ast-mcp-server 0.3.0 → 0.5.0

package/CHANGELOG.md

@@ -4,6 +4,48 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.0] - 2026-08-03
+
+### Added
+
+- Relevance-ranked symbol search with `selectors`, `summary`, and backward-compatible `full` detail profiles.
+- Progressive semantic references with location-only defaults and opt-in bounded source context.
+- `ast_scaffold_class`, a strict prepare-only class scaffold that generates explicit throwing method placeholders and uses the existing preview/apply protocol.
+- Hash-bound absent-file plans with atomic no-clobber creation, conservative rollback, persisted replay, and idempotent receipt recovery.
+- A checked result-shaping workflow benchmark covering ranking, selector chaining, multi-file references, JSON/TOON payloads, call counts, evidence preservation, and complete tool metadata.
+
+### Changed
+
+- Symbol search now defaults to ranked `summary` records with a 20-result page. Use `detail: "full", limit: 100` for the v0.4.0 result profile.
+- Reference search now defaults to `locations` without source lines. Use `detail: "context"` for the v0.4.0 record profile.
+- MCP and batch client identity now derives from the shipped package version instead of a separately maintained literal.
+
+### Security
+
+- Scaffold inputs are strict and bounded; identifiers, duplicates, raw TypeScript fragments, paths, parent directories, and target absence are validated before plan creation.
+- File creation uses same-directory exclusive staging plus an atomic hard-link commit and never falls back to overwrite-capable rename.
+- Existing rename/body-replacement plan hashes and persisted modification plans remain compatible.
+
+## [0.4.0] - 2026-08-03
+
+### Added
+
+- Opt-in TOON envelopes for symbol search, semantic references, and diagnostics through `output_format: "toon"`.
+- Plain TOON output for final read-only batch results through `ast-tool run --output-format toon`.
+- Reproducible JSON/TOON benchmark with lossless round trips, UTF-8 bytes, `o200k_base` token estimates, encoding latency, static tool-metadata accounting, and negative controls.
+
+### Changed
+
+- Collection-heavy tools validate their canonical Zod outputs explicitly before returning JSON or TOON. They no longer advertise one MCP `outputSchema` because their structured result can be either the canonical object or a TOON envelope.
+- The bundled structural-code-editing skill now selects TOON only for measured collection shapes and keeps JSON for source, outline, file-list, and mutation workflows.
+
+### Security
+
+- Batch steps reject statically declared and dynamically resolved TOON intermediates before the affected invocation.
+- TOON output is capped at 10 MiB after UTF-8 encoding, and mutation preparation remains JSON-only so review coordinates are never compacted.
+- Successful MCP TOON output is decode-checked against the validated value, and MCP error text is capped at 64 KiB of UTF-8.
+- The transitive Hono runtime is resolved to `4.12.34`, closing GHSA-8j4g-w8fx-2239 from the MCP SDK dependency tree.
+
 ## [0.3.0] - 2026-08-03
 
 Initial public release.
@@ -24,4 +66,6 @@ Initial public release.
 - Apply rejects stale workspaces, changed configs/sources, mismatched plan hashes, unsafe plan files, conflicting MCP registrations, and conflicting skill content.
 - Inputs, outputs, operation stores, subprocesses, batch fan-out, plan lifetime, and filesystem access are bounded.
 
+[0.5.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.5.0
+[0.4.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.4.0
 [0.3.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.3.0

package/README.md

@@ -39,7 +39,7 @@ Reads can start with a compact outline and fetch exact source only when needed.
 
 AST-aware editing is not a proof that a change is semantically correct. The safety comes from combining structural selection with diagnostics, exact previews, reviewed hashes, freshness checks, and fail-closed apply semantics.
 
-The included batch benchmark records a 50% reduction in model round-trips and a 95.21% reduction in serialized context for its search-to-source scenario. These are reproducible scenario measurements, not universal token or latency claims.
+The included batch benchmark records a 50% reduction in model round-trips and a 94.67% reduction in serialized context for its search-to-source scenario. The result-shaping corpus records a 68.80% reduction in aggregate model-facing TOON tokens while preserving declared selectors/reference coordinates with the same six logical calls. The separate format benchmark records 25.87% across its eligible collection corpus. These are reproducible local `o200k_base` estimates, not universal token, billing, cache, or latency claims.
 
 ## Requirements
 
@@ -224,11 +224,22 @@ Project-scoped tools accept `project_root`, either the project directory or an e
 | `ast_get_diagnostics`       | Project- or file-scoped TypeScript diagnostics                       | No            |
 | `ast_rename_symbol`         | Prepare a project-wide rename                                        | No            |
 | `ast_replace_symbol_body`   | Prepare a body-only replacement while preserving the signature       | No            |
+| `ast_scaffold_class`        | Prepare one new class file with explicit placeholder methods         | No            |
 | `ast_get_operation_preview` | Retrieve the complete retained diff for a prepared plan              | No            |
 | `ast_apply_operation`       | Apply one reviewed, hash-bound plan                                  | Yes           |
 
 Read results use project-relative paths, deterministic ordering, structured MCP output, and pagination where result sets can grow with the project.
 
+Symbol search is relevance-ranked and defaults to at most 20 `summary` records containing `file`, a directly reusable `selector`, `kind`, and body-free `signature`. Request `detail: "selectors"` for routing coordinates only, or `detail: "full", limit: 100` for the v0.4.0 fields/page. References default to `detail: "locations"`; request `detail: "context"` only when the bounded source line is needed.
+
+### Optional TOON results
+
+`ast_search_symbols`, `ast_find_references`, and `ast_get_diagnostics` accept `output_format: "toon"` for collection-heavy results consumed directly by a model. JSON remains the default and preserves the canonical structured object.
+
+MCP TOON is returned once as structured content shaped like `{ "format": "toon", "data": "..." }`; `data` is the lossless TOON document. The complete JSON result is not duplicated. These three tools validate their canonical Zod result and verify an encode/decode deep-equality round trip before presentation, but do not advertise a single MCP `outputSchema` because their successful structured content has two representations.
+
+Do not request TOON for source, outlines, file lists, previews, or mutation results. Checked negative controls show that the MCP envelope makes those shapes larger. TOON is an explicit shape-specific optimization, not a new dialect for every object in sight.
+
 ## Batch CLI
 
 `ast-tool` lets Claude Code and other Bash-capable clients collapse a known structural pipeline into one shell call:
@@ -236,6 +247,7 @@ Read results use project-relative paths, deterministic ordering, structured MCP
 ```bash
 ast-tool validate pipeline.json
 ast-tool run pipeline.json
+ast-tool run pipeline.json --output-format toon
 cat pipeline.json | ast-tool run -
 ```
 
@@ -256,7 +268,7 @@ Example search-to-source pipeline:
       "tool": "ast_get_symbol_source",
       "input": {
         "file_path": { "$ref": "#/steps/search/symbols/0/file" },
-        "symbol_path": { "$ref": "#/steps/search/symbols/0/symbol_path" }
+        "symbol_path": { "$ref": "#/steps/search/symbols/0/selector" }
       }
     }
   ],
@@ -299,21 +311,23 @@ A `$ref` is an RFC 6901 JSON Pointer rooted at prior step results. References ca
 - One project root per pipeline.
 - No branches, eval, embedded JavaScript, while loops, or arbitrary transformations.
 
-Success is one JSON value on stdout. Errors are structured JSON on stderr. Exit code 0 is success, 1 is execution/apply failure, and 2 is usage or schema failure.
+Success is one compact JSON value on stdout by default. `ast-tool run --output-format toon` writes one plain TOON document for a read-only batch; internal steps remain structured JSON, and prepare batches reject TOON before execution. Encoding and output-limit failures write no partial stdout and use stable `ENCODING_ERROR` or `OUTPUT_LIMIT` codes. Errors are always structured JSON on stderr. Exit code 0 is success, 1 is execution/apply failure, and 2 is usage or schema failure.
 
 ## Reviewed mutations
 
 ### MCP process
 
-Rename and body replacement never write directly:
+Rename, body replacement, and class scaffold never write directly during preparation:
 
-1. Call `ast_rename_symbol` or `ast_replace_symbol_body`.
+1. Call `ast_rename_symbol`, `ast_replace_symbol_body`, or `ast_scaffold_class`.
 2. Review the diagnostic delta, affected files, `blocked`, and `plan_hash`.
 3. Fetch complete diffs with `ast_get_operation_preview` when needed.
 4. Call `ast_apply_operation` with both `operation_id` and `plan_hash`.
 
 MCP plans live in a bounded in-memory store and do not survive a server restart.
 
+`ast_scaffold_class` accepts structured imports, heritage, decorators, constructor parameter properties, initialized properties, and one or more method signatures. It creates an in-memory preview for one absent project-relative `.ts`/`.tsx` target. Each generated method initially contains only `throw new Error("Not implemented: Class.method")`. Review the `/dev/null` creation diff and diagnostics, apply the scaffold, then replace each pending method body with `ast_replace_symbol_body`. Existing targets and symbolic/traversing parents fail closed.
+
 ### CLI process boundary
 
 A batch may contain at most one prepare operation. It must be the final step and cannot use foreach. `ast_apply_operation` and arbitrary preview calls are forbidden inside batch documents.
@@ -370,13 +384,19 @@ yarn pack --dry-run
 yarn benchmark /absolute/project --sample 20 --output benchmark/results/project.json
 yarn benchmark:corpus benchmark/task-corpus.json --output benchmark/results/self-corpus.json
 yarn benchmark:batch --iterations 5 --output benchmark/results/self-batch.json
+yarn benchmark:formats
+yarn benchmark:shapes
 ```
 
-The batch benchmark compares two separate client calls with one batch invocation in fresh Node processes, recording model round-trips, actual tool invocations, wall time, maximum RSS, and serialized character counts. Character counts are not model-specific token estimates. See `benchmark/README.md` for methodology and limitations.
+The batch benchmark compares two separate client calls with one batch invocation in fresh Node processes, recording model round-trips, actual tool invocations, wall time, maximum RSS, and serialized character counts. Character counts are not model-specific token estimates.
+
+The format benchmark runs real tools against this repository plus deterministic reference/diagnostic fixtures. It checks JSON→TOON→value equality, UTF-8 bytes, `gpt-tokenizer` `o200k_base` estimates, encode/decode latency, the actual MCP envelope, tool metadata, and negative controls. Its checked result is `benchmark/results/self-formats.json`; local tokenizer estimates do not establish provider-side billing or cache savings. See `benchmark/README.md` for methodology and limitations.
+
+The result-shaping benchmark compares the v0.4.0-compatible `full/100/context` profiles with the new public defaults across exact-name, exact-path, prefix, broad-substring, and multi-file-reference tasks. It fails on missing evidence, extra logical calls, fewer than 11 tools, or less than 35% aggregate TOON token reduction. Its checked result is `benchmark/results/self-result-shapes.json`.
 
 ## Scope
 
 - TypeScript and JavaScript projects understood by the TypeScript compiler.
-- Structural rename and callable-body replacement.
+- Structural rename, callable-body replacement, and reviewed creation of one class scaffold.
 - Declarative DAG-like pipelines with prior-result references and bounded foreach.
-- No arbitrary signature migration, file creation/deletion plans, cross-language refactors, or general-purpose scripting language.
+- No arbitrary signature migration, general file creation/deletion, cross-language refactors, or general-purpose scripting language.

package/dist/batch/runner.js

@@ -1,7 +1,7 @@
 import { performance } from "node:perf_hooks";
 import { Client } from "@modelcontextprotocol/sdk/client/index.js";
 import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
-import { createServer } from "../server.js";
+import { createServer, PACKAGE_VERSION } from "../server.js";
 import { MAX_BATCH_CONTEXT_BYTES, MAX_BATCH_INVOCATIONS, MAX_BATCH_OUTPUT_BYTES, MAX_FOREACH_ITEMS, isItemTemplate, isPrepareBatchTool, isReadBatchTool, isReferenceTemplate, parseBatchDocument as parseDocument, } from "./schema.js";
 export { parseDocument as parseBatchDocument };
 export class BatchExecutionError extends Error {
@@ -65,6 +65,9 @@ function injectProjectRoot(resolved, projectRoot, step) {
         throw new BatchExecutionError(`Step "${step.id}" input must resolve to an object.`, "INVALID_STEP_INPUT", step.id);
     }
     const input = resolved;
+    if (input.output_format === "toon") {
+        throw new BatchExecutionError(`Step "${step.id}" cannot resolve TOON for an intermediate batch result.`, "TOON_INTERMEDIATE_FORBIDDEN", step.id);
+    }
     if (Object.prototype.hasOwnProperty.call(input, "project_root") &&
         input.project_root !== projectRoot) {
         throw new BatchExecutionError(`Step "${step.id}" resolves a conflicting project_root.`, "PROJECT_ROOT_CONFLICT", step.id);
@@ -73,7 +76,7 @@ function injectProjectRoot(resolved, projectRoot, step) {
 }
 async function connectBatchClient() {
     const server = createServer();
-    const client = new Client({ name: "ast-tool-batch", version: "0.3.0" });
+    const client = new Client({ name: "ast-tool-batch", version: PACKAGE_VERSION });
     const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair();
     await Promise.all([server.connect(serverTransport), client.connect(clientTransport)]);
     return { client, server };
@@ -170,6 +173,11 @@ export async function runBatchDocument(document, options = {}) {
                 if (items.length > MAX_FOREACH_ITEMS) {
                     throw new BatchExecutionError(`Step "${step.id}" expands to ${items.length} items; maximum is ${MAX_FOREACH_ITEMS}.`, "FOREACH_LIMIT", step.id);
                 }
+                // Resolve once up front so one forbidden item cannot race valid sibling invocations.
+                // Resolve again inside mapLimit to avoid retaining up to 200 cloned input payloads.
+                for (const item of items) {
+                    injectProjectRoot(resolveTemplate(step.input, context, item), document.project_root, step);
+                }
                 reserveInvocations(items.length, step);
                 const results = await mapLimit(items, document.limits.concurrency, async (item) => {
                     const resolved = resolveTemplate(step.input, context, item);

package/dist/batch/schema.js

@@ -16,7 +16,11 @@ export const READ_BATCH_TOOLS = [
     "ast_find_references",
     "ast_get_diagnostics",
 ];
-export const PREPARE_BATCH_TOOLS = ["ast_rename_symbol", "ast_replace_symbol_body"];
+export const PREPARE_BATCH_TOOLS = [
+    "ast_rename_symbol",
+    "ast_replace_symbol_body",
+    "ast_scaffold_class",
+];
 export const BATCH_TOOLS = [...READ_BATCH_TOOLS, ...PREPARE_BATCH_TOOLS];
 const BatchToolSchema = z.enum(BATCH_TOOLS);
 const BatchStepSchema = z
@@ -112,6 +116,9 @@ export function parseBatchDocument(input) {
     for (const [index, step] of parsed.steps.entries()) {
         if (ids.has(step.id))
             throw new Error(`Duplicate batch step id "${step.id}".`);
+        if (step.input.output_format === "toon") {
+            throw new Error(`Step "${step.id}" cannot request TOON for an intermediate batch result. Use CLI --output-format toon for the final output.`);
+        }
         if (Object.hasOwn(step.input, "project_root")) {
             if (step.input.project_root !== parsed.project_root) {
                 throw new Error(`Step "${step.id}" project_root conflicts with the pipeline project_root.`);

package/dist/cli-output.js

@@ -0,0 +1,28 @@
+import { Buffer } from "node:buffer";
+import { encode } from "@toon-format/toon";
+import { MAX_BATCH_OUTPUT_BYTES } from "./batch/schema.js";
+export class CliOutputError extends Error {
+    code;
+    constructor(message, code, options) {
+        super(message, options);
+        this.code = code;
+        this.name = "CliOutputError";
+    }
+}
+export function serializeCliSuccess(value, outputFormat, maxBytes = MAX_BATCH_OUTPUT_BYTES) {
+    let serialized;
+    try {
+        serialized = outputFormat === "toon" ? encode(value) : JSON.stringify(value);
+    }
+    catch (error) {
+        throw new CliOutputError(`Failed to encode ${outputFormat.toUpperCase()} output.`, "ENCODING_ERROR", { cause: error });
+    }
+    if (serialized === undefined) {
+        throw new CliOutputError(`${outputFormat.toUpperCase()} output did not produce a document.`, "ENCODING_ERROR");
+    }
+    const bytes = Buffer.byteLength(serialized, "utf8");
+    if (bytes > maxBytes) {
+        throw new CliOutputError(`${outputFormat === "toon" ? "TOON" : "JSON"} output exceeds the ${maxBytes}-byte limit (${bytes} bytes encoded).`, "OUTPUT_LIMIT");
+    }
+    return serialized;
+}

package/dist/cli.js

@@ -4,7 +4,8 @@ import process from "node:process";
 import { createInterface } from "node:readline/promises";
 import { ZodError } from "zod";
 import { BatchExecutionError, parseBatchDocument, runBatchDocument } from "./batch/runner.js";
-import { MAX_BATCH_INPUT_BYTES } from "./batch/schema.js";
+import { MAX_BATCH_INPUT_BYTES, isPrepareBatchTool } from "./batch/schema.js";
+import { CliOutputError, serializeCliSuccess } from "./cli-output.js";
 import { applyPersistedOperation, persistOperationPlan } from "./services/operation-plan-file.js";
 import { AgentSetupError, detectInstalledAgents, resolveServerEntryPath, runAgentSetup, } from "./services/agent-setup.js";
 import { installBundledSkill, resolveBundledSkillPath, } from "./services/skill-installer.js";
@@ -28,7 +29,7 @@ class CliError extends Error {
 function usage() {
     return [
         "Usage:",
-        "  ast-tool run <pipeline.json|->",
+        "  ast-tool run <pipeline.json|-> [--output-format json|toon]",
         "  ast-tool validate <pipeline.json|->",
         "  ast-tool apply <plan.astplan> --plan-hash <sha256>",
         "  ast-tool install-skill [claude|hermes|all] [--scope user|project] [--project-root <path>] [--force]",
@@ -68,6 +69,32 @@ function parseApplyArgs(args) {
     }
     return { planFile: args[0], planHash: args[2] };
 }
+function parseRunArgs(args) {
+    let source;
+    let outputFormat = "json";
+    let outputFormatSeen = false;
+    for (let index = 0; index < args.length; index += 1) {
+        const argument = args[index];
+        if (argument === "--output-format") {
+            const value = args[index + 1];
+            if (outputFormatSeen || (value !== "json" && value !== "toon")) {
+                throw new CliError(usage(), "USAGE", 2, "run");
+            }
+            outputFormat = value;
+            outputFormatSeen = true;
+            index += 1;
+            continue;
+        }
+        if (source === undefined && !argument.startsWith("--")) {
+            source = argument;
+            continue;
+        }
+        throw new CliError(usage(), "USAGE", 2, "run");
+    }
+    if (!source)
+        throw new CliError(usage(), "USAGE", 2, "run");
+    return { source, outputFormat };
+}
 function parseInstallSkillArgs(args) {
     let target = "all";
     let targetSeen = false;
@@ -168,10 +195,14 @@ async function parseDocumentForCommand(source, command) {
 export async function runCli(args) {
     const [command, ...commandArgs] = args;
     if (command === "run" || command === "validate") {
-        if (commandArgs.length !== 1 || !commandArgs[0]) {
+        const runArgs = command === "run"
+            ? parseRunArgs(commandArgs)
+            : commandArgs.length === 1 && commandArgs[0]
+                ? { source: commandArgs[0], outputFormat: "json" }
+                : undefined;
+        if (!runArgs)
             throw new CliError(usage(), "USAGE", 2, command);
-        }
-        const document = await parseDocumentForCommand(commandArgs[0], command);
+        const document = await parseDocumentForCommand(runArgs.source, command);
         if (command === "validate") {
             return {
                 version: 1,
@@ -179,6 +210,10 @@ export async function runCli(args) {
                 step_count: document.steps.length,
             };
         }
+        if (runArgs.outputFormat === "toon" &&
+            document.steps.some((step) => isPrepareBatchTool(step.tool))) {
+            throw new CliError("TOON output is available only for read-only batches; prepare results require JSON review coordinates.", "INVALID_BATCH", 2, command);
+        }
         try {
             return await runBatchDocument(document, {
                 persistPreparedOperation: (operationId) => persistOperationPlan(operationId),
@@ -262,7 +297,7 @@ export async function runCli(args) {
     }
     throw new CliError(usage(), "USAGE", 2, command ?? null);
 }
-function failure(error) {
+function failure(error, command = null) {
     if (error instanceof CliError) {
         return {
             value: {
@@ -276,6 +311,17 @@ function failure(error) {
             exitCode: error.exitCode,
         };
     }
+    if (error instanceof CliOutputError) {
+        return {
+            value: {
+                status: "error",
+                command,
+                code: error.code,
+                message: error.message,
+            },
+            exitCode: 1,
+        };
+    }
     return {
         value: {
             status: "error",
@@ -287,12 +333,14 @@ function failure(error) {
     };
 }
 async function main() {
+    const args = process.argv.slice(2);
     try {
-        const result = await runCli(process.argv.slice(2));
-        process.stdout.write(`${JSON.stringify(result)}\n`);
+        const result = await runCli(args);
+        const outputFormat = args[0] === "run" ? parseRunArgs(args.slice(1)).outputFormat : "json";
+        process.stdout.write(`${serializeCliSuccess(result, outputFormat)}\n`);
     }
     catch (error) {
-        const failed = failure(error);
+        const failed = failure(error, args[0] ?? null);
         process.stderr.write(`${JSON.stringify(failed.value)}\n`);
         process.exitCode = failed.exitCode;
     }

package/dist/server.js

@@ -1,3 +1,4 @@
+import { createRequire } from "node:module";
 import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
 import { registerApplyOperation } from "./tools/apply_operation.js";
 import { registerFindReferences } from "./tools/find_references.js";
@@ -8,11 +9,14 @@ import { registerGetSymbolSource } from "./tools/get_symbol_source.js";
 import { registerListFiles } from "./tools/list_files.js";
 import { registerRenameSymbol } from "./tools/rename_symbol.js";
 import { registerReplaceSymbolBody } from "./tools/replace_symbol_body.js";
+import { registerScaffoldClass } from "./tools/scaffold_class.js";
 import { registerSearchSymbols } from "./tools/search_symbols.js";
+const packageMetadata = createRequire(import.meta.url)("../package.json");
+export const PACKAGE_VERSION = packageMetadata.version;
 export function createServer() {
     const server = new McpServer({
         name: "ast-mcp-server",
-        version: "0.3.0",
+        version: PACKAGE_VERSION,
     });
     registerListFiles(server);
     registerGetOutline(server);
@@ -22,6 +26,7 @@ export function createServer() {
     registerGetDiagnostics(server);
     registerRenameSymbol(server);
     registerReplaceSymbolBody(server);
+    registerScaffoldClass(server);
     registerGetOperationPreview(server);
     registerApplyOperation(server);
     return server;

package/dist/services/agent-setup.js

@@ -17,6 +17,7 @@ const EXPECTED_HERMES_TOOLS = [
     "ast_get_diagnostics",
     "ast_rename_symbol",
     "ast_replace_symbol_body",
+    "ast_scaffold_class",
     "ast_get_operation_preview",
     "ast_apply_operation",
 ];

package/dist/services/operation-plan-file.js

@@ -42,7 +42,7 @@ const PersistedOperationSchema = z
     .object({
     operation_id: z.string().uuid(),
     plan_hash: HashSchema,
-    kind: z.enum(["rename_symbol", "replace_symbol_body"]),
+    kind: z.enum(["rename_symbol", "replace_symbol_body", "scaffold_class"]),
     status: z.enum(["prepared", "applied"]),
     project_root: z.string().min(1),
     created_at: IsoDateSchema,