ast-mcp-server 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +15 -3
- package/benchmark/README.md +33 -1
- package/dist/batch/runner.js +8 -0
- package/dist/batch/schema.js +3 -0
- package/dist/cli-output.js +28 -0
- package/dist/cli.js +57 -9
- package/dist/tools/find_references.js +4 -4
- package/dist/tools/get_diagnostics.js +4 -4
- package/dist/tools/result.js +45 -1
- package/dist/tools/search_symbols.js +4 -4
- package/package.json +7 -1
- package/skills/structural-code-editing/SKILL.md +16 -2
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,26 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.4.0] - 2026-08-03
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- Opt-in TOON envelopes for symbol search, semantic references, and diagnostics through `output_format: "toon"`.
|
|
12
|
+
- Plain TOON output for final read-only batch results through `ast-tool run --output-format toon`.
|
|
13
|
+
- Reproducible JSON/TOON benchmark with lossless round trips, UTF-8 bytes, `o200k_base` token estimates, encoding latency, static tool-metadata accounting, and negative controls.
|
|
14
|
+
|
|
15
|
+
### Changed
|
|
16
|
+
|
|
17
|
+
- Collection-heavy tools validate their canonical Zod outputs explicitly before returning JSON or TOON. They no longer advertise one MCP `outputSchema` because their structured result can be either the canonical object or a TOON envelope.
|
|
18
|
+
- The bundled structural-code-editing skill now selects TOON only for measured collection shapes and keeps JSON for source, outline, file-list, and mutation workflows.
|
|
19
|
+
|
|
20
|
+
### Security
|
|
21
|
+
|
|
22
|
+
- Batch steps reject statically declared and dynamically resolved TOON intermediates before the affected invocation.
|
|
23
|
+
- TOON output is capped at 10 MiB after UTF-8 encoding, and mutation preparation remains JSON-only so review coordinates are never compacted.
|
|
24
|
+
- Successful MCP TOON output is decode-checked against the validated value, and MCP error text is capped at 64 KiB of UTF-8.
|
|
25
|
+
- The transitive Hono runtime is resolved to `4.12.34`, closing GHSA-8j4g-w8fx-2239 from the MCP SDK dependency tree.
|
|
26
|
+
|
|
7
27
|
## [0.3.0] - 2026-08-03
|
|
8
28
|
|
|
9
29
|
Initial public release.
|
|
@@ -24,4 +44,5 @@ Initial public release.
|
|
|
24
44
|
- Apply rejects stale workspaces, changed configs/sources, mismatched plan hashes, unsafe plan files, conflicting MCP registrations, and conflicting skill content.
|
|
25
45
|
- Inputs, outputs, operation stores, subprocesses, batch fan-out, plan lifetime, and filesystem access are bounded.
|
|
26
46
|
|
|
47
|
+
[0.4.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.4.0
|
|
27
48
|
[0.3.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.3.0
|
package/README.md
CHANGED
|
@@ -39,7 +39,7 @@ Reads can start with a compact outline and fetch exact source only when needed.
|
|
|
39
39
|
|
|
40
40
|
AST-aware editing is not a proof that a change is semantically correct. The safety comes from combining structural selection with diagnostics, exact previews, reviewed hashes, freshness checks, and fail-closed apply semantics.
|
|
41
41
|
|
|
42
|
-
The included batch benchmark records a 50% reduction in model round-trips and a 95.21% reduction in serialized context for its search-to-source scenario. These are reproducible scenario measurements, not universal token or latency claims.
|
|
42
|
+
The included batch benchmark records a 50% reduction in model round-trips and a 95.21% reduction in serialized context for its search-to-source scenario. The format benchmark records an `o200k_base` estimate of 25.72% fewer model-facing tokens for a 100-row symbol search and 26.56% across the eligible collection corpus. These are reproducible scenario measurements, not universal token, billing, cache, or latency claims.
|
|
43
43
|
|
|
44
44
|
## Requirements
|
|
45
45
|
|
|
@@ -229,6 +229,14 @@ Project-scoped tools accept `project_root`, either the project directory or an e
|
|
|
229
229
|
|
|
230
230
|
Read results use project-relative paths, deterministic ordering, structured MCP output, and pagination where result sets can grow with the project.
|
|
231
231
|
|
|
232
|
+
### Optional TOON results
|
|
233
|
+
|
|
234
|
+
`ast_search_symbols`, `ast_find_references`, and `ast_get_diagnostics` accept `output_format: "toon"` for collection-heavy results consumed directly by a model. JSON remains the default and preserves the canonical structured object.
|
|
235
|
+
|
|
236
|
+
MCP TOON is returned once as structured content shaped like `{ "format": "toon", "data": "..." }`; `data` is the lossless TOON document. The complete JSON result is not duplicated. These three tools validate their canonical Zod result and verify an encode/decode deep-equality round trip before presentation, but do not advertise a single MCP `outputSchema` because their successful structured content has two representations.
|
|
237
|
+
|
|
238
|
+
Do not request TOON for source, outlines, file lists, previews, or mutation results. Checked negative controls show that the MCP envelope makes those shapes larger. TOON is an explicit shape-specific optimization, not a new dialect for every object in sight.
|
|
239
|
+
|
|
232
240
|
## Batch CLI
|
|
233
241
|
|
|
234
242
|
`ast-tool` lets Claude Code and other Bash-capable clients collapse a known structural pipeline into one shell call:
|
|
@@ -236,6 +244,7 @@ Read results use project-relative paths, deterministic ordering, structured MCP
|
|
|
236
244
|
```bash
|
|
237
245
|
ast-tool validate pipeline.json
|
|
238
246
|
ast-tool run pipeline.json
|
|
247
|
+
ast-tool run pipeline.json --output-format toon
|
|
239
248
|
cat pipeline.json | ast-tool run -
|
|
240
249
|
```
|
|
241
250
|
|
|
@@ -299,7 +308,7 @@ A `$ref` is an RFC 6901 JSON Pointer rooted at prior step results. References ca
|
|
|
299
308
|
- One project root per pipeline.
|
|
300
309
|
- No branches, eval, embedded JavaScript, while loops, or arbitrary transformations.
|
|
301
310
|
|
|
302
|
-
Success is one JSON value on stdout. Errors are structured JSON on stderr. Exit code 0 is success, 1 is execution/apply failure, and 2 is usage or schema failure.
|
|
311
|
+
Success is one compact JSON value on stdout by default. `ast-tool run --output-format toon` writes one plain TOON document for a read-only batch; internal steps remain structured JSON, and prepare batches reject TOON before execution. Encoding and output-limit failures write no partial stdout and use stable `ENCODING_ERROR` or `OUTPUT_LIMIT` codes. Errors are always structured JSON on stderr. Exit code 0 is success, 1 is execution/apply failure, and 2 is usage or schema failure.
|
|
303
312
|
|
|
304
313
|
## Reviewed mutations
|
|
305
314
|
|
|
@@ -370,9 +379,12 @@ yarn pack --dry-run
|
|
|
370
379
|
yarn benchmark /absolute/project --sample 20 --output benchmark/results/project.json
|
|
371
380
|
yarn benchmark:corpus benchmark/task-corpus.json --output benchmark/results/self-corpus.json
|
|
372
381
|
yarn benchmark:batch --iterations 5 --output benchmark/results/self-batch.json
|
|
382
|
+
yarn benchmark:formats
|
|
373
383
|
```
|
|
374
384
|
|
|
375
|
-
The batch benchmark compares two separate client calls with one batch invocation in fresh Node processes, recording model round-trips, actual tool invocations, wall time, maximum RSS, and serialized character counts. Character counts are not model-specific token estimates.
|
|
385
|
+
The batch benchmark compares two separate client calls with one batch invocation in fresh Node processes, recording model round-trips, actual tool invocations, wall time, maximum RSS, and serialized character counts. Character counts are not model-specific token estimates.
|
|
386
|
+
|
|
387
|
+
The format benchmark runs real tools against this repository plus deterministic reference/diagnostic fixtures. It checks JSON→TOON→value equality, UTF-8 bytes, `gpt-tokenizer` `o200k_base` estimates, encode/decode latency, the actual MCP envelope, tool metadata, and negative controls. Its checked result is `benchmark/results/self-formats.json`; local tokenizer estimates do not establish provider-side billing or cache savings. See `benchmark/README.md` for methodology and limitations.
|
|
376
388
|
|
|
377
389
|
## Scope
|
|
378
390
|
|
package/benchmark/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Benchmark methodology
|
|
2
2
|
|
|
3
|
-
The benchmark suite measures
|
|
3
|
+
The benchmark suite measures structural payload reduction, batch orchestration, and model-facing result formats. The first two benchmarks deliberately avoid converting characters into tokens. The format benchmark pins and names one tokenizer so its estimates are reproducible rather than universal.
|
|
4
4
|
|
|
5
5
|
## Project payload and latency
|
|
6
6
|
|
|
@@ -64,6 +64,27 @@ yarn benchmark:batch --iterations 5 --output benchmark/results/self-batch.json
|
|
|
64
64
|
|
|
65
65
|
The report records logical model round-trips, actual tool invocations, median wall time, process maximum RSS, final-result characters, and total characters exposed to the model. It does not include model inference latency, so the practical wall-time benefit of removing a model round-trip will be larger than this local process-only comparison. Maximum RSS includes the Node runtime and TypeScript project in each fresh worker.
|
|
66
66
|
|
|
67
|
+
## Model-facing JSON and TOON
|
|
68
|
+
|
|
69
|
+
`scripts/benchmark-formats.mjs` compares compact JSON with TOON for actual MCP logical results. It uses this repository for broad symbol search and deterministic temporary TypeScript fixtures for repeated references and non-empty diagnostics. File lists, outlines, exact source, and a prepared rename are retained as negative controls.
|
|
70
|
+
|
|
71
|
+
For each payload it records:
|
|
72
|
+
|
|
73
|
+
- exact JSON→TOON→value round-trip equality;
|
|
74
|
+
- UTF-8 bytes;
|
|
75
|
+
- `gpt-tokenizer@3.4.0` `o200k_base` token estimates;
|
|
76
|
+
- plain TOON for CLI output and the real MCP `{format,data}` envelope;
|
|
77
|
+
- median encode/decode time over 50 local iterations;
|
|
78
|
+
- serialized `tools/list` metadata against the checked v0.3.0 baseline.
|
|
79
|
+
|
|
80
|
+
Run:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
yarn benchmark:formats
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
The script fails unless every round trip is exact, the broad-search MCP envelope reduces estimated tokens by at least 20%, and the aggregate eligible corpus reduces them by at least 15%. Tokenizer estimates are not provider usage records and do not establish billed-token or prompt-cache savings.
|
|
87
|
+
|
|
67
88
|
## Checked results
|
|
68
89
|
|
|
69
90
|
Results generated on 2026-08-03 with Node.js v24.16.0:
|
|
@@ -84,3 +105,14 @@ The checked broad search-to-source batch result used five fresh-process samples:
|
|
|
84
105
|
| Batch | 1 | 2 | 552 | 535.24 ms | 378.35 MB |
|
|
85
106
|
|
|
86
107
|
That run reduced model round-trips by 50% and serialized context by 95.21%. Local execution latency decreased by 0.18% and RSS increased by 0.21%, both small enough to treat as noise rather than a performance claim. The batch value is orchestration/context reduction; it does not make TypeScript analysis free.
|
|
108
|
+
|
|
109
|
+
The checked model-facing format corpus used the complete MCP envelope for TOON:
|
|
110
|
+
|
|
111
|
+
| Payload | Records | JSON tokens | MCP TOON tokens | Reduction |
|
|
112
|
+
| ------------------- | ------: | ----------: | --------------: | --------: |
|
|
113
|
+
| Broad symbol search | 100 | 5,225 | 3,881 | 25.72% |
|
|
114
|
+
| References | 71 | 3,086 | 2,150 | 30.33% |
|
|
115
|
+
| Diagnostics | 30 | 1,309 | 1,034 | 21.01% |
|
|
116
|
+
| Eligible aggregate | 201 | 9,620 | 7,065 | 26.56% |
|
|
117
|
+
|
|
118
|
+
All seven positive and negative payloads round-tripped exactly. File list, outline, source, and prepare envelopes ranged from 5.50% to 23.40% worse in estimated tokens, which is why they remain JSON-only. Current serialized tool metadata is 2,451 characters smaller than the v0.3.0 baseline despite the three new input fields, because multi-shape tools no longer advertise a single incompatible output schema.
|
package/dist/batch/runner.js
CHANGED
|
@@ -65,6 +65,9 @@ function injectProjectRoot(resolved, projectRoot, step) {
|
|
|
65
65
|
throw new BatchExecutionError(`Step "${step.id}" input must resolve to an object.`, "INVALID_STEP_INPUT", step.id);
|
|
66
66
|
}
|
|
67
67
|
const input = resolved;
|
|
68
|
+
if (input.output_format === "toon") {
|
|
69
|
+
throw new BatchExecutionError(`Step "${step.id}" cannot resolve TOON for an intermediate batch result.`, "TOON_INTERMEDIATE_FORBIDDEN", step.id);
|
|
70
|
+
}
|
|
68
71
|
if (Object.prototype.hasOwnProperty.call(input, "project_root") &&
|
|
69
72
|
input.project_root !== projectRoot) {
|
|
70
73
|
throw new BatchExecutionError(`Step "${step.id}" resolves a conflicting project_root.`, "PROJECT_ROOT_CONFLICT", step.id);
|
|
@@ -170,6 +173,11 @@ export async function runBatchDocument(document, options = {}) {
|
|
|
170
173
|
if (items.length > MAX_FOREACH_ITEMS) {
|
|
171
174
|
throw new BatchExecutionError(`Step "${step.id}" expands to ${items.length} items; maximum is ${MAX_FOREACH_ITEMS}.`, "FOREACH_LIMIT", step.id);
|
|
172
175
|
}
|
|
176
|
+
// Resolve once up front so one forbidden item cannot race valid sibling invocations.
|
|
177
|
+
// Resolve again inside mapLimit to avoid retaining up to 200 cloned input payloads.
|
|
178
|
+
for (const item of items) {
|
|
179
|
+
injectProjectRoot(resolveTemplate(step.input, context, item), document.project_root, step);
|
|
180
|
+
}
|
|
173
181
|
reserveInvocations(items.length, step);
|
|
174
182
|
const results = await mapLimit(items, document.limits.concurrency, async (item) => {
|
|
175
183
|
const resolved = resolveTemplate(step.input, context, item);
|
package/dist/batch/schema.js
CHANGED
|
@@ -112,6 +112,9 @@ export function parseBatchDocument(input) {
|
|
|
112
112
|
for (const [index, step] of parsed.steps.entries()) {
|
|
113
113
|
if (ids.has(step.id))
|
|
114
114
|
throw new Error(`Duplicate batch step id "${step.id}".`);
|
|
115
|
+
if (step.input.output_format === "toon") {
|
|
116
|
+
throw new Error(`Step "${step.id}" cannot request TOON for an intermediate batch result. Use CLI --output-format toon for the final output.`);
|
|
117
|
+
}
|
|
115
118
|
if (Object.hasOwn(step.input, "project_root")) {
|
|
116
119
|
if (step.input.project_root !== parsed.project_root) {
|
|
117
120
|
throw new Error(`Step "${step.id}" project_root conflicts with the pipeline project_root.`);
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
import { Buffer } from "node:buffer";
|
|
2
|
+
import { encode } from "@toon-format/toon";
|
|
3
|
+
import { MAX_BATCH_OUTPUT_BYTES } from "./batch/schema.js";
|
|
4
|
+
export class CliOutputError extends Error {
|
|
5
|
+
code;
|
|
6
|
+
constructor(message, code, options) {
|
|
7
|
+
super(message, options);
|
|
8
|
+
this.code = code;
|
|
9
|
+
this.name = "CliOutputError";
|
|
10
|
+
}
|
|
11
|
+
}
|
|
12
|
+
export function serializeCliSuccess(value, outputFormat, maxBytes = MAX_BATCH_OUTPUT_BYTES) {
|
|
13
|
+
let serialized;
|
|
14
|
+
try {
|
|
15
|
+
serialized = outputFormat === "toon" ? encode(value) : JSON.stringify(value);
|
|
16
|
+
}
|
|
17
|
+
catch (error) {
|
|
18
|
+
throw new CliOutputError(`Failed to encode ${outputFormat.toUpperCase()} output.`, "ENCODING_ERROR", { cause: error });
|
|
19
|
+
}
|
|
20
|
+
if (serialized === undefined) {
|
|
21
|
+
throw new CliOutputError(`${outputFormat.toUpperCase()} output did not produce a document.`, "ENCODING_ERROR");
|
|
22
|
+
}
|
|
23
|
+
const bytes = Buffer.byteLength(serialized, "utf8");
|
|
24
|
+
if (bytes > maxBytes) {
|
|
25
|
+
throw new CliOutputError(`${outputFormat === "toon" ? "TOON" : "JSON"} output exceeds the ${maxBytes}-byte limit (${bytes} bytes encoded).`, "OUTPUT_LIMIT");
|
|
26
|
+
}
|
|
27
|
+
return serialized;
|
|
28
|
+
}
|
package/dist/cli.js
CHANGED
|
@@ -4,7 +4,8 @@ import process from "node:process";
|
|
|
4
4
|
import { createInterface } from "node:readline/promises";
|
|
5
5
|
import { ZodError } from "zod";
|
|
6
6
|
import { BatchExecutionError, parseBatchDocument, runBatchDocument } from "./batch/runner.js";
|
|
7
|
-
import { MAX_BATCH_INPUT_BYTES } from "./batch/schema.js";
|
|
7
|
+
import { MAX_BATCH_INPUT_BYTES, isPrepareBatchTool } from "./batch/schema.js";
|
|
8
|
+
import { CliOutputError, serializeCliSuccess } from "./cli-output.js";
|
|
8
9
|
import { applyPersistedOperation, persistOperationPlan } from "./services/operation-plan-file.js";
|
|
9
10
|
import { AgentSetupError, detectInstalledAgents, resolveServerEntryPath, runAgentSetup, } from "./services/agent-setup.js";
|
|
10
11
|
import { installBundledSkill, resolveBundledSkillPath, } from "./services/skill-installer.js";
|
|
@@ -28,7 +29,7 @@ class CliError extends Error {
|
|
|
28
29
|
function usage() {
|
|
29
30
|
return [
|
|
30
31
|
"Usage:",
|
|
31
|
-
" ast-tool run <pipeline.json|->",
|
|
32
|
+
" ast-tool run <pipeline.json|-> [--output-format json|toon]",
|
|
32
33
|
" ast-tool validate <pipeline.json|->",
|
|
33
34
|
" ast-tool apply <plan.astplan> --plan-hash <sha256>",
|
|
34
35
|
" ast-tool install-skill [claude|hermes|all] [--scope user|project] [--project-root <path>] [--force]",
|
|
@@ -68,6 +69,32 @@ function parseApplyArgs(args) {
|
|
|
68
69
|
}
|
|
69
70
|
return { planFile: args[0], planHash: args[2] };
|
|
70
71
|
}
|
|
72
|
+
function parseRunArgs(args) {
|
|
73
|
+
let source;
|
|
74
|
+
let outputFormat = "json";
|
|
75
|
+
let outputFormatSeen = false;
|
|
76
|
+
for (let index = 0; index < args.length; index += 1) {
|
|
77
|
+
const argument = args[index];
|
|
78
|
+
if (argument === "--output-format") {
|
|
79
|
+
const value = args[index + 1];
|
|
80
|
+
if (outputFormatSeen || (value !== "json" && value !== "toon")) {
|
|
81
|
+
throw new CliError(usage(), "USAGE", 2, "run");
|
|
82
|
+
}
|
|
83
|
+
outputFormat = value;
|
|
84
|
+
outputFormatSeen = true;
|
|
85
|
+
index += 1;
|
|
86
|
+
continue;
|
|
87
|
+
}
|
|
88
|
+
if (source === undefined && !argument.startsWith("--")) {
|
|
89
|
+
source = argument;
|
|
90
|
+
continue;
|
|
91
|
+
}
|
|
92
|
+
throw new CliError(usage(), "USAGE", 2, "run");
|
|
93
|
+
}
|
|
94
|
+
if (!source)
|
|
95
|
+
throw new CliError(usage(), "USAGE", 2, "run");
|
|
96
|
+
return { source, outputFormat };
|
|
97
|
+
}
|
|
71
98
|
function parseInstallSkillArgs(args) {
|
|
72
99
|
let target = "all";
|
|
73
100
|
let targetSeen = false;
|
|
@@ -168,10 +195,14 @@ async function parseDocumentForCommand(source, command) {
|
|
|
168
195
|
export async function runCli(args) {
|
|
169
196
|
const [command, ...commandArgs] = args;
|
|
170
197
|
if (command === "run" || command === "validate") {
|
|
171
|
-
|
|
198
|
+
const runArgs = command === "run"
|
|
199
|
+
? parseRunArgs(commandArgs)
|
|
200
|
+
: commandArgs.length === 1 && commandArgs[0]
|
|
201
|
+
? { source: commandArgs[0], outputFormat: "json" }
|
|
202
|
+
: undefined;
|
|
203
|
+
if (!runArgs)
|
|
172
204
|
throw new CliError(usage(), "USAGE", 2, command);
|
|
173
|
-
|
|
174
|
-
const document = await parseDocumentForCommand(commandArgs[0], command);
|
|
205
|
+
const document = await parseDocumentForCommand(runArgs.source, command);
|
|
175
206
|
if (command === "validate") {
|
|
176
207
|
return {
|
|
177
208
|
version: 1,
|
|
@@ -179,6 +210,10 @@ export async function runCli(args) {
|
|
|
179
210
|
step_count: document.steps.length,
|
|
180
211
|
};
|
|
181
212
|
}
|
|
213
|
+
if (runArgs.outputFormat === "toon" &&
|
|
214
|
+
document.steps.some((step) => isPrepareBatchTool(step.tool))) {
|
|
215
|
+
throw new CliError("TOON output is available only for read-only batches; prepare results require JSON review coordinates.", "INVALID_BATCH", 2, command);
|
|
216
|
+
}
|
|
182
217
|
try {
|
|
183
218
|
return await runBatchDocument(document, {
|
|
184
219
|
persistPreparedOperation: (operationId) => persistOperationPlan(operationId),
|
|
@@ -262,7 +297,7 @@ export async function runCli(args) {
|
|
|
262
297
|
}
|
|
263
298
|
throw new CliError(usage(), "USAGE", 2, command ?? null);
|
|
264
299
|
}
|
|
265
|
-
function failure(error) {
|
|
300
|
+
function failure(error, command = null) {
|
|
266
301
|
if (error instanceof CliError) {
|
|
267
302
|
return {
|
|
268
303
|
value: {
|
|
@@ -276,6 +311,17 @@ function failure(error) {
|
|
|
276
311
|
exitCode: error.exitCode,
|
|
277
312
|
};
|
|
278
313
|
}
|
|
314
|
+
if (error instanceof CliOutputError) {
|
|
315
|
+
return {
|
|
316
|
+
value: {
|
|
317
|
+
status: "error",
|
|
318
|
+
command,
|
|
319
|
+
code: error.code,
|
|
320
|
+
message: error.message,
|
|
321
|
+
},
|
|
322
|
+
exitCode: 1,
|
|
323
|
+
};
|
|
324
|
+
}
|
|
279
325
|
return {
|
|
280
326
|
value: {
|
|
281
327
|
status: "error",
|
|
@@ -287,12 +333,14 @@ function failure(error) {
|
|
|
287
333
|
};
|
|
288
334
|
}
|
|
289
335
|
async function main() {
|
|
336
|
+
const args = process.argv.slice(2);
|
|
290
337
|
try {
|
|
291
|
-
const result = await runCli(
|
|
292
|
-
|
|
338
|
+
const result = await runCli(args);
|
|
339
|
+
const outputFormat = args[0] === "run" ? parseRunArgs(args.slice(1)).outputFormat : "json";
|
|
340
|
+
process.stdout.write(`${serializeCliSuccess(result, outputFormat)}\n`);
|
|
293
341
|
}
|
|
294
342
|
catch (error) {
|
|
295
|
-
const failed = failure(error);
|
|
343
|
+
const failed = failure(error, args[0] ?? null);
|
|
296
344
|
process.stderr.write(`${JSON.stringify(failed.value)}\n`);
|
|
297
345
|
process.exitCode = failed.exitCode;
|
|
298
346
|
}
|
|
@@ -3,7 +3,7 @@ import { Node } from "ts-morph";
|
|
|
3
3
|
import { z } from "zod";
|
|
4
4
|
import { PaginationInputSchema, PaginationOutputSchema, paginate } from "../services/pagination.js";
|
|
5
5
|
import { findDeclarationByName, getSourceFileOrThrow, withProject } from "../services/project.js";
|
|
6
|
-
import { errorResult,
|
|
6
|
+
import { errorResult, formattedResult, ToolOutputFormatInputSchema } from "./result.js";
|
|
7
7
|
const AstFindReferencesInputSchema = z.object({
|
|
8
8
|
project_root: z
|
|
9
9
|
.string()
|
|
@@ -11,6 +11,7 @@ const AstFindReferencesInputSchema = z.object({
|
|
|
11
11
|
file_path: z.string().describe("File containing the declaration."),
|
|
12
12
|
symbol_path: z.string().describe("Exact symbol path returned by an outline or symbol search."),
|
|
13
13
|
include_declaration: z.boolean().default(true),
|
|
14
|
+
...ToolOutputFormatInputSchema,
|
|
14
15
|
...PaginationInputSchema,
|
|
15
16
|
});
|
|
16
17
|
const ReferenceSchema = z.object({
|
|
@@ -34,14 +35,13 @@ export function registerFindReferences(server) {
|
|
|
34
35
|
title: "Find semantic symbol references",
|
|
35
36
|
description: "Finds type-resolved references and returns bounded project-relative locations, including declaration impact by default.",
|
|
36
37
|
inputSchema: AstFindReferencesInputSchema,
|
|
37
|
-
outputSchema: AstFindReferencesOutputSchema,
|
|
38
38
|
annotations: {
|
|
39
39
|
readOnlyHint: true,
|
|
40
40
|
destructiveHint: false,
|
|
41
41
|
idempotentHint: true,
|
|
42
42
|
openWorldHint: false,
|
|
43
43
|
},
|
|
44
|
-
}, async ({ project_root, file_path, symbol_path, include_declaration, offset, limit }) => {
|
|
44
|
+
}, async ({ project_root, file_path, symbol_path, include_declaration, output_format, offset, limit, }) => {
|
|
45
45
|
try {
|
|
46
46
|
const structuredContent = await withProject(project_root, ({ project, projectRoot }) => {
|
|
47
47
|
const sourceFile = getSourceFileOrThrow(project, file_path);
|
|
@@ -88,7 +88,7 @@ export function registerFindReferences(server) {
|
|
|
88
88
|
...metadata,
|
|
89
89
|
};
|
|
90
90
|
});
|
|
91
|
-
return
|
|
91
|
+
return formattedResult(AstFindReferencesOutputSchema, structuredContent, output_format);
|
|
92
92
|
}
|
|
93
93
|
catch (error) {
|
|
94
94
|
return errorResult(error);
|
|
@@ -2,7 +2,7 @@ import { z } from "zod";
|
|
|
2
2
|
import { normalizeDiagnostic } from "../services/diagnostics.js";
|
|
3
3
|
import { PaginationInputSchema, PaginationOutputSchema, paginate } from "../services/pagination.js";
|
|
4
4
|
import { getSourceFileOrThrow, withProject } from "../services/project.js";
|
|
5
|
-
import { errorResult,
|
|
5
|
+
import { errorResult, formattedResult, ToolOutputFormatInputSchema } from "./result.js";
|
|
6
6
|
const DiagnosticSchema = z.object({
|
|
7
7
|
code: z.number().int(),
|
|
8
8
|
category: z.string(),
|
|
@@ -19,6 +19,7 @@ const AstGetDiagnosticsInputSchema = z.object({
|
|
|
19
19
|
.string()
|
|
20
20
|
.optional()
|
|
21
21
|
.describe("Optional project-relative or absolute file path. Omit for project diagnostics."),
|
|
22
|
+
...ToolOutputFormatInputSchema,
|
|
22
23
|
...PaginationInputSchema,
|
|
23
24
|
});
|
|
24
25
|
const AstGetDiagnosticsOutputSchema = z.object({
|
|
@@ -33,14 +34,13 @@ export function registerGetDiagnostics(server) {
|
|
|
33
34
|
title: "Get TypeScript diagnostics",
|
|
34
35
|
description: "Returns bounded normalized TypeScript diagnostics for a project or one source file. Existing errors are preserved as evidence for write-operation delta checks.",
|
|
35
36
|
inputSchema: AstGetDiagnosticsInputSchema,
|
|
36
|
-
outputSchema: AstGetDiagnosticsOutputSchema,
|
|
37
37
|
annotations: {
|
|
38
38
|
readOnlyHint: true,
|
|
39
39
|
destructiveHint: false,
|
|
40
40
|
idempotentHint: true,
|
|
41
41
|
openWorldHint: false,
|
|
42
42
|
},
|
|
43
|
-
}, async ({ project_root, file_path, offset, limit }) => {
|
|
43
|
+
}, async ({ project_root, file_path, output_format, offset, limit }) => {
|
|
44
44
|
try {
|
|
45
45
|
const structuredContent = await withProject(project_root, ({ project, projectRoot }) => {
|
|
46
46
|
const startedAt = performance.now();
|
|
@@ -61,7 +61,7 @@ export function registerGetDiagnostics(server) {
|
|
|
61
61
|
...metadata,
|
|
62
62
|
};
|
|
63
63
|
});
|
|
64
|
-
return
|
|
64
|
+
return formattedResult(AstGetDiagnosticsOutputSchema, structuredContent, output_format);
|
|
65
65
|
}
|
|
66
66
|
catch (error) {
|
|
67
67
|
return errorResult(error);
|
package/dist/tools/result.js
CHANGED
|
@@ -1,12 +1,56 @@
|
|
|
1
|
+
import { Buffer } from "node:buffer";
|
|
2
|
+
import { isDeepStrictEqual } from "node:util";
|
|
3
|
+
import { decode, encode } from "@toon-format/toon";
|
|
4
|
+
import { z } from "zod";
|
|
5
|
+
export const MAX_TOOL_ERROR_BYTES = 64 * 1024;
|
|
6
|
+
export const MAX_TOON_RESULT_BYTES = 10 * 1024 * 1024;
|
|
7
|
+
export const ToolOutputFormatSchema = z.enum(["json", "toon"]).default("json");
|
|
8
|
+
export const ToolOutputFormatInputSchema = {
|
|
9
|
+
output_format: ToolOutputFormatSchema.describe("Result representation. TOON wraps one compact encoded value in structuredContent."),
|
|
10
|
+
};
|
|
11
|
+
export const ToonResultSchema = z.object({
|
|
12
|
+
format: z.literal("toon"),
|
|
13
|
+
data: z.string(),
|
|
14
|
+
});
|
|
1
15
|
export function structuredResult(structuredContent) {
|
|
2
16
|
return { content: [], structuredContent };
|
|
3
17
|
}
|
|
18
|
+
export function formattedResult(outputSchema, candidate, outputFormat, maxToonBytes = MAX_TOON_RESULT_BYTES) {
|
|
19
|
+
const validated = outputSchema.parse(candidate);
|
|
20
|
+
if (outputFormat === "json") {
|
|
21
|
+
return structuredResult(validated);
|
|
22
|
+
}
|
|
23
|
+
const text = encode(validated);
|
|
24
|
+
const byteLength = Buffer.byteLength(text, "utf8");
|
|
25
|
+
if (byteLength > maxToonBytes) {
|
|
26
|
+
throw new Error(`TOON result exceeds the ${maxToonBytes}-byte limit (${byteLength} bytes encoded).`);
|
|
27
|
+
}
|
|
28
|
+
if (!isDeepStrictEqual(decode(text), validated)) {
|
|
29
|
+
throw new Error("TOON result cannot represent the validated value losslessly.");
|
|
30
|
+
}
|
|
31
|
+
return structuredResult({ format: "toon", data: text });
|
|
32
|
+
}
|
|
33
|
+
function truncateUtf8(value, maxBytes) {
|
|
34
|
+
if (Buffer.byteLength(value, "utf8") <= maxBytes)
|
|
35
|
+
return value;
|
|
36
|
+
const suffix = "... [truncated]";
|
|
37
|
+
const source = Buffer.from(value, "utf8");
|
|
38
|
+
let end = Math.max(0, maxBytes - Buffer.byteLength(suffix, "utf8"));
|
|
39
|
+
while (end > 0 && (source[end] & 0xc0) === 0x80)
|
|
40
|
+
end -= 1;
|
|
41
|
+
let prefix = source.subarray(0, end).toString("utf8");
|
|
42
|
+
while (Buffer.byteLength(prefix + suffix, "utf8") > maxBytes) {
|
|
43
|
+
prefix = prefix.slice(0, -1);
|
|
44
|
+
}
|
|
45
|
+
return prefix + suffix;
|
|
46
|
+
}
|
|
4
47
|
export function errorResult(error) {
|
|
48
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
5
49
|
return {
|
|
6
50
|
content: [
|
|
7
51
|
{
|
|
8
52
|
type: "text",
|
|
9
|
-
text:
|
|
53
|
+
text: truncateUtf8(message, MAX_TOOL_ERROR_BYTES),
|
|
10
54
|
},
|
|
11
55
|
],
|
|
12
56
|
isError: true,
|
|
@@ -4,7 +4,7 @@ import { buildFileOutline } from "../services/outline.js";
|
|
|
4
4
|
import { PaginationInputSchema, PaginationOutputSchema, paginate } from "../services/pagination.js";
|
|
5
5
|
import { withProject } from "../services/project.js";
|
|
6
6
|
import { collectSymbols } from "../services/symbols.js";
|
|
7
|
-
import { errorResult,
|
|
7
|
+
import { errorResult, formattedResult, ToolOutputFormatInputSchema } from "./result.js";
|
|
8
8
|
const AstSearchSymbolsInputSchema = z.object({
|
|
9
9
|
project_root: z
|
|
10
10
|
.string()
|
|
@@ -21,6 +21,7 @@ const AstSearchSymbolsInputSchema = z.object({
|
|
|
21
21
|
.string()
|
|
22
22
|
.optional()
|
|
23
23
|
.describe("Optional case-insensitive substring matched against project-relative file paths."),
|
|
24
|
+
...ToolOutputFormatInputSchema,
|
|
24
25
|
...PaginationInputSchema,
|
|
25
26
|
});
|
|
26
27
|
const SearchSymbolSchema = z.object({
|
|
@@ -42,14 +43,13 @@ export function registerSearchSymbols(server) {
|
|
|
42
43
|
title: "Search project symbols",
|
|
43
44
|
description: "Searches declarations structurally and returns exact file/symbol selectors that can be passed to the other AST tools.",
|
|
44
45
|
inputSchema: AstSearchSymbolsInputSchema,
|
|
45
|
-
outputSchema: AstSearchSymbolsOutputSchema,
|
|
46
46
|
annotations: {
|
|
47
47
|
readOnlyHint: true,
|
|
48
48
|
destructiveHint: false,
|
|
49
49
|
idempotentHint: true,
|
|
50
50
|
openWorldHint: false,
|
|
51
51
|
},
|
|
52
|
-
}, async ({ project_root, query, kinds, file_filter, offset, limit }) => {
|
|
52
|
+
}, async ({ project_root, query, kinds, file_filter, output_format, offset, limit }) => {
|
|
53
53
|
try {
|
|
54
54
|
const structuredContent = await withProject(project_root, ({ project, projectRoot }) => {
|
|
55
55
|
const startedAt = performance.now();
|
|
@@ -92,7 +92,7 @@ export function registerSearchSymbols(server) {
|
|
|
92
92
|
...metadata,
|
|
93
93
|
};
|
|
94
94
|
});
|
|
95
|
-
return
|
|
95
|
+
return formattedResult(AstSearchSymbolsOutputSchema, structuredContent, output_format);
|
|
96
96
|
}
|
|
97
97
|
catch (error) {
|
|
98
98
|
return errorResult(error);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ast-mcp-server",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"description": "Structural code MCP server and batch CLI for compact TypeScript/JavaScript reads and hash-bound AST edits.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"mcp",
|
|
@@ -40,6 +40,9 @@
|
|
|
40
40
|
"engines": {
|
|
41
41
|
"node": ">=20.19"
|
|
42
42
|
},
|
|
43
|
+
"resolutions": {
|
|
44
|
+
"hono": "4.12.34"
|
|
45
|
+
},
|
|
43
46
|
"scripts": {
|
|
44
47
|
"build": "tsc",
|
|
45
48
|
"typecheck": "tsc --noEmit && tsc -p tsconfig.test.json",
|
|
@@ -51,6 +54,7 @@
|
|
|
51
54
|
"benchmark": "yarn build && node scripts/benchmark.mjs",
|
|
52
55
|
"benchmark:corpus": "yarn build && node scripts/benchmark-corpus.mjs",
|
|
53
56
|
"benchmark:batch": "yarn build && node scripts/benchmark-batch.mjs",
|
|
57
|
+
"benchmark:formats": "yarn build && node scripts/benchmark-formats.mjs",
|
|
54
58
|
"lint": "eslint src test scripts vitest.config.ts eslint.config.js",
|
|
55
59
|
"format": "prettier --write .",
|
|
56
60
|
"format:check": "prettier --check .",
|
|
@@ -62,6 +66,7 @@
|
|
|
62
66
|
},
|
|
63
67
|
"dependencies": {
|
|
64
68
|
"@modelcontextprotocol/sdk": "^1.12.0",
|
|
69
|
+
"@toon-format/toon": "4.1.0",
|
|
65
70
|
"diff": "^9.0.0",
|
|
66
71
|
"ts-morph": "24.0.0",
|
|
67
72
|
"zod": "^3.23.8"
|
|
@@ -70,6 +75,7 @@
|
|
|
70
75
|
"@eslint/js": "^10.0.1",
|
|
71
76
|
"@types/node": "^22.5.0",
|
|
72
77
|
"eslint": "^10.8.0",
|
|
78
|
+
"gpt-tokenizer": "3.4.0",
|
|
73
79
|
"prettier": "^3.9.6",
|
|
74
80
|
"typescript": "^5.6.0",
|
|
75
81
|
"typescript-eslint": "^8.65.0",
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: structural-code-editing
|
|
3
3
|
description: Leer, navegar y editar proyectos TypeScript/JavaScript mediante el resolver real del compilador y operaciones AST preparadas, revisadas y vinculadas por hash.
|
|
4
|
-
version: "3.
|
|
4
|
+
version: "3.4.0"
|
|
5
5
|
author: "yail"
|
|
6
6
|
license: "ISC"
|
|
7
7
|
metadata:
|
|
@@ -49,6 +49,20 @@ El setup falla cerrado ante un registro `ast` conflictivo. No eliminar ni reempl
|
|
|
49
49
|
|
|
50
50
|
Los clientes pueden prefijar los nombres publicados (`ast_*`) según su convención MCP; elegir por el nombre base y el schema, no adivinar el prefijo.
|
|
51
51
|
|
|
52
|
+
## Formato de resultado para el modelo
|
|
53
|
+
|
|
54
|
+
`ast_search_symbols`, `ast_find_references` y `ast_get_diagnostics` aceptan `output_format: "toon"`. Usarlo cuando la respuesta vaya directo al modelo y se espere una colección uniforme con varios registros. Omitirlo, o usar `json`, para automatización que dependa del objeto canónico.
|
|
55
|
+
|
|
56
|
+
En MCP, TOON llega como un único envelope estructurado `{ "format": "toon", "data": "..." }`; `data` contiene el documento TOON lossless. No hay una copia JSON completa en paralelo. Las tres tools validan primero el valor canónico con Zod, pero no publican un `outputSchema` MCP único porque tienen dos representaciones de éxito.
|
|
57
|
+
|
|
58
|
+
No pedir TOON para `ast_list_files`, outlines, source, previews ni mutaciones: los benchmarks muestran que el envelope empeora esas formas pequeñas, multiline o diff-heavy. Para un pipeline read-only conocido, se puede compactar únicamente el resultado final con:
|
|
59
|
+
|
|
60
|
+
```text
|
|
61
|
+
ast-tool run pipeline.json --output-format toon
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Los pasos internos del batch siempre son JSON estructurado. TOON dentro de `step.input` y TOON final para un batch de preparación se rechazan antes de ejecutar la operación.
|
|
65
|
+
|
|
52
66
|
## Batch CLI para clientes con Bash
|
|
53
67
|
|
|
54
68
|
Cuando un pipeline conocido requiere varias llamadas MCP dependientes y el cliente tiene Bash, usar `ast-tool run pipeline.json` para colapsar los roundtrips del modelo. No usar batch para exploración abierta ni asumir que reduce el trabajo interno del compilador.
|
|
@@ -59,7 +73,7 @@ Cuando un pipeline conocido requiere varias llamadas MCP dependientes y el clien
|
|
|
59
73
|
- Mantener paginación y filtros: batch elimina roundtrips, no vuelve razonable leer un monorepo entero.
|
|
60
74
|
- No generar JavaScript/eval dentro del documento; el contrato es declarativo y limitado.
|
|
61
75
|
|
|
62
|
-
`ast-tool validate pipeline.json` valida schema, orden de referencias y política sin cargar el proyecto. Límites por defecto: 50 steps, 500 invocaciones, 200 items por foreach, concurrencia 4 (máximo 16), input 1 MiB, 10 MiB por resultado retenido/output y 50 MiB de contexto intermedio acumulado.
|
|
76
|
+
`ast-tool validate pipeline.json` valida schema, orden de referencias y política sin cargar el proyecto. Límites por defecto: 50 steps, 500 invocaciones, 200 items por foreach, concurrencia 4 (máximo 16), input 1 MiB, 10 MiB por resultado retenido/output y 50 MiB de contexto intermedio acumulado. Los errores del CLI permanecen como JSON en stderr aunque el output exitoso solicitado sea TOON.
|
|
63
77
|
|
|
64
78
|
## Flujo obligatorio de mutación
|
|
65
79
|
|