ast-mcp-server 0.5.0 → 0.6.0

package/CHANGELOG.md

@@ -4,6 +4,33 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.6.0] - 2026-08-05
+
+### Added
+
+- Read-only `ast_get_file` with bounded exact source lines, SHA-256 byte hashes, compiler snapshot state, and `symbols_only` body-free output.
+- Project-scoped file snapshot validation with UTF-8 checks, traversal rejection, symlink containment checks, and ambiguous-path failures.
+- Read-only `ast_explore` with query/file/symbol routing, progressive source and compiler-reference evidence, freshness/completeness metadata, unresolved selectors, and serialized byte budgets.
+- A deterministic context workflow benchmark comparing full-file, primitive, and composed exploration workflows.
+
+## [0.5.1] - 2026-08-05
+
+### Added
+
+- Bounded JSON-safe project status contracts and the read-only `ast_get_project_status` tool.
+- Session freshness metadata with separate source, config, and canonical fingerprints.
+- Serialized operation-queue accounting and status integration without changing reviewed mutation semantics.
+
+### Changed
+
+- Source synchronization verifies stability across refresh, snapshot, refresh, and verification before reporting fresh state.
+- Phase 1 keeps the symbol index disabled while exposing explicit stale/degraded recovery state.
+
+### Security
+
+- Status projections redact identities, absolute paths, credentials, and multi-token `Authorization` values at both transition and projection boundaries.
+- Noncanonical external fingerprints are converted to bounded opaque SHA-256 digests before projection.
+
 ## [0.5.0] - 2026-08-03
 
 ### Added
@@ -66,6 +93,7 @@ Initial public release.
 - Apply rejects stale workspaces, changed configs/sources, mismatched plan hashes, unsafe plan files, conflicting MCP registrations, and conflicting skill content.
 - Inputs, outputs, operation stores, subprocesses, batch fan-out, plan lifetime, and filesystem access are bounded.
 
+[0.5.1]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.5.1
 [0.5.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.5.0
 [0.4.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.4.0
 [0.3.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.3.0

package/README.md

@@ -23,13 +23,25 @@ This MCP server gives the agent structural code tools in addition to generic fil
 
 | Need                           | Structural operation      | Returned scope                                        |
 | ------------------------------ | ------------------------- | ----------------------------------------------------- |
+| Read a bounded file            | `ast_get_file`            | Exact selected source lines, hash, and snapshot state |
+| Explore bounded context        | `ast_explore`             | Ranked selectors plus optional source and references  |
 | Understand a file              | `ast_get_outline`         | Signatures without implementation bodies              |
 | Inspect one declaration        | `ast_get_symbol_source`   | Exact source for one function, method, class, or type |
 | Find usages across the project | `ast_find_references`     | Compiler-resolved reference locations                 |
 | Rename a symbol everywhere     | `ast_rename_symbol`       | A reviewed project-wide rename plan                   |
 | Change one implementation      | `ast_replace_symbol_body` | A body-only plan that preserves the declaration       |
 
-Reads can start with a compact outline and fetch exact source only when needed. Mutations are prepared in memory first, compared against baseline diagnostics, and returned as immutable, hash-bound plans. Nothing is written until the caller reviews and explicitly applies the plan.
+Reads can start with a bounded file slice, a compact outline, or exact source only for the declaration that needs inspection. Mutations are prepared in memory first, compared against baseline diagnostics, and returned as immutable, hash-bound plans. Nothing is written until the caller reviews and explicitly applies the plan.
+
+### Choosing a read tool
+
+- Use `ast_get_file` when the file path is known and the agent needs exact source lines. It is read-only, uses zero-based `offset` and bounded `limit`, returns one-based line records, a SHA-256 byte hash, and `snapshot_state`.
+- Use `ast_get_file` with `symbols_only: true` when only selectors and body-free signatures are needed from one known file.
+- Use `ast_explore` when the question spans discovery and evidence. Its default summary is bounded; use `detail: "context"` for selected source and `detail: "full"` for source plus compiler references.
+- Use `ast_get_outline` for a compact body-free view of a known file without source lines.
+- Use `ast_get_symbol_source` when one declaration or implementation is the required evidence.
+
+`snapshot_state: "fresh"` means that the returned file bytes match the synchronized compiler snapshot. It does not mean that the project has zero TypeScript diagnostics; use `ast_get_diagnostics` for compiler errors and warnings.
 
 ## Why this helps
 
@@ -39,7 +51,7 @@ Reads can start with a compact outline and fetch exact source only when needed.
 
 AST-aware editing is not a proof that a change is semantically correct. The safety comes from combining structural selection with diagnostics, exact previews, reviewed hashes, freshness checks, and fail-closed apply semantics.
 
-The included batch benchmark records a 50% reduction in model round-trips and a 94.67% reduction in serialized context for its search-to-source scenario. The result-shaping corpus records a 68.80% reduction in aggregate model-facing TOON tokens while preserving declared selectors/reference coordinates with the same six logical calls. The separate format benchmark records 25.87% across its eligible collection corpus. These are reproducible local `o200k_base` estimates, not universal token, billing, cache, or latency claims.
+The included batch benchmark records a 50% reduction in model round-trips and a 94.67% reduction in serialized context for its search-to-source scenario. The result-shaping corpus records a 68.80% reduction in aggregate model-facing TOON tokens while preserving declared selectors/reference coordinates with the same six logical calls. The separate format benchmark records 25.87% across its eligible collection corpus. The context workflow benchmark verifies evidence preservation and call bounds for full-file, primitive, and `ast_explore` workflows. These are reproducible local `o200k_base` estimates, not universal token, billing, cache, or latency claims.
 
 ## Requirements
 
@@ -217,6 +229,9 @@ Project-scoped tools accept `project_root`, either the project directory or an e
 | Tool                        | Purpose                                                              | Mutates files |
 | --------------------------- | -------------------------------------------------------------------- | ------------- |
 | `ast_list_files`            | Paginated, project-relative source file inventory                    | No            |
+| `ast_get_project_status`    | Read-only compiler, freshness, index, and operation status           | No            |
+| `ast_explore`               | Bounded composed selectors, source evidence, and references          | No            |
+| `ast_get_file`              | Bounded exact source lines, byte hash, and snapshot state            | No            |
 | `ast_get_outline`           | Body-free declaration signatures; detailed symbol metadata is opt-in | No            |
 | `ast_get_symbol_source`     | Exact source for one declaration                                     | No            |
 | `ast_search_symbols`        | Paginated structural symbol discovery                                | No            |
@@ -230,6 +245,8 @@ Project-scoped tools accept `project_root`, either the project directory or an e
 
 Read results use project-relative paths, deterministic ordering, structured MCP output, and pagination where result sets can grow with the project.
 
+`ast_explore` supports query, exact file, and exact symbol routes. Its default `summary` profile returns bounded reusable selectors; `context` adds selected source and `full` adds compiler references. Every response reports freshness, completeness, truncation, unresolved selectors, record limits, and a serialized byte budget. Use the primitive tools when a single exact operation is clearer or when preparing a mutation.
+
 Symbol search is relevance-ranked and defaults to at most 20 `summary` records containing `file`, a directly reusable `selector`, `kind`, and body-free `signature`. Request `detail: "selectors"` for routing coordinates only, or `detail: "full", limit: 100` for the v0.4.0 fields/page. References default to `detail: "locations"`; request `detail: "context"` only when the bounded source line is needed.
 
 ### Optional TOON results
@@ -392,7 +409,7 @@ The batch benchmark compares two separate client calls with one batch invocation
 
 The format benchmark runs real tools against this repository plus deterministic reference/diagnostic fixtures. It checks JSON→TOON→value equality, UTF-8 bytes, `gpt-tokenizer` `o200k_base` estimates, encode/decode latency, the actual MCP envelope, tool metadata, and negative controls. Its checked result is `benchmark/results/self-formats.json`; local tokenizer estimates do not establish provider-side billing or cache savings. See `benchmark/README.md` for methodology and limitations.
 
-The result-shaping benchmark compares the v0.4.0-compatible `full/100/context` profiles with the new public defaults across exact-name, exact-path, prefix, broad-substring, and multi-file-reference tasks. It fails on missing evidence, extra logical calls, fewer than 11 tools, or less than 35% aggregate TOON token reduction. Its checked result is `benchmark/results/self-result-shapes.json`.
+The result-shaping benchmark compares the v0.4.0-compatible `full/100/context` profiles with the new public defaults across exact-name, exact-path, prefix, broad-substring, and multi-file-reference tasks. It fails on missing evidence, extra required calls, fewer than the benchmark's required minimum tool surface, or less than 35% aggregate TOON token reduction. Its checked result is `benchmark/results/self-result-shapes.json`.
 
 ## Scope
 

package/dist/server.js

@@ -3,14 +3,17 @@ import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
 import { registerApplyOperation } from "./tools/apply_operation.js";
 import { registerFindReferences } from "./tools/find_references.js";
 import { registerGetDiagnostics } from "./tools/get_diagnostics.js";
+import { registerGetFile } from "./tools/get_file.js";
 import { registerGetOperationPreview } from "./tools/get_operation_preview.js";
 import { registerGetOutline } from "./tools/get_outline.js";
+import { registerGetProjectStatus } from "./tools/get_project_status.js";
 import { registerGetSymbolSource } from "./tools/get_symbol_source.js";
 import { registerListFiles } from "./tools/list_files.js";
 import { registerRenameSymbol } from "./tools/rename_symbol.js";
 import { registerReplaceSymbolBody } from "./tools/replace_symbol_body.js";
 import { registerScaffoldClass } from "./tools/scaffold_class.js";
 import { registerSearchSymbols } from "./tools/search_symbols.js";
+import { registerExplore } from "./tools/explore.js";
 const packageMetadata = createRequire(import.meta.url)("../package.json");
 export const PACKAGE_VERSION = packageMetadata.version;
 export function createServer() {
@@ -19,11 +22,14 @@ export function createServer() {
         version: PACKAGE_VERSION,
     });
     registerListFiles(server);
+    registerGetProjectStatus(server);
+    registerExplore(server);
     registerGetOutline(server);
     registerGetSymbolSource(server);
     registerSearchSymbols(server);
     registerFindReferences(server);
     registerGetDiagnostics(server);
+    registerGetFile(server);
     registerRenameSymbol(server);
     registerReplaceSymbolBody(server);
     registerScaffoldClass(server);

package/dist/services/context-builder.js

@@ -0,0 +1,234 @@
+import { Buffer } from "node:buffer";
+import path from "node:path";
+import { buildFileOutline, nodeSourceWithLocation } from "./outline.js";
+import { paginate } from "./pagination.js";
+import { findDeclarationByName, getSourceFileOrThrow } from "./project.js";
+import { collectSymbolReferences } from "./references.js";
+import { searchProjectSymbols, sourceFileSymbols, symbolMatchRank, } from "./symbols.js";
+export const EXPLORE_DEFAULT_LIMIT = 20;
+export const EXPLORE_DEFAULT_REFERENCE_LIMIT = 20;
+export const EXPLORE_DEFAULT_MAX_BYTES = 64 * 1024;
+export const EXPLORE_MAX_BYTES = 1024 * 1024;
+function matchesQuery(record, query) {
+    const normalizedQuery = query.toLowerCase();
+    return (record.name.toLowerCase().includes(normalizedQuery) ||
+        record.symbol_path.toLowerCase().includes(normalizedQuery) ||
+        record.selector.toLowerCase().includes(normalizedQuery));
+}
+function sortRecords(records, query) {
+    return records.sort((left, right) => {
+        if (query) {
+            const rank = symbolMatchRank(query, {
+                symbolPath: left.symbol_path,
+                name: left.name,
+                line: left.line,
+            }) -
+                symbolMatchRank(query, {
+                    symbolPath: right.symbol_path,
+                    name: right.name,
+                    line: right.line,
+                });
+            if (rank !== 0)
+                return rank;
+        }
+        return (left.file.localeCompare(right.file) ||
+            left.line - right.line ||
+            left.symbol_path.localeCompare(right.symbol_path) ||
+            left.kind.localeCompare(right.kind));
+    });
+}
+function recordFromOutlineSymbol(sourceFile, projectRoot, symbol) {
+    return {
+        file: path.relative(projectRoot, sourceFile.getFilePath()).replaceAll("\\", "/"),
+        symbol_path: symbol.symbolPath,
+        selector: `${symbol.symbolPath}@${symbol.startLine}`,
+        name: symbol.name,
+        kind: symbol.kind,
+        line: symbol.startLine,
+        signature: symbol.signature,
+    };
+}
+function fileRecords(sourceFile, projectRoot) {
+    const outline = buildFileOutline(sourceFile);
+    if (outline.symbols.length > 0) {
+        return outline.symbols.map((symbol) => recordFromOutlineSymbol(sourceFile, projectRoot, symbol));
+    }
+    return sourceFileSymbols(sourceFile, projectRoot);
+}
+function projectRecords(context, request) {
+    const { project, projectRoot } = context;
+    if (request.filePath) {
+        const sourceFile = getSourceFileOrThrow(project, request.filePath);
+        const file = path.relative(projectRoot, sourceFile.getFilePath()).replaceAll("\\", "/");
+        const requestedSymbolPath = request.symbolPath;
+        if (requestedSymbolPath) {
+            const node = findDeclarationByName(sourceFile, requestedSymbolPath);
+            const record = fileRecords(sourceFile, projectRoot).find((candidate) => candidate.symbol_path === requestedSymbolPath ||
+                candidate.selector === requestedSymbolPath ||
+                (candidate.symbol_path === requestedSymbolPath.split("@")[0] &&
+                    candidate.line === node.getStartLineNumber()));
+            if (!record) {
+                throw new Error(`Symbol "${requestedSymbolPath}" could not be projected from ${file}.`);
+            }
+            return {
+                route: "symbol",
+                file,
+                symbol: record.selector,
+                records: [record],
+            };
+        }
+        const records = fileRecords(sourceFile, projectRoot)
+            .filter((record) => !request.query || matchesQuery(record, request.query))
+            .filter((record) => !request.kinds || request.kinds.includes(record.kind));
+        return { route: "file", file, symbol: null, records: sortRecords(records, request.query) };
+    }
+    if (!request.query) {
+        throw new Error("ast_explore requires query, file_path, or both file_path and symbol_path.");
+    }
+    return {
+        route: "query",
+        file: null,
+        symbol: null,
+        records: searchProjectSymbols(project, projectRoot, {
+            query: request.query,
+            kinds: request.kinds,
+            fileFilter: request.fileFilter,
+        }),
+    };
+}
+function projectSymbol(detail, record) {
+    const base = { file: record.file, selector: record.selector, kind: record.kind };
+    if (detail === "selectors")
+        return base;
+    if (detail === "summary" || detail === "context") {
+        return { ...base, signature: record.signature };
+    }
+    return {
+        ...base,
+        symbol_path: record.symbol_path,
+        name: record.name,
+        line: record.line,
+        signature: record.signature,
+    };
+}
+function jsonBytes(value) {
+    return Buffer.byteLength(JSON.stringify(value), "utf8");
+}
+function effectiveExpansion(request) {
+    return {
+        source: request.includeSource ?? (request.detail === "context" || request.detail === "full"),
+        references: request.includeReferences ?? request.detail === "full",
+    };
+}
+export function buildExploreContext(context, request) {
+    const routed = projectRecords(context, request);
+    const page = paginate(routed.records, request.offset, request.limit);
+    const expansion = effectiveExpansion(request);
+    const unresolved = [];
+    const evidence = [];
+    for (const record of page.items) {
+        let sourceFile;
+        try {
+            sourceFile = getSourceFileOrThrow(context.project, record.file);
+        }
+        catch {
+            unresolved.push({ selector: record.selector, reason: "source_unresolved" });
+            continue;
+        }
+        let node;
+        try {
+            node = findDeclarationByName(sourceFile, record.selector);
+        }
+        catch {
+            unresolved.push({ selector: record.selector, reason: "source_unresolved" });
+            continue;
+        }
+        let source;
+        let references;
+        if (expansion.source)
+            source = nodeSourceWithLocation(node, context.projectRoot);
+        if (expansion.references) {
+            try {
+                references = collectSymbolReferences(node, context.projectRoot, record.selector, true, request.referenceDetail, 0, request.referenceLimit);
+            }
+            catch {
+                unresolved.push({ selector: record.selector, reason: "references_unresolved" });
+            }
+        }
+        if (source || references) {
+            evidence.push({
+                selector: record.selector,
+                ...(source ? { source } : {}),
+                ...(references ? { references } : {}),
+            });
+        }
+    }
+    const symbols = page.items.map((record) => projectSymbol(request.detail, record));
+    let returnedSymbols = [...symbols];
+    let returnedEvidence = [...evidence];
+    let reason = page.has_more ? "record_limit" : null;
+    let result = createResult(context, request, routed.route, routed.file, routed.symbol, returnedSymbols, returnedEvidence, page.total, unresolved, reason);
+    while (jsonBytes(result) > request.maxBytes &&
+        (returnedEvidence.length > 0 || returnedSymbols.length > 0)) {
+        reason = "byte_limit";
+        if (returnedEvidence.length > 0) {
+            returnedEvidence = returnedEvidence.slice(0, -1);
+        }
+        else {
+            returnedSymbols = returnedSymbols.slice(0, -1);
+            const selectors = new Set(returnedSymbols.map((symbol) => symbol.selector));
+            returnedEvidence = returnedEvidence.filter((item) => selectors.has(item.selector));
+        }
+        result = createResult(context, request, routed.route, routed.file, routed.symbol, returnedSymbols, returnedEvidence, page.total, unresolved, reason);
+    }
+    const usedBytes = jsonBytes(result);
+    return {
+        ...result,
+        budget: { ...result.budget, used_bytes: usedBytes },
+    };
+}
+function createResult(context, request, route, file, symbol, symbols, evidence, total, unresolved, reason) {
+    const hasMore = request.offset + symbols.length < total;
+    const nextOffset = hasMore ? request.offset + symbols.length : null;
+    const expansion = effectiveExpansion(request);
+    const evidenceComplete = unresolved.length === 0 &&
+        evidence.length >= (expansion.source || expansion.references ? symbols.length : 0);
+    const symbolsComplete = !hasMore;
+    return {
+        route,
+        query: request.query ?? null,
+        file,
+        symbol,
+        detail: request.detail,
+        symbols,
+        evidence,
+        offset: request.offset,
+        limit: request.limit,
+        total,
+        has_more: hasMore,
+        next_offset: nextOffset,
+        freshness: {
+            state: context.status.state,
+            causes: context.status.causes,
+            checked_at: context.status.lastSuccessfulSyncAt,
+        },
+        completeness: {
+            complete: symbolsComplete && evidenceComplete,
+            symbols_complete: symbolsComplete,
+            evidence_complete: evidenceComplete,
+            unresolved,
+        },
+        budget: {
+            max_records: request.limit,
+            max_bytes: request.maxBytes,
+            max_depth: null,
+            max_edges: null,
+            max_invocations: 1,
+            used_bytes: 0,
+        },
+        truncation: {
+            truncated: reason !== null || hasMore,
+            reason: reason ?? (hasMore ? "record_limit" : null),
+        },
+    };
+}

package/dist/services/file-snapshot.js

@@ -0,0 +1,96 @@
+import { TextDecoder } from "node:util";
+import { lstat, readFile, realpath } from "node:fs/promises";
+import path from "node:path";
+import { getSourceFileOrThrow } from "./project.js";
+import { hashBytes } from "./workspace.js";
+export const DEFAULT_FILE_LINE_LIMIT = 200;
+export const MAX_FILE_LINE_LIMIT = 500;
+function assertBoundedRange({ offset, limit }) {
+    if (!Number.isInteger(offset) || offset < 0) {
+        throw new Error("File offset must be a non-negative integer.");
+    }
+    if (!Number.isInteger(limit) || limit < 1 || limit > MAX_FILE_LINE_LIMIT) {
+        throw new Error(`File limit must be an integer between 1 and ${MAX_FILE_LINE_LIMIT}.`);
+    }
+}
+function rejectTraversalPath(filePath) {
+    const normalized = filePath.replaceAll("\\", "/");
+    const segments = normalized.split("/");
+    const isWindowsAbsolute = /^[A-Za-z]:\//.test(normalized) || normalized.startsWith("//");
+    if (segments.includes("..")) {
+        throw new Error("File path traversal is not allowed.");
+    }
+    if (isWindowsAbsolute && !path.isAbsolute(filePath)) {
+        throw new Error("Windows absolute paths are not valid on this host.");
+    }
+}
+function isContained(root, candidate) {
+    const relative = path.relative(root, candidate);
+    return (relative.length > 0 &&
+        relative !== ".." &&
+        !relative.startsWith(`..${path.sep}`) &&
+        !path.isAbsolute(relative));
+}
+function projectRelative(root, filePath) {
+    return path.relative(root, filePath).split(path.sep).join("/");
+}
+function decodeUtf8(bytes, filePath) {
+    try {
+        return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(bytes);
+    }
+    catch {
+        throw new Error(`Source file "${filePath}" is not valid UTF-8.`);
+    }
+}
+function splitLines(source) {
+    if (source.length === 0)
+        return [];
+    const lines = source.split(/\r\n|\n|\r/);
+    if (source.endsWith("\n") || source.endsWith("\r"))
+        lines.pop();
+    return lines;
+}
+function compilerSnapshotHashes(sourceText) {
+    const sourceBytes = Buffer.from(sourceText, "utf8");
+    const hashes = [hashBytes(sourceBytes)];
+    if (sourceText.charCodeAt(0) !== 0xfeff) {
+        hashes.push(hashBytes(Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), sourceBytes])));
+    }
+    return hashes;
+}
+export async function readFileSnapshot(project, projectRoot, filePath, options) {
+    assertBoundedRange(options);
+    rejectTraversalPath(filePath);
+    const sourceFile = getSourceFileOrThrow(project, filePath);
+    const canonicalRoot = await realpath(projectRoot);
+    const sourcePath = sourceFile.getFilePath();
+    const canonicalFile = await realpath(sourcePath);
+    if (!isContained(canonicalRoot, canonicalFile)) {
+        throw new Error("Source file resolves outside the configured project root.");
+    }
+    const stat = await lstat(canonicalFile);
+    if (!stat.isFile()) {
+        throw new Error("Source file is not a regular file.");
+    }
+    const bytes = await readFile(canonicalFile);
+    const source = decodeUtf8(bytes, projectRelative(canonicalRoot, canonicalFile));
+    const fileHash = hashBytes(bytes);
+    const compilerHashes = compilerSnapshotHashes(sourceFile.getFullText());
+    const snapshotState = compilerHashes.includes(fileHash) ? "fresh" : "stale";
+    const allLines = splitLines(source);
+    const selectedLines = allLines.slice(options.offset, options.offset + options.limit);
+    return {
+        file: projectRelative(canonicalRoot, canonicalFile),
+        range: {
+            offset: options.offset,
+            limit: options.limit,
+            total_lines: allLines.length,
+        },
+        lines: selectedLines.map((text, index) => ({
+            line: options.offset + index + 1,
+            text,
+        })),
+        file_hash: fileHash,
+        snapshot_state: snapshotState,
+    };
+}