swagger-digest 0.1.1 → 0.2.0

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2026 swagger-digest contributors
+Copyright (c) 2026 VladLegkowski
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+SOFTWARE.

package/README.md

@@ -25,8 +25,99 @@ For a direct spec URL:
 npx swagger-digest --url https://example.com/openapi.yaml --out-dir ./docs
 ```
 
+If the Swagger UI initializer lists several spec definitions (`urls: [{ url, name }, …]`), the tool probes them in the order they appear and uses the first one that parses, printing a notice to stderr naming the pick and the alternatives. To choose a specific spec yourself, pass it explicitly:
+
+```bash
+npx swagger-digest --url https://api.example.com/swagger-ui \
+  --spec https://api.example.com/v2/tenant-a.json --out-dir ./docs
+```
+
+`--spec` overrides all discovery: the given URL is fetched directly (with the same header scoping and size limits) and the Swagger UI page is never consulted when it parses successfully.
+
+### Writing to stdout
+
+```bash
+npx swagger-digest --url https://petstore.swagger.io/ --stdout > petstore.md
+npx swagger-digest --url https://api.example.com/openapi.json --stdout | pbcopy
+```
+
+`--stdout` prints the digest to standard output instead of saving a file — handy for piping into other tools or copying to the clipboard. Progress and warnings go to stderr, so the pipe receives only the Markdown. `--stdout` cannot be combined with `--out-dir` or `--filename`; the CLI exits with a clear error if you try.
+
 Run `npx swagger-digest --help` for all CLI options. The package requires Node.js 20 or newer.
 
+## MCP server
+
+The package ships an [MCP](https://modelcontextprotocol.io) server exposing the same pipeline as a tool for AI agents, so they can fetch and digest API docs on their own instead of being handed a Markdown file.
+
+```bash
+npx swagger-digest-mcp
+```
+
+It speaks the stdio transport: the MCP client spawns the process and communicates over stdin/stdout. Register it in your client's configuration:
+
+```json
+{
+  "mcp": {
+    "swagger-digest": {
+      "command": ["npx", "-y", "swagger-digest-mcp"]
+    }
+  }
+}
+```
+
+### The `swagger_digest` tool
+
+| Argument        | Type             | Description                                                                |
+| --------------- | ---------------- | -------------------------------------------------------------------------- |
+| `url`           | string, required | Swagger UI page URL or direct OpenAPI spec URL                             |
+| `spec_url`      | string, optional | Explicit spec URL; overrides discovery from the page                       |
+| `headers`       | object, optional | Request headers (e.g. `Authorization`); sent only to the origin of `url`   |
+| `include_tags`  | string[]         | Keep only operations with these tags                                       |
+| `exclude_tags`  | string[]         | Drop operations with these tags                                            |
+| `path_prefix`   | string           | Keep only paths starting with this prefix                                  |
+| `max_tokens`    | number           | Trim the digest to an approximate token budget                             |
+| `write_to_file` | object, optional | `{ directory, filename }` — save the digest instead of returning it inline |
+
+By default the digest is returned directly in the tool result (so it lands in the agent's context), with a metadata footer: retrieval method, operation/model counts, and a token estimate.
+
+### Environment variables
+
+- `SWAGGER_DIGEST_HEADER` — newline-separated `Name: value` pairs, merged with (and overridden by) the tool call's `headers` argument. Keeps secrets out of the conversation.
+- `SWAGGER_DIGEST_ALLOW_PRIVATE=1` — opt out of the private-network guard (see below).
+
+### Security differences from the CLI
+
+- **SSRF guard on by default:** URLs resolving to loopback, private, or link-local (cloud metadata) addresses are rejected, because MCP clients supply URLs on the model's behalf. Opt out with `SWAGGER_DIGEST_ALLOW_PRIVATE=1` — for example to digest an internal corporate API — only if you trust the hosts involved.
+- **No headless-browser fallback:** the optional Playwright fallback is disabled in the MCP server.
+- Diagnostics go to stderr only; stdout carries protocol frames exclusively.
+
+### Filtering and token budget
+
+Large specs produce large digests. The CLI can remove whole operations before rendering and/or trim the remaining digest to an approximate token budget:
+
+```bash
+# Keep only operations tagged "Payments", plus everything under /invoices
+npx swagger-digest --url https://api.example.com/openapi.json \
+  --include-tag Payments --path-prefix /payments/ --out-dir ./docs
+
+# Drop a noisy tag
+npx swagger-digest --url https://api.example.com/openapi.json --exclude-tag Internal
+
+# Fit the digest into a model's context window
+npx swagger-digest --url https://api.example.com/openapi.json --max-tokens 50000
+```
+
+How `--max-tokens` trims (in this fixed order, re-checked against the estimate after each step):
+
+1. Drop declared example payloads (synthetic examples are never emitted).
+2. Shorten long descriptions.
+3. Collapse parameter/response detail per tag — the **tag priority rule** is that the tag appearing **last in the document** is collapsed first (lowest priority), one tag per step.
+4. Keep only the Quick index entries (no endpoint detail, no data types).
+
+A `## Trimmed` note at the end of the digest records exactly which steps were applied. After every step, schemas that became unreachable (referenced only by collapsed/removed operations) are dropped from the Data types section. The token estimator is a heuristic, **not** a real tokenizer — treat `--max-tokens` as a guide, not a guarantee. Unknown tag names passed to `--include-tag`/`--exclude-tag` produce a warning on stderr listing the tags that exist in the spec.
+
+The same options are available programmatically via the third argument of `transformOpenApiToMarkdown(spec, sourceUrl, { includeTags, excludeTags, pathPrefix, maxTokens, onWarning })`.
+
 ## What it extracts
 
 The retrieval strategy deliberately avoids scraping rendered operation accordions or simulating UI clicks:
@@ -45,6 +136,37 @@ npx playwright install chromium
 
 Then run the CLI again. Use `--no-browser` to disable the fallback.
 
+### Authentication headers
+
+Some Swagger UI pages and spec URLs live behind authentication. Pass custom headers with `-H`/`--header` (repeatable) or `--cookie`:
+
+```bash
+npx swagger-digest \
+  --url https://api.example.com/swagger-ui \
+  --header "Authorization: Bearer eyJhbGciOi..." \
+  --out-dir ./docs
+```
+
+Or with a session cookie:
+
+```bash
+npx swagger-digest --url https://api.example.com/swagger-ui --cookie "session=abc123"
+```
+
+You can also supply headers through the `SWAGGER_DIGEST_HEADER` environment variable, with one `Name: value` pair per line. `-H` flags override environment entries on conflicting header names:
+
+```bash
+export SWAGGER_DIGEST_HEADER=$'Authorization: Bearer eyJ…\nX-Tenant: acme'
+npx swagger-digest --url https://api.example.com/swagger-ui
+```
+
+**Scoping and safety:**
+
+- Custom headers are sent **only** to the origin of the URL you supply. If the discovered spec, a redirect, or a Swagger UI script lives on a different origin, that request is made without your headers and a warning naming the other origin is printed to stderr.
+- In the headless-browser fallback, headers are injected per-request (only for your origin) — they never reach CDNs or third-party scripts loaded by the page.
+- Header values are never logged, written to the digest, or included in error messages.
+- ⚠️ **Shell history:** secrets passed with `-H` can end up in your shell history file (`~/.bash_history`, `~/.zsh_history`). Prefer the `SWAGGER_DIGEST_HEADER` environment variable, or prefix the command with a space if your shell is configured to ignore leading-space entries.
+
 ### Troubleshooting access through a VPN
 
 The ordinary HTTP fetches use Node.js networking; they do not automatically reuse a browser's VPN extension, cookies, or signed-in session. If the Swagger page works in Chrome but not in the CLI, first test the same URL from the terminal:
@@ -55,6 +177,14 @@ curl -v -L 'https://your-host.example/path/to/swagger-ui' -o /tmp/swagger-page.h
 
 If `curl` cannot connect or receives `401`/`403`, connect a system-level VPN that also routes terminal applications, or configure the relevant corporate proxy/authentication for the terminal. Installing Playwright alone does not make a browser-only VPN extension available to its separate Chromium instance. If terminal access succeeds, the extractor now also reads same-origin Swagger UI initializer/config scripts and tries spec paths relative to the supplied page's context path.
 
+## Security notes
+
+**What the tool fetches:** the URL you supply (a Swagger UI page or direct OpenAPI JSON/YAML URL), same-origin Swagger UI initializer/config scripts, the spec URL referenced by that page, and a short list of common spec paths on the same origin (`/v3/api-docs`, `/swagger.json`, …). The optional Playwright fallback loads the same page in a headless browser.
+
+**What it does not forward:** cookies or Authorization headers are never sent automatically. Custom headers you pass with `--header`/`--cookie` (see below) are sent **only** to the origin of the URL you supplied. If the page redirects to another origin, or the discovered spec lives on a different origin, those requests are made without your headers and a warning naming the other origin is printed to stderr. Response bodies are capped (10 MB by default, configurable with `--max-bytes`), and redirects are followed at most 5 times.
+
+**Embedding in a server:** if you call `fetchOpenApiSpec` from a server that accepts user-supplied URLs, enable `blockPrivateNetworks: true` in the fetch options. Without it, a caller can point the tool at `http://169.254.169.254/…` or other internal addresses from your infrastructure (SSRF). The check rejects loopback, link-local (including the cloud-metadata address 169.254.169.254), private, and unique-local addresses, and applies after redirects. Note the documented DNS-rebinding limitation in the code — for hostile multi-tenant input, use a dedicated egress proxy.
+
 ## Markdown format
 
 The generated document includes:
@@ -95,12 +225,16 @@ src/
   transformer.ts  OpenAPI/Swagger to compact Markdown and TypeScript-like types
   writer.ts       Output path, filename defaults and file writing
   index.ts        Public exports
+  mcp/
+    index.ts      MCP stdio entry point (`swagger-digest-mcp` bin)
+    server.ts     MCP server and tool registration (transport-agnostic)
+    digest.ts     Tool handler: fetch → transform → tool result
 ```
 
 ## Limitations
 
 - The tool downloads specs exposed by the supplied server. It does not bypass authentication, browser access controls, or network restrictions.
-- External schema references are retained as named type references where possible; the tool does not currently fetch and bundle arbitrary external documents.
+- External `$ref` documents are bundled best-effort: references to other HTTP(S) documents are fetched (one level deep, at most 5 documents) and their schemas inlined under namespaced names such as `meta_SharedMeta`. If an external document is unreachable or unparsable, the reference degrades to a named type reference and the digest still succeeds.
 - Declared request/response examples are preserved. When an example is absent but a schema exists, a representative JSON example is generated from that schema and clearly marked as illustrative rather than spec-authored. The transformer includes the primary success response and applicable common error responses to balance useful payload context against digest size.
 - A generated digest is a compact developer reference, not a replacement for the source OpenAPI contract.
 

package/dist/chunk-2NORV6KM.js

@@ -0,0 +1,143 @@
+import {
+  VERSION,
+  estimateTokens,
+  fetchOpenApiSpec,
+  mergeHeaders,
+  parseHeaderPairs,
+  transformOpenApiToMarkdown,
+  writeDigest
+} from "./chunk-U7OLLL6J.js";
+
+// src/mcp/digest.ts
+var TOKEN_HINT_THRESHOLD = 8e4;
+function textContent(text) {
+  return [{ type: "text", text }];
+}
+function asError(message) {
+  return { content: textContent(message), isError: true };
+}
+function isHttpUrl(value) {
+  try {
+    const url = new URL(value);
+    return url.protocol === "http:" || url.protocol === "https:";
+  } catch {
+    return false;
+  }
+}
+function countDigestOperations(markdown) {
+  return (markdown.match(/^\| \*\*(?:GET|PUT|POST|DELETE|OPTIONS|HEAD|PATCH|TRACE)\*\* \|/gm) ?? []).length;
+}
+function countDigestModels(markdown) {
+  return (markdown.match(/^export (?:interface|type) \w+/gm) ?? []).length;
+}
+function buildRequestHeaders(args) {
+  const fromEnv = process.env.SWAGGER_DIGEST_HEADER ? parseHeaderPairs(process.env.SWAGGER_DIGEST_HEADER) : {};
+  return mergeHeaders(fromEnv, args.headers);
+}
+async function runSwaggerDigest(args, options = {}) {
+  if (!isHttpUrl(args.url)) {
+    return asError(`Invalid url "${args.url}": use an http:// or https:// URL.`);
+  }
+  if (args.spec_url !== void 0 && !isHttpUrl(args.spec_url)) {
+    return asError(`Invalid spec_url "${args.spec_url}": use an http:// or https:// URL.`);
+  }
+  if (args.max_tokens !== void 0 && (!Number.isInteger(args.max_tokens) || args.max_tokens <= 0)) {
+    return asError(`Invalid max_tokens ${args.max_tokens}: use a positive integer.`);
+  }
+  const blockPrivateNetworks = options.blockPrivateNetworks ?? process.env.SWAGGER_DIGEST_ALLOW_PRIVATE !== "1";
+  const warnings = [];
+  const warn = (message) => {
+    warnings.push(message);
+  };
+  let fetchResult;
+  try {
+    const fetchOptions = {
+      useBrowserFallback: false,
+      maxBytes: 1e7,
+      headers: buildRequestHeaders(args),
+      specUrl: args.spec_url,
+      blockPrivateNetworks,
+      warn
+    };
+    fetchResult = await fetchOpenApiSpec(args.url, fetchOptions);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    return asError(`Could not extract an OpenAPI definition from ${args.url}: ${message}`);
+  }
+  const markdown = transformOpenApiToMarkdown(fetchResult.spec, fetchResult.sourceUrl, {
+    includeTags: args.include_tags,
+    excludeTags: args.exclude_tags,
+    pathPrefix: args.path_prefix,
+    maxTokens: args.max_tokens,
+    onWarning: warn
+  });
+  const operationCount = countDigestOperations(markdown);
+  const modelCount = countDigestModels(markdown);
+  const tokens = estimateTokens(markdown).models.claude.tokens;
+  const metadata = [
+    `source: ${fetchResult.sourceUrl} (via ${fetchResult.retrievalMethod})`,
+    `${operationCount} operation${operationCount === 1 ? "" : "s"}`,
+    `${modelCount} data model${modelCount === 1 ? "" : "s"}`,
+    `~${tokens.toLocaleString("en-US")} tokens estimated`
+  ].join(" \xB7 ");
+  const warningBlock = warnings.length > 0 ? `
+
+_Warnings:_
+${warnings.map((w) => `- ${w}`).join("\n")}` : "";
+  const budgetHint = tokens > TOKEN_HINT_THRESHOLD && args.max_tokens === void 0 ? "\n\n_This digest is large. Consider re-running with max_tokens, include_tags, exclude_tags or path_prefix to fit it into context._" : "";
+  if (args.write_to_file) {
+    try {
+      const outputPath = await writeDigest({
+        markdown,
+        directory: args.write_to_file.directory,
+        filename: args.write_to_file.filename
+      });
+      return { content: textContent(`Digest written to ${outputPath}
+
+_${metadata}_${warningBlock}`) };
+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error);
+      return asError(`Could not write digest: ${message}`);
+    }
+  }
+  return { content: textContent(`${markdown}
+
+---
+_Digest metadata: ${metadata}_${warningBlock}${budgetHint}`) };
+}
+
+// src/mcp/server.ts
+import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
+import { z } from "zod";
+function createMcpServer(options = {}) {
+  const server = new McpServer({ name: "swagger-digest", version: VERSION });
+  server.registerTool(
+    "swagger_digest",
+    {
+      title: "Generate API digest",
+      description: "Fetch a Swagger UI page or an OpenAPI/Swagger spec (JSON or YAML, versions 2.0/3.0/3.1) and convert it into a compact Markdown API reference optimized for LLM context: endpoint index, request/response details, authentication schemes, and TypeScript-like data types. Handles Swagger UI discovery automatically (initializer scripts, multi-spec dropdowns, common spec paths). Use this whenever the user asks to understand, summarize, or work against an API documented with Swagger/OpenAPI.",
+      inputSchema: {
+        url: z.string().describe("Swagger UI page URL or direct OpenAPI spec URL (http/https)"),
+        spec_url: z.string().optional().describe("Explicit OpenAPI spec URL; overrides discovery from the Swagger UI page"),
+        headers: z.record(z.string(), z.string()).optional().describe(
+          "Request headers (e.g. Authorization or Cookie). Sent ONLY to the origin of url; cross-origin requests never receive them. Values are not logged."
+        ),
+        include_tags: z.array(z.string()).optional().describe("Keep only operations carrying at least one of these tags"),
+        exclude_tags: z.array(z.string()).optional().describe("Drop operations carrying any of these tags"),
+        path_prefix: z.string().optional().describe("Keep only operations whose path starts with this prefix"),
+        max_tokens: z.number().int().positive().optional().describe(
+          "Trim the digest until an approximate token estimate fits this budget (heuristic, not tokenizer-accurate)"
+        ),
+        write_to_file: z.object({ directory: z.string(), filename: z.string() }).optional().describe("Save the digest to a file instead of returning it inline")
+      }
+    },
+    (args) => runSwaggerDigest(args, options)
+  );
+  return server;
+}
+
+export {
+  buildRequestHeaders,
+  runSwaggerDigest,
+  createMcpServer
+};