@j0hanz/superfetch 1.1.9 → 1.2.1

package/README.md

@@ -1,8 +1,8 @@
-# 🚀 superFetch MCP Server
+# superFetch MCP Server
 
 <img src="docs/logo.png" alt="SuperFetch MCP Logo" width="200">
 
-[![npm version](https://img.shields.io/npm/v/@j0hanz/superfetch.svg)](https://www.npmjs.com/package/@j0hanz/superfetch) [![Node.js](https://img.shields.io/badge/Node.js-≥20.0.0-339933?logo=nodedotjs&logoColor=white)](https://nodejs.org/) [![TypeScript](https://img.shields.io/badge/TypeScript-5.9-3178C6?logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
+[![npm version](https://img.shields.io/npm/v/@j0hanz/superfetch.svg)](https://www.npmjs.com/package/@j0hanz/superfetch) [![Node.js](https://img.shields.io/badge/Node.js-%3E=20.12-339933?logo=nodedotjs&logoColor=white)](https://nodejs.org/) [![TypeScript](https://img.shields.io/badge/TypeScript-5.9-3178C6?logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
 
 ## One-Click Install
 
@@ -10,71 +10,64 @@
 
 [![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/install-mcp?name=superfetch&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovc3VwZXJmZXRjaEBsYXRlc3QiLCItLXN0ZGlvIl19)
 
-A [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) server that fetches, extracts, and transforms web content into AI-optimized formats using Mozilla Readability.
+A [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) server that fetches web pages, extracts readable content with Mozilla Readability, and returns AI-friendly JSONL or Markdown.
 
-[Quick Start](#quick-start) · [How to Choose a Tool](#-how-to-choose-a-tool) · [Tools](#available-tools) · [Configuration](#configuration) · [Contributing](#contributing)
+[Quick Start](#quick-start) | [How to Choose a Tool](#how-to-choose-a-tool) | [Tools](#available-tools) | [Resources](#resources) | [Configuration](#configuration) | [Security](#security) | [Development](#development)
 
-> 📦 **Published to [MCP Registry](https://registry.modelcontextprotocol.io/)** — Search for `io.github.j0hanz/superfetch`
+> **Published to [MCP Registry](https://registry.modelcontextprotocol.io/)** - Search for `io.github.j0hanz/superfetch`
 
 ---
 
 > [!CAUTION]
 > This server can access URLs on behalf of AI assistants. Built-in SSRF protection blocks private IP ranges and cloud metadata endpoints, but exercise caution when deploying in sensitive environments.
 
-## ✨ Features
+## Features
 
-| Feature                   | Description                                                   |
-| ------------------------- | ------------------------------------------------------------- |
-| 🧠 **Smart Extraction**   | Mozilla Readability removes ads, navigation, and boilerplate  |
-| 📄 **Multiple Formats**   | JSONL semantic blocks or clean Markdown with YAML frontmatter |
-| 🔗 **Link Discovery**     | Extract and classify internal/external links                  |
-| ⚡ **Built-in Caching**   | Configurable TTL and max entries                              |
-| 🛡️ **Security First**     | SSRF protection, URL validation, header sanitization          |
-| 🔄 **Resilient Fetching** | Exponential backoff with jitter                               |
-| 📊 **Monitoring**         | Stats resource for cache performance and health               |
+| Feature            | Description                                                               |
+| ------------------ | ------------------------------------------------------------------------- |
+| Smart extraction   | Mozilla Readability removes ads, navigation, and boilerplate when enabled |
+| JSONL + Markdown   | JSONL semantic blocks or clean Markdown with frontmatter                  |
+| Structured blocks  | Headings, paragraphs, lists, code, tables, images, blockquotes            |
+| Built-in caching   | In-memory cache with TTL, max keys, and resource subscriptions            |
+| Resilient fetching | Redirect handling plus retry with exponential backoff + jitter            |
+| Security first     | URL validation, SSRF/DNS/IP blocklists, header sanitization               |
+| HTTP mode          | API key auth, session management, rate limiting, CORS                     |
 
 ---
 
-## 🎯 How to Choose a Tool
+## How to Choose a Tool
 
-Use this guide to select the right tool for your web content extraction needs:
+Use this guide to select the right tool for your web content extraction needs.
 
 ### Decision Tree
 
 ```text
 Need web content for AI?
-├─ Single URL?
-│   ├─ Need structured semantic blocks → fetch-url (JSONL)
-│   ├─ Need readable markdown → fetch-markdown
-│   └─ Need links only → fetch-links
-└─ Multiple URLs?
-    └─ Use fetch-urls (batch processing)
+- Want structured JSONL blocks -> fetch-url (format: jsonl)
+- Want clean Markdown -> fetch-markdown
+- Want Markdown but also need contentBlocks count -> fetch-url (format: markdown)
 ```
 
 ### Quick Reference Table
 
-| Tool             | Best For                         | Output Format           | Use When                                    |
-| ---------------- | -------------------------------- | ----------------------- | ------------------------------------------- |
-| `fetch-url`      | Single page → structured content | JSONL semantic blocks   | AI analysis, RAG pipelines, content parsing |
-| `fetch-markdown` | Single page → readable format    | Clean Markdown + TOC    | Documentation, human-readable output        |
-| `fetch-links`    | Link discovery & classification  | URL array with types    | Sitemap building, finding related pages     |
-| `fetch-urls`     | Batch processing multiple pages  | Multiple JSONL/Markdown | Comparing pages, bulk extraction            |
+| Tool             | Best For                           | Output Format                    | Use When                                  |
+| ---------------- | ---------------------------------- | -------------------------------- | ----------------------------------------- |
+| `fetch-url`      | Single page with structured blocks | JSONL (or Markdown via `format`) | RAG pipelines, content parsing, analytics |
+| `fetch-markdown` | Single page in readable format     | Markdown + frontmatter           | Documentation, summaries, human review    |
 
 ### Common Use Cases
 
 | Task                     | Recommended Tool                         | Why                                                  |
 | ------------------------ | ---------------------------------------- | ---------------------------------------------------- |
 | Parse a blog post for AI | `fetch-url`                              | Returns semantic blocks (headings, paragraphs, code) |
-| Generate documentation   | `fetch-markdown`                         | Clean markdown with optional TOC                     |
-| Build a sitemap          | `fetch-links`                            | Extracts and classifies all links                    |
-| Compare multiple docs    | `fetch-urls`                             | Parallel fetching with concurrency control           |
+| Generate documentation   | `fetch-markdown`                         | Clean markdown with frontmatter                      |
 | Extract article for RAG  | `fetch-url` + `extractMainContent: true` | Removes ads/nav, keeps main content                  |
 
 ---
 
 ## Quick Start
 
-Add superFetch to your MCP client configuration — no installation required!
+Add superFetch to your MCP client configuration - no installation required.
 
 ### Claude Desktop
 
@@ -108,25 +101,8 @@ Add to `.vscode/mcp.json` in your workspace:
 
 ### With Custom Configuration
 
-Configure SuperFetch behavior by adding environment variables to the `env` property:
-
-```json
-{
-  "servers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"],
-      "env": {
-        "CACHE_TTL": "7200",
-        "LOG_LEVEL": "debug",
-        "FETCH_TIMEOUT": "60000"
-      }
-    }
-  }
-}
-```
-
-See [Configuration](#configuration) section below for all available options and presets.
+Add environment variables in your MCP client config under `env`.
+See [Configuration](#configuration) or `CONFIGURATION.md` for all available options and presets.
 
 ### Cursor
 
@@ -146,7 +122,7 @@ See [Configuration](#configuration) section below for all available options and
 }
 ```
 
-> **Tip:** On Windows, if you encounter issues, try: `cmd /c "npx -y @j0hanz/superfetch@latest --stdio"`
+> **Tip (Windows):** If you encounter issues, try: `cmd /c "npx -y @j0hanz/superfetch@latest --stdio"`
 
 <details>
 <summary><strong>Codex IDE</strong></summary>
@@ -161,16 +137,9 @@ command = "npx"
 args = ["-y", "@j0hanz/superfetch@latest", "--stdio"]
 ```
 
-**With Environment Variables:**
+**With Environment Variables:** See `CONFIGURATION.md` for examples.
 
-```toml
-[mcp_servers.superfetch]
-command = "npx"
-args = ["-y", "@j0hanz/superfetch@latest", "--stdio"]
-env = { CACHE_TTL = "7200", LOG_LEVEL = "debug", FETCH_TIMEOUT = "60000" }
-```
-
-> **Access config file:** Click the gear icon → "Codex Settings &gt; Open config.toml"
+> **Access config file:** Click the gear icon -> "Codex Settings > Open config.toml"
 >
 > **Documentation:** [Codex MCP Guide](https://codex.com/docs/mcp)
 
@@ -261,7 +230,7 @@ npm install -g @j0hanz/superfetch
 # Run in stdio mode
 superfetch --stdio
 
-# Run HTTP server
+# Run HTTP server (requires API_KEY)
 superfetch
 ```
 
@@ -276,24 +245,6 @@ npm run build
 
 ### Running the Server
 
-<details>
-<summary><strong>HTTP Mode</strong> (default)</summary>
-
-```bash
-# Development with hot reload
-npm run dev
-
-# Production
-npm start
-```
-
-Server runs at `http://127.0.0.1:3000`:
-
-- Health check: `GET /health`
-- MCP endpoint: `POST /mcp`
-
-</details>
-
 <details>
 <summary><strong>stdio Mode</strong> (direct MCP integration)</summary>
 
@@ -303,433 +254,199 @@ node dist/index.js --stdio
 
 </details>
 
----
-
-## Available Tools
-
-> **Note:** If extracted content exceeds `MAX_INLINE_CONTENT_CHARS`, the tool response includes a `resourceUri` and a `resource_link` content block instead of embedding the full text.
-
-### `fetch-url`
-
-Fetches a webpage and converts it to AI-readable JSONL format with semantic content blocks.
-
-| Parameter            | Type    | Default    | Description                                  |
-| -------------------- | ------- | ---------- | -------------------------------------------- |
-| `url`                | string  | _required_ | URL to fetch                                 |
-| `extractMainContent` | boolean | `true`     | Use Readability to extract main content      |
-| `includeMetadata`    | boolean | `true`     | Include page metadata (title, description)   |
-| `maxContentLength`   | number  | –          | Maximum content length in characters         |
-| `customHeaders`      | object  | –          | Custom HTTP headers for the request          |
-| `timeout`            | number  | `30000`    | Request timeout in milliseconds (1000-60000) |
-| `retries`            | number  | `3`        | Number of retry attempts (1-10)              |
+<details>
+<summary><strong>HTTP Mode</strong> (default)</summary>
 
-**Example Response:**
+HTTP mode requires `API_KEY` and only binds to loopback addresses unless `ALLOW_REMOTE=true`.
 
-```json
-{
-  "url": "https://example.com/article",
-  "title": "Example Article",
-  "fetchedAt": "2025-12-11T10:30:00.000Z",
-  "contentBlocks": [
-    {
-      "type": "metadata",
-      "title": "Example Article",
-      "description": "A sample article"
-    },
-    { "type": "heading", "level": 1, "text": "Introduction" },
-    {
-      "type": "paragraph",
-      "text": "This is the main content of the article..."
-    },
-    {
-      "type": "code",
-      "language": "javascript",
-      "content": "console.log('Hello');"
-    }
-  ],
-  "cached": false
-}
+```bash
+API_KEY=supersecret npx -y @j0hanz/superfetch@latest
+# Server runs at http://127.0.0.1:3000
 ```
 
-### `fetch-links`
-
-Extracts hyperlinks from a webpage with classification. Supports filtering, image links, and link limits.
-
-| Parameter         | Type    | Default    | Description                                  |
-| ----------------- | ------- | ---------- | -------------------------------------------- |
-| `url`             | string  | _required_ | URL to extract links from                    |
-| `includeExternal` | boolean | `true`     | Include external links                       |
-| `includeInternal` | boolean | `true`     | Include internal links                       |
-| `includeImages`   | boolean | `false`    | Include image links (img src attributes)     |
-| `maxLinks`        | number  | –          | Maximum number of links to return (1-1000)   |
-| `filterPattern`   | string  | –          | Regex pattern to filter links (matches href) |
-| `customHeaders`   | object  | –          | Custom HTTP headers for the request          |
-| `timeout`         | number  | `30000`    | Request timeout in milliseconds (1000-60000) |
-| `retries`         | number  | `3`        | Number of retry attempts (1-10)              |
+**Windows (PowerShell):**
 
-**Example Response:**
-
-```json
-{
-  "url": "https://example.com/",
-  "linkCount": 15,
-  "links": [
-    {
-      "href": "https://example.com/about",
-      "text": "About Us",
-      "type": "internal"
-    },
-    {
-      "href": "https://github.com/example",
-      "text": "GitHub",
-      "type": "external"
-    },
-    { "href": "https://example.com/logo.png", "text": "", "type": "image" }
-  ],
-  "cached": false,
-  "truncated": false
-}
+```powershell
+$env:API_KEY = "supersecret"
+npx -y @j0hanz/superfetch@latest
 ```
 
-### `fetch-markdown`
+Endpoints (all require `Authorization: Bearer <API_KEY>` or `X-API-Key: <API_KEY>`):
 
-Fetches a webpage and converts it to clean Markdown with optional table of contents.
+- `GET /health`
+- `POST /mcp`
+- `GET /mcp` (SSE stream)
+- `DELETE /mcp`
+- `GET /mcp/downloads/:namespace/:hash`
 
-| Parameter            | Type    | Default    | Description                                  |
-| -------------------- | ------- | ---------- | -------------------------------------------- |
-| `url`                | string  | _required_ | URL to fetch                                 |
-| `extractMainContent` | boolean | `true`     | Extract main content only                    |
-| `includeMetadata`    | boolean | `true`     | Include YAML frontmatter                     |
-| `maxContentLength`   | number  | –          | Maximum content length in characters         |
-| `generateToc`        | boolean | `false`    | Generate table of contents from headings     |
-| `customHeaders`      | object  | –          | Custom HTTP headers for the request          |
-| `timeout`            | number  | `30000`    | Request timeout in milliseconds (1000-60000) |
-| `retries`            | number  | `3`        | Number of retry attempts (1-10)              |
-
-**Example Response:**
-
-````json
-{
-  "url": "https://example.com/docs",
-  "title": "Documentation",
-  "fetchedAt": "2025-12-11T10:30:00.000Z",
-  "markdown": "---\ntitle: Documentation\nsource: \"https://example.com/docs\"\n---\n\n# Getting Started\n\nWelcome to our documentation...\n\n## Installation\n\n```bash\nnpm install example\n```",
-  "toc": [
-    { "level": 1, "text": "Getting Started", "slug": "getting-started" },
-    { "level": 2, "text": "Installation", "slug": "installation" }
-  ],
-  "cached": false,
-  "truncated": false
-}
-````
-
-### `fetch-urls` (Batch)
-
-Fetches multiple URLs in parallel with concurrency control. Ideal for comparing content or processing multiple pages efficiently.
-
-| Parameter            | Type     | Default    | Description                                  |
-| -------------------- | -------- | ---------- | -------------------------------------------- |
-| `urls`               | string[] | _required_ | Array of URLs to fetch (1-10 URLs)           |
-| `extractMainContent` | boolean  | `true`     | Use Readability to extract main content      |
-| `includeMetadata`    | boolean  | `true`     | Include page metadata                        |
-| `maxContentLength`   | number   | –          | Maximum content length per URL in characters |
-| `format`             | string   | `'jsonl'`  | Output format: `'jsonl'` or `'markdown'`     |
-| `concurrency`        | number   | `3`        | Maximum concurrent requests (1-5)            |
-| `continueOnError`    | boolean  | `true`     | Continue processing if some URLs fail        |
-| `customHeaders`      | object   | –          | Custom HTTP headers for all requests         |
-| `timeout`            | number   | `30000`    | Request timeout in milliseconds (1000-60000) |
-| `retries`            | number   | `3`        | Number of retry attempts (1-10)              |
-
-**Example Output:**
-
-```json
-{
-  "results": [
-    {
-      "url": "https://example.com",
-      "success": true,
-      "title": "Example",
-      "content": "...",
-      "cached": false
-    },
-    {
-      "url": "https://example.org",
-      "success": true,
-      "title": "Example Org",
-      "content": "...",
-      "cached": false
-    }
-  ],
-  "summary": {
-    "total": 2,
-    "successful": 2,
-    "failed": 0,
-    "cached": 0,
-    "totalContentBlocks": 15
-  },
-  "fetchedAt": "2024-12-11T10:30:00.000Z"
-}
-```
+Sessions are managed via the `mcp-session-id` header (see [HTTP Mode Details](#http-mode-details)).
 
-### Resources
-
-| URI                   | Description                                         |
-| --------------------- | --------------------------------------------------- |
-| `superfetch://stats`  | Server statistics and cache metrics                 |
-| `superfetch://health` | Real-time server health and dependency status       |
-| Dynamic resources     | Cached content available via resource subscriptions |
+</details>
 
 ---
 
-## Configuration
-
-### Configuration Presets
-
-<details open>
-<summary><strong>Default (Recommended)</strong> — No configuration needed</summary>
-
-```json
-{
-  "servers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"]
-    }
-  }
-}
-```
-
-</details>
+## Available Tools
 
-<details>
-<summary><strong>Debug Mode</strong> — Verbose logging and no cache</summary>
+### Tool Response Notes
 
-**VS Code** (`.vscode/mcp.json`):
+Both tools return:
 
-```json
-{
-  "servers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"],
-      "env": {
-        "LOG_LEVEL": "debug",
-        "CACHE_ENABLED": "false"
-      }
-    }
-  }
-}
-```
+- `structuredContent` for machine-readable fields (includes `contentSize`, `cached`, and optional `resourceUri`/`resourceMimeType`/`truncated`)
+- `content` blocks that include:
+  - a `text` block containing JSON of `structuredContent`
+  - in stdio mode, a `resource` block with a `file:///...` URI containing the full content
+  - in HTTP mode, a `resource` block when inline content is available; large payloads include a `resource_link` block when cache is enabled
 
-**Claude Desktop** (`claude_desktop_config.json`):
+If content exceeds `MAX_INLINE_CONTENT_CHARS` and cache is disabled, the server truncates output and appends `...[truncated]`.
 
-```json
-{
-  "mcpServers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"],
-      "env": {
-        "LOG_LEVEL": "debug",
-        "CACHE_ENABLED": "false"
-      }
-    }
-  }
-}
-```
+---
 
-**Cursor** (MCP settings):
+### `fetch-url`
 
-```json
-{
-  "mcpServers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"],
-      "env": {
-        "LOG_LEVEL": "debug",
-        "CACHE_ENABLED": "false"
-      }
-    }
-  }
-}
-```
+Fetches a webpage and converts it to AI-readable JSONL format with semantic content blocks. You can also request Markdown with `format: "markdown"`.
 
-</details>
+| Parameter            | Type                  | Default   | Description                                   |
+| -------------------- | --------------------- | --------- | --------------------------------------------- |
+| `url`                | string                | required  | URL to fetch                                  |
+| `format`             | "jsonl" \| "markdown" | `"jsonl"` | Output format                                 |
+| `extractMainContent` | boolean               | `true`    | Use Readability to extract main content       |
+| `includeMetadata`    | boolean               | `true`    | Include page metadata                         |
+| `maxContentLength`   | number                | -         | Maximum content length in characters          |
+| `customHeaders`      | object                | -         | Custom HTTP headers (sanitized)               |
+| `timeout`            | number                | `30000`   | Request timeout in milliseconds (1000-120000) |
+| `retries`            | number                | `3`       | Number of retry attempts (1-10)               |
 
-<details>
-<summary><strong>Performance Mode</strong> — Aggressive caching for speed</summary>
+**Example `structuredContent`:**
 
 ```json
 {
-  "servers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"],
-      "env": {
-        "CACHE_TTL": "7200",
-        "CACHE_MAX_KEYS": "500",
-        "LOG_LEVEL": "warn"
-      }
-    }
-  }
+  "url": "https://example.com/article",
+  "title": "Example Article",
+  "contentBlocks": 42,
+  "fetchedAt": "2025-12-11T10:30:00.000Z",
+  "format": "jsonl",
+  "contentSize": 12345,
+  "cached": false,
+  "content": "{\"type\":\"metadata\",\"title\":\"Example Article\",\"url\":\"https://example.com/article\"}\n{\"type\":\"heading\",\"level\":1,\"text\":\"Introduction\"}"
 }
 ```
 
-</details>
+---
 
-<details>
-<summary><strong>Custom User Agent</strong> — For sites that block bots</summary>
+### `fetch-markdown`
 
-```json
-{
-  "servers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"],
-      "env": {
-        "USER_AGENT": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
-      }
-    }
-  }
-}
-```
+Fetches a webpage and converts it to clean Markdown with optional frontmatter.
 
-</details>
+| Parameter            | Type    | Default  | Description                                   |
+| -------------------- | ------- | -------- | --------------------------------------------- |
+| `url`                | string  | required | URL to fetch                                  |
+| `extractMainContent` | boolean | `true`   | Extract main content only                     |
+| `includeMetadata`    | boolean | `true`   | Include YAML frontmatter                      |
+| `maxContentLength`   | number  | -        | Maximum content length in characters          |
+| `customHeaders`      | object  | -        | Custom HTTP headers (sanitized)               |
+| `timeout`            | number  | `30000`  | Request timeout in milliseconds (1000-120000) |
+| `retries`            | number  | `3`      | Number of retry attempts (1-10)               |
 
-<details>
-<summary><strong>Slow Networks / CI/CD</strong> — Extended timeouts</summary>
+**Example `structuredContent`:**
 
 ```json
 {
-  "servers": {
-    "superFetch": {
-      "command": "npx",
-      "args": ["-y", "@j0hanz/superfetch@latest", "--stdio"],
-      "env": {
-        "FETCH_TIMEOUT": "60000",
-        "CACHE_ENABLED": "false",
-        "LOG_LEVEL": "warn"
-      }
-    }
+  "url": "https://example.com/docs",
+  "title": "Documentation",
+  "fetchedAt": "2025-12-11T10:30:00.000Z",
+  "markdown": "---\ntitle: Documentation\nsource: \"https://example.com/docs\"\n---\n\n# Getting Started\n\nWelcome...",
+  "contentSize": 9876,
+  "cached": false,
+  "truncated": false,
+  "file": {
+    "downloadUrl": "/mcp/downloads/markdown/abc123def456",
+    "fileName": "documentation.md",
+    "expiresAt": "2025-12-11T11:30:00.000Z"
   }
 }
 ```
 
-</details>
-
-### Available Environment Variables
-
-Configure SuperFetch behavior by adding environment variables to your MCP client configuration's `env` property.
-
-#### 🌐 Fetcher Settings
+`file` is included only in HTTP mode when content is cached and too large to inline.
 
-| Variable        | Default              | Valid Values         | Description                                                     |
-| --------------- | -------------------- | -------------------- | --------------------------------------------------------------- |
-| `FETCH_TIMEOUT` | `30000`              | `5000`-`120000`      | Request timeout in milliseconds (5s-2min)                       |
-| `USER_AGENT`    | `superFetch-MCP/1.0` | Any valid user agent | Custom user agent for requests (useful for sites blocking bots) |
+---
 
-#### 💾 Cache Settings
+### Large Content Handling
 
-| Variable         | Default | Valid Values     | Description                            |
-| ---------------- | ------- | ---------------- | -------------------------------------- |
-| `CACHE_ENABLED`  | `true`  | `true` / `false` | Enable response caching                |
-| `CACHE_TTL`      | `3600`  | `60`-`86400`     | Cache lifetime in seconds (1min-24hrs) |
-| `CACHE_MAX_KEYS` | `100`   | `10`-`1000`      | Maximum number of cached entries       |
+- Inline limit is configurable via `MAX_INLINE_CONTENT_CHARS` (see `CONFIGURATION.md`).
+- If content exceeds the limit and cache is enabled, responses include `resourceUri` and a `resource_link` block.
+- If cache is disabled, content is truncated with `...[truncated]`.
+- Use `maxContentLength` per request to enforce a lower limit.
+- Upstream fetch size is capped at 10 MB of HTML; larger responses fail.
 
-#### 📦 Output Settings
+---
 
-| Variable                   | Default | Valid Values    | Description                                                     |
-| -------------------------- | ------- | --------------- | --------------------------------------------------------------- |
-| `MAX_INLINE_CONTENT_CHARS` | `20000` | `1000`-`200000` | Inline content limit before returning a `resource_link` instead |
+## Resources
 
-#### 📝 Logging Settings
+| URI                                        | Description                                                                |
+| ------------------------------------------ | -------------------------------------------------------------------------- |
+| `superfetch://health`                      | Real-time server health and memory checks                                  |
+| `superfetch://stats`                       | Server stats and cache metrics                                             |
+| `superfetch://cache/list`                  | List cached entries and their resource URIs                                |
+| `superfetch://cache/{namespace}/{urlHash}` | Cached content entry (`namespace`: `url`, `markdown`; `links` is reserved) |
 
-| Variable         | Default | Valid Values                        | Description                |
-| ---------------- | ------- | ----------------------------------- | -------------------------- |
-| `LOG_LEVEL`      | `info`  | `debug` / `info` / `warn` / `error` | Logging verbosity level    |
-| `ENABLE_LOGGING` | `true`  | `true` / `false`                    | Enable/disable all logging |
+Resource subscriptions notify clients when cache entries update.
 
-#### 🔍 Extraction Settings
+---
 
-| Variable               | Default | Valid Values     | Description                                        |
-| ---------------------- | ------- | ---------------- | -------------------------------------------------- |
-| `EXTRACT_MAIN_CONTENT` | `true`  | `true` / `false` | Use Mozilla Readability to extract main content    |
-| `INCLUDE_METADATA`     | `true`  | `true` / `false` | Include page metadata (title, description, author) |
+## Download Endpoint (HTTP Mode)
 
-#### 🛡️ Security Settings
+When running in HTTP mode, cached content can be downloaded directly. Downloads are available only when cache is enabled.
 
-| Variable       | Default | Description                                              |
-| -------------- | ------- | -------------------------------------------------------- |
-| `API_KEY`      | -       | API Key for HTTP authentication (required for HTTP mode) |
-| `ALLOW_REMOTE` | `false` | Allow binding to non-loopback interfaces                 |
+### Endpoint
 
-#### Rate Limiting
+```text
+GET /mcp/downloads/:namespace/:hash
+```
 
-| Variable                | Default | Valid Values      | Description                          |
-| ----------------------- | ------- | ----------------- | ------------------------------------ |
-| `RATE_LIMIT_ENABLED`    | `true`  | `true` / `false`  | Enable/disable HTTP rate limiting    |
-| `RATE_LIMIT_MAX`        | `100`   | `1`-`10000`       | Max requests per window per IP       |
-| `RATE_LIMIT_WINDOW_MS`  | `60000` | `1000`-`3600000`  | Rate limit window in milliseconds    |
-| `RATE_LIMIT_CLEANUP_MS` | `60000` | `10000`-`3600000` | Cleanup interval for limiter entries |
+- `namespace`: `markdown` or `url`
+- Auth required (`Authorization: Bearer <API_KEY>` or `X-API-Key: <API_KEY>`)
 
-### HTTP Mode Configuration
+### Response Headers
 
-<details>
-<summary><strong>HTTP Mode</strong> (Advanced) — For running as a standalone HTTP server</summary>
+| Header                | Value                                                                   |
+| --------------------- | ----------------------------------------------------------------------- |
+| `Content-Type`        | `text/markdown; charset=utf-8` or `application/x-ndjson; charset=utf-8` |
+| `Content-Disposition` | `attachment; filename="<name>"`                                         |
+| `Cache-Control`       | `private, max-age=<CACHE_TTL>`                                          |
 
-SuperFetch can run as an HTTP server for custom integrations. HTTP mode requires additional configuration and an `API_KEY` for authenticated access (send `Authorization: Bearer <key>` or `X-API-Key: <key>`).
-
-#### Start HTTP Server
+### Example Usage
 
 ```bash
-npx -y @j0hanz/superfetch@latest
-# Server runs at http://127.0.0.1:3000
+curl -H "Authorization: Bearer $API_KEY" \
+  http://localhost:3000/mcp/downloads/markdown/abc123.def456 \
+  -o article.md
 ```
 
-#### HTTP-Specific Environment Variables
+### Error Responses
 
-| Variable                  | Default     | Description                                      |
-| ------------------------- | ----------- | ------------------------------------------------ |
-| `PORT`                    | `3000`      | HTTP server port                                 |
-| `HOST`                    | `127.0.0.1` | HTTP server host (`0.0.0.0` for Docker/K8s)      |
-| `ALLOWED_ORIGINS`         | `[]`        | Comma-separated CORS origins                     |
-| `CORS_ALLOW_ALL`          | `false`     | Allow all CORS origins (dev only, security risk) |
-| `SESSION_TTL_MS`          | `1800000`   | Session time-to-live in milliseconds (30 mins)   |
-| `SESSION_INIT_TIMEOUT_MS` | `10000`     | Time allowed for session initialization (ms)     |
-| `MAX_SESSIONS`            | `200`       | Maximum number of active sessions                |
+| Status | Code                  | Description                      |
+| ------ | --------------------- | -------------------------------- |
+| 400    | `BAD_REQUEST`         | Invalid namespace or hash format |
+| 404    | `NOT_FOUND`           | Content not found or expired     |
+| 503    | `SERVICE_UNAVAILABLE` | Download service disabled        |
 
-#### VS Code HTTP Mode Setup
+---
 
-```json
-{
-  "servers": {
-    "superFetch": {
-      "type": "http",
-      "url": "http://127.0.0.1:3000/mcp"
-    }
-  }
-}
-```
+## Configuration
 
-#### Docker/Kubernetes Example
+Configuration details live in `CONFIGURATION.md`, including all environment variables, defaults, ranges, presets, and dev-only flags.
 
-```bash
-PORT=8080 HOST=0.0.0.0 ALLOWED_ORIGINS=https://myapp.com npx @j0hanz/superfetch@latest
-```
+---
 
-</details>
+## HTTP Mode Details
+
+HTTP mode uses the MCP Streamable HTTP transport. The workflow is:
 
-### Configuration Cookbook
+1. `POST /mcp` with an `initialize` request and **no** `mcp-session-id` header.
+2. The server returns `mcp-session-id` in the response headers.
+3. Use that header for subsequent `POST /mcp`, `GET /mcp`, and `DELETE /mcp` requests.
 
-| Use Case                     | Configuration                                                  |
-| ---------------------------- | -------------------------------------------------------------- |
-| 🐛 **Debugging issues**      | `LOG_LEVEL=debug`, `CACHE_ENABLED=false`                       |
-| 🚀 **Maximum performance**   | `CACHE_TTL=7200`, `CACHE_MAX_KEYS=500`, `LOG_LEVEL=error`      |
-| 🌐 **Slow target sites**     | `FETCH_TIMEOUT=60000`                                          |
-| 🤖 **Bypass bot detection**  | `USER_AGENT="Mozilla/5.0 (compatible; MyBot/1.0)"`             |
-| 🔄 **CI/CD (always fresh)**  | `CACHE_ENABLED=false`, `FETCH_TIMEOUT=60000`, `LOG_LEVEL=warn` |
-| 📊 **Production monitoring** | `LOG_LEVEL=warn` or `error`                                    |
+If `MAX_SESSIONS` is reached, the server evicts the oldest session when possible, otherwise returns a 503.
 
 ---
 
@@ -737,15 +454,16 @@ PORT=8080 HOST=0.0.0.0 ALLOWED_ORIGINS=https://myapp.com npx @j0hanz/superfetch@
 
 JSONL output includes semantic content blocks:
 
-| Type        | Description                                     |
-| ----------- | ----------------------------------------------- |
-| `metadata`  | Page title, description, author, URL, timestamp |
-| `heading`   | Headings (h1-h6) with level indicator           |
-| `paragraph` | Text paragraphs                                 |
-| `list`      | Ordered/unordered lists                         |
-| `code`      | Code blocks with language                       |
-| `table`     | Tables with headers and rows                    |
-| `image`     | Images with src and alt text                    |
+| Type         | Description                              |
+| ------------ | ---------------------------------------- |
+| `metadata`   | Minimal page metadata (type, title, url) |
+| `heading`    | Headings (h1-h6) with level indicator    |
+| `paragraph`  | Text paragraphs                          |
+| `list`       | Ordered/unordered lists                  |
+| `code`       | Code blocks with optional language       |
+| `table`      | Tables with headers and rows             |
+| `image`      | Images with src and alt text             |
+| `blockquote` | Block quote text                         |
 
 ---
 
@@ -753,12 +471,19 @@ JSONL output includes semantic content blocks:
 
 ### SSRF Protection
 
-Blocked destinations:
+Blocked destinations include:
 
 - Localhost and loopback addresses
 - Private IP ranges (`10.x.x.x`, `172.16-31.x.x`, `192.168.x.x`)
 - Cloud metadata endpoints (AWS, GCP, Azure)
 - IPv6 link-local and unique local addresses
+- Internal suffixes such as `.local` and `.internal`
+
+### URL Validation
+
+- Only `http` and `https` URLs
+- No embedded credentials in URLs
+- Max URL length: 2048 characters
 
 ### Header Sanitization
 
@@ -766,21 +491,7 @@ Blocked headers: `host`, `authorization`, `cookie`, `x-forwarded-for`, `x-real-i
 
 ### Rate Limiting
 
-Default: **100 requests/minute** per IP. Configure with `RATE_LIMIT_MAX` and
-`RATE_LIMIT_WINDOW_MS`.
-
-### HTTP Mode Endpoints
-
-When running without `--stdio`, the following endpoints are available:
-
-| Endpoint  | Method | Description                             |
-| --------- | ------ | --------------------------------------- |
-| `/health` | GET    | Health check with uptime and version    |
-| `/mcp`    | POST   | MCP request handling (requires session) |
-| `/mcp`    | GET    | SSE stream for notifications            |
-| `/mcp`    | DELETE | Close session                           |
-
-Sessions are managed via `mcp-session-id` header with 30-minute TTL.
+Rate limiting thresholds are configurable via `RATE_LIMIT_MAX` and `RATE_LIMIT_WINDOW_MS` (see `CONFIGURATION.md`).
 
 ---
 
@@ -807,14 +518,13 @@ Sessions are managed via `mcp-session-id` header with 30-minute TTL.
 
 | Category           | Technology                        |
 | ------------------ | --------------------------------- |
-| Runtime            | Node.js ≥20.0.0                   |
+| Runtime            | Node.js >=20.12                   |
 | Language           | TypeScript 5.9                    |
 | MCP SDK            | @modelcontextprotocol/sdk ^1.25.1 |
 | Content Extraction | @mozilla/readability ^0.6.0       |
 | HTML Parsing       | Cheerio ^1.1.2, LinkeDOM ^0.18.12 |
 | Markdown           | Turndown ^7.2.2                   |
-| HTTP               | Express ^5.2.1, Axios ^1.7.9      |
-| Caching            | node-cache ^5.1.2                 |
+| HTTP               | Express ^5.2.1, undici ^6.22.0    |
 | Validation         | Zod ^3.24.1                       |
 
 ---