@j0hanz/fetch-url-mcp 1.6.0 → 1.7.0

package/README.md

@@ -1,610 +1,714 @@
 # Fetch URL MCP Server
 
-[![npm version](https://img.shields.io/npm/v/%40j0hanz%2Ffetch-url-mcp)](https://www.npmjs.com/package/@j0hanz/fetch-url-mcp) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Node.js](https://img.shields.io/badge/node-%3E%3D24-3c873a)](https://nodejs.org) [![TypeScript](https://img.shields.io/badge/TypeScript-5.9-3178c6?logo=typescript&logoColor=white)](https://www.typescriptlang.org) [![MCP SDK](https://img.shields.io/badge/MCP%20SDK-1.26-7c3aed)](https://modelcontextprotocol.io)
+[![npm version](https://img.shields.io/npm/v/%40j0hanz%2Ffetch-url-mcp?style=flat-square&logo=npm)](https://www.npmjs.com/package/%40j0hanz%2Ffetch-url-mcp) [![License](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](#contributing-and-license)
 
-[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0078d7?logo=visual-studio-code&logoColor=white)](https://insiders.vscode.dev/redirect?url=vscode%3Amcp%2Finstall%3F%7B%22name%22%3A%22fetch-url-mcp%22%2C%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%2C%22--stdio%22%5D%7D) [![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?logo=visual-studio-code&logoColor=white)](https://insiders.vscode.dev/redirect?url=vscode-insiders%3Amcp%2Finstall%3F%7B%22name%22%3A%22fetch-url-mcp%22%2C%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%2C%22--stdio%22%5D%7D) [![Install in Cursor](https://img.shields.io/badge/Cursor-Install-f97316?logo=cursor&logoColor=white)](https://cursor.com/install-mcp?name=fetch-url-mcp&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiLCItLXN0ZGlvIl19)
+[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_Server-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url-mcp&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D) [![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install_Server-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url-mcp&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D&quality=insiders) [![Install in Visual Studio](https://img.shields.io/badge/Visual_Studio-Install_Server-C16FDE?logo=visualstudio&logoColor=white)](https://vs-open.link/mcp-install?%7B%22fetch-url-mcp%22%3A%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D%7D)
 
-Fetch public web pages and convert them into clean, AI-readable Markdown.
+[![Add to LM Studio](https://files.lmstudio.ai/deeplink/mcp-install-light.svg)](https://lmstudio.ai/install-mcp?name=fetch-url-mcp&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D) [![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=fetch-url-mcp&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D) [![Install in Goose](https://block.github.io/goose/img/extension-install-dark.svg)](https://block.github.io/goose/extension?cmd=npx&arg=-y&arg=%40j0hanz%2Ffetch-url-mcp%40latest&id=%40j0hanz%2Ffetch-url-mcp&name=fetch-url-mcp&description=fetch-url-mcp%20MCP%20server)
 
-## Overview
+A web content fetcher MCP server that converts HTML to clean, AI and human readable markdown.
 
-Fetch URL is a [Model Context Protocol](https://modelcontextprotocol.io) (MCP) server that fetches public web pages, extracts meaningful content using Mozilla's Readability algorithm, and converts the result into clean Markdown optimized for LLM context windows. It handles noise removal, caching, SSRF protection, async task execution, and supports both **stdio** and **Streamable HTTP** transports.
+## Overview
 
-> [!NOTE]
-> Content extraction quality varies depending on the HTML structure and complexity of the source page. Fetch URL works best with standard article and documentation layouts. Pages relying on client-side JavaScript rendering may yield incomplete results.
+The Fetch URL MCP Server provides a standardized interface for fetching public web content and transforming it into Markdown enriched with structured metadata. It validates URLs, applies noise removal heuristics, and caches results for reuse. The server supports both inline and task-based execution modes, making it suitable for a wide range of client applications and LLM interactions.
 
 ## Key Features
 
-- **HTML to Markdown** — Content extraction via Mozilla Readability + node-html-markdown
-- **Noise removal** — Strips navigation, ads, cookie banners, and other non-content elements
-- **In-memory LRU cache** — Faster repeat fetches with configurable TTL (24 h default)
-- **Raw URL rewriting** — Auto-converts GitHub, GitLab, Bitbucket, and Gist URLs to raw content endpoints
+- `fetch-url` validates public HTTP(S) URLs, fetches the page, and returns cleaned Markdown plus structured metadata.
+- The tool advertises optional task support and emits progress updates while fetching and transforming larger pages.
+- GitHub, GitLab, Bitbucket, and Gist page URLs are rewritten to raw-content endpoints when possible before fetch.
+- `internal://instructions` and `internal://cache/{namespace}/{hash}` expose built-in guidance and cached Markdown as MCP resources.
+- HTTP mode adds host/origin validation, auth, rate limiting, health checks, OAuth protected-resource metadata, and cached-download URLs.
 
-## Tech Stack
+## Requirements
 
-| Component           | Technology                          |
-| ------------------- | ----------------------------------- |
-| Runtime             | Node.js >= 24                       |
-| Language            | TypeScript 5.9                      |
-| MCP SDK             | `@modelcontextprotocol/sdk` ^1.26.0 |
-| Content Extraction  | `@mozilla/readability` ^0.6.0       |
-| DOM Parsing         | `linkedom` ^0.18.12                 |
-| Markdown Conversion | `node-html-markdown` ^2.0.0         |
-| Schema Validation   | `zod` ^4.3.6                        |
-| Package Manager     | npm                                 |
+- Node.js >=24 (from `package.json`)
+- Docker is optional if you want to run the published container image.
 
-## Architecture
+## Quick Start
 
-```text
-URL → Validate → DNS Preflight → HTTP Fetch → Decompress
-  → Truncate HTML → Readability Extract → Noise Removal
-  → Markdown Convert → Cleanup Pipeline → Cache → Response
+Use this standard MCP client configuration:
+
+```json
+{
+  "mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
+}
 ```
 
-1. **URL Validation** — Normalize, block private hosts, transform raw-content URLs (GitHub, GitLab, Bitbucket)
-2. **Fetch** — HTTP request with redirect following, DNS preflight SSRF checks, and size limits (10 MB)
-3. **Transform** — Offloaded to worker threads: parse HTML with `linkedom`, extract with Readability, remove DOM noise, convert to Markdown
-4. **Cleanup** — Multi-pass Markdown normalization (heading promotion, spacing, skip-link removal)
-5. **Cache + Respond** — Store result in LRU cache, apply inline content limits, return structured content
+## Client Configuration
 
-## Repository Structure
+<details>
+<summary><b>Install in VS Code</b></summary>
 
-```text
-fetch-url-mcp/
-├── assets/              # Server icon (logo.svg)
-├── examples/            # Client examples
-├── scripts/             # Build & test orchestration
-├── src/
-│   ├── workers/         # Worker-thread child for HTML transforms
-│   ├── index.ts         # CLI entrypoint, transport wiring, shutdown
-│   ├── server.ts        # McpServer lifecycle and registration
-│   ├── tools.ts         # fetch-url tool definition and pipeline
-│   ├── fetch.ts         # URL normalization, SSRF, HTTP fetch
-│   ├── transform.ts     # HTML-to-Markdown pipeline, worker pool
-│   ├── config.ts        # Env-driven configuration
-│   ├── resources.ts     # MCP resource/template registration
-│   ├── prompts.ts       # MCP prompt registration (get-help)
-│   ├── mcp.ts           # Task execution management
-│   ├── http-native.ts   # Streamable HTTP server, auth, sessions
-│   └── instructions.md  # Server instructions embedded at runtime
-├── tests/               # Unit/integration tests (Node.js test runner)
-├── package.json
-├── tsconfig.json
-└── AGENTS.md
-```
+[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_Server-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url-mcp&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D)
 
-## Requirements
+Add to `.vscode/mcp.json`:
 
-- **Node.js** >= 24
+```json
+{
+  "servers": {
+    "fetch-url-mcp": {
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
+}
+```
 
-## Quickstart
+Or install via CLI:
 
-```bash
-npx -y @j0hanz/fetch-url-mcp@latest --stdio
+```sh
+code --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'
 ```
 
-Add to your MCP client configuration:
+For more info, see [VS Code MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).
+
+</details>
+
+<details>
+<summary><b>Install in VS Code Insiders</b></summary>
+
+[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install_Server-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url-mcp&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D&quality=insiders)
+
+Add to `.vscode/mcp.json`:
 
 ```json
 {
-  "mcpServers": {
+  "servers": {
     "fetch-url-mcp": {
+      "type": "stdio",
       "command": "npx",
-      "args": ["-y", "@j0hanz/fetch-url-mcp@latest", "--stdio"]
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
     }
   }
 }
 ```
 
-## Client Example (CLI)
-
-Build the server and examples, then run the client:
+Or install via CLI:
 
-```bash
-npm run build
-node dist/examples/mcp-fetch-url-client.js https://example.com
+```sh
+code-insiders --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'
 ```
 
-Optional flags:
+For more info, see [VS Code Insiders MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).
 
-- `--full` reads the cached markdown resource to avoid inline truncation.
-- `--task` enables task-based execution with streamed status updates.
-- `--task-ttl <ms>` sets task TTL; `--task-poll <ms>` sets poll interval.
-- `--http http://localhost:3000/mcp` connects to the Streamable HTTP server.
-- Progress updates (when emitted) are printed to stderr.
+</details>
 
-## Installation
+<details>
+<summary><b>Install in Cursor</b></summary>
 
-### NPX (Recommended)
+[![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=fetch-url-mcp&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D)
 
-No installation required — runs directly:
+Add to `~/.cursor/mcp.json`:
 
-```bash
-npx -y @j0hanz/fetch-url-mcp@latest --stdio
+```json
+{
+  "mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
+}
 ```
 
-### Global Install
+For more info, see [Cursor MCP docs](https://docs.cursor.com/context/model-context-protocol).
 
-```bash
-npm install -g @j0hanz/fetch-url-mcp
-fetch-url-mcp --stdio
-```
+</details>
 
-### From Source
+<details>
+<summary><b>Install in Visual Studio</b></summary>
 
-```bash
-git clone https://github.com/j0hanz/fetch-url-mcp.git
-cd fetch-url-mcp
-npm install
-npm run build
-node dist/index.js --stdio
-```
+[![Install in Visual Studio](https://img.shields.io/badge/Visual_Studio-Install_Server-C16FDE?logo=visualstudio&logoColor=white)](https://vs-open.link/mcp-install?%7B%22fetch-url-mcp%22%3A%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D%7D)
 
-### Docker
+For solution-scoped setup, add this to `.mcp.json` at the solution root:
 
-```bash
-docker compose up --build
+```json
+{
+  "servers": {
+    "fetch-url-mcp": {
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
+}
 ```
 
-## Configuration
+For more info, see [Visual Studio MCP docs](https://learn.microsoft.com/en-us/visualstudio/ide/mcp-servers).
 
-### Runtime Modes
-
-| Flag              | Description                                                |
-| ----------------- | ---------------------------------------------------------- |
-| `--stdio`, `-s`   | Run in stdio mode (for desktop MCP clients; default)       |
-| `--http`          | Run in HTTP mode (Streamable HTTP on port 3000 by default) |
-| `--help`, `-h`    | Show usage help                                            |
-| `--version`, `-v` | Print server version                                       |
-
-When no transport flag is passed, the server starts in **stdio mode**.
-
-### Environment Variables
-
-#### Core Settings
-
-| Variable                             | Default                   | Description                                                                                             |
-| ------------------------------------ | ------------------------- | ------------------------------------------------------------------------------------------------------- |
-| `HOST`                               | `127.0.0.1`               | HTTP server bind address                                                                                |
-| `PORT`                               | `3000`                    | HTTP server port (1024–65535)                                                                           |
-| `LOG_LEVEL`                          | `info`                    | Log level: `debug`, `info`, `warn`, `error`                                                             |
-| `FETCH_TIMEOUT_MS`                   | `15000`                   | HTTP fetch timeout in ms (1000–60000)                                                                   |
-| `CACHE_ENABLED`                      | `true`                    | Enable/disable in-memory content cache                                                                  |
-| `USER_AGENT`                         | `fetch-url-mcp/{version}` | Custom User-Agent header                                                                                |
-| `ALLOW_REMOTE`                       | `false`                   | Allow remote connections in HTTP mode                                                                   |
-| `ALLOWED_HOSTS`                      | _(empty)_                 | Comma-separated host/origin allowlist for HTTP mode                                                     |
-| `MCP_STRICT_PROTOCOL_VERSION_HEADER` | `true`                    | Require `MCP-Protocol-Version` on HTTP session initialize (`false` allows legacy headerless initialize) |
-
-#### Task Management
-
-| Variable                     | Default | Description                                              |
-| ---------------------------- | ------- | -------------------------------------------------------- |
-| `TASKS_MAX_TOTAL`            | `5000`  | Maximum retained task records across all owners          |
-| `TASKS_MAX_PER_OWNER`        | `1000`  | Maximum retained task records per session/client         |
-| `TASKS_STATUS_NOTIFICATIONS` | `false` | Emit experimental `notifications/tasks/status` extension |
-
-#### Authentication (HTTP Mode)
-
-| Variable                  | Default   | Description                             |
-| ------------------------- | --------- | --------------------------------------- |
-| `ACCESS_TOKENS`           | _(empty)_ | Comma-separated static bearer tokens    |
-| `API_KEY`                 | _(empty)_ | Single API key (added to static tokens) |
-| `OAUTH_ISSUER_URL`        | _(empty)_ | OAuth issuer URL (enables OAuth mode)   |
-| `OAUTH_AUTHORIZATION_URL` | _(empty)_ | OAuth authorization endpoint            |
-| `OAUTH_TOKEN_URL`         | _(empty)_ | OAuth token endpoint                    |
-| `OAUTH_INTROSPECTION_URL` | _(empty)_ | OAuth token introspection endpoint      |
-| `OAUTH_REVOCATION_URL`    | _(empty)_ | OAuth token revocation endpoint         |
-| `OAUTH_REGISTRATION_URL`  | _(empty)_ | OAuth dynamic client registration       |
-| `OAUTH_REQUIRED_SCOPES`   | _(empty)_ | Required OAuth scopes                   |
-| `OAUTH_CLIENT_ID`         | _(empty)_ | OAuth client ID                         |
-| `OAUTH_CLIENT_SECRET`     | _(empty)_ | OAuth client secret                     |
-
-#### Transform & Workers
-
-| Variable                                   | Default   | Description                               |
-| ------------------------------------------ | --------- | ----------------------------------------- |
-| `TRANSFORM_WORKER_MODE`                    | `threads` | Worker mode: `threads` or `process`       |
-| `TRANSFORM_WORKER_MAX_OLD_GENERATION_MB`   | _(unset)_ | V8 old generation heap limit per worker   |
-| `TRANSFORM_WORKER_MAX_YOUNG_GENERATION_MB` | _(unset)_ | V8 young generation heap limit per worker |
-| `TRANSFORM_WORKER_CODE_RANGE_MB`           | _(unset)_ | V8 code range limit per worker            |
-| `TRANSFORM_WORKER_STACK_MB`                | _(unset)_ | Stack size limit per worker               |
-
-#### Content Tuning
-
-| Variable                              | Default           | Description                                      |
-| ------------------------------------- | ----------------- | ------------------------------------------------ |
-| `MAX_INLINE_CONTENT_CHARS`            | `0`               | Global inline markdown limit (`0` = unlimited)   |
-| `FETCH_URL_MCP_EXTRA_NOISE_TOKENS`    | _(empty)_         | Additional CSS class/id tokens for noise removal |
-| `FETCH_URL_MCP_EXTRA_NOISE_SELECTORS` | _(empty)_         | Additional CSS selectors for noise removal       |
-| `MARKDOWN_HEADING_KEYWORDS`           | _(built-in list)_ | Keywords triggering heading promotion            |
-| `FETCH_URL_MCP_LOCALE`                | _(system)_        | Locale for content processing                    |
-
-#### Server Tuning
-
-| Variable                           | Default         | Description                              |
-| ---------------------------------- | --------------- | ---------------------------------------- |
-| `SERVER_MAX_CONNECTIONS`           | `0` (unlimited) | Maximum concurrent HTTP connections      |
-| `SERVER_BLOCK_PRIVATE_CONNECTIONS` | `false`         | Block connections from private IP ranges |
-
-### Hardcoded Defaults
-
-| Setting                  | Value                           |
-| ------------------------ | ------------------------------- |
-| Max HTML size            | 10 MB                           |
-| Max inline content chars | 0 (unlimited, configurable)     |
-| Fetch timeout            | 15 s                            |
-| Transform timeout        | 30 s                            |
-| Tool timeout             | Fetch + Transform + 5 s padding |
-| Max redirects            | 5                               |
-| Cache TTL                | 86400 s (24 h)                  |
-| Cache max keys           | 100                             |
-| Rate limit               | 100 requests / 60 s             |
-| Max sessions             | 200                             |
-| Session TTL              | 30 min                          |
-| Max URL length           | 2048 chars                      |
-| Worker pool max scale    | 4                               |
-
-## Usage
-
-### Stdio Mode
-
-```bash
-fetch-url-mcp --stdio
+</details>
+
+<details>
+<summary><b>Install in Goose</b></summary>
+
+[![Install in Goose](https://block.github.io/goose/img/extension-install-dark.svg)](https://block.github.io/goose/extension?cmd=npx&arg=-y&arg=%40j0hanz%2Ffetch-url-mcp%40latest&id=%40j0hanz%2Ffetch-url-mcp&name=fetch-url-mcp&description=A%20web%20content%20fetcher%20MCP%20server%20that%20converts%20HTML%20to%20clean%2C%20AI%20and%20human%20readable%20markdown.)
+
+Add to `~/.config/goose/config.yaml` on macOS/Linux or `%APPDATA%\Block\goose\config\config.yaml` on Windows:
+
+```yaml
+extensions:
+  fetch-url-mcp:
+    name: fetch-url-mcp
+    cmd: npx
+    args: ['-y', '@j0hanz/fetch-url-mcp@latest']
+    enabled: true
+    type: stdio
+    timeout: 300
 ```
 
-The server communicates via JSON-RPC over stdin/stdout. All MCP clients that support stdio transport can connect directly.
+For more info, see [Goose extension docs](https://block.github.io/goose/docs/getting-started/using-extensions/).
+
+</details>
+
+<details>
+<summary><b>Install in LM Studio</b></summary>
+
+[![Add to LM Studio](https://files.lmstudio.ai/deeplink/mcp-install-light.svg)](https://lmstudio.ai/install-mcp?name=fetch-url-mcp&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D)
 
-### HTTP Mode
+Add to `~/.lmstudio/mcp.json` on macOS/Linux or `%USERPROFILE%/.lmstudio/mcp.json` on Windows:
 
-```bash
-fetch-url-mcp
-# or
-PORT=8080 HOST=0.0.0.0 ALLOW_REMOTE=true fetch-url-mcp
+```json
+{
+  "mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
+}
 ```
 
-The server starts a Streamable HTTP endpoint at `/mcp`. Authenticate with bearer tokens via the `ACCESS_TOKENS` or `API_KEY` environment variables.
+For more info, see [LM Studio MCP docs](https://lmstudio.ai/docs/basics/mcp).
 
-For `POST /mcp`, clients should send:
+</details>
 
-- `Accept: application/json, text/event-stream`
-- `MCP-Protocol-Version: 2025-11-25` (or `2025-03-26` for legacy clients)
+<details>
+<summary><b>Install in Claude Desktop</b></summary>
 
-## MCP Surface
+Add to `claude_desktop_config.json`:
 
-### Tools
+```json
+{
+  "mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
+}
+```
 
-#### `fetch-url`
+For more info, see [Claude Desktop MCP docs](https://modelcontextprotocol.io/quickstart/user).
 
-Fetches a webpage and converts it to clean Markdown format optimized for LLM context.
+</details>
 
-**Useful for:**
+<details>
+<summary><b>Install in Claude Code</b></summary>
 
-- Reading documentation, blog posts, or articles
-- Extracting main content while removing navigation and ads
-- Caching content to speed up repeated queries
+Use the CLI:
 
-**Limitations:**
+```sh
+claude mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest
+```
 
-- Does not execute complex client-side JavaScript interactions
-- Inline output may be truncated when `MAX_INLINE_CONTENT_CHARS` is set
+For project-scoped config, Claude Code writes `.mcp.json` with:
 
-##### Parameters
+```json
+{
+  "mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
+      "env": {}
+    }
+  }
+}
+```
 
-| Parameter          | Type           | Required | Default | Description                                                                |
-| ------------------ | -------------- | -------- | ------- | -------------------------------------------------------------------------- |
-| `url`              | `string` (URL) | Yes      | —       | The URL of the webpage to fetch (http/https, max 2048 chars)               |
-| `skipNoiseRemoval` | `boolean`      | No       | `false` | Preserve navigation, footers, and other elements normally filtered         |
-| `forceRefresh`     | `boolean`      | No       | `false` | Bypass cache and fetch fresh content                                       |
-| `maxInlineChars`   | `number`       | No       | `0`     | Per-call inline markdown limit (`0` = unlimited; global cap still applies) |
+For more info, see [Claude Code MCP docs](https://docs.anthropic.com/en/docs/claude-code/mcp).
 
-##### Returns
+</details>
+
+<details>
+<summary><b>Install in Windsurf</b></summary>
+
+Add to `~/.codeium/windsurf/mcp_config.json`:
 
 ```json
 {
-  "url": "https://example.com",
-  "inputUrl": "https://example.com",
-  "resolvedUrl": "https://example.com",
-  "finalUrl": "https://example.com",
-  "title": "Example Domain",
-  "metadata": {
-    "title": "Example Domain",
-    "description": "...",
-    "author": "...",
-    "image": "...",
-    "favicon": "...",
-    "publishedAt": "...",
-    "modifiedAt": "..."
-  },
-  "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
-  "fromCache": false,
-  "fetchedAt": "2026-02-11T12:00:00.000Z",
-  "contentSize": 1234,
-  "truncated": false
+  "mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
 }
 ```
 
-| Field         | Type       | Description                                                                              |
-| ------------- | ---------- | ---------------------------------------------------------------------------------------- |
-| `url`         | `string`   | The canonical URL (pre-raw-transform)                                                    |
-| `inputUrl`    | `string?`  | The original URL provided by the caller                                                  |
-| `resolvedUrl` | `string?`  | The normalized/transformed URL that was fetched                                          |
-| `finalUrl`    | `string?`  | Final response URL after redirects                                                       |
-| `title`       | `string?`  | Extracted page title                                                                     |
-| `metadata`    | `object?`  | Extracted metadata (title, description, author, image, favicon, publishedAt, modifiedAt) |
-| `markdown`    | `string?`  | Extracted content in Markdown format                                                     |
-| `fromCache`   | `boolean?` | Whether the response was served from cache                                               |
-| `fetchedAt`   | `string?`  | ISO timestamp for fetch/cache retrieval                                                  |
-| `contentSize` | `number?`  | Full markdown size before inline truncation                                              |
-| `truncated`   | `boolean?` | Whether inline markdown was truncated                                                    |
-| `error`       | `string?`  | Error message if the request failed                                                      |
-| `statusCode`  | `number?`  | HTTP status code for failed requests                                                     |
-| `details`     | `object?`  | Additional error details                                                                 |
-
-##### Annotations
+For more info, see [Windsurf MCP docs](https://docs.windsurf.com/windsurf/cascade/mcp).
 
-| Annotation        | Value   |
-| ----------------- | ------- |
-| `readOnlyHint`    | `true`  |
-| `destructiveHint` | `false` |
-| `idempotentHint`  | `true`  |
-| `openWorldHint`   | `true`  |
+</details>
 
-##### Async Task Execution
+<details>
+<summary><b>Install in Amp</b></summary>
 
-The `fetch-url` tool supports optional async task execution (`execution.taskSupport: "optional"`). Include a `task` field in the tool call to run the fetch in the background:
+Add to `~/.config/amp/settings.json` on macOS/Linux, `%USERPROFILE%\.config\amp\settings.json` on Windows, or `.amp/settings.json` for workspace-scoped config:
 
 ```json
 {
-  "method": "tools/call",
-  "params": {
-    "name": "fetch-url",
-    "arguments": { "url": "https://example.com" },
-    "task": { "ttl": 30000 }
+  "amp.mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
   }
 }
 ```
 
-Then poll `tasks/get` until the task status is `completed` or `failed`, and retrieve the result via `tasks/result`.
+Or install via CLI:
 
-### Prompts
+```sh
+amp mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest
+```
 
-| Name       | Description                       |
-| ---------- | --------------------------------- |
-| `get-help` | Returns server usage instructions |
+For more info, see [Amp docs](https://ampcode.com/manual).
 
-### Resources
+</details>
 
-| URI Pattern                           | MIME Type       | Description                                          |
-| ------------------------------------- | --------------- | ---------------------------------------------------- |
-| `internal://instructions`             | `text/markdown` | Server instructions and usage guidance               |
-| `internal://cache/{namespace}/{hash}` | `text/markdown` | Cached markdown entries from prior `fetch-url` calls |
+<details>
+<summary><b>Install in Cline</b></summary>
 
-### Tasks
+Open the MCP Servers panel, choose `Configure MCP Servers`, and add this to `cline_mcp_settings.json`:
 
-The server declares full MCP task support:
+```json
+{
+  "mcpServers": {
+    "fetch-url-mcp": {
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
+    }
+  }
+}
+```
 
-| Endpoint       | Description                          |
-| -------------- | ------------------------------------ |
-| `tasks/list`   | List tasks (scoped to session/owner) |
-| `tasks/get`    | Get task status by ID                |
-| `tasks/result` | Retrieve completed task result       |
-| `tasks/cancel` | Cancel an in-flight task             |
+For more info, see [Cline MCP docs](https://docs.cline.bot/mcp/configuring-mcp-servers).
 
-## HTTP Mode Endpoints
+</details>
 
-| Method   | Path                                | Auth  | Description                              |
-| -------- | ----------------------------------- | ----- | ---------------------------------------- |
-| `GET`    | `/health`                           | No    | Health check (minimal payload)           |
-| `GET`    | `/health?verbose=true`              | Yes\* | Detailed diagnostics and runtime metrics |
-| `POST`   | `/mcp`                              | Yes   | MCP JSON-RPC (Streamable HTTP)           |
-| `GET`    | `/mcp`                              | Yes   | SSE stream for server-initiated messages |
-| `DELETE` | `/mcp`                              | Yes   | Terminate MCP session                    |
-| `GET`    | `/mcp/downloads/{namespace}/{hash}` | Yes   | Download cached content                  |
+<details>
+<summary><b>Install in Codex CLI</b></summary>
 
-\* `verbose=true` can be read without auth only for local-only deployments (`ALLOW_REMOTE=false`).
+Use the CLI:
 
-### Session Behavior
+```sh
+codex mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest
+```
 
-- Sessions are created on the first `POST /mcp` request with an `initialize` message
-- Session ID is returned in the `mcp-session-id` response header
-- Sessions expire after 30 minutes of inactivity (max 200 concurrent)
+Or add this to `~/.codex/config.toml` or project-scoped `.codex/config.toml`:
 
-### Authentication
+```toml
+[mcp_servers.fetch-url-mcp]
+command = "npx"
+args = ["-y", "@j0hanz/fetch-url-mcp@latest"]
+```
 
-- **Static tokens**: Set `ACCESS_TOKENS` or `API_KEY` environment variables; pass as `Authorization: Bearer <token>`
-- **OAuth**: Configure `OAUTH_*` environment variables to enable OAuth 2.0 token introspection
+For more info, see [Codex MCP docs](https://developers.openai.com/codex/mcp/).
 
-## Client Configuration Examples
+</details>
 
 <details>
-<summary>VS Code / VS Code Insiders</summary>
+<summary><b>Install in GitHub Copilot</b></summary>
 
-Add to your VS Code settings (`.vscode/mcp.json` or User Settings):
+Add to `.vscode/mcp.json`:
 
 ```json
 {
   "servers": {
     "fetch-url-mcp": {
+      "type": "stdio",
       "command": "npx",
-      "args": ["-y", "@j0hanz/fetch-url-mcp@latest", "--stdio"]
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
     }
   }
 }
 ```
 
+For more info, see [GitHub Copilot MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).
+
 </details>
 
 <details>
-<summary>Claude Desktop</summary>
+<summary><b>Install in Warp</b></summary>
 
-Add to `claude_desktop_config.json`:
+Open `Personal > MCP Servers` in Warp, choose `+ Add`, and either add a CLI server with:
+
+- `command`: `npx`
+- `args`: `["-y", "@j0hanz/fetch-url-mcp@latest"]`
+
+Or paste this JSON snippet when using Warp's multi-server import flow:
 
 ```json
 {
   "mcpServers": {
     "fetch-url-mcp": {
       "command": "npx",
-      "args": ["-y", "@j0hanz/fetch-url-mcp@latest", "--stdio"]
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
     }
   }
 }
 ```
 
+For more info, see [Warp MCP docs](https://docs.warp.dev/features/warp-ai/mcp).
+
 </details>
 
 <details>
-<summary>Cursor</summary>
+<summary><b>Install in Kiro</b></summary>
+
+Use Kiro's MCP Servers panel or the `Add to Kiro` install flow. Kiro stores workspace-scoped MCP config in `.kiro/settings/mcp.json` and user-scoped config in `~/.kiro/settings/mcp.json`.
+
+For this server, use:
 
-[![Install in Cursor](https://img.shields.io/badge/Cursor-Install-f97316?logo=cursor&logoColor=white)](https://cursor.com/install-mcp?name=fetch-url-mcp&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiLCItLXN0ZGlvIl19)
+- `command`: `npx`
+- `args`: `["-y", "@j0hanz/fetch-url-mcp@latest"]`
 
-Or manually add to Cursor MCP settings:
+For more info, see [Kiro MCP docs](https://kiro.dev/blog/unlock-your-development-productivity-with-kiro-and-mcp/).
+
+</details>
+
+<details>
+<summary><b>Install in Gemini CLI</b></summary>
+
+Add to `~/.gemini/settings.json`:
 
 ```json
 {
   "mcpServers": {
     "fetch-url-mcp": {
       "command": "npx",
-      "args": ["-y", "@j0hanz/fetch-url-mcp@latest", "--stdio"]
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
     }
   }
 }
 ```
 
+For more info, see [Gemini CLI MCP docs](https://google-gemini.github.io/gemini-cli/docs/tools/mcp-server.html).
+
 </details>
 
 <details>
-<summary>Windsurf</summary>
+<summary><b>Install in Zed</b></summary>
 
-Add to your Windsurf MCP configuration:
+Add to `~/.config/zed/settings.json`:
 
 ```json
 {
-  "mcpServers": {
+  "context_servers": {
     "fetch-url-mcp": {
       "command": "npx",
-      "args": ["-y", "@j0hanz/fetch-url-mcp@latest", "--stdio"]
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
+      "env": {}
     }
   }
 }
 ```
 
+For more info, see [Zed MCP docs](https://zed.dev/docs/ai/mcp).
+
 </details>
 
 <details>
-<summary>Docker</summary>
+<summary><b>Install in Augment</b></summary>
 
-Use the published image from GitHub Container Registry:
+Use the Augment Settings panel and either add the server manually or choose `Import from JSON`:
 
 ```json
 {
   "mcpServers": {
     "fetch-url-mcp": {
-      "command": "docker",
-      "args": [
-        "run",
-        "-i",
-        "--rm",
-        "ghcr.io/j0hanz/fetch-url-mcp:latest",
-        "--stdio"
-      ]
+      "command": "npx",
+      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
     }
   }
 }
 ```
 
-Or build and run locally:
-
-```bash
-docker build -t fetch-url-mcp .
-docker run -i --rm fetch-url-mcp --stdio
-```
+For more info, see [Augment MCP docs](https://docs.augmentcode.com/setup-augment/mcp).
 
 </details>
 
-## Security
+<details>
+<summary><b>Install in Roo Code</b></summary>
 
-### SSRF Protection
+Use Roo Code's MCP Servers UI or marketplace flow.
 
-Fetch URL blocks requests to private and internal network addresses:
+For this server, use:
 
-- **Blocked hosts**: `localhost`, `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `100.64.0.0/10`
-- **Blocked IPv6**: `::1`, `fc00::/7`, `fe80::/10`, IPv4-mapped private addresses
-- **Cloud metadata**: `169.254.169.254` (AWS), `metadata.google.internal`, `metadata.azure.com`, `100.100.100.200` (Azure IMDS)
+- `command`: `npx`
+- `args`: `["-y", "@j0hanz/fetch-url-mcp@latest"]`
 
-DNS preflight checks run on every redirect hop to prevent DNS rebinding attacks.
+For more info, see [Roo Code docs](https://docs.roocode.com/).
 
-### Stdio Transport Safety
+</details>
 
-The server never writes non-protocol data to stdout. All logs and diagnostics go to stderr.
+<details>
+<summary><b>Install in Kilo Code</b></summary>
 
-### Rate Limiting
+Use Kilo Code's MCP Servers UI or marketplace flow.
 
-HTTP mode enforces a rate limit of 100 requests per 60-second window per client.
+For this server, use:
 
-### Content Safety
+- `command`: `npx`
+- `args`: `["-y", "@j0hanz/fetch-url-mcp@latest"]`
 
-- HTML downloads are capped at 10 MB
-- Worker threads run in isolation with configurable resource limits
-- Auth tokens are stored in-memory only and compared using timing-safe equality
+For more info, see [Kilo Code docs](https://kilocode.ai/docs).
 
-## Development Workflow
+</details>
 
-### Install Dependencies
+## Use Cases
 
-```bash
-npm install
-```
+- Fetch documentation pages, blog posts, or reference material into Markdown before sending them to an LLM.
+- Retrieve repository-hosted content from GitHub, GitLab, Bitbucket, or Gists and let the server rewrite page URLs to raw endpoints when possible.
+- Reuse cached Markdown through `internal://cache/{namespace}/{hash}` or bypass the cache with `forceRefresh` for time-sensitive pages.
+- Use task mode for large pages or slower sites when the inline response would otherwise be truncated or delayed.
 
-### Scripts
-
-| Script          | Command                 | Description                                  |
-| --------------- | ----------------------- | -------------------------------------------- |
-| `dev`           | `npm run dev`           | TypeScript watch mode                        |
-| `dev:run`       | `npm run dev:run`       | Run compiled output with watch + `.env`      |
-| `build`         | `npm run build`         | Clean, compile, copy assets, make executable |
-| `start`         | `npm start`             | Run compiled server                          |
-| `test`          | `npm test`              | Run test suite (Node.js native test runner)  |
-| `test:coverage` | `npm run test:coverage` | Run tests with coverage                      |
-| `lint`          | `npm run lint`          | ESLint                                       |
-| `lint:fix`      | `npm run lint:fix`      | ESLint with auto-fix                         |
-| `format`        | `npm run format`        | Prettier                                     |
-| `type-check`    | `npm run type-check`    | TypeScript type checking                     |
-| `inspector`     | `npm run inspector`     | Build and launch MCP Inspector               |
+## Architecture
 
-## Build and Release
+```text
+[MCP Client]
+  ├─ stdio -> `src/index.ts` -> `startStdioServer()` -> `createMcpServer()`
+  └─ HTTP (`--http`) -> `src/index.ts` -> `startHttpServer()` -> HTTP dispatcher
+       ├─ `GET /health`
+       ├─ `GET /.well-known/oauth-protected-resource`
+       ├─ `GET /.well-known/oauth-protected-resource/mcp`
+       ├─ `GET /mcp/downloads/{namespace}/{hash}`
+       └─ `POST|GET|DELETE /mcp`
+
+`createMcpServer()`
+  ├─ registers tool: `fetch-url`
+  ├─ registers prompt: `get-help`
+  ├─ registers resources:
+  │    - `internal://instructions`
+  │    - `internal://cache/{namespace}/{hash}`
+  ├─ enables capabilities: completions, logging, resources, prompts, tasks
+  └─ installs task handlers, log-level handling, and shutdown cleanup
+
+`fetch-url` execution
+  ├─ validate input with `fetchUrlInputSchema`
+  ├─ normalize URL and block local/private targets unless allowed
+  ├─ rewrite supported code-host URLs to raw endpoints when possible
+  ├─ fetch and cache content via the shared pipeline
+  ├─ transform HTML into Markdown in the transform worker path
+  └─ validate `structuredContent` with `fetchUrlOutputSchema`
+```
 
-```bash
-npm run build           # Clean → Compile → Copy Assets → chmod
-npm run prepublishOnly  # Lint → Type-Check → Build
-npm publish             # Publish to npm
+### Request Lifecycle
+
+```text
+[Client] -- initialize {protocolVersion, capabilities} --> [Server]
+[Server] -- {protocolVersion, capabilities, serverInfo} --> [Client]
+[Client] -- notifications/initialized --> [Server]
+[Client] -- tools/call {name, arguments} --> [Server]
+[Server] -- {content: [{type, text}], structuredContent?, isError?} --> [Client]
 ```
 
-CI/CD is handled via a GitHub Actions workflow (`release.yml`) that runs lint, type-check, test, build, and publishes to npm with version bumping.
+## MCP Surface
 
-## Troubleshooting
+### Tools
 
-### MCP Inspector
+#### `fetch-url`
+
+Fetch public webpages and convert HTML into AI-readable Markdown. The tool is read-only, does not execute page JavaScript, can bypass the cache with `forceRefresh`, and supports optional task mode for larger or slower fetches.
+
+| Parameter          | Type      | Required | Description                                                                                 |
+| ------------------ | --------- | -------- | ------------------------------------------------------------------------------------------- |
+| `url`              | `string`  | yes      | Target URL. Max 2048 chars.                                                                 |
+| `skipNoiseRemoval` | `boolean` | no       | Preserve navigation/footers (disable noise filtering).                                      |
+| `forceRefresh`     | `boolean` | no       | Bypass cache and fetch fresh content.                                                       |
+| `maxInlineChars`   | `integer` | no       | Inline markdown limit (0-10485760, 0=unlimited). Lower of this or the global limit applies. |
 
-Use the built-in inspector to test the server interactively:
+The response is returned as MCP text content and, when validation succeeds, as `structuredContent` containing `url`, `resolvedUrl`, `finalUrl`, `title`, `metadata`, `markdown`, `fromCache`, `fetchedAt`, `contentSize`, and `truncated`.
 
-```bash
-npm run inspector
+```text
+1. [Client] -- tools/call {name: "fetch-url", arguments} --> [Server]
+2. [Server] -- dispatch("fetch-url") --> [src/tools/fetch-url.ts]
+3. [Handler] -- validate(fetchUrlInputSchema) --> normalize / fetch / transform
+4. [Handler] -- validate(fetchUrlOutputSchema) --> assemble content + structuredContent
+5. [Server] -- result or tool error --> [Client]
 ```
 
-### Common Issues
+### Resources
+
+| Resource                     | URI                                   | MIME Type       | Description                                                   |
+| ---------------------------- | ------------------------------------- | --------------- | ------------------------------------------------------------- |
+| `fetch-url-mcp-instructions` | `internal://instructions`             | `text/markdown` | Guidance for using the Fetch URL MCP server.                  |
+| `fetch-url-mcp-cache-entry`  | `internal://cache/{namespace}/{hash}` | `text/markdown` | Read cached markdown generated by previous `fetch-url` calls. |
+
+### Prompts
+
+| Prompt     | Arguments | Description                                                                                  |
+| ---------- | --------- | -------------------------------------------------------------------------------------------- |
+| `get-help` | none      | Return Fetch URL server instructions: workflows, cache usage, task mode, and error handling. |
+
+## MCP Capabilities
+
+| Capability                      | Status    | Notes                                                                                                                     |
+| ------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------- |
+| completions                     | confirmed | Advertised in `createServerCapabilities()` and used by the cache resource template for `namespace` and `hash` completion. |
+| logging                         | confirmed | Advertised in `createServerCapabilities()` and handled through `SetLevelRequestSchema`.                                   |
+| resources subscribe/listChanged | confirmed | Advertised in `createServerCapabilities()` and implemented for cache resource subscriptions and list changes.             |
+| prompts                         | confirmed | `get-help` is registered during server startup.                                                                           |
+| tasks                           | confirmed | Advertised in `createServerCapabilities()` and backed by registered task handlers plus optional tool task support.        |
+| progress notifications          | confirmed | Tool execution reports `notifications/progress` updates during fetch and transform stages.                                |
+
+### Tool Annotations
+
+| Annotation        | Value   |
+| ----------------- | ------- |
+| `readOnlyHint`    | `true`  |
+| `destructiveHint` | `false` |
+| `idempotentHint`  | `true`  |
+| `openWorldHint`   | `true`  |
+
+### Structured Output
+
+- `fetch-url` publishes an explicit `outputSchema` and returns `structuredContent` when the assembled response passes validation.
+
+## Configuration
+
+| Variable                                   | Default                   | Applies To        | Notes                                                                 |
+| ------------------------------------------ | ------------------------- | ----------------- | --------------------------------------------------------------------- |
+| `HOST`                                     | `127.0.0.1`               | HTTP mode         | Bind address. Non-loopback bindings also require `ALLOW_REMOTE=true`. |
+| `PORT`                                     | `3000`                    | HTTP mode         | Listening port for `--http`.                                          |
+| `ALLOW_REMOTE`                             | `false`                   | HTTP mode         | Must be enabled to bind to a non-loopback interface.                  |
+| `ACCESS_TOKENS`                            | unset                     | HTTP mode         | Comma- or space-separated static bearer tokens.                       |
+| `API_KEY`                                  | unset                     | HTTP mode         | Alternate static token source for header auth.                        |
+| `OAUTH_ISSUER_URL`                         | unset                     | HTTP mode         | Enables OAuth mode when combined with the other OAuth URLs.           |
+| `OAUTH_AUTHORIZATION_URL`                  | unset                     | HTTP mode         | Optional explicit authorization endpoint.                             |
+| `OAUTH_TOKEN_URL`                          | unset                     | HTTP mode         | Optional explicit token endpoint.                                     |
+| `OAUTH_REVOCATION_URL`                     | unset                     | HTTP mode         | Optional OAuth revocation endpoint.                                   |
+| `OAUTH_REGISTRATION_URL`                   | unset                     | HTTP mode         | Optional OAuth dynamic client registration endpoint.                  |
+| `OAUTH_INTROSPECTION_URL`                  | unset                     | HTTP mode         | Required for OAuth token introspection.                               |
+| `OAUTH_REQUIRED_SCOPES`                    | empty                     | HTTP mode         | Required scopes enforced after auth.                                  |
+| `OAUTH_CLIENT_ID`                          | unset                     | HTTP mode         | Optional introspection client ID.                                     |
+| `OAUTH_CLIENT_SECRET`                      | unset                     | HTTP mode         | Optional introspection client secret.                                 |
+| `SERVER_TLS_KEY_FILE`                      | unset                     | HTTP mode         | Enable HTTPS when set together with `SERVER_TLS_CERT_FILE`.           |
+| `SERVER_TLS_CERT_FILE`                     | unset                     | HTTP mode         | TLS certificate path.                                                 |
+| `SERVER_TLS_CA_FILE`                       | unset                     | HTTP mode         | Optional custom CA bundle.                                            |
+| `SERVER_MAX_CONNECTIONS`                   | `0`                       | HTTP mode         | Optional connection cap.                                              |
+| `SERVER_HEADERS_TIMEOUT_MS`                | unset                     | HTTP mode         | Optional Node server tuning.                                          |
+| `SERVER_REQUEST_TIMEOUT_MS`                | unset                     | HTTP mode         | Optional Node server tuning.                                          |
+| `SERVER_KEEP_ALIVE_TIMEOUT_MS`             | unset                     | HTTP mode         | Optional keep-alive tuning.                                           |
+| `SERVER_KEEP_ALIVE_TIMEOUT_BUFFER_MS`      | unset                     | HTTP mode         | Optional keep-alive tuning buffer.                                    |
+| `SERVER_MAX_HEADERS_COUNT`                 | unset                     | HTTP mode         | Optional header count limit.                                          |
+| `SERVER_BLOCK_PRIVATE_CONNECTIONS`         | `false`                   | HTTP mode         | Enables inbound private-network protections.                          |
+| `MCP_STRICT_PROTOCOL_VERSION_HEADER`       | `true`                    | HTTP mode         | Requires `MCP-Protocol-Version` on session init.                      |
+| `ALLOWED_HOSTS`                            | empty                     | HTTP mode         | Additional allowed `Host` and `Origin` values.                        |
+| `ALLOW_LOCAL_FETCH`                        | `false`                   | Fetching          | Allows loopback and private-network fetch targets.                    |
+| `FETCH_TIMEOUT_MS`                         | `15000`                   | Fetching          | Network fetch timeout in milliseconds.                                |
+| `USER_AGENT`                               | `fetch-url-mcp/<version>` | Fetching          | Override the outbound user agent string.                              |
+| `MAX_INLINE_CONTENT_CHARS`                 | `0`                       | Tool output       | `0` means no explicit inline truncation limit.                        |
+| `CACHE_ENABLED`                            | `true`                    | Caching           | Enables in-memory fetch result caching.                               |
+| `TASKS_MAX_TOTAL`                          | `5000`                    | Tasks             | Total task capacity.                                                  |
+| `TASKS_MAX_PER_OWNER`                      | `1000`                    | Tasks             | Per-owner task cap, clamped to the total cap.                         |
+| `TASKS_STATUS_NOTIFICATIONS`               | `false`                   | Tasks             | Enables status notifications for tasks.                               |
+| `TASKS_REQUIRE_INTERCEPTION`               | `true`                    | Tasks             | Requires task interception for task-capable tool execution.           |
+| `TRANSFORM_CANCEL_ACK_TIMEOUT_MS`          | `200`                     | Transform workers | Cancellation acknowledgement timeout.                                 |
+| `TRANSFORM_WORKER_MODE`                    | `threads`                 | Transform workers | Worker execution mode.                                                |
+| `TRANSFORM_WORKER_MAX_OLD_GENERATION_MB`   | unset                     | Transform workers | Optional worker memory limit.                                         |
+| `TRANSFORM_WORKER_MAX_YOUNG_GENERATION_MB` | unset                     | Transform workers | Optional worker memory limit.                                         |
+| `TRANSFORM_WORKER_CODE_RANGE_MB`           | unset                     | Transform workers | Optional worker memory limit.                                         |
+| `TRANSFORM_WORKER_STACK_MB`                | unset                     | Transform workers | Optional worker stack size.                                           |
+| `FETCH_URL_MCP_EXTRA_NOISE_TOKENS`         | empty                     | Content cleanup   | Extra noise-removal tokens.                                           |
+| `FETCH_URL_MCP_EXTRA_NOISE_SELECTORS`      | empty                     | Content cleanup   | Extra DOM selectors for noise removal.                                |
+| `FETCH_URL_MCP_LOCALE`                     | system default            | Content cleanup   | Locale override for extraction heuristics.                            |
+| `MARKDOWN_HEADING_KEYWORDS`                | built-in list             | Markdown cleanup  | Override heading keywords used by cleanup.                            |
+| `LOG_LEVEL`                                | `info`                    | Logging           | `debug`, `info`, `warn`, or `error`.                                  |
+| `LOG_FORMAT`                               | `text`                    | Logging           | Set to `json` for structured logs.                                    |
+
+## HTTP Endpoints
+
+| Method   | Path                                        | Auth                                       | Purpose                                                 |
+| -------- | ------------------------------------------- | ------------------------------------------ | ------------------------------------------------------- |
+| `GET`    | `/health`                                   | no, unless `?verbose=1` on a remote server | Basic health response, with optional diagnostics.       |
+| `GET`    | `/.well-known/oauth-protected-resource`     | no                                         | OAuth protected-resource metadata.                      |
+| `GET`    | `/.well-known/oauth-protected-resource/mcp` | no                                         | OAuth protected-resource metadata for the MCP endpoint. |
+| `POST`   | `/mcp`                                      | yes                                        | Session initialization and JSON-RPC requests.           |
+| `GET`    | `/mcp`                                      | yes                                        | Session-bound server-to-client stream handling.         |
+| `DELETE` | `/mcp`                                      | yes                                        | Session shutdown.                                       |
+| `GET`    | `/mcp/downloads/{namespace}/{hash}`         | yes                                        | Download route used by HTTP-mode cached fetch results.  |
+
+## Security
+
+| Control                    | Status      | Notes                                                                                                                                    |
+| -------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
+| Host and origin validation | implemented | HTTP requests are rejected unless `Host` and `Origin` match the allowlist built from loopback, the configured host, and `ALLOWED_HOSTS`. |
+| Authentication             | implemented | HTTP mode supports static bearer tokens locally or OAuth token introspection; remote bindings require OAuth.                             |
+| Protocol version checks    | implemented | HTTP sessions validate `MCP-Protocol-Version` and pin it to the negotiated session version.                                              |
+| Rate limiting              | implemented | Requests pass through the HTTP rate limiter before route dispatch.                                                                       |
+| Outbound SSRF protections  | implemented | Local/private IPs, metadata endpoints, and `.local`/`.internal` hosts are blocked unless `ALLOW_LOCAL_FETCH=true`.                       |
+| TLS                        | optional    | HTTPS is enabled when both TLS key and certificate files are configured.                                                                 |
+| Stdio logging safety       | implemented | Server logs are written to stderr, not stdout, so stdio MCP traffic stays clean.                                                         |
+
+## Development
+
+| Script                   | Command                                                                                                             |
+| ------------------------ | ------------------------------------------------------------------------------------------------------------------- |
+| `clean`                  | `node scripts/tasks.mjs clean`                                                                                      |
+| `build`                  | `node scripts/tasks.mjs build`                                                                                      |
+| `copy:assets`            | `node scripts/tasks.mjs copy:assets`                                                                                |
+| `prepare`                | `npm run build`                                                                                                     |
+| `dev`                    | `tsc --watch --preserveWatchOutput`                                                                                 |
+| `dev:run`                | `node --env-file=.env --watch dist/index.js`                                                                        |
+| `start`                  | `node dist/index.js`                                                                                                |
+| `format`                 | `prettier --write .`                                                                                                |
+| `type-check`             | `node scripts/tasks.mjs type-check`                                                                                 |
+| `type-check:src`         | `node node_modules/typescript/bin/tsc -p tsconfig.json --noEmit`                                                    |
+| `type-check:tests`       | `node node_modules/typescript/bin/tsc -p tsconfig.test.json --noEmit`                                               |
+| `type-check:diagnostics` | `tsc --noEmit --extendedDiagnostics`                                                                                |
+| `type-check:trace`       | `node -e "require('fs').rmSync('.ts-trace',{recursive:true,force:true})" && tsc --noEmit --generateTrace .ts-trace` |
+| `lint`                   | `eslint .`                                                                                                          |
+| `lint:tests`             | `eslint src/__tests__`                                                                                              |
+| `lint:fix`               | `eslint . --fix`                                                                                                    |
+| `test`                   | `node scripts/tasks.mjs test`                                                                                       |
+| `test:fast`              | `node --test --import tsx/esm src/__tests__/**/*.test.ts node-tests/**/*.test.ts`                                   |
+| `test:coverage`          | `node scripts/tasks.mjs test --coverage`                                                                            |
+| `knip`                   | `knip`                                                                                                              |
+| `knip:fix`               | `knip --fix`                                                                                                        |
+| `inspector`              | `npm run build && npx -y @modelcontextprotocol/inspector node dist/index.js --stdio`                                |
+| `prepublishOnly`         | `npm run lint && npm run type-check && npm run build`                                                               |
+
+## Build and Release
+
+- The repository includes release automation under `.github/workflows/`.
+- `Dockerfile` and `docker-compose.yml` are available for container-based packaging and local runs.
+- `npm run prepublishOnly` runs the release gate: lint, type-check, and build.
+
+## Troubleshooting
 
-| Issue                     | Solution                                                                              |
-| ------------------------- | ------------------------------------------------------------------------------------- |
-| `VALIDATION_ERROR` on URL | URL is blocked (private IP/localhost) or malformed. Do not retry.                     |
-| `queue_full` error        | Worker pool busy. Wait briefly, then retry or use async task mode.                    |
-| Garbled output            | Binary content (images, PDFs) cannot be converted. Ensure the URL serves HTML.        |
-| No output in stdio mode   | If you intended HTTP mode, pass `--http`. Stdio is the default transport.             |
-| Auth errors in HTTP mode  | Set `ACCESS_TOKENS` or `API_KEY` env var and pass as `Authorization: Bearer <token>`. |
+- For stdio mode, avoid writing logs to stdout; keep logs on stderr.
+- For HTTP mode, verify MCP protocol headers and endpoint routing.
+- Update client snippets when client MCP configuration formats change.
 
-### Stdout / Stderr Guidance
+## Credits
 
-In stdio mode, **stdout** is reserved exclusively for MCP JSON-RPC messages. Logs and diagnostics are written to **stderr**. Never pipe stdout to a log file when using stdio transport.
+| Dependency                                                                           | Registry |
+| ------------------------------------------------------------------------------------ | -------- |
+| [@modelcontextprotocol/sdk](https://www.npmjs.com/package/@modelcontextprotocol/sdk) | npm      |
+| [@mozilla/readability](https://www.npmjs.com/package/@mozilla/readability)           | npm      |
+| [linkedom](https://www.npmjs.com/package/linkedom)                                   | npm      |
+| [node-html-markdown](https://www.npmjs.com/package/node-html-markdown)               | npm      |
+| [undici](https://www.npmjs.com/package/undici)                                       | npm      |
+| [zod](https://www.npmjs.com/package/zod)                                             | npm      |
 
-## License
+## Contributing and License
 
-[MIT](https://opensource.org/licenses/MIT)
+- License: MIT
+- Contributions are welcome via pull requests.