@page2ai/mcp 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/PRIVACY.md +51 -0
- package/README.md +135 -0
- package/dist/index.d.ts +3 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +48 -0
- package/dist/index.js.map +1 -0
- package/dist/tools/page-to-markdown.d.ts +62 -0
- package/dist/tools/page-to-markdown.d.ts.map +1 -0
- package/dist/tools/page-to-markdown.js +100 -0
- package/dist/tools/page-to-markdown.js.map +1 -0
- package/manifest.json +55 -0
- package/package.json +69 -0
- package/server.json +19 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Igor Saevets
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/PRIVACY.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Page2AI MCP Server — Privacy Policy
|
|
2
|
+
|
|
3
|
+
**Last updated:** 2026-07-23
|
|
4
|
+
**Applies to:** `@page2ai/mcp` npm package v0.1.0+
|
|
5
|
+
|
|
6
|
+
## What data this MCP server collects
|
|
7
|
+
|
|
8
|
+
**None.**
|
|
9
|
+
|
|
10
|
+
`@page2ai/mcp` is a stateless local Node.js process. When invoked with a URL:
|
|
11
|
+
|
|
12
|
+
1. It fetches the URL via Node's built-in `fetch()` API
|
|
13
|
+
2. Parses the HTML response via [linkedom](https://github.com/WebReflection/linkedom) (a lightweight in-memory DOM parser)
|
|
14
|
+
3. Extracts main content and converts it to Markdown
|
|
15
|
+
4. Returns the Markdown to the calling MCP client (Claude Desktop, Cursor, Windsurf, Zed, VS Code, etc.)
|
|
16
|
+
|
|
17
|
+
The server does not:
|
|
18
|
+
- Store any content on disk beyond ephemeral memory during processing
|
|
19
|
+
- Send telemetry, analytics, crash reports, or any other data to any third party
|
|
20
|
+
- Maintain user accounts, API keys, or authentication state
|
|
21
|
+
- Track requests or responses across invocations
|
|
22
|
+
- Log requests or responses to any location outside the calling MCP client's own logs
|
|
23
|
+
|
|
24
|
+
## What data the MCP client may log
|
|
25
|
+
|
|
26
|
+
The MCP client (Claude Desktop, Cursor, etc.) that invokes `@page2ai/mcp` may log tool invocations and results according to its own privacy policy. Refer to the specific client's privacy policy for details.
|
|
27
|
+
|
|
28
|
+
## Network communication
|
|
29
|
+
|
|
30
|
+
`@page2ai/mcp` makes outbound HTTP/HTTPS requests only to the URLs explicitly provided as tool arguments. It makes no other network calls. It does not phone home, does not check for updates over the network, and does not send crash reports.
|
|
31
|
+
|
|
32
|
+
## Authentication
|
|
33
|
+
|
|
34
|
+
`@page2ai/mcp` has no authentication mechanism because it has no user accounts or state. It cannot access content behind login walls or authenticated sessions.
|
|
35
|
+
|
|
36
|
+
## Source code and audit
|
|
37
|
+
|
|
38
|
+
The full source code is available at:
|
|
39
|
+
- https://github.com/igorsaevets/page2ai-extension (in the `packages/mcp/` directory once published)
|
|
40
|
+
|
|
41
|
+
Users can audit the source code to verify these privacy claims.
|
|
42
|
+
|
|
43
|
+
## Reporting concerns
|
|
44
|
+
|
|
45
|
+
If you believe you have found a privacy issue with `@page2ai/mcp`:
|
|
46
|
+
- **Email:** igorsaevets@gmail.com
|
|
47
|
+
- **Subject line:** `[Page2AI MCP Privacy]`
|
|
48
|
+
|
|
49
|
+
## Changes to this policy
|
|
50
|
+
|
|
51
|
+
This policy will be versioned in the same repository as the source code. Material changes will be reflected in a new version of the npm package with an accompanying changelog entry.
|
package/README.md
ADDED
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# @page2ai/mcp — Web to Markdown for LLM Context
|
|
2
|
+
|
|
3
|
+
Turn any web page into clean Markdown for Claude, ChatGPT, or your own LLM.
|
|
4
|
+
|
|
5
|
+
Companion to the [Page2AI Chrome extension](https://github.com/igorsaevets/page2ai-extension). Shares the same `@page2ai/core` extraction library. Zero external API calls — runs entirely on your machine using [linkedom](https://github.com/WebReflection/linkedom).
|
|
6
|
+
|
|
7
|
+
## Install
|
|
8
|
+
|
|
9
|
+
### Claude Desktop
|
|
10
|
+
|
|
11
|
+
Add to `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%/Claude/claude_desktop_config.json` (Windows):
|
|
12
|
+
|
|
13
|
+
```json
|
|
14
|
+
{
|
|
15
|
+
"mcpServers": {
|
|
16
|
+
"page2ai": {
|
|
17
|
+
"command": "npx",
|
|
18
|
+
"args": ["-y", "@page2ai/mcp"]
|
|
19
|
+
}
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Restart Claude Desktop.
|
|
25
|
+
|
|
26
|
+
### Cursor
|
|
27
|
+
|
|
28
|
+
Add to `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project):
|
|
29
|
+
|
|
30
|
+
```json
|
|
31
|
+
{
|
|
32
|
+
"mcpServers": {
|
|
33
|
+
"page2ai": {
|
|
34
|
+
"command": "npx",
|
|
35
|
+
"args": ["-y", "@page2ai/mcp"]
|
|
36
|
+
}
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
### Windsurf
|
|
42
|
+
|
|
43
|
+
Add to `~/.windsurf/mcp.json`:
|
|
44
|
+
|
|
45
|
+
```json
|
|
46
|
+
{
|
|
47
|
+
"mcpServers": {
|
|
48
|
+
"page2ai": {
|
|
49
|
+
"command": "npx",
|
|
50
|
+
"args": ["-y", "@page2ai/mcp"]
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### Zed
|
|
57
|
+
|
|
58
|
+
Add to `settings.json`:
|
|
59
|
+
|
|
60
|
+
```json
|
|
61
|
+
{
|
|
62
|
+
"context_servers": {
|
|
63
|
+
"page2ai": {
|
|
64
|
+
"command": {
|
|
65
|
+
"path": "npx",
|
|
66
|
+
"args": ["-y", "@page2ai/mcp"]
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### VS Code (with Continue or GitHub Copilot Chat)
|
|
74
|
+
|
|
75
|
+
Refer to your MCP-compatible extension's documentation. The command is `npx -y @page2ai/mcp`.
|
|
76
|
+
|
|
77
|
+
## Tools
|
|
78
|
+
|
|
79
|
+
| Tool | Description | Read-only | Example |
|
|
80
|
+
|------|-------------|-----------|---------|
|
|
81
|
+
| `page_to_markdown` | Fetch a web page URL and convert to clean Markdown | ✅ | `page_to_markdown(url="https://docs.anthropic.com/en/api/messages")` |
|
|
82
|
+
|
|
83
|
+
## Example prompts
|
|
84
|
+
|
|
85
|
+
**1. Fetch documentation and answer questions:**
|
|
86
|
+
> "Use page_to_markdown to fetch https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking and summarize the three main use cases for extended thinking."
|
|
87
|
+
|
|
88
|
+
**2. Extract API reference into a code snippet:**
|
|
89
|
+
> "Fetch https://ai.google.dev/gemini-api/docs/thinking with page_to_markdown, then generate a Python code sample using the thinking budget parameter."
|
|
90
|
+
|
|
91
|
+
**3. Compare two documentation pages:**
|
|
92
|
+
> "Fetch both https://docs.anthropic.com/en/docs/prompt-engineering and https://platform.openai.com/docs/guides/prompt-engineering with page_to_markdown, then summarize the differences in approach."
|
|
93
|
+
|
|
94
|
+
## Configuration
|
|
95
|
+
|
|
96
|
+
None required in v0.1. All extraction options use sensible defaults.
|
|
97
|
+
|
|
98
|
+
Future versions will support options via tool arguments:
|
|
99
|
+
- `include_images` (boolean, default `false`)
|
|
100
|
+
- `include_frontmatter` (boolean, default `true`)
|
|
101
|
+
- `profile` (string, one of `auto | docs | marketing | research | dashboard | wordpress-marketing`, default `auto`)
|
|
102
|
+
|
|
103
|
+
## Privacy
|
|
104
|
+
|
|
105
|
+
`@page2ai/mcp` collects no data, sends no telemetry, and makes no external network calls beyond the URLs you explicitly provide. See [PRIVACY.md](./PRIVACY.md) for details.
|
|
106
|
+
|
|
107
|
+
## Development
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
git clone https://github.com/igorsaevets/page2ai-mcp
|
|
111
|
+
cd page2ai-mcp
|
|
112
|
+
npm install
|
|
113
|
+
npm run build
|
|
114
|
+
node dist/index.js # runs stdio MCP server
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
Test with MCP Inspector:
|
|
118
|
+
```bash
|
|
119
|
+
npx @modelcontextprotocol/inspector node dist/index.js
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
## Support
|
|
123
|
+
|
|
124
|
+
- GitHub issues: https://github.com/igorsaevets/page2ai-mcp/issues
|
|
125
|
+
- Author: Igor Saevets (igorsaevets@gmail.com)
|
|
126
|
+
|
|
127
|
+
## License
|
|
128
|
+
|
|
129
|
+
MIT — see [LICENSE](./LICENSE). Copyright © 2026 Igor Saevets.
|
|
130
|
+
|
|
131
|
+
## Related
|
|
132
|
+
|
|
133
|
+
- **Page2AI Chrome extension** — https://github.com/igorsaevets/page2ai-extension (same extraction core, distributed as a browser extension for humans)
|
|
134
|
+
- **@page2ai/core** — https://npmjs.com/package/@page2ai/core (the shared extraction library)
|
|
135
|
+
- **Software Heritage archive** — SWHID `swh:1:snp:05123c51ef9e7c0aeb06f42b1263c07a8d26999a`
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":""}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// @page2ai/mcp — MCP server entry point.
|
|
3
|
+
// Runs as a stdio subprocess spawned by Claude Desktop, Cursor, Windsurf, Zed.
|
|
4
|
+
// Registers a single tool `page_to_markdown` that converts URLs to Markdown
|
|
5
|
+
// via @page2ai/core (SSRF-guarded, 10MB size cap, 15s timeout by default).
|
|
6
|
+
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
|
7
|
+
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
|
8
|
+
import { CallToolRequestSchema, ListToolsRequestSchema, } from '@modelcontextprotocol/sdk/types.js';
|
|
9
|
+
import { handlePageToMarkdown, pageToMarkdownDefinition } from './tools/page-to-markdown.js';
|
|
10
|
+
const SERVER_NAME = 'page2ai-mcp';
|
|
11
|
+
const SERVER_VERSION = '0.1.0';
|
|
12
|
+
const server = new Server({
|
|
13
|
+
name: SERVER_NAME,
|
|
14
|
+
version: SERVER_VERSION,
|
|
15
|
+
}, {
|
|
16
|
+
capabilities: {
|
|
17
|
+
tools: {},
|
|
18
|
+
},
|
|
19
|
+
});
|
|
20
|
+
server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
21
|
+
tools: [pageToMarkdownDefinition],
|
|
22
|
+
}));
|
|
23
|
+
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
24
|
+
const toolName = request.params.name;
|
|
25
|
+
const toolArgs = request.params.arguments ?? {};
|
|
26
|
+
if (toolName === 'page_to_markdown') {
|
|
27
|
+
return handlePageToMarkdown(toolArgs);
|
|
28
|
+
}
|
|
29
|
+
return {
|
|
30
|
+
isError: true,
|
|
31
|
+
content: [{ type: 'text', text: `Unknown tool: ${toolName}` }],
|
|
32
|
+
};
|
|
33
|
+
});
|
|
34
|
+
// Log to stderr — stdio transport uses stdout for JSON-RPC frames, so anything
|
|
35
|
+
// written to stderr is safe (Claude Desktop logs it to the debug console).
|
|
36
|
+
const log = (msg) => {
|
|
37
|
+
process.stderr.write(`[${SERVER_NAME}] ${msg}\n`);
|
|
38
|
+
};
|
|
39
|
+
const main = async () => {
|
|
40
|
+
const transport = new StdioServerTransport();
|
|
41
|
+
await server.connect(transport);
|
|
42
|
+
log(`server ${SERVER_VERSION} ready on stdio (tools: page_to_markdown)`);
|
|
43
|
+
};
|
|
44
|
+
main().catch((err) => {
|
|
45
|
+
log(`fatal: ${err instanceof Error ? err.stack || err.message : String(err)}`);
|
|
46
|
+
process.exit(1);
|
|
47
|
+
});
|
|
48
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AACA,yCAAyC;AACzC,+EAA+E;AAC/E,4EAA4E;AAC5E,2EAA2E;AAE3E,OAAO,EAAE,MAAM,EAAE,MAAM,2CAA2C,CAAC;AACnE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AACjF,OAAO,EACL,qBAAqB,EACrB,sBAAsB,GACvB,MAAM,oCAAoC,CAAC;AAC5C,OAAO,EAAE,oBAAoB,EAAE,wBAAwB,EAAE,MAAM,6BAA6B,CAAC;AAE7F,MAAM,WAAW,GAAG,aAAa,CAAC;AAClC,MAAM,cAAc,GAAG,OAAO,CAAC;AAE/B,MAAM,MAAM,GAAG,IAAI,MAAM,CACvB;IACE,IAAI,EAAE,WAAW;IACjB,OAAO,EAAE,cAAc;CACxB,EACD;IACE,YAAY,EAAE;QACZ,KAAK,EAAE,EAAE;KACV;CACF,CACF,CAAC;AAEF,MAAM,CAAC,iBAAiB,CAAC,sBAAsB,EAAE,KAAK,IAAI,EAAE,CAAC,CAAC;IAC5D,KAAK,EAAE,CAAC,wBAAwB,CAAC;CAClC,CAAC,CAAC,CAAC;AAEJ,MAAM,CAAC,iBAAiB,CAAC,qBAAqB,EAAE,KAAK,EAAE,OAAO,EAAE,EAAE;IAChE,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC;IACrC,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM,CAAC,SAAS,IAAI,EAAE,CAAC;IAChD,IAAI,QAAQ,KAAK,kBAAkB,EAAE,CAAC;QACpC,OAAO,oBAAoB,CAAC,QAAQ,CAAC,CAAC;IACxC,CAAC;IACD,OAAO;QACL,OAAO,EAAE,IAAI;QACb,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,MAAe,EAAE,IAAI,EAAE,iBAAiB,QAAQ,EAAE,EAAE,CAAC;KACxE,CAAC;AACJ,CAAC,CAAC,CAAC;AAEH,+EAA+E;AAC/E,2EAA2E;AAC3E,MAAM,GAAG,GAAG,CAAC,GAAW,EAAQ,EAAE;IAChC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,WAAW,KAAK,GAAG,IAAI,CAAC,CAAC;AACpD,CAAC,CAAC;AAEF,MAAM,IAAI,GAAG,KAAK,IAAmB,EAAE;IACrC,MAAM,SAAS,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAC7C,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAChC,GAAG,CAAC,UAAU,cAAc,2CAA2C,CAAC,CAAC;AAC3E,CAAC,CAAC;AAEF,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;IACnB,GAAG,CAAC,UAAU,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,KAAK,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAC/E,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
import type { CallToolResult } from '@modelcontextprotocol/sdk/types.js';
|
|
2
|
+
import { z } from 'zod';
|
|
3
|
+
export declare const pageToMarkdownInputSchema: z.ZodObject<{
|
|
4
|
+
url: z.ZodString;
|
|
5
|
+
include_images: z.ZodDefault<z.ZodOptional<z.ZodBoolean>>;
|
|
6
|
+
include_frontmatter: z.ZodDefault<z.ZodOptional<z.ZodBoolean>>;
|
|
7
|
+
timeout_ms: z.ZodDefault<z.ZodOptional<z.ZodNumber>>;
|
|
8
|
+
}, "strip", z.ZodTypeAny, {
|
|
9
|
+
url: string;
|
|
10
|
+
include_images: boolean;
|
|
11
|
+
include_frontmatter: boolean;
|
|
12
|
+
timeout_ms: number;
|
|
13
|
+
}, {
|
|
14
|
+
url: string;
|
|
15
|
+
include_images?: boolean | undefined;
|
|
16
|
+
include_frontmatter?: boolean | undefined;
|
|
17
|
+
timeout_ms?: number | undefined;
|
|
18
|
+
}>;
|
|
19
|
+
export type PageToMarkdownInput = z.infer<typeof pageToMarkdownInputSchema>;
|
|
20
|
+
export type PageToMarkdownResult = CallToolResult;
|
|
21
|
+
export declare const pageToMarkdownDefinition: {
|
|
22
|
+
name: string;
|
|
23
|
+
title: string;
|
|
24
|
+
description: string;
|
|
25
|
+
inputSchema: {
|
|
26
|
+
type: "object";
|
|
27
|
+
properties: {
|
|
28
|
+
url: {
|
|
29
|
+
type: "string";
|
|
30
|
+
format: string;
|
|
31
|
+
description: string;
|
|
32
|
+
};
|
|
33
|
+
include_images: {
|
|
34
|
+
type: "boolean";
|
|
35
|
+
default: boolean;
|
|
36
|
+
description: string;
|
|
37
|
+
};
|
|
38
|
+
include_frontmatter: {
|
|
39
|
+
type: "boolean";
|
|
40
|
+
default: boolean;
|
|
41
|
+
description: string;
|
|
42
|
+
};
|
|
43
|
+
timeout_ms: {
|
|
44
|
+
type: "integer";
|
|
45
|
+
minimum: number;
|
|
46
|
+
maximum: number;
|
|
47
|
+
default: number;
|
|
48
|
+
description: string;
|
|
49
|
+
};
|
|
50
|
+
};
|
|
51
|
+
required: string[];
|
|
52
|
+
additionalProperties: boolean;
|
|
53
|
+
};
|
|
54
|
+
annotations: {
|
|
55
|
+
readOnlyHint: boolean;
|
|
56
|
+
openWorldHint: boolean;
|
|
57
|
+
idempotentHint: boolean;
|
|
58
|
+
destructiveHint: boolean;
|
|
59
|
+
};
|
|
60
|
+
};
|
|
61
|
+
export declare const handlePageToMarkdown: (rawInput: unknown) => Promise<PageToMarkdownResult>;
|
|
62
|
+
//# sourceMappingURL=page-to-markdown.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"page-to-markdown.d.ts","sourceRoot":"","sources":["../../src/tools/page-to-markdown.ts"],"names":[],"mappings":"AAMA,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,oCAAoC,CAAC;AACzE,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,eAAO,MAAM,yBAAyB;;;;;;;;;;;;;;;EAapC,CAAC;AAEH,MAAM,MAAM,mBAAmB,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,yBAAyB,CAAC,CAAC;AAG5E,MAAM,MAAM,oBAAoB,GAAG,cAAc,CAAC;AAElD,eAAO,MAAM,wBAAwB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CAgDpC,CAAC;AASF,eAAO,MAAM,oBAAoB,GAC/B,UAAU,OAAO,KAChB,OAAO,CAAC,oBAAoB,CAiC9B,CAAC"}
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
// page_to_markdown tool: fetches a URL and returns clean Markdown for LLM context.
|
|
2
|
+
// Uses @page2ai/core under the hood — inherits SSRF protection, 10MB size cap,
|
|
3
|
+
// 15s AbortController timeout, `.md`-first attempt (Mintlify convention), and
|
|
4
|
+
// static tab discovery (emits `### Tab: {label}` sections for docs sites).
|
|
5
|
+
import { fetchAndConvert, SsrfBlockedError, FetchProtectionError } from '@page2ai/core';
|
|
6
|
+
import { z } from 'zod';
|
|
7
|
+
export const pageToMarkdownInputSchema = z.object({
|
|
8
|
+
url: z.string().url().describe('Absolute HTTP or HTTPS URL of the web page to convert. Blocks private ranges, loopback, and cloud metadata endpoints.'),
|
|
9
|
+
include_images: z.boolean().optional().default(true).describe('Include image references () in the output. Default: true.'),
|
|
10
|
+
include_frontmatter: z.boolean().optional().default(true).describe('Prepend a YAML frontmatter block with title, source URL, capture timestamp, language, description, and canonical URL. Default: true.'),
|
|
11
|
+
timeout_ms: z.number().int().min(1000).max(60000).optional().default(15000).describe('Abort the fetch after N milliseconds. Range: 1000-60000. Default: 15000.'),
|
|
12
|
+
});
|
|
13
|
+
export const pageToMarkdownDefinition = {
|
|
14
|
+
name: 'page_to_markdown',
|
|
15
|
+
title: 'Convert Page to Markdown',
|
|
16
|
+
description: 'Fetch a web page URL and convert it to clean Markdown optimized for LLM context. ' +
|
|
17
|
+
'Preserves headings, code blocks (with language hints), links, and tables; strips ads, ' +
|
|
18
|
+
'navigation, and cookie banners. For Mintlify docs (docs.anthropic.com, OpenAI platform, ' +
|
|
19
|
+
'Vercel, Stripe, etc.) tries the `URL.md` convention first for cleanest output. ' +
|
|
20
|
+
'Discovers tab groups statically and emits each panel as a `### Tab: {label}` section ' +
|
|
21
|
+
'instead of concatenating (prevents Python+TypeScript examples merging into one broken block). ' +
|
|
22
|
+
'Zero external API calls — parses locally via linkedom.',
|
|
23
|
+
inputSchema: {
|
|
24
|
+
type: 'object',
|
|
25
|
+
properties: {
|
|
26
|
+
url: {
|
|
27
|
+
type: 'string',
|
|
28
|
+
format: 'uri',
|
|
29
|
+
description: 'Absolute HTTP or HTTPS URL of the web page to convert. Blocks private ranges, loopback, and cloud metadata endpoints.',
|
|
30
|
+
},
|
|
31
|
+
include_images: {
|
|
32
|
+
type: 'boolean',
|
|
33
|
+
default: true,
|
|
34
|
+
description: 'Include image references () in the output.',
|
|
35
|
+
},
|
|
36
|
+
include_frontmatter: {
|
|
37
|
+
type: 'boolean',
|
|
38
|
+
default: true,
|
|
39
|
+
description: 'Prepend a YAML frontmatter block with title, source URL, capture timestamp, language, description, and canonical URL.',
|
|
40
|
+
},
|
|
41
|
+
timeout_ms: {
|
|
42
|
+
type: 'integer',
|
|
43
|
+
minimum: 1000,
|
|
44
|
+
maximum: 60000,
|
|
45
|
+
default: 15000,
|
|
46
|
+
description: 'Abort the fetch after N milliseconds. Range: 1000-60000.',
|
|
47
|
+
},
|
|
48
|
+
},
|
|
49
|
+
required: ['url'],
|
|
50
|
+
additionalProperties: false,
|
|
51
|
+
},
|
|
52
|
+
annotations: {
|
|
53
|
+
readOnlyHint: true,
|
|
54
|
+
openWorldHint: true,
|
|
55
|
+
idempotentHint: false, // the fetched page may change between calls
|
|
56
|
+
destructiveHint: false,
|
|
57
|
+
},
|
|
58
|
+
};
|
|
59
|
+
// Wraps a plain-text error into an MCP tool result with isError:true so the
|
|
60
|
+
// LLM sees the error message instead of a JSON-RPC failure.
|
|
61
|
+
const errorResult = (message) => ({
|
|
62
|
+
isError: true,
|
|
63
|
+
content: [{ type: 'text', text: message }],
|
|
64
|
+
});
|
|
65
|
+
export const handlePageToMarkdown = async (rawInput) => {
|
|
66
|
+
let input;
|
|
67
|
+
try {
|
|
68
|
+
input = pageToMarkdownInputSchema.parse(rawInput);
|
|
69
|
+
}
|
|
70
|
+
catch (e) {
|
|
71
|
+
const msg = e instanceof z.ZodError
|
|
72
|
+
? e.errors.map((err) => `${err.path.join('.')}: ${err.message}`).join('; ')
|
|
73
|
+
: String(e);
|
|
74
|
+
return errorResult(`Invalid input: ${msg}`);
|
|
75
|
+
}
|
|
76
|
+
try {
|
|
77
|
+
const result = await fetchAndConvert(input.url, {
|
|
78
|
+
timeoutMs: input.timeout_ms,
|
|
79
|
+
includeImages: input.include_images,
|
|
80
|
+
includeFrontmatter: input.include_frontmatter,
|
|
81
|
+
});
|
|
82
|
+
return {
|
|
83
|
+
content: [{ type: 'text', text: result.markdown }],
|
|
84
|
+
};
|
|
85
|
+
}
|
|
86
|
+
catch (e) {
|
|
87
|
+
if (e instanceof SsrfBlockedError) {
|
|
88
|
+
return errorResult(`URL blocked by SSRF guard: ${e.message}`);
|
|
89
|
+
}
|
|
90
|
+
if (e instanceof FetchProtectionError) {
|
|
91
|
+
return errorResult(`Fetch failed: ${e.message}`);
|
|
92
|
+
}
|
|
93
|
+
if (e instanceof Error && e.name === 'AbortError') {
|
|
94
|
+
return errorResult(`Fetch timed out after ${input.timeout_ms}ms: ${input.url}`);
|
|
95
|
+
}
|
|
96
|
+
const msg = e instanceof Error ? e.message : String(e);
|
|
97
|
+
return errorResult(`Unexpected error: ${msg}`);
|
|
98
|
+
}
|
|
99
|
+
};
|
|
100
|
+
//# sourceMappingURL=page-to-markdown.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"page-to-markdown.js","sourceRoot":"","sources":["../../src/tools/page-to-markdown.ts"],"names":[],"mappings":"AAAA,mFAAmF;AACnF,+EAA+E;AAC/E,8EAA8E;AAC9E,2EAA2E;AAE3E,OAAO,EAAE,eAAe,EAAE,gBAAgB,EAAE,oBAAoB,EAAE,MAAM,eAAe,CAAC;AAExF,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,MAAM,CAAC,MAAM,yBAAyB,GAAG,CAAC,CAAC,MAAM,CAAC;IAChD,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,QAAQ,CAC5B,uHAAuH,CACxH;IACD,cAAc,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,QAAQ,CAC3D,sEAAsE,CACvE;IACD,mBAAmB,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,QAAQ,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,QAAQ,CAChE,sIAAsI,CACvI;IACD,UAAU,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,QAAQ,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,QAAQ,CAClF,0EAA0E,CAC3E;CACF,CAAC,CAAC;AAOH,MAAM,CAAC,MAAM,wBAAwB,GAAG;IACtC,IAAI,EAAE,kBAAkB;IACxB,KAAK,EAAE,0BAA0B;IACjC,WAAW,EACT,mFAAmF;QACnF,wFAAwF;QACxF,0FAA0F;QAC1F,iFAAiF;QACjF,uFAAuF;QACvF,gGAAgG;QAChG,wDAAwD;IAC1D,WAAW,EAAE;QACX,IAAI,EAAE,QAAiB;QACvB,UAAU,EAAE;YACV,GAAG,EAAE;gBACH,IAAI,EAAE,QAAiB;gBACvB,MAAM,EAAE,KAAK;gBACb,WAAW,EACT,uHAAuH;aAC1H;YACD,cAAc,EAAE;gBACd,IAAI,EAAE,SAAkB;gBACxB,OAAO,EAAE,IAAI;gBACb,WAAW,EAAE,uDAAuD;aACrE;YACD,mBAAmB,EAAE;gBACnB,IAAI,EAAE,SAAkB;gBACxB,OAAO,EAAE,IAAI;gBACb,WAAW,EACT,uHAAuH;aAC1H;YACD,UAAU,EAAE;gBACV,IAAI,EAAE,SAAkB;gBACxB,OAAO,EAAE,IAAI;gBACb,OAAO,EAAE,KAAK;gBACd,OAAO,EAAE,KAAK;gBACd,WAAW,EAAE,0DAA0D;aACxE;SACF;QACD,QAAQ,EAAE,CAAC,KAAK,CAAC;QACjB,oBAAoB,EAAE,KAAK;KAC5B;IACD,WAAW,EAAE;QACX,YAAY,EAAE,IAAI;QAClB,aAAa,EAAE,IAAI;QACnB,cAAc,EAAE,KAAK,EAAI,4CAA4C;QACrE,eAAe,EAAE,KAAK;KACvB;CACF,CAAC;AAEF,4EAA4E;AAC5E,4DAA4D;AAC5D,MAAM,WAAW,GAAG,CAAC,OAAe,EAAwB,EAAE,CAAC,CAAC;IAC9D,OAAO,EAAE,IAAI;IACb,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,MAAe,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC;CACpD,CAAC,CAAC;AAEH,MAAM,CAAC,MAAM,oBAAoB,GAAG,KAAK,EACvC,QAAiB,EACc,EAAE;IACjC,IAAI,KAA0B,CAAC;IAC/B,IAAI,CAAC;QACH,KAAK,GAAG,yBAAyB,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC;IACpD,CAAC;IAAC,OAAO,CAAC,EAAE,CAAC;QACX,MAAM,GAAG,GAAG,CAAC,YAAY,CAAC,CAAC,QAAQ;YACjC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,GAAG,CAAC,OAAO,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC;YAC3E,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;QACd,OAAO,WAAW,CAAC,kBAAkB,GAAG,EAAE,CAAC,CAAC;IAC9C,CAAC;IAED,IAAI,CAAC;QACH,MAAM,MAAM,GAAG,MAAM,eAAe,CAAC,KAAK,CAAC,GAAG,EAAE;YAC9C,SAAS,EAAE,KAAK,CAAC,UAAU;YAC3B,aAAa,EAAE,KAAK,CAAC,cAAc;YACnC,kBAAkB,EAAE,KAAK,CAAC,mBAAmB;SAC9C,CAAC,CAAC;QACH,OAAO;YACL,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,MAAe,EAAE,IAAI,EAAE,MAAM,CAAC,QAAQ,EAAE,CAAC;SAC5D,CAAC;IACJ,CAAC;IAAC,OAAO,CAAU,EAAE,CAAC;QACpB,IAAI,CAAC,YAAY,gBAAgB,EAAE,CAAC;YAClC,OAAO,WAAW,CAAC,8BAA8B,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC;QAChE,CAAC;QACD,IAAI,CAAC,YAAY,oBAAoB,EAAE,CAAC;YACtC,OAAO,WAAW,CAAC,iBAAiB,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC;QACnD,CAAC;QACD,IAAI,CAAC,YAAY,KAAK,IAAI,CAAC,CAAC,IAAI,KAAK,YAAY,EAAE,CAAC;YAClD,OAAO,WAAW,CAAC,yBAAyB,KAAK,CAAC,UAAU,OAAO,KAAK,CAAC,GAAG,EAAE,CAAC,CAAC;QAClF,CAAC;QACD,MAAM,GAAG,GAAG,CAAC,YAAY,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;QACvD,OAAO,WAAW,CAAC,qBAAqB,GAAG,EAAE,CAAC,CAAC;IACjD,CAAC;AACH,CAAC,CAAC"}
|
package/manifest.json
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
{
|
|
2
|
+
"manifest_version": "0.3",
|
|
3
|
+
"name": "page2ai-mcp",
|
|
4
|
+
"display_name": "Page2AI — Web to Markdown",
|
|
5
|
+
"version": "0.1.0",
|
|
6
|
+
"description": "Convert any web page or HTML string to clean Markdown for LLM context",
|
|
7
|
+
"long_description": "Page2AI MCP converts web pages to Markdown optimized for AI context windows. Extracts main content using DOM-position-aware dedup, preserves code blocks with language hints, converts tables and lists, and strips navigation, ads, and cookie banners. Works offline against local HTML strings or online against URLs. Zero external API calls in v0.1 — pure Node + linkedom.\n\nCompanion to the Page2AI Chrome extension. Shares the same extraction core (@page2ai/core) with the browser extension available at https://github.com/igorsaevets/page2ai-extension.",
|
|
8
|
+
"author": {
|
|
9
|
+
"name": "Igor Saevets",
|
|
10
|
+
"email": "igorsaevets@gmail.com",
|
|
11
|
+
"url": "https://github.com/igorsaevets"
|
|
12
|
+
},
|
|
13
|
+
"repository": "https://github.com/igorsaevets/page2ai-extension",
|
|
14
|
+
"homepage": "https://github.com/igorsaevets/page2ai-extension",
|
|
15
|
+
"documentation": "https://github.com/igorsaevets/page2ai-extension#readme",
|
|
16
|
+
"support": "https://github.com/igorsaevets/page2ai-extension/issues",
|
|
17
|
+
"icon": "icon.png",
|
|
18
|
+
"license": "MIT",
|
|
19
|
+
"keywords": [
|
|
20
|
+
"mcp",
|
|
21
|
+
"markdown",
|
|
22
|
+
"html-to-markdown",
|
|
23
|
+
"readability",
|
|
24
|
+
"web-scraping",
|
|
25
|
+
"llm",
|
|
26
|
+
"claude",
|
|
27
|
+
"rag"
|
|
28
|
+
],
|
|
29
|
+
"privacy_policies": [
|
|
30
|
+
"https://github.com/igorsaevets/page2ai-extension/blob/main/packages/mcp/PRIVACY.md"
|
|
31
|
+
],
|
|
32
|
+
"server": {
|
|
33
|
+
"type": "node",
|
|
34
|
+
"entry_point": "server/index.js",
|
|
35
|
+
"mcp_config": {
|
|
36
|
+
"command": "node",
|
|
37
|
+
"args": ["${__dirname}/server/index.js"]
|
|
38
|
+
}
|
|
39
|
+
},
|
|
40
|
+
"tools": [
|
|
41
|
+
{
|
|
42
|
+
"name": "page_to_markdown",
|
|
43
|
+
"title": "Convert Page to Markdown",
|
|
44
|
+
"description": "Fetch a web page URL and convert it to clean Markdown. Preserves headings, code blocks, links, and tables; strips ads, navigation, cookie banners, and footers.",
|
|
45
|
+
"readOnlyHint": true
|
|
46
|
+
}
|
|
47
|
+
],
|
|
48
|
+
"compatibility": {
|
|
49
|
+
"claude_desktop": ">=1.0.0",
|
|
50
|
+
"platforms": ["darwin", "win32", "linux"],
|
|
51
|
+
"runtimes": {
|
|
52
|
+
"node": ">=18.0.0"
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
}
|
package/package.json
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@page2ai/mcp",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "MCP server that converts any web page URL to clean Markdown for LLM context. Zero external API calls, SSRF-guarded, 100% local.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"page2ai-mcp": "./dist/index.js"
|
|
8
|
+
},
|
|
9
|
+
"main": "./dist/index.js",
|
|
10
|
+
"types": "./dist/index.d.ts",
|
|
11
|
+
"files": [
|
|
12
|
+
"dist",
|
|
13
|
+
"README.md",
|
|
14
|
+
"PRIVACY.md",
|
|
15
|
+
"LICENSE",
|
|
16
|
+
"manifest.json",
|
|
17
|
+
"server.json"
|
|
18
|
+
],
|
|
19
|
+
"engines": {
|
|
20
|
+
"node": ">=18.0.0"
|
|
21
|
+
},
|
|
22
|
+
"scripts": {
|
|
23
|
+
"build": "tsc -p tsconfig.build.json && node -e \"import('node:fs').then(({chmodSync})=>chmodSync('dist/index.js',0o755))\"",
|
|
24
|
+
"build:watch": "tsc -p tsconfig.build.json --watch",
|
|
25
|
+
"start": "node dist/index.js",
|
|
26
|
+
"test": "vitest run",
|
|
27
|
+
"typecheck": "tsc --noEmit",
|
|
28
|
+
"prepublishOnly": "npm run build && npm run test"
|
|
29
|
+
},
|
|
30
|
+
"publishConfig": {
|
|
31
|
+
"access": "public"
|
|
32
|
+
},
|
|
33
|
+
"dependencies": {
|
|
34
|
+
"@modelcontextprotocol/sdk": "^1.29.0",
|
|
35
|
+
"@page2ai/core": "^0.1.0",
|
|
36
|
+
"zod": "^3.24.0"
|
|
37
|
+
},
|
|
38
|
+
"devDependencies": {
|
|
39
|
+
"@types/node": "^22.10.0",
|
|
40
|
+
"typescript": "^5.7.2",
|
|
41
|
+
"vitest": "^3.2.7"
|
|
42
|
+
},
|
|
43
|
+
"keywords": [
|
|
44
|
+
"mcp",
|
|
45
|
+
"model-context-protocol",
|
|
46
|
+
"markdown",
|
|
47
|
+
"html-to-markdown",
|
|
48
|
+
"webpage-to-markdown",
|
|
49
|
+
"web-scraping",
|
|
50
|
+
"llm",
|
|
51
|
+
"ai",
|
|
52
|
+
"claude",
|
|
53
|
+
"cursor",
|
|
54
|
+
"windsurf",
|
|
55
|
+
"zed",
|
|
56
|
+
"rag",
|
|
57
|
+
"page2ai"
|
|
58
|
+
],
|
|
59
|
+
"author": "Igor Saevets, AI Expert and Entrepreneur",
|
|
60
|
+
"license": "MIT",
|
|
61
|
+
"homepage": "https://github.com/igorsaevets/page2ai-mcp",
|
|
62
|
+
"repository": {
|
|
63
|
+
"type": "git",
|
|
64
|
+
"url": "https://github.com/igorsaevets/page2ai-mcp.git"
|
|
65
|
+
},
|
|
66
|
+
"bugs": {
|
|
67
|
+
"url": "https://github.com/igorsaevets/page2ai-mcp/issues"
|
|
68
|
+
}
|
|
69
|
+
}
|
package/server.json
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
|
|
3
|
+
"name": "io.github.igorsaevets/page2ai-mcp",
|
|
4
|
+
"title": "Page2AI — Web to Markdown",
|
|
5
|
+
"description": "Convert any web page or HTML string to clean Markdown for LLM context. Preserves headings, code blocks, links, and tables; strips ads, navigation, cookie banners, and footers. Zero external API calls — runs entirely on your machine using linkedom.",
|
|
6
|
+
"repository": {
|
|
7
|
+
"url": "https://github.com/igorsaevets/page2ai-extension",
|
|
8
|
+
"source": "github"
|
|
9
|
+
},
|
|
10
|
+
"version": "0.1.0",
|
|
11
|
+
"packages": [
|
|
12
|
+
{
|
|
13
|
+
"registryType": "npm",
|
|
14
|
+
"identifier": "@page2ai/mcp",
|
|
15
|
+
"version": "0.1.0",
|
|
16
|
+
"transport": { "type": "stdio" }
|
|
17
|
+
}
|
|
18
|
+
]
|
|
19
|
+
}
|