html2llm 0.1.0 → 0.1.2

package/README.md

@@ -1,161 +1,191 @@
-# html2llm
-
-Convert HTML to **CSX (Compact S-Expression)** — a token-efficient format for feeding web content to LLMs.
-
-## Why CSX?
-
-Raw HTML is token-expensive: every `<div class="wrapper">` pays the cost of angle brackets, tag names, attribute syntax, and closing tags. CSX strips all noise and encodes the same structure in a format LLMs understand natively.
-
-| Format | Example |
-|--------|---------|
-| HTML | `<p class="lead">Hello <strong>world</strong></p>` |
-| Hiccup | `[:p {:class "lead"} "Hello " [:strong "world"]]` |
-| **CSX** | `(p.lead Hello (b world))` |
-
-## Install
-
-```bash
-npm install html2llm
-```
-
-*(Not yet published to npm — clone and build from source: `git clone && npm install && npm run build`)*
-
-## Usage
-
-### As a module
-
-```typescript
-import { htmlToCSX, urlToCSX } from "html2llm";
-
-// Convert HTML string to CSX
-const html = `<article class="post">
-  <h1>Title</h1>
-  <p>This is <strong>important</strong> content.</p>
-</article>`;
-
-console.log(htmlToCSX(html));
-// (article.post (h1 Title) (p "This is" (b important) content.))
-
-console.log(htmlToCSX(html, { minify: false }));
-// (article.post
-//   (h1 Title)
-//   (p "This is" (b important) content.))
-
-// Fetch a URL and convert its HTML to CSX
-const csx = await urlToCSX("https://example.com");
-console.log(csx);
-// (div (h1 "Example Domain") (p ...))
-
-// Pretty-printed
-const prettyCsx = await urlToCSX("https://example.com", { minify: false });
-
-// Use headless Chromium to bypass JS anti-bot walls (zhihu, etc.)
-const zhihuCsx = await urlToCSX("https://www.zhihu.com/question/...", {
-  headless: true,
-});
-```
-
-### As a CLI
-
-```bash
-# From file
-html2llm page.html
-
-# From URL (fetches and converts)
-html2llm https://example.com
-
-# From URL with headless browser (bypasses anti-bot JS challenges)
-html2llm https://www.zhihu.com/question/... --headless
-
-# From stdin
-curl -s https://example.com | html2llm
-
-# Pretty-printed
-html2llm page.html --pretty
-html2llm https://example.com --pretty
-```
-
-### As a web service (like Jina)
-
-Start the server:
-
-```bash
-# Development (hot reload)
-npm run dev:server
-
-# Production
-npm run build && npm start
-```
-
-Then curl any URL to get CSX:
-
-```bash
-# Fetch a page
-curl "localhost:3000/https://example.com"
-
-# Pretty-printed
-curl "localhost:3000/https://example.com?pretty"
-
-# Bypass JS anti-bot challenges with headless Chromium
-curl "localhost:3000/https://www.zhihu.com/question/...?headless"
-
-# Combine both
-curl "localhost:3000/https://www.zhihu.com/question/...?headless&pretty"
-
-# Auto-prepends https:// if you omit the protocol
-curl "localhost:3000/example.com"
-
-# Health check
-curl "localhost:3000/health"
-```
-
-The server:
-- Listens on `PORT` env (default 3000)
-- Fetches URLs server-side with a 15s timeout
-- Limits responses to 5 MB
-- Returns CSX as `text/plain; charset=utf-8`
-- Sets `X-Original-URL` response header
-
-### Docker
-
-```bash
-docker build -t html2llm .
-docker run -p 3000:3000 html2llm
-
-# With custom port
-docker run -p 8080:8080 -e PORT=8080 html2llm
-```
-
-## CSX Format Rules
-
-| Rule | Example |
-|------|---------|
-| Element | `(tag children...)` |
-| ID shorthand | `(div#main ...)` |
-| Class shorthand | `(div.card.dark ...)` |
-| Attributes | `(img src=photo.jpg alt=photo)` |
-| Quoted attr value | `(img alt="a nice photo")` |
-| Single-word text | `(p Hello)` |
-| Multi-word text | `(p "Hello world")` |
-| Boolean attribute | `(input type=checkbox checked)` |
-| Tag synonyms | `strong→b`, `em→i` |
-
-**Stripped:** `script`, `style`, `svg`, `noscript`, `iframe`, `link`, `meta`, HTML comments, all non-semantic attributes.
-
-**Flattened:** bare `div`/`span` with no attributes and a single child are hoisted.
-
-## Benchmark
-
-Token counts approximated at 4 characters/token (GPT-family).
-
-| URL | HTML tokens | CSX tokens | Reduction |
-|-----|-------------|------------|-----------|
-| [github.com](https://en.wikipedia.org/wiki/Large_language_model) |142589 | 33022 | 76.8% |
-| [news.ycombinator.com](https://news.ycombinator.com) | 8,619 | 5,793 | 32.8% |
-| [bbc.com/news](https://www.bbc.com/news) | 81,821 | 20,038 | 75.5% |
-
-> Results from running `npm run benchmark` against live URLs on 2026-05-15.
-
-## License
-
-MIT
+# html2llm
+
+Convert HTML to **CSX (Compact S-Expression)** — a token-efficient format for feeding web content to LLMs.
+
+## Why CSX?
+
+Raw HTML is token-expensive: every `<div class="wrapper">` pays the cost of angle brackets, tag names, attribute syntax, and closing tags. CSX strips all noise and encodes the same structure in a format LLMs understand natively.
+
+| Format | Example |
+|--------|---------|
+| HTML | `<p class="lead">Hello <strong>world</strong></p>` |
+| Hiccup | `[:p {:class "lead"} "Hello " [:strong "world"]]` |
+| **CSX** | `(p.lead Hello (b world))` |
+
+## Install
+
+```bash
+npm install html2llm
+```
+
+## Usage
+
+### As a module
+
+```typescript
+import { htmlToCSX, urlToCSX } from "html2llm";
+
+// Convert HTML string to CSX
+const html = `<article class="post">
+  <h1>Title</h1>
+  <p>This is <strong>important</strong> content.</p>
+</article>`;
+
+console.log(htmlToCSX(html));
+// (article.post (h1 Title) (p "This is" (b important) content.))
+
+console.log(htmlToCSX(html, { minify: false }));
+// (article.post
+//   (h1 Title)
+//   (p "This is" (b important) content.))
+
+// Fetch a URL and convert its HTML to CSX
+const csx = await urlToCSX("https://example.com");
+console.log(csx);
+// (div (h1 "Example Domain") (p ...))
+
+// Pretty-printed
+const prettyCsx = await urlToCSX("https://example.com", { minify: false });
+
+// Use headless Chromium to bypass JS anti-bot walls (zhihu, etc.)
+const zhihuCsx = await urlToCSX("https://www.zhihu.com/question/...", {
+  headless: true,
+});
+```
+
+### As a CLI
+
+```bash
+# From file
+html2llm page.html
+
+# From URL (fetches and converts)
+html2llm https://example.com
+
+# From URL with headless browser (bypasses anti-bot JS challenges)
+html2llm https://www.zhihu.com/question/... --headless
+
+# From stdin
+curl -s https://example.com | html2llm
+
+# Pretty-printed
+html2llm page.html --pretty
+html2llm https://example.com --pretty
+```
+
+### As a web service (like Jina)
+
+Start the server:
+
+```bash
+# Development (hot reload)
+npm run dev:server
+
+# Production
+npm run build && npm start
+```
+
+Then curl any URL to get CSX:
+
+```bash
+# Fetch a page
+curl "localhost:3000/https://example.com"
+
+# Pretty-printed
+curl "localhost:3000/https://example.com?pretty"
+
+# Bypass JS anti-bot challenges with headless Chromium
+curl "localhost:3000/https://www.zhihu.com/question/...?headless"
+
+# Combine both
+curl "localhost:3000/https://www.zhihu.com/question/...?headless&pretty"
+
+# Auto-prepends https:// if you omit the protocol
+curl "localhost:3000/example.com"
+
+# Health check
+curl "localhost:3000/health"
+```
+
+The server:
+- Listens on `PORT` env (default 3000)
+- Fetches URLs server-side with a 15s timeout
+- Limits responses to 5 MB
+- Returns CSX as `text/plain; charset=utf-8`
+- Sets `X-Original-URL` response header
+
+### Public instance
+
+A hosted instance is available at **html2llm.cyncyn.xyz** — use it like `r.jina.ai`:
+
+```bash
+# Fetch any webpage as CSX
+curl "https://html2llm.cyncyn.xyz/https://example.com"
+
+# Pretty-printed for readability
+curl "https://html2llm.cyncyn.xyz/https://example.com?pretty"
+
+# Feed to an LLM in one pipeline
+curl -s "https://html2llm.cyncyn.xyz/https://example.com" | llm "summarize this page"
+
+# Bypass JS anti-bot walls with headless Chromium
+curl "https://html2llm.cyncyn.xyz/https://www.zhihu.com/question/...?headless"
+
+# Omit https:// — auto-prepended
+curl "https://html2llm.cyncyn.xyz/example.com"
+
+# Health check
+curl "https://html2llm.cyncyn.xyz/health"
+```
+
+Query params:
+
+| Param | Effect |
+|---|---|
+| `?pretty` | Indented, human-readable output |
+| `?headless` | Use headless Chromium to bypass JS challenges (slower) |
+| `?pretty&headless` | Combine both |
+
+### Docker
+
+```bash
+docker build -t html2llm .
+docker run -p 3000:3000 html2llm
+
+# With custom port
+docker run -p 8080:8080 -e PORT=8080 html2llm
+```
+
+## CSX Format Rules
+
+| Rule | Example |
+|------|---------|
+| Element | `(tag children...)` |
+| ID shorthand | `(div#main ...)` |
+| Class shorthand | `(div.card.dark ...)` |
+| Attributes | `(img src=photo.jpg alt=photo)` |
+| Quoted attr value | `(img alt="a nice photo")` |
+| Single-word text | `(p Hello)` |
+| Multi-word text | `(p "Hello world")` |
+| Boolean attribute | `(input type=checkbox checked)` |
+| Tag synonyms | `strong→b`, `em→i` |
+
+**Stripped:** `script`, `style`, `svg`, `noscript`, `iframe`, `link`, `meta`, HTML comments, all non-semantic attributes.
+
+**Flattened:** bare `div`/`span` with no attributes and a single child are hoisted.
+
+## Benchmark
+
+Token counts approximated at 4 characters/token (GPT-family).
+
+| URL | HTML tokens | CSX tokens | Reduction |
+|-----|-------------|------------|-----------|
+| [github.com](https://en.wikipedia.org/wiki/Large_language_model) |142589 | 33022 | 76.8% |
+| [news.ycombinator.com](https://news.ycombinator.com) | 8,619 | 5,793 | 32.8% |
+| [bbc.com/news](https://www.bbc.com/news) | 81,821 | 20,038 | 75.5% |
+
+> Results from running `npm run benchmark` against live URLs on 2026-05-15.
+
+## License
+
+MIT

package/package.json

@@ -1,44 +1,44 @@
-{
-  "name": "html2llm",
-  "version": "0.1.0",
-  "description": "Convert HTML to CSX (Compact S-Expression) for token-efficient LLM context",
-  "type": "module",
-  "main": "dist/html2llm.js",
-  "types": "dist/html2llm.d.ts",
-  "exports": {
-    ".": "./dist/html2llm.js",
-    "./cli": "./dist/cli.js"
-  },
-  "files": [
-    "dist"
-  ],
-  "bin": {
-    "html2llm": "./dist/cli.js"
-  },
-  "scripts": {
-    "build": "tsc",
-    "test": "vitest run",
-    "test:watch": "vitest",
-    "benchmark": "tsx tests/benchmark.ts",
-    "dev": "tsx src/cli.ts",
-    "start": "node dist/server.js",
-    "dev:server": "tsx watch src/server.ts",
-    "postinstall": "npx playwright install chromium"
-  },
-  "dependencies": {
-    "@hono/node-server": "^1.0.0",
-    "cheerio": "^1.0.0",
-    "got-scraping": "^4.0.0",
-    "hono": "^4.0.0",
-    "playwright": "^1.52.0",
-    "playwright-extra": "^4.3.0",
-    "puppeteer-extra-plugin-stealth": "^2.11.0"
-  },
-  "devDependencies": {
-    "@types/node": "^20.0.0",
-    "axios": "^1.6.0",
-    "tsx": "^4.0.0",
-    "typescript": "^5.0.0",
-    "vitest": "^1.0.0"
-  }
-}
+{
+  "name": "html2llm",
+  "version": "0.1.2",
+  "description": "Convert HTML to CSX (Compact S-Expression) for token-efficient LLM context",
+  "type": "module",
+  "main": "dist/html2llm.js",
+  "types": "dist/html2llm.d.ts",
+  "exports": {
+    ".": "./dist/html2llm.js",
+    "./cli": "./dist/cli.js"
+  },
+  "files": [
+    "dist"
+  ],
+  "bin": {
+    "html2llm": "./dist/cli.js"
+  },
+  "scripts": {
+    "build": "tsc",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "benchmark": "tsx tests/benchmark.ts",
+    "dev": "tsx src/cli.ts",
+    "start": "node dist/server.js",
+    "dev:server": "tsx watch src/server.ts",
+    "postinstall": "npx playwright install chromium"
+  },
+  "dependencies": {
+    "@hono/node-server": "^1.0.0",
+    "cheerio": "^1.0.0",
+    "got-scraping": "^4.0.0",
+    "hono": "^4.0.0",
+    "playwright": "^1.52.0",
+    "playwright-extra": "^4.3.0",
+    "puppeteer-extra-plugin-stealth": "^2.11.0"
+  },
+  "devDependencies": {
+    "@types/node": "^20.0.0",
+    "axios": "^1.6.0",
+    "tsx": "^4.0.0",
+    "typescript": "^5.0.0",
+    "vitest": "^1.0.0"
+  }
+}

package/dist/browser.d.ts

@@ -1,14 +0,0 @@
-/**
- * Fetch HTML from a URL using a headless Chromium browser with stealth patches.
- * Handles JS challenge walls (e.g., zhihu) that got-scraping can't bypass.
- *
- * @param url - The URL to fetch
- * @param timeoutMs - Page load timeout (default 30s)
- * @returns The full HTML after JS execution
- */
-export declare function getHTML(url: string, timeoutMs?: number): Promise<string>;
-/**
- * Gracefully close the browser. Safe to call multiple times.
- * Registered on process exit signals.
- */
-export declare function closeBrowser(): Promise<void>;

package/dist/browser.js

@@ -1,97 +0,0 @@
-/**
- * Headless browser manager — singleton Chromium instance with stealth plugin.
- * Used as a fallback for sites that block got-scraping (JS challenge walls).
- */
-import { chromium } from "playwright-extra";
-import StealthPlugin from "puppeteer-extra-plugin-stealth";
-// Apply stealth plugin once at module load
-chromium.use(StealthPlugin());
-let browser = null;
-let launchPromise = null;
-let shuttingDown = false;
-const BROWSER_ARGS = [
-    "--no-sandbox",
-    "--disable-dev-shm-usage",
-    "--disable-gpu",
-    "--disable-setuid-sandbox",
-    "--single-process", // lighter weight for single-page fetches
-];
-async function getBrowser() {
-    if (shuttingDown) {
-        throw new Error("Browser is shutting down");
-    }
-    if (browser?.isConnected()) {
-        return browser;
-    }
-    // Avoid concurrent launches
-    if (!launchPromise) {
-        launchPromise = chromium.launch({
-            headless: true,
-            args: BROWSER_ARGS,
-        });
-    }
-    browser = await launchPromise;
-    launchPromise = null;
-    return browser;
-}
-async function getPage() {
-    const b = await getBrowser();
-    return b.newPage();
-}
-/**
- * Fetch HTML from a URL using a headless Chromium browser with stealth patches.
- * Handles JS challenge walls (e.g., zhihu) that got-scraping can't bypass.
- *
- * @param url - The URL to fetch
- * @param timeoutMs - Page load timeout (default 30s)
- * @returns The full HTML after JS execution
- */
-export async function getHTML(url, timeoutMs = 30_000) {
-    let page = null;
-    try {
-        page = await getPage();
-        // Stealth extras: realistic viewport
-        await page.setViewportSize({ width: 1366, height: 768 });
-        await page.goto(url, {
-            waitUntil: "domcontentloaded",
-            timeout: timeoutMs,
-        });
-        // Wait a bit for JS-rendered content to appear (anti-bot challenges, SPAs)
-        await page.waitForTimeout(3000);
-        const html = await page.content();
-        // 5 MB guard
-        if (html.length > 5_000_000) {
-            throw new Error("Response too large (>5 MB)");
-        }
-        return html;
-    }
-    finally {
-        if (page) {
-            await page.close().catch(() => { });
-        }
-    }
-}
-/**
- * Gracefully close the browser. Safe to call multiple times.
- * Registered on process exit signals.
- */
-export async function closeBrowser() {
-    shuttingDown = true;
-    if (browser) {
-        const b = browser;
-        browser = null;
-        await b.close().catch(() => { });
-    }
-}
-// Clean shutdown on exit
-process.on("exit", () => {
-    if (browser) {
-        browser.close().catch(() => { });
-    }
-});
-process.on("SIGTERM", () => {
-    closeBrowser().then(() => process.exit(0));
-});
-process.on("SIGINT", () => {
-    closeBrowser().then(() => process.exit(0));
-});

package/dist/cli.d.ts

@@ -1,2 +0,0 @@
-#!/usr/bin/env node
-export {};

package/dist/cli.js

@@ -1,45 +0,0 @@
-#!/usr/bin/env node
-import { readFileSync } from "fs";
-import { htmlToCSX, urlToCSX } from "./html2llm.js";
-import { closeBrowser } from "./browser.js";
-const args = process.argv.slice(2);
-const prettyFlag = args.includes("--pretty");
-const headlessFlag = args.includes("--headless");
-const fileArg = args.find((a) => !a.startsWith("--"));
-function isURL(arg) {
-    return arg.startsWith("http://") || arg.startsWith("https://");
-}
-async function main() {
-    const options = { minify: !prettyFlag, headless: headlessFlag };
-    if (fileArg && isURL(fileArg)) {
-        const result = await urlToCSX(fileArg, options);
-        console.log(result);
-        return;
-    }
-    let html;
-    if (fileArg) {
-        try {
-            html = readFileSync(fileArg, "utf-8");
-        }
-        catch (e) {
-            console.error(`Error reading file: ${fileArg}: ${e.message}`);
-            process.exit(1);
-        }
-    }
-    else {
-        // Read from stdin
-        const chunks = [];
-        for await (const chunk of process.stdin) {
-            chunks.push(chunk);
-        }
-        html = Buffer.concat(chunks).toString("utf-8");
-    }
-    const result = htmlToCSX(html, options);
-    console.log(result);
-}
-main()
-    .then(() => closeBrowser())
-    .catch((e) => {
-    console.error(e);
-    closeBrowser().then(() => process.exit(1));
-});

package/dist/html2llm.d.ts

@@ -1,12 +0,0 @@
-export interface CSXOptions {
-    minify?: boolean;
-    /** Use headless Chromium to bypass JS anti-bot challenges (slower but stronger) */
-    headless?: boolean;
-}
-export declare function htmlToCSX(html: string, options?: CSXOptions): string;
-/**
- * Fetch a URL and convert its HTML to CSX.
- * Uses got-scraping under the hood for TLS fingerprint spoofing
- * and browser-like headers to avoid 403 blocks.
- */
-export declare function urlToCSX(url: string, options?: CSXOptions): Promise<string>;

package/dist/html2llm.js

@@ -1,152 +0,0 @@
-import * as cheerio from "cheerio";
-import { gotScraping } from "got-scraping";
-import { getHTML } from "./browser.js";
-// Tags to remove entirely (with all children)
-const STRIP_TAGS = new Set([
-    "script", "style", "svg", "noscript", "iframe", "link", "meta",
-]);
-// Tags to rename (synonym collapse)
-const TAG_MAP = {
-    strong: "b",
-    em: "i",
-};
-// Attributes to keep (beyond id/class which become shorthand)
-const KEEP_ATTRS = new Set(["href", "src", "alt", "title", "type"]);
-// Boolean attributes to keep
-const BOOL_ATTRS = new Set([
-    "checked", "disabled", "readonly", "required", "selected", "open",
-]);
-function isDataAttr(name) {
-    return name.startsWith("data-");
-}
-function normalizeTag(tag) {
-    return TAG_MAP[tag] ?? tag;
-}
-function getShorthand(el) {
-    const attribs = el.attribs ?? {};
-    const id = attribs["id"] ? `#${attribs["id"]}` : "";
-    const classes = attribs["class"]
-        ? attribs["class"].trim().split(/\s+/).map((c) => `.${c}`).join("")
-        : "";
-    return id + classes;
-}
-function getAttrs(el) {
-    const attribs = el.attribs ?? {};
-    const parts = [];
-    for (const [name, value] of Object.entries(attribs)) {
-        if (name === "id" || name === "class")
-            continue; // handled in shorthand
-        if (BOOL_ATTRS.has(name) && (value === "" || value === name || value === "true")) {
-            parts.push(name);
-        }
-        else if (KEEP_ATTRS.has(name) || isDataAttr(name)) {
-            const needsQuotes = value.includes(" ") || value.includes("=") || value.includes("(") || value.includes(")") || value.includes("#");
-            const escapedValue = needsQuotes ? value.replace(/"/g, '\\"') : value;
-            parts.push(needsQuotes ? `${name}="${escapedValue}"` : `${name}=${value}`);
-        }
-    }
-    return parts.join(" ");
-}
-function serializeText(text) {
-    const trimmed = text.replace(/\s+/g, " ").trim();
-    if (!trimmed)
-        return "";
-    const escaped = trimmed.replace(/"/g, '\\"');
-    const needsQuotes = escaped.includes(" ") || escaped.includes("(") || escaped.includes(")") || trimmed.includes('"');
-    return needsQuotes ? `"${escaped}"` : escaped;
-}
-function convertNode(node, $, minify, depth = 0) {
-    // Strip HTML comments
-    if (node.type === "comment")
-        return null;
-    // Text nodes
-    if (node.type === "text") {
-        const text = node.data ?? "";
-        return serializeText(text) || null;
-    }
-    // Only process element (tag) nodes
-    if (node.type !== "tag")
-        return null;
-    const el = node;
-    const rawTag = el.name.toLowerCase();
-    // Strip noise tags entirely
-    if (STRIP_TAGS.has(rawTag))
-        return null;
-    const tag = normalizeTag(rawTag);
-    const shorthand = getShorthand(el);
-    const attrs = getAttrs(el);
-    // Check if this node is a candidate for flattening BEFORE recursing children
-    const isFlattenable = (rawTag === "div" || rawTag === "span") && !shorthand && !attrs;
-    const children = $(el)
-        .contents()
-        .toArray()
-        .map((child) => convertNode(child, $, minify, depth + 1))
-        .filter((s) => s !== null && s.length > 0);
-    // Structural flattening: skip bare div/span with exactly one child
-    if (isFlattenable && children.length === 1) {
-        if (minify)
-            return children[0];
-        // In pretty mode, depth matters — re-render at current depth (not depth+1)
-        const childNodes = $(el).contents().toArray();
-        for (const child of childNodes) {
-            const rendered = convertNode(child, $, minify, depth);
-            if (rendered)
-                return rendered;
-        }
-    }
-    const tagPart = tag + shorthand;
-    const attrPart = attrs ? ` ${attrs}` : "";
-    if (children.length === 0) {
-        return `(${tagPart}${attrPart})`;
-    }
-    if (minify) {
-        return `(${tagPart}${attrPart} ${children.join(" ")})`;
-    }
-    else {
-        const indent = "  ".repeat(depth + 1);
-        const childStr = children.map((c) => `${indent}${c}`).join("\n");
-        return `(${tagPart}${attrPart}\n${childStr})`;
-    }
-}
-export function htmlToCSX(html, options) {
-    const minify = options?.minify ?? true;
-    let $;
-    try {
-        $ = cheerio.load(html);
-    }
-    catch {
-        $ = cheerio.load("");
-    }
-    const results = [];
-    $("body")
-        .contents()
-        .toArray()
-        .forEach((node) => {
-        const result = convertNode(node, $, minify, 0);
-        if (result)
-            results.push(result);
-    });
-    return results.join(minify ? " " : "\n");
-}
-/**
- * Fetch a URL and convert its HTML to CSX.
- * Uses got-scraping under the hood for TLS fingerprint spoofing
- * and browser-like headers to avoid 403 blocks.
- */
-export async function urlToCSX(url, options) {
-    const headless = options?.headless ?? false;
-    if (headless) {
-        const html = await getHTML(url);
-        return htmlToCSX(html, options);
-    }
-    const res = await gotScraping({
-        url,
-        responseType: "text",
-        timeout: { request: 15_000 },
-        retry: { limit: 1 },
-    });
-    if (res.body.length > 5_000_000) {
-        throw new Error("Response too large (>5 MB)");
-    }
-    return htmlToCSX(res.body, options);
-}

package/dist/server.d.ts

@@ -1,2 +0,0 @@
-import { Hono } from "hono";
-export declare const app: Hono<import("hono/types").BlankEnv, import("hono/types").BlankSchema, "/">;

package/dist/server.js

@@ -1,58 +0,0 @@
-import { Hono } from "hono";
-import { serve } from "@hono/node-server";
-import { urlToCSX } from "./html2llm.js";
-export const app = new Hono();
-// Health check
-app.get("/health", (c) => {
-    return c.json({ status: "ok" });
-});
-// Main endpoint: URL-as-path (like Jina)
-// GET /https://example.com/page?pretty
-app.get("/*", async (c) => {
-    const rawUrl = c.req.path.slice(1); // remove leading "/"
-    if (!rawUrl) {
-        return c.text("html2llm — Convert any webpage to CSX for LLM context.\n\n" +
-            "Usage: GET /https://example.com/page\n\n" +
-            "Query params:\n" +
-            "  ?pretty   — pretty-printed output for debugging\n" +
-            "  ?headless — use headless Chromium to bypass JS anti-bot challenges\n", 400);
-    }
-    // Auto-prepend https:// if no protocol specified
-    let url;
-    try {
-        url = rawUrl.startsWith("http://") || rawUrl.startsWith("https://")
-            ? rawUrl
-            : `https://${rawUrl}`;
-        new URL(url); // validate
-    }
-    catch {
-        return c.text(`Invalid URL: ${rawUrl}`, 400);
-    }
-    const pretty = c.req.query("pretty") !== undefined;
-    const headless = c.req.query("headless") !== undefined;
-    try {
-        const csx = await urlToCSX(url, { minify: !pretty, headless });
-        return new Response(csx, {
-            status: 200,
-            headers: {
-                "Content-Type": "text/plain; charset=utf-8",
-                "X-Original-URL": url,
-            },
-        });
-    }
-    catch (err) {
-        const name = err.name;
-        if (name === "TimeoutError") {
-            return c.text(`Request timed out (15s): ${url}`, 504);
-        }
-        if (name === "HTTPError") {
-            const httpErr = err;
-            return c.text(`Failed to fetch URL: ${url} (HTTP ${httpErr.response.statusCode})`, 502);
-        }
-        return c.text(`Failed to fetch URL: ${url}\n${err.message}`, 502);
-    }
-});
-// ---- Entry point ----
-const port = parseInt(process.env.PORT || "3000");
-serve({ fetch: app.fetch, port });
-console.log(`html2llm server running on http://localhost:${port}`);