cdp-toolkit 2.5.0 → 2.6.0

package/CHANGELOG.md

@@ -5,6 +5,25 @@ All notable changes to cdp-toolkit are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.6.0] - 2026-09-15
+
+**Bun is now a first-class, CI-tested runtime, and the MCP server can serve streamable HTTP.** The package has always declared `engines.bun`, but nothing proved the server actually booted under Bun — or under Node, for that matter. This release pins both: the server and CLI run under Bun ≥ 1.1 or Node ≥ 22, importing `dist/mcp.js` / `dist/cli.js` as libraries is now silent (no server start, no CLI usage dump), and under Bun the same `mcp.js` entry can serve the full MCP surface over loopback HTTP with `--transport streamable-http` for hosts that launch it as an HTTP service instead of a stdio child.
+
+### Added
+
+- **`--transport streamable-http` (Bun only).** `mcp.js` accepts `--transport <stdio|streamable-http>` (default `stdio`, byte-identical behavior to previous releases), plus `--port` (default 3000; `0` picks an OS port, announced on the stderr listening line) and `--host` (default 127.0.0.1). The HTTP path is built on the MCP SDK 2.0's Web-standard `createMcpHandler` behind the same `buildServer()` factory stdio uses, so both transports can never drift apart; loopback host-header and origin validation are enforced in front of it. Launched under Node with `--transport streamable-http`, the server fails fast with a clear stderr message rather than half-serving. The listening URL goes to stderr only — stdout remains the stdio JSON-RPC channel. (`src/mcp.ts`)
+- **CI `runtime` job that gates the badge.** Boots the BUILT `dist/mcp.js` under both `node` (22, matching the `engines.node` floor) and `bun` on every push/PR: silent-import checks for `dist/mcp.js` and `dist/cli.js` under both runtimes, a hand-rolled stdio JSON-RPC `initialize` + `tools/list` handshake under both runtimes (no SDK client, so the wire is what's asserted), and a streamable-http `initialize` round-trip under Bun. No browser involved, so every failure is a real regression. New `bun run test:runtime` runs the same suite locally (`test/runtime-entry.test.ts`, 7 tests). (`.github/workflows/ci.yml`)
+
+### Fixed
+
+- **Importing the entries is now silent under both runtimes.** `dist/mcp.js` and `dist/cli.js` used to start serving / print usage the moment any bundler or `require.resolve`-style host imported them. Both files now gate on a direct-run check (`realpathSync(process.argv[1])` against `import.meta.url`, which handles bin symlinks under npm/bunx and works identically under Node and Bun, ESM-only): only direct/bin invocation runs the server or CLI; library import is side-effect free. `buildServer` is exported from `src/mcp.ts` for embedders. (`src/mcp.ts`, `src/cli.ts`)
+
+## [2.5.1] - 2026-09-12
+
+### Fixed
+
+- **`extract_page { prompt: "schematron" }` no longer sends `response_format`.** Measured against a locally served open-weight Schematron-8B on `mlx_vlm.server` (llguidance-constrained decoding): with `response_format` present, a real CNN zone came back as `{"stories": []}` in 6 completion tokens; the IDENTICAL messages with `response_format` omitted returned valid, schema-conformant JSON with 5 stories in 285 tokens. A 3-link toy page worked either way, so the collapse only shows on real markup. The schema already rides inline in the schematron prompt, so constrained decoding on top of it was redundant and, on real pages, actively harmful; `response_format` is now omitted entirely (never sent as `null`) in `schematron` mode, unchanged in `html` mode. Since nothing constrains the shape server-side anymore in `schematron` mode, `extractPage` now also rejects a syntactically valid but non-object JSON response (array/null/primitive) with the same invalid-JSON error shape. One new unit test pins `html` mode still sending `response_format`; the existing schematron wire-shape test now asserts its absence (`test/extract.test.ts`).
+
 ## [2.5.0] - 2026-09-12
 
 ### Added

package/README.md

@@ -87,7 +87,7 @@ npx -y --package cdp-toolkit cdp navigate_page --target index:0 --url https://ex
 # …or from a clone: `bun run src/cli.ts <tool> …`
 ```
 
-**Requirements:** Node ≥ 22 **or** Bun ≥ 1.1: `npx -y cdp-toolkit` and `bunx -y cdp-toolkit` both work (the published bins are plain Node ESM). Chrome/Chromium with `--remote-debugging-port=9222`. Smoke-check the port: `curl -s http://127.0.0.1:9222/json/version`.
+**Requirements:** Node ≥ 22 **or** Bun ≥ 1.1: `npx -y cdp-toolkit` and `bunx -y cdp-toolkit` both work (the published bins are plain Node ESM, and the server + CLI are CI-tested under both runtimes). Chrome/Chromium with `--remote-debugging-port=9222`. Smoke-check the port: `curl -s http://127.0.0.1:9222/json/version`. Under Bun only, `--transport streamable-http` (with `--port`/`--host`, default `127.0.0.1:3000`) serves the same MCP surface over loopback HTTP instead of stdio.
 
 ### MCP client setup
 
@@ -254,7 +254,7 @@ That shape is deliberate. The MCP **2026-07-28** revision puts lazy discovery on
 With no `name` it returns the grouped catalog of everything this server can run. The whole 48-tool surface:
 
 ```
-cdp-toolkit 2.5.0 · browser=chrome · 48 tools available, 49 in tools/list (CDP_TOOL_PROFILE=full)
+cdp-toolkit 2.5.1 · browser=chrome · 48 tools available, 49 in tools/list (CDP_TOOL_PROFILE=full)
 [listed] core (12): list_pages, new_page, close_page, select_page, navigate_page, wait_for, take_snapshot, click, fill, type_text, evaluate_script, take_screenshot
 [listed] input (9): hover, drag, scroll, dispatch_mouse, press_key, fill_form, upload_file, focus_emulation, click_focus_gated
 [listed] cookies (3): list_cookies, set_cookie, delete_cookies
@@ -314,7 +314,7 @@ Measured 2026-09-01: Claude Code 2.1.258 and Codex CLI 0.147.0 both connect and
 {"tool": "extract_page", "arguments": {"target": "index:0", "schema": {"type": "object", "properties": {"title": {"type": "string", "description": "the page title, exactly as printed"}}}}}
 ```
 
-**Prompt shape (`prompt`, `CDP_EXTRACT_PROMPT`).** By default (`prompt:"html"`) the cleaned HTML is the *only* user message and your schema rides in `response_format` alone — which is what the **hosted** Schematron API expects, since it injects the schema into the prompt server-side. Point `baseUrl` at a **locally served open-weight Schematron** (e.g. an MLX `schematron8B`) and that breaks: the open weights were fine-tuned on a prompt with the schema *inside* the user message, and a bare-HTML prompt returns garbage even under constrained decoding. `prompt:"schematron"` sends the model card's own messages instead — a `You are a helpful assistant` system message plus a user message carrying the compact `JSON.stringify`'d schema, then the HTML, then `MAKE SURE ITS VALID JSON.` That adds roughly the stringified schema's length to the prompt (`maxChars` caps the **HTML** only; the true count still comes back in `usage.promptTokens`). `response_format` is sent in both modes. The per-call arg wins over `CDP_EXTRACT_PROMPT`, and anything but `html`/`schematron` is rejected before any page or network work.
+**Prompt shape (`prompt`, `CDP_EXTRACT_PROMPT`).** By default (`prompt:"html"`) the cleaned HTML is the *only* user message and your schema rides in `response_format` alone — which is what the **hosted** Schematron API expects, since it injects the schema into the prompt server-side. Point `baseUrl` at a **locally served open-weight Schematron** (e.g. an MLX `schematron8B`) and that breaks: the open weights were fine-tuned on a prompt with the schema *inside* the user message, and a bare-HTML prompt returns garbage even under constrained decoding. `prompt:"schematron"` sends the model card's own messages instead — a `You are a helpful assistant` system message plus a user message carrying the compact `JSON.stringify`'d schema, then the HTML, then `MAKE SURE ITS VALID JSON.` That adds roughly the stringified schema's length to the prompt (`maxChars` caps the **HTML** only; the true count still comes back in `usage.promptTokens`). `schematron` mode sends **no `response_format`** at all — the schema already lives in the prompt, and the client validates only that the response parses as JSON. (Measured 2026-09-12 against a locally served open-weight Schematron-8B under llguidance-constrained decoding: identical messages returned `{"stories": []}` in 6 completion tokens with `response_format` present, versus a valid, schema-conformant 5-story result in 285 tokens with it omitted — constrained decoding on top of an already-inlined schema is redundant and, on real markup, harmful.) The per-call arg wins over `CDP_EXTRACT_PROMPT`, and anything but `html`/`schematron` is rejected before any page or network work.
 
 Works on both backends (Chrome and Firefox); see the `extract_page` row in [The tools](#the-tools-29-parity--19-superset--48) for the full parameter set.
 

package/dist/cli.js

@@ -8187,12 +8187,13 @@ function buildExtractionMessages(mode, html, schema) {
 }
 async function callExtractionEndpoint(opts) {
   const url = `${opts.baseUrl.replace(/\/+$/, "")}/chat/completions`;
+  const promptMode = opts.prompt ?? "html";
   const body = JSON.stringify({
     model: opts.model,
-    messages: buildExtractionMessages(opts.prompt ?? "html", opts.html, opts.schema),
+    messages: buildExtractionMessages(promptMode, opts.html, opts.schema),
     temperature: 0,
     max_tokens: COMPLESION_TOKENS,
-    response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } }
+    ...promptMode === "schematron" ? {} : { response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } } }
   });
   const headers = { "content-type": "application/json" };
   if (opts.apiKey)
@@ -8286,6 +8287,9 @@ async function extractPage(driver, args) {
     } catch (e) {
       throw new CdpError(`extract_page: endpoint returned invalid JSON (${e.message}${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`);
     }
+    if (data === null || typeof data !== "object" || Array.isArray(data)) {
+      throw new CdpError(`extract_page: endpoint returned invalid JSON (expected a JSON object${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`);
+    }
     const result = {
       data,
       model: respondedModel,
@@ -11225,6 +11229,18 @@ Warning: could not find the ${browser} binary at its expected location. The alia
 }
 
 // src/cli.ts
+import { realpathSync } from "node:fs";
+import { fileURLToPath as fileURLToPath2 } from "node:url";
+function isDirectRun() {
+  const entry = process.argv[1];
+  if (!entry)
+    return false;
+  try {
+    return fileURLToPath2(import.meta.url) === realpathSync(entry);
+  } catch {
+    return false;
+  }
+}
 var USAGE = `cdp-toolkit: raw single-target CDP, 29-tool chrome-devtools-mcp parity, plus a Firefox backend over WebDriver BiDi.
 
 Usage:
@@ -11462,9 +11478,11 @@ async function main() {
     }
   });
 }
-main().then((code) => process.exit(code)).catch((err) => {
-  const message = err instanceof Error ? err.message : String(err);
-  process.stderr.write(`${JSON.stringify({ error: message })}
+if (isDirectRun()) {
+  main().then((code) => process.exit(code)).catch((err) => {
+    const message = err instanceof Error ? err.message : String(err);
+    process.stderr.write(`${JSON.stringify({ error: message })}
 `);
-  process.exit(1);
-});
+    process.exit(1);
+  });
+}

package/dist/index.js

@@ -5589,12 +5589,13 @@ function buildExtractionMessages(mode, html, schema) {
 }
 async function callExtractionEndpoint(opts) {
   const url = `${opts.baseUrl.replace(/\/+$/, "")}/chat/completions`;
+  const promptMode = opts.prompt ?? "html";
   const body = JSON.stringify({
     model: opts.model,
-    messages: buildExtractionMessages(opts.prompt ?? "html", opts.html, opts.schema),
+    messages: buildExtractionMessages(promptMode, opts.html, opts.schema),
     temperature: 0,
     max_tokens: COMPLESION_TOKENS,
-    response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } }
+    ...promptMode === "schematron" ? {} : { response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } } }
   });
   const headers = { "content-type": "application/json" };
   if (opts.apiKey)
@@ -5688,6 +5689,9 @@ async function extractPage(driver, args) {
     } catch (e) {
       throw new CdpError(`extract_page: endpoint returned invalid JSON (${e.message}${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`);
     }
+    if (data === null || typeof data !== "object" || Array.isArray(data)) {
+      throw new CdpError(`extract_page: endpoint returned invalid JSON (expected a JSON object${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`);
+    }
     const result = {
       data,
       model: respondedModel,

package/dist/mcp.js

@@ -3697,6 +3697,15 @@ var init_driver2 = __esm(() => {
 // src/mcp.ts
 import { Server } from "@modelcontextprotocol/server";
 import { serveStdio } from "@modelcontextprotocol/server/stdio";
+import {
+  createMcpHandler,
+  hostHeaderValidationResponse,
+  localhostAllowedHostnames,
+  localhostAllowedOrigins,
+  originValidationResponse
+} from "@modelcontextprotocol/server";
+import { realpathSync } from "node:fs";
+import { fileURLToPath as fileURLToPath2 } from "node:url";
 
 // src/cdp/driver.ts
 init_client();
@@ -8191,12 +8200,13 @@ function buildExtractionMessages(mode, html, schema) {
 }
 async function callExtractionEndpoint(opts) {
   const url = `${opts.baseUrl.replace(/\/+$/, "")}/chat/completions`;
+  const promptMode = opts.prompt ?? "html";
   const body = JSON.stringify({
     model: opts.model,
-    messages: buildExtractionMessages(opts.prompt ?? "html", opts.html, opts.schema),
+    messages: buildExtractionMessages(promptMode, opts.html, opts.schema),
     temperature: 0,
     max_tokens: COMPLESION_TOKENS,
-    response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } }
+    ...promptMode === "schematron" ? {} : { response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } } }
   });
   const headers = { "content-type": "application/json" };
   if (opts.apiKey)
@@ -8290,6 +8300,9 @@ async function extractPage(driver, args) {
     } catch (e) {
       throw new CdpError(`extract_page: endpoint returned invalid JSON (${e.message}${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`);
     }
+    if (data === null || typeof data !== "object" || Array.isArray(data)) {
+      throw new CdpError(`extract_page: endpoint returned invalid JSON (expected a JSON object${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`);
+    }
     const result = {
       data,
       model: respondedModel,
@@ -11248,7 +11261,7 @@ var TOOL_DOCS = {
     params: {}
   },
   extract_page: {
-    description: "Structured extraction: take the target page's HTML, clean it, and send it to an OpenAI-compatible /chat/completions endpoint along with your JSON Schema, getting back schema-conformant JSON instead of prose. THE DEFAULT ENDPOINT IS LOOPBACK - llm-ferry at http://127.0.0.1:8090/v1 with model 'schematron' - so page content stays ON THIS MACHINE unless the operator overrides baseUrl (or sets CDP_EXTRACT_BASE_URL). Configuration: CDP_EXTRACT_BASE_URL (endpoint override), CDP_EXTRACT_MODEL (model override), CDP_EXTRACT_API_KEY (bearer token; never logged and REDACTED from every error), CDP_EXTRACT_TIMEOUT_MS, CDP_EXTRACT_MAX_CHARS, CDP_EXTRACT_PROMPT. PROMPT SHAPE: by default only the cleaned HTML is sent and the schema rides in response_format, which is what the HOSTED Schematron API expects (it injects the schema server-side); set prompt:'schematron' when baseUrl points at a LOCALLY SERVED open-weight Schematron (e.g. an MLX schematron8B), because that model was fine-tuned on a prompt with the schema INSIDE the user message and an HTML-only prompt returns garbage from it even under constrained decoding. THE SCHEMA-DESCRIPTION RULE: this is prompt-driven extraction, and the extraction INSTRUCTIONS live in the schema's property descriptions - write them like prompts ('the job title, exactly as printed', 'ISO date, not 'yesterday'). A schema whose properties carry no descriptions is REJECTED up front with an error saying so, because it would produce garbage extraction with no hint why. Source: 'dom' (live DOM, the rendered, JS-executed page) is the default; 'served' (the raw server response before rendering) currently returns a not-implemented error rather than guessing. Size guard: cleaned HTML over maxChars (default 300000) fails with html_too_large naming the actual size - shrink the payload with 'selector' to scope to a subtree, clean:'aggressive' to strip harder (standard already drops scripts/styles/hidden), or a larger maxChars if you truly need it all. Endpoint failures surface as upstream_error (with status and body, API key redacted) or timeout; the response never contains the key. A completion the endpoint cut off at its 8192-token output cap fails naming finish_reason=length and the fix (narrow with 'selector', or ask the schema for less) - it is NOT reported as invalid JSON, because a truncated tail is a size problem, not a model one. COST IS ALWAYS VISIBLE: every successful call reports the endpoint's usage token counts inline, so a slow/bloated extraction is visible in the answer itself, not discovered on a bill later.",
+    description: "Structured extraction: take the target page's HTML, clean it, and send it to an OpenAI-compatible /chat/completions endpoint along with your JSON Schema, getting back schema-conformant JSON instead of prose. THE DEFAULT ENDPOINT IS LOOPBACK - llm-ferry at http://127.0.0.1:8090/v1 with model 'schematron' - so page content stays ON THIS MACHINE unless the operator overrides baseUrl (or sets CDP_EXTRACT_BASE_URL). Configuration: CDP_EXTRACT_BASE_URL (endpoint override), CDP_EXTRACT_MODEL (model override), CDP_EXTRACT_API_KEY (bearer token; never logged and REDACTED from every error), CDP_EXTRACT_TIMEOUT_MS, CDP_EXTRACT_MAX_CHARS, CDP_EXTRACT_PROMPT. PROMPT SHAPE: by default only the cleaned HTML is sent and the schema rides in response_format, which is what the HOSTED Schematron API expects (it injects the schema server-side); set prompt:'schematron' when baseUrl points at a LOCALLY SERVED open-weight Schematron (e.g. an MLX schematron8B), because that model was fine-tuned on a prompt with the schema INSIDE the user message and an HTML-only prompt returns garbage from it even under constrained decoding. In 'schematron' mode NO response_format is sent at all - the schema already lives in the prompt, and the client validates only that the response parses as JSON. THE SCHEMA-DESCRIPTION RULE: this is prompt-driven extraction, and the extraction INSTRUCTIONS live in the schema's property descriptions - write them like prompts ('the job title, exactly as printed', 'ISO date, not 'yesterday'). A schema whose properties carry no descriptions is REJECTED up front with an error saying so, because it would produce garbage extraction with no hint why. Source: 'dom' (live DOM, the rendered, JS-executed page) is the default; 'served' (the raw server response before rendering) currently returns a not-implemented error rather than guessing. Size guard: cleaned HTML over maxChars (default 300000) fails with html_too_large naming the actual size - shrink the payload with 'selector' to scope to a subtree, clean:'aggressive' to strip harder (standard already drops scripts/styles/hidden), or a larger maxChars if you truly need it all. Endpoint failures surface as upstream_error (with status and body, API key redacted) or timeout; the response never contains the key. A completion the endpoint cut off at its 8192-token output cap fails naming finish_reason=length and the fix (narrow with 'selector', or ask the schema for less) - it is NOT reported as invalid JSON, because a truncated tail is a size problem, not a model one. COST IS ALWAYS VISIBLE: every successful call reports the endpoint's usage token counts inline, so a slow/bloated extraction is visible in the answer itself, not discovered on a bill later.",
     params: {
       lease: "Lease token from claim_page. Omit for a tab this process holds; required for a tab held by another process or one claimed explicitly. Auto-acquired under CDP_REQUIRE_LEASE (then pass target, not lease). See server instructions.",
       target: "Page selector: 'active' (default), 'index:N', 'url:<substr>', 'title:<substr>', 'label:<name>', or a 32-hex '<targetId>'. Grammar: see server instructions.",
@@ -11258,7 +11271,7 @@ var TOOL_DOCS = {
       clean: "HTML cleaning level before the payload is measured and sent. 'standard' (default) strips scripts, styles, hidden elements and other non-content noise. 'aggressive' additionally collapses whitespace, drops nav/footer/boilerplate - use when html_too_large bites or the extractor chases junk. 'none' sends the raw HTML; almost never what you want.",
       maxChars: "Character cap on the cleaned HTML payload. Default 300000 (also settable via CDP_EXTRACT_MAX_CHARS). Exceeding it fails with html_too_large reporting actual vs cap - it does NOT silently truncate, because a half page extracted as though it were whole is a wrong answer that looks right.",
       timeoutMs: "Total budget in ms for the endpoint call. Default 90000, hard max 300000 (CDP_EXTRACT_TIMEOUT_MS overrides the default). Expiry fails with a timeout error, never a hang.",
-      prompt: "Prompt shape sent upstream. 'html' (default, also CDP_EXTRACT_PROMPT=html) sends the cleaned HTML as the only user message and lets response_format carry the schema - correct for the hosted Schematron API, which injects the schema server-side. 'schematron' sends the open-weight Schematron model card's own messages: a 'You are a helpful assistant' system message plus a user message with the compact JSON-stringified schema inline, then the HTML, then 'MAKE SURE ITS VALID JSON.' - use it when baseUrl points at a locally served open-weight Schematron. It adds roughly the stringified schema's length to the prompt (maxChars caps the HTML only); response_format is sent in both modes. The arg wins over CDP_EXTRACT_PROMPT, and any other value is rejected up front.",
+      prompt: "Prompt shape sent upstream. 'html' (default, also CDP_EXTRACT_PROMPT=html) sends the cleaned HTML as the only user message and lets response_format carry the schema - correct for the hosted Schematron API, which injects the schema server-side. 'schematron' sends the open-weight Schematron model card's own messages: a 'You are a helpful assistant' system message plus a user message with the compact JSON-stringified schema inline, then the HTML, then 'MAKE SURE ITS VALID JSON.' - use it when baseUrl points at a locally served open-weight Schematron. It adds roughly the stringified schema's length to the prompt (maxChars caps the HTML only); 'schematron' mode sends NO response_format - the schema lives in the prompt and the client validates only that the response parses as JSON. The arg wins over CDP_EXTRACT_PROMPT, and any other value is rejected up front.",
       model: "Model name for the endpoint call. Default 'schematron' on the loopback llm-ferry; override per-call or via CDP_EXTRACT_MODEL.",
       baseUrl: "OpenAI-compatible endpoint base URL (the server appends /chat/completions). DEFAULT IS http://127.0.0.1:8090/v1 - loopback llm-ferry, page content never leaves the machine. Setting this (or CDP_EXTRACT_BASE_URL) to a remote host is an operator decision about where page content may travel; the response never echoes the URL with credentials.",
       savePath: "Write the extracted JSON here instead of returning it inline: the response then carries the file path and usage counts only, keeping large payloads out of the transcript. An absolute path (starting with /) is used as-is; a relative path resolves under the artifact dir (/tmp/cdp-toolkit); missing parents are created."
@@ -11267,9 +11280,48 @@ var TOOL_DOCS = {
 };
 
 // src/version.ts
-var VERSION = "2.5.0";
+var VERSION = "2.6.0";
 
 // src/mcp.ts
+function isDirectRun() {
+  const entry = process.argv[1];
+  if (!entry)
+    return false;
+  try {
+    return fileURLToPath2(import.meta.url) === realpathSync(entry);
+  } catch {
+    return false;
+  }
+}
+function parseServeOptions(argv) {
+  const out = { transport: "stdio", port: 3000, host: "127.0.0.1" };
+  const transportIdx = argv.indexOf("--transport");
+  if (transportIdx !== -1) {
+    const value = argv[transportIdx + 1];
+    if (value !== "stdio" && value !== "streamable-http") {
+      throw new Error(`unknown --transport '${value ?? ""}': expected 'stdio' or 'streamable-http'`);
+    }
+    out.transport = value;
+  }
+  const portIdx = argv.indexOf("--port");
+  if (portIdx !== -1) {
+    const raw = argv[portIdx + 1];
+    const port = Number(raw);
+    if (raw === undefined || !/^\d+$/.test(raw) || port < 0 || port > 65535) {
+      throw new Error(`invalid --port '${raw ?? ""}': expected an integer 0-65535`);
+    }
+    out.port = port;
+  }
+  const hostIdx = argv.indexOf("--host");
+  if (hostIdx !== -1) {
+    const raw = argv[hostIdx + 1];
+    if (raw === undefined || raw === "") {
+      throw new Error(`invalid --host '${raw ?? ""}': expected a hostname or address`);
+    }
+    out.host = raw;
+  }
+  return out;
+}
 var { kind: BROWSER, endpoint: FIREFOX_ENDPOINT } = resolveBackend(process.argv.slice(2));
 var AVAILABILITY = toolAvailability(BROWSER);
 var AVAILABLE_NAMES = new Set(AVAILABILITY.available);
@@ -11403,25 +11455,64 @@ function buildServer() {
   });
   return server;
 }
-auditCoverage();
-markLongLivedProcess();
-var handle = serveStdio(({ era }) => {
-  console.error(`[cdp-toolkit] connection pinned to the ${era} protocol era`);
-  return buildServer();
-}, { onerror: (err) => console.error(`[cdp-toolkit] stdio: ${err.message}`) });
-console.error(`[cdp-toolkit] MCP server v${VERSION} ready, browser=${BROWSER}${FIREFOX_ENDPOINT ? ` (attach ${FIREFOX_ENDPOINT})` : ""}, ${AVAILABILITY.available.length} tools available, ${LISTING.length} listed (CDP_TOOL_PROFILE=${PROFILE.label}), CDP_BASE=${BASE}`);
-var shuttingDown = false;
-async function shutdown() {
-  if (shuttingDown)
-    return;
-  shuttingDown = true;
-  await disposeFirefoxSession();
-  await disposeBrowserSession();
-  await handle.close().catch(() => {
+async function run2(argv) {
+  const serve = parseServeOptions(argv);
+  auditCoverage();
+  markLongLivedProcess();
+  let stdioHandle;
+  let httpHandler;
+  let listener;
+  let shuttingDown = false;
+  async function shutdown() {
+    if (shuttingDown)
+      return;
+    shuttingDown = true;
+    await disposeFirefoxSession();
+    await disposeBrowserSession();
+    if (stdioHandle)
+      await stdioHandle.close().catch(() => {
+        return;
+      });
+    if (httpHandler)
+      await httpHandler.close().catch(() => {
+        return;
+      });
+    if (listener)
+      listener.stop();
+    process.exit(0);
+  }
+  process.on("SIGINT", () => void shutdown());
+  process.on("SIGTERM", () => void shutdown());
+  if (serve.transport === "stdio") {
+    stdioHandle = serveStdio(({ era }) => {
+      console.error(`[cdp-toolkit] connection pinned to the ${era} protocol era`);
+      return buildServer();
+    }, { onerror: (err) => console.error(`[cdp-toolkit] stdio: ${err.message}`) });
+    process.stdin.on("close", () => void shutdown());
+    console.error(`[cdp-toolkit] MCP server v${VERSION} ready, browser=${BROWSER}${FIREFOX_ENDPOINT ? ` (attach ${FIREFOX_ENDPOINT})` : ""}, ${AVAILABILITY.available.length} tools available, ${LISTING.length} listed (CDP_TOOL_PROFILE=${PROFILE.label}), CDP_BASE=${BASE}`);
     return;
+  }
+  if (typeof Bun === "undefined") {
+    console.error("[cdp-toolkit] --transport streamable-http needs Bun's HTTP listener (Bun.serve). Re-run with bun, or use the default stdio transport.");
+    process.exit(1);
+  }
+  httpHandler = createMcpHandler(({ era }) => {
+    console.error(`[cdp-toolkit] connection pinned to the ${era} protocol era`);
+    return buildServer();
+  }, { onerror: (err) => console.error(`[cdp-toolkit] http: ${err.message}`) });
+  listener = Bun.serve({
+    port: serve.port,
+    hostname: serve.host,
+    fetch: (req) => hostHeaderValidationResponse(req, localhostAllowedHostnames()) ?? originValidationResponse(req, localhostAllowedOrigins()) ?? httpHandler.fetch(req)
   });
-  process.exit(0);
+  console.error(`[cdp-toolkit] MCP server v${VERSION} listening on streamable-http port ${listener.port} (http://${serve.host}:${listener.port}/), browser=${BROWSER}${FIREFOX_ENDPOINT ? ` (attach ${FIREFOX_ENDPOINT})` : ""}, ${AVAILABILITY.available.length} tools available, ${LISTING.length} listed (CDP_TOOL_PROFILE=${PROFILE.label}), CDP_BASE=${BASE}`);
 }
-process.on("SIGINT", () => void shutdown());
-process.on("SIGTERM", () => void shutdown());
-process.stdin.on("close", () => void shutdown());
+if (isDirectRun()) {
+  run2(process.argv.slice(2)).catch((err) => {
+    console.error(`[cdp-toolkit] ${err instanceof Error ? err.message : String(err)}`);
+    process.exit(1);
+  });
+}
+export {
+  buildServer
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cdp-toolkit",
-  "version": "2.5.0",
+  "version": "2.6.0",
   "description": "A lightweight chrome-devtools-mcp alternative: an MCP server + CLI that drives the tabs you name, one target per call over one direct WebSocket, with a bounded timeout on every call, so a stuck page can't wedge your agent or force a /mcp restart. Chrome over CDP and Firefox over WebDriver BiDi, with tools a backend can't run absent from tools/list rather than throwing mid-task. 48 tools including built-in network mocking, a cookie group, tab leases with human-activity detection, real HTML5 drag-and-drop, raw mouse/scroll dispatch, downloads, permission grants, and tab-to-video screen recording so several agents can drive one browser without stealing each other's tabs or a human's. No Puppeteer, no all-target fan-out.",
   "type": "module",
   "license": "MIT",
@@ -73,6 +73,7 @@
     "mcp": "bun run ./src/mcp.ts",
     "typecheck": "tsc --noEmit",
     "test": "bun test",
+    "test:runtime": "bun test test/runtime-entry.test.ts",
     "smoke": "bun run ./test/smoke.ts",
     "mock:smoke": "bun run ./test/mock-smoke.ts",
     "screencast:smoke": "bun run ./test/screencast-smoke.ts",

package/src/cli.ts

@@ -29,6 +29,26 @@ import { FIREFOX_TOOLS } from "./firefox-tools.ts";
 import { leaseFromArgs, withLeaseScope } from "./leases.ts";
 import { MANIFEST } from "./manifest.ts";
 import { runInstaller } from "./install/wizard.ts";
+import { realpathSync } from "node:fs";
+import { fileURLToPath } from "node:url";
+
+/**
+ * True when this module is the process's entry script — the ONLY condition under which main()
+ * runs. Works under both node and bun, ESM included: compare this module's real path with the
+ * realpath of argv[1] (realpath resolves the bunx/npm bin symlink and any /tmp-style symlinked
+ * directory). Importing this module (tests, embedders) leaves argv[1] pointing elsewhere, so the
+ * import stays silent — no tool dispatch, no usage message.
+ */
+function isDirectRun(): boolean {
+  const entry = process.argv[1];
+  if (!entry) return false;
+  try {
+    return fileURLToPath(import.meta.url) === realpathSync(entry);
+  } catch {
+    // argv[1] may not exist as a file (eval strings, wrappers); never treat that as direct.
+    return false;
+  }
+}
 
 const USAGE = `cdp-toolkit: raw single-target CDP, 29-tool chrome-devtools-mcp parity, plus a Firefox backend over WebDriver BiDi.
 
@@ -330,10 +350,12 @@ async function main(): Promise<number> {
   });
 }
 
-main()
-  .then((code) => process.exit(code))
-  .catch((err: unknown) => {
-    const message = err instanceof Error ? err.message : String(err);
-    process.stderr.write(`${JSON.stringify({ error: message })}\n`);
-    process.exit(1);
-  });
+if (isDirectRun()) {
+  main()
+    .then((code) => process.exit(code))
+    .catch((err: unknown) => {
+      const message = err instanceof Error ? err.message : String(err);
+      process.stderr.write(`${JSON.stringify({ error: message })}\n`);
+      process.exit(1);
+    });
+}

package/src/mcp.ts

@@ -1,14 +1,20 @@
 #!/usr/bin/env bun
 /**
- * cdp-toolkit MCP server (stdio).
+ * cdp-toolkit MCP server (stdio by default, streamable-http on request).
  *
  * Exposes the toolkit's raw-CDP tools to any MCP client (Claude Code, etc.) over
- * the standard stdio transport. It does NOT connect to Chrome at startup: each
- * tool call lazily opens a single-target CDP connection (with its own timeout),
- * so the server loads cleanly even when Chrome isn't running; individual calls
- * then fail with a clear error if the browser is unreachable.
+ * the standard stdio transport, or — with `--transport streamable-http` — over
+ * an HTTP/SSE listener served by Bun. It does NOT connect to Chrome at startup:
+ * each tool call lazily opens a single-target CDP connection (with its own
+ * timeout), so the server loads cleanly even when Chrome isn't running;
+ * individual calls then fail with a clear error if the browser is unreachable.
+ *
+ * Importing this module is side-effect-free: the server starts only when the
+ * file is the directly executed entry (bin invocation or `bun run src/mcp.ts`),
+ * never when imported (see isDirectRun below).
  *
  * Launch: `bunx -y cdp-toolkit`  (or `bun run src/mcp.ts` from a checkout)
+ *   HTTP:  `bun run src/mcp.ts --transport streamable-http [--port 3000] [--host 127.0.0.1]`
  * Config:  CDP_BASE (default http://127.0.0.1:9222), CDP_TIMEOUT_MS, CDP_ARTIFACT_DIR,
  *          CDP_TOOL_PROFILE (`full` — the default — advertises every group; `core` advertises
  *          just the 12 everyday tools; or a comma-separated group list, e.g. `core,network`.
@@ -21,6 +27,15 @@
 import { Server } from "@modelcontextprotocol/server";
 import type { Tool } from "@modelcontextprotocol/server";
 import { serveStdio } from "@modelcontextprotocol/server/stdio";
+import {
+  createMcpHandler,
+  hostHeaderValidationResponse,
+  localhostAllowedHostnames,
+  localhostAllowedOrigins,
+  originValidationResponse,
+} from "@modelcontextprotocol/server";
+import { realpathSync } from "node:fs";
+import { fileURLToPath } from "node:url";
 import { TOOLS, TOOL_NAMES, BASE } from "./index.ts";
 import { MANIFEST } from "./manifest.ts";
 import { resolveBackend, getOrCreateFirefoxSession, disposeFirefoxSession } from "./backend.ts";
@@ -32,6 +47,84 @@ import { resolveProfile, TOOL_GROUP, TOOL_GROUPS, GROUP_TOOLS } from "./toolGrou
 import { TOOL_DOCS } from "./toolDocs.ts";
 import { VERSION } from "./version.ts";
 
+// Bun.serve is the HTTP transport's listener: a Bun-only global, typed here (not via bun-types,
+// which CONTRACT.md keeps out of the dependency set) in the narrow shape this file uses. The code
+// below never touches it unless --transport streamable-http was requested, and checks
+// `typeof Bun === "undefined"` first so running under plain node fails with a clear message
+// instead of a ReferenceError. The declaration emits nothing — at runtime the identifier resolves
+// to the real global when one exists.
+declare const Bun: {
+  serve(opts: {
+    port: number;
+    hostname: string;
+    fetch(req: Request): Promise<Response> | Response;
+  }): { stop(): void; port: number };
+};
+
+/**
+ * True when this module is the process's entry script — the ONLY condition under which the
+ * server starts. Works under both node and bun, ESM included: compare this module's real path
+ * with the realpath of argv[1]. realpathSync resolves the bunx/npm bin symlink (argv[1] is the
+ * symlink, the module URL is its target) and any /tmp-style symlinked directory; when argv[1] is
+ * absent (e.g. `node -e`/REPL importing this file) the answer is simply "not direct". A plain
+ * `import(".../mcp.js")` from another module leaves argv[1] pointing at that other entry, so the
+ * import stays silent.
+ */
+function isDirectRun(): boolean {
+  const entry = process.argv[1];
+  if (!entry) return false;
+  try {
+    return fileURLToPath(import.meta.url) === realpathSync(entry);
+  } catch {
+    // argv[1] may not exist as a file (eval strings, wrappers); never treat that as direct.
+    return false;
+  }
+}
+
+/** What --transport/--port/--host asked for. stdio is the default and stays byte-for-byte unchanged. */
+interface ServeOptions {
+  transport: "stdio" | "streamable-http";
+  port: number;
+  host: string;
+}
+
+/**
+ * Parse the serving flags from the same argv resolveBackend reads (it scans only --browser and
+ * --connect, so these extra tokens were already invisible to it). An unknown --transport value or
+ * a non-port --port is a hard configuration error, mirroring resolveBackend's loud failures.
+ */
+function parseServeOptions(argv: readonly string[]): ServeOptions {
+  const out: ServeOptions = { transport: "stdio", port: 3000, host: "127.0.0.1" };
+  const transportIdx = argv.indexOf("--transport");
+  if (transportIdx !== -1) {
+    const value = argv[transportIdx + 1];
+    if (value !== "stdio" && value !== "streamable-http") {
+      throw new Error(`unknown --transport '${value ?? ""}': expected 'stdio' or 'streamable-http'`);
+    }
+    out.transport = value;
+  }
+  const portIdx = argv.indexOf("--port");
+  if (portIdx !== -1) {
+    const raw = argv[portIdx + 1];
+    const port = Number(raw);
+    // 0 is valid HERE (unlike backend.ts's dial-out endpoints): for a listener it means "let the
+    // OS pick a free port", and the resolved port is then announced on stderr (see run()).
+    if (raw === undefined || !/^\d+$/.test(raw) || port < 0 || port > 65_535) {
+      throw new Error(`invalid --port '${raw ?? ""}': expected an integer 0-65535`);
+    }
+    out.port = port;
+  }
+  const hostIdx = argv.indexOf("--host");
+  if (hostIdx !== -1) {
+    const raw = argv[hostIdx + 1];
+    if (raw === undefined || raw === "") {
+      throw new Error(`invalid --host '${raw ?? ""}': expected a hostname or address`);
+    }
+    out.host = raw;
+  }
+  return out;
+}
+
 // Backend + (Firefox only) attach endpoint are read once at startup (MCP has no per-call notion
 // of backend): --browser flag / CDP_BROWSER env, else "chrome" (zero behavior change for existing
 // users/configs); --connect flag / CDP_FIREFOX_ENDPOINT env selects Firefox ATTACH mode and
@@ -163,7 +256,7 @@ const LIST_CACHE_HINT = { ttlMs: 3_600_000, cacheScope: "public" } as const;
  * our manifest is plain JSON Schema (src/manifest.ts). McpServer also cannot express
  * "hidden tools remain callable by name", which is the whole point of CDP_TOOL_PROFILE.
  */
-function buildServer(): Server {
+export function buildServer(): Server {
   const server = new Server(
     { name: "cdp-toolkit", version: VERSION },
     {
@@ -236,47 +329,105 @@ function buildServer(): Server {
   return server;
 }
 
-auditCoverage();
-// This process is the long-lived MCP server, which is what makes strict mode
-// (CDP_REQUIRE_LEASE) safe to honor here and unsafe in cli.ts. See requireLease.
-markLongLivedProcess();
+/**
+ * Start serving: stdio (the default) or the streamable-http listener. Only ever called from the
+ * isDirectRun guard below — importing this module must stay silent.
+ */
+async function run(argv: string[]): Promise<void> {
+  const serve = parseServeOptions(argv);
+  auditCoverage();
+  // This process is the long-lived MCP server, which is what makes strict mode
+  // (CDP_REQUIRE_LEASE) safe to honor here and unsafe in cli.ts. See requireLease.
+  markLongLivedProcess();
 
-// serveStdio owns the era decision for the connection: a modern opening (server/discover
-// with the per-request _meta envelope) pins a 2026-07-28 instance, a 2025-era `initialize`
-// pins a legacy one. legacy:'serve' (the default) is deliberate — 2025-era clients keep the
-// initialize handshake they expect, so this upgrade is invisible to them.
-const handle = serveStdio(
-  ({ era }) => {
-    console.error(`[cdp-toolkit] connection pinned to the ${era} protocol era`);
-    return buildServer();
-  },
-  { onerror: (err) => console.error(`[cdp-toolkit] stdio: ${err.message}`) },
-);
-console.error(
-  `[cdp-toolkit] MCP server v${VERSION} ready, browser=${BROWSER}${FIREFOX_ENDPOINT ? ` (attach ${FIREFOX_ENDPOINT})` : ""}, ${AVAILABILITY.available.length} tools available, ${LISTING.length} listed (CDP_TOOL_PROFILE=${PROFILE.label}), CDP_BASE=${BASE}`,
-);
+  // Per-transport state the shutdown path closes (only one is ever set).
+  let stdioHandle: Awaited<ReturnType<typeof serveStdio>> | undefined;
+  let httpHandler: ReturnType<typeof createMcpHandler> | undefined;
+  let listener: { stop(): void; port: number } | undefined;
+
+  // LAUNCH-mode Firefox owns a real OS process (see bidi/launch.ts): it must be reaped on every
+  // shutdown path, not just a clean exit. ATTACH-mode Firefox (CDP_FIREFOX_ENDPOINT / --connect)
+  // owns no process here — disposeFirefoxSession() below only ends its BiDi session (freeing
+  // Firefox's single session slot) and closes the socket, leaving the user's browser running.
+  // SIGINT/SIGTERM cover ctrl-C and a supervising client killing the server; in stdio mode the
+  // stdin 'close' event covers the normal MCP shutdown (the client closes the pipe). All are
+  // idempotent through disposeFirefoxSession(), and a no-op entirely when Firefox was never
+  // launched/attached.
+  let shuttingDown = false;
+  async function shutdown(): Promise<void> {
+    if (shuttingDown) return;
+    shuttingDown = true;
+    await disposeFirefoxSession();
+    // The standing browser-endpoint connection behind wait_for_download / grant_permissions (1.8.0
+    // Track P3). Unlike Firefox it owns no OS process, so process.exit would collect it anyway; it is
+    // closed explicitly so shutdown does not depend on that, and is a no-op when neither tool ran.
+    await disposeBrowserSession();
+    // Close the active transport last, so a tool call still in flight over it cannot outlive the
+    // browser resources it was driving. For http this also stops accepting new connections.
+    if (stdioHandle) await stdioHandle.close().catch(() => undefined);
+    if (httpHandler) await httpHandler.close().catch(() => undefined);
+    if (listener) listener.stop();
+    process.exit(0);
+  }
+  process.on("SIGINT", () => void shutdown());
+  process.on("SIGTERM", () => void shutdown());
 
-// LAUNCH-mode Firefox owns a real OS process (see bidi/launch.ts): it must be reaped on every
-// shutdown path, not just a clean exit. ATTACH-mode Firefox (CDP_FIREFOX_ENDPOINT / --connect)
-// owns no process here — disposeFirefoxSession() below only ends its BiDi session (freeing
-// Firefox's single session slot) and closes the socket, leaving the user's browser running.
-// SIGINT/SIGTERM cover ctrl-C and a supervising client killing the server; stdin 'close' covers
-// the normal MCP shutdown (the client closes the stdio pipe). All three are idempotent through
-// disposeFirefoxSession(), and a no-op entirely when Firefox was never launched/attached.
-let shuttingDown = false;
-async function shutdown(): Promise<void> {
-  if (shuttingDown) return;
-  shuttingDown = true;
-  await disposeFirefoxSession();
-  // The standing browser-endpoint connection behind wait_for_download / grant_permissions (1.8.0
-  // Track P3). Unlike Firefox it owns no OS process, so process.exit would collect it anyway; it is
-  // closed explicitly so shutdown does not depend on that, and is a no-op when neither tool ran.
-  await disposeBrowserSession();
-  // Close the stdio connection (pinned instance + transport) last, so a tool call still
-  // in flight over it cannot outlive the browser resources it was driving.
-  await handle.close().catch(() => undefined);
-  process.exit(0);
+  if (serve.transport === "stdio") {
+    // serveStdio owns the era decision for the connection: a modern opening (server/discover
+    // with the per-request _meta envelope) pins a 2026-07-28 instance, a 2025-era `initialize`
+    // pins a legacy one. legacy:'serve' (the default) is deliberate — 2025-era clients keep the
+    // initialize handshake they expect, so this upgrade is invisible to them.
+    stdioHandle = serveStdio(
+      ({ era }) => {
+        console.error(`[cdp-toolkit] connection pinned to the ${era} protocol era`);
+        return buildServer();
+      },
+      { onerror: (err) => console.error(`[cdp-toolkit] stdio: ${err.message}`) },
+    );
+    process.stdin.on("close", () => void shutdown());
+    console.error(
+      `[cdp-toolkit] MCP server v${VERSION} ready, browser=${BROWSER}${FIREFOX_ENDPOINT ? ` (attach ${FIREFOX_ENDPOINT})` : ""}, ${AVAILABILITY.available.length} tools available, ${LISTING.length} listed (CDP_TOOL_PROFILE=${PROFILE.label}), CDP_BASE=${BASE}`,
+    );
+    return;
+  }
+
+  // streamable-http: Bun.serve + the SDK's web-standard handler. DNS-rebinding protection comes
+  // from the SDK's host/origin validators composed in front (each returns an error Response for a
+  // disallowed request, undefined to fall through); the localhost allowlists match the default
+  // loopback bind. Everything diagnostic goes to stderr — stdout stays the stdio JSON-RPC channel
+  // only, and even here it must carry nothing (an http client never reads it).
+  if (typeof Bun === "undefined") {
+    console.error(
+      "[cdp-toolkit] --transport streamable-http needs Bun's HTTP listener (Bun.serve). Re-run with bun, or use the default stdio transport.",
+    );
+    process.exit(1);
+  }
+  httpHandler = createMcpHandler(
+    ({ era }) => {
+      console.error(`[cdp-toolkit] connection pinned to the ${era} protocol era`);
+      return buildServer();
+    },
+    { onerror: (err) => console.error(`[cdp-toolkit] http: ${err.message}`) },
+  );
+  listener = Bun.serve({
+    port: serve.port,
+    hostname: serve.host,
+    fetch: (req) =>
+      hostHeaderValidationResponse(req, localhostAllowedHostnames()) ??
+      originValidationResponse(req, localhostAllowedOrigins()) ??
+      httpHandler!.fetch(req),
+  });
+  // The resolved port is announced as a bare `port N` token BEFORE the URL text, so a stderr
+  // scanner finds the real listening port first (matters under --port 0, where N is the
+  // OS-assigned one and the only place it is knowable).
+  console.error(
+    `[cdp-toolkit] MCP server v${VERSION} listening on streamable-http port ${listener.port} (http://${serve.host}:${listener.port}/), browser=${BROWSER}${FIREFOX_ENDPOINT ? ` (attach ${FIREFOX_ENDPOINT})` : ""}, ${AVAILABILITY.available.length} tools available, ${LISTING.length} listed (CDP_TOOL_PROFILE=${PROFILE.label}), CDP_BASE=${BASE}`,
+  );
+}
+
+if (isDirectRun()) {
+  run(process.argv.slice(2)).catch((err: unknown) => {
+    console.error(`[cdp-toolkit] ${err instanceof Error ? err.message : String(err)}`);
+    process.exit(1);
+  });
 }
-process.on("SIGINT", () => void shutdown());
-process.on("SIGTERM", () => void shutdown());
-process.stdin.on("close", () => void shutdown());

package/src/toolDocs.ts

@@ -486,7 +486,7 @@ export const TOOL_DOCS: Record<string, ToolDoc> = {
     "params": {}
   },
   "extract_page": {
-    "description": "Structured extraction: take the target page's HTML, clean it, and send it to an OpenAI-compatible /chat/completions endpoint along with your JSON Schema, getting back schema-conformant JSON instead of prose. THE DEFAULT ENDPOINT IS LOOPBACK - llm-ferry at http://127.0.0.1:8090/v1 with model 'schematron' - so page content stays ON THIS MACHINE unless the operator overrides baseUrl (or sets CDP_EXTRACT_BASE_URL). Configuration: CDP_EXTRACT_BASE_URL (endpoint override), CDP_EXTRACT_MODEL (model override), CDP_EXTRACT_API_KEY (bearer token; never logged and REDACTED from every error), CDP_EXTRACT_TIMEOUT_MS, CDP_EXTRACT_MAX_CHARS, CDP_EXTRACT_PROMPT. PROMPT SHAPE: by default only the cleaned HTML is sent and the schema rides in response_format, which is what the HOSTED Schematron API expects (it injects the schema server-side); set prompt:'schematron' when baseUrl points at a LOCALLY SERVED open-weight Schematron (e.g. an MLX schematron8B), because that model was fine-tuned on a prompt with the schema INSIDE the user message and an HTML-only prompt returns garbage from it even under constrained decoding. THE SCHEMA-DESCRIPTION RULE: this is prompt-driven extraction, and the extraction INSTRUCTIONS live in the schema's property descriptions - write them like prompts ('the job title, exactly as printed', 'ISO date, not 'yesterday'). A schema whose properties carry no descriptions is REJECTED up front with an error saying so, because it would produce garbage extraction with no hint why. Source: 'dom' (live DOM, the rendered, JS-executed page) is the default; 'served' (the raw server response before rendering) currently returns a not-implemented error rather than guessing. Size guard: cleaned HTML over maxChars (default 300000) fails with html_too_large naming the actual size - shrink the payload with 'selector' to scope to a subtree, clean:'aggressive' to strip harder (standard already drops scripts/styles/hidden), or a larger maxChars if you truly need it all. Endpoint failures surface as upstream_error (with status and body, API key redacted) or timeout; the response never contains the key. A completion the endpoint cut off at its 8192-token output cap fails naming finish_reason=length and the fix (narrow with 'selector', or ask the schema for less) - it is NOT reported as invalid JSON, because a truncated tail is a size problem, not a model one. COST IS ALWAYS VISIBLE: every successful call reports the endpoint's usage token counts inline, so a slow/bloated extraction is visible in the answer itself, not discovered on a bill later.",
+    "description": "Structured extraction: take the target page's HTML, clean it, and send it to an OpenAI-compatible /chat/completions endpoint along with your JSON Schema, getting back schema-conformant JSON instead of prose. THE DEFAULT ENDPOINT IS LOOPBACK - llm-ferry at http://127.0.0.1:8090/v1 with model 'schematron' - so page content stays ON THIS MACHINE unless the operator overrides baseUrl (or sets CDP_EXTRACT_BASE_URL). Configuration: CDP_EXTRACT_BASE_URL (endpoint override), CDP_EXTRACT_MODEL (model override), CDP_EXTRACT_API_KEY (bearer token; never logged and REDACTED from every error), CDP_EXTRACT_TIMEOUT_MS, CDP_EXTRACT_MAX_CHARS, CDP_EXTRACT_PROMPT. PROMPT SHAPE: by default only the cleaned HTML is sent and the schema rides in response_format, which is what the HOSTED Schematron API expects (it injects the schema server-side); set prompt:'schematron' when baseUrl points at a LOCALLY SERVED open-weight Schematron (e.g. an MLX schematron8B), because that model was fine-tuned on a prompt with the schema INSIDE the user message and an HTML-only prompt returns garbage from it even under constrained decoding. In 'schematron' mode NO response_format is sent at all - the schema already lives in the prompt, and the client validates only that the response parses as JSON. THE SCHEMA-DESCRIPTION RULE: this is prompt-driven extraction, and the extraction INSTRUCTIONS live in the schema's property descriptions - write them like prompts ('the job title, exactly as printed', 'ISO date, not 'yesterday'). A schema whose properties carry no descriptions is REJECTED up front with an error saying so, because it would produce garbage extraction with no hint why. Source: 'dom' (live DOM, the rendered, JS-executed page) is the default; 'served' (the raw server response before rendering) currently returns a not-implemented error rather than guessing. Size guard: cleaned HTML over maxChars (default 300000) fails with html_too_large naming the actual size - shrink the payload with 'selector' to scope to a subtree, clean:'aggressive' to strip harder (standard already drops scripts/styles/hidden), or a larger maxChars if you truly need it all. Endpoint failures surface as upstream_error (with status and body, API key redacted) or timeout; the response never contains the key. A completion the endpoint cut off at its 8192-token output cap fails naming finish_reason=length and the fix (narrow with 'selector', or ask the schema for less) - it is NOT reported as invalid JSON, because a truncated tail is a size problem, not a model one. COST IS ALWAYS VISIBLE: every successful call reports the endpoint's usage token counts inline, so a slow/bloated extraction is visible in the answer itself, not discovered on a bill later.",
     "params": {
       "lease": "Lease token from claim_page. Omit for a tab this process holds; required for a tab held by another process or one claimed explicitly. Auto-acquired under CDP_REQUIRE_LEASE (then pass target, not lease). See server instructions.",
       "target": "Page selector: 'active' (default), 'index:N', 'url:<substr>', 'title:<substr>', 'label:<name>', or a 32-hex '<targetId>'. Grammar: see server instructions.",
@@ -496,7 +496,7 @@ export const TOOL_DOCS: Record<string, ToolDoc> = {
       "clean": "HTML cleaning level before the payload is measured and sent. 'standard' (default) strips scripts, styles, hidden elements and other non-content noise. 'aggressive' additionally collapses whitespace, drops nav/footer/boilerplate - use when html_too_large bites or the extractor chases junk. 'none' sends the raw HTML; almost never what you want.",
       "maxChars": "Character cap on the cleaned HTML payload. Default 300000 (also settable via CDP_EXTRACT_MAX_CHARS). Exceeding it fails with html_too_large reporting actual vs cap - it does NOT silently truncate, because a half page extracted as though it were whole is a wrong answer that looks right.",
       "timeoutMs": "Total budget in ms for the endpoint call. Default 90000, hard max 300000 (CDP_EXTRACT_TIMEOUT_MS overrides the default). Expiry fails with a timeout error, never a hang.",
-      "prompt": "Prompt shape sent upstream. 'html' (default, also CDP_EXTRACT_PROMPT=html) sends the cleaned HTML as the only user message and lets response_format carry the schema - correct for the hosted Schematron API, which injects the schema server-side. 'schematron' sends the open-weight Schematron model card's own messages: a 'You are a helpful assistant' system message plus a user message with the compact JSON-stringified schema inline, then the HTML, then 'MAKE SURE ITS VALID JSON.' - use it when baseUrl points at a locally served open-weight Schematron. It adds roughly the stringified schema's length to the prompt (maxChars caps the HTML only); response_format is sent in both modes. The arg wins over CDP_EXTRACT_PROMPT, and any other value is rejected up front.",
+      "prompt": "Prompt shape sent upstream. 'html' (default, also CDP_EXTRACT_PROMPT=html) sends the cleaned HTML as the only user message and lets response_format carry the schema - correct for the hosted Schematron API, which injects the schema server-side. 'schematron' sends the open-weight Schematron model card's own messages: a 'You are a helpful assistant' system message plus a user message with the compact JSON-stringified schema inline, then the HTML, then 'MAKE SURE ITS VALID JSON.' - use it when baseUrl points at a locally served open-weight Schematron. It adds roughly the stringified schema's length to the prompt (maxChars caps the HTML only); 'schematron' mode sends NO response_format - the schema lives in the prompt and the client validates only that the response parses as JSON. The arg wins over CDP_EXTRACT_PROMPT, and any other value is rejected up front.",
       "model": "Model name for the endpoint call. Default 'schematron' on the loopback llm-ferry; override per-call or via CDP_EXTRACT_MODEL.",
       "baseUrl": "OpenAI-compatible endpoint base URL (the server appends /chat/completions). DEFAULT IS http://127.0.0.1:8090/v1 - loopback llm-ferry, page content never leaves the machine. Setting this (or CDP_EXTRACT_BASE_URL) to a remote host is an operator decision about where page content may travel; the response never echoes the URL with credentials.",
       "savePath": "Write the extracted JSON here instead of returning it inline: the response then carries the file path and usage counts only, keeping large payloads out of the transcript. An absolute path (starting with /) is used as-is; a relative path resolves under the artifact dir (/tmp/cdp-toolkit); missing parents are created."

package/src/tools/extract.ts

@@ -14,10 +14,13 @@
  * remote host.
  *
  * The caller's JSON Schema is BOTH the response contract (sent as
- * response_format json_schema strict) AND the extraction prompt: Schematron
- * reads the schema's property DESCRIPTIONS as its only instructions. A schema
- * with no descriptions anywhere would extract garbage with no hint why, so it
- * is rejected up front, before any page work happens.
+ * response_format json_schema strict in "html" mode) AND the extraction
+ * prompt: Schematron reads the schema's property DESCRIPTIONS as its only
+ * instructions. A schema with no descriptions anywhere would extract garbage
+ * with no hint why, so it is rejected up front, before any page work happens.
+ * In "schematron" mode response_format is OMITTED — the schema already rides
+ * inline in the prompt, and constrained decoding on top of it collapses real
+ * pages to an empty result (see buildExtractionMessages/callExtractionEndpoint).
  *
  * COST IS ALWAYS VISIBLE: every successful call reports the endpoint's token
  * counts, so a bloated extraction is visible in the answer itself, not
@@ -99,7 +102,9 @@ export interface ExtractPageArgs {
    * locally served open-weight Schematron was fine-tuned on: without it, an
    * HTML-only prompt yields garbage even under constrained decoding. It adds
    * roughly the JSON-stringified schema's length to the prompt. Default from
-   * CDP_EXTRACT_PROMPT, then "html". response_format is sent in BOTH modes.
+   * CDP_EXTRACT_PROMPT, then "html". "schematron" mode sends NO
+   * response_format — the schema is already inline in the prompt, and the
+   * client validates only that the response parses as JSON.
    */
   prompt?: "html" | "schematron";
   /** Model name override (default CDP_EXTRACT_MODEL, then "schematron"). */
@@ -564,12 +569,23 @@ export async function callExtractionEndpoint(opts: {
   prompt?: "html" | "schematron";
 }): Promise<{ content: string; usage: ExtractUsage; model: string; finishReason: string | undefined }> {
   const url = `${opts.baseUrl.replace(/\/+$/, "")}/chat/completions`;
+  const promptMode = opts.prompt ?? "html";
+  // "schematron" mode OMITS response_format entirely (never sends it as null).
+  // Measured 2026-09-12: against a locally served open-weight Schematron-8B
+  // (llguidance-constrained decoding), the IDENTICAL messages with
+  // response_format present returned {"stories": []} in 6 completion tokens
+  // for a real CNN zone, while omitting response_format returned valid,
+  // schema-conformant JSON with 5 stories (285 tokens) — the schema is
+  // already inline in the schematron user message, so constrained decoding
+  // on top of it is redundant and, on real markup, harmful.
   const body = JSON.stringify({
     model: opts.model,
-    messages: buildExtractionMessages(opts.prompt ?? "html", opts.html, opts.schema),
+    messages: buildExtractionMessages(promptMode, opts.html, opts.schema),
     temperature: 0,
     max_tokens: COMPLESION_TOKENS,
-    response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } },
+    ...(promptMode === "schematron"
+      ? {}
+      : { response_format: { type: "json_schema", json_schema: { name: "extract", strict: true, schema: opts.schema } } }),
   });
   const headers: Record<string, string> = { "content-type": "application/json" };
   if (opts.apiKey) headers.authorization = `Bearer ${opts.apiKey}`;
@@ -715,6 +731,15 @@ export async function extractPage(
         `extract_page: endpoint returned invalid JSON (${(e as Error).message}${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`,
       );
     }
+    // In "schematron" mode nothing constrains the shape server-side (no
+    // response_format), so a syntactically valid but non-object payload
+    // (array/null/primitive) is caught here with the same invalid-JSON error
+    // shape used above.
+    if (data === null || typeof data !== "object" || Array.isArray(data)) {
+      throw new CdpError(
+        `extract_page: endpoint returned invalid JSON (expected a JSON object${finishReason !== undefined ? `, finish_reason=${finishReason}` : ""}): ${excerpt(redactSecrets(content, apiKey), 200)}`,
+      );
+    }
 
     const result: ExtractPageResult = {
       data,
@@ -759,7 +784,8 @@ export async function extractPage(
  * method is used, and the live DOM is never mutated (cleaning runs on a clone).
  *
  * Upstream dependency (global fetch, zero npm deps): POST ${baseUrl}/chat/completions
- * with response_format json_schema strict; default baseUrl is the loopback
+ * with response_format json_schema strict in "html" mode only ("schematron"
+ * mode omits it — see callExtractionEndpoint); default baseUrl is the loopback
  * llm-ferry (http://127.0.0.1:8090/v1, model "schematron"). If it is not
  * running, the network error names the URL and asks "is ferry running?".
  *

package/src/version.ts

@@ -6,4 +6,4 @@
  * ready line; keeping it in its own module means a release bump touches one
  * line here plus package.json, not a const buried in the server.
  */
-export const VERSION = "2.5.0";
+export const VERSION = "2.6.0";