mcp-researchpowerpack 6.0.16 → 6.0.18

package/README.md

@@ -2,7 +2,7 @@
 
 # mcp-researchpowerpack
 
-HTTP MCP server for research. Three tools, orientation-first, built for agents that run multi-pass research loops.
+HTTP MCP server for research. Five tools, orientation-first, built for agents that run multi-pass research loops.
 
 Built on [mcp-use](https://github.com/nicepkg/mcp-use). No stdio, HTTP only.
 
@@ -11,8 +11,10 @@ Built on [mcp-use](https://github.com/nicepkg/mcp-use). No stdio, HTTP only.
 | tool | what it does | needs |
 |------|-------------|-------|
 | `start-research` | returns a goal-tailored brief: `primary_branch` (reddit / web / both), exact `first_call_sequence`, 25–50 keyword seeds, iteration hints, gaps to watch, stop criteria. Call FIRST every session. | `LLM_API_KEY` + `LLM_BASE_URL` + `LLM_MODEL` for non-degraded brief generation (optional) |
-| `web-search` | parallel Google search, up to 50 queries per call, parallel-callable across turns. `scope: "web" \| "reddit" \| "both"` — reddit mode filters to post permalinks. Queries should be retrieval probes, not topic labels: rewrite vague phrases into source-aware searches with anchors such as `site:`, exact quoted terms, versions, error text, package names, or community filters. Returns tiered markdown (HIGHLY_RELEVANT / MAYBE_RELEVANT / OTHER) + grounded synthesis + gaps + refine suggestions. | `SERPER_API_KEY` |
-| `scrape-links` | fetch URLs in parallel with per-URL LLM extraction. Auto-detects `reddit.com/r/.../comments/` permalinks and routes them through the Reddit API (threaded post + comments); PDF / DOCX / PPTX / XLSX URLs route through Jina Reader; non-reddit, non-document web URLs flow through Scrape.do. Parallel-callable. | `SCRAPEDO_API_KEY` for web URLs (+ `REDDIT_CLIENT_ID` / `REDDIT_CLIENT_SECRET` for reddit URLs; optional `JINA_API_KEY` for higher document limits) |
+| `raw-web-search` | parallel search, up to 50 `keywords` per call. Serper is primary; Jina Search is fallback when Serper is missing, fails, or yields empty query results. Returns the raw ranked markdown list directly. Use for broad discovery, audit trails, and Reddit permalink discovery via explicit `site:reddit.com/r/.../comments` probes. | `SERPER_API_KEY` or `JINA_API_KEY` |
+| `smart-web-search` | parallel search, up to 50 `keywords` per call, plus required `extract`. Serper/Jina provider order matches raw search. Always runs LLM classification and returns tiered markdown (HIGHLY_RELEVANT / MAYBE_RELEVANT / OTHER) + grounded synthesis + gaps + refine suggestions. Supports `scope: "web" \| "reddit" \| "both"`. | `SERPER_API_KEY` or `JINA_API_KEY` + LLM env |
+| `raw-scrape-links` | fetch URLs in parallel and return full markdown directly. Reddit post permalinks route through the Reddit API with threaded comments. Non-Reddit URLs use Jina Reader first, then Jina Reader through Scrape.do proxy mode, then optional Kernel browser rendering for web pages. | optional `REDDIT_CLIENT_ID` / `REDDIT_CLIENT_SECRET`, `SCRAPEDO_API_KEY`, `JINA_API_KEY`, `KERNEL_API_KEY` |
+| `smart-scrape-links` | same fetch stack as raw scrape, then always runs per-URL LLM extraction with required `extract`. Use for focused evidence packs with `## Source`, `## Matches`, `## Not found`, and `## Follow-up signals`. | raw scrape providers + LLM env |
 
 Also exposes `/health` and `health://status`.
 
@@ -20,7 +22,7 @@ Also exposes `/health` and `health://status`.
 
 Call `start-research` once at the beginning of each session with your goal. The server returns a brief that tells the agent exactly which tool to call first (reddit-first for sentiment/migration, web-first for spec/bug/pricing, both when opinion-heavy AND needs official sources), what keyword seeds to fire, and when to stop.
 
-For search fan-out, use bad → better rewrite thinking before calling `web-search`: turn broad phrases like `<feature> support`, `<product> pricing`, `<library> bug fix`, or `<tool> reviews` into source-aware probes such as `site:<official-docs-domain> "<feature>" "<platform-or-version>"`, `site:<vendor-domain> "<product>" pricing "enterprise" OR "free tier"`, `"<exact error text>" "<library-or-package>" "<version>" site:github.com`, or `site:reddit.com/r/<community>/comments "<tool>" "migration" OR "regression"`.
+For search fan-out, use bad → better rewrite thinking before calling `raw-web-search` or `smart-web-search`: turn broad phrases like `<feature> support`, `<product> pricing`, `<library> bug fix`, or `<tool> reviews` into source-aware probes such as `site:<official-docs-domain> "<feature>" "<platform-or-version>"`, `site:<vendor-domain> "<product>" pricing "enterprise" OR "free tier"`, `"<exact error text>" "<library-or-package>" "<version>" site:github.com`, or `site:reddit.com/r/<community>/comments "<tool>" "migration" OR "regression"`.
 
 Pair the server with the [`run-research`](https://github.com/yigitkonur/skills-by-yigitkonur/tree/main/skills/run-research) skill for the full agentic playbook:
 
@@ -69,11 +71,13 @@ Copy `.env.example`, set only what you need. Missing keys don't crash the server
 
 | var | enables |
 |-----|---------|
-| `SERPER_API_KEY` | `web-search` (all scopes) |
-| `SCRAPEDO_API_KEY` | `scrape-links` for non-reddit, non-document web URLs |
-| `REDDIT_CLIENT_ID` + `REDDIT_CLIENT_SECRET` | `scrape-links` for reddit.com permalinks (threaded post + comments) |
-| `JINA_API_KEY` | optional higher-rate `scrape-links` document conversion for PDF / DOCX / PPTX / XLSX URLs via Jina Reader |
-| `LLM_API_KEY` + `LLM_BASE_URL` + `LLM_MODEL` | goal-tailored brief, AI extraction, search classification, raw-mode refine suggestions |
+| `SERPER_API_KEY` | primary raw/smart web search provider |
+| `SCRAPEDO_API_KEY` | Scrape.do proxy-mode retry for Jina Reader (`X-Proxy-Url`) |
+| `REDDIT_CLIENT_ID` + `REDDIT_CLIENT_SECRET` | raw/smart scrape for reddit.com permalinks (threaded post + comments) |
+| `JINA_API_KEY` | Jina Search fallback and authenticated Jina Reader requests |
+| `KERNEL_API_KEY` | optional Kernel browser-render fallback after Jina direct + proxy fail |
+| `KERNEL_PROJECT` | optional Kernel project scoping header |
+| `LLM_API_KEY` + `LLM_BASE_URL` + `LLM_MODEL` | goal-tailored brief, `smart-web-search`, `smart-scrape-links` |
 
 ### llm (AI extraction + classification)
 
@@ -133,11 +137,12 @@ src/
   config/                env parsing, capability detection, lazy proxy config
   clients/               provider API clients (serper, reddit, scrapedo, jina)
   tools/
-    registry.ts          registerAllTools() — wires 3 tools
+    registry.ts          registerAllTools() — wires 5 tools
     start-research.ts    goal-tailored brief + static playbook
-    search.ts            web-search handler (with CTR-weighted URL aggregation + LLM classification)
-    scrape.ts            scrape-links handler (reddit + web + document branches in parallel)
-    mcp-helpers.ts       response builders (markdown + structured MCP output)
+    search.ts            raw/smart search handlers (CTR ranking + optional LLM classification)
+    scrape.ts            raw/smart scrape handlers (Reddit API, Jina Reader, Scrape.do proxy,
+                         optional Kernel, optional LLM extraction)
+    mcp-helpers.ts       markdown response builders
     utils.ts             shared formatters
   services/
     llm-processor.ts     AI extraction, classification, brief generation — primary + fallback model, always low reasoning
@@ -153,7 +158,7 @@ src/
     logger.ts            mcpLog() — stderr-only (MCP-safe)
 ```
 
-Key patterns: capability detection at startup, description-led tool routing (no bootstrap gate), always-on structured MCP tool output, tiered classified output in `web-search`, parallel reddit + web + document branches in `scrape-links`, Jina fallback for binary/document content, bounded concurrency via `p-map`, CTR-based URL ranking, tools never throw (always return `toolFailure`), and structured errors with retry classification.
+Key patterns: capability detection at startup, description-led tool routing (no bootstrap gate), markdown-only MCP tool output, raw/smart tool split, tiered classified output in `smart-web-search`, Reddit API routing in scrape tools, Jina Reader first for non-Reddit URLs, Scrape.do proxy-mode retry through `X-Proxy-Url`, optional Kernel browser-render fallback, bounded concurrency via `p-map`, CTR-based URL ranking, tools never throw (always return `toolFailure`), and structured errors with retry classification.
 
 ## license
 

package/dist/index.js

@@ -14,7 +14,7 @@ import { getLLMHealth } from "./src/services/llm-processor.js";
 import { registerAllTools } from "./src/tools/registry.js";
 const DEFAULT_PORT = 3e3;
 const SHUTDOWN_TIMEOUT_MS = 1e4;
-const WEBSITE_URL = "https://github.com/yigitkonur/mcp-researchpowerpack-http";
+const WEBSITE_URL = "https://github.com/yigitkonur/mcp-researchpowerpack";
 const LOCAL_DEFAULT_HOST = "127.0.0.1";
 const startupLogger = Logger.get("startup");
 function parseCsvEnv(value) {

package/dist/index.js.map

@@ -1,7 +1,7 @@
 {
   "version": 3,
   "sources": ["../index.ts"],
-  "sourcesContent": ["#!/usr/bin/env node\n\n// Expand libuv thread pool for parallel DNS lookups (default 4 is too low for 20+ concurrent connections)\nif (!process.env.UV_THREADPOOL_SIZE) {\n  process.env.UV_THREADPOOL_SIZE = '8';\n}\n\nimport { Logger } from 'mcp-use';\nimport {\n  InMemorySessionStore,\n  InMemoryStreamManager,\n  MCPServer,\n  object,\n  type ServerConfig,\n} from 'mcp-use/server';\n\nimport { SERVER } from './src/config/index.js';\nimport { getLLMHealth } from './src/services/llm-processor.js';\nimport { registerAllTools } from './src/tools/registry.js';\n\nconst DEFAULT_PORT = 3000 as const;\nconst SHUTDOWN_TIMEOUT_MS = 10_000 as const;\nconst WEBSITE_URL = 'https://github.com/yigitkonur/mcp-researchpowerpack-http' as const;\nconst LOCAL_DEFAULT_HOST = '127.0.0.1' as const;\n\ntype CleanupFn = () => Promise<void>;\n\nconst startupLogger = Logger.get('startup');\n\nfunction parseCsvEnv(value: string | undefined): string[] | undefined {\n  if (!value) return undefined;\n\n  const parts = value\n    .split(',')\n    .map((part) => part.trim())\n    .filter(Boolean);\n\n  return parts.length > 0 ? parts : undefined;\n}\n\nfunction parsePort(value: string | undefined, fallback: number): number {\n  const parsed = Number.parseInt(value ?? '', 10);\n  if (Number.isFinite(parsed) && parsed > 0) {\n    return parsed;\n  }\n\n  return fallback;\n}\n\nfunction resolvePort(): number {\n  const portFlagIndex = process.argv.findIndex((arg) => arg === '--port');\n  if (portFlagIndex >= 0) {\n    return parsePort(process.argv[portFlagIndex + 1], DEFAULT_PORT);\n  }\n\n  return parsePort(process.env.PORT, DEFAULT_PORT);\n}\n\nfunction resolveHost(): string {\n  const explicitHost = process.env.HOST?.trim();\n  if (explicitHost) {\n    return explicitHost;\n  }\n\n  // Cloud runtimes typically inject PORT and expect the process to listen on all interfaces.\n  if (process.env.PORT?.trim()) {\n    return '0.0.0.0';\n  }\n\n  return LOCAL_DEFAULT_HOST;\n}\n\nfunction buildCors(allowedOrigins: string[] | undefined): ServerConfig['cors'] {\n  if (!allowedOrigins || allowedOrigins.length === 0) {\n    return undefined;\n  }\n\n  return {\n    origin: allowedOrigins,\n    allowMethods: ['GET', 'HEAD', 'POST', 'PUT', 'DELETE', 'OPTIONS'],\n    allowHeaders: [\n      'Content-Type',\n      'Accept',\n      'Authorization',\n      'mcp-protocol-version',\n      'mcp-session-id',\n      'X-Proxy-Token',\n      'X-Target-URL',\n    ],\n    exposeHeaders: ['mcp-session-id'],\n  };\n}\n\nfunction configureLogging(): void {\n  Logger.configure({\n    level: process.env.NODE_ENV === 'production' ? 'info' : 'debug',\n    format: 'minimal',\n  });\n\n  const debug = process.env.DEBUG?.trim();\n  if (debug === '2') {\n    Logger.setDebug(2);\n  } else if (debug) {\n    Logger.setDebug(1);\n  }\n}\n\nfunction normalizeOrigin(value: string, envName: string): string {\n  try {\n    return new URL(value).origin;\n  } catch {\n    throw new Error(`${envName} must contain absolute URLs with protocol. Received: ${value}`);\n  }\n}\n\nfunction resolveAllowedOrigins(): string[] | undefined {\n  const explicitOrigins = parseCsvEnv(process.env.ALLOWED_ORIGINS);\n  if (explicitOrigins && explicitOrigins.length > 0) {\n    return explicitOrigins.map(origin => normalizeOrigin(origin, 'ALLOWED_ORIGINS'));\n  }\n\n  return undefined;\n}\n\nfunction buildSessionConfig(): {\n  sessionConfig: Pick<ServerConfig, 'sessionStore' | 'streamManager'>;\n  cleanupFns: CleanupFn[];\n} {\n  return {\n    sessionConfig: {\n      sessionStore: new InMemorySessionStore(),\n      streamManager: new InMemoryStreamManager(),\n    },\n    cleanupFns: [],\n  };\n}\n\nfunction buildHealthPayload(server: MCPServer, startedAt: number) {\n  const llm = getLLMHealth();\n  // Distinguish \"never probed\" (checkedAt === null) from \"probed and failed\"\n  // (checkedAt set, ok=false). The raw `lastPlannerOk` defaults to `false`\n  // at startup, which would mislead operators into thinking the LLM is\n  // broken before it has been exercised once.\n  const plannerOkForHealth = llm.lastPlannerCheckedAt === null ? null : llm.lastPlannerOk;\n  const extractorOkForHealth = llm.lastExtractorCheckedAt === null ? null : llm.lastExtractorOk;\n  return {\n    status: 'ok',\n    name: SERVER.NAME,\n    version: SERVER.VERSION,\n    transport: 'http',\n    uptime_seconds: Math.floor((Date.now() - startedAt) / 1000),\n    active_sessions: server.getActiveSessions().length,\n    llm_planner_ok: plannerOkForHealth,\n    llm_extractor_ok: extractorOkForHealth,\n    llm_planner_checked_at: llm.lastPlannerCheckedAt,\n    llm_extractor_checked_at: llm.lastExtractorCheckedAt,\n    llm_planner_error: llm.lastPlannerError,\n    llm_extractor_error: llm.lastExtractorError,\n    planner_configured: llm.plannerConfigured,\n    extractor_configured: llm.extractorConfigured,\n    // Counter surfacing lets operators diagnose gate behavior from outside\n    // the process (see src/tools/start-research.ts for the gate semantics).\n    consecutive_planner_failures: llm.consecutivePlannerFailures,\n    consecutive_extractor_failures: llm.consecutiveExtractorFailures,\n    timestamp: new Date().toISOString(),\n  };\n}\n\nasync function main(): Promise<void> {\n  configureLogging();\n\n  const isProduction = process.env.NODE_ENV === 'production';\n  const host = resolveHost();\n  const port = resolvePort();\n  const baseUrl = process.env.MCP_URL?.trim() || undefined;\n  const allowedOrigins = resolveAllowedOrigins();\n\n  const { sessionConfig, cleanupFns } = buildSessionConfig();\n\n  startupLogger.info(`Starting ${SERVER.NAME} v${SERVER.VERSION}`);\n  startupLogger.info(`Binding HTTP server to ${host}:${port}`);\n  if (allowedOrigins && allowedOrigins.length > 0) {\n    startupLogger.info(`Host validation enabled for origins: ${allowedOrigins.join(', ')}`);\n  } else if (isProduction) {\n    if (!baseUrl) {\n      startupLogger.error(\n        'Production mode requires ALLOWED_ORIGINS or MCP_URL to be set. ' +\n        'Without host validation, the server is vulnerable to DNS rebinding attacks. ' +\n        'Set ALLOWED_ORIGINS to the public deployment URL or custom domain.',\n      );\n      process.exit(1);\n    }\n    startupLogger.warn(\n      'Host validation is disabled because ALLOWED_ORIGINS is not set. ' +\n      'MCP_URL is set, so the server will start \u2014 but set ALLOWED_ORIGINS for full origin protection.',\n    );\n  } else {\n    startupLogger.info('Host validation disabled for local development');\n  }\n\n  const server = new MCPServer({\n    name: SERVER.NAME,\n    title: 'Research Powerpack',\n    version: SERVER.VERSION,\n    description: SERVER.DESCRIPTION,\n    websiteUrl: WEBSITE_URL,\n    host,\n    baseUrl,\n    cors: buildCors(allowedOrigins),\n    allowedOrigins,\n    ...sessionConfig,\n  });\n\n  registerAllTools(server);\n\n  // Advertise our LLM-augmentation capability via the MCP `experimental`\n  // namespace so capability-aware clients can branch at initialize-time\n  // instead of parsing per-call footers. mcp-use creates a fresh native MCP\n  // server per session via `getServerForSession()`, so we patch that factory\n  // to register our experimental capability on every session. The capability\n  // values are read fresh on each session so health flips are observable.\n  // See: docs/code-review/context/06-mcp-use-best-practices-primer.md (#3, #6).\n  try {\n    type Native = { server?: { registerCapabilities?: (caps: Record<string, unknown>) => void } };\n    type Patched = { getServerForSession?: (sessionId?: string) => Native };\n    const patched = server as unknown as Patched;\n    const original = patched.getServerForSession?.bind(server);\n    if (original) {\n      patched.getServerForSession = (sessionId?: string): Native => {\n        const native = original(sessionId);\n        try {\n          const llm = getLLMHealth();\n          native.server?.registerCapabilities?.({\n            experimental: {\n              research_powerpack: {\n                planner_available: llm.plannerConfigured,\n                extractor_available: llm.extractorConfigured,\n                planner_model: process.env.LLM_MODEL ?? null,\n                extractor_model: process.env.LLM_MODEL ?? null,\n              },\n            },\n          });\n        } catch {\n          // Capability registration is advisory; never block session creation.\n        }\n        return native;\n      };\n    }\n  } catch (err) {\n    startupLogger.warn(`Could not patch session-server factory: ${String(err)}`);\n  }\n\n  const startedAt = Date.now();\n\n  server.get('/health', (c) => c.json(buildHealthPayload(server, startedAt)));\n  server.get('/healthz', (c) => c.json(buildHealthPayload(server, startedAt)));\n\n  // Some MCP clients (Claude Desktop, Cursor, VS Code) proactively probe\n  // /.well-known/oauth-protected-resource before receiving any 401, per the\n  // MCP 2025-03-26 spec. Without these routes the server returns 404 and some\n  // clients surface a spurious \"authentication required\" error. A minimal PRM\n  // response with no authorization_servers field explicitly signals that this\n  // server requires no authentication.\n  const resourceBaseUrl = baseUrl ?? `http://${host}:${port}`;\n  server.get('/.well-known/oauth-protected-resource', (c) =>\n    c.json({ resource: resourceBaseUrl }),\n  );\n  server.get('/.well-known/oauth-protected-resource/mcp', (c) =>\n    c.json({ resource: `${resourceBaseUrl}/mcp` }),\n  );\n\n  server.resource(\n    {\n      name: 'server-health',\n      uri: 'health://status',\n      description: 'Current server health, uptime, and active MCP session count.',\n      mimeType: 'application/json',\n    },\n    async () => object(buildHealthPayload(server, startedAt)),\n  );\n\n  let isShuttingDown = false;\n\n  async function shutdown(signal: string, exitCode: number): Promise<void> {\n    if (isShuttingDown) return;\n    isShuttingDown = true;\n\n    const forceExit = setTimeout(() => {\n      startupLogger.error(`Forced exit after ${SHUTDOWN_TIMEOUT_MS}ms (${signal})`);\n      process.exit(1);\n    }, SHUTDOWN_TIMEOUT_MS);\n\n    try {\n      startupLogger.warn(`Shutdown signal received: ${signal}`);\n      await server.close();\n\n      for (const cleanupFn of cleanupFns) {\n        await cleanupFn();\n      }\n\n      clearTimeout(forceExit);\n      process.exit(exitCode);\n    } catch (error) {\n      clearTimeout(forceExit);\n      const message = error instanceof Error ? (error.stack ?? error.message) : String(error);\n      startupLogger.error(`Error while stopping server: ${message}`);\n      process.exit(1);\n    }\n  }\n\n  process.on('SIGTERM', () => {\n    void shutdown('SIGTERM', 0);\n  });\n\n  process.on('SIGINT', () => {\n    void shutdown('SIGINT', 0);\n  });\n\n  process.on('uncaughtException', (error) => {\n    startupLogger.error(`Uncaught exception: ${error.stack ?? error.message}`);\n    void shutdown('uncaughtException', 1);\n  });\n\n  process.on('unhandledRejection', (reason) => {\n    startupLogger.error(`Unhandled rejection: ${String(reason)}`);\n    void shutdown('unhandledRejection', 1);\n  });\n\n  await server.listen(port);\n\n  startupLogger.info(`${SERVER.NAME} v${SERVER.VERSION} listening on http://${host}:${port}/mcp`);\n}\n\nvoid main().catch((error) => {\n  const message = error instanceof Error ? (error.stack ?? error.message) : String(error);\n  startupLogger.error(`Server failed to start: ${message}`);\n  process.exit(1);\n});\n"],
+  "sourcesContent": ["#!/usr/bin/env node\n\n// Expand libuv thread pool for parallel DNS lookups (default 4 is too low for 20+ concurrent connections)\nif (!process.env.UV_THREADPOOL_SIZE) {\n  process.env.UV_THREADPOOL_SIZE = '8';\n}\n\nimport { Logger } from 'mcp-use';\nimport {\n  InMemorySessionStore,\n  InMemoryStreamManager,\n  MCPServer,\n  object,\n  type ServerConfig,\n} from 'mcp-use/server';\n\nimport { SERVER } from './src/config/index.js';\nimport { getLLMHealth } from './src/services/llm-processor.js';\nimport { registerAllTools } from './src/tools/registry.js';\n\nconst DEFAULT_PORT = 3000 as const;\nconst SHUTDOWN_TIMEOUT_MS = 10_000 as const;\nconst WEBSITE_URL = 'https://github.com/yigitkonur/mcp-researchpowerpack' as const;\nconst LOCAL_DEFAULT_HOST = '127.0.0.1' as const;\n\ntype CleanupFn = () => Promise<void>;\n\nconst startupLogger = Logger.get('startup');\n\nfunction parseCsvEnv(value: string | undefined): string[] | undefined {\n  if (!value) return undefined;\n\n  const parts = value\n    .split(',')\n    .map((part) => part.trim())\n    .filter(Boolean);\n\n  return parts.length > 0 ? parts : undefined;\n}\n\nfunction parsePort(value: string | undefined, fallback: number): number {\n  const parsed = Number.parseInt(value ?? '', 10);\n  if (Number.isFinite(parsed) && parsed > 0) {\n    return parsed;\n  }\n\n  return fallback;\n}\n\nfunction resolvePort(): number {\n  const portFlagIndex = process.argv.findIndex((arg) => arg === '--port');\n  if (portFlagIndex >= 0) {\n    return parsePort(process.argv[portFlagIndex + 1], DEFAULT_PORT);\n  }\n\n  return parsePort(process.env.PORT, DEFAULT_PORT);\n}\n\nfunction resolveHost(): string {\n  const explicitHost = process.env.HOST?.trim();\n  if (explicitHost) {\n    return explicitHost;\n  }\n\n  // Cloud runtimes typically inject PORT and expect the process to listen on all interfaces.\n  if (process.env.PORT?.trim()) {\n    return '0.0.0.0';\n  }\n\n  return LOCAL_DEFAULT_HOST;\n}\n\nfunction buildCors(allowedOrigins: string[] | undefined): ServerConfig['cors'] {\n  if (!allowedOrigins || allowedOrigins.length === 0) {\n    return undefined;\n  }\n\n  return {\n    origin: allowedOrigins,\n    allowMethods: ['GET', 'HEAD', 'POST', 'PUT', 'DELETE', 'OPTIONS'],\n    allowHeaders: [\n      'Content-Type',\n      'Accept',\n      'Authorization',\n      'mcp-protocol-version',\n      'mcp-session-id',\n      'X-Proxy-Token',\n      'X-Target-URL',\n    ],\n    exposeHeaders: ['mcp-session-id'],\n  };\n}\n\nfunction configureLogging(): void {\n  Logger.configure({\n    level: process.env.NODE_ENV === 'production' ? 'info' : 'debug',\n    format: 'minimal',\n  });\n\n  const debug = process.env.DEBUG?.trim();\n  if (debug === '2') {\n    Logger.setDebug(2);\n  } else if (debug) {\n    Logger.setDebug(1);\n  }\n}\n\nfunction normalizeOrigin(value: string, envName: string): string {\n  try {\n    return new URL(value).origin;\n  } catch {\n    throw new Error(`${envName} must contain absolute URLs with protocol. Received: ${value}`);\n  }\n}\n\nfunction resolveAllowedOrigins(): string[] | undefined {\n  const explicitOrigins = parseCsvEnv(process.env.ALLOWED_ORIGINS);\n  if (explicitOrigins && explicitOrigins.length > 0) {\n    return explicitOrigins.map(origin => normalizeOrigin(origin, 'ALLOWED_ORIGINS'));\n  }\n\n  return undefined;\n}\n\nfunction buildSessionConfig(): {\n  sessionConfig: Pick<ServerConfig, 'sessionStore' | 'streamManager'>;\n  cleanupFns: CleanupFn[];\n} {\n  return {\n    sessionConfig: {\n      sessionStore: new InMemorySessionStore(),\n      streamManager: new InMemoryStreamManager(),\n    },\n    cleanupFns: [],\n  };\n}\n\nfunction buildHealthPayload(server: MCPServer, startedAt: number) {\n  const llm = getLLMHealth();\n  // Distinguish \"never probed\" (checkedAt === null) from \"probed and failed\"\n  // (checkedAt set, ok=false). The raw `lastPlannerOk` defaults to `false`\n  // at startup, which would mislead operators into thinking the LLM is\n  // broken before it has been exercised once.\n  const plannerOkForHealth = llm.lastPlannerCheckedAt === null ? null : llm.lastPlannerOk;\n  const extractorOkForHealth = llm.lastExtractorCheckedAt === null ? null : llm.lastExtractorOk;\n  return {\n    status: 'ok',\n    name: SERVER.NAME,\n    version: SERVER.VERSION,\n    transport: 'http',\n    uptime_seconds: Math.floor((Date.now() - startedAt) / 1000),\n    active_sessions: server.getActiveSessions().length,\n    llm_planner_ok: plannerOkForHealth,\n    llm_extractor_ok: extractorOkForHealth,\n    llm_planner_checked_at: llm.lastPlannerCheckedAt,\n    llm_extractor_checked_at: llm.lastExtractorCheckedAt,\n    llm_planner_error: llm.lastPlannerError,\n    llm_extractor_error: llm.lastExtractorError,\n    planner_configured: llm.plannerConfigured,\n    extractor_configured: llm.extractorConfigured,\n    // Counter surfacing lets operators diagnose gate behavior from outside\n    // the process (see src/tools/start-research.ts for the gate semantics).\n    consecutive_planner_failures: llm.consecutivePlannerFailures,\n    consecutive_extractor_failures: llm.consecutiveExtractorFailures,\n    timestamp: new Date().toISOString(),\n  };\n}\n\nasync function main(): Promise<void> {\n  configureLogging();\n\n  const isProduction = process.env.NODE_ENV === 'production';\n  const host = resolveHost();\n  const port = resolvePort();\n  const baseUrl = process.env.MCP_URL?.trim() || undefined;\n  const allowedOrigins = resolveAllowedOrigins();\n\n  const { sessionConfig, cleanupFns } = buildSessionConfig();\n\n  startupLogger.info(`Starting ${SERVER.NAME} v${SERVER.VERSION}`);\n  startupLogger.info(`Binding HTTP server to ${host}:${port}`);\n  if (allowedOrigins && allowedOrigins.length > 0) {\n    startupLogger.info(`Host validation enabled for origins: ${allowedOrigins.join(', ')}`);\n  } else if (isProduction) {\n    if (!baseUrl) {\n      startupLogger.error(\n        'Production mode requires ALLOWED_ORIGINS or MCP_URL to be set. ' +\n        'Without host validation, the server is vulnerable to DNS rebinding attacks. ' +\n        'Set ALLOWED_ORIGINS to the public deployment URL or custom domain.',\n      );\n      process.exit(1);\n    }\n    startupLogger.warn(\n      'Host validation is disabled because ALLOWED_ORIGINS is not set. ' +\n      'MCP_URL is set, so the server will start \u2014 but set ALLOWED_ORIGINS for full origin protection.',\n    );\n  } else {\n    startupLogger.info('Host validation disabled for local development');\n  }\n\n  const server = new MCPServer({\n    name: SERVER.NAME,\n    title: 'Research Powerpack',\n    version: SERVER.VERSION,\n    description: SERVER.DESCRIPTION,\n    websiteUrl: WEBSITE_URL,\n    host,\n    baseUrl,\n    cors: buildCors(allowedOrigins),\n    allowedOrigins,\n    ...sessionConfig,\n  });\n\n  registerAllTools(server);\n\n  // Advertise our LLM-augmentation capability via the MCP `experimental`\n  // namespace so capability-aware clients can branch at initialize-time\n  // instead of parsing per-call footers. mcp-use creates a fresh native MCP\n  // server per session via `getServerForSession()`, so we patch that factory\n  // to register our experimental capability on every session. The capability\n  // values are read fresh on each session so health flips are observable.\n  // See: docs/code-review/context/06-mcp-use-best-practices-primer.md (#3, #6).\n  try {\n    type Native = { server?: { registerCapabilities?: (caps: Record<string, unknown>) => void } };\n    type Patched = { getServerForSession?: (sessionId?: string) => Native };\n    const patched = server as unknown as Patched;\n    const original = patched.getServerForSession?.bind(server);\n    if (original) {\n      patched.getServerForSession = (sessionId?: string): Native => {\n        const native = original(sessionId);\n        try {\n          const llm = getLLMHealth();\n          native.server?.registerCapabilities?.({\n            experimental: {\n              research_powerpack: {\n                planner_available: llm.plannerConfigured,\n                extractor_available: llm.extractorConfigured,\n                planner_model: process.env.LLM_MODEL ?? null,\n                extractor_model: process.env.LLM_MODEL ?? null,\n              },\n            },\n          });\n        } catch {\n          // Capability registration is advisory; never block session creation.\n        }\n        return native;\n      };\n    }\n  } catch (err) {\n    startupLogger.warn(`Could not patch session-server factory: ${String(err)}`);\n  }\n\n  const startedAt = Date.now();\n\n  server.get('/health', (c) => c.json(buildHealthPayload(server, startedAt)));\n  server.get('/healthz', (c) => c.json(buildHealthPayload(server, startedAt)));\n\n  // Some MCP clients (Claude Desktop, Cursor, VS Code) proactively probe\n  // /.well-known/oauth-protected-resource before receiving any 401, per the\n  // MCP 2025-03-26 spec. Without these routes the server returns 404 and some\n  // clients surface a spurious \"authentication required\" error. A minimal PRM\n  // response with no authorization_servers field explicitly signals that this\n  // server requires no authentication.\n  const resourceBaseUrl = baseUrl ?? `http://${host}:${port}`;\n  server.get('/.well-known/oauth-protected-resource', (c) =>\n    c.json({ resource: resourceBaseUrl }),\n  );\n  server.get('/.well-known/oauth-protected-resource/mcp', (c) =>\n    c.json({ resource: `${resourceBaseUrl}/mcp` }),\n  );\n\n  server.resource(\n    {\n      name: 'server-health',\n      uri: 'health://status',\n      description: 'Current server health, uptime, and active MCP session count.',\n      mimeType: 'application/json',\n    },\n    async () => object(buildHealthPayload(server, startedAt)),\n  );\n\n  let isShuttingDown = false;\n\n  async function shutdown(signal: string, exitCode: number): Promise<void> {\n    if (isShuttingDown) return;\n    isShuttingDown = true;\n\n    const forceExit = setTimeout(() => {\n      startupLogger.error(`Forced exit after ${SHUTDOWN_TIMEOUT_MS}ms (${signal})`);\n      process.exit(1);\n    }, SHUTDOWN_TIMEOUT_MS);\n\n    try {\n      startupLogger.warn(`Shutdown signal received: ${signal}`);\n      await server.close();\n\n      for (const cleanupFn of cleanupFns) {\n        await cleanupFn();\n      }\n\n      clearTimeout(forceExit);\n      process.exit(exitCode);\n    } catch (error) {\n      clearTimeout(forceExit);\n      const message = error instanceof Error ? (error.stack ?? error.message) : String(error);\n      startupLogger.error(`Error while stopping server: ${message}`);\n      process.exit(1);\n    }\n  }\n\n  process.on('SIGTERM', () => {\n    void shutdown('SIGTERM', 0);\n  });\n\n  process.on('SIGINT', () => {\n    void shutdown('SIGINT', 0);\n  });\n\n  process.on('uncaughtException', (error) => {\n    startupLogger.error(`Uncaught exception: ${error.stack ?? error.message}`);\n    void shutdown('uncaughtException', 1);\n  });\n\n  process.on('unhandledRejection', (reason) => {\n    startupLogger.error(`Unhandled rejection: ${String(reason)}`);\n    void shutdown('unhandledRejection', 1);\n  });\n\n  await server.listen(port);\n\n  startupLogger.info(`${SERVER.NAME} v${SERVER.VERSION} listening on http://${host}:${port}/mcp`);\n}\n\nvoid main().catch((error) => {\n  const message = error instanceof Error ? (error.stack ?? error.message) : String(error);\n  startupLogger.error(`Server failed to start: ${message}`);\n  process.exit(1);\n});\n"],
   "mappings": ";AAGA,IAAI,CAAC,QAAQ,IAAI,oBAAoB;AACnC,UAAQ,IAAI,qBAAqB;AACnC;AAEA,SAAS,cAAc;AACvB;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OAEK;AAEP,SAAS,cAAc;AACvB,SAAS,oBAAoB;AAC7B,SAAS,wBAAwB;AAEjC,MAAM,eAAe;AACrB,MAAM,sBAAsB;AAC5B,MAAM,cAAc;AACpB,MAAM,qBAAqB;AAI3B,MAAM,gBAAgB,OAAO,IAAI,SAAS;AAE1C,SAAS,YAAY,OAAiD;AACpE,MAAI,CAAC,MAAO,QAAO;AAEnB,QAAM,QAAQ,MACX,MAAM,GAAG,EACT,IAAI,CAAC,SAAS,KAAK,KAAK,CAAC,EACzB,OAAO,OAAO;AAEjB,SAAO,MAAM,SAAS,IAAI,QAAQ;AACpC;AAEA,SAAS,UAAU,OAA2B,UAA0B;AACtE,QAAM,SAAS,OAAO,SAAS,SAAS,IAAI,EAAE;AAC9C,MAAI,OAAO,SAAS,MAAM,KAAK,SAAS,GAAG;AACzC,WAAO;AAAA,EACT;AAEA,SAAO;AACT;AAEA,SAAS,cAAsB;AAC7B,QAAM,gBAAgB,QAAQ,KAAK,UAAU,CAAC,QAAQ,QAAQ,QAAQ;AACtE,MAAI,iBAAiB,GAAG;AACtB,WAAO,UAAU,QAAQ,KAAK,gBAAgB,CAAC,GAAG,YAAY;AAAA,EAChE;AAEA,SAAO,UAAU,QAAQ,IAAI,MAAM,YAAY;AACjD;AAEA,SAAS,cAAsB;AAC7B,QAAM,eAAe,QAAQ,IAAI,MAAM,KAAK;AAC5C,MAAI,cAAc;AAChB,WAAO;AAAA,EACT;AAGA,MAAI,QAAQ,IAAI,MAAM,KAAK,GAAG;AAC5B,WAAO;AAAA,EACT;AAEA,SAAO;AACT;AAEA,SAAS,UAAU,gBAA4D;AAC7E,MAAI,CAAC,kBAAkB,eAAe,WAAW,GAAG;AAClD,WAAO;AAAA,EACT;AAEA,SAAO;AAAA,IACL,QAAQ;AAAA,IACR,cAAc,CAAC,OAAO,QAAQ,QAAQ,OAAO,UAAU,SAAS;AAAA,IAChE,cAAc;AAAA,MACZ;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,IACF;AAAA,IACA,eAAe,CAAC,gBAAgB;AAAA,EAClC;AACF;AAEA,SAAS,mBAAyB;AAChC,SAAO,UAAU;AAAA,IACf,OAAO,QAAQ,IAAI,aAAa,eAAe,SAAS;AAAA,IACxD,QAAQ;AAAA,EACV,CAAC;AAED,QAAM,QAAQ,QAAQ,IAAI,OAAO,KAAK;AACtC,MAAI,UAAU,KAAK;AACjB,WAAO,SAAS,CAAC;AAAA,EACnB,WAAW,OAAO;AAChB,WAAO,SAAS,CAAC;AAAA,EACnB;AACF;AAEA,SAAS,gBAAgB,OAAe,SAAyB;AAC/D,MAAI;AACF,WAAO,IAAI,IAAI,KAAK,EAAE;AAAA,EACxB,QAAQ;AACN,UAAM,IAAI,MAAM,GAAG,OAAO,wDAAwD,KAAK,EAAE;AAAA,EAC3F;AACF;AAEA,SAAS,wBAA8C;AACrD,QAAM,kBAAkB,YAAY,QAAQ,IAAI,eAAe;AAC/D,MAAI,mBAAmB,gBAAgB,SAAS,GAAG;AACjD,WAAO,gBAAgB,IAAI,YAAU,gBAAgB,QAAQ,iBAAiB,CAAC;AAAA,EACjF;AAEA,SAAO;AACT;AAEA,SAAS,qBAGP;AACA,SAAO;AAAA,IACL,eAAe;AAAA,MACb,cAAc,IAAI,qBAAqB;AAAA,MACvC,eAAe,IAAI,sBAAsB;AAAA,IAC3C;AAAA,IACA,YAAY,CAAC;AAAA,EACf;AACF;AAEA,SAAS,mBAAmB,QAAmB,WAAmB;AAChE,QAAM,MAAM,aAAa;AAKzB,QAAM,qBAAqB,IAAI,yBAAyB,OAAO,OAAO,IAAI;AAC1E,QAAM,uBAAuB,IAAI,2BAA2B,OAAO,OAAO,IAAI;AAC9E,SAAO;AAAA,IACL,QAAQ;AAAA,IACR,MAAM,OAAO;AAAA,IACb,SAAS,OAAO;AAAA,IAChB,WAAW;AAAA,IACX,gBAAgB,KAAK,OAAO,KAAK,IAAI,IAAI,aAAa,GAAI;AAAA,IAC1D,iBAAiB,OAAO,kBAAkB,EAAE;AAAA,IAC5C,gBAAgB;AAAA,IAChB,kBAAkB;AAAA,IAClB,wBAAwB,IAAI;AAAA,IAC5B,0BAA0B,IAAI;AAAA,IAC9B,mBAAmB,IAAI;AAAA,IACvB,qBAAqB,IAAI;AAAA,IACzB,oBAAoB,IAAI;AAAA,IACxB,sBAAsB,IAAI;AAAA;AAAA;AAAA,IAG1B,8BAA8B,IAAI;AAAA,IAClC,gCAAgC,IAAI;AAAA,IACpC,YAAW,oBAAI,KAAK,GAAE,YAAY;AAAA,EACpC;AACF;AAEA,eAAe,OAAsB;AACnC,mBAAiB;AAEjB,QAAM,eAAe,QAAQ,IAAI,aAAa;AAC9C,QAAM,OAAO,YAAY;AACzB,QAAM,OAAO,YAAY;AACzB,QAAM,UAAU,QAAQ,IAAI,SAAS,KAAK,KAAK;AAC/C,QAAM,iBAAiB,sBAAsB;AAE7C,QAAM,EAAE,eAAe,WAAW,IAAI,mBAAmB;AAEzD,gBAAc,KAAK,YAAY,OAAO,IAAI,KAAK,OAAO,OAAO,EAAE;AAC/D,gBAAc,KAAK,0BAA0B,IAAI,IAAI,IAAI,EAAE;AAC3D,MAAI,kBAAkB,eAAe,SAAS,GAAG;AAC/C,kBAAc,KAAK,wCAAwC,eAAe,KAAK,IAAI,CAAC,EAAE;AAAA,EACxF,WAAW,cAAc;AACvB,QAAI,CAAC,SAAS;AACZ,oBAAc;AAAA,QACZ;AAAA,MAGF;AACA,cAAQ,KAAK,CAAC;AAAA,IAChB;AACA,kBAAc;AAAA,MACZ;AAAA,IAEF;AAAA,EACF,OAAO;AACL,kBAAc,KAAK,gDAAgD;AAAA,EACrE;AAEA,QAAM,SAAS,IAAI,UAAU;AAAA,IAC3B,MAAM,OAAO;AAAA,IACb,OAAO;AAAA,IACP,SAAS,OAAO;AAAA,IAChB,aAAa,OAAO;AAAA,IACpB,YAAY;AAAA,IACZ;AAAA,IACA;AAAA,IACA,MAAM,UAAU,cAAc;AAAA,IAC9B;AAAA,IACA,GAAG;AAAA,EACL,CAAC;AAED,mBAAiB,MAAM;AASvB,MAAI;AAGF,UAAM,UAAU;AAChB,UAAM,WAAW,QAAQ,qBAAqB,KAAK,MAAM;AACzD,QAAI,UAAU;AACZ,cAAQ,sBAAsB,CAAC,cAA+B;AAC5D,cAAM,SAAS,SAAS,SAAS;AACjC,YAAI;AACF,gBAAM,MAAM,aAAa;AACzB,iBAAO,QAAQ,uBAAuB;AAAA,YACpC,cAAc;AAAA,cACZ,oBAAoB;AAAA,gBAClB,mBAAmB,IAAI;AAAA,gBACvB,qBAAqB,IAAI;AAAA,gBACzB,eAAe,QAAQ,IAAI,aAAa;AAAA,gBACxC,iBAAiB,QAAQ,IAAI,aAAa;AAAA,cAC5C;AAAA,YACF;AAAA,UACF,CAAC;AAAA,QACH,QAAQ;AAAA,QAER;AACA,eAAO;AAAA,MACT;AAAA,IACF;AAAA,EACF,SAAS,KAAK;AACZ,kBAAc,KAAK,2CAA2C,OAAO,GAAG,CAAC,EAAE;AAAA,EAC7E;AAEA,QAAM,YAAY,KAAK,IAAI;AAE3B,SAAO,IAAI,WAAW,CAAC,MAAM,EAAE,KAAK,mBAAmB,QAAQ,SAAS,CAAC,CAAC;AAC1E,SAAO,IAAI,YAAY,CAAC,MAAM,EAAE,KAAK,mBAAmB,QAAQ,SAAS,CAAC,CAAC;AAQ3E,QAAM,kBAAkB,WAAW,UAAU,IAAI,IAAI,IAAI;AACzD,SAAO;AAAA,IAAI;AAAA,IAAyC,CAAC,MACnD,EAAE,KAAK,EAAE,UAAU,gBAAgB,CAAC;AAAA,EACtC;AACA,SAAO;AAAA,IAAI;AAAA,IAA6C,CAAC,MACvD,EAAE,KAAK,EAAE,UAAU,GAAG,eAAe,OAAO,CAAC;AAAA,EAC/C;AAEA,SAAO;AAAA,IACL;AAAA,MACE,MAAM;AAAA,MACN,KAAK;AAAA,MACL,aAAa;AAAA,MACb,UAAU;AAAA,IACZ;AAAA,IACA,YAAY,OAAO,mBAAmB,QAAQ,SAAS,CAAC;AAAA,EAC1D;AAEA,MAAI,iBAAiB;AAErB,iBAAe,SAAS,QAAgB,UAAiC;AACvE,QAAI,eAAgB;AACpB,qBAAiB;AAEjB,UAAM,YAAY,WAAW,MAAM;AACjC,oBAAc,MAAM,qBAAqB,mBAAmB,OAAO,MAAM,GAAG;AAC5E,cAAQ,KAAK,CAAC;AAAA,IAChB,GAAG,mBAAmB;AAEtB,QAAI;AACF,oBAAc,KAAK,6BAA6B,MAAM,EAAE;AACxD,YAAM,OAAO,MAAM;AAEnB,iBAAW,aAAa,YAAY;AAClC,cAAM,UAAU;AAAA,MAClB;AAEA,mBAAa,SAAS;AACtB,cAAQ,KAAK,QAAQ;AAAA,IACvB,SAAS,OAAO;AACd,mBAAa,SAAS;AACtB,YAAM,UAAU,iBAAiB,QAAS,MAAM,SAAS,MAAM,UAAW,OAAO,KAAK;AACtF,oBAAc,MAAM,gCAAgC,OAAO,EAAE;AAC7D,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF;AAEA,UAAQ,GAAG,WAAW,MAAM;AAC1B,SAAK,SAAS,WAAW,CAAC;AAAA,EAC5B,CAAC;AAED,UAAQ,GAAG,UAAU,MAAM;AACzB,SAAK,SAAS,UAAU,CAAC;AAAA,EAC3B,CAAC;AAED,UAAQ,GAAG,qBAAqB,CAAC,UAAU;AACzC,kBAAc,MAAM,uBAAuB,MAAM,SAAS,MAAM,OAAO,EAAE;AACzE,SAAK,SAAS,qBAAqB,CAAC;AAAA,EACtC,CAAC;AAED,UAAQ,GAAG,sBAAsB,CAAC,WAAW;AAC3C,kBAAc,MAAM,wBAAwB,OAAO,MAAM,CAAC,EAAE;AAC5D,SAAK,SAAS,sBAAsB,CAAC;AAAA,EACvC,CAAC;AAED,QAAM,OAAO,OAAO,IAAI;AAExB,gBAAc,KAAK,GAAG,OAAO,IAAI,KAAK,OAAO,OAAO,wBAAwB,IAAI,IAAI,IAAI,MAAM;AAChG;AAEA,KAAK,KAAK,EAAE,MAAM,CAAC,UAAU;AAC3B,QAAM,UAAU,iBAAiB,QAAS,MAAM,SAAS,MAAM,UAAW,OAAO,KAAK;AACtF,gBAAc,MAAM,2BAA2B,OAAO,EAAE;AACxD,UAAQ,KAAK,CAAC;AAChB,CAAC;",
   "names": []
 }

package/dist/mcp-use.json

@@ -1,7 +1,7 @@
 {
   "includeInspector": false,
-  "buildTime": "2026-04-30T20:36:35.153Z",
-  "buildId": "7cde881e8eedd617",
+  "buildTime": "2026-05-05T10:02:24.122Z",
+  "buildId": "94ba0cb513ead054",
   "entryPoint": "dist/index.js",
   "widgets": {}
 }

package/dist/src/clients/jina.js

@@ -7,8 +7,20 @@ import {
 import { calculateBackoff } from "../utils/retry.js";
 import { mcpLog } from "../utils/logger.js";
 const JINA_READER_BASE = "https://r.jina.ai/";
-const DEFAULT_TIMEOUT_MS = 6e4;
+const JINA_SEARCH_BASE = "https://s.jina.ai/";
+const DEFAULT_TIMEOUT_SECONDS = 15;
+const DEFAULT_TIMEOUT_MS = DEFAULT_TIMEOUT_SECONDS * 1e3;
 const MAX_RETRIES = 2;
+const SEARCH_RESULTS_PER_QUERY = 10;
+function buildJinaSearchUrl(query) {
+  const params = new URLSearchParams({ q: query });
+  return `${JINA_SEARCH_BASE}?${params.toString()}`;
+}
+function buildScrapeDoProxyUrl(token, parameters = "render=false") {
+  const trimmed = token.trim();
+  if (!trimmed) return "";
+  return `http://${encodeURIComponent(trimmed)}:${parameters}@proxy.scrape.do:8080`;
+}
 class JinaClient {
   apiKey;
   constructor(apiKey) {
@@ -20,7 +32,13 @@ class JinaClient {
    * NEVER throws — always returns a JinaConvertResponse (possibly with error).
    */
   async convert(request) {
-    const { url, timeoutMs = DEFAULT_TIMEOUT_MS } = request;
+    const {
+      url,
+      timeoutSeconds = DEFAULT_TIMEOUT_SECONDS,
+      proxyUrl,
+      noCache = false,
+      allowProxyRetry = false
+    } = request;
     try {
       new URL(url);
     } catch {
@@ -31,105 +49,116 @@ class JinaClient {
         error: { code: ErrorCode.INVALID_INPUT, message: `Invalid URL: ${url}`, retryable: false }
       };
     }
-    const jinaUrl = `${JINA_READER_BASE}${url}`;
+    const first = await this.convertOnce({
+      url,
+      timeoutSeconds,
+      proxyUrl,
+      noCache
+    });
+    if (!first.error || !allowProxyRetry || proxyUrl || isTerminalReaderError(first.error)) {
+      return first;
+    }
+    mcpLog("warning", `Jina Reader failed for ${url}; retrying with Jina proxy`, "jina");
+    return this.convertOnce({
+      url,
+      timeoutSeconds,
+      proxyUrl: "auto",
+      noCache: true
+    });
+  }
+  async searchMultiple(queries) {
+    const startTime = Date.now();
+    if (queries.length === 0) {
+      return {
+        searches: [],
+        totalQueries: 0,
+        executionTime: 0,
+        error: { code: ErrorCode.INVALID_INPUT, message: "No queries provided", retryable: false }
+      };
+    }
+    if (!this.apiKey) {
+      return {
+        searches: [],
+        totalQueries: queries.length,
+        executionTime: Date.now() - startTime,
+        error: { code: ErrorCode.AUTH_ERROR, message: "Jina Search requires JINA_API_KEY", retryable: false }
+      };
+    }
+    const searches = await Promise.all(queries.map((query) => this.searchOne(query)));
+    const firstError = searches.find((search) => search.error)?.error;
+    const allFailed = searches.every((search) => search.error);
+    return {
+      searches,
+      totalQueries: queries.length,
+      executionTime: Date.now() - startTime,
+      ...allFailed && firstError ? { error: firstError } : {}
+    };
+  }
+  async convertOnce(request) {
     const headers = {
-      Accept: "text/markdown"
+      Accept: "application/json",
+      "Content-Type": "application/json"
     };
-    if (this.apiKey) {
-      headers["Authorization"] = `Bearer ${this.apiKey}`;
+    if (this.apiKey) headers["Authorization"] = `Bearer ${this.apiKey}`;
+    if (request.proxyUrl && request.proxyUrl !== "auto") {
+      headers["X-Proxy-Url"] = request.proxyUrl;
     }
+    const body = {
+      url: request.url,
+      respondWith: "markdown",
+      timeout: request.timeoutSeconds,
+      base: "final",
+      removeOverlay: true
+    };
+    if (request.proxyUrl === "auto") body["proxy"] = "auto";
+    if (request.noCache) body["noCache"] = true;
+    return this.fetchReader(body, headers, request.timeoutSeconds);
+  }
+  async fetchReader(body, headers, timeoutSeconds) {
     let lastError;
     for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
       try {
-        const response = await fetchWithTimeout(jinaUrl, {
-          method: "GET",
+        const response = await fetchWithTimeout(JINA_READER_BASE, {
+          method: "POST",
           headers,
-          timeoutMs
+          body: JSON.stringify(body),
+          timeoutMs: (timeoutSeconds + 5) * 1e3
         });
-        let content;
-        try {
-          content = await response.text();
-        } catch (readError) {
-          content = `Failed to read Jina response: ${readError instanceof Error ? readError.message : String(readError)}`;
-        }
+        const raw = await response.text().catch(
+          (readError) => `Failed to read Jina response: ${readError instanceof Error ? readError.message : String(readError)}`
+        );
         const usageHeader = response.headers.get("x-usage-tokens");
         const usageTokens = usageHeader ? Number(usageHeader) : void 0;
+        const parsed = parseReaderContent(raw);
         if (response.ok) {
-          if (!content.trim()) {
-            return {
-              content: "Jina returned an empty body",
-              statusCode: response.status,
-              credits: 0,
-              usageTokens: Number.isFinite(usageTokens) ? usageTokens : void 0,
-              error: {
-                code: ErrorCode.UNSUPPORTED_BINARY_CONTENT,
-                message: "Jina Reader returned empty content for this URL",
-                retryable: false
-              }
-            };
+          if (!parsed.content.trim()) {
+            return emptyReaderResponse(response.status, usageTokens);
           }
           return {
-            content,
+            content: parsed.content,
             statusCode: response.status,
             credits: 0,
             usageTokens: Number.isFinite(usageTokens) ? usageTokens : void 0
           };
         }
-        if (response.status === 401 || response.status === 403) {
-          return {
-            content: `Jina auth/quota error (${response.status}): ${content.slice(0, 200)}`,
-            statusCode: response.status,
-            credits: 0,
-            error: {
-              code: response.status === 401 ? ErrorCode.AUTH_ERROR : ErrorCode.QUOTA_EXCEEDED,
-              message: response.status === 401 ? "Jina Reader auth failed \u2014 check JINA_API_KEY" : "Jina Reader quota exceeded",
-              retryable: false,
-              statusCode: response.status
-            }
-          };
-        }
-        if (response.status === 404) {
-          return {
-            content: `Jina could not fetch the target URL (404)`,
-            statusCode: 404,
-            credits: 0,
-            error: {
-              code: ErrorCode.NOT_FOUND,
-              message: "Target URL not reachable by Jina Reader",
-              retryable: false,
-              statusCode: 404
-            }
-          };
-        }
-        if (response.status === 429 || response.status >= 500) {
-          lastError = classifyError({ status: response.status, message: content.slice(0, 200) });
-          if (attempt < MAX_RETRIES) {
-            const delayMs = calculateBackoff(attempt);
-            mcpLog(
-              "warning",
-              `Jina ${response.status} on attempt ${attempt + 1}/${MAX_RETRIES + 1}. Retrying in ${delayMs}ms`,
-              "jina"
-            );
-            await sleep(delayMs);
-            continue;
-          }
-          return {
-            content: `Jina Reader error (${response.status}): ${content.slice(0, 200)}`,
-            statusCode: response.status,
-            credits: 0,
-            error: lastError
-          };
+        const terminal = terminalReaderResponse(response.status, parsed.content || raw);
+        if (terminal) return terminal;
+        lastError = classifyError({ status: response.status, message: raw.slice(0, 200) });
+        if (lastError.retryable && attempt < MAX_RETRIES) {
+          const delayMs = calculateBackoff(attempt);
+          mcpLog(
+            "warning",
+            `Jina ${response.status} on attempt ${attempt + 1}/${MAX_RETRIES + 1}. Retrying in ${delayMs}ms`,
+            "jina"
+          );
+          await sleep(delayMs);
+          continue;
         }
         return {
-          content: `Jina Reader error (${response.status}): ${content.slice(0, 200)}`,
+          content: `Jina Reader error (${response.status}): ${raw.slice(0, 200)}`,
           statusCode: response.status,
           credits: 0,
-          error: {
-            code: ErrorCode.INVALID_INPUT,
-            message: `Jina Reader returned ${response.status}`,
-            retryable: false,
-            statusCode: response.status
-          }
+          error: lastError
         };
       } catch (error) {
         lastError = classifyError(error);
@@ -158,8 +187,161 @@ class JinaClient {
       error: lastError ?? { code: ErrorCode.UNKNOWN_ERROR, message: "All retries exhausted", retryable: false }
     };
   }
+  async searchOne(query) {
+    const headers = {
+      Accept: "application/json",
+      Authorization: `Bearer ${this.apiKey ?? ""}`
+    };
+    try {
+      const response = await fetchWithTimeout(buildJinaSearchUrl(query), {
+        method: "GET",
+        headers,
+        timeoutMs: DEFAULT_TIMEOUT_MS
+      });
+      const raw = await response.text().catch(
+        (readError) => `Failed to read Jina Search response: ${readError instanceof Error ? readError.message : String(readError)}`
+      );
+      if (!response.ok) {
+        return {
+          query,
+          results: [],
+          totalResults: 0,
+          related: [],
+          error: classifyError({ status: response.status, message: raw.slice(0, 200) })
+        };
+      }
+      const results = parseSearchResults(raw);
+      return { query, results, totalResults: results.length, related: [] };
+    } catch (error) {
+      return {
+        query,
+        results: [],
+        totalResults: 0,
+        related: [],
+        error: classifyError(error)
+      };
+    }
+  }
+}
+function parseReaderContent(raw) {
+  try {
+    const parsed = JSON.parse(raw);
+    const data = readRecord(parsed, "data");
+    const content = readString(data, "content");
+    if (content) return { content };
+  } catch {
+  }
+  return { content: raw };
+}
+function emptyReaderResponse(statusCode, usageTokens) {
+  return {
+    content: "Jina returned an empty body",
+    statusCode,
+    credits: 0,
+    usageTokens: Number.isFinite(usageTokens) ? usageTokens : void 0,
+    error: {
+      code: ErrorCode.UNSUPPORTED_BINARY_CONTENT,
+      message: "Jina Reader returned empty content for this URL",
+      retryable: false
+    }
+  };
+}
+function terminalReaderResponse(statusCode, content) {
+  if (statusCode === 401 || statusCode === 403) {
+    return {
+      content: `Jina auth/quota error (${statusCode}): ${content.slice(0, 200)}`,
+      statusCode,
+      credits: 0,
+      error: {
+        code: statusCode === 401 ? ErrorCode.AUTH_ERROR : ErrorCode.QUOTA_EXCEEDED,
+        message: statusCode === 401 ? "Jina Reader auth failed \u2014 check JINA_API_KEY" : "Jina Reader quota exceeded",
+        retryable: false,
+        statusCode
+      }
+    };
+  }
+  if (statusCode === 404) {
+    return {
+      content: "Jina could not fetch the target URL (404)",
+      statusCode: 404,
+      credits: 0,
+      error: {
+        code: ErrorCode.NOT_FOUND,
+        message: "Target URL not reachable by Jina Reader",
+        retryable: false,
+        statusCode: 404
+      }
+    };
+  }
+  if (statusCode >= 400 && statusCode < 500 && statusCode !== 429) {
+    return {
+      content: `Jina Reader error (${statusCode}): ${content.slice(0, 200)}`,
+      statusCode,
+      credits: 0,
+      error: {
+        code: ErrorCode.INVALID_INPUT,
+        message: `Jina Reader returned ${statusCode}`,
+        retryable: false,
+        statusCode
+      }
+    };
+  }
+  return null;
+}
+function isTerminalReaderError(error) {
+  return !error.retryable && (error.code === ErrorCode.AUTH_ERROR || error.code === ErrorCode.QUOTA_EXCEEDED || error.code === ErrorCode.NOT_FOUND || error.code === ErrorCode.INVALID_INPUT);
+}
+function parseSearchResults(raw) {
+  let data;
+  try {
+    const parsed = JSON.parse(raw);
+    data = readUnknown(parsed, "data");
+  } catch {
+    data = parseMarkdownSearchResults(raw);
+  }
+  const items = Array.isArray(data) ? data : [];
+  return items.map((item, index) => normalizeSearchItem(item, index)).filter((item) => item !== null).slice(0, SEARCH_RESULTS_PER_QUERY);
+}
+function normalizeSearchItem(item, index) {
+  const link = readString(item, "url") ?? readString(item, "link");
+  if (!link) return null;
+  return {
+    title: readString(item, "title") || link,
+    link,
+    snippet: (readString(item, "snippet") || readString(item, "description") || readString(item, "content") || "").slice(0, 500),
+    date: readString(item, "date") ?? readString(item, "publishedTime"),
+    position: index + 1
+  };
+}
+function parseMarkdownSearchResults(raw) {
+  const items = [];
+  const markdownLink = /\[([^\]]+)\]\((https?:\/\/[^)]+)\)/g;
+  let match;
+  while ((match = markdownLink.exec(raw)) !== null && items.length < SEARCH_RESULTS_PER_QUERY) {
+    const title = match[1];
+    const url = match[2];
+    if (title && url) items.push({ title, url });
+  }
+  return items;
+}
+function isRecord(value) {
+  return typeof value === "object" && value !== null;
+}
+function readUnknown(value, key) {
+  return isRecord(value) ? value[key] : void 0;
+}
+function readRecord(value, key) {
+  const child = readUnknown(value, key);
+  return isRecord(child) ? child : void 0;
+}
+function readString(value, key) {
+  const child = readUnknown(value, key);
+  return typeof child === "string" ? child : void 0;
 }
 export {
-  JinaClient
+  JinaClient,
+  buildJinaSearchUrl,
+  buildScrapeDoProxyUrl,
+  isTerminalReaderError
 };
 //# sourceMappingURL=jina.js.map