firecrawl-mcp 3.23.4 → 3.23.7

package/README.md

@@ -35,15 +35,29 @@ Connect to the remote hosted server with no setup:
 https://mcp.firecrawl.dev/v2/mcp
 ```
 
-On the keyless free tier, `scrape`, `search`, and `interact` work without an API key (rate-limited). Other tools such as `crawl`, `map`, `agent`, and `extract` still need a key.
+On the keyless free tier, `scrape`, `search`, and `parse` work without an API key (rate-limited). Other tools such as `crawl`, `map`, and `agent` still need a key.
 
-Prefer an API key or OAuth whenever the human can sign up. It unlocks the full tool set and higher limits. With a key, use:
+Prefer OAuth or an API key whenever the human can sign up. It unlocks the full tool set and higher limits.
+
+For an interactive account connection, configure your MCP client to use this server URL. This is an MCP endpoint, **not a browser page**; use the client's account-connection flow and do not add a second Firecrawl server entry when reconnecting:
+
+```
+https://mcp.firecrawl.dev/v2/mcp-oauth
+```
+
+For an API-key connection (for example, an unattended integration), keep the server URL as:
+
+```
+https://mcp.firecrawl.dev/v2/mcp
+```
+
+Then configure the client's secure header or secret setting with:
 
 ```
-https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp
+Authorization: Bearer <FIRECRAWL_API_KEY>
 ```
 
-See the [MCP server docs](https://docs.firecrawl.dev/mcp-server) and the [agent onboarding guide](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for setup details.
+Never put an API key in the server URL. Never put an API key in an agent chat. Configure it directly in the client or secret manager. See the [hosted MCP setup guide](https://docs.firecrawl.dev/mcp-server) and the [agent onboarding guide](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for client-specific instructions.
 
 #### Search-only endpoint
 
@@ -290,7 +304,6 @@ Use this guide to select the right tool for your task:
 | map          | Discovering URLs on a site                     | URL[]                          |
 | crawl        | Multi-page extraction (with limits)            | final crawl status/data after internal polling |
 | parse        | Files and hosted upload refs                   | markdown, JSON, or document output |
-| extract      | Structured extraction from URLs                | JSON structured data           |
 | search       | Web search for info                            | results[]                      |
 | developer    | Programming questions over developer sources   | results[] with passages        |
 | agent        | Complex multi-source research                  | JSON (structured data)         |
@@ -643,79 +656,34 @@ Parse local files or hosted upload references with Firecrawl's `/v2/parse` endpo
 
 **Returns:** Parsed document content or hosted upload instructions with a `nextToolCall`.
 
-### 7. Extract Tool (`firecrawl_extract`)
-
-Extract structured information from web pages using LLM capabilities. Supports both cloud AI and self-hosted LLM extraction.
-
-**Best for:**
-
-- Extracting specific structured data like prices, names, details.
-
-**Not recommended for:**
-
-- When you need the full content of a page (use scrape)
-- When you're not looking for specific structured data
-
-**Arguments:**
-
-- `urls`: Array of URLs to extract information from
-- `prompt`: Custom prompt for the LLM extraction
-- `systemPrompt`: System prompt to guide the LLM
-- `schema`: JSON schema for structured data extraction
-- `allowExternalLinks`: Allow extraction from external links
-- `enableWebSearch`: Enable web search for additional context
-- `includeSubdomains`: Include subdomains in extraction
-
-When using a self-hosted instance, the extraction will use your configured LLM. For cloud API, it uses Firecrawl's managed LLM service.
-**Prompt Example:**
-
-> "Extract the product name, price, and description from these product pages."
+### 7. Structured data with Scrape JSON
 
-**Usage Example:**
+For structured data from a known page, call `firecrawl_scrape` once per URL with `formats: ["json"]`. Put the extraction prompt and JSON schema in `jsonOptions`.
 
 ```json
 {
-  "name": "firecrawl_extract",
+  "name": "firecrawl_scrape",
   "arguments": {
-    "urls": ["https://example.com/page1", "https://example.com/page2"],
-    "prompt": "Extract product information including name, price, and description",
-    "systemPrompt": "You are a helpful assistant that extracts product information",
-    "schema": {
-      "type": "object",
-      "properties": {
-        "name": { "type": "string" },
-        "price": { "type": "number" },
-        "description": { "type": "string" }
-      },
-      "required": ["name", "price"]
-    },
-    "allowExternalLinks": false,
-    "enableWebSearch": false,
-    "includeSubdomains": false
-  }
-}
-```
-
-**Returns:**
-
-- Extracted structured data as defined by your schema
-
-```json
-{
-  "content": [
-    {
-      "type": "text",
-      "text": {
-        "name": "Example Product",
-        "price": 99.99,
-        "description": "This is an example product description"
+    "url": "https://example.com/product",
+    "formats": ["json"],
+    "jsonOptions": {
+      "prompt": "Extract the product name, price, and description.",
+      "schema": {
+        "type": "object",
+        "properties": {
+          "name": { "type": "string" },
+          "price": { "type": "number" },
+          "description": { "type": "string" }
+        },
+        "required": ["name", "price"]
       }
     }
-  ],
-  "isError": false
+  }
 }
 ```
 
+For unknown URLs or multi-source research, use `firecrawl_search` or `firecrawl_agent` before Scrape.
+
 ### 8. Agent Tool (`firecrawl_agent`)
 
 Autonomous web research agent. This is a separate AI agent layer that independently browses the internet, searches for information, navigates through pages, and extracts structured data based on your query.

package/dist/index.js

@@ -802,6 +802,11 @@ Search indexed public GitHub issue, pull-request, and README content. Returns ra
   });
 }
 
+// src/www-authenticate.ts
+function escapeWWWAuthenticateValue(value) {
+  return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
+}
+
 // src/index.ts
 dotenv.config({ debug: false, quiet: true });
 var require2 = createRequire(import.meta.url);
@@ -849,6 +854,9 @@ var DEFAULT_MCP_RESOURCE_URL = "https://mcp.firecrawl.dev/v2/mcp";
 var DEFAULT_MCP_OAUTH_RESOURCE_URL = "https://mcp.firecrawl.dev/v2/mcp-oauth";
 var DEFAULT_MCP_SEARCH_RESOURCE_URL = "https://mcp.firecrawl.dev/v2/mcp-search";
 var DEFAULT_MCP_SEARCH_ENDPOINT = "/v2/mcp-search";
+var MCP_CONNECTION_GUIDE_URL = "https://docs.firecrawl.dev/mcp-server";
+var MCP_OAUTH_SERVER_URL = "https://mcp.firecrawl.dev/v2/mcp-oauth";
+var API_KEY_SIGNUP_URL = "https://www.firecrawl.dev/app/api-keys";
 function withoutTrailingSlash(value) {
   return value.replace(/\/+$/, "");
 }
@@ -884,23 +892,23 @@ function getOAuthProtectedResourceMetadataUrl(profile) {
   const base = `${resource.origin}/.well-known/oauth-protected-resource`;
   return profile.id === "full" ? base : `${base}${resource.pathname}`;
 }
-function escapeWWWAuthenticateValue(value) {
-  return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
-}
-function createOAuthChallengeResponse(error, profile) {
+function createOAuthChallengeResponse(error, profile, details = {}) {
   if (!isMcpOAuthEnabled()) {
     return void 0;
   }
   const errorMessage = error instanceof Error ? error.message : String(error || "Unauthorized");
   const wwwAuthenticate = [
-    `resource_metadata="${escapeWWWAuthenticateValue(getOAuthProtectedResourceMetadataUrl(profile))}"`,
+    ...profile.advertiseOAuth ? [
+      `resource_metadata="${escapeWWWAuthenticateValue(getOAuthProtectedResourceMetadataUrl(profile))}"`
+    ] : [],
     'error="invalid_token"',
     `error_description="${escapeWWWAuthenticateValue(errorMessage)}"`
   ].join(", ");
   return new Response(
     JSON.stringify({
       error: "invalid_token",
-      error_description: errorMessage
+      error_description: errorMessage,
+      ...details
     }),
     {
       headers: {
@@ -911,11 +919,26 @@ function createOAuthChallengeResponse(error, profile) {
     }
   );
 }
-function createInvalidCredentialResponse(error) {
+function createInvalidCredentialResponse(_error) {
+  const recovery = invalidApiKeyRecoveryPayload();
   return new Response(
     JSON.stringify({
       error: "invalid_api_key",
-      error_description: error.message
+      error_description: recovery.message,
+      ...recovery
+    }),
+    {
+      headers: { "Content-Type": "application/json" },
+      status: 401
+    }
+  );
+}
+function createInvalidOAuthRecoveryResponse(recovery) {
+  return new Response(
+    JSON.stringify({
+      error: "invalid_token",
+      error_description: recovery.message,
+      ...recovery
     }),
     {
       headers: { "Content-Type": "application/json" },
@@ -941,6 +964,12 @@ var InvalidFirecrawlCredentialError = class extends Error {
     this.name = "InvalidFirecrawlCredentialError";
   }
 };
+var InvalidOAuthCredentialError = class extends Error {
+  constructor() {
+    super("Invalid OAuth access token");
+    this.name = "InvalidOAuthCredentialError";
+  }
+};
 var MCP_GLOBAL_SCOPE = "firecrawl:global";
 function values(value) {
   if (typeof value === "string") return value.split(/\s+/).filter(Boolean);
@@ -1019,7 +1048,7 @@ async function resolveCredentialFromHeaders(headers, profile) {
   }
   if (!data.active || !data.api_key) {
     if (isFirecrawlOAuthAccessToken(token)) {
-      throw new Error("Invalid OAuth access token");
+      throw new InvalidOAuthCredentialError();
     }
     return { invalid: true };
   }
@@ -1171,6 +1200,15 @@ function makeAuthenticate(profile) {
       if (error instanceof InvalidFirecrawlCredentialError) {
         throw createInvalidCredentialResponse(error);
       }
+      if (error instanceof InvalidOAuthCredentialError) {
+        const recovery = invalidOAuthRecoveryPayload(profile);
+        const oauthChallenge2 = createOAuthChallengeResponse(
+          new Error(recovery.message),
+          profile,
+          recovery
+        );
+        throw oauthChallenge2 ?? createInvalidOAuthRecoveryResponse(recovery);
+      }
       if (error instanceof CredentialValidationUnavailableError) {
         throw new Response(
           JSON.stringify({
@@ -1273,8 +1311,8 @@ var ConsoleLogger = class {
 var openAiAppsChallengeToken = normalizeHeader(
   process.env.OPENAI_APPS_CHALLENGE_TOKEN
 );
-var FULL_PROFILE_INSTRUCTIONS = `Firecrawl provides web search, page retrieval, site URL discovery, multi-page collection, structured extraction, monitoring, and asynchronous research. Match the requested operation to the tool boundary: firecrawl_scrape retrieves one supplied page, firecrawl_map enumerates URLs under a site without retrieving their content, and firecrawl_agent starts multi-source research whose result is read with firecrawl_agent_status. Provide only the required inputs and account for stated network or external side effects.`;
-var KEYLESS_PROFILE_INSTRUCTIONS = `Without authentication, this endpoint exposes Search, Scrape, and Parse with usage limits. An OAuth connection or Authorization bearer API key exposes account tools; unavailable tools return connection guidance. Firecrawl provides web search, page retrieval, site URL discovery, multi-page collection, structured extraction, monitoring, and asynchronous research. Match the requested operation to the tool boundary: firecrawl_scrape retrieves one supplied page, firecrawl_map enumerates URLs under a site without retrieving their content, and firecrawl_agent starts multi-source research whose result is read with firecrawl_agent_status. Provide only the required inputs.`;
+var FULL_PROFILE_INSTRUCTIONS = `Firecrawl provides web search, page retrieval, site URL discovery, multi-page collection, structured page data, monitoring, and asynchronous research. Match the requested operation to the tool boundary: firecrawl_scrape retrieves one supplied page and can return JSON matching a supplied schema, firecrawl_map enumerates URLs under a site without retrieving their content, and firecrawl_agent starts multi-source research whose result is read with firecrawl_agent_status. Provide only the required inputs and account for stated network or external side effects.`;
+var KEYLESS_PROFILE_INSTRUCTIONS = `Without authentication, this endpoint exposes Search, Scrape, and Parse with usage limits. An OAuth connection or Authorization bearer API key exposes account tools; unavailable tools return connection guidance. Firecrawl provides web search, page retrieval, site URL discovery, multi-page collection, structured page data, monitoring, and asynchronous research. Match the requested operation to the tool boundary: firecrawl_scrape retrieves one supplied page and can return JSON matching a supplied schema, firecrawl_map enumerates URLs under a site without retrieving their content, and firecrawl_agent starts multi-source research whose result is read with firecrawl_agent_status. Provide only the required inputs.`;
 var SEARCH_PROFILE_INSTRUCTIONS = `Firecrawl provides web, developer, and research search. Use firecrawl_search to find relevant results across the web and specialized indexes. For a programming question, use firecrawl_search with categories: ["developer"] to search indexed GitHub issues, merged pull requests, READMEs, and documentation. Use the firecrawl_research_* tools to search academic and research literature, expand from anchor papers via the citation graph, read full-text passages from a specific paper, and search public code repositories. All tools are read-only and return ranked results.`;
 var SEARCH_PROFILE_TOOLS = /* @__PURE__ */ new Set([
   "firecrawl_search",
@@ -1296,6 +1334,7 @@ function makeFullProfile() {
     allowKeyless: !account,
     acceptApiKeys: true,
     acceptLegacyAudience: account && process.env.MCP_OAUTH_ACCEPT_LEGACY_V2_MCP_AUD !== "false",
+    advertiseOAuth: account,
     primary: true
   };
 }
@@ -1323,6 +1362,7 @@ function makeSearchProfile({ primary = false } = {}) {
     // deployment explicitly enables the same profile flag used by primary.
     acceptApiKeys: !oauthOnly,
     requireManagedOAuth: oauthOnly,
+    advertiseOAuth: true,
     primary
   };
 }
@@ -1337,7 +1377,7 @@ function createServer(profile) {
     logger: new ConsoleLogger(),
     roots: { enabled: false },
     oauth: {
-      enabled: isMcpOAuthEnabled(),
+      enabled: isMcpOAuthEnabled() && profile.advertiseOAuth,
       protectedResource: {
         authorizationServers: [getOAuthIssuer()],
         bearerMethodsSupported: ["header"],
@@ -1369,30 +1409,107 @@ function isHostedKeylessSession(session) {
 function isLocalKeylessStartup() {
   return process.env.CLOUD_SERVICE !== "true" && !isHttpStreamingTransport() && !resolveCredentialFromEnv() && !normalizeHeader(process.env.FIRECRAWL_API_URL);
 }
+var NEVER_SHARE_API_KEY_SENTENCE = "Never ask for, accept, or put an API key in chat or in an MCP URL.";
+var RETRY_AFTER_CONFIRM_SUFFIX = "start a new client session or run and retry the original task.";
+var HUMAN_CONNECTION_GUIDANCE = `A human or operator must complete the connection because this session cannot reconnect Firecrawl itself. Ask the human to choose before changing anything: (1) in the MCP client's settings, update or replace the existing Firecrawl server entry so its URL is ${MCP_OAUTH_SERVER_URL}, then complete sign-in through the client. That URL is a client configuration value, not a page to open in a browser. Do not add a second Firecrawl server or change configuration without approval; or (2) have an operator create an API key at ${API_KEY_SIGNUP_URL} and configure it in the client or secret manager outside this chat. ${NEVER_SHARE_API_KEY_SENTENCE} After the human confirms setup, ${RETRY_AFTER_CONFIRM_SUFFIX} Connection guide: ${MCP_CONNECTION_GUIDE_URL}`;
+var ACCOUNT_ONLY_TOOL_GUIDANCE = `This tool needs a connected Firecrawl account. Search, Scrape, and Parse remain available, so continue with those if they can complete the task. Only if this task specifically requires this tool, tell the user that a human must update the existing Firecrawl connection using ${MCP_CONNECTION_GUIDE_URL}. Never ask for or accept an API key in chat. After setup, start a new client session and retry.`;
+var HUMAN_RECONNECT_ACCOUNT_ACTION = {
+  kind: "human_reconnect_account",
+  actor: "human",
+  requires_user_consent: true,
+  existing_server_only: true,
+  server_url: MCP_OAUTH_SERVER_URL,
+  open_server_url_in_browser: false,
+  docs_url: MCP_CONNECTION_GUIDE_URL
+};
+var OPERATOR_CONFIGURE_API_KEY_ACTION = {
+  kind: "operator_configure_api_key",
+  actor: "human_or_operator",
+  requires_user_consent: true,
+  credential_delivery: "outside_agent_chat",
+  signup_url: API_KEY_SIGNUP_URL
+};
+var HUMAN_CONNECTION_ACTIONS = [
+  HUMAN_RECONNECT_ACCOUNT_ACTION,
+  OPERATOR_CONFIGURE_API_KEY_ACTION
+];
+function connectionRecoveryPayload(params) {
+  return {
+    code: params.code,
+    auth_mode: params.authMode,
+    message: params.message,
+    docs_url: MCP_CONNECTION_GUIDE_URL,
+    next_actions: params.nextActions
+  };
+}
+function invalidApiKeyRecoveryPayload() {
+  return connectionRecoveryPayload({
+    code: "CREDENTIAL_INVALID",
+    authMode: "api_key",
+    message: `The Firecrawl API key configured for this server is invalid or revoked. Ask a human or operator to replace it in this existing server configuration or secret manager outside this chat. ${NEVER_SHARE_API_KEY_SENTENCE} After the human confirms the change, ${RETRY_AFTER_CONFIRM_SUFFIX}`,
+    nextActions: [OPERATOR_CONFIGURE_API_KEY_ACTION]
+  });
+}
+function invalidOAuthRecoveryPayload(profile) {
+  if (isMcpOAuthEnabled() && profile.advertiseOAuth) {
+    return connectionRecoveryPayload({
+      code: "OAUTH_CONNECTION_INVALID",
+      authMode: "oauth",
+      message: `This Firecrawl account connection is no longer valid. Ask the human to sign in again through this MCP client's account-connection flow. Do not add a second Firecrawl server or open the MCP server URL directly in a browser. After sign-in, ${RETRY_AFTER_CONFIRM_SUFFIX}`,
+      nextActions: [HUMAN_RECONNECT_ACCOUNT_ACTION, OPERATOR_CONFIGURE_API_KEY_ACTION]
+    });
+  }
+  return connectionRecoveryPayload({
+    code: "OAUTH_CONNECTION_INVALID",
+    authMode: "oauth",
+    message: `This Firecrawl account connection is no longer valid, and this server does not start account sign-in. Ask the human to choose before changing anything: (1) have an operator create a Firecrawl API key at ${API_KEY_SIGNUP_URL} and configure it on this existing server outside this chat; or (2) update this existing server's URL to ${MCP_OAUTH_SERVER_URL} and complete sign-in through the MCP client. That URL is a client configuration value, not a page to open in a browser. ${NEVER_SHARE_API_KEY_SENTENCE} After the human confirms the change, ${RETRY_AFTER_CONFIRM_SUFFIX}`,
+    nextActions: [OPERATOR_CONFIGURE_API_KEY_ACTION, HUMAN_RECONNECT_ACCOUNT_ACTION]
+  });
+}
 function recoveryPayload(code, requestId = randomUUID(), options = {}) {
   const retryAfterSeconds = options.retryAfterSeconds;
   const isQuotaExhausted = code === "KEYLESS_QUOTA_EXHAUSTED" || code === "KEYLESS_LIMIT_REACHED";
   const isToolUnavailable = code === "KEYLESS_TOOL_NOT_AVAILABLE";
   const isKeylessAccessUnavailable = code === "KEYLESS_ACCESS_NOT_AVAILABLE";
   const isKeylessEligibilityUnavailable = code === "KEYLESS_ELIGIBILITY_UNAVAILABLE";
-  const oauthUrl = "https://mcp.firecrawl.dev/v2/mcp-oauth";
   return {
     code,
     request_id: requestId,
     auth_mode: code === "CREDENTIAL_INVALID" ? "credential_error" : "keyless",
-    message: code === "CREDENTIAL_INVALID" ? "The supplied Firecrawl credential is invalid or revoked. Replace it or reconnect the account, then retry." : isQuotaExhausted ? `The free daily limit for this network has been reached${retryAfterSeconds ? `; try again in about ${retryAfterSeconds} seconds` : ""}. To continue now, connect a Firecrawl account via OAuth or configure an API key.` : isToolUnavailable ? "The free tier includes Search, Scrape, and Parse. This tool needs a connected account; Search, Scrape, and Parse still work, so no action is required." : isKeylessAccessUnavailable ? "Anonymous keyless access is unavailable for this request. To continue, connect a Firecrawl account via OAuth or configure an API key." : isKeylessEligibilityUnavailable ? "The anonymous keyless eligibility check is temporarily unavailable. Retry shortly." : "This tool requires a Firecrawl account or API key. Connect an account or configure Authorization: Bearer <FIRECRAWL_API_KEY>, then retry.",
+    message: code === "CREDENTIAL_INVALID" ? `The supplied Firecrawl credential is invalid or revoked. ${HUMAN_CONNECTION_GUIDANCE}` : isQuotaExhausted ? `The free daily limit for this network has been reached${retryAfterSeconds ? `; try again in about ${retryAfterSeconds} seconds` : ""}. To continue now: ${HUMAN_CONNECTION_GUIDANCE}` : isToolUnavailable ? ACCOUNT_ONLY_TOOL_GUIDANCE : isKeylessAccessUnavailable ? `Anonymous keyless access is unavailable for this request. To continue: ${HUMAN_CONNECTION_GUIDANCE}` : isKeylessEligibilityUnavailable ? "The anonymous keyless eligibility check is temporarily unavailable. Retry shortly." : `This tool requires a Firecrawl account or API key. ${HUMAN_CONNECTION_GUIDANCE}`,
     ...isKeylessAccessUnavailable ? {} : { available_tools: [...KEYLESS_TOOL_NAMES] },
-    docs_url: "https://docs.firecrawl.dev/mcp-server",
+    docs_url: MCP_CONNECTION_GUIDE_URL,
     ...retryAfterSeconds ? { retry_after_seconds: retryAfterSeconds } : {},
-    next_actions: isKeylessEligibilityUnavailable ? [{ kind: "retry_later", after_seconds: 30 }] : isQuotaExhausted || isKeylessAccessUnavailable ? [
-      { kind: "connect_oauth", url: oauthUrl, client_commands: [{ client: "claude-code", command: "claude mcp add --transport http firecrawl https://mcp.firecrawl.dev/v2/mcp-oauth" }] },
-      { kind: "configure_api_key", header: "Authorization: Bearer <FIRECRAWL_API_KEY>" }
-    ] : isToolUnavailable ? [{ kind: "continue_keyless", tools: [...KEYLESS_TOOL_NAMES] }] : [
-      { kind: "connect_oauth", url: oauthUrl },
-      { kind: "configure_api_key", header: "Authorization: Bearer <FIRECRAWL_API_KEY>" }
+    next_actions: isKeylessEligibilityUnavailable ? [{ kind: "retry_later", after_seconds: 30 }] : isQuotaExhausted || isKeylessAccessUnavailable ? HUMAN_CONNECTION_ACTIONS : isToolUnavailable ? [
+      { kind: "continue_keyless", tools: [...KEYLESS_TOOL_NAMES] },
+      ...HUMAN_CONNECTION_ACTIONS
+    ] : [
+      ...HUMAN_CONNECTION_ACTIONS
     ]
   };
 }
+function deprecatedExtractPayload() {
+  return {
+    code: "DEPRECATED_TOOL",
+    message: 'firecrawl_extract is deprecated and unavailable through MCP. For structured data from a known page, call firecrawl_scrape once per URL with formats: ["json"] and jsonOptions containing the prompt and schema. For unknown URLs or multi-source research, use firecrawl_search or firecrawl_agent first.',
+    replacement: {
+      name: "firecrawl_scrape",
+      instructions: 'Call once per known URL. Set formats to ["json"] and pass the extraction prompt and JSON schema in jsonOptions.',
+      example_arguments: {
+        url: "https://example.com/page",
+        formats: ["json"],
+        jsonOptions: {
+          prompt: "Extract the requested fields from this page.",
+          schema: {
+            type: "object",
+            properties: {}
+          }
+        }
+      }
+    },
+    docs_url: "https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor"
+  };
+}
 function emitActionLog(toolName, status, session, error, requestId = randomUUID(), code) {
   if (process.env.CLOUD_SERVICE !== "true") return;
   const payload = {
@@ -1428,22 +1545,39 @@ function emitActionLog(toolName, status, session, error, requestId = randomUUID(
 function guardHostedTool(tool, { logActions }) {
   const keylessTool = KEYLESS_TOOL_NAMES.has(tool.name);
   const execute = tool.execute;
+  const canList = tool.canList;
+  const beforeValidate = tool.beforeValidate;
   return {
     ...tool,
-    canList: (session) => !session?.credentialError && (!isHostedKeylessSession(session) || keylessTool),
-    beforeValidate: (_args, session) => {
+    canList: (session) => !session?.credentialError && (!isHostedKeylessSession(session) || keylessTool) && (canList?.(session) ?? true),
+    beforeValidate: async (args2, session) => {
       const code = session?.credentialError ? "CREDENTIAL_INVALID" : isHostedKeylessSession(session) && !keylessTool ? "KEYLESS_TOOL_NOT_AVAILABLE" : void 0;
-      if (!code) return void 0;
-      const requestId = randomUUID();
-      const payload = recoveryPayload(code, requestId);
-      if (logActions) {
-        emitActionLog(tool.name, "error", session, new UserError(String(payload.message), payload), requestId, code);
+      if (code) {
+        const requestId = randomUUID();
+        const payload2 = recoveryPayload(code, requestId);
+        if (logActions) {
+          emitActionLog(tool.name, "error", session, new UserError(String(payload2.message), payload2), requestId, code);
+        }
+        return {
+          content: [{ type: "text", text: String(payload2.message) }],
+          isError: true,
+          structuredContent: payload2
+        };
       }
-      return {
-        content: [{ type: "text", text: String(payload.message) }],
-        isError: true,
-        structuredContent: payload
-      };
+      const earlyResult = await beforeValidate?.(args2, session);
+      const payload = earlyResult?.structuredContent;
+      const recoveryCode = payload && typeof payload === "object" && "code" in payload && typeof payload.code === "string" ? payload.code : void 0;
+      if (logActions && earlyResult?.isError && recoveryCode) {
+        emitActionLog(
+          tool.name,
+          "error",
+          session,
+          new UserError(`Tool validation failed: ${recoveryCode}`, payload),
+          randomUUID(),
+          recoveryCode
+        );
+      }
+      return earlyResult;
     },
     execute: async (args2, context) => {
       const requestId = randomUUID();
@@ -1734,7 +1868,7 @@ var parseOptionParamsSchema = z4.object({
   skipTlsVerification: z4.boolean().optional(),
   storeInCache: z4.boolean().optional(),
   zeroDataRetention: z4.boolean().optional(),
-  maxAge: z4.number().optional(),
+  maxAge: z4.number().optional().describe("Ignored: parse never reuses or stores indexed content."),
   proxy: z4.enum(["basic", "auto"]).optional()
 });
 var localParseParamsSchema = parseOptionParamsSchema.extend({
@@ -1972,6 +2106,8 @@ Retrieve and extract content from one supplied URL through Firecrawl. Use this w
 
 This tool operates on a known page. For a set of pages use \`firecrawl_crawl\`, and to discover page URLs use \`firecrawl_map\` or \`firecrawl_search\`. Options include JavaScript render delay, cache age, main-content filtering, PII redaction, and lockdown cache-only retrieval. Browser actions may change the live page when interactive actions are enabled.
 
+Firecrawl may reuse recently indexed content instead of refetching the page, and the reuse window varies by domain. Set \`maxAge: 0\` to force a live fetch, or a smaller \`maxAge\` to bound how stale reused content may be. A successful response does not by itself confirm that the state it describes is still current.
+
 Returns the selected content formats and page metadata.
 `,
   parameters: scrapeParamsSchema,
@@ -2058,7 +2194,7 @@ Search web, news, or image sources and return ranked results. Operators include
 
 For a programming question, add \`categories: ["developer"]\`. It searches an index of GitHub issues, merged pull requests, repository READMEs, and curated documentation sites, and returns the hits in \`data.developer\` beside the web results.
 
-\`scrapeOptions\` can attach extracted page content. Returns source-type result groups and usage metadata. Authenticated responses can include an \`id\` for optional search feedback.
+\`scrapeOptions\` can attach extracted page content; pages fetched this way use a fixed reuse window and ignore \`maxAge\`, so use \`firecrawl_scrape\` when a live fetch is required. Returns source-type result groups and usage metadata. Authenticated responses can include an \`id\` for optional search feedback.
 `,
   parameters: z4.object({
     ...searchToolBaseFields,
@@ -2576,18 +2712,13 @@ Retrieve the current status, progress, and available results for an existing cra
 server.addTool({
   name: "firecrawl_extract",
   annotations: {
-    title: "Extract structured data",
+    title: "Deprecated: use Scrape JSON",
     readOnlyHint: true,
-    // Uses LLM extraction to pull structured data from URLs without modifying those sites.
     openWorldHint: true,
-    // Accepts arbitrary user-supplied URLs on the public web.
     destructiveHint: false
-    // Read-only extraction; no destructive changes to external content.
   },
   description: `
-Extract structured information from one or more URLs with an optional natural-language prompt and JSON schema. It can include subdomains, follow external links, or use web search when those options are enabled.
-
-Use this for a defined structured result rather than full page content. Returns data shaped by the supplied schema or prompt.
+Deprecated compatibility entry point. Use firecrawl_scrape once per known URL with formats: ["json"] and jsonOptions containing the prompt and schema. Use firecrawl_search or firecrawl_agent before Scrape when URLs are not known.
 `,
   parameters: z4.object({
     urls: z4.array(z4.string()),
@@ -2597,23 +2728,18 @@ Use this for a defined structured result rather than full page content. Returns
     enableWebSearch: z4.boolean().optional(),
     includeSubdomains: z4.boolean().optional()
   }),
-  execute: async (args2, { session, log }) => {
-    const client = getClient(session);
-    const a = args2;
-    log.info("Extracting from URLs", {
-      count: Array.isArray(a.urls) ? a.urls.length : 0
-    });
-    const extractBody = removeEmptyTopLevel({
-      urls: a.urls,
-      prompt: a.prompt,
-      schema: a.schema || void 0,
-      allowExternalLinks: a.allowExternalLinks,
-      enableWebSearch: a.enableWebSearch,
-      includeSubdomains: a.includeSubdomains,
-      origin: ORIGIN
-    });
-    const res = await client.extract(extractBody);
-    return asText2(res);
+  canList: () => false,
+  beforeValidate: () => {
+    const payload = deprecatedExtractPayload();
+    return {
+      content: [{ type: "text", text: payload.message }],
+      isError: true,
+      structuredContent: payload
+    };
+  },
+  execute: async () => {
+    const payload = deprecatedExtractPayload();
+    throw new UserError(payload.message, payload);
   }
 });
 server.addTool({

package/dist/www-authenticate.js

@@ -0,0 +1,7 @@
+// src/www-authenticate.ts
+function escapeWWWAuthenticateValue(value) {
+  return value.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
+}
+export {
+  escapeWWWAuthenticateValue
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "firecrawl-mcp",
-  "version": "3.23.4",
+  "version": "3.23.7",
   "description": "MCP server for Firecrawl — search, scrape, and interact with the web. Supports both cloud and self-hosted instances. Features include web search, scraping, page interaction, batch processing, and LLM-powered content analysis.",
   "type": "module",
   "mcpName": "io.github.firecrawl/firecrawl-mcp-server",