firecrawl-mcp 3.27.2 → 3.27.3

package/README.md

@@ -419,6 +419,7 @@ Scrape content from a single URL with advanced options.
 
 **Branding format:** Extracts comprehensive brand identity (colors, fonts, typography, spacing, logo, UI components) for design analysis or style replication.
 **Privacy:** Set `redactPII: true` to return content with personally identifiable information redacted.
+**Hosted server:** On the hosted server (`CLOUD_SERVICE=true`) scrape is read-only. It takes no browser `actions` and cannot accept provider terms. A named `profile` loads saved browser state without saving changes to it. An organization admin accepts terms in the dashboard.
 
 **Returns:**
 
@@ -747,17 +748,17 @@ The agent performs web searches, follows links, reads pages, and gathers data au
   - `enabled`, `toolkits` (up to 5 provider slugs), `maxCalls` (1 to 30), `requireApproval` (paid calls end the turn with a `pendingApproval`; needs `mode: "chat"` on the same call, even on a follow-up)
   - `onTermsRequired`: what to do when an Alexandria provider the agent would use needs data terms your team has not accepted. Gated providers are never called in any mode. Omitted on a follow-up keeps the previous turn's value.
     - `"skip"` (default): answer with accepted providers only. `exchange.skippedProviders` on the status result lists the gated providers that would have helped.
-    - `"ask"`: the same, plus a terms `pendingApproval` and `exchange.requiresAction` with the exact `terms/show` and `terms/accept` calls for each provider. Each provider's `digest` is always present and is `string | null`; when it is `null`, `terms/show` returns the current digest to send.
+    - `"ask"`: the same, plus a terms `pendingApproval` and `exchange.requiresAction` with the approval ID and provider requirements. Read terms with `terms/show`; an organization admin accepts them in the Firecrawl dashboard.
   - `approve`: `{ approvalId, callIds?, always? }` answers yes to the `pendingApproval` the previous turn ended on. `callIds` and `always` apply to paid-call approvals only.
   - `decline`: `{ approvalId }` answers no. A declined terms offer keeps those providers out of the rest of the thread.
   - `approve` and `decline` need `threadId`, and only one of them can be sent.
 
-**Provider terms (ask mode):** there is no auto-accept mode. When a turn ends on a terms offer, the status result carries `pendingApproval` (`kind: "terms"`) and `exchange.requiresAction` with the `approvalId` and the exact `terms/show` and `terms/accept` calls. To use the provider:
+**Provider terms (ask mode):** there is no auto-accept mode. When a turn ends on a terms offer, the status result carries `pendingApproval` (`kind: "terms"`) and `exchange.requiresAction` with the `approvalId` and provider requirements. Any `terms/accept` descriptor in that API payload is unavailable through MCP. To use the provider:
 
 1. Show the user the terms (`terms/show` through `firecrawl_scrape` with `alexandria`).
-2. Get the user's explicit consent to that provider's terms. A data request is not consent.
-3. Run the `terms/accept` call through `firecrawl_scrape`.
-4. Continue the same thread: call `firecrawl_agent` with the same `threadId` and `exchange.approve: { "approvalId": "..." }`.
+2. Direct an organization admin to accept the terms at the provider's URL, or [data sources settings](https://www.firecrawl.dev/app/settings?tab=data-sources). A data request is not consent.
+3. Wait for the admin to confirm acceptance in the dashboard.
+4. Continue the same thread: call `firecrawl_agent` with the same `threadId` and `exchange.approve: { "approvalId": "..." }`. This resumes research and does not accept terms.
 
 If the user says no, call `firecrawl_agent` with the same `threadId` and `exchange.decline: { "approvalId": "..." }` instead.
 
@@ -818,13 +819,13 @@ Then poll with `firecrawl_agent_status` using the returned job ID.
 }
 ```
 
-**Usage Example (continue the thread after the user accepted a provider's terms):**
+**Usage Example (continue the thread after an admin confirmed dashboard acceptance):**
 
 ```json
 {
   "name": "firecrawl_agent",
   "arguments": {
-    "prompt": "I accepted the Apollo terms. Continue.",
+    "prompt": "The admin confirmed acceptance of the Apollo terms in the dashboard. Continue.",
     "threadId": "0199a1b2-0000-7000-8000-000000000031",
     "exchange": { "approve": { "approvalId": "0199a1b2-0000-7000-8000-000000000033" } }
   }
@@ -1113,16 +1114,14 @@ HTTP 403 and this body:
 The tool result relays it as an error with `structuredContent` carrying `code`,
 `status: 403`, `requestId`, the `requiresAction` object unchanged, and
 `next_actions` (`human_action_required` then `retry_same_request`). Accepting
-terms is a legal act. Use the returned `nextTool` call to read the agreement through `firecrawl_scrape`
-with `alexandria: [{provider: "firecrawl", capability: "terms/show", options: {provider: "<provider>"}}]`.
-Present it to the user and obtain explicit authorization to bind their organization
-before calling `firecrawl_scrape` with capability `terms/accept` under provider `firecrawl`.
-Its options are `provider`, the exact reviewed `version`
-and 64-character lowercase hexadecimal `digest`, and `confirmed: true`. A request
-for data is not consent. Authority or eligibility errors may require an organization
-admin to use `requiresAction.url`. No automatic acceptance or uncertain retries occur.
-Send terms calls separately from execution. These are nested capabilities, not top-level MCP tools.
-No credits are charged for the blocked retrieval. After confirmed acceptance, call the same
+terms is a legal act, so an organization admin accepts them in the Firecrawl dashboard, not
+through MCP. Use the returned `nextTool` call to read the agreement through `firecrawl_scrape`
+with `alexandria: [{provider: "firecrawl", capability: "terms/show", options: {provider: "<provider>"}}]`,
+sent separately from provider execution, and present it to the user. An organization admin then
+accepts it at `requiresAction.url`, or at https://www.firecrawl.dev/app/settings?tab=data-sources.
+`firecrawl_scrape` refuses every other `terms/*` capability, so it makes no account changes. A request
+for data is not consent, and no automatic acceptance or uncertain retries occur.
+No credits are charged for the blocked retrieval. After the admin confirms acceptance, call the same
 tool again with the identical payload and `requestId`.
 
 ### 16. Credit Usage Tool
@@ -1288,7 +1287,7 @@ The existing `firecrawl_feedback` tool accepts `endpoint: "alexandria"`:
 }
 ```
 
-`objective` is the underlying goal of the session: what the agent or its user was ultimately trying to accomplish, beyond the single website in `requestedWebsite`.
+The optional `objective` is the underlying goal of the session: what the agent or its user was ultimately trying to accomplish, beyond the single website in `requestedWebsite`.
 
 This uses authenticated `POST /v2/feedback`, without a job ID, job-age deadline, or credit refund. Optional `providerFeedback` and `capabilityFeedback` arrays describe coverage gaps and execution issues; the tool schema lists supported issue values. A `new_capability_request` requires `requestedFunctionality`; `missing_capability` (the provider exists but lacks the capability) does not. Existing feedback opt-out and authentication controls apply.
 

package/dist/index.js

@@ -76,8 +76,7 @@ var alexandriaSessionFeedbackSchema = z.strictObject({
   rating: z.enum(["good", "bad", "partial"]),
   ...alexandriaFeedbackFields,
   requestedWebsite: alexandriaFeedbackFields.requestedWebsite.unwrap(),
-  rationale: detail,
-  objective: detail
+  rationale: detail
 });
 
 // src/index.ts
@@ -2271,6 +2270,10 @@ var agentStatusOutputSchema = z4.object({
   status: str("Job state: `processing` is non-terminal; `completed` and `failed` are terminal."),
   progress: unknown("Progress information for a running job."),
   data: unknown("Research result, present once the job has completed."),
+  partial: unknown("Incomplete best-effort result of a `failed` run; never a completed answer."),
+  partialSchemaValid: bool("Whether `partial` matches the requested schema, when one was supplied."),
+  stopReason: str("Why a `failed` run stopped early; `credit_limit_reached` means it hit maxCredits."),
+  notice: str("Set when the run stopped at its credit limit: what that means and how to continue."),
   creditsUsed: num("Credits the job consumed."),
   sources: unknown("Sources the agent read."),
   expiresAt: str("When the job result expires and can no longer be read."),
@@ -2281,10 +2284,10 @@ var agentStatusOutputSchema = z4.object({
   message: unknown("The agent's reply; in chat mode, the short answer to a follow-up."),
   suggestions: unknown("Follow-ups the agent offers; send one as the prompt of the next turn with this threadId."),
   exchange: unknown(
-    "What the job did with Alexandria providers: onTermsRequired, paidCalls, creditsUsed, skippedProviders (gated providers that would have helped), and requiresAction (approvalId plus the terms/show and terms/accept calls, each provider digest string | null and always present; call accept only with the user's explicit consent, then continue the thread with exchange.approve: {approvalId})."
+    "What the job did with Alexandria providers: onTermsRequired, paidCalls, creditsUsed, skippedProviders (gated providers that would have helped), and requiresAction (approvalId and provider requirements). Read terms with terms/show. Ignore any terms/accept call in this API payload: an organization admin accepts terms in the Firecrawl dashboard, then confirms before the thread resumes with exchange.approve: {approvalId}."
   ),
   pendingApproval: unknown(
-    'Set when the job ended waiting on the caller. Answer it by calling firecrawl_agent with this threadId and exchange.approve or exchange.decline carrying its id. kind "terms" lists providers whose data terms need accepting; otherwise calls lists paid calls waiting for approval.'
+    'Set when the job ended waiting on the caller. Answer it by calling firecrawl_agent with this threadId and exchange.approve or exchange.decline carrying its id. kind "terms" lists providers whose data terms need acceptance by an organization admin in the Firecrawl dashboard; approve only after the admin confirms, and approval does not accept terms. Otherwise calls lists paid calls waiting for approval.'
   )
 }).describe("Progress or final result of a research agent job.");
 var interactOutputSchema = z4.object({
@@ -4155,6 +4158,8 @@ var searchDomainSchema = z10.string().trim().toLowerCase().min(1).max(253).regex
 var searchToolBaseFields = {
   toolDetail: z10.enum(["compact", "summary", "full"]).optional().describe("Compact by default. Compact returns only provider, capability and description; full includes contracts. Inspect selected compact tools with firecrawl_find_tools providers and capabilities."),
   query: z10.string().min(1).describe("Query for web and semantic tool discovery. Operators include quoted phrases, `-term`, `site:host`, `inurl:term`, `intitle:term`, and `related:host`; the set is non-exhaustive. Catalogue browsing is available through firecrawl_find_tools."),
+  objective: z10.string().trim().min(1).max(5e3).optional().describe("Optional broader goal for this search, if known. Avoid sensitive information."),
+  clientModel: z10.string().trim().min(1).max(128).optional().describe("Optional model identifier, if known."),
   domainTools: z10.boolean().optional().describe(
     "Include domain-matched tools for result URLs. Defaults to true when Alexandria is combined with web, news or images; semantic-only search leaves domain matching off."
   ),
@@ -4198,6 +4203,15 @@ function assertExchangeCredential(session) {
   throw new UserError(payload.message, payload);
 }
 var TERMS_REQUIRED_CODE = "THIRD_PARTY_DATA_TERMS_REQUIRED";
+var DATA_SOURCES_SETTINGS_URL = "https://www.firecrawl.dev/app/settings?tab=data-sources";
+function isTermsWrite(call) {
+  const capability = call.capability.trim().toLowerCase();
+  return call.provider.trim().toLowerCase() === "firecrawl" && capability.startsWith("terms/") && capability !== "terms/show";
+}
+function termsWriteError() {
+  const message = `Provider terms are accepted in the Firecrawl dashboard, not through this connection. Read them with terms/show, then ask an organization admin to accept them at ${DATA_SOURCES_SETTINGS_URL}.`;
+  return new UserError(message, { code: "invalid_option", status: 400, message });
+}
 function termsRequiredAction(body) {
   const data = body;
   if (data?.code !== TERMS_REQUIRED_CODE) return void 0;
@@ -4218,8 +4232,8 @@ function termsRequiredError(action, context, hints) {
   return new UserError(
     `${message}
 
-1. Read the agreement with firecrawl_scrape using alexandria: {provider: "firecrawl", capability: "terms/show", options: {provider: "${action.terms}"}} and present it to the user. Only after explicit authorization to accept that exact version and digest, use firecrawl_scrape with alexandria: {provider: "firecrawl", capability: "terms/accept", options: {provider: "${action.terms}", version: "<reviewed-version>", digest: "<reviewed-digest>", confirmed: true}}. Send terms calls separately from provider execution. If authority or eligibility requires a dashboard action, ask an organization admin to visit ${action.url} or https://www.firecrawl.dev/app/settings?tab=data-sources if that page is unavailable. Never infer acceptance from a data request.
-2. After they confirm, call ${context.tool} again with the identical payload${retryIdentity}. If the same terms error persists, stop and ask an organization admin to check access at https://www.firecrawl.dev/app/settings?tab=data-sources.
+1. Read the agreement with firecrawl_scrape using alexandria: {provider: "firecrawl", capability: "terms/show", options: {provider: "${action.terms}"}} and present it to the user. Send terms/show separately from provider execution. Terms are accepted in the Firecrawl dashboard, not through this connection: ask an organization admin to accept them at ${action.url}, or ${DATA_SOURCES_SETTINGS_URL} if that page is unavailable. Never infer acceptance from a data request.
+2. After the admin confirms, call ${context.tool} again with the identical payload${retryIdentity}. If the same terms error persists, stop and ask an organization admin to check access at ${DATA_SOURCES_SETTINGS_URL}.
 
 Do not retry until acceptance is confirmed.`,
     {
@@ -4281,7 +4295,7 @@ async function relayExchangeError(run, context) {
     const disabledProvider = response.status === 403 ? context.providers?.find((provider) => message === `Access to ${provider} is disabled for this organization.`) : void 0;
     if (disabledProvider) {
       throw new UserError(
-        `${message} Read the provider terms and status using nextTool and present them to the user. Never infer acceptance from a data request. Only after explicit authorization for the reviewed version and digest may you call firecrawl_scrape with alexandria:{provider:"firecrawl",capability:"terms/accept",options:{provider:"${disabledProvider}",version:"<reviewed-version>",digest:"<reviewed-digest>",confirmed:true}}. Send terms calls separately. Acceptance may not restore disabled access; an organization admin may need to review https://www.firecrawl.dev/app/settings?tab=data-sources. Retry the original request only after access is restored.`,
+        `${message} Read the provider terms and status using nextTool and present them to the user. Never infer acceptance from a data request. Terms are accepted in the Firecrawl dashboard, not through this connection; acceptance may not restore disabled access, and an organization admin can review access at ${DATA_SOURCES_SETTINGS_URL}. Retry the original request only after access is restored.`,
         {
           code: typeof data?.code === "string" ? data.code : "exchange_error",
           status: 403,
@@ -5321,8 +5335,9 @@ var scrapeTool = {
   name: "firecrawl_scrape",
   annotations: {
     title: "Firecrawl scrape",
-    readOnlyHint: false,
-    // Alexandria capabilities can record provider agreement acceptance.
+    // Hosted scrape omits browser actions and refuses provider terms writes
+    // before execution.
+    readOnlyHint: SAFE_MODE,
     openWorldHint: true,
     // Accepts any user-supplied URL on the public web.
     destructiveHint: false
@@ -5349,6 +5364,9 @@ Alexandria mode, on an authenticated session with Alexandria access: \`alexandri
     } = args2;
     if (alexandria) {
       assertExchangeCredential(session);
+      if ((Array.isArray(alexandria) ? alexandria : [alexandria]).some(isTermsWrite)) {
+        throw termsWriteError();
+      }
       log.info("Executing Alexandria capabilities", {
         count: Array.isArray(alexandria) ? alexandria.length : 1
       });
@@ -5596,7 +5614,7 @@ var findToolsTool = {
 };
 server.addTool(findToolsTool);
 var SEARCH_SURFACE_SCRAPE_DESCRIPTION = `
-Scrape one URL and return its content, or execute catalogued Alexandria capabilities. URL mode returns markdown by default, or HTML, links, screenshots, branding data, a targeted answer, or JSON matching a supplied schema, plus page metadata. Firecrawl may serve recently indexed content; set \`maxAge: 0\` for a live fetch. A successful response does not by itself confirm the page is still current. Browser actions can change the live page when interactive actions are enabled, and a named browser profile can load saved session data and overwrite its stored state.
+Scrape one URL and return its content, or execute catalogued Alexandria capabilities. URL mode returns markdown by default, or HTML, links, screenshots, branding data, a targeted answer, or JSON matching a supplied schema, plus page metadata. Firecrawl may serve recently indexed content; set \`maxAge: 0\` for a live fetch. A successful response does not by itself confirm the page is still current. A named browser profile loads saved session data.
 
 \`firecrawl_search\` with \`sources\` unset and \`firecrawl_find_tools\` can discover providers for the same fields across several pages; a matching Alexandria provider returns typed records in one call.
 
@@ -6206,13 +6224,13 @@ var agentExchangeSchema = z10.strictObject({
     callIds: z10.array(z10.string()).optional(),
     always: z10.boolean().optional()
   }).optional().describe(
-    "Answer yes to the pendingApproval the previous turn of this thread ended on (its id, also exchange.requiresAction.approvalId). Needs threadId. For a terms offer, send it only after the user explicitly agreed and terms/accept succeeded; callIds and always are ignored on terms offers. For paid calls, callIds picks a subset (default all) and always stops asking for the rest of the thread."
+    "Answer yes to the pendingApproval the previous turn of this thread ended on (its id, also exchange.requiresAction.approvalId). Needs threadId. For a terms offer, send it only after an organization admin confirms acceptance in the Firecrawl dashboard; this does not accept terms. callIds and always are ignored on terms offers. For paid calls, callIds picks a subset (default all) and always stops asking for the rest of the thread."
   ),
   decline: z10.strictObject({ approvalId: z10.string().uuid() }).optional().describe(
     "Answer no to that pendingApproval. Needs threadId. A declined terms offer keeps those providers out of the rest of the thread."
   ),
   onTermsRequired: z10.enum(["skip", "ask"]).optional().describe(
-    `What to do when a provider the agent would use needs data terms the team has not accepted. Gated providers are never called. "skip" (default): answer with accepted providers and list the rest in exchange.skippedProviders. "ask": the same, plus a terms pendingApproval and exchange.requiresAction with the terms/show and terms/accept calls. Each provider digest is string | null and always present; when null, terms/show returns it. There is no auto-accept. Omitted on a follow-up keeps the previous turn's value.`
+    `What to do when a provider the agent would use needs data terms the team has not accepted. Gated providers are never called. "skip" (default): answer with accepted providers and list the rest in exchange.skippedProviders. "ask": the same, plus a terms pendingApproval and exchange.requiresAction. Read terms with terms/show; an organization admin accepts them in the Firecrawl dashboard. There is no auto-accept. Omitted on a follow-up keeps the previous turn's value.`
   )
 }).describe(
   "Alexandria provider settings for this turn, forwarded as the request's exchange object."
@@ -6235,7 +6253,7 @@ This call returns only a job ID, not the research result. Read the job with \`fi
 
 The job also returns a \`threadId\`. To continue that thread, pass it with a follow-up \`prompt\`; omitted \`mode\`, \`urls\`, \`schema\` and exchange settings carry over from the previous turn.
 
-The agent only calls Alexandria providers whose data terms the team has accepted. The status result's \`exchange.skippedProviders\` lists gated providers that would have helped. With \`exchange.onTermsRequired\` "ask", a terms offer ends the turn: \`pendingApproval\` (kind "terms") and \`exchange.requiresAction\` carry the \`approvalId\` and the exact terms/show and terms/accept calls. Show the user the terms, get their EXPLICIT consent, run terms/accept through \`firecrawl_scrape\`, then call \`firecrawl_agent\` with the same \`threadId\` and \`exchange.approve: {approvalId}\`. If they decline, send \`exchange.decline: {approvalId}\` instead. Never call terms/accept without that consent; a data request is not consent.
+The agent only calls Alexandria providers whose terms the team has accepted. \`exchange.skippedProviders\` lists gated providers. With \`exchange.onTermsRequired\` "ask", a terms \`pendingApproval\` and \`exchange.requiresAction\` carry the \`approvalId\` and provider requirements. Read terms/show through \`firecrawl_scrape\`. An organization admin must accept terms in the Firecrawl dashboard at the provider URL or ${DATA_SOURCES_SETTINGS_URL}. Ignore any terms/accept call in the API response. Only after the admin confirms acceptance, resume with the same \`threadId\` and \`exchange.approve: {approvalId}\`; this does not accept terms. To decline, use \`exchange.decline: {approvalId}\`. Never infer acceptance from a data request.
 `,
   outputSchema: agentOutputSchema,
   parameters: z10.object({
@@ -6331,9 +6349,41 @@ Returns job status, progress information, and result data when completed.
       `/v2/agent/${encodeURIComponent(id)}`,
       originHeaders(requestOrigin(mcpClient, session))
     );
-    return structuredText(res?.data ?? {});
+    return agentStatusResult(res?.data ?? {});
   }
 });
+function agentStatusResult(data) {
+  const run = data;
+  if (!run || typeof run !== "object" || run.status !== "failed" || run.stopReason !== "credit_limit_reached") {
+    return structuredText(data);
+  }
+  const lines = [
+    "This agent run stopped at its credit limit (maxCredits) before finishing."
+  ];
+  const hasPartial = run.partial !== void 0 && run.partial !== null;
+  if (hasPartial) {
+    const validity = run.partialSchemaValid === true ? " It matches the schema this run was given." : run.partialSchemaValid === false ? " It does not match the schema this run was given." : "";
+    lines.push(
+      `\`partial\` is an INCOMPLETE best-effort result, not a finished answer; tell the user it is incomplete.${validity}`
+    );
+  } else {
+    lines.push("No partial result was recovered.");
+  }
+  if (typeof run.message === "string" && run.message.trim()) {
+    lines.push(`Agent message: ${run.message.trim()}`);
+  }
+  const threadId = typeof run.threadId === "string" ? run.threadId : void 0;
+  lines.push(
+    threadId ? `To continue, call firecrawl_agent with threadId "${threadId}" and a prompt to finish the task (the follow-up ${hasPartial ? "resumes from this partial" : "keeps this thread"}), or start a new run with a higher maxCredits.` : "To continue, start a new firecrawl_agent run with a higher maxCredits."
+  );
+  const notice = lines.join(" ");
+  return withStructured(
+    `${notice}
+
+${JSON.stringify(data, null, 2)}`,
+    { ...run, notice }
+  );
+}
 server.addTool({
   name: "firecrawl_interact",
   annotations: {
@@ -6568,6 +6618,8 @@ Returns result groups in \`data\` and an operation \`id\`.
     execute: async (args2, { session, log, client: mcpClient }) => {
       const {
         query,
+        objective,
+        clientModel,
         includeDomains,
         excludeDomains,
         limit,
@@ -6585,6 +6637,8 @@ Returns result groups in \`data\` and an operation \`id\`.
       const searchBody = {
         query: searchQuery,
         ...removeEmptyTopLevel({
+          objective,
+          clientModel,
           limit,
           includeDomains,
           excludeDomains,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "firecrawl-mcp",
-  "version": "3.27.2",
+  "version": "3.27.3",
   "description": "MCP server for Firecrawl — search, scrape, and interact with the web, and search scientific papers. Supports both cloud and self-hosted instances. Features include web search, scraping, page interaction, batch processing, LLM-powered content analysis, and research paper search over biomedical and arXiv literature (PubMed, bioRxiv, medRxiv, arXiv) with citation-graph expansion and full-text reading.",
   "type": "module",
   "mcpName": "io.github.firecrawl/firecrawl-mcp-server",