firecrawl-mcp 3.21.4 → 3.22.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +134 -199
  2. package/dist/index.js +122 -27
  3. package/package.json +1 -1
package/README.md CHANGED
@@ -26,6 +26,24 @@ A Model Context Protocol (MCP) server that brings [Firecrawl](https://github.com
26
26
 
27
27
  ## Installation
28
28
 
29
+ ### Hosted MCP (keyless free tier)
30
+
31
+ Connect to the remote hosted server with no setup:
32
+
33
+ ```
34
+ https://mcp.firecrawl.dev/v2/mcp
35
+ ```
36
+
37
+ On the keyless free tier, `scrape`, `search`, and `interact` work without an API key (rate-limited). Other tools such as `crawl`, `map`, `agent`, and `extract` still need a key.
38
+
39
+ Prefer an API key or OAuth whenever the human can sign up. It unlocks the full tool set and higher limits. With a key, use:
40
+
41
+ ```
42
+ https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp
43
+ ```
44
+
45
+ See the [MCP server docs](https://docs.firecrawl.dev/mcp-server) and the [agent onboarding guide](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for setup details.
46
+
29
47
  ### Running with npx
30
48
 
31
49
  ```bash
@@ -195,37 +213,12 @@ Hosted Firecrawl can issue OAuth **access tokens** (`fco_…`) via the authoriza
195
213
 
196
214
  Use **access** tokens (`fco_…`) only. Refresh tokens (`fcr_…`) must be exchanged at the token endpoint, not passed to the scrape/search API.
197
215
 
198
- #### Optional Configuration
199
-
200
- ##### Retry Configuration
201
-
202
- - `FIRECRAWL_RETRY_MAX_ATTEMPTS`: Maximum number of retry attempts (default: 3)
203
- - `FIRECRAWL_RETRY_INITIAL_DELAY`: Initial delay in milliseconds before first retry (default: 1000)
204
- - `FIRECRAWL_RETRY_MAX_DELAY`: Maximum delay in milliseconds between retries (default: 10000)
205
- - `FIRECRAWL_RETRY_BACKOFF_FACTOR`: Exponential backoff multiplier (default: 2)
206
-
207
- ##### Credit Usage Monitoring
208
-
209
- - `FIRECRAWL_CREDIT_WARNING_THRESHOLD`: Credit usage warning threshold (default: 1000)
210
- - `FIRECRAWL_CREDIT_CRITICAL_THRESHOLD`: Credit usage critical threshold (default: 100)
211
-
212
216
  ### Configuration Examples
213
217
 
214
- For cloud API usage with custom retry and credit monitoring:
218
+ For cloud API usage:
215
219
 
216
220
  ```bash
217
- # Required for cloud API
218
221
  export FIRECRAWL_API_KEY=your-api-key
219
-
220
- # Optional retry configuration
221
- export FIRECRAWL_RETRY_MAX_ATTEMPTS=5 # Increase max retry attempts
222
- export FIRECRAWL_RETRY_INITIAL_DELAY=2000 # Start with 2s delay
223
- export FIRECRAWL_RETRY_MAX_DELAY=30000 # Maximum 30s delay
224
- export FIRECRAWL_RETRY_BACKOFF_FACTOR=3 # More aggressive backoff
225
-
226
- # Optional credit monitoring
227
- export FIRECRAWL_CREDIT_WARNING_THRESHOLD=2000 # Warning at 2000 credits
228
- export FIRECRAWL_CREDIT_CRITICAL_THRESHOLD=500 # Critical at 500 credits
229
222
  ```
230
223
 
231
224
  For self-hosted instance:
@@ -236,10 +229,6 @@ export FIRECRAWL_API_URL=https://firecrawl.your-domain.com
236
229
 
237
230
  # Optional authentication for self-hosted
238
231
  export FIRECRAWL_API_KEY=your-api-key # If your instance requires auth
239
-
240
- # Custom retry configuration
241
- export FIRECRAWL_RETRY_MAX_ATTEMPTS=10
242
- export FIRECRAWL_RETRY_INITIAL_DELAY=500 # Start with faster retries
243
232
  ```
244
233
 
245
234
  ### Usage with Claude Desktop
@@ -253,96 +242,43 @@ Add this to your `claude_desktop_config.json`:
253
242
  "command": "npx",
254
243
  "args": ["-y", "firecrawl-mcp"],
255
244
  "env": {
256
- "FIRECRAWL_API_KEY": "YOUR_API_KEY_HERE",
257
-
258
- "FIRECRAWL_RETRY_MAX_ATTEMPTS": "5",
259
- "FIRECRAWL_RETRY_INITIAL_DELAY": "2000",
260
- "FIRECRAWL_RETRY_MAX_DELAY": "30000",
261
- "FIRECRAWL_RETRY_BACKOFF_FACTOR": "3",
262
-
263
- "FIRECRAWL_CREDIT_WARNING_THRESHOLD": "2000",
264
- "FIRECRAWL_CREDIT_CRITICAL_THRESHOLD": "500"
245
+ "FIRECRAWL_API_KEY": "YOUR_API_KEY_HERE"
265
246
  }
266
247
  }
267
248
  }
268
249
  }
269
250
  ```
270
251
 
271
- ### System Configuration
272
-
273
- The server includes several configurable parameters that can be set via environment variables. Here are the default values if not configured:
274
-
275
- ```typescript
276
- const CONFIG = {
277
- retry: {
278
- maxAttempts: 3, // Number of retry attempts for rate-limited requests
279
- initialDelay: 1000, // Initial delay before first retry (in milliseconds)
280
- maxDelay: 10000, // Maximum delay between retries (in milliseconds)
281
- backoffFactor: 2, // Multiplier for exponential backoff
282
- },
283
- credit: {
284
- warningThreshold: 1000, // Warn when credit usage reaches this level
285
- criticalThreshold: 100, // Critical alert when credit usage reaches this level
286
- },
287
- };
288
- ```
289
-
290
- These configurations control:
291
-
292
- 1. **Retry Behavior**
293
-
294
- - Automatically retries failed requests due to rate limits
295
- - Uses exponential backoff to avoid overwhelming the API
296
- - Example: With default settings, retries will be attempted at:
297
- - 1st retry: 1 second delay
298
- - 2nd retry: 2 seconds delay
299
- - 3rd retry: 4 seconds delay (capped at maxDelay)
300
-
301
- 2. **Credit Usage Monitoring**
302
- - Tracks API credit consumption for cloud API usage
303
- - Provides warnings at specified thresholds
304
- - Helps prevent unexpected service interruption
305
- - Example: With default settings:
306
- - Warning at 1000 credits remaining
307
- - Critical alert at 100 credits remaining
308
-
309
- ### Rate Limiting and Batch Processing
310
-
311
- The server utilizes Firecrawl's built-in rate limiting and batch processing capabilities:
312
-
313
- - Automatic rate limit handling with exponential backoff
314
- - Efficient parallel processing for batch operations
315
- - Smart request queuing and throttling
316
- - Automatic retries for transient errors
317
-
318
252
  ## How to Choose a Tool
319
253
 
320
254
  Use this guide to select the right tool for your task:
321
255
 
322
- - **If you know the exact URL(s) you want:**
323
- - For one: use **scrape** (with JSON format for structured data)
324
- - For many: use **batch_scrape**
256
+ - **If you know the exact URL you want:** use **scrape** (with JSON format for structured data)
257
+ - **If you have multiple known URLs:** call **scrape** for each URL. If you specifically need one bulk API operation, use the Firecrawl API batch endpoint outside MCP.
325
258
  - **If you need to discover URLs on a site:** use **map**
326
259
  - **If you want to search the web for info:** use **search**
327
260
  - **If you need complex research across multiple unknown sources:** use **agent**
328
261
  - **If you want to analyze a whole site or section:** use **crawl** (with limits!)
329
- - **If you need interactive browser automation** (click, type, navigate): use **scrape** + **interact**
262
+ - **If you need interactive browser automation** (click, type, navigate): use **interact** with a URL for a fresh page, or **scrape** + **interact** when you already scraped the page or need tighter scrape control
330
263
 
331
264
  ### Quick Reference Table
332
265
 
333
266
  | Tool | Best for | Returns |
334
267
  | ------------ | ---------------------------------------------- | ------------------------------ |
335
268
  | scrape | Single page content | JSON (preferred) or markdown |
336
- | interact | Interact with a scraped page | Execution result |
337
- | batch_scrape | Multiple known URLs | JSON (preferred) or markdown[] |
269
+ | interact | Interact with a URL or scraped page | Execution result + scrapeId for URL mode |
338
270
  | map | Discovering URLs on a site | URL[] |
339
- | crawl | Multi-page extraction (with limits) | markdown/html[] |
271
+ | crawl | Multi-page extraction (with limits) | final crawl status/data after internal polling |
272
+ | parse | Files and hosted upload refs | markdown, JSON, or document output |
273
+ | extract | Structured extraction from URLs | JSON structured data |
340
274
  | search | Web search for info | results[] |
341
275
  | agent | Complex multi-source research | JSON (structured data) |
276
+ | monitor | Recurring page checks | monitor/check metadata and diffs |
277
+ | research | Paper and GitHub repository research | research results and repo matches |
342
278
 
343
279
  ### Format Selection Guide
344
280
 
345
- When using `scrape` or `batch_scrape`, choose the right format:
281
+ When using `scrape`, choose the right format:
346
282
 
347
283
  - **JSON format (recommended for most cases):** Use when you need specific data from a page. Define a schema based on what you need to extract. This keeps responses small and avoids context window overflow.
348
284
  - **Markdown format (use sparingly):** Only when you genuinely need the full page content, such as reading an entire article for summarization or analyzing page structure.
@@ -359,12 +295,12 @@ Scrape content from a single URL with advanced options.
359
295
 
360
296
  **Not recommended for:**
361
297
 
362
- - Extracting content from multiple pages (use batch_scrape for known URLs, or map + batch_scrape to discover URLs first, or crawl for full page content)
298
+ - Extracting content from multiple pages (use repeated scrape calls for known URLs, or map + scrape to discover URLs first, or crawl for full page content)
363
299
  - When you're unsure which page contains the information (use search)
364
300
 
365
301
  **Common mistakes:**
366
302
 
367
- - Using scrape for a list of URLs (use batch_scrape instead).
303
+ - Passing a list of URLs to one scrape call. Call scrape once per URL in MCP. If you specifically need one bulk API operation, use the Firecrawl API batch endpoint outside MCP.
368
304
  - Using markdown format by default (use JSON format to extract only what you need).
369
305
 
370
306
  **Choosing the right format:**
@@ -434,72 +370,7 @@ Scrape content from a single URL with advanced options.
434
370
 
435
371
  - JSON structured data, markdown, branding profile, or other formats as specified.
436
372
 
437
- ### 2. Batch Scrape Tool (`firecrawl_batch_scrape`)
438
-
439
- Scrape multiple URLs efficiently with built-in rate limiting and parallel processing.
440
-
441
- **Best for:**
442
-
443
- - Retrieving content from multiple pages, when you know exactly which pages to scrape.
444
-
445
- **Not recommended for:**
446
-
447
- - Discovering URLs (use map first if you don't know the URLs)
448
- - Scraping a single page (use scrape)
449
-
450
- **Common mistakes:**
451
-
452
- - Using batch_scrape with too many URLs at once (may hit rate limits or token overflow)
453
-
454
- **Prompt Example:**
455
-
456
- > "Get the content of these three blog posts: [url1, url2, url3]."
457
-
458
- **Usage Example:**
459
-
460
- ```json
461
- {
462
- "name": "firecrawl_batch_scrape",
463
- "arguments": {
464
- "urls": ["https://example1.com", "https://example2.com"],
465
- "options": {
466
- "formats": ["markdown"],
467
- "onlyMainContent": true
468
- }
469
- }
470
- }
471
- ```
472
-
473
- **Returns:**
474
-
475
- - Response includes operation ID for status checking:
476
-
477
- ```json
478
- {
479
- "content": [
480
- {
481
- "type": "text",
482
- "text": "Batch operation queued with ID: batch_1. Use firecrawl_check_batch_status to check progress."
483
- }
484
- ],
485
- "isError": false
486
- }
487
- ```
488
-
489
- ### 3. Check Batch Status (`firecrawl_check_batch_status`)
490
-
491
- Check the status of a batch operation.
492
-
493
- ```json
494
- {
495
- "name": "firecrawl_check_batch_status",
496
- "arguments": {
497
- "id": "batch_1"
498
- }
499
- }
500
- ```
501
-
502
- ### 4. Map Tool (`firecrawl_map`)
373
+ ### 2. Map Tool (`firecrawl_map`)
503
374
 
504
375
  Map a website to discover all indexed URLs on the site.
505
376
 
@@ -510,7 +381,7 @@ Map a website to discover all indexed URLs on the site.
510
381
 
511
382
  **Not recommended for:**
512
383
 
513
- - When you already know which specific URL you need (use scrape or batch_scrape)
384
+ - When you already know which specific URL you need (use scrape)
514
385
  - When you need the content of the pages (use scrape after mapping)
515
386
 
516
387
  **Common mistakes:**
@@ -536,7 +407,7 @@ Map a website to discover all indexed URLs on the site.
536
407
 
537
408
  - Array of URLs found on the site
538
409
 
539
- ### 5. Search Tool (`firecrawl_search`)
410
+ ### 3. Search Tool (`firecrawl_search`)
540
411
 
541
412
  Search the web and optionally extract content from search results.
542
413
 
@@ -581,7 +452,7 @@ Search the web and optionally extract content from search results.
581
452
 
582
453
  > "Find the latest research papers on AI published in 2023."
583
454
 
584
- ### 5b. Search Feedback Tool (`firecrawl_search_feedback`)
455
+ ### 3b. Search Feedback Tool (`firecrawl_search_feedback`)
585
456
 
586
457
  Sends structured feedback on a previous `firecrawl_search` result. The first feedback per search id refunds 1 credit and improves Firecrawl's search quality. Idempotent per search id.
587
458
 
@@ -623,7 +494,7 @@ Sends structured feedback on a previous `firecrawl_search` result. The first fee
623
494
 
624
495
  - `{ success, feedbackId, creditsRefunded, alreadySubmitted? }` JSON.
625
496
 
626
- ### 5c. Generic Feedback Tool (`firecrawl_feedback`)
497
+ ### 3c. Generic Feedback Tool (`firecrawl_feedback`)
627
498
 
628
499
  Sends structured feedback for a completed v2 endpoint job through `/v2/feedback`.
629
500
  Use this for endpoint-level feedback on `scrape`, `parse`, `map`, or `search`
@@ -660,9 +531,9 @@ and small metadata objects. Do not include raw scrape/parse outputs.
660
531
 
661
532
  - `{ success, feedbackId, creditsRefunded, creditsRefundedToday?, dailyRefundCap?, dailyCapReached?, alreadySubmitted?, warning? }` JSON.
662
533
 
663
- ### 6. Crawl Tool (`firecrawl_crawl`)
534
+ ### 4. Crawl Tool (`firecrawl_crawl`)
664
535
 
665
- Starts an asynchronous crawl job on a website and extract content from all pages.
536
+ Starts a crawl job, polls until it reaches a terminal state, and returns the final crawl status/data.
666
537
 
667
538
  **Best for:**
668
539
 
@@ -671,14 +542,14 @@ Starts an asynchronous crawl job on a website and extract content from all pages
671
542
  **Not recommended for:**
672
543
 
673
544
  - Extracting content from a single page (use scrape)
674
- - When token limits are a concern (use map + batch_scrape)
545
+ - When token limits are a concern (use map + scrape for tighter control)
675
546
  - When you need fast results (crawling can be slow)
676
547
 
677
- **Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + batch_scrape for better control.
548
+ **Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + scrape for tighter control.
678
549
 
679
550
  **Common mistakes:**
680
551
 
681
- - Setting limit or maxDepth too high (causes token overflow)
552
+ - Setting limit or maxDiscoveryDepth too high (causes token overflow)
682
553
  - Using crawl for a single page (use scrape instead)
683
554
 
684
555
  **Prompt Example:**
@@ -692,7 +563,7 @@ Starts an asynchronous crawl job on a website and extract content from all pages
692
563
  "name": "firecrawl_crawl",
693
564
  "arguments": {
694
565
  "url": "https://example.com/blog/*",
695
- "maxDepth": 2,
566
+ "maxDiscoveryDepth": 2,
696
567
  "limit": 100,
697
568
  "allowExternalLinks": false,
698
569
  "deduplicateSimilarURLs": true
@@ -700,25 +571,14 @@ Starts an asynchronous crawl job on a website and extract content from all pages
700
571
  }
701
572
  ```
702
573
 
703
- **Returns:**
704
574
 
705
- - Response includes operation ID for status checking:
575
+ **Returns:**
706
576
 
707
- ```json
708
- {
709
- "content": [
710
- {
711
- "type": "text",
712
- "text": "Started crawl for: https://example.com/* with job ID: 550e8400-e29b-41d4-a716-446655440000. Use firecrawl_check_crawl_status to check progress."
713
- }
714
- ],
715
- "isError": false
716
- }
717
- ```
577
+ - Final crawl status and data after internal polling, including `id`, `status`, `completed`, `total`, `creditsUsed`, `expiresAt`, `next`, and `data`. Use the returned `id` with `firecrawl_check_crawl_status` if you need to re-check the job later.
718
578
 
719
- ### 7. Check Crawl Status (`firecrawl_check_crawl_status`)
579
+ ### 5. Check Crawl Status (`firecrawl_check_crawl_status`)
720
580
 
721
- Check the status of a crawl job.
581
+ Check the status and results of an existing crawl job by ID.
722
582
 
723
583
  ```json
724
584
  {
@@ -733,7 +593,33 @@ Check the status of a crawl job.
733
593
 
734
594
  - Response includes the status of the crawl job:
735
595
 
736
- ### 8. Extract Tool (`firecrawl_extract`)
596
+ ### 6. Parse Tool (`firecrawl_parse`)
597
+
598
+ Parse local files or hosted upload references with Firecrawl's `/v2/parse` endpoint.
599
+
600
+ **Best for:** PDFs, Word documents, spreadsheets, HTML files, and other documents that need markdown or structured JSON output. Hosted MCP supports a two-step upload-ref flow; local direct file reads require a self-hosted `FIRECRAWL_API_URL`.
601
+
602
+ **Not recommended for:** Remote URLs (use scrape), multiple files in one call (call parse once per file), or browser-only actions such as screenshots and clicks.
603
+
604
+ **Hosted MCP flow:** Hosted MCP cannot read the caller's filesystem directly. Call `firecrawl_parse` with `filePath` to receive a short-lived upload command and `nextToolCall`, upload the file locally, then call `firecrawl_parse` again with the returned `uploadRef`. Minting the hosted upload URL requires Firecrawl auth or keyless eligibility. In local `npx firecrawl-mcp` mode, direct file parsing currently requires `FIRECRAWL_API_URL` pointing to a self-hosted Firecrawl API; a plain cloud API-key-only local server cannot read and upload files through this tool.
605
+
606
+ **Usage Example:**
607
+
608
+ ```json
609
+ {
610
+ "name": "firecrawl_parse",
611
+ "arguments": {
612
+ "filePath": "/absolute/path/to/document.pdf",
613
+ "formats": ["markdown"],
614
+ "parsers": ["pdf"],
615
+ "zeroDataRetention": true
616
+ }
617
+ }
618
+ ```
619
+
620
+ **Returns:** Parsed document content or hosted upload instructions with a `nextToolCall`.
621
+
622
+ ### 7. Extract Tool (`firecrawl_extract`)
737
623
 
738
624
  Extract structured information from web pages using LLM capabilities. Supports both cloud AI and self-hosted LLM extraction.
739
625
 
@@ -806,7 +692,7 @@ When using a self-hosted instance, the extraction will use your configured LLM.
806
692
  }
807
693
  ```
808
694
 
809
- ### 9. Agent Tool (`firecrawl_agent`)
695
+ ### 8. Agent Tool (`firecrawl_agent`)
810
696
 
811
697
  Autonomous web research agent. This is a separate AI agent layer that independently browses the internet, searches for information, navigates through pages, and extracts structured data based on your query.
812
698
 
@@ -887,7 +773,7 @@ Then poll with `firecrawl_agent_status` using the returned job ID.
887
773
 
888
774
  - Job ID for status checking. Use `firecrawl_agent_status` to poll for results.
889
775
 
890
- ### 10. Check Agent Status (`firecrawl_agent_status`)
776
+ ### 9. Check Agent Status (`firecrawl_agent_status`)
891
777
 
892
778
  Check the status of an agent job and retrieve results when complete. Use this to poll for results after starting an agent.
893
779
 
@@ -908,7 +794,60 @@ Check the status of an agent job and retrieve results when complete. Use this to
908
794
  - `completed`: Research finished - response includes the extracted data
909
795
  - `failed`: An error occurred
910
796
 
911
- ### 11. Monitor Tools (`firecrawl_monitor_*`)
797
+ ### 10. Interact Tool (`firecrawl_interact`)
798
+
799
+ Interact with a fresh URL or with a page that was already opened by `firecrawl_scrape`.
800
+
801
+ **Best for:** Clicking, typing, navigating, and extracting state from dynamic pages without restoring the deprecated browser tools.
802
+
803
+ **Usage options:**
804
+
805
+ - Pass `url` to scrape and open a page for interaction in one MCP call.
806
+ - Pass `scrapeId` to continue interacting with an existing scraped page.
807
+ - Pass exactly one of `url` or `scrapeId`, plus either `prompt` or `code`.
808
+
809
+ **Usage Example:**
810
+
811
+ ```json
812
+ {
813
+ "name": "firecrawl_interact",
814
+ "arguments": {
815
+ "url": "https://example.com",
816
+ "prompt": "Click the pricing link and summarize the visible plans"
817
+ }
818
+ }
819
+ ```
820
+
821
+ **Returns:** Interaction result and, for URL mode, the derived `scrapeId` for follow-up or cleanup.
822
+
823
+ ### 11. Stop Interact Tool (`firecrawl_interact_stop`)
824
+
825
+ Stop an interact session for a scraped page when you are done interacting.
826
+
827
+ ```json
828
+ {
829
+ "name": "firecrawl_interact_stop",
830
+ "arguments": {
831
+ "scrapeId": "scrape-id-here"
832
+ }
833
+ }
834
+ ```
835
+
836
+ ### 12. Research Tools (`firecrawl_research_*`)
837
+
838
+ Search and inspect papers and GitHub repositories through the research MCP tools.
839
+
840
+ **Available research tools:**
841
+
842
+ - `firecrawl_research_search_papers`: search research papers.
843
+ - `firecrawl_research_inspect_paper`: inspect one paper.
844
+ - `firecrawl_research_related_papers`: find related papers.
845
+ - `firecrawl_research_read_paper`: read paper content.
846
+ - `firecrawl_research_search_github`: search GitHub repositories.
847
+
848
+ **Best for:** Literature review, paper lookup, and repository discovery workflows where the agent needs a focused research surface instead of general web scraping.
849
+
850
+ ### 13. Monitor Tools (`firecrawl_monitor_*`)
912
851
 
913
852
  Create and manage recurring page monitors. Monitors run scheduled scrapes or crawls, diff each result against the last retained snapshot, and can notify by webhook or email.
914
853
 
@@ -973,6 +912,7 @@ Pass `body` when you need crawl targets, JSON change tracking, custom retention,
973
912
  - `firecrawl_monitor_get`: get one monitor.
974
913
  - `firecrawl_monitor_update`: update fields including `goal`, `judgeEnabled`, `webhook`, and `notification`.
975
914
  - `firecrawl_monitor_run`: trigger a check now.
915
+ - `firecrawl_monitor_delete`: delete a monitor (destructive; only call when the user intends to remove it).
976
916
  - `firecrawl_monitor_checks`: list checks, optionally filtered by status.
977
917
  - `firecrawl_monitor_check`: get page-level results, including `diff`, `snapshot`, `judgment.meaningful`, and `judgment.meaningfulChanges`.
978
918
 
@@ -982,7 +922,6 @@ The server includes comprehensive logging:
982
922
 
983
923
  - Operation status and progress
984
924
  - Performance metrics
985
- - Credit usage monitoring
986
925
  - Rate limit tracking
987
926
  - Error conditions
988
927
 
@@ -991,19 +930,15 @@ Example log messages:
991
930
  ```
992
931
  [INFO] Firecrawl MCP Server initialized successfully
993
932
  [INFO] Starting scrape for URL: https://example.com
994
- [INFO] Batch operation queued with ID: batch_1
995
- [WARNING] Credit usage has reached warning threshold
996
- [ERROR] Rate limit exceeded, retrying in 2s...
933
+ [ERROR] Rate limit exceeded
997
934
  ```
998
935
 
999
936
  ## Error Handling
1000
937
 
1001
938
  The server provides robust error handling:
1002
939
 
1003
- - Automatic retries for transient errors
1004
- - Rate limit handling with backoff
940
+ - API rate-limit errors surfaced to the MCP client
1005
941
  - Detailed error messages
1006
- - Credit usage warnings
1007
942
  - Network resilience
1008
943
 
1009
944
  Example error response:
@@ -1013,7 +948,7 @@ Example error response:
1013
948
  "content": [
1014
949
  {
1015
950
  "type": "text",
1016
- "text": "Error: Rate limit exceeded. Retrying in 2 seconds..."
951
+ "text": "Error: Rate limit exceeded"
1017
952
  }
1018
953
  ],
1019
954
  "isError": true
package/dist/index.js CHANGED
@@ -1385,9 +1385,11 @@ function buildMonitorCreateBody(args2) {
1385
1385
  args2.page,
1386
1386
  args2.pages
1387
1387
  );
1388
- if (urls.length === 0) {
1388
+ const queries = Array.isArray(args2.queries) ? args2.queries.filter((q) => typeof q === "string").map((q) => q.trim()).filter(Boolean) : [];
1389
+ const isSearch = queries.length > 0;
1390
+ if (urls.length === 0 && !isSearch) {
1389
1391
  throw new Error(
1390
- "firecrawl_monitor_create requires either `body`, `page`, or `pages`."
1392
+ "firecrawl_monitor_create requires either `body`, `page`/`pages`, or `queries`."
1391
1393
  );
1392
1394
  }
1393
1395
  const goal = typeof args2.goal === "string" ? args2.goal.trim() : "";
@@ -1396,6 +1398,25 @@ function buildMonitorCreateBody(args2) {
1396
1398
  "firecrawl_monitor_create shorthand requires `goal`. Use `body` for advanced requests without a goal."
1397
1399
  );
1398
1400
  }
1401
+ let target;
1402
+ if (isSearch) {
1403
+ const includeDomains = Array.isArray(args2.includeDomains) ? args2.includeDomains.filter(
1404
+ (d) => typeof d === "string"
1405
+ ) : void 0;
1406
+ const excludeDomains = Array.isArray(args2.excludeDomains) ? args2.excludeDomains.filter(
1407
+ (d) => typeof d === "string"
1408
+ ) : void 0;
1409
+ target = {
1410
+ type: "search",
1411
+ queries,
1412
+ ...typeof args2.searchWindow === "string" && args2.searchWindow.trim() ? { searchWindow: args2.searchWindow.trim() } : {},
1413
+ ...typeof args2.maxResults === "number" ? { maxResults: args2.maxResults } : {},
1414
+ ...includeDomains && includeDomains.length > 0 ? { includeDomains } : {},
1415
+ ...excludeDomains && excludeDomains.length > 0 ? { excludeDomains } : {}
1416
+ };
1417
+ } else {
1418
+ target = { type: "scrape", urls };
1419
+ }
1399
1420
  const webhookUrl = typeof args2.webhookUrl === "string" ? args2.webhookUrl.trim() : "";
1400
1421
  const email = typeof args2.email === "string" && args2.email.trim() ? {
1401
1422
  email: {
@@ -1405,13 +1426,13 @@ function buildMonitorCreateBody(args2) {
1405
1426
  }
1406
1427
  } : void 0;
1407
1428
  return {
1408
- name: typeof args2.name === "string" && args2.name.trim() ? args2.name.trim() : `Monitor ${urls[0]}`,
1429
+ name: typeof args2.name === "string" && args2.name.trim() ? args2.name.trim() : isSearch ? `Monitor ${queries[0]}` : `Monitor ${urls[0]}`,
1409
1430
  schedule: {
1410
1431
  text: typeof args2.scheduleText === "string" && args2.scheduleText.trim() ? args2.scheduleText.trim() : "every 30 minutes",
1411
1432
  timezone: typeof args2.timezone === "string" && args2.timezone.trim() ? args2.timezone.trim() : "UTC"
1412
1433
  },
1413
1434
  goal,
1414
- targets: [{ type: "scrape", urls }],
1435
+ targets: [target],
1415
1436
  ...email ? { notification: email } : {},
1416
1437
  ...webhookUrl ? {
1417
1438
  webhook: {
@@ -1434,20 +1455,38 @@ function registerMonitorTools(server2) {
1434
1455
  // Additive; creates a new monitor without deleting existing monitors or external content.
1435
1456
  },
1436
1457
  description: `
1437
- Create a Firecrawl monitor \u2014 a recurring scrape or crawl that diffs each result against the last retained snapshot.
1458
+ Create a Firecrawl monitor \u2014 a recurring scrape, crawl, or search that diffs each result against the last retained snapshot.
1438
1459
 
1439
- Prefer the simple path: pass \`page\` or \`pages\` plus \`goal\`. The tool will create a scrape monitor with a 30-minute schedule and meaningful-change judging enabled by the API. Use \`body\` only for advanced requests such as crawl targets, JSON change tracking, custom retention, or manual \`judgeEnabled\` control.
1460
+ Prefer the simple path: pass \`page\` or \`pages\` plus \`goal\` to monitor specific URLs, OR pass \`queries\` plus \`goal\` to monitor web search results for new/changed hits. The tool will create the monitor with a 30-minute schedule and meaningful-change judging enabled by the API. Use \`body\` only for advanced requests such as crawl targets, JSON change tracking, custom retention, or manual \`judgeEnabled\` control.
1440
1461
 
1441
1462
  Meaningful-change judge: set \`goal\` to a plain-language description of what the user actually cares about. \`judgeEnabled\` defaults to true when \`goal\` is set, so providing \`goal\` is enough. Page webhooks expose \`isMeaningful\` and \`judgment\` on \`monitor.page\` events.
1442
1463
 
1443
1464
  Simple fields:
1444
1465
  - \`page\`: one page URL to monitor.
1445
1466
  - \`pages\`: multiple page URLs to monitor.
1446
- - \`goal\`: plain-English instruction for what changes matter. Required for the simple path.
1467
+ - \`queries\`: one or more search queries (1-12) to monitor instead of fixed URLs. Each check runs the searches and diffs the result set, so you get alerted when new or changed results appear. Mutually exclusive with \`page\`/\`pages\` in the simple path.
1468
+ - \`searchWindow\`: optional recency window for search targets \u2014 one of \`5m\`, \`15m\`, \`1h\`, \`6h\`, \`24h\`, \`7d\` (default \`24h\`).
1469
+ - \`maxResults\`: optional max results per search, 1-50 (default 10).
1470
+ - \`includeDomains\` / \`excludeDomains\`: optional domain allow/deny lists for search targets.
1471
+ - \`goal\`: plain-English instruction for what changes matter. Required for the simple path (and always required when \`queries\` are set \u2014 web monitors must have a goal).
1447
1472
  - \`scheduleText\`: optional natural-language schedule, default \`every 30 minutes\`.
1448
1473
  - \`email\`: optional email recipient for summaries.
1449
1474
  - \`webhookUrl\`: optional webhook URL. Configures \`monitor.page\` and \`monitor.check.completed\`.
1450
1475
 
1476
+ **Search-mode example:**
1477
+
1478
+ \`\`\`json
1479
+ {
1480
+ "name": "firecrawl_monitor_create",
1481
+ "arguments": {
1482
+ "queries": ["new LLM release", "frontier model launch"],
1483
+ "goal": "Notify me about major new LLM model releases.",
1484
+ "searchWindow": "24h",
1485
+ "maxResults": 10
1486
+ }
1487
+ }
1488
+ \`\`\`
1489
+
1451
1490
  Goal guidance:
1452
1491
  - Expand the user's one-line monitoring intent into a concise 2-3 sentence monitor goal.
1453
1492
  - State what should trigger an alert, restate any scope the user gave, and include intent-specific exclusions only when obvious from the user's request.
@@ -1456,7 +1495,7 @@ Goal guidance:
1456
1495
  - If the user says they do not care about something, include that explicitly. It is okay to ask whether they want to ignore specific noise when it is likely to matter.
1457
1496
  - Do not invent page-specific sections, thresholds, entities, or business rules unless the user mentioned them.
1458
1497
 
1459
- Full \`body\` requests require: \`name\`, \`schedule\` (with \`cron\` or \`text\`), and \`targets\` (one or more \`{ type: 'scrape', urls: [...] }\` or \`{ type: 'crawl', url: '...' }\`). Optional: \`goal\`, \`judgeEnabled\`, \`webhook\`, \`notification\`, \`retentionDays\`.
1498
+ Full \`body\` requests require: \`name\`, \`schedule\` (with \`cron\` or \`text\`), and \`targets\` (one or more \`{ type: 'scrape', urls: [...] }\`, \`{ type: 'crawl', url: '...' }\`, or \`{ type: 'search', queries: [...], searchWindow?, maxResults?, includeDomains?, excludeDomains? }\`). Optional: \`goal\` (required when any search target is present), \`judgeEnabled\`, \`webhook\`, \`notification\`, \`retentionDays\`.
1460
1499
 
1461
1500
  **Markdown-mode (default):** Each check produces a unified text diff of the page's markdown. No extra configuration needed.
1462
1501
 
@@ -1532,6 +1571,11 @@ Full \`body\` requests require: \`name\`, \`schedule\` (with \`cron\` or \`text\
1532
1571
  body: z2.record(z2.string(), z2.any()).optional(),
1533
1572
  page: z2.string().optional(),
1534
1573
  pages: z2.array(z2.string()).optional(),
1574
+ queries: z2.array(z2.string()).optional(),
1575
+ searchWindow: z2.enum(["5m", "15m", "1h", "6h", "24h", "7d"]).optional(),
1576
+ maxResults: z2.number().int().min(1).max(50).optional(),
1577
+ includeDomains: z2.array(z2.string()).optional(),
1578
+ excludeDomains: z2.array(z2.string()).optional(),
1535
1579
  goal: z2.string().optional(),
1536
1580
  name: z2.string().optional(),
1537
1581
  scheduleText: z2.string().optional(),
@@ -3484,20 +3528,20 @@ Do not store multi-MB outputs in feedback. Use concise notes, issue codes, URLs,
3484
3528
  server.addTool({
3485
3529
  name: "firecrawl_crawl",
3486
3530
  annotations: {
3487
- title: "Start a site crawl",
3531
+ title: "Run a site crawl",
3488
3532
  readOnlyHint: false,
3489
- // Starts an asynchronous crawl job, creating a persistent server-side job.
3533
+ // Starts a server-side crawl job and polls until the job reaches a terminal state.
3490
3534
  openWorldHint: true,
3491
3535
  // Crawls user-specified URLs across the public web.
3492
3536
  destructiveHint: false
3493
3537
  // Reads pages from target sites; does not delete or alter external websites.
3494
3538
  },
3495
3539
  description: `
3496
- Starts a crawl job on a website and extracts content from all pages.
3540
+ Starts a crawl job on a website, polls until it reaches a terminal state, and returns the final crawl status/data.
3497
3541
 
3498
3542
  **Best for:** Extracting content from multiple related pages, when you need comprehensive coverage.
3499
- **Not recommended for:** Extracting content from a single page (use scrape); when token limits are a concern (use map + batch_scrape); when you need fast results (crawling can be slow).
3500
- **Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + batch_scrape for better control.
3543
+ **Not recommended for:** Extracting content from a single page (use scrape); when token limits are a concern (use map + scrape for tighter control); when you need fast results (crawling can be slow).
3544
+ **Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + scrape for tighter control.
3501
3545
  **Common mistakes:** Setting limit or maxDiscoveryDepth too high (causes token overflow) or too low (causes missing pages); using crawl for a single page (use scrape instead). Using a /* wildcard is not recommended.
3502
3546
  **Prompt Example:** "Get all blog posts from the first two levels of example.com/blog."
3503
3547
  **Usage Example:**
@@ -3514,7 +3558,7 @@ server.addTool({
3514
3558
  }
3515
3559
  }
3516
3560
  \`\`\`
3517
- **Returns:** Operation ID for status checking; use firecrawl_check_crawl_status to check progress.
3561
+ **Returns:** Final crawl status and data after internal polling, including the crawl id. Use firecrawl_check_crawl_status only when you need to re-check an existing crawl ID later.
3518
3562
  ${SAFE_MODE ? "**Safe Mode:** Read-only crawling. Webhooks and interactive actions are disabled for security." : ""}
3519
3563
  `,
3520
3564
  parameters: z4.object({
@@ -3846,24 +3890,28 @@ server.addTool({
3846
3890
  // Transient page interactions only; does not delete monitors, jobs, or external sites.
3847
3891
  },
3848
3892
  description: `
3849
- Interact with a previously scraped page in a live browser session. Scrape a page first with firecrawl_scrape, then use the returned scrapeId to click buttons, fill forms, extract dynamic content, or navigate deeper.
3893
+ Interact with a page in a live browser session: click buttons, fill forms, extract dynamic content, or navigate deeper.
3850
3894
 
3851
3895
  **Best for:** Multi-step workflows on a single page \u2014 searching a site, clicking through results, filling forms, extracting data that requires interaction.
3852
- **Requires:** A scrapeId from a previous firecrawl_scrape call (found in the metadata of the scrape response).
3896
+ **Two ways to target a page:**
3897
+ - Pass a \`url\` to interact directly. The session is opened for you in one call (use this for a fresh page).
3898
+ - Pass a \`scrapeId\` from a previous firecrawl_scrape to reuse that already-loaded page (cheaper when you just scraped it).
3853
3899
 
3854
3900
  **Arguments:**
3855
- - scrapeId: The scrape job ID from a previous scrape (required)
3901
+ - url: Page to interact with; opens a session for you (use this OR scrapeId)
3902
+ - scrapeId: Scrape job ID from a previous scrape, found in its metadata (use this OR url)
3856
3903
  - prompt: Natural language instruction describing the action to take (use this OR code)
3857
3904
  - code: Code to execute in the browser session (use this OR prompt)
3858
3905
  - language: "bash", "python", or "node" (optional, defaults to "node", only used with code)
3859
- - timeout: Execution timeout in seconds, 1-300 (optional, defaults to 30)
3906
+ - timeout: Interact execution timeout in seconds, 1-300 (optional, defaults to 30)
3907
+ - scrapeOptions: Optional scrape controls used only with url mode, such as waitFor, maxAge, proxy, or zeroDataRetention
3860
3908
 
3861
- **Usage Example (prompt):**
3909
+ **Usage Example (prompt, direct via url):**
3862
3910
  \`\`\`json
3863
3911
  {
3864
3912
  "name": "firecrawl_interact",
3865
3913
  "arguments": {
3866
- "scrapeId": "scrape-id-from-previous-scrape",
3914
+ "url": "https://example.com/products",
3867
3915
  "prompt": "Click on the first product and tell me its price"
3868
3916
  }
3869
3917
  }
@@ -3883,24 +3931,71 @@ Interact with a previously scraped page in a live browser session. Scrape a page
3883
3931
  **Returns:** Execution result including output, stdout, stderr, exit code, and live view URLs.
3884
3932
  `,
3885
3933
  parameters: z4.object({
3886
- scrapeId: z4.string(),
3887
- prompt: z4.string().optional(),
3888
- code: z4.string().optional(),
3934
+ scrapeId: z4.string().trim().min(1).optional(),
3935
+ url: z4.string().trim().url().optional(),
3936
+ prompt: z4.string().trim().min(1).optional(),
3937
+ code: z4.string().trim().min(1).optional(),
3889
3938
  language: z4.enum(["bash", "python", "node"]).optional(),
3890
- timeout: z4.number().min(1).max(300).optional()
3939
+ timeout: z4.number().min(1).max(300).optional(),
3940
+ scrapeOptions: scrapeParamsSchema.omit({ url: true }).partial().optional()
3941
+ }).refine((data) => Boolean(data.scrapeId) !== Boolean(data.url), {
3942
+ message: "Provide either 'url' (interact directly) or 'scrapeId' (reuse a previous scrape), not both."
3943
+ }).refine((data) => !data.scrapeOptions || Boolean(data.url), {
3944
+ message: "scrapeOptions can only be used with 'url' mode."
3891
3945
  }).refine((data) => data.code || data.prompt, {
3892
3946
  message: "Either 'code' or 'prompt' must be provided."
3893
3947
  }),
3894
3948
  execute: async (args2, { session, log }) => {
3895
3949
  const client = getClient(session);
3896
- const { scrapeId, prompt, code, language, timeout } = args2;
3897
- log.info("Interacting with scraped page", { scrapeId });
3950
+ const {
3951
+ scrapeId: providedScrapeId,
3952
+ url,
3953
+ prompt,
3954
+ code,
3955
+ language,
3956
+ timeout,
3957
+ scrapeOptions
3958
+ } = args2;
3959
+ let scrapeId = providedScrapeId;
3960
+ const openedFromUrl = !scrapeId;
3961
+ if (openedFromUrl) {
3962
+ log.info("Opening interact session from url", { url });
3963
+ const cleanedScrapeOptions = removeEmptyTopLevel(scrapeOptions ?? {});
3964
+ const scraped = await client.scrape(String(url), {
3965
+ ...cleanedScrapeOptions,
3966
+ origin: ORIGIN
3967
+ });
3968
+ scrapeId = scraped?.metadata?.scrapeId;
3969
+ if (!scrapeId) {
3970
+ return asText2({
3971
+ error: "Could not open an interact session: the scrape did not return a scrapeId. Try firecrawl_scrape first, then pass its scrapeId.",
3972
+ url
3973
+ });
3974
+ }
3975
+ }
3976
+ if (!scrapeId) {
3977
+ return asText2({
3978
+ error: "Could not open an interact session: missing scrapeId.",
3979
+ url
3980
+ });
3981
+ }
3982
+ const activeScrapeId = scrapeId;
3983
+ log.info("Interacting with page", { scrapeId: activeScrapeId });
3898
3984
  const interactArgs = { origin: ORIGIN };
3899
3985
  if (prompt) interactArgs.prompt = prompt;
3900
3986
  if (code) interactArgs.code = code;
3901
3987
  if (language) interactArgs.language = language;
3902
3988
  if (timeout != null) interactArgs.timeout = timeout;
3903
- const res = await client.interact(scrapeId, interactArgs);
3989
+ const res = await client.interact(activeScrapeId, interactArgs);
3990
+ if (openedFromUrl && res && typeof res === "object" && !Array.isArray(res)) {
3991
+ return asText2({
3992
+ ...res,
3993
+ scrapeId: activeScrapeId
3994
+ });
3995
+ }
3996
+ if (openedFromUrl) {
3997
+ return asText2({ scrapeId: activeScrapeId, result: res });
3998
+ }
3904
3999
  return asText2(res);
3905
4000
  }
3906
4001
  });
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "firecrawl-mcp",
3
- "version": "3.21.4",
3
+ "version": "3.22.1",
4
4
  "description": "MCP server for Firecrawl — search, scrape, and interact with the web. Supports both cloud and self-hosted instances. Features include web search, scraping, page interaction, batch processing, and LLM-powered content analysis.",
5
5
  "type": "module",
6
6
  "mcpName": "io.github.firecrawl/firecrawl-mcp-server",