firecrawl-mcp 2.0.1 → 2.0.2

package/README.md

@@ -311,8 +311,6 @@ Use this guide to select the right tool for your task:
 - **If you want to search the web for info:** use **search**
 - **If you want to extract structured data:** use **extract**
 - **If you want to analyze a whole site or section:** use **crawl** (with limits!)
-- **If you want to do in-depth research:** use **deep_research**
-- **If you want to generate LLMs.txt:** use **generate_llmstxt**
 
 ### Quick Reference Table
 
@@ -324,8 +322,6 @@ Use this guide to select the right tool for your task:
 | crawl               | Multi-page extraction (with limits)      | markdown/html[] |
 | search              | Web search for info                      | results[]       |
 | extract             | Structured data from pages               | JSON            |
-| deep_research       | In-depth, multi-source research          | summary, sources|
-| generate_llmstxt    | LLMs.txt for a domain                    | text            |
 
 ## Available Tools
 
@@ -629,78 +625,6 @@ When using a self-hosted instance, the extraction will use your configured LLM.
 }
 ```
 
-### 9. Deep Research Tool (`firecrawl_deep_research`)
-
-Conduct deep web research on a query using intelligent crawling, search, and LLM analysis.
-
-**Best for:**
-- Complex research questions requiring multiple sources, in-depth analysis.
-
-**Not recommended for:**
-- Simple questions that can be answered with a single search
-- When you need very specific information from a known page (use scrape)
-- When you need results quickly (deep research can take time)
-
-**Arguments:**
-- query (string, required): The research question or topic to explore.
-- maxDepth (number, optional): Maximum recursive depth for crawling/search (default: 3).
-- timeLimit (number, optional): Time limit in seconds for the research session (default: 120).
-- maxUrls (number, optional): Maximum number of URLs to analyze (default: 50).
-
-**Prompt Example:**
-> "Research the environmental impact of electric vehicles versus gasoline vehicles."
-
-**Usage Example:**
-```json
-{
-  "name": "firecrawl_deep_research",
-  "arguments": {
-    "query": "What are the environmental impacts of electric vehicles compared to gasoline vehicles?",
-    "maxDepth": 3,
-    "timeLimit": 120,
-    "maxUrls": 50
-  }
-}
-```
-
-**Returns:**
-- Final analysis generated by an LLM based on research. (data.finalAnalysis)
-- May also include structured activities and sources used in the research process.
-
-### 10. Generate LLMs.txt Tool (`firecrawl_generate_llmstxt`)
-
-Generate a standardized llms.txt (and optionally llms-full.txt) file for a given domain. This file defines how large language models should interact 
-with the site.
-
-**Best for:**
-- Creating machine-readable permission guidelines for AI models.
-
-**Not recommended for:**
-- General content extraction or research
-
-**Arguments:**
-- url (string, required): The base URL of the website to analyze.
-- maxUrls (number, optional): Max number of URLs to include (default: 10).
-- showFullText (boolean, optional): Whether to include llms-full.txt contents in the response.
-
-**Prompt Example:**
-> "Generate an LLMs.txt file for example.com."
-
-**Usage Example:**
-```json
-{
-  "name": "firecrawl_generate_llmstxt",
-  "arguments": {
-    "url": "https://example.com",
-    "maxUrls": 20,
-    "showFullText": true
-  }
-}
-```
-
-**Returns:**
-- LLMs.txt file contents (and optionally llms-full.txt)
-
 ## Logging System
 
 The server includes comprehensive logging:

package/dist/index.js

@@ -55,6 +55,7 @@ This is the most powerful, fastest and most reliable scraper tool, if available
                                 'links',
                                 'extract',
                                 'summary',
+                                'changeTracking',
                             ],
                         },
                         {
@@ -256,7 +257,7 @@ const CRAWL_TOOL = {
  **Best for:** Extracting content from multiple related pages, when you need comprehensive coverage.
  **Not recommended for:** Extracting content from a single page (use scrape); when token limits are a concern (use map + batch_scrape); when you need fast results (crawling can be slow).
  **Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + batch_scrape for better control.
- **Common mistakes:** Setting limit or maxDiscoveryDepth too high (causes token overflow); using crawl for a single page (use scrape instead).
+ **Common mistakes:** Setting limit or maxDiscoveryDepth too high (causes token overflow) or too low (causes missing pages); using crawl for a single page (use scrape instead). Using a /* wildcard is not recommended.
  **Prompt Example:** "Get all blog posts from the first two levels of example.com/blog."
  **Usage Example:**
  \`\`\`json
@@ -264,8 +265,8 @@ const CRAWL_TOOL = {
    "name": "firecrawl_crawl",
    "arguments": {
      "url": "https://example.com/blog/*",
-     "maxDiscoveryDepth": 2,
-     "limit": 100,
+     "maxDiscoveryDepth": 5,
+     "limit": 20,
      "allowExternalLinks": false,
      "deduplicateSimilarURLs": true,
      "sitemap": "include"
@@ -702,12 +703,6 @@ function isExtractOptions(args) {
     return (Array.isArray(urls) &&
         urls.every((url) => typeof url === 'string'));
 }
-function isGenerateLLMsTextOptions(args) {
-    return (typeof args === 'object' &&
-        args !== null &&
-        'url' in args &&
-        typeof args.url === 'string');
-}
 function removeEmptyTopLevel(obj) {
     const out = {};
     for (const [k, v] of Object.entries(obj)) {
@@ -905,7 +900,10 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
                 }
                 return {
                     content: [
-                        { type: 'text', text: trimResponseText(JSON.stringify(response.links, null, 2)) },
+                        {
+                            type: 'text',
+                            text: trimResponseText(JSON.stringify(response.links, null, 2)),
+                        },
                     ],
                     isError: false,
                 };
@@ -1040,44 +1038,6 @@ ${response.data.length > 0 ? '\nResults:\n' + formatResults(response.data) : ''}
                     };
                 }
             }
-            case 'firecrawl_generate_llmstxt': {
-                if (!isGenerateLLMsTextOptions(args)) {
-                    throw new Error('Invalid arguments for firecrawl_generate_llmstxt');
-                }
-                try {
-                    const { url, ...params } = args;
-                    const generateStartTime = Date.now();
-                    safeLog('info', `Starting LLMs.txt generation for URL: ${url}`);
-                    // Start the generation process
-                    const response = await withRetry(async () => 
-                    // @ts-expect-error Extended API options including origin
-                    client.generateLLMsText(url, { ...params, origin: 'mcp-server' }), 'LLMs.txt generation');
-                    if (!response.success) {
-                        throw new Error(response.error || 'LLMs.txt generation failed');
-                    }
-                    // Log performance metrics
-                    safeLog('info', `LLMs.txt generation completed in ${Date.now() - generateStartTime}ms`);
-                    // Format the response
-                    let resultText = '';
-                    if ('data' in response) {
-                        resultText = `LLMs.txt content:\n\n${response.data.llmstxt}`;
-                        if (args.showFullText && response.data.llmsfulltxt) {
-                            resultText += `\n\nLLMs-full.txt content:\n\n${response.data.llmsfulltxt}`;
-                        }
-                    }
-                    return {
-                        content: [{ type: 'text', text: trimResponseText(resultText) }],
-                        isError: false,
-                    };
-                }
-                catch (error) {
-                    const errorMessage = error instanceof Error ? error.message : String(error);
-                    return {
-                        content: [{ type: 'text', text: trimResponseText(errorMessage) }],
-                        isError: true,
-                    };
-                }
-            }
             default:
                 return {
                     content: [

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "firecrawl-mcp",
-  "version": "2.0.1",
-  "description": "MCP server for Firecrawl web scraping integration. Supports both cloud and self-hosted instances. Features include web scraping, batch processing, structured data extraction, and LLM-powered content analysis.",
+  "version": "2.0.2",
+  "description": "MCP server for Firecrawl web scraping integration. Supports both cloud and self-hosted instances. Features include web scraping, search, batch processing, structured data extraction, and LLM-powered content analysis.",
   "type": "module",
   "bin": {
     "firecrawl-mcp": "dist/index.js"