@ariangibson/firecrawl-lite-mcp-server 1.1.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 ariangibson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE. 

package/README.md

@@ -0,0 +1,287 @@
+# Firecrawl Lite MCP Server
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+A **privacy-first, standalone** MCP server that provides web scraping and data extraction tools using local browser automation and your own LLM API key. **No external dependencies or API keys required** - completely decoupled from Firecrawl's cloud service.
+
+## � **What Makes Firecrawl Lite Special**
+
+### **🔒 Privacy-First Architecture**
+- **Local Processing** - All web scraping and data extraction happens on your machine
+- **Your Data Stays Local** - Content is processed locally, not sent to third parties
+- **No External Service Lock-in** - Doesn't require Firecrawl's cloud API
+- **Complete Control** - You own your data and infrastructure
+
+### **💰 Cost-Effective & Transparent**
+- **Pay Only for LLM Usage** - No additional subscription or API fees
+- **Your LLM Provider** - Compatible with OpenAI, xAI, Anthropic, Ollama, etc.
+- **Predictable Costs** - Transparent pricing based on your chosen LLM rates
+
+### **⚡ Performance & Simplicity**
+- **Lightning-Fast Startup** - Lightweight design means quick initialization
+- **Single Container** - Simple deployment with Docker support
+- **Minimal Resource Usage** - Optimized for efficiency and low memory footprint
+
+## 📊 **Feature Comparison**
+
+| Feature | Firecrawl Lite ✅ | Original Firecrawl ❌ |
+|---------|-------------------|----------------------|
+| **🏠 Deployment** | **Standalone/Local** | Cloud Service |
+| **🔑 API Keys Required** | **Your LLM key only** | Firecrawl API + LLM keys |
+| **🔒 Data Privacy** | **100% local processing** | Cloud processing |
+| **💰 Cost Model** | **LLM usage only** | Subscription + LLM costs |
+| **⚙️ Setup Complexity** | **Single container** | Multi-service deployment |
+| **📦 Bundle Size** | **~50MB lightweight** | Heavy multi-service |
+| **🏠 Local LLM Support** | **✅ Ollama/Local LLMs** | Limited local options |
+| **🎛️ Customization** | **Full control** | Limited customization |
+| **🚀 Startup Time** | **< 5 seconds** | Variable (cloud dependent) |
+| **🔧 Maintenance** | **Self-managed** | Managed service |
+
+## �️ **Available Tools**
+
+This standalone version provides local web scraping and data extraction using Puppeteer and your own LLM:
+
+### ✅ **`scrape_page`** - Extract content from a single webpage
+- **Implementation**: Local browser automation with Puppeteer
+- **Use case**: Get webpage content for LLMs to read
+- **Parameters**: `url`, `onlyMainContent`
+- **Privacy**: All data processed locally
+
+### ✅ **`batch_scrape`** - Scrape multiple URLs in a single request
+- **Implementation**: Sequential local scraping with rate limiting
+- **Use case**: Process multiple pages efficiently
+- **Parameters**: `urls[]`, `onlyMainContent`
+- **Privacy**: All data processed locally
+
+### ✅ **`extract_data`** - Extract structured data using LLM
+- **Implementation**: Local scraping + your LLM for data extraction
+- **Use case**: Pull specific data from pages using natural language prompts
+- **Parameters**: `urls[]`, `prompt`, `enableWebSearch`
+- **Privacy**: Content scraped locally, sent to your LLM only
+
+### ✅ **`extract_with_schema`** - Extract data using JSON schema
+- **Implementation**: Local scraping + schema-guided LLM extraction
+- **Use case**: Extract structured data with predefined schema
+- **Parameters**: `urls[]`, `schema`, `prompt`, `enableWebSearch`
+- **Privacy**: Content scraped locally, sent to your LLM only
+
+## 🚀 **Quick Start**
+
+### **1. Install the package:**
+```bash
+npm install -g @ariangibson/firecrawl-lite-mcp-server
+```
+
+Or use npx to run without global installation (recommended).
+
+### **2. Configure your LLM:**
+```bash
+# Create a .env file or set environment variables
+LLM_API_KEY=your_llm_api_key_here
+LLM_PROVIDER_BASE_URL=https://api.x.ai/v1
+LLM_MODEL=grok-code-fast-1
+```
+
+### **3. Configure your MCP client:**
+
+#### **Claude Desktop**
+Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:
+```json
+{
+  "mcpServers": {
+    "firecrawl-lite": {
+      "command": "npx",
+      "args": ["-y", "@ariangibson/firecrawl-lite-mcp-server"],
+      "env": {
+        "LLM_API_KEY": "your_llm_api_key_here",
+        "LLM_PROVIDER_BASE_URL": "https://api.x.ai/v1",
+        "LLM_MODEL": "grok-code-fast-1"
+      }
+    }
+  }
+}
+```
+
+#### **Claude Code (CLI)**
+```bash
+claude config mcp add firecrawl-lite \
+  --command "npx" \
+  --args "-y" --args "@ariangibson/firecrawl-lite-mcp-server" \
+  --env LLM_API_KEY=your_llm_api_key_here \
+  --env LLM_PROVIDER_BASE_URL=https://api.x.ai/v1 \
+  --env LLM_MODEL=grok-code-fast-1
+```
+
+### **4. Restart your MCP client and start scraping!**
+
+## ⚙️ **Configuration Guide**
+
+### **Required Environment Variables**
+```bash
+# Your LLM API key (xAI, OpenAI, Anthropic, etc.)
+LLM_API_KEY=your_api_key_here
+
+# LLM provider base URL
+LLM_PROVIDER_BASE_URL=https://api.x.ai/v1
+
+# LLM model name
+LLM_MODEL=grok-code-fast-1
+```
+
+### **LLM Provider Examples**
+```bash
+# xAI (Grok)
+LLM_PROVIDER_BASE_URL=https://api.x.ai/v1
+LLM_API_KEY=xai-your-key-here
+LLM_MODEL=grok-code-fast-1
+
+# OpenAI
+LLM_PROVIDER_BASE_URL=https://api.openai.com/v1
+LLM_API_KEY=sk-your-key-here
+LLM_MODEL=gpt-4o-mini
+
+# Anthropic
+LLM_PROVIDER_BASE_URL=https://api.anthropic.com
+LLM_API_KEY=sk-ant-your-key-here
+LLM_MODEL=claude-3-haiku-20240307
+
+# Local LLM (Ollama)
+LLM_PROVIDER_BASE_URL=http://localhost:11434/v1
+LLM_API_KEY=your-local-key
+LLM_MODEL=llama2
+```
+
+### **Optional Configuration**
+```bash
+# Proxy configuration (for web scraping and LLM API calls)
+PROXY_SERVER_URL=http://your-proxy.com:8080
+PROXY_SERVER_USERNAME=your_proxy_username
+PROXY_SERVER_PASSWORD=your_proxy_password
+
+# Scraping configuration (anti-detection and rate limiting)
+SCRAPE_USER_AGENT=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
+SCRAPE_VIEWPORT_WIDTH=1920
+SCRAPE_VIEWPORT_HEIGHT=1080
+SCRAPE_DELAY_MIN=1000
+SCRAPE_DELAY_MAX=3000
+```
+
+## 🛡️ **Anti-Detection Features**
+
+Firecrawl Lite includes sophisticated anti-detection measures to handle modern websites with bot protection:
+
+### ✅ **Built-in Anti-Detection**
+- **Realistic Browser Fingerprinting**: Spoofs navigator properties, plugins, and browser APIs
+- **Random Delays**: Adds human-like delays between requests (configurable)
+- **Modern User Agent**: Uses up-to-date Chrome user agent strings
+- **Viewport Simulation**: Sets realistic desktop viewport sizes
+- **Headless Optimization**: Configured for maximum stealth in headless mode
+
+### ✅ **Configurable Settings**
+```bash
+# Control delays (in milliseconds)
+SCRAPE_DELAY_MIN=1000      # Minimum delay before navigation
+SCRAPE_DELAY_MAX=3000      # Maximum delay before navigation
+SCRAPE_BATCH_DELAY_MIN=2000 # Minimum delay between batch requests
+SCRAPE_BATCH_DELAY_MAX=5000 # Maximum delay between batch requests
+```
+
+## � **Docker Deployment (Optional)**
+
+If you prefer Docker deployment:
+
+```bash
+# Build and run with Docker
+docker-compose up --build
+
+# Run in background
+docker-compose up -d --build
+```
+
+The server will be available at `http://localhost:3000` with a health endpoint at `http://localhost:3000/health`.
+
+## 📊 **Usage Examples**
+
+### Scrape a webpage
+```json
+{
+  "name": "scrape_page",
+  "arguments": {
+    "url": "https://example.com"
+  }
+}
+```
+
+### Batch scrape multiple URLs
+```json
+{
+  "name": "batch_scrape",
+  "arguments": {
+    "urls": ["https://example.com", "https://example.org"],
+    "onlyMainContent": true
+  }
+}
+```
+
+### Extract data with prompt
+```json
+{
+  "name": "extract_data",
+  "arguments": {
+    "urls": ["https://example.com"],
+    "prompt": "Extract the main article title and summary"
+  }
+}
+```
+
+### Extract with schema
+```json
+{
+  "name": "extract_with_schema",
+  "arguments": {
+    "urls": ["https://example.com"],
+    "schema": {
+      "type": "object",
+      "properties": {
+        "title": {"type": "string"},
+        "description": {"type": "string"}
+      }
+    }
+  }
+}
+```
+
+## ❓ **Important Notes**
+
+### **🌐 Internet Requirements**
+- **Requires Internet Access** - Still needs to access target websites
+- **LLM API Access** - Requires connection to your chosen LLM provider
+- **No Offline Operation** - Cannot work completely offline
+
+### **� Intentionally Excluded Features**
+By design, this lite version excludes advanced features to maintain simplicity:
+- Web search functionality
+- Website URL discovery/mapping
+- Multi-page website crawling
+- LLMs.txt file generation
+- Advanced research capabilities
+- Crawl job status checking
+
+## 🙏 **Credits & Acknowledgments**
+
+This project is inspired by and builds upon the excellent work of the original Firecrawl projects:
+
+### 🔥 **[Firecrawl](https://firecrawl.com)**
+The original Firecrawl project by **Mendable.ai** - a comprehensive web scraping and crawling platform with advanced features like website mapping, multi-page crawling, and deep research capabilities.
+
+### 🔥 **[Firecrawl MCP Server](https://github.com/firecrawl/firecrawl-mcp-server)**
+The official MCP server implementation by the Firecrawl team, providing MCP integration for their cloud-based scraping service.
+
+**We give huge thanks to the Firecrawl team for their pioneering work in web scraping and MCP integration!** 🚀
+
+> **💡 Looking for a very generous free tier and dead-simple cloud-hosted solution?**  
+> Visit **[firecrawl.com](https://firecrawl.com)** and sign up for a Firecrawl account! Their cloud service offers enterprise-grade web scraping with zero setup complexity.
+
+## �📝 **License**
+
+MIT License - see [LICENSE](LICENSE) for details.

package/dist/index.js

@@ -0,0 +1,890 @@
+#!/usr/bin/env node
+// Core MCP SDK imports
+import { Server } from '@modelcontextprotocol/sdk/server/index.js';
+import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
+import { SSEServerTransport } from '@modelcontextprotocol/sdk/server/sse.js';
+import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
+import { CallToolRequestSchema, ListToolsRequestSchema, } from '@modelcontextprotocol/sdk/types.js';
+// Web framework and utilities
+import express from 'express';
+import { randomUUID } from 'node:crypto';
+// External dependencies
+import dotenv from 'dotenv';
+import axios from 'axios';
+import puppeteer from 'puppeteer';
+dotenv.config();
+// Constants
+const DEFAULT_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36';
+const DEFAULT_VIEWPORT_WIDTH = 1920;
+const DEFAULT_VIEWPORT_HEIGHT = 1080;
+const DEFAULT_SCRAPE_DELAY_MIN = 1000;
+const DEFAULT_SCRAPE_DELAY_MAX = 3000;
+const DEFAULT_BATCH_DELAY_MIN = 2000;
+const DEFAULT_BATCH_DELAY_MAX = 5000;
+const DEFAULT_RETRY_ATTEMPTS = 3;
+const DEFAULT_RETRY_INITIAL_DELAY = 1000;
+const DEFAULT_RETRY_MAX_DELAY = 10000;
+const DEFAULT_RETRY_BACKOFF_FACTOR = 2;
+// Security constants
+const MAX_URLS_PER_REQUEST = 10;
+// Input validation utilities
+function isValidUrl(url) {
+    try {
+        const parsedUrl = new URL(url);
+        // Only allow http and https protocols
+        return ['http:', 'https:'].includes(parsedUrl.protocol);
+    }
+    catch {
+        return false;
+    }
+}
+function sanitizeUrl(url) {
+    // Remove any potentially dangerous characters
+    return url.trim().replace(/[<>'"]/g, '');
+}
+function validatePrompt(prompt) {
+    // Basic prompt validation - prevent extremely long prompts
+    return prompt.length > 0 && prompt.length < 10000;
+}
+const SCRAPE_TOOL = {
+    name: 'scrape_page',
+    description: 'Extract content from a single webpage',
+    inputSchema: {
+        type: 'object',
+        properties: {
+            url: { type: 'string', description: 'Webpage URL to scrape' },
+            onlyMainContent: {
+                type: 'boolean',
+                description: 'Extract only main content',
+                default: true
+            },
+        },
+        required: ['url'],
+    },
+};
+const BATCH_SCRAPE_TOOL = {
+    name: 'batch_scrape',
+    description: 'Scrape multiple URLs in a single request',
+    inputSchema: {
+        type: 'object',
+        properties: {
+            urls: {
+                type: 'array',
+                items: { type: 'string' },
+                description: 'Array of URLs to scrape'
+            },
+            onlyMainContent: {
+                type: 'boolean',
+                description: 'Extract only main content',
+                default: true
+            },
+        },
+        required: ['urls'],
+    },
+};
+const EXTRACT_DATA_TOOL = {
+    name: 'extract_data',
+    description: 'Extract structured data from webpages using LLM',
+    inputSchema: {
+        type: 'object',
+        properties: {
+            urls: {
+                type: 'array',
+                items: { type: 'string' },
+                description: 'URLs to extract data from'
+            },
+            prompt: {
+                type: 'string',
+                description: 'Instructions for what data to extract'
+            },
+            enableWebSearch: {
+                type: 'boolean',
+                description: 'Enable web search for additional context',
+                default: false
+            },
+        },
+        required: ['urls', 'prompt'],
+    },
+};
+const EXTRACT_WITH_SCHEMA_TOOL = {
+    name: 'extract_with_schema',
+    description: 'Extract structured data using a JSON schema',
+    inputSchema: {
+        type: 'object',
+        properties: {
+            urls: {
+                type: 'array',
+                items: { type: 'string' },
+                description: 'URLs to extract data from'
+            },
+            schema: {
+                type: 'object',
+                description: 'JSON schema defining the data structure to extract'
+            },
+            prompt: {
+                type: 'string',
+                description: 'Optional instructions for extraction'
+            },
+            enableWebSearch: {
+                type: 'boolean',
+                description: 'Enable web search for additional context',
+                default: false
+            },
+        },
+        required: ['urls', 'schema'],
+    },
+};
+// Lightweight tool definitions for essential Firecrawl functionality
+// Local web scraping functions
+async function scrapeWebpage(url, onlyMainContent = true) {
+    // SECURITY: Validate and sanitize URL
+    if (!isValidUrl(url)) {
+        return {
+            url,
+            title: '',
+            content: '',
+            markdown: '',
+            html: '',
+            success: false,
+            error: 'Invalid URL format. Only HTTP and HTTPS URLs are allowed.'
+        };
+    }
+    const sanitizedUrl = sanitizeUrl(url);
+    let browser;
+    try {
+        // Get proxy configuration
+        const proxyUrl = CONFIG.proxy.url;
+        const proxyUsername = CONFIG.proxy.username;
+        const proxyPassword = CONFIG.proxy.password;
+        // Get scraping configuration
+        const customUserAgent = CONFIG.scraping.userAgent;
+        const viewportWidth = CONFIG.scraping.viewportWidth;
+        const viewportHeight = CONFIG.scraping.viewportHeight;
+        const delayMin = CONFIG.scraping.delayMin;
+        const delayMax = CONFIG.scraping.delayMax;
+        // Build Puppeteer launch options with enhanced anti-detection
+        // SECURITY: Removed --disable-web-security which is a major security risk
+        const launchOptions = {
+            headless: true,
+            args: [
+                '--no-sandbox',
+                '--disable-setuid-sandbox',
+                '--disable-dev-shm-usage',
+                '--disable-accelerated-2d-canvas',
+                '--no-first-run',
+                '--no-zygote',
+                '--disable-gpu',
+                '--disable-features=VizDisplayCompositor',
+                `--user-agent=${customUserAgent}`
+            ]
+        };
+        // Add proxy configuration if available
+        if (proxyUrl) {
+            launchOptions.args.push(`--proxy-server=${proxyUrl}`);
+            // If proxy requires authentication, we'll handle it in the page setup
+            if (proxyUsername && proxyPassword) {
+                console.error(`Using authenticated proxy: ${proxyUrl}`);
+            }
+            else {
+                console.error(`Using proxy: ${proxyUrl}`);
+            }
+        }
+        browser = await puppeteer.launch(launchOptions);
+        const page = await browser.newPage();
+        // Enhanced anti-detection setup
+        await page.setUserAgent(customUserAgent);
+        // Set viewport to common desktop size
+        await page.setViewport({ width: viewportWidth, height: viewportHeight });
+        // Add common browser properties to avoid detection
+        await page.evaluateOnNewDocument(() => {
+            // Override navigator properties
+            Object.defineProperty(navigator, 'webdriver', { get: () => undefined });
+            Object.defineProperty(navigator, 'plugins', { get: () => [1, 2, 3, 4, 5] });
+            Object.defineProperty(navigator, 'languages', { get: () => ['en-US', 'en'] });
+            // Mock common browser APIs
+            window.chrome = { runtime: {} };
+        });
+        // Handle proxy authentication if credentials are provided
+        if (proxyUrl && proxyUsername && proxyPassword) {
+            await page.authenticate({
+                username: proxyUsername,
+                password: proxyPassword
+            });
+        }
+        // Add random delay before navigation
+        const delay = Math.floor(Math.random() * (delayMax - delayMin)) + delayMin;
+        await new Promise(resolve => setTimeout(resolve, delay));
+        // SECURITY: Use sanitized URL and add timeout
+        await page.goto(sanitizedUrl, {
+            waitUntil: 'networkidle2',
+            timeout: 30000
+        });
+        // Wait additional time for dynamic content
+        await new Promise(resolve => setTimeout(resolve, 2000));
+        // Extract title
+        const title = await page.title();
+        // Extract content based on preference
+        let content = '';
+        let markdown = '';
+        if (onlyMainContent) {
+            // Try to extract main content using common selectors
+            const mainContent = await page.evaluate(() => {
+                const selectors = [
+                    'main',
+                    '[role="main"]',
+                    '.content',
+                    '.post-content',
+                    '.entry-content',
+                    'article',
+                    '.article-content',
+                    '#content',
+                    '.main-content'
+                ];
+                for (const selector of selectors) {
+                    const element = document.querySelector(selector);
+                    if (element && element.textContent && element.textContent.trim().length > 100) {
+                        return element.textContent.trim();
+                    }
+                }
+                // Fallback to body content
+                return document.body.textContent || '';
+            });
+            content = mainContent;
+            markdown = content;
+        }
+        else {
+            // Extract full page content
+            content = await page.evaluate(() => document.body.textContent || '');
+            markdown = content;
+        }
+        // Get HTML
+        const html = await page.content();
+        return {
+            url,
+            title,
+            content,
+            markdown,
+            html,
+            success: true
+        };
+    }
+    catch (error) {
+        return {
+            url,
+            title: '',
+            content: '',
+            markdown: '',
+            html: '',
+            success: false,
+            error: error instanceof Error ? error.message : String(error)
+        };
+    }
+    finally {
+        if (browser) {
+            await browser.close();
+        }
+    }
+}
+async function extractDataWithLLM(url, prompt, schema) {
+    // SECURITY: Validate inputs
+    if (!isValidUrl(url)) {
+        return {
+            url,
+            data: null,
+            success: false,
+            error: 'Invalid URL format. Only HTTP and HTTPS URLs are allowed.'
+        };
+    }
+    if (!validatePrompt(prompt)) {
+        return {
+            url,
+            data: null,
+            success: false,
+            error: 'Invalid prompt. Prompt must be between 1 and 10,000 characters.'
+        };
+    }
+    const sanitizedUrl = sanitizeUrl(url);
+    const sanitizedPrompt = prompt.trim();
+    try {
+        // First scrape the webpage
+        const scraped = await scrapeWebpage(url, true);
+        if (!scraped.success) {
+            return {
+                url,
+                data: null,
+                success: false,
+                error: scraped.error
+            };
+        }
+        // Get LLM configuration
+        const LLM_API_KEY = CONFIG.llm.apiKey;
+        const LLM_PROVIDER_BASE_URL = CONFIG.llm.providerBaseUrl;
+        const LLM_MODEL = CONFIG.llm.model;
+        if (!LLM_API_KEY || !LLM_PROVIDER_BASE_URL || !LLM_MODEL) {
+            return {
+                url,
+                data: null,
+                success: false,
+                error: 'LLM configuration not available'
+            };
+        }
+        // Prepare the extraction prompt
+        const extractionPrompt = `
+You are a data extraction assistant. Extract information from the following webpage content based on the user's request.
+
+Webpage URL: ${sanitizedUrl}
+Webpage Title: ${scraped.title}
+
+Content:
+${scraped.content}
+
+${schema ? `Extract data according to this JSON schema: ${JSON.stringify(schema, null, 2)}` : ''}
+
+User Request: ${sanitizedPrompt}
+
+Please provide the extracted data in JSON format. ${schema ? 'Ensure the response matches the provided schema.' : 'Structure the data logically based on the content and request.'}
+`;
+        // Get proxy configuration for LLM API calls
+        const proxyUrl = CONFIG.proxy.url;
+        const proxyUsername = CONFIG.proxy.username;
+        const proxyPassword = CONFIG.proxy.password;
+        // Build axios configuration
+        const axiosConfig = {
+            headers: {
+                'Authorization': `Bearer ${LLM_API_KEY}`,
+                'Content-Type': 'application/json'
+            }
+        };
+        // Add proxy configuration if available
+        if (proxyUrl) {
+            const proxyConfig = {
+                host: proxyUrl.replace(/^https?:\/\//, '').split(':')[0],
+                port: parseInt(proxyUrl.split(':').pop() || '80'),
+                protocol: proxyUrl.startsWith('https') ? 'https' : 'http'
+            };
+            if (proxyUsername && proxyPassword) {
+                proxyConfig.auth = {
+                    username: proxyUsername,
+                    password: proxyPassword
+                };
+            }
+            axiosConfig.proxy = proxyConfig;
+            console.error(`Using proxy for LLM API: ${proxyUrl}`);
+        }
+        // Call LLM API with timeout for security
+        const response = await axios.post(`${LLM_PROVIDER_BASE_URL}/chat/completions`, {
+            model: LLM_MODEL,
+            messages: [
+                {
+                    role: 'user',
+                    content: extractionPrompt
+                }
+            ],
+            temperature: 0.1,
+            max_tokens: 2000
+        }, {
+            ...axiosConfig,
+            timeout: 60000, // 60 second timeout for security
+            maxContentLength: 10 * 1024 * 1024, // 10MB max response size
+            maxBodyLength: 10 * 1024 * 1024
+        });
+        const llmResponse = response.data.choices[0].message.content;
+        // Try to parse JSON from the response
+        try {
+            const extractedData = JSON.parse(llmResponse);
+            return {
+                url,
+                data: extractedData,
+                success: true
+            };
+        }
+        catch (parseError) {
+            // If JSON parsing fails, return the raw response
+            return {
+                url,
+                data: { raw_response: llmResponse },
+                success: true
+            };
+        }
+    }
+    catch (error) {
+        // SECURITY: Prevent information disclosure in error messages
+        const isAxiosError = axios.isAxiosError(error);
+        let safeErrorMessage = 'An error occurred while processing the request';
+        if (isAxiosError) {
+            // Only expose safe error information
+            if (error.response?.status === 401) {
+                safeErrorMessage = 'Authentication failed with LLM provider';
+            }
+            else if (error.response?.status === 429) {
+                safeErrorMessage = 'Rate limit exceeded with LLM provider';
+            }
+            else if (error.code === 'ECONNABORTED') {
+                safeErrorMessage = 'Request timeout - LLM provider took too long to respond';
+            }
+        }
+        return {
+            url: sanitizedUrl,
+            data: null,
+            success: false,
+            error: safeErrorMessage
+        };
+    }
+}
+// Type guards
+function isScrapeOptions(args) {
+    return (typeof args === 'object' &&
+        args !== null &&
+        'url' in args &&
+        typeof args.url === 'string');
+}
+function isBatchScrapeOptions(args) {
+    return (typeof args === 'object' &&
+        args !== null &&
+        'urls' in args &&
+        Array.isArray(args.urls));
+}
+function isExtractOptions(args) {
+    return (typeof args === 'object' &&
+        args !== null &&
+        'urls' in args &&
+        'prompt' in args &&
+        Array.isArray(args.urls) &&
+        typeof args.prompt === 'string');
+}
+// Remove all complex tools - keep only essential ones above
+// Server implementation
+const server = new Server({
+    name: 'firecrawl-lite-mcp-server',
+    version: '1.0.0',
+}, {
+    capabilities: {
+        tools: {},
+    },
+});
+// Configuration for retries and monitoring
+const CONFIG = {
+    scraping: {
+        userAgent: process.env.SCRAPE_USER_AGENT || DEFAULT_USER_AGENT,
+        viewportWidth: Number(process.env.SCRAPE_VIEWPORT_WIDTH) || DEFAULT_VIEWPORT_WIDTH,
+        viewportHeight: Number(process.env.SCRAPE_VIEWPORT_HEIGHT) || DEFAULT_VIEWPORT_HEIGHT,
+        delayMin: Number(process.env.SCRAPE_DELAY_MIN) || DEFAULT_SCRAPE_DELAY_MIN,
+        delayMax: Number(process.env.SCRAPE_DELAY_MAX) || DEFAULT_SCRAPE_DELAY_MAX,
+        batchDelayMin: Number(process.env.SCRAPE_BATCH_DELAY_MIN) || DEFAULT_BATCH_DELAY_MIN,
+        batchDelayMax: Number(process.env.SCRAPE_BATCH_DELAY_MAX) || DEFAULT_BATCH_DELAY_MAX,
+    },
+    retry: {
+        maxAttempts: Number(process.env.FIRECRAWL_RETRY_MAX_ATTEMPTS) || DEFAULT_RETRY_ATTEMPTS,
+        initialDelay: Number(process.env.FIRECRAWL_RETRY_INITIAL_DELAY) || DEFAULT_RETRY_INITIAL_DELAY,
+        maxDelay: Number(process.env.FIRECRAWL_RETRY_MAX_DELAY) || DEFAULT_RETRY_MAX_DELAY,
+        backoffFactor: Number(process.env.FIRECRAWL_RETRY_BACKOFF_FACTOR) || DEFAULT_RETRY_BACKOFF_FACTOR,
+    },
+    llm: {
+        apiKey: process.env.LLM_API_KEY,
+        providerBaseUrl: process.env.LLM_PROVIDER_BASE_URL,
+        model: process.env.LLM_MODEL,
+    },
+    proxy: {
+        url: process.env.PROXY_SERVER_URL,
+        username: process.env.PROXY_SERVER_USERNAME,
+        password: process.env.PROXY_SERVER_PASSWORD,
+    },
+};
+// Get LLM configuration
+const LLM_API_KEY = CONFIG.llm.apiKey;
+const LLM_PROVIDER_BASE_URL = CONFIG.llm.providerBaseUrl;
+const LLM_MODEL = CONFIG.llm.model;
+// Add utility function for delay
+function delay(ms) {
+    return new Promise((resolve) => setTimeout(resolve, ms));
+}
+let isStdioTransport = false;
+function safeLog(level, data) {
+    try {
+        // Always log to stderr to avoid relying on MCP logging capability
+        const message = `[${level}] ${typeof data === 'object' ? JSON.stringify(data) : String(data)}`;
+        console.error(message);
+    }
+    catch (_) {
+        // ignore
+    }
+}
+// Tool handlers
+server.setRequestHandler(ListToolsRequestSchema, async function listToolsRequestHandler() {
+    return {
+        tools: [
+            SCRAPE_TOOL,
+            BATCH_SCRAPE_TOOL,
+            EXTRACT_DATA_TOOL,
+            EXTRACT_WITH_SCHEMA_TOOL,
+        ],
+    };
+});
+server.setRequestHandler(CallToolRequestSchema, async function callToolRequestHandler(request) {
+    const startTime = Date.now();
+    try {
+        const { name, arguments: args } = request.params;
+        // Log incoming request with timestamp
+        safeLog('info', `[${new Date().toISOString()}] Received request for tool: ${name}`);
+        if (!args) {
+            throw new Error('No arguments provided');
+        }
+        switch (name) {
+            case 'scrape_page': {
+                if (!isScrapeOptions(args)) {
+                    throw new Error('Invalid arguments for scrape_page');
+                }
+                // SECURITY: Validate URL before processing
+                if (!isValidUrl(args.url)) {
+                    throw new Error('Invalid URL format. Only HTTP and HTTPS URLs are allowed.');
+                }
+                const result = await scrapeWebpage(args.url, args.onlyMainContent !== false);
+                return {
+                    content: [{ type: 'text', text: result.success ? result.markdown : `Error: ${result.error}` }],
+                    isError: !result.success,
+                };
+            }
+            case 'batch_scrape': {
+                if (!isBatchScrapeOptions(args)) {
+                    throw new Error('Invalid arguments for batch_scrape: urls array required');
+                }
+                // SECURITY: Validate all URLs before processing
+                const invalidUrls = args.urls.filter(url => !isValidUrl(url));
+                if (invalidUrls.length > 0) {
+                    throw new Error(`Invalid URL format detected: ${invalidUrls.join(', ')}. Only HTTP and HTTPS URLs are allowed.`);
+                }
+                // SECURITY: Limit batch size to prevent abuse
+                if (args.urls.length > 10) {
+                    throw new Error('Batch size limited to 10 URLs maximum for security and performance reasons.');
+                }
+                const results = [];
+                for (const url of args.urls) {
+                    try {
+                        const result = await scrapeWebpage(url, args.onlyMainContent !== false);
+                        results.push({
+                            url,
+                            success: result.success,
+                            title: result.title,
+                            content: result.success ? result.markdown : `Error: ${result.error}`
+                        });
+                    }
+                    catch (error) {
+                        results.push({
+                            url,
+                            success: false,
+                            error: error instanceof Error ? error.message : String(error)
+                        });
+                    }
+                    // Add random delay between requests to avoid rate limiting
+                    const batchDelay = Math.floor(Math.random() * (CONFIG.scraping.batchDelayMax - CONFIG.scraping.batchDelayMin)) + CONFIG.scraping.batchDelayMin;
+                    await delay(batchDelay);
+                }
+                return {
+                    content: [{ type: 'text', text: JSON.stringify(results, null, 2) }],
+                    isError: false,
+                };
+            }
+            case 'extract_data': {
+                if (!isExtractOptions(args)) {
+                    throw new Error('Invalid arguments for extract_data: urls array and prompt required');
+                }
+                // SECURITY: Validate all URLs
+                const invalidUrls = args.urls.filter(url => !isValidUrl(url));
+                if (invalidUrls.length > 0) {
+                    throw new Error(`Invalid URL format detected: ${invalidUrls.join(', ')}. Only HTTP and HTTPS URLs are allowed.`);
+                }
+                // SECURITY: Validate prompt
+                if (!validatePrompt(args.prompt)) {
+                    throw new Error('Invalid prompt. Prompt must be between 1 and 10,000 characters.');
+                }
+                // SECURITY: Limit batch size
+                if (args.urls.length > 5) {
+                    throw new Error('Extraction limited to 5 URLs maximum for security and performance reasons.');
+                }
+                const results = [];
+                for (const url of args.urls) {
+                    try {
+                        const result = await extractDataWithLLM(url, args.prompt);
+                        results.push({
+                            url,
+                            success: result.success,
+                            data: result.success ? result.data : `Error: ${result.error}`
+                        });
+                    }
+                    catch (error) {
+                        results.push({
+                            url,
+                            success: false,
+                            error: error instanceof Error ? error.message : String(error)
+                        });
+                    }
+                    // Add delay between requests
+                    await delay(1000);
+                }
+                return {
+                    content: [{ type: 'text', text: JSON.stringify(results, null, 2) }],
+                    isError: false,
+                };
+            }
+            case 'extract_with_schema': {
+                if (!isExtractOptions(args) || !args.schema) {
+                    throw new Error('Invalid arguments for extract_with_schema: urls array, schema, and prompt required');
+                }
+                // Security validations
+                if (!Array.isArray(args.urls) || args.urls.length === 0) {
+                    throw new Error('Invalid arguments: urls must be a non-empty array');
+                }
+                if (args.urls.length > MAX_URLS_PER_REQUEST) {
+                    throw new Error(`Too many URLs: maximum ${MAX_URLS_PER_REQUEST} URLs allowed per request`);
+                }
+                // Validate and sanitize each URL
+                const sanitizedUrls = [];
+                for (const url of args.urls) {
+                    if (typeof url !== 'string') {
+                        throw new Error('Invalid URL format: all URLs must be strings');
+                    }
+                    if (!isValidUrl(url)) {
+                        throw new Error(`Invalid URL: ${url}`);
+                    }
+                    const sanitizedUrl = sanitizeUrl(url);
+                    if (!sanitizedUrl) {
+                        throw new Error(`Failed to sanitize URL: ${url}`);
+                    }
+                    sanitizedUrls.push(sanitizedUrl);
+                }
+                // Validate and sanitize prompt
+                if (typeof args.prompt !== 'string') {
+                    throw new Error('Invalid prompt: must be a string');
+                }
+                if (!validatePrompt(args.prompt)) {
+                    throw new Error('Invalid prompt: must be between 1 and 10,000 characters');
+                }
+                const sanitizedPrompt = args.prompt.trim();
+                // Validate schema (basic validation)
+                if (typeof args.schema !== 'object' || args.schema === null) {
+                    throw new Error('Invalid schema: must be a valid object');
+                }
+                const results = [];
+                for (const url of sanitizedUrls) {
+                    try {
+                        const result = await extractDataWithLLM(url, sanitizedPrompt, args.schema);
+                        results.push({
+                            url,
+                            success: result.success,
+                            data: result.success ? result.data : `Error: ${result.error}`
+                        });
+                    }
+                    catch (error) {
+                        // Prevent information disclosure in error messages
+                        const safeErrorMessage = error instanceof Error ? error.message : 'Unknown error occurred';
+                        results.push({
+                            url,
+                            success: false,
+                            error: safeErrorMessage.replace(/[^\w\s\-.:]/g, '') // Remove potentially sensitive characters
+                        });
+                    }
+                    // Add delay between requests to prevent rate limiting
+                    await delay(1000);
+                }
+                return {
+                    content: [{ type: 'text', text: JSON.stringify(results, null, 2) }],
+                    isError: false,
+                };
+            }
+            default:
+                return {
+                    content: [{ type: 'text', text: `Unknown tool: ${name}` }],
+                    isError: true,
+                };
+        }
+    }
+    catch (error) {
+        // Log detailed error information
+        safeLog('error', {
+            message: `Request failed: ${error instanceof Error ? error.message : String(error)}`,
+            tool: request.params.name,
+            arguments: request.params.arguments,
+            timestamp: new Date().toISOString(),
+            duration: Date.now() - startTime,
+        });
+        return {
+            content: [
+                {
+                    type: 'text',
+                    text: trimResponseText(`Error: ${error instanceof Error ? error.message : String(error)}`),
+                },
+            ],
+            isError: true,
+        };
+    }
+    finally {
+        // Log request completion with performance metrics
+        safeLog('info', `Request completed in ${Date.now() - startTime}ms`);
+    }
+});
+// Helper function to format results
+function formatResults(data) {
+    return data
+        .map((doc) => {
+        const content = doc.markdown || doc.content || 'No content';
+        return `Title: ${doc.title}
+Content: ${content.substring(0, 100)}${content.length > 100 ? '...' : ''}`;
+    })
+        .join('\n\n');
+}
+// Utility function to trim trailing whitespace from text responses
+// This prevents Claude API errors with "final assistant content cannot end with trailing whitespace"
+function trimResponseText(text) {
+    return text.trim();
+}
+// Server startup
+async function runLocalServer() {
+    try {
+        console.error('Initializing Firecrawl Lite MCP Server...');
+        const transport = new StdioServerTransport();
+        // Detect if we're using stdio transport
+        isStdioTransport = transport instanceof StdioServerTransport;
+        if (isStdioTransport) {
+            console.error('Running in stdio mode, logging will be directed to stderr');
+        }
+        await server.connect(transport);
+        // Now that we're connected, we can send logging messages
+        safeLog('info', 'Firecrawl Lite MCP Server initialized successfully');
+        safeLog('info', `LLM Configuration: ${LLM_PROVIDER_BASE_URL ? 'Configured' : 'Not configured'} (${LLM_MODEL || 'no model'})`);
+        console.error('Firecrawl Lite MCP Server running on stdio');
+    }
+    catch (error) {
+        console.error('Fatal error running server:', error);
+        process.exit(1);
+    }
+}
+async function runSSELocalServer() {
+    let transport = null;
+    const app = express();
+    app.get('/sse', async (req, res) => {
+        transport = new SSEServerTransport(`/messages`, res);
+        res.on('close', () => {
+            transport = null;
+        });
+        await server.connect(transport);
+    });
+    // Endpoint for the client to POST messages
+    // Remove express.json() middleware - let the transport handle the body
+    app.post('/messages', (req, res) => {
+        if (transport) {
+            transport.handlePostMessage(req, res);
+        }
+    });
+    const PORT = process.env.PORT || 3000;
+    console.log('Starting server on port', PORT);
+    try {
+        app.listen(PORT, () => {
+            console.log(`MCP SSE Server listening on http://localhost:${PORT}`);
+            console.log(`SSE endpoint: http://localhost:${PORT}/sse`);
+            console.log(`Message endpoint: http://localhost:${PORT}/messages`);
+        });
+    }
+    catch (error) {
+        console.error('Error starting server:', error);
+    }
+}
+async function runHTTPStreamableServer() {
+    const app = express();
+    app.use(express.json());
+    // Health check endpoint
+    app.get('/health', (req, res) => {
+        res.status(200).json({
+            status: 'OK',
+            server: 'Firecrawl Lite MCP Server',
+            version: '1.0.0',
+            timestamp: new Date().toISOString()
+        });
+    });
+    const transports = {};
+    // A single endpoint handles all MCP requests.
+    app.all('/mcp', async (req, res) => {
+        try {
+            const sessionId = req.headers['mcp-session-id'];
+            let transport;
+            if (sessionId && transports[sessionId]) {
+                transport = transports[sessionId];
+            }
+            else if (!sessionId &&
+                req.method === 'POST' &&
+                req.body &&
+                typeof req.body === 'object' &&
+                req.body.method === 'initialize') {
+                transport = new StreamableHTTPServerTransport({
+                    sessionIdGenerator: () => {
+                        const id = randomUUID();
+                        return id;
+                    },
+                    onsessioninitialized: (sid) => {
+                        transports[sid] = transport;
+                    },
+                });
+                transport.onclose = () => {
+                    const sid = transport.sessionId;
+                    if (sid && transports[sid]) {
+                        delete transports[sid];
+                    }
+                };
+                console.log('Creating server instance');
+                console.log('Connecting transport to server');
+                await server.connect(transport);
+                await transport.handleRequest(req, res, req.body);
+                return;
+            }
+            else {
+                res.status(400).json({
+                    jsonrpc: '2.0',
+                    error: {
+                        code: -32000,
+                        message: 'Invalid or missing session ID',
+                    },
+                    id: null,
+                });
+                return;
+            }
+            await transport.handleRequest(req, res, req.body);
+        }
+        catch (error) {
+            if (!res.headersSent) {
+                res.status(500).json({
+                    jsonrpc: '2.0',
+                    error: {
+                        code: -32603,
+                        message: 'Internal server error',
+                    },
+                    id: null,
+                });
+            }
+        }
+    });
+    const PORT = 3000;
+    const appServer = app.listen(PORT, () => {
+        console.log(`MCP Streamable HTTP Server listening on port ${PORT}`);
+    });
+    process.on('SIGINT', async () => {
+        console.log('Shutting down server...');
+        for (const sessionId in transports) {
+            try {
+                console.log(`Closing transport for session ${sessionId}`);
+                await transports[sessionId].close();
+                delete transports[sessionId];
+            }
+            catch (error) {
+                console.error(`Error closing transport for session ${sessionId}:`, error);
+            }
+        }
+        appServer.close(() => {
+            console.log('Server shutdown complete');
+            process.exit(0);
+        });
+    });
+}
+// Server startup - standalone MCP server
+runLocalServer().catch((error) => {
+    console.error('Fatal error running server:', error);
+    process.exit(1);
+});

package/package.json

@@ -0,0 +1,45 @@
+{
+  "name": "@ariangibson/firecrawl-lite-mcp-server",
+  "version": "1.1.1",
+  "description": "Privacy-first, standalone MCP server for web scraping and data extraction using local browser automation and your own LLM API key",
+  "type": "module",
+  "bin": {
+    "firecrawl-lite-mcp-server": "dist/index.js"
+  },
+  "files": [
+    "dist"
+  ],
+  "scripts": {
+    "build": "tsc",
+    "start": "node dist/index.js",
+    "dev": "tsc && node dist/index.js",
+    "lint": "tsc --noEmit"
+  },
+  "license": "MIT",
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.17.3",
+    "@types/cheerio": "^0.22.35",
+    "axios": "^1.11.0",
+    "cheerio": "^1.1.2",
+    "dotenv": "^16.4.7",
+    "express": "^5.1.0",
+    "puppeteer": "^24.17.1",
+    "ws": "^8.18.1"
+  },
+  "devDependencies": {
+    "@types/express": "^5.0.1",
+    "@types/node": "^20.10.5",
+    "typescript": "^5.9.2"
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/ariangibson/firecrawl-lite-mcp-server.git"
+  },
+  "homepage": "https://github.com/ariangibson/firecrawl-lite-mcp-server#readme",
+  "bugs": {
+    "url": "https://github.com/ariangibson/firecrawl-lite-mcp-server/issues"
+  }
+}