scraperapi-mcp-server 0.1.0__tar.gz

scraperapi_mcp_server-0.1.0/PKG-INFO

@@ -0,0 +1,207 @@
+Metadata-Version: 2.3
+Name: scraperapi-mcp-server
+Version: 0.1.0
+Summary: ScraperAPI MCP server
+Keywords: http,mcp,llm,automation,scraping
+Author: ScraperAPI
+Requires-Python: >=3.11
+Classifier: Development Status :: 1 - Planning
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.10
+Requires-Dist: mcp[cli] (>=1.6.0)
+Requires-Dist: pydantic (>=2.11.3)
+Requires-Dist: pydantic-settings (>=2.9.1)
+Requires-Dist: requests (>=2.32.3)
+Description-Content-Type: text/markdown
+
+# ScraperAPI MCP server
+
+The ScraperAPI MCP server enables LLM clients to retrieve and process web scraping requests using the ScraperAPI services.
+
+## Table of Contents
+
+- [Features](#features)
+- [Architecture](#architecture)
+- [Installation](#installation)
+- [API Reference](#api-reference)
+- [Configuration](#configuration)
+- [Development](#development)
+
+## Features
+
+- Full implementation of the Model Context Protocol specification
+- Seamless integration with ScraperAPI for web scraping
+- Simple setup with Docker or Python
+
+## Architecture
+
+```
+          ┌───────────────┐     ┌───────────────────────┐     ┌───────────────┐
+          │  LLM Client   │────▶│  Scraper MCP Server   │────▶│    AI Model   │
+          └───────────────┘     └───────────────────────┘     └───────────────┘
+                                            │
+                                            ▼
+                                  ┌──────────────────┐
+                                  │  ScraperAPI API  │
+                                  └──────────────────┘
+```
+
+## Installation
+
+The ScraperAPI MCP Server is designed to run as a local server on your machine, your LLM client will launch it automatically when configured.
+
+### Prerequisites
+- Python 3.11+
+- Docker (optional)
+
+### Using Python
+
+Add this to your client configuration file:
+
+```json
+{
+  "mcpServers": {
+    "ScraperAPI": {
+      "command": "python",
+      "args": ["-m", "scraperapi_mcp_server"],
+      "env": {
+        "API_KEY": "<YOUR_SCRAPERAPI_API_KEY>"
+      }
+    }
+  }
+}
+```
+
+### Using Docker 
+
+Add this to your client configuration file:
+
+```json
+{
+  "mcpServers": {
+    "ScraperAPI": {
+      "command": "docker",
+      "args": [
+        "run",
+        "-i",
+        "-e",
+        "API_KEY=${API_KEY}",
+        "--rm",
+        "scraperapi-mcp-server"]
+    }
+  }
+}
+```
+
+</br>
+
+> [!TIP]
+>
+> If your command is not working (for example, you see a `package not found` error when trying to start the server), double-check the path you are using. To find the correct path, activate your virtual environment first, then run:
+>    ```bash
+>    which <YOUR_COMMAND>
+>    ```
+
+## API Reference
+
+### Available Tools
+
+- `scrape`
+  - Scrape a URL from the internet using ScraperAPI
+  - Parameters:
+    - `url` (string, required): URL to scrape
+    - `render` (boolean, optional): Whether to render the page using JavaScript. Defaults to `False`. Set to `True` only if the page requires JavaScript rendering to display its content.
+    - `country_code` (string, optional): Activate country geotargeting (ISO 2-letter code)
+    - `premium` (boolean, optional): Activate premium residential and mobile IPs
+    - `ultra_premium` (boolean, optional): Activate advanced bypass mechanisms. Can not combine with `premium`
+    - `device_type` (string, optional): Set request to use `mobile` or `desktop` user agents
+  - Returns: The scraped content as a string
+
+### Prompt templates
+
+- Please scrape this URL `<URL>`. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.
+- Can you scrape URL `<URL>` to extract `<SPECIFIC_DATA>`? If the request returns missing/incomplete`<SPECIFIC_DATA>`, set render=true to enable JS Rendering.
+
+## Configuration
+
+### Settings
+
+- `API_KEY`: Your ScraperAPI API key.
+
+### Configure for Claude Desktop App
+
+1. Open Claude Desktop Application
+2. Access the Settings Menu
+3. Click on the settings icon (typically a gear or three dots in the upper right corner)
+4. Select the "Developer" tab
+5. Click on "Edit Config" and paste [the JSON configuration file](#installation).
+
+## Development
+
+### Local setup
+
+1. **Clone the repository:**
+   ```bash
+   git clone https://github.com/scraperapi/scraperapi-mcp
+   cd scraperapi-mcp
+   ```
+
+2. **Install dependencies and run the package locally:**
+   - **Using Python:**
+     ```bash
+     # Create virtual environment and activate it
+     python -m venv .venv
+     source .venv/bin/activate # MacOS/Linux
+     # OR
+     .venv/Scripts/activate # Windows
+
+     # Install the local package in editable mode
+     pip install -e .
+     ```
+
+   - **Using Docker:**
+     ```bash
+     # Build the Docker image locally
+     docker build -t scraperapi-mcp-server .
+     ```
+
+### Run the server
+   - **Using Python:**
+     ```bash
+     python -m scraperapi_mcp_server
+     ```
+
+   - **Using Docker:**
+     ```bash
+     # Run the Docker container with your API key
+     docker run -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> scraperapi-mcp-server
+     ```
+
+### Debug
+
+```bash
+python3 -m scraperapi_mcp_server --debug
+```
+
+### Testing
+
+This project uses [pytest](https://docs.pytest.org/en/stable/) for testing.
+
+#### Install Test Dependencies
+```bash
+# Install the package with test dependencies
+pip install -e ".[test]"
+```
+
+#### Running Tests
+
+```bash
+# Run All Tests
+pytest
+
+# Run Specific Test
+pytest <TEST_FILE_PATH>
+```

scraperapi_mcp_server-0.1.0/README.md

@@ -0,0 +1,188 @@
+# ScraperAPI MCP server
+
+The ScraperAPI MCP server enables LLM clients to retrieve and process web scraping requests using the ScraperAPI services.
+
+## Table of Contents
+
+- [Features](#features)
+- [Architecture](#architecture)
+- [Installation](#installation)
+- [API Reference](#api-reference)
+- [Configuration](#configuration)
+- [Development](#development)
+
+## Features
+
+- Full implementation of the Model Context Protocol specification
+- Seamless integration with ScraperAPI for web scraping
+- Simple setup with Docker or Python
+
+## Architecture
+
+```
+          ┌───────────────┐     ┌───────────────────────┐     ┌───────────────┐
+          │  LLM Client   │────▶│  Scraper MCP Server   │────▶│    AI Model   │
+          └───────────────┘     └───────────────────────┘     └───────────────┘
+                                            │
+                                            ▼
+                                  ┌──────────────────┐
+                                  │  ScraperAPI API  │
+                                  └──────────────────┘
+```
+
+## Installation
+
+The ScraperAPI MCP Server is designed to run as a local server on your machine, your LLM client will launch it automatically when configured.
+
+### Prerequisites
+- Python 3.11+
+- Docker (optional)
+
+### Using Python
+
+Add this to your client configuration file:
+
+```json
+{
+  "mcpServers": {
+    "ScraperAPI": {
+      "command": "python",
+      "args": ["-m", "scraperapi_mcp_server"],
+      "env": {
+        "API_KEY": "<YOUR_SCRAPERAPI_API_KEY>"
+      }
+    }
+  }
+}
+```
+
+### Using Docker 
+
+Add this to your client configuration file:
+
+```json
+{
+  "mcpServers": {
+    "ScraperAPI": {
+      "command": "docker",
+      "args": [
+        "run",
+        "-i",
+        "-e",
+        "API_KEY=${API_KEY}",
+        "--rm",
+        "scraperapi-mcp-server"]
+    }
+  }
+}
+```
+
+</br>
+
+> [!TIP]
+>
+> If your command is not working (for example, you see a `package not found` error when trying to start the server), double-check the path you are using. To find the correct path, activate your virtual environment first, then run:
+>    ```bash
+>    which <YOUR_COMMAND>
+>    ```
+
+## API Reference
+
+### Available Tools
+
+- `scrape`
+  - Scrape a URL from the internet using ScraperAPI
+  - Parameters:
+    - `url` (string, required): URL to scrape
+    - `render` (boolean, optional): Whether to render the page using JavaScript. Defaults to `False`. Set to `True` only if the page requires JavaScript rendering to display its content.
+    - `country_code` (string, optional): Activate country geotargeting (ISO 2-letter code)
+    - `premium` (boolean, optional): Activate premium residential and mobile IPs
+    - `ultra_premium` (boolean, optional): Activate advanced bypass mechanisms. Can not combine with `premium`
+    - `device_type` (string, optional): Set request to use `mobile` or `desktop` user agents
+  - Returns: The scraped content as a string
+
+### Prompt templates
+
+- Please scrape this URL `<URL>`. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.
+- Can you scrape URL `<URL>` to extract `<SPECIFIC_DATA>`? If the request returns missing/incomplete`<SPECIFIC_DATA>`, set render=true to enable JS Rendering.
+
+## Configuration
+
+### Settings
+
+- `API_KEY`: Your ScraperAPI API key.
+
+### Configure for Claude Desktop App
+
+1. Open Claude Desktop Application
+2. Access the Settings Menu
+3. Click on the settings icon (typically a gear or three dots in the upper right corner)
+4. Select the "Developer" tab
+5. Click on "Edit Config" and paste [the JSON configuration file](#installation).
+
+## Development
+
+### Local setup
+
+1. **Clone the repository:**
+   ```bash
+   git clone https://github.com/scraperapi/scraperapi-mcp
+   cd scraperapi-mcp
+   ```
+
+2. **Install dependencies and run the package locally:**
+   - **Using Python:**
+     ```bash
+     # Create virtual environment and activate it
+     python -m venv .venv
+     source .venv/bin/activate # MacOS/Linux
+     # OR
+     .venv/Scripts/activate # Windows
+
+     # Install the local package in editable mode
+     pip install -e .
+     ```
+
+   - **Using Docker:**
+     ```bash
+     # Build the Docker image locally
+     docker build -t scraperapi-mcp-server .
+     ```
+
+### Run the server
+   - **Using Python:**
+     ```bash
+     python -m scraperapi_mcp_server
+     ```
+
+   - **Using Docker:**
+     ```bash
+     # Run the Docker container with your API key
+     docker run -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> scraperapi-mcp-server
+     ```
+
+### Debug
+
+```bash
+python3 -m scraperapi_mcp_server --debug
+```
+
+### Testing
+
+This project uses [pytest](https://docs.pytest.org/en/stable/) for testing.
+
+#### Install Test Dependencies
+```bash
+# Install the package with test dependencies
+pip install -e ".[test]"
+```
+
+#### Running Tests
+
+```bash
+# Run All Tests
+pytest
+
+# Run Specific Test
+pytest <TEST_FILE_PATH>
+```

scraperapi_mcp_server-0.1.0/pyproject.toml

@@ -0,0 +1,35 @@
+[tool.poetry]
+name = "scraperapi-mcp-server"
+version = "0.1.0"
+description = "ScraperAPI MCP server"
+readme = "README.md"
+authors = ["ScraperAPI"]
+keywords = ["http", "mcp", "llm", "automation", "scraping"]
+classifiers = [
+    "Development Status :: 1 - Planning",
+    "Programming Language :: Python :: 3.10",
+]
+packages = [{include = "scraperapi_mcp_server", from = "src"}]
+
+[tool.poetry.dependencies]
+python = ">=3.11"
+mcp = {extras = ["cli"], version = ">=1.6.0"}
+pydantic = ">=2.11.3"
+pydantic-settings = ">=2.9.1"
+requests = ">=2.32.3"
+
+[tool.poetry.group.dev.dependencies]
+pytest = ">=7.4.0"
+pytest-mock = ">=3.10.0"
+
+[tool.poetry.scripts]
+scraperapi-mcp-server = "scraperapi_mcp_server:main"
+
+[tool.pytest.ini_options]
+markers = [
+    "integration: marks tests as integration tests",
+]
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/__init__.py

@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+import logging
+from scraperapi_mcp_server.server import mcp
+
+__version__ = "0.1.0"
+
+
+def main():
+    """ScraperAPI MCP server main module."""
+    logging.info("Starting ScraperAPI MCP server main module.")
+    # Run the server
+    try:
+        logging.debug("Running ScraperAPI MCP server...")
+        mcp.run()
+    except Exception as e:
+        logging.exception(
+            f"Unhandled exception in ScraperAPI MCP server main loop: {e}"
+        )
+        raise
+
+
+if __name__ == "__main__":
+    logging.basicConfig(
+        level=logging.INFO,
+        format="%(asctime)s %(levelname)s %(message)s",
+    )
+    main()

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/__main__.py

@@ -0,0 +1,5 @@
+# __main__.py
+
+from scraperapi_mcp_server import main
+
+main()

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/config.py

@@ -0,0 +1,25 @@
+import os
+from dotenv import load_dotenv
+from pydantic_settings import BaseSettings
+from scraperapi_mcp_server.utils.exceptions import ApiKeyEnvVarNotSetError
+
+
+load_dotenv()
+
+
+class Settings(BaseSettings):
+    """ScraperAPI MCP Server settings."""
+
+    API_KEY: str = os.getenv("API_KEY", "")
+    API_URL: str = "https://api.scraperapi.com"
+    API_TIMEOUT_SECONDS: int = 70
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        if not self.API_KEY:
+            raise ApiKeyEnvVarNotSetError(
+                "API_KEY environment variable is not set. Please set it when installing the MCP server. Check the README for more information."
+            )
+
+
+settings = Settings()

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/scrape/models.py

@@ -0,0 +1,31 @@
+from typing import Annotated
+from pydantic import BaseModel, Field, AnyUrl
+
+
+class Scrape(BaseModel):
+    """Parameters for scraping a URL."""
+
+    url: Annotated[AnyUrl, Field(description="URL to scrape")]
+    render: Annotated[
+        bool,
+        Field(
+            default=False,
+            description="Whether to render the page using JavaScript. Set to `True` only if the page requires JavaScript rendering to display its content.",
+        ),
+    ]
+    country_code: Annotated[
+        str, Field(default=None, description="Country code to scrape from")
+    ]
+    premium: Annotated[
+        bool, Field(default=False, description="Whether to use premium scraping")
+    ]
+    ultra_premium: Annotated[
+        bool, Field(default=False, description="Whether to use ultra premium scraping")
+    ]
+    device_type: Annotated[
+        str,
+        Field(
+            default=None,
+            description="Device type to scrape from. Set request to use `mobile` or `desktop` user agents",
+        ),
+    ]

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/scrape/scrape.py

@@ -0,0 +1,45 @@
+import requests
+from scraperapi_mcp_server.config import settings
+from scraperapi_mcp_server.utils.exceptions import handle_scraper_error
+import logging
+
+
+def basic_scrape(
+    url: str,
+    render: bool = None,
+    country_code: str = None,
+    premium: bool = None,
+    ultra_premium: bool = None,
+    device_type: str = None,
+) -> str:
+    logging.info(f"Starting scrape for URL: {url}")
+    payload = {
+        "api_key": settings.API_KEY,
+        "url": url,
+        "output_format": "markdown",
+        "scraper_sdk": "mcp-server",
+    }
+    logging.debug(f"Initial payload: {payload}")
+    optional_params = {
+        "render": (render, lambda v: str(v).lower()),
+        "country_code": (country_code, str),
+        "premium": (premium, lambda v: str(v).lower()),
+        "ultra_premium": (ultra_premium, lambda v: str(v).lower()),
+        "device_type": (device_type, str),
+    }
+    for key, (value, formatter) in optional_params.items():
+        if value is not None:
+            payload[key] = formatter(value)
+            logging.debug(f"Added optional param: {key}={payload[key]}")
+    try:
+        logging.info(f"Sending request to {settings.API_URL} with params: {payload}")
+        response = requests.get(
+            settings.API_URL, params=payload, timeout=settings.API_TIMEOUT_SECONDS
+        )
+        response.raise_for_status()
+        logging.info(f"Scrape successful for URL: {url}")
+        return response.text
+    except Exception as e:
+        logging.error(f"Error during scrape for URL: {url}", exc_info=True)
+        error_obj = handle_scraper_error(e, url, payload)
+        raise Exception(error_obj.error.message) from e

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/server.py

@@ -0,0 +1,45 @@
+from mcp.server.fastmcp import FastMCP
+from scraperapi_mcp_server.scrape.models import Scrape
+from scraperapi_mcp_server.scrape.scrape import basic_scrape
+import logging
+
+mcp = FastMCP("ScraperAPI")
+
+
+@mcp.tool()
+def scrape(params: Scrape) -> str:
+    """
+    Execute a web scrape using ScraperAPI with the specified parameters.
+
+    Parameters:
+        params: Scrape model containing:
+            url: Target URL to scrape (required)
+            render: Enable JavaScript rendering only when needed for dynamic content (default: False)
+                    Set to True ONLY if the content you need is missing from the initial HTML response and is loaded dynamically by JavaScript.
+                    For most websites, including many modern ones, the main content is available without JavaScript rendering.
+            country_code: Two-letter country code for geo-specific scraping
+            premium: Use premium residential/mobile proxies for higher success rate (costs more, incompatible with ultra_premium)
+            ultra_premium: Activate advanced bypass mechanisms (costs more, incompatible with premium)
+            device_type: 'mobile' or 'desktop' for device-specific user agents
+
+    Returns:
+        Scraped content as a string
+    """
+
+    logging.info(f"Invoking scrape tool with params: {params}")
+    try:
+        result = basic_scrape(
+            url=str(params.url),
+            render=params.render,
+            country_code=params.country_code,
+            premium=params.premium,
+            ultra_premium=params.ultra_premium,
+            device_type=params.device_type,
+        )
+        logging.info(f"Scrape tool completed for URL: {params.url}")
+        return result
+    except Exception as e:
+        logging.error(
+            f"Scrape tool failed for URL: {params.url}. Error: {e}", exc_info=True
+        )
+        raise

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/utils/country_codes.py

@@ -0,0 +1,416 @@
+"""
+Mapping of country names to their ISO 3166-1 alpha-2 country codes.
+Used for scraping from specific geographic locations.
+"""
+
+COUNTRY_CODES = {
+    # North America
+    "us": "us",
+    "usa": "us",
+    "united states": "us",
+    "united states of america": "us",
+    "ca": "ca",
+    "canada": "ca",
+    "mx": "mx",
+    "mexico": "mx",
+    "cu": "cu",
+    "cuba": "cu",
+    "do": "do",
+    "dominican republic": "do",
+    "ht": "ht",
+    "haiti": "ht",
+    "jm": "jm",
+    "jamaica": "jm",
+    "pr": "pr",
+    "puerto rico": "pr",
+    # Central America
+    "bz": "bz",
+    "belize": "bz",
+    "cr": "cr",
+    "costa rica": "cr",
+    "sv": "sv",
+    "el salvador": "sv",
+    "gt": "gt",
+    "guatemala": "gt",
+    "hn": "hn",
+    "honduras": "hn",
+    "ni": "ni",
+    "nicaragua": "ni",
+    "pa": "pa",
+    "panama": "pa",
+    # South America
+    "ar": "ar",
+    "argentina": "ar",
+    "bo": "bo",
+    "bolivia": "bo",
+    "br": "br",
+    "brazil": "br",
+    "cl": "cl",
+    "chile": "cl",
+    "co": "co",
+    "colombia": "co",
+    "ec": "ec",
+    "ecuador": "ec",
+    "gy": "gy",
+    "guyana": "gy",
+    "py": "py",
+    "paraguay": "py",
+    "pe": "pe",
+    "peru": "pe",
+    "sr": "sr",
+    "suriname": "sr",
+    "uy": "uy",
+    "uruguay": "uy",
+    "ve": "ve",
+    "venezuela": "ve",
+    # Europe
+    "al": "al",
+    "albania": "al",
+    "ad": "ad",
+    "andorra": "ad",
+    "at": "at",
+    "austria": "at",
+    "by": "by",
+    "belarus": "by",
+    "be": "be",
+    "belgium": "be",
+    "ba": "ba",
+    "bosnia and herzegovina": "ba",
+    "bg": "bg",
+    "bulgaria": "bg",
+    "hr": "hr",
+    "croatia": "hr",
+    "cz": "cz",
+    "czech republic": "cz",
+    "dk": "dk",
+    "denmark": "dk",
+    "ee": "ee",
+    "estonia": "ee",
+    "fi": "fi",
+    "finland": "fi",
+    "fr": "fr",
+    "france": "fr",
+    "de": "de",
+    "germany": "de",
+    "gr": "gr",
+    "greece": "gr",
+    "hu": "hu",
+    "hungary": "hu",
+    "is": "is",
+    "iceland": "is",
+    "ie": "ie",
+    "ireland": "ie",
+    "it": "it",
+    "italy": "it",
+    "lv": "lv",
+    "latvia": "lv",
+    "li": "li",
+    "liechtenstein": "li",
+    "lt": "lt",
+    "lithuania": "lt",
+    "lu": "lu",
+    "luxembourg": "lu",
+    "mt": "mt",
+    "malta": "mt",
+    "md": "md",
+    "moldova": "md",
+    "mc": "mc",
+    "monaco": "mc",
+    "me": "me",
+    "montenegro": "me",
+    "nl": "nl",
+    "netherlands": "nl",
+    "mk": "mk",
+    "north macedonia": "mk",
+    "no": "no",
+    "norway": "no",
+    "pl": "pl",
+    "poland": "pl",
+    "pt": "pt",
+    "portugal": "pt",
+    "ro": "ro",
+    "romania": "ro",
+    "ru": "ru",
+    "russia": "ru",
+    "russian federation": "ru",
+    "sm": "sm",
+    "san marino": "sm",
+    "rs": "rs",
+    "serbia": "rs",
+    "sk": "sk",
+    "slovakia": "sk",
+    "si": "si",
+    "slovenia": "si",
+    "es": "es",
+    "spain": "es",
+    "se": "se",
+    "sweden": "se",
+    "ch": "ch",
+    "switzerland": "ch",
+    "ua": "ua",
+    "ukraine": "ua",
+    "uk": "gb",
+    "united kingdom": "gb",
+    "great britain": "gb",
+    "gb": "gb",
+    "va": "va",
+    "vatican city": "va",
+    # Asia
+    "af": "af",
+    "afghanistan": "af",
+    "am": "am",
+    "armenia": "am",
+    "az": "az",
+    "azerbaijan": "az",
+    "bh": "bh",
+    "bahrain": "bh",
+    "bd": "bd",
+    "bangladesh": "bd",
+    "bt": "bt",
+    "bhutan": "bt",
+    "bn": "bn",
+    "brunei": "bn",
+    "kh": "kh",
+    "cambodia": "kh",
+    "cn": "cn",
+    "china": "cn",
+    "peoples republic of china": "cn",
+    "cy": "cy",
+    "cyprus": "cy",
+    "ge": "ge",
+    "georgia": "ge",
+    "in": "in",
+    "india": "in",
+    "id": "id",
+    "indonesia": "id",
+    "ir": "ir",
+    "iran": "ir",
+    "iq": "iq",
+    "iraq": "iq",
+    "il": "il",
+    "israel": "il",
+    "jp": "jp",
+    "japan": "jp",
+    "jo": "jo",
+    "jordan": "jo",
+    "kz": "kz",
+    "kazakhstan": "kz",
+    "kw": "kw",
+    "kuwait": "kw",
+    "kg": "kg",
+    "kyrgyzstan": "kg",
+    "la": "la",
+    "laos": "la",
+    "lb": "lb",
+    "lebanon": "lb",
+    "my": "my",
+    "malaysia": "my",
+    "mv": "mv",
+    "maldives": "mv",
+    "mn": "mn",
+    "mongolia": "mn",
+    "mm": "mm",
+    "myanmar": "mm",
+    "burma": "mm",
+    "np": "np",
+    "nepal": "np",
+    "kp": "kp",
+    "north korea": "kp",
+    "dprk": "kp",
+    "om": "om",
+    "oman": "om",
+    "pk": "pk",
+    "pakistan": "pk",
+    "ph": "ph",
+    "philippines": "ph",
+    "qa": "qa",
+    "qatar": "qa",
+    "sa": "sa",
+    "saudi arabia": "sa",
+    "sg": "sg",
+    "singapore": "sg",
+    "kr": "kr",
+    "south korea": "kr",
+    "korea": "kr",
+    "lk": "lk",
+    "sri lanka": "lk",
+    "sy": "sy",
+    "syria": "sy",
+    "tw": "tw",
+    "taiwan": "tw",
+    "tj": "tj",
+    "tajikistan": "tj",
+    "th": "th",
+    "thailand": "th",
+    "tr": "tr",
+    "turkey": "tr",
+    "tm": "tm",
+    "turkmenistan": "tm",
+    "ae": "ae",
+    "uae": "ae",
+    "united arab emirates": "ae",
+    "uz": "uz",
+    "uzbekistan": "uz",
+    "vn": "vn",
+    "vietnam": "vn",
+    "ye": "ye",
+    "yemen": "ye",
+    # Africa
+    "dz": "dz",
+    "algeria": "dz",
+    "ao": "ao",
+    "angola": "ao",
+    "bj": "bj",
+    "benin": "bj",
+    "bw": "bw",
+    "botswana": "bw",
+    "bf": "bf",
+    "burkina faso": "bf",
+    "bi": "bi",
+    "burundi": "bi",
+    "cm": "cm",
+    "cameroon": "cm",
+    "cv": "cv",
+    "cape verde": "cv",
+    "cf": "cf",
+    "central african republic": "cf",
+    "td": "td",
+    "chad": "td",
+    "km": "km",
+    "comoros": "km",
+    "cg": "cg",
+    "congo": "cg",
+    "cd": "cd",
+    "drc": "cd",
+    "democratic republic of the congo": "cd",
+    "ci": "ci",
+    "cote divoire": "ci",
+    "ivory coast": "ci",
+    "dj": "dj",
+    "djibouti": "dj",
+    "eg": "eg",
+    "egypt": "eg",
+    "gq": "gq",
+    "equatorial guinea": "gq",
+    "er": "er",
+    "eritrea": "er",
+    "et": "et",
+    "ethiopia": "et",
+    "ga": "ga",
+    "gabon": "ga",
+    "gm": "gm",
+    "gambia": "gm",
+    "gh": "gh",
+    "ghana": "gh",
+    "gn": "gn",
+    "guinea": "gn",
+    "gw": "gw",
+    "guinea bissau": "gw",
+    "ke": "ke",
+    "kenya": "ke",
+    "ls": "ls",
+    "lesotho": "ls",
+    "lr": "lr",
+    "liberia": "lr",
+    "ly": "ly",
+    "libya": "ly",
+    "mg": "mg",
+    "madagascar": "mg",
+    "mw": "mw",
+    "malawi": "mw",
+    "ml": "ml",
+    "mali": "ml",
+    "mr": "mr",
+    "mauritania": "mr",
+    "mu": "mu",
+    "mauritius": "mu",
+    "ma": "ma",
+    "morocco": "ma",
+    "mz": "mz",
+    "mozambique": "mz",
+    "na": "na",
+    "namibia": "na",
+    "ne": "ne",
+    "niger": "ne",
+    "ng": "ng",
+    "nigeria": "ng",
+    "rw": "rw",
+    "rwanda": "rw",
+    "st": "st",
+    "sao tome and principe": "st",
+    "sn": "sn",
+    "senegal": "sn",
+    "sc": "sc",
+    "seychelles": "sc",
+    "sl": "sl",
+    "sierra leone": "sl",
+    "so": "so",
+    "somalia": "so",
+    "za": "za",
+    "south africa": "za",
+    "ss": "ss",
+    "south sudan": "ss",
+    "sd": "sd",
+    "sudan": "sd",
+    "sz": "sz",
+    "swaziland": "sz",
+    "eswatini": "sz",
+    "tz": "tz",
+    "tanzania": "tz",
+    "tg": "tg",
+    "togo": "tg",
+    "tn": "tn",
+    "tunisia": "tn",
+    "ug": "ug",
+    "uganda": "ug",
+    "zm": "zm",
+    "zambia": "zm",
+    "zw": "zw",
+    "zimbabwe": "zw",
+    # Oceania
+    "au": "au",
+    "australia": "au",
+    "fj": "fj",
+    "fiji": "fj",
+    "ki": "ki",
+    "kiribati": "ki",
+    "mh": "mh",
+    "marshall islands": "mh",
+    "fm": "fm",
+    "micronesia": "fm",
+    "nr": "nr",
+    "nauru": "nr",
+    "nz": "nz",
+    "new zealand": "nz",
+    "pw": "pw",
+    "palau": "pw",
+    "pg": "pg",
+    "papua new guinea": "pg",
+    "ws": "ws",
+    "samoa": "ws",
+    "sb": "sb",
+    "solomon islands": "sb",
+    "to": "to",
+    "tonga": "to",
+    "tv": "tv",
+    "tuvalu": "tv",
+    "vu": "vu",
+    "vanuatu": "vu",
+    # Territories and Special Cases
+    "hk": "hk",
+    "hong kong": "hk",
+    "mo": "mo",
+    "macau": "mo",
+    "ps": "ps",
+    "palestine": "ps",
+    "re": "re",
+    "reunion": "re",
+    "tf": "tf",
+    "french southern territories": "tf",
+    "um": "um",
+    "united states minor outlying islands": "um",
+    "wf": "wf",
+    "wallis and futuna": "wf",
+    "yt": "yt",
+    "mayotte": "yt",
+}

scraperapi_mcp_server-0.1.0/src/scraperapi_mcp_server/utils/exceptions.py

@@ -0,0 +1,49 @@
+from typing import Dict, Any
+from requests.exceptions import RequestException, HTTPError as RequestsHTTPError
+from mcp.shared.exceptions import McpError
+from mcp.types import ErrorData, INTERNAL_ERROR
+import logging
+
+
+class ApiKeyEnvVarNotSetError(Exception):
+    """Raised when the API key environment variable is not set."""
+
+    pass
+
+
+def handle_scraper_error(
+    e: Exception, url: str, payload: Dict[str, Any] = None
+) -> McpError:
+    """
+    Handle all scraper errors with simple error messages.
+
+    Args:
+        e: The exception that occurred
+        url: The URL that was being scraped
+        payload: Optional parameters used in the API call
+
+    Returns:
+        An McpError with error information
+    """
+    if isinstance(e, RequestsHTTPError) and hasattr(e, "response"):
+        status_code = e.response.status_code
+        error_message = f"HTTP error {status_code} when scraping '{url}': {str(e)}"
+    elif isinstance(e, RequestException):
+        error_message = f"Connection error when scraping '{url}': {str(e)}"
+    else:
+        error_message = f"Error when scraping '{url}': {str(e)}"
+
+    # Include the parameters we tried in the error message if available
+    if payload and isinstance(e, RequestsHTTPError):
+        param_summary = " ".join(
+            [f"{k}={v}" for k, v in payload.items() if k != "api_key"]
+        )
+        error_message += f" Parameters used: {param_summary}"
+
+    logging.error(f"handle_scraper_error: {error_message}", exc_info=True)
+    return McpError(
+        ErrorData(
+            code=INTERNAL_ERROR,
+            message=error_message,
+        )
+    )