mcpscraper-sdk 0.13.0__tar.gz

mcpscraper_sdk-0.13.0/.gitignore

@@ -0,0 +1,6 @@
+.venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+dist/
+*.egg-info/

mcpscraper_sdk-0.13.0/PKG-INFO

@@ -0,0 +1,128 @@
+Metadata-Version: 2.4
+Name: mcpscraper-sdk
+Version: 0.13.0
+Summary: Official Python client for all 165 mcpscraper.dev MCP tools and the REST API
+Project-URL: Repository, https://github.com/VilovietaSEO/mcpscraper-sdk
+License-Expression: MIT
+Requires-Python: >=3.10
+Requires-Dist: pydantic>=2.0
+Requires-Dist: requests>=2.31
+Description-Content-Type: text/markdown
+
+# mcpscraper-sdk (Python)
+
+Official Python client for the [mcpscraper.dev](https://mcpscraper.dev) REST API — SERP search, People-Also-Ask harvesting, single-page and whole-site extraction, YouTube, Facebook/Google Ads Transparency, Instagram, Reddit, video breakdown, Google Maps, and directory/rank-tracking workflows.
+
+This is a thin HTTP client generated against [`../../contracts/scraper.openapi.yaml`](../../contracts/scraper.openapi.yaml), the public contract for the hosted API. It contains no scraping, proxy, or billing logic — only typed request/response plumbing over `requests`.
+
+## Install
+
+```bash
+pip install "mcpscraper-sdk @ git+https://github.com/VilovietaSEO/mcpscraper-sdk.git#subdirectory=packages/scraper-python"
+# or
+uv add "mcpscraper-sdk @ git+https://github.com/VilovietaSEO/mcpscraper-sdk.git#subdirectory=packages/scraper-python"
+```
+
+## Usage
+
+```python
+from mcpscraper import ScraperClient, ScraperApiError
+
+client = ScraperClient(api_key="sk_live_your_key")
+
+serp = client.search_serp(query="roof repair Denver")
+
+try:
+    places = client.maps.search({"query": "roofers", "location": "Denver, CO"})
+    print(places)
+except ScraperApiError as err:
+    if err.is_insufficient_balance():
+        print(f"Need {err.body['required_credits']} credits, have {err.body['balance_credits']}. Top up: {err.body['topup_url']}")
+    else:
+        raise
+```
+
+## Errors
+
+Every non-2xx response raises a `ScraperApiError` with `.status`, `.code`, and the raw response `.body`. Two cases have narrowing helpers:
+
+- `err.is_insufficient_balance()` — `err.body` has `balance_credits`, `required_credits`, `topup_url`.
+- `err.is_concurrency_limit_exceeded()` — `err.body` has `active`, `limit`, `retryable`.
+
+## API surface
+
+`client.tools` is the generated 165-tool MCP surface — 76 MCP Scraper tools plus 89 mirrored memory tools — with one typed snake_case method per tool:
+
+Use `client.tools.call_tool_result(name, args)` when a multimodal tool must preserve its native MCP image, audio, or resource blocks. `call_tool(...)` remains the parsed, backward-compatible path.
+
+```python
+client.tools.search.search_serp(query="roof repair Denver")
+client.tools.memory.search(query="roofing warranty terms")
+client.tools.connections.export_connected_service_data(
+    connection_id="conn_123",
+    dataset="resend_data",
+    last_days=7,
+)
+```
+
+The connected-data export performs bounded Gmail, Calendar, Google Search Console, Zoom, Meta Marketing, or Resend pagination server-side and returns small results inline or a private seven-day JSONL artifact. Use `search_console_performance` for bounded Search Analytics rows across every accessible property, and `meta_ads_insights` for daily account, campaign, ad-set, and ad reporting across connected Meta ad accounts. Resend can aggregate sent/received mail, logs, contacts, broadcasts, and templates with `resend_data`. Resume partial exports with the returned `continuation` object; renew an expired signed URL with `client.tools.connections.renew_connected_data_download(artifact_id="artifact_123")`. Use `list_service_connections` for verified grants and per-tool permission blockers, then `describe_service_connection_tool` for the exact provider-native schema before calling through the generic connection bridges.
+
+Core operations are flat on the client: `search_serp`, `harvest_paa`, `extract_url`, `map_site_urls`, `extract_site`, `audit_site`, `get_extract_site_status`, `list_jobs`, `get_job`, `get_history`, `get_ledger`.
+
+Everything else is namespaced by product area, matching the OpenAPI spec's tags: `client.youtube`, `client.screenshot`, `client.facebook`, `client.google_ads`, `client.instagram`, `client.reddit`, `client.video`, `client.maps`, `client.directory`, `client.serp_intelligence`, `client.workflows`.
+
+Typed Pydantic models for every request/response schema are available from `mcpscraper.models` if you want to construct or validate payloads explicitly — the client methods themselves accept plain keyword arguments/dicts and return parsed JSON.
+
+### Retry-safe SERP Intelligence captures
+
+Pass a stable idempotency key when a capture may be retried. The regular method continues to return the response body; `capture_with_receipt` also returns the key accepted or generated by the server so it can be reused after a timeout or uncertain response:
+
+```python
+result = client.serp_intelligence.capture(
+    {"query": "roofers near me"},
+    idempotency_key="serp-run-2026-07-14-001",
+)
+
+receipt = client.serp_intelligence.capture_with_receipt({"query": "roofers near me"})
+print(receipt["data"]["billing"]["creditsUsed"], receipt["idempotency_key"])
+```
+
+Reusing a key with the same body recovers the original debit and settlement. Reusing it with a different body raises `ScraperApiError` with status `409` and code `idempotency_conflict`.
+
+## Scrape → memory vault
+
+```python
+page = client.extract_url(url="https://example.com/pricing", depositToVault=True, vaultName="competitors")
+print(page["memory"])  # {"deposited": True, "vault": "competitors", "noteId": "...", "chunks": 4}
+```
+
+## Memory tools, using only this API key
+
+`client.memory_tools.call_tool(name, args)` dispatches to any of the 87 memory.mcpscraper.dev tools through `POST /memory/mcp-call`, using only this mcpscraper.dev API key — no separate memory key needed:
+
+```python
+hits = client.memory_tools.call_tool("searchTool", {"query": "competitor pricing pages"})
+vaults = client.memory_tools.call_tool("listVaultsTool")
+```
+
+This generic compatibility bridge remains available, but new integrations should use `client.tools`, whose generated namespaces provide one typed method for every one of the 165 unified MCP tools.
+
+## Regenerating models
+
+`src/mcpscraper/models.py` is generated from the OpenAPI spec and checked in. After editing `../../contracts/scraper.openapi.yaml`, regenerate with:
+
+```bash
+uv run datamodel-codegen \
+  --input ../../contracts/scraper.openapi.yaml \
+  --input-file-type openapi \
+  --output src/mcpscraper/models.py \
+  --output-model-type pydantic_v2.BaseModel \
+  --field-constraints \
+  --use-schema-description \
+  --target-python-version 3.10 \
+  --disable-timestamp
+```
+
+## See also
+
+[Repo README](../../README.md) (multi-language examples with real sample output) · [`mcpscraper-sdk` on npm](../scraper) (Node, full `memory_tools` typed surface) · [`mcpscraper-memory-sdk` Python source package](../memory-python) · [`mcpscraper-cli`](../cli)

mcpscraper_sdk-0.13.0/README.md

@@ -0,0 +1,117 @@
+# mcpscraper-sdk (Python)
+
+Official Python client for the [mcpscraper.dev](https://mcpscraper.dev) REST API — SERP search, People-Also-Ask harvesting, single-page and whole-site extraction, YouTube, Facebook/Google Ads Transparency, Instagram, Reddit, video breakdown, Google Maps, and directory/rank-tracking workflows.
+
+This is a thin HTTP client generated against [`../../contracts/scraper.openapi.yaml`](../../contracts/scraper.openapi.yaml), the public contract for the hosted API. It contains no scraping, proxy, or billing logic — only typed request/response plumbing over `requests`.
+
+## Install
+
+```bash
+pip install "mcpscraper-sdk @ git+https://github.com/VilovietaSEO/mcpscraper-sdk.git#subdirectory=packages/scraper-python"
+# or
+uv add "mcpscraper-sdk @ git+https://github.com/VilovietaSEO/mcpscraper-sdk.git#subdirectory=packages/scraper-python"
+```
+
+## Usage
+
+```python
+from mcpscraper import ScraperClient, ScraperApiError
+
+client = ScraperClient(api_key="sk_live_your_key")
+
+serp = client.search_serp(query="roof repair Denver")
+
+try:
+    places = client.maps.search({"query": "roofers", "location": "Denver, CO"})
+    print(places)
+except ScraperApiError as err:
+    if err.is_insufficient_balance():
+        print(f"Need {err.body['required_credits']} credits, have {err.body['balance_credits']}. Top up: {err.body['topup_url']}")
+    else:
+        raise
+```
+
+## Errors
+
+Every non-2xx response raises a `ScraperApiError` with `.status`, `.code`, and the raw response `.body`. Two cases have narrowing helpers:
+
+- `err.is_insufficient_balance()` — `err.body` has `balance_credits`, `required_credits`, `topup_url`.
+- `err.is_concurrency_limit_exceeded()` — `err.body` has `active`, `limit`, `retryable`.
+
+## API surface
+
+`client.tools` is the generated 165-tool MCP surface — 76 MCP Scraper tools plus 89 mirrored memory tools — with one typed snake_case method per tool:
+
+Use `client.tools.call_tool_result(name, args)` when a multimodal tool must preserve its native MCP image, audio, or resource blocks. `call_tool(...)` remains the parsed, backward-compatible path.
+
+```python
+client.tools.search.search_serp(query="roof repair Denver")
+client.tools.memory.search(query="roofing warranty terms")
+client.tools.connections.export_connected_service_data(
+    connection_id="conn_123",
+    dataset="resend_data",
+    last_days=7,
+)
+```
+
+The connected-data export performs bounded Gmail, Calendar, Google Search Console, Zoom, Meta Marketing, or Resend pagination server-side and returns small results inline or a private seven-day JSONL artifact. Use `search_console_performance` for bounded Search Analytics rows across every accessible property, and `meta_ads_insights` for daily account, campaign, ad-set, and ad reporting across connected Meta ad accounts. Resend can aggregate sent/received mail, logs, contacts, broadcasts, and templates with `resend_data`. Resume partial exports with the returned `continuation` object; renew an expired signed URL with `client.tools.connections.renew_connected_data_download(artifact_id="artifact_123")`. Use `list_service_connections` for verified grants and per-tool permission blockers, then `describe_service_connection_tool` for the exact provider-native schema before calling through the generic connection bridges.
+
+Core operations are flat on the client: `search_serp`, `harvest_paa`, `extract_url`, `map_site_urls`, `extract_site`, `audit_site`, `get_extract_site_status`, `list_jobs`, `get_job`, `get_history`, `get_ledger`.
+
+Everything else is namespaced by product area, matching the OpenAPI spec's tags: `client.youtube`, `client.screenshot`, `client.facebook`, `client.google_ads`, `client.instagram`, `client.reddit`, `client.video`, `client.maps`, `client.directory`, `client.serp_intelligence`, `client.workflows`.
+
+Typed Pydantic models for every request/response schema are available from `mcpscraper.models` if you want to construct or validate payloads explicitly — the client methods themselves accept plain keyword arguments/dicts and return parsed JSON.
+
+### Retry-safe SERP Intelligence captures
+
+Pass a stable idempotency key when a capture may be retried. The regular method continues to return the response body; `capture_with_receipt` also returns the key accepted or generated by the server so it can be reused after a timeout or uncertain response:
+
+```python
+result = client.serp_intelligence.capture(
+    {"query": "roofers near me"},
+    idempotency_key="serp-run-2026-07-14-001",
+)
+
+receipt = client.serp_intelligence.capture_with_receipt({"query": "roofers near me"})
+print(receipt["data"]["billing"]["creditsUsed"], receipt["idempotency_key"])
+```
+
+Reusing a key with the same body recovers the original debit and settlement. Reusing it with a different body raises `ScraperApiError` with status `409` and code `idempotency_conflict`.
+
+## Scrape → memory vault
+
+```python
+page = client.extract_url(url="https://example.com/pricing", depositToVault=True, vaultName="competitors")
+print(page["memory"])  # {"deposited": True, "vault": "competitors", "noteId": "...", "chunks": 4}
+```
+
+## Memory tools, using only this API key
+
+`client.memory_tools.call_tool(name, args)` dispatches to any of the 87 memory.mcpscraper.dev tools through `POST /memory/mcp-call`, using only this mcpscraper.dev API key — no separate memory key needed:
+
+```python
+hits = client.memory_tools.call_tool("searchTool", {"query": "competitor pricing pages"})
+vaults = client.memory_tools.call_tool("listVaultsTool")
+```
+
+This generic compatibility bridge remains available, but new integrations should use `client.tools`, whose generated namespaces provide one typed method for every one of the 165 unified MCP tools.
+
+## Regenerating models
+
+`src/mcpscraper/models.py` is generated from the OpenAPI spec and checked in. After editing `../../contracts/scraper.openapi.yaml`, regenerate with:
+
+```bash
+uv run datamodel-codegen \
+  --input ../../contracts/scraper.openapi.yaml \
+  --input-file-type openapi \
+  --output src/mcpscraper/models.py \
+  --output-model-type pydantic_v2.BaseModel \
+  --field-constraints \
+  --use-schema-description \
+  --target-python-version 3.10 \
+  --disable-timestamp
+```
+
+## See also
+
+[Repo README](../../README.md) (multi-language examples with real sample output) · [`mcpscraper-sdk` on npm](../scraper) (Node, full `memory_tools` typed surface) · [`mcpscraper-memory-sdk` Python source package](../memory-python) · [`mcpscraper-cli`](../cli)

mcpscraper_sdk-0.13.0/pyproject.toml

@@ -0,0 +1,28 @@
+[project]
+name = "mcpscraper-sdk"
+version = "0.13.0"
+description = "Official Python client for all 165 mcpscraper.dev MCP tools and the REST API"
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.10"
+dependencies = [
+    "requests>=2.31",
+    "pydantic>=2.0",
+]
+
+[project.urls]
+Repository = "https://github.com/VilovietaSEO/mcpscraper-sdk"
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0",
+    "responses>=0.25",
+    "datamodel-code-generator>=0.25",
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/mcpscraper"]

mcpscraper_sdk-0.13.0/src/mcpscraper/__init__.py

@@ -0,0 +1,9 @@
+from .client import ScraperClient
+from .errors import ScraperApiError
+from .mcp_client import McpToolsClient, McpToolError
+from ._mcp_generated_client import MCP_TOOL_BINDINGS, MCP_TOOL_COUNT
+
+__all__ = [
+    "ScraperClient", "ScraperApiError", "McpToolsClient", "McpToolError",
+    "MCP_TOOL_BINDINGS", "MCP_TOOL_COUNT",
+]