thordata-mcp-server 0.4.4__py3-none-any.whl → 0.5.0__py3-none-any.whl

{thordata_mcp_server-0.4.4.dist-info → thordata_mcp_server-0.5.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: thordata-mcp-server
-Version: 0.4.4
+Version: 0.5.0
 Summary: Official MCP Server for Thordata.
 Author-email: Thordata Developer Team <support@thordata.com>
 License-Expression: MIT
@@ -8,7 +8,7 @@ Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 Requires-Dist: mcp[cli]>=1.0.0
 Requires-Dist: sse-starlette>=1.6.1
-Requires-Dist: thordata-sdk>=1.6.0
+Requires-Dist: thordata-sdk>=1.7.0
 Requires-Dist: pydantic-settings
 Requires-Dist: markdownify
 Requires-Dist: html2text
@@ -23,14 +23,14 @@ Requires-Dist: uvicorn
 
 **Give your AI Agents real-time web scraping superpowers.**
 
-This MCP Server version has been **streamlined to focus on scraping**, concentrating on four core products:
+This MCP Server version has been **streamlined to focus on scraping**, concentrating on a compact, LLM‑friendly tool surface:
 
-- **SERP API** (Search result scraping)
+- **Search Engine** (LLM-friendly web search wrapper)
+- **SERP API** (Search result scraping, internal plumbing)
 - **Web Unlocker / Universal Scraper** (Universal page unlocking & scraping)
-- **Web Scraper API** (Structured task flow)
 - **Scraping Browser** (Browser-level scraping)
 
-Earlier versions exposed `proxy.*` / `account.*` / `proxy_users.*` proxy and account management tools. This version has removed these control plane interfaces, keeping only scraping-related capabilities for a clean tool surface in Cursor / MCP clients.
+Earlier versions exposed `proxy.*` / `account.*` / `proxy_users.*` proxy and account management tools, and a large `web_scraper` task surface. This version removes those control plane interfaces from MCP, keeping only scraping-related capabilities that are easy for LLMs to use.
 
 ## 🚀 Features
 
@@ -76,38 +76,15 @@ THORDATA_BROWSER_PASSWORD=your_password
 
 ### Tool Exposure Modes
 
-Current implementation provides **streamlined scraping tool surface only**, no longer exposing proxy and account management tools:
+Current implementation provides a **compact scraping tool surface**, optimized for Cursor / LLM tool callers:
 
-- **SERP SCRAPER**: `serp` (actions: `search`, `batch_search`)
-- **WEB UNLOCKER**: `unlocker` (actions: `fetch`, `batch_fetch`)
-- **WEB SCRAPER (100+ structured tasks + task management)**: `web_scraper` (actions: `catalog`, `groups`, `run`, `batch_run`, `status`, `status_batch`, `wait`, `result`, `result_batch`, `list_tasks`, `cancel`)
-- **BROWSER SCRAPER**: `browser` (actions: `navigate`, `snapshot`)
-- **Smart (auto tool + fallback)**: `smart_scrape`
+- **`search_engine`** (recommended for LLMs): high-level web search wrapper, returns a light `results[]` array with `title/link/description`. Internally delegates to the SERP backend.
+- **`search_engine_batch`**: batch variant of `search_engine` with per-item `ok/error` results.
+- **`unlocker`**: actions `fetch`, `batch_fetch` – universal page unlock & content extraction (HTML/Markdown), with per-item error reporting for batch.
+- **`browser`**: action `snapshot` – navigate (optional `url`) and capture an ARIA-focused snapshot for interactive elements.
+- **`smart_scrape`**: auto-picks the best scraper (SERP, Web Scraper, Unlocker) for a given URL and returns a unified, LLM-friendly response.
 
-> Note: This version focuses on scraping functionality and no longer includes `proxy.*` / `account.*` control plane tools.
-
-### Web Scraper discovery (100+ tools, no extra env required)
-
-Use `web_scraper` with `action="catalog"` / `action="groups"` to discover tools.
-This keeps Cursor/LLMs usable while still supporting **100+ tools** under a single entrypoint.
-
-```env
-# Default: curated + limit 60
-THORDATA_TASKS_LIST_MODE=curated
-THORDATA_TASKS_LIST_DEFAULT_LIMIT=60
-
-# Which groups are included when mode=curated
-THORDATA_TASKS_GROUPS=ecommerce,social,video,search,travel,code,professional
-
-# Optional safety/UX: restrict which tools can actually run
-# (comma-separated prefixes or exact tool keys)
-# Example:
-# THORDATA_TASKS_ALLOWLIST=thordata.tools.video.,thordata.tools.ecommerce.Amazon.ProductByAsin
-THORDATA_TASKS_ALLOWLIST=
-```
-
-If you want Cursor to **never** see the full 300+ tool list, keep `THORDATA_TASKS_LIST_MODE=curated`
-and optionally set `THORDATA_TASKS_ALLOWLIST` to the small subset you actually want to support.
+Internally, the server still uses structured SERP and Web Scraper capabilities, but they are not exposed as large tool surfaces by default to avoid overwhelming LLMs.
 
 ### Deployment (Optional)
 
@@ -162,19 +139,17 @@ Add this to your `claude_desktop_config.json`:
 Notes:
 - `THORDATA_BROWSER_USERNAME` / `THORDATA_BROWSER_PASSWORD` are required for `browser.*` tools (Scraping Browser).
 
-## 🛠️ Available Tools
-
-### Available Tools (All directly related to scraping)
+## 🛠️ Available Tools (Compact Surface)
 
-Current MCP Server only exposes the following **5 scraping-related tools**:
+By default, the MCP server exposes a **small, LLM-friendly tool set**:
 
-- **`serp`**: action `search`, `batch_search`
-- **`unlocker`**: action `fetch`, `batch_fetch`
-- **`web_scraper`**: action `catalog`, `groups`, `run`, `batch_run`, `status`, `status_batch`, `wait`, `result`, `result_batch`, `list_tasks`, `cancel`
-- **`browser`**: action `navigate`, `snapshot`
-- **`smart_scrape`**: auto-pick structured tool; fallback to unlocker
+- **`search_engine`**: single-query web search (`params.q`, optional `params.num`, `params.engine`).
+- **`search_engine_batch`**: batch web search with per-item `ok/error` in `results[]`.
+- **`unlocker`**: universal scraping via `fetch` / `batch_fetch`.
+- **`browser`**: `snapshot` with optional `url`, `max_items`, and `max_chars`.
+- **`smart_scrape`**: smart router for `url` with optional preview limit parameters.
 
-> Proxy network related APIs can still be used via other Thordata SDKs / HTTP APIs, but are not exposed through MCP to avoid introducing complex management operations in LLMs.
+Advanced / internal tools (e.g. low-level `serp.*`, full `web_scraper.*` surfaces, proxy/account control plane) remain available via HTTP APIs and SDKs, but are not exposed directly as MCP tools to keep the surface manageable for agents and LLMs.
 
 ## 🏗️ Architecture
 
@@ -189,14 +164,14 @@ thordata_mcp/
 ├── utils.py             # Common utilities (error handling, responses)
 ├── browser_session.py   # Browser session management (Playwright)
 ├── aria_snapshot.py     # ARIA snapshot filtering
-└── tools/
-    ├── product_compact.py  # Streamlined 5-tool entry point (serp/unlocker/web_scraper/browser/smart_scrape)
-    ├── product.py          # Full product implementation for internal use (reused by compact version)
-    ├── data/               # Data plane tools (only scraping-related namespaces retained)
-    │   ├── serp.py         # serp.*
-    │   ├── universal.py    # universal.*
-    │   ├── browser.py      # browser.*
-    │   └── tasks.py        # tasks.*
+    └── tools/
+        ├── product_compact.py  # Streamlined MCP entrypoint (search_engine / unlocker / browser / smart_scrape, plus batch variants)
+        ├── product.py          # Full product implementation for internal use (reused by compact version)
+        ├── data/               # Data plane tools (only scraping-related namespaces retained)
+        │   ├── serp.py         # SERP backend integration
+        │   ├── universal.py    # Universal / Unlocker backend integration
+        │   ├── browser.py      # Browser / Playwright helpers
+        │   └── tasks.py        # Structured scraping tasks (used by smart_scrape and internal flows)
 ```
 
 ## 🎯 Design Principles

thordata_mcp_server-0.5.0.dist-info/RECORD

@@ -0,0 +1,26 @@
+thordata_mcp/__init__.py,sha256=EMlQmThUq4YcxjIewML7g2ZyT3W3TXaYGoVciewYbwg,61
+thordata_mcp/aria_snapshot.py,sha256=SW8d_MxudDUlrl0OPjfG1brg67WyC-I4fMnwPOpk9oU,3293
+thordata_mcp/browser_session.py,sha256=p3rM00kHcL8DVSBRoqZfwl8X8Kfi4sx6AsCX9ACi-z4,17654
+thordata_mcp/config.py,sha256=M5gTGrYdvKOlUCI6ZHoSMEWPWjAz7Onu5XXAnat9bts,2957
+thordata_mcp/context.py,sha256=rptQ55f7SP47tbHb9mrebkdrSvPm4gZaabSr15WpnLU,1304
+thordata_mcp/debug_http.py,sha256=6bioJS7M8luF2wr39cQeD9jqgYBPeTAPWxbing6JhD0,7045
+thordata_mcp/main.py,sha256=qcqTdF6YM0lwmp3Zj2R7vlA8LmczflykANCvcOKJZcQ,3743
+thordata_mcp/monitoring.py,sha256=JIFJa0mJb1f9tyzSyERVtOsm_YbKv3i9-E_8aBhI7Ho,7458
+thordata_mcp/registry.py,sha256=4Vp49bNB8BeBAFfyPJO9SPRU2A4Ez6EM5OClAEY1NYE,1339
+thordata_mcp/utils.py,sha256=c_U3jle0oLFOyBYz68ECZxUpRX2-NdyEk12LK1nLnH8,15997
+thordata_mcp/tools/__init__.py,sha256=6XOtYo7kqcHuBQLE0xZLGilP5NxNmZArIjy_tKIMep0,618
+thordata_mcp/tools/debug.py,sha256=Lunt2cm7HvAiQJK1T1damy2mVSVm1RUqr4cPTl8IXko,5468
+thordata_mcp/tools/params_utils.py,sha256=u3QCD9cS6p7APZxMFJM5gkfvQxTeWB_4d0mVWeiozwk,3483
+thordata_mcp/tools/product.py,sha256=sY39fAOSBLcErgyroN3x6W0dftVnYvHH6178DRz9c_4,75573
+thordata_mcp/tools/product_compact.py,sha256=d_WwW1dBDPOHM7E-WSq3eyVxfmZRElrMPM1EowiOXiU,105964
+thordata_mcp/tools/utils.py,sha256=BaD3150Dt_vhLDZliep-_LZaAiN7IAC3HPV2Db8WBZc,4538
+thordata_mcp/tools/data/__init__.py,sha256=KMD19WJlkPe3Y3e-U1v9IjAO3E_qEu2dtzS5yj_h1XU,398
+thordata_mcp/tools/data/browser.py,sha256=KVhNtYUpIV5FwE7uCpSbCM1TWMMezNk679UPaXbDp1Q,20307
+thordata_mcp/tools/data/serp.py,sha256=BdxVMiQ5WwHqK1sPIfk1Ehx9MCG-QFZKLu8ZlZE2IlA,3439
+thordata_mcp/tools/data/tasks.py,sha256=_spv5Uz3ZGdP4sSCrJT3nJg5UQILGxEijjK4Z-zJZVA,16274
+thordata_mcp/tools/data/universal.py,sha256=JUQUOwN2x_p7coRt71bCykQxOzZCnyiN4NvSq4dfZaI,9482
+thordata_mcp_server-0.5.0.dist-info/METADATA,sha256=keM0rodUXSyI35OaNCxF0IrHHPf6qhKM1NXuf6YR8zE,7666
+thordata_mcp_server-0.5.0.dist-info/WHEEL,sha256=wUyA8OaulRlbfwMtmQsvNngGrxQHAvkKcvRmdizlJi0,92
+thordata_mcp_server-0.5.0.dist-info/entry_points.txt,sha256=DI4aWMlUKBmTJTbNanK9m1LWoKeFvuLYS4sPd0n5GIQ,56
+thordata_mcp_server-0.5.0.dist-info/top_level.txt,sha256=9ODiyY_ikjIGCkF7_zk3HAd07KUkH63Uu7Ax-cz7DA8,13
+thordata_mcp_server-0.5.0.dist-info/RECORD,,

thordata_mcp_server-0.4.4.dist-info/RECORD

@@ -1,24 +0,0 @@
-thordata_mcp/__init__.py,sha256=yPIlzOnjL77vd9e1iuXpJs7wUiGYSAnSZtrrdrdQlOg,61
-thordata_mcp/aria_snapshot.py,sha256=SW8d_MxudDUlrl0OPjfG1brg67WyC-I4fMnwPOpk9oU,3293
-thordata_mcp/browser_session.py,sha256=I804ixmgl2d4B4MC3bqVniBUlYZ5693V618mZ7tYGkU,11425
-thordata_mcp/config.py,sha256=HJzFZxRnhhu2wbl4xMgEJIFy4cifSMQ-TYyVGcyVSug,2655
-thordata_mcp/context.py,sha256=WmfoEbk_y3lxkssWIsDRpr00fwvuomzclskIUnMlQs0,1305
-thordata_mcp/debug_http.py,sha256=6bioJS7M8luF2wr39cQeD9jqgYBPeTAPWxbing6JhD0,7045
-thordata_mcp/main.py,sha256=qcqTdF6YM0lwmp3Zj2R7vlA8LmczflykANCvcOKJZcQ,3743
-thordata_mcp/monitoring.py,sha256=JIFJa0mJb1f9tyzSyERVtOsm_YbKv3i9-E_8aBhI7Ho,7458
-thordata_mcp/registry.py,sha256=4Vp49bNB8BeBAFfyPJO9SPRU2A4Ez6EM5OClAEY1NYE,1339
-thordata_mcp/utils.py,sha256=bxrQiJPJu1TvlOSQXUR-LFwFN5r-6qIrKLNEkSAgyIM,13251
-thordata_mcp/tools/__init__.py,sha256=6XOtYo7kqcHuBQLE0xZLGilP5NxNmZArIjy_tKIMep0,618
-thordata_mcp/tools/product.py,sha256=38Se8OAKLmuRm6bXyLS28PBOXa_LVJ-lYPg6u-cVUhM,72230
-thordata_mcp/tools/product_compact.py,sha256=mVOPP3Bde2TzD3m6PV3uHiDMdaqnYb5nhF2fSwIdURQ,48705
-thordata_mcp/tools/utils.py,sha256=76YoU2hk1G9GOOxQhvNwYK5mZsQKJT8DnKzhzu-b4r8,4375
-thordata_mcp/tools/data/__init__.py,sha256=KMD19WJlkPe3Y3e-U1v9IjAO3E_qEu2dtzS5yj_h1XU,398
-thordata_mcp/tools/data/browser.py,sha256=1NXFlqeMCCHWfmklCYAohcs9A_ceBpqO9f7ZNRJZR6c,16122
-thordata_mcp/tools/data/serp.py,sha256=BdxVMiQ5WwHqK1sPIfk1Ehx9MCG-QFZKLu8ZlZE2IlA,3439
-thordata_mcp/tools/data/tasks.py,sha256=_spv5Uz3ZGdP4sSCrJT3nJg5UQILGxEijjK4Z-zJZVA,16274
-thordata_mcp/tools/data/universal.py,sha256=JUQUOwN2x_p7coRt71bCykQxOzZCnyiN4NvSq4dfZaI,9482
-thordata_mcp_server-0.4.4.dist-info/METADATA,sha256=9_X3K4gIA2U641GWkvbbRpqBbEb51opEWD7pHZ2uJxA,8000
-thordata_mcp_server-0.4.4.dist-info/WHEEL,sha256=wUyA8OaulRlbfwMtmQsvNngGrxQHAvkKcvRmdizlJi0,92
-thordata_mcp_server-0.4.4.dist-info/entry_points.txt,sha256=DI4aWMlUKBmTJTbNanK9m1LWoKeFvuLYS4sPd0n5GIQ,56
-thordata_mcp_server-0.4.4.dist-info/top_level.txt,sha256=9ODiyY_ikjIGCkF7_zk3HAd07KUkH63Uu7Ax-cz7DA8,13
-thordata_mcp_server-0.4.4.dist-info/RECORD,,