thordata-sdk 0.3.1__tar.gz → 1.0.0__tar.gz

thordata_sdk-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Thordata · AI Proxy & Web Data
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

thordata_sdk-1.0.0/PKG-INFO

@@ -0,0 +1,208 @@
+Metadata-Version: 2.4
+Name: thordata-sdk
+Version: 1.0.0
+Summary: The Official Python SDK for Thordata - AI Data Infrastructure & Proxy Network.
+Author-email: Thordata Developer Team <support@thordata.com>
+License: MIT
+Project-URL: Homepage, https://www.thordata.com
+Project-URL: Documentation, https://github.com/Thordata/thordata-python-sdk#readme
+Project-URL: Source, https://github.com/Thordata/thordata-python-sdk
+Project-URL: Tracker, https://github.com/Thordata/thordata-python-sdk/issues
+Project-URL: Changelog, https://github.com/Thordata/thordata-python-sdk/blob/main/CHANGELOG.md
+Keywords: web scraping,proxy,residential proxy,datacenter proxy,ai,llm,data-mining,serp,thordata,web scraper,anti-bot bypass
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Internet :: Proxy Servers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25.0
+Requires-Dist: aiohttp>=3.9.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: pytest-httpserver>=1.0.0; extra == "dev"
+Requires-Dist: python-dotenv>=1.0.0; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: types-requests>=2.28.0; extra == "dev"
+Requires-Dist: aioresponses>=0.7.6; extra == "dev"
+Dynamic: license-file
+
+# Thordata Python SDK
+
+<div align="center">
+
+**Official Python client for Thordata's Proxy Network, SERP API, Web Unlocker, and Web Scraper API.**
+
+*Async-ready, type-safe, built for AI agents and large-scale data collection.*
+
+[![PyPI](https://img.shields.io/pypi/v/thordata-sdk?color=blue)](https://pypi.org/project/thordata-sdk/)
+[![Python](https://img.shields.io/badge/python-3.9+-blue)](https://python.org)
+[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+[Documentation](https://doc.thordata.com) • [Dashboard](https://www.thordata.com) • [Examples](examples/)
+
+</div>
+
+---
+
+## ✨ Features
+
+- 🌐 **Proxy Network**: Residential, Mobile, Datacenter, ISP proxies with geo-targeting
+- 🔍 **SERP API**: Google, Bing, Yandex, DuckDuckGo search results
+- 🔓 **Web Unlocker**: Bypass Cloudflare, CAPTCHAs, anti-bot systems
+- 🕷️ **Web Scraper API**: Async task-based scraping (Text & Video/Audio)
+- 📊 **Account Management**: Usage stats, sub-users, IP whitelist
+- ⚡ **Async Support**: Full async/await support with aiohttp
+- 🔄 **Auto Retry**: Configurable retry with exponential backoff
+
+---
+
+## 📦 Installation
+
+```bash
+pip install thordata-sdk
+```
+
+---
+
+## 🔐 Configuration
+
+Set environment variables:
+
+```bash
+# Required for Scraper APIs (SERP, Universal, Tasks)
+export THORDATA_SCRAPER_TOKEN=your_token
+
+# Public/Location APIs (Dashboard -> My account -> API)
+export THORDATA_PUBLIC_TOKEN=your_public_token
+export THORDATA_PUBLIC_KEY=your_public_key
+
+```
+
+---
+
+## 🚀 Quick Start
+
+```python
+from thordata import ThordataClient, Engine
+
+# Initialize (reads from env vars)
+client = ThordataClient(
+    scraper_token="your_token", 
+    public_token="pub_token", 
+    public_key="pub_key"
+)
+
+# SERP Search
+results = client.serp_search("python tutorial", engine=Engine.GOOGLE)
+print(f"Found {len(results.get('organic', []))} results")
+
+# Universal Scrape
+html = client.universal_scrape("https://httpbin.org/html")
+print(html[:100])
+```
+
+---
+
+## 📖 Feature Guide
+
+### SERP API
+
+```python
+from thordata import SerpRequest
+
+# Advanced search
+results = client.serp_search_advanced(SerpRequest(
+    query="pizza",
+    engine="google_local",
+    country="us",
+    location="New York",
+    num=10
+))
+```
+
+### Web Scraper API (Async Tasks)
+
+**Create Task:**
+```python
+task_id = client.create_scraper_task(
+    file_name="my_task",
+    spider_id="universal",
+    spider_name="universal",
+    parameters={"url": "https://example.com"}
+)
+```
+
+**Video Download (New):**
+```python
+from thordata import CommonSettings
+
+task_id = client.create_video_task(
+    file_name="{{VideoID}}",
+    spider_id="youtube_video_by-url",
+    spider_name="youtube.com",
+    parameters={"url": "https://youtube.com/watch?v=..."},
+    common_settings=CommonSettings(resolution="1080p")
+)
+```
+
+**Wait & Download:**
+```python
+status = client.wait_for_task(task_id)
+if status == "ready":
+    url = client.get_task_result(task_id)
+    print(url)
+```
+
+### Account Management
+
+```python
+# Usage Statistics
+stats = client.get_usage_statistics("2024-01-01", "2024-01-31")
+print(f"Balance: {stats.balance_gb():.2f} GB")
+
+# Proxy Users
+users = client.list_proxy_users()
+print(f"Sub-users: {users.user_count}")
+
+# Whitelist IP
+client.add_whitelist_ip("1.2.3.4")
+```
+
+### Proxy Network
+
+```python
+from thordata import ProxyConfig
+
+# Generate Proxy URL
+proxy_url = client.build_proxy_url(
+    username="proxy_user",
+    password="proxy_pass",
+    country="us",
+    city="ny"
+)
+
+# Use with requests
+import requests
+requests.get("https://httpbin.org/ip", proxies={"http": proxy_url, "https": proxy_url})
+```
+
+---
+
+## 📄 License
+
+MIT License

thordata_sdk-1.0.0/README.md

@@ -0,0 +1,165 @@
+# Thordata Python SDK
+
+<div align="center">
+
+**Official Python client for Thordata's Proxy Network, SERP API, Web Unlocker, and Web Scraper API.**
+
+*Async-ready, type-safe, built for AI agents and large-scale data collection.*
+
+[![PyPI](https://img.shields.io/pypi/v/thordata-sdk?color=blue)](https://pypi.org/project/thordata-sdk/)
+[![Python](https://img.shields.io/badge/python-3.9+-blue)](https://python.org)
+[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+[Documentation](https://doc.thordata.com) • [Dashboard](https://www.thordata.com) • [Examples](examples/)
+
+</div>
+
+---
+
+## ✨ Features
+
+- 🌐 **Proxy Network**: Residential, Mobile, Datacenter, ISP proxies with geo-targeting
+- 🔍 **SERP API**: Google, Bing, Yandex, DuckDuckGo search results
+- 🔓 **Web Unlocker**: Bypass Cloudflare, CAPTCHAs, anti-bot systems
+- 🕷️ **Web Scraper API**: Async task-based scraping (Text & Video/Audio)
+- 📊 **Account Management**: Usage stats, sub-users, IP whitelist
+- ⚡ **Async Support**: Full async/await support with aiohttp
+- 🔄 **Auto Retry**: Configurable retry with exponential backoff
+
+---
+
+## 📦 Installation
+
+```bash
+pip install thordata-sdk
+```
+
+---
+
+## 🔐 Configuration
+
+Set environment variables:
+
+```bash
+# Required for Scraper APIs (SERP, Universal, Tasks)
+export THORDATA_SCRAPER_TOKEN=your_token
+
+# Public/Location APIs (Dashboard -> My account -> API)
+export THORDATA_PUBLIC_TOKEN=your_public_token
+export THORDATA_PUBLIC_KEY=your_public_key
+
+```
+
+---
+
+## 🚀 Quick Start
+
+```python
+from thordata import ThordataClient, Engine
+
+# Initialize (reads from env vars)
+client = ThordataClient(
+    scraper_token="your_token", 
+    public_token="pub_token", 
+    public_key="pub_key"
+)
+
+# SERP Search
+results = client.serp_search("python tutorial", engine=Engine.GOOGLE)
+print(f"Found {len(results.get('organic', []))} results")
+
+# Universal Scrape
+html = client.universal_scrape("https://httpbin.org/html")
+print(html[:100])
+```
+
+---
+
+## 📖 Feature Guide
+
+### SERP API
+
+```python
+from thordata import SerpRequest
+
+# Advanced search
+results = client.serp_search_advanced(SerpRequest(
+    query="pizza",
+    engine="google_local",
+    country="us",
+    location="New York",
+    num=10
+))
+```
+
+### Web Scraper API (Async Tasks)
+
+**Create Task:**
+```python
+task_id = client.create_scraper_task(
+    file_name="my_task",
+    spider_id="universal",
+    spider_name="universal",
+    parameters={"url": "https://example.com"}
+)
+```
+
+**Video Download (New):**
+```python
+from thordata import CommonSettings
+
+task_id = client.create_video_task(
+    file_name="{{VideoID}}",
+    spider_id="youtube_video_by-url",
+    spider_name="youtube.com",
+    parameters={"url": "https://youtube.com/watch?v=..."},
+    common_settings=CommonSettings(resolution="1080p")
+)
+```
+
+**Wait & Download:**
+```python
+status = client.wait_for_task(task_id)
+if status == "ready":
+    url = client.get_task_result(task_id)
+    print(url)
+```
+
+### Account Management
+
+```python
+# Usage Statistics
+stats = client.get_usage_statistics("2024-01-01", "2024-01-31")
+print(f"Balance: {stats.balance_gb():.2f} GB")
+
+# Proxy Users
+users = client.list_proxy_users()
+print(f"Sub-users: {users.user_count}")
+
+# Whitelist IP
+client.add_whitelist_ip("1.2.3.4")
+```
+
+### Proxy Network
+
+```python
+from thordata import ProxyConfig
+
+# Generate Proxy URL
+proxy_url = client.build_proxy_url(
+    username="proxy_user",
+    password="proxy_pass",
+    country="us",
+    city="ny"
+)
+
+# Use with requests
+import requests
+requests.get("https://httpbin.org/ip", proxies={"http": proxy_url, "https": proxy_url})
+```
+
+---
+
+## 📄 License
+
+MIT License

thordata_sdk-1.0.0/pyproject.toml

@@ -0,0 +1,136 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "thordata-sdk"
+version = "1.0.0"
+description = "The Official Python SDK for Thordata - AI Data Infrastructure & Proxy Network."
+readme = "README.md"
+requires-python = ">=3.9"
+license = {text = "MIT"}
+authors = [
+  {name = "Thordata Developer Team", email = "support@thordata.com"}
+]
+keywords = [
+    "web scraping", 
+    "proxy", 
+    "residential proxy",
+    "datacenter proxy",
+    "ai", 
+    "llm", 
+    "data-mining", 
+    "serp", 
+    "thordata",
+    "web scraper",
+    "anti-bot bypass"
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+    "Topic :: Internet :: WWW/HTTP",
+    "Topic :: Internet :: Proxy Servers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Typing :: Typed",
+]
+
+dependencies = [
+    "requests>=2.25.0",
+    "aiohttp>=3.9.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0.0",
+    "pytest-asyncio>=0.21.0",
+    "pytest-cov>=4.0.0",
+    "pytest-httpserver>=1.0.0",
+    "python-dotenv>=1.0.0",
+    "black>=23.0.0",
+    "ruff>=0.1.0",
+    "mypy>=1.0.0",
+    "types-requests>=2.28.0",
+    "aioresponses>=0.7.6",
+]
+
+[project.urls]
+"Homepage" = "https://www.thordata.com"
+"Documentation" = "https://github.com/Thordata/thordata-python-sdk#readme"
+"Source" = "https://github.com/Thordata/thordata-python-sdk"
+"Tracker" = "https://github.com/Thordata/thordata-python-sdk/issues"
+"Changelog" = "https://github.com/Thordata/thordata-python-sdk/blob/main/CHANGELOG.md"
+
+# Setuptools setup
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+thordata = ["py.typed"]
+
+[tool.black]
+line-length = 88
+target-version = ['py39', 'py310', 'py311', 'py312']
+include = '\.pyi?$'
+
+[tool.ruff]
+line-length = 88
+target-version = "py39"
+
+[tool.ruff.lint]
+select = [
+    "E",   # pycodestyle errors
+    "W",   # pycodestyle warnings
+    "F",   # pyflakes
+    "I",   # isort (import sorting)
+    "B",   # flake8-bugbear
+]
+ignore = [
+    "E501",  # line too long (handled by black)
+    "E731",  # do not assign a lambda expression
+    "F401",  # imported but unused (we have some intentional re-exports)
+]
+
+[tool.ruff.lint.isort]
+known-first-party = ["thordata"]
+
+# Mypy
+[tool.mypy]
+python_version = "3.9"
+warn_return_any = false
+warn_unused_ignores = false
+disallow_untyped_defs = false
+disallow_incomplete_defs = false
+check_untyped_defs = false
+strict_optional = false
+show_error_codes = true
+ignore_missing_imports = true
+
+[[tool.mypy.overrides]]
+module = ["aiohttp.*", "requests.*"]
+ignore_missing_imports = true
+
+# Pytest setup
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+asyncio_mode = "auto"
+addopts = "-v --cov=thordata --cov-report=term-missing"
+
+# Coverage setup
+[tool.coverage.run]
+source = ["src/thordata"]
+branch = true
+
+[tool.coverage.report]
+exclude_lines = [
+    "pragma: no cover",
+    "def __repr__",
+    "raise NotImplementedError",
+    "if TYPE_CHECKING:",
+]

{thordata_sdk-0.3.1 → thordata_sdk-1.0.0}/setup.cfg

@@ -1,4 +1,4 @@
-[egg_info]
-tag_build = 
-tag_date = 0
-
+[egg_info]
+tag_build = 
+tag_date = 0
+

thordata_sdk-1.0.0/src/thordata/__init__.py

@@ -0,0 +1,151 @@
+"""
+Thordata Python SDK
+
+Official Python client for Thordata's Proxy Network, SERP API,
+Universal Scraping API (Web Unlocker), and Web Scraper API.
+
+Basic Usage:
+    >>> from thordata import ThordataClient
+    >>>
+    >>> client = ThordataClient(
+    ...     scraper_token="your_token",
+    ...     public_token="your_public_token",
+    ...     public_key="your_public_key"
+    ... )
+    >>>
+    >>> # Proxy request
+    >>> response = client.get("https://httpbin.org/ip")
+    >>>
+    >>> # SERP search
+    >>> results = client.serp_search("python tutorial", engine="google")
+    >>>
+    >>> # Universal scrape
+    >>> html = client.universal_scrape("https://example.com", js_render=True)
+
+Async Usage:
+    >>> from thordata import AsyncThordataClient
+    >>> import asyncio
+    >>>
+    >>> async def main():
+    ...     async with AsyncThordataClient(
+    ...         scraper_token="your_token"
+    ...     ) as client:
+    ...         response = await client.get("https://httpbin.org/ip")
+    >>>
+    >>> asyncio.run(main())
+"""
+
+__version__ = "1.0.0"
+__author__ = "Thordata Developer Team"
+__email__ = "support@thordata.com"
+
+# Main clients
+from .async_client import AsyncThordataClient
+from .client import ThordataClient
+
+# Enums
+from .enums import (
+    BingSearchType,
+    Continent,
+    Country,
+    DataFormat,
+    Device,
+    Engine,
+    GoogleSearchType,
+    GoogleTbm,
+    OutputFormat,
+    ProxyHost,
+    ProxyPort,
+    ProxyType,
+    SessionType,
+    TaskStatus,
+    TimeRange,
+)
+
+# Exceptions
+from .exceptions import (
+    ThordataAPIError,
+    ThordataAuthError,
+    ThordataConfigError,
+    ThordataError,
+    ThordataNetworkError,
+    ThordataNotCollectedError,
+    ThordataRateLimitError,
+    ThordataServerError,
+    ThordataTimeoutError,
+    ThordataValidationError,
+)
+
+# Models
+from .models import (
+    CommonSettings,
+    ProxyConfig,
+    ProxyProduct,
+    ProxyServer,
+    ProxyUser,
+    ProxyUserList,
+    ScraperTaskConfig,
+    SerpRequest,
+    StaticISPProxy,
+    StickySession,
+    TaskStatusResponse,
+    UniversalScrapeRequest,
+    UsageStatistics,
+    VideoTaskConfig,
+)
+
+# Retry utilities
+from .retry import RetryConfig
+
+# Public API
+__all__ = [
+    # Version
+    "__version__",
+    # Clients
+    "ThordataClient",
+    "AsyncThordataClient",
+    # Enums
+    "Engine",
+    "GoogleSearchType",
+    "BingSearchType",
+    "ProxyType",
+    "SessionType",
+    "Continent",
+    "Country",
+    "OutputFormat",
+    "DataFormat",
+    "TaskStatus",
+    "Device",
+    "TimeRange",
+    "ProxyHost",
+    "ProxyPort",
+    "GoogleTbm",
+    # Models
+    "ProxyConfig",
+    "ProxyProduct",
+    "ProxyServer",
+    "ProxyUser",
+    "ProxyUserList",
+    "UsageStatistics",
+    "StaticISPProxy",
+    "StickySession",
+    "SerpRequest",
+    "UniversalScrapeRequest",
+    "ScraperTaskConfig",
+    "CommonSettings",
+    "VideoTaskConfig",
+    "TaskStatusResponse",
+    # Exceptions
+    "ThordataError",
+    "ThordataConfigError",
+    "ThordataNetworkError",
+    "ThordataTimeoutError",
+    "ThordataAPIError",
+    "ThordataAuthError",
+    "ThordataRateLimitError",
+    "ThordataServerError",
+    "ThordataValidationError",
+    "ThordataNotCollectedError",
+    # Retry
+    "RetryConfig",
+]