web-distiller 0.1.0__tar.gz

web_distiller-0.1.0/PKG-INFO

@@ -0,0 +1,89 @@
+Metadata-Version: 2.4
+Name: web-distiller
+Version: 0.1.0
+Summary: Python client for Distiller — agent-ready web extraction with AI cleaning
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/nickandbro/distiller
+Project-URL: Documentation, https://webdistiller.dev/docs
+Project-URL: Source, https://github.com/nickandbro/distiller
+Project-URL: Issues, https://github.com/nickandbro/distiller/issues
+Keywords: distiller,web-distiller,markdown,agents,llm,web-scraping,url-to-markdown
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing :: Markup :: Markdown
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: httpx>=0.27.0
+
+# web-distiller
+
+Python client for the [Distiller](https://webdistiller.dev) API — agent-ready web extraction with AI cleaning.
+
+`pip install web-distiller` installs the client for the hosted Distiller API. It does not install or run the Distiller server.
+
+Use it with:
+- free-tier API keys (Trafilatura cleaning),
+- the Starter paid plan (Stripe-backed AI cleaning),
+- or alternate x402 payment-compatible deployments.
+
+## Install
+
+```bash
+pip install web-distiller
+set DISTILLER_API_BASE=https://webdistiller.dev
+```
+
+## Quick start
+
+```python
+from distiller_client import DistillerClient
+
+client = DistillerClient(api_key="your-key")
+result = client.markdown("https://example.com")
+print(result["cleaned_markdown"])
+```
+
+Or as a one-liner:
+
+```python
+import distiller_client
+
+print(distiller_client.markdown("https://example.com")["cleaned_markdown"])
+```
+
+## CLI
+
+```bash
+web-distiller https://example.com
+web-distiller https://example.com --format text
+web-distiller https://example.com --use-browser
+web-distiller https://example.com --force-refresh
+```
+
+## Configuration
+
+| Env var | Purpose |
+|---------|---------|
+| `DISTILLER_API_BASE` | API base URL (default: production) |
+| `DISTILLER_API_KEY` | Bearer token for authenticated access |
+| `DISTILLER_REFERRER_TOKEN` | Referral attribution token |
+
+All env vars can also be passed as constructor arguments.
+
+Typical flow:
+- use `web-distiller` without an API key for open/demo deployments,
+- register for an API key when you want the free 1,000-call tier,
+- upgrade that account to Starter through Stripe when you want AI cleaning and `/extract`.
+
+## Releases
+
+`web-distiller` is released to PyPI from GitHub Actions via Trusted Publishing.
+
+- Use the `.github/workflows/publish-web-distiller.yml` workflow for manual publishes.
+- Or push a tag in the format `web-distiller-vX.Y.Z` after updating `distiller-client/pyproject.toml`.

web_distiller-0.1.0/README.md

@@ -0,0 +1,66 @@
+# web-distiller
+
+Python client for the [Distiller](https://webdistiller.dev) API — agent-ready web extraction with AI cleaning.
+
+`pip install web-distiller` installs the client for the hosted Distiller API. It does not install or run the Distiller server.
+
+Use it with:
+- free-tier API keys (Trafilatura cleaning),
+- the Starter paid plan (Stripe-backed AI cleaning),
+- or alternate x402 payment-compatible deployments.
+
+## Install
+
+```bash
+pip install web-distiller
+set DISTILLER_API_BASE=https://webdistiller.dev
+```
+
+## Quick start
+
+```python
+from distiller_client import DistillerClient
+
+client = DistillerClient(api_key="your-key")
+result = client.markdown("https://example.com")
+print(result["cleaned_markdown"])
+```
+
+Or as a one-liner:
+
+```python
+import distiller_client
+
+print(distiller_client.markdown("https://example.com")["cleaned_markdown"])
+```
+
+## CLI
+
+```bash
+web-distiller https://example.com
+web-distiller https://example.com --format text
+web-distiller https://example.com --use-browser
+web-distiller https://example.com --force-refresh
+```
+
+## Configuration
+
+| Env var | Purpose |
+|---------|---------|
+| `DISTILLER_API_BASE` | API base URL (default: production) |
+| `DISTILLER_API_KEY` | Bearer token for authenticated access |
+| `DISTILLER_REFERRER_TOKEN` | Referral attribution token |
+
+All env vars can also be passed as constructor arguments.
+
+Typical flow:
+- use `web-distiller` without an API key for open/demo deployments,
+- register for an API key when you want the free 1,000-call tier,
+- upgrade that account to Starter through Stripe when you want AI cleaning and `/extract`.
+
+## Releases
+
+`web-distiller` is released to PyPI from GitHub Actions via Trusted Publishing.
+
+- Use the `.github/workflows/publish-web-distiller.yml` workflow for manual publishes.
+- Or push a tag in the format `web-distiller-vX.Y.Z` after updating `distiller-client/pyproject.toml`.

web_distiller-0.1.0/pyproject.toml

@@ -0,0 +1,38 @@
+[project]
+name = "web-distiller"
+version = "0.1.0"
+description = "Python client for Distiller — agent-ready web extraction with AI cleaning"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+keywords = ["distiller", "web-distiller", "markdown", "agents", "llm", "web-scraping", "url-to-markdown"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Internet :: WWW/HTTP",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+    "Topic :: Text Processing :: Markup :: Markdown",
+]
+dependencies = [
+    "httpx>=0.27.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/nickandbro/distiller"
+Documentation = "https://webdistiller.dev/docs"
+Source = "https://github.com/nickandbro/distiller"
+Issues = "https://github.com/nickandbro/distiller/issues"
+
+[project.scripts]
+web-distiller = "distiller_client:main"
+
+[build-system]
+requires = ["setuptools>=70.0"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]

web_distiller-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

web_distiller-0.1.0/src/distiller_client/__init__.py

@@ -0,0 +1,173 @@
+"""Lightweight Python client for the Distiller API."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+from typing import Any
+
+import httpx
+
+__all__ = ["DistillerClient", "distill", "markdown", "main"]
+
+DEFAULT_BASE_URL = "https://webdistiller.dev"
+
+
+class DistillerClient:
+    """Minimal client for the Distiller web-extraction API.
+
+    Usage::
+
+        client = DistillerClient(api_key="your-key")
+        result = client.markdown("https://example.com")
+        print(result["cleaned_markdown"])
+    """
+
+    def __init__(
+        self,
+        *,
+        api_key: str | None = None,
+        base_url: str | None = None,
+        referrer_token: str | None = None,
+        timeout: float = 30.0,
+    ) -> None:
+        self.base_url = (base_url or os.getenv("DISTILLER_API_BASE", DEFAULT_BASE_URL)).rstrip("/")
+        self.api_key = api_key or os.getenv("DISTILLER_API_KEY")
+        self.referrer_token = referrer_token or os.getenv("DISTILLER_REFERRER_TOKEN")
+        self.timeout = timeout
+
+    def _headers(self) -> dict[str, str]:
+        headers: dict[str, str] = {}
+        if self.api_key:
+            headers["Authorization"] = f"Bearer {self.api_key}"
+        if self.referrer_token:
+            headers["X-Distiller-Referrer"] = self.referrer_token
+        return headers
+
+    def markdown(
+        self,
+        url: str,
+        *,
+        use_browser: bool = False,
+        force_refresh: bool = False,
+        force_refetch: bool = False,
+    ) -> dict[str, Any]:
+        """Fetch cleaned Markdown for *url*."""
+        return self._post(
+            "/markdown", url,
+            use_browser=use_browser,
+            force_refresh=force_refresh,
+            force_refetch=force_refetch,
+        )
+
+    def distill(
+        self,
+        url: str,
+        *,
+        use_browser: bool = False,
+        force_refresh: bool = False,
+        force_refetch: bool = False,
+    ) -> dict[str, Any]:
+        """Fetch full distillation (HTML + Markdown + text) for *url*."""
+        return self._post(
+            "/distill", url,
+            use_browser=use_browser,
+            force_refresh=force_refresh,
+            force_refetch=force_refetch,
+        )
+
+    def start_paid_ai_checkout(self) -> dict[str, Any]:
+        """Create a Stripe checkout session for paid AI tier."""
+        resp = httpx.post(
+            f"{self.base_url}/billing/checkout",
+            headers=self._headers(),
+            timeout=self.timeout,
+        )
+        resp.raise_for_status()
+        return resp.json()
+
+    def _post(
+        self,
+        endpoint: str,
+        url: str,
+        *,
+        use_browser: bool = False,
+        force_refresh: bool = False,
+        force_refetch: bool = False,
+    ) -> dict[str, Any]:
+        headers = self._headers()
+        body: dict[str, Any] = {"url": url, "use_browser": use_browser}
+        if force_refresh:
+            body["force_refresh"] = True
+        if force_refetch:
+            body["force_refetch"] = True
+        resp = httpx.post(
+            f"{self.base_url}{endpoint}",
+            json=body,
+            headers=headers,
+            timeout=self.timeout,
+        )
+        try:
+            resp.raise_for_status()
+        except httpx.HTTPStatusError as exc:
+            if exc.response is not None and exc.response.status_code == 402:
+                raise RuntimeError("Payment required (402). Configure paid auth or provide payment headers.") from exc
+            raise
+        return resp.json()
+
+
+def distill(url: str, **kwargs: Any) -> dict[str, Any]:
+    """One-shot helper: ``distiller_client.distill("https://…")``."""
+    return DistillerClient(**kwargs).distill(url)
+
+
+def markdown(url: str, **kwargs: Any) -> dict[str, Any]:
+    """One-shot helper: ``distiller_client.markdown("https://…")``."""
+    return DistillerClient(**kwargs).markdown(url)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        prog="web-distiller",
+        description="Fetch cleaned content from the Distiller API.",
+    )
+    parser.add_argument("url", help="URL to distill")
+    parser.add_argument("--api-base", default=None, help="Distiller API base URL")
+    parser.add_argument("--api-key", default=None, help="API key (or set DISTILLER_API_KEY)")
+    parser.add_argument("--use-browser", action="store_true", help="Force browser rendering")
+    parser.add_argument("--force-refresh", action="store_true", help="Bypass response cache (content dedup still applies)")
+    parser.add_argument("--force-refetch", action="store_true", help="Bypass ALL caches and re-fetch from upstream (always billed)")
+    parser.add_argument(
+        "--format",
+        choices=("html", "markdown", "text", "json"),
+        default="markdown",
+        help="Output format (default: markdown)",
+    )
+    args = parser.parse_args()
+
+    client = DistillerClient(api_key=args.api_key, base_url=args.api_base)
+
+    call_kwargs = {
+        "use_browser": args.use_browser,
+        "force_refresh": args.force_refresh,
+        "force_refetch": args.force_refetch,
+    }
+
+    if args.format in ("html", "json"):
+        result = client.distill(args.url, **call_kwargs)
+    else:
+        result = client.markdown(args.url, **call_kwargs)
+
+    if args.format == "html":
+        print(result["cleaned_html"])
+    elif args.format == "markdown":
+        print(result["cleaned_markdown"])
+    elif args.format == "text":
+        print(result["cleaned_text"])
+    else:
+        print(json.dumps(result, indent=2))
+
+
+if __name__ == "__main__":
+    main()

web_distiller-0.1.0/src/web_distiller.egg-info/PKG-INFO

@@ -0,0 +1,89 @@
+Metadata-Version: 2.4
+Name: web-distiller
+Version: 0.1.0
+Summary: Python client for Distiller — agent-ready web extraction with AI cleaning
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/nickandbro/distiller
+Project-URL: Documentation, https://webdistiller.dev/docs
+Project-URL: Source, https://github.com/nickandbro/distiller
+Project-URL: Issues, https://github.com/nickandbro/distiller/issues
+Keywords: distiller,web-distiller,markdown,agents,llm,web-scraping,url-to-markdown
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing :: Markup :: Markdown
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: httpx>=0.27.0
+
+# web-distiller
+
+Python client for the [Distiller](https://webdistiller.dev) API — agent-ready web extraction with AI cleaning.
+
+`pip install web-distiller` installs the client for the hosted Distiller API. It does not install or run the Distiller server.
+
+Use it with:
+- free-tier API keys (Trafilatura cleaning),
+- the Starter paid plan (Stripe-backed AI cleaning),
+- or alternate x402 payment-compatible deployments.
+
+## Install
+
+```bash
+pip install web-distiller
+set DISTILLER_API_BASE=https://webdistiller.dev
+```
+
+## Quick start
+
+```python
+from distiller_client import DistillerClient
+
+client = DistillerClient(api_key="your-key")
+result = client.markdown("https://example.com")
+print(result["cleaned_markdown"])
+```
+
+Or as a one-liner:
+
+```python
+import distiller_client
+
+print(distiller_client.markdown("https://example.com")["cleaned_markdown"])
+```
+
+## CLI
+
+```bash
+web-distiller https://example.com
+web-distiller https://example.com --format text
+web-distiller https://example.com --use-browser
+web-distiller https://example.com --force-refresh
+```
+
+## Configuration
+
+| Env var | Purpose |
+|---------|---------|
+| `DISTILLER_API_BASE` | API base URL (default: production) |
+| `DISTILLER_API_KEY` | Bearer token for authenticated access |
+| `DISTILLER_REFERRER_TOKEN` | Referral attribution token |
+
+All env vars can also be passed as constructor arguments.
+
+Typical flow:
+- use `web-distiller` without an API key for open/demo deployments,
+- register for an API key when you want the free 1,000-call tier,
+- upgrade that account to Starter through Stripe when you want AI cleaning and `/extract`.
+
+## Releases
+
+`web-distiller` is released to PyPI from GitHub Actions via Trusted Publishing.
+
+- Use the `.github/workflows/publish-web-distiller.yml` workflow for manual publishes.
+- Or push a tag in the format `web-distiller-vX.Y.Z` after updating `distiller-client/pyproject.toml`.

web_distiller-0.1.0/src/web_distiller.egg-info/SOURCES.txt

@@ -0,0 +1,9 @@
+README.md
+pyproject.toml
+src/distiller_client/__init__.py
+src/web_distiller.egg-info/PKG-INFO
+src/web_distiller.egg-info/SOURCES.txt
+src/web_distiller.egg-info/dependency_links.txt
+src/web_distiller.egg-info/entry_points.txt
+src/web_distiller.egg-info/requires.txt
+src/web_distiller.egg-info/top_level.txt

web_distiller-0.1.0/src/web_distiller.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

web_distiller-0.1.0/src/web_distiller.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+web-distiller = distiller_client:main

web_distiller-0.1.0/src/web_distiller.egg-info/requires.txt

@@ -0,0 +1 @@
+httpx>=0.27.0

web_distiller-0.1.0/src/web_distiller.egg-info/top_level.txt

@@ -0,0 +1 @@
+distiller_client