tinyfish 0.2.2__tar.gz

tinyfish-0.2.2/.gitignore

@@ -0,0 +1 @@
+playground/

tinyfish-0.2.2/PKG-INFO

@@ -0,0 +1,8 @@
+Metadata-Version: 2.4
+Name: tinyfish
+Version: 0.2.2
+Summary: Official Python SDK for the TinyFish API
+Requires-Python: >=3.11
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: tenacity>=8.0.0

tinyfish-0.2.2/README.md

@@ -0,0 +1,334 @@
+# TinyFish Python SDK
+
+The official Python SDK for [TinyFish](https://agent.tinyfish.ai)
+
+## Installation
+
+```bash
+pip install tinyfish
+```
+
+Requires Python 3.11+.
+
+## Get your API key
+
+Sign up and grab your key at [agent.tinyfish.ai/api-keys](https://agent.tinyfish.ai/api-keys).
+
+## Quickstart
+
+```python
+from tinyfish import TinyFish
+
+client = TinyFish(api_key="your-api-key")
+
+response = client.agent.run(
+    goal="What is the current Bitcoin price?",
+    url="https://www.coinbase.com/price/bitcoin",
+)
+print(response.result)
+```
+
+Or set the `TINYFISH_API_KEY` environment variable and omit `api_key`:
+
+```python
+client = TinyFish()
+```
+
+## Methods
+
+Every method below is available on both `TinyFish` (sync) and `AsyncTinyFish` (async). Async versions have the same signatures — just `await` them.
+
+| Method | Description | Returns | Blocks? |
+|--------|-------------|---------|---------|
+| [`agent.run()`](#agentrun--block-until-done) | Run an automation, wait for the result | `AgentRunResponse` | Yes |
+| [`agent.queue()`](#agentqueue--fire-and-forget) | Start an automation, return immediately | `AgentRunAsyncResponse` | No |
+| [`agent.stream()`](#agentstream--real-time-events) | Stream live SSE events as the agent works | `AgentStream` | No |
+| [`runs.get()`](#runsget--retrieve-a-single-run) | Retrieve a single run by ID | `Run` | — |
+| [`runs.list()`](#runslist--list-and-filter-runs) | List runs with filtering, sorting, pagination | `RunListResponse` | — |
+
+---
+
+### `agent.run()` — block until done
+
+Sends the automation and waits for it to finish. Returns the full result in one shot.
+
+```python
+from tinyfish import TinyFish, RunStatus, BrowserProfile, ProxyConfig, ProxyCountryCode
+
+client = TinyFish()
+
+response = client.agent.run(
+    goal="Extract the top 5 headlines",              # required — what to do on the page
+    url="https://news.ycombinator.com",              # required — URL to open
+    browser_profile=BrowserProfile.STEALTH,          # optional — "lite" (default) or "stealth"
+    proxy_config=ProxyConfig(                        # optional — proxy settings
+        enabled=True,
+        country_code=ProxyCountryCode.US,            # optional — US, GB, CA, DE, FR, JP, AU
+    ),
+)
+
+if response.status == RunStatus.COMPLETED:
+    print(response.result)
+else:
+    print(f"Failed: {response.error.message}")
+```
+
+**Returns `AgentRunResponse`:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `status` | `RunStatus` | `COMPLETED`, `FAILED`, etc. |
+| `run_id` | `str \| None` | Unique run identifier |
+| `result` | `dict \| None` | Extracted data (`None` if failed) |
+| `error` | `RunError \| None` | Error details (`None` if succeeded) |
+| `num_of_steps` | `int` | Number of steps the agent took |
+| `started_at` | `datetime \| None` | When the run started |
+| `finished_at` | `datetime \| None` | When the run finished |
+
+---
+
+### `agent.queue()` — fire and forget
+
+Starts the automation in the background and returns a `run_id` immediately. Poll with `runs.get()` when you're ready for the result.
+
+```python
+import time
+from tinyfish import TinyFish, RunStatus
+
+client = TinyFish()
+
+queued = client.agent.queue(
+    goal="Extract the top 5 headlines",              # required — what to do on the page
+    url="https://news.ycombinator.com",              # required — URL to open
+    browser_profile=None,                            # optional — "lite" (default) or "stealth"
+    proxy_config=None,                               # optional — proxy settings
+)
+print(f"Run started: {queued.run_id}")
+
+# Poll for completion
+while True:
+    run = client.runs.get(queued.run_id)
+    if run.status in (RunStatus.COMPLETED, RunStatus.FAILED):
+        break
+    time.sleep(5)
+
+print(run.result)
+```
+
+**Returns `AgentRunAsyncResponse`:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `run_id` | `str \| None` | Run ID to poll with `runs.get()` |
+| `error` | `RunError \| None` | Error if queuing itself failed |
+
+---
+
+### `agent.stream()` — real-time events
+
+Opens a Server-Sent Events stream. You get live progress updates as the agent works, plus a WebSocket URL for a live browser preview.
+
+```python
+from tinyfish import TinyFish, CompleteEvent, ProgressEvent
+
+client = TinyFish()
+
+with client.agent.stream(
+    goal="Extract the top 5 headlines",              # required — what to do on the page
+    url="https://news.ycombinator.com",              # required — URL to open
+    browser_profile=None,                            # optional — "lite" (default) or "stealth"
+    proxy_config=None,                               # optional — proxy settings
+    on_started=lambda e: print(f"Started: {e.run_id}"),          # optional — called when run starts
+    on_streaming_url=lambda e: print(f"Watch: {e.streaming_url}"),  # optional — called with live browser URL
+    on_progress=lambda e: print(f"  > {e.purpose}"),             # optional — called on each step
+    on_heartbeat=lambda e: None,                                 # optional — called on keepalive pings
+    on_complete=lambda e: print(f"Done: {e.status}"),            # optional — called when run finishes
+) as stream:
+    for event in stream:
+        # Callbacks fire automatically during iteration.
+        # You can also inspect events directly:
+        if isinstance(event, CompleteEvent):
+            print(event.result_json)
+```
+
+**Returns `AgentStream`** — a context manager you iterate over. Events arrive in order: `STARTED` → `STREAMING_URL` → `PROGRESS` (repeated) → `COMPLETE`.
+
+See the [Streaming Guide](docs/streaming-guide.md) for the full event lifecycle, event types, and advanced patterns.
+
+---
+
+### `runs.get()` — retrieve a single run
+
+Fetch the full details of a run by its ID.
+
+```python
+run = client.runs.get(
+    "run_abc123",   # required — the run ID
+)
+
+print(run.status)   # PENDING, RUNNING, COMPLETED, FAILED, CANCELLED
+print(run.result)
+print(run.goal)
+print(run.streaming_url)          # live browser URL (while RUNNING)
+print(run.browser_config)         # proxy/browser settings that were used
+```
+
+**Returns `Run`:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `run_id` | `str` | Unique identifier |
+| `status` | `RunStatus` | `PENDING`, `RUNNING`, `COMPLETED`, `FAILED`, `CANCELLED` |
+| `goal` | `str` | The goal that was given |
+| `result` | `dict \| None` | Extracted data (`None` if not completed) |
+| `error` | `RunError \| None` | Error details (`None` if succeeded) |
+| `streaming_url` | `str \| None` | Live browser URL (available while running) |
+| `browser_config` | `BrowserConfig \| None` | Proxy/browser settings used |
+| `created_at` | `datetime` | When the run was created |
+| `started_at` | `datetime \| None` | When execution started |
+| `finished_at` | `datetime \| None` | When execution finished |
+
+**Raises:** `ValueError` if `run_id` is empty. `NotFoundError` if no run exists with that ID.
+
+---
+
+### `runs.list()` — list and filter runs
+
+List runs with optional filtering, sorting, and cursor-based pagination. All parameters are optional.
+
+```python
+from tinyfish import RunStatus, SortDirection
+
+response = client.runs.list(
+    status=RunStatus.COMPLETED,                      # optional — filter by status
+    goal="headlines",                                # optional — filter by goal text
+    created_after="2025-01-01T00:00:00Z",            # optional — ISO 8601 lower bound
+    created_before="2025-12-31T23:59:59Z",           # optional — ISO 8601 upper bound
+    sort_direction=SortDirection.DESC,                # optional — "asc" or "desc"
+    limit=10,                                        # optional — max runs per page
+    cursor=None,                                     # optional — pagination cursor from previous response
+)
+
+for run in response.data:
+    print(f"{run.run_id} | {run.goal}")
+
+# Pagination
+if response.pagination.has_more:
+    next_page = client.runs.list(cursor=response.pagination.next_cursor)
+```
+
+**Returns `RunListResponse`:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `data` | `list[Run]` | List of runs |
+| `pagination.total` | `int` | Total runs matching filters |
+| `pagination.has_more` | `bool` | Whether more pages exist |
+| `pagination.next_cursor` | `str \| None` | Pass to `cursor=` for the next page |
+
+See the [Pagination Guide](docs/pagination-guide.md) for full pagination loop examples.
+
+---
+
+## Sync vs Async
+
+Use `AsyncTinyFish` when you're in an async context (FastAPI, aiohttp, etc.):
+
+**Sync:**
+
+```python
+from tinyfish import TinyFish
+
+client = TinyFish()
+response = client.agent.run(goal="...", url="...")
+```
+
+**Async:**
+
+```python
+from tinyfish import AsyncTinyFish
+
+client = AsyncTinyFish()
+response = await client.agent.run(goal="...", url="...")
+```
+
+All five methods (`agent.run()`, `agent.queue()`, `agent.stream()`, `runs.get()`, `runs.list()`) work the same way — same parameters, just `await`-ed.
+
+## Configuration
+
+### Client options
+
+```python
+client = TinyFish(
+    api_key="your-api-key",         # optional — or set TINYFISH_API_KEY env var
+    base_url="https://agent.tinyfish.ai",  # optional — default shown
+    timeout=600.0,                  # optional — seconds (default: 600)
+    max_retries=2,                  # optional — retry attempts (default: 2)
+)
+```
+
+The SDK retries `408`, `429`, and `5xx` errors automatically with exponential backoff (0.5s multiplier, max 8s wait).
+
+### Browser profiles
+
+Control the browser environment with `browser_profile`:
+
+- **`lite`** (default) — fast, lightweight. Good for most sites.
+- **`stealth`** — anti-detection mode. Use for sites with bot protection.
+
+```python
+from tinyfish import BrowserProfile
+
+response = client.agent.run(
+    goal="...",
+    url="...",
+    browser_profile=BrowserProfile.STEALTH,
+)
+```
+
+### Proxy configuration
+
+Route requests through a proxy, optionally pinned to a country:
+
+```python
+from tinyfish import ProxyConfig, ProxyCountryCode
+
+response = client.agent.run(
+    goal="...",
+    url="...",
+    proxy_config=ProxyConfig(enabled=True, country_code=ProxyCountryCode.US),
+)
+```
+
+Available countries: `US`, `GB`, `CA`, `DE`, `FR`, `JP`, `AU`.
+
+See the [Proxy & Browser Profiles Guide](docs/proxy-and-browser-profiles.md) for more details.
+
+## Error handling
+
+```python
+from tinyfish import TinyFish, AuthenticationError, RateLimitError, SDKError
+
+client = TinyFish()
+
+try:
+    response = client.agent.run(goal="...", url="...")
+except AuthenticationError:
+    print("Invalid API key")
+except RateLimitError:
+    print("Rate limited (retries exhausted)")
+except SDKError:
+    print("Something else went wrong")
+```
+
+The SDK automatically retries transient errors (`408`, `429`, `5xx`) up to `max_retries` times with exponential backoff. Non-retryable errors (`401`, `400`, `404`) raise immediately.
+
+For the full exception hierarchy and internal architecture, see [docs/internal/exceptions-and-errors-guide.md](docs/internal/exceptions-and-errors-guide.md).
+
+## Guides
+
+- [Streaming Guide](docs/streaming-guide.md) — event lifecycle, callbacks vs iteration, event type reference
+- [Proxy & Browser Profiles](docs/proxy-and-browser-profiles.md) — stealth mode, proxy countries
+- [Pagination Guide](docs/pagination-guide.md) — filtering, sorting, cursor-based pagination
+- [Exceptions & Error Handling (internal)](docs/internal/exceptions-and-errors-guide.md) — layer-by-layer architecture
+- [Testing Guide](tests/testing_guide.md) — running and writing tests

tinyfish-0.2.2/RELEASE.md

@@ -0,0 +1,71 @@
+# Releasing the Python SDK to PyPI
+
+This document covers how to publish a new version of the TinyFish Python SDK to the public PyPI registry so users can `pip install tinyfish`.
+
+## Prerequisites (one-time infra setup)
+
+The `PYPI_API_TOKEN` secret exists in AWS Secrets Manager (`pypi-token`) and is already used by other repos. It needs to be added to the `ux-labs` repo via a PR to `github-control/repos.tf`:
+
+```hcl
+"ux-labs" : {
+  secrets = {
+    # ... existing secrets ...
+    PYPI_API_TOKEN = data.aws_secretsmanager_secret_version.pypi-token.secret_string
+  }
+}
+```
+
+This is a one-time change. Once merged and applied, the workflow will have access to the token.
+
+## Release process
+
+### Step 1: Bump the version
+
+Edit `sdk/sdk-python/pyproject.toml` and update the `version` field:
+
+```toml
+[project]
+version = "0.2.0"  # bump this
+```
+
+Commit and merge to main.
+
+### Step 2: Create a GitHub Release
+
+1. Go to the `ux-labs` repo on GitHub → **Releases** → **Draft a new release**
+2. Set the tag to match the version exactly, prefixed with `v`:
+   - Version `0.2.0` → tag `v0.2.0`
+3. Set the target to `main`
+4. Write release notes summarizing what changed
+5. Click **Publish release**
+
+The workflow validates that the tag matches the version in `pyproject.toml` — if they don't match, the build fails before anything is published.
+
+### Step 3: Monitor the workflow
+
+The `SDK Python CD - Publish to PyPI` workflow triggers automatically on release publish. It runs two jobs in sequence:
+
+1. **Build** — validates the tag matches `pyproject.toml` version, builds the package with `uv build`
+2. **Publish to PyPI** — publishes to `pypi.org` using the `PYPI_API_TOKEN` secret
+
+Monitor progress at: `github.com/tinyfish-io/ux-labs/actions`
+
+### Step 4: Verify
+
+After the workflow completes, verify the release:
+
+```bash
+pip install tinyfish==0.2.0
+python -c "from tinyfish import Tinyfish; print('ok')"
+```
+
+## Troubleshooting
+
+**Tag does not match pyproject.toml version**
+The build job validates that the git tag (e.g., `v0.2.0`) matches the version in `pyproject.toml` (e.g., `0.2.0`). Fix by updating `pyproject.toml` to match the tag before publishing, then delete and recreate the release.
+
+**403 / authentication error**
+Verify that `PYPI_API_TOKEN` has been added to the `ux-labs` repo secrets in `github-control`. The token must exist as a GitHub Actions secret before the workflow can publish.
+
+**Package already exists on PyPI**
+PyPI does not allow re-uploading the same version. Bump the version in `pyproject.toml` and create a new release.

tinyfish-0.2.2/docs/exceptions-and-errors-guide.md

@@ -0,0 +1,110 @@
+# Exceptions & Error Handling
+
+---
+
+## Layer 1 — httpx (the bottom, third-party)
+
+`httpx` is the HTTP library doing the actual network calls. It can throw two kinds of errors (regardless of async or sync):
+
+- `httpx.TimeoutException` — the request took too long
+- `httpx.RequestError` — any other network failure (DNS, connection refused, etc.)
+
+These are not SDK errors yet. The job of the layers above is to catch and translate them.
+
+---
+
+## Layer 2 — `_make_status_error` in `_base.py` (the translator)
+
+When the server responds but with a bad status code, httpx itself doesn't raise — it just returns a response with `response.is_error == True`. So `_base.py:60` has `_make_status_error()` which:
+
+1. Looks up the status code in a map → picks the right exception class
+2. Tries to parse the JSON body for a human-readable message
+3. Falls back to raw response text if the body isn't JSON
+
+```
+404  → NotFoundError
+401  → AuthenticationError
+429  → RateLimitError
+5xx  → InternalServerError
+...etc
+```
+
+This is where the raw HTTP world gets converted into SDK-typed exceptions.
+
+---
+
+## Layer 3 — `_request()` in `sync.py` / `async_.py` (the retry + catch layer)
+
+This is the core of the logic. It has two jobs:
+
+**Job 1 — retry certain errors via tenacity:**
+
+```
+RequestTimeoutError  ← server sent 408
+RateLimitError       ← server sent 429
+InternalServerError  ← server sent 5xx
+httpx.RequestError   ← network failure
+```
+
+These are retried up to `max_retries` times (default 2) with exponential backoff (0.5s multiplier, max 8s wait).
+
+**Job 2 — translate the remaining httpx errors that survived all retries:**
+
+```python
+except httpx.TimeoutException → raise APITimeoutError
+except httpx.RequestError     → raise APIConnectionError
+```
+
+Notice what's *not* retried: `BadRequestError` (400), `AuthenticationError` (401), `NotFoundError` (404), etc. Those propagate immediately on the first attempt — there's no point retrying them.
+
+---
+
+## Layer 4 — `_get` / `_post` / `_post_stream` (the interface layer)
+
+These call `_request()` and either parse the response via Pydantic (`_parse_response`) or yield raw lines (for SSE streams). They don't add any error handling — they just let exceptions propagate upward.
+
+---
+
+## Layer 5 — The exception hierarchy (`exceptions.py`)
+
+Everything the user can catch inherits from `SDKError`:
+
+```
+SDKError
+└─ APIError                   ← carries .request and .response
+   ├─ APIConnectionError       ← network failed
+   │  └─ APITimeoutError       ← specifically a timeout
+   └─ APIStatusError           ← carries .status_code too
+      ├─ BadRequestError       (400)
+      ├─ AuthenticationError   (401)
+      ├─ PermissionDeniedError (403)
+      ├─ NotFoundError         (404)
+      ├─ RequestTimeoutError   (408)  ← server-side timeout
+      ├─ ConflictError         (409)
+      ├─ UnprocessableEntityError (422)
+      ├─ RateLimitError        (429)
+      └─ InternalServerError   (500+)
+```
+
+Note the distinction between `APITimeoutError` (client-side — the request never completed) and `RequestTimeoutError` (server-side — server responded with 408).
+
+---
+
+## What this means for SDK users
+
+They can be as broad or specific as they want:
+
+```python
+from tinyfish import TinyFish, RateLimitError, AuthenticationError, SDKError
+
+client = TinyFish(api_key="...")
+
+try:
+    result = client.agent.run(...)
+except AuthenticationError:
+    # bad API key — don't retry
+except RateLimitError:
+    # already retried internally, give up
+except SDKError:
+    # catch anything else
+```

tinyfish-0.2.2/docs/internal/exceptions-and-errors-guide.md

@@ -0,0 +1,110 @@
+# Exceptions & Error Handling
+
+---
+
+## Layer 1 — httpx (the bottom, third-party)
+
+`httpx` is the HTTP library doing the actual network calls. It can throw two kinds of errors (regardless of async or sync):
+
+- `httpx.TimeoutException` — the request took too long
+- `httpx.RequestError` — any other network failure (DNS, connection refused, etc.)
+
+These are not SDK errors yet. The job of the layers above is to catch and translate them.
+
+---
+
+## Layer 2 — `_make_status_error` in `_base.py` (the translator)
+
+When the server responds but with a bad status code, httpx itself doesn't raise — it just returns a response with `response.is_error == True`. So `_base.py:60` has `_make_status_error()` which:
+
+1. Looks up the status code in a map → picks the right exception class
+2. Tries to parse the JSON body for a human-readable message
+3. Falls back to raw response text if the body isn't JSON
+
+```
+404  → NotFoundError
+401  → AuthenticationError
+429  → RateLimitError
+5xx  → InternalServerError
+...etc
+```
+
+This is where the raw HTTP world gets converted into SDK-typed exceptions.
+
+---
+
+## Layer 3 — `_request()` in `sync.py` / `async_.py` (the retry + catch layer)
+
+This is the core of the logic. It has two jobs:
+
+**Job 1 — retry certain errors via tenacity:**
+
+```
+RequestTimeoutError  ← server sent 408
+RateLimitError       ← server sent 429
+InternalServerError  ← server sent 5xx
+httpx.RequestError   ← network failure
+```
+
+These are retried up to `max_retries` times (default 2) with exponential backoff (0.5s multiplier, max 8s wait).
+
+**Job 2 — translate the remaining httpx errors that survived all retries:**
+
+```python
+except httpx.TimeoutException → raise APITimeoutError
+except httpx.RequestError     → raise APIConnectionError
+```
+
+Notice what's *not* retried: `BadRequestError` (400), `AuthenticationError` (401), `NotFoundError` (404), etc. Those propagate immediately on the first attempt — there's no point retrying them.
+
+---
+
+## Layer 4 — `_get` / `_post` / `_post_stream` (the interface layer)
+
+These call `_request()` and either parse the response via Pydantic (`_parse_response`) or yield raw lines (for SSE streams). They don't add any error handling — they just let exceptions propagate upward.
+
+---
+
+## Layer 5 — The exception hierarchy (`exceptions.py`)
+
+Everything the user can catch inherits from `SDKError`:
+
+```
+SDKError
+└─ APIError                   ← carries .request and .response
+   ├─ APIConnectionError       ← network failed
+   │  └─ APITimeoutError       ← specifically a timeout
+   └─ APIStatusError           ← carries .status_code too
+      ├─ BadRequestError       (400)
+      ├─ AuthenticationError   (401)
+      ├─ PermissionDeniedError (403)
+      ├─ NotFoundError         (404)
+      ├─ RequestTimeoutError   (408)  ← server-side timeout
+      ├─ ConflictError         (409)
+      ├─ UnprocessableEntityError (422)
+      ├─ RateLimitError        (429)
+      └─ InternalServerError   (500+)
+```
+
+Note the distinction between `APITimeoutError` (client-side — the request never completed) and `RequestTimeoutError` (server-side — server responded with 408).
+
+---
+
+## What this means for SDK users
+
+They can be as broad or specific as they want:
+
+```python
+from tinyfish import TinyFish, RateLimitError, AuthenticationError, SDKError
+
+client = TinyFish(api_key="...")
+
+try:
+    result = client.agent.run(...)
+except AuthenticationError:
+    # bad API key — don't retry
+except RateLimitError:
+    # already retried internally, give up
+except SDKError:
+    # catch anything else
+```

tinyfish-0.2.2/docs/internal/publishing-private-guide.md

@@ -0,0 +1,59 @@
+# Publishing to the TinyFish Private Registry
+
+## Prerequisites
+
+- [uv](https://docs.astral.sh/uv/) installed
+- PyPI credentials from 1Password
+- Need to be connected to VPN (or company wifi)
+
+## 1. Clean and build
+
+```bash
+cd sdk/sdk-python
+rm -rf dist/
+uv build
+```
+
+This outputs `dist/tinyfish-<version>-py3-none-any.whl` and `dist/tinyfish-<version>.tar.gz`.
+
+## 2. Publish
+
+Set credentials via environment variables (values are in 1Password):
+
+```bash
+export UV_PUBLISH_USERNAME="<from 1password>"
+export UV_PUBLISH_PASSWORD="<from 1password>"
+```
+
+Then publish:
+
+```bash
+uv publish dist/* --publish-url https://pypi.production.tinyfish.io/
+```
+
+## 3. Verify the install
+
+Configure pip to authenticate with the private registry. Add to `~/.pip/pip.conf` (or `pip.ini` on Windows):
+
+```ini
+[global]
+extra-index-url = https://pypi.production.tinyfish.io/
+```
+
+Then set credentials via environment variables:
+
+```bash
+export PIP_EXTRA_INDEX_URL="https://${TINYFISH_PYPI_USER}:${TINYFISH_PYPI_PASS}@pypi.production.tinyfish.io/"
+```
+
+```bash
+pip install tinyfish
+python -c "from tinyfish import TinyFish; print('ok')"
+```
+
+Use `--extra-index-url` (not `--index-url`) so pip still falls back to public PyPI for dependencies like `httpx`, `pydantic`, and `tenacity`.
+
+## Notes
+
+- **Bump the version** before each publish — the registry rejects duplicate versions. Update the version in `pyproject.toml`.
+- **Never put credentials inline** in commands — they leak into shell history and logs. Always use environment variables or config files.