hotdata 0.4.1__tar.gz → 0.6.0__tar.gz

hotdata-0.6.0/PKG-INFO

@@ -0,0 +1,200 @@
+Metadata-Version: 2.4
+Name: hotdata
+Version: 0.6.0
+Summary: Hotdata API
+Author-email: Hotdata <developers@hotdata.dev>
+License: MIT
+Project-URL: Homepage, https://www.hotdata.dev
+Project-URL: Repository, https://github.com/hotdata-dev/sdk-python
+Keywords: hotdata,api-client,data-platform
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: urllib3<3.0.0,>=2.1.0
+Requires-Dist: python-dateutil>=2.8.2
+Requires-Dist: pydantic>=2.4.0
+Requires-Dist: typing-extensions>=4.7.1
+Provides-Extra: arrow
+Requires-Dist: pyarrow>=14.0.1; extra == "arrow"
+
+# hotdata
+
+Official Python client for the [Hotdata](https://www.hotdata.dev) HTTP API: workspaces, connections, datasets, SQL queries, results, secrets, uploads, indexes, jobs, embedding providers, and workspace context.
+
+## Requirements
+
+Python 3.9+
+
+## Install
+
+```sh
+pip install hotdata
+```
+
+For an unreleased revision:
+
+```sh
+pip install "git+https://github.com/hotdata-dev/sdk-python.git"
+```
+
+From a local checkout (editable):
+
+```sh
+pip install -e .
+```
+
+## Authentication
+
+The API uses an **API key** sent as `Authorization: Bearer <key>`, plus an **`X-Workspace-Id`** header on requests scoped to a workspace.
+
+```python
+import hotdata
+
+configuration = hotdata.Configuration(
+    api_key="YOUR_API_KEY",
+    workspace_id="YOUR_WORKSPACE_ID",
+)
+```
+
+`host` defaults to `https://api.hotdata.dev`. Override it if you target another environment.
+
+## Usage
+
+```python
+import hotdata
+from hotdata.rest import ApiException
+
+configuration = hotdata.Configuration(
+    api_key="YOUR_API_KEY",
+    workspace_id="YOUR_WORKSPACE_ID",
+)
+
+with hotdata.ApiClient(configuration) as api_client:
+    workspaces = hotdata.WorkspacesApi(api_client)
+    try:
+        response = workspaces.list_workspaces()
+    except ApiException as e:
+        print(f"API error: {e.status} {e.reason}\n{e.body}")
+```
+
+Each `Api` class groups endpoints by resource. Construct the client, then call the typed methods you need.
+
+## Arrow results
+
+Query results can be fetched as an [Apache Arrow](https://arrow.apache.org/) IPC stream instead of JSON, which is faster and far more memory-efficient for large result sets. Install the optional extra:
+
+```sh
+pip install 'hotdata[arrow]'
+```
+
+Use `hotdata.arrow.ResultsApi` (a drop-in subclass of `ResultsApi` that adds Arrow methods):
+
+```python
+from hotdata import ApiClient, Configuration
+from hotdata.arrow import ResultsApi
+
+with ApiClient(Configuration(api_key="...", workspace_id="...")) as client:
+    results = ResultsApi(client)
+
+    # Results are scoped to a database via the required `X-Database-Id` header,
+    # so pass the id of the database the query ran in.
+    database_id = "your_database_id"
+
+    # Buffered: returns a pyarrow.Table.
+    table = results.get_result_arrow(result_id, database_id)
+
+    # Streaming: yields a pyarrow.RecordBatchStreamReader without
+    # materializing the full table in memory.
+    with results.stream_result_arrow(result_id, database_id) as reader:
+        for batch in reader:
+            ...
+```
+
+Both methods accept `offset` and `limit` for pagination. They raise `hotdata.arrow.ResultNotReadyError` if the result is still pending or processing — poll `results.get_result(result_id, database_id)` until `status == "ready"` first.
+
+## File uploads
+
+`hotdata.uploads.UploadsApi` (also the default `hotdata.UploadsApi`) adds
+`upload_file`, which uploads a local file **directly to object storage** and
+finalizes it in one call. It opens an upload session, `PUT`s the bytes straight
+to storage — a single `PUT` for a small file, concurrent part `PUT`s for a large
+one — then finalizes. The bytes never round-trip through the API.
+
+```python
+from hotdata import ApiClient, Configuration, UploadsApi
+
+with ApiClient(Configuration(api_key="...", workspace_id="...")) as client:
+    uploads = UploadsApi(client)
+
+    finalized = uploads.upload_file(
+        "data.parquet",
+        content_type="application/parquet",
+        progress=lambda done, total: print(f"{done}/{total} bytes"),
+    )
+
+    # Pass finalized.upload_id to the managed-table load endpoint.
+    print(finalized.upload_id)
+```
+
+`upload_file` accepts a path, raw `bytes`, or a seekable binary file object
+(`size` is inferred for all three; a file object is read from its current
+position to the end). The SDK picks single vs. multipart from the size,
+auto-scales the part size, and bounds part concurrency to a peak-memory budget
+(override with `part_size` / `max_concurrency` / `part_retry`). A file larger than
+8 MiB uploads via a **streaming** session — the SDK mints each part's URL just
+before its `PUT`, so a presigned URL can't expire mid-transfer on a slow upload.
+Storage `PUT`s go through a dedicated, header-isolated connection pool (auth and
+workspace headers never reach object storage, which would otherwise reject the
+upload) with a 30s connect timeout so a dead endpoint fails fast. Finalize is sent
+with retries disabled so the exactly-once call is never accidentally replayed.
+
+Every failure is a subclass of `hotdata.uploads.UploadError` (also importable as
+`from hotdata import UploadError`), so a single `except UploadError` catches the
+whole flow: `SessionCreateError` (opening the session — check `.status` for a
+`501` `PRESIGN_UNSUPPORTED`), `StorageError` (storage returned a non-2xx; `.exhausted`
+is `True` if it outlived every retry round), `StorageTransportError` (the PUT
+failed before any response), `MissingETagError`, `MintPartError` (minting a part
+URL), `FinalizeError`, `MalformedSessionError`, `SizeLimitError`, and
+`UploadCancelledError`. The phase errors that wrap a control-plane call chain the
+underlying `hotdata.exceptions.ApiException` as `__cause__` (and expose it as
+`.api_exception` / `.status`). A local file read error surfaces as `OSError`.
+
+The `progress` callback receives a **cumulative** `(bytes_done, total)` — for a
+tqdm bar (whose `update(n)` wants a delta, and which isn't thread-safe under
+multipart) use the ready-made adapter:
+
+```python
+from hotdata import tqdm_progress
+from tqdm import tqdm
+
+with tqdm(total=size, unit="B", unit_scale=True) as bar:
+    uploads.upload_file("data.parquet", progress=tqdm_progress(bar))
+```
+
+Pass a `threading.Event` as `cancel_event` to abort an in-flight upload; tune the
+control-plane calls with `request_timeout` (storage-PUT timeouts are automatic and
+size-scaled).
+
+For that fallback (or to upload from a non-seekable stream), use `upload_stream`,
+which sends the bytes to the legacy `POST /v1/files` endpoint in one request,
+streaming a file object without buffering it in memory:
+
+```python
+with open("data.parquet", "rb") as f:
+    resp = uploads.upload_stream(f, content_type="application/parquet")
+print(resp.id)
+```
+
+Note `upload_file` shadows the generated raw-body `upload_file(body=...)`; that
+raw operation is still reachable at
+`hotdata.api.uploads_api.UploadsApi.upload_file`.
+
+## API reference
+
+Generated Markdown for every operation and model is in [`docs/`](https://github.com/hotdata-dev/sdk-python/tree/main/docs):
+
+- Resource APIs: `docs/*Api.md` (for example [`QueryApi.md`](https://github.com/hotdata-dev/sdk-python/blob/main/docs/QueryApi.md))
+- Request and response models: `docs/<ModelName>.md`
+
+## Support
+
+Questions and issues: [github.com/hotdata-dev/sdk-python](https://github.com/hotdata-dev/sdk-python).

hotdata-0.6.0/README.md

@@ -0,0 +1,182 @@
+# hotdata
+
+Official Python client for the [Hotdata](https://www.hotdata.dev) HTTP API: workspaces, connections, datasets, SQL queries, results, secrets, uploads, indexes, jobs, embedding providers, and workspace context.
+
+## Requirements
+
+Python 3.9+
+
+## Install
+
+```sh
+pip install hotdata
+```
+
+For an unreleased revision:
+
+```sh
+pip install "git+https://github.com/hotdata-dev/sdk-python.git"
+```
+
+From a local checkout (editable):
+
+```sh
+pip install -e .
+```
+
+## Authentication
+
+The API uses an **API key** sent as `Authorization: Bearer <key>`, plus an **`X-Workspace-Id`** header on requests scoped to a workspace.
+
+```python
+import hotdata
+
+configuration = hotdata.Configuration(
+    api_key="YOUR_API_KEY",
+    workspace_id="YOUR_WORKSPACE_ID",
+)
+```
+
+`host` defaults to `https://api.hotdata.dev`. Override it if you target another environment.
+
+## Usage
+
+```python
+import hotdata
+from hotdata.rest import ApiException
+
+configuration = hotdata.Configuration(
+    api_key="YOUR_API_KEY",
+    workspace_id="YOUR_WORKSPACE_ID",
+)
+
+with hotdata.ApiClient(configuration) as api_client:
+    workspaces = hotdata.WorkspacesApi(api_client)
+    try:
+        response = workspaces.list_workspaces()
+    except ApiException as e:
+        print(f"API error: {e.status} {e.reason}\n{e.body}")
+```
+
+Each `Api` class groups endpoints by resource. Construct the client, then call the typed methods you need.
+
+## Arrow results
+
+Query results can be fetched as an [Apache Arrow](https://arrow.apache.org/) IPC stream instead of JSON, which is faster and far more memory-efficient for large result sets. Install the optional extra:
+
+```sh
+pip install 'hotdata[arrow]'
+```
+
+Use `hotdata.arrow.ResultsApi` (a drop-in subclass of `ResultsApi` that adds Arrow methods):
+
+```python
+from hotdata import ApiClient, Configuration
+from hotdata.arrow import ResultsApi
+
+with ApiClient(Configuration(api_key="...", workspace_id="...")) as client:
+    results = ResultsApi(client)
+
+    # Results are scoped to a database via the required `X-Database-Id` header,
+    # so pass the id of the database the query ran in.
+    database_id = "your_database_id"
+
+    # Buffered: returns a pyarrow.Table.
+    table = results.get_result_arrow(result_id, database_id)
+
+    # Streaming: yields a pyarrow.RecordBatchStreamReader without
+    # materializing the full table in memory.
+    with results.stream_result_arrow(result_id, database_id) as reader:
+        for batch in reader:
+            ...
+```
+
+Both methods accept `offset` and `limit` for pagination. They raise `hotdata.arrow.ResultNotReadyError` if the result is still pending or processing — poll `results.get_result(result_id, database_id)` until `status == "ready"` first.
+
+## File uploads
+
+`hotdata.uploads.UploadsApi` (also the default `hotdata.UploadsApi`) adds
+`upload_file`, which uploads a local file **directly to object storage** and
+finalizes it in one call. It opens an upload session, `PUT`s the bytes straight
+to storage — a single `PUT` for a small file, concurrent part `PUT`s for a large
+one — then finalizes. The bytes never round-trip through the API.
+
+```python
+from hotdata import ApiClient, Configuration, UploadsApi
+
+with ApiClient(Configuration(api_key="...", workspace_id="...")) as client:
+    uploads = UploadsApi(client)
+
+    finalized = uploads.upload_file(
+        "data.parquet",
+        content_type="application/parquet",
+        progress=lambda done, total: print(f"{done}/{total} bytes"),
+    )
+
+    # Pass finalized.upload_id to the managed-table load endpoint.
+    print(finalized.upload_id)
+```
+
+`upload_file` accepts a path, raw `bytes`, or a seekable binary file object
+(`size` is inferred for all three; a file object is read from its current
+position to the end). The SDK picks single vs. multipart from the size,
+auto-scales the part size, and bounds part concurrency to a peak-memory budget
+(override with `part_size` / `max_concurrency` / `part_retry`). A file larger than
+8 MiB uploads via a **streaming** session — the SDK mints each part's URL just
+before its `PUT`, so a presigned URL can't expire mid-transfer on a slow upload.
+Storage `PUT`s go through a dedicated, header-isolated connection pool (auth and
+workspace headers never reach object storage, which would otherwise reject the
+upload) with a 30s connect timeout so a dead endpoint fails fast. Finalize is sent
+with retries disabled so the exactly-once call is never accidentally replayed.
+
+Every failure is a subclass of `hotdata.uploads.UploadError` (also importable as
+`from hotdata import UploadError`), so a single `except UploadError` catches the
+whole flow: `SessionCreateError` (opening the session — check `.status` for a
+`501` `PRESIGN_UNSUPPORTED`), `StorageError` (storage returned a non-2xx; `.exhausted`
+is `True` if it outlived every retry round), `StorageTransportError` (the PUT
+failed before any response), `MissingETagError`, `MintPartError` (minting a part
+URL), `FinalizeError`, `MalformedSessionError`, `SizeLimitError`, and
+`UploadCancelledError`. The phase errors that wrap a control-plane call chain the
+underlying `hotdata.exceptions.ApiException` as `__cause__` (and expose it as
+`.api_exception` / `.status`). A local file read error surfaces as `OSError`.
+
+The `progress` callback receives a **cumulative** `(bytes_done, total)` — for a
+tqdm bar (whose `update(n)` wants a delta, and which isn't thread-safe under
+multipart) use the ready-made adapter:
+
+```python
+from hotdata import tqdm_progress
+from tqdm import tqdm
+
+with tqdm(total=size, unit="B", unit_scale=True) as bar:
+    uploads.upload_file("data.parquet", progress=tqdm_progress(bar))
+```
+
+Pass a `threading.Event` as `cancel_event` to abort an in-flight upload; tune the
+control-plane calls with `request_timeout` (storage-PUT timeouts are automatic and
+size-scaled).
+
+For that fallback (or to upload from a non-seekable stream), use `upload_stream`,
+which sends the bytes to the legacy `POST /v1/files` endpoint in one request,
+streaming a file object without buffering it in memory:
+
+```python
+with open("data.parquet", "rb") as f:
+    resp = uploads.upload_stream(f, content_type="application/parquet")
+print(resp.id)
+```
+
+Note `upload_file` shadows the generated raw-body `upload_file(body=...)`; that
+raw operation is still reachable at
+`hotdata.api.uploads_api.UploadsApi.upload_file`.
+
+## API reference
+
+Generated Markdown for every operation and model is in [`docs/`](https://github.com/hotdata-dev/sdk-python/tree/main/docs):
+
+- Resource APIs: `docs/*Api.md` (for example [`QueryApi.md`](https://github.com/hotdata-dev/sdk-python/blob/main/docs/QueryApi.md))
+- Request and response models: `docs/<ModelName>.md`
+
+## Support
+
+Questions and issues: [github.com/hotdata-dev/sdk-python](https://github.com/hotdata-dev/sdk-python).