datalab-python-sdk 0.5.0__py3-none-any.whl → 0.7.0__py3-none-any.whl

datalab_python_sdk-0.7.0.dist-info/METADATA

@@ -0,0 +1,159 @@
+Metadata-Version: 2.5
+Name: datalab-python-sdk
+Version: 0.7.0
+Summary: SDK for the Datalab document intelligence API
+Project-URL: Homepage, https://www.datalab.to
+Project-URL: Documentation, https://documentation.datalab.to
+Project-URL: Repository, https://github.com/datalab-to/sdk
+Author-email: Datalab Team <hi@datalab.to>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: api,datalab,document-intelligence,sdk
+Requires-Python: >=3.11
+Requires-Dist: aiohttp>=3.12.14
+Requires-Dist: click>=8.2.1
+Requires-Dist: ijson>=3.3.0
+Requires-Dist: pydantic-settings>=2.10.1
+Requires-Dist: pydantic>=2.11.7
+Requires-Dist: tenacity>=8.2.3
+Requires-Dist: tqdm>=4.66.0
+Description-Content-Type: text/markdown
+
+# Datalab SDK
+
+A Python SDK for the [Datalab API](https://www.datalab.to/?utm_source=gh-sdk) - it turns documents into verifiable knowledge your agents act on. Datalab is the team behind [Chandra](https://github.com/datalab-to/chandra), [marker](https://github.com/datalab-to/marker), [surya](https://github.com/datalab-to/surya), and [lift](https://github.com/datalab-to/lift).
+
+See the full documentation at [https://documentation.datalab.to](https://documentation.datalab.to).
+
+## Installation
+
+```bash
+pip install datalab-python-sdk
+```
+
+## Quick Start
+
+### Authentication
+
+[Sign up](https://www.datalab.to/?utm_source=gh-sdk), then get your API key from [https://www.datalab.to/app/keys](https://www.datalab.to/app/keys):
+
+```bash
+export DATALAB_API_KEY="your_api_key_here"
+```
+
+Or sign in from a terminal, which also works for coding agents:
+
+```bash
+datalab login
+```
+
+It shows a code and a link. Open the link, sign in or sign up, approve the code, and type the 6-digit code that we email you. The SDK saves the key to `~/.config/datalab/config.json`, readable only by your user, and uses it when `DATALAB_API_KEY` is not set. `datalab logout` deletes it.
+
+### Basic Usage
+
+```python
+from datalab_sdk import DatalabClient
+
+client = DatalabClient() # use env var from above, or pass api_key="your_api_key_here"
+
+# Convert PDF to markdown
+result = client.convert("document.pdf")
+print(result.markdown)
+```
+
+`AsyncDatalabClient` has the same methods as `DatalabClient`, as coroutines.
+
+## Conversion options
+
+```python
+from datalab_sdk import ConvertOptions
+
+options = ConvertOptions(
+    mode="balanced",               # lite, fast, balanced, accurate (SDK default: fast)
+    output_format="markdown,html", # markdown, html, json, chunks (comma-separated)
+    page_range="0-4",
+    extras="chart_understanding,table_cell_bboxes",
+    merge_cross_page=True,         # merge tables, paragraphs and lists split across pages
+    word_bboxes=True,              # per-word bounding boxes in the HTML output
+    save_checkpoint=True,          # reuse the parse in extract() or segment()
+)
+result = client.convert("document.pdf", options=options)
+```
+
+Other options include `paginate`, `add_block_ids`, `disable_html_prettify`, `exact_transcription`, `token_efficient_markdown`, `include_markdown_in_chunks`, `disable_image_extraction`, `disable_image_captions`, `fence_synthetic_captions`, `keep_spreadsheet_formatting`, `keep_pageheader_in_output`, `keep_pagefooter_in_output`, `additional_config`, `webhook_url`, `skip_cache` and `processing_location`. The [API reference](https://documentation.datalab.to/api-reference/convert-document) describes each option.
+
+## Structured extraction
+
+```python
+from datalab_sdk import ExtractOptions
+
+result = client.extract(
+    "invoice.pdf",
+    options=ExtractOptions(
+        page_schema='{"properties": {"total": {"type": "number"}}}',  # or schema_id="sch_..."
+        extraction_mode="balanced",  # turbo, fast, balanced, accurate
+    ),
+)
+print(result.extraction_schema_json, result.extraction_score_average)
+```
+
+When you do not set the parse `mode`, the API picks the parse mode that matches `extraction_mode`.
+
+## Other endpoints
+
+| Method | Endpoint |
+| --- | --- |
+| `run_agent()` | **Beta:** run an existing published processor (`AgentOptions(processor="cp_...")`) |
+| `segment()` | Split a document into sections (`SegmentOptions`) |
+| `fill()` | Fill a PDF or image form (`FormFillingOptions`) |
+| `form_fields()` | Find the fillable fields in a PDF or image |
+| `track_changes()` | Read tracked changes and comments from DOCX or PDF |
+| `create_document()` | Make a DOCX from markdown with track changes markup |
+| `generate_extraction_schemas()` | Suggest extraction schemas for a saved checkpoint |
+| `delete_result()` | Result deletion, by `result.request_id` |
+| `upload_files()`, `list_files()`, ... | File storage. Pass the returned `datalab://file-...` reference as `file_url` |
+| `create_extraction_schema()`, ... | Saved extraction schemas |
+| `create_pipeline()`, `run_pipeline()`, ... | [Pipelines](https://documentation.datalab.to/docs/welcome/sdk/pipelines) |
+
+`run_custom_processor()` and `ocr()` call deprecated API endpoints. Use `run_agent()` and `convert()` instead.
+
+Published-agent execution is in open beta; the API may change. Processor design and raw-prompt execution are not included.
+
+## Inputs, polling and errors
+
+- Send a local file with `file_path`, or a URL or a `datalab://file-...` reference with `file_url`. When you set `processing_location`, send the document as `file_url`.
+- Processing methods poll until the result is ready. Set `max_polls` and `poll_interval` to change the limit (default: 300 polls, 1 second apart). Use `stream_response_to` to write a large result straight to disk.
+- The SDK retries a submission that gets HTTP 429 or 503, and polls that get 408, 429, 5xx or a connection error. It waits for the `Retry-After` header when the API sends one.
+- API errors raise `DatalabAPIError` (with `status_code` and `response_data`) or one of its subclasses: `DatalabAuthenticationError` (401, 403), `DatalabPaymentRequiredError` (402), `DatalabNotFoundError` (404), `DatalabRateLimitError` (429) and `DatalabServerError` (5xx). Invalid arguments raise `DatalabValidationError`, which is also a `ValueError`.
+
+## Workflows
+
+The Datalab API removed the Workflows API. The workflow methods and CLI commands in this SDK are deprecated, and the API does not answer them. Use pipelines instead.
+
+## CLI Usage
+
+The SDK includes a command-line interface:
+
+```bash
+# Convert a document or a directory of documents
+datalab convert document.pdf --mode balanced --format markdown,html
+
+# Extract with a saved schema
+datalab extract invoice.pdf --schema_id sch_XXXXX --extraction_mode fast
+
+# Other commands
+datalab agent --help
+datalab segment --help
+datalab fill --help
+datalab form-fields --help
+datalab track-changes --help
+datalab create-document --help
+```
+
+## Contributing
+
+Issues and bug reports are welcome here. We develop the SDK in a private repository and mirror each release to this one, so we do not merge pull requests here directly. Open an issue, or open a pull request to show the change, and we will port it and credit you.
+
+## License
+
+MIT License

datalab_python_sdk-0.7.0.dist-info/RECORD

@@ -0,0 +1,13 @@
+datalab_sdk/__init__.py,sha256=BLAqVuqjxhdjKabkDUePLSmGeVMo0BnWfVbLDt8haP4,2397
+datalab_sdk/auth.py,sha256=Z7eaKSxhDFt2uUbSIlU8yuOC2myDWXl-rY9JLuThg4A,4278
+datalab_sdk/cli.py,sha256=icUy91brz0J8hzvC1G-pBtZPszc5oRd4H3sEpYhKr7Y,54186
+datalab_sdk/client.py,sha256=aNPW4_gUdyW7UbLHI3YRzPA_Ae2dQmlQwCV38qsw0hA,125702
+datalab_sdk/exceptions.py,sha256=5-lzlfvB7FKGOhvgyacceTFRRUxton_JqcIyuB8k_mU,2625
+datalab_sdk/mimetypes.py,sha256=a4ULJmHxoR_RMsD0j51hx0t2Q7In_PzNm7UlWX07eEE,932
+datalab_sdk/models.py,sha256=OpiIAdOVcL2tdfBkzRNHZ8MjGjRiBSjaKtFVf_U1l8w,32494
+datalab_sdk/settings.py,sha256=B_B42QQADAz6ePXQTOqENZllcppFR4JpwonEk2vlzhw,693
+datalab_python_sdk-0.7.0.dist-info/METADATA,sha256=z3hdjWxjWWjORewPlI58rO_stgNdHKUkwvf0Y4JemRg,6963
+datalab_python_sdk-0.7.0.dist-info/WHEEL,sha256=2iZrJkDIkeDh0k9SikKMqFuRP-ua6mb4Ir1AsER4BEw,87
+datalab_python_sdk-0.7.0.dist-info/entry_points.txt,sha256=rqWxO21UV57JxxyeF20iBTxHpzBYOA69hCHIqwdn-Ok,48
+datalab_python_sdk-0.7.0.dist-info/licenses/LICENSE,sha256=XGjT-Uz6t7UZzr3JCZfilsGcZEgMm_1aONcRunohxWg,1064
+datalab_python_sdk-0.7.0.dist-info/RECORD,,

{datalab_python_sdk-0.5.0.dist-info → datalab_python_sdk-0.7.0.dist-info}/WHEEL

@@ -1,4 +1,4 @@
 Wheel-Version: 1.0
-Generator: hatchling 1.29.0
+Generator: hatchling 1.33.0
 Root-Is-Purelib: true
 Tag: py3-none-any

datalab_sdk/__init__.py

@@ -6,7 +6,18 @@ supporting document conversion, extraction, segmentation, and more.
 """
 
 from .client import DatalabClient, AsyncDatalabClient
-from .exceptions import DatalabError, DatalabAPIError, DatalabTimeoutError
+from .exceptions import (
+    DatalabError,
+    DatalabAPIError,
+    DatalabAuthenticationError,
+    DatalabPaymentRequiredError,
+    DatalabNotFoundError,
+    DatalabRateLimitError,
+    DatalabServerError,
+    DatalabTimeoutError,
+    DatalabFileError,
+    DatalabValidationError,
+)
 from .models import (
     ConversionResult,
     CreateDocumentResult,
@@ -21,6 +32,11 @@ from .models import (
     OCROptions,
     FormFillingOptions,
     FormFillingResult,
+    FormFieldsOptions,
+    FormFieldsResult,
+    AgentOptions,
+    AgentResult,
+    ExtractionSchemaSuggestions,
     Workflow,
     WorkflowStep,
     WorkflowExecution,
@@ -43,7 +59,14 @@ __all__ = [
     "AsyncDatalabClient",
     "DatalabError",
     "DatalabAPIError",
+    "DatalabAuthenticationError",
+    "DatalabPaymentRequiredError",
+    "DatalabNotFoundError",
+    "DatalabRateLimitError",
+    "DatalabServerError",
     "DatalabTimeoutError",
+    "DatalabFileError",
+    "DatalabValidationError",
     "ConversionResult",
     "CreateDocumentResult",
     "FileResult",
@@ -57,6 +80,11 @@ __all__ = [
     "OCROptions",
     "FormFillingOptions",
     "FormFillingResult",
+    "FormFieldsOptions",
+    "FormFieldsResult",
+    "AgentOptions",
+    "AgentResult",
+    "ExtractionSchemaSuggestions",
     "Workflow",
     "WorkflowStep",
     "WorkflowExecution",

datalab_sdk/auth.py

@@ -0,0 +1,130 @@
+"""Sign in from a terminal with a device code (`datalab login`), and keep the API key on disk.
+
+The flow (server side: /api/auth/device/code, /app/device, /api/auth/device/token):
+1. Ask the server for a code. Show the human the code and a URL, and open the browser.
+2. The human approves in the browser, where they are signed in as usual.
+3. Poll until the server returns a new API key, then save it to the config file.
+
+The key is saved to $XDG_CONFIG_HOME/datalab/config.json (default ~/.config/datalab/config.json)
+with mode 0600. DATALAB_API_KEY in the environment always wins over the saved key.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import stat
+import time
+import urllib.error
+import urllib.request
+import webbrowser
+from pathlib import Path
+from typing import Callable, Optional
+
+from datalab_sdk.exceptions import DatalabAPIError
+
+KEYS_PAGE = "https://www.datalab.to/app/keys"
+
+
+def config_path() -> Path:
+    base = os.environ.get("XDG_CONFIG_HOME") or os.path.join(Path.home(), ".config")
+    return Path(base) / "datalab" / "config.json"
+
+
+def load_saved_api_key() -> Optional[str]:
+    try:
+        return json.loads(config_path().read_text()).get("api_key") or None
+    except (OSError, ValueError):
+        return None
+
+
+def save_api_key(api_key: str, host: str, team_name: Optional[str] = None) -> Path:
+    path = config_path()
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(
+        json.dumps({"api_key": api_key, "host": host, "team_name": team_name}, indent=2)
+    )
+    os.chmod(path, stat.S_IRUSR | stat.S_IWUSR)
+    return path
+
+
+def delete_saved_api_key() -> bool:
+    try:
+        config_path().unlink()
+        return True
+    except FileNotFoundError:
+        return False
+
+
+def _post(url: str, payload: dict, headers: dict) -> tuple[int, dict]:
+    req = urllib.request.Request(
+        url,
+        data=json.dumps(payload).encode(),
+        headers={
+            "Content-Type": "application/json",
+            "Accept": "application/json",
+            **headers,
+        },
+        method="POST",
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=30) as res:
+            return res.status, json.loads(res.read() or b"{}")
+    except urllib.error.HTTPError as e:
+        try:
+            return e.code, json.loads(e.read() or b"{}")
+        except ValueError:
+            return e.code, {}
+
+
+def device_login(
+    host: str = "https://www.datalab.to",
+    client_name: str = "Datalab CLI",
+    open_browser: bool = True,
+    extra_headers: Optional[dict] = None,
+    on_code: Optional[Callable[[dict], None]] = None,
+    sleep: Callable[[float], None] = time.sleep,
+) -> dict:
+    """Run the device flow and return {"api_key", "team_name", "key_name"}. Raises DatalabAPIError."""
+    host = host.rstrip("/")
+    headers = extra_headers or {}
+    status, code = _post(
+        f"{host}/api/auth/device/code", {"client_name": client_name}, headers
+    )
+    if status != 200:
+        raise DatalabAPIError(
+            f"Could not start the sign-in ({status}): {code.get('detail', code)}"
+        )
+    if on_code:
+        on_code(code)
+    if open_browser:
+        try:
+            webbrowser.open(code["verification_uri_complete"])
+        except Exception:
+            pass
+
+    interval = float(code.get("interval", 5))
+    deadline = time.monotonic() + float(code.get("expires_in", 600))
+    while time.monotonic() < deadline:
+        sleep(interval)
+        status, body = _post(
+            f"{host}/api/auth/device/token",
+            {"device_code": code["device_code"]},
+            headers,
+        )
+        if status == 200 and body.get("api_key"):
+            return body
+        error = body.get("error")
+        if error == "authorization_pending":
+            continue
+        if error == "slow_down":
+            interval += 5
+            continue
+        if error == "access_denied":
+            raise DatalabAPIError("The sign-in was denied in the browser.")
+        if error == "expired_token":
+            raise DatalabAPIError(
+                "The sign-in code expired. Run `datalab login` again."
+            )
+        raise DatalabAPIError(f"Sign-in failed ({status}): {body}")
+    raise DatalabAPIError("The sign-in code expired. Run `datalab login` again.")