sference-sdk 0.0.1__tar.gz

sference_sdk-0.0.1/.gitignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+dist-sdk/
+dist-cli/
+build/
+.venv/
+*.egg
+.pytest_cache/

sference_sdk-0.0.1/PKG-INFO

@@ -0,0 +1,186 @@
+Metadata-Version: 2.4
+Name: sference-sdk
+Version: 0.0.1
+Summary: Python SDK for the sference batch API
+Requires-Python: >=3.12
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: pydantic>=2.12.5
+Description-Content-Type: text/markdown
+
+# sference Python SDK
+
+Installable package: `sference-sdk` (import: `sference_sdk`). Used by the `sference` CLI and your own automation.
+
+## Install
+
+```bash
+pip install sference-sdk
+# or: uv add sference-sdk
+```
+
+## Usage
+
+Set `SFERENCE_API_KEY` and optional `SFERENCE_BASE_URL` (default `https://api.sference.com`), or pass `api_key=` / `base_url=` to the client.
+
+### Batches (sync)
+
+Best for a **fixed JSONL workload**: one submit, poll until terminal, then fetch structured results or download JSONL via the API.
+
+```python
+from sference_sdk import SferenceClient
+
+client = SferenceClient(api_key="sk_...", base_url="https://api.sference.com")
+
+batch = client.submit_batch(
+    input_file="./workload.jsonl",
+    model="Qwen/Qwen2.5-7B-Instruct",
+    window="24h",
+)
+done = client.wait_for_completion(batch.id, poll_interval=2.0, timeout=3600.0)
+results = client.get_results(done.id)
+print(results.status, results.output_url)
+```
+
+Use a `model` supported by your sference deployment.
+
+### OpenAI-compatible responses (sync)
+
+Standalone or stream-associated jobs via `POST /v1/responses`. Keys need `responses:read` and `responses:write` (default on newly issued keys).
+
+```python
+from sference_sdk import SferenceClient
+
+client = SferenceClient(api_key="sk_...", base_url="https://api.sference.com")
+
+created = client.create_response(
+    model="Qwen/Qwen2.5-7B-Instruct",
+    input=[{"role": "user", "content": "Hello"}],
+    metadata={"completion_window": "24h"},
+)
+row = client.get_response(created.id)
+```
+
+For a stream, add `stream_id` inside `metadata` next to `completion_window`.
+
+### OpenAI Python SDK (`openai` package)
+
+If you already use the official OpenAI client, point it at a sference-compatible **`/v1`** base URL and the same API key (with `responses:read` and `responses:write`).
+
+```bash
+pip install openai
+```
+
+```python
+import asyncio
+import os
+
+from openai import AsyncOpenAI
+
+
+async def main() -> None:
+    client = AsyncOpenAI(
+        base_url="https://api.sference.com/v1",
+        api_key=os.environ["SFERENCE_API_KEY"],
+    )
+
+    response = await client.responses.create(
+        model="zai-org/GLM-5",
+        input=[{"role": "user", "content": "Hello, world!"}],
+        background=True,
+    )
+    # Poll GET /v1/responses/{id} until terminal; your openai version may expose
+    # something like await client.responses.retrieve(response.id), or use
+    # AsyncSferenceClient.get_response(response.id) with the same host and key.
+
+
+asyncio.run(main())
+```
+
+**Self-hosted** (local API): use `base_url="http://127.0.0.1:8000/v1"` (or your `SFERENCE_BASE_URL` + `"/v1"`). **`model`** must match a model your inference workers consume.
+
+**Metadata:** to set `completion_window` or `stream_id` like the native SDK, pass them in the request body your `openai` version supports (for example `metadata=` on `create`, or `extra_body={"metadata": {...}}` if the helper does not list those fields yet).
+
+### Async client — batches
+
+`AsyncSferenceClient` uses `httpx.AsyncClient` so batch polling can run alongside other async I/O without blocking threads.
+
+**Use case:** You already know the full set of prompts (for example a JSONL file) and want one scheduled unit of work with a clear terminal state and bulk results.
+
+**Benefits:** Simple lifecycle (submit → wait → fetch results), fits large static workloads and JSONL-heavy pipelines.
+
+```python
+import asyncio
+
+from sference_sdk import AsyncSferenceClient
+
+
+async def main() -> None:
+    async with AsyncSferenceClient(api_key="sk_...", base_url="https://api.sference.com") as client:
+        batch = await client.submit_batch(
+            input_file="./workload.jsonl",
+            model="Qwen/Qwen2.5-7B-Instruct",
+            window="24h",
+        )
+        done = await client.wait_for_completion(batch.id, poll_interval=2.0, timeout=3600.0)
+        results = await client.get_results(done.id)
+        print(results.status, results.output_url)
+
+
+asyncio.run(main())
+```
+
+### Async client — streams
+
+Stream-associated jobs use `create_response(..., metadata={"stream_id": ..., "completion_window": "24h"})`. Consume completions with `list_stream_events` (optional `wait_ms` long-poll) or `iter_stream_events` (paged replay; optional checkpoints align with CLI `stream tail`).
+
+**Use case:** Work arrives over time, or you want one id to group many responses and observe completions as they land.
+
+**Benefits:** Independent submits with aggregated progress, stream-level status in the API/UI, and efficient event tailing.
+
+```python
+import asyncio
+
+from sference_sdk import AsyncSferenceClient
+
+
+async def main() -> None:
+    async with AsyncSferenceClient(api_key="sk_...", base_url="https://api.sference.com") as client:
+        stream = await client.create_stream(name="sdk-demo", window="24h")
+        await client.create_response(
+            model="Qwen/Qwen2.5-7B-Instruct",
+            input=[{"role": "user", "content": "Hello"}],
+            metadata={"stream_id": stream.id, "completion_window": "24h"},
+        )
+        async for ev in client.iter_stream_events(stream.id, checkpoint=False):
+            print(ev.completion_id, ev.status)
+
+
+asyncio.run(main())
+```
+
+## cURL (same API the SDK calls)
+
+API keys need `responses:read` and `responses:write` (default on newly issued keys). `X-API-Key` or `Authorization: Bearer sk_...` are both accepted.
+
+```bash
+export TOKEN=sk_...
+BASE_URL=https://api.sference.com
+
+RID=$(curl -sS -X POST "${BASE_URL}/v1/responses" \
+  -H "X-API-Key: $TOKEN" \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "Qwen/Qwen2.5-7B-Instruct",
+    "input": [{"role": "user", "content": "Hello"}],
+    "metadata": {"completion_window": "24h"}
+  }' | jq -r '.id')
+
+curl -sS "${BASE_URL}/v1/responses/${RID}" \
+  -H "X-API-Key: $TOKEN"
+```
+
+For self-hosted APIs, set `BASE_URL` to your API origin (no `/v1` suffix on `BASE_URL` here—the paths already include `/v1`). Without `jq`, read `id` from the POST JSON and substitute it in the GET URL.
+
+## CLI
+
+For `sference batch …` and `sference stream …` commands, see the [CLI README](../cli/README.md).

sference_sdk-0.0.1/README.md

@@ -0,0 +1,177 @@
+# sference Python SDK
+
+Installable package: `sference-sdk` (import: `sference_sdk`). Used by the `sference` CLI and your own automation.
+
+## Install
+
+```bash
+pip install sference-sdk
+# or: uv add sference-sdk
+```
+
+## Usage
+
+Set `SFERENCE_API_KEY` and optional `SFERENCE_BASE_URL` (default `https://api.sference.com`), or pass `api_key=` / `base_url=` to the client.
+
+### Batches (sync)
+
+Best for a **fixed JSONL workload**: one submit, poll until terminal, then fetch structured results or download JSONL via the API.
+
+```python
+from sference_sdk import SferenceClient
+
+client = SferenceClient(api_key="sk_...", base_url="https://api.sference.com")
+
+batch = client.submit_batch(
+    input_file="./workload.jsonl",
+    model="Qwen/Qwen2.5-7B-Instruct",
+    window="24h",
+)
+done = client.wait_for_completion(batch.id, poll_interval=2.0, timeout=3600.0)
+results = client.get_results(done.id)
+print(results.status, results.output_url)
+```
+
+Use a `model` supported by your sference deployment.
+
+### OpenAI-compatible responses (sync)
+
+Standalone or stream-associated jobs via `POST /v1/responses`. Keys need `responses:read` and `responses:write` (default on newly issued keys).
+
+```python
+from sference_sdk import SferenceClient
+
+client = SferenceClient(api_key="sk_...", base_url="https://api.sference.com")
+
+created = client.create_response(
+    model="Qwen/Qwen2.5-7B-Instruct",
+    input=[{"role": "user", "content": "Hello"}],
+    metadata={"completion_window": "24h"},
+)
+row = client.get_response(created.id)
+```
+
+For a stream, add `stream_id` inside `metadata` next to `completion_window`.
+
+### OpenAI Python SDK (`openai` package)
+
+If you already use the official OpenAI client, point it at a sference-compatible **`/v1`** base URL and the same API key (with `responses:read` and `responses:write`).
+
+```bash
+pip install openai
+```
+
+```python
+import asyncio
+import os
+
+from openai import AsyncOpenAI
+
+
+async def main() -> None:
+    client = AsyncOpenAI(
+        base_url="https://api.sference.com/v1",
+        api_key=os.environ["SFERENCE_API_KEY"],
+    )
+
+    response = await client.responses.create(
+        model="zai-org/GLM-5",
+        input=[{"role": "user", "content": "Hello, world!"}],
+        background=True,
+    )
+    # Poll GET /v1/responses/{id} until terminal; your openai version may expose
+    # something like await client.responses.retrieve(response.id), or use
+    # AsyncSferenceClient.get_response(response.id) with the same host and key.
+
+
+asyncio.run(main())
+```
+
+**Self-hosted** (local API): use `base_url="http://127.0.0.1:8000/v1"` (or your `SFERENCE_BASE_URL` + `"/v1"`). **`model`** must match a model your inference workers consume.
+
+**Metadata:** to set `completion_window` or `stream_id` like the native SDK, pass them in the request body your `openai` version supports (for example `metadata=` on `create`, or `extra_body={"metadata": {...}}` if the helper does not list those fields yet).
+
+### Async client — batches
+
+`AsyncSferenceClient` uses `httpx.AsyncClient` so batch polling can run alongside other async I/O without blocking threads.
+
+**Use case:** You already know the full set of prompts (for example a JSONL file) and want one scheduled unit of work with a clear terminal state and bulk results.
+
+**Benefits:** Simple lifecycle (submit → wait → fetch results), fits large static workloads and JSONL-heavy pipelines.
+
+```python
+import asyncio
+
+from sference_sdk import AsyncSferenceClient
+
+
+async def main() -> None:
+    async with AsyncSferenceClient(api_key="sk_...", base_url="https://api.sference.com") as client:
+        batch = await client.submit_batch(
+            input_file="./workload.jsonl",
+            model="Qwen/Qwen2.5-7B-Instruct",
+            window="24h",
+        )
+        done = await client.wait_for_completion(batch.id, poll_interval=2.0, timeout=3600.0)
+        results = await client.get_results(done.id)
+        print(results.status, results.output_url)
+
+
+asyncio.run(main())
+```
+
+### Async client — streams
+
+Stream-associated jobs use `create_response(..., metadata={"stream_id": ..., "completion_window": "24h"})`. Consume completions with `list_stream_events` (optional `wait_ms` long-poll) or `iter_stream_events` (paged replay; optional checkpoints align with CLI `stream tail`).
+
+**Use case:** Work arrives over time, or you want one id to group many responses and observe completions as they land.
+
+**Benefits:** Independent submits with aggregated progress, stream-level status in the API/UI, and efficient event tailing.
+
+```python
+import asyncio
+
+from sference_sdk import AsyncSferenceClient
+
+
+async def main() -> None:
+    async with AsyncSferenceClient(api_key="sk_...", base_url="https://api.sference.com") as client:
+        stream = await client.create_stream(name="sdk-demo", window="24h")
+        await client.create_response(
+            model="Qwen/Qwen2.5-7B-Instruct",
+            input=[{"role": "user", "content": "Hello"}],
+            metadata={"stream_id": stream.id, "completion_window": "24h"},
+        )
+        async for ev in client.iter_stream_events(stream.id, checkpoint=False):
+            print(ev.completion_id, ev.status)
+
+
+asyncio.run(main())
+```
+
+## cURL (same API the SDK calls)
+
+API keys need `responses:read` and `responses:write` (default on newly issued keys). `X-API-Key` or `Authorization: Bearer sk_...` are both accepted.
+
+```bash
+export TOKEN=sk_...
+BASE_URL=https://api.sference.com
+
+RID=$(curl -sS -X POST "${BASE_URL}/v1/responses" \
+  -H "X-API-Key: $TOKEN" \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "Qwen/Qwen2.5-7B-Instruct",
+    "input": [{"role": "user", "content": "Hello"}],
+    "metadata": {"completion_window": "24h"}
+  }' | jq -r '.id')
+
+curl -sS "${BASE_URL}/v1/responses/${RID}" \
+  -H "X-API-Key: $TOKEN"
+```
+
+For self-hosted APIs, set `BASE_URL` to your API origin (no `/v1` suffix on `BASE_URL` here—the paths already include `/v1`). Without `jq`, read `id` from the POST JSON and substitute it in the GET URL.
+
+## CLI
+
+For `sference batch …` and `sference stream …` commands, see the [CLI README](../cli/README.md).

sference_sdk-0.0.1/pyproject.toml

@@ -0,0 +1,20 @@
+[project]
+name = "sference-sdk"
+version = "0.0.1"
+description = "Python SDK for the sference batch API"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "httpx>=0.28.1",
+    "pydantic>=2.12.5",
+]
+
+[build-system]
+requires = ["hatchling>=1.29.0"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["sference_sdk"]
+
+[tool.hatch.build.targets.sdist]
+include = ["sference_sdk"]

sference_sdk-0.0.1/sference_sdk/__init__.py

@@ -0,0 +1,32 @@
+from .async_client import AsyncSferenceClient
+from .checkpoint import clear_checkpoint, load_checkpoint, save_checkpoint
+from .client import ApiError, SferenceClient
+from .models import (
+    Batch,
+    BatchList,
+    BatchResults,
+    InferenceRequest,
+    LoginResponse,
+    Stream,
+    StreamInferenceCompletionEvent,
+    StreamEventList,
+    StreamList,
+)
+
+__all__ = [
+    "ApiError",
+    "AsyncSferenceClient",
+    "SferenceClient",
+    "Batch",
+    "BatchList",
+    "BatchResults",
+    "LoginResponse",
+    "InferenceRequest",
+    "Stream",
+    "StreamInferenceCompletionEvent",
+    "StreamEventList",
+    "StreamList",
+    "clear_checkpoint",
+    "load_checkpoint",
+    "save_checkpoint",
+]

sference_sdk-0.0.1/sference_sdk/async_client.py

@@ -0,0 +1,295 @@
+from __future__ import annotations
+
+import asyncio
+import json
+import os
+import time
+import warnings
+from pathlib import Path
+from typing import Any, AsyncIterator, BinaryIO
+
+import httpx
+
+from .client import ApiError, SferenceClient
+from .checkpoint import clear_checkpoint, load_checkpoint, save_checkpoint
+from .models import (
+    Batch,
+    BatchCreatePayload,
+    BatchList,
+    BatchResults,
+    InferenceRequest,
+    LoginResponse,
+    Response,
+    ResponseList,
+    Stream,
+    StreamInferenceCompletionEvent,
+    StreamEventList,
+    StreamList,
+    StreamWindow,
+)
+
+
+class AsyncSferenceClient:
+    """Async HTTP client for the sference batch API (``httpx.AsyncClient``)."""
+
+    def __init__(
+        self,
+        base_url: str | None = None,
+        api_key: str | None = None,
+        transport: httpx.AsyncBaseTransport | None = None,
+    ) -> None:
+        self.base_url = base_url or os.getenv("SFERENCE_BASE_URL", "https://api.sference.com")
+        self._token = api_key or os.getenv("SFERENCE_API_KEY")
+        self._client = httpx.AsyncClient(base_url=self.base_url, timeout=30.0, transport=transport)
+
+    async def aclose(self) -> None:
+        await self._client.aclose()
+
+    async def __aenter__(self) -> AsyncSferenceClient:
+        return self
+
+    async def __aexit__(self, *_: Any) -> None:
+        await self.aclose()
+
+    def _headers(self) -> dict[str, str]:
+        headers = {"content-type": "application/json"}
+        if self._token:
+            headers["authorization"] = f"Bearer {self._token}"
+        return headers
+
+    async def _request(self, method: str, path: str, json_body: dict[str, Any] | None = None) -> Any:
+        response = await self._client.request(method, path, headers=self._headers(), json=json_body)
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+            except Exception:
+                payload = {"detail": response.text}
+            raise ApiError(f"{response.status_code}: {payload}")
+        return response.json()
+
+    async def _request_response(
+        self, method: str, path: str, json_body: dict[str, Any] | None = None
+    ) -> httpx.Response:
+        response = await self._client.request(method, path, headers=self._headers(), json=json_body)
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+            except Exception:
+                payload = {"detail": response.text}
+            raise ApiError(f"{response.status_code}: {payload}")
+        return response
+
+    async def login(self, username: str, password: str) -> LoginResponse:
+        payload = await self._request("POST", "/v1/auth/login", {"username": username, "password": password})
+        result = LoginResponse.model_validate(payload)
+        self._token = result.access_token
+        return result
+
+    async def get_me(self) -> dict[str, Any]:
+        return await self._request("GET", "/v1/auth/me")
+
+    async def submit_batch(
+        self,
+        *,
+        input_file: str | None = None,
+        requests: list[dict[str, Any]] | None = None,
+        model: str | None = None,
+        window: str = "24h",
+    ) -> Batch:
+        if window != "24h":
+            raise ValueError('Only window "24h" is supported in MVP.')
+        resolved = requests or []
+        if input_file:
+            resolved = SferenceClient.parse_inference_requests_jsonl(Path(input_file), model=model)
+        if not resolved:
+            raise ValueError("At least one request is required (use input_file or non-empty requests)")
+        if isinstance(resolved, list) and resolved and isinstance(resolved[0], dict):
+            reqs: list[InferenceRequest] = []
+            for r in resolved:
+                if not isinstance(r, dict):
+                    raise TypeError("requests must be a list of dicts or InferenceRequest objects")
+                body = r.get("body")
+                if isinstance(body, dict) and {"method", "url", "body"}.issubset(r.keys()):
+                    reqs.append(InferenceRequest(custom_id=r.get("custom_id"), body=body))
+                else:
+                    reqs.append(InferenceRequest.model_validate(r))
+            resolved = reqs
+        payload = BatchCreatePayload(window="24h", requests=resolved)  # type: ignore[arg-type]
+        response = await self._request("POST", "/v1/batches", payload.model_dump())
+        return Batch.model_validate(response)
+
+    async def get_batch(self, batch_id: str) -> Batch:
+        response = await self._request("GET", f"/v1/batches/{batch_id}")
+        return Batch.model_validate(response)
+
+    async def list_batches(self) -> BatchList:
+        response = await self._request("GET", "/v1/batches")
+        return BatchList.model_validate(response)
+
+    async def get_results(self, batch_id: str) -> BatchResults:
+        response = await self._request("GET", f"/v1/batches/{batch_id}/results")
+        return BatchResults.model_validate(response)
+
+    async def cancel_batch(self, batch_id: str) -> Batch:
+        response = await self._request("POST", f"/v1/batches/{batch_id}/cancel")
+        return Batch.model_validate(response)
+
+    async def download_results_jsonl(self, batch_id: str, out: str | Path | BinaryIO) -> None:
+        resp = await self._request_response("GET", f"/v1/batches/{batch_id}/results.jsonl")
+        if isinstance(out, (str, Path)):
+            Path(out).write_bytes(resp.content)
+            return
+        out.write(resp.content)
+
+    async def wait_for_completion(self, batch_id: str, poll_interval: float = 1.0, timeout: float = 30.0) -> Batch:
+        deadline = time.time() + timeout
+        while True:
+            batch = await self.get_batch(batch_id)
+            if batch.status in ("completed", "failed", "cancelled"):
+                return batch
+            if time.time() >= deadline:
+                raise TimeoutError(f"Timed out waiting for batch {batch_id}")
+            await asyncio.sleep(poll_interval)
+
+    async def create_stream(self, name: str, window: StreamWindow = "24h") -> Stream:
+        response = await self._request("POST", "/v1/streams", {"name": name, "window": window})
+        return Stream.model_validate(response)
+
+    async def list_streams(self) -> StreamList:
+        response = await self._request("GET", "/v1/streams")
+        return StreamList.model_validate(response)
+
+    async def get_stream(self, stream_id: str) -> Stream:
+        response = await self._request("GET", f"/v1/streams/{stream_id}")
+        return Stream.model_validate(response)
+
+    # Note: POST /v1/streams/{id}/items has been removed. Use create_response() with metadata.stream_id instead.
+
+    async def create_response(
+        self,
+        *,
+        model: str,
+        input: list[dict[str, str]],  # [{"role": "user", "content": "..."}]
+        instructions: str | None = None,
+        max_output_tokens: int | None = None,
+        temperature: float | None = None,
+        top_p: float | None = None,
+        metadata: dict[str, Any] | None = None,
+    ) -> Response:
+        """Create a standalone response or stream-associated response.
+
+        For stream-associated, include stream_id in metadata:
+            metadata={"stream_id": "uuid", "completion_window": "24h"}
+        """
+        payload: dict[str, Any] = {
+            "model": model,
+            "input": input,
+        }
+        if instructions is not None:
+            payload["instructions"] = instructions
+        if max_output_tokens is not None:
+            payload["max_output_tokens"] = max_output_tokens
+        if temperature is not None:
+            payload["temperature"] = temperature
+        if top_p is not None:
+            payload["top_p"] = top_p
+        if metadata is not None:
+            payload["metadata"] = metadata
+
+        response = await self._request("POST", "/v1/responses", payload)
+        return Response.model_validate(response)
+
+    async def get_response(self, response_id: str) -> Response:
+        """Get a response by ID."""
+        response = await self._request("GET", f"/v1/responses/{response_id}")
+        return Response.model_validate(response)
+
+    async def cancel_response(self, response_id: str) -> Response:
+        """Cancel a pending or running response."""
+        response = await self._request("DELETE", f"/v1/responses/{response_id}")
+        return Response.model_validate(response)
+
+    async def list_responses(
+        self,
+        *,
+        limit: int = 100,
+        stream_id: str | None = None,
+    ) -> ResponseList:
+        """List responses. Optionally filter by stream_id."""
+        params: dict[str, Any] = {"limit": limit}
+        if stream_id is not None:
+            params["stream_id"] = stream_id
+
+        response = await self._request("GET", "/v1/responses", params=params)
+        return ResponseList.model_validate(response)
+
+    async def archive_stream(self, stream_id: str) -> Stream:
+        response = await self._request("PATCH", f"/v1/streams/{stream_id}", {"status": "archived"})
+        return Stream.model_validate(response)
+
+    async def cancel_stream(self, stream_id: str) -> Stream:
+        response = await self._request("PATCH", f"/v1/streams/{stream_id}", {"status": "cancelled"})
+        return Stream.model_validate(response)
+
+    async def list_stream_events(
+        self,
+        stream_id: str,
+        *,
+        limit: int = 20,
+        starting_after: str | None = None,
+        ending_before: str | None = None,
+        wait_ms: int = 0,
+    ) -> StreamEventList:
+        params: dict[str, Any] = {"limit": limit, "wait_ms": wait_ms}
+        if starting_after is not None:
+            params["starting_after"] = starting_after
+        if ending_before is not None:
+            params["ending_before"] = ending_before
+        response = await self._client.request(
+            "GET",
+            f"/v1/streams/{stream_id}/events",
+            headers=self._headers(),
+            params=params,
+        )
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+            except Exception:
+                payload = {"detail": response.text}
+            raise ApiError(f"{response.status_code}: {payload}")
+        return StreamEventList.model_validate(response.json())
+
+    async def iter_stream_events(
+        self,
+        stream_id: str,
+        *,
+        consumer_name: str = "default",
+        starting_after: str | None = None,
+        checkpoint: bool = True,
+        from_latest: bool = False,
+    ) -> AsyncIterator[StreamInferenceCompletionEvent]:
+        if from_latest:
+            clear_checkpoint(self.base_url, stream_id, consumer_name)
+        cur: str | None = starting_after
+        if checkpoint and cur is None:
+            cur = load_checkpoint(self.base_url, stream_id, consumer_name)
+
+        if cur is None:
+            page = await self.list_stream_events(stream_id, limit=100, wait_ms=0)
+            for ev in reversed(page.data):
+                yield ev
+                if checkpoint:
+                    save_checkpoint(self.base_url, stream_id, consumer_name, ev.completion_id)
+            return
+
+        while True:
+            page = await self.list_stream_events(stream_id, limit=100, starting_after=cur, wait_ms=0)
+            if not page.data:
+                return
+            for ev in page.data:
+                yield ev
+                if checkpoint:
+                    save_checkpoint(self.base_url, stream_id, consumer_name, ev.completion_id)
+                cur = ev.completion_id
+            if not page.has_more:
+                return

sference_sdk-0.0.1/sference_sdk/checkpoint.py

@@ -0,0 +1,62 @@
+from __future__ import annotations
+
+import json
+import os
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+
+def _store_path() -> Path:
+    raw = os.environ.get("SFERENCE_STREAM_CHECKPOINTS")
+    if raw:
+        return Path(raw)
+    return Path.home() / ".sference" / "stream_checkpoints.json"
+
+
+def _checkpoint_key(base_url: str, stream_id: str, consumer_name: str) -> str:
+    return f"{base_url.rstrip('/')}:{stream_id}:{consumer_name}"
+
+
+def _load_all(path: Path) -> dict[str, Any]:
+    if not path.exists():
+        return {}
+    try:
+        data = json.loads(path.read_text(encoding="utf-8"))
+        return data if isinstance(data, dict) else {}
+    except (json.JSONDecodeError, OSError):
+        return {}
+
+
+def _write_all(path: Path, data: dict[str, Any]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+
+
+def load_checkpoint(base_url: str, stream_id: str, consumer_name: str) -> str | None:
+    path = _store_path()
+    key = _checkpoint_key(base_url, stream_id, consumer_name)
+    entry = _load_all(path).get(key)
+    if not isinstance(entry, dict):
+        return None
+    last = entry.get("last_completion_id") or entry.get("last_event_id")
+    return str(last) if last else None
+
+
+def save_checkpoint(base_url: str, stream_id: str, consumer_name: str, last_completion_id: str) -> None:
+    path = _store_path()
+    key = _checkpoint_key(base_url, stream_id, consumer_name)
+    data = _load_all(path)
+    data[key] = {
+        "last_completion_id": last_completion_id,
+        "updated_at": datetime.now(tz=timezone.utc).isoformat(),
+    }
+    _write_all(path, data)
+
+
+def clear_checkpoint(base_url: str, stream_id: str, consumer_name: str) -> None:
+    path = _store_path()
+    key = _checkpoint_key(base_url, stream_id, consumer_name)
+    data = _load_all(path)
+    data.pop(key, None)
+    _write_all(path, data)

sference_sdk-0.0.1/sference_sdk/client.py

@@ -0,0 +1,357 @@
+from __future__ import annotations
+
+import json
+import os
+import time
+import warnings
+from pathlib import Path
+from typing import Any, BinaryIO, Iterator
+
+import httpx
+
+from .checkpoint import clear_checkpoint, load_checkpoint, save_checkpoint
+from .models import (
+    Batch,
+    BatchCreatePayload,
+    BatchList,
+    BatchResults,
+    InferenceRequest,
+    LoginResponse,
+    Response,
+    ResponseList,
+    Stream,
+    StreamInferenceCompletionEvent,
+    StreamEventList,
+    StreamList,
+    StreamWindow,
+)
+
+
+class ApiError(Exception):
+    pass
+
+
+class SferenceClient:
+    def __init__(
+        self,
+        base_url: str | None = None,
+        api_key: str | None = None,
+        transport: httpx.BaseTransport | None = None,
+    ) -> None:
+        self.base_url = base_url or os.getenv("SFERENCE_BASE_URL", "https://api.sference.com")
+        self._token = api_key or os.getenv("SFERENCE_API_KEY")
+        self._client = httpx.Client(base_url=self.base_url, timeout=30.0, transport=transport)
+
+    def close(self) -> None:
+        self._client.close()
+
+    def __enter__(self) -> "SferenceClient":
+        return self
+
+    def __exit__(self, *_: Any) -> None:
+        self.close()
+
+    def _headers(self) -> dict[str, str]:
+        headers = {"content-type": "application/json"}
+        if self._token:
+            headers["authorization"] = f"Bearer {self._token}"
+        return headers
+
+    def _request(self, method: str, path: str, json_body: dict[str, Any] | None = None) -> Any:
+        response = self._client.request(method, path, headers=self._headers(), json=json_body)
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+            except Exception:
+                payload = {"detail": response.text}
+            raise ApiError(f"{response.status_code}: {payload}")
+        return response.json()
+
+    def _request_response(self, method: str, path: str, json_body: dict[str, Any] | None = None) -> httpx.Response:
+        response = self._client.request(method, path, headers=self._headers(), json=json_body)
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+            except Exception:
+                payload = {"detail": response.text}
+            raise ApiError(f"{response.status_code}: {payload}")
+        return response
+
+    def login(self, username: str, password: str) -> LoginResponse:
+        payload = self._request("POST", "/v1/auth/login", {"username": username, "password": password})
+        result = LoginResponse.model_validate(payload)
+        self._token = result.access_token
+        return result
+
+    def get_me(self) -> dict[str, Any]:
+        return self._request("GET", "/v1/auth/me")
+
+    def submit_batch(
+        self,
+        *,
+        input_file: str | None = None,
+        requests: list[dict[str, Any]] | None = None,
+        model: str | None = None,
+        window: str = "24h",
+    ) -> Batch:
+        if window != "24h":
+            raise ValueError('Only window "24h" is supported in MVP.')
+        resolved: list[InferenceRequest] | list[dict[str, Any]] = requests or []
+        if input_file:
+            resolved = self.parse_inference_requests_jsonl(Path(input_file), model=model)
+        if not resolved:
+            raise ValueError("At least one request is required (use input_file or non-empty requests)")
+        if isinstance(resolved, list) and resolved and isinstance(resolved[0], dict):
+            # Back-compat: accept OpenAI batch-style envelopes in `requests=[...]`.
+            reqs: list[InferenceRequest] = []
+            for r in resolved:
+                if not isinstance(r, dict):
+                    raise TypeError("requests must be a list of dicts or InferenceRequest objects")
+                body = r.get("body")
+                if isinstance(body, dict) and {"method", "url", "body"}.issubset(r.keys()):
+                    reqs.append(InferenceRequest(custom_id=r.get("custom_id"), body=body))
+                else:
+                    reqs.append(InferenceRequest.model_validate(r))
+            resolved = reqs
+        payload = BatchCreatePayload(window="24h", requests=resolved)  # type: ignore[arg-type]
+        response = self._request("POST", "/v1/batches", payload.model_dump())
+        return Batch.model_validate(response)
+
+    def get_batch(self, batch_id: str) -> Batch:
+        response = self._request("GET", f"/v1/batches/{batch_id}")
+        return Batch.model_validate(response)
+
+    def list_batches(self) -> BatchList:
+        response = self._request("GET", "/v1/batches")
+        return BatchList.model_validate(response)
+
+    def get_results(self, batch_id: str) -> BatchResults:
+        response = self._request("GET", f"/v1/batches/{batch_id}/results")
+        return BatchResults.model_validate(response)
+
+    def cancel_batch(self, batch_id: str) -> Batch:
+        response = self._request("POST", f"/v1/batches/{batch_id}/cancel")
+        return Batch.model_validate(response)
+
+    def download_results_jsonl(self, batch_id: str, out: str | Path | BinaryIO) -> None:
+        resp = self._request_response("GET", f"/v1/batches/{batch_id}/results.jsonl")
+        if isinstance(out, (str, Path)):
+            Path(out).write_bytes(resp.content)
+            return
+        out.write(resp.content)
+
+    def wait_for_completion(self, batch_id: str, poll_interval: float = 1.0, timeout: float = 30.0) -> Batch:
+        deadline = time.time() + timeout
+        while True:
+            batch = self.get_batch(batch_id)
+            if batch.status in ("completed", "failed", "cancelled"):
+                return batch
+            if time.time() >= deadline:
+                raise TimeoutError(f"Timed out waiting for batch {batch_id}")
+            time.sleep(poll_interval)
+
+    def create_stream(self, name: str, window: StreamWindow = "24h") -> Stream:
+        response = self._request("POST", "/v1/streams", {"name": name, "window": window})
+        return Stream.model_validate(response)
+
+    def list_streams(self) -> StreamList:
+        response = self._request("GET", "/v1/streams")
+        return StreamList.model_validate(response)
+
+    def get_stream(self, stream_id: str) -> Stream:
+        response = self._request("GET", f"/v1/streams/{stream_id}")
+        return Stream.model_validate(response)
+
+    # Note: POST /v1/streams/{id}/items has been removed. Use create_response() with metadata.stream_id instead.
+
+    def create_response(
+        self,
+        *,
+        model: str,
+        input: list[dict[str, str]],  # [{"role": "user", "content": "..."}]
+        instructions: str | None = None,
+        max_output_tokens: int | None = None,
+        temperature: float | None = None,
+        top_p: float | None = None,
+        metadata: dict[str, Any] | None = None,
+    ) -> Response:
+        """Create a standalone response or stream-associated response.
+
+        For stream-associated, include stream_id in metadata:
+            metadata={"stream_id": "uuid", "completion_window": "24h"}
+        """
+        from sference_sdk.models import Response, ResponseCreatePayload
+
+        payload: dict[str, Any] = {
+            "model": model,
+            "input": input,
+        }
+        if instructions is not None:
+            payload["instructions"] = instructions
+        if max_output_tokens is not None:
+            payload["max_output_tokens"] = max_output_tokens
+        if temperature is not None:
+            payload["temperature"] = temperature
+        if top_p is not None:
+            payload["top_p"] = top_p
+        if metadata is not None:
+            payload["metadata"] = metadata
+
+        response = self._request("POST", "/v1/responses", payload)
+        return Response.model_validate(response)
+
+    def get_response(self, response_id: str) -> Response:
+        """Get a response by ID."""
+        from sference_sdk.models import Response
+
+        response = self._request("GET", f"/v1/responses/{response_id}")
+        return Response.model_validate(response)
+
+    def cancel_response(self, response_id: str) -> Response:
+        """Cancel a pending or running response."""
+        from sference_sdk.models import Response
+
+        response = self._request("DELETE", f"/v1/responses/{response_id}")
+        return Response.model_validate(response)
+
+    def list_responses(
+        self,
+        *,
+        limit: int = 100,
+        stream_id: str | None = None,
+    ) -> ResponseList:
+        """List responses. Optionally filter by stream_id."""
+        from sference_sdk.models import ResponseList
+
+        params: dict[str, Any] = {"limit": limit}
+        if stream_id is not None:
+            params["stream_id"] = stream_id
+
+        response = self._request("GET", "/v1/responses", params=params)
+        return ResponseList.model_validate(response)
+
+    def archive_stream(self, stream_id: str) -> Stream:
+        response = self._request("PATCH", f"/v1/streams/{stream_id}", {"status": "archived"})
+        return Stream.model_validate(response)
+
+    def cancel_stream(self, stream_id: str) -> Stream:
+        response = self._request("PATCH", f"/v1/streams/{stream_id}", {"status": "cancelled"})
+        return Stream.model_validate(response)
+
+    def list_stream_events(
+        self,
+        stream_id: str,
+        *,
+        limit: int = 20,
+        starting_after: str | None = None,
+        ending_before: str | None = None,
+        wait_ms: int = 0,
+    ) -> StreamEventList:
+        params: dict[str, Any] = {"limit": limit, "wait_ms": wait_ms}
+        if starting_after is not None:
+            params["starting_after"] = starting_after
+        if ending_before is not None:
+            params["ending_before"] = ending_before
+        response = self._client.request(
+            "GET",
+            f"/v1/streams/{stream_id}/events",
+            headers=self._headers(),
+            params=params,
+        )
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+            except Exception:
+                payload = {"detail": response.text}
+            raise ApiError(f"{response.status_code}: {payload}")
+        return StreamEventList.model_validate(response.json())
+
+    def iter_stream_events(
+        self,
+        stream_id: str,
+        *,
+        consumer_name: str = "default",
+        starting_after: str | None = None,
+        checkpoint: bool = True,
+        from_latest: bool = False,
+    ) -> Iterator[StreamInferenceCompletionEvent]:
+        if from_latest:
+            clear_checkpoint(self.base_url, stream_id, consumer_name)
+        cur: str | None = starting_after
+        if checkpoint and cur is None:
+            cur = load_checkpoint(self.base_url, stream_id, consumer_name)
+
+        if cur is None:
+            page = self.list_stream_events(stream_id, limit=100, wait_ms=0)
+            for ev in reversed(page.data):
+                yield ev
+                if checkpoint:
+                    save_checkpoint(self.base_url, stream_id, consumer_name, ev.completion_id)
+            return
+
+        while True:
+            page = self.list_stream_events(stream_id, limit=100, starting_after=cur, wait_ms=0)
+            if not page.data:
+                return
+            for ev in page.data:
+                yield ev
+                if checkpoint:
+                    save_checkpoint(self.base_url, stream_id, consumer_name, ev.completion_id)
+                cur = ev.completion_id
+            if not page.has_more:
+                return
+
+    @staticmethod
+    def parse_inference_requests_jsonl(path: Path, model: str | None = None) -> list[InferenceRequest]:
+        parsed: list[InferenceRequest] = []
+        warn_model_ignored = False
+        with path.open("r", encoding="utf-8") as f:
+            for idx, line in enumerate(f, start=1):
+                line = line.strip()
+                if not line:
+                    continue
+                obj = json.loads(line)
+                # OpenAI batch-style envelope: {"custom_id", "method", "url", "body"}
+                if isinstance(obj, dict) and {"method", "url", "body"}.issubset(obj.keys()):
+                    if model:
+                        warn_model_ignored = True
+                    body = obj.get("body")
+                    if not isinstance(body, dict) or not body.get("model"):
+                        raise ValueError(f"Invalid JSONL line {idx}: body.model is required")
+                    parsed.append(
+                        InferenceRequest(
+                            custom_id=obj.get("custom_id"),
+                            body=body,
+                        )
+                    )
+                    continue
+                # Content-only format: {"content": "..."}
+                if isinstance(obj, dict) and "content" in obj:
+                    if not model:
+                        raise ValueError("model is required for content-only JSONL format")
+                    parsed.append(
+                        InferenceRequest(
+                            custom_id=obj.get("custom_id") or f"request-{idx}",
+                            body={
+                                "model": model,
+                                "messages": [{"role": "user", "content": obj["content"]}],
+                            },
+                        )
+                    )
+                    continue
+                raise ValueError(f"Unsupported JSONL line format at line {idx}")
+        if warn_model_ignored:
+            warnings.warn(
+                "model argument is ignored for OpenAI-compatible JSONL lines",
+                stacklevel=2,
+            )
+        return parsed
+
+    @staticmethod
+    def _parse_jsonl(path: Path, model: str | None = None) -> list[dict[str, Any]]:
+        """Backwards-compatible JSONL parser (returns plain dicts).
+
+        Prefer `parse_inference_requests_jsonl`, which returns `InferenceRequest` objects.
+        """
+
+        return [r.model_dump() for r in SferenceClient.parse_inference_requests_jsonl(path, model=model)]

sference_sdk-0.0.1/sference_sdk/models.py

@@ -0,0 +1,185 @@
+from __future__ import annotations
+
+from typing import Any, Literal
+
+from pydantic import BaseModel, ConfigDict, Field
+
+
+BatchStatus = Literal["pending", "running", "completed", "failed", "cancelled"]
+
+
+class InferenceRequest(BaseModel):
+    """Unified inference request schema for batches and streams.
+
+    custom_id is user-provided and can be used for correlation.
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    custom_id: str | None = Field(default=None, description="User-provided identifier for correlation.")
+    body: dict[str, Any] = Field(description="OpenAI-style request body. Must include `model`.")
+
+
+class LoginResponse(BaseModel):
+    access_token: str
+    token_type: str
+    user: dict[str, str]
+
+
+class Batch(BaseModel):
+    id: str
+    status: BatchStatus
+    window: Literal["24h"]
+    request_count: int
+    created_at: str
+    updated_at: str
+    completed_at: str | None = None
+    total_prompt_tokens: int = 0
+    total_completion_tokens: int = 0
+    total_tokens: int = 0
+
+
+class BatchList(BaseModel):
+    items: list[Batch]
+
+
+class BatchResults(BaseModel):
+    batch_id: str
+    status: BatchStatus
+    output_url: str | None = None
+    completed_at: str | None = None
+    results: list[dict[str, Any]] | None = None
+
+
+class BatchCreatePayload(BaseModel):
+    window: Literal["24h"] = "24h"
+    requests: list[InferenceRequest] = Field(min_length=1)
+
+
+StreamStatus = Literal["open", "cancelled", "archived"]
+StreamWindow = Literal["1h", "24h"]
+
+
+class Stream(BaseModel):
+    """Stream detail (GET /v1/streams/{id}); list responses omit counter fields (defaults apply)."""
+
+    id: str
+    name: str
+    window: StreamWindow
+    status: StreamStatus
+    created_at: str
+    updated_at: str
+    cancelled_at: str | None = None
+    archived_at: str | None = None
+    total_items: int = 0
+    pending_items: int = 0
+    running_items: int = 0
+    completed_items: int = 0
+    failed_items: int = 0
+    cancelled_items: int = 0
+    completion_ratio: float = 0.0
+    total_prompt_tokens: int = 0
+    total_completion_tokens: int = 0
+    total_tokens: int = 0
+    last_completion_at: str | None = None
+    latest_completion_id: str | None = None
+
+
+class StreamInferenceCompletionEvent(BaseModel):
+    completion_id: str
+    custom_id: str | None = None
+    status: str
+    result: dict[str, Any] | None = None
+    error: dict[str, Any] | None = None
+    prompt_tokens: int | None = None
+    completion_tokens: int | None = None
+    total_tokens: int | None = None
+    completed_at: str | None = None
+
+
+class StreamEventList(BaseModel):
+    object: str = "list"
+    data: list[StreamInferenceCompletionEvent]
+    has_more: bool
+
+
+class StreamList(BaseModel):
+    items: list[Stream]
+
+
+# Response (OpenAI-compatible) models
+ResponseStatus = Literal["in_progress", "completed", "failed", "cancelled"]
+
+
+class ResponseInputMessage(BaseModel):
+    """Message in the input array for a response."""
+
+    role: Literal["user", "assistant", "system", "developer"]
+    content: str
+
+
+class ResponseCreatePayload(BaseModel):
+    """Request body for POST /v1/responses."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    model: str = Field(description='Model identifier, e.g. "zai-org/GLM-5"')
+    input: list[ResponseInputMessage] = Field(min_length=1)
+    instructions: str | None = None
+    max_output_tokens: int | None = None
+    temperature: float | None = Field(default=None, ge=0.0, le=2.0)
+    top_p: float | None = Field(default=None, ge=0.0, le=1.0)
+    metadata: dict[str, Any] = Field(default_factory=dict)
+
+
+class ResponseOutputContent(BaseModel):
+    """Content item in a response output."""
+
+    type: Literal["output_text"] = "output_text"
+    text: str
+
+
+class ResponseUsage(BaseModel):
+    """Token usage for a response."""
+
+    input_tokens: int
+    output_tokens: int
+    total_tokens: int
+
+
+class ResponseError(BaseModel):
+    """Error information for a failed response."""
+
+    code: str
+    message: str
+
+
+class Response(BaseModel):
+    """OpenAI-compatible response object."""
+
+    id: str
+    object: Literal["response"] = "response"
+    created_at: int
+    model: str
+    status: ResponseStatus
+    output: list[ResponseOutputContent] | None = None
+    error: ResponseError | None = None
+    usage: ResponseUsage | None = None
+
+
+class ResponseListItem(BaseModel):
+    """Item in a response list."""
+
+    id: str
+    object: Literal["response"] = "response"
+    created_at: int
+    model: str
+    status: ResponseStatus
+
+
+class ResponseList(BaseModel):
+    """Paginated list of responses."""
+
+    object: Literal["list"] = "list"
+    data: list[ResponseListItem]
+    has_more: bool = False