belfort-ml 0.2.0__tar.gz

belfort_ml-0.2.0/.gitignore

@@ -0,0 +1,11 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+.venv/
+build/
+dist/

belfort_ml-0.2.0/PKG-INFO

@@ -0,0 +1,9 @@
+Metadata-Version: 2.4
+Name: belfort-ml
+Version: 0.2.0
+Summary: Model-provider SDK for Belfort's encrypted Machine Learning platform
+Requires-Python: >=3.12
+Requires-Dist: belfort-ml-bucket==0.2.0
+Requires-Dist: belfort-torch-mlir==2026.10.9
+Requires-Dist: httpx==0.28.1
+Requires-Dist: torch==2.13.0

belfort_ml-0.2.0/README.md

@@ -0,0 +1,235 @@
+# Belfort ML Python Client
+
+`belfort_ml`: an httpx client for the Belfort CaaS gateway. A model provider
+uses it to compile a PyTorch model, register it, and run inference on it while
+it stays encrypted.
+
+## Install
+
+Not published. It is a `uv` workspace member, so run `uv sync` from the repo
+root.
+
+## Configuration
+
+```python
+import belfort_ml
+
+provider = belfort_ml.Provider()  # reads the environment
+provider = belfort_ml.Provider(api_key="blf_live_...")  # or pass it in
+```
+
+| Argument | Environment | Default |
+|---|---|---|
+| `api_key` | `BELFORT_API_KEY` | required |
+| `base_url` | `BELFORT_BASE_URL` | `https://eml.belfortlabs.com/api/v1` |
+| `timeouts` | — | `Timeouts()` |
+| `max_retries` | — | `3` |
+
+Explicit arguments beat the environment. A missing key raises
+`ConfigurationError` at construction. `Timeouts` is per operation class:
+`standard` 30s, `upload_part` 300s, `inference` 120s, `poll` 10s.
+
+## Quick start
+
+```python
+with belfort_ml.Provider() as provider:
+    account = provider.account.get()
+    print(account["mp_id"], account["credits"]["available"])
+```
+
+The provider holds a connection pool for its lifetime, so use it as a context
+manager or call `close()`.
+
+`provider.account` and `provider.credits` return plain dicts. `provider.models` and
+`provider.deploys` return typed objects — `Model`, `ClientKeySet`, `Deploy`,
+`LoadedKeys`, `InferenceResult` — that carry their own methods, so
+`model.deploy()` works without passing ids around.
+
+## The gateway surface
+
+| Method | Endpoint |
+|---|---|
+| `provider.account.get()` | `GET /account` |
+| `provider.credits.get()` | `GET /credits` |
+| `provider.credits.transactions(cursor=, limit=, kind=)` | `GET /credits/transactions` |
+| `provider.models.create(artifacts, name=)` | `POST /models`, S3 PUTs, `POST /models/{m_id}/compile`, then polls |
+| `provider.models.get(id)` / `.get(name=)` / `.list()` | `GET /models/{m_id}` / `GET /models` |
+| `model.refresh()` / `.delete()` | `GET .../status` / `DELETE /models/{m_id}` |
+| `model.fetch_artifacts(dir, bucket_url=)` | Reads the compiled artifacts from the bucket |
+| `model.create_client(label, ttl_seconds=)` | `POST /models/{m_id}/client-tokens`: the token one end client runs on |
+| `model.artifact_downloads()` | One presigned GET per artifact of the model's compile |
+| `model.clients(status=)` / `model.delete_client(c_id)` | `GET /models/{m_id}/clients`, `DELETE /models/{m_id}/clients/{c_id}` |
+| `model.deploy(key_set=, max_lifetime=, progress=)` | `POST /deploys` for the key set it serves, then polls and writes each launch step to stderr |
+| `provider.deploys.list()` / `.get(id)` / `.stop_all()` | `GET /deploys`, `GET /deploys/{d_id}`, `DELETE` each running one |
+| `deploy.refresh()` / `.extend(n)` / `.stop()` | `GET .../status` / `POST .../extend` / `DELETE /deploys/{d_id}` |
+| `deploy.load_keys()` / `.unload_keys(k)` | `POST .../keys` for the deploy's key set / `DELETE .../keys/{k_id}` |
+| `deploy.infer(loaded, bytes)` / `.infer_batch(...)` | `POST /deploys/{d_id}/infer` |
+
+Nothing calls `GET /jobs/{job_id}`. Every `wait=True` path polls the resource's
+own `/status` route: 2s backing off to 10s, then `belfort_ml.TimeoutError`. A ^C
+while a deploy starts prints the deploy id and that the instance is still
+billing, then re-raises.
+
+`deploy(...)` is a context manager. Exit stops the instance whether the block
+completed or raised, and a deploy someone else already stopped is not an error.
+
+`provider.credits.transactions` returns one page, newest first, keyset-paginated
+on `tx_id`; follow `next_cursor` yourself. The models and deploys lists follow
+cursors lazily.
+
+Uploads go straight to S3 through presigned multipart URLs, without the Belfort
+credential. The file is split into the presigned part count, with 4 parts in
+flight, each retried on its own and streamed from disk.
+
+## Compiling locally
+
+```python
+from belfort_ml._compile._export import load_calibration_data, load_pytorch_model
+
+model = load_pytorch_model("smallnet.pt")  # eval mode, CPU
+calibration = load_calibration_data(  # 512 examples by default
+    "calibration.npz", key="inputs", input_shape=(1, 3, 32, 32)
+)
+artifacts = belfort_ml.compile(
+    model, calibration[0], "smallnet", "./build", calibration_inputs=calibration
+)
+```
+
+`compile` writes `build/smallnet_torch.mlir` and returns the `Artifacts` that
+`provider.models.create` takes: the file and the parameters it declares. No
+network is involved.
+
+`name` becomes the generated C++ code's name, so it must be a C++ identifier:
+ASCII letters, digits and underscores, not starting with a digit, at most 200
+characters, with no `__` and no leading `_` before an uppercase letter. It may
+not be a C++ keyword, `std`, `heir`, `detail` or one of the native build's
+macros (`PERSEUS_MODEL`, `PERSEUS_HEADER`, `PERSEUS_SERVER`, `DEFAULT_STREAM`)
+or a standard-library macro (`NULL`, `EOF`, `errno`, `stdin`, `stdout`, `stderr`). Any other name raises `ValueError`
+before export.
+
+`example_input` fixes the shapes. The graph is specialized to it, so a deployed
+model serves that input shape only. A model the frontend cannot lower raises
+`CompileError` carrying the frontend's message.
+
+`compile` warns with `MultiCiphertextWarning` when a layer needs more slots
+than one ciphertext holds (32768), so the compiler must split its tensors
+across ciphertexts. The estimate is
+[`belfort_bucket.slots`](../bucket/belfort_bucket/slots.py), the one Perseus
+sizes the compile with. It is a lower bound: no warning does not prove that
+each tensor fits one ciphertext.
+
+Two annotations travel with the IR. Every input carries `{secret.secret}` unless
+the model marks a subset with `belfort_ml.Secret`, as
+[`examples/public-examples/mlp/mlp.py`](../examples/public-examples/mlp/mlp.py) does; and each op the compiler
+must approximate carries the value range the calibration examples put through
+it. Only those bounds reach the IR, never the examples.
+
+The end client is its own component, [`belfort-ml-client/`](../belfort-ml-client).
+
+## Reading the compiled artifacts
+
+Perseus writes under `{mp_id}/{m_id}/compiles/{compile_id}/`. The gateway
+reports a status and the chosen parameters; the files themselves are read from
+the bucket:
+
+```python
+compiled = model.fetch_artifacts(
+    "./artifacts", bucket_url="http://localhost:9000/belfort-caas-models"
+)
+compiled.compile_id  # the model's compile
+compiled.params  # what the compiler actually chose
+compiled.entry_function  # the exported function the C++ is named after
+compiled.files  # every downloaded file, tree preserved
+```
+
+| Argument | Environment | Default |
+|---|---|---|
+| `bucket_url` | `BELFORT_BUCKET_URL` | required |
+| `access_key_id` | `BELFORT_BUCKET_ACCESS_KEY_ID` | boto3's own chain |
+| `secret_access_key` | `BELFORT_BUCKET_SECRET_ACCESS_KEY` | boto3's own chain |
+| `region` | `BELFORT_BUCKET_REGION` | `us-east-1` |
+
+`bucket_url` names the endpoint and the bucket together, as
+[`bucket/`](../bucket) describes; a URL carrying a key prefix raises
+`ConfigurationError`.
+
+It reads the model's compile, `model.active_compile_id`. Pass `store=` a
+`belfort_bucket.Bucket` to reuse one reader across several models.
+
+## The end client
+
+`model.create_client(label)` opens a key set for one end client and returns a
+`ClientToken`. Hand its `token` to the client on your own channel; the client
+then talks to the gateway itself, with no bucket credential and no API key:
+
+```bash
+belfort-ml-client infer --token "$BELFORT_CLIENT_TOKEN" --input input.json
+```
+
+The client downloads and links its code, generates its key pair, uploads the
+evaluation keys, and waits for the gateway to load them onto the deploy started
+for its key set with `model.deploy(key_set=minted, max_lifetime=...)`. [`belfort-ml-client`](../belfort-ml-client) describes that side, and
+[`e2e/`](../tests/e2e) runs both.
+
+A key set travels as a directory, one object per evaluation key: `<n>.evk` and
+one `<n>.s2d`.
+
+## Errors
+
+Everything descends from `BelfortMLError`, and local failures separate from
+gateway failures.
+
+| Raised locally | When |
+|---|---|
+| `ConfigurationError` | No API key, or a `base_url` that is not http(s) |
+| `CompileError` | Local or gateway-side compile failure |
+| `UploadError` | An S3 part failed after retries |
+| `DownloadError` | Reading the bucket failed, or it holds no such compile |
+| `KeyMaterialError` | A key directory holds something that may not be published |
+| `TimeoutError` | Poll deadline exceeded |
+| `ConnectionError` | Gateway unreachable after retries; httpx's error rides along as `__cause__` |
+
+`APIError` covers anything the gateway returned, carrying `message`, `code`,
+`detail`, `status_code` and `request_id`. Quote `request_id` when you report a
+failure; it locates the request in the gateway logs.
+
+| Status | Class | Extra |
+|---|---|---|
+| 400, 413 | `ValidationError` | |
+| 401 | `AuthenticationError` | |
+| 402 | `InsufficientCredits` | `.required`, `.available` |
+| 403 | `PermissionError` | |
+| 404 | `NotFoundError` | |
+| 409 | `ConflictError` | `DeployLimitReached` for `deploy_limit_reached`, with `.limit`, `.running` |
+| 429 | `RateLimitError` | `.retry_after` |
+| 503 | `ServiceUnavailable` | |
+
+A response that is not the gateway's `{"error": {...}}` envelope — a proxy page,
+a crash — degrades to the raw text.
+
+## Retries, idempotency and logging
+
+Connection errors, timeouts, 429 and 500/502/504 are retried `max_retries`
+times, with exponential backoff and full jitter capped at 30s. A 429's
+`Retry-After` wins over the computed backoff. 4xx is never retried, and neither
+is inference. A read or write timeout can come after the gateway acted on the
+request, so after one only GET and HEAD are sent again.
+
+Idempotent requests carry one `Idempotency-Key` per logical operation, reused
+across that operation's retries. The gateway does not honour it yet.
+
+Logging goes to the standard `belfort_ml` logger and is silent without a
+handler. DEBUG lines carry method, path, status and request id only, never
+headers or bodies.
+
+## Examples and tests
+
+[`examples/`](./examples/) covers the account, credits and error handling.
+
+```bash
+cd gateway && docker compose up -d minio    # the artifact tests need it
+uv run pytest                                # from belfort-ml/
+```
+
+CI runs this package from
+[`.github/workflows/test-packages.yml`](../.github/workflows/test-packages.yml).

belfort_ml-0.2.0/belfort_ml/__init__.py

@@ -0,0 +1,76 @@
+from importlib.metadata import version
+
+__version__ = version("belfort-ml")
+
+from belfort_bucket import CompiledArtifacts
+
+from belfort_ml._compile import compile  # noqa: A004
+from belfort_ml._compile._annotations import Secret
+from belfort_ml.errors import (
+    APIError,
+    AuthenticationError,
+    BelfortMLError,
+    CompileError,
+    ConfigurationError,
+    ConflictError,
+    ConnectionError,
+    DeployLimitReached,
+    DownloadError,
+    InsufficientCredits,
+    KeyMaterialError,
+    MultiCiphertextWarning,
+    NotFoundError,
+    PermissionError,
+    RateLimitError,
+    ServiceUnavailable,
+    TimeoutError,
+    UploadError,
+    ValidationError,
+)
+from belfort_ml.gateway_connector.deploys import (
+    Deploy,
+    InferenceResult,
+    LoadedKeys,
+)
+from belfort_ml.gateway_connector.models import (
+    ClientKeySet,
+    ClientToken,
+    Model,
+)
+from belfort_ml.gateway_connector.provider import Provider
+from belfort_ml.types import Artifacts, CompileParams
+
+__all__ = [
+    "APIError",
+    "Artifacts",
+    "AuthenticationError",
+    "ClientKeySet",
+    "CompileError",
+    "CompileParams",
+    "CompiledArtifacts",
+    "Deploy",
+    "DeployLimitReached",
+    "ClientToken",
+    "InferenceResult",
+    "KeyMaterialError",
+    "LoadedKeys",
+    "Model",
+    "MultiCiphertextWarning",
+    "ConfigurationError",
+    "ConflictError",
+    "ConnectionError",
+    "DownloadError",
+    "InsufficientCredits",
+    "BelfortMLError",
+    "NotFoundError",
+    "PermissionError",
+    "Provider",
+    "RateLimitError",
+    "Secret",
+    "ServiceUnavailable",
+    "TimeoutError",
+    "UploadError",
+    "ValidationError",
+    "__version__",
+    "compile",
+]

belfort_ml-0.2.0/belfort_ml/_compile/__init__.py

@@ -0,0 +1,85 @@
+from __future__ import annotations
+
+import inspect
+import re
+import shutil
+from pathlib import Path
+from typing import Any
+
+from belfort_ml._compile._export import SampleInput, export_torch_mlir
+from belfort_ml.types import Artifacts
+
+# `name` names the C++ Perseus generates (`heir::generated::<name>`) and its
+# artifacts, so it must be a C++ identifier that is neither a keyword, a
+# namespace that code refers to, nor a macro the native build or the standard
+# headers define, and short enough that `<name>_cheddar.mlir` fits a 255-byte
+# file name.
+_NAME = re.compile(r"[A-Za-z_][A-Za-z0-9_]{0,199}")
+_RESERVED_NAMES = frozenset(
+    """
+    alignas alignof and and_eq asm auto bitand bitor bool break case catch char
+    char8_t char16_t char32_t class compl concept const consteval constexpr
+    constinit const_cast continue co_await co_return co_yield decltype default
+    delete do double dynamic_cast else enum explicit export extern false float
+    for friend goto if inline int long mutable namespace new noexcept not not_eq
+    nullptr operator or or_eq private protected public register reinterpret_cast
+    requires return short signed sizeof static static_assert static_cast struct
+    switch template this thread_local throw true try typedef typeid typename
+    union unsigned using virtual void volatile wchar_t while xor xor_eq
+    detail heir std
+    DEFAULT_STREAM PERSEUS_HEADER PERSEUS_MODEL PERSEUS_SERVER
+    EOF NULL errno stderr stdin stdout
+    """.split()
+)
+
+
+def _validate_name(name: str) -> None:
+    if not _NAME.fullmatch(name):
+        raise ValueError(
+            f"model name {name!r} may use only ASCII letters, digits and "
+            "underscores, may not start with a digit, and may be at most 200 "
+            "characters long"
+        )
+    # C++ reserves these for the implementation (`__FILE__`, `_Bool`).
+    if name in _RESERVED_NAMES or "__" in name or re.match(r"_[A-Z]", name):
+        raise ValueError(
+            f"model name {name!r} is reserved in the C++ the compiler generates"
+        )
+
+
+def compile(  # noqa: A001
+    model: Any,  # torch.nn.Module
+    example_input: SampleInput,
+    name: str,
+    output_dir: str | Path,
+    *,
+    calibration_inputs: list[SampleInput] | None = None,
+    allow_unsupported_ops: bool = False,
+) -> Artifacts:
+    _validate_name(name)
+    mlir_path = export_torch_mlir(
+        model,
+        example_input,
+        name,
+        output_dir,
+        calibration_inputs=calibration_inputs,
+        allow_unsupported_ops=allow_unsupported_ops,
+    )
+    return Artifacts(
+        mlir_path=mlir_path, source_path=_copy_source(model, name, output_dir)
+    )
+
+
+def _copy_source(model: Any, name: str, output_dir: str | Path) -> Path | None:
+    """Copy the file that defines the model's class to `<name>_model.py` next to
+    the MLIR, or return None when the class has no source file."""
+    try:
+        source = inspect.getsourcefile(type(model))
+    except TypeError:  # a built-in class
+        return None
+    if source is None or not Path(source).is_file():
+        return None
+    target = Path(output_dir) / f"{name}_model.py"
+    if Path(source).resolve() != target.resolve():
+        shutil.copyfile(source, target)
+    return target