pydantic-open-inference 1.0.0__tar.gz

pydantic_open_inference-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 1&1 Mail&Media
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pydantic_open_inference-1.0.0/PKG-INFO

@@ -0,0 +1,193 @@
+Metadata-Version: 2.4
+Name: pydantic-open-inference
+Version: 1.0.0
+Summary: Call an Open Inference server using pydantic models defining your model inputs and outputs.
+Keywords: pydantic,open inference,kserve,triton,machine learning,ml,inference
+Author: Anton Vikström
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Operating System :: OS Independent
+Classifier: Typing :: Typed
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: Intended Audience :: Other Audience
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Framework :: Pydantic
+Classifier: Framework :: Pydantic :: 2
+Classifier: Development Status :: 5 - Production/Stable
+Requires-Dist: httpx==0.28.*
+Requires-Dist: more-itertools==10.* ; python_full_version < '3.13'
+Requires-Dist: pydantic==2.*
+Requires-Python: >=3.10
+Project-URL: Repository, https://github.com/mam-dev/pydantic-open-inference
+Description-Content-Type: text/markdown
+
+# pydantic-open-inference
+
+[![image](https://img.shields.io/pypi/v/pydantic-open-inference
+)](https://pypi.python.org/pypi/pydantic-open-inference)
+[![image](https://img.shields.io/github/license/mam-dev/pydantic-open-inference
+)](https://github.com/mam-dev/pydantic-open-inference/blob/main/LICENSE)
+[![image](https://img.shields.io/pypi/pyversions/pydantic-open-inference)](https://pypi.python.org/pypi/pydantic-open-inference)
+[![Actions status](https://github.com/mam-dev/pydantic-open-inference/actions/workflows/ci.yaml/badge.svg)](https://github.com/mam-dev/pydantic-open-inference/actions)
+[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![Mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://www.mypy-lang.org)
+
+## What is it?
+Provides a client-side python interface for calling [open inference servers](https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol).
+Define [pydantic](https://docs.pydantic.dev/latest/) models for the model inputs and outputs, and it gets automatically
+converted to and from the open inference protocol, removing the need for boilerplate
+coding handling raw tensors and their shapes.
+
+## Features
+* ✅ Simple, type-safe API toward an open inference server using user-defined pydantic models
+* ✅ Automatic shape inference from pydantic models
+* ✅ Automatic datatype mapping (`int`→INT64, `float`→FP32, `str`→BYTES)
+* ✅ Datatype override for fine-grained control of model inputs
+* ✅ Automatically request only a subset of model outputs by omitting fields in your pydantic model
+* ✅ Take advantage of the full power of pydantic to validate and type-coerce your model calls
+* ✅ Multi-input/output models - each input/output maps to a pydantic field
+* ✅ Custom timeout configuration - configurable per model
+* ✅ Support calling inference for versioned and unversioned models
+* ✅ Error handling with custom exceptions - handle, e.g., specific error codes from the inference server differently
+* ✅ Fully typed - use, e.g., [mypy](https://mypy-lang.org/) and its [pydantic plugin](https://docs.pydantic.dev/latest/integrations/mypy/) to type check your code
+
+## Installation
+Installation using [pip](https://github.com/pypa/pip):
+```bash
+pip install pydantic-open-inference
+```
+
+Adding it to your project dependencies using [uv](https://docs.astral.sh/uv/):
+```bash
+uv add pydantic-open-inference
+```
+
+## How to use it
+Assume we have an inference server serving an Open Inference V2 REST API at `http://localhost:8080`
+and that the server contains a model named `example_model`, which has the following model **inputs**
+
+| name | shape | datatype | description              | example data       |
+|------|-------|----------|--------------------------|--------------------|
+| text | [1]   | BYTES    | A single string of text. | `["This is a text"]` |
+
+and the following model **outputs**:
+
+| name     | shape   | datatype | description                                                                                                                                                                                                                                  | example data                                                     |
+|----------|---------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------|
+| entities | [-1, 4] | BYTES    | <div style="width:200px">A list of entities extracted from the text. Each entity is made up of (in order) a label (either "order-id" or "invoice-id"), a confidence score, a start index for the entity in the text, and an end index.</div> | `[["order-id", "0.9", "23", "30"],["invoice-id","0.8","34","46"]]` |
+
+We then define a pydantic model for the inputs, inheriting from `InputsBaseModel`.
+Each model input name needs to correspond to a field in the pydantic model, and the type of the
+field needs to make sense given the shape and datatype of the input. In this case, there
+is just a single input named "text" that is supposed to hold a single text, so we simply define
+```python
+from pydantic_open_inference import InputsBaseModel
+
+class ExampleModelInputs(InputsBaseModel):
+    text: str
+
+```
+
+In the same way, we define a pydantic model for the outputs, inheriting from `OutputsBaseModel`.
+There is one model outputs named "entities", so that will be the only field we define.
+There can be any number of entities (corresponding to the -1 in the shape), so we define it to
+be a `list`. A list of what? Here we have some choices. We could simply define `entities: list[list[str]]`
+and get each entity as a list of strings, e.g., `["order-id", "0.98", "12", "24"]`, but this is not
+very useful to us, so we instead define, e.g., a _named tuple_ with ordered, named, and typed fields:
+```python
+from typing import NamedTuple
+from enum import StrEnum
+from pydantic_open_inference import OutputsBaseModel
+
+class Label(StrEnum):
+    ORDER_ID = "order-id"
+    INVOICE_ID = "invoice-id"
+
+class Entity(NamedTuple):
+    label: Label
+    score: float
+    start: int
+    end: int
+
+class ExampleModelOutputs(OutputsBaseModel):
+    entities: list[Entity]
+```
+Note that we could also have used `typing.Literal` instead of `enum.StrEnum`, and even added further validation,
+e.g., that the score is between 0.0 and 1.0, and that the start index is not negative etc. Pydantic offers many
+possibilities, but for now, let us stick with this.
+
+We now create a `RemoteModel` instance. The `RemoteModel` class represents a model in the inference server,
+in this case our "example_model". Instantiation requires the URL of the inference server, the name of the
+model, and the pydantic models we defined for the input and output:
+```python
+from pydantic_open_inference import RemoteModel
+
+INFERENCE_SERVER_URL = "http://localhost:8080"
+
+example_model = RemoteModel(
+    server_url=INFERENCE_SERVER_URL,
+    model_name="example_model",
+    inputs_model=ExampleModelInputs,
+    outputs_model=ExampleModelOutputs,
+)
+```
+
+In order to call the inference server, we simply call the `RemoteModel.infer` method, giving
+it an instance of `ExampleModelInputs`, and get back an instance of `ExampleModelOutputs`:
+```python
+example_input = ExampleModelInputs(text="Your order id is 123456.")
+example_output = example_model.infer(example_input)
+# ExampleModelOutputs(
+#    entities=[
+#        Entity(label=Label.ORDER_ID, score=0.95, start=17, end=23),
+#    ]
+# )
+```
+We have thus not only made a perfectly type-safe and validated call to the inference server,
+we have also gotten the response back in a useful format! We never had to bother with
+handling the shape of the data, parsing it in row-major order etc, as all of that is handled under the hood!
+
+The only time you need to think about "shape" is when defining the fields of the pydantic models.
+Use types like, e.g., `list`, `tuple`, and `typing.NamedTuple`, to type the pydantic-model fields
+so that they map sensibly to their corresponding model input/output. Each input/output name must
+match to a pydantic field and the shape must correspond to the structure of the field, e.g.,
+`list[tuple[int, int]]` for shape [-1, 2], or `tuple[tuple[float, float], tuple[float, float]]`
+for shape [2, 2], etc.
+
+The "datatype" of the model _inputs_ is automatically set based on the innermost type of the
+corresponding field in the `InputsBaseModel` subclass, according to:
+
+| python type | datatype |
+|-------------|----------|
+| bool        | BOOL     |
+| int         | INT64    |
+| float       | FP32     |
+| str         | BYTES    |
+
+However, you can always _override_ the datatype of the model inputs to whatever you want using
+`pydantic_open_inference.DatatypeOverride`; see its docstring for more information.
+The datatypes of the model _outputs_ are disregarded; all that matters is 
+that the data we got back from the model can be mapped to the defined `OutputsBaseModel` subclass.
+
+When an inference call to a model is made, it automatically requests only the specific outputs
+for which you have defined fields in your `OutputsBaseModel` subclass. So if you are not
+interested in an output, simply do not define a pydantic field for it.
+
+## Examples
+
+See the [examples directory](examples) for more examples.

pydantic_open_inference-1.0.0/README.md

@@ -0,0 +1,156 @@
+# pydantic-open-inference
+
+[![image](https://img.shields.io/pypi/v/pydantic-open-inference
+)](https://pypi.python.org/pypi/pydantic-open-inference)
+[![image](https://img.shields.io/github/license/mam-dev/pydantic-open-inference
+)](https://github.com/mam-dev/pydantic-open-inference/blob/main/LICENSE)
+[![image](https://img.shields.io/pypi/pyversions/pydantic-open-inference)](https://pypi.python.org/pypi/pydantic-open-inference)
+[![Actions status](https://github.com/mam-dev/pydantic-open-inference/actions/workflows/ci.yaml/badge.svg)](https://github.com/mam-dev/pydantic-open-inference/actions)
+[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![Mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://www.mypy-lang.org)
+
+## What is it?
+Provides a client-side python interface for calling [open inference servers](https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol).
+Define [pydantic](https://docs.pydantic.dev/latest/) models for the model inputs and outputs, and it gets automatically
+converted to and from the open inference protocol, removing the need for boilerplate
+coding handling raw tensors and their shapes.
+
+## Features
+* ✅ Simple, type-safe API toward an open inference server using user-defined pydantic models
+* ✅ Automatic shape inference from pydantic models
+* ✅ Automatic datatype mapping (`int`→INT64, `float`→FP32, `str`→BYTES)
+* ✅ Datatype override for fine-grained control of model inputs
+* ✅ Automatically request only a subset of model outputs by omitting fields in your pydantic model
+* ✅ Take advantage of the full power of pydantic to validate and type-coerce your model calls
+* ✅ Multi-input/output models - each input/output maps to a pydantic field
+* ✅ Custom timeout configuration - configurable per model
+* ✅ Support calling inference for versioned and unversioned models
+* ✅ Error handling with custom exceptions - handle, e.g., specific error codes from the inference server differently
+* ✅ Fully typed - use, e.g., [mypy](https://mypy-lang.org/) and its [pydantic plugin](https://docs.pydantic.dev/latest/integrations/mypy/) to type check your code
+
+## Installation
+Installation using [pip](https://github.com/pypa/pip):
+```bash
+pip install pydantic-open-inference
+```
+
+Adding it to your project dependencies using [uv](https://docs.astral.sh/uv/):
+```bash
+uv add pydantic-open-inference
+```
+
+## How to use it
+Assume we have an inference server serving an Open Inference V2 REST API at `http://localhost:8080`
+and that the server contains a model named `example_model`, which has the following model **inputs**
+
+| name | shape | datatype | description              | example data       |
+|------|-------|----------|--------------------------|--------------------|
+| text | [1]   | BYTES    | A single string of text. | `["This is a text"]` |
+
+and the following model **outputs**:
+
+| name     | shape   | datatype | description                                                                                                                                                                                                                                  | example data                                                     |
+|----------|---------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------|
+| entities | [-1, 4] | BYTES    | <div style="width:200px">A list of entities extracted from the text. Each entity is made up of (in order) a label (either "order-id" or "invoice-id"), a confidence score, a start index for the entity in the text, and an end index.</div> | `[["order-id", "0.9", "23", "30"],["invoice-id","0.8","34","46"]]` |
+
+We then define a pydantic model for the inputs, inheriting from `InputsBaseModel`.
+Each model input name needs to correspond to a field in the pydantic model, and the type of the
+field needs to make sense given the shape and datatype of the input. In this case, there
+is just a single input named "text" that is supposed to hold a single text, so we simply define
+```python
+from pydantic_open_inference import InputsBaseModel
+
+class ExampleModelInputs(InputsBaseModel):
+    text: str
+
+```
+
+In the same way, we define a pydantic model for the outputs, inheriting from `OutputsBaseModel`.
+There is one model outputs named "entities", so that will be the only field we define.
+There can be any number of entities (corresponding to the -1 in the shape), so we define it to
+be a `list`. A list of what? Here we have some choices. We could simply define `entities: list[list[str]]`
+and get each entity as a list of strings, e.g., `["order-id", "0.98", "12", "24"]`, but this is not
+very useful to us, so we instead define, e.g., a _named tuple_ with ordered, named, and typed fields:
+```python
+from typing import NamedTuple
+from enum import StrEnum
+from pydantic_open_inference import OutputsBaseModel
+
+class Label(StrEnum):
+    ORDER_ID = "order-id"
+    INVOICE_ID = "invoice-id"
+
+class Entity(NamedTuple):
+    label: Label
+    score: float
+    start: int
+    end: int
+
+class ExampleModelOutputs(OutputsBaseModel):
+    entities: list[Entity]
+```
+Note that we could also have used `typing.Literal` instead of `enum.StrEnum`, and even added further validation,
+e.g., that the score is between 0.0 and 1.0, and that the start index is not negative etc. Pydantic offers many
+possibilities, but for now, let us stick with this.
+
+We now create a `RemoteModel` instance. The `RemoteModel` class represents a model in the inference server,
+in this case our "example_model". Instantiation requires the URL of the inference server, the name of the
+model, and the pydantic models we defined for the input and output:
+```python
+from pydantic_open_inference import RemoteModel
+
+INFERENCE_SERVER_URL = "http://localhost:8080"
+
+example_model = RemoteModel(
+    server_url=INFERENCE_SERVER_URL,
+    model_name="example_model",
+    inputs_model=ExampleModelInputs,
+    outputs_model=ExampleModelOutputs,
+)
+```
+
+In order to call the inference server, we simply call the `RemoteModel.infer` method, giving
+it an instance of `ExampleModelInputs`, and get back an instance of `ExampleModelOutputs`:
+```python
+example_input = ExampleModelInputs(text="Your order id is 123456.")
+example_output = example_model.infer(example_input)
+# ExampleModelOutputs(
+#    entities=[
+#        Entity(label=Label.ORDER_ID, score=0.95, start=17, end=23),
+#    ]
+# )
+```
+We have thus not only made a perfectly type-safe and validated call to the inference server,
+we have also gotten the response back in a useful format! We never had to bother with
+handling the shape of the data, parsing it in row-major order etc, as all of that is handled under the hood!
+
+The only time you need to think about "shape" is when defining the fields of the pydantic models.
+Use types like, e.g., `list`, `tuple`, and `typing.NamedTuple`, to type the pydantic-model fields
+so that they map sensibly to their corresponding model input/output. Each input/output name must
+match to a pydantic field and the shape must correspond to the structure of the field, e.g.,
+`list[tuple[int, int]]` for shape [-1, 2], or `tuple[tuple[float, float], tuple[float, float]]`
+for shape [2, 2], etc.
+
+The "datatype" of the model _inputs_ is automatically set based on the innermost type of the
+corresponding field in the `InputsBaseModel` subclass, according to:
+
+| python type | datatype |
+|-------------|----------|
+| bool        | BOOL     |
+| int         | INT64    |
+| float       | FP32     |
+| str         | BYTES    |
+
+However, you can always _override_ the datatype of the model inputs to whatever you want using
+`pydantic_open_inference.DatatypeOverride`; see its docstring for more information.
+The datatypes of the model _outputs_ are disregarded; all that matters is 
+that the data we got back from the model can be mapped to the defined `OutputsBaseModel` subclass.
+
+When an inference call to a model is made, it automatically requests only the specific outputs
+for which you have defined fields in your `OutputsBaseModel` subclass. So if you are not
+interested in an output, simply do not define a pydantic field for it.
+
+## Examples
+
+See the [examples directory](examples) for more examples.

pydantic_open_inference-1.0.0/pyproject.toml

@@ -0,0 +1,197 @@
+[project]
+name = "pydantic-open-inference"
+version = "1.0.0"
+description = "Call an Open Inference server using pydantic models defining your model inputs and outputs."
+authors = [{ name = "Anton Vikström" }]
+license = "MIT"
+license-files = ["LICENSE"]
+readme = "README.md"
+requires-python = ">=3.10"
+keywords = [
+  "pydantic",
+  "open inference",
+  "kserve",
+  "triton",
+  "machine learning",
+  "ml",
+  "inference",
+]
+classifiers = [
+  "Natural Language :: English",
+  "Programming Language :: Python",
+  "Programming Language :: Python :: 3 :: Only",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Programming Language :: Python :: 3.14",
+  "Operating System :: OS Independent",
+  "Typing :: Typed",
+  "Intended Audience :: Science/Research",
+  "Intended Audience :: Developers",
+  "Intended Audience :: System Administrators",
+  "Intended Audience :: Other Audience",
+  "Topic :: Software Development :: Libraries",
+  "Topic :: Software Development :: Libraries :: Python Modules",
+  "Topic :: Scientific/Engineering",
+  "Topic :: Scientific/Engineering :: Artificial Intelligence",
+  "Framework :: Pydantic",
+  "Framework :: Pydantic :: 2",
+  "Development Status :: 5 - Production/Stable",
+]
+dependencies = [
+  "httpx==0.28.*",
+  "more-itertools==10.* ; python_full_version < '3.13'",
+  "pydantic==2.*",
+]
+
+[dependency-groups]
+dev = [
+  "coverage>=7.0.0",
+  "mypy>=1.18.2",
+  "pytest>=9.0.0",
+  "pytest-cov>=7.0.0",
+  "pytest-integration>=0.2.3",
+  "pytest-random-order>=1.2.0",
+  "ruff>=0.14.4",
+  "typing-extensions>=4.15.0 ; python_full_version < '3.12'",
+]
+
+[build-system]
+requires = ["uv_build>=0.9.8,<0.10.0"]
+build-backend = "uv_build"
+
+[project.urls]
+Repository = "https://github.com/mam-dev/pydantic-open-inference"
+
+[tool.uv]
+cache-keys = [{ file = "pyproject.toml" }, { git = true }]
+
+[[tool.uv.index]]
+url = "https://pypi.python.org/simple"
+default = true
+
+[tool.coverage.run]
+branch = true
+source = ["src", "examples"]
+
+[tool.coverage.report]
+show_missing = true
+fail_under = 100
+
+[tool.pytest.ini_options]
+addopts = ["--cov", "--random-order"]
+testpaths = ["tests"]
+
+[tool.mypy]
+files = ["src", "tests", "examples"]
+plugins = ["pydantic.mypy"]
+pretty = true
+warn_no_return = true
+warn_return_any = true
+warn_unused_configs = true
+warn_unused_ignores = true
+warn_redundant_casts = true
+warn_unreachable = true
+check_untyped_defs = true
+disallow_any_generics = true
+disallow_subclassing_any = true
+disallow_untyped_calls = true
+disallow_untyped_defs = true
+disallow_incomplete_defs = true
+disallow_untyped_decorators = true
+no_implicit_optional = true
+no_implicit_reexport = true
+strict_equality = true
+extra_checks = true
+enable_error_code = """
+explicit-override,\
+deprecated,\
+truthy-iterable,\
+ignore-without-code,\
+possibly-undefined,\
+redundant-expr,\
+truthy-bool"""
+
+[tool.pydantic-mypy]
+init_forbid_extra = true
+init_typed = true
+warn_required_dynamic_aliases = true
+warn_untyped_fields = true
+
+[tool.ruff]
+line-length = 120
+extend-include = ["tests/**/.py", "examples/**/*.py"]
+
+[tool.ruff.lint]
+select = [
+  "E",   # pycodestyle errors
+  "W",   # pycodestyle warnings
+  "F",   # pyflakes
+  "UP",  # pyupgrade
+  "S",   # flake8-bandit
+  "D",   # pydocstyle
+  "PT",  # flake8-pytest-style
+  "I",   # isort
+  "RUF", # Ruff-specific rules
+  "PTH", # flake8-use-pathlib
+  "ERA", # eradicate
+  "PL",  # pylint
+  "FBT", # flake8-boolean-trap
+  "B",   # flake8-bugbear
+  "A",   # flake8-builtins
+  "ISC", # flake8-implicit-str-concat
+  "INP", # flake8-no-pep420
+  "SLF", # flake8-self
+  "SIM", # flake8-simplify
+  "TID", # flake8-tidy-imports
+  "ARG", # flake8-unused-arguments
+  "TRY", # tryceratops
+  "FLY",  # flynt
+  "RSE",  # flake8-raise
+  "RET",  # flake8-return
+  "FIX",  # flake8-fixme
+  "Q",    # flake8-quotes
+  "C4",   # flake8-comprehensions
+  "DTZ",  # flake8-datetimez
+  "T10",  # flake8-debugger
+  "T20",  # flake8-print
+  "TCH",  # flake8-type-checking
+  "ASYNC",  # flake8-async
+  "BLE",  # flake8-blind-except
+  "LOG",  # flake8-logging
+  "SLOT",  # flake8-slots
+  "NPY",  # Numpy-specific rules
+  "PD",  # pandas-vet
+  "PERF",  # Perflint
+]
+
+[tool.ruff.lint.per-file-ignores]
+"tests/**/*.py" = [
+  "D101",  # undocumented public class
+  "D103",  # undocumented public function
+  "FBT001",  # boolean positional arg
+  "SLF001",  # private member access
+  "PLR2004",  # magic-value compare
+  "PLR0913",  # too many args
+  "ARG001",  # unused arg
+  "S101",  # use of 'assert'
+]
+"examples/**/*.py" = [
+  "T201",  # print statement
+  "INP001",  # implicit namespace
+  "ERA001",  # commented-out code
+]
+
+[tool.ruff.lint.pydocstyle]
+convention = "google"
+
+[tool.ruff.lint.flake8-pytest-style]
+fixture-parentheses = false
+mark-parentheses = false
+parametrize-names-type = "csv"
+parametrize-values-type = "list"
+
+[tool.ruff.lint.flake8-tidy-imports]
+ban-relative-imports = "parents"

pydantic_open_inference-1.0.0/src/pydantic_open_inference/__init__.py

@@ -0,0 +1,14 @@
+"""Package for wrapping calls to an Open Inference server REST API."""
+
+from ._client import BadStatusCodeFromServerError
+from ._remote_model import InputsBaseModel, OutputsBaseModel, RemoteModel
+from ._utils import DatatypeOverride, PydanticOpenInferenceError
+
+__all__ = (
+    "BadStatusCodeFromServerError",
+    "DatatypeOverride",
+    "InputsBaseModel",
+    "OutputsBaseModel",
+    "PydanticOpenInferenceError",
+    "RemoteModel",
+)

pydantic_open_inference-1.0.0/src/pydantic_open_inference/_client.py

@@ -0,0 +1,72 @@
+from __future__ import annotations
+
+import atexit
+import json
+from typing import cast
+
+import httpx
+
+from ._utils import (
+    OpenInferenceAPIInput,
+    OpenInferenceAPIOutput,
+    OpenInferenceAPIRequestedOutput,
+    PydanticOpenInferenceError,
+    Singleton,
+)
+
+
+class OpenInferenceHTTPClientAPIError(PydanticOpenInferenceError):
+    """Exceptions from the OpenInferenceHTTPClientAPI."""
+
+
+class BadStatusCodeFromServerError(OpenInferenceHTTPClientAPIError):
+    """A bad status code was returned from the open inference server."""
+
+    def __init__(self, status_code: int) -> None:
+        self._status_code = status_code
+        message = f"Inference server responded with HTTP {status_code}"
+        super().__init__(message)
+
+    @property
+    def status_code(self) -> int:
+        return self._status_code
+
+
+class OpenInferenceHTTPClientAPI(metaclass=Singleton):
+    def __init__(self, base_url: str) -> None:
+        if not base_url.endswith("/"):
+            base_url += "/"
+        self._client = httpx.Client(
+            base_url=base_url,
+            headers={"Content-Type": "application/json"},
+        )
+        atexit.register(self._client.close)
+
+    def infer(
+        self,
+        model_name: str,
+        inputs: list[OpenInferenceAPIInput],
+        outputs: list[OpenInferenceAPIRequestedOutput] | None = None,
+        model_version: str | None = None,
+        timeout_seconds: float | None = None,
+    ) -> list[OpenInferenceAPIOutput]:
+        try:
+            payload: dict[str, list[OpenInferenceAPIInput] | list[OpenInferenceAPIRequestedOutput]] = {"inputs": inputs}
+            if outputs is not None:
+                payload["outputs"] = outputs
+            if model_version is None:
+                url = f"v2/models/{model_name}/infer"
+            else:
+                url = f"v2/models/{model_name}/versions/{model_version}/infer"
+            response = self._client.post(
+                url,
+                json=payload,
+                timeout=timeout_seconds,
+            ).raise_for_status()
+            return cast("list[OpenInferenceAPIOutput]", response.json().get("outputs", []))
+        except httpx.HTTPStatusError as error:
+            raise BadStatusCodeFromServerError(
+                status_code=error.response.status_code,
+            ) from error
+        except (httpx.HTTPError, json.JSONDecodeError) as error:
+            raise OpenInferenceHTTPClientAPIError(error) from error

pydantic_open_inference-1.0.0/src/pydantic_open_inference/_remote_model.py

@@ -0,0 +1,256 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, Generic, TypeVar
+
+if TYPE_CHECKING:
+    import sys
+
+    if sys.version_info < (3, 11):
+        from typing_extensions import Self
+    else:
+        from typing import Self
+
+import pydantic
+
+from ._client import OpenInferenceHTTPClientAPI
+from ._utils import (
+    OpenInferenceAPIInput,
+    OpenInferenceAPIOutput,
+    OpenInferenceAPIRequestedOutput,
+    get_data,
+    get_datatype,
+    get_shape,
+    is_listlike,
+    unflatten_data,
+)
+
+
+class InputsBaseModel(pydantic.BaseModel):
+    """Base class for model inputs.
+
+    Subclass this class to define the model inputs, e.g.,
+
+        class MyModelInput(InputsBaseModel):
+            values: list[int]
+
+    The field names MUST correspond to the names of the model inputs.
+
+    The structure of a field must be mappable to the shape of the
+    corresponding input, e.g.,
+    - "list[tuple[int, int]]" for a shape of [-1, 2],
+    - "tuple[float, float]" for  a shape of [2], or
+    - "list[tuple[tuple[int, int], tuple[int, int]]]" for a shape of [-1, 2, 2].
+    The actual shape in the input that is sent will have the actual size, not -1.
+
+    NOTE: You can define an input field that is neither a list nor a tuple, just
+    a simple datatype like int, float, str; this will correspond to a shape of [1].
+
+    You can use typing.Namedtuple instead of a tuple to get named fields.
+
+    Example:
+        class Point(typing.Namedtuple):
+            x: float
+            y: float
+
+        class ExampleInput(InputsBaseModel):
+            points: list[Point]  # name: "points", shape: [-1, 2]
+            matrix: tuple[tuple[int, int, int], tuple[int, int, int]]  # name: "matrix", shape: [2, 3]
+            text: str  # name: "text", shape: [1]
+
+    The datatype of an input is automatically set depending on the field type, e.g.,
+    for a field that is "list[int]" (or "list[tuple[int, int]]" etc) the datatype
+    will be "INT64". The actual mapping is
+
+        bool: "BOOL",
+        int: "INT64",
+        float: "FP32",
+        str: "BYTES",
+
+    You can override these defaults using DatatypeOverride, e.g.,
+
+        class ExampleInput(InputsBaseModel):
+            points: list[Point]  # datatype: "INT64"
+            more_points: Annotated[list[Point], DatatypeOverride("INT32")]  # datatype: "INT32"
+
+    See DatatypeOverride docs for more information.
+
+    NOTE: Because there is just one datatype per input, it means that fields with a mix of types
+    like "tuple[int, float, str]" should be avoided.
+
+    """
+
+    model_config = pydantic.ConfigDict(extra="forbid", frozen=True, coerce_numbers_to_str=True)
+
+    def to_inputs(self) -> list[OpenInferenceAPIInput]:
+        return [
+            {
+                "name": field_name,
+                "shape": get_shape(field_value),
+                "datatype": get_datatype(field_value, type(self).model_fields[field_name]),
+                "data": get_data(field_value),
+            }
+            for field_name, field_value in self.model_dump(mode="json", warnings=False).items()
+        ]
+
+
+class OutputsBaseModel(pydantic.BaseModel):
+    """Base class for model outputs.
+
+    Subclass this class to define the model inputs, e.g.,
+
+        class MyModelIOutput(OutputsBaseModel):
+            values: list[float]
+
+    The field names MUST correspond to the names of the model outputs.
+
+    The structure of a field must be mappable to the shape of the
+    corresponding output, e.g.,
+    - "list[tuple[int, int]]" for a shape of [-1, 2],
+    - "tuple[float, float]" for  a shape of [2], or
+    - "list[tuple[tuple[int, int], tuple[int, int]]]" for a shape of [-1, 2, 2].
+
+    NOTE: You can define an output field that is neither a list nor a tuple, just
+    a simple datatype like int, float, str; this will correspond to a shape of [1].
+
+    You can use typing.Namedtuple instead of a tuple to get named fields.
+
+    Example:
+        class Point(typing.Namedtuple):
+            x: float
+            y: float
+
+        class ExampleInput(OutputsBaseModel):
+            points: list[Point]  # name: "points", shape: [-1, 2]
+            matrix: tuple[tuple[int, int, int], tuple[int, int, int]]  # name: "matrix", shape: [2, 3]
+            text: str  # name: "text", shape: [1]
+
+    You can take advantage of pydantic's type coercion by defining the field types
+    as you please, including mixed values.
+
+    Example:
+        Model outputs of the form
+            [
+                {
+                    "name": "labels",
+                    "datatype": "BYTES",
+                    "shape": [3, 2],
+                    "data": [
+                        "SPAM",
+                        "0.8",
+                        "HAM",
+                        "0.3",
+                        "SPAM",
+                        "0.9"
+                    ]
+                }
+            ]
+
+        can be, using this OutputsBaseModel:
+
+            class LabelAndProbability(typing.Namedtuple):
+                label: typing.Literal["SPAM", "HAM"]
+                probability: float
+
+            class ExampleInput(OutputsBaseModel):
+                labels: list[LabelAndProbability]
+
+        coerced to
+
+            ExampleInput(
+                labels=[
+                    LabelAndProbability(label="SPAM", probability=0.8),
+                    LabelAndProbability(label="HAM", probability=0.3),
+                    LabelAndProbability(label="SPAM", probability=0.9),
+                ]
+            )
+
+        i.e., we get named fields of the desired type.
+
+    """
+
+    model_config = pydantic.ConfigDict(extra="forbid", frozen=True, coerce_numbers_to_str=True)
+
+    @classmethod
+    def from_outputs(cls: type[Self], outputs: list[OpenInferenceAPIOutput]) -> Self:
+        values: dict[str, Any] = {}
+        for output in outputs:
+            value = unflatten_data(output["shape"], output["data"])
+            if output["shape"] == [1] and not is_listlike(cls.model_fields[output["name"]].annotation):
+                values[output["name"]] = value[0]
+            else:
+                values[output["name"]] = value
+        return cls(**values)
+
+    @classmethod
+    def get_requested_outputs(cls) -> list[OpenInferenceAPIRequestedOutput]:
+        return [{"name": field_name} for field_name in cls.model_fields]
+
+
+InputsModelT = TypeVar("InputsModelT", bound=InputsBaseModel)
+OutputsModelT = TypeVar("OutputsModelT", bound=OutputsBaseModel)
+
+
+class RemoteModel(Generic[InputsModelT, OutputsModelT]):
+    """Interface toward a remote model called via the Open Inference protocol.
+
+    Define the server, model, inputs, and outputs, then call
+    the "infer" method to call the remote model, automatically
+    handling conversion between the inputs/outputs models and
+    the Open Inference protocol.
+
+    This class is designed to be thread-safe.
+
+    """
+
+    def __init__(  # noqa: PLR0913
+        self,
+        *,
+        model_name: str,
+        inputs_model: type[InputsModelT],
+        outputs_model: type[OutputsModelT],
+        server_url: str,
+        request_timeout_seconds: float | None = None,
+        model_version: str | None = None,
+    ) -> None:
+        """Initialize a RemoteModel.
+
+        Args:
+            model_name: The name of the model in the inference server.
+            inputs_model: InputsBaseModel subclass defining the model inputs.
+            outputs_model: OutputsBaseModel subclass defining the model outputs.
+            server_url: The URL of the inference server (not including the
+                API version "v2/..." and onward.)
+            request_timeout_seconds: Optional timeout (seconds) for requests to the inference server.
+            model_version: Optional model version.
+
+        """
+        self._model_name = model_name
+        self._model_version = model_version
+        self._inputs_model = inputs_model
+        self._outputs_model = outputs_model
+        self._client_API = OpenInferenceHTTPClientAPI(base_url=server_url)
+        self._request_timeout_seconds = request_timeout_seconds
+
+    def infer(self, inputs: InputsModelT) -> OutputsModelT:
+        """Use the remote model for inference.
+
+        Args:
+            inputs: The model inputs in the form of the InputsBaseModel
+                subclass defined for this RemoteModel instance.
+
+        Returns:
+            outputs: The model outputs in the form of the OutputsBaseModel
+                subclass defined for this RemoteModel instance.
+
+        """
+        if not isinstance(inputs, self._inputs_model):
+            raise TypeError(f"Bad inputs type: {type(inputs)}")  # noqa: TRY003
+        return self._outputs_model.from_outputs(
+            self._client_API.infer(
+                model_name=self._model_name,
+                model_version=self._model_version,
+                inputs=inputs.to_inputs(),
+                outputs=self._outputs_model.get_requested_outputs(),
+                timeout_seconds=self._request_timeout_seconds,
+            )
+        )

pydantic_open_inference-1.0.0/src/pydantic_open_inference/_utils.py

@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import itertools
+import sys
+import types
+from typing import TYPE_CHECKING, Any, ClassVar, Final, Literal, TypedDict
+
+if sys.version_info < (3, 11):
+    from typing_extensions import NotRequired  # pragma: no cover
+else:
+    from typing import NotRequired  # pragma: no cover
+
+if sys.version_info < (3, 12):
+    from typing_extensions import override  # pragma: no cover
+else:
+    from typing import override  # pragma: no cover
+
+if sys.version_info < (3, 13):
+    from more_itertools import batched  # pragma: no cover
+else:
+    from itertools import batched  # pragma: no cover
+if TYPE_CHECKING:
+    from collections.abc import Mapping
+
+    from pydantic.fields import FieldInfo
+
+Shape = list[int]
+Datatype = Literal[
+    "BOOL",
+    "UINT8",
+    "UINT16",
+    "UINT32",
+    "UINT64",
+    "INT8",
+    "INT16",
+    "INT32",
+    "INT64",
+    "FP16",
+    "FP32",
+    "FP64",
+    "BYTES",
+]
+Data = list[Any]
+_TYPE_TO_DATATYPE_MAP: Final[Mapping[type, Datatype]] = {
+    bool: "BOOL",
+    int: "INT64",
+    float: "FP32",
+    str: "BYTES",
+}
+
+
+class PydanticOpenInferenceError(Exception):
+    """Package base exception."""
+
+
+class ShapeDataMismatchError(PydanticOpenInferenceError):
+    """Exception raised when shape and data do not match."""
+
+
+class DatatypeOverride:
+    """Use with typing.Annotated to override the default datatype in inputs.
+
+    In this example InputsBaseModel, the datatype of
+    "values" would be "INT64":
+
+        class IntInputsBaseModel(InputsBaseModel):
+            values: list[int]
+
+    Using DatatypeOverride, we can instead force the datatype
+    to be anything we want, e.g., "INT16":
+
+        class IntInputsBaseModel(InputsBaseModel):
+            values: Annotated[list[int], DatatypeOverride("INT16")]
+
+    Note that this simply sets the datatype as given. There
+    are no additional checks (for sign, size, etc).
+
+    """
+
+    __slots__ = ("_datatype",)
+
+    def __init__(self, datatype: Datatype) -> None:
+        self._datatype = datatype
+
+    @property
+    def datatype(self) -> Datatype:
+        return self._datatype
+
+
+def is_flat(data: Data) -> bool:
+    return not data or not any(isinstance(x, (list, tuple)) for x in data)
+
+
+def parse_row_major_order(shape: Shape, data: list[Any]) -> list[Any]:
+    if len(shape) == 1:
+        if shape[0] != len(data):
+            raise ShapeDataMismatchError
+        return data
+    new_shape: Shape = list(shape[:-1])
+    return parse_row_major_order(new_shape, list(batched(data, n=shape[-1], strict=True)))
+
+
+def is_listlike(annotation: type[Any] | None) -> bool:
+    if annotation is None:
+        return False
+    if isinstance(annotation, types.GenericAlias):
+        annotation = annotation.__origin__  # type: ignore[unreachable]
+    return any(t in (list, tuple) for t in itertools.chain((annotation,), annotation.__bases__))
+
+
+def unflatten_data(shape: Shape, data: Data) -> Data:
+    if is_flat(data):
+        return parse_row_major_order(shape, data)
+    return data
+
+
+def get_shape(value: Any) -> Shape:
+    shape: Shape = []
+    while isinstance(value, (list, tuple)):
+        shape.append(len(value))
+        value = value[0]
+    return shape or [1]
+
+
+def get_datatype(value: Any, field_info: FieldInfo) -> Datatype:
+    overrides: list[DatatypeOverride] = [x for x in field_info.metadata if isinstance(x, DatatypeOverride)]
+    if overrides:
+        return overrides[0].datatype
+    while isinstance(value, (list, tuple)):
+        value = value[0]
+    return _TYPE_TO_DATATYPE_MAP[type(value)]
+
+
+def get_data(value: Any) -> Data:
+    if not isinstance(value, (tuple, list)):
+        return [value]
+    return list(value)
+
+
+class _OpenInferenceAPIPut(TypedDict):
+    name: str
+    shape: Shape
+    datatype: Datatype
+    data: Data
+
+
+class OpenInferenceAPIInput(_OpenInferenceAPIPut):
+    parameters: NotRequired[dict[str, Any]]
+
+
+class OpenInferenceAPIRequestedOutput(TypedDict):
+    name: str
+    parameters: NotRequired[dict[str, Any]]
+
+
+class OpenInferenceAPIOutput(_OpenInferenceAPIPut):
+    parameters: NotRequired[dict[str, Any]]
+
+
+class Singleton(type):
+    _instances: ClassVar[dict[tuple[type, tuple[Any, ...], str], Singleton]] = {}
+
+    @override
+    def __call__(cls, *args, **kwargs):  # type: ignore[no-untyped-def]
+        key = (cls, args, str(kwargs))
+        if key not in cls._instances:
+            cls._instances[key] = super().__call__(*args, **kwargs)
+        return cls._instances[key]