singlepipe 0.1.0__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- singlepipe-0.3.0/PKG-INFO +425 -0
- singlepipe-0.3.0/README.md +391 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/pyproject.toml +4 -2
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/__init__.py +20 -2
- singlepipe-0.3.0/singlepipe/_fanout.py +157 -0
- singlepipe-0.3.0/singlepipe/_graph/__init__.py +12 -0
- singlepipe-0.3.0/singlepipe/_graph/_context.py +196 -0
- singlepipe-0.3.0/singlepipe/_graph/graph.py +152 -0
- singlepipe-0.3.0/singlepipe/_graph/node.py +294 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/adapters/base.py +67 -5
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/adapters/hf_adapter.py +13 -4
- singlepipe-0.3.0/singlepipe/ops.py +49 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/modalities/image.py +140 -16
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/primitives.py +42 -8
- singlepipe-0.3.0/singlepipe.egg-info/PKG-INFO +425 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/SOURCES.txt +8 -0
- singlepipe-0.3.0/tests/test_fanout.py +167 -0
- singlepipe-0.3.0/tests/test_ops.py +72 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/tests/test_package.py +9 -0
- singlepipe-0.1.0/PKG-INFO +0 -249
- singlepipe-0.1.0/README.md +0 -215
- singlepipe-0.1.0/singlepipe.egg-info/PKG-INFO +0 -249
- {singlepipe-0.1.0 → singlepipe-0.3.0}/LICENSE +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/setup.cfg +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/adapters/__init__.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/py.typed +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/__init__.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/containers.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/contract.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/__init__.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/base.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/classification.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/detection.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/modalities/__init__.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/tasks.py +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/dependency_links.txt +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/requires.txt +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/top_level.txt +0 -0
- {singlepipe-0.1.0 → singlepipe-0.3.0}/tests/test_tasks.py +0 -0
|
@@ -0,0 +1,425 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: singlepipe
|
|
3
|
+
Version: 0.3.0
|
|
4
|
+
Summary: Typed, async, batching inference for ML models: pydantic schemas in and out, any backend
|
|
5
|
+
Author-email: Gleb <glebov274@gmail.com>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Keywords: ml,inference,huggingface,pytorch,pydantic,typed,async,batching
|
|
8
|
+
Classifier: Development Status :: 3 - Alpha
|
|
9
|
+
Classifier: Intended Audience :: Developers
|
|
10
|
+
Classifier: Programming Language :: Python :: 3
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
14
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
15
|
+
Classifier: Typing :: Typed
|
|
16
|
+
Requires-Python: >=3.12
|
|
17
|
+
Description-Content-Type: text/markdown
|
|
18
|
+
License-File: LICENSE
|
|
19
|
+
Requires-Dist: httpx>=0.28.1
|
|
20
|
+
Requires-Dist: pillow>=12.3.0
|
|
21
|
+
Requires-Dist: pydantic>=2.13.4
|
|
22
|
+
Requires-Dist: numpy>=2.0
|
|
23
|
+
Provides-Extra: hf
|
|
24
|
+
Requires-Dist: transformers>=5.14.1; extra == "hf"
|
|
25
|
+
Requires-Dist: torch>=2.13.0; extra == "hf"
|
|
26
|
+
Provides-Extra: detr
|
|
27
|
+
Requires-Dist: singlepipe[hf]; extra == "detr"
|
|
28
|
+
Requires-Dist: timm>=1.0.28; extra == "detr"
|
|
29
|
+
Provides-Extra: torch
|
|
30
|
+
Requires-Dist: torch>=2.13.0; extra == "torch"
|
|
31
|
+
Provides-Extra: all
|
|
32
|
+
Requires-Dist: singlepipe[detr,hf,torch]; extra == "all"
|
|
33
|
+
Dynamic: license-file
|
|
34
|
+
|
|
35
|
+
# SinglePipe
|
|
36
|
+
|
|
37
|
+
**Typed, async, batching inference for ML models.** Pydantic schemas in and out,
|
|
38
|
+
any backend underneath.
|
|
39
|
+
|
|
40
|
+
SinglePipe does not reimplement HuggingFace, PyTorch or scikit-learn. It puts a
|
|
41
|
+
typed boundary around them: inputs are validated and normalised, outputs are
|
|
42
|
+
typed objects your editor and mypy understand, and the model runs off the event
|
|
43
|
+
loop with dynamic batching. Several models are composed into a graph — a plain
|
|
44
|
+
async function — that tells you where an error happened, records what every
|
|
45
|
+
model saw and returned, and serves as a FastAPI endpoint in one line.
|
|
46
|
+
|
|
47
|
+
> **Status: alpha (0.3.0).** Image classification and object detection on
|
|
48
|
+
> HuggingFace models, the glue between models (crops, coordinates, fan-out) and
|
|
49
|
+
> graphs with a run log. The API will change before 1.0; breaking changes are
|
|
50
|
+
> listed in `CHANGELOG.md`.
|
|
51
|
+
|
|
52
|
+
## Install
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
pip install "singlepipe[hf]" # HuggingFace models (transformers + torch)
|
|
56
|
+
pip install "singlepipe[detr]" # + timm, for DETR detection models
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The core (`pip install singlepipe`) has no ML backend: schemas, tasks, image
|
|
60
|
+
handling, glue and graphs only. Python 3.12+.
|
|
61
|
+
|
|
62
|
+
## Quick start
|
|
63
|
+
|
|
64
|
+
```python
|
|
65
|
+
import asyncio
|
|
66
|
+
|
|
67
|
+
import singlepipe as sp
|
|
68
|
+
from singlepipe.adapters.hf_adapter import HFAdapter
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
async def main() -> None:
|
|
72
|
+
classifier = HFAdapter("google/vit-base-patch16-224", sp.tasks.image_classification)
|
|
73
|
+
|
|
74
|
+
out = await classifier("cats.jpg") # path, URL, data URI, bytes, PIL, numpy, torch tensor
|
|
75
|
+
best = out.predictions.top1 # typed: Prediction | None
|
|
76
|
+
if best:
|
|
77
|
+
print(best.label, best.score)
|
|
78
|
+
print(out.predictions.top(3).labels)
|
|
79
|
+
|
|
80
|
+
classifier.close()
|
|
81
|
+
|
|
82
|
+
|
|
83
|
+
asyncio.run(main())
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
`out` is an `ImageClassificationOutput`, not a dict: mypy knows it without a
|
|
87
|
+
`cast`. In Jupyter, `await` works directly in a cell.
|
|
88
|
+
|
|
89
|
+
## Object detection
|
|
90
|
+
|
|
91
|
+
```python
|
|
92
|
+
detector = HFAdapter("facebook/detr-resnet-50", sp.tasks.object_detection)
|
|
93
|
+
|
|
94
|
+
out = await detector("http://images.cocodataset.org/val2017/000000039769.jpg")
|
|
95
|
+
for det in out.predictions.filter_by_label("cat").filter_by_score(0.9):
|
|
96
|
+
print(det.label, round(det.score, 3), det.box.as_tuple())
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Boxes are in pixels of the original image (`out.image_width`, `out.image_height`)
|
|
100
|
+
and are returned as the model gave them — even partly outside the image, on any
|
|
101
|
+
side, as detectors do for objects at the edge. `box.clip(w, h)` cuts a box to
|
|
102
|
+
the image; `image.crop(box)` does that by itself.
|
|
103
|
+
|
|
104
|
+
## Many models: a graph
|
|
105
|
+
|
|
106
|
+
Detect bottles on a shelf, classify every bottle, count the brands:
|
|
107
|
+
|
|
108
|
+
```python
|
|
109
|
+
from collections import Counter
|
|
110
|
+
|
|
111
|
+
import singlepipe as sp
|
|
112
|
+
from singlepipe.adapters.hf_adapter import HFAdapter
|
|
113
|
+
from singlepipe.schemas.modalities import Image
|
|
114
|
+
|
|
115
|
+
detector = HFAdapter("facebook/detr-resnet-50", sp.tasks.object_detection)
|
|
116
|
+
classifier = HFAdapter("my-org/sku-vit", sp.tasks.image_classification, max_batch_size=16)
|
|
117
|
+
|
|
118
|
+
|
|
119
|
+
@sp.graph
|
|
120
|
+
async def shelf(image: Image) -> dict[str, int]:
|
|
121
|
+
dets = await detector(image)
|
|
122
|
+
bottles = dets.predictions.filter_by_label("bottle").filter_by_score(0.5)
|
|
123
|
+
results = await classifier.map(sp.ops.crops(image, bottles, scale=1.1))
|
|
124
|
+
return dict(Counter(r.predictions.top1.label for r in results if r.predictions.top1))
|
|
125
|
+
|
|
126
|
+
|
|
127
|
+
counts = await shelf(Image("shelf.jpg")) # {"cola": 7, "water": 3}
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
A graph is plain Python: loops, `if`, fallbacks and parallel branches
|
|
131
|
+
(`asyncio.TaskGroup`) need nothing special. What `@sp.graph` adds:
|
|
132
|
+
|
|
133
|
+
- **Boundaries.** Called from outside — a script, FastAPI — the arguments are
|
|
134
|
+
validated leniently: a path, URL, bytes or PIL image becomes an `Image` (mypy
|
|
135
|
+
still wants the annotated type, hence `Image(...)` above). Called from inside
|
|
136
|
+
another graph they are validated strictly: there the data is yours, and a
|
|
137
|
+
silent conversion would hide a bug. The return value is always checked.
|
|
138
|
+
- **Where an error happened** (below) and **what every model saw** (the run log).
|
|
139
|
+
- **One line to a service** (FastAPI, below).
|
|
140
|
+
|
|
141
|
+
In the graph:
|
|
142
|
+
|
|
143
|
+
- **`classifier.map(items)`** sends every item at once, so the adapter batches
|
|
144
|
+
them. A loop of `await classifier(c)` would send them one at a time and
|
|
145
|
+
batching would never kick in. Results come back in input order.
|
|
146
|
+
- **`sp.ops.crops`** cuts one crop per detection (or bare `BBox`). `scale=1.1`
|
|
147
|
+
makes every box 1.1× wider and taller about its center; at the image edge the
|
|
148
|
+
crop simply stops there. For anything per box, write the list comprehension:
|
|
149
|
+
`[image.crop(d.box.scale(k)) for d in dets if ...]`.
|
|
150
|
+
- A crop is clipped to the image and widened to whole pixels; nothing is padded.
|
|
151
|
+
|
|
152
|
+
### The run log
|
|
153
|
+
|
|
154
|
+
`shelf(image)` records nothing. `shelf.run(image)` returns the result together
|
|
155
|
+
with every step that produced it — each model call, node and nested graph, with
|
|
156
|
+
references to its inputs and output, in the order they finished:
|
|
157
|
+
|
|
158
|
+
```python
|
|
159
|
+
run = await shelf.run(Image("shelf.jpg"))
|
|
160
|
+
for step in run.steps:
|
|
161
|
+
item = f"[{step.item}]" if step.item is not None else ""
|
|
162
|
+
print(" > ".join(step.path), item, type(step.output).__name__)
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
```
|
|
166
|
+
shelf > facebook/detr-resnet-50 ObjectDetectionOutput
|
|
167
|
+
shelf > my-org/sku-vit [2] ImageClassificationOutput
|
|
168
|
+
shelf > my-org/sku-vit [0] ImageClassificationOutput
|
|
169
|
+
shelf > my-org/sku-vit [1] ImageClassificationOutput
|
|
170
|
+
shelf dict
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
`step.item` is the index of the `map` item the step ran for; `step.inputs`
|
|
174
|
+
holds the crop the classifier saw, `step.output` what it answered,
|
|
175
|
+
`step.seconds` how long it took.
|
|
176
|
+
|
|
177
|
+
### When something fails
|
|
178
|
+
|
|
179
|
+
An error leaves the graph as it was raised — a bad input is still a
|
|
180
|
+
`pydantic.ValidationError`, so `except` clauses and FastAPI handlers keep
|
|
181
|
+
working — with notes saying where:
|
|
182
|
+
|
|
183
|
+
```
|
|
184
|
+
pydantic_core._pydantic_core.ValidationError: 1 validation error for ...
|
|
185
|
+
in node: shelf > my-org/sku-vit
|
|
186
|
+
my-org/sku-vit.map: item at index 6 of 14
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Inside `shelf.run(...)`, the steps recorded before the failure are on the
|
|
190
|
+
exception: `sp.steps_of(error)`.
|
|
191
|
+
|
|
192
|
+
By default the first failing item of a `map` cancels the remaining ones. To keep
|
|
193
|
+
going, ask for it explicitly; the result type says that an item may be an
|
|
194
|
+
error, so mypy makes you handle it:
|
|
195
|
+
|
|
196
|
+
```python
|
|
197
|
+
results = await classifier.map(crops, errors="collect")
|
|
198
|
+
# list[ImageClassificationOutput | Exception], same length and order as crops
|
|
199
|
+
|
|
200
|
+
for crop, result in zip(crops, results):
|
|
201
|
+
if isinstance(result, Exception):
|
|
202
|
+
print("failed:", crop.origin, result)
|
|
203
|
+
elif result.predictions.top1:
|
|
204
|
+
print(result.predictions.top1.label)
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
Which one is right depends on the task: a content filter must not pass an image
|
|
208
|
+
it failed to check; a catalogue search can return 49 results out of 50.
|
|
209
|
+
|
|
210
|
+
### Your own steps: `@sp.node`
|
|
211
|
+
|
|
212
|
+
Any function becomes a step of the graph — named in errors and recorded in the
|
|
213
|
+
run log:
|
|
214
|
+
|
|
215
|
+
```python
|
|
216
|
+
@sp.node
|
|
217
|
+
def share(counts: dict[str, int], brand: str) -> float:
|
|
218
|
+
return counts.get(brand, 0) / max(sum(counts.values()), 1)
|
|
219
|
+
|
|
220
|
+
|
|
221
|
+
cola = share(counts, "cola")
|
|
222
|
+
per_shelf = share.map([counts_a, counts_b], brand="cola") # list[float]
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
- Annotated arguments and the return value are validated strictly; an
|
|
226
|
+
unannotated argument is passed as it is. No annotation, no check — name a type
|
|
227
|
+
only where you want the guarantee.
|
|
228
|
+
- A plain function stays plain: called without `await`, its `map` returns a
|
|
229
|
+
list. Heavy synchronous work (pure-Python image processing, a big regex pass)
|
|
230
|
+
would freeze the event loop: mark it `@sp.node(thread=True)` and `await` it.
|
|
231
|
+
An `async def` node is awaited as usual.
|
|
232
|
+
- `.map(items, **shared)` on any node or graph works like `adapter.map`:
|
|
233
|
+
the item is the first argument, `shared` goes to every call, `errors="collect"`
|
|
234
|
+
is there too.
|
|
235
|
+
|
|
236
|
+
## Cascades: back to the original frame
|
|
237
|
+
|
|
238
|
+
A crop remembers where it was cut (`crop.origin`), so a box found inside it maps
|
|
239
|
+
back to the original image:
|
|
240
|
+
|
|
241
|
+
```python
|
|
242
|
+
from singlepipe.schemas.primitives import BBox
|
|
243
|
+
|
|
244
|
+
|
|
245
|
+
@sp.graph
|
|
246
|
+
async def find_plate(frame: Image) -> BBox | None:
|
|
247
|
+
car = (await car_detector(frame)).predictions.filter_by_label("car").top1
|
|
248
|
+
if car is None:
|
|
249
|
+
return None
|
|
250
|
+
car_crop = frame.crop(car.box)
|
|
251
|
+
|
|
252
|
+
plate = (await plate_detector(car_crop)).predictions.top1
|
|
253
|
+
if plate is None:
|
|
254
|
+
return None
|
|
255
|
+
return car_crop.to_frame(plate.box) # in pixels of `frame`
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
The offset is the region actually cut — after clipping to the image and
|
|
259
|
+
rounding to whole pixels — not the requested box, so the mapped box is exact.
|
|
260
|
+
A crop of a crop maps straight to the first image.
|
|
261
|
+
|
|
262
|
+
## Images
|
|
263
|
+
|
|
264
|
+
`Image` accepts a file path, an `http(s)://` URL, a base64 data URI, raw bytes,
|
|
265
|
+
a PIL image, a numpy array (HWC or HW) or a torch tensor (CHW), and normalises
|
|
266
|
+
it to RGB. It never guesses silently:
|
|
267
|
+
|
|
268
|
+
- pixel arrays must be `uint8` or float in `[0, 1]`; a float array in `0..255`,
|
|
269
|
+
an `int16` array, NaN or an empty array raise `ValueError` with a hint, instead
|
|
270
|
+
of turning into a white or wrapped-around picture;
|
|
271
|
+
- EXIF orientation is applied, as HuggingFace does: a phone photo reaches the
|
|
272
|
+
model upright, and boxes are in the frame people see.
|
|
273
|
+
|
|
274
|
+
## FastAPI
|
|
275
|
+
|
|
276
|
+
A graph is an endpoint:
|
|
277
|
+
|
|
278
|
+
```python
|
|
279
|
+
from contextlib import asynccontextmanager
|
|
280
|
+
from typing import Annotated
|
|
281
|
+
|
|
282
|
+
from fastapi import Body, FastAPI
|
|
283
|
+
|
|
284
|
+
|
|
285
|
+
@sp.graph
|
|
286
|
+
async def shelf_api(image: Annotated[Image, Body()]) -> dict[str, int]:
|
|
287
|
+
return await shelf(image)
|
|
288
|
+
|
|
289
|
+
|
|
290
|
+
@asynccontextmanager
|
|
291
|
+
async def lifespan(app: FastAPI):
|
|
292
|
+
yield
|
|
293
|
+
detector.close()
|
|
294
|
+
classifier.close()
|
|
295
|
+
|
|
296
|
+
|
|
297
|
+
app = FastAPI(lifespan=lifespan)
|
|
298
|
+
app.post("/shelf")(shelf_api)
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
The request body is the image as a JSON string (a data URI); a bad image is a
|
|
302
|
+
422. `Body()` is needed because an `Image` is a string in the JSON schema, and
|
|
303
|
+
FastAPI takes plain strings from the query string by default.
|
|
304
|
+
|
|
305
|
+
Concurrent requests share batches: with `max_batch_size` set, calls that arrive
|
|
306
|
+
while the model is busy run together in the next forward pass.
|
|
307
|
+
|
|
308
|
+
> **Do not expose this to untrusted clients yet.** A string given to `Image` may
|
|
309
|
+
> be a local file path or any URL, so a public endpoint can read files on the
|
|
310
|
+
> server or reach internal hosts. A mode for untrusted input is the next thing
|
|
311
|
+
> on the roadmap; until then, put the service behind your own authentication,
|
|
312
|
+
> accept only data URIs and check them yourself.
|
|
313
|
+
|
|
314
|
+
## Your own task on a HuggingFace model
|
|
315
|
+
|
|
316
|
+
A task is a pair of schemas. Built-in tasks are ready objects; a custom one is a
|
|
317
|
+
line of code.
|
|
318
|
+
|
|
319
|
+
```python
|
|
320
|
+
from typing import Any, Self
|
|
321
|
+
|
|
322
|
+
import singlepipe as sp
|
|
323
|
+
from singlepipe.adapters.hf_adapter import HFAdapter
|
|
324
|
+
from singlepipe.schemas.contract import OutputSchema
|
|
325
|
+
from singlepipe.schemas.image.base import ImageTaskInput
|
|
326
|
+
|
|
327
|
+
|
|
328
|
+
class IsCat(OutputSchema):
|
|
329
|
+
is_cat: bool
|
|
330
|
+
confidence: float
|
|
331
|
+
|
|
332
|
+
@classmethod
|
|
333
|
+
def from_pipeline_output(cls, raw: Any, **context: Any) -> Self:
|
|
334
|
+
cat = next((p for p in raw if "cat" in p["label"]), None)
|
|
335
|
+
return cls(is_cat=cat is not None, confidence=cat["score"] if cat else 0.0)
|
|
336
|
+
|
|
337
|
+
|
|
338
|
+
is_cat = sp.Task("is-cat", ImageTaskInput, IsCat)
|
|
339
|
+
adapter = HFAdapter("google/vit-base-patch16-224", is_cat, hf_task="image-classification")
|
|
340
|
+
|
|
341
|
+
answer = await adapter("cats.jpg")
|
|
342
|
+
answer.is_cat # typed: bool
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
## Your own adapter, any library
|
|
346
|
+
|
|
347
|
+
Everything except the backend call lives in `Adapter`: async calls, validation,
|
|
348
|
+
the worker thread, batching, `map`, its place in a graph, `close()`. A backend
|
|
349
|
+
implements one method.
|
|
350
|
+
|
|
351
|
+
```python
|
|
352
|
+
from collections.abc import Sequence
|
|
353
|
+
from typing import Any
|
|
354
|
+
|
|
355
|
+
import numpy as np
|
|
356
|
+
import onnxruntime
|
|
357
|
+
|
|
358
|
+
import singlepipe as sp
|
|
359
|
+
from singlepipe.adapters.base import Adapter
|
|
360
|
+
from singlepipe.schemas.image.classification import (
|
|
361
|
+
ImageClassificationInput,
|
|
362
|
+
ImageClassificationOutput,
|
|
363
|
+
)
|
|
364
|
+
|
|
365
|
+
|
|
366
|
+
class OnnxClassifier(Adapter[ImageClassificationInput, ImageClassificationOutput]):
|
|
367
|
+
def __init__(self, path: str, labels: list[str], **batching: Any) -> None:
|
|
368
|
+
self.session = onnxruntime.InferenceSession(path) # load the model first,
|
|
369
|
+
self.labels = labels
|
|
370
|
+
super().__init__(sp.tasks.image_classification, name=path, **batching) # then start
|
|
371
|
+
|
|
372
|
+
def run_batch(self, inputs: Sequence[Any]) -> Sequence[Any]:
|
|
373
|
+
x = np.stack([preprocess(image) for image in inputs]) # your resize + normalise
|
|
374
|
+
probabilities = self.session.run(None, {"input": x})[0]
|
|
375
|
+
return [
|
|
376
|
+
[{"label": label, "score": float(p)} for label, p in zip(self.labels, row)]
|
|
377
|
+
for row in probabilities
|
|
378
|
+
]
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
`run_batch` receives what the input schema produces (here RGB PIL images) and
|
|
382
|
+
returns one raw output per input, in the format the output schema reads.
|
|
383
|
+
`name` is how the adapter appears in error notes and the run log; it defaults to
|
|
384
|
+
`ClassName(task)` (an `HFAdapter` uses the model id).
|
|
385
|
+
|
|
386
|
+
## Performance
|
|
387
|
+
|
|
388
|
+
Measured on one Tesla T4 with `google/vit-base-patch16-224`:
|
|
389
|
+
|
|
390
|
+
- **The typed layer costs 0–0.7 ms per call** against a bare `transformers`
|
|
391
|
+
pipeline (16.6 ms per image in fp32).
|
|
392
|
+
- **fp16 doubles throughput** (×2.2–2.4): pass it to the pipeline,
|
|
393
|
+
`HFAdapter(..., dtype=torch.float16)`.
|
|
394
|
+
- **For a server, batch without waiting:** `max_batch_size=16` to `32` and the
|
|
395
|
+
default `max_wait_ms=0`. Batches grow by themselves when requests arrive while
|
|
396
|
+
the model is busy (30 of 32 on average at 64 concurrent callers): +54%
|
|
397
|
+
throughput and −35% latency in fp16. A wait window adds nothing under load
|
|
398
|
+
and slows down a lone caller.
|
|
399
|
+
|
|
400
|
+
## What is there today
|
|
401
|
+
|
|
402
|
+
| | |
|
|
403
|
+
| --------- | ------------------------------------------------------------------------------- |
|
|
404
|
+
| Tasks | `image_classification`, `object_detection` |
|
|
405
|
+
| Backends | HuggingFace `transformers` pipelines; anything else via `Adapter` |
|
|
406
|
+
| Inputs | file path, URL, data URI, bytes, PIL image, numpy array, torch tensor |
|
|
407
|
+
| Execution | async calls, a worker thread per model, dynamic batching, `map` |
|
|
408
|
+
| Graphs | `@sp.graph`, `@sp.node`, `.map` on any node, FastAPI endpoints |
|
|
409
|
+
| Debugging | node paths in errors, run log (`graph.run`, `sp.steps_of`) |
|
|
410
|
+
| Glue | `Image.crop`, `origin`, `to_frame`; `BBox.scale`, `clip`, `iou`; `sp.ops.crops` |
|
|
411
|
+
| Typing | outputs typed for mypy without `cast` (`py.typed` included) |
|
|
412
|
+
|
|
413
|
+
## Roadmap
|
|
414
|
+
|
|
415
|
+
- Safe mode for untrusted input in FastAPI (next)
|
|
416
|
+
- Text tasks: classification, named entities
|
|
417
|
+
- Audio: speech recognition, audio classification
|
|
418
|
+
- scikit-learn / XGBoost adapters and tabular tasks; PyTorch adapter
|
|
419
|
+
- `Connector`: picks the adapter from the model object
|
|
420
|
+
- A faster HuggingFace path that skips the pipeline's own overhead
|
|
421
|
+
- Later: a node as a remote service (`sp.serve`, `RemoteAdapter`)
|
|
422
|
+
|
|
423
|
+
## License
|
|
424
|
+
|
|
425
|
+
MIT
|