singlepipe 0.1.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. singlepipe-0.3.0/PKG-INFO +425 -0
  2. singlepipe-0.3.0/README.md +391 -0
  3. {singlepipe-0.1.0 → singlepipe-0.3.0}/pyproject.toml +4 -2
  4. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/__init__.py +20 -2
  5. singlepipe-0.3.0/singlepipe/_fanout.py +157 -0
  6. singlepipe-0.3.0/singlepipe/_graph/__init__.py +12 -0
  7. singlepipe-0.3.0/singlepipe/_graph/_context.py +196 -0
  8. singlepipe-0.3.0/singlepipe/_graph/graph.py +152 -0
  9. singlepipe-0.3.0/singlepipe/_graph/node.py +294 -0
  10. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/adapters/base.py +67 -5
  11. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/adapters/hf_adapter.py +13 -4
  12. singlepipe-0.3.0/singlepipe/ops.py +49 -0
  13. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/modalities/image.py +140 -16
  14. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/primitives.py +42 -8
  15. singlepipe-0.3.0/singlepipe.egg-info/PKG-INFO +425 -0
  16. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/SOURCES.txt +8 -0
  17. singlepipe-0.3.0/tests/test_fanout.py +167 -0
  18. singlepipe-0.3.0/tests/test_ops.py +72 -0
  19. {singlepipe-0.1.0 → singlepipe-0.3.0}/tests/test_package.py +9 -0
  20. singlepipe-0.1.0/PKG-INFO +0 -249
  21. singlepipe-0.1.0/README.md +0 -215
  22. singlepipe-0.1.0/singlepipe.egg-info/PKG-INFO +0 -249
  23. {singlepipe-0.1.0 → singlepipe-0.3.0}/LICENSE +0 -0
  24. {singlepipe-0.1.0 → singlepipe-0.3.0}/setup.cfg +0 -0
  25. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/adapters/__init__.py +0 -0
  26. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/py.typed +0 -0
  27. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/__init__.py +0 -0
  28. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/containers.py +0 -0
  29. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/contract.py +0 -0
  30. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/__init__.py +0 -0
  31. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/base.py +0 -0
  32. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/classification.py +0 -0
  33. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/image/detection.py +0 -0
  34. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/schemas/modalities/__init__.py +0 -0
  35. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe/tasks.py +0 -0
  36. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/dependency_links.txt +0 -0
  37. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/requires.txt +0 -0
  38. {singlepipe-0.1.0 → singlepipe-0.3.0}/singlepipe.egg-info/top_level.txt +0 -0
  39. {singlepipe-0.1.0 → singlepipe-0.3.0}/tests/test_tasks.py +0 -0
@@ -0,0 +1,425 @@
1
+ Metadata-Version: 2.4
2
+ Name: singlepipe
3
+ Version: 0.3.0
4
+ Summary: Typed, async, batching inference for ML models: pydantic schemas in and out, any backend
5
+ Author-email: Gleb <glebov274@gmail.com>
6
+ License-Expression: MIT
7
+ Keywords: ml,inference,huggingface,pytorch,pydantic,typed,async,batching
8
+ Classifier: Development Status :: 3 - Alpha
9
+ Classifier: Intended Audience :: Developers
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: Programming Language :: Python :: 3.12
12
+ Classifier: Programming Language :: Python :: 3.13
13
+ Classifier: Programming Language :: Python :: 3.14
14
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
15
+ Classifier: Typing :: Typed
16
+ Requires-Python: >=3.12
17
+ Description-Content-Type: text/markdown
18
+ License-File: LICENSE
19
+ Requires-Dist: httpx>=0.28.1
20
+ Requires-Dist: pillow>=12.3.0
21
+ Requires-Dist: pydantic>=2.13.4
22
+ Requires-Dist: numpy>=2.0
23
+ Provides-Extra: hf
24
+ Requires-Dist: transformers>=5.14.1; extra == "hf"
25
+ Requires-Dist: torch>=2.13.0; extra == "hf"
26
+ Provides-Extra: detr
27
+ Requires-Dist: singlepipe[hf]; extra == "detr"
28
+ Requires-Dist: timm>=1.0.28; extra == "detr"
29
+ Provides-Extra: torch
30
+ Requires-Dist: torch>=2.13.0; extra == "torch"
31
+ Provides-Extra: all
32
+ Requires-Dist: singlepipe[detr,hf,torch]; extra == "all"
33
+ Dynamic: license-file
34
+
35
+ # SinglePipe
36
+
37
+ **Typed, async, batching inference for ML models.** Pydantic schemas in and out,
38
+ any backend underneath.
39
+
40
+ SinglePipe does not reimplement HuggingFace, PyTorch or scikit-learn. It puts a
41
+ typed boundary around them: inputs are validated and normalised, outputs are
42
+ typed objects your editor and mypy understand, and the model runs off the event
43
+ loop with dynamic batching. Several models are composed into a graph — a plain
44
+ async function — that tells you where an error happened, records what every
45
+ model saw and returned, and serves as a FastAPI endpoint in one line.
46
+
47
+ > **Status: alpha (0.3.0).** Image classification and object detection on
48
+ > HuggingFace models, the glue between models (crops, coordinates, fan-out) and
49
+ > graphs with a run log. The API will change before 1.0; breaking changes are
50
+ > listed in `CHANGELOG.md`.
51
+
52
+ ## Install
53
+
54
+ ```bash
55
+ pip install "singlepipe[hf]" # HuggingFace models (transformers + torch)
56
+ pip install "singlepipe[detr]" # + timm, for DETR detection models
57
+ ```
58
+
59
+ The core (`pip install singlepipe`) has no ML backend: schemas, tasks, image
60
+ handling, glue and graphs only. Python 3.12+.
61
+
62
+ ## Quick start
63
+
64
+ ```python
65
+ import asyncio
66
+
67
+ import singlepipe as sp
68
+ from singlepipe.adapters.hf_adapter import HFAdapter
69
+
70
+
71
+ async def main() -> None:
72
+ classifier = HFAdapter("google/vit-base-patch16-224", sp.tasks.image_classification)
73
+
74
+ out = await classifier("cats.jpg") # path, URL, data URI, bytes, PIL, numpy, torch tensor
75
+ best = out.predictions.top1 # typed: Prediction | None
76
+ if best:
77
+ print(best.label, best.score)
78
+ print(out.predictions.top(3).labels)
79
+
80
+ classifier.close()
81
+
82
+
83
+ asyncio.run(main())
84
+ ```
85
+
86
+ `out` is an `ImageClassificationOutput`, not a dict: mypy knows it without a
87
+ `cast`. In Jupyter, `await` works directly in a cell.
88
+
89
+ ## Object detection
90
+
91
+ ```python
92
+ detector = HFAdapter("facebook/detr-resnet-50", sp.tasks.object_detection)
93
+
94
+ out = await detector("http://images.cocodataset.org/val2017/000000039769.jpg")
95
+ for det in out.predictions.filter_by_label("cat").filter_by_score(0.9):
96
+ print(det.label, round(det.score, 3), det.box.as_tuple())
97
+ ```
98
+
99
+ Boxes are in pixels of the original image (`out.image_width`, `out.image_height`)
100
+ and are returned as the model gave them — even partly outside the image, on any
101
+ side, as detectors do for objects at the edge. `box.clip(w, h)` cuts a box to
102
+ the image; `image.crop(box)` does that by itself.
103
+
104
+ ## Many models: a graph
105
+
106
+ Detect bottles on a shelf, classify every bottle, count the brands:
107
+
108
+ ```python
109
+ from collections import Counter
110
+
111
+ import singlepipe as sp
112
+ from singlepipe.adapters.hf_adapter import HFAdapter
113
+ from singlepipe.schemas.modalities import Image
114
+
115
+ detector = HFAdapter("facebook/detr-resnet-50", sp.tasks.object_detection)
116
+ classifier = HFAdapter("my-org/sku-vit", sp.tasks.image_classification, max_batch_size=16)
117
+
118
+
119
+ @sp.graph
120
+ async def shelf(image: Image) -> dict[str, int]:
121
+ dets = await detector(image)
122
+ bottles = dets.predictions.filter_by_label("bottle").filter_by_score(0.5)
123
+ results = await classifier.map(sp.ops.crops(image, bottles, scale=1.1))
124
+ return dict(Counter(r.predictions.top1.label for r in results if r.predictions.top1))
125
+
126
+
127
+ counts = await shelf(Image("shelf.jpg")) # {"cola": 7, "water": 3}
128
+ ```
129
+
130
+ A graph is plain Python: loops, `if`, fallbacks and parallel branches
131
+ (`asyncio.TaskGroup`) need nothing special. What `@sp.graph` adds:
132
+
133
+ - **Boundaries.** Called from outside — a script, FastAPI — the arguments are
134
+ validated leniently: a path, URL, bytes or PIL image becomes an `Image` (mypy
135
+ still wants the annotated type, hence `Image(...)` above). Called from inside
136
+ another graph they are validated strictly: there the data is yours, and a
137
+ silent conversion would hide a bug. The return value is always checked.
138
+ - **Where an error happened** (below) and **what every model saw** (the run log).
139
+ - **One line to a service** (FastAPI, below).
140
+
141
+ In the graph:
142
+
143
+ - **`classifier.map(items)`** sends every item at once, so the adapter batches
144
+ them. A loop of `await classifier(c)` would send them one at a time and
145
+ batching would never kick in. Results come back in input order.
146
+ - **`sp.ops.crops`** cuts one crop per detection (or bare `BBox`). `scale=1.1`
147
+ makes every box 1.1× wider and taller about its center; at the image edge the
148
+ crop simply stops there. For anything per box, write the list comprehension:
149
+ `[image.crop(d.box.scale(k)) for d in dets if ...]`.
150
+ - A crop is clipped to the image and widened to whole pixels; nothing is padded.
151
+
152
+ ### The run log
153
+
154
+ `shelf(image)` records nothing. `shelf.run(image)` returns the result together
155
+ with every step that produced it — each model call, node and nested graph, with
156
+ references to its inputs and output, in the order they finished:
157
+
158
+ ```python
159
+ run = await shelf.run(Image("shelf.jpg"))
160
+ for step in run.steps:
161
+ item = f"[{step.item}]" if step.item is not None else ""
162
+ print(" > ".join(step.path), item, type(step.output).__name__)
163
+ ```
164
+
165
+ ```
166
+ shelf > facebook/detr-resnet-50 ObjectDetectionOutput
167
+ shelf > my-org/sku-vit [2] ImageClassificationOutput
168
+ shelf > my-org/sku-vit [0] ImageClassificationOutput
169
+ shelf > my-org/sku-vit [1] ImageClassificationOutput
170
+ shelf dict
171
+ ```
172
+
173
+ `step.item` is the index of the `map` item the step ran for; `step.inputs`
174
+ holds the crop the classifier saw, `step.output` what it answered,
175
+ `step.seconds` how long it took.
176
+
177
+ ### When something fails
178
+
179
+ An error leaves the graph as it was raised — a bad input is still a
180
+ `pydantic.ValidationError`, so `except` clauses and FastAPI handlers keep
181
+ working — with notes saying where:
182
+
183
+ ```
184
+ pydantic_core._pydantic_core.ValidationError: 1 validation error for ...
185
+ in node: shelf > my-org/sku-vit
186
+ my-org/sku-vit.map: item at index 6 of 14
187
+ ```
188
+
189
+ Inside `shelf.run(...)`, the steps recorded before the failure are on the
190
+ exception: `sp.steps_of(error)`.
191
+
192
+ By default the first failing item of a `map` cancels the remaining ones. To keep
193
+ going, ask for it explicitly; the result type says that an item may be an
194
+ error, so mypy makes you handle it:
195
+
196
+ ```python
197
+ results = await classifier.map(crops, errors="collect")
198
+ # list[ImageClassificationOutput | Exception], same length and order as crops
199
+
200
+ for crop, result in zip(crops, results):
201
+ if isinstance(result, Exception):
202
+ print("failed:", crop.origin, result)
203
+ elif result.predictions.top1:
204
+ print(result.predictions.top1.label)
205
+ ```
206
+
207
+ Which one is right depends on the task: a content filter must not pass an image
208
+ it failed to check; a catalogue search can return 49 results out of 50.
209
+
210
+ ### Your own steps: `@sp.node`
211
+
212
+ Any function becomes a step of the graph — named in errors and recorded in the
213
+ run log:
214
+
215
+ ```python
216
+ @sp.node
217
+ def share(counts: dict[str, int], brand: str) -> float:
218
+ return counts.get(brand, 0) / max(sum(counts.values()), 1)
219
+
220
+
221
+ cola = share(counts, "cola")
222
+ per_shelf = share.map([counts_a, counts_b], brand="cola") # list[float]
223
+ ```
224
+
225
+ - Annotated arguments and the return value are validated strictly; an
226
+ unannotated argument is passed as it is. No annotation, no check — name a type
227
+ only where you want the guarantee.
228
+ - A plain function stays plain: called without `await`, its `map` returns a
229
+ list. Heavy synchronous work (pure-Python image processing, a big regex pass)
230
+ would freeze the event loop: mark it `@sp.node(thread=True)` and `await` it.
231
+ An `async def` node is awaited as usual.
232
+ - `.map(items, **shared)` on any node or graph works like `adapter.map`:
233
+ the item is the first argument, `shared` goes to every call, `errors="collect"`
234
+ is there too.
235
+
236
+ ## Cascades: back to the original frame
237
+
238
+ A crop remembers where it was cut (`crop.origin`), so a box found inside it maps
239
+ back to the original image:
240
+
241
+ ```python
242
+ from singlepipe.schemas.primitives import BBox
243
+
244
+
245
+ @sp.graph
246
+ async def find_plate(frame: Image) -> BBox | None:
247
+ car = (await car_detector(frame)).predictions.filter_by_label("car").top1
248
+ if car is None:
249
+ return None
250
+ car_crop = frame.crop(car.box)
251
+
252
+ plate = (await plate_detector(car_crop)).predictions.top1
253
+ if plate is None:
254
+ return None
255
+ return car_crop.to_frame(plate.box) # in pixels of `frame`
256
+ ```
257
+
258
+ The offset is the region actually cut — after clipping to the image and
259
+ rounding to whole pixels — not the requested box, so the mapped box is exact.
260
+ A crop of a crop maps straight to the first image.
261
+
262
+ ## Images
263
+
264
+ `Image` accepts a file path, an `http(s)://` URL, a base64 data URI, raw bytes,
265
+ a PIL image, a numpy array (HWC or HW) or a torch tensor (CHW), and normalises
266
+ it to RGB. It never guesses silently:
267
+
268
+ - pixel arrays must be `uint8` or float in `[0, 1]`; a float array in `0..255`,
269
+ an `int16` array, NaN or an empty array raise `ValueError` with a hint, instead
270
+ of turning into a white or wrapped-around picture;
271
+ - EXIF orientation is applied, as HuggingFace does: a phone photo reaches the
272
+ model upright, and boxes are in the frame people see.
273
+
274
+ ## FastAPI
275
+
276
+ A graph is an endpoint:
277
+
278
+ ```python
279
+ from contextlib import asynccontextmanager
280
+ from typing import Annotated
281
+
282
+ from fastapi import Body, FastAPI
283
+
284
+
285
+ @sp.graph
286
+ async def shelf_api(image: Annotated[Image, Body()]) -> dict[str, int]:
287
+ return await shelf(image)
288
+
289
+
290
+ @asynccontextmanager
291
+ async def lifespan(app: FastAPI):
292
+ yield
293
+ detector.close()
294
+ classifier.close()
295
+
296
+
297
+ app = FastAPI(lifespan=lifespan)
298
+ app.post("/shelf")(shelf_api)
299
+ ```
300
+
301
+ The request body is the image as a JSON string (a data URI); a bad image is a
302
+ 422. `Body()` is needed because an `Image` is a string in the JSON schema, and
303
+ FastAPI takes plain strings from the query string by default.
304
+
305
+ Concurrent requests share batches: with `max_batch_size` set, calls that arrive
306
+ while the model is busy run together in the next forward pass.
307
+
308
+ > **Do not expose this to untrusted clients yet.** A string given to `Image` may
309
+ > be a local file path or any URL, so a public endpoint can read files on the
310
+ > server or reach internal hosts. A mode for untrusted input is the next thing
311
+ > on the roadmap; until then, put the service behind your own authentication,
312
+ > accept only data URIs and check them yourself.
313
+
314
+ ## Your own task on a HuggingFace model
315
+
316
+ A task is a pair of schemas. Built-in tasks are ready objects; a custom one is a
317
+ line of code.
318
+
319
+ ```python
320
+ from typing import Any, Self
321
+
322
+ import singlepipe as sp
323
+ from singlepipe.adapters.hf_adapter import HFAdapter
324
+ from singlepipe.schemas.contract import OutputSchema
325
+ from singlepipe.schemas.image.base import ImageTaskInput
326
+
327
+
328
+ class IsCat(OutputSchema):
329
+ is_cat: bool
330
+ confidence: float
331
+
332
+ @classmethod
333
+ def from_pipeline_output(cls, raw: Any, **context: Any) -> Self:
334
+ cat = next((p for p in raw if "cat" in p["label"]), None)
335
+ return cls(is_cat=cat is not None, confidence=cat["score"] if cat else 0.0)
336
+
337
+
338
+ is_cat = sp.Task("is-cat", ImageTaskInput, IsCat)
339
+ adapter = HFAdapter("google/vit-base-patch16-224", is_cat, hf_task="image-classification")
340
+
341
+ answer = await adapter("cats.jpg")
342
+ answer.is_cat # typed: bool
343
+ ```
344
+
345
+ ## Your own adapter, any library
346
+
347
+ Everything except the backend call lives in `Adapter`: async calls, validation,
348
+ the worker thread, batching, `map`, its place in a graph, `close()`. A backend
349
+ implements one method.
350
+
351
+ ```python
352
+ from collections.abc import Sequence
353
+ from typing import Any
354
+
355
+ import numpy as np
356
+ import onnxruntime
357
+
358
+ import singlepipe as sp
359
+ from singlepipe.adapters.base import Adapter
360
+ from singlepipe.schemas.image.classification import (
361
+ ImageClassificationInput,
362
+ ImageClassificationOutput,
363
+ )
364
+
365
+
366
+ class OnnxClassifier(Adapter[ImageClassificationInput, ImageClassificationOutput]):
367
+ def __init__(self, path: str, labels: list[str], **batching: Any) -> None:
368
+ self.session = onnxruntime.InferenceSession(path) # load the model first,
369
+ self.labels = labels
370
+ super().__init__(sp.tasks.image_classification, name=path, **batching) # then start
371
+
372
+ def run_batch(self, inputs: Sequence[Any]) -> Sequence[Any]:
373
+ x = np.stack([preprocess(image) for image in inputs]) # your resize + normalise
374
+ probabilities = self.session.run(None, {"input": x})[0]
375
+ return [
376
+ [{"label": label, "score": float(p)} for label, p in zip(self.labels, row)]
377
+ for row in probabilities
378
+ ]
379
+ ```
380
+
381
+ `run_batch` receives what the input schema produces (here RGB PIL images) and
382
+ returns one raw output per input, in the format the output schema reads.
383
+ `name` is how the adapter appears in error notes and the run log; it defaults to
384
+ `ClassName(task)` (an `HFAdapter` uses the model id).
385
+
386
+ ## Performance
387
+
388
+ Measured on one Tesla T4 with `google/vit-base-patch16-224`:
389
+
390
+ - **The typed layer costs 0–0.7 ms per call** against a bare `transformers`
391
+ pipeline (16.6 ms per image in fp32).
392
+ - **fp16 doubles throughput** (×2.2–2.4): pass it to the pipeline,
393
+ `HFAdapter(..., dtype=torch.float16)`.
394
+ - **For a server, batch without waiting:** `max_batch_size=16` to `32` and the
395
+ default `max_wait_ms=0`. Batches grow by themselves when requests arrive while
396
+ the model is busy (30 of 32 on average at 64 concurrent callers): +54%
397
+ throughput and −35% latency in fp16. A wait window adds nothing under load
398
+ and slows down a lone caller.
399
+
400
+ ## What is there today
401
+
402
+ | | |
403
+ | --------- | ------------------------------------------------------------------------------- |
404
+ | Tasks | `image_classification`, `object_detection` |
405
+ | Backends | HuggingFace `transformers` pipelines; anything else via `Adapter` |
406
+ | Inputs | file path, URL, data URI, bytes, PIL image, numpy array, torch tensor |
407
+ | Execution | async calls, a worker thread per model, dynamic batching, `map` |
408
+ | Graphs | `@sp.graph`, `@sp.node`, `.map` on any node, FastAPI endpoints |
409
+ | Debugging | node paths in errors, run log (`graph.run`, `sp.steps_of`) |
410
+ | Glue | `Image.crop`, `origin`, `to_frame`; `BBox.scale`, `clip`, `iou`; `sp.ops.crops` |
411
+ | Typing | outputs typed for mypy without `cast` (`py.typed` included) |
412
+
413
+ ## Roadmap
414
+
415
+ - Safe mode for untrusted input in FastAPI (next)
416
+ - Text tasks: classification, named entities
417
+ - Audio: speech recognition, audio classification
418
+ - scikit-learn / XGBoost adapters and tabular tasks; PyTorch adapter
419
+ - `Connector`: picks the adapter from the model object
420
+ - A faster HuggingFace path that skips the pipeline's own overhead
421
+ - Later: a node as a remote service (`sp.serve`, `RemoteAdapter`)
422
+
423
+ ## License
424
+
425
+ MIT