singlepipe 0.2.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. {singlepipe-0.2.0/singlepipe.egg-info → singlepipe-0.3.0}/PKG-INFO +168 -73
  2. {singlepipe-0.2.0 → singlepipe-0.3.0}/README.md +167 -72
  3. {singlepipe-0.2.0 → singlepipe-0.3.0}/pyproject.toml +4 -2
  4. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/__init__.py +19 -1
  5. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/_fanout.py +53 -7
  6. singlepipe-0.3.0/singlepipe/_graph/__init__.py +12 -0
  7. singlepipe-0.3.0/singlepipe/_graph/_context.py +196 -0
  8. singlepipe-0.3.0/singlepipe/_graph/graph.py +152 -0
  9. singlepipe-0.3.0/singlepipe/_graph/node.py +294 -0
  10. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/adapters/base.py +27 -5
  11. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/adapters/hf_adapter.py +13 -4
  12. {singlepipe-0.2.0 → singlepipe-0.3.0/singlepipe.egg-info}/PKG-INFO +168 -73
  13. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe.egg-info/SOURCES.txt +4 -0
  14. {singlepipe-0.2.0 → singlepipe-0.3.0}/tests/test_package.py +9 -0
  15. {singlepipe-0.2.0 → singlepipe-0.3.0}/LICENSE +0 -0
  16. {singlepipe-0.2.0 → singlepipe-0.3.0}/setup.cfg +0 -0
  17. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/adapters/__init__.py +0 -0
  18. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/ops.py +0 -0
  19. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/py.typed +0 -0
  20. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/__init__.py +0 -0
  21. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/containers.py +0 -0
  22. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/contract.py +0 -0
  23. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/image/__init__.py +0 -0
  24. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/image/base.py +0 -0
  25. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/image/classification.py +0 -0
  26. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/image/detection.py +0 -0
  27. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/modalities/__init__.py +0 -0
  28. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/modalities/image.py +0 -0
  29. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/schemas/primitives.py +0 -0
  30. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe/tasks.py +0 -0
  31. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe.egg-info/dependency_links.txt +0 -0
  32. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe.egg-info/requires.txt +0 -0
  33. {singlepipe-0.2.0 → singlepipe-0.3.0}/singlepipe.egg-info/top_level.txt +0 -0
  34. {singlepipe-0.2.0 → singlepipe-0.3.0}/tests/test_fanout.py +0 -0
  35. {singlepipe-0.2.0 → singlepipe-0.3.0}/tests/test_ops.py +0 -0
  36. {singlepipe-0.2.0 → singlepipe-0.3.0}/tests/test_tasks.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: singlepipe
3
- Version: 0.2.0
3
+ Version: 0.3.0
4
4
  Summary: Typed, async, batching inference for ML models: pydantic schemas in and out, any backend
5
5
  Author-email: Gleb <glebov274@gmail.com>
6
6
  License-Expression: MIT
@@ -40,13 +40,14 @@ any backend underneath.
40
40
  SinglePipe does not reimplement HuggingFace, PyTorch or scikit-learn. It puts a
41
41
  typed boundary around them: inputs are validated and normalised, outputs are
42
42
  typed objects your editor and mypy understand, and the model runs off the event
43
- loop with dynamic batching — so several models can work together in one
44
- program or one FastAPI service.
43
+ loop with dynamic batching. Several models are composed into a graph — a plain
44
+ async function — that tells you where an error happened, records what every
45
+ model saw and returned, and serves as a FastAPI endpoint in one line.
45
46
 
46
- > **Status: alpha (0.2.0).** Image classification and object detection on
47
- > HuggingFace models, plus the glue between models: crops, coordinates and
48
- > fan-out. The API will change before 1.0; breaking changes are listed in
49
- > `CHANGELOG.md`.
47
+ > **Status: alpha (0.3.0).** Image classification and object detection on
48
+ > HuggingFace models, the glue between models (crops, coordinates, fan-out) and
49
+ > graphs with a run log. The API will change before 1.0; breaking changes are
50
+ > listed in `CHANGELOG.md`.
50
51
 
51
52
  ## Install
52
53
 
@@ -56,7 +57,7 @@ pip install "singlepipe[detr]" # + timm, for DETR detection models
56
57
  ```
57
58
 
58
59
  The core (`pip install singlepipe`) has no ML backend: schemas, tasks, image
59
- handling and glue only. Python 3.12+.
60
+ handling, glue and graphs only. Python 3.12+.
60
61
 
61
62
  ## Quick start
62
63
 
@@ -100,32 +101,45 @@ and are returned as the model gave them — even partly outside the image, on an
100
101
  side, as detectors do for objects at the edge. `box.clip(w, h)` cuts a box to
101
102
  the image; `image.crop(box)` does that by itself.
102
103
 
103
- ## Many models: detect, crop, classify
104
+ ## Many models: a graph
105
+
106
+ Detect bottles on a shelf, classify every bottle, count the brands:
104
107
 
105
108
  ```python
106
109
  from collections import Counter
107
110
 
111
+ import singlepipe as sp
112
+ from singlepipe.adapters.hf_adapter import HFAdapter
108
113
  from singlepipe.schemas.modalities import Image
109
114
 
110
115
  detector = HFAdapter("facebook/detr-resnet-50", sp.tasks.object_detection)
111
- classifier = HFAdapter(
112
- "my-org/sku-vit",
113
- sp.tasks.image_classification,
114
- max_batch_size=16,
115
- max_wait_ms=5, # dynamic batching
116
- )
116
+ classifier = HFAdapter("my-org/sku-vit", sp.tasks.image_classification, max_batch_size=16)
117
117
 
118
118
 
119
- async def shelf(path: str) -> Counter[str]:
120
- image = Image(path)
119
+ @sp.graph
120
+ async def shelf(image: Image) -> dict[str, int]:
121
121
  dets = await detector(image)
122
122
  bottles = dets.predictions.filter_by_label("bottle").filter_by_score(0.5)
123
+ results = await classifier.map(sp.ops.crops(image, bottles, scale=1.1))
124
+ return dict(Counter(r.predictions.top1.label for r in results if r.predictions.top1))
125
+
123
126
 
124
- crops = sp.ops.crops(image, bottles, scale=1.1) # one crop per box, a little context
125
- results = await classifier.map(crops) # batched: up to 16 crops per forward pass
126
- return Counter(r.predictions.top1.label for r in results if r.predictions.top1)
127
+ counts = await shelf(Image("shelf.jpg")) # {"cola": 7, "water": 3}
127
128
  ```
128
129
 
130
+ A graph is plain Python: loops, `if`, fallbacks and parallel branches
131
+ (`asyncio.TaskGroup`) need nothing special. What `@sp.graph` adds:
132
+
133
+ - **Boundaries.** Called from outside — a script, FastAPI — the arguments are
134
+ validated leniently: a path, URL, bytes or PIL image becomes an `Image` (mypy
135
+ still wants the annotated type, hence `Image(...)` above). Called from inside
136
+ another graph they are validated strictly: there the data is yours, and a
137
+ silent conversion would hide a bug. The return value is always checked.
138
+ - **Where an error happened** (below) and **what every model saw** (the run log).
139
+ - **One line to a service** (FastAPI, below).
140
+
141
+ In the graph:
142
+
129
143
  - **`classifier.map(items)`** sends every item at once, so the adapter batches
130
144
  them. A loop of `await classifier(c)` would send them one at a time and
131
145
  batching would never kick in. Results come back in input order.
@@ -135,18 +149,49 @@ async def shelf(path: str) -> Counter[str]:
135
149
  `[image.crop(d.box.scale(k)) for d in dets if ...]`.
136
150
  - A crop is clipped to the image and widened to whole pixels; nothing is padded.
137
151
 
138
- ### When one item fails
152
+ ### The run log
153
+
154
+ `shelf(image)` records nothing. `shelf.run(image)` returns the result together
155
+ with every step that produced it — each model call, node and nested graph, with
156
+ references to its inputs and output, in the order they finished:
157
+
158
+ ```python
159
+ run = await shelf.run(Image("shelf.jpg"))
160
+ for step in run.steps:
161
+ item = f"[{step.item}]" if step.item is not None else ""
162
+ print(" > ".join(step.path), item, type(step.output).__name__)
163
+ ```
164
+
165
+ ```
166
+ shelf > facebook/detr-resnet-50 ObjectDetectionOutput
167
+ shelf > my-org/sku-vit [2] ImageClassificationOutput
168
+ shelf > my-org/sku-vit [0] ImageClassificationOutput
169
+ shelf > my-org/sku-vit [1] ImageClassificationOutput
170
+ shelf dict
171
+ ```
172
+
173
+ `step.item` is the index of the `map` item the step ran for; `step.inputs`
174
+ holds the crop the classifier saw, `step.output` what it answered,
175
+ `step.seconds` how long it took.
176
+
177
+ ### When something fails
139
178
 
140
- By default the first failure cancels the remaining calls and is raised as is —
141
- a bad input is still a `pydantic.ValidationError` — with a note naming the item:
179
+ An error leaves the graph as it was raised — a bad input is still a
180
+ `pydantic.ValidationError`, so `except` clauses and FastAPI handlers keep
181
+ working — with notes saying where:
142
182
 
143
183
  ```
144
184
  pydantic_core._pydantic_core.ValidationError: 1 validation error for ...
145
- HFAdapter(task='image-classification').map: item at index 6 of 14
185
+ in node: shelf > my-org/sku-vit
186
+ my-org/sku-vit.map: item at index 6 of 14
146
187
  ```
147
188
 
148
- To keep going, ask for it explicitly; the result type says that an item may be
149
- an error, so mypy makes you handle it:
189
+ Inside `shelf.run(...)`, the steps recorded before the failure are on the
190
+ exception: `sp.steps_of(error)`.
191
+
192
+ By default the first failing item of a `map` cancels the remaining ones. To keep
193
+ going, ask for it explicitly; the result type says that an item may be an
194
+ error, so mypy makes you handle it:
150
195
 
151
196
  ```python
152
197
  results = await classifier.map(crops, errors="collect")
@@ -162,6 +207,32 @@ for crop, result in zip(crops, results):
162
207
  Which one is right depends on the task: a content filter must not pass an image
163
208
  it failed to check; a catalogue search can return 49 results out of 50.
164
209
 
210
+ ### Your own steps: `@sp.node`
211
+
212
+ Any function becomes a step of the graph — named in errors and recorded in the
213
+ run log:
214
+
215
+ ```python
216
+ @sp.node
217
+ def share(counts: dict[str, int], brand: str) -> float:
218
+ return counts.get(brand, 0) / max(sum(counts.values()), 1)
219
+
220
+
221
+ cola = share(counts, "cola")
222
+ per_shelf = share.map([counts_a, counts_b], brand="cola") # list[float]
223
+ ```
224
+
225
+ - Annotated arguments and the return value are validated strictly; an
226
+ unannotated argument is passed as it is. No annotation, no check — name a type
227
+ only where you want the guarantee.
228
+ - A plain function stays plain: called without `await`, its `map` returns a
229
+ list. Heavy synchronous work (pure-Python image processing, a big regex pass)
230
+ would freeze the event loop: mark it `@sp.node(thread=True)` and `await` it.
231
+ An `async def` node is awaited as usual.
232
+ - `.map(items, **shared)` on any node or graph works like `adapter.map`:
233
+ the item is the first argument, `shared` goes to every call, `errors="collect"`
234
+ is there too.
235
+
165
236
  ## Cascades: back to the original frame
166
237
 
167
238
  A crop remembers where it was cut (`crop.origin`), so a box found inside it maps
@@ -171,6 +242,7 @@ back to the original image:
171
242
  from singlepipe.schemas.primitives import BBox
172
243
 
173
244
 
245
+ @sp.graph
174
246
  async def find_plate(frame: Image) -> BBox | None:
175
247
  car = (await car_detector(frame)).predictions.filter_by_label("car").top1
176
248
  if car is None:
@@ -199,6 +271,46 @@ it to RGB. It never guesses silently:
199
271
  - EXIF orientation is applied, as HuggingFace does: a phone photo reaches the
200
272
  model upright, and boxes are in the frame people see.
201
273
 
274
+ ## FastAPI
275
+
276
+ A graph is an endpoint:
277
+
278
+ ```python
279
+ from contextlib import asynccontextmanager
280
+ from typing import Annotated
281
+
282
+ from fastapi import Body, FastAPI
283
+
284
+
285
+ @sp.graph
286
+ async def shelf_api(image: Annotated[Image, Body()]) -> dict[str, int]:
287
+ return await shelf(image)
288
+
289
+
290
+ @asynccontextmanager
291
+ async def lifespan(app: FastAPI):
292
+ yield
293
+ detector.close()
294
+ classifier.close()
295
+
296
+
297
+ app = FastAPI(lifespan=lifespan)
298
+ app.post("/shelf")(shelf_api)
299
+ ```
300
+
301
+ The request body is the image as a JSON string (a data URI); a bad image is a
302
+ 422. `Body()` is needed because an `Image` is a string in the JSON schema, and
303
+ FastAPI takes plain strings from the query string by default.
304
+
305
+ Concurrent requests share batches: with `max_batch_size` set, calls that arrive
306
+ while the model is busy run together in the next forward pass.
307
+
308
+ > **Do not expose this to untrusted clients yet.** A string given to `Image` may
309
+ > be a local file path or any URL, so a public endpoint can read files on the
310
+ > server or reach internal hosts. A mode for untrusted input is the next thing
311
+ > on the roadmap; until then, put the service behind your own authentication,
312
+ > accept only data URIs and check them yourself.
313
+
202
314
  ## Your own task on a HuggingFace model
203
315
 
204
316
  A task is a pair of schemas. Built-in tasks are ready objects; a custom one is a
@@ -233,7 +345,8 @@ answer.is_cat # typed: bool
233
345
  ## Your own adapter, any library
234
346
 
235
347
  Everything except the backend call lives in `Adapter`: async calls, validation,
236
- the worker thread, batching, `map`, `close()`. A backend implements one method.
348
+ the worker thread, batching, `map`, its place in a graph, `close()`. A backend
349
+ implements one method.
237
350
 
238
351
  ```python
239
352
  from collections.abc import Sequence
@@ -254,7 +367,7 @@ class OnnxClassifier(Adapter[ImageClassificationInput, ImageClassificationOutput
254
367
  def __init__(self, path: str, labels: list[str], **batching: Any) -> None:
255
368
  self.session = onnxruntime.InferenceSession(path) # load the model first,
256
369
  self.labels = labels
257
- super().__init__(sp.tasks.image_classification, **batching) # then start the worker
370
+ super().__init__(sp.tasks.image_classification, name=path, **batching) # then start
258
371
 
259
372
  def run_batch(self, inputs: Sequence[Any]) -> Sequence[Any]:
260
373
  x = np.stack([preprocess(image) for image in inputs]) # your resize + normalise
@@ -267,62 +380,44 @@ class OnnxClassifier(Adapter[ImageClassificationInput, ImageClassificationOutput
267
380
 
268
381
  `run_batch` receives what the input schema produces (here RGB PIL images) and
269
382
  returns one raw output per input, in the format the output schema reads.
383
+ `name` is how the adapter appears in error notes and the run log; it defaults to
384
+ `ClassName(task)` (an `HFAdapter` uses the model id).
270
385
 
271
- ## FastAPI
386
+ ## Performance
272
387
 
273
- ```python
274
- from contextlib import asynccontextmanager
275
-
276
- from fastapi import FastAPI
277
-
278
- from singlepipe.schemas.image.classification import ImageClassificationOutput
279
-
280
-
281
- @asynccontextmanager
282
- async def lifespan(app: FastAPI):
283
- app.state.classifier = HFAdapter(
284
- "google/vit-base-patch16-224",
285
- sp.tasks.image_classification,
286
- max_batch_size=32,
287
- max_wait_ms=10, # concurrent requests share batches
288
- )
289
- yield
290
- app.state.classifier.close()
291
-
292
-
293
- app = FastAPI(lifespan=lifespan)
388
+ Measured on one Tesla T4 with `google/vit-base-patch16-224`:
294
389
 
295
-
296
- @app.post("/classify")
297
- async def classify(url: str) -> ImageClassificationOutput:
298
- return await app.state.classifier(url)
299
- ```
300
-
301
- > **Do not expose this to untrusted clients yet.** A string given to `Image` may
302
- > be a local file path or any URL, so a public endpoint like this one can read
303
- > server files or reach internal hosts. A mode for untrusted input is planned;
304
- > until then, accept only data URIs or bytes and check them yourself.
390
+ - **The typed layer costs 0–0.7 ms per call** against a bare `transformers`
391
+ pipeline (16.6 ms per image in fp32).
392
+ - **fp16 doubles throughput** (×2.2–2.4): pass it to the pipeline,
393
+ `HFAdapter(..., dtype=torch.float16)`.
394
+ - **For a server, batch without waiting:** `max_batch_size=16` to `32` and the
395
+ default `max_wait_ms=0`. Batches grow by themselves when requests arrive while
396
+ the model is busy (30 of 32 on average at 64 concurrent callers): +54%
397
+ throughput and −35% latency in fp16. A wait window adds nothing under load
398
+ and slows down a lone caller.
305
399
 
306
400
  ## What is there today
307
401
 
308
- | | |
309
- | --------- | --------------------------------------------------------------------- |
310
- | Tasks | `image_classification`, `object_detection` |
311
- | Backends | HuggingFace `transformers` pipelines; anything else via `Adapter` |
312
- | Inputs | file path, URL, data URI, bytes, PIL image, numpy array, torch tensor |
313
- | Execution | async calls, a worker thread per model, dynamic batching, `map` |
402
+ | | |
403
+ | --------- | ------------------------------------------------------------------------------- |
404
+ | Tasks | `image_classification`, `object_detection` |
405
+ | Backends | HuggingFace `transformers` pipelines; anything else via `Adapter` |
406
+ | Inputs | file path, URL, data URI, bytes, PIL image, numpy array, torch tensor |
407
+ | Execution | async calls, a worker thread per model, dynamic batching, `map` |
408
+ | Graphs | `@sp.graph`, `@sp.node`, `.map` on any node, FastAPI endpoints |
409
+ | Debugging | node paths in errors, run log (`graph.run`, `sp.steps_of`) |
314
410
  | Glue | `Image.crop`, `origin`, `to_frame`; `BBox.scale`, `clip`, `iou`; `sp.ops.crops` |
315
- | Typing | outputs typed for mypy without `cast` (`py.typed` included) |
411
+ | Typing | outputs typed for mypy without `cast` (`py.typed` included) |
316
412
 
317
413
  ## Roadmap
318
414
 
319
- - `@sp.node` and `@sp.graph`: a graph is an async function — loops, fallbacks and
320
- parallel branches are plain Python — with a run log, node paths in errors,
321
- `.map` on any node and FastAPI endpoints
322
- - Safe mode for untrusted input in FastAPI
415
+ - Safe mode for untrusted input in FastAPI (next)
416
+ - Text tasks: classification, named entities
417
+ - Audio: speech recognition, audio classification
418
+ - scikit-learn / XGBoost adapters and tabular tasks; PyTorch adapter
323
419
  - `Connector`: picks the adapter from the model object
324
- - PyTorch and scikit-learn / XGBoost adapters, tabular tasks
325
- - Text and audio tasks
420
+ - A faster HuggingFace path that skips the pipeline's own overhead
326
421
  - Later: a node as a remote service (`sp.serve`, `RemoteAdapter`)
327
422
 
328
423
  ## License