stagepipe 0.0.2__tar.gz → 0.0.3__tar.gz

{stagepipe-0.0.2/src/stagepipe.egg-info → stagepipe-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: stagepipe
-Version: 0.0.2
+Version: 0.0.3
 Summary: Staged pipelines for blocking Python code: per-stage thread workers, no stage barrier, ordered stages, bounded in-flight. Standard library only.
 Author: Rafael Borja
 License-Expression: Apache-2.0
@@ -14,6 +14,11 @@ Classifier: Intended Audience :: Developers
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Typing :: Typed
 Requires-Python: >=3.10
@@ -23,6 +28,12 @@ Dynamic: license-file
 
 # stagepipe
 
+[![PyPI](https://img.shields.io/pypi/v/stagepipe)](https://pypi.org/project/stagepipe/)
+[![CI](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml/badge.svg)](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml)
+[![Python](https://img.shields.io/badge/python-3.10%20to%203.14-blue)](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml)
+[![License](https://img.shields.io/pypi/l/stagepipe)](https://github.com/rafaelborja/stagepipe/blob/main/LICENSE)
+[![Downloads](https://img.shields.io/pypi/dm/stagepipe)](https://pypistats.org/packages/stagepipe)
+
 **Run your slow, blocking steps in parallel, and let every item move on the moment the next step is free.**
 
 A tiny pipeline for plain Python: no event loop, no server, no dependencies, about 130 lines of
@@ -88,6 +99,11 @@ aiostream is excellent and well maintained, and most of that difference is simpl
 being loaded. If you already run an event loop it costs you nothing extra. If you do not, and
 memory is tight, a thread-and-queue design is the lighter tool. Reproduce with `bench/memory.py`.
 
+Continuous integration repeats the footprint measurement on Linux, Windows and macOS for every
+Python from 3.10 to 3.14 (see `bench/footprint.py` and the run summaries). Across those 15
+combinations, importing stagepipe cost between 1.6 and 2.6 MB of resident memory and the
+machinery cost between 2 and 17 microseconds per item per stage.
+
 **Seconds, not nanoseconds.** stagepipe is for pipelines whose steps take milliseconds to minutes
 (an OCR page, an API call, a transcode), where running in parallel saves *seconds*. Ten
 microseconds of bookkeeping per item disappears next to that. So whenever speed and footprint
@@ -171,12 +187,39 @@ docs = run(urls, [
 | `ordered=True` | The stage sees items in index order (needs `workers=1`). Use it when a stage carries state across items. Earlier stages still run out of order. |
 | `max_in_flight=M` | At most M items are being processed at any moment. Bounds memory and keeps stages balanced. |
 | `cancelled=callable` | Checked before every call; when it returns true the run stops and raises `Cancelled`. |
-| `on_done=callable` | Called with `(index, value)` as each item leaves the last stage. Good for progress bars. **It runs in a worker thread**, and concurrently if the last stage has several workers, so it must be thread-safe (or make the last stage `ordered`, which has exactly one worker). |
+| `on_done=callable` | Called with `(index, value)` as each item leaves the last stage. Good for progress bars. **It runs in a worker thread**, and concurrently if the last stage has several workers, so it must be thread-safe (or make the last stage `ordered`, which has exactly one worker). If it raises, the run is aborted and the exception is re-raised from `run()`, like an error in a stage. |
 
 If any stage raises, new work stops, workers drain, and the first exception is re-raised from `run()`.
 
 ---
 
+## Examples
+
+Two runnable scripts in [`examples/`](https://github.com/rafaelborja/stagepipe/tree/main/examples).
+Both run straight from a clone, with no install and no extra packages.
+
+**A pizza kitchen** (`examples/pizza_kitchen.py`), the playful one. Eight orders go through dough,
+toppings, two ovens and a boxing station. It prints when each pizza leaves, then a timeline of
+what every station was doing, and compares with a kitchen where each station finishes the whole
+batch first:
+
+```
+  dough    222223333355555...........788888....................................
+  toppings .....22222233333335555555......7888888..............................
+  oven     ...........111111111111113333333333333555555555555588888888888888...
+  box      .........................12...........34...........56...........788.
+
+Kitchen 1 finished in 2.02s, kitchen 2 in 3.25s: 1.6x faster, and the first pizza left
+after 0.78s instead of 3.25s.
+```
+
+**Compress a folder** (`examples/compress_folder.py`), the useful one. Read, gzip and write every
+file in a folder as separate stages, with `max_in_flight=6` so only six files are ever in memory,
+however big the folder is. On 24 files it ran about **4x faster than one file at a time**.
+Point it at your own folders with `python examples/compress_folder.py SRC OUT`.
+
+---
+
 ## Where it fits
 
 stagepipe is for **blocking work that spends its time waiting or in native code**, which is
@@ -225,7 +268,31 @@ These come from running stagepipe on a real five-stage document pipeline.
 
 - Not a job queue or scheduler: no broker, no workers on other machines, no persistence (yet, see the roadmap).
 - Not a workflow engine with branching graphs (yet): stages form a straight line.
-- Not async. If your code is already `async`, use [aiostream](https://github.com/vxgmichel/aiostream).
+- Not async: there is no event loop. If your code is already `async`, see the next section.
+
+---
+
+## Which one should I use?
+
+**If your code is already async, use [aiostream](https://github.com/vxgmichel/aiostream).**
+"Already async" means your steps are `async def` functions that you `await`: an async HTTP client,
+an async database driver, `asyncio` tasks. Your program already runs an event loop, and
+aiostream chains those steps on it directly. Pushing async code through threads would be a step
+backwards, and stagepipe would add nothing.
+
+**If your steps are ordinary blocking functions, use stagepipe.** OCR engines, onnxruntime,
+PyTorch, `requests`, PDF and image libraries, subprocesses: plain `def` functions you can call
+and test on their own. There is no event loop to start and nothing to wrap in `async`.
+
+Two more reasons to pick stagepipe, both about running small:
+
+- **Small memory footprint.** No event loop and no heavy imports. Loading `asyncio` alone costs
+  about 6 MB of resident memory on CPython 3.12; stagepipe is about 2 MB today and the roadmap
+  takes it to a few hundred KB. That matters in serverless functions and on small devices.
+- **Surviving a crash without a database** (planned for 0.2, not in 0.0.x yet). The design is
+  plain files: one small file per finished item and an append-only journal, with memory use that
+  does not grow with the number of items. aiostream has no persistence of any kind, so with it
+  you would build that yourself.
 
 ---
 
@@ -244,8 +311,8 @@ Checked on Python 3.12 in a clean virtual environment, with blocking functions.
 
 ## Status and known limits
 
-**Alpha, version 0.0.2.** It was extracted from a working document-processing pipeline where it
-replaces a hand-rolled look-ahead loop, and it passes its stress suite on CPython 3.12 and 3.15.
+**Alpha, version 0.0.3.** It was extracted from a working document-processing pipeline where it
+replaces a hand-rolled look-ahead loop, and it passes its stress suite in CI on CPython 3.10 to 3.14 on Linux, Windows and macOS (and locally on the 3.15 release candidate).
 In its first real use, five stages with different worker counts and at most 8 items in flight
 gave output identical to the sequential loop, and cancelling then resuming worked.
 
@@ -287,12 +354,16 @@ do not cover and that the older thread-based libraries have left behind.
     are handed over as they finish;
   - workers let go of the last item they handled as soon as they are idle;
   - optional smaller thread stacks for very small containers.
-- [ ] Failure policy per run: `collect` failed items and finish the rest, `skip`, or `retry(n, backoff)`.
+- [ ] Failure policy: `on_error` per stage or per run (`collect` failed items and finish the rest, `skip`,
+      `raise`, or `retry(n, backoff)`), so one bad item never kills a long job. *Asked for by both
+      production users, who each wrapped every stage function in try/except.*
+- [ ] Partial results on cancel: return what finished, with a marker for unfinished items.
 - [ ] Per-worker setup: `Stage(init=...)`, run once in each worker thread, e.g. one model
       session per worker. *Asked for by the first real user, who had to build it by hand with
       `threading.local()`.*
-- [ ] A bottleneck report after every run: per stage, busy time against waiting time, items
-      done and queue depth, so the stage to scale is obvious. *Also asked for by the first real
+- [ ] A bottleneck report after every run: per stage and per item, **queue wait time** (item ready, no
+      free worker) against run time, busy and idle totals, items done and queue depth, so the
+      stage to scale is obvious. Wait time cannot be measured from inside a stage function. *Also asked for by the first real
       user, who timed inside their stage functions.*
 
 ### Then: make it safe to rerun and easy to watch (0.2)
@@ -302,7 +373,9 @@ do not cover and that the older thread-based libraries have left behind.
 - [ ] **Events for dashboards**: a stable, versioned event stream (enter/exit per stage, worker,
       duration, queue depth) written as JSON lines or sent to a callback. A dashboard can be a
       separate project that just reads it.
-- [ ] Shared limits across stages, e.g. a `gpu=1` slot that three stages compete for.
+- [ ] Shared limits across stages, e.g. a `gpu=1` slot that three stages compete for, and
+      `Stage(limiter=<Semaphore>)` so one semaphore caps a resource across several runs in the same
+      process (three runs of 3 workers must not put 9 calls on a service that accepts 4).
 - [ ] A memory budget in bytes, not only an item count.
 
 ### Later: bigger shapes (0.3 and beyond)
@@ -315,8 +388,10 @@ do not cover and that the older thread-based libraries have left behind.
 - [ ] Verified on free-threaded Python builds.
 
 ### Python versions
-Developed against CPython 3.12 and tested on 3.15 (release candidate). Python 3.15's lazy
-imports (PEP 810) are on the list: the optional features above will load only when used, so
+Tested in CI on CPython 3.10, 3.11, 3.12, 3.13 and 3.14, on Linux, Windows and macOS, and the
+release workflow only publishes if the whole matrix passes. Python 3.15 works locally (release
+candidate) and joins the CI matrix and the release workflow as soon as GitHub's runners offer it.
+Python 3.15's lazy imports (PEP 810) are on the list: the optional features above will load only when used, so
 `import stagepipe` stays near-instant.
 
 Want something on this list sooner, or something that is missing? Open an
@@ -333,6 +408,8 @@ pip install -e .
 python tests/stress.py        # plain script, no pytest needed
 python bench/bench.py         # speed numbers above
 python bench/memory.py        # memory numbers above (needs psutil; aiostream optional)
+python bench/footprint.py     # memory and time on your Python and OS
+python examples/pizza_kitchen.py
 ```
 
 ## Changelog

{stagepipe-0.0.2 → stagepipe-0.0.3}/README.md

@@ -1,5 +1,11 @@
 # stagepipe
 
+[![PyPI](https://img.shields.io/pypi/v/stagepipe)](https://pypi.org/project/stagepipe/)
+[![CI](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml/badge.svg)](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml)
+[![Python](https://img.shields.io/badge/python-3.10%20to%203.14-blue)](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml)
+[![License](https://img.shields.io/pypi/l/stagepipe)](https://github.com/rafaelborja/stagepipe/blob/main/LICENSE)
+[![Downloads](https://img.shields.io/pypi/dm/stagepipe)](https://pypistats.org/packages/stagepipe)
+
 **Run your slow, blocking steps in parallel, and let every item move on the moment the next step is free.**
 
 A tiny pipeline for plain Python: no event loop, no server, no dependencies, about 130 lines of
@@ -65,6 +71,11 @@ aiostream is excellent and well maintained, and most of that difference is simpl
 being loaded. If you already run an event loop it costs you nothing extra. If you do not, and
 memory is tight, a thread-and-queue design is the lighter tool. Reproduce with `bench/memory.py`.
 
+Continuous integration repeats the footprint measurement on Linux, Windows and macOS for every
+Python from 3.10 to 3.14 (see `bench/footprint.py` and the run summaries). Across those 15
+combinations, importing stagepipe cost between 1.6 and 2.6 MB of resident memory and the
+machinery cost between 2 and 17 microseconds per item per stage.
+
 **Seconds, not nanoseconds.** stagepipe is for pipelines whose steps take milliseconds to minutes
 (an OCR page, an API call, a transcode), where running in parallel saves *seconds*. Ten
 microseconds of bookkeeping per item disappears next to that. So whenever speed and footprint
@@ -148,12 +159,39 @@ docs = run(urls, [
 | `ordered=True` | The stage sees items in index order (needs `workers=1`). Use it when a stage carries state across items. Earlier stages still run out of order. |
 | `max_in_flight=M` | At most M items are being processed at any moment. Bounds memory and keeps stages balanced. |
 | `cancelled=callable` | Checked before every call; when it returns true the run stops and raises `Cancelled`. |
-| `on_done=callable` | Called with `(index, value)` as each item leaves the last stage. Good for progress bars. **It runs in a worker thread**, and concurrently if the last stage has several workers, so it must be thread-safe (or make the last stage `ordered`, which has exactly one worker). |
+| `on_done=callable` | Called with `(index, value)` as each item leaves the last stage. Good for progress bars. **It runs in a worker thread**, and concurrently if the last stage has several workers, so it must be thread-safe (or make the last stage `ordered`, which has exactly one worker). If it raises, the run is aborted and the exception is re-raised from `run()`, like an error in a stage. |
 
 If any stage raises, new work stops, workers drain, and the first exception is re-raised from `run()`.
 
 ---
 
+## Examples
+
+Two runnable scripts in [`examples/`](https://github.com/rafaelborja/stagepipe/tree/main/examples).
+Both run straight from a clone, with no install and no extra packages.
+
+**A pizza kitchen** (`examples/pizza_kitchen.py`), the playful one. Eight orders go through dough,
+toppings, two ovens and a boxing station. It prints when each pizza leaves, then a timeline of
+what every station was doing, and compares with a kitchen where each station finishes the whole
+batch first:
+
+```
+  dough    222223333355555...........788888....................................
+  toppings .....22222233333335555555......7888888..............................
+  oven     ...........111111111111113333333333333555555555555588888888888888...
+  box      .........................12...........34...........56...........788.
+
+Kitchen 1 finished in 2.02s, kitchen 2 in 3.25s: 1.6x faster, and the first pizza left
+after 0.78s instead of 3.25s.
+```
+
+**Compress a folder** (`examples/compress_folder.py`), the useful one. Read, gzip and write every
+file in a folder as separate stages, with `max_in_flight=6` so only six files are ever in memory,
+however big the folder is. On 24 files it ran about **4x faster than one file at a time**.
+Point it at your own folders with `python examples/compress_folder.py SRC OUT`.
+
+---
+
 ## Where it fits
 
 stagepipe is for **blocking work that spends its time waiting or in native code**, which is
@@ -202,7 +240,31 @@ These come from running stagepipe on a real five-stage document pipeline.
 
 - Not a job queue or scheduler: no broker, no workers on other machines, no persistence (yet, see the roadmap).
 - Not a workflow engine with branching graphs (yet): stages form a straight line.
-- Not async. If your code is already `async`, use [aiostream](https://github.com/vxgmichel/aiostream).
+- Not async: there is no event loop. If your code is already `async`, see the next section.
+
+---
+
+## Which one should I use?
+
+**If your code is already async, use [aiostream](https://github.com/vxgmichel/aiostream).**
+"Already async" means your steps are `async def` functions that you `await`: an async HTTP client,
+an async database driver, `asyncio` tasks. Your program already runs an event loop, and
+aiostream chains those steps on it directly. Pushing async code through threads would be a step
+backwards, and stagepipe would add nothing.
+
+**If your steps are ordinary blocking functions, use stagepipe.** OCR engines, onnxruntime,
+PyTorch, `requests`, PDF and image libraries, subprocesses: plain `def` functions you can call
+and test on their own. There is no event loop to start and nothing to wrap in `async`.
+
+Two more reasons to pick stagepipe, both about running small:
+
+- **Small memory footprint.** No event loop and no heavy imports. Loading `asyncio` alone costs
+  about 6 MB of resident memory on CPython 3.12; stagepipe is about 2 MB today and the roadmap
+  takes it to a few hundred KB. That matters in serverless functions and on small devices.
+- **Surviving a crash without a database** (planned for 0.2, not in 0.0.x yet). The design is
+  plain files: one small file per finished item and an append-only journal, with memory use that
+  does not grow with the number of items. aiostream has no persistence of any kind, so with it
+  you would build that yourself.
 
 ---
 
@@ -221,8 +283,8 @@ Checked on Python 3.12 in a clean virtual environment, with blocking functions.
 
 ## Status and known limits
 
-**Alpha, version 0.0.2.** It was extracted from a working document-processing pipeline where it
-replaces a hand-rolled look-ahead loop, and it passes its stress suite on CPython 3.12 and 3.15.
+**Alpha, version 0.0.3.** It was extracted from a working document-processing pipeline where it
+replaces a hand-rolled look-ahead loop, and it passes its stress suite in CI on CPython 3.10 to 3.14 on Linux, Windows and macOS (and locally on the 3.15 release candidate).
 In its first real use, five stages with different worker counts and at most 8 items in flight
 gave output identical to the sequential loop, and cancelling then resuming worked.
 
@@ -264,12 +326,16 @@ do not cover and that the older thread-based libraries have left behind.
     are handed over as they finish;
   - workers let go of the last item they handled as soon as they are idle;
   - optional smaller thread stacks for very small containers.
-- [ ] Failure policy per run: `collect` failed items and finish the rest, `skip`, or `retry(n, backoff)`.
+- [ ] Failure policy: `on_error` per stage or per run (`collect` failed items and finish the rest, `skip`,
+      `raise`, or `retry(n, backoff)`), so one bad item never kills a long job. *Asked for by both
+      production users, who each wrapped every stage function in try/except.*
+- [ ] Partial results on cancel: return what finished, with a marker for unfinished items.
 - [ ] Per-worker setup: `Stage(init=...)`, run once in each worker thread, e.g. one model
       session per worker. *Asked for by the first real user, who had to build it by hand with
       `threading.local()`.*
-- [ ] A bottleneck report after every run: per stage, busy time against waiting time, items
-      done and queue depth, so the stage to scale is obvious. *Also asked for by the first real
+- [ ] A bottleneck report after every run: per stage and per item, **queue wait time** (item ready, no
+      free worker) against run time, busy and idle totals, items done and queue depth, so the
+      stage to scale is obvious. Wait time cannot be measured from inside a stage function. *Also asked for by the first real
       user, who timed inside their stage functions.*
 
 ### Then: make it safe to rerun and easy to watch (0.2)
@@ -279,7 +345,9 @@ do not cover and that the older thread-based libraries have left behind.
 - [ ] **Events for dashboards**: a stable, versioned event stream (enter/exit per stage, worker,
       duration, queue depth) written as JSON lines or sent to a callback. A dashboard can be a
       separate project that just reads it.
-- [ ] Shared limits across stages, e.g. a `gpu=1` slot that three stages compete for.
+- [ ] Shared limits across stages, e.g. a `gpu=1` slot that three stages compete for, and
+      `Stage(limiter=<Semaphore>)` so one semaphore caps a resource across several runs in the same
+      process (three runs of 3 workers must not put 9 calls on a service that accepts 4).
 - [ ] A memory budget in bytes, not only an item count.
 
 ### Later: bigger shapes (0.3 and beyond)
@@ -292,8 +360,10 @@ do not cover and that the older thread-based libraries have left behind.
 - [ ] Verified on free-threaded Python builds.
 
 ### Python versions
-Developed against CPython 3.12 and tested on 3.15 (release candidate). Python 3.15's lazy
-imports (PEP 810) are on the list: the optional features above will load only when used, so
+Tested in CI on CPython 3.10, 3.11, 3.12, 3.13 and 3.14, on Linux, Windows and macOS, and the
+release workflow only publishes if the whole matrix passes. Python 3.15 works locally (release
+candidate) and joins the CI matrix and the release workflow as soon as GitHub's runners offer it.
+Python 3.15's lazy imports (PEP 810) are on the list: the optional features above will load only when used, so
 `import stagepipe` stays near-instant.
 
 Want something on this list sooner, or something that is missing? Open an
@@ -310,6 +380,8 @@ pip install -e .
 python tests/stress.py        # plain script, no pytest needed
 python bench/bench.py         # speed numbers above
 python bench/memory.py        # memory numbers above (needs psutil; aiostream optional)
+python bench/footprint.py     # memory and time on your Python and OS
+python examples/pizza_kitchen.py
 ```
 
 ## Changelog

{stagepipe-0.0.2 → stagepipe-0.0.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "stagepipe"
-version = "0.0.2"
+version = "0.0.3"
 description = "Staged pipelines for blocking Python code: per-stage thread workers, no stage barrier, ordered stages, bounded in-flight. Standard library only."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -18,6 +18,11 @@ classifiers = [
   "Operating System :: OS Independent",
   "Programming Language :: Python :: 3",
   "Programming Language :: Python :: 3 :: Only",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Programming Language :: Python :: 3.14",
   "Topic :: Software Development :: Libraries :: Python Modules",
   "Typing :: Typed",
 ]

{stagepipe-0.0.2 → stagepipe-0.0.3}/src/stagepipe/__init__.py

@@ -2,4 +2,4 @@
 from .pipeline import Cancelled, Stage, run
 
 __all__ = ["Cancelled", "Stage", "run"]
-__version__ = "0.0.2"
+__version__ = "0.0.3"

{stagepipe-0.0.2 → stagepipe-0.0.3/src/stagepipe.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: stagepipe
-Version: 0.0.2
+Version: 0.0.3
 Summary: Staged pipelines for blocking Python code: per-stage thread workers, no stage barrier, ordered stages, bounded in-flight. Standard library only.
 Author: Rafael Borja
 License-Expression: Apache-2.0
@@ -14,6 +14,11 @@ Classifier: Intended Audience :: Developers
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Typing :: Typed
 Requires-Python: >=3.10
@@ -23,6 +28,12 @@ Dynamic: license-file
 
 # stagepipe
 
+[![PyPI](https://img.shields.io/pypi/v/stagepipe)](https://pypi.org/project/stagepipe/)
+[![CI](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml/badge.svg)](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml)
+[![Python](https://img.shields.io/badge/python-3.10%20to%203.14-blue)](https://github.com/rafaelborja/stagepipe/actions/workflows/ci.yml)
+[![License](https://img.shields.io/pypi/l/stagepipe)](https://github.com/rafaelborja/stagepipe/blob/main/LICENSE)
+[![Downloads](https://img.shields.io/pypi/dm/stagepipe)](https://pypistats.org/packages/stagepipe)
+
 **Run your slow, blocking steps in parallel, and let every item move on the moment the next step is free.**
 
 A tiny pipeline for plain Python: no event loop, no server, no dependencies, about 130 lines of
@@ -88,6 +99,11 @@ aiostream is excellent and well maintained, and most of that difference is simpl
 being loaded. If you already run an event loop it costs you nothing extra. If you do not, and
 memory is tight, a thread-and-queue design is the lighter tool. Reproduce with `bench/memory.py`.
 
+Continuous integration repeats the footprint measurement on Linux, Windows and macOS for every
+Python from 3.10 to 3.14 (see `bench/footprint.py` and the run summaries). Across those 15
+combinations, importing stagepipe cost between 1.6 and 2.6 MB of resident memory and the
+machinery cost between 2 and 17 microseconds per item per stage.
+
 **Seconds, not nanoseconds.** stagepipe is for pipelines whose steps take milliseconds to minutes
 (an OCR page, an API call, a transcode), where running in parallel saves *seconds*. Ten
 microseconds of bookkeeping per item disappears next to that. So whenever speed and footprint
@@ -171,12 +187,39 @@ docs = run(urls, [
 | `ordered=True` | The stage sees items in index order (needs `workers=1`). Use it when a stage carries state across items. Earlier stages still run out of order. |
 | `max_in_flight=M` | At most M items are being processed at any moment. Bounds memory and keeps stages balanced. |
 | `cancelled=callable` | Checked before every call; when it returns true the run stops and raises `Cancelled`. |
-| `on_done=callable` | Called with `(index, value)` as each item leaves the last stage. Good for progress bars. **It runs in a worker thread**, and concurrently if the last stage has several workers, so it must be thread-safe (or make the last stage `ordered`, which has exactly one worker). |
+| `on_done=callable` | Called with `(index, value)` as each item leaves the last stage. Good for progress bars. **It runs in a worker thread**, and concurrently if the last stage has several workers, so it must be thread-safe (or make the last stage `ordered`, which has exactly one worker). If it raises, the run is aborted and the exception is re-raised from `run()`, like an error in a stage. |
 
 If any stage raises, new work stops, workers drain, and the first exception is re-raised from `run()`.
 
 ---
 
+## Examples
+
+Two runnable scripts in [`examples/`](https://github.com/rafaelborja/stagepipe/tree/main/examples).
+Both run straight from a clone, with no install and no extra packages.
+
+**A pizza kitchen** (`examples/pizza_kitchen.py`), the playful one. Eight orders go through dough,
+toppings, two ovens and a boxing station. It prints when each pizza leaves, then a timeline of
+what every station was doing, and compares with a kitchen where each station finishes the whole
+batch first:
+
+```
+  dough    222223333355555...........788888....................................
+  toppings .....22222233333335555555......7888888..............................
+  oven     ...........111111111111113333333333333555555555555588888888888888...
+  box      .........................12...........34...........56...........788.
+
+Kitchen 1 finished in 2.02s, kitchen 2 in 3.25s: 1.6x faster, and the first pizza left
+after 0.78s instead of 3.25s.
+```
+
+**Compress a folder** (`examples/compress_folder.py`), the useful one. Read, gzip and write every
+file in a folder as separate stages, with `max_in_flight=6` so only six files are ever in memory,
+however big the folder is. On 24 files it ran about **4x faster than one file at a time**.
+Point it at your own folders with `python examples/compress_folder.py SRC OUT`.
+
+---
+
 ## Where it fits
 
 stagepipe is for **blocking work that spends its time waiting or in native code**, which is
@@ -225,7 +268,31 @@ These come from running stagepipe on a real five-stage document pipeline.
 
 - Not a job queue or scheduler: no broker, no workers on other machines, no persistence (yet, see the roadmap).
 - Not a workflow engine with branching graphs (yet): stages form a straight line.
-- Not async. If your code is already `async`, use [aiostream](https://github.com/vxgmichel/aiostream).
+- Not async: there is no event loop. If your code is already `async`, see the next section.
+
+---
+
+## Which one should I use?
+
+**If your code is already async, use [aiostream](https://github.com/vxgmichel/aiostream).**
+"Already async" means your steps are `async def` functions that you `await`: an async HTTP client,
+an async database driver, `asyncio` tasks. Your program already runs an event loop, and
+aiostream chains those steps on it directly. Pushing async code through threads would be a step
+backwards, and stagepipe would add nothing.
+
+**If your steps are ordinary blocking functions, use stagepipe.** OCR engines, onnxruntime,
+PyTorch, `requests`, PDF and image libraries, subprocesses: plain `def` functions you can call
+and test on their own. There is no event loop to start and nothing to wrap in `async`.
+
+Two more reasons to pick stagepipe, both about running small:
+
+- **Small memory footprint.** No event loop and no heavy imports. Loading `asyncio` alone costs
+  about 6 MB of resident memory on CPython 3.12; stagepipe is about 2 MB today and the roadmap
+  takes it to a few hundred KB. That matters in serverless functions and on small devices.
+- **Surviving a crash without a database** (planned for 0.2, not in 0.0.x yet). The design is
+  plain files: one small file per finished item and an append-only journal, with memory use that
+  does not grow with the number of items. aiostream has no persistence of any kind, so with it
+  you would build that yourself.
 
 ---
 
@@ -244,8 +311,8 @@ Checked on Python 3.12 in a clean virtual environment, with blocking functions.
 
 ## Status and known limits
 
-**Alpha, version 0.0.2.** It was extracted from a working document-processing pipeline where it
-replaces a hand-rolled look-ahead loop, and it passes its stress suite on CPython 3.12 and 3.15.
+**Alpha, version 0.0.3.** It was extracted from a working document-processing pipeline where it
+replaces a hand-rolled look-ahead loop, and it passes its stress suite in CI on CPython 3.10 to 3.14 on Linux, Windows and macOS (and locally on the 3.15 release candidate).
 In its first real use, five stages with different worker counts and at most 8 items in flight
 gave output identical to the sequential loop, and cancelling then resuming worked.
 
@@ -287,12 +354,16 @@ do not cover and that the older thread-based libraries have left behind.
     are handed over as they finish;
   - workers let go of the last item they handled as soon as they are idle;
   - optional smaller thread stacks for very small containers.
-- [ ] Failure policy per run: `collect` failed items and finish the rest, `skip`, or `retry(n, backoff)`.
+- [ ] Failure policy: `on_error` per stage or per run (`collect` failed items and finish the rest, `skip`,
+      `raise`, or `retry(n, backoff)`), so one bad item never kills a long job. *Asked for by both
+      production users, who each wrapped every stage function in try/except.*
+- [ ] Partial results on cancel: return what finished, with a marker for unfinished items.
 - [ ] Per-worker setup: `Stage(init=...)`, run once in each worker thread, e.g. one model
       session per worker. *Asked for by the first real user, who had to build it by hand with
       `threading.local()`.*
-- [ ] A bottleneck report after every run: per stage, busy time against waiting time, items
-      done and queue depth, so the stage to scale is obvious. *Also asked for by the first real
+- [ ] A bottleneck report after every run: per stage and per item, **queue wait time** (item ready, no
+      free worker) against run time, busy and idle totals, items done and queue depth, so the
+      stage to scale is obvious. Wait time cannot be measured from inside a stage function. *Also asked for by the first real
       user, who timed inside their stage functions.*
 
 ### Then: make it safe to rerun and easy to watch (0.2)
@@ -302,7 +373,9 @@ do not cover and that the older thread-based libraries have left behind.
 - [ ] **Events for dashboards**: a stable, versioned event stream (enter/exit per stage, worker,
       duration, queue depth) written as JSON lines or sent to a callback. A dashboard can be a
       separate project that just reads it.
-- [ ] Shared limits across stages, e.g. a `gpu=1` slot that three stages compete for.
+- [ ] Shared limits across stages, e.g. a `gpu=1` slot that three stages compete for, and
+      `Stage(limiter=<Semaphore>)` so one semaphore caps a resource across several runs in the same
+      process (three runs of 3 workers must not put 9 calls on a service that accepts 4).
 - [ ] A memory budget in bytes, not only an item count.
 
 ### Later: bigger shapes (0.3 and beyond)
@@ -315,8 +388,10 @@ do not cover and that the older thread-based libraries have left behind.
 - [ ] Verified on free-threaded Python builds.
 
 ### Python versions
-Developed against CPython 3.12 and tested on 3.15 (release candidate). Python 3.15's lazy
-imports (PEP 810) are on the list: the optional features above will load only when used, so
+Tested in CI on CPython 3.10, 3.11, 3.12, 3.13 and 3.14, on Linux, Windows and macOS, and the
+release workflow only publishes if the whole matrix passes. Python 3.15 works locally (release
+candidate) and joins the CI matrix and the release workflow as soon as GitHub's runners offer it.
+Python 3.15's lazy imports (PEP 810) are on the list: the optional features above will load only when used, so
 `import stagepipe` stays near-instant.
 
 Want something on this list sooner, or something that is missing? Open an
@@ -333,6 +408,8 @@ pip install -e .
 python tests/stress.py        # plain script, no pytest needed
 python bench/bench.py         # speed numbers above
 python bench/memory.py        # memory numbers above (needs psutil; aiostream optional)
+python bench/footprint.py     # memory and time on your Python and OS
+python examples/pizza_kitchen.py
 ```
 
 ## Changelog