flowevent 1.2__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- eventflow/__init__.py +123 -0
- eventflow/_callbacks.py +22 -0
- eventflow/_lifecycle.py +31 -0
- eventflow/actions.py +53 -0
- eventflow/async_event_bus.py +640 -0
- eventflow/base.py +180 -0
- eventflow/callback.py +318 -0
- eventflow/compatibility.py +14 -0
- eventflow/context.py +103 -0
- eventflow/contracts.py +40 -0
- eventflow/engine.py +466 -0
- eventflow/errors.py +21 -0
- eventflow/event_bus.py +594 -0
- eventflow/filtering.py +57 -0
- eventflow/history.py +111 -0
- eventflow/observability.py +133 -0
- eventflow/pipeline/__init__.py +7 -0
- eventflow/pipeline/async_.py +129 -0
- eventflow/pipeline/base.py +203 -0
- eventflow/pipeline/contracts.py +24 -0
- eventflow/pipeline/registry.py +95 -0
- eventflow/pipeline/sync.py +92 -0
- eventflow/py.typed +0 -0
- eventflow/scopes.py +185 -0
- eventflow/stream.py +241 -0
- eventflow/subscription.py +116 -0
- flowevent-1.2.dist-info/METADATA +1171 -0
- flowevent-1.2.dist-info/RECORD +31 -0
- flowevent-1.2.dist-info/WHEEL +5 -0
- flowevent-1.2.dist-info/licenses/LICENSE +21 -0
- flowevent-1.2.dist-info/top_level.txt +1 -0
|
@@ -0,0 +1,1171 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: flowevent
|
|
3
|
+
Version: 1.2
|
|
4
|
+
Summary: EventFlow: synchronous and asynchronous event delivery for Python.
|
|
5
|
+
Author-email: TahsinCr <Tahsincr@outlook.com>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/TahsinCr/python-eventflow
|
|
8
|
+
Project-URL: Repository, https://github.com/TahsinCr/python-eventflow
|
|
9
|
+
Project-URL: Changelog, https://github.com/TahsinCr/python-eventflow/blob/main/CHANGELOG.md
|
|
10
|
+
Project-URL: Issues, https://github.com/TahsinCr/python-eventflow/issues
|
|
11
|
+
Keywords: event-bus,event-driven,asyncio,pub-sub
|
|
12
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Classifier: Framework :: AsyncIO
|
|
22
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
23
|
+
Requires-Python: >=3.10
|
|
24
|
+
Description-Content-Type: text/markdown
|
|
25
|
+
License-File: LICENSE
|
|
26
|
+
Requires-Dist: backports.strenum<2,>=1.3.1; python_version < "3.11"
|
|
27
|
+
Requires-Dist: exceptiongroup<2,>=1.3.1; python_version < "3.11"
|
|
28
|
+
Provides-Extra: dev
|
|
29
|
+
Requires-Dist: pyright<2,>=1.1.400; extra == "dev"
|
|
30
|
+
Requires-Dist: ruff<1,>=0.12; extra == "dev"
|
|
31
|
+
Dynamic: license-file
|
|
32
|
+
|
|
33
|
+
[![Version: v1.2][version-shield]](https://github.com/TahsinCr/python-eventflow/blob/main/CHANGELOG.md)
|
|
34
|
+
[![Python 3.10+][python-shield]](https://www.python.org/downloads/)
|
|
35
|
+
[![MIT License][license-shield]](https://github.com/TahsinCr/python-eventflow/blob/main/LICENSE)
|
|
36
|
+
|
|
37
|
+
**English** | [Türkçe](https://github.com/TahsinCr/python-eventflow/blob/main/README-TR.md)
|
|
38
|
+
|
|
39
|
+
# EventFlow
|
|
40
|
+
|
|
41
|
+
[Repository](https://github.com/TahsinCr/python-eventflow) · [Latest release](https://github.com/TahsinCr/python-eventflow/releases/latest) · [MIT License](https://github.com/TahsinCr/python-eventflow/blob/main/LICENSE)
|
|
42
|
+
|
|
43
|
+
EventFlow lets components in one Python process communicate through ordinary
|
|
44
|
+
Python objects. It provides a synchronous event bus, an asyncio-native event
|
|
45
|
+
bus, optional in-memory history, named channels, and explicit subscription
|
|
46
|
+
lifecycle management.
|
|
47
|
+
|
|
48
|
+
Optional pipelines transform values through named stages and around middleware.
|
|
49
|
+
Async streams compose lazy transformations with bounded buffering. Scopes,
|
|
50
|
+
event envelopes and opt-in measurements let these pieces work together without
|
|
51
|
+
requiring a framework or changing ordinary bus publication.
|
|
52
|
+
|
|
53
|
+
The library is for in-process delivery. It does not persist events, serialize
|
|
54
|
+
them, or transport them between processes. Those responsibilities remain with
|
|
55
|
+
the application or its messaging infrastructure.
|
|
56
|
+
|
|
57
|
+
EventFlow was extracted from the event subsystem of
|
|
58
|
+
[Vehicle Autonomy Core](https://github.com/TahsinCr/vehicle-autonomy-core) and
|
|
59
|
+
is maintained here as a standalone library. The source project is not needed
|
|
60
|
+
to install or use this package.
|
|
61
|
+
|
|
62
|
+
## Contents
|
|
63
|
+
|
|
64
|
+
- [Getting started](#getting-started)
|
|
65
|
+
- [Install](#install)
|
|
66
|
+
- [Version](#version)
|
|
67
|
+
- [Publish your first event](#first-event)
|
|
68
|
+
- [Choose a bus](#choose-a-bus)
|
|
69
|
+
- [Core concepts](#core-concepts)
|
|
70
|
+
- [Events and subscriptions](#subscriptions)
|
|
71
|
+
- [Filtering and delivery limits](#filtering)
|
|
72
|
+
- [Failures and results](#failures)
|
|
73
|
+
- [Working with events](#working-with-events)
|
|
74
|
+
- [History and replay](#history)
|
|
75
|
+
- [Waiting for an event](#wait-for-event)
|
|
76
|
+
- [Periodic publishing](#periodic)
|
|
77
|
+
- [Async applications](#async-applications)
|
|
78
|
+
- [Organizing channels](#channels)
|
|
79
|
+
- [Callback policies](#callback-policies)
|
|
80
|
+
- [Pipelines and middleware](#pipelines)
|
|
81
|
+
- [Async pipelines and streams](#streams)
|
|
82
|
+
- [Subscription scopes](#scopes)
|
|
83
|
+
- [Event context and measurements](#event-context)
|
|
84
|
+
- [Security and operational boundaries](#security-and-operational-boundaries)
|
|
85
|
+
- [Composing the components](#composition)
|
|
86
|
+
- [Lifecycle and reference](#reference)
|
|
87
|
+
- [Defaults](#defaults)
|
|
88
|
+
- [Bus methods](#bus-api)
|
|
89
|
+
- [Engine methods](#engine-api)
|
|
90
|
+
- [Public types and errors](#public-types)
|
|
91
|
+
- [Development and license](#development)
|
|
92
|
+
|
|
93
|
+
## Getting started
|
|
94
|
+
|
|
95
|
+
### Install
|
|
96
|
+
|
|
97
|
+
Python 3.10 or newer is required. From a checkout, install the source into the
|
|
98
|
+
active environment:
|
|
99
|
+
|
|
100
|
+
~~~bash
|
|
101
|
+
python -m pip install .
|
|
102
|
+
~~~
|
|
103
|
+
|
|
104
|
+
After the PyPI release is published, install EventFlow with:
|
|
105
|
+
|
|
106
|
+
~~~bash
|
|
107
|
+
python -m pip install flowevent
|
|
108
|
+
~~~
|
|
109
|
+
|
|
110
|
+
`flowevent` is the PyPI distribution name. The library is called EventFlow,
|
|
111
|
+
and Python code continues to import the `eventflow` package:
|
|
112
|
+
|
|
113
|
+
~~~python
|
|
114
|
+
from eventflow import EventBus
|
|
115
|
+
~~~
|
|
116
|
+
|
|
117
|
+
Python 3.10 installs `backports.strenum` and `exceptiongroup` to supply features
|
|
118
|
+
that are built into Python 3.11 and newer. Package metadata selects these
|
|
119
|
+
dependencies only on Python 3.10.
|
|
120
|
+
|
|
121
|
+
The distribution includes inline type annotations and a `py.typed` marker, so
|
|
122
|
+
type checkers can discover EventFlow's types when the package is installed.
|
|
123
|
+
|
|
124
|
+
Because the distribution installs files in the shared `eventflow` import
|
|
125
|
+
package, installing another distribution that provides that package in the
|
|
126
|
+
same environment can overwrite or conflict with these files. Use a separate
|
|
127
|
+
virtual environment when both distributions are needed.
|
|
128
|
+
|
|
129
|
+
For editable development, install the optional tools and run the repository
|
|
130
|
+
checks from the project root:
|
|
131
|
+
|
|
132
|
+
~~~bash
|
|
133
|
+
python -m pip install -e ".[dev]"
|
|
134
|
+
python -m unittest discover -s tests -v
|
|
135
|
+
ruff check .
|
|
136
|
+
pyright
|
|
137
|
+
~~~
|
|
138
|
+
|
|
139
|
+
The `dev` extra provides Ruff and Pyright. The tests use Python's standard
|
|
140
|
+
`unittest` framework.
|
|
141
|
+
|
|
142
|
+
### Version
|
|
143
|
+
|
|
144
|
+
The installed package exposes its version through `eventflow.__version__`:
|
|
145
|
+
|
|
146
|
+
~~~python
|
|
147
|
+
from eventflow import __version__
|
|
148
|
+
|
|
149
|
+
print(__version__) # 1.2
|
|
150
|
+
~~~
|
|
151
|
+
|
|
152
|
+
<a id="first-event"></a>
|
|
153
|
+
### Publish your first event
|
|
154
|
+
|
|
155
|
+
An event is a value that describes something that has happened. It can be any
|
|
156
|
+
Python object; a frozen dataclass is a useful choice when the event should be
|
|
157
|
+
explicit and immutable.
|
|
158
|
+
|
|
159
|
+
~~~python
|
|
160
|
+
from dataclasses import dataclass
|
|
161
|
+
|
|
162
|
+
from eventflow import EventBus
|
|
163
|
+
|
|
164
|
+
|
|
165
|
+
@dataclass(frozen=True)
|
|
166
|
+
class OrderPlaced:
|
|
167
|
+
order_id: str
|
|
168
|
+
|
|
169
|
+
|
|
170
|
+
def send_confirmation(event: OrderPlaced) -> None:
|
|
171
|
+
print(f"Order placed: {event.order_id}")
|
|
172
|
+
|
|
173
|
+
|
|
174
|
+
with EventBus[OrderPlaced]() as bus:
|
|
175
|
+
with bus.subscribe(send_confirmation):
|
|
176
|
+
result = bus.publish(OrderPlaced("order-42"))
|
|
177
|
+
print(f"delivered={result.delivered}, failed={result.failed}")
|
|
178
|
+
~~~
|
|
179
|
+
|
|
180
|
+
Output:
|
|
181
|
+
|
|
182
|
+
~~~text
|
|
183
|
+
Order placed: order-42
|
|
184
|
+
delivered=1, failed=0
|
|
185
|
+
~~~
|
|
186
|
+
|
|
187
|
+
The publisher creates an `OrderPlaced` value and passes it to the bus. The bus
|
|
188
|
+
calls the subscribed handler before `publish()` returns. The inner context
|
|
189
|
+
manager cancels the subscription when it ends; the outer one closes the bus.
|
|
190
|
+
This makes the lifetime of both resources visible at the point of use.
|
|
191
|
+
|
|
192
|
+
The generic annotation in `EventBus[OrderPlaced]` helps type checkers and IDEs.
|
|
193
|
+
It does not validate event types at runtime. Use a filter when a runtime type
|
|
194
|
+
check is needed.
|
|
195
|
+
|
|
196
|
+
### Choose a bus
|
|
197
|
+
|
|
198
|
+
| | `EventBus` | `AsyncEventBus` |
|
|
199
|
+
| --- | --- | --- |
|
|
200
|
+
| Handler | Synchronous callable | Async callable |
|
|
201
|
+
| Publish | Blocks until selected handlers finish | Await `publish()` |
|
|
202
|
+
| Execution context | Calling thread by default; optional executor | One owning asyncio event loop |
|
|
203
|
+
| Delivery order | Concurrent publisher calls may overlap | Sequential by default; optional concurrent delivery |
|
|
204
|
+
|
|
205
|
+
Choose `EventBus` for synchronous code. By default, each handler runs on the
|
|
206
|
+
thread that called `publish()`. An optional `concurrent.futures.ThreadPoolExecutor`
|
|
207
|
+
can run handlers on worker threads, but the publishing call still waits for them.
|
|
208
|
+
When buses share a pool, a callback that publishes through another bus using
|
|
209
|
+
that same pool delivers the nested event inline on the active worker. This
|
|
210
|
+
prevents worker starvation during cross-bus reentrant publication.
|
|
211
|
+
|
|
212
|
+
Publishing from multiple threads is safe. Two simultaneous publish calls may
|
|
213
|
+
run their handlers at the same time, so handlers that update shared state must
|
|
214
|
+
provide their own synchronization.
|
|
215
|
+
|
|
216
|
+
Choose `AsyncEventBus` when the application already uses asyncio and handlers
|
|
217
|
+
are coroutines. Bus operations belong to the first running event loop that
|
|
218
|
+
uses the bus. Do not share an instance across different loops. To publish from
|
|
219
|
+
a worker thread, use `publish_threadsafe()`; see [Async applications](#async-applications).
|
|
220
|
+
|
|
221
|
+
## Core concepts
|
|
222
|
+
|
|
223
|
+
A bus stores active subscriptions and delivers each published event to the
|
|
224
|
+
subscriptions whose filters match it. Subscriptions are independent: a handler
|
|
225
|
+
can be cancelled without affecting other handlers or the bus.
|
|
226
|
+
|
|
227
|
+
<a id="subscriptions"></a>
|
|
228
|
+
### Events and subscriptions
|
|
229
|
+
|
|
230
|
+
Events do not inherit from a library base class. Define the values that suit
|
|
231
|
+
the application, then publish them with `publish(event)`. Keep event objects
|
|
232
|
+
small and give their fields names that describe the information a handler
|
|
233
|
+
needs.
|
|
234
|
+
|
|
235
|
+
`subscribe(callback)` returns a `Subscription` for `EventBus` and an
|
|
236
|
+
`AsyncSubscription` for `AsyncEventBus`. Keep the returned handle while the
|
|
237
|
+
handler is needed. Call `cancel()` when it is no longer needed, or manage the
|
|
238
|
+
handle with a context manager as in the first example. Cancellation is safe to
|
|
239
|
+
repeat. The handle exposes `id` and `active`.
|
|
240
|
+
|
|
241
|
+
<a id="filtering"></a>
|
|
242
|
+
### Filtering and delivery limits
|
|
243
|
+
|
|
244
|
+
A subscription can receive every event, events of one or more classes, or
|
|
245
|
+
events that pass a predicate. A type check and predicate can be combined:
|
|
246
|
+
|
|
247
|
+
~~~python
|
|
248
|
+
from dataclasses import dataclass
|
|
249
|
+
|
|
250
|
+
from eventflow import EventBus, EventFilter
|
|
251
|
+
|
|
252
|
+
|
|
253
|
+
@dataclass(frozen=True)
|
|
254
|
+
class PaymentReceived:
|
|
255
|
+
account_id: str
|
|
256
|
+
amount_cents: int
|
|
257
|
+
|
|
258
|
+
|
|
259
|
+
large_payment = EventFilter(
|
|
260
|
+
event_type=PaymentReceived,
|
|
261
|
+
predicate=lambda event: event.amount_cents >= 10_000,
|
|
262
|
+
)
|
|
263
|
+
|
|
264
|
+
|
|
265
|
+
def write_audit_record(event: PaymentReceived) -> None:
|
|
266
|
+
print(f"Audit {event.account_id}: {event.amount_cents}")
|
|
267
|
+
|
|
268
|
+
|
|
269
|
+
with EventBus[PaymentReceived]() as bus:
|
|
270
|
+
with bus.subscribe(write_audit_record, event_filter=large_payment):
|
|
271
|
+
bus.publish(PaymentReceived("acct-7", 2_500))
|
|
272
|
+
bus.publish(PaymentReceived("acct-7", 12_500))
|
|
273
|
+
~~~
|
|
274
|
+
|
|
275
|
+
Only the second payment reaches the handler. A type filter may also be a tuple
|
|
276
|
+
of classes. Predicates are synchronous functions that receive the event and
|
|
277
|
+
return a boolean, including when attached to an async bus. Async predicates
|
|
278
|
+
and predicates that return awaitables are rejected because filtering itself
|
|
279
|
+
is synchronous.
|
|
280
|
+
|
|
281
|
+
Use either an `EventFilter` or the direct `event_type`/`predicate` arguments
|
|
282
|
+
on a bus method; do not combine the two forms. For a handler that should stop
|
|
283
|
+
after its first match, use `once=True` or `once(callback, ...)`. For a fixed
|
|
284
|
+
number of matching deliveries, use `times=N`. These limits count matching
|
|
285
|
+
events, not every event published to the bus.
|
|
286
|
+
|
|
287
|
+
<a id="failures"></a>
|
|
288
|
+
### Failures and results
|
|
289
|
+
|
|
290
|
+
The default `ErrorPolicy.ISOLATE` records handler failures and continues with
|
|
291
|
+
the other selected handlers. Each call to `publish()` returns a `PublishResult`:
|
|
292
|
+
|
|
293
|
+
| Field | Meaning |
|
|
294
|
+
| --- | --- |
|
|
295
|
+
| `matched` | Subscriptions selected for immediate delivery by this publish call |
|
|
296
|
+
| `delivered` | Handlers that completed successfully |
|
|
297
|
+
| `failed` / `errors` | Collected delivery or action failures |
|
|
298
|
+
| `successful` | `True` when no failures were collected |
|
|
299
|
+
|
|
300
|
+
Events buffered for a subscription while replay is in progress are excluded
|
|
301
|
+
from `matched`, even when its filter accepts them.
|
|
302
|
+
|
|
303
|
+
A synchronous `on_error` hook can report a failure without coupling the handler
|
|
304
|
+
to a particular logging system:
|
|
305
|
+
|
|
306
|
+
~~~python
|
|
307
|
+
from eventflow import EventBus, EventErrorContext
|
|
308
|
+
|
|
309
|
+
|
|
310
|
+
def report_failure(context: EventErrorContext[str]) -> None:
|
|
311
|
+
print(f"{context.event}: {type(context.error).__name__}")
|
|
312
|
+
|
|
313
|
+
|
|
314
|
+
def persist_order(event: str) -> None:
|
|
315
|
+
raise RuntimeError("storage is unavailable")
|
|
316
|
+
|
|
317
|
+
|
|
318
|
+
with EventBus[str](on_error=report_failure) as bus:
|
|
319
|
+
bus.subscribe(persist_order)
|
|
320
|
+
result = bus.publish("order-42")
|
|
321
|
+
print(result.matched, result.delivered, result.failed)
|
|
322
|
+
~~~
|
|
323
|
+
|
|
324
|
+
Set `error_policy=ErrorPolicy.RAISE` when the caller should handle failures
|
|
325
|
+
directly. The bus attempts delivery to the selected handlers, then raises an
|
|
326
|
+
`ExceptionGroup` if failures were collected. That publish call does not return
|
|
327
|
+
a `PublishResult`. The `bus.stats` property provides cumulative
|
|
328
|
+
`published`, `delivered`, and `failed` counts.
|
|
329
|
+
|
|
330
|
+
These counters are not a per-publish invariant. `published` is incremented
|
|
331
|
+
before callbacks run; `delivered` and `failed` are finalized afterward. An
|
|
332
|
+
in-flight or cancelled async publish can leave them different. `failed` counts
|
|
333
|
+
collected errors from handlers, filters, and actions, not only failed handlers.
|
|
334
|
+
|
|
335
|
+
To catch grouped failures with the same import on Python 3.10 and newer, import
|
|
336
|
+
`ExceptionGroup` from `eventflow.compatibility`. On Python 3.11 and newer, that
|
|
337
|
+
name refers to the built-in exception group class.
|
|
338
|
+
|
|
339
|
+
## Working with events
|
|
340
|
+
|
|
341
|
+
<a id="history"></a>
|
|
342
|
+
### History and replay
|
|
343
|
+
|
|
344
|
+
History is disabled by default. Set `history` to a positive capacity to keep
|
|
345
|
+
recent events in a bounded, thread-safe `MemoryEventHistory`. When the buffer
|
|
346
|
+
is full, each new event replaces the oldest one. History lives in memory; it
|
|
347
|
+
is not a durable event log.
|
|
348
|
+
|
|
349
|
+
~~~python
|
|
350
|
+
from eventflow import EventBus
|
|
351
|
+
|
|
352
|
+
|
|
353
|
+
with EventBus[str](history=3) as bus:
|
|
354
|
+
for status in ("created", "paid", "packed", "shipped"):
|
|
355
|
+
bus.publish(status)
|
|
356
|
+
|
|
357
|
+
print(bus.query(limit=2))
|
|
358
|
+
print(bus.latest())
|
|
359
|
+
~~~
|
|
360
|
+
|
|
361
|
+
Output:
|
|
362
|
+
|
|
363
|
+
~~~text
|
|
364
|
+
('packed', 'shipped')
|
|
365
|
+
shipped
|
|
366
|
+
~~~
|
|
367
|
+
|
|
368
|
+
`query()` returns retained events in publication order. It accepts the same
|
|
369
|
+
filter options as a subscription and an optional positive `limit`. With a
|
|
370
|
+
limit, it returns the newest matching events in chronological order.
|
|
371
|
+
`latest()` returns the newest match or `None`. Without history,
|
|
372
|
+
`query()` returns an empty tuple and `latest()` returns `None`.
|
|
373
|
+
History order reflects when events were recorded, not when callbacks start or
|
|
374
|
+
finish. Async publish hooks can yield after recording, and separate publish
|
|
375
|
+
calls may overlap.
|
|
376
|
+
|
|
377
|
+
An unfiltered query with `limit` reads only the requested recent window. A
|
|
378
|
+
filtered query evaluates the predicate outside the history lock on a stable
|
|
379
|
+
snapshot. It searches newest to oldest when limited, but a rare match near the
|
|
380
|
+
oldest entry can still require scanning the full snapshot.
|
|
381
|
+
On `AsyncEventBus`, this filtered lookup runs synchronously on the owning event
|
|
382
|
+
loop. A rare match in a large history can therefore delay unrelated tasks;
|
|
383
|
+
bound the retained history and keep predicates inexpensive when replay latency
|
|
384
|
+
matters.
|
|
385
|
+
|
|
386
|
+
A new subscriber can receive recent events before it starts handling live
|
|
387
|
+
ones. Pass `replay=N` to deliver up to the N newest retained events that match
|
|
388
|
+
the subscription filter:
|
|
389
|
+
|
|
390
|
+
~~~python
|
|
391
|
+
with EventBus[str](history=10) as bus:
|
|
392
|
+
bus.publish("started")
|
|
393
|
+
bus.publish("ready")
|
|
394
|
+
|
|
395
|
+
with bus.subscribe(print, replay=2):
|
|
396
|
+
pass
|
|
397
|
+
~~~
|
|
398
|
+
|
|
399
|
+
The handler prints `started` and `ready` as the subscription is created. Live
|
|
400
|
+
events that arrive during replay are buffered per subscriber.
|
|
401
|
+
`replay_buffer_limit` (default 1000) bounds this temporary queue. If the queue
|
|
402
|
+
overflows, that subscription is cancelled and the overflow is reported using
|
|
403
|
+
the bus's error policy.
|
|
404
|
+
|
|
405
|
+
To provide a custom history store, implement `EventHistory`:
|
|
406
|
+
`append()`, `latest()`, `query()`, `clear()`, and `__len__()`. Implementations
|
|
407
|
+
must synchronize all five operations and return `latest()`/`query()` results
|
|
408
|
+
from a consistent snapshot. A history shared by multiple buses must synchronize
|
|
409
|
+
across those buses too. The bus publication lock serializes `append()` only for
|
|
410
|
+
that bus; history queries run outside it. Keep `append()` fast and non-blocking,
|
|
411
|
+
and do not publish recursively through the same bus from it.
|
|
412
|
+
|
|
413
|
+
Bus `clear()` removes active subscriptions and wakes pending waiters; it does
|
|
414
|
+
not erase history or stop periodic schedules. Call the returned schedule's
|
|
415
|
+
`cancel()` to stop it, or close the bus to stop all schedules. To delete retained
|
|
416
|
+
events, call `clear()` on the configured history object.
|
|
417
|
+
|
|
418
|
+
<a id="wait-for-event"></a>
|
|
419
|
+
### Waiting for an event
|
|
420
|
+
|
|
421
|
+
Use `wait_for()` when the caller needs one matching event as a value instead
|
|
422
|
+
of registering a long-lived handler. It accepts the same filter options as
|
|
423
|
+
`subscribe()` and an optional timeout. It returns the event, or `None` if the
|
|
424
|
+
wait times out or the bus closes before a match. A synchronous wait blocks the
|
|
425
|
+
calling thread. Timeouts must be finite, non-negative numbers; zero requests
|
|
426
|
+
an immediate wait.
|
|
427
|
+
|
|
428
|
+
This async example schedules a publisher and then waits for its event. The
|
|
429
|
+
wait registers its temporary subscription before the publisher task gets to
|
|
430
|
+
run:
|
|
431
|
+
|
|
432
|
+
~~~python
|
|
433
|
+
import asyncio
|
|
434
|
+
|
|
435
|
+
from eventflow import AsyncEventBus
|
|
436
|
+
|
|
437
|
+
|
|
438
|
+
async def main() -> None:
|
|
439
|
+
async with AsyncEventBus[str]() as bus:
|
|
440
|
+
publish_task = asyncio.create_task(bus.publish("ready"))
|
|
441
|
+
event = await bus.wait_for(event_type=str, timeout=1.0)
|
|
442
|
+
await publish_task
|
|
443
|
+
print(event)
|
|
444
|
+
|
|
445
|
+
|
|
446
|
+
asyncio.run(main())
|
|
447
|
+
~~~
|
|
448
|
+
|
|
449
|
+
The output is `ready`. In synchronous code, use
|
|
450
|
+
`bus.wait_for(event_type=..., timeout=...)`; omit `timeout` to wait without a
|
|
451
|
+
deadline. A configured `on_timeout` hook receives an `EventTimeoutContext`
|
|
452
|
+
with the filter and timeout that expired.
|
|
453
|
+
|
|
454
|
+
<a id="periodic"></a>
|
|
455
|
+
### Periodic publishing
|
|
456
|
+
|
|
457
|
+
`publish_every(event, interval, times=None, immediately=True)` publishes the
|
|
458
|
+
same event repeatedly and returns a cancellable subscription. `interval` must
|
|
459
|
+
be positive. The first publication happens immediately unless
|
|
460
|
+
`immediately=False`. Set `times` to a positive integer for a finite schedule,
|
|
461
|
+
or cancel the returned subscription to stop early. A bus allows at most
|
|
462
|
+
`max_schedules` active schedules (default 64).
|
|
463
|
+
|
|
464
|
+
~~~python
|
|
465
|
+
import time
|
|
466
|
+
|
|
467
|
+
from eventflow import EventBus
|
|
468
|
+
|
|
469
|
+
|
|
470
|
+
with EventBus[str]() as bus:
|
|
471
|
+
bus.subscribe(print)
|
|
472
|
+
schedule = bus.publish_every("heartbeat", interval=1.0, times=3)
|
|
473
|
+
|
|
474
|
+
while schedule.active:
|
|
475
|
+
time.sleep(0.05)
|
|
476
|
+
~~~
|
|
477
|
+
|
|
478
|
+
The handler receives three heartbeats. `AsyncEventBus` provides the awaited
|
|
479
|
+
version and runs it on the owning loop:
|
|
480
|
+
|
|
481
|
+
~~~python
|
|
482
|
+
import asyncio
|
|
483
|
+
|
|
484
|
+
from eventflow import AsyncEventBus
|
|
485
|
+
|
|
486
|
+
|
|
487
|
+
async def main() -> None:
|
|
488
|
+
async with AsyncEventBus[str]() as bus:
|
|
489
|
+
|
|
490
|
+
async def report(message: str) -> None:
|
|
491
|
+
print(message)
|
|
492
|
+
|
|
493
|
+
await bus.subscribe(report)
|
|
494
|
+
schedule = await bus.publish_every("heartbeat", 0.05, times=3)
|
|
495
|
+
while schedule.active:
|
|
496
|
+
await asyncio.sleep(0.01)
|
|
497
|
+
|
|
498
|
+
|
|
499
|
+
asyncio.run(main())
|
|
500
|
+
~~~
|
|
501
|
+
|
|
502
|
+
Closing a bus cancels its schedules. With `ErrorPolicy.RAISE`, an error during
|
|
503
|
+
a periodic publish stops that schedule. A schedule does not return each
|
|
504
|
+
`PublishResult`; configure the bus `on_error` hook to observe handler, filter,
|
|
505
|
+
or action failures. With `ErrorPolicy.ISOLATE`, the schedule continues and the
|
|
506
|
+
publish result is otherwise discarded.
|
|
507
|
+
|
|
508
|
+
## Async applications
|
|
509
|
+
|
|
510
|
+
`AsyncEventBus` accepts async handlers and async bus hooks. Its default
|
|
511
|
+
`DeliveryMode.SEQUENTIAL` awaits matching handlers in subscription order.
|
|
512
|
+
`DeliveryMode.CONCURRENT` awaits them together. Concurrent mode can be useful
|
|
513
|
+
when handlers are independent, but their completion order is not guaranteed.
|
|
514
|
+
Sequential ordering applies within one `publish()` call; separate publish calls
|
|
515
|
+
may overlap. In sequential mode, a handler-raised `asyncio.CancelledError`
|
|
516
|
+
propagates from `publish()` and stops delivery to later handlers. In concurrent
|
|
517
|
+
mode, an independently cancelled handler task is recorded as a failed delivery
|
|
518
|
+
in `PublishResult`. Cancelling the task that called `publish()` propagates in
|
|
519
|
+
either mode and cancels in-flight handler work. Cancellation cannot undo side
|
|
520
|
+
effects a handler produced before it was cancelled.
|
|
521
|
+
|
|
522
|
+
~~~python
|
|
523
|
+
import asyncio
|
|
524
|
+
|
|
525
|
+
from eventflow import AsyncEventBus, DeliveryMode
|
|
526
|
+
|
|
527
|
+
|
|
528
|
+
async def main() -> None:
|
|
529
|
+
async with AsyncEventBus[int](
|
|
530
|
+
delivery_mode=DeliveryMode.CONCURRENT,
|
|
531
|
+
max_concurrency=8,
|
|
532
|
+
) as bus:
|
|
533
|
+
|
|
534
|
+
async def save_metric(value: int) -> None:
|
|
535
|
+
print(f"Saved metric: {value}")
|
|
536
|
+
|
|
537
|
+
async def notify_dashboard(value: int) -> None:
|
|
538
|
+
print(f"Dashboard notified: {value}")
|
|
539
|
+
|
|
540
|
+
await bus.subscribe(save_metric)
|
|
541
|
+
await bus.subscribe(notify_dashboard)
|
|
542
|
+
result = await bus.publish(42)
|
|
543
|
+
print(f"delivered={result.delivered}")
|
|
544
|
+
|
|
545
|
+
|
|
546
|
+
asyncio.run(main())
|
|
547
|
+
~~~
|
|
548
|
+
|
|
549
|
+
Bus operations such as `subscribe()`, `publish()`, `wait_for()`,
|
|
550
|
+
`publish_every()`, `clear()`, and `close()` are awaited. The
|
|
551
|
+
`publish_threadsafe(event)` entry point is for a different thread: it returns
|
|
552
|
+
a `concurrent.futures.Future` and schedules delivery on the owning loop.
|
|
553
|
+
|
|
554
|
+
Concurrent delivery is unbounded by default to preserve the existing behavior.
|
|
555
|
+
Set `max_concurrency=N` with `DeliveryMode.CONCURRENT` to cap callback fan-out
|
|
556
|
+
for each publish. When a callback finishes, its slot is reused immediately for
|
|
557
|
+
the next subscriber instead of waiting for the rest of a fixed group. Overlapping
|
|
558
|
+
publish calls each have their own limit; this does not bound how many publish
|
|
559
|
+
calls the application creates. Choose a finite value when one event's fan-out
|
|
560
|
+
could otherwise create too many tasks or use too many external resources.
|
|
561
|
+
|
|
562
|
+
~~~python
|
|
563
|
+
import asyncio
|
|
564
|
+
|
|
565
|
+
from eventflow import AsyncEventBus
|
|
566
|
+
|
|
567
|
+
|
|
568
|
+
async def main() -> None:
|
|
569
|
+
async with AsyncEventBus[str]() as bus:
|
|
570
|
+
|
|
571
|
+
async def report(message: str) -> None:
|
|
572
|
+
print(message)
|
|
573
|
+
|
|
574
|
+
await bus.subscribe(report) # binds the bus to this loop
|
|
575
|
+
future = await asyncio.to_thread(
|
|
576
|
+
bus.publish_threadsafe,
|
|
577
|
+
"from a worker",
|
|
578
|
+
)
|
|
579
|
+
result = await asyncio.wrap_future(future)
|
|
580
|
+
print(result.delivered)
|
|
581
|
+
|
|
582
|
+
|
|
583
|
+
asyncio.run(main())
|
|
584
|
+
~~~
|
|
585
|
+
|
|
586
|
+
Do not call `future.result()` on the event-loop thread: waiting there would
|
|
587
|
+
block the loop needed to deliver the event.
|
|
588
|
+
|
|
589
|
+
<a id="channels"></a>
|
|
590
|
+
## Organizing channels
|
|
591
|
+
|
|
592
|
+
`EventEngine` creates synchronous buses lazily by channel name. Use it when
|
|
593
|
+
several parts of an application need separate event channels with shared
|
|
594
|
+
defaults:
|
|
595
|
+
|
|
596
|
+
~~~python
|
|
597
|
+
from eventflow import EventEngine
|
|
598
|
+
|
|
599
|
+
|
|
600
|
+
def update_order_view(event: str) -> None:
|
|
601
|
+
print(f"Order view updated: {event}")
|
|
602
|
+
|
|
603
|
+
|
|
604
|
+
with EventEngine(history=50) as engine:
|
|
605
|
+
engine.subscribe("orders", update_order_view)
|
|
606
|
+
result = engine.publish("orders", "order-42")
|
|
607
|
+
print(result.delivered)
|
|
608
|
+
~~~
|
|
609
|
+
|
|
610
|
+
Channel names are stripped and lowercased, so `" Orders "` and `"orders"`
|
|
611
|
+
refer to the same channel. `engine.channel(name)` returns a channel, creating
|
|
612
|
+
it when necessary. Use `engine.add(name, bus)` to register a bus with custom
|
|
613
|
+
settings. `remove(name)` closes one channel; `stop()` closes every channel.
|
|
614
|
+
An engine can own a bus instance under only one channel name at a time. A
|
|
615
|
+
successful `add()` claims that bus for the engine; another engine cannot claim
|
|
616
|
+
the same instance and receives `ValueError`. Removing or stopping the owner
|
|
617
|
+
closes the bus.
|
|
618
|
+
The engine exposes `running` and a sorted `channel_names` tuple.
|
|
619
|
+
During `stop()`, both engines attempt every channel. If some closes fail, the
|
|
620
|
+
failures are raised together and the engine retains those channels so a later
|
|
621
|
+
`stop()` can retry cleanup.
|
|
622
|
+
|
|
623
|
+
`AsyncEventEngine` offers the same channel model for async buses. Its
|
|
624
|
+
`channel()` and `add()` methods return directly. Publish, subscribe, once,
|
|
625
|
+
wait, remove, start, and stop operations are awaited:
|
|
626
|
+
|
|
627
|
+
~~~python
|
|
628
|
+
import asyncio
|
|
629
|
+
|
|
630
|
+
from eventflow import AsyncEventEngine
|
|
631
|
+
|
|
632
|
+
|
|
633
|
+
async def main() -> None:
|
|
634
|
+
async with AsyncEventEngine(history=50) as engine:
|
|
635
|
+
|
|
636
|
+
async def update_order_view(event: str) -> None:
|
|
637
|
+
print(f"Order view updated: {event}")
|
|
638
|
+
|
|
639
|
+
await engine.subscribe("orders", update_order_view)
|
|
640
|
+
result = await engine.publish("orders", "order-42")
|
|
641
|
+
print(result.delivered)
|
|
642
|
+
|
|
643
|
+
|
|
644
|
+
asyncio.run(main())
|
|
645
|
+
~~~
|
|
646
|
+
|
|
647
|
+
Engine defaults are passed to the buses the engine creates. `EventEngine`
|
|
648
|
+
carries history, error policy, executor, and hooks; its optional executor must
|
|
649
|
+
be a `ThreadPoolExecutor`. `AsyncEventEngine` carries history, error policy,
|
|
650
|
+
delivery mode, `max_concurrency`, and async hooks.
|
|
651
|
+
|
|
652
|
+
## Callback policies
|
|
653
|
+
|
|
654
|
+
`CallbackSubscription` is a lower-level utility for applying limits and hooks
|
|
655
|
+
to one callback when it is invoked directly. It is independent of a bus and is
|
|
656
|
+
not created by `EventBus.subscribe()`. A caller supplies its identifier, a
|
|
657
|
+
cancellation function, and the callback:
|
|
658
|
+
|
|
659
|
+
~~~python
|
|
660
|
+
from eventflow import CallbackSubscription
|
|
661
|
+
|
|
662
|
+
|
|
663
|
+
def handle_message(message: str) -> str:
|
|
664
|
+
return message.upper()
|
|
665
|
+
|
|
666
|
+
|
|
667
|
+
policy = CallbackSubscription(
|
|
668
|
+
identifier=1,
|
|
669
|
+
cancel=lambda: None,
|
|
670
|
+
callback=handle_message,
|
|
671
|
+
max_calls=3,
|
|
672
|
+
frequency_hz=2,
|
|
673
|
+
on_success=lambda context: print(context.result),
|
|
674
|
+
)
|
|
675
|
+
|
|
676
|
+
print(policy.invoke("ready"))
|
|
677
|
+
~~~
|
|
678
|
+
|
|
679
|
+
`invoke()` applies the policy to a synchronous callback. For an asynchronous
|
|
680
|
+
callback, set `asynchronous=True` and await `invoke_async()`:
|
|
681
|
+
|
|
682
|
+
~~~python
|
|
683
|
+
import asyncio
|
|
684
|
+
|
|
685
|
+
from eventflow import CallbackSubscription
|
|
686
|
+
|
|
687
|
+
|
|
688
|
+
async def handle_message(message: str) -> str:
|
|
689
|
+
return message.upper()
|
|
690
|
+
|
|
691
|
+
|
|
692
|
+
async def main() -> None:
|
|
693
|
+
policy = CallbackSubscription(
|
|
694
|
+
identifier=2,
|
|
695
|
+
cancel=lambda: None,
|
|
696
|
+
callback=handle_message,
|
|
697
|
+
asynchronous=True,
|
|
698
|
+
timeout=1.0,
|
|
699
|
+
)
|
|
700
|
+
print(await policy.invoke_async("ready"))
|
|
701
|
+
|
|
702
|
+
|
|
703
|
+
asyncio.run(main())
|
|
704
|
+
~~~
|
|
705
|
+
|
|
706
|
+
The policy can limit accepted calls with `once` or `max_calls`, cap the rate
|
|
707
|
+
with `frequency_hz`, apply a timeout, filter with a synchronous predicate,
|
|
708
|
+
and be toggled with `enable()`/`disable()`. It supports before, success,
|
|
709
|
+
error, timeout, and after hooks; each hook receives a `CallbackContext` with
|
|
710
|
+
the event, result, error, and elapsed time. Disabled, filtered, and
|
|
711
|
+
rate-limited events do not consume the call limit. An invocation that passes
|
|
712
|
+
those checks consumes a call even if the callback later raises.
|
|
713
|
+
|
|
714
|
+
Use `on_before()`, `on_success()`, `on_error()`, `on_timeout()`, or
|
|
715
|
+
`on_after()` to register hooks after construction. Calling the wrapper itself
|
|
716
|
+
invokes the underlying callback directly and bypasses its policies; use
|
|
717
|
+
`invoke()` or `invoke_async()` when those policies should apply.
|
|
718
|
+
|
|
719
|
+
A synchronous timeout is observational: Python cannot safely terminate a
|
|
720
|
+
running thread, so the callback completes before an overrun is reported.
|
|
721
|
+
An asynchronous timeout is cooperative: it requests cancellation and waits for
|
|
722
|
+
the callback task to finish before raising `CallbackTimeoutError`. A callback
|
|
723
|
+
that suppresses cancellation, or a hook that does not return, can extend the
|
|
724
|
+
invocation beyond its configured budget. Python cannot force-stop a coroutine.
|
|
725
|
+
A callback's own `TimeoutError` is distinct from `CallbackTimeoutError`.
|
|
726
|
+
|
|
727
|
+
<!-- integrated execution documentation -->
|
|
728
|
+
<a id="pipelines"></a>
|
|
729
|
+
## Pipelines and middleware
|
|
730
|
+
|
|
731
|
+
Use `Pipeline[T]` for a sequence of transformations from `T` to `T`. Every stage
|
|
732
|
+
receives the previous stage's result. The pipeline returns the final value and
|
|
733
|
+
does not publish it automatically. An empty pipeline returns its input.
|
|
734
|
+
|
|
735
|
+
~~~python
|
|
736
|
+
from dataclasses import dataclass, replace
|
|
737
|
+
from eventflow import Pipeline
|
|
738
|
+
|
|
739
|
+
|
|
740
|
+
@dataclass(frozen=True)
|
|
741
|
+
class Request:
|
|
742
|
+
amount: int
|
|
743
|
+
|
|
744
|
+
|
|
745
|
+
pipeline = Pipeline[Request]()
|
|
746
|
+
|
|
747
|
+
|
|
748
|
+
@pipeline.stage("validate")
|
|
749
|
+
def validate(request: Request) -> Request:
|
|
750
|
+
if request.amount < 0:
|
|
751
|
+
raise ValueError("amount must be non-negative")
|
|
752
|
+
return request
|
|
753
|
+
|
|
754
|
+
|
|
755
|
+
def double(request: Request) -> Request:
|
|
756
|
+
return replace(request, amount=request.amount * 2)
|
|
757
|
+
|
|
758
|
+
|
|
759
|
+
pipeline.add("double", double, after="validate")
|
|
760
|
+
|
|
761
|
+
|
|
762
|
+
@pipeline.middleware("audit")
|
|
763
|
+
def audit(request, next_):
|
|
764
|
+
print("before", request.amount)
|
|
765
|
+
result = next_(request)
|
|
766
|
+
print("after", result.amount)
|
|
767
|
+
return result
|
|
768
|
+
|
|
769
|
+
|
|
770
|
+
print(pipeline.run(Request(4))) # Request(amount=8)
|
|
771
|
+
~~~
|
|
772
|
+
|
|
773
|
+
`stage()` and `middleware()` register and return the original function. Its
|
|
774
|
+
identity, name and direct call behavior remain intact. Decorator registration
|
|
775
|
+
and `add()` use the same registry; neither wraps the registered function.
|
|
776
|
+
|
|
777
|
+
| Operation | Behavior |
|
|
778
|
+
| --- | --- |
|
|
779
|
+
| `add(name, callback, *, before=None, after=None)` | Add a stage with optional ordering constraints. |
|
|
780
|
+
| `stage(name, *, before=None, after=None)` | Decorator equivalent of `add`. |
|
|
781
|
+
| `remove(name)` | Remove the stage and references to it; return its callback. |
|
|
782
|
+
| `replace(name, callback)` | Keep the name, ordering constraints and enabled state. |
|
|
783
|
+
| `enable(name)` / `disable(name)` | Change inclusion in future runs. |
|
|
784
|
+
| `stage_names` | Ordered names, including disabled stages. |
|
|
785
|
+
| `add_middleware(...)` / `middleware(...)` | Register around middleware in its own namespace. |
|
|
786
|
+
| `remove_middleware`, `replace_middleware`, `enable_middleware`, `disable_middleware` | Apply the corresponding registry operation to middleware. |
|
|
787
|
+
| `middleware_names` | Names from outermost to innermost, including disabled middleware. |
|
|
788
|
+
|
|
789
|
+
Names are stripped, case-sensitive, non-empty and unique within each registry.
|
|
790
|
+
`before` and `after` must reference existing names. Unknown references raise
|
|
791
|
+
`KeyError`; duplicate names, empty names and ordering cycles raise `ValueError`.
|
|
792
|
+
Invalid mutations leave the existing plan intact. Registration order resolves
|
|
793
|
+
ties. Disabled stages retain their ordering constraints. Removing a stage
|
|
794
|
+
removes references to it from the remaining stages.
|
|
795
|
+
|
|
796
|
+
Ordering is prepared when configuration changes. Each run takes one immutable
|
|
797
|
+
execution snapshot; replacement or disabling during a run affects subsequent
|
|
798
|
+
runs. Configuration is thread-safe and callbacks execute outside its lock.
|
|
799
|
+
The snapshot does not copy payloads or make a callback's own mutable state safe
|
|
800
|
+
for concurrent use.
|
|
801
|
+
|
|
802
|
+
Middleware wraps the entire pipeline: the first registered middleware is the
|
|
803
|
+
outermost. It may call `next_(value)` once, transform its return value, catch a
|
|
804
|
+
`PipelineExecutionError` to recover, or return directly to short-circuit. A
|
|
805
|
+
continuation is valid only inside its middleware call and on the same thread
|
|
806
|
+
or async task. A second call, deferred call or detached-task call raises
|
|
807
|
+
`RuntimeError`. Existing bus actions and callback hooks retain their own
|
|
808
|
+
semantics; they are not executed as pipeline middleware.
|
|
809
|
+
Because middleware uses nested continuations, each pipeline accepts at most 64
|
|
810
|
+
registered middleware entries, including disabled entries. The limit is checked
|
|
811
|
+
when adding middleware and avoids reaching Python's recursion limit during a run.
|
|
812
|
+
|
|
813
|
+
Stage failures stop later stages and raise `PipelineExecutionError`. Inspect
|
|
814
|
+
`name`, `phase`, `context` and `__cause__` to locate the original failure.
|
|
815
|
+
The context is the value passed into the failing component. A synchronous
|
|
816
|
+
stage or middleware returning an awaitable is rejected, and returned coroutine
|
|
817
|
+
objects are closed. Python cannot forcibly stop a synchronous callback.
|
|
818
|
+
|
|
819
|
+
<a id="streams"></a>
|
|
820
|
+
## Async pipelines and streams
|
|
821
|
+
|
|
822
|
+
`AsyncPipeline[T]` has the same registry and decorators. Its callbacks return
|
|
823
|
+
awaitables; `await pipeline.run(value)` executes on the first owning loop.
|
|
824
|
+
Synchronous callbacks are never moved to threads. Concurrent runs use
|
|
825
|
+
independent snapshots and continuations. `CancelledError` propagates into the
|
|
826
|
+
active stage and is not converted into a delivery failure or retried.
|
|
827
|
+
Stage and middleware results must be resolved values: returning another
|
|
828
|
+
awaitable after the outer await (for example, forgetting `await next_`) is
|
|
829
|
+
rejected and any returned coroutine is closed.
|
|
830
|
+
|
|
831
|
+
`AsyncStream[T]` owns an `AsyncIterable[T]`. Its transformations accept ordinary
|
|
832
|
+
or awaitable callbacks, making both pipeline `run` methods directly usable as
|
|
833
|
+
map functions. Mapping, filtering and batching are lazy and start no workers.
|
|
834
|
+
|
|
835
|
+
~~~python
|
|
836
|
+
import asyncio
|
|
837
|
+
from eventflow import AsyncPipeline, AsyncStream
|
|
838
|
+
|
|
839
|
+
|
|
840
|
+
async def main() -> None:
|
|
841
|
+
pipeline = AsyncPipeline[int]()
|
|
842
|
+
|
|
843
|
+
@pipeline.stage("double")
|
|
844
|
+
async def double(value: int) -> int:
|
|
845
|
+
return value * 2
|
|
846
|
+
|
|
847
|
+
async def source():
|
|
848
|
+
for value in range(5):
|
|
849
|
+
yield value
|
|
850
|
+
|
|
851
|
+
stream = (
|
|
852
|
+
AsyncStream(source())
|
|
853
|
+
.map(pipeline.run)
|
|
854
|
+
.filter(lambda value: value % 4 == 0)
|
|
855
|
+
.buffer(2)
|
|
856
|
+
.batch(2)
|
|
857
|
+
)
|
|
858
|
+
async with stream:
|
|
859
|
+
async for batch in stream:
|
|
860
|
+
print(batch) # (0, 4), then (8,)
|
|
861
|
+
|
|
862
|
+
|
|
863
|
+
asyncio.run(main())
|
|
864
|
+
~~~
|
|
865
|
+
|
|
866
|
+
| Operation | Result and ownership |
|
|
867
|
+
| --- | --- |
|
|
868
|
+
| `AsyncStream(source)` | A single-use stream on one owning loop. |
|
|
869
|
+
| `map(callback)` | Transform values sequentially; await awaitable results. |
|
|
870
|
+
| `filter(predicate)` | Keep accepted values; synchronous or awaitable predicate. |
|
|
871
|
+
| `batch(size, *, partial=True)` | Yield tuples; optionally include the final partial tuple. |
|
|
872
|
+
| `buffer(capacity)` | Add one producer and a bounded queue with backpressure. |
|
|
873
|
+
| `aclose()` / `async with stream` | Close the entire upstream chain and await owned cleanup. |
|
|
874
|
+
| `closed` | Whether new reads have stopped. |
|
|
875
|
+
|
|
876
|
+
Batch sizes and buffer capacities must be positive integers; booleans, zero,
|
|
877
|
+
fractions and infinity are rejected. A full buffer blocks its producer. There
|
|
878
|
+
is room for `capacity` queued values plus one source value currently waiting
|
|
879
|
+
to enter the queue. Values are never dropped. Source failures reach the
|
|
880
|
+
consumer after earlier queued values; normal completion remains distinct from
|
|
881
|
+
failure and cancellation. Capacity counts objects, not bytes; a small number of
|
|
882
|
+
large payloads can still use substantial memory. Apply payload-size limits or
|
|
883
|
+
an application-owned byte budget when events have highly variable sizes.
|
|
884
|
+
|
|
885
|
+
A stream is consumed once. A second iterator acquisition and concurrent reads
|
|
886
|
+
are rejected. Use `async with` on the **terminal stream** when a consumer may
|
|
887
|
+
break early or raise. Python's `async for` protocol does not notify a source
|
|
888
|
+
when arbitrary consumer code uses `break`; `aclose()` supplies that explicit
|
|
889
|
+
ownership boundary. It also closes an unstarted upstream chain. Closing from
|
|
890
|
+
another task cancels an outstanding reader, then waits for source cleanup.
|
|
891
|
+
Cancellation of the caller is propagated after owned cleanup finishes, even
|
|
892
|
+
if cancellation is repeated. Sources and user cleanup still need to cooperate
|
|
893
|
+
with cancellation; a non-returning source can delay closure.
|
|
894
|
+
A callback running in the active read task cannot await that same terminal
|
|
895
|
+
stream's `aclose()`; it raises `RuntimeError` to prevent a cleanup deadlock.
|
|
896
|
+
The read's normal error path then closes the stream and its upstream chain.
|
|
897
|
+
|
|
898
|
+
<a id="scopes"></a>
|
|
899
|
+
## Subscription scopes
|
|
900
|
+
|
|
901
|
+
Scopes group existing cancellation handles without changing what `subscribe`
|
|
902
|
+
returns. `add(handle)` returns the same handle and ignores repeated registration
|
|
903
|
+
of the same object. Separate scopes may own unrelated buses or callback policies.
|
|
904
|
+
|
|
905
|
+
~~~python
|
|
906
|
+
from eventflow import EventBus, SubscriptionScope
|
|
907
|
+
|
|
908
|
+
with EventBus[int]() as bus, SubscriptionScope() as scope:
|
|
909
|
+
handle = scope.add(bus.subscribe(print))
|
|
910
|
+
bus.publish(42)
|
|
911
|
+
assert not handle.active
|
|
912
|
+
~~~
|
|
913
|
+
|
|
914
|
+
`SubscriptionScope.close()` cancels synchronous handles in reverse registration
|
|
915
|
+
order. `AsyncSubscriptionScope.aclose()` awaits async handles in the same order
|
|
916
|
+
and supports `async with`. Both accept duck-typed handles with the corresponding
|
|
917
|
+
`cancel()` method. They attempt every handle, collect failures in the existing
|
|
918
|
+
Python 3.10-compatible `ExceptionGroup`, and retain failed handles for retry.
|
|
919
|
+
An async handle's `cancel()` must return an awaitable. The scope checks this
|
|
920
|
+
when closing because `add()` cannot validate the result without cancelling the
|
|
921
|
+
handle.
|
|
922
|
+
Handle implementations should make repeated cancellation safe. A handle's own
|
|
923
|
+
async cancellation is propagated after the other handles have been attempted.
|
|
924
|
+
Other handle-owned `BaseException` interrupts also retain their type after all
|
|
925
|
+
handles are attempted; additional failures are chained as an exception group.
|
|
926
|
+
Closed scopes reject additions. Async cleanup is shared by concurrent close
|
|
927
|
+
callers and finishes before caller cancellation is re-raised.
|
|
928
|
+
|
|
929
|
+
Scopes cancel registrations; they do not acquire ownership of the underlying
|
|
930
|
+
bus or forcibly stop callbacks already running. Keep bus/engine context managers
|
|
931
|
+
when those resources also need to close. The async scope itself belongs to one
|
|
932
|
+
loop and rejects reentrant cleanup that would await itself.
|
|
933
|
+
|
|
934
|
+
<a id="event-context"></a>
|
|
935
|
+
## Event context and measurements
|
|
936
|
+
|
|
937
|
+
`publish(event)` continues to accept raw objects. `EventEnvelope[T]` is an
|
|
938
|
+
optional frozen container with `event`, generated `id`, `correlation_id`,
|
|
939
|
+
`source`, UTC `timestamp` and `metadata`. A timezone-aware timestamp supplied
|
|
940
|
+
in another zone is normalized to UTC. Metadata is copied into a read-only
|
|
941
|
+
mapping, including after `copy.deepcopy()`; payloads and nested metadata values
|
|
942
|
+
retain their own mutability. `dataclasses.asdict()` and pickle work when the
|
|
943
|
+
event and metadata values themselves support those operations. Only unpickle
|
|
944
|
+
data from trusted sources: Python pickle can execute code during loading. Use
|
|
945
|
+
`dataclasses.replace` to transform an envelope's payload.
|
|
946
|
+
|
|
947
|
+
~~~python
|
|
948
|
+
from dataclasses import replace
|
|
949
|
+
from eventflow import EventEnvelope, Pipeline, PipelineStats, correlation_context
|
|
950
|
+
|
|
951
|
+
stats = PipelineStats()
|
|
952
|
+
pipeline = Pipeline[EventEnvelope[int]](observer=stats)
|
|
953
|
+
pipeline.add("increment", lambda envelope: replace(envelope, event=envelope.event + 1))
|
|
954
|
+
with correlation_context("request-42"):
|
|
955
|
+
event = EventEnvelope(1, source="example", metadata={"attempt": 1})
|
|
956
|
+
result = pipeline.run(event)
|
|
957
|
+
print(result.event, result.correlation_id) # 2 request-42
|
|
958
|
+
print(stats.snapshot.runs.successful) # 1
|
|
959
|
+
~~~
|
|
960
|
+
|
|
961
|
+
`correlation_context(id)` is a normal context manager, usable in async functions
|
|
962
|
+
too. `current_correlation_id()` reads the current identifier. An envelope uses
|
|
963
|
+
an explicit correlation ID, then the bound context, then its own ID. Context
|
|
964
|
+
is isolated across asyncio tasks and reset on exit. New threads and the bus
|
|
965
|
+
executor do not automatically inherit it; use `contextvars.copy_context` in
|
|
966
|
+
application code when propagation is required.
|
|
967
|
+
|
|
968
|
+
Pass `observer=PipelineStats()` or a synchronous `PipelineObserver` callable to
|
|
969
|
+
either pipeline. Each completed run, stage and middleware produces a
|
|
970
|
+
`PipelineObservation(kind, name, elapsed, status, correlation_id, error)`.
|
|
971
|
+
Statuses are `success`, `error` and `cancelled`; elapsed times are seconds.
|
|
972
|
+
Bind correlation around the caller's entire run for consistent run and stage
|
|
973
|
+
notifications. Middleware-local context applies only while that middleware is
|
|
974
|
+
active. With no observer, clocks and observation objects are omitted.
|
|
975
|
+
|
|
976
|
+
`stats.snapshot` returns immutable `runs`, `stages` and `middleware` aggregates.
|
|
977
|
+
Each `ExecutionMetrics` includes `calls`, `successful`, `failed`, `cancelled`,
|
|
978
|
+
`total_seconds` and `max_seconds`. Counts describe actual execution, so a
|
|
979
|
+
short-circuit does not count skipped stages. Run and middleware durations
|
|
980
|
+
include nested stages; do not add them together. `reset()` clears aggregates.
|
|
981
|
+
No payloads or traceback objects are retained by the collector. Metric keys are
|
|
982
|
+
stage and middleware names; stable names keep the collector small. Dynamically
|
|
983
|
+
generated names remain in the snapshot until `reset()` is called, so avoid
|
|
984
|
+
unbounded user or request values as component names.
|
|
985
|
+
|
|
986
|
+
Observers are synchronous and execute inline, including in `AsyncPipeline`;
|
|
987
|
+
they must be fast and non-blocking to avoid delaying the event loop. For logging
|
|
988
|
+
or telemetry I/O, enqueue a compact observation into an application-owned
|
|
989
|
+
bounded queue. An observer failure aborts an otherwise successful run with
|
|
990
|
+
`phase="observer"`. During an existing pipeline failure, observer errors are
|
|
991
|
+
available as `observer_errors` on the original `PipelineExecutionError`,
|
|
992
|
+
preserving its cause. Existing `EventBusStats` is unchanged.
|
|
993
|
+
|
|
994
|
+
### Security and operational boundaries
|
|
995
|
+
|
|
996
|
+
EventFlow invokes callbacks, predicates, middleware and stream sources in the
|
|
997
|
+
application process with that process's permissions. It does not sandbox
|
|
998
|
+
untrusted plugins; isolate such code with an operating-system process or
|
|
999
|
+
container boundary and explicit permissions and resource limits. Async
|
|
1000
|
+
cancellation and callback timeouts are cooperative and do not provide a hard
|
|
1001
|
+
deadline for code that suppresses cancellation or blocks the event loop.
|
|
1002
|
+
|
|
1003
|
+
`PipelineExecutionError.context` and observer errors may reference application
|
|
1004
|
+
objects, including sensitive data. Avoid logging these objects wholesale;
|
|
1005
|
+
redact fields in the application's structured logging layer.
|
|
1006
|
+
|
|
1007
|
+
<a id="composition"></a>
|
|
1008
|
+
## Composing the components
|
|
1009
|
+
|
|
1010
|
+
Use the components independently or connect their normal callable/iterator
|
|
1011
|
+
interfaces. A stage can be `CallbackSubscription.invoke`, a stream map can be
|
|
1012
|
+
`Pipeline.run` or `AsyncPipeline.run`, and a scope can own the handle returned
|
|
1013
|
+
by either bus. Bus publication does not implicitly invoke a pipeline or add
|
|
1014
|
+
an envelope.
|
|
1015
|
+
|
|
1016
|
+
Two complete, network-free examples are executable from the repository root:
|
|
1017
|
+
|
|
1018
|
+
~~~bash
|
|
1019
|
+
python -m examples.integrated
|
|
1020
|
+
python -m examples.bus_stream
|
|
1021
|
+
~~~
|
|
1022
|
+
|
|
1023
|
+
[integrated.py](https://github.com/TahsinCr/python-eventflow/blob/main/examples/integrated.py)
|
|
1024
|
+
combines sync and async pipeline stages, decorators, callback policies,
|
|
1025
|
+
correlation, metrics, a buffered stream and an async bus.
|
|
1026
|
+
[bus_stream.py](https://github.com/TahsinCr/python-eventflow/blob/main/examples/bus_stream.py)
|
|
1027
|
+
uses a bounded application queue as a bus subscription and stream source.
|
|
1028
|
+
It owns one publisher task and joins it on exit; no task is created per event.
|
|
1029
|
+
|
|
1030
|
+
<a id="reference"></a>
|
|
1031
|
+
## Lifecycle and reference
|
|
1032
|
+
|
|
1033
|
+
### Defaults
|
|
1034
|
+
|
|
1035
|
+
| Option | `EventBus` | `AsyncEventBus` |
|
|
1036
|
+
| --- | --- | --- |
|
|
1037
|
+
| `history` | `None` | `None` |
|
|
1038
|
+
| `error_policy` | `ErrorPolicy.ISOLATE` | `ErrorPolicy.ISOLATE` |
|
|
1039
|
+
| `executor` | `None` | Not used |
|
|
1040
|
+
| `delivery_mode` | Not used | `DeliveryMode.SEQUENTIAL` |
|
|
1041
|
+
| `max_concurrency` | Not used | `None` (unbounded) |
|
|
1042
|
+
| `replay_buffer_limit` | 1000 | 1000 |
|
|
1043
|
+
| `max_schedules` | 64 | 64 |
|
|
1044
|
+
| `shutdown_timeout` | 5 seconds | Not used |
|
|
1045
|
+
| `actions` or direct `on_*` hooks | None | None |
|
|
1046
|
+
|
|
1047
|
+
`history` accepts a positive capacity or an `EventHistory` implementation.
|
|
1048
|
+
Pass either an `EventBusActions`/`AsyncEventBusActions` object or individual
|
|
1049
|
+
`on_before`, `on_after`, `on_error`, and `on_timeout` callbacks, not both.
|
|
1050
|
+
Hooks on `EventBus` are synchronous; hooks on `AsyncEventBus` are async.
|
|
1051
|
+
|
|
1052
|
+
Use `close()` when a bus is no longer needed; context managers do this
|
|
1053
|
+
automatically. A closed bus rejects new subscriptions and publications with
|
|
1054
|
+
`EventBusClosedError`. `clear()` removes subscriptions, leaves history intact,
|
|
1055
|
+
does not stop periodic schedules, and completes pending `wait_for()` calls with
|
|
1056
|
+
`None` on both bus types. A synchronous wait also returns `None` when the bus
|
|
1057
|
+
closes.
|
|
1058
|
+
|
|
1059
|
+
The synchronous bus waits up to `shutdown_timeout` for periodic workers to
|
|
1060
|
+
stop. If a worker does not stop in time, `EventShutdownTimeoutError` is
|
|
1061
|
+
raised. The worker is not forcibly terminated, so `close()` can be retried
|
|
1062
|
+
after the blocking handler finishes. Async close cancels and awaits periodic
|
|
1063
|
+
tasks. Closing a bus does not wait for active `publish()` calls: a callback
|
|
1064
|
+
that has already started may finish after `close()` returns.
|
|
1065
|
+
|
|
1066
|
+
<a id="bus-api"></a>
|
|
1067
|
+
### Bus methods
|
|
1068
|
+
|
|
1069
|
+
| Method | Behavior |
|
|
1070
|
+
| --- | --- |
|
|
1071
|
+
| `subscribe(callback, *, event_filter=None, event_type=None, predicate=None, once=False, times=None, replay=0)` | Register a handler. Supply either `event_filter` or direct filter arguments. |
|
|
1072
|
+
| `once(callback, *, event_filter=None, event_type=None, predicate=None, replay=0)` | Register for the first matching event. |
|
|
1073
|
+
| `publish(event)` | Deliver an event and return a `PublishResult`, unless the error policy raises. |
|
|
1074
|
+
| `wait_for(*, event_filter=None, event_type=None, predicate=None, timeout=None)` | Return one matching event or `None`. |
|
|
1075
|
+
| `publish_every(event, interval, *, times=None, immediately=True)` | Start a cancellable periodic schedule. |
|
|
1076
|
+
| `latest(...)` / `query(..., limit=None)` | Read retained history. These queries are synchronous on both bus types. |
|
|
1077
|
+
| `clear()` / `close()` | Clear subscriptions (schedules continue) or close the bus (schedules stop). Both are awaited on `AsyncEventBus`. |
|
|
1078
|
+
|
|
1079
|
+
On `AsyncEventBus`, await `subscribe()`, `once()`, `publish()`,
|
|
1080
|
+
`wait_for()`, `publish_every()`, `clear()`, `close()`, and subscription
|
|
1081
|
+
cancellation. `publish_threadsafe(event)` is called from another thread
|
|
1082
|
+
without `await` and returns a `concurrent.futures.Future`.
|
|
1083
|
+
|
|
1084
|
+
<a id="engine-api"></a>
|
|
1085
|
+
### Engine methods
|
|
1086
|
+
|
|
1087
|
+
`EventEngine` provides `channel()`, `add()`, `remove()`, `publish()`,
|
|
1088
|
+
`subscribe()`, `once()`, `wait_for()`, `publish_every()`, `start()`, and
|
|
1089
|
+
`stop()`. `channel()` and `add()` return directly.
|
|
1090
|
+
|
|
1091
|
+
Engine `subscribe()` accepts `event_filter`, `event_type`, `predicate`, `once`,
|
|
1092
|
+
`times`, and `replay`. Engine `once()` accepts the filter options and `replay`.
|
|
1093
|
+
The async engine exposes the same typed options for async handlers.
|
|
1094
|
+
|
|
1095
|
+
`AsyncEventEngine` provides the same operations for async buses. `channel()`
|
|
1096
|
+
and `add()` return directly; publishing, subscription, waiting, removal, start,
|
|
1097
|
+
and stop are awaited. Both engines support context managers; the async engine
|
|
1098
|
+
uses `async with`.
|
|
1099
|
+
|
|
1100
|
+
After a successful `stop()`, an engine can be reused: a later `channel()` or
|
|
1101
|
+
`add()` call creates or registers new open channels and marks it running again.
|
|
1102
|
+
|
|
1103
|
+
<a id="public-types"></a>
|
|
1104
|
+
### Public types and errors
|
|
1105
|
+
|
|
1106
|
+
The package exports the following public names:
|
|
1107
|
+
|
|
1108
|
+
| Area | Public names |
|
|
1109
|
+
| --- | --- |
|
|
1110
|
+
| Buses | `EventBus`, `AsyncEventBus`, `BaseEventBus` |
|
|
1111
|
+
| Engines | `EventEngine`, `AsyncEventEngine` |
|
|
1112
|
+
| Subscriptions | `Subscription`, `AsyncSubscription`, `CallbackSubscription` |
|
|
1113
|
+
| Filters and history | `EventFilter`, `EventHistory`, `MemoryEventHistory` |
|
|
1114
|
+
| Results and modes | `PublishResult`, `EventBusStats`, `ErrorPolicy`, `DeliveryMode` |
|
|
1115
|
+
| Actions and contexts | `EventBusActions`, `AsyncEventBusActions`, `EventErrorContext`, `EventTimeoutContext`, `CallbackContext` |
|
|
1116
|
+
| Errors | `EventBusError`, `EventBusClosedError`, `InvalidEventHandlerError`, `CallbackTimeoutError`, `EventShutdownTimeoutError` |
|
|
1117
|
+
|
|
1118
|
+
`EventFilter` accepts an event class or tuple of classes and an optional
|
|
1119
|
+
predicate. `MemoryEventHistory(capacity=1000)` is the built-in bounded,
|
|
1120
|
+
thread-safe history; `capacity` reports its configured size. Buses expose
|
|
1121
|
+
`closed`, `subscriber_count`, `stats`, `history`, and `error_policy`.
|
|
1122
|
+
Engines expose `running` and `channel_names`. Subscription handles expose
|
|
1123
|
+
`id` and `active`; `CallbackSubscription` also exposes `enabled`.
|
|
1124
|
+
|
|
1125
|
+
| Error | Raised when |
|
|
1126
|
+
| --- | --- |
|
|
1127
|
+
| `EventBusError` | A bus operation or schedule capacity cannot be completed. |
|
|
1128
|
+
| `EventBusClosedError` | A publish or registration is attempted after close. |
|
|
1129
|
+
| `InvalidEventHandlerError` | A handler or hook does not match the bus's sync/async mode. |
|
|
1130
|
+
| `CallbackTimeoutError` | A callback exceeds its configured time budget. |
|
|
1131
|
+
| `EventShutdownTimeoutError` | A synchronous periodic worker outlives `shutdown_timeout`. |
|
|
1132
|
+
|
|
1133
|
+
`BaseEventBus` is the abstract base for custom bus implementations.
|
|
1134
|
+
`EventHistory` is the interface for custom history stores. Most applications
|
|
1135
|
+
can work entirely with `EventBus` or `AsyncEventBus`.
|
|
1136
|
+
|
|
1137
|
+
<a id="development"></a>
|
|
1138
|
+
## Development and license
|
|
1139
|
+
|
|
1140
|
+
The project supports Python 3.10 and newer and uses setuptools as its build
|
|
1141
|
+
backend. The compatibility dependencies are installed only on Python 3.10.
|
|
1142
|
+
Development setup and checks are described under
|
|
1143
|
+
[Install](https://github.com/TahsinCr/python-eventflow/blob/main/README.md#install).
|
|
1144
|
+
Run `python -m benchmarks.benchmark_replay` from the repository root to compare
|
|
1145
|
+
history `latest()`/`query()` calls through the history and bus APIs, along with
|
|
1146
|
+
bounded sync and async replay setup. The standard-library benchmark defaults
|
|
1147
|
+
to 100,000 retained events and can be adjusted with `--events`, `--replay`,
|
|
1148
|
+
`--iterations`, and `--samples`.
|
|
1149
|
+
|
|
1150
|
+
Additional repeatable measurements use `python -m benchmarks.benchmark_publish`,
|
|
1151
|
+
`python -m benchmarks.benchmark_pipeline`, `python -m benchmarks.benchmark_stream`
|
|
1152
|
+
and `python -m benchmarks.benchmark_runtime`. The runtime-load benchmark compares
|
|
1153
|
+
sync and async publisher contention, async `max_concurrency` values, callback
|
|
1154
|
+
error rates, periodic schedule resources, and traced memory over a configurable
|
|
1155
|
+
event soak. It defaults to five samples, 500 publishes per publisher in each
|
|
1156
|
+
sample, and 100,000 soak events; use `--iterations`, `--publisher-counts`,
|
|
1157
|
+
`--schedule-counts`, `--samples` or `--soak-operations` to adjust the workload.
|
|
1158
|
+
Use `--output results.json` to retain the environment, throughput, p50/p95/p99
|
|
1159
|
+
latency and separately traced peak allocations. Stream rates count items per
|
|
1160
|
+
whole-stream operation. Runtime memory samples use `tracemalloc`, not process
|
|
1161
|
+
RSS, and include event history when configured. Local microbenchmarks are not
|
|
1162
|
+
deployment guarantees. Recorded results and methodology live under
|
|
1163
|
+
`benchmarks/results`.
|
|
1164
|
+
|
|
1165
|
+
Copyright © 2026 TahsinCr. EventFlow is licensed under the MIT License.
|
|
1166
|
+
See [LICENSE](https://github.com/TahsinCr/python-eventflow/blob/main/LICENSE) and
|
|
1167
|
+
[CHANGELOG](https://github.com/TahsinCr/python-eventflow/blob/main/CHANGELOG.md).
|
|
1168
|
+
|
|
1169
|
+
[version-shield]: https://img.shields.io/badge/version-v1.2-b7a3e3.svg?style=for-the-badge
|
|
1170
|
+
[python-shield]: https://img.shields.io/badge/Python-3.10%2B-3776AB.svg?style=for-the-badge&logo=python&logoColor=white
|
|
1171
|
+
[license-shield]: https://img.shields.io/badge/license-MIT-blue.svg?style=for-the-badge
|