multiloop 0.1.2__tar.gz → 0.1.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {multiloop-0.1.2 → multiloop-0.1.3}/AGENTS.md +17 -73
- {multiloop-0.1.2 → multiloop-0.1.3}/CHANGELOG.md +19 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/CITATION.cff +2 -2
- {multiloop-0.1.2 → multiloop-0.1.3}/CONTRIBUTING.md +0 -2
- {multiloop-0.1.2 → multiloop-0.1.3}/Cargo.lock +1 -8
- {multiloop-0.1.2 → multiloop-0.1.3}/Cargo.toml +3 -4
- {multiloop-0.1.2 → multiloop-0.1.3}/Makefile +1 -1
- {multiloop-0.1.2 → multiloop-0.1.3}/PKG-INFO +43 -133
- multiloop-0.1.3/README.md +271 -0
- multiloop-0.1.3/README_ZH.md +271 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/docs/API.md +54 -69
- {multiloop-0.1.2 → multiloop-0.1.3}/docs/API_ZH.md +55 -70
- {multiloop-0.1.2 → multiloop-0.1.3}/docs/CHOOSING.md +0 -3
- {multiloop-0.1.2 → multiloop-0.1.3}/docs/CHOOSING_ZH.md +0 -3
- {multiloop-0.1.2 → multiloop-0.1.3}/docs/CONCURRENCY.md +10 -39
- {multiloop-0.1.2 → multiloop-0.1.3}/docs/CONCURRENCY_ZH.md +10 -37
- multiloop-0.1.3/examples/05_multicore_pipeline.py +76 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/examples/README.md +1 -2
- {multiloop-0.1.2 → multiloop-0.1.3}/examples/README_ZH.md +1 -2
- {multiloop-0.1.2 → multiloop-0.1.3}/pyproject.toml +1 -8
- {multiloop-0.1.2 → multiloop-0.1.3}/src/channel.rs +261 -106
- {multiloop-0.1.2 → multiloop-0.1.3}/src/lib.rs +17 -73
- {multiloop-0.1.2 → multiloop-0.1.3}/src/metrics.rs +11 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/__init__.py +6 -8
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/__init__.pyi +65 -91
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_cancel.py +11 -4
- multiloop-0.1.3/src/multiloop/_channel_base.py +80 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_metrics.py +1 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_multiloop_core.pyi +19 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_sync.py +45 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_taskgroup.py +238 -32
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/context.py +37 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/exceptions.py +22 -2
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/pool.py +102 -55
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/primitives.py +262 -194
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/rwlock.py +79 -44
- {multiloop-0.1.2 → multiloop-0.1.3}/src/pool.rs +26 -19
- {multiloop-0.1.2 → multiloop-0.1.3}/src/rwlock.rs +84 -7
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_channel.py +109 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_pool.py +42 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_regression_bugs.py +86 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_sync_lock_event.py +24 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_sync_primitives.py +50 -6
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_taskgroup.py +145 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/uv.lock +1 -70
- multiloop-0.1.2/README.md +0 -361
- multiloop-0.1.2/README_ZH.md +0 -360
- multiloop-0.1.2/benchmarks/bench_asgi_throughput.py +0 -490
- multiloop-0.1.2/benchmarks/bench_wrk_asgi.py +0 -343
- multiloop-0.1.2/examples/05_asgi_websocket.py +0 -71
- multiloop-0.1.2/examples/06_wsgi_flask.py +0 -37
- multiloop-0.1.2/src/http.rs +0 -1286
- multiloop-0.1.2/src/multiloop/_channel_base.py +0 -338
- multiloop-0.1.2/src/multiloop/_http11.py +0 -413
- multiloop-0.1.2/src/multiloop/_lifespan.py +0 -223
- multiloop-0.1.2/src/multiloop/_websocket.py +0 -484
- multiloop-0.1.2/src/multiloop/asgi.py +0 -144
- multiloop-0.1.2/src/multiloop/cli.py +0 -438
- multiloop-0.1.2/src/multiloop/server.py +0 -342
- multiloop-0.1.2/src/multiloop/wsgi.py +0 -748
- multiloop-0.1.2/src/websocket.rs +0 -292
- multiloop-0.1.2/tests/test_asgi_backpressure.py +0 -71
- multiloop-0.1.2/tests/test_asgi_lifespan_ws.py +0 -781
- multiloop-0.1.2/tests/test_cli.py +0 -81
- multiloop-0.1.2/tests/test_http_pipelining.py +0 -70
- multiloop-0.1.2/tests/test_server.py +0 -454
- multiloop-0.1.2/tests/test_wsgi.py +0 -221
- {multiloop-0.1.2 → multiloop-0.1.3}/.editorconfig +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/FUNDING.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/ISSUE_TEMPLATE/bug_report.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/ISSUE_TEMPLATE/feature_request.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/dependabot.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/pull_request_template.md +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/release-drafter.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/workflows/ci.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/workflows/publish.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.github/workflows/release-drafter.yml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.gitignore +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.pre-commit-config.yaml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/.python-version +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/CODEOWNERS +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/CODE_OF_CONDUCT.md +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/LICENSE +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/SECURITY.md +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/benchmarks/bench_multithread_loops.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/benchmarks/benchmark_pull_model.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/build.rs +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/deny.toml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/examples/00_pool_basics.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/examples/01_channels_select.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/examples/02_waitgroup_once.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/examples/03_taskgroup_timeout.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/examples/04_sync_primitives.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/rust-toolchain.toml +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_logging.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_options.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/_rust.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/py.typed +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/multiloop/testing/__init__.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/src/waitgroup.rs +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/conftest.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_cancel_scope.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_docs.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_free_threading.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_go_parity.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_logging.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_loop_factory.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_perf_regression.py +0 -0
- {multiloop-0.1.2 → multiloop-0.1.3}/tests/test_sync_condition_barrier.py +0 -0
|
@@ -34,22 +34,15 @@ invariants, token conservation laws, and multi-thread signal forwarding**.
|
|
|
34
34
|
```
|
|
35
35
|
multiloop/
|
|
36
36
|
├── src/
|
|
37
|
-
│ ├── multiloop/ # Python package (
|
|
37
|
+
│ ├── multiloop/ # Python package (15 modules)
|
|
38
38
|
│ │ ├── __init__.py # Public API surface — start here
|
|
39
39
|
│ │ ├── pool.py # EventLoopThreadPool engine
|
|
40
40
|
│ │ ├── primitives.py # Channel, select_channel, AsyncWaitGroup, AsyncOnce
|
|
41
41
|
│ │ ├── context.py # AsyncContext (O(1) set tracking)
|
|
42
42
|
│ │ ├── _cancel.py # CancelScope, fail_after, move_on_after, shield
|
|
43
|
-
│ │ ├── _sync.py # Lock, Semaphore, Event, Condition, Barrier
|
|
43
|
+
│ │ ├── _sync.py # Lock, Semaphore, Event, Condition, Barrier, CapacityLimiter
|
|
44
44
|
│ │ ├── _taskgroup.py # TaskGroup, TaskHandle
|
|
45
45
|
│ │ ├── rwlock.py # AsyncRWMutex (async read-write lock, Rust-backed)
|
|
46
|
-
│ │ ├── asgi.py # MultiloopASGIWorker (ASGI 3.0, Lifespan, WebSockets)
|
|
47
|
-
│ │ ├── _http11.py # Http11Protocol (Rust SIMD httparse & fused TCP write)
|
|
48
|
-
│ │ ├── _websocket.py # WebSocketProtocol (Rust SIMD unmasking & RFC 6455)
|
|
49
|
-
│ │ ├── _lifespan.py # ASGILifespanManager (ASGI lifespan state machine)
|
|
50
|
-
│ │ ├── wsgi.py # MultiloopWSGIWorker (WSGI 1.0.1, PEP 3333 synchronous runner)
|
|
51
|
-
│ │ ├── cli.py # multiloop run CLI server runner
|
|
52
|
-
│ │ ├── server.py # ConnectionPinningServer (platform-adaptive multi-core listener)
|
|
53
46
|
│ │ ├── _channel_base.py # Shared waiter deques, _wake_all, _wakeup_next
|
|
54
47
|
│ │ ├── _rust.py # _try_import_rust_class helper
|
|
55
48
|
│ │ ├── _metrics.py # MetricsCollector
|
|
@@ -62,11 +55,9 @@ multiloop/
|
|
|
62
55
|
│ ├── pool.rs # NativeWorkerPool & PollerGuard
|
|
63
56
|
│ ├── channel.rs # Channel, RawAsyncChannel (Anti-Barging FIFO)
|
|
64
57
|
│ ├── waitgroup.rs # RawAsyncWaitGroup & WaitGroupInner
|
|
65
|
-
│
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
├── tests/ # 27 test files (pytest + pytest-asyncio)
|
|
69
|
-
├── benchmarks/ # 5 benchmark scripts (including bench_wrk_asgi.py)
|
|
58
|
+
│ └── rwlock.rs # RawAsyncRWMutex (64-bit atomic state machine)
|
|
59
|
+
├── tests/ # 22 test files (pytest + pytest-asyncio)
|
|
60
|
+
├── benchmarks/ # Multi-thread benchmarks
|
|
70
61
|
├── examples/ # Runnable examples (python examples/00_*.py)
|
|
71
62
|
├── docs/
|
|
72
63
|
│ ├── API.md # Complete API reference
|
|
@@ -167,14 +158,13 @@ searches, never modify.
|
|
|
167
158
|
`_worker_dispatcher` is the per-worker event loop that calls `pop_work()`
|
|
168
159
|
and spawns tasks.
|
|
169
160
|
|
|
170
|
-
3. **`src/lib.rs`** — Rust backend.
|
|
171
|
-
(global + per-worker flume queues, batch-pull pop_work), `
|
|
172
|
-
(
|
|
173
|
-
`RawAsyncWaitGroup` (atomic counter + mutex-protected waiter list)
|
|
161
|
+
3. **`src/lib.rs`** — Rust backend. Five `#[pyclass]` types: `NativeWorkerPool`
|
|
162
|
+
(global + per-worker flume queues, batch-pull pop_work), `RawAsyncChannel`
|
|
163
|
+
(native async channel with anti-barging FIFO), `AtomicMetrics` (padded atomic counters),
|
|
164
|
+
`RawAsyncWaitGroup` (atomic counter + mutex-protected waiter list), and `RawAsyncRWMutex`.
|
|
174
165
|
|
|
175
|
-
4. **`src/multiloop/primitives.py`** — `Channel` Python
|
|
176
|
-
|
|
177
|
-
`AsyncOnce`.
|
|
166
|
+
4. **`src/multiloop/primitives.py`** — `Channel` Python facade over native Rust
|
|
167
|
+
`RawAsyncChannel`, `select_channel`, `AsyncWaitGroup`, `AsyncOnce`.
|
|
178
168
|
|
|
179
169
|
5. **`src/multiloop/_cancel.py`** — `CancelScope` with shield semantics, using
|
|
180
170
|
`task.uncancel()` / `task.cancel()` snapshot/restore on Python 3.11+.
|
|
@@ -196,7 +186,6 @@ __init__.py ──┬── pool.py ────────── _rust.py ─
|
|
|
196
186
|
├── _taskgroup.py
|
|
197
187
|
├── context.py
|
|
198
188
|
├── rwlock.py
|
|
199
|
-
├── asgi.py ────────── server.py
|
|
200
189
|
├── exceptions.py
|
|
201
190
|
├── _metrics.py
|
|
202
191
|
├── _options.py
|
|
@@ -269,9 +258,6 @@ pops so no single worker greedily drains the global queue.
|
|
|
269
258
|
| `AtomicMetrics` | `src/metrics.rs` | `std::sync::atomic` | 64-byte-padded per-worker counters (`active`, `completed`, `global_pull_count`, `park_count`, etc.) — prevents false sharing |
|
|
270
259
|
| `RawAsyncWaitGroup` | `src/waitgroup.rs` | parking_lot `Mutex<WaitGroupInner>` | Go-style atomic counter + generation + waiter list with single-mutex state machine |
|
|
271
260
|
| `RawAsyncRWMutex` | `src/rwlock.rs` | parking_lot `Mutex<RWMutexInner>` | 64-bit atomic state machine, writer-preference async read-write lock with cancellation cleanup |
|
|
272
|
-
| `FastHttpConnection` | `src/http.rs` | httparse + PyO3 | Stateful HTTP/1.1 & WebSocket connection state machine, RFC 9112 chunk parser, request smuggling defense & zero-alloc response serializer |
|
|
273
|
-
| `FastHttpParser` | `src/http.rs` | httparse | SIMD zero-copy HTTP/1.x header parser with 22 interned headers, zero-alloc path/value interning & PyBuffer zero-copy |
|
|
274
|
-
| `FastWebSocketParser` & `fast_websocket_unmask` | `src/websocket.rs` | PyO3 | 32-byte 4-way SIMD vectorization unmasking, Option<[u8; 4]> allocation-free mask parsing, and RFC 6455 frame header parsing |
|
|
275
261
|
|
|
276
262
|
### Python-side patterns
|
|
277
263
|
|
|
@@ -290,17 +276,17 @@ pops so no single worker greedily drains the global queue.
|
|
|
290
276
|
while re-acquiring the lock after being notified, it automatically forwards
|
|
291
277
|
the notification to the next waiter to conserve notification tokens.
|
|
292
278
|
|
|
293
|
-
###
|
|
279
|
+
### Seven non-obvious design decisions
|
|
294
280
|
|
|
295
281
|
These are the places where a change that looks like a simplification usually
|
|
296
282
|
breaks a real correctness or performance property. **Read each before modifying
|
|
297
283
|
the related code.**
|
|
298
284
|
|
|
299
|
-
1. **
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
285
|
+
1. **Anti-barging FIFO token conservation in `Channel`** (`channel.rs:270-360`, `primitives.py:180-240`):
|
|
286
|
+
`RawAsyncChannel` enforces strict FIFO ordering so external `try_send` cannot barge ahead
|
|
287
|
+
of queued putters. Woken putters hold reservations via `in_flight_putters` until `claim_put`.
|
|
288
|
+
If a woken putter is cancelled, `unregister_putter` automatically forwards the reservation
|
|
289
|
+
to the next queued putter or decrements the reservation count, conserving tokens and preventing deadlocks.
|
|
304
290
|
|
|
305
291
|
2. **`CancelScope` shield semantics** (`_cancel.py:97-205`):
|
|
306
292
|
Shielded scopes snapshot and clear `task.cancelling()`, then restore it on
|
|
@@ -337,48 +323,6 @@ the related code.**
|
|
|
337
323
|
full, preserving liveness. Drain phase validates all three sources (global queue,
|
|
338
324
|
private buffer, local channel) are completely empty before worker loop termination.
|
|
339
325
|
|
|
340
|
-
8. **Darwin vs Linux `SO_REUSEPORT` asymmetric routing trap** (`src/multiloop/server.py:99-130`):
|
|
341
|
-
Linux `SO_REUSEPORT` hashes 4-tuples across multiple listening sockets with kernel-level
|
|
342
|
-
load balancing. On macOS (Darwin/BSD), `SO_REUSEPORT` does not implement symmetric
|
|
343
|
-
hashing and routes 100% of loopback connections to the newest bound socket, causing
|
|
344
|
-
severe worker starvation and degrading multi-core throughput to single-worker limits (~50k QPS).
|
|
345
|
-
`server.py` restricts `SO_REUSEPORT` to Linux only; on macOS/Darwin, workers share a single
|
|
346
|
-
listening socket and race concurrently on `loop.sock_accept(shared_sock)`, guaranteeing
|
|
347
|
-
perfect 25% load distribution and ~96k+ multi-core QPS.
|
|
348
|
-
|
|
349
|
-
9. **Python 3.14t Free-Threaded memory safety via `PyBytes::new_with`** (`src/websocket.rs:80-110`, `src/http.rs`):
|
|
350
|
-
Under Python 3.14t (No-GIL), modifying Python buffer objects in-place across threads
|
|
351
|
-
causes data races and invalidates Python's immutability invariant. Rust SIMD extensions must
|
|
352
|
-
allocate Python objects through atomic single-pass constructors like
|
|
353
|
-
`PyBytes::new_with(py, len, |buf| ...)` to populate unmasked payload bytes and serialized
|
|
354
|
-
HTTP responses without intermediate copies or in-place mutations.
|
|
355
|
-
|
|
356
|
-
10. **Unified Protocol Calculator (Rust) & Pure Transport Messenger (Python)** (`src/http.rs`, `_http11.py`):
|
|
357
|
-
`multiloop` strictly unifies HTTP/1.1 and WebSocket processing by eliminating parsing brain-split.
|
|
358
|
-
Rust `FastHttpConnection` owns 100% of protocol computation: SIMD header parsing, RFC 9112 chunked
|
|
359
|
-
streaming decoding with trailer verification, request smuggling defense (CL-TE/TE-TE rejection),
|
|
360
|
-
zero-allocation response wire serialization, and WebSocket frame fusion. Python `Http11Protocol` acts
|
|
361
|
-
purely as an asynchronous transport messenger (~380 lines), driving socket I/O, managing `_body_queue`
|
|
362
|
-
backpressure with `pump_events()` residue draining, and dispatching ASGI 3.0 coroutines.
|
|
363
|
-
|
|
364
|
-
11. **Worker EventLoop Network Disconnection Exception Filtering** (`src/multiloop/pool.py:230-240`):
|
|
365
|
-
During high-concurrency client disconnects or abrupt socket termination, background transport socket
|
|
366
|
-
buffer flushes in `_SelectorSocketTransport` trigger `BrokenPipeError`, `ConnectionResetError`, or
|
|
367
|
-
`ConnectionAbortedError` without an active awaiter. Custom loop exception handler intercepts and suppresses
|
|
368
|
-
these normal network disconnection events, preventing unretrieved Future logs while keeping unexpected
|
|
369
|
-
runtime bugs intact.
|
|
370
|
-
|
|
371
|
-
12. **Multi-Thread `server.close()` Task Set Snapshot Isolation** (`src/multiloop/server.py:290-305`):
|
|
372
|
-
Under Python 3.14t true physical multi-core execution, multiple worker threads concurrently finish
|
|
373
|
-
requests and discard tasks from `self._worker_conn_tasks` while `server.close()` iterates over them.
|
|
374
|
-
`server.py` snapshots task collections with `list(tasks)` and `list(self._worker_conn_tasks.values())`
|
|
375
|
-
to prevent `RuntimeError: Set changed size during iteration`.
|
|
376
|
-
|
|
377
|
-
13. **ASGI Lifespan State Isolation via `.copy()`** (`src/multiloop/_http11.py:170`, `src/multiloop/_websocket.py:133`):
|
|
378
|
-
To conform to the ASGI 3.0 specification and prevent cross-request context pollution (e.g. `request.state.user_id`
|
|
379
|
-
leaking between users) and multi-core dictionary write contention under Python 3.14t, `scope["state"]` is
|
|
380
|
-
instantiated via `self.lifespan_state.copy()`.
|
|
381
|
-
|
|
382
326
|
---
|
|
383
327
|
|
|
384
328
|
## 6. Quick Commands
|
|
@@ -5,6 +5,25 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.1.3] - 2026-09-17
|
|
9
|
+
|
|
10
|
+
### Added & Optimized
|
|
11
|
+
- **Cross-Loop Structured Concurrency (`TaskGroup` & `AsyncContext`)**:
|
|
12
|
+
- `TaskGroup(pool=...)` multi-loop task distribution with isolated per-task child `CancelScope` instances.
|
|
13
|
+
- Remote drain barrier on context exit, bidirectional cascading cancellation, and unified `ExceptionGroup` aggregation.
|
|
14
|
+
- Unified `AsyncContext.task_group(pool=...)` lifecycle tracking and automatic cascading cancellation.
|
|
15
|
+
- **Native Synchronous Channel Communication (`Channel.send_sync` & `Channel.recv_sync`)**:
|
|
16
|
+
- Replaced busy-spin polling with native `parking_lot::Condvar` wait/notify in `RawAsyncChannel`.
|
|
17
|
+
- Releases the Python GIL via `py.detach()` during blocking waits, eliminating CPU waste.
|
|
18
|
+
- Added `ReceiveChannel.recv_sync(timeout=None)` to segregation views.
|
|
19
|
+
- **Concurrency & Free-Threaded (Python 3.14t) Hardening**:
|
|
20
|
+
- `AsyncRWMutex`: Protected reentrancy depth and writer task with `self._depth_lock = threading.Lock()`, with out-of-lock `_wake_all()` execution to eliminate deadlocks.
|
|
21
|
+
- `EventLoopThreadPool`: Implemented condition-variable drain barrier `self._drain_barrier` and event-driven worker dispatcher.
|
|
22
|
+
- `AtomicMetrics`: Added `add_global_pull` batch counter accumulation and upgraded pool metrics to `RwLock`.
|
|
23
|
+
- `_channel_base.py`: Enforced $O(1)$ waiter unregistration via `OrderedDict.pop()`.
|
|
24
|
+
- `AsyncWaitGroup`: Removed GC finalizer `self._wg.done()` invocation in `_TrackedCoroutine.__del__` to prevent cross-thread re-entrancy and deadlocks under multi-threaded GC.
|
|
25
|
+
- Standardized cross-language `ThreadPoolClosedError` exception hierarchy.
|
|
26
|
+
|
|
8
27
|
## [0.1.2] - 2026-08-21
|
|
9
28
|
|
|
10
29
|
### Added & Optimized
|
|
@@ -122,8 +122,6 @@ Performance-sensitive changes should be validated against the benchmark suite:
|
|
|
122
122
|
```bash
|
|
123
123
|
uv run python benchmarks/bench_multithread_loops.py
|
|
124
124
|
uv run python benchmarks/benchmark_pull_model.py
|
|
125
|
-
uv run python benchmarks/bench_asgi_throughput.py
|
|
126
|
-
uv run python benchmarks/benchmark_winuvloop.py
|
|
127
125
|
```
|
|
128
126
|
|
|
129
127
|
---
|
|
@@ -81,12 +81,6 @@ version = "0.5.0"
|
|
|
81
81
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
82
82
|
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
|
|
83
83
|
|
|
84
|
-
[[package]]
|
|
85
|
-
name = "httparse"
|
|
86
|
-
version = "1.10.1"
|
|
87
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
88
|
-
checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87"
|
|
89
|
-
|
|
90
84
|
[[package]]
|
|
91
85
|
name = "js-sys"
|
|
92
86
|
version = "0.3.103"
|
|
@@ -115,10 +109,9 @@ dependencies = [
|
|
|
115
109
|
|
|
116
110
|
[[package]]
|
|
117
111
|
name = "multiloop-core"
|
|
118
|
-
version = "0.1.
|
|
112
|
+
version = "0.1.3"
|
|
119
113
|
dependencies = [
|
|
120
114
|
"flume",
|
|
121
|
-
"httparse",
|
|
122
115
|
"parking_lot",
|
|
123
116
|
"pyo3",
|
|
124
117
|
"pyo3-build-config",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[package]
|
|
2
2
|
name = "multiloop-core"
|
|
3
|
-
version = "0.1.
|
|
3
|
+
version = "0.1.3"
|
|
4
4
|
edition = "2021"
|
|
5
5
|
readme = "README.md"
|
|
6
6
|
|
|
@@ -12,7 +12,6 @@ crate-type = ["cdylib", "rlib"]
|
|
|
12
12
|
pyo3 = "0.29"
|
|
13
13
|
flume = "0.11"
|
|
14
14
|
parking_lot = "0.12"
|
|
15
|
-
httparse = "1.10.1"
|
|
16
15
|
|
|
17
16
|
[build-dependencies]
|
|
18
17
|
pyo3-build-config = "0.29"
|
|
@@ -24,7 +23,7 @@ test-init = ["pyo3/auto-initialize"]
|
|
|
24
23
|
|
|
25
24
|
[profile.release]
|
|
26
25
|
opt-level = 3
|
|
27
|
-
lto =
|
|
26
|
+
lto = "thin"
|
|
28
27
|
codegen-units = 1
|
|
29
28
|
panic = "unwind"
|
|
30
|
-
strip =
|
|
29
|
+
strip = false
|
|
@@ -16,7 +16,7 @@ test-slow:
|
|
|
16
16
|
uv run pytest -m slow -o addopts=""
|
|
17
17
|
|
|
18
18
|
bench:
|
|
19
|
-
for f in benchmarks/benchmark_pull_model.py benchmarks/
|
|
19
|
+
for f in benchmarks/benchmark_pull_model.py benchmarks/bench_multithread_loops.py; do \
|
|
20
20
|
uv run python "$$f"; \
|
|
21
21
|
done
|
|
22
22
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: multiloop
|
|
3
|
-
Version: 0.1.
|
|
3
|
+
Version: 0.1.3
|
|
4
4
|
Classifier: Development Status :: 4 - Beta
|
|
5
5
|
Classifier: Intended Audience :: Developers
|
|
6
6
|
Classifier: License :: OSI Approved :: MIT License
|
|
@@ -14,7 +14,7 @@ Classifier: Topic :: System :: Networking
|
|
|
14
14
|
Classifier: Framework :: AsyncIO
|
|
15
15
|
License-File: LICENSE
|
|
16
16
|
Summary: High-performance multi-event-loop concurrency engine for Python 3.14t with Rust core
|
|
17
|
-
Keywords: asyncio,concurrency,multithreading,free-threading,no-gil,golang,channel,eventloop,multieventloop,sync,AsyncRwLock,rwlock
|
|
17
|
+
Keywords: asyncio,concurrency,multithreading,free-threading,no-gil,golang,channel,eventloop,multieventloop,sync,AsyncRwLock,rwlock
|
|
18
18
|
Author-email: Han Kaihong <hankaihong@outlook.com>
|
|
19
19
|
License-Expression: MIT
|
|
20
20
|
Requires-Python: >=3.14
|
|
@@ -29,11 +29,10 @@ Project-URL: Repository, https://github.com/hankaihong1/multiloop
|
|
|
29
29
|
|
|
30
30
|
[](https://github.com/hankaihong1/multiloop/actions/workflows/ci.yml)
|
|
31
31
|
[](https://www.python.org/)
|
|
32
|
-
[](benchmarks/bench_wrk_asgi.py)
|
|
32
|
+
[](https://www.rust-lang.org/)
|
|
34
33
|
[](LICENSE)
|
|
35
34
|
|
|
36
|
-
> ⚡ **A high-performance multi-event-loop concurrency
|
|
35
|
+
> ⚡ **A high-performance multi-event-loop concurrency engine and Go-style concurrency toolkit for Python 3.14t (Free-Threaded / No-GIL), powered by an ultra-fast Rust core.**
|
|
37
36
|
|
|
38
37
|
**[中文版 (Chinese)](README_ZH.md)**
|
|
39
38
|
|
|
@@ -42,11 +41,10 @@ Project-URL: Repository, https://github.com/hankaihong1/multiloop
|
|
|
42
41
|
## Table of Contents
|
|
43
42
|
|
|
44
43
|
- [1. Installation](#1-installation)
|
|
45
|
-
- [2.
|
|
46
|
-
- [3.
|
|
47
|
-
- [4.
|
|
48
|
-
- [5.
|
|
49
|
-
- [6. Miscellaneous & Community](#6-miscellaneous--community)
|
|
44
|
+
- [2. Multi-Core Performance & Benchmarks](#2-multi-core-performance--benchmarks)
|
|
45
|
+
- [3. Python API Usage](#3-python-api-usage)
|
|
46
|
+
- [4. Architecture & Developer Guide](#4-architecture--developer-guide)
|
|
47
|
+
- [5. Miscellaneous & Community](#5-miscellaneous--community)
|
|
50
48
|
|
|
51
49
|
---
|
|
52
50
|
|
|
@@ -84,106 +82,36 @@ maturin develop --release
|
|
|
84
82
|
|
|
85
83
|
---
|
|
86
84
|
|
|
87
|
-
## 2.
|
|
85
|
+
## 2. Multi-Core Performance & Benchmarks
|
|
88
86
|
|
|
89
87
|
### Multi-Core Throughput (Python 3.14t)
|
|
90
88
|
|
|
91
|
-
`multiloop` achieves true physical multi-core scalability without GIL bottlenecks.
|
|
89
|
+
`multiloop` achieves true physical multi-core scalability without GIL bottlenecks across isolated event loops.
|
|
92
90
|
|
|
93
|
-
*
|
|
91
|
+
*Multi-core scheduling and throughput benchmarks on Apple M1 (8 Cores, 8GB RAM, Python 3.14.6 Free-Threaded No-GIL, release build):*
|
|
94
92
|
|
|
95
|
-
| Workload (Benchmark Scenario) |
|
|
96
|
-
|
|
97
|
-
| **
|
|
98
|
-
| **
|
|
99
|
-
| **CPU-bound Dispatch (40 × 2M ops)** | 2.88 s | 0.79 s | **0.60 s** | **4.83x** | — |
|
|
93
|
+
| Workload (Benchmark Scenario) | 1 Worker Loop | 4 Worker Loops | 8 Worker Loops | Max Speedup |
|
|
94
|
+
|---|---|---|---|---|
|
|
95
|
+
| **CPU-bound Dispatch (40 × 2M ops)** | 2.88 s | 0.79 s | **0.60 s** | **4.83x** |
|
|
96
|
+
| **Cross-Thread Channel Ping-Pong (100k)** | 185k msgs/s | 340k msgs/s | **410k msgs/s** | **2.21x** |
|
|
100
97
|
|
|
101
98
|
### Reproducing Benchmarks
|
|
102
99
|
|
|
103
|
-
Run the
|
|
100
|
+
Run the built-in multi-thread benchmark suite to measure physical multi-core throughput:
|
|
104
101
|
|
|
105
102
|
```bash
|
|
106
|
-
# Measure
|
|
107
|
-
uv run python benchmarks/
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
You can also run the built-in cross-platform pure-Python socket benchmark suite directly (zero external dependencies required):
|
|
111
|
-
|
|
112
|
-
```bash
|
|
113
|
-
uv run python benchmarks/bench_asgi_throughput.py
|
|
114
|
-
```
|
|
115
|
-
|
|
116
|
-
---
|
|
117
|
-
|
|
118
|
-
## 3. Web Server CLI
|
|
119
|
-
|
|
120
|
-
`multiloop` includes a high-performance CLI server runner. Unlike traditional multi-process process managers (`gunicorn -w 4` or `uvicorn --workers 4`), `multiloop run` operates across **multi-threaded isolated event loops in a single process** with shared memory and zero IPC serialization overhead.
|
|
121
|
-
|
|
122
|
-
### Run a FastAPI Application
|
|
123
|
-
|
|
124
|
-
Create `main.py`:
|
|
125
|
-
|
|
126
|
-
```python
|
|
127
|
-
from fastapi import FastAPI
|
|
128
|
-
|
|
129
|
-
app = FastAPI(title="My multiloop API")
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
@app.get("/")
|
|
133
|
-
def read_root():
|
|
134
|
-
return {"message": "Hello from FastAPI on multiloop!"}
|
|
135
|
-
```
|
|
136
|
-
|
|
137
|
-
Start the multi-worker server with 1 command:
|
|
138
|
-
|
|
139
|
-
```bash
|
|
140
|
-
multiloop run main:app --port 8000 --workers 4 --reload
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
Open [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs) to view the live interactive Swagger UI!
|
|
103
|
+
# Measure multi-thread loop pool scaling
|
|
104
|
+
uv run python benchmarks/bench_multithread_loops.py
|
|
144
105
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
`multiloop run` automatically detects WSGI applications (PEP 3333). Create `app.py`:
|
|
148
|
-
|
|
149
|
-
```python
|
|
150
|
-
from flask import Flask
|
|
151
|
-
|
|
152
|
-
app = Flask(__name__)
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
@app.route("/")
|
|
156
|
-
def index():
|
|
157
|
-
return "Hello from Flask running on multiloop WSGI thread pool!"
|
|
158
|
-
```
|
|
159
|
-
|
|
160
|
-
Launch the WSGI application with multi-thread pool offloading:
|
|
161
|
-
|
|
162
|
-
```bash
|
|
163
|
-
multiloop run app:app --port 5000 --workers 4
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
### CLI Parameter Reference
|
|
167
|
-
|
|
168
|
-
```bash
|
|
169
|
-
multiloop run <module:app> [OPTIONS]
|
|
106
|
+
# Measure lock-free work stealing channel pull throughput
|
|
107
|
+
uv run python benchmarks/benchmark_pull_model.py
|
|
170
108
|
```
|
|
171
109
|
|
|
172
|
-
| Option | Default | Description |
|
|
173
|
-
| :--- | :--- | :--- |
|
|
174
|
-
| `<module:app>` | *(Required)* | Application import string, e.g. `main:app` or `my_project.wsgi:application` |
|
|
175
|
-
| `--host` | `127.0.0.1` | Network interface to bind on (`0.0.0.0` for public access) |
|
|
176
|
-
| `--port` | `8000` | Port to bind on (`0` for ephemeral random port) |
|
|
177
|
-
| `--workers` | `auto` | Number of worker event loop threads (defaults to CPU core count) |
|
|
178
|
-
| `--reload` | `off` | Enable automatic hot-reloading upon file modifications |
|
|
179
|
-
| `--interface` | `auto` | Protocol interface: `auto`, `asgi` (FastAPI/Starlette), or `wsgi` (Django/Flask) |
|
|
180
|
-
| `--log-level` | `info` | Logging verbosity: `debug`, `info`, `warning`, `error` |
|
|
181
|
-
|
|
182
110
|
---
|
|
183
111
|
|
|
184
|
-
##
|
|
112
|
+
## 3. Python API Usage
|
|
185
113
|
|
|
186
|
-
### 1. Multi-Core Thread Pool
|
|
114
|
+
### 1. Multi-Core Thread Pool
|
|
187
115
|
|
|
188
116
|
```python
|
|
189
117
|
import asyncio
|
|
@@ -198,21 +126,17 @@ async def heavy_task(x: int) -> int:
|
|
|
198
126
|
async def main() -> None:
|
|
199
127
|
# async with manages the pool lifecycle automatically
|
|
200
128
|
async with multiloop.EventLoopThreadPool(num_threads=4) as pool:
|
|
201
|
-
#
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
fut2 = pool.submit(heavy_task, 21, pin_to=0)
|
|
206
|
-
|
|
207
|
-
# Await results computed across physical CPU cores
|
|
208
|
-
print("Results:", await fut1, await fut2) # Output: 42 42
|
|
129
|
+
# submit() round-robins across workers with work stealing
|
|
130
|
+
fut = pool.submit(heavy_task, 42)
|
|
131
|
+
result = await fut
|
|
132
|
+
print(f"Computed result: {result}")
|
|
209
133
|
|
|
210
134
|
|
|
211
135
|
if __name__ == "__main__":
|
|
212
136
|
asyncio.run(main())
|
|
213
137
|
```
|
|
214
138
|
|
|
215
|
-
### 2. Go-Style
|
|
139
|
+
### 2. Go-Style Channels & select_channel
|
|
216
140
|
|
|
217
141
|
```python
|
|
218
142
|
import asyncio
|
|
@@ -220,12 +144,11 @@ import multiloop
|
|
|
220
144
|
|
|
221
145
|
|
|
222
146
|
async def main() -> None:
|
|
223
|
-
ch1
|
|
224
|
-
ch2
|
|
147
|
+
ch1 = multiloop.Channel(maxsize=10)
|
|
148
|
+
ch2 = multiloop.Channel(maxsize=10)
|
|
225
149
|
|
|
226
150
|
async def producer() -> None:
|
|
227
|
-
await ch1.send("
|
|
228
|
-
ch1.close()
|
|
151
|
+
await ch1.send("Message from Worker A")
|
|
229
152
|
|
|
230
153
|
asyncio.create_task(producer())
|
|
231
154
|
|
|
@@ -304,7 +227,7 @@ Explore more standalone scripts in [`examples/`](examples/README.md).
|
|
|
304
227
|
|
|
305
228
|
---
|
|
306
229
|
|
|
307
|
-
##
|
|
230
|
+
## 4. Architecture & Developer Guide
|
|
308
231
|
|
|
309
232
|
### Core Architecture
|
|
310
233
|
|
|
@@ -312,7 +235,7 @@ Explore more standalone scripts in [`examples/`](examples/README.md).
|
|
|
312
235
|
|
|
313
236
|
```mermaid
|
|
314
237
|
graph TD
|
|
315
|
-
UserApp[User Application /
|
|
238
|
+
UserApp[User Application / Task Pipeline] -->|pool.submit| Scheduler[Round-Robin Scheduler]
|
|
316
239
|
|
|
317
240
|
subgraph multiloop Core Engine
|
|
318
241
|
Scheduler -->|Round-Robin Notify| W1[Worker Loop Thread 1]
|
|
@@ -329,6 +252,7 @@ graph TD
|
|
|
329
252
|
FastChan <--> Select[multiloop.select_channel]
|
|
330
253
|
FastChan <--> Context[AsyncContext]
|
|
331
254
|
RustCore <--> WaitGroup[AsyncWaitGroup]
|
|
255
|
+
RustCore <--> RWMutex[AsyncRWMutex]
|
|
332
256
|
end
|
|
333
257
|
```
|
|
334
258
|
|
|
@@ -341,41 +265,27 @@ make develop
|
|
|
341
265
|
# 2. Run all linter & type checks (0 warnings, strict typing)
|
|
342
266
|
make lint
|
|
343
267
|
|
|
344
|
-
# 3. Run complete test suite (
|
|
268
|
+
# 3. Run complete test suite (320+ tests)
|
|
345
269
|
make test
|
|
346
270
|
```
|
|
347
271
|
|
|
348
272
|
---
|
|
349
273
|
|
|
350
|
-
##
|
|
274
|
+
## 5. Miscellaneous & Community
|
|
351
275
|
|
|
352
|
-
###
|
|
276
|
+
### Formal Concurrency Guarantees
|
|
353
277
|
|
|
354
|
-
|
|
|
278
|
+
| Invariant / Primitive | Physical Concurrency Guarantee | Underlying State Machine |
|
|
355
279
|
|---|---|---|
|
|
356
|
-
|
|
|
357
|
-
| `Barrier`
|
|
358
|
-
| `select_channel`
|
|
359
|
-
| Waiter
|
|
360
|
-
| `AsyncContext.cancel()` |
|
|
361
|
-
| `CancelScope`
|
|
362
|
-
| Windows | Proactor: single acceptor listener model | Documented platform behavior |
|
|
280
|
+
| Python 3.14t Free-Threading | Physical multi-core parallelism across isolated event loops | Lock-free Rust queues + OS mutex protected waiter lists |
|
|
281
|
+
| `Barrier` Cancellation | Auto-healing: cancelled party immediately breaks round & wakes all waiters | Monotonic generation counter + atomic `_broken` state machine |
|
|
282
|
+
| `select_channel` Arbitration | 100% deterministic arbitration without message loss or starvation | Two-phase arbiter: uniform random probe + single-arbiter registration |
|
|
283
|
+
| Waiter Unregistration | $O(1)$ constant time cancellation per waiter | `collections.OrderedDict` hash lookup & removal |
|
|
284
|
+
| `AsyncContext.cancel()` | Directly cancels active coroutines on worker OS threads | Injected `CancelScope` cross-thread cancellation |
|
|
285
|
+
| `CancelScope` Shielding | Zero-leak symmetric cancellation accounting | CPython 3.11+ `task.cancelling()` snapshot & restore |
|
|
363
286
|
|
|
364
287
|
For formal invariants and concurrency design principles, see [docs/CONCURRENCY.md](docs/CONCURRENCY.md).
|
|
365
288
|
|
|
366
|
-
### Live Demo
|
|
367
|
-
|
|
368
|
-
Want to test a live production setup without writing code?
|
|
369
|
-
[multiloop-fastapi-demo](https://github.com/hankaihong1/multiloop-fastapi-demo) is
|
|
370
|
-
a real FastAPI application served directly by `multiloop run` / `MultiloopASGIWorker` without uvicorn:
|
|
371
|
-
|
|
372
|
-
```bash
|
|
373
|
-
git clone https://github.com/hankaihong1/multiloop-fastapi-demo
|
|
374
|
-
cd multiloop-fastapi-demo
|
|
375
|
-
uv sync
|
|
376
|
-
uv run python app.py # then open http://127.0.0.1:8000
|
|
377
|
-
```
|
|
378
|
-
|
|
379
289
|
### Community & License
|
|
380
290
|
|
|
381
291
|
- Complete API Reference: [docs/API.md](docs/API.md)
|