rwlocker 2.0__tar.gz → 3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {rwlocker-2.0/rwlocker.egg-info → rwlocker-3.0}/PKG-INFO +54 -21
- {rwlocker-2.0 → rwlocker-3.0}/README-pypi.md +51 -18
- {rwlocker-2.0 → rwlocker-3.0}/pyproject.toml +5 -3
- rwlocker-3.0/rwlocker/__init__.py +67 -0
- {rwlocker-2.0 → rwlocker-3.0}/rwlocker/async_rwlock.py +472 -150
- {rwlocker-2.0 → rwlocker-3.0}/rwlocker/thread_rwlock.py +438 -91
- {rwlocker-2.0 → rwlocker-3.0/rwlocker.egg-info}/PKG-INFO +54 -21
- rwlocker-2.0/rwlocker/__init__.py +0 -54
- {rwlocker-2.0 → rwlocker-3.0}/LICENSE +0 -0
- {rwlocker-2.0 → rwlocker-3.0}/MANIFEST.in +0 -0
- {rwlocker-2.0 → rwlocker-3.0}/rwlocker.egg-info/SOURCES.txt +0 -0
- {rwlocker-2.0 → rwlocker-3.0}/rwlocker.egg-info/dependency_links.txt +0 -0
- {rwlocker-2.0 → rwlocker-3.0}/rwlocker.egg-info/top_level.txt +0 -0
- {rwlocker-2.0 → rwlocker-3.0}/setup.cfg +0 -0
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: rwlocker
|
|
3
|
-
Version:
|
|
3
|
+
Version: 3.0
|
|
4
4
|
Summary: Advanced, High-Performance, and State-Machine Based Synchronous/Asynchronous Read-Write Locks.
|
|
5
5
|
Author-email: TahsinCr <TahsinCrs@gmail.com>
|
|
6
6
|
License: MIT
|
|
7
7
|
Project-URL: Homepage, https://github.com/TahsinCr/python-rwlocker
|
|
8
8
|
Project-URL: Repository, https://github.com/TahsinCr/python-rwlocker
|
|
9
9
|
Project-URL: Bug Tracker, https://github.com/TahsinCr/python-rwlocker/issues
|
|
10
|
-
Project-URL: Changelog, https://github.com/TahsinCr/python-rwlocker/blob/
|
|
11
|
-
Keywords: rwlock,read-write-lock,thread,concurrency,lock,thread-safe,async,thundering-herd
|
|
10
|
+
Project-URL: Changelog, https://github.com/TahsinCr/python-rwlocker/blob/main/CHANGELOG.md
|
|
11
|
+
Keywords: rwlock,read-write-lock,thread,concurrency,lock,thread-safe,async,thundering-herd,rwcondition,condition
|
|
12
12
|
Classifier: Development Status :: 5 - Production/Stable
|
|
13
13
|
Classifier: Intended Audience :: Developers
|
|
14
14
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
@@ -75,6 +75,9 @@ Standard `Condition` structures in the library wake up all waiting threads/tasks
|
|
|
75
75
|
* **O(1) Condition Queuing (Stampede Protection):** Unlike standard libraries, it does not perform O(N) scanning on `notify_all()` calls. It wakes up hundreds of tasks instantly without choking the CPU.
|
|
76
76
|
* **Smart Signaling:** The ability to accurately wake up only the exact number of tasks you need, such as `notify(n=5)`, without creating a "Thundering Herd" in the system.
|
|
77
77
|
* **Flawless Cancellation Shielding:** If a task is cancelled from the outside (`CancelledError`) while waiting in an asynchronous `Condition.wait()`, the lock state is never corrupted. The lock is safely re-acquired and passed on to other waiters.
|
|
78
|
+
* **100% Drop-in Replacement:** You can inject your advanced locks (`RWLockFair`, etc.) directly into third-party libraries (SQLAlchemy, requests, FastAPI, etc.) expecting standard `threading.Lock` or `asyncio.Lock` instances without making any code changes. Standard API calls (e.g., `lock.acquire()`) are automatically and safely routed to the `.write` (exclusive) proxy.
|
|
79
|
+
* **"Happy Path" Performance Isolation:** A next-generation, OS-Interrupt-resilient micro-queue architecture that completely bypasses O(N) cost cleanup operations upon successful lock wake-ups, completing the process with zero CPU overhead.
|
|
80
|
+
* **Standard Adapters:** `Lock`, `Condition`, `AsyncLock`, and `AsyncCondition` standard wrapper classes for dependency injections where advanced RWLock features are not required but the same architectural signature (`.read`, `.write`) is desired.
|
|
78
81
|
|
|
79
82
|
### 🛡️ Lock Strategies
|
|
80
83
|
|
|
@@ -84,7 +87,7 @@ You can select the right lock strategy based on your system's bottleneck profile
|
|
|
84
87
|
| --- | --- | --- | --- |
|
|
85
88
|
| **Writer-Preferring** | `RWLockWrite` / `AsyncRWLockWrite` | Forbids new readers from entering if there is a waiting writer. Prevents writer starvation. | To prevent writers from being overwhelmed in read-heavy systems. |
|
|
86
89
|
| **Reader-Preferring** | `RWLockRead` / `AsyncRWLockRead` | Continuously allows new readers in, even if writers are waiting. Provides maximum parallelism. | In cache structures where write operations are very rare or non-critical. |
|
|
87
|
-
| **Fair
|
|
90
|
+
| **Fair** | `RWLockFair` / `AsyncRWLockFair` | Grants access alternately between readers and writers (interleaving). Prevents starvation for both sides. | In high-frequency, bidirectional traffic (MAVLink, WebSockets, etc.). |
|
|
88
91
|
> 💡 **Condition Compatibility:** The `RWCondition` and `AsyncRWCondition` classes in the library are designed to encapsulate all the lock strategies mentioned above (Dependency Injection). You can choose the lock that best fits your system and transform it into a state machine running at pure O(1) speed.
|
|
89
92
|
<br/>
|
|
90
93
|
|
|
@@ -99,9 +102,11 @@ Lock classes establish a circular reference graph (Lock -> Proxy -> Lock) when c
|
|
|
99
102
|
3. **Strict Nested Write Locks:**
|
|
100
103
|
In `ReentrantWriter` variants, only "Write" locks can be nested. If a writer wants to acquire a reader lock, it cannot do so implicitly; it must explicitly call the `.downgrade()` method. This is a strict architectural decision made to prevent deadlocks at the structural level.
|
|
101
104
|
4. **The Cost of Fairness:**
|
|
102
|
-
If you use the `
|
|
105
|
+
If you use the `Fair` strategy, the system forces a strict order-based context switch between readers and writers to guarantee that no one starves (No Starvation). Especially in **`RWCondition`** uses and write-heavy scenarios, this effort to maintain fair order causes a certain slowdown compared to the standard, rule-less C-based `Condition` object (this is why Fair scores 0.50x in benchmarks). This is not a bug or a lack of optimization; it is the engineering price paid to ensure "fairness".
|
|
103
106
|
5. **Condition Memory vs CPU Trade-off:**
|
|
104
107
|
While a standard `threading.Condition` keeps a simple C-level counter in the background, `rwlocker` stores a tiny `Lock` or `asyncio.Future` object in memory for each waiting task/thread to guarantee O(1) wake-up speed. This completely resolves CPU bottlenecks (Cache Stampede), but in extreme cases where tens of thousands of tasks are waiting, it creates a small memory footprint in RAM.
|
|
108
|
+
6. **Drop-in Security Assumption:**
|
|
109
|
+
If you use the lock objects directly like a standard lock without specifying the `.read` or `.write` proxies (e.g., `with lock:` or `await cond.wait()`), the system automatically acquires the **Write (Exclusive)** lock to ensure backward compatibility and absolute data security. This is a "Secure by Default" approach established to prevent external libraries from corrupting data.
|
|
105
110
|
|
|
106
111
|
<br/>
|
|
107
112
|
|
|
@@ -114,7 +119,7 @@ While a standard `threading.Condition` keeps a simple C-level counter in the bac
|
|
|
114
119
|
* **🚀 Read-Heavy Scenario (100 Readers, 2 Writers):**
|
|
115
120
|
While standard locks queue readers single-file and choke the system, `rwlocker` allows readers to access the data simultaneously. This achieves **~37x FASTER** speed and throughput in **Threading** and **~30x FASTER** in **Asyncio**.
|
|
116
121
|
* **⚖️ Balanced Scenario (50 Readers, 50 Writers):**
|
|
117
|
-
Thanks to the Fair
|
|
122
|
+
Thanks to the Fair state machine, read operations are squeezed in parallel between write queues. It increases performance by **2x** compared to standard locks without creating a system bottleneck.
|
|
118
123
|
* **🛡️ Write-Heavy Scenario (2 Readers, 100 Writers):**
|
|
119
124
|
Even though write operations inherently cannot be executed concurrently (in parallel), thanks to `rwlocker`'s zero-allocation smart proxy architecture, it runs **7-8% faster** than standard `C`-based locks. Even the O(1) cost "ReentrantWriter" (reentrancy) feature adds almost no overhead to performance.
|
|
120
125
|
|
|
@@ -127,9 +132,9 @@ When a single writer updates the database and wakes up hundreds of waiting reade
|
|
|
127
132
|
* **🔀 Balanced Pub/Sub (50 Writers, 50 Readers):**
|
|
128
133
|
In mixed waiting and waking scenarios, our Condition locks with the `Write-Pref` strategy ran **~2x FASTER** than the standard library.
|
|
129
134
|
* **📉 Write-Heavy Limit (Stress Test - 100 Writers, 2 Readers):**
|
|
130
|
-
In this brutal scenario where writers constantly block each other and call `notify()`, C-based standard locks utilize their raw speed advantage. `rwlocker`'s Write-Pref model holds its ground neck-and-neck (1.0x) with the standard lock, while the
|
|
135
|
+
In this brutal scenario where writers constantly block each other and call `notify()`, C-based standard locks utilize their raw speed advantage. `rwlocker`'s Write-Pref model holds its ground neck-and-neck (1.0x) with the standard lock, while the Fair and Read-Pref models intentionally slow down (0.5x - 0.7x) for the sake of maintaining fairness.
|
|
131
136
|
|
|
132
|
-
*(Note: All lock and condition classes have passed **
|
|
137
|
+
*(Note: All lock, adapter, and condition classes have passed a massive suite of **266 different unit tests** covering reentrancy, deadlock, timeout, OS interrupts, O(N) leaks, and cancellation safety scenarios with 0 errors, and this entire test suite was completed in just **8.5 seconds**.)*
|
|
133
138
|
|
|
134
139
|
|
|
135
140
|
<br/>
|
|
@@ -268,19 +273,19 @@ class AuthTokenManager:
|
|
|
268
273
|
|
|
269
274
|
```
|
|
270
275
|
|
|
271
|
-
#### 4. High-Frequency Telemetry (Fair
|
|
276
|
+
#### 4. High-Frequency Telemetry (Fair Distribution)
|
|
272
277
|
|
|
273
|
-
Data arrives from a sensor 100 times per second (Write), and 200 WebSockets read this data (Read). The
|
|
278
|
+
Data arrives from a sensor 100 times per second (Write), and 200 WebSockets read this data (Read). The Fair architecture prevents both sides from starving.
|
|
274
279
|
|
|
275
280
|
```python
|
|
276
281
|
import asyncio
|
|
277
282
|
from typing import Dict
|
|
278
|
-
from rwlocker.async_rwlock import
|
|
283
|
+
from rwlocker.async_rwlock import AsyncRWLockFair
|
|
279
284
|
|
|
280
285
|
class TelemetryDispatcher:
|
|
281
286
|
def __init__(self):
|
|
282
|
-
#
|
|
283
|
-
self._lock =
|
|
287
|
+
# Fair prevents read and write intensities from choking each other.
|
|
288
|
+
self._lock = AsyncRWLockFair()
|
|
284
289
|
self._state = {"alt": 0.0, "lat": 0.0, "lon": 0.0}
|
|
285
290
|
|
|
286
291
|
async def ingest_sensor_data(self, new_data: Dict[str, float]):
|
|
@@ -304,7 +309,7 @@ class TelemetryDispatcher:
|
|
|
304
309
|
|
|
305
310
|
```
|
|
306
311
|
|
|
307
|
-
#### 5. Event-Driven Cache Refresh (
|
|
312
|
+
#### 5. Event-Driven Cache Refresh (Thundering Herd Protection)
|
|
308
313
|
|
|
309
314
|
If thousands of tasks try to fetch an expired token from the database simultaneously, the DB crashes. With `AsyncRWCondition`, while 1 task updates the data, the other 999 tasks safely sleep without choking the CPU (at O(1) speed) and are awakened all at once afterward.
|
|
310
315
|
|
|
@@ -348,12 +353,12 @@ When 3 new jobs arrive in the system, instead of waking up all 50 idle worker th
|
|
|
348
353
|
```python
|
|
349
354
|
from collections import deque
|
|
350
355
|
import threading
|
|
351
|
-
from rwlocker.thread_rwlock import
|
|
356
|
+
from rwlocker.thread_rwlock import RWLockFair, RWCondition
|
|
352
357
|
|
|
353
358
|
class ImageProcessingQueue:
|
|
354
359
|
def __init__(self):
|
|
355
|
-
# Fair
|
|
356
|
-
self._cond = RWCondition(
|
|
360
|
+
# Fair strategy to prevent Producers and Consumers from crushing each other
|
|
361
|
+
self._cond = RWCondition(RWLockFair())
|
|
357
362
|
self._queue = deque()
|
|
358
363
|
|
|
359
364
|
def add_jobs(self, jobs: list[str]):
|
|
@@ -377,6 +382,34 @@ class ImageProcessingQueue:
|
|
|
377
382
|
|
|
378
383
|
```
|
|
379
384
|
|
|
385
|
+
#### 7. 100% Drop-in Replacement Compatibility
|
|
386
|
+
|
|
387
|
+
Inject the power of `rwlocker` into your system without changing your legacy code or third-party libraries that expect a standard `threading.Lock` or `asyncio.Lock`.
|
|
388
|
+
|
|
389
|
+
```python
|
|
390
|
+
import threading
|
|
391
|
+
from rwlocker.thread_rwlock import RWLockFair, Lock
|
|
392
|
+
|
|
393
|
+
# Scenario: A third-party function expects a standard threading.Lock
|
|
394
|
+
def third_party_worker(standard_lock: threading.Lock, data: list):
|
|
395
|
+
# The external library doesn't know about ".write" or ".read" proxies.
|
|
396
|
+
# It directly uses "with lock:".
|
|
397
|
+
with standard_lock:
|
|
398
|
+
data.append("Processed")
|
|
399
|
+
print("Lock acquired via standard API!")
|
|
400
|
+
|
|
401
|
+
# METHOD 1: You can pass an advanced RWLock object directly!
|
|
402
|
+
# RWLockFair detects these calls and automatically switches to the
|
|
403
|
+
# .write (exclusive) mode because it is the safest assumption.
|
|
404
|
+
advanced_lock = RWLockFair()
|
|
405
|
+
third_party_worker(advanced_lock, [])
|
|
406
|
+
|
|
407
|
+
# METHOD 2: If you only need standard lock behavior,
|
|
408
|
+
# you can use standard adapters that share the same signature.
|
|
409
|
+
simple_adapter_lock = Lock()
|
|
410
|
+
third_party_worker(simple_adapter_lock, [])
|
|
411
|
+
```
|
|
412
|
+
|
|
380
413
|
*For more examples, please check the [examples][examples-url] directory.*
|
|
381
414
|
|
|
382
415
|
See the [open issues][issues-url] for a full list of proposed features (and known issues).
|
|
@@ -438,9 +471,9 @@ Email: TahsinCrs@gmail.com
|
|
|
438
471
|
|
|
439
472
|
[examples-url]: https://github.com/TahsinCr/python-rwlocker/wiki
|
|
440
473
|
|
|
441
|
-
[license-url]: https://github.com/TahsinCr/python-rwlocker/blob/
|
|
474
|
+
[license-url]: https://github.com/TahsinCr/python-rwlocker/blob/main/LICENSE
|
|
442
475
|
|
|
443
|
-
[changelog-url]:https://github.com/TahsinCr/python-rwlocker/blob/
|
|
476
|
+
[changelog-url]:https://github.com/TahsinCr/python-rwlocker/blob/main/CHANGELOG.md
|
|
444
477
|
|
|
445
478
|
|
|
446
479
|
|
|
@@ -454,6 +487,6 @@ Email: TahsinCrs@gmail.com
|
|
|
454
487
|
|
|
455
488
|
<!-- File URL -->
|
|
456
489
|
|
|
457
|
-
[lang-tr-url]: https://github.com/TahsinCr/python-rwlocker/blob/
|
|
490
|
+
[lang-tr-url]: https://github.com/TahsinCr/python-rwlocker/blob/main/README_TR.md
|
|
458
491
|
|
|
459
|
-
[lang-en-url]: https://github.com/TahsinCr/python-rwlocker/blob/
|
|
492
|
+
[lang-en-url]: https://github.com/TahsinCr/python-rwlocker/blob/main/README.md
|
|
@@ -46,6 +46,9 @@ Standard `Condition` structures in the library wake up all waiting threads/tasks
|
|
|
46
46
|
* **O(1) Condition Queuing (Stampede Protection):** Unlike standard libraries, it does not perform O(N) scanning on `notify_all()` calls. It wakes up hundreds of tasks instantly without choking the CPU.
|
|
47
47
|
* **Smart Signaling:** The ability to accurately wake up only the exact number of tasks you need, such as `notify(n=5)`, without creating a "Thundering Herd" in the system.
|
|
48
48
|
* **Flawless Cancellation Shielding:** If a task is cancelled from the outside (`CancelledError`) while waiting in an asynchronous `Condition.wait()`, the lock state is never corrupted. The lock is safely re-acquired and passed on to other waiters.
|
|
49
|
+
* **100% Drop-in Replacement:** You can inject your advanced locks (`RWLockFair`, etc.) directly into third-party libraries (SQLAlchemy, requests, FastAPI, etc.) expecting standard `threading.Lock` or `asyncio.Lock` instances without making any code changes. Standard API calls (e.g., `lock.acquire()`) are automatically and safely routed to the `.write` (exclusive) proxy.
|
|
50
|
+
* **"Happy Path" Performance Isolation:** A next-generation, OS-Interrupt-resilient micro-queue architecture that completely bypasses O(N) cost cleanup operations upon successful lock wake-ups, completing the process with zero CPU overhead.
|
|
51
|
+
* **Standard Adapters:** `Lock`, `Condition`, `AsyncLock`, and `AsyncCondition` standard wrapper classes for dependency injections where advanced RWLock features are not required but the same architectural signature (`.read`, `.write`) is desired.
|
|
49
52
|
|
|
50
53
|
### 🛡️ Lock Strategies
|
|
51
54
|
|
|
@@ -55,7 +58,7 @@ You can select the right lock strategy based on your system's bottleneck profile
|
|
|
55
58
|
| --- | --- | --- | --- |
|
|
56
59
|
| **Writer-Preferring** | `RWLockWrite` / `AsyncRWLockWrite` | Forbids new readers from entering if there is a waiting writer. Prevents writer starvation. | To prevent writers from being overwhelmed in read-heavy systems. |
|
|
57
60
|
| **Reader-Preferring** | `RWLockRead` / `AsyncRWLockRead` | Continuously allows new readers in, even if writers are waiting. Provides maximum parallelism. | In cache structures where write operations are very rare or non-critical. |
|
|
58
|
-
| **Fair
|
|
61
|
+
| **Fair** | `RWLockFair` / `AsyncRWLockFair` | Grants access alternately between readers and writers (interleaving). Prevents starvation for both sides. | In high-frequency, bidirectional traffic (MAVLink, WebSockets, etc.). |
|
|
59
62
|
> 💡 **Condition Compatibility:** The `RWCondition` and `AsyncRWCondition` classes in the library are designed to encapsulate all the lock strategies mentioned above (Dependency Injection). You can choose the lock that best fits your system and transform it into a state machine running at pure O(1) speed.
|
|
60
63
|
<br/>
|
|
61
64
|
|
|
@@ -70,9 +73,11 @@ Lock classes establish a circular reference graph (Lock -> Proxy -> Lock) when c
|
|
|
70
73
|
3. **Strict Nested Write Locks:**
|
|
71
74
|
In `ReentrantWriter` variants, only "Write" locks can be nested. If a writer wants to acquire a reader lock, it cannot do so implicitly; it must explicitly call the `.downgrade()` method. This is a strict architectural decision made to prevent deadlocks at the structural level.
|
|
72
75
|
4. **The Cost of Fairness:**
|
|
73
|
-
If you use the `
|
|
76
|
+
If you use the `Fair` strategy, the system forces a strict order-based context switch between readers and writers to guarantee that no one starves (No Starvation). Especially in **`RWCondition`** uses and write-heavy scenarios, this effort to maintain fair order causes a certain slowdown compared to the standard, rule-less C-based `Condition` object (this is why Fair scores 0.50x in benchmarks). This is not a bug or a lack of optimization; it is the engineering price paid to ensure "fairness".
|
|
74
77
|
5. **Condition Memory vs CPU Trade-off:**
|
|
75
78
|
While a standard `threading.Condition` keeps a simple C-level counter in the background, `rwlocker` stores a tiny `Lock` or `asyncio.Future` object in memory for each waiting task/thread to guarantee O(1) wake-up speed. This completely resolves CPU bottlenecks (Cache Stampede), but in extreme cases where tens of thousands of tasks are waiting, it creates a small memory footprint in RAM.
|
|
79
|
+
6. **Drop-in Security Assumption:**
|
|
80
|
+
If you use the lock objects directly like a standard lock without specifying the `.read` or `.write` proxies (e.g., `with lock:` or `await cond.wait()`), the system automatically acquires the **Write (Exclusive)** lock to ensure backward compatibility and absolute data security. This is a "Secure by Default" approach established to prevent external libraries from corrupting data.
|
|
76
81
|
|
|
77
82
|
<br/>
|
|
78
83
|
|
|
@@ -85,7 +90,7 @@ While a standard `threading.Condition` keeps a simple C-level counter in the bac
|
|
|
85
90
|
* **🚀 Read-Heavy Scenario (100 Readers, 2 Writers):**
|
|
86
91
|
While standard locks queue readers single-file and choke the system, `rwlocker` allows readers to access the data simultaneously. This achieves **~37x FASTER** speed and throughput in **Threading** and **~30x FASTER** in **Asyncio**.
|
|
87
92
|
* **⚖️ Balanced Scenario (50 Readers, 50 Writers):**
|
|
88
|
-
Thanks to the Fair
|
|
93
|
+
Thanks to the Fair state machine, read operations are squeezed in parallel between write queues. It increases performance by **2x** compared to standard locks without creating a system bottleneck.
|
|
89
94
|
* **🛡️ Write-Heavy Scenario (2 Readers, 100 Writers):**
|
|
90
95
|
Even though write operations inherently cannot be executed concurrently (in parallel), thanks to `rwlocker`'s zero-allocation smart proxy architecture, it runs **7-8% faster** than standard `C`-based locks. Even the O(1) cost "ReentrantWriter" (reentrancy) feature adds almost no overhead to performance.
|
|
91
96
|
|
|
@@ -98,9 +103,9 @@ When a single writer updates the database and wakes up hundreds of waiting reade
|
|
|
98
103
|
* **🔀 Balanced Pub/Sub (50 Writers, 50 Readers):**
|
|
99
104
|
In mixed waiting and waking scenarios, our Condition locks with the `Write-Pref` strategy ran **~2x FASTER** than the standard library.
|
|
100
105
|
* **📉 Write-Heavy Limit (Stress Test - 100 Writers, 2 Readers):**
|
|
101
|
-
In this brutal scenario where writers constantly block each other and call `notify()`, C-based standard locks utilize their raw speed advantage. `rwlocker`'s Write-Pref model holds its ground neck-and-neck (1.0x) with the standard lock, while the
|
|
106
|
+
In this brutal scenario where writers constantly block each other and call `notify()`, C-based standard locks utilize their raw speed advantage. `rwlocker`'s Write-Pref model holds its ground neck-and-neck (1.0x) with the standard lock, while the Fair and Read-Pref models intentionally slow down (0.5x - 0.7x) for the sake of maintaining fairness.
|
|
102
107
|
|
|
103
|
-
*(Note: All lock and condition classes have passed **
|
|
108
|
+
*(Note: All lock, adapter, and condition classes have passed a massive suite of **266 different unit tests** covering reentrancy, deadlock, timeout, OS interrupts, O(N) leaks, and cancellation safety scenarios with 0 errors, and this entire test suite was completed in just **8.5 seconds**.)*
|
|
104
109
|
|
|
105
110
|
|
|
106
111
|
<br/>
|
|
@@ -239,19 +244,19 @@ class AuthTokenManager:
|
|
|
239
244
|
|
|
240
245
|
```
|
|
241
246
|
|
|
242
|
-
#### 4. High-Frequency Telemetry (Fair
|
|
247
|
+
#### 4. High-Frequency Telemetry (Fair Distribution)
|
|
243
248
|
|
|
244
|
-
Data arrives from a sensor 100 times per second (Write), and 200 WebSockets read this data (Read). The
|
|
249
|
+
Data arrives from a sensor 100 times per second (Write), and 200 WebSockets read this data (Read). The Fair architecture prevents both sides from starving.
|
|
245
250
|
|
|
246
251
|
```python
|
|
247
252
|
import asyncio
|
|
248
253
|
from typing import Dict
|
|
249
|
-
from rwlocker.async_rwlock import
|
|
254
|
+
from rwlocker.async_rwlock import AsyncRWLockFair
|
|
250
255
|
|
|
251
256
|
class TelemetryDispatcher:
|
|
252
257
|
def __init__(self):
|
|
253
|
-
#
|
|
254
|
-
self._lock =
|
|
258
|
+
# Fair prevents read and write intensities from choking each other.
|
|
259
|
+
self._lock = AsyncRWLockFair()
|
|
255
260
|
self._state = {"alt": 0.0, "lat": 0.0, "lon": 0.0}
|
|
256
261
|
|
|
257
262
|
async def ingest_sensor_data(self, new_data: Dict[str, float]):
|
|
@@ -275,7 +280,7 @@ class TelemetryDispatcher:
|
|
|
275
280
|
|
|
276
281
|
```
|
|
277
282
|
|
|
278
|
-
#### 5. Event-Driven Cache Refresh (
|
|
283
|
+
#### 5. Event-Driven Cache Refresh (Thundering Herd Protection)
|
|
279
284
|
|
|
280
285
|
If thousands of tasks try to fetch an expired token from the database simultaneously, the DB crashes. With `AsyncRWCondition`, while 1 task updates the data, the other 999 tasks safely sleep without choking the CPU (at O(1) speed) and are awakened all at once afterward.
|
|
281
286
|
|
|
@@ -319,12 +324,12 @@ When 3 new jobs arrive in the system, instead of waking up all 50 idle worker th
|
|
|
319
324
|
```python
|
|
320
325
|
from collections import deque
|
|
321
326
|
import threading
|
|
322
|
-
from rwlocker.thread_rwlock import
|
|
327
|
+
from rwlocker.thread_rwlock import RWLockFair, RWCondition
|
|
323
328
|
|
|
324
329
|
class ImageProcessingQueue:
|
|
325
330
|
def __init__(self):
|
|
326
|
-
# Fair
|
|
327
|
-
self._cond = RWCondition(
|
|
331
|
+
# Fair strategy to prevent Producers and Consumers from crushing each other
|
|
332
|
+
self._cond = RWCondition(RWLockFair())
|
|
328
333
|
self._queue = deque()
|
|
329
334
|
|
|
330
335
|
def add_jobs(self, jobs: list[str]):
|
|
@@ -348,6 +353,34 @@ class ImageProcessingQueue:
|
|
|
348
353
|
|
|
349
354
|
```
|
|
350
355
|
|
|
356
|
+
#### 7. 100% Drop-in Replacement Compatibility
|
|
357
|
+
|
|
358
|
+
Inject the power of `rwlocker` into your system without changing your legacy code or third-party libraries that expect a standard `threading.Lock` or `asyncio.Lock`.
|
|
359
|
+
|
|
360
|
+
```python
|
|
361
|
+
import threading
|
|
362
|
+
from rwlocker.thread_rwlock import RWLockFair, Lock
|
|
363
|
+
|
|
364
|
+
# Scenario: A third-party function expects a standard threading.Lock
|
|
365
|
+
def third_party_worker(standard_lock: threading.Lock, data: list):
|
|
366
|
+
# The external library doesn't know about ".write" or ".read" proxies.
|
|
367
|
+
# It directly uses "with lock:".
|
|
368
|
+
with standard_lock:
|
|
369
|
+
data.append("Processed")
|
|
370
|
+
print("Lock acquired via standard API!")
|
|
371
|
+
|
|
372
|
+
# METHOD 1: You can pass an advanced RWLock object directly!
|
|
373
|
+
# RWLockFair detects these calls and automatically switches to the
|
|
374
|
+
# .write (exclusive) mode because it is the safest assumption.
|
|
375
|
+
advanced_lock = RWLockFair()
|
|
376
|
+
third_party_worker(advanced_lock, [])
|
|
377
|
+
|
|
378
|
+
# METHOD 2: If you only need standard lock behavior,
|
|
379
|
+
# you can use standard adapters that share the same signature.
|
|
380
|
+
simple_adapter_lock = Lock()
|
|
381
|
+
third_party_worker(simple_adapter_lock, [])
|
|
382
|
+
```
|
|
383
|
+
|
|
351
384
|
*For more examples, please check the [examples][examples-url] directory.*
|
|
352
385
|
|
|
353
386
|
See the [open issues][issues-url] for a full list of proposed features (and known issues).
|
|
@@ -409,9 +442,9 @@ Email: TahsinCrs@gmail.com
|
|
|
409
442
|
|
|
410
443
|
[examples-url]: https://github.com/TahsinCr/python-rwlocker/wiki
|
|
411
444
|
|
|
412
|
-
[license-url]: https://github.com/TahsinCr/python-rwlocker/blob/
|
|
445
|
+
[license-url]: https://github.com/TahsinCr/python-rwlocker/blob/main/LICENSE
|
|
413
446
|
|
|
414
|
-
[changelog-url]:https://github.com/TahsinCr/python-rwlocker/blob/
|
|
447
|
+
[changelog-url]:https://github.com/TahsinCr/python-rwlocker/blob/main/CHANGELOG.md
|
|
415
448
|
|
|
416
449
|
|
|
417
450
|
|
|
@@ -425,6 +458,6 @@ Email: TahsinCrs@gmail.com
|
|
|
425
458
|
|
|
426
459
|
<!-- File URL -->
|
|
427
460
|
|
|
428
|
-
[lang-tr-url]: https://github.com/TahsinCr/python-rwlocker/blob/
|
|
461
|
+
[lang-tr-url]: https://github.com/TahsinCr/python-rwlocker/blob/main/README_TR.md
|
|
429
462
|
|
|
430
|
-
[lang-en-url]: https://github.com/TahsinCr/python-rwlocker/blob/
|
|
463
|
+
[lang-en-url]: https://github.com/TahsinCr/python-rwlocker/blob/main/README.md
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "rwlocker"
|
|
7
|
-
version = "
|
|
7
|
+
version = "3.0"
|
|
8
8
|
description = "Advanced, High-Performance, and State-Machine Based Synchronous/Asynchronous Read-Write Locks."
|
|
9
9
|
readme = {file = "README-pypi.md", content-type = "text/markdown"}
|
|
10
10
|
license = {text = "MIT"}
|
|
@@ -21,7 +21,9 @@ keywords = [
|
|
|
21
21
|
"lock",
|
|
22
22
|
"thread-safe",
|
|
23
23
|
"async",
|
|
24
|
-
"thundering-herd"
|
|
24
|
+
"thundering-herd",
|
|
25
|
+
"rwcondition",
|
|
26
|
+
"condition"
|
|
25
27
|
]
|
|
26
28
|
|
|
27
29
|
classifiers = [
|
|
@@ -46,7 +48,7 @@ dependencies = []
|
|
|
46
48
|
Homepage = "https://github.com/TahsinCr/python-rwlocker"
|
|
47
49
|
Repository = "https://github.com/TahsinCr/python-rwlocker"
|
|
48
50
|
"Bug Tracker" = "https://github.com/TahsinCr/python-rwlocker/issues"
|
|
49
|
-
"Changelog" = "https://github.com/TahsinCr/python-rwlocker/blob/
|
|
51
|
+
"Changelog" = "https://github.com/TahsinCr/python-rwlocker/blob/main/CHANGELOG.md"
|
|
50
52
|
|
|
51
53
|
[tool.setuptools.packages.find]
|
|
52
54
|
include = ["rwlocker*"]
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
"""
|
|
2
|
+
Advanced Read-Write Lock (RWLock) and Condition Concurrency Primitives.
|
|
3
|
+
|
|
4
|
+
This package provides a comprehensive, highly optimized, state-machine-based
|
|
5
|
+
suite of Read-Write locks and Condition variables for both Synchronous
|
|
6
|
+
(`threading`) and Asynchronous (`asyncio`) Python applications.
|
|
7
|
+
|
|
8
|
+
Designed for high-performance systems (e.g., telemetry processing, data streams),
|
|
9
|
+
it guarantees strict data safety while maximizing read concurrency and
|
|
10
|
+
preventing CPU/Event-Loop bottlenecks.
|
|
11
|
+
|
|
12
|
+
Key Architectural Features:
|
|
13
|
+
- **Multiple Scheduling Strategies**: Choose between Write-preferring,
|
|
14
|
+
Read-preferring, and Fair (FIFO) algorithms to prevent starvation based
|
|
15
|
+
on your specific workload.
|
|
16
|
+
- **O(1) Condition Queuing (Stampede Protection)**: Condition variables
|
|
17
|
+
(`RWCondition`, `AsyncRWCondition`) utilize pure O(1) waiter queues
|
|
18
|
+
to completely eliminate O(N) cache stampedes and event-loop blocking
|
|
19
|
+
during massive `notify_all()` calls.
|
|
20
|
+
- **Smart Proxies**: Locks and conditions are interacted with via `.read`
|
|
21
|
+
and `.write` attributes. These proxies intelligently route `release()`
|
|
22
|
+
operations, even after complex state transitions, preventing deadlocks.
|
|
23
|
+
- **Atomic Downgrading**: Transition from a Write lock to a Read lock seamlessly.
|
|
24
|
+
The `.downgrade()` operation ensures no other writer can hijack the lock
|
|
25
|
+
during the transition.
|
|
26
|
+
- **Adapter Pattern & Solid Base**: Standard locks and conditions are
|
|
27
|
+
encapsulated via `Lock`, `Condition`, `AsyncLock`, and `AsyncCondition`
|
|
28
|
+
adapters, sharing the exact same API signatures for seamless dependency injection.
|
|
29
|
+
- **Zero-Allocation Fast-Paths**: Standard synchronous lock acquisition
|
|
30
|
+
and release are optimized to avoid runtime object creation.
|
|
31
|
+
- **Flawless Cancellation Shielding (Async)**: Asynchronous locks and
|
|
32
|
+
condition `wait()` operations are strictly resilient to `asyncio.CancelledError`,
|
|
33
|
+
ensuring safe state recovery during task aborts.
|
|
34
|
+
|
|
35
|
+
Important Usage Notes & Gotchas:
|
|
36
|
+
- **Reentrancy (`ReentrantWriter` variants)**: Reentrancy is STRICTLY supported for
|
|
37
|
+
nested *write* operations by the same Thread/Task. It does NOT implicitly
|
|
38
|
+
grant read locks. You must use `.downgrade()` if you need to read.
|
|
39
|
+
- **Downgrade Performance Cost**: Calling `.downgrade()` registers the current
|
|
40
|
+
Thread/Task ID into a tracking set. This adds a minor O(1) hash lookup cost
|
|
41
|
+
during the subsequent `release()` operation. Standard operations remain O(0).
|
|
42
|
+
- **Circular References**: Base lock classes hold references to their proxies,
|
|
43
|
+
and proxies hold references back to the base. Memory is reclaimed via
|
|
44
|
+
Python's cyclic GC. Do not rely on `__del__` for cleanup.
|
|
45
|
+
|
|
46
|
+
Basic Example:
|
|
47
|
+
>>> lock = RWLockFIFOReentrantWriter()
|
|
48
|
+
>>> with lock.write:
|
|
49
|
+
... # Exclusive write access
|
|
50
|
+
... lock.write.downgrade()
|
|
51
|
+
... # Atomically downgraded to shared read access
|
|
52
|
+
|
|
53
|
+
>>> cond = RWCondition(RWLockWrite())
|
|
54
|
+
>>> with cond.read:
|
|
55
|
+
... # Sleep at O(1) cost without blocking other readers
|
|
56
|
+
... cond.read.wait_for(lambda: True)
|
|
57
|
+
|
|
58
|
+
>>> async_cond = AsyncRWCondition(AsyncRWLockRead())
|
|
59
|
+
>>> async with async_cond.write:
|
|
60
|
+
... # Wake up thousands of tasks instantly without event-loop lag
|
|
61
|
+
... async_cond.write.notify_all()
|
|
62
|
+
"""
|
|
63
|
+
|
|
64
|
+
from .thread_rwlock import *
|
|
65
|
+
from .async_rwlock import *
|
|
66
|
+
|
|
67
|
+
__version__ = '3.0'
|