py-sharedmemory 3.0.0__tar.gz → 3.0.1__tar.gz

{py_sharedmemory-3.0.0 → py_sharedmemory-3.0.1}/CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## 3.0.1
+
+- Fix resource-tracker leaks on Linux and macOS: receivers no longer register blocks they
+  attach to, so `fork` workers no longer accumulate tracker entries or print
+  "leaked shared_memory" warnings at exit.
+- Ship type hints (`py.typed`), checked with strict mypy; expose `memory.__version__`.
+- Support and test Python 3.15; test Linux, macOS, and Windows for every Python version.
+- Require 100% branch coverage; add pre-commit hooks, Dependabot, and a trusted-publishing
+  release workflow.
+
 ## 3.0.0
 
 - Python 3.11+, standard-library-only runtime.

{py_sharedmemory-3.0.0 → py_sharedmemory-3.0.1}/LICENSE

@@ -1,21 +1,21 @@
-MIT License
-
-Copyright (c) 2026 Tobias Würth
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+MIT License
+
+Copyright (c) 2026 Tobias Würth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

py_sharedmemory-3.0.1/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.4
+Name: py-sharedmemory
+Version: 3.0.1
+Summary: Shared-memory queues for large Python objects with less pipe I/O
+Author: Tobias Würth
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/tobiaswuerth/python_shared_memory_queue
+Project-URL: Repository, https://github.com/tobiaswuerth/python_shared_memory_queue
+Project-URL: Changelog, https://github.com/tobiaswuerth/python_shared_memory_queue/blob/main/CHANGELOG.md
+Project-URL: Bug Tracker, https://github.com/tobiaswuerth/python_shared_memory_queue/issues
+Keywords: queue,multiprocessing,shared-memory,inter-process-communication,ipc,numpy,zero-copy,pickle
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Distributed Computing
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: test
+Requires-Dist: pytest>=7.4; extra == "test"
+Requires-Dist: pytest-cov>=4.1; extra == "test"
+Requires-Dist: pytest-timeout>=2.3; extra == "test"
+Requires-Dist: numpy>=1.26; extra == "test"
+Provides-Extra: benchmark
+Requires-Dist: numpy>=1.26; extra == "benchmark"
+Requires-Dist: matplotlib>=3.8; extra == "benchmark"
+Provides-Extra: dev
+Requires-Dist: py-sharedmemory[benchmark,test]; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: mypy>=1.10; extra == "dev"
+Requires-Dist: pre-commit>=3.7; extra == "dev"
+Requires-Dist: ruff>=0.15; extra == "dev"
+Requires-Dist: twine>=5; extra == "dev"
+Dynamic: license-file
+
+# py-sharedmemory
+
+[![CI](https://github.com/tobiaswuerth/python_shared_memory_queue/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/tobiaswuerth/python_shared_memory_queue/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/py-sharedmemory)](https://pypi.org/project/py-sharedmemory/)
+[![Python](https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftobiaswuerth%2Fpython_shared_memory_queue%2Fmain%2Fpyproject.toml&query=%24.project%5B%27requires-python%27%5D&label=python&logo=python&logoColor=white)](https://pypi.org/project/py-sharedmemory/)
+[![Downloads](https://img.shields.io/pypi/dm/py-sharedmemory)](https://pypistats.org/packages/py-sharedmemory)
+[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen)](https://github.com/tobiaswuerth/python_shared_memory_queue/actions/workflows/ci.yml)
+[![License](https://img.shields.io/github/license/tobiaswuerth/python_shared_memory_queue)](LICENSE)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)
+[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit)
+
+Send large Python objects between processes with less pipe I/O.
+
+Originally built to reduce the heavy Windows pipe I/O and uneven throughput caused by
+large `multiprocessing.Queue` transfers. Payloads travel through shared memory;
+queues carry only small metadata, avoiding bulk transfers through the pipe.
+
+Useful for **ML pipelines, reinforcement-learning actors and replay buffers,
+scientific computing, and image/video processing**. Supports pickleable Python
+objects with familiar `put/get` calls. No runtime dependencies, fully typed.
+Python 3.11+ on Linux, macOS, and Windows.
+
+```sh
+pip install py-sharedmemory
+```
+
+## Measured performance
+
+![Transfer speed relative to multiprocessing.Queue, from 16 bytes to 10 GB](benchmarks/results/windows-numpy.png)
+
+About **2× faster at 1–10 MB**; **10 GB transferred in 8.0 s**.
+`mp.Queue` failed at 10 GB (Windows error 87). Small messages favor standard queues.
+
+Core i9-14900K, 191.8 GiB RAM, Windows 11, Python 3.12.10, NumPy 2.5.3.
+2026-10-08; contiguous float32 arrays; `spawn`; capacity 2; median of 5 runs,
+excluding startup and warmup. Up to 1,000 messages or 512 MiB/run, at least one
+message. GB = 10⁹ bytes. [Raw results](benchmarks/results/windows-numpy.json).
+These measurements show transfer speed; I/O and stability depend on the workload.
+
+## Usage
+
+```python
+import multiprocessing as mp
+from memory import create_shared_memory_pair
+
+
+def produce(sender):
+    with sender:
+        sender.put({"step": 42, "frames": bytearray(16_000_000)})
+        sender.wait_for_all_ack()
+
+
+if __name__ == "__main__":
+    ctx = mp.get_context("spawn")
+    sender, receiver = create_shared_memory_pair(capacity=2, ctx=ctx)
+    worker = ctx.Process(target=produce, args=(sender,))
+    worker.start()
+    with receiver:
+        batch = receiver.get(timeout=30)
+    worker.join()
+    sender.close()
+```
+
+One producing process per pair; multiple threads and consumers are supported.
+Pass endpoints to workers before use. `put/get` support `block`, `timeout`, and
+`*_nowait`. Capacity counts outstanding messages; 0/`None` is unbounded.
+
+**Drain before producer close or exit:** `wait_for_all_ack(timeout=...)` returns
+`False` on timeout. Close discards unread data. Within one process, drain before
+closing either endpoint. Received data stays valid after cleanup; arrays preserve mutability.
+
+A [complete example](examples/basic.py) includes error handling.
+
+### API
+
+| Call | Behavior |
+| --- | --- |
+| `create_shared_memory_pair(capacity=0, *, ctx=None)` | Returns `(sender, receiver)` built from one multiprocessing context. |
+| `sender.put(obj, block=True, timeout=None)` / `put_nowait(obj)` | Copies `obj` into a new shared-memory block; raises `queue.Full`. |
+| `sender.wait_for_all_ack(timeout=None)` | Waits until every block is received and freed; `False` on timeout. |
+| `receiver.get(block=True, timeout=None)` / `get_nowait()` | Returns an independent local copy; raises `queue.Empty`. |
+| `close()`, `closed`, `with endpoint:` | Both endpoints; using a closed endpoint raises `BrokenPipeError`. |
+
+## Development
+
+```sh
+python -m pip install -e ".[dev]"
+pre-commit install
+python -m pytest --cov
+pre-commit run --all-files
+```
+
+CI runs the tests on Linux, macOS, and Windows with Python 3.11–3.15 and requires
+100% branch coverage on each. Pre-commit runs Ruff, mypy (strict), and file checks.
+
+To release, update `__version__` in `memory/__init__.py` and the changelog, then
+publish a GitHub release tagged `vX.Y.Z`. The release workflow uploads it to PyPI.
+
+### Benchmark
+
+```sh
+python -m benchmarks.benchmark --sizes 16 100 1000 10000 100000 1000000 10000000 100000000 1000000000 10000000000 --repeats 5 --json benchmarks/results/windows-numpy.json
+python -m benchmarks.plot benchmarks/results/windows-numpy.json benchmarks/results/windows-numpy.png
+```
+
+The full benchmark needs enough RAM for several payload copies.

py_sharedmemory-3.0.0/py_sharedmemory.egg-info/PKG-INFO → py_sharedmemory-3.0.1/README.md

@@ -1,98 +1,107 @@
-Metadata-Version: 2.4
-Name: py-sharedmemory
-Version: 3.0.0
-Summary: Shared-memory queues for large Python objects with less pipe I/O
-Author: Tobias Würth
-License-Expression: MIT
-Project-URL: Repository, https://github.com/tobiaswuerth/python_shared_memory_queue
-Project-URL: Bug Tracker, https://github.com/tobiaswuerth/python_shared_memory_queue/issues
-Keywords: queue,multiprocessing,numpy,inter-process-communication,shared-memory
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3 :: Only
-Classifier: Operating System :: OS Independent
-Requires-Python: >=3.11
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Provides-Extra: test
-Requires-Dist: pytest>=7.4; extra == "test"
-Requires-Dist: pytest-cov>=4.1; extra == "test"
-Requires-Dist: pytest-timeout>=2.3; extra == "test"
-Requires-Dist: numpy>=1.26; extra == "test"
-Provides-Extra: benchmark
-Requires-Dist: numpy>=1.26; extra == "benchmark"
-Requires-Dist: matplotlib>=3.8; extra == "benchmark"
-Dynamic: license-file
-
-# py-sharedmemory
-
-Send large Python objects between processes with less pipe I/O.
-
-Originally built to reduce the heavy Windows pipe I/O and uneven throughput caused by
-large `multiprocessing.Queue` transfers. Payloads travel through shared memory;
-queues carry only small metadata, avoiding bulk transfers through the pipe.
-
-Useful for **ML pipelines, reinforcement-learning actors and replay buffers,
-scientific computing, and image/video processing**. Supports pickleable Python
-objects with familiar `put/get` calls. No runtime dependencies. Python 3.11+.
-
-```sh
-pip install py-sharedmemory
-```
-
-## Measured performance
-
-![Transfer speed relative to multiprocessing.Queue, from 16 bytes to 10 GB](benchmarks/results/windows-numpy.png)
-
-About **2× faster at 1–10 MB**; **10 GB transferred in 8.0 s**.
-`mp.Queue` failed at 10 GB (Windows error 87). Small messages favor standard queues.
-
-Core i9-14900K, 191.8 GiB RAM, Windows 11, Python 3.12.10, NumPy 2.5.3.
-2026-10-08; contiguous float32 arrays; `spawn`; capacity 2; median of 5 runs,
-excluding startup and warmup. Up to 1,000 messages or 512 MiB/run, at least one
-message. GB = 10⁹ bytes. [Raw results](benchmarks/results/windows-numpy.json).
-These measurements show transfer speed; I/O and stability depend on the workload.
-
-## Usage
-
-```python
-import multiprocessing as mp
-from memory import create_shared_memory_pair
-
-
-def produce(sender):
-    with sender:
-        sender.put({"step": 42, "frames": bytearray(16_000_000)})
-        sender.wait_for_all_ack()
-
-
-if __name__ == "__main__":
-    ctx = mp.get_context("spawn")
-    sender, receiver = create_shared_memory_pair(capacity=2, ctx=ctx)
-    worker = ctx.Process(target=produce, args=(sender,))
-    worker.start()
-    with receiver:
-        batch = receiver.get(timeout=30)
-    worker.join()
-    sender.close()
-```
-
-One producing process per pair; multiple threads and consumers are supported.
-Pass endpoints to workers before use. `put/get` support `block`, `timeout`, and
-`*_nowait`. Capacity counts outstanding messages; 0/`None` is unbounded.
-
-**Drain before producer close or exit:** `wait_for_all_ack(timeout=...)` returns
-`False` on timeout. Close discards unread data. Within one process, drain before
-closing either endpoint. Received data stays valid after cleanup; arrays preserve mutability.
-
-A [complete example](examples/basic.py) includes error handling.
-
-## Test and benchmark
-
-```sh
-python -m pip install -e ".[test,benchmark]"
-python -m pytest --cov=memory
-python -m benchmarks.benchmark --sizes 16 100 1000 10000 100000 1000000 10000000 100000000 1000000000 10000000000 --repeats 5 --json benchmarks/results/windows-numpy.json
-python -m benchmarks.plot benchmarks/results/windows-numpy.json benchmarks/results/windows-numpy.png
-```
-
-The full benchmark needs enough RAM for several payload copies.
+# py-sharedmemory
+
+[![CI](https://github.com/tobiaswuerth/python_shared_memory_queue/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/tobiaswuerth/python_shared_memory_queue/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/py-sharedmemory)](https://pypi.org/project/py-sharedmemory/)
+[![Python](https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftobiaswuerth%2Fpython_shared_memory_queue%2Fmain%2Fpyproject.toml&query=%24.project%5B%27requires-python%27%5D&label=python&logo=python&logoColor=white)](https://pypi.org/project/py-sharedmemory/)
+[![Downloads](https://img.shields.io/pypi/dm/py-sharedmemory)](https://pypistats.org/packages/py-sharedmemory)
+[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen)](https://github.com/tobiaswuerth/python_shared_memory_queue/actions/workflows/ci.yml)
+[![License](https://img.shields.io/github/license/tobiaswuerth/python_shared_memory_queue)](LICENSE)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)
+[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit)
+
+Send large Python objects between processes with less pipe I/O.
+
+Originally built to reduce the heavy Windows pipe I/O and uneven throughput caused by
+large `multiprocessing.Queue` transfers. Payloads travel through shared memory;
+queues carry only small metadata, avoiding bulk transfers through the pipe.
+
+Useful for **ML pipelines, reinforcement-learning actors and replay buffers,
+scientific computing, and image/video processing**. Supports pickleable Python
+objects with familiar `put/get` calls. No runtime dependencies, fully typed.
+Python 3.11+ on Linux, macOS, and Windows.
+
+```sh
+pip install py-sharedmemory
+```
+
+## Measured performance
+
+![Transfer speed relative to multiprocessing.Queue, from 16 bytes to 10 GB](benchmarks/results/windows-numpy.png)
+
+About **2× faster at 1–10 MB**; **10 GB transferred in 8.0 s**.
+`mp.Queue` failed at 10 GB (Windows error 87). Small messages favor standard queues.
+
+Core i9-14900K, 191.8 GiB RAM, Windows 11, Python 3.12.10, NumPy 2.5.3.
+2026-10-08; contiguous float32 arrays; `spawn`; capacity 2; median of 5 runs,
+excluding startup and warmup. Up to 1,000 messages or 512 MiB/run, at least one
+message. GB = 10⁹ bytes. [Raw results](benchmarks/results/windows-numpy.json).
+These measurements show transfer speed; I/O and stability depend on the workload.
+
+## Usage
+
+```python
+import multiprocessing as mp
+from memory import create_shared_memory_pair
+
+
+def produce(sender):
+    with sender:
+        sender.put({"step": 42, "frames": bytearray(16_000_000)})
+        sender.wait_for_all_ack()
+
+
+if __name__ == "__main__":
+    ctx = mp.get_context("spawn")
+    sender, receiver = create_shared_memory_pair(capacity=2, ctx=ctx)
+    worker = ctx.Process(target=produce, args=(sender,))
+    worker.start()
+    with receiver:
+        batch = receiver.get(timeout=30)
+    worker.join()
+    sender.close()
+```
+
+One producing process per pair; multiple threads and consumers are supported.
+Pass endpoints to workers before use. `put/get` support `block`, `timeout`, and
+`*_nowait`. Capacity counts outstanding messages; 0/`None` is unbounded.
+
+**Drain before producer close or exit:** `wait_for_all_ack(timeout=...)` returns
+`False` on timeout. Close discards unread data. Within one process, drain before
+closing either endpoint. Received data stays valid after cleanup; arrays preserve mutability.
+
+A [complete example](examples/basic.py) includes error handling.
+
+### API
+
+| Call | Behavior |
+| --- | --- |
+| `create_shared_memory_pair(capacity=0, *, ctx=None)` | Returns `(sender, receiver)` built from one multiprocessing context. |
+| `sender.put(obj, block=True, timeout=None)` / `put_nowait(obj)` | Copies `obj` into a new shared-memory block; raises `queue.Full`. |
+| `sender.wait_for_all_ack(timeout=None)` | Waits until every block is received and freed; `False` on timeout. |
+| `receiver.get(block=True, timeout=None)` / `get_nowait()` | Returns an independent local copy; raises `queue.Empty`. |
+| `close()`, `closed`, `with endpoint:` | Both endpoints; using a closed endpoint raises `BrokenPipeError`. |
+
+## Development
+
+```sh
+python -m pip install -e ".[dev]"
+pre-commit install
+python -m pytest --cov
+pre-commit run --all-files
+```
+
+CI runs the tests on Linux, macOS, and Windows with Python 3.11–3.15 and requires
+100% branch coverage on each. Pre-commit runs Ruff, mypy (strict), and file checks.
+
+To release, update `__version__` in `memory/__init__.py` and the changelog, then
+publish a GitHub release tagged `vX.Y.Z`. The release workflow uploads it to PyPI.
+
+### Benchmark
+
+```sh
+python -m benchmarks.benchmark --sizes 16 100 1000 10000 100000 1000000 10000000 100000000 1000000000 10000000000 --repeats 5 --json benchmarks/results/windows-numpy.json
+python -m benchmarks.plot benchmarks/results/windows-numpy.json benchmarks/results/windows-numpy.png
+```
+
+The full benchmark needs enough RAM for several payload copies.

{py_sharedmemory-3.0.0 → py_sharedmemory-3.0.1}/benchmarks/benchmark.py

@@ -129,10 +129,7 @@ def main(argv=None):
     print(f"{'Bytes':>12}  {'Transport':>9}  {'MiB/s':>10}  {'ms/message':>12}")
     for size in args.sizes:
         count = args.messages or min(1_000, max(1, 2**29 // size))
-        if args.payload == "numpy":
-            payload = np.ones(size // 4, dtype=np.float32)
-        else:
-            payload = b"x" * size
+        payload = np.ones(size // 4, dtype=np.float32) if args.payload == "numpy" else b"x" * size
         for transport in ("standard", "shared"):
             row = {"transport": transport, "size": size, "messages": count}
             durations = []

{py_sharedmemory-3.0.0 → py_sharedmemory-3.0.1}/benchmarks/plot.py

@@ -7,7 +7,7 @@ from pathlib import Path
 import matplotlib
 
 matplotlib.use("Agg")
-import matplotlib.pyplot as plt  # noqa: E402
+import matplotlib.pyplot as plt
 
 
 def plot(report, output):
@@ -68,11 +68,11 @@ def _size_label(size):
     return f"{size} B"
 
 
-def main():
+def main(argv=None):
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("results", type=Path)
     parser.add_argument("output", type=Path)
-    args = parser.parse_args()
+    args = parser.parse_args(argv)
     plot(json.loads(args.results.read_text(encoding="utf-8")), args.output)