slick-queue-py 2.3.0__tar.gz → 2.4.0__tar.gz

{slick_queue_py-2.3.0 → slick_queue_py-2.4.0}/API_DIFFERENCES.md

@@ -1,8 +1,8 @@
 # C++ / Python API Differences
 
-This document explains the intentional API differences between the C++ `slick::queue<T>` and Python `SlickQueue` implementations.
+This document explains the intentional API differences between the C++ `slick::queue<T>` and Python `SlickQueue` implementations, and between C++ `slick::spsc_queue<T>` and Python `SpscQueue` (section 8).
 
-Both track the same version line; this document describes slick-queue v2.3.0.
+Both track the same version line; this document describes slick-queue v2.4.0.
 
 ## Core Compatibility
 
@@ -267,6 +267,39 @@ the process exits.
 
 ---
 
+### 8. SpscQueue: Held Slot vs. Copy
+
+**C++:**
+```cpp
+slick::spsc_queue<uint64_t> q("ticks");
+if (uint64_t* v = q.read()) { /* slot held until the next read(), read_last() or release() */ }
+uint64_t dropped = 0;
+uint64_t* last = q.read_last(dropped);
+```
+
+**Python:**
+```python
+q = SpscQueue(name='ticks', element_size=8)
+data = q.read()                 # bytes copy; the slot is already back with the producer
+data, dropped = q.read_last()
+```
+
+**Rationale:** Python copies the element anyway, so once `read()` returns there is nothing
+left to hold, and it hands the slot back at once. A Python consumer therefore gets the full
+`size` as capacity, even at size 1, and has no `release()`. A C++ consumer that exits while
+holding a slot still has that element delivered again to whoever attaches next, Python
+included.
+
+| C++ | Python |
+|-----|--------|
+| `try_reserve()` returns `invalid_index` | Same, as `SpscQueue.invalid_index` |
+| `read()` returns `T*`, `nullptr` when empty | `bytes`, `None` when empty |
+| `read_last(dropped)` with an out-parameter | `read_last()` returns `(data, dropped)` |
+| `release()` | Not needed: nothing is held |
+| `spsc_queue(name)` adopts `size`, and `T` gives the element size | `SpscQueue(name=..., element_size=...)` adopts `size`; `element_size` is always required |
+
+---
+
 ## Interoperability Examples
 
 ### Example 1: C++ Producer → Python Consumer
@@ -363,5 +396,7 @@ int main() {
 | Misconfigured traits | Compile error (concept / `static_assert`) | `TypeError` / `RuntimeError` at runtime | No compile step |
 | Default loss detection | Off in Release | Always on | +0.2% here, and the only overrun signal a caller has |
 | Layout marker | `'SLQ1'` / `'SLQ0'` | Identical, and validated against C++ peers | Shared contract |
+| SPSC `read()` | `T*` held until the next read | `bytes` copy, slot freed at once | Python copies anyway |
+| SPSC `read_last()` dropped count | Out-parameter | Second item of the returned tuple | Python has no pass-by-reference |
 
 **Bottom Line:** The Python API is designed to be Pythonic while maintaining **100% binary compatibility** with C++. All atomic operations and memory layouts are identical, enabling seamless C++/Python interoperability.

{slick_queue_py-2.3.0 → slick_queue_py-2.4.0}/CHANGELOG.md

@@ -1,5 +1,59 @@
 # Changelogs
 
+## [v2.4.0] - 2026-11-09
+
+Tracks C++ slick-queue v2.4.0, which added `slick::spsc_queue`. The `SlickQueue`
+shared-memory layout is unchanged, so 2.3.0 peers in either language interoperate
+with it as before.
+
+### Added
+
+- `SpscQueue`, a port of C++ `slick::spsc_queue<T>`: a lossless single-producer
+  single-consumer ring in local or shared memory. `try_reserve(n)` returns
+  `SpscQueue.invalid_index` when the ring is full and `reserve(n)` waits until the
+  consumer frees room. `publish()` never moves the tail backwards, so publishing an
+  earlier reservation after a later one is a no-op.
+- `read()` returns the next element as `bytes`. `read_last()` returns
+  `(data, dropped)`, the newest element and how many unread ones it skipped, from
+  one load of the producer's tail however large the backlog.
+- Both reads copy the element and hand its slot back before they return. C++ holds
+  the slot until the consumer's next call, so a Python consumer gets the full `size`
+  as capacity, nothing it read is delivered again after it reattaches, and there is
+  no `release()`.
+- `SpscQueueTraits` with one setting, `enable_cpu_relax`, which yields the GIL while
+  `reserve()` waits. `q.traits` is a read-only snapshot, as for `SlickQueue`.
+- The C++ shared-memory layout: a 384-byte header with the tail at offset 128 and the
+  head at 256, then the data array. Size, element size, the `'SLS0'` marker and init
+  state keep the `SlickQueue` offsets. Both cursors live in the segment, so a
+  reattached peer resumes where the previous one stopped.
+- A creator that claims a segment left uninitialized, whose own creator died before
+  claiming it, checks the mapping holds its geometry first. If it doesn't, it raises
+  and hands the claim back, instead of writing a header for a ring the mapping can't
+  hold. Every attacher checks the mapping size too.
+- Using an `SpscQueue` after `close()` raises `RuntimeError("queue is closed")`.
+
+### Changed
+
+- `SlickQueue` attaching to an `SpscQueue` segment now raises "holds a
+  slick::spsc_queue" instead of the generic missing-marker error, matching C++, and
+  `SpscQueue` names a `SlickQueue` segment the same way.
+- Name handling, opening and closing the segment, `unlink()`, `remove_shm()`,
+  `remove()` and the context manager moved into a base class both queues share.
+  `SlickQueue` behaves as before.
+- `close()` still never raises, but no longer prints the exception it swallows.
+
+### Tests
+
+- `tests/test_spsc.py` covers the local and shared-memory API: wraps, a full ring,
+  out-of-order publish, element sizes other than 8, `read_last()` accounting under a
+  concurrent producer, reset, reattach, a too-small leftover segment, use after
+  `close()` and the cross-type marker checks.
+- `tests/test_spsc_interop.py` runs against C++ through the new `cpp_spsc_tester`:
+  data in both directions, attaching to a C++-created segment, `read_last()` from
+  each side, and each language rejecting the other queue type.
+
+Checked against C++ slick-queue v2.4.0 with the interop tests on Windows.
+
 ## [v2.3.0] - 2026-10-17
 
 Tracks C++ slick-queue v2.3.0: no queue removes its shared-memory segment on its

{slick_queue_py-2.3.0 → slick_queue_py-2.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: slick-queue-py
-Version: 2.3.0
+Version: 2.4.0
 Summary: Lock-free MPMC queue with C++ interoperability via shared memory
 Home-page: https://github.com/SlickQuant/slick-queue-py
 Author: Slick Quant
@@ -36,7 +36,7 @@ Dynamic: requires-python
 
 # slick-queue-py
 
-Python implementation of SlickQueue - a lock-free multi-producer multi-consumer (MPMC) queue with C++ interoperability through shared memory.
+Python implementation of SlickQueue - a lock-free multi-producer multi-consumer (MPMC) queue, plus a lossless single-producer single-consumer (SPSC) queue, with C++ interoperability through shared memory.
 
 This is the Python binding for the [SlickQueue C++ library](https://github.com/SlickQuant/slick-queue). The Python implementation maintains exact binary compatibility with the C++ version, enabling seamless interprocess communication between Python and C++ applications.
 
@@ -50,6 +50,8 @@ This is the Python binding for the [SlickQueue C++ library](https://github.com/S
   - **Local Memory Mode**: In-process queue using local memory (no shared memory overhead)
   - **Shared Memory Mode**: Inter-process queue for interprocess communication
 - **Lock-Free Multi-Producer Multi-Consumer**: True MPMC support using atomic operations
+- **Lossless SPSC Queue**: `SpscQueue` ports C++ `slick::spsc_queue`. It never overwrites an
+  unread element and reports a full ring to the producer instead. See [SpscQueue](#spscqueue)
 - **C++/Python Interoperability**: Python and C++ processes can share the same queue
 - **Cross-Platform**: Windows and Linux/macOS support (x86-64)
 - **Memory Layout Compatible**: Exact binary compatibility with C++ `slick::queue<T>`
@@ -642,6 +644,51 @@ sends the tracker nothing at all: `track=False` on Python 3.13+, and on older PO
 `SharedMemory` subclass that maps the segment without registering it, so there is no
 register/unregister pair to race between threads or between processes sharing one tracker.
 
+### SpscQueue
+
+A lossless single-producer single-consumer ring, compatible with C++ `slick::spsc_queue<T>`. One
+thread or process produces and one consumes. Unlike `SlickQueue` it never overwrites an unread
+element: `try_reserve()` reports a full ring instead.
+
+```python
+import struct
+from slick_queue_py import SpscQueue
+
+q = SpscQueue(name='ticks', size=1024, element_size=8)  # create, or open with the same geometry
+peer = SpscQueue(name='ticks', element_size=8)          # attach, adopting the segment's size
+local = SpscQueue(size=1024, element_size=8)            # local memory
+
+# Producer
+i = q.try_reserve()                 # SpscQueue.invalid_index when full; reserve() waits instead
+if i != SpscQueue.invalid_index:
+    q[i][:8] = struct.pack('<Q', 42)
+    q.publish(i)
+
+# Consumer
+data = peer.read()                  # next element as bytes, or None
+data, dropped = peer.read_last()    # newest element, and how many unread ones it skipped
+```
+
+- `reserve(n)` and `try_reserve(n)` reserve `n` separate elements. Write element `k` through
+  `q[i + k]`, because a batch may wrap around the end of the ring. Publish in reservation order.
+  Publishing a later batch first also exposes the earlier one, and publishing the earlier one
+  afterwards is a no-op.
+- `read()` and `read_last()` copy the element and hand its slot back before they return. C++ holds
+  the slot until the consumer's next call, so here the full `size` is always usable and there is no
+  `release()`.
+- `read_last()` loads the producer's tail and moves the read cursor there, so its cost doesn't grow
+  with the backlog. It returns `(None, 0)` if nothing was published since the last read. Note
+  that the second item counts skipped elements, where `SlickQueue.read_last()` returns a size.
+- Both cursors live in the segment, and a peer that reattaches resumes where the previous one
+  stopped. Each object caches its own side's cursor, so keep one live object per side. After
+  another process has produced or consumed, attach a fresh object rather than reusing an old one.
+- `reset()` empties the queue. It isn't thread-safe, and a peer in another process must reattach.
+- `close()`, `unlink()`, `remove_shm()`, `SpscQueue.remove(name)`, `get_shm_name()` and the
+  context manager behave as they do for `SlickQueue`. Using the queue after `close()` raises
+  `RuntimeError("queue is closed")`.
+- `traits=` takes an `SpscQueueTraits` subclass. Its one setting is `enable_cpu_relax` (default
+  `True`), which yields the GIL while `reserve()` waits. `q.traits` is a read-only snapshot.
+
 ### AtomicCursor
 
 The `AtomicCursor` class enables multi-consumer work-stealing patterns by providing an atomic read cursor that multiple consumers can coordinate through. Works in both local mode (multi-threading) and shared memory mode (multi-process).
@@ -817,6 +864,34 @@ Bits 2-3 of the nibble are reserved and must be 0; a marker that sets one is rej
 newer than this build understands. Segments created before slick-queue v1.4.0 carry no
 marker at all and are rejected at attach time.
 
+### SpscQueue layout
+
+`SpscQueue` matches C++ `slick::spsc_queue<T>`. Each cursor sits on its own 128-byte line on every
+platform:
+
+```
+Offset | Size          | Content
+-------|---------------|------------------
+0      | 8 bytes       | reserved (0)
+8      | 4 bytes       | uint32_t size (queue capacity)
+12     | 4 bytes       | uint32_t element_size
+16     | 8 bytes       | reserved (0)
+24     | 4 bytes       | uint32_t header_magic - 'SLS0' (0x534C5330)
+28     | 20 bytes      | reserved (0)
+48     | 4 bytes       | uint32_t init_state (atomic)
+52     | 76 bytes      | padding (to 128)
+128    | 8 bytes       | uint64_t tail (atomic), written by the producer
+136    | 120 bytes     | padding
+256    | 8 bytes       | uint64_t head (atomic), written by the consumer
+264    | 120 bytes     | padding (to 384)
+384    | elem*size     | data array
+```
+
+`size`, `element_size`, the marker and `init_state` sit at the same offsets as in a `SlickQueue`
+segment. Each queue type therefore turns the other's segment away at attach time, with *"holds a
+slick::spsc_queue"* or *"holds a slick::queue"*, instead of timing out. The marker's low nibble is
+reserved for future layout features and must be 0. The C++ side requires `alignof(T) <= 128`.
+
 ## Platform Support
 
 ### Fully Supported (Lock-Free)
@@ -959,6 +1034,9 @@ python tests/test_wrap_invariants.py
 
 # Segment lifetime: unlink(), remove_shm(), remove(), process exit
 python tests/test_segment_lifetime.py
+
+# SpscQueue, local and shared memory
+python tests/test_spsc.py
 ```
 
 Or run everything with pytest:
@@ -987,6 +1065,9 @@ python tests/test_interop.py --test cpp_producer_python_consumer
 python tests/test_interop.py --test multi_producer_interop
 python tests/test_interop.py --test stress_interop
 python tests/test_interop.py --test cpp_shm_creation
+
+# SpscQueue against C++ slick::spsc_queue (uses cpp_spsc_tester)
+python tests/test_spsc_interop.py
 ```
 
 The interop tests verify:
@@ -1052,6 +1133,7 @@ queue so consumers keep up, and use `loss_count()` to detect when they have not.
 | read(atomic cursor) multi-consumer | ✅ | ✅ |
 | Feature traits | Template parameter | `traits=` argument |
 | Shared layout marker | `'SLQ1'` / `'SLQ0'` | Same, and validated against C++ peers |
+| Lossless SPSC queue | `slick::spsc_queue<T>` | `SpscQueue`, same `'SLS0'` layout |
 
 ## Contributing
 

{slick_queue_py-2.3.0 → slick_queue_py-2.4.0}/README.md

@@ -1,6 +1,6 @@
 # slick-queue-py
 
-Python implementation of SlickQueue - a lock-free multi-producer multi-consumer (MPMC) queue with C++ interoperability through shared memory.
+Python implementation of SlickQueue - a lock-free multi-producer multi-consumer (MPMC) queue, plus a lossless single-producer single-consumer (SPSC) queue, with C++ interoperability through shared memory.
 
 This is the Python binding for the [SlickQueue C++ library](https://github.com/SlickQuant/slick-queue). The Python implementation maintains exact binary compatibility with the C++ version, enabling seamless interprocess communication between Python and C++ applications.
 
@@ -14,6 +14,8 @@ This is the Python binding for the [SlickQueue C++ library](https://github.com/S
   - **Local Memory Mode**: In-process queue using local memory (no shared memory overhead)
   - **Shared Memory Mode**: Inter-process queue for interprocess communication
 - **Lock-Free Multi-Producer Multi-Consumer**: True MPMC support using atomic operations
+- **Lossless SPSC Queue**: `SpscQueue` ports C++ `slick::spsc_queue`. It never overwrites an
+  unread element and reports a full ring to the producer instead. See [SpscQueue](#spscqueue)
 - **C++/Python Interoperability**: Python and C++ processes can share the same queue
 - **Cross-Platform**: Windows and Linux/macOS support (x86-64)
 - **Memory Layout Compatible**: Exact binary compatibility with C++ `slick::queue<T>`
@@ -606,6 +608,51 @@ sends the tracker nothing at all: `track=False` on Python 3.13+, and on older PO
 `SharedMemory` subclass that maps the segment without registering it, so there is no
 register/unregister pair to race between threads or between processes sharing one tracker.
 
+### SpscQueue
+
+A lossless single-producer single-consumer ring, compatible with C++ `slick::spsc_queue<T>`. One
+thread or process produces and one consumes. Unlike `SlickQueue` it never overwrites an unread
+element: `try_reserve()` reports a full ring instead.
+
+```python
+import struct
+from slick_queue_py import SpscQueue
+
+q = SpscQueue(name='ticks', size=1024, element_size=8)  # create, or open with the same geometry
+peer = SpscQueue(name='ticks', element_size=8)          # attach, adopting the segment's size
+local = SpscQueue(size=1024, element_size=8)            # local memory
+
+# Producer
+i = q.try_reserve()                 # SpscQueue.invalid_index when full; reserve() waits instead
+if i != SpscQueue.invalid_index:
+    q[i][:8] = struct.pack('<Q', 42)
+    q.publish(i)
+
+# Consumer
+data = peer.read()                  # next element as bytes, or None
+data, dropped = peer.read_last()    # newest element, and how many unread ones it skipped
+```
+
+- `reserve(n)` and `try_reserve(n)` reserve `n` separate elements. Write element `k` through
+  `q[i + k]`, because a batch may wrap around the end of the ring. Publish in reservation order.
+  Publishing a later batch first also exposes the earlier one, and publishing the earlier one
+  afterwards is a no-op.
+- `read()` and `read_last()` copy the element and hand its slot back before they return. C++ holds
+  the slot until the consumer's next call, so here the full `size` is always usable and there is no
+  `release()`.
+- `read_last()` loads the producer's tail and moves the read cursor there, so its cost doesn't grow
+  with the backlog. It returns `(None, 0)` if nothing was published since the last read. Note
+  that the second item counts skipped elements, where `SlickQueue.read_last()` returns a size.
+- Both cursors live in the segment, and a peer that reattaches resumes where the previous one
+  stopped. Each object caches its own side's cursor, so keep one live object per side. After
+  another process has produced or consumed, attach a fresh object rather than reusing an old one.
+- `reset()` empties the queue. It isn't thread-safe, and a peer in another process must reattach.
+- `close()`, `unlink()`, `remove_shm()`, `SpscQueue.remove(name)`, `get_shm_name()` and the
+  context manager behave as they do for `SlickQueue`. Using the queue after `close()` raises
+  `RuntimeError("queue is closed")`.
+- `traits=` takes an `SpscQueueTraits` subclass. Its one setting is `enable_cpu_relax` (default
+  `True`), which yields the GIL while `reserve()` waits. `q.traits` is a read-only snapshot.
+
 ### AtomicCursor
 
 The `AtomicCursor` class enables multi-consumer work-stealing patterns by providing an atomic read cursor that multiple consumers can coordinate through. Works in both local mode (multi-threading) and shared memory mode (multi-process).
@@ -781,6 +828,34 @@ Bits 2-3 of the nibble are reserved and must be 0; a marker that sets one is rej
 newer than this build understands. Segments created before slick-queue v1.4.0 carry no
 marker at all and are rejected at attach time.
 
+### SpscQueue layout
+
+`SpscQueue` matches C++ `slick::spsc_queue<T>`. Each cursor sits on its own 128-byte line on every
+platform:
+
+```
+Offset | Size          | Content
+-------|---------------|------------------
+0      | 8 bytes       | reserved (0)
+8      | 4 bytes       | uint32_t size (queue capacity)
+12     | 4 bytes       | uint32_t element_size
+16     | 8 bytes       | reserved (0)
+24     | 4 bytes       | uint32_t header_magic - 'SLS0' (0x534C5330)
+28     | 20 bytes      | reserved (0)
+48     | 4 bytes       | uint32_t init_state (atomic)
+52     | 76 bytes      | padding (to 128)
+128    | 8 bytes       | uint64_t tail (atomic), written by the producer
+136    | 120 bytes     | padding
+256    | 8 bytes       | uint64_t head (atomic), written by the consumer
+264    | 120 bytes     | padding (to 384)
+384    | elem*size     | data array
+```
+
+`size`, `element_size`, the marker and `init_state` sit at the same offsets as in a `SlickQueue`
+segment. Each queue type therefore turns the other's segment away at attach time, with *"holds a
+slick::spsc_queue"* or *"holds a slick::queue"*, instead of timing out. The marker's low nibble is
+reserved for future layout features and must be 0. The C++ side requires `alignof(T) <= 128`.
+
 ## Platform Support
 
 ### Fully Supported (Lock-Free)
@@ -923,6 +998,9 @@ python tests/test_wrap_invariants.py
 
 # Segment lifetime: unlink(), remove_shm(), remove(), process exit
 python tests/test_segment_lifetime.py
+
+# SpscQueue, local and shared memory
+python tests/test_spsc.py
 ```
 
 Or run everything with pytest:
@@ -951,6 +1029,9 @@ python tests/test_interop.py --test cpp_producer_python_consumer
 python tests/test_interop.py --test multi_producer_interop
 python tests/test_interop.py --test stress_interop
 python tests/test_interop.py --test cpp_shm_creation
+
+# SpscQueue against C++ slick::spsc_queue (uses cpp_spsc_tester)
+python tests/test_spsc_interop.py
 ```
 
 The interop tests verify:
@@ -1016,6 +1097,7 @@ queue so consumers keep up, and use `loss_count()` to detect when they have not.
 | read(atomic cursor) multi-consumer | ✅ | ✅ |
 | Feature traits | Template parameter | `traits=` argument |
 | Shared layout marker | `'SLQ1'` / `'SLQ0'` | Same, and validated against C++ peers |
+| Lossless SPSC queue | `slick::spsc_queue<T>` | `SpscQueue`, same `'SLS0'` layout |
 
 ## Contributing
 

{slick_queue_py-2.3.0 → slick_queue_py-2.4.0}/slick_queue_py.egg-info/SOURCES.txt

@@ -25,6 +25,7 @@ tests/cpp_consumer.cpp
 tests/cpp_multi_producer.cpp
 tests/cpp_producer.cpp
 tests/cpp_read_last_tester.cpp
+tests/cpp_spsc_tester.cpp
 tests/cpp_work_stealing_consumer.cpp
 tests/run_test.py
 tests/test_atomic_cursor.py
@@ -40,5 +41,7 @@ tests/test_read_last.py
 tests/test_read_last_interop.py
 tests/test_reset_detection.py
 tests/test_segment_lifetime.py
+tests/test_spsc.py
+tests/test_spsc_interop.py
 tests/test_traits.py
 tests/test_wrap_invariants.py