slick-queue-py 2.1.0__tar.gz → 2.3.0__tar.gz

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/API_DIFFERENCES.md

@@ -2,7 +2,7 @@
 
 This document explains the intentional API differences between the C++ `slick::queue<T>` and Python `SlickQueue` implementations.
 
-Both track the same version line; this document describes slick-queue v2.1.0.
+Both track the same version line; this document describes slick-queue v2.3.0.
 
 ## Core Compatibility
 
@@ -227,27 +227,44 @@ q[idx][:4] = struct.pack("<i", 42)  # Manual packing
 {
     slick::queue<T> q(size, "name");
     // ... use queue ...
-}  // Destructor automatically cleans up
+    q.remove_shm();  // Coordinator, once no peer will attach again; q's mapping stays valid
+}  // Destructor unmaps; it never removes the segment
+
+slick::shm::shared_memory::remove("name");  // By name, without a queue - e.g. at startup
 ```
 
 **Python:**
 ```python
 # Option 1: Manual cleanup
-q = SlickQueue(name='name', size=size, element_size=elem_size, create=True)
+q = SlickQueue(name='name', size=size, element_size=elem_size)
 try:
     # ... use queue ...
 finally:
     q.close()
-    q.unlink()  # Only call from creator process
+    q.unlink()  # Only from the process that coordinates the queue's lifetime
 
-# Option 2: Context manager (recommended)
-with SlickQueue(name='name', size=size, element_size=elem_size, create=True) as q:
+# Option 2: Context manager - closes, never unlinks
+with SlickQueue(name='name', size=size, element_size=elem_size) as q:
     # ... use queue ...
-    pass  # Automatic cleanup
+    pass
+
+SlickQueue.remove('name')  # By name, without attaching - e.g. at startup
 ```
 
 **Rationale:** Python's garbage collection is non-deterministic, so explicit cleanup or context managers are needed.
 
+Neither side deletes a segment on its own, matching C++ v2.3.0: not the C++ destructor, and not
+Python's `close()`, context manager or process exit. Python keeps its segments out of
+`multiprocessing`'s resource tracker for this, since the tracker would otherwise unlink them when
+the process exits.
+
+| C++ | Python |
+|-----|--------|
+| `remove_shm()` | `remove_shm()` - same contract, unlinks by name on every call |
+| - | `unlink()` - unlinks once, valid after `close()`; a repeat is a no-op |
+| `slick::shm::shared_memory::remove(name)` | `SlickQueue.remove(name)` - also accepts a `/`-prefixed name |
+| `shm_name()` - `""` for a local queue | `get_shm_name()` - `None` for a local queue, `/`-prefixed on POSIX |
+
 ---
 
 ## Interoperability Examples
@@ -339,6 +356,7 @@ int main() {
 | Element access | Pointer | `memoryview` | Python memory safety |
 | Type safety | Templates | `struct` module | Language difference |
 | Cleanup | RAII (destructor) | Explicit or context manager | Python GC is non-deterministic |
+| Segment removal | `remove_shm()` | `remove_shm()`, `unlink()`, `SlickQueue.remove()` | Neither side unlinks on its own |
 | `read()` data | `T*` into the ring | `bytes` copy | No safe raw pointers in Python |
 | Overwritten-in-flight data | Caller sees a volatile pointer | Copy may hold a newer record's bytes | Detect with `loss_count()` |
 | Feature configuration | `Traits` template parameter | `traits=` argument | Python has no templates |

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/CHANGELOG.md

@@ -1,5 +1,79 @@
 # Changelogs
 
+## [v2.3.0] - 2026-10-17
+
+Tracks C++ slick-queue v2.3.0: no queue removes its shared-memory segment on its
+own any more, and `remove_shm()` is how the coordinating process does. The
+version moves from 2.1.0 to 2.3.0 to stay on the C++ number. C++ v2.2.0 added
+only `shm_name()`, which Python already had as `get_shm_name()`. The
+shared-memory layout is unchanged, so 2.1.0 peers in either language
+interoperate as before.
+
+### Segment lifetime (behavior change)
+
+- On POSIX, a segment is no longer unlinked when the process that created it
+  exits. `SharedMemory` registered every segment with `multiprocessing`'s
+  resource tracker, which unlinks what is registered when the process exits, so
+  a creator that exited first stranded every peer still attached on an orphaned
+  mapping, while its restart silently created a fresh segment under the same
+  name that those peers never saw. C++ v2.3.0 stopped its destructor doing the
+  same. A name now lives until `unlink()`, `remove_shm()` or
+  `SlickQueue.remove(name)`, and a restarted process reattaches to the existing
+  segment and keeps its contents. Windows is unaffected: a section dies with its
+  last handle.
+- The queue sends the resource tracker nothing at all: `track=False` on Python
+  3.13+, and on older POSIX Pythons a `SharedMemory` subclass that maps the
+  segment without registering it. With no register/unregister pair there is
+  nothing to race between threads, or between `multiprocessing` children, which
+  share their parent's tracker.
+- Migration: code that relied on the cleanup at exit now leaves segments in
+  `/dev/shm`, and its next run attaches to them - a creator joins the existing
+  segment instead of initializing it, the contents are the old run's, and a
+  creator with a different geometry raises `ValueError`. Call `SlickQueue.remove(name)` at startup for a clean start, or
+  `unlink()` / `remove_shm()` at shutdown once no peer will attach again.
+
+### Added
+
+- `remove_shm() -> bool`, matching C++: unlinks the segment's name so the next
+  queue created with it starts fresh, leaving open mappings - this queue's own
+  included - valid. It unlinks by name on every call, and leaves `unlink()`
+  nothing to do afterwards, so a later `unlink()` cannot delete a replacement
+  created under the name. Returns `False` for a local-memory queue or when the
+  name is already gone, and always `True` for a shared queue on Windows.
+- `SlickQueue.remove(name) -> bool`, matching C++
+  `slick::shm::shared_memory::remove()`: unlinks a segment by name without
+  attaching to it. Accepts the `/`-prefixed value of `get_shm_name()`, and is
+  always `True` on Windows.
+
+### Fixed
+
+- `unlink()` was a silent no-op once `close()` had run, because `close()` drops
+  the `SharedMemory` object it went through. The documented `shared_memory`
+  idiom - close every handle, then unlink the name once - therefore leaked the
+  segment on POSIX. The name is now kept alongside the mapping, so `unlink()`
+  works after `close()`, and repeating it is a no-op, so it cannot delete a
+  segment a later run has created under the same name.
+- Before Python 3.13 on POSIX, a process that only attached to a queue unlinked
+  its segment when it exited, because attaching registered the name with that
+  process's resource tracker too.
+
+### Tests
+
+- New `tests/test_segment_lifetime.py` with 12 tests, registered with CTest:
+  `unlink()` after `close()` and repeated, `close()` and the context manager
+  keeping the segment, `remove_shm()` on local and shared queues, `unlink()`
+  after `remove_shm()` sparing a replacement segment, `remove()` by name, a
+  creator's and an attacher's exit in a fresh interpreter, no resource-tracker
+  traffic at all, a child forked while another thread is constructing a queue,
+  and spawned children attaching at once.
+- `test_traits.py` and `test_items_per_slot.py` open their hand-built raw
+  segments through the library's opener, so those handles and the queue no
+  longer both claim a name in the resource tracker.
+
+Checked against C++ slick-queue v2.3.0 with the interop tests on Windows and on
+Linux, run twice on Linux so a segment left behind would fail the second run,
+and on Linux under Python 3.12 (the subclass) and 3.14 (`track=False`).
+
 ## [v2.1.0] - 2026-09-24
 
 Tracks C++ slick-queue v2.1.0. Adds `items_per_slot`, so one control slot can

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: slick-queue-py
-Version: 2.1.0
+Version: 2.3.0
 Summary: Lock-free MPMC queue with C++ interoperability via shared memory
 Home-page: https://github.com/SlickQuant/slick-queue-py
 Author: Slick Quant
@@ -600,11 +600,47 @@ Get memoryview for writing to reserved slot.
 
 #### `close()`
 
-Close the shared memory connection. Always call this before unlinking.
+Close the shared memory connection. Never deletes the segment.
 
 #### `unlink()`
 
-Delete the shared memory segment. Only call from the process that created it.
+Delete the shared memory segment this queue is attached to. Valid before or after `close()`
+(close every handle, then unlink once), and a no-op when repeated, so it cannot delete a segment
+a later run has since created under the same name. A no-op for local memory mode.
+
+#### `remove_shm() -> bool`
+
+Unlink the segment's name so the next queue created with it starts fresh, matching C++
+`remove_shm()`. Mappings already open, including this queue's own, stay valid. Unlike `unlink()`
+it unlinks by name on every call. Returns `False` for a local-memory queue or if the name is
+already gone, and always `True` for a shared queue on Windows.
+
+#### `SlickQueue.remove(name) -> bool`
+
+Unlink a segment by name without attaching to it, matching C++
+`slick::shm::shared_memory::remove()`. Typically called at startup to clear a segment a previous
+run left behind. Accepts the `/`-prefixed value of `get_shm_name()`. Always `True` on Windows.
+
+#### `get_shm_name() -> Optional[str]`
+
+The segment name to pass to C++ (`/`-prefixed on POSIX), or `None` for a local-memory queue.
+
+#### Segment lifetime
+
+No queue deletes its segment on its own: not `close()`, not the context manager, and not process
+exit, matching C++ slick-queue v2.3.0. On POSIX a name outlives every mapping until it is
+unlinked, so a departing process that unlinked it while a peer was still attached would leave that
+peer reading an orphaned mapping while a restarted process silently created a fresh segment under
+the same name. The process that coordinates the queue's lifetime calls `unlink()`, `remove_shm()`
+or `SlickQueue.remove(name)` once no peer will attach again; until then a restarted process
+reattaches to the existing segment and keeps its contents. On Windows a segment is freed when its
+last handle closes.
+
+The queue keeps its segments out of `multiprocessing`'s resource tracker, which would otherwise
+unlink them when the process exits - before Python 3.13, even in a process that only attached. It
+sends the tracker nothing at all: `track=False` on Python 3.13+, and on older POSIX Pythons a
+`SharedMemory` subclass that maps the segment without registering it, so there is no
+register/unregister pair to race between threads or between processes sharing one tracker.
 
 ### AtomicCursor
 
@@ -920,6 +956,9 @@ python tests/test_reset_detection.py
 
 # Wrapping-producer record/size invariants
 python tests/test_wrap_invariants.py
+
+# Segment lifetime: unlink(), remove_shm(), remove(), process exit
+python tests/test_segment_lifetime.py
 ```
 
 Or run everything with pytest:
@@ -963,7 +1002,7 @@ The interop tests verify:
 
 1. **Buffer Cleanup Warning**: You may see a `BufferError: cannot close exported pointers exist` warning during garbage collection. This is a **harmless warning** caused by Python's ctypes creating internal buffer references that persist beyond explicit cleanup. It occurs during program exit and **does not affect functionality, performance, or correctness**. The queue works perfectly despite this warning.
 
-2. **UserWarning**: On Linux you may see `UserWarning: resource_tracker: There appear to be 4 leaked shared_memory objects to clean up at shutdown`. This is a **harmless warning** caused by Python's ctypes creating internal buffer references that persist beyond explicit cleanup. It occurs during program exit and **does not affect functionality, performance, or correctness**. The queue works perfectly despite this warning.
+2. **UserWarning**: On Linux you may see `UserWarning: resource_tracker: There appear to be 4 leaked shared_memory objects to clean up at shutdown`. `SlickQueue` keeps its own segments out of the resource tracker (see [Segment lifetime](#segment-lifetime)), so the warning comes from a `multiprocessing.shared_memory.SharedMemory` your code opened directly, such as one holding an `AtomicCursor`. The tracker unlinks those segments at exit; pass `track=False` (Python 3.13+) to keep them.
 
 ## Architecture
 

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/README.md

@@ -564,11 +564,47 @@ Get memoryview for writing to reserved slot.
 
 #### `close()`
 
-Close the shared memory connection. Always call this before unlinking.
+Close the shared memory connection. Never deletes the segment.
 
 #### `unlink()`
 
-Delete the shared memory segment. Only call from the process that created it.
+Delete the shared memory segment this queue is attached to. Valid before or after `close()`
+(close every handle, then unlink once), and a no-op when repeated, so it cannot delete a segment
+a later run has since created under the same name. A no-op for local memory mode.
+
+#### `remove_shm() -> bool`
+
+Unlink the segment's name so the next queue created with it starts fresh, matching C++
+`remove_shm()`. Mappings already open, including this queue's own, stay valid. Unlike `unlink()`
+it unlinks by name on every call. Returns `False` for a local-memory queue or if the name is
+already gone, and always `True` for a shared queue on Windows.
+
+#### `SlickQueue.remove(name) -> bool`
+
+Unlink a segment by name without attaching to it, matching C++
+`slick::shm::shared_memory::remove()`. Typically called at startup to clear a segment a previous
+run left behind. Accepts the `/`-prefixed value of `get_shm_name()`. Always `True` on Windows.
+
+#### `get_shm_name() -> Optional[str]`
+
+The segment name to pass to C++ (`/`-prefixed on POSIX), or `None` for a local-memory queue.
+
+#### Segment lifetime
+
+No queue deletes its segment on its own: not `close()`, not the context manager, and not process
+exit, matching C++ slick-queue v2.3.0. On POSIX a name outlives every mapping until it is
+unlinked, so a departing process that unlinked it while a peer was still attached would leave that
+peer reading an orphaned mapping while a restarted process silently created a fresh segment under
+the same name. The process that coordinates the queue's lifetime calls `unlink()`, `remove_shm()`
+or `SlickQueue.remove(name)` once no peer will attach again; until then a restarted process
+reattaches to the existing segment and keeps its contents. On Windows a segment is freed when its
+last handle closes.
+
+The queue keeps its segments out of `multiprocessing`'s resource tracker, which would otherwise
+unlink them when the process exits - before Python 3.13, even in a process that only attached. It
+sends the tracker nothing at all: `track=False` on Python 3.13+, and on older POSIX Pythons a
+`SharedMemory` subclass that maps the segment without registering it, so there is no
+register/unregister pair to race between threads or between processes sharing one tracker.
 
 ### AtomicCursor
 
@@ -884,6 +920,9 @@ python tests/test_reset_detection.py
 
 # Wrapping-producer record/size invariants
 python tests/test_wrap_invariants.py
+
+# Segment lifetime: unlink(), remove_shm(), remove(), process exit
+python tests/test_segment_lifetime.py
 ```
 
 Or run everything with pytest:
@@ -927,7 +966,7 @@ The interop tests verify:
 
 1. **Buffer Cleanup Warning**: You may see a `BufferError: cannot close exported pointers exist` warning during garbage collection. This is a **harmless warning** caused by Python's ctypes creating internal buffer references that persist beyond explicit cleanup. It occurs during program exit and **does not affect functionality, performance, or correctness**. The queue works perfectly despite this warning.
 
-2. **UserWarning**: On Linux you may see `UserWarning: resource_tracker: There appear to be 4 leaked shared_memory objects to clean up at shutdown`. This is a **harmless warning** caused by Python's ctypes creating internal buffer references that persist beyond explicit cleanup. It occurs during program exit and **does not affect functionality, performance, or correctness**. The queue works perfectly despite this warning.
+2. **UserWarning**: On Linux you may see `UserWarning: resource_tracker: There appear to be 4 leaked shared_memory objects to clean up at shutdown`. `SlickQueue` keeps its own segments out of the resource tracker (see [Segment lifetime](#segment-lifetime)), so the warning comes from a `multiprocessing.shared_memory.SharedMemory` your code opened directly, such as one holding an `AtomicCursor`. The tracker unlinks those segments at exit; pass `track=False` (Python 3.13+) to keep them.
 
 ## Architecture
 

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/slick_queue_py.egg-info/SOURCES.txt

@@ -39,5 +39,6 @@ tests/test_queue.py
 tests/test_read_last.py
 tests/test_read_last_interop.py
 tests/test_reset_detection.py
+tests/test_segment_lifetime.py
 tests/test_traits.py
 tests/test_wrap_invariants.py

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/slick_queue_py.py

@@ -29,8 +29,11 @@ Supported on Python 3.8+ (uses multiprocessing.shared_memory).
 """
 from __future__ import annotations
 
-__version__ = '2.1.0'
+__version__ = '2.3.0'
 
+import inspect
+import mmap
+import os
 import struct
 import sys
 import time
@@ -40,6 +43,81 @@ from atomic_ops import AtomicReservedInfo, AtomicUInt64, AtomicCursor, check_pla
 # Use Python's built-in shared memory (available in Python 3.8+)
 from multiprocessing.shared_memory import SharedMemory
 
+# Python >= 3.13 supports SharedMemory(track=False), which keeps a segment out of
+# multiprocessing's resource tracker.
+_SHM_SUPPORTS_TRACK = 'track' in inspect.signature(SharedMemory.__init__).parameters
+
+if not _SHM_SUPPORTS_TRACK and sys.platform != 'win32':
+    import _posixshmem
+
+    class _UntrackedSharedMemory(SharedMemory):
+        """``SharedMemory(track=False)`` for POSIX Python < 3.13, where that
+        argument does not exist.
+
+        The base class touches the resource tracker in exactly two places, and
+        only those are replaced: ``__init__``, which registers the name, and
+        ``unlink()``, which unregisters it. This ``__init__`` sets the same
+        attributes the base one does, which ``close()``, ``buf``, ``name``,
+        ``size``, ``__del__`` and ``__reduce__`` rely on.
+
+        Sending the tracker nothing at all is what makes this safe. Registering
+        and then retracting at once needed a lock against other threads, which a
+        fork could leave held in the child, and still raced against other
+        processes sharing the tracker - multiprocessing children share their
+        parent's - which answers an UNREGISTER for a name it no longer holds by
+        printing a KeyError traceback.
+        """
+
+        def __init__(self, name: str, create: bool = False, size: int = 0):
+            flags = os.O_RDWR
+            if create:
+                if size <= 0:
+                    raise ValueError("'size' must be a positive number different from zero")
+                flags |= os.O_CREAT | os.O_EXCL
+            self._name = '/' + name
+            self._fd = _posixshmem.shm_open(self._name, flags, mode=self._mode)
+            try:
+                if create:
+                    os.ftruncate(self._fd, size)
+                size = os.fstat(self._fd).st_size
+                self._mmap = mmap.mmap(self._fd, size)
+            except OSError:
+                os.close(self._fd)
+                self._fd = -1
+                if create:  # only a segment this call made is ours to delete
+                    try:
+                        _posixshmem.shm_unlink(self._name)
+                    except OSError:
+                        pass
+                raise
+            self._size = size
+            self._buf = memoryview(self._mmap)
+
+        def unlink(self) -> None:
+            if self._name:
+                _posixshmem.shm_unlink(self._name)
+
+
+def _open_shm(name: str, create: bool, size: int = 0) -> SharedMemory:
+    """Create or attach to a segment without registering it with
+    multiprocessing's resource tracker.
+
+    A registered name is unlinked by the tracker when the process exits, which on
+    POSIX is the lifetime C++ slick-queue v2.3.0 dropped: a peer still attached
+    keeps reading the orphaned mapping while a restarted process silently
+    creates a fresh segment under the same name, so the two never meet again.
+    Python before 3.13 registers on attach as well as on create, so any peer's
+    exit removed the segment. The name now outlives every process until
+    remove_shm(), unlink() or remove() is called. Windows has no tracker; a
+    section there dies with its last handle.
+    """
+    if _SHM_SUPPORTS_TRACK:
+        return SharedMemory(name=name, create=create, size=size, track=False)
+    if sys.platform == 'win32':
+        return SharedMemory(name=name, create=create, size=size)
+    return _UntrackedSharedMemory(name, create, size)
+
+
 # Layout constants
 # Shared memory header layout (64 bytes total):
 # Offset 0-7:   std::atomic<reserved_info> (8 bytes)
@@ -318,6 +396,9 @@ class SlickQueue:
 
         self.use_shm = name is not None
         self._shm: Optional[SharedMemory] = None
+        # Tracker-form name of the mapping. Outlives close(), which drops _shm, so
+        # unlink() still knows what to unlink - see unlink().
+        self._shm_name: Optional[str] = None
         self._local_buf: Optional[bytearray] = None
         self.size = None
         self._own = False
@@ -344,11 +425,10 @@ class SlickQueue:
                     raise ValueError("size and element_size required when creating")
                 total = self._layout_data_offset() + self.element_size * self.size
                 try:
-                    self._shm = SharedMemory(name=self.name, create=True, size=total)
-                    # print(f"**** create new shm {self.name}")
+                    self._shm = _open_shm(self.name, create=True, size=total)
                 except FileExistsError:
-                    # print(f"**** shm already exists, opening {self.name}")
-                    self._shm = SharedMemory(name=self.name, create=False)
+                    self._shm = _open_shm(self.name, create=False)
+                self._shm_name = self._shm._name
 
                 # Use CAS on init_state to determine ownership (matches C++ queue.h:618-648)
                 buf = self._shm.buf
@@ -428,7 +508,8 @@ class SlickQueue:
                     raise ValueError("element_size must be provided when opening existing shared memory")
 
                 # Open existing shared memory (size parameter not needed/ignored)
-                self._shm = SharedMemory(name=self.name, create=False)
+                self._shm = _open_shm(self.name, create=False)
+                self._shm_name = self._shm._name
                 buf = self._shm.buf
 
                 # Wait for initialization to complete (matches C++ queue.h:558-562)
@@ -1199,18 +1280,88 @@ class SlickQueue:
             pass
 
     def unlink(self) -> None:
-        """Unlink (delete) the shared memory segment.
+        """Unlink (delete) the shared memory segment this queue is attached to.
+
+        Only applicable for shared memory mode; a no-op for local memory mode. No
+        queue ever does this on its own - not close(), not the context manager and
+        not process exit - so call it from the process that coordinates the queue's
+        lifetime, once no peer will attach again.
+
+        Valid after :meth:`close`, which is the documented ``shared_memory`` idiom -
+        close every handle, then unlink the name once. Deleting a name never touches
+        mappings that already exist, open or closed. Unlinking twice is a no-op, so a
+        repeat cannot delete a segment a later run has since created under the same
+        name; :meth:`remove_shm` is the C++-style call that unlinks on every call.
+        """
+        prefixed = self._shm_name
+        if prefixed is None:
+            return
+        self._shm_name = None  # unlink once, however many times this is called
+        SlickQueue.remove(prefixed)
+
+    def remove_shm(self) -> bool:
+        """Unlink this queue's segment name so the next queue created with it starts
+        fresh. Matches C++ ``remove_shm()``.
+
+        Mappings that are already open, including this queue's own, stay valid until
+        they are closed. Unlike :meth:`unlink` it unlinks by name on every call, so
+        it deletes whatever holds the name at the time.
+
+        After this call :meth:`unlink` is a no-op, whatever the result: a name
+        removed here, or one already gone, can be taken by a replacement segment,
+        which the pending unlink() would otherwise delete.
 
-        Only applicable for shared memory mode. Does nothing for local memory mode.
+        Returns:
+            True if the name was removed; False for a local-memory queue or if
+            removal failed (on POSIX, typically because the name is already gone).
+            Always True for a shared queue on Windows, where the segment is freed
+            when its last handle closes and there is no name to remove.
         """
         if not self.use_shm:
-            return  # Nothing to unlink for local memory
+            return False
+        self._shm_name = None  # unlink() is done with this name - see above
+        return SlickQueue.remove(self.name)
+
+    @staticmethod
+    def remove(name: str) -> bool:
+        """Unlink a shared memory segment by name without attaching to it. Matches
+        C++ ``slick::shm::shared_memory::remove()``.
+
+        Typically called once at startup to clear a segment a previous run left
+        behind, which a creator would otherwise reattach to with its old contents::
+
+            SlickQueue.remove("my_queue")
+            q = SlickQueue(name="my_queue", size=1024, element_size=64)
 
+        A leading ``/`` is accepted, so the value of :meth:`get_shm_name` can be
+        passed back.
+
+        Returns:
+            True if the segment was removed (always True on Windows).
+
+        Warning:
+            Only call this when no process will attach again. On POSIX the name is
+            unlinked immediately, so the next creator gets a fresh segment while any
+            process still mapped to the old one keeps reading the orphaned copy. On
+            Windows this is a no-op - a section disappears once its last handle
+            closes.
+        """
+        if name.startswith('/'):
+            name = name[1:]
+        if sys.platform == 'win32':
+            return True
+        # Straight to shm_unlink rather than SharedMemory.unlink(), which before
+        # Python 3.13 also retracts a tracker registration this module never keeps -
+        # see _open_shm().
         try:
-            if self._shm:
-                self._shm.unlink()
-        except Exception:
-            pass
+            from _posixshmem import shm_unlink
+        except ImportError:  # pragma: no cover - POSIX always has it
+            return False
+        try:
+            shm_unlink('/' + name)
+        except OSError:  # not there, or not ours to unlink
+            return False
+        return True
 
     def __enter__(self):
         """Context manager entry."""

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/tests/test_items_per_slot.py

@@ -11,8 +11,6 @@ from pathlib import Path
 
 sys.path.insert(0, str(Path(__file__).parent.parent))
 
-from multiprocessing.shared_memory import SharedMemory
-
 from atomic_ops import AtomicCursor
 from slick_queue_py import (
     SlickQueue,
@@ -21,6 +19,7 @@ from slick_queue_py import (
     SLOT_SIZE,
     HEADER_MAGIC_OFFSET,
     ITEMS_PER_SLOT_OFFSET,
+    _open_shm,
 )
 
 SLQ1 = 0x534C5131
@@ -37,7 +36,7 @@ def _marker(q):
 
 
 def _set_raw_marker(q, marker):
-    raw = SharedMemory(name=q._shm.name, create=False)
+    raw = _open_shm(q._shm.name, create=False)
     try:
         struct.pack_into("<I", raw.buf, HEADER_MAGIC_OFFSET, marker)
     finally:
@@ -364,7 +363,7 @@ def test_zero_in_header_reads_as_one():
     name = _unique("slq_ips_legacy")
     creator = SlickQueue(name=name, size=8, element_size=8)
     try:
-        raw = SharedMemory(name=creator._shm.name, create=False)
+        raw = _open_shm(creator._shm.name, create=False)
         try:
             struct.pack_into("<I", raw.buf, ITEMS_PER_SLOT_OFFSET, 0)
         finally:

slick_queue_py-2.3.0/tests/test_segment_lifetime.py

@@ -0,0 +1,410 @@
+"""
+Tests for shared-memory segment lifetime (matching C++ slick-queue v2.3.0).
+
+No queue unlinks its segment on its own - not close(), not the context manager and
+not process exit through multiprocessing's resource tracker. The name lives until
+unlink(), remove_shm() or SlickQueue.remove() is called.
+"""
+import os
+import signal
+import struct
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+import warnings
+from multiprocessing import resource_tracker
+from pathlib import Path
+
+ROOT = Path(__file__).parent.parent
+sys.path.insert(0, str(ROOT))
+
+from slick_queue_py import SlickQueue
+
+IS_WINDOWS = sys.platform == 'win32'
+
+# A fresh interpreter, so it has its own resource tracker - a multiprocessing child
+# would share the parent's, and the parent's exit would be what unlinked the name.
+CHILD = r"""
+import struct, sys
+sys.path.insert(0, sys.argv[1])
+from slick_queue_py import SlickQueue
+name, mode, value = sys.argv[2], sys.argv[3], int(sys.argv[4])
+if mode == 'create':
+    q = SlickQueue(name=name, size=16, element_size=8)
+    assert q._own, "child expected to create the segment"
+else:
+    q = SlickQueue(name=name, element_size=8)
+idx = q.reserve()
+q[idx][:8] = struct.pack('<Q', value)
+q.publish(idx)
+q.close()
+"""
+
+
+def _unique(prefix):
+    return f"{prefix}_{os.getpid()}"
+
+
+def _run_child(name, mode, value):
+    result = subprocess.run(
+        [sys.executable, "-c", CHILD, str(ROOT), name, mode, str(value)],
+        capture_output=True, text=True, timeout=60)
+    assert result.returncode == 0, f"child failed:\n{result.stdout}\n{result.stderr}"
+    # The tracker warns about every registered name it cleans up at shutdown
+    assert "resource_tracker" not in result.stderr, result.stderr
+    return result
+
+
+def _values(q):
+    """Every value published so far, from the start of the queue."""
+    out = []
+    cursor = 0
+    while True:
+        data, size, cursor = q.read(cursor)
+        if data is None:
+            return out
+        out.append(struct.unpack('<Q', data[:8])[0])
+
+
+def _expect_gone(name):
+    try:
+        q = SlickQueue(name=name, element_size=8)
+    except FileNotFoundError:
+        return
+    q.close()
+    raise AssertionError(f"segment {name!r} still exists")
+
+
+def test_unlink_after_close_removes_segment():
+    """unlink() after close() was a silent no-op: close() dropped the SharedMemory
+    object unlink() went through, so the documented idiom leaked the segment."""
+    name = _unique("lt_unlink_after_close")
+    SlickQueue.remove(name)
+    q = SlickQueue(name=name, size=16, element_size=8)
+    q.close()
+    q.unlink()
+    _expect_gone(name)
+
+
+def test_unlink_twice_is_noop():
+    """A repeated unlink() must not delete a segment a later run created under the
+    same name."""
+    name = _unique("lt_unlink_twice")
+    SlickQueue.remove(name)
+    first = SlickQueue(name=name, size=16, element_size=8)
+    first.close()
+    first.unlink()
+
+    second = SlickQueue(name=name, size=16, element_size=8)
+    try:
+        assert second._own
+        first.unlink()
+        reopened = SlickQueue(name=name, element_size=8)  # still there
+        reopened.close()
+    finally:
+        second.close()
+        second.unlink()
+
+
+def test_close_and_context_manager_keep_segment():
+    name = _unique("lt_close_keeps")
+    SlickQueue.remove(name)
+    keeper = SlickQueue(name=name, size=16, element_size=8)  # holds it on Windows
+    try:
+        with SlickQueue(name=name, element_size=8) as q:
+            idx = q.reserve()
+            q[idx][:8] = struct.pack('<Q', 7)
+            q.publish(idx)
+        q.close()
+        reopened = SlickQueue(name=name, element_size=8)
+        try:
+            assert _values(reopened) == [7]
+        finally:
+            reopened.close()
+    finally:
+        keeper.close()
+        keeper.unlink()
+
+
+def test_remove_shm_local_queue_returns_false():
+    q = SlickQueue(size=16, element_size=8)
+    try:
+        assert q.remove_shm() is False
+    finally:
+        q.close()
+
+
+def test_remove_shm_keeps_open_mapping_valid():
+    name = _unique("lt_remove_shm")
+    SlickQueue.remove(name)
+    q = SlickQueue(name=name, size=16, element_size=8)
+    try:
+        idx = q.reserve()
+        q[idx][:8] = struct.pack('<Q', 42)
+        q.publish(idx)
+
+        assert q.remove_shm() is True
+        # The mapping outlives its name
+        assert _values(q) == [42]
+        idx = q.reserve()
+        q[idx][:8] = struct.pack('<Q', 43)
+        q.publish(idx)
+        assert _values(q) == [42, 43]
+
+        if not IS_WINDOWS:
+            # The next creator starts fresh rather than reattaching
+            assert q.remove_shm() is False  # already gone
+            fresh = SlickQueue(name=name, size=16, element_size=8)
+            try:
+                assert fresh._own
+                assert _values(fresh) == []
+            finally:
+                fresh.close()
+                fresh.unlink()
+    finally:
+        q.close()
+        q.unlink()
+
+
+def test_remove_by_name():
+    name = _unique("lt_remove_static")
+    SlickQueue.remove(name)
+    q = SlickQueue(name=name, size=16, element_size=8)
+    shm_name = q.get_shm_name()
+    q.close()
+    # get_shm_name()'s value, '/'-prefixed on POSIX, is accepted back
+    assert SlickQueue.remove(shm_name) is True
+    _expect_gone(name)
+    # Windows has no name to remove, so it always reports success
+    assert SlickQueue.remove(name) is IS_WINDOWS
+
+
+def test_creator_exit_keeps_segment():
+    """A creator's exit used to unlink the name on POSIX through the resource
+    tracker, stranding any peer on an orphaned mapping. On Windows a section dies
+    with its last handle, which no one else holds here."""
+    if IS_WINDOWS:
+        return
+    name = _unique("lt_creator_exit")
+    SlickQueue.remove(name)
+    _run_child(name, 'create', 11)
+    q = SlickQueue(name=name, element_size=8)
+    try:
+        assert _values(q) == [11]
+        # A restarted creator reattaches and keeps the contents
+        again = SlickQueue(name=name, size=16, element_size=8)
+        try:
+            assert not again._own
+        finally:
+            again.close()
+    finally:
+        q.close()
+        q.unlink()
+
+
+def test_attacher_exit_keeps_segment():
+    """Before Python 3.13 attaching registered the name with the attacher's resource
+    tracker too, so any peer's exit unlinked the segment on POSIX."""
+    name = _unique("lt_attacher_exit")
+    SlickQueue.remove(name)
+    q = SlickQueue(name=name, size=16, element_size=8)
+    try:
+        _run_child(name, 'open', 21)
+        _run_child(name, 'open', 22)
+        reopened = SlickQueue(name=name, element_size=8)
+        try:
+            assert _values(reopened) == [21, 22]
+        finally:
+            reopened.close()
+    finally:
+        q.close()
+        q.unlink()
+
+
+def test_unlink_after_remove_shm_spares_replacement():
+    """Regression: remove_shm() left unlink()'s pending name in place, so once the
+    old segment was removed and a replacement created under the same name, the old
+    queue's unlink() deleted the replacement. Also when remove_shm() finds the name
+    already gone, since a replacement can take it just the same."""
+    name = _unique("lt_rm_unlink")
+    for gone_first in (False, True):
+        SlickQueue.remove(name)
+        old = SlickQueue(name=name, size=16, element_size=8)
+        try:
+            if gone_first:
+                SlickQueue.remove(name)  # someone else got there first
+                assert old.remove_shm() is IS_WINDOWS
+            else:
+                assert old.remove_shm() is True
+            replacement = SlickQueue(name=name, size=16, element_size=8)
+            try:
+                if not IS_WINDOWS:
+                    assert replacement._own
+                old.unlink()
+                SlickQueue(name=name, element_size=8).close()  # still there
+            finally:
+                replacement.close()
+                replacement.unlink()
+        finally:
+            old.close()
+
+
+def test_never_talks_to_resource_tracker():
+    """No REGISTER or UNREGISTER for a queue's segment reaches multiprocessing's
+    resource tracker - not on create, attach, close or unlink. Before Python 3.13
+    the library registered each name and then retracted it at once, which raced
+    between processes sharing one tracker and needed a lock that a fork could
+    leave held (see the two tests below)."""
+    name = _unique("lt_no_tracker")
+    SlickQueue.remove(name)
+    calls = []
+    orig_register, orig_unregister = resource_tracker.register, resource_tracker.unregister
+    resource_tracker.register = lambda n, rtype: calls.append(("register", n))
+    resource_tracker.unregister = lambda n, rtype: calls.append(("unregister", n))
+    try:
+        creator = SlickQueue(name=name, size=16, element_size=8)
+        opener = SlickQueue(name=name, element_size=8)
+        second = SlickQueue(name=name, size=16, element_size=8)  # attaches
+        for q in (second, opener, creator):
+            q.close()
+        creator.unlink()
+    finally:
+        resource_tracker.register, resource_tracker.unregister = orig_register, orig_unregister
+        SlickQueue.remove(name)
+    assert calls == [], calls
+
+
+def test_fork_during_construction_does_not_deadlock():
+    """A child forked while another thread is constructing a queue can still
+    construct one. Python < 3.13 held a module lock around SharedMemory() for the
+    tracker fallback, and a fork inside that window handed the child a copy of
+    the lock that nothing would ever release, so its next construction hung."""
+    if not hasattr(os, 'fork'):
+        return
+    name = _unique("lt_fork")
+    SlickQueue.remove(name)
+    keeper = SlickQueue(name=name, size=16, element_size=8)
+    stop = threading.Event()
+
+    def churn():
+        while not stop.is_set():
+            SlickQueue(name=name, element_size=8).close()
+
+    thread = threading.Thread(target=churn, daemon=True)
+    thread.start()
+    try:
+        with warnings.catch_warnings():
+            # Python 3.12+ warns about forking a multi-threaded process
+            warnings.simplefilter("ignore", DeprecationWarning)
+            for attempt in range(20):
+                pid = os.fork()
+                if pid == 0:
+                    try:
+                        SlickQueue(name=name, element_size=8).close()
+                    except BaseException:
+                        os._exit(1)
+                    os._exit(0)
+                deadline = time.monotonic() + 5
+                while True:
+                    done, status = os.waitpid(pid, os.WNOHANG)
+                    if done:
+                        break
+                    if time.monotonic() > deadline:
+                        os.kill(pid, signal.SIGKILL)
+                        os.waitpid(pid, 0)
+                        raise AssertionError(
+                            f"forked child {attempt} hung constructing a queue")
+                    time.sleep(0.01)
+                assert os.WIFEXITED(status) and os.WEXITSTATUS(status) == 0, \
+                    f"forked child {attempt} failed (status {status})"
+    finally:
+        stop.set()
+        thread.join()
+        keeper.close()
+        keeper.unlink()
+
+
+RELATED_PROCESSES = r"""
+import multiprocessing
+import sys
+sys.path.insert(0, sys.argv[1])
+from slick_queue_py import SlickQueue
+
+
+def churn(name, n):
+    for _ in range(n):
+        SlickQueue(name=name, element_size=8).close()
+
+
+if __name__ == '__main__':
+    name = sys.argv[2]
+    SlickQueue.remove(name)
+    q = SlickQueue(name=name, size=16, element_size=8)
+    ctx = multiprocessing.get_context('spawn')
+    procs = [ctx.Process(target=churn, args=(name, 200)) for _ in range(4)]
+    for p in procs:
+        p.start()
+    for p in procs:
+        p.join()
+    q.close()
+    q.unlink()
+    # Every worker must succeed: max() let a killed one's negative code hide
+    # behind any worker that exited 0
+    codes = [p.exitcode for p in procs]
+    if codes != [0] * len(procs):
+        sys.exit(f"worker exit codes: {codes}")
+"""
+
+
+def test_related_processes_do_not_race_in_tracker():
+    """multiprocessing children share their parent's resource tracker, so before
+    Python 3.13 two of them attaching at once could send REGISTER, REGISTER,
+    UNREGISTER, UNREGISTER and make the tracker print a KeyError traceback. The
+    tracker writes to the stderr it inherited, so the scenario runs in a fresh
+    interpreter whose stderr this test captures. A spawned child re-imports the
+    script by path, hence a file rather than -c."""
+    with tempfile.TemporaryDirectory() as tmp:
+        script = Path(tmp) / "related_processes.py"
+        script.write_text(RELATED_PROCESSES)
+        result = subprocess.run(
+            [sys.executable, str(script), str(ROOT), _unique("lt_related")],
+            capture_output=True, text=True, timeout=120)
+    assert result.returncode == 0, f"script failed:\n{result.stdout}\n{result.stderr}"
+    assert "KeyError" not in result.stderr, result.stderr
+    assert "resource_tracker" not in result.stderr, result.stderr
+
+
+def run_all_tests():
+    tests = [(name, obj) for name, obj in sorted(globals().items())
+             if name.startswith("test_") and callable(obj)]
+
+    print("=" * 70)
+    print("Segment lifetime tests")
+    print("=" * 70)
+
+    passed = 0
+    failed = 0
+    for name, test_func in tests:
+        try:
+            print(f"\n[TEST] {name}...", end=" ")
+            test_func()
+            print("[PASSED]")
+            passed += 1
+        except Exception as e:
+            print("[FAILED]")
+            print(f"  Error: {e}")
+            import traceback
+            traceback.print_exc()
+            failed += 1
+
+    print("\n" + "=" * 70)
+    print(f"Results: {passed} passed, {failed} failed")
+    print("=" * 70)
+    return failed == 0
+
+
+if __name__ == '__main__':
+    success = run_all_tests()
+    sys.exit(0 if success else 1)

{slick_queue_py-2.1.0 → slick_queue_py-2.3.0}/tests/test_traits.py

@@ -9,8 +9,6 @@ from pathlib import Path
 
 sys.path.insert(0, str(Path(__file__).parent.parent))
 
-from multiprocessing.shared_memory import SharedMemory
-
 from slick_queue_py import (
     SlickQueue,
     QueueTraits,
@@ -24,6 +22,7 @@ from slick_queue_py import (
     INIT_STATE_OFFSET,
     INIT_STATE_READY,
     K_INVALID_INDEX,
+    _open_shm,
 )
 
 
@@ -475,9 +474,12 @@ def test_differently_configured_queues_coexist():
 
 
 def _make_raw_segment(name, size, element_size, magic):
-    """Build a segment by hand so a marker the library never writes can be tested."""
+    """Build a segment by hand so a marker the library never writes can be tested.
+
+    Through the library's own opener, so this handle and the queue that attaches to
+    the same name do not both claim it in the resource tracker before Python 3.13."""
     total = HEADER_SIZE + SLOT_SIZE * size + element_size * size
-    shm = SharedMemory(name=name, create=True, size=total)
+    shm = _open_shm(name, create=True, size=total)
     buf = shm.buf
     buf[:HEADER_SIZE] = bytes(HEADER_SIZE)
     struct.pack_into("<I I", buf, SIZE_OFFSET, size, element_size)
@@ -505,10 +507,7 @@ def test_attach_rejects_segment_without_marker():
             raise AssertionError("expected RuntimeError for an unmarked segment")
     finally:
         shm.close()
-        try:
-            shm.unlink()
-        except Exception:
-            pass
+        SlickQueue.remove(name)
 
 
 def test_attach_rejects_unknown_feature_bits():
@@ -526,10 +525,7 @@ def test_attach_rejects_unknown_feature_bits():
             raise AssertionError("expected RuntimeError for unknown feature bits")
     finally:
         shm.close()
-        try:
-            shm.unlink()
-        except Exception:
-            pass
+        SlickQueue.remove(name)
 
 
 # ------------------------------------------------------------------ reserve()