redis-message-queue 8.0.2__tar.gz → 8.0.3__tar.gz

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: redis-message-queue
-Version: 8.0.2
+Version: 8.0.3
 Summary: Python message queuing with Redis and message deduplication
 License: MIT
 License-File: LICENSE
@@ -68,6 +68,13 @@ with queue.process_message() as message:
 `RedisMessageQueue` itself is not a context manager. Use
 `with queue.process_message() as message:` for each message.
 
+> **Important:** In the sync API, work inside `process_message()` must be
+> synchronous. If your handler is `async def`, returns a coroutine, or returns
+> any other awaitable, use `redis_message_queue.asyncio.RedisMessageQueue`.
+> The sync context manager does not inspect the handler's return value; an
+> unawaited coroutine can be dropped while the message is acked. An ergonomic
+> callback API that detects this is planned for v8.1.
+
 ### Async quickstart
 
 ```python
@@ -119,6 +126,12 @@ All features are optional and can be enabled or disabled as needed.
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
 
+> **Important:** Handler exceptions are terminal. This library is a payload
+> queue, not a task framework: raising inside `process_message()` does not
+> requeue the message. With `enable_failed_queue=False`, the message is removed
+> from `processing`; with `enable_failed_queue=True`, it is moved to the failed
+> list.
+
 ## Configuration
 
 ### Deduplication
@@ -159,6 +172,13 @@ Avoid fallback patterns such as `lambda msg: msg.get("order_id", "")`.
 Missing fields should fail loudly instead of collapsing unrelated messages into
 one deduplication key.
 
+Deduplication markers and publish retry-safety markers are Redis TTL keys. A
+large forward step in the Redis server expiration clock during an in-call retry
+window can expire those markers before the Python-side monotonic retry budget
+elapses, allowing a duplicate publish. This is an extreme anomaly, mainly
+relevant under cluster-wide NTP step corrections while a producer is retrying
+after an ambiguous Redis write.
+
 ### Success and failure tracking
 
 ```python
@@ -234,6 +254,11 @@ queue = RedisMessageQueue(
 )
 ```
 
+> **Important:** `visibility_timeout_seconds` is a lease, not a handler runtime
+> cap. rmq never interrupts a long-running handler. If the lease expires while
+> the handler continues, another consumer can reclaim and process the same
+> message concurrently.
+
 This enables lease-based redelivery for messages left in `processing` by a crashed worker and renews the lease while a healthy long-running handler is still working.
 Tradeoffs:
 - delivery becomes at-least-once after lease expiry
@@ -258,6 +283,13 @@ The callback is **advisory** — it may fire briefly after a successful `process
 
 Without a visibility timeout, messages already moved to `processing` remain there indefinitely after a consumer crash and are not redelivered, even if the crash happened before your handler started running.
 
+Visibility deadlines use Redis server time (`TIME`), not Python process time.
+A forward step in the Redis server clock can make a live lease appear expired
+and allow premature redelivery while the original consumer is still processing;
+a backward step can delay reclaim of truly abandoned messages. Treat NTP step
+corrections on Redis hosts as a deployment risk. Prefer time-synchronization
+discipline that slews corrections rather than stepping the Redis clock.
+
 ### Ordering and multi-consumer fairness
 
 The built-in queue is a shared-pull Redis list. Successful publishes push to the
@@ -354,6 +386,11 @@ while not interrupt.is_interrupted():
 > `ValueError`. A repeated owned signal falls back to the default behavior
 > (for example, a second Ctrl+C raises `KeyboardInterrupt`). If you need multiple
 > shutdown hooks, use a single handler and fan out in your own code.
+>
+> Process-global signal ownership cannot be safely chained with task-worker
+> CLIs such as Celery, RQ, or Dramatiq. Run sibling workers in separate
+> processes, or install one top-level signal owner that calls `queue.drain()`
+> / `queue.aclose()` or sets an application stop event.
 
 There are three distinct shutdown shapes; pick the one that matches your runtime:
 
@@ -383,7 +420,10 @@ if a publish is already inside the queue instance's publish path, drain waits
 for that publish to finish before it returns; publishes that arrive after the
 drained flag is set are rejected. The drained state is local to that Python
 queue object and is not written to Redis, so constructing a fresh
-`RedisMessageQueue(...)` over the same keys remains usable.
+`RedisMessageQueue(...)` over the same keys remains usable. A separate process
+or separate queue instance against the same Redis keys is not marked drained by
+this call. For multi-process graceful shutdown, each process must drain its own
+queue instances.
 
 Drain does not cancel in-flight handlers — the caller must arrange handler
 exit through normal thread/task coordination. Returns `True` if all in-memory
@@ -391,6 +431,24 @@ pending claim IDs were recovered within the timeout; `False` if the deadline
 fired or transient Redis errors left claim IDs pending (call again to retry).
 `timeout=0` reports current state without attempting recovery.
 
+#### Abandoned in-flight messages
+
+Abandoned in-flight messages are recovered lazily. Async tasks cancelled
+without `aclose()`, or sync processes killed mid-handler, can leave the message
+and its processing/lease metadata in Redis until a later consumer claim path
+triggers visibility-timeout reclaim. With visibility timeouts enabled, this is
+the designed at-least-once recovery path: the message is delayed by the lease,
+not lost. With `visibility_timeout_seconds=None`, there is no automatic reclaim
+path. For low-visibility-timeout workloads, prefer an explicit `drain()` /
+`aclose()` during shutdown so local pending claim IDs are recovered before
+process exit.
+
+`drain()` / `aclose()` timeouts are measured with Python monotonic clocks, but
+any lease deadlines they recover were created from Redis server time. The same
+Redis-clock step caveats from
+[Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout)
+apply to when abandoned work becomes reclaimable.
+
 > **Heartbeat caveat (best-effort stop):** when `heartbeat_interval_seconds` is
 > set, the heartbeat sidecar's `stop()` is bounded but not strictly quiescent —
 > a slow renewal in flight when `process_message` exits may still write to
@@ -495,6 +553,42 @@ await client.aclose()
 For the sync Redis client, call `client.close()` during application shutdown when
 you own the client lifecycle.
 
+## Migrating from RQ / Celery / Dramatiq / taskiq
+
+redis-message-queue is a payload queue, not a task framework. It has no task
+registry, job object, result backend, scheduler, workflow canvas, callback
+graph, or handler-level retry policy. Producers publish a `str` or `dict`
+payload, and consumers decide what that payload means.
+
+The most important semantic differences from sibling task libraries are:
+
+- Handler exceptions are terminal. Raising inside `process_message()` removes
+  the message from `processing`, or moves it to the failed list when
+  `enable_failed_queue=True`; it does not requeue or retry the message.
+- `visibility_timeout_seconds` is a crash/stall recovery lease, not a runtime
+  limit. Slow handlers are not interrupted; after the lease expires another
+  consumer can process the same payload concurrently.
+- `on_event` is telemetry only. Callback exceptions are logged and emitted as
+  `RuntimeWarning`, but they do not affect ack/nack, failed-queue movement, or
+  any other message outcome. Do not use `on_event` for sagas, follow-up writes,
+  billing callbacks, or other correctness-critical work.
+- Dict payloads are JSON data, not Python call arguments. JSON does not
+  preserve every Python type: tuples become lists, and sets or custom objects
+  raise unless you encode them into JSON-native values first.
+- Process-global signal ownership cannot be safely chained with Celery, RQ, or
+  Dramatiq CLI workers. Prefer one top-level owner that calls `queue.drain()`
+  or sets an application stop event, and run sibling workers in separate
+  processes.
+
+When migrating on the same Redis deployment, prefer separate Redis DBs or hard
+namespaces. Do not point a Celery, RQ, Dramatiq, or taskiq worker at an rmq
+pending key. A sibling worker can pop the rmq stored message, fail its own
+decoder, and leave the rmq queue without that message. Also avoid custom
+`key_separator` values that synthesize another library's key namespace, such as
+using `":queue:"` with a queue name that overlaps RQ keys. rmq has no fixed
+library prefix; generated keys share the Redis DB namespace with every other
+Redis user.
+
 ## Production notes
 
 ### Fork safety and pre-fork servers
@@ -610,8 +704,10 @@ Events cover publish, dedup hits, claim/empty polls, reclaim, ack/nack,
 completed/failed cleanup, DLQ moves, heartbeat renewal, stale leases, cleanup
 and trim failures, and retry attempts. Callback exceptions are logged and
 reported with `RuntimeWarning`, but never propagate into queue operations.
-Package logs remain diagnostic; use `on_event` rather than log parsing for
-metrics.
+`on_event` is telemetry only: use it for metrics, tracing, and logging, not for
+sagas, follow-up writes, billing callbacks, or other correctness-critical
+work. Package logs remain diagnostic; use `on_event` rather than log parsing
+for metrics.
 
 ```python
 from opentelemetry import trace

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/README.md

@@ -42,6 +42,13 @@ with queue.process_message() as message:
 `RedisMessageQueue` itself is not a context manager. Use
 `with queue.process_message() as message:` for each message.
 
+> **Important:** In the sync API, work inside `process_message()` must be
+> synchronous. If your handler is `async def`, returns a coroutine, or returns
+> any other awaitable, use `redis_message_queue.asyncio.RedisMessageQueue`.
+> The sync context manager does not inspect the handler's return value; an
+> unawaited coroutine can be dropped while the message is acked. An ergonomic
+> callback API that detects this is planned for v8.1.
+
 ### Async quickstart
 
 ```python
@@ -93,6 +100,12 @@ All features are optional and can be enabled or disabled as needed.
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
 
+> **Important:** Handler exceptions are terminal. This library is a payload
+> queue, not a task framework: raising inside `process_message()` does not
+> requeue the message. With `enable_failed_queue=False`, the message is removed
+> from `processing`; with `enable_failed_queue=True`, it is moved to the failed
+> list.
+
 ## Configuration
 
 ### Deduplication
@@ -133,6 +146,13 @@ Avoid fallback patterns such as `lambda msg: msg.get("order_id", "")`.
 Missing fields should fail loudly instead of collapsing unrelated messages into
 one deduplication key.
 
+Deduplication markers and publish retry-safety markers are Redis TTL keys. A
+large forward step in the Redis server expiration clock during an in-call retry
+window can expire those markers before the Python-side monotonic retry budget
+elapses, allowing a duplicate publish. This is an extreme anomaly, mainly
+relevant under cluster-wide NTP step corrections while a producer is retrying
+after an ambiguous Redis write.
+
 ### Success and failure tracking
 
 ```python
@@ -208,6 +228,11 @@ queue = RedisMessageQueue(
 )
 ```
 
+> **Important:** `visibility_timeout_seconds` is a lease, not a handler runtime
+> cap. rmq never interrupts a long-running handler. If the lease expires while
+> the handler continues, another consumer can reclaim and process the same
+> message concurrently.
+
 This enables lease-based redelivery for messages left in `processing` by a crashed worker and renews the lease while a healthy long-running handler is still working.
 Tradeoffs:
 - delivery becomes at-least-once after lease expiry
@@ -232,6 +257,13 @@ The callback is **advisory** — it may fire briefly after a successful `process
 
 Without a visibility timeout, messages already moved to `processing` remain there indefinitely after a consumer crash and are not redelivered, even if the crash happened before your handler started running.
 
+Visibility deadlines use Redis server time (`TIME`), not Python process time.
+A forward step in the Redis server clock can make a live lease appear expired
+and allow premature redelivery while the original consumer is still processing;
+a backward step can delay reclaim of truly abandoned messages. Treat NTP step
+corrections on Redis hosts as a deployment risk. Prefer time-synchronization
+discipline that slews corrections rather than stepping the Redis clock.
+
 ### Ordering and multi-consumer fairness
 
 The built-in queue is a shared-pull Redis list. Successful publishes push to the
@@ -328,6 +360,11 @@ while not interrupt.is_interrupted():
 > `ValueError`. A repeated owned signal falls back to the default behavior
 > (for example, a second Ctrl+C raises `KeyboardInterrupt`). If you need multiple
 > shutdown hooks, use a single handler and fan out in your own code.
+>
+> Process-global signal ownership cannot be safely chained with task-worker
+> CLIs such as Celery, RQ, or Dramatiq. Run sibling workers in separate
+> processes, or install one top-level signal owner that calls `queue.drain()`
+> / `queue.aclose()` or sets an application stop event.
 
 There are three distinct shutdown shapes; pick the one that matches your runtime:
 
@@ -357,7 +394,10 @@ if a publish is already inside the queue instance's publish path, drain waits
 for that publish to finish before it returns; publishes that arrive after the
 drained flag is set are rejected. The drained state is local to that Python
 queue object and is not written to Redis, so constructing a fresh
-`RedisMessageQueue(...)` over the same keys remains usable.
+`RedisMessageQueue(...)` over the same keys remains usable. A separate process
+or separate queue instance against the same Redis keys is not marked drained by
+this call. For multi-process graceful shutdown, each process must drain its own
+queue instances.
 
 Drain does not cancel in-flight handlers — the caller must arrange handler
 exit through normal thread/task coordination. Returns `True` if all in-memory
@@ -365,6 +405,24 @@ pending claim IDs were recovered within the timeout; `False` if the deadline
 fired or transient Redis errors left claim IDs pending (call again to retry).
 `timeout=0` reports current state without attempting recovery.
 
+#### Abandoned in-flight messages
+
+Abandoned in-flight messages are recovered lazily. Async tasks cancelled
+without `aclose()`, or sync processes killed mid-handler, can leave the message
+and its processing/lease metadata in Redis until a later consumer claim path
+triggers visibility-timeout reclaim. With visibility timeouts enabled, this is
+the designed at-least-once recovery path: the message is delayed by the lease,
+not lost. With `visibility_timeout_seconds=None`, there is no automatic reclaim
+path. For low-visibility-timeout workloads, prefer an explicit `drain()` /
+`aclose()` during shutdown so local pending claim IDs are recovered before
+process exit.
+
+`drain()` / `aclose()` timeouts are measured with Python monotonic clocks, but
+any lease deadlines they recover were created from Redis server time. The same
+Redis-clock step caveats from
+[Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout)
+apply to when abandoned work becomes reclaimable.
+
 > **Heartbeat caveat (best-effort stop):** when `heartbeat_interval_seconds` is
 > set, the heartbeat sidecar's `stop()` is bounded but not strictly quiescent —
 > a slow renewal in flight when `process_message` exits may still write to
@@ -469,6 +527,42 @@ await client.aclose()
 For the sync Redis client, call `client.close()` during application shutdown when
 you own the client lifecycle.
 
+## Migrating from RQ / Celery / Dramatiq / taskiq
+
+redis-message-queue is a payload queue, not a task framework. It has no task
+registry, job object, result backend, scheduler, workflow canvas, callback
+graph, or handler-level retry policy. Producers publish a `str` or `dict`
+payload, and consumers decide what that payload means.
+
+The most important semantic differences from sibling task libraries are:
+
+- Handler exceptions are terminal. Raising inside `process_message()` removes
+  the message from `processing`, or moves it to the failed list when
+  `enable_failed_queue=True`; it does not requeue or retry the message.
+- `visibility_timeout_seconds` is a crash/stall recovery lease, not a runtime
+  limit. Slow handlers are not interrupted; after the lease expires another
+  consumer can process the same payload concurrently.
+- `on_event` is telemetry only. Callback exceptions are logged and emitted as
+  `RuntimeWarning`, but they do not affect ack/nack, failed-queue movement, or
+  any other message outcome. Do not use `on_event` for sagas, follow-up writes,
+  billing callbacks, or other correctness-critical work.
+- Dict payloads are JSON data, not Python call arguments. JSON does not
+  preserve every Python type: tuples become lists, and sets or custom objects
+  raise unless you encode them into JSON-native values first.
+- Process-global signal ownership cannot be safely chained with Celery, RQ, or
+  Dramatiq CLI workers. Prefer one top-level owner that calls `queue.drain()`
+  or sets an application stop event, and run sibling workers in separate
+  processes.
+
+When migrating on the same Redis deployment, prefer separate Redis DBs or hard
+namespaces. Do not point a Celery, RQ, Dramatiq, or taskiq worker at an rmq
+pending key. A sibling worker can pop the rmq stored message, fail its own
+decoder, and leave the rmq queue without that message. Also avoid custom
+`key_separator` values that synthesize another library's key namespace, such as
+using `":queue:"` with a queue name that overlaps RQ keys. rmq has no fixed
+library prefix; generated keys share the Redis DB namespace with every other
+Redis user.
+
 ## Production notes
 
 ### Fork safety and pre-fork servers
@@ -584,8 +678,10 @@ Events cover publish, dedup hits, claim/empty polls, reclaim, ack/nack,
 completed/failed cleanup, DLQ moves, heartbeat renewal, stale leases, cleanup
 and trim failures, and retry attempts. Callback exceptions are logged and
 reported with `RuntimeWarning`, but never propagate into queue operations.
-Package logs remain diagnostic; use `on_event` rather than log parsing for
-metrics.
+`on_event` is telemetry only: use it for metrics, tracing, and logging, not for
+sagas, follow-up writes, billing callbacks, or other correctness-critical
+work. Package logs remain diagnostic; use `on_event` rather than log parsing
+for metrics.
 
 ```python
 from opentelemetry import trace

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "redis-message-queue"
-version = "8.0.2"
+version = "8.0.3"
 description = "Python message queuing with Redis and message deduplication"
 authors = ["Elijas <4084885+Elijas@users.noreply.github.com>"]
 readme = "README.md"

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/redis_message_queue/_config.py

@@ -50,6 +50,9 @@ def is_redis_retryable_exception(exception):
             ),
         )
 
+    if isinstance(exception, redis.exceptions.ClusterError) and "TTL exhausted" in str(exception):
+        return True
+
     # 2. Explicit retryable exceptions (BusyLoadingError is a ConnectionError
     #    subclass, so it is already handled by branch 1 above)
     return isinstance(

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/redis_message_queue/_queue_key_manager.py

@@ -17,6 +17,14 @@ def validate_callable_deduplication_key(dedup_key: object, message: str | dict)
 
 
 class QueueKeyManager:
+    """Build Redis keys for one rmq queue namespace.
+
+    ``key_separator`` is part of every generated key and rmq has no fixed
+    library prefix. Do not choose a separator that overlaps another Redis task
+    library's namespace, such as ``":queue:"`` with RQ-style keys; user-chosen
+    separators interact with every Redis user on the same DB.
+    """
+
     # Logs message existence to prevent duplication.
     # Messages are marked for the duration of their lifecycle.
     _MESSAGE_DEDUPLICATION_LOG = "deduplication"

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/redis_message_queue/_redis_cluster.py

@@ -1,4 +1,5 @@
 import re
+from collections.abc import Mapping
 
 from redis.crc import key_slot
 
@@ -6,12 +7,36 @@ from redis_message_queue._exceptions import ConfigurationError
 from redis_message_queue._queue_key_manager import QueueKeyManager
 
 _HASH_TAG_PATTERN = re.compile(r"\{([^{}]+)\}")
+PLAIN_REDIS_CLUSTER_CLIENT_MESSAGE = (
+    "The provided Redis client is a plain {client_type} connected to a Redis Cluster node "
+    "('INFO cluster' reports cluster_enabled=1). Use redis.RedisCluster or "
+    "redis.asyncio.RedisCluster instead, and use a hash-tagged queue name such as '{{myqueue}}' "
+    "so all queue keys share one Redis Cluster slot."
+)
 
 
 def _redis_cluster_key_slot(key: str) -> int:
     return key_slot(key.encode("utf-8"))
 
 
+def redis_info_reports_cluster_enabled(info: object) -> bool:
+    if not isinstance(info, Mapping):
+        return False
+
+    value = info.get("cluster_enabled")
+    if value is None:
+        value = info.get(b"cluster_enabled")
+    if isinstance(value, bytes):
+        value = value.decode("utf-8", errors="replace")
+    if isinstance(value, str):
+        return value.strip() == "1"
+    return value == 1
+
+
+def plain_redis_cluster_client_error(client_type: str) -> ConfigurationError:
+    return ConfigurationError(PLAIN_REDIS_CLUSTER_CLIENT_MESSAGE.format(client_type=client_type))
+
+
 def validate_queue_keys_for_redis_cluster(
     key_manager: QueueKeyManager,
     *,

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/redis_message_queue/asyncio/redis_message_queue.py

@@ -27,7 +27,11 @@ from redis_message_queue._exceptions import (
     QueueDrainedError,
 )
 from redis_message_queue._queue_key_manager import QueueKeyManager, validate_callable_deduplication_key
-from redis_message_queue._redis_cluster import validate_queue_keys_for_redis_cluster
+from redis_message_queue._redis_cluster import (
+    plain_redis_cluster_client_error,
+    redis_info_reports_cluster_enabled,
+    validate_queue_keys_for_redis_cluster,
+)
 from redis_message_queue._stored_message import (
     ClaimedMessage,
     MessageData,
@@ -242,16 +246,32 @@ def _validate_cluster_configuration(
     client: redis.asyncio.Redis | None = None,
     gateway: AbstractRedisGateway | None = None,
     dead_letter_queue: str | None = None,
-) -> None:
-    if client is not None and isinstance(client, redis.asyncio.RedisCluster):
-        validate_queue_keys_for_redis_cluster(key_manager, dead_letter_queue=dead_letter_queue)
-        return
+) -> bool:
+    if client is not None:
+        if isinstance(client, redis.asyncio.RedisCluster):
+            validate_queue_keys_for_redis_cluster(key_manager, dead_letter_queue=dead_letter_queue)
+            return False
+        return type(client) is redis.asyncio.Redis
     if gateway is None or not gateway.is_redis_cluster:
-        return
+        return False
     validate_queue_keys_for_redis_cluster(
         key_manager,
         dead_letter_queue=gateway.dead_letter_queue,
     )
+    return False
+
+
+async def _plain_redis_client_reports_cluster(client: redis.asyncio.Redis) -> bool:
+    try:
+        info = await client.info("cluster")
+    except redis.exceptions.RedisError as exc:
+        logger.warning(
+            "Could not verify whether plain Redis client is connected to a Redis Cluster node; "
+            "trusting the provided client: %s",
+            exc,
+        )
+        return False
+    return redis_info_reports_cluster_enabled(info)
 
 
 def _derive_dead_letter_queue(name: str, key_separator: str) -> str:
@@ -508,6 +528,12 @@ class RedisMessageQueue:
         disable lease-based crash recovery; messages left in ``processing`` by a
         crashed worker are then not reclaimed automatically.
 
+        ``visibility_timeout_seconds`` is a Redis server-time lease, not a
+        handler runtime limit. Long-running handlers are not interrupted; if the
+        lease expires, another consumer can reclaim and process the same message
+        concurrently. A forward step in the Redis server clock can make a live
+        lease appear expired before that much real processing time has elapsed.
+
         ``max_delivery_count`` defaults to 10 on the built-in ``client=`` path.
         Messages reclaimed more than this many times are routed to the
         auto-derived dead-letter queue. Set it to ``None`` for unlimited
@@ -531,13 +557,19 @@ class RedisMessageQueue:
         waits for capacity before raising ``QueueBackpressureError``. ``0``
         performs a single immediate capacity check.
 
+        ``key_separator`` only controls generated Redis key names; rmq has no
+        fixed library prefix. Do not customize it to overlap another Redis
+        task library's namespace, such as ``":queue:"`` with RQ-style keys.
+
         ``interrupt`` accepts a ``BaseGracefulInterruptHandler``; pass
         ``GracefulInterruptHandler()`` for prompt Ctrl-C / termination handling
         in polling waits. ``on_heartbeat_failure`` is a zero-argument callable
         or coroutine callable invoked when lease renewal fails. ``on_event`` is
-        an async callback receiving best-effort QueueEvent lifecycle
-        notifications; callback failures are logged and converted to
-        RuntimeWarning without interrupting queue operations.
+        telemetry only: an async callback receiving best-effort QueueEvent
+        lifecycle notifications. Callback failures are logged and converted to
+        RuntimeWarning without influencing ack/nack or any other message
+        outcome. Do not use it for correctness-critical callbacks or follow-up
+        writes.
         """
         self.key = QueueKeyManager(name, key_separator=key_separator)
         if not isinstance(deduplication, bool):
@@ -639,6 +671,7 @@ class RedisMessageQueue:
         self._drained = False
         self._publish_lock = asyncio.Lock()
         self._aclose_lock = asyncio.Lock()
+        self._cluster_validation_lock = asyncio.Lock()
         self._aclose_result: bool | None = None
         self._deduplication = deduplication
         self._enable_completed_queue = enable_completed_queue
@@ -650,6 +683,7 @@ class RedisMessageQueue:
         self._heartbeat_interval_seconds = None
         self._warned_no_lease_for_heartbeat = False
         self._requires_claimed_message = False
+        self._plain_redis_cluster_probe_client: redis.asyncio.Redis | None = None
 
         if gateway is not None:
             visibility_timeout_was_configured = visibility_timeout_seconds not in (
@@ -712,7 +746,8 @@ class RedisMessageQueue:
             dead_letter_queue = (
                 _derive_dead_letter_queue(name, key_separator) if max_delivery_count is not None else None
             )
-            _validate_cluster_configuration(self.key, client=client, dead_letter_queue=dead_letter_queue)
+            if _validate_cluster_configuration(self.key, client=client, dead_letter_queue=dead_letter_queue):
+                self._plain_redis_cluster_probe_client = client
             self._heartbeat_interval_seconds = _validate_heartbeat_interval_seconds(
                 heartbeat_interval_seconds,
                 visibility_timeout_seconds,
@@ -791,6 +826,18 @@ class RedisMessageQueue:
                     stacklevel=2,
                 )
 
+    async def _ensure_plain_redis_client_is_not_cluster(self) -> None:
+        client = self._plain_redis_cluster_probe_client
+        if client is None:
+            return
+        async with self._cluster_validation_lock:
+            client = self._plain_redis_cluster_probe_client
+            if client is None:
+                return
+            if await _plain_redis_client_reports_cluster(client):
+                raise plain_redis_cluster_client_error(type(client).__name__)
+            self._plain_redis_cluster_probe_client = None
+
     async def publish(self, message: str | dict) -> bool:
         """Publish a message.
 
@@ -799,6 +846,17 @@ class RedisMessageQueue:
         ``TypeError`` to avoid silent ``json.dumps`` coercion that would
         collapse distinct keys into the same dedup key (e.g. ``{1: "x"}``
         vs ``{"1": "x"}``).
+
+        Dict payloads are JSON-encoded data, not Python object serialization.
+        JSON does not preserve every Python type: tuples become lists, raw set
+        values raise unless converted to lists before publish, and custom
+        objects raise. Plan dict payload schemas in JSON-native types only.
+
+        Deduplication and publish retry-safety markers are Redis TTL keys. A
+        large forward step in Redis server expiration time during a retry
+        window can expire those markers before the Python-side monotonic retry
+        budget elapses, allowing a duplicate publish under that extreme
+        anomaly.
         """
         async with self._publish_lock:
             if self._drained:
@@ -808,6 +866,7 @@ class RedisMessageQueue:
     async def _publish_unlocked(self, message: str | dict) -> bool:
         started_at = time.perf_counter()
         try:
+            await self._ensure_plain_redis_client_is_not_cluster()
             if not isinstance(message, (str, dict)):
                 raise TypeError(f"'message' must be a str or dict, got {type(message).__name__}")
             if isinstance(message, dict):
@@ -866,6 +925,20 @@ class RedisMessageQueue:
 
         Yields ``str`` if your client uses ``decode_responses=True``, else
         ``bytes``. Match the client setting to the type your handler expects.
+
+        Important: exceptions raised inside the ``async with`` block are
+        terminal. rmq is a payload queue, not a task framework; handler
+        exceptions do not requeue the message. With
+        ``enable_failed_queue=False``, the message is removed from
+        ``processing``; with ``enable_failed_queue=True``, it is moved to the
+        failed list.
+
+        If the task is cancelled after a message is claimed and cleanup cannot
+        run, the claimed message and lease metadata remain in Redis until a
+        later consumer claim triggers visibility-timeout reclaim. With
+        visibility timeouts enabled, this is at-least-once recovery semantics:
+        the message is delayed by the lease, not lost. Use ``aclose()`` for an
+        explicit async drain path during shutdown.
         """
         claim_started_at = time.perf_counter()
         if self._draining:
@@ -873,6 +946,7 @@ class RedisMessageQueue:
             yield None
             return
         try:
+            await self._ensure_plain_redis_client_is_not_cluster()
             claimed_message = await self._wait_for_message_and_move()
             if claimed_message is not None:
                 if not isinstance(claimed_message, (ClaimedMessage, str, bytes)):
@@ -1157,6 +1231,16 @@ class RedisMessageQueue:
         but no further claims are taken. Callers must await any
         in-flight ``process_message`` tasks separately — ``aclose()`` does
         not cancel them.
+
+        ``timeout`` is measured with the event loop's monotonic clock, but
+        visibility leases being recovered are anchored to Redis server
+        ``TIME``. A forward step in the Redis server clock can make leases
+        eligible for reclaim earlier than real elapsed handler time.
+
+        ``aclose()`` is queue-instance and process-local. A separate process,
+        or a separate ``RedisMessageQueue`` instance using the same Redis keys,
+        is not marked drained by this call. For multi-process graceful
+        shutdown, each process must drain its own queue instances.
         """
         if timeout is not None and (not isinstance(timeout, (int, float)) or isinstance(timeout, bool)):
             raise TypeError(f"'timeout' must be a number or None, got {type(timeout).__name__}")
@@ -1192,7 +1276,11 @@ class RedisMessageQueue:
             return result
 
     async def drain(self, timeout: float | None = None) -> bool:
-        """Alias of :meth:`aclose` for explicit async drain naming."""
+        """Alias of :meth:`aclose` for explicit async drain naming.
+
+        See :meth:`aclose` for process-local drain and Redis server-time lease
+        caveats.
+        """
         return await self.aclose(timeout)
 
     def __repr__(self) -> str:

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/redis_message_queue/interrupt_handler/_implementation.py

@@ -37,6 +37,12 @@ class GracefulInterruptHandler(BaseGracefulInterruptHandler):
     repeated signal for an owned handler falls back to the previous/default
     disposition so operators can still force termination (for example, a second
     Ctrl+C raises ``KeyboardInterrupt``).
+
+    Process-global signal ownership cannot be safely chained. If rmq runs in
+    the same process as Celery, RQ, or Dramatiq CLI workers, the libraries may
+    overwrite each other's SIGTERM/SIGINT handlers. Prefer one top-level signal
+    owner that calls ``queue.drain()`` or sets an application stop event, and
+    run sibling workers in separate processes.
     """
 
     _DEFAULT_SIGNALS = (

{redis_message_queue-8.0.2 → redis_message_queue-8.0.3}/redis_message_queue/redis_message_queue.py

@@ -28,7 +28,11 @@ from redis_message_queue._exceptions import (
     QueueDrainedError,
 )
 from redis_message_queue._queue_key_manager import QueueKeyManager, validate_callable_deduplication_key
-from redis_message_queue._redis_cluster import validate_queue_keys_for_redis_cluster
+from redis_message_queue._redis_cluster import (
+    plain_redis_cluster_client_error,
+    redis_info_reports_cluster_enabled,
+    validate_queue_keys_for_redis_cluster,
+)
 from redis_message_queue._redis_gateway import RedisGateway
 from redis_message_queue._stored_message import (
     ClaimedMessage,
@@ -177,8 +181,11 @@ def _validate_cluster_configuration(
     gateway: AbstractRedisGateway | None = None,
     dead_letter_queue: str | None = None,
 ) -> None:
-    if client is not None and isinstance(client, redis.RedisCluster):
-        validate_queue_keys_for_redis_cluster(key_manager, dead_letter_queue=dead_letter_queue)
+    if client is not None:
+        if isinstance(client, redis.RedisCluster):
+            validate_queue_keys_for_redis_cluster(key_manager, dead_letter_queue=dead_letter_queue)
+            return
+        _validate_plain_redis_client_not_cluster(client)
         return
     if gateway is None or not gateway.is_redis_cluster:
         return
@@ -188,6 +195,22 @@ def _validate_cluster_configuration(
     )
 
 
+def _validate_plain_redis_client_not_cluster(client: redis.Redis) -> None:
+    if type(client) is not redis.Redis:
+        return
+    try:
+        info = client.info("cluster")
+    except redis.exceptions.RedisError as exc:
+        logger.warning(
+            "Could not verify whether plain Redis client is connected to a Redis Cluster node; "
+            "trusting the provided client: %s",
+            exc,
+        )
+        return
+    if redis_info_reports_cluster_enabled(info):
+        raise plain_redis_cluster_client_error(type(client).__name__)
+
+
 def _derive_dead_letter_queue(name: str, key_separator: str) -> str:
     return f"{name}{key_separator}{_AUTO_DEAD_LETTER_QUEUE_SUFFIX}"
 
@@ -449,6 +472,12 @@ class RedisMessageQueue:
         disable lease-based crash recovery; messages left in ``processing`` by a
         crashed worker are then not reclaimed automatically.
 
+        ``visibility_timeout_seconds`` is a Redis server-time lease, not a
+        handler runtime limit. Long-running handlers are not interrupted; if the
+        lease expires, another consumer can reclaim and process the same message
+        concurrently. A forward step in the Redis server clock can make a live
+        lease appear expired before that much real processing time has elapsed.
+
         ``max_delivery_count`` defaults to 10 on the built-in ``client=`` path.
         Messages reclaimed more than this many times are routed to the
         auto-derived dead-letter queue. Set it to ``None`` for unlimited
@@ -472,12 +501,18 @@ class RedisMessageQueue:
         waits for capacity before raising ``QueueBackpressureError``. ``0``
         performs a single immediate capacity check.
 
+        ``key_separator`` only controls generated Redis key names; rmq has no
+        fixed library prefix. Do not customize it to overlap another Redis
+        task library's namespace, such as ``":queue:"`` with RQ-style keys.
+
         ``interrupt`` accepts a ``BaseGracefulInterruptHandler``; pass
         ``GracefulInterruptHandler()`` for prompt Ctrl-C / termination handling
         in polling waits. ``on_heartbeat_failure`` is a zero-argument callable
-        invoked when lease renewal fails. ``on_event`` receives best-effort
-        QueueEvent lifecycle notifications; callback failures are logged and
-        converted to RuntimeWarning without interrupting queue operations.
+        invoked when lease renewal fails. ``on_event`` is telemetry only and
+        receives best-effort QueueEvent lifecycle notifications; callback
+        failures are logged and converted to RuntimeWarning without influencing
+        ack/nack or any other message outcome. Do not use it for
+        correctness-critical callbacks or follow-up writes.
         """
         self.key = QueueKeyManager(name, key_separator=key_separator)
         if not isinstance(deduplication, bool):
@@ -752,6 +787,17 @@ class RedisMessageQueue:
         ``TypeError`` to avoid silent ``json.dumps`` coercion that would
         collapse distinct keys into the same dedup key (e.g. ``{1: "x"}``
         vs ``{"1": "x"}``).
+
+        Dict payloads are JSON-encoded data, not Python object serialization.
+        JSON does not preserve every Python type: tuples become lists, raw set
+        values raise unless converted to lists before publish, and custom
+        objects raise. Plan dict payload schemas in JSON-native types only.
+
+        Deduplication and publish retry-safety markers are Redis TTL keys. A
+        large forward step in Redis server expiration time during a retry
+        window can expire those markers before the Python-side monotonic retry
+        budget elapses, allowing a duplicate publish under that extreme
+        anomaly.
         """
         with self._publish_lock:
             if self._drained.is_set():
@@ -829,6 +875,25 @@ class RedisMessageQueue:
 
         Yields ``str`` if your client uses ``decode_responses=True``, else
         ``bytes``. Match the client setting to the type your handler expects.
+
+        Important: exceptions raised inside the ``with`` block are terminal.
+        rmq is a payload queue, not a task framework; handler exceptions do not
+        requeue the message. With ``enable_failed_queue=False``, the message is
+        removed from ``processing``; with ``enable_failed_queue=True``, it is
+        moved to the failed list.
+
+        This sync context manager only observes whether the block raises. It
+        does not inspect handler return values; if your handler returns a
+        coroutine or other awaitable, the awaitable can be dropped while the
+        message is acked. Use ``redis_message_queue.asyncio.RedisMessageQueue``
+        for async handlers. An ergonomic callback API that detects this is
+        planned for v8.1.
+
+        If the process is killed mid-handler, the claimed message and lease
+        metadata remain in Redis until a later consumer claim triggers
+        visibility-timeout reclaim. With visibility timeouts enabled, this is
+        at-least-once recovery semantics: the message is delayed by the lease,
+        not lost.
         """
         claim_started_at = time.perf_counter()
         if self._draining:
@@ -1108,10 +1173,20 @@ class RedisMessageQueue:
         ``None`` waits indefinitely, ``0`` skips the loop entirely. The
         flag is set regardless of the timeout value.
 
+        ``timeout`` is measured with Python monotonic time, but visibility
+        leases being recovered are anchored to Redis server ``TIME``. A forward
+        step in the Redis server clock can make leases eligible for reclaim
+        earlier than real elapsed handler time.
+
         Returns ``True`` if all pending claim ids were recovered (or none
         were present); ``False`` if recovery hit the deadline or a
         transient Redis error left claim ids pending.
 
+        Drain is queue-instance and process-local. A separate process, or a
+        separate ``RedisMessageQueue`` instance using the same Redis keys, is
+        not marked drained by this call. For multi-process graceful shutdown,
+        each process must drain its own queue instances.
+
         Drain does **not** cancel in-flight ``process_message`` handlers;
         the caller must coordinate handler exits via its own scheduling
         (joining threads / awaiting tasks). Heartbeat stop remains