redis-message-queue 8.0.2__tar.gz → 8.1.0__tar.gz

{redis_message_queue-8.0.2 → redis_message_queue-8.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: redis-message-queue
-Version: 8.0.2
+Version: 8.1.0
 Summary: Python message queuing with Redis and message deduplication
 License: MIT
 License-File: LICENSE
@@ -68,6 +68,13 @@ with queue.process_message() as message:
 `RedisMessageQueue` itself is not a context manager. Use
 `with queue.process_message() as message:` for each message.
 
+> **Important:** In the sync API, work inside `process_message()` must be
+> synchronous. If your handler is `async def`, returns a coroutine, or returns
+> any other awaitable, use `redis_message_queue.asyncio.RedisMessageQueue`.
+> The sync context manager does not inspect the handler's return value; an
+> unawaited coroutine can be dropped while the message is acked. An ergonomic
+> callback API that detects this is planned for v8.1.
+
 ### Async quickstart
 
 ```python
@@ -119,6 +126,12 @@ All features are optional and can be enabled or disabled as needed.
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
 
+> **Important:** Handler exceptions are terminal. This library is a payload
+> queue, not a task framework: raising inside `process_message()` does not
+> requeue the message. With `enable_failed_queue=False`, the message is removed
+> from `processing`; with `enable_failed_queue=True`, it is moved to the failed
+> list.
+
 ## Configuration
 
 ### Deduplication
@@ -159,6 +172,13 @@ Avoid fallback patterns such as `lambda msg: msg.get("order_id", "")`.
 Missing fields should fail loudly instead of collapsing unrelated messages into
 one deduplication key.
 
+Deduplication markers and publish retry-safety markers are Redis TTL keys. A
+large forward step in the Redis server expiration clock during an in-call retry
+window can expire those markers before the Python-side monotonic retry budget
+elapses, allowing a duplicate publish. This is an extreme anomaly, mainly
+relevant under cluster-wide NTP step corrections while a producer is retrying
+after an ambiguous Redis write.
+
 ### Success and failure tracking
 
 ```python
@@ -234,6 +254,11 @@ queue = RedisMessageQueue(
 )
 ```
 
+> **Important:** `visibility_timeout_seconds` is a lease, not a handler runtime
+> cap. rmq never interrupts a long-running handler. If the lease expires while
+> the handler continues, another consumer can reclaim and process the same
+> message concurrently.
+
 This enables lease-based redelivery for messages left in `processing` by a crashed worker and renews the lease while a healthy long-running handler is still working.
 Tradeoffs:
 - delivery becomes at-least-once after lease expiry
@@ -258,6 +283,13 @@ The callback is **advisory** — it may fire briefly after a successful `process
 
 Without a visibility timeout, messages already moved to `processing` remain there indefinitely after a consumer crash and are not redelivered, even if the crash happened before your handler started running.
 
+Visibility deadlines use Redis server time (`TIME`), not Python process time.
+A forward step in the Redis server clock can make a live lease appear expired
+and allow premature redelivery while the original consumer is still processing;
+a backward step can delay reclaim of truly abandoned messages. Treat NTP step
+corrections on Redis hosts as a deployment risk. Prefer time-synchronization
+discipline that slews corrections rather than stepping the Redis clock.
+
 ### Ordering and multi-consumer fairness
 
 The built-in queue is a shared-pull Redis list. Successful publishes push to the
@@ -354,6 +386,37 @@ while not interrupt.is_interrupted():
 > `ValueError`. A repeated owned signal falls back to the default behavior
 > (for example, a second Ctrl+C raises `KeyboardInterrupt`). If you need multiple
 > shutdown hooks, use a single handler and fan out in your own code.
+>
+> Process-global signal ownership cannot be safely chained with task-worker
+> CLIs such as Celery, RQ, or Dramatiq. Run sibling workers in separate
+> processes, or install one top-level signal owner that calls `queue.drain()`
+> / `queue.aclose()` or sets an application stop event.
+
+If another library owns SIGTERM/SIGINT in the same process, adapt its shutdown
+signal to rmq with a user-owned event instead of installing rmq signal handlers:
+
+```python
+import threading
+
+from redis_message_queue import EventDrivenInterruptHandler, RedisMessageQueue
+
+stop_event = threading.Event()
+interrupt = EventDrivenInterruptHandler(stop_event)
+queue = RedisMessageQueue("q", client=client, interrupt=interrupt)
+
+while not interrupt.is_interrupted():
+    with queue.process_message() as message:
+        if message is not None:
+            process(message)
+
+# In the sibling library's shutdown hook:
+stop_event.set()
+queue.drain(timeout=25)
+```
+
+The caller MUST set `stop_event` before exiting. rmq observes
+`is_interrupted()` and exits cooperatively; it does not call `sys.exit()` or
+otherwise force process shutdown.
 
 There are three distinct shutdown shapes; pick the one that matches your runtime:
 
@@ -383,7 +446,10 @@ if a publish is already inside the queue instance's publish path, drain waits
 for that publish to finish before it returns; publishes that arrive after the
 drained flag is set are rejected. The drained state is local to that Python
 queue object and is not written to Redis, so constructing a fresh
-`RedisMessageQueue(...)` over the same keys remains usable.
+`RedisMessageQueue(...)` over the same keys remains usable. A separate process
+or separate queue instance against the same Redis keys is not marked drained by
+this call. For multi-process graceful shutdown, each process must drain its own
+queue instances.
 
 Drain does not cancel in-flight handlers — the caller must arrange handler
 exit through normal thread/task coordination. Returns `True` if all in-memory
@@ -391,6 +457,24 @@ pending claim IDs were recovered within the timeout; `False` if the deadline
 fired or transient Redis errors left claim IDs pending (call again to retry).
 `timeout=0` reports current state without attempting recovery.
 
+#### Abandoned in-flight messages
+
+Abandoned in-flight messages are recovered lazily. Async tasks cancelled
+without `aclose()`, or sync processes killed mid-handler, can leave the message
+and its processing/lease metadata in Redis until a later consumer claim path
+triggers visibility-timeout reclaim. With visibility timeouts enabled, this is
+the designed at-least-once recovery path: the message is delayed by the lease,
+not lost. With `visibility_timeout_seconds=None`, there is no automatic reclaim
+path. For low-visibility-timeout workloads, prefer an explicit `drain()` /
+`aclose()` during shutdown so local pending claim IDs are recovered before
+process exit.
+
+`drain()` / `aclose()` timeouts are measured with Python monotonic clocks, but
+any lease deadlines they recover were created from Redis server time. The same
+Redis-clock step caveats from
+[Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout)
+apply to when abandoned work becomes reclaimable.
+
 > **Heartbeat caveat (best-effort stop):** when `heartbeat_interval_seconds` is
 > set, the heartbeat sidecar's `stop()` is bounded but not strictly quiescent —
 > a slow renewal in flight when `process_message` exits may still write to
@@ -495,6 +579,47 @@ await client.aclose()
 For the sync Redis client, call `client.close()` during application shutdown when
 you own the client lifecycle.
 
+## Migrating from RQ / Celery / Dramatiq / taskiq
+
+redis-message-queue is a payload queue, not a task framework. It has no task
+registry, job object, result backend, scheduler, workflow canvas, callback
+graph, or handler-level retry policy. Producers publish a `str` or `dict`
+payload, and consumers decide what that payload means.
+
+The most important semantic differences from sibling task libraries are:
+
+- Handler exceptions are terminal. Raising inside `process_message()` removes
+  the message from `processing`, or moves it to the failed list when
+  `enable_failed_queue=True`; it does not requeue or retry the message.
+- `visibility_timeout_seconds` is a crash/stall recovery lease, not a runtime
+  limit. Slow handlers are not interrupted; after the lease expires another
+  consumer can process the same payload concurrently.
+- `on_event` is telemetry only. Callback exceptions are logged and emitted as
+  `RuntimeWarning`, but they do not affect ack/nack, failed-queue movement, or
+  any other message outcome. Do not use `on_event` for sagas, follow-up writes,
+  billing callbacks, or other correctness-critical work.
+- Dict payloads are JSON data, not Python call arguments. JSON does not
+  preserve every Python type: tuples become lists, and sets or custom objects
+  raise unless you encode them into JSON-native values first.
+- Process-global signal ownership cannot be safely chained with Celery, RQ, or
+  Dramatiq CLI workers. Prefer one top-level owner that calls `queue.drain()`
+  or sets an application stop event, and run sibling workers in separate
+  processes.
+
+When migrating on the same Redis deployment, prefer separate Redis DBs or hard
+namespaces. Do not point a Celery, RQ, Dramatiq, or taskiq worker at an rmq
+pending key. A sibling worker can pop the rmq stored message, fail its own
+decoder, and leave the rmq queue without that message. Also avoid custom
+`key_separator` values that synthesize another library's key namespace, such as
+using `":queue:"` with a queue name that overlaps RQ keys. rmq has no fixed
+library prefix; generated keys share the Redis DB namespace with every other
+Redis user.
+
+Set `strict_envelope_decoding=True` if this Redis is shared with sibling task
+libraries (Celery, RQ, Dramatiq) to fail-fast on foreign payloads. With the
+default `False`, non-rmq values that do not start with the rmq envelope prefix
+remain backward-compatible raw messages and are yielded to the handler.
+
 ## Production notes
 
 ### Fork safety and pre-fork servers
@@ -610,8 +735,10 @@ Events cover publish, dedup hits, claim/empty polls, reclaim, ack/nack,
 completed/failed cleanup, DLQ moves, heartbeat renewal, stale leases, cleanup
 and trim failures, and retry attempts. Callback exceptions are logged and
 reported with `RuntimeWarning`, but never propagate into queue operations.
-Package logs remain diagnostic; use `on_event` rather than log parsing for
-metrics.
+`on_event` is telemetry only: use it for metrics, tracing, and logging, not for
+sagas, follow-up writes, billing callbacks, or other correctness-critical
+work. Package logs remain diagnostic; use `on_event` rather than log parsing
+for metrics.
 
 ```python
 from opentelemetry import trace

{redis_message_queue-8.0.2 → redis_message_queue-8.1.0}/README.md

@@ -42,6 +42,13 @@ with queue.process_message() as message:
 `RedisMessageQueue` itself is not a context manager. Use
 `with queue.process_message() as message:` for each message.
 
+> **Important:** In the sync API, work inside `process_message()` must be
+> synchronous. If your handler is `async def`, returns a coroutine, or returns
+> any other awaitable, use `redis_message_queue.asyncio.RedisMessageQueue`.
+> The sync context manager does not inspect the handler's return value; an
+> unawaited coroutine can be dropped while the message is acked. An ergonomic
+> callback API that detects this is planned for v8.1.
+
 ### Async quickstart
 
 ```python
@@ -93,6 +100,12 @@ All features are optional and can be enabled or disabled as needed.
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
 
+> **Important:** Handler exceptions are terminal. This library is a payload
+> queue, not a task framework: raising inside `process_message()` does not
+> requeue the message. With `enable_failed_queue=False`, the message is removed
+> from `processing`; with `enable_failed_queue=True`, it is moved to the failed
+> list.
+
 ## Configuration
 
 ### Deduplication
@@ -133,6 +146,13 @@ Avoid fallback patterns such as `lambda msg: msg.get("order_id", "")`.
 Missing fields should fail loudly instead of collapsing unrelated messages into
 one deduplication key.
 
+Deduplication markers and publish retry-safety markers are Redis TTL keys. A
+large forward step in the Redis server expiration clock during an in-call retry
+window can expire those markers before the Python-side monotonic retry budget
+elapses, allowing a duplicate publish. This is an extreme anomaly, mainly
+relevant under cluster-wide NTP step corrections while a producer is retrying
+after an ambiguous Redis write.
+
 ### Success and failure tracking
 
 ```python
@@ -208,6 +228,11 @@ queue = RedisMessageQueue(
 )
 ```
 
+> **Important:** `visibility_timeout_seconds` is a lease, not a handler runtime
+> cap. rmq never interrupts a long-running handler. If the lease expires while
+> the handler continues, another consumer can reclaim and process the same
+> message concurrently.
+
 This enables lease-based redelivery for messages left in `processing` by a crashed worker and renews the lease while a healthy long-running handler is still working.
 Tradeoffs:
 - delivery becomes at-least-once after lease expiry
@@ -232,6 +257,13 @@ The callback is **advisory** — it may fire briefly after a successful `process
 
 Without a visibility timeout, messages already moved to `processing` remain there indefinitely after a consumer crash and are not redelivered, even if the crash happened before your handler started running.
 
+Visibility deadlines use Redis server time (`TIME`), not Python process time.
+A forward step in the Redis server clock can make a live lease appear expired
+and allow premature redelivery while the original consumer is still processing;
+a backward step can delay reclaim of truly abandoned messages. Treat NTP step
+corrections on Redis hosts as a deployment risk. Prefer time-synchronization
+discipline that slews corrections rather than stepping the Redis clock.
+
 ### Ordering and multi-consumer fairness
 
 The built-in queue is a shared-pull Redis list. Successful publishes push to the
@@ -328,6 +360,37 @@ while not interrupt.is_interrupted():
 > `ValueError`. A repeated owned signal falls back to the default behavior
 > (for example, a second Ctrl+C raises `KeyboardInterrupt`). If you need multiple
 > shutdown hooks, use a single handler and fan out in your own code.
+>
+> Process-global signal ownership cannot be safely chained with task-worker
+> CLIs such as Celery, RQ, or Dramatiq. Run sibling workers in separate
+> processes, or install one top-level signal owner that calls `queue.drain()`
+> / `queue.aclose()` or sets an application stop event.
+
+If another library owns SIGTERM/SIGINT in the same process, adapt its shutdown
+signal to rmq with a user-owned event instead of installing rmq signal handlers:
+
+```python
+import threading
+
+from redis_message_queue import EventDrivenInterruptHandler, RedisMessageQueue
+
+stop_event = threading.Event()
+interrupt = EventDrivenInterruptHandler(stop_event)
+queue = RedisMessageQueue("q", client=client, interrupt=interrupt)
+
+while not interrupt.is_interrupted():
+    with queue.process_message() as message:
+        if message is not None:
+            process(message)
+
+# In the sibling library's shutdown hook:
+stop_event.set()
+queue.drain(timeout=25)
+```
+
+The caller MUST set `stop_event` before exiting. rmq observes
+`is_interrupted()` and exits cooperatively; it does not call `sys.exit()` or
+otherwise force process shutdown.
 
 There are three distinct shutdown shapes; pick the one that matches your runtime:
 
@@ -357,7 +420,10 @@ if a publish is already inside the queue instance's publish path, drain waits
 for that publish to finish before it returns; publishes that arrive after the
 drained flag is set are rejected. The drained state is local to that Python
 queue object and is not written to Redis, so constructing a fresh
-`RedisMessageQueue(...)` over the same keys remains usable.
+`RedisMessageQueue(...)` over the same keys remains usable. A separate process
+or separate queue instance against the same Redis keys is not marked drained by
+this call. For multi-process graceful shutdown, each process must drain its own
+queue instances.
 
 Drain does not cancel in-flight handlers — the caller must arrange handler
 exit through normal thread/task coordination. Returns `True` if all in-memory
@@ -365,6 +431,24 @@ pending claim IDs were recovered within the timeout; `False` if the deadline
 fired or transient Redis errors left claim IDs pending (call again to retry).
 `timeout=0` reports current state without attempting recovery.
 
+#### Abandoned in-flight messages
+
+Abandoned in-flight messages are recovered lazily. Async tasks cancelled
+without `aclose()`, or sync processes killed mid-handler, can leave the message
+and its processing/lease metadata in Redis until a later consumer claim path
+triggers visibility-timeout reclaim. With visibility timeouts enabled, this is
+the designed at-least-once recovery path: the message is delayed by the lease,
+not lost. With `visibility_timeout_seconds=None`, there is no automatic reclaim
+path. For low-visibility-timeout workloads, prefer an explicit `drain()` /
+`aclose()` during shutdown so local pending claim IDs are recovered before
+process exit.
+
+`drain()` / `aclose()` timeouts are measured with Python monotonic clocks, but
+any lease deadlines they recover were created from Redis server time. The same
+Redis-clock step caveats from
+[Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout)
+apply to when abandoned work becomes reclaimable.
+
 > **Heartbeat caveat (best-effort stop):** when `heartbeat_interval_seconds` is
 > set, the heartbeat sidecar's `stop()` is bounded but not strictly quiescent —
 > a slow renewal in flight when `process_message` exits may still write to
@@ -469,6 +553,47 @@ await client.aclose()
 For the sync Redis client, call `client.close()` during application shutdown when
 you own the client lifecycle.
 
+## Migrating from RQ / Celery / Dramatiq / taskiq
+
+redis-message-queue is a payload queue, not a task framework. It has no task
+registry, job object, result backend, scheduler, workflow canvas, callback
+graph, or handler-level retry policy. Producers publish a `str` or `dict`
+payload, and consumers decide what that payload means.
+
+The most important semantic differences from sibling task libraries are:
+
+- Handler exceptions are terminal. Raising inside `process_message()` removes
+  the message from `processing`, or moves it to the failed list when
+  `enable_failed_queue=True`; it does not requeue or retry the message.
+- `visibility_timeout_seconds` is a crash/stall recovery lease, not a runtime
+  limit. Slow handlers are not interrupted; after the lease expires another
+  consumer can process the same payload concurrently.
+- `on_event` is telemetry only. Callback exceptions are logged and emitted as
+  `RuntimeWarning`, but they do not affect ack/nack, failed-queue movement, or
+  any other message outcome. Do not use `on_event` for sagas, follow-up writes,
+  billing callbacks, or other correctness-critical work.
+- Dict payloads are JSON data, not Python call arguments. JSON does not
+  preserve every Python type: tuples become lists, and sets or custom objects
+  raise unless you encode them into JSON-native values first.
+- Process-global signal ownership cannot be safely chained with Celery, RQ, or
+  Dramatiq CLI workers. Prefer one top-level owner that calls `queue.drain()`
+  or sets an application stop event, and run sibling workers in separate
+  processes.
+
+When migrating on the same Redis deployment, prefer separate Redis DBs or hard
+namespaces. Do not point a Celery, RQ, Dramatiq, or taskiq worker at an rmq
+pending key. A sibling worker can pop the rmq stored message, fail its own
+decoder, and leave the rmq queue without that message. Also avoid custom
+`key_separator` values that synthesize another library's key namespace, such as
+using `":queue:"` with a queue name that overlaps RQ keys. rmq has no fixed
+library prefix; generated keys share the Redis DB namespace with every other
+Redis user.
+
+Set `strict_envelope_decoding=True` if this Redis is shared with sibling task
+libraries (Celery, RQ, Dramatiq) to fail-fast on foreign payloads. With the
+default `False`, non-rmq values that do not start with the rmq envelope prefix
+remain backward-compatible raw messages and are yielded to the handler.
+
 ## Production notes
 
 ### Fork safety and pre-fork servers
@@ -584,8 +709,10 @@ Events cover publish, dedup hits, claim/empty polls, reclaim, ack/nack,
 completed/failed cleanup, DLQ moves, heartbeat renewal, stale leases, cleanup
 and trim failures, and retry attempts. Callback exceptions are logged and
 reported with `RuntimeWarning`, but never propagate into queue operations.
-Package logs remain diagnostic; use `on_event` rather than log parsing for
-metrics.
+`on_event` is telemetry only: use it for metrics, tracing, and logging, not for
+sagas, follow-up writes, billing callbacks, or other correctness-critical
+work. Package logs remain diagnostic; use `on_event` rather than log parsing
+for metrics.
 
 ```python
 from opentelemetry import trace

{redis_message_queue-8.0.2 → redis_message_queue-8.1.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "redis-message-queue"
-version = "8.0.2"
+version = "8.1.0"
 description = "Python message queuing with Redis and message deduplication"
 authors = ["Elijas <4084885+Elijas@users.noreply.github.com>"]
 readme = "README.md"

{redis_message_queue-8.0.2 → redis_message_queue-8.1.0}/redis_message_queue/__init__.py

@@ -3,6 +3,7 @@ from redis_message_queue._event import EventOperation, EventOutcome, QueueEvent
 from redis_message_queue._exceptions import (
     CleanupFailedError,
     ConfigurationError,
+    DrainFailedError,
     GatewayContractError,
     LuaScriptError,
     MalformedStoredMessageError,
@@ -15,6 +16,7 @@ from redis_message_queue._redis_gateway import RedisGateway
 from redis_message_queue._stored_message import ClaimedMessage, MessageData
 from redis_message_queue.interrupt_handler import (
     BaseGracefulInterruptHandler,
+    EventDrivenInterruptHandler,
     GracefulInterruptHandler,
 )
 from redis_message_queue.redis_message_queue import RedisMessageQueue
@@ -25,6 +27,7 @@ __all__ = [
     "AbstractRedisGateway",
     "ClaimedMessage",
     "MessageData",
+    "EventDrivenInterruptHandler",
     "GracefulInterruptHandler",
     "BaseGracefulInterruptHandler",
     "QueueEvent",
@@ -32,6 +35,7 @@ __all__ = [
     "EventOutcome",
     "RedisMessageQueueError",
     "ConfigurationError",
+    "DrainFailedError",
     "GatewayContractError",
     "LuaScriptError",
     "MalformedStoredMessageError",

{redis_message_queue-8.0.2 → redis_message_queue-8.1.0}/redis_message_queue/_config.py

@@ -50,6 +50,9 @@ def is_redis_retryable_exception(exception):
             ),
         )
 
+    if isinstance(exception, redis.exceptions.ClusterError) and "TTL exhausted" in str(exception):
+        return True
+
     # 2. Explicit retryable exceptions (BusyLoadingError is a ConnectionError
     #    subclass, so it is already handled by branch 1 above)
     return isinstance(

{redis_message_queue-8.0.2 → redis_message_queue-8.1.0}/redis_message_queue/_event.py

@@ -24,6 +24,7 @@ class EventOperation(StrEnum):
     TRIM_FAILED = "trim_failed"
     RETRY_ATTEMPT = "retry_attempt"
     RETRY_EXHAUSTED = "retry_exhausted"
+    DRAIN = "drain"
 
 
 class EventOutcome(StrEnum):
@@ -32,6 +33,7 @@ class EventOutcome(StrEnum):
     SUCCESS = "success"
     FAILURE = "failure"
     SKIPPED = "skipped"
+    START = "start"
 
 
 @dataclass(frozen=True)
@@ -68,3 +70,7 @@ class QueueEvent:
     the actual exception object when one was raised; pass to OpenTelemetry
     `span.record_exception(...)` for full trace attribution
     """
+    timeout_seconds: float | None = None
+    """the caller-requested timeout for drain operations, when applicable"""
+    pending_claim_ids: int | None = None
+    """number of unresolved pending claim ids for drain operations, when applicable"""

redis_message_queue-8.1.0/redis_message_queue/_exceptions.py

@@ -0,0 +1,149 @@
+import redis.exceptions
+
+
+class RedisMessageQueueError(Exception):
+    """Base class for redis-message-queue specific errors."""
+
+    def __init__(
+        self,
+        *args: object,
+        queue: str | None = None,
+        message_id: str | None = None,
+        operation: str | None = None,
+    ) -> None:
+        super().__init__(*args)
+        self.queue = queue
+        self.message_id = message_id
+        self.operation = operation
+
+
+class ConfigurationError(RedisMessageQueueError, ValueError):
+    """Bad parameter values or combinations."""
+
+
+class GatewayContractError(RedisMessageQueueError, TypeError):
+    """Custom gateway returned wrong type or violated contract."""
+
+
+class LuaScriptError(redis.exceptions.ResponseError, RedisMessageQueueError):
+    """A Lua script returned an unexpected error_reply."""
+
+    def __init__(
+        self,
+        *args: object,
+        queue: str | None = None,
+        message_id: str | None = None,
+        operation: str | None = None,
+    ) -> None:
+        RedisMessageQueueError.__init__(
+            self,
+            *args,
+            queue=queue,
+            message_id=message_id,
+            operation=operation,
+        )
+
+
+class CleanupFailedError(RedisMessageQueueError):
+    """Cleanup after handler completion failed."""
+
+
+class DrainFailedError(RedisMessageQueueError):
+    """Wraps a non-RMQ exception caught during drain pending-claim recovery.
+
+    drain() returns False as the bool result; this exception carries
+    F7 context (queue, operation="drain") into the drain/failure event
+    payload so users diagnosing drain incidents via on_event see the
+    same structured attrs as elsewhere.
+    """
+
+
+class MalformedStoredMessageError(RedisMessageQueueError):
+    """Stored value is not a valid RMQ envelope for the configured decode mode."""
+
+
+class QueueBackpressureError(RedisMessageQueueError):
+    """Publish rejected because the pending queue is at its configured limit."""
+
+    _REMEDIATION = (
+        "consider increasing `max_pending_length`, switching to "
+        "`pending_overload_policy='block'`, or adding consumer capacity."
+    )
+
+    def __init__(
+        self,
+        *args: object,
+        queue: str | None = None,
+        message_id: str | None = None,
+        operation: str | None = None,
+    ) -> None:
+        message = "Pending queue reached its configured limit" if not args else args[0]
+        if not isinstance(message, str) or len(args) > 1:
+            super().__init__(*args, queue=queue, message_id=message_id, operation=operation)
+            return
+        if self._REMEDIATION not in message:
+            message = f"{message}; {self._REMEDIATION}"
+        super().__init__(message, queue=queue, message_id=message_id, operation=operation)
+
+
+class QueueDrainedError(RedisMessageQueueError):
+    """Raised when publish() is called after drain() or aclose()."""
+
+
+class RetryBudgetExhaustedError(redis.exceptions.RedisError, RedisMessageQueueError):
+    """Tenacity retry budget exhausted; underlying redis-py exception is .__cause__."""
+
+    _REMEDIATION = (
+        "verify Redis connectivity and consider increasing `retry_budget_seconds` if transient failures are expected."
+    )
+
+    def __init__(
+        self,
+        *args: object,
+        queue: str | None = None,
+        message_id: str | None = None,
+        operation: str | None = None,
+    ) -> None:
+        message = "Redis retry budget exhausted" if not args else args[0]
+        if not isinstance(message, str) or len(args) > 1:
+            RedisMessageQueueError.__init__(
+                self,
+                *args,
+                queue=queue,
+                message_id=message_id,
+                operation=operation,
+            )
+            return
+        if self._REMEDIATION not in message:
+            message = f"{message}; {self._REMEDIATION}"
+        RedisMessageQueueError.__init__(
+            self,
+            message,
+            queue=queue,
+            message_id=message_id,
+            operation=operation,
+        )
+
+
+def _set_exception_context(
+    exc: BaseException,
+    *,
+    queue: str | None = None,
+    message_id: str | None = None,
+    operation: str | None = None,
+) -> None:
+    if not isinstance(exc, RedisMessageQueueError):
+        return
+    if queue is not None:
+        exc.queue = queue
+    if message_id is not None:
+        exc.message_id = message_id
+    if operation is not None:
+        exc.operation = operation
+
+
+def wrap_lua_response_error(exc: redis.exceptions.ResponseError) -> LuaScriptError | None:
+    message = str(exc)
+    if message.startswith("WRONGTYPE ") or message.startswith("OOM during publish;"):
+        return LuaScriptError(message)
+    return None

{redis_message_queue-8.0.2 → redis_message_queue-8.1.0}/redis_message_queue/_queue_key_manager.py

@@ -17,6 +17,14 @@ def validate_callable_deduplication_key(dedup_key: object, message: str | dict)
 
 
 class QueueKeyManager:
+    """Build Redis keys for one rmq queue namespace.
+
+    ``key_separator`` is part of every generated key and rmq has no fixed
+    library prefix. Do not choose a separator that overlaps another Redis task
+    library's namespace, such as ``":queue:"`` with RQ-style keys; user-chosen
+    separators interact with every Redis user on the same DB.
+    """
+
     # Logs message existence to prevent duplication.
     # Messages are marked for the duration of their lifecycle.
     _MESSAGE_DEDUPLICATION_LOG = "deduplication"