redis-message-queue 8.2.9__tar.gz → 8.3.0__tar.gz

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: redis-message-queue
-Version: 8.2.9
+Version: 8.3.0
 Summary: Python message queuing with Redis and message deduplication
 Project-URL: Homepage, https://github.com/Elijas/redis-message-queue
 Project-URL: Repository, https://github.com/Elijas/redis-message-queue
@@ -34,10 +34,10 @@ Description-Content-Type: text/markdown
 **Lightweight Python message queuing with Redis and built-in publish-side deduplication.** Deduplicate publishes within a TTL window, with optional crash recovery — across any number of producers and consumers.
 
 ```bash
-pip install "redis-message-queue>=8.2.9,<9.0.0"
+pip install "redis-message-queue>=8.3.0,<9.0.0"
 ```
 
-Requires Redis server >= 6.2.
+Requires Python >= 3.12 and Redis server >= 6.2.
 
 ## Quickstart
 
@@ -764,22 +764,35 @@ work. Package logs remain diagnostic; use `on_event` rather than log parsing
 for metrics.
 
 ```python
-from opentelemetry import trace
-from prometheus_client import Counter
 from redis_message_queue import QueueEvent, RedisMessageQueue
 
-events_total = Counter(
-    "rmq_events_total",
-    "redis-message-queue lifecycle events",
-    ["queue", "operation", "outcome", "exception_type"],
+try:
+    from opentelemetry import trace
+except ImportError:
+    trace = None
+
+try:
+    from prometheus_client import Counter
+except ImportError:
+    Counter = None
+
+events_total = (
+    Counter(
+        "rmq_events_total",
+        "redis-message-queue lifecycle events",
+        ["queue", "operation", "outcome", "exception_type"],
+    )
+    if Counter is not None
+    else None
 )
 SPAN_SINK_TRUSTED = False
 
 def observe(event: QueueEvent) -> None:
-    events_total.labels(
-        event.queue, event.operation, event.outcome, event.exception_type or ""
-    ).inc()
-    if event.error is not None and SPAN_SINK_TRUSTED:
+    if events_total is not None:
+        events_total.labels(
+            event.queue, event.operation, event.outcome, event.exception_type or ""
+        ).inc()
+    if event.error is not None and SPAN_SINK_TRUSTED and trace is not None:
         trace.get_current_span().record_exception(event.error)
 
 queue = RedisMessageQueue("jobs", client=client, on_event=observe)
@@ -830,6 +843,13 @@ Callbacks fire inline:
   copies the context present when the heartbeat was started, so contextvars and
   OpenTelemetry spans bound at handler entry are visible.
 
+> **Warning:** Because callbacks fire inline and may run while an internal
+> publish/drain lock is held, an `on_event` callback must not call back into the
+> same queue instance's `publish()`, `drain()`, `close()` (sync), or `aclose()`
+> (async). Those locks are non-reentrant, so re-entering deadlocks and wedges
+> the caller permanently. Re-entering a *different* queue instance, or scheduling
+> the follow-up work outside the callback, is safe.
+
 #### Event timing vs. Redis commit
 
 Most events are post-commit, emitted after the Redis command or Lua script
@@ -889,6 +909,17 @@ The following operations have no `on_event` surface by design:
   ack/remove, move-to-completed/failed, and lease renewal collapse into the
   terminal operation's failure event. There is no per-attempt event for those
   paths.
+- **Claim cache-replay after a lost reply:** a visibility-timeout claim can
+  commit server-side — dead-lettering a poison message (`dlq`) or reclaiming an
+  expired lease (`claim_reclaim`) before claiming the next live message — and
+  then lose its reply (for example, a dropped connection). The claim loop
+  retries the same claim ID and hits the `claim_result` cache-replay, which
+  re-asserts the lease and returns the stored claim but does not re-run those
+  side effects, so their `claim_reclaim` / `dlq` event payloads are not
+  re-emitted. Queue state stays correct (the poison message stays
+  dead-lettered, the live message is claimed exactly once); only telemetry for
+  the lost-reply attempt is dropped. Reconcile poison-message alerting against
+  `LLEN {name}::dlq` rather than the `on_event` stream alone.
 
 The public exception hierarchy is rooted at `RedisMessageQueueError`. The
 current exported queue-owned exception classes are:

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/README.md

@@ -11,10 +11,10 @@
 **Lightweight Python message queuing with Redis and built-in publish-side deduplication.** Deduplicate publishes within a TTL window, with optional crash recovery — across any number of producers and consumers.
 
 ```bash
-pip install "redis-message-queue>=8.2.9,<9.0.0"
+pip install "redis-message-queue>=8.3.0,<9.0.0"
 ```
 
-Requires Redis server >= 6.2.
+Requires Python >= 3.12 and Redis server >= 6.2.
 
 ## Quickstart
 
@@ -741,22 +741,35 @@ work. Package logs remain diagnostic; use `on_event` rather than log parsing
 for metrics.
 
 ```python
-from opentelemetry import trace
-from prometheus_client import Counter
 from redis_message_queue import QueueEvent, RedisMessageQueue
 
-events_total = Counter(
-    "rmq_events_total",
-    "redis-message-queue lifecycle events",
-    ["queue", "operation", "outcome", "exception_type"],
+try:
+    from opentelemetry import trace
+except ImportError:
+    trace = None
+
+try:
+    from prometheus_client import Counter
+except ImportError:
+    Counter = None
+
+events_total = (
+    Counter(
+        "rmq_events_total",
+        "redis-message-queue lifecycle events",
+        ["queue", "operation", "outcome", "exception_type"],
+    )
+    if Counter is not None
+    else None
 )
 SPAN_SINK_TRUSTED = False
 
 def observe(event: QueueEvent) -> None:
-    events_total.labels(
-        event.queue, event.operation, event.outcome, event.exception_type or ""
-    ).inc()
-    if event.error is not None and SPAN_SINK_TRUSTED:
+    if events_total is not None:
+        events_total.labels(
+            event.queue, event.operation, event.outcome, event.exception_type or ""
+        ).inc()
+    if event.error is not None and SPAN_SINK_TRUSTED and trace is not None:
         trace.get_current_span().record_exception(event.error)
 
 queue = RedisMessageQueue("jobs", client=client, on_event=observe)
@@ -807,6 +820,13 @@ Callbacks fire inline:
   copies the context present when the heartbeat was started, so contextvars and
   OpenTelemetry spans bound at handler entry are visible.
 
+> **Warning:** Because callbacks fire inline and may run while an internal
+> publish/drain lock is held, an `on_event` callback must not call back into the
+> same queue instance's `publish()`, `drain()`, `close()` (sync), or `aclose()`
+> (async). Those locks are non-reentrant, so re-entering deadlocks and wedges
+> the caller permanently. Re-entering a *different* queue instance, or scheduling
+> the follow-up work outside the callback, is safe.
+
 #### Event timing vs. Redis commit
 
 Most events are post-commit, emitted after the Redis command or Lua script
@@ -866,6 +886,17 @@ The following operations have no `on_event` surface by design:
   ack/remove, move-to-completed/failed, and lease renewal collapse into the
   terminal operation's failure event. There is no per-attempt event for those
   paths.
+- **Claim cache-replay after a lost reply:** a visibility-timeout claim can
+  commit server-side — dead-lettering a poison message (`dlq`) or reclaiming an
+  expired lease (`claim_reclaim`) before claiming the next live message — and
+  then lose its reply (for example, a dropped connection). The claim loop
+  retries the same claim ID and hits the `claim_result` cache-replay, which
+  re-asserts the lease and returns the stored claim but does not re-run those
+  side effects, so their `claim_reclaim` / `dlq` event payloads are not
+  re-emitted. Queue state stays correct (the poison message stays
+  dead-lettered, the live message is claimed exactly once); only telemetry for
+  the lost-reply attempt is dropped. Reconcile poison-message alerting against
+  `LLEN {name}::dlq` rather than the `on_event` stream alone.
 
 The public exception hierarchy is rooted at `RedisMessageQueueError`. The
 current exported queue-owned exception classes are:

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "redis-message-queue"
-version = "8.2.9"
+version = "8.3.0"
 description = "Python message queuing with Redis and message deduplication"
 authors = [{ name = "Elijas", email = "4084885+Elijas@users.noreply.github.com" }]
 readme = "README.md"
@@ -48,7 +48,7 @@ default-groups = ["dev", "test"]
 ##############################
 
 [tool.bumpversion]
-current_version = "8.2.9"
+current_version = "8.3.0"
 parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
 serialize = ["{major}.{minor}.{patch}"]
 search = "{current_version}"

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/redis_message_queue/_config.py

@@ -358,7 +358,13 @@ def validate_pending_backpressure_parameters(
         raise ConfigurationError(
             "drop_oldest requires max_pending_length to be set. "
             "Use a positive max_pending_length to define what can be dropped, or use "
-            "pending_overload_policy='raise' or 'block' for unbounded queues."
+            "pending_overload_policy='raise' for an unbounded queue."
+        )
+    if pending_overload_policy == "block" and max_pending_length is None:
+        raise ConfigurationError(
+            "block requires max_pending_length to be set. "
+            "Use a positive max_pending_length to define the threshold to block on, or use "
+            "pending_overload_policy='raise' for an unbounded queue."
         )
     if pending_overload_policy == "drop_oldest" and (deduplication or get_deduplication_key_configured):
         raise ConfigurationError(
@@ -857,35 +863,53 @@ end
 -- Cap at 100 to bound Lua execution time (Redis blocks during scripts).
 -- With a single consumer polling at default interval, 1000 expired leases drain in ~2.5s.
 local expired = redis.call('ZRANGEBYSCORE', KEYS[3], '-inf', now_ms, 'LIMIT', 0, 100)
-local to_requeue = {}
 local reclaimed_events = {}
 for i = #expired, 1, -1 do
-    local expired_lease_token = redis.call('HGET', KEYS[4], expired[i])
-    redis.call('ZREM', KEYS[3], expired[i])
-    redis.call('HDEL', KEYS[4], expired[i])
-    if expired_lease_token then
-        local claim_result_key = redis.call('HGET', KEYS[9], expired_lease_token)
-        if claim_result_key then
-            -- Use pcall: in Redis Cluster, claim_result_key was read from KEYS[9] (claim_result_refs)
-            -- and is therefore not in the declared EVAL KEYS[] set. Cluster may reject the DEL;
-            -- TTL on the claim_result string (PX visibility_timeout_seconds) bounds the orphan.
-            redis.pcall('DEL', claim_result_key)
-            redis.call('HDEL', KEYS[9], expired_lease_token)
+    local stored = expired[i]
+    local expired_lease_token = redis.call('HGET', KEYS[4], stored)
+
+    -- Durable-before-destructive (mirror RETURN_MESSAGE_TO_PENDING): requeue to
+    -- pending BEFORE removing from processing or deleting lease metadata. If this
+    -- write fails, the message remains in processing with its metadata intact for
+    -- a future reclaim attempt.
+    redis.call('RPUSH', KEYS[1], stored)
+    if redis.call('LREM', KEYS[2], 1, stored) == 1 then
+        redis.call('ZREM', KEYS[3], stored)
+        redis.call('HDEL', KEYS[4], stored)
+        if expired_lease_token then
+            local claim_result_key = redis.call('HGET', KEYS[9], expired_lease_token)
+            if claim_result_key then
+                -- Use pcall: in Redis Cluster, claim_result_key was read from KEYS[9] (claim_result_refs)
+                -- and is therefore not in the declared EVAL KEYS[] set. Cluster may reject the DEL;
+                -- TTL on the claim_result string (PX visibility_timeout_seconds) bounds the orphan.
+                redis.pcall('DEL', claim_result_key)
+                redis.call('HDEL', KEYS[9], expired_lease_token)
+            end
+            local claim_id = redis.call('HGET', KEYS[11], expired_lease_token)
+            if claim_id then
+                redis.call('HDEL', KEYS[10], claim_id)
+                redis.call('HDEL', KEYS[11], expired_lease_token)
+            end
         end
-        local claim_id = redis.call('HGET', KEYS[11], expired_lease_token)
-        if claim_id then
-            redis.call('HDEL', KEYS[10], claim_id)
-            redis.call('HDEL', KEYS[11], expired_lease_token)
+        local delivery_count = redis.call('HGET', KEYS[6], stored)
+        table.insert(reclaimed_events, {redis_message_queue_message_id(stored), tostring(delivery_count or '0')})
+    else
+        redis.call('LREM', KEYS[1], 1, stored)
+        redis.call('ZREM', KEYS[3], stored)
+        redis.call('HDEL', KEYS[4], stored)
+        if expired_lease_token then
+            local claim_result_key = redis.call('HGET', KEYS[9], expired_lease_token)
+            if claim_result_key then
+                redis.pcall('DEL', claim_result_key)
+                redis.call('HDEL', KEYS[9], expired_lease_token)
+            end
+            local claim_id = redis.call('HGET', KEYS[11], expired_lease_token)
+            if claim_id then
+                redis.call('HDEL', KEYS[10], claim_id)
+                redis.call('HDEL', KEYS[11], expired_lease_token)
+            end
         end
     end
-    if redis.call('LREM', KEYS[2], 1, expired[i]) == 1 then
-        table.insert(to_requeue, expired[i])
-        local delivery_count = redis.call('HGET', KEYS[6], expired[i])
-        table.insert(reclaimed_events, {redis_message_queue_message_id(expired[i]), tostring(delivery_count or '0')})
-    end
-end
-if #to_requeue > 0 then
-    redis.call('RPUSH', KEYS[1], unpack(to_requeue))
 end
 local dead_lettered_events = {}
 local claim_store_failed_sentinel = string.char(0) .. '__rmq_claim_store_failed__'

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/redis_message_queue/_redis_gateway.py

@@ -402,6 +402,15 @@ class RedisGateway(AbstractRedisGateway):
     def is_redis_cluster(self) -> bool:
         return isinstance(self._redis_client, redis.RedisCluster)
 
+    def _raise_if_drop_oldest_deduplicated_publish(self) -> None:
+        if self._pending_overload_policy == "drop_oldest":
+            raise ConfigurationError(
+                "'pending_overload_policy=drop_oldest' cannot be used with RedisGateway.publish_message "
+                "because dropped messages leave their deduplication keys in Redis, causing future publishes "
+                "of the same payload to be silently suppressed. Use add_message for non-deduplicated lossy "
+                "queues, or use 'raise' or 'block' for deduplicated publishes."
+            )
+
     def publish_message(self, queue: str, message: str, dedup_key: str) -> bool:
         if not isinstance(dedup_key, str):
             raise TypeError(f"'dedup_key' must be a str, got {type(dedup_key).__name__}")
@@ -410,6 +419,7 @@ class RedisGateway(AbstractRedisGateway):
                 "'dedup_key' must be a non-empty string; "
                 "an empty key would create a bare-prefix Redis marker that silently suppresses unrelated messages"
             )
+        self._raise_if_drop_oldest_deduplicated_publish()
         stored_message = encode_stored_message(message)
         message_id = extract_stored_message_id(stored_message)
         operation_id = uuid.uuid4().hex

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/redis_message_queue/asyncio/_redis_gateway.py

@@ -381,6 +381,15 @@ class RedisGateway(AbstractRedisGateway):
     def is_redis_cluster(self) -> bool:
         return isinstance(self._redis_client, redis.asyncio.RedisCluster)
 
+    def _raise_if_drop_oldest_deduplicated_publish(self) -> None:
+        if self._pending_overload_policy == "drop_oldest":
+            raise ConfigurationError(
+                "'pending_overload_policy=drop_oldest' cannot be used with RedisGateway.publish_message "
+                "because dropped messages leave their deduplication keys in Redis, causing future publishes "
+                "of the same payload to be silently suppressed. Use add_message for non-deduplicated lossy "
+                "queues, or use 'raise' or 'block' for deduplicated publishes."
+            )
+
     async def publish_message(self, queue: str, message: str, dedup_key: str) -> bool:
         if not isinstance(dedup_key, str):
             raise TypeError(f"'dedup_key' must be a str, got {type(dedup_key).__name__}")
@@ -389,6 +398,7 @@ class RedisGateway(AbstractRedisGateway):
                 "'dedup_key' must be a non-empty string; "
                 "an empty key would create a bare-prefix Redis marker that silently suppresses unrelated messages"
             )
+        self._raise_if_drop_oldest_deduplicated_publish()
         stored_message = encode_stored_message(message)
         message_id = extract_stored_message_id(stored_message)
         operation_id = uuid.uuid4().hex

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/redis_message_queue/asyncio/redis_message_queue.py

@@ -511,6 +511,29 @@ class _LeaseHeartbeat:
                     stacklevel=1,
                 )
 
+    async def _report_renewal_failure(self, exc: BaseException) -> None:
+        if self._stop_event.is_set():
+            return
+        logger.exception("Failed to renew message lease")
+        await self._emit(
+            "lease_renew_failed",
+            "failure",
+            message_id=self._message_id,
+            lease_token_hash=self._lease_token_hash,
+            exception_type=type(exc).__name__,
+            error=exc,
+        )
+        with warnings.catch_warnings():
+            warnings.simplefilter("always", RuntimeWarning)
+            warnings.warn(
+                "Failed to renew message lease "
+                f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
+                "when the visibility timeout expires",
+                RuntimeWarning,
+                stacklevel=1,
+            )
+        await self._invoke_failure_callback()
+
     async def _run(self) -> None:
         try:
             while True:
@@ -531,30 +554,14 @@ class _LeaseHeartbeat:
                             f"gateway.renew_message_lease() must return bool, got {type(renewed).__name__}. "
                             "See AbstractRedisGateway.renew_message_lease for the full contract."
                         )
-                except asyncio.CancelledError:
-                    raise
+                except asyncio.CancelledError as exc:
+                    current_task = asyncio.current_task()
+                    if self._stop_event.is_set() or (current_task is not None and current_task.cancelling() > 0):
+                        raise
+                    await self._report_renewal_failure(exc)
+                    return
                 except Exception as exc:
-                    if self._stop_event.is_set():
-                        return
-                    logger.exception("Failed to renew message lease")
-                    await self._emit(
-                        "lease_renew_failed",
-                        "failure",
-                        message_id=self._message_id,
-                        lease_token_hash=self._lease_token_hash,
-                        exception_type=type(exc).__name__,
-                        error=exc,
-                    )
-                    with warnings.catch_warnings():
-                        warnings.simplefilter("always", RuntimeWarning)
-                        warnings.warn(
-                            "Failed to renew message lease "
-                            f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
-                            "when the visibility timeout expires",
-                            RuntimeWarning,
-                            stacklevel=1,
-                        )
-                    await self._invoke_failure_callback()
+                    await self._report_renewal_failure(exc)
                     return
                 if not renewed:
                     await self._emit(
@@ -660,6 +667,8 @@ class RedisMessageQueue:
         ``"drop_oldest"`` evicts the oldest pending message before enqueueing
         the new one. ``"drop_oldest"`` requires ``max_pending_length`` and is
         not compatible with deduplication or ``max_delivery_count``.
+        ``"block"`` also requires ``max_pending_length`` (the threshold to
+        block on); only the default ``"raise"`` operates on an unbounded queue.
 
         ``pending_overload_block_timeout_seconds`` bounds how long ``"block"``
         waits for capacity before raising ``QueueBackpressureError``. ``0``
@@ -672,12 +681,23 @@ class RedisMessageQueue:
         ``interrupt`` accepts a ``BaseGracefulInterruptHandler``; pass
         ``GracefulInterruptHandler()`` for prompt Ctrl-C / termination handling
         in polling waits. ``on_heartbeat_failure`` is a zero-argument callable
-        or coroutine callable invoked when lease renewal fails. ``on_event`` is
-        telemetry only: a callable returning an awaitable and receiving
+        or coroutine callable invoked when lease renewal fails. It runs on the
+        heartbeat's background thread (sync queue) or on the event loop (async
+        queue); in the async queue it MUST NOT block (no ``time.sleep``,
+        blocking I/O, or CPU-heavy work; use ``await``), because a blocking
+        callback freezes the event loop, delaying lease renewal for every other
+        in-flight message (whose leases can then expire and be reclaimed) and
+        stalling ``aclose()``. Keep it quick and offload slow work yourself.
+        ``on_event`` is telemetry only: a callable returning an awaitable and
+        receiving
         best-effort QueueEvent lifecycle notifications. Callback failures are
         logged and converted to RuntimeWarning without influencing ack/nack or
         any other message outcome. Do not use it for correctness-critical
-        callbacks or follow-up writes.
+        callbacks or follow-up writes. ``on_event`` is awaited inline in the
+        current asyncio task and may execute while an internal publish/drain
+        lock is held, so the callback must not call back into the same queue
+        instance's ``publish()``, ``drain()``, or ``aclose()``; that lock is
+        non-reentrant, so re-entering deadlocks the caller permanently.
         """
         self.key = QueueKeyManager(name, key_separator=key_separator)
         if not isinstance(deduplication, bool):
@@ -762,6 +782,18 @@ class RedisMessageQueue:
             deduplication=deduplication,
             get_deduplication_key=get_deduplication_key,
         )
+        if gateway is not None:
+            # Before the generic validator so gateway-incompat wins over the drop_oldest runaround; non-default only.
+            if pending_overload_policy != "raise":
+                raise ConfigurationError(
+                    "'pending_overload_policy' cannot be provided alongside 'gateway'."
+                    " Configure publish backpressure on the gateway directly instead."
+                )
+            if pending_overload_block_timeout_seconds != DEFAULT_PENDING_OVERLOAD_BLOCK_TIMEOUT_SECONDS:
+                raise ConfigurationError(
+                    "'pending_overload_block_timeout_seconds' cannot be provided alongside 'gateway'."
+                    " Configure publish backpressure on the gateway directly instead."
+                )
         validate_pending_backpressure_parameters(
             max_pending_length,
             pending_overload_policy,
@@ -1179,7 +1211,7 @@ class RedisMessageQueue:
                             "visibility timeouts are in use."
                         )
                         logger.warning(no_lease_token_warning)
-                        warnings.warn(no_lease_token_warning, RuntimeWarning, stacklevel=2)
+                        _warn_runtime_warning(no_lease_token_warning, stacklevel=2)
 
                 if lease_token is None and self._requires_claimed_message:
                     raise GatewayContractError(
@@ -1232,11 +1264,14 @@ class RedisMessageQueue:
         )
 
         lease_heartbeat = self._build_lease_heartbeat(stored_message, lease_token, message_id, lease_token_hash)
-        if lease_heartbeat is not None:
-            lease_heartbeat.start()
         finished_without_error = False
         processing_started_at = time.perf_counter()
         try:
+            # start() must be inside this try so its finally always stop()s the
+            # heartbeat; an exception in the start()->yield window would
+            # otherwise orphan it.
+            if lease_heartbeat is not None:
+                lease_heartbeat.start()
             yield message  # type: ignore
         except BaseException as exc:
             skip_cleanup = _should_skip_message_cleanup(exc)
@@ -1497,7 +1532,7 @@ class RedisMessageQueue:
         timeout_seconds = None if timeout is None else float(timeout)
         async with self._aclose_lock:
             cleanup_lease_counter = getattr(self._redis, "_cleanup_drained_lease_token_counter", None)
-            if self._aclose_result is not None:
+            if self._aclose_result is True:
                 pending_claim_ids = self._pending_claim_ids_count()
                 if pending_claim_ids:
                     self._aclose_result = None

{redis_message_queue-8.2.9 → redis_message_queue-8.3.0}/redis_message_queue/redis_message_queue.py

@@ -482,6 +482,29 @@ class _LeaseHeartbeat:
                     stacklevel=1,
                 )
 
+    def _report_renewal_failure(self, exc: BaseException) -> None:
+        if self._stop_event.is_set():
+            return
+        logger.exception("Failed to renew message lease")
+        self._emit(
+            "lease_renew_failed",
+            "failure",
+            message_id=self._message_id,
+            lease_token_hash=self._lease_token_hash,
+            exception_type=type(exc).__name__,
+            error=exc,
+        )
+        with warnings.catch_warnings():
+            warnings.simplefilter("always", RuntimeWarning)
+            warnings.warn(
+                "Failed to renew message lease "
+                f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
+                "when the visibility timeout expires",
+                RuntimeWarning,
+                stacklevel=1,
+            )
+        self._invoke_failure_callback()
+
     def _run(self) -> None:
         # No explicit _is_interrupted() check here. Heartbeat lifetime is owned
         # by process_message, which sets _stop_event in its finally block on any
@@ -498,28 +521,11 @@ class _LeaseHeartbeat:
                         f"gateway.renew_message_lease() must return bool, got {type(renewed).__name__}. "
                         "See AbstractRedisGateway.renew_message_lease for the full contract."
                     )
+            except asyncio.CancelledError as exc:
+                self._report_renewal_failure(exc)
+                return
             except Exception as exc:
-                if self._stop_event.is_set():
-                    return
-                logger.exception("Failed to renew message lease")
-                self._emit(
-                    "lease_renew_failed",
-                    "failure",
-                    message_id=self._message_id,
-                    lease_token_hash=self._lease_token_hash,
-                    exception_type=type(exc).__name__,
-                    error=exc,
-                )
-                with warnings.catch_warnings():
-                    warnings.simplefilter("always", RuntimeWarning)
-                    warnings.warn(
-                        "Failed to renew message lease "
-                        f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
-                        "when the visibility timeout expires",
-                        RuntimeWarning,
-                        stacklevel=1,
-                    )
-                self._invoke_failure_callback()
+                self._report_renewal_failure(exc)
                 return
             if not renewed:
                 self._emit(
@@ -623,6 +629,8 @@ class RedisMessageQueue:
         ``"drop_oldest"`` evicts the oldest pending message before enqueueing
         the new one. ``"drop_oldest"`` requires ``max_pending_length`` and is
         not compatible with deduplication or ``max_delivery_count``.
+        ``"block"`` also requires ``max_pending_length`` (the threshold to
+        block on); only the default ``"raise"`` operates on an unbounded queue.
 
         ``pending_overload_block_timeout_seconds`` bounds how long ``"block"``
         waits for capacity before raising ``QueueBackpressureError``. ``0``
@@ -635,11 +643,21 @@ class RedisMessageQueue:
         ``interrupt`` accepts a ``BaseGracefulInterruptHandler``; pass
         ``GracefulInterruptHandler()`` for prompt Ctrl-C / termination handling
         in polling waits. ``on_heartbeat_failure`` is a zero-argument callable
-        invoked when lease renewal fails. ``on_event`` is telemetry only and
-        receives best-effort QueueEvent lifecycle notifications; callback
+        invoked when lease renewal fails. It runs on the heartbeat's background
+        thread (sync queue) or on the event loop (async queue); in the async
+        queue it MUST NOT block (no ``time.sleep``, blocking I/O, or CPU-heavy
+        work; use ``await``), because a blocking callback freezes the event
+        loop, delaying lease renewal for every other in-flight message (whose
+        leases can then expire and be reclaimed) and stalling ``aclose()``.
+        Keep it quick and offload slow work yourself. ``on_event`` is telemetry
+        only and receives best-effort QueueEvent lifecycle notifications; callback
         failures are logged and converted to RuntimeWarning without influencing
         ack/nack or any other message outcome. Do not use it for
-        correctness-critical callbacks or follow-up writes.
+        correctness-critical callbacks or follow-up writes. ``on_event`` runs
+        inline in the caller's thread and may execute while an internal
+        publish/drain lock is held, so the callback must not call back into the
+        same queue instance's ``publish()``, ``drain()``, or ``close()``; that
+        lock is non-reentrant, so re-entering deadlocks the caller permanently.
         """
         self.key = QueueKeyManager(name, key_separator=key_separator)
         if not isinstance(deduplication, bool):
@@ -728,6 +746,18 @@ class RedisMessageQueue:
             deduplication=deduplication,
             get_deduplication_key=get_deduplication_key,
         )
+        if gateway is not None:
+            # Before the generic validator so gateway-incompat wins over the drop_oldest runaround; non-default only.
+            if pending_overload_policy != "raise":
+                raise ConfigurationError(
+                    "'pending_overload_policy' cannot be provided alongside 'gateway'."
+                    " Configure publish backpressure on the gateway directly instead."
+                )
+            if pending_overload_block_timeout_seconds != DEFAULT_PENDING_OVERLOAD_BLOCK_TIMEOUT_SECONDS:
+                raise ConfigurationError(
+                    "'pending_overload_block_timeout_seconds' cannot be provided alongside 'gateway'."
+                    " Configure publish backpressure on the gateway directly instead."
+                )
         validate_pending_backpressure_parameters(
             max_pending_length,
             pending_overload_policy,
@@ -1150,7 +1180,7 @@ class RedisMessageQueue:
                             "visibility timeouts are in use."
                         )
                         logger.warning(no_lease_token_warning)
-                        warnings.warn(no_lease_token_warning, RuntimeWarning, stacklevel=2)
+                        _warn_runtime_warning(no_lease_token_warning, stacklevel=2)
 
                 if lease_token is None and self._requires_claimed_message:
                     raise GatewayContractError(
@@ -1203,10 +1233,13 @@ class RedisMessageQueue:
         )
 
         lease_heartbeat = self._build_lease_heartbeat(stored_message, lease_token, message_id, lease_token_hash)
-        if lease_heartbeat is not None:
-            lease_heartbeat.start()
         processing_started_at = time.perf_counter()
         try:
+            # start() must be inside this try so its finally always stop()s the
+            # heartbeat; an exception in the start()->yield window (e.g. a
+            # signal-induced KeyboardInterrupt) would otherwise orphan it.
+            if lease_heartbeat is not None:
+                lease_heartbeat.start()
             yield message  # type: ignore
         except BaseException as exc:
             skip_cleanup = _should_skip_message_cleanup(exc)