redis-message-queue 8.2.8__tar.gz → 8.2.10__tar.gz

{redis_message_queue-8.2.8 → redis_message_queue-8.2.10}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: redis-message-queue
-Version: 8.2.8
+Version: 8.2.10
 Summary: Python message queuing with Redis and message deduplication
 Project-URL: Homepage, https://github.com/Elijas/redis-message-queue
 Project-URL: Repository, https://github.com/Elijas/redis-message-queue
@@ -34,7 +34,7 @@ Description-Content-Type: text/markdown
 **Lightweight Python message queuing with Redis and built-in publish-side deduplication.** Deduplicate publishes within a TTL window, with optional crash recovery — across any number of producers and consumers.
 
 ```bash
-pip install "redis-message-queue>=8.2.8,<9.0.0"
+pip install "redis-message-queue>=8.2.10,<9.0.0"
 ```
 
 Requires Redis server >= 6.2.
@@ -848,6 +848,11 @@ Pre-commit and mid-flight exceptions:
   exceptions. Under an ambiguous lost response, Redis may have committed
   despite the exception. Treat them as "operation did not succeed from the
   caller's perspective", not "Redis did not commit".
+- Visibility-timeout claim-store write failures raise
+  `ClaimStoreFailedError` and emit `claim/failure`. When the compensating
+  return-to-pending write succeeds, the payload is back in pending; if
+  return-to-pending also fails, the payload remains in processing so there is
+  still a live queue copy.
 
 #### Drain events
 
@@ -875,11 +880,6 @@ The following operations have no `on_event` surface by design:
   preserves queue safety on Cluster `CROSSSLOT` rejection but cannot be
   observed through `on_event`. Operators watching key-TTL behavior or Redis
   slow logs can detect orphans.
-- **VT claim-store OOM compensation:** if the visibility-timeout Lua script
-  cannot store the claim result, it removes the message from processing, pushes
-  it back to pending, and returns `false`. Python translates that into
-  `claim_empty/skipped`, the same shape as an empty poll. This is intentional
-  fail-safe behavior; the message is not lost.
 - **`drop_oldest` evictions:** when publish backpressure uses
   `pending_overload_policy="drop_oldest"`, the oldest pending message is
   discarded before the new message is enqueued. The successful enqueue emits
@@ -890,14 +890,22 @@ The following operations have no `on_event` surface by design:
   terminal operation's failure event. There is no per-attempt event for those
   paths.
 
-The public exception hierarchy is rooted at `RedisMessageQueueError`.
-Configuration value/combinations raise `ConfigurationError` (also a
-`ValueError`), custom gateway contract violations raise `GatewayContractError`
-(also a `TypeError`), and Lua `redis.error_reply(...)` failures raise
-`LuaScriptError` (also a redis-py `ResponseError`). Publish overload raises
-`QueueBackpressureError`; publish after explicit drain raises
-`QueueDrainedError`. `CleanupFailedError` and `RetryBudgetExhaustedError` are
-reserved categories for cleanup and retry surfaces.
+The public exception hierarchy is rooted at `RedisMessageQueueError`. The
+current exported queue-owned exception classes are:
+
+- `RedisMessageQueueError` (base)
+  - `ClaimStoreFailedError` - visibility-timeout claim metadata could not be stored
+  - `ConfigurationError` - invalid constructor args; also a `ValueError`
+  - `DrainFailedError` - drain pending-claim recovery failed
+  - `GatewayContractError` - custom gateway protocol violation; also a `TypeError`
+  - `LuaScriptError` - Lua `redis.error_reply(...)`; also a redis-py `ResponseError`
+  - `MalformedStoredMessageError` - stored value is not a valid RMQ envelope
+  - `PayloadTooLargeError` - serialized payload exceeds `max_payload_bytes`; also a `ValueError`
+  - `PayloadTooDeepError` - payload nesting exceeds `max_payload_depth`; also a `ValueError`
+  - `QueueBackpressureError` - `pending_overload_policy="raise"` rejected enqueue
+  - `QueueDrainedError` - `publish()` called after explicit drain/aclose
+  - `CleanupFailedError` - cleanup after handler completion failed
+  - `RetryBudgetExhaustedError` - Redis retry budget exhausted; also a redis-py `RedisError`
 
 ## Known limitations
 

{redis_message_queue-8.2.8 → redis_message_queue-8.2.10}/README.md

@@ -11,7 +11,7 @@
 **Lightweight Python message queuing with Redis and built-in publish-side deduplication.** Deduplicate publishes within a TTL window, with optional crash recovery — across any number of producers and consumers.
 
 ```bash
-pip install "redis-message-queue>=8.2.8,<9.0.0"
+pip install "redis-message-queue>=8.2.10,<9.0.0"
 ```
 
 Requires Redis server >= 6.2.
@@ -825,6 +825,11 @@ Pre-commit and mid-flight exceptions:
   exceptions. Under an ambiguous lost response, Redis may have committed
   despite the exception. Treat them as "operation did not succeed from the
   caller's perspective", not "Redis did not commit".
+- Visibility-timeout claim-store write failures raise
+  `ClaimStoreFailedError` and emit `claim/failure`. When the compensating
+  return-to-pending write succeeds, the payload is back in pending; if
+  return-to-pending also fails, the payload remains in processing so there is
+  still a live queue copy.
 
 #### Drain events
 
@@ -852,11 +857,6 @@ The following operations have no `on_event` surface by design:
   preserves queue safety on Cluster `CROSSSLOT` rejection but cannot be
   observed through `on_event`. Operators watching key-TTL behavior or Redis
   slow logs can detect orphans.
-- **VT claim-store OOM compensation:** if the visibility-timeout Lua script
-  cannot store the claim result, it removes the message from processing, pushes
-  it back to pending, and returns `false`. Python translates that into
-  `claim_empty/skipped`, the same shape as an empty poll. This is intentional
-  fail-safe behavior; the message is not lost.
 - **`drop_oldest` evictions:** when publish backpressure uses
   `pending_overload_policy="drop_oldest"`, the oldest pending message is
   discarded before the new message is enqueued. The successful enqueue emits
@@ -867,14 +867,22 @@ The following operations have no `on_event` surface by design:
   terminal operation's failure event. There is no per-attempt event for those
   paths.
 
-The public exception hierarchy is rooted at `RedisMessageQueueError`.
-Configuration value/combinations raise `ConfigurationError` (also a
-`ValueError`), custom gateway contract violations raise `GatewayContractError`
-(also a `TypeError`), and Lua `redis.error_reply(...)` failures raise
-`LuaScriptError` (also a redis-py `ResponseError`). Publish overload raises
-`QueueBackpressureError`; publish after explicit drain raises
-`QueueDrainedError`. `CleanupFailedError` and `RetryBudgetExhaustedError` are
-reserved categories for cleanup and retry surfaces.
+The public exception hierarchy is rooted at `RedisMessageQueueError`. The
+current exported queue-owned exception classes are:
+
+- `RedisMessageQueueError` (base)
+  - `ClaimStoreFailedError` - visibility-timeout claim metadata could not be stored
+  - `ConfigurationError` - invalid constructor args; also a `ValueError`
+  - `DrainFailedError` - drain pending-claim recovery failed
+  - `GatewayContractError` - custom gateway protocol violation; also a `TypeError`
+  - `LuaScriptError` - Lua `redis.error_reply(...)`; also a redis-py `ResponseError`
+  - `MalformedStoredMessageError` - stored value is not a valid RMQ envelope
+  - `PayloadTooLargeError` - serialized payload exceeds `max_payload_bytes`; also a `ValueError`
+  - `PayloadTooDeepError` - payload nesting exceeds `max_payload_depth`; also a `ValueError`
+  - `QueueBackpressureError` - `pending_overload_policy="raise"` rejected enqueue
+  - `QueueDrainedError` - `publish()` called after explicit drain/aclose
+  - `CleanupFailedError` - cleanup after handler completion failed
+  - `RetryBudgetExhaustedError` - Redis retry budget exhausted; also a redis-py `RedisError`
 
 ## Known limitations
 

{redis_message_queue-8.2.8 → redis_message_queue-8.2.10}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "redis-message-queue"
-version = "8.2.8"
+version = "8.2.10"
 description = "Python message queuing with Redis and message deduplication"
 authors = [{ name = "Elijas", email = "4084885+Elijas@users.noreply.github.com" }]
 readme = "README.md"
@@ -48,7 +48,7 @@ default-groups = ["dev", "test"]
 ##############################
 
 [tool.bumpversion]
-current_version = "8.2.8"
+current_version = "8.2.10"
 parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
 serialize = ["{major}.{minor}.{patch}"]
 search = "{current_version}"

{redis_message_queue-8.2.8 → redis_message_queue-8.2.10}/redis_message_queue/asyncio/redis_message_queue.py

@@ -87,6 +87,12 @@ def _warning_exception_name(exc: BaseException) -> str:
     return type(exc).__name__
 
 
+def _warn_runtime_warning(message: str, *, stacklevel: int) -> None:
+    with warnings.catch_warnings():
+        warnings.simplefilter("always", RuntimeWarning)
+        warnings.warn(message, RuntimeWarning, stacklevel=stacklevel + 1)
+
+
 def _find_non_string_dict_keys(value: object) -> list[object]:
     non_str_keys: list[object] = []
     seen: set[int] = set()
@@ -505,6 +511,29 @@ class _LeaseHeartbeat:
                     stacklevel=1,
                 )
 
+    async def _report_renewal_failure(self, exc: BaseException) -> None:
+        if self._stop_event.is_set():
+            return
+        logger.exception("Failed to renew message lease")
+        await self._emit(
+            "lease_renew_failed",
+            "failure",
+            message_id=self._message_id,
+            lease_token_hash=self._lease_token_hash,
+            exception_type=type(exc).__name__,
+            error=exc,
+        )
+        with warnings.catch_warnings():
+            warnings.simplefilter("always", RuntimeWarning)
+            warnings.warn(
+                "Failed to renew message lease "
+                f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
+                "when the visibility timeout expires",
+                RuntimeWarning,
+                stacklevel=1,
+            )
+        await self._invoke_failure_callback()
+
     async def _run(self) -> None:
         try:
             while True:
@@ -525,30 +554,14 @@ class _LeaseHeartbeat:
                             f"gateway.renew_message_lease() must return bool, got {type(renewed).__name__}. "
                             "See AbstractRedisGateway.renew_message_lease for the full contract."
                         )
-                except asyncio.CancelledError:
-                    raise
+                except asyncio.CancelledError as exc:
+                    current_task = asyncio.current_task()
+                    if self._stop_event.is_set() or (current_task is not None and current_task.cancelling() > 0):
+                        raise
+                    await self._report_renewal_failure(exc)
+                    return
                 except Exception as exc:
-                    if self._stop_event.is_set():
-                        return
-                    logger.exception("Failed to renew message lease")
-                    await self._emit(
-                        "lease_renew_failed",
-                        "failure",
-                        message_id=self._message_id,
-                        lease_token_hash=self._lease_token_hash,
-                        exception_type=type(exc).__name__,
-                        error=exc,
-                    )
-                    with warnings.catch_warnings():
-                        warnings.simplefilter("always", RuntimeWarning)
-                        warnings.warn(
-                            "Failed to renew message lease "
-                            f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
-                            "when the visibility timeout expires",
-                            RuntimeWarning,
-                            stacklevel=1,
-                        )
-                    await self._invoke_failure_callback()
+                    await self._report_renewal_failure(exc)
                     return
                 if not renewed:
                     await self._emit(
@@ -1173,7 +1186,7 @@ class RedisMessageQueue:
                             "visibility timeouts are in use."
                         )
                         logger.warning(no_lease_token_warning)
-                        warnings.warn(no_lease_token_warning, RuntimeWarning, stacklevel=2)
+                        _warn_runtime_warning(no_lease_token_warning, stacklevel=2)
 
                 if lease_token is None and self._requires_claimed_message:
                     raise GatewayContractError(
@@ -1273,7 +1286,7 @@ class RedisMessageQueue:
                         message_id=message_id,
                         lease_token_hash=lease_token_hash,
                     )
-                    warnings.warn(_STALE_LEASE_NACK_WARNING, RuntimeWarning, stacklevel=2)
+                    _warn_runtime_warning(_STALE_LEASE_NACK_WARNING, stacklevel=2)
             except BaseException as cleanup_exc:
                 # The handler exception is the user-visible failure; cleanup failure is secondary.
                 logger.exception("Failed to clean up message from processing queue")
@@ -1286,10 +1299,9 @@ class RedisMessageQueue:
                     error=cleanup_exc,
                     duration_ms=_duration_ms(cleanup_started_at),
                 )
-                warnings.warn(
+                _warn_runtime_warning(
                     f"Cleanup raised after handler exception ({_warning_exception_name(cleanup_exc)}); "
                     "see logs for both tracebacks",
-                    RuntimeWarning,
                     stacklevel=2,
                 )
             raise
@@ -1349,7 +1361,7 @@ class RedisMessageQueue:
                     message_id=message_id,
                     lease_token_hash=lease_token_hash,
                 )
-                warnings.warn(_STALE_LEASE_ACK_WARNING, RuntimeWarning, stacklevel=2)
+                _warn_runtime_warning(_STALE_LEASE_ACK_WARNING, stacklevel=2)
             finished_without_error = True
         finally:
             if lease_heartbeat is not None:
@@ -1438,9 +1450,8 @@ class RedisMessageQueue:
                     exception_type=type(exc).__name__,
                     error=exc,
                 )
-                warnings.warn(
+                _warn_runtime_warning(
                     f"Failed to trim queue {destination_queue} ({type(exc).__name__}); list may exceed max_*_length",
-                    RuntimeWarning,
                     stacklevel=3,
                 )
 

{redis_message_queue-8.2.8 → redis_message_queue-8.2.10}/redis_message_queue/redis_message_queue.py

@@ -100,6 +100,12 @@ def _warning_exception_name(exc: BaseException) -> str:
     return type(exc).__name__
 
 
+def _warn_runtime_warning(message: str, *, stacklevel: int) -> None:
+    with warnings.catch_warnings():
+        warnings.simplefilter("always", RuntimeWarning)
+        warnings.warn(message, RuntimeWarning, stacklevel=stacklevel + 1)
+
+
 def _find_non_string_dict_keys(value: object) -> list[object]:
     non_str_keys: list[object] = []
     seen: set[int] = set()
@@ -339,12 +345,16 @@ def _close_or_cancel_awaitable(awaitable: object) -> None:
         try:
             close()
             return
+        except asyncio.CancelledError:
+            pass
         except Exception:
             pass
     cancel = getattr(awaitable, "cancel", None)
     if callable(cancel):
         try:
             cancel()
+        except asyncio.CancelledError:
+            pass
         except Exception:
             pass
 
@@ -472,6 +482,29 @@ class _LeaseHeartbeat:
                     stacklevel=1,
                 )
 
+    def _report_renewal_failure(self, exc: BaseException) -> None:
+        if self._stop_event.is_set():
+            return
+        logger.exception("Failed to renew message lease")
+        self._emit(
+            "lease_renew_failed",
+            "failure",
+            message_id=self._message_id,
+            lease_token_hash=self._lease_token_hash,
+            exception_type=type(exc).__name__,
+            error=exc,
+        )
+        with warnings.catch_warnings():
+            warnings.simplefilter("always", RuntimeWarning)
+            warnings.warn(
+                "Failed to renew message lease "
+                f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
+                "when the visibility timeout expires",
+                RuntimeWarning,
+                stacklevel=1,
+            )
+        self._invoke_failure_callback()
+
     def _run(self) -> None:
         # No explicit _is_interrupted() check here. Heartbeat lifetime is owned
         # by process_message, which sets _stop_event in its finally block on any
@@ -488,28 +521,11 @@ class _LeaseHeartbeat:
                         f"gateway.renew_message_lease() must return bool, got {type(renewed).__name__}. "
                         "See AbstractRedisGateway.renew_message_lease for the full contract."
                     )
+            except asyncio.CancelledError as exc:
+                self._report_renewal_failure(exc)
+                return
             except Exception as exc:
-                if self._stop_event.is_set():
-                    return
-                logger.exception("Failed to renew message lease")
-                self._emit(
-                    "lease_renew_failed",
-                    "failure",
-                    message_id=self._message_id,
-                    lease_token_hash=self._lease_token_hash,
-                    exception_type=type(exc).__name__,
-                    error=exc,
-                )
-                with warnings.catch_warnings():
-                    warnings.simplefilter("always", RuntimeWarning)
-                    warnings.warn(
-                        "Failed to renew message lease "
-                        f"({_warning_exception_name(exc)}); message will be reclaimed by another consumer "
-                        "when the visibility timeout expires",
-                        RuntimeWarning,
-                        stacklevel=1,
-                    )
-                self._invoke_failure_callback()
+                self._report_renewal_failure(exc)
                 return
             if not renewed:
                 self._emit(
@@ -1140,7 +1156,7 @@ class RedisMessageQueue:
                             "visibility timeouts are in use."
                         )
                         logger.warning(no_lease_token_warning)
-                        warnings.warn(no_lease_token_warning, RuntimeWarning, stacklevel=2)
+                        _warn_runtime_warning(no_lease_token_warning, stacklevel=2)
 
                 if lease_token is None and self._requires_claimed_message:
                     raise GatewayContractError(
@@ -1235,7 +1251,7 @@ class RedisMessageQueue:
                         message_id=message_id,
                         lease_token_hash=lease_token_hash,
                     )
-                    warnings.warn(_STALE_LEASE_NACK_WARNING, RuntimeWarning, stacklevel=2)
+                    _warn_runtime_warning(_STALE_LEASE_NACK_WARNING, stacklevel=2)
             except BaseException as cleanup_exc:
                 # The handler exception is the user-visible failure; cleanup failure is secondary.
                 logger.exception("Failed to clean up message from processing queue")
@@ -1248,10 +1264,9 @@ class RedisMessageQueue:
                     error=cleanup_exc,
                     duration_ms=_duration_ms(cleanup_started_at),
                 )
-                warnings.warn(
+                _warn_runtime_warning(
                     f"Cleanup raised after handler exception ({_warning_exception_name(cleanup_exc)}); "
                     "see logs for both tracebacks",
-                    RuntimeWarning,
                     stacklevel=2,
                 )
             raise
@@ -1310,7 +1325,7 @@ class RedisMessageQueue:
                     message_id=message_id,
                     lease_token_hash=lease_token_hash,
                 )
-                warnings.warn(_STALE_LEASE_ACK_WARNING, RuntimeWarning, stacklevel=2)
+                _warn_runtime_warning(_STALE_LEASE_ACK_WARNING, stacklevel=2)
         finally:
             if lease_heartbeat is not None:
                 lease_heartbeat.stop()
@@ -1400,9 +1415,8 @@ class RedisMessageQueue:
                     exception_type=type(exc).__name__,
                     error=exc,
                 )
-                warnings.warn(
+                _warn_runtime_warning(
                     f"Failed to trim queue {destination_queue} ({type(exc).__name__}); list may exceed max_*_length",
-                    RuntimeWarning,
                     stacklevel=3,
                 )