redis-message-queue 8.2.6__tar.gz → 8.2.8__tar.gz

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: redis-message-queue
-Version: 8.2.6
+Version: 8.2.8
 Summary: Python message queuing with Redis and message deduplication
 Project-URL: Homepage, https://github.com/Elijas/redis-message-queue
 Project-URL: Repository, https://github.com/Elijas/redis-message-queue
@@ -34,7 +34,7 @@ Description-Content-Type: text/markdown
 **Lightweight Python message queuing with Redis and built-in publish-side deduplication.** Deduplicate publishes within a TTL window, with optional crash recovery — across any number of producers and consumers.
 
 ```bash
-pip install "redis-message-queue>=8.2.6,<9.0.0"
+pip install "redis-message-queue>=8.2.8,<9.0.0"
 ```
 
 Requires Redis server >= 6.2.
@@ -129,9 +129,12 @@ All features are optional and can be enabled or disabled as needed.
 | Configuration | Delivery guarantee |
 |---|---|
 | Default (`visibility_timeout_seconds=300`) | **At-least-once** — expired messages are reclaimed and redelivered |
-| With `visibility_timeout_seconds=None` | **At-most-once** — a consumer crash loses the in-flight message |
+| With `visibility_timeout_seconds=None, max_delivery_count=None` | **At-most-once** — a consumer crash loses the in-flight message |
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
+Because delivery-count limits depend on visibility-timeout reclaim, disabling
+lease-based crash recovery requires setting both `visibility_timeout_seconds=None`
+and `max_delivery_count=None`.
 
 > **Important:** Handler exceptions are terminal. This library is a payload
 > queue, not a task framework: raising inside `process_message()` does not
@@ -290,7 +293,10 @@ queue = RedisMessageQueue(
 
 The callback is **advisory** — it may fire briefly after a successful `process_message` exit when a final renewal coincided with the success path. Use it for metrics or alerting, not as a correctness signal. For the async queue (`redis_message_queue.asyncio`), the callback may also be `async def`.
 
-Without a visibility timeout, messages already moved to `processing` remain there indefinitely after a consumer crash and are not redelivered, even if the crash happened before your handler started running.
+With `visibility_timeout_seconds=None, max_delivery_count=None`, messages
+already moved to `processing` remain there indefinitely after a consumer crash
+and are not redelivered, even if the crash happened before your handler
+started running.
 
 Visibility deadlines use Redis server time (`TIME`), not Python process time.
 A forward step in the Redis server clock can make a live lease appear expired
@@ -473,10 +479,10 @@ without `aclose()`, or sync processes killed mid-handler, can leave the message
 and its processing/lease metadata in Redis until a later consumer claim path
 triggers visibility-timeout reclaim. With visibility timeouts enabled, this is
 the designed at-least-once recovery path: the message is delayed by the lease,
-not lost. With `visibility_timeout_seconds=None`, there is no automatic reclaim
-path. For low-visibility-timeout workloads, prefer an explicit `drain()` /
-`aclose()` during shutdown so local pending claim IDs are recovered before
-process exit.
+not lost. With `visibility_timeout_seconds=None, max_delivery_count=None`,
+there is no automatic reclaim path. For low-visibility-timeout workloads,
+prefer an explicit `drain()` / `aclose()` during shutdown so local pending claim
+IDs are recovered before process exit.
 
 `drain()` / `aclose()` timeouts are measured with Python monotonic clocks, but
 any lease deadlines they recover were created from Redis server time. The same

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/README.md

@@ -11,7 +11,7 @@
 **Lightweight Python message queuing with Redis and built-in publish-side deduplication.** Deduplicate publishes within a TTL window, with optional crash recovery — across any number of producers and consumers.
 
 ```bash
-pip install "redis-message-queue>=8.2.6,<9.0.0"
+pip install "redis-message-queue>=8.2.8,<9.0.0"
 ```
 
 Requires Redis server >= 6.2.
@@ -106,9 +106,12 @@ All features are optional and can be enabled or disabled as needed.
 | Configuration | Delivery guarantee |
 |---|---|
 | Default (`visibility_timeout_seconds=300`) | **At-least-once** — expired messages are reclaimed and redelivered |
-| With `visibility_timeout_seconds=None` | **At-most-once** — a consumer crash loses the in-flight message |
+| With `visibility_timeout_seconds=None, max_delivery_count=None` | **At-most-once** — a consumer crash loses the in-flight message |
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
+Because delivery-count limits depend on visibility-timeout reclaim, disabling
+lease-based crash recovery requires setting both `visibility_timeout_seconds=None`
+and `max_delivery_count=None`.
 
 > **Important:** Handler exceptions are terminal. This library is a payload
 > queue, not a task framework: raising inside `process_message()` does not
@@ -267,7 +270,10 @@ queue = RedisMessageQueue(
 
 The callback is **advisory** — it may fire briefly after a successful `process_message` exit when a final renewal coincided with the success path. Use it for metrics or alerting, not as a correctness signal. For the async queue (`redis_message_queue.asyncio`), the callback may also be `async def`.
 
-Without a visibility timeout, messages already moved to `processing` remain there indefinitely after a consumer crash and are not redelivered, even if the crash happened before your handler started running.
+With `visibility_timeout_seconds=None, max_delivery_count=None`, messages
+already moved to `processing` remain there indefinitely after a consumer crash
+and are not redelivered, even if the crash happened before your handler
+started running.
 
 Visibility deadlines use Redis server time (`TIME`), not Python process time.
 A forward step in the Redis server clock can make a live lease appear expired
@@ -450,10 +456,10 @@ without `aclose()`, or sync processes killed mid-handler, can leave the message
 and its processing/lease metadata in Redis until a later consumer claim path
 triggers visibility-timeout reclaim. With visibility timeouts enabled, this is
 the designed at-least-once recovery path: the message is delayed by the lease,
-not lost. With `visibility_timeout_seconds=None`, there is no automatic reclaim
-path. For low-visibility-timeout workloads, prefer an explicit `drain()` /
-`aclose()` during shutdown so local pending claim IDs are recovered before
-process exit.
+not lost. With `visibility_timeout_seconds=None, max_delivery_count=None`,
+there is no automatic reclaim path. For low-visibility-timeout workloads,
+prefer an explicit `drain()` / `aclose()` during shutdown so local pending claim
+IDs are recovered before process exit.
 
 `drain()` / `aclose()` timeouts are measured with Python monotonic clocks, but
 any lease deadlines they recover were created from Redis server time. The same

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "redis-message-queue"
-version = "8.2.6"
+version = "8.2.8"
 description = "Python message queuing with Redis and message deduplication"
 authors = [{ name = "Elijas", email = "4084885+Elijas@users.noreply.github.com" }]
 readme = "README.md"
@@ -48,7 +48,7 @@ default-groups = ["dev", "test"]
 ##############################
 
 [tool.bumpversion]
-current_version = "8.2.6"
+current_version = "8.2.8"
 parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
 serialize = ["{major}.{minor}.{patch}"]
 search = "{current_version}"

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/redis_message_queue/_config.py

@@ -891,10 +891,11 @@ local dead_lettered_events = {}
 local claim_store_failed_sentinel = string.char(0) .. '__rmq_claim_store_failed__'
 
 local function store_claim_and_return(stored)
-    -- pcall guards against OOM mid-write: compensate by returning message to pending
+    -- pcall guards against OOM mid-write: fail fast while preserving a live payload copy.
+    local lease_token = nil
     local ok, result = pcall(function()
         redis.call('INCR', KEYS[5])
-        local lease_token = redis.call('GET', KEYS[5])
+        lease_token = redis.call('GET', KEYS[5])
         local claim_payload = cjson.encode({stored, lease_token})
         redis.call('ZADD', KEYS[3], now_ms + tonumber(ARGV[1]), stored)
         redis.call('HSET', KEYS[4], stored, lease_token)
@@ -906,8 +907,20 @@ local function store_claim_and_return(stored)
     end)
     if not ok then
         redis.call('HINCRBY', KEYS[6], stored, -1)
+        local return_result = redis.pcall('RPUSH', KEYS[1], stored)
+        if type(return_result) == 'table' and return_result['err'] then
+            local failure = tostring(result) .. '; return-to-pending failed: ' .. tostring(return_result['err'])
+            return {claim_store_failed_sentinel, failure, stored}
+        end
         redis.call('LREM', KEYS[2], 1, stored)
-        redis.pcall('RPUSH', KEYS[1], stored)
+        redis.call('ZREM', KEYS[3], stored)
+        redis.call('HDEL', KEYS[4], stored)
+        redis.call('DEL', KEYS[8])
+        redis.call('HDEL', KEYS[10], ARGV[4])
+        if lease_token then
+            redis.call('HDEL', KEYS[9], lease_token)
+            redis.call('HDEL', KEYS[11], lease_token)
+        end
         return {claim_store_failed_sentinel, tostring(result), stored}
     end
     return result

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/redis_message_queue/_exceptions.py

@@ -51,8 +51,9 @@ class CleanupFailedError(RedisMessageQueueError):
 class ClaimStoreFailedError(RedisMessageQueueError):
     """Raised when the VT-claim Lua store_claim_and_return pcall failed.
 
-    The script decremented the speculative delivery_count increment and
-    compensated by returning the message to pending before surfacing this error.
+    The script decremented the speculative delivery_count increment. It returns
+    the message to pending when that write succeeds; if that write fails, it
+    leaves the message in processing so the payload still has a live queue copy.
     """
 
 

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/redis_message_queue/_redis_gateway.py

@@ -987,7 +987,8 @@ class RedisGateway(AbstractRedisGateway):
             stored_message = result[2] if len(result) > 2 else None
             message_id = extract_stored_message_id(stored_message) if isinstance(stored_message, (str, bytes)) else None
             raise ClaimStoreFailedError(
-                f"VT claim store failed after delivery_count rollback: {_decode_lua_error(result[1])}",
+                f"VT claim store failed after delivery_count rollback and payload preservation: "
+                f"{_decode_lua_error(result[1])}",
                 queue=from_queue,
                 message_id=message_id,
                 operation="claim",

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/redis_message_queue/asyncio/_redis_gateway.py

@@ -967,7 +967,8 @@ class RedisGateway(AbstractRedisGateway):
             stored_message = result[2] if len(result) > 2 else None
             message_id = extract_stored_message_id(stored_message) if isinstance(stored_message, (str, bytes)) else None
             raise ClaimStoreFailedError(
-                f"VT claim store failed after delivery_count rollback: {_decode_lua_error(result[1])}",
+                f"VT claim store failed after delivery_count rollback and payload preservation: "
+                f"{_decode_lua_error(result[1])}",
                 queue=from_queue,
                 message_id=message_id,
                 operation="claim",

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/redis_message_queue/asyncio/redis_message_queue.py

@@ -607,8 +607,9 @@ class RedisMessageQueue:
     ):
         """Create a queue bound to an async Redis client or custom gateway.
 
-        ``visibility_timeout_seconds`` defaults to 300. Set it to ``None`` to
-        disable lease-based crash recovery; messages left in ``processing`` by a
+        ``visibility_timeout_seconds`` defaults to 300. To disable
+        lease-based crash recovery, set both ``visibility_timeout_seconds=None``
+        and ``max_delivery_count=None``; messages left in ``processing`` by a
         crashed worker are then not reclaimed automatically.
 
         ``visibility_timeout_seconds`` is a Redis server-time lease, not a
@@ -1072,9 +1073,23 @@ class RedisMessageQueue:
                         "See AbstractRedisGateway.add_message for the full contract."
                     )
             else:
-                dedup_key = self._get_deduplication_key(message)
-                if inspect.isawaitable(dedup_key):
-                    dedup_key = await dedup_key
+                try:
+                    dedup_key = self._get_deduplication_key(message)
+                    if inspect.isawaitable(dedup_key):
+                        dedup_key = await dedup_key
+                except asyncio.CancelledError as exc:
+                    current_task = asyncio.current_task()
+                    if current_task is not None and current_task.cancelling() > 0:
+                        raise
+                    _set_exception_context(exc, queue=self._queue_name, operation="publish")
+                    await self._emit_event(
+                        "publish",
+                        "failure",
+                        exception_type=type(exc).__name__,
+                        error=exc,
+                        duration_ms=_duration_ms(started_at),
+                    )
+                    raise
                 dedup_key = validate_callable_deduplication_key(dedup_key, message)
                 dedup_key = self.key.deduplication(dedup_key)
                 result = await self._redis.publish_message(self.key.pending, message_str, dedup_key)

{redis_message_queue-8.2.6 → redis_message_queue-8.2.8}/redis_message_queue/redis_message_queue.py

@@ -80,6 +80,14 @@ def _duration_ms(started_at: float) -> float:
     return (time.perf_counter() - started_at) * 1000
 
 
+def _current_async_task_is_cancelling() -> bool:
+    try:
+        current_task = asyncio.current_task()
+    except RuntimeError:
+        return False
+    return current_task is not None and current_task.cancelling() > 0
+
+
 def _hash_lease_token(lease_token: str | None) -> str | None:
     if lease_token is None:
         return None
@@ -558,8 +566,9 @@ class RedisMessageQueue:
     ):
         """Create a queue bound to a Redis client or custom gateway.
 
-        ``visibility_timeout_seconds`` defaults to 300. Set it to ``None`` to
-        disable lease-based crash recovery; messages left in ``processing`` by a
+        ``visibility_timeout_seconds`` defaults to 300. To disable
+        lease-based crash recovery, set both ``visibility_timeout_seconds=None``
+        and ``max_delivery_count=None``; messages left in ``processing`` by a
         crashed worker are then not reclaimed automatically.
 
         ``visibility_timeout_seconds`` is a Redis server-time lease, not a
@@ -1017,7 +1026,20 @@ class RedisMessageQueue:
                         "See AbstractRedisGateway.add_message for the full contract."
                     )
             else:
-                dedup_key = self._get_deduplication_key(message)
+                try:
+                    dedup_key = self._get_deduplication_key(message)
+                except asyncio.CancelledError as exc:
+                    if _current_async_task_is_cancelling():
+                        raise
+                    _set_exception_context(exc, queue=self._queue_name, operation="publish")
+                    self._emit_event(
+                        "publish",
+                        "failure",
+                        exception_type=type(exc).__name__,
+                        error=exc,
+                        duration_ms=_duration_ms(started_at),
+                    )
+                    raise
                 if inspect.isawaitable(dedup_key):
                     is_coroutine = inspect.iscoroutine(dedup_key)
                     _close_or_cancel_awaitable(dedup_key)