redis-message-queue 3.1.1__tar.gz → 5.0.0__tar.gz

{redis_message_queue-3.1.1 → redis_message_queue-5.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: redis-message-queue
-Version: 3.1.1
+Version: 5.0.0
 Summary: Python message queuing with Redis and message deduplication
 License: MIT
 License-File: LICENSE
@@ -26,7 +26,7 @@ Description-Content-Type: text/markdown
 
 # redis-message-queue
 
-[![PyPI Version](https://img.shields.io/badge/v3.1.1-version?color=43cd0f&style=flat&label=pypi)](https://pypi.org/project/redis-message-queue)
+[![PyPI Version](https://img.shields.io/badge/v5.0.0-version?color=43cd0f&style=flat&label=pypi)](https://pypi.org/project/redis-message-queue)
 [![PyPI Downloads](https://img.shields.io/pypi/dm/redis-message-queue?color=43cd0f&style=flat&label=downloads)](https://pypistats.org/packages/redis-message-queue)
 [![License: MIT](https://img.shields.io/badge/License-MIT-43cd0f.svg?style=flat&label=license)](LICENSE)
 [![Maintained: yes](https://img.shields.io/badge/yes-43cd0f.svg?style=flat&label=maintained)](https://github.com/Elijas/redis-message-queue/issues)
@@ -50,7 +50,7 @@ Requires Redis server >= 6.2.
 from redis import Redis
 from redis_message_queue import RedisMessageQueue
 
-client = Redis.from_url("redis://localhost:6379/0")
+client = Redis.from_url("redis://localhost:6379/0", decode_responses=True)
 queue = RedisMessageQueue("my_queue", client=client, deduplication=True)
 
 queue.publish("order:1234")           # returns True
@@ -74,6 +74,9 @@ while True:
             # Auto-acknowledged on success; cleaned up on exception
 ```
 
+`RedisMessageQueue` itself is not a context manager. Use
+`with queue.process_message() as message:` for each message.
+
 ## Why redis-message-queue
 
 **The problem:** You're sending messages between services or workers and need guarantees. Simple Redis LPUSH/BRPOP loses messages on crashes, doesn't deduplicate, and gives you no visibility into what succeeded or failed.
@@ -97,8 +100,8 @@ All features are optional and can be enabled or disabled as needed.
 
 | Configuration | Delivery guarantee |
 |---|---|
-| Default (no visibility timeout) | **At-most-once** — a consumer crash loses the in-flight message |
-| With `visibility_timeout_seconds` | **At-least-once** — expired messages are reclaimed and redelivered |
+| Default (`visibility_timeout_seconds=300`) | **At-least-once** — expired messages are reclaimed and redelivered |
+| With `visibility_timeout_seconds=None` | **At-most-once** — a consumer crash loses the in-flight message |
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
 
@@ -107,7 +110,7 @@ See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-tim
 ### Deduplication
 
 ```python
-# Default: deduplicate by full message content (1-hour TTL)
+# Default: deduplicate by SHA-256 hash of canonical message content (1-hour TTL)
 queue = RedisMessageQueue("q", client=client, deduplication=True)
 
 # Custom dedup key (e.g., deduplicate by order ID only)
@@ -131,7 +134,8 @@ queue = RedisMessageQueue(
 )
 ```
 
-To prevent unbounded growth, cap the queue lengths:
+Completed and failed tracking queues are capped at 1,000 entries by default
+when enabled. Override the caps when you need a different retention window:
 
 ```python
 queue = RedisMessageQueue(
@@ -144,6 +148,8 @@ queue = RedisMessageQueue(
 ```
 
 When set, `LTRIM` is called after each message is moved to the completed/failed queue. This is best-effort cleanup — if the trim fails, the queue is slightly longer until the next successful trim.
+Pass `max_completed_length=None` or `max_failed_length=None` explicitly if you
+want unbounded tracking queues.
 
 ### Crash recovery with visibility timeout
 
@@ -191,14 +197,14 @@ queue = RedisMessageQueue(
 )
 ```
 
-When a message has been delivered more than `max_delivery_count` times (due to consumer crashes causing visibility-timeout reclaim), it is automatically routed to a dead-letter queue (`{name}::dead_letter`) instead of being redelivered. This prevents poison messages from cycling indefinitely.
+When a message has been delivered more than `max_delivery_count` times (due to consumer crashes causing visibility-timeout reclaim), it is automatically routed to a dead-letter queue (`{name}::dlq`) instead of being redelivered. `max_delivery_count` defaults to `10` on the built-in `client=` path, with the DLQ name auto-derived from the queue name. This prevents poison messages from cycling indefinitely.
 
 Notes:
 - requires `visibility_timeout_seconds` to be set (poison messages are only a concern with VT reclaim)
 - the delivery count is tracked per-message in a Redis HASH and cleaned up on successful ack or move to completed/failed
 - the delivery count increments when Redis grants the claim/lease, not when your handler begins running. If a process exits after Redis claims a message, that claim still counts toward `max_delivery_count`
 - `max_delivery_count=1` means the message is delivered once; any reclaim routes it to the dead-letter queue
-- without `max_delivery_count`, messages are redelivered indefinitely (existing behavior)
+- set `max_delivery_count=None` explicitly for unlimited redelivery
 - dead-lettered messages contain the **raw payload** only — the internal envelope (which carries a per-delivery UUID) is stripped before pushing to the DLQ, consistent with how completed/failed queues store messages. Two identical payloads dead-lettered separately are indistinguishable in the DLQ
 
 ### Graceful shutdown
@@ -227,7 +233,7 @@ while not interrupt.is_interrupted():
 ### Custom gateway
 
 ```python
-from redis_message_queue._redis_gateway import RedisGateway
+from redis_message_queue import RedisGateway
 
 # Tune retry budget, dedup TTL, or wait interval
 gateway = RedisGateway(
@@ -253,8 +259,8 @@ impossible. Note: tenacity may allow one additional attempt beyond the budget if
 
 To plug in a different retry library (`backoff`, `asyncstdlib.retry`, or your
 own logic) or fundamentally different semantics, subclass
-`AbstractRedisGateway` from `redis_message_queue._abstract_redis_gateway`
-(or `redis_message_queue.asyncio._abstract_redis_gateway`) and override the
+`AbstractRedisGateway` from `redis_message_queue` (or
+`redis_message_queue.asyncio` for the async sibling) and override the
 operation methods directly.
 
 If your custom gateway uses visibility timeouts, it must expose a public
@@ -285,6 +291,16 @@ Use a separate gateway instance per queue when `max_delivery_count` is enabled.
 Dead-letter routing is gateway-scoped, so reusing the same gateway across different
 queues is rejected.
 
+If you use Redis Sentinel, pass the Redis client returned by
+`sentinel.master_for(name)` to `client=` or `RedisGateway(redis_client=...)`, not
+the `sentinel` object itself.
+
+### Connection pool sizing
+
+Each queue with `heartbeat_interval_seconds` set uses up to 2 simultaneous
+connections: one for the main operation and one for heartbeat renewal. Size Redis
+client pools with `max_connections >= 2 * number_of_queues + headroom`.
+
 ## Async API
 
 Replace the import to use the async variant — the API is identical:
@@ -293,6 +309,11 @@ Replace the import to use the async variant — the API is identical:
 from redis_message_queue.asyncio import RedisMessageQueue
 ```
 
+The sync and async classes intentionally share names. In modules that use both,
+alias the imports explicitly, for example
+`from redis_message_queue import RedisMessageQueue as SyncRedisMessageQueue` and
+`from redis_message_queue.asyncio import RedisMessageQueue as AsyncRedisMessageQueue`.
+
 All examples work the same way. Remember to close the connection when done:
 
 ```python
@@ -303,6 +324,9 @@ client = redis.Redis()
 await client.aclose()
 ```
 
+For the sync Redis client, call `client.close()` during application shutdown when
+you own the client lifecycle.
+
 ## Known limitations
 
 - **No metrics or observability hooks.** The library logs warnings (stale leases, heartbeat failures, transient errors) via Python's `logging` module but does not expose callbacks, event hooks, or metric counters. To monitor queue health, inspect the underlying Redis keys directly or parse log output.
@@ -312,10 +336,34 @@ await client.aclose()
 - **Claim-attempt loop limit of 100 per poll.** The VT claim Lua script attempts at most 100 LMOVE+delivery-count checks per invocation. Under pathological conditions (>100 consecutive poison messages in pending), a single poll returns no message even though non-poison messages exist deeper in the queue. Subsequent polls drain the poison batch 100 at a time.
 - **Cluster detection uses `isinstance(client, RedisCluster)`.** Wrapped or instrumented cluster clients that delegate without inheriting will bypass hash-tag validation. Custom gateways should set `is_redis_cluster = True` explicitly.
 - **Redis Cluster requires hash tags.** The built-in queue uses multiple Redis keys per operation. Wrap the queue name in hash tags (for example `{myqueue}`) so every generated key lands in the same slot. When you pass a Redis Cluster client to the built-in queue/gateway path, incompatible names are rejected early.
+- **Non-ASCII payloads use ~2x storage.** The default `ensure_ascii=True` in JSON serialization encodes non-ASCII characters as `\uXXXX` escape sequences. This is a deliberate compatibility choice.
 - **Client-side `Retry` can duplicate non-deduplicated publishes.** If you construct your `redis.Redis` client with `retry=Retry(...)`, redis-py retries `ConnectionError` / `TimeoutError` at the connection layer — *below* this library. Idempotent operations (deduplicated `publish()`, lease-scoped cleanup) are safe because their Lua scripts replay the original result. `add_message()` (used by `publish()` when `deduplication=False`) is a bare `LPUSH`: this library deliberately does not retry it, but a client-level `Retry` will, and if the server executed the command before the response was lost the message is enqueued twice. Leave `retry=None` (the default) if you need strict at-most-once semantics for non-deduplicated publishes, or accept the duplication risk. More broadly, any non-idempotent `LPUSH` path is vulnerable if the connection drops after server execution but before the client receives the response; all other built-in operations (deduplicated publish, lease-scoped ack/move, lease renewal) use replay markers and are safe under client-level `Retry`.
+- **Redis Cluster default retry can stack with this library's retry budget.** In redis-py 6.0+, `RedisCluster()` constructs a default `ExponentialWithJitterBackoff` retry below this library's `retry_budget_seconds`. If you need a single retry surface, pass `retry=Retry(NoBackoff(), 0)` to the cluster client or reduce `retry_budget_seconds` to account for the lower-level retry window.
 
 For a full analysis, see [docs/production-readiness.md](docs/production-readiness.md).
 
+## Upgrading
+
+### Configuration changes on live queues
+
+> **Warning:** These changes are destructive on live queues. Drain the queue completely before applying them.
+
+- **Do not change `name` or `key_separator` on a live queue.** Both settings define the Redis key namespace. Existing Redis keys become invisible to the new key scheme. Drain the queue completely before changing either value.
+- **Do not rename `dead_letter_queue` on a live queue.** Existing DLQ records stay in the old list, while new failures route to the new list. Inspect or drain the old DLQ manually before switching names.
+- **Do not toggle visibility timeout in either direction with messages in processing.** Messages claimed by non-VT consumers have no lease metadata, so VT-enabled consumers cannot reclaim them. Disabling VT later orphans existing lease deadline, lease token, and delivery count metadata and removes crash recovery for those in-flight messages. Drain the processing queue first.
+- **Reducing `max_delivery_count` retroactively DLQs messages.** The delivery count hash persists across restarts. Messages whose accumulated count exceeds the new limit are immediately dead-lettered on next claim.
+- **Changing `max_delivery_count` from a number to `None` leaves delivery metadata behind.** The delivery count hash continues to exist but is no longer consulted. Use this only after draining or after planning manual cleanup of the delivery-count hash.
+- **Changing `get_deduplication_key` changes the dedup keyspace.** Existing dedup records become inert for the duration of their TTL. Drain the queue or clear the old deduplication keys before switching between the default hash, explicit `None`, or a custom key function.
+- **Disabling `deduplication` has a retention-window overlap.** Existing dedup records remain in Redis until their TTL expires, but new publishes bypass them. Republishes that would have been suppressed under the old setting can enqueue during that window.
+- **Disabling `enable_failed_queue` stops recording handler failures.** Existing failed entries remain in Redis, but new failures are removed from `processing` without being appended to the failed queue. If `max_delivery_count=None` is also set, repeated handler failures can be dropped with no DLQ or failed-queue record; see [Dead-letter queue](#dead-letter-queue).
+- **Lowering `max_completed_length` or `max_failed_length` trims existing history.** The next completed or failed move calls `LTRIM`, so changing `None` to `N` or lowering `N` can immediately reduce historical entries to the new cap.
+- **Do not switch sync and async gateway instances mid-process while claims are active.** Redis state is compatible across deploys, but each gateway instance keeps its own pending claim-recovery IDs. In-flight claim recovery state does not transfer between instances.
+- **Switching between `gateway=` and `client=` can retarget the DLQ.** The built-in `client=` path derives the DLQ from the queue name. If a custom gateway used a different `dead_letter_queue`, switching paths has the same orphaning impact as renaming the DLQ.
+
+### v2 to v3 migration
+
+v3.0.0 replaced the `retry_strategy: Callable` constructor parameter with `retry_budget_seconds`, `retry_max_delay_seconds`, and `retry_initial_delay_seconds`. Users with custom retry strategies should subclass `AbstractRedisGateway` instead (see [Custom gateway](#custom-gateway)).
+
 ## Running locally
 
 You'll need a Redis server:

{redis_message_queue-3.1.1 → redis_message_queue-5.0.0}/README.md

@@ -1,6 +1,6 @@
 # redis-message-queue
 
-[![PyPI Version](https://img.shields.io/badge/v3.1.1-version?color=43cd0f&style=flat&label=pypi)](https://pypi.org/project/redis-message-queue)
+[![PyPI Version](https://img.shields.io/badge/v5.0.0-version?color=43cd0f&style=flat&label=pypi)](https://pypi.org/project/redis-message-queue)
 [![PyPI Downloads](https://img.shields.io/pypi/dm/redis-message-queue?color=43cd0f&style=flat&label=downloads)](https://pypistats.org/packages/redis-message-queue)
 [![License: MIT](https://img.shields.io/badge/License-MIT-43cd0f.svg?style=flat&label=license)](LICENSE)
 [![Maintained: yes](https://img.shields.io/badge/yes-43cd0f.svg?style=flat&label=maintained)](https://github.com/Elijas/redis-message-queue/issues)
@@ -24,7 +24,7 @@ Requires Redis server >= 6.2.
 from redis import Redis
 from redis_message_queue import RedisMessageQueue
 
-client = Redis.from_url("redis://localhost:6379/0")
+client = Redis.from_url("redis://localhost:6379/0", decode_responses=True)
 queue = RedisMessageQueue("my_queue", client=client, deduplication=True)
 
 queue.publish("order:1234")           # returns True
@@ -48,6 +48,9 @@ while True:
             # Auto-acknowledged on success; cleaned up on exception
 ```
 
+`RedisMessageQueue` itself is not a context manager. Use
+`with queue.process_message() as message:` for each message.
+
 ## Why redis-message-queue
 
 **The problem:** You're sending messages between services or workers and need guarantees. Simple Redis LPUSH/BRPOP loses messages on crashes, doesn't deduplicate, and gives you no visibility into what succeeded or failed.
@@ -71,8 +74,8 @@ All features are optional and can be enabled or disabled as needed.
 
 | Configuration | Delivery guarantee |
 |---|---|
-| Default (no visibility timeout) | **At-most-once** — a consumer crash loses the in-flight message |
-| With `visibility_timeout_seconds` | **At-least-once** — expired messages are reclaimed and redelivered |
+| Default (`visibility_timeout_seconds=300`) | **At-least-once** — expired messages are reclaimed and redelivered |
+| With `visibility_timeout_seconds=None` | **At-most-once** — a consumer crash loses the in-flight message |
 
 See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-timeout) for details and tradeoffs.
 
@@ -81,7 +84,7 @@ See [Crash recovery with visibility timeout](#crash-recovery-with-visibility-tim
 ### Deduplication
 
 ```python
-# Default: deduplicate by full message content (1-hour TTL)
+# Default: deduplicate by SHA-256 hash of canonical message content (1-hour TTL)
 queue = RedisMessageQueue("q", client=client, deduplication=True)
 
 # Custom dedup key (e.g., deduplicate by order ID only)
@@ -105,7 +108,8 @@ queue = RedisMessageQueue(
 )
 ```
 
-To prevent unbounded growth, cap the queue lengths:
+Completed and failed tracking queues are capped at 1,000 entries by default
+when enabled. Override the caps when you need a different retention window:
 
 ```python
 queue = RedisMessageQueue(
@@ -118,6 +122,8 @@ queue = RedisMessageQueue(
 ```
 
 When set, `LTRIM` is called after each message is moved to the completed/failed queue. This is best-effort cleanup — if the trim fails, the queue is slightly longer until the next successful trim.
+Pass `max_completed_length=None` or `max_failed_length=None` explicitly if you
+want unbounded tracking queues.
 
 ### Crash recovery with visibility timeout
 
@@ -165,14 +171,14 @@ queue = RedisMessageQueue(
 )
 ```
 
-When a message has been delivered more than `max_delivery_count` times (due to consumer crashes causing visibility-timeout reclaim), it is automatically routed to a dead-letter queue (`{name}::dead_letter`) instead of being redelivered. This prevents poison messages from cycling indefinitely.
+When a message has been delivered more than `max_delivery_count` times (due to consumer crashes causing visibility-timeout reclaim), it is automatically routed to a dead-letter queue (`{name}::dlq`) instead of being redelivered. `max_delivery_count` defaults to `10` on the built-in `client=` path, with the DLQ name auto-derived from the queue name. This prevents poison messages from cycling indefinitely.
 
 Notes:
 - requires `visibility_timeout_seconds` to be set (poison messages are only a concern with VT reclaim)
 - the delivery count is tracked per-message in a Redis HASH and cleaned up on successful ack or move to completed/failed
 - the delivery count increments when Redis grants the claim/lease, not when your handler begins running. If a process exits after Redis claims a message, that claim still counts toward `max_delivery_count`
 - `max_delivery_count=1` means the message is delivered once; any reclaim routes it to the dead-letter queue
-- without `max_delivery_count`, messages are redelivered indefinitely (existing behavior)
+- set `max_delivery_count=None` explicitly for unlimited redelivery
 - dead-lettered messages contain the **raw payload** only — the internal envelope (which carries a per-delivery UUID) is stripped before pushing to the DLQ, consistent with how completed/failed queues store messages. Two identical payloads dead-lettered separately are indistinguishable in the DLQ
 
 ### Graceful shutdown
@@ -201,7 +207,7 @@ while not interrupt.is_interrupted():
 ### Custom gateway
 
 ```python
-from redis_message_queue._redis_gateway import RedisGateway
+from redis_message_queue import RedisGateway
 
 # Tune retry budget, dedup TTL, or wait interval
 gateway = RedisGateway(
@@ -227,8 +233,8 @@ impossible. Note: tenacity may allow one additional attempt beyond the budget if
 
 To plug in a different retry library (`backoff`, `asyncstdlib.retry`, or your
 own logic) or fundamentally different semantics, subclass
-`AbstractRedisGateway` from `redis_message_queue._abstract_redis_gateway`
-(or `redis_message_queue.asyncio._abstract_redis_gateway`) and override the
+`AbstractRedisGateway` from `redis_message_queue` (or
+`redis_message_queue.asyncio` for the async sibling) and override the
 operation methods directly.
 
 If your custom gateway uses visibility timeouts, it must expose a public
@@ -259,6 +265,16 @@ Use a separate gateway instance per queue when `max_delivery_count` is enabled.
 Dead-letter routing is gateway-scoped, so reusing the same gateway across different
 queues is rejected.
 
+If you use Redis Sentinel, pass the Redis client returned by
+`sentinel.master_for(name)` to `client=` or `RedisGateway(redis_client=...)`, not
+the `sentinel` object itself.
+
+### Connection pool sizing
+
+Each queue with `heartbeat_interval_seconds` set uses up to 2 simultaneous
+connections: one for the main operation and one for heartbeat renewal. Size Redis
+client pools with `max_connections >= 2 * number_of_queues + headroom`.
+
 ## Async API
 
 Replace the import to use the async variant — the API is identical:
@@ -267,6 +283,11 @@ Replace the import to use the async variant — the API is identical:
 from redis_message_queue.asyncio import RedisMessageQueue
 ```
 
+The sync and async classes intentionally share names. In modules that use both,
+alias the imports explicitly, for example
+`from redis_message_queue import RedisMessageQueue as SyncRedisMessageQueue` and
+`from redis_message_queue.asyncio import RedisMessageQueue as AsyncRedisMessageQueue`.
+
 All examples work the same way. Remember to close the connection when done:
 
 ```python
@@ -277,6 +298,9 @@ client = redis.Redis()
 await client.aclose()
 ```
 
+For the sync Redis client, call `client.close()` during application shutdown when
+you own the client lifecycle.
+
 ## Known limitations
 
 - **No metrics or observability hooks.** The library logs warnings (stale leases, heartbeat failures, transient errors) via Python's `logging` module but does not expose callbacks, event hooks, or metric counters. To monitor queue health, inspect the underlying Redis keys directly or parse log output.
@@ -286,10 +310,34 @@ await client.aclose()
 - **Claim-attempt loop limit of 100 per poll.** The VT claim Lua script attempts at most 100 LMOVE+delivery-count checks per invocation. Under pathological conditions (>100 consecutive poison messages in pending), a single poll returns no message even though non-poison messages exist deeper in the queue. Subsequent polls drain the poison batch 100 at a time.
 - **Cluster detection uses `isinstance(client, RedisCluster)`.** Wrapped or instrumented cluster clients that delegate without inheriting will bypass hash-tag validation. Custom gateways should set `is_redis_cluster = True` explicitly.
 - **Redis Cluster requires hash tags.** The built-in queue uses multiple Redis keys per operation. Wrap the queue name in hash tags (for example `{myqueue}`) so every generated key lands in the same slot. When you pass a Redis Cluster client to the built-in queue/gateway path, incompatible names are rejected early.
+- **Non-ASCII payloads use ~2x storage.** The default `ensure_ascii=True` in JSON serialization encodes non-ASCII characters as `\uXXXX` escape sequences. This is a deliberate compatibility choice.
 - **Client-side `Retry` can duplicate non-deduplicated publishes.** If you construct your `redis.Redis` client with `retry=Retry(...)`, redis-py retries `ConnectionError` / `TimeoutError` at the connection layer — *below* this library. Idempotent operations (deduplicated `publish()`, lease-scoped cleanup) are safe because their Lua scripts replay the original result. `add_message()` (used by `publish()` when `deduplication=False`) is a bare `LPUSH`: this library deliberately does not retry it, but a client-level `Retry` will, and if the server executed the command before the response was lost the message is enqueued twice. Leave `retry=None` (the default) if you need strict at-most-once semantics for non-deduplicated publishes, or accept the duplication risk. More broadly, any non-idempotent `LPUSH` path is vulnerable if the connection drops after server execution but before the client receives the response; all other built-in operations (deduplicated publish, lease-scoped ack/move, lease renewal) use replay markers and are safe under client-level `Retry`.
+- **Redis Cluster default retry can stack with this library's retry budget.** In redis-py 6.0+, `RedisCluster()` constructs a default `ExponentialWithJitterBackoff` retry below this library's `retry_budget_seconds`. If you need a single retry surface, pass `retry=Retry(NoBackoff(), 0)` to the cluster client or reduce `retry_budget_seconds` to account for the lower-level retry window.
 
 For a full analysis, see [docs/production-readiness.md](docs/production-readiness.md).
 
+## Upgrading
+
+### Configuration changes on live queues
+
+> **Warning:** These changes are destructive on live queues. Drain the queue completely before applying them.
+
+- **Do not change `name` or `key_separator` on a live queue.** Both settings define the Redis key namespace. Existing Redis keys become invisible to the new key scheme. Drain the queue completely before changing either value.
+- **Do not rename `dead_letter_queue` on a live queue.** Existing DLQ records stay in the old list, while new failures route to the new list. Inspect or drain the old DLQ manually before switching names.
+- **Do not toggle visibility timeout in either direction with messages in processing.** Messages claimed by non-VT consumers have no lease metadata, so VT-enabled consumers cannot reclaim them. Disabling VT later orphans existing lease deadline, lease token, and delivery count metadata and removes crash recovery for those in-flight messages. Drain the processing queue first.
+- **Reducing `max_delivery_count` retroactively DLQs messages.** The delivery count hash persists across restarts. Messages whose accumulated count exceeds the new limit are immediately dead-lettered on next claim.
+- **Changing `max_delivery_count` from a number to `None` leaves delivery metadata behind.** The delivery count hash continues to exist but is no longer consulted. Use this only after draining or after planning manual cleanup of the delivery-count hash.
+- **Changing `get_deduplication_key` changes the dedup keyspace.** Existing dedup records become inert for the duration of their TTL. Drain the queue or clear the old deduplication keys before switching between the default hash, explicit `None`, or a custom key function.
+- **Disabling `deduplication` has a retention-window overlap.** Existing dedup records remain in Redis until their TTL expires, but new publishes bypass them. Republishes that would have been suppressed under the old setting can enqueue during that window.
+- **Disabling `enable_failed_queue` stops recording handler failures.** Existing failed entries remain in Redis, but new failures are removed from `processing` without being appended to the failed queue. If `max_delivery_count=None` is also set, repeated handler failures can be dropped with no DLQ or failed-queue record; see [Dead-letter queue](#dead-letter-queue).
+- **Lowering `max_completed_length` or `max_failed_length` trims existing history.** The next completed or failed move calls `LTRIM`, so changing `None` to `N` or lowering `N` can immediately reduce historical entries to the new cap.
+- **Do not switch sync and async gateway instances mid-process while claims are active.** Redis state is compatible across deploys, but each gateway instance keeps its own pending claim-recovery IDs. In-flight claim recovery state does not transfer between instances.
+- **Switching between `gateway=` and `client=` can retarget the DLQ.** The built-in `client=` path derives the DLQ from the queue name. If a custom gateway used a different `dead_letter_queue`, switching paths has the same orphaning impact as renaming the DLQ.
+
+### v2 to v3 migration
+
+v3.0.0 replaced the `retry_strategy: Callable` constructor parameter with `retry_budget_seconds`, `retry_max_delay_seconds`, and `retry_initial_delay_seconds`. Users with custom retry strategies should subclass `AbstractRedisGateway` instead (see [Custom gateway](#custom-gateway)).
+
 ## Running locally
 
 You'll need a Redis server:

{redis_message_queue-3.1.1 → redis_message_queue-5.0.0}/pyproject.toml

@@ -1,10 +1,11 @@
 [tool.poetry]
 name = "redis-message-queue"
-version = "3.1.1"
+version = "5.0.0"
 description = "Python message queuing with Redis and message deduplication"
 authors = ["Elijas <4084885+Elijas@users.noreply.github.com>"]
 readme = "README.md"
 license = "MIT"
+include = ["redis_message_queue/py.typed"]
 keywords = ["redis", "message-queue", "deduplication", "task-queue"]
 classifiers = [
     "Development Status :: 5 - Production/Stable",
@@ -46,6 +47,16 @@ ignore = ["E731"]
 
 [tool.pytest.ini_options]
 asyncio_default_fixture_loop_scope = "function"
+filterwarnings = [
+    # Channel-rule warnings are emitted at user-actionable failure sites
+    # (see _STALE_LEASE_*_WARNING, heartbeat-failure, etc.). Dedicated B8
+    # tests assert emission via pytest.warns(); other tests trigger these
+    # warnings incidentally as a side effect of exercising lifecycle paths
+    # and would fail under "error::" promotion. Ignore at the default
+    # filter so emission is silent in incidental tests but pytest.warns
+    # assertions still verify emission where intended.
+    "ignore::RuntimeWarning:redis_message_queue",
+]
 markers = [
     "integration: tests that require a real Redis server",
 ]

{redis_message_queue-3.1.1 → redis_message_queue-5.0.0}/redis_message_queue/__init__.py

@@ -1,4 +1,5 @@
 from redis_message_queue._abstract_redis_gateway import AbstractRedisGateway
+from redis_message_queue._redis_gateway import RedisGateway
 from redis_message_queue._stored_message import ClaimedMessage, MessageData
 from redis_message_queue.interrupt_handler import (
     BaseGracefulInterruptHandler,
@@ -8,6 +9,7 @@ from redis_message_queue.redis_message_queue import RedisMessageQueue
 
 __all__ = [
     "RedisMessageQueue",
+    "RedisGateway",
     "AbstractRedisGateway",
     "ClaimedMessage",
     "MessageData",

{redis_message_queue-3.1.1 → redis_message_queue-5.0.0}/redis_message_queue/_abstract_redis_gateway.py

@@ -20,12 +20,18 @@ class AbstractRedisGateway(ABC):
     causing the queue to treat the gateway as a non-lease implementation.
 
     The queue also reads ``max_delivery_count`` and ``dead_letter_queue``
-    from the gateway via ``getattr``. Avoid using these attribute names for
-    unrelated purposes on custom gateway implementations.
-
-    Gateways that wrap a Redis Cluster client should expose an
-    ``is_redis_cluster`` property returning ``True`` so the queue can apply
-    hash-tag validation at construction time.
+    from the gateway. The abstract base provides ``None`` defaults via
+    ``@property``; lease + DLQ-enabled custom gateways MUST override both
+    to enable poison-message routing. Avoid using these attribute names
+    for unrelated purposes on custom gateway implementations.
+    When DLQ routing is enabled, the queue also attaches an internal
+    ``_rmq_bound_pending_queue`` attribute to gateway instances to reject
+    reusing one DLQ-bound gateway across multiple pending queues.
+
+    Gateways that wrap a Redis Cluster client should override the
+    ``is_redis_cluster`` property to return ``True`` so the queue can
+    apply hash-tag validation at construction time. The abstract base
+    provides ``False`` as the default; non-cluster gateways inherit it.
 
     Concurrency
     -----------
@@ -35,8 +41,29 @@ class AbstractRedisGateway(ABC):
     ``remove_message``. Implementations must be safe for concurrent calls
     across these methods. The built-in gateway achieves this via atomic
     Lua scripts.
+
+    Server-side atomicity (Lua scripts, redis-py transactions, or a
+    single-command-per-mutation discipline) is the recommended pattern.
+    Python-level locks (``threading.Lock``, ``asyncio.Lock``) shared across
+    ``renew_message_lease`` and ``move_message`` / ``remove_message`` are an
+    anti-pattern: they serialize without giving Redis-server-side atomicity,
+    leave partial-failure orphans, and can deadlock against the heartbeat
+    lifecycle (the heartbeat is awaited from the same finally block that
+    issues the cleanup move/remove).
     """
 
+    @property
+    def is_redis_cluster(self) -> bool:
+        return False
+
+    @property
+    def max_delivery_count(self) -> int | None:
+        return None
+
+    @property
+    def dead_letter_queue(self) -> str | None:
+        return None
+
     @abstractmethod
     def publish_message(self, queue: str, message: str, dedup_key: str) -> bool:
         """Publish a message with deduplication.
@@ -48,7 +75,20 @@ class AbstractRedisGateway(ABC):
 
     @abstractmethod
     def add_message(self, queue: str, message: str) -> None:
-        """Unconditionally enqueue a message. No deduplication is performed."""
+        """Unconditionally enqueue a message. No deduplication is performed.
+
+        This library deliberately does not wrap the underlying enqueue in a
+        retry — retrying after the server may already have executed the
+        command can silently duplicate the message. The caller can still
+        retry (accepting duplicates).
+
+        Note: a client-level retry policy bypasses this guarantee. If the
+        underlying ``redis.Redis`` / ``redis.asyncio.Redis`` client was
+        constructed with ``retry=Retry(...)``, redis-py retries on
+        ``ConnectionError`` / ``TimeoutError`` below this call and may
+        duplicate. Pass ``retry=None`` (the default) when strict at-most-once
+        is required for non-deduplicated publishes.
+        """
 
     @abstractmethod
     def move_message(
@@ -128,6 +168,15 @@ class AbstractRedisGateway(ABC):
         use leases.
 
         Return None if no message was available (e.g. timeout or interrupt).
+
+        Implementations MUST respect a reasonable timeout or return None
+        periodically so the consumer can check for interrupts. Blocking
+        indefinitely without returning prevents graceful shutdown. As a
+        concrete reference, the built-in gateway uses a 5s outer wait
+        decomposed into 0.25s polling steps (see
+        ``_VISIBILITY_TIMEOUT_POLL_INTERVAL_SECONDS``); custom
+        implementations should keep the longest single block below ~5s so
+        an interrupt is observed within one polling step.
         """
 
     @abstractmethod