redqueue 0.13.1__tar.gz → 0.14.0__tar.gz

{redqueue-0.13.1 → redqueue-0.14.0}/CHANGELOG.md

@@ -7,26 +7,44 @@ All notable public release changes are documented here.
 Development versions are tracked separately from formal release versions.
 开发版本与正式版本分开管理。
 
-## [0.13.1] - 2026-06-27
-
-### Fixed
+## [0.14.0] - 2026-06-27
 
-- Fixed a compatibility regression where legacy messages with blank
-  `headers["trace_id"]` values could fail to construct or consume after
-  upgrading to `0.13.0`.
-- Fixed sync and async `publish(..., delay=..., message_id=...)` so delayed
-  publishing preserves caller-provided message ids.
+### Added
 
-### 修复
+- Added opt-in Redis-backed message deduplication with
+  `DeduplicationConfig`.
+- Added `dedup_key` support to sync and async `publish()` and `delay()`.
+- Added duplicate-hit behavior that returns the first message id without
+  enqueueing or scheduling another message.
+- Added `message.deduplicated` monitoring events.
 
-- 修复升级到 `0.13.0` 后，历史消息中空白 `headers["trace_id"]` 可能导致
-  消息构造或消费失败的兼容性回归。
-- 修复同步和异步 `publish(..., delay=..., message_id=...)`，现在延迟发布会
-  保留调用方传入的消息 ID。
+### 新增
 
-## [0.13.0] - 2026-06-27
+- 新增基于 Redis 的可选消息去重能力，通过 `DeduplicationConfig` 启用。
+- 同步和异步 `publish()`、`delay()` 新增 `dedup_key` 参数。
+- 重复命中时返回第一次消息 ID，不再重复入队或重复调度。
+- 新增 `message.deduplicated` 监控事件。
 
-### Added
+## [0.13.1] - 2026-06-27
+
+### Fixed
+
+- Fixed a compatibility regression where legacy messages with blank
+  `headers["trace_id"]` values could fail to construct or consume after
+  upgrading to `0.13.0`.
+- Fixed sync and async `publish(..., delay=..., message_id=...)` so delayed
+  publishing preserves caller-provided message ids.
+
+### 修复
+
+- 修复升级到 `0.13.0` 后，历史消息中空白 `headers["trace_id"]` 可能导致
+  消息构造或消费失败的兼容性回归。
+- 修复同步和异步 `publish(..., delay=..., message_id=...)`，现在延迟发布会
+  保留调用方传入的消息 ID。
+
+## [0.13.0] - 2026-06-27
+
+### Added
 
 - Added first-class `trace_id` support to `Message`, sync and async
   `publish()`, sync and async `delay()`, List, Streams, and delayed task

{redqueue-0.13.1 → redqueue-0.14.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: redqueue
-Version: 0.13.1
-Summary: Redis-backed Python message queue library with List, Streams, delayed tasks, and monitoring.
+Version: 0.14.0
+Summary: Redis-backed Python message queue library with List, Streams, delayed tasks, deduplication, and monitoring.
 Project-URL: Homepage, https://github.com/SpringMirror-pear/redqueue
 Project-URL: Repository, https://github.com/SpringMirror-pear/redqueue.git
 Project-URL: Issues, https://github.com/SpringMirror-pear/redqueue/issues
@@ -52,6 +52,7 @@ https://github.com/SpringMirror-pear/redqueue.git
 - Redis connection pool managers for shared sync and async resources.
 - `redqueue` CLI for local debugging and operational checks.
 - First-class `trace_id` propagation for lifecycle tracing.
+- Opt-in message deduplication with Redis `SET NX` and TTL windows.
 - Unified exception hierarchy with structured context.
 - Monitoring events for publish, consume, ack, nack, retry, dead letter, delay,
   and backend errors.
@@ -75,6 +76,7 @@ Redis:
 | Streams | `>=5.0` | Uses `XADD`, `XGROUP CREATE`, `XREADGROUP` |
 | Streams auto claim | `>=6.2` | Uses `XAUTOCLAIM`; Redis 5.x uses `XPENDING`/`XCLAIM` fallback |
 | Delayed tasks | `>=1.2` | Uses `ZADD` and timestamp scores |
+| Message deduplication | `>=2.6.12` | Uses `SET` with `NX` and `EX`/`PX` |
 
 ## Installation
 
@@ -210,6 +212,23 @@ client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123"
 released = client.schedule_due(limit=100)
 ```
 
+Message deduplication:
+
+```python
+from redqueue import DeduplicationConfig, QueueClient
+
+client = QueueClient.from_url(
+    "redis://127.0.0.1:6379/0",
+    queue="emails",
+    deduplication=DeduplicationConfig(enabled=True, ttl_seconds=3600),
+)
+
+first_id = client.publish({"order": 1}, dedup_key="order-1")
+second_id = client.publish({"order": 1}, dedup_key="order-1")
+
+assert second_id == first_id
+```
+
 Trace IDs:
 
 ```python
@@ -367,17 +386,21 @@ REDQUEUE_REDIS_URL=redis://127.0.0.1:6379/0 PYTHONPATH=src python -m pytest -m "
 
 Latest local run on Python `3.14.5`:
 
-- Full test suite without `REDQUEUE_REDIS_URL`: `102 passed, 8 skipped`.
-- Real Redis availability suite: `3 passed` with
+- Full test suite without `REDQUEUE_REDIS_URL`: `114 passed, 10 skipped`.
+- Real Redis integration suite: `10 passed` with
+  `redis://127.0.0.1:6379/0`.
+- Real Redis availability suite: `5 passed` with
   `redis://127.0.0.1:6379/0`.
 - Real Redis server: Redis for Windows `5.0.14.1`.
 - Availability suite: covers List processing recovery, List dead-letter
   requeue, Streams Redis `<5.0` compatibility rejection, Streams Redis 5.x
   pending recovery fallback, Streams dead-letter requeue, delayed task publish
-  failure rollback, missing delayed payload errors, async List recovery, async
-  Streams dead-letter requeue, and async delayed task rollback.
+  failure rollback, missing delayed payload errors, publish-time deduplication,
+  delayed task deduplication, async List recovery, async Streams dead-letter
+  requeue, and async delayed task rollback.
 - Real Redis availability suite additionally validates List recovery, Streams
-  dead-letter requeue, and delayed task rollback against a running Redis server.
+  dead-letter requeue, delayed task rollback, List deduplication, and delayed
+  task deduplication against a running Redis server.
 
 ## Performance Results
 

{redqueue-0.13.1 → redqueue-0.14.0}/README-zh-CN.md

@@ -15,10 +15,11 @@ https://github.com/SpringMirror-pear/redqueue.git
 - 基于 Redis Streams 的消费组后端，Streams 要求 Redis `>=5.0`。
 - 基于 Redis Sorted Set 的延迟任务。
 - 同步客户端 `QueueClient` 与异步客户端 `AsyncQueueClient`。
-- 支持同步和异步 Redis 连接池管理器，方便多个客户端共享连接池。
-- 提供 `redqueue` CLI，用于本地调试和运行时检查。
-- 一等 `trace_id` 链路追踪能力，贯穿消息生命周期。
-- 带结构化上下文的统一异常体系。
+- 支持同步和异步 Redis 连接池管理器，方便多个客户端共享连接池。
+- 提供 `redqueue` CLI，用于本地调试和运行时检查。
+- 一等 `trace_id` 链路追踪能力，贯穿消息生命周期。
+- 可选消息去重能力，基于 Redis `SET NX` 和 TTL 窗口。
+- 带结构化上下文的统一异常体系。
 - 针对发布、消费、确认、拒绝、重试、死信、延迟和后端错误的监控事件。
 - 通过 `INFO server` 探测 Redis 能力。
 - Apache License 2.0。
@@ -37,9 +38,10 @@ Redis：
 | --- | --- | --- |
 | List 阻塞消费 | `>=2.0` | 以 `BLPOP` 系列能力为基础 |
 | List 可靠搬移 | `>=2.2` | 使用 `BRPOPLPUSH`；Redis `>=6.2` 优先使用 `BLMOVE` |
-| Streams | `>=5.0` | 使用 `XADD`、`XGROUP CREATE`、`XREADGROUP` |
-| Streams 自动认领 | `>=6.2` | 使用 `XAUTOCLAIM`；Redis 5.x 回退 `XPENDING`/`XCLAIM` |
-| 延迟任务 | `>=1.2` | 使用 `ZADD` 和时间戳 score |
+| Streams | `>=5.0` | 使用 `XADD`、`XGROUP CREATE`、`XREADGROUP` |
+| Streams 自动认领 | `>=6.2` | 使用 `XAUTOCLAIM`；Redis 5.x 回退 `XPENDING`/`XCLAIM` |
+| 延迟任务 | `>=1.2` | 使用 `ZADD` 和时间戳 score |
+| 消息去重 | `>=2.6.12` | 使用带 `NX` 和 `EX`/`PX` 的 `SET` |
 
 ## 安装
 
@@ -165,17 +167,34 @@ async def main() -> None:
 asyncio.run(main())
 ```
 
-延迟任务：
+延迟任务：
 
 ```python
 from redqueue import QueueClient
 
 client = QueueClient.from_url("redis://127.0.0.1:6379/0", queue="emails")
 client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123")
-released = client.schedule_due(limit=100)
-```
-
-链路追踪：
+released = client.schedule_due(limit=100)
+```
+
+消息去重：
+
+```python
+from redqueue import DeduplicationConfig, QueueClient
+
+client = QueueClient.from_url(
+    "redis://127.0.0.1:6379/0",
+    queue="emails",
+    deduplication=DeduplicationConfig(enabled=True, ttl_seconds=3600),
+)
+
+first_id = client.publish({"order": 1}, dedup_key="order-1")
+second_id = client.publish({"order": 1}, dedup_key="order-1")
+
+assert second_id == first_id
+```
+
+链路追踪：
 
 ```python
 from redqueue import InMemoryMonitoringHook, QueueClient, new_trace_id
@@ -329,15 +348,16 @@ REDQUEUE_REDIS_URL=redis://127.0.0.1:6379/0 PYTHONPATH=src python -m pytest -m "
 
 Python `3.14.5` 最新本地运行结果：
 
-- 未设置 `REDQUEUE_REDIS_URL` 时的完整测试套件：`102 passed, 8 skipped`。
-- 真实 Redis 可用性套件：使用 `redis://127.0.0.1:6379/0` 时 `3 passed`。
+- 未设置 `REDQUEUE_REDIS_URL` 时的完整测试套件：`114 passed, 10 skipped`。
+- 真实 Redis 集成套件：使用 `redis://127.0.0.1:6379/0` 时 `10 passed`。
+- 真实 Redis 可用性套件：使用 `redis://127.0.0.1:6379/0` 时 `5 passed`。
 - 真实 Redis 服务端：Redis for Windows `5.0.14.1`。
 - 可用性套件覆盖：List processing 恢复、List 死信重放、Streams Redis
   `<5.0` 兼容性拒绝、Streams Redis 5.x pending 恢复回退、Streams 死信
-  重放、延迟任务发布失败回滚、延迟 payload 缺失错误、异步 List 恢复、异步
-  Streams 死信重放和异步延迟任务回滚。
-- 真实 Redis 可用性套件额外验证了运行中 Redis 服务上的 List 恢复、Streams
-  死信重放和延迟任务回滚。
+  重放、延迟任务发布失败回滚、延迟 payload 缺失错误、发布时去重、延迟任务
+  去重、异步 List 恢复、异步 Streams 死信重放和异步延迟任务回滚。
+- 真实 Redis 可用性套件额外验证了运行中 Redis 服务上的 List 恢复、Streams
+  死信重放、延迟任务回滚、List 去重和延迟任务去重。
 
 ## 性能测试结果
 

{redqueue-0.13.1 → redqueue-0.14.0}/README.md

@@ -16,10 +16,11 @@ https://github.com/SpringMirror-pear/redqueue.git
 - Redis Streams backend with consumer groups. Streams require Redis `>=5.0`.
 - Delayed tasks based on Redis Sorted Set.
 - Sync client `QueueClient` and async client `AsyncQueueClient`.
-- Redis connection pool managers for shared sync and async resources.
-- `redqueue` CLI for local debugging and operational checks.
-- First-class `trace_id` propagation for lifecycle tracing.
-- Unified exception hierarchy with structured context.
+- Redis connection pool managers for shared sync and async resources.
+- `redqueue` CLI for local debugging and operational checks.
+- First-class `trace_id` propagation for lifecycle tracing.
+- Opt-in message deduplication with Redis `SET NX` and TTL windows.
+- Unified exception hierarchy with structured context.
 - Monitoring events for publish, consume, ack, nack, retry, dead letter, delay,
   and backend errors.
 - Redis capability detection from `INFO server`.
@@ -39,9 +40,10 @@ Redis:
 | --- | --- | --- |
 | List blocking consume | `>=2.0` | Uses `BLPOP` family compatibility baseline |
 | List reliable move | `>=2.2` | Uses `BRPOPLPUSH`; `BLMOVE` preferred on `>=6.2` |
-| Streams | `>=5.0` | Uses `XADD`, `XGROUP CREATE`, `XREADGROUP` |
-| Streams auto claim | `>=6.2` | Uses `XAUTOCLAIM`; Redis 5.x uses `XPENDING`/`XCLAIM` fallback |
-| Delayed tasks | `>=1.2` | Uses `ZADD` and timestamp scores |
+| Streams | `>=5.0` | Uses `XADD`, `XGROUP CREATE`, `XREADGROUP` |
+| Streams auto claim | `>=6.2` | Uses `XAUTOCLAIM`; Redis 5.x uses `XPENDING`/`XCLAIM` fallback |
+| Delayed tasks | `>=1.2` | Uses `ZADD` and timestamp scores |
+| Message deduplication | `>=2.6.12` | Uses `SET` with `NX` and `EX`/`PX` |
 
 ## Installation
 
@@ -167,17 +169,34 @@ async def main() -> None:
 asyncio.run(main())
 ```
 
-Delayed task:
+Delayed task:
 
 ```python
 from redqueue import QueueClient
 
 client = QueueClient.from_url("redis://127.0.0.1:6379/0", queue="emails")
 client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123")
-released = client.schedule_due(limit=100)
-```
-
-Trace IDs:
+released = client.schedule_due(limit=100)
+```
+
+Message deduplication:
+
+```python
+from redqueue import DeduplicationConfig, QueueClient
+
+client = QueueClient.from_url(
+    "redis://127.0.0.1:6379/0",
+    queue="emails",
+    deduplication=DeduplicationConfig(enabled=True, ttl_seconds=3600),
+)
+
+first_id = client.publish({"order": 1}, dedup_key="order-1")
+second_id = client.publish({"order": 1}, dedup_key="order-1")
+
+assert second_id == first_id
+```
+
+Trace IDs:
 
 ```python
 from redqueue import InMemoryMonitoringHook, QueueClient, new_trace_id
@@ -334,17 +353,21 @@ REDQUEUE_REDIS_URL=redis://127.0.0.1:6379/0 PYTHONPATH=src python -m pytest -m "
 
 Latest local run on Python `3.14.5`:
 
-- Full test suite without `REDQUEUE_REDIS_URL`: `102 passed, 8 skipped`.
-- Real Redis availability suite: `3 passed` with
-  `redis://127.0.0.1:6379/0`.
+- Full test suite without `REDQUEUE_REDIS_URL`: `114 passed, 10 skipped`.
+- Real Redis integration suite: `10 passed` with
+  `redis://127.0.0.1:6379/0`.
+- Real Redis availability suite: `5 passed` with
+  `redis://127.0.0.1:6379/0`.
 - Real Redis server: Redis for Windows `5.0.14.1`.
 - Availability suite: covers List processing recovery, List dead-letter
   requeue, Streams Redis `<5.0` compatibility rejection, Streams Redis 5.x
   pending recovery fallback, Streams dead-letter requeue, delayed task publish
-  failure rollback, missing delayed payload errors, async List recovery, async
-  Streams dead-letter requeue, and async delayed task rollback.
-- Real Redis availability suite additionally validates List recovery, Streams
-  dead-letter requeue, and delayed task rollback against a running Redis server.
+  failure rollback, missing delayed payload errors, publish-time deduplication,
+  delayed task deduplication, async List recovery, async Streams dead-letter
+  requeue, and async delayed task rollback.
+- Real Redis availability suite additionally validates List recovery, Streams
+  dead-letter requeue, delayed task rollback, List deduplication, and delayed
+  task deduplication against a running Redis server.
 
 ## Performance Results
 

{redqueue-0.13.1 → redqueue-0.14.0}/docs/API.md

@@ -1,8 +1,8 @@
 # RedQueue API / RedQueue API 文档
 
-This document describes the public API available in RedQueue `0.13.1`.
+This document describes the public API available in RedQueue `0.14.0`.
 
-本文档描述 RedQueue `0.13.1` 的公开 API。
+本文档描述 RedQueue `0.14.0` 的公开 API。
 
 ## Clients / 客户端
 
@@ -27,12 +27,12 @@ Methods / 方法：
 - `from_url(url, *, queue, backend="list", connection_manager=None, **options) -> QueueClient`
   - Advanced options include `pool_options`, injected `redis`, injected
     `capabilities`, and `owns_redis`.
-- `publish(payload, *, delay=None, headers=None, message_id=None, trace_id=None) -> str`
+- `publish(payload, *, delay=None, headers=None, message_id=None, trace_id=None, dedup_key=None) -> str`
 - `consume(*, timeout=None, batch_size=1) -> Message | list[Message] | None`
 - `ack(message) -> None`
 - `nack(message, *, requeue=True) -> None`
 - `retry(message, *, delay=None, reason=None) -> None`
-- `delay(payload, *, delay_seconds=None, run_at=None, headers=None, message_id=None, trace_id=None) -> str`
+- `delay(payload, *, delay_seconds=None, run_at=None, headers=None, message_id=None, trace_id=None, dedup_key=None) -> str`
 - `schedule_due(*, limit=100, now=None) -> int`
 - `recover_stale(*, min_idle_ms=None, limit=100) -> int`
 - `dead_letters(*, limit=100) -> list[Message]`
@@ -61,12 +61,12 @@ Methods / 方法：
 - `await from_url(url, *, queue, backend="list", connection_manager=None, **options) -> AsyncQueueClient`
   - Advanced options include `pool_options`, injected `redis`, injected
     `capabilities`, and `owns_redis`.
-- `await publish(payload, *, delay=None, headers=None, message_id=None, trace_id=None) -> str`
+- `await publish(payload, *, delay=None, headers=None, message_id=None, trace_id=None, dedup_key=None) -> str`
 - `await consume(*, timeout=None, batch_size=1) -> Message | list[Message] | None`
 - `await ack(message) -> None`
 - `await nack(message, *, requeue=True) -> None`
 - `await retry(message, *, delay=None, reason=None) -> None`
-- `await delay(payload, *, delay_seconds=None, run_at=None, headers=None, message_id=None, trace_id=None) -> str`
+- `await delay(payload, *, delay_seconds=None, run_at=None, headers=None, message_id=None, trace_id=None, dedup_key=None) -> str`
 - `await schedule_due(*, limit=100, now=None) -> int`
 - `await recover_stale(*, min_idle_ms=None, limit=100) -> int`
 - `await dead_letters(*, limit=100) -> list[Message]`
@@ -147,9 +147,10 @@ Important fields / 重要字段：
 - `queue: str`
 - `backend: BackendType | str = BackendType.LIST`
 - `enable_delay: bool = False`
-- `namespace: str = "rq"`
-- `retry: RetryConfig`
-- `monitoring: MonitoringHook`
+- `namespace: str = "rq"`
+- `retry: RetryConfig`
+- `deduplication: DeduplicationConfig`
+- `monitoring: MonitoringHook`
 - `serializer: Serializer`
 - `visibility_timeout_seconds: float = 300.0`
 - `consumer_group: str = "redqueue"`
@@ -165,9 +166,44 @@ Retry policy for failed messages.
 - `max_retries: int = 3`
 - `base_delay_seconds: float = 0.0`
 - `max_delay_seconds: float | None = None`
-- `next_delay(attempts: int) -> float`
-
-### `BackendType`
+- `next_delay(attempts: int) -> float`
+
+### `DeduplicationConfig`
+
+Opt-in publish-time deduplication window.
+
+可选的发布时去重窗口配置。
+
+- `enabled: bool = False`
+- `ttl_seconds: float = 86400.0`
+
+When enabled, callers may pass `dedup_key` to `publish()` or `delay()`. RedQueue
+stores the first message id in Redis with `SET NX` and the configured TTL.
+Duplicate calls with the same key return the stored message id and do not
+enqueue or schedule another message. Deduplication keys are not deleted on
+`ack()`; the TTL controls the deduplication window.
+
+启用后，调用方可以在 `publish()` 或 `delay()` 中传入 `dedup_key`。RedQueue
+会用 Redis `SET NX` 和配置的 TTL 存储第一次消息 ID。相同 key 的重复调用会
+返回已存储的消息 ID，不会再次入队或调度。去重 key 不会在 `ack()` 时删除；
+TTL 控制去重窗口。
+
+```python
+from redqueue import DeduplicationConfig, QueueClient
+
+client = QueueClient.from_url(
+    "redis://127.0.0.1:6379/0",
+    queue="emails",
+    deduplication=DeduplicationConfig(enabled=True, ttl_seconds=3600),
+)
+
+first_id = client.publish({"order": 1}, dedup_key="order-1")
+second_id = client.publish({"order": 1}, dedup_key="order-1")
+
+assert second_id == first_id
+```
+
+### `BackendType`
 
 Supported backends.
 
@@ -309,9 +345,10 @@ All RedQueue custom exceptions inherit from `RedQueueError`.
 - `message.consumed`
 - `message.acked`
 - `message.nacked`
-- `message.retried`
-- `message.dead_lettered`
-- `delay.scheduled`
+- `message.retried`
+- `message.dead_lettered`
+- `message.deduplicated`
+- `delay.scheduled`
 - `delay.released`
 - `backend.error`
 
@@ -345,10 +382,10 @@ Streams are rejected with `RedisCompatibilityError` when Redis is below `5.0`.
 
 ## CLI / 命令行工具
 
-RedQueue `0.13.1` provides a `redqueue` console command and a
+RedQueue `0.14.0` provides a `redqueue` console command and a
 `python -m redqueue` module entry point for developer diagnostics.
 
-RedQueue `0.13.1` 提供 `redqueue` 控制台命令和 `python -m redqueue`
+RedQueue `0.14.0` 提供 `redqueue` 控制台命令和 `python -m redqueue`
 模块入口，用于开发者调试。
 
 Commands / 命令：