redqueue 0.13.0__tar.gz → 0.14.0__tar.gz

{redqueue-0.13.0 → redqueue-0.14.0}/CHANGELOG.md

@@ -7,35 +7,70 @@ All notable public release changes are documented here.
 Development versions are tracked separately from formal release versions.
 开发版本与正式版本分开管理。
 
-## [0.13.0] - 2026-06-27
+## [0.14.0] - 2026-06-27
 
 ### Added
 
-- Added first-class `trace_id` support to `Message`, sync and async
-  `publish()`, sync and async `delay()`, List, Streams, and delayed task
-  backends.
-- Added trace propagation through message envelopes, retry, dead-letter,
-  delayed release, and requeue flows while remaining compatible with existing
-  `headers["trace_id"]` messages.
-- Added `trace_id` to monitoring events and CLI message output for lifecycle
-  correlation.
-- Added CLI `--trace-id` support for `publish` and `delay`.
-- Added `new_trace_id()` helper for callers that want RedQueue-generated trace
-  identifiers.
+- Added opt-in Redis-backed message deduplication with
+  `DeduplicationConfig`.
+- Added `dedup_key` support to sync and async `publish()` and `delay()`.
+- Added duplicate-hit behavior that returns the first message id without
+  enqueueing or scheduling another message.
+- Added `message.deduplicated` monitoring events.
 
 ### 新增
 
-- `Message`、同步和异步 `publish()`、同步和异步 `delay()`、List、Streams
-  和延迟任务后端新增一等 `trace_id` 支持。
-- trace 会穿过消息 envelope、重试、死信、延迟释放和重放流程，并兼容已有的
-  `headers["trace_id"]` 消息。
-- 监控事件和 CLI 消息输出新增 `trace_id`，方便按链路聚合生命周期事件。
-- CLI 的 `publish` 和 `delay` 命令新增 `--trace-id`。
-- 新增 `new_trace_id()` helper，方便调用方生成 RedQueue trace 标识。
+- 新增基于 Redis 的可选消息去重能力，通过 `DeduplicationConfig` 启用。
+- 同步和异步 `publish()`、`delay()` 新增 `dedup_key` 参数。
+- 重复命中时返回第一次消息 ID，不再重复入队或重复调度。
+- 新增 `message.deduplicated` 监控事件。
 
-## [0.12.0] - 2026-06-23
-
-### Added
+## [0.13.1] - 2026-06-27
+
+### Fixed
+
+- Fixed a compatibility regression where legacy messages with blank
+  `headers["trace_id"]` values could fail to construct or consume after
+  upgrading to `0.13.0`.
+- Fixed sync and async `publish(..., delay=..., message_id=...)` so delayed
+  publishing preserves caller-provided message ids.
+
+### 修复
+
+- 修复升级到 `0.13.0` 后，历史消息中空白 `headers["trace_id"]` 可能导致
+  消息构造或消费失败的兼容性回归。
+- 修复同步和异步 `publish(..., delay=..., message_id=...)`，现在延迟发布会
+  保留调用方传入的消息 ID。
+
+## [0.13.0] - 2026-06-27
+
+### Added
+
+- Added first-class `trace_id` support to `Message`, sync and async
+  `publish()`, sync and async `delay()`, List, Streams, and delayed task
+  backends.
+- Added trace propagation through message envelopes, retry, dead-letter,
+  delayed release, and requeue flows while remaining compatible with existing
+  `headers["trace_id"]` messages.
+- Added `trace_id` to monitoring events and CLI message output for lifecycle
+  correlation.
+- Added CLI `--trace-id` support for `publish` and `delay`.
+- Added `new_trace_id()` helper for callers that want RedQueue-generated trace
+  identifiers.
+
+### 新增
+
+- `Message`、同步和异步 `publish()`、同步和异步 `delay()`、List、Streams
+  和延迟任务后端新增一等 `trace_id` 支持。
+- trace 会穿过消息 envelope、重试、死信、延迟释放和重放流程，并兼容已有的
+  `headers["trace_id"]` 消息。
+- 监控事件和 CLI 消息输出新增 `trace_id`，方便按链路聚合生命周期事件。
+- CLI 的 `publish` 和 `delay` 命令新增 `--trace-id`。
+- 新增 `new_trace_id()` helper，方便调用方生成 RedQueue trace 标识。
+
+## [0.12.0] - 2026-06-23
+
+### Added
 
 - Added the `redqueue` CLI and `python -m redqueue` module entry point for
   developer debugging.

{redqueue-0.13.0 → redqueue-0.14.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: redqueue
-Version: 0.13.0
-Summary: Redis-backed Python message queue library with List, Streams, delayed tasks, and monitoring.
+Version: 0.14.0
+Summary: Redis-backed Python message queue library with List, Streams, delayed tasks, deduplication, and monitoring.
 Project-URL: Homepage, https://github.com/SpringMirror-pear/redqueue
 Project-URL: Repository, https://github.com/SpringMirror-pear/redqueue.git
 Project-URL: Issues, https://github.com/SpringMirror-pear/redqueue/issues
@@ -52,6 +52,7 @@ https://github.com/SpringMirror-pear/redqueue.git
 - Redis connection pool managers for shared sync and async resources.
 - `redqueue` CLI for local debugging and operational checks.
 - First-class `trace_id` propagation for lifecycle tracing.
+- Opt-in message deduplication with Redis `SET NX` and TTL windows.
 - Unified exception hierarchy with structured context.
 - Monitoring events for publish, consume, ack, nack, retry, dead letter, delay,
   and backend errors.
@@ -75,6 +76,7 @@ Redis:
 | Streams | `>=5.0` | Uses `XADD`, `XGROUP CREATE`, `XREADGROUP` |
 | Streams auto claim | `>=6.2` | Uses `XAUTOCLAIM`; Redis 5.x uses `XPENDING`/`XCLAIM` fallback |
 | Delayed tasks | `>=1.2` | Uses `ZADD` and timestamp scores |
+| Message deduplication | `>=2.6.12` | Uses `SET` with `NX` and `EX`/`PX` |
 
 ## Installation
 
@@ -210,6 +212,23 @@ client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123"
 released = client.schedule_due(limit=100)
 ```
 
+Message deduplication:
+
+```python
+from redqueue import DeduplicationConfig, QueueClient
+
+client = QueueClient.from_url(
+    "redis://127.0.0.1:6379/0",
+    queue="emails",
+    deduplication=DeduplicationConfig(enabled=True, ttl_seconds=3600),
+)
+
+first_id = client.publish({"order": 1}, dedup_key="order-1")
+second_id = client.publish({"order": 1}, dedup_key="order-1")
+
+assert second_id == first_id
+```
+
 Trace IDs:
 
 ```python
@@ -367,17 +386,21 @@ REDQUEUE_REDIS_URL=redis://127.0.0.1:6379/0 PYTHONPATH=src python -m pytest -m "
 
 Latest local run on Python `3.14.5`:
 
-- Full test suite without `REDQUEUE_REDIS_URL`: `77 passed, 8 skipped`.
-- Real Redis availability suite: `3 passed` with
+- Full test suite without `REDQUEUE_REDIS_URL`: `114 passed, 10 skipped`.
+- Real Redis integration suite: `10 passed` with
+  `redis://127.0.0.1:6379/0`.
+- Real Redis availability suite: `5 passed` with
   `redis://127.0.0.1:6379/0`.
 - Real Redis server: Redis for Windows `5.0.14.1`.
 - Availability suite: covers List processing recovery, List dead-letter
   requeue, Streams Redis `<5.0` compatibility rejection, Streams Redis 5.x
   pending recovery fallback, Streams dead-letter requeue, delayed task publish
-  failure rollback, missing delayed payload errors, async List recovery, async
-  Streams dead-letter requeue, and async delayed task rollback.
+  failure rollback, missing delayed payload errors, publish-time deduplication,
+  delayed task deduplication, async List recovery, async Streams dead-letter
+  requeue, and async delayed task rollback.
 - Real Redis availability suite additionally validates List recovery, Streams
-  dead-letter requeue, and delayed task rollback against a running Redis server.
+  dead-letter requeue, delayed task rollback, List deduplication, and delayed
+  task deduplication against a running Redis server.
 
 ## Performance Results
 

{redqueue-0.13.0 → redqueue-0.14.0}/README-zh-CN.md

@@ -14,10 +14,11 @@ https://github.com/SpringMirror-pear/redqueue.git
   `BRPOPLPUSH`。
 - 基于 Redis Streams 的消费组后端，Streams 要求 Redis `>=5.0`。
 - 基于 Redis Sorted Set 的延迟任务。
-- 同步客户端 `QueueClient` 与异步客户端 `AsyncQueueClient`。
+- 同步客户端 `QueueClient` 与异步客户端 `AsyncQueueClient`。
 - 支持同步和异步 Redis 连接池管理器，方便多个客户端共享连接池。
 - 提供 `redqueue` CLI，用于本地调试和运行时检查。
 - 一等 `trace_id` 链路追踪能力，贯穿消息生命周期。
+- 可选消息去重能力，基于 Redis `SET NX` 和 TTL 窗口。
 - 带结构化上下文的统一异常体系。
 - 针对发布、消费、确认、拒绝、重试、死信、延迟和后端错误的监控事件。
 - 通过 `INFO server` 探测 Redis 能力。
@@ -37,9 +38,10 @@ Redis：
 | --- | --- | --- |
 | List 阻塞消费 | `>=2.0` | 以 `BLPOP` 系列能力为基础 |
 | List 可靠搬移 | `>=2.2` | 使用 `BRPOPLPUSH`；Redis `>=6.2` 优先使用 `BLMOVE` |
-| Streams | `>=5.0` | 使用 `XADD`、`XGROUP CREATE`、`XREADGROUP` |
-| Streams 自动认领 | `>=6.2` | 使用 `XAUTOCLAIM`；Redis 5.x 回退 `XPENDING`/`XCLAIM` |
-| 延迟任务 | `>=1.2` | 使用 `ZADD` 和时间戳 score |
+| Streams | `>=5.0` | 使用 `XADD`、`XGROUP CREATE`、`XREADGROUP` |
+| Streams 自动认领 | `>=6.2` | 使用 `XAUTOCLAIM`；Redis 5.x 回退 `XPENDING`/`XCLAIM` |
+| 延迟任务 | `>=1.2` | 使用 `ZADD` 和时间戳 score |
+| 消息去重 | `>=2.6.12` | 使用带 `NX` 和 `EX`/`PX` 的 `SET` |
 
 ## 安装
 
@@ -72,16 +74,16 @@ redqueue stats --url redis://127.0.0.1:6379/0 --queue emails
 
 发布和消费消息：
 
-```bash
-redqueue publish --queue emails --payload '{"to":"user@example.com"}'
-redqueue consume --queue emails --timeout 1 --ack
-```
-
-发布时指定 trace：
-
-```bash
-redqueue publish --queue emails --payload '{"to":"user@example.com"}' --trace-id trace-123
-```
+```bash
+redqueue publish --queue emails --payload '{"to":"user@example.com"}'
+redqueue consume --queue emails --timeout 1 --ack
+```
+
+发布时指定 trace：
+
+```bash
+redqueue publish --queue emails --payload '{"to":"user@example.com"}' --trace-id trace-123
+```
 
 调试延迟任务：
 
@@ -102,8 +104,8 @@ redqueue dead-letters --queue emails --limit 20
 
 同步 List 队列：
 
-```python
-from redqueue import QueueClient, new_trace_id
+```python
+from redqueue import QueueClient, new_trace_id
 
 client = QueueClient.from_url(
     "redis://127.0.0.1:6379/0",
@@ -111,15 +113,15 @@ client = QueueClient.from_url(
     backend="list",
 )
 
-trace_id = new_trace_id()
-message_id = client.publish({"to": "user@example.com"}, trace_id=trace_id)
-message = client.consume(timeout=1)
-
-if message is not None:
-    try:
-        print(message.trace_id)
-        print(message.payload)
-        client.ack(message)
+trace_id = new_trace_id()
+message_id = client.publish({"to": "user@example.com"}, trace_id=trace_id)
+message = client.consume(timeout=1)
+
+if message is not None:
+    try:
+        print(message.trace_id)
+        print(message.payload)
+        client.ack(message)
     except Exception:
         client.retry(message, reason="handler failed")
 ```
@@ -165,35 +167,52 @@ async def main() -> None:
 asyncio.run(main())
 ```
 
-延迟任务：
+延迟任务：
 
 ```python
 from redqueue import QueueClient
 
-client = QueueClient.from_url("redis://127.0.0.1:6379/0", queue="emails")
-client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123")
+client = QueueClient.from_url("redis://127.0.0.1:6379/0", queue="emails")
+client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123")
 released = client.schedule_due(limit=100)
 ```
 
-链路追踪：
+消息去重：
 
 ```python
-from redqueue import InMemoryMonitoringHook, QueueClient, new_trace_id
+from redqueue import DeduplicationConfig, QueueClient
 
-hook = InMemoryMonitoringHook()
 client = QueueClient.from_url(
     "redis://127.0.0.1:6379/0",
     queue="emails",
-    monitoring=hook,
+    deduplication=DeduplicationConfig(enabled=True, ttl_seconds=3600),
 )
 
-trace_id = new_trace_id()
-client.publish({"to": "user@example.com"}, trace_id=trace_id)
-message = client.consume(timeout=1)
+first_id = client.publish({"order": 1}, dedup_key="order-1")
+second_id = client.publish({"order": 1}, dedup_key="order-1")
 
-assert message.trace_id == trace_id
-assert hook.events[-1].trace_id == trace_id
+assert second_id == first_id
 ```
+
+链路追踪：
+
+```python
+from redqueue import InMemoryMonitoringHook, QueueClient, new_trace_id
+
+hook = InMemoryMonitoringHook()
+client = QueueClient.from_url(
+    "redis://127.0.0.1:6379/0",
+    queue="emails",
+    monitoring=hook,
+)
+
+trace_id = new_trace_id()
+client.publish({"to": "user@example.com"}, trace_id=trace_id)
+message = client.consume(timeout=1)
+
+assert message.trace_id == trace_id
+assert hook.events[-1].trace_id == trace_id
+```
 
 连接池管理：
 
@@ -329,15 +348,16 @@ REDQUEUE_REDIS_URL=redis://127.0.0.1:6379/0 PYTHONPATH=src python -m pytest -m "
 
 Python `3.14.5` 最新本地运行结果：
 
-- 未设置 `REDQUEUE_REDIS_URL` 时的完整测试套件：`77 passed, 8 skipped`。
-- 真实 Redis 可用性套件：使用 `redis://127.0.0.1:6379/0` 时 `3 passed`。
+- 未设置 `REDQUEUE_REDIS_URL` 时的完整测试套件：`114 passed, 10 skipped`。
+- 真实 Redis 集成套件：使用 `redis://127.0.0.1:6379/0` 时 `10 passed`。
+- 真实 Redis 可用性套件：使用 `redis://127.0.0.1:6379/0` 时 `5 passed`。
 - 真实 Redis 服务端：Redis for Windows `5.0.14.1`。
 - 可用性套件覆盖：List processing 恢复、List 死信重放、Streams Redis
   `<5.0` 兼容性拒绝、Streams Redis 5.x pending 恢复回退、Streams 死信
-  重放、延迟任务发布失败回滚、延迟 payload 缺失错误、异步 List 恢复、异步
-  Streams 死信重放和异步延迟任务回滚。
-- 真实 Redis 可用性套件额外验证了运行中 Redis 服务上的 List 恢复、Streams
-  死信重放和延迟任务回滚。
+  重放、延迟任务发布失败回滚、延迟 payload 缺失错误、发布时去重、延迟任务
+  去重、异步 List 恢复、异步 Streams 死信重放和异步延迟任务回滚。
+- 真实 Redis 可用性套件额外验证了运行中 Redis 服务上的 List 恢复、Streams
+  死信重放、延迟任务回滚、List 去重和延迟任务去重。
 
 ## 性能测试结果
 

{redqueue-0.13.0 → redqueue-0.14.0}/README.md

@@ -15,10 +15,11 @@ https://github.com/SpringMirror-pear/redqueue.git
   fallback on older compatible Redis versions.
 - Redis Streams backend with consumer groups. Streams require Redis `>=5.0`.
 - Delayed tasks based on Redis Sorted Set.
-- Sync client `QueueClient` and async client `AsyncQueueClient`.
+- Sync client `QueueClient` and async client `AsyncQueueClient`.
 - Redis connection pool managers for shared sync and async resources.
 - `redqueue` CLI for local debugging and operational checks.
 - First-class `trace_id` propagation for lifecycle tracing.
+- Opt-in message deduplication with Redis `SET NX` and TTL windows.
 - Unified exception hierarchy with structured context.
 - Monitoring events for publish, consume, ack, nack, retry, dead letter, delay,
   and backend errors.
@@ -39,9 +40,10 @@ Redis:
 | --- | --- | --- |
 | List blocking consume | `>=2.0` | Uses `BLPOP` family compatibility baseline |
 | List reliable move | `>=2.2` | Uses `BRPOPLPUSH`; `BLMOVE` preferred on `>=6.2` |
-| Streams | `>=5.0` | Uses `XADD`, `XGROUP CREATE`, `XREADGROUP` |
-| Streams auto claim | `>=6.2` | Uses `XAUTOCLAIM`; Redis 5.x uses `XPENDING`/`XCLAIM` fallback |
-| Delayed tasks | `>=1.2` | Uses `ZADD` and timestamp scores |
+| Streams | `>=5.0` | Uses `XADD`, `XGROUP CREATE`, `XREADGROUP` |
+| Streams auto claim | `>=6.2` | Uses `XAUTOCLAIM`; Redis 5.x uses `XPENDING`/`XCLAIM` fallback |
+| Delayed tasks | `>=1.2` | Uses `ZADD` and timestamp scores |
+| Message deduplication | `>=2.6.12` | Uses `SET` with `NX` and `EX`/`PX` |
 
 ## Installation
 
@@ -74,16 +76,16 @@ redqueue stats --url redis://127.0.0.1:6379/0 --queue emails
 
 Publish and consume messages:
 
-```bash
-redqueue publish --queue emails --payload '{"to":"user@example.com"}'
-redqueue consume --queue emails --timeout 1 --ack
-```
-
-Publish with trace correlation:
-
-```bash
-redqueue publish --queue emails --payload '{"to":"user@example.com"}' --trace-id trace-123
-```
+```bash
+redqueue publish --queue emails --payload '{"to":"user@example.com"}'
+redqueue consume --queue emails --timeout 1 --ack
+```
+
+Publish with trace correlation:
+
+```bash
+redqueue publish --queue emails --payload '{"to":"user@example.com"}' --trace-id trace-123
+```
 
 Delayed task debugging:
 
@@ -104,8 +106,8 @@ All command output is JSON so it can be piped into scripts or log processors.
 
 Synchronous List queue:
 
-```python
-from redqueue import QueueClient, new_trace_id
+```python
+from redqueue import QueueClient, new_trace_id
 
 client = QueueClient.from_url(
     "redis://127.0.0.1:6379/0",
@@ -113,15 +115,15 @@ client = QueueClient.from_url(
     backend="list",
 )
 
-trace_id = new_trace_id()
-message_id = client.publish({"to": "user@example.com"}, trace_id=trace_id)
-message = client.consume(timeout=1)
-
-if message is not None:
-    try:
-        print(message.trace_id)
-        print(message.payload)
-        client.ack(message)
+trace_id = new_trace_id()
+message_id = client.publish({"to": "user@example.com"}, trace_id=trace_id)
+message = client.consume(timeout=1)
+
+if message is not None:
+    try:
+        print(message.trace_id)
+        print(message.payload)
+        client.ack(message)
     except Exception:
         client.retry(message, reason="handler failed")
 ```
@@ -167,35 +169,52 @@ async def main() -> None:
 asyncio.run(main())
 ```
 
-Delayed task:
+Delayed task:
 
 ```python
 from redqueue import QueueClient
 
-client = QueueClient.from_url("redis://127.0.0.1:6379/0", queue="emails")
-client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123")
+client = QueueClient.from_url("redis://127.0.0.1:6379/0", queue="emails")
+client.delay({"to": "later@example.com"}, delay_seconds=60, trace_id="trace-123")
 released = client.schedule_due(limit=100)
 ```
 
-Trace IDs:
+Message deduplication:
 
 ```python
-from redqueue import InMemoryMonitoringHook, QueueClient, new_trace_id
+from redqueue import DeduplicationConfig, QueueClient
 
-hook = InMemoryMonitoringHook()
 client = QueueClient.from_url(
     "redis://127.0.0.1:6379/0",
     queue="emails",
-    monitoring=hook,
+    deduplication=DeduplicationConfig(enabled=True, ttl_seconds=3600),
 )
 
-trace_id = new_trace_id()
-client.publish({"to": "user@example.com"}, trace_id=trace_id)
-message = client.consume(timeout=1)
+first_id = client.publish({"order": 1}, dedup_key="order-1")
+second_id = client.publish({"order": 1}, dedup_key="order-1")
 
-assert message.trace_id == trace_id
-assert hook.events[-1].trace_id == trace_id
+assert second_id == first_id
 ```
+
+Trace IDs:
+
+```python
+from redqueue import InMemoryMonitoringHook, QueueClient, new_trace_id
+
+hook = InMemoryMonitoringHook()
+client = QueueClient.from_url(
+    "redis://127.0.0.1:6379/0",
+    queue="emails",
+    monitoring=hook,
+)
+
+trace_id = new_trace_id()
+client.publish({"to": "user@example.com"}, trace_id=trace_id)
+message = client.consume(timeout=1)
+
+assert message.trace_id == trace_id
+assert hook.events[-1].trace_id == trace_id
+```
 
 Connection pool management:
 
@@ -334,17 +353,21 @@ REDQUEUE_REDIS_URL=redis://127.0.0.1:6379/0 PYTHONPATH=src python -m pytest -m "
 
 Latest local run on Python `3.14.5`:
 
-- Full test suite without `REDQUEUE_REDIS_URL`: `77 passed, 8 skipped`.
-- Real Redis availability suite: `3 passed` with
-  `redis://127.0.0.1:6379/0`.
+- Full test suite without `REDQUEUE_REDIS_URL`: `114 passed, 10 skipped`.
+- Real Redis integration suite: `10 passed` with
+  `redis://127.0.0.1:6379/0`.
+- Real Redis availability suite: `5 passed` with
+  `redis://127.0.0.1:6379/0`.
 - Real Redis server: Redis for Windows `5.0.14.1`.
 - Availability suite: covers List processing recovery, List dead-letter
   requeue, Streams Redis `<5.0` compatibility rejection, Streams Redis 5.x
   pending recovery fallback, Streams dead-letter requeue, delayed task publish
-  failure rollback, missing delayed payload errors, async List recovery, async
-  Streams dead-letter requeue, and async delayed task rollback.
-- Real Redis availability suite additionally validates List recovery, Streams
-  dead-letter requeue, and delayed task rollback against a running Redis server.
+  failure rollback, missing delayed payload errors, publish-time deduplication,
+  delayed task deduplication, async List recovery, async Streams dead-letter
+  requeue, and async delayed task rollback.
+- Real Redis availability suite additionally validates List recovery, Streams
+  dead-letter requeue, delayed task rollback, List deduplication, and delayed
+  task deduplication against a running Redis server.
 
 ## Performance Results