streamgate 1.0.0__tar.gz → 2.0.0__tar.gz

{streamgate-1.0.0 → streamgate-2.0.0}/.github/workflows/ci.yml

@@ -100,9 +100,9 @@ jobs:
           assert streamgate.__version__
 
           from streamgate import (
-              Consumer, ConsumerOptions, DefaultErrorClassifier, ErrorKind,
-              InMemoryAdmission, IngestBinding, IngestGateway,
-              ManualBackpressureSignal,
+              Consumer, ConsumerOptions, DedupOptions, DefaultErrorClassifier,
+              ErrorKind, InMemoryDedupCarrier, ManualBackpressureSignal,
+              Producer, ProducerOptions,
           )
 
           try:
@@ -110,6 +110,11 @@ jobs:
           except TypeError as e:
               assert "handler" in str(e)
 
+          try:
+              Producer()
+          except ValueError as e:
+              assert "topic is required" in str(e)
+
           print("bare-install import smoke OK")
           EOF
       - name: "Smoke: pure_pipeline end to end (produce → Kafka → consume → sqlite)"
@@ -120,7 +125,7 @@ jobs:
           sleep 15
           KAFKA__BOOTSTRAP_SERVERS=localhost:29092 ../../../bare/bin/python produce.py | tee produce.out
           grep -q "o-1: accepted" produce.out
-          grep -q "o-1: conflict" produce.out   # in-memory uniqueness contract
+          grep -q "o-1: duplicate" produce.out   # in-process dedup contract
           grep -q "o-2: accepted" produce.out
           sleep 20   # allow batch-timeout flush + offset commit
           kill -INT $CONSUMER_PID

{streamgate-1.0.0 → streamgate-2.0.0}/CHANGELOG.md

@@ -5,6 +5,63 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.0.0] - 2026-09-11
+
+### BREAKING — unified backfill contract (feat!)
+
+`BackfillSource.load` / `SqlBackfill.load` now speak one mapped contract:
+
+```text
+await load(scope: str) -> dict[str, JsonObject] | None
+```
+
+where `scope` is an identity key (single-key carriers) or a group id (group
+carriers), and the return is a `{identity: summary}` mapping — `None` and `{}`
+are equivalent (no records).
+
+- old single-summary `None` (confirmed absent) → `{}` / key-not-present
+- old single-summary hit → `{identity: summary}`
+- duplicate / allow judgment paths are preserved verbatim
+
+**Runtime breakage:** external custom `BackfillSource` implementations must
+switch `load`'s return shape to the mapped form. Mechanical migration, no
+other surface changed.
+
+### Added
+
+- **`RedisGroupDedupCarrier`** — a group-scoped `DedupCarrier`
+  (`guarantee="distributed"`) in `streamgate.contrib.redis_dedup`. A group
+  (batch / task / import session) is one Redis HASH
+  (`{prefix}dedup_group:{group}`, field = group-internal identity);
+  `group_key(record)` extracts the group id, `key(record)` the identity.
+  Three-stage admit: in-group HASH hit → group-exists fast path (**zero DB**) →
+  new-group cold load of the **whole group** (group-level single-flight +
+  concurrency gate). The group key's lifecycle is immutable — created only by a
+  successful whole-group backfill or the first placeholder after an
+  empty-group confirm — so a present group key makes in-group judgment
+  authoritative with no safety valve.
+- **`RedisGroupDedupCache`** — HASH storage for the group carrier: atomic
+  per-field Lua placeholder, idempotent HSET+EXPIRE writes, chunked-pipeline
+  whole-group backfill, `group_exists` / `get_group_fields` / `get_ttl` /
+  `delete_group`.
+- **`SqlBackfill(group_column=...)`** — optional group-mode backfill
+  (`WHERE group_column = :scope`, returns the whole group); unset keeps the
+  1.0.0 single-key behavior unchanged.
+- `RedisConfig.group_key_prefix` / `RedisConfig.group_ttl_seconds` for the
+  group HASH keys.
+
+### Notes
+
+- The 1.0.0 single-key `RedisDedupCarrier` / `RedisDedupCache` keep their public
+  API; only their internal consumption of `load` moved to the mapped contract.
+  Both carriers coexist in one Redis instance (`dedup:` STRING vs
+  `dedup_group:` HASH, no type clash).
+- Large active groups: whole-group backfill is chunked internally (implementation
+  detail — no config, no cap). Mind the `group_ttl_seconds` vs write-rate
+  relationship (idle-GC renewal grows the memory watermark of active groups);
+  a low-rate group that crosses its TTL mid-batch simply re-sources once, which
+  is normal self-healing.
+
 ## [1.0.0] - 2026-09-10
 
 ### BREAKING — both sides redesigned as flat entry points

{streamgate-1.0.0 → streamgate-2.0.0}/CONFIGURATION.md

@@ -94,13 +94,17 @@ These are plain constructor arguments (not env-mapped). All belong to
 |-------|---------|-------------|
 | `url` | `redis://localhost:6379/0` | Redis connection URL. |
 | `key_prefix` | `streamgate:` | Key prefix for dedup keys (`<prefix>dedup:<identity>`). |
-| `identity_ttl_seconds` | `18000` | Placeholder/summary TTL (idle-GC heartbeat: every write renews it). |
+| `identity_ttl_seconds` | `18000` | Single-key placeholder/summary TTL (idle-GC heartbeat: every write renews it). |
+| `group_key_prefix` | `dedup_group:` | Group-carrier key prefix (`<prefix>dedup_group:<group>` HASH; type-isolated from the single-key STRING). |
+| `group_ttl_seconds` | `18000` | Group HASH TTL (idle-GC heartbeat: any group write renews it). |
 | `socket_timeout_ms` | `1000` | Read/query path timeout. |
 | `recv_timeout_ms` | `500` | Push path (reserve/summary write) timeout. |
 
 Strategy-level knobs live on `RedisDedupCarrierConfig`
 (`fail_closed_on_unavailable`, `cold_path_max_concurrency`, gate-full /
-unavailable `retry_after` seconds, `log_context` mapping).
+unavailable `retry_after` seconds, `log_context` mapping) and on
+`RedisGroupDedupCarrierConfig` (`group_cold_path_max_concurrency`, same
+family of knobs).
 
 ### `contrib.sql_upsert.DbConfig`
 

{streamgate-1.0.0 → streamgate-2.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: streamgate
-Version: 1.0.0
+Version: 2.0.0
 Summary: A pure-core Kafka data pipeline framework: conditional admission, reliable delivery, and one obvious consume-loop outlet via a typed handler.
 Project-URL: Homepage, https://github.com/pwg-code/streamgate
 Project-URL: Repository, https://github.com/pwg-code/streamgate
@@ -162,7 +162,7 @@ The package contains **zero web-framework code** (no fastapi/uvicorn), and the c
 
 | Extra | Install | Subpackage(s) | Provides |
 |-------|---------|---------------|----------|
-| `[redis]` | `pip install "streamgate[redis]"` | `contrib.redis_dedup` | `RedisDedupCarrier` (`DedupCarrier` impl, `guarantee="distributed"`), `RedisDedupCache`, `RedisDedupCarrierConfig`, `RedisConfig` |
+| `[redis]` | `pip install "streamgate[redis]"` | `contrib.redis_dedup` | `RedisDedupCarrier` (`DedupCarrier` impl, `guarantee="distributed"`) + `RedisDedupCache`, and the **group-scoped** `RedisGroupDedupCarrier` + `RedisGroupDedupCache` (whole-group one-shot cold backfill, group single-flight); `RedisDedupCarrierConfig`, `RedisConfig` |
 | `[sql]` | `pip install "streamgate[sql]"` | `contrib.sql_upsert` / `contrib.sqlite_upsert` / `contrib.mssql_upsert` | `Upsert` + `upsert_outlet` (batch handler + single-record probe, SQLite `ON CONFLICT` + MSSQL `MERGE`+HOLDLOCK), `SqliteConsumer` / `MssqlConsumer` pre-assembled factories, engine factory, `SqlBackfill` (`BackfillSource` impl), `SQLAlchemyErrorClassifier`, `DbConfig` |
 | `[http]` | `pip install "streamgate[http]"` | `contrib.http_probe` | `HttpProbeSignal` / `HysteresisController` (`BackpressureSignal` impl with trip/recover hysteresis, fail-closed probing) |
 

{streamgate-1.0.0 → streamgate-2.0.0}/README.md

@@ -126,7 +126,7 @@ The package contains **zero web-framework code** (no fastapi/uvicorn), and the c
 
 | Extra | Install | Subpackage(s) | Provides |
 |-------|---------|---------------|----------|
-| `[redis]` | `pip install "streamgate[redis]"` | `contrib.redis_dedup` | `RedisDedupCarrier` (`DedupCarrier` impl, `guarantee="distributed"`), `RedisDedupCache`, `RedisDedupCarrierConfig`, `RedisConfig` |
+| `[redis]` | `pip install "streamgate[redis]"` | `contrib.redis_dedup` | `RedisDedupCarrier` (`DedupCarrier` impl, `guarantee="distributed"`) + `RedisDedupCache`, and the **group-scoped** `RedisGroupDedupCarrier` + `RedisGroupDedupCache` (whole-group one-shot cold backfill, group single-flight); `RedisDedupCarrierConfig`, `RedisConfig` |
 | `[sql]` | `pip install "streamgate[sql]"` | `contrib.sql_upsert` / `contrib.sqlite_upsert` / `contrib.mssql_upsert` | `Upsert` + `upsert_outlet` (batch handler + single-record probe, SQLite `ON CONFLICT` + MSSQL `MERGE`+HOLDLOCK), `SqliteConsumer` / `MssqlConsumer` pre-assembled factories, engine factory, `SqlBackfill` (`BackfillSource` impl), `SQLAlchemyErrorClassifier`, `DbConfig` |
 | `[http]` | `pip install "streamgate[http]"` | `contrib.http_probe` | `HttpProbeSignal` / `HysteresisController` (`BackpressureSignal` impl with trip/recover hysteresis, fail-closed probing) |
 

{streamgate-1.0.0 → streamgate-2.0.0}/README.zh-CN.md

@@ -194,8 +194,9 @@ producer = Producer(..., options=ProducerOptions(
 from streamgate.protocols import BackfillSource
 
 class MyBackfill:                              # 能力清单只有一个方法
-    async def load(self, identity: str) -> dict | None:
-        ...   # 返回既有摘要 dict；None = 确认不存在
+    async def load(self, scope: str) -> dict[str, dict] | None:
+        ...   # 返回 {identity: summary} 映射；scope = 身份键（单键载体）
+              # 或组号（组载体，须返回该组全量身份）；无记录返回 {}
               # 核实不了就抛异常（按"不确定"处理：宁可拒，不重复）
 
 carrier = MyCarrier(backfill=MyBackfill())      # 回源传给你的载体，不是 Producer！
@@ -241,7 +242,7 @@ producer = Producer(..., options=ProducerOptions(signal=MySignal()))   # 注入
 
 | Extra | 安装 | 子包 | 给你什么 |
 |---|---|---|---|
-| `[redis]` | `pip install "streamgate[redis]"` | `streamgate.contrib.redis_dedup` | Redis 分布式判重载体（多实例安全，Lua 原子占位/冷回源/fail-closed，`guarantee="distributed"`） |
+| `[redis]` | `pip install "streamgate[redis]"` | `streamgate.contrib.redis_dedup` | Redis 分布式判重载体（多实例安全，Lua 原子占位/冷回源/fail-closed，`guarantee="distributed"`）；单身份键版 `RedisDedupCarrier` + 组版 `RedisGroupDedupCarrier`（组内候选判重） |
 | `[sql]` | `pip install "streamgate[sql]"` | `streamgate.contrib.sqlite_upsert` / `.mssql_upsert` / `.sql_upsert` | SQL 幂等出口全套：`SqliteConsumer` / `MssqlConsumer` 预装配工厂（批量 upsert handler + 单条探针 + 数据库异常分类器三件套）、`upsert_outlet` 原语、引擎工厂、`SqlBackfill` 回源 |
 | `[http]` | `pip install "streamgate[http]"` | `streamgate.contrib.http_probe` | 积压探针 + 磁滞限流状态机（trip/recover 防抖、fail-closed） |
 
@@ -272,6 +273,63 @@ if result.kind == "duplicate":
     ...   # result.summary = 已有记录的摘要
 ```
 
+同一场景，如果判重边界是"组"（批次/任务/导入会话：一个组号 + 组内身份不重复），
+换 **`RedisGroupDedupCarrier`**——组是一整个 HASH，冷路径**整组一次回源预热**，
+组内后续记录纯 Redis 命中，零 DB 查询；只有"全新组"才触发回源（组级单飞合并并发）：
+
+```python
+from streamgate import DedupOptions, Producer, ProducerOptions
+from streamgate.contrib.redis_dedup import (
+    RedisConfig, RedisGroupDedupCache, RedisGroupDedupCarrier,
+)
+
+redis_config = RedisConfig(url="redis://localhost:6379/0")
+producer = Producer(
+    "localhost:29092",
+    topic="orders",
+    options=ProducerOptions(
+        dedup=DedupOptions(
+            key=lambda r: r.order_id,          # 组内身份键声明
+            carrier=RedisGroupDedupCarrier(    # 仅组内判重（组 = 判重完整边界）
+                cache=RedisGroupDedupCache(redis_config),
+                group_key=lambda r: r.batch_no,   # payload 组号（组 → 组内身份 HASH）
+                key=lambda r: r.order_id,
+                summary=lambda r: {"amount": r.amount},
+                redis_config=redis_config,
+            ),
+        ),
+    ),
+)
+result = await producer.push(order)
+if result.kind == "duplicate":
+    ...   # result.summary = 组内既有记录的摘要
+```
+
+组载体搭配 `SqlBackfill` 做冷回源时，只需加一个列名，**整组**一次查回预热：
+
+```python
+from streamgate.contrib.sql_upsert import SqlBackfill
+
+carrier = RedisGroupDedupCarrier(
+    cache=RedisGroupDedupCache(redis_config),
+    group_key=lambda r: r.batch_no,
+    key=lambda r: r.order_id,
+    backfill=SqlBackfill(            # group_column 已设 → 整组回源
+        engine=...,
+        table="orders",
+        key_column="order_id",
+        summary_columns={"amount": "amount"},
+        order_columns=["seq_no"],
+        group_column="batch_no",
+    ),
+)
+```
+
+> **大组注意（仅备注，不构成限制）**：整组回填按 pipeline 分片写入（纯实现细节）；
+> 活跃大组的 Redis 内存水位随 `group_ttl_seconds`（idle-GC 续期）增长；单次回源
+> 查询体积大时留意 `query_wait_seconds`（库内分页由回源实现自定）；写入间隔接近
+> TTL 的低速率组可能在批未结束时过期，触发一次额外回源属正常自愈。
+
 SQL 出口的开箱路径——工厂预接三件套，其余与核心 `Consumer` 完全一致：
 
 ```python

{streamgate-1.0.0 → streamgate-2.0.0}/examples/pure_pipeline/produce.py

@@ -28,7 +28,7 @@ async def main() -> None:
         options=ProducerOptions(
             message_type="order",
             # 一行开启进程内判重：身份键 = order_id，duplicate 时回给已有摘要
-            dedup=DedupOptions(
+            dedup=DedupOptions[OrderIn](
                 key=lambda r: r.order_id,
                 summary=lambda r: {"amount": r.amount},
             ),

{streamgate-1.0.0 → streamgate-2.0.0}/src/streamgate/__init__.py

@@ -85,7 +85,7 @@ from streamgate.resilience.health import (
 from streamgate.transport.codec import JsonEnvelopeCodec
 from streamgate.transport.kafka import KafkaConsumerService
 
-__version__ = "1.0.0"
+__version__ = "2.0.0"
 
 __all__ = [
     "AllowAllSignal",

{streamgate-1.0.0 → streamgate-2.0.0}/src/streamgate/contrib/__init__.py

@@ -5,7 +5,8 @@
 核心层不得 import 本层；本层可使用核心公共 API 与所在 extra 的第三方库。
 
 子包与 extras 对照（缺依赖时 import 会给出安装指引）：
-- redis_dedup：Redis 分布式判重载体（身份键原子占位 / 冷回源，guarantee="distributed"）
+- redis_dedup：Redis 分布式判重载体（身份键原子占位 / 冷回源，guarantee="distributed"；
+  含组版 RedisGroupDedupCarrier：组 → 组内身份 HASH，整组一次冷回源）
     → pip install streamgate[redis]
 - sql_upsert / sqlite_upsert / mssql_upsert：SQL 幂等 upsert 出口（基座 + 双方言预装配工厂）
     → pip install streamgate[sql]

{streamgate-1.0.0 → streamgate-2.0.0}/src/streamgate/contrib/redis_dedup/__init__.py

@@ -1,8 +1,10 @@
 """Redis 分布式判重载体（extras：streamgate[redis]）。
 
 RedisDedupCarrier（DedupCarrier 协议实现，guarantee="distributed"）+
-RedisDedupCache（身份键 → summary 缓存）+ RedisConfig。配合
-streamgate.contrib.sql_upsert.SqlBackfill 可实现冷身份 DB 回源。
+RedisDedupCache（身份键 → summary 缓存）+ RedisGroupDedupCarrier/
+RedisGroupDedupCache（组 → 组内身份 HASH 缓存）+ RedisConfig。
+配合 streamgate.contrib.sql_upsert.SqlBackfill（可选 group_column 整组回源）
+可实现冷身份/冷整组 DB 回源。
 """
 
 from streamgate.contrib._deps import require_extra_import
@@ -20,6 +22,11 @@ try:
         RejectReason,
     )
     from streamgate.contrib.redis_dedup.config import RedisConfig
+    from streamgate.contrib.redis_dedup.group_cache import RedisGroupDedupCache
+    from streamgate.contrib.redis_dedup.group_carrier import (
+        RedisGroupDedupCarrier,
+        RedisGroupDedupCarrierConfig,
+    )
 except ModuleNotFoundError as e:
     require_extra_import(e)
 
@@ -29,6 +36,9 @@ __all__ = [
     "RedisDedupCache",
     "RedisDedupCarrier",
     "RedisDedupCarrierConfig",
+    "RedisGroupDedupCache",
+    "RedisGroupDedupCarrier",
+    "RedisGroupDedupCarrierConfig",
     "RejectReason",
     "dumps_summary",
     "parse_summary",

{streamgate-1.0.0 → streamgate-2.0.0}/src/streamgate/contrib/redis_dedup/carrier.py

@@ -243,7 +243,7 @@ class RedisDedupCarrier(Generic[RecordModelT]):
 
         # ---- 阶段 2：冷身份回源（并发闸门；无回源时直接占位）----
         try:
-            backfill_summary = await self._gated_load(identity)
+            backfill_map = await self._gated_load(identity)
         except ColdPathGateFullError:
             logger.warning("cold_path_gate_rejected", **self._context(identity))
             return self._reject_gate_full()
@@ -253,27 +253,28 @@ class RedisDedupCarrier(Generic[RecordModelT]):
             )
             return self._reject_dependency()
 
-        if backfill_summary is not None:
-            await self._backfill_best_effort(identity, backfill_summary)
-            return Decision.duplicate(backfill_summary)
+        if backfill_map is not None and identity in backfill_map:
+            await self._backfill_best_effort(identity, backfill_map[identity])
+            return Decision.duplicate(backfill_map[identity])
 
         # DB 确认空：推入路径直接原子占位
         return await self._reserve(identity, meta)
 
-    async def _gated_load(self, identity: str) -> JsonObject | None:
+    async def _gated_load(self, scope: str) -> dict[str, JsonObject] | None:
         """闸门满时不触碰 DB，立即抛 ColdPathGateFullError。
 
+        回源契约：scope = 身份键（单键载体），返回 {identity: summary} 单条目映射。
         单 event loop 内 locked() 检查与 acquire 之间无 await 点，无竞态窗口
         （asyncio.Semaphore 槽位空闲时 acquire 不挂起）。
         """
         if self._cold_gate is None:
-            logger.info("cold_path_load", **self._context(identity))
-            return await self._backfill.load(identity)
+            logger.info("cold_path_load", **self._context(scope))
+            return await self._backfill.load(scope)
         if self._cold_gate.locked():
-            raise ColdPathGateFullError(identity)
+            raise ColdPathGateFullError(scope)
         async with self._cold_gate:
-            logger.info("cold_path_load", **self._context(identity))
-            return await self._backfill.load(identity)
+            logger.info("cold_path_load", **self._context(scope))
+            return await self._backfill.load(scope)
 
     async def _reserve(
         self, identity: str, meta: JsonObject
@@ -304,7 +305,7 @@ class RedisDedupCarrier(Generic[RecordModelT]):
             error=str(error),
         )
         try:
-            backfill_summary = await self._gated_load(identity)
+            backfill_map = await self._gated_load(identity)
         except ColdPathGateFullError:
             logger.warning("cold_path_gate_rejected", **self._context(identity))
             return self._reject_gate_full()
@@ -315,8 +316,8 @@ class RedisDedupCarrier(Generic[RecordModelT]):
                 error=str(db_err),
             )
             return self._reject_dependency()
-        if backfill_summary is not None:
-            return Decision.duplicate(backfill_summary)
+        if backfill_map is not None and identity in backfill_map:
+            return Decision.duplicate(backfill_map[identity])
         return None
 
     async def _backfill_best_effort(

{streamgate-1.0.0 → streamgate-2.0.0}/src/streamgate/contrib/redis_dedup/config.py

@@ -9,6 +9,8 @@ from pydantic import BaseModel
 class RedisConfig(BaseModel):
     url: str = "redis://localhost:6379/0"
     key_prefix: str = "streamgate:"
-    identity_ttl_seconds: int = 18000        # 占位/摘要 TTL 5h（idle GC：写即续期回满额）
+    identity_ttl_seconds: int = 18000        # 单键占位/摘要 TTL 5h（idle GC：写即续期回满额）
+    group_key_prefix: str = "dedup_group:"   # 组键前缀（组载体 <prefix>dedup_group:<group>，与单键 STRING 隔离）
+    group_ttl_seconds: int = 18000           # 组 HASH TTL 5h（idle GC：组内任何写即续期回满额）
     socket_timeout_ms: int = 1000            # 查询/校验路径超时
     recv_timeout_ms: int = 500               # 推入路径占位/摘要写超时

streamgate-2.0.0/src/streamgate/contrib/redis_dedup/group_cache.py

@@ -0,0 +1,213 @@
+"""RedisGroupDedupCache：组 → 组内身份 的 Redis HASH 缓存（分组判重载体的参考存储）。
+
+机制（HASH，field = 组内身份，value = summary JSON）：
+- 原子组内占位（单 key Lua，Cluster 兼容）：HGET 命中即返回既有摘要（竞态输家
+  不写入、不续期）；未命中 HSET + EXPIRE（idle GC 续期）
+- 幂等摘要写：HSET + EXPIRE 无条件续期（force / on_persisted / 回填共用）
+- 整组回填 write_group_fields：pipeline 分片写入（分片为纯实现细节，无配置、无上界），
+  写后统一 EXPIRE
+- 组存在探测 group_exists：EXISTS。组 key 生命周期不可变：仅"整组回填成功"或
+  "空组确认后首身份占位"两条路径创建 → 组 key 存在 ⇒ 组内判定可信，无需空组哨兵
+- fail-closed 与否是判重载体的参数，缓存层只如实报告错误
+"""
+
+import asyncio
+from collections.abc import Awaitable, Mapping
+from typing import TypeVar, cast
+
+import redis.asyncio as aioredis
+
+from streamgate import logger
+from streamgate.contrib.redis_dedup.cache import dumps_summary, parse_summary
+from streamgate.contrib.redis_dedup.config import RedisConfig
+from streamgate.protocols import JsonObject
+
+_T = TypeVar("_T")
+
+_HEALTH_TIMEOUT_S = 2.0
+
+# 整组回填分片大小（纯实现细节：一次 HSET 的 field 数量上限，非业务配置）
+_WRITE_CHUNK_SIZE = 200
+
+# 原子组内占位（单 key，Cluster 兼容）：命中即返回既有摘要，未命中写入 + TTL
+# KEYS[1]=group key, ARGV=[identity, summary_json, ttl_seconds]
+# 返回 [1, ""] = 占位成功；[0, existing_json] = 已被占（竞态输家未写入，不续期）
+_RESERVE_LUA = """
+local existing = redis.call('HGET', KEYS[1], ARGV[1])
+if existing then
+    return {0, existing}
+end
+redis.call('HSET', KEYS[1], ARGV[1], ARGV[2])
+redis.call('EXPIRE', KEYS[1], tonumber(ARGV[3]))
+return {1, ''}
+"""
+
+# 幂等摘要写（force 路径 / 消费端权威刷新 / 回填共用）：无条件 HSET + 无条件续期
+# KEYS[1]=group key, ARGV=[identity, summary_json, ttl_seconds]
+_WRITE_LUA = """
+redis.call('HSET', KEYS[1], ARGV[1], ARGV[2])
+redis.call('EXPIRE', KEYS[1], tonumber(ARGV[3]))
+return 1
+"""
+
+
+class RedisGroupDedupCache:
+    def __init__(
+        self, config: RedisConfig, client: aioredis.Redis | None = None
+    ) -> None:
+        self._config = config
+        self._injected = client
+        self._client: aioredis.Redis | None = None
+
+    @property
+    def group_ttl_seconds(self) -> int:
+        return self._config.group_ttl_seconds
+
+    async def start(self) -> None:
+        if self._client is not None:
+            return
+        if self._injected is not None:
+            self._client = self._injected
+            return
+        self._client = aioredis.from_url(
+            self._config.url,
+            socket_timeout=self._config.socket_timeout_ms / 1000,
+            socket_connect_timeout=self._config.socket_timeout_ms / 1000,
+            decode_responses=True,
+        )
+        logger.info("redis_client_created", url=self._config.url)
+
+    async def close(self) -> None:
+        if self._client is None:
+            return
+        if self._injected is None:  # 注入的客户端归调用方关闭
+            await self._client.aclose()
+        self._client = None
+        logger.info("redis_client_closed")
+
+    async def check_health_detail(self) -> tuple[bool, str | None]:
+        """健康预检。返回 (是否可用, 错误信息)；可用时 error 为 None。"""
+        if self._client is None:
+            return False, "redis client not started"
+        try:
+            await asyncio.wait_for(self._client.ping(), timeout=_HEALTH_TIMEOUT_S)
+            return True, None
+        except Exception as e:
+            logger.debug("redis_health_check_failed", error=str(e))
+            return False, str(e)
+
+    async def check_health(self) -> bool:
+        ok, _ = await self.check_health_detail()
+        return ok
+
+    def group_key(self, group: str) -> str:
+        return f"{self._config.key_prefix}{self._config.group_key_prefix}{group}"
+
+    # ---------- 内部工具 ----------
+
+    def _require_client(self) -> aioredis.Redis:
+        if self._client is None:
+            raise RuntimeError("RedisGroupDedupCache not started, call start() first")
+        return self._client
+
+    async def _read(self, coro: Awaitable[_T]) -> _T:
+        """读操作统一超时（查询/校验路径 socket_timeout_ms）。"""
+        return await asyncio.wait_for(
+            coro, timeout=self._config.socket_timeout_ms / 1000
+        )
+
+    async def _write(self, coro: Awaitable[_T]) -> _T:
+        """写操作统一超时（推入路径 recv_timeout_ms）。"""
+        return await asyncio.wait_for(coro, timeout=self._config.recv_timeout_ms / 1000)
+
+    # ---------- 读 ----------
+
+    async def get_identity_meta(self, group: str, identity: str) -> JsonObject | None:
+        # redis-py 命令类型为 ResponseT（Awaitable|值 联合，同步模式兼容）；
+        # 异步 client 恒返回 Awaitable，cast 仅为收窄静态类型
+        raw = await self._read(
+            cast(
+                "Awaitable[str | None]",
+                self._require_client().hget(self.group_key(group), identity),
+            )
+        )
+        if raw is None:
+            return None
+        return parse_summary(str(raw))
+
+    async def get_group_fields(self, group: str) -> dict[str, JsonObject]:
+        raw = await self._read(
+            cast(
+                "Awaitable[dict[str, str]]",
+                self._require_client().hgetall(self.group_key(group)),
+            )
+        )
+        return {identity: parse_summary(str(payload)) for identity, payload in raw.items()}
+
+    async def get_ttl(self, group: str) -> int:
+        return int(
+            await self._read(self._require_client().ttl(self.group_key(group)))
+        )
+
+    async def group_exists(self, group: str) -> bool:
+        hits = int(
+            await self._read(self._require_client().exists(self.group_key(group)))
+        )
+        return hits > 0
+
+    # ---------- 写 ----------
+
+    async def reserve(
+        self, group: str, identity: str, summary: JsonObject
+    ) -> tuple[bool, JsonObject | None]:
+        """原子检查 + 组内占位。返回 (True, None) = 占位成功；(False, 既有摘要) = 已被占。"""
+        client = self._require_client()
+        script = client.register_script(_RESERVE_LUA)
+        result = await self._write(
+            script(
+                keys=[self.group_key(group)],
+                args=[identity, dumps_summary(summary), self._config.group_ttl_seconds],
+            )
+        )
+        reserved, existing = int(result[0]), result[1]  # type: ignore[index]
+        if reserved == 1:
+            return (True, None)
+        return (False, parse_summary(str(existing)))
+
+    async def write_summary(self, group: str, identity: str, summary: JsonObject) -> None:
+        """幂等摘要写（force 路径 / 消费端权威刷新），无条件续期。"""
+        client = self._require_client()
+        script = client.register_script(_WRITE_LUA)
+        await self._write(
+            script(
+                keys=[self.group_key(group)],
+                args=[identity, dumps_summary(summary), self._config.group_ttl_seconds],
+            )
+        )
+
+    async def write_group_fields(self, group: str, mapping: dict[str, JsonObject]) -> None:
+        """整组回填：pipeline 分片写入（纯实现细节），写后统一续期。
+
+        best-effort 语义（失败抛异常，由调用方 WARN 告警）：
+        - 冷路径：本方法之前组 key 不存在，首片写入即建组；分片中途失败时
+          半组对上层不可判定（阶段2组存在快路径要求 group_exists=True 才走零 DB，
+          判定路径在回填抛异常时统一 REJECT，不把半组当全集）
+        - 幂等：对既有组重复回填全量映射无害（HASH 覆盖写 + 续期）
+        """
+        if not mapping:
+            return
+        client = self._require_client()
+        key = self.group_key(group)
+        items = [
+            (identity, dumps_summary(summary)) for identity, summary in mapping.items()
+        ]
+        for i in range(0, len(items), _WRITE_CHUNK_SIZE):
+            chunk = dict(items[i : i + _WRITE_CHUNK_SIZE])
+            pipe = client.pipeline()
+            pipe.hset(key, mapping=cast("Mapping[str, str]", chunk))  # type: ignore[reportArgumentType]
+            pipe.expire(key, self._config.group_ttl_seconds)
+            await self._write(pipe.execute())
+
+    async def delete_group(self, group: str) -> None:
+        """删除整组（供告警/运维删键修复）。"""
+        await self._write(self._require_client().delete(self.group_key(group)))