@x12i/countex 1.8.5 → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +867 -860
- package/dist/cli.js +6 -6
- package/dist/core/countex.d.ts +61 -1
- package/dist/core/countex.d.ts.map +1 -1
- package/dist/core/countex.js +241 -0
- package/dist/core/countex.js.map +1 -1
- package/dist/core/managed.d.ts +200 -0
- package/dist/core/managed.d.ts.map +1 -0
- package/dist/core/managed.js +431 -0
- package/dist/core/managed.js.map +1 -0
- package/dist/core/timeframe.d.ts +5 -0
- package/dist/core/timeframe.d.ts.map +1 -1
- package/dist/core/timeframe.js +36 -0
- package/dist/core/timeframe.js.map +1 -1
- package/dist/index.d.ts +6 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +9 -1
- package/dist/index.js.map +1 -1
- package/dist/insights/advisors/resource-advisor.d.ts.map +1 -1
- package/dist/insights/advisors/resource-advisor.js +5 -1
- package/dist/insights/advisors/resource-advisor.js.map +1 -1
- package/dist/insights/types.d.ts +4 -0
- package/dist/insights/types.d.ts.map +1 -1
- package/dist/storage/managed-memory.d.ts +33 -0
- package/dist/storage/managed-memory.d.ts.map +1 -0
- package/dist/storage/managed-memory.js +211 -0
- package/dist/storage/managed-memory.js.map +1 -0
- package/dist/storage/memory.d.ts +7 -4
- package/dist/storage/memory.d.ts.map +1 -1
- package/dist/storage/memory.js.map +1 -1
- package/dist/storage/postgres.d.ts +123 -0
- package/dist/storage/postgres.d.ts.map +1 -0
- package/dist/storage/postgres.js +1325 -0
- package/dist/storage/postgres.js.map +1 -0
- package/dist/storage/redis.d.ts +6 -0
- package/dist/storage/redis.d.ts.map +1 -1
- package/dist/storage/redis.js +21 -0
- package/dist/storage/redis.js.map +1 -1
- package/dist/types/index.d.ts +63 -5
- package/dist/types/index.d.ts.map +1 -1
- package/dist/utils/errors.d.ts +12 -0
- package/dist/utils/errors.d.ts.map +1 -1
- package/dist/utils/errors.js +22 -1
- package/dist/utils/errors.js.map +1 -1
- package/docs/DEPLOYMENT.md +399 -375
- package/docs/END_TO_END_EXAMPLE.md +316 -316
- package/docs/INSIGHTS.md +59 -59
- package/docs/INSIGHTS_ANOMALY_DETECTION.md +33 -33
- package/docs/INSIGHTS_LLM_COST_AND_TOKENS.md +26 -26
- package/docs/INSIGHTS_RESOURCE_ADVISOR.md +33 -31
- package/docs/countex_gap_analysis_complete_checklist.md +1340 -1340
- package/docs/guides/README.md +37 -34
- package/docs/guides/adapter-memory.md +60 -60
- package/docs/guides/adapter-mongo.md +64 -62
- package/docs/guides/adapter-postgres.md +92 -0
- package/docs/guides/adapter-redis.md +89 -89
- package/docs/guides/backup-export.md +135 -135
- package/docs/guides/cardinality.md +50 -50
- package/docs/guides/cli.md +53 -53
- package/docs/guides/client-events.md +65 -65
- package/docs/guides/counter-maps.md +160 -160
- package/docs/guides/idempotency.md +39 -39
- package/docs/guides/ingestion-and-flush.md +67 -67
- package/docs/guides/insights-advisors-and-optimization.md +39 -39
- package/docs/guides/insights-anomalies-and-rules.md +48 -48
- package/docs/guides/insights-comparison-and-trends.md +47 -47
- package/docs/guides/insights-forecasting-and-budget.md +61 -61
- package/docs/guides/insights-overview.md +82 -82
- package/docs/guides/insights-ranking.md +81 -81
- package/docs/guides/managed-commit.md +119 -0
- package/docs/guides/measures.md +72 -71
- package/docs/guides/migrating-mongo-to-postgres.md +127 -0
- package/docs/guides/preview.md +92 -92
- package/docs/guides/querying.md +178 -178
- package/docs/guides/simulation-profiles.md +33 -33
- package/docs/guides/simulation-resource-estimation.md +31 -30
- package/docs/guides/simulation.md +36 -36
- package/docs/guides/timeframes-and-bucketing.md +55 -55
- package/docs/internals/README.md +21 -21
- package/docs/internals/internal-architecture-overview.md +65 -65
- package/docs/internals/internal-backup-stream.md +23 -23
- package/docs/internals/internal-buffer-merge-and-flush.md +51 -51
- package/docs/internals/internal-cardinality.md +30 -30
- package/docs/internals/internal-dimension-hash.md +34 -34
- package/docs/internals/internal-idempotency.md +36 -36
- package/docs/internals/internal-ingest-pipeline.md +52 -52
- package/docs/internals/internal-insights-layer.md +36 -36
- package/docs/internals/internal-mapping-validation-and-extraction.md +59 -59
- package/docs/internals/internal-measures-and-scales.md +35 -35
- package/docs/internals/internal-query-path.md +46 -46
- package/docs/internals/internal-storage-memory.md +13 -13
- package/docs/internals/internal-storage-mongo.md +44 -44
- package/docs/internals/internal-storage-postgres.md +32 -0
- package/docs/internals/internal-storage-redis.md +63 -63
- package/docs/internals/internal-timeframe-bucketing.md +33 -33
- package/docs/use-cases/README.md +56 -56
- package/docs/use-cases/audit-security-log-counts.md +550 -550
- package/docs/use-cases/llm-cost-and-token-metering.md +255 -255
- package/docs/use-cases/multi-tenant-isolation.md +203 -203
- package/docs/use-cases/ops-dashboard-daily-rollups.md +255 -255
- package/docs/use-cases/processing-workflow-activity.md +165 -165
- package/docs/use-cases/product-feature-usage.md +240 -240
- package/docs/use-cases/webhook-idempotent-ingestion.md +188 -188
- package/package.json +190 -170
|
@@ -1,188 +1,188 @@
|
|
|
1
|
-
# Technical playbook: Idempotent webhooks and queues
|
|
2
|
-
|
|
3
|
-
HTTP webhooks and brokers (SQS, Pub/Sub, Rabbit) usually deliver **at least once**. Retries and consumer crashes re-deliver the same logical message. Without dedupe, Countex **double-counts**.
|
|
4
|
-
|
|
5
|
-
This playbook specifies **exact** configuration and handler semantics using Countex **idempotency** + Redis **`markEventSeen`**.
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## 1. Preconditions
|
|
10
|
-
|
|
11
|
-
| Requirement | Detail |
|
|
12
|
-
|-------------|--------|
|
|
13
|
-
| Node.js | ≥ 18 |
|
|
14
|
-
| Packages | `@x12i/countex`, `mongodb` (durable), `ioredis` (cache + dedupe) |
|
|
15
|
-
| Stable id | Unique string per logical delivery, available **before** ingest |
|
|
16
|
-
|
|
17
|
-
---
|
|
18
|
-
|
|
19
|
-
## 2. Idempotency mechanics (internal contract)
|
|
20
|
-
|
|
21
|
-
1. During `extractIncrements`, Countex reads `eventIdPath` → string `eventId` (optional warning if missing).
|
|
22
|
-
2. Before buffering, `ingestBatch` calls `cache.markEventSeen(scope, dedupeKey, ttl)` where:
|
|
23
|
-
|
|
24
|
-
```
|
|
25
|
-
dedupeKey = `${counterId}:${eventId}`
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
See `src/core/countex.ts` — dedupe is **per map** so two maps on same raw event do not share keys incorrectly.
|
|
29
|
-
|
|
30
|
-
3. If `markEventSeen` returns **`false`**, increments for that map slice are cleared and event **`duplicate`** fires.
|
|
31
|
-
|
|
32
|
-
4. Redis adapter stores `countex:{scope}:dedupe:{eventId}` — note the adapter receives **already-namespaced** `eventId` from Countex (includes `counterId:` prefix).
|
|
33
|
-
|
|
34
|
-
---
|
|
35
|
-
|
|
36
|
-
## 3. Canonical payload
|
|
37
|
-
|
|
38
|
-
```json
|
|
39
|
-
{
|
|
40
|
-
"tenant": { "id": "org_1" },
|
|
41
|
-
"delivery": { "id": "wh_8f3a2c9e1d004f2a" },
|
|
42
|
-
"at": "2026-05-08T18:00:00.000Z",
|
|
43
|
-
"userId": "u_1",
|
|
44
|
-
"kind": "invoice.paid"
|
|
45
|
-
}
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
**Invariant:** `delivery.id` is unique **for all time** or unique within TTL horizon — if ids recycle after TTL expiry, duplicates may re-count.
|
|
49
|
-
|
|
50
|
-
---
|
|
51
|
-
|
|
52
|
-
## 4. Counter map (exact)
|
|
53
|
-
|
|
54
|
-
`src/countex/maps/billingWebhookMap.ts`:
|
|
55
|
-
|
|
56
|
-
```ts
|
|
57
|
-
import type { CounterMap } from "@x12i/countex";
|
|
58
|
-
|
|
59
|
-
export const billingWebhookMap: CounterMap = {
|
|
60
|
-
counterId: "billing_events",
|
|
61
|
-
|
|
62
|
-
scope: { path: "$.tenant.id" },
|
|
63
|
-
eventTime: { path: "$.at" },
|
|
64
|
-
|
|
65
|
-
subjects: [{ type: "user", path: "$.userId", required: true }],
|
|
66
|
-
metrics: [{ path: "$.kind", required: true }],
|
|
67
|
-
|
|
68
|
-
delta: { default: 1 },
|
|
69
|
-
timeframes: ["day"],
|
|
70
|
-
|
|
71
|
-
idempotency: {
|
|
72
|
-
enabled: true,
|
|
73
|
-
eventIdPath: "$.delivery.id",
|
|
74
|
-
ttlSeconds: 604800, // 7d — MUST exceed max retry horizon
|
|
75
|
-
},
|
|
76
|
-
};
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
### 4.1 Choosing `ttlSeconds`
|
|
80
|
-
|
|
81
|
-
Set:
|
|
82
|
-
|
|
83
|
-
```
|
|
84
|
-
ttlSeconds ≥ (max broker retention) + (max consumer lag) + safety_margin
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
Example: SQS visibility timeout retries for 4 days → use ≥ 5–7 days.
|
|
88
|
-
|
|
89
|
-
---
|
|
90
|
-
|
|
91
|
-
## 5. Factory (exact)
|
|
92
|
-
|
|
93
|
-
```ts
|
|
94
|
-
import { MongoClient } from "mongodb";
|
|
95
|
-
import IORedis from "ioredis";
|
|
96
|
-
import { createCountex } from "@x12i/countex";
|
|
97
|
-
import { MongoDurable } from "@x12i/countex/adapters/mongo";
|
|
98
|
-
import { RedisCache } from "@x12i/countex/adapters/redis";
|
|
99
|
-
import { billingWebhookMap } from "./maps/billingWebhookMap.js";
|
|
100
|
-
|
|
101
|
-
export async function createWebhookCountex() {
|
|
102
|
-
const mongo = new MongoClient(process.env.MONGO_URI!);
|
|
103
|
-
await mongo.connect();
|
|
104
|
-
const redis = new IORedis(process.env.REDIS_URL!);
|
|
105
|
-
|
|
106
|
-
const durable = new MongoDurable({ client: mongo, dbResolver: ({ scope }) => `countex_${scope}` });
|
|
107
|
-
const cache = new RedisCache({ client: redis, keyPrefix: "countex", ttlSeconds: 14 * 24 * 3600 });
|
|
108
|
-
|
|
109
|
-
const countex = createCountex({
|
|
110
|
-
maps: [billingWebhookMap],
|
|
111
|
-
durable,
|
|
112
|
-
cache,
|
|
113
|
-
flush: { everyMs: 5_000, maxBufferedCells: 20_000, maxBufferedEvents: 10_000, flushOnShutdown: true, onOverflow: "block" },
|
|
114
|
-
});
|
|
115
|
-
|
|
116
|
-
await countex.init();
|
|
117
|
-
return { countex, mongo, redis };
|
|
118
|
-
}
|
|
119
|
-
```
|
|
120
|
-
|
|
121
|
-
**Critical:** Without `cache` implementing `markEventSeen`, idempotency **does nothing** — duplicates still buffer.
|
|
122
|
-
|
|
123
|
-
---
|
|
124
|
-
|
|
125
|
-
## 6. Consumer handler (exact steps)
|
|
126
|
-
|
|
127
|
-
### Step 6.1 — Parse
|
|
128
|
-
|
|
129
|
-
```ts
|
|
130
|
-
const payload = JSON.parse(messageBody) as unknown;
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
### Step 6.2 — Validate schema (your code)
|
|
134
|
-
|
|
135
|
-
Reject before ingest if `delivery.id` missing while idempotency enabled — otherwise you get warnings and **non-deduped** increments.
|
|
136
|
-
|
|
137
|
-
### Step 6.3 — Ingest
|
|
138
|
-
|
|
139
|
-
```ts
|
|
140
|
-
await countex.ingest(payload);
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
### Step 6.4 — Observe duplicates
|
|
144
|
-
|
|
145
|
-
```ts
|
|
146
|
-
countex.on("duplicate", ({ scopeId, eventId }) => {
|
|
147
|
-
// eventId here is the NAMESPACED key "billing_events:wh_8f3a2c..."
|
|
148
|
-
});
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
Return **200 OK / ack message** even on duplicate — delivery succeeded idempotently.
|
|
152
|
-
|
|
153
|
-
### Step 6.5 — Durability boundary
|
|
154
|
-
|
|
155
|
-
Either:
|
|
156
|
-
|
|
157
|
-
- Rely on **`flush.everyMs`**, or
|
|
158
|
-
- **`await countex.flush()`** after batch if webhook framework requires strong durability before ack (latency tradeoff).
|
|
159
|
-
|
|
160
|
-
**Ack ordering policy:** industry-standard is **process → flush OR tolerate timer → ack**; never ack before deserialize if you cannot handle poison pills.
|
|
161
|
-
|
|
162
|
-
---
|
|
163
|
-
|
|
164
|
-
## 7. Verification tests
|
|
165
|
-
|
|
166
|
-
| Test | Procedure | Expected |
|
|
167
|
-
|------|-----------|----------|
|
|
168
|
-
| Dedupe works | Same payload twice | Second `duplicate`; Mongo count unchanged. |
|
|
169
|
-
| TTL expiry | Simulate key delete | Third delivery increments again — expected. |
|
|
170
|
-
| Multi-map | Two maps, same raw `delivery.id` | Distinct dedupe keys (`counterId:` prefix) → both run once each. |
|
|
171
|
-
|
|
172
|
-
---
|
|
173
|
-
|
|
174
|
-
## 8. Failure modes
|
|
175
|
-
|
|
176
|
-
| Symptom | Cause | Fix |
|
|
177
|
-
|---------|-------|-----|
|
|
178
|
-
| Duplicates still increment | Memory-only cache adapter | Use `RedisCache` or implement `markEventSeen`. |
|
|
179
|
-
| Redis down | `markEventSeen` throws | Decide fail-open vs fail-closed; today ingest path uses cache — surface errors to DLQ. |
|
|
180
|
-
| Mass replays after TTL | TTL too short | Raise `ttlSeconds`. |
|
|
181
|
-
|
|
182
|
-
---
|
|
183
|
-
|
|
184
|
-
## 9. References
|
|
185
|
-
|
|
186
|
-
- [`docs/guides/idempotency.md`](../guides/idempotency.md)
|
|
187
|
-
- [`docs/internals/internal-idempotency.md`](../internals/internal-idempotency.md)
|
|
188
|
-
- `src/storage/redis.ts` dedupe key format
|
|
1
|
+
# Technical playbook: Idempotent webhooks and queues
|
|
2
|
+
|
|
3
|
+
HTTP webhooks and brokers (SQS, Pub/Sub, Rabbit) usually deliver **at least once**. Retries and consumer crashes re-deliver the same logical message. Without dedupe, Countex **double-counts**.
|
|
4
|
+
|
|
5
|
+
This playbook specifies **exact** configuration and handler semantics using Countex **idempotency** + Redis **`markEventSeen`**.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 1. Preconditions
|
|
10
|
+
|
|
11
|
+
| Requirement | Detail |
|
|
12
|
+
|-------------|--------|
|
|
13
|
+
| Node.js | ≥ 18 |
|
|
14
|
+
| Packages | `@x12i/countex`, `mongodb` (durable), `ioredis` (cache + dedupe) |
|
|
15
|
+
| Stable id | Unique string per logical delivery, available **before** ingest |
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## 2. Idempotency mechanics (internal contract)
|
|
20
|
+
|
|
21
|
+
1. During `extractIncrements`, Countex reads `eventIdPath` → string `eventId` (optional warning if missing).
|
|
22
|
+
2. Before buffering, `ingestBatch` calls `cache.markEventSeen(scope, dedupeKey, ttl)` where:
|
|
23
|
+
|
|
24
|
+
```
|
|
25
|
+
dedupeKey = `${counterId}:${eventId}`
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
See `src/core/countex.ts` — dedupe is **per map** so two maps on same raw event do not share keys incorrectly.
|
|
29
|
+
|
|
30
|
+
3. If `markEventSeen` returns **`false`**, increments for that map slice are cleared and event **`duplicate`** fires.
|
|
31
|
+
|
|
32
|
+
4. Redis adapter stores `countex:{scope}:dedupe:{eventId}` — note the adapter receives **already-namespaced** `eventId` from Countex (includes `counterId:` prefix).
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## 3. Canonical payload
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
{
|
|
40
|
+
"tenant": { "id": "org_1" },
|
|
41
|
+
"delivery": { "id": "wh_8f3a2c9e1d004f2a" },
|
|
42
|
+
"at": "2026-05-08T18:00:00.000Z",
|
|
43
|
+
"userId": "u_1",
|
|
44
|
+
"kind": "invoice.paid"
|
|
45
|
+
}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
**Invariant:** `delivery.id` is unique **for all time** or unique within TTL horizon — if ids recycle after TTL expiry, duplicates may re-count.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## 4. Counter map (exact)
|
|
53
|
+
|
|
54
|
+
`src/countex/maps/billingWebhookMap.ts`:
|
|
55
|
+
|
|
56
|
+
```ts
|
|
57
|
+
import type { CounterMap } from "@x12i/countex";
|
|
58
|
+
|
|
59
|
+
export const billingWebhookMap: CounterMap = {
|
|
60
|
+
counterId: "billing_events",
|
|
61
|
+
|
|
62
|
+
scope: { path: "$.tenant.id" },
|
|
63
|
+
eventTime: { path: "$.at" },
|
|
64
|
+
|
|
65
|
+
subjects: [{ type: "user", path: "$.userId", required: true }],
|
|
66
|
+
metrics: [{ path: "$.kind", required: true }],
|
|
67
|
+
|
|
68
|
+
delta: { default: 1 },
|
|
69
|
+
timeframes: ["day"],
|
|
70
|
+
|
|
71
|
+
idempotency: {
|
|
72
|
+
enabled: true,
|
|
73
|
+
eventIdPath: "$.delivery.id",
|
|
74
|
+
ttlSeconds: 604800, // 7d — MUST exceed max retry horizon
|
|
75
|
+
},
|
|
76
|
+
};
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
### 4.1 Choosing `ttlSeconds`
|
|
80
|
+
|
|
81
|
+
Set:
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
ttlSeconds ≥ (max broker retention) + (max consumer lag) + safety_margin
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Example: SQS visibility timeout retries for 4 days → use ≥ 5–7 days.
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## 5. Factory (exact)
|
|
92
|
+
|
|
93
|
+
```ts
|
|
94
|
+
import { MongoClient } from "mongodb";
|
|
95
|
+
import IORedis from "ioredis";
|
|
96
|
+
import { createCountex } from "@x12i/countex";
|
|
97
|
+
import { MongoDurable } from "@x12i/countex/adapters/mongo";
|
|
98
|
+
import { RedisCache } from "@x12i/countex/adapters/redis";
|
|
99
|
+
import { billingWebhookMap } from "./maps/billingWebhookMap.js";
|
|
100
|
+
|
|
101
|
+
export async function createWebhookCountex() {
|
|
102
|
+
const mongo = new MongoClient(process.env.MONGO_URI!);
|
|
103
|
+
await mongo.connect();
|
|
104
|
+
const redis = new IORedis(process.env.REDIS_URL!);
|
|
105
|
+
|
|
106
|
+
const durable = new MongoDurable({ client: mongo, dbResolver: ({ scope }) => `countex_${scope}` });
|
|
107
|
+
const cache = new RedisCache({ client: redis, keyPrefix: "countex", ttlSeconds: 14 * 24 * 3600 });
|
|
108
|
+
|
|
109
|
+
const countex = createCountex({
|
|
110
|
+
maps: [billingWebhookMap],
|
|
111
|
+
durable,
|
|
112
|
+
cache,
|
|
113
|
+
flush: { everyMs: 5_000, maxBufferedCells: 20_000, maxBufferedEvents: 10_000, flushOnShutdown: true, onOverflow: "block" },
|
|
114
|
+
});
|
|
115
|
+
|
|
116
|
+
await countex.init();
|
|
117
|
+
return { countex, mongo, redis };
|
|
118
|
+
}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
**Critical:** Without `cache` implementing `markEventSeen`, idempotency **does nothing** — duplicates still buffer.
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
## 6. Consumer handler (exact steps)
|
|
126
|
+
|
|
127
|
+
### Step 6.1 — Parse
|
|
128
|
+
|
|
129
|
+
```ts
|
|
130
|
+
const payload = JSON.parse(messageBody) as unknown;
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
### Step 6.2 — Validate schema (your code)
|
|
134
|
+
|
|
135
|
+
Reject before ingest if `delivery.id` missing while idempotency enabled — otherwise you get warnings and **non-deduped** increments.
|
|
136
|
+
|
|
137
|
+
### Step 6.3 — Ingest
|
|
138
|
+
|
|
139
|
+
```ts
|
|
140
|
+
await countex.ingest(payload);
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
### Step 6.4 — Observe duplicates
|
|
144
|
+
|
|
145
|
+
```ts
|
|
146
|
+
countex.on("duplicate", ({ scopeId, eventId }) => {
|
|
147
|
+
// eventId here is the NAMESPACED key "billing_events:wh_8f3a2c..."
|
|
148
|
+
});
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Return **200 OK / ack message** even on duplicate — delivery succeeded idempotently.
|
|
152
|
+
|
|
153
|
+
### Step 6.5 — Durability boundary
|
|
154
|
+
|
|
155
|
+
Either:
|
|
156
|
+
|
|
157
|
+
- Rely on **`flush.everyMs`**, or
|
|
158
|
+
- **`await countex.flush()`** after batch if webhook framework requires strong durability before ack (latency tradeoff).
|
|
159
|
+
|
|
160
|
+
**Ack ordering policy:** industry-standard is **process → flush OR tolerate timer → ack**; never ack before deserialize if you cannot handle poison pills.
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## 7. Verification tests
|
|
165
|
+
|
|
166
|
+
| Test | Procedure | Expected |
|
|
167
|
+
|------|-----------|----------|
|
|
168
|
+
| Dedupe works | Same payload twice | Second `duplicate`; Mongo count unchanged. |
|
|
169
|
+
| TTL expiry | Simulate key delete | Third delivery increments again — expected. |
|
|
170
|
+
| Multi-map | Two maps, same raw `delivery.id` | Distinct dedupe keys (`counterId:` prefix) → both run once each. |
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
174
|
+
## 8. Failure modes
|
|
175
|
+
|
|
176
|
+
| Symptom | Cause | Fix |
|
|
177
|
+
|---------|-------|-----|
|
|
178
|
+
| Duplicates still increment | Memory-only cache adapter | Use `RedisCache` or implement `markEventSeen`. |
|
|
179
|
+
| Redis down | `markEventSeen` throws | Decide fail-open vs fail-closed; today ingest path uses cache — surface errors to DLQ. |
|
|
180
|
+
| Mass replays after TTL | TTL too short | Raise `ttlSeconds`. |
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## 9. References
|
|
185
|
+
|
|
186
|
+
- [`docs/guides/idempotency.md`](../guides/idempotency.md)
|
|
187
|
+
- [`docs/internals/internal-idempotency.md`](../internals/internal-idempotency.md)
|
|
188
|
+
- `src/storage/redis.ts` dedupe key format
|