@siglo/redis 1.0.1 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +189 -77
- package/package.json +15 -4
package/README.md
CHANGED
|
@@ -1,67 +1,102 @@
|
|
|
1
1
|
# @siglo/redis Adapter Guide
|
|
2
2
|
|
|
3
|
-
`@siglo/redis` provides an in-memory distributed persistence, step response caching, atomic idempotency deduplication,
|
|
3
|
+
`@siglo/redis` provides an in-memory distributed persistence, step response caching, atomic idempotency deduplication, distributed single-flight coordination, and webhook replay prevention layer for `Siglo` workflows. It is powered by Redis and compatible with standalone Redis, Redis Sentinel, and Redis Cluster via `ioredis` (or any client implementing `RedisClientInterface`).
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
## 1. Overview & Architecture
|
|
8
8
|
|
|
9
|
-
The Redis adapter family encompasses
|
|
9
|
+
The Redis adapter family encompasses four core modules:
|
|
10
|
+
|
|
11
|
+
1. **`RedisStorageAdapter`**: Implements the Siglo `StorageAdapter` interface for execution stack checkpointing (with tenant isolation), durable wait record indexing for event-driven suspensions, step response caching, and non-blocking context mutation streaming.
|
|
12
|
+
2. **`RedisIdempotencyStore`**: Implements the Siglo `IdempotencyStore` interface for atomic, distributed single-flight execution claims, cached result retrieval, and automated lock releasing on failure.
|
|
13
|
+
3. **`RedisLock`**: Distributed mutex coordinator with unique per-worker token verification, single-flight helper execution (`withLock`), and optional strict Lua atomicity (`strictAtomicity`) to eliminate TOCTOU release races.
|
|
14
|
+
4. **`RedisNonceStore`**: Distributed atomic deduplication store for webhook signatures and replay attack prevention, enforcing single-use delivery within tolerance windows.
|
|
10
15
|
|
|
11
16
|
```mermaid
|
|
12
17
|
flowchart TD
|
|
13
18
|
subgraph Package["@siglo/redis"]
|
|
14
19
|
direction LR
|
|
15
|
-
SA["RedisStorageAdapter<br/>•
|
|
20
|
+
SA["RedisStorageAdapter<br/>• Checkpoints & Wait Records<br/>• Step Output Caching<br/>• Context Hash Stream"]
|
|
16
21
|
IS["RedisIdempotencyStore<br/>• Atomic NX Claims<br/>• Result Caching<br/>• Fast Lock Release"]
|
|
17
22
|
LK["RedisLock<br/>• Distributed Mutex<br/>• Token Verification<br/>• Safe Lua Releasing"]
|
|
23
|
+
NS["RedisNonceStore<br/>• Replay Attack Prevention<br/>• Atomic NX Claims<br/>• Webhook Signature Dedup"]
|
|
18
24
|
end
|
|
19
25
|
|
|
20
26
|
subgraph RedisServer["Redis Server"]
|
|
21
|
-
Keys1["siglo:[ns:]checkpoint:id<br/>siglo:cache:key"]
|
|
27
|
+
Keys1["siglo:[ns:]checkpoint:id<br/>siglo:wait:key<br/>siglo:cache:key"]
|
|
22
28
|
Keys2["siglo:idemp:key"]
|
|
23
29
|
Keys3["siglo:lock:key<br/>siglo:[ns:]context:id:step"]
|
|
30
|
+
Keys4["siglo:wh:dedup:key"]
|
|
24
31
|
end
|
|
25
32
|
|
|
26
33
|
SA --> Keys1
|
|
27
34
|
SA --> Keys3
|
|
28
35
|
IS --> Keys2
|
|
29
36
|
LK --> Keys3
|
|
37
|
+
NS --> Keys4
|
|
30
38
|
```
|
|
31
39
|
|
|
32
40
|
---
|
|
33
41
|
|
|
34
42
|
## 2. Installation & Requirements
|
|
35
43
|
|
|
36
|
-
Install `@siglo/redis` alongside `ioredis`:
|
|
44
|
+
Install `@siglo/redis` alongside `@siglo/core` and `ioredis`:
|
|
37
45
|
|
|
38
46
|
```bash
|
|
39
47
|
npm install @siglo/core @siglo/redis ioredis
|
|
40
48
|
```
|
|
41
49
|
|
|
42
|
-
Peer
|
|
43
|
-
|
|
44
|
-
|
|
50
|
+
### Peer Dependency Matrix
|
|
51
|
+
|
|
52
|
+
| Package | Version | Requirement |
|
|
53
|
+
| :--- | :--- | :--- |
|
|
54
|
+
| `@siglo/core` | `^1.0.0` | Core workflow engine & interfaces (`StorageAdapter`, `IdempotencyStore`) |
|
|
55
|
+
| `ioredis` | `^5.0.0 \|\| ^6.0.0` | Optional peer dependency (any client matching `RedisClientInterface`) |
|
|
56
|
+
| Redis Server | `6.0+` | Requires `SET ... PX ... NX`, hashes (`HSET`, `HGETALL`), and optional Lua scripting (`EVAL`) |
|
|
45
57
|
|
|
46
58
|
---
|
|
47
59
|
|
|
48
60
|
## 3. Key Topology & Redis Data Structures
|
|
49
61
|
|
|
50
|
-
`@siglo/redis` uses structured key namespaces:
|
|
62
|
+
`@siglo/redis` uses structured key namespaces across all components:
|
|
51
63
|
|
|
52
64
|
| Key Pattern | Redis Type | Default TTL | Description |
|
|
53
65
|
| :--- | :--- | :--- | :--- |
|
|
54
66
|
| `siglo:[<ns>:]checkpoint:<stackId>` | `string` (JSON) | 7 days (`604800000ms`) | Serialized execution stack for workflow resumption. Partitioned by tenant namespace if provided (`siglo:<ns>:checkpoint:<stackId>`). |
|
|
55
|
-
| `siglo:cache:<key>` | `string` (JSON) | 24 hours (`86400000ms`) | Cached step evaluation output. |
|
|
56
|
-
| `siglo:
|
|
57
|
-
| `siglo:
|
|
58
|
-
| `siglo:
|
|
67
|
+
| `siglo:cache:<key>` | `string` (JSON) | 24 hours (`86400000ms`) | Cached step evaluation output. Can be overridden per `setCachedStep` call. |
|
|
68
|
+
| `siglo:wait:<key>` | `string` (JSON) | 7 days (`604800000ms`) | Durable wait/continuation record. Indexed by `correlationKey` and/or `stackId`. |
|
|
69
|
+
| `siglo:[<ns>:]context:<stackId>:<stepId>` | `hash` | 7 days (`604800000ms`) | Active context field mutations for auditing. Partitioned by tenant namespace if provided. TTL refreshed via `PEXPIRE`. |
|
|
70
|
+
| `siglo:idemp:<key>` | `string` | 24 hours (`86400000ms`) | Idempotency lock status (`in-progress` or `COMPLETED:<json>`). Can be overridden per claim/complete. |
|
|
71
|
+
| `siglo:lock:<key>` | `string` | 10 seconds (`10000ms`) | Distributed mutex lock holding worker token (`lock_<timestamp>_<randomHex>` or custom). |
|
|
72
|
+
| `siglo:wh:dedup:<key>` | `string` (`"1"`) | Caller specified (`ttlSeconds * 1000ms`) | Nonce deduplication token for webhook replay attack prevention. |
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## 4. `RedisClientInterface` & Driver Compatibility
|
|
77
|
+
|
|
78
|
+
`@siglo/redis` communicates through `RedisClientInterface`, decoupling the adapter from a specific driver version and simplifying mock testing:
|
|
79
|
+
|
|
80
|
+
```typescript
|
|
81
|
+
export interface RedisClientInterface {
|
|
82
|
+
get(key: string): Promise<string | null>;
|
|
83
|
+
set(key: string, value: string, ...args: any[]): Promise<'OK' | string | null>;
|
|
84
|
+
del(key: string | string[]): Promise<number>;
|
|
85
|
+
hset(key: string, field: string | Record<string, string>, value?: string): Promise<number>;
|
|
86
|
+
hgetall?(key: string): Promise<Record<string, string>>;
|
|
87
|
+
expire?(key: string, seconds: number): Promise<number>;
|
|
88
|
+
pexpire?(key: string, milliseconds: number): Promise<number>;
|
|
89
|
+
eval?(script: string, numKeys: number, ...args: (string | number)[]): Promise<any>;
|
|
90
|
+
}
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Standard `ioredis` instances (`Redis`, `Cluster`, `Sentinel`) fulfill this interface out-of-the-box without requiring wrappers.
|
|
59
94
|
|
|
60
95
|
---
|
|
61
96
|
|
|
62
|
-
##
|
|
97
|
+
## 5. `RedisStorageAdapter` Reference
|
|
63
98
|
|
|
64
|
-
Implements the
|
|
99
|
+
Implements the Siglo `StorageAdapter` interface (`name = 'redis'`) for high-performance checkpointing, step output caching, wait record indexing, and context mutation streaming.
|
|
65
100
|
|
|
66
101
|
```typescript
|
|
67
102
|
import { RedisStorageAdapter } from '@siglo/redis';
|
|
@@ -71,25 +106,36 @@ import { RedisStorageAdapter } from '@siglo/redis';
|
|
|
71
106
|
|
|
72
107
|
| Option | Type | Default | Description |
|
|
73
108
|
| :--- | :--- | :--- | :--- |
|
|
74
|
-
| `client` | `RedisClientInterface
|
|
75
|
-
| `keyPrefix` | `string` | `'siglo:'` | Root prefix for all Redis keys. |
|
|
76
|
-
| `checkpointTtlMs` | `number` | `604800000` (7 days) |
|
|
109
|
+
| `client` | `RedisClientInterface` | *Required* | Active Redis client instance (e.g., `ioredis`). Throws if omitted. |
|
|
110
|
+
| `keyPrefix` | `string` | `'siglo:'` | Root prefix for all Redis keys created by this adapter. |
|
|
111
|
+
| `checkpointTtlMs` | `number` | `604800000` (7 days) | TTL for checkpoints, wait records, and context hashes. |
|
|
77
112
|
| `cacheTtlMs` | `number` | `86400000` (24 hours) | Default step response cache expiration TTL. |
|
|
78
|
-
| `recordContextChanges` | `boolean` | `true` | When `true`, streams field updates to Redis
|
|
113
|
+
| `recordContextChanges` | `boolean` | `true` | When `true`, streams field updates to Redis hashes. |
|
|
79
114
|
|
|
80
115
|
### Methods
|
|
81
116
|
|
|
82
|
-
- **`saveCheckpoint(checkpoint)
|
|
83
|
-
-
|
|
84
|
-
- **`
|
|
85
|
-
-
|
|
86
|
-
- **`
|
|
117
|
+
- **`saveCheckpoint(checkpoint: SerializedExecutionStack): Promise<void>`**:
|
|
118
|
+
Saves the JSON-serialized execution stack with millisecond TTL (`SET key val PX checkpointTtlMs`). If `checkpoint.namespace` is provided, stores under `siglo:<ns>:checkpoint:<stackId>`. Returns early if `checkpoint` or `checkpoint.stackId` is missing.
|
|
119
|
+
- **`getCheckpoint(stackId: string, namespace?: string): Promise<SerializedExecutionStack | undefined>`**:
|
|
120
|
+
Retrieves and parses the serialized checkpoint. If `namespace` is specified, checks the tenant-namespaced key first, falling back to the un-namespaced key if not found. Returns `undefined` if missing or if parsing fails.
|
|
121
|
+
- **`saveWaitRecord(record: any): Promise<void>`**:
|
|
122
|
+
Persists durable wait/continuation records for event-driven workflows (e.g., `waitForEvent`). Dual-indexes the record under `siglo:wait:<correlationKey>` (if present) and `siglo:wait:<stackId>` (if present) with `checkpointTtlMs`.
|
|
123
|
+
- **`getWaitRecord(correlationKeyOrStackId: string): Promise<any | undefined>`**:
|
|
124
|
+
Retrieves a wait record by correlation key or stack ID from `siglo:wait:<key>`. If not found, falls back to inspecting `cp?.metadata?.waitRecord` from `getCheckpoint(correlationKeyOrStackId)`.
|
|
125
|
+
- **`cleanExpiredCheckpoints(): Promise<number>`**:
|
|
126
|
+
Returns `0`. TTL expiration is handled automatically by Redis native keyspace expiration, eliminating background cleanup sweeps.
|
|
127
|
+
- **`getCachedStep(key: string): Promise<any | undefined>`**:
|
|
128
|
+
Retrieves cached step response from `siglo:cache:<key>`. Returns parsed JSON, or the raw string if not JSON, or `undefined` if key does not exist.
|
|
129
|
+
- **`setCachedStep(key: string, value: any, ttlMs?: number): Promise<void>`**:
|
|
130
|
+
Writes cached step evaluation to `siglo:cache:<key>` with `PX` TTL (`ttlMs || cacheTtlMs`). Objects are serialized directly; primitives are wrapped as `{ value }`.
|
|
131
|
+
- **`onContextChange(entry): Promise<void>`**:
|
|
132
|
+
When `recordContextChanges` is `true`, writes field mutations to `HSET` at `siglo:[<ns>:]context:<stackId>:<stepId>` and refreshes expiration with `PEXPIRE` (`checkpointTtlMs`). Errors are silently caught to ensure non-blocking telemetry.
|
|
87
133
|
|
|
88
134
|
---
|
|
89
135
|
|
|
90
|
-
##
|
|
136
|
+
## 6. `RedisIdempotencyStore` Reference
|
|
91
137
|
|
|
92
|
-
Provides distributed, atomic idempotency guarantees across multi-instance clusters using Redis `SET key val PX ttl NX` semantics.
|
|
138
|
+
Provides distributed, atomic idempotency guarantees across multi-instance clusters using Redis `SET key val PX ttl NX` semantics. Implements the Siglo `IdempotencyStore` interface.
|
|
93
139
|
|
|
94
140
|
```typescript
|
|
95
141
|
import { RedisIdempotencyStore } from '@siglo/redis';
|
|
@@ -99,36 +145,39 @@ import { RedisIdempotencyStore } from '@siglo/redis';
|
|
|
99
145
|
|
|
100
146
|
| Option | Type | Default | Description |
|
|
101
147
|
| :--- | :--- | :--- | :--- |
|
|
102
|
-
| `client` | `RedisClientInterface
|
|
103
|
-
| `keyPrefix` | `string` | `'siglo:idemp:'` |
|
|
104
|
-
| `defaultTtlMs` | `number` | `86400000` (24 hours) | Default lock TTL. |
|
|
105
|
-
|
|
106
|
-
### Claim States & Transitions
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
-
|
|
110
|
-
- If
|
|
111
|
-
- If key exists with
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
148
|
+
| `client` | `RedisClientInterface` | *Required* | Active Redis client instance. Throws if omitted. |
|
|
149
|
+
| `keyPrefix` | `string` | `'siglo:idemp:'` | Root prefix for idempotency keys. |
|
|
150
|
+
| `defaultTtlMs` | `number` | `86400000` (24 hours) | Default lock expiration TTL. |
|
|
151
|
+
|
|
152
|
+
### Claim States & Lifecycle Transitions
|
|
153
|
+
|
|
154
|
+
1. **`claim(key: string, ttlMs?: number): Promise<'claimed' | 'completed' | 'in-progress'>`**:
|
|
155
|
+
- Executes atomic Redis `SET <keyPrefix><key> "in-progress" PX <effectiveTtl> NX`.
|
|
156
|
+
- If Redis returns `'OK'`: Returns `'claimed'` (caller acquired exclusive execution right).
|
|
157
|
+
- If the key already exists and starts with `COMPLETED:`: Returns `'completed'`.
|
|
158
|
+
- Otherwise returns `'in-progress'` (another worker is currently processing this key).
|
|
159
|
+
2. **`complete(key: string, result: unknown, ttlMs?: number): Promise<void>`**:
|
|
160
|
+
- Overwrites key with `COMPLETED:${JSON.stringify(result ?? null)}` and resets TTL to `effectiveTtl`.
|
|
161
|
+
3. **`fail(key: string, _error: unknown): Promise<void>`**:
|
|
162
|
+
- Reads the key and deletes it (`DEL`) **only** if its current value is `'in-progress'`. This frees the lock for immediate retry execution while preserving already completed results.
|
|
163
|
+
4. **`getResult(key: string): Promise<unknown | undefined>`**:
|
|
164
|
+
- Reads the key. If it starts with `COMPLETED:`, deserializes and returns the JSON payload. Returns `undefined` if missing or if parsing fails.
|
|
165
|
+
5. **`get(key: string): Promise<unknown | undefined>`**:
|
|
166
|
+
- Direct alias delegating to `getResult(key)`.
|
|
118
167
|
|
|
119
168
|
---
|
|
120
169
|
|
|
121
|
-
##
|
|
170
|
+
## 7. `RedisLock` (Distributed Mutex & Single-Flight Coordination)
|
|
122
171
|
|
|
123
172
|
`RedisLock` prevents duplicate parallel execution of sensitive workflows (e.g., end-of-month payroll, nightly data synchronization) across multiple server instances.
|
|
124
173
|
|
|
125
174
|
```typescript
|
|
126
175
|
import { RedisLock, RedisLockOptions } from '@siglo/redis';
|
|
127
176
|
|
|
128
|
-
// Simple prefix configuration:
|
|
177
|
+
// 1. Simple prefix configuration:
|
|
129
178
|
const lock = new RedisLock(redis, 'siglo:lock:');
|
|
130
179
|
|
|
131
|
-
// Full options configuration:
|
|
180
|
+
// 2. Full options configuration:
|
|
132
181
|
const strictLock = new RedisLock(redis, {
|
|
133
182
|
keyPrefix: 'siglo:lock:',
|
|
134
183
|
strictAtomicity: true, // Enforces Redis eval (Lua) for atomic lock release
|
|
@@ -140,65 +189,102 @@ const strictLock = new RedisLock(redis, {
|
|
|
140
189
|
| Option | Type | Default | Description |
|
|
141
190
|
| :--- | :--- | :--- | :--- |
|
|
142
191
|
| `keyPrefix` | `string` | `'siglo:lock:'` | Root prefix for distributed mutex keys. |
|
|
143
|
-
| `strictAtomicity` | `boolean` | `false` | When `true`, enforces
|
|
192
|
+
| `strictAtomicity` | `boolean` | `false` | When `true`, enforces atomic Lua evaluation (`client.eval`) during `release()`. If Lua scripting is unsupported, throws an error rather than falling back to non-atomic commands, preventing TOCTOU race conditions. |
|
|
193
|
+
|
|
194
|
+
### Methods
|
|
195
|
+
|
|
196
|
+
#### `acquire(lockKey: string, ttlMs: number = 10000, customToken?: string): Promise<RedisLockResult>`
|
|
197
|
+
Attempts atomic acquisition via `SET <keyPrefix><lockKey> <token> PX <ttlMs> NX`.
|
|
198
|
+
- Returns `RedisLockResult`: `{ acquired: boolean, token: string }`.
|
|
199
|
+
- `token` defaults to `lock_${Date.now()}_${randomBytes(16).toString('hex')}` unless `customToken` is supplied.
|
|
200
|
+
|
|
201
|
+
#### `release(lockKey: string, token: string): Promise<boolean>`
|
|
202
|
+
Safely releases the lock only if the current value matches the caller's unique token.
|
|
203
|
+
- **When `client.eval` is available**: Executes atomic Lua script:
|
|
204
|
+
```lua
|
|
205
|
+
if redis.call("get", KEYS[1]) == ARGV[1] then
|
|
206
|
+
return redis.call("del", KEYS[1])
|
|
207
|
+
else
|
|
208
|
+
return 0
|
|
209
|
+
end
|
|
210
|
+
```
|
|
211
|
+
- **When `client.eval` is unavailable**:
|
|
212
|
+
- If `strictAtomicity: true`: Throws `Error('RedisLock: client.eval is required for atomic release to prevent TOCTOU race condition.')`.
|
|
213
|
+
- If `strictAtomicity: false`: Falls back to two-step check-and-delete (`get` followed by `del`).
|
|
214
|
+
|
|
215
|
+
#### `withLock<T>(lockKey: string, fn: () => Promise<T>, ttlMs: number = 10000): Promise<T>`
|
|
216
|
+
Executes an asynchronous function wrapped in single-flight lock protection:
|
|
217
|
+
1. Calls `acquire(lockKey, ttlMs)`.
|
|
218
|
+
2. If acquisition fails, throws `Error('Failed to acquire lock for key: ${lockKey} (in-use or concurrent execution)')`.
|
|
219
|
+
3. Executes `await fn()`.
|
|
220
|
+
4. In a `finally` block, releases the lock via `release(lockKey, token).catch(() => {})`, guaranteeing no leaked locks even on unhandled exceptions.
|
|
144
221
|
|
|
145
|
-
|
|
222
|
+
---
|
|
223
|
+
|
|
224
|
+
## 8. `RedisNonceStore` (Webhook Replay Attack Prevention)
|
|
146
225
|
|
|
147
|
-
|
|
148
|
-
|
|
226
|
+
`RedisNonceStore` provides distributed, atomic signature and nonce deduplication for webhook endpoints and signature verification (used in `@siglo/crypto` and standalone webhook handlers).
|
|
227
|
+
|
|
228
|
+
```typescript
|
|
229
|
+
import { RedisNonceStore } from '@siglo/redis';
|
|
149
230
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
return redis.call("del", KEYS[1])
|
|
155
|
-
else
|
|
156
|
-
return 0
|
|
157
|
-
end
|
|
231
|
+
const dedupStore = new RedisNonceStore({
|
|
232
|
+
client: redis,
|
|
233
|
+
keyPrefix: 'siglo:wh:dedup:', // default
|
|
234
|
+
});
|
|
158
235
|
```
|
|
159
|
-
When `strictAtomicity` is enabled, failure or absence of Lua `eval` will throw an error to prevent non-atomic release races.
|
|
160
236
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
237
|
+
### Options (`RedisNonceStoreOptions`)
|
|
238
|
+
|
|
239
|
+
| Option | Type | Default | Description |
|
|
240
|
+
| :--- | :--- | :--- | :--- |
|
|
241
|
+
| `client` | `RedisClientInterface` | *Required* | Active Redis client instance. Throws if omitted. |
|
|
242
|
+
| `keyPrefix` | `string` | `'siglo:wh:dedup:'` | Key prefix for replay prevention tokens. |
|
|
243
|
+
|
|
244
|
+
### Methods
|
|
245
|
+
|
|
246
|
+
#### `claim(key: string, ttlSeconds: number): Promise<boolean>`
|
|
247
|
+
- Calculates `ttlMs = Math.max(1, Math.floor(ttlSeconds * 1000))`.
|
|
248
|
+
- Executes atomic `SET <keyPrefix><key> "1" PX <ttlMs> NX`.
|
|
249
|
+
- Returns `true` if the key was claimed (first delivery attempt within tolerance window).
|
|
250
|
+
- Returns `false` if the key already exists (replay attempt detected, request should be rejected).
|
|
167
251
|
|
|
168
252
|
---
|
|
169
253
|
|
|
170
|
-
##
|
|
254
|
+
## 9. Complete Production Example
|
|
171
255
|
|
|
172
256
|
```typescript
|
|
173
257
|
import { Redis } from 'ioredis';
|
|
174
|
-
import { Step, flow } from '@siglo/core';
|
|
258
|
+
import { Step, flow, useStorage } from '@siglo/core';
|
|
175
259
|
import {
|
|
176
260
|
RedisStorageAdapter,
|
|
177
261
|
RedisIdempotencyStore,
|
|
178
|
-
RedisLock
|
|
262
|
+
RedisLock,
|
|
263
|
+
RedisNonceStore,
|
|
179
264
|
} from '@siglo/redis';
|
|
180
265
|
|
|
181
266
|
// 1. Initialize Redis Client
|
|
182
267
|
const redis = new Redis('redis://localhost:6379');
|
|
183
268
|
|
|
184
|
-
// 2. Register Global Storage
|
|
269
|
+
// 2. Register Global Storage Adapter
|
|
185
270
|
const storageAdapter = new RedisStorageAdapter({
|
|
186
271
|
client: redis,
|
|
187
272
|
checkpointTtlMs: 86400000, // 24 hours
|
|
273
|
+
cacheTtlMs: 300000, // 5 minutes
|
|
274
|
+
recordContextChanges: true,
|
|
188
275
|
});
|
|
189
|
-
|
|
276
|
+
useStorage(storageAdapter);
|
|
190
277
|
|
|
191
|
-
// 3. Initialize Idempotency &
|
|
278
|
+
// 3. Initialize Idempotency, Mutex Lock & Webhook Dedup
|
|
192
279
|
const idempotencyStore = new RedisIdempotencyStore({ client: redis });
|
|
193
280
|
const redisLock = new RedisLock(redis, { strictAtomicity: true });
|
|
281
|
+
const nonceStore = new RedisNonceStore({ client: redis });
|
|
194
282
|
|
|
195
283
|
// 4. Define Workflow with Step Caching
|
|
196
284
|
const fetchExchangeRates = Step.create('fetchRates', async () => {
|
|
197
|
-
//
|
|
285
|
+
// External HTTP call cached for 5 minutes
|
|
198
286
|
return { EUR: 1.08, GBP: 1.27, JPY: 154.2 };
|
|
199
|
-
})
|
|
200
|
-
.cache({ ttlMs: 300000, key: () => 'currency-rates-latest' }) // 5-minute cache
|
|
201
|
-
;
|
|
287
|
+
}).cache({ ttlMs: 300000, key: () => 'currency-rates-latest' });
|
|
202
288
|
|
|
203
289
|
const settleInvoice = Step.create('settleInvoice', async ({ invoiceId, amount, currency }: any) => {
|
|
204
290
|
return { invoiceId, settled: true, amountUSD: amount * 1.08 };
|
|
@@ -206,9 +292,8 @@ const settleInvoice = Step.create('settleInvoice', async ({ invoiceId, amount, c
|
|
|
206
292
|
|
|
207
293
|
const billingFlow = flow(fetchExchangeRates, settleInvoice);
|
|
208
294
|
|
|
209
|
-
// 5. Single-Flight Execution Runner
|
|
295
|
+
// 5. Single-Flight Execution Runner with Distributed Lock
|
|
210
296
|
async function processDailySettlement() {
|
|
211
|
-
// Only one cluster node can execute this block at a time
|
|
212
297
|
return await redisLock.withLock('lock:daily-settlement', async () => {
|
|
213
298
|
console.log('Acquired single-flight lock. Starting settlement flow...');
|
|
214
299
|
return await billingFlow.run({
|
|
@@ -219,7 +304,34 @@ async function processDailySettlement() {
|
|
|
219
304
|
}, 15000); // 15-second lock timeout
|
|
220
305
|
}
|
|
221
306
|
|
|
222
|
-
// 6.
|
|
307
|
+
// 6. Webhook Ingestion with Replay Protection & Idempotency
|
|
308
|
+
async function handleWebhook(signature: string, payload: { id: string; amount: number }) {
|
|
309
|
+
// Check replay attack window (e.g. 5 minutes = 300 seconds)
|
|
310
|
+
const fresh = await nonceStore.claim(signature, 300);
|
|
311
|
+
if (!fresh) {
|
|
312
|
+
throw new Error('Duplicate or replayed webhook signature detected');
|
|
313
|
+
}
|
|
314
|
+
|
|
315
|
+
// Atomic idempotency claim
|
|
316
|
+
const claimState = await idempotencyStore.claim(payload.id);
|
|
317
|
+
if (claimState === 'completed') {
|
|
318
|
+
return await idempotencyStore.getResult(payload.id);
|
|
319
|
+
}
|
|
320
|
+
if (claimState === 'in-progress') {
|
|
321
|
+
throw new Error('Concurrent execution already in progress');
|
|
322
|
+
}
|
|
323
|
+
|
|
324
|
+
try {
|
|
325
|
+
const result = { processed: true, id: payload.id };
|
|
326
|
+
await idempotencyStore.complete(payload.id, result);
|
|
327
|
+
return result;
|
|
328
|
+
} catch (err) {
|
|
329
|
+
await idempotencyStore.fail(payload.id, err);
|
|
330
|
+
throw err;
|
|
331
|
+
}
|
|
332
|
+
}
|
|
333
|
+
|
|
334
|
+
// 7. Graceful Shutdown
|
|
223
335
|
process.on('SIGTERM', async () => {
|
|
224
336
|
await redis.quit();
|
|
225
337
|
process.exit(0);
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@siglo/redis",
|
|
3
|
-
"version": "1.0
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "1.1.0",
|
|
4
|
+
"description": "Redis adapter for Siglo TypeScript workflows with distributed caching, durable checkpoints, locking, and idempotency.",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"types": "dist/index.d.ts",
|
|
7
7
|
"exports": {
|
|
@@ -37,7 +37,7 @@
|
|
|
37
37
|
"bugs": {
|
|
38
38
|
"url": "https://github.com/jsoftgem/siglo/issues"
|
|
39
39
|
},
|
|
40
|
-
"homepage": "https://github.com/jsoftgem/siglo#readme",
|
|
40
|
+
"homepage": "https://github.com/jsoftgem/siglo/tree/main/packages/redis#readme",
|
|
41
41
|
"dependencies": {
|
|
42
42
|
"@siglo/core": "^1.0.0"
|
|
43
43
|
},
|
|
@@ -57,5 +57,16 @@
|
|
|
57
57
|
"mocha": "^12.0.3",
|
|
58
58
|
"tsx": "^4.23.15",
|
|
59
59
|
"typescript": "~5.7.0"
|
|
60
|
-
}
|
|
60
|
+
},
|
|
61
|
+
"keywords": [
|
|
62
|
+
"siglo",
|
|
63
|
+
"workflow",
|
|
64
|
+
"typescript",
|
|
65
|
+
"redis",
|
|
66
|
+
"cache",
|
|
67
|
+
"distributed-cache",
|
|
68
|
+
"checkpoint",
|
|
69
|
+
"distributed-lock",
|
|
70
|
+
"idempotency"
|
|
71
|
+
]
|
|
61
72
|
}
|