outbox-event-bus 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -6
- package/package.json +2 -2
- package/docs/docs/API_REFERENCE.md +0 -717
- package/docs/docs/CONTRIBUTING.md +0 -43
- package/docs/docs/PUBLISHING.md +0 -144
- package/docs/docs/images/choose_publisher.png +0 -0
- package/docs/docs/images/event_life_cycle.png +0 -0
- package/docs/docs/images/problem.png +0 -0
- package/docs/docs/images/solution.png +0 -0
package/README.md
CHANGED
|
@@ -3,7 +3,6 @@
|
|
|
3
3
|
<div align="center">
|
|
4
4
|
|
|
5
5
|

|
|
6
|
-

|
|
7
6
|

|
|
8
7
|

|
|
9
8
|
|
|
@@ -15,11 +14,11 @@
|
|
|
15
14
|
|
|
16
15
|
**The Problem**: You save data to your database and attempt to emit a relevant event. If your process crashes or the network fails before the event is sent, your system becomes inconsistent.
|
|
17
16
|
|
|
18
|
-

|
|
19
18
|
|
|
20
19
|
**The Solution**: `outbox-event-bus` stores events in your database *within the same transaction* as your data. A background worker then reliably delivers them.
|
|
21
20
|
|
|
22
|
-

|
|
23
22
|
|
|
24
23
|
## Quick Start (Postgres + Drizzle ORM + SQS Example)
|
|
25
24
|
|
|
@@ -93,7 +92,6 @@ await db.transaction(async (transaction) => {
|
|
|
93
92
|
|
|
94
93
|
### Strict 1:1 Command Bus Pattern
|
|
95
94
|
|
|
96
|
-
> [!IMPORTANT]
|
|
97
95
|
> This library enforces a **Command Bus pattern**: Each event type can have exactly **one** handler.
|
|
98
96
|
|
|
99
97
|
**Why?**
|
|
@@ -129,7 +127,7 @@ bus.on('send.analytics', async (event) => {
|
|
|
129
127
|
|
|
130
128
|
Events flow through several states from creation to completion:
|
|
131
129
|
|
|
132
|
-

|
|
133
131
|
|
|
134
132
|
**State Descriptions:**
|
|
135
133
|
|
|
@@ -429,7 +427,7 @@ These send your events to the world.
|
|
|
429
427
|
|
|
430
428
|
### Choosing the Right Publisher
|
|
431
429
|
|
|
432
|
-

|
|
433
431
|
|
|
434
432
|
## Production Guide
|
|
435
433
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "outbox-event-bus",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.1",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -37,7 +37,7 @@
|
|
|
37
37
|
"typescript": "5.9.3",
|
|
38
38
|
"vite-tsconfig-paths": "^6.0.3",
|
|
39
39
|
"vitest": "^4.0.16",
|
|
40
|
-
"@outbox-event-bus/config": "
|
|
40
|
+
"@outbox-event-bus/config": "1.0.1"
|
|
41
41
|
},
|
|
42
42
|
"scripts": {
|
|
43
43
|
"build": "tsdown",
|
|
@@ -1,717 +0,0 @@
|
|
|
1
|
-
# API Reference
|
|
2
|
-
|
|
3
|
-
## Core Components
|
|
4
|
-
|
|
5
|
-
### OutboxEventBus
|
|
6
|
-
|
|
7
|
-
The main orchestrator for event emission and handling. The API is inspired by [the Node.js EventEmitter](https://nodejs.org/api/events.html#class-eventemitter).
|
|
8
|
-
|
|
9
|
-
#### Constructor
|
|
10
|
-
|
|
11
|
-
```typescript
|
|
12
|
-
new OutboxEventBus<TTransaction = unknown>(
|
|
13
|
-
outbox: IOutbox<TTransaction>,
|
|
14
|
-
onError: ErrorHandler
|
|
15
|
-
)
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
**Parameters:**
|
|
19
|
-
- `outbox`: Storage adapter implementing `IOutbox` interface
|
|
20
|
-
- `onError`: Required error handler for processing failures. Receives `OutboxError` with event bundled in `error.context?.event` for event-related errors.
|
|
21
|
-
|
|
22
|
-
#### Methods
|
|
23
|
-
|
|
24
|
-
##### `start()`
|
|
25
|
-
Starts the background worker that polls and processes events.
|
|
26
|
-
|
|
27
|
-
```typescript
|
|
28
|
-
bus.start(): void
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
##### `stop()`
|
|
32
|
-
Stops the background worker and cleans up resources.
|
|
33
|
-
|
|
34
|
-
```typescript
|
|
35
|
-
bus.stop(): Promise<void>
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
##### `emit()`
|
|
39
|
-
Emits a single event to the outbox.
|
|
40
|
-
|
|
41
|
-
```typescript
|
|
42
|
-
bus.emit(
|
|
43
|
-
event: BusEventInput,
|
|
44
|
-
transaction?: TTransaction
|
|
45
|
-
): Promise<void>
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
**Parameters:**
|
|
49
|
-
- `event`: Event object with `type` and `payload`
|
|
50
|
-
- `transaction`: Optional transaction context
|
|
51
|
-
|
|
52
|
-
**Example:**
|
|
53
|
-
```typescript
|
|
54
|
-
await bus.emit({ type: 'user.created', payload: user }, transaction);
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
##### `emitMany()`
|
|
58
|
-
Emits multiple events in a single operation.
|
|
59
|
-
|
|
60
|
-
```typescript
|
|
61
|
-
bus.emitMany(
|
|
62
|
-
events: BusEventInput[],
|
|
63
|
-
transaction?: TTransaction
|
|
64
|
-
): Promise<void>
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
##### `on()`
|
|
68
|
-
Registers a handler for a specific event type.
|
|
69
|
-
|
|
70
|
-
```typescript
|
|
71
|
-
bus.on(
|
|
72
|
-
eventType: string,
|
|
73
|
-
handler: (event: BusEvent) => Promise<void>
|
|
74
|
-
): this
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
**Note:** Only one handler per event type (1:1 Command Bus pattern).
|
|
78
|
-
|
|
79
|
-
##### `off()`
|
|
80
|
-
Removes the handler for a specific event type.
|
|
81
|
-
|
|
82
|
-
```typescript
|
|
83
|
-
bus.off(
|
|
84
|
-
eventType: string,
|
|
85
|
-
handler: (event: BusEvent) => Promise<void>
|
|
86
|
-
): this
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
> [!TIP]
|
|
90
|
-
> `off()` and `removeListener()` are equivalent methods. Similarly, `on()` and `addListener()` are equivalent. Use whichever naming convention you prefer.
|
|
91
|
-
|
|
92
|
-
##### `removeAllListeners()`
|
|
93
|
-
Removes all registered handlers.
|
|
94
|
-
|
|
95
|
-
```typescript
|
|
96
|
-
bus.removeAllListeners(eventType?: string): this
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
##### `subscribe()`
|
|
100
|
-
|
|
101
|
-
Subscribes a single handler to multiple event types.
|
|
102
|
-
|
|
103
|
-
```typescript
|
|
104
|
-
bus.subscribe(
|
|
105
|
-
eventTypes: string[],
|
|
106
|
-
handler: (event: BusEvent) => Promise<void>
|
|
107
|
-
): this
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
**Parameters:**
|
|
111
|
-
|
|
112
|
-
- `eventTypes`: Array of event type strings
|
|
113
|
-
- `handler`: Function to handle the events
|
|
114
|
-
|
|
115
|
-
**Example:**
|
|
116
|
-
|
|
117
|
-
```typescript
|
|
118
|
-
bus.subscribe(['user.created', 'user.updated'], async (event) => {
|
|
119
|
-
console.log('User changed:', event.type);
|
|
120
|
-
});
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
##### `waitFor()`
|
|
124
|
-
Waits for a specific event type to be processed (useful for testing).
|
|
125
|
-
|
|
126
|
-
```typescript
|
|
127
|
-
bus.waitFor(
|
|
128
|
-
eventType: string,
|
|
129
|
-
timeoutMs?: number
|
|
130
|
-
): Promise<BusEvent>
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
##### `getFailedEvents()`
|
|
134
|
-
Retrieves a list of events that have failed processing.
|
|
135
|
-
|
|
136
|
-
```typescript
|
|
137
|
-
bus.getFailedEvents(): Promise<FailedBusEvent[]>
|
|
138
|
-
```
|
|
139
|
-
|
|
140
|
-
##### `retryEvents()`
|
|
141
|
-
Resets the status of failed events to 'created' so they can be retried.
|
|
142
|
-
|
|
143
|
-
```typescript
|
|
144
|
-
bus.retryEvents(eventIds: string[]): Promise<void>
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
### InMemoryOutbox
|
|
148
|
-
|
|
149
|
-
A lightweight in-memory outbox, primarily useful for testing or non-persistent workflows.
|
|
150
|
-
|
|
151
|
-
#### Constructor
|
|
152
|
-
|
|
153
|
-
```typescript
|
|
154
|
-
new InMemoryOutbox(config?: InMemoryOutboxConfig)
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
#### Configuration
|
|
158
|
-
|
|
159
|
-
```typescript
|
|
160
|
-
interface InMemoryOutboxConfig {
|
|
161
|
-
maxRetries?: number; // Default: 3
|
|
162
|
-
}
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
### IOutbox Interface
|
|
166
|
-
|
|
167
|
-
Storage adapters must implement this interface.
|
|
168
|
-
|
|
169
|
-
```typescript
|
|
170
|
-
interface IOutbox<TTransaction = unknown> {
|
|
171
|
-
publish(events: BusEvent[], transaction?: TTransaction): Promise<void>;
|
|
172
|
-
start(handler: (event: BusEvent) => Promise<void>, onError: ErrorHandler): void;
|
|
173
|
-
stop(): Promise<void>;
|
|
174
|
-
getFailedEvents(): Promise<FailedBusEvent[]>;
|
|
175
|
-
retryEvents(eventIds: string[]): Promise<void>;
|
|
176
|
-
}
|
|
177
|
-
```
|
|
178
|
-
|
|
179
|
-
### IOutboxEventBus Interface
|
|
180
|
-
|
|
181
|
-
Public interface for the event bus, useful for dependency injection.
|
|
182
|
-
|
|
183
|
-
```typescript
|
|
184
|
-
interface IOutboxEventBus<TTransaction> {
|
|
185
|
-
emit<T extends string, P>(
|
|
186
|
-
event: BusEventInput<T, P>,
|
|
187
|
-
transaction?: TTransaction
|
|
188
|
-
): Promise<void>;
|
|
189
|
-
emitMany<T extends string, P>(
|
|
190
|
-
events: BusEventInput<T, P>[],
|
|
191
|
-
transaction?: TTransaction
|
|
192
|
-
): Promise<void>;
|
|
193
|
-
on<T extends string, P = unknown>(
|
|
194
|
-
eventType: T,
|
|
195
|
-
handler: (event: BusEvent<T, P>) => Promise<void>
|
|
196
|
-
): this;
|
|
197
|
-
addListener<T extends string, P = unknown>(
|
|
198
|
-
eventType: T,
|
|
199
|
-
handler: (event: BusEvent<T, P>) => Promise<void>
|
|
200
|
-
): this;
|
|
201
|
-
off<T extends string, P = unknown>(
|
|
202
|
-
eventType: T,
|
|
203
|
-
handler: (event: BusEvent<T, P>) => Promise<void>
|
|
204
|
-
): this;
|
|
205
|
-
removeListener<T extends string, P = unknown>(
|
|
206
|
-
eventType: T,
|
|
207
|
-
handler: (event: BusEvent<T, P>) => Promise<void>
|
|
208
|
-
): this;
|
|
209
|
-
once<T extends string, P = unknown>(
|
|
210
|
-
eventType: T,
|
|
211
|
-
handler: (event: BusEvent<T, P>) => Promise<void>
|
|
212
|
-
): this;
|
|
213
|
-
removeAllListeners<T extends string>(eventType?: T): this;
|
|
214
|
-
getSubscriptionCount(): number;
|
|
215
|
-
listenerCount(eventType: string): number;
|
|
216
|
-
getListener(eventType: string): AnyListener | undefined;
|
|
217
|
-
eventNames(): string[];
|
|
218
|
-
start(): void;
|
|
219
|
-
stop(): Promise<void>;
|
|
220
|
-
subscribe<T extends string, P = unknown>(
|
|
221
|
-
eventTypes: T[],
|
|
222
|
-
handler: (event: BusEvent<T, P>) => Promise<void>
|
|
223
|
-
): this;
|
|
224
|
-
waitFor<T extends string, P = unknown>(
|
|
225
|
-
eventType: T,
|
|
226
|
-
timeoutMs?: number
|
|
227
|
-
): Promise<BusEvent<T, P>>;
|
|
228
|
-
getFailedEvents(): Promise<FailedBusEvent[]>;
|
|
229
|
-
retryEvents(eventIds: string[]): Promise<void>;
|
|
230
|
-
}
|
|
231
|
-
```
|
|
232
|
-
|
|
233
|
-
## Event Types
|
|
234
|
-
|
|
235
|
-
### `BusEventInput`
|
|
236
|
-
Event structure when emitting (before persistence). Optional `id` and `occurredAt` are auto-generated if not provided.
|
|
237
|
-
|
|
238
|
-
```typescript
|
|
239
|
-
interface BusEventInput<T extends string = string, P = unknown> {
|
|
240
|
-
id?: string; // Optional, auto-generated if not provided
|
|
241
|
-
type: T; // Event type (e.g., 'user.created')
|
|
242
|
-
payload: P; // Event data
|
|
243
|
-
occurredAt?: Date; // Optional, auto-set to now if not provided
|
|
244
|
-
metadata?: Record<string, unknown>; // Optional metadata
|
|
245
|
-
}
|
|
246
|
-
```
|
|
247
|
-
|
|
248
|
-
### `BusEvent`
|
|
249
|
-
Event structure after persistence (includes required metadata).
|
|
250
|
-
|
|
251
|
-
```typescript
|
|
252
|
-
interface BusEvent<T extends string = string, P = unknown> {
|
|
253
|
-
id: string; // Unique event ID (required)
|
|
254
|
-
type: T; // Event type
|
|
255
|
-
payload: P; // Event data
|
|
256
|
-
occurredAt: Date; // When the event occurred (required)
|
|
257
|
-
metadata?: Record<string, unknown>; // Optional metadata
|
|
258
|
-
}
|
|
259
|
-
```
|
|
260
|
-
|
|
261
|
-
### `FailedBusEvent`
|
|
262
|
-
Represents an event that has failed processing.
|
|
263
|
-
|
|
264
|
-
```typescript
|
|
265
|
-
type FailedBusEvent<T extends string = string, P = unknown> = BusEvent<T, P> & {
|
|
266
|
-
error?: string;
|
|
267
|
-
retryCount: number;
|
|
268
|
-
lastAttemptAt?: Date;
|
|
269
|
-
}
|
|
270
|
-
```
|
|
271
|
-
|
|
272
|
-
### `EventStatus`
|
|
273
|
-
Possible states for an event in the outbox.
|
|
274
|
-
|
|
275
|
-
```typescript
|
|
276
|
-
const EventStatus = {
|
|
277
|
-
CREATED: 'created',
|
|
278
|
-
ACTIVE: 'active',
|
|
279
|
-
FAILED: 'failed',
|
|
280
|
-
COMPLETED: 'completed',
|
|
281
|
-
} as const;
|
|
282
|
-
```
|
|
283
|
-
|
|
284
|
-
## Adapters
|
|
285
|
-
|
|
286
|
-
### PostgresPrismaOutbox
|
|
287
|
-
|
|
288
|
-
Adapter for PostgreSQL using Prisma.
|
|
289
|
-
|
|
290
|
-
#### Constructor
|
|
291
|
-
|
|
292
|
-
```typescript
|
|
293
|
-
new PostgresPrismaOutbox(config: PostgresPrismaOutboxConfig)
|
|
294
|
-
```
|
|
295
|
-
|
|
296
|
-
#### Configuration
|
|
297
|
-
|
|
298
|
-
```typescript
|
|
299
|
-
interface PostgresPrismaOutboxConfig extends OutboxConfig {
|
|
300
|
-
prisma: PrismaClient;
|
|
301
|
-
getTransaction?: () => PrismaClient | undefined; // Optional context getter
|
|
302
|
-
models?: {
|
|
303
|
-
outbox?: string; // Default: "outboxEvent"
|
|
304
|
-
archive?: string; // Default: "outboxEventArchive"
|
|
305
|
-
};
|
|
306
|
-
tableName?: string; // Default: "outbox_events"
|
|
307
|
-
}
|
|
308
|
-
```
|
|
309
|
-
|
|
310
|
-
### DynamoDBAwsSdkOutbox
|
|
311
|
-
|
|
312
|
-
Adapter for AWS DynamoDB.
|
|
313
|
-
|
|
314
|
-
#### Constructor
|
|
315
|
-
|
|
316
|
-
```typescript
|
|
317
|
-
new DynamoDBAwsSdkOutbox(config: DynamoDBAwsSdkOutboxConfig)
|
|
318
|
-
```
|
|
319
|
-
|
|
320
|
-
#### Configuration
|
|
321
|
-
|
|
322
|
-
```typescript
|
|
323
|
-
interface DynamoDBAwsSdkOutboxConfig extends OutboxConfig {
|
|
324
|
-
client: DynamoDBClient;
|
|
325
|
-
tableName: string;
|
|
326
|
-
statusIndexName?: string; // Default: "status-gsiSortKey-index"
|
|
327
|
-
processingTimeoutMs?: number; // Time before stuck events are retried (default: 30000ms)
|
|
328
|
-
getCollector?: () => DynamoDBAwsSdkTransactionCollector | undefined;
|
|
329
|
-
}
|
|
330
|
-
```
|
|
331
|
-
|
|
332
|
-
### MongoMongodbOutbox
|
|
333
|
-
|
|
334
|
-
Adapter for MongoDB.
|
|
335
|
-
|
|
336
|
-
#### Constructor
|
|
337
|
-
|
|
338
|
-
```typescript
|
|
339
|
-
new MongoMongodbOutbox(config: MongoMongodbOutboxConfig)
|
|
340
|
-
```
|
|
341
|
-
|
|
342
|
-
#### Configuration
|
|
343
|
-
|
|
344
|
-
```typescript
|
|
345
|
-
interface MongoMongodbOutboxConfig extends OutboxConfig {
|
|
346
|
-
client: MongoClient;
|
|
347
|
-
dbName: string;
|
|
348
|
-
collectionName?: string; // Default: "outbox_events"
|
|
349
|
-
getSession?: () => ClientSession | undefined;
|
|
350
|
-
}
|
|
351
|
-
```
|
|
352
|
-
|
|
353
|
-
### PostgresDrizzleOutbox
|
|
354
|
-
|
|
355
|
-
Adapter for PostgreSQL using Drizzle ORM.
|
|
356
|
-
|
|
357
|
-
#### Constructor
|
|
358
|
-
|
|
359
|
-
```typescript
|
|
360
|
-
new PostgresDrizzleOutbox(config: PostgresDrizzleOutboxConfig)
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
#### Configuration
|
|
364
|
-
|
|
365
|
-
```typescript
|
|
366
|
-
interface PostgresDrizzleOutboxConfig extends OutboxConfig {
|
|
367
|
-
db: PostgresJsDatabase<Record<string, unknown>>;
|
|
368
|
-
getTransaction?: () => PostgresJsDatabase<Record<string, unknown>> | undefined;
|
|
369
|
-
tables?: {
|
|
370
|
-
outboxEvents: Table;
|
|
371
|
-
outboxEventsArchive?: Table;
|
|
372
|
-
};
|
|
373
|
-
}
|
|
374
|
-
```
|
|
375
|
-
|
|
376
|
-
### RedisIoRedisOutbox
|
|
377
|
-
|
|
378
|
-
Adapter for Redis (using IOredis).
|
|
379
|
-
|
|
380
|
-
#### Constructor
|
|
381
|
-
|
|
382
|
-
```typescript
|
|
383
|
-
new RedisIoRedisOutbox(config: RedisIoRedisOutboxConfig)
|
|
384
|
-
```
|
|
385
|
-
|
|
386
|
-
#### Configuration
|
|
387
|
-
|
|
388
|
-
```typescript
|
|
389
|
-
interface RedisIoRedisOutboxConfig extends OutboxConfig {
|
|
390
|
-
redis: Redis;
|
|
391
|
-
keyPrefix?: string; // Default: "outbox"
|
|
392
|
-
processingTimeoutMs?: number; // Default: 30000ms
|
|
393
|
-
getPipeline?: () => ChainableCommander | undefined;
|
|
394
|
-
}
|
|
395
|
-
```
|
|
396
|
-
|
|
397
|
-
### SqliteBetterSqlite3Outbox
|
|
398
|
-
|
|
399
|
-
Adapter for SQLite (using better-sqlite3).
|
|
400
|
-
|
|
401
|
-
#### Constructor
|
|
402
|
-
|
|
403
|
-
```typescript
|
|
404
|
-
new SqliteBetterSqlite3Outbox(config: SqliteBetterSqlite3OutboxConfig)
|
|
405
|
-
```
|
|
406
|
-
|
|
407
|
-
#### Configuration
|
|
408
|
-
|
|
409
|
-
```typescript
|
|
410
|
-
interface SqliteBetterSqlite3OutboxConfig extends OutboxConfig {
|
|
411
|
-
dbPath?: string;
|
|
412
|
-
db?: Database.Database;
|
|
413
|
-
getTransaction?: () => Database.Database | undefined;
|
|
414
|
-
tableName?: string; // Default: "outbox_events"
|
|
415
|
-
archiveTableName?: string; // Default: "outbox_events_archive"
|
|
416
|
-
}
|
|
417
|
-
```
|
|
418
|
-
|
|
419
|
-
### InMemoryOutbox
|
|
420
|
-
|
|
421
|
-
Simple in-memory adapter for testing and development. Does not provide persistence across restarts.
|
|
422
|
-
|
|
423
|
-
#### Constructor
|
|
424
|
-
|
|
425
|
-
```typescript
|
|
426
|
-
new InMemoryOutbox(config?: InMemoryOutboxConfig)
|
|
427
|
-
```
|
|
428
|
-
|
|
429
|
-
#### Configuration
|
|
430
|
-
|
|
431
|
-
```typescript
|
|
432
|
-
interface InMemoryOutboxConfig extends OutboxConfig {
|
|
433
|
-
maxEvents?: number; // Optional limit for memory safety
|
|
434
|
-
}
|
|
435
|
-
```
|
|
436
|
-
|
|
437
|
-
## Publishers
|
|
438
|
-
|
|
439
|
-
### SQSPublisher
|
|
440
|
-
|
|
441
|
-
Publisher for AWS SQS.
|
|
442
|
-
|
|
443
|
-
#### Constructor
|
|
444
|
-
|
|
445
|
-
```typescript
|
|
446
|
-
new SQSPublisher(
|
|
447
|
-
bus: IOutboxEventBus<TTransaction>,
|
|
448
|
-
config: SQSPublisherConfig
|
|
449
|
-
)
|
|
450
|
-
```
|
|
451
|
-
|
|
452
|
-
#### Configuration
|
|
453
|
-
|
|
454
|
-
```typescript
|
|
455
|
-
interface SQSPublisherConfig extends PublisherConfig {
|
|
456
|
-
sqsClient: SQSClient;
|
|
457
|
-
queueUrl: string;
|
|
458
|
-
}
|
|
459
|
-
```
|
|
460
|
-
|
|
461
|
-
### SNSPublisher
|
|
462
|
-
|
|
463
|
-
Publisher for AWS SNS.
|
|
464
|
-
|
|
465
|
-
#### Constructor
|
|
466
|
-
|
|
467
|
-
```typescript
|
|
468
|
-
new SNSPublisher(
|
|
469
|
-
bus: IOutboxEventBus<TTransaction>,
|
|
470
|
-
config: SNSPublisherConfig
|
|
471
|
-
)
|
|
472
|
-
```
|
|
473
|
-
|
|
474
|
-
#### Configuration
|
|
475
|
-
|
|
476
|
-
```typescript
|
|
477
|
-
interface SNSPublisherConfig extends PublisherConfig {
|
|
478
|
-
snsClient: SNSClient;
|
|
479
|
-
topicArn: string;
|
|
480
|
-
}
|
|
481
|
-
```
|
|
482
|
-
|
|
483
|
-
### EventBridgePublisher
|
|
484
|
-
|
|
485
|
-
Publisher for AWS EventBridge.
|
|
486
|
-
|
|
487
|
-
#### Constructor
|
|
488
|
-
|
|
489
|
-
```typescript
|
|
490
|
-
new EventBridgePublisher(
|
|
491
|
-
bus: IOutboxEventBus<TTransaction>,
|
|
492
|
-
config: EventBridgePublisherConfig
|
|
493
|
-
)
|
|
494
|
-
```
|
|
495
|
-
|
|
496
|
-
#### Configuration
|
|
497
|
-
|
|
498
|
-
```typescript
|
|
499
|
-
interface EventBridgePublisherConfig extends PublisherConfig {
|
|
500
|
-
eventBridgeClient: EventBridgeClient;
|
|
501
|
-
eventBusName?: string;
|
|
502
|
-
source: string;
|
|
503
|
-
}
|
|
504
|
-
```
|
|
505
|
-
|
|
506
|
-
### KafkaPublisher
|
|
507
|
-
|
|
508
|
-
Publisher for Apache Kafka (using kafkajs).
|
|
509
|
-
|
|
510
|
-
#### Constructor
|
|
511
|
-
|
|
512
|
-
```typescript
|
|
513
|
-
new KafkaPublisher(
|
|
514
|
-
bus: IOutboxEventBus<TTransaction>,
|
|
515
|
-
config: KafkaPublisherConfig
|
|
516
|
-
)
|
|
517
|
-
```
|
|
518
|
-
|
|
519
|
-
#### Configuration
|
|
520
|
-
|
|
521
|
-
```typescript
|
|
522
|
-
interface KafkaPublisherConfig extends PublisherConfig {
|
|
523
|
-
producer: Producer;
|
|
524
|
-
topic: string;
|
|
525
|
-
}
|
|
526
|
-
```
|
|
527
|
-
|
|
528
|
-
### RabbitMQPublisher
|
|
529
|
-
|
|
530
|
-
Publisher for RabbitMQ (using amqplib).
|
|
531
|
-
|
|
532
|
-
#### Constructor
|
|
533
|
-
|
|
534
|
-
```typescript
|
|
535
|
-
new RabbitMQPublisher(
|
|
536
|
-
bus: IOutboxEventBus<TTransaction>,
|
|
537
|
-
config: RabbitMQPublisherConfig
|
|
538
|
-
)
|
|
539
|
-
```
|
|
540
|
-
|
|
541
|
-
#### Configuration
|
|
542
|
-
|
|
543
|
-
```typescript
|
|
544
|
-
interface RabbitMQPublisherConfig extends PublisherConfig {
|
|
545
|
-
channel: Channel;
|
|
546
|
-
exchange: string;
|
|
547
|
-
routingKey?: string;
|
|
548
|
-
}
|
|
549
|
-
```
|
|
550
|
-
|
|
551
|
-
### RedisStreamsPublisher
|
|
552
|
-
|
|
553
|
-
Publisher for Redis Streams.
|
|
554
|
-
|
|
555
|
-
#### Constructor
|
|
556
|
-
|
|
557
|
-
```typescript
|
|
558
|
-
new RedisStreamsPublisher(
|
|
559
|
-
bus: IOutboxEventBus<TTransaction>,
|
|
560
|
-
config: RedisStreamsPublisherConfig
|
|
561
|
-
)
|
|
562
|
-
```
|
|
563
|
-
|
|
564
|
-
#### Configuration
|
|
565
|
-
|
|
566
|
-
```typescript
|
|
567
|
-
interface RedisStreamsPublisherConfig extends PublisherConfig {
|
|
568
|
-
redisClient: Redis;
|
|
569
|
-
streamKey: string;
|
|
570
|
-
}
|
|
571
|
-
```
|
|
572
|
-
|
|
573
|
-
## Global Configuration
|
|
574
|
-
|
|
575
|
-
### Base Outbox Configuration
|
|
576
|
-
|
|
577
|
-
All adapters inherit from `OutboxConfig`.
|
|
578
|
-
|
|
579
|
-
```typescript
|
|
580
|
-
interface OutboxConfig {
|
|
581
|
-
maxRetries?: number; // Max retry attempts (default: 5)
|
|
582
|
-
baseBackoffMs?: number; // Initial delay before retry (default: 1000ms)
|
|
583
|
-
pollIntervalMs?: number; // How often to poll for events (default: 1000ms)
|
|
584
|
-
batchSize?: number; // Max events per batch (default: 50)
|
|
585
|
-
processingTimeoutMs?: number; // Timeout for processing/locking (default: 30000ms)
|
|
586
|
-
maxErrorBackoffMs?: number; // Max backoff delay (default: 30000ms)
|
|
587
|
-
}
|
|
588
|
-
```
|
|
589
|
-
|
|
590
|
-
> [!NOTE]
|
|
591
|
-
> All outbox adapters implement an exponential backoff with a **+/- 10% jitter** to prevent thundering herd problems when multiple workers are recovering from failures.
|
|
592
|
-
|
|
593
|
-
### Publisher Configuration
|
|
594
|
-
|
|
595
|
-
All publishers inherit from `PublisherConfig`.
|
|
596
|
-
|
|
597
|
-
```typescript
|
|
598
|
-
type PublisherConfig = {
|
|
599
|
-
retryConfig?: RetryOptions;
|
|
600
|
-
processingConfig?: ProcessingOptions;
|
|
601
|
-
}
|
|
602
|
-
|
|
603
|
-
type RetryOptions = {
|
|
604
|
-
maxAttempts?: number; // Default: 3
|
|
605
|
-
initialDelayMs?: number; // Default: 1000ms
|
|
606
|
-
maxDelayMs?: number; // Default: 10000ms
|
|
607
|
-
}
|
|
608
|
-
|
|
609
|
-
type ProcessingOptions = {
|
|
610
|
-
bufferSize?: number; // Max events to buffer before processing (default: 50)
|
|
611
|
-
bufferTimeoutMs?: number; // Max wait time for buffer to fill (default: 100ms)
|
|
612
|
-
concurrency?: number; // Max concurrent batch requests (default: 5)
|
|
613
|
-
maxBatchSize?: number; // Max items per downstream batch (e.g. SQS limit)
|
|
614
|
-
}
|
|
615
|
-
```
|
|
616
|
-
|
|
617
|
-
## Error Handling
|
|
618
|
-
|
|
619
|
-
The library provides a structured error hierarchy to help you distinguish between configuration issues, validation errors, and operational failures. All custom errors inherit from `OutboxError`.
|
|
620
|
-
|
|
621
|
-
### Error Hierarchy
|
|
622
|
-
|
|
623
|
-
| Error Class | Category | Description |
|
|
624
|
-
| :--- | :--- | :--- |
|
|
625
|
-
| `OutboxError` | - | **Base class** for all errors in the library. Includes a `.context` property. |
|
|
626
|
-
| `ConfigurationError`| Category | Base class for setup-related issues. |
|
|
627
|
-
| `DuplicateListenerError` | Configuration | Thrown by `.on()` if a listener is already registered for an event type. |
|
|
628
|
-
| `UnsupportedOperationError` | Configuration | Thrown if an outbox doesn't support management APIs like `getFailedEvents`. |
|
|
629
|
-
| `ValidationError` | Category | Base class for validation-related issues. |
|
|
630
|
-
| `BatchSizeLimitError` | Validation | Thrown by `.emit()` if the number of events exceeds adapter limits. |
|
|
631
|
-
| `OperationalError` | Category | Base class for runtime/processing failures. |
|
|
632
|
-
| `TimeoutError` | Operational | Thrown by `.waitFor()` if the event does not occur within the timeout period. |
|
|
633
|
-
| `BackpressureError` | Operational | Thrown by publishers if the underlying storage/channel is full. |
|
|
634
|
-
| `MaintenanceError` | Operational | Reported to `onError` if a background maintenance task fails. |
|
|
635
|
-
| `MaxRetriesExceededError` | Operational | Reported to `onError` when an event exhausts its configured retry attempts. |
|
|
636
|
-
| `HandlerError` | Operational | Reported to `onError` when an event handler throws an error (before max retries). |
|
|
637
|
-
|
|
638
|
-
### The `onError` Callback
|
|
639
|
-
|
|
640
|
-
The `onError` callback is the primary place to handle background processing failures. It receives the error instance with event information bundled in `error.context.event` for event-related errors.
|
|
641
|
-
|
|
642
|
-
#### Example: Specific Error Handling
|
|
643
|
-
|
|
644
|
-
```typescript
|
|
645
|
-
import { OutboxEventBus, MaxRetriesExceededError, MaintenanceError } from 'outbox-event-bus';
|
|
646
|
-
|
|
647
|
-
const bus = new OutboxEventBus(outbox, (error: OutboxError) => {
|
|
648
|
-
// error is always an OutboxError instance
|
|
649
|
-
if (error instanceof MaxRetriesExceededError) {
|
|
650
|
-
// Access strongly typed event and cause
|
|
651
|
-
console.error(`Event ${error.event.id} permanently failed after ${error.retryCount} attempts`);
|
|
652
|
-
console.error('Original cause:', error.cause);
|
|
653
|
-
} else if (error instanceof MaintenanceError) {
|
|
654
|
-
// Background tasks like stuck event recovery failed.
|
|
655
|
-
console.error(`Maintenance failed: ${error.message}`);
|
|
656
|
-
} else {
|
|
657
|
-
// Generic operational or handler error.
|
|
658
|
-
console.error('Background processing error:', error);
|
|
659
|
-
}
|
|
660
|
-
});
|
|
661
|
-
```
|
|
662
|
-
|
|
663
|
-
### Contextual Data
|
|
664
|
-
|
|
665
|
-
Every `OutboxError` contains a `context` property with additional debugging information:
|
|
666
|
-
|
|
667
|
-
```typescript
|
|
668
|
-
try {
|
|
669
|
-
bus.on('my-event', handler);
|
|
670
|
-
bus.on('my-event', handler); // Throws DuplicateListenerError
|
|
671
|
-
} catch (error) {
|
|
672
|
-
if (error instanceof DuplicateListenerError) {
|
|
673
|
-
console.log(error.context.eventType); // "my-event"
|
|
674
|
-
}
|
|
675
|
-
}
|
|
676
|
-
```
|
|
677
|
-
|
|
678
|
-
## Advanced API
|
|
679
|
-
|
|
680
|
-
> [!TIP]
|
|
681
|
-
> These APIs are for advanced use cases. Most users won't need them.
|
|
682
|
-
|
|
683
|
-
#### `PollingService`
|
|
684
|
-
Low-level polling service used internally by outbox adapters. Useful if you're implementing a custom adapter.
|
|
685
|
-
|
|
686
|
-
```typescript
|
|
687
|
-
class PollingService {
|
|
688
|
-
constructor(config: PollingServiceConfig);
|
|
689
|
-
start(): void;
|
|
690
|
-
stop(): Promise<void>;
|
|
691
|
-
}
|
|
692
|
-
|
|
693
|
-
interface PollingServiceConfig {
|
|
694
|
-
pollIntervalMs: number; // How often to poll
|
|
695
|
-
baseBackoffMs: number; // Base backoff on error
|
|
696
|
-
maxErrorBackoffMs?: number; // Max backoff on error
|
|
697
|
-
processBatch: (handler) => Promise<void>; // Batch processor
|
|
698
|
-
performMaintenance?: () => Promise<void>; // Optional maintenance
|
|
699
|
-
}
|
|
700
|
-
```
|
|
701
|
-
|
|
702
|
-
#### `EventPublisher`
|
|
703
|
-
Generic event publisher with batching and retry logic. Used internally by all publisher implementations.
|
|
704
|
-
|
|
705
|
-
```typescript
|
|
706
|
-
class EventPublisher<TTransaction = unknown> {
|
|
707
|
-
constructor(
|
|
708
|
-
bus: IOutboxEventBus<TTransaction>,
|
|
709
|
-
config?: PublisherConfig
|
|
710
|
-
);
|
|
711
|
-
|
|
712
|
-
subscribe(
|
|
713
|
-
eventTypes: string[],
|
|
714
|
-
handler: (events: BusEvent[]) => Promise<void>
|
|
715
|
-
): void;
|
|
716
|
-
}
|
|
717
|
-
```
|
|
@@ -1,43 +0,0 @@
|
|
|
1
|
-
# Contributing
|
|
2
|
-
|
|
3
|
-
This repository is a monorepo managed with [pnpm](https://pnpm.io/) workspaces.
|
|
4
|
-
|
|
5
|
-
## Setup
|
|
6
|
-
|
|
7
|
-
1. **Install Node.js**: Ensure you have Node.js (version 18 or higher) installed.
|
|
8
|
-
```bash
|
|
9
|
-
npm install -g pnpm
|
|
10
|
-
```
|
|
11
|
-
3. **Install dependencies**:
|
|
12
|
-
```bash
|
|
13
|
-
pnpm install
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
## Development Workflow
|
|
17
|
-
|
|
18
|
-
- **Build**: `pnpm build` - Builds all packages.
|
|
19
|
-
- **Test**: `pnpm test` - Runs unit tests.
|
|
20
|
-
- **E2E Test**: `pnpm test:e2e` - Runs end-to-end tests (requires local DBs).
|
|
21
|
-
- **Lint/Format**: `pnpm format` - Runs Biome to check and fix code style.
|
|
22
|
-
|
|
23
|
-
## Project Structure
|
|
24
|
-
|
|
25
|
-
- `core`: The main event bus logic.
|
|
26
|
-
- `adapters/*`: Database persistence adapters (Postgres, Mongo, DynamoDB, etc.).
|
|
27
|
-
- `publishers/*`: Event transport publishers (SQS, SNS, Kafka, etc.).
|
|
28
|
-
|
|
29
|
-
## Pull Request Process
|
|
30
|
-
|
|
31
|
-
1. **Branch**: Create a feature branch from `main`.
|
|
32
|
-
2. **Code**: Implement your changes. Ensure tests pass and code is formatted.
|
|
33
|
-
3. **Changeset**: If your changes affect published packages, you **must** create a changeset:
|
|
34
|
-
```bash
|
|
35
|
-
pnpm changeset
|
|
36
|
-
```
|
|
37
|
-
Follow the prompts to select packages and describe the change (major/minor/patch).
|
|
38
|
-
4. **Commit**: Commit your changes and the generated `.changeset` file.
|
|
39
|
-
5. **Submit**: Open a Pull Request against `main`.
|
|
40
|
-
|
|
41
|
-
## Reporting Issues
|
|
42
|
-
|
|
43
|
-
Use the [GitHub Issues](https://github.com/dunika/outbox-event-bus/issues) tracker to report bugs or suggest features.
|
package/docs/docs/PUBLISHING.md
DELETED
|
@@ -1,144 +0,0 @@
|
|
|
1
|
-
# Publishing Guide
|
|
2
|
-
|
|
3
|
-
## Quick Start
|
|
4
|
-
|
|
5
|
-
```bash
|
|
6
|
-
# Local
|
|
7
|
-
pnpm publish:interactive --dry-run # test first
|
|
8
|
-
pnpm publish:interactive # publish (handles auth automatically)
|
|
9
|
-
|
|
10
|
-
# CI/CD
|
|
11
|
-
# Automated via "Version Packages" PR and GitHub Actions
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
## Prerequisites
|
|
15
|
-
|
|
16
|
-
- Access to `@outbox-event-bus` npm organization
|
|
17
|
-
- `NPM_TOKEN` configured (for CI) or logged in locally
|
|
18
|
-
|
|
19
|
-
> The publish script automatically runs build, lint, and test across the workspace.
|
|
20
|
-
|
|
21
|
-
## Local Publishing
|
|
22
|
-
|
|
23
|
-
**Step 1: Test with Dry-run**
|
|
24
|
-
|
|
25
|
-
```bash
|
|
26
|
-
pnpm publish:interactive --dry-run
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
This will:
|
|
30
|
-
- ✓ Check authentication
|
|
31
|
-
- ✓ Show what would be built, linted, tested, and published
|
|
32
|
-
- ✗ Not actually publish to npm
|
|
33
|
-
|
|
34
|
-
**Step 2: Publish**
|
|
35
|
-
|
|
36
|
-
```bash
|
|
37
|
-
pnpm publish:interactive
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
The script will:
|
|
41
|
-
1. **Check authentication** - If not logged in, it will prompt you to choose:
|
|
42
|
-
- **Interactive Login**: Runs `npm login`
|
|
43
|
-
- **Environment Variable**: Checks `NPM_TOKEN`
|
|
44
|
-
2. **Build** all packages (`pnpm -r build`)
|
|
45
|
-
3. **Lint** all packages (`pnpm -r lint`)
|
|
46
|
-
4. **Test** all packages (`pnpm -r test`)
|
|
47
|
-
5. **E2E Test** all packages (`pnpm -r test:e2e`)
|
|
48
|
-
6. **Publish** using Changesets (`pnpm release`)
|
|
49
|
-
|
|
50
|
-
**Two-Factor Authentication (2FA)**
|
|
51
|
-
|
|
52
|
-
Since this project uses Changesets for publishing, the underlying `npm publish` command handles 2FA. You will be prompted for your OTP code if required during the `pnpm release` step.
|
|
53
|
-
|
|
54
|
-
> [!NOTE]
|
|
55
|
-
> If you don't have 2FA enabled yet, run `npm profile enable-2fa auth-and-writes` to set it up.
|
|
56
|
-
|
|
57
|
-
## CI/CD Setup
|
|
58
|
-
|
|
59
|
-
The project is configured to use **npm Trusted Publishing** for secure, passwordless publishing via GitHub Actions.
|
|
60
|
-
|
|
61
|
-
> **Important**: You must publish the packages **manually** at least once before you can configure Trusted Publishing. Use the [Local Publishing](#local-publishing) steps below for the initial release.
|
|
62
|
-
|
|
63
|
-
1. **Configure Trusted Publishing on npm**:
|
|
64
|
-
- Log in to [npmjs.com](https://www.npmjs.com/)
|
|
65
|
-
- Navigate to your package settings (e.g., `https://www.npmjs.com/package/@outbox-event-bus/core/access`)
|
|
66
|
-
- Go to **Publishing Access**
|
|
67
|
-
- Click **Connect GitHub**
|
|
68
|
-
- Select this repository: `dunika/outbox-event-bus`
|
|
69
|
-
- Workflow filename: `release.yml`
|
|
70
|
-
- Environment: Leave empty (unless you configured one in GitHub)
|
|
71
|
-
- *Repeat this for each package in the monorepo.*
|
|
72
|
-
|
|
73
|
-
2. **Check Permissions**:
|
|
74
|
-
- Go to **Settings** > **Actions** > **General**
|
|
75
|
-
- Ensure **Workflow permissions** is set to **Read and write permissions**
|
|
76
|
-
- This is required for the Changesets action to push version commits and tags back to the repo.
|
|
77
|
-
|
|
78
|
-
> **Note**: The `release.yml` workflow is already configured with `permissions: id-token: write` to support Trusted Publishing.
|
|
79
|
-
|
|
80
|
-
## CI/CD Publishing
|
|
81
|
-
|
|
82
|
-
Publishing is automated via GitHub Actions and Changesets.
|
|
83
|
-
|
|
84
|
-
**Step 1: Create Changeset**
|
|
85
|
-
|
|
86
|
-
When making changes, run:
|
|
87
|
-
```bash
|
|
88
|
-
pnpm changeset
|
|
89
|
-
```
|
|
90
|
-
Follow the prompts to select packages and bump types (patch/minor/major).
|
|
91
|
-
|
|
92
|
-
**Step 2: Push Changes**
|
|
93
|
-
|
|
94
|
-
Commit the `package.json` updates and the new changeset file.
|
|
95
|
-
|
|
96
|
-
**Step 3: Version Packages (Automated)**
|
|
97
|
-
|
|
98
|
-
When changes are merged to `main`, the "Version Packages" PR will be automatically created/updated by the Changesets bot.
|
|
99
|
-
- This PR consumes changesets and updates versions/changelogs.
|
|
100
|
-
|
|
101
|
-
**Step 4: Release (Automated)**
|
|
102
|
-
|
|
103
|
-
Merging the "Version Packages" PR triggers the release workflow which runs `pnpm release` to publish the updated packages to npm.
|
|
104
|
-
|
|
105
|
-
## Troubleshooting
|
|
106
|
-
|
|
107
|
-
### Authentication Failed
|
|
108
|
-
|
|
109
|
-
**Local:**
|
|
110
|
-
- Check: `npm whoami`
|
|
111
|
-
- Login: `npm login`
|
|
112
|
-
|
|
113
|
-
**CI:**
|
|
114
|
-
- Set `NPM_TOKEN` secret in: Repository → Settings → Secrets and variables → Actions
|
|
115
|
-
|
|
116
|
-
### Build/Lint/Test Failures
|
|
117
|
-
|
|
118
|
-
Run the workspace commands manually to debug:
|
|
119
|
-
```bash
|
|
120
|
-
pnpm -r build
|
|
121
|
-
pnpm -r lint
|
|
122
|
-
pnpm -r test
|
|
123
|
-
```
|
|
124
|
-
|
|
125
|
-
### Version Already Published
|
|
126
|
-
|
|
127
|
-
- Check currently published versions on npm under the `@outbox-event-bus` scope.
|
|
128
|
-
- Ensure you have a new changeset defined if you intend to publish a new version.
|
|
129
|
-
|
|
130
|
-
## Scripts Reference
|
|
131
|
-
|
|
132
|
-
| Script | Description |
|
|
133
|
-
|--------|-------------|
|
|
134
|
-
| `pnpm publish:interactive` | Full publish pipeline wrapper (auth → build → lint → test → release) |
|
|
135
|
-
| `pnpm release` | Changesets publish command (build + changeset publish) |
|
|
136
|
-
| `pnpm changeset` | Create a new changeset |
|
|
137
|
-
| `pnpm version-packages` | Consume changesets and update versions |
|
|
138
|
-
|
|
139
|
-
## Checklist
|
|
140
|
-
|
|
141
|
-
- [ ] Changeset created (`pnpm changeset`)
|
|
142
|
-
- [ ] Changes merged to `main`
|
|
143
|
-
- [ ] "Version Packages" PR reviewed and merged
|
|
144
|
-
- [ ] Release workflow succeeded
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|