@dotpress/events 0.0.0-stage → 2.0.0-alpha.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,708 @@
1
- # Temporary Holding Version
1
+ # @dotpress/events
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Typed events for [Dotpress](https://dotpress.dev): emit a fact, and let listeners react to it in workers, through interchangeable transports. The code that emits never knows where or how the listeners run. Works without a Dotpress app: in a script, a worker or a Lambda function.
4
+
5
+ > 🚧 Dotpress V2 is in alpha (`2.0.0-alpha`): the API may still change before 2.0.0.
6
+
7
+ ```sh
8
+ pnpm add @dotpress/events zod
9
+ pnpm add @dotpress/core # for eventsPlugin
10
+ ```
11
+
12
+ ## Quick start
13
+
14
+ ```ts
15
+ import {
16
+ createEventBus,
17
+ createMemoryTransport,
18
+ defineEvent,
19
+ defineListener,
20
+ } from '@dotpress/events'
21
+ import { z } from 'zod'
22
+
23
+ // An event: a fact, with the schema of its payload.
24
+ export const userCreated = defineEvent({
25
+ name: 'user.created',
26
+ schema: z.object({ userId: z.string() }),
27
+ })
28
+
29
+ // Listeners react to it, independently.
30
+ export const sendWelcomeMail = defineListener(userCreated, {
31
+ name: 'send-welcome-mail',
32
+ handle: async ({ payload, logger }) => {
33
+ const user = await findUser(payload.userId)
34
+ await mailer.send(welcomeMail, { to: user.email, data: { … } })
35
+ },
36
+ })
37
+
38
+ export const geocodeUser = defineListener(userCreated, {
39
+ name: 'geocode-user',
40
+ load: () => import('./listeners/geocode-user.js'), // loaded by workers only
41
+ })
42
+
43
+ const events = createEventBus({
44
+ transport: createMemoryTransport(), // a real queue in production
45
+ listeners: [sendWelcomeMail, geocodeUser],
46
+ })
47
+
48
+ await events.emit(userCreated, { userId: '42' })
49
+ ```
50
+
51
+ ## Concepts
52
+
53
+ | | Role | Where |
54
+ | --- | --- | --- |
55
+ | **Event** (`defineEvent`) | a fact, with a name and a payload schema | shared code |
56
+ | **Listener** (`defineListener`) | a reaction to an event, with a unique name | shared code |
57
+ | **Bus** (`createEventBus`) | validates emitted events, sends one message per listener to its transport | the code that emits (API, scripts) |
58
+ | **Transport** | sends messages to a queue: `sync`, `memory`, SQS, Redis… | bus configuration |
59
+ | **Worker** (`createWorker`) | runs the listeners of the messages it receives | Lambda, Cloudflare Worker, container, `dotpress-events work` |
60
+
61
+ `emit()` sends **one message per listener**: each listener is retried on its own, and can run on another platform than the others.
62
+
63
+ ## Events
64
+
65
+ ```ts
66
+ defineEvent({
67
+ name: 'invoice.paid', // letters, digits, ".", "_", ":" and "-"
68
+ schema: z.object({ invoiceId: z.string(), paidAt: z.iso.datetime() }),
69
+ transport: 'edge', // optional: see Several transports
70
+ })
71
+ ```
72
+
73
+ **Payloads travel as JSON.** The bus serializes the payload, then validates the result with the schema, so an invalid payload throws an `EventPayloadError` at `emit()`, not later in a worker. Describe JSON values: dates as ISO strings (convert them in the schema with `.transform()` or `z.coerce.date()` if listeners need `Date` objects). Workers validate the payload again: a message emitted by a previous version of your code is checked against the current schema.
74
+
75
+ **Pass identifiers, not secrets.** Messages are stored by queues, shown in their consoles and kept in dead-letter queues: send `{ userId }`, never a password or a token. Listeners load what they need.
76
+
77
+ ## Listeners
78
+
79
+ ```ts
80
+ defineListener(invoicePaid, {
81
+ name: 'send-receipt', // unique: it identifies the listener in messages
82
+ handle: async (context) => { … },
83
+ // or: load: () => import('./listeners/send-receipt.js'), with a default export
84
+ transport: 'edge', // optional: another transport than its event
85
+ })
86
+ ```
87
+
88
+ The handler receives:
89
+
90
+ | Field | Content |
91
+ | --- | --- |
92
+ | `payload` | the payload, parsed by the schema (transforms applied) |
93
+ | `event`, `listener` | names |
94
+ | `id` | identifier of the message: the same for every attempt |
95
+ | `emissionId` | identifier of the emission, shared by the messages of its listeners |
96
+ | `attempt` | attempt number, from 1 |
97
+ | `emittedAt` | date of the emission |
98
+ | `metadata` | metadata of the emission (`requestId` with `eventsPlugin`) |
99
+ | `logger` | logger of the worker |
100
+
101
+ **Throw to have the message retried.** The transport retries it, then moves it to its dead-letter queue (see Failures).
102
+
103
+ **Make listeners idempotent.** Queues deliver messages at least once: a message can run twice (a retry after a timeout, a duplicate delivery). Use `id` to ignore duplicates (a unique constraint, a "sent" flag), or design the listener so that running it twice is harmless.
104
+
105
+ **Renaming a listener** orphans the messages still in queues under the old name: deploy workers that handle both names first, or drain the queues.
106
+
107
+ `load` keeps the code of the listeners (database clients, SDKs…) out of the code that emits: the bus only needs the names and routes of the listeners.
108
+
109
+ ## Emitting
110
+
111
+ ```ts
112
+ const { emissionId, messages } = await events.emit(
113
+ invoicePaid,
114
+ { invoiceId: 'F-1', paidAt: new Date().toISOString() },
115
+ {
116
+ delay: 3600, // seconds, within the limit of the transport
117
+ metadata: { source: 'stripe' }, // passed to the listeners
118
+ }
119
+ )
120
+ ```
121
+
122
+ `messages` lists one `{ id, listener, transport }` per listener. An event without listeners is not sent.
123
+
124
+ **Emit after the transaction commits.** An event describes something that happened: emitted inside a database transaction that rolls back, it describes something that did not. Commit first, then emit.
125
+
126
+ ```ts
127
+ const user = await db.transaction(async (tx) => createUser(tx, input))
128
+ await events.emit(userCreated, { userId: user.id })
129
+ ```
130
+
131
+ When some messages cannot be sent (a queue is down), `emit()` throws an `EventEmitError` listing the `sent` and `failed` messages. Emitting again is safe: message ids depend on the emission and the listener, so listeners that use `id` ignore the duplicates.
132
+
133
+ ## Several transports
134
+
135
+ Name the transports in the bus, and route events (or single listeners) to them. Without `transport`, an event uses `default`.
136
+
137
+ ```ts
138
+ declare module '@dotpress/events' {
139
+ interface EventTransports {
140
+ edge: true // "transport: 'edge'" is now type-checked
141
+ }
142
+ }
143
+
144
+ export const recordingUploaded = defineEvent({
145
+ name: 'recording.uploaded',
146
+ schema: z.object({ recordingId: z.string() }),
147
+ transport: 'edge',
148
+ })
149
+
150
+ const events = createEventBus({
151
+ transports: {
152
+ default: createSqsTransport({ queueUrl }), // handled by a Lambda worker
153
+ edge: createCloudflareQueueTransport({ … }), // handled by a Cloudflare worker
154
+ },
155
+ listeners,
156
+ })
157
+ ```
158
+
159
+ The code that emits does not change: `events.emit(recordingUploaded, …)` reaches the right queue. The bus checks at startup that every listener has its transport.
160
+
161
+ ## Transports
162
+
163
+ | Transport | Import | Use |
164
+ | --- | --- | --- |
165
+ | `createSyncTransport()` | `@dotpress/events` | runs the listeners inside `emit()`, which waits for them and throws an `EventEmitError` if one fails; delays are ignored. Development, tests, apps without a queue |
166
+ | `createMemoryTransport()` | `@dotpress/events` | in-process queue with retries (3 attempts, 100 ms then doubled; `maxAttempts`, `retryDelay`), `sent`, `drain()`, `clear()`. Development and tests; messages are lost when the process stops |
167
+ | `createSqsTransport()` | `@dotpress/events/aws-sqs` | Amazon SQS (standard and FIFO queues), with a Lambda handler and a long-polling source |
168
+ | `createBullmqTransport()` | `@dotpress/events/bullmq` | Redis, through BullMQ: retries with backoff, delays, deduplication |
169
+ | `createCloudflareQueueTransport()` | `@dotpress/events/cloudflare-queues` | Cloudflare Queues, from a Worker (binding) or from anywhere (REST API), with a `queue()` handler and an HTTP pull source |
170
+ | `createPgBossTransport()` | `@dotpress/events/pg-boss` | Postgres, through pg-boss: no other infrastructure than your database |
171
+ | `createPubSubTransport()` | `@dotpress/events/google-pubsub` | Google Cloud Pub/Sub, over the REST API (any runtime), with a push endpoint and a pull source |
172
+ | `createServiceBusTransport()` | `@dotpress/events/azure-service-bus` | Azure Service Bus queues, with a long-running source and an Azure Functions handler |
173
+
174
+ ```ts
175
+ // Tests: emit, wait, check
176
+ const transport = createMemoryTransport({ retryDelay: () => 0 })
177
+ const events = createEventBus({ transport, listeners })
178
+ await signUp(events, 'ada@example.com')
179
+ await transport.drain()
180
+ expect(transport.sent).toMatchObject([{ event: 'user.created', listener: 'send-welcome-mail' }])
181
+ ```
182
+
183
+ To test the code that emits without running listeners, use `createFakeEventBus()`: it checks each emission like a real bus (event, payload, delay, metadata) and records it in `emitted`. `expectEventEmitted()` of `@dotpress/testing` reads both:
184
+
185
+ ```ts
186
+ const events = createFakeEventBus()
187
+ await signUp(events, 'ada@example.com')
188
+ expectEventEmitted(events, userCreated, { email: 'ada@example.com' })
189
+ ```
190
+
191
+ ## Amazon SQS
192
+
193
+ ```sh
194
+ pnpm add @aws-sdk/client-sqs
195
+ ```
196
+
197
+ ### Sending
198
+
199
+ ```ts
200
+ import { createSqsTransport } from '@dotpress/events/aws-sqs'
201
+
202
+ const events = createEventBus({
203
+ transport: createSqsTransport({ queueUrl: process.env.EVENTS_QUEUE_URL! }),
204
+ listeners,
205
+ })
206
+ ```
207
+
208
+ Messages are sent in batches of 10 (`SendMessageBatch`), with `event` and `listener` message attributes. The client is created from the environment (region, credentials), or from `clientConfig`; pass `client` to reuse yours.
209
+
210
+ | Limit | Value |
211
+ | --- | --- |
212
+ | Message size, attributes included | 1 MiB: larger messages fail (`EventEmitError`), the others are sent |
213
+ | `delay` | 900 seconds (15 minutes) |
214
+
215
+ **FIFO queues** (URL ending with `.fifo`): messages are ordered within a message group, the listener by default (`messageGroupId: (envelope) => …` to group by entity), and deduplicated by their id for 5 minutes. They do not accept per-message delays.
216
+
217
+ ### Lambda workers
218
+
219
+ ```ts
220
+ import { createSqsHandler } from '@dotpress/events/aws-sqs'
221
+
222
+ export const handler = createSqsHandler(
223
+ createWorker({ listeners: [sendWelcomeMail, geocodeUser], logger }),
224
+ {
225
+ deadLetterQueueUrl: process.env.EVENTS_DLQ_URL, // optional, see below
226
+ retryDelay: (attempt) => 30 * 2 ** (attempt - 1), // optional backoff, in seconds
227
+ }
228
+ )
229
+ ```
230
+
231
+ **Enable `ReportBatchItemFailures`** on the event source mapping: the handler returns the failed messages (`batchItemFailures`), so only they are retried. Without it, a failure retries the whole batch. Messages of a batch run in parallel; with a FIFO queue, in order, stopping at the first failure.
232
+
233
+ ### Long-running workers
234
+
235
+ ```ts
236
+ import { createSqsSource } from '@dotpress/events/aws-sqs'
237
+
238
+ // worker.ts, for startWorker() or `dotpress-events work ./worker.ts`
239
+ export default {
240
+ worker: createWorker({ listeners }),
241
+ source: createSqsSource({
242
+ queueUrl: process.env.EVENTS_QUEUE_URL!,
243
+ visibilityTimeout: 60, // extended while a listener runs
244
+ }),
245
+ }
246
+ ```
247
+
248
+ The source reads the queue by long polling (`waitTimeSeconds`, 20 by default; `batchSize`, 10 by default), runs the messages of a batch in parallel, and deletes the handled ones. With `visibilityTimeout`, it keeps a message invisible while its listener runs (every half timeout), so that a long listener is not delivered twice. Receive errors are logged and retried, after 1 second then up to 30 seconds. At startup, it warns when the queue has no dead-letter queue.
249
+
250
+ ### Retries and dead-letter queue
251
+
252
+ Configure the queue in your infrastructure:
253
+
254
+ - a **dead-letter queue**, with a redrive policy: `maxReceiveCount` is the number of attempts before a message moves there;
255
+ - a **visibility timeout** of at least 6 times the timeout of the Lambda function (AWS recommendation), or longer than your slowest listener for a long-running worker;
256
+ - for Lambda, `ReportBatchItemFailures` on the event source mapping.
257
+
258
+ A failed message becomes visible again after the visibility timeout; `retryDelay(attempt)` sets another delay (`ChangeMessageVisibility`, up to 12 hours). Set `maxAttempts` on the worker to the `maxReceiveCount` of the queue: `onFailure` then reports the last attempt as `final`.
259
+
260
+ **Messages that can never be handled** (invalid payload, unknown listener): with `deadLetterQueueUrl`, they are copied to the dead-letter queue at once, with `reason` and `error` attributes, and deleted from the queue. Without it, they fail until `maxReceiveCount`, like other failures.
261
+
262
+ ```ts
263
+ // AWS CDK
264
+ const dlq = new sqs.Queue(this, 'EventsDlq', { retentionPeriod: Duration.days(14) })
265
+ const queue = new sqs.Queue(this, 'Events', {
266
+ visibilityTimeout: Duration.seconds(180), // 6 × the timeout of the function
267
+ deadLetterQueue: { queue: dlq, maxReceiveCount: 5 },
268
+ })
269
+ worker.addEventSource(
270
+ new SqsEventSource(queue, { batchSize: 10, reportBatchItemFailures: true })
271
+ )
272
+ ```
273
+
274
+ ```hcl
275
+ # Terraform
276
+ resource "aws_sqs_queue" "events_dlq" {
277
+ name = "events-dlq"
278
+ message_retention_seconds = 1209600
279
+ }
280
+
281
+ resource "aws_sqs_queue" "events" {
282
+ name = "events"
283
+ visibility_timeout_seconds = 180
284
+ redrive_policy = jsonencode({
285
+ deadLetterTargetArn = aws_sqs_queue.events_dlq.arn
286
+ maxReceiveCount = 5
287
+ })
288
+ }
289
+
290
+ resource "aws_lambda_event_source_mapping" "events" {
291
+ event_source_arn = aws_sqs_queue.events.arn
292
+ function_name = aws_lambda_function.worker.arn
293
+ batch_size = 10
294
+ function_response_types = ["ReportBatchItemFailures"]
295
+ }
296
+ ```
297
+
298
+ **IAM permissions**: `sqs:SendMessage` for the code that emits; for workers, `sqs:ReceiveMessage`, `sqs:DeleteMessage` and `sqs:GetQueueAttributes` (granted by the Lambda event source mapping, or yours for a long-running worker), `sqs:ChangeMessageVisibility` with `retryDelay` or `visibilityTimeout`, and `sqs:SendMessage` on the dead-letter queue with `deadLetterQueueUrl`.
299
+
300
+ ## Redis (BullMQ)
301
+
302
+ ```sh
303
+ pnpm add bullmq ioredis
304
+ ```
305
+
306
+ ```ts
307
+ import { createBullmqSource, createBullmqTransport } from '@dotpress/events/bullmq'
308
+
309
+ const connection = { host: 'localhost', port: 6379 }
310
+
311
+ // The code that emits
312
+ const transport = createBullmqTransport({ name: 'events', connection })
313
+ const events = createEventBus({ transport, listeners })
314
+ // On shutdown: await transport.close()
315
+
316
+ // worker.ts, for startWorker() or `dotpress-events work ./worker.ts`
317
+ export default {
318
+ worker: createWorker({ listeners, maxAttempts: 3 }),
319
+ source: createBullmqSource({ name: 'events', connection, concurrency: 5 }),
320
+ }
321
+ ```
322
+
323
+ Each message becomes a [BullMQ](https://docs.bullmq.io) job named after its listener, with the message id as job id: emitting again adds no duplicate while the job exists. `delay` has no upper limit.
324
+
325
+ **Job options** (`jobOptions`, merged with the defaults): 3 `attempts`, `exponential` backoff from 1 second, completed jobs removed (`removeOnComplete: true`), failed jobs kept (`removeOnFail: false`). Set `maxAttempts` of the worker to the same number of attempts, so that `onFailure` reports the last one as `final`.
326
+
327
+ **Failed jobs are the dead-letter queue**: after their last attempt, jobs stay in the failed set of the queue, with their error, where you can inspect and retry them (BullMQ API, or a dashboard such as Bull Board). Messages that can never be handled (invalid payload, unknown listener) fail at once, without retries (`UnrecoverableError`).
328
+
329
+ **Source**: `concurrency` jobs at a time (1 by default). On stop, the source waits for the jobs in progress, up to `closeTimeout` (30 seconds by default); their jobs then become stalled and another worker retries them. While Redis is unreachable, BullMQ cannot close: the stop gives up waiting after `closeTimeout`, and BullMQ may keep reconnecting until the process exits.
330
+
331
+ **Redis requirements** (BullMQ): Redis 6.2 or later, with `maxmemory-policy=noeviction`. Use `prefix` for the key prefix, not the `keyPrefix` of ioredis. An ioredis instance passed as `connection` to a source needs `maxRetriesPerRequest: null`. Pass `queue` instead of `name` and `connection` to reuse a `Queue` (shared connection, another BullMQ backend).
332
+
333
+ BullMQ runs on Node.js and Bun.
334
+
335
+ ## Cloudflare Queues
336
+
337
+ No dependency: the transport uses the queue binding inside Workers, and `fetch` elsewhere.
338
+
339
+ ```jsonc
340
+ // wrangler.jsonc
341
+ {
342
+ "queues": {
343
+ "producers": [{ "queue": "events", "binding": "EVENTS_QUEUE" }],
344
+ "consumers": [
345
+ {
346
+ "queue": "events",
347
+ "max_batch_size": 10,
348
+ "max_retries": 5,
349
+ "dead_letter_queue": "events-dlq"
350
+ }
351
+ ]
352
+ }
353
+ }
354
+ ```
355
+
356
+ ### Sending
357
+
358
+ ```ts
359
+ import { createCloudflareQueueTransport } from '@dotpress/events/cloudflare-queues'
360
+
361
+ // Inside a Worker: the producer binding
362
+ import { env } from 'cloudflare:workers'
363
+ const transport = createCloudflareQueueTransport({ queue: () => env.EVENTS_QUEUE })
364
+
365
+ // Anywhere else (Node.js, Lambda…): the REST API
366
+ const transport = createCloudflareQueueTransport({
367
+ accountId: process.env.CF_ACCOUNT_ID!,
368
+ queueId: process.env.CF_QUEUE_ID!, // the queue ID, not its name
369
+ apiToken: process.env.CF_API_TOKEN!, // Queues Write permission
370
+ })
371
+ ```
372
+
373
+ `queue` also accepts the binding itself. Messages are sent as JSON, in batches of 100 messages and 256 KB at most.
374
+
375
+ | Limit | Value |
376
+ | --- | --- |
377
+ | Message size | 128 KB, including about 100 bytes of metadata: larger messages fail (`EventEmitError`) |
378
+ | `delay` | 86,400 seconds (24 hours) |
379
+
380
+ ### Worker consumers
381
+
382
+ ```ts
383
+ import { createCloudflareQueueHandler } from '@dotpress/events/cloudflare-queues'
384
+
385
+ export default {
386
+ queue: createCloudflareQueueHandler(createWorker({ listeners, maxAttempts: 5 }), {
387
+ retryDelay: (attempt) => 30 * attempt, // optional, in seconds
388
+ deadLetterQueue: () => env.EVENTS_DLQ, // optional producer binding, see below
389
+ }),
390
+ }
391
+ ```
392
+
393
+ Each message is acknowledged (`ack()`) or retried (`retry()`) on its own: a failure does not retry the whole batch. After `max_retries`, Cloudflare moves the message to the `dead_letter_queue` of the consumer (without it, the message is deleted). Set `maxAttempts` of the worker to `max_retries + 1`.
394
+
395
+ **Messages that can never be handled**: with `deadLetterQueue` (a producer binding to the dead-letter queue), they are sent there at once and acknowledged. Without it, they are retried until `max_retries`.
396
+
397
+ ### Pull consumers
398
+
399
+ To run workers outside Cloudflare (a container, your machine with `dotpress-events work`), enable an HTTP pull consumer on the queue (`wrangler queues consumer http add events`; a queue has either a Worker consumer or a pull consumer):
400
+
401
+ ```ts
402
+ import { createCloudflarePullSource } from '@dotpress/events/cloudflare-queues'
403
+
404
+ export default {
405
+ worker: createWorker({ listeners }),
406
+ source: createCloudflarePullSource({
407
+ accountId,
408
+ queueId,
409
+ apiToken, // Queues Read and Write permissions
410
+ visibilityTimeoutMs: 60_000, // lease: longer than your slowest listener
411
+ deadLetterQueueId, // optional: dead messages are pushed there at once
412
+ }),
413
+ }
414
+ ```
415
+
416
+ The source pulls `batchSize` messages (10 by default, up to 100), runs them in parallel, then acknowledges or retries them in one request. The API cannot extend a lease: set `visibilityTimeoutMs` (30 seconds by default, up to 12 hours) above the duration of your slowest listener, or the message is delivered again while it runs. An empty queue is pulled again after `pollInterval` (1 second by default). Errors are logged and retried, after 1 second then up to 30 seconds.
417
+
418
+ ## Postgres (pg-boss)
419
+
420
+ ```sh
421
+ pnpm add pg-boss
422
+ ```
423
+
424
+ ```ts
425
+ import { createPgBossSource, createPgBossTransport } from '@dotpress/events/pg-boss'
426
+
427
+ const boss = process.env.DATABASE_URL! // or pg-boss options, or a started instance
428
+ const queueOptions = { deadLetter: 'events-dlq' } // optional
429
+
430
+ // The code that emits
431
+ const transport = createPgBossTransport({ boss, queue: 'events', queueOptions })
432
+ const events = createEventBus({ transport, listeners })
433
+ // On shutdown: await transport.close()
434
+
435
+ // worker.ts, for startWorker() or `dotpress-events work ./worker.ts`
436
+ export default {
437
+ worker: createWorker({ listeners, maxAttempts: 3 }),
438
+ source: createPgBossSource({ boss, queue: 'events', queueOptions, batchSize: 5 }),
439
+ }
440
+ ```
441
+
442
+ [pg-boss](https://pgboss.io) keeps the queue in your Postgres database (13 or later; pg-boss needs Node.js 22.12 or later, or Bun). With a connection string or options, the transport and the source create and start their own pg-boss instance (`close()` stops the one of the transport); pass a started instance to share it.
443
+
444
+ **Queues** are created on first use when they do not exist, with `queueOptions` merged with the defaults: 3 attempts (`retryLimit: 2`), exponential backoff from 1 second (`retryDelay: 1`, `retryBackoff: true`). Existing queues are not changed. With `deadLetter`, the dead-letter queue is created too: failed jobs go there after their last attempt, and messages that can never be handled at once. Set `maxAttempts` of the worker to `retryLimit + 1`.
445
+
446
+ Each message is a job whose id is the message id: emitting again is ignored by pg-boss, which keeps the first job. `delay` has no upper limit.
447
+
448
+ **Source**: `batchSize` jobs at a time (1 by default), handled in parallel, each with its own result (`perJobResults`): one failure does not retry the others. `pollingIntervalSeconds` sets the wait on an empty queue (0.5 at least). On stop, the source waits for the jobs in progress, up to `closeTimeout` (30 seconds by default). pg-boss errors are logged.
449
+
450
+ Emit after the transaction commits, as with the other transports: an event emitted inside a transaction that rolls back describes something that did not happen.
451
+
452
+ ## Google Cloud Pub/Sub
453
+
454
+ ```sh
455
+ pnpm add google-auth-library # to publish with Application Default Credentials
456
+ pnpm add jose # push endpoints: to verify the tokens
457
+ pnpm add @google-cloud/pubsub # pull workers
458
+ ```
459
+
460
+ Each of these is optional: install only what you use.
461
+
462
+ ### Sending
463
+
464
+ ```ts
465
+ import { createPubSubTransport } from '@dotpress/events/google-pubsub'
466
+
467
+ const transport = createPubSubTransport({ topic: 'projects/my-project/topics/events' })
468
+ ```
469
+
470
+ The transport publishes over the REST API with `fetch`: it runs on Node.js, Bun, Deno and Cloudflare Workers. Each message carries the attributes `event` and `listener`; a subscription can filter on them (`--message-filter='attributes.listener = "send-welcome-mail"'`) to run some listeners on other workers.
471
+
472
+ **Authentication**: by default, an OAuth token from Application Default Credentials, through `google-auth-library` (the service account of Cloud Run, GKE or Cloud Functions, `GOOGLE_APPLICATION_CREDENTIALS`, or `gcloud auth application-default login`). The account needs the Pub/Sub Publisher role (`roles/pubsub.publisher`) on the topic. Elsewhere, pass `getAccessToken`, which returns a token with the `https://www.googleapis.com/auth/pubsub` scope. For the emulator: `baseUrl: 'http://localhost:8085', getAccessToken: false`.
473
+
474
+ | Limit | Value |
475
+ | --- | --- |
476
+ | Request | 10 MB and 1,000 messages: larger emissions are split into several requests |
477
+ | Message size | 10 MB with the base64-encoded envelope (about 7.5 MB of JSON): larger messages fail (`EventEmitError`) |
478
+ | `delay` | not supported by Pub/Sub: an emission with a delay fails (`EventEmitError`, `RangeError`) |
479
+
480
+ ### Subscriptions, retries and dead letters
481
+
482
+ Subscriptions are your infrastructure. Pub/Sub delivers each message at least once: use `context.id` to ignore duplicates.
483
+
484
+ ```sh
485
+ gcloud pubsub topics create events
486
+ gcloud pubsub topics create events-dlq
487
+ gcloud pubsub subscriptions create events-worker --topic=events \
488
+ --ack-deadline=60 \
489
+ --min-retry-delay=10s --max-retry-delay=600s \
490
+ --dead-letter-topic=events-dlq --max-delivery-attempts=5
491
+ ```
492
+
493
+ With a dead-letter policy, Pub/Sub counts the delivery attempts, and moves the message to the dead-letter topic after `--max-delivery-attempts` (5 to 100); its service account needs the Publisher role on the dead-letter topic and the Subscriber role on the subscription. Set `maxAttempts` of the worker to `--max-delivery-attempts`. Without a dead-letter policy, every delivery counts as the first attempt, and a failing message is delivered again until the retention of the subscription expires (7 days by default).
494
+
495
+ **Messages that can never be handled** (not an envelope, unknown listener, invalid payload): with the `deadLetter` option of the push handler or `deadLetterTopic` of the pull source, they are published at once to that topic, with the attributes `reason` and `error`, and acknowledged. Without it, they are delivered again until the dead-letter policy moves them.
496
+
497
+ ### Push endpoints
498
+
499
+ A push subscription POSTs each message to your endpoint, with an OIDC token signed by Google:
500
+
501
+ ```sh
502
+ gcloud pubsub subscriptions create events-push --topic=events \
503
+ --push-endpoint=https://api.example.com/pubsub \
504
+ --push-auth-service-account=pubsub-push@my-project.iam.gserviceaccount.com \
505
+ --ack-deadline=60 \
506
+ --dead-letter-topic=events-dlq --max-delivery-attempts=5
507
+ ```
508
+
509
+ ```ts
510
+ import { createPubSubPushHandler } from '@dotpress/events/google-pubsub'
511
+
512
+ const push = createPubSubPushHandler(createWorker({ listeners, maxAttempts: 5 }), {
513
+ audience: 'https://api.example.com/pubsub', // the push endpoint, or --push-auth-token-audience
514
+ serviceAccountEmail: 'pubsub-push@my-project.iam.gserviceaccount.com',
515
+ deadLetter: { topic: 'projects/my-project/topics/events-dlq' }, // optional, see above
516
+ })
517
+
518
+ // Web-standard runtimes (Cloudflare Workers, Hono, Bun, Deno)
519
+ app.post('/pubsub', (c) => push.fetch(c.req.raw))
520
+ // Other frameworks: the parsed JSON body and the Authorization header
521
+ app.post('/pubsub', async (req, res) => {
522
+ res.status(await push.handle(req.body, req.headers.authorization)).end()
523
+ })
524
+ ```
525
+
526
+ The handler verifies the token with `jose`: signature with the Google keys, issuer, `audience`, a verified `email`, equal to `serviceAccountEmail` when set. It answers 204 when the message is handled (or moved to `deadLetter`), and 503 to have it delivered again; 401 and 403 for invalid tokens, 400 for bodies that are not push messages (Pub/Sub delivers these again too). Set `verifyToken: false` only behind a proxy that checks the token. Pub/Sub waits for the answer up to the ack deadline of the subscription (10 seconds by default, up to 600): set it above your slowest listener.
527
+
528
+ ### Pull workers
529
+
530
+ ```ts
531
+ import { createPubSubPullSource } from '@dotpress/events/google-pubsub'
532
+
533
+ // worker.ts, for startWorker() or `dotpress-events work ./worker.ts`
534
+ export default {
535
+ worker: createWorker({ listeners, maxAttempts: 5 }),
536
+ source: createPubSubPullSource({
537
+ subscription: 'projects/my-project/subscriptions/events-worker',
538
+ maxMessages: 10, // messages handled at the same time (default)
539
+ deadLetterTopic: 'projects/my-project/topics/events-dlq', // optional, see above
540
+ }),
541
+ }
542
+ ```
543
+
544
+ The source uses the streaming pull of `@google-cloud/pubsub` (Node.js), which extends the ack deadline of the messages in progress. Credentials come from Application Default Credentials, `clientConfig` (options of `new PubSub()`), or the `client` you pass; the client uses the emulator when `PUBSUB_EMULATOR_HOST` is set. On stop, the source waits for the messages in progress, up to `closeTimeout` (30 seconds by default). Subscription errors are logged.
545
+
546
+ ## Azure Service Bus
547
+
548
+ ```sh
549
+ pnpm add @azure/service-bus
550
+ ```
551
+
552
+ ```ts
553
+ import {
554
+ createServiceBusSource,
555
+ createServiceBusTransport,
556
+ } from '@dotpress/events/azure-service-bus'
557
+
558
+ const connection = { connectionString: process.env.SERVICE_BUS_CONNECTION! }
559
+ // Or Microsoft Entra ID:
560
+ // const connection = { client: new ServiceBusClient('my-namespace.servicebus.windows.net', new DefaultAzureCredential()) }
561
+
562
+ // The code that emits
563
+ const transport = createServiceBusTransport({ ...connection, queue: 'events' })
564
+ const events = createEventBus({ transport, listeners })
565
+ // On shutdown: await transport.close()
566
+
567
+ // worker.ts, for startWorker() or `dotpress-events work ./worker.ts`
568
+ export default {
569
+ worker: createWorker({ listeners, maxAttempts: 10 }), // the maxDeliveryCount of the queue
570
+ source: createServiceBusSource({ ...connection, queue: 'events', maxConcurrentCalls: 4 }),
571
+ }
572
+ ```
573
+
574
+ The SDK speaks AMQP: Node.js only. With a connection string, the transport and the source create their own client (`close()` closes the one of the transport); pass a `client` to share it, or to use Microsoft Entra ID credentials (`@azure/identity`, with the Azure Service Bus Data Sender or Data Receiver role).
575
+
576
+ Each message carries the envelope as JSON, `subject` (the event) and the application properties `event` and `listener`. Its `messageId` is the message id: with duplicate detection on the queue (`requiresDuplicateDetection`, set when the queue is created; Standard and Premium tiers), emitting again within the detection window is ignored. `delay` schedules the message (`scheduledEnqueueTimeUtc`).
577
+
578
+ | Limit | Value |
579
+ | --- | --- |
580
+ | Message size | 256 KB in the Standard tier (properties included); in the Premium tier, 1 MB unless raised on the queue (up to 100 MB): larger messages fail (`EventEmitError`) |
581
+ | Batch | filled up to the size the queue accepts, 1 MB at most; larger Premium messages are sent alone |
582
+ | `delay` | no limit |
583
+
584
+ **Source**: `maxConcurrentCalls` messages at a time (1 by default), in peek-lock mode. The lock of a message in progress is renewed for `maxAutoLockRenewalDurationInMs` (5 minutes by default; the lock itself lasts 1 minute by default, 5 at most). A handled message is completed; a failed one is abandoned, and Service Bus delivers it again at once, until the `maxDeliveryCount` of the queue (10 by default), then moves it to the dead-letter queue (`MaxDeliveryCountExceeded`). Messages that can never be handled are moved to the dead-letter queue at once, with `deadLetterReason` (`invalid-envelope`, `unknown-listener`, `invalid-payload`) and `deadLetterErrorDescription`. On stop, the source waits for the messages in progress, up to `closeTimeout` (30 seconds by default). Receiver errors are logged.
585
+
586
+ Service Bus has no delay between attempts: a listener that calls a service which is down fails its attempts within seconds. Keep `maxDeliveryCount` high enough, and retry short outages inside the listener if needed.
587
+
588
+ ### Azure Functions
589
+
590
+ ```ts
591
+ import { app } from '@azure/functions'
592
+ import { createServiceBusFunctionHandler } from '@dotpress/events/azure-service-bus'
593
+
594
+ app.serviceBusQueue('events', {
595
+ connection: 'SERVICE_BUS_CONNECTION',
596
+ queueName: 'events',
597
+ handler: createServiceBusFunctionHandler(createWorker({ listeners, maxAttempts: 10 }), {
598
+ // optional, see below
599
+ deadLetter: { connectionString: process.env.SERVICE_BUS_CONNECTION!, queue: 'events-failed' },
600
+ }),
601
+ })
602
+ ```
603
+
604
+ The handler takes one message per invocation (the default `cardinality: 'one'`). A failure throws: the Functions host abandons the message, delivered again until `maxDeliveryCount`, then moved to the dead-letter queue. `attempt` is the `deliveryCount` of the trigger metadata.
605
+
606
+ **Messages that can never be handled**: a function cannot move a message to the dead-letter queue (only the preview SDK bindings can). With `deadLetter`, they are sent at once to that queue, with the application properties `reason` and `error`, and the invocation succeeds. Without it, they fail until `maxDeliveryCount`.
607
+
608
+ ## Workers
609
+
610
+ A worker runs the listeners of the messages it receives. Give it the listeners it is responsible for:
611
+
612
+ ```ts
613
+ const worker = createWorker({
614
+ listeners: [sendWelcomeMail, geocodeUser],
615
+ logger,
616
+ onFailure: (error, { envelope, attempt, final, reason }) => { … },
617
+ maxAttempts: 5, // the retry limit of the queue, to report the last attempt as final
618
+ })
619
+ ```
620
+
621
+ Each transport provides the entry points of its platforms (a Lambda handler, a Cloudflare `queue()` handler…), built on `worker.handle(message, { attempt })`. It returns `success`, `retry` (the listener threw) or `dead` (the message can never be handled), and never throws.
622
+
623
+ ### Long-running workers
624
+
625
+ `startWorker(worker, source)` reads messages from a queue until stopped: in a container, a VM, or locally.
626
+
627
+ ```ts
628
+ const running = startWorker(worker, source, { logger, signal })
629
+ await running.stop() // finishes the messages in progress
630
+ ```
631
+
632
+ ### `dotpress-events work`
633
+
634
+ Like `queue:work` in Laravel: runs a worker next to your app, until Ctrl+C.
635
+
636
+ ```sh
637
+ npx dotpress-events work ./src/events/worker.ts
638
+ npx dotpress-events work ./src/events/worker.ts --watch # restarts when a file changes
639
+ ```
640
+
641
+ The module exports `{ worker, source }` (and optionally a `logger`), or a function returning them; choose another export with `--export`. Logs are written to the standard error as JSON lines. With an SQS transport, it reads your development queue as the Lambda would, without deploying; in production, it is also the entry point of a container.
642
+
643
+ TypeScript modules require Node.js 22.18 or later (erasable syntax, relative imports ending in `.ts`); otherwise, run the compiled module, or the command with `NODE_OPTIONS="--import tsx"`.
644
+
645
+ ## Failures
646
+
647
+ | Failure | What happens |
648
+ | --- | --- |
649
+ | The listener throws | `retry`: the transport delivers the message again, then moves it to its dead-letter queue after its retry limit |
650
+ | Invalid payload, unknown listener, not an envelope | `dead`: retrying would not help. The message goes to the dead-letter queue when the transport can send it there directly; otherwise it fails until the retry limit |
651
+
652
+ `onFailure(error, info)` is called for every failure, with the `envelope` (undefined when the message is not one), the `reason` (`listener-error`, `invalid-envelope`, `unknown-listener`, `invalid-payload`), the `attempt`, and `final` (dead message, or last attempt with `maxAttempts`). Use it for alerts; its own errors are logged and ignored.
653
+
654
+ **Dead-letter queues are part of your infrastructure** (CDK, Terraform, Wrangler…), not created by this package: each transport documents the configuration it expects.
655
+
656
+ An unknown listener usually means a deployment mistake: a listener routed to a queue whose workers do not run it. Check the logs of your workers after adding a listener.
657
+
658
+ ## Dotpress plugin
659
+
660
+ ```ts
661
+ import { eventsPlugin } from '@dotpress/events/plugin'
662
+
663
+ await createApp({ adapter, plugins: [eventsPlugin(events)] })
664
+
665
+ defineRoute({
666
+ method: 'post',
667
+ path: '/users',
668
+ handler: async ({ events, body }) => {
669
+ const user = await createUser(body)
670
+ await events.emit(userCreated, { userId: user.id })
671
+ return user
672
+ },
673
+ })
674
+ ```
675
+
676
+ `ctx.events` logs with the request logger, and adds the request ID to the metadata of the emitted events (`requestId`): listeners can link their logs to the request. Using `ctx.events` without the plugin throws an explicit error. `events.withContext({ logger, metadata })` does the same outside Dotpress.
677
+
678
+ ## Logs
679
+
680
+ With a `logger` (a Dotpress or Pino logger, or any object with `info`, `warn` and `error` methods, and optionally `debug`):
681
+
682
+ | Level | Message |
683
+ | --- | --- |
684
+ | `info` | `Event emitted` (event, emission id, number of messages); `Event worker started` / `stopped` |
685
+ | `debug` | `Event emitted without listeners`; `Event listener succeeded` |
686
+ | `warn` | `Event listener failed, the message is retried` |
687
+ | `error` | `Event listener failed, the message is not retried`; `onFailure hook failed` |
688
+
689
+ Payloads are not logged.
690
+
691
+ ## Errors
692
+
693
+ | Error | When |
694
+ | --- | --- |
695
+ | `TypeError` | invalid options of `defineEvent()`, `defineListener()`, `createEventBus()`, `emit()`… |
696
+ | `EventPayloadError` | the payload does not match the schema, or is not serializable as JSON (`event`, `issues`) |
697
+ | `EventEmitError` | some messages could not be sent (`sent`, `failed`, `cause`) |
698
+ | `PartialSendError` | for transport authors: some messages of a `send()` call failed |
699
+
700
+ The error classes extend `EventsError`.
701
+
702
+ ## Custom transports
703
+
704
+ A transport has a `kind` and a `send(envelopes, { delay })` method; a source has a `kind` and a `consume(worker, { signal, logger })` method. Envelopes are plain JSON objects: `{ v: 1, id, emissionId, event, listener, payload, emittedAt, metadata }`.
705
+
706
+ ## License
707
+
708
+ MIT