@warlock.js/herald 5.19.1 → 5.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -1
- package/llms-full.txt +465 -465
- package/llms.txt +3 -3
- package/package.json +4 -4
- package/skills/consume-message/SKILL.md +219 -219
- package/skills/index.md +20 -0
- package/skills/publish-message/SKILL.md +172 -172
- package/skills/request-and-respond/SKILL.md +74 -74
package/llms.txt
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
## Skills
|
|
8
8
|
|
|
9
|
-
- [consume-message](@warlock.js/herald/consume-message/SKILL.md):
|
|
9
|
+
- [consume-message](@warlock.js/herald/consume-message/SKILL.md): (no description)
|
|
10
10
|
- [herald-basics](@warlock.js/herald/herald-basics/SKILL.md): Start here for @warlock.js/herald event publishing, consumers, request-response messaging, and configuration.
|
|
11
|
-
- [publish-message](@warlock.js/herald/publish-message/SKILL.md):
|
|
12
|
-
- [request-and-respond](@warlock.js/herald/request-and-respond/SKILL.md):
|
|
11
|
+
- [publish-message](@warlock.js/herald/publish-message/SKILL.md): (no description)
|
|
12
|
+
- [request-and-respond](@warlock.js/herald/request-and-respond/SKILL.md): (no description)
|
package/package.json
CHANGED
|
@@ -7,9 +7,9 @@
|
|
|
7
7
|
"dependencies": {
|
|
8
8
|
"@mongez/copper": "^2.1.2",
|
|
9
9
|
"@mongez/events": "^2.2.7",
|
|
10
|
-
"@mongez/reinforcements": "^4.0.
|
|
11
|
-
"@warlock.js/logger": "5.
|
|
12
|
-
"@warlock.js/seal": "5.
|
|
10
|
+
"@mongez/reinforcements": "^4.0.2",
|
|
11
|
+
"@warlock.js/logger": "5.21.0",
|
|
12
|
+
"@warlock.js/seal": "5.21.0"
|
|
13
13
|
},
|
|
14
14
|
"repository": {
|
|
15
15
|
"type": "git",
|
|
@@ -36,7 +36,7 @@
|
|
|
36
36
|
],
|
|
37
37
|
"author": "hassanzohdy",
|
|
38
38
|
"license": "MIT",
|
|
39
|
-
"version": "5.
|
|
39
|
+
"version": "5.21.0",
|
|
40
40
|
"main": "./cjs/index.cjs",
|
|
41
41
|
"module": "./esm/index.mjs",
|
|
42
42
|
"types": "./esm/index.d.mts",
|
|
@@ -1,219 +1,219 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: consume-message
|
|
3
|
-
description: "
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Consume messages
|
|
7
|
-
|
|
8
|
-
Subscribe to a channel; receive each message with a flow-control `ctx`.
|
|
9
|
-
|
|
10
|
-
## Minimal subscribe
|
|
11
|
-
|
|
12
|
-
```ts
|
|
13
|
-
import { herald } from "@warlock.js/herald";
|
|
14
|
-
|
|
15
|
-
herald()
|
|
16
|
-
.channel<{ userId: number; email: string }>("user.created")
|
|
17
|
-
.subscribe(async (message, ctx) => {
|
|
18
|
-
try {
|
|
19
|
-
await sendWelcomeEmail(message.payload.email);
|
|
20
|
-
await ctx.ack(); // success — message removed from queue
|
|
21
|
-
} catch (error) {
|
|
22
|
-
await ctx.nack(true); // failure — requeue for redelivery
|
|
23
|
-
}
|
|
24
|
-
});
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
**Smart auto-ack is on by default** (when `autoAck` is unset or `false`). The consumer runs with manual-ack enabled, and herald acks/nacks based on how your handler ends:
|
|
28
|
-
|
|
29
|
-
- Handler returns without calling any `ctx` method → herald **auto-acks**.
|
|
30
|
-
- Handler **throws** → herald **auto-nacks** (requeue, or DLQ/reject once retries are exhausted — see below).
|
|
31
|
-
- Handler calls `ctx.ack()` / `ctx.nack()` / `ctx.reject()` / `ctx.retry()` → herald respects that and does nothing further.
|
|
32
|
-
|
|
33
|
-
So the `try/catch` above is optional — letting the error throw produces the same auto-nack. Call `ctx` methods explicitly only when you want a *non-default* outcome (reject without requeue, route to DLQ, delayed retry). The one mode that loses messages is `autoAck: true`, where the broker acks on delivery before your handler runs.
|
|
34
|
-
|
|
35
|
-
## Message context — flow control
|
|
36
|
-
|
|
37
|
-
```ts
|
|
38
|
-
ctx.ack(); // acknowledge — message removed from the queue
|
|
39
|
-
ctx.nack(requeue?); // negative ack — requeue (true) or send to DLQ (false)
|
|
40
|
-
ctx.reject(); // shorthand for nack(false)
|
|
41
|
-
ctx.retry(delayMs); // delayed retry — requeue after delay
|
|
42
|
-
ctx.reply(payload); // for request-response pattern (see request-and-respond skill)
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
Pick by intent:
|
|
46
|
-
|
|
47
|
-
| Intent | Use |
|
|
48
|
-
| --- | --- |
|
|
49
|
-
| Processed cleanly | `ctx.ack()` |
|
|
50
|
-
| Transient failure, try again | `ctx.nack(true)` or `ctx.retry(5000)` |
|
|
51
|
-
| Permanent failure, send to DLQ | `ctx.nack(false)` or `ctx.reject()` |
|
|
52
|
-
| Validation failure (bad message) | `ctx.reject()` — don't requeue |
|
|
53
|
-
|
|
54
|
-
## Subscribe options
|
|
55
|
-
|
|
56
|
-
```ts
|
|
57
|
-
await channel.subscribe(handler, {
|
|
58
|
-
group: "email-workers", // consumer group / tag — multiple consumers share work
|
|
59
|
-
prefetch: 10, // concurrency — how many in-flight messages this consumer holds
|
|
60
|
-
autoAck: false, // default; keep it false in production
|
|
61
|
-
exclusive: false, // single consumer only on this channel?
|
|
62
|
-
retry: {
|
|
63
|
-
maxRetries: 3, // redelivery ceiling on handler throw (delay not auto-applied — see below)
|
|
64
|
-
delay: 1000,
|
|
65
|
-
},
|
|
66
|
-
deadLetter: {
|
|
67
|
-
channel: "user.created.failed",
|
|
68
|
-
preserveOriginal: true,
|
|
69
|
-
},
|
|
70
|
-
});
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
`group` is the unit of "share work" — N consumers in the same group split messages across them. Different groups each receive every message (fanout).
|
|
74
|
-
|
|
75
|
-
`prefetch` is the per-consumer concurrency cap. Higher = throughput; lower = even spread. For CPU-bound handlers, set ~= CPU cores. For IO-bound, can go higher (50+).
|
|
76
|
-
|
|
77
|
-
## Retry policy
|
|
78
|
-
|
|
79
|
-
```ts
|
|
80
|
-
retry: {
|
|
81
|
-
maxRetries: 3,
|
|
82
|
-
delay: 1000, // see the caveat below — delay is not auto-applied
|
|
83
|
-
}
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
`maxRetries` is the part that does the work: when a handler **throws**, herald reads the message's `x-retry-count` header and, while it's under `maxRetries`, republishes with the header incremented so the count actually advances (a plain requeue never advances it — the broker doesn't add that header). Once `x-retry-count` reaches `maxRetries`, it dead-letters (if `deadLetter` is configured) or drops the message with a loud `log.error` — a poison message can no longer ping-pong forever. This is shared by the automatic throw path and the explicit `ctx.retry()` call, so both honor the same cap.
|
|
87
|
-
|
|
88
|
-
**Caveat on `delay`.** `RetryOptions.delay` (number or `(attempt) => number`) is **not applied on the automatic throw path** — a thrown handler requeues immediately, with no wait. The only place a delay takes effect is the explicit `ctx.retry(delayMs)` call, which republishes the message with an `x-delay` header — and even that needs the RabbitMQ delayed-message-exchange plugin installed, or the delay is ignored. So if you need real backoff, call `ctx.retry(ms)` from inside the handler and install the plugin; don't rely on the channel-level `retry.delay` for timing.
|
|
89
|
-
|
|
90
|
-
## Dead-letter queue
|
|
91
|
-
|
|
92
|
-
```ts
|
|
93
|
-
deadLetter: {
|
|
94
|
-
channel: "user.created.failed",
|
|
95
|
-
preserveOriginal: true, // accepted, but currently a no-op (see note)
|
|
96
|
-
}
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
The DLQ always forwards the full original payload plus the message metadata, so `preserveOriginal` makes no observable difference today — the field is part of the type but the RabbitMQ driver doesn't branch on it yet. Set it or omit it; behaviour is the same.
|
|
100
|
-
|
|
101
|
-
Messages that exhausted retries land in `user.created.failed`. Subscribe to it separately for alerting / manual inspection:
|
|
102
|
-
|
|
103
|
-
```ts
|
|
104
|
-
herald().channel("user.created.failed").subscribe(async (message, ctx) => {
|
|
105
|
-
await alerts.notify(`Failed: ${JSON.stringify(message.payload)}`);
|
|
106
|
-
await ctx.ack();
|
|
107
|
-
});
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
## Decorator-style consumers — `@Consumable` + `EventConsumer`
|
|
111
|
-
|
|
112
|
-
For class-based consumers that pair with the [`EventMessage`](@warlock.js/herald/publish-message/SKILL.md) producer layer. This is a **different handler shape** from the raw `.subscribe()` above — ack/nack is automatic, and you get the unwrapped payload plus an event-metadata object (no `ctx`):
|
|
113
|
-
|
|
114
|
-
```ts
|
|
115
|
-
import { Consumable, EventConsumer } from "@warlock.js/herald";
|
|
116
|
-
import type { ConsumedEventMessage } from "@warlock.js/herald";
|
|
117
|
-
|
|
118
|
-
@Consumable() // or @Consumable({ broker: "analytics" }) to target a non-default broker
|
|
119
|
-
export class UserCreatedConsumer extends EventConsumer<{ id: number; email: string }> {
|
|
120
|
-
public static eventName = "user.created";
|
|
121
|
-
|
|
122
|
-
// handle(payload, event) — NOT (message, ctx). No ctx.ack() here.
|
|
123
|
-
public async handle(payload: { id: number; email: string }, event: ConsumedEventMessage) {
|
|
124
|
-
await sendWelcomeEmail(payload.email);
|
|
125
|
-
// return cleanly → herald acks. throw → herald retries (bounded — see below).
|
|
126
|
-
}
|
|
127
|
-
}
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
The handler receives the **already-unwrapped payload** as the first argument and a `ConsumedEventMessage` as the second — `{ messageId, eventName, payload, version?, occurredAt?, metadata?, message }` (where `message` is the raw `Message` if you need `message.metadata.headers`). There is **no `ctx`**: the framework auto-acks when `handle` resolves, so you never call `ack`/`nack` yourself in this style.
|
|
131
|
-
|
|
132
|
-
**A throw is a bounded retry, not an infinite requeue.** `handle` throwing routes through the same bounded-retry routine the raw `.subscribe()` throw path uses (below) — republishes with an incremented `x-retry-count`, capped at `maxRetries: 3` (there's no per-consumer `retry`/`deadLetter` config yet, so the cap is always the default). Once the cap is hit, the message is dropped with a loud `log.error` — never a silent, unbounded ack/nack loop.
|
|
133
|
-
|
|
134
|
-
Wiring: the channel name comes from `static eventName`, and `@Consumable` self-registers the moment the class module is **imported** — if a broker is already connected it subscribes immediately, otherwise it buffers and subscribes once `connectToBroker` fires. So the only wiring you need is to import the consumer file on the boot path (e.g. your module's `main.ts`).
|
|
135
|
-
|
|
136
|
-
**Pair it with the producer:** an `EventConsumer` expects the envelope shape that `publishEvent(new SomeEvent(...))` emits — it reads `envelope.payload`. Don't point one at a channel fed by a bare `channel(name).publish(rawBody)`; the payload nesting won't match.
|
|
137
|
-
|
|
138
|
-
### Validation + version gating
|
|
139
|
-
|
|
140
|
-
Two optional gates run before `handle`, both driven by fields on the class:
|
|
141
|
-
|
|
142
|
-
```ts
|
|
143
|
-
import { Consumable, EventConsumer } from "@warlock.js/herald";
|
|
144
|
-
import { v } from "@warlock.js/seal";
|
|
145
|
-
|
|
146
|
-
@Consumable()
|
|
147
|
-
export class UserCreatedConsumer extends EventConsumer {
|
|
148
|
-
public static eventName = "user.created";
|
|
149
|
-
|
|
150
|
-
// Only accept events whose `version` falls in [minVersion, maxVersion].
|
|
151
|
-
// Out-of-range events are acked and skipped (not requeued).
|
|
152
|
-
public static minVersion = 2;
|
|
153
|
-
public static maxVersion = 3;
|
|
154
|
-
|
|
155
|
-
// Seal schema — invalid payloads are nacked before handle() runs.
|
|
156
|
-
public schema = v.object({ id: v.int(), email: v.string().email() });
|
|
157
|
-
|
|
158
|
-
public async handle(payload, event) {
|
|
159
|
-
// payload is validated + version-accepted here
|
|
160
|
-
}
|
|
161
|
-
}
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
### `defineConsumer` — no-class shorthand
|
|
165
|
-
|
|
166
|
-
```ts
|
|
167
|
-
import { defineConsumer } from "@warlock.js/herald";
|
|
168
|
-
|
|
169
|
-
export const userCreatedConsumer = defineConsumer<{ id: number; email: string }>("user.created", {
|
|
170
|
-
schema: userCreatedSchema, // optional seal validation
|
|
171
|
-
handle: async (payload, event) => {
|
|
172
|
-
await sendWelcomeEmail(payload.email);
|
|
173
|
-
},
|
|
174
|
-
});
|
|
175
|
-
```
|
|
176
|
-
|
|
177
|
-
`defineConsumer` self-registers via `@Consumable` internally, so importing the module is all the wiring it needs — same as the class form. (`subscribeConsumer(SomeConsumerClass)` is the imperative equivalent if you'd rather register explicitly.)
|
|
178
|
-
|
|
179
|
-
## Subscription handle
|
|
180
|
-
|
|
181
|
-
`subscribe()` resolves to a `Subscription` you can manage:
|
|
182
|
-
|
|
183
|
-
```ts
|
|
184
|
-
const subscription = await channel.subscribe(handler);
|
|
185
|
-
|
|
186
|
-
subscription.id; // the consumer id (string)
|
|
187
|
-
subscription.isActive(); // boolean
|
|
188
|
-
|
|
189
|
-
await subscription.unsubscribe(); // cancel the consumer on the broker
|
|
190
|
-
await subscription.pause(); // also cancels the consumer (RabbitMQ has no native pause)
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
On the RabbitMQ driver, `pause()` and `unsubscribe()` both cancel the consumer; `resume()` is **not supported and throws** — create a fresh subscription to start consuming again. You can also cancel by id from the channel: `await channel.unsubscribeById(consumerId)`, or drop every consumer on the channel with `await channel.stopConsuming()`.
|
|
194
|
-
|
|
195
|
-
## Channel admin
|
|
196
|
-
|
|
197
|
-
The channel exposes queue-management helpers beyond pub/sub:
|
|
198
|
-
|
|
199
|
-
```ts
|
|
200
|
-
const { messageCount, consumerCount } = await channel.stats(); // queue depth + live consumers
|
|
201
|
-
const purged = await channel.purge(); // drop all pending messages, returns the count
|
|
202
|
-
const ok = await channel.exists(); // does the queue exist on the broker?
|
|
203
|
-
await channel.assert(); // create the queue with its options (idempotent)
|
|
204
|
-
await channel.delete(); // remove the queue entirely
|
|
205
|
-
```
|
|
206
|
-
|
|
207
|
-
`assert()` runs lazily on the first `publish` / `subscribe`, so you rarely call it directly — reach for it only to pre-create a queue before any traffic.
|
|
208
|
-
|
|
209
|
-
## Things NOT to do
|
|
210
|
-
|
|
211
|
-
- Don't catch the handler's error and return normally. Smart auto-ack reads a clean return as success and **acks** — silently dropping the work. Let it throw (auto-nack) or call `ctx.reject()` / `ctx.nack(false)` deliberately.
|
|
212
|
-
- Don't `ctx.nack(true)` in an infinite loop. With no `retry` policy, a perpetually-failing message ping-pongs forever. Set `retry.maxRetries` + a `deadLetter` so failures eventually move out.
|
|
213
|
-
- Don't ignore the `deadLetter` channel. Subscribe to it (or at least monitor its depth) — DLQ growth means real failures piling up.
|
|
214
|
-
- Don't use `autoAck: true` for messages that matter. The broker acks on delivery, so a crash mid-handling loses the message.
|
|
215
|
-
|
|
216
|
-
## See also
|
|
217
|
-
|
|
218
|
-
- [`@warlock.js/herald/publish-message/SKILL.md`](@warlock.js/herald/publish-message/SKILL.md) — the producing side
|
|
219
|
-
- [`@warlock.js/herald/request-and-respond/SKILL.md`](@warlock.js/herald/request-and-respond/SKILL.md) — when you need to reply to a message
|
|
1
|
+
---
|
|
2
|
+
name: consume-message
|
|
3
|
+
description: 'Subscribe to a channel via .subscribe(handler, options?) — handler receives (message, ctx). Control flow via ctx.ack / ctx.nack(requeue?) / ctx.reject / ctx.retry(delayMs). Configure prefetch, retry policy, dead-letter, consumer groups. Triggers: `subscribe`, `ctx.ack`, `ctx.nack`, `ctx.reject`, `ctx.retry`, `prefetch`, `group`, `retry`, `deadLetter`, `@Consumable`, `EventConsumer`, `defineConsumer`, `autoAck`; "consume messages from a queue", "implement a worker", "background queue processor", "set up retry and dead-letter", "class-based event consumer"; typical import `import { herald, Consumable, EventConsumer } from "@warlock.js/herald"`. Skip: setup — `@warlock.js/herald/herald-basics/SKILL.md`; producing — `@warlock.js/herald/publish-message/SKILL.md`; RPC reply — `@warlock.js/herald/request-and-respond/SKILL.md`; competing libs `amqplib`, `bullmq`, `kafkajs`; NestJS `@MessagePattern` / `@EventPattern`.'
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Consume messages
|
|
7
|
+
|
|
8
|
+
Subscribe to a channel; receive each message with a flow-control `ctx`.
|
|
9
|
+
|
|
10
|
+
## Minimal subscribe
|
|
11
|
+
|
|
12
|
+
```ts
|
|
13
|
+
import { herald } from "@warlock.js/herald";
|
|
14
|
+
|
|
15
|
+
herald()
|
|
16
|
+
.channel<{ userId: number; email: string }>("user.created")
|
|
17
|
+
.subscribe(async (message, ctx) => {
|
|
18
|
+
try {
|
|
19
|
+
await sendWelcomeEmail(message.payload.email);
|
|
20
|
+
await ctx.ack(); // success — message removed from queue
|
|
21
|
+
} catch (error) {
|
|
22
|
+
await ctx.nack(true); // failure — requeue for redelivery
|
|
23
|
+
}
|
|
24
|
+
});
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
**Smart auto-ack is on by default** (when `autoAck` is unset or `false`). The consumer runs with manual-ack enabled, and herald acks/nacks based on how your handler ends:
|
|
28
|
+
|
|
29
|
+
- Handler returns without calling any `ctx` method → herald **auto-acks**.
|
|
30
|
+
- Handler **throws** → herald **auto-nacks** (requeue, or DLQ/reject once retries are exhausted — see below).
|
|
31
|
+
- Handler calls `ctx.ack()` / `ctx.nack()` / `ctx.reject()` / `ctx.retry()` → herald respects that and does nothing further.
|
|
32
|
+
|
|
33
|
+
So the `try/catch` above is optional — letting the error throw produces the same auto-nack. Call `ctx` methods explicitly only when you want a *non-default* outcome (reject without requeue, route to DLQ, delayed retry). The one mode that loses messages is `autoAck: true`, where the broker acks on delivery before your handler runs.
|
|
34
|
+
|
|
35
|
+
## Message context — flow control
|
|
36
|
+
|
|
37
|
+
```ts
|
|
38
|
+
ctx.ack(); // acknowledge — message removed from the queue
|
|
39
|
+
ctx.nack(requeue?); // negative ack — requeue (true) or send to DLQ (false)
|
|
40
|
+
ctx.reject(); // shorthand for nack(false)
|
|
41
|
+
ctx.retry(delayMs); // delayed retry — requeue after delay
|
|
42
|
+
ctx.reply(payload); // for request-response pattern (see request-and-respond skill)
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Pick by intent:
|
|
46
|
+
|
|
47
|
+
| Intent | Use |
|
|
48
|
+
| --- | --- |
|
|
49
|
+
| Processed cleanly | `ctx.ack()` |
|
|
50
|
+
| Transient failure, try again | `ctx.nack(true)` or `ctx.retry(5000)` |
|
|
51
|
+
| Permanent failure, send to DLQ | `ctx.nack(false)` or `ctx.reject()` |
|
|
52
|
+
| Validation failure (bad message) | `ctx.reject()` — don't requeue |
|
|
53
|
+
|
|
54
|
+
## Subscribe options
|
|
55
|
+
|
|
56
|
+
```ts
|
|
57
|
+
await channel.subscribe(handler, {
|
|
58
|
+
group: "email-workers", // consumer group / tag — multiple consumers share work
|
|
59
|
+
prefetch: 10, // concurrency — how many in-flight messages this consumer holds
|
|
60
|
+
autoAck: false, // default; keep it false in production
|
|
61
|
+
exclusive: false, // single consumer only on this channel?
|
|
62
|
+
retry: {
|
|
63
|
+
maxRetries: 3, // redelivery ceiling on handler throw (delay not auto-applied — see below)
|
|
64
|
+
delay: 1000,
|
|
65
|
+
},
|
|
66
|
+
deadLetter: {
|
|
67
|
+
channel: "user.created.failed",
|
|
68
|
+
preserveOriginal: true,
|
|
69
|
+
},
|
|
70
|
+
});
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`group` is the unit of "share work" — N consumers in the same group split messages across them. Different groups each receive every message (fanout).
|
|
74
|
+
|
|
75
|
+
`prefetch` is the per-consumer concurrency cap. Higher = throughput; lower = even spread. For CPU-bound handlers, set ~= CPU cores. For IO-bound, can go higher (50+).
|
|
76
|
+
|
|
77
|
+
## Retry policy
|
|
78
|
+
|
|
79
|
+
```ts
|
|
80
|
+
retry: {
|
|
81
|
+
maxRetries: 3,
|
|
82
|
+
delay: 1000, // see the caveat below — delay is not auto-applied
|
|
83
|
+
}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
`maxRetries` is the part that does the work: when a handler **throws**, herald reads the message's `x-retry-count` header and, while it's under `maxRetries`, republishes with the header incremented so the count actually advances (a plain requeue never advances it — the broker doesn't add that header). Once `x-retry-count` reaches `maxRetries`, it dead-letters (if `deadLetter` is configured) or drops the message with a loud `log.error` — a poison message can no longer ping-pong forever. This is shared by the automatic throw path and the explicit `ctx.retry()` call, so both honor the same cap.
|
|
87
|
+
|
|
88
|
+
**Caveat on `delay`.** `RetryOptions.delay` (number or `(attempt) => number`) is **not applied on the automatic throw path** — a thrown handler requeues immediately, with no wait. The only place a delay takes effect is the explicit `ctx.retry(delayMs)` call, which republishes the message with an `x-delay` header — and even that needs the RabbitMQ delayed-message-exchange plugin installed, or the delay is ignored. So if you need real backoff, call `ctx.retry(ms)` from inside the handler and install the plugin; don't rely on the channel-level `retry.delay` for timing.
|
|
89
|
+
|
|
90
|
+
## Dead-letter queue
|
|
91
|
+
|
|
92
|
+
```ts
|
|
93
|
+
deadLetter: {
|
|
94
|
+
channel: "user.created.failed",
|
|
95
|
+
preserveOriginal: true, // accepted, but currently a no-op (see note)
|
|
96
|
+
}
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
The DLQ always forwards the full original payload plus the message metadata, so `preserveOriginal` makes no observable difference today — the field is part of the type but the RabbitMQ driver doesn't branch on it yet. Set it or omit it; behaviour is the same.
|
|
100
|
+
|
|
101
|
+
Messages that exhausted retries land in `user.created.failed`. Subscribe to it separately for alerting / manual inspection:
|
|
102
|
+
|
|
103
|
+
```ts
|
|
104
|
+
herald().channel("user.created.failed").subscribe(async (message, ctx) => {
|
|
105
|
+
await alerts.notify(`Failed: ${JSON.stringify(message.payload)}`);
|
|
106
|
+
await ctx.ack();
|
|
107
|
+
});
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## Decorator-style consumers — `@Consumable` + `EventConsumer`
|
|
111
|
+
|
|
112
|
+
For class-based consumers that pair with the [`EventMessage`](@warlock.js/herald/publish-message/SKILL.md) producer layer. This is a **different handler shape** from the raw `.subscribe()` above — ack/nack is automatic, and you get the unwrapped payload plus an event-metadata object (no `ctx`):
|
|
113
|
+
|
|
114
|
+
```ts
|
|
115
|
+
import { Consumable, EventConsumer } from "@warlock.js/herald";
|
|
116
|
+
import type { ConsumedEventMessage } from "@warlock.js/herald";
|
|
117
|
+
|
|
118
|
+
@Consumable() // or @Consumable({ broker: "analytics" }) to target a non-default broker
|
|
119
|
+
export class UserCreatedConsumer extends EventConsumer<{ id: number; email: string }> {
|
|
120
|
+
public static eventName = "user.created";
|
|
121
|
+
|
|
122
|
+
// handle(payload, event) — NOT (message, ctx). No ctx.ack() here.
|
|
123
|
+
public async handle(payload: { id: number; email: string }, event: ConsumedEventMessage) {
|
|
124
|
+
await sendWelcomeEmail(payload.email);
|
|
125
|
+
// return cleanly → herald acks. throw → herald retries (bounded — see below).
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
The handler receives the **already-unwrapped payload** as the first argument and a `ConsumedEventMessage` as the second — `{ messageId, eventName, payload, version?, occurredAt?, metadata?, message }` (where `message` is the raw `Message` if you need `message.metadata.headers`). There is **no `ctx`**: the framework auto-acks when `handle` resolves, so you never call `ack`/`nack` yourself in this style.
|
|
131
|
+
|
|
132
|
+
**A throw is a bounded retry, not an infinite requeue.** `handle` throwing routes through the same bounded-retry routine the raw `.subscribe()` throw path uses (below) — republishes with an incremented `x-retry-count`, capped at `maxRetries: 3` (there's no per-consumer `retry`/`deadLetter` config yet, so the cap is always the default). Once the cap is hit, the message is dropped with a loud `log.error` — never a silent, unbounded ack/nack loop.
|
|
133
|
+
|
|
134
|
+
Wiring: the channel name comes from `static eventName`, and `@Consumable` self-registers the moment the class module is **imported** — if a broker is already connected it subscribes immediately, otherwise it buffers and subscribes once `connectToBroker` fires. So the only wiring you need is to import the consumer file on the boot path (e.g. your module's `main.ts`).
|
|
135
|
+
|
|
136
|
+
**Pair it with the producer:** an `EventConsumer` expects the envelope shape that `publishEvent(new SomeEvent(...))` emits — it reads `envelope.payload`. Don't point one at a channel fed by a bare `channel(name).publish(rawBody)`; the payload nesting won't match.
|
|
137
|
+
|
|
138
|
+
### Validation + version gating
|
|
139
|
+
|
|
140
|
+
Two optional gates run before `handle`, both driven by fields on the class:
|
|
141
|
+
|
|
142
|
+
```ts
|
|
143
|
+
import { Consumable, EventConsumer } from "@warlock.js/herald";
|
|
144
|
+
import { v } from "@warlock.js/seal";
|
|
145
|
+
|
|
146
|
+
@Consumable()
|
|
147
|
+
export class UserCreatedConsumer extends EventConsumer {
|
|
148
|
+
public static eventName = "user.created";
|
|
149
|
+
|
|
150
|
+
// Only accept events whose `version` falls in [minVersion, maxVersion].
|
|
151
|
+
// Out-of-range events are acked and skipped (not requeued).
|
|
152
|
+
public static minVersion = 2;
|
|
153
|
+
public static maxVersion = 3;
|
|
154
|
+
|
|
155
|
+
// Seal schema — invalid payloads are nacked before handle() runs.
|
|
156
|
+
public schema = v.object({ id: v.int(), email: v.string().email() });
|
|
157
|
+
|
|
158
|
+
public async handle(payload, event) {
|
|
159
|
+
// payload is validated + version-accepted here
|
|
160
|
+
}
|
|
161
|
+
}
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### `defineConsumer` — no-class shorthand
|
|
165
|
+
|
|
166
|
+
```ts
|
|
167
|
+
import { defineConsumer } from "@warlock.js/herald";
|
|
168
|
+
|
|
169
|
+
export const userCreatedConsumer = defineConsumer<{ id: number; email: string }>("user.created", {
|
|
170
|
+
schema: userCreatedSchema, // optional seal validation
|
|
171
|
+
handle: async (payload, event) => {
|
|
172
|
+
await sendWelcomeEmail(payload.email);
|
|
173
|
+
},
|
|
174
|
+
});
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
`defineConsumer` self-registers via `@Consumable` internally, so importing the module is all the wiring it needs — same as the class form. (`subscribeConsumer(SomeConsumerClass)` is the imperative equivalent if you'd rather register explicitly.)
|
|
178
|
+
|
|
179
|
+
## Subscription handle
|
|
180
|
+
|
|
181
|
+
`subscribe()` resolves to a `Subscription` you can manage:
|
|
182
|
+
|
|
183
|
+
```ts
|
|
184
|
+
const subscription = await channel.subscribe(handler);
|
|
185
|
+
|
|
186
|
+
subscription.id; // the consumer id (string)
|
|
187
|
+
subscription.isActive(); // boolean
|
|
188
|
+
|
|
189
|
+
await subscription.unsubscribe(); // cancel the consumer on the broker
|
|
190
|
+
await subscription.pause(); // also cancels the consumer (RabbitMQ has no native pause)
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
On the RabbitMQ driver, `pause()` and `unsubscribe()` both cancel the consumer; `resume()` is **not supported and throws** — create a fresh subscription to start consuming again. You can also cancel by id from the channel: `await channel.unsubscribeById(consumerId)`, or drop every consumer on the channel with `await channel.stopConsuming()`.
|
|
194
|
+
|
|
195
|
+
## Channel admin
|
|
196
|
+
|
|
197
|
+
The channel exposes queue-management helpers beyond pub/sub:
|
|
198
|
+
|
|
199
|
+
```ts
|
|
200
|
+
const { messageCount, consumerCount } = await channel.stats(); // queue depth + live consumers
|
|
201
|
+
const purged = await channel.purge(); // drop all pending messages, returns the count
|
|
202
|
+
const ok = await channel.exists(); // does the queue exist on the broker?
|
|
203
|
+
await channel.assert(); // create the queue with its options (idempotent)
|
|
204
|
+
await channel.delete(); // remove the queue entirely
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
`assert()` runs lazily on the first `publish` / `subscribe`, so you rarely call it directly — reach for it only to pre-create a queue before any traffic.
|
|
208
|
+
|
|
209
|
+
## Things NOT to do
|
|
210
|
+
|
|
211
|
+
- Don't catch the handler's error and return normally. Smart auto-ack reads a clean return as success and **acks** — silently dropping the work. Let it throw (auto-nack) or call `ctx.reject()` / `ctx.nack(false)` deliberately.
|
|
212
|
+
- Don't `ctx.nack(true)` in an infinite loop. With no `retry` policy, a perpetually-failing message ping-pongs forever. Set `retry.maxRetries` + a `deadLetter` so failures eventually move out.
|
|
213
|
+
- Don't ignore the `deadLetter` channel. Subscribe to it (or at least monitor its depth) — DLQ growth means real failures piling up.
|
|
214
|
+
- Don't use `autoAck: true` for messages that matter. The broker acks on delivery, so a crash mid-handling loses the message.
|
|
215
|
+
|
|
216
|
+
## See also
|
|
217
|
+
|
|
218
|
+
- [`@warlock.js/herald/publish-message/SKILL.md`](@warlock.js/herald/publish-message/SKILL.md) — the producing side
|
|
219
|
+
- [`@warlock.js/herald/request-and-respond/SKILL.md`](@warlock.js/herald/request-and-respond/SKILL.md) — when you need to reply to a message
|
package/skills/index.md
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Message bus over RabbitMQ (Kafka planned): publish, subscribe, and request-response between services. Exports `connectToBroker`, `herald`, `heraldChannel`, `Broker`, `defineEvent`, `defineConsumer`, `EventMessage`, `EventConsumer`, `Consumable`, `brokerRegistry`. Use for: \"publish an event to RabbitMQ\", \"subscribe to a channel\", \"ack or nack a message\", \"request and reply over the bus\", \"cross-service messaging\". Not this package: durable retried background jobs on Redis → @warlock.js/queue; cron/recurring work → @warlock.js/scheduler."
|
|
3
|
+
---
|
|
4
|
+
# @warlock.js/herald
|
|
5
|
+
|
|
6
|
+
Herald is the broker-agnostic message bus. `connectToBroker(config)` boots a broker once; `herald()` returns it (`herald("name")` for a named one); `.channel(name)` is the queue/topic you publish into and subscribe from. Channels are typed with generics and can validate with a seal schema. Server-only.
|
|
7
|
+
|
|
8
|
+
## The 80% path
|
|
9
|
+
1. Orient with `herald-basics.md` (the map and foundations).
|
|
10
|
+
2. Connect at startup, then publish: `publish-message.md`.
|
|
11
|
+
3. Consume with `subscribe` handlers or consumer classes: `consume-message.md`.
|
|
12
|
+
4. Need a reply? `request-and-respond.md`.
|
|
13
|
+
|
|
14
|
+
## Conventions and pitfalls
|
|
15
|
+
- Call `connectToBroker` once before any publish or subscribe.
|
|
16
|
+
- Smart auto-ack is the default: a clean return acks, a throw nacks with requeue. `autoAck: true` loses messages on a crash; avoid it.
|
|
17
|
+
- Use `ctx.reject()` / `ctx.retry(ms)` only for a non-default outcome.
|
|
18
|
+
- Herald is transport, not a job runner: no retries-as-jobs, no scheduling. Use `@warlock.js/queue` for retried jobs, `@warlock.js/scheduler` for cron.
|
|
19
|
+
- Install the driver (`amqplib`) yourself; Kafka is not shipped yet.
|
|
20
|
+
- Do not value-import from client code.
|