@nestjs-transactional/outbox-microservices 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -10,27 +10,44 @@ Mark an event `@Externalized`, and once its local handlers have
|
|
|
10
10
|
completed, [`@nestjs-transactional/outbox`](https://www.npmjs.com/package/@nestjs-transactional/outbox)
|
|
11
11
|
hands it to this package, which emits it over a `ClientProxy` you
|
|
12
12
|
already configured. One implementation covers every transport
|
|
13
|
-
`@nestjs/microservices` supports — Kafka, RabbitMQ,
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
13
|
+
`@nestjs/microservices` supports — Kafka, RabbitMQ, MQTT, Redis, NATS,
|
|
14
|
+
and custom strategies. gRPC is the exception; see below.
|
|
15
|
+
|
|
16
|
+
## What a successful publish means on your transport
|
|
17
|
+
|
|
18
|
+
A publication is marked `COMPLETED` when `emit()` resolves, and what
|
|
19
|
+
`emit()` waits for is not the same on every transport. Kafka and
|
|
20
|
+
RabbitMQ wait for a real broker acknowledgement, so an unreachable
|
|
21
|
+
broker marks the publication `FAILED` and the outbox's retry and
|
|
22
|
+
resubmit machinery engages. NATS and TCP do not wait for anything.
|
|
23
|
+
|
|
24
|
+
| Transport | `emit()` resolves when | Acknowledged by the broker? |
|
|
25
|
+
| --- | --- | --- |
|
|
26
|
+
| Kafka | `producer.send()` settles; `kafkajs` defaults to `acks: -1`, every in-sync replica | Yes |
|
|
27
|
+
| RabbitMQ | the publisher confirm arrives (`amqp-connection-manager` enables confirms by default) | Yes, but set `persistent: true` |
|
|
28
|
+
| MQTT | PUBACK at QoS 1 and above, immediately at QoS 0 | At QoS 1 and above |
|
|
29
|
+
| Redis | the `PUBLISH` command replies | The server got it, but Redis pub/sub does not persist: only live subscribers receive it |
|
|
30
|
+
| TCP | the message is written to the socket | No |
|
|
31
|
+
| NATS | immediately: core `publish()` returns `void`, and the client resolves unconditionally | No |
|
|
32
|
+
| gRPC | never: `dispatchEvent` throws `Method is not supported in gRPC mode` | Not usable for externalization |
|
|
33
|
+
|
|
34
|
+
Two things worth acting on:
|
|
35
|
+
|
|
36
|
+
- **RabbitMQ publishes non-persistent by default.** NestJS defaults
|
|
37
|
+
`persistent` to `false`, and RabbitMQ confirms a non-persistent
|
|
38
|
+
message without writing it to disk, so a broker restart loses it.
|
|
39
|
+
Pass `persistent: true` in your `ClientsModule.register()` options.
|
|
40
|
+
- **NATS gives you no delivery signal at all.** Core NATS publish is
|
|
41
|
+
fire-and-forget by protocol. If you need a guarantee there, this
|
|
42
|
+
externalizer is not the right one; the `EVENT_EXTERNALIZER` SPI is
|
|
43
|
+
public and a JetStream-based implementation slots into the same
|
|
44
|
+
place.
|
|
45
|
+
|
|
46
|
+
The outbox itself guarantees crash-consistent **enqueueing** and
|
|
47
|
+
at-least-once delivery to **local** handlers regardless of transport.
|
|
48
|
+
Measurements, the reading of each client's `dispatchEvent`, and what
|
|
49
|
+
remains genuinely unguaranteed:
|
|
50
|
+
[ADR-021](https://github.com/igorgolovanov/nestjs-transactional/blob/main/docs/adr/021-externalization-acknowledgement-per-transport.md).
|
|
34
51
|
|
|
35
52
|
## Install
|
|
36
53
|
|
|
@@ -97,34 +114,38 @@ publication, so a typo surfaces on deploy instead of in the middle of
|
|
|
97
114
|
the night. Pass `validateOnBootstrap: false` to defer resolution if you
|
|
98
115
|
register clients late.
|
|
99
116
|
|
|
100
|
-
##
|
|
117
|
+
## Hardening the delivery
|
|
101
118
|
|
|
102
|
-
|
|
103
|
-
|
|
119
|
+
The defaults are reasonable on Kafka and RabbitMQ. What is left is
|
|
120
|
+
mostly about not weakening them, and about the consumer side.
|
|
104
121
|
|
|
105
|
-
1. **
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
not
|
|
110
|
-
|
|
122
|
+
1. **Do not configure the acknowledgement away.** Kafka `acks: 0`,
|
|
123
|
+
MQTT QoS 0, and RabbitMQ without `persistent: true` each give up a
|
|
124
|
+
guarantee you had for free. `producer.idempotent: true` on Kafka
|
|
125
|
+
additionally protects against duplicates from producer retries.
|
|
126
|
+
This package reuses whatever proxy you registered and does not
|
|
127
|
+
interfere with any of it.
|
|
128
|
+
2. **Deduplicate on the consumer.** At-least-once means duplicates are
|
|
129
|
+
expected, not exceptional. Track processed message ids on the
|
|
111
130
|
receiving side and alert on gaps. The listener id plus the event id
|
|
112
131
|
is enough to identify a message.
|
|
113
|
-
3. **
|
|
114
|
-
`
|
|
115
|
-
|
|
116
|
-
|
|
132
|
+
3. **Watch the `FAILED` publications.** A broker rejection now reaches
|
|
133
|
+
you as a `FAILED` row with a readable `failureReason`, which is what
|
|
134
|
+
`FailedEventPublications.resubmit` and the retry scheduler act on.
|
|
135
|
+
That path is only useful if someone is looking at it.
|
|
117
136
|
|
|
118
137
|
## Limitations
|
|
119
138
|
|
|
120
139
|
- **Headers and `routingKey`** are accepted by `@Externalized` but not
|
|
121
140
|
yet applied to the emitted payload.
|
|
122
|
-
- **
|
|
141
|
+
- **gRPC cannot be used** as an externalization transport:
|
|
142
|
+
`ClientGrpcProxy.dispatchEvent` throws.
|
|
143
|
+
- **NATS and TCP give no delivery signal** — see the table at the top.
|
|
123
144
|
|
|
124
145
|
## Documentation
|
|
125
146
|
|
|
126
147
|
- [Getting started and full docs](https://github.com/igorgolovanov/nestjs-transactional#readme)
|
|
127
|
-
- [
|
|
148
|
+
- [Acknowledgement per transport (ADR-021)](https://github.com/igorgolovanov/nestjs-transactional/blob/main/docs/adr/021-externalization-acknowledgement-per-transport.md)
|
|
128
149
|
- [Externalization architecture (ADR-015)](https://github.com/igorgolovanov/nestjs-transactional/blob/main/docs/adr/015-event-externalization-architecture.md)
|
|
129
150
|
- [Architecture: event externalization](https://github.com/igorgolovanov/nestjs-transactional/blob/main/docs/architecture/event-externalization.md)
|
|
130
151
|
- Runnable examples:
|
|
@@ -64,12 +64,12 @@ let MicroservicesEventExternalizer = MicroservicesEventExternalizer_1 = class Mi
|
|
|
64
64
|
this.logger.log(`Externalization configured with default client: ${formatToken(this.options.defaultClient)}`);
|
|
65
65
|
}
|
|
66
66
|
catch (err) {
|
|
67
|
-
const
|
|
67
|
+
const reason = (0, outbox_1.describeThrown)(err);
|
|
68
68
|
throw new Error(`OutboxMicroservicesModule: defaultClient '${formatToken(this.options.defaultClient)}' ` +
|
|
69
69
|
`is not registered in the DI container. Register the ClientProxy via ` +
|
|
70
70
|
`ClientsModule.register() / ClientsModule.registerAsync(), or pass ` +
|
|
71
71
|
`validateOnBootstrap: false to defer resolution to the first event. ` +
|
|
72
|
-
`Original error: ${cause}
|
|
72
|
+
`Original error: ${reason}`, { cause: err });
|
|
73
73
|
}
|
|
74
74
|
}
|
|
75
75
|
async externalize(event, metadata) {
|
|
@@ -85,7 +85,7 @@ let MicroservicesEventExternalizer = MicroservicesEventExternalizer_1 = class Mi
|
|
|
85
85
|
}
|
|
86
86
|
catch (err) {
|
|
87
87
|
const cause = err instanceof Error ? err : undefined;
|
|
88
|
-
const causeMessage =
|
|
88
|
+
const causeMessage = (0, outbox_1.describeThrown)(err);
|
|
89
89
|
throw new outbox_1.ExternalizationError(`ClientProxy '${formatToken(clientToken)}' not found in the DI container. ` +
|
|
90
90
|
`Ensure it is registered via ClientsModule.register() / registerAsync(). ` +
|
|
91
91
|
`Original error: ${causeMessage}`, metadata.eventType, metadata.target, cause);
|
|
@@ -103,7 +103,7 @@ let MicroservicesEventExternalizer = MicroservicesEventExternalizer_1 = class Mi
|
|
|
103
103
|
}
|
|
104
104
|
catch (err) {
|
|
105
105
|
const cause = err instanceof Error ? err : undefined;
|
|
106
|
-
const causeMessage =
|
|
106
|
+
const causeMessage = (0, outbox_1.describeThrown)(err);
|
|
107
107
|
throw new outbox_1.ExternalizationError(`Failed to publish ${metadata.eventType} to ${metadata.target}: ${causeMessage}`, metadata.eventType, metadata.target, cause);
|
|
108
108
|
}
|
|
109
109
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"microservices-event-externalizer.js","sourceRoot":"","sources":["../../src/externalizer/microservices-event-externalizer.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;AAAA,2CAMwB;AACxB,uCAAyC;AAEzC,
|
|
1
|
+
{"version":3,"file":"microservices-event-externalizer.js","sourceRoot":"","sources":["../../src/externalizer/microservices-event-externalizer.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;AAAA,2CAMwB;AACxB,uCAAyC;AAEzC,yDAKsC;AACtC,+BAAsC;AAEtC,8CAAiG;AAEjG;;;;;;;;;;;;;;;;;;;;;;GAsBG;AAEI,IAAM,8BAA8B,sCAApC,MAAM,8BAA8B;IAItB;IAEA;IALF,MAAM,GAAG,IAAI,eAAM,CAAC,gCAA8B,CAAC,IAAI,CAAC,CAAC;IAE1E,YACmB,SAAoB,EAEpB,OAAmC;QAFnC,cAAS,GAAT,SAAS,CAAW;QAEpB,YAAO,GAAP,OAAO,CAA4B;IACnD,CAAC;IAEJ,sBAAsB;QACpB,IAAI,IAAI,CAAC,OAAO,CAAC,mBAAmB,KAAK,KAAK,EAAE,CAAC;YAC/C,IAAI,CAAC,MAAM,CAAC,GAAG,CACb,4FAA4F,CAC7F,CAAC;YACF,OAAO;QACT,CAAC;QAED,IAAI,IAAI,CAAC,OAAO,CAAC,aAAa,KAAK,SAAS,EAAE,CAAC;YAC7C,IAAI,CAAC,MAAM,CAAC,GAAG,CACb,qFAAqF,CACtF,CAAC;YACF,OAAO;QACT,CAAC;QAED,IAAI,CAAC;YACH,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC;YAC/C,IAAI,CAAC,MAAM,CAAC,GAAG,CACb,mDAAmD,WAAW,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,CAAC,EAAE,CAC7F,CAAC;QACJ,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,MAAM,GAAG,IAAA,uBAAc,EAAC,GAAG,CAAC,CAAC;YACnC,MAAM,IAAI,KAAK,CACb,6CAA6C,WAAW,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,CAAC,IAAI;gBACtF,sEAAsE;gBACtE,oEAAoE;gBACpE,qEAAqE;gBACrE,mBAAmB,MAAM,EAAE,EAC7B,EAAE,KAAK,EAAE,GAAG,EAAE,CACf,CAAC;QACJ,CAAC;IACH,CAAC;IAED,KAAK,CAAC,WAAW,CAAC,KAAc,EAAE,QAAiC;QACjE,MAAM,WAAW,GAAG,QAAQ,CAAC,MAAM,IAAI,IAAI,CAAC,OAAO,CAAC,aAAa,CAAC;QAClE,IAAI,WAAW,KAAK,SAAS,EAAE,CAAC;YAC9B,MAAM,IAAI,6BAAoB,CAC5B,sCAAsC,QAAQ,CAAC,SAAS,IAAI;gBAC1D,mEAAmE;gBACnE,oDAAoD,EACtD,QAAQ,CAAC,SAAS,EAClB,QAAQ,CAAC,MAAM,CAChB,CAAC;QACJ,CAAC;QAED,IAAI,MAAmB,CAAC;QACxB,IAAI,CAAC;YACH,MAAM,GAAG,IAAI,CAAC,aAAa,CAAC,WAAW,CAAC,CAAC;QAC3C,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,KAAK,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,SAAS,CAAC;YACrD,MAAM,YAAY,GAAG,IAAA,uBAAc,EAAC,GAAG,CAAC,CAAC;YACzC,MAAM,IAAI,6BAAoB,CAC5B,gBAAgB,WAAW,CAAC,WAAW,CAAC,mCAAmC;gBACzE,0EAA0E;gBAC1E,mBAAmB,YAAY,EAAE,EACnC,QAAQ,CAAC,SAAS,EAClB,QAAQ,CAAC,MAAM,EACf,KAAK,CACN,CAAC;QACJ,CAAC;QAED,IAAI,QAAQ,CAAC,OAAO,KAAK,SAAS,IAAI,QAAQ,CAAC,UAAU,KAAK,SAAS,EAAE,CAAC;YACxE,uDAAuD;YACvD,8DAA8D;YAC9D,8DAA8D;YAC9D,wCAAwC;YACxC,IAAI,CAAC,MAAM,CAAC,KAAK,CACf,GAAG,QAAQ,CAAC,SAAS,kGAAkG,IAAI,CAAC,SAAS,CACnI,EAAE,OAAO,EAAE,QAAQ,CAAC,OAAO,EAAE,UAAU,EAAE,QAAQ,CAAC,UAAU,EAAE,CAC/D,EAAE,CACJ,CAAC;QACJ,CAAC;QAED,IAAI,CAAC;YACH,MAAM,IAAA,qBAAc,EAAC,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC,CAAC;YAC1D,IAAI,CAAC,MAAM,CAAC,KAAK,CACf,gBAAgB,QAAQ,CAAC,SAAS,MAAM,QAAQ,CAAC,MAAM,aAAa,WAAW,CAAC,WAAW,CAAC,GAAG,CAChG,CAAC;QACJ,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,KAAK,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,SAAS,CAAC;YACrD,MAAM,YAAY,GAAG,IAAA,uBAAc,EAAC,GAAG,CAAC,CAAC;YACzC,MAAM,IAAI,6BAAoB,CAC5B,qBAAqB,QAAQ,CAAC,SAAS,OAAO,QAAQ,CAAC,MAAM,KAAK,YAAY,EAAE,EAChF,QAAQ,CAAC,SAAS,EAClB,QAAQ,CAAC,MAAM,EACf,KAAK,CACN,CAAC;QACJ,CAAC;IACH,CAAC;IAEO,aAAa,CAAC,KAAqB;QACzC,MAAM,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,CAAc,KAAK,EAAE,EAAE,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC;QACxE,IAAI,KAAK,KAAK,IAAI,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;YAC1C,MAAM,IAAI,KAAK,CAAC,qCAAqC,CAAC,CAAC;QACzD,CAAC;QACD,OAAO,KAAK,CAAC;IACf,CAAC;CACF,CAAA;AA1GY,wEAA8B;yCAA9B,8BAA8B;IAD1C,IAAA,mBAAU,GAAE;IAMR,WAAA,IAAA,eAAM,EAAC,sCAA4B,CAAC,CAAA;qCADT,gBAAS;GAJ5B,8BAA8B,CA0G1C;AAED,SAAS,WAAW,CAAC,KAAqB;IACxC,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;QAC9B,OAAO,KAAK,CAAC,QAAQ,EAAE,CAAC;IAC1B,CAAC;IACD,IAAI,OAAO,KAAK,KAAK,UAAU,EAAE,CAAC;QAChC,OAAO,KAAK,CAAC,IAAI,IAAI,4BAA4B,CAAC;IACpD,CAAC;IACD,OAAO,MAAM,CAAC,KAAK,CAAC,CAAC;AACvB,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@nestjs-transactional/outbox-microservices",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "Event externalization for @nestjs-transactional/outbox via @nestjs/microservices ClientProxy — Spring Modulith @Externalized parity",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "commonjs",
|
|
@@ -53,30 +53,36 @@
|
|
|
53
53
|
"@nestjs/microservices": "^10.0.0 || ^11.0.0",
|
|
54
54
|
"reflect-metadata": "^0.1.13 || ^0.2.0",
|
|
55
55
|
"rxjs": "^7.0.0",
|
|
56
|
-
"@nestjs-transactional/outbox": "^1.
|
|
56
|
+
"@nestjs-transactional/outbox": "^1.1.0"
|
|
57
57
|
},
|
|
58
58
|
"devDependencies": {
|
|
59
|
-
"@nestjs/common": "^11.
|
|
60
|
-
"@nestjs/core": "^11.
|
|
61
|
-
"@nestjs/microservices": "^11.
|
|
62
|
-
"@nestjs/testing": "^11.
|
|
63
|
-
"@
|
|
64
|
-
"jest": "^
|
|
59
|
+
"@nestjs/common": "^11.2.3",
|
|
60
|
+
"@nestjs/core": "^11.2.3",
|
|
61
|
+
"@nestjs/microservices": "^11.2.3",
|
|
62
|
+
"@nestjs/testing": "^11.2.3",
|
|
63
|
+
"@testcontainers/kafka": "^12.1.0",
|
|
64
|
+
"@types/jest": "^30.0.0",
|
|
65
|
+
"@types/node": "^22.20.1",
|
|
66
|
+
"amqplib": "^2.0.1",
|
|
67
|
+
"jest": "^30.4.2",
|
|
68
|
+
"kafkajs": "^2.2.4",
|
|
65
69
|
"reflect-metadata": "^0.2.2",
|
|
66
70
|
"rxjs": "^7.8.1",
|
|
67
|
-
"
|
|
68
|
-
"
|
|
71
|
+
"testcontainers": "^12.1.0",
|
|
72
|
+
"ts-jest": "^29.4.12",
|
|
73
|
+
"typescript": "^6.0.3",
|
|
69
74
|
"@nestjs-transactional/core": "1.0.0",
|
|
70
|
-
"@nestjs-transactional/outbox": "1.
|
|
75
|
+
"@nestjs-transactional/outbox": "1.1.0"
|
|
71
76
|
},
|
|
72
77
|
"scripts": {
|
|
73
78
|
"build": "tsc -p tsconfig.build.json",
|
|
74
79
|
"clean": "rimraf dist *.tsbuildinfo coverage",
|
|
75
80
|
"test": "jest",
|
|
76
81
|
"test:watch": "jest --watch",
|
|
82
|
+
"test:integration": "jest --config jest.integration.config.js",
|
|
77
83
|
"test:cov": "jest --coverage",
|
|
78
84
|
"type-check": "tsc --noEmit",
|
|
79
|
-
"lint": "eslint
|
|
85
|
+
"lint": "eslint .",
|
|
80
86
|
"api:check": "api-extractor run",
|
|
81
87
|
"api:update": "api-extractor run --local",
|
|
82
88
|
"publish:check": "publint && attw --pack ."
|