comq 0.23.0 → 0.24.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/discussions/amqp-1.0.md +127 -0
- package/features/rpc.call.feature +23 -1
- package/features/rpc.call.recovery.feature +0 -21
- package/features/rpc.parked.feature +6 -6
- package/features/steps/call.js +90 -30
- package/features/steps/events.js +1 -1
- package/features/steps/parking.js +5 -5
- package/package.json +6 -4
- package/readme.md +37 -42
- package/source/connection.js +1 -4
- package/source/io.js +16 -10
- package/test/amqplib.mock.js +1 -1
- package/test/connection.test.js +1 -1
- package/test/{io.request.terms.test.js → io.call.terms.test.js} +31 -30
- package/test/io.call.test.js +0 -10
- package/test/io.request.test.js +13 -0
- package/test/memory.test.js +1 -1
- package/types/amqp.d.ts +1 -1
- package/types/channel.d.ts +1 -1
- package/types/io.d.ts +9 -8
- package/features/rpc.timeout.feature +0 -19
- package/source/batch.js +0 -62
- package/test/batch.test.js +0 -94
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# AMQP 1.0
|
|
2
|
+
|
|
3
|
+
**comq stays on AMQP 0-9-1 over [amqplib](https://github.com/amqp-node/amqplib).**
|
|
4
|
+
|
|
5
|
+
RabbitMQ 4 speaks AMQP 1.0 natively and keeps 0-9-1 as a first-class protocol; both are
|
|
6
|
+
supported out of the box and neither is on its way out. The choice is therefore comq's to make on
|
|
7
|
+
what each protocol and each client gives it, and the answer today is that 1.0 costs a rewrite of
|
|
8
|
+
the transport layer, changes comq's public API, breaks the wire between comq versions, and
|
|
9
|
+
returns nothing comq uses.
|
|
10
|
+
|
|
11
|
+
Two separate questions hide in "switch to the official client", and they have different answers.
|
|
12
|
+
The client is unusable for comq today. The protocol is usable and in places cleaner, and still
|
|
13
|
+
does not pay for itself.
|
|
14
|
+
|
|
15
|
+
Measured on 2026-09-20 against RabbitMQ 4.3.6, with `rabbitmq-amqp-js-client` 1.0.0, `rhea` 3.0.5
|
|
16
|
+
and `amqplib` 2.0.1.
|
|
17
|
+
|
|
18
|
+
## The client
|
|
19
|
+
|
|
20
|
+
[`rabbitmq-amqp-js-client`](https://github.com/coders51/rabbitmq-amqp-js-client) is listed among
|
|
21
|
+
RabbitMQ's AMQP 1.0 client libraries and is maintained by coders51 rather than by the RabbitMQ
|
|
22
|
+
team, unlike the Java, .NET, Go and Python clients. It is a thin wrapper over
|
|
23
|
+
[rhea](https://github.com/amqp/rhea), which does the protocol.
|
|
24
|
+
|
|
25
|
+
What it offers an application is a connection, a publisher, a consumer and a management link.
|
|
26
|
+
What comq needs from a client, it does not reach:
|
|
27
|
+
|
|
28
|
+
| comq | the client |
|
|
29
|
+
|---|---|
|
|
30
|
+
| 16 [diagnostic events](../readme.md#diagnostics) | connection, publisher and consumer state changes are unobservable ([#40](https://github.com/coders51/rabbitmq-amqp-js-client/issues/40)) |
|
|
31
|
+
| [`seal`](../readme.md#sealing) stops consuming and waits | `Consumer.close` is synchronous and returns nothing ([#34](https://github.com/coders51/rabbitmq-amqp-js-client/issues/34)) |
|
|
32
|
+
| topology recovered after a reconnect | the links come back, the topology does not ([#42](https://github.com/coders51/rabbitmq-amqp-js-client/issues/42)) |
|
|
33
|
+
| prefetch 300, per channel | rhea's default credit window of 1000, fixed |
|
|
34
|
+
| Requests and Replies are published unconfirmed | `publish` always awaits settlement |
|
|
35
|
+
| `amqp://user:pass@host/vhost`, heartbeat 15s | host, port, username, password |
|
|
36
|
+
|
|
37
|
+
Three defects it has today would each stop comq on its own:
|
|
38
|
+
|
|
39
|
+
- **A publish to a queue that does not exist never settles.** The broker refuses the attach with
|
|
40
|
+
`amqp:not-found`; `createPublisher` resolves regardless, and the `publish` promise stays
|
|
41
|
+
pending forever.
|
|
42
|
+
- **Attaching to another connection's exclusive queue kills the process.** `createConsumer`
|
|
43
|
+
resolves, the broker ends the session, and rhea emits an unhandled `error` on a container the
|
|
44
|
+
client keeps private. That is exactly what [`back`](../readme.md#addressed-requests) does on
|
|
45
|
+
every key already held.
|
|
46
|
+
- **Every body goes out as an `amqp-value` section**, Buffers included, so a 0-9-1 consumer
|
|
47
|
+
receives the section bytes in front of the payload and `type: 'amqp-1.0'` among the properties.
|
|
48
|
+
A process on 0-9-1 and a process on the client cannot exchange a message, which rules out a
|
|
49
|
+
rolling migration. rhea with an explicit `data_section` interoperates cleanly, so this is the
|
|
50
|
+
client's message model rather than the protocol.
|
|
51
|
+
|
|
52
|
+
Queue `arguments` are typed `Record<string, string>` while the broker requires integers for
|
|
53
|
+
`x-message-ttl` (`expected integer, got longstr`) and accepts the numbers the typing forbids. The
|
|
54
|
+
dependency on rhea is `github:amqp/rhea` — a git reference that resolves to `git+ssh` in a
|
|
55
|
+
lockfile and builds on install, which would land in comq's dependency tree and in every CI that
|
|
56
|
+
installs comq.
|
|
57
|
+
|
|
58
|
+
Adopted with those gaps filled locally, "the official client" becomes "rhea, plus a management
|
|
59
|
+
link we wrote" — which is the layer comq already is, over a client with fifteen years behind it.
|
|
60
|
+
|
|
61
|
+
## The protocol
|
|
62
|
+
|
|
63
|
+
Everything comq does is expressible over AMQP 1.0, and three mechanisms fit it better than 0-9-1
|
|
64
|
+
does:
|
|
65
|
+
|
|
66
|
+
- **An unroutable publish comes back as a `RELEASED` outcome on the send itself**, where 0-9-1
|
|
67
|
+
reports it asynchronously through `mandatory` and `basic.return` and comq has to correlate it.
|
|
68
|
+
- **The `ttl` header is honoured per message**, so a [`call`](../readme.md#timeout)'s expiration
|
|
69
|
+
carries unchanged.
|
|
70
|
+
- **Credit is granted per link**, which is what comq's [channel segregation](../readme.md#channels)
|
|
71
|
+
approximates with a prefetch count per channel.
|
|
72
|
+
|
|
73
|
+
The rest holds: an exclusive queue declared over the management link is refused to a second
|
|
74
|
+
connection (`400 cannot obtain exclusive access`), so `back` still holds a Key; numeric queue
|
|
75
|
+
arguments are accepted, so the [retry ladder](../readme.md#retries) and `comq.parked` are
|
|
76
|
+
unchanged. Streams with offsets and server-side filters are the one capability 0-9-1 cannot match,
|
|
77
|
+
and comq consumes no streams.
|
|
78
|
+
|
|
79
|
+
### What it costs
|
|
80
|
+
|
|
81
|
+
**comq's public API is shaped by 0-9-1.** `Properties` is in the consumer signature, `emit`,
|
|
82
|
+
`enqueue` and `route` take publish options, and `discard`, `retry` and `return` hand listeners a
|
|
83
|
+
raw amqplib message. Every one of those changes, and so does what reads them — `@toa.io/bindings.amqp`
|
|
84
|
+
reads `message.properties.correlationId` and `properties.headers`.
|
|
85
|
+
|
|
86
|
+
**Message properties land elsewhere.** A 0-9-1 header prefixed `x-` arrives over 1.0 as a message
|
|
87
|
+
annotation rather than an application property, beside the broker's own `x-exchange` and
|
|
88
|
+
`x-routing-key`; `replyTo: 'rq'` arrives as `/queues/rq`. comq's [`x-comq-*` headers](../docs/headers.md)
|
|
89
|
+
move with them.
|
|
90
|
+
|
|
91
|
+
**The management UI and API stop reporting message rates** for AMQP 1.0 connections; queue depth
|
|
92
|
+
survives, `message_stats` is null. Frame-intercepting plugins and transactions are 0-9-1 only,
|
|
93
|
+
and comq uses neither.
|
|
94
|
+
|
|
95
|
+
**The work is the transport layer and the unit suite.** `source/connection.js` and
|
|
96
|
+
`source/channel.js` are 1,256 lines written against amqplib's channel API, `source/io.js` another
|
|
97
|
+
797 that speak its vocabulary, and `test/` is built on `test/amqplib.mock.js` — some 6,400 lines
|
|
98
|
+
that go with the thing they mock. The 88 scenarios in `features/` are behaviour-level and would
|
|
99
|
+
carry over, which makes them the whole safety net for the change.
|
|
100
|
+
|
|
101
|
+
### What it does not buy
|
|
102
|
+
|
|
103
|
+
Throughput is level and the latency comq optimises for is worse. Same broker over loopback,
|
|
104
|
+
20,000 messages of ~180 bytes, three runs, `TCP_NODELAY` on both:
|
|
105
|
+
|
|
106
|
+
| | amqplib | rhea |
|
|
107
|
+
|---|---|---|
|
|
108
|
+
| publish, awaiting confirmation or settlement | 77–88k msg/s | 79–103k msg/s |
|
|
109
|
+
| consume and acknowledge | 121–143k msg/s | 110–128k msg/s |
|
|
110
|
+
|
|
111
|
+
Sequential request and reply, 2,000 round trips, one in flight:
|
|
112
|
+
|
|
113
|
+
| | p50 | p99 |
|
|
114
|
+
|---|---|---|
|
|
115
|
+
| amqplib | 0.20 ms | 0.36–0.41 ms |
|
|
116
|
+
| rhea | 0.26–0.29 ms | 0.51–0.57 ms |
|
|
117
|
+
|
|
118
|
+
rhea enables `TCP_NODELAY` when a receiver is attached and leaves it to the `tcp_no_delay` option
|
|
119
|
+
otherwise, so a connection that only publishes is left to Nagle — the same delay comq took out of
|
|
120
|
+
amqplib in 0.20.1, and worth 22 ms per round trip when it is missed.
|
|
121
|
+
|
|
122
|
+
## What would reverse this
|
|
123
|
+
|
|
124
|
+
- comq needs RabbitMQ streams with server-side filtering.
|
|
125
|
+
- The client exposes connection and link state, a consumer close that waits, a credit setting and
|
|
126
|
+
a vhost — the four things its own issues already name.
|
|
127
|
+
- RabbitMQ announces an end for AMQP 0-9-1.
|
|
@@ -65,7 +65,29 @@ Feature: Calls under a key
|
|
|
65
65
|
Scenario: A caller stops waiting for a holder that crashed
|
|
66
66
|
Given a holder never answering `void` under the `a` key
|
|
67
67
|
When the consumer calls `void` under the `a` key with a 1000ms timeout
|
|
68
|
-
And
|
|
68
|
+
And the silent holder has taken the call
|
|
69
69
|
And the silent holder crashes
|
|
70
70
|
Then the consumer stops waiting
|
|
71
71
|
And a call to `void` under the `a` key is refused within 10 seconds
|
|
72
|
+
|
|
73
|
+
Scenario: A caller stops waiting at its timeout
|
|
74
|
+
Given a holder on another connection answering `slow` under the `a` key in 1000ms
|
|
75
|
+
When the consumer calls `slow` under the `a` key with a 200ms timeout
|
|
76
|
+
Then the consumer stops waiting
|
|
77
|
+
|
|
78
|
+
Scenario: A call its holder has not taken in time is never processed
|
|
79
|
+
Given a holder not taking `later` under the `a` key
|
|
80
|
+
When the consumer calls `later` under the `a` key with a 200ms timeout
|
|
81
|
+
Then the consumer stops waiting
|
|
82
|
+
When after 300ms
|
|
83
|
+
And the idle holder starts taking
|
|
84
|
+
Then the idle holder takes nothing
|
|
85
|
+
|
|
86
|
+
Scenario: A call abandoned by its signal stays in its queue
|
|
87
|
+
Given a holder not taking `stray` under the `a` key
|
|
88
|
+
When the consumer calls `stray` under the `a` key with a signal
|
|
89
|
+
And after 100ms
|
|
90
|
+
And the signal aborts
|
|
91
|
+
Then the consumer stops waiting for the aborted call
|
|
92
|
+
When the idle holder starts taking
|
|
93
|
+
Then the idle holder takes the call
|
|
@@ -39,13 +39,6 @@ Feature: Calls under a key across failures
|
|
|
39
39
|
And the broker is up
|
|
40
40
|
Then the call ends
|
|
41
41
|
|
|
42
|
-
Scenario: A caller stops waiting for a request at its timeout while the broker is down
|
|
43
|
-
Given an active connection to the broker
|
|
44
|
-
And a producer replying `echo` queue
|
|
45
|
-
When the broker has crashed
|
|
46
|
-
And the consumer sends a request to the `echo` queue with a 1000ms timeout
|
|
47
|
-
Then the consumer stops waiting within 5 seconds
|
|
48
|
-
|
|
49
42
|
Scenario: A caller stops waiting for a call at its timeout while the broker is down
|
|
50
43
|
Given an active connection to the broker
|
|
51
44
|
And a holder on another connection answering `echo` under the `a` key
|
|
@@ -53,20 +46,6 @@ Feature: Calls under a key across failures
|
|
|
53
46
|
And the consumer calls `echo` under the `a` key with a 1000ms timeout
|
|
54
47
|
Then the consumer stops waiting within 5 seconds
|
|
55
48
|
|
|
56
|
-
Scenario: A request re-sent after the broker restarts expires at its caller's deadline
|
|
57
|
-
|
|
58
|
-
The request is lost with the broker and sent again once the connection is restored, carrying
|
|
59
|
-
the time its caller has left.
|
|
60
|
-
|
|
61
|
-
Given an active connection to the broker
|
|
62
|
-
When the consumer sends a request to the `resent` queue with a 30000ms timeout
|
|
63
|
-
And after 200ms
|
|
64
|
-
And the broker has crashed
|
|
65
|
-
And the broker is up
|
|
66
|
-
Then the consumer stops waiting within 40 seconds
|
|
67
|
-
When a producer counting requests to the `resent` queue
|
|
68
|
-
Then the producer receives nothing
|
|
69
|
-
|
|
70
49
|
Scenario: A call reaches its holder while one of the brokers is down
|
|
71
50
|
Given an active sharded connection
|
|
72
51
|
And a holder on another connection answering `echo` under the `a` key
|
|
@@ -5,13 +5,13 @@ Feature: Parked requests
|
|
|
5
5
|
|
|
6
6
|
Scenario: A parked request can still be answered
|
|
7
7
|
|
|
8
|
-
A caller
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
8
|
+
A caller is certain of an eventual answer for as long as it is alive, and parking
|
|
9
|
+
is what keeps that true when no consumer can produce one: the request is kept, along
|
|
10
|
+
with who was asking. Somebody else can answer it, and the caller — still waiting,
|
|
11
|
+
still holding an exclusive reply queue — receives it.
|
|
12
12
|
|
|
13
|
-
|
|
14
|
-
|
|
13
|
+
This is why a request has no timeout and cannot be withdrawn: either would leave the
|
|
14
|
+
parked message addressed to a caller that is no longer listening.
|
|
15
15
|
|
|
16
16
|
Given a producer failing every request to the `unanswerable` queue
|
|
17
17
|
When a request is sent to the `unanswerable` queue
|
package/features/steps/call.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
'use strict'
|
|
2
2
|
|
|
3
3
|
const assert = require('node:assert')
|
|
4
|
-
const amqplib = require('amqplib')
|
|
4
|
+
const amqplib = require('@toa.io/amqplib')
|
|
5
5
|
const { Given, When, Then, After } = require('@cucumber/cucumber')
|
|
6
6
|
|
|
7
7
|
const { connect, Unroutable } = require('../../')
|
|
@@ -84,11 +84,76 @@ Given('a holder never answering {token} under the {token} key',
|
|
|
84
84
|
await channel.assertExchange(exchange, 'direct', { durable: true })
|
|
85
85
|
await channel.assertQueue(queue, { exclusive: true })
|
|
86
86
|
await channel.bindQueue(queue, exchange, key)
|
|
87
|
-
await channel.consume(queue, () =>
|
|
87
|
+
await channel.consume(queue, () => (this.handed = true))
|
|
88
88
|
|
|
89
89
|
this.silent = connection
|
|
90
90
|
})
|
|
91
91
|
|
|
92
|
+
Then('the silent holder has taken the call',
|
|
93
|
+
/**
|
|
94
|
+
* The publication is awaited nowhere, and what the crash has to find already queued is what
|
|
95
|
+
* the holder has been handed.
|
|
96
|
+
*
|
|
97
|
+
* @this {comq.features.Context}
|
|
98
|
+
*/
|
|
99
|
+
async function () {
|
|
100
|
+
assert.equal(await until(() => this.handed === true, 10_000), true,
|
|
101
|
+
'The silent holder was handed nothing')
|
|
102
|
+
})
|
|
103
|
+
|
|
104
|
+
Given('a holder not taking {token} under the {token} key',
|
|
105
|
+
/**
|
|
106
|
+
* Holds the key as comq does, and takes nothing until told to, so that a call waits in its
|
|
107
|
+
* queue. Each scenario holds a key of its own: the broker releases an exclusive queue once it
|
|
108
|
+
* has noticed the connection close, and a scenario that declared it again before then would be
|
|
109
|
+
* refused it.
|
|
110
|
+
*
|
|
111
|
+
* @param {string} exchange
|
|
112
|
+
* @param {string} key
|
|
113
|
+
* @this {comq.features.Context}
|
|
114
|
+
*/
|
|
115
|
+
async function (exchange, key) {
|
|
116
|
+
const connection = await amqplib.connect(url(0))
|
|
117
|
+
const channel = await connection.createChannel()
|
|
118
|
+
const queue = exchange + '.' + key
|
|
119
|
+
|
|
120
|
+
await channel.assertExchange(exchange, 'direct', { durable: true })
|
|
121
|
+
await channel.assertQueue(queue, { exclusive: true })
|
|
122
|
+
await channel.bindQueue(queue, exchange, key)
|
|
123
|
+
|
|
124
|
+
this.idle = { connection, channel, queue, taken: 0 }
|
|
125
|
+
})
|
|
126
|
+
|
|
127
|
+
When('the idle holder starts taking',
|
|
128
|
+
/**
|
|
129
|
+
* @this {comq.features.Context}
|
|
130
|
+
*/
|
|
131
|
+
async function () {
|
|
132
|
+
const idle = this.idle
|
|
133
|
+
|
|
134
|
+
await idle.channel.consume(idle.queue, () => idle.taken++, { noAck: true })
|
|
135
|
+
})
|
|
136
|
+
|
|
137
|
+
When('the consumer calls {token} under the {token} key with a signal',
|
|
138
|
+
/**
|
|
139
|
+
* @param {string} exchange
|
|
140
|
+
* @param {string} key
|
|
141
|
+
* @this {comq.features.Context}
|
|
142
|
+
*/
|
|
143
|
+
async function (exchange, key) {
|
|
144
|
+
this.controller = new AbortController()
|
|
145
|
+
|
|
146
|
+
call(this, this.io.call(exchange, key, { key }, { signal: this.controller.signal }))
|
|
147
|
+
})
|
|
148
|
+
|
|
149
|
+
When('the signal aborts',
|
|
150
|
+
/**
|
|
151
|
+
* @this {comq.features.Context}
|
|
152
|
+
*/
|
|
153
|
+
function () {
|
|
154
|
+
this.controller.abort()
|
|
155
|
+
})
|
|
156
|
+
|
|
92
157
|
When('the silent holder crashes',
|
|
93
158
|
/**
|
|
94
159
|
* Its socket is destroyed without a word, as a process that dies leaves it.
|
|
@@ -149,16 +214,6 @@ When('the consumer calls {token} under the {token} key {number} times',
|
|
|
149
214
|
this.replies = await Promise.all(calls)
|
|
150
215
|
})
|
|
151
216
|
|
|
152
|
-
When('the consumer sends a request to the {token} queue with a {number}ms timeout',
|
|
153
|
-
/**
|
|
154
|
-
* @param {string} queue
|
|
155
|
-
* @param {number} ms
|
|
156
|
-
* @this {comq.features.Context}
|
|
157
|
-
*/
|
|
158
|
-
async function (queue, ms) {
|
|
159
|
-
call(this, this.io.request(queue, { queue }, { timeout: ms }))
|
|
160
|
-
})
|
|
161
|
-
|
|
162
217
|
When('the holder is sealed',
|
|
163
218
|
/**
|
|
164
219
|
* @this {comq.features.Context}
|
|
@@ -192,19 +247,32 @@ When('the first holder disconnects',
|
|
|
192
247
|
await this.holder.close()
|
|
193
248
|
})
|
|
194
249
|
|
|
195
|
-
|
|
250
|
+
Then('the idle holder takes nothing',
|
|
196
251
|
/**
|
|
197
|
-
* @param {string} queue
|
|
198
252
|
* @this {comq.features.Context}
|
|
199
253
|
*/
|
|
200
|
-
async function (
|
|
201
|
-
|
|
254
|
+
async function () {
|
|
255
|
+
await timeout(500)
|
|
202
256
|
|
|
203
|
-
|
|
204
|
-
|
|
257
|
+
assert.equal(this.idle.taken, 0, 'An expired call was taken')
|
|
258
|
+
})
|
|
205
259
|
|
|
206
|
-
|
|
207
|
-
|
|
260
|
+
Then('the idle holder takes the call',
|
|
261
|
+
/**
|
|
262
|
+
* @this {comq.features.Context}
|
|
263
|
+
*/
|
|
264
|
+
async function () {
|
|
265
|
+
await until(() => this.idle.taken > 0, 5000)
|
|
266
|
+
|
|
267
|
+
assert.equal(this.idle.taken, 1)
|
|
268
|
+
})
|
|
269
|
+
|
|
270
|
+
Then('the consumer stops waiting for the aborted call',
|
|
271
|
+
/**
|
|
272
|
+
* @this {comq.features.Context}
|
|
273
|
+
*/
|
|
274
|
+
async function () {
|
|
275
|
+
await assert.rejects(this.reply, (error) => error.name === 'AbortError')
|
|
208
276
|
})
|
|
209
277
|
|
|
210
278
|
Then('the call is refused as unroutable',
|
|
@@ -373,16 +441,6 @@ Then('every call is answered',
|
|
|
373
441
|
assert.deepEqual(this.replies, this.replies.map((_, n) => n))
|
|
374
442
|
})
|
|
375
443
|
|
|
376
|
-
Then('the producer receives nothing',
|
|
377
|
-
/**
|
|
378
|
-
* @this {comq.features.Context}
|
|
379
|
-
*/
|
|
380
|
-
async function () {
|
|
381
|
-
await timeout(500)
|
|
382
|
-
|
|
383
|
-
assert.equal(this.produced, 0, 'An expired request was processed')
|
|
384
|
-
})
|
|
385
|
-
|
|
386
444
|
After(
|
|
387
445
|
/**
|
|
388
446
|
* @this {comq.features.Context}
|
|
@@ -390,9 +448,11 @@ After(
|
|
|
390
448
|
async function () {
|
|
391
449
|
await Promise.all((this.holders ?? []).map((io) => io.close()))
|
|
392
450
|
await this.silent?.close().catch(() => undefined)
|
|
451
|
+
await this.idle?.connection.close().catch(() => undefined)
|
|
393
452
|
|
|
394
453
|
this.holders = []
|
|
395
454
|
this.silent = undefined
|
|
455
|
+
this.idle = undefined
|
|
396
456
|
})
|
|
397
457
|
|
|
398
458
|
/**
|
package/features/steps/events.js
CHANGED
|
@@ -225,7 +225,7 @@ async function consumed (group) {
|
|
|
225
225
|
/**
|
|
226
226
|
* @param {string} exchange
|
|
227
227
|
* @param {any} message
|
|
228
|
-
* @param {import('amqplib').Options.Publish} [properties]
|
|
228
|
+
* @param {import('@toa.io/amqplib').Options.Publish} [properties]
|
|
229
229
|
* @return {Promise<void>}
|
|
230
230
|
* @this {comq.features.Context}
|
|
231
231
|
*/
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
'use strict'
|
|
2
2
|
|
|
3
3
|
const assert = require('node:assert')
|
|
4
|
-
const amqplib = require('amqplib')
|
|
4
|
+
const amqplib = require('@toa.io/amqplib')
|
|
5
5
|
const { Given, When, Then } = require('@cucumber/cucumber')
|
|
6
6
|
|
|
7
7
|
const { getAddress, USER, PASSWORD } = require('./brokers')
|
|
@@ -58,7 +58,7 @@ Then('a message parked from the {token} exchange is in the parking queue',
|
|
|
58
58
|
* Runs `read` against a connection of its own, so that what it does to the parking
|
|
59
59
|
* queue is not what comq is doing to it.
|
|
60
60
|
*
|
|
61
|
-
* @param {(channel: import('amqplib').Channel) => Promise<void>} read
|
|
61
|
+
* @param {(channel: import('@toa.io/amqplib').Channel) => Promise<void>} read
|
|
62
62
|
*/
|
|
63
63
|
async function reading (read) {
|
|
64
64
|
const connection = await amqplib.connect(`amqp://${USER}:${PASSWORD}@${getAddress(0)}`)
|
|
@@ -77,9 +77,9 @@ async function reading (read) {
|
|
|
77
77
|
* every scenario before this one left in it, so everything that does not match is given
|
|
78
78
|
* back; the match is left unacknowledged, for the caller to take or to return.
|
|
79
79
|
*
|
|
80
|
-
* @param {import('amqplib').Channel} channel
|
|
81
|
-
* @param {(message: import('amqplib').GetMessage) => boolean} matches
|
|
82
|
-
* @returns {Promise<import('amqplib').GetMessage | undefined>}
|
|
80
|
+
* @param {import('@toa.io/amqplib').Channel} channel
|
|
81
|
+
* @param {(message: import('@toa.io/amqplib').GetMessage) => boolean} matches
|
|
82
|
+
* @returns {Promise<import('@toa.io/amqplib').GetMessage | undefined>}
|
|
83
83
|
*/
|
|
84
84
|
async function pick (channel, matches) {
|
|
85
85
|
const deadline = Date.now() + DEADLINE
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "comq",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.24.0",
|
|
4
4
|
"description": "ComQ",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Artem Gurtovoi",
|
|
@@ -34,7 +34,6 @@
|
|
|
34
34
|
"@semantic-release/npm": "^13.1.2",
|
|
35
35
|
"@semantic-release/release-notes-generator": "^14.1.0",
|
|
36
36
|
"@testcontainers/rabbitmq": "^12.1.0",
|
|
37
|
-
"@types/amqplib": "^0.10.8",
|
|
38
37
|
"@types/jest": "^30.0.0",
|
|
39
38
|
"conventional-changelog-conventionalcommits": "^8.0.0",
|
|
40
39
|
"fast-glob": "^3.3.3",
|
|
@@ -49,13 +48,16 @@
|
|
|
49
48
|
"testcontainers": "^12.0.4"
|
|
50
49
|
},
|
|
51
50
|
"dependencies": {
|
|
52
|
-
"amqplib": "^
|
|
51
|
+
"@toa.io/amqplib": "^0.1.0",
|
|
53
52
|
"promex": "^0.1.1",
|
|
54
53
|
"reretry": "^0.1.0"
|
|
55
54
|
},
|
|
55
|
+
"engines": {
|
|
56
|
+
"node": ">=24.0.0"
|
|
57
|
+
},
|
|
56
58
|
"scripts": {
|
|
57
59
|
"prepare": "husky",
|
|
58
|
-
"test": "standard && jest",
|
|
60
|
+
"test": "standard && NODE_OPTIONS=--experimental-vm-modules jest",
|
|
59
61
|
"features": "cucumber-js --fail-fast -t 'not @manual'",
|
|
60
62
|
"features:fast": "cucumber-js --fail-fast -t 'not @manual and not @heavy'",
|
|
61
63
|
"benchmark": "node benchmarks/run.js",
|
package/readme.md
CHANGED
|
@@ -6,8 +6,8 @@ for distributed, eventually consistent systems running on Node.js.
|
|
|
6
6
|
## Features
|
|
7
7
|
|
|
8
8
|
- [Dynamic topology](#topology)
|
|
9
|
-
- [Request](#request)-[reply](#reply) (RPC)
|
|
10
|
-
- [Addressed requests](#addressed-requests) to the one connection holding a Key
|
|
9
|
+
- [Request](#request)-[reply](#reply) (RPC)
|
|
10
|
+
- [Addressed requests](#addressed-requests) to the one connection holding a Key, with a [timeout](#timeout)
|
|
11
11
|
- Events ([pub](#emission)/[sub](#consumption)), fanned out or [routed](#routing)
|
|
12
12
|
- [Tasks](#tasks)
|
|
13
13
|
- [Pipelines](#pipelines)
|
|
@@ -107,51 +107,22 @@ await io.reply('add_numbers', ({ a, b }) => (a + b))
|
|
|
107
107
|
|
|
108
108
|
## Request
|
|
109
109
|
|
|
110
|
-
`async IO.request(queue: string, payload: any,
|
|
110
|
+
`async IO.request(queue: string, payload: any, encoding?: string): any`
|
|
111
111
|
|
|
112
112
|
Send encoded Request message with `replyTo` and `correlationId` properties set and
|
|
113
|
-
return decoded Reply content. The promise stays pending until the Reply arrives,
|
|
114
|
-
|
|
113
|
+
return decoded Reply content. The promise stays pending until the Reply arrives, however long
|
|
114
|
+
that takes: a Request has no timeout and cannot be withdrawn. One no Producer can answer is
|
|
115
|
+
[parked](#parked-messages), and can still be answered while its caller waits.
|
|
115
116
|
|
|
116
117
|
On the initial call, the queue for Requests is asserted, and so is the one this `IO` receives
|
|
117
118
|
every Reply on.
|
|
118
119
|
|
|
119
|
-
`options` is the encoding, or an object:
|
|
120
|
-
|
|
121
|
-
| Option | Type | Default |
|
|
122
|
-
|------------|---------------|--------------------|
|
|
123
|
-
| `encoding` | `string` | `application/json` |
|
|
124
|
-
| `timeout` | `number`, ms | none |
|
|
125
|
-
| `signal` | `AbortSignal` | none |
|
|
126
|
-
|
|
127
120
|
### Example
|
|
128
121
|
|
|
129
122
|
```javascript
|
|
130
123
|
const sum = await io.request('add_numbers', { a: 1, b: 2 })
|
|
131
124
|
```
|
|
132
125
|
|
|
133
|
-
### Timeout
|
|
134
|
-
|
|
135
|
-
A Request with a `timeout` rejects once it passes, with the `TimeoutError` of
|
|
136
|
-
[`AbortSignal.timeout`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal/timeout_static).
|
|
137
|
-
It is published with that much
|
|
138
|
-
[expiration](https://www.rabbitmq.com/docs/ttl#per-message-ttl-in-publishers), so a Request no
|
|
139
|
-
Producer has taken by then is dropped by the broker and never processed. A Request a Producer has
|
|
140
|
-
already taken is processed to the end, and its Reply is discarded. A Request re-sent after a lost
|
|
141
|
-
connection carries the time it has left.
|
|
142
|
-
|
|
143
|
-
A `signal` rejects the Request with its reason once aborted, within the `timeout` where both are
|
|
144
|
-
given. It ends the wait and leaves the Request where it is: in its queue until the `timeout`
|
|
145
|
-
passes, or, without a `timeout`, until a Producer takes it. A Request abandoned by its `signal` may
|
|
146
|
-
therefore still be processed.
|
|
147
|
-
|
|
148
|
-
A Request that failed and waits for its [next attempt](#retries) loses its expiration on the way
|
|
149
|
-
back to its queue, so it may be processed after its caller has stopped waiting.
|
|
150
|
-
|
|
151
|
-
```javascript
|
|
152
|
-
const sum = await io.request('add_numbers', { a: 1, b: 2 }, { timeout: 5000 })
|
|
153
|
-
```
|
|
154
|
-
|
|
155
126
|
## Consumption
|
|
156
127
|
|
|
157
128
|
`async IO.consume(exchange: string, group?: string, consumer): void`
|
|
@@ -231,7 +202,7 @@ await io.route('records', 'store.customers', { id: 2 }) // not delivered to the
|
|
|
231
202
|
|
|
232
203
|
`async IO.back(exchange: string, key: string, producer): void`
|
|
233
204
|
|
|
234
|
-
`async IO.call(exchange: string, key: string, payload: any, options?: string |
|
|
205
|
+
`async IO.call(exchange: string, key: string, payload: any, options?: string | CallOptions): any`
|
|
235
206
|
|
|
236
207
|
A Request to the one connection holding a Key, where a [Request](#request) goes to whichever
|
|
237
208
|
Producer takes it first.
|
|
@@ -245,8 +216,15 @@ is let go when its connection closes, however it closes — once the broker has
|
|
|
245
216
|
connection that went without a word. `back` returns once a broker holds the Key. The rest of the
|
|
246
217
|
connection works throughout, and a connection that is restored claims its Keys again the same way.
|
|
247
218
|
|
|
248
|
-
`call` publishes the encoded Request to the exchange under the `key` and returns the decoded Reply
|
|
249
|
-
|
|
219
|
+
`call` publishes the encoded Request to the exchange under the `key` and returns the decoded Reply.
|
|
220
|
+
|
|
221
|
+
`options` is the encoding, or an object:
|
|
222
|
+
|
|
223
|
+
| Option | Type | Default |
|
|
224
|
+
|------------|---------------|--------------------|
|
|
225
|
+
| `encoding` | `string` | `application/json` |
|
|
226
|
+
| `timeout` | `number`, ms | none |
|
|
227
|
+
| `signal` | `AbortSignal` | none |
|
|
250
228
|
|
|
251
229
|
A call ends in one of three ways:
|
|
252
230
|
|
|
@@ -272,6 +250,24 @@ same Key can therefore hold it on different shards and both answer calls, and `t
|
|
|
272
250
|
so. A call returned by one shard is published on the next, and refused once every shard has
|
|
273
251
|
returned it.
|
|
274
252
|
|
|
253
|
+
### Timeout
|
|
254
|
+
|
|
255
|
+
A call with a `timeout` rejects once it passes, with the `TimeoutError` of
|
|
256
|
+
[`AbortSignal.timeout`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal/timeout_static).
|
|
257
|
+
It is published with that much
|
|
258
|
+
[expiration](https://www.rabbitmq.com/docs/ttl#per-message-ttl-in-publishers), so a call its holder
|
|
259
|
+
has not taken by then is dropped by the broker and never processed. A call the holder has already
|
|
260
|
+
taken is processed to the end, and its Reply is discarded. A call re-sent after a lost connection
|
|
261
|
+
carries the time it has left.
|
|
262
|
+
|
|
263
|
+
A `signal` rejects the call with its reason once aborted, within the `timeout` where both are
|
|
264
|
+
given. It ends the wait and leaves the call where it is: in its queue until the `timeout` passes,
|
|
265
|
+
or, without a `timeout`, until its holder takes it. A call abandoned by its `signal` may therefore
|
|
266
|
+
still be processed.
|
|
267
|
+
|
|
268
|
+
A call that failed and waits for its [next attempt](#retries) loses its expiration on the way back
|
|
269
|
+
to its queue, so it may be processed after its caller has stopped waiting.
|
|
270
|
+
|
|
275
271
|
### Example
|
|
276
272
|
|
|
277
273
|
```javascript
|
|
@@ -661,10 +657,9 @@ The queue grows until somebody drains it, which is deliberate — the alternativ
|
|
|
661
657
|
evidence. Nothing comq does removes it or what it holds. Alert on [`discard`](#diagnostics), and
|
|
662
658
|
do not delete `comq.retry.*` or `comq.parked` on a running system.
|
|
663
659
|
|
|
664
|
-
> **A parked Request is never answered.** A Consumer awaiting its Reply waits
|
|
665
|
-
> [timeout](#timeout)
|
|
666
|
-
>
|
|
667
|
-
> it — and it keeps `replyTo` and `correlationId`, so a Reply can still be produced from it by
|
|
660
|
+
> **A parked Request is never answered.** A Consumer awaiting its Reply waits indefinitely — a
|
|
661
|
+
> call until its [timeout](#timeout) — and with a limited prefetch that can deadlock it. Parking
|
|
662
|
+
> keeps the Request rather than deleting it — and it keeps `replyTo` and `correlationId`, so a Reply can still be produced from it by
|
|
668
663
|
> hand while the caller is alive — but comq itself sends no Reply and reports no error to the
|
|
669
664
|
> caller.
|
|
670
665
|
|
package/source/connection.js
CHANGED
|
@@ -1,12 +1,11 @@
|
|
|
1
1
|
'use strict'
|
|
2
2
|
|
|
3
3
|
const { setTimeout: delay } = require('node:timers/promises')
|
|
4
|
-
const amqp = require('amqplib')
|
|
4
|
+
const amqp = require('@toa.io/amqplib')
|
|
5
5
|
const { Promex } = require('promex')
|
|
6
6
|
const { retry } = require('reretry')
|
|
7
7
|
|
|
8
8
|
const { failsafe } = require('./attributes')
|
|
9
|
-
const { batch } = require('./batch')
|
|
10
9
|
const presets = require('./topology')
|
|
11
10
|
const channels = require('./channel')
|
|
12
11
|
const emitter = require('./emitter')
|
|
@@ -148,8 +147,6 @@ class Connection {
|
|
|
148
147
|
connection.on('close', (error) => this.#close(connection, error))
|
|
149
148
|
this.#connection = connection
|
|
150
149
|
|
|
151
|
-
batch(connection.connection?.stream)
|
|
152
|
-
|
|
153
150
|
this.#armWatchdog(connection)
|
|
154
151
|
this.#diagnostics.emit('open')
|
|
155
152
|
|
package/source/io.js
CHANGED
|
@@ -84,16 +84,21 @@ class IO {
|
|
|
84
84
|
})
|
|
85
85
|
|
|
86
86
|
/**
|
|
87
|
+
* Waits for the Reply for as long as it takes: a Request is answered eventually, and one no
|
|
88
|
+
* Producer can answer is parked rather than abandoned.
|
|
89
|
+
*
|
|
87
90
|
* @param {string} queue
|
|
88
91
|
* @param {any | Readable} payload
|
|
89
|
-
* @param {comq.Encoding
|
|
92
|
+
* @param {comq.Encoding} [encoding]
|
|
90
93
|
* @returns {Promise<any | Readable>}
|
|
91
94
|
*/
|
|
92
|
-
request (queue, payload,
|
|
93
|
-
|
|
95
|
+
async request (queue, payload, encoding) {
|
|
96
|
+
// a caller of the version that took a timeout here would otherwise wait on in silence
|
|
97
|
+
if (typeof encoding === 'object' && encoding !== null) {
|
|
98
|
+
throw new TypeError('A Request takes an encoding and waits for its Reply')
|
|
99
|
+
}
|
|
94
100
|
|
|
95
|
-
|
|
96
|
-
return abortable(this.#request(queue, payload, settled), settled.signal)
|
|
101
|
+
return await this.#request(queue, payload, { encoding })
|
|
97
102
|
}
|
|
98
103
|
|
|
99
104
|
/**
|
|
@@ -103,12 +108,13 @@ class IO {
|
|
|
103
108
|
* @param {string} exchange
|
|
104
109
|
* @param {string} key
|
|
105
110
|
* @param {any} payload
|
|
106
|
-
* @param {comq.Encoding | comq.
|
|
111
|
+
* @param {comq.Encoding | comq.CallOptions} [options]
|
|
107
112
|
* @returns {Promise<any | Readable>}
|
|
108
113
|
*/
|
|
109
114
|
call (exchange, key, payload, options) {
|
|
110
115
|
const settled = terms(options)
|
|
111
116
|
|
|
117
|
+
// the channels a first call declares are waited for within the terms too
|
|
112
118
|
return abortable(this.#call(exchange, key, payload, settled), settled.signal)
|
|
113
119
|
}
|
|
114
120
|
|
|
@@ -129,8 +135,7 @@ class IO {
|
|
|
129
135
|
await this.#requests.held(exchange, exchange + '.' + key, key, consumer)
|
|
130
136
|
})
|
|
131
137
|
|
|
132
|
-
// failsafe is aimed to retransmit unanswered messages
|
|
133
|
-
// re-sent Request keeps the deadline of the first
|
|
138
|
+
// failsafe is aimed to retransmit unanswered messages
|
|
134
139
|
#request = lazy(this, [this.#createRequestReplyChannels, this.#consumeReplies],
|
|
135
140
|
failsafe(this, this.#recover,
|
|
136
141
|
/**
|
|
@@ -152,6 +157,7 @@ class IO {
|
|
|
152
157
|
(buffer, properties) => this.#requests.send(queue, buffer, properties))
|
|
153
158
|
}))
|
|
154
159
|
|
|
160
|
+
// the terms are taken once, so a re-sent call keeps the deadline of the first
|
|
155
161
|
#call = lazy(this, [this.#createRequestReplyChannels, this.#consumeReplies],
|
|
156
162
|
failsafe(this, this.#recover,
|
|
157
163
|
/**
|
|
@@ -708,10 +714,10 @@ const REPLY = 'comq.reply'
|
|
|
708
714
|
const CONTROL = 'control'
|
|
709
715
|
|
|
710
716
|
/**
|
|
711
|
-
* What a caller passed, settled once: a timeout becomes the moment the
|
|
717
|
+
* What a caller passed, settled once: a timeout becomes the moment the call expires, so that
|
|
712
718
|
* every attempt of it shares one deadline.
|
|
713
719
|
*
|
|
714
|
-
* @param {comq.Encoding | comq.
|
|
720
|
+
* @param {comq.Encoding | comq.CallOptions} [options]
|
|
715
721
|
* @returns {comq.Terms}
|
|
716
722
|
*/
|
|
717
723
|
function terms (options) {
|
package/test/amqplib.mock.js
CHANGED
|
@@ -58,7 +58,7 @@ class Connection extends EventEmitter {
|
|
|
58
58
|
|
|
59
59
|
const connect = async (url) => new Connection(url)
|
|
60
60
|
|
|
61
|
-
/** @type {jest.MockedObject<import('amqplib')>} */
|
|
61
|
+
/** @type {jest.MockedObject<import('@toa.io/amqplib')>} */
|
|
62
62
|
const amqplib = {
|
|
63
63
|
connect: jest.fn(connect)
|
|
64
64
|
}
|
package/test/connection.test.js
CHANGED
|
@@ -10,7 +10,7 @@ const { amqplib, connect } = require('./amqplib.mock')
|
|
|
10
10
|
const { channel: create } = require('./connection.mock')
|
|
11
11
|
const mock = { amqplib, channel: { create } }
|
|
12
12
|
|
|
13
|
-
jest.mock('amqplib', () => mock.amqplib)
|
|
13
|
+
jest.mock('@toa.io/amqplib', () => mock.amqplib)
|
|
14
14
|
jest.mock('../source/channel', () => mock.channel)
|
|
15
15
|
|
|
16
16
|
const presets = require('../source/topology')
|
|
@@ -21,7 +21,8 @@ let requests
|
|
|
21
21
|
/** @type {jest.MockedObject<comq.Channel>} */
|
|
22
22
|
let replies
|
|
23
23
|
|
|
24
|
-
const
|
|
24
|
+
const exchange = generate()
|
|
25
|
+
const key = generate()
|
|
25
26
|
const payload = { [generate()]: generate() }
|
|
26
27
|
|
|
27
28
|
beforeEach(async () => {
|
|
@@ -34,12 +35,12 @@ beforeEach(async () => {
|
|
|
34
35
|
// endregion
|
|
35
36
|
|
|
36
37
|
describe('timeout', () => {
|
|
37
|
-
it('should expire the
|
|
38
|
-
io.
|
|
38
|
+
it('should expire the call when the caller stops waiting', async () => {
|
|
39
|
+
io.call(exchange, key, payload, { timeout: 10000 }).catch(noop)
|
|
39
40
|
|
|
40
41
|
await initialized()
|
|
41
42
|
|
|
42
|
-
const expiration = Number(requests.
|
|
43
|
+
const expiration = Number(requests.route.mock.calls[0][3].expiration)
|
|
43
44
|
|
|
44
45
|
expect(expiration).toBeGreaterThan(9000)
|
|
45
46
|
expect(expiration).toBeLessThanOrEqual(10000)
|
|
@@ -48,29 +49,29 @@ describe('timeout', () => {
|
|
|
48
49
|
it('should keep the encoding', async () => {
|
|
49
50
|
const encoding = 'application/octet-stream'
|
|
50
51
|
|
|
51
|
-
io.
|
|
52
|
+
io.call(exchange, key, randomBytes(8), { encoding, timeout: 10000 }).catch(noop)
|
|
52
53
|
|
|
53
54
|
await initialized()
|
|
54
55
|
|
|
55
|
-
expect(requests.
|
|
56
|
+
expect(requests.route.mock.calls[0][3].contentType).toStrictEqual(encoding)
|
|
56
57
|
})
|
|
57
58
|
|
|
58
59
|
it('should stop waiting while the channels are being created', async () => {
|
|
59
60
|
connection.createChannel.mockImplementation(() => new Promise(noop))
|
|
60
61
|
|
|
61
|
-
const promise = io.
|
|
62
|
+
const promise = io.call(exchange, key, payload, { timeout: 20 })
|
|
62
63
|
|
|
63
64
|
await expect(promise).rejects.toMatchObject({ name: 'TimeoutError' })
|
|
64
65
|
})
|
|
65
66
|
|
|
66
67
|
it('should reject once it has passed', async () => {
|
|
67
|
-
const promise = io.
|
|
68
|
+
const promise = io.call(exchange, key, payload, { timeout: 20 })
|
|
68
69
|
|
|
69
70
|
await expect(promise).rejects.toMatchObject({ name: 'TimeoutError' })
|
|
70
71
|
})
|
|
71
72
|
|
|
72
73
|
it('should resolve with a reply that arrives in time', async () => {
|
|
73
|
-
const promise = io.
|
|
74
|
+
const promise = io.call(exchange, key, payload, { timeout: 10000 })
|
|
74
75
|
const content = randomBytes(8)
|
|
75
76
|
|
|
76
77
|
await initialized()
|
|
@@ -80,7 +81,7 @@ describe('timeout', () => {
|
|
|
80
81
|
})
|
|
81
82
|
|
|
82
83
|
it('should re-send with the time that is left', async () => {
|
|
83
|
-
io.
|
|
84
|
+
io.call(exchange, key, payload, { timeout: 10000 }).catch(noop)
|
|
84
85
|
|
|
85
86
|
await initialized()
|
|
86
87
|
await timeout(20)
|
|
@@ -89,10 +90,10 @@ describe('timeout', () => {
|
|
|
89
90
|
|
|
90
91
|
await immediate()
|
|
91
92
|
|
|
92
|
-
expect(requests.
|
|
93
|
+
expect(requests.route).toHaveBeenCalledTimes(2)
|
|
93
94
|
|
|
94
|
-
const first = Number(requests.
|
|
95
|
-
const second = Number(requests.
|
|
95
|
+
const first = Number(requests.route.mock.calls[0][3].expiration)
|
|
96
|
+
const second = Number(requests.route.mock.calls[1][3].expiration)
|
|
96
97
|
|
|
97
98
|
expect(second).toBeLessThan(first)
|
|
98
99
|
})
|
|
@@ -102,7 +103,7 @@ describe('signal', () => {
|
|
|
102
103
|
it('should reject with the reason it was aborted for', async () => {
|
|
103
104
|
const controller = new AbortController()
|
|
104
105
|
const reason = new Error(generate())
|
|
105
|
-
const promise = io.
|
|
106
|
+
const promise = io.call(exchange, key, payload, { signal: controller.signal })
|
|
106
107
|
|
|
107
108
|
await initialized()
|
|
108
109
|
|
|
@@ -111,14 +112,14 @@ describe('signal', () => {
|
|
|
111
112
|
await expect(promise).rejects.toBe(reason)
|
|
112
113
|
})
|
|
113
114
|
|
|
114
|
-
it('should leave the
|
|
115
|
+
it('should leave the call in its queue', async () => {
|
|
115
116
|
const controller = new AbortController()
|
|
116
117
|
|
|
117
|
-
io.
|
|
118
|
+
io.call(exchange, key, payload, { signal: controller.signal }).catch(noop)
|
|
118
119
|
|
|
119
120
|
await initialized()
|
|
120
121
|
|
|
121
|
-
expect(requests.
|
|
122
|
+
expect(requests.route.mock.calls[0][3].expiration).toBeUndefined()
|
|
122
123
|
})
|
|
123
124
|
|
|
124
125
|
it('should reject at once when aborted before the channels exist', async () => {
|
|
@@ -129,27 +130,27 @@ describe('signal', () => {
|
|
|
129
130
|
|
|
130
131
|
controller.abort(reason)
|
|
131
132
|
|
|
132
|
-
await expect(io.
|
|
133
|
+
await expect(io.call(exchange, key, payload, { signal: controller.signal })).rejects.toBe(reason)
|
|
133
134
|
})
|
|
134
135
|
|
|
135
136
|
it('should send nothing once aborted', async () => {
|
|
136
137
|
const controller = new AbortController()
|
|
137
138
|
|
|
138
|
-
// the channels exist before the aborted
|
|
139
|
-
io.
|
|
139
|
+
// the channels exist before the aborted call is made
|
|
140
|
+
io.call(exchange, key, payload).catch(noop)
|
|
140
141
|
|
|
141
142
|
await initialized()
|
|
142
143
|
|
|
143
144
|
controller.abort()
|
|
144
145
|
|
|
145
|
-
await expect(io.
|
|
146
|
+
await expect(io.call(exchange, key, payload, { signal: controller.signal })).rejects.toBeDefined()
|
|
146
147
|
|
|
147
|
-
expect(requests.
|
|
148
|
+
expect(requests.route).toHaveBeenCalledTimes(1)
|
|
148
149
|
})
|
|
149
150
|
|
|
150
151
|
it('should not re-send what it abandoned', async () => {
|
|
151
152
|
const controller = new AbortController()
|
|
152
|
-
const promise = io.
|
|
153
|
+
const promise = io.call(exchange, key, payload, { signal: controller.signal })
|
|
153
154
|
|
|
154
155
|
await initialized()
|
|
155
156
|
|
|
@@ -161,19 +162,19 @@ describe('signal', () => {
|
|
|
161
162
|
|
|
162
163
|
await immediate()
|
|
163
164
|
|
|
164
|
-
expect(requests.
|
|
165
|
+
expect(requests.route).toHaveBeenCalledTimes(1)
|
|
165
166
|
})
|
|
166
167
|
|
|
167
168
|
it('should stop waiting for a publication', async () => {
|
|
168
|
-
io.
|
|
169
|
+
io.call(exchange, key, payload).catch(noop)
|
|
169
170
|
|
|
170
171
|
await initialized()
|
|
171
172
|
|
|
172
|
-
requests.
|
|
173
|
+
requests.route.mockImplementationOnce(() => new Promise(noop))
|
|
173
174
|
|
|
174
175
|
const controller = new AbortController()
|
|
175
176
|
const reason = new Error(generate())
|
|
176
|
-
const promise = io.
|
|
177
|
+
const promise = io.call(exchange, key, payload, { signal: controller.signal })
|
|
177
178
|
|
|
178
179
|
await immediate()
|
|
179
180
|
|
|
@@ -184,7 +185,7 @@ describe('signal', () => {
|
|
|
184
185
|
|
|
185
186
|
it('should discard a reply that arrives afterwards', async () => {
|
|
186
187
|
const controller = new AbortController()
|
|
187
|
-
const promise = io.
|
|
188
|
+
const promise = io.call(exchange, key, payload, { signal: controller.signal })
|
|
188
189
|
|
|
189
190
|
await initialized()
|
|
190
191
|
|
|
@@ -197,7 +198,7 @@ describe('signal', () => {
|
|
|
197
198
|
|
|
198
199
|
it('should end the wait within the timeout', async () => {
|
|
199
200
|
const controller = new AbortController()
|
|
200
|
-
const promise = io.
|
|
201
|
+
const promise = io.call(exchange, key, payload, { signal: controller.signal, timeout: 20 })
|
|
201
202
|
|
|
202
203
|
await expect(promise).rejects.toMatchObject({ name: 'TimeoutError' })
|
|
203
204
|
})
|
|
@@ -216,7 +217,7 @@ async function initialized () {
|
|
|
216
217
|
* @param {Buffer} [content]
|
|
217
218
|
*/
|
|
218
219
|
async function answer (index, content = randomBytes(8)) {
|
|
219
|
-
const { correlationId } = requests.
|
|
220
|
+
const { correlationId } = requests.route.mock.calls[index][3]
|
|
220
221
|
const deliver = replies.consume.mock.calls[0][1]
|
|
221
222
|
|
|
222
223
|
await deliver({ content, properties: { correlationId } })
|
package/test/io.call.test.js
CHANGED
|
@@ -98,16 +98,6 @@ describe('call', () => {
|
|
|
98
98
|
})
|
|
99
99
|
})
|
|
100
100
|
|
|
101
|
-
describe('timeout', () => {
|
|
102
|
-
it('should stop waiting while the channels are being created', async () => {
|
|
103
|
-
connection.createChannel.mockImplementation(() => new Promise(() => undefined))
|
|
104
|
-
|
|
105
|
-
const promise = io.call(exchange, key, payload, { timeout: 20 })
|
|
106
|
-
|
|
107
|
-
await expect(promise).rejects.toMatchObject({ name: 'TimeoutError' })
|
|
108
|
-
})
|
|
109
|
-
})
|
|
110
|
-
|
|
111
101
|
describe('back', () => {
|
|
112
102
|
const producer = jest.fn(() => generate())
|
|
113
103
|
|
package/test/io.request.test.js
CHANGED
|
@@ -115,6 +115,19 @@ describe('send', () => {
|
|
|
115
115
|
expect(call[2]).toMatchObject({ contentType })
|
|
116
116
|
})
|
|
117
117
|
|
|
118
|
+
it('should not expire', async () => {
|
|
119
|
+
expect(call[2].expiration).toBeUndefined()
|
|
120
|
+
})
|
|
121
|
+
|
|
122
|
+
it('should refuse options other than the encoding', async () => {
|
|
123
|
+
requests.send.mockClear()
|
|
124
|
+
|
|
125
|
+
await expect(io.request(queue, payload, { timeout: 1000 })).rejects.toBeInstanceOf(TypeError)
|
|
126
|
+
await expect(io.request(queue, payload, { signal: new AbortController().signal })).rejects.toBeInstanceOf(TypeError)
|
|
127
|
+
|
|
128
|
+
expect(requests.send).not.toHaveBeenCalled()
|
|
129
|
+
})
|
|
130
|
+
|
|
118
131
|
it('should throw if encoding is not supported', async () => {
|
|
119
132
|
const encoding = /** @type {comq.Encoding} */ 'wtf/' + generate()
|
|
120
133
|
|
package/test/memory.test.js
CHANGED
|
@@ -12,7 +12,7 @@ const { timeout } = require('./helpers')
|
|
|
12
12
|
const mock = require('./amqplib.mock')
|
|
13
13
|
const { amqplib, connect } = mock
|
|
14
14
|
|
|
15
|
-
jest.mock('amqplib', () => mock.amqplib)
|
|
15
|
+
jest.mock('@toa.io/amqplib', () => mock.amqplib)
|
|
16
16
|
|
|
17
17
|
const { connect: open, assert } = require('../source')
|
|
18
18
|
const { SingletonConnection } = require('../source/singleton')
|
package/types/amqp.d.ts
CHANGED
package/types/channel.d.ts
CHANGED
package/types/io.d.ts
CHANGED
|
@@ -37,19 +37,19 @@ declare namespace comq {
|
|
|
37
37
|
signal?: AbortSignal
|
|
38
38
|
}
|
|
39
39
|
|
|
40
|
-
/** How long a caller waits for a
|
|
41
|
-
interface
|
|
40
|
+
/** How long a caller waits for the Reply to a call. */
|
|
41
|
+
interface CallOptions {
|
|
42
42
|
encoding?: _encoding.Encoding
|
|
43
43
|
|
|
44
44
|
/**
|
|
45
|
-
* Milliseconds to wait for the Reply. A
|
|
45
|
+
* Milliseconds to wait for the Reply. A call nobody has taken by then is dropped by the
|
|
46
46
|
* broker; one already taken may still be processed, and its Reply is discarded.
|
|
47
47
|
*/
|
|
48
48
|
timeout?: number
|
|
49
49
|
|
|
50
50
|
/**
|
|
51
|
-
* Ends the wait when aborted, within the timeout. The
|
|
52
|
-
* timeout passes, or, without one, until
|
|
51
|
+
* Ends the wait when aborted, within the timeout. The call stays in its queue until the
|
|
52
|
+
* timeout passes, or, without one, until its holder takes it.
|
|
53
53
|
*/
|
|
54
54
|
signal?: AbortSignal
|
|
55
55
|
}
|
|
@@ -64,15 +64,16 @@ declare namespace comq {
|
|
|
64
64
|
interface IO extends _diagnostics.Diagnosable {
|
|
65
65
|
reply(queue: string, produce: Producer): Promise<void>
|
|
66
66
|
|
|
67
|
-
|
|
67
|
+
/** Waits for the Reply for as long as it takes. */
|
|
68
|
+
request<Reply = any, Request = any>(queue: string, payload: Request, encoding?: _encoding.Encoding): Promise<Reply> | Promise<Readable>
|
|
68
69
|
|
|
69
|
-
request(queue: string, stream: Readable,
|
|
70
|
+
request(queue: string, stream: Readable, encoding?: _encoding.Encoding): Promise<Readable>
|
|
70
71
|
|
|
71
72
|
/**
|
|
72
73
|
* Sends a Request to whoever holds `key` on the routed `exchange`, through `back`. One that
|
|
73
74
|
* nobody holds is refused with `Unroutable`.
|
|
74
75
|
*/
|
|
75
|
-
call<Reply = any, Request = any>(exchange: string, key: string, payload: Request, options?: _encoding.Encoding |
|
|
76
|
+
call<Reply = any, Request = any>(exchange: string, key: string, payload: Request, options?: _encoding.Encoding | CallOptions): Promise<Reply> | Promise<Readable>
|
|
76
77
|
|
|
77
78
|
/**
|
|
78
79
|
* Answers the Requests `call` sends under `key`, from a queue this connection alone holds.
|
|
@@ -1,19 +0,0 @@
|
|
|
1
|
-
Feature: Request timeout
|
|
2
|
-
|
|
3
|
-
A caller may say how long it waits for a reply. A request nobody took by then is dropped by the
|
|
4
|
-
broker; one already taken is still processed, and its reply is discarded.
|
|
5
|
-
|
|
6
|
-
Background:
|
|
7
|
-
Given an active connection to the broker
|
|
8
|
-
|
|
9
|
-
Scenario: A caller stops waiting at its timeout
|
|
10
|
-
Given a producer replying `slow` queue in 1000ms
|
|
11
|
-
When the consumer sends a request to the `slow` queue with a 200ms timeout
|
|
12
|
-
Then the consumer stops waiting
|
|
13
|
-
|
|
14
|
-
Scenario: A request nobody took in time is never processed
|
|
15
|
-
When the consumer sends a request to the `later` queue with a 200ms timeout
|
|
16
|
-
Then the consumer stops waiting
|
|
17
|
-
When after 300ms
|
|
18
|
-
And a producer counting requests to the `later` queue
|
|
19
|
-
Then the producer receives nothing
|
package/source/batch.js
DELETED
|
@@ -1,62 +0,0 @@
|
|
|
1
|
-
'use strict'
|
|
2
|
-
|
|
3
|
-
/**
|
|
4
|
-
* Writes as one what a connection sends in one turn of the event loop.
|
|
5
|
-
*
|
|
6
|
-
* amqplib hands its frames to the socket one write at a time, and a request served is two of
|
|
7
|
-
* them: the reply, and the acknowledgement of the request it answers. With Nagle's algorithm off,
|
|
8
|
-
* which is what a socket sending without delay runs under, each is a syscall and a packet of its
|
|
9
|
-
* own. Corked, they are held in the socket's buffer and leave together, in one `writev`, at the
|
|
10
|
-
* end of the same turn: nothing waits for a timer, and nothing waits for a write that may never
|
|
11
|
-
* come.
|
|
12
|
-
*
|
|
13
|
-
* This belongs in amqplib, whose `Mux` writes the frames one at a time, and where corking that
|
|
14
|
-
* loop is six lines and needs no wrapping of a socket the library does not own. It is done here
|
|
15
|
-
* until amqplib carries it, and goes when it does.
|
|
16
|
-
*
|
|
17
|
-
* @param {import('node:net').Socket} socket
|
|
18
|
-
*/
|
|
19
|
-
function batch (socket) {
|
|
20
|
-
// whatever a connection is carried over is left as it is unless it is a stream that corks
|
|
21
|
-
if (!corkable(socket) || socket[BATCHED] === true) return
|
|
22
|
-
|
|
23
|
-
socket[BATCHED] = true
|
|
24
|
-
|
|
25
|
-
const write = socket.write.bind(socket)
|
|
26
|
-
|
|
27
|
-
const flush = () => {
|
|
28
|
-
socket[CORKED] = false
|
|
29
|
-
socket.uncork()
|
|
30
|
-
}
|
|
31
|
-
|
|
32
|
-
const cork = () => {
|
|
33
|
-
if (socket[CORKED] === true) return
|
|
34
|
-
|
|
35
|
-
socket[CORKED] = true
|
|
36
|
-
socket.cork()
|
|
37
|
-
|
|
38
|
-
process.nextTick(flush)
|
|
39
|
-
}
|
|
40
|
-
|
|
41
|
-
socket.write = (...args) => {
|
|
42
|
-
cork()
|
|
43
|
-
|
|
44
|
-
return write(...args)
|
|
45
|
-
}
|
|
46
|
-
}
|
|
47
|
-
|
|
48
|
-
/**
|
|
49
|
-
* @param {any} socket
|
|
50
|
-
* @returns {boolean}
|
|
51
|
-
*/
|
|
52
|
-
function corkable (socket) {
|
|
53
|
-
return socket !== null && socket !== undefined &&
|
|
54
|
-
typeof socket.write === 'function' &&
|
|
55
|
-
typeof socket.cork === 'function' &&
|
|
56
|
-
typeof socket.uncork === 'function'
|
|
57
|
-
}
|
|
58
|
-
|
|
59
|
-
const BATCHED = Symbol('comq.batched')
|
|
60
|
-
const CORKED = Symbol('comq.corked')
|
|
61
|
-
|
|
62
|
-
exports.batch = batch
|
package/test/batch.test.js
DELETED
|
@@ -1,94 +0,0 @@
|
|
|
1
|
-
'use strict'
|
|
2
|
-
|
|
3
|
-
const { Writable } = require('node:stream')
|
|
4
|
-
|
|
5
|
-
const { batch } = require('../source/batch')
|
|
6
|
-
|
|
7
|
-
/** A socket, as far as writing to it goes: what it was given, and how many times. */
|
|
8
|
-
function socket () {
|
|
9
|
-
const writes = []
|
|
10
|
-
|
|
11
|
-
return Object.assign(new Writable({
|
|
12
|
-
write (chunk, _encoding, callback) {
|
|
13
|
-
writes.push([chunk.toString()])
|
|
14
|
-
callback()
|
|
15
|
-
},
|
|
16
|
-
writev (chunks, callback) {
|
|
17
|
-
writes.push(chunks.map(({ chunk }) => chunk.toString()))
|
|
18
|
-
callback()
|
|
19
|
-
}
|
|
20
|
-
}), { writes })
|
|
21
|
-
}
|
|
22
|
-
|
|
23
|
-
const tick = () => new Promise((resolve) => process.nextTick(resolve))
|
|
24
|
-
|
|
25
|
-
it('should be', async () => {
|
|
26
|
-
expect(batch).toBeDefined()
|
|
27
|
-
})
|
|
28
|
-
|
|
29
|
-
it('should write what a turn wrote as one', async () => {
|
|
30
|
-
const stream = socket()
|
|
31
|
-
|
|
32
|
-
batch(stream)
|
|
33
|
-
|
|
34
|
-
stream.write('a')
|
|
35
|
-
stream.write('b')
|
|
36
|
-
|
|
37
|
-
await tick()
|
|
38
|
-
|
|
39
|
-
expect(stream.writes).toStrictEqual([['a', 'b']])
|
|
40
|
-
})
|
|
41
|
-
|
|
42
|
-
it('should write what a later turn wrote on its own', async () => {
|
|
43
|
-
const stream = socket()
|
|
44
|
-
|
|
45
|
-
batch(stream)
|
|
46
|
-
|
|
47
|
-
stream.write('a')
|
|
48
|
-
|
|
49
|
-
await tick()
|
|
50
|
-
|
|
51
|
-
stream.write('b')
|
|
52
|
-
|
|
53
|
-
await tick()
|
|
54
|
-
|
|
55
|
-
expect(stream.writes).toStrictEqual([['a'], ['b']])
|
|
56
|
-
})
|
|
57
|
-
|
|
58
|
-
it('should not hold a write past its turn', async () => {
|
|
59
|
-
const stream = socket()
|
|
60
|
-
|
|
61
|
-
batch(stream)
|
|
62
|
-
|
|
63
|
-
stream.write('a')
|
|
64
|
-
|
|
65
|
-
await tick()
|
|
66
|
-
|
|
67
|
-
expect(stream.writes).toStrictEqual([['a']])
|
|
68
|
-
})
|
|
69
|
-
|
|
70
|
-
it('should take a socket once', async () => {
|
|
71
|
-
const stream = socket()
|
|
72
|
-
|
|
73
|
-
batch(stream)
|
|
74
|
-
|
|
75
|
-
const wrapped = stream.write
|
|
76
|
-
|
|
77
|
-
batch(stream)
|
|
78
|
-
|
|
79
|
-
expect(stream.write).toStrictEqual(wrapped)
|
|
80
|
-
})
|
|
81
|
-
|
|
82
|
-
it('should leave alone what does not cork', async () => {
|
|
83
|
-
for (const value of [undefined, null, {}, { write: () => undefined }]) {
|
|
84
|
-
expect(() => batch(value)).not.toThrow()
|
|
85
|
-
}
|
|
86
|
-
|
|
87
|
-
const written = []
|
|
88
|
-
const plain = { write: (chunk) => written.push(chunk) }
|
|
89
|
-
|
|
90
|
-
batch(plain)
|
|
91
|
-
plain.write('a')
|
|
92
|
-
|
|
93
|
-
expect(written).toStrictEqual(['a'])
|
|
94
|
-
})
|