ac-sqs 4.0.4 → 4.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/node.js.yml +32 -0
- package/CHANGELOG.md +36 -0
- package/README.md +19 -11
- package/index.js +3 -3
- package/package.json +10 -8
- package/test/test.js +603 -0
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
|
|
2
|
+
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
|
|
3
|
+
|
|
4
|
+
name: Node.js CI
|
|
5
|
+
|
|
6
|
+
on:
|
|
7
|
+
push:
|
|
8
|
+
branches: [ develop, master ]
|
|
9
|
+
pull_request:
|
|
10
|
+
branches: [ develop, master ]
|
|
11
|
+
|
|
12
|
+
jobs:
|
|
13
|
+
build:
|
|
14
|
+
|
|
15
|
+
permissions:
|
|
16
|
+
contents: read
|
|
17
|
+
|
|
18
|
+
runs-on: ubuntu-latest
|
|
19
|
+
|
|
20
|
+
strategy:
|
|
21
|
+
matrix:
|
|
22
|
+
node-version: [22.x, 24.x]
|
|
23
|
+
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
|
|
24
|
+
|
|
25
|
+
steps:
|
|
26
|
+
- uses: actions/checkout@v5
|
|
27
|
+
- name: Use Node.js ${{ matrix.node-version }}
|
|
28
|
+
uses: actions/setup-node@v5
|
|
29
|
+
with:
|
|
30
|
+
node-version: ${{ matrix.node-version }}
|
|
31
|
+
- run: yarn install
|
|
32
|
+
- run: yarn run test
|
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,39 @@
|
|
|
1
|
+
## [4.0.6](https://github.com/admiralcloud/ac-sqs/compare/v4.0.5..v4.0.6) (2026-04-04 10:08:41)
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
### Bug Fix
|
|
5
|
+
|
|
6
|
+
|
|
7
|
+
* **App:** Make batchExtendInterval configurable | MP | [dda69e856f588882e4e293e62a0df474e483a46b](https://github.com/admiralcloud/ac-sqs/commit/dda69e856f588882e4e293e62a0df474e483a46b)
|
|
8
|
+
Make batchExtendInterval configurable but it still defaults to 5s - so there is no change
|
|
9
|
+
Related issues:
|
|
10
|
+
* **App:** Package updates | MP | [254924ccbd447b18e230fb4c5f7455abf9fdc060](https://github.com/admiralcloud/ac-sqs/commit/254924ccbd447b18e230fb4c5f7455abf9fdc060)
|
|
11
|
+
Package updates
|
|
12
|
+
Related issues:
|
|
13
|
+
* **App:** Package updates | MP | [161e2bc77e4eadc76614dd2c51dcc35dd08a611e](https://github.com/admiralcloud/ac-sqs/commit/161e2bc77e4eadc76614dd2c51dcc35dd08a611e)
|
|
14
|
+
Package updates
|
|
15
|
+
Related issues:
|
|
16
|
+
### Tests
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
* **App:** Added tests | MP | [01bb547ae0940ef4d41bc5b206e1f21888c2a87b](https://github.com/admiralcloud/ac-sqs/commit/01bb547ae0940ef4d41bc5b206e1f21888c2a87b)
|
|
20
|
+
Added tests
|
|
21
|
+
Related issues:
|
|
22
|
+
### Documentation
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
* **App:** Added badges | MP | [3806edd85fd4dbff240b1ceb33c4cb46dc2d57bd](https://github.com/admiralcloud/ac-sqs/commit/3806edd85fd4dbff240b1ceb33c4cb46dc2d57bd)
|
|
26
|
+
Added badges
|
|
27
|
+
Related issues:
|
|
28
|
+
|
|
29
|
+
## [4.0.5](https://github.com/admiralcloud/ac-sqs/compare/v4.0.4..v4.0.5) (2026-03-21 11:11:45)
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
### Bug Fix
|
|
33
|
+
|
|
34
|
+
* **Misc:** Package updates | MP | [5d39527151c36c166599414912d2b4266a53385c](https://github.com/admiralcloud/ac-sqs/commit/5d39527151c36c166599414912d2b4266a53385c)
|
|
35
|
+
Package updates
|
|
36
|
+
Related issues:
|
|
1
37
|
|
|
2
38
|
## [4.0.4](https://github.com/admiralcloud/ac-sqs/compare/v4.0.3..v4.0.4) (2026-02-24 19:19:56)
|
|
3
39
|
|
package/README.md
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# AC SQS
|
|
2
2
|
This tool is a wrapper for AWS SDK's SQS function. It includes handling of big SQS messages using S3.
|
|
3
3
|
|
|
4
|
+
[](https://github.com/AdmiralCloud/ac-sqs/actions/workflows/node.js.yml) [](https://github.com/AdmiralCloud/ac-sqs/actions/workflows/github-code-scanning/codeql)
|
|
5
|
+
|
|
4
6
|
## Breaking changes - version 2
|
|
5
7
|
This new class-based wrapper is not compatible with older versions.
|
|
6
8
|
|
|
@@ -40,6 +42,9 @@ Array of AWS SQS lists that will be used by this function. Every item in the lis
|
|
|
40
42
|
+ debug -> if true, all SQS payloads for that list will be logged (level debug)
|
|
41
43
|
+ throwError -> if true, error will throw otherwise they will only be logged (default)
|
|
42
44
|
|
|
45
|
+
**batchExtendInterval [optional]**
|
|
46
|
+
Interval in milliseconds at which the visibility timeout extension loop runs. Defaults to 5000 (5 seconds). Can be set to a lower value in test environments for faster shutdown.
|
|
47
|
+
|
|
43
48
|
Name should be the plain name of the list. Parameters like fifo or test (in test environment) or localPrefixes (for local development) should not be part of the list name. LocalPrefix can be used if multiple developers work on your project and you want to make sure they all work on their own SQS list without changing the name of all SQS lists in your main project.
|
|
44
49
|
|
|
45
50
|
Example for a FIFO list:
|
|
@@ -116,20 +121,23 @@ https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_GetQue
|
|
|
116
121
|
Call with all lists and let the function check if the queues exist. If not, the missing queue is created.
|
|
117
122
|
|
|
118
123
|
# Test
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
Preparations you have to make before running the tests:
|
|
124
|
+
Run tests using **yarn test** or **npm test**.
|
|
122
125
|
|
|
123
|
-
|
|
124
|
-
+ export the AWS account id using **export awsaccount=12345**
|
|
125
|
-
+ create a SQS list named "test_acsqs"
|
|
126
|
-
+ create a bucket and export the name using **export bucket=sqstest.admiralcloud.com**
|
|
127
|
-
+ export the node test environment using **export NODE_ENV=test**
|
|
126
|
+
No AWS credentials or real infrastructure required. All SQS and S3 calls are stubbed using [sinon](https://sinonjs.org/), so tests run fully offline and complete in under 2 seconds.
|
|
128
127
|
|
|
129
|
-
|
|
130
|
-
|
|
128
|
+
```
|
|
129
|
+
NODE_ENV=test yarn test
|
|
130
|
+
```
|
|
131
131
|
|
|
132
|
-
|
|
132
|
+
The test suite covers:
|
|
133
|
+
+ send, receive and delete (single and batch)
|
|
134
|
+
+ large message handling via S3 (PutObject / GetObject / DeleteObjects)
|
|
135
|
+
+ automatic visibility timeout extension (`processBatchExtensions`)
|
|
136
|
+
+ max extensions and invalid receipt handle cleanup
|
|
137
|
+
+ `createQueues` (existing and missing queues)
|
|
138
|
+
+ `extendVisibility` legacy method
|
|
139
|
+
+ `getVisibilityStats` breakdown
|
|
140
|
+
+ error handling at class and function level (`throwError`)
|
|
133
141
|
|
|
134
142
|
# Misc
|
|
135
143
|
## Links
|
package/index.js
CHANGED
|
@@ -6,19 +6,19 @@ const { SQSClient, SendMessageCommand, SendMessageBatchCommand, ReceiveMessageCo
|
|
|
6
6
|
const { S3Client, GetObjectCommand, PutObjectCommand, DeleteObjectsCommand } = require("@aws-sdk/client-s3")
|
|
7
7
|
|
|
8
8
|
class ACSQS {
|
|
9
|
-
constructor({ region = 'eu-central-1', account, availableLists, useS3 = { enabled: true, bucket: undefined }, messageThreshold = 1000e3, logger = console, throwError = false, maxConcurrentMessages = 3000 }) {
|
|
9
|
+
constructor({ region = 'eu-central-1', account, availableLists, useS3 = { enabled: true, bucket: undefined }, messageThreshold = 1000e3, logger = console, throwError = false, maxConcurrentMessages = 3000, batchExtendInterval = 5000 }) {
|
|
10
10
|
this.region = region
|
|
11
11
|
this.account = account
|
|
12
12
|
this.availableLists = availableLists
|
|
13
13
|
this.logger = logger
|
|
14
14
|
this.throwError = throwError
|
|
15
15
|
this.maxConcurrentMessages = maxConcurrentMessages
|
|
16
|
-
|
|
16
|
+
|
|
17
17
|
// Improved visibility management
|
|
18
18
|
this.visibilityManagement = new Map()
|
|
19
19
|
this.batchExtendRunning = false
|
|
20
20
|
this.stopBatchExtend = false
|
|
21
|
-
this.batchExtendInterval =
|
|
21
|
+
this.batchExtendInterval = batchExtendInterval
|
|
22
22
|
|
|
23
23
|
const awsConfig = {
|
|
24
24
|
region,
|
package/package.json
CHANGED
|
@@ -3,19 +3,20 @@
|
|
|
3
3
|
"author": "Mark Poepping (https://www.admiralcloud.com)",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"repository": "admiralcloud/ac-sqs",
|
|
6
|
-
"version": "4.0.
|
|
6
|
+
"version": "4.0.6",
|
|
7
7
|
"dependencies": {
|
|
8
|
-
"@aws-sdk/client-s3": "^3.
|
|
9
|
-
"@aws-sdk/client-sqs": "^3.
|
|
10
|
-
"lodash": "^4.
|
|
8
|
+
"@aws-sdk/client-s3": "^3.1024.0",
|
|
9
|
+
"@aws-sdk/client-sqs": "^3.1024.0",
|
|
10
|
+
"lodash": "^4.18.1",
|
|
11
11
|
"uuid": "^11.1.0"
|
|
12
12
|
},
|
|
13
13
|
"devDependencies": {
|
|
14
|
-
"ac-semantic-release": "^0.
|
|
14
|
+
"ac-semantic-release": "^1.0.1",
|
|
15
15
|
"chai": "^4.5.0",
|
|
16
|
-
"eslint": "^10.0
|
|
17
|
-
"globals": "^17.
|
|
18
|
-
"mocha": "^11.7.5"
|
|
16
|
+
"eslint": "^10.2.0",
|
|
17
|
+
"globals": "^17.4.0",
|
|
18
|
+
"mocha": "^11.7.5",
|
|
19
|
+
"sinon": "^21.0.3"
|
|
19
20
|
},
|
|
20
21
|
"scripts": {
|
|
21
22
|
"test": "mocha --reporter spec"
|
|
@@ -26,6 +27,7 @@
|
|
|
26
27
|
"resolutions": {
|
|
27
28
|
"mocha/chokidar/braces": "^3.0.3",
|
|
28
29
|
"mocha/diff": "^8.0.3",
|
|
30
|
+
"mocha/serialize-javascript": "^7.0.3",
|
|
29
31
|
"minimatch": "^10.2.1"
|
|
30
32
|
},
|
|
31
33
|
"packageManager": "yarn@1.22.22+sha512.a6b2f7906b721bba3d67d4aff083df04dad64c399707841b7acf00f6b133b7ac24255f2652fa22ae3534329dc6180534e98d17432037ff6fd140556e2bb3137e"
|
package/test/test.js
ADDED
|
@@ -0,0 +1,603 @@
|
|
|
1
|
+
const _ = require('lodash')
|
|
2
|
+
const { expect } = require('chai')
|
|
3
|
+
const sinon = require('sinon')
|
|
4
|
+
const ACSQS = require('../index')
|
|
5
|
+
|
|
6
|
+
const name = 'acsqs'
|
|
7
|
+
const FAKE_ACCOUNT = '123456789012'
|
|
8
|
+
const FAKE_BUCKET = 'test-bucket'
|
|
9
|
+
const FAKE_QUEUE_ARN = `arn:aws:sqs:eu-central-1:${FAKE_ACCOUNT}:test_acsqs`
|
|
10
|
+
|
|
11
|
+
// Suppress logger output during tests
|
|
12
|
+
const silentLogger = { debug: () => {}, info: () => {}, warn: () => {}, error: () => {} }
|
|
13
|
+
|
|
14
|
+
function makeConfig(overrides = {}) {
|
|
15
|
+
return {
|
|
16
|
+
account: FAKE_ACCOUNT,
|
|
17
|
+
availableLists: [{
|
|
18
|
+
name,
|
|
19
|
+
waitTime: 0,
|
|
20
|
+
visibilityTimeout: 30,
|
|
21
|
+
messageThreshold: 250e3
|
|
22
|
+
}],
|
|
23
|
+
useS3: {
|
|
24
|
+
enabled: true,
|
|
25
|
+
bucket: FAKE_BUCKET,
|
|
26
|
+
},
|
|
27
|
+
logger: silentLogger,
|
|
28
|
+
batchExtendInterval: 50, // fast shutdown in tests
|
|
29
|
+
...overrides
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
describe('Test basics', function () {
|
|
35
|
+
let acsqs, sqsSend, s3Send
|
|
36
|
+
let newMessage = {}
|
|
37
|
+
const message = 'This is a message'
|
|
38
|
+
|
|
39
|
+
before(() => {
|
|
40
|
+
acsqs = new ACSQS(makeConfig())
|
|
41
|
+
sqsSend = sinon.stub(acsqs.sqs, 'send')
|
|
42
|
+
s3Send = sinon.stub(acsqs.s3, 'send')
|
|
43
|
+
})
|
|
44
|
+
|
|
45
|
+
after(async () => {
|
|
46
|
+
await acsqs.shutdown()
|
|
47
|
+
sinon.restore()
|
|
48
|
+
})
|
|
49
|
+
|
|
50
|
+
afterEach(() => {
|
|
51
|
+
sqsSend.reset()
|
|
52
|
+
s3Send.reset()
|
|
53
|
+
acsqs.visibilityManagement.clear()
|
|
54
|
+
})
|
|
55
|
+
|
|
56
|
+
it('getAllLists', async () => {
|
|
57
|
+
sqsSend.resolves({ $metadata: { httpStatusCode: 200 }, Attributes: { QueueArn: FAKE_QUEUE_ARN } })
|
|
58
|
+
const response = await acsqs.getAllLists()
|
|
59
|
+
expect(_.first(response)).to.have.property('value', FAKE_QUEUE_ARN)
|
|
60
|
+
})
|
|
61
|
+
|
|
62
|
+
it('sendSQSMessage', async () => {
|
|
63
|
+
sqsSend.resolves({
|
|
64
|
+
$metadata: { httpStatusCode: 200 },
|
|
65
|
+
MessageId: 'msg-id-1',
|
|
66
|
+
MD5OfMessageBody: '78745dd27ccc2f660afba9841f58259b'
|
|
67
|
+
})
|
|
68
|
+
const response = await acsqs.sendSQSMessage({ name, message })
|
|
69
|
+
expect(response.$metadata.httpStatusCode).to.eql(200)
|
|
70
|
+
expect(response.MD5OfMessageBody).to.eql('78745dd27ccc2f660afba9841f58259b')
|
|
71
|
+
newMessage = { Id: response.MessageId }
|
|
72
|
+
})
|
|
73
|
+
|
|
74
|
+
it('getQueueAttributes - shows 1 message', async () => {
|
|
75
|
+
sqsSend.resolves({
|
|
76
|
+
$metadata: { httpStatusCode: 200 },
|
|
77
|
+
Attributes: { ApproximateNumberOfMessages: '1' }
|
|
78
|
+
})
|
|
79
|
+
const response = await acsqs.getQueueAttributes({ name })
|
|
80
|
+
expect(response.$metadata.httpStatusCode).to.eql(200)
|
|
81
|
+
expect(response.Attributes.ApproximateNumberOfMessages).to.eql('1')
|
|
82
|
+
})
|
|
83
|
+
|
|
84
|
+
it('receiveSQSMessages', async () => {
|
|
85
|
+
sqsSend.resolves({
|
|
86
|
+
Messages: [{
|
|
87
|
+
MessageId: 'msg-id-1',
|
|
88
|
+
MD5OfBody: '78745dd27ccc2f660afba9841f58259b',
|
|
89
|
+
Body: message,
|
|
90
|
+
ReceiptHandle: 'rh-1'
|
|
91
|
+
}]
|
|
92
|
+
})
|
|
93
|
+
const response = await acsqs.receiveSQSMessages({ name })
|
|
94
|
+
const first = _.first(response)
|
|
95
|
+
expect(first.MD5OfBody).to.eql('78745dd27ccc2f660afba9841f58259b')
|
|
96
|
+
expect(first.Body).to.eql(message)
|
|
97
|
+
expect(first).to.have.property('ReceiptHandle')
|
|
98
|
+
newMessage.ReceiptHandle = first.ReceiptHandle
|
|
99
|
+
})
|
|
100
|
+
|
|
101
|
+
it('deleteSQSMessages', async () => {
|
|
102
|
+
sqsSend.resolves({
|
|
103
|
+
$metadata: { httpStatusCode: 200 },
|
|
104
|
+
Successful: [{ Id: 'msg-id-1' }],
|
|
105
|
+
Failed: []
|
|
106
|
+
})
|
|
107
|
+
const response = await acsqs.deleteSQSMessages({ name, items: [newMessage] })
|
|
108
|
+
expect(response.$metadata.httpStatusCode).to.eql(200)
|
|
109
|
+
expect(_.first(response.Successful)).to.have.property('Id', newMessage.Id)
|
|
110
|
+
})
|
|
111
|
+
|
|
112
|
+
it('getQueueAttributes - shows 0 messages after delete', async () => {
|
|
113
|
+
sqsSend.resolves({
|
|
114
|
+
$metadata: { httpStatusCode: 200 },
|
|
115
|
+
Attributes: { ApproximateNumberOfMessages: '0' }
|
|
116
|
+
})
|
|
117
|
+
const response = await acsqs.getQueueAttributes({ name })
|
|
118
|
+
expect(response.Attributes.ApproximateNumberOfMessages).to.eql('0')
|
|
119
|
+
})
|
|
120
|
+
|
|
121
|
+
it('receiveSQSMessages - returns undefined on empty queue', async () => {
|
|
122
|
+
sqsSend.resolves({ Messages: [] })
|
|
123
|
+
const response = await acsqs.receiveSQSMessages({ name })
|
|
124
|
+
expect(response).to.be.undefined
|
|
125
|
+
})
|
|
126
|
+
|
|
127
|
+
it('sendSQSMessage - passes optional fields (groupId, deduplicationId, delay)', async () => {
|
|
128
|
+
sqsSend.resolves({ $metadata: { httpStatusCode: 200 }, MessageId: 'msg-fifo-1' })
|
|
129
|
+
await acsqs.sendSQSMessage({ name, message, messageGroupId: 'group1', deDuplicationId: 'dedup1', delay: 5 })
|
|
130
|
+
const payload = sqsSend.firstCall.args[0].input
|
|
131
|
+
expect(payload.MessageGroupId).to.eql('group1')
|
|
132
|
+
expect(payload.MessageDeduplicationId).to.eql('dedup1')
|
|
133
|
+
expect(payload.DelaySeconds).to.eql(5)
|
|
134
|
+
})
|
|
135
|
+
})
|
|
136
|
+
|
|
137
|
+
|
|
138
|
+
describe('Test message batch', function () {
|
|
139
|
+
let acsqs, sqsSend, s3Send
|
|
140
|
+
let messageItems = []
|
|
141
|
+
|
|
142
|
+
before(() => {
|
|
143
|
+
acsqs = new ACSQS(makeConfig())
|
|
144
|
+
sqsSend = sinon.stub(acsqs.sqs, 'send')
|
|
145
|
+
s3Send = sinon.stub(acsqs.s3, 'send')
|
|
146
|
+
})
|
|
147
|
+
|
|
148
|
+
after(async () => {
|
|
149
|
+
await acsqs.shutdown()
|
|
150
|
+
sinon.restore()
|
|
151
|
+
})
|
|
152
|
+
|
|
153
|
+
afterEach(() => {
|
|
154
|
+
sqsSend.reset()
|
|
155
|
+
s3Send.reset()
|
|
156
|
+
acsqs.visibilityManagement.clear()
|
|
157
|
+
})
|
|
158
|
+
|
|
159
|
+
it('sendSQSMessageBatch', async () => {
|
|
160
|
+
sqsSend.resolves({
|
|
161
|
+
$metadata: { httpStatusCode: 200 },
|
|
162
|
+
Successful: [
|
|
163
|
+
{ Id: '0', MD5OfMessageBody: '21fa8c4ea3be7cf61328c3f6aeb1dc78' },
|
|
164
|
+
{ Id: '1', MD5OfMessageBody: 'bfef4766d0fa3755cee4f499c5ab3626' }
|
|
165
|
+
],
|
|
166
|
+
Failed: []
|
|
167
|
+
})
|
|
168
|
+
const messages = [{ messageBody: 'Message #1' }, { messageBody: 'Message #2' }]
|
|
169
|
+
const response = await acsqs.sendSQSMessageBatch({ name, messages })
|
|
170
|
+
expect(response.$metadata.httpStatusCode).to.eql(200)
|
|
171
|
+
expect(_.last(response.Successful).MD5OfMessageBody).to.eql('bfef4766d0fa3755cee4f499c5ab3626')
|
|
172
|
+
})
|
|
173
|
+
|
|
174
|
+
it('receiveSQSMessages batch', async () => {
|
|
175
|
+
sqsSend.resolves({
|
|
176
|
+
Messages: [
|
|
177
|
+
{ MessageId: 'id-0', MD5OfBody: '21fa8c4ea3be7cf61328c3f6aeb1dc78', Body: 'Message #1', ReceiptHandle: 'rh-0' },
|
|
178
|
+
{ MessageId: 'id-1', MD5OfBody: 'bfef4766d0fa3755cee4f499c5ab3626', Body: 'Message #2', ReceiptHandle: 'rh-1' }
|
|
179
|
+
]
|
|
180
|
+
})
|
|
181
|
+
const response = await acsqs.receiveSQSMessages({ name })
|
|
182
|
+
messageItems = response.map(item => _.pick(item, ['MessageId', 'ReceiptHandle']))
|
|
183
|
+
expect(_.first(response).MD5OfBody).to.eql('21fa8c4ea3be7cf61328c3f6aeb1dc78')
|
|
184
|
+
expect(_.first(response).Body).to.eql('Message #1')
|
|
185
|
+
expect(_.last(response).MD5OfBody).to.eql('bfef4766d0fa3755cee4f499c5ab3626')
|
|
186
|
+
expect(_.last(response).Body).to.eql('Message #2')
|
|
187
|
+
})
|
|
188
|
+
|
|
189
|
+
it('deleteSQSMessages batch', async () => {
|
|
190
|
+
sqsSend.resolves({
|
|
191
|
+
$metadata: { httpStatusCode: 200 },
|
|
192
|
+
Successful: messageItems.map(item => ({ Id: item.MessageId })),
|
|
193
|
+
Failed: []
|
|
194
|
+
})
|
|
195
|
+
const response = await acsqs.deleteSQSMessages({ name, items: messageItems })
|
|
196
|
+
expect(response.$metadata.httpStatusCode).to.eql(200)
|
|
197
|
+
expect(_.first(response.Successful)).to.have.property('Id', _.first(messageItems).MessageId)
|
|
198
|
+
})
|
|
199
|
+
|
|
200
|
+
it('sendSQSMessageBatch - passes optional fields per entry', async () => {
|
|
201
|
+
sqsSend.resolves({ $metadata: { httpStatusCode: 200 }, Successful: [], Failed: [] })
|
|
202
|
+
const messages = [{
|
|
203
|
+
messageBody: 'test',
|
|
204
|
+
messageGroupId: 'g1',
|
|
205
|
+
messageDeduplicationId: 'dedup1',
|
|
206
|
+
delaySeconds: 10
|
|
207
|
+
}]
|
|
208
|
+
await acsqs.sendSQSMessageBatch({ name, messages })
|
|
209
|
+
const entries = sqsSend.firstCall.args[0].input.Entries
|
|
210
|
+
expect(entries[0].MessageGroupId).to.eql('g1')
|
|
211
|
+
expect(entries[0].MessageDeduplicationId).to.eql('dedup1')
|
|
212
|
+
expect(entries[0].DelaySeconds).to.eql(10)
|
|
213
|
+
})
|
|
214
|
+
})
|
|
215
|
+
|
|
216
|
+
|
|
217
|
+
describe('Test with S3', function () {
|
|
218
|
+
let acsqs, sqsSend, s3Send
|
|
219
|
+
let newMessage = {}
|
|
220
|
+
let bigMessage
|
|
221
|
+
|
|
222
|
+
before(() => {
|
|
223
|
+
const buffer = Buffer.alloc(400 * 1024)
|
|
224
|
+
buffer.fill('A')
|
|
225
|
+
bigMessage = buffer.toString()
|
|
226
|
+
|
|
227
|
+
acsqs = new ACSQS(makeConfig())
|
|
228
|
+
sqsSend = sinon.stub(acsqs.sqs, 'send')
|
|
229
|
+
s3Send = sinon.stub(acsqs.s3, 'send')
|
|
230
|
+
})
|
|
231
|
+
|
|
232
|
+
after(async () => {
|
|
233
|
+
await acsqs.shutdown()
|
|
234
|
+
sinon.restore()
|
|
235
|
+
})
|
|
236
|
+
|
|
237
|
+
afterEach(() => {
|
|
238
|
+
sqsSend.reset()
|
|
239
|
+
s3Send.reset()
|
|
240
|
+
acsqs.visibilityManagement.clear()
|
|
241
|
+
})
|
|
242
|
+
|
|
243
|
+
it('sendSQSMessage - stores large message in S3', async () => {
|
|
244
|
+
s3Send.resolves({ $metadata: { httpStatusCode: 200 } })
|
|
245
|
+
sqsSend.resolves({
|
|
246
|
+
$metadata: { httpStatusCode: 200 },
|
|
247
|
+
MessageId: 'msg-s3-1',
|
|
248
|
+
MD5OfMessageBody: 'some-md5'
|
|
249
|
+
})
|
|
250
|
+
const response = await acsqs.sendSQSMessage({ name, message: bigMessage })
|
|
251
|
+
expect(response.$metadata.httpStatusCode).to.eql(200)
|
|
252
|
+
expect(s3Send.calledOnce).to.be.true
|
|
253
|
+
const sqsPayload = sqsSend.firstCall.args[0].input
|
|
254
|
+
expect(sqsPayload.MessageBody).to.match(/^s3:/)
|
|
255
|
+
newMessage = { Id: response.MessageId }
|
|
256
|
+
})
|
|
257
|
+
|
|
258
|
+
it('receiveSQSMessages - retrieves large message body from S3', async () => {
|
|
259
|
+
const s3Key = 'some-uuid-key'
|
|
260
|
+
sqsSend.resolves({
|
|
261
|
+
Messages: [{
|
|
262
|
+
MessageId: 'msg-s3-1',
|
|
263
|
+
Body: `s3:${s3Key}`,
|
|
264
|
+
ReceiptHandle: 'rh-s3-1'
|
|
265
|
+
}]
|
|
266
|
+
})
|
|
267
|
+
s3Send.resolves({
|
|
268
|
+
Body: { transformToString: sinon.stub().resolves(bigMessage) }
|
|
269
|
+
})
|
|
270
|
+
const response = await acsqs.receiveSQSMessages({ name })
|
|
271
|
+
const first = _.first(response)
|
|
272
|
+
expect(first.Body).to.eql(bigMessage)
|
|
273
|
+
expect(first.s3key).to.eql(s3Key)
|
|
274
|
+
newMessage.ReceiptHandle = first.ReceiptHandle
|
|
275
|
+
newMessage.s3key = first.s3key
|
|
276
|
+
})
|
|
277
|
+
|
|
278
|
+
it('deleteSQSMessages - triggers S3 cleanup for s3key', async () => {
|
|
279
|
+
sqsSend.resolves({
|
|
280
|
+
$metadata: { httpStatusCode: 200 },
|
|
281
|
+
Successful: [{ Id: newMessage.Id }],
|
|
282
|
+
Failed: []
|
|
283
|
+
})
|
|
284
|
+
s3Send.resolves({ $metadata: { httpStatusCode: 200 } })
|
|
285
|
+
const response = await acsqs.deleteSQSMessages({ name, items: [newMessage] })
|
|
286
|
+
expect(response.$metadata.httpStatusCode).to.eql(200)
|
|
287
|
+
expect(sqsSend.calledOnce).to.be.true
|
|
288
|
+
})
|
|
289
|
+
})
|
|
290
|
+
|
|
291
|
+
|
|
292
|
+
describe('Test visibility extension', function () {
|
|
293
|
+
let acsqs, sqsSend
|
|
294
|
+
|
|
295
|
+
before(() => {
|
|
296
|
+
acsqs = new ACSQS(makeConfig({
|
|
297
|
+
availableLists: [{
|
|
298
|
+
name,
|
|
299
|
+
waitTime: 0,
|
|
300
|
+
visibilityTimeout: 30,
|
|
301
|
+
messageThreshold: 250e3
|
|
302
|
+
}]
|
|
303
|
+
}))
|
|
304
|
+
sqsSend = sinon.stub(acsqs.sqs, 'send')
|
|
305
|
+
sinon.stub(acsqs.s3, 'send')
|
|
306
|
+
})
|
|
307
|
+
|
|
308
|
+
after(async () => {
|
|
309
|
+
await acsqs.shutdown()
|
|
310
|
+
sinon.restore()
|
|
311
|
+
})
|
|
312
|
+
|
|
313
|
+
afterEach(() => {
|
|
314
|
+
sqsSend.reset()
|
|
315
|
+
acsqs.visibilityManagement.clear()
|
|
316
|
+
})
|
|
317
|
+
|
|
318
|
+
it('receiveSQSMessages adds message to visibility tracking', async () => {
|
|
319
|
+
sqsSend.resolves({
|
|
320
|
+
Messages: [{ MessageId: 'vis-1', Body: 'test', ReceiptHandle: 'rh-1' }]
|
|
321
|
+
})
|
|
322
|
+
await acsqs.receiveSQSMessages({ name })
|
|
323
|
+
expect(acsqs.getVisibilityStats().totalTracked).to.eql(1)
|
|
324
|
+
})
|
|
325
|
+
|
|
326
|
+
it('processBatchExtensions - extends visibility for due messages', async () => {
|
|
327
|
+
acsqs.visibilityManagement.set('vis-1', {
|
|
328
|
+
messageId: 'vis-1', queueName: name, receiptHandle: 'rh-1',
|
|
329
|
+
extensionCount: 0, maxExtensions: 12,
|
|
330
|
+
nextExtendTime: Date.now() - 1, createdAt: Date.now() - 5000
|
|
331
|
+
})
|
|
332
|
+
sqsSend.resolves({
|
|
333
|
+
$metadata: { httpStatusCode: 200 },
|
|
334
|
+
Successful: [{ Id: 'vis-1' }],
|
|
335
|
+
Failed: []
|
|
336
|
+
})
|
|
337
|
+
await acsqs.processBatchExtensions()
|
|
338
|
+
expect(sqsSend.calledOnce).to.be.true
|
|
339
|
+
expect(acsqs.visibilityManagement.get('vis-1').extensionCount).to.eql(1)
|
|
340
|
+
expect(acsqs.getVisibilityStats().totalTracked).to.eql(1)
|
|
341
|
+
})
|
|
342
|
+
|
|
343
|
+
it('processBatchExtensions - skips messages not yet due', async () => {
|
|
344
|
+
acsqs.visibilityManagement.set('vis-future', {
|
|
345
|
+
messageId: 'vis-future', queueName: name, receiptHandle: 'rh-2',
|
|
346
|
+
extensionCount: 0, maxExtensions: 12,
|
|
347
|
+
nextExtendTime: Date.now() + 60000, createdAt: Date.now()
|
|
348
|
+
})
|
|
349
|
+
await acsqs.processBatchExtensions()
|
|
350
|
+
expect(sqsSend.called).to.be.false
|
|
351
|
+
})
|
|
352
|
+
|
|
353
|
+
it('processBatchExtensions - removes message when max extensions reached', async () => {
|
|
354
|
+
acsqs.visibilityManagement.set('vis-max', {
|
|
355
|
+
messageId: 'vis-max', queueName: name, receiptHandle: 'rh-max',
|
|
356
|
+
extensionCount: 12, maxExtensions: 12,
|
|
357
|
+
nextExtendTime: Date.now() - 1, createdAt: Date.now() - 10000
|
|
358
|
+
})
|
|
359
|
+
await acsqs.processBatchExtensions()
|
|
360
|
+
expect(sqsSend.called).to.be.false
|
|
361
|
+
expect(acsqs.visibilityManagement.has('vis-max')).to.be.false
|
|
362
|
+
})
|
|
363
|
+
|
|
364
|
+
it('processBatchExtensions - removes message on ReceiptHandleIsInvalid', async () => {
|
|
365
|
+
acsqs.visibilityManagement.set('vis-invalid', {
|
|
366
|
+
messageId: 'vis-invalid', queueName: name, receiptHandle: 'rh-invalid',
|
|
367
|
+
extensionCount: 0, maxExtensions: 12,
|
|
368
|
+
nextExtendTime: Date.now() - 1, createdAt: Date.now()
|
|
369
|
+
})
|
|
370
|
+
sqsSend.resolves({
|
|
371
|
+
Successful: [],
|
|
372
|
+
Failed: [{ Id: 'vis-invalid', Code: 'ReceiptHandleIsInvalid', Message: 'The receipt handle has expired.' }]
|
|
373
|
+
})
|
|
374
|
+
await acsqs.processBatchExtensions()
|
|
375
|
+
expect(acsqs.visibilityManagement.has('vis-invalid')).to.be.false
|
|
376
|
+
})
|
|
377
|
+
|
|
378
|
+
it('deleteSQSMessages removes message from visibility tracking', async () => {
|
|
379
|
+
acsqs.visibilityManagement.set('vis-1', {
|
|
380
|
+
messageId: 'vis-1', queueName: name, receiptHandle: 'rh-1',
|
|
381
|
+
extensionCount: 2, maxExtensions: 12,
|
|
382
|
+
nextExtendTime: Date.now() + 10000, createdAt: Date.now()
|
|
383
|
+
})
|
|
384
|
+
sqsSend.resolves({
|
|
385
|
+
$metadata: { httpStatusCode: 200 },
|
|
386
|
+
Successful: [{ Id: 'vis-1' }],
|
|
387
|
+
Failed: []
|
|
388
|
+
})
|
|
389
|
+
await acsqs.deleteSQSMessages({ name, items: [{ Id: 'vis-1', ReceiptHandle: 'rh-1' }] })
|
|
390
|
+
expect(acsqs.visibilityManagement.has('vis-1')).to.be.false
|
|
391
|
+
expect(acsqs.getVisibilityStats().totalTracked).to.eql(0)
|
|
392
|
+
})
|
|
393
|
+
|
|
394
|
+
it('addVisibilityTracking - rejects when maxConcurrentMessages is reached', () => {
|
|
395
|
+
const limited = new ACSQS(makeConfig({ maxConcurrentMessages: 2, logger: silentLogger }))
|
|
396
|
+
limited.addVisibilityTracking('m1', name, 'rh1', { visibilityTimeout: 30 })
|
|
397
|
+
limited.addVisibilityTracking('m2', name, 'rh2', { visibilityTimeout: 30 })
|
|
398
|
+
const result = limited.addVisibilityTracking('m3', name, 'rh3', { visibilityTimeout: 30 })
|
|
399
|
+
expect(result).to.be.false
|
|
400
|
+
expect(limited.visibilityManagement.size).to.eql(2)
|
|
401
|
+
limited.shutdown()
|
|
402
|
+
})
|
|
403
|
+
|
|
404
|
+
it('getVisibilityStats - returns breakdown by queue', () => {
|
|
405
|
+
// Use direct map manipulation to control createdAt timestamp reliably
|
|
406
|
+
const past = Date.now() - 1000
|
|
407
|
+
acsqs.visibilityManagement.set('stat-1', {
|
|
408
|
+
messageId: 'stat-1', queueName: name, receiptHandle: 'rh-s1',
|
|
409
|
+
extensionCount: 1, maxExtensions: 12,
|
|
410
|
+
nextExtendTime: Date.now() + 10000, createdAt: past
|
|
411
|
+
})
|
|
412
|
+
acsqs.visibilityManagement.set('stat-2', {
|
|
413
|
+
messageId: 'stat-2', queueName: name, receiptHandle: 'rh-s2',
|
|
414
|
+
extensionCount: 3, maxExtensions: 12,
|
|
415
|
+
nextExtendTime: Date.now() + 10000, createdAt: past + 100
|
|
416
|
+
})
|
|
417
|
+
const stats = acsqs.getVisibilityStats()
|
|
418
|
+
expect(stats.totalTracked).to.eql(2)
|
|
419
|
+
expect(stats.queueBreakdown[name]).to.eql(2)
|
|
420
|
+
expect(stats.avgExtensions).to.eql(2) // (1+3)/2
|
|
421
|
+
expect(stats.oldestMessage).to.not.be.null
|
|
422
|
+
expect(stats.oldestMessage.messageId).to.eql('stat-1')
|
|
423
|
+
})
|
|
424
|
+
})
|
|
425
|
+
|
|
426
|
+
|
|
427
|
+
describe('Test extendVisibility (legacy)', function () {
|
|
428
|
+
let acsqs
|
|
429
|
+
|
|
430
|
+
before(() => {
|
|
431
|
+
acsqs = new ACSQS(makeConfig())
|
|
432
|
+
sinon.stub(acsqs.sqs, 'send')
|
|
433
|
+
sinon.stub(acsqs.s3, 'send')
|
|
434
|
+
})
|
|
435
|
+
|
|
436
|
+
after(async () => {
|
|
437
|
+
await acsqs.shutdown()
|
|
438
|
+
sinon.restore()
|
|
439
|
+
})
|
|
440
|
+
|
|
441
|
+
afterEach(() => acsqs.visibilityManagement.clear())
|
|
442
|
+
|
|
443
|
+
it('adds untracked message to visibility tracking', async () => {
|
|
444
|
+
const message = { MessageId: 'ext-1', ReceiptHandle: 'rh-ext-1' }
|
|
445
|
+
await acsqs.extendVisibility({ name, message })
|
|
446
|
+
expect(acsqs.visibilityManagement.has('ext-1')).to.be.true
|
|
447
|
+
})
|
|
448
|
+
|
|
449
|
+
it('forces immediate extension for already-tracked message', async () => {
|
|
450
|
+
const message = { MessageId: 'ext-1', ReceiptHandle: 'rh-ext-1' }
|
|
451
|
+
await acsqs.extendVisibility({ name, message })
|
|
452
|
+
const tracking = acsqs.visibilityManagement.get('ext-1')
|
|
453
|
+
// Set to future to detect the change
|
|
454
|
+
tracking.nextExtendTime = Date.now() + 60000
|
|
455
|
+
|
|
456
|
+
await acsqs.extendVisibility({ name, message })
|
|
457
|
+
expect(tracking.nextExtendTime).to.be.lessThanOrEqual(Date.now())
|
|
458
|
+
})
|
|
459
|
+
|
|
460
|
+
it('throws configurationForListMissing for unknown queue', async () => {
|
|
461
|
+
try {
|
|
462
|
+
await acsqs.extendVisibility({ name: 'unknown', message: { MessageId: 'x', ReceiptHandle: 'y' } })
|
|
463
|
+
expect.fail('Should have thrown')
|
|
464
|
+
}
|
|
465
|
+
catch(e) {
|
|
466
|
+
expect(e.message).to.eql('configurationForListMissing')
|
|
467
|
+
}
|
|
468
|
+
})
|
|
469
|
+
})
|
|
470
|
+
|
|
471
|
+
|
|
472
|
+
describe('Test createQueues', function () {
|
|
473
|
+
let acsqs, sqsSend
|
|
474
|
+
|
|
475
|
+
before(() => {
|
|
476
|
+
acsqs = new ACSQS(makeConfig({
|
|
477
|
+
availableLists: [{
|
|
478
|
+
name,
|
|
479
|
+
waitTime: 0,
|
|
480
|
+
visibilityTimeout: 30,
|
|
481
|
+
attributes: { MessageRetentionPeriod: '86400' }
|
|
482
|
+
}]
|
|
483
|
+
}))
|
|
484
|
+
sqsSend = sinon.stub(acsqs.sqs, 'send')
|
|
485
|
+
sinon.stub(acsqs.s3, 'send')
|
|
486
|
+
})
|
|
487
|
+
|
|
488
|
+
after(async () => {
|
|
489
|
+
await acsqs.shutdown()
|
|
490
|
+
sinon.restore()
|
|
491
|
+
})
|
|
492
|
+
|
|
493
|
+
afterEach(() => sqsSend.reset())
|
|
494
|
+
|
|
495
|
+
it('skips creation if queue already exists', async () => {
|
|
496
|
+
sqsSend.resolves({ QueueUrl: 'https://sqs.eu-central-1.amazonaws.com/123/test_acsqs' })
|
|
497
|
+
await acsqs.createQueues({ lists: [{ name }] })
|
|
498
|
+
expect(sqsSend.calledOnce).to.be.true
|
|
499
|
+
})
|
|
500
|
+
|
|
501
|
+
it('creates queue when it does not exist', async () => {
|
|
502
|
+
sqsSend.onFirstCall().rejects(new Error('AWS.SimpleQueueService.NonExistentQueue'))
|
|
503
|
+
sqsSend.onSecondCall().resolves({ QueueUrl: 'https://sqs.eu-central-1.amazonaws.com/123/test_acsqs' })
|
|
504
|
+
await acsqs.createQueues({ lists: [{ name }], debug: true })
|
|
505
|
+
expect(sqsSend.calledTwice).to.be.true
|
|
506
|
+
})
|
|
507
|
+
|
|
508
|
+
it('throws configurationForListMissing for unconfigured queue', async () => {
|
|
509
|
+
try {
|
|
510
|
+
await acsqs.createQueues({ lists: [{ name: 'unconfigured' }] })
|
|
511
|
+
expect.fail('Should have thrown')
|
|
512
|
+
}
|
|
513
|
+
catch(e) {
|
|
514
|
+
expect(e.message).to.eql('configurationForListMissing')
|
|
515
|
+
}
|
|
516
|
+
})
|
|
517
|
+
})
|
|
518
|
+
|
|
519
|
+
|
|
520
|
+
describe('Error handling', function () {
|
|
521
|
+
afterEach(() => sinon.restore())
|
|
522
|
+
|
|
523
|
+
it('getAllLists - missing queue silently returns undefined value', async () => {
|
|
524
|
+
const acsqs = new ACSQS(makeConfig())
|
|
525
|
+
sinon.stub(acsqs.sqs, 'send').rejects(new Error('The specified queue does not exist.'))
|
|
526
|
+
const response = await acsqs.getAllLists()
|
|
527
|
+
expect(_.first(response)).to.have.property('value', undefined)
|
|
528
|
+
await acsqs.shutdown()
|
|
529
|
+
})
|
|
530
|
+
|
|
531
|
+
it('getAllLists - throwError: true throws the error', async () => {
|
|
532
|
+
const acsqs = new ACSQS(makeConfig({ throwError: true }))
|
|
533
|
+
sinon.stub(acsqs.sqs, 'send').rejects(new Error('The specified queue does not exist.'))
|
|
534
|
+
try {
|
|
535
|
+
await acsqs.getAllLists()
|
|
536
|
+
expect.fail('Should have thrown')
|
|
537
|
+
}
|
|
538
|
+
catch(e) {
|
|
539
|
+
expect(e.message).to.eql('The specified queue does not exist.')
|
|
540
|
+
}
|
|
541
|
+
await acsqs.shutdown()
|
|
542
|
+
})
|
|
543
|
+
|
|
544
|
+
it('getAllLists - throwError at function level throws the error', async () => {
|
|
545
|
+
const acsqs = new ACSQS(makeConfig())
|
|
546
|
+
sinon.stub(acsqs.sqs, 'send').rejects(new Error('The specified queue does not exist.'))
|
|
547
|
+
try {
|
|
548
|
+
await acsqs.getAllLists({ throwError: true })
|
|
549
|
+
expect.fail('Should have thrown')
|
|
550
|
+
}
|
|
551
|
+
catch(e) {
|
|
552
|
+
expect(e.message).to.eql('The specified queue does not exist.')
|
|
553
|
+
}
|
|
554
|
+
await acsqs.shutdown()
|
|
555
|
+
})
|
|
556
|
+
|
|
557
|
+
it('sendSQSMessage - throws configurationForListMissing for unknown queue', async () => {
|
|
558
|
+
const acsqs = new ACSQS(makeConfig())
|
|
559
|
+
try {
|
|
560
|
+
await acsqs.sendSQSMessage({ name: 'unknown', message: 'test' })
|
|
561
|
+
expect.fail('Should have thrown')
|
|
562
|
+
}
|
|
563
|
+
catch(e) {
|
|
564
|
+
expect(e.message).to.eql('configurationForListMissing')
|
|
565
|
+
}
|
|
566
|
+
await acsqs.shutdown()
|
|
567
|
+
})
|
|
568
|
+
|
|
569
|
+
it('receiveSQSMessages - throws configurationForListMissing for unknown queue', async () => {
|
|
570
|
+
const acsqs = new ACSQS(makeConfig())
|
|
571
|
+
try {
|
|
572
|
+
await acsqs.receiveSQSMessages({ name: 'unknown' })
|
|
573
|
+
expect.fail('Should have thrown')
|
|
574
|
+
}
|
|
575
|
+
catch(e) {
|
|
576
|
+
expect(e.message).to.eql('configurationForListMissing')
|
|
577
|
+
}
|
|
578
|
+
await acsqs.shutdown()
|
|
579
|
+
})
|
|
580
|
+
|
|
581
|
+
it('deleteSQSMessages - returns early when items list is empty', async () => {
|
|
582
|
+
const acsqs = new ACSQS(makeConfig())
|
|
583
|
+
sinon.stub(acsqs.sqs, 'send')
|
|
584
|
+
const response = await acsqs.deleteSQSMessages({ name, items: [] })
|
|
585
|
+
expect(response).to.be.undefined
|
|
586
|
+
expect(acsqs.sqs.send.called).to.be.false
|
|
587
|
+
await acsqs.shutdown()
|
|
588
|
+
})
|
|
589
|
+
|
|
590
|
+
it('sendSQSMessage - throwError at function level', async () => {
|
|
591
|
+
const acsqs = new ACSQS(makeConfig())
|
|
592
|
+
sinon.stub(acsqs.sqs, 'send').rejects(new Error('ServiceUnavailable'))
|
|
593
|
+
sinon.stub(acsqs.s3, 'send')
|
|
594
|
+
try {
|
|
595
|
+
await acsqs.sendSQSMessage({ name, message: 'test', throwError: true })
|
|
596
|
+
expect.fail('Should have thrown')
|
|
597
|
+
}
|
|
598
|
+
catch(e) {
|
|
599
|
+
expect(e.message).to.eql('ServiceUnavailable')
|
|
600
|
+
}
|
|
601
|
+
await acsqs.shutdown()
|
|
602
|
+
})
|
|
603
|
+
})
|