sqs-consumer 6.1.0 → 6.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/coverage.yml +1 -1
- package/.github/workflows/docs.yml +55 -0
- package/.prettierignore +2 -1
- package/README.md +18 -42
- package/dist/bind.d.ts +4 -0
- package/dist/bind.js +9 -0
- package/dist/consumer.d.ts +90 -21
- package/dist/consumer.js +244 -217
- package/dist/errors.d.ts +13 -1
- package/dist/errors.js +32 -1
- package/dist/types.d.ts +134 -1
- package/dist/types.js +28 -0
- package/dist/validation.d.ts +13 -0
- package/dist/validation.js +36 -0
- package/package.json +13 -13
- package/src/bind.ts +9 -0
- package/src/consumer.ts +289 -279
- package/src/errors.ts +36 -1
- package/src/types.ts +146 -1
- package/src/validation.ts +45 -0
- package/typedoc.json +13 -0
|
@@ -30,6 +30,6 @@ jobs:
|
|
|
30
30
|
- name: Report Coverage
|
|
31
31
|
uses: paambaati/codeclimate-action@v3.2.0
|
|
32
32
|
env:
|
|
33
|
-
CC_TEST_REPORTER_ID:
|
|
33
|
+
CC_TEST_REPORTER_ID: 760097cb88b4c685dce427cf94a8e12a5f082774d06b4f4f5daef839ffc07821
|
|
34
34
|
with:
|
|
35
35
|
coverageCommand: npm run lcov
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Simple workflow for deploying static content to GitHub Pages
|
|
2
|
+
name: Deploy static content to Pages
|
|
3
|
+
|
|
4
|
+
on:
|
|
5
|
+
# Runs on pushes targeting the default branch
|
|
6
|
+
push:
|
|
7
|
+
branches: ["main"]
|
|
8
|
+
|
|
9
|
+
# Allows you to run this workflow manually from the Actions tab
|
|
10
|
+
workflow_dispatch:
|
|
11
|
+
|
|
12
|
+
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
|
|
13
|
+
permissions:
|
|
14
|
+
contents: read
|
|
15
|
+
pages: write
|
|
16
|
+
id-token: write
|
|
17
|
+
|
|
18
|
+
# Allow one concurrent deployment
|
|
19
|
+
concurrency:
|
|
20
|
+
group: "pages"
|
|
21
|
+
cancel-in-progress: true
|
|
22
|
+
|
|
23
|
+
jobs:
|
|
24
|
+
# Single deploy job since we're just deploying
|
|
25
|
+
deploy:
|
|
26
|
+
environment:
|
|
27
|
+
name: github-pages
|
|
28
|
+
url: ${{ steps.deployment.outputs.page_url }}
|
|
29
|
+
runs-on: ubuntu-latest
|
|
30
|
+
steps:
|
|
31
|
+
- name: Checkout
|
|
32
|
+
uses: actions/checkout@v3
|
|
33
|
+
|
|
34
|
+
- name: Setup Node.js
|
|
35
|
+
uses: actions/setup-node@v3
|
|
36
|
+
with:
|
|
37
|
+
node-version: 16.x
|
|
38
|
+
|
|
39
|
+
- name: Install Node Modules
|
|
40
|
+
run: npm ci
|
|
41
|
+
|
|
42
|
+
- name: Build Docs
|
|
43
|
+
run: npm run generate-docs
|
|
44
|
+
|
|
45
|
+
- name: Setup Pages
|
|
46
|
+
uses: actions/configure-pages@v2
|
|
47
|
+
|
|
48
|
+
- name: Upload artifact
|
|
49
|
+
uses: actions/upload-pages-artifact@v1
|
|
50
|
+
with:
|
|
51
|
+
path: './public'
|
|
52
|
+
|
|
53
|
+
- name: Deploy to GitHub Pages
|
|
54
|
+
id: deployment
|
|
55
|
+
uses: actions/deploy-pages@v1
|
package/.prettierignore
CHANGED
package/README.md
CHANGED
|
@@ -2,15 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://npmjs.org/package/sqs-consumer)
|
|
4
4
|
[](https://github.com/bbc/sqs-consumer/actions/workflows/test.yml)
|
|
5
|
-
[](https://codeclimate.com/github/bbc/sqs-consumer/maintainability)
|
|
6
|
+
[](https://codeclimate.com/github/bbc/sqs-consumer/test_coverage)
|
|
7
7
|
|
|
8
8
|
Build SQS-based applications without the boilerplate. Just define an async function that handles the SQS message processing.
|
|
9
9
|
|
|
10
10
|
## Installation
|
|
11
11
|
|
|
12
|
+
To install this package, simply enter the following command into your terminal (or the variant of whatever package manager you are using):
|
|
13
|
+
|
|
12
14
|
```bash
|
|
13
|
-
npm install
|
|
15
|
+
npm install --save-dev sqs-consumer
|
|
14
16
|
```
|
|
15
17
|
|
|
16
18
|
> **Note**
|
|
@@ -43,16 +45,16 @@ app.on('processing_error', (err) => {
|
|
|
43
45
|
app.start();
|
|
44
46
|
```
|
|
45
47
|
|
|
46
|
-
- The queue is polled continuously for messages using [long polling](
|
|
48
|
+
- The queue is polled continuously for messages using [long polling](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html).
|
|
47
49
|
- Messages are deleted from the queue once the handler function has completed successfully.
|
|
48
|
-
- Throwing an error (or returning a rejected promise) from the handler function will cause the message to be left on the queue. An [SQS redrive policy](
|
|
50
|
+
- Throwing an error (or returning a rejected promise) from the handler function will cause the message to be left on the queue. An [SQS redrive policy](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/SQSDeadLetterQueue.html) can be used to move messages that cannot be processed to a dead letter queue.
|
|
49
51
|
- By default messages are processed one at a time – a new message won't be received until the first one has been processed. To process messages in parallel, use the `batchSize` option [detailed below](#options).
|
|
50
52
|
|
|
51
53
|
You can also find some examples of sqs-consumer implemented in various ways within the [examples directory](./examples/).
|
|
52
54
|
|
|
53
55
|
### Credentials
|
|
54
56
|
|
|
55
|
-
By default the consumer will look for AWS credentials in the places [specified by the AWS SDK](
|
|
57
|
+
By default the consumer will look for AWS credentials in the places [specified by the AWS SDK](https://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-configuring.html#Setting_AWS_Credentials). The simplest option is to export your credentials as environment variables:
|
|
56
58
|
|
|
57
59
|
```bash
|
|
58
60
|
export AWS_SECRET_ACCESS_KEY=...
|
|
@@ -102,28 +104,7 @@ Consumer will receive and delete messages from the SQS queue. Ensure `sqs:Receiv
|
|
|
102
104
|
|
|
103
105
|
### `Consumer.create(options)`
|
|
104
106
|
|
|
105
|
-
Creates a new SQS consumer.
|
|
106
|
-
|
|
107
|
-
#### Options
|
|
108
|
-
|
|
109
|
-
| Option | Type | Description |
|
|
110
|
-
| ---------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
111
|
-
| `queueUrl` | String | The SQS queue URL |
|
|
112
|
-
| `region` | String | The AWS region (default `eu-west-1`) |
|
|
113
|
-
| `handleMessage` | Function | An `async` function (or function that returns a `Promise`) to be called whenever a message is received. Receives an SQS message object as its first argument. |
|
|
114
|
-
| `handleMessageBatch` | Function | An `async` function (or function that returns a `Promise`) to be called whenever a batch of messages is received. Similar to `handleMessage` but will receive the list of messages, not each message individually. **If both are set, `handleMessageBatch` overrides `handleMessage`**. In the case that you need to ack only some of the messages, return an array with the successful messages only. |
|
|
115
|
-
| `handleMessageTimeout` | Number | Time in ms to wait for `handleMessage` to process a message before timing out. Emits `timeout_error` on timeout. By default, if `handleMessage` times out, the unprocessed message returns to the end of the queue. |
|
|
116
|
-
| `attributeNames` | Array | List of queue attributes to retrieve (i.e. `['All', 'ApproximateFirstReceiveTimestamp', 'ApproximateReceiveCount']`). |
|
|
117
|
-
| `messageAttributeNames` | Array | List of message attributes to retrieve (i.e. `['name', 'address']`). |
|
|
118
|
-
| `batchSize` | Number | The number of messages to request from SQS when polling (default `1`). This cannot be higher than the [AWS limit of 10](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html). |
|
|
119
|
-
| `visibilityTimeout` | Number | The duration (in seconds) that the received messages are hidden from subsequent retrieve requests after being retrieved by a ReceiveMessage request. |
|
|
120
|
-
| `heartbeatInterval` | Number | The interval (in seconds) between requests to extend the message visibility timeout. On each heartbeat the visibility is extended by adding `visibilityTimeout` to the number of seconds since the start of the handler function. This value must less than `visibilityTimeout`. |
|
|
121
|
-
| `terminateVisibilityTimeout` | Boolean | If true, sets the message visibility timeout to 0 after a `processing_error` (defaults to `false`). |
|
|
122
|
-
| `waitTimeSeconds` | Number | The duration (in seconds) for which the call will wait for a message to arrive in the queue before returning (defaults to `20`). |
|
|
123
|
-
| `authenticationErrorTimeout` | Number | The duration (in milliseconds) to wait before retrying after an authentication error (defaults to `10000`). |
|
|
124
|
-
| `pollingWaitTimeMs` | Number | The duration (in milliseconds) to wait before repolling the queue (defaults to `0`). |
|
|
125
|
-
| `sqs` | SQSClient | An optional [SQS Client](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-sqs/classes/sqsclient.html) object to use if you need to configure the client manually |
|
|
126
|
-
| `shouldDeleteMessages` | Boolean | Default to `true`, if you don't want the package to delete messages from sqs set this to `false` |
|
|
107
|
+
Creates a new SQS consumer using the [defined options](https://bbc.github.io/sqs-consumer/interfaces/ConsumerOptions.html).
|
|
127
108
|
|
|
128
109
|
### `consumer.start()`
|
|
129
110
|
|
|
@@ -131,7 +112,7 @@ Start polling the queue for messages.
|
|
|
131
112
|
|
|
132
113
|
### `consumer.stop()`
|
|
133
114
|
|
|
134
|
-
Stop polling the queue for messages.
|
|
115
|
+
Stop polling the queue for messages (pre existing requests will still be made until concluded).
|
|
135
116
|
|
|
136
117
|
### `consumer.isRunning`
|
|
137
118
|
|
|
@@ -139,19 +120,14 @@ Returns the current polling state of the consumer: `true` if it is actively poll
|
|
|
139
120
|
|
|
140
121
|
### Events
|
|
141
122
|
|
|
142
|
-
Each consumer is an [`EventEmitter`](
|
|
123
|
+
Each consumer is an [`EventEmitter`](https://nodejs.org/api/events.html) and [emits these events](https://bbc.github.io/sqs-consumer/interfaces/Events.html).
|
|
124
|
+
|
|
125
|
+
## Contributing
|
|
126
|
+
|
|
127
|
+
We welcome and appreciate contributions for anyone who would like to take the time to fix a bug or implement a new feature.
|
|
143
128
|
|
|
144
|
-
|
|
145
|
-
| -------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- |
|
|
146
|
-
| `error` | `err`, `[message]` | Fired when an error occurs interacting with the queue. If the error correlates to a message, that message is included in Params |
|
|
147
|
-
| `processing_error` | `err`, `message` | Fired when an error occurs processing the message. |
|
|
148
|
-
| `timeout_error` | `err`, `message` | Fired when `handleMessageTimeout` is supplied as an option and if `handleMessage` times out. |
|
|
149
|
-
| `message_received` | `message` | Fired when a message is received. |
|
|
150
|
-
| `message_processed` | `message` | Fired when a message is successfully processed and removed from the queue. |
|
|
151
|
-
| `response_processed` | None | Fired after one batch of items (up to `batchSize`) has been successfully processed. |
|
|
152
|
-
| `stopped` | None | Fired when the consumer finally stops its work. |
|
|
153
|
-
| `empty` | None | Fired when the queue is empty (All messages have been consumed). |
|
|
129
|
+
But before you get started, [please read the contributing guidelines](https://github.com/bbc/sqs-consumer/blob/main/.github/CONTRIBUTING.md) and [code of conduct](https://github.com/bbc/sqs-consumer/blob/main/.github/CODE_OF_CONDUCT.md).
|
|
154
130
|
|
|
155
|
-
|
|
131
|
+
## License
|
|
156
132
|
|
|
157
|
-
|
|
133
|
+
SQS Consumer is distributed under the Apache License, Version 2.0, see [LICENSE](https://github.com/bbc/sqs-consumer/blob/main/LICENSE) for more information.
|
package/dist/bind.d.ts
CHANGED
package/dist/bind.js
CHANGED
|
@@ -1,9 +1,18 @@
|
|
|
1
1
|
"use strict";
|
|
2
2
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
3
|
exports.autoBind = void 0;
|
|
4
|
+
/**
|
|
5
|
+
* Determines if the property is a method
|
|
6
|
+
* @param propertyName the name of the property
|
|
7
|
+
* @param value the value of the property
|
|
8
|
+
*/
|
|
4
9
|
function isMethod(propertyName, value) {
|
|
5
10
|
return propertyName !== 'constructor' && typeof value === 'function';
|
|
6
11
|
}
|
|
12
|
+
/**
|
|
13
|
+
* Auto binds the provided properties
|
|
14
|
+
* @param obj an object containing the available properties
|
|
15
|
+
*/
|
|
7
16
|
function autoBind(obj) {
|
|
8
17
|
const propertyNames = Object.getOwnPropertyNames(obj.constructor.prototype);
|
|
9
18
|
propertyNames.forEach((propertyName) => {
|
package/dist/consumer.d.ts
CHANGED
|
@@ -1,42 +1,111 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
1
|
+
import { ConsumerOptions, TypedEventEmitter } from './types';
|
|
2
|
+
/**
|
|
3
|
+
* [Usage](https://bbc.github.io/sqs-consumer/index.html#usage)
|
|
4
|
+
*/
|
|
5
|
+
export declare class Consumer extends TypedEventEmitter {
|
|
6
|
+
private pollingTimeoutId;
|
|
7
|
+
private heartbeatTimeoutId;
|
|
8
|
+
private handleMessageTimeoutId;
|
|
9
|
+
private stopped;
|
|
5
10
|
private queueUrl;
|
|
6
11
|
private handleMessage;
|
|
7
12
|
private handleMessageBatch;
|
|
13
|
+
private sqs;
|
|
8
14
|
private handleMessageTimeout;
|
|
9
15
|
private attributeNames;
|
|
10
16
|
private messageAttributeNames;
|
|
11
|
-
private
|
|
17
|
+
private shouldDeleteMessages;
|
|
12
18
|
private batchSize;
|
|
13
19
|
private visibilityTimeout;
|
|
20
|
+
private terminateVisibilityTimeout;
|
|
14
21
|
private waitTimeSeconds;
|
|
15
22
|
private authenticationErrorTimeout;
|
|
16
23
|
private pollingWaitTimeMs;
|
|
17
|
-
private terminateVisibilityTimeout;
|
|
18
24
|
private heartbeatInterval;
|
|
19
|
-
private sqs;
|
|
20
|
-
private shouldDeleteMessages;
|
|
21
25
|
constructor(options: ConsumerOptions);
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
get isRunning(): boolean;
|
|
26
|
+
/**
|
|
27
|
+
* Creates a new SQS consumer.
|
|
28
|
+
*/
|
|
26
29
|
static create(options: ConsumerOptions): Consumer;
|
|
30
|
+
/**
|
|
31
|
+
* Start polling the queue for messages.
|
|
32
|
+
*/
|
|
27
33
|
start(): void;
|
|
34
|
+
/**
|
|
35
|
+
* Stop polling the queue for messages (pre existing requests will still be made until concluded).
|
|
36
|
+
*/
|
|
28
37
|
stop(): void;
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
38
|
+
/**
|
|
39
|
+
* Returns the current polling state of the consumer: `true` if it is actively polling, `false` if it is not.
|
|
40
|
+
*/
|
|
41
|
+
get isRunning(): boolean;
|
|
42
|
+
/**
|
|
43
|
+
* Emit one of the consumer's error events depending on the error received.
|
|
44
|
+
* @param err The error object to forward on
|
|
45
|
+
* @param message The message that the error occurred on
|
|
46
|
+
*/
|
|
35
47
|
private emitError;
|
|
48
|
+
/**
|
|
49
|
+
* Poll for new messages from SQS
|
|
50
|
+
*/
|
|
36
51
|
private poll;
|
|
52
|
+
/**
|
|
53
|
+
* Send a request to SQS to retrieve messages
|
|
54
|
+
* @param params The required params to receive messages from SQS
|
|
55
|
+
*/
|
|
56
|
+
private receiveMessage;
|
|
57
|
+
/**
|
|
58
|
+
* Handles the response from AWS SQS, determining if we should proceed to
|
|
59
|
+
* the message handler.
|
|
60
|
+
* @param response The output from AWS SQS
|
|
61
|
+
*/
|
|
62
|
+
private handleSqsResponse;
|
|
63
|
+
/**
|
|
64
|
+
* Process a message that has been received from SQS. This will execute the message
|
|
65
|
+
* handler and delete the message once complete.
|
|
66
|
+
* @param message The message that was delivered from SQS
|
|
67
|
+
*/
|
|
68
|
+
private processMessage;
|
|
69
|
+
/**
|
|
70
|
+
* Process a batch of messages from the SQS queue.
|
|
71
|
+
* @param messages The messages that were delivered from SQS
|
|
72
|
+
*/
|
|
37
73
|
private processMessageBatch;
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
74
|
+
/**
|
|
75
|
+
* Trigger a function on a set interval
|
|
76
|
+
* @param heartbeatFn The function that should be triggered
|
|
77
|
+
*/
|
|
41
78
|
private startHeartbeat;
|
|
79
|
+
/**
|
|
80
|
+
* Change the visibility timeout on a message
|
|
81
|
+
* @param message The message to change the value of
|
|
82
|
+
* @param timeout The new timeout that should be set
|
|
83
|
+
*/
|
|
84
|
+
private changeVisibilityTimeout;
|
|
85
|
+
/**
|
|
86
|
+
* Change the visibility timeout on a batch of messages
|
|
87
|
+
* @param messages The messages to change the value of
|
|
88
|
+
* @param timeout The new timeout that should be set
|
|
89
|
+
*/
|
|
90
|
+
private changeVisibilityTimeoutBatch;
|
|
91
|
+
/**
|
|
92
|
+
* Trigger the applications handleMessage function
|
|
93
|
+
* @param message The message that was received from SQS
|
|
94
|
+
*/
|
|
95
|
+
private executeHandler;
|
|
96
|
+
/**
|
|
97
|
+
* Execute the application's message batch handler
|
|
98
|
+
* @param messages The messages that should be forwarded from the SQS queue
|
|
99
|
+
*/
|
|
100
|
+
private executeBatchHandler;
|
|
101
|
+
/**
|
|
102
|
+
* Delete a single message from SQS
|
|
103
|
+
* @param message The message to delete from the SQS queue
|
|
104
|
+
*/
|
|
105
|
+
private deleteMessage;
|
|
106
|
+
/**
|
|
107
|
+
* Delete a batch of messages from the SQS queue.
|
|
108
|
+
* @param messages The messages that should be deleted from SQS
|
|
109
|
+
*/
|
|
110
|
+
private deleteMessageBatch;
|
|
42
111
|
}
|