@or-sdk/contacts 4.0.2-beta.2365.0 → 4.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/README.md +37 -0
- package/dist/cjs/api/baseWithPolingApi.js +28 -12
- package/dist/cjs/api/baseWithPolingApi.js.map +1 -1
- package/dist/cjs/api/bulkContactsCreateApi.js +152 -123
- package/dist/cjs/api/bulkContactsCreateApi.js.map +1 -1
- package/dist/cjs/constants.js +3 -2
- package/dist/cjs/constants.js.map +1 -1
- package/dist/cjs/utils.js +10 -2
- package/dist/cjs/utils.js.map +1 -1
- package/dist/esm/api/baseWithPolingApi.js +15 -7
- package/dist/esm/api/baseWithPolingApi.js.map +1 -1
- package/dist/esm/api/bulkContactsCreateApi.js +82 -54
- package/dist/esm/api/bulkContactsCreateApi.js.map +1 -1
- package/dist/esm/constants.js +2 -1
- package/dist/esm/constants.js.map +1 -1
- package/dist/esm/utils.js +7 -1
- package/dist/esm/utils.js.map +1 -1
- package/dist/types/api/baseWithPolingApi.d.ts +4 -1
- package/dist/types/api/baseWithPolingApi.d.ts.map +1 -1
- package/dist/types/api/bulkContactsCreateApi.d.ts +2 -2
- package/dist/types/api/bulkContactsCreateApi.d.ts.map +1 -1
- package/dist/types/constants.d.ts +2 -1
- package/dist/types/constants.d.ts.map +1 -1
- package/dist/types/types.d.ts +9 -1
- package/dist/types/types.d.ts.map +1 -1
- package/dist/types/utils.d.ts +1 -0
- package/dist/types/utils.d.ts.map +1 -1
- package/package.json +4 -3
- package/src/api/baseWithPolingApi.ts +22 -7
- package/src/api/bulkContactsCreateApi.ts +141 -64
- package/src/constants.ts +3 -1
- package/src/types.ts +9 -1
- package/src/utils.ts +8 -1
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,15 @@
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
|
|
5
5
|
|
|
6
|
+
## [4.0.2](https://gitlab.internal.onereach.io/onereach/platform/or-sdk-next/compare/@or-sdk/contacts@4.0.1...@or-sdk/contacts@4.0.2) (2024-01-31)
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
### Bug Fixes
|
|
10
|
+
|
|
11
|
+
* **contacts:** fix bulk gateway timeout ([f37f6c3](https://gitlab.internal.onereach.io/onereach/platform/or-sdk-next/commit/f37f6c3267a52415a4a91a9585844420eee94331))
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
|
|
6
15
|
## [4.0.1](https://gitlab.internal.onereach.io/onereach/platform/or-sdk-next/compare/@or-sdk/contacts@4.0.0...@or-sdk/contacts@4.0.1) (2024-01-30)
|
|
7
16
|
|
|
8
17
|
|
package/README.md
CHANGED
|
@@ -100,4 +100,41 @@ for await (const bulkResults of bulkCreateApi.trackBulkCreateContacts('bulkName'
|
|
|
100
100
|
}
|
|
101
101
|
|
|
102
102
|
```
|
|
103
|
+
|
|
104
|
+
## Tuning bulk
|
|
105
|
+
|
|
106
|
+
Create bulk consumes lots of resources, in terms of the quantity of launched lambdas (it's huge!), and in terms of created workload on the DB (DB server memory, CPU, connections to the DB).
|
|
107
|
+
So basically it means that the more activities are taking place at particular point of time the higher is the probability of bulk failure. It is strongly recommended to execute the bulk create operation during non working hours!
|
|
108
|
+
|
|
109
|
+
|
|
110
|
+
Bulk provides some interface for tuning the workload by means of the `options` argument. With the default options, the bulk successfully completes 200k contacts for approximately 5 mins. Tuning options may result into the bulk operation performance degrade, however it might dramatically reduce workload on the system and turn failed bulk into successful one!
|
|
111
|
+
|
|
112
|
+
Here is the short description of the bulk create contacts mechanism to understand how to affect it by tuning options:
|
|
113
|
+
- bulk examines the size of passed data and if it is more that 4000000 bytes the data is broken into array of data with the same type, however each element of this array is less than the allowed batchSize; each element of this array is called `batch`
|
|
114
|
+
- bulk executes 4 (by default, might be tuned with the `parallelBatchesAmount`) batches in parallel - we call it `batches series`; **executing** here means sending the batch data to the server where contacts, actually, are created and inserted into DB
|
|
115
|
+
- batch execution duration can't be determined and to avoid the gateway timeout error the execution results are poled. Every poling attempt are repeated in the amount of time specified by `polingDuration`
|
|
116
|
+
- if batches series fails (it might happen due to DB server overload, for example), one more attempt is made after the amount of time specified with `repeatFailedParallelBatchesIn`
|
|
117
|
+
- when all batches are executed, bulk results is formed
|
|
118
|
+
|
|
119
|
+
Here is the options:
|
|
120
|
+
```js
|
|
121
|
+
{
|
|
122
|
+
|
|
123
|
+
batchSize?: number; // default is 4000000 bytes.
|
|
124
|
+
// Reducing this number is the very first step you should try.
|
|
125
|
+
parallelBatchesAmount?: number; // default is 4. Reducing it also improves bulk reliability,
|
|
126
|
+
// however it is making it slower.
|
|
127
|
+
batchesToProcess?: number; // default is ALL batches. If specified, executes only the number
|
|
128
|
+
//of batches passed here.
|
|
129
|
+
polingDuration?: number; // default is 15000 milliseconds,
|
|
130
|
+
//specifies the pause duration between poling batch process results
|
|
131
|
+
repeatFailedParallelBatchesIn?: number; // default is 30000 milliseconds (30 seconds) -
|
|
132
|
+
// the amount of time bulk waits until repeating an attempt
|
|
133
|
+
// to execute again a failed series of batches
|
|
134
|
+
logger?: BulkLogger; // logger, pass here an object that has log() function;
|
|
135
|
+
// to be more precise, the log function should be of the following type:
|
|
136
|
+
// (message?: unknown, ...optionalParams: unknown[]) => void
|
|
137
|
+
};
|
|
138
|
+
```
|
|
139
|
+
|
|
103
140
|
|
|
@@ -64,29 +64,45 @@ var BaseWithPoling = (function (_super) {
|
|
|
64
64
|
_this.batchProcessApi = batchProcessApi;
|
|
65
65
|
return _this;
|
|
66
66
|
}
|
|
67
|
-
BaseWithPoling.prototype.
|
|
67
|
+
BaseWithPoling.prototype.setLogger = function (logger) {
|
|
68
|
+
this.logger = logger;
|
|
69
|
+
};
|
|
70
|
+
BaseWithPoling.prototype.polling = function (batchId, polingDuration, repeats) {
|
|
71
|
+
var _a;
|
|
72
|
+
if (polingDuration === void 0) { polingDuration = constants_1.BULK_POLING_PAUSE_DURATION; }
|
|
68
73
|
if (repeats === void 0) { repeats = 0; }
|
|
69
74
|
return __awaiter(this, void 0, void 0, function () {
|
|
70
75
|
var batchProcess, e_1;
|
|
71
76
|
var _this = this;
|
|
72
|
-
return __generator(this, function (
|
|
73
|
-
switch (
|
|
77
|
+
return __generator(this, function (_b) {
|
|
78
|
+
switch (_b.label) {
|
|
74
79
|
case 0:
|
|
75
|
-
|
|
76
|
-
return [4, (0, utils_1.debouncePromise)(function () { return _this.batchProcessApi.getBatchProcess(batchId); },
|
|
80
|
+
_b.trys.push([0, 2, , 5]);
|
|
81
|
+
return [4, (0, utils_1.debouncePromise)(function () { return _this.batchProcessApi.getBatchProcess(batchId); }, polingDuration)];
|
|
77
82
|
case 1:
|
|
78
|
-
batchProcess =
|
|
83
|
+
batchProcess = _b.sent();
|
|
79
84
|
return [3, 5];
|
|
80
85
|
case 2:
|
|
81
|
-
e_1 =
|
|
86
|
+
e_1 = _b.sent();
|
|
82
87
|
if (!(repeats < constants_1.FAILED_REQUEST_REPEATS
|
|
83
88
|
&& 'statusCode' in e_1
|
|
84
|
-
&& e_1.statusCode ===
|
|
85
|
-
|
|
86
|
-
|
|
89
|
+
&& e_1.statusCode === 'ERR_BAD_REQUEST')) return [3, 4];
|
|
90
|
+
(_a = this.logger) === null || _a === void 0 ? void 0 : _a.log(batchId, ' NOT FOUND IN POLING');
|
|
91
|
+
return [4, (0, utils_1.debouncePromise)(function () { return __awaiter(_this, void 0, void 0, function () {
|
|
92
|
+
var result;
|
|
93
|
+
return __generator(this, function (_a) {
|
|
94
|
+
switch (_a.label) {
|
|
95
|
+
case 0: return [4, this.polling(batchId, polingDuration, repeats + 1)];
|
|
96
|
+
case 1:
|
|
97
|
+
result = _a.sent();
|
|
98
|
+
return [2, result];
|
|
99
|
+
}
|
|
100
|
+
});
|
|
101
|
+
}); }, (repeats + 1) * polingDuration)];
|
|
102
|
+
case 3: return [2, _b.sent()];
|
|
87
103
|
case 4:
|
|
88
104
|
if (repeats < constants_1.FAILED_REQUEST_REPEATS) {
|
|
89
|
-
return [2, this.polling(batchId, repeats + 1)];
|
|
105
|
+
return [2, this.polling(batchId, polingDuration, repeats + 1)];
|
|
90
106
|
}
|
|
91
107
|
throw new apiError_1.CreateContactsBatchError(e_1.message, batchId);
|
|
92
108
|
case 5:
|
|
@@ -94,7 +110,7 @@ var BaseWithPoling = (function (_super) {
|
|
|
94
110
|
throw new apiError_1.CreateContactsBatchError('Could not complete batch process', batchId, batchProcess.messages);
|
|
95
111
|
}
|
|
96
112
|
return [2, batchProcess.status === types_contacts_api_1.BatchProcessStatus.pending
|
|
97
|
-
? this.polling(batchId)
|
|
113
|
+
? this.polling(batchId, polingDuration)
|
|
98
114
|
: batchProcess];
|
|
99
115
|
}
|
|
100
116
|
});
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"baseWithPolingApi.js","sourceRoot":"","sources":["../../../src/api/baseWithPolingApi.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AACA,qCAAoC;AAEpC,mEAA2F;AAC3F,kCAA2C;AAC3C,
|
|
1
|
+
{"version":3,"file":"baseWithPolingApi.js","sourceRoot":"","sources":["../../../src/api/baseWithPolingApi.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AACA,qCAAoC;AAEpC,mEAA2F;AAC3F,kCAA2C;AAC3C,0CAAkF;AAClF,wCAAiE;AAGjE;IAA4C,kCAAO;IAGjD,wBACqB,OAAgD,EACzD,eAAgC;QAF5C,YAIE,kBAAM,OAAO,CAAC,SACf;QAJoB,aAAO,GAAP,OAAO,CAAyC;QACzD,qBAAe,GAAf,eAAe,CAAiB;;IAG5C,CAAC;IAES,kCAAS,GAAnB,UAAoB,MAA8B;QAChD,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;IACvB,CAAC;IAKe,gCAAO,GAAvB,UACE,OAAe,EACf,cAA2C,EAC3C,OAAW;;QADX,+BAAA,EAAA,iBAAiB,sCAA0B;QAC3C,wBAAA,EAAA,WAAW;;;;;;;;wBAKM,WAAM,IAAA,uBAAe,EAAC,cAAM,OAAA,KAAI,CAAC,eAAe,CAAC,eAAe,CAAC,OAAO,CAAC,EAA7C,CAA6C,EAAE,cAAc,CAAC,EAAA;;wBAAzG,YAAY,GAAG,SAA0F,CAAC;;;;6BAGxG,CAAA,OAAO,GAAG,kCAAsB;+BAC7B,YAAY,IAAK,GAAY;+BAC5B,GAAyB,CAAC,UAAU,KAAK,iBAAiB,CAAA,EAF9D,cAE8D;wBAE9D,MAAA,IAAI,CAAC,MAAM,0CAAE,GAAG,CAAC,OAAO,EAAE,sBAAsB,CAAC,CAAC;wBAC3C,WAAM,IAAA,uBAAe,EAAC;;;;gDACZ,WAAM,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,cAAc,EAAE,OAAO,GAAG,CAAC,CAAC,EAAA;;4CAAjE,MAAM,GAAG,SAAwD;4CACvE,WAAO,MAAM,EAAC;;;iCACf,EAAE,CAAC,OAAO,GAAG,CAAC,CAAC,GAAG,cAAc,CAAC,EAAA;4BAHlC,WAAO,SAG2B,EAAC;;wBAErC,IAAI,OAAO,GAAG,kCAAsB,EAAE;4BACpC,WAAO,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,cAAc,EAAE,OAAO,GAAG,CAAC,CAAC,EAAC;yBAC3D;wBACD,MAAM,IAAI,mCAAwB,CAAE,GAAsB,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;;wBAG/E,IAAI,YAAY,CAAC,MAAM,KAAK,uCAAkB,CAAC,MAAM,EAAE;4BACrD,MAAM,IAAI,mCAAwB,CAChC,kCAAkC,EAClC,OAAO,EACP,YAAY,CAAC,QAAQ,CACtB,CAAC;yBACH;wBAED,WAAO,YAAY,CAAC,MAAM,KAAK,uCAAkB,CAAC,OAAO;gCACvD,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,cAAc,CAAC;gCACvC,CAAC,CAAC,YAAY,EAAC;;;;KAClB;IACH,qBAAC;AAAD,CAAC,AAxDD,CAA4C,iBAAO,GAwDlD"}
|
|
@@ -153,73 +153,89 @@ var BulkContactsCreateApi = (function (_super) {
|
|
|
153
153
|
});
|
|
154
154
|
});
|
|
155
155
|
};
|
|
156
|
+
BulkContactsCreateApi.prototype.logBulkDetails = function (size, totalContacts, totalBatches, options) {
|
|
157
|
+
options.logger.log('data size = ', size, ' bytes');
|
|
158
|
+
options.logger.log('total contacts = ', totalContacts);
|
|
159
|
+
options.logger.log('batches amount = ', totalBatches);
|
|
160
|
+
options.logger.log('average batch size ~ ', Math.ceil(totalContacts / totalBatches), ' contacts');
|
|
161
|
+
options.logger.log('parallel batches size = ', options.parallelBatchesAmount || constants_1.MAX_PARALLEL_BATCH_PROCESSING);
|
|
162
|
+
options.logger.log('batch size = ', options.batchSize || constants_1.REQUEST_PAYLOAD_MAX_BYTES, ' bytes');
|
|
163
|
+
};
|
|
156
164
|
BulkContactsCreateApi.prototype.trackBulkCreateContacts = function (bulkName, data, options) {
|
|
165
|
+
var _a, _b;
|
|
157
166
|
if (options === void 0) { options = {}; }
|
|
158
167
|
return __asyncGenerator(this, arguments, function trackBulkCreateContacts_1() {
|
|
159
|
-
var dataWithContactKeys, contacts, rest, contactsMaxSize, contactsChunks,
|
|
160
|
-
var e_2,
|
|
161
|
-
return __generator(this, function (
|
|
162
|
-
switch (
|
|
168
|
+
var startTime, dataWithContactKeys, contacts, rest, contactsMaxSize, contactsChunks, results, _c, _d, results, endTime, e_2_1;
|
|
169
|
+
var e_2, _e;
|
|
170
|
+
return __generator(this, function (_f) {
|
|
171
|
+
switch (_f.label) {
|
|
163
172
|
case 0:
|
|
173
|
+
this.setLogger(options.logger);
|
|
174
|
+
(_a = options.logger) === null || _a === void 0 ? void 0 : _a.log('Starting ', bulkName, ' bulk');
|
|
175
|
+
startTime = performance.now();
|
|
164
176
|
dataWithContactKeys = this.updateContactsWithContactKeys(data);
|
|
165
|
-
_d.label = 1;
|
|
166
|
-
case 1:
|
|
167
|
-
_d.trys.push([1, , 17, 18]);
|
|
168
177
|
contacts = dataWithContactKeys.contacts, rest = __rest(dataWithContactKeys, ["contacts"]);
|
|
169
178
|
contactsMaxSize = options.batchSize || constants_1.REQUEST_PAYLOAD_MAX_BYTES - (0, utils_1.getObjectSizeInBytes)(__assign({}, rest));
|
|
170
179
|
contactsChunks = (0, utils_1.chunkArrByMaxSize)(contacts, contactsMaxSize);
|
|
171
|
-
if (
|
|
172
|
-
|
|
180
|
+
if (options.logger) {
|
|
181
|
+
this.logBulkDetails((0, utils_1.getObjectSizeInBytes)(data), contacts.length, contactsChunks.length, options);
|
|
182
|
+
}
|
|
183
|
+
_f.label = 1;
|
|
184
|
+
case 1:
|
|
185
|
+
_f.trys.push([1, , 19, 20]);
|
|
186
|
+
if (!(contactsChunks.length === 1 || options.batchesToProcess === 1)) return [3, 5];
|
|
187
|
+
return [4, __await(this.createContactsInSingleBatch(bulkName, __assign(__assign({}, dataWithContactKeys), { batchIdx: 0 }), options))];
|
|
173
188
|
case 2:
|
|
174
|
-
|
|
175
|
-
return [3, 16];
|
|
176
|
-
case 3:
|
|
177
|
-
_d.trys.push([3, 10, 11, 16]);
|
|
178
|
-
_a = __asyncValues(this.createContactsInMultiBatches(contactsChunks, rest, bulkName, options));
|
|
179
|
-
_d.label = 4;
|
|
180
|
-
case 4: return [4, __await(_a.next())];
|
|
181
|
-
case 5:
|
|
182
|
-
if (!(_b = _d.sent(), !_b.done)) return [3, 9];
|
|
183
|
-
progress = _b.value;
|
|
189
|
+
results = _f.sent();
|
|
184
190
|
return [4, __await({
|
|
185
|
-
type: '
|
|
186
|
-
results:
|
|
191
|
+
type: 'results',
|
|
192
|
+
results: this.mergeResults({
|
|
193
|
+
created: {},
|
|
194
|
+
failed: {},
|
|
195
|
+
}, [results]),
|
|
187
196
|
})];
|
|
188
|
-
case
|
|
197
|
+
case 3: return [4, _f.sent()];
|
|
198
|
+
case 4:
|
|
199
|
+
_f.sent();
|
|
200
|
+
return [3, 18];
|
|
201
|
+
case 5:
|
|
202
|
+
_f.trys.push([5, 12, 13, 18]);
|
|
203
|
+
_c = __asyncValues(this.createContactsInMultiBatches(contactsChunks, rest, bulkName, options));
|
|
204
|
+
_f.label = 6;
|
|
205
|
+
case 6: return [4, __await(_c.next())];
|
|
189
206
|
case 7:
|
|
190
|
-
_d.sent();
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
case
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
207
|
+
if (!(_d = _f.sent(), !_d.done)) return [3, 11];
|
|
208
|
+
results = _d.value;
|
|
209
|
+
if (results.type === 'results') {
|
|
210
|
+
endTime = performance.now();
|
|
211
|
+
(_b = options.logger) === null || _b === void 0 ? void 0 : _b.log("Bulk ".concat(bulkName, " completed for ").concat(Math.floor((endTime - startTime) / 1000), " seconds"));
|
|
212
|
+
}
|
|
213
|
+
return [4, __await(results)];
|
|
214
|
+
case 8: return [4, _f.sent()];
|
|
215
|
+
case 9:
|
|
216
|
+
_f.sent();
|
|
217
|
+
_f.label = 10;
|
|
218
|
+
case 10: return [3, 6];
|
|
219
|
+
case 11: return [3, 18];
|
|
202
220
|
case 12:
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
221
|
+
e_2_1 = _f.sent();
|
|
222
|
+
e_2 = { error: e_2_1 };
|
|
223
|
+
return [3, 18];
|
|
224
|
+
case 13:
|
|
225
|
+
_f.trys.push([13, , 16, 17]);
|
|
226
|
+
if (!(_d && !_d.done && (_e = _c.return))) return [3, 15];
|
|
227
|
+
return [4, __await(_e.call(_c))];
|
|
206
228
|
case 14:
|
|
229
|
+
_f.sent();
|
|
230
|
+
_f.label = 15;
|
|
231
|
+
case 15: return [3, 17];
|
|
232
|
+
case 16:
|
|
207
233
|
if (e_2) throw e_2.error;
|
|
208
234
|
return [7];
|
|
209
|
-
case 15: return [7];
|
|
210
|
-
case 16: return [3, 18];
|
|
211
235
|
case 17: return [7];
|
|
212
|
-
case 18: return [
|
|
213
|
-
case 19:
|
|
214
|
-
|
|
215
|
-
return [4, __await({
|
|
216
|
-
type: 'results',
|
|
217
|
-
results: results,
|
|
218
|
-
})];
|
|
219
|
-
case 20: return [4, _d.sent()];
|
|
220
|
-
case 21:
|
|
221
|
-
_d.sent();
|
|
222
|
-
return [2];
|
|
236
|
+
case 18: return [3, 20];
|
|
237
|
+
case 19: return [7];
|
|
238
|
+
case 20: return [2];
|
|
223
239
|
}
|
|
224
240
|
});
|
|
225
241
|
});
|
|
@@ -284,110 +300,123 @@ var BulkContactsCreateApi = (function (_super) {
|
|
|
284
300
|
}
|
|
285
301
|
});
|
|
286
302
|
};
|
|
287
|
-
BulkContactsCreateApi.prototype.
|
|
303
|
+
BulkContactsCreateApi.prototype.createContactsInSingleBatch = function (batchGroupId, singleBatchData, options) {
|
|
304
|
+
var _a;
|
|
288
305
|
return __awaiter(this, void 0, void 0, function () {
|
|
289
|
-
var
|
|
306
|
+
var batchId, batchIdx, data, batch;
|
|
307
|
+
var _this = this;
|
|
290
308
|
return __generator(this, function (_b) {
|
|
291
309
|
switch (_b.label) {
|
|
292
|
-
case 0: return [4, this.callGetBulkResults(batchGroupId, 0)];
|
|
293
|
-
case 1:
|
|
294
|
-
_a = _b.sent(), total = _a.total, firstResults = _a.results, size = _a.size;
|
|
295
|
-
results = __assign({}, firstResults);
|
|
296
|
-
pageSize = options.bulkResultsPageSize || constants_1.BULK_RESULT_PAGE_SIZE;
|
|
297
|
-
if (!(total > size)) return [3, 5];
|
|
298
|
-
i = 1;
|
|
299
|
-
_b.label = 2;
|
|
300
|
-
case 2:
|
|
301
|
-
if (!(i <= Math.ceil((total - size) / size))) return [3, 5];
|
|
302
|
-
return [4, this.callGetBulkResults(batchGroupId, i * pageSize)];
|
|
303
|
-
case 3:
|
|
304
|
-
nextResults = (_b.sent()).results;
|
|
305
|
-
results = {
|
|
306
|
-
created: __assign(__assign({}, results.created), nextResults.created),
|
|
307
|
-
failed: __assign(__assign({}, results.failed), nextResults.created),
|
|
308
|
-
};
|
|
309
|
-
_b.label = 4;
|
|
310
|
-
case 4:
|
|
311
|
-
i++;
|
|
312
|
-
return [3, 2];
|
|
313
|
-
case 5: return [2, results];
|
|
314
|
-
}
|
|
315
|
-
});
|
|
316
|
-
});
|
|
317
|
-
};
|
|
318
|
-
BulkContactsCreateApi.prototype.callGetBulkResults = function (batchGroupId, skip) {
|
|
319
|
-
return __awaiter(this, void 0, void 0, function () {
|
|
320
|
-
return __generator(this, function (_a) {
|
|
321
|
-
return [2, this.apiCall({
|
|
322
|
-
method: 'GET',
|
|
323
|
-
route: "".concat(this.apiBasePath, "/bulk-results/").concat(batchGroupId, "/").concat(skip),
|
|
324
|
-
})];
|
|
325
|
-
});
|
|
326
|
-
});
|
|
327
|
-
};
|
|
328
|
-
BulkContactsCreateApi.prototype.createContactsInSingleBatch = function (data, batchGroupId) {
|
|
329
|
-
return __awaiter(this, void 0, void 0, function () {
|
|
330
|
-
var batchId;
|
|
331
|
-
var _this = this;
|
|
332
|
-
return __generator(this, function (_a) {
|
|
333
|
-
switch (_a.label) {
|
|
334
310
|
case 0:
|
|
335
311
|
batchId = (0, uuid_1.v4)();
|
|
312
|
+
batchIdx = singleBatchData.batchIdx, data = __rest(singleBatchData, ["batchIdx"]);
|
|
336
313
|
this.apiCall({
|
|
337
314
|
method: 'POST',
|
|
338
315
|
route: "".concat(this.apiBasePath, "/bulk"),
|
|
339
316
|
data: __assign(__assign({}, data), { batchId: batchId, batchGroupId: batchGroupId }),
|
|
340
317
|
});
|
|
341
|
-
return [4, (0, utils_1.debouncePromise)(function () { return _this.polling(batchId); },
|
|
318
|
+
return [4, (0, utils_1.debouncePromise)(function () { return _this.polling(batchId, options.polingDuration); }, 5000)];
|
|
342
319
|
case 1:
|
|
343
|
-
|
|
344
|
-
|
|
320
|
+
batch = _b.sent();
|
|
321
|
+
(_a = options.logger) === null || _a === void 0 ? void 0 : _a.log('Completed ', batchIdx, '-th batch');
|
|
322
|
+
return [2, batch];
|
|
345
323
|
}
|
|
346
324
|
});
|
|
347
325
|
});
|
|
348
326
|
};
|
|
327
|
+
BulkContactsCreateApi.prototype.mergeResults = function (current, batchProcesses) {
|
|
328
|
+
return batchProcesses.reduce(function (acc, batch) {
|
|
329
|
+
var createdContacts = batch.results.reduce(function (acc, result) { return (0, utils_1.mergeObjects)(acc, JSON.parse(result)); }, {});
|
|
330
|
+
var failedContacts = batch.messages.reduce(function (acc, message) { return (0, utils_1.mergeObjects)(acc, JSON.parse(message)); }, {});
|
|
331
|
+
return {
|
|
332
|
+
created: __assign(__assign({}, acc.created), createdContacts),
|
|
333
|
+
failed: __assign(__assign({}, acc.failed), failedContacts),
|
|
334
|
+
};
|
|
335
|
+
}, current);
|
|
336
|
+
};
|
|
349
337
|
BulkContactsCreateApi.prototype.createContactsInMultiBatches = function (contactsBatches, data, batchGroupId, options) {
|
|
338
|
+
var _a;
|
|
350
339
|
return __asyncGenerator(this, arguments, function createContactsInMultiBatches_1() {
|
|
351
|
-
var batchPromises, parallelBatchesAmount,
|
|
340
|
+
var repeatIn, batchPromises, progress, results, parallelBatchesAmount, _loop_1, this_1, i;
|
|
352
341
|
var _this = this;
|
|
353
|
-
return __generator(this, function (
|
|
354
|
-
switch (
|
|
342
|
+
return __generator(this, function (_b) {
|
|
343
|
+
switch (_b.label) {
|
|
355
344
|
case 0:
|
|
345
|
+
repeatIn = options.repeatFailedParallelBatchesIn || constants_1.BULK_FAILED_SERIES_REPEAT_IN;
|
|
356
346
|
batchPromises = contactsBatches
|
|
357
|
-
.map(function (chunkContacts) { return function () { return
|
|
358
|
-
return __generator(this, function (_a) {
|
|
359
|
-
return [2, this.createContactsInSingleBatch(__assign({ contacts: chunkContacts }, data), batchGroupId)];
|
|
360
|
-
});
|
|
361
|
-
}); }; });
|
|
362
|
-
parallelBatchesAmount = options.parallelBatchesAmount || constants_1.MAX_PARALLEL_BATCH_PROCESSING;
|
|
347
|
+
.map(function (chunkContacts, batchIdx) { return function () { return _this.createContactsInSingleBatch(batchGroupId, __assign({ batchIdx: batchIdx, contacts: chunkContacts }, data), options); }; });
|
|
363
348
|
progress = {
|
|
364
349
|
totalBatches: contactsBatches.length,
|
|
365
350
|
totalContacts: contactsBatches.map(function (b) { return b.length; }).reduce(function (acc, curr) { return acc + curr; }, 0),
|
|
366
351
|
completedBatches: 0,
|
|
367
352
|
completedContacts: 0,
|
|
368
353
|
};
|
|
354
|
+
results = {
|
|
355
|
+
created: {},
|
|
356
|
+
failed: {},
|
|
357
|
+
};
|
|
358
|
+
parallelBatchesAmount = options.parallelBatchesAmount || constants_1.MAX_PARALLEL_BATCH_PROCESSING;
|
|
359
|
+
_loop_1 = function (i) {
|
|
360
|
+
var startIdx, endIdx, batchesSeries, seriesResults, e_3;
|
|
361
|
+
return __generator(this, function (_c) {
|
|
362
|
+
switch (_c.label) {
|
|
363
|
+
case 0:
|
|
364
|
+
startIdx = i;
|
|
365
|
+
endIdx = i + parallelBatchesAmount;
|
|
366
|
+
batchesSeries = batchPromises.slice(i, i + parallelBatchesAmount);
|
|
367
|
+
seriesResults = [];
|
|
368
|
+
_c.label = 1;
|
|
369
|
+
case 1:
|
|
370
|
+
_c.trys.push([1, 3, , 5]);
|
|
371
|
+
return [4, __await(Promise.all(batchesSeries.map(function (p) { return p(); })))];
|
|
372
|
+
case 2:
|
|
373
|
+
seriesResults = _c.sent();
|
|
374
|
+
return [3, 5];
|
|
375
|
+
case 3:
|
|
376
|
+
e_3 = _c.sent();
|
|
377
|
+
(_a = options.logger) === null || _a === void 0 ? void 0 : _a.log("Failed to execute of ".concat(parallelBatchesAmount, " batches, from ").concat(startIdx, " to ").concat(endIdx, ", repeating attempt..."));
|
|
378
|
+
return [4, __await((0, utils_1.debouncePromise)(function () { return Promise.all(batchesSeries.map(function (p) { return p(); })); }, repeatIn))];
|
|
379
|
+
case 4:
|
|
380
|
+
seriesResults = _c.sent();
|
|
381
|
+
return [3, 5];
|
|
382
|
+
case 5:
|
|
383
|
+
results = this_1.mergeResults(results, seriesResults);
|
|
384
|
+
progress.completedBatches += batchesSeries.length;
|
|
385
|
+
progress.completedContacts += contactsBatches
|
|
386
|
+
.slice(i, i + parallelBatchesAmount)
|
|
387
|
+
.map(function (batch) { return batch.length; })
|
|
388
|
+
.reduce(function (acc, curr) { return acc + curr; }, 0);
|
|
389
|
+
return [4, __await({
|
|
390
|
+
type: 'progress',
|
|
391
|
+
results: progress,
|
|
392
|
+
})];
|
|
393
|
+
case 6: return [4, _c.sent()];
|
|
394
|
+
case 7:
|
|
395
|
+
_c.sent();
|
|
396
|
+
return [2];
|
|
397
|
+
}
|
|
398
|
+
});
|
|
399
|
+
};
|
|
400
|
+
this_1 = this;
|
|
369
401
|
i = 0;
|
|
370
|
-
|
|
402
|
+
_b.label = 1;
|
|
371
403
|
case 1:
|
|
372
|
-
if (!(i < (options.batchesToProcess || batchPromises.length))) return [3,
|
|
373
|
-
|
|
374
|
-
return [4, __await(Promise.all(batch.map(function (p) { return p(); })))];
|
|
404
|
+
if (!(i < (options.batchesToProcess || batchPromises.length))) return [3, 4];
|
|
405
|
+
return [5, _loop_1(i)];
|
|
375
406
|
case 2:
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
.slice(i, i + parallelBatchesAmount)
|
|
380
|
-
.map(function (batch) { return batch.length; })
|
|
381
|
-
.reduce(function (acc, curr) { return acc + curr; }, 0);
|
|
382
|
-
return [4, __await(progress)];
|
|
383
|
-
case 3: return [4, _a.sent()];
|
|
384
|
-
case 4:
|
|
385
|
-
_a.sent();
|
|
386
|
-
_a.label = 5;
|
|
387
|
-
case 5:
|
|
407
|
+
_b.sent();
|
|
408
|
+
_b.label = 3;
|
|
409
|
+
case 3:
|
|
388
410
|
i += parallelBatchesAmount;
|
|
389
411
|
return [3, 1];
|
|
390
|
-
case
|
|
412
|
+
case 4: return [4, __await({
|
|
413
|
+
type: 'results',
|
|
414
|
+
results: results,
|
|
415
|
+
})];
|
|
416
|
+
case 5: return [4, _b.sent()];
|
|
417
|
+
case 6:
|
|
418
|
+
_b.sent();
|
|
419
|
+
return [2];
|
|
391
420
|
}
|
|
392
421
|
});
|
|
393
422
|
});
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"bulkContactsCreateApi.js","sourceRoot":"","sources":["../../../src/api/bulkContactsCreateApi.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AACA,0EAAiD;AAEjD,
|
|
1
|
+
{"version":3,"file":"bulkContactsCreateApi.js","sourceRoot":"","sources":["../../../src/api/bulkContactsCreateApi.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AACA,0EAAiD;AAEjD,0CAIsB;AACtB,kCAAkG;AAClG,6BAA0B;AAkB1B;IAAmD,yCAAc;IAC/D,+BACqB,OAAgD,EACzD,eAAgC;QAF5C,YAIE,kBAAM,OAAO,EAAE,eAAe,CAAC,SAChC;QAJoB,aAAO,GAAP,OAAO,CAAyC;QACzD,qBAAe,GAAf,eAAe,CAAiB;QAK3B,iBAAW,GAAG,aAAa,CAAC;;IAF7C,CAAC;IAmCY,kDAAkB,GAA/B,UACE,QAAgB,EAChB,IAAoB,EACpB,OAAyB;;QAAzB,wBAAA,EAAA,YAAyB;;;;;;;wBAEa,KAAA,cAAA,IAAI,CAAC,uBAAuB,CAChE,QAAQ,EACR,IAAI,EACJ,OAAO,CACR,CAAA;;;;;wBAJgB,aAAiB,EAAf,IAAI,UAAA,EAAE,OAAO,aAAA,CAAE;wBAKhC,IAAI,IAAI,KAAK,SAAS,EAAE;4BACtB,WAAO,OAA+B,EAAC;yBACxC;;;;;;;;;;;;;;;;;;;;6BAEH,WAAO;4BACL,OAAO,EAAE,EAAE;4BACX,MAAM,EAAE,EAAE;yBACX,EAAC;;;;KACH;IAEO,8CAAc,GAAtB,UACE,IAAY,EACZ,aAAqB,EACrB,YAAoB,EACpB,OAA8B;QAE9B,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,0BAA0B,EAAE,IAAI,EAAE,QAAQ,CAAC,CAAC;QAC/D,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,0BAA0B,EAAE,aAAa,CAAC,CAAC;QAC9D,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,0BAA0B,EAAE,YAAY,CAAC,CAAC;QAC7D,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,0BAA0B,EAAE,IAAI,CAAC,IAAI,CAAC,aAAa,GAAG,YAAY,CAAC,EAAE,WAAW,CAAC,CAAC;QACrG,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,0BAA0B,EAAE,OAAO,CAAC,qBAAqB,IAAI,yCAA6B,CAAC,CAAC;QAC/G,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,0BAA0B,EAAE,OAAO,CAAC,SAAS,IAAI,qCAAyB,EAAE,QAAQ,CAAC,CAAC;IAC3G,CAAC;IAEc,uDAAuB,GAAtC,UACE,QAAgB,EAChB,IAAoB,EACpB,OAAyB;;QAAzB,wBAAA,EAAA,YAAyB;;;;;;;wBAEzB,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;wBAC/B,MAAA,OAAO,CAAC,MAAM,0CAAE,GAAG,CAAC,WAAW,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;wBAC9C,SAAS,GAAG,WAAW,CAAC,GAAG,EAAE,CAAC;wBAC9B,mBAAmB,GAAG,IAAI,CAAC,6BAA6B,CAAC,IAAI,CAAC,CAAC;wBAC7D,QAAQ,GAAc,mBAAmB,SAAjC,EAAK,IAAI,UAAK,mBAAmB,EAA3C,YAAqB,CAAF,CAAyB;wBAC5C,eAAe,GAAG,OAAO,CAAC,SAAS,IAAI,qCAAyB,GAAG,IAAA,4BAAoB,eAAM,IAAI,EAAG,CAAC;wBACrG,cAAc,GAAG,IAAA,yBAAiB,EAAC,QAAQ,EAAE,eAAe,CAAC,CAAC;wBACpE,IAAI,OAAO,CAAC,MAAM,EAAE;4BAClB,IAAI,CAAC,cAAc,CACjB,IAAA,4BAAoB,EAAC,IAAI,CAAC,EAC1B,QAAQ,CAAC,MAAM,EACf,cAAc,CAAC,MAAM,EACrB,OAAgC,CACjC,CAAC;yBACH;;;;6BAMK,CAAA,cAAc,CAAC,MAAM,KAAK,CAAC,IAAI,OAAO,CAAC,gBAAgB,KAAK,CAAC,CAAA,EAA7D,cAA6D;wBAC/C,mBAAM,IAAI,CAAC,2BAA2B,CACpD,QAAQ,wBAEF,mBAA2C,KAC/C,QAAQ,EAAE,CAAC,KAEb,OAAO,CACR,GAAA;;wBAPK,OAAO,GAAG,SAOf;2CACK;gCACJ,IAAI,EAAE,SAAS;gCACf,OAAO,EAAE,IAAI,CAAC,YAAY,CAAC;oCACzB,OAAO,EAAE,EAAE;oCACX,MAAM,EAAE,EAAE;iCACX,EAAE,CAAC,OAAO,CAAC,CAAC;6BACd;4BAND,sBAMC;;wBAND,SAMC,CAAC;;;;wBAE0B,KAAA,cAAA,IAAI,CAAC,4BAA4B,CAC3D,cAAoC,EACpC,IAAI,EACJ,QAAQ,EACR,OAAO,CACR,CAAA;;;;;wBALgB,OAAO,WAAA,CAAA;wBAMtB,IAAI,OAAO,CAAC,IAAI,KAAK,SAAS,EAAE;4BACxB,OAAO,GAAG,WAAW,CAAC,GAAG,EAAE,CAAC;4BAClC,MAAA,OAAO,CAAC,MAAM,0CAAE,GAAG,CAAC,eAAQ,QAAQ,4BAAkB,IAAI,CAAC,KAAK,CAAC,CAAC,OAAO,GAAG,SAAS,CAAC,GAAG,IAAI,CAAC,aAAU,CAAC,CAAC;yBAC3G;2CACK,OAAO;4BAAb,sBAAa;;wBAAb,SAAa,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;KAOrB;IAEO,6DAA6B,GAArC,UAAsC,IAAoB;QACxD,IAAM,mBAAmB,yBACpB,IAAI,KACP,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,UAAC,CAAC,EAAE,GAAG;gBACjC,IAAI,CAAC,CAAC,UAAU,KAAK,SAAS,EAAE;oBAC9B,OAAO,CAAC,CAAC;iBACV;gBACD,6BACK,CAAC,KACJ,UAAU,EAAE,UAAG,GAAG,CAAE,IACpB;YACJ,CAAC,CAAqB,GACvB,CAAC;QAEF,IAAI,CAAC,4BAA4B,CAAC,mBAAmB,CAAC,QAAQ,CAAC,CAAC;QAEhE,OAAO,mBAAmB,CAAC;IAC7B,CAAC;IAEa,yCAAS,GAAvB;;;;;;;wBACQ,gBAAgB,GAAG,IAAA,SAAE,GAAE,CAAC;wBAC9B,IAAI,CAAC,OAAO,CAAC;4BACX,MAAM,EAAE,MAAM;4BACd,KAAK,EAAE,UAAG,IAAI,CAAC,WAAW,yBAAe,gBAAgB,CAAE;yBAC5D,CAAC,CAAC;wBACH,WAAM,IAAA,uBAAe,EAAC,cAAM,OAAA,KAAI,CAAC,OAAO,CAAC,gBAAgB,CAAC,EAA9B,CAA8B,EAAE,IAAI,CAAC,EAAA;;wBAAjE,SAAiE,CAAC;;;;;KACnE;IAEa,0CAAU,GAAxB;;;;;;;wBACQ,iBAAiB,GAAG,IAAA,SAAE,GAAE,CAAC;wBAC/B,IAAI,CAAC,OAAO,CAAC;4BACX,MAAM,EAAE,MAAM;4BACd,KAAK,EAAE,UAAG,IAAI,CAAC,WAAW,0BAAgB,iBAAiB,CAAE;yBAC9D,CAAC,CAAC;wBACH,WAAM,IAAA,uBAAe,EAAC,cAAM,OAAA,KAAI,CAAC,OAAO,CAAC,iBAAiB,CAAC,EAA/B,CAA+B,EAAE,IAAI,CAAC,EAAA;;wBAAlE,SAAkE,CAAC;;;;;KACpE;IAEO,4DAA4B,GAApC,UAAqC,QAA0B;QAC7D,QAAQ,CAAC,GAAG,CAAC,UAAC,EAAc;gBAAZ,UAAU,gBAAA;YAAO,OAAA,UAAU;QAAV,CAAU,CAAC,CAAC,OAAO,CAAC,UAAC,GAAG,EAAE,KAAK,EAAE,KAAK;YACrE,IAAI,KAAK,KAAK,KAAK,CAAC,WAAW,CAAC,GAAG,CAAC,EAAE;gBACpC,MAAM,IAAI,KAAK,CAAC,cAAO,GAAG,sDAAmD,CAAC,CAAC;aAChF;QACH,CAAC,CAAC,CAAC;IACL,CAAC;IAEa,2DAA2B,GAAzC,UACE,YAAoB,EACpB,eAA0C,EAC1C,OAAoB;;;;;;;;wBAEd,OAAO,GAAG,IAAA,SAAE,GAAE,CAAC;wBACb,QAAQ,GAAc,eAAe,SAA7B,EAAK,IAAI,UAAK,eAAe,EAAvC,YAAqB,CAAF,CAAqB;wBAE9C,IAAI,CAAC,OAAO,CAAC;4BACX,MAAM,EAAE,MAAM;4BACd,KAAK,EAAE,UAAG,IAAI,CAAC,WAAW,UAAO;4BACjC,IAAI,wBACC,IAAI,KACP,OAAO,SAAA,EACP,YAAY,cAAA,GACb;yBACF,CAAC,CAAC;wBAGW,WAAM,IAAA,uBAAe,EAAC,cAAM,OAAA,KAAI,CAAC,OAAO,CAAC,OAAO,EAAE,OAAO,CAAC,cAAc,CAAC,EAA7C,CAA6C,EAAE,IAAI,CAAC,EAAA;;wBAAxF,KAAK,GAAG,SAAgF;wBAC9F,MAAA,OAAO,CAAC,MAAM,0CAAE,GAAG,CAAC,YAAY,EAAE,QAAQ,EAAE,WAAW,CAAC,CAAC;wBACzD,WAAO,KAAK,EAAC;;;;KACd;IAEO,4CAAY,GAApB,UAAqB,OAA6B,EAAE,cAAyC;QAC3F,OAAO,cAAc,CAAC,MAAM,CAAuB,UAAC,GAAG,EAAE,KAAK;YAC5D,IAAM,eAAe,GAAG,KAAK,CAAC,OAAO,CAAC,MAAM,CAC1C,UAAC,GAAG,EAAE,MAAM,IAAK,OAAA,IAAA,oBAAY,EAAC,GAAG,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,EAArC,CAAqC,EACtD,EAAE,CACH,CAAC;YACF,IAAM,cAAc,GAAG,KAAK,CAAC,QAAQ,CAAC,MAAM,CAC1C,UAAC,GAAG,EAAE,OAAO,IAAK,OAAA,IAAA,oBAAY,EAAC,GAAG,EAAE,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,EAAtC,CAAsC,EACxD,EAAE,CACH,CAAC;YACF,OAAO;gBACL,OAAO,wBACF,GAAG,CAAC,OAAO,GACX,eAAe,CACnB;gBACD,MAAM,wBACD,GAAG,CAAC,MAAM,GACV,cAAc,CAClB;aACF,CAAC;QACJ,CAAC,EAAE,OAAO,CAAC,CAAC;IACd,CAAC;IAEe,4DAA4B,GAA5C,UACE,eAAmC,EACnC,IAAsC,EACtC,YAAoB,EACpB,OAAoB;;;;;;;;wBAEd,QAAQ,GAAG,OAAO,CAAC,6BAA6B,IAAI,wCAA4B,CAAC;wBACjF,aAAa,GAAG,eAAe;6BAClC,GAAG,CAAC,UAAC,aAAa,EAAE,QAAQ,IAAK,OAAA,cAAM,OAAA,KAAI,CAAC,2BAA2B,CACtE,YAAY,aACV,QAAQ,UAAA,EACR,QAAQ,EAAE,aAAa,IACpB,IAAI,GAET,OAAO,CACR,EAPuC,CAOvC,EAPiC,CAOjC,CAAC,CAAC;wBAEC,QAAQ,GAAiB;4BAC7B,YAAY,EAAE,eAAe,CAAC,MAAM;4BACpC,aAAa,EAAE,eAAe,CAAC,GAAG,CAAC,UAAC,CAAC,IAAK,OAAA,CAAC,CAAC,MAAM,EAAR,CAAQ,CAAC,CAAC,MAAM,CAAC,UAAC,GAAG,EAAE,IAAI,IAAK,OAAA,GAAG,GAAG,IAAI,EAAV,CAAU,EAAE,CAAC,CAAC;4BACxF,gBAAgB,EAAE,CAAC;4BACnB,iBAAiB,EAAE,CAAC;yBACrB,CAAC;wBAEE,OAAO,GAAyB;4BAClC,OAAO,EAAE,EAAE;4BACX,MAAM,EAAE,EAAE;yBACX,CAAC;wBAEI,qBAAqB,GAAG,OAAO,CAAC,qBAAqB,IAAI,yCAA6B,CAAC;4CAEpF,CAAC;;;;;wCACF,QAAQ,GAAG,CAAC,CAAC;wCACb,MAAM,GAAG,CAAC,GAAG,qBAAqB,CAAC;wCACnC,aAAa,GAAG,aAAa,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,qBAAqB,CAAC,CAAC;wCACpE,aAAa,GAA8B,EAAE,CAAC;;;;wCAEhC,mBAAM,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC,GAAG,CAAC,UAAC,CAAC,IAAK,OAAA,CAAC,EAAE,EAAH,CAAG,CAAC,CAAC,GAAA;;wCAAhE,aAAa,GAAG,SAAgD,CAAC;;;;wCAEjE,MAAA,OAAO,CAAC,MAAM,0CAAE,GAAG,CAAC,+BAClB,qBAAqB,4BACL,QAAQ,iBAAO,MAAM,2BAAwB,CAAC,CAAC;wCACjD,mBAAM,IAAA,uBAAe,EAAC,cAAM,OAAA,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC,GAAG,CAAC,UAAC,CAAC,IAAK,OAAA,CAAC,EAAE,EAAH,CAAG,CAAC,CAAC,EAA1C,CAA0C,EAAE,QAAQ,CAAC,GAAA;;wCAAjG,aAAa,GAAG,SAAiF,CAAC;;;wCAGpG,OAAO,GAAG,OAAK,YAAY,CAAC,OAAO,EAAE,aAAa,CAAC,CAAC;wCACpD,QAAQ,CAAC,gBAAgB,IAAI,aAAa,CAAC,MAAM,CAAC;wCAClD,QAAQ,CAAC,iBAAiB,IAAI,eAAe;6CAC1C,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,qBAAqB,CAAC;6CACnC,GAAG,CAAC,UAAA,KAAK,IAAI,OAAA,KAAK,CAAC,MAAM,EAAZ,CAAY,CAAC;6CAC1B,MAAM,CAAC,UAAC,GAAG,EAAE,IAAI,IAAK,OAAA,GAAG,GAAG,IAAI,EAAV,CAAU,EAAE,CAAC,CAAC,CAAC;2DAClC;gDACJ,IAAI,EAAE,UAAU;gDAChB,OAAO,EAAE,QAAQ;6CAClB;4CAHD,sBAGC;;wCAHD,SAGC,CAAC;;;;;;wBAvBK,CAAC,GAAG,CAAC;;;6BAAC,CAAA,CAAC,GAAG,CAAC,OAAO,CAAC,gBAAgB,IAAI,aAAa,CAAC,MAAM,CAAC,CAAA;2CAA5D,CAAC;;;;;wBAA4D,CAAC,IAAI,qBAAqB,CAAA;;+CAyB1F;4BACJ,IAAI,EAAE,SAAS;4BACf,OAAO,SAAA;yBACR;4BAHD,sBAGC;;wBAHD,SAGC,CAAC;;;;;KACH;IAEH,4BAAC;AAAD,CAAC,AAlSD,CAAmD,2BAAc,GAkShE"}
|
package/dist/cjs/constants.js
CHANGED
|
@@ -1,10 +1,11 @@
|
|
|
1
1
|
"use strict";
|
|
2
2
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
-
exports.
|
|
3
|
+
exports.BULK_FAILED_SERIES_REPEAT_IN = exports.BULK_POLING_PAUSE_DURATION = exports.CONTACTS_DELETE_MAX_AMOUNT = exports.FAILED_REQUEST_REPEATS = exports.MAX_PARALLEL_BATCH_PROCESSING = exports.REQUEST_PAYLOAD_MAX_BYTES = exports.CONTACTS_SERVICE_KEY = void 0;
|
|
4
4
|
exports.CONTACTS_SERVICE_KEY = 'contacts-api';
|
|
5
5
|
exports.REQUEST_PAYLOAD_MAX_BYTES = 4000000;
|
|
6
6
|
exports.MAX_PARALLEL_BATCH_PROCESSING = 4;
|
|
7
7
|
exports.FAILED_REQUEST_REPEATS = 3;
|
|
8
8
|
exports.CONTACTS_DELETE_MAX_AMOUNT = 1000;
|
|
9
|
-
exports.
|
|
9
|
+
exports.BULK_POLING_PAUSE_DURATION = 1500;
|
|
10
|
+
exports.BULK_FAILED_SERIES_REPEAT_IN = 30000;
|
|
10
11
|
//# sourceMappingURL=constants.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"constants.js","sourceRoot":"","sources":["../../src/constants.ts"],"names":[],"mappings":";;;AAAa,QAAA,oBAAoB,GAAG,cAAc,CAAC;AAEtC,QAAA,yBAAyB,GAAG,OAAO,CAAC;AAEpC,QAAA,6BAA6B,GAAG,CAAC,CAAC;AAElC,QAAA,sBAAsB,GAAG,CAAC,CAAC;AAE3B,QAAA,0BAA0B,GAAG,IAAI,CAAC;AAElC,QAAA,
|
|
1
|
+
{"version":3,"file":"constants.js","sourceRoot":"","sources":["../../src/constants.ts"],"names":[],"mappings":";;;AAAa,QAAA,oBAAoB,GAAG,cAAc,CAAC;AAEtC,QAAA,yBAAyB,GAAG,OAAO,CAAC;AAEpC,QAAA,6BAA6B,GAAG,CAAC,CAAC;AAElC,QAAA,sBAAsB,GAAG,CAAC,CAAC;AAE3B,QAAA,0BAA0B,GAAG,IAAI,CAAC;AAElC,QAAA,0BAA0B,GAAG,IAAI,CAAC;AAElC,QAAA,4BAA4B,GAAG,KAAK,CAAC"}
|
package/dist/cjs/utils.js
CHANGED
|
@@ -14,7 +14,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
|
14
14
|
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
15
15
|
};
|
|
16
16
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
17
|
-
exports.buildHttpAgents = exports.debouncePromise = exports.chunkArrByMaxSize = exports.getObjectSizeInBytes = exports.adaptListParams = void 0;
|
|
17
|
+
exports.mergeObjects = exports.buildHttpAgents = exports.debouncePromise = exports.chunkArrByMaxSize = exports.getObjectSizeInBytes = exports.adaptListParams = void 0;
|
|
18
18
|
var agentkeepalive_1 = __importDefault(require("agentkeepalive"));
|
|
19
19
|
var propertyToEscape = ['from', 'size', 'orderProperty', 'orderDirection'];
|
|
20
20
|
var adaptListParams = function (params) {
|
|
@@ -63,7 +63,7 @@ var chunkArrByMaxSize = function (arr, maxSize) {
|
|
|
63
63
|
if (chunksLength < arr.length) {
|
|
64
64
|
chunks[chunkIdx + 1] = arr.slice(chunksLength);
|
|
65
65
|
}
|
|
66
|
-
return chunks;
|
|
66
|
+
return chunks.filter(function (c) { return !!c; });
|
|
67
67
|
};
|
|
68
68
|
exports.chunkArrByMaxSize = chunkArrByMaxSize;
|
|
69
69
|
function debouncePromise(caller, delay) {
|
|
@@ -91,4 +91,12 @@ function buildHttpAgents() {
|
|
|
91
91
|
};
|
|
92
92
|
}
|
|
93
93
|
exports.buildHttpAgents = buildHttpAgents;
|
|
94
|
+
var mergeObjects = function (first, second) {
|
|
95
|
+
for (var _i = 0, _a = Object.entries(second); _i < _a.length; _i++) {
|
|
96
|
+
var _b = _a[_i], key = _b[0], value = _b[1];
|
|
97
|
+
first[key] = value;
|
|
98
|
+
}
|
|
99
|
+
return first;
|
|
100
|
+
};
|
|
101
|
+
exports.mergeObjects = mergeObjects;
|
|
94
102
|
//# sourceMappingURL=utils.js.map
|
package/dist/cjs/utils.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"utils.js","sourceRoot":"","sources":["../../src/utils.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;AAAA,kEAAmC;AAInC,IAAM,gBAAgB,GAAG,CAAC,MAAM,EAAE,MAAM,EAAE,eAAe,EAAE,gBAAgB,CAAU,CAAC;AAI/E,IAAM,eAAe,GAAG,UAC7B,MAAS;IAET,IAAM,SAAS,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,MAAM,CAAa,UAAC,GAAG,EAAE,CAAC;QAC9D,IAAI,gBAAgB,CAAC,OAAO,CAAC,CAAoC,CAAC,KAAK,CAAC,CAAC,EAAE;YAExE,GAAW,CAAC,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;SAC7B;QACD,OAAO,GAAG,CAAC;IACb,CAAC,EAAE,EAAgB,CAAC,CAAC;IAErB,+CACK,SAAS,GACT,CACD,CAAC,MAAM,CAAC,aAAa,IAAI,MAAM,CAAC,cAAc,CAAC;QAC/C,EAAE,KAAK,EAAE,UAAG,MAAM,CAAC,aAAa,cAAI,MAAM,CAAC,cAAc,CAAE,EAAE,CAC9D,GACE,CAAC,MAAM,CAAC,IAAI,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,GACtC,CAAC,MAAM,CAAC,IAAI,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,EACzC;AACJ,CAAC,CAAC;AApBW,QAAA,eAAe,mBAoB1B;AAEK,IAAM,oBAAoB,GAAG,UAAI,GAAM,IAAa,OAAA,MAAM;KAC9D,IAAI,CAAC,kBAAkB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC;KAC7C,MAAM,EAFkD,CAElD,CAAC;AAFG,QAAA,oBAAoB,wBAEvB;AAEH,IAAM,iBAAiB,GAAG,UAAI,GAAQ,EAAE,OAAe;IAC5D,IAAM,IAAI,GAAG,IAAA,4BAAoB,EAAC,GAAG,CAAC,CAAC;IAEvC,IAAI,IAAI,GAAG,OAAO,EAAE;QAClB,OAAO,CAAC,GAAG,CAAC,CAAC;KACd;SAAM,IAAI,GAAG,CAAC,MAAM,KAAK,CAAC,EAAE;QAC3B,MAAM,IAAI,KAAK,CAAC,qDAAqD,CAAC,CAAC;KACxE;IAED,IAAM,MAAM,GAAG,EAAE,CAAC;IAClB,IAAI,QAAQ,GAAG,CAAC,CAAC;IACjB,IAAI,SAAS,GAAG,CAAC,CAAC;IAClB,IAAI,aAAa,GAAG,CAAC,CAAC;IAEtB,KAAK,IAAI,GAAG,GAAG,CAAC,EAAC,GAAG,GAAG,GAAG,CAAC,MAAM,EAAC,GAAG,EAAG,EAAE;QACxC,IAAM,QAAQ,GAAG,IAAA,4BAAoB,EAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC;QAChD,IAAI,QAAQ,GAAG,OAAO,EAAE;YACtB,MAAM,IAAI,KAAK,CAAC,iEACd,GAAG,yBACU,QAAQ,oCAA0B,OAAO,CAAE,CAAC,CAAC;SAC7D;QAED,IAAI,SAAS,GAAG,QAAQ,GAAG,OAAO,EAAE;YAClC,SAAS,IAAI,QAAQ,CAAC;SACvB;aAAM;YACL,MAAM,CAAC,QAAQ,CAAC,GAAG,GAAG,CAAC,KAAK,CAAC,aAAa,EAAE,GAAG,CAAC,CAAC;YACjD,QAAQ,IAAI,CAAC,CAAC;YACd,SAAS,GAAG,QAAQ,CAAC;YACrB,aAAa,GAAG,GAAG,CAAC;SACrB;KACF;IAED,IAAM,YAAY,GAAG,MAAM,CAAC,MAAM,CAAC,UAAC,GAAG,EAAE,KAAK,IAAK,OAAA,GAAG,GAAG,KAAK,CAAC,MAAM,EAAlB,CAAkB,EAAE,CAAC,CAAC,CAAC;IAC1E,IAAI,YAAY,GAAG,GAAG,CAAC,MAAM,EAAE;QAC7B,MAAM,CAAC,QAAQ,GAAG,CAAC,CAAC,GAAG,GAAG,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;KAChD;IAED,OAAO,MAAM,CAAC;
|
|
1
|
+
{"version":3,"file":"utils.js","sourceRoot":"","sources":["../../src/utils.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;AAAA,kEAAmC;AAInC,IAAM,gBAAgB,GAAG,CAAC,MAAM,EAAE,MAAM,EAAE,eAAe,EAAE,gBAAgB,CAAU,CAAC;AAI/E,IAAM,eAAe,GAAG,UAC7B,MAAS;IAET,IAAM,SAAS,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,MAAM,CAAa,UAAC,GAAG,EAAE,CAAC;QAC9D,IAAI,gBAAgB,CAAC,OAAO,CAAC,CAAoC,CAAC,KAAK,CAAC,CAAC,EAAE;YAExE,GAAW,CAAC,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;SAC7B;QACD,OAAO,GAAG,CAAC;IACb,CAAC,EAAE,EAAgB,CAAC,CAAC;IAErB,+CACK,SAAS,GACT,CACD,CAAC,MAAM,CAAC,aAAa,IAAI,MAAM,CAAC,cAAc,CAAC;QAC/C,EAAE,KAAK,EAAE,UAAG,MAAM,CAAC,aAAa,cAAI,MAAM,CAAC,cAAc,CAAE,EAAE,CAC9D,GACE,CAAC,MAAM,CAAC,IAAI,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,GACtC,CAAC,MAAM,CAAC,IAAI,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,EACzC;AACJ,CAAC,CAAC;AApBW,QAAA,eAAe,mBAoB1B;AAEK,IAAM,oBAAoB,GAAG,UAAI,GAAM,IAAa,OAAA,MAAM;KAC9D,IAAI,CAAC,kBAAkB,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC;KAC7C,MAAM,EAFkD,CAElD,CAAC;AAFG,QAAA,oBAAoB,wBAEvB;AAEH,IAAM,iBAAiB,GAAG,UAAI,GAAQ,EAAE,OAAe;IAC5D,IAAM,IAAI,GAAG,IAAA,4BAAoB,EAAC,GAAG,CAAC,CAAC;IAEvC,IAAI,IAAI,GAAG,OAAO,EAAE;QAClB,OAAO,CAAC,GAAG,CAAC,CAAC;KACd;SAAM,IAAI,GAAG,CAAC,MAAM,KAAK,CAAC,EAAE;QAC3B,MAAM,IAAI,KAAK,CAAC,qDAAqD,CAAC,CAAC;KACxE;IAED,IAAM,MAAM,GAAG,EAAE,CAAC;IAClB,IAAI,QAAQ,GAAG,CAAC,CAAC;IACjB,IAAI,SAAS,GAAG,CAAC,CAAC;IAClB,IAAI,aAAa,GAAG,CAAC,CAAC;IAEtB,KAAK,IAAI,GAAG,GAAG,CAAC,EAAC,GAAG,GAAG,GAAG,CAAC,MAAM,EAAC,GAAG,EAAG,EAAE;QACxC,IAAM,QAAQ,GAAG,IAAA,4BAAoB,EAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC;QAChD,IAAI,QAAQ,GAAG,OAAO,EAAE;YACtB,MAAM,IAAI,KAAK,CAAC,iEACd,GAAG,yBACU,QAAQ,oCAA0B,OAAO,CAAE,CAAC,CAAC;SAC7D;QAED,IAAI,SAAS,GAAG,QAAQ,GAAG,OAAO,EAAE;YAClC,SAAS,IAAI,QAAQ,CAAC;SACvB;aAAM;YACL,MAAM,CAAC,QAAQ,CAAC,GAAG,GAAG,CAAC,KAAK,CAAC,aAAa,EAAE,GAAG,CAAC,CAAC;YACjD,QAAQ,IAAI,CAAC,CAAC;YACd,SAAS,GAAG,QAAQ,CAAC;YACrB,aAAa,GAAG,GAAG,CAAC;SACrB;KACF;IAED,IAAM,YAAY,GAAG,MAAM,CAAC,MAAM,CAAC,UAAC,GAAG,EAAE,KAAK,IAAK,OAAA,GAAG,GAAG,KAAK,CAAC,MAAM,EAAlB,CAAkB,EAAE,CAAC,CAAC,CAAC;IAC1E,IAAI,YAAY,GAAG,GAAG,CAAC,MAAM,EAAE;QAC7B,MAAM,CAAC,QAAQ,GAAG,CAAC,CAAC,GAAG,GAAG,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;KAChD;IAED,OAAO,MAAM,CAAC,MAAM,CAAC,UAAC,CAAC,IAAK,OAAA,CAAC,CAAC,CAAC,EAAH,CAAG,CAAC,CAAC;AACnC,CAAC,CAAC;AAtCW,QAAA,iBAAiB,qBAsC5B;AAEF,SAAgB,eAAe,CAC7B,MAAwB,EACxB,KAAa;IAEb,IAAI,OAAO,GAA0B,IAAI,CAAC;IAE1C,OAAO,IAAI,OAAO,CAAI,UAAC,GAAG,EAAE,GAAG;QAC7B,IAAI,OAAO,EAAE;YACX,YAAY,CAAC,OAAO,CAAC,CAAC;SACvB;QACD,OAAO,GAAG,UAAU,CAClB,cAAM,OAAA,MAAM,EAAE,CAAC,IAAI,CAAC,UAAC,CAAC,IAAK,OAAA,GAAG,CAAC,CAAC,CAAC,EAAN,CAAM,CAAC,CAAC,KAAK,CAAC,UAAC,GAAG,IAAK,OAAA,GAAG,CAAC,GAAG,CAAC,EAAR,CAAQ,CAAC,EAArD,CAAqD,EAC3D,KAAK,CACN,CAAC;IACJ,CAAC,CAAC,CAAC;AACL,CAAC;AAfD,0CAeC;AAED,SAAgB,eAAe;IAC7B,IAAM,kBAAkB,GAAG;QACzB,UAAU,EAAE,GAAG;QACf,cAAc,EAAE,EAAE;QAElB,OAAO,EAAE,KAAK;QAEd,iBAAiB,EAAE,KAAK;KACzB,CAAC;IAEF,IAAM,SAAS,GAAG,IAAI,wBAAK,CAAC,kBAAkB,CAAC,CAAC;IAChD,IAAM,UAAU,GAAG,IAAI,wBAAK,CAAC,UAAU,CAAC,kBAAkB,CAAC,CAAC;IAC5D,OAAO;QACL,SAAS,WAAA;QACT,UAAU,YAAA;KACX,CAAC;AACJ,CAAC;AAhBD,0CAgBC;AAEM,IAAM,YAAY,GAAG,UAAC,KAA6B,EAAE,MAA8B;IACxF,KAA2B,UAAsB,EAAtB,KAAA,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,EAAtB,cAAsB,EAAtB,IAAsB,EAAE;QAAxC,IAAA,WAAY,EAAX,GAAG,QAAA,EAAE,KAAK,QAAA;QACpB,KAAK,CAAC,GAAG,CAAC,GAAG,KAAK,CAAC;KACpB;IACD,OAAO,KAAK,CAAC;AACf,CAAC,CAAC;AALW,QAAA,YAAY,gBAKvB"}
|