@bymax-one/nest-storage 1.1.2 → 1.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -1
- package/README.md +4 -1
- package/dist/server/index.cjs +99 -10
- package/dist/server/index.d.cts +7 -0
- package/dist/server/index.d.ts +7 -0
- package/dist/server/index.mjs +100 -11
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -13,6 +13,31 @@ note on each entry is the compatibility contract — read it, not the number.
|
|
|
13
13
|
|
|
14
14
|
## [Unreleased]
|
|
15
15
|
|
|
16
|
+
## [1.1.3] - 2026-11-01
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
|
|
20
|
+
- **A stream that stops delivering no longer holds a multipart upload open forever.** A multipart upload
|
|
21
|
+
waiting for the next bytes of a stream has no request in flight, so `requestTimeoutMs` never fired:
|
|
22
|
+
a client that sent a few MiB and went silent left `upload()` pending and the upload (and its parts)
|
|
23
|
+
open on the provider until a lifecycle rule reaped it. A new `uploadIdleTimeoutMs` bounds the silence.
|
|
24
|
+
Past it the library sends `AbortMultipartUpload`, destroys your stream and rejects `upload()` with
|
|
25
|
+
`STORAGE_TIMEOUT` and `details.phase: 'source-idle'`. The clock restarts on every chunk and runs only
|
|
26
|
+
while the library is waiting for data.
|
|
27
|
+
|
|
28
|
+
**Apply to a derived backend:** a behaviour change, on by default. A stream source that delivers no
|
|
29
|
+
bytes for `uploadIdleTimeoutMs` now aborts the multipart upload and rejects `STORAGE_TIMEOUT`; the
|
|
30
|
+
limit defaults to `requestTimeoutMs` (30 s). Set `uploadIdleTimeoutMs: 0` to restore the old
|
|
31
|
+
behaviour, or raise it for a source that legitimately pauses longer. It applies only to a stream that
|
|
32
|
+
takes the multipart path (no `size`, or at or above `multipart.thresholdBytes`); single-shot uploads
|
|
33
|
+
and `Buffer` bodies are unchanged. An invalid value (negative, fractional, not a number) is rejected
|
|
34
|
+
at startup as `STORAGE_INVALID_CONFIG`.
|
|
35
|
+
- **Reporting a failure can no longer replace it.** `deleteMany()`, the post-upload cleanup, `exists()` and
|
|
36
|
+
the multipart error mapping built a message with `String(err)` or `err.message`; for a rejection value
|
|
37
|
+
with no usable `toString` (an object with no prototype, one whose `toString` throws) that raised from
|
|
38
|
+
inside the catch and the new error escaped in place of the one being reported. A value is now described
|
|
39
|
+
by its string `message`, else its string form, else `non-printable rejection value`, and never throws.
|
|
40
|
+
|
|
16
41
|
## [1.1.2] - 2026-10-31
|
|
17
42
|
|
|
18
43
|
### Changed
|
|
@@ -455,4 +480,5 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
455
480
|
[1.1.0]: https://github.com/bymaxone/nest-storage/compare/v1.0.6...v1.1.0
|
|
456
481
|
[1.1.1]: https://github.com/bymaxone/nest-storage/compare/v1.1.0...v1.1.1
|
|
457
482
|
[1.1.2]: https://github.com/bymaxone/nest-storage/compare/v1.1.1...v1.1.2
|
|
458
|
-
[
|
|
483
|
+
[1.1.3]: https://github.com/bymaxone/nest-storage/compare/v1.1.2...v1.1.3
|
|
484
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.3...HEAD
|
package/README.md
CHANGED
|
@@ -267,6 +267,7 @@ The full configuration reference lives in `docs/technical_specification.md` §4.
|
|
|
267
267
|
| `defaultCacheControl` | `string` | `public, max-age=31536000, immutable` | Default `Cache-Control` on uploads |
|
|
268
268
|
| `defaultContentDisposition` | `'inline' \| 'attachment'` | `inline` | Default `Content-Disposition` on uploads |
|
|
269
269
|
| `requestTimeoutMs` | `number` | `30000` | Per attempt, until the response headers arrive; see below the table |
|
|
270
|
+
| `uploadIdleTimeoutMs` | `number` | `30000` | Longest silence from a stream body during a multipart upload; follows `requestTimeoutMs` unless set, `0` disables; see below the table |
|
|
270
271
|
| `signedUrls.defaultGetTtlSeconds` | `number` | `300` | Default TTL for `getDownloadUrl` |
|
|
271
272
|
| `signedUrls.defaultPutTtlSeconds` | `number` | `300` | Default TTL for `getUploadUrl` |
|
|
272
273
|
| `signedUrls.maxTtlSeconds` | `number` | `604800` | Per-request TTLs above this are clamped; a value above 604800 (the 7-day SigV4 limit) is reduced to it at init |
|
|
@@ -280,6 +281,8 @@ The full configuration reference lives in `docs/technical_specification.md` §4.
|
|
|
280
281
|
|
|
281
282
|
**`requestTimeoutMs`** is passed to the S3 client's request handler as `requestTimeout` with `throwOnRequestTimeout` ([AWS SDK for JavaScript v3: `requestHandler: { requestTimeout: 3_000, ... }`][sdk-request-timeout]). It bounds one attempt, from sending the request until the response headers arrive, and an upload body counts toward it, so a large single-shot upload on a slow link needs a larger value. It does not limit reading a download stream. With `maxAttempts` above 1 a stalled request is retried, so the total wait is the timeout times the attempts.
|
|
282
283
|
|
|
284
|
+
**`uploadIdleTimeoutMs`** covers the gap `requestTimeoutMs` cannot: a multipart upload that is waiting for the next bytes of a stream has no request in flight, so a source that stops delivering (a client that stalls, a pipe that never closes) would hold the upload open on the provider indefinitely. When the source delivers nothing for this long, the upload is aborted (the provider is sent `AbortMultipartUpload`), your stream is destroyed and `upload()` rejects with `STORAGE_TIMEOUT` and `details.phase` set to `'source-idle'`. The clock restarts on every chunk and runs only while the library is waiting for data, so a slow but live source succeeds and a slow provider does not trip it. It defaults to the value of `requestTimeoutMs`; `0` turns it off. It applies only to a stream that takes the multipart path; a single-shot upload is bounded by `requestTimeoutMs`, and a `Buffer` cannot stall.
|
|
285
|
+
|
|
283
286
|
[aws-presigned-expiry]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
|
|
284
287
|
[aws-get-object]: https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
|
|
285
288
|
[aws-mpu-limits]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/qfacts.html
|
|
@@ -742,7 +745,7 @@ All errors are thrown as `StorageException extends HttpException`. The response
|
|
|
742
745
|
| `STORAGE_BUCKET_UNDEFINED` | 400 | Operation without a bucket and no default in config |
|
|
743
746
|
| `STORAGE_MULTIPART_ABORTED` | 500 | Multipart upload failed and was aborted |
|
|
744
747
|
| `STORAGE_INVALID_CONFIG` | 500 | `BymaxStorageModuleOptions` validation failed at initialization |
|
|
745
|
-
| `STORAGE_TIMEOUT` | 504 | An attempt got no response headers within `requestTimeoutMs` (the SDK's `TimeoutError`); a failure inside a multipart upload surfaces as `STORAGE_MULTIPART_ABORTED` |
|
|
748
|
+
| `STORAGE_TIMEOUT` | 504 | An attempt got no response headers within `requestTimeoutMs` (the SDK's `TimeoutError`); a multipart source that stays silent for `uploadIdleTimeoutMs` also surfaces as `STORAGE_TIMEOUT` (`details.phase: 'source-idle'`); any other failure inside a multipart upload surfaces as `STORAGE_MULTIPART_ABORTED` |
|
|
746
749
|
| `STORAGE_LIFECYCLE_RULE_INVALID` | 400 | `BucketLifecycleService` input rejected: empty or over-long `id`, non-positive or non-integer `expirationDays`, bad `prefix`, or an id naming a rule outside `keyPrefix` (`details.reason`) |
|
|
747
750
|
| `STORAGE_LIFECYCLE_UNSUPPORTED` | 501 | The provider does not implement the lifecycle API |
|
|
748
751
|
| `STORAGE_LIFECYCLE_REJECTED` | 422 | The provider refused the lifecycle configuration |
|
package/dist/server/index.cjs
CHANGED
|
@@ -160,6 +160,10 @@ function validateOptions(options) {
|
|
|
160
160
|
if (isObject(multipart) && typeof multipart.partSizeBytes === "number" && multipart.partSizeBytes < MIN_PART_SIZE_BYTES) {
|
|
161
161
|
throw invalid("multipart.partSizeBytes must be >= 5 MB (S3 hard limit)");
|
|
162
162
|
}
|
|
163
|
+
const idle = options.uploadIdleTimeoutMs;
|
|
164
|
+
if (idle !== void 0 && (typeof idle !== "number" || !Number.isInteger(idle) || idle < 0)) {
|
|
165
|
+
throw invalid("uploadIdleTimeoutMs must be an integer >= 0 (0 disables the guard)");
|
|
166
|
+
}
|
|
163
167
|
if (options.serverSideEncryption === "aws:kms" && !options.kmsKeyId) {
|
|
164
168
|
throw invalid('kmsKeyId is required when serverSideEncryption === "aws:kms"');
|
|
165
169
|
}
|
|
@@ -213,6 +217,7 @@ function applyDefaults(options) {
|
|
|
213
217
|
const hasCredentials = Boolean(options.credentials.accessKeyId) && Boolean(options.credentials.secretAccessKey);
|
|
214
218
|
const publicBaseUrl = options.publicBaseUrl ?? `${trimTrailingSlashes(options.endpoint)}/${options.bucket}`;
|
|
215
219
|
const credentials = { ...options.credentials };
|
|
220
|
+
const requestTimeoutMs = options.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS;
|
|
216
221
|
const resolved = {
|
|
217
222
|
endpoint: options.endpoint,
|
|
218
223
|
region: options.region,
|
|
@@ -233,7 +238,8 @@ function applyDefaults(options) {
|
|
|
233
238
|
requestChecksumCalculation: options.requestChecksumCalculation ?? DEFAULT_CHECKSUM_CALCULATION,
|
|
234
239
|
responseChecksumValidation: options.responseChecksumValidation ?? DEFAULT_CHECKSUM_VALIDATION,
|
|
235
240
|
maxAttempts: options.maxAttempts ?? DEFAULT_MAX_ATTEMPTS,
|
|
236
|
-
requestTimeoutMs
|
|
241
|
+
requestTimeoutMs,
|
|
242
|
+
uploadIdleTimeoutMs: options.uploadIdleTimeoutMs ?? requestTimeoutMs,
|
|
237
243
|
hasCredentials
|
|
238
244
|
};
|
|
239
245
|
Object.defineProperty(resolved, "credentials", {
|
|
@@ -996,6 +1002,93 @@ FileScannerService = __decorateClass([
|
|
|
996
1002
|
__decorateParam(1, common.Inject(BYMAX_STORAGE_FILE_SCANNER))
|
|
997
1003
|
], FileScannerService);
|
|
998
1004
|
|
|
1005
|
+
// src/server/utils/describe-error.ts
|
|
1006
|
+
var NON_PRINTABLE_REJECTION = "non-printable rejection value";
|
|
1007
|
+
function describeError(value) {
|
|
1008
|
+
try {
|
|
1009
|
+
if (typeof value === "object" && value !== null) {
|
|
1010
|
+
const message = value.message;
|
|
1011
|
+
if (typeof message === "string") {
|
|
1012
|
+
return message;
|
|
1013
|
+
}
|
|
1014
|
+
}
|
|
1015
|
+
return String(value);
|
|
1016
|
+
} catch {
|
|
1017
|
+
return NON_PRINTABLE_REJECTION;
|
|
1018
|
+
}
|
|
1019
|
+
}
|
|
1020
|
+
var SourceIdleError = class extends Error {
|
|
1021
|
+
constructor(idleMs) {
|
|
1022
|
+
super(`The upload source delivered no bytes for ${String(idleMs)} ms`);
|
|
1023
|
+
this.idleMs = idleMs;
|
|
1024
|
+
this.name = "SourceIdleError";
|
|
1025
|
+
}
|
|
1026
|
+
idleMs;
|
|
1027
|
+
};
|
|
1028
|
+
function withinIdleTime(next, idleMs) {
|
|
1029
|
+
return new Promise((resolve, reject) => {
|
|
1030
|
+
const timer = setTimeout(() => {
|
|
1031
|
+
reject(new SourceIdleError(idleMs));
|
|
1032
|
+
}, idleMs);
|
|
1033
|
+
next.then(
|
|
1034
|
+
(value) => {
|
|
1035
|
+
clearTimeout(timer);
|
|
1036
|
+
resolve(value);
|
|
1037
|
+
},
|
|
1038
|
+
(error) => {
|
|
1039
|
+
clearTimeout(timer);
|
|
1040
|
+
reject(error instanceof Error ? error : new Error(describeError(error)));
|
|
1041
|
+
}
|
|
1042
|
+
);
|
|
1043
|
+
});
|
|
1044
|
+
}
|
|
1045
|
+
function withIdleTimeout(source, idleMs) {
|
|
1046
|
+
const iterator = source[Symbol.asyncIterator]();
|
|
1047
|
+
const wrapper = new stream.Readable({
|
|
1048
|
+
objectMode: false,
|
|
1049
|
+
// No read-ahead: with a buffer to fill, Node calls read() while the consumer is paused, and the
|
|
1050
|
+
// clock would run for bytes nobody has asked for. At 0, read() runs only on demand.
|
|
1051
|
+
highWaterMark: 0,
|
|
1052
|
+
read() {
|
|
1053
|
+
withinIdleTime(iterator.next(), idleMs).then(
|
|
1054
|
+
(step) => {
|
|
1055
|
+
wrapper.push(step.done === true ? null : step.value);
|
|
1056
|
+
},
|
|
1057
|
+
(error) => {
|
|
1058
|
+
wrapper.destroy(error);
|
|
1059
|
+
}
|
|
1060
|
+
);
|
|
1061
|
+
},
|
|
1062
|
+
destroy(error, callback) {
|
|
1063
|
+
destroyStream(source);
|
|
1064
|
+
callback(error);
|
|
1065
|
+
}
|
|
1066
|
+
});
|
|
1067
|
+
wrapper.once("error", () => void 0);
|
|
1068
|
+
return wrapper;
|
|
1069
|
+
}
|
|
1070
|
+
function guardIdleBody(body, idleMs) {
|
|
1071
|
+
if (isBufferLike(body)) {
|
|
1072
|
+
return body;
|
|
1073
|
+
}
|
|
1074
|
+
return idleMs === 0 ? body : withIdleTimeout(body, idleMs);
|
|
1075
|
+
}
|
|
1076
|
+
|
|
1077
|
+
// src/server/errors/multipart-error-mapper.ts
|
|
1078
|
+
function mapMultipartError(err, context) {
|
|
1079
|
+
if (err instanceof SourceIdleError) {
|
|
1080
|
+
return new StorageException(STORAGE_ERROR_CODES.STORAGE_TIMEOUT, common.HttpStatus.GATEWAY_TIMEOUT, {
|
|
1081
|
+
...context,
|
|
1082
|
+
phase: "source-idle",
|
|
1083
|
+
idleMs: err.idleMs
|
|
1084
|
+
});
|
|
1085
|
+
}
|
|
1086
|
+
return new StorageException(STORAGE_ERROR_CODES.STORAGE_MULTIPART_ABORTED, void 0, {
|
|
1087
|
+
...context,
|
|
1088
|
+
awsMessage: describeError(err)
|
|
1089
|
+
});
|
|
1090
|
+
}
|
|
1091
|
+
|
|
999
1092
|
// src/server/utils/header-utils.ts
|
|
1000
1093
|
function buildContentDisposition(perCall, defaultValue) {
|
|
1001
1094
|
return perCall ?? defaultValue;
|
|
@@ -1191,7 +1284,7 @@ exports.StorageService = class StorageService {
|
|
|
1191
1284
|
await this.delete(finalKey, { bucket }).catch((deleteErr) => {
|
|
1192
1285
|
this.logger.error(
|
|
1193
1286
|
`Post-upload cleanup failed for infected/inconclusive object: ${finalKey}`,
|
|
1194
|
-
deleteErr
|
|
1287
|
+
describeError(deleteErr)
|
|
1195
1288
|
);
|
|
1196
1289
|
});
|
|
1197
1290
|
throw scanErr;
|
|
@@ -1232,7 +1325,7 @@ exports.StorageService = class StorageService {
|
|
|
1232
1325
|
if (err instanceof StorageException && err.code === STORAGE_ERROR_CODES.STORAGE_OBJECT_NOT_FOUND) {
|
|
1233
1326
|
return false;
|
|
1234
1327
|
}
|
|
1235
|
-
this.logger.warn(`exists() \u2014 treating a non-404 error as "false": ${err
|
|
1328
|
+
this.logger.warn(`exists() \u2014 treating a non-404 error as "false": ${describeError(err)}`);
|
|
1236
1329
|
return false;
|
|
1237
1330
|
}
|
|
1238
1331
|
}
|
|
@@ -1416,7 +1509,7 @@ exports.StorageService = class StorageService {
|
|
|
1416
1509
|
const params = {
|
|
1417
1510
|
Bucket: bucket,
|
|
1418
1511
|
Key: finalKey,
|
|
1419
|
-
Body: options.body,
|
|
1512
|
+
Body: guardIdleBody(options.body, this.options.uploadIdleTimeoutMs),
|
|
1420
1513
|
ContentType: options.contentType,
|
|
1421
1514
|
CacheControl: buildCacheControl(options.cacheControl, this.options.defaultCacheControl),
|
|
1422
1515
|
ContentDisposition: buildContentDisposition(
|
|
@@ -1452,11 +1545,7 @@ exports.StorageService = class StorageService {
|
|
|
1452
1545
|
multipart: true
|
|
1453
1546
|
});
|
|
1454
1547
|
} catch (err) {
|
|
1455
|
-
throw
|
|
1456
|
-
key: finalKey,
|
|
1457
|
-
bucket,
|
|
1458
|
-
awsMessage: err.message
|
|
1459
|
-
});
|
|
1548
|
+
throw mapMultipartError(err, { key: finalKey, bucket });
|
|
1460
1549
|
}
|
|
1461
1550
|
}
|
|
1462
1551
|
/** Converts a `Uint8Array` body to a `Buffer` for SDK safety; passes others through. */
|
|
@@ -1624,7 +1713,7 @@ exports.StorageService = class StorageService {
|
|
|
1624
1713
|
}
|
|
1625
1714
|
}
|
|
1626
1715
|
} catch (err) {
|
|
1627
|
-
const message =
|
|
1716
|
+
const message = describeError(err);
|
|
1628
1717
|
for (const key of chunk) {
|
|
1629
1718
|
failed.push({ key: this.keyResolver.stripPrefix(key), error: message });
|
|
1630
1719
|
}
|
package/dist/server/index.d.cts
CHANGED
|
@@ -168,6 +168,12 @@ interface BymaxStorageModuleOptions {
|
|
|
168
168
|
maxAttempts?: number;
|
|
169
169
|
/** Per-request timeout. Default: `30000` ms. */
|
|
170
170
|
requestTimeoutMs?: number;
|
|
171
|
+
/**
|
|
172
|
+
* Longest silence, in milliseconds, tolerated from a stream body while a multipart upload
|
|
173
|
+
* waits for its next bytes; past it the upload is aborted and rejects `STORAGE_TIMEOUT`.
|
|
174
|
+
* An integer of at least `0`; `0` turns the guard off. Default: `requestTimeoutMs`.
|
|
175
|
+
*/
|
|
176
|
+
uploadIdleTimeoutMs?: number;
|
|
171
177
|
}
|
|
172
178
|
/**
|
|
173
179
|
* Asynchronous configuration for `BymaxStorageModule.forRootAsync()` — the
|
|
@@ -296,6 +302,7 @@ interface ResolvedBymaxStorageOptions {
|
|
|
296
302
|
responseChecksumValidation: 'WHEN_SUPPORTED' | 'WHEN_REQUIRED';
|
|
297
303
|
maxAttempts: number;
|
|
298
304
|
requestTimeoutMs: number;
|
|
305
|
+
uploadIdleTimeoutMs: number;
|
|
299
306
|
/** True when both `accessKeyId` and `secretAccessKey` are non-empty. */
|
|
300
307
|
hasCredentials: boolean;
|
|
301
308
|
}
|
package/dist/server/index.d.ts
CHANGED
|
@@ -168,6 +168,12 @@ interface BymaxStorageModuleOptions {
|
|
|
168
168
|
maxAttempts?: number;
|
|
169
169
|
/** Per-request timeout. Default: `30000` ms. */
|
|
170
170
|
requestTimeoutMs?: number;
|
|
171
|
+
/**
|
|
172
|
+
* Longest silence, in milliseconds, tolerated from a stream body while a multipart upload
|
|
173
|
+
* waits for its next bytes; past it the upload is aborted and rejects `STORAGE_TIMEOUT`.
|
|
174
|
+
* An integer of at least `0`; `0` turns the guard off. Default: `requestTimeoutMs`.
|
|
175
|
+
*/
|
|
176
|
+
uploadIdleTimeoutMs?: number;
|
|
171
177
|
}
|
|
172
178
|
/**
|
|
173
179
|
* Asynchronous configuration for `BymaxStorageModule.forRootAsync()` — the
|
|
@@ -296,6 +302,7 @@ interface ResolvedBymaxStorageOptions {
|
|
|
296
302
|
responseChecksumValidation: 'WHEN_SUPPORTED' | 'WHEN_REQUIRED';
|
|
297
303
|
maxAttempts: number;
|
|
298
304
|
requestTimeoutMs: number;
|
|
305
|
+
uploadIdleTimeoutMs: number;
|
|
299
306
|
/** True when both `accessKeyId` and `secretAccessKey` are non-empty. */
|
|
300
307
|
hasCredentials: boolean;
|
|
301
308
|
}
|
package/dist/server/index.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { HttpStatus, Injectable, Inject, Global, Module, Logger, HttpException } from '@nestjs/common';
|
|
2
2
|
import { S3Client, GetBucketLifecycleConfigurationCommand, DeleteBucketLifecycleCommand, PutBucketLifecycleConfigurationCommand, HeadObjectCommand, GetObjectCommand, DeleteObjectCommand, PutObjectCommand, ListObjectsV2Command, CopyObjectCommand, DeleteObjectsCommand, CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand } from '@aws-sdk/client-s3';
|
|
3
3
|
import { Upload } from '@aws-sdk/lib-storage';
|
|
4
|
-
import { PassThrough } from 'stream';
|
|
4
|
+
import { PassThrough, Readable } from 'stream';
|
|
5
5
|
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
|
|
6
6
|
import { createHash } from 'crypto';
|
|
7
7
|
|
|
@@ -158,6 +158,10 @@ function validateOptions(options) {
|
|
|
158
158
|
if (isObject(multipart) && typeof multipart.partSizeBytes === "number" && multipart.partSizeBytes < MIN_PART_SIZE_BYTES) {
|
|
159
159
|
throw invalid("multipart.partSizeBytes must be >= 5 MB (S3 hard limit)");
|
|
160
160
|
}
|
|
161
|
+
const idle = options.uploadIdleTimeoutMs;
|
|
162
|
+
if (idle !== void 0 && (typeof idle !== "number" || !Number.isInteger(idle) || idle < 0)) {
|
|
163
|
+
throw invalid("uploadIdleTimeoutMs must be an integer >= 0 (0 disables the guard)");
|
|
164
|
+
}
|
|
161
165
|
if (options.serverSideEncryption === "aws:kms" && !options.kmsKeyId) {
|
|
162
166
|
throw invalid('kmsKeyId is required when serverSideEncryption === "aws:kms"');
|
|
163
167
|
}
|
|
@@ -211,6 +215,7 @@ function applyDefaults(options) {
|
|
|
211
215
|
const hasCredentials = Boolean(options.credentials.accessKeyId) && Boolean(options.credentials.secretAccessKey);
|
|
212
216
|
const publicBaseUrl = options.publicBaseUrl ?? `${trimTrailingSlashes(options.endpoint)}/${options.bucket}`;
|
|
213
217
|
const credentials = { ...options.credentials };
|
|
218
|
+
const requestTimeoutMs = options.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS;
|
|
214
219
|
const resolved = {
|
|
215
220
|
endpoint: options.endpoint,
|
|
216
221
|
region: options.region,
|
|
@@ -231,7 +236,8 @@ function applyDefaults(options) {
|
|
|
231
236
|
requestChecksumCalculation: options.requestChecksumCalculation ?? DEFAULT_CHECKSUM_CALCULATION,
|
|
232
237
|
responseChecksumValidation: options.responseChecksumValidation ?? DEFAULT_CHECKSUM_VALIDATION,
|
|
233
238
|
maxAttempts: options.maxAttempts ?? DEFAULT_MAX_ATTEMPTS,
|
|
234
|
-
requestTimeoutMs
|
|
239
|
+
requestTimeoutMs,
|
|
240
|
+
uploadIdleTimeoutMs: options.uploadIdleTimeoutMs ?? requestTimeoutMs,
|
|
235
241
|
hasCredentials
|
|
236
242
|
};
|
|
237
243
|
Object.defineProperty(resolved, "credentials", {
|
|
@@ -994,6 +1000,93 @@ FileScannerService = __decorateClass([
|
|
|
994
1000
|
__decorateParam(1, Inject(BYMAX_STORAGE_FILE_SCANNER))
|
|
995
1001
|
], FileScannerService);
|
|
996
1002
|
|
|
1003
|
+
// src/server/utils/describe-error.ts
|
|
1004
|
+
var NON_PRINTABLE_REJECTION = "non-printable rejection value";
|
|
1005
|
+
function describeError(value) {
|
|
1006
|
+
try {
|
|
1007
|
+
if (typeof value === "object" && value !== null) {
|
|
1008
|
+
const message = value.message;
|
|
1009
|
+
if (typeof message === "string") {
|
|
1010
|
+
return message;
|
|
1011
|
+
}
|
|
1012
|
+
}
|
|
1013
|
+
return String(value);
|
|
1014
|
+
} catch {
|
|
1015
|
+
return NON_PRINTABLE_REJECTION;
|
|
1016
|
+
}
|
|
1017
|
+
}
|
|
1018
|
+
var SourceIdleError = class extends Error {
|
|
1019
|
+
constructor(idleMs) {
|
|
1020
|
+
super(`The upload source delivered no bytes for ${String(idleMs)} ms`);
|
|
1021
|
+
this.idleMs = idleMs;
|
|
1022
|
+
this.name = "SourceIdleError";
|
|
1023
|
+
}
|
|
1024
|
+
idleMs;
|
|
1025
|
+
};
|
|
1026
|
+
function withinIdleTime(next, idleMs) {
|
|
1027
|
+
return new Promise((resolve, reject) => {
|
|
1028
|
+
const timer = setTimeout(() => {
|
|
1029
|
+
reject(new SourceIdleError(idleMs));
|
|
1030
|
+
}, idleMs);
|
|
1031
|
+
next.then(
|
|
1032
|
+
(value) => {
|
|
1033
|
+
clearTimeout(timer);
|
|
1034
|
+
resolve(value);
|
|
1035
|
+
},
|
|
1036
|
+
(error) => {
|
|
1037
|
+
clearTimeout(timer);
|
|
1038
|
+
reject(error instanceof Error ? error : new Error(describeError(error)));
|
|
1039
|
+
}
|
|
1040
|
+
);
|
|
1041
|
+
});
|
|
1042
|
+
}
|
|
1043
|
+
function withIdleTimeout(source, idleMs) {
|
|
1044
|
+
const iterator = source[Symbol.asyncIterator]();
|
|
1045
|
+
const wrapper = new Readable({
|
|
1046
|
+
objectMode: false,
|
|
1047
|
+
// No read-ahead: with a buffer to fill, Node calls read() while the consumer is paused, and the
|
|
1048
|
+
// clock would run for bytes nobody has asked for. At 0, read() runs only on demand.
|
|
1049
|
+
highWaterMark: 0,
|
|
1050
|
+
read() {
|
|
1051
|
+
withinIdleTime(iterator.next(), idleMs).then(
|
|
1052
|
+
(step) => {
|
|
1053
|
+
wrapper.push(step.done === true ? null : step.value);
|
|
1054
|
+
},
|
|
1055
|
+
(error) => {
|
|
1056
|
+
wrapper.destroy(error);
|
|
1057
|
+
}
|
|
1058
|
+
);
|
|
1059
|
+
},
|
|
1060
|
+
destroy(error, callback) {
|
|
1061
|
+
destroyStream(source);
|
|
1062
|
+
callback(error);
|
|
1063
|
+
}
|
|
1064
|
+
});
|
|
1065
|
+
wrapper.once("error", () => void 0);
|
|
1066
|
+
return wrapper;
|
|
1067
|
+
}
|
|
1068
|
+
function guardIdleBody(body, idleMs) {
|
|
1069
|
+
if (isBufferLike(body)) {
|
|
1070
|
+
return body;
|
|
1071
|
+
}
|
|
1072
|
+
return idleMs === 0 ? body : withIdleTimeout(body, idleMs);
|
|
1073
|
+
}
|
|
1074
|
+
|
|
1075
|
+
// src/server/errors/multipart-error-mapper.ts
|
|
1076
|
+
function mapMultipartError(err, context) {
|
|
1077
|
+
if (err instanceof SourceIdleError) {
|
|
1078
|
+
return new StorageException(STORAGE_ERROR_CODES.STORAGE_TIMEOUT, HttpStatus.GATEWAY_TIMEOUT, {
|
|
1079
|
+
...context,
|
|
1080
|
+
phase: "source-idle",
|
|
1081
|
+
idleMs: err.idleMs
|
|
1082
|
+
});
|
|
1083
|
+
}
|
|
1084
|
+
return new StorageException(STORAGE_ERROR_CODES.STORAGE_MULTIPART_ABORTED, void 0, {
|
|
1085
|
+
...context,
|
|
1086
|
+
awsMessage: describeError(err)
|
|
1087
|
+
});
|
|
1088
|
+
}
|
|
1089
|
+
|
|
997
1090
|
// src/server/utils/header-utils.ts
|
|
998
1091
|
function buildContentDisposition(perCall, defaultValue) {
|
|
999
1092
|
return perCall ?? defaultValue;
|
|
@@ -1189,7 +1282,7 @@ var StorageService = class {
|
|
|
1189
1282
|
await this.delete(finalKey, { bucket }).catch((deleteErr) => {
|
|
1190
1283
|
this.logger.error(
|
|
1191
1284
|
`Post-upload cleanup failed for infected/inconclusive object: ${finalKey}`,
|
|
1192
|
-
deleteErr
|
|
1285
|
+
describeError(deleteErr)
|
|
1193
1286
|
);
|
|
1194
1287
|
});
|
|
1195
1288
|
throw scanErr;
|
|
@@ -1230,7 +1323,7 @@ var StorageService = class {
|
|
|
1230
1323
|
if (err instanceof StorageException && err.code === STORAGE_ERROR_CODES.STORAGE_OBJECT_NOT_FOUND) {
|
|
1231
1324
|
return false;
|
|
1232
1325
|
}
|
|
1233
|
-
this.logger.warn(`exists() \u2014 treating a non-404 error as "false": ${err
|
|
1326
|
+
this.logger.warn(`exists() \u2014 treating a non-404 error as "false": ${describeError(err)}`);
|
|
1234
1327
|
return false;
|
|
1235
1328
|
}
|
|
1236
1329
|
}
|
|
@@ -1414,7 +1507,7 @@ var StorageService = class {
|
|
|
1414
1507
|
const params = {
|
|
1415
1508
|
Bucket: bucket,
|
|
1416
1509
|
Key: finalKey,
|
|
1417
|
-
Body: options.body,
|
|
1510
|
+
Body: guardIdleBody(options.body, this.options.uploadIdleTimeoutMs),
|
|
1418
1511
|
ContentType: options.contentType,
|
|
1419
1512
|
CacheControl: buildCacheControl(options.cacheControl, this.options.defaultCacheControl),
|
|
1420
1513
|
ContentDisposition: buildContentDisposition(
|
|
@@ -1450,11 +1543,7 @@ var StorageService = class {
|
|
|
1450
1543
|
multipart: true
|
|
1451
1544
|
});
|
|
1452
1545
|
} catch (err) {
|
|
1453
|
-
throw
|
|
1454
|
-
key: finalKey,
|
|
1455
|
-
bucket,
|
|
1456
|
-
awsMessage: err.message
|
|
1457
|
-
});
|
|
1546
|
+
throw mapMultipartError(err, { key: finalKey, bucket });
|
|
1458
1547
|
}
|
|
1459
1548
|
}
|
|
1460
1549
|
/** Converts a `Uint8Array` body to a `Buffer` for SDK safety; passes others through. */
|
|
@@ -1622,7 +1711,7 @@ var StorageService = class {
|
|
|
1622
1711
|
}
|
|
1623
1712
|
}
|
|
1624
1713
|
} catch (err) {
|
|
1625
|
-
const message =
|
|
1714
|
+
const message = describeError(err);
|
|
1626
1715
|
for (const key of chunk) {
|
|
1627
1716
|
failed.push({ key: this.keyResolver.stripPrefix(key), error: message });
|
|
1628
1717
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bymax-one/nest-storage",
|
|
3
|
-
"version": "1.1.
|
|
3
|
+
"version": "1.1.3",
|
|
4
4
|
"description": "Provider-agnostic S3-compatible object storage for NestJS. Works with AWS S3, DigitalOcean Spaces, Cloudflare R2, Backblaze B2, MinIO, Wasabi.",
|
|
5
5
|
"author": "Bymax One <support@bymax.one>",
|
|
6
6
|
"license": "MIT",
|