@bymax-one/nest-storage 1.1.14 → 1.1.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +44 -1
- package/README.md +9 -3
- package/dist/server/index.cjs +147 -88
- package/dist/server/index.d.cts +12 -4
- package/dist/server/index.d.ts +12 -4
- package/dist/server/index.mjs +147 -88
- package/package.json +3 -1
package/CHANGELOG.md
CHANGED
|
@@ -13,6 +13,47 @@ note on each entry is the compatibility contract — read it, not the number.
|
|
|
13
13
|
|
|
14
14
|
## [Unreleased]
|
|
15
15
|
|
|
16
|
+
## [1.1.16] - 2026-11-17
|
|
17
|
+
|
|
18
|
+
### Changed
|
|
19
|
+
|
|
20
|
+
- **The scanner's free-form `details` no longer reaches a client.** `STORAGE_SCAN_INFECTED` and
|
|
21
|
+
`STORAGE_SCAN_INCONCLUSIVE` copied the `details` an `IFileScanner` returned into the body of the error,
|
|
22
|
+
and that text is the scanner's own: an engine message, a quarantine path, an internal address. The body
|
|
23
|
+
now carries the `engine` and, for an infected object, the `threat` name, and nothing else the scanner
|
|
24
|
+
wrote. The `details` go to the warning log, written as one escaped, truncated token; a
|
|
25
|
+
rejected inconclusive result is now logged too.
|
|
26
|
+
|
|
27
|
+
**Apply to a derived backend:** a host that read `details.details` from either error must read the log
|
|
28
|
+
line instead, or put what it needs in `threat` or `engine`. The other fields of the body are unchanged.
|
|
29
|
+
A rejection whose `details` could not be serialised (a circular object, a `BigInt`, a `toJSON` that
|
|
30
|
+
throws) used to fail while the error body was written, and the host answered 500; it is now the typed
|
|
31
|
+
422.
|
|
32
|
+
|
|
33
|
+
- **The `abortSignal` option documentation states what a body read ends with.** It said a `download()` stream
|
|
34
|
+
errors with the raw `AbortError` when `bodyIdleTimeoutMs` is `0` and left out `downloadBuffer()`. It now says
|
|
35
|
+
that `downloadBuffer()` and a guarded `download()` stream fail `STORAGE_REQUEST_ABORTED`, and that with the
|
|
36
|
+
guard off a `download()` stream is the SDK's own and errors with the SDK's error. The behaviour is unchanged.
|
|
37
|
+
|
|
38
|
+
**Apply to a derived backend:** nothing.
|
|
39
|
+
|
|
40
|
+
## [1.1.15] - 2026-11-17
|
|
41
|
+
|
|
42
|
+
### Fixed
|
|
43
|
+
|
|
44
|
+
- **A download body that fails after its headers no longer reaches the caller as the SDK's raw error.** A
|
|
45
|
+
connection reset in the middle of a body (`ECONNRESET`, message `aborted`) passed through unmapped: a
|
|
46
|
+
caller that caught only `StorageException` missed it, and a Nest handler answered 500 for it. With the idle
|
|
47
|
+
guard on (the default), a failure of the body is now mapped as any other failure of the provider
|
|
48
|
+
(`STORAGE_PROVIDER_ERROR`, **503** for a cut connection, with its fixed `reason`), and `downloadBuffer()`
|
|
49
|
+
maps it in both modes, with `STORAGE_REQUEST_ABORTED` when the caller's signal aborted. Nothing a client
|
|
50
|
+
receives carries the driver's text.
|
|
51
|
+
|
|
52
|
+
**Apply to a derived backend:** a host that caught the raw `ECONNRESET` from `download()` or
|
|
53
|
+
`downloadBuffer()` must catch `StorageException` instead (the status is 503 for a reset). With
|
|
54
|
+
`bodyIdleTimeoutMs: 0`, the stream `download()` returns is still the SDK's own and still errors with the
|
|
55
|
+
SDK's error, as documented; only `downloadBuffer()` changes there.
|
|
56
|
+
|
|
16
57
|
## [1.1.14] - 2026-11-16
|
|
17
58
|
|
|
18
59
|
### Changed
|
|
@@ -865,4 +906,6 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
865
906
|
[1.1.12]: https://github.com/bymaxone/nest-storage/compare/v1.1.11...v1.1.12
|
|
866
907
|
[1.1.13]: https://github.com/bymaxone/nest-storage/compare/v1.1.12...v1.1.13
|
|
867
908
|
[1.1.14]: https://github.com/bymaxone/nest-storage/compare/v1.1.13...v1.1.14
|
|
868
|
-
[
|
|
909
|
+
[1.1.15]: https://github.com/bymaxone/nest-storage/compare/v1.1.14...v1.1.15
|
|
910
|
+
[1.1.16]: https://github.com/bymaxone/nest-storage/compare/v1.1.15...v1.1.16
|
|
911
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.16...HEAD
|
package/README.md
CHANGED
|
@@ -241,7 +241,7 @@ Options accepted by `forRoot` and `forRootAsync`:
|
|
|
241
241
|
|
|
242
242
|
**`uploadIdleTimeoutMs`** covers what `requestTimeoutMs` cannot: a multipart upload waiting for the next bytes of a stream has no request in flight, so a source that stops delivering would hold the upload open on the provider indefinitely. After this much silence the upload is aborted (the provider is sent `AbortMultipartUpload`), your stream is destroyed and `upload()` rejects with `STORAGE_TIMEOUT` and `details.phase` set to `'source-idle'`. The clock restarts on every chunk and runs only while the library is waiting for data, so a slow but live source succeeds and a slow provider does not trip it. It defaults to the value of `requestTimeoutMs`; `0` turns it off. It applies only to a stream on the multipart path: a single-shot upload is bounded by `requestTimeoutMs`, and a `Buffer` cannot stall.
|
|
243
243
|
|
|
244
|
-
**`bodyIdleTimeoutMs`** covers the other direction: `requestTimeoutMs` ends when the response headers arrive, so a provider that stops sending in the middle of a download would leave the stream, its connection and the caller waiting without limit. After this much silence while a read is pending, the stream fails with `STORAGE_TIMEOUT` and `details.phase` set to `'body-idle'`, and the connection is released. The clock runs only while the caller is waiting for data, so a consumer that is slow to read, for example a response piped to a slow client, never trips it, and every chunk restarts it. It defaults to the value of `requestTimeoutMs`; `0` turns it off. It applies to `download` and `downloadBuffer`.
|
|
244
|
+
**`bodyIdleTimeoutMs`** covers the other direction: `requestTimeoutMs` ends when the response headers arrive, so a provider that stops sending in the middle of a download would leave the stream, its connection and the caller waiting without limit. After this much silence while a read is pending, the stream fails with `STORAGE_TIMEOUT` and `details.phase` set to `'body-idle'`, and the connection is released. The clock runs only while the caller is waiting for data, so a consumer that is slow to read, for example a response piped to a slow client, never trips it, and every chunk restarts it. It defaults to the value of `requestTimeoutMs`; `0` turns it off. It applies to `download` and `downloadBuffer`. A body that fails after its headers arrived (a reset connection) ends as a `StorageException` like any other failure of the provider (`STORAGE_PROVIDER_ERROR`, **503** when the connection was cut); with the guard off only `download()` is the exception, its stream being the SDK's own and so erroring with the SDK's own error, while `downloadBuffer()` is mapped in both modes.
|
|
245
245
|
|
|
246
246
|
|
|
247
247
|
[aws-presigned-expiry]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
|
|
@@ -444,8 +444,9 @@ await this.storage.upload({
|
|
|
444
444
|
same error and `details.stored: true`: the object is already stored. The scan carries on, and an
|
|
445
445
|
object it finds infected is still deleted.
|
|
446
446
|
- A **download** body that is being read when the signal aborts errors as `STORAGE_REQUEST_ABORTED`
|
|
447
|
-
while the idle guard is on (the default `bodyIdleTimeoutMs`)
|
|
448
|
-
|
|
447
|
+
while the idle guard is on (the default `bodyIdleTimeoutMs`), and `downloadBuffer()` rejects with it in
|
|
448
|
+
both modes. With `bodyIdleTimeoutMs: 0` the stream `download()` returns is the SDK's own and errors with
|
|
449
|
+
the SDK's own error, whichever it is where the abort lands.
|
|
449
450
|
- The library keeps no listener on your signal after a call, so one signal can be reused across many
|
|
450
451
|
calls.
|
|
451
452
|
- `copy`, `deleteMany` and the signed-URL methods take no `abortSignal`.
|
|
@@ -594,6 +595,11 @@ stream longer than the limit is refused with `STORAGE_VALIDATION_FAILED` as soon
|
|
|
594
595
|
with no `maxSizeBytes` it is refused with a reason that says to set it or use `post-upload`. Buffer
|
|
595
596
|
bodies are not affected, and `post-upload` hands the scanner the key only, no body.
|
|
596
597
|
|
|
598
|
+
**What a client is told.** `STORAGE_SCAN_INFECTED` carries `details.engine` and `details.threat`, and
|
|
599
|
+
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`. The free-form `details` your scanner returns is the
|
|
600
|
+
scanner's own text, so it is written to the warning log (escaped and truncated) and never to the
|
|
601
|
+
body of the error.
|
|
602
|
+
|
|
597
603
|
**A rejected object that could not be removed.** In `post-upload` mode the object is already stored
|
|
598
604
|
when the scanner rejects it, so the library deletes it and rejects `upload()` with the verdict
|
|
599
605
|
(`STORAGE_SCAN_INFECTED` or `STORAGE_SCAN_INCONCLUSIVE`). If that delete fails the verdict is still the
|
package/dist/server/index.cjs
CHANGED
|
@@ -215,28 +215,7 @@ function assertTimerMs(options, name, min) {
|
|
|
215
215
|
);
|
|
216
216
|
}
|
|
217
217
|
}
|
|
218
|
-
function
|
|
219
|
-
if (!isObject(options)) {
|
|
220
|
-
throw invalid("options object is required");
|
|
221
|
-
}
|
|
222
|
-
if (!isNonEmptyString(options.endpoint)) {
|
|
223
|
-
throw invalid("options.endpoint must be a non-empty string");
|
|
224
|
-
}
|
|
225
|
-
for (const name of ["endpoint", "publicBaseUrl", "cdnBaseUrl"]) {
|
|
226
|
-
const value = options[name];
|
|
227
|
-
if (typeof value === "string" && urlCarriesCredentials(value)) {
|
|
228
|
-
throw invalid(`options.${name} must not carry credentials; pass them in options.credentials`);
|
|
229
|
-
}
|
|
230
|
-
}
|
|
231
|
-
if (!isNonEmptyString(options.region)) {
|
|
232
|
-
throw invalid("options.region must be a non-empty string");
|
|
233
|
-
}
|
|
234
|
-
if (!isNonEmptyString(options.bucket)) {
|
|
235
|
-
throw invalid("options.bucket must be a non-empty string");
|
|
236
|
-
}
|
|
237
|
-
if (!isObject(options.credentials)) {
|
|
238
|
-
throw invalid("options.credentials is required");
|
|
239
|
-
}
|
|
218
|
+
function assertLimitsAndEncryption(options) {
|
|
240
219
|
const signedUrls = options.signedUrls;
|
|
241
220
|
if (isObject(signedUrls) && typeof signedUrls.maxTtlSeconds === "number" && signedUrls.maxTtlSeconds <= 0) {
|
|
242
221
|
throw invalid("signedUrls.maxTtlSeconds must be > 0");
|
|
@@ -261,6 +240,30 @@ function validateOptions(options) {
|
|
|
261
240
|
}
|
|
262
241
|
}
|
|
263
242
|
}
|
|
243
|
+
function validateOptions(options) {
|
|
244
|
+
if (!isObject(options)) {
|
|
245
|
+
throw invalid("options object is required");
|
|
246
|
+
}
|
|
247
|
+
if (!isNonEmptyString(options.endpoint)) {
|
|
248
|
+
throw invalid("options.endpoint must be a non-empty string");
|
|
249
|
+
}
|
|
250
|
+
for (const name of ["endpoint", "publicBaseUrl", "cdnBaseUrl"]) {
|
|
251
|
+
const value = options[name];
|
|
252
|
+
if (typeof value === "string" && urlCarriesCredentials(value)) {
|
|
253
|
+
throw invalid(`options.${name} must not carry credentials; pass them in options.credentials`);
|
|
254
|
+
}
|
|
255
|
+
}
|
|
256
|
+
if (!isNonEmptyString(options.region)) {
|
|
257
|
+
throw invalid("options.region must be a non-empty string");
|
|
258
|
+
}
|
|
259
|
+
if (!isNonEmptyString(options.bucket)) {
|
|
260
|
+
throw invalid("options.bucket must be a non-empty string");
|
|
261
|
+
}
|
|
262
|
+
if (!isObject(options.credentials)) {
|
|
263
|
+
throw invalid("options.credentials is required");
|
|
264
|
+
}
|
|
265
|
+
assertLimitsAndEncryption(options);
|
|
266
|
+
}
|
|
264
267
|
|
|
265
268
|
// src/server/utils/trim-trailing-slashes.ts
|
|
266
269
|
var SLASH = "/";
|
|
@@ -306,6 +309,13 @@ function resolveSignedUrls(configured) {
|
|
|
306
309
|
const merged = { ...DEFAULT_SIGNED_URLS, ...configured ?? {} };
|
|
307
310
|
return { ...merged, maxTtlSeconds: Math.min(merged.maxTtlSeconds, MAX_SIGNED_URL_TTL_SECONDS) };
|
|
308
311
|
}
|
|
312
|
+
function attachCredentials(resolved, credentials) {
|
|
313
|
+
Object.defineProperty(resolved, "credentials", {
|
|
314
|
+
get: () => credentials,
|
|
315
|
+
enumerable: false,
|
|
316
|
+
configurable: false
|
|
317
|
+
});
|
|
318
|
+
}
|
|
309
319
|
function applyDefaults(options) {
|
|
310
320
|
const hasCredentials = Boolean(options.credentials.accessKeyId) && Boolean(options.credentials.secretAccessKey);
|
|
311
321
|
const publicBaseUrl = options.publicBaseUrl ?? `${trimTrailingSlashes(options.endpoint)}/${options.bucket}`;
|
|
@@ -336,11 +346,7 @@ function applyDefaults(options) {
|
|
|
336
346
|
bodyIdleTimeoutMs: options.bodyIdleTimeoutMs ?? requestTimeoutMs,
|
|
337
347
|
hasCredentials
|
|
338
348
|
};
|
|
339
|
-
|
|
340
|
-
get: () => credentials,
|
|
341
|
-
enumerable: false,
|
|
342
|
-
configurable: false
|
|
343
|
-
});
|
|
349
|
+
attachCredentials(resolved, credentials);
|
|
344
350
|
return resolved;
|
|
345
351
|
}
|
|
346
352
|
|
|
@@ -641,6 +647,31 @@ function awsCodeOf(err) {
|
|
|
641
647
|
function isNoSuchBucket(err) {
|
|
642
648
|
return awsCodeOf(err) === "NoSuchBucket";
|
|
643
649
|
}
|
|
650
|
+
function mapAnsweredStatus(e, details) {
|
|
651
|
+
const status = e.$metadata?.httpStatusCode;
|
|
652
|
+
if (e.name === "NotFound" || status === 404) {
|
|
653
|
+
return new StorageException(
|
|
654
|
+
STORAGE_ERROR_CODES.STORAGE_OBJECT_NOT_FOUND,
|
|
655
|
+
common.HttpStatus.NOT_FOUND,
|
|
656
|
+
details
|
|
657
|
+
);
|
|
658
|
+
}
|
|
659
|
+
if (status === 304) {
|
|
660
|
+
return new StorageException(
|
|
661
|
+
STORAGE_ERROR_CODES.STORAGE_NOT_MODIFIED,
|
|
662
|
+
common.HttpStatus.NOT_MODIFIED,
|
|
663
|
+
details
|
|
664
|
+
);
|
|
665
|
+
}
|
|
666
|
+
if (status === 412) {
|
|
667
|
+
return new StorageException(
|
|
668
|
+
STORAGE_ERROR_CODES.STORAGE_PRECONDITION_FAILED,
|
|
669
|
+
common.HttpStatus.PRECONDITION_FAILED,
|
|
670
|
+
details
|
|
671
|
+
);
|
|
672
|
+
}
|
|
673
|
+
return void 0;
|
|
674
|
+
}
|
|
644
675
|
function mapAwsError(err, context) {
|
|
645
676
|
const e = err ?? {};
|
|
646
677
|
const httpStatus = e.$metadata?.httpStatusCode;
|
|
@@ -661,27 +692,8 @@ function mapAwsError(err, context) {
|
|
|
661
692
|
details
|
|
662
693
|
);
|
|
663
694
|
}
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
STORAGE_ERROR_CODES.STORAGE_OBJECT_NOT_FOUND,
|
|
667
|
-
common.HttpStatus.NOT_FOUND,
|
|
668
|
-
details
|
|
669
|
-
);
|
|
670
|
-
}
|
|
671
|
-
if (httpStatus === 304) {
|
|
672
|
-
return new StorageException(
|
|
673
|
-
STORAGE_ERROR_CODES.STORAGE_NOT_MODIFIED,
|
|
674
|
-
common.HttpStatus.NOT_MODIFIED,
|
|
675
|
-
details
|
|
676
|
-
);
|
|
677
|
-
}
|
|
678
|
-
if (httpStatus === 412) {
|
|
679
|
-
return new StorageException(
|
|
680
|
-
STORAGE_ERROR_CODES.STORAGE_PRECONDITION_FAILED,
|
|
681
|
-
common.HttpStatus.PRECONDITION_FAILED,
|
|
682
|
-
details
|
|
683
|
-
);
|
|
684
|
-
}
|
|
695
|
+
const answered = mapAnsweredStatus(e, details);
|
|
696
|
+
if (answered) return answered;
|
|
685
697
|
if (e.name === "TimeoutError") {
|
|
686
698
|
logProviderFailure(err, context);
|
|
687
699
|
return new StorageException(STORAGE_ERROR_CODES.STORAGE_TIMEOUT, void 0, details);
|
|
@@ -1216,6 +1228,21 @@ ValidationService = __decorateClass([
|
|
|
1216
1228
|
__decorateParam(0, common.Inject(BYMAX_STORAGE_OPTIONS)),
|
|
1217
1229
|
__decorateParam(1, common.Inject(BYMAX_STORAGE_UPLOAD_VALIDATORS))
|
|
1218
1230
|
], ValidationService);
|
|
1231
|
+
var UNSERIALISABLE = "[unserialisable]";
|
|
1232
|
+
var MAX_SCANNER_DETAILS_LENGTH = 300;
|
|
1233
|
+
function toJson(value) {
|
|
1234
|
+
return JSON.stringify(value);
|
|
1235
|
+
}
|
|
1236
|
+
function scannerDetailsForLog(details) {
|
|
1237
|
+
if (details === void 0) return "";
|
|
1238
|
+
let text;
|
|
1239
|
+
try {
|
|
1240
|
+
text = toJson(details) ?? UNSERIALISABLE;
|
|
1241
|
+
} catch {
|
|
1242
|
+
text = UNSERIALISABLE;
|
|
1243
|
+
}
|
|
1244
|
+
return ` details=${logValue(text.slice(0, MAX_SCANNER_DETAILS_LENGTH))}`;
|
|
1245
|
+
}
|
|
1219
1246
|
var FileScannerService = class {
|
|
1220
1247
|
constructor(options, scanner) {
|
|
1221
1248
|
this.options = options;
|
|
@@ -1267,18 +1294,17 @@ var FileScannerService = class {
|
|
|
1267
1294
|
}
|
|
1268
1295
|
const result = await this.scanner.scan(input);
|
|
1269
1296
|
if (result.status === "infected") {
|
|
1270
|
-
this.logger.warn(`Infected file detected: key=${logValue(input.key)} engine=${logValue(result.engine)} threat=${logValue(result.threat ?? "unknown")}`);
|
|
1297
|
+
this.logger.warn(`Infected file detected: key=${logValue(input.key)} engine=${logValue(result.engine)} threat=${logValue(result.threat ?? "unknown")}${scannerDetailsForLog(result.details)}`);
|
|
1271
1298
|
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_SCAN_INFECTED, void 0, {
|
|
1272
1299
|
engine: result.engine,
|
|
1273
|
-
threat: result.threat
|
|
1274
|
-
details: result.details
|
|
1300
|
+
threat: result.threat
|
|
1275
1301
|
});
|
|
1276
1302
|
}
|
|
1277
1303
|
if (result.status === "unknown") {
|
|
1278
1304
|
if (this.options.scanner?.rejectOnUnknown) {
|
|
1305
|
+
this.logger.warn(`Inconclusive scan result rejected: key=${logValue(input.key)} engine=${logValue(result.engine)}${scannerDetailsForLog(result.details)}`);
|
|
1279
1306
|
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_SCAN_INCONCLUSIVE, void 0, {
|
|
1280
|
-
engine: result.engine
|
|
1281
|
-
details: result.details
|
|
1307
|
+
engine: result.engine
|
|
1282
1308
|
});
|
|
1283
1309
|
}
|
|
1284
1310
|
this.logger.warn(`Inconclusive scan result accepted: key=${logValue(input.key)} engine=${logValue(result.engine)}`);
|
|
@@ -1299,6 +1325,10 @@ var SourceIdleError = class extends Error {
|
|
|
1299
1325
|
}
|
|
1300
1326
|
idleMs;
|
|
1301
1327
|
};
|
|
1328
|
+
function isError(value) {
|
|
1329
|
+
const candidate = value;
|
|
1330
|
+
return typeof candidate?.name === "string" && typeof candidate.message === "string";
|
|
1331
|
+
}
|
|
1302
1332
|
function withinIdleTime(next, idleMs) {
|
|
1303
1333
|
return new Promise((resolve, reject) => {
|
|
1304
1334
|
const timer = setTimeout(() => {
|
|
@@ -1311,7 +1341,7 @@ function withinIdleTime(next, idleMs) {
|
|
|
1311
1341
|
},
|
|
1312
1342
|
(error) => {
|
|
1313
1343
|
clearTimeout(timer);
|
|
1314
|
-
reject(error
|
|
1344
|
+
reject(isError(error) ? error : new Error(describeError(error)));
|
|
1315
1345
|
}
|
|
1316
1346
|
);
|
|
1317
1347
|
});
|
|
@@ -1701,6 +1731,24 @@ function boundStreamSize(body, declaredSize, options) {
|
|
|
1701
1731
|
}
|
|
1702
1732
|
return withSizeLimit(body, limit, declaredSize === void 0 ? "max" : "declared");
|
|
1703
1733
|
}
|
|
1734
|
+
async function bufferStreamForScan(body, finalKey, maxSizeBytes) {
|
|
1735
|
+
if (maxSizeBytes === void 0) {
|
|
1736
|
+
destroyStream(body);
|
|
1737
|
+
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1738
|
+
key: finalKey,
|
|
1739
|
+
reason: "Pre-upload scanning of a stream needs validation.maxSizeBytes to bound the buffer; set it or use scanner mode post-upload"
|
|
1740
|
+
});
|
|
1741
|
+
}
|
|
1742
|
+
const buffered = await readStreamUpTo(body, maxSizeBytes);
|
|
1743
|
+
if (buffered === void 0) {
|
|
1744
|
+
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1745
|
+
key: finalKey,
|
|
1746
|
+
maxSizeBytes,
|
|
1747
|
+
reason: "The stream is longer than validation.maxSizeBytes, so it cannot be scanned before upload"
|
|
1748
|
+
});
|
|
1749
|
+
}
|
|
1750
|
+
return buffered;
|
|
1751
|
+
}
|
|
1704
1752
|
|
|
1705
1753
|
// src/server/utils/download-body.ts
|
|
1706
1754
|
function guardDownloadBody(body, idleMs, context, signal) {
|
|
@@ -1715,9 +1763,15 @@ function guardDownloadBody(body, idleMs, context, signal) {
|
|
|
1715
1763
|
idleMs: error.idleMs
|
|
1716
1764
|
});
|
|
1717
1765
|
}
|
|
1718
|
-
return signal?.aborted ? requestAborted(signal, context) : error;
|
|
1766
|
+
return signal?.aborted ? requestAborted(signal, context) : mapAwsError(error, context);
|
|
1719
1767
|
});
|
|
1720
1768
|
}
|
|
1769
|
+
function mapBodyReadFailure(err, signal, context) {
|
|
1770
|
+
if (err instanceof StorageException) {
|
|
1771
|
+
return err;
|
|
1772
|
+
}
|
|
1773
|
+
return signal?.aborted ? requestAborted(signal, context) : mapAwsError(err, context);
|
|
1774
|
+
}
|
|
1721
1775
|
|
|
1722
1776
|
// src/server/utils/public-url.ts
|
|
1723
1777
|
function buildPublicUrl(options, finalKey, bucket) {
|
|
@@ -1807,7 +1861,7 @@ function assertMaxBytes(maxBytes) {
|
|
|
1807
1861
|
});
|
|
1808
1862
|
}
|
|
1809
1863
|
}
|
|
1810
|
-
async function
|
|
1864
|
+
async function readBody(stream, metadata, maxBytes) {
|
|
1811
1865
|
if (maxBytes === void 0) {
|
|
1812
1866
|
return readStream(stream);
|
|
1813
1867
|
}
|
|
@@ -1822,6 +1876,17 @@ async function readDownloadBody(stream, metadata, maxBytes) {
|
|
|
1822
1876
|
}
|
|
1823
1877
|
return buffer;
|
|
1824
1878
|
}
|
|
1879
|
+
async function readDownloadBody(stream, metadata, maxBytes, signal) {
|
|
1880
|
+
try {
|
|
1881
|
+
return await readBody(stream, metadata, maxBytes);
|
|
1882
|
+
} catch (err) {
|
|
1883
|
+
throw mapBodyReadFailure(err, signal, {
|
|
1884
|
+
key: metadata.key,
|
|
1885
|
+
bucket: metadata.bucket,
|
|
1886
|
+
op: "downloadBuffer"
|
|
1887
|
+
});
|
|
1888
|
+
}
|
|
1889
|
+
}
|
|
1825
1890
|
|
|
1826
1891
|
// src/server/services/storage.service.ts
|
|
1827
1892
|
var MAX_LIST_KEYS = 1e3;
|
|
@@ -1878,7 +1943,8 @@ exports.StorageService = class StorageService {
|
|
|
1878
1943
|
}
|
|
1879
1944
|
const finalKey = this.keyResolver.normalize(options.key);
|
|
1880
1945
|
const bucket = this.resolveBucket(options.bucket);
|
|
1881
|
-
|
|
1946
|
+
const context = { key: finalKey, bucket, op: "upload" };
|
|
1947
|
+
throwIfAborted(options.abortSignal, context);
|
|
1882
1948
|
if (options.idempotencyKey) {
|
|
1883
1949
|
const cacheKey = this.idempotencyCache.computeKey(options.idempotencyKey, finalKey);
|
|
1884
1950
|
const cached = this.idempotencyCache.get(cacheKey);
|
|
@@ -1888,7 +1954,25 @@ exports.StorageService = class StorageService {
|
|
|
1888
1954
|
return { ...cached, fromIdempotencyCache: true };
|
|
1889
1955
|
}
|
|
1890
1956
|
}
|
|
1891
|
-
const
|
|
1957
|
+
const validatedOptions = await this.validateAndScan(options, finalKey, bucket, context);
|
|
1958
|
+
throwIfAborted(options.abortSignal, context);
|
|
1959
|
+
const strategy = pickUploadStrategy(
|
|
1960
|
+
validatedOptions.body,
|
|
1961
|
+
validatedOptions.size,
|
|
1962
|
+
this.options.multipart.thresholdBytes
|
|
1963
|
+
);
|
|
1964
|
+
const result = strategy === "multipart" ? await this.uploadMultipart(validatedOptions, finalKey, bucket) : await this.uploadSingleShot(validatedOptions, finalKey, bucket);
|
|
1965
|
+
const stored = { ...context, stored: true };
|
|
1966
|
+
await abortable(this.runPostUploadScan(finalKey, bucket, options), options.abortSignal, stored);
|
|
1967
|
+
this.rememberUpload(options, finalKey, result);
|
|
1968
|
+
return result;
|
|
1969
|
+
}
|
|
1970
|
+
/**
|
|
1971
|
+
* Validates the upload and runs the pre-upload scan, returning the options with the body to send.
|
|
1972
|
+
* Reading the body (a validator peeking at it, the scan buffering it) happens before any request, so
|
|
1973
|
+
* the signal is watched here: a silent stream must not hold the call.
|
|
1974
|
+
*/
|
|
1975
|
+
async validateAndScan(options, finalKey, bucket, context) {
|
|
1892
1976
|
const signal = options.abortSignal;
|
|
1893
1977
|
const input = validationInput(options, finalKey);
|
|
1894
1978
|
const validated = await abortable(this.validation.validate(input), signal, context);
|
|
@@ -1900,17 +1984,7 @@ exports.StorageService = class StorageService {
|
|
|
1900
1984
|
const body = boundStreamSize(scanned, options.size, this.options);
|
|
1901
1985
|
const validatedOptions = { ...options, body };
|
|
1902
1986
|
await abortable(this.runPreUploadScan(body, finalKey, bucket, options), signal, context);
|
|
1903
|
-
|
|
1904
|
-
const strategy = pickUploadStrategy(
|
|
1905
|
-
validatedOptions.body,
|
|
1906
|
-
validatedOptions.size,
|
|
1907
|
-
this.options.multipart.thresholdBytes
|
|
1908
|
-
);
|
|
1909
|
-
const result = strategy === "multipart" ? await this.uploadMultipart(validatedOptions, finalKey, bucket) : await this.uploadSingleShot(validatedOptions, finalKey, bucket);
|
|
1910
|
-
const stored = { ...context, stored: true };
|
|
1911
|
-
await abortable(this.runPostUploadScan(finalKey, bucket, options), signal, stored);
|
|
1912
|
-
this.rememberUpload(options, finalKey, result);
|
|
1913
|
-
return result;
|
|
1987
|
+
return validatedOptions;
|
|
1914
1988
|
}
|
|
1915
1989
|
/**
|
|
1916
1990
|
* In pre-upload scan mode a stream body is read into one `Buffer`, so the scanner
|
|
@@ -1923,23 +1997,7 @@ exports.StorageService = class StorageService {
|
|
|
1923
1997
|
if (!this.scanner.isEnabled() || this.scanner.getMode() !== "pre-upload" || isBufferLike(body)) {
|
|
1924
1998
|
return body;
|
|
1925
1999
|
}
|
|
1926
|
-
|
|
1927
|
-
if (maxSizeBytes === void 0) {
|
|
1928
|
-
destroyStream(body);
|
|
1929
|
-
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1930
|
-
key: finalKey,
|
|
1931
|
-
reason: "Pre-upload scanning of a stream needs validation.maxSizeBytes to bound the buffer; set it or use scanner mode post-upload"
|
|
1932
|
-
});
|
|
1933
|
-
}
|
|
1934
|
-
const buffered = await readStreamUpTo(body, maxSizeBytes);
|
|
1935
|
-
if (buffered === void 0) {
|
|
1936
|
-
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1937
|
-
key: finalKey,
|
|
1938
|
-
maxSizeBytes,
|
|
1939
|
-
reason: "The stream is longer than validation.maxSizeBytes, so it cannot be scanned before upload"
|
|
1940
|
-
});
|
|
1941
|
-
}
|
|
1942
|
-
return buffered;
|
|
2000
|
+
return bufferStreamForScan(body, finalKey, this.options.validation?.maxSizeBytes);
|
|
1943
2001
|
}
|
|
1944
2002
|
/** Runs a pre-upload scan when configured; throws on infected. */
|
|
1945
2003
|
async runPreUploadScan(body, finalKey, bucket, options) {
|
|
@@ -2095,7 +2153,7 @@ exports.StorageService = class StorageService {
|
|
|
2095
2153
|
const stream = guardDownloadBody(
|
|
2096
2154
|
response.Body,
|
|
2097
2155
|
this.options.bodyIdleTimeoutMs,
|
|
2098
|
-
|
|
2156
|
+
context,
|
|
2099
2157
|
options.abortSignal
|
|
2100
2158
|
);
|
|
2101
2159
|
return { stream, metadata: this.toObjectMetadata(response, finalKey, bucket) };
|
|
@@ -2118,7 +2176,8 @@ exports.StorageService = class StorageService {
|
|
|
2118
2176
|
async downloadBuffer(options) {
|
|
2119
2177
|
assertMaxBytes(options.maxBytes);
|
|
2120
2178
|
const { stream, metadata } = await this.download(options);
|
|
2121
|
-
|
|
2179
|
+
const buffer = await readDownloadBody(stream, metadata, options.maxBytes, options.abortSignal);
|
|
2180
|
+
return { buffer, metadata };
|
|
2122
2181
|
}
|
|
2123
2182
|
/**
|
|
2124
2183
|
* Deletes an object. Idempotent: a missing object is a no-op (logged as a
|
package/dist/server/index.d.cts
CHANGED
|
@@ -70,6 +70,7 @@ interface FileScanResult {
|
|
|
70
70
|
engine: string;
|
|
71
71
|
/** Threat name when `status === 'infected'`. */
|
|
72
72
|
threat?: string;
|
|
73
|
+
/** Free-form scanner text. It is logged, escaped and cut at 300 characters, and is never put in an error body. */
|
|
73
74
|
details?: Record<string, unknown>;
|
|
74
75
|
}
|
|
75
76
|
|
|
@@ -380,10 +381,11 @@ interface DownloadOptions {
|
|
|
380
381
|
/**
|
|
381
382
|
* Ends the call when it aborts: the request is dropped and the call rejects
|
|
382
383
|
* `STORAGE_REQUEST_ABORTED` with `details.reason` set to the signal's reason name. A signal that is
|
|
383
|
-
* already aborted makes no request.
|
|
384
|
-
* `
|
|
385
|
-
*
|
|
386
|
-
*
|
|
384
|
+
* already aborted makes no request. It also ends a body that is still being read, in
|
|
385
|
+
* `downloadBuffer()` and in a `download()` stream while the idle guard is on (the default
|
|
386
|
+
* `bodyIdleTimeoutMs`): both fail `STORAGE_REQUEST_ABORTED`. With `bodyIdleTimeoutMs` set to `0`,
|
|
387
|
+
* a `download()` stream is the SDK's own and errors with the SDK's error, which is not a
|
|
388
|
+
* `StorageException`. The library keeps no listener on the signal after the call.
|
|
387
389
|
*/
|
|
388
390
|
abortSignal?: AbortSignal;
|
|
389
391
|
}
|
|
@@ -805,6 +807,12 @@ declare class StorageService {
|
|
|
805
807
|
upload(options: UploadOptions): Promise<UploadResult>;
|
|
806
808
|
/** The upload itself; `upload()` destroys a stream body when this rejects. */
|
|
807
809
|
private uploadOwningBody;
|
|
810
|
+
/**
|
|
811
|
+
* Validates the upload and runs the pre-upload scan, returning the options with the body to send.
|
|
812
|
+
* Reading the body (a validator peeking at it, the scan buffering it) happens before any request, so
|
|
813
|
+
* the signal is watched here: a silent stream must not hold the call.
|
|
814
|
+
*/
|
|
815
|
+
private validateAndScan;
|
|
808
816
|
/**
|
|
809
817
|
* In pre-upload scan mode a stream body is read into one `Buffer`, so the scanner
|
|
810
818
|
* and the upload see the same bytes; a stream is single-use, and one the scanner
|
package/dist/server/index.d.ts
CHANGED
|
@@ -70,6 +70,7 @@ interface FileScanResult {
|
|
|
70
70
|
engine: string;
|
|
71
71
|
/** Threat name when `status === 'infected'`. */
|
|
72
72
|
threat?: string;
|
|
73
|
+
/** Free-form scanner text. It is logged, escaped and cut at 300 characters, and is never put in an error body. */
|
|
73
74
|
details?: Record<string, unknown>;
|
|
74
75
|
}
|
|
75
76
|
|
|
@@ -380,10 +381,11 @@ interface DownloadOptions {
|
|
|
380
381
|
/**
|
|
381
382
|
* Ends the call when it aborts: the request is dropped and the call rejects
|
|
382
383
|
* `STORAGE_REQUEST_ABORTED` with `details.reason` set to the signal's reason name. A signal that is
|
|
383
|
-
* already aborted makes no request.
|
|
384
|
-
* `
|
|
385
|
-
*
|
|
386
|
-
*
|
|
384
|
+
* already aborted makes no request. It also ends a body that is still being read, in
|
|
385
|
+
* `downloadBuffer()` and in a `download()` stream while the idle guard is on (the default
|
|
386
|
+
* `bodyIdleTimeoutMs`): both fail `STORAGE_REQUEST_ABORTED`. With `bodyIdleTimeoutMs` set to `0`,
|
|
387
|
+
* a `download()` stream is the SDK's own and errors with the SDK's error, which is not a
|
|
388
|
+
* `StorageException`. The library keeps no listener on the signal after the call.
|
|
387
389
|
*/
|
|
388
390
|
abortSignal?: AbortSignal;
|
|
389
391
|
}
|
|
@@ -805,6 +807,12 @@ declare class StorageService {
|
|
|
805
807
|
upload(options: UploadOptions): Promise<UploadResult>;
|
|
806
808
|
/** The upload itself; `upload()` destroys a stream body when this rejects. */
|
|
807
809
|
private uploadOwningBody;
|
|
810
|
+
/**
|
|
811
|
+
* Validates the upload and runs the pre-upload scan, returning the options with the body to send.
|
|
812
|
+
* Reading the body (a validator peeking at it, the scan buffering it) happens before any request, so
|
|
813
|
+
* the signal is watched here: a silent stream must not hold the call.
|
|
814
|
+
*/
|
|
815
|
+
private validateAndScan;
|
|
808
816
|
/**
|
|
809
817
|
* In pre-upload scan mode a stream body is read into one `Buffer`, so the scanner
|
|
810
818
|
* and the upload see the same bytes; a stream is single-use, and one the scanner
|
package/dist/server/index.mjs
CHANGED
|
@@ -213,28 +213,7 @@ function assertTimerMs(options, name, min) {
|
|
|
213
213
|
);
|
|
214
214
|
}
|
|
215
215
|
}
|
|
216
|
-
function
|
|
217
|
-
if (!isObject(options)) {
|
|
218
|
-
throw invalid("options object is required");
|
|
219
|
-
}
|
|
220
|
-
if (!isNonEmptyString(options.endpoint)) {
|
|
221
|
-
throw invalid("options.endpoint must be a non-empty string");
|
|
222
|
-
}
|
|
223
|
-
for (const name of ["endpoint", "publicBaseUrl", "cdnBaseUrl"]) {
|
|
224
|
-
const value = options[name];
|
|
225
|
-
if (typeof value === "string" && urlCarriesCredentials(value)) {
|
|
226
|
-
throw invalid(`options.${name} must not carry credentials; pass them in options.credentials`);
|
|
227
|
-
}
|
|
228
|
-
}
|
|
229
|
-
if (!isNonEmptyString(options.region)) {
|
|
230
|
-
throw invalid("options.region must be a non-empty string");
|
|
231
|
-
}
|
|
232
|
-
if (!isNonEmptyString(options.bucket)) {
|
|
233
|
-
throw invalid("options.bucket must be a non-empty string");
|
|
234
|
-
}
|
|
235
|
-
if (!isObject(options.credentials)) {
|
|
236
|
-
throw invalid("options.credentials is required");
|
|
237
|
-
}
|
|
216
|
+
function assertLimitsAndEncryption(options) {
|
|
238
217
|
const signedUrls = options.signedUrls;
|
|
239
218
|
if (isObject(signedUrls) && typeof signedUrls.maxTtlSeconds === "number" && signedUrls.maxTtlSeconds <= 0) {
|
|
240
219
|
throw invalid("signedUrls.maxTtlSeconds must be > 0");
|
|
@@ -259,6 +238,30 @@ function validateOptions(options) {
|
|
|
259
238
|
}
|
|
260
239
|
}
|
|
261
240
|
}
|
|
241
|
+
function validateOptions(options) {
|
|
242
|
+
if (!isObject(options)) {
|
|
243
|
+
throw invalid("options object is required");
|
|
244
|
+
}
|
|
245
|
+
if (!isNonEmptyString(options.endpoint)) {
|
|
246
|
+
throw invalid("options.endpoint must be a non-empty string");
|
|
247
|
+
}
|
|
248
|
+
for (const name of ["endpoint", "publicBaseUrl", "cdnBaseUrl"]) {
|
|
249
|
+
const value = options[name];
|
|
250
|
+
if (typeof value === "string" && urlCarriesCredentials(value)) {
|
|
251
|
+
throw invalid(`options.${name} must not carry credentials; pass them in options.credentials`);
|
|
252
|
+
}
|
|
253
|
+
}
|
|
254
|
+
if (!isNonEmptyString(options.region)) {
|
|
255
|
+
throw invalid("options.region must be a non-empty string");
|
|
256
|
+
}
|
|
257
|
+
if (!isNonEmptyString(options.bucket)) {
|
|
258
|
+
throw invalid("options.bucket must be a non-empty string");
|
|
259
|
+
}
|
|
260
|
+
if (!isObject(options.credentials)) {
|
|
261
|
+
throw invalid("options.credentials is required");
|
|
262
|
+
}
|
|
263
|
+
assertLimitsAndEncryption(options);
|
|
264
|
+
}
|
|
262
265
|
|
|
263
266
|
// src/server/utils/trim-trailing-slashes.ts
|
|
264
267
|
var SLASH = "/";
|
|
@@ -304,6 +307,13 @@ function resolveSignedUrls(configured) {
|
|
|
304
307
|
const merged = { ...DEFAULT_SIGNED_URLS, ...configured ?? {} };
|
|
305
308
|
return { ...merged, maxTtlSeconds: Math.min(merged.maxTtlSeconds, MAX_SIGNED_URL_TTL_SECONDS) };
|
|
306
309
|
}
|
|
310
|
+
function attachCredentials(resolved, credentials) {
|
|
311
|
+
Object.defineProperty(resolved, "credentials", {
|
|
312
|
+
get: () => credentials,
|
|
313
|
+
enumerable: false,
|
|
314
|
+
configurable: false
|
|
315
|
+
});
|
|
316
|
+
}
|
|
307
317
|
function applyDefaults(options) {
|
|
308
318
|
const hasCredentials = Boolean(options.credentials.accessKeyId) && Boolean(options.credentials.secretAccessKey);
|
|
309
319
|
const publicBaseUrl = options.publicBaseUrl ?? `${trimTrailingSlashes(options.endpoint)}/${options.bucket}`;
|
|
@@ -334,11 +344,7 @@ function applyDefaults(options) {
|
|
|
334
344
|
bodyIdleTimeoutMs: options.bodyIdleTimeoutMs ?? requestTimeoutMs,
|
|
335
345
|
hasCredentials
|
|
336
346
|
};
|
|
337
|
-
|
|
338
|
-
get: () => credentials,
|
|
339
|
-
enumerable: false,
|
|
340
|
-
configurable: false
|
|
341
|
-
});
|
|
347
|
+
attachCredentials(resolved, credentials);
|
|
342
348
|
return resolved;
|
|
343
349
|
}
|
|
344
350
|
|
|
@@ -639,6 +645,31 @@ function awsCodeOf(err) {
|
|
|
639
645
|
function isNoSuchBucket(err) {
|
|
640
646
|
return awsCodeOf(err) === "NoSuchBucket";
|
|
641
647
|
}
|
|
648
|
+
function mapAnsweredStatus(e, details) {
|
|
649
|
+
const status = e.$metadata?.httpStatusCode;
|
|
650
|
+
if (e.name === "NotFound" || status === 404) {
|
|
651
|
+
return new StorageException(
|
|
652
|
+
STORAGE_ERROR_CODES.STORAGE_OBJECT_NOT_FOUND,
|
|
653
|
+
HttpStatus.NOT_FOUND,
|
|
654
|
+
details
|
|
655
|
+
);
|
|
656
|
+
}
|
|
657
|
+
if (status === 304) {
|
|
658
|
+
return new StorageException(
|
|
659
|
+
STORAGE_ERROR_CODES.STORAGE_NOT_MODIFIED,
|
|
660
|
+
HttpStatus.NOT_MODIFIED,
|
|
661
|
+
details
|
|
662
|
+
);
|
|
663
|
+
}
|
|
664
|
+
if (status === 412) {
|
|
665
|
+
return new StorageException(
|
|
666
|
+
STORAGE_ERROR_CODES.STORAGE_PRECONDITION_FAILED,
|
|
667
|
+
HttpStatus.PRECONDITION_FAILED,
|
|
668
|
+
details
|
|
669
|
+
);
|
|
670
|
+
}
|
|
671
|
+
return void 0;
|
|
672
|
+
}
|
|
642
673
|
function mapAwsError(err, context) {
|
|
643
674
|
const e = err ?? {};
|
|
644
675
|
const httpStatus = e.$metadata?.httpStatusCode;
|
|
@@ -659,27 +690,8 @@ function mapAwsError(err, context) {
|
|
|
659
690
|
details
|
|
660
691
|
);
|
|
661
692
|
}
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
STORAGE_ERROR_CODES.STORAGE_OBJECT_NOT_FOUND,
|
|
665
|
-
HttpStatus.NOT_FOUND,
|
|
666
|
-
details
|
|
667
|
-
);
|
|
668
|
-
}
|
|
669
|
-
if (httpStatus === 304) {
|
|
670
|
-
return new StorageException(
|
|
671
|
-
STORAGE_ERROR_CODES.STORAGE_NOT_MODIFIED,
|
|
672
|
-
HttpStatus.NOT_MODIFIED,
|
|
673
|
-
details
|
|
674
|
-
);
|
|
675
|
-
}
|
|
676
|
-
if (httpStatus === 412) {
|
|
677
|
-
return new StorageException(
|
|
678
|
-
STORAGE_ERROR_CODES.STORAGE_PRECONDITION_FAILED,
|
|
679
|
-
HttpStatus.PRECONDITION_FAILED,
|
|
680
|
-
details
|
|
681
|
-
);
|
|
682
|
-
}
|
|
693
|
+
const answered = mapAnsweredStatus(e, details);
|
|
694
|
+
if (answered) return answered;
|
|
683
695
|
if (e.name === "TimeoutError") {
|
|
684
696
|
logProviderFailure(err, context);
|
|
685
697
|
return new StorageException(STORAGE_ERROR_CODES.STORAGE_TIMEOUT, void 0, details);
|
|
@@ -1214,6 +1226,21 @@ ValidationService = __decorateClass([
|
|
|
1214
1226
|
__decorateParam(0, Inject(BYMAX_STORAGE_OPTIONS)),
|
|
1215
1227
|
__decorateParam(1, Inject(BYMAX_STORAGE_UPLOAD_VALIDATORS))
|
|
1216
1228
|
], ValidationService);
|
|
1229
|
+
var UNSERIALISABLE = "[unserialisable]";
|
|
1230
|
+
var MAX_SCANNER_DETAILS_LENGTH = 300;
|
|
1231
|
+
function toJson(value) {
|
|
1232
|
+
return JSON.stringify(value);
|
|
1233
|
+
}
|
|
1234
|
+
function scannerDetailsForLog(details) {
|
|
1235
|
+
if (details === void 0) return "";
|
|
1236
|
+
let text;
|
|
1237
|
+
try {
|
|
1238
|
+
text = toJson(details) ?? UNSERIALISABLE;
|
|
1239
|
+
} catch {
|
|
1240
|
+
text = UNSERIALISABLE;
|
|
1241
|
+
}
|
|
1242
|
+
return ` details=${logValue(text.slice(0, MAX_SCANNER_DETAILS_LENGTH))}`;
|
|
1243
|
+
}
|
|
1217
1244
|
var FileScannerService = class {
|
|
1218
1245
|
constructor(options, scanner) {
|
|
1219
1246
|
this.options = options;
|
|
@@ -1265,18 +1292,17 @@ var FileScannerService = class {
|
|
|
1265
1292
|
}
|
|
1266
1293
|
const result = await this.scanner.scan(input);
|
|
1267
1294
|
if (result.status === "infected") {
|
|
1268
|
-
this.logger.warn(`Infected file detected: key=${logValue(input.key)} engine=${logValue(result.engine)} threat=${logValue(result.threat ?? "unknown")}`);
|
|
1295
|
+
this.logger.warn(`Infected file detected: key=${logValue(input.key)} engine=${logValue(result.engine)} threat=${logValue(result.threat ?? "unknown")}${scannerDetailsForLog(result.details)}`);
|
|
1269
1296
|
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_SCAN_INFECTED, void 0, {
|
|
1270
1297
|
engine: result.engine,
|
|
1271
|
-
threat: result.threat
|
|
1272
|
-
details: result.details
|
|
1298
|
+
threat: result.threat
|
|
1273
1299
|
});
|
|
1274
1300
|
}
|
|
1275
1301
|
if (result.status === "unknown") {
|
|
1276
1302
|
if (this.options.scanner?.rejectOnUnknown) {
|
|
1303
|
+
this.logger.warn(`Inconclusive scan result rejected: key=${logValue(input.key)} engine=${logValue(result.engine)}${scannerDetailsForLog(result.details)}`);
|
|
1277
1304
|
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_SCAN_INCONCLUSIVE, void 0, {
|
|
1278
|
-
engine: result.engine
|
|
1279
|
-
details: result.details
|
|
1305
|
+
engine: result.engine
|
|
1280
1306
|
});
|
|
1281
1307
|
}
|
|
1282
1308
|
this.logger.warn(`Inconclusive scan result accepted: key=${logValue(input.key)} engine=${logValue(result.engine)}`);
|
|
@@ -1297,6 +1323,10 @@ var SourceIdleError = class extends Error {
|
|
|
1297
1323
|
}
|
|
1298
1324
|
idleMs;
|
|
1299
1325
|
};
|
|
1326
|
+
function isError(value) {
|
|
1327
|
+
const candidate = value;
|
|
1328
|
+
return typeof candidate?.name === "string" && typeof candidate.message === "string";
|
|
1329
|
+
}
|
|
1300
1330
|
function withinIdleTime(next, idleMs) {
|
|
1301
1331
|
return new Promise((resolve, reject) => {
|
|
1302
1332
|
const timer = setTimeout(() => {
|
|
@@ -1309,7 +1339,7 @@ function withinIdleTime(next, idleMs) {
|
|
|
1309
1339
|
},
|
|
1310
1340
|
(error) => {
|
|
1311
1341
|
clearTimeout(timer);
|
|
1312
|
-
reject(error
|
|
1342
|
+
reject(isError(error) ? error : new Error(describeError(error)));
|
|
1313
1343
|
}
|
|
1314
1344
|
);
|
|
1315
1345
|
});
|
|
@@ -1699,6 +1729,24 @@ function boundStreamSize(body, declaredSize, options) {
|
|
|
1699
1729
|
}
|
|
1700
1730
|
return withSizeLimit(body, limit, declaredSize === void 0 ? "max" : "declared");
|
|
1701
1731
|
}
|
|
1732
|
+
async function bufferStreamForScan(body, finalKey, maxSizeBytes) {
|
|
1733
|
+
if (maxSizeBytes === void 0) {
|
|
1734
|
+
destroyStream(body);
|
|
1735
|
+
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1736
|
+
key: finalKey,
|
|
1737
|
+
reason: "Pre-upload scanning of a stream needs validation.maxSizeBytes to bound the buffer; set it or use scanner mode post-upload"
|
|
1738
|
+
});
|
|
1739
|
+
}
|
|
1740
|
+
const buffered = await readStreamUpTo(body, maxSizeBytes);
|
|
1741
|
+
if (buffered === void 0) {
|
|
1742
|
+
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1743
|
+
key: finalKey,
|
|
1744
|
+
maxSizeBytes,
|
|
1745
|
+
reason: "The stream is longer than validation.maxSizeBytes, so it cannot be scanned before upload"
|
|
1746
|
+
});
|
|
1747
|
+
}
|
|
1748
|
+
return buffered;
|
|
1749
|
+
}
|
|
1702
1750
|
|
|
1703
1751
|
// src/server/utils/download-body.ts
|
|
1704
1752
|
function guardDownloadBody(body, idleMs, context, signal) {
|
|
@@ -1713,9 +1761,15 @@ function guardDownloadBody(body, idleMs, context, signal) {
|
|
|
1713
1761
|
idleMs: error.idleMs
|
|
1714
1762
|
});
|
|
1715
1763
|
}
|
|
1716
|
-
return signal?.aborted ? requestAborted(signal, context) : error;
|
|
1764
|
+
return signal?.aborted ? requestAborted(signal, context) : mapAwsError(error, context);
|
|
1717
1765
|
});
|
|
1718
1766
|
}
|
|
1767
|
+
function mapBodyReadFailure(err, signal, context) {
|
|
1768
|
+
if (err instanceof StorageException) {
|
|
1769
|
+
return err;
|
|
1770
|
+
}
|
|
1771
|
+
return signal?.aborted ? requestAborted(signal, context) : mapAwsError(err, context);
|
|
1772
|
+
}
|
|
1719
1773
|
|
|
1720
1774
|
// src/server/utils/public-url.ts
|
|
1721
1775
|
function buildPublicUrl(options, finalKey, bucket) {
|
|
@@ -1805,7 +1859,7 @@ function assertMaxBytes(maxBytes) {
|
|
|
1805
1859
|
});
|
|
1806
1860
|
}
|
|
1807
1861
|
}
|
|
1808
|
-
async function
|
|
1862
|
+
async function readBody(stream, metadata, maxBytes) {
|
|
1809
1863
|
if (maxBytes === void 0) {
|
|
1810
1864
|
return readStream(stream);
|
|
1811
1865
|
}
|
|
@@ -1820,6 +1874,17 @@ async function readDownloadBody(stream, metadata, maxBytes) {
|
|
|
1820
1874
|
}
|
|
1821
1875
|
return buffer;
|
|
1822
1876
|
}
|
|
1877
|
+
async function readDownloadBody(stream, metadata, maxBytes, signal) {
|
|
1878
|
+
try {
|
|
1879
|
+
return await readBody(stream, metadata, maxBytes);
|
|
1880
|
+
} catch (err) {
|
|
1881
|
+
throw mapBodyReadFailure(err, signal, {
|
|
1882
|
+
key: metadata.key,
|
|
1883
|
+
bucket: metadata.bucket,
|
|
1884
|
+
op: "downloadBuffer"
|
|
1885
|
+
});
|
|
1886
|
+
}
|
|
1887
|
+
}
|
|
1823
1888
|
|
|
1824
1889
|
// src/server/services/storage.service.ts
|
|
1825
1890
|
var MAX_LIST_KEYS = 1e3;
|
|
@@ -1876,7 +1941,8 @@ var StorageService = class {
|
|
|
1876
1941
|
}
|
|
1877
1942
|
const finalKey = this.keyResolver.normalize(options.key);
|
|
1878
1943
|
const bucket = this.resolveBucket(options.bucket);
|
|
1879
|
-
|
|
1944
|
+
const context = { key: finalKey, bucket, op: "upload" };
|
|
1945
|
+
throwIfAborted(options.abortSignal, context);
|
|
1880
1946
|
if (options.idempotencyKey) {
|
|
1881
1947
|
const cacheKey = this.idempotencyCache.computeKey(options.idempotencyKey, finalKey);
|
|
1882
1948
|
const cached = this.idempotencyCache.get(cacheKey);
|
|
@@ -1886,7 +1952,25 @@ var StorageService = class {
|
|
|
1886
1952
|
return { ...cached, fromIdempotencyCache: true };
|
|
1887
1953
|
}
|
|
1888
1954
|
}
|
|
1889
|
-
const
|
|
1955
|
+
const validatedOptions = await this.validateAndScan(options, finalKey, bucket, context);
|
|
1956
|
+
throwIfAborted(options.abortSignal, context);
|
|
1957
|
+
const strategy = pickUploadStrategy(
|
|
1958
|
+
validatedOptions.body,
|
|
1959
|
+
validatedOptions.size,
|
|
1960
|
+
this.options.multipart.thresholdBytes
|
|
1961
|
+
);
|
|
1962
|
+
const result = strategy === "multipart" ? await this.uploadMultipart(validatedOptions, finalKey, bucket) : await this.uploadSingleShot(validatedOptions, finalKey, bucket);
|
|
1963
|
+
const stored = { ...context, stored: true };
|
|
1964
|
+
await abortable(this.runPostUploadScan(finalKey, bucket, options), options.abortSignal, stored);
|
|
1965
|
+
this.rememberUpload(options, finalKey, result);
|
|
1966
|
+
return result;
|
|
1967
|
+
}
|
|
1968
|
+
/**
|
|
1969
|
+
* Validates the upload and runs the pre-upload scan, returning the options with the body to send.
|
|
1970
|
+
* Reading the body (a validator peeking at it, the scan buffering it) happens before any request, so
|
|
1971
|
+
* the signal is watched here: a silent stream must not hold the call.
|
|
1972
|
+
*/
|
|
1973
|
+
async validateAndScan(options, finalKey, bucket, context) {
|
|
1890
1974
|
const signal = options.abortSignal;
|
|
1891
1975
|
const input = validationInput(options, finalKey);
|
|
1892
1976
|
const validated = await abortable(this.validation.validate(input), signal, context);
|
|
@@ -1898,17 +1982,7 @@ var StorageService = class {
|
|
|
1898
1982
|
const body = boundStreamSize(scanned, options.size, this.options);
|
|
1899
1983
|
const validatedOptions = { ...options, body };
|
|
1900
1984
|
await abortable(this.runPreUploadScan(body, finalKey, bucket, options), signal, context);
|
|
1901
|
-
|
|
1902
|
-
const strategy = pickUploadStrategy(
|
|
1903
|
-
validatedOptions.body,
|
|
1904
|
-
validatedOptions.size,
|
|
1905
|
-
this.options.multipart.thresholdBytes
|
|
1906
|
-
);
|
|
1907
|
-
const result = strategy === "multipart" ? await this.uploadMultipart(validatedOptions, finalKey, bucket) : await this.uploadSingleShot(validatedOptions, finalKey, bucket);
|
|
1908
|
-
const stored = { ...context, stored: true };
|
|
1909
|
-
await abortable(this.runPostUploadScan(finalKey, bucket, options), signal, stored);
|
|
1910
|
-
this.rememberUpload(options, finalKey, result);
|
|
1911
|
-
return result;
|
|
1985
|
+
return validatedOptions;
|
|
1912
1986
|
}
|
|
1913
1987
|
/**
|
|
1914
1988
|
* In pre-upload scan mode a stream body is read into one `Buffer`, so the scanner
|
|
@@ -1921,23 +1995,7 @@ var StorageService = class {
|
|
|
1921
1995
|
if (!this.scanner.isEnabled() || this.scanner.getMode() !== "pre-upload" || isBufferLike(body)) {
|
|
1922
1996
|
return body;
|
|
1923
1997
|
}
|
|
1924
|
-
|
|
1925
|
-
if (maxSizeBytes === void 0) {
|
|
1926
|
-
destroyStream(body);
|
|
1927
|
-
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1928
|
-
key: finalKey,
|
|
1929
|
-
reason: "Pre-upload scanning of a stream needs validation.maxSizeBytes to bound the buffer; set it or use scanner mode post-upload"
|
|
1930
|
-
});
|
|
1931
|
-
}
|
|
1932
|
-
const buffered = await readStreamUpTo(body, maxSizeBytes);
|
|
1933
|
-
if (buffered === void 0) {
|
|
1934
|
-
throw new StorageException(STORAGE_ERROR_CODES.STORAGE_VALIDATION_FAILED, void 0, {
|
|
1935
|
-
key: finalKey,
|
|
1936
|
-
maxSizeBytes,
|
|
1937
|
-
reason: "The stream is longer than validation.maxSizeBytes, so it cannot be scanned before upload"
|
|
1938
|
-
});
|
|
1939
|
-
}
|
|
1940
|
-
return buffered;
|
|
1998
|
+
return bufferStreamForScan(body, finalKey, this.options.validation?.maxSizeBytes);
|
|
1941
1999
|
}
|
|
1942
2000
|
/** Runs a pre-upload scan when configured; throws on infected. */
|
|
1943
2001
|
async runPreUploadScan(body, finalKey, bucket, options) {
|
|
@@ -2093,7 +2151,7 @@ var StorageService = class {
|
|
|
2093
2151
|
const stream = guardDownloadBody(
|
|
2094
2152
|
response.Body,
|
|
2095
2153
|
this.options.bodyIdleTimeoutMs,
|
|
2096
|
-
|
|
2154
|
+
context,
|
|
2097
2155
|
options.abortSignal
|
|
2098
2156
|
);
|
|
2099
2157
|
return { stream, metadata: this.toObjectMetadata(response, finalKey, bucket) };
|
|
@@ -2116,7 +2174,8 @@ var StorageService = class {
|
|
|
2116
2174
|
async downloadBuffer(options) {
|
|
2117
2175
|
assertMaxBytes(options.maxBytes);
|
|
2118
2176
|
const { stream, metadata } = await this.download(options);
|
|
2119
|
-
|
|
2177
|
+
const buffer = await readDownloadBody(stream, metadata, options.maxBytes, options.abortSignal);
|
|
2178
|
+
return { buffer, metadata };
|
|
2120
2179
|
}
|
|
2121
2180
|
/**
|
|
2122
2181
|
* Deletes an object. Idempotent: a missing object is a no-op (logged as a
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bymax-one/nest-storage",
|
|
3
|
-
"version": "1.1.
|
|
3
|
+
"version": "1.1.16",
|
|
4
4
|
"description": "Provider-agnostic S3-compatible object storage for NestJS. Works with AWS S3, DigitalOcean Spaces, Cloudflare R2, Backblaze B2, MinIO, Wasabi.",
|
|
5
5
|
"author": "Bymax One <support@bymax.one>",
|
|
6
6
|
"license": "MIT",
|
|
@@ -49,6 +49,8 @@
|
|
|
49
49
|
"check:exports": "attw --pack . --profile strict",
|
|
50
50
|
"check:mutants": "node scripts/check-mutation-directives.mjs",
|
|
51
51
|
"check:file-length": "node scripts/check-file-length.mjs",
|
|
52
|
+
"check:spec-headers": "node scripts/check-spec-headers.mjs",
|
|
53
|
+
"check:function-length": "node scripts/check-function-length.mjs",
|
|
52
54
|
"check:published": "node scripts/check-published-surface.mjs",
|
|
53
55
|
"check:runtime": "node scripts/check-consumer-runtime.mjs",
|
|
54
56
|
"clean": "rm -rf dist coverage",
|