@bymax-one/nest-storage 1.1.16 → 1.1.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +201 -1
- package/README.md +30 -13
- package/dist/server/index.cjs +443 -59
- package/dist/server/index.d.cts +39 -13
- package/dist/server/index.d.ts +39 -13
- package/dist/server/index.mjs +443 -59
- package/package.json +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -13,6 +13,204 @@ note on each entry is the compatibility contract — read it, not the number.
|
|
|
13
13
|
|
|
14
14
|
## [Unreleased]
|
|
15
15
|
|
|
16
|
+
## [1.1.18] - 2026-11-18
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
|
|
20
|
+
- **A per-call value of the wrong kind no longer reaches the provider call or a log line.** A per-call `bucket` that
|
|
21
|
+
is not a string (a `Symbol`, a function, an array, an object, a number) was repeated in `details.bucket` of the
|
|
22
|
+
error it caused, on every method that takes a bucket, and an `abortSignal` that was not an `AbortSignal` (a string,
|
|
23
|
+
a `Symbol`, a `BigInt`) was handed to the SDK, which named the value in its own error message, and that message is
|
|
24
|
+
logged as `Provider call failed`. Both are now refused before any request with `STORAGE_VALIDATION_FAILED` (400)
|
|
25
|
+
naming `bucket` or `abortSignal`, and nothing of the value. `getMultipartUploadUrls` no longer puts a `parts` that
|
|
26
|
+
is not a number in `details.provided`. A body that has `pipe`, `on` and `once` but no `pause` or `resume` is refused
|
|
27
|
+
with `STORAGE_VALIDATION_FAILED` naming `body`, where it used to be waited on until `STORAGE_TIMEOUT`.
|
|
28
|
+
|
|
29
|
+
**Apply to a derived backend:** a call that passed a per-call `bucket` that is not a string, or an `abortSignal` that
|
|
30
|
+
is not an `AbortSignal`, now fails with `STORAGE_VALIDATION_FAILED` (`details.field` is `bucket` or `abortSignal`)
|
|
31
|
+
where it used to reach the provider (`exists` answers `false` and logs a warning, as for any failure that is not a
|
|
32
|
+
missing object). `details.provided` of `STORAGE_INVALID_PART_COUNT` is present only when `parts` is a finite number. A
|
|
33
|
+
stream-like body without `pause` and `resume` is refused with `STORAGE_VALIDATION_FAILED`.
|
|
34
|
+
|
|
35
|
+
- **A configuration of the wrong kind is refused at boot, and a `null` that means "off" is refused.** `keyPrefix` that is
|
|
36
|
+
an array or an object was accepted and failed later as a raw `TypeError`; `validation.mimeWhitelist` that is a string
|
|
37
|
+
was accepted; `validation.customValidators` that is not an array was accepted; and a `region` with a space, a slash
|
|
38
|
+
or a newline in it was sent on to the SDK, which repeats it in its error messages. `forRoot` now refuses each with
|
|
39
|
+
`STORAGE_INVALID_CONFIG` naming the option and repeating nothing of the value: `keyPrefix` must be a string,
|
|
40
|
+
`mimeWhitelist` an array of strings, `customValidators` an array, and `region` 1 to 63 letters, digits, `.`, `_` or
|
|
41
|
+
`-` starting with a letter or digit. `validation.maxSizeBytes: null`, `validation.mimeWhitelist: null` and
|
|
42
|
+
`validation.customValidators: null` are refused as well: leaving each out means no size limit, every type allowed or
|
|
43
|
+
no custom validator, so a `null` from a config lookup of a missing key must not switch the protection off (1.1.17
|
|
44
|
+
read `maxSizeBytes: null` as "no limit").
|
|
45
|
+
|
|
46
|
+
**Apply to a derived backend:** a configuration that booted and is now refused: a `keyPrefix` that is not a string, a
|
|
47
|
+
`validation.mimeWhitelist` or `customValidators` that is not an array (and a `mimeWhitelist` with a non-string
|
|
48
|
+
entry), a `region` outside the shape above, and a `null` for `validation.maxSizeBytes`, `validation.mimeWhitelist` or
|
|
49
|
+
`validation.customValidators`. Omit the option for "none": `null` no longer means it. A `null` for any other option
|
|
50
|
+
inside a group is still the default.
|
|
51
|
+
|
|
52
|
+
**`scanner: null` is refused; omit the key for no scanner.** 1.1.17 read a `scanner` of `null` as "no scanner", so a
|
|
53
|
+
config lookup that returned `null` for a missing key booted with scanning off. `forRoot` now fails with
|
|
54
|
+
`STORAGE_INVALID_CONFIG` ("scanner must not be null; omit it for no scanner"); a `scanner` that is `false`, `0`,
|
|
55
|
+
`''` or `[]` was already refused.
|
|
56
|
+
|
|
57
|
+
**Apply to a derived backend:** a `scanner` of `null` now fails `forRoot` and `forRootAsync` with
|
|
58
|
+
`STORAGE_INVALID_CONFIG`; omit the key to run no scanner.
|
|
59
|
+
|
|
60
|
+
- **A value that throws when it is read no longer hangs a source read.** The check that tells an `Error` from any other
|
|
61
|
+
value in the idle-timeout wrapper read its `name` and `message` outside a guard: a rejection value whose property
|
|
62
|
+
getter throws left the wrapped read pending for good, with its timer already cleared. It is now treated as not
|
|
63
|
+
being an error, and the read ends with one.
|
|
64
|
+
|
|
65
|
+
**Apply to a derived backend:** nothing to change; a read that used to stay pending in that case now ends with an error.
|
|
66
|
+
|
|
67
|
+
- **Every option is read once.** Validation and resolution each read the options by property name, so a getter that
|
|
68
|
+
returned a valid value for the check and a hostile one for the use (`scanner.mode` that answers `'pre-upload'`
|
|
69
|
+
first and `'preupload'` after, a size limit that answers a number and then `NaN`) passed the check and was used as
|
|
70
|
+
the hostile value. `forRoot` and `forRootAsync` now copy each known option, and each key of the option groups, into a
|
|
71
|
+
plain object once; the copy is what is validated and what is resolved. The arrays of `validation` are copied once
|
|
72
|
+
as well.
|
|
73
|
+
|
|
74
|
+
**Apply to a derived backend:** nothing to change for a plain options object. An options object whose property
|
|
75
|
+
getters answer differently from one read to the next now gets the first answer everywhere; a key that is not an
|
|
76
|
+
option is not read at all.
|
|
77
|
+
|
|
78
|
+
- **A cut of scanner text never leaves half of a surrogate pair, and a scan that times out is logged.** The engine and
|
|
79
|
+
threat names of a scan result (in the response body and the log), the unrecognised status, the scanner's `details`
|
|
80
|
+
and the message of a scanner that throws were cut at a fixed length, and a cut between the two halves of an
|
|
81
|
+
emoji left a lone surrogate in a JSON body or a log line. The cut now drops the whole pair. A scan that outlasts
|
|
82
|
+
`scanner.timeoutMs` logs one `Scan timed out` warning with the key and the limit; until now only the typed
|
|
83
|
+
`STORAGE_TIMEOUT` carried it.
|
|
84
|
+
|
|
85
|
+
**Apply to a derived backend:** an engine or threat name cut at its limit is one character shorter when an
|
|
86
|
+
emoji sits at the cut; alert rules that matched the absence of a timeout line can match `Scan timed out`.
|
|
87
|
+
|
|
88
|
+
### Changed
|
|
89
|
+
|
|
90
|
+
- **Behaviour of 1.1.17 that its notes did not name.** Each is a change from 1.1.16; the `Apply` line says what to do.
|
|
91
|
+
- **A WHATWG `ReadableStream` body is refused (breaking).** 1.1.16 accepted it and stored it; 1.1.17 and later fail
|
|
92
|
+
with `STORAGE_VALIDATION_FAILED` (400, `details.field: 'body'`). **Apply:** convert it with
|
|
93
|
+
`Readable.fromWeb(stream)` before the call.
|
|
94
|
+
- **A `scanner` that is not an object is refused at boot.** `false`, `0`, `''` and `[]` booted in 1.1.16 and scanned
|
|
95
|
+
nothing; they fail `forRoot` with `STORAGE_INVALID_CONFIG` (`null` is refused too, see above). **Apply:**
|
|
96
|
+
omit `scanner` for no scanner.
|
|
97
|
+
- **`scanner.impl` must be an object with a `scan` function.** A function with a static `scan` (a class passed
|
|
98
|
+
where an instance belongs) is refused; 1.1.16 accepted it and called the static method. **Apply:** pass an
|
|
99
|
+
instance, or an object that holds the `scan` function.
|
|
100
|
+
- **Keys that are not options are dropped from the resolved `signedUrls`, `multipart`, `validation` and `scanner`.**
|
|
101
|
+
1.1.16 copied them through. **Apply:** keep such values outside the options.
|
|
102
|
+
- **`validation: null` resolves to an empty `validation`**; 1.1.16 kept the `null`. **Apply:** none; read
|
|
103
|
+
`resolved.validation` as an object, never as `null`.
|
|
104
|
+
- **A group that is a class instance is honoured.** A value served from a prototype getter (a config object built
|
|
105
|
+
as a class) is kept; 1.1.16 copied own properties only and dropped it silently.
|
|
106
|
+
- **`validation.maxSizeBytes: null` meant "no limit" in 1.1.17**; it is refused from this release (see above).
|
|
107
|
+
|
|
108
|
+
## [1.1.17] - 2026-11-17
|
|
109
|
+
|
|
110
|
+
### Fixed
|
|
111
|
+
|
|
112
|
+
- **A size limit or a multipart setting of `NaN` or `Infinity` no longer boots.** `validation.maxSizeBytes`,
|
|
113
|
+
`signedUrls.defaultGetTtlSeconds`, `defaultPutTtlSeconds`, `multipart.thresholdBytes` and `multipart.queueSize`
|
|
114
|
+
had no check, and `signedUrls.maxTtlSeconds` and `multipart.partSizeBytes` checked only one side of their range,
|
|
115
|
+
which `NaN` passes. `NaN` for `maxSizeBytes` switched the size limit off: a buffer and a stream declared at
|
|
116
|
+
5 GB were both sent. `NaN` for a TTL put `X-Amz-Expires=NaN` in every signed URL. `Infinity` for `queueSize`
|
|
117
|
+
kept the process busy from the first multipart upload on. A threshold of `NaN` or `Infinity` never chose a
|
|
118
|
+
multipart upload and a part size of `Infinity` buffered the whole stream as one part. Each option is now a
|
|
119
|
+
whole number in a stated range, and `forRoot` fails with `STORAGE_INVALID_CONFIG` naming the option. Per
|
|
120
|
+
call, a `ttlSeconds` that is not a whole number above 0 fails with `STORAGE_SIGNED_URL_TTL_INVALID`, and a
|
|
121
|
+
declared upload `size` or a list `maxKeys` that is not a whole number fails with `STORAGE_VALIDATION_FAILED`,
|
|
122
|
+
before any request, whatever the body is (a buffer, a view, a string or a stream). A key set to `undefined` or
|
|
123
|
+
`null` inside `signedUrls`, `multipart`, `validation` or `scanner` means the default: `maxTtlSeconds: undefined`
|
|
124
|
+
used to resolve to `NaN` and put `X-Amz-Expires=NaN` in every signed URL. `maxAttempts` is refused above the
|
|
125
|
+
safe-integer ceiling. The error for a TTL names no value except a finite number: a string, a `Symbol` or a
|
|
126
|
+
function the caller passed is not echoed.
|
|
127
|
+
|
|
128
|
+
**Apply to a derived backend:** a configuration that booted before and is now refused at `forRoot`:
|
|
129
|
+
`validation.maxSizeBytes`, a TTL or `multipart.thresholdBytes` that is `0`, negative, `NaN`, `Infinity`,
|
|
130
|
+
fractional or a string; `signedUrls.maxTtlSeconds: Infinity` (it was lowered to 7 days with a warning; a finite
|
|
131
|
+
value above that still is); `multipart.partSizeBytes` that is a string, `NaN`, `Infinity` or above 5 GiB;
|
|
132
|
+
`multipart.queueSize` that is not a whole number from 1 to 1000. Per call, `ttlSeconds: Infinity` (it was
|
|
133
|
+
clamped to the maximum), a fractional or string TTL, a fractional or negative `size`, and a `maxKeys` of
|
|
134
|
+
`Infinity`, `0`, a fraction or a string are now refused with the codes above. A per-call `ttlSeconds`, `size` or
|
|
135
|
+
`maxKeys` of `null` is refused too (a `ttlSeconds` of `null` no longer reads as absent), while `undefined` or
|
|
136
|
+
`null` on an option inside a group is the default; `maxAttempts` above 9007199254740991 is refused. A
|
|
137
|
+
`getResponse()` of a TTL refusal no longer carries a non-numeric `details.provided`.
|
|
138
|
+
|
|
139
|
+
- **A scanner that would leave scanning silently off is refused at boot.** `scanner: { impl: undefined }` (an optional
|
|
140
|
+
provider or a factory that returned nothing) and a `scanner.mode` that is a typo (`'preupload'`,
|
|
141
|
+
`'pre_upload'`) made the library skip both the pre-upload and the post-upload scan, so an infected upload was
|
|
142
|
+
stored (measured with an infected stand-in: 9 bytes stored under each). `forRoot` now fails with
|
|
143
|
+
`STORAGE_INVALID_CONFIG` unless `scanner.impl` is an object with a `scan` function and `scanner.mode` is
|
|
144
|
+
`'pre-upload'` or `'post-upload'`; `scanner.rejectOnUnknown` must be a boolean (unset or `null` is the default), and is no longer read as one
|
|
145
|
+
(`'false'` rejected unknown results). A `scanner` of `null` is no scanner, as `undefined` is, and an
|
|
146
|
+
`undefined` or `null` mode is the default `'pre-upload'`.
|
|
147
|
+
|
|
148
|
+
**Apply to a derived backend:** a host whose `scanner.impl` could be unset, whose `mode` was misspelt, or
|
|
149
|
+
whose `rejectOnUnknown` was a string now fails at boot instead of running unscanned; fix the configuration.
|
|
150
|
+
|
|
151
|
+
- **A scan result outside the contract no longer uploads.** `scan()` returned any result that was not
|
|
152
|
+
`'infected'` or an `'unknown'` it was told to reject, so no result, `null`, `'CLEAN'`, `' clean'` or an object
|
|
153
|
+
as the status stored the object. Only the exact string `'clean'` passes now; `'unknown'` keeps following
|
|
154
|
+
`rejectOnUnknown`, and anything else is rejected as `STORAGE_SCAN_INCONCLUSIVE` with
|
|
155
|
+
`details.reason: 'unrecognised scanner status'`, with or without `rejectOnUnknown`, and logged: a string status is
|
|
156
|
+
cut to 40 characters in the log line and any other status is named by its type, so none of the scanner's own
|
|
157
|
+
code runs to print it.
|
|
158
|
+
|
|
159
|
+
**Apply to a derived backend:** a scanner that returned a status other than `'clean'`, `'infected'` or
|
|
160
|
+
`'unknown'` (or nothing) on a clean file was uploading; fix the scanner, it is rejected now.
|
|
161
|
+
|
|
162
|
+
- **What a pre-upload scan approves is what is stored.** A caller that wrote into its own `Buffer` or
|
|
163
|
+
`Uint8Array` while the scan ran had the changed bytes stored (measured: `ZAAA` stored after the scanner
|
|
164
|
+
approved `AAAA`), because the scanner and the upload read the same memory at different times. In pre-upload
|
|
165
|
+
mode a buffer body is now copied once, and the scanner and the upload both use the copy.
|
|
166
|
+
|
|
167
|
+
**Apply to a derived backend:** a pre-upload scan holds a second copy of a buffer body while the upload
|
|
168
|
+
runs; the extra memory is the size of the body.
|
|
169
|
+
|
|
170
|
+
- **A stream whose declared `size` is wrong no longer leaves a truncated object.** A stream below the multipart
|
|
171
|
+
threshold goes as one request with `Content-Length` set to `size`; a provider stores the object once it has
|
|
172
|
+
that many bytes, so a longer body (measured on MinIO: 500 declared, 5000 sent) stored the first 500 bytes as
|
|
173
|
+
if it were the whole. The last byte is now held back until the source has ended: a longer body fails
|
|
174
|
+
`STORAGE_SIZE_EXCEEDED` before the request completes and a shorter one fails `STORAGE_VALIDATION_FAILED`
|
|
175
|
+
(`details.field: 'size'`), each in bounded time with nothing stored. A declared `size` that is not a whole
|
|
176
|
+
number of at least 0 also fails `STORAGE_VALIDATION_FAILED` (see the entry above). With a pre-upload scanner
|
|
177
|
+
the stream is read into memory first, and the same rule is applied to the length that was read.
|
|
178
|
+
|
|
179
|
+
**Apply to a derived backend:** a caller that passed a `size` that did not match its stream, and got an object
|
|
180
|
+
of the declared length back, now gets an error; pass the true size, or none (a stream with no size uses
|
|
181
|
+
multipart). The multipart path is unchanged for a shorter stream. A single-shot stream that delivers its
|
|
182
|
+
declared bytes and then never ends now fails `STORAGE_TIMEOUT` after `requestTimeoutMs`, because the last byte
|
|
183
|
+
is held until the source ends; it completed before. End the stream.
|
|
184
|
+
|
|
185
|
+
- **A body that is not a body now fails as a validation error.** A `Blob`, an array of chunks or any other object
|
|
186
|
+
that is not a `Buffer`, `Uint8Array`, `ArrayBuffer`, string or `Readable` reached the upload as if it were a
|
|
187
|
+
stream and failed as `STORAGE_MULTIPART_ABORTED` (500), which blames the multipart upload. It now fails
|
|
188
|
+
`STORAGE_VALIDATION_FAILED` (400, `details.field: 'body'`) before any request. A WHATWG `ReadableStream` is
|
|
189
|
+
refused the same way, and so is an object that has a `pipe` function but is not a Node readable (it was a raw
|
|
190
|
+
`TypeError`).
|
|
191
|
+
|
|
192
|
+
**Apply to a derived backend:** a handler that mapped that 500 to a client error can drop the mapping; pass
|
|
193
|
+
a supported body.
|
|
194
|
+
|
|
195
|
+
- **The scanner's engine and threat names are capped, and a scanner that throws is logged.** `details.engine`
|
|
196
|
+
and `details.threat` in the body, and the same names in the log line, are cut to 200 characters and left out
|
|
197
|
+
when the scanner gave something that is not a string. A scanner that threw was answered 500 with no log line
|
|
198
|
+
at all; it now logs `Scanner failed` with its message, cut at 300 characters. The error the caller gets is
|
|
199
|
+
unchanged.
|
|
200
|
+
|
|
201
|
+
**Apply to a derived backend:** a scanner that names a threat longer than 200 characters is cut in the
|
|
202
|
+
response; keep the names short, or read the full text from your scanner's own log.
|
|
203
|
+
|
|
204
|
+
### Added
|
|
205
|
+
|
|
206
|
+
- **`scanner.timeoutMs` bounds a file scan.** A scanner that never settled held the upload until the caller
|
|
207
|
+
aborted. A scan now ends with `STORAGE_TIMEOUT` (503, `details.phase: 'scan'`) after `scanner.timeoutMs`,
|
|
208
|
+
by default 60 s, and the scanner is handed an `AbortSignal` (`signal` on its input) that aborts then. There is
|
|
209
|
+
no setting that waits without limit. A scanner that ignores the signal keeps running after the timeout.
|
|
210
|
+
|
|
211
|
+
**Apply to a derived backend:** a scan that took longer than 60 s used to complete and now fails; set
|
|
212
|
+
`scanner.timeoutMs` to what your scanner needs, and have the scanner honour `signal`.
|
|
213
|
+
|
|
16
214
|
## [1.1.16] - 2026-11-17
|
|
17
215
|
|
|
18
216
|
### Changed
|
|
@@ -908,4 +1106,6 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
908
1106
|
[1.1.14]: https://github.com/bymaxone/nest-storage/compare/v1.1.13...v1.1.14
|
|
909
1107
|
[1.1.15]: https://github.com/bymaxone/nest-storage/compare/v1.1.14...v1.1.15
|
|
910
1108
|
[1.1.16]: https://github.com/bymaxone/nest-storage/compare/v1.1.15...v1.1.16
|
|
911
|
-
[
|
|
1109
|
+
[1.1.17]: https://github.com/bymaxone/nest-storage/compare/v1.1.16...v1.1.17
|
|
1110
|
+
[1.1.18]: https://github.com/bymaxone/nest-storage/compare/v1.1.17...v1.1.18
|
|
1111
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.18...HEAD
|
package/README.md
CHANGED
|
@@ -209,13 +209,13 @@ Options accepted by `forRoot` and `forRootAsync`:
|
|
|
209
209
|
| Option | Type | Default | Notes |
|
|
210
210
|
| ------------------------------ | ------------------------------------------------- | ----------------- | ---------------------------------------------- |
|
|
211
211
|
| `endpoint` | `string` | — | Required. S3-compatible endpoint URL; it must not carry credentials (`https://user:pass@host` is refused), pass them in `credentials` |
|
|
212
|
-
| `region` | `string` | — | Required. Provider region or `'auto'` (R2)
|
|
213
|
-
| `bucket` | `string` | — | Default bucket (can be overridden per-call)
|
|
212
|
+
| `region` | `string` | — | Required. Provider region or `'auto'` (R2): 1 to 63 letters, digits, `.`, `_` or `-`, starting with a letter or digit |
|
|
213
|
+
| `bucket` | `string` | — | Default bucket (can be overridden per-call by a string; any other per-call kind fails with `STORAGE_VALIDATION_FAILED`) |
|
|
214
214
|
| `credentials` | `{ accessKeyId, secretAccessKey, sessionToken? }` | — | Load from env / Secrets Manager |
|
|
215
215
|
| `forcePathStyle` | `boolean` | `false` | Set to `true` for MinIO and self-hosted |
|
|
216
216
|
| `publicBaseUrl` | `string` | — | Base URL for public `getPublicUrl()` results; it must not carry credentials (a user or password is refused) |
|
|
217
217
|
| `cdnBaseUrl` | `string` | — | CDN edge URL (preferred over `publicBaseUrl`); it must not carry credentials either |
|
|
218
|
-
| `keyPrefix` | `string` | — | Prepended to every key before it reaches the provider; fixed per module instance |
|
|
218
|
+
| `keyPrefix` | `string` | — | Prepended to every key before it reaches the provider; fixed per module instance. A value that is not a string fails `forRoot`; `null` is read as unset |
|
|
219
219
|
| `maxAttempts` | `number` | `3` | SDK v3 retry count; an integer of at least `1` |
|
|
220
220
|
| `serverSideEncryption` | `'AES256' \| 'aws:kms'` | — | Global SSE policy; `aws:kms` needs `kmsKeyId`. `'NONE'` is accepted per upload only |
|
|
221
221
|
| `requestChecksumCalculation` | `'WHEN_SUPPORTED' \| 'WHEN_REQUIRED'` | SDK default | Set to `'WHEN_REQUIRED'` for non-AWS providers |
|
|
@@ -232,8 +232,10 @@ Options accepted by `forRoot` and `forRootAsync`:
|
|
|
232
232
|
| `multipart.thresholdBytes` | `number` | `5 MiB` | Switch to multipart at or above this size |
|
|
233
233
|
| `multipart.partSizeBytes` | `number` | `5 MiB` | Part size (S3 minimum: 5 MiB) |
|
|
234
234
|
| `multipart.queueSize` | `number` | `4` | Concurrent parts |
|
|
235
|
-
| `validation.mimeWhitelist` | `string[]` | — | No MIME check when absent. Wildcards: `'image/*'`. A value that lists several types never matches |
|
|
236
|
-
| `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes; counted as a stream is read, so it holds with or without `size` |
|
|
235
|
+
| `validation.mimeWhitelist` | `string[]` | — | No MIME check when absent. Wildcards: `'image/*'`. A value that lists several types never matches. An array of strings: any other kind fails `forRoot`, and `null` is refused (omit it to allow every type) |
|
|
236
|
+
| `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes; counted as a stream is read, so it holds with or without `size`. `null` is refused: omit it for no limit |
|
|
237
|
+
|
|
238
|
+
**Numeric options are whole numbers, checked at boot.** `validation.maxSizeBytes`, the `signedUrls` TTLs and `multipart.thresholdBytes` take a whole number of at least `1`; `multipart.partSizeBytes` takes one from 5 MiB (the S3 minimum) to 5 GiB (the S3 maximum) and `multipart.queueSize` one from `1` to `1000`. `NaN`, `Infinity`, a fraction, a string or a value out of range fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the option: a comparison alone lets `NaN` through, and `NaN` on a size limit would turn the limit off. The same holds per call: a `ttlSeconds` that is not a whole number above `0` fails with `STORAGE_SIGNED_URL_TTL_INVALID`, and a declared upload `size` or a list `maxKeys` that is not a whole number (at least `0`, and at least `1`) fails with `STORAGE_VALIDATION_FAILED` naming the field, before any request is made. A key set to `undefined` or `null` inside `signedUrls`, `multipart`, `validation` or `scanner` means the default, except `scanner.impl` and the `validation` options named below, which refuse `null`. The per-call values are refused on `null`, and the error for a TTL echoes no value but a finite number. Where leaving an option out means a limit or a protection is off, `null` is refused for it as well, so a config lookup that returns `null` for a missing key cannot switch it off: `validation.maxSizeBytes`, `validation.mimeWhitelist` and `validation.customValidators` (the last must be an array).
|
|
237
239
|
|
|
238
240
|
> **Do not use `maxRetries` or `signatureVersion`** — these are AWS SDK v2 options that do not exist in v3. The v3 SDK is SigV4-only. Use `maxAttempts` (default `3`) for retry configuration.
|
|
239
241
|
|
|
@@ -339,6 +341,10 @@ await storage.upload({
|
|
|
339
341
|
|
|
340
342
|
Uploads are automatically routed to multipart when `size >= multipart.thresholdBytes` (default 5 MiB). A `Readable` without a `size` always uses multipart. Multipart aborts automatically on failure (`leavePartsOnError: false`).
|
|
341
343
|
|
|
344
|
+
**What `body` may be.** A `Buffer`, a `Uint8Array`, an `ArrayBuffer`, a string or a Node `Readable`. Anything else (a `Blob`, an array of chunks, a plain object) fails with `STORAGE_VALIDATION_FAILED` (`details.field: 'body'`) before any request; it used to reach the upload as if it were a stream and fail as `STORAGE_MULTIPART_ABORTED` (500).
|
|
345
|
+
|
|
346
|
+
**A stream's declared size must be true.** A stream below the multipart threshold is sent as one request that carries `Content-Length` set to `size`, and a provider stores an object as soon as it has received that many bytes. So a body longer than `size` is refused with `STORAGE_SIZE_EXCEEDED` (`details.declaredSize`) before its last byte is sent, and a body shorter than `size` with `STORAGE_VALIDATION_FAILED` (`details.field: 'size'`, `details.declaredSize` and `details.receivedSize`); either way the request is abandoned and nothing is stored. A stream on the multipart path is still refused when it is longer than `size`; a shorter one is uploaded as it is. With a pre-upload scanner the stream is read into memory first, and the same rules apply to the length that was read. A `size` that is not a whole number of at least `0` fails with `STORAGE_VALIDATION_FAILED` before any request.
|
|
347
|
+
|
|
342
348
|
**The library owns a `Readable` once you pass it to `upload()`**, as with `stream.pipeline`. When the upload is rejected for any reason, the stream is destroyed without an error, which releases the file handle or socket behind it: a listener on the stream sees `'close'`, and the failure is the rejection of `upload()`. A call answered from the idempotency cache destroys the stream unread. Do not reuse a stream you passed in; open a new one to retry. No listener on the stream is needed to avoid an unhandled `'error'` event.
|
|
343
349
|
|
|
344
350
|
### Idempotency
|
|
@@ -418,7 +424,9 @@ A 304 is therefore thrown, not returned: catch `StorageException` and branch on
|
|
|
418
424
|
`abortSignal`. When it aborts, the call rejects `STORAGE_REQUEST_ABORTED` (HTTP 499) with
|
|
419
425
|
`details.reason` set to the **name** of the signal's reason (`AbortError`, `TimeoutError`), never its
|
|
420
426
|
message, so a host can tell its own timeout from a client disconnect. `exists` rejects too, rather
|
|
421
|
-
than answering `false`.
|
|
427
|
+
than answering `false`. A value that is not an `AbortSignal` fails with `STORAGE_VALIDATION_FAILED` naming
|
|
428
|
+
`abortSignal` and nothing of the value (`exists` answers `false` and logs a warning for it, as it does for
|
|
429
|
+
any failure that is not a missing object).
|
|
422
430
|
|
|
423
431
|
```typescript
|
|
424
432
|
const controller = new AbortController()
|
|
@@ -585,21 +593,30 @@ BymaxStorageModule.forRoot({
|
|
|
585
593
|
impl: new ClamAvScanner(),
|
|
586
594
|
mode: 'pre-upload', // 'pre-upload' | 'post-upload'
|
|
587
595
|
rejectOnUnknown: false, // treat inconclusive scans as pass (true = reject)
|
|
596
|
+
timeoutMs: 60_000, // refuse the upload when a scan takes longer (default 60000)
|
|
588
597
|
},
|
|
589
598
|
})
|
|
590
599
|
```
|
|
591
600
|
|
|
592
601
|
**Streams in pre-upload mode.** A stream can be read once, so in `pre-upload` mode it is first read
|
|
593
|
-
into memory, bounded by `validation.maxSizeBytes`, and the scanner and the upload share that buffer. A
|
|
602
|
+
into memory, bounded by `validation.maxSizeBytes`, and the scanner and the upload share that buffer. A `Buffer` or `Uint8Array` body is copied once in this mode, so a change the caller makes to its own array while the scan runs reaches neither (the second copy of the body is held until the upload ends, so the body takes twice its size in memory). The library validates and scans the bytes at call time; do not change a buffer you passed until the call settles. A
|
|
594
603
|
stream longer than the limit is refused with `STORAGE_VALIDATION_FAILED` as soon as it passes it, and
|
|
595
|
-
with no `maxSizeBytes` it is refused with a reason that says to set it or use `post-upload`.
|
|
596
|
-
|
|
604
|
+
with no `maxSizeBytes` it is refused with a reason that says to set it or use `post-upload`.
|
|
605
|
+
`post-upload` hands the scanner the key only, no body.
|
|
606
|
+
|
|
607
|
+
**The scanner configuration is checked at boot.** `scanner.impl` must be an object with a `scan` function, `scanner.mode` must be `'pre-upload'` or `'post-upload'` (unset or `null` is `'pre-upload'`) and `scanner.rejectOnUnknown` must be a boolean (unset or `null` is the default); anything else fails `forRoot` with `STORAGE_INVALID_CONFIG`, because a missing implementation or a mode that is neither value would scan nothing. Omit `scanner` for no scanner: a `scanner` of `null` is refused, so a config lookup that returns `null` for a missing key cannot switch scanning off.
|
|
608
|
+
|
|
609
|
+
**A scan has a deadline.** `scanner.timeoutMs` (a whole number of milliseconds from `1` to `2147483647`, default `60000`; there is no setting that waits without limit) bounds a scan: past it the upload is refused with `STORAGE_TIMEOUT` (503, `details.phase: 'scan'`), and in `post-upload` mode the stored object is removed as for any other rejection. The scanner is handed an `AbortSignal` (`signal` on its input) that aborts at the deadline. **A scanner that ignores the signal keeps running after the timeout**, so repeated timeouts pile up scanner work: honour the signal, or give the scanner a limit of its own. A listener the scanner adds to the signal and that throws is reported by Node as an uncaught exception, as for any event target: it cannot be caught around the abort.
|
|
610
|
+
|
|
611
|
+
**What counts as a verdict.** Only the exact strings `'clean'`, `'infected'` and `'unknown'` are results. Anything else (no result, `null`, `'CLEAN'`, `' clean'`, an object, a made-up status) is rejected as `STORAGE_SCAN_INCONCLUSIVE` with `details.reason: 'unrecognised scanner status'`, whatever `rejectOnUnknown` says: an object the scanner did not call clean is not stored.
|
|
597
612
|
|
|
598
613
|
**What a client is told.** `STORAGE_SCAN_INFECTED` carries `details.engine` and `details.threat`, and
|
|
599
|
-
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine
|
|
614
|
+
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `Scanner failed` warning with the message (cut) so there is a trace of why; a scan that outlasts `scanner.timeoutMs` is logged as `Scan timed out` with the key and the limit. A cut never leaves half of a surrogate pair, so an emoji at the cut is dropped whole. The free-form `details` your scanner returns is the
|
|
600
615
|
scanner's own text, so it is written to the warning log (escaped and truncated) and never to the
|
|
601
616
|
body of the error.
|
|
602
617
|
|
|
618
|
+
In `post-upload` mode your scanner is given the key and reads the object itself, so another writer that overwrites the same key between your upload and the scan makes the verdict about their object, and an infected verdict deletes the key whichever object it now holds.
|
|
619
|
+
|
|
603
620
|
**A rejected object that could not be removed.** In `post-upload` mode the object is already stored
|
|
604
621
|
when the scanner rejects it, so the library deletes it and rejects `upload()` with the verdict
|
|
605
622
|
(`STORAGE_SCAN_INFECTED` or `STORAGE_SCAN_INCONCLUSIVE`). If that delete fails the verdict is still the
|
|
@@ -762,14 +779,14 @@ All errors are thrown as `StorageException extends HttpException`. The response
|
|
|
762
779
|
| `STORAGE_CONTENT_TYPE_REQUIRED` | 400 | `upload()` called without `contentType` |
|
|
763
780
|
| `STORAGE_MIME_NOT_ALLOWED` | 415 | `contentType` outside `mimeWhitelist`, at upload or at presign |
|
|
764
781
|
| `STORAGE_SIZE_EXCEEDED` | 413 | `size > maxSizeBytes` |
|
|
765
|
-
| `STORAGE_VALIDATION_FAILED` | 400 | `IUploadValidator` rejected the file, or a header-bound value is not a header value (`details.field`, `details.reason
|
|
782
|
+
| `STORAGE_VALIDATION_FAILED` | 400 | `IUploadValidator` rejected the file, or a header-bound value is not a header value, or a per-call `bucket` or `abortSignal` is of the wrong kind (`details.field`, `details.reason`; the value is never repeated) |
|
|
766
783
|
| `STORAGE_SCAN_INFECTED` | 422 | Scanner returned `'infected'` (`details.threat`); `details.cleanup: 'failed'` means the object is still stored |
|
|
767
|
-
| `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true
|
|
784
|
+
| `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true`, or a result that is not a recognised status |
|
|
768
785
|
| `STORAGE_OBJECT_NOT_FOUND` | 404 | `head()`, `download()`, or `copy()` on a nonexistent key |
|
|
769
786
|
| `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (`unreachable`, `provider-error`, `credentials-refused` or `rejected`; never the text of the driver or of the provider) plus the operation context (`op`, `bucket`, and `key` or `prefix`) |
|
|
770
787
|
| `STORAGE_SIGNED_URL_TTL_INVALID` | 400 | Per-request `ttlSeconds` ≤ 0 |
|
|
771
788
|
| `STORAGE_PART_TOO_SMALL` | 400 | Never thrown: a `multipart.partSizeBytes` below 5 MiB fails at init as `STORAGE_INVALID_CONFIG` |
|
|
772
|
-
| `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts` not an integer from 1 to 10 000 (`details.
|
|
789
|
+
| `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts` not an integer from 1 to 10 000 (`details.reason`; `details.provided` only when `parts` is a finite number) |
|
|
773
790
|
| `STORAGE_BUCKET_UNDEFINED` | 400 | Operation without a bucket and no default in config |
|
|
774
791
|
| `STORAGE_MULTIPART_ABORTED` | 500 | A multipart upload failed (**503** when the cause is an outage, as for `STORAGE_PROVIDER_ERROR`). `details.abort` is `'completed'` when the library aborted it, `'failed'` when the abort itself failed (logged as a warning, with what it threw): that upload then stays open on the provider until a lifecycle rule removes it |
|
|
775
792
|
| `STORAGE_INVALID_CONFIG` | 500 | `BymaxStorageModuleOptions` validation failed at initialization |
|