@bymax-one/nest-storage 1.1.18 → 1.1.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +145 -1
- package/README.md +184 -16
- package/dist/observability/index.cjs +151 -0
- package/dist/observability/index.d.cts +72 -0
- package/dist/observability/index.d.ts +72 -0
- package/dist/observability/index.mjs +145 -0
- package/dist/server/index.cjs +831 -444
- package/dist/server/index.d.cts +151 -12
- package/dist/server/index.d.ts +151 -12
- package/dist/server/index.mjs +831 -447
- package/package.json +32 -5
package/CHANGELOG.md
CHANGED
|
@@ -13,6 +13,148 @@ note on each entry is the compatibility contract — read it, not the number.
|
|
|
13
13
|
|
|
14
14
|
## [Unreleased]
|
|
15
15
|
|
|
16
|
+
## [1.1.20] - 2026-11-21
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
|
|
20
|
+
- **A throwing `onProgress` no longer fails an upload that stored its object.** On the single-shot path the hook was
|
|
21
|
+
called in the upload's own `try` block, so a hook that threw made `upload()` reject with `STORAGE_PROVIDER_ERROR`
|
|
22
|
+
after `PutObject` had succeeded; on the multipart path a throw reached the uploader. An async hook that rejected was
|
|
23
|
+
an unhandled rejection that ends the Node process. Both paths now ignore what the hook throws or rejects with.
|
|
24
|
+
|
|
25
|
+
**Apply to a derived backend:** nothing to change. A hook that threw used to turn a stored upload into an error; it
|
|
26
|
+
now leaves the result alone, so an application that relied on a throwing hook to fail an upload must fail it itself.
|
|
27
|
+
|
|
28
|
+
- **An option that throws when it is read is refused with `STORAGE_INVALID_CONFIG`.** A getter or a Proxy of the
|
|
29
|
+
host's as the options object, a group or a validator, and a `then` that throws on the object a `useFactory` returns,
|
|
30
|
+
surfaced the host's own error from `forRoot` and `forRootAsync`. They now fail `STORAGE_INVALID_CONFIG`
|
|
31
|
+
(`reason: 'the options could not be read'`), the host's error is kept as the `cause` of the exception and none of
|
|
32
|
+
its text is in the response, and a whitelist or a validator list whose iterator throws is refused the same way instead
|
|
33
|
+
of as "must be an array of strings". `StorageException` takes an optional `cause`.
|
|
34
|
+
|
|
35
|
+
**Apply to a derived backend:** a factory that throws is now reported as `STORAGE_INVALID_CONFIG` with the
|
|
36
|
+
factory's error as the `cause`; read `cause` where you matched the raw error.
|
|
37
|
+
|
|
38
|
+
- **A lone surrogate in a logged value is escaped.** `escapeForLog` left a lone surrogate (`\ud800`) as it was, and a
|
|
39
|
+
logger writes it as an invalid character. It shows as `\ud800` now, like the other characters that cannot be shown;
|
|
40
|
+
a well-formed pair, an emoji, stays readable.
|
|
41
|
+
|
|
42
|
+
**Apply to a derived backend:** nothing to change; a log rule that matched the raw character reads the escape.
|
|
43
|
+
|
|
44
|
+
- **A very large sum of uploaded bytes no longer fails the metrics scrape.** `bymax_storage_bytes_uploaded_total` is
|
|
45
|
+
the sum of every object's size, and two finite sizes can add up past the largest number, to `Infinity`, which a
|
|
46
|
+
counter refuses: the whole `/metrics` answer then failed. The count in the tally now stops at the largest finite
|
|
47
|
+
number.
|
|
48
|
+
|
|
49
|
+
**Apply to a derived backend:** nothing to change; a scrape that failed on that sum now answers.
|
|
50
|
+
|
|
51
|
+
- **The README names what the multipart path does on a failure.** It said the SDK aborts the upload
|
|
52
|
+
(`leavePartsOnError: false`); the library passes `true` and sends `AbortMultipartUpload` itself, so a failing abort
|
|
53
|
+
cannot replace the error that caused it, and the outcome is in `details.abort`.
|
|
54
|
+
|
|
55
|
+
**Apply to a derived backend:** nothing to change; only the text was wrong.
|
|
56
|
+
|
|
57
|
+
### Changed
|
|
58
|
+
|
|
59
|
+
- **The span attributes are named, and the scan span takes the two scan modes.** `StorageSpanAttributes` was
|
|
60
|
+
`Readonly<Record<string, string>>`; it is now an interface naming the attributes the library sets, so a call site cannot add a key
|
|
61
|
+
that carries a bucket or a payload, and the scan span's `mode` parameter is `'pre-upload' | 'post-upload'` instead of
|
|
62
|
+
`string`.
|
|
63
|
+
|
|
64
|
+
**Apply to a derived backend:** code that calls the scan span with a mode the interface does not declare, that builds
|
|
65
|
+
attributes with another key, or that reads an attribute by a key it does not declare does not compile against the
|
|
66
|
+
interface; use the declared modes and names. Nothing changes at run time.
|
|
67
|
+
|
|
68
|
+
- **`defaultPublicRead` and `forcePathStyle` must be booleans, and so must a per-call `publicRead`.** A string from an
|
|
69
|
+
environment variable, `'false'`, is truthy: it passed the `??` default and made every upload `public-read`. A value
|
|
70
|
+
of another kind for either option is now refused at boot with `STORAGE_INVALID_CONFIG`, and a per-call `publicRead`
|
|
71
|
+
that is set and is not a boolean fails that call with `STORAGE_VALIDATION_FAILED` (`details.field: 'publicRead'`);
|
|
72
|
+
`null` is refused like any other non-boolean, and an absent value still takes the default.
|
|
73
|
+
|
|
74
|
+
**Apply to a derived backend:** a host that passes `process.env.X` straight into either option now gets a boot
|
|
75
|
+
refusal; compare it to `'true'` and pass the boolean.
|
|
76
|
+
|
|
77
|
+
- **`validation.mimeWhitelist` and `validation.customValidators` hold at most 1000 entries.** A longer array, or an
|
|
78
|
+
array whose reported length is not a count, is refused at boot with `STORAGE_INVALID_CONFIG` before any entry is
|
|
79
|
+
read, so a sparse array of billions of holes no longer stalls the start.
|
|
80
|
+
|
|
81
|
+
**Apply to a derived backend:** nothing to change unless a list holds more than 1000 entries; a whitelist that long
|
|
82
|
+
can use a wildcard instead.
|
|
83
|
+
|
|
84
|
+
## [1.1.19] - 2026-11-20
|
|
85
|
+
|
|
86
|
+
### Added
|
|
87
|
+
|
|
88
|
+
- **An optional `./observability` subpath and a span seam.** `StorageMetrics` publishes
|
|
89
|
+
`bymax_storage_operations_total`, `bymax_storage_operation_duration_seconds`, `bymax_storage_bytes_uploaded_total`
|
|
90
|
+
and `bymax_storage_scans_total` with labels from closed sets, and `StorageHealthIndicator` reports the bucket as
|
|
91
|
+
`storage` from one `HeadBucket`. The main entry now exports `StorageMetricsTally` and the closed sets
|
|
92
|
+
(`STORAGE_OPERATIONS`, `STORAGE_OUTCOMES`, `SCAN_VERDICTS`), counts every public operation, and starts the
|
|
93
|
+
`storage.upload`, `storage.download_url` and `storage.scan` spans when `@opentelemetry/api` is installed. `bymax_storage_bytes_uploaded_total` counts the bytes an upload sent, including a stream without a declared `size`. The new
|
|
94
|
+
peers `@bymax-one/nest-core >=1.6.15`, `prom-client ^15` and `@opentelemetry/api ^1.9` are optional.
|
|
95
|
+
|
|
96
|
+
**Apply to a derived backend:** nothing changes unless you use it. To publish the metrics, add `StorageMetrics` and
|
|
97
|
+
`StorageHealthIndicator` to the providers of a module and install `prom-client`; the README "Observability"
|
|
98
|
+
section has the table of metrics, labels and spans.
|
|
99
|
+
|
|
100
|
+
### Changed
|
|
101
|
+
|
|
102
|
+
- **The library's log lines are object events.** Every operational line is now an object with an `event` code
|
|
103
|
+
(`STORAGE_*`), a `message` with the library's own text and named fields, through the Nest `Logger`; the README has
|
|
104
|
+
a "Log events" table. What the provider, the scanner or the host raised goes under `err` as a fresh error (name,
|
|
105
|
+
provider code, masked and cut message), never into the text, and never the SDK's own error. The key you passed is
|
|
106
|
+
no longer logged by the provider-failure and scan events, the client-ready line no longer carries the endpoint or
|
|
107
|
+
the region, and a host logger that throws can no longer fail the call being logged. A host `LoggerService` method that is
|
|
108
|
+
async and rejects is still an unhandled rejection, which ends the Node process unless the application
|
|
109
|
+
handles `unhandledRejection`: Nest's `Logger` drops the value the method returns before the library can see it.
|
|
110
|
+
|
|
111
|
+
**Apply to a derived backend:** a log rule or an alert that matched the text of these lines (`Provider call failed`,
|
|
112
|
+
`Scan timed out`, `Scanner failed`, `Infected file detected`, `S3Client initialized`, the exists and delete
|
|
113
|
+
warnings, the TTL clamp) now matches the `event` code and reads the fields; the library text is kept in `message`.
|
|
114
|
+
A sink that stringifies the message sees an object: use a logger that writes object messages as fields. Anything
|
|
115
|
+
that relied on the key, the endpoint or the region in these lines reads them from the call or the configuration.
|
|
116
|
+
A quote or a backslash in a logged value is no longer escaped by the library, since the logger escapes it when it
|
|
117
|
+
serialises the field; control, bidirectional and format characters still show as escapes.
|
|
118
|
+
The `cleanupFailure` field of `STORAGE_SCAN_CLEANUP_FAILED` is the code of a `StorageException`; any other failure is masked, cut and escaped like `err`.
|
|
119
|
+
|
|
120
|
+
### Fixed
|
|
121
|
+
|
|
122
|
+
- **An option group of the wrong kind is refused at boot.** `validation`, `signedUrls` or `multipart` given as `false`,
|
|
123
|
+
`0`, a string, `true` or an array booted without error and was ignored, so `validation: enabled && { ... }` with
|
|
124
|
+
`enabled` false left uploads unvalidated. `forRoot` and `forRootAsync` now fail with `STORAGE_INVALID_CONFIG`
|
|
125
|
+
(`options.<group> must be an object`), repeating nothing of the value; `undefined` and `null` still mean "not
|
|
126
|
+
configured". `credentials` given as an array is refused as well; before, `forRoot` copied any array given for a
|
|
127
|
+
group into an empty group.
|
|
128
|
+
|
|
129
|
+
**Apply to a derived backend:** a configuration that passed one of those values for a group now fails at boot.
|
|
130
|
+
Pass an object, or leave the group out.
|
|
131
|
+
|
|
132
|
+
- **`validation.customValidators` entries are checked at boot.** An entry that is not an object with a `validate`
|
|
133
|
+
function (`42`, `null`, `{}`, a bare function) booted and then failed the first upload with a raw `TypeError`
|
|
134
|
+
(500). It now fails `forRoot` with `STORAGE_INVALID_CONFIG`. A `mimeWhitelist` that throws when it is read (a proxy,
|
|
135
|
+
a throwing index getter) is refused as "must be an array of strings" instead of surfacing the thrown error.
|
|
136
|
+
|
|
137
|
+
**Apply to a derived backend:** a list that held anything but validator objects now fails at boot; a class
|
|
138
|
+
instance that serves `validate` from its prototype is accepted. The README option table now has the
|
|
139
|
+
`validation.customValidators` row.
|
|
140
|
+
|
|
141
|
+
- **An error no longer repeats what the host configured.** `details` carried the module's `bucket` (when the call
|
|
142
|
+
named none), the `keyPrefix` (inside the `key` of an upload, head, download, delete, copy and signed-URL
|
|
143
|
+
failure, and the `prefix` of a list failure) and the whole `validation.mimeWhitelist` (in `STORAGE_MIME_NOT_ALLOWED`). The `key` is now the one the
|
|
144
|
+
caller passed, the `bucket` appears only when the caller passed one, and the whitelist is not repeated; a `key`,
|
|
145
|
+
`bucket`, `prefix`, `sourceKey` or `destKey` longer than 1024 characters is left out.
|
|
146
|
+
|
|
147
|
+
The same applies to a scan: a `STORAGE_TIMEOUT` for a scan that outlasts its deadline repeated the prefixed key and
|
|
148
|
+
the module bucket, and now carries the `key` the caller passed and a per-call `bucket`. `details.storedKey`,
|
|
149
|
+
which a rejected scan whose cleanup delete failed carried, is removed from the response: the operator gets the key as
|
|
150
|
+
stored in the `STORAGE_SCAN_CLEANUP_FAILED` log entry, in its `storedKey` field.
|
|
151
|
+
|
|
152
|
+
**Apply to a derived backend:** a client or a log rule that read `details.bucket` of a call that named no bucket,
|
|
153
|
+
`details.whitelist` of `STORAGE_MIME_NOT_ALLOWED`, `details.storedKey` of a rejected scan whose cleanup failed, or a
|
|
154
|
+
`keyPrefix` in `details.key` finds none, and reads the module configuration instead. A rule on the text of the
|
|
155
|
+
`STORAGE_SCAN_CLEANUP_FAILED` log message now reads the `event` and `storedKey` fields of the entry. A `key` and a
|
|
156
|
+
`bucket` that the caller passed are unchanged.
|
|
157
|
+
|
|
16
158
|
## [1.1.18] - 2026-11-18
|
|
17
159
|
|
|
18
160
|
### Fixed
|
|
@@ -1108,4 +1250,6 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
1108
1250
|
[1.1.16]: https://github.com/bymaxone/nest-storage/compare/v1.1.15...v1.1.16
|
|
1109
1251
|
[1.1.17]: https://github.com/bymaxone/nest-storage/compare/v1.1.16...v1.1.17
|
|
1110
1252
|
[1.1.18]: https://github.com/bymaxone/nest-storage/compare/v1.1.17...v1.1.18
|
|
1111
|
-
[
|
|
1253
|
+
[1.1.19]: https://github.com/bymaxone/nest-storage/compare/v1.1.18...v1.1.19
|
|
1254
|
+
[1.1.20]: https://github.com/bymaxone/nest-storage/compare/v1.1.19...v1.1.20
|
|
1255
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.20...HEAD
|
package/README.md
CHANGED
|
@@ -58,8 +58,8 @@ minimal.
|
|
|
58
58
|
|
|
59
59
|
## 🔥 Features
|
|
60
60
|
|
|
61
|
-
- ✅ **Multipart upload** — `@aws-sdk/lib-storage` with
|
|
62
|
-
|
|
61
|
+
- ✅ **Multipart upload** — `@aws-sdk/lib-storage` with an abort on failure that the library runs
|
|
62
|
+
itself, progress events, and a configurable threshold and part size
|
|
63
63
|
- ✅ **Content validation** — MIME allowlist with wildcards (`image/*`) and a size limit, then any
|
|
64
64
|
number of `IUploadValidator`s, all refused before a byte is stored
|
|
65
65
|
- ✅ **Virus scanning hook** — `IFileScanner` pre-upload or post-upload, for ClamAV, AWS Macie or
|
|
@@ -81,6 +81,9 @@ minimal.
|
|
|
81
81
|
provider-specific operations this surface does not wrap
|
|
82
82
|
- ✅ **Key prefix and server-side encryption** — `keyPrefix` is prepended after normalization and the
|
|
83
83
|
keys the library returns are relative to it; AES256 or `aws:kms`, globally or per upload
|
|
84
|
+
- ✅ **Metrics, readiness and spans** — an optional `./observability` subpath publishes Prometheus
|
|
85
|
+
metrics and a bucket readiness check; spans start when `@opentelemetry/api` is installed. Labels are
|
|
86
|
+
closed sets: no key, bucket or prefix is ever a label
|
|
84
87
|
- ✅ **Typed end to end** — TypeScript `strict` with `exactOptionalPropertyTypes` and
|
|
85
88
|
`noUncheckedIndexedAccess`; zero `any`
|
|
86
89
|
|
|
@@ -91,11 +94,12 @@ minimal.
|
|
|
91
94
|
| Subpath | Contents |
|
|
92
95
|
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
93
96
|
| `.` | Server runtime: `BymaxStorageModule`, `StorageService`, `SignedUrlService`, `BucketLifecycleService`, `providerRecipes`, DI tokens, interfaces, `StorageException`, `NoOpUploadValidator`, `NoOpFileScanner` |
|
|
97
|
+
| `./observability` | Optional: `StorageMetrics` (Prometheus metrics), `StorageHealthIndicator` (readiness of the bucket) |
|
|
94
98
|
| `./shared` | Framework-free types (`UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`, `StorageErrorResponse`) + `STORAGE_ERROR_CODES` + `StorageErrorCode` + the default MIME whitelists and the TTL and multipart default constants |
|
|
95
99
|
|
|
96
100
|
The root re-exports some `./shared` names for convenience; import the rest from the subpath.
|
|
97
101
|
|
|
98
|
-
|
|
102
|
+
All subpaths ship ESM **and** CommonJS with declarations for each format, so a
|
|
99
103
|
`require()` consumer receives CommonJS declarations rather than ESM ones. The
|
|
100
104
|
`pnpm check:exports` gate verifies this against the packed tarball.
|
|
101
105
|
|
|
@@ -103,7 +107,8 @@ Both subpaths ship ESM **and** CommonJS with declarations for each format, so a
|
|
|
103
107
|
|
|
104
108
|
| Subpath | Required peers |
|
|
105
109
|
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
106
|
-
| `.` (server) | `@nestjs/common ^11.0.16`, `@nestjs/core ^11.1.18`, `@aws-sdk/client-s3 ^3.700`, `@aws-sdk/lib-storage ^3.700`, `@aws-sdk/s3-request-presigner ^3.700`, `reflect-metadata ^0.2` |
|
|
110
|
+
| `.` (server) | `@nestjs/common ^11.0.16`, `@nestjs/core ^11.1.18`, `@aws-sdk/client-s3 ^3.700`, `@aws-sdk/lib-storage ^3.700`, `@aws-sdk/s3-request-presigner ^3.700`, `reflect-metadata ^0.2`; optional: `@opentelemetry/api ^1.9` (the spans start only when it is installed) |
|
|
111
|
+
| `./observability` | The peers of `.`, and `prom-client ^15` for the metrics; `@bymax-one/nest-core >=1.6.15` is how a host discovers the two classes. `prom-client` and `@bymax-one/nest-core` are optional for the package, needed only to use this subpath |
|
|
107
112
|
| `./shared` | None |
|
|
108
113
|
|
|
109
114
|
---
|
|
@@ -232,8 +237,15 @@ Options accepted by `forRoot` and `forRootAsync`:
|
|
|
232
237
|
| `multipart.thresholdBytes` | `number` | `5 MiB` | Switch to multipart at or above this size |
|
|
233
238
|
| `multipart.partSizeBytes` | `number` | `5 MiB` | Part size (S3 minimum: 5 MiB) |
|
|
234
239
|
| `multipart.queueSize` | `number` | `4` | Concurrent parts |
|
|
240
|
+
| `scanner.impl` | `IFileScanner` | — | The scanner. Without it no scan runs; an object with a `scan` function, see Virus Scanning |
|
|
241
|
+
| `scanner.mode` | `'pre-upload' \| 'post-upload'` | `'pre-upload'` | When the scan runs: before the bytes are stored, or after, on the stored key |
|
|
242
|
+
| `scanner.rejectOnUnknown` | `boolean` | `false` | `true` rejects an inconclusive (`'unknown'`) scan; `false` lets it through with a warning |
|
|
243
|
+
| `scanner.timeoutMs` | `number` | `60000` | Deadline of one scan; a whole number from `1` to `2147483647`; past it the upload is refused with `STORAGE_TIMEOUT` |
|
|
235
244
|
| `validation.mimeWhitelist` | `string[]` | — | No MIME check when absent. Wildcards: `'image/*'`. A value that lists several types never matches. An array of strings: any other kind fails `forRoot`, and `null` is refused (omit it to allow every type) |
|
|
236
245
|
| `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes; counted as a stream is read, so it holds with or without `size`. `null` is refused: omit it for no limit |
|
|
246
|
+
| `validation.customValidators` | `IUploadValidator[]` | — | Run in order before the upload. An array of objects that each have a `validate` function: any other kind, or an entry that is not such an object, fails `forRoot`; `null` is refused (omit it to run none) |
|
|
247
|
+
|
|
248
|
+
**Option groups are objects.** `signedUrls`, `multipart` and `validation` are each an object, or left out (`null` is read as left out). `false`, `0`, a string, `true` or an array fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the group and repeating nothing of the value: ignoring it would boot without the protection that a toggle such as `validation: enabled && { ... }` was meant to switch on.
|
|
237
249
|
|
|
238
250
|
**Numeric options are whole numbers, checked at boot.** `validation.maxSizeBytes`, the `signedUrls` TTLs and `multipart.thresholdBytes` take a whole number of at least `1`; `multipart.partSizeBytes` takes one from 5 MiB (the S3 minimum) to 5 GiB (the S3 maximum) and `multipart.queueSize` one from `1` to `1000`. `NaN`, `Infinity`, a fraction, a string or a value out of range fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the option: a comparison alone lets `NaN` through, and `NaN` on a size limit would turn the limit off. The same holds per call: a `ttlSeconds` that is not a whole number above `0` fails with `STORAGE_SIGNED_URL_TTL_INVALID`, and a declared upload `size` or a list `maxKeys` that is not a whole number (at least `0`, and at least `1`) fails with `STORAGE_VALIDATION_FAILED` naming the field, before any request is made. A key set to `undefined` or `null` inside `signedUrls`, `multipart`, `validation` or `scanner` means the default, except `scanner.impl` and the `validation` options named below, which refuse `null`. The per-call values are refused on `null`, and the error for a TTL echoes no value but a finite number. Where leaving an option out means a limit or a protection is off, `null` is refused for it as well, so a config lookup that returns `null` for a missing key cannot switch it off: `validation.maxSizeBytes`, `validation.mimeWhitelist` and `validation.customValidators` (the last must be an array).
|
|
239
251
|
|
|
@@ -339,7 +351,12 @@ await storage.upload({
|
|
|
339
351
|
})
|
|
340
352
|
```
|
|
341
353
|
|
|
342
|
-
|
|
354
|
+
`onProgress` receives `{ loaded, total?, part? }`: the bytes sent so far, the total when it is known, and the
|
|
355
|
+
part number on a multipart upload. A single-shot upload reports once, when it ends; a multipart upload reports
|
|
356
|
+
as each part is sent.
|
|
357
|
+
A hook that throws, or an async one that rejects, is ignored: the upload still resolves with its result and the process is not ended.
|
|
358
|
+
|
|
359
|
+
Uploads are automatically routed to multipart when `size >= multipart.thresholdBytes` (default 5 MiB). A `Readable` without a `size` always uses multipart. A failed multipart upload is aborted by the library, which tells the SDK to leave the parts (`leavePartsOnError: true`) and runs `AbortMultipartUpload` itself, so an abort that fails cannot replace the error that caused it; the outcome (`completed` or `failed`) is in the error's `details.abort`.
|
|
343
360
|
|
|
344
361
|
**What `body` may be.** A `Buffer`, a `Uint8Array`, an `ArrayBuffer`, a string or a Node `Readable`. Anything else (a `Blob`, an array of chunks, a plain object) fails with `STORAGE_VALIDATION_FAILED` (`details.field: 'body'`) before any request; it used to reach the upload as if it were a stream and fail as `STORAGE_MULTIPART_ABORTED` (500).
|
|
345
362
|
|
|
@@ -598,6 +615,11 @@ BymaxStorageModule.forRoot({
|
|
|
598
615
|
})
|
|
599
616
|
```
|
|
600
617
|
|
|
618
|
+
**What your scanner is given.** `scan(input)` receives `mode` (`'pre-upload'` or `'post-upload'`, the configured
|
|
619
|
+
`scanner.mode`), the `key` and `bucket` as the provider holds them, the `contentType`, the `size` when known, a
|
|
620
|
+
`signal` that aborts at `scanner.timeoutMs`, and a `body` (a `Buffer` or a stream) only when `mode` is
|
|
621
|
+
`'pre-upload'`. In `'post-upload'` mode the scanner reads the object itself.
|
|
622
|
+
|
|
601
623
|
**Streams in pre-upload mode.** A stream can be read once, so in `pre-upload` mode it is first read
|
|
602
624
|
into memory, bounded by `validation.maxSizeBytes`, and the scanner and the upload share that buffer. A `Buffer` or `Uint8Array` body is copied once in this mode, so a change the caller makes to its own array while the scan runs reaches neither (the second copy of the body is held until the upload ends, so the body takes twice its size in memory). The library validates and scans the bytes at call time; do not change a buffer you passed until the call settles. A
|
|
603
625
|
stream longer than the limit is refused with `STORAGE_VALIDATION_FAILED` as soon as it passes it, and
|
|
@@ -611,7 +633,7 @@ with no `maxSizeBytes` it is refused with a reason that says to set it or use `p
|
|
|
611
633
|
**What counts as a verdict.** Only the exact strings `'clean'`, `'infected'` and `'unknown'` are results. Anything else (no result, `null`, `'CLEAN'`, `' clean'`, an object, a made-up status) is rejected as `STORAGE_SCAN_INCONCLUSIVE` with `details.reason: 'unrecognised scanner status'`, whatever `rejectOnUnknown` says: an object the scanner did not call clean is not stored.
|
|
612
634
|
|
|
613
635
|
**What a client is told.** `STORAGE_SCAN_INFECTED` carries `details.engine` and `details.threat`, and
|
|
614
|
-
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `
|
|
636
|
+
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `STORAGE_SCAN_FAILED` event with the error under `err` (its message cut) so there is a trace of why; a scan that outlasts `scanner.timeoutMs` is logged as `STORAGE_SCAN_TIMEOUT` with the limit (see [Log events](#-log-events)). The free-form `details` your scanner returns is the
|
|
615
637
|
scanner's own text, so it is written to the warning log (escaped and truncated) and never to the
|
|
616
638
|
body of the error.
|
|
617
639
|
|
|
@@ -620,9 +642,10 @@ In `post-upload` mode your scanner is given the key and reads the object itself,
|
|
|
620
642
|
**A rejected object that could not be removed.** In `post-upload` mode the object is already stored
|
|
621
643
|
when the scanner rejects it, so the library deletes it and rejects `upload()` with the verdict
|
|
622
644
|
(`STORAGE_SCAN_INFECTED` or `STORAGE_SCAN_INCONCLUSIVE`). If that delete fails the verdict is still the
|
|
623
|
-
error, and `details` carries `cleanup: 'failed'`, the `key`
|
|
624
|
-
`
|
|
625
|
-
|
|
645
|
+
error, and `details` carries `cleanup: 'failed'`, the `key` without any `keyPrefix` and the code of the delete
|
|
646
|
+
failure in `cleanupError`. The log gets an error entry with `event: 'STORAGE_SCAN_CLEANUP_FAILED'` and the fields `message`, `storedKey`
|
|
647
|
+
(the key as the provider holds it, with any `keyPrefix`, as a bucket listing shows it, so an operator can find and remove the
|
|
648
|
+
object) and `cleanupFailure` (why the delete failed); the `storedKey` is never in the response. With no `details.cleanup` the object was
|
|
626
649
|
removed; with `cleanup: 'failed'` it is **still stored** and has to be removed by other means.
|
|
627
650
|
|
|
628
651
|
---
|
|
@@ -643,6 +666,18 @@ if (page1.nextContinuationToken) {
|
|
|
643
666
|
}
|
|
644
667
|
```
|
|
645
668
|
|
|
669
|
+
| Option | Meaning |
|
|
670
|
+
| --- | --- |
|
|
671
|
+
| `prefix` | Only keys that start with it, after `keyPrefix` |
|
|
672
|
+
| `maxKeys` | Page size, a whole number of at least `1`; above `1000` it is lowered to `1000` (the S3 maximum). Default `1000` |
|
|
673
|
+
| `continuationToken` | The `nextContinuationToken` of the previous page |
|
|
674
|
+
| `delimiter` | Groups keys under sub-prefixes: with `'/'`, objects below `avatars/2026/` are not listed, and `avatars/2026/` is returned in `commonPrefixes` instead |
|
|
675
|
+
| `bucket` | Lists another bucket for this call |
|
|
676
|
+
| `abortSignal` | Ends the call when it aborts, see Cancelling a call |
|
|
677
|
+
|
|
678
|
+
The result is `{ objects, commonPrefixes, isTruncated, nextContinuationToken? }`. Keys and prefixes come back
|
|
679
|
+
relative to `keyPrefix`, so they can be passed to any other method as they are.
|
|
680
|
+
|
|
646
681
|
### Copy (server-side)
|
|
647
682
|
|
|
648
683
|
```typescript
|
|
@@ -652,6 +687,10 @@ const { etag } = await storage.copy({
|
|
|
652
687
|
})
|
|
653
688
|
```
|
|
654
689
|
|
|
690
|
+
`sourceBucket` and `destinationBucket` default to the module's bucket and may each name another, so a copy can
|
|
691
|
+
cross buckets; both keys are relative to `keyPrefix`. `publicRead` and `cacheControl` set the ACL and the
|
|
692
|
+
`Cache-Control` of the copy.
|
|
693
|
+
|
|
655
694
|
### Delete / Batch Delete
|
|
656
695
|
|
|
657
696
|
```typescript
|
|
@@ -771,6 +810,8 @@ BymaxStorageModule.forRootAsync({
|
|
|
771
810
|
|
|
772
811
|
All errors are thrown as `StorageException extends HttpException`. The response body is `{ error: { code, message, details? } }`.
|
|
773
812
|
|
|
813
|
+
**What `details` repeats.** Only what the caller supplied in the request that failed: the `key` (without the module's `keyPrefix`), the per-call `bucket`, the `prefix` of a `list()`, the `sourceKey` and `destKey` of a `copy()`, and the `contentType` of a refused upload (left out when it is not a valid header value). The others are strings of at most 1024 characters (S3's limit for a key) and are left out when longer or not a string. What the host set as configuration is never repeated: the module's `bucket` when the call did not name one, the `keyPrefix`, the `validation.mimeWhitelist` (a `STORAGE_MIME_NOT_ALLOWED` carries the refused `contentType` and not the list) and the `region`.
|
|
814
|
+
|
|
774
815
|
| Code | HTTP | When |
|
|
775
816
|
| -------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
776
817
|
| `STORAGE_NOT_CONFIGURED` | 503 | Credentials missing and an operation was called |
|
|
@@ -783,7 +824,7 @@ All errors are thrown as `StorageException extends HttpException`. The response
|
|
|
783
824
|
| `STORAGE_SCAN_INFECTED` | 422 | Scanner returned `'infected'` (`details.threat`); `details.cleanup: 'failed'` means the object is still stored |
|
|
784
825
|
| `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true`, or a result that is not a recognised status |
|
|
785
826
|
| `STORAGE_OBJECT_NOT_FOUND` | 404 | `head()`, `download()`, or `copy()` on a nonexistent key |
|
|
786
|
-
| `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (
|
|
827
|
+
| `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (never the text of the driver or of the provider) plus the operation context (`op`, and the `bucket`, `key` or `prefix` the caller gave; see _What `details` repeats_) |
|
|
787
828
|
| `STORAGE_SIGNED_URL_TTL_INVALID` | 400 | Per-request `ttlSeconds` ≤ 0 |
|
|
788
829
|
| `STORAGE_PART_TOO_SMALL` | 400 | Never thrown: a `multipart.partSizeBytes` below 5 MiB fails at init as `STORAGE_INVALID_CONFIG` |
|
|
789
830
|
| `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts` not an integer from 1 to 10 000 (`details.reason`; `details.provided` only when `parts` is a finite number) |
|
|
@@ -816,6 +857,118 @@ try {
|
|
|
816
857
|
|
|
817
858
|
---
|
|
818
859
|
|
|
860
|
+
## 📊 Observability
|
|
861
|
+
|
|
862
|
+
The main entry counts and times every operation and starts spans. The packages below are
|
|
863
|
+
optional peers: install what you use.
|
|
864
|
+
|
|
865
|
+
```bash
|
|
866
|
+
pnpm add prom-client @bymax-one/nest-core @opentelemetry/api
|
|
867
|
+
```
|
|
868
|
+
|
|
869
|
+
### Metrics
|
|
870
|
+
|
|
871
|
+
```ts
|
|
872
|
+
import { Module } from '@nestjs/common'
|
|
873
|
+
import { BymaxCoreModule } from '@bymax-one/nest-core'
|
|
874
|
+
import { BymaxStorageModule } from '@bymax-one/nest-storage'
|
|
875
|
+
import { StorageHealthIndicator, StorageMetrics } from '@bymax-one/nest-storage/observability'
|
|
876
|
+
|
|
877
|
+
@Module({
|
|
878
|
+
imports: [
|
|
879
|
+
BymaxCoreModule.forRoot({ metrics: { enabled: true }, health: { autoDiscover: true } }),
|
|
880
|
+
BymaxStorageModule.forRoot({
|
|
881
|
+
endpoint: process.env.S3_ENDPOINT ?? '',
|
|
882
|
+
region: 'us-east-1',
|
|
883
|
+
bucket: 'uploads',
|
|
884
|
+
credentials: { accessKeyId: process.env.S3_KEY ?? '', secretAccessKey: process.env.S3_SECRET ?? '' },
|
|
885
|
+
}),
|
|
886
|
+
],
|
|
887
|
+
providers: [StorageMetrics, StorageHealthIndicator],
|
|
888
|
+
})
|
|
889
|
+
export class AppModule {}
|
|
890
|
+
```
|
|
891
|
+
|
|
892
|
+
`@bymax-one/nest-core` finds both classes by their markers and calls `registerMetrics` on `StorageMetrics` once at bootstrap.
|
|
893
|
+
|
|
894
|
+
| Metric | Type | Labels | What it counts |
|
|
895
|
+
| --- | --- | --- | --- |
|
|
896
|
+
| `bymax_storage_operations_total` | counter | `operation`, `outcome` | Operations that ended |
|
|
897
|
+
| `bymax_storage_operation_duration_seconds` | histogram | `operation` | How long they took |
|
|
898
|
+
| `bymax_storage_bytes_uploaded_total` | counter | none | Bytes sent by uploads that succeeded |
|
|
899
|
+
| `bymax_storage_scans_total` | counter | `verdict` | File scans |
|
|
900
|
+
|
|
901
|
+
The labels come from closed sets, so the number of series never grows with traffic and no key, bucket, prefix or
|
|
902
|
+
message is ever a label:
|
|
903
|
+
|
|
904
|
+
- `operation`: `upload`, `download`, `head`, `delete`, `delete_many`, `list`, `copy`, `get_download_url`,
|
|
905
|
+
`get_upload_url`, `get_multipart_upload_urls`, `check_bucket`, `lifecycle_get`, `lifecycle_upsert`,
|
|
906
|
+
`lifecycle_remove`.
|
|
907
|
+
- `outcome`: `success`, `not_found` (the object was absent), `aborted` (your `abortSignal`, or an aborted multipart
|
|
908
|
+
upload), `error`.
|
|
909
|
+
- `verdict`: `clean`, `infected`, `unknown`, `timeout`, `error`.
|
|
910
|
+
|
|
911
|
+
`exists()` is counted as `head`, `downloadBuffer()` as `download`, and the delete that removes an object a post-upload scan rejected as `delete`. A download is timed to the response, not to the
|
|
912
|
+
last byte. A result returned from the idempotency cache counts as an upload but adds no bytes. The bytes are the ones the transfer sent: a stream without a declared `size` is counted too, and a `size` the stream did not reach is not; a failed upload adds none. The counters start
|
|
913
|
+
with the module; the histogram starts when `StorageMetrics` registers.
|
|
914
|
+
|
|
915
|
+
`delete_many` reports how it ended as a call, not per key: `deleteMany()` returns the keys that failed instead of
|
|
916
|
+
throwing, so a call in which every key failed is counted as `success`; read `failed` in its result for that. A `head`
|
|
917
|
+
of a bucket that does not exist is answered 404 by the provider as it is for a missing object, and is counted as
|
|
918
|
+
`not_found`.
|
|
919
|
+
|
|
920
|
+
`StorageMetricsTally` is what the metrics read, and the library writes to it as it works: `operation(operation,
|
|
921
|
+
outcome, seconds)` and `scan(verdict)` count, `bytesUploaded(bytes)` adds to the byte counter, `onDuration(listener)`
|
|
922
|
+
hands the listener the duration of every operation that ends from then on, and `snapshot()` returns the counts so
|
|
923
|
+
far as copies. A host that publishes its own metrics reads `snapshot()`.
|
|
924
|
+
|
|
925
|
+
### Readiness
|
|
926
|
+
|
|
927
|
+
`StorageHealthIndicator` reports under the name `storage`. It asks `StorageService.checkBucket()` for one
|
|
928
|
+
`HeadBucket` and answers `up` or `down` with no detail, so nothing about the bucket, the region or the endpoint
|
|
929
|
+
leaves with it. The probe has its own deadline, the answer is kept, and checks that arrive
|
|
930
|
+
while one is running share it. The probe is counted as `check_bucket`. The answer lags the bucket's real state by up to the 5 seconds it is kept plus the 2 seconds the probe may take,
|
|
931
|
+
so a bucket that stops answering can still read `up` for up to 7 seconds.
|
|
932
|
+
|
|
933
|
+
### Spans
|
|
934
|
+
|
|
935
|
+
Spans start through `@opentelemetry/api` when it is installed and a tracer is registered; without it the operation
|
|
936
|
+
runs as it is.
|
|
937
|
+
|
|
938
|
+
| Span | Around |
|
|
939
|
+
| --- | --- |
|
|
940
|
+
| `storage.upload` | `upload()` |
|
|
941
|
+
| `storage.download_url` | `getDownloadUrl()` |
|
|
942
|
+
| `storage.scan` | a file scan |
|
|
943
|
+
|
|
944
|
+
Attributes are `bymax.storage.outcome` (the outcome above), `bymax.storage.multipart` (`upload` only: `true` or
|
|
945
|
+
`false`) and `bymax.storage.scan_mode` (`scan` only: `pre-upload` or `post-upload`). A failed operation sets the
|
|
946
|
+
span status to error with the library error code, never its message.
|
|
947
|
+
|
|
948
|
+
---
|
|
949
|
+
|
|
950
|
+
## 📋 Log events
|
|
951
|
+
|
|
952
|
+
Every operational line the library writes is an **object** through the Nest `Logger`: an `event` code, a `message` with the library's own text, and the fields below. What the provider, the scanner or the host raised goes under `err`, as a fresh error built from its name, its provider code and its message (secrets masked, cut, control characters escaped); never into `message`, and never the SDK's own error, so no response, header or `cause` reaches a log. No field of an event carries the key you passed, the bucket, the endpoint or the region, except `storedKey` of `STORAGE_SCAN_CLEANUP_FAILED`, which an operator needs to remove the object. What the provider or the scanner wrote under `err` is their own text and can name the host, the key or the bucket. A host logger that throws does not change the call being logged. A host `LoggerService` method that is async and rejects is an unhandled rejection, and Node ends the process on it unless the application handles `unhandledRejection`: Nest's `Logger` drops the value a host method returns before the library can see it, so handle rejections inside your logger.
|
|
953
|
+
|
|
954
|
+
| Code | Level | When | Fields |
|
|
955
|
+
| --- | --- | --- | --- |
|
|
956
|
+
| `STORAGE_TTL_CLAMPED` | warn | `signedUrls.maxTtlSeconds` is above the SigV4 limit and the limit is used | `configuredSeconds`, `appliedSeconds` |
|
|
957
|
+
| `STORAGE_CREDENTIALS_MISSING` | warn | the module starts without credentials; operations then throw `STORAGE_NOT_CONFIGURED` | none |
|
|
958
|
+
| `STORAGE_CLIENT_READY` | log | the S3 client is created | none |
|
|
959
|
+
| `STORAGE_PROVIDER_CALL_FAILED` | warn | a call to the provider failed (a request you cancelled is not logged) | `operation`, `awsCode`, `httpStatus`, `requestId`, `reason`, `err` |
|
|
960
|
+
| `STORAGE_MULTIPART_ABORT_FAILED` | warn | aborting a failed multipart upload failed, so the upload stays open on the provider | `err` |
|
|
961
|
+
| `STORAGE_SCAN_INFECTED` | warn | the scanner said `infected` | `engine`, `threat`, `details` (the scanner's own, cut) |
|
|
962
|
+
| `STORAGE_SCAN_INCONCLUSIVE_ACCEPTED` | warn | the scanner said `unknown` and `rejectOnUnknown` is off | `engine` |
|
|
963
|
+
| `STORAGE_SCAN_INCONCLUSIVE_REJECTED` | warn | `unknown` with `rejectOnUnknown`, or a result outside the contract | `engine`, `status` (when unrecognised), `details` |
|
|
964
|
+
| `STORAGE_SCAN_FAILED` | warn | the scanner threw | `err` |
|
|
965
|
+
| `STORAGE_SCAN_TIMEOUT` | warn | the scan outlasted `scanner.timeoutMs` | `timeoutMs` |
|
|
966
|
+
| `STORAGE_SCAN_CLEANUP_FAILED` | error | the object a scan rejected could not be deleted and is still stored | `storedKey`, `cleanupFailure` |
|
|
967
|
+
| `STORAGE_EXISTS_FAILED` | warn | `exists()` met a failure that is not a 404 and answered `false` | `err` |
|
|
968
|
+
| `STORAGE_DELETE_NOT_FOUND` | warn | `delete()` found no such key (an idempotent no-op) | none |
|
|
969
|
+
|
|
970
|
+
---
|
|
971
|
+
|
|
819
972
|
## 🚫 What This Library Does NOT Do
|
|
820
973
|
|
|
821
974
|
- **Detect a file's real type.** The MIME check reads the declared `Content-Type` only; plug in an
|
|
@@ -911,12 +1064,13 @@ injected into every service, so `credentials` is attached as a non-enumerable ac
|
|
|
911
1064
|
is what a structured logger or an error reporter capturing a scope would otherwise pick up. Reading it
|
|
912
1065
|
on purpose is unchanged.
|
|
913
1066
|
|
|
914
|
-
### Error payloads carry the operation, and
|
|
1067
|
+
### Error payloads carry the operation, and the key you passed
|
|
915
1068
|
|
|
916
|
-
`details` holds the provider's error code
|
|
917
|
-
operation context the call site supplies: the operation
|
|
918
|
-
|
|
919
|
-
|
|
1069
|
+
`details` holds the provider's error code, the HTTP status, the request id, a fixed `reason`, and the
|
|
1070
|
+
operation context the call site supplies: the operation and the `key`, `bucket` or `prefix` the caller
|
|
1071
|
+
passed (see _What `details` repeats_). Neither the text of the provider nor anything the host configured is
|
|
1072
|
+
in it. An object key you passed still reaches whatever consumes the exception, so if a key is itself
|
|
1073
|
+
sensitive in your deployment, do not log the envelope verbatim.
|
|
920
1074
|
|
|
921
1075
|
### `keyPrefix` is a namespace, not an access boundary
|
|
922
1076
|
|
|
@@ -1030,7 +1184,14 @@ supply through it first: the guard below, then `keyPrefix`.
|
|
|
1030
1184
|
`deleteMany` result all omit the prefix, so any of them can be passed straight back to `head`,
|
|
1031
1185
|
`download`, `delete`, `copy` or a signed-URL call, which prepend it once. `UploadResult.publicUrl`
|
|
1032
1186
|
is the one full value: it is a URL, and the prefix is part of the object's real path. Error
|
|
1033
|
-
`details` carry the
|
|
1187
|
+
`details` carry the `key` you passed, without the prefix (see "What `details` repeats").
|
|
1188
|
+
|
|
1189
|
+
**The fields of the results.** `UploadResult` holds `key`, `bucket`, `etag`, `contentType`, `publicUrl`, `multipart`
|
|
1190
|
+
and `fromIdempotencyCache`, and `size` and `versionId` when known (`versionId` only on a versioned bucket).
|
|
1191
|
+
`ObjectMetadata` (from `head` and `download`) holds `key`, `bucket`, `size`, `contentType`, `etag`, `lastModified`
|
|
1192
|
+
and `metadata` (the `x-amz-meta-*` pairs), and `cacheControl`, `contentDisposition`, `storageClass` and `versionId`
|
|
1193
|
+
when the provider reports them. A `ListedObject` holds `key`, `size`, `etag` and `lastModified`, and `storageClass`
|
|
1194
|
+
when reported.
|
|
1034
1195
|
|
|
1035
1196
|
### Exported types and constants
|
|
1036
1197
|
|
|
@@ -1049,6 +1210,13 @@ is the one full value: it is a URL, and the prefix is part of the object's real
|
|
|
1049
1210
|
`LifecycleBucketOptions` (`{ bucket? }`).
|
|
1050
1211
|
- **Extension points:** `IUploadValidator`, `IFileScanner`, `FileScanResult`, and the helpers
|
|
1051
1212
|
`NoOpUploadValidator` and `NoOpFileScanner`.
|
|
1213
|
+
- **Observability (main entry):** `StorageMetricsTally`, the counts the `./observability` subpath publishes, and the
|
|
1214
|
+
closed sets they are keyed by: `STORAGE_OPERATIONS` and `StorageOperation`, `STORAGE_OUTCOMES` and
|
|
1215
|
+
`StorageOutcome`, `SCAN_VERDICTS` and `ScanVerdict`.
|
|
1216
|
+
- **From `./observability`:** `StorageMetrics`, `StorageHealthIndicator`, the name it reports under
|
|
1217
|
+
(`STORAGE_HEALTH_INDICATOR_NAME`), its deadline and cache (`STORAGE_HEALTH_TIMEOUT_MS`,
|
|
1218
|
+
`STORAGE_HEALTH_CACHE_MS`), and the metadata keys `@bymax-one/nest-core` discovers them by
|
|
1219
|
+
(`METRICS_CONTRIBUTOR_METADATA`, `HEALTH_INDICATOR_METADATA`).
|
|
1052
1220
|
- **From `./shared`:** `UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`,
|
|
1053
1221
|
`StorageErrorResponse`, `StorageErrorCode`, `STORAGE_ERROR_CODES`, the MIME presets
|
|
1054
1222
|
`DEFAULT_IMAGE_MIME_WHITELIST`, `DEFAULT_VIDEO_MIME_WHITELIST`, `DEFAULT_DOC_MIME_WHITELIST`, and the
|