@bymax-one/nest-storage 1.1.17 → 1.1.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +169 -1
- package/README.md +172 -21
- package/dist/observability/index.cjs +151 -0
- package/dist/observability/index.d.cts +72 -0
- package/dist/observability/index.d.ts +72 -0
- package/dist/observability/index.mjs +145 -0
- package/dist/server/index.cjs +895 -420
- package/dist/server/index.d.cts +137 -11
- package/dist/server/index.d.ts +137 -11
- package/dist/server/index.mjs +895 -423
- package/package.json +32 -5
package/CHANGELOG.md
CHANGED
|
@@ -13,6 +13,172 @@ note on each entry is the compatibility contract — read it, not the number.
|
|
|
13
13
|
|
|
14
14
|
## [Unreleased]
|
|
15
15
|
|
|
16
|
+
## [1.1.19] - 2026-11-20
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **An optional `./observability` subpath and a span seam.** `StorageMetrics` publishes
|
|
21
|
+
`bymax_storage_operations_total`, `bymax_storage_operation_duration_seconds`, `bymax_storage_bytes_uploaded_total`
|
|
22
|
+
and `bymax_storage_scans_total` with labels from closed sets, and `StorageHealthIndicator` reports the bucket as
|
|
23
|
+
`storage` from one `HeadBucket`. The main entry now exports `StorageMetricsTally` and the closed sets
|
|
24
|
+
(`STORAGE_OPERATIONS`, `STORAGE_OUTCOMES`, `SCAN_VERDICTS`), counts every public operation, and starts the
|
|
25
|
+
`storage.upload`, `storage.download_url` and `storage.scan` spans when `@opentelemetry/api` is installed. `bymax_storage_bytes_uploaded_total` counts the bytes an upload sent, including a stream without a declared `size`. The new
|
|
26
|
+
peers `@bymax-one/nest-core >=1.6.15`, `prom-client ^15` and `@opentelemetry/api ^1.9` are optional.
|
|
27
|
+
|
|
28
|
+
**Apply to a derived backend:** nothing changes unless you use it. To publish the metrics, add `StorageMetrics` and
|
|
29
|
+
`StorageHealthIndicator` to the providers of a module and install `prom-client`; the README "Observability"
|
|
30
|
+
section has the table of metrics, labels and spans.
|
|
31
|
+
|
|
32
|
+
### Changed
|
|
33
|
+
|
|
34
|
+
- **The library's log lines are object events.** Every operational line is now an object with an `event` code
|
|
35
|
+
(`STORAGE_*`), a `message` with the library's own text and named fields, through the Nest `Logger`; the README has
|
|
36
|
+
a "Log events" table. What the provider, the scanner or the host raised goes under `err` as a fresh error (name,
|
|
37
|
+
provider code, masked and cut message), never into the text, and never the SDK's own error. The key you passed is
|
|
38
|
+
no longer logged by the provider-failure and scan events, the client-ready line no longer carries the endpoint or
|
|
39
|
+
the region, and a host logger that throws can no longer fail the call being logged. A host `LoggerService` method that is
|
|
40
|
+
async and rejects is still an unhandled rejection, which ends the Node process unless the application
|
|
41
|
+
handles `unhandledRejection`: Nest's `Logger` drops the value the method returns before the library can see it.
|
|
42
|
+
|
|
43
|
+
**Apply to a derived backend:** a log rule or an alert that matched the text of these lines (`Provider call failed`,
|
|
44
|
+
`Scan timed out`, `Scanner failed`, `Infected file detected`, `S3Client initialized`, the exists and delete
|
|
45
|
+
warnings, the TTL clamp) now matches the `event` code and reads the fields; the library text is kept in `message`.
|
|
46
|
+
A sink that stringifies the message sees an object: use a logger that writes object messages as fields. Anything
|
|
47
|
+
that relied on the key, the endpoint or the region in these lines reads them from the call or the configuration.
|
|
48
|
+
A quote or a backslash in a logged value is no longer escaped by the library, since the logger escapes it when it
|
|
49
|
+
serialises the field; control, bidirectional and format characters still show as escapes.
|
|
50
|
+
The `cleanupFailure` field of `STORAGE_SCAN_CLEANUP_FAILED` is the code of a `StorageException`; any other failure is masked, cut and escaped like `err`.
|
|
51
|
+
|
|
52
|
+
### Fixed
|
|
53
|
+
|
|
54
|
+
- **An option group of the wrong kind is refused at boot.** `validation`, `signedUrls` or `multipart` given as `false`,
|
|
55
|
+
`0`, a string, `true` or an array booted without error and was ignored, so `validation: enabled && { ... }` with
|
|
56
|
+
`enabled` false left uploads unvalidated. `forRoot` and `forRootAsync` now fail with `STORAGE_INVALID_CONFIG`
|
|
57
|
+
(`options.<group> must be an object`), repeating nothing of the value; `undefined` and `null` still mean "not
|
|
58
|
+
configured". `credentials` given as an array is refused as well; before, `forRoot` copied any array given for a
|
|
59
|
+
group into an empty group.
|
|
60
|
+
|
|
61
|
+
**Apply to a derived backend:** a configuration that passed one of those values for a group now fails at boot.
|
|
62
|
+
Pass an object, or leave the group out.
|
|
63
|
+
|
|
64
|
+
- **`validation.customValidators` entries are checked at boot.** An entry that is not an object with a `validate`
|
|
65
|
+
function (`42`, `null`, `{}`, a bare function) booted and then failed the first upload with a raw `TypeError`
|
|
66
|
+
(500). It now fails `forRoot` with `STORAGE_INVALID_CONFIG`. A `mimeWhitelist` that throws when it is read (a proxy,
|
|
67
|
+
a throwing index getter) is refused as "must be an array of strings" instead of surfacing the thrown error.
|
|
68
|
+
|
|
69
|
+
**Apply to a derived backend:** a list that held anything but validator objects now fails at boot; a class
|
|
70
|
+
instance that serves `validate` from its prototype is accepted. The README option table now has the
|
|
71
|
+
`validation.customValidators` row.
|
|
72
|
+
|
|
73
|
+
- **An error no longer repeats what the host configured.** `details` carried the module's `bucket` (when the call
|
|
74
|
+
named none), the `keyPrefix` (inside the `key` of an upload, head, download, delete, copy and signed-URL
|
|
75
|
+
failure, and the `prefix` of a list failure) and the whole `validation.mimeWhitelist` (in `STORAGE_MIME_NOT_ALLOWED`). The `key` is now the one the
|
|
76
|
+
caller passed, the `bucket` appears only when the caller passed one, and the whitelist is not repeated; a `key`,
|
|
77
|
+
`bucket`, `prefix`, `sourceKey` or `destKey` longer than 1024 characters is left out.
|
|
78
|
+
|
|
79
|
+
The same applies to a scan: a `STORAGE_TIMEOUT` for a scan that outlasts its deadline repeated the prefixed key and
|
|
80
|
+
the module bucket, and now carries the `key` the caller passed and a per-call `bucket`. `details.storedKey`,
|
|
81
|
+
which a rejected scan whose cleanup delete failed carried, is removed from the response: the operator gets the key as
|
|
82
|
+
stored in the `STORAGE_SCAN_CLEANUP_FAILED` log entry, in its `storedKey` field.
|
|
83
|
+
|
|
84
|
+
**Apply to a derived backend:** a client or a log rule that read `details.bucket` of a call that named no bucket,
|
|
85
|
+
`details.whitelist` of `STORAGE_MIME_NOT_ALLOWED`, `details.storedKey` of a rejected scan whose cleanup failed, or a
|
|
86
|
+
`keyPrefix` in `details.key` finds none, and reads the module configuration instead. A rule on the text of the
|
|
87
|
+
`STORAGE_SCAN_CLEANUP_FAILED` log message now reads the `event` and `storedKey` fields of the entry. A `key` and a
|
|
88
|
+
`bucket` that the caller passed are unchanged.
|
|
89
|
+
|
|
90
|
+
## [1.1.18] - 2026-11-18
|
|
91
|
+
|
|
92
|
+
### Fixed
|
|
93
|
+
|
|
94
|
+
- **A per-call value of the wrong kind no longer reaches the provider call or a log line.** A per-call `bucket` that
|
|
95
|
+
is not a string (a `Symbol`, a function, an array, an object, a number) was repeated in `details.bucket` of the
|
|
96
|
+
error it caused, on every method that takes a bucket, and an `abortSignal` that was not an `AbortSignal` (a string,
|
|
97
|
+
a `Symbol`, a `BigInt`) was handed to the SDK, which named the value in its own error message, and that message is
|
|
98
|
+
logged as `Provider call failed`. Both are now refused before any request with `STORAGE_VALIDATION_FAILED` (400)
|
|
99
|
+
naming `bucket` or `abortSignal`, and nothing of the value. `getMultipartUploadUrls` no longer puts a `parts` that
|
|
100
|
+
is not a number in `details.provided`. A body that has `pipe`, `on` and `once` but no `pause` or `resume` is refused
|
|
101
|
+
with `STORAGE_VALIDATION_FAILED` naming `body`, where it used to be waited on until `STORAGE_TIMEOUT`.
|
|
102
|
+
|
|
103
|
+
**Apply to a derived backend:** a call that passed a per-call `bucket` that is not a string, or an `abortSignal` that
|
|
104
|
+
is not an `AbortSignal`, now fails with `STORAGE_VALIDATION_FAILED` (`details.field` is `bucket` or `abortSignal`)
|
|
105
|
+
where it used to reach the provider (`exists` answers `false` and logs a warning, as for any failure that is not a
|
|
106
|
+
missing object). `details.provided` of `STORAGE_INVALID_PART_COUNT` is present only when `parts` is a finite number. A
|
|
107
|
+
stream-like body without `pause` and `resume` is refused with `STORAGE_VALIDATION_FAILED`.
|
|
108
|
+
|
|
109
|
+
- **A configuration of the wrong kind is refused at boot, and a `null` that means "off" is refused.** `keyPrefix` that is
|
|
110
|
+
an array or an object was accepted and failed later as a raw `TypeError`; `validation.mimeWhitelist` that is a string
|
|
111
|
+
was accepted; `validation.customValidators` that is not an array was accepted; and a `region` with a space, a slash
|
|
112
|
+
or a newline in it was sent on to the SDK, which repeats it in its error messages. `forRoot` now refuses each with
|
|
113
|
+
`STORAGE_INVALID_CONFIG` naming the option and repeating nothing of the value: `keyPrefix` must be a string,
|
|
114
|
+
`mimeWhitelist` an array of strings, `customValidators` an array, and `region` 1 to 63 letters, digits, `.`, `_` or
|
|
115
|
+
`-` starting with a letter or digit. `validation.maxSizeBytes: null`, `validation.mimeWhitelist: null` and
|
|
116
|
+
`validation.customValidators: null` are refused as well: leaving each out means no size limit, every type allowed or
|
|
117
|
+
no custom validator, so a `null` from a config lookup of a missing key must not switch the protection off (1.1.17
|
|
118
|
+
read `maxSizeBytes: null` as "no limit").
|
|
119
|
+
|
|
120
|
+
**Apply to a derived backend:** a configuration that booted and is now refused: a `keyPrefix` that is not a string, a
|
|
121
|
+
`validation.mimeWhitelist` or `customValidators` that is not an array (and a `mimeWhitelist` with a non-string
|
|
122
|
+
entry), a `region` outside the shape above, and a `null` for `validation.maxSizeBytes`, `validation.mimeWhitelist` or
|
|
123
|
+
`validation.customValidators`. Omit the option for "none": `null` no longer means it. A `null` for any other option
|
|
124
|
+
inside a group is still the default.
|
|
125
|
+
|
|
126
|
+
**`scanner: null` is refused; omit the key for no scanner.** 1.1.17 read a `scanner` of `null` as "no scanner", so a
|
|
127
|
+
config lookup that returned `null` for a missing key booted with scanning off. `forRoot` now fails with
|
|
128
|
+
`STORAGE_INVALID_CONFIG` ("scanner must not be null; omit it for no scanner"); a `scanner` that is `false`, `0`,
|
|
129
|
+
`''` or `[]` was already refused.
|
|
130
|
+
|
|
131
|
+
**Apply to a derived backend:** a `scanner` of `null` now fails `forRoot` and `forRootAsync` with
|
|
132
|
+
`STORAGE_INVALID_CONFIG`; omit the key to run no scanner.
|
|
133
|
+
|
|
134
|
+
- **A value that throws when it is read no longer hangs a source read.** The check that tells an `Error` from any other
|
|
135
|
+
value in the idle-timeout wrapper read its `name` and `message` outside a guard: a rejection value whose property
|
|
136
|
+
getter throws left the wrapped read pending for good, with its timer already cleared. It is now treated as not
|
|
137
|
+
being an error, and the read ends with one.
|
|
138
|
+
|
|
139
|
+
**Apply to a derived backend:** nothing to change; a read that used to stay pending in that case now ends with an error.
|
|
140
|
+
|
|
141
|
+
- **Every option is read once.** Validation and resolution each read the options by property name, so a getter that
|
|
142
|
+
returned a valid value for the check and a hostile one for the use (`scanner.mode` that answers `'pre-upload'`
|
|
143
|
+
first and `'preupload'` after, a size limit that answers a number and then `NaN`) passed the check and was used as
|
|
144
|
+
the hostile value. `forRoot` and `forRootAsync` now copy each known option, and each key of the option groups, into a
|
|
145
|
+
plain object once; the copy is what is validated and what is resolved. The arrays of `validation` are copied once
|
|
146
|
+
as well.
|
|
147
|
+
|
|
148
|
+
**Apply to a derived backend:** nothing to change for a plain options object. An options object whose property
|
|
149
|
+
getters answer differently from one read to the next now gets the first answer everywhere; a key that is not an
|
|
150
|
+
option is not read at all.
|
|
151
|
+
|
|
152
|
+
- **A cut of scanner text never leaves half of a surrogate pair, and a scan that times out is logged.** The engine and
|
|
153
|
+
threat names of a scan result (in the response body and the log), the unrecognised status, the scanner's `details`
|
|
154
|
+
and the message of a scanner that throws were cut at a fixed length, and a cut between the two halves of an
|
|
155
|
+
emoji left a lone surrogate in a JSON body or a log line. The cut now drops the whole pair. A scan that outlasts
|
|
156
|
+
`scanner.timeoutMs` logs one `Scan timed out` warning with the key and the limit; until now only the typed
|
|
157
|
+
`STORAGE_TIMEOUT` carried it.
|
|
158
|
+
|
|
159
|
+
**Apply to a derived backend:** an engine or threat name cut at its limit is one character shorter when an
|
|
160
|
+
emoji sits at the cut; alert rules that matched the absence of a timeout line can match `Scan timed out`.
|
|
161
|
+
|
|
162
|
+
### Changed
|
|
163
|
+
|
|
164
|
+
- **Behaviour of 1.1.17 that its notes did not name.** Each is a change from 1.1.16; the `Apply` line says what to do.
|
|
165
|
+
- **A WHATWG `ReadableStream` body is refused (breaking).** 1.1.16 accepted it and stored it; 1.1.17 and later fail
|
|
166
|
+
with `STORAGE_VALIDATION_FAILED` (400, `details.field: 'body'`). **Apply:** convert it with
|
|
167
|
+
`Readable.fromWeb(stream)` before the call.
|
|
168
|
+
- **A `scanner` that is not an object is refused at boot.** `false`, `0`, `''` and `[]` booted in 1.1.16 and scanned
|
|
169
|
+
nothing; they fail `forRoot` with `STORAGE_INVALID_CONFIG` (`null` is refused too, see above). **Apply:**
|
|
170
|
+
omit `scanner` for no scanner.
|
|
171
|
+
- **`scanner.impl` must be an object with a `scan` function.** A function with a static `scan` (a class passed
|
|
172
|
+
where an instance belongs) is refused; 1.1.16 accepted it and called the static method. **Apply:** pass an
|
|
173
|
+
instance, or an object that holds the `scan` function.
|
|
174
|
+
- **Keys that are not options are dropped from the resolved `signedUrls`, `multipart`, `validation` and `scanner`.**
|
|
175
|
+
1.1.16 copied them through. **Apply:** keep such values outside the options.
|
|
176
|
+
- **`validation: null` resolves to an empty `validation`**; 1.1.16 kept the `null`. **Apply:** none; read
|
|
177
|
+
`resolved.validation` as an object, never as `null`.
|
|
178
|
+
- **A group that is a class instance is honoured.** A value served from a prototype getter (a config object built
|
|
179
|
+
as a class) is kept; 1.1.16 copied own properties only and dropped it silently.
|
|
180
|
+
- **`validation.maxSizeBytes: null` meant "no limit" in 1.1.17**; it is refused from this release (see above).
|
|
181
|
+
|
|
16
182
|
## [1.1.17] - 2026-11-17
|
|
17
183
|
|
|
18
184
|
### Fixed
|
|
@@ -1015,4 +1181,6 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
1015
1181
|
[1.1.15]: https://github.com/bymaxone/nest-storage/compare/v1.1.14...v1.1.15
|
|
1016
1182
|
[1.1.16]: https://github.com/bymaxone/nest-storage/compare/v1.1.15...v1.1.16
|
|
1017
1183
|
[1.1.17]: https://github.com/bymaxone/nest-storage/compare/v1.1.16...v1.1.17
|
|
1018
|
-
[
|
|
1184
|
+
[1.1.18]: https://github.com/bymaxone/nest-storage/compare/v1.1.17...v1.1.18
|
|
1185
|
+
[1.1.19]: https://github.com/bymaxone/nest-storage/compare/v1.1.18...v1.1.19
|
|
1186
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.19...HEAD
|
package/README.md
CHANGED
|
@@ -81,6 +81,9 @@ minimal.
|
|
|
81
81
|
provider-specific operations this surface does not wrap
|
|
82
82
|
- ✅ **Key prefix and server-side encryption** — `keyPrefix` is prepended after normalization and the
|
|
83
83
|
keys the library returns are relative to it; AES256 or `aws:kms`, globally or per upload
|
|
84
|
+
- ✅ **Metrics, readiness and spans** — an optional `./observability` subpath publishes Prometheus
|
|
85
|
+
metrics and a bucket readiness check; spans start when `@opentelemetry/api` is installed. Labels are
|
|
86
|
+
closed sets: no key, bucket or prefix is ever a label
|
|
84
87
|
- ✅ **Typed end to end** — TypeScript `strict` with `exactOptionalPropertyTypes` and
|
|
85
88
|
`noUncheckedIndexedAccess`; zero `any`
|
|
86
89
|
|
|
@@ -91,11 +94,12 @@ minimal.
|
|
|
91
94
|
| Subpath | Contents |
|
|
92
95
|
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
93
96
|
| `.` | Server runtime: `BymaxStorageModule`, `StorageService`, `SignedUrlService`, `BucketLifecycleService`, `providerRecipes`, DI tokens, interfaces, `StorageException`, `NoOpUploadValidator`, `NoOpFileScanner` |
|
|
97
|
+
| `./observability` | Optional: `StorageMetrics` (Prometheus metrics), `StorageHealthIndicator` (readiness of the bucket) |
|
|
94
98
|
| `./shared` | Framework-free types (`UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`, `StorageErrorResponse`) + `STORAGE_ERROR_CODES` + `StorageErrorCode` + the default MIME whitelists and the TTL and multipart default constants |
|
|
95
99
|
|
|
96
100
|
The root re-exports some `./shared` names for convenience; import the rest from the subpath.
|
|
97
101
|
|
|
98
|
-
|
|
102
|
+
All subpaths ship ESM **and** CommonJS with declarations for each format, so a
|
|
99
103
|
`require()` consumer receives CommonJS declarations rather than ESM ones. The
|
|
100
104
|
`pnpm check:exports` gate verifies this against the packed tarball.
|
|
101
105
|
|
|
@@ -104,6 +108,7 @@ Both subpaths ship ESM **and** CommonJS with declarations for each format, so a
|
|
|
104
108
|
| Subpath | Required peers |
|
|
105
109
|
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
106
110
|
| `.` (server) | `@nestjs/common ^11.0.16`, `@nestjs/core ^11.1.18`, `@aws-sdk/client-s3 ^3.700`, `@aws-sdk/lib-storage ^3.700`, `@aws-sdk/s3-request-presigner ^3.700`, `reflect-metadata ^0.2` |
|
|
111
|
+
| `./observability` | `@bymax-one/nest-core >=1.6.15` (to be discovered), `prom-client ^15`; all optional |
|
|
107
112
|
| `./shared` | None |
|
|
108
113
|
|
|
109
114
|
---
|
|
@@ -209,13 +214,13 @@ Options accepted by `forRoot` and `forRootAsync`:
|
|
|
209
214
|
| Option | Type | Default | Notes |
|
|
210
215
|
| ------------------------------ | ------------------------------------------------- | ----------------- | ---------------------------------------------- |
|
|
211
216
|
| `endpoint` | `string` | — | Required. S3-compatible endpoint URL; it must not carry credentials (`https://user:pass@host` is refused), pass them in `credentials` |
|
|
212
|
-
| `region` | `string` | — | Required. Provider region or `'auto'` (R2)
|
|
213
|
-
| `bucket` | `string` | — | Default bucket (can be overridden per-call)
|
|
217
|
+
| `region` | `string` | — | Required. Provider region or `'auto'` (R2): 1 to 63 letters, digits, `.`, `_` or `-`, starting with a letter or digit |
|
|
218
|
+
| `bucket` | `string` | — | Default bucket (can be overridden per-call by a string; any other per-call kind fails with `STORAGE_VALIDATION_FAILED`) |
|
|
214
219
|
| `credentials` | `{ accessKeyId, secretAccessKey, sessionToken? }` | — | Load from env / Secrets Manager |
|
|
215
220
|
| `forcePathStyle` | `boolean` | `false` | Set to `true` for MinIO and self-hosted |
|
|
216
221
|
| `publicBaseUrl` | `string` | — | Base URL for public `getPublicUrl()` results; it must not carry credentials (a user or password is refused) |
|
|
217
222
|
| `cdnBaseUrl` | `string` | — | CDN edge URL (preferred over `publicBaseUrl`); it must not carry credentials either |
|
|
218
|
-
| `keyPrefix` | `string` | — | Prepended to every key before it reaches the provider; fixed per module instance |
|
|
223
|
+
| `keyPrefix` | `string` | — | Prepended to every key before it reaches the provider; fixed per module instance. A value that is not a string fails `forRoot`; `null` is read as unset |
|
|
219
224
|
| `maxAttempts` | `number` | `3` | SDK v3 retry count; an integer of at least `1` |
|
|
220
225
|
| `serverSideEncryption` | `'AES256' \| 'aws:kms'` | — | Global SSE policy; `aws:kms` needs `kmsKeyId`. `'NONE'` is accepted per upload only |
|
|
221
226
|
| `requestChecksumCalculation` | `'WHEN_SUPPORTED' \| 'WHEN_REQUIRED'` | SDK default | Set to `'WHEN_REQUIRED'` for non-AWS providers |
|
|
@@ -232,10 +237,17 @@ Options accepted by `forRoot` and `forRootAsync`:
|
|
|
232
237
|
| `multipart.thresholdBytes` | `number` | `5 MiB` | Switch to multipart at or above this size |
|
|
233
238
|
| `multipart.partSizeBytes` | `number` | `5 MiB` | Part size (S3 minimum: 5 MiB) |
|
|
234
239
|
| `multipart.queueSize` | `number` | `4` | Concurrent parts |
|
|
235
|
-
| `
|
|
236
|
-
| `
|
|
240
|
+
| `scanner.impl` | `IFileScanner` | — | The scanner. Without it no scan runs; an object with a `scan` function, see Virus Scanning |
|
|
241
|
+
| `scanner.mode` | `'pre-upload' \| 'post-upload'` | `'pre-upload'` | When the scan runs: before the bytes are stored, or after, on the stored key |
|
|
242
|
+
| `scanner.rejectOnUnknown` | `boolean` | `false` | `true` rejects an inconclusive (`'unknown'`) scan; `false` lets it through with a warning |
|
|
243
|
+
| `scanner.timeoutMs` | `number` | `60000` | Deadline of one scan; a whole number from `1` to `2147483647`; past it the upload is refused with `STORAGE_TIMEOUT` |
|
|
244
|
+
| `validation.mimeWhitelist` | `string[]` | — | No MIME check when absent. Wildcards: `'image/*'`. A value that lists several types never matches. An array of strings: any other kind fails `forRoot`, and `null` is refused (omit it to allow every type) |
|
|
245
|
+
| `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes; counted as a stream is read, so it holds with or without `size`. `null` is refused: omit it for no limit |
|
|
246
|
+
| `validation.customValidators` | `IUploadValidator[]` | — | Run in order before the upload. An array of objects that each have a `validate` function: any other kind, or an entry that is not such an object, fails `forRoot`; `null` is refused (omit it to run none) |
|
|
237
247
|
|
|
238
|
-
**
|
|
248
|
+
**Option groups are objects.** `signedUrls`, `multipart` and `validation` are each an object, or left out (`null` is read as left out). `false`, `0`, a string, `true` or an array fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the group and repeating nothing of the value: ignoring it would boot without the protection that a toggle such as `validation: enabled && { ... }` was meant to switch on.
|
|
249
|
+
|
|
250
|
+
**Numeric options are whole numbers, checked at boot.** `validation.maxSizeBytes`, the `signedUrls` TTLs and `multipart.thresholdBytes` take a whole number of at least `1`; `multipart.partSizeBytes` takes one from 5 MiB (the S3 minimum) to 5 GiB (the S3 maximum) and `multipart.queueSize` one from `1` to `1000`. `NaN`, `Infinity`, a fraction, a string or a value out of range fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the option: a comparison alone lets `NaN` through, and `NaN` on a size limit would turn the limit off. The same holds per call: a `ttlSeconds` that is not a whole number above `0` fails with `STORAGE_SIGNED_URL_TTL_INVALID`, and a declared upload `size` or a list `maxKeys` that is not a whole number (at least `0`, and at least `1`) fails with `STORAGE_VALIDATION_FAILED` naming the field, before any request is made. A key set to `undefined` or `null` inside `signedUrls`, `multipart`, `validation` or `scanner` means the default, except `scanner.impl` and the `validation` options named below, which refuse `null`. The per-call values are refused on `null`, and the error for a TTL echoes no value but a finite number. Where leaving an option out means a limit or a protection is off, `null` is refused for it as well, so a config lookup that returns `null` for a missing key cannot switch it off: `validation.maxSizeBytes`, `validation.mimeWhitelist` and `validation.customValidators` (the last must be an array).
|
|
239
251
|
|
|
240
252
|
> **Do not use `maxRetries` or `signatureVersion`** — these are AWS SDK v2 options that do not exist in v3. The v3 SDK is SigV4-only. Use `maxAttempts` (default `3`) for retry configuration.
|
|
241
253
|
|
|
@@ -339,6 +351,10 @@ await storage.upload({
|
|
|
339
351
|
})
|
|
340
352
|
```
|
|
341
353
|
|
|
354
|
+
`onProgress` receives `{ loaded, total?, part? }`: the bytes sent so far, the total when it is known, and the
|
|
355
|
+
part number on a multipart upload. A single-shot upload reports once, when it ends; a multipart upload reports
|
|
356
|
+
as each part is sent.
|
|
357
|
+
|
|
342
358
|
Uploads are automatically routed to multipart when `size >= multipart.thresholdBytes` (default 5 MiB). A `Readable` without a `size` always uses multipart. Multipart aborts automatically on failure (`leavePartsOnError: false`).
|
|
343
359
|
|
|
344
360
|
**What `body` may be.** A `Buffer`, a `Uint8Array`, an `ArrayBuffer`, a string or a Node `Readable`. Anything else (a `Blob`, an array of chunks, a plain object) fails with `STORAGE_VALIDATION_FAILED` (`details.field: 'body'`) before any request; it used to reach the upload as if it were a stream and fail as `STORAGE_MULTIPART_ABORTED` (500).
|
|
@@ -424,7 +440,9 @@ A 304 is therefore thrown, not returned: catch `StorageException` and branch on
|
|
|
424
440
|
`abortSignal`. When it aborts, the call rejects `STORAGE_REQUEST_ABORTED` (HTTP 499) with
|
|
425
441
|
`details.reason` set to the **name** of the signal's reason (`AbortError`, `TimeoutError`), never its
|
|
426
442
|
message, so a host can tell its own timeout from a client disconnect. `exists` rejects too, rather
|
|
427
|
-
than answering `false`.
|
|
443
|
+
than answering `false`. A value that is not an `AbortSignal` fails with `STORAGE_VALIDATION_FAILED` naming
|
|
444
|
+
`abortSignal` and nothing of the value (`exists` answers `false` and logs a warning for it, as it does for
|
|
445
|
+
any failure that is not a missing object).
|
|
428
446
|
|
|
429
447
|
```typescript
|
|
430
448
|
const controller = new AbortController()
|
|
@@ -596,20 +614,25 @@ BymaxStorageModule.forRoot({
|
|
|
596
614
|
})
|
|
597
615
|
```
|
|
598
616
|
|
|
617
|
+
**What your scanner is given.** `scan(input)` receives `mode` (`'pre-upload'` or `'post-upload'`, the configured
|
|
618
|
+
`scanner.mode`), the `key` and `bucket` as the provider holds them, the `contentType`, the `size` when known, a
|
|
619
|
+
`signal` that aborts at `scanner.timeoutMs`, and a `body` (a `Buffer` or a stream) only when `mode` is
|
|
620
|
+
`'pre-upload'`. In `'post-upload'` mode the scanner reads the object itself.
|
|
621
|
+
|
|
599
622
|
**Streams in pre-upload mode.** A stream can be read once, so in `pre-upload` mode it is first read
|
|
600
623
|
into memory, bounded by `validation.maxSizeBytes`, and the scanner and the upload share that buffer. A `Buffer` or `Uint8Array` body is copied once in this mode, so a change the caller makes to its own array while the scan runs reaches neither (the second copy of the body is held until the upload ends, so the body takes twice its size in memory). The library validates and scans the bytes at call time; do not change a buffer you passed until the call settles. A
|
|
601
624
|
stream longer than the limit is refused with `STORAGE_VALIDATION_FAILED` as soon as it passes it, and
|
|
602
625
|
with no `maxSizeBytes` it is refused with a reason that says to set it or use `post-upload`.
|
|
603
626
|
`post-upload` hands the scanner the key only, no body.
|
|
604
627
|
|
|
605
|
-
**The scanner configuration is checked at boot.** `scanner.impl` must be an object with a `scan` function, `scanner.mode` must be `'pre-upload'` or `'post-upload'` (unset or `null` is `'pre-upload'`) and `scanner.rejectOnUnknown` must be a boolean (unset or `null` is the default); anything else fails `forRoot` with `STORAGE_INVALID_CONFIG`, because a missing implementation or a mode that is neither value would scan nothing.
|
|
628
|
+
**The scanner configuration is checked at boot.** `scanner.impl` must be an object with a `scan` function, `scanner.mode` must be `'pre-upload'` or `'post-upload'` (unset or `null` is `'pre-upload'`) and `scanner.rejectOnUnknown` must be a boolean (unset or `null` is the default); anything else fails `forRoot` with `STORAGE_INVALID_CONFIG`, because a missing implementation or a mode that is neither value would scan nothing. Omit `scanner` for no scanner: a `scanner` of `null` is refused, so a config lookup that returns `null` for a missing key cannot switch scanning off.
|
|
606
629
|
|
|
607
630
|
**A scan has a deadline.** `scanner.timeoutMs` (a whole number of milliseconds from `1` to `2147483647`, default `60000`; there is no setting that waits without limit) bounds a scan: past it the upload is refused with `STORAGE_TIMEOUT` (503, `details.phase: 'scan'`), and in `post-upload` mode the stored object is removed as for any other rejection. The scanner is handed an `AbortSignal` (`signal` on its input) that aborts at the deadline. **A scanner that ignores the signal keeps running after the timeout**, so repeated timeouts pile up scanner work: honour the signal, or give the scanner a limit of its own. A listener the scanner adds to the signal and that throws is reported by Node as an uncaught exception, as for any event target: it cannot be caught around the abort.
|
|
608
631
|
|
|
609
632
|
**What counts as a verdict.** Only the exact strings `'clean'`, `'infected'` and `'unknown'` are results. Anything else (no result, `null`, `'CLEAN'`, `' clean'`, an object, a made-up status) is rejected as `STORAGE_SCAN_INCONCLUSIVE` with `details.reason: 'unrecognised scanner status'`, whatever `rejectOnUnknown` says: an object the scanner did not call clean is not stored.
|
|
610
633
|
|
|
611
634
|
**What a client is told.** `STORAGE_SCAN_INFECTED` carries `details.engine` and `details.threat`, and
|
|
612
|
-
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut
|
|
635
|
+
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `STORAGE_SCAN_FAILED` event with the error under `err` (its message cut) so there is a trace of why; a scan that outlasts `scanner.timeoutMs` is logged as `STORAGE_SCAN_TIMEOUT` with the limit (see [Log events](#-log-events)). The free-form `details` your scanner returns is the
|
|
613
636
|
scanner's own text, so it is written to the warning log (escaped and truncated) and never to the
|
|
614
637
|
body of the error.
|
|
615
638
|
|
|
@@ -618,9 +641,10 @@ In `post-upload` mode your scanner is given the key and reads the object itself,
|
|
|
618
641
|
**A rejected object that could not be removed.** In `post-upload` mode the object is already stored
|
|
619
642
|
when the scanner rejects it, so the library deletes it and rejects `upload()` with the verdict
|
|
620
643
|
(`STORAGE_SCAN_INFECTED` or `STORAGE_SCAN_INCONCLUSIVE`). If that delete fails the verdict is still the
|
|
621
|
-
error, and `details` carries `cleanup: 'failed'`, the `key`
|
|
622
|
-
`
|
|
623
|
-
|
|
644
|
+
error, and `details` carries `cleanup: 'failed'`, the `key` without any `keyPrefix` and the code of the delete
|
|
645
|
+
failure in `cleanupError`. The log gets an error entry with `event: 'STORAGE_SCAN_CLEANUP_FAILED'` and the fields `message`, `storedKey`
|
|
646
|
+
(the key as the provider holds it, with any `keyPrefix`, as a bucket listing shows it, so an operator can find and remove the
|
|
647
|
+
object) and `cleanupFailure` (why the delete failed); the `storedKey` is never in the response. With no `details.cleanup` the object was
|
|
624
648
|
removed; with `cleanup: 'failed'` it is **still stored** and has to be removed by other means.
|
|
625
649
|
|
|
626
650
|
---
|
|
@@ -641,6 +665,18 @@ if (page1.nextContinuationToken) {
|
|
|
641
665
|
}
|
|
642
666
|
```
|
|
643
667
|
|
|
668
|
+
| Option | Meaning |
|
|
669
|
+
| --- | --- |
|
|
670
|
+
| `prefix` | Only keys that start with it, after `keyPrefix` |
|
|
671
|
+
| `maxKeys` | Page size, a whole number of at least `1`; above `1000` it is lowered to `1000` (the S3 maximum). Default `1000` |
|
|
672
|
+
| `continuationToken` | The `nextContinuationToken` of the previous page |
|
|
673
|
+
| `delimiter` | Groups keys under sub-prefixes: with `'/'`, objects below `avatars/2026/` are not listed, and `avatars/2026/` is returned in `commonPrefixes` instead |
|
|
674
|
+
| `bucket` | Lists another bucket for this call |
|
|
675
|
+
| `abortSignal` | Ends the call when it aborts, see Cancelling a call |
|
|
676
|
+
|
|
677
|
+
The result is `{ objects, commonPrefixes, isTruncated, nextContinuationToken? }`. Keys and prefixes come back
|
|
678
|
+
relative to `keyPrefix`, so they can be passed to any other method as they are.
|
|
679
|
+
|
|
644
680
|
### Copy (server-side)
|
|
645
681
|
|
|
646
682
|
```typescript
|
|
@@ -650,6 +686,10 @@ const { etag } = await storage.copy({
|
|
|
650
686
|
})
|
|
651
687
|
```
|
|
652
688
|
|
|
689
|
+
`sourceBucket` and `destinationBucket` default to the module's bucket and may each name another, so a copy can
|
|
690
|
+
cross buckets; both keys are relative to `keyPrefix`. `publicRead` and `cacheControl` set the ACL and the
|
|
691
|
+
`Cache-Control` of the copy.
|
|
692
|
+
|
|
653
693
|
### Delete / Batch Delete
|
|
654
694
|
|
|
655
695
|
```typescript
|
|
@@ -769,6 +809,8 @@ BymaxStorageModule.forRootAsync({
|
|
|
769
809
|
|
|
770
810
|
All errors are thrown as `StorageException extends HttpException`. The response body is `{ error: { code, message, details? } }`.
|
|
771
811
|
|
|
812
|
+
**What `details` repeats.** Only what the caller supplied in the request that failed: the `key` (without the module's `keyPrefix`), the per-call `bucket`, the `prefix` of a `list()`, the `sourceKey` and `destKey` of a `copy()`, and the `contentType` of a refused upload (left out when it is not a valid header value). The others are strings of at most 1024 characters (S3's limit for a key) and are left out when longer or not a string. What the host set as configuration is never repeated: the module's `bucket` when the call did not name one, the `keyPrefix`, the `validation.mimeWhitelist` (a `STORAGE_MIME_NOT_ALLOWED` carries the refused `contentType` and not the list) and the `region`.
|
|
813
|
+
|
|
772
814
|
| Code | HTTP | When |
|
|
773
815
|
| -------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
774
816
|
| `STORAGE_NOT_CONFIGURED` | 503 | Credentials missing and an operation was called |
|
|
@@ -777,14 +819,14 @@ All errors are thrown as `StorageException extends HttpException`. The response
|
|
|
777
819
|
| `STORAGE_CONTENT_TYPE_REQUIRED` | 400 | `upload()` called without `contentType` |
|
|
778
820
|
| `STORAGE_MIME_NOT_ALLOWED` | 415 | `contentType` outside `mimeWhitelist`, at upload or at presign |
|
|
779
821
|
| `STORAGE_SIZE_EXCEEDED` | 413 | `size > maxSizeBytes` |
|
|
780
|
-
| `STORAGE_VALIDATION_FAILED` | 400 | `IUploadValidator` rejected the file, or a header-bound value is not a header value (`details.field`, `details.reason
|
|
822
|
+
| `STORAGE_VALIDATION_FAILED` | 400 | `IUploadValidator` rejected the file, or a header-bound value is not a header value, or a per-call `bucket` or `abortSignal` is of the wrong kind (`details.field`, `details.reason`; the value is never repeated) |
|
|
781
823
|
| `STORAGE_SCAN_INFECTED` | 422 | Scanner returned `'infected'` (`details.threat`); `details.cleanup: 'failed'` means the object is still stored |
|
|
782
824
|
| `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true`, or a result that is not a recognised status |
|
|
783
825
|
| `STORAGE_OBJECT_NOT_FOUND` | 404 | `head()`, `download()`, or `copy()` on a nonexistent key |
|
|
784
|
-
| `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (
|
|
826
|
+
| `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (never the text of the driver or of the provider) plus the operation context (`op`, and the `bucket`, `key` or `prefix` the caller gave; see _What `details` repeats_) |
|
|
785
827
|
| `STORAGE_SIGNED_URL_TTL_INVALID` | 400 | Per-request `ttlSeconds` ≤ 0 |
|
|
786
828
|
| `STORAGE_PART_TOO_SMALL` | 400 | Never thrown: a `multipart.partSizeBytes` below 5 MiB fails at init as `STORAGE_INVALID_CONFIG` |
|
|
787
|
-
| `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts` not an integer from 1 to 10 000 (`details.
|
|
829
|
+
| `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts` not an integer from 1 to 10 000 (`details.reason`; `details.provided` only when `parts` is a finite number) |
|
|
788
830
|
| `STORAGE_BUCKET_UNDEFINED` | 400 | Operation without a bucket and no default in config |
|
|
789
831
|
| `STORAGE_MULTIPART_ABORTED` | 500 | A multipart upload failed (**503** when the cause is an outage, as for `STORAGE_PROVIDER_ERROR`). `details.abort` is `'completed'` when the library aborted it, `'failed'` when the abort itself failed (logged as a warning, with what it threw): that upload then stays open on the provider until a lifecycle rule removes it |
|
|
790
832
|
| `STORAGE_INVALID_CONFIG` | 500 | `BymaxStorageModuleOptions` validation failed at initialization |
|
|
@@ -814,6 +856,107 @@ try {
|
|
|
814
856
|
|
|
815
857
|
---
|
|
816
858
|
|
|
859
|
+
## 📊 Observability
|
|
860
|
+
|
|
861
|
+
The main entry counts and times every operation and starts spans. The packages below are
|
|
862
|
+
optional peers: install what you use.
|
|
863
|
+
|
|
864
|
+
```bash
|
|
865
|
+
pnpm add prom-client @bymax-one/nest-core @opentelemetry/api
|
|
866
|
+
```
|
|
867
|
+
|
|
868
|
+
### Metrics
|
|
869
|
+
|
|
870
|
+
```ts
|
|
871
|
+
import { Module } from '@nestjs/common'
|
|
872
|
+
import { BymaxCoreModule } from '@bymax-one/nest-core'
|
|
873
|
+
import { BymaxStorageModule } from '@bymax-one/nest-storage'
|
|
874
|
+
import { StorageHealthIndicator, StorageMetrics } from '@bymax-one/nest-storage/observability'
|
|
875
|
+
|
|
876
|
+
@Module({
|
|
877
|
+
imports: [
|
|
878
|
+
BymaxCoreModule.forRoot({ metrics: { enabled: true }, health: { autoDiscover: true } }),
|
|
879
|
+
BymaxStorageModule.forRoot({
|
|
880
|
+
endpoint: process.env.S3_ENDPOINT ?? '',
|
|
881
|
+
region: 'us-east-1',
|
|
882
|
+
bucket: 'uploads',
|
|
883
|
+
credentials: { accessKeyId: process.env.S3_KEY ?? '', secretAccessKey: process.env.S3_SECRET ?? '' },
|
|
884
|
+
}),
|
|
885
|
+
],
|
|
886
|
+
providers: [StorageMetrics, StorageHealthIndicator],
|
|
887
|
+
})
|
|
888
|
+
export class AppModule {}
|
|
889
|
+
```
|
|
890
|
+
|
|
891
|
+
`@bymax-one/nest-core` finds both classes by their markers and calls `registerMetrics` on `StorageMetrics` once at bootstrap.
|
|
892
|
+
|
|
893
|
+
| Metric | Type | Labels | What it counts |
|
|
894
|
+
| --- | --- | --- | --- |
|
|
895
|
+
| `bymax_storage_operations_total` | counter | `operation`, `outcome` | Operations that ended |
|
|
896
|
+
| `bymax_storage_operation_duration_seconds` | histogram | `operation` | How long they took |
|
|
897
|
+
| `bymax_storage_bytes_uploaded_total` | counter | none | Bytes sent by uploads that succeeded |
|
|
898
|
+
| `bymax_storage_scans_total` | counter | `verdict` | File scans |
|
|
899
|
+
|
|
900
|
+
The labels come from closed sets, so the number of series never grows with traffic and no key, bucket, prefix or
|
|
901
|
+
message is ever a label:
|
|
902
|
+
|
|
903
|
+
- `operation`: `upload`, `download`, `head`, `delete`, `delete_many`, `list`, `copy`, `get_download_url`,
|
|
904
|
+
`get_upload_url`, `get_multipart_upload_urls`, `check_bucket`, `lifecycle_get`, `lifecycle_upsert`,
|
|
905
|
+
`lifecycle_remove`.
|
|
906
|
+
- `outcome`: `success`, `not_found` (the object was absent), `aborted` (your `abortSignal`, or an aborted multipart
|
|
907
|
+
upload), `error`.
|
|
908
|
+
- `verdict`: `clean`, `infected`, `unknown`, `timeout`, `error`.
|
|
909
|
+
|
|
910
|
+
`exists()` is counted as `head`, `downloadBuffer()` as `download`, and the delete that removes an object a post-upload scan rejected as `delete`. A download is timed to the response, not to the
|
|
911
|
+
last byte. A result returned from the idempotency cache counts as an upload but adds no bytes. The bytes are the ones the transfer sent: a stream without a declared `size` is counted too, and a `size` the stream did not reach is not; a failed upload adds none. The counters start
|
|
912
|
+
with the module; the histogram starts when `StorageMetrics` registers.
|
|
913
|
+
|
|
914
|
+
### Readiness
|
|
915
|
+
|
|
916
|
+
`StorageHealthIndicator` reports under the name `storage`. It asks `StorageService.checkBucket()` for one
|
|
917
|
+
`HeadBucket` and answers `up` or `down` with no detail, so nothing about the bucket, the region or the endpoint
|
|
918
|
+
leaves with it. The probe has its own deadline, the answer is kept, and checks that arrive
|
|
919
|
+
while one is running share it. The probe is counted as `check_bucket`.
|
|
920
|
+
|
|
921
|
+
### Spans
|
|
922
|
+
|
|
923
|
+
Spans start through `@opentelemetry/api` when it is installed and a tracer is registered; without it the operation
|
|
924
|
+
runs as it is.
|
|
925
|
+
|
|
926
|
+
| Span | Around |
|
|
927
|
+
| --- | --- |
|
|
928
|
+
| `storage.upload` | `upload()` |
|
|
929
|
+
| `storage.download_url` | `getDownloadUrl()` |
|
|
930
|
+
| `storage.scan` | a file scan |
|
|
931
|
+
|
|
932
|
+
Attributes are `bymax.storage.outcome` (the outcome above), `bymax.storage.multipart` (`upload` only: `true` or
|
|
933
|
+
`false`) and `bymax.storage.scan_mode` (`scan` only: `pre-upload` or `post-upload`). A failed operation sets the
|
|
934
|
+
span status to error with the library error code, never its message.
|
|
935
|
+
|
|
936
|
+
---
|
|
937
|
+
|
|
938
|
+
## 📋 Log events
|
|
939
|
+
|
|
940
|
+
Every operational line the library writes is an **object** through the Nest `Logger`: an `event` code, a `message` with the library's own text, and the fields below. What the provider, the scanner or the host raised goes under `err`, as a fresh error built from its name, its provider code and its message (secrets masked, cut, control characters escaped); never into `message`, and never the SDK's own error, so no response, header or `cause` reaches a log. No field of an event carries the key you passed, the bucket, the endpoint or the region, except `storedKey` of `STORAGE_SCAN_CLEANUP_FAILED`, which an operator needs to remove the object. What the provider or the scanner wrote under `err` is their own text and can name the host, the key or the bucket. A host logger that throws does not change the call being logged. A host `LoggerService` method that is async and rejects is an unhandled rejection, and Node ends the process on it unless the application handles `unhandledRejection`: Nest's `Logger` drops the value a host method returns before the library can see it, so handle rejections inside your logger.
|
|
941
|
+
|
|
942
|
+
| Code | Level | When | Fields |
|
|
943
|
+
| --- | --- | --- | --- |
|
|
944
|
+
| `STORAGE_TTL_CLAMPED` | warn | `signedUrls.maxTtlSeconds` is above the SigV4 limit and the limit is used | `configuredSeconds`, `appliedSeconds` |
|
|
945
|
+
| `STORAGE_CREDENTIALS_MISSING` | warn | the module starts without credentials; operations then throw `STORAGE_NOT_CONFIGURED` | none |
|
|
946
|
+
| `STORAGE_CLIENT_READY` | log | the S3 client is created | none |
|
|
947
|
+
| `STORAGE_PROVIDER_CALL_FAILED` | warn | a call to the provider failed (a request you cancelled is not logged) | `operation`, `awsCode`, `httpStatus`, `requestId`, `reason`, `err` |
|
|
948
|
+
| `STORAGE_MULTIPART_ABORT_FAILED` | warn | aborting a failed multipart upload failed, so the upload stays open on the provider | `err` |
|
|
949
|
+
| `STORAGE_SCAN_INFECTED` | warn | the scanner said `infected` | `engine`, `threat`, `details` (the scanner's own, cut) |
|
|
950
|
+
| `STORAGE_SCAN_INCONCLUSIVE_ACCEPTED` | warn | the scanner said `unknown` and `rejectOnUnknown` is off | `engine` |
|
|
951
|
+
| `STORAGE_SCAN_INCONCLUSIVE_REJECTED` | warn | `unknown` with `rejectOnUnknown`, or a result outside the contract | `engine`, `status` (when unrecognised), `details` |
|
|
952
|
+
| `STORAGE_SCAN_FAILED` | warn | the scanner threw | `err` |
|
|
953
|
+
| `STORAGE_SCAN_TIMEOUT` | warn | the scan outlasted `scanner.timeoutMs` | `timeoutMs` |
|
|
954
|
+
| `STORAGE_SCAN_CLEANUP_FAILED` | error | the object a scan rejected could not be deleted and is still stored | `storedKey`, `cleanupFailure` |
|
|
955
|
+
| `STORAGE_EXISTS_FAILED` | warn | `exists()` met a failure that is not a 404 and answered `false` | `err` |
|
|
956
|
+
| `STORAGE_DELETE_NOT_FOUND` | warn | `delete()` found no such key (an idempotent no-op) | none |
|
|
957
|
+
|
|
958
|
+
---
|
|
959
|
+
|
|
817
960
|
## 🚫 What This Library Does NOT Do
|
|
818
961
|
|
|
819
962
|
- **Detect a file's real type.** The MIME check reads the declared `Content-Type` only; plug in an
|
|
@@ -909,12 +1052,13 @@ injected into every service, so `credentials` is attached as a non-enumerable ac
|
|
|
909
1052
|
is what a structured logger or an error reporter capturing a scope would otherwise pick up. Reading it
|
|
910
1053
|
on purpose is unchanged.
|
|
911
1054
|
|
|
912
|
-
### Error payloads carry the operation, and
|
|
1055
|
+
### Error payloads carry the operation, and the key you passed
|
|
913
1056
|
|
|
914
|
-
`details` holds the provider's error code
|
|
915
|
-
operation context the call site supplies: the operation
|
|
916
|
-
|
|
917
|
-
|
|
1057
|
+
`details` holds the provider's error code, the HTTP status, the request id, a fixed `reason`, and the
|
|
1058
|
+
operation context the call site supplies: the operation and the `key`, `bucket` or `prefix` the caller
|
|
1059
|
+
passed (see _What `details` repeats_). Neither the text of the provider nor anything the host configured is
|
|
1060
|
+
in it. An object key you passed still reaches whatever consumes the exception, so if a key is itself
|
|
1061
|
+
sensitive in your deployment, do not log the envelope verbatim.
|
|
918
1062
|
|
|
919
1063
|
### `keyPrefix` is a namespace, not an access boundary
|
|
920
1064
|
|
|
@@ -1047,6 +1191,13 @@ is the one full value: it is a URL, and the prefix is part of the object's real
|
|
|
1047
1191
|
`LifecycleBucketOptions` (`{ bucket? }`).
|
|
1048
1192
|
- **Extension points:** `IUploadValidator`, `IFileScanner`, `FileScanResult`, and the helpers
|
|
1049
1193
|
`NoOpUploadValidator` and `NoOpFileScanner`.
|
|
1194
|
+
- **Observability (main entry):** `StorageMetricsTally`, the counts the `./observability` subpath publishes, and the
|
|
1195
|
+
closed sets they are keyed by: `STORAGE_OPERATIONS` and `StorageOperation`, `STORAGE_OUTCOMES` and
|
|
1196
|
+
`StorageOutcome`, `SCAN_VERDICTS` and `ScanVerdict`.
|
|
1197
|
+
- **From `./observability`:** `StorageMetrics`, `StorageHealthIndicator`, the name it reports under
|
|
1198
|
+
(`STORAGE_HEALTH_INDICATOR_NAME`), its deadline and cache (`STORAGE_HEALTH_TIMEOUT_MS`,
|
|
1199
|
+
`STORAGE_HEALTH_CACHE_MS`), and the metadata keys `@bymax-one/nest-core` discovers them by
|
|
1200
|
+
(`METRICS_CONTRIBUTOR_METADATA`, `HEALTH_INDICATOR_METADATA`).
|
|
1050
1201
|
- **From `./shared`:** `UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`,
|
|
1051
1202
|
`StorageErrorResponse`, `StorageErrorCode`, `STORAGE_ERROR_CODES`, the MIME presets
|
|
1052
1203
|
`DEFAULT_IMAGE_MIME_WHITELIST`, `DEFAULT_VIDEO_MIME_WHITELIST`, `DEFAULT_DOC_MIME_WHITELIST`, and the
|