@bymax-one/nest-storage 1.1.18 → 1.1.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +76 -1
- package/README.md +160 -11
- package/dist/observability/index.cjs +151 -0
- package/dist/observability/index.d.cts +72 -0
- package/dist/observability/index.d.ts +72 -0
- package/dist/observability/index.mjs +145 -0
- package/dist/server/index.cjs +751 -421
- package/dist/server/index.d.cts +137 -11
- package/dist/server/index.d.ts +137 -11
- package/dist/server/index.mjs +751 -424
- package/package.json +32 -5
package/CHANGELOG.md
CHANGED
|
@@ -13,6 +13,80 @@ note on each entry is the compatibility contract — read it, not the number.
|
|
|
13
13
|
|
|
14
14
|
## [Unreleased]
|
|
15
15
|
|
|
16
|
+
## [1.1.19] - 2026-11-20
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **An optional `./observability` subpath and a span seam.** `StorageMetrics` publishes
|
|
21
|
+
`bymax_storage_operations_total`, `bymax_storage_operation_duration_seconds`, `bymax_storage_bytes_uploaded_total`
|
|
22
|
+
and `bymax_storage_scans_total` with labels from closed sets, and `StorageHealthIndicator` reports the bucket as
|
|
23
|
+
`storage` from one `HeadBucket`. The main entry now exports `StorageMetricsTally` and the closed sets
|
|
24
|
+
(`STORAGE_OPERATIONS`, `STORAGE_OUTCOMES`, `SCAN_VERDICTS`), counts every public operation, and starts the
|
|
25
|
+
`storage.upload`, `storage.download_url` and `storage.scan` spans when `@opentelemetry/api` is installed. `bymax_storage_bytes_uploaded_total` counts the bytes an upload sent, including a stream without a declared `size`. The new
|
|
26
|
+
peers `@bymax-one/nest-core >=1.6.15`, `prom-client ^15` and `@opentelemetry/api ^1.9` are optional.
|
|
27
|
+
|
|
28
|
+
**Apply to a derived backend:** nothing changes unless you use it. To publish the metrics, add `StorageMetrics` and
|
|
29
|
+
`StorageHealthIndicator` to the providers of a module and install `prom-client`; the README "Observability"
|
|
30
|
+
section has the table of metrics, labels and spans.
|
|
31
|
+
|
|
32
|
+
### Changed
|
|
33
|
+
|
|
34
|
+
- **The library's log lines are object events.** Every operational line is now an object with an `event` code
|
|
35
|
+
(`STORAGE_*`), a `message` with the library's own text and named fields, through the Nest `Logger`; the README has
|
|
36
|
+
a "Log events" table. What the provider, the scanner or the host raised goes under `err` as a fresh error (name,
|
|
37
|
+
provider code, masked and cut message), never into the text, and never the SDK's own error. The key you passed is
|
|
38
|
+
no longer logged by the provider-failure and scan events, the client-ready line no longer carries the endpoint or
|
|
39
|
+
the region, and a host logger that throws can no longer fail the call being logged. A host `LoggerService` method that is
|
|
40
|
+
async and rejects is still an unhandled rejection, which ends the Node process unless the application
|
|
41
|
+
handles `unhandledRejection`: Nest's `Logger` drops the value the method returns before the library can see it.
|
|
42
|
+
|
|
43
|
+
**Apply to a derived backend:** a log rule or an alert that matched the text of these lines (`Provider call failed`,
|
|
44
|
+
`Scan timed out`, `Scanner failed`, `Infected file detected`, `S3Client initialized`, the exists and delete
|
|
45
|
+
warnings, the TTL clamp) now matches the `event` code and reads the fields; the library text is kept in `message`.
|
|
46
|
+
A sink that stringifies the message sees an object: use a logger that writes object messages as fields. Anything
|
|
47
|
+
that relied on the key, the endpoint or the region in these lines reads them from the call or the configuration.
|
|
48
|
+
A quote or a backslash in a logged value is no longer escaped by the library, since the logger escapes it when it
|
|
49
|
+
serialises the field; control, bidirectional and format characters still show as escapes.
|
|
50
|
+
The `cleanupFailure` field of `STORAGE_SCAN_CLEANUP_FAILED` is the code of a `StorageException`; any other failure is masked, cut and escaped like `err`.
|
|
51
|
+
|
|
52
|
+
### Fixed
|
|
53
|
+
|
|
54
|
+
- **An option group of the wrong kind is refused at boot.** `validation`, `signedUrls` or `multipart` given as `false`,
|
|
55
|
+
`0`, a string, `true` or an array booted without error and was ignored, so `validation: enabled && { ... }` with
|
|
56
|
+
`enabled` false left uploads unvalidated. `forRoot` and `forRootAsync` now fail with `STORAGE_INVALID_CONFIG`
|
|
57
|
+
(`options.<group> must be an object`), repeating nothing of the value; `undefined` and `null` still mean "not
|
|
58
|
+
configured". `credentials` given as an array is refused as well; before, `forRoot` copied any array given for a
|
|
59
|
+
group into an empty group.
|
|
60
|
+
|
|
61
|
+
**Apply to a derived backend:** a configuration that passed one of those values for a group now fails at boot.
|
|
62
|
+
Pass an object, or leave the group out.
|
|
63
|
+
|
|
64
|
+
- **`validation.customValidators` entries are checked at boot.** An entry that is not an object with a `validate`
|
|
65
|
+
function (`42`, `null`, `{}`, a bare function) booted and then failed the first upload with a raw `TypeError`
|
|
66
|
+
(500). It now fails `forRoot` with `STORAGE_INVALID_CONFIG`. A `mimeWhitelist` that throws when it is read (a proxy,
|
|
67
|
+
a throwing index getter) is refused as "must be an array of strings" instead of surfacing the thrown error.
|
|
68
|
+
|
|
69
|
+
**Apply to a derived backend:** a list that held anything but validator objects now fails at boot; a class
|
|
70
|
+
instance that serves `validate` from its prototype is accepted. The README option table now has the
|
|
71
|
+
`validation.customValidators` row.
|
|
72
|
+
|
|
73
|
+
- **An error no longer repeats what the host configured.** `details` carried the module's `bucket` (when the call
|
|
74
|
+
named none), the `keyPrefix` (inside the `key` of an upload, head, download, delete, copy and signed-URL
|
|
75
|
+
failure, and the `prefix` of a list failure) and the whole `validation.mimeWhitelist` (in `STORAGE_MIME_NOT_ALLOWED`). The `key` is now the one the
|
|
76
|
+
caller passed, the `bucket` appears only when the caller passed one, and the whitelist is not repeated; a `key`,
|
|
77
|
+
`bucket`, `prefix`, `sourceKey` or `destKey` longer than 1024 characters is left out.
|
|
78
|
+
|
|
79
|
+
The same applies to a scan: a `STORAGE_TIMEOUT` for a scan that outlasts its deadline repeated the prefixed key and
|
|
80
|
+
the module bucket, and now carries the `key` the caller passed and a per-call `bucket`. `details.storedKey`,
|
|
81
|
+
which a rejected scan whose cleanup delete failed carried, is removed from the response: the operator gets the key as
|
|
82
|
+
stored in the `STORAGE_SCAN_CLEANUP_FAILED` log entry, in its `storedKey` field.
|
|
83
|
+
|
|
84
|
+
**Apply to a derived backend:** a client or a log rule that read `details.bucket` of a call that named no bucket,
|
|
85
|
+
`details.whitelist` of `STORAGE_MIME_NOT_ALLOWED`, `details.storedKey` of a rejected scan whose cleanup failed, or a
|
|
86
|
+
`keyPrefix` in `details.key` finds none, and reads the module configuration instead. A rule on the text of the
|
|
87
|
+
`STORAGE_SCAN_CLEANUP_FAILED` log message now reads the `event` and `storedKey` fields of the entry. A `key` and a
|
|
88
|
+
`bucket` that the caller passed are unchanged.
|
|
89
|
+
|
|
16
90
|
## [1.1.18] - 2026-11-18
|
|
17
91
|
|
|
18
92
|
### Fixed
|
|
@@ -1108,4 +1182,5 @@ have regressed from. They are kept because the reasoning is worth having.
|
|
|
1108
1182
|
[1.1.16]: https://github.com/bymaxone/nest-storage/compare/v1.1.15...v1.1.16
|
|
1109
1183
|
[1.1.17]: https://github.com/bymaxone/nest-storage/compare/v1.1.16...v1.1.17
|
|
1110
1184
|
[1.1.18]: https://github.com/bymaxone/nest-storage/compare/v1.1.17...v1.1.18
|
|
1111
|
-
[
|
|
1185
|
+
[1.1.19]: https://github.com/bymaxone/nest-storage/compare/v1.1.18...v1.1.19
|
|
1186
|
+
[Unreleased]: https://github.com/bymaxone/nest-storage/compare/v1.1.19...HEAD
|
package/README.md
CHANGED
|
@@ -81,6 +81,9 @@ minimal.
|
|
|
81
81
|
provider-specific operations this surface does not wrap
|
|
82
82
|
- ✅ **Key prefix and server-side encryption** — `keyPrefix` is prepended after normalization and the
|
|
83
83
|
keys the library returns are relative to it; AES256 or `aws:kms`, globally or per upload
|
|
84
|
+
- ✅ **Metrics, readiness and spans** — an optional `./observability` subpath publishes Prometheus
|
|
85
|
+
metrics and a bucket readiness check; spans start when `@opentelemetry/api` is installed. Labels are
|
|
86
|
+
closed sets: no key, bucket or prefix is ever a label
|
|
84
87
|
- ✅ **Typed end to end** — TypeScript `strict` with `exactOptionalPropertyTypes` and
|
|
85
88
|
`noUncheckedIndexedAccess`; zero `any`
|
|
86
89
|
|
|
@@ -91,11 +94,12 @@ minimal.
|
|
|
91
94
|
| Subpath | Contents |
|
|
92
95
|
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
93
96
|
| `.` | Server runtime: `BymaxStorageModule`, `StorageService`, `SignedUrlService`, `BucketLifecycleService`, `providerRecipes`, DI tokens, interfaces, `StorageException`, `NoOpUploadValidator`, `NoOpFileScanner` |
|
|
97
|
+
| `./observability` | Optional: `StorageMetrics` (Prometheus metrics), `StorageHealthIndicator` (readiness of the bucket) |
|
|
94
98
|
| `./shared` | Framework-free types (`UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`, `StorageErrorResponse`) + `STORAGE_ERROR_CODES` + `StorageErrorCode` + the default MIME whitelists and the TTL and multipart default constants |
|
|
95
99
|
|
|
96
100
|
The root re-exports some `./shared` names for convenience; import the rest from the subpath.
|
|
97
101
|
|
|
98
|
-
|
|
102
|
+
All subpaths ship ESM **and** CommonJS with declarations for each format, so a
|
|
99
103
|
`require()` consumer receives CommonJS declarations rather than ESM ones. The
|
|
100
104
|
`pnpm check:exports` gate verifies this against the packed tarball.
|
|
101
105
|
|
|
@@ -104,6 +108,7 @@ Both subpaths ship ESM **and** CommonJS with declarations for each format, so a
|
|
|
104
108
|
| Subpath | Required peers |
|
|
105
109
|
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
106
110
|
| `.` (server) | `@nestjs/common ^11.0.16`, `@nestjs/core ^11.1.18`, `@aws-sdk/client-s3 ^3.700`, `@aws-sdk/lib-storage ^3.700`, `@aws-sdk/s3-request-presigner ^3.700`, `reflect-metadata ^0.2` |
|
|
111
|
+
| `./observability` | `@bymax-one/nest-core >=1.6.15` (to be discovered), `prom-client ^15`; all optional |
|
|
107
112
|
| `./shared` | None |
|
|
108
113
|
|
|
109
114
|
---
|
|
@@ -232,8 +237,15 @@ Options accepted by `forRoot` and `forRootAsync`:
|
|
|
232
237
|
| `multipart.thresholdBytes` | `number` | `5 MiB` | Switch to multipart at or above this size |
|
|
233
238
|
| `multipart.partSizeBytes` | `number` | `5 MiB` | Part size (S3 minimum: 5 MiB) |
|
|
234
239
|
| `multipart.queueSize` | `number` | `4` | Concurrent parts |
|
|
240
|
+
| `scanner.impl` | `IFileScanner` | — | The scanner. Without it no scan runs; an object with a `scan` function, see Virus Scanning |
|
|
241
|
+
| `scanner.mode` | `'pre-upload' \| 'post-upload'` | `'pre-upload'` | When the scan runs: before the bytes are stored, or after, on the stored key |
|
|
242
|
+
| `scanner.rejectOnUnknown` | `boolean` | `false` | `true` rejects an inconclusive (`'unknown'`) scan; `false` lets it through with a warning |
|
|
243
|
+
| `scanner.timeoutMs` | `number` | `60000` | Deadline of one scan; a whole number from `1` to `2147483647`; past it the upload is refused with `STORAGE_TIMEOUT` |
|
|
235
244
|
| `validation.mimeWhitelist` | `string[]` | — | No MIME check when absent. Wildcards: `'image/*'`. A value that lists several types never matches. An array of strings: any other kind fails `forRoot`, and `null` is refused (omit it to allow every type) |
|
|
236
245
|
| `validation.maxSizeBytes` | `number` | — | Maximum upload size in bytes; counted as a stream is read, so it holds with or without `size`. `null` is refused: omit it for no limit |
|
|
246
|
+
| `validation.customValidators` | `IUploadValidator[]` | — | Run in order before the upload. An array of objects that each have a `validate` function: any other kind, or an entry that is not such an object, fails `forRoot`; `null` is refused (omit it to run none) |
|
|
247
|
+
|
|
248
|
+
**Option groups are objects.** `signedUrls`, `multipart` and `validation` are each an object, or left out (`null` is read as left out). `false`, `0`, a string, `true` or an array fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the group and repeating nothing of the value: ignoring it would boot without the protection that a toggle such as `validation: enabled && { ... }` was meant to switch on.
|
|
237
249
|
|
|
238
250
|
**Numeric options are whole numbers, checked at boot.** `validation.maxSizeBytes`, the `signedUrls` TTLs and `multipart.thresholdBytes` take a whole number of at least `1`; `multipart.partSizeBytes` takes one from 5 MiB (the S3 minimum) to 5 GiB (the S3 maximum) and `multipart.queueSize` one from `1` to `1000`. `NaN`, `Infinity`, a fraction, a string or a value out of range fails `forRoot` with `STORAGE_INVALID_CONFIG` naming the option: a comparison alone lets `NaN` through, and `NaN` on a size limit would turn the limit off. The same holds per call: a `ttlSeconds` that is not a whole number above `0` fails with `STORAGE_SIGNED_URL_TTL_INVALID`, and a declared upload `size` or a list `maxKeys` that is not a whole number (at least `0`, and at least `1`) fails with `STORAGE_VALIDATION_FAILED` naming the field, before any request is made. A key set to `undefined` or `null` inside `signedUrls`, `multipart`, `validation` or `scanner` means the default, except `scanner.impl` and the `validation` options named below, which refuse `null`. The per-call values are refused on `null`, and the error for a TTL echoes no value but a finite number. Where leaving an option out means a limit or a protection is off, `null` is refused for it as well, so a config lookup that returns `null` for a missing key cannot switch it off: `validation.maxSizeBytes`, `validation.mimeWhitelist` and `validation.customValidators` (the last must be an array).
|
|
239
251
|
|
|
@@ -339,6 +351,10 @@ await storage.upload({
|
|
|
339
351
|
})
|
|
340
352
|
```
|
|
341
353
|
|
|
354
|
+
`onProgress` receives `{ loaded, total?, part? }`: the bytes sent so far, the total when it is known, and the
|
|
355
|
+
part number on a multipart upload. A single-shot upload reports once, when it ends; a multipart upload reports
|
|
356
|
+
as each part is sent.
|
|
357
|
+
|
|
342
358
|
Uploads are automatically routed to multipart when `size >= multipart.thresholdBytes` (default 5 MiB). A `Readable` without a `size` always uses multipart. Multipart aborts automatically on failure (`leavePartsOnError: false`).
|
|
343
359
|
|
|
344
360
|
**What `body` may be.** A `Buffer`, a `Uint8Array`, an `ArrayBuffer`, a string or a Node `Readable`. Anything else (a `Blob`, an array of chunks, a plain object) fails with `STORAGE_VALIDATION_FAILED` (`details.field: 'body'`) before any request; it used to reach the upload as if it were a stream and fail as `STORAGE_MULTIPART_ABORTED` (500).
|
|
@@ -598,6 +614,11 @@ BymaxStorageModule.forRoot({
|
|
|
598
614
|
})
|
|
599
615
|
```
|
|
600
616
|
|
|
617
|
+
**What your scanner is given.** `scan(input)` receives `mode` (`'pre-upload'` or `'post-upload'`, the configured
|
|
618
|
+
`scanner.mode`), the `key` and `bucket` as the provider holds them, the `contentType`, the `size` when known, a
|
|
619
|
+
`signal` that aborts at `scanner.timeoutMs`, and a `body` (a `Buffer` or a stream) only when `mode` is
|
|
620
|
+
`'pre-upload'`. In `'post-upload'` mode the scanner reads the object itself.
|
|
621
|
+
|
|
601
622
|
**Streams in pre-upload mode.** A stream can be read once, so in `pre-upload` mode it is first read
|
|
602
623
|
into memory, bounded by `validation.maxSizeBytes`, and the scanner and the upload share that buffer. A `Buffer` or `Uint8Array` body is copied once in this mode, so a change the caller makes to its own array while the scan runs reaches neither (the second copy of the body is held until the upload ends, so the body takes twice its size in memory). The library validates and scans the bytes at call time; do not change a buffer you passed until the call settles. A
|
|
603
624
|
stream longer than the limit is refused with `STORAGE_VALIDATION_FAILED` as soon as it passes it, and
|
|
@@ -611,7 +632,7 @@ with no `maxSizeBytes` it is refused with a reason that says to set it or use `p
|
|
|
611
632
|
**What counts as a verdict.** Only the exact strings `'clean'`, `'infected'` and `'unknown'` are results. Anything else (no result, `null`, `'CLEAN'`, `' clean'`, an object, a made-up status) is rejected as `STORAGE_SCAN_INCONCLUSIVE` with `details.reason: 'unrecognised scanner status'`, whatever `rejectOnUnknown` says: an object the scanner did not call clean is not stored.
|
|
612
633
|
|
|
613
634
|
**What a client is told.** `STORAGE_SCAN_INFECTED` carries `details.engine` and `details.threat`, and
|
|
614
|
-
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `
|
|
635
|
+
`STORAGE_SCAN_INCONCLUSIVE` carries `details.engine`; each name is cut, in the body and in the log, and left out when the scanner gave something that is not a string. A scanner that throws is answered with a 500, and the library logs a `STORAGE_SCAN_FAILED` event with the error under `err` (its message cut) so there is a trace of why; a scan that outlasts `scanner.timeoutMs` is logged as `STORAGE_SCAN_TIMEOUT` with the limit (see [Log events](#-log-events)). The free-form `details` your scanner returns is the
|
|
615
636
|
scanner's own text, so it is written to the warning log (escaped and truncated) and never to the
|
|
616
637
|
body of the error.
|
|
617
638
|
|
|
@@ -620,9 +641,10 @@ In `post-upload` mode your scanner is given the key and reads the object itself,
|
|
|
620
641
|
**A rejected object that could not be removed.** In `post-upload` mode the object is already stored
|
|
621
642
|
when the scanner rejects it, so the library deletes it and rejects `upload()` with the verdict
|
|
622
643
|
(`STORAGE_SCAN_INFECTED` or `STORAGE_SCAN_INCONCLUSIVE`). If that delete fails the verdict is still the
|
|
623
|
-
error, and `details` carries `cleanup: 'failed'`, the `key`
|
|
624
|
-
`
|
|
625
|
-
|
|
644
|
+
error, and `details` carries `cleanup: 'failed'`, the `key` without any `keyPrefix` and the code of the delete
|
|
645
|
+
failure in `cleanupError`. The log gets an error entry with `event: 'STORAGE_SCAN_CLEANUP_FAILED'` and the fields `message`, `storedKey`
|
|
646
|
+
(the key as the provider holds it, with any `keyPrefix`, as a bucket listing shows it, so an operator can find and remove the
|
|
647
|
+
object) and `cleanupFailure` (why the delete failed); the `storedKey` is never in the response. With no `details.cleanup` the object was
|
|
626
648
|
removed; with `cleanup: 'failed'` it is **still stored** and has to be removed by other means.
|
|
627
649
|
|
|
628
650
|
---
|
|
@@ -643,6 +665,18 @@ if (page1.nextContinuationToken) {
|
|
|
643
665
|
}
|
|
644
666
|
```
|
|
645
667
|
|
|
668
|
+
| Option | Meaning |
|
|
669
|
+
| --- | --- |
|
|
670
|
+
| `prefix` | Only keys that start with it, after `keyPrefix` |
|
|
671
|
+
| `maxKeys` | Page size, a whole number of at least `1`; above `1000` it is lowered to `1000` (the S3 maximum). Default `1000` |
|
|
672
|
+
| `continuationToken` | The `nextContinuationToken` of the previous page |
|
|
673
|
+
| `delimiter` | Groups keys under sub-prefixes: with `'/'`, objects below `avatars/2026/` are not listed, and `avatars/2026/` is returned in `commonPrefixes` instead |
|
|
674
|
+
| `bucket` | Lists another bucket for this call |
|
|
675
|
+
| `abortSignal` | Ends the call when it aborts, see Cancelling a call |
|
|
676
|
+
|
|
677
|
+
The result is `{ objects, commonPrefixes, isTruncated, nextContinuationToken? }`. Keys and prefixes come back
|
|
678
|
+
relative to `keyPrefix`, so they can be passed to any other method as they are.
|
|
679
|
+
|
|
646
680
|
### Copy (server-side)
|
|
647
681
|
|
|
648
682
|
```typescript
|
|
@@ -652,6 +686,10 @@ const { etag } = await storage.copy({
|
|
|
652
686
|
})
|
|
653
687
|
```
|
|
654
688
|
|
|
689
|
+
`sourceBucket` and `destinationBucket` default to the module's bucket and may each name another, so a copy can
|
|
690
|
+
cross buckets; both keys are relative to `keyPrefix`. `publicRead` and `cacheControl` set the ACL and the
|
|
691
|
+
`Cache-Control` of the copy.
|
|
692
|
+
|
|
655
693
|
### Delete / Batch Delete
|
|
656
694
|
|
|
657
695
|
```typescript
|
|
@@ -771,6 +809,8 @@ BymaxStorageModule.forRootAsync({
|
|
|
771
809
|
|
|
772
810
|
All errors are thrown as `StorageException extends HttpException`. The response body is `{ error: { code, message, details? } }`.
|
|
773
811
|
|
|
812
|
+
**What `details` repeats.** Only what the caller supplied in the request that failed: the `key` (without the module's `keyPrefix`), the per-call `bucket`, the `prefix` of a `list()`, the `sourceKey` and `destKey` of a `copy()`, and the `contentType` of a refused upload (left out when it is not a valid header value). The others are strings of at most 1024 characters (S3's limit for a key) and are left out when longer or not a string. What the host set as configuration is never repeated: the module's `bucket` when the call did not name one, the `keyPrefix`, the `validation.mimeWhitelist` (a `STORAGE_MIME_NOT_ALLOWED` carries the refused `contentType` and not the list) and the `region`.
|
|
813
|
+
|
|
774
814
|
| Code | HTTP | When |
|
|
775
815
|
| -------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
776
816
|
| `STORAGE_NOT_CONFIGURED` | 503 | Credentials missing and an operation was called |
|
|
@@ -783,7 +823,7 @@ All errors are thrown as `StorageException extends HttpException`. The response
|
|
|
783
823
|
| `STORAGE_SCAN_INFECTED` | 422 | Scanner returned `'infected'` (`details.threat`); `details.cleanup: 'failed'` means the object is still stored |
|
|
784
824
|
| `STORAGE_SCAN_INCONCLUSIVE` | 422 | Scanner returned `'unknown'` and `rejectOnUnknown: true`, or a result that is not a recognised status |
|
|
785
825
|
| `STORAGE_OBJECT_NOT_FOUND` | 404 | `head()`, `download()`, or `copy()` on a nonexistent key |
|
|
786
|
-
| `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (
|
|
826
|
+
| `STORAGE_PROVIDER_ERROR` | 502 | The provider rejected the request (`AccessControlListNotSupported`, an `AccessDenied` on an object, any other rejection). **503** when it could not be reached (connection refused or reset, DNS failure) or answered with a server error (`SlowDown`, 500, 502, 504; not 501), which is worth retrying; **500** when it refused the credentials (`InvalidAccessKeyId`, `SignatureDoesNotMatch`), a deployment fault that a retry does not fix (a `HEAD` answer carries no body and so no code: a refused `head()` stays **502** with reason `rejected`, and the credentials case is not told apart there). `details` carries `awsCode`, `httpStatus`, `requestId` and a fixed `reason` (never the text of the driver or of the provider) plus the operation context (`op`, and the `bucket`, `key` or `prefix` the caller gave; see _What `details` repeats_) |
|
|
787
827
|
| `STORAGE_SIGNED_URL_TTL_INVALID` | 400 | Per-request `ttlSeconds` ≤ 0 |
|
|
788
828
|
| `STORAGE_PART_TOO_SMALL` | 400 | Never thrown: a `multipart.partSizeBytes` below 5 MiB fails at init as `STORAGE_INVALID_CONFIG` |
|
|
789
829
|
| `STORAGE_INVALID_PART_COUNT` | 400 | `getMultipartUploadUrls()` with `parts` not an integer from 1 to 10 000 (`details.reason`; `details.provided` only when `parts` is a finite number) |
|
|
@@ -816,6 +856,107 @@ try {
|
|
|
816
856
|
|
|
817
857
|
---
|
|
818
858
|
|
|
859
|
+
## 📊 Observability
|
|
860
|
+
|
|
861
|
+
The main entry counts and times every operation and starts spans. The packages below are
|
|
862
|
+
optional peers: install what you use.
|
|
863
|
+
|
|
864
|
+
```bash
|
|
865
|
+
pnpm add prom-client @bymax-one/nest-core @opentelemetry/api
|
|
866
|
+
```
|
|
867
|
+
|
|
868
|
+
### Metrics
|
|
869
|
+
|
|
870
|
+
```ts
|
|
871
|
+
import { Module } from '@nestjs/common'
|
|
872
|
+
import { BymaxCoreModule } from '@bymax-one/nest-core'
|
|
873
|
+
import { BymaxStorageModule } from '@bymax-one/nest-storage'
|
|
874
|
+
import { StorageHealthIndicator, StorageMetrics } from '@bymax-one/nest-storage/observability'
|
|
875
|
+
|
|
876
|
+
@Module({
|
|
877
|
+
imports: [
|
|
878
|
+
BymaxCoreModule.forRoot({ metrics: { enabled: true }, health: { autoDiscover: true } }),
|
|
879
|
+
BymaxStorageModule.forRoot({
|
|
880
|
+
endpoint: process.env.S3_ENDPOINT ?? '',
|
|
881
|
+
region: 'us-east-1',
|
|
882
|
+
bucket: 'uploads',
|
|
883
|
+
credentials: { accessKeyId: process.env.S3_KEY ?? '', secretAccessKey: process.env.S3_SECRET ?? '' },
|
|
884
|
+
}),
|
|
885
|
+
],
|
|
886
|
+
providers: [StorageMetrics, StorageHealthIndicator],
|
|
887
|
+
})
|
|
888
|
+
export class AppModule {}
|
|
889
|
+
```
|
|
890
|
+
|
|
891
|
+
`@bymax-one/nest-core` finds both classes by their markers and calls `registerMetrics` on `StorageMetrics` once at bootstrap.
|
|
892
|
+
|
|
893
|
+
| Metric | Type | Labels | What it counts |
|
|
894
|
+
| --- | --- | --- | --- |
|
|
895
|
+
| `bymax_storage_operations_total` | counter | `operation`, `outcome` | Operations that ended |
|
|
896
|
+
| `bymax_storage_operation_duration_seconds` | histogram | `operation` | How long they took |
|
|
897
|
+
| `bymax_storage_bytes_uploaded_total` | counter | none | Bytes sent by uploads that succeeded |
|
|
898
|
+
| `bymax_storage_scans_total` | counter | `verdict` | File scans |
|
|
899
|
+
|
|
900
|
+
The labels come from closed sets, so the number of series never grows with traffic and no key, bucket, prefix or
|
|
901
|
+
message is ever a label:
|
|
902
|
+
|
|
903
|
+
- `operation`: `upload`, `download`, `head`, `delete`, `delete_many`, `list`, `copy`, `get_download_url`,
|
|
904
|
+
`get_upload_url`, `get_multipart_upload_urls`, `check_bucket`, `lifecycle_get`, `lifecycle_upsert`,
|
|
905
|
+
`lifecycle_remove`.
|
|
906
|
+
- `outcome`: `success`, `not_found` (the object was absent), `aborted` (your `abortSignal`, or an aborted multipart
|
|
907
|
+
upload), `error`.
|
|
908
|
+
- `verdict`: `clean`, `infected`, `unknown`, `timeout`, `error`.
|
|
909
|
+
|
|
910
|
+
`exists()` is counted as `head`, `downloadBuffer()` as `download`, and the delete that removes an object a post-upload scan rejected as `delete`. A download is timed to the response, not to the
|
|
911
|
+
last byte. A result returned from the idempotency cache counts as an upload but adds no bytes. The bytes are the ones the transfer sent: a stream without a declared `size` is counted too, and a `size` the stream did not reach is not; a failed upload adds none. The counters start
|
|
912
|
+
with the module; the histogram starts when `StorageMetrics` registers.
|
|
913
|
+
|
|
914
|
+
### Readiness
|
|
915
|
+
|
|
916
|
+
`StorageHealthIndicator` reports under the name `storage`. It asks `StorageService.checkBucket()` for one
|
|
917
|
+
`HeadBucket` and answers `up` or `down` with no detail, so nothing about the bucket, the region or the endpoint
|
|
918
|
+
leaves with it. The probe has its own deadline, the answer is kept, and checks that arrive
|
|
919
|
+
while one is running share it. The probe is counted as `check_bucket`.
|
|
920
|
+
|
|
921
|
+
### Spans
|
|
922
|
+
|
|
923
|
+
Spans start through `@opentelemetry/api` when it is installed and a tracer is registered; without it the operation
|
|
924
|
+
runs as it is.
|
|
925
|
+
|
|
926
|
+
| Span | Around |
|
|
927
|
+
| --- | --- |
|
|
928
|
+
| `storage.upload` | `upload()` |
|
|
929
|
+
| `storage.download_url` | `getDownloadUrl()` |
|
|
930
|
+
| `storage.scan` | a file scan |
|
|
931
|
+
|
|
932
|
+
Attributes are `bymax.storage.outcome` (the outcome above), `bymax.storage.multipart` (`upload` only: `true` or
|
|
933
|
+
`false`) and `bymax.storage.scan_mode` (`scan` only: `pre-upload` or `post-upload`). A failed operation sets the
|
|
934
|
+
span status to error with the library error code, never its message.
|
|
935
|
+
|
|
936
|
+
---
|
|
937
|
+
|
|
938
|
+
## 📋 Log events
|
|
939
|
+
|
|
940
|
+
Every operational line the library writes is an **object** through the Nest `Logger`: an `event` code, a `message` with the library's own text, and the fields below. What the provider, the scanner or the host raised goes under `err`, as a fresh error built from its name, its provider code and its message (secrets masked, cut, control characters escaped); never into `message`, and never the SDK's own error, so no response, header or `cause` reaches a log. No field of an event carries the key you passed, the bucket, the endpoint or the region, except `storedKey` of `STORAGE_SCAN_CLEANUP_FAILED`, which an operator needs to remove the object. What the provider or the scanner wrote under `err` is their own text and can name the host, the key or the bucket. A host logger that throws does not change the call being logged. A host `LoggerService` method that is async and rejects is an unhandled rejection, and Node ends the process on it unless the application handles `unhandledRejection`: Nest's `Logger` drops the value a host method returns before the library can see it, so handle rejections inside your logger.
|
|
941
|
+
|
|
942
|
+
| Code | Level | When | Fields |
|
|
943
|
+
| --- | --- | --- | --- |
|
|
944
|
+
| `STORAGE_TTL_CLAMPED` | warn | `signedUrls.maxTtlSeconds` is above the SigV4 limit and the limit is used | `configuredSeconds`, `appliedSeconds` |
|
|
945
|
+
| `STORAGE_CREDENTIALS_MISSING` | warn | the module starts without credentials; operations then throw `STORAGE_NOT_CONFIGURED` | none |
|
|
946
|
+
| `STORAGE_CLIENT_READY` | log | the S3 client is created | none |
|
|
947
|
+
| `STORAGE_PROVIDER_CALL_FAILED` | warn | a call to the provider failed (a request you cancelled is not logged) | `operation`, `awsCode`, `httpStatus`, `requestId`, `reason`, `err` |
|
|
948
|
+
| `STORAGE_MULTIPART_ABORT_FAILED` | warn | aborting a failed multipart upload failed, so the upload stays open on the provider | `err` |
|
|
949
|
+
| `STORAGE_SCAN_INFECTED` | warn | the scanner said `infected` | `engine`, `threat`, `details` (the scanner's own, cut) |
|
|
950
|
+
| `STORAGE_SCAN_INCONCLUSIVE_ACCEPTED` | warn | the scanner said `unknown` and `rejectOnUnknown` is off | `engine` |
|
|
951
|
+
| `STORAGE_SCAN_INCONCLUSIVE_REJECTED` | warn | `unknown` with `rejectOnUnknown`, or a result outside the contract | `engine`, `status` (when unrecognised), `details` |
|
|
952
|
+
| `STORAGE_SCAN_FAILED` | warn | the scanner threw | `err` |
|
|
953
|
+
| `STORAGE_SCAN_TIMEOUT` | warn | the scan outlasted `scanner.timeoutMs` | `timeoutMs` |
|
|
954
|
+
| `STORAGE_SCAN_CLEANUP_FAILED` | error | the object a scan rejected could not be deleted and is still stored | `storedKey`, `cleanupFailure` |
|
|
955
|
+
| `STORAGE_EXISTS_FAILED` | warn | `exists()` met a failure that is not a 404 and answered `false` | `err` |
|
|
956
|
+
| `STORAGE_DELETE_NOT_FOUND` | warn | `delete()` found no such key (an idempotent no-op) | none |
|
|
957
|
+
|
|
958
|
+
---
|
|
959
|
+
|
|
819
960
|
## 🚫 What This Library Does NOT Do
|
|
820
961
|
|
|
821
962
|
- **Detect a file's real type.** The MIME check reads the declared `Content-Type` only; plug in an
|
|
@@ -911,12 +1052,13 @@ injected into every service, so `credentials` is attached as a non-enumerable ac
|
|
|
911
1052
|
is what a structured logger or an error reporter capturing a scope would otherwise pick up. Reading it
|
|
912
1053
|
on purpose is unchanged.
|
|
913
1054
|
|
|
914
|
-
### Error payloads carry the operation, and
|
|
1055
|
+
### Error payloads carry the operation, and the key you passed
|
|
915
1056
|
|
|
916
|
-
`details` holds the provider's error code
|
|
917
|
-
operation context the call site supplies: the operation
|
|
918
|
-
|
|
919
|
-
|
|
1057
|
+
`details` holds the provider's error code, the HTTP status, the request id, a fixed `reason`, and the
|
|
1058
|
+
operation context the call site supplies: the operation and the `key`, `bucket` or `prefix` the caller
|
|
1059
|
+
passed (see _What `details` repeats_). Neither the text of the provider nor anything the host configured is
|
|
1060
|
+
in it. An object key you passed still reaches whatever consumes the exception, so if a key is itself
|
|
1061
|
+
sensitive in your deployment, do not log the envelope verbatim.
|
|
920
1062
|
|
|
921
1063
|
### `keyPrefix` is a namespace, not an access boundary
|
|
922
1064
|
|
|
@@ -1049,6 +1191,13 @@ is the one full value: it is a URL, and the prefix is part of the object's real
|
|
|
1049
1191
|
`LifecycleBucketOptions` (`{ bucket? }`).
|
|
1050
1192
|
- **Extension points:** `IUploadValidator`, `IFileScanner`, `FileScanResult`, and the helpers
|
|
1051
1193
|
`NoOpUploadValidator` and `NoOpFileScanner`.
|
|
1194
|
+
- **Observability (main entry):** `StorageMetricsTally`, the counts the `./observability` subpath publishes, and the
|
|
1195
|
+
closed sets they are keyed by: `STORAGE_OPERATIONS` and `StorageOperation`, `STORAGE_OUTCOMES` and
|
|
1196
|
+
`StorageOutcome`, `SCAN_VERDICTS` and `ScanVerdict`.
|
|
1197
|
+
- **From `./observability`:** `StorageMetrics`, `StorageHealthIndicator`, the name it reports under
|
|
1198
|
+
(`STORAGE_HEALTH_INDICATOR_NAME`), its deadline and cache (`STORAGE_HEALTH_TIMEOUT_MS`,
|
|
1199
|
+
`STORAGE_HEALTH_CACHE_MS`), and the metadata keys `@bymax-one/nest-core` discovers them by
|
|
1200
|
+
(`METRICS_CONTRIBUTOR_METADATA`, `HEALTH_INDICATOR_METADATA`).
|
|
1052
1201
|
- **From `./shared`:** `UploadResult`, `ObjectMetadata`, `ListedObject`, `SignedUrlResult`,
|
|
1053
1202
|
`StorageErrorResponse`, `StorageErrorCode`, `STORAGE_ERROR_CODES`, the MIME presets
|
|
1054
1203
|
`DEFAULT_IMAGE_MIME_WHITELIST`, `DEFAULT_VIDEO_MIME_WHITELIST`, `DEFAULT_DOC_MIME_WHITELIST`, and the
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
var common = require('@nestjs/common');
|
|
4
|
+
var promClient = require('prom-client');
|
|
5
|
+
var nestStorage = require('@bymax-one/nest-storage');
|
|
6
|
+
|
|
7
|
+
var __defProp = Object.defineProperty;
|
|
8
|
+
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
|
|
9
|
+
var __typeError = (msg) => {
|
|
10
|
+
throw TypeError(msg);
|
|
11
|
+
};
|
|
12
|
+
var __defNormalProp = (obj, key, value) => key in obj ? __defProp(obj, key, { enumerable: true, configurable: true, writable: true, value }) : obj[key] = value;
|
|
13
|
+
var __decorateClass = (decorators, target, key, kind) => {
|
|
14
|
+
var result = kind > 1 ? void 0 : kind ? __getOwnPropDesc(target, key) : target;
|
|
15
|
+
for (var i = decorators.length - 1, decorator; i >= 0; i--)
|
|
16
|
+
if (decorator = decorators[i])
|
|
17
|
+
result = (decorator(result)) || result;
|
|
18
|
+
return result;
|
|
19
|
+
};
|
|
20
|
+
var __decorateParam = (index, decorator) => (target, key) => decorator(target, key, index);
|
|
21
|
+
var __publicField = (obj, key, value) => __defNormalProp(obj, key + "" , value);
|
|
22
|
+
var __accessCheck = (obj, member, msg) => member.has(obj) || __typeError("Cannot " + msg);
|
|
23
|
+
var __privateGet = (obj, member, getter) => (__accessCheck(obj, member, "read from private field"), getter ? getter.call(obj) : member.get(obj));
|
|
24
|
+
var __privateAdd = (obj, member, value) => member.has(obj) ? __typeError("Cannot add the same private member more than once") : member instanceof WeakSet ? member.add(obj) : member.set(obj, value);
|
|
25
|
+
var __privateSet = (obj, member, value, setter) => (__accessCheck(obj, member, "write to private field"), member.set(obj, value), value);
|
|
26
|
+
var METRICS_CONTRIBUTOR_METADATA = "bymax-one:metrics-contributor";
|
|
27
|
+
exports.StorageMetrics = class StorageMetrics {
|
|
28
|
+
constructor(tally) {
|
|
29
|
+
this.tally = tally;
|
|
30
|
+
}
|
|
31
|
+
tally;
|
|
32
|
+
/** Registers the collectors; called once at bootstrap by the metrics module. */
|
|
33
|
+
registerMetrics(registry) {
|
|
34
|
+
const { tally } = this;
|
|
35
|
+
new promClient.Counter({
|
|
36
|
+
name: "bymax_storage_operations_total",
|
|
37
|
+
help: "Storage operations by operation and outcome (success, not_found, aborted, error).",
|
|
38
|
+
labelNames: ["operation", "outcome"],
|
|
39
|
+
registers: [registry],
|
|
40
|
+
collect() {
|
|
41
|
+
this.reset();
|
|
42
|
+
const { operations } = tally.snapshot();
|
|
43
|
+
for (const operation of nestStorage.STORAGE_OPERATIONS) {
|
|
44
|
+
for (const outcome of nestStorage.STORAGE_OUTCOMES) {
|
|
45
|
+
this.inc({ operation, outcome }, operations[operation][outcome]);
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
});
|
|
50
|
+
const duration = new promClient.Histogram({
|
|
51
|
+
name: "bymax_storage_operation_duration_seconds",
|
|
52
|
+
help: "Time a storage operation took, by operation: to the response for a download, to the end otherwise.",
|
|
53
|
+
labelNames: ["operation"],
|
|
54
|
+
registers: [registry]
|
|
55
|
+
});
|
|
56
|
+
for (const operation of nestStorage.STORAGE_OPERATIONS) duration.zero({ operation });
|
|
57
|
+
tally.onDuration((operation, seconds) => {
|
|
58
|
+
duration.observe({ operation }, seconds);
|
|
59
|
+
});
|
|
60
|
+
new promClient.Counter({
|
|
61
|
+
name: "bymax_storage_bytes_uploaded_total",
|
|
62
|
+
help: "Bytes of objects uploaded, counted when the upload succeeded and not for a result the idempotency cache returned.",
|
|
63
|
+
registers: [registry],
|
|
64
|
+
collect() {
|
|
65
|
+
this.reset();
|
|
66
|
+
this.inc(tally.snapshot().bytesUploaded);
|
|
67
|
+
}
|
|
68
|
+
});
|
|
69
|
+
new promClient.Counter({
|
|
70
|
+
name: "bymax_storage_scans_total",
|
|
71
|
+
help: "File scans by verdict (clean, infected, unknown, timeout, error).",
|
|
72
|
+
labelNames: ["verdict"],
|
|
73
|
+
registers: [registry],
|
|
74
|
+
collect() {
|
|
75
|
+
this.reset();
|
|
76
|
+
const { scans } = tally.snapshot();
|
|
77
|
+
for (const verdict of nestStorage.SCAN_VERDICTS) this.inc({ verdict }, scans[verdict]);
|
|
78
|
+
}
|
|
79
|
+
});
|
|
80
|
+
}
|
|
81
|
+
};
|
|
82
|
+
exports.StorageMetrics = __decorateClass([
|
|
83
|
+
common.SetMetadata(METRICS_CONTRIBUTOR_METADATA, true),
|
|
84
|
+
common.Injectable(),
|
|
85
|
+
__decorateParam(0, common.Inject(nestStorage.StorageMetricsTally))
|
|
86
|
+
], exports.StorageMetrics);
|
|
87
|
+
var HEALTH_INDICATOR_METADATA = "bymax-one:health-indicator";
|
|
88
|
+
var STORAGE_HEALTH_INDICATOR_NAME = "storage";
|
|
89
|
+
var STORAGE_HEALTH_TIMEOUT_MS = 2e3;
|
|
90
|
+
var STORAGE_HEALTH_CACHE_MS = 5e3;
|
|
91
|
+
var _answer, _running;
|
|
92
|
+
exports.StorageHealthIndicator = class StorageHealthIndicator {
|
|
93
|
+
constructor(storage) {
|
|
94
|
+
this.storage = storage;
|
|
95
|
+
__publicField(this, "name", STORAGE_HEALTH_INDICATOR_NAME);
|
|
96
|
+
__privateAdd(this, _answer);
|
|
97
|
+
__privateAdd(this, _running);
|
|
98
|
+
}
|
|
99
|
+
storage;
|
|
100
|
+
/** Whether the bucket answers in time; the aggregator's signal ends this caller's wait. */
|
|
101
|
+
async check(signal) {
|
|
102
|
+
const kept = __privateGet(this, _answer);
|
|
103
|
+
if (kept !== void 0 && Date.now() - kept.at < STORAGE_HEALTH_CACHE_MS) {
|
|
104
|
+
return { status: kept.status };
|
|
105
|
+
}
|
|
106
|
+
const status = await this.waitFor(__privateGet(this, _running) ?? this.probe(), signal);
|
|
107
|
+
return { status };
|
|
108
|
+
}
|
|
109
|
+
/** The shared probe: one `HeadBucket` under its own deadline, remembered when it ends. */
|
|
110
|
+
probe() {
|
|
111
|
+
const running = this.storage.checkBucket({ abortSignal: AbortSignal.timeout(STORAGE_HEALTH_TIMEOUT_MS) }).then(
|
|
112
|
+
() => "up",
|
|
113
|
+
() => "down"
|
|
114
|
+
).then((status) => {
|
|
115
|
+
__privateSet(this, _answer, { at: Date.now(), status });
|
|
116
|
+
__privateSet(this, _running, void 0);
|
|
117
|
+
return status;
|
|
118
|
+
});
|
|
119
|
+
__privateSet(this, _running, running);
|
|
120
|
+
return running;
|
|
121
|
+
}
|
|
122
|
+
/** `probe`'s answer, or `down` as soon as `signal` aborts. */
|
|
123
|
+
waitFor(probe, signal) {
|
|
124
|
+
if (signal === void 0) return probe;
|
|
125
|
+
if (signal.aborted) return Promise.resolve("down");
|
|
126
|
+
return new Promise((resolve) => {
|
|
127
|
+
const stop = () => {
|
|
128
|
+
signal.removeEventListener("abort", stop);
|
|
129
|
+
resolve("down");
|
|
130
|
+
};
|
|
131
|
+
signal.addEventListener("abort", stop);
|
|
132
|
+
void probe.then((status) => {
|
|
133
|
+
signal.removeEventListener("abort", stop);
|
|
134
|
+
resolve(status);
|
|
135
|
+
});
|
|
136
|
+
});
|
|
137
|
+
}
|
|
138
|
+
};
|
|
139
|
+
_answer = new WeakMap();
|
|
140
|
+
_running = new WeakMap();
|
|
141
|
+
exports.StorageHealthIndicator = __decorateClass([
|
|
142
|
+
common.SetMetadata(HEALTH_INDICATOR_METADATA, true),
|
|
143
|
+
common.Injectable(),
|
|
144
|
+
__decorateParam(0, common.Inject(nestStorage.StorageService))
|
|
145
|
+
], exports.StorageHealthIndicator);
|
|
146
|
+
|
|
147
|
+
exports.HEALTH_INDICATOR_METADATA = HEALTH_INDICATOR_METADATA;
|
|
148
|
+
exports.METRICS_CONTRIBUTOR_METADATA = METRICS_CONTRIBUTOR_METADATA;
|
|
149
|
+
exports.STORAGE_HEALTH_CACHE_MS = STORAGE_HEALTH_CACHE_MS;
|
|
150
|
+
exports.STORAGE_HEALTH_INDICATOR_NAME = STORAGE_HEALTH_INDICATOR_NAME;
|
|
151
|
+
exports.STORAGE_HEALTH_TIMEOUT_MS = STORAGE_HEALTH_TIMEOUT_MS;
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
import { Registry } from 'prom-client';
|
|
2
|
+
import { StorageMetricsTally, StorageService } from '@bymax-one/nest-storage';
|
|
3
|
+
|
|
4
|
+
/** The registry the application's scrape endpoint serves. */
|
|
5
|
+
type MetricsRegistry = Registry;
|
|
6
|
+
/** What a component that publishes metrics implements: it receives the registry at bootstrap. */
|
|
7
|
+
interface MetricsContributor {
|
|
8
|
+
registerMetrics(registry: MetricsRegistry): void;
|
|
9
|
+
}
|
|
10
|
+
/** The metadata key a metrics module discovers contributors by. */
|
|
11
|
+
declare const METRICS_CONTRIBUTOR_METADATA = "bymax-one:metrics-contributor";
|
|
12
|
+
/**
|
|
13
|
+
* Registers the storage collectors on the registry the scrape endpoint serves.
|
|
14
|
+
*
|
|
15
|
+
* The counters are read from the tally on each scrape, after a reset, instead of being written as events
|
|
16
|
+
* arrive: the tally starts counting before this class runs, and a counter fed by events would drop that window.
|
|
17
|
+
* The duration histogram cannot be rebuilt from counts, so it is fed as operations end, from the moment this
|
|
18
|
+
* class registers it. Labels are the closed sets the tally is keyed by; no key, bucket, prefix, tenant or message
|
|
19
|
+
* ever becomes one.
|
|
20
|
+
*
|
|
21
|
+
* @example
|
|
22
|
+
* // providers: [StorageMetrics] (BymaxCoreModule discovers it and calls registerMetrics once at bootstrap)
|
|
23
|
+
*/
|
|
24
|
+
declare class StorageMetrics implements MetricsContributor {
|
|
25
|
+
private readonly tally;
|
|
26
|
+
constructor(tally: StorageMetricsTally);
|
|
27
|
+
/** Registers the collectors; called once at bootstrap by the metrics module. */
|
|
28
|
+
registerMetrics(registry: MetricsRegistry): void;
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
/** The outcome of one readiness check. */
|
|
32
|
+
interface HealthIndicatorResult {
|
|
33
|
+
status: 'up' | 'down';
|
|
34
|
+
details?: Record<string, unknown>;
|
|
35
|
+
}
|
|
36
|
+
/** What a readiness indicator implements. */
|
|
37
|
+
interface HealthIndicator {
|
|
38
|
+
readonly name: string;
|
|
39
|
+
check(signal?: AbortSignal): Promise<HealthIndicatorResult>;
|
|
40
|
+
}
|
|
41
|
+
/** The metadata key a health module discovers indicators by. */
|
|
42
|
+
declare const HEALTH_INDICATOR_METADATA = "bymax-one:health-indicator";
|
|
43
|
+
/** The name the indicator reports under. */
|
|
44
|
+
declare const STORAGE_HEALTH_INDICATOR_NAME = "storage";
|
|
45
|
+
/** How long the probe may take before it is `down`. */
|
|
46
|
+
declare const STORAGE_HEALTH_TIMEOUT_MS = 2000;
|
|
47
|
+
/** How long an answer is kept before the bucket is asked again, so a readiness poll cannot become a request flood. */
|
|
48
|
+
declare const STORAGE_HEALTH_CACHE_MS = 5000;
|
|
49
|
+
/**
|
|
50
|
+
* The readiness check of the bucket.
|
|
51
|
+
*
|
|
52
|
+
* It asks `StorageService.checkBucket()` for one `HeadBucket`: nothing is written and no object is read. A bucket
|
|
53
|
+
* that cannot be reached, a refusal, or an answer that does not come within {@link STORAGE_HEALTH_TIMEOUT_MS} is
|
|
54
|
+
* `down`. The answer, up or down, is kept for {@link STORAGE_HEALTH_CACHE_MS}, and checks that arrive while one is
|
|
55
|
+
* running share it. No detail is reported, so nothing about the store (its name, its region or its endpoint)
|
|
56
|
+
* leaves with it. The aggregator's signal ends the wait of that caller only: the shared probe runs under its own
|
|
57
|
+
* deadline, so one caller giving up cannot fail the others.
|
|
58
|
+
*/
|
|
59
|
+
declare class StorageHealthIndicator implements HealthIndicator {
|
|
60
|
+
#private;
|
|
61
|
+
private readonly storage;
|
|
62
|
+
readonly name = "storage";
|
|
63
|
+
constructor(storage: StorageService);
|
|
64
|
+
/** Whether the bucket answers in time; the aggregator's signal ends this caller's wait. */
|
|
65
|
+
check(signal?: AbortSignal): Promise<HealthIndicatorResult>;
|
|
66
|
+
/** The shared probe: one `HeadBucket` under its own deadline, remembered when it ends. */
|
|
67
|
+
private probe;
|
|
68
|
+
/** `probe`'s answer, or `down` as soon as `signal` aborts. */
|
|
69
|
+
private waitFor;
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
export { HEALTH_INDICATOR_METADATA, METRICS_CONTRIBUTOR_METADATA, STORAGE_HEALTH_CACHE_MS, STORAGE_HEALTH_INDICATOR_NAME, STORAGE_HEALTH_TIMEOUT_MS, StorageHealthIndicator, StorageMetrics };
|