@sdflc/backend-helpers 1.0.140 → 1.0.141
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +405 -119
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @sdflc/backend-helpers
|
|
2
2
|
|
|
3
|
-
A comprehensive backend framework for Node.js applications providing database access, authentication, caching, IP-based rate limiting, GraphQL resolver helpers, REST controllers, CLI utilities, and
|
|
3
|
+
A comprehensive backend framework for Node.js applications providing database access, authentication, caching, IP-based rate limiting, GraphQL resolver helpers, REST controllers, CLI utilities, migration tools, ready-made gateways to outside services (SendGrid, Telegram, Twilio, Slack, Discord, signed HTTP webhooks), and emergency alerts a service raises by itself.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -27,13 +27,26 @@ A comprehensive backend framework for Node.js applications providing database ac
|
|
|
27
27
|
- [BaseRestApiCrudController](#baserestapicrudcontroller)
|
|
28
28
|
- [BaseExternalApiGateway](#baseexternalapigateway)
|
|
29
29
|
- [Gateways to outside services](#gateways-to-outside-services)
|
|
30
|
+
- [SendGridGw](#sendgridgw)
|
|
31
|
+
- [TelegramGw](#telegramgw)
|
|
32
|
+
- [TwilioGw](#twiliogw)
|
|
33
|
+
- [SlackWebhookGw](#slackwebhookgw)
|
|
34
|
+
- [DiscordWebhookGw](#discordwebhookgw)
|
|
35
|
+
- [HttpWebhookGw](#httpwebhookgw)
|
|
36
|
+
- [Outbound URL safety](#outbound-url-safety)
|
|
37
|
+
- [Testing a gateway](#testing-a-gateway)
|
|
38
|
+
- [Emergency alerts](#emergency-alerts)
|
|
39
|
+
- [Wiring it into a service](#wiring-it-into-a-service)
|
|
40
|
+
- [The settings fromConfig reads](#the-settings-fromconfig-reads)
|
|
41
|
+
- [Building it yourself](#building-it-yourself)
|
|
42
|
+
- [What it answers](#what-it-answers)
|
|
43
|
+
- [The rules it keeps](#the-rules-it-keeps)
|
|
30
44
|
- [IP rate limiting](#ip-rate-limiting)
|
|
31
45
|
- [How it works](#how-it-works)
|
|
32
46
|
- [Setup](#setup)
|
|
33
47
|
- [Global tier](#global-tier)
|
|
34
48
|
- [Core-level rules](#core-level-rules)
|
|
35
49
|
- [Action-level rules](#action-level-rules)
|
|
36
|
-
- [Per-call override](#per-call-override)
|
|
37
50
|
- [Disabling rate limiting](#disabling-rate-limiting)
|
|
38
51
|
- [Resetting a counter after success](#resetting-a-counter-after-success)
|
|
39
52
|
- [Custom limiter](#custom-limiter)
|
|
@@ -57,7 +70,6 @@ A comprehensive backend framework for Node.js applications providing database ac
|
|
|
57
70
|
- [Utility functions](#utility-functions)
|
|
58
71
|
- [downloadFile](#downloadfile)
|
|
59
72
|
- [passwordHelpers](#passwordhelpers)
|
|
60
|
-
- [Emergency alerts](#emergency-alerts)
|
|
61
73
|
- [Constants](#constants)
|
|
62
74
|
- [Interfaces reference](#interfaces-reference)
|
|
63
75
|
|
|
@@ -73,7 +85,7 @@ npm install @sdflc/backend-helpers
|
|
|
73
85
|
|
|
74
86
|
## Requirements
|
|
75
87
|
|
|
76
|
-
- Node.js >= 22
|
|
88
|
+
- Node.js >= 22. The gateways use the built-in `fetch`, `AbortSignal.timeout` and `net.BlockList`
|
|
77
89
|
- TypeScript >= 5.7
|
|
78
90
|
- PostgreSQL or MSSQL (via Knex)
|
|
79
91
|
|
|
@@ -93,17 +105,21 @@ Business layer → Core classes (BaseCore subclasses)
|
|
|
93
105
|
validation, IP rate limiting, enforce business rules,
|
|
94
106
|
and normalise responses into a consistent OpResult shape.
|
|
95
107
|
|
|
96
|
-
Data layer → Gateway classes
|
|
97
|
-
|
|
98
|
-
queries with joins and filters
|
|
99
|
-
|
|
108
|
+
Data layer → Gateway classes
|
|
109
|
+
BaseGateway subclasses perform CRUD operations on database
|
|
110
|
+
tables and run complex queries with joins and filters.
|
|
111
|
+
BaseExternalApiGateway subclasses talk to outside services
|
|
112
|
+
(email, messaging, webhooks, payment providers).
|
|
100
113
|
```
|
|
101
114
|
|
|
102
115
|
Each inbound request — whether from an HTTP client, a GraphQL operation, or a CLI command — creates a fresh **ApiContext** that carries the database connection, all gateway instances, all core instances, and request-scoped data (userId, spaceId, headers, etc.).
|
|
103
116
|
|
|
104
117
|
**Key invariants:**
|
|
105
118
|
|
|
106
|
-
-
|
|
119
|
+
- Database gateways (`BaseGateway`) and cores are instantiated **per-request** and must never be reused across requests
|
|
120
|
+
- Gateways to outside services (`BaseExternalApiGateway`) hold no request state: create one once (or per request, if it
|
|
121
|
+
sits in your gateway barrel) and `configure()` it whenever its settings change. `configure` **replaces** the settings,
|
|
122
|
+
so nothing of one tenant carries over to the next
|
|
107
123
|
- The data layer (gateways) has no knowledge of HTTP, GraphQL, or authentication — it only handles data
|
|
108
124
|
- The business layer (cores) has no knowledge of transport details — it receives plain args and returns OpResults
|
|
109
125
|
- The application layer (controllers/resolvers/CLI) has no direct database access — it goes through cores
|
|
@@ -1163,58 +1179,369 @@ acme.configure({ apiKey: await keyOf(spaceId) }); // per tenant, whenever
|
|
|
1163
1179
|
const result = await acme.ping();
|
|
1164
1180
|
```
|
|
1165
1181
|
|
|
1166
|
-
|
|
1167
|
-
|
|
1182
|
+
What a subclass overrides, and what it uses:
|
|
1183
|
+
|
|
1184
|
+
| Member | Purpose |
|
|
1185
|
+
|---|---|
|
|
1186
|
+
| `validateSettings(settings)` | Override. What's missing or malformed, in words (never a value). Drives `isConfigured()` / `getConfigProblems()` |
|
|
1187
|
+
| `secretValues()` | Override. The values scrubbed out of everything the gateway returns (keys, tokens, secret URLs) |
|
|
1188
|
+
| `errorMessageOf(reply)` | Override. The refusal's words, from the service's own error shape |
|
|
1189
|
+
| `shouldRetry(result)` | Override. Refuse hopeless retries (default: network, timeout, 429, 5xx) |
|
|
1190
|
+
| `retryAfterMs(result)` | Override. How long the service asked to wait (default: the `Retry-After` header) |
|
|
1191
|
+
| `onConfigured()` | Override. Called after every `configure` (to rebuild an SDK client) |
|
|
1192
|
+
| `settings` | Use. The current settings (they hold secrets: never log or return them) |
|
|
1193
|
+
| `request(req)` / `requestWithRetries(req)` | Use. One HTTP attempt, or attempts with retries. `req`: `{ url, method?, headers?, json?, body?, timeoutMs?, secrets? }`; `secrets` are per-request values to scrub |
|
|
1194
|
+
| `notConfigured()` | Use. The standard refusal while the gateway isn't set up |
|
|
1195
|
+
| `scrub(text, extra?)` / `scrubValue(value, extra?)` | Use. Replace every secret with `***` |
|
|
1196
|
+
|
|
1197
|
+
ⓘ `setCredentials()` / `onCredentialsSet()` still work for gateways written before `configure()`.
|
|
1168
1198
|
|
|
1169
1199
|
---
|
|
1170
1200
|
|
|
1171
1201
|
## Gateways to outside services
|
|
1172
1202
|
|
|
1173
|
-
Ready-made gateways on `BaseExternalApiGateway`
|
|
1203
|
+
Ready-made gateways on [`BaseExternalApiGateway`](#baseexternalapigateway): no SDKs, one REST call each, an `OpResult`
|
|
1204
|
+
out, never a throw. Create one **once**, then `configure()` it whenever the settings change (per brand, per tenant, per
|
|
1205
|
+
request). `configure()` **replaces** the settings; it never merges them.
|
|
1206
|
+
|
|
1207
|
+
| Gateway | `configure({...})` | Operation | OK data |
|
|
1208
|
+
|---|---|---|---|
|
|
1209
|
+
| [`SendGridGw`](#sendgridgw) | `apiKey`, `fromEmail?`, `fromName?` | `sendEmail(email)` | `{ messageId }` |
|
|
1210
|
+
| [`TelegramGw`](#telegramgw) | `botToken`, `chatId?`, `parseMode?` | `sendMessage(message)` | `{ messageId }` |
|
|
1211
|
+
| [`TwilioGw`](#twiliogw) | `accountSid`, `authToken`, `from?`, `messagingServiceSid?` | `sendMessage(message)` | `{ sid, status }` |
|
|
1212
|
+
| [`SlackWebhookGw`](#slackwebhookgw) | `webhookUrl` | `postMessage(message)` | `{ messageId: null }` |
|
|
1213
|
+
| [`DiscordWebhookGw`](#discordwebhookgw) | `webhookUrl`, `username?`, `avatarUrl?` | `postMessage(message)` | `{ messageId }` |
|
|
1214
|
+
| [`HttpWebhookGw`](#httpwebhookgw) | `url`, `method?`, `headers?`, `signingSecret?` | `send(request)` | `{ status, body }` |
|
|
1215
|
+
|
|
1216
|
+
Every operation answers the same way:
|
|
1174
1217
|
|
|
1175
|
-
|
|
|
1218
|
+
| Outcome | `OpResult` code | Retried? |
|
|
1176
1219
|
|---|---|---|
|
|
1177
|
-
|
|
|
1178
|
-
|
|
|
1179
|
-
|
|
|
1180
|
-
|
|
|
1181
|
-
|
|
|
1182
|
-
|
|
|
1220
|
+
| Delivered | `OK`, with the data above | — |
|
|
1221
|
+
| Not configured, or the message itself is wrong (no recipient, too long, a bad number…) | `VALIDATION_FAILED` (nothing is sent) | no |
|
|
1222
|
+
| The service refused it | the code for its HTTP status: `UNAUTHORIZED` 401, `FORBIDDEN` 403, `NOT_FOUND` 404, `CONFLICT` 409, `VALIDATION_FAILED` other 4xx | no |
|
|
1223
|
+
| Throttled | `TOO_MANY_REQUESTS` 429 | yes, after its `Retry-After` |
|
|
1224
|
+
| The service failed | `FAILED` 5xx | yes |
|
|
1225
|
+
| No answer | `NETWORK_ERROR`, or `TIMEOUT` (also 408) | yes (email and SMS: not after a timeout) |
|
|
1226
|
+
|
|
1227
|
+
The message is the service's own words (`result.getErrorSummary('')`). The data carries `{ status, headers, body }` of
|
|
1228
|
+
its reply, scrubbed of every secret.
|
|
1183
1229
|
|
|
1184
1230
|
```typescript
|
|
1185
|
-
import { SendGridGw
|
|
1231
|
+
import { SendGridGw } from '@sdflc/backend-helpers';
|
|
1232
|
+
|
|
1233
|
+
const mail = new SendGridGw(); // once
|
|
1234
|
+
mail.configure({ apiKey: brand.sendgridKey, fromEmail: brand.fromEmail, fromName: brand.name }); // per brand, whenever
|
|
1186
1235
|
|
|
1187
|
-
const mail = new SendGridGw().configure({ apiKey, fromEmail: 'hello@brand.com', fromName: 'Brand' });
|
|
1188
1236
|
const sent = await mail.sendEmail({ to: 'guest@example.com', subject: 'Welcome', html: '<p>Hi</p>' });
|
|
1189
1237
|
|
|
1190
1238
|
if (sent.didFail()) {
|
|
1191
|
-
logger.error(sent.getErrorSummary(''));
|
|
1239
|
+
logger.error(`welcome email: ${sent.getErrorSummary('')}`); // the service's words, no key in them
|
|
1192
1240
|
}
|
|
1193
1241
|
```
|
|
1194
1242
|
|
|
1195
|
-
|
|
1196
|
-
|
|
1197
|
-
|
|
1198
|
-
|
|
1199
|
-
|
|
1200
|
-
|
|
1201
|
-
|
|
1202
|
-
|
|
1203
|
-
|
|
1204
|
-
|
|
1205
|
-
|
|
1206
|
-
|
|
1207
|
-
|
|
1208
|
-
|
|
1209
|
-
|
|
1210
|
-
|
|
1211
|
-
|
|
1212
|
-
|
|
1213
|
-
|
|
1214
|
-
|
|
1215
|
-
|
|
1216
|
-
|
|
1217
|
-
|
|
1243
|
+
### SendGridGw
|
|
1244
|
+
|
|
1245
|
+
Email through SendGrid's v3 Mail Send API: plain text, HTML, or a dynamic template.
|
|
1246
|
+
|
|
1247
|
+
```typescript
|
|
1248
|
+
const mail = new SendGridGw().configure({ apiKey, fromEmail: 'hello@brand.com', fromName: 'Brand' });
|
|
1249
|
+
|
|
1250
|
+
await mail.sendEmail({
|
|
1251
|
+
to: ['a@example.com', { email: 'b@example.com', name: 'B' }], // a SendGridAddress, or several
|
|
1252
|
+
cc: 'c@example.com',
|
|
1253
|
+
replyTo: 'support@brand.com',
|
|
1254
|
+
subject: 'Your ticket',
|
|
1255
|
+
text: 'Plain version',
|
|
1256
|
+
html: '<p>Rich version</p>',
|
|
1257
|
+
categories: ['tickets'],
|
|
1258
|
+
customArgs: { orderId: 'o-1' },
|
|
1259
|
+
});
|
|
1260
|
+
|
|
1261
|
+
await mail.sendEmail({ to: 'a@example.com', templateId: 'd-123', dynamicTemplateData: { name: 'A' } }); // no subject needed
|
|
1262
|
+
```
|
|
1263
|
+
|
|
1264
|
+
- The From (`from`, else the configured `fromEmail`) must be on a domain authenticated in that SendGrid account;
|
|
1265
|
+
SendGrid refuses anything else.
|
|
1266
|
+
- A malformed recipient is **refused**, never silently dropped.
|
|
1267
|
+
- Not retried after a timeout: SendGrid may well have accepted the mail, and a retry would send it twice.
|
|
1268
|
+
|
|
1269
|
+
### TelegramGw
|
|
1270
|
+
|
|
1271
|
+
Messages through the Telegram Bot API.
|
|
1272
|
+
|
|
1273
|
+
```typescript
|
|
1274
|
+
const telegram = new TelegramGw().configure({ botToken: '123456:ABC…', chatId: '-1001234567890' });
|
|
1275
|
+
|
|
1276
|
+
await telegram.sendMessage({ text: '<b>Deployed</b> v1.2' }); // HTML by default
|
|
1277
|
+
await telegram.sendMessage({ text: 'plain', parseMode: null, chatId: '@ops_channel' }); // another chat, no markup
|
|
1278
|
+
await telegram.sendMessage({ text: 'Open', inlineKeyboard: [[{ text: 'Dashboard', url: 'https://…' }]] });
|
|
1279
|
+
```
|
|
1280
|
+
|
|
1281
|
+
- A chat is a numeric id (negative for a group) or `@channelname`; `isTelegramChatId(value)` checks one.
|
|
1282
|
+
- A text over 4096 characters is **refused, not cut**. It's counted the way Telegram counts it (`telegramVisibleLength`:
|
|
1283
|
+
tags are free, an entity is one character). Cutting finished HTML can split an entity, and Telegram then refuses the
|
|
1284
|
+
whole message, so cut your text **before** adding markup.
|
|
1285
|
+
- A 200 with `ok: false` is a failure. A permanent refusal (chat not found, bot blocked, broken markup) is not retried.
|
|
1286
|
+
Telegram's own `retry_after` is honoured.
|
|
1287
|
+
- The bot token sits in the request URL. It is scrubbed out of everything returned.
|
|
1288
|
+
|
|
1289
|
+
### TwilioGw
|
|
1290
|
+
|
|
1291
|
+
SMS, or WhatsApp with `whatsapp:+…` addresses, through Twilio's Messages API.
|
|
1292
|
+
|
|
1293
|
+
```typescript
|
|
1294
|
+
const twilio = new TwilioGw().configure({ accountSid: 'AC…', authToken, from: '+15551234567' });
|
|
1295
|
+
// or configure({ accountSid, authToken, messagingServiceSid: 'MG…' }) to send through a messaging service
|
|
1296
|
+
|
|
1297
|
+
const result = await twilio.sendMessage({ to: '+15557654321', body: 'Your code is 123456', statusCallback: 'https://…' });
|
|
1298
|
+
// OK → [{ sid: 'SM…', status: 'queued' }]
|
|
1299
|
+
```
|
|
1300
|
+
|
|
1301
|
+
- Numbers must be E.164. A body over 1600 characters is refused before anything is sent.
|
|
1302
|
+
- Not retried after a timeout (it may have gone out, and a retry texts the person twice).
|
|
1303
|
+
- The auth token (and the Basic credentials built from it) is scrubbed out of everything returned.
|
|
1304
|
+
|
|
1305
|
+
### SlackWebhookGw
|
|
1306
|
+
|
|
1307
|
+
Posts to a Slack channel through an incoming webhook.
|
|
1308
|
+
|
|
1309
|
+
```typescript
|
|
1310
|
+
const slack = new SlackWebhookGw().configure({ webhookUrl: 'https://hooks.slack.com/services/…' });
|
|
1311
|
+
|
|
1312
|
+
await slack.postMessage({ text: 'Deployed', blocks: [/* Block Kit */] });
|
|
1313
|
+
```
|
|
1314
|
+
|
|
1315
|
+
- The webhook URL **is** the credential; it is scrubbed out of everything returned. Only `https://hooks.slack.com/…` is
|
|
1316
|
+
accepted.
|
|
1317
|
+
- Slack's webhook answers no id, so the OK data is `{ messageId: null }`, the same shape as the others.
|
|
1318
|
+
|
|
1319
|
+
### DiscordWebhookGw
|
|
1320
|
+
|
|
1321
|
+
Posts to a Discord channel through a webhook.
|
|
1322
|
+
|
|
1323
|
+
```typescript
|
|
1324
|
+
const discord = new DiscordWebhookGw().configure({ webhookUrl: 'https://discord.com/api/webhooks/…', username: 'Alerts' });
|
|
1325
|
+
|
|
1326
|
+
await discord.postMessage({ content: 'Deployed', embeds: [{ title: 'v1.2', color: 0x5865f2 }] });
|
|
1327
|
+
```
|
|
1328
|
+
|
|
1329
|
+
- Content is at most 2000 characters, with at most 10 embeds.
|
|
1330
|
+
- **Nobody is pinged** unless `allowedMentions` says so: `@everyone` in a text stays text.
|
|
1331
|
+
- An unknown webhook or a bad webhook token is not retried. The webhook URL is scrubbed out of everything returned.
|
|
1332
|
+
|
|
1333
|
+
### HttpWebhookGw
|
|
1334
|
+
|
|
1335
|
+
Sends JSON to any URL, for anything without a gateway of its own, optionally **signed**.
|
|
1336
|
+
|
|
1337
|
+
```typescript
|
|
1338
|
+
const hook = new HttpWebhookGw(); // once: the address protection is on
|
|
1339
|
+
hook.configure({ url: tenant.webhookUrl, headers: { 'X-Api-Key': tenant.apiKey }, signingSecret: tenant.signingSecret });
|
|
1340
|
+
|
|
1341
|
+
const result = await hook.send({ payload: { event: 'order.paid', id }, headers: { 'X-Trace': traceId } });
|
|
1342
|
+
// OK → [{ status: 200, body: … }] (the receiver's answer, scrubbed)
|
|
1343
|
+
```
|
|
1344
|
+
|
|
1345
|
+
- **The URL a user typed must not reach your own network** (see [Outbound URL safety](#outbound-url-safety)). It is
|
|
1346
|
+
checked before the request, and the address the socket actually connects to is checked again at connect time.
|
|
1347
|
+
- `allowPrivateNetwork: true` is a **constructor** option, never a setting: `new HttpWebhookGw({ allowPrivateNetwork: true })`.
|
|
1348
|
+
Use it only for a URL **you** configure (an internal service). Settings often come from a tenant's own row, and a
|
|
1349
|
+
tenant must not be able to switch the protection off.
|
|
1350
|
+
- Every send carries `X-Webhook-Id`, which stays the same on its retries so a receiver can drop a duplicate.
|
|
1351
|
+
- With `signingSecret` it also carries `X-Webhook-Timestamp` (unix seconds) and
|
|
1352
|
+
`X-Webhook-Signature: sha256=<hex HMAC-SHA256 of "<timestamp>.<body>">`. A configured or per-request header can never
|
|
1353
|
+
replace these.
|
|
1354
|
+
- A header with a line break, or a payload that can't be sent as JSON (a function), is refused.
|
|
1355
|
+
- Every header value is scrubbed out of everything returned, configured or per-request, whole and word by word.
|
|
1356
|
+
|
|
1357
|
+
A receiver checks a signed request like this:
|
|
1358
|
+
|
|
1359
|
+
```typescript
|
|
1360
|
+
import { timingSafeEqual } from 'crypto';
|
|
1361
|
+
import { webhookSignatureOf } from '@sdflc/backend-helpers';
|
|
1362
|
+
|
|
1363
|
+
const timestamp = Number(req.headers['x-webhook-timestamp']);
|
|
1364
|
+
const expected = Buffer.from(webhookSignatureOf(secret, timestamp, rawBody)); // the RAW body, before JSON parsing
|
|
1365
|
+
const given = Buffer.from(String(req.headers['x-webhook-signature'] ?? ''));
|
|
1366
|
+
const fresh = Math.abs(Date.now() / 1000 - timestamp) < 300;
|
|
1367
|
+
const genuine = fresh && expected.length === given.length && timingSafeEqual(expected, given);
|
|
1368
|
+
```
|
|
1369
|
+
|
|
1370
|
+
### Outbound URL safety
|
|
1371
|
+
|
|
1372
|
+
For any code that sends a request to a URL a **user** gave (webhooks, callbacks, "import from URL"):
|
|
1373
|
+
|
|
1374
|
+
- **`checkExternalUrl(url, { protocols?, lookup?, field? })`**: `OK` when the URL may be requested, else
|
|
1375
|
+
`VALIDATION_FAILED` saying why. It refuses:
|
|
1376
|
+
- anything but http(s);
|
|
1377
|
+
- a user name or password in the URL;
|
|
1378
|
+
- `localhost`, `*.localhost` and `*.internal` names, and the cloud metadata host names;
|
|
1379
|
+
- any host that is, or resolves to, a blocked address. **Every** address it resolves to is checked: a name with one
|
|
1380
|
+
private address among public ones is refused.
|
|
1381
|
+
- **`isBlockedAddress(ip)`**: true for the addresses a user-given URL must never reach, however they're written. IPv4:
|
|
1382
|
+
this-network, private, carrier-grade NAT, loopback, link-local (169.254.169.254, the metadata service), protocol
|
|
1383
|
+
assignments, test networks, the 6to4 relay, benchmarking, multicast, reserved. IPv6: unspecified, loopback,
|
|
1384
|
+
IPv4-compatible, IPv4-mapped (checked as IPv4), NAT64 (both prefixes), 6to4, discard-only, documentation,
|
|
1385
|
+
unique-local `fc00::/7`, link-local, multicast.
|
|
1386
|
+
- **`createGuardedFetch({ dnsLookup?, isBlocked? })`**: a `fetch` that checks the address **at the moment the socket
|
|
1387
|
+
connects**. A check made only beforehand loses to DNS rebinding (a server that answers a public address first and an
|
|
1388
|
+
internal one to the second lookup). It also checks an IP written in the URL directly, never follows redirects, reads
|
|
1389
|
+
at most 64 KB, and ends at its signal. `HttpWebhookGw` uses it by default.
|
|
1390
|
+
|
|
1391
|
+
### Testing a gateway
|
|
1392
|
+
|
|
1393
|
+
Every gateway takes a `fetch` in its constructor props, so a test runs the real gateway against a stub and nothing leaves
|
|
1394
|
+
the machine:
|
|
1395
|
+
|
|
1396
|
+
```typescript
|
|
1397
|
+
const calls: any[] = [];
|
|
1398
|
+
const fetch = async (url: string, init: any) => {
|
|
1399
|
+
calls.push({ url, body: JSON.parse(init.body) });
|
|
1400
|
+
|
|
1401
|
+
return { status: 202, headers: { forEach: () => null }, text: async () => '' };
|
|
1402
|
+
};
|
|
1403
|
+
|
|
1404
|
+
const mail = new SendGridGw({ fetch, retries: 0 }).configure({ apiKey: 'SG.test' });
|
|
1405
|
+
const result = await mail.sendEmail({ to: 'a@b.test', from: 'x@b.test', subject: 's', text: 't' });
|
|
1406
|
+
// result.didSucceed() === true; calls[0].body.personalizations[0].to[0].email === 'a@b.test'
|
|
1407
|
+
```
|
|
1408
|
+
|
|
1409
|
+
`HttpWebhookGw` also takes `lookup` (the pre-check's resolver) and `dnsLookup` (the connect-time resolver).
|
|
1410
|
+
|
|
1411
|
+
---
|
|
1412
|
+
|
|
1413
|
+
## Emergency alerts
|
|
1414
|
+
|
|
1415
|
+
A service raises an alarm **by itself**, by email and/or Telegram, when it starts with missing configuration or
|
|
1416
|
+
crashes. `EmergencyAlerter` decides **what** is said, **to whom** and **how often**. It sends through the gateways above
|
|
1417
|
+
(`SendGridGw`, `TelegramGw`) and contains no transport code itself. Each alert names the app, the service and the
|
|
1418
|
+
environment (`[partylabz · ms_storage · production]`), because several apps can run the same services side by side on
|
|
1419
|
+
one machine.
|
|
1420
|
+
|
|
1421
|
+
### Wiring it into a service
|
|
1422
|
+
|
|
1423
|
+
```typescript
|
|
1424
|
+
// src/index.ts
|
|
1425
|
+
import { EmergencyAlerter, CriticalConfigItem } from '@sdflc/backend-helpers';
|
|
1426
|
+
|
|
1427
|
+
// What this service starts WITHOUT (optional to the config loader) but cannot fully work without.
|
|
1428
|
+
// APP_NAME is always checked as well. `key` (the built-config name) is derived from `env`; pass it only when it differs.
|
|
1429
|
+
const CRITICAL_CONFIG: CriticalConfigItem[] = [
|
|
1430
|
+
{ env: 'INTERSERVICE_API_KEY', effect: 'Every call from our other services is refused.' },
|
|
1431
|
+
];
|
|
1432
|
+
|
|
1433
|
+
const alerter = EmergencyAlerter.fromConfig(config, 'ms_storage', { log: (m) => logger.error(m) });
|
|
1434
|
+
|
|
1435
|
+
alerter.installCrashHandlers(process); // uncaught exception / unhandled rejection → alert, then exit(1)
|
|
1436
|
+
|
|
1437
|
+
const run = async () => {
|
|
1438
|
+
const app = express();
|
|
1439
|
+
|
|
1440
|
+
configureApp(app);
|
|
1441
|
+
|
|
1442
|
+
app.listen(config.port, async () => {
|
|
1443
|
+
logger.log(`Server started on port ${config.port}`);
|
|
1444
|
+
await alerter.checkConfig(config, CRITICAL_CONFIG); // the service still starts; it says loudly what is missing
|
|
1445
|
+
});
|
|
1446
|
+
};
|
|
1447
|
+
|
|
1448
|
+
run().catch(async (e) => {
|
|
1449
|
+
await alerter.send({ key: 'crash:startup', subject: `Failed to start: ${e?.message ?? e}`, lines: [String(e?.stack ?? e)] });
|
|
1450
|
+
process.exit(1);
|
|
1451
|
+
});
|
|
1452
|
+
```
|
|
1453
|
+
|
|
1454
|
+
Anything else worth waking someone for:
|
|
1455
|
+
|
|
1456
|
+
```typescript
|
|
1457
|
+
const result = await alerter.send({ key: 'queue:stalled', subject: 'The queue stopped draining', lines: ['…'] });
|
|
1458
|
+
```
|
|
1459
|
+
|
|
1460
|
+
### The settings fromConfig reads
|
|
1461
|
+
|
|
1462
|
+
From the built config (`buildConfig`). All of them are optional, and each channel needs all of its own. Leave them
|
|
1463
|
+
commented out in `.env.example` so a service without them still starts.
|
|
1464
|
+
|
|
1465
|
+
| Env | Meaning |
|
|
1466
|
+
|---|---|
|
|
1467
|
+
| `APP_NAME` | The app this service belongs to. Alerts say `unknown-app` without it, and `checkConfig` reports it as missing |
|
|
1468
|
+
| `EMERGENCY_SENDGRID_API_KEY` | Email: a SendGrid key allowed to send mail |
|
|
1469
|
+
| `EMERGENCY_EMAIL_FROM` | Email: a sender on a domain authenticated in that SendGrid account. Required: a guessed one is refused |
|
|
1470
|
+
| `EMERGENCY_EMAIL_TO` | Email: recipients, comma-separated |
|
|
1471
|
+
| `TELEGRAM_BOT_ID` | Telegram: the bot token `<id>:<secret>` from @BotFather |
|
|
1472
|
+
| `TELEGRAM_CHAT_ID` | Telegram: the chat id (negative for a group) or `@channel`. The bot must be in it |
|
|
1473
|
+
| `EMERGENCY_EMAIL_MIN_INTERVAL_MINUTES` | The same alert at most once per this many minutes, on every channel. Default 60; 0 = every time |
|
|
1474
|
+
|
|
1475
|
+
`fromConfig(config, serviceName, { log?, redact?, memory?, now?, hostname? })` gives each provider a 4 s timeout and one
|
|
1476
|
+
short retry, so both channels together fit the crash wait. It says once, through `log`, when a channel was set up but
|
|
1477
|
+
can't be used (for example a malformed token, or an email missing its From).
|
|
1478
|
+
|
|
1479
|
+
### Building it yourself
|
|
1480
|
+
|
|
1481
|
+
Credentials live in the gateways; who to alert lives in the alerter.
|
|
1482
|
+
|
|
1483
|
+
```typescript
|
|
1484
|
+
const alerter = new EmergencyAlerter({
|
|
1485
|
+
email: new SendGridGw({ timeoutMs: 4000, retries: 1 }).configure({ apiKey }),
|
|
1486
|
+
telegram: new TelegramGw({ timeoutMs: 4000, retries: 1 }).configure({ botToken }),
|
|
1487
|
+
settings: {
|
|
1488
|
+
appName: 'partylabz',
|
|
1489
|
+
serviceName: 'ms_storage',
|
|
1490
|
+
environment: 'production',
|
|
1491
|
+
minIntervalMinutes: 60,
|
|
1492
|
+
emailTo: ['ops@brand.com'],
|
|
1493
|
+
emailFrom: 'alerts@brand.com',
|
|
1494
|
+
telegramChatId: '-100123',
|
|
1495
|
+
},
|
|
1496
|
+
log: (m) => logger.error(m),
|
|
1497
|
+
redact: (text) => text.replace(/password=\S+/g, 'password=***'),
|
|
1498
|
+
});
|
|
1499
|
+
```
|
|
1500
|
+
|
|
1501
|
+
`configure(settings)` **replaces** the settings, and `useEmail(gateway | null)` / `useTelegram(gateway | null)` swap a
|
|
1502
|
+
channel's gateway.
|
|
1503
|
+
|
|
1504
|
+
### What it answers
|
|
1505
|
+
|
|
1506
|
+
Every method returns an `OpResult` and never throws.
|
|
1507
|
+
|
|
1508
|
+
| Call | OK | Not OK |
|
|
1509
|
+
|---|---|---|
|
|
1510
|
+
| `send(alert)` | `[{ status: 'sent', channels: { email, telegram } }]` (each `'sent'`, `'failed'` or `'off'`), or `[{ status: 'recently-sent' }]` | `VALIDATION_FAILED` `[{ status: 'unconfigured' }]` when no channel can be used; `FAILED` `[{ status: 'failed', channels }]` when no channel delivered it, with each channel's error merged in |
|
|
1511
|
+
| `checkConfig(config, items)` | `[]` when nothing is missing | otherwise whatever `send` answered for the missing-configuration alert |
|
|
1512
|
+
|
|
1513
|
+
Also available:
|
|
1514
|
+
- `getLabel()`: `app · service · environment`.
|
|
1515
|
+
- `getChannelProblems()`: why each channel can't be used, in words, never a value.
|
|
1516
|
+
- `getMinIntervalMs()`.
|
|
1517
|
+
- `emailOf(alert, host, at)` and `telegramTextOf(alert, host, at)`: exactly what would be sent.
|
|
1518
|
+
- The pure helpers `missingCriticalConfig(config, items)`, `missingConfigAlert(config, items)`, `emergencyOneLine(value, max)`,
|
|
1519
|
+
`emergencyIntervalText(ms)` and `emergencyFileMemory(app, service, dir?)`.
|
|
1520
|
+
|
|
1521
|
+
### The rules it keeps
|
|
1522
|
+
|
|
1523
|
+
- **Never throws, never blocks for long.** Even a `log` or `redact` that throws can't make it throw. A crash waits for its
|
|
1524
|
+
alert at most 10 s, then exits, and **always** exits, whatever was thrown (even a value that can't be turned into
|
|
1525
|
+
text).
|
|
1526
|
+
- **One alert per key per interval, across all channels.** It's remembered in a file under the OS temp folder named after
|
|
1527
|
+
the app and service, so a crash loop sends one message rather than one per restart.
|
|
1528
|
+
- Sends of the same key **at the same time** go out once.
|
|
1529
|
+
- A send that no channel delivered is not remembered, so the next one tries again.
|
|
1530
|
+
- A remembered time in the future (a clock moved back) doesn't silence anything.
|
|
1531
|
+
- **A crash's key names which crash** (`crash:<kind>:<fingerprint of the message and top frame>`). The same crash in a
|
|
1532
|
+
loop is sent once, and a different crash is never silenced by it.
|
|
1533
|
+
- **A channel set up wrongly is said once, at the start**, never silently skipped.
|
|
1534
|
+
- **What leaves is redacted.** `fromConfig` removes every config value whose **name** looks secret (password, secret,
|
|
1535
|
+
token, key…) from every subject, line and log line, because an error message can carry SQL with its values. Pass
|
|
1536
|
+
`redact` for more.
|
|
1537
|
+
- **Subjects are one line**, because a crash's subject is the error's message, which often spans lines. The Telegram text
|
|
1538
|
+
is cut **before** it's escaped, to whatever the heading and footer leave, so a long stack trace neither splits an
|
|
1539
|
+
entity nor overruns Telegram's limit.
|
|
1540
|
+
- **It goes straight to the providers**, never through another service of yours. It has to work when the rest doesn't.
|
|
1541
|
+
- **Never put a secret, a token or a database row in an alert yourself.** It leaves your infrastructure.
|
|
1542
|
+
|
|
1543
|
+
Proofs: `npm run build && npm run prove`. They cover the gateways and the alerter, each against a stub `fetch`, plus a
|
|
1544
|
+
real local server for the connect-time address check.
|
|
1218
1545
|
|
|
1219
1546
|
---
|
|
1220
1547
|
|
|
@@ -2067,79 +2394,6 @@ const isCorrect = await checkPassword(plaintextPassword, storedHash);
|
|
|
2067
2394
|
|
|
2068
2395
|
---
|
|
2069
2396
|
|
|
2070
|
-
## Emergency alerts
|
|
2071
|
-
|
|
2072
|
-
A service raises an alarm **by itself** — by email and/or Telegram — when it starts with missing configuration or
|
|
2073
|
-
crashes. `EmergencyAlerter` decides **what** is said, **to whom** and **how often**; it sends through the gateways above
|
|
2074
|
-
(`SendGridGw`, `TelegramGw`). Each alert names the app, the service and the environment
|
|
2075
|
-
(`[partylabz · ms_storage · production]`), because several apps can run the same services side by side on one machine.
|
|
2076
|
-
|
|
2077
|
-
```typescript
|
|
2078
|
-
import { EmergencyAlerter, CriticalConfigItem } from '@sdflc/backend-helpers';
|
|
2079
|
-
|
|
2080
|
-
// What this service starts WITHOUT (optional to the config loader) but cannot fully work without.
|
|
2081
|
-
// APP_NAME is always checked as well; `key` (the built-config name) is derived from `env`.
|
|
2082
|
-
const CRITICAL_CONFIG: CriticalConfigItem[] = [
|
|
2083
|
-
{ env: 'INTERSERVICE_API_KEY', effect: 'Every call from our other services is refused.' },
|
|
2084
|
-
];
|
|
2085
|
-
|
|
2086
|
-
const alerter = EmergencyAlerter.fromConfig(config, 'ms_storage', { log: (m) => logger.error(m) });
|
|
2087
|
-
|
|
2088
|
-
alerter.installCrashHandlers(process); // uncaught exception / unhandled rejection → alert, then exit(1)
|
|
2089
|
-
|
|
2090
|
-
httpServer.listen(port, async () => {
|
|
2091
|
-
await alerter.checkConfig(config, CRITICAL_CONFIG); // the service still starts; it says loudly what is missing
|
|
2092
|
-
});
|
|
2093
|
-
|
|
2094
|
-
const result = await alerter.send({ key: 'queue:stalled', subject: 'The queue stopped draining', lines: ['…'] });
|
|
2095
|
-
// OK → data [{ status: 'sent', channels: { email: 'sent', telegram: 'failed' } }] or [{ status: 'recently-sent' }]
|
|
2096
|
-
// VALIDATION_FAILED → no channel can be used; FAILED → no channel delivered it (each channel's error merged in)
|
|
2097
|
-
```
|
|
2098
|
-
|
|
2099
|
-
Or build it yourself — credentials live in the gateways, the addressing in the alerter:
|
|
2100
|
-
|
|
2101
|
-
```typescript
|
|
2102
|
-
const alerter = new EmergencyAlerter({
|
|
2103
|
-
email: new SendGridGw().configure({ apiKey }),
|
|
2104
|
-
telegram: new TelegramGw().configure({ botToken }),
|
|
2105
|
-
settings: { appName: 'partylabz', serviceName: 'ms_storage', environment: 'production',
|
|
2106
|
-
emailTo: ['ops@brand.com'], emailFrom: 'alerts@brand.com', telegramChatId: '-100123' },
|
|
2107
|
-
});
|
|
2108
|
-
```
|
|
2109
|
-
|
|
2110
|
-
Settings `fromConfig` reads (all optional; each channel needs all of its own):
|
|
2111
|
-
|
|
2112
|
-
| Env | Meaning |
|
|
2113
|
-
|---|---|
|
|
2114
|
-
| `APP_NAME` | The app this service belongs to; alerts say `unknown-app` without it (and `checkConfig` reports it) |
|
|
2115
|
-
| `EMERGENCY_SENDGRID_API_KEY` | Email: a SendGrid key allowed to send mail |
|
|
2116
|
-
| `EMERGENCY_EMAIL_FROM` | Email: a sender on a domain authenticated in that SendGrid account (required — a guessed one is refused) |
|
|
2117
|
-
| `EMERGENCY_EMAIL_TO` | Email: recipients, comma-separated |
|
|
2118
|
-
| `TELEGRAM_BOT_ID` | Telegram: the bot token `<id>:<secret>` from @BotFather |
|
|
2119
|
-
| `TELEGRAM_CHAT_ID` | Telegram: the chat id (negative for a group) or `@channel` |
|
|
2120
|
-
| `EMERGENCY_EMAIL_MIN_INTERVAL_MINUTES` | The same alert at most once per this many minutes, on every channel (default 60; 0 = always) |
|
|
2121
|
-
|
|
2122
|
-
Rules it keeps:
|
|
2123
|
-
|
|
2124
|
-
- **Never throws, never blocks for long.** `fromConfig` gives each provider a 4 s timeout (one short retry); a crash
|
|
2125
|
-
waits for its alert at most 10 s, then exits — and **always** exits, whatever was thrown.
|
|
2126
|
-
- **A channel set up wrongly is said once, at the start** — never silently skipped (`getChannelProblems()`).
|
|
2127
|
-
- **What leaves is redacted:** `fromConfig` removes every config value whose NAME looks secret (password, secret,
|
|
2128
|
-
token, key…) from every subject, line and log line — an error message can carry SQL with its values. Pass `redact`
|
|
2129
|
-
for more.
|
|
2130
|
-
- **A crash's key names which crash** (its message and top frame): one crash never silences a different one.
|
|
2131
|
-
- **One alert per key per interval across all channels**, remembered in a file under the OS temp folder named after
|
|
2132
|
-
the app and service — a crash loop sends one message, not one per restart. Sends of the same key **at the same time**
|
|
2133
|
-
go out once; a send no channel delivered is not remembered, so the next one tries again.
|
|
2134
|
-
- **Goes straight to the providers**, never through another service — it has to work when the rest does not.
|
|
2135
|
-
- **Subjects are one line**; the Telegram text is cut **before** it is escaped, to what the heading and footer leave,
|
|
2136
|
-
so a long stack trace neither splits an entity nor overruns Telegram's limit.
|
|
2137
|
-
- **Never put a secret, a token or a database row in an alert** — it leaves your infrastructure.
|
|
2138
|
-
|
|
2139
|
-
Proofs: `npm run build && npm run prove` (the gateways and the alerter, each against a stub `fetch`).
|
|
2140
|
-
|
|
2141
|
-
---
|
|
2142
|
-
|
|
2143
2397
|
## Constants
|
|
2144
2398
|
|
|
2145
2399
|
```typescript
|
|
@@ -2177,6 +2431,22 @@ ORDER_BYS.ASC; // 'asc'
|
|
|
2177
2431
|
ORDER_BYS.DESC; // 'desc'
|
|
2178
2432
|
```
|
|
2179
2433
|
|
|
2434
|
+
Gateways and emergency alerts:
|
|
2435
|
+
|
|
2436
|
+
```typescript
|
|
2437
|
+
MAX_REPLY_BYTES; // 65536: the most of a reply a gateway reads
|
|
2438
|
+
MAX_RETRY_WAIT_MS; // 10000: a longer Retry-After is not waited for (the failure comes back at once)
|
|
2439
|
+
TELEGRAM_MAX_MESSAGE_LENGTH; // 4096
|
|
2440
|
+
TWILIO_MAX_BODY_LENGTH; // 1600
|
|
2441
|
+
DISCORD_MAX_CONTENT_LENGTH; // 2000
|
|
2442
|
+
DISCORD_MAX_EMBEDS; // 10
|
|
2443
|
+
WEBHOOK_ID_HEADER; // 'X-Webhook-Id'
|
|
2444
|
+
WEBHOOK_TIMESTAMP_HEADER; // 'X-Webhook-Timestamp'
|
|
2445
|
+
WEBHOOK_SIGNATURE_HEADER; // 'X-Webhook-Signature'
|
|
2446
|
+
UNKNOWN_APP_NAME; // 'unknown-app': what an alert says without APP_NAME
|
|
2447
|
+
APP_NAME_CONFIG; // the CriticalConfigItem every checkConfig includes
|
|
2448
|
+
```
|
|
2449
|
+
|
|
2180
2450
|
---
|
|
2181
2451
|
|
|
2182
2452
|
## Interfaces reference
|
|
@@ -2188,7 +2458,23 @@ ORDER_BYS.DESC; // 'desc'
|
|
|
2188
2458
|
| `BaseCorePropsInterface` | Constructor props for `BaseCore` subclasses |
|
|
2189
2459
|
| `BaseCoreRateLimitOnActionInterface` | Per-action rate-limit rule map used in `BaseCorePropsInterface` |
|
|
2190
2460
|
| `BaseGatewayPropsInterface` | Constructor props for `BaseGateway` subclasses — includes `statusField`, `nonCreatableFields`, `nonUpdatableFields`, and the filter rule arrays |
|
|
2191
|
-
| `BaseExternalApiGatewayPropsInterface` | Constructor props for `BaseExternalApiGateway`
|
|
2461
|
+
| `BaseExternalApiGatewayPropsInterface` | Constructor props for `BaseExternalApiGateway`: `settings`, `timeoutMs`, `retries`, `retryDelaysMs`, `fetch`, `logger`, `baseUrl`, `credentials` (legacy) |
|
|
2462
|
+
| `ExternalApiFetch` | The HTTP client shape a gateway calls (the platform `fetch`, or a test's stub) |
|
|
2463
|
+
| `ExternalApiRequest` | One outbound request for `request()` / `requestWithRetries()`: `{ url, method?, headers?, json?, body?, timeoutMs?, secrets? }` |
|
|
2464
|
+
| `ExternalApiReply` | What a request answered, in its OpResult data: `{ status, headers, body }`, scrubbed and capped |
|
|
2465
|
+
| `SendGridSettings` / `SendGridEmail` / `SendGridAddress` | `SendGridGw` settings, the email, an address (`'a@b.com'` or `{ email, name? }`) |
|
|
2466
|
+
| `TelegramSettings` / `TelegramMessage` | `TelegramGw` settings and message |
|
|
2467
|
+
| `TwilioSettings` / `TwilioMessage` | `TwilioGw` settings and message |
|
|
2468
|
+
| `SlackWebhookSettings` / `SlackMessage` | `SlackWebhookGw` settings and message |
|
|
2469
|
+
| `DiscordWebhookSettings` / `DiscordMessage` / `DiscordEmbed` | `DiscordWebhookGw` settings, message and embed |
|
|
2470
|
+
| `HttpWebhookSettings` / `HttpWebhookSend` / `HttpWebhookGwProps` | `HttpWebhookGw` settings, one send, and its constructor props (`allowPrivateNetwork`, `lookup`, `dnsLookup`) |
|
|
2471
|
+
| `ExternalUrlOptions` | Options for `checkExternalUrl()`: `{ protocols?, lookup?, field? }` |
|
|
2472
|
+
| `GuardedFetchOptions` | Options for `createGuardedFetch()`: `{ dnsLookup?, isBlocked? }` |
|
|
2473
|
+
| `EmergencyAlert` | One alert: `{ key, subject, lines }` |
|
|
2474
|
+
| `EmergencyAlerterSettings` | `EmergencyAlerter` settings: `appName`, `serviceName`, `environment`, `minIntervalMinutes`, `emailTo`, `emailFrom`, `telegramChatId` |
|
|
2475
|
+
| `EmergencyAlerterProps` | `EmergencyAlerter` constructor props: `settings`, `email`, `telegram`, `log`, `redact`, `memory`, `now`, `hostname` |
|
|
2476
|
+
| `EmergencyAlertMemory` | Where sent alerts are remembered: `{ read, write }` (default: a file under the OS temp folder, per app and service) |
|
|
2477
|
+
| `CriticalConfigItem` | A setting a service cannot fully work without: `{ env, key?, effect }` |
|
|
2192
2478
|
| `BaseRestApiCrudControllerPropsInterface` | Constructor props for `BaseRestApiCrudController` |
|
|
2193
2479
|
| `BuildDefaultResolvers` | Arguments for `buildDefaultResolvers()` |
|
|
2194
2480
|
| `BuildListQueryArgsInterface` | Arguments passed to `buildListQuery()` overrides: `{ filter, params, fieldsToUse, useCache }` |
|