@sdflc/backend-helpers 1.0.139 → 1.0.141
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +471 -29
- package/build/EmergencyAlerter.d.ts +140 -0
- package/build/EmergencyAlerter.js +465 -0
- package/build/esm/EmergencyAlerter.d.ts +140 -0
- package/build/esm/EmergencyAlerter.js +454 -0
- package/build/esm/gateways/BaseExternalApiGateway.d.ts +94 -0
- package/build/esm/gateways/BaseExternalApiGateway.js +319 -0
- package/build/esm/gateways/DiscordWebhookGw.d.ts +50 -0
- package/build/esm/gateways/DiscordWebhookGw.js +70 -0
- package/build/esm/gateways/HttpWebhookGw.d.ts +43 -0
- package/build/esm/gateways/HttpWebhookGw.js +131 -0
- package/build/esm/gateways/SendGridGw.d.ts +41 -0
- package/build/esm/gateways/SendGridGw.js +116 -0
- package/build/esm/gateways/SlackWebhookGw.d.ts +23 -0
- package/build/esm/gateways/SlackWebhookGw.js +59 -0
- package/build/esm/gateways/TelegramGw.d.ts +45 -0
- package/build/esm/gateways/TelegramGw.js +110 -0
- package/build/esm/gateways/TwilioGw.d.ts +33 -0
- package/build/esm/gateways/TwilioGw.js +107 -0
- package/build/esm/gateways/externalUrl.d.ts +16 -0
- package/build/esm/gateways/externalUrl.js +116 -0
- package/build/esm/gateways/guardedFetch.d.ts +15 -0
- package/build/esm/gateways/guardedFetch.js +133 -0
- package/build/esm/gateways/index.d.ts +9 -0
- package/build/esm/gateways/index.js +9 -0
- package/build/esm/index.d.ts +2 -1
- package/build/esm/index.js +2 -1
- package/build/esm/interfaces/BaseExternalApiGatewayPropsInterface.d.ts +16 -2
- package/build/gateways/BaseExternalApiGateway.d.ts +94 -0
- package/build/gateways/BaseExternalApiGateway.js +326 -0
- package/build/gateways/DiscordWebhookGw.d.ts +50 -0
- package/build/gateways/DiscordWebhookGw.js +73 -0
- package/build/gateways/HttpWebhookGw.d.ts +43 -0
- package/build/gateways/HttpWebhookGw.js +135 -0
- package/build/gateways/SendGridGw.d.ts +41 -0
- package/build/gateways/SendGridGw.js +119 -0
- package/build/gateways/SlackWebhookGw.d.ts +23 -0
- package/build/gateways/SlackWebhookGw.js +62 -0
- package/build/gateways/TelegramGw.d.ts +45 -0
- package/build/gateways/TelegramGw.js +115 -0
- package/build/gateways/TwilioGw.d.ts +33 -0
- package/build/gateways/TwilioGw.js +110 -0
- package/build/gateways/externalUrl.d.ts +16 -0
- package/build/gateways/externalUrl.js +121 -0
- package/build/gateways/guardedFetch.d.ts +15 -0
- package/build/gateways/guardedFetch.js +141 -0
- package/build/gateways/index.d.ts +9 -0
- package/build/gateways/index.js +25 -0
- package/build/index.d.ts +2 -1
- package/build/index.js +2 -1
- package/build/interfaces/BaseExternalApiGatewayPropsInterface.d.ts +16 -2
- package/package.json +2 -1
- package/build/BaseExternalApiGateway.d.ts +0 -16
- package/build/BaseExternalApiGateway.js +0 -54
- package/build/esm/BaseExternalApiGateway.d.ts +0 -16
- package/build/esm/BaseExternalApiGateway.js +0 -48
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @sdflc/backend-helpers
|
|
2
2
|
|
|
3
|
-
A comprehensive backend framework for Node.js applications providing database access, authentication, caching, IP-based rate limiting, GraphQL resolver helpers, REST controllers, CLI utilities, and
|
|
3
|
+
A comprehensive backend framework for Node.js applications providing database access, authentication, caching, IP-based rate limiting, GraphQL resolver helpers, REST controllers, CLI utilities, migration tools, ready-made gateways to outside services (SendGrid, Telegram, Twilio, Slack, Discord, signed HTTP webhooks), and emergency alerts a service raises by itself.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -25,14 +25,28 @@ A comprehensive backend framework for Node.js applications providing database ac
|
|
|
25
25
|
- [BaseGateway](#basegateway)
|
|
26
26
|
- [BaseController](#basecontroller)
|
|
27
27
|
- [BaseRestApiCrudController](#baserestapicrudcontroller)
|
|
28
|
-
- [BaseExternalApiGateway](#
|
|
28
|
+
- [BaseExternalApiGateway](#baseexternalapigateway)
|
|
29
|
+
- [Gateways to outside services](#gateways-to-outside-services)
|
|
30
|
+
- [SendGridGw](#sendgridgw)
|
|
31
|
+
- [TelegramGw](#telegramgw)
|
|
32
|
+
- [TwilioGw](#twiliogw)
|
|
33
|
+
- [SlackWebhookGw](#slackwebhookgw)
|
|
34
|
+
- [DiscordWebhookGw](#discordwebhookgw)
|
|
35
|
+
- [HttpWebhookGw](#httpwebhookgw)
|
|
36
|
+
- [Outbound URL safety](#outbound-url-safety)
|
|
37
|
+
- [Testing a gateway](#testing-a-gateway)
|
|
38
|
+
- [Emergency alerts](#emergency-alerts)
|
|
39
|
+
- [Wiring it into a service](#wiring-it-into-a-service)
|
|
40
|
+
- [The settings fromConfig reads](#the-settings-fromconfig-reads)
|
|
41
|
+
- [Building it yourself](#building-it-yourself)
|
|
42
|
+
- [What it answers](#what-it-answers)
|
|
43
|
+
- [The rules it keeps](#the-rules-it-keeps)
|
|
29
44
|
- [IP rate limiting](#ip-rate-limiting)
|
|
30
45
|
- [How it works](#how-it-works)
|
|
31
46
|
- [Setup](#setup)
|
|
32
47
|
- [Global tier](#global-tier)
|
|
33
48
|
- [Core-level rules](#core-level-rules)
|
|
34
49
|
- [Action-level rules](#action-level-rules)
|
|
35
|
-
- [Per-call override](#per-call-override)
|
|
36
50
|
- [Disabling rate limiting](#disabling-rate-limiting)
|
|
37
51
|
- [Resetting a counter after success](#resetting-a-counter-after-success)
|
|
38
52
|
- [Custom limiter](#custom-limiter)
|
|
@@ -71,7 +85,7 @@ npm install @sdflc/backend-helpers
|
|
|
71
85
|
|
|
72
86
|
## Requirements
|
|
73
87
|
|
|
74
|
-
- Node.js >= 22
|
|
88
|
+
- Node.js >= 22. The gateways use the built-in `fetch`, `AbortSignal.timeout` and `net.BlockList`
|
|
75
89
|
- TypeScript >= 5.7
|
|
76
90
|
- PostgreSQL or MSSQL (via Knex)
|
|
77
91
|
|
|
@@ -91,17 +105,21 @@ Business layer → Core classes (BaseCore subclasses)
|
|
|
91
105
|
validation, IP rate limiting, enforce business rules,
|
|
92
106
|
and normalise responses into a consistent OpResult shape.
|
|
93
107
|
|
|
94
|
-
Data layer → Gateway classes
|
|
95
|
-
|
|
96
|
-
queries with joins and filters
|
|
97
|
-
|
|
108
|
+
Data layer → Gateway classes
|
|
109
|
+
BaseGateway subclasses perform CRUD operations on database
|
|
110
|
+
tables and run complex queries with joins and filters.
|
|
111
|
+
BaseExternalApiGateway subclasses talk to outside services
|
|
112
|
+
(email, messaging, webhooks, payment providers).
|
|
98
113
|
```
|
|
99
114
|
|
|
100
115
|
Each inbound request — whether from an HTTP client, a GraphQL operation, or a CLI command — creates a fresh **ApiContext** that carries the database connection, all gateway instances, all core instances, and request-scoped data (userId, spaceId, headers, etc.).
|
|
101
116
|
|
|
102
117
|
**Key invariants:**
|
|
103
118
|
|
|
104
|
-
-
|
|
119
|
+
- Database gateways (`BaseGateway`) and cores are instantiated **per-request** and must never be reused across requests
|
|
120
|
+
- Gateways to outside services (`BaseExternalApiGateway`) hold no request state: create one once (or per request, if it
|
|
121
|
+
sits in your gateway barrel) and `configure()` it whenever its settings change. `configure` **replaces** the settings,
|
|
122
|
+
so nothing of one tenant carries over to the next
|
|
105
123
|
- The data layer (gateways) has no knowledge of HTTP, GraphQL, or authentication — it only handles data
|
|
106
124
|
- The business layer (cores) has no knowledge of transport details — it receives plain args and returns OpResults
|
|
107
125
|
- The application layer (controllers/resolvers/CLI) has no direct database access — it goes through cores
|
|
@@ -1101,38 +1119,430 @@ Routes registered for `root = '/api/notes'`:
|
|
|
1101
1119
|
|
|
1102
1120
|
### BaseExternalApiGateway
|
|
1103
1121
|
|
|
1104
|
-
|
|
1122
|
+
The base of every gateway to an outside service. **One way to set it up** — `configure(settings)` — and **one way to
|
|
1123
|
+
answer** — an `OpResult`, never a throw.
|
|
1124
|
+
|
|
1125
|
+
- A gateway is created **once** and configured — and re-configured — at run time: per brand, per tenant, per request.
|
|
1126
|
+
`configure` **replaces** the settings (it never merges), so a shared instance re-pointed at another tenant keeps
|
|
1127
|
+
nothing of the last one.
|
|
1128
|
+
- `isConfigured()` / `getConfigProblems()` say what is missing or malformed, in words — never a value. Every operation
|
|
1129
|
+
refuses with `VALIDATION_FAILED` until it is fixed.
|
|
1130
|
+
- `configure(settings)` takes the gateway's FULL settings type, so a missing required setting is a compile error.
|
|
1131
|
+
- HTTP refusals map onto the standard codes — 401 `UNAUTHORIZED`, 403 `FORBIDDEN`, 404 `NOT_FOUND`, 408 `TIMEOUT`,
|
|
1132
|
+
409 `CONFLICT`, 429 `TOO_MANY_REQUESTS`, 5xx `FAILED`, any other 4xx `VALIDATION_FAILED`; no answer `NETWORK_ERROR` /
|
|
1133
|
+
`TIMEOUT`; a request that cannot even be made (a malformed header, a refused address) `VALIDATION_FAILED` — so
|
|
1134
|
+
`result.isRetryableError()` means the same thing for every gateway.
|
|
1135
|
+
- Retries (default 2, after 1 s and 2 s) on network errors, timeouts, 429 and 5xx — after the service's own
|
|
1136
|
+
`Retry-After` when it gives one, and not at all when it asks for longer than 10 s. A gateway refuses to retry the
|
|
1137
|
+
hopeless ones (a chat that does not exist, an unknown webhook), and email/SMS never retry after a TIMEOUT (it may have
|
|
1138
|
+
gone out — a retry would send it twice).
|
|
1139
|
+
- Redirects are never followed; every request has a timeout (default 10 s); a reply is read at most 64 KB.
|
|
1140
|
+
- The secrets a gateway names (keys, tokens, webhook URLs) — and a request's own (per-request headers) — are scrubbed out
|
|
1141
|
+
of EVERYTHING it returns: the error message AND the reply in the data (body and headers; `set-cookie` is dropped).
|
|
1142
|
+
|
|
1143
|
+
Constructor props (all optional): `settings`, `timeoutMs`, `retries`, `retryDelaysMs`, `fetch` (a test passes its
|
|
1144
|
+
own), `logger`, `baseUrl`.
|
|
1145
|
+
|
|
1146
|
+
Writing a gateway:
|
|
1105
1147
|
|
|
1106
1148
|
```typescript
|
|
1107
|
-
|
|
1108
|
-
import { BaseExternalApiGateway } from '@sdflc/backend-helpers';
|
|
1109
|
-
import Stripe from 'stripe';
|
|
1149
|
+
import { BaseExternalApiGateway, OpResult } from '@sdflc/backend-helpers';
|
|
1110
1150
|
|
|
1111
|
-
interface
|
|
1112
|
-
|
|
1151
|
+
interface AcmeSettings {
|
|
1152
|
+
apiKey: string;
|
|
1113
1153
|
}
|
|
1114
1154
|
|
|
1115
|
-
class
|
|
1116
|
-
|
|
1155
|
+
class AcmeGw extends BaseExternalApiGateway<AcmeSettings> {
|
|
1156
|
+
protected validateSettings(settings: Partial<AcmeSettings>): string[] {
|
|
1157
|
+
return settings.apiKey ? [] : ['the API key is missing'];
|
|
1158
|
+
}
|
|
1117
1159
|
|
|
1118
|
-
protected
|
|
1119
|
-
|
|
1120
|
-
this.client = new Stripe(secretKey);
|
|
1160
|
+
protected secretValues(): string[] {
|
|
1161
|
+
return [String(this.settings.apiKey ?? '')];
|
|
1121
1162
|
}
|
|
1122
1163
|
|
|
1123
|
-
async
|
|
1124
|
-
this.
|
|
1125
|
-
|
|
1164
|
+
public async ping(): Promise<OpResult> {
|
|
1165
|
+
if (!this.isConfigured()) {
|
|
1166
|
+
return this.notConfigured();
|
|
1167
|
+
}
|
|
1168
|
+
|
|
1169
|
+
return this.requestWithRetries({
|
|
1170
|
+
url: 'https://api.acme.test/ping',
|
|
1171
|
+
headers: { Authorization: `Bearer ${this.settings.apiKey}` },
|
|
1172
|
+
json: { hello: 'world' },
|
|
1173
|
+
});
|
|
1126
1174
|
}
|
|
1127
1175
|
}
|
|
1128
1176
|
|
|
1129
|
-
|
|
1130
|
-
|
|
1131
|
-
const
|
|
1132
|
-
|
|
1133
|
-
|
|
1177
|
+
const acme = new AcmeGw(); // once
|
|
1178
|
+
acme.configure({ apiKey: await keyOf(spaceId) }); // per tenant, whenever
|
|
1179
|
+
const result = await acme.ping();
|
|
1180
|
+
```
|
|
1181
|
+
|
|
1182
|
+
What a subclass overrides, and what it uses:
|
|
1183
|
+
|
|
1184
|
+
| Member | Purpose |
|
|
1185
|
+
|---|---|
|
|
1186
|
+
| `validateSettings(settings)` | Override. What's missing or malformed, in words (never a value). Drives `isConfigured()` / `getConfigProblems()` |
|
|
1187
|
+
| `secretValues()` | Override. The values scrubbed out of everything the gateway returns (keys, tokens, secret URLs) |
|
|
1188
|
+
| `errorMessageOf(reply)` | Override. The refusal's words, from the service's own error shape |
|
|
1189
|
+
| `shouldRetry(result)` | Override. Refuse hopeless retries (default: network, timeout, 429, 5xx) |
|
|
1190
|
+
| `retryAfterMs(result)` | Override. How long the service asked to wait (default: the `Retry-After` header) |
|
|
1191
|
+
| `onConfigured()` | Override. Called after every `configure` (to rebuild an SDK client) |
|
|
1192
|
+
| `settings` | Use. The current settings (they hold secrets: never log or return them) |
|
|
1193
|
+
| `request(req)` / `requestWithRetries(req)` | Use. One HTTP attempt, or attempts with retries. `req`: `{ url, method?, headers?, json?, body?, timeoutMs?, secrets? }`; `secrets` are per-request values to scrub |
|
|
1194
|
+
| `notConfigured()` | Use. The standard refusal while the gateway isn't set up |
|
|
1195
|
+
| `scrub(text, extra?)` / `scrubValue(value, extra?)` | Use. Replace every secret with `***` |
|
|
1196
|
+
|
|
1197
|
+
ⓘ `setCredentials()` / `onCredentialsSet()` still work for gateways written before `configure()`.
|
|
1198
|
+
|
|
1199
|
+
---
|
|
1200
|
+
|
|
1201
|
+
## Gateways to outside services
|
|
1202
|
+
|
|
1203
|
+
Ready-made gateways on [`BaseExternalApiGateway`](#baseexternalapigateway): no SDKs, one REST call each, an `OpResult`
|
|
1204
|
+
out, never a throw. Create one **once**, then `configure()` it whenever the settings change (per brand, per tenant, per
|
|
1205
|
+
request). `configure()` **replaces** the settings; it never merges them.
|
|
1206
|
+
|
|
1207
|
+
| Gateway | `configure({...})` | Operation | OK data |
|
|
1208
|
+
|---|---|---|---|
|
|
1209
|
+
| [`SendGridGw`](#sendgridgw) | `apiKey`, `fromEmail?`, `fromName?` | `sendEmail(email)` | `{ messageId }` |
|
|
1210
|
+
| [`TelegramGw`](#telegramgw) | `botToken`, `chatId?`, `parseMode?` | `sendMessage(message)` | `{ messageId }` |
|
|
1211
|
+
| [`TwilioGw`](#twiliogw) | `accountSid`, `authToken`, `from?`, `messagingServiceSid?` | `sendMessage(message)` | `{ sid, status }` |
|
|
1212
|
+
| [`SlackWebhookGw`](#slackwebhookgw) | `webhookUrl` | `postMessage(message)` | `{ messageId: null }` |
|
|
1213
|
+
| [`DiscordWebhookGw`](#discordwebhookgw) | `webhookUrl`, `username?`, `avatarUrl?` | `postMessage(message)` | `{ messageId }` |
|
|
1214
|
+
| [`HttpWebhookGw`](#httpwebhookgw) | `url`, `method?`, `headers?`, `signingSecret?` | `send(request)` | `{ status, body }` |
|
|
1215
|
+
|
|
1216
|
+
Every operation answers the same way:
|
|
1217
|
+
|
|
1218
|
+
| Outcome | `OpResult` code | Retried? |
|
|
1219
|
+
|---|---|---|
|
|
1220
|
+
| Delivered | `OK`, with the data above | — |
|
|
1221
|
+
| Not configured, or the message itself is wrong (no recipient, too long, a bad number…) | `VALIDATION_FAILED` (nothing is sent) | no |
|
|
1222
|
+
| The service refused it | the code for its HTTP status: `UNAUTHORIZED` 401, `FORBIDDEN` 403, `NOT_FOUND` 404, `CONFLICT` 409, `VALIDATION_FAILED` other 4xx | no |
|
|
1223
|
+
| Throttled | `TOO_MANY_REQUESTS` 429 | yes, after its `Retry-After` |
|
|
1224
|
+
| The service failed | `FAILED` 5xx | yes |
|
|
1225
|
+
| No answer | `NETWORK_ERROR`, or `TIMEOUT` (also 408) | yes (email and SMS: not after a timeout) |
|
|
1226
|
+
|
|
1227
|
+
The message is the service's own words (`result.getErrorSummary('')`). The data carries `{ status, headers, body }` of
|
|
1228
|
+
its reply, scrubbed of every secret.
|
|
1229
|
+
|
|
1230
|
+
```typescript
|
|
1231
|
+
import { SendGridGw } from '@sdflc/backend-helpers';
|
|
1232
|
+
|
|
1233
|
+
const mail = new SendGridGw(); // once
|
|
1234
|
+
mail.configure({ apiKey: brand.sendgridKey, fromEmail: brand.fromEmail, fromName: brand.name }); // per brand, whenever
|
|
1235
|
+
|
|
1236
|
+
const sent = await mail.sendEmail({ to: 'guest@example.com', subject: 'Welcome', html: '<p>Hi</p>' });
|
|
1237
|
+
|
|
1238
|
+
if (sent.didFail()) {
|
|
1239
|
+
logger.error(`welcome email: ${sent.getErrorSummary('')}`); // the service's words, no key in them
|
|
1240
|
+
}
|
|
1241
|
+
```
|
|
1242
|
+
|
|
1243
|
+
### SendGridGw
|
|
1244
|
+
|
|
1245
|
+
Email through SendGrid's v3 Mail Send API: plain text, HTML, or a dynamic template.
|
|
1246
|
+
|
|
1247
|
+
```typescript
|
|
1248
|
+
const mail = new SendGridGw().configure({ apiKey, fromEmail: 'hello@brand.com', fromName: 'Brand' });
|
|
1249
|
+
|
|
1250
|
+
await mail.sendEmail({
|
|
1251
|
+
to: ['a@example.com', { email: 'b@example.com', name: 'B' }], // a SendGridAddress, or several
|
|
1252
|
+
cc: 'c@example.com',
|
|
1253
|
+
replyTo: 'support@brand.com',
|
|
1254
|
+
subject: 'Your ticket',
|
|
1255
|
+
text: 'Plain version',
|
|
1256
|
+
html: '<p>Rich version</p>',
|
|
1257
|
+
categories: ['tickets'],
|
|
1258
|
+
customArgs: { orderId: 'o-1' },
|
|
1259
|
+
});
|
|
1260
|
+
|
|
1261
|
+
await mail.sendEmail({ to: 'a@example.com', templateId: 'd-123', dynamicTemplateData: { name: 'A' } }); // no subject needed
|
|
1262
|
+
```
|
|
1263
|
+
|
|
1264
|
+
- The From (`from`, else the configured `fromEmail`) must be on a domain authenticated in that SendGrid account;
|
|
1265
|
+
SendGrid refuses anything else.
|
|
1266
|
+
- A malformed recipient is **refused**, never silently dropped.
|
|
1267
|
+
- Not retried after a timeout: SendGrid may well have accepted the mail, and a retry would send it twice.
|
|
1268
|
+
|
|
1269
|
+
### TelegramGw
|
|
1270
|
+
|
|
1271
|
+
Messages through the Telegram Bot API.
|
|
1272
|
+
|
|
1273
|
+
```typescript
|
|
1274
|
+
const telegram = new TelegramGw().configure({ botToken: '123456:ABC…', chatId: '-1001234567890' });
|
|
1275
|
+
|
|
1276
|
+
await telegram.sendMessage({ text: '<b>Deployed</b> v1.2' }); // HTML by default
|
|
1277
|
+
await telegram.sendMessage({ text: 'plain', parseMode: null, chatId: '@ops_channel' }); // another chat, no markup
|
|
1278
|
+
await telegram.sendMessage({ text: 'Open', inlineKeyboard: [[{ text: 'Dashboard', url: 'https://…' }]] });
|
|
1279
|
+
```
|
|
1280
|
+
|
|
1281
|
+
- A chat is a numeric id (negative for a group) or `@channelname`; `isTelegramChatId(value)` checks one.
|
|
1282
|
+
- A text over 4096 characters is **refused, not cut**. It's counted the way Telegram counts it (`telegramVisibleLength`:
|
|
1283
|
+
tags are free, an entity is one character). Cutting finished HTML can split an entity, and Telegram then refuses the
|
|
1284
|
+
whole message, so cut your text **before** adding markup.
|
|
1285
|
+
- A 200 with `ok: false` is a failure. A permanent refusal (chat not found, bot blocked, broken markup) is not retried.
|
|
1286
|
+
Telegram's own `retry_after` is honoured.
|
|
1287
|
+
- The bot token sits in the request URL. It is scrubbed out of everything returned.
|
|
1288
|
+
|
|
1289
|
+
### TwilioGw
|
|
1290
|
+
|
|
1291
|
+
SMS, or WhatsApp with `whatsapp:+…` addresses, through Twilio's Messages API.
|
|
1292
|
+
|
|
1293
|
+
```typescript
|
|
1294
|
+
const twilio = new TwilioGw().configure({ accountSid: 'AC…', authToken, from: '+15551234567' });
|
|
1295
|
+
// or configure({ accountSid, authToken, messagingServiceSid: 'MG…' }) to send through a messaging service
|
|
1296
|
+
|
|
1297
|
+
const result = await twilio.sendMessage({ to: '+15557654321', body: 'Your code is 123456', statusCallback: 'https://…' });
|
|
1298
|
+
// OK → [{ sid: 'SM…', status: 'queued' }]
|
|
1299
|
+
```
|
|
1300
|
+
|
|
1301
|
+
- Numbers must be E.164. A body over 1600 characters is refused before anything is sent.
|
|
1302
|
+
- Not retried after a timeout (it may have gone out, and a retry texts the person twice).
|
|
1303
|
+
- The auth token (and the Basic credentials built from it) is scrubbed out of everything returned.
|
|
1304
|
+
|
|
1305
|
+
### SlackWebhookGw
|
|
1306
|
+
|
|
1307
|
+
Posts to a Slack channel through an incoming webhook.
|
|
1308
|
+
|
|
1309
|
+
```typescript
|
|
1310
|
+
const slack = new SlackWebhookGw().configure({ webhookUrl: 'https://hooks.slack.com/services/…' });
|
|
1311
|
+
|
|
1312
|
+
await slack.postMessage({ text: 'Deployed', blocks: [/* Block Kit */] });
|
|
1313
|
+
```
|
|
1314
|
+
|
|
1315
|
+
- The webhook URL **is** the credential; it is scrubbed out of everything returned. Only `https://hooks.slack.com/…` is
|
|
1316
|
+
accepted.
|
|
1317
|
+
- Slack's webhook answers no id, so the OK data is `{ messageId: null }`, the same shape as the others.
|
|
1318
|
+
|
|
1319
|
+
### DiscordWebhookGw
|
|
1320
|
+
|
|
1321
|
+
Posts to a Discord channel through a webhook.
|
|
1322
|
+
|
|
1323
|
+
```typescript
|
|
1324
|
+
const discord = new DiscordWebhookGw().configure({ webhookUrl: 'https://discord.com/api/webhooks/…', username: 'Alerts' });
|
|
1325
|
+
|
|
1326
|
+
await discord.postMessage({ content: 'Deployed', embeds: [{ title: 'v1.2', color: 0x5865f2 }] });
|
|
1327
|
+
```
|
|
1328
|
+
|
|
1329
|
+
- Content is at most 2000 characters, with at most 10 embeds.
|
|
1330
|
+
- **Nobody is pinged** unless `allowedMentions` says so: `@everyone` in a text stays text.
|
|
1331
|
+
- An unknown webhook or a bad webhook token is not retried. The webhook URL is scrubbed out of everything returned.
|
|
1332
|
+
|
|
1333
|
+
### HttpWebhookGw
|
|
1334
|
+
|
|
1335
|
+
Sends JSON to any URL, for anything without a gateway of its own, optionally **signed**.
|
|
1336
|
+
|
|
1337
|
+
```typescript
|
|
1338
|
+
const hook = new HttpWebhookGw(); // once: the address protection is on
|
|
1339
|
+
hook.configure({ url: tenant.webhookUrl, headers: { 'X-Api-Key': tenant.apiKey }, signingSecret: tenant.signingSecret });
|
|
1340
|
+
|
|
1341
|
+
const result = await hook.send({ payload: { event: 'order.paid', id }, headers: { 'X-Trace': traceId } });
|
|
1342
|
+
// OK → [{ status: 200, body: … }] (the receiver's answer, scrubbed)
|
|
1343
|
+
```
|
|
1344
|
+
|
|
1345
|
+
- **The URL a user typed must not reach your own network** (see [Outbound URL safety](#outbound-url-safety)). It is
|
|
1346
|
+
checked before the request, and the address the socket actually connects to is checked again at connect time.
|
|
1347
|
+
- `allowPrivateNetwork: true` is a **constructor** option, never a setting: `new HttpWebhookGw({ allowPrivateNetwork: true })`.
|
|
1348
|
+
Use it only for a URL **you** configure (an internal service). Settings often come from a tenant's own row, and a
|
|
1349
|
+
tenant must not be able to switch the protection off.
|
|
1350
|
+
- Every send carries `X-Webhook-Id`, which stays the same on its retries so a receiver can drop a duplicate.
|
|
1351
|
+
- With `signingSecret` it also carries `X-Webhook-Timestamp` (unix seconds) and
|
|
1352
|
+
`X-Webhook-Signature: sha256=<hex HMAC-SHA256 of "<timestamp>.<body>">`. A configured or per-request header can never
|
|
1353
|
+
replace these.
|
|
1354
|
+
- A header with a line break, or a payload that can't be sent as JSON (a function), is refused.
|
|
1355
|
+
- Every header value is scrubbed out of everything returned, configured or per-request, whole and word by word.
|
|
1356
|
+
|
|
1357
|
+
A receiver checks a signed request like this:
|
|
1358
|
+
|
|
1359
|
+
```typescript
|
|
1360
|
+
import { timingSafeEqual } from 'crypto';
|
|
1361
|
+
import { webhookSignatureOf } from '@sdflc/backend-helpers';
|
|
1362
|
+
|
|
1363
|
+
const timestamp = Number(req.headers['x-webhook-timestamp']);
|
|
1364
|
+
const expected = Buffer.from(webhookSignatureOf(secret, timestamp, rawBody)); // the RAW body, before JSON parsing
|
|
1365
|
+
const given = Buffer.from(String(req.headers['x-webhook-signature'] ?? ''));
|
|
1366
|
+
const fresh = Math.abs(Date.now() / 1000 - timestamp) < 300;
|
|
1367
|
+
const genuine = fresh && expected.length === given.length && timingSafeEqual(expected, given);
|
|
1368
|
+
```
|
|
1369
|
+
|
|
1370
|
+
### Outbound URL safety
|
|
1371
|
+
|
|
1372
|
+
For any code that sends a request to a URL a **user** gave (webhooks, callbacks, "import from URL"):
|
|
1373
|
+
|
|
1374
|
+
- **`checkExternalUrl(url, { protocols?, lookup?, field? })`**: `OK` when the URL may be requested, else
|
|
1375
|
+
`VALIDATION_FAILED` saying why. It refuses:
|
|
1376
|
+
- anything but http(s);
|
|
1377
|
+
- a user name or password in the URL;
|
|
1378
|
+
- `localhost`, `*.localhost` and `*.internal` names, and the cloud metadata host names;
|
|
1379
|
+
- any host that is, or resolves to, a blocked address. **Every** address it resolves to is checked: a name with one
|
|
1380
|
+
private address among public ones is refused.
|
|
1381
|
+
- **`isBlockedAddress(ip)`**: true for the addresses a user-given URL must never reach, however they're written. IPv4:
|
|
1382
|
+
this-network, private, carrier-grade NAT, loopback, link-local (169.254.169.254, the metadata service), protocol
|
|
1383
|
+
assignments, test networks, the 6to4 relay, benchmarking, multicast, reserved. IPv6: unspecified, loopback,
|
|
1384
|
+
IPv4-compatible, IPv4-mapped (checked as IPv4), NAT64 (both prefixes), 6to4, discard-only, documentation,
|
|
1385
|
+
unique-local `fc00::/7`, link-local, multicast.
|
|
1386
|
+
- **`createGuardedFetch({ dnsLookup?, isBlocked? })`**: a `fetch` that checks the address **at the moment the socket
|
|
1387
|
+
connects**. A check made only beforehand loses to DNS rebinding (a server that answers a public address first and an
|
|
1388
|
+
internal one to the second lookup). It also checks an IP written in the URL directly, never follows redirects, reads
|
|
1389
|
+
at most 64 KB, and ends at its signal. `HttpWebhookGw` uses it by default.
|
|
1390
|
+
|
|
1391
|
+
### Testing a gateway
|
|
1392
|
+
|
|
1393
|
+
Every gateway takes a `fetch` in its constructor props, so a test runs the real gateway against a stub and nothing leaves
|
|
1394
|
+
the machine:
|
|
1395
|
+
|
|
1396
|
+
```typescript
|
|
1397
|
+
const calls: any[] = [];
|
|
1398
|
+
const fetch = async (url: string, init: any) => {
|
|
1399
|
+
calls.push({ url, body: JSON.parse(init.body) });
|
|
1400
|
+
|
|
1401
|
+
return { status: 202, headers: { forEach: () => null }, text: async () => '' };
|
|
1402
|
+
};
|
|
1403
|
+
|
|
1404
|
+
const mail = new SendGridGw({ fetch, retries: 0 }).configure({ apiKey: 'SG.test' });
|
|
1405
|
+
const result = await mail.sendEmail({ to: 'a@b.test', from: 'x@b.test', subject: 's', text: 't' });
|
|
1406
|
+
// result.didSucceed() === true; calls[0].body.personalizations[0].to[0].email === 'a@b.test'
|
|
1407
|
+
```
|
|
1408
|
+
|
|
1409
|
+
`HttpWebhookGw` also takes `lookup` (the pre-check's resolver) and `dnsLookup` (the connect-time resolver).
|
|
1410
|
+
|
|
1411
|
+
---
|
|
1412
|
+
|
|
1413
|
+
## Emergency alerts
|
|
1414
|
+
|
|
1415
|
+
A service raises an alarm **by itself**, by email and/or Telegram, when it starts with missing configuration or
|
|
1416
|
+
crashes. `EmergencyAlerter` decides **what** is said, **to whom** and **how often**. It sends through the gateways above
|
|
1417
|
+
(`SendGridGw`, `TelegramGw`) and contains no transport code itself. Each alert names the app, the service and the
|
|
1418
|
+
environment (`[partylabz · ms_storage · production]`), because several apps can run the same services side by side on
|
|
1419
|
+
one machine.
|
|
1420
|
+
|
|
1421
|
+
### Wiring it into a service
|
|
1422
|
+
|
|
1423
|
+
```typescript
|
|
1424
|
+
// src/index.ts
|
|
1425
|
+
import { EmergencyAlerter, CriticalConfigItem } from '@sdflc/backend-helpers';
|
|
1426
|
+
|
|
1427
|
+
// What this service starts WITHOUT (optional to the config loader) but cannot fully work without.
|
|
1428
|
+
// APP_NAME is always checked as well. `key` (the built-config name) is derived from `env`; pass it only when it differs.
|
|
1429
|
+
const CRITICAL_CONFIG: CriticalConfigItem[] = [
|
|
1430
|
+
{ env: 'INTERSERVICE_API_KEY', effect: 'Every call from our other services is refused.' },
|
|
1431
|
+
];
|
|
1432
|
+
|
|
1433
|
+
const alerter = EmergencyAlerter.fromConfig(config, 'ms_storage', { log: (m) => logger.error(m) });
|
|
1434
|
+
|
|
1435
|
+
alerter.installCrashHandlers(process); // uncaught exception / unhandled rejection → alert, then exit(1)
|
|
1436
|
+
|
|
1437
|
+
const run = async () => {
|
|
1438
|
+
const app = express();
|
|
1439
|
+
|
|
1440
|
+
configureApp(app);
|
|
1441
|
+
|
|
1442
|
+
app.listen(config.port, async () => {
|
|
1443
|
+
logger.log(`Server started on port ${config.port}`);
|
|
1444
|
+
await alerter.checkConfig(config, CRITICAL_CONFIG); // the service still starts; it says loudly what is missing
|
|
1445
|
+
});
|
|
1446
|
+
};
|
|
1447
|
+
|
|
1448
|
+
run().catch(async (e) => {
|
|
1449
|
+
await alerter.send({ key: 'crash:startup', subject: `Failed to start: ${e?.message ?? e}`, lines: [String(e?.stack ?? e)] });
|
|
1450
|
+
process.exit(1);
|
|
1451
|
+
});
|
|
1452
|
+
```
|
|
1453
|
+
|
|
1454
|
+
Anything else worth waking someone for:
|
|
1455
|
+
|
|
1456
|
+
```typescript
|
|
1457
|
+
const result = await alerter.send({ key: 'queue:stalled', subject: 'The queue stopped draining', lines: ['…'] });
|
|
1458
|
+
```
|
|
1459
|
+
|
|
1460
|
+
### The settings fromConfig reads
|
|
1461
|
+
|
|
1462
|
+
From the built config (`buildConfig`). All of them are optional, and each channel needs all of its own. Leave them
|
|
1463
|
+
commented out in `.env.example` so a service without them still starts.
|
|
1464
|
+
|
|
1465
|
+
| Env | Meaning |
|
|
1466
|
+
|---|---|
|
|
1467
|
+
| `APP_NAME` | The app this service belongs to. Alerts say `unknown-app` without it, and `checkConfig` reports it as missing |
|
|
1468
|
+
| `EMERGENCY_SENDGRID_API_KEY` | Email: a SendGrid key allowed to send mail |
|
|
1469
|
+
| `EMERGENCY_EMAIL_FROM` | Email: a sender on a domain authenticated in that SendGrid account. Required: a guessed one is refused |
|
|
1470
|
+
| `EMERGENCY_EMAIL_TO` | Email: recipients, comma-separated |
|
|
1471
|
+
| `TELEGRAM_BOT_ID` | Telegram: the bot token `<id>:<secret>` from @BotFather |
|
|
1472
|
+
| `TELEGRAM_CHAT_ID` | Telegram: the chat id (negative for a group) or `@channel`. The bot must be in it |
|
|
1473
|
+
| `EMERGENCY_EMAIL_MIN_INTERVAL_MINUTES` | The same alert at most once per this many minutes, on every channel. Default 60; 0 = every time |
|
|
1474
|
+
|
|
1475
|
+
`fromConfig(config, serviceName, { log?, redact?, memory?, now?, hostname? })` gives each provider a 4 s timeout and one
|
|
1476
|
+
short retry, so both channels together fit the crash wait. It says once, through `log`, when a channel was set up but
|
|
1477
|
+
can't be used (for example a malformed token, or an email missing its From).
|
|
1478
|
+
|
|
1479
|
+
### Building it yourself
|
|
1480
|
+
|
|
1481
|
+
Credentials live in the gateways; who to alert lives in the alerter.
|
|
1482
|
+
|
|
1483
|
+
```typescript
|
|
1484
|
+
const alerter = new EmergencyAlerter({
|
|
1485
|
+
email: new SendGridGw({ timeoutMs: 4000, retries: 1 }).configure({ apiKey }),
|
|
1486
|
+
telegram: new TelegramGw({ timeoutMs: 4000, retries: 1 }).configure({ botToken }),
|
|
1487
|
+
settings: {
|
|
1488
|
+
appName: 'partylabz',
|
|
1489
|
+
serviceName: 'ms_storage',
|
|
1490
|
+
environment: 'production',
|
|
1491
|
+
minIntervalMinutes: 60,
|
|
1492
|
+
emailTo: ['ops@brand.com'],
|
|
1493
|
+
emailFrom: 'alerts@brand.com',
|
|
1494
|
+
telegramChatId: '-100123',
|
|
1495
|
+
},
|
|
1496
|
+
log: (m) => logger.error(m),
|
|
1497
|
+
redact: (text) => text.replace(/password=\S+/g, 'password=***'),
|
|
1498
|
+
});
|
|
1134
1499
|
```
|
|
1135
1500
|
|
|
1501
|
+
`configure(settings)` **replaces** the settings, and `useEmail(gateway | null)` / `useTelegram(gateway | null)` swap a
|
|
1502
|
+
channel's gateway.
|
|
1503
|
+
|
|
1504
|
+
### What it answers
|
|
1505
|
+
|
|
1506
|
+
Every method returns an `OpResult` and never throws.
|
|
1507
|
+
|
|
1508
|
+
| Call | OK | Not OK |
|
|
1509
|
+
|---|---|---|
|
|
1510
|
+
| `send(alert)` | `[{ status: 'sent', channels: { email, telegram } }]` (each `'sent'`, `'failed'` or `'off'`), or `[{ status: 'recently-sent' }]` | `VALIDATION_FAILED` `[{ status: 'unconfigured' }]` when no channel can be used; `FAILED` `[{ status: 'failed', channels }]` when no channel delivered it, with each channel's error merged in |
|
|
1511
|
+
| `checkConfig(config, items)` | `[]` when nothing is missing | otherwise whatever `send` answered for the missing-configuration alert |
|
|
1512
|
+
|
|
1513
|
+
Also available:
|
|
1514
|
+
- `getLabel()`: `app · service · environment`.
|
|
1515
|
+
- `getChannelProblems()`: why each channel can't be used, in words, never a value.
|
|
1516
|
+
- `getMinIntervalMs()`.
|
|
1517
|
+
- `emailOf(alert, host, at)` and `telegramTextOf(alert, host, at)`: exactly what would be sent.
|
|
1518
|
+
- The pure helpers `missingCriticalConfig(config, items)`, `missingConfigAlert(config, items)`, `emergencyOneLine(value, max)`,
|
|
1519
|
+
`emergencyIntervalText(ms)` and `emergencyFileMemory(app, service, dir?)`.
|
|
1520
|
+
|
|
1521
|
+
### The rules it keeps
|
|
1522
|
+
|
|
1523
|
+
- **Never throws, never blocks for long.** Even a `log` or `redact` that throws can't make it throw. A crash waits for its
|
|
1524
|
+
alert at most 10 s, then exits, and **always** exits, whatever was thrown (even a value that can't be turned into
|
|
1525
|
+
text).
|
|
1526
|
+
- **One alert per key per interval, across all channels.** It's remembered in a file under the OS temp folder named after
|
|
1527
|
+
the app and service, so a crash loop sends one message rather than one per restart.
|
|
1528
|
+
- Sends of the same key **at the same time** go out once.
|
|
1529
|
+
- A send that no channel delivered is not remembered, so the next one tries again.
|
|
1530
|
+
- A remembered time in the future (a clock moved back) doesn't silence anything.
|
|
1531
|
+
- **A crash's key names which crash** (`crash:<kind>:<fingerprint of the message and top frame>`). The same crash in a
|
|
1532
|
+
loop is sent once, and a different crash is never silenced by it.
|
|
1533
|
+
- **A channel set up wrongly is said once, at the start**, never silently skipped.
|
|
1534
|
+
- **What leaves is redacted.** `fromConfig` removes every config value whose **name** looks secret (password, secret,
|
|
1535
|
+
token, key…) from every subject, line and log line, because an error message can carry SQL with its values. Pass
|
|
1536
|
+
`redact` for more.
|
|
1537
|
+
- **Subjects are one line**, because a crash's subject is the error's message, which often spans lines. The Telegram text
|
|
1538
|
+
is cut **before** it's escaped, to whatever the heading and footer leave, so a long stack trace neither splits an
|
|
1539
|
+
entity nor overruns Telegram's limit.
|
|
1540
|
+
- **It goes straight to the providers**, never through another service of yours. It has to work when the rest doesn't.
|
|
1541
|
+
- **Never put a secret, a token or a database row in an alert yourself.** It leaves your infrastructure.
|
|
1542
|
+
|
|
1543
|
+
Proofs: `npm run build && npm run prove`. They cover the gateways and the alerter, each against a stub `fetch`, plus a
|
|
1544
|
+
real local server for the connect-time address check.
|
|
1545
|
+
|
|
1136
1546
|
---
|
|
1137
1547
|
|
|
1138
1548
|
## IP rate limiting
|
|
@@ -2021,6 +2431,22 @@ ORDER_BYS.ASC; // 'asc'
|
|
|
2021
2431
|
ORDER_BYS.DESC; // 'desc'
|
|
2022
2432
|
```
|
|
2023
2433
|
|
|
2434
|
+
Gateways and emergency alerts:
|
|
2435
|
+
|
|
2436
|
+
```typescript
|
|
2437
|
+
MAX_REPLY_BYTES; // 65536: the most of a reply a gateway reads
|
|
2438
|
+
MAX_RETRY_WAIT_MS; // 10000: a longer Retry-After is not waited for (the failure comes back at once)
|
|
2439
|
+
TELEGRAM_MAX_MESSAGE_LENGTH; // 4096
|
|
2440
|
+
TWILIO_MAX_BODY_LENGTH; // 1600
|
|
2441
|
+
DISCORD_MAX_CONTENT_LENGTH; // 2000
|
|
2442
|
+
DISCORD_MAX_EMBEDS; // 10
|
|
2443
|
+
WEBHOOK_ID_HEADER; // 'X-Webhook-Id'
|
|
2444
|
+
WEBHOOK_TIMESTAMP_HEADER; // 'X-Webhook-Timestamp'
|
|
2445
|
+
WEBHOOK_SIGNATURE_HEADER; // 'X-Webhook-Signature'
|
|
2446
|
+
UNKNOWN_APP_NAME; // 'unknown-app': what an alert says without APP_NAME
|
|
2447
|
+
APP_NAME_CONFIG; // the CriticalConfigItem every checkConfig includes
|
|
2448
|
+
```
|
|
2449
|
+
|
|
2024
2450
|
---
|
|
2025
2451
|
|
|
2026
2452
|
## Interfaces reference
|
|
@@ -2032,7 +2458,23 @@ ORDER_BYS.DESC; // 'desc'
|
|
|
2032
2458
|
| `BaseCorePropsInterface` | Constructor props for `BaseCore` subclasses |
|
|
2033
2459
|
| `BaseCoreRateLimitOnActionInterface` | Per-action rate-limit rule map used in `BaseCorePropsInterface` |
|
|
2034
2460
|
| `BaseGatewayPropsInterface` | Constructor props for `BaseGateway` subclasses — includes `statusField`, `nonCreatableFields`, `nonUpdatableFields`, and the filter rule arrays |
|
|
2035
|
-
| `BaseExternalApiGatewayPropsInterface` | Constructor props for `BaseExternalApiGateway`
|
|
2461
|
+
| `BaseExternalApiGatewayPropsInterface` | Constructor props for `BaseExternalApiGateway`: `settings`, `timeoutMs`, `retries`, `retryDelaysMs`, `fetch`, `logger`, `baseUrl`, `credentials` (legacy) |
|
|
2462
|
+
| `ExternalApiFetch` | The HTTP client shape a gateway calls (the platform `fetch`, or a test's stub) |
|
|
2463
|
+
| `ExternalApiRequest` | One outbound request for `request()` / `requestWithRetries()`: `{ url, method?, headers?, json?, body?, timeoutMs?, secrets? }` |
|
|
2464
|
+
| `ExternalApiReply` | What a request answered, in its OpResult data: `{ status, headers, body }`, scrubbed and capped |
|
|
2465
|
+
| `SendGridSettings` / `SendGridEmail` / `SendGridAddress` | `SendGridGw` settings, the email, an address (`'a@b.com'` or `{ email, name? }`) |
|
|
2466
|
+
| `TelegramSettings` / `TelegramMessage` | `TelegramGw` settings and message |
|
|
2467
|
+
| `TwilioSettings` / `TwilioMessage` | `TwilioGw` settings and message |
|
|
2468
|
+
| `SlackWebhookSettings` / `SlackMessage` | `SlackWebhookGw` settings and message |
|
|
2469
|
+
| `DiscordWebhookSettings` / `DiscordMessage` / `DiscordEmbed` | `DiscordWebhookGw` settings, message and embed |
|
|
2470
|
+
| `HttpWebhookSettings` / `HttpWebhookSend` / `HttpWebhookGwProps` | `HttpWebhookGw` settings, one send, and its constructor props (`allowPrivateNetwork`, `lookup`, `dnsLookup`) |
|
|
2471
|
+
| `ExternalUrlOptions` | Options for `checkExternalUrl()`: `{ protocols?, lookup?, field? }` |
|
|
2472
|
+
| `GuardedFetchOptions` | Options for `createGuardedFetch()`: `{ dnsLookup?, isBlocked? }` |
|
|
2473
|
+
| `EmergencyAlert` | One alert: `{ key, subject, lines }` |
|
|
2474
|
+
| `EmergencyAlerterSettings` | `EmergencyAlerter` settings: `appName`, `serviceName`, `environment`, `minIntervalMinutes`, `emailTo`, `emailFrom`, `telegramChatId` |
|
|
2475
|
+
| `EmergencyAlerterProps` | `EmergencyAlerter` constructor props: `settings`, `email`, `telegram`, `log`, `redact`, `memory`, `now`, `hostname` |
|
|
2476
|
+
| `EmergencyAlertMemory` | Where sent alerts are remembered: `{ read, write }` (default: a file under the OS temp folder, per app and service) |
|
|
2477
|
+
| `CriticalConfigItem` | A setting a service cannot fully work without: `{ env, key?, effect }` |
|
|
2036
2478
|
| `BaseRestApiCrudControllerPropsInterface` | Constructor props for `BaseRestApiCrudController` |
|
|
2037
2479
|
| `BuildDefaultResolvers` | Arguments for `buildDefaultResolvers()` |
|
|
2038
2480
|
| `BuildListQueryArgsInterface` | Arguments passed to `buildListQuery()` overrides: `{ filter, params, fieldsToUse, useCache }` |
|