@sdflc/backend-helpers 1.0.141 → 1.0.143
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -16
- package/build/BaseGateway.d.ts +7 -0
- package/build/BaseGateway.js +52 -1
- package/build/EmergencyAlerter.d.ts +35 -0
- package/build/EmergencyAlerter.js +134 -7
- package/build/esm/BaseGateway.d.ts +7 -0
- package/build/esm/BaseGateway.js +52 -1
- package/build/esm/EmergencyAlerter.d.ts +35 -0
- package/build/esm/EmergencyAlerter.js +132 -6
- package/build/esm/interfaces/BaseGatewayPropsInterface.d.ts +11 -0
- package/build/interfaces/BaseGatewayPropsInterface.d.ts +11 -0
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1434,23 +1434,32 @@ const alerter = EmergencyAlerter.fromConfig(config, 'ms_storage', { log: (m) =>
|
|
|
1434
1434
|
|
|
1435
1435
|
alerter.installCrashHandlers(process); // uncaught exception / unhandled rejection → alert, then exit(1)
|
|
1436
1436
|
|
|
1437
|
-
|
|
1438
|
-
|
|
1439
|
-
|
|
1440
|
-
configureApp(
|
|
1441
|
-
|
|
1442
|
-
|
|
1437
|
+
// The whole start. `start` builds the app and answers the HTTP server to listen on (for Express + Apollo, the
|
|
1438
|
+
// http.Server configureApp creates — listening on it is what makes Apollo's drain plugin work).
|
|
1439
|
+
void alerter.startService({
|
|
1440
|
+
start: () => configureApp(express()),
|
|
1441
|
+
port: config.port,
|
|
1442
|
+
config, // checked AFTER the server listens: the service still starts, and says loudly what is missing
|
|
1443
|
+
critical: CRITICAL_CONFIG,
|
|
1444
|
+
afterListen: () => {
|
|
1443
1445
|
logger.log(`Server started on port ${config.port}`);
|
|
1444
|
-
|
|
1445
|
-
}
|
|
1446
|
-
};
|
|
1447
|
-
|
|
1448
|
-
run().catch(async (e) => {
|
|
1449
|
-
await alerter.send({ key: 'crash:startup', subject: `Failed to start: ${e?.message ?? e}`, lines: [String(e?.stack ?? e)] });
|
|
1450
|
-
process.exit(1);
|
|
1446
|
+
startBackgroundJobs();
|
|
1447
|
+
},
|
|
1451
1448
|
});
|
|
1452
1449
|
```
|
|
1453
1450
|
|
|
1451
|
+
`startService(opts)` answers `[{ status: 'listening', port }]`, and never throws:
|
|
1452
|
+
|
|
1453
|
+
- **A start that fails** — `start` throws, the port is in use, the port is not a port — is alerted (keyed
|
|
1454
|
+
`crash:startup:<fingerprint>`, so one failure does not silence a different one), then `exit(1)`.
|
|
1455
|
+
- **A start that hangs** is alerted once after `readyTimeoutMs` (default `START_READY_TIMEOUT_MS`, a minute) and keeps
|
|
1456
|
+
waiting; when it listens at last, that is said too. ⚠ This is the common one: Redis's client retries a refused
|
|
1457
|
+
connection for ever, so an unreachable Redis (or a wrong `REDIS_PASSWORD`) leaves the port closed with nothing but a
|
|
1458
|
+
log line every half second.
|
|
1459
|
+
- **`afterListen` failing** is alerted, never fatal — the service is already answering.
|
|
1460
|
+
- Options: `start`, `port`, `config?` + `critical?` (omit `config` to skip the check), `afterListen?`, `readyTimeoutMs?`,
|
|
1461
|
+
`exit?` (default `process.exit`).
|
|
1462
|
+
|
|
1454
1463
|
Anything else worth waking someone for:
|
|
1455
1464
|
|
|
1456
1465
|
```typescript
|
|
@@ -1509,6 +1518,7 @@ Every method returns an `OpResult` and never throws.
|
|
|
1509
1518
|
|---|---|---|
|
|
1510
1519
|
| `send(alert)` | `[{ status: 'sent', channels: { email, telegram } }]` (each `'sent'`, `'failed'` or `'off'`), or `[{ status: 'recently-sent' }]` | `VALIDATION_FAILED` `[{ status: 'unconfigured' }]` when no channel can be used; `FAILED` `[{ status: 'failed', channels }]` when no channel delivered it, with each channel's error merged in |
|
|
1511
1520
|
| `checkConfig(config, items)` | `[]` when nothing is missing | otherwise whatever `send` answered for the missing-configuration alert |
|
|
1521
|
+
| `startService(opts)` | `[{ status: 'listening', port }]` | the start failed — alerted, then `exit(1)` — with the error |
|
|
1512
1522
|
|
|
1513
1523
|
Also available:
|
|
1514
1524
|
- `getLabel()`: `app · service · environment`.
|
|
@@ -1516,7 +1526,7 @@ Also available:
|
|
|
1516
1526
|
- `getMinIntervalMs()`.
|
|
1517
1527
|
- `emailOf(alert, host, at)` and `telegramTextOf(alert, host, at)`: exactly what would be sent.
|
|
1518
1528
|
- The pure helpers `missingCriticalConfig(config, items)`, `missingConfigAlert(config, items)`, `emergencyOneLine(value, max)`,
|
|
1519
|
-
`emergencyIntervalText(ms)
|
|
1529
|
+
`emergencyIntervalText(ms)`, `emergencyFileMemory(app, service, dir?)` and `redactKnownSecrets(text)`.
|
|
1520
1530
|
|
|
1521
1531
|
### The rules it keeps
|
|
1522
1532
|
|
|
@@ -1532,8 +1542,11 @@ Also available:
|
|
|
1532
1542
|
loop is sent once, and a different crash is never silenced by it.
|
|
1533
1543
|
- **A channel set up wrongly is said once, at the start**, never silently skipped.
|
|
1534
1544
|
- **What leaves is redacted.** `fromConfig` removes every config value whose **name** looks secret (password, secret,
|
|
1535
|
-
token, key…) from every subject, line and log line, because an error message can carry SQL with its values.
|
|
1536
|
-
`redact`
|
|
1545
|
+
token, key…) from every subject, line and log line, because an error message can carry SQL with its values. And
|
|
1546
|
+
EVERY alerter — under any custom `redact` — removes what LOOKS like a secret wherever it came from
|
|
1547
|
+
(`redactKnownSecrets`): SendGrid keys, Stripe secret/restricted keys and webhook secrets, Telegram bot tokens, JWTs,
|
|
1548
|
+
Azure `AccountKey=`/`SharedAccess…=`, `Bearer`/`Basic` values, credentials inside a URL, and `password=`/`token=`/
|
|
1549
|
+
`api_key=`-style pairs — a per-tenant secret read from a database row is known to no config. Pass `redact` for more.
|
|
1537
1550
|
- **Subjects are one line**, because a crash's subject is the error's message, which often spans lines. The Telegram text
|
|
1538
1551
|
is cut **before** it's escaped, to whatever the heading and footer leave, so a long stack trace neither splits an
|
|
1539
1552
|
entity nor overruns Telegram's limit.
|
|
@@ -2445,6 +2458,7 @@ WEBHOOK_TIMESTAMP_HEADER; // 'X-Webhook-Timestamp'
|
|
|
2445
2458
|
WEBHOOK_SIGNATURE_HEADER; // 'X-Webhook-Signature'
|
|
2446
2459
|
UNKNOWN_APP_NAME; // 'unknown-app': what an alert says without APP_NAME
|
|
2447
2460
|
APP_NAME_CONFIG; // the CriticalConfigItem every checkConfig includes
|
|
2461
|
+
START_READY_TIMEOUT_MS; // 60000: startService alerts when the server is not listening after this long
|
|
2448
2462
|
```
|
|
2449
2463
|
|
|
2450
2464
|
---
|
|
@@ -2475,6 +2489,8 @@ APP_NAME_CONFIG; // the CriticalConfigItem every checkConfig includes
|
|
|
2475
2489
|
| `EmergencyAlerterProps` | `EmergencyAlerter` constructor props: `settings`, `email`, `telegram`, `log`, `redact`, `memory`, `now`, `hostname` |
|
|
2476
2490
|
| `EmergencyAlertMemory` | Where sent alerts are remembered: `{ read, write }` (default: a file under the OS temp folder, per app and service) |
|
|
2477
2491
|
| `CriticalConfigItem` | A setting a service cannot fully work without: `{ env, key?, effect }` |
|
|
2492
|
+
| `EmergencyStartOptions` | `startService()` options: `start`, `port`, `config?`, `critical?`, `afterListen?`, `readyTimeoutMs?`, `exit?` |
|
|
2493
|
+
| `EmergencyStartServer` | What `startService()` listens on: `{ listen, once, removeListener }` — an `http.Server` |
|
|
2478
2494
|
| `BaseRestApiCrudControllerPropsInterface` | Constructor props for `BaseRestApiCrudController` |
|
|
2479
2495
|
| `BuildDefaultResolvers` | Arguments for `buildDefaultResolvers()` |
|
|
2480
2496
|
| `BuildListQueryArgsInterface` | Arguments passed to `buildListQuery()` overrides: `{ filter, params, fieldsToUse, useCache }` |
|
package/build/BaseGateway.d.ts
CHANGED
|
@@ -153,6 +153,13 @@ declare class BaseGateway {
|
|
|
153
153
|
* (e.g. a table with a UUID primary key but a separate "slug" as update target).
|
|
154
154
|
*/
|
|
155
155
|
getIdFieldUpdateRemove(): string;
|
|
156
|
+
/**
|
|
157
|
+
* The column `list()` appends as the last sort term when it applies a limit — so tied rows keep one order across
|
|
158
|
+
* pages. `config.listTieBreaker` when set (a string names the column, `false` turns it off → null); otherwise the id
|
|
159
|
+
* field, qualified with the table's OWN name (the last segment of `config.table`, so `SCHEMA.Table` gives `Table.id`)
|
|
160
|
+
* — a list query that joins another table with an `id` stays unambiguous. Override to compute it differently.
|
|
161
|
+
*/
|
|
162
|
+
getListTieBreaker(): string | null;
|
|
156
163
|
/**
|
|
157
164
|
* Applies a whereIn or where-equals filter for scalar or array values.
|
|
158
165
|
*
|
package/build/BaseGateway.js
CHANGED
|
@@ -18,6 +18,24 @@ const DEFAULT_CHUNCK_SIZE = 2500;
|
|
|
18
18
|
* filters must be precise identifiers, not fuzzy searches or time windows.
|
|
19
19
|
*/
|
|
20
20
|
const UPDATE_REMOVE_SUPPORTED_TYPES = new Set(['array', 'eq', 'bool', 'nullableBool']);
|
|
21
|
+
/** `[table, column]` of a column reference, lower-cased; the table is '' when unqualified. */
|
|
22
|
+
const splitColumn = (reference) => {
|
|
23
|
+
const parts = String(reference || '').split('.');
|
|
24
|
+
const column = (parts.pop() || '').toLowerCase();
|
|
25
|
+
const table = (parts.pop() || '').toLowerCase();
|
|
26
|
+
return [table, column];
|
|
27
|
+
};
|
|
28
|
+
/**
|
|
29
|
+
* Whether a sort already orders by `column` — matched by the column's own name, and by its table when BOTH name one:
|
|
30
|
+
* a bare `id` and `party_appearance.id` are the same column here (a duplicate would be refused by SQL Server).
|
|
31
|
+
*/
|
|
32
|
+
const sortNamesColumn = (sortBy, column) => {
|
|
33
|
+
const [table, name] = splitColumn(column);
|
|
34
|
+
return (sortBy || []).some((item) => {
|
|
35
|
+
const [itemTable, itemName] = splitColumn(String(item?.column ?? ''));
|
|
36
|
+
return itemName === name && (!itemTable || !table || itemTable === table);
|
|
37
|
+
});
|
|
38
|
+
};
|
|
21
39
|
/**
|
|
22
40
|
* BaseGateway is the foundation for all database-backed gateway classes.
|
|
23
41
|
*
|
|
@@ -229,6 +247,29 @@ class BaseGateway {
|
|
|
229
247
|
getIdFieldUpdateRemove() {
|
|
230
248
|
return this.config.idFieldUpdateRemove || this.config.idField || constants_1.ID_FIELD;
|
|
231
249
|
}
|
|
250
|
+
/**
|
|
251
|
+
* The column `list()` appends as the last sort term when it applies a limit — so tied rows keep one order across
|
|
252
|
+
* pages. `config.listTieBreaker` when set (a string names the column, `false` turns it off → null); otherwise the id
|
|
253
|
+
* field, qualified with the table's OWN name (the last segment of `config.table`, so `SCHEMA.Table` gives `Table.id`)
|
|
254
|
+
* — a list query that joins another table with an `id` stays unambiguous. Override to compute it differently.
|
|
255
|
+
*/
|
|
256
|
+
getListTieBreaker() {
|
|
257
|
+
const configured = this.config.listTieBreaker;
|
|
258
|
+
if (configured === false) {
|
|
259
|
+
return null;
|
|
260
|
+
}
|
|
261
|
+
if (typeof configured === 'string' && configured) {
|
|
262
|
+
return configured;
|
|
263
|
+
}
|
|
264
|
+
const idField = this.getIdField();
|
|
265
|
+
if (idField.includes('.')) {
|
|
266
|
+
return idField;
|
|
267
|
+
}
|
|
268
|
+
const table = String(this.config.table || '')
|
|
269
|
+
.split('.')
|
|
270
|
+
.pop();
|
|
271
|
+
return table ? `${table}.${idField}` : idField;
|
|
272
|
+
}
|
|
232
273
|
// ─── Protected filter helpers ───────────────────────────────────────────────
|
|
233
274
|
//
|
|
234
275
|
// These helpers encode the common WHERE clause patterns used across gateway
|
|
@@ -1198,12 +1239,22 @@ class BaseGateway {
|
|
|
1198
1239
|
const fieldsToUse = !shouldUseCache && Array.isArray(this.config.selectFields) ? this.config.selectFields : this.getIdField();
|
|
1199
1240
|
const { query } = await this.buildListQuery({ filter, params, fieldsToUse, useCache: shouldUseCache });
|
|
1200
1241
|
await this.onListFilter(query, filter);
|
|
1201
|
-
|
|
1242
|
+
const isLimited = !!limit && !isNaN(limit) && limit > 0;
|
|
1243
|
+
if (isLimited) {
|
|
1202
1244
|
query.offset(offset).limit(limit);
|
|
1203
1245
|
}
|
|
1204
1246
|
if (sortBy) {
|
|
1205
1247
|
query.orderBy(sortBy);
|
|
1206
1248
|
}
|
|
1249
|
+
// A limited read ends its order with a unique column, so rows that tie on the sort (one bulk insert's created_at,
|
|
1250
|
+
// a shared order_no or name) come back in the same order on every page: without it Postgres may hand page 2 a row
|
|
1251
|
+
// page 1 already had and never hand over another. Not when the sort already names that column (SQL Server refuses
|
|
1252
|
+
// a column twice in ORDER BY), and not on an unlimited read (it returns every row whatever their order). An empty
|
|
1253
|
+
// sort gets it too — that is exactly the read with no order at all.
|
|
1254
|
+
const tieBreaker = isLimited ? this.getListTieBreaker() : null;
|
|
1255
|
+
if (tieBreaker && !sortNamesColumn(sortBy, tieBreaker)) {
|
|
1256
|
+
query.orderBy(tieBreaker, 'asc');
|
|
1257
|
+
}
|
|
1207
1258
|
const items = await query;
|
|
1208
1259
|
const result = shouldUseCache ? await this.getMany(items.map((row) => row.id)) : items;
|
|
1209
1260
|
return result.map((item) => this.transformOnOut(item));
|
|
@@ -1,6 +1,28 @@
|
|
|
1
1
|
import { OpResult } from '@sdflc/api-helpers';
|
|
2
2
|
import { SendGridGw, SendGridEmail } from './gateways/SendGridGw';
|
|
3
3
|
import { TelegramGw } from './gateways/TelegramGw';
|
|
4
|
+
/** What `startService` listens on — an `http.Server` (for Express + Apollo, the one `configureApp` creates). */
|
|
5
|
+
export interface EmergencyStartServer {
|
|
6
|
+
listen: (port: number, callback: () => void) => any;
|
|
7
|
+
once: (event: 'error', handler: (error: any) => void) => any;
|
|
8
|
+
removeListener: (event: 'error', handler: (error: any) => void) => any;
|
|
9
|
+
}
|
|
10
|
+
export interface EmergencyStartOptions {
|
|
11
|
+
/** Builds the app and answers the server to listen on. Everything slow (Redis, the database, Apollo) happens here. */
|
|
12
|
+
start: () => Promise<EmergencyStartServer> | EmergencyStartServer;
|
|
13
|
+
port: number | string;
|
|
14
|
+
/** The service config — checked against `critical` (and APP_NAME) AFTER the server listens. Omit to skip the check. */
|
|
15
|
+
config?: any;
|
|
16
|
+
critical?: CriticalConfigItem[];
|
|
17
|
+
/** Runs once the server listens: a startup log, background jobs, the service's own checks. A failure is alerted, never fatal. */
|
|
18
|
+
afterListen?: () => Promise<unknown> | unknown;
|
|
19
|
+
/** Still not listening this long after the start began → one alert; the start keeps waiting. Default 60 s. */
|
|
20
|
+
readyTimeoutMs?: number;
|
|
21
|
+
/** Called with 1 when the start fails, after its alert. Default `process.exit`. */
|
|
22
|
+
exit?: (code: number) => any;
|
|
23
|
+
}
|
|
24
|
+
/** The default wait before "still not listening" is alerted. */
|
|
25
|
+
export declare const START_READY_TIMEOUT_MS = 60000;
|
|
4
26
|
/** What an alert says when APP_NAME is not set. */
|
|
5
27
|
export declare const UNKNOWN_APP_NAME = "unknown-app";
|
|
6
28
|
export interface EmergencyAlert {
|
|
@@ -74,6 +96,8 @@ export declare const missingCriticalConfig: (config: any, items?: CriticalConfig
|
|
|
74
96
|
}>;
|
|
75
97
|
/** The alert for missing configuration, or null when nothing is missing. Pure. */
|
|
76
98
|
export declare const missingConfigAlert: (config: any, items?: CriticalConfigItem[]) => EmergencyAlert | null;
|
|
99
|
+
/** `text` with everything that looks like a secret replaced by `***`. Pure; never throws. */
|
|
100
|
+
export declare const redactKnownSecrets: (text: string) => string;
|
|
77
101
|
/** The memory in a file under the OS temp folder, one per app AND service (several may share a temp folder). */
|
|
78
102
|
export declare const emergencyFileMemory: (appName: string, serviceName: string, dir?: string) => EmergencyAlertMemory;
|
|
79
103
|
declare class EmergencyAlerter {
|
|
@@ -126,6 +150,17 @@ declare class EmergencyAlerter {
|
|
|
126
150
|
* when nothing is missing; otherwise what `send` answered. Never throws.
|
|
127
151
|
*/
|
|
128
152
|
checkConfig(config: any, items?: CriticalConfigItem[]): Promise<OpResult>;
|
|
153
|
+
/**
|
|
154
|
+
* The whole start of a service. Listens on the server `start` answers; then runs `afterListen` and checks the
|
|
155
|
+
* configuration (a missing value is alerted, never fatal — the service answers what it can). OK with
|
|
156
|
+
* `{ status: 'listening', port }`. When the start throws, or the port cannot be listened on, it alerts (keyed by WHICH
|
|
157
|
+
* failure) and exits 1. Never throws.
|
|
158
|
+
*
|
|
159
|
+
* ⚠ WHY THE TIMER: a dependency that never answers — Redis's client retries a refused connection for ever — hangs the
|
|
160
|
+
* start with the port closed and nothing but a log line every half second. Not listening after `readyTimeoutMs` is
|
|
161
|
+
* alerted once; the start keeps waiting, and when it listens at last that is said too.
|
|
162
|
+
*/
|
|
163
|
+
startService(opts: EmergencyStartOptions): Promise<OpResult>;
|
|
129
164
|
/**
|
|
130
165
|
* Alerts on an uncaught exception or an unhandled rejection, then exits 1 — the process is in an unknown state after an
|
|
131
166
|
* uncaught error, and a restart is the recovery (what Node would have done without the handler). The exit waits for the
|
|
@@ -3,7 +3,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
|
3
3
|
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
4
4
|
};
|
|
5
5
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
6
|
-
exports.EmergencyAlerter = exports.emergencyFileMemory = exports.missingConfigAlert = exports.missingCriticalConfig = exports.emergencyIntervalText = exports.emergencyOneLine = exports.APP_NAME_CONFIG = exports.UNKNOWN_APP_NAME = void 0;
|
|
6
|
+
exports.EmergencyAlerter = exports.emergencyFileMemory = exports.redactKnownSecrets = exports.missingConfigAlert = exports.missingCriticalConfig = exports.emergencyIntervalText = exports.emergencyOneLine = exports.APP_NAME_CONFIG = exports.UNKNOWN_APP_NAME = exports.START_READY_TIMEOUT_MS = void 0;
|
|
7
7
|
// ./src/EmergencyAlerter.ts
|
|
8
8
|
//
|
|
9
9
|
// An EMERGENCY alert a service sends BY ITSELF — for what nobody would otherwise notice until somebody trips over it: a
|
|
@@ -12,7 +12,11 @@ exports.EmergencyAlerter = exports.emergencyFileMemory = exports.missingConfigAl
|
|
|
12
12
|
//
|
|
13
13
|
// const alerter = EmergencyAlerter.fromConfig(config, 'ms_storage', { log: (m) => logger.error(m) });
|
|
14
14
|
// alerter.installCrashHandlers(process);
|
|
15
|
-
//
|
|
15
|
+
// void alerter.startService({ start: () => configureApp(express()), port: config.port, config, critical: CRITICAL_CONFIG });
|
|
16
|
+
//
|
|
17
|
+
// `startService` is the whole start: listen on the server `start` answers, check the configuration AFTER listening (a
|
|
18
|
+
// missing value never stops the start), alert when the server is still not listening after a while (a Redis or database
|
|
19
|
+
// that never answers hangs the start without a word otherwise), and alert + exit 1 when the start fails.
|
|
16
20
|
//
|
|
17
21
|
// Every alert names its app, service and environment — `[partylabz · ms_storage · production]` — because several apps
|
|
18
22
|
// run the same services side by side on one machine, and "ms_storage crashed" alone does not say whose.
|
|
@@ -37,11 +41,14 @@ const crypto_1 = require("crypto");
|
|
|
37
41
|
const fs_1 = __importDefault(require("fs"));
|
|
38
42
|
const os_1 = __importDefault(require("os"));
|
|
39
43
|
const path_1 = __importDefault(require("path"));
|
|
44
|
+
const timers_1 = require("timers");
|
|
40
45
|
const promises_1 = require("timers/promises");
|
|
41
46
|
const lodash_1 = require("lodash");
|
|
42
47
|
const api_helpers_1 = require("@sdflc/api-helpers");
|
|
43
48
|
const SendGridGw_1 = require("./gateways/SendGridGw");
|
|
44
49
|
const TelegramGw_1 = require("./gateways/TelegramGw");
|
|
50
|
+
/** The default wait before "still not listening" is alerted. */
|
|
51
|
+
exports.START_READY_TIMEOUT_MS = 60_000;
|
|
45
52
|
/** What an alert says when APP_NAME is not set. */
|
|
46
53
|
exports.UNKNOWN_APP_NAME = 'unknown-app';
|
|
47
54
|
/** Checked in every service: without it no alert can say which app it comes from. */
|
|
@@ -113,6 +120,37 @@ const escapedPrefix = (text, budget) => {
|
|
|
113
120
|
const keep = low > 0 && code >= 0xd800 && code <= 0xdbff ? low - 1 : low;
|
|
114
121
|
return escapeHtml(text.slice(0, keep));
|
|
115
122
|
};
|
|
123
|
+
/**
|
|
124
|
+
* Secrets that look like secrets whatever variable held them — provider keys, tokens, credentials inside URLs, connection
|
|
125
|
+
* strings and `name=value` pairs. Alerts carry error messages and stacks, which can quote a per-tenant secret read from a
|
|
126
|
+
* database row (that no config scrub knows about). Applied to EVERY alert, under any custom `redact`.
|
|
127
|
+
*/
|
|
128
|
+
const KNOWN_SECRET_PATTERNS = [
|
|
129
|
+
[/\bSG\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/g, 'SG.***'], // SendGrid
|
|
130
|
+
[/\b(sk|rk)_(live|test)_[A-Za-z0-9]{8,}/g, '$1_$2_***'], // Stripe secret / restricted
|
|
131
|
+
[/\bwhsec_[A-Za-z0-9]{8,}/g, 'whsec_***'], // Stripe webhook secret
|
|
132
|
+
[/\b\d{6,12}:[A-Za-z0-9_-]{30,}/g, '***:***'], // Telegram bot token
|
|
133
|
+
[/\beyJ[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}/g, '***.***.***'], // JWT
|
|
134
|
+
[/\b(AccountKey|SharedAccessKey|SharedAccessSignature)=[^;\s"']+/gi, '$1=***'], // Azure connection strings
|
|
135
|
+
[/\b(Bearer|Basic)\s+[A-Za-z0-9._~+/-]{8,}=*/gi, '$1 ***'], // Authorization values
|
|
136
|
+
[/(:\/\/[^/\s:@]+:)[^@\s/]+@/g, '$1***@'], // credentials inside a URL
|
|
137
|
+
[/\b(password|passwd|pwd|secret|token|api[_-]?key|apikey)(["']?\s*[:=]\s*["']?)[^\s"',;&]{6,}/gi, '$1$2***'], // name=value
|
|
138
|
+
];
|
|
139
|
+
/** `text` with everything that looks like a secret replaced by `***`. Pure; never throws. */
|
|
140
|
+
const redactKnownSecrets = (text) => {
|
|
141
|
+
try {
|
|
142
|
+
return KNOWN_SECRET_PATTERNS.reduce((out, [pattern, replacement]) => out.replace(pattern, replacement), String(text));
|
|
143
|
+
}
|
|
144
|
+
catch {
|
|
145
|
+
return '[redacted]';
|
|
146
|
+
}
|
|
147
|
+
};
|
|
148
|
+
exports.redactKnownSecrets = redactKnownSecrets;
|
|
149
|
+
/** Which failure this is — its message and top frame — so one failure does not silence a different one. Pure. */
|
|
150
|
+
const failureFingerprintOf = (message, stack) => (0, crypto_1.createHash)('sha256')
|
|
151
|
+
.update(`${message}\n${stack[1] ?? ''}`)
|
|
152
|
+
.digest('hex')
|
|
153
|
+
.slice(0, 12);
|
|
116
154
|
/** The memory in a file under the OS temp folder, one per app AND service (several may share a temp folder). */
|
|
117
155
|
const emergencyFileMemory = (appName, serviceName, dir = os_1.default.tmpdir()) => {
|
|
118
156
|
const safe = (name) => name.replace(/[^A-Za-z0-9_.-]/g, '_');
|
|
@@ -164,9 +202,10 @@ class EmergencyAlerter {
|
|
|
164
202
|
}
|
|
165
203
|
};
|
|
166
204
|
const redact = props.redact ?? ((text) => text);
|
|
205
|
+
// ⚠ Known secret shapes are removed ALWAYS, after any custom redact: a custom hook narrows nothing.
|
|
167
206
|
this.redact = (text) => {
|
|
168
207
|
try {
|
|
169
|
-
return String(redact(String(text ?? '')));
|
|
208
|
+
return (0, exports.redactKnownSecrets)(String(redact(String(text ?? ''))));
|
|
170
209
|
}
|
|
171
210
|
catch {
|
|
172
211
|
return '[redacted: the redact hook failed]';
|
|
@@ -413,6 +452,97 @@ class EmergencyAlerter {
|
|
|
413
452
|
const alert = (0, exports.missingConfigAlert)(config, items);
|
|
414
453
|
return alert ? this.send(alert) : api_helpers_1.OpResult.ok([]);
|
|
415
454
|
}
|
|
455
|
+
/**
|
|
456
|
+
* The whole start of a service. Listens on the server `start` answers; then runs `afterListen` and checks the
|
|
457
|
+
* configuration (a missing value is alerted, never fatal — the service answers what it can). OK with
|
|
458
|
+
* `{ status: 'listening', port }`. When the start throws, or the port cannot be listened on, it alerts (keyed by WHICH
|
|
459
|
+
* failure) and exits 1. Never throws.
|
|
460
|
+
*
|
|
461
|
+
* ⚠ WHY THE TIMER: a dependency that never answers — Redis's client retries a refused connection for ever — hangs the
|
|
462
|
+
* start with the port closed and nothing but a log line every half second. Not listening after `readyTimeoutMs` is
|
|
463
|
+
* alerted once; the start keeps waiting, and when it listens at last that is said too.
|
|
464
|
+
*/
|
|
465
|
+
async startService(opts) {
|
|
466
|
+
const exit = opts?.exit ?? ((code) => process.exit(code));
|
|
467
|
+
const readyMs = Number.isFinite(Number(opts?.readyTimeoutMs))
|
|
468
|
+
? Number(opts.readyTimeoutMs)
|
|
469
|
+
: exports.START_READY_TIMEOUT_MS;
|
|
470
|
+
const port = Number(opts?.port);
|
|
471
|
+
let listening = false;
|
|
472
|
+
let lateAlerted = false;
|
|
473
|
+
const startedAt = Date.now();
|
|
474
|
+
// ⚠ Not unref'd: a start that hangs on nothing at all must still live long enough to say so.
|
|
475
|
+
const timer = (0, timers_1.setTimeout)(() => {
|
|
476
|
+
if (listening) {
|
|
477
|
+
return;
|
|
478
|
+
}
|
|
479
|
+
lateAlerted = true;
|
|
480
|
+
void this.send({
|
|
481
|
+
key: 'startup:not-listening',
|
|
482
|
+
subject: `Not listening ${(0, exports.emergencyIntervalText)(readyMs)} after the start`,
|
|
483
|
+
lines: [
|
|
484
|
+
`The start is still waiting, and port ${opts?.port} is closed: nothing is answered.`,
|
|
485
|
+
'It waits on something that does not answer — Redis, the database, or another service. It keeps waiting.',
|
|
486
|
+
],
|
|
487
|
+
});
|
|
488
|
+
}, readyMs);
|
|
489
|
+
try {
|
|
490
|
+
if (!Number.isInteger(port) || port < 0 || port > 65535) {
|
|
491
|
+
throw new Error(`The port '${opts?.port}' is not a port`);
|
|
492
|
+
}
|
|
493
|
+
const server = await opts.start();
|
|
494
|
+
await new Promise((resolve, reject) => {
|
|
495
|
+
const onError = (error) => reject(error);
|
|
496
|
+
server.once('error', onError);
|
|
497
|
+
server.listen(port, () => {
|
|
498
|
+
server.removeListener('error', onError);
|
|
499
|
+
resolve();
|
|
500
|
+
});
|
|
501
|
+
});
|
|
502
|
+
listening = true;
|
|
503
|
+
(0, timers_1.clearTimeout)(timer);
|
|
504
|
+
if (lateAlerted) {
|
|
505
|
+
void this.send({
|
|
506
|
+
key: 'startup:listening-late',
|
|
507
|
+
subject: `Listening at last, ${(0, exports.emergencyIntervalText)(Date.now() - startedAt)} after the start`,
|
|
508
|
+
lines: ['What it waited on answered. Nothing to do unless it happens again.'],
|
|
509
|
+
});
|
|
510
|
+
}
|
|
511
|
+
}
|
|
512
|
+
catch (e) {
|
|
513
|
+
(0, timers_1.clearTimeout)(timer);
|
|
514
|
+
const { message, stack } = describeError(e);
|
|
515
|
+
await this.send({
|
|
516
|
+
key: `crash:startup:${failureFingerprintOf(message, stack)}`,
|
|
517
|
+
subject: `Could not start: ${(0, exports.emergencyOneLine)(message, 120)}`,
|
|
518
|
+
lines: [message, '', ...stack],
|
|
519
|
+
});
|
|
520
|
+
try {
|
|
521
|
+
exit(1);
|
|
522
|
+
}
|
|
523
|
+
catch {
|
|
524
|
+
// An exit that throws leaves nothing more to do.
|
|
525
|
+
}
|
|
526
|
+
return api_helpers_1.OpResult.fromException(e);
|
|
527
|
+
}
|
|
528
|
+
if (opts.afterListen) {
|
|
529
|
+
try {
|
|
530
|
+
await opts.afterListen();
|
|
531
|
+
}
|
|
532
|
+
catch (e) {
|
|
533
|
+
const { message, stack } = describeError(e);
|
|
534
|
+
await this.send({
|
|
535
|
+
key: `startup:after-listen:${failureFingerprintOf(message, stack)}`,
|
|
536
|
+
subject: `Started, but a startup step failed: ${(0, exports.emergencyOneLine)(message, 120)}`,
|
|
537
|
+
lines: [message, '', ...stack],
|
|
538
|
+
});
|
|
539
|
+
}
|
|
540
|
+
}
|
|
541
|
+
if (opts.config !== undefined) {
|
|
542
|
+
await this.checkConfig(opts.config, opts.critical ?? []);
|
|
543
|
+
}
|
|
544
|
+
return api_helpers_1.OpResult.ok([{ status: 'listening', port }]);
|
|
545
|
+
}
|
|
416
546
|
/**
|
|
417
547
|
* Alerts on an uncaught exception or an unhandled rejection, then exits 1 — the process is in an unknown state after an
|
|
418
548
|
* uncaught error, and a restart is the recovery (what Node would have done without the handler). The exit waits for the
|
|
@@ -429,10 +559,7 @@ class EmergencyAlerter {
|
|
|
429
559
|
try {
|
|
430
560
|
const { message, stack } = describeError(error);
|
|
431
561
|
// ⚠ The key names WHICH crash (its message and top frame): one crash must not silence a different one for an hour.
|
|
432
|
-
const fingerprint = (
|
|
433
|
-
.update(`${message}\n${stack[1] ?? ''}`)
|
|
434
|
-
.digest('hex')
|
|
435
|
-
.slice(0, 12);
|
|
562
|
+
const fingerprint = failureFingerprintOf(message, stack);
|
|
436
563
|
const alert = {
|
|
437
564
|
key: `crash:${kind}:${fingerprint}`,
|
|
438
565
|
subject: `Crashed (${kind}): ${(0, exports.emergencyOneLine)(message, 120)}`,
|
|
@@ -153,6 +153,13 @@ declare class BaseGateway {
|
|
|
153
153
|
* (e.g. a table with a UUID primary key but a separate "slug" as update target).
|
|
154
154
|
*/
|
|
155
155
|
getIdFieldUpdateRemove(): string;
|
|
156
|
+
/**
|
|
157
|
+
* The column `list()` appends as the last sort term when it applies a limit — so tied rows keep one order across
|
|
158
|
+
* pages. `config.listTieBreaker` when set (a string names the column, `false` turns it off → null); otherwise the id
|
|
159
|
+
* field, qualified with the table's OWN name (the last segment of `config.table`, so `SCHEMA.Table` gives `Table.id`)
|
|
160
|
+
* — a list query that joins another table with an `id` stays unambiguous. Override to compute it differently.
|
|
161
|
+
*/
|
|
162
|
+
getListTieBreaker(): string | null;
|
|
156
163
|
/**
|
|
157
164
|
* Applies a whereIn or where-equals filter for scalar or array values.
|
|
158
165
|
*
|
package/build/esm/BaseGateway.js
CHANGED
|
@@ -12,6 +12,24 @@ const DEFAULT_CHUNCK_SIZE = 2500;
|
|
|
12
12
|
* filters must be precise identifiers, not fuzzy searches or time windows.
|
|
13
13
|
*/
|
|
14
14
|
const UPDATE_REMOVE_SUPPORTED_TYPES = new Set(['array', 'eq', 'bool', 'nullableBool']);
|
|
15
|
+
/** `[table, column]` of a column reference, lower-cased; the table is '' when unqualified. */
|
|
16
|
+
const splitColumn = (reference) => {
|
|
17
|
+
const parts = String(reference || '').split('.');
|
|
18
|
+
const column = (parts.pop() || '').toLowerCase();
|
|
19
|
+
const table = (parts.pop() || '').toLowerCase();
|
|
20
|
+
return [table, column];
|
|
21
|
+
};
|
|
22
|
+
/**
|
|
23
|
+
* Whether a sort already orders by `column` — matched by the column's own name, and by its table when BOTH name one:
|
|
24
|
+
* a bare `id` and `party_appearance.id` are the same column here (a duplicate would be refused by SQL Server).
|
|
25
|
+
*/
|
|
26
|
+
const sortNamesColumn = (sortBy, column) => {
|
|
27
|
+
const [table, name] = splitColumn(column);
|
|
28
|
+
return (sortBy || []).some((item) => {
|
|
29
|
+
const [itemTable, itemName] = splitColumn(String(item?.column ?? ''));
|
|
30
|
+
return itemName === name && (!itemTable || !table || itemTable === table);
|
|
31
|
+
});
|
|
32
|
+
};
|
|
15
33
|
/**
|
|
16
34
|
* BaseGateway is the foundation for all database-backed gateway classes.
|
|
17
35
|
*
|
|
@@ -223,6 +241,29 @@ class BaseGateway {
|
|
|
223
241
|
getIdFieldUpdateRemove() {
|
|
224
242
|
return this.config.idFieldUpdateRemove || this.config.idField || ID_FIELD;
|
|
225
243
|
}
|
|
244
|
+
/**
|
|
245
|
+
* The column `list()` appends as the last sort term when it applies a limit — so tied rows keep one order across
|
|
246
|
+
* pages. `config.listTieBreaker` when set (a string names the column, `false` turns it off → null); otherwise the id
|
|
247
|
+
* field, qualified with the table's OWN name (the last segment of `config.table`, so `SCHEMA.Table` gives `Table.id`)
|
|
248
|
+
* — a list query that joins another table with an `id` stays unambiguous. Override to compute it differently.
|
|
249
|
+
*/
|
|
250
|
+
getListTieBreaker() {
|
|
251
|
+
const configured = this.config.listTieBreaker;
|
|
252
|
+
if (configured === false) {
|
|
253
|
+
return null;
|
|
254
|
+
}
|
|
255
|
+
if (typeof configured === 'string' && configured) {
|
|
256
|
+
return configured;
|
|
257
|
+
}
|
|
258
|
+
const idField = this.getIdField();
|
|
259
|
+
if (idField.includes('.')) {
|
|
260
|
+
return idField;
|
|
261
|
+
}
|
|
262
|
+
const table = String(this.config.table || '')
|
|
263
|
+
.split('.')
|
|
264
|
+
.pop();
|
|
265
|
+
return table ? `${table}.${idField}` : idField;
|
|
266
|
+
}
|
|
226
267
|
// ─── Protected filter helpers ───────────────────────────────────────────────
|
|
227
268
|
//
|
|
228
269
|
// These helpers encode the common WHERE clause patterns used across gateway
|
|
@@ -1192,12 +1233,22 @@ class BaseGateway {
|
|
|
1192
1233
|
const fieldsToUse = !shouldUseCache && Array.isArray(this.config.selectFields) ? this.config.selectFields : this.getIdField();
|
|
1193
1234
|
const { query } = await this.buildListQuery({ filter, params, fieldsToUse, useCache: shouldUseCache });
|
|
1194
1235
|
await this.onListFilter(query, filter);
|
|
1195
|
-
|
|
1236
|
+
const isLimited = !!limit && !isNaN(limit) && limit > 0;
|
|
1237
|
+
if (isLimited) {
|
|
1196
1238
|
query.offset(offset).limit(limit);
|
|
1197
1239
|
}
|
|
1198
1240
|
if (sortBy) {
|
|
1199
1241
|
query.orderBy(sortBy);
|
|
1200
1242
|
}
|
|
1243
|
+
// A limited read ends its order with a unique column, so rows that tie on the sort (one bulk insert's created_at,
|
|
1244
|
+
// a shared order_no or name) come back in the same order on every page: without it Postgres may hand page 2 a row
|
|
1245
|
+
// page 1 already had and never hand over another. Not when the sort already names that column (SQL Server refuses
|
|
1246
|
+
// a column twice in ORDER BY), and not on an unlimited read (it returns every row whatever their order). An empty
|
|
1247
|
+
// sort gets it too — that is exactly the read with no order at all.
|
|
1248
|
+
const tieBreaker = isLimited ? this.getListTieBreaker() : null;
|
|
1249
|
+
if (tieBreaker && !sortNamesColumn(sortBy, tieBreaker)) {
|
|
1250
|
+
query.orderBy(tieBreaker, 'asc');
|
|
1251
|
+
}
|
|
1201
1252
|
const items = await query;
|
|
1202
1253
|
const result = shouldUseCache ? await this.getMany(items.map((row) => row.id)) : items;
|
|
1203
1254
|
return result.map((item) => this.transformOnOut(item));
|
|
@@ -1,6 +1,28 @@
|
|
|
1
1
|
import { OpResult } from '@sdflc/api-helpers';
|
|
2
2
|
import { SendGridGw, SendGridEmail } from './gateways/SendGridGw';
|
|
3
3
|
import { TelegramGw } from './gateways/TelegramGw';
|
|
4
|
+
/** What `startService` listens on — an `http.Server` (for Express + Apollo, the one `configureApp` creates). */
|
|
5
|
+
export interface EmergencyStartServer {
|
|
6
|
+
listen: (port: number, callback: () => void) => any;
|
|
7
|
+
once: (event: 'error', handler: (error: any) => void) => any;
|
|
8
|
+
removeListener: (event: 'error', handler: (error: any) => void) => any;
|
|
9
|
+
}
|
|
10
|
+
export interface EmergencyStartOptions {
|
|
11
|
+
/** Builds the app and answers the server to listen on. Everything slow (Redis, the database, Apollo) happens here. */
|
|
12
|
+
start: () => Promise<EmergencyStartServer> | EmergencyStartServer;
|
|
13
|
+
port: number | string;
|
|
14
|
+
/** The service config — checked against `critical` (and APP_NAME) AFTER the server listens. Omit to skip the check. */
|
|
15
|
+
config?: any;
|
|
16
|
+
critical?: CriticalConfigItem[];
|
|
17
|
+
/** Runs once the server listens: a startup log, background jobs, the service's own checks. A failure is alerted, never fatal. */
|
|
18
|
+
afterListen?: () => Promise<unknown> | unknown;
|
|
19
|
+
/** Still not listening this long after the start began → one alert; the start keeps waiting. Default 60 s. */
|
|
20
|
+
readyTimeoutMs?: number;
|
|
21
|
+
/** Called with 1 when the start fails, after its alert. Default `process.exit`. */
|
|
22
|
+
exit?: (code: number) => any;
|
|
23
|
+
}
|
|
24
|
+
/** The default wait before "still not listening" is alerted. */
|
|
25
|
+
export declare const START_READY_TIMEOUT_MS = 60000;
|
|
4
26
|
/** What an alert says when APP_NAME is not set. */
|
|
5
27
|
export declare const UNKNOWN_APP_NAME = "unknown-app";
|
|
6
28
|
export interface EmergencyAlert {
|
|
@@ -74,6 +96,8 @@ export declare const missingCriticalConfig: (config: any, items?: CriticalConfig
|
|
|
74
96
|
}>;
|
|
75
97
|
/** The alert for missing configuration, or null when nothing is missing. Pure. */
|
|
76
98
|
export declare const missingConfigAlert: (config: any, items?: CriticalConfigItem[]) => EmergencyAlert | null;
|
|
99
|
+
/** `text` with everything that looks like a secret replaced by `***`. Pure; never throws. */
|
|
100
|
+
export declare const redactKnownSecrets: (text: string) => string;
|
|
77
101
|
/** The memory in a file under the OS temp folder, one per app AND service (several may share a temp folder). */
|
|
78
102
|
export declare const emergencyFileMemory: (appName: string, serviceName: string, dir?: string) => EmergencyAlertMemory;
|
|
79
103
|
declare class EmergencyAlerter {
|
|
@@ -126,6 +150,17 @@ declare class EmergencyAlerter {
|
|
|
126
150
|
* when nothing is missing; otherwise what `send` answered. Never throws.
|
|
127
151
|
*/
|
|
128
152
|
checkConfig(config: any, items?: CriticalConfigItem[]): Promise<OpResult>;
|
|
153
|
+
/**
|
|
154
|
+
* The whole start of a service. Listens on the server `start` answers; then runs `afterListen` and checks the
|
|
155
|
+
* configuration (a missing value is alerted, never fatal — the service answers what it can). OK with
|
|
156
|
+
* `{ status: 'listening', port }`. When the start throws, or the port cannot be listened on, it alerts (keyed by WHICH
|
|
157
|
+
* failure) and exits 1. Never throws.
|
|
158
|
+
*
|
|
159
|
+
* ⚠ WHY THE TIMER: a dependency that never answers — Redis's client retries a refused connection for ever — hangs the
|
|
160
|
+
* start with the port closed and nothing but a log line every half second. Not listening after `readyTimeoutMs` is
|
|
161
|
+
* alerted once; the start keeps waiting, and when it listens at last that is said too.
|
|
162
|
+
*/
|
|
163
|
+
startService(opts: EmergencyStartOptions): Promise<OpResult>;
|
|
129
164
|
/**
|
|
130
165
|
* Alerts on an uncaught exception or an unhandled rejection, then exits 1 — the process is in an unknown state after an
|
|
131
166
|
* uncaught error, and a restart is the recovery (what Node would have done without the handler). The exit waits for the
|
|
@@ -6,7 +6,11 @@
|
|
|
6
6
|
//
|
|
7
7
|
// const alerter = EmergencyAlerter.fromConfig(config, 'ms_storage', { log: (m) => logger.error(m) });
|
|
8
8
|
// alerter.installCrashHandlers(process);
|
|
9
|
-
//
|
|
9
|
+
// void alerter.startService({ start: () => configureApp(express()), port: config.port, config, critical: CRITICAL_CONFIG });
|
|
10
|
+
//
|
|
11
|
+
// `startService` is the whole start: listen on the server `start` answers, check the configuration AFTER listening (a
|
|
12
|
+
// missing value never stops the start), alert when the server is still not listening after a while (a Redis or database
|
|
13
|
+
// that never answers hangs the start without a word otherwise), and alert + exit 1 when the start fails.
|
|
10
14
|
//
|
|
11
15
|
// Every alert names its app, service and environment — `[partylabz · ms_storage · production]` — because several apps
|
|
12
16
|
// run the same services side by side on one machine, and "ms_storage crashed" alone does not say whose.
|
|
@@ -31,11 +35,14 @@ import { createHash } from 'crypto';
|
|
|
31
35
|
import fs from 'fs';
|
|
32
36
|
import os from 'os';
|
|
33
37
|
import path from 'path';
|
|
38
|
+
import { clearTimeout, setTimeout as setTimer } from 'timers';
|
|
34
39
|
import { setTimeout as sleep } from 'timers/promises';
|
|
35
40
|
import { camelCase } from 'lodash';
|
|
36
41
|
import { OP_RESULT_CODES, OpResult } from '@sdflc/api-helpers';
|
|
37
42
|
import { SendGridGw } from './gateways/SendGridGw';
|
|
38
43
|
import { TelegramGw, TELEGRAM_MAX_MESSAGE_LENGTH, isTelegramChatId } from './gateways/TelegramGw';
|
|
44
|
+
/** The default wait before "still not listening" is alerted. */
|
|
45
|
+
export const START_READY_TIMEOUT_MS = 60_000;
|
|
39
46
|
/** What an alert says when APP_NAME is not set. */
|
|
40
47
|
export const UNKNOWN_APP_NAME = 'unknown-app';
|
|
41
48
|
/** Checked in every service: without it no alert can say which app it comes from. */
|
|
@@ -103,6 +110,36 @@ const escapedPrefix = (text, budget) => {
|
|
|
103
110
|
const keep = low > 0 && code >= 0xd800 && code <= 0xdbff ? low - 1 : low;
|
|
104
111
|
return escapeHtml(text.slice(0, keep));
|
|
105
112
|
};
|
|
113
|
+
/**
|
|
114
|
+
* Secrets that look like secrets whatever variable held them — provider keys, tokens, credentials inside URLs, connection
|
|
115
|
+
* strings and `name=value` pairs. Alerts carry error messages and stacks, which can quote a per-tenant secret read from a
|
|
116
|
+
* database row (that no config scrub knows about). Applied to EVERY alert, under any custom `redact`.
|
|
117
|
+
*/
|
|
118
|
+
const KNOWN_SECRET_PATTERNS = [
|
|
119
|
+
[/\bSG\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/g, 'SG.***'], // SendGrid
|
|
120
|
+
[/\b(sk|rk)_(live|test)_[A-Za-z0-9]{8,}/g, '$1_$2_***'], // Stripe secret / restricted
|
|
121
|
+
[/\bwhsec_[A-Za-z0-9]{8,}/g, 'whsec_***'], // Stripe webhook secret
|
|
122
|
+
[/\b\d{6,12}:[A-Za-z0-9_-]{30,}/g, '***:***'], // Telegram bot token
|
|
123
|
+
[/\beyJ[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}/g, '***.***.***'], // JWT
|
|
124
|
+
[/\b(AccountKey|SharedAccessKey|SharedAccessSignature)=[^;\s"']+/gi, '$1=***'], // Azure connection strings
|
|
125
|
+
[/\b(Bearer|Basic)\s+[A-Za-z0-9._~+/-]{8,}=*/gi, '$1 ***'], // Authorization values
|
|
126
|
+
[/(:\/\/[^/\s:@]+:)[^@\s/]+@/g, '$1***@'], // credentials inside a URL
|
|
127
|
+
[/\b(password|passwd|pwd|secret|token|api[_-]?key|apikey)(["']?\s*[:=]\s*["']?)[^\s"',;&]{6,}/gi, '$1$2***'], // name=value
|
|
128
|
+
];
|
|
129
|
+
/** `text` with everything that looks like a secret replaced by `***`. Pure; never throws. */
|
|
130
|
+
export const redactKnownSecrets = (text) => {
|
|
131
|
+
try {
|
|
132
|
+
return KNOWN_SECRET_PATTERNS.reduce((out, [pattern, replacement]) => out.replace(pattern, replacement), String(text));
|
|
133
|
+
}
|
|
134
|
+
catch {
|
|
135
|
+
return '[redacted]';
|
|
136
|
+
}
|
|
137
|
+
};
|
|
138
|
+
/** Which failure this is — its message and top frame — so one failure does not silence a different one. Pure. */
|
|
139
|
+
const failureFingerprintOf = (message, stack) => createHash('sha256')
|
|
140
|
+
.update(`${message}\n${stack[1] ?? ''}`)
|
|
141
|
+
.digest('hex')
|
|
142
|
+
.slice(0, 12);
|
|
106
143
|
/** The memory in a file under the OS temp folder, one per app AND service (several may share a temp folder). */
|
|
107
144
|
export const emergencyFileMemory = (appName, serviceName, dir = os.tmpdir()) => {
|
|
108
145
|
const safe = (name) => name.replace(/[^A-Za-z0-9_.-]/g, '_');
|
|
@@ -153,9 +190,10 @@ class EmergencyAlerter {
|
|
|
153
190
|
}
|
|
154
191
|
};
|
|
155
192
|
const redact = props.redact ?? ((text) => text);
|
|
193
|
+
// ⚠ Known secret shapes are removed ALWAYS, after any custom redact: a custom hook narrows nothing.
|
|
156
194
|
this.redact = (text) => {
|
|
157
195
|
try {
|
|
158
|
-
return String(redact(String(text ?? '')));
|
|
196
|
+
return redactKnownSecrets(String(redact(String(text ?? ''))));
|
|
159
197
|
}
|
|
160
198
|
catch {
|
|
161
199
|
return '[redacted: the redact hook failed]';
|
|
@@ -402,6 +440,97 @@ class EmergencyAlerter {
|
|
|
402
440
|
const alert = missingConfigAlert(config, items);
|
|
403
441
|
return alert ? this.send(alert) : OpResult.ok([]);
|
|
404
442
|
}
|
|
443
|
+
/**
|
|
444
|
+
* The whole start of a service. Listens on the server `start` answers; then runs `afterListen` and checks the
|
|
445
|
+
* configuration (a missing value is alerted, never fatal — the service answers what it can). OK with
|
|
446
|
+
* `{ status: 'listening', port }`. When the start throws, or the port cannot be listened on, it alerts (keyed by WHICH
|
|
447
|
+
* failure) and exits 1. Never throws.
|
|
448
|
+
*
|
|
449
|
+
* ⚠ WHY THE TIMER: a dependency that never answers — Redis's client retries a refused connection for ever — hangs the
|
|
450
|
+
* start with the port closed and nothing but a log line every half second. Not listening after `readyTimeoutMs` is
|
|
451
|
+
* alerted once; the start keeps waiting, and when it listens at last that is said too.
|
|
452
|
+
*/
|
|
453
|
+
async startService(opts) {
|
|
454
|
+
const exit = opts?.exit ?? ((code) => process.exit(code));
|
|
455
|
+
const readyMs = Number.isFinite(Number(opts?.readyTimeoutMs))
|
|
456
|
+
? Number(opts.readyTimeoutMs)
|
|
457
|
+
: START_READY_TIMEOUT_MS;
|
|
458
|
+
const port = Number(opts?.port);
|
|
459
|
+
let listening = false;
|
|
460
|
+
let lateAlerted = false;
|
|
461
|
+
const startedAt = Date.now();
|
|
462
|
+
// ⚠ Not unref'd: a start that hangs on nothing at all must still live long enough to say so.
|
|
463
|
+
const timer = setTimer(() => {
|
|
464
|
+
if (listening) {
|
|
465
|
+
return;
|
|
466
|
+
}
|
|
467
|
+
lateAlerted = true;
|
|
468
|
+
void this.send({
|
|
469
|
+
key: 'startup:not-listening',
|
|
470
|
+
subject: `Not listening ${emergencyIntervalText(readyMs)} after the start`,
|
|
471
|
+
lines: [
|
|
472
|
+
`The start is still waiting, and port ${opts?.port} is closed: nothing is answered.`,
|
|
473
|
+
'It waits on something that does not answer — Redis, the database, or another service. It keeps waiting.',
|
|
474
|
+
],
|
|
475
|
+
});
|
|
476
|
+
}, readyMs);
|
|
477
|
+
try {
|
|
478
|
+
if (!Number.isInteger(port) || port < 0 || port > 65535) {
|
|
479
|
+
throw new Error(`The port '${opts?.port}' is not a port`);
|
|
480
|
+
}
|
|
481
|
+
const server = await opts.start();
|
|
482
|
+
await new Promise((resolve, reject) => {
|
|
483
|
+
const onError = (error) => reject(error);
|
|
484
|
+
server.once('error', onError);
|
|
485
|
+
server.listen(port, () => {
|
|
486
|
+
server.removeListener('error', onError);
|
|
487
|
+
resolve();
|
|
488
|
+
});
|
|
489
|
+
});
|
|
490
|
+
listening = true;
|
|
491
|
+
clearTimeout(timer);
|
|
492
|
+
if (lateAlerted) {
|
|
493
|
+
void this.send({
|
|
494
|
+
key: 'startup:listening-late',
|
|
495
|
+
subject: `Listening at last, ${emergencyIntervalText(Date.now() - startedAt)} after the start`,
|
|
496
|
+
lines: ['What it waited on answered. Nothing to do unless it happens again.'],
|
|
497
|
+
});
|
|
498
|
+
}
|
|
499
|
+
}
|
|
500
|
+
catch (e) {
|
|
501
|
+
clearTimeout(timer);
|
|
502
|
+
const { message, stack } = describeError(e);
|
|
503
|
+
await this.send({
|
|
504
|
+
key: `crash:startup:${failureFingerprintOf(message, stack)}`,
|
|
505
|
+
subject: `Could not start: ${emergencyOneLine(message, 120)}`,
|
|
506
|
+
lines: [message, '', ...stack],
|
|
507
|
+
});
|
|
508
|
+
try {
|
|
509
|
+
exit(1);
|
|
510
|
+
}
|
|
511
|
+
catch {
|
|
512
|
+
// An exit that throws leaves nothing more to do.
|
|
513
|
+
}
|
|
514
|
+
return OpResult.fromException(e);
|
|
515
|
+
}
|
|
516
|
+
if (opts.afterListen) {
|
|
517
|
+
try {
|
|
518
|
+
await opts.afterListen();
|
|
519
|
+
}
|
|
520
|
+
catch (e) {
|
|
521
|
+
const { message, stack } = describeError(e);
|
|
522
|
+
await this.send({
|
|
523
|
+
key: `startup:after-listen:${failureFingerprintOf(message, stack)}`,
|
|
524
|
+
subject: `Started, but a startup step failed: ${emergencyOneLine(message, 120)}`,
|
|
525
|
+
lines: [message, '', ...stack],
|
|
526
|
+
});
|
|
527
|
+
}
|
|
528
|
+
}
|
|
529
|
+
if (opts.config !== undefined) {
|
|
530
|
+
await this.checkConfig(opts.config, opts.critical ?? []);
|
|
531
|
+
}
|
|
532
|
+
return OpResult.ok([{ status: 'listening', port }]);
|
|
533
|
+
}
|
|
405
534
|
/**
|
|
406
535
|
* Alerts on an uncaught exception or an unhandled rejection, then exits 1 — the process is in an unknown state after an
|
|
407
536
|
* uncaught error, and a restart is the recovery (what Node would have done without the handler). The exit waits for the
|
|
@@ -418,10 +547,7 @@ class EmergencyAlerter {
|
|
|
418
547
|
try {
|
|
419
548
|
const { message, stack } = describeError(error);
|
|
420
549
|
// ⚠ The key names WHICH crash (its message and top frame): one crash must not silence a different one for an hour.
|
|
421
|
-
const fingerprint =
|
|
422
|
-
.update(`${message}\n${stack[1] ?? ''}`)
|
|
423
|
-
.digest('hex')
|
|
424
|
-
.slice(0, 12);
|
|
550
|
+
const fingerprint = failureFingerprintOf(message, stack);
|
|
425
551
|
const alert = {
|
|
426
552
|
key: `crash:${kind}:${fingerprint}`,
|
|
427
553
|
subject: `Crashed (${kind}): ${emergencyOneLine(message, 120)}`,
|
|
@@ -74,6 +74,17 @@ export interface BaseGatewayPropsInterface extends CacheInterface {
|
|
|
74
74
|
*/
|
|
75
75
|
localizationForField?: string;
|
|
76
76
|
defaultSorting?: SortItemInterface[];
|
|
77
|
+
/**
|
|
78
|
+
* The column `list()` adds as the LAST sort term whenever it applies a limit, so rows that tie on the caller's sort
|
|
79
|
+
* come back in the same order on every page. Defaults to the id field, qualified with the table's own name
|
|
80
|
+
* (`getListTieBreaker()`). `false` turns it off — for a list query it would break (one that groups, selects
|
|
81
|
+
* DISTINCT, or reads the table under an alias). A string names a different unique column.
|
|
82
|
+
*
|
|
83
|
+
* ⚠ Without it, Postgres does not promise the same order of tied rows from one LIMIT/OFFSET to the next: page 2 can
|
|
84
|
+
* repeat a row from page 1 and skip another for good — rows created together (one bulk insert = one created_at) or a
|
|
85
|
+
* shared `order_no` / name tie all the time. (SQL Server also refuses OFFSET/FETCH — page 2 — with no ORDER BY.)
|
|
86
|
+
*/
|
|
87
|
+
listTieBreaker?: string | false;
|
|
77
88
|
/**
|
|
78
89
|
* Field to apply keyword (ILIKE) search against when filterParams.keyword is set.
|
|
79
90
|
*/
|
|
@@ -74,6 +74,17 @@ export interface BaseGatewayPropsInterface extends CacheInterface {
|
|
|
74
74
|
*/
|
|
75
75
|
localizationForField?: string;
|
|
76
76
|
defaultSorting?: SortItemInterface[];
|
|
77
|
+
/**
|
|
78
|
+
* The column `list()` adds as the LAST sort term whenever it applies a limit, so rows that tie on the caller's sort
|
|
79
|
+
* come back in the same order on every page. Defaults to the id field, qualified with the table's own name
|
|
80
|
+
* (`getListTieBreaker()`). `false` turns it off — for a list query it would break (one that groups, selects
|
|
81
|
+
* DISTINCT, or reads the table under an alias). A string names a different unique column.
|
|
82
|
+
*
|
|
83
|
+
* ⚠ Without it, Postgres does not promise the same order of tied rows from one LIMIT/OFFSET to the next: page 2 can
|
|
84
|
+
* repeat a row from page 1 and skip another for good — rows created together (one bulk insert = one created_at) or a
|
|
85
|
+
* shared `order_no` / name tie all the time. (SQL Server also refuses OFFSET/FETCH — page 2 — with no ORDER BY.)
|
|
86
|
+
*/
|
|
87
|
+
listTieBreaker?: string | false;
|
|
77
88
|
/**
|
|
78
89
|
* Field to apply keyword (ILIKE) search against when filterParams.keyword is set.
|
|
79
90
|
*/
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sdflc/backend-helpers",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.143",
|
|
4
4
|
"description": "Set of helper functions and classes to simplify building backend apps within @sdflc ecosystem",
|
|
5
5
|
"type": "commonjs",
|
|
6
6
|
"exports": {
|
|
@@ -20,7 +20,7 @@
|
|
|
20
20
|
"lint": "ESLINT_USE_FLAT_CONFIG=true eslint .",
|
|
21
21
|
"test": "echo No Tests Available Yet",
|
|
22
22
|
"testToDo": "jest --config jest.config.ts",
|
|
23
|
-
"prove": "node scripts/proveCreateId.cjs && node scripts/proveGateways.cjs && node scripts/proveEmergencyAlerter.cjs",
|
|
23
|
+
"prove": "node scripts/proveCreateId.cjs && node scripts/proveGateways.cjs && node scripts/proveEmergencyAlerter.cjs && node scripts/proveListOrder.cjs",
|
|
24
24
|
"format": "prettier --write \"src/**/*.ts\"",
|
|
25
25
|
"prepare": "husky",
|
|
26
26
|
"prepare1": "husky install",
|