express-file-cluster 0.3.14 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +204 -52
- package/dist/auth/index.cjs +12 -2
- package/dist/auth/index.cjs.map +1 -1
- package/dist/auth/index.d.cts +13 -2
- package/dist/auth/index.d.ts +13 -2
- package/dist/auth/index.js +12 -2
- package/dist/chunk-3VWD2ZED.js +60 -0
- package/dist/chunk-3VWD2ZED.js.map +1 -0
- package/dist/chunk-47K2532L.cjs +196 -0
- package/dist/chunk-47K2532L.cjs.map +1 -0
- package/dist/chunk-5PDAWMGF.js +438 -0
- package/dist/chunk-5PDAWMGF.js.map +1 -0
- package/dist/chunk-7YEX4MQV.cjs +38 -0
- package/dist/chunk-7YEX4MQV.cjs.map +1 -0
- package/dist/chunk-ALBDM5R3.cjs +60 -0
- package/dist/chunk-ALBDM5R3.cjs.map +1 -0
- package/dist/chunk-JJJJTACK.js +196 -0
- package/dist/chunk-JJJJTACK.js.map +1 -0
- package/dist/chunk-LJNSJFOL.js +38 -0
- package/dist/chunk-LJNSJFOL.js.map +1 -0
- package/dist/chunk-M7MUPWCE.js +116 -0
- package/dist/chunk-M7MUPWCE.js.map +1 -0
- package/dist/chunk-PMF4GMHD.js +154 -0
- package/dist/chunk-PMF4GMHD.js.map +1 -0
- package/dist/chunk-QTRGJ2EN.cjs +2635 -0
- package/dist/chunk-QTRGJ2EN.cjs.map +1 -0
- package/dist/chunk-RSBW2B3D.cjs +126 -0
- package/dist/chunk-RSBW2B3D.cjs.map +1 -0
- package/dist/chunk-RTFJ32T5.js +2635 -0
- package/dist/chunk-RTFJ32T5.js.map +1 -0
- package/dist/chunk-SFVEQ2T5.cjs +438 -0
- package/dist/chunk-SFVEQ2T5.cjs.map +1 -0
- package/dist/chunk-SZT36SGB.cjs +154 -0
- package/dist/chunk-SZT36SGB.cjs.map +1 -0
- package/dist/chunk-TI24L4XE.cjs.map +1 -1
- package/dist/chunk-ULQXD4YZ.cjs +116 -0
- package/dist/chunk-ULQXD4YZ.cjs.map +1 -0
- package/dist/chunk-VYHJX54R.js +126 -0
- package/dist/chunk-VYHJX54R.js.map +1 -0
- package/dist/cli/db-runner.cjs +497 -0
- package/dist/cli/db-runner.cjs.map +1 -0
- package/dist/cli/db-runner.d.cts +2 -0
- package/dist/cli/db-runner.d.ts +2 -0
- package/dist/cli/db-runner.js +497 -0
- package/dist/cli/db-runner.js.map +1 -0
- package/dist/cli/index.cjs +447 -69
- package/dist/cli/index.cjs.map +1 -1
- package/dist/cli/index.js +473 -95
- package/dist/cli/index.js.map +1 -1
- package/dist/docs/api-reference/auth.md +232 -0
- package/dist/docs/api-reference/compose.md +78 -0
- package/dist/docs/api-reference/db.md +351 -0
- package/dist/docs/api-reference/http-error.md +109 -0
- package/dist/docs/api-reference/ignite.md +323 -0
- package/dist/docs/api-reference/table.md +250 -0
- package/dist/docs/api-reference/tasks.md +140 -0
- package/dist/docs/cli/db.md +38 -0
- package/dist/docs/cli/diagnostics.md +145 -0
- package/dist/docs/cli/generate.md +120 -0
- package/dist/docs/cli/index.md +48 -0
- package/dist/docs/cli/start.md +112 -0
- package/dist/docs/contributing/index.md +139 -0
- package/dist/docs/core-concepts/background-tasks.md +228 -0
- package/dist/docs/core-concepts/clustering.md +114 -0
- package/dist/docs/core-concepts/file-based-routing.md +235 -0
- package/dist/docs/core-concepts/index.md +79 -0
- package/dist/docs/core-concepts/middleware.md +144 -0
- package/dist/docs/getting-started/index.md +91 -0
- package/dist/docs/getting-started/project-structure.md +154 -0
- package/dist/docs/guides/ai-agents.md +48 -0
- package/dist/docs/guides/authentication.md +196 -0
- package/dist/docs/guides/caching.md +124 -0
- package/dist/docs/guides/database.md +217 -0
- package/dist/docs/guides/deployment.md +171 -0
- package/dist/docs/guides/environment-variables.md +106 -0
- package/dist/docs/guides/error-handling.md +172 -0
- package/dist/docs/guides/generated-portals.md +115 -0
- package/dist/docs/guides/logging.md +70 -0
- package/dist/docs/guides/mailer.md +166 -0
- package/dist/docs/guides/migrations.md +106 -0
- package/dist/docs/guides/rbac.md +89 -0
- package/dist/docs/guides/validation.md +95 -0
- package/dist/docs/index.md +93 -0
- package/dist/docs/skills/SKILL.md +44 -0
- package/dist/docs/skills/api-surface.md +308 -0
- package/dist/docs/skills/auth-internals.md +141 -0
- package/dist/docs/skills/pre-flight.md +142 -0
- package/dist/docs/skills/routing-rules.md +114 -0
- package/dist/docs/skills/tasks-internals.md +172 -0
- package/dist/docs/skills/what-not-to-invent.md +251 -0
- package/dist/index.cjs +289 -391
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +472 -7
- package/dist/index.d.ts +472 -7
- package/dist/index.js +283 -385
- package/dist/index.js.map +1 -1
- package/dist/pgboss-backend-3E6ATY42.cjs +14 -0
- package/dist/pgboss-backend-3E6ATY42.cjs.map +1 -0
- package/dist/pgboss-backend-UUJOVEYK.js +14 -0
- package/dist/pgboss-backend-UUJOVEYK.js.map +1 -0
- package/dist/tasks/index.cjs.map +1 -1
- package/dist/tasks/index.d.cts +1 -1
- package/dist/tasks/index.d.ts +1 -1
- package/dist/types-DJCPAsM4.d.cts +541 -0
- package/dist/types-DJCPAsM4.d.ts +541 -0
- package/package.json +28 -5
- package/dist/chunk-6522HS2J.cjs +0 -99
- package/dist/chunk-6522HS2J.cjs.map +0 -1
- package/dist/chunk-DGVOJNR6.cjs +0 -55
- package/dist/chunk-DGVOJNR6.cjs.map +0 -1
- package/dist/chunk-KOHLLUN7.js +0 -55
- package/dist/chunk-KOHLLUN7.js.map +0 -1
- package/dist/chunk-VIP3PFWZ.js +0 -99
- package/dist/chunk-VIP3PFWZ.js.map +0 -1
- package/dist/types-BusbSenn.d.cts +0 -162
- package/dist/types-BusbSenn.d.ts +0 -162
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
# Database Guide
|
|
2
|
+
|
|
3
|
+
EFC supports **MongoDB** (via mongoose) and **PostgreSQL** (via Drizzle) behind one model API. Application code — `defineModel`, filters, updates, transactions — is the same on both; only `DATABASE_URL` and the installed driver differ.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Choosing an engine
|
|
8
|
+
|
|
9
|
+
`ignite()` picks the engine from `database`, or from the `DATABASE_URL` scheme when `database` is omitted:
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
mongodb://… mongodb+srv://… → 'mongodb'
|
|
13
|
+
postgres://… postgresql://… → 'postgresql'
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
| | MongoDB | PostgreSQL |
|
|
17
|
+
|---|---|---|
|
|
18
|
+
| Install | `npm install mongoose` | `npm install pg drizzle-orm` |
|
|
19
|
+
| `DATABASE_URL` | `mongodb://user:pass@host:27017/db` | `postgres://user:pass@host:5432/db` |
|
|
20
|
+
| Record ids | `ObjectId` strings | UUIDs |
|
|
21
|
+
| Embedded objects / arrays | subdocuments | `jsonb` columns |
|
|
22
|
+
| Sequences | `efc_counters` collection | `SEQUENCE` |
|
|
23
|
+
| `db` client | mongoose `Connection` | node-postgres `Pool` |
|
|
24
|
+
| Transactions | need a replica set | always available |
|
|
25
|
+
| `aggregate()` | ✓ | use `db.query()` / Drizzle via `raw()` |
|
|
26
|
+
| TTL indexes | ✓ | — |
|
|
27
|
+
|
|
28
|
+
```ts
|
|
29
|
+
// efc.config.ts — ignite() never reads process.env itself
|
|
30
|
+
const config: EFCConfig = {
|
|
31
|
+
databaseUrl: process.env.DATABASE_URL,
|
|
32
|
+
// database: 'postgresql', // optional — inferred from the URL
|
|
33
|
+
};
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Each worker process connects once during Pre-Flight, before routes mount — no cold connection on the first request.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Defining models
|
|
41
|
+
|
|
42
|
+
```ts
|
|
43
|
+
// src/model/Post.ts
|
|
44
|
+
import { defineModel, type InferRecord } from 'express-file-cluster';
|
|
45
|
+
|
|
46
|
+
export const Post = defineModel('Post', {
|
|
47
|
+
title: { type: 'string', required: true, minLength: 3, maxLength: 120 },
|
|
48
|
+
slug: { type: 'string', required: true, unique: true, match: /^[a-z0-9-]+$/ },
|
|
49
|
+
status: { type: 'string', enum: ['draft', 'published', 'archived'], default: 'draft', index: true },
|
|
50
|
+
author: { type: 'objectId', ref: 'User', required: true, default: '$currentUser.id' },
|
|
51
|
+
seo: { description: { type: 'string', maxLength: 160 }, keywords: { type: 'array', of: 'string' } },
|
|
52
|
+
revisions: [{ body: { type: 'string', required: true }, at: { type: 'date', default: '$now' } }],
|
|
53
|
+
});
|
|
54
|
+
|
|
55
|
+
export type Post = InferRecord<typeof Post>;
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
The types come from the schema — no interface to write. `Post.create()` requires `title` and `slug` (required, no default), `status` is `'draft' | 'published' | 'archived'`, and every record has `id`, `createdAt` and `updatedAt`. A typo in a field name or an enum value is a compile error.
|
|
59
|
+
|
|
60
|
+
Full field reference: [Database API](../api-reference/db.md).
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## The schema takes care of itself
|
|
65
|
+
|
|
66
|
+
On both engines you just define the model and use it. The first time a model is used in a process, EFC reconciles its table (PostgreSQL) or collection and indexes (MongoDB) with the definition — the way mongoose has always created collections and indexes for you.
|
|
67
|
+
|
|
68
|
+
Editing a model later works the same on both engines:
|
|
69
|
+
|
|
70
|
+
| You change… | MongoDB | PostgreSQL (what sync does) |
|
|
71
|
+
|---|---|---|
|
|
72
|
+
| Add a field | new documents have it | `ADD COLUMN` |
|
|
73
|
+
| Remove a field | old documents keep it, it's ignored | the column stays (no data loss), it's ignored |
|
|
74
|
+
| Make a field optional / drop `required` | works | relaxes any `NOT NULL` |
|
|
75
|
+
| Add or remove `enum` values | works | rebuilds the enum `CHECK` (existing rows aren't re-checked) |
|
|
76
|
+
| Change a field's `type` | new writes use it | `ALTER COLUMN … TYPE … USING <cast>` — fails clearly if existing data can't convert |
|
|
77
|
+
| Add / remove `unique`, `index`, `indexes` | indexes synced | indexes created / EFC-created ones dropped |
|
|
78
|
+
|
|
79
|
+
On MongoDB the first query also waits for the collection's indexes, so `unique` is enforced from the very first write. If an index can't be built (e.g. duplicates are already stored) EFC logs a warning and the model keeps working without it — fix the data and run `efc db sync`.
|
|
80
|
+
|
|
81
|
+
Required fields, enums, bounds and custom validators are enforced by the model on writes, as on MongoDB. Columns and rows are never dropped by sync.
|
|
82
|
+
|
|
83
|
+
Fields the model doesn't declare behave like mongoose's defaults on both engines: ignored on writes, treated as absent in filters (`{ legacy: null }` and `{ legacy: { $exists: false } }` match), ignored in `sort`/`select`.
|
|
84
|
+
|
|
85
|
+
**Seeing and controlling it:**
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
efc db sync --dry-run # show exactly what would change (SQL on PostgreSQL)
|
|
89
|
+
efc db sync # apply it now
|
|
90
|
+
efc db studio # browse rows and each table's structure
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Prefer every schema change to go through review? Set `autoSync: false` in `ignite()` and apply changes with `efc db sync` or migrations (`Model.sync()` in `up()`). See [Migrations & Seeds](./migrations.md).
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Querying
|
|
98
|
+
|
|
99
|
+
```ts
|
|
100
|
+
// Published posts tagged "node", newest first, second page
|
|
101
|
+
const posts = await Post.find(
|
|
102
|
+
{ status: 'published', 'seo.keywords': 'node' },
|
|
103
|
+
{ sort: { createdAt: -1 }, skip: 20, limit: 20, populate: 'author' },
|
|
104
|
+
);
|
|
105
|
+
|
|
106
|
+
const total = await Post.count({ status: 'published' });
|
|
107
|
+
const recent = await Post.find({ createdAt: { $gte: new Date(Date.now() - 7 * 864e5) } });
|
|
108
|
+
const either = await Post.find({ $or: [{ status: 'draft' }, { title: { $regex: /wip/i } }] });
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Filters, sorting and pagination behave the same on both engines.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Writing
|
|
116
|
+
|
|
117
|
+
```ts
|
|
118
|
+
const post = await Post.create({ title: 'Hello', slug: 'hello', seo: { keywords: ['intro'] } });
|
|
119
|
+
|
|
120
|
+
await Post.update(post.id, { status: 'published' });
|
|
121
|
+
await Post.update(post.id, { $push: { revisions: { body: 'v2' } }, $set: { 'seo.description': 'First post' } });
|
|
122
|
+
|
|
123
|
+
const archived = await Post.updateMany({ status: 'draft', createdAt: { $lt: cutoff } }, { $set: { status: 'archived' } });
|
|
124
|
+
const removed = await Post.deleteMany({ status: 'archived' });
|
|
125
|
+
|
|
126
|
+
// One record by filter, in one statement
|
|
127
|
+
const claimed = await Post.updateOne({ status: 'draft' }, { status: 'published' });
|
|
128
|
+
await Post.deleteOne({ slug: 'hello' });
|
|
129
|
+
|
|
130
|
+
// Insert-or-update (put a unique index on the filtered field)
|
|
131
|
+
await Post.upsert({ slug: 'about' }, { title: 'About', slug: 'about' }, { $set: { title: 'About' } });
|
|
132
|
+
|
|
133
|
+
if (await Post.exists({ slug })) throw new HttpError(409, 'Slug taken');
|
|
134
|
+
const statuses = await Post.distinct('status'); // ('draft' | 'published' | 'archived')[]
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Every write is validated — including updates. Invalid data throws `ModelValidationError`, which the global error handler turns into a **400** with field `issues`; a duplicate `unique` value becomes a **409**. Route handlers usually don't need to catch either.
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Transactions
|
|
142
|
+
|
|
143
|
+
```ts
|
|
144
|
+
import { transaction } from 'express-file-cluster';
|
|
145
|
+
|
|
146
|
+
await transaction(async () => {
|
|
147
|
+
const order = await Order.create({ total: 40 });
|
|
148
|
+
await Wallet.update(walletId, { $inc: { balance: -40 } });
|
|
149
|
+
await Ledger.create({ order: order.id, amount: -40 });
|
|
150
|
+
});
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Model calls inside join the transaction through `AsyncLocalStorage` — no session or `tx` objects to pass around. On MongoDB this needs a replica set.
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## PostgreSQL-native: `defineTable`
|
|
158
|
+
|
|
159
|
+
Building on PostgreSQL only? `defineTable` gives you Prisma's query vocabulary and SQL types,
|
|
160
|
+
on the same foundation (schema format, validation, auto-sync, `transaction()`, inferred types):
|
|
161
|
+
|
|
162
|
+
```ts
|
|
163
|
+
import { defineTable } from 'express-file-cluster';
|
|
164
|
+
|
|
165
|
+
export const Order = defineTable('Order', {
|
|
166
|
+
number: { type: 'string', required: true, unique: true, maxLength: 20 },
|
|
167
|
+
status: { type: 'string', enum: ['open', 'paid'], default: 'open', nativeEnum: true },
|
|
168
|
+
total: { type: 'decimal', precision: 10, scale: 2, required: true }, // '19.99', exact
|
|
169
|
+
qty: { type: 'integer', required: true },
|
|
170
|
+
tags: { type: 'array', of: 'string', default: [] }, // text[]
|
|
171
|
+
}, { id: 'serial', naming: 'snake_case' });
|
|
172
|
+
|
|
173
|
+
const page = await Order.findMany({
|
|
174
|
+
where: { status: 'paid', total: { gte: '100' }, tags: { has: 'priority' } },
|
|
175
|
+
orderBy: { createdAt: 'desc' },
|
|
176
|
+
take: 20, cursor: { id: lastId }, skip: 1,
|
|
177
|
+
select: { id: true, number: true, total: true },
|
|
178
|
+
});
|
|
179
|
+
await Order.upsert({ where: { number }, create: { number, total: '0', qty: 1 }, update: { qty: { increment: 1 } } });
|
|
180
|
+
const revenue = await Order.groupBy({ by: ['status'], _sum: { total: true }, orderBy: { _sum: { total: 'desc' } } });
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
`defineModel` and `defineTable` can share a table, so you can move models over one at a time.
|
|
184
|
+
Full reference: [defineTable API](../api-reference/table.md).
|
|
185
|
+
|
|
186
|
+
---
|
|
187
|
+
|
|
188
|
+
## Beyond the model API
|
|
189
|
+
|
|
190
|
+
```ts
|
|
191
|
+
import { db } from 'express-file-cluster';
|
|
192
|
+
|
|
193
|
+
// MongoDB — aggregation
|
|
194
|
+
const byRole = await User.aggregate([{ $group: { _id: '$role', n: { $sum: 1 } } }]);
|
|
195
|
+
|
|
196
|
+
// PostgreSQL — SQL on the pool
|
|
197
|
+
const { rows } = await db.query('SELECT date_trunc($1, "createdAt") d, count(*) FROM posts GROUP BY d', ['day']);
|
|
198
|
+
|
|
199
|
+
// Either — the native model/table
|
|
200
|
+
const PostModel = await Post.raw(); // mongoose Model or Drizzle pgTable
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
---
|
|
204
|
+
|
|
205
|
+
## Models in tasks and migrations
|
|
206
|
+
|
|
207
|
+
Models work the same in task handlers, migrations and seeds:
|
|
208
|
+
|
|
209
|
+
```ts
|
|
210
|
+
// src/tasks/PurgeSessions.ts
|
|
211
|
+
import { defineTask } from 'express-file-cluster/tasks';
|
|
212
|
+
import { Session } from '../model/Session.js';
|
|
213
|
+
|
|
214
|
+
export default defineTask({ schedule: '0 * * * *' }, async () => {
|
|
215
|
+
await Session.deleteMany({ expiresAt: { $lt: new Date() } });
|
|
216
|
+
});
|
|
217
|
+
```
|
|
@@ -0,0 +1,171 @@
|
|
|
1
|
+
# Deployment Guide
|
|
2
|
+
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
## Build step
|
|
6
|
+
|
|
7
|
+
Before deploying, compile TypeScript to JavaScript:
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npm run build
|
|
11
|
+
# → efc build prod
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
`efc build prod` type-checks, then transpiles every file in `src/` (plus `efc.config.ts`) into `dist/`, keeping the directory tree so routes and tasks are found at runtime. The entry point for production is `dist/src/index.js`.
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Running in production
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
npm start
|
|
22
|
+
# → efc start prod
|
|
23
|
+
# → node dist/src/index.js (NODE_ENV=production)
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
`efc start prod` sets `NODE_ENV=production` and spawns the compiled server with `node`. It does **not** read `.env` — secrets must be injected through the platform's secret management.
|
|
27
|
+
|
|
28
|
+
Stopping is graceful at every layer: `efc start prod` forwards `SIGTERM`/`SIGINT` to the server, the cluster primary relays it to each worker, and each worker stops accepting connections, finishes in-flight requests and task jobs, then exits. This works whether the signal reaches the whole process group (Ctrl+C) or only PID 1 (`docker stop`, Kubernetes).
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Environment variables
|
|
33
|
+
|
|
34
|
+
Set these in your deployment platform:
|
|
35
|
+
|
|
36
|
+
| Variable | Notes |
|
|
37
|
+
|---|---|
|
|
38
|
+
| `NODE_ENV` | Set to `production` — enables clustering, `Secure` cookie, production log level |
|
|
39
|
+
| `PORT` | Usually set by the platform (Heroku, Railway, etc.) |
|
|
40
|
+
| `DATABASE_URL` | Production database connection string |
|
|
41
|
+
| `JWT_SECRET` | Production signing key — different from dev |
|
|
42
|
+
| `REDIS_URL` | Required if using BullMQ (pg-boss uses `DATABASE_URL`) |
|
|
43
|
+
| `CORS_ORIGINS` | Your production frontend domain(s) |
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## Migrations
|
|
48
|
+
|
|
49
|
+
Tables and indexes follow the models automatically (`autoSync`). If you turn that off, run `npx efc db sync --dist` as a release step. Data migrations (renames, backfills) still go through `efc db migrate` — run pending ones before the new version starts serving, as a release step on your platform or in the container command:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
npx efc db migrate --dist # --dist: use compiled dist/src/migrations (no tsx in production)
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
On PostgreSQL concurrent `migrate` runs take an advisory lock, so it's safe if several instances run it at deploy time. See [Migrations & Seeds](./migrations.md).
|
|
56
|
+
|
|
57
|
+
## Clustering in production
|
|
58
|
+
|
|
59
|
+
With `NODE_ENV=production`, `cluster` defaults to `true`. The primary process forks `os.cpus().length` workers. To override:
|
|
60
|
+
|
|
61
|
+
```ts
|
|
62
|
+
ignite({ cluster: true, workers: 2 });
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
For containerised deployments where you want one process per container (and rely on the orchestrator for scaling), set:
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
ignite({ cluster: false });
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Or use a single-CPU container image and let `os.cpus().length` naturally be `1`.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
## Graceful shutdown
|
|
76
|
+
|
|
77
|
+
Always wire `gracefulShutdown` to handle SIGTERM (sent by orchestrators before replacing a pod/dyno):
|
|
78
|
+
|
|
79
|
+
```ts
|
|
80
|
+
ignite(config)
|
|
81
|
+
.then(gracefulShutdown);
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
The default drain window is 10 seconds. Adjust for long-running requests:
|
|
85
|
+
|
|
86
|
+
```ts
|
|
87
|
+
.then((server) => gracefulShutdown(server, 30_000));
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
What happens on `SIGTERM`:
|
|
91
|
+
|
|
92
|
+
1. `efc start prod` (if it's your entrypoint) forwards the signal to the server process.
|
|
93
|
+
2. The cluster primary relays it to every worker — orchestrators like `docker stop` and Kubernetes signal only PID 1.
|
|
94
|
+
3. Each worker stops accepting connections, finishes in-flight requests and running task jobs, then exits `0`.
|
|
95
|
+
4. The primary exits once all workers have; it force-exits after `timeoutMs + 2s` if one hangs.
|
|
96
|
+
|
|
97
|
+
Set your orchestrator's grace period (`terminationGracePeriodSeconds`, `stop_grace_period`) above `timeoutMs`.
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## Docker
|
|
102
|
+
|
|
103
|
+
```dockerfile
|
|
104
|
+
FROM node:20-alpine AS builder
|
|
105
|
+
WORKDIR /app
|
|
106
|
+
COPY package*.json ./
|
|
107
|
+
RUN npm ci
|
|
108
|
+
COPY . .
|
|
109
|
+
RUN npm run build
|
|
110
|
+
|
|
111
|
+
FROM node:20-alpine
|
|
112
|
+
WORKDIR /app
|
|
113
|
+
COPY package*.json ./
|
|
114
|
+
RUN npm ci --omit=dev
|
|
115
|
+
COPY --from=builder /app/dist ./dist
|
|
116
|
+
EXPOSE 3000
|
|
117
|
+
CMD ["sh", "-c", "npx efc db migrate --dist && node dist/src/index.js"]
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Inject environment variables via `docker run -e` or a secrets manager. Do not copy `.env` into the image.
|
|
121
|
+
|
|
122
|
+
---
|
|
123
|
+
|
|
124
|
+
## Railway / Render / Heroku
|
|
125
|
+
|
|
126
|
+
These platforms:
|
|
127
|
+
|
|
128
|
+
1. Detect `npm start` from `package.json`.
|
|
129
|
+
2. Inject `PORT` automatically.
|
|
130
|
+
3. Let you set env vars through their dashboard.
|
|
131
|
+
|
|
132
|
+
No extra configuration is needed — `npm start` (`efc start prod`) handles the rest.
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## Health check endpoint
|
|
137
|
+
|
|
138
|
+
The scaffolder generates `src/api/health.ts`:
|
|
139
|
+
|
|
140
|
+
```ts
|
|
141
|
+
export const GET = async (_req, res) => {
|
|
142
|
+
res.json({ status: 'OK', timestamp: new Date().toISOString() });
|
|
143
|
+
};
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
This maps to `GET /health`. Point your load balancer / Kubernetes liveness probe at it:
|
|
147
|
+
|
|
148
|
+
```yaml
|
|
149
|
+
livenessProbe:
|
|
150
|
+
httpGet:
|
|
151
|
+
path: /health
|
|
152
|
+
port: 3000
|
|
153
|
+
initialDelaySeconds: 5
|
|
154
|
+
periodSeconds: 10
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
## Production checklist
|
|
160
|
+
|
|
161
|
+
- [ ] `NODE_ENV=production` set in the platform
|
|
162
|
+
- [ ] `JWT_SECRET` is a 64-byte random hex string unique to production
|
|
163
|
+
- [ ] `.env` is **not** copied into the Docker image or committed to git
|
|
164
|
+
- [ ] `CORS_ORIGINS` is set to your actual frontend domain(s)
|
|
165
|
+
- [ ] `gracefulShutdown` is wired in `src/index.ts`
|
|
166
|
+
- [ ] Health check endpoint is configured in the load balancer
|
|
167
|
+
- [ ] Redis is available if using BullMQ tasks (pg-boss needs only Postgres)
|
|
168
|
+
- [ ] `efc db migrate` runs before the new version serves traffic
|
|
169
|
+
- [ ] Orchestrator grace period > `gracefulShutdown` timeout
|
|
170
|
+
- [ ] Logs are collected from stdout (JSON lines with `reqId`)
|
|
171
|
+
- [ ] `efc doctor` passes locally with production-like env vars
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# Environment Variables
|
|
2
|
+
|
|
3
|
+
`ignite()` never reads `process.env` for application config — the one exception is `NODE_ENV`, a Node/Express-wide runtime-mode signal, not an app secret (see [Precedence](#precedence)). Every other runtime value (`PORT`, `DATABASE_URL`, `JWT_SECRET`, `CORS_ORIGINS`, ...) has to be read from `process.env` explicitly and passed into `ignite()` — there is no framework-level fallback to grab it for you.
|
|
4
|
+
|
|
5
|
+
The scaffolder wires this for you: it generates `.env` (gitignored, secrets pre-filled), `.env.example` (committed, documented placeholders), and `efc.config.ts` — which reads each variable explicitly and builds the `EFCConfig` object that `src/index.ts` spreads into `ignite()`. `efc.config.ts` is the single source of truth for what's actually applied; `.env` only matters because `efc.config.ts` chooses to read it.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Variable reference
|
|
10
|
+
|
|
11
|
+
These are the variables the **scaffolded** `efc.config.ts` reads. If you hand-write your own config file, this list is a menu, not a contract — read only what you need.
|
|
12
|
+
|
|
13
|
+
| Variable | Required | Default | Wired into |
|
|
14
|
+
|---|---|---|---|
|
|
15
|
+
| `PORT` | No | `3000` | `config.port` |
|
|
16
|
+
| `NODE_ENV` | No | — | Read directly by the framework (not via config) — see [Precedence](#precedence). |
|
|
17
|
+
| `DATABASE_URL` | Yes (if using a database) | — | `config.databaseUrl`. Format determines the engine: `mongodb://...` or `postgres://...`. Also used by pg-boss and by `efc db` / `efc tasks` commands. |
|
|
18
|
+
| `JWT_SECRET` | Yes (if using auth) | — | `config.jwtSecret`. Generate with `openssl rand -hex 64`. |
|
|
19
|
+
| `JWT_EXPIRES_IN` | No | `7d` | `config.jwtExpiresIn`. Accepts `15m`, `1h`, `7d`, `30d`, etc. |
|
|
20
|
+
| `COOKIE_DOMAIN` | No | — | `config.cookieDomain`. Blank for localhost. |
|
|
21
|
+
| `REDIS_URL` | If tasks use BullMQ | `redis://localhost:6379` | `config.tasks.redisUrl`. Supports `rediss://` (TLS), ACL users and `/db`. Omitted for pg-boss projects — `efc tasks failed` picks BullMQ when it's set. |
|
|
22
|
+
| `EFC_DEBUG` | No | — | `1` makes `efc db` commands print full stack traces on failure. |
|
|
23
|
+
| `CORS_ORIGINS` | No | (all allowed) | `config.cors.origin` (comma-separated list, split into an array). Example: `http://localhost:3000,https://myapp.com` |
|
|
24
|
+
| `APP_URL` | If Mailer enabled | `http://localhost:3000` | Read directly in task/route code that builds email links (e.g. `${APP_URL}/auth/verify-email?token=...`) — not part of `EFCConfig`. |
|
|
25
|
+
| `SMTP_HOST` | If Mailer enabled | `smtp.gmail.com` | Read directly by the generated mailer task — not part of `EFCConfig`. |
|
|
26
|
+
| `SMTP_PORT` | If Mailer enabled | `465` (Gmail) / `587` (custom) | Same as above. `465` implies TLS (`secure: true`). |
|
|
27
|
+
| `SMTP_USER` | If Mailer enabled | — | Same as above. Sending email address / SMTP username. |
|
|
28
|
+
| `SMTP_PASS` | If Mailer enabled | — | Same as above. **For Gmail this must be a 16-character App Password** (Google Account → Security → 2-Step Verification → App passwords) — your regular Gmail password will be rejected by Google. See [Mailer](./mailer.md). |
|
|
29
|
+
| `SMTP_FROM` | If Mailer enabled | same as `SMTP_USER` | Same as above. `From:` address on outgoing mail. |
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Precedence
|
|
34
|
+
|
|
35
|
+
There's no env-vs-config precedence to reason about for most values, because the framework itself never reads them from `process.env` — `config.X` is whatever your `efc.config.ts` (or hand-written `ignite()` call) computes it to be. If you want an env var to win, read it yourself:
|
|
36
|
+
|
|
37
|
+
```ts
|
|
38
|
+
// efc.config.ts
|
|
39
|
+
const config: EFCConfig = {
|
|
40
|
+
port: process.env.PORT ? Number(process.env.PORT) : undefined,
|
|
41
|
+
jwtSecret: process.env.JWT_SECRET,
|
|
42
|
+
};
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
`NODE_ENV` is the one exception — it's a Node/Express-wide runtime-mode signal set by the platform (not authored in your app's `.env`), and the framework still reads it directly in three places: the cluster-in-production default (`config.cluster ?? process.env.NODE_ENV === 'production'`), the dev-only dashboard toggle, and the `http-only` auth strategy's cookie `Secure` flag. An explicit `config.cluster` always overrides the `NODE_ENV`-derived default.
|
|
46
|
+
|
|
47
|
+
---
|
|
48
|
+
|
|
49
|
+
## Secret hygiene
|
|
50
|
+
|
|
51
|
+
- **Never commit `.env`.** The scaffolder adds it to `.gitignore` automatically.
|
|
52
|
+
- **Use a different `JWT_SECRET` per environment** (dev, staging, production). Rotating the secret invalidates all existing tokens.
|
|
53
|
+
- **Regenerate if exposed:**
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
openssl rand -hex 64
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Paste the output into your `.env` and your deployment platform's secret store.
|
|
60
|
+
|
|
61
|
+
- **Do not log secrets.** EFC never echoes `JWT_SECRET` or `REDIS_URL` to stdout.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## `.env` vs `efc.config.ts`
|
|
66
|
+
|
|
67
|
+
| File | Purpose | Committed? |
|
|
68
|
+
|---|---|---|
|
|
69
|
+
| `.env` | Runtime secrets and per-environment values, read by `efc.config.ts` | **No** |
|
|
70
|
+
| `.env.example` | Documented template of every required variable | Yes |
|
|
71
|
+
| `efc.config.ts` | Reads `.env` explicitly and builds the `EFCConfig` object `ignite()` receives — both runtime values and structural choices (`authStrategy`, `tasks.backend`) live here | Yes |
|
|
72
|
+
|
|
73
|
+
There is no `apiDir`/`tasksDir` option — `src/api/` and `src/tasks/` are fixed conventions.
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Development `.env` example
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
PORT=3000
|
|
81
|
+
NODE_ENV=development
|
|
82
|
+
|
|
83
|
+
DATABASE_URL=mongodb://localhost:27017/my-api-dev
|
|
84
|
+
|
|
85
|
+
JWT_SECRET=<generated-by-scaffolder>
|
|
86
|
+
JWT_EXPIRES_IN=7d
|
|
87
|
+
|
|
88
|
+
REDIS_URL=redis://localhost:6379
|
|
89
|
+
|
|
90
|
+
CORS_ORIGINS=http://localhost:5173
|
|
91
|
+
|
|
92
|
+
# Only present if the Mailer feature was enabled during scaffolding
|
|
93
|
+
SMTP_HOST=smtp.gmail.com
|
|
94
|
+
SMTP_PORT=465
|
|
95
|
+
SMTP_USER=you@example.com
|
|
96
|
+
SMTP_PASS=abcdefghijklmnop # Gmail App Password (16 chars) — not your Gmail login password
|
|
97
|
+
SMTP_FROM=you@example.com
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## Loading in development
|
|
103
|
+
|
|
104
|
+
In development (`efc start dev`), the CLI loads `.env` into `process.env` before your app runs, so `efc.config.ts`'s `process.env.X` reads see it. It re-reads `.env` fresh on every `tsx watch` restart — editing `.env` and saving a source file is enough to pick up the change, no need to stop and restart `efc start dev` itself. Variables already present in `process.env` (e.g. real OS-exported vars, CI-injected secrets) take precedence over `.env`.
|
|
105
|
+
|
|
106
|
+
In production (`efc start prod`), `.env` is **not** loaded. Platforms (Docker, Kubernetes, Railway, Heroku) inject secrets directly into `process.env`, and `efc.config.ts`'s `process.env.X` reads pick those up the same way. This is intentional — avoid reading secret files from disk in production.
|
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
# Error Handling Guide
|
|
2
|
+
|
|
3
|
+
EFC provides a structured error model that converts exceptions into clean HTTP responses without try/catch in every handler.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## How it works
|
|
8
|
+
|
|
9
|
+
Every route handler is wrapped in `asyncWrap`, which passes any thrown error or rejected promise to Express's `next(err)`. The global error handler at the bottom of the middleware stack then inspects the error and responds.
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
throw new HttpError(404, 'Not found')
|
|
13
|
+
↓
|
|
14
|
+
asyncWrap catches it → next(err)
|
|
15
|
+
↓
|
|
16
|
+
EFC global error handler
|
|
17
|
+
↓
|
|
18
|
+
HTTP 404 { "error": "Not found", "statusCode": 404 }
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## `HttpError`
|
|
24
|
+
|
|
25
|
+
Throw `HttpError` to signal a specific HTTP status:
|
|
26
|
+
|
|
27
|
+
```ts
|
|
28
|
+
import { HttpError } from 'express-file-cluster';
|
|
29
|
+
|
|
30
|
+
export const GET = async (req, res) => {
|
|
31
|
+
const user = await User.findById(req.params.id);
|
|
32
|
+
if (!user) throw new HttpError(404, 'User not found');
|
|
33
|
+
res.json(user);
|
|
34
|
+
};
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
The default response shape:
|
|
38
|
+
|
|
39
|
+
```json
|
|
40
|
+
{
|
|
41
|
+
"error": "User not found",
|
|
42
|
+
"statusCode": 404
|
|
43
|
+
}
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
See the full [`HttpError` reference](../api-reference/http-error.md).
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Unhandled errors
|
|
51
|
+
|
|
52
|
+
Any `throw` that is not an `HttpError` (e.g. a database error, a null pointer, an unexpected exception) is treated as a server error:
|
|
53
|
+
|
|
54
|
+
```json
|
|
55
|
+
{
|
|
56
|
+
"error": "Internal Server Error",
|
|
57
|
+
"statusCode": 500
|
|
58
|
+
}
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The original error is logged at `error` level with the request ID and stack (see [Logging](./logging.md)):
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
12:00:01 ERROR [w1] 3f9c2a1b Unhandled error
|
|
65
|
+
TypeError: Cannot read properties of null
|
|
66
|
+
at GET (src/api/users/[id].ts:8:21)
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## Validation and database errors
|
|
72
|
+
|
|
73
|
+
These arrive as `HttpError` subclasses, so they need no handling in route code:
|
|
74
|
+
|
|
75
|
+
| Source | Status | Body |
|
|
76
|
+
|---|---|---|
|
|
77
|
+
| Route `schema` rejects the request | `400` | `{ "error": "Validation failed", "statusCode": 400, "issues": [{ "location", "path", "message" }] }` |
|
|
78
|
+
| `defineModel` write fails validation (`ModelValidationError`) | `400` | `{ "error": "User validation failed: email: is required", "statusCode": 400, "issues": [{ "path", "message" }] }` |
|
|
79
|
+
| Duplicate value of a `unique` field | `409` | `{ "error": "User with this email already exists", "statusCode": 409 }` |
|
|
80
|
+
|
|
81
|
+
Catch `ModelValidationError` only when you want to react differently:
|
|
82
|
+
|
|
83
|
+
```ts
|
|
84
|
+
import { ModelValidationError } from 'express-file-cluster';
|
|
85
|
+
|
|
86
|
+
try {
|
|
87
|
+
await User.create(req.body);
|
|
88
|
+
} catch (err) {
|
|
89
|
+
if (err instanceof ModelValidationError && err.issues.some((i) => i.path === 'email')) {
|
|
90
|
+
throw new HttpError(422, 'Please provide a valid email');
|
|
91
|
+
}
|
|
92
|
+
throw err;
|
|
93
|
+
}
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## Custom global error handler
|
|
99
|
+
|
|
100
|
+
Override the built-in handler with `onError` in `ignite()`:
|
|
101
|
+
|
|
102
|
+
```ts
|
|
103
|
+
import { HttpError } from 'express-file-cluster';
|
|
104
|
+
import type { ErrorRequestHandler } from 'express';
|
|
105
|
+
|
|
106
|
+
const errorHandler: ErrorRequestHandler = (err, req, res, _next) => {
|
|
107
|
+
const status = err instanceof HttpError ? err.statusCode : 500;
|
|
108
|
+
const message = err instanceof HttpError ? err.message : 'Internal Server Error';
|
|
109
|
+
|
|
110
|
+
logger.error({ err, url: req.url, method: req.method });
|
|
111
|
+
|
|
112
|
+
res.status(status).json({
|
|
113
|
+
error: message,
|
|
114
|
+
path: req.path,
|
|
115
|
+
requestId: req.headers['x-request-id'],
|
|
116
|
+
});
|
|
117
|
+
};
|
|
118
|
+
|
|
119
|
+
ignite({ onError: errorHandler });
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
---
|
|
123
|
+
|
|
124
|
+
## Errors in middleware
|
|
125
|
+
|
|
126
|
+
Middleware errors are forwarded the same way — call `next(err)` or `throw` from an async middleware and the global handler picks it up:
|
|
127
|
+
|
|
128
|
+
```ts
|
|
129
|
+
// src/middlewares/validate.ts
|
|
130
|
+
import type { RequestHandler } from 'express';
|
|
131
|
+
import { HttpError } from 'express-file-cluster';
|
|
132
|
+
import { z } from 'zod';
|
|
133
|
+
|
|
134
|
+
export function validateBody(schema: z.ZodType): RequestHandler {
|
|
135
|
+
return async (req, _res, next) => {
|
|
136
|
+
const result = schema.safeParse(req.body);
|
|
137
|
+
if (!result.success) {
|
|
138
|
+
return next(new HttpError(422, result.error.message));
|
|
139
|
+
}
|
|
140
|
+
req.body = result.data;
|
|
141
|
+
next();
|
|
142
|
+
};
|
|
143
|
+
}
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
## Errors in tasks
|
|
149
|
+
|
|
150
|
+
Task errors do not crash the application. The backend (BullMQ or pg-boss) catches failed jobs, retries them according to `retries` and `backoff`, and dead-letters them after the last attempt. Each failure is logged with the task, job ID and attempt:
|
|
151
|
+
|
|
152
|
+
```
|
|
153
|
+
12:00:01 ERROR [w2] Task SendEmail failed task=SendEmail jobId=1842 attempt=2
|
|
154
|
+
Error: SMTP connection refused
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Inspect, retry or purge dead-lettered jobs with `efc tasks failed | retry | purge` — see [Background Tasks › Dead letters](../core-concepts/background-tasks.md#dead-letters).
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## Error response reference
|
|
162
|
+
|
|
163
|
+
| Scenario | Status | Body |
|
|
164
|
+
|---|---|---|
|
|
165
|
+
| `throw new HttpError(N, msg)` | `N` | `{ "error": msg, "statusCode": N }` |
|
|
166
|
+
| Any other thrown error | `500` | `{ "error": "Internal Server Error", "statusCode": 500 }` |
|
|
167
|
+
| Method not implemented | `405` | `{ "error": "Method Not Allowed" }` + `Allow` header |
|
|
168
|
+
| Route `schema` validation failed | `400` | `{ "error": "Validation failed", "statusCode": 400, "issues": [...] }` |
|
|
169
|
+
| Model validation failed | `400` | `{ "error": "...", "statusCode": 400, "issues": [...] }` |
|
|
170
|
+
| Unique constraint violated | `409` | `{ "error": "<Model> with this <field> already exists", "statusCode": 409 }` |
|
|
171
|
+
| `requestTimeout` exceeded | `408` | `{ "error": "Request Timeout", "statusCode": 408 }` |
|
|
172
|
+
| Missing/invalid auth token | `401` | `{ "error": "Unauthorized" }` |
|