@luckystack/devkit 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/CLAUDE.md +111 -0
- package/LICENSE +21 -0
- package/README.md +44 -0
- package/dist/cli/validateDeploy.js +318 -0
- package/dist/cli/validateDeploy.js.map +1 -0
- package/dist/index.d.ts +207 -0
- package/dist/index.js +4606 -0
- package/dist/index.js.map +1 -0
- package/dist/templates/api.template.ts +55 -0
- package/dist/templates/page_dashboard.template.tsx +35 -0
- package/dist/templates/page_plain.template.tsx +32 -0
- package/dist/templates/sync_client.template.ts +41 -0
- package/dist/templates/sync_client_paired.template.ts +38 -0
- package/dist/templates/sync_client_standalone.template.ts +36 -0
- package/dist/templates/sync_server.template.ts +65 -0
- package/docs/cli.md +245 -0
- package/docs/hot-reload.md +365 -0
- package/docs/loader-pipeline.md +324 -0
- package/docs/runtime-type-resolver.md +258 -0
- package/docs/supervisor.md +292 -0
- package/docs/template-customization.md +136 -0
- package/docs/ts-program-cache.md +199 -0
- package/docs/type-map-generation.md +392 -0
- package/package.json +65 -0
|
@@ -0,0 +1,324 @@
|
|
|
1
|
+
# Loader Pipeline (`initializeAll`, `initializeApis`, `initializeSyncs`, `initializeFunctions`, upsert/remove)
|
|
2
|
+
|
|
3
|
+
> Dev-only. `@luckystack/devkit` is published as a `devDependency`. Nothing in this document runs in a production bundle — the prod runtime maps loader in `@luckystack/server` reads from in-memory `devApis`/`devSyncs`/`devFunctions` only when `NODE_ENV !== 'production'`, and from the on-disk generated maps otherwise.
|
|
4
|
+
|
|
5
|
+
The loader pipeline is the dev-time mirror of the production route registry. It walks the configured `srcDir` (from `getProjectConfig().paths`) and each `serverFunctionDirs` root (legacy singular `serverFunctionsDir` still honored when set), evaluates each route file via dynamic `import()`, and populates three in-memory maps:
|
|
6
|
+
|
|
7
|
+
| Map | Shape | Mirror of |
|
|
8
|
+
|---|---|---|
|
|
9
|
+
| `devApis` | `Record<string, { main, auth, rateLimit, httpMethod, schema, inputType, inputTypeFilePath }>` keyed by `api/<page-location\|'system'>/<name>/v<n>` | every `_api/<name>_v<n>.ts` |
|
|
10
|
+
| `devSyncs` | `Record<string, { main, auth, inputType, inputTypeFilePath } \| Function>` keyed by `sync/<page-location>/<name>/v<n>_<server\|client>` | every `_sync/<name>_(server\|client)_v<n>.ts` |
|
|
11
|
+
| `devFunctions` | nested `Record<string, unknown>` mirror of the on-disk tree, leaves are merged named + default exports per file | every `<root>/**/*.ts` under each `serverFunctionDirs` root (legacy singular `serverFunctionsDir` still honored when set) |
|
|
12
|
+
|
|
13
|
+
All three maps are mutated in place. Consumers (the dev server boot path, the prod runtime maps loader's dev branch) hold a stable reference to the exported objects and read them on every request.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## `initializeAll()`
|
|
18
|
+
|
|
19
|
+
```typescript
|
|
20
|
+
export const initializeAll = async () => {
|
|
21
|
+
assertValidRouteNaming({
|
|
22
|
+
srcDir: getSrcDir(),
|
|
23
|
+
context: 'starting dev server (npm run server)',
|
|
24
|
+
});
|
|
25
|
+
|
|
26
|
+
await Promise.all([initializeApis(), initializeSyncs(), initializeFunctions()]);
|
|
27
|
+
};
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Order matters:
|
|
31
|
+
|
|
32
|
+
1. **Route naming validation first.** `assertValidRouteNaming` walks the file tree once and throws if any `_api/` or `_sync/` file doesn't match the version regex (e.g. `_v1.ts`, `_server_v2.ts`). Aborting before any file evaluates means a typo can't produce half-loaded state.
|
|
33
|
+
2. **Three initializers in parallel** via `Promise.all`. They share the TypeScript program cache (see `ts-program-cache.md`) — running them sequentially would not be faster, since the cache only gets built on first access from inside the extractors.
|
|
34
|
+
|
|
35
|
+
After `initializeAll()` returns, the dev server should call `setupWatchers()` exactly once (see `hot-reload.md`) so subsequent file changes incrementally mutate the same maps.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## `initializeApis()`
|
|
40
|
+
|
|
41
|
+
```typescript
|
|
42
|
+
export const initializeApis = async () => {
|
|
43
|
+
for (const key of Object.keys(devApis)) delete devApis[key];
|
|
44
|
+
//? No invalidateProgramCache() here — cachedProgram starts as null on
|
|
45
|
+
//? module-load (tsProgram.ts), so the first getServerProgram() call
|
|
46
|
+
//? builds it from scratch. With initializeApis + initializeSyncs running
|
|
47
|
+
//? in parallel via Promise.all, invalidating here forced a redundant
|
|
48
|
+
//? double-build (~3-4s waste). Hot-reload paths (upsertApiFromFile,
|
|
49
|
+
//? removeApiFromFile etc.) DO invalidate — that's where it's needed.
|
|
50
|
+
clearRuntimeTypeResolverCache();
|
|
51
|
+
const srcFolder = fs.readdirSync(getSrcDir());
|
|
52
|
+
|
|
53
|
+
for (const file of srcFolder) {
|
|
54
|
+
await scanApiFolder(file);
|
|
55
|
+
}
|
|
56
|
+
};
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Flow:
|
|
60
|
+
|
|
61
|
+
1. Empty `devApis` by deleting every key. The exported reference stays stable.
|
|
62
|
+
2. **Deliberately skip `invalidateProgramCache()`.** Module-load init in `tsProgram.ts` starts with `cachedProgram = null`, so the first `getServerProgram()` call from inside the extractors will build it from scratch. Calling `invalidateProgramCache()` here used to force a redundant ~3-4s double-build because `initializeApis` and `initializeSyncs` run concurrently and would each invalidate after the other had warmed the cache. The hot-reload upsert/remove paths DO invalidate, because there the cache really is stale.
|
|
63
|
+
3. Clear the runtime type resolver cache (`clearRuntimeTypeResolverCache()`) so any string-based identifier resolution starts fresh.
|
|
64
|
+
4. Recurse the project's `srcDir` via `scanApiFolder`. The walker descends through every directory looking for one whose lowercase name ends in `api` (the resolved `apiMarker` from `getRoutingRules()` — default `_api`).
|
|
65
|
+
5. For each `<name>_v<n>.ts` inside that folder, parse the version with `apiVersionRegex`, import the file, extract `main` / `auth` / `rateLimit` / `httpMethod` / `schema`, derive the inline `inputType` text via `getInputTypeFromFile()`, and store the entry under `api/<pageLocation>/<name>/v<n>`. `pageLocation` is `''` when the API lives at `src/_api/` and is mapped to `'system'` by `mapApiPageLocation()`; otherwise it's the slash-joined path segments above the marker.
|
|
66
|
+
|
|
67
|
+
`auth` is normalized at load time:
|
|
68
|
+
|
|
69
|
+
```typescript
|
|
70
|
+
auth: {
|
|
71
|
+
login: auth.login || false,
|
|
72
|
+
additional: auth.additional || [],
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Missing `main` (or non-function `main`) silently skips the file — the entry is removed and the route key is absent from `devApis`. Filename parse failures log in red to `[loader][api]` and continue.
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## `initializeSyncs()`
|
|
81
|
+
|
|
82
|
+
Same shape as `initializeApis()`, keyed on folders whose lowercase name ends in `sync` (the resolved `syncMarker`). Each `_sync/<name>_(server|client)_v<n>.ts` is matched by `syncVersionRegex`, yielding the `kind` (`server` or `client`) and version. The route base key is:
|
|
83
|
+
|
|
84
|
+
- `sync/<pageLocation>/<syncName>/v<n>` (when `pageLocation` is non-empty)
|
|
85
|
+
- `sync/<syncName>/v<n>` (root-level sync)
|
|
86
|
+
|
|
87
|
+
Final entry key appends `_server` or `_client`. Storage differs by kind:
|
|
88
|
+
|
|
89
|
+
- **Server kind** stores a record:
|
|
90
|
+
```typescript
|
|
91
|
+
devSyncs[`${routeBaseKey}_server`] = {
|
|
92
|
+
main: resolvedSyncModule.main,
|
|
93
|
+
auth: resolvedSyncModule.auth || {},
|
|
94
|
+
inputType, // derived via getSyncClientDataType()
|
|
95
|
+
inputTypeFilePath: filePath,
|
|
96
|
+
};
|
|
97
|
+
```
|
|
98
|
+
- **Client kind** stores the bare `main` callback (the request dispatcher calls it directly with `{ clientOutput, serverOutput }`):
|
|
99
|
+
```typescript
|
|
100
|
+
devSyncs[`${routeBaseKey}_client`] = resolvedSyncModule.main;
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Like `initializeApis()`, the boot path skips `invalidateProgramCache()` and only clears the runtime type resolver cache.
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## `initializeFunctions()`
|
|
108
|
+
|
|
109
|
+
```typescript
|
|
110
|
+
export const initializeFunctions = async () => {
|
|
111
|
+
for (const key of Object.keys(devFunctions)) delete devFunctions[key];
|
|
112
|
+
|
|
113
|
+
for (const serverFunctionsDir of getServerFunctionDirs()) {
|
|
114
|
+
if (fs.existsSync(serverFunctionsDir)) {
|
|
115
|
+
await scanFunctionsFolder(serverFunctionsDir);
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
};
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
> `getServerFunctionDirs()` returns the array from `projectConfig.paths.serverFunctionDirs`. The legacy singular `serverFunctionsDir` form is still honored — when set it is merged into the array at config load.
|
|
122
|
+
|
|
123
|
+
Walks every configured root recursively. Per `.ts` file:
|
|
124
|
+
|
|
125
|
+
1. Import via `importFile()` (see below).
|
|
126
|
+
2. Run `resolveFunctionModule(module, fileName)`:
|
|
127
|
+
- Module with only named exports -> the named exports object (default merged in only if it's the sole export).
|
|
128
|
+
- Module with no named exports but a default export -> `{ [fileName]: defaultExport }`. This lets `import functionName from './function'` show up at `devFunctions[folder]?.[functionName]?.[functionName]`.
|
|
129
|
+
- Module that is itself a function -> returned as-is so it can be merged.
|
|
130
|
+
3. Walk into the `devFunctions` tree along the `basePath` segments, creating nested `Record<string, unknown>` subtrees on demand. Each level is structurally a record but typed as `unknown` after one level of indexing — `resolveFunctionModule` re-narrows before descent.
|
|
131
|
+
4. If a previous scan left a node at the same leaf path, `Object.assign(resolvedFunctionModule, existingAtFileName)` merges them. This is what lets two files in the same folder contribute to the same logical bag of helpers.
|
|
132
|
+
|
|
133
|
+
Failure modes are logged with `[loader][function]` and the file is skipped.
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## Per-file import strategy
|
|
138
|
+
|
|
139
|
+
```typescript
|
|
140
|
+
const importFile = async (absolutePath: string) => {
|
|
141
|
+
const url = `${pathToFileURL(absolutePath).href}?v=${Date.now()}`;
|
|
142
|
+
return import(url);
|
|
143
|
+
};
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
- `pathToFileURL` converts Windows-style paths and special characters into a proper `file://` URL.
|
|
147
|
+
- The `?v=<timestamp>` query is a cachebust that forces the ESM loader to return a fresh evaluation each call. Without it, `import()` is memoized and a saved file would not re-evaluate.
|
|
148
|
+
- Switching from CommonJS `require()` to dynamic `import()` also stops module evaluation from blocking the event loop. With CJS, evaluating a single ~50 ms file held the Socket.io thread; with ESM the evaluation yields and Socket.io stays responsive during a save burst.
|
|
149
|
+
|
|
150
|
+
The `tryCatch` wrapper from `@luckystack/core` captures evaluation errors:
|
|
151
|
+
|
|
152
|
+
```typescript
|
|
153
|
+
const [err, module] = await tryCatch(async () => importFile(routeMeta.absolutePath));
|
|
154
|
+
if (err) {
|
|
155
|
+
console.log(`[loader][api] failed to import ${routeMeta.routeKey} from ${routeMeta.absolutePath}:`, err, 'red');
|
|
156
|
+
return;
|
|
157
|
+
}
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
A failed import leaves the existing `devApis` / `devSyncs` entry untouched (no partial replacement).
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## Route key shape
|
|
165
|
+
|
|
166
|
+
API key:
|
|
167
|
+
|
|
168
|
+
```
|
|
169
|
+
api/<pageLocation|'system'>/<apiName>/v<n>
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
`pageLocation` is the slash-joined path segments BEFORE the `_api` marker. `system` is used when the API sits at the project root (e.g. `src/_api/healthCheck_v1.ts` -> `api/system/healthCheck/v1`).
|
|
173
|
+
|
|
174
|
+
Sync key:
|
|
175
|
+
|
|
176
|
+
```
|
|
177
|
+
sync/<pageLocation>/<syncName>/v<n>_<server|client>
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
If the sync lives at the project root (no page above the `_sync` marker), the key drops the page segment: `sync/<syncName>/v<n>_<server|client>`.
|
|
181
|
+
|
|
182
|
+
These shapes match the route keys produced by the production runtime maps loader in `@luckystack/server`. Switching between dev and prod is a matter of which map the request dispatcher reads from.
|
|
183
|
+
|
|
184
|
+
---
|
|
185
|
+
|
|
186
|
+
## Filename validation regexes
|
|
187
|
+
|
|
188
|
+
```typescript
|
|
189
|
+
export const API_VERSION_TOKEN_REGEX = /_v(\d+)$/;
|
|
190
|
+
export const SYNC_VERSION_TOKEN_REGEX = /_(server|client)_v(\d+)$/;
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Re-exported from `routeConventions.ts` for consumers (e.g. the docs UI) that need to parse filenames.
|
|
194
|
+
|
|
195
|
+
`assertValidRouteNaming({ srcDir, context })` walks the tree and throws an `Error` whose message includes:
|
|
196
|
+
|
|
197
|
+
- The `context` string passed in (e.g. `'starting dev server (npm run server)'` or `'generating API/sync type maps'`).
|
|
198
|
+
- Every offending file path with the expected pattern.
|
|
199
|
+
|
|
200
|
+
`assertNoDuplicateNormalizedRouteKeys({ srcDir, context })` is called only by the type-map generator. It catches the case where two differently-cased files (`getSettings_v1.ts` vs `getsettings_v1.ts`) normalize to the same route key on case-insensitive filesystems.
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## Routing rules registry
|
|
205
|
+
|
|
206
|
+
```typescript
|
|
207
|
+
export interface RoutingRules {
|
|
208
|
+
apiMarker: string; // default '_api'
|
|
209
|
+
syncMarker: string; // default '_sync'
|
|
210
|
+
apiVersionRegex: RegExp; // default API_VERSION_TOKEN_REGEX
|
|
211
|
+
syncVersionRegex: RegExp;// default SYNC_VERSION_TOKEN_REGEX
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
export const registerRoutingRules = (rules: Partial<RoutingRules>): void;
|
|
215
|
+
export const getRoutingRules = (): RoutingRules;
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
Defaults are exported from `routingRules.ts` and consumed by every walker (`scanApiFolder`, `scanSyncFolder`, `resolveApiRouteMetaFromPath`, `resolveSyncRouteMetaFromPath`, the watcher) plus the filename predicates:
|
|
219
|
+
|
|
220
|
+
```typescript
|
|
221
|
+
export const apiMarkerSegment = (): string;
|
|
222
|
+
export const syncMarkerSegment = (): string;
|
|
223
|
+
export const isApiFileName = (name: string): boolean;
|
|
224
|
+
export const isSyncFileName = (name: string): boolean;
|
|
225
|
+
export const isSyncServerFileName = (name: string): boolean;
|
|
226
|
+
export const isSyncClientFileName = (name: string): boolean;
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
A consumer registering custom markers (`{ apiMarker: 'api', syncMarker: 'live' }`) sees them flow through to the watcher segment computation in `setupWatchers()` (see `hot-reload.md`) and the loader scans, so non-default layouts work without forking the package.
|
|
230
|
+
|
|
231
|
+
---
|
|
232
|
+
|
|
233
|
+
## Hot-reload single-file paths
|
|
234
|
+
|
|
235
|
+
All four hot-reload functions follow the same shape:
|
|
236
|
+
|
|
237
|
+
```typescript
|
|
238
|
+
export const upsertApiFromFile = async (filePath: string): Promise<void> => {
|
|
239
|
+
const routeMeta = resolveApiRouteMetaFromPath(filePath);
|
|
240
|
+
if (!routeMeta) {
|
|
241
|
+
// log + return — filename rejection
|
|
242
|
+
return;
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
invalidateProgramCache();
|
|
246
|
+
clearRuntimeTypeResolverCache();
|
|
247
|
+
|
|
248
|
+
const [err, module] = await tryCatch(async () => importFile(routeMeta.absolutePath));
|
|
249
|
+
if (err) { /* log + return */ return; }
|
|
250
|
+
|
|
251
|
+
const resolvedModule = module?.default ? { ...module.default, ...module } : module;
|
|
252
|
+
const { auth = {}, main, rateLimit, httpMethod, schema } = resolvedModule;
|
|
253
|
+
|
|
254
|
+
if (!main || typeof main !== 'function') {
|
|
255
|
+
delete devApis[routeMeta.routeKey];
|
|
256
|
+
return;
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
const inputType = getInputTypeFromFile(routeMeta.absolutePath);
|
|
260
|
+
|
|
261
|
+
devApis[routeMeta.routeKey] = { main, auth: { ... }, rateLimit, httpMethod, schema, inputType, inputTypeFilePath: routeMeta.absolutePath };
|
|
262
|
+
};
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
Key differences from the boot initializers:
|
|
266
|
+
|
|
267
|
+
| Step | Boot (`initializeApis`/`initializeSyncs`) | Hot reload (`upsert*`/`remove*`) |
|
|
268
|
+
|---|---|---|
|
|
269
|
+
| `invalidateProgramCache()` | Skipped — cache is null on module load | Always called — file content changed |
|
|
270
|
+
| `clearRuntimeTypeResolverCache()` | Called | Called |
|
|
271
|
+
| Resolve filename via marker walk | Recursive tree scan | `resolveApiRouteMetaFromPath` / `resolveSyncRouteMetaFromPath` on the single path |
|
|
272
|
+
| Missing `main` | Silently skip | Explicit `delete devApis[key]` to keep table in sync with file state |
|
|
273
|
+
|
|
274
|
+
`removeApiFromFile` / `removeSyncFromFile` follow the same skeleton minus the import: they invalidate caches and `delete` the matching entry. If the file path doesn't normalize to a route key (wrong marker segment, wrong extension), the call is a no-op — the watcher fan-out from `enqueueAffectedRoutesFromDependency` may enqueue paths that aren't routes, and silently dropping them is the right behavior.
|
|
275
|
+
|
|
276
|
+
Sync kind handling in `upsertSyncFromFile`:
|
|
277
|
+
|
|
278
|
+
- `kind === 'server'`: store `{ main, auth, inputType, inputTypeFilePath }` if `main` is a function, otherwise delete the entry.
|
|
279
|
+
- `kind === 'client'`: store the bare `main` callback if it's a function, otherwise delete the entry.
|
|
280
|
+
|
|
281
|
+
---
|
|
282
|
+
|
|
283
|
+
## Consumer contract
|
|
284
|
+
|
|
285
|
+
Dev server boot:
|
|
286
|
+
|
|
287
|
+
```typescript
|
|
288
|
+
import { initializeAll, setupWatchers } from '@luckystack/devkit';
|
|
289
|
+
|
|
290
|
+
await initializeAll();
|
|
291
|
+
setupWatchers();
|
|
292
|
+
// then start HTTP server / Socket.io
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
Production runtime maps loader (`packages/server/src/runtimeMapsLoader.ts`):
|
|
296
|
+
|
|
297
|
+
```typescript
|
|
298
|
+
if (process.env.NODE_ENV !== 'production') {
|
|
299
|
+
const devkit = await import('@luckystack/devkit');
|
|
300
|
+
return {
|
|
301
|
+
apis: devkit.devApis,
|
|
302
|
+
syncs: devkit.devSyncs,
|
|
303
|
+
functions: devkit.devFunctions,
|
|
304
|
+
};
|
|
305
|
+
}
|
|
306
|
+
|
|
307
|
+
// prod: read from generated maps on disk
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
The dev loader is never invoked under `NODE_ENV=production`; the generated `apiTypes.generated.ts` / `apiInputSchemas.generated.ts` (see `type-map-generation.md`) drive the prod path instead.
|
|
311
|
+
|
|
312
|
+
---
|
|
313
|
+
|
|
314
|
+
## Failure modes (summary)
|
|
315
|
+
|
|
316
|
+
| Symptom | Cause | What happens |
|
|
317
|
+
|---|---|---|
|
|
318
|
+
| `[loader][api] invalid filename: ...` | filename doesn't match `apiVersionRegex` | file skipped, route key absent from `devApis` |
|
|
319
|
+
| `[loader][sync] invalid filename: ...` | filename doesn't match `syncVersionRegex` | file skipped, route key absent from `devSyncs` |
|
|
320
|
+
| `[loader][api] failed to import ...` | runtime error during `import()` | previous entry left untouched, error logged |
|
|
321
|
+
| `[loader][function] failed to import ...` | runtime error in a server-functions file | previous nested node left untouched |
|
|
322
|
+
| `assertValidRouteNaming` throw at startup | any `_api/`/`_sync/` file fails the regex | dev server boot aborts before any file evaluates |
|
|
323
|
+
| Missing `main` after a save | consumer removed the export | `delete devApis[key]` (or `devSyncs[key]`) keeps the table consistent with disk state |
|
|
324
|
+
| Hot reload picks up a typo, then the next save fixes it | `upsertApiFromFile` runs twice | second run replaces the entry; no orphans |
|
|
@@ -0,0 +1,258 @@
|
|
|
1
|
+
# Runtime Type Resolver (`runtimeTypeResolver.ts`)
|
|
2
|
+
|
|
3
|
+
> Dev-only. Lives in `@luckystack/devkit` and is consumed exclusively by `@luckystack/core/src/runtimeTypeValidation.ts` via a lazy `await import('@luckystack/devkit')` that fires only when `NODE_ENV !== 'production'`. Production servers never load this module — runtime input validation in prod uses the pre-generated Zod schemas in `apiInputSchemas.generated.ts`, not the deep resolver.
|
|
4
|
+
|
|
5
|
+
The resolver bridges two worlds:
|
|
6
|
+
|
|
7
|
+
1. **What the loader stored.** When the dev loader (`loader.ts`) imports an API or sync file, it calls `getInputTypeFromFile(filePath)` (a public extractor) and stashes the result on `devApis[routeKey].inputType` along with `inputTypeFilePath`. The stored text is what the extractor produced at import time — usually a fully-expanded inline form, occasionally a named alias that the extractor could not expand standalone.
|
|
8
|
+
2. **What the runtime validator needs.** Each incoming request is type-checked against the stored `inputType`. If the stored text still contains identifiers (e.g. `UserInput`, `Partial<RegisterPayload>`, `Pick<Prisma.User, 'id' | 'name'>`), the validator cannot match payload keys to type members until those identifiers are recursively expanded into a structural shape.
|
|
9
|
+
|
|
10
|
+
The resolver takes the stored text + the original file path, follows imports across the project via the cached `ts.Program`, applies a handful of TypeScript utility types (`Partial`, `Required`, `Pick`, `Omit`, `Record`, `Array`), and returns a self-contained inline type string. The result is cached so a hot request path does not redo work.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Module surface
|
|
15
|
+
|
|
16
|
+
```typescript
|
|
17
|
+
type ResolveResult =
|
|
18
|
+
| { status: 'success'; typeText: string }
|
|
19
|
+
| { status: 'error'; message: string };
|
|
20
|
+
|
|
21
|
+
export const resolveRuntimeTypeText = ({
|
|
22
|
+
typeText,
|
|
23
|
+
filePath,
|
|
24
|
+
}: {
|
|
25
|
+
typeText: string;
|
|
26
|
+
filePath?: string;
|
|
27
|
+
}): ResolveResult;
|
|
28
|
+
|
|
29
|
+
export const clearRuntimeTypeResolverCache = (): void;
|
|
30
|
+
export const isUnresolvedTypeMarker = (value: string): boolean;
|
|
31
|
+
export const getUnresolvedTypeMessage = (value: string): string;
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Internal-only:
|
|
35
|
+
|
|
36
|
+
- `resolveExpression(typeText, filePath, depth, state)` — recursive worker.
|
|
37
|
+
- `resolveIdentifier(identifier, filePath)` — TypeChecker-backed identifier lookup, follows local declarations + import statements.
|
|
38
|
+
- `applyUtilityType({ utilityName, utilityArgs, filePath, depth, state })` — `Partial`, `Required`, `Pick`, `Omit`, `Record`.
|
|
39
|
+
- `parseObjectFields(typeText)` / `serializeObjectFields({ fields, indexSignatures })` — round-trip an object literal type through a typed AST.
|
|
40
|
+
- `splitTopLevel(value, '|' | '&' | ',')` — depth-aware splitter (respects `(`, `{`, `[`, `<`).
|
|
41
|
+
- `parseLiteralUnionKeys(value)` — extract `'a' | 'b'` into `['a', 'b']` for `Pick` / `Omit` / `Record` key lists.
|
|
42
|
+
- Constants: `MAX_DEPTH = 20`, `PRIMITIVE_TYPES`, `SKIP_EXPANSION`, `unresolvedPrefix = '__RUNTIME_UNRESOLVED__::'`.
|
|
43
|
+
- Types: `ObjectField`, `ObjectIndexSignature`, `ResolveState`.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## `resolveRuntimeTypeText(...)` flow
|
|
48
|
+
|
|
49
|
+
```typescript
|
|
50
|
+
export const resolveRuntimeTypeText = ({ typeText, filePath }) => {
|
|
51
|
+
const cleanType = typeText.trim();
|
|
52
|
+
if (!cleanType || !filePath) {
|
|
53
|
+
return { status: 'success', typeText: cleanType };
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
const cacheKey = `${filePath}::${cleanType}`;
|
|
57
|
+
const cached = resolvedTypeCache.get(cacheKey);
|
|
58
|
+
if (cached) {
|
|
59
|
+
return cached;
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
const resolved = resolveExpression(cleanType, filePath, 0, { stack: new Set() });
|
|
63
|
+
const result: ResolveResult = isUnresolvedTypeMarker(resolved)
|
|
64
|
+
? { status: 'error', message: getUnresolvedTypeMessage(resolved) }
|
|
65
|
+
: { status: 'success', typeText: resolved };
|
|
66
|
+
|
|
67
|
+
resolvedTypeCache.set(cacheKey, result);
|
|
68
|
+
return result;
|
|
69
|
+
};
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
1. **No-op short-circuit.** Empty input or missing `filePath` — return as-is. The validator passes an empty string for routes with `{ [key: string]: never }` "no input" syntax that the extractor reduced to nothing; callers expect a success in that case.
|
|
73
|
+
2. **Cache hit.** `resolvedTypeCache: Map<string, ResolveResult>` keyed by `${filePath}::${typeText}`. The path is part of the key because `import { Foo } from './a'` and `import { Foo } from './b'` resolve to different `Foo`s.
|
|
74
|
+
3. **Recursive resolve.** `resolveExpression(...)` walks the type expression text and returns either a fully expanded form or a string prefixed with `__RUNTIME_UNRESOLVED__::<reason>`.
|
|
75
|
+
4. **Result shape.** Unresolved-marker prefix is mapped to `{ status: 'error', message }`; clean text is mapped to `{ status: 'success', typeText }`. Both branches are cached.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## `resolveExpression(...)` — the recursive worker
|
|
80
|
+
|
|
81
|
+
The worker tracks two pieces of state:
|
|
82
|
+
|
|
83
|
+
- `state.stack: Set<string>` — visited `(filePath, typeText)` pairs in the current resolution. Re-entering one returns `__RUNTIME_UNRESOLVED__::cyclic type reference <type>`.
|
|
84
|
+
- `depth: number` — bumped on every recursion. When it exceeds `MAX_DEPTH = 20`, the worker returns `__RUNTIME_UNRESOLVED__::resolution depth exceeded for <type>`.
|
|
85
|
+
|
|
86
|
+
The decision tree (in order):
|
|
87
|
+
|
|
88
|
+
1. **Already unresolved.** If `type.startsWith('__RUNTIME_UNRESOLVED__::')`, return it as-is.
|
|
89
|
+
2. **Parenthesized group.** `(T)` — strip outer parens, recurse, re-wrap on success.
|
|
90
|
+
3. **Top-level union.** `splitTopLevel(type, '|')` returns more than one part — recurse on each, propagate the first unresolved marker found, otherwise join with ` | `.
|
|
91
|
+
4. **Top-level intersection.** Same pattern with `&`.
|
|
92
|
+
5. **Array suffix.** `T[]` — recurse on `T`, wrap result with `[]`.
|
|
93
|
+
6. **Object literal.** Starts with `{` and ends with `}` — parse via `parseObjectFields`, recurse on every field type and every index signature key/value type, serialize back via `serializeObjectFields`.
|
|
94
|
+
7. **Generic application.** Matches `^([A-Za-z_][A-Za-z0-9_]*)<(.+)>$` — split args at top-level commas, then:
|
|
95
|
+
- `Array<T>` (single arg) — same as `T[]`.
|
|
96
|
+
- `Partial`, `Required`, `Pick`, `Omit`, `Record` — delegate to `applyUtilityType`.
|
|
97
|
+
- Anything else — recurse on every type argument and re-render as `Name<arg1, arg2>`. The generic name is preserved verbatim; the validator can decide what to do with it.
|
|
98
|
+
8. **Bare identifier.** Matches `^[A-Za-z_][A-Za-z0-9_]*$` — delegate to `resolveIdentifier`.
|
|
99
|
+
9. **Everything else.** Returned as-is. This covers literal types (`'foo'`, `42`, `true`), already-primitive forms, and anything the regex set above does not classify.
|
|
100
|
+
|
|
101
|
+
`splitTopLevel(value, splitter)` is the depth-aware splitter used by steps 3, 4, 7. It tracks `(`, `{`, `[`, `<` depth counters and only splits when all four are zero. This is what lets the worker safely split `Pick<User, 'a' | 'b'>` on commas without trimming the inner union.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## `resolveIdentifier(identifier, filePath)`
|
|
106
|
+
|
|
107
|
+
```typescript
|
|
108
|
+
const resolveIdentifier = (identifier: string, filePath: string): string => {
|
|
109
|
+
if (PRIMITIVE_TYPES.has(identifier.toLowerCase())) return identifier.toLowerCase();
|
|
110
|
+
if (SKIP_EXPANSION.has(identifier)) return identifier;
|
|
111
|
+
|
|
112
|
+
const program = getServerProgram();
|
|
113
|
+
const sourceFile = program.getSourceFile(filePath);
|
|
114
|
+
if (!sourceFile) return toUnresolved(`unresolved type ${identifier}`);
|
|
115
|
+
|
|
116
|
+
const checker = program.getTypeChecker();
|
|
117
|
+
|
|
118
|
+
for (const stmt of sourceFile.statements) {
|
|
119
|
+
// 1. Local interface / type alias / enum
|
|
120
|
+
if (
|
|
121
|
+
(ts.isInterfaceDeclaration(stmt) || ts.isTypeAliasDeclaration(stmt) || ts.isEnumDeclaration(stmt))
|
|
122
|
+
&& stmt.name.text === identifier
|
|
123
|
+
) {
|
|
124
|
+
const symbol = checker.getSymbolAtLocation(stmt.name);
|
|
125
|
+
if (symbol) {
|
|
126
|
+
return expandType(checker.getDeclaredTypeOfSymbol(symbol), checker);
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
// 2. Import declarations — follow aliases
|
|
131
|
+
if (ts.isImportDeclaration(stmt) && stmt.importClause) {
|
|
132
|
+
// namedBindings: { Foo, Bar } — match Foo or Bar
|
|
133
|
+
// defaultName: Foo from '...'
|
|
134
|
+
// For matches: getSymbolAtLocation -> getAliasedSymbol -> getDeclaredTypeOfSymbol -> expandType
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
return toUnresolved(`unresolved type ${identifier}`);
|
|
139
|
+
};
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Two short-circuit lists:
|
|
143
|
+
|
|
144
|
+
- `PRIMITIVE_TYPES` — `string`, `number`, `boolean`, `true`, `false`, `null`, `undefined`, `any`, `unknown`, `void`, `never`. Returned lowercased; covers the case where the extractor emitted `String` instead of `string`.
|
|
145
|
+
- `SKIP_EXPANSION` — `Date`, `Promise`, `Array`, `Record`, `Partial`, `Required`, `Pick`, `Omit`, `Function`, `Map`, `Set`, `Buffer`, `Uint8Array`, `Object`, `WeakMap`, `WeakSet`. Structurally opaque; the validator treats them as "any value of this name".
|
|
146
|
+
|
|
147
|
+
For everything else, the worker walks the file's top-level statements and looks for:
|
|
148
|
+
|
|
149
|
+
1. A local `interface`, `type`, or `enum` declaration matching the identifier. If found, resolve the symbol, fetch its declared type, and recursively expand via `expandType` from `tsProgram.ts`.
|
|
150
|
+
2. An import declaration whose named or default binding matches the identifier. If found, resolve the symbol, hop through `getAliasedSymbol` to reach the original declaration, then expand.
|
|
151
|
+
|
|
152
|
+
If neither path produces a result, return the unresolved marker. The `try { ... } catch { ... }` wrapper around the body ensures any TypeChecker exception (rare, but possible on partially-typed sources) degrades to `__RUNTIME_UNRESOLVED__::unresolved type <identifier>` rather than crashing the request.
|
|
153
|
+
|
|
154
|
+
`getServerProgram()` is shared with the type-map emitter. The resolver does not call `invalidateProgramCache()` — that's the loader's job on hot-reload. Resolver-side, `clearRuntimeTypeResolverCache()` is the matching reset.
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## `applyUtilityType(...)`
|
|
159
|
+
|
|
160
|
+
Handles five utility types. Each requires its target type to resolve to an object literal first; if the target resolves to a non-object form, the utility returns an unresolved marker.
|
|
161
|
+
|
|
162
|
+
- **`Partial<T>`** — resolve `T` to an object, mark every field optional via `{ ...field, optional: true }`, preserve index signatures. Pure-additive utility, no field filtering.
|
|
163
|
+
- **`Required<T>`** — symmetric; mark every field non-optional.
|
|
164
|
+
- **`Pick<T, K>`** — resolve `T` to an object, parse `K` via `parseLiteralUnionKeys` into a string set, drop every field whose key is not in the set. Preserves index signatures.
|
|
165
|
+
- **`Omit<T, K>`** — same as `Pick` with inverted membership test.
|
|
166
|
+
- **`Record<K, V>`** — resolve `K` and `V` first. If `K` resolves to a literal-string union, materialize each literal as a concrete field with type `V` (renders as a closed object). Otherwise leave as `Record<K, V>` (treated as an opaque container at runtime).
|
|
167
|
+
|
|
168
|
+
If any argument to a utility resolves to an unresolved marker, that marker is propagated unchanged.
|
|
169
|
+
|
|
170
|
+
If an argument list does not have the expected arity (e.g. `Partial<A, B>`), the utility returns `__RUNTIME_UNRESOLVED__::unresolved utility <Name><...>`.
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
174
|
+
## Object literal round-trip
|
|
175
|
+
|
|
176
|
+
`parseObjectFields(typeText)` and `serializeObjectFields({ fields, indexSignatures })` are the small AST-like helpers that let the worker rewrite an object body.
|
|
177
|
+
|
|
178
|
+
`parseObjectFields(typeText)`:
|
|
179
|
+
|
|
180
|
+
- Strips outer braces.
|
|
181
|
+
- Walks the inner text character by character, tracking nesting depth for `{`, `[`, `(`, `<`.
|
|
182
|
+
- Splits on `;` at depth 0.
|
|
183
|
+
- For each segment, runs two regexes:
|
|
184
|
+
- Field regex: `^("']?[A-Za-z_][A-Za-z0-9_]*["']?)(\?)?\s*:\s*([\s\S]+)$` — captures key, optional flag, and type. Strips surrounding quotes from the key.
|
|
185
|
+
- Index signature regex: `^\[\s*([A-Za-z_][A-Za-z0-9_]*)\s*:\s*([^\]]+)\]\s*:\s*([\s\S]+)$` — captures the key name (e.g. `key`), the key type (e.g. `string`), and the value type.
|
|
186
|
+
|
|
187
|
+
`serializeObjectFields`:
|
|
188
|
+
|
|
189
|
+
- Renders each field as `${name}${optional ? '?' : ''}: ${type}` and each index signature as `[${keyName}: ${keyType}]: ${type}`.
|
|
190
|
+
- Joins with `; ` and wraps with `{ ... }`. Returns `{ }` for an empty body (matches the extractor's "no input" form).
|
|
191
|
+
|
|
192
|
+
---
|
|
193
|
+
|
|
194
|
+
## Unresolved-marker protocol
|
|
195
|
+
|
|
196
|
+
```typescript
|
|
197
|
+
const unresolvedPrefix = '__RUNTIME_UNRESOLVED__::';
|
|
198
|
+
|
|
199
|
+
const toUnresolved = (message: string): string => `${unresolvedPrefix}${message}`;
|
|
200
|
+
export const isUnresolvedTypeMarker = (value: string): boolean => value.startsWith(unresolvedPrefix);
|
|
201
|
+
export const getUnresolvedTypeMessage = (value: string): string => value.slice(unresolvedPrefix.length).trim();
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
The worker uses marker strings — not exceptions — to propagate failure. The reason:
|
|
205
|
+
|
|
206
|
+
- One unresolved leaf inside a deep object should not abort the whole resolve; the validator wants to know which symbol failed.
|
|
207
|
+
- Throw-based propagation forces every call site to wrap in try/catch; marker-based propagation lets the worker fall through identical code paths for success and failure (the worker only needs to check the result of each recursion).
|
|
208
|
+
|
|
209
|
+
Callers (`@luckystack/core/src/runtimeTypeValidation.ts`) inspect `result.status`. On `'error'`, they log the `message` and degrade to a soft-pass for the request rather than refusing it — the deep resolver is informational, not a hard gate.
|
|
210
|
+
|
|
211
|
+
---
|
|
212
|
+
|
|
213
|
+
## Cache lifecycle
|
|
214
|
+
|
|
215
|
+
```typescript
|
|
216
|
+
const resolvedTypeCache = new Map<string, ResolveResult>();
|
|
217
|
+
|
|
218
|
+
export const clearRuntimeTypeResolverCache = () => {
|
|
219
|
+
resolvedTypeCache.clear();
|
|
220
|
+
};
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
The cache is keyed by `${filePath}::${typeText}`. It survives across requests but must be cleared whenever the underlying source files change.
|
|
224
|
+
|
|
225
|
+
Invalidation points (all in `loader.ts`):
|
|
226
|
+
|
|
227
|
+
- `initializeApis()` and `initializeSyncs()` — `clearRuntimeTypeResolverCache()` is called before re-scanning the source tree.
|
|
228
|
+
- `upsertApiFromFile(filePath)`, `removeApiFromFile(filePath)`, `upsertSyncFromFile(filePath)`, `removeSyncFromFile(filePath)` — every per-file mutation clears the whole cache (not just the entry for the changed file). Selective invalidation would require tracking transitive imports; whole-cache flush is the simpler, safer choice and the resolver is not on the hot path of a steady-state running app.
|
|
229
|
+
|
|
230
|
+
The cache must always be flushed alongside `invalidateProgramCache()` from `tsProgram.ts`. The two are paired in every invalidation site.
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
234
|
+
## Consumer contract
|
|
235
|
+
|
|
236
|
+
- **Only `@luckystack/core` may import this module.** The lazy import lives in `@luckystack/core/src/runtimeTypeValidation.ts` and fires only when `process.env.NODE_ENV !== 'production'`. Consumers must not introduce any additional callers.
|
|
237
|
+
- **Do not call from production code paths.** The resolver depends on `getServerProgram()`, which needs `tsconfig.server.json` and the consumer's source tree on disk. Production tarballs have neither.
|
|
238
|
+
- **Treat results as ephemeral.** A cached success may turn into an error after the next hot reload (or vice versa). Always re-call `resolveRuntimeTypeText({ typeText, filePath })` rather than memoizing on the consumer side.
|
|
239
|
+
- **Markers are strings, not exceptions.** Callers must check `status` before assuming success.
|
|
240
|
+
|
|
241
|
+
---
|
|
242
|
+
|
|
243
|
+
## Failure modes
|
|
244
|
+
|
|
245
|
+
- **Cycle detected.** Returns `{ status: 'error', message: 'cyclic type reference <type>' }`. Happens for self-referential aliases that the type-map emitter could not flatten (e.g. recursive tree types whose Prisma client form was elided).
|
|
246
|
+
- **Depth exceeded.** Returns `{ status: 'error', message: 'resolution depth exceeded for <type>' }`. `MAX_DEPTH = 20` is well above the depth limit in `tsProgram.ts` (`DEPTH_LIMIT = 12`), so this only fires when a type chains many import hops in addition to deep nesting.
|
|
247
|
+
- **Identifier not found.** Returns `{ status: 'error', message: 'unresolved type <name>' }`. The file in question may not exist in the cached `ts.Program` (recently deleted, or outside the `tsconfig.server.json` `include` glob).
|
|
248
|
+
- **Utility-arity mismatch.** Returns `{ status: 'error', message: 'unresolved utility <Name><...>' }`. The validator falls back to a soft-pass.
|
|
249
|
+
- **Program rebuild during a resolve.** A concurrent hot-reload event could call `invalidateProgramCache()` mid-resolution. The next `getServerProgram()` call inside `resolveIdentifier` rebuilds; results from before and after the rebuild may differ. This is a tolerated race — the next steady-state request will see consistent state.
|
|
250
|
+
|
|
251
|
+
---
|
|
252
|
+
|
|
253
|
+
## Constants
|
|
254
|
+
|
|
255
|
+
- `MAX_DEPTH = 20` — recursion cap for `resolveExpression`. Distinct from `DEPTH_LIMIT = 12` in `tsProgram.ts`; the resolver allows deeper recursion because each step crosses at most one file boundary.
|
|
256
|
+
- `PRIMITIVE_TYPES` — 11 entries, lowercased on return.
|
|
257
|
+
- `SKIP_EXPANSION` — 17 entries; passes through opaque containers and utility names that are themselves the subject of recursion elsewhere.
|
|
258
|
+
- `unresolvedPrefix = '__RUNTIME_UNRESOLVED__::'` — sentinel string used by `toUnresolved` / `isUnresolvedTypeMarker` / `getUnresolvedTypeMessage`.
|