tina4-nodejs 3.13.24 → 3.13.26
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +19 -14
- package/package.json +1 -1
- package/packages/core/src/cache.ts +75 -33
- package/packages/core/src/middleware.ts +15 -6
- package/packages/core/src/server.ts +2 -2
- package/packages/orm/src/cachedDatabase.ts +157 -33
- package/packages/orm/src/database.ts +25 -6
package/CLAUDE.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
# CLAUDE.md — AI Developer Guide for tina4-nodejs (v3.13.
|
|
1
|
+
# CLAUDE.md — AI Developer Guide for tina4-nodejs (v3.13.26)
|
|
2
2
|
|
|
3
3
|
> This file helps AI assistants (Claude, Copilot, Cursor, etc.) understand and work on this codebase effectively.
|
|
4
4
|
|
|
5
5
|
## What This Project Is
|
|
6
6
|
|
|
7
|
-
Tina4 for Node.js/TypeScript v3.13.
|
|
7
|
+
Tina4 for Node.js/TypeScript v3.13.26 — The Intelligent Native Application 4ramework. A convention-over-configuration structural paradigm. The developer writes TypeScript; Tina4 is invisible infrastructure.
|
|
8
8
|
|
|
9
9
|
The philosophy: zero ceremony, batteries included, file system as source of truth.
|
|
10
10
|
|
|
@@ -575,7 +575,10 @@ db.delete(table, filter?, params?): DatabaseWriteResult
|
|
|
575
575
|
db.getLastId(): string | number | null
|
|
576
576
|
db.getError(): string | null
|
|
577
577
|
|
|
578
|
-
// Transactions — autoCommit defaults to
|
|
578
|
+
// Transactions — autoCommit defaults to ON: a standalone write commits on its
|
|
579
|
+
// own connection (durable + visible across the pool); inside startTransaction()
|
|
580
|
+
// the per-statement commit is suppressed so the transaction stays atomic. Set
|
|
581
|
+
// TINA4_AUTOCOMMIT=false for strict manual-commit mode.
|
|
579
582
|
db.startTransaction(): void
|
|
580
583
|
db.commit(): void
|
|
581
584
|
db.rollback(): void
|
|
@@ -591,13 +594,15 @@ db.getNextId(table, pkColumn?, generatorName?): number
|
|
|
591
594
|
|
|
592
595
|
// DB query cache — request-scoped auto cache is ON by default (TINA4_AUTO_CACHING=true,
|
|
593
596
|
// TTL TINA4_AUTO_CACHING_TTL=5s): dedupes identical db.fetch()/ORM reads within a request,
|
|
594
|
-
// flushed on any write. Persistent cross-request cache opt-in
|
|
595
|
-
// (TTL TINA4_DB_CACHE_TTL=30s), configured via TINA4_DB_CACHE_BACKEND
|
|
596
|
-
//
|
|
597
|
-
//
|
|
598
|
-
//
|
|
599
|
-
//
|
|
600
|
-
|
|
597
|
+
// flushed on any write (always in-process, fastest). Persistent cross-request cache opt-in
|
|
598
|
+
// via TINA4_DB_CACHE=true (TTL TINA4_DB_CACHE_TTL=30s), configured via TINA4_DB_CACHE_BACKEND
|
|
599
|
+
// + TINA4_DB_CACHE_URL. The persistent layer routes through the SAME unified async backend
|
|
600
|
+
// set (memory default = in-process; redis/valkey/memcached/mongodb/database distribute), so
|
|
601
|
+
// multiple instances share one cache with global write-invalidation — full parity with
|
|
602
|
+
// Python/PHP/Ruby. (Node's read path — db.fetch → fetchAsync — is async, so the backend's
|
|
603
|
+
// async get/set work directly; the KV/middleware API is async, await it.) cacheStats() reports
|
|
604
|
+
// mode + backend; cacheClear() is real.
|
|
605
|
+
db.cacheStats(): { enabled, size, ttl, mode, backend } // mode: "request" | "persistent" | "off"
|
|
601
606
|
db.cacheClear(): void
|
|
602
607
|
|
|
603
608
|
// Connection pool access (null when pooling disabled)
|
|
@@ -906,7 +911,7 @@ await cacheClear();
|
|
|
906
911
|
await cacheStats(); // { hits, misses, size, backend } — reflects the real KV backend
|
|
907
912
|
```
|
|
908
913
|
|
|
909
|
-
**
|
|
914
|
+
**Async everywhere (full parity):** the KV API, the `responseCache` middleware, and the persistent DB query cache all route through the same unified async backend set with native async clients (no child processes). The `responseCache` middleware and persistent DB cache distribute cross-instance when a network backend (redis/valkey/memcached/mongodb/database) is selected — exactly like Python/PHP/Ruby. The default `memory` backend keeps both in-process (fastest), so default behaviour is unchanged. Because the KV/middleware API is async, **`await` it** (`await cacheGet`/`await cacheSet`/`await clearCache`; the middleware runner and `db.fetch` are async). Request-scoped DB caching (`TINA4_AUTO_CACHING`) always stays in-process.
|
|
910
915
|
|
|
911
916
|
**Graceful fallback**: if a configured backend's driver is missing or the service/credentials are unreachable or wrong, the cache logs a warning and falls back to the **file** backend — a real persistent cache, never a silent no-op.
|
|
912
917
|
|
|
@@ -1112,13 +1117,13 @@ When adding new features, add a corresponding `test/<feature>.test.ts` file.
|
|
|
1112
1117
|
## v3 Features Summary
|
|
1113
1118
|
|
|
1114
1119
|
- **45 built-in features**, zero third-party dependencies
|
|
1115
|
-
- **3,
|
|
1120
|
+
- **3,748 tests** passing across all modules
|
|
1116
1121
|
- **Race-safe `getNextId()`** with atomic sequence table (`tina4_sequences`) for SQLite/MySQL/MSSQL; PostgreSQL auto-creates sequences
|
|
1117
1122
|
- **Frond template engine optimizations**: pre-compiled regexes, lazy loop context (copy-on-write), filter chain caching, path split caching, inline common filters (11-15% speedup)
|
|
1118
1123
|
- **Production server auto-detect**: `npx tina4nodejs serve --production` auto-uses cluster mode
|
|
1119
1124
|
- **`npx tina4nodejs generate`**: model, route, migration, middleware scaffolding
|
|
1120
|
-
- **Database**: 5 engines (SQLite, PostgreSQL, MySQL, MSSQL, Firebird), DB query caching — request-scoped auto cache **on by default** (`TINA4_AUTO_CACHING=true`, TTL `TINA4_AUTO_CACHING_TTL=5`s) dedupes identical `db.fetch()`/ORM reads within a request and flushes on writes; persistent cross-request cache opt-in via `TINA4_DB_CACHE=true` (TTL `TINA4_DB_CACHE_TTL=30`s)
|
|
1121
|
-
- **Cache**: unified backend set — `memory` (default), `file`, `redis`, `valkey`, `memcached`, `mongodb`, `database` — via `TINA4_CACHE_BACKEND` (+ `TINA4_CACHE_URL`/credentials); file-backend fallback if a backend is unreachable. KV API
|
|
1125
|
+
- **Database**: 5 engines (SQLite, PostgreSQL, MySQL, MSSQL, Firebird), DB query caching — request-scoped auto cache **on by default** (`TINA4_AUTO_CACHING=true`, TTL `TINA4_AUTO_CACHING_TTL=5`s) dedupes identical `db.fetch()`/ORM reads within a request and flushes on writes (always in-process); persistent cross-request cache opt-in via `TINA4_DB_CACHE=true` (TTL `TINA4_DB_CACHE_TTL=30`s) routed through the unified async backend set via `TINA4_DB_CACHE_BACKEND` (memory/file/redis/valkey/memcached/mongodb/database) + `TINA4_DB_CACHE_URL`, so instances share one cache with global write-invalidation (full parity with Python/PHP/Ruby). `db.cacheStats()` reports `mode` (request/persistent/off) + `backend`
|
|
1126
|
+
- **Cache**: unified backend set — `memory` (default), `file`, `redis`, `valkey`, `memcached`, `mongodb`, `database` — via `TINA4_CACHE_BACKEND` (+ `TINA4_CACHE_URL`/credentials); file-backend fallback if a backend is unreachable. The KV API, the `responseCache` middleware, and the persistent DB query cache all route through this async backend set (native async clients, no child processes) — a network backend distributes them cross-instance, full parity with Python/PHP/Ruby; `memory` (default) keeps them in-process. `await` the async API (`cacheGet`/`cacheSet`/`clearCache`)
|
|
1122
1127
|
- **Sessions**: file backend (default). `TINA4_SESSION_SAMESITE` env var (default: Lax)
|
|
1123
1128
|
- **Queue**: file/RabbitMQ/Kafka/MongoDB backends, configured via env vars
|
|
1124
1129
|
- **Cache**: memory/Redis/file backends
|
package/package.json
CHANGED
|
@@ -1199,17 +1199,55 @@ export async function createBackend(config?: {
|
|
|
1199
1199
|
|
|
1200
1200
|
// ── Response cache store (for middleware) ──────────────────────────
|
|
1201
1201
|
|
|
1202
|
-
|
|
1202
|
+
/**
|
|
1203
|
+
* The responseCache middleware's backend. Built lazily (and memoised) from the
|
|
1204
|
+
* SAME unified `createBackend()` factory as the KV API, so cached GET responses
|
|
1205
|
+
* distribute across instances via redis/valkey/memcached/mongodb/database
|
|
1206
|
+
* (parity with Python's ResponseCache, which routes through `_create_backend`).
|
|
1207
|
+
*
|
|
1208
|
+
* The default backend is `memory` (in-process), so the DEFAULT behaviour is
|
|
1209
|
+
* unchanged — only an explicit redis/etc. backend distributes. Each middleware
|
|
1210
|
+
* instance resolves the shared module-level backend; a fresh middleware
|
|
1211
|
+
* instance therefore serves hits stored by an earlier one (cross-instance via a
|
|
1212
|
+
* network backend, same-process via memory).
|
|
1213
|
+
*
|
|
1214
|
+
* Response entries are stored as a plain JSON-serialisable object so every
|
|
1215
|
+
* backend (redis SETEX, mongo doc, etc.) can round-trip them.
|
|
1216
|
+
*/
|
|
1217
|
+
let _responseBackend: CacheBackend | null = null;
|
|
1218
|
+
let _responseBackendPromise: Promise<CacheBackend> | null = null;
|
|
1219
|
+
|
|
1220
|
+
function _getResponseBackend(config?: ResponseCacheConfig): Promise<CacheBackend> {
|
|
1221
|
+
if (_responseBackend) return Promise.resolve(_responseBackend);
|
|
1222
|
+
if (!_responseBackendPromise) {
|
|
1223
|
+
_responseBackendPromise = createBackend({
|
|
1224
|
+
backend: config?.backend,
|
|
1225
|
+
cacheUrl: config?.cacheUrl,
|
|
1226
|
+
cacheDir: config?.cacheDir,
|
|
1227
|
+
maxEntries: config?.maxEntries,
|
|
1228
|
+
}).then((b) => {
|
|
1229
|
+
_responseBackend = b;
|
|
1230
|
+
return b;
|
|
1231
|
+
});
|
|
1232
|
+
}
|
|
1233
|
+
return _responseBackendPromise;
|
|
1234
|
+
}
|
|
1203
1235
|
|
|
1204
1236
|
/**
|
|
1205
1237
|
* Response cache middleware for GET requests.
|
|
1206
|
-
* Caches the full response body, content-type, and status code
|
|
1207
|
-
* Cache key is method + url (including query string).
|
|
1238
|
+
* Caches the full response body, content-type, and status code through the
|
|
1239
|
+
* unified async backend. Cache key is method + url (including query string).
|
|
1240
|
+
*
|
|
1241
|
+
* The middleware is ASYNC: the before-path awaits `backend.get` (serve hit) and
|
|
1242
|
+
* the after-path awaits `backend.set` (store on the captured `res.raw.end`).
|
|
1243
|
+
* The framework's middleware chain (`runRouteMiddlewares` / `MiddlewareChain`)
|
|
1244
|
+
* already awaits middleware, so async is transparent. Honors ttl/statusCodes/
|
|
1245
|
+
* maxEntries. With the default `memory` backend behaviour is unchanged; a
|
|
1246
|
+
* redis/etc. backend distributes cross-instance.
|
|
1208
1247
|
*/
|
|
1209
1248
|
export function responseCache(config?: ResponseCacheConfig): Middleware {
|
|
1210
1249
|
const ttl = config?.ttl
|
|
1211
1250
|
?? (process.env.TINA4_CACHE_TTL ? parseInt(process.env.TINA4_CACHE_TTL, 10) : 60);
|
|
1212
|
-
const maxEntries = config?.maxEntries ?? 1000;
|
|
1213
1251
|
const allowedCodes = new Set(config?.statusCodes ?? [200]);
|
|
1214
1252
|
|
|
1215
1253
|
if (ttl <= 0) {
|
|
@@ -1217,34 +1255,26 @@ export function responseCache(config?: ResponseCacheConfig): Middleware {
|
|
|
1217
1255
|
return (_req, _res, next) => next();
|
|
1218
1256
|
}
|
|
1219
1257
|
|
|
1220
|
-
|
|
1221
|
-
const cleanupTimer = setInterval(() => {
|
|
1222
|
-
const now = Date.now();
|
|
1223
|
-
for (const [key, entry] of store) {
|
|
1224
|
-
if (now > entry.expiresAt) store.delete(key);
|
|
1225
|
-
}
|
|
1226
|
-
}, 30_000);
|
|
1227
|
-
if (cleanupTimer.unref) cleanupTimer.unref();
|
|
1228
|
-
|
|
1229
|
-
return (req, res, next) => {
|
|
1258
|
+
return async (req, res, next) => {
|
|
1230
1259
|
// Only cache GET requests
|
|
1231
1260
|
if (req.method !== "GET") {
|
|
1232
1261
|
next();
|
|
1233
1262
|
return;
|
|
1234
1263
|
}
|
|
1235
1264
|
|
|
1236
|
-
const
|
|
1237
|
-
const
|
|
1265
|
+
const backend = await _getResponseBackend(config);
|
|
1266
|
+
const cacheKey = `response:GET:${req.url}`;
|
|
1267
|
+
const cached = (await backend.get(cacheKey)) as CacheEntry | undefined;
|
|
1238
1268
|
|
|
1239
|
-
if (cached &&
|
|
1240
|
-
// Cache HIT
|
|
1269
|
+
if (cached && typeof cached === "object" && typeof cached.body === "string") {
|
|
1270
|
+
// Cache HIT — serve from the (possibly distributed) backend.
|
|
1241
1271
|
res.header("X-Cache", "HIT");
|
|
1242
1272
|
res.header("Content-Type", cached.contentType);
|
|
1243
1273
|
res(cached.body, cached.statusCode, cached.contentType);
|
|
1244
1274
|
return;
|
|
1245
1275
|
}
|
|
1246
1276
|
|
|
1247
|
-
// Cache MISS — intercept the response to capture it
|
|
1277
|
+
// Cache MISS — intercept the response to capture and store it.
|
|
1248
1278
|
const originalEnd = res.raw.end.bind(res.raw);
|
|
1249
1279
|
let captured = false;
|
|
1250
1280
|
|
|
@@ -1254,18 +1284,16 @@ export function responseCache(config?: ResponseCacheConfig): Middleware {
|
|
|
1254
1284
|
const body = typeof chunk === "string" ? chunk : chunk?.toString() ?? "";
|
|
1255
1285
|
const contentType = String(res.raw.getHeader("Content-Type") ?? "application/octet-stream");
|
|
1256
1286
|
|
|
1257
|
-
//
|
|
1258
|
-
|
|
1259
|
-
|
|
1260
|
-
|
|
1261
|
-
|
|
1262
|
-
|
|
1263
|
-
store.set(cacheKey, {
|
|
1287
|
+
// backend.set is async; the captured end() must stay synchronous (Node
|
|
1288
|
+
// flushes the body here), so fire-and-forget the store. The backend's
|
|
1289
|
+
// TTL + maxEntries handle expiry/eviction. Errors are swallowed so a
|
|
1290
|
+
// cache write can never break the response.
|
|
1291
|
+
void backend.set(cacheKey, {
|
|
1264
1292
|
body,
|
|
1265
1293
|
contentType,
|
|
1266
1294
|
statusCode: res.raw.statusCode,
|
|
1267
1295
|
expiresAt: Date.now() + ttl * 1000,
|
|
1268
|
-
});
|
|
1296
|
+
} as CacheEntry, ttl).catch(() => { /* best effort */ });
|
|
1269
1297
|
}
|
|
1270
1298
|
|
|
1271
1299
|
res.header("X-Cache", "MISS");
|
|
@@ -1276,9 +1304,20 @@ export function responseCache(config?: ResponseCacheConfig): Middleware {
|
|
|
1276
1304
|
};
|
|
1277
1305
|
}
|
|
1278
1306
|
|
|
1279
|
-
/**
|
|
1280
|
-
|
|
1281
|
-
|
|
1307
|
+
/**
|
|
1308
|
+
* Clear all cached responses (the responseCache middleware backend).
|
|
1309
|
+
* ASYNC on Node — callers `await clearCache()` — because the backend may be a
|
|
1310
|
+
* network backend (redis/etc.). Resets the backend's namespace; mirrors the
|
|
1311
|
+
* Python ResponseCache.clear_cache() which clears its backend.
|
|
1312
|
+
*/
|
|
1313
|
+
export async function clearCache(): Promise<void> {
|
|
1314
|
+
if (!_responseBackend && !_responseBackendPromise) {
|
|
1315
|
+
// Nothing built yet — build the default so a clear before first use still
|
|
1316
|
+
// resolves to a real (empty) backend rather than silently no-op'ing.
|
|
1317
|
+
await _getResponseBackend();
|
|
1318
|
+
}
|
|
1319
|
+
const backend = await _getResponseBackend();
|
|
1320
|
+
await backend.clear();
|
|
1282
1321
|
}
|
|
1283
1322
|
|
|
1284
1323
|
/**
|
|
@@ -1358,10 +1397,13 @@ export async function cacheBackendStats(): Promise<{ hits: number; misses: numbe
|
|
|
1358
1397
|
|
|
1359
1398
|
/** Reset the default backend (for testing). Closes any pooled connection. */
|
|
1360
1399
|
export function _resetBackend(): void {
|
|
1361
|
-
const b
|
|
1362
|
-
|
|
1363
|
-
|
|
1400
|
+
for (const b of [_defaultBackend, _responseBackend] as any[]) {
|
|
1401
|
+
if (b && typeof b.client?.close === "function") { try { b.client.close(); } catch { /* noop */ } }
|
|
1402
|
+
if (b && typeof b.client?.close !== "function" && typeof b.db?.close === "function") { /* leave shared ORM db */ }
|
|
1403
|
+
}
|
|
1364
1404
|
_defaultBackend = null;
|
|
1365
1405
|
_defaultBackendPromise = null;
|
|
1406
|
+
_responseBackend = null;
|
|
1407
|
+
_responseBackendPromise = null;
|
|
1366
1408
|
_defaultTtl = null;
|
|
1367
1409
|
}
|
|
@@ -95,18 +95,23 @@ export class MiddlewareRunner {
|
|
|
95
95
|
* boolean indicating whether the route handler should still run.
|
|
96
96
|
*
|
|
97
97
|
* Short-circuits when a before method sets a status >= 400.
|
|
98
|
+
*
|
|
99
|
+
* ASYNC — each hook is awaited so middleware can perform async work (e.g.
|
|
100
|
+
* the distributed responseCache before-hook awaiting `backend.get`). Awaiting
|
|
101
|
+
* a synchronous hook that returns an array is harmless (the array resolves
|
|
102
|
+
* immediately), so existing sync hooks keep working unchanged.
|
|
98
103
|
*/
|
|
99
|
-
static runBefore(
|
|
104
|
+
static async runBefore(
|
|
100
105
|
classes: any[],
|
|
101
106
|
req: Tina4Request,
|
|
102
107
|
res: Tina4Response,
|
|
103
|
-
): [Tina4Request, Tina4Response, boolean] {
|
|
108
|
+
): Promise<[Tina4Request, Tina4Response, boolean]> {
|
|
104
109
|
for (const cls of classes) {
|
|
105
110
|
const methods = Object.getOwnPropertyNames(cls).filter(
|
|
106
111
|
(name) => typeof cls[name] === "function" && name.startsWith("before"),
|
|
107
112
|
);
|
|
108
113
|
for (const method of methods) {
|
|
109
|
-
const result = cls[method](req, res);
|
|
114
|
+
const result = await cls[method](req, res);
|
|
110
115
|
if (Array.isArray(result)) {
|
|
111
116
|
[req, res] = result as [Tina4Request, Tina4Response];
|
|
112
117
|
}
|
|
@@ -122,18 +127,22 @@ export class MiddlewareRunner {
|
|
|
122
127
|
/**
|
|
123
128
|
* Execute every afterX static method found on the supplied classes,
|
|
124
129
|
* in order. Returns the (possibly mutated) request and response pair.
|
|
130
|
+
*
|
|
131
|
+
* ASYNC — each hook is awaited (e.g. the responseCache after-hook awaiting
|
|
132
|
+
* `backend.set`). Awaiting a synchronous hook is harmless, so existing sync
|
|
133
|
+
* after-hooks keep working unchanged.
|
|
125
134
|
*/
|
|
126
|
-
static runAfter(
|
|
135
|
+
static async runAfter(
|
|
127
136
|
classes: any[],
|
|
128
137
|
req: Tina4Request,
|
|
129
138
|
res: Tina4Response,
|
|
130
|
-
): [Tina4Request, Tina4Response] {
|
|
139
|
+
): Promise<[Tina4Request, Tina4Response]> {
|
|
131
140
|
for (const cls of classes) {
|
|
132
141
|
const methods = Object.getOwnPropertyNames(cls).filter(
|
|
133
142
|
(name) => typeof cls[name] === "function" && name.startsWith("after"),
|
|
134
143
|
);
|
|
135
144
|
for (const method of methods) {
|
|
136
|
-
const result = cls[method](req, res);
|
|
145
|
+
const result = await cls[method](req, res);
|
|
137
146
|
if (Array.isArray(result)) {
|
|
138
147
|
[req, res] = result as [Tina4Request, Tina4Response];
|
|
139
148
|
}
|
|
@@ -1142,7 +1142,7 @@ ${reset}
|
|
|
1142
1142
|
...new Set([...Router.getClassMiddlewares(), ...MiddlewareRunner.getGlobal()]),
|
|
1143
1143
|
];
|
|
1144
1144
|
if (globalMiddleware.length > 0) {
|
|
1145
|
-
const [, , proceed] = MiddlewareRunner.runBefore(globalMiddleware, req, res);
|
|
1145
|
+
const [, , proceed] = await MiddlewareRunner.runBefore(globalMiddleware, req, res);
|
|
1146
1146
|
if (!proceed || res.raw.writableEnded) return;
|
|
1147
1147
|
}
|
|
1148
1148
|
|
|
@@ -1240,7 +1240,7 @@ ${reset}
|
|
|
1240
1240
|
// Header mutations here are no-ops once the response is flushed (Node
|
|
1241
1241
|
// sends headers with the body) — set response headers in beforeX.
|
|
1242
1242
|
if (globalMiddleware.length > 0) {
|
|
1243
|
-
MiddlewareRunner.runAfter(globalMiddleware, req, res);
|
|
1243
|
+
await MiddlewareRunner.runAfter(globalMiddleware, req, res);
|
|
1244
1244
|
}
|
|
1245
1245
|
|
|
1246
1246
|
if (!res.raw.writableEnded) {
|
|
@@ -31,17 +31,12 @@
|
|
|
31
31
|
|
|
32
32
|
import { QueryCache } from "./sqlTranslation.js";
|
|
33
33
|
import type { DatabaseAdapter, DatabaseResult, ColumnInfo, FieldDefinition } from "./types.js";
|
|
34
|
+
import type { CacheBackend } from "@tina4/core";
|
|
34
35
|
|
|
35
36
|
function isTruthy(val: string | undefined): boolean {
|
|
36
37
|
return ["true", "1", "yes", "on"].includes((val ?? "").trim().toLowerCase());
|
|
37
38
|
}
|
|
38
39
|
|
|
39
|
-
/** Network backends that cannot serve a SYNCHRONOUS db.fetch() path. */
|
|
40
|
-
const NETWORK_DB_CACHE_BACKENDS = new Set(["redis", "valkey", "memcached", "memcache", "mongodb", "mongo"]);
|
|
41
|
-
|
|
42
|
-
/** One-time guard so the distributed-DB-cache warning logs once per process. */
|
|
43
|
-
let _warnedNetworkDbCache = false;
|
|
44
|
-
|
|
45
40
|
/**
|
|
46
41
|
* Options for wrapping an adapter with a query cache. When several pooled
|
|
47
42
|
* connections must share one cache store (so a write on any connection
|
|
@@ -77,6 +72,25 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
77
72
|
private hits: number = 0;
|
|
78
73
|
private misses: number = 0;
|
|
79
74
|
|
|
75
|
+
/**
|
|
76
|
+
* Persistent-mode distributed backend (TINA4_DB_CACHE=true). Built lazily from
|
|
77
|
+
* the SAME unified `createBackend()` factory the response/KV cache uses, so
|
|
78
|
+
* multiple Database instances share one cache with global write-invalidation
|
|
79
|
+
* (parity with Python's connection.py, which routes the persistent DB cache
|
|
80
|
+
* through `_create_backend`). The read path (`fetchAsync`/`fetchOneAsync`/
|
|
81
|
+
* `queryAsync`) is async, so the backend's async get/set work directly — no
|
|
82
|
+
* sync-path restriction. Request-scoped mode keeps the in-process QueryCache
|
|
83
|
+
* above (ephemeral, fastest, never serialized).
|
|
84
|
+
*
|
|
85
|
+
* `null` until the first async read builds it; a `memory` backend (the
|
|
86
|
+
* default) means the persistent layer behaves in-process exactly as before, so
|
|
87
|
+
* default behaviour is unchanged and only an explicit redis/etc. backend
|
|
88
|
+
* distributes.
|
|
89
|
+
*/
|
|
90
|
+
private backend: CacheBackend | null = null;
|
|
91
|
+
private backendPromise: Promise<CacheBackend | null> | null = null;
|
|
92
|
+
private backendName: string;
|
|
93
|
+
|
|
80
94
|
constructor(adapter: DatabaseAdapter, options: CachedAdapterOptions = {}) {
|
|
81
95
|
this.adapter = adapter;
|
|
82
96
|
this.cachePersistent = options.persistent ?? isTruthy(process.env.TINA4_DB_CACHE);
|
|
@@ -94,32 +108,52 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
94
108
|
|
|
95
109
|
this.cache = options.sharedCache ?? new QueryCache({ defaultTtl: this.ttl, maxSize: 10000 });
|
|
96
110
|
|
|
97
|
-
// Persistent mode
|
|
98
|
-
//
|
|
99
|
-
//
|
|
100
|
-
//
|
|
101
|
-
//
|
|
102
|
-
//
|
|
103
|
-
//
|
|
104
|
-
|
|
105
|
-
// store here — the unified network backends live behind the async KV API.)
|
|
106
|
-
if (this.cachePersistent) {
|
|
107
|
-
const backendName = (process.env.TINA4_DB_CACHE_BACKEND ?? "memory").toLowerCase().trim();
|
|
108
|
-
if (NETWORK_DB_CACHE_BACKENDS.has(backendName) && !_warnedNetworkDbCache) {
|
|
109
|
-
_warnedNetworkDbCache = true;
|
|
110
|
-
// eslint-disable-next-line no-console
|
|
111
|
-
console.warn(
|
|
112
|
-
`[tina4] TINA4_DB_CACHE_BACKEND='${backendName}' selects a distributed cache, ` +
|
|
113
|
-
`but Node's db.fetch() is synchronous so distributed DB-query caching is not ` +
|
|
114
|
-
`available on that path — falling back to the in-process memory store. ` +
|
|
115
|
-
`For distributed caching use the response cache (cacheGet/cacheSet).`,
|
|
116
|
-
);
|
|
117
|
-
}
|
|
118
|
-
}
|
|
111
|
+
// Persistent mode now routes through the unified async backend (the read
|
|
112
|
+
// path — fetchAsync/fetchOneAsync/queryAsync — is async, so the backend's
|
|
113
|
+
// async get/set work directly). TINA4_DB_CACHE_BACKEND + TINA4_DB_CACHE_URL
|
|
114
|
+
// select the backend (default `memory` = in-process, unchanged behaviour).
|
|
115
|
+
// The backend is built lazily on first async read so an unreachable network
|
|
116
|
+
// backend degrades to `file` (via createBackend's own fallback) without
|
|
117
|
+
// blocking construction.
|
|
118
|
+
this.backendName = (process.env.TINA4_DB_CACHE_BACKEND ?? "memory").toLowerCase().trim();
|
|
119
119
|
|
|
120
120
|
CachedDatabaseAdapter.instances.add(this);
|
|
121
121
|
}
|
|
122
122
|
|
|
123
|
+
/**
|
|
124
|
+
* Whether the persistent layer should use a distributed/serialised backend.
|
|
125
|
+
* For the default `memory` backend we keep the in-process QueryCache (fast,
|
|
126
|
+
* no serialisation) so behaviour is identical to before; only an explicit
|
|
127
|
+
* non-memory backend (redis/valkey/memcached/mongodb/database/file) routes
|
|
128
|
+
* through the unified async backend for cross-instance sharing.
|
|
129
|
+
*/
|
|
130
|
+
private usesPersistentBackend(): boolean {
|
|
131
|
+
return this.cachePersistent && this.backendName !== "memory";
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
/** Lazily build (and memoise) the persistent backend via createBackend(). */
|
|
135
|
+
private async getBackend(): Promise<CacheBackend | null> {
|
|
136
|
+
if (this.backend) return this.backend;
|
|
137
|
+
if (!this.backendPromise) {
|
|
138
|
+
this.backendPromise = (async () => {
|
|
139
|
+
try {
|
|
140
|
+
// Dynamic import keeps @tina4/orm free of an import-time cycle with
|
|
141
|
+
// @tina4/core (whose `database` backend dynamically imports @tina4/orm).
|
|
142
|
+
const core: any = await import("@tina4/core");
|
|
143
|
+
const b: CacheBackend = await core.createBackend({
|
|
144
|
+
backend: this.backendName,
|
|
145
|
+
cacheUrl: process.env.TINA4_DB_CACHE_URL,
|
|
146
|
+
});
|
|
147
|
+
this.backend = b;
|
|
148
|
+
return b;
|
|
149
|
+
} catch {
|
|
150
|
+
return null; // fall back to the in-process QueryCache
|
|
151
|
+
}
|
|
152
|
+
})();
|
|
153
|
+
}
|
|
154
|
+
return this.backendPromise;
|
|
155
|
+
}
|
|
156
|
+
|
|
123
157
|
// ── Cache mode helpers ────────────────────────────────────
|
|
124
158
|
|
|
125
159
|
/** Current cache mode: "persistent" | "request" | "off". */
|
|
@@ -171,9 +205,14 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
171
205
|
mode: this.cacheMode(),
|
|
172
206
|
hits: this.hits,
|
|
173
207
|
misses: this.misses,
|
|
208
|
+
// `size` is the in-process figure. When a distributed persistent backend
|
|
209
|
+
// is active the authoritative size lives in redis/etc.; the hit/miss
|
|
210
|
+
// counters here are still real (tracked locally per the read path).
|
|
174
211
|
size: this.cache.size(),
|
|
175
212
|
ttl: this.ttl,
|
|
176
|
-
backend
|
|
213
|
+
// Report the actually-configured persistent backend so the operator sees
|
|
214
|
+
// where cross-instance entries land (parity with Python's cache_stats).
|
|
215
|
+
backend: this.usesPersistentBackend() ? this.backendName : "memory",
|
|
177
216
|
};
|
|
178
217
|
}
|
|
179
218
|
|
|
@@ -182,11 +221,68 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
182
221
|
this.cache.clear();
|
|
183
222
|
this.hits = 0;
|
|
184
223
|
this.misses = 0;
|
|
224
|
+
// Best-effort flush of the distributed backend too (fire-and-forget — this
|
|
225
|
+
// method is synchronous to keep db.cacheClear() ergonomic).
|
|
226
|
+
if (this.usesPersistentBackend()) {
|
|
227
|
+
void this.getBackend().then((b) => b?.clear()).catch(() => { /* best effort */ });
|
|
228
|
+
}
|
|
185
229
|
}
|
|
186
230
|
|
|
187
231
|
/** Clear the entire query cache (called on writes). */
|
|
188
232
|
private invalidate(): void {
|
|
189
233
|
this.cache.clear();
|
|
234
|
+
// Persistent distributed backend: clear it too so a write on ANY instance
|
|
235
|
+
// invalidates entries cached by ALL instances (global write-invalidation,
|
|
236
|
+
// parity with Python's _cache_invalidate → backend.clear()). Fire-and-forget
|
|
237
|
+
// on the sync write path (execute/insert/...); the async write methods below
|
|
238
|
+
// await invalidateAsync() instead for deterministic ordering.
|
|
239
|
+
if (this.usesPersistentBackend()) {
|
|
240
|
+
void this.getBackend().then((b) => b?.clear()).catch(() => { /* best effort */ });
|
|
241
|
+
}
|
|
242
|
+
}
|
|
243
|
+
|
|
244
|
+
/** Async write-invalidation — awaits the distributed backend clear. */
|
|
245
|
+
private async invalidateAsync(): Promise<void> {
|
|
246
|
+
this.cache.clear();
|
|
247
|
+
if (this.usesPersistentBackend()) {
|
|
248
|
+
const b = await this.getBackend();
|
|
249
|
+
if (b) { try { await b.clear(); } catch { /* best effort */ } }
|
|
250
|
+
}
|
|
251
|
+
}
|
|
252
|
+
|
|
253
|
+
// ── Persistent-backend get/set helpers (serialised rows) ──
|
|
254
|
+
//
|
|
255
|
+
// The persistent backend stores the adapter's row payload (T[] for fetch/
|
|
256
|
+
// query, {row:T|null} for fetchOne) as plain JSON — every backend (redis
|
|
257
|
+
// SETEX, mongo doc, db row) round-trips it. fetchOne is wrapped in an object
|
|
258
|
+
// so a genuine `null` row is distinguishable from a cache miss (the backend
|
|
259
|
+
// returns undefined on miss).
|
|
260
|
+
|
|
261
|
+
private async backendGetRows<T>(key: string): Promise<T[] | undefined> {
|
|
262
|
+
const b = await this.getBackend();
|
|
263
|
+
if (!b) return undefined;
|
|
264
|
+
const raw = await b.get(key);
|
|
265
|
+
return Array.isArray(raw) ? (raw as T[]) : undefined;
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
private async backendSetRows<T>(key: string, rows: T[]): Promise<void> {
|
|
269
|
+
const b = await this.getBackend();
|
|
270
|
+
if (b) { try { await b.set(key, rows, this.ttl); } catch { /* best effort */ } }
|
|
271
|
+
}
|
|
272
|
+
|
|
273
|
+
private async backendGetOne<T>(key: string): Promise<{ row: T | null } | undefined> {
|
|
274
|
+
const b = await this.getBackend();
|
|
275
|
+
if (!b) return undefined;
|
|
276
|
+
const raw = await b.get(key);
|
|
277
|
+
if (raw && typeof raw === "object" && "row" in (raw as object)) {
|
|
278
|
+
return raw as { row: T | null };
|
|
279
|
+
}
|
|
280
|
+
return undefined;
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
private async backendSetOne<T>(key: string, row: T | null): Promise<void> {
|
|
284
|
+
const b = await this.getBackend();
|
|
285
|
+
if (b) { try { await b.set(key, { row }, this.ttl); } catch { /* best effort */ } }
|
|
190
286
|
}
|
|
191
287
|
|
|
192
288
|
// ── DatabaseAdapter interface — writes flush, reads cache ──
|
|
@@ -327,6 +423,18 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
327
423
|
: this.adapter.fetch<T>(sql, params, limit, skip);
|
|
328
424
|
if (this.enabled) {
|
|
329
425
|
const key = QueryCache.queryKey(sql + `:L${limit}:S${skip}`, params as unknown[] | undefined);
|
|
426
|
+
// Persistent distributed backend is AUTHORITATIVE (mirrors Python, where a
|
|
427
|
+
// configured _cache_backend bypasses the in-process dict). This keeps
|
|
428
|
+
// cross-instance write-invalidation deterministic: a write clears the
|
|
429
|
+
// shared backend, so every instance misses on its next read.
|
|
430
|
+
if (this.usesPersistentBackend()) {
|
|
431
|
+
const shared = await this.backendGetRows<T>(key);
|
|
432
|
+
if (shared !== undefined) { this.hits++; return shared; }
|
|
433
|
+
const result = await run();
|
|
434
|
+
await this.backendSetRows(key, result);
|
|
435
|
+
this.misses++;
|
|
436
|
+
return result;
|
|
437
|
+
}
|
|
330
438
|
const cached = this.cache.get<T[]>(key);
|
|
331
439
|
if (cached !== undefined) {
|
|
332
440
|
this.hits++;
|
|
@@ -346,6 +454,14 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
346
454
|
: this.adapter.fetchOne<T>(sql, params);
|
|
347
455
|
if (this.enabled) {
|
|
348
456
|
const key = QueryCache.queryKey(sql + ":ONE", params as unknown[] | undefined);
|
|
457
|
+
if (this.usesPersistentBackend()) {
|
|
458
|
+
const shared = await this.backendGetOne<T>(key);
|
|
459
|
+
if (shared !== undefined) { this.hits++; return shared.row; }
|
|
460
|
+
const result = await run();
|
|
461
|
+
await this.backendSetOne(key, result);
|
|
462
|
+
this.misses++;
|
|
463
|
+
return result;
|
|
464
|
+
}
|
|
349
465
|
const cached = this.cache.get<T | null>(key);
|
|
350
466
|
if (cached !== undefined) {
|
|
351
467
|
this.hits++;
|
|
@@ -365,6 +481,14 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
365
481
|
: this.adapter.query<T>(sql, params);
|
|
366
482
|
if (this.enabled) {
|
|
367
483
|
const key = QueryCache.queryKey(sql + ":Q", params as unknown[] | undefined);
|
|
484
|
+
if (this.usesPersistentBackend()) {
|
|
485
|
+
const shared = await this.backendGetRows<T>(key);
|
|
486
|
+
if (shared !== undefined) { this.hits++; return shared; }
|
|
487
|
+
const result = await run();
|
|
488
|
+
await this.backendSetRows(key, result);
|
|
489
|
+
this.misses++;
|
|
490
|
+
return result;
|
|
491
|
+
}
|
|
368
492
|
const cached = this.cache.get<T[]>(key);
|
|
369
493
|
if (cached !== undefined) {
|
|
370
494
|
this.hits++;
|
|
@@ -379,28 +503,28 @@ export class CachedDatabaseAdapter implements DatabaseAdapter {
|
|
|
379
503
|
}
|
|
380
504
|
|
|
381
505
|
async executeAsync(sql: string, params?: unknown[]): Promise<unknown> {
|
|
382
|
-
if (this.enabled) this.
|
|
506
|
+
if (this.enabled) await this.invalidateAsync();
|
|
383
507
|
return (this.adapter as any).executeAsync
|
|
384
508
|
? await (this.adapter as any).executeAsync(sql, params)
|
|
385
509
|
: this.adapter.execute(sql, params);
|
|
386
510
|
}
|
|
387
511
|
|
|
388
512
|
async insertAsync(table: string, data: Record<string, unknown> | Record<string, unknown>[]): Promise<DatabaseResult> {
|
|
389
|
-
if (this.enabled) this.
|
|
513
|
+
if (this.enabled) await this.invalidateAsync();
|
|
390
514
|
return (this.adapter as any).insertAsync
|
|
391
515
|
? await (this.adapter as any).insertAsync(table, data)
|
|
392
516
|
: this.adapter.insert(table, data);
|
|
393
517
|
}
|
|
394
518
|
|
|
395
519
|
async updateAsync(table: string, data: Record<string, unknown>, filter: Record<string, unknown>, params?: unknown[]): Promise<DatabaseResult> {
|
|
396
|
-
if (this.enabled) this.
|
|
520
|
+
if (this.enabled) await this.invalidateAsync();
|
|
397
521
|
return (this.adapter as any).updateAsync
|
|
398
522
|
? await (this.adapter as any).updateAsync(table, data, filter, params)
|
|
399
523
|
: this.adapter.update(table, data, filter);
|
|
400
524
|
}
|
|
401
525
|
|
|
402
526
|
async deleteAsync(table: string, filter: Record<string, unknown> | string | Record<string, unknown>[], params?: unknown[]): Promise<DatabaseResult> {
|
|
403
|
-
if (this.enabled) this.
|
|
527
|
+
if (this.enabled) await this.invalidateAsync();
|
|
404
528
|
return (this.adapter as any).deleteAsync
|
|
405
529
|
? await (this.adapter as any).deleteAsync(table, filter, params)
|
|
406
530
|
: this.adapter.delete(table, filter as Record<string, unknown>);
|
|
@@ -458,8 +458,17 @@ export class Database {
|
|
|
458
458
|
/** Factory for creating new adapters (used by pool) */
|
|
459
459
|
private adapterFactory: (() => Promise<DatabaseAdapter>) | null = null;
|
|
460
460
|
|
|
461
|
-
/**
|
|
462
|
-
|
|
461
|
+
/**
|
|
462
|
+
* Whether a standalone write auto-commits. ON by default — a write made
|
|
463
|
+
* outside an explicit transaction commits on its own connection before
|
|
464
|
+
* returning (so it's durable and visible across pooled connections). Inside
|
|
465
|
+
* startTransaction()/commit()/rollback() the per-statement commit is
|
|
466
|
+
* suppressed, so explicit transactions stay atomic. Set TINA4_AUTOCOMMIT=false
|
|
467
|
+
* for strict manual-commit mode.
|
|
468
|
+
*/
|
|
469
|
+
private autoCommit: boolean = ["true", "1", "yes"].includes(
|
|
470
|
+
(process.env.TINA4_AUTOCOMMIT ?? "true").toLowerCase(),
|
|
471
|
+
);
|
|
463
472
|
private lastError: string | null = null;
|
|
464
473
|
|
|
465
474
|
/** Database engine type (sqlite, postgres, mysql, mssql, firebird) */
|
|
@@ -675,7 +684,7 @@ export class Database {
|
|
|
675
684
|
try {
|
|
676
685
|
const adapter = this.getNextAdapter();
|
|
677
686
|
const result = await adapterExecute(adapter, sql, params);
|
|
678
|
-
if (this.autoCommit) {
|
|
687
|
+
if (this.autoCommit && !this.inExplicitTransaction()) {
|
|
679
688
|
try { await adapterCommit(adapter); } catch { /* no active transaction */ }
|
|
680
689
|
}
|
|
681
690
|
this.lastError = null;
|
|
@@ -697,7 +706,7 @@ export class Database {
|
|
|
697
706
|
const result = (adapter as any).insertAsync
|
|
698
707
|
? await (adapter as any).insertAsync(table, data)
|
|
699
708
|
: adapter.insert(table, data);
|
|
700
|
-
if (this.autoCommit) {
|
|
709
|
+
if (this.autoCommit && !this.inExplicitTransaction()) {
|
|
701
710
|
try { await adapterCommit(adapter); } catch { /* no active transaction */ }
|
|
702
711
|
}
|
|
703
712
|
return result;
|
|
@@ -709,7 +718,7 @@ export class Database {
|
|
|
709
718
|
const result = (adapter as any).updateAsync
|
|
710
719
|
? await (adapter as any).updateAsync(table, data, filter ?? {}, params)
|
|
711
720
|
: adapter.update(table, data, filter ?? {}, params);
|
|
712
|
-
if (this.autoCommit) {
|
|
721
|
+
if (this.autoCommit && !this.inExplicitTransaction()) {
|
|
713
722
|
try { await adapterCommit(adapter); } catch { /* no active transaction */ }
|
|
714
723
|
}
|
|
715
724
|
return result;
|
|
@@ -721,7 +730,7 @@ export class Database {
|
|
|
721
730
|
const result = (adapter as any).deleteAsync
|
|
722
731
|
? await (adapter as any).deleteAsync(table, filter ?? {}, params)
|
|
723
732
|
: adapter.delete(table, filter ?? {}, params);
|
|
724
|
-
if (this.autoCommit) {
|
|
733
|
+
if (this.autoCommit && !this.inExplicitTransaction()) {
|
|
725
734
|
try { await adapterCommit(adapter); } catch { /* no active transaction */ }
|
|
726
735
|
}
|
|
727
736
|
return result;
|
|
@@ -741,6 +750,16 @@ export class Database {
|
|
|
741
750
|
}
|
|
742
751
|
}
|
|
743
752
|
|
|
753
|
+
/**
|
|
754
|
+
* True while an explicit transaction is active on the current async context.
|
|
755
|
+
* startTransaction() pins an adapter into txStore; commit()/rollback() clear
|
|
756
|
+
* it. Standalone writes only auto-commit when this is false, so per-statement
|
|
757
|
+
* commits never break the atomicity of an explicit transaction.
|
|
758
|
+
*/
|
|
759
|
+
private inExplicitTransaction(): boolean {
|
|
760
|
+
return !!this.txStore.getStore()?.adapter;
|
|
761
|
+
}
|
|
762
|
+
|
|
744
763
|
/**
|
|
745
764
|
* Start a transaction. Pins the adapter to the current async context for
|
|
746
765
|
* the whole transaction so executes and the final commit/rollback all run
|