@jskit-ai/database-runtime 0.1.261 → 0.1.263

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,14 @@
1
+ # Changelog
2
+
3
+ ## Unreleased
4
+
5
+ - `@jskit-ai/database-runtime-mysql` and `@jskit-ai/database-runtime-postgres` take `@jskit-ai/database-runtime` as a required peer at the same version, where it was a dependency, as `@jskit-ai/resource-runtime` and `@jskit-ai/crud-core` do: the application declares it beside its driver (`npm install @jskit-ai/database-runtime @jskit-ai/database-runtime-mysql`) and holds its one copy. npm installs a required peer as it installed the dependency.
6
+ - **Breaking (data-access redesign).** `./shared/visibility` is removed (`applyVisibility`, `applyVisibilityOwners`, `DEFAULT_VISIBILITY_COLUMNS`, also re-exported from `./shared`): a table resource declares its `ownership`, and every handle statement is scoped by it. `./shared` and `./shared/repositoryOptions` no longer export `applyForUpdate`, `createWithTransaction`, `joinOrOpenTransaction`, `lockScopedRecordId`, `resolveQueryOptions`, `resolveRepoClient`, `resolveTransaction` and `runInTransaction`: a transaction is the table handle's `withTransaction(options, work)`, a row lock its `lock` or `lockMany`, and a call's client and transaction ride in the `options` it passes on. The Before | Now table in the existing-application migration guide maps each one.
7
+ - `./server/testDatabases` exports `requireTestDatabase(environment)`, which answers `TEST_DB_NAME` and throws `TestDatabaseMissingError` without it, so a data test fails instead of skipping, and `tableNames(knex)`, the tables of the test database.
8
+ - `retryLostInsertRace(options, run)` (from `./shared`) runs a transaction once more when its insert lost a race for a unique key: `run`, usually `() => repository.withTransaction(options, work)`, failed with a duplicate whose transaction rolled back in full, outside a caller's transaction (`isLostDuplicateInsertRace`). The database reports the duplicate only once the winner has committed, so the second run's lookup by the key finds the winner's row; a second duplicate is final, and inside a caller's transaction (`options.trx`) the error reaches the caller without a rerun. Apps wrote this retry by hand around each idempotent write.
9
+ - The migration configuration lists the application's `migrations/constraints/` only when the directory exists, as it already did for a package's: an application without one failed `db:migrate` and `jskit test --databases` with ENOENT (scandir `migrations/constraints`).
10
+ - `retryDeadlocks` (and so `withTransaction` and every handle write outside a transaction) also runs again a transaction that MariaDB failed as "Record has changed since last read" (errno 1020, `ER_CHECKREAD`): with `innodb_snapshot_isolation` on, the second of two writers whose filters matched one row fails so when it locks the row the first changed. json-rest-api has rolled the transaction back, so the new run reads a fresh snapshot, and a `patchWhere` or `deleteWhere` then leaves out the row that no longer matches. `isDeadlockError` recognises it.
11
+ - **Breaking.** `runtime.database` is `{ driver, knex }`: its `transactionManager`, with `inTransaction(work, { trx })`, and the `RuntimeTransactionManager` type are removed, so a transaction has one way in, the table handle's `withTransaction(options, work)`, which joins the caller's transaction and retries a deadlocked outermost one. Nothing in JSKIT called it. Code that opened a transaction with it opens it on a table's handle (or its owner's API) instead.
12
+ - A PostgreSQL session runs in UTC whatever the server's default time zone, as a MySQL connection does with `timezone: "Z"`: the runtime's Knex and the migration configuration (`createKnexMigrationConfig`) set `TimeZone` to UTC in the pool's `afterCreate`, and the test databases' own connections keep it. PostgreSQL reads a date-time written without an offset, as the `datetime-utc` storage writes it, in the session's zone, so on a server set to another zone a `timestamptz` value moved by that zone's offset.
13
+ - The test databases' `tableNames()` and the migration ledger read accept every name Knex gives PostgreSQL's client (`pg`, `postgres`, `postgresql`); `postgres` used to be read as MySQL.
14
+ - `retryDeadlocks` runs a transaction again only when the database chose it as a deadlock victim, never after a lock wait timed out (MySQL's 1205 and PostgreSQL's `55P03`): MySQL rolls back only the timed-out statement, so a rerun could repeat the transaction's earlier work. Behaviour is unchanged; tests now pin it.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@jskit-ai/database-runtime",
3
- "version": "0.1.261",
3
+ "version": "0.1.263",
4
4
  "type": "module",
5
5
  "scripts": {
6
6
  "test": "jskit test"
@@ -17,8 +17,7 @@
17
17
  "./shared/databaseClient": "./src/shared/databaseClient.js",
18
18
  "./shared/dialect": "./src/shared/dialect.js",
19
19
  "./shared/duplicateEntry": "./src/shared/duplicateEntry.js",
20
- "./shared/repositoryOptions": "./src/shared/repositoryOptions.js",
21
- "./shared/visibility": "./src/shared/visibility.js"
20
+ "./shared/repositoryOptions": "./src/shared/repositoryOptions.js"
22
21
  },
23
22
  "dependencies": {
24
23
  "knex": "^3.1.0"
@@ -72,7 +71,7 @@
72
71
  },
73
72
  {
74
73
  "subpath": "./shared",
75
- "summary": "Exports shared repository helpers (transactions, record ids, JSON columns, visibility, duplicate-entry detection and date/time serialization)."
74
+ "summary": "Exports shared database helpers (deadlock retries, record ids, JSON columns, duplicate-entry detection and date/time serialization)."
76
75
  },
77
76
  {
78
77
  "subpath": "./server/types",
@@ -83,6 +82,6 @@
83
82
  }
84
83
  },
85
84
  "peerDependencies": {
86
- "@jskit-ai/kernel": "0.2.12"
85
+ "@jskit-ai/kernel": "0.2.14"
87
86
  }
88
87
  }
@@ -1,7 +1,6 @@
1
1
  import diagnosticsChannel from "node:diagnostics_channel";
2
2
  import { normalizeDatabaseClient, toKnexClientId } from "../shared/databaseClient.js";
3
- import { resolveKnexConnectionFromEnvironment } from "../shared/databaseConnection.js";
4
- import { createTransactionManager } from "../shared/transactionManager.js";
3
+ import { resolveKnexConnectionFromEnvironment, resolveKnexPool } from "../shared/databaseConnection.js";
5
4
  import { loadKnexFactory } from "./knexFactory.js";
6
5
 
7
6
  // The diagnostics channel that carries each statement the shared Knex sends (Knex's `query` event: its `sql`,
@@ -18,7 +17,8 @@ function resolveDriverDialectId(driver) {
18
17
  }
19
18
 
20
19
  /**
21
- * The database connection every feature shares: the installed driver, its Knex client and a transaction manager.
20
+ * The database connection every feature shares: the installed driver and its Knex client. Table work opens its
21
+ * transactions through its handle's withTransaction.
22
22
  * @param {object} dependencies
23
23
  * @param {import("./types.js").RuntimeDatabaseDriver} dependencies.driver
24
24
  * @param {import("@jskit-ai/kernel/server/types").RuntimeEnv} dependencies.env
@@ -30,6 +30,7 @@ function createDatabase({ driver, env }) {
30
30
  throw new Error(`DB_CLIENT="${configuredClient}" does not match installed database driver "${dialectId}".`);
31
31
  }
32
32
 
33
+ const pool = resolveKnexPool(dialectId);
33
34
  /** @type {import("./types.js").RuntimeKnex} */
34
35
  const knex = loadKnexFactory()({
35
36
  client: toKnexClientId(dialectId),
@@ -37,16 +38,15 @@ function createDatabase({ driver, env }) {
37
38
  client: dialectId,
38
39
  defaultPort: dialectId === "pg" ? 5432 : 3306,
39
40
  context: "database runtime"
40
- })
41
+ }),
42
+ ...(pool ? { pool } : {})
41
43
  });
42
44
  const statements = diagnosticsChannel.channel(RUNTIME_QUERY_CHANNEL);
43
45
  knex.on("query", (query) => statements.publish(query));
44
- const transactionManager = createTransactionManager({ knex });
45
46
 
46
47
  return Object.freeze({
47
48
  driver,
48
- knex,
49
- transactionManager
49
+ knex
50
50
  });
51
51
  }
52
52
 
@@ -1,7 +1,7 @@
1
1
  import path from "node:path";
2
2
  import { createKnexMigrationConfigFromApp } from "./knexMigrationConfig.js";
3
3
  import { loadKnexFactory } from "./knexFactory.js";
4
- import { runInTransaction } from "../shared/repositoryOptions.js";
4
+ import { retryDeadlocks } from "../shared/repositoryOptions.js";
5
5
 
6
6
  function normalizeMigrationResult(result) {
7
7
  const [batchValue, migrationValues] = Array.isArray(result) ? result : [];
@@ -44,13 +44,13 @@ async function runDatabaseSetup({ knex, seed, appRoot = process.cwd(), environme
44
44
  });
45
45
  }
46
46
 
47
- await runInTransaction(knex, async (transaction) => {
47
+ await retryDeadlocks(() => knex.transaction(async (transaction) => {
48
48
  await seed(Object.freeze({
49
49
  knex: transaction,
50
50
  appRoot: path.resolve(appRoot),
51
51
  environment
52
52
  }));
53
- });
53
+ }));
54
54
 
55
55
  return Object.freeze({
56
56
  ...migrationResult,
@@ -1,3 +1,4 @@
1
+ import { existsSync } from "node:fs";
1
2
  import path from "node:path";
2
3
  import { access } from "node:fs/promises";
3
4
  import { discoverInstalledPackages } from "@jskit-ai/kernel/server/support";
@@ -5,7 +6,7 @@ import {
5
6
  normalizeDatabaseClient,
6
7
  toKnexClientId
7
8
  } from "../shared/databaseClient.js";
8
- import { resolveKnexConnectionFromEnvironment } from "../shared/databaseConnection.js";
9
+ import { resolveKnexConnectionFromEnvironment, resolveKnexPool } from "../shared/databaseConnection.js";
9
10
 
10
11
  /**
11
12
  * The Knex configuration for migrating the database the application's environment names.
@@ -32,6 +33,9 @@ function createKnexMigrationConfig({
32
33
  }
33
34
  const dialectId = normalizeDatabaseClient(client);
34
35
  const resolvedMigrationsDirectory = path.resolve(appRoot, migrationsDirectory);
36
+ // An application needs no constraints/ until it has a constraint; Knex refuses a missing directory.
37
+ const applicationConstraintsDirectory = path.join(resolvedMigrationsDirectory, "constraints");
38
+ const pool = resolveKnexPool(dialectId);
35
39
 
36
40
  return {
37
41
  client: toKnexClientId(dialectId),
@@ -40,6 +44,7 @@ function createKnexMigrationConfig({
40
44
  defaultPort: dialectId === "pg" ? 5432 : 3306,
41
45
  context: "knex migrations"
42
46
  }),
47
+ ...(pool ? { pool } : {}),
43
48
  migrations: {
44
49
  // Packages create the tables application migrations build on; package constraints
45
50
  // can then reference application tables such as users.
@@ -47,7 +52,7 @@ function createKnexMigrationConfig({
47
52
  ...packageMigrationDirectories.map((directory) => path.resolve(directory)),
48
53
  resolvedMigrationsDirectory,
49
54
  ...packageConstraintDirectories.map((directory) => path.resolve(directory)),
50
- path.join(resolvedMigrationsDirectory, "constraints")
55
+ ...(existsSync(applicationConstraintsDirectory) ? [applicationConstraintsDirectory] : [])
51
56
  ],
52
57
  extension: "cjs",
53
58
  sortDirsSeparately: true
@@ -19,7 +19,9 @@
19
19
  // PROD_DB_NAME are never touched. Two runs at once need different database names, as before.
20
20
  //
21
21
  // A database test that guards a path's cost counts the statements the path sends with countQueries(), or, for a route
22
- // of a server the test booted, countRuntimeQueries().
22
+ // of a server the test booted, countRuntimeQueries(); expectFlatQueries() in @jskit-ai/resource-runtime/test compares
23
+ // those counts at a few rows and at many. A data test never skips: requireTestDatabase() fails it when TEST_DB_NAME is
24
+ // not set.
23
25
  import { createHash } from "node:crypto";
24
26
  import diagnosticsChannel from "node:diagnostics_channel";
25
27
  import { existsSync, mkdirSync, mkdtempSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
@@ -27,6 +29,7 @@ import os from "node:os";
27
29
  import path from "node:path";
28
30
  import { fileURLToPath } from "node:url";
29
31
  import { parseDatabaseUrl } from "../shared/databaseConnection.js";
32
+ import { detectDialectFromClient } from "../shared/dialect.js";
30
33
  import { RUNTIME_QUERY_CHANNEL } from "./database.js";
31
34
  import { loadKnexFactory } from "./knexFactory.js";
32
35
 
@@ -127,7 +130,8 @@ function knexOn(knexConfig, database, connectionOptions = {}) {
127
130
  return loadKnexFactory()({
128
131
  ...knexConfig,
129
132
  connection: { ...knexConfig.connection, ...connectionOptions, database },
130
- pool: { min: 0, max: 1 }
133
+ // The configuration's own pool settings stay, such as a PostgreSQL session's UTC time zone.
134
+ pool: { ...knexConfig.pool, min: 0, max: 1 }
131
135
  });
132
136
  }
133
137
 
@@ -178,7 +182,7 @@ async function testDatabaseSnapshotKey(knexConfig, { appRoot = process.cwd(), sc
178
182
  async function ledgerNames(knex, ledger, tables) {
179
183
  if (!tables.includes(ledger)) return [];
180
184
  const result = await knex.raw("SELECT name FROM ?? ORDER BY id", [ledger]);
181
- return (knex.client.config.client === "pg" ? result.rows : result[0]).map(({ name }) => name);
185
+ return (detectDialectFromClient(knex) === "postgres" ? result.rows : result[0]).map(({ name }) => name);
182
186
  }
183
187
 
184
188
  const mysql = {
@@ -571,14 +575,59 @@ async function countRuntimeQueries(work) {
571
575
  }, work);
572
576
  }
573
577
 
578
+ /** A data test that has no test database: it fails rather than skip. */
579
+ class TestDatabaseMissingError extends Error {
580
+ constructor() {
581
+ super("TEST_DB_NAME is not set; data tests never skip.");
582
+ this.name = "TestDatabaseMissingError";
583
+ }
584
+ }
585
+
586
+ /**
587
+ * The test database a data test runs on. A data test never skips: without TEST_DB_NAME it throws
588
+ * TestDatabaseMissingError, so a run without a database fails instead of passing what it never ran.
589
+ * @param {import("@jskit-ai/kernel/server/types").RuntimeEnv} environment The test's environment, such as `process.env`.
590
+ * @returns {string}
591
+ */
592
+ function requireTestDatabase(environment) {
593
+ const name = String(environment.TEST_DB_NAME || "").trim();
594
+ if (!name) {
595
+ throw new TestDatabaseMissingError();
596
+ }
597
+ return name;
598
+ }
599
+
600
+ /**
601
+ * The tables of the database `knex` is connected to, sorted: MySQL's and MariaDB's base tables, PostgreSQL's in the
602
+ * current schema, SQLite's own tables (its internal ones aside).
603
+ * @param {import("knex").Knex | import("knex").Knex.Transaction} knex
604
+ * @returns {Promise<string[]>}
605
+ */
606
+ async function tableNames(knex) {
607
+ const client = String(knex.client.config.client);
608
+ if (client.includes("sqlite")) {
609
+ const rows = await knex.raw("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'");
610
+ return rows.map(({ name }) => name).sort();
611
+ }
612
+ // Knex names PostgreSQL "pg", "postgres" or "postgresql".
613
+ if (detectDialectFromClient(knex) === "postgres") {
614
+ return postgres.tables(knex);
615
+ }
616
+ const [[{ name: database }]] = await knex.raw("SELECT DATABASE() AS name");
617
+ return mysql.tables(knex, database);
618
+ }
619
+
574
620
  export {
575
621
  RUN_SETTINGS_FILE,
622
+ TestDatabaseMissingError,
576
623
  countQueries,
577
624
  countRuntimeQueries,
578
625
  matchesSnapshot,
579
626
  prepareTestDatabases,
580
627
  requireDisposableDatabase,
628
+ requireTestDatabase,
581
629
  restoreTestDatabase,
630
+ tableNames,
582
631
  testDatabaseNames,
583
632
  testDatabaseRun,
584
633
  testDatabaseSnapshotKey
@@ -2,8 +2,7 @@
2
2
 
3
3
  /** @typedef {ReturnType<typeof import("./database.js").createDatabase>} RuntimeDatabase */
4
4
  /** @typedef {import("knex").Knex} RuntimeKnex A Knex client: the one `runtime.database` shares, or one an application passes in. */
5
- /** @typedef {RuntimeDatabase["transactionManager"]} RuntimeTransactionManager The transaction manager `runtime.database` shares. */
6
- /** @typedef {import("knex").Knex.Transaction} RuntimeKnexTransaction A Knex transaction: the `trx` a repository or service call runs in, as the transaction manager hands it to its work. */
5
+ /** @typedef {import("knex").Knex.Transaction} RuntimeKnexTransaction A Knex transaction: the `trx` a repository or service call runs in. */
7
6
  /** @typedef {import("knex").Knex.QueryBuilder} RuntimeKnexQueryBuilder A Knex query that a hook or helper narrows. */
8
7
 
9
8
  /**
@@ -141,9 +141,28 @@ function resolveKnexConnectionFromEnvironment(
141
141
  return connection;
142
142
  }
143
143
 
144
+ // Runs on each new PostgreSQL connection before the pool hands it out.
145
+ function setPostgresSessionToUtc(connection, done) {
146
+ connection.query("SET TIME ZONE 'UTC'", (error) => done(error, connection));
147
+ }
148
+
149
+ /**
150
+ * The Knex pool settings of a dialect: a PostgreSQL session runs in UTC whatever
151
+ * the server's default time zone, as a MySQL connection does with timezone "Z".
152
+ * PostgreSQL reads a date-time written without an offset (as the "datetime-utc"
153
+ * storage writes it) in the session's time zone, and formats timestamptz values
154
+ * and date functions in it, so a non-UTC session would shift stored instants.
155
+ * @param {string} client The dialect id: "mysql2" or "pg".
156
+ * @returns {{ afterCreate: (connection: any, done: (error: unknown, connection: any) => void) => void } | undefined}
157
+ */
158
+ function resolveKnexPool(client) {
159
+ return normalizeDatabaseClient(client) === "pg" ? { afterCreate: setPostgresSessionToUtc } : undefined;
160
+ }
161
+
144
162
  export {
145
163
  parseDatabaseUrl,
146
164
  resolveDatabaseClientFromEnvironment,
147
165
  resolveDatabaseConnectionFromEnvironment,
148
- resolveKnexConnectionFromEnvironment
166
+ resolveKnexConnectionFromEnvironment,
167
+ resolveKnexPool
149
168
  };
@@ -15,6 +15,13 @@ function isPostgresDuplicateEntryError(error) {
15
15
  return code === "23505";
16
16
  }
17
17
 
18
+ // better-sqlite3 names the constraint a duplicate broke: a unique index or the primary key.
19
+ const SQLITE_DUPLICATE_CODES = Object.freeze(new Set(["SQLITE_CONSTRAINT_UNIQUE", "SQLITE_CONSTRAINT_PRIMARYKEY"]));
20
+
21
+ function isSqliteDuplicateEntryError(error) {
22
+ return SQLITE_DUPLICATE_CODES.has(String(error?.code || "").trim().toUpperCase());
23
+ }
24
+
18
25
  function findDuplicateEntryError(error, { dialect = "", client = null } = {}) {
19
26
  const resolvedDialect =
20
27
  normalizeDialect(dialect) || (client ? normalizeDialect(detectDialectFromClient(client)) : "");
@@ -28,8 +35,10 @@ function findDuplicateEntryError(error, { dialect = "", client = null } = {}) {
28
35
  matches = isPostgresDuplicateEntryError(error);
29
36
  } else if (resolvedDialect === "mysql") {
30
37
  matches = isMysqlDuplicateEntryError(error);
38
+ } else if (resolvedDialect.includes("sqlite")) {
39
+ matches = isSqliteDuplicateEntryError(error);
31
40
  } else {
32
- matches = isMysqlDuplicateEntryError(error) || isPostgresDuplicateEntryError(error);
41
+ matches = isMysqlDuplicateEntryError(error) || isPostgresDuplicateEntryError(error) || isSqliteDuplicateEntryError(error);
33
42
  }
34
43
 
35
44
  if (matches) {
@@ -10,12 +10,7 @@ export {
10
10
  export { normalizeDialect, detectDialectFromClient } from "./dialect.js";
11
11
  export { normalizeDatabaseClient, toKnexClientId } from "./databaseClient.js";
12
12
  export { findDuplicateEntryError, isDuplicateEntryError, isLostDuplicateInsertRace } from "./duplicateEntry.js";
13
- export { DEFAULT_VISIBILITY_COLUMNS, applyVisibility, applyVisibilityOwners } from "./visibility.js";
14
13
  export {
15
- resolveTransaction,
16
- resolveQueryOptions,
17
- resolveRepoClient,
18
- applyForUpdate,
19
14
  parseJsonObject,
20
15
  stringifyJsonObject,
21
16
  normalizeNullableString,
@@ -25,8 +20,5 @@ export {
25
20
  toDbJson,
26
21
  isDeadlockError,
27
22
  retryDeadlocks,
28
- runInTransaction,
29
- createWithTransaction,
30
- joinOrOpenTransaction,
31
- lockScopedRecordId
23
+ retryLostInsertRace
32
24
  } from "./repositoryOptions.js";
@@ -1,43 +1,5 @@
1
1
  import { normalizeCanonicalRecordIdText } from "@jskit-ai/kernel/shared/support/normalize";
2
-
3
- /**
4
- * The transaction a repository or service call runs in: its options' `trx`,
5
- * or null for a call outside one (a call may pass no options).
6
- * @param {{ trx?: import("../server/types.js").RuntimeKnexTransaction | null } | null} [options]
7
- * @returns {import("../server/types.js").RuntimeKnexTransaction | null}
8
- */
9
- function resolveTransaction(options) {
10
- if (!options || typeof options !== "object") {
11
- return null;
12
- }
13
- return options.trx || null;
14
- }
15
-
16
- /**
17
- * A query's transaction (see resolveTransaction) and whether it locks the rows it reads.
18
- * @param {{ trx?: import("../server/types.js").RuntimeKnexTransaction | null, forUpdate?: boolean } | null} [options]
19
- */
20
- function resolveQueryOptions(options) {
21
- return {
22
- trx: resolveTransaction(options),
23
- forUpdate: options?.forUpdate === true
24
- };
25
- }
26
-
27
- function resolveRepoClient(dbClient, options = {}) {
28
- const { trx } = resolveQueryOptions(options);
29
- return trx || dbClient;
30
- }
31
-
32
- function applyForUpdate(query, options = {}) {
33
- const { forUpdate } = resolveQueryOptions(options);
34
- // readable-check: allow method-probe - platform object: a Knex query builder that cannot lock rows is used as is
35
- if (forUpdate && typeof query?.forUpdate === "function") {
36
- return query.forUpdate();
37
- }
38
-
39
- return query;
40
- }
2
+ import { isLostDuplicateInsertRace } from "./duplicateEntry.js";
41
3
 
42
4
  function parseJsonObject(value, fallback = {}) {
43
5
  const source = String(value || "").trim();
@@ -164,8 +126,14 @@ function toDbJson(value) {
164
126
 
165
127
  /**
166
128
  * True when the error reports a transaction the database aborted, and rolled
167
- * back in full, as a deadlock victim. A wrapper that reports its transaction
168
- * committed (json-rest-api's transactionOutcome) never is one.
129
+ * back in full, as a deadlock victim, a SQLite write that found the database
130
+ * locked by another connection (SQLITE_BUSY), or, with MariaDB's
131
+ * innodb_snapshot_isolation on, a locking read of a row another transaction
132
+ * changed after this one's snapshot ("Record has changed since last read",
133
+ * errno 1020, ER_CHECKREAD); after the last two json-rest-api rolls its
134
+ * transaction back, so a new run starts from a fresh snapshot. A wrapper that
135
+ * reports its transaction committed (json-rest-api's transactionOutcome)
136
+ * never is one.
169
137
  * @param {unknown} error
170
138
  */
171
139
  function isDeadlockError(error) {
@@ -176,7 +144,8 @@ function isDeadlockError(error) {
176
144
  return false;
177
145
  }
178
146
  const code = String(error.code || "").toUpperCase();
179
- if (code === "ER_LOCK_DEADLOCK" || code === "40P01" || Number(error.errno) === 1213) {
147
+ if (code === "ER_LOCK_DEADLOCK" || code === "ER_CHECKREAD" || code === "40P01" || code === "SQLITE_BUSY" ||
148
+ Number(error.errno) === 1213 || Number(error.errno) === 1020) {
180
149
  return true;
181
150
  }
182
151
  error = error.cause;
@@ -187,6 +156,8 @@ function isDeadlockError(error) {
187
156
  /**
188
157
  * Runs a whole transaction again when the database chose it as a deadlock
189
158
  * victim, so `run` must be safe to repeat: no external effects inside it.
159
+ * A lock wait that timed out is never run again: MySQL rolls back only the
160
+ * timed-out statement, so the transaction's earlier work could repeat.
190
161
  * @template T
191
162
  * @param {() => Promise<T>} run
192
163
  * @param {{ retries?: number }} [options]
@@ -205,88 +176,42 @@ async function retryDeadlocks(run, { retries = 3 } = {}) {
205
176
  }
206
177
  }
207
178
 
208
- async function runInTransaction(knex, callback, options = {}) {
209
- if (typeof callback !== "function") {
210
- throw new TypeError("runInTransaction requires callback.");
211
- }
212
- if (!knex || typeof knex !== "function") {
213
- throw new TypeError("runInTransaction requires knex client.");
214
- }
215
-
216
- // readable-check: allow method-probe - platform object: a Knex client without transaction() runs the callback on itself
217
- if (typeof knex.transaction !== "function") {
218
- return callback(knex);
219
- }
220
- // Inside a transaction this opens a savepoint; a deadlock aborts the
221
- // outermost transaction, and only that one retries.
222
- if (knex.isTransaction) {
223
- return knex.transaction(callback);
224
- }
225
- return retryDeadlocks(() => knex.transaction(callback), options);
226
- }
227
-
228
- function createWithTransaction(knex) {
229
- return function withTransaction(work) {
230
- return runInTransaction(knex, work);
231
- };
232
- }
233
-
234
179
  /**
235
- * Joins the caller's transaction or opens one: runs work in the call's
236
- * `options.trx` when the caller holds a transaction, else in the one that
237
- * withTransaction opens.
180
+ * Runs `run` once more when its insert lost a race for a unique key: another
181
+ * request inserted the same key first (an idempotency or request key, a
182
+ * revision number, a computed id), and `run`'s own transaction rolled back in
183
+ * full (isLostDuplicateInsertRace). The database reports the duplicate only
184
+ * once the winner has committed, so the second run reads a snapshot that holds
185
+ * the winner's row: its lookup by the key finds it, and every other check sees
186
+ * the current state. A second duplicate is final. `run` is a transaction
187
+ * opened with these `options`, usually `() => repository.withTransaction(options,
188
+ * work)`, and must be safe to repeat, as for a deadlock retry. Inside a
189
+ * caller's transaction (`options.trx`) the failed insert has failed that
190
+ * transaction, so `run` never runs again there: the error reaches the caller,
191
+ * whose own retry, if it has one, runs the whole of it again.
238
192
  * @template T
239
- * @param {{ trx?: import("../server/types.js").RuntimeKnexTransaction | null }} options The call's options.
240
- * @param {(work: (trx: import("../server/types.js").RuntimeKnexTransaction) => Promise<T>) => Promise<T>} withTransaction
241
- * The repository's withTransaction: opens a transaction and runs work in it.
242
- * @param {(trx: import("../server/types.js").RuntimeKnexTransaction) => Promise<T>} work
193
+ * @param {{ trx?: unknown }} options The options `run` opens its transaction with.
194
+ * @param {() => Promise<T>} run
243
195
  * @returns {Promise<T>}
244
196
  */
245
- function joinOrOpenTransaction(options, withTransaction, work) {
246
- return options.trx ? work(options.trx) : withTransaction(work);
247
- }
248
-
249
- function requireSqlIdentifier(value, label) {
250
- const identifier = String(value || "").trim();
251
- if (!/^[A-Za-z_][A-Za-z0-9_]*$/u.test(identifier)) {
252
- throw new TypeError(`${label} must be a safe SQL identifier.`);
253
- }
254
- return identifier;
255
- }
256
-
257
- async function lockScopedRecordId({
258
- trx,
259
- tableName,
260
- recordId,
261
- scopeId,
262
- idColumn = "id",
263
- scopeColumn = "workspace_id"
264
- } = {}) {
265
- if (typeof trx !== "function") {
266
- throw new TypeError("lockScopedRecordId requires a transaction.");
197
+ async function retryLostInsertRace(options, run, ...rest) {
198
+ if (!options || typeof options !== "object" || typeof run !== "function" || rest.length > 0) {
199
+ throw new TypeError(
200
+ "retryLostInsertRace() takes (options, run): the options run opens its transaction with, then " +
201
+ "run, such as () => repository.withTransaction(options, work)."
202
+ );
267
203
  }
268
- const table = requireSqlIdentifier(tableName, "lockScopedRecordId tableName");
269
- const recordKey = requireSqlIdentifier(idColumn, "lockScopedRecordId idColumn");
270
- const scopeKey = requireSqlIdentifier(scopeColumn, "lockScopedRecordId scopeColumn");
271
- const normalizedRecordId = normalizeDbRecordId(recordId);
272
- const normalizedScopeId = normalizeDbRecordId(scopeId);
273
- if (!normalizedRecordId || !normalizedScopeId) {
274
- return null;
204
+ try {
205
+ return await run();
206
+ } catch (error) {
207
+ if (!isLostDuplicateInsertRace(error, options)) {
208
+ throw error;
209
+ }
210
+ return run();
275
211
  }
276
- const row = await trx(table)
277
- .select(recordKey)
278
- .where(recordKey, normalizedRecordId)
279
- .andWhere(scopeKey, normalizedScopeId)
280
- .forUpdate()
281
- .first();
282
- return normalizeDbRecordId(row?.[recordKey]);
283
212
  }
284
213
 
285
214
  export {
286
- resolveTransaction,
287
- resolveQueryOptions,
288
- resolveRepoClient,
289
- applyForUpdate,
290
215
  parseJsonObject,
291
216
  stringifyJsonObject,
292
217
  normalizeNullableString,
@@ -296,8 +221,5 @@ export {
296
221
  toDbJson,
297
222
  isDeadlockError,
298
223
  retryDeadlocks,
299
- runInTransaction,
300
- createWithTransaction,
301
- joinOrOpenTransaction,
302
- lockScopedRecordId
224
+ retryLostInsertRace
303
225
  };
@@ -0,0 +1,66 @@
1
+ import assert from "node:assert/strict";
2
+ import test from "node:test";
3
+ import knexFactory from "knex";
4
+ import { createDatabase } from "../src/server/database.js";
5
+ import { resolveKnexConnectionFromEnvironment } from "../src/shared/databaseConnection.js";
6
+ import { toDatabaseDateTimeUtc } from "../src/shared/dateUtils.js";
7
+
8
+ // Runs beside the PostgreSQL test database: run with DB_NAME set to TEST_DB_NAME (and the usual DB_HOST, DB_PORT,
9
+ // DB_USER and DB_PASSWORD), and DB_CLIENT unset or pg.
10
+ const TEST_DB_NAME = process.env.TEST_DB_NAME?.trim();
11
+ const ON_TEST_DATABASE = Boolean(TEST_DB_NAME) && process.env.DB_NAME === TEST_DB_NAME;
12
+ const POSTGRES_CLIENT = !process.env.DB_CLIENT || process.env.DB_CLIENT === "pg";
13
+ const SKIP = ON_TEST_DATABASE && POSTGRES_CLIENT ? false : "not on a PostgreSQL test database (DB_NAME=$TEST_DB_NAME)";
14
+
15
+ // The default time zone of the server in this test. node-postgres sends PGOPTIONS as each new connection's startup
16
+ // options, so every session starts in it, as on a server configured for that zone.
17
+ const SERVER_TIME_ZONE = "Australia/Perth";
18
+
19
+ /**
20
+ * Writes one date-time as the "datetime-utc" storage writes it (UTC, without an offset) into a timestamptz column,
21
+ * and reads it back, on one connection.
22
+ * @param {import("knex").Knex} knex
23
+ * @param {Date} instant
24
+ */
25
+ function roundTrip(knex, instant) {
26
+ return knex.transaction(async (trx) => {
27
+ await trx.raw("CREATE TEMPORARY TABLE stamped (at timestamptz NOT NULL) ON COMMIT DROP");
28
+ await trx.raw("INSERT INTO stamped (at) VALUES (?)", [toDatabaseDateTimeUtc(instant)]);
29
+ const { rows: [{ TimeZone: session }] } = await trx.raw("SHOW TIME ZONE");
30
+ const { rows: [{ at, text }] } = await trx.raw("SELECT at, at::text AS text FROM stamped");
31
+ return { session, at, text };
32
+ });
33
+ }
34
+
35
+ test("the runtime's PostgreSQL sessions run in UTC on a server that is not, so a timestamptz keeps the instant written", {
36
+ skip: SKIP
37
+ }, async (context) => {
38
+ const previousOptions = process.env.PGOPTIONS;
39
+ process.env.PGOPTIONS = `-c TimeZone=${SERVER_TIME_ZONE}`;
40
+ context.after(() => {
41
+ if (previousOptions === undefined) {
42
+ delete process.env.PGOPTIONS;
43
+ } else {
44
+ process.env.PGOPTIONS = previousOptions;
45
+ }
46
+ });
47
+ const env = { ...process.env, DB_CLIENT: "pg" };
48
+ const serverDefault = knexFactory({
49
+ client: "pg",
50
+ connection: resolveKnexConnectionFromEnvironment(env, { client: "pg", defaultPort: 5432 }),
51
+ pool: { min: 0, max: 1 }
52
+ });
53
+ const { knex } = createDatabase({ driver: { DIALECT_ID: "pg" }, env });
54
+ context.after(() => Promise.all([serverDefault.destroy(), knex.destroy()]));
55
+ const instant = new Date("2026-11-08T04:05:06.789Z");
56
+
57
+ // Without the runtime's session setting, the server reads the UTC text in its own zone: eight hours early.
58
+ const shifted = await roundTrip(serverDefault, instant);
59
+ assert.equal(shifted.session, SERVER_TIME_ZONE);
60
+ assert.equal(instant.getTime() - shifted.at.getTime(), 8 * 60 * 60 * 1000);
61
+
62
+ const kept = await roundTrip(knex, instant);
63
+ assert.equal(kept.session, "UTC");
64
+ assert.equal(kept.at.toISOString(), instant.toISOString());
65
+ assert.equal(kept.text, "2026-11-08 04:05:06.789+00");
66
+ });