@jskit-ai/database-runtime 0.1.262 → 0.1.264

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,16 @@
1
+ # Changelog
2
+
3
+ ## Unreleased
4
+
5
+ - `./server/testDatabases` exports `createPackageTestKnex(t, { environment })`, the Knex a package's own data test runs on when it creates its tables in the test: an in-memory SQLite database, or on a `DB_CLIENT=pg` run a schema of the test's own in `TEST_DB_NAME` (on DB_HOST, DB_PORT, DB_USER and DB_PASSWORD), in UTC, with a pool of up to eight connections for concurrent transactions, dropped with everything in it after the test. It refuses `NODE_ENV=production`, and a pg run without `TEST_DB_NAME` fails (`TestDatabaseMissingError`).
6
+ - `listForeignKeys(knex)` (`./server/foreignKeyMigrations`) answers every foreign key of the connected database's tables in one statement, each with its table, its columns in order and the table and columns it references, on MySQL, MariaDB, PostgreSQL (the current schema) and SQLite (whose keys have no name). `testDatabaseRun` takes `checkDatabase(knex)`, run once on the prepared `TEST_DB_NAME` before any test file: its answer joins the summary and a throw fails the run.
7
+ - `@jskit-ai/database-runtime-mysql` and `@jskit-ai/database-runtime-postgres` take `@jskit-ai/database-runtime` as a required peer at the same version, where it was a dependency, as `@jskit-ai/resource-runtime` and `@jskit-ai/crud-core` do: the application declares it beside its driver (`npm install @jskit-ai/database-runtime @jskit-ai/database-runtime-mysql`) and holds its one copy. npm installs a required peer as it installed the dependency.
8
+ - **Breaking (data-access redesign).** `./shared/visibility` is removed (`applyVisibility`, `applyVisibilityOwners`, `DEFAULT_VISIBILITY_COLUMNS`, also re-exported from `./shared`): a table resource declares its `ownership`, and every handle statement is scoped by it. `./shared` and `./shared/repositoryOptions` no longer export `applyForUpdate`, `createWithTransaction`, `joinOrOpenTransaction`, `lockScopedRecordId`, `resolveQueryOptions`, `resolveRepoClient`, `resolveTransaction` and `runInTransaction`: a transaction is the table handle's `withTransaction(options, work)`, a row lock its `lock` or `lockMany`, and a call's client and transaction ride in the `options` it passes on. The Before | Now table in the existing-application migration guide maps each one.
9
+ - `./server/testDatabases` exports `requireTestDatabase(environment)`, which answers `TEST_DB_NAME` and throws `TestDatabaseMissingError` without it, so a data test fails instead of skipping, and `tableNames(knex)`, the tables of the test database.
10
+ - `retryLostInsertRace(options, run)` (from `./shared`) runs a transaction once more when its insert lost a race for a unique key: `run`, usually `() => repository.withTransaction(options, work)`, failed with a duplicate whose transaction rolled back in full, outside a caller's transaction (`isLostDuplicateInsertRace`). The database reports the duplicate only once the winner has committed, so the second run's lookup by the key finds the winner's row; a second duplicate is final, and inside a caller's transaction (`options.trx`) the error reaches the caller without a rerun. Apps wrote this retry by hand around each idempotent write.
11
+ - The migration configuration lists the application's `migrations/constraints/` only when the directory exists, as it already did for a package's: an application without one failed `db:migrate` and `jskit test --databases` with ENOENT (scandir `migrations/constraints`).
12
+ - `retryDeadlocks` (and so `withTransaction` and every handle write outside a transaction) also runs again a transaction that MariaDB failed as "Record has changed since last read" (errno 1020, `ER_CHECKREAD`): with `innodb_snapshot_isolation` on, the second of two writers whose filters matched one row fails so when it locks the row the first changed. json-rest-api has rolled the transaction back, so the new run reads a fresh snapshot, and a `patchWhere` or `deleteWhere` then leaves out the row that no longer matches. `isDeadlockError` recognises it.
13
+ - **Breaking.** `runtime.database` is `{ driver, knex }`: its `transactionManager`, with `inTransaction(work, { trx })`, and the `RuntimeTransactionManager` type are removed, so a transaction has one way in, the table handle's `withTransaction(options, work)`, which joins the caller's transaction and retries a deadlocked outermost one. Nothing in JSKIT called it. Code that opened a transaction with it opens it on a table's handle (or its owner's API) instead.
14
+ - A PostgreSQL session runs in UTC whatever the server's default time zone, as a MySQL connection does with `timezone: "Z"`: the runtime's Knex and the migration configuration (`createKnexMigrationConfig`) set `TimeZone` to UTC in the pool's `afterCreate`, and the test databases' own connections keep it. PostgreSQL reads a date-time written without an offset, as the `datetime-utc` storage writes it, in the session's zone, so on a server set to another zone a `timestamptz` value moved by that zone's offset.
15
+ - The test databases' `tableNames()` and the migration ledger read accept every name Knex gives PostgreSQL's client (`pg`, `postgres`, `postgresql`); `postgres` used to be read as MySQL.
16
+ - `retryDeadlocks` runs a transaction again only when the database chose it as a deadlock victim, never after a lock wait timed out (MySQL's 1205 and PostgreSQL's `55P03`): MySQL rolls back only the timed-out statement, so a rerun could repeat the transaction's earlier work. Behaviour is unchanged; tests now pin it.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@jskit-ai/database-runtime",
3
- "version": "0.1.262",
3
+ "version": "0.1.264",
4
4
  "type": "module",
5
5
  "scripts": {
6
6
  "test": "jskit test"
@@ -17,8 +17,7 @@
17
17
  "./shared/databaseClient": "./src/shared/databaseClient.js",
18
18
  "./shared/dialect": "./src/shared/dialect.js",
19
19
  "./shared/duplicateEntry": "./src/shared/duplicateEntry.js",
20
- "./shared/repositoryOptions": "./src/shared/repositoryOptions.js",
21
- "./shared/visibility": "./src/shared/visibility.js"
20
+ "./shared/repositoryOptions": "./src/shared/repositoryOptions.js"
22
21
  },
23
22
  "dependencies": {
24
23
  "knex": "^3.1.0"
@@ -64,15 +63,15 @@
64
63
  },
65
64
  {
66
65
  "subpath": "./server/foreignKeyMigrations",
67
- "summary": "Adds a single-column foreign key only when it is missing, refusing rows without a parent, for constraints migrations on fresh and already-migrated databases."
66
+ "summary": "Adds a single-column foreign key only when it is missing, refusing rows without a parent, for constraints migrations on fresh and already-migrated databases, and lists every foreign key the connected database holds."
68
67
  },
69
68
  {
70
69
  "subpath": "./server/testDatabases",
71
- "summary": "Gives each test file or browser worker running at once its own disposable copy of TEST_DB_NAME or BROWSER_TEST_DB_NAME, restored from a schema snapshot keyed by the migrations' names and contents (MySQL, MariaDB and PostgreSQL), and counts the statements a piece of work sends, for query-count guards."
70
+ "summary": "Gives each test file or browser worker running at once its own disposable copy of TEST_DB_NAME or BROWSER_TEST_DB_NAME, restored from a schema snapshot keyed by the migrations' names and contents (MySQL, MariaDB and PostgreSQL), counts the statements a piece of work sends, for query-count guards, and gives a package's own data test its Knex: SQLite in memory, or a PostgreSQL schema of its own on a DB_CLIENT=pg run (createPackageTestKnex)."
72
71
  },
73
72
  {
74
73
  "subpath": "./shared",
75
- "summary": "Exports shared repository helpers (transactions, record ids, JSON columns, visibility, duplicate-entry detection and date/time serialization)."
74
+ "summary": "Exports shared database helpers (deadlock retries, record ids, JSON columns, duplicate-entry detection and date/time serialization)."
76
75
  },
77
76
  {
78
77
  "subpath": "./server/types",
@@ -83,6 +82,6 @@
83
82
  }
84
83
  },
85
84
  "peerDependencies": {
86
- "@jskit-ai/kernel": "0.2.13"
85
+ "@jskit-ai/kernel": "0.2.15"
87
86
  }
88
87
  }
@@ -1,7 +1,6 @@
1
1
  import diagnosticsChannel from "node:diagnostics_channel";
2
2
  import { normalizeDatabaseClient, toKnexClientId } from "../shared/databaseClient.js";
3
- import { resolveKnexConnectionFromEnvironment } from "../shared/databaseConnection.js";
4
- import { createTransactionManager } from "../shared/transactionManager.js";
3
+ import { resolveKnexConnectionFromEnvironment, resolveKnexPool } from "../shared/databaseConnection.js";
5
4
  import { loadKnexFactory } from "./knexFactory.js";
6
5
 
7
6
  // The diagnostics channel that carries each statement the shared Knex sends (Knex's `query` event: its `sql`,
@@ -18,7 +17,8 @@ function resolveDriverDialectId(driver) {
18
17
  }
19
18
 
20
19
  /**
21
- * The database connection every feature shares: the installed driver, its Knex client and a transaction manager.
20
+ * The database connection every feature shares: the installed driver and its Knex client. Table work opens its
21
+ * transactions through its handle's withTransaction.
22
22
  * @param {object} dependencies
23
23
  * @param {import("./types.js").RuntimeDatabaseDriver} dependencies.driver
24
24
  * @param {import("@jskit-ai/kernel/server/types").RuntimeEnv} dependencies.env
@@ -30,6 +30,7 @@ function createDatabase({ driver, env }) {
30
30
  throw new Error(`DB_CLIENT="${configuredClient}" does not match installed database driver "${dialectId}".`);
31
31
  }
32
32
 
33
+ const pool = resolveKnexPool(dialectId);
33
34
  /** @type {import("./types.js").RuntimeKnex} */
34
35
  const knex = loadKnexFactory()({
35
36
  client: toKnexClientId(dialectId),
@@ -37,16 +38,15 @@ function createDatabase({ driver, env }) {
37
38
  client: dialectId,
38
39
  defaultPort: dialectId === "pg" ? 5432 : 3306,
39
40
  context: "database runtime"
40
- })
41
+ }),
42
+ ...(pool ? { pool } : {})
41
43
  });
42
44
  const statements = diagnosticsChannel.channel(RUNTIME_QUERY_CHANNEL);
43
45
  knex.on("query", (query) => statements.publish(query));
44
- const transactionManager = createTransactionManager({ knex });
45
46
 
46
47
  return Object.freeze({
47
48
  driver,
48
- knex,
49
- transactionManager
49
+ knex
50
50
  });
51
51
  }
52
52
 
@@ -1,7 +1,7 @@
1
1
  import path from "node:path";
2
2
  import { createKnexMigrationConfigFromApp } from "./knexMigrationConfig.js";
3
3
  import { loadKnexFactory } from "./knexFactory.js";
4
- import { runInTransaction } from "../shared/repositoryOptions.js";
4
+ import { retryDeadlocks } from "../shared/repositoryOptions.js";
5
5
 
6
6
  function normalizeMigrationResult(result) {
7
7
  const [batchValue, migrationValues] = Array.isArray(result) ? result : [];
@@ -44,13 +44,13 @@ async function runDatabaseSetup({ knex, seed, appRoot = process.cwd(), environme
44
44
  });
45
45
  }
46
46
 
47
- await runInTransaction(knex, async (transaction) => {
47
+ await retryDeadlocks(() => knex.transaction(async (transaction) => {
48
48
  await seed(Object.freeze({
49
49
  knex: transaction,
50
50
  appRoot: path.resolve(appRoot),
51
51
  environment
52
52
  }));
53
- });
53
+ }));
54
54
 
55
55
  return Object.freeze({
56
56
  ...migrationResult,
@@ -119,4 +119,67 @@ async function dropForeignKeyIfPresent(knex, key) {
119
119
  return true;
120
120
  }
121
121
 
122
- export { addMissingForeignKey, dropForeignKeyIfPresent, hasForeignKey };
122
+ // Every foreign key of the connected database's tables, one row per column: the key's name, its table and column,
123
+ // the referenced table and column, and the column's place in the key. Keys to another schema are left out.
124
+ const FOREIGN_KEY_COLUMNS = Object.freeze({
125
+ mysql: `SELECT CONSTRAINT_NAME AS name, TABLE_NAME AS tableName, COLUMN_NAME AS columnName,
126
+ REFERENCED_TABLE_NAME AS referencedTableName, REFERENCED_COLUMN_NAME AS referencedColumnName
127
+ FROM information_schema.KEY_COLUMN_USAGE
128
+ WHERE TABLE_SCHEMA = DATABASE() AND REFERENCED_TABLE_SCHEMA = DATABASE()
129
+ ORDER BY TABLE_NAME, CONSTRAINT_NAME, ORDINAL_POSITION`,
130
+ postgres: `SELECT c.conname AS name, t.relname AS "tableName", a.attname AS "columnName",
131
+ r.relname AS "referencedTableName", p.attname AS "referencedColumnName"
132
+ FROM pg_constraint c
133
+ JOIN pg_class t ON t.oid = c.conrelid
134
+ JOIN pg_class r ON r.oid = c.confrelid
135
+ CROSS JOIN LATERAL unnest(c.conkey, c.confkey) WITH ORDINALITY AS k(attnum, refnum, position)
136
+ JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = k.attnum
137
+ JOIN pg_attribute p ON p.attrelid = c.confrelid AND p.attnum = k.refnum
138
+ WHERE c.contype = 'f' AND t.relnamespace = to_regnamespace(current_schema()) AND r.relnamespace = t.relnamespace
139
+ ORDER BY t.relname, c.conname, k.position`,
140
+ // SQLite keys have no name: each is told apart by its table and id.
141
+ sqlite: `SELECT m.name || '#' || k.id AS name, m.name AS tableName, k."from" AS columnName,
142
+ k."table" AS referencedTableName, k."to" AS referencedColumnName
143
+ FROM sqlite_master m JOIN pragma_foreign_key_list(m.name) k
144
+ WHERE m.type = 'table'
145
+ ORDER BY m.name, k.id, k.seq`
146
+ });
147
+
148
+ // The rows of a raw statement, as each driver answers them.
149
+ const RAW_ROWS = Object.freeze({
150
+ mysql: ([rows]) => rows,
151
+ postgres: ({ rows }) => rows,
152
+ sqlite: (rows) => rows
153
+ });
154
+
155
+ /**
156
+ * The foreign keys the connected database holds, each with its table, its columns in order, and the table and
157
+ * columns it references. A SQLite key's name is "", since SQLite keeps none.
158
+ * @param {import("./types.js").RuntimeKnex} knex
159
+ * @returns {Promise<Array<{ name: string, tableName: string, columns: string[], referencedTableName: string, referencedColumns: string[] }>>}
160
+ */
161
+ async function listForeignKeys(knex) {
162
+ const detected = detectDialectFromClient(knex);
163
+ const dialect = detected.includes("sqlite") ? "sqlite" : detected;
164
+ if (!Object.hasOwn(FOREIGN_KEY_COLUMNS, dialect)) {
165
+ throw new Error(`Listing foreign keys does not support the ${detected || "unknown"} dialect.`);
166
+ }
167
+ const keys = new Map();
168
+ for (const row of RAW_ROWS[dialect](await knex.raw(FOREIGN_KEY_COLUMNS[dialect]))) {
169
+ const id = `${row.tableName}\u0000${row.name}`;
170
+ if (!keys.has(id)) {
171
+ keys.set(id, {
172
+ name: dialect === "sqlite" ? "" : row.name,
173
+ tableName: row.tableName,
174
+ columns: [],
175
+ referencedTableName: row.referencedTableName,
176
+ referencedColumns: []
177
+ });
178
+ }
179
+ keys.get(id).columns.push(row.columnName);
180
+ keys.get(id).referencedColumns.push(row.referencedColumnName);
181
+ }
182
+ return [...keys.values()];
183
+ }
184
+
185
+ export { addMissingForeignKey, dropForeignKeyIfPresent, hasForeignKey, listForeignKeys };
@@ -1,3 +1,4 @@
1
+ import { existsSync } from "node:fs";
1
2
  import path from "node:path";
2
3
  import { access } from "node:fs/promises";
3
4
  import { discoverInstalledPackages } from "@jskit-ai/kernel/server/support";
@@ -5,7 +6,7 @@ import {
5
6
  normalizeDatabaseClient,
6
7
  toKnexClientId
7
8
  } from "../shared/databaseClient.js";
8
- import { resolveKnexConnectionFromEnvironment } from "../shared/databaseConnection.js";
9
+ import { resolveKnexConnectionFromEnvironment, resolveKnexPool } from "../shared/databaseConnection.js";
9
10
 
10
11
  /**
11
12
  * The Knex configuration for migrating the database the application's environment names.
@@ -32,6 +33,9 @@ function createKnexMigrationConfig({
32
33
  }
33
34
  const dialectId = normalizeDatabaseClient(client);
34
35
  const resolvedMigrationsDirectory = path.resolve(appRoot, migrationsDirectory);
36
+ // An application needs no constraints/ until it has a constraint; Knex refuses a missing directory.
37
+ const applicationConstraintsDirectory = path.join(resolvedMigrationsDirectory, "constraints");
38
+ const pool = resolveKnexPool(dialectId);
35
39
 
36
40
  return {
37
41
  client: toKnexClientId(dialectId),
@@ -40,6 +44,7 @@ function createKnexMigrationConfig({
40
44
  defaultPort: dialectId === "pg" ? 5432 : 3306,
41
45
  context: "knex migrations"
42
46
  }),
47
+ ...(pool ? { pool } : {}),
43
48
  migrations: {
44
49
  // Packages create the tables application migrations build on; package constraints
45
50
  // can then reference application tables such as users.
@@ -47,7 +52,7 @@ function createKnexMigrationConfig({
47
52
  ...packageMigrationDirectories.map((directory) => path.resolve(directory)),
48
53
  resolvedMigrationsDirectory,
49
54
  ...packageConstraintDirectories.map((directory) => path.resolve(directory)),
50
- path.join(resolvedMigrationsDirectory, "constraints")
55
+ ...(existsSync(applicationConstraintsDirectory) ? [applicationConstraintsDirectory] : [])
51
56
  ],
52
57
  extension: "cjs",
53
58
  sortDirsSeparately: true
@@ -19,14 +19,17 @@
19
19
  // PROD_DB_NAME are never touched. Two runs at once need different database names, as before.
20
20
  //
21
21
  // A database test that guards a path's cost counts the statements the path sends with countQueries(), or, for a route
22
- // of a server the test booted, countRuntimeQueries().
23
- import { createHash } from "node:crypto";
22
+ // of a server the test booted, countRuntimeQueries(); expectFlatQueries() in @jskit-ai/resource-runtime/test compares
23
+ // those counts at a few rows and at many. A data test never skips: requireTestDatabase() fails it when TEST_DB_NAME is
24
+ // not set.
25
+ import { createHash, randomBytes } from "node:crypto";
24
26
  import diagnosticsChannel from "node:diagnostics_channel";
25
27
  import { existsSync, mkdirSync, mkdtempSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
26
28
  import os from "node:os";
27
29
  import path from "node:path";
28
30
  import { fileURLToPath } from "node:url";
29
- import { parseDatabaseUrl } from "../shared/databaseConnection.js";
31
+ import { parseDatabaseUrl, resolveKnexConnectionFromEnvironment, resolveKnexPool } from "../shared/databaseConnection.js";
32
+ import { detectDialectFromClient } from "../shared/dialect.js";
30
33
  import { RUNTIME_QUERY_CHANNEL } from "./database.js";
31
34
  import { loadKnexFactory } from "./knexFactory.js";
32
35
 
@@ -127,7 +130,8 @@ function knexOn(knexConfig, database, connectionOptions = {}) {
127
130
  return loadKnexFactory()({
128
131
  ...knexConfig,
129
132
  connection: { ...knexConfig.connection, ...connectionOptions, database },
130
- pool: { min: 0, max: 1 }
133
+ // The configuration's own pool settings stay, such as a PostgreSQL session's UTC time zone.
134
+ pool: { ...knexConfig.pool, min: 0, max: 1 }
131
135
  });
132
136
  }
133
137
 
@@ -178,7 +182,7 @@ async function testDatabaseSnapshotKey(knexConfig, { appRoot = process.cwd(), sc
178
182
  async function ledgerNames(knex, ledger, tables) {
179
183
  if (!tables.includes(ledger)) return [];
180
184
  const result = await knex.raw("SELECT name FROM ?? ORDER BY id", [ledger]);
181
- return (knex.client.config.client === "pg" ? result.rows : result[0]).map(({ name }) => name);
185
+ return (detectDialectFromClient(knex) === "postgres" ? result.rows : result[0]).map(({ name }) => name);
182
186
  }
183
187
 
184
188
  const mysql = {
@@ -478,8 +482,10 @@ function refusedPrivilege(error) {
478
482
  * @param {number} options.count How many test files run at once: at most the run's test budget.
479
483
  * @param {TestDatabaseSchema} [options.schema]
480
484
  * @param {string} [options.appRoot]
485
+ * @param {(knex: import("knex").Knex) => Promise<string>} [options.checkDatabase] Runs once on the prepared TEST_DB_NAME,
486
+ * before any test file: what it answers joins the summary, and a throw fails the run.
481
487
  */
482
- async function testDatabaseRun({ knexConfig, environment, count, schema, appRoot = process.cwd() }) {
488
+ async function testDatabaseRun({ knexConfig, environment, count, schema, appRoot = process.cwd(), checkDatabase }) {
483
489
  const oneAtATime = (reason) => ({
484
490
  concurrency: 1,
485
491
  env: {},
@@ -499,6 +505,7 @@ async function testDatabaseRun({ knexConfig, environment, count, schema, appRoot
499
505
  if (!refusedPrivilege(error)) throw error;
500
506
  return oneAtATime(`the database user may not prepare ${testDatabase} (${error.code})`);
501
507
  }
508
+ const checked = checkDatabase ? await withKnex(knexConfig, prepared.databases[0], checkDatabase) : "";
502
509
  // mkdtemp makes the directory for this user alone; the settings file holds the connection password.
503
510
  const directory = mkdtempSync(path.join(os.tmpdir(), "jskit-test-databases-"));
504
511
  const { client, connection, migrations } = knexConfig;
@@ -512,7 +519,7 @@ async function testDatabaseRun({ knexConfig, environment, count, schema, appRoot
512
519
  concurrency: prepared.databases.length,
513
520
  env: { JSKIT_TEST_DATABASES: directory },
514
521
  preload: TEST_DATABASE_FILE,
515
- summary: prepared.summary,
522
+ summary: `${prepared.summary}${checked ? `; ${checked}` : ""}`,
516
523
  cleanup: () => rmSync(directory, { recursive: true, force: true })
517
524
  };
518
525
  }
@@ -571,14 +578,109 @@ async function countRuntimeQueries(work) {
571
578
  }, work);
572
579
  }
573
580
 
581
+ /** A data test that has no test database: it fails rather than skip. */
582
+ class TestDatabaseMissingError extends Error {
583
+ constructor() {
584
+ super("TEST_DB_NAME is not set; data tests never skip.");
585
+ this.name = "TestDatabaseMissingError";
586
+ }
587
+ }
588
+
589
+ /**
590
+ * The test database a data test runs on. A data test never skips: without TEST_DB_NAME it throws
591
+ * TestDatabaseMissingError, so a run without a database fails instead of passing what it never ran.
592
+ * @param {import("@jskit-ai/kernel/server/types").RuntimeEnv} environment The test's environment, such as `process.env`.
593
+ * @returns {string}
594
+ */
595
+ function requireTestDatabase(environment) {
596
+ const name = String(environment.TEST_DB_NAME || "").trim();
597
+ if (!name) {
598
+ throw new TestDatabaseMissingError();
599
+ }
600
+ return name;
601
+ }
602
+
603
+ // A package test's PostgreSQL connections: enough for the concurrent transactions a lock or race test opens at once.
604
+ const PACKAGE_TEST_POOL = Object.freeze({ min: 0, max: 8 });
605
+
606
+ /**
607
+ * The Knex a package's own data test runs on, for a test that creates its own tables: an in-memory SQLite database,
608
+ * or, on a run whose DB_CLIENT is pg, a schema of the test's own in TEST_DB_NAME on the server DB_HOST, DB_PORT,
609
+ * DB_USER and DB_PASSWORD name, its sessions in UTC as an application's are. So one test file proves its data paths on
610
+ * both: the PostgreSQL run of the same files is the proof of the row locks, the many-row writes and the upserts there.
611
+ * The Knex is destroyed after the test, and the schema dropped with everything in it. A pg run without TEST_DB_NAME
612
+ * fails (TestDatabaseMissingError), as every data test does; `knex.client.config.client` tells a test which it got.
613
+ * @param {{ after(cleanup: () => unknown): unknown }} context The test's context (node:test's `t`).
614
+ * @param {{ environment: import("@jskit-ai/kernel/server/types").RuntimeEnv }} options The test's environment, such as
615
+ * `process.env`.
616
+ * @returns {Promise<import("knex").Knex>}
617
+ */
618
+ async function createPackageTestKnex(context, { environment }) {
619
+ const knexFactory = loadKnexFactory();
620
+ if (String(environment.DB_CLIENT || "").trim() !== "pg") {
621
+ const knex = knexFactory({ client: "better-sqlite3", connection: { filename: ":memory:" }, useNullAsDefault: true });
622
+ context.after(() => knex.destroy());
623
+ return knex;
624
+ }
625
+ if (environment.NODE_ENV === "production") {
626
+ throw new Error("Package data tests never run with NODE_ENV=production.");
627
+ }
628
+ const connection = {
629
+ ...resolveKnexConnectionFromEnvironment(environment, { client: "pg", defaultPort: 5432 }),
630
+ database: requireTestDatabase(environment)
631
+ };
632
+ const schema = `jskit_test_${process.pid}_${randomBytes(6).toString("hex")}`;
633
+ const admin = knexFactory({ client: "pg", connection, pool: { min: 0, max: 1 } });
634
+ try {
635
+ await admin.raw("CREATE SCHEMA ??", [schema]);
636
+ } catch (error) {
637
+ await admin.destroy();
638
+ throw error;
639
+ }
640
+ const knex = knexFactory({ client: "pg", connection, searchPath: [schema], pool: { ...resolveKnexPool("pg"), ...PACKAGE_TEST_POOL } });
641
+ context.after(async () => {
642
+ try {
643
+ await knex.destroy();
644
+ await admin.raw("DROP SCHEMA IF EXISTS ?? CASCADE", [schema]);
645
+ } finally {
646
+ await admin.destroy();
647
+ }
648
+ });
649
+ return knex;
650
+ }
651
+
652
+ /**
653
+ * The tables of the database `knex` is connected to, sorted: MySQL's and MariaDB's base tables, PostgreSQL's in the
654
+ * current schema, SQLite's own tables (its internal ones aside).
655
+ * @param {import("knex").Knex | import("knex").Knex.Transaction} knex
656
+ * @returns {Promise<string[]>}
657
+ */
658
+ async function tableNames(knex) {
659
+ const client = String(knex.client.config.client);
660
+ if (client.includes("sqlite")) {
661
+ const rows = await knex.raw("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'");
662
+ return rows.map(({ name }) => name).sort();
663
+ }
664
+ // Knex names PostgreSQL "pg", "postgres" or "postgresql".
665
+ if (detectDialectFromClient(knex) === "postgres") {
666
+ return postgres.tables(knex);
667
+ }
668
+ const [[{ name: database }]] = await knex.raw("SELECT DATABASE() AS name");
669
+ return mysql.tables(knex, database);
670
+ }
671
+
574
672
  export {
575
673
  RUN_SETTINGS_FILE,
674
+ TestDatabaseMissingError,
576
675
  countQueries,
676
+ createPackageTestKnex,
577
677
  countRuntimeQueries,
578
678
  matchesSnapshot,
579
679
  prepareTestDatabases,
580
680
  requireDisposableDatabase,
681
+ requireTestDatabase,
581
682
  restoreTestDatabase,
683
+ tableNames,
582
684
  testDatabaseNames,
583
685
  testDatabaseRun,
584
686
  testDatabaseSnapshotKey
@@ -2,8 +2,7 @@
2
2
 
3
3
  /** @typedef {ReturnType<typeof import("./database.js").createDatabase>} RuntimeDatabase */
4
4
  /** @typedef {import("knex").Knex} RuntimeKnex A Knex client: the one `runtime.database` shares, or one an application passes in. */
5
- /** @typedef {RuntimeDatabase["transactionManager"]} RuntimeTransactionManager The transaction manager `runtime.database` shares. */
6
- /** @typedef {import("knex").Knex.Transaction} RuntimeKnexTransaction A Knex transaction: the `trx` a repository or service call runs in, as the transaction manager hands it to its work. */
5
+ /** @typedef {import("knex").Knex.Transaction} RuntimeKnexTransaction A Knex transaction: the `trx` a repository or service call runs in. */
7
6
  /** @typedef {import("knex").Knex.QueryBuilder} RuntimeKnexQueryBuilder A Knex query that a hook or helper narrows. */
8
7
 
9
8
  /**
@@ -13,8 +12,6 @@
13
12
  * contract; `runtime.database` reads its dialect from it.
14
13
  * @typedef {object} RuntimeDatabaseDriver
15
14
  * @property {string} DIALECT_ID The dialect id: "mysql2" or "pg".
16
- * @property {(knex: RuntimeKnex, options?: { tableName?: string, idColumn?: string }) => Promise<object>} [introspectCrudTableSnapshot]
17
- * Reads a table's columns for CRUD generation; the MySQL driver has it.
18
15
  */
19
16
 
20
17
  export {};
@@ -141,9 +141,28 @@ function resolveKnexConnectionFromEnvironment(
141
141
  return connection;
142
142
  }
143
143
 
144
+ // Runs on each new PostgreSQL connection before the pool hands it out.
145
+ function setPostgresSessionToUtc(connection, done) {
146
+ connection.query("SET TIME ZONE 'UTC'", (error) => done(error, connection));
147
+ }
148
+
149
+ /**
150
+ * The Knex pool settings of a dialect: a PostgreSQL session runs in UTC whatever
151
+ * the server's default time zone, as a MySQL connection does with timezone "Z".
152
+ * PostgreSQL reads a date-time written without an offset (as the "datetime-utc"
153
+ * storage writes it) in the session's time zone, and formats timestamptz values
154
+ * and date functions in it, so a non-UTC session would shift stored instants.
155
+ * @param {string} client The dialect id: "mysql2" or "pg".
156
+ * @returns {{ afterCreate: (connection: any, done: (error: unknown, connection: any) => void) => void } | undefined}
157
+ */
158
+ function resolveKnexPool(client) {
159
+ return normalizeDatabaseClient(client) === "pg" ? { afterCreate: setPostgresSessionToUtc } : undefined;
160
+ }
161
+
144
162
  export {
145
163
  parseDatabaseUrl,
146
164
  resolveDatabaseClientFromEnvironment,
147
165
  resolveDatabaseConnectionFromEnvironment,
148
- resolveKnexConnectionFromEnvironment
166
+ resolveKnexConnectionFromEnvironment,
167
+ resolveKnexPool
149
168
  };
@@ -15,6 +15,13 @@ function isPostgresDuplicateEntryError(error) {
15
15
  return code === "23505";
16
16
  }
17
17
 
18
+ // better-sqlite3 names the constraint a duplicate broke: a unique index or the primary key.
19
+ const SQLITE_DUPLICATE_CODES = Object.freeze(new Set(["SQLITE_CONSTRAINT_UNIQUE", "SQLITE_CONSTRAINT_PRIMARYKEY"]));
20
+
21
+ function isSqliteDuplicateEntryError(error) {
22
+ return SQLITE_DUPLICATE_CODES.has(String(error?.code || "").trim().toUpperCase());
23
+ }
24
+
18
25
  function findDuplicateEntryError(error, { dialect = "", client = null } = {}) {
19
26
  const resolvedDialect =
20
27
  normalizeDialect(dialect) || (client ? normalizeDialect(detectDialectFromClient(client)) : "");
@@ -28,8 +35,10 @@ function findDuplicateEntryError(error, { dialect = "", client = null } = {}) {
28
35
  matches = isPostgresDuplicateEntryError(error);
29
36
  } else if (resolvedDialect === "mysql") {
30
37
  matches = isMysqlDuplicateEntryError(error);
38
+ } else if (resolvedDialect.includes("sqlite")) {
39
+ matches = isSqliteDuplicateEntryError(error);
31
40
  } else {
32
- matches = isMysqlDuplicateEntryError(error) || isPostgresDuplicateEntryError(error);
41
+ matches = isMysqlDuplicateEntryError(error) || isPostgresDuplicateEntryError(error) || isSqliteDuplicateEntryError(error);
33
42
  }
34
43
 
35
44
  if (matches) {
@@ -10,12 +10,7 @@ export {
10
10
  export { normalizeDialect, detectDialectFromClient } from "./dialect.js";
11
11
  export { normalizeDatabaseClient, toKnexClientId } from "./databaseClient.js";
12
12
  export { findDuplicateEntryError, isDuplicateEntryError, isLostDuplicateInsertRace } from "./duplicateEntry.js";
13
- export { DEFAULT_VISIBILITY_COLUMNS, applyVisibility, applyVisibilityOwners } from "./visibility.js";
14
13
  export {
15
- resolveTransaction,
16
- resolveQueryOptions,
17
- resolveRepoClient,
18
- applyForUpdate,
19
14
  parseJsonObject,
20
15
  stringifyJsonObject,
21
16
  normalizeNullableString,
@@ -25,8 +20,5 @@ export {
25
20
  toDbJson,
26
21
  isDeadlockError,
27
22
  retryDeadlocks,
28
- runInTransaction,
29
- createWithTransaction,
30
- joinOrOpenTransaction,
31
- lockScopedRecordId
23
+ retryLostInsertRace
32
24
  } from "./repositoryOptions.js";