velocious 1.0.613 → 1.0.615

package/README.md

@@ -7,7 +7,7 @@
 * Connection-scoped advisory locks with automatic cleanup before pooled connections are reused or closed (see [docs/advisory-locks.md](docs/advisory-locks.md))
 * Built-in record auditing for model lifecycle changes (see [docs/auditing.md](docs/auditing.md))
 * Declarative state machines for models, with typed event methods generated into the base model (see [docs/state-machine.md](docs/state-machine.md))
-* Migrations for schema changes and UTC datetime storage (see [docs/database-migrations.md](docs/database-migrations.md))
+* Migrations for schema changes and UTC datetime storage, including recorded `changeTable` batches that combine operations into one `ALTER` on bulk-capable drivers (see [docs/database-migrations.md](docs/database-migrations.md) and [docs/change-table.md](docs/change-table.md))
 * Tenant-selected base-model and structure generation with one immutable, fail-closed physical database context; tenant-only model metadata initializes only after that context is active (see [docs/tenant-selected-database-generation.md](docs/tenant-selected-database-generation.md))
 * Read-only tenant migration deploy preflight with stable JSON output and fail-closed ledger reads (see [docs/tenant-migration-deploy-preflight.md](docs/tenant-migration-deploy-preflight.md))
 * External packages (engines) that contribute data models, frontend-model resources and migrations to a consuming app (see [docs/packages.md](docs/packages.md))
@@ -2625,7 +2625,7 @@ backgroundJobs: {
 }
 ```
 
-A job with no queue runs on `"default"`; a queue with no cap is unlimited. Caps are enforced through the durable per-key concurrency mechanism (the reserved `queue:<name>` key) and hold regardless of how many worker processes run. Changing a cap is reconciled against the existing backlog only when `background-jobs-main` starts (serialized across processes with a database advisory lock); `db:migrate`, `db:tenants:*`, and routine store/application initialization never reconcile the backlog and stay read-only regarding queued jobs. Scheduled jobs honor a job's `static queue` too.
+A job with no queue runs on `"default"`; a queue with no cap is unlimited. Caps are enforced through the durable per-key concurrency mechanism (the reserved `queue:<name>` key) and hold regardless of how many worker processes run. Changing a cap and rebuilding durable active counts happen only when `background-jobs-main` starts (serialized across processes with a database advisory lock and logged with database identifier/duration); `db:migrate`, `db:tenants:*`, and routine store/application initialization with an intact jobs table never adopt queued jobs or rebuild global concurrency counts. If schema repair must recreate a missing `background_jobs` table while the migration marker and concurrency table survive, it resets the now-orphaned active counts against that newly empty table. Scheduled jobs honor a job's `static queue` too.
 
 Set `priority` (default `0`) to dispatch a queue ahead of lower-priority ones regardless of enqueue order, so a small time-critical queue is never starved by a flood of low-priority work sharing a worker pool. Unlike Sidekiq's strict queue ordering, priority composes with the caps: a higher-priority queue already at its `maxConcurrent` is skipped and dispatch falls through to the next eligible job. See [docs/background-jobs.md](docs/background-jobs.md#queues-per-queue-concurrency-caps).
 

package/build/background-jobs/store.js

@@ -46,6 +46,7 @@ import {
 const MIGRATIONS_TABLE = "velocious_internal_migrations"
 const MIGRATION_SCOPE = "background_jobs"
 const MIGRATION_VERSION = "20250215000000"
+const SCHEMA_RECOVERY_PENDING_VERSION = "schema-recovery-pending"
 const EXECUTION_MODE_BACKFILL_MIGRATION_VERSION = "20260607131010"
 // Drops the redundant legacy `forked` boolean column and rewrites pooled rows to
 // persist `execution_mode = "pooled"` directly (retiring the pooled-as-forked
@@ -186,6 +187,13 @@ export default class BackgroundJobsStore extends BackgroundJobsAdapter {
   async reconcileQueueConcurrency() {
     if (this._queueConcurrencyReconciled) return
 
+    const databaseIdentifier = this.getDatabaseIdentifier()
+    const startedAtMs = Date.now()
+
+    await this.logger.info(() => [
+      "Starting background jobs queue-concurrency startup reconciliation",
+      {databaseIdentifier}
+    ])
     await this.ensureReady()
 
     await this._withDb(async (db) => {
@@ -206,6 +214,11 @@ export default class BackgroundJobsStore extends BackgroundJobsAdapter {
         await db.releaseAdvisoryLock(lockName)
       }
     })
+
+    await this.logger.info(() => [
+      "Completed background jobs queue-concurrency startup reconciliation",
+      {databaseIdentifier, durationMs: Date.now() - startedAtMs}
+    ])
   }
 
   /**
@@ -1514,24 +1527,29 @@ export default class BackgroundJobsStore extends BackgroundJobsAdapter {
     await this._ensureMigrationsTable(db)
 
     const alreadyApplied = await this._hasMigration(db)
+    const schemaRecoveryPending = await this._hasMigration(db, SCHEMA_RECOVERY_PENDING_VERSION)
+    const jobsTableExists = await db.tableExists(JOBS_TABLE)
 
     // Even when the migration row is present, the jobs table itself can have
     // been dropped underneath us by a transaction rollback in another caller
     // (DDL is transactional on SQLite/MSSQL). Verify the table physically
     // exists and recreate it when missing rather than trusting the migration
     // row alone, otherwise later callers fail with "no such table".
-    if (alreadyApplied && await db.tableExists(JOBS_TABLE)) {
+    if (alreadyApplied && jobsTableExists && !schemaRecoveryPending) {
       await this._ensureJobsTableColumns(db)
       await this._ensureIdempotencyKeysTable(db)
       await this._ensureMailDeliveryOperationsTable(db)
       await this._ensureScheduleKeysTable(db)
       await this._ensureConcurrencyTable(db)
       await this._ensureCountRevisionTable(db)
-      await this._reconcileConcurrency(db)
 
       return
     }
 
+    if (alreadyApplied && !schemaRecoveryPending) {
+      await this._recordMigration(db, SCHEMA_RECOVERY_PENDING_VERSION)
+    }
+
     await this._applyMigrations(db)
     await this._ensureJobsTableColumns(db)
     await this._ensureIdempotencyKeysTable(db)
@@ -1539,9 +1557,18 @@ export default class BackgroundJobsStore extends BackgroundJobsAdapter {
     await this._ensureScheduleKeysTable(db)
     await this._ensureConcurrencyTable(db)
     await this._ensureCountRevisionTable(db)
-    await this._reconcileConcurrency(db)
 
-    if (alreadyApplied) return
+    if (alreadyApplied) {
+      // The recreated jobs table is empty, but the surviving concurrency table
+      // can still count handoffs that disappeared with the dropped jobs table.
+      await this._reconcileConcurrency(db)
+      await db.delete({
+        tableName: MIGRATIONS_TABLE,
+        conditions: {key: this._migrationKey(SCHEMA_RECOVERY_PENDING_VERSION)}
+      })
+
+      return
+    }
 
     await this._recordMigration(db, MIGRATION_VERSION)
   }
@@ -2578,7 +2605,7 @@ export default class BackgroundJobsStore extends BackgroundJobsAdapter {
   }
 
   /**
-   * Rebuilds durable counts from active handoffs after startup.
+   * Rebuilds durable counts from active handoffs.
    * @param {import("../database/drivers/base.js").default} db - Database connection.
    * @returns {Promise<void>} - Resolves when reconciled.
    */

package/build/database/drivers/base.js

@@ -908,6 +908,25 @@ export default class VelociousDatabaseDriversBase {
     throw new Error("'type' not implemented")
   }
 
+  /**
+   * Whether this driver can combine unrelated alter-table operations into a
+   * single `ALTER TABLE` statement (Rails' `supports_bulk_alter`).
+   * @returns {boolean} - Whether bulk alter is supported.
+   */
+  supportsBulkAlter() {
+    return false
+  }
+
+  /**
+   * Whether a bulk `ALTER TABLE` statement can also carry `ADD INDEX` clauses.
+   * Only drivers that support this keep index adds inside the combined batch;
+   * the rest execute each index as its own statement.
+   * @returns {boolean} - Whether indexes can be added inside a bulk alter.
+   */
+  supportsBulkAlterIndexes() {
+    return false
+  }
+
   /**
    * Runs insert.
    * @param {InsertSqlArgsType} args - Options object.

package/build/database/drivers/mysql/index.js

@@ -345,6 +345,18 @@ export default class VelociousDatabaseDriversMysql extends Base{
    */
   getType() { return "mysql" }
 
+  /**
+   * Whether this driver supports combining operations into one bulk `ALTER`.
+   * @returns {boolean} - Whether bulk alter is supported.
+   */
+  supportsBulkAlter() { return true }
+
+  /**
+   * Whether the bulk `ALTER` can also carry `ADD INDEX` clauses.
+   * @returns {boolean} - Whether indexes can be added inside a bulk alter.
+   */
+  supportsBulkAlterIndexes() { return true }
+
   /**
    * Runs retryable database error.
    * @param {Error} error - Error instance.

package/build/database/drivers/mysql/sql/alter-table.js

@@ -26,13 +26,25 @@ export default class VelociousDatabaseConnectionDriversMysqlSqlAlterTable extend
     const indexes = this.tableData.getIndexes()
 
     if (indexes.length === 0) return sqls
-    if (sqls.length !== 1) throw new Error("Expected one MySQL ALTER TABLE statement when adding indexes")
+    if (sqls.length > 1) throw new Error("Expected one MySQL ALTER TABLE statement when adding indexes")
 
     const options = this.getOptions()
     let sql = sqls[0]
+    let needsIndexSeparator = true
+
+    if (sql === undefined) {
+      sql = `ALTER TABLE ${options.quoteTableName(this.tableData.getName())} `
+      needsIndexSeparator = false
+    }
 
     for (const index of indexes) {
-      sql += ", ADD"
+      if (needsIndexSeparator) {
+        sql += ", "
+      } else {
+        needsIndexSeparator = true
+      }
+
+      sql += "ADD"
 
       if (index.getUnique()) sql += " UNIQUE"
 

package/build/database/drivers/pgsql/index.js

@@ -203,6 +203,19 @@ export default class VelociousDatabaseDriversPgsql extends Base{
 
   getType() { return "pgsql" }
 
+  /**
+   * Whether this driver supports combining operations into one bulk `ALTER`.
+   * @returns {boolean} - Whether bulk alter is supported.
+   */
+  supportsBulkAlter() { return true }
+
+  /**
+   * Whether the bulk `ALTER` can also carry `ADD INDEX` clauses. PostgreSQL's
+   * `ALTER TABLE` cannot express index creation, so indexes stay standalone.
+   * @returns {boolean} - Whether indexes can be added inside a bulk alter.
+   */
+  supportsBulkAlterIndexes() { return false }
+
   /**
    * Runs query actual.
    * @param {string} sql - SQL string.

package/build/database/migration/change-table.js

@@ -0,0 +1,324 @@
+// @ts-check
+
+/**
+ * ChangeTableAddIndexArgsType type.
+ * @typedef {object} ChangeTableAddIndexArgsType
+ * @property {boolean} [ifNotExists] - Skip creation if the index already exists.
+ * @property {string} [name] - Explicit index name to use.
+ * @property {boolean} [unique] - Whether the index should be unique.
+ */
+
+/**
+ * ChangeTableRemoveIndexArgsType type.
+ * @typedef {object} ChangeTableRemoveIndexArgsType
+ * @property {string} [name] - Explicit index name to remove.
+ */
+
+/**
+ * ChangeTableRemoveReferenceArgsType type.
+ * @typedef {object} ChangeTableRemoveReferenceArgsType
+ * @property {string} [columnName] - Override the derived reference column name.
+ * @property {string} [indexName] - Explicit generated index name to remove.
+ */
+
+/**
+ * ChangeTableAddColumnOperationType type.
+ * @typedef {object} ChangeTableAddColumnOperationType
+ * @property {"addColumn"} type - Operation type.
+ * @property {string} columnName - Column name.
+ * @property {string} columnType - Column type.
+ * @property {import("../table-data/table-column.js").TableColumnArgsType | undefined} args - Column args.
+ */
+
+/**
+ * ChangeTableRemoveColumnOperationType type.
+ * @typedef {object} ChangeTableRemoveColumnOperationType
+ * @property {"removeColumn"} type - Operation type.
+ * @property {string} columnName - Column name.
+ */
+
+/**
+ * ChangeTableAddIndexOperationType type.
+ * @typedef {object} ChangeTableAddIndexOperationType
+ * @property {"addIndex"} type - Operation type.
+ * @property {Array<string | import("../table-data/table-column.js").default>} columns - Columns to index.
+ * @property {ChangeTableAddIndexArgsType | undefined} args - Index args.
+ */
+
+/**
+ * ChangeTableRemoveIndexOperationType type.
+ * @typedef {object} ChangeTableRemoveIndexOperationType
+ * @property {"removeIndex"} type - Operation type.
+ * @property {string | Array<string | import("../table-data/table-column.js").default>} nameOrColumns - Index name or columns.
+ * @property {ChangeTableRemoveIndexArgsType | undefined} args - Index args.
+ */
+
+/**
+ * ChangeTableAddReferenceOperationType type.
+ * @typedef {object} ChangeTableAddReferenceOperationType
+ * @property {"addReference"} type - Operation type.
+ * @property {string} referenceName - Reference name.
+ * @property {object | undefined} args - Reference args.
+ */
+
+/**
+ * ChangeTableRemoveReferenceOperationType type.
+ * @typedef {object} ChangeTableRemoveReferenceOperationType
+ * @property {"removeReference"} type - Operation type.
+ * @property {string} referenceName - Reference name.
+ * @property {ChangeTableRemoveReferenceArgsType | undefined} args - Reference args.
+ */
+
+/**
+ * ChangeTableRenameColumnOperationType type.
+ * @typedef {object} ChangeTableRenameColumnOperationType
+ * @property {"renameColumn"} type - Operation type.
+ * @property {string} oldColumnName - Previous column name.
+ * @property {string} newColumnName - New column name.
+ */
+
+/**
+ * ChangeTableChangeColumnNullOperationType type.
+ * @typedef {object} ChangeTableChangeColumnNullOperationType
+ * @property {"changeColumnNull"} type - Operation type.
+ * @property {string} columnName - Column name.
+ * @property {boolean} nullable - Whether the column becomes nullable.
+ */
+
+/**
+ * ChangeTableOperationType type.
+ * @typedef {ChangeTableAddColumnOperationType | ChangeTableRemoveColumnOperationType | ChangeTableAddIndexOperationType | ChangeTableRemoveIndexOperationType | ChangeTableAddReferenceOperationType | ChangeTableRemoveReferenceOperationType | ChangeTableRenameColumnOperationType | ChangeTableChangeColumnNullOperationType} ChangeTableOperationType
+ */
+
+/**
+ * Table-scoped recorder used by `migration.changeTable`. Each call records a
+ * single DDL operation synchronously; `changeTable` replays them after the
+ * callback completes so a failed callback executes zero recorded DDL.
+ */
+export default class VelociousDatabaseMigrationChangeTable {
+  /**
+   * Operations.
+   * @type {ChangeTableOperationType[]} */
+  _operations = []
+
+  /**
+   * Runs constructor.
+   * @param {object} args - Options object.
+   * @param {string} args.tableName - Table name.
+   */
+  constructor({tableName}) {
+    if (!tableName) throw new Error(`Invalid table name: ${tableName}`)
+
+    this._tableName = tableName
+  }
+
+  /**
+   * Runs get table name.
+   * @returns {string} - The table name.
+   */
+  getTableName() { return this._tableName }
+
+  /**
+   * Runs get operations.
+   * @returns {ChangeTableOperationType[]} - The recorded operations.
+   */
+  getOperations() { return this._operations }
+
+  /**
+   * Records a new column.
+   * @param {string} name - Column name.
+   * @param {string} type - Column type.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  column(name, type, args) {
+    this._operations.push({type: "addColumn", columnName: name, columnType: type, args})
+  }
+
+  /**
+   * Records a bigint column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  bigint(name, args) { this.column(name, "bigint", args) }
+
+  /**
+   * Records a blob column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  blob(name, args) { this.column(name, "blob", args) }
+
+  /**
+   * Records a boolean column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  boolean(name, args) { this.column(name, "boolean", args) }
+
+  /**
+   * Records a datetime column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  datetime(name, args) { this.column(name, "datetime", args) }
+
+  /**
+   * Records a decimal column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  decimal(name, args) { this.column(name, "decimal", args) }
+
+  /**
+   * Records an integer column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  integer(name, args) { this.column(name, "integer", args) }
+
+  /**
+   * Records a json column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  json(name, args) { this.column(name, "json", args) }
+
+  /**
+   * Records a string column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  string(name, args) { this.column(name, "string", args) }
+
+  /**
+   * Records a text column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  text(name, args) { this.column(name, "text", args) }
+
+  /**
+   * Records a tinyint column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  tinyint(name, args) { this.column(name, "tinyint", args) }
+
+  /**
+   * Records a uuid column.
+   * @param {string} name - Column name.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  uuid(name, args) { this.column(name, "uuid", args) }
+
+  /**
+   * Records created_at and updated_at datetime columns.
+   * @param {import("../table-data/table-column.js").TableColumnArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  timestamps(args) {
+    this.datetime("created_at", args)
+    this.datetime("updated_at", args)
+  }
+
+  /**
+   * Records a new index.
+   * @param {string | Array<string | import("../table-data/table-column.js").default>} columns - Column name or array of column names.
+   * @param {ChangeTableAddIndexArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  index(columns, args) {
+    const normalizedColumns = typeof columns == "string" ? [columns] : columns
+
+    this._operations.push({type: "addIndex", columns: normalizedColumns, args})
+  }
+
+  /**
+   * Records a reference column, index, and optional foreign key.
+   * @param {string} name - Reference name.
+   * @param {object} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  references(name, args) {
+    this._operations.push({type: "addReference", referenceName: name, args})
+  }
+
+  /**
+   * Alias for {@link references}.
+   * @param {string} name - Reference name.
+   * @param {object} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  belongsTo(name, args) { this.references(name, args) }
+
+  /**
+   * Records removal of one or more columns.
+   * @param {string[]} columnNames - Column names to remove.
+   * @returns {void} - No return value.
+   */
+  remove(...columnNames) {
+    for (const columnName of columnNames) {
+      this._operations.push({type: "removeColumn", columnName})
+    }
+  }
+
+  /**
+   * Records removal of an index.
+   * @param {string | Array<string | import("../table-data/table-column.js").default>} nameOrColumns - Index name or columns.
+   * @param {ChangeTableRemoveIndexArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  removeIndex(nameOrColumns, args) {
+    this._operations.push({type: "removeIndex", nameOrColumns, args})
+  }
+
+  /**
+   * Records removal of a reference column and its generated index and foreign keys.
+   * @param {string} name - Reference name.
+   * @param {ChangeTableRemoveReferenceArgsType} [args] - Options object.
+   * @returns {void} - No return value.
+   */
+  removeReferences(name, args) {
+    this._operations.push({type: "removeReference", referenceName: name, args})
+  }
+
+  /**
+   * Records removal of the created_at and updated_at columns.
+   * @returns {void} - No return value.
+   */
+  removeTimestamps() {
+    this.remove("created_at", "updated_at")
+  }
+
+  /**
+   * Records a column rename.
+   * @param {string} oldColumnName - Previous column name.
+   * @param {string} newColumnName - New column name.
+   * @returns {void} - No return value.
+   */
+  rename(oldColumnName, newColumnName) {
+    this._operations.push({type: "renameColumn", oldColumnName, newColumnName})
+  }
+
+  /**
+   * Records a change to a column's nullability.
+   * @param {string} columnName - Column name.
+   * @param {boolean} nullable - Whether the column becomes nullable.
+   * @returns {void} - No return value.
+   */
+  changeNull(columnName, nullable) {
+    this._operations.push({type: "changeColumnNull", columnName, nullable})
+  }
+}