@travetto/model-sql 8.0.0-alpha.9 → 8.0.1

package/README.md

@@ -13,18 +13,38 @@ npm install @travetto/model-sql
 yarn add @travetto/model-sql
 ```
 
-The current SQL client support stands at:
-   *  [MySQL Model Service](https://github.com/travetto/travetto/tree/main/module/model-mysql#readme "MySQL backing for the travetto model module, with real-time modeling support for SQL schemas.") - MySQL 8.6+
+This module provides the core SQL foundation for [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations.") datastores. The current SQL client implementations include:
+   *  [MySQL Model Service](https://github.com/travetto/travetto/tree/main/module/model-mysql#readme "MySQL backing for the travetto model module, with real-time modeling support for SQL schemas.") - MySQL 8.0+
    *  [PostgreSQL Model Service](https://github.com/travetto/travetto/tree/main/module/model-postgres#readme "PostgreSQL backing for the travetto model module, with real-time modeling support for SQL schemas.") - Postgres 14+
-   *  [SQLite Model Service](https://github.com/travetto/travetto/tree/main/module/model-sqlite#readme "SQLite backing for the travetto model module, with real-time modeling support for SQL schemas.") - (bettersqlite 8.0+)
-   *  `SQL Server` - Currently unsupported
-   *  `Oracle` - Currently unsupported
+   *  [SQLite Model Service](https://github.com/travetto/travetto/tree/main/module/model-sqlite#readme "SQLite backing for the travetto model module, with real-time modeling support for SQL schemas.") - SQLite (Node native `DatabaseSync`)
 
 **Note**: Wider client support will roll out as usage increases.
 
-## Assumed Behavior
-The [SQL Model Service](https://github.com/travetto/travetto/tree/main/module/model-sql#readme "SQL backing for the travetto model module, with real-time modeling support for SQL schemas.") works quite a bit different than the average [Object Relationship Mapping](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) in that it makes assertions about how data is stored in the database.  The primary goal of the [SQL](https://en.wikipedia.org/wiki/SQL) support is not to handle every scenario that a relational database can provide, but to integrate with the [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations.") structure, while leverage relational datastores to the best of their abilities. 
+## Assumed Behavior & Schema Design
+The [SQL Model Service](https://github.com/travetto/travetto/tree/main/module/model-sql#readme "SQL backing for the travetto model module, with real-time modeling support for SQL schemas.") works quite a bit differently than the average [Object Relationship Mapping](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) in that it makes assertions about how data is stored in the database. The primary goal of the [SQL](https://en.wikipedia.org/wiki/SQL) support is not to handle every legacy relational database scenario, but to integrate with the [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations.") structure while leveraging relational datastores to the best of their abilities. 
 
-The primary difference is around unique identifiers, and how parent/child relationships are managed.  In a normal database primary keys could be composite values between various fields.  For example a unique identifier could be a combination of `date` + `orderNumber` + `customerNumber`.  This is perfectly normal in a relational model, but [SQL Model Service](https://github.com/travetto/travetto/tree/main/module/model-sql#readme "SQL backing for the travetto model module, with real-time modeling support for SQL schemas.") assumes unique identifiers ([UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)) as 32-character hexadecimal values.  In addition to these unique values, the parent's identifier is required in all children values.  This allows for some fairly optimized querying, updates, and deletions on changes. 
+The primary design maps each model class to a single table where simple fields (primitives, dates, enums) are mapped to individual columns, and complex fields (objects and arrays) are serialized and stored as native JSON document columns. Every table requires a primary key column (`id`), and indices are compiled directly to standard database columns or SQL functional expressions over JSON paths.
 
-What this translates to, is that the framework here dictates the final schema, and doesn't support adapting to existing relational data stores. In greenfield projects, this is not an issue, but will most likely preclude its use in adapting to existing relational data sets.
+In development mode, storage modifications are applied dynamically in real time to match model definitions, minimizing manual database migrations.
+
+## Transactions
+Transaction state is tracked seamlessly using [Async Context](https://github.com/travetto/travetto/tree/main/module/context#readme "Async-aware state management, maintaining context across asynchronous calls."). Methods can be wrapped using the [Transactional](https://github.com/travetto/travetto/tree/main/module/model-sql/src/connection.ts#L18) decorator to run operations within a managed transaction.
+
+Supported transaction modes include:
+   *  `required` - Joins an active transaction if one exists, or starts a new top-level transaction.
+   *  `isolated` - Begins a SAVEPOINT / nested transaction isolated from the surrounding context.
+   *  `force` - Always creates a separate SAVEPOINT for nested operations.
+
+## Bulk Operations
+Bulk operations (`processBulk`) batch insert, delete, and update statements for high performance. Bulk updates utilize standard ANSI SQL `CASE ... WHEN` constructs to update multiple records in a single database query across all SQL engines.
+
+## Supported Features
+All SQL model service implementations derive from [BaseSQLModelService](https://github.com/travetto/travetto/tree/main/module/model-sql/src/service.ts#L68) and support:
+   *  [Bulk](https://github.com/travetto/travetto/tree/main/module/model/src/types/bulk.ts#L60)
+   *  [CRUD](https://github.com/travetto/travetto/tree/main/module/model/src/types/crud.ts#L10)
+   *  [Expiry](https://github.com/travetto/travetto/tree/main/module/model/src/types/expiry.ts#L10)
+   *  [Indexed](https://github.com/travetto/travetto/tree/main/module/model-indexed/src/types/service.ts#L21)
+   *  [Query Crud](https://github.com/travetto/travetto/tree/main/module/model-query/src/types/crud.ts#L11)
+   *  [Facet](https://github.com/travetto/travetto/tree/main/module/model-query/src/types/facet.ts#L14)
+   *  [Suggest](https://github.com/travetto/travetto/tree/main/module/model-query/src/types/suggest.ts#L12)
+   *  [Query](https://github.com/travetto/travetto/tree/main/module/model-query/src/types/query.ts#L10)

package/__index__.ts

@@ -1,7 +1,5 @@
-export * from './src/config.ts';
+export * from './src/connection.ts';
+export * from './src/dialect.ts';
+export * from './src/schema.ts';
 export * from './src/service.ts';
-export * from './src/connection/base.ts';
-export * from './src/connection/decorator.ts';
-export * from './src/dialect/base.ts';
-export * from './src/util.ts';
-export * from './src/types.ts';
+export * from './src/types.ts';

package/package.json

@@ -1,41 +1,45 @@
 {
   "name": "@travetto/model-sql",
-  "version": "8.0.0-alpha.9",
-  "type": "module",
+  "version": "8.0.1",
   "description": "SQL backing for the travetto model module, with real-time modeling support for SQL schemas.",
   "keywords": [
-    "sql",
     "data-modeling",
-    "real-time",
     "model",
+    "real-time",
+    "sql",
     "travetto",
     "typescript"
   ],
   "homepage": "https://travetto.io",
   "license": "MIT",
   "author": {
-    "email": "travetto.framework@gmail.com",
-    "name": "Travetto Framework"
+    "name": "Travetto Framework",
+    "email": "travetto.framework@gmail.com"
+  },
+  "repository": {
+    "url": "git+https://github.com/travetto/travetto.git",
+    "directory": "module/model-sql"
   },
   "files": [
     "__index__.ts",
     "src",
     "support"
   ],
+  "type": "module",
   "main": "__index__.ts",
-  "repository": {
-    "url": "git+https://github.com/travetto/travetto.git",
-    "directory": "module/model-sql"
+  "publishConfig": {
+    "access": "public"
   },
   "dependencies": {
-    "@travetto/config": "^8.0.0-alpha.9",
-    "@travetto/context": "^8.0.0-alpha.9",
-    "@travetto/model": "^8.0.0-alpha.9",
-    "@travetto/model-query": "^8.0.0-alpha.9"
+    "@travetto/config": "^8.0.1",
+    "@travetto/context": "^8.0.1",
+    "@travetto/model": "^8.0.1",
+    "@travetto/model-indexed": "^8.0.1",
+    "@travetto/model-query": "^8.0.1"
   },
   "peerDependencies": {
-    "@travetto/cli": "^8.0.0-alpha.14",
-    "@travetto/test": "^8.0.0-alpha.9"
+    "@travetto/cli": "^8.0.1",
+    "@travetto/test": "^8.0.1"
   },
   "peerDependenciesMeta": {
     "@travetto/cli": {
@@ -47,8 +51,5 @@
   },
   "travetto": {
     "displayName": "SQL Model Service"
-  },
-  "publishConfig": {
-    "access": "public"
   }
 }

package/src/connection.ts

@@ -0,0 +1,218 @@
+import { type AsyncContext, AsyncContextValue } from '@travetto/context';
+import { ModelRegistryIndex, type ModelType } from '@travetto/model';
+import { type AsyncMethodDescriptor, type Class, castTo, Util } from '@travetto/runtime';
+
+import type { AbstractANSI99Dialect } from './dialect.ts';
+import { SQLModelSchemaUtil } from './schema.ts';
+import type { TableContext } from './types.ts';
+
+export type TransactionType = 'required' | 'isolated' | 'force';
+
+export interface ConnectionAware {
+  connection: SQLConnection;
+}
+
+/**
+ * Decorator to ensure a method runs inside a database transaction
+ */
+export function Transactional(mode: TransactionType = 'required') {
+  return function <Target extends ConnectionAware>(target: Target, property: string, descriptor: AsyncMethodDescriptor<Target>): void {
+    const originalMethod = descriptor.value!;
+    descriptor.value = function (...args: unknown[]): ReturnType<typeof originalMethod> {
+      return this.connection.runWithTransaction(mode, () => originalMethod.call(this, ...args));
+    };
+  };
+}
+
+/**
+ * Base abstract connection manager for SQL Model services.
+ * Uses @travetto/context to track active client connections and transaction state.
+ */
+export abstract class SQLConnection<
+  C = unknown,
+  F extends { namespace?: string; database?: string } = { namespace?: string; database?: string }
+> {
+  isolatedTransactions = true;
+  nestedTransactions = true;
+
+  abstract readonly dialect: AbstractANSI99Dialect;
+
+  readonly context: AsyncContext;
+
+  #activeConnection = new AsyncContextValue<C>(this, { failIfUnbound: { read: false } });
+  #activeTransaction = new AsyncContextValue<boolean>(this, { failIfUnbound: { read: false } });
+
+  constructor(context: AsyncContext) {
+    this.context = context;
+  }
+
+  abstract config: F;
+
+  getContext<T extends ModelType>(modelClass: Class<T>): TableContext<T> {
+    let tableName = ModelRegistryIndex.getStoreName(modelClass);
+    if (this.config.namespace) {
+      tableName = `${this.config.namespace}_${tableName}`;
+    }
+    return { tableName, ...SQLModelSchemaUtil.getSchemaContext(modelClass) };
+  }
+
+  get active(): C | undefined {
+    return this.#activeConnection.get();
+  }
+
+  get activeTransaction(): boolean {
+    return !!this.#activeTransaction.get();
+  }
+
+  /**
+   * Initializes the connection
+   */
+  abstract init(): Promise<void> | void;
+
+  /**
+   * Acquires a client connection
+   */
+  abstract acquire(): Promise<C>;
+
+  /**
+   * Releases a client connection
+   */
+  abstract release(connection: C): void;
+
+  /**
+   * Executes a SQL query
+   */
+  abstract execute<Type = unknown>(query: string, values?: unknown[]): Promise<{ count: number; records: Type[] }>;
+
+  /**
+   * Runs an operation with an active connection (allocating one if not already active)
+   */
+  async runWithConnection<Result>(operation: () => Promise<Result>): Promise<Result> {
+    if (this.active) {
+      return await operation();
+    }
+
+    return this.context.run(async () => {
+      let connection: C | undefined;
+      try {
+        connection = await this.acquire();
+        this.#activeConnection.set(connection);
+        return await operation();
+      } finally {
+        if (connection) {
+          this.release(connection);
+        }
+      }
+    });
+  }
+
+  /**
+   * Iterates with an active connection
+   */
+  async *iterateWithConnection<Result>(operation: () => AsyncIterable<Result>): AsyncIterable<Result> {
+    if (this.active) {
+      yield* operation();
+      return;
+    }
+
+    const self = castTo<SQLConnection<C>>(this);
+    yield* this.context.iterate(async function* () {
+      let connection: C | undefined;
+      try {
+        connection = await self.acquire();
+        self.#activeConnection.set(connection);
+        yield* operation();
+      } finally {
+        if (connection) {
+          self.release(connection);
+        }
+      }
+    });
+  }
+
+  /**
+   * Runs an operation within a database transaction context
+   */
+  async runWithTransaction<Result>(mode: TransactionType, operation: () => Promise<Result>): Promise<Result> {
+    if (this.activeTransaction) {
+      if (mode === 'isolated' || mode === 'force') {
+        const transactionId = mode === 'isolated' ? `tx${Util.uuid()}` : undefined;
+        try {
+          await this.startTransaction(transactionId);
+          const result = await operation();
+          await this.commitTransaction(transactionId);
+          return result;
+        } catch (error) {
+          try {
+            await this.rollbackTransaction(transactionId);
+          } catch {}
+          throw error;
+        }
+      } else {
+        return await operation();
+      }
+    } else {
+      return this.runWithConnection(async () => {
+        this.#activeTransaction.set(true);
+        if (this.isolatedTransactions) {
+          await this.execute(this.dialect.transactionStatements.isolate);
+        }
+        await this.execute(this.dialect.transactionStatements.begin);
+        try {
+          const result = await operation();
+          await this.execute(this.dialect.transactionStatements.commit);
+          return result;
+        } catch (error) {
+          try {
+            await this.execute(this.dialect.transactionStatements.rollback);
+          } catch {}
+          throw error;
+        } finally {
+          this.#activeTransaction.set(false);
+        }
+      });
+    }
+  }
+
+  /**
+   * Starts a transaction or SAVEPOINT
+   */
+  async startTransaction(transactionId?: string): Promise<void> {
+    if (transactionId) {
+      if (this.nestedTransactions) {
+        await this.execute(this.dialect.transactionStatements.beginNested, [transactionId]);
+      }
+    } else {
+      if (this.isolatedTransactions) {
+        await this.execute(this.dialect.transactionStatements.isolate);
+      }
+      await this.execute(this.dialect.transactionStatements.begin);
+    }
+  }
+
+  /**
+   * Commits the transaction or release SAVEPOINT
+   */
+  async commitTransaction(transactionId?: string): Promise<void> {
+    if (transactionId) {
+      if (this.nestedTransactions) {
+        await this.execute(this.dialect.transactionStatements.commitNested, [transactionId]);
+      }
+    } else {
+      await this.execute(this.dialect.transactionStatements.commit);
+    }
+  }
+
+  /**
+   * Rolls back the transaction or SAVEPOINT
+   */
+  async rollbackTransaction(transactionId?: string): Promise<void> {
+    if (transactionId) {
+      if (this.nestedTransactions) {
+        await this.execute(this.dialect.transactionStatements.rollbackNested, [transactionId]);
+      }
+    } else {
+      await this.execute(this.dialect.transactionStatements.rollback);
+    }
+  }
+}