@mastra/mssql 1.0.0-beta.11 → 1.0.0-beta.13

package/CHANGELOG.md

@@ -1,5 +1,109 @@
 # @mastra/mssql
 
+## 1.0.0-beta.13
+
+### Patch Changes
+
+- Fixed duplicate spans migration issue across all storage backends. When upgrading from older versions, existing duplicate (traceId, spanId) combinations in the spans table could prevent the unique constraint from being created. The migration deduplicates spans before adding the constraint. ([#12073](https://github.com/mastra-ai/mastra/pull/12073))
+
+  **Deduplication rules (in priority order):**
+  1. Keep completed spans (those with `endedAt` set) over incomplete spans
+  2. Among spans with the same completion status, keep the one with the newest `updatedAt`
+  3. Use `createdAt` as the final tiebreaker
+
+  **What changed:**
+  - Added `migrateSpans()` method to observability stores for manual migration
+  - Added `checkSpansMigrationStatus()` method to check if migration is needed
+  - All stores use optimized single-query deduplication to avoid memory issues on large tables
+
+  **Usage example:**
+
+  ```typescript
+  const observability = await storage.getStore('observability');
+  const status = await observability.checkSpansMigrationStatus();
+  if (status.needsMigration) {
+    const result = await observability.migrateSpans();
+    console.log(`Migration complete: ${result.duplicatesRemoved} duplicates removed`);
+  }
+  ```
+
+  Fixes #11840
+
+- Renamed MastraStorage to MastraCompositeStore for better clarity. The old MastraStorage name remains available as a deprecated alias for backward compatibility, but will be removed in a future version. ([#12093](https://github.com/mastra-ai/mastra/pull/12093))
+
+  **Migration:**
+
+  Update your imports and usage:
+
+  ```typescript
+  // Before
+  import { MastraStorage } from '@mastra/core/storage';
+
+  const storage = new MastraStorage({
+    id: 'composite',
+    domains: { ... }
+  });
+
+  // After
+  import { MastraCompositeStore } from '@mastra/core/storage';
+
+  const storage = new MastraCompositeStore({
+    id: 'composite',
+    domains: { ... }
+  });
+  ```
+
+  The new name better reflects that this is a composite storage implementation that routes different domains (workflows, traces, messages) to different underlying stores, avoiding confusion with the general "Mastra Storage" concept.
+
+- Updated dependencies [[`026b848`](https://github.com/mastra-ai/mastra/commit/026b8483fbf5b6d977be8f7e6aac8d15c75558ac), [`ffa553a`](https://github.com/mastra-ai/mastra/commit/ffa553a3edc1bd17d73669fba66d6b6f4ac10897)]:
+  - @mastra/core@1.0.0-beta.26
+
+## 1.0.0-beta.12
+
+### Patch Changes
+
+- Added new `listThreads` method for flexible thread filtering across all storage adapters. ([#11832](https://github.com/mastra-ai/mastra/pull/11832))
+
+  **New Features**
+  - Filter threads by `resourceId`, `metadata`, or both (with AND logic for metadata key-value pairs)
+  - All filter parameters are optional, allowing you to list all threads or filter as needed
+  - Full pagination and sorting support
+
+  **Example Usage**
+
+  ```typescript
+  // List all threads
+  const allThreads = await memory.listThreads({});
+
+  // Filter by resourceId only
+  const userThreads = await memory.listThreads({
+    filter: { resourceId: 'user-123' },
+  });
+
+  // Filter by metadata only
+  const supportThreads = await memory.listThreads({
+    filter: { metadata: { category: 'support' } },
+  });
+
+  // Filter by both with pagination
+  const filteredThreads = await memory.listThreads({
+    filter: {
+      resourceId: 'user-123',
+      metadata: { priority: 'high', status: 'open' },
+    },
+    orderBy: { field: 'updatedAt', direction: 'DESC' },
+    page: 0,
+    perPage: 20,
+  });
+  ```
+
+  **Security Improvements**
+  - Added validation to prevent SQL injection via malicious metadata keys
+  - Added pagination parameter validation to prevent integer overflow attacks
+
+- Updated dependencies [[`ed3e3dd`](https://github.com/mastra-ai/mastra/commit/ed3e3ddec69d564fe2b125e083437f76331f1283), [`6833c69`](https://github.com/mastra-ai/mastra/commit/6833c69607418d257750bbcdd84638993d343539), [`47b1c16`](https://github.com/mastra-ai/mastra/commit/47b1c16a01c7ffb6765fe1e499b49092f8b7eba3), [`3a76a80`](https://github.com/mastra-ai/mastra/commit/3a76a80284cb71a0faa975abb3d4b2a9631e60cd), [`8538a0d`](https://github.com/mastra-ai/mastra/commit/8538a0d232619bf55dad7ddc2a8b0ca77c679a87), [`9312dcd`](https://github.com/mastra-ai/mastra/commit/9312dcd1c6f5b321929e7d382e763d95fdc030f5)]:
+  - @mastra/core@1.0.0-beta.25
+
 ## 1.0.0-beta.11
 
 ### Patch Changes

package/dist/docs/README.md

@@ -28,4 +28,4 @@ docs/
 ## Version
 
 Package: @mastra/mssql
-Version: 1.0.0-beta.11
+Version: 1.0.0-beta.13

package/dist/docs/SKILL.md

@@ -5,7 +5,7 @@ description: Documentation for @mastra/mssql. Includes links to type definitions
 
 # @mastra/mssql Documentation
 
-> **Version**: 1.0.0-beta.11
+> **Version**: 1.0.0-beta.13
 > **Package**: @mastra/mssql
 
 ## Quick Navigation

package/dist/docs/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.0.0-beta.11",
+  "version": "1.0.0-beta.13",
   "package": "@mastra/mssql",
   "exports": {},
   "modules": {}

package/dist/index.cjs

@@ -340,15 +340,49 @@ ${columns}
         );
         const pkExists = Array.isArray(pkResult.recordset) && pkResult.recordset.length > 0;
         if (!pkExists) {
-          try {
-            const addPkSql = `ALTER TABLE ${getTableName({ indexName: tableName, schemaName: getSchemaName(this.schemaName) })} ADD CONSTRAINT [${pkConstraintName}] PRIMARY KEY ([traceId], [spanId])`;
-            await this.pool.request().query(addPkSql);
-          } catch (pkError) {
-            this.logger?.warn?.(`Failed to add composite primary key to spans table:`, pkError);
+          const duplicateInfo = await this.checkForDuplicateSpans();
+          if (duplicateInfo.hasDuplicates) {
+            const errorMessage = `
+===========================================================================
+MIGRATION REQUIRED: Duplicate spans detected in ${duplicateInfo.tableName}
+===========================================================================
+
+Found ${duplicateInfo.duplicateCount} duplicate (traceId, spanId) combinations.
+
+The spans table requires a unique constraint on (traceId, spanId), but your
+database contains duplicate entries that must be resolved first.
+
+To fix this, run the manual migration command:
+
+  npx mastra migrate
+
+This command will:
+  1. Remove duplicate spans (keeping the most complete/recent version)
+  2. Add the required unique constraint
+
+Note: This migration may take some time for large tables.
+===========================================================================
+`;
+            throw new error.MastraError({
+              id: storage.createStorageErrorId("MSSQL", "MIGRATION_REQUIRED", "DUPLICATE_SPANS"),
+              domain: error.ErrorDomain.STORAGE,
+              category: error.ErrorCategory.USER,
+              text: errorMessage
+            });
+          } else {
+            try {
+              const addPkSql = `ALTER TABLE ${getTableName({ indexName: tableName, schemaName: getSchemaName(this.schemaName) })} ADD CONSTRAINT [${pkConstraintName}] PRIMARY KEY ([traceId], [spanId])`;
+              await this.pool.request().query(addPkSql);
+            } catch (pkError) {
+              this.logger?.warn?.(`Failed to add composite primary key to spans table:`, pkError);
+            }
           }
         }
       }
     } catch (error$1) {
+      if (error$1 instanceof error.MastraError) {
+        throw error$1;
+      }
       throw new error.MastraError(
         {
           id: storage.createStorageErrorId("MSSQL", "CREATE_TABLE", "FAILED"),
@@ -390,6 +424,154 @@ ${columns}
       this.logger?.warn?.(`Failed to migrate spans table ${fullTableName}:`, error);
     }
   }
+  /**
+   * Deduplicates spans with the same (traceId, spanId) combination.
+   * This is needed for databases that existed before the unique constraint was added.
+   *
+   * Priority for keeping spans:
+   * 1. Completed spans (endedAt IS NOT NULL) over incomplete spans
+   * 2. Most recent updatedAt
+   * 3. Most recent createdAt (as tiebreaker)
+   *
+   * Note: This prioritizes migration completion over perfect data preservation.
+   * Old trace data may be lost, which is acceptable for this use case.
+   */
+  async deduplicateSpans() {
+    const fullTableName = getTableName({ indexName: storage.TABLE_SPANS, schemaName: getSchemaName(this.schemaName) });
+    try {
+      const duplicateCheck = await this.pool.request().query(`
+        SELECT TOP 1 1 as has_duplicates
+        FROM ${fullTableName}
+        GROUP BY [traceId], [spanId]
+        HAVING COUNT(*) > 1
+      `);
+      if (!duplicateCheck.recordset || duplicateCheck.recordset.length === 0) {
+        this.logger?.debug?.(`No duplicate spans found in ${fullTableName}`);
+        return;
+      }
+      this.logger?.info?.(`Duplicate spans detected in ${fullTableName}, starting deduplication...`);
+      const result = await this.pool.request().query(`
+        WITH RankedSpans AS (
+          SELECT *, ROW_NUMBER() OVER (
+            PARTITION BY [traceId], [spanId]
+            ORDER BY
+              CASE WHEN [endedAt] IS NOT NULL THEN 0 ELSE 1 END,
+              [updatedAt] DESC,
+              [createdAt] DESC
+          ) as rn
+          FROM ${fullTableName}
+        )
+        DELETE FROM RankedSpans WHERE rn > 1
+      `);
+      this.logger?.info?.(
+        `Deduplication complete: removed ${result.rowsAffected?.[0] ?? 0} duplicate spans from ${fullTableName}`
+      );
+    } catch (error) {
+      this.logger?.warn?.("Failed to deduplicate spans:", error);
+    }
+  }
+  /**
+   * Checks for duplicate (traceId, spanId) combinations in the spans table.
+   * Returns information about duplicates for logging/CLI purposes.
+   */
+  async checkForDuplicateSpans() {
+    const fullTableName = getTableName({ indexName: storage.TABLE_SPANS, schemaName: getSchemaName(this.schemaName) });
+    try {
+      const result = await this.pool.request().query(`
+        SELECT COUNT(*) as duplicate_count
+        FROM (
+          SELECT [traceId], [spanId]
+          FROM ${fullTableName}
+          GROUP BY [traceId], [spanId]
+          HAVING COUNT(*) > 1
+        ) duplicates
+      `);
+      const duplicateCount = result.recordset?.[0]?.duplicate_count ?? 0;
+      return {
+        hasDuplicates: duplicateCount > 0,
+        duplicateCount,
+        tableName: fullTableName
+      };
+    } catch (error) {
+      this.logger?.debug?.(`Could not check for duplicates: ${error}`);
+      return { hasDuplicates: false, duplicateCount: 0, tableName: fullTableName };
+    }
+  }
+  /**
+   * Checks if the PRIMARY KEY constraint on (traceId, spanId) already exists on the spans table.
+   */
+  async spansPrimaryKeyExists() {
+    const schemaPrefix = this.schemaName ? `${utils.parseSqlIdentifier(this.schemaName, "schema name")}_` : "";
+    const pkConstraintName = `${schemaPrefix}mastra_ai_spans_traceid_spanid_pk`;
+    const checkPkRequest = this.pool.request();
+    checkPkRequest.input("constraintName", pkConstraintName);
+    const pkResult = await checkPkRequest.query(
+      `SELECT 1 AS found FROM sys.key_constraints WHERE name = @constraintName`
+    );
+    return Array.isArray(pkResult.recordset) && pkResult.recordset.length > 0;
+  }
+  /**
+   * Manually run the spans migration to deduplicate and add the unique constraint.
+   * This is intended to be called from the CLI when duplicates are detected.
+   *
+   * @returns Migration result with status and details
+   */
+  async migrateSpans() {
+    const fullTableName = getTableName({ indexName: storage.TABLE_SPANS, schemaName: getSchemaName(this.schemaName) });
+    const pkExists = await this.spansPrimaryKeyExists();
+    if (pkExists) {
+      return {
+        success: true,
+        alreadyMigrated: true,
+        duplicatesRemoved: 0,
+        message: `Migration already complete. PRIMARY KEY constraint exists on ${fullTableName}.`
+      };
+    }
+    const duplicateInfo = await this.checkForDuplicateSpans();
+    if (duplicateInfo.hasDuplicates) {
+      this.logger?.info?.(
+        `Found ${duplicateInfo.duplicateCount} duplicate (traceId, spanId) combinations. Starting deduplication...`
+      );
+      await this.deduplicateSpans();
+    } else {
+      this.logger?.info?.(`No duplicate spans found.`);
+    }
+    const schemaPrefix = this.schemaName ? `${utils.parseSqlIdentifier(this.schemaName, "schema name")}_` : "";
+    const pkConstraintName = `${schemaPrefix}mastra_ai_spans_traceid_spanid_pk`;
+    const addPkSql = `ALTER TABLE ${fullTableName} ADD CONSTRAINT [${pkConstraintName}] PRIMARY KEY ([traceId], [spanId])`;
+    await this.pool.request().query(addPkSql);
+    return {
+      success: true,
+      alreadyMigrated: false,
+      duplicatesRemoved: duplicateInfo.duplicateCount,
+      message: duplicateInfo.hasDuplicates ? `Migration complete. Removed duplicates and added PRIMARY KEY constraint to ${fullTableName}.` : `Migration complete. Added PRIMARY KEY constraint to ${fullTableName}.`
+    };
+  }
+  /**
+   * Check migration status for the spans table.
+   * Returns information about whether migration is needed.
+   */
+  async checkSpansMigrationStatus() {
+    const fullTableName = getTableName({ indexName: storage.TABLE_SPANS, schemaName: getSchemaName(this.schemaName) });
+    const pkExists = await this.spansPrimaryKeyExists();
+    if (pkExists) {
+      return {
+        needsMigration: false,
+        hasDuplicates: false,
+        duplicateCount: 0,
+        constraintExists: true,
+        tableName: fullTableName
+      };
+    }
+    const duplicateInfo = await this.checkForDuplicateSpans();
+    return {
+      needsMigration: true,
+      hasDuplicates: duplicateInfo.hasDuplicates,
+      duplicateCount: duplicateInfo.duplicateCount,
+      constraintExists: false,
+      tableName: fullTableName
+    };
+  }
   /**
    * Alters table schema to add columns if they don't exist
    * @param tableName Name of the table
@@ -1265,28 +1447,76 @@ var MemoryMSSQL = class _MemoryMSSQL extends storage.MemoryStorage {
       );
     }
   }
-  async listThreadsByResourceId(args) {
-    const { resourceId, page = 0, perPage: perPageInput, orderBy } = args;
-    if (page < 0) {
+  async listThreads(args) {
+    const { page = 0, perPage: perPageInput, orderBy, filter } = args;
+    try {
+      this.validatePaginationInput(page, perPageInput ?? 100);
+    } catch (error$1) {
       throw new error.MastraError({
-        id: storage.createStorageErrorId("MSSQL", "LIST_THREADS_BY_RESOURCE_ID", "INVALID_PAGE"),
+        id: storage.createStorageErrorId("MSSQL", "LIST_THREADS", "INVALID_PAGE"),
         domain: error.ErrorDomain.STORAGE,
         category: error.ErrorCategory.USER,
-        text: "Page number must be non-negative",
-        details: {
-          resourceId,
-          page
-        }
+        text: error$1 instanceof Error ? error$1.message : "Invalid pagination parameters",
+        details: { page, ...perPageInput !== void 0 && { perPage: perPageInput } }
       });
     }
     const perPage = storage.normalizePerPage(perPageInput, 100);
+    try {
+      this.validateMetadataKeys(filter?.metadata);
+    } catch (error$1) {
+      throw new error.MastraError({
+        id: storage.createStorageErrorId("MSSQL", "LIST_THREADS", "INVALID_METADATA_KEY"),
+        domain: error.ErrorDomain.STORAGE,
+        category: error.ErrorCategory.USER,
+        text: error$1 instanceof Error ? error$1.message : "Invalid metadata key",
+        details: { metadataKeys: filter?.metadata ? Object.keys(filter.metadata).join(", ") : "" }
+      });
+    }
     const { offset, perPage: perPageForResponse } = storage.calculatePagination(page, perPageInput, perPage);
     const { field, direction } = this.parseOrderBy(orderBy);
     try {
-      const baseQuery = `FROM ${getTableName2({ indexName: storage.TABLE_THREADS, schemaName: getSchemaName2(this.schema) })} WHERE [resourceId] = @resourceId`;
+      const tableName = getTableName2({ indexName: storage.TABLE_THREADS, schemaName: getSchemaName2(this.schema) });
+      const whereClauses = [];
+      const params = {};
+      if (filter?.resourceId) {
+        whereClauses.push("[resourceId] = @resourceId");
+        params.resourceId = filter.resourceId;
+      }
+      if (filter?.metadata && Object.keys(filter.metadata).length > 0) {
+        let metadataIndex = 0;
+        for (const [key, value] of Object.entries(filter.metadata)) {
+          if (value !== null && typeof value === "object") {
+            throw new error.MastraError({
+              id: storage.createStorageErrorId("MSSQL", "LIST_THREADS", "INVALID_METADATA_VALUE"),
+              domain: error.ErrorDomain.STORAGE,
+              category: error.ErrorCategory.USER,
+              text: `Metadata filter value for key "${key}" must be a scalar type (string, number, boolean, or null), got ${Array.isArray(value) ? "array" : "object"}`,
+              details: { key, valueType: Array.isArray(value) ? "array" : "object" }
+            });
+          }
+          if (value === null) {
+            whereClauses.push(`JSON_VALUE(metadata, '$.${key}') IS NULL`);
+          } else {
+            const paramName = `metadata${metadataIndex}`;
+            whereClauses.push(`JSON_VALUE(metadata, '$.${key}') = @${paramName}`);
+            if (typeof value === "string") {
+              params[paramName] = value;
+            } else if (typeof value === "boolean") {
+              params[paramName] = value ? "true" : "false";
+            } else {
+              params[paramName] = String(value);
+            }
+          }
+          metadataIndex++;
+        }
+      }
+      const whereClause = whereClauses.length > 0 ? `WHERE ${whereClauses.join(" AND ")}` : "";
+      const baseQuery = `FROM ${tableName} ${whereClause}`;
       const countQuery = `SELECT COUNT(*) as count ${baseQuery}`;
       const countRequest = this.pool.request();
-      countRequest.input("resourceId", resourceId);
+      for (const [key, value] of Object.entries(params)) {
+        countRequest.input(key, value);
+      }
       const countResult = await countRequest.query(countQuery);
       const total = parseInt(countResult.recordset[0]?.count ?? "0", 10);
       if (total === 0) {
@@ -1303,7 +1533,9 @@ var MemoryMSSQL = class _MemoryMSSQL extends storage.MemoryStorage {
       const limitValue = perPageInput === false ? total : perPage;
       const dataQuery = `SELECT id, [resourceId], title, metadata, [createdAt], [updatedAt] ${baseQuery} ORDER BY ${orderByField} ${dir} OFFSET @offset ROWS FETCH NEXT @perPage ROWS ONLY`;
       const dataRequest = this.pool.request();
-      dataRequest.input("resourceId", resourceId);
+      for (const [key, value] of Object.entries(params)) {
+        dataRequest.input(key, value);
+      }
       dataRequest.input("offset", offset);
       if (limitValue > 2147483647) {
         dataRequest.input("perPage", sql__default.default.BigInt, limitValue);
@@ -1326,13 +1558,17 @@ var MemoryMSSQL = class _MemoryMSSQL extends storage.MemoryStorage {
         hasMore: perPageInput === false ? false : offset + perPage < total
       };
     } catch (error$1) {
+      if (error$1 instanceof error.MastraError && error$1.category === error.ErrorCategory.USER) {
+        throw error$1;
+      }
       const mastraError = new error.MastraError(
         {
-          id: storage.createStorageErrorId("MSSQL", "LIST_THREADS_BY_RESOURCE_ID", "FAILED"),
+          id: storage.createStorageErrorId("MSSQL", "LIST_THREADS", "FAILED"),
           domain: error.ErrorDomain.STORAGE,
           category: error.ErrorCategory.THIRD_PARTY,
           details: {
-            resourceId,
+            ...filter?.resourceId && { resourceId: filter.resourceId },
+            hasMetadataFilter: !!filter?.metadata,
             page
           }
         },
@@ -2214,6 +2450,22 @@ var ObservabilityMSSQL = class _ObservabilityMSSQL extends storage.Observability
   async dangerouslyClearAll() {
     await this.db.clearTable({ tableName: storage.TABLE_SPANS });
   }
+  /**
+   * Manually run the spans migration to deduplicate and add the unique constraint.
+   * This is intended to be called from the CLI when duplicates are detected.
+   *
+   * @returns Migration result with status and details
+   */
+  async migrateSpans() {
+    return this.db.migrateSpans();
+  }
+  /**
+   * Check migration status for the spans table.
+   * Returns information about whether migration is needed.
+   */
+  async checkSpansMigrationStatus() {
+    return this.db.checkSpansMigrationStatus();
+  }
   get tracingStrategy() {
     return {
       preferred: "batch-with-updates",
@@ -3613,7 +3865,7 @@ var WorkflowsMSSQL = class _WorkflowsMSSQL extends storage.WorkflowsStorage {
 var isPoolConfig = (config) => {
   return "pool" in config;
 };
-var MSSQLStore = class extends storage.MastraStorage {
+var MSSQLStore = class extends storage.MastraCompositeStore {
   pool;
   schema;
   isConnected = null;
@@ -3684,6 +3936,9 @@ var MSSQLStore = class extends storage.MastraStorage {
       await super.init();
     } catch (error$1) {
       this.isConnected = null;
+      if (error$1 instanceof error.MastraError) {
+        throw error$1;
+      }
       throw new error.MastraError(
         {
           id: storage.createStorageErrorId("MSSQL", "INIT", "FAILED"),