@memberjunction/integration-engine 0.0.1 → 5.8.0

package/README.md

@@ -1,45 +1,159 @@
 # @memberjunction/integration-engine
 
-## ⚠️ IMPORTANT NOTICE ⚠️
+MemberJunction Integration Engine — orchestration, field mapping, and connector framework for synchronizing external systems with MJ entities.
+
+## Architecture
+
+The integration engine follows a pipeline architecture:
 
-**This package is created solely for the purpose of setting up OIDC (OpenID Connect) trusted publishing with npm.**
+```
+External System → Connector → FieldMappingEngine → MatchEngine → MJ Entity Persistence
+```
 
-This is **NOT** a functional package and contains **NO** code or functionality beyond the OIDC setup configuration.
+### Core Components
+
+| Component | Responsibility |
+|-----------|---------------|
+| `BaseIntegrationConnector` | Abstract base for external system connectors |
+| `ConnectorFactory` | Resolves connector instances via MJGlobal.ClassFactory |
+| `FieldMappingEngine` | Applies field-level transformations from external→MJ fields |
+| `MatchEngine` | Determines Create/Update/Delete/Skip for each record |
+| `WatermarkService` | Manages incremental sync watermarks |
+| `IntegrationOrchestrator` | Top-level coordinator that runs end-to-end sync |
+
+## Quick Start
+
+```typescript
+import { IntegrationOrchestrator } from '@memberjunction/integration-engine';
+
+const orchestrator = new IntegrationOrchestrator();
+const result = await orchestrator.RunSync(companyIntegrationID, contextUser, 'Manual');
+
+console.log(`Processed: ${result.RecordsProcessed}`);
+console.log(`Created: ${result.RecordsCreated}, Updated: ${result.RecordsUpdated}`);
+console.log(`Errors: ${result.RecordsErrored}`);
+```
+
+## Creating a Custom Connector
+
+```typescript
+import { RegisterClass } from '@memberjunction/global';
+import {
+  BaseIntegrationConnector,
+  ConnectionTestResult,
+  ExternalObjectSchema,
+  ExternalFieldSchema,
+  FetchContext,
+  FetchBatchResult,
+} from '@memberjunction/integration-engine';
 
-## Purpose
+@RegisterClass(BaseIntegrationConnector, 'HubSpotConnector')
+export class HubSpotConnector extends BaseIntegrationConnector {
+  async TestConnection(companyIntegration, contextUser): Promise<ConnectionTestResult> {
+    // Test HubSpot API connectivity
+  }
 
-This package exists to:
-1. Configure OIDC trusted publishing for the package name `@memberjunction/integration-engine`
-2. Enable secure, token-less publishing from CI/CD workflows
-3. Establish provenance for packages published under this name
+  async DiscoverObjects(companyIntegration, contextUser): Promise<ExternalObjectSchema[]> {
+    // Return available HubSpot objects (Contacts, Companies, Deals, etc.)
+  }
 
-## What is OIDC Trusted Publishing?
+  async DiscoverFields(companyIntegration, objectName, contextUser): Promise<ExternalFieldSchema[]> {
+    // Return fields for a specific HubSpot object
+  }
 
-OIDC trusted publishing allows package maintainers to publish packages directly from their CI/CD workflows without needing to manage npm access tokens. Instead, it uses OpenID Connect to establish trust between the CI/CD provider (like GitHub Actions) and npm.
+  async FetchChanges(ctx: FetchContext): Promise<FetchBatchResult> {
+    // Fetch changed records using HubSpot's API with watermark-based pagination
+  }
+}
+```
 
-## Setup Instructions
+## Transform Types
 
-To properly configure OIDC trusted publishing for this package:
+The `FieldMappingEngine` supports 9 transform types configured via JSON in the `TransformPipeline` field:
 
-1. Go to [npmjs.com](https://www.npmjs.com/) and navigate to your package settings
-2. Configure the trusted publisher (e.g., GitHub Actions)
-3. Specify the repository and workflow that should be allowed to publish
-4. Use the configured workflow to publish your actual package
+| Type | Description | Config |
+|------|-------------|--------|
+| `direct` | Pass-through with optional default | `{ DefaultValue?: unknown }` |
+| `regex` | Regex find/replace | `{ Pattern, Replacement, Flags? }` |
+| `split` | Split string, extract by index | `{ Delimiter, Index }` |
+| `combine` | Merge multiple fields | `{ SourceFields, Separator }` |
+| `lookup` | Case-insensitive value mapping | `{ Map, Default? }` |
+| `format` | Date/number formatting | `{ FormatString, FormatType }` |
+| `coerce` | Type conversion | `{ TargetType: 'string'|'number'|'boolean'|'date' }` |
+| `substring` | Extract portion of string | `{ Start, Length? }` |
+| `custom` | JavaScript expression | `{ Expression }` |
 
-## DO NOT USE THIS PACKAGE
+### Transform Pipeline Example
 
-This package is a placeholder for OIDC configuration only. It:
-- Contains no executable code
-- Provides no functionality
-- Should not be installed as a dependency
-- Exists only for administrative purposes
+```json
+[
+  { "Type": "regex", "Config": { "Pattern": "[^0-9]", "Replacement": "", "Flags": "g" } },
+  { "Type": "substring", "Config": { "Start": 0, "Length": 3 } }
+]
+```
 
-## More Information
+This pipeline strips non-numeric characters then extracts the first 3 digits (e.g., area code from phone number).
 
-For more details about npm's trusted publishing feature, see:
-- [npm Trusted Publishing Documentation](https://docs.npmjs.com/generating-provenance-statements)
-- [GitHub Actions OIDC Documentation](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect)
+### Error Handling
 
----
+Each transform step can specify `OnError`:
+- `"Fail"` (default) — throws, halting the record
+- `"Skip"` — skips the field entirely
+- `"Null"` — sets the field to null
 
-**Maintained for OIDC setup purposes only**
+## API Reference
+
+### Notifications
+
+After a sync completes or fails, an optional `onNotification` callback receives a `SyncNotification` with pre-formatted subject and body text suitable for email delivery:
+
+```typescript
+const result = await orchestrator.RunSync(
+    companyIntegrationID, contextUser, 'Scheduled',
+    undefined,  // onProgress
+    (notification) => {
+        if (notification.Severity !== 'Info') {
+            emailService.send({
+                to: 'ops@example.com',
+                subject: notification.Subject,
+                text: notification.Body,
+            });
+        }
+    }
+);
+```
+
+**SyncNotification** properties:
+- `Event`: `'SyncCompleted'` | `'SyncCompletedWithErrors'` | `'SyncFailed'`
+- `Severity`: `'Info'` | `'Warning'` | `'Error'`
+- `Subject` / `Body`: human-readable text ready for email
+- `Result`: the full `SyncResult` for programmatic access
+
+### IntegrationOrchestrator
+
+- `RunSync(companyIntegrationID, contextUser, triggerType?, onProgress?, onNotification?)` — Executes a full sync run
+
+### FieldMappingEngine
+
+- `Apply(records, fieldMaps, entityName)` — Maps external records to MJ entity fields
+
+### MatchEngine
+
+- `Resolve(records, entityMap, fieldMaps, contextUser)` — Resolves Create/Update/Delete/Skip
+
+### WatermarkService
+
+- `Load(entityMapID, contextUser)` — Loads current watermark
+- `Update(entityMapID, newValue, contextUser)` — Updates or creates watermark
+
+### ConnectorFactory
+
+- `Resolve(integration, sourceTypes)` — Creates connector instance via ClassFactory
+
+### BaseIntegrationConnector (abstract)
+
+- `TestConnection(companyIntegration, contextUser)` — Tests external system connectivity
+- `DiscoverObjects(companyIntegration, contextUser)` — Lists available external objects
+- `DiscoverFields(companyIntegration, objectName, contextUser)` — Lists fields on an object
+- `FetchChanges(ctx)` — Fetches a batch of changed records
+- `GetDefaultFieldMappings(objectName, entityName)` — Suggests default mappings

package/dist/BaseIntegrationConnector.d.ts

@@ -0,0 +1,142 @@
+import type { UserInfo } from '@memberjunction/core';
+import type { MJCompanyIntegrationEntity } from '@memberjunction/core-entities';
+import type { ExternalRecord, DefaultFieldMapping, SourceSchemaInfo } from './types.js';
+/** Result of testing a connection to an external system */
+export interface ConnectionTestResult {
+    /** Whether the connection was successful */
+    Success: boolean;
+    /** Human-readable status message */
+    Message: string;
+    /** Server or API version reported by the external system */
+    ServerVersion?: string;
+}
+/** Schema description of an object/table in an external system */
+export interface ExternalObjectSchema {
+    /** API name of the object (e.g., "Contact", "Account") */
+    Name: string;
+    /** Human-readable label */
+    Label: string;
+    /** Whether this object supports incremental sync via watermarks */
+    SupportsIncrementalSync: boolean;
+    /** Whether this object can be created/updated from MJ (push) */
+    SupportsWrite: boolean;
+}
+/** Schema description of a single field on an external object */
+export interface ExternalFieldSchema {
+    /** API name of the field */
+    Name: string;
+    /** Human-readable label */
+    Label: string;
+    /** Field data type in the external system */
+    DataType: string;
+    /** Whether the field is required */
+    IsRequired: boolean;
+    /** Whether the field is a unique identifier */
+    IsUniqueKey: boolean;
+    /** Whether the field is read-only */
+    IsReadOnly: boolean;
+}
+/** Context passed to FetchChanges for incremental data retrieval */
+export interface FetchContext {
+    /** The company integration entity providing connection details */
+    CompanyIntegration: MJCompanyIntegrationEntity;
+    /** External object name to fetch from */
+    ObjectName: string;
+    /** Current watermark value for incremental fetch, or null for full fetch */
+    WatermarkValue: string | null;
+    /** Maximum number of records to fetch in a single batch */
+    BatchSize: number;
+    /** User context for authorization */
+    ContextUser: UserInfo;
+}
+/** Result of a FetchChanges call, containing a batch of records */
+export interface FetchBatchResult {
+    /** Records retrieved in this batch */
+    Records: ExternalRecord[];
+    /** Whether there are more records to fetch after this batch */
+    HasMore: boolean;
+    /** Updated watermark value after this batch */
+    NewWatermarkValue?: string;
+}
+/** Configurable timeout values for connector operations */
+export interface OperationTimeouts {
+    /** Timeout for TestConnection in milliseconds. Default: 5000 */
+    TestConnectionMs: number;
+    /** Timeout for DiscoverObjects in milliseconds. Default: 10000 */
+    DiscoverObjectsMs: number;
+    /** Timeout for DiscoverFields in milliseconds. Default: 10000 */
+    DiscoverFieldsMs: number;
+    /** Timeout for FetchChanges in milliseconds. Default: 30000 */
+    FetchChangesMs: number;
+}
+/** Default timeout values for connector operations */
+export declare const DEFAULT_OPERATION_TIMEOUTS: OperationTimeouts;
+/**
+ * Wraps a promise with a timeout. Rejects with a timeout error if the
+ * promise does not resolve within the specified duration.
+ *
+ * @param promise - The promise to wrap
+ * @param timeoutMs - Timeout in milliseconds
+ * @param operationName - Name of the operation for error messaging
+ * @returns The result of the promise
+ * @throws Error if the operation times out
+ */
+export declare function WithTimeout<T>(promise: Promise<T>, timeoutMs: number, operationName: string): Promise<T>;
+/**
+ * Abstract base class for integration connectors.
+ * Each external system (HubSpot, Salesforce, etc.) implements this class
+ * to provide system-specific data access and discovery.
+ */
+export declare abstract class BaseIntegrationConnector {
+    /**
+     * Tests connectivity to the external system.
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Connection test result with success/failure and message
+     */
+    abstract TestConnection(companyIntegration: MJCompanyIntegrationEntity, contextUser: UserInfo): Promise<ConnectionTestResult>;
+    /**
+     * Discovers available objects/tables in the external system.
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Array of object schemas available for integration
+     */
+    abstract DiscoverObjects(companyIntegration: MJCompanyIntegrationEntity, contextUser: UserInfo): Promise<ExternalObjectSchema[]>;
+    /**
+     * Discovers fields on a specific external object.
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param objectName - Name of the external object to inspect
+     * @param contextUser - User context for authorization
+     * @returns Array of field schemas for the specified object
+     */
+    abstract DiscoverFields(companyIntegration: MJCompanyIntegrationEntity, objectName: string, contextUser: UserInfo): Promise<ExternalFieldSchema[]>;
+    /**
+     * Fetches a batch of changed records from the external system.
+     * Supports incremental fetching via watermarks.
+     * @param ctx - Context containing connection info, object name, and watermark
+     * @returns Batch of external records with pagination info
+     */
+    abstract FetchChanges(ctx: FetchContext): Promise<FetchBatchResult>;
+    /**
+     * Returns suggested default field mappings for an external object to MJ entity.
+     * Override in subclasses to provide intelligent defaults.
+     * @param _objectName - Name of the external object
+     * @param _entityName - Name of the target MJ entity
+     * @returns Array of default field mappings (empty by default)
+     */
+    GetDefaultFieldMappings(_objectName: string, _entityName: string): DefaultFieldMapping[];
+    /**
+     * Introspects the source system's schema — returns metadata about available
+     * objects, their fields, primary keys, and foreign key relationships.
+     * Used by the Schema Builder to generate local DDL.
+     *
+     * Default implementation builds SourceSchemaInfo from DiscoverObjects + DiscoverFields.
+     * Override in subclasses for richer metadata (e.g., FK relationships, type details).
+     *
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Full schema info for all source objects
+     */
+    IntrospectSchema(companyIntegration: MJCompanyIntegrationEntity, contextUser: UserInfo): Promise<SourceSchemaInfo>;
+}
+//# sourceMappingURL=BaseIntegrationConnector.d.ts.map

package/dist/BaseIntegrationConnector.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"BaseIntegrationConnector.d.ts","sourceRoot":"","sources":["../src/BaseIntegrationConnector.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,KAAK,EAAE,0BAA0B,EAAE,MAAM,+BAA+B,CAAC;AAChF,OAAO,KAAK,EAAE,cAAc,EAAE,mBAAmB,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAExF,2DAA2D;AAC3D,MAAM,WAAW,oBAAoB;IACjC,4CAA4C;IAC5C,OAAO,EAAE,OAAO,CAAC;IACjB,oCAAoC;IACpC,OAAO,EAAE,MAAM,CAAC;IAChB,4DAA4D;IAC5D,aAAa,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,kEAAkE;AAClE,MAAM,WAAW,oBAAoB;IACjC,0DAA0D;IAC1D,IAAI,EAAE,MAAM,CAAC;IACb,2BAA2B;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,mEAAmE;IACnE,uBAAuB,EAAE,OAAO,CAAC;IACjC,gEAAgE;IAChE,aAAa,EAAE,OAAO,CAAC;CAC1B;AAED,iEAAiE;AACjE,MAAM,WAAW,mBAAmB;IAChC,4BAA4B;IAC5B,IAAI,EAAE,MAAM,CAAC;IACb,2BAA2B;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,6CAA6C;IAC7C,QAAQ,EAAE,MAAM,CAAC;IACjB,oCAAoC;IACpC,UAAU,EAAE,OAAO,CAAC;IACpB,+CAA+C;IAC/C,WAAW,EAAE,OAAO,CAAC;IACrB,qCAAqC;IACrC,UAAU,EAAE,OAAO,CAAC;CACvB;AAED,oEAAoE;AACpE,MAAM,WAAW,YAAY;IACzB,kEAAkE;IAClE,kBAAkB,EAAE,0BAA0B,CAAC;IAC/C,yCAAyC;IACzC,UAAU,EAAE,MAAM,CAAC;IACnB,4EAA4E;IAC5E,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,2DAA2D;IAC3D,SAAS,EAAE,MAAM,CAAC;IAClB,qCAAqC;IACrC,WAAW,EAAE,QAAQ,CAAC;CACzB;AAED,mEAAmE;AACnE,MAAM,WAAW,gBAAgB;IAC7B,sCAAsC;IACtC,OAAO,EAAE,cAAc,EAAE,CAAC;IAC1B,+DAA+D;IAC/D,OAAO,EAAE,OAAO,CAAC;IACjB,+CAA+C;IAC/C,iBAAiB,CAAC,EAAE,MAAM,CAAC;CAC9B;AAED,2DAA2D;AAC3D,MAAM,WAAW,iBAAiB;IAC9B,gEAAgE;IAChE,gBAAgB,EAAE,MAAM,CAAC;IACzB,kEAAkE;IAClE,iBAAiB,EAAE,MAAM,CAAC;IAC1B,iEAAiE;IACjE,gBAAgB,EAAE,MAAM,CAAC;IACzB,+DAA+D;IAC/D,cAAc,EAAE,MAAM,CAAC;CAC1B;AAED,sDAAsD;AACtD,eAAO,MAAM,0BAA0B,EAAE,iBAKxC,CAAC;AAEF;;;;;;;;;GASG;AACH,wBAAsB,WAAW,CAAC,CAAC,EAC/B,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC,EACnB,SAAS,EAAE,MAAM,EACjB,aAAa,EAAE,MAAM,GACtB,OAAO,CAAC,CAAC,CAAC,CAiBZ;AAED;;;;GAIG;AACH,8BAAsB,wBAAwB;IAC1C;;;;;OAKG;aACa,cAAc,CAC1B,kBAAkB,EAAE,0BAA0B,EAC9C,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,oBAAoB,CAAC;IAEhC;;;;;OAKG;aACa,eAAe,CAC3B,kBAAkB,EAAE,0BAA0B,EAC9C,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,oBAAoB,EAAE,CAAC;IAElC;;;;;;OAMG;aACa,cAAc,CAC1B,kBAAkB,EAAE,0BAA0B,EAC9C,UAAU,EAAE,MAAM,EAClB,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,mBAAmB,EAAE,CAAC;IAEjC;;;;;OAKG;aACa,YAAY,CAAC,GAAG,EAAE,YAAY,GAAG,OAAO,CAAC,gBAAgB,CAAC;IAE1E;;;;;;OAMG;IACI,uBAAuB,CAAC,WAAW,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM,GAAG,mBAAmB,EAAE;IAI/F;;;;;;;;;;;OAWG;IACU,gBAAgB,CACzB,kBAAkB,EAAE,0BAA0B,EAC9C,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,gBAAgB,CAAC;CA6B/B"}

package/dist/BaseIntegrationConnector.js

@@ -0,0 +1,91 @@
+/** Default timeout values for connector operations */
+export const DEFAULT_OPERATION_TIMEOUTS = {
+    TestConnectionMs: 5000,
+    DiscoverObjectsMs: 10000,
+    DiscoverFieldsMs: 10000,
+    FetchChangesMs: 30000,
+};
+/**
+ * Wraps a promise with a timeout. Rejects with a timeout error if the
+ * promise does not resolve within the specified duration.
+ *
+ * @param promise - The promise to wrap
+ * @param timeoutMs - Timeout in milliseconds
+ * @param operationName - Name of the operation for error messaging
+ * @returns The result of the promise
+ * @throws Error if the operation times out
+ */
+export async function WithTimeout(promise, timeoutMs, operationName) {
+    let timeoutHandle;
+    const timeoutPromise = new Promise((_resolve, reject) => {
+        timeoutHandle = setTimeout(() => {
+            reject(new Error(`Operation '${operationName}' timed out after ${timeoutMs}ms`));
+        }, timeoutMs);
+    });
+    try {
+        const result = await Promise.race([promise, timeoutPromise]);
+        return result;
+    }
+    finally {
+        if (timeoutHandle !== undefined) {
+            clearTimeout(timeoutHandle);
+        }
+    }
+}
+/**
+ * Abstract base class for integration connectors.
+ * Each external system (HubSpot, Salesforce, etc.) implements this class
+ * to provide system-specific data access and discovery.
+ */
+export class BaseIntegrationConnector {
+    /**
+     * Returns suggested default field mappings for an external object to MJ entity.
+     * Override in subclasses to provide intelligent defaults.
+     * @param _objectName - Name of the external object
+     * @param _entityName - Name of the target MJ entity
+     * @returns Array of default field mappings (empty by default)
+     */
+    GetDefaultFieldMappings(_objectName, _entityName) {
+        return [];
+    }
+    /**
+     * Introspects the source system's schema — returns metadata about available
+     * objects, their fields, primary keys, and foreign key relationships.
+     * Used by the Schema Builder to generate local DDL.
+     *
+     * Default implementation builds SourceSchemaInfo from DiscoverObjects + DiscoverFields.
+     * Override in subclasses for richer metadata (e.g., FK relationships, type details).
+     *
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Full schema info for all source objects
+     */
+    async IntrospectSchema(companyIntegration, contextUser) {
+        const objects = await this.DiscoverObjects(companyIntegration, contextUser);
+        const result = { Objects: [] };
+        for (const obj of objects) {
+            const fields = await this.DiscoverFields(companyIntegration, obj.Name, contextUser);
+            result.Objects.push({
+                ExternalName: obj.Name,
+                ExternalLabel: obj.Label,
+                Fields: fields.map(f => ({
+                    Name: f.Name,
+                    Label: f.Label,
+                    SourceType: f.DataType,
+                    IsRequired: f.IsRequired,
+                    MaxLength: null,
+                    Precision: null,
+                    Scale: null,
+                    DefaultValue: null,
+                    IsPrimaryKey: f.IsUniqueKey,
+                    IsForeignKey: false,
+                    ForeignKeyTarget: null,
+                })),
+                PrimaryKeyFields: fields.filter(f => f.IsUniqueKey).map(f => f.Name),
+                Relationships: [],
+            });
+        }
+        return result;
+    }
+}
+//# sourceMappingURL=BaseIntegrationConnector.js.map

package/dist/BaseIntegrationConnector.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"BaseIntegrationConnector.js","sourceRoot":"","sources":["../src/BaseIntegrationConnector.ts"],"names":[],"mappings":"AA8EA,sDAAsD;AACtD,MAAM,CAAC,MAAM,0BAA0B,GAAsB;IACzD,gBAAgB,EAAE,IAAI;IACtB,iBAAiB,EAAE,KAAK;IACxB,gBAAgB,EAAE,KAAK;IACvB,cAAc,EAAE,KAAK;CACxB,CAAC;AAEF;;;;;;;;;GASG;AACH,MAAM,CAAC,KAAK,UAAU,WAAW,CAC7B,OAAmB,EACnB,SAAiB,EACjB,aAAqB;IAErB,IAAI,aAAwD,CAAC;IAE7D,MAAM,cAAc,GAAG,IAAI,OAAO,CAAQ,CAAC,QAAQ,EAAE,MAAM,EAAE,EAAE;QAC3D,aAAa,GAAG,UAAU,CAAC,GAAG,EAAE;YAC5B,MAAM,CAAC,IAAI,KAAK,CAAC,cAAc,aAAa,qBAAqB,SAAS,IAAI,CAAC,CAAC,CAAC;QACrF,CAAC,EAAE,SAAS,CAAC,CAAC;IAClB,CAAC,CAAC,CAAC;IAEH,IAAI,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC,OAAO,EAAE,cAAc,CAAC,CAAC,CAAC;QAC7D,OAAO,MAAM,CAAC;IAClB,CAAC;YAAS,CAAC;QACP,IAAI,aAAa,KAAK,SAAS,EAAE,CAAC;YAC9B,YAAY,CAAC,aAAa,CAAC,CAAC;QAChC,CAAC;IACL,CAAC;AACL,CAAC;AAED;;;;GAIG;AACH,MAAM,OAAgB,wBAAwB;IA4C1C;;;;;;OAMG;IACI,uBAAuB,CAAC,WAAmB,EAAE,WAAmB;QACnE,OAAO,EAAE,CAAC;IACd,CAAC;IAED;;;;;;;;;;;OAWG;IACI,KAAK,CAAC,gBAAgB,CACzB,kBAA8C,EAC9C,WAAqB;QAErB,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,eAAe,CAAC,kBAAkB,EAAE,WAAW,CAAC,CAAC;QAC5E,MAAM,MAAM,GAAqB,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;QAEjD,KAAK,MAAM,GAAG,IAAI,OAAO,EAAE,CAAC;YACxB,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,cAAc,CAAC,kBAAkB,EAAE,GAAG,CAAC,IAAI,EAAE,WAAW,CAAC,CAAC;YACpF,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC;gBAChB,YAAY,EAAE,GAAG,CAAC,IAAI;gBACtB,aAAa,EAAE,GAAG,CAAC,KAAK;gBACxB,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;oBACrB,IAAI,EAAE,CAAC,CAAC,IAAI;oBACZ,KAAK,EAAE,CAAC,CAAC,KAAK;oBACd,UAAU,EAAE,CAAC,CAAC,QAAQ;oBACtB,UAAU,EAAE,CAAC,CAAC,UAAU;oBACxB,SAAS,EAAE,IAAI;oBACf,SAAS,EAAE,IAAI;oBACf,KAAK,EAAE,IAAI;oBACX,YAAY,EAAE,IAAI;oBAClB,YAAY,EAAE,CAAC,CAAC,WAAW;oBAC3B,YAAY,EAAE,KAAK;oBACnB,gBAAgB,EAAE,IAAI;iBACzB,CAAC,CAAC;gBACH,gBAAgB,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;gBACpE,aAAa,EAAE,EAAE;aACpB,CAAC,CAAC;QACP,CAAC;QAED,OAAO,MAAM,CAAC;IAClB,CAAC;CACJ"}

package/dist/ConnectorFactory.d.ts

@@ -0,0 +1,30 @@
+import type { MJIntegrationEntity } from '@memberjunction/core-entities';
+import type { IIntegrationSourceType } from './entity-types.js';
+import { BaseIntegrationConnector } from './BaseIntegrationConnector.js';
+/**
+ * Factory for resolving integration connectors using MJGlobal.ClassFactory.
+ * Connectors register themselves via `@RegisterClass(BaseIntegrationConnector, 'DriverClassName')`.
+ */
+export declare class ConnectorFactory {
+    /**
+     * Resolves and instantiates the appropriate connector for an integration.
+     * Looks up the DriverClass from the integration's source type and uses
+     * MJGlobal.ClassFactory to create the connector instance.
+     *
+     * @param integration - The integration entity defining which external system to connect to
+     * @param sourceTypes - Array of available integration source type entities
+     * @returns An instance of the appropriate BaseIntegrationConnector subclass
+     * @throws Error if no source type matches or no connector is registered for the driver class
+     */
+    static Resolve(integration: MJIntegrationEntity, sourceTypes: IIntegrationSourceType[]): BaseIntegrationConnector;
+    /**
+     * Finds the DriverClass name from the source types that matches the integration's ClassName.
+     */
+    private static FindDriverClass;
+    /**
+     * Creates a connector instance via MJGlobal.ClassFactory.
+     * First verifies a registration exists to avoid falling back to the abstract base class.
+     */
+    private static CreateConnectorInstance;
+}
+//# sourceMappingURL=ConnectorFactory.d.ts.map

package/dist/ConnectorFactory.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"ConnectorFactory.d.ts","sourceRoot":"","sources":["../src/ConnectorFactory.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,+BAA+B,CAAC;AACzE,OAAO,KAAK,EAAE,sBAAsB,EAAE,MAAM,mBAAmB,CAAC;AAChE,OAAO,EAAE,wBAAwB,EAAE,MAAM,+BAA+B,CAAC;AAEzE;;;GAGG;AACH,qBAAa,gBAAgB;IACzB;;;;;;;;;OASG;WACW,OAAO,CACjB,WAAW,EAAE,mBAAmB,EAChC,WAAW,EAAE,sBAAsB,EAAE,GACtC,wBAAwB;IAK3B;;OAEG;IACH,OAAO,CAAC,MAAM,CAAC,eAAe;IAqB9B;;;OAGG;IACH,OAAO,CAAC,MAAM,CAAC,uBAAuB;CAoBzC"}

package/dist/ConnectorFactory.js

@@ -0,0 +1,50 @@
+import { MJGlobal } from '@memberjunction/global';
+import { BaseIntegrationConnector } from './BaseIntegrationConnector.js';
+/**
+ * Factory for resolving integration connectors using MJGlobal.ClassFactory.
+ * Connectors register themselves via `@RegisterClass(BaseIntegrationConnector, 'DriverClassName')`.
+ */
+export class ConnectorFactory {
+    /**
+     * Resolves and instantiates the appropriate connector for an integration.
+     * Looks up the DriverClass from the integration's source type and uses
+     * MJGlobal.ClassFactory to create the connector instance.
+     *
+     * @param integration - The integration entity defining which external system to connect to
+     * @param sourceTypes - Array of available integration source type entities
+     * @returns An instance of the appropriate BaseIntegrationConnector subclass
+     * @throws Error if no source type matches or no connector is registered for the driver class
+     */
+    static Resolve(integration, sourceTypes) {
+        const driverClass = ConnectorFactory.FindDriverClass(integration, sourceTypes);
+        return ConnectorFactory.CreateConnectorInstance(driverClass);
+    }
+    /**
+     * Finds the DriverClass name from the source types that matches the integration's ClassName.
+     */
+    static FindDriverClass(integration, sourceTypes) {
+        const className = integration.ClassName;
+        if (!className) {
+            throw new Error(`Integration "${integration.Name}" does not have a ClassName configured.`);
+        }
+        const sourceType = sourceTypes.find(st => st.DriverClass === className);
+        if (!sourceType) {
+            throw new Error(`No IntegrationSourceType found with DriverClass "${className}" for integration "${integration.Name}".`);
+        }
+        return sourceType.DriverClass;
+    }
+    /**
+     * Creates a connector instance via MJGlobal.ClassFactory.
+     * First verifies a registration exists to avoid falling back to the abstract base class.
+     */
+    static CreateConnectorInstance(driverClass) {
+        const registration = MJGlobal.Instance.ClassFactory.GetRegistration(BaseIntegrationConnector, driverClass);
+        if (!registration) {
+            throw new Error(`No connector registered for driver class "${driverClass}". ` +
+                `Ensure a class is decorated with @RegisterClass(BaseIntegrationConnector, '${driverClass}').`);
+        }
+        const instance = MJGlobal.Instance.ClassFactory.CreateInstance(BaseIntegrationConnector, driverClass);
+        return instance;
+    }
+}
+//# sourceMappingURL=ConnectorFactory.js.map

package/dist/ConnectorFactory.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"ConnectorFactory.js","sourceRoot":"","sources":["../src/ConnectorFactory.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,wBAAwB,CAAC;AAGlD,OAAO,EAAE,wBAAwB,EAAE,MAAM,+BAA+B,CAAC;AAEzE;;;GAGG;AACH,MAAM,OAAO,gBAAgB;IACzB;;;;;;;;;OASG;IACI,MAAM,CAAC,OAAO,CACjB,WAAgC,EAChC,WAAqC;QAErC,MAAM,WAAW,GAAG,gBAAgB,CAAC,eAAe,CAAC,WAAW,EAAE,WAAW,CAAC,CAAC;QAC/E,OAAO,gBAAgB,CAAC,uBAAuB,CAAC,WAAW,CAAC,CAAC;IACjE,CAAC;IAED;;OAEG;IACK,MAAM,CAAC,eAAe,CAC1B,WAAgC,EAChC,WAAqC;QAErC,MAAM,SAAS,GAAG,WAAW,CAAC,SAAS,CAAC;QACxC,IAAI,CAAC,SAAS,EAAE,CAAC;YACb,MAAM,IAAI,KAAK,CACX,gBAAgB,WAAW,CAAC,IAAI,yCAAyC,CAC5E,CAAC;QACN,CAAC;QAED,MAAM,UAAU,GAAG,WAAW,CAAC,IAAI,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,WAAW,KAAK,SAAS,CAAC,CAAC;QACxE,IAAI,CAAC,UAAU,EAAE,CAAC;YACd,MAAM,IAAI,KAAK,CACX,oDAAoD,SAAS,sBAAsB,WAAW,CAAC,IAAI,IAAI,CAC1G,CAAC;QACN,CAAC;QAED,OAAO,UAAU,CAAC,WAAW,CAAC;IAClC,CAAC;IAED;;;OAGG;IACK,MAAM,CAAC,uBAAuB,CAAC,WAAmB;QACtD,MAAM,YAAY,GAAG,QAAQ,CAAC,QAAQ,CAAC,YAAY,CAAC,eAAe,CAC/D,wBAAwB,EACxB,WAAW,CACd,CAAC;QAEF,IAAI,CAAC,YAAY,EAAE,CAAC;YAChB,MAAM,IAAI,KAAK,CACX,6CAA6C,WAAW,KAAK;gBAC7D,8EAA8E,WAAW,KAAK,CACjG,CAAC;QACN,CAAC;QAED,MAAM,QAAQ,GAAG,QAAQ,CAAC,QAAQ,CAAC,YAAY,CAAC,cAAc,CAC1D,wBAAwB,EACxB,WAAW,CACd,CAAC;QAEF,OAAO,QAAS,CAAC;IACrB,CAAC;CACJ"}

package/dist/FieldMappingEngine.d.ts

@@ -0,0 +1,94 @@
+import type { ICompanyIntegrationFieldMap } from './entity-types.js';
+import type { ExternalRecord, MappedRecord } from './types.js';
+/**
+ * Engine responsible for applying field-level mappings and transformations
+ * from external records to MJ entity fields.
+ */
+export declare class FieldMappingEngine {
+    /**
+     * Applies field mappings to a batch of external records, producing mapped records
+     * ready for match resolution and persistence.
+     *
+     * @param records - External records to map
+     * @param fieldMaps - Active field map entities defining source→destination mappings
+     * @param entityName - Target MJ entity name
+     * @returns Array of mapped records with transformed field values
+     */
+    Apply(records: ExternalRecord[], fieldMaps: ICompanyIntegrationFieldMap[], entityName: string): MappedRecord[];
+    /**
+     * Maps a single external record through all active field mappings.
+     */
+    private MapSingleRecord;
+    /**
+     * Applies a single field mapping, including the full transform pipeline.
+     * Returns undefined if the field should be skipped (OnError: Skip).
+     */
+    private ApplyFieldMapping;
+    /**
+     * Parses the JSON transform pipeline string into typed TransformStep objects.
+     */
+    private ParseTransformPipeline;
+    /**
+     * Executes a single transform step, applying the configured transformation
+     * and handling errors according to the step's OnError strategy.
+     */
+    private ExecuteTransformStep;
+    /**
+     * Dispatches to the appropriate transform handler based on step type.
+     */
+    private DispatchTransform;
+    /**
+     * Handles a transform error according to the OnError strategy.
+     */
+    private HandleTransformError;
+    /**
+     * Direct pass-through, applying default value if source is null/undefined.
+     */
+    private ApplyDirect;
+    /**
+     * Applies a regex replacement to a string value.
+     */
+    private ApplyRegex;
+    /**
+     * Splits a string value and extracts a part by index.
+     */
+    private ApplySplit;
+    /**
+     * Combines multiple source fields with a separator.
+     */
+    private ApplyCombine;
+    /**
+     * Performs a case-insensitive value lookup/mapping.
+     */
+    private ApplyLookup;
+    /**
+     * Applies date/number/string formatting.
+     */
+    private ApplyFormat;
+    /**
+     * Basic date formatting supporting ISO and common format tokens.
+     */
+    private FormatDate;
+    /**
+     * Coerces a value to the specified target type.
+     */
+    private ApplyCoerce;
+    /**
+     * Coerces a value to a number, throwing on NaN.
+     */
+    private CoerceToNumber;
+    /**
+     * Coerces a value to boolean, supporting common truthy/falsy strings.
+     */
+    private CoerceToBoolean;
+    /**
+     * Extracts a substring from a string value.
+     */
+    private ApplySubstring;
+    /**
+     * Evaluates a custom JavaScript expression.
+     * The expression has access to `value` (current field value) and `fields` (all record fields).
+     */
+    private ApplyCustom;
+}
+//# sourceMappingURL=FieldMappingEngine.d.ts.map

package/dist/FieldMappingEngine.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"FieldMappingEngine.d.ts","sourceRoot":"","sources":["../src/FieldMappingEngine.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,2BAA2B,EAAE,MAAM,mBAAmB,CAAC;AACrE,OAAO,KAAK,EAAE,cAAc,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAe/D;;;GAGG;AACH,qBAAa,kBAAkB;IAC3B;;;;;;;;OAQG;IACI,KAAK,CACR,OAAO,EAAE,cAAc,EAAE,EACzB,SAAS,EAAE,2BAA2B,EAAE,EACxC,UAAU,EAAE,MAAM,GACnB,YAAY,EAAE;IAKjB;;OAEG;IACH,OAAO,CAAC,eAAe;IAsBvB;;;OAGG;IACH,OAAO,CAAC,iBAAiB;IAgBzB;;OAEG;IACH,OAAO,CAAC,sBAAsB;IAY9B;;;OAGG;IACH,OAAO,CAAC,oBAAoB;IAe5B;;OAEG;IACH,OAAO,CAAC,iBAAiB;IA6BzB;;OAEG;IACH,OAAO,CAAC,oBAAoB;IAW5B;;OAEG;IACH,OAAO,CAAC,WAAW;IAOnB;;OAEG;IACH,OAAO,CAAC,UAAU;IAMlB;;OAEG;IACH,OAAO,CAAC,UAAU;IAMlB;;OAEG;IACH,OAAO,CAAC,YAAY;IAKpB;;OAEG;IACH,OAAO,CAAC,WAAW;IASnB;;OAEG;IACH,OAAO,CAAC,WAAW;IAYnB;;OAEG;IACH,OAAO,CAAC,UAAU;IAOlB;;OAEG;IACH,OAAO,CAAC,WAAW;IAenB;;OAEG;IACH,OAAO,CAAC,cAAc;IAQtB;;OAEG;IACH,OAAO,CAAC,eAAe;IAOvB;;OAEG;IACH,OAAO,CAAC,cAAc;IAQtB;;;OAGG;IACH,OAAO,CAAC,WAAW;CActB"}