@memberjunction/integration-engine 0.0.1 → 5.9.0

package/README.md

@@ -1,45 +1,159 @@
 # @memberjunction/integration-engine
 
-## ⚠️ IMPORTANT NOTICE ⚠️
+MemberJunction Integration Engine — orchestration, field mapping, and connector framework for synchronizing external systems with MJ entities.
+
+## Architecture
+
+The integration engine follows a pipeline architecture:
 
-**This package is created solely for the purpose of setting up OIDC (OpenID Connect) trusted publishing with npm.**
+```
+External System → Connector → FieldMappingEngine → MatchEngine → MJ Entity Persistence
+```
 
-This is **NOT** a functional package and contains **NO** code or functionality beyond the OIDC setup configuration.
+### Core Components
+
+| Component | Responsibility |
+|-----------|---------------|
+| `BaseIntegrationConnector` | Abstract base for external system connectors |
+| `ConnectorFactory` | Resolves connector instances via MJGlobal.ClassFactory |
+| `FieldMappingEngine` | Applies field-level transformations from external→MJ fields |
+| `MatchEngine` | Determines Create/Update/Delete/Skip for each record |
+| `WatermarkService` | Manages incremental sync watermarks |
+| `IntegrationOrchestrator` | Top-level coordinator that runs end-to-end sync |
+
+## Quick Start
+
+```typescript
+import { IntegrationOrchestrator } from '@memberjunction/integration-engine';
+
+const orchestrator = new IntegrationOrchestrator();
+const result = await orchestrator.RunSync(companyIntegrationID, contextUser, 'Manual');
+
+console.log(`Processed: ${result.RecordsProcessed}`);
+console.log(`Created: ${result.RecordsCreated}, Updated: ${result.RecordsUpdated}`);
+console.log(`Errors: ${result.RecordsErrored}`);
+```
+
+## Creating a Custom Connector
+
+```typescript
+import { RegisterClass } from '@memberjunction/global';
+import {
+  BaseIntegrationConnector,
+  ConnectionTestResult,
+  ExternalObjectSchema,
+  ExternalFieldSchema,
+  FetchContext,
+  FetchBatchResult,
+} from '@memberjunction/integration-engine';
 
-## Purpose
+@RegisterClass(BaseIntegrationConnector, 'HubSpotConnector')
+export class HubSpotConnector extends BaseIntegrationConnector {
+  async TestConnection(companyIntegration, contextUser): Promise<ConnectionTestResult> {
+    // Test HubSpot API connectivity
+  }
 
-This package exists to:
-1. Configure OIDC trusted publishing for the package name `@memberjunction/integration-engine`
-2. Enable secure, token-less publishing from CI/CD workflows
-3. Establish provenance for packages published under this name
+  async DiscoverObjects(companyIntegration, contextUser): Promise<ExternalObjectSchema[]> {
+    // Return available HubSpot objects (Contacts, Companies, Deals, etc.)
+  }
 
-## What is OIDC Trusted Publishing?
+  async DiscoverFields(companyIntegration, objectName, contextUser): Promise<ExternalFieldSchema[]> {
+    // Return fields for a specific HubSpot object
+  }
 
-OIDC trusted publishing allows package maintainers to publish packages directly from their CI/CD workflows without needing to manage npm access tokens. Instead, it uses OpenID Connect to establish trust between the CI/CD provider (like GitHub Actions) and npm.
+  async FetchChanges(ctx: FetchContext): Promise<FetchBatchResult> {
+    // Fetch changed records using HubSpot's API with watermark-based pagination
+  }
+}
+```
 
-## Setup Instructions
+## Transform Types
 
-To properly configure OIDC trusted publishing for this package:
+The `FieldMappingEngine` supports 9 transform types configured via JSON in the `TransformPipeline` field:
 
-1. Go to [npmjs.com](https://www.npmjs.com/) and navigate to your package settings
-2. Configure the trusted publisher (e.g., GitHub Actions)
-3. Specify the repository and workflow that should be allowed to publish
-4. Use the configured workflow to publish your actual package
+| Type | Description | Config |
+|------|-------------|--------|
+| `direct` | Pass-through with optional default | `{ DefaultValue?: unknown }` |
+| `regex` | Regex find/replace | `{ Pattern, Replacement, Flags? }` |
+| `split` | Split string, extract by index | `{ Delimiter, Index }` |
+| `combine` | Merge multiple fields | `{ SourceFields, Separator }` |
+| `lookup` | Case-insensitive value mapping | `{ Map, Default? }` |
+| `format` | Date/number formatting | `{ FormatString, FormatType }` |
+| `coerce` | Type conversion | `{ TargetType: 'string'|'number'|'boolean'|'date' }` |
+| `substring` | Extract portion of string | `{ Start, Length? }` |
+| `custom` | JavaScript expression | `{ Expression }` |
 
-## DO NOT USE THIS PACKAGE
+### Transform Pipeline Example
 
-This package is a placeholder for OIDC configuration only. It:
-- Contains no executable code
-- Provides no functionality
-- Should not be installed as a dependency
-- Exists only for administrative purposes
+```json
+[
+  { "Type": "regex", "Config": { "Pattern": "[^0-9]", "Replacement": "", "Flags": "g" } },
+  { "Type": "substring", "Config": { "Start": 0, "Length": 3 } }
+]
+```
 
-## More Information
+This pipeline strips non-numeric characters then extracts the first 3 digits (e.g., area code from phone number).
 
-For more details about npm's trusted publishing feature, see:
-- [npm Trusted Publishing Documentation](https://docs.npmjs.com/generating-provenance-statements)
-- [GitHub Actions OIDC Documentation](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect)
+### Error Handling
 
----
+Each transform step can specify `OnError`:
+- `"Fail"` (default) — throws, halting the record
+- `"Skip"` — skips the field entirely
+- `"Null"` — sets the field to null
 
-**Maintained for OIDC setup purposes only**
+## API Reference
+
+### Notifications
+
+After a sync completes or fails, an optional `onNotification` callback receives a `SyncNotification` with pre-formatted subject and body text suitable for email delivery:
+
+```typescript
+const result = await orchestrator.RunSync(
+    companyIntegrationID, contextUser, 'Scheduled',
+    undefined,  // onProgress
+    (notification) => {
+        if (notification.Severity !== 'Info') {
+            emailService.send({
+                to: 'ops@example.com',
+                subject: notification.Subject,
+                text: notification.Body,
+            });
+        }
+    }
+);
+```
+
+**SyncNotification** properties:
+- `Event`: `'SyncCompleted'` | `'SyncCompletedWithErrors'` | `'SyncFailed'`
+- `Severity`: `'Info'` | `'Warning'` | `'Error'`
+- `Subject` / `Body`: human-readable text ready for email
+- `Result`: the full `SyncResult` for programmatic access
+
+### IntegrationOrchestrator
+
+- `RunSync(companyIntegrationID, contextUser, triggerType?, onProgress?, onNotification?)` — Executes a full sync run
+
+### FieldMappingEngine
+
+- `Apply(records, fieldMaps, entityName)` — Maps external records to MJ entity fields
+
+### MatchEngine
+
+- `Resolve(records, entityMap, fieldMaps, contextUser)` — Resolves Create/Update/Delete/Skip
+
+### WatermarkService
+
+- `Load(entityMapID, contextUser)` — Loads current watermark
+- `Update(entityMapID, newValue, contextUser)` — Updates or creates watermark
+
+### ConnectorFactory
+
+- `Resolve(integration, sourceTypes)` — Creates connector instance via ClassFactory
+
+### BaseIntegrationConnector (abstract)
+
+- `TestConnection(companyIntegration, contextUser)` — Tests external system connectivity
+- `DiscoverObjects(companyIntegration, contextUser)` — Lists available external objects
+- `DiscoverFields(companyIntegration, objectName, contextUser)` — Lists fields on an object
+- `FetchChanges(ctx)` — Fetches a batch of changed records
+- `GetDefaultFieldMappings(objectName, entityName)` — Suggests default mappings

package/dist/BaseIntegrationConnector.d.ts

@@ -0,0 +1,177 @@
+import type { UserInfo } from '@memberjunction/core';
+import type { MJCompanyIntegrationEntity } from '@memberjunction/core-entities';
+import type { ExternalRecord, DefaultFieldMapping, SourceSchemaInfo } from './types.js';
+/** Result of testing a connection to an external system */
+export interface ConnectionTestResult {
+    /** Whether the connection was successful */
+    Success: boolean;
+    /** Human-readable status message */
+    Message: string;
+    /** Server or API version reported by the external system */
+    ServerVersion?: string;
+}
+/** Schema description of an object/table in an external system */
+export interface ExternalObjectSchema {
+    /** API name of the object (e.g., "Contact", "Account") */
+    Name: string;
+    /** Human-readable label */
+    Label: string;
+    /** Human-readable description of the object's purpose */
+    Description?: string;
+    /** Whether this object supports incremental sync via watermarks */
+    SupportsIncrementalSync: boolean;
+    /** Whether this object can be created/updated from MJ (push) */
+    SupportsWrite: boolean;
+}
+/** Schema description of a single field on an external object */
+export interface ExternalFieldSchema {
+    /** API name of the field */
+    Name: string;
+    /** Human-readable label */
+    Label: string;
+    /** Human-readable description of the field's purpose */
+    Description?: string;
+    /** Field data type in the external system */
+    DataType: string;
+    /** Whether the field is required */
+    IsRequired: boolean;
+    /** Whether the field is a unique identifier */
+    IsUniqueKey: boolean;
+    /** Whether the field is read-only */
+    IsReadOnly: boolean;
+    /** Whether this field is a foreign key */
+    IsForeignKey?: boolean;
+    /** If FK, which source object it references */
+    ForeignKeyTarget?: string | null;
+}
+/** Context passed to FetchChanges for incremental data retrieval */
+export interface FetchContext {
+    /** The company integration entity providing connection details */
+    CompanyIntegration: MJCompanyIntegrationEntity;
+    /** External object name to fetch from */
+    ObjectName: string;
+    /** Current watermark value for incremental fetch, or null for full fetch */
+    WatermarkValue: string | null;
+    /** Maximum number of records to fetch in a single batch */
+    BatchSize: number;
+    /** User context for authorization */
+    ContextUser: UserInfo;
+}
+/** Result of a FetchChanges call, containing a batch of records */
+export interface FetchBatchResult {
+    /** Records retrieved in this batch */
+    Records: ExternalRecord[];
+    /** Whether there are more records to fetch after this batch */
+    HasMore: boolean;
+    /** Updated watermark value after this batch */
+    NewWatermarkValue?: string;
+}
+/** Configurable timeout values for connector operations */
+export interface OperationTimeouts {
+    /** Timeout for TestConnection in milliseconds. Default: 5000 */
+    TestConnectionMs: number;
+    /** Timeout for DiscoverObjects in milliseconds. Default: 10000 */
+    DiscoverObjectsMs: number;
+    /** Timeout for DiscoverFields in milliseconds. Default: 10000 */
+    DiscoverFieldsMs: number;
+    /** Timeout for FetchChanges in milliseconds. Default: 30000 */
+    FetchChangesMs: number;
+}
+/** Default timeout values for connector operations */
+export declare const DEFAULT_OPERATION_TIMEOUTS: OperationTimeouts;
+/**
+ * Wraps a promise with a timeout. Rejects with a timeout error if the
+ * promise does not resolve within the specified duration.
+ *
+ * @param promise - The promise to wrap
+ * @param timeoutMs - Timeout in milliseconds
+ * @param operationName - Name of the operation for error messaging
+ * @returns The result of the promise
+ * @throws Error if the operation times out
+ */
+export declare function WithTimeout<T>(promise: Promise<T>, timeoutMs: number, operationName: string): Promise<T>;
+/** Proposed default configuration for a quick-start setup */
+export interface DefaultObjectConfig {
+    /** Source object name in the external system */
+    SourceObjectName: string;
+    /** Proposed target table name in the MJ database */
+    TargetTableName: string;
+    /** Proposed MJ entity name */
+    TargetEntityName: string;
+    /** Whether to enable sync by default */
+    SyncEnabled: boolean;
+    /** Proposed field mappings */
+    FieldMappings: DefaultFieldMapping[];
+}
+/** Full default configuration returned by a connector for quick setup */
+export interface DefaultIntegrationConfig {
+    /** Proposed DB schema name for new tables (e.g., "YourMembership", "HubSpot") */
+    DefaultSchemaName: string;
+    /** Objects to sync by default with proposed table/entity names */
+    DefaultObjects: DefaultObjectConfig[];
+}
+/**
+ * Abstract base class for integration connectors.
+ * Each external system (HubSpot, Salesforce, etc.) implements this class
+ * to provide system-specific data access and discovery.
+ */
+export declare abstract class BaseIntegrationConnector {
+    /**
+     * Tests connectivity to the external system.
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Connection test result with success/failure and message
+     */
+    abstract TestConnection(companyIntegration: MJCompanyIntegrationEntity, contextUser: UserInfo): Promise<ConnectionTestResult>;
+    /**
+     * Discovers available objects/tables in the external system.
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Array of object schemas available for integration
+     */
+    abstract DiscoverObjects(companyIntegration: MJCompanyIntegrationEntity, contextUser: UserInfo): Promise<ExternalObjectSchema[]>;
+    /**
+     * Discovers fields on a specific external object.
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param objectName - Name of the external object to inspect
+     * @param contextUser - User context for authorization
+     * @returns Array of field schemas for the specified object
+     */
+    abstract DiscoverFields(companyIntegration: MJCompanyIntegrationEntity, objectName: string, contextUser: UserInfo): Promise<ExternalFieldSchema[]>;
+    /**
+     * Fetches a batch of changed records from the external system.
+     * Supports incremental fetching via watermarks.
+     * @param ctx - Context containing connection info, object name, and watermark
+     * @returns Batch of external records with pagination info
+     */
+    abstract FetchChanges(ctx: FetchContext): Promise<FetchBatchResult>;
+    /**
+     * Returns suggested default field mappings for an external object to MJ entity.
+     * Override in subclasses to provide intelligent defaults.
+     * @param _objectName - Name of the external object
+     * @param _entityName - Name of the target MJ entity
+     * @returns Array of default field mappings (empty by default)
+     */
+    GetDefaultFieldMappings(_objectName: string, _entityName: string): DefaultFieldMapping[];
+    /**
+     * Returns a proposed default configuration for quick setup.
+     * Override in subclasses to provide connector-specific defaults
+     * including schema name, objects to sync, and field mappings.
+     * Returns null by default (no quick setup available).
+     */
+    GetDefaultConfiguration(): DefaultIntegrationConfig | null;
+    /**
+     * Introspects the source system's schema — returns metadata about available
+     * objects, their fields, primary keys, and foreign key relationships.
+     * Used by the Schema Builder to generate local DDL.
+     *
+     * Default implementation builds SourceSchemaInfo from DiscoverObjects + DiscoverFields.
+     * Override in subclasses for richer metadata (e.g., FK relationships, type details).
+     *
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Full schema info for all source objects
+     */
+    IntrospectSchema(companyIntegration: MJCompanyIntegrationEntity, contextUser: UserInfo): Promise<SourceSchemaInfo>;
+}
+//# sourceMappingURL=BaseIntegrationConnector.d.ts.map

package/dist/BaseIntegrationConnector.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"BaseIntegrationConnector.d.ts","sourceRoot":"","sources":["../src/BaseIntegrationConnector.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,KAAK,EAAE,0BAA0B,EAAE,MAAM,+BAA+B,CAAC;AAChF,OAAO,KAAK,EAAE,cAAc,EAAE,mBAAmB,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAExF,2DAA2D;AAC3D,MAAM,WAAW,oBAAoB;IACjC,4CAA4C;IAC5C,OAAO,EAAE,OAAO,CAAC;IACjB,oCAAoC;IACpC,OAAO,EAAE,MAAM,CAAC;IAChB,4DAA4D;IAC5D,aAAa,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,kEAAkE;AAClE,MAAM,WAAW,oBAAoB;IACjC,0DAA0D;IAC1D,IAAI,EAAE,MAAM,CAAC;IACb,2BAA2B;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,yDAAyD;IACzD,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,mEAAmE;IACnE,uBAAuB,EAAE,OAAO,CAAC;IACjC,gEAAgE;IAChE,aAAa,EAAE,OAAO,CAAC;CAC1B;AAED,iEAAiE;AACjE,MAAM,WAAW,mBAAmB;IAChC,4BAA4B;IAC5B,IAAI,EAAE,MAAM,CAAC;IACb,2BAA2B;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,wDAAwD;IACxD,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,6CAA6C;IAC7C,QAAQ,EAAE,MAAM,CAAC;IACjB,oCAAoC;IACpC,UAAU,EAAE,OAAO,CAAC;IACpB,+CAA+C;IAC/C,WAAW,EAAE,OAAO,CAAC;IACrB,qCAAqC;IACrC,UAAU,EAAE,OAAO,CAAC;IACpB,0CAA0C;IAC1C,YAAY,CAAC,EAAE,OAAO,CAAC;IACvB,+CAA+C;IAC/C,gBAAgB,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;CACpC;AAED,oEAAoE;AACpE,MAAM,WAAW,YAAY;IACzB,kEAAkE;IAClE,kBAAkB,EAAE,0BAA0B,CAAC;IAC/C,yCAAyC;IACzC,UAAU,EAAE,MAAM,CAAC;IACnB,4EAA4E;IAC5E,cAAc,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,2DAA2D;IAC3D,SAAS,EAAE,MAAM,CAAC;IAClB,qCAAqC;IACrC,WAAW,EAAE,QAAQ,CAAC;CACzB;AAED,mEAAmE;AACnE,MAAM,WAAW,gBAAgB;IAC7B,sCAAsC;IACtC,OAAO,EAAE,cAAc,EAAE,CAAC;IAC1B,+DAA+D;IAC/D,OAAO,EAAE,OAAO,CAAC;IACjB,+CAA+C;IAC/C,iBAAiB,CAAC,EAAE,MAAM,CAAC;CAC9B;AAED,2DAA2D;AAC3D,MAAM,WAAW,iBAAiB;IAC9B,gEAAgE;IAChE,gBAAgB,EAAE,MAAM,CAAC;IACzB,kEAAkE;IAClE,iBAAiB,EAAE,MAAM,CAAC;IAC1B,iEAAiE;IACjE,gBAAgB,EAAE,MAAM,CAAC;IACzB,+DAA+D;IAC/D,cAAc,EAAE,MAAM,CAAC;CAC1B;AAED,sDAAsD;AACtD,eAAO,MAAM,0BAA0B,EAAE,iBAKxC,CAAC;AAEF;;;;;;;;;GASG;AACH,wBAAsB,WAAW,CAAC,CAAC,EAC/B,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC,EACnB,SAAS,EAAE,MAAM,EACjB,aAAa,EAAE,MAAM,GACtB,OAAO,CAAC,CAAC,CAAC,CAiBZ;AAED,6DAA6D;AAC7D,MAAM,WAAW,mBAAmB;IAChC,gDAAgD;IAChD,gBAAgB,EAAE,MAAM,CAAC;IACzB,oDAAoD;IACpD,eAAe,EAAE,MAAM,CAAC;IACxB,8BAA8B;IAC9B,gBAAgB,EAAE,MAAM,CAAC;IACzB,wCAAwC;IACxC,WAAW,EAAE,OAAO,CAAC;IACrB,8BAA8B;IAC9B,aAAa,EAAE,mBAAmB,EAAE,CAAC;CACxC;AAED,yEAAyE;AACzE,MAAM,WAAW,wBAAwB;IACrC,iFAAiF;IACjF,iBAAiB,EAAE,MAAM,CAAC;IAC1B,kEAAkE;IAClE,cAAc,EAAE,mBAAmB,EAAE,CAAC;CACzC;AAED;;;;GAIG;AACH,8BAAsB,wBAAwB;IAC1C;;;;;OAKG;aACa,cAAc,CAC1B,kBAAkB,EAAE,0BAA0B,EAC9C,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,oBAAoB,CAAC;IAEhC;;;;;OAKG;aACa,eAAe,CAC3B,kBAAkB,EAAE,0BAA0B,EAC9C,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,oBAAoB,EAAE,CAAC;IAElC;;;;;;OAMG;aACa,cAAc,CAC1B,kBAAkB,EAAE,0BAA0B,EAC9C,UAAU,EAAE,MAAM,EAClB,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,mBAAmB,EAAE,CAAC;IAEjC;;;;;OAKG;aACa,YAAY,CAAC,GAAG,EAAE,YAAY,GAAG,OAAO,CAAC,gBAAgB,CAAC;IAE1E;;;;;;OAMG;IACI,uBAAuB,CAAC,WAAW,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM,GAAG,mBAAmB,EAAE;IAI/F;;;;;OAKG;IACI,uBAAuB,IAAI,wBAAwB,GAAG,IAAI;IAIjE;;;;;;;;;;;OAWG;IACU,gBAAgB,CACzB,kBAAkB,EAAE,0BAA0B,EAC9C,WAAW,EAAE,QAAQ,GACtB,OAAO,CAAC,gBAAgB,CAAC;CAsC/B"}

package/dist/BaseIntegrationConnector.js

@@ -0,0 +1,110 @@
+/** Default timeout values for connector operations */
+export const DEFAULT_OPERATION_TIMEOUTS = {
+    TestConnectionMs: 5000,
+    DiscoverObjectsMs: 10000,
+    DiscoverFieldsMs: 10000,
+    FetchChangesMs: 30000,
+};
+/**
+ * Wraps a promise with a timeout. Rejects with a timeout error if the
+ * promise does not resolve within the specified duration.
+ *
+ * @param promise - The promise to wrap
+ * @param timeoutMs - Timeout in milliseconds
+ * @param operationName - Name of the operation for error messaging
+ * @returns The result of the promise
+ * @throws Error if the operation times out
+ */
+export async function WithTimeout(promise, timeoutMs, operationName) {
+    let timeoutHandle;
+    const timeoutPromise = new Promise((_resolve, reject) => {
+        timeoutHandle = setTimeout(() => {
+            reject(new Error(`Operation '${operationName}' timed out after ${timeoutMs}ms`));
+        }, timeoutMs);
+    });
+    try {
+        const result = await Promise.race([promise, timeoutPromise]);
+        return result;
+    }
+    finally {
+        if (timeoutHandle !== undefined) {
+            clearTimeout(timeoutHandle);
+        }
+    }
+}
+/**
+ * Abstract base class for integration connectors.
+ * Each external system (HubSpot, Salesforce, etc.) implements this class
+ * to provide system-specific data access and discovery.
+ */
+export class BaseIntegrationConnector {
+    /**
+     * Returns suggested default field mappings for an external object to MJ entity.
+     * Override in subclasses to provide intelligent defaults.
+     * @param _objectName - Name of the external object
+     * @param _entityName - Name of the target MJ entity
+     * @returns Array of default field mappings (empty by default)
+     */
+    GetDefaultFieldMappings(_objectName, _entityName) {
+        return [];
+    }
+    /**
+     * Returns a proposed default configuration for quick setup.
+     * Override in subclasses to provide connector-specific defaults
+     * including schema name, objects to sync, and field mappings.
+     * Returns null by default (no quick setup available).
+     */
+    GetDefaultConfiguration() {
+        return null;
+    }
+    /**
+     * Introspects the source system's schema — returns metadata about available
+     * objects, their fields, primary keys, and foreign key relationships.
+     * Used by the Schema Builder to generate local DDL.
+     *
+     * Default implementation builds SourceSchemaInfo from DiscoverObjects + DiscoverFields.
+     * Override in subclasses for richer metadata (e.g., FK relationships, type details).
+     *
+     * @param companyIntegration - The company integration entity with connection credentials
+     * @param contextUser - User context for authorization
+     * @returns Full schema info for all source objects
+     */
+    async IntrospectSchema(companyIntegration, contextUser) {
+        const objects = await this.DiscoverObjects(companyIntegration, contextUser);
+        const result = { Objects: [] };
+        for (const obj of objects) {
+            let fields;
+            try {
+                fields = await this.DiscoverFields(companyIntegration, obj.Name, contextUser);
+            }
+            catch (err) {
+                const msg = err instanceof Error ? err.message : String(err);
+                console.warn(`WARNING: Skipping object "${obj.Name}" — DiscoverFields failed: ${msg}`);
+                continue;
+            }
+            result.Objects.push({
+                ExternalName: obj.Name,
+                ExternalLabel: obj.Label,
+                Description: obj.Description,
+                Fields: fields.map(f => ({
+                    Name: f.Name,
+                    Label: f.Label,
+                    Description: f.Description,
+                    SourceType: f.DataType,
+                    IsRequired: f.IsRequired,
+                    MaxLength: null,
+                    Precision: null,
+                    Scale: null,
+                    DefaultValue: null,
+                    IsPrimaryKey: f.IsUniqueKey,
+                    IsForeignKey: f.IsForeignKey ?? false,
+                    ForeignKeyTarget: f.ForeignKeyTarget ?? null,
+                })),
+                PrimaryKeyFields: fields.filter(f => f.IsUniqueKey).map(f => f.Name),
+                Relationships: [],
+            });
+        }
+        return result;
+    }
+}
+//# sourceMappingURL=BaseIntegrationConnector.js.map

package/dist/BaseIntegrationConnector.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"BaseIntegrationConnector.js","sourceRoot":"","sources":["../src/BaseIntegrationConnector.ts"],"names":[],"mappings":"AAsFA,sDAAsD;AACtD,MAAM,CAAC,MAAM,0BAA0B,GAAsB;IACzD,gBAAgB,EAAE,IAAI;IACtB,iBAAiB,EAAE,KAAK;IACxB,gBAAgB,EAAE,KAAK;IACvB,cAAc,EAAE,KAAK;CACxB,CAAC;AAEF;;;;;;;;;GASG;AACH,MAAM,CAAC,KAAK,UAAU,WAAW,CAC7B,OAAmB,EACnB,SAAiB,EACjB,aAAqB;IAErB,IAAI,aAAwD,CAAC;IAE7D,MAAM,cAAc,GAAG,IAAI,OAAO,CAAQ,CAAC,QAAQ,EAAE,MAAM,EAAE,EAAE;QAC3D,aAAa,GAAG,UAAU,CAAC,GAAG,EAAE;YAC5B,MAAM,CAAC,IAAI,KAAK,CAAC,cAAc,aAAa,qBAAqB,SAAS,IAAI,CAAC,CAAC,CAAC;QACrF,CAAC,EAAE,SAAS,CAAC,CAAC;IAClB,CAAC,CAAC,CAAC;IAEH,IAAI,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC,OAAO,EAAE,cAAc,CAAC,CAAC,CAAC;QAC7D,OAAO,MAAM,CAAC;IAClB,CAAC;YAAS,CAAC;QACP,IAAI,aAAa,KAAK,SAAS,EAAE,CAAC;YAC9B,YAAY,CAAC,aAAa,CAAC,CAAC;QAChC,CAAC;IACL,CAAC;AACL,CAAC;AAwBD;;;;GAIG;AACH,MAAM,OAAgB,wBAAwB;IA4C1C;;;;;;OAMG;IACI,uBAAuB,CAAC,WAAmB,EAAE,WAAmB;QACnE,OAAO,EAAE,CAAC;IACd,CAAC;IAED;;;;;OAKG;IACI,uBAAuB;QAC1B,OAAO,IAAI,CAAC;IAChB,CAAC;IAED;;;;;;;;;;;OAWG;IACI,KAAK,CAAC,gBAAgB,CACzB,kBAA8C,EAC9C,WAAqB;QAErB,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,eAAe,CAAC,kBAAkB,EAAE,WAAW,CAAC,CAAC;QAC5E,MAAM,MAAM,GAAqB,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;QAEjD,KAAK,MAAM,GAAG,IAAI,OAAO,EAAE,CAAC;YACxB,IAAI,MAA6B,CAAC;YAClC,IAAI,CAAC;gBACD,MAAM,GAAG,MAAM,IAAI,CAAC,cAAc,CAAC,kBAAkB,EAAE,GAAG,CAAC,IAAI,EAAE,WAAW,CAAC,CAAC;YAClF,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACX,MAAM,GAAG,GAAG,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;gBAC7D,OAAO,CAAC,IAAI,CAAC,6BAA6B,GAAG,CAAC,IAAI,8BAA8B,GAAG,EAAE,CAAC,CAAC;gBACvF,SAAS;YACb,CAAC;YACD,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC;gBAChB,YAAY,EAAE,GAAG,CAAC,IAAI;gBACtB,aAAa,EAAE,GAAG,CAAC,KAAK;gBACxB,WAAW,EAAE,GAAG,CAAC,WAAW;gBAC5B,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;oBACrB,IAAI,EAAE,CAAC,CAAC,IAAI;oBACZ,KAAK,EAAE,CAAC,CAAC,KAAK;oBACd,WAAW,EAAE,CAAC,CAAC,WAAW;oBAC1B,UAAU,EAAE,CAAC,CAAC,QAAQ;oBACtB,UAAU,EAAE,CAAC,CAAC,UAAU;oBACxB,SAAS,EAAE,IAAI;oBACf,SAAS,EAAE,IAAI;oBACf,KAAK,EAAE,IAAI;oBACX,YAAY,EAAE,IAAI;oBAClB,YAAY,EAAE,CAAC,CAAC,WAAW;oBAC3B,YAAY,EAAE,CAAC,CAAC,YAAY,IAAI,KAAK;oBACrC,gBAAgB,EAAE,CAAC,CAAC,gBAAgB,IAAI,IAAI;iBAC/C,CAAC,CAAC;gBACH,gBAAgB,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;gBACpE,aAAa,EAAE,EAAE;aACpB,CAAC,CAAC;QACP,CAAC;QAED,OAAO,MAAM,CAAC;IAClB,CAAC;CACJ"}