npm - @fluentcommerce/fc-connect-sdk - Versions diffs - 0.1.54 → 0.1.56 - Mend

@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (476) hide show

package/docs/04-REFERENCE/architecture/architecture-05-integration-architecture.md CHANGED Viewed

@@ -1,655 +1,655 @@
-# Integration Architecture
-This document explains how the FC Connect SDK integrates with external systems including Fluent Commerce, Amazon S3, SFTP servers, and the Versori platform.
-## Overview
-The SDK provides robust integration capabilities across multiple systems:
-```mermaid
-graph TB
-    SDK[FC Connect SDK]
-    subgraph "External Systems"
-        FLUENT[Fluent Commerce API]
-        S3[Amazon S3]
-        SFTP[SFTP Servers]
-        VERSORI[Versori Platform]
-    end
-    subgraph "Integration Methods"
-        AUTH[Authentication]
-        API[API Communication]
-        STORAGE[File Storage]
-        PLATFORM[Platform Services]
-    end
-    SDK --> FLUENT
-    SDK --> S3
-    SDK --> SFTP
-    SDK --> VERSORI
-    FLUENT --> AUTH
-    S3 --> STORAGE
-    SFTP --> STORAGE
-    VERSORI --> PLATFORM
-    AUTH --> OAUTH[OAuth2]
-    AUTH --> CONN[Connection-based]
-    STORAGE --> PRESIGN[Presigned URLs]
-    STORAGE --> SDK_UPLOAD[SDK Upload]
-    STORAGE --> SSH[SSH Keys]
-    PLATFORM --> KV[KV Storage]
-    PLATFORM --> WEBHOOK[Webhooks]
-    PLATFORM --> LOG[Logging]
-    style SDK fill:#e3f2fd
-    style FLUENT fill:#ffebee
-    style S3 fill:#fff3e0
-    style SFTP fill:#f3e5f5
-    style VERSORI fill:#e8f5e9
-```
-## Fluent Commerce Integration
-### Authentication Flows
-```mermaid
-sequenceDiagram
-    participant SDK
-    participant AuthService
-    participant FluentOAuth
-    participant FluentAPI
-    Note over SDK,FluentAPI: OAuth2 Client Credentials Flow
-    SDK->>AuthService: Initialize with credentials
-    AuthService->>AuthService: Store clientId, clientSecret
-    SDK->>AuthService: Request access token
-    AuthService->>FluentOAuth: POST /oauth/token
-    Note right of AuthService: grant_type: client_credentials<br/>client_id: xxx<br/>client_secret: yyy
-    FluentOAuth-->>AuthService: { access_token, expires_in }
-    AuthService->>AuthService: Cache token with expiry
-    SDK->>AuthService: Get token for API call
-    AuthService->>AuthService: Check token expiry
-    alt Token Valid
-        AuthService-->>SDK: Return cached token
-    else Token Expired
-        AuthService->>FluentOAuth: Request new token
-        FluentOAuth-->>AuthService: New token
-        AuthService->>AuthService: Update cache
-        AuthService-->>SDK: Return new token
-    end
-    SDK->>FluentAPI: GraphQL request (Bearer token)
-    FluentAPI-->>SDK: Response
-```
-### API Communication Patterns
-```mermaid
-graph TB
-    SDK[SDK] --> DECIDE{Operation Type}
-    DECIDE -->|Batch| BATCH[Batch API]
-    DECIDE -->|Query| GRAPHQL[GraphQL API]
-    DECIDE -->|Mutation| GRAPHQL
-    BATCH --> JOB[Create Job]
-    JOB --> SEND[Send Batches]
-    SEND --> POLL[Poll Status]
-    GRAPHQL --> BUILD[Build Query/Mutation]
-    BUILD --> EXECUTE[Execute Request]
-    EXECUTE --> PAGINATE{More Pages?}
-    PAGINATE -->|Yes| CURSOR[Fetch Next Cursor]
-    PAGINATE -->|No| COMPLETE[Return Results]
-    CURSOR --> EXECUTE
-    style SDK fill:#e3f2fd
-    style BATCH fill:#fff3e0
-    style GRAPHQL fill:#f3e5f5
-```
-**Batch API Flow:**
-```typescript
-// 1. Create job
-const job = await client.createJob({
-  name: 'inventory-sync',
-  retailerId: 'my-retailer',
-});
-// 2. Send batches
-const batch = await client.sendBatch(job.id, {
-  entities: inventoryData,
-});
-// 3. Poll status
-const status = await client.getBatchStatus(job.id, batch.id);
-```
-**GraphQL Flow:**
-```typescript
-// Query with pagination (auto-enabled for queries that include pagination vars)
-const results = await client.graphql({
-  query: `
-    query GetInventory($first: Int!, $after: String) {
-      inventories(first: $first, after: $after) {
-        edges {
-          cursor
-          node { id ref productRef locationRef qty }
-        }
-        pageInfo { hasNextPage }
-      }
-    }
-  `,
-  variables: { first: 100 },
-  // To opt out: pagination: { enabled: false }
-});
-```
-### Error Handling & Retry Logic
-**Retry Configuration:**
-```typescript
-const client = await createClient({
-  config: {
-    baseUrl: 'https://api.fluentcommerce.com',
-    clientId: process.env.FLUENT_CLIENT_ID!,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
-    retryAttempts: 3,
-    retryDelay: 1000,
-  },
-});
-```
-## Amazon S3 Integration
-### Dual S3 Implementation
-The SDK supports two S3 integration methods:
-```mermaid
-graph TB
-    SDK[SDK] --> CHOOSE{S3 Method}
-    CHOOSE -->|Default| PRESIGN[Presigned URLs<br/>No AWS credentials needed]
-    CHOOSE -->|Optional| AWS_SDK[AWS SDK v3<br/>Direct S3 access]
-    PRESIGN --> SIG_SERVICE[S3PresignService]
-    SIG_SERVICE --> GENERATE[Generate SigV4 URL]
-    GENERATE --> HTTP[Standard HTTP Upload/Download]
-    AWS_SDK --> S3_CLIENT[S3Client]
-    S3_CLIENT --> PUT[PutObject]
-    S3_CLIENT --> GET[GetObject]
-    S3_CLIENT --> LIST[ListObjects]
-    HTTP --> WORKS[Works Everywhere<br/>Node.js, Deno, Versori]
-    PUT --> WORKS2[Requires AWS SDK<br/>More features]
-    style PRESIGN fill:#e8f5e9
-    style AWS_SDK fill:#fff3e0
-    style WORKS fill:#e3f2fd
-```
-### Presigned URL Flow
-```mermaid
-sequenceDiagram
-    participant SDK
-    participant PresignService
-    participant S3
-    Note over SDK,S3: Upload Flow with Presigned URLs
-    SDK->>PresignService: getPresignedUploadUrl(bucket, key, expiry)
-    PresignService->>PresignService: Generate SigV4 signature
-    PresignService-->>SDK: Presigned PUT URL (valid 1 hour)
-    SDK->>S3: PUT {presigned_url} (file data)
-    S3-->>SDK: 200 OK
-    Note over SDK,S3: Download Flow with Presigned URLs
-    SDK->>PresignService: getPresignedDownloadUrl(bucket, key, expiry)
-    PresignService->>PresignService: Generate SigV4 signature
-    PresignService-->>SDK: Presigned GET URL (valid 1 hour)
-    SDK->>S3: GET {presigned_url}
-    S3-->>SDK: File data
-```
-**Presigned URL Benefits:**
-- No AWS credentials needed in application
-- Works in restricted environments (Versori, browser)
-- Temporary, secure access
-- Cross-account and cross-region support
-**Example:**
-```typescript
-const presignService = new S3PresignService({
-  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
-  region: 'us-east-1',
-});
-// Upload
-const uploadUrl = await presignService.getPresignedUploadUrl(
-  'my-bucket',
-  'path/to/file.csv',
-  3600 // 1 hour
-);
-await fetch(uploadUrl, {
-  method: 'PUT',
-  body: fileData,
-});
-// Download
-const downloadUrl = await presignService.getPresignedDownloadUrl(
-  'my-bucket',
-  'path/to/file.csv',
-  3600
-);
-const response = await fetch(downloadUrl);
-const data = await response.text();
-```
-### S3 Operations
-```mermaid
-graph TB
-    S3_SOURCE[S3DataSource]
-    S3_SOURCE --> LIST[List Files]
-    S3_SOURCE --> DOWNLOAD[Download]
-    S3_SOURCE --> UPLOAD[Upload]
-    S3_SOURCE --> ARCHIVE[Archive]
-    LIST --> PREFIX[Prefix Filtering<br/>folder/]
-    style S3_SOURCE fill:#e3f2fd
-    style LIST fill:#fff3e0
-    style DOWNLOAD fill:#f3e5f5
-    style UPLOAD fill:#e8f5e9
-    style ARCHIVE fill:#fce4ec
-```
-- Note: S3 supports prefix filtering (e.g., `prefix: 'folder/'`). Wildcard patterns (e.g., `*.csv`) are not supported on S3.
-## SFTP Integration
-### Connection Management
-```mermaid
-sequenceDiagram
-    participant SDK
-    participant SftpDataSource
-    participant SSHClient
-    participant SftpServer
-    SDK->>SftpDataSource: Initialize connection
-    SftpDataSource->>SftpDataSource: Load SSH key or password
-    SftpDataSource->>SSHClient: Connect
-    SSHClient->>SftpServer: SSH handshake
-    alt Key-based Auth
-        SSHClient->>SftpServer: Authenticate with SSH key
-    else Password Auth
-        SSHClient->>SftpServer: Authenticate with password
-    end
-    SftpServer-->>SSHClient: Authentication success
-    SSHClient-->>SftpDataSource: Connection established
-    SDK->>SftpDataSource: listFiles(pattern)
-    SftpDataSource->>SftpServer: List directory
-    SftpServer-->>SftpDataSource: File list
-    SftpDataSource-->>SDK: Matching files
-    SDK->>SftpDataSource: downloadFile(path)
-    SftpDataSource->>SftpServer: Get file
-    SftpServer-->>SftpDataSource: File stream
-    SftpDataSource-->>SDK: File data
-    SDK->>SftpDataSource: Disconnect
-    SftpDataSource->>SSHClient: Close connection
-```
-**Configuration:**
-```typescript
-const sftp = new SftpDataSource(
-  {
-    /* host, auth, path, filePattern */
-  },
-  logger
-);
-// List files with wildcards
-const files = await sftp.listFiles({ remotePath: '/incoming', filePattern: '*.csv' });
-// Download file (Buffer or string depending on options)
-const content = await sftp.downloadFile('/incoming/file.csv');
-```
-### SFTP Operations
-```mermaid
-graph TB
-    SFTP[SftpDataSource]
-    SFTP --> CONN[Connection Pool]
-    SFTP --> LIST[List Files]
-    SFTP --> DOWN[Download]
-    SFTP --> UP[Upload]
-    SFTP --> DEL[Delete]
-    CONN --> REUSE[Reuse Connections]
-    CONN --> TIMEOUT[Auto-disconnect on Idle]
-    LIST --> FILTER[Pattern Filtering<br/>*.csv, *.xml]
-    DOWN --> STREAM[Streaming Support]
-    UP --> RETRY[Retry on Failure]
-    DEL --> SAFE[Safe Delete with Verification]
-    style SFTP fill:#e3f2fd
-    style CONN fill:#fff3e0
-    style LIST fill:#f3e5f5
-    style DOWN fill:#e8f5e9
-```
-## Versori Platform Integration
-### Platform Context Detection
-```mermaid
-flowchart TD
-    START[SDK Initialization] --> DETECT{Detect Context}
-    DETECT -->|Has 'activation'| VERSORI[Versori Platform]
-    DETECT -->|No 'activation'| STANDALONE[Standalone Runtime]
-    VERSORI --> PLATFORM_SERVICES[Platform Services]
-    STANDALONE --> MANUAL_CONFIG[Manual Configuration]
-    PLATFORM_SERVICES --> KV[KV Storage]
-    PLATFORM_SERVICES --> LOGGER[Platform Logger]
-    PLATFORM_SERVICES --> FETCH[Authenticated Fetch]
-    PLATFORM_SERVICES --> CONN[Connections]
-    MANUAL_CONFIG --> ENV[Environment Variables]
-    MANUAL_CONFIG --> CONFIG[Config Objects]
-    style START fill:#e3f2fd
-    style VERSORI fill:#e8f5e9
-    style STANDALONE fill:#fff3e0
-```
-### Versori Services Integration
-```mermaid
-graph TB
-    subgraph "Versori Platform"
-        HTTP[HTTP Webhook]
-        SCHEDULE[Scheduled Workflow]
-        KV_STORE[KV Storage]
-        CONNECTIONS[Connections]
-        LOGGING[Platform Logging]
-    end
-    subgraph "SDK Adapters"
-        KV_ADAPTER[VersoriKVAdapter]
-        FILE_TRACKER[VersoriFileTracker]
-        CLIENT[FluentVersoriClient]
-    end
-    HTTP --> CLIENT
-    SCHEDULE --> CLIENT
-    KV_STORE --> KV_ADAPTER
-    KV_ADAPTER --> FILE_TRACKER
-    CONNECTIONS --> CLIENT
-    LOGGING --> CLIENT
-    CLIENT --> FLUENT[Fluent Commerce]
-    style HTTP fill:#e3f2fd
-    style KV_STORE fill:#fff3e0
-    style CONNECTIONS fill:#f3e5f5
-    style CLIENT fill:#e8f5e9
-```
-### KV Storage Adapter
-```mermaid
-sequenceDiagram
-    participant SDK
-    participant VersoriKVAdapter
-    participant VersoriKV
-    SDK->>VersoriKVAdapter: Initialize with openKv()
-    VersoriKVAdapter->>VersoriKV: Open KV store
-    SDK->>VersoriKVAdapter: get('state')
-    VersoriKVAdapter->>VersoriKV: Read key
-    VersoriKV-->>VersoriKVAdapter: Value
-    VersoriKVAdapter-->>SDK: Parsed state object
-    SDK->>VersoriKVAdapter: set('state', newState)
-    VersoriKVAdapter->>VersoriKV: Write key
-    VersoriKV-->>VersoriKVAdapter: Confirmation
-    VersoriKVAdapter-->>SDK: Success
-    SDK->>VersoriKVAdapter: delete('old-key')
-    VersoriKVAdapter->>VersoriKV: Delete key
-    VersoriKV-->>VersoriKVAdapter: Deleted
-    VersoriKVAdapter-->>SDK: Success
-```
-**Usage:**
-```typescript
-import { VersoriKVAdapter, StateService } from '@fluentcommerce/fc-connect-sdk';
-// ✅ CORRECT: Access openKv from Versori context
-// import { openKv } from '@versori/run'; // ❌ WRONG - Not a direct export
-// In Versori workflow handler:
-const { openKv } = ctx;
-const kvAdapter = new VersoriKVAdapter(openKv(':project:'));
-const stateService = new StateService(logger); // logger is optional
-// Use state service for deduplication (kv is first parameter)
-const processed = await stateService.isFileProcessed(kvAdapter, 'file-key', 'my-workflow');
-```
-### Webhook Integration
-```mermaid
-sequenceDiagram
-    participant External
-    participant Versori
-    participant SDK
-    participant FluentAPI
-    External->>Versori: POST /webhook (data)
-    Versori->>Versori: Validate signature
-    Versori->>SDK: Trigger workflow
-    SDK->>SDK: Parse webhook payload
-    SDK->>SDK: Apply field mappings
-    SDK->>SDK: Build GraphQL mutation
-    SDK->>FluentAPI: Execute mutation
-    FluentAPI-->>SDK: Result
-    SDK-->>Versori: Response
-    Versori-->>External: 200 OK
-```
-**Webhook Example:**
-```typescript
-import { webhook } from '@versori/run';
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-export const processOrder = webhook('process-order', async ctx => {
-  const client = await createClient(ctx);
-  // Parse webhook payload
-  const orderData = await ctx.request.json();
-  // Transform and send to Fluent via GraphQL
-  const result = await client.graphql({
-    query: `mutation($input: CreateOrderInput!) { createOrder(input: $input) { id } }`,
-    variables: { input: orderData },
-  });
-  return { success: true, orderId: (result.data as any)?.createOrder?.id };
-});
-```
-## Cross-System Data Flows
-### Ingestion Flow (S3 → Fluent)
-```mermaid
-sequenceDiagram
-    participant S3
-    participant SDK
-    participant State
-    participant Fluent
-    SDK->>S3: List files
-    S3-->>SDK: [file1.csv, file2.csv]
-    loop For Each File
-        SDK->>State: isFileProcessed(file)
-        State-->>SDK: false
-        SDK->>S3: Download file
-        S3-->>SDK: File data
-        SDK->>SDK: Parse CSV
-        SDK->>SDK: Map fields
-        SDK->>Fluent: Create job
-        Fluent-->>SDK: Job ID
-        SDK->>Fluent: Send batches
-        Fluent-->>SDK: Batch IDs
-        SDK->>State: updateSyncState(kv, processedFiles)
-        SDK->>S3: Archive file
-    end
-```
-### Extraction Flow (Fluent → S3)
-```mermaid
-sequenceDiagram
-    participant SDK
-    participant Fluent
-    participant S3
-    SDK->>Fluent: Execute query (page 1)
-    Fluent-->>SDK: Results + cursor
-    loop Pagination
-        SDK->>Fluent: Fetch next page
-        Fluent-->>SDK: Results + cursor
-    end
-    SDK->>SDK: Transform data
-    SDK->>SDK: Apply mappings
-    SDK->>S3: Upload JSON
-    S3-->>SDK: Upload confirmation
-```
-### Transformation Flow (External → Fluent)
-```mermaid
-sequenceDiagram
-    participant External
-    participant SDK
-    participant Fluent
-    External->>SDK: POST webhook (XML/JSON)
-    SDK->>SDK: Parse input
-    SDK->>SDK: Load mapping config
-    SDK->>SDK: Build GraphQL mutation
-    SDK->>Fluent: Execute mutation
-    Fluent-->>SDK: Created entity
-    SDK-->>External: 200 OK (entity ID)
-```
-## Security Considerations
-### Authentication Security
-```mermaid
-graph TB
-    SECURITY[Security Measures]
-    SECURITY --> TOKEN[Token Management]
-    SECURITY --> CREDS[Credential Storage]
-    SECURITY --> NETWORK[Network Security]
-    TOKEN --> CACHE[In-memory Cache Only]
-    TOKEN --> REFRESH[Auto-refresh Before Expiry]
-    TOKEN --> EXPIRE[Short-lived Tokens]
-    CREDS --> ENV[Environment Variables]
-    CREDS --> VAULT[Secret Vaults]
-    CREDS --> NEVER[Never Hardcode]
-    NETWORK --> HTTPS[HTTPS Only]
-    NETWORK --> VALIDATE[Validate Certificates]
-    NETWORK --> TIMEOUT[Request Timeouts]
-    style SECURITY fill:#ffebee
-    style TOKEN fill:#fff3e0
-    style CREDS fill:#f3e5f5
-    style NETWORK fill:#e8f5e9
-```
-**Best Practices:**
-- Store credentials in environment variables or secret managers
-- Use short-lived OAuth2 tokens (auto-refreshed)
-- Enable HTTPS for all API communication
-- Implement request timeouts
-- Validate webhook signatures (Versori)
-- Use presigned URLs with expiration for S3
-**Versori Client Features:**
-- GraphQL auto-pagination parity with standalone client
-- Excludes internal fields from responses
-- Aligned endpoints (sendBatch uses POST, status endpoints)
-- Automatic retry/backoff for 5xx errors
-- Exposes `validateWebhook()` for signature validation
-## Next Steps
-- [State Management](./architecture-06-state-management.md) - Distributed state and deduplication
-- [Client Architecture](./architecture-02-client-architecture.md) - Client layer details
-- [Data Flow](./architecture-03-data-flow.md) - Data processing patterns
+# Integration Architecture
+This document explains how the FC Connect SDK integrates with external systems including Fluent Commerce, Amazon S3, SFTP servers, and the Versori platform.
+## Overview
+The SDK provides robust integration capabilities across multiple systems:
+```mermaid
+graph TB
+    SDK[FC Connect SDK]
+    subgraph "External Systems"
+        FLUENT[Fluent Commerce API]
+        S3[Amazon S3]
+        SFTP[SFTP Servers]
+        VERSORI[Versori Platform]
+    end
+    subgraph "Integration Methods"
+        AUTH[Authentication]
+        API[API Communication]
+        STORAGE[File Storage]
+        PLATFORM[Platform Services]
+    end
+    SDK --> FLUENT
+    SDK --> S3
+    SDK --> SFTP
+    SDK --> VERSORI
+    FLUENT --> AUTH
+    S3 --> STORAGE
+    SFTP --> STORAGE
+    VERSORI --> PLATFORM
+    AUTH --> OAUTH[OAuth2]
+    AUTH --> CONN[Connection-based]
+    STORAGE --> PRESIGN[Presigned URLs]
+    STORAGE --> SDK_UPLOAD[SDK Upload]
+    STORAGE --> SSH[SSH Keys]
+    PLATFORM --> KV[KV Storage]
+    PLATFORM --> WEBHOOK[Webhooks]
+    PLATFORM --> LOG[Logging]
+    style SDK fill:#e3f2fd
+    style FLUENT fill:#ffebee
+    style S3 fill:#fff3e0
+    style SFTP fill:#f3e5f5
+    style VERSORI fill:#e8f5e9
+```
+## Fluent Commerce Integration
+### Authentication Flows
+```mermaid
+sequenceDiagram
+    participant SDK
+    participant AuthService
+    participant FluentOAuth
+    participant FluentAPI
+    Note over SDK,FluentAPI: OAuth2 Client Credentials Flow
+    SDK->>AuthService: Initialize with credentials
+    AuthService->>AuthService: Store clientId, clientSecret
+    SDK->>AuthService: Request access token
+    AuthService->>FluentOAuth: POST /oauth/token
+    Note right of AuthService: grant_type: client_credentials<br/>client_id: xxx<br/>client_secret: yyy
+    FluentOAuth-->>AuthService: { access_token, expires_in }
+    AuthService->>AuthService: Cache token with expiry
+    SDK->>AuthService: Get token for API call
+    AuthService->>AuthService: Check token expiry
+    alt Token Valid
+        AuthService-->>SDK: Return cached token
+    else Token Expired
+        AuthService->>FluentOAuth: Request new token
+        FluentOAuth-->>AuthService: New token
+        AuthService->>AuthService: Update cache
+        AuthService-->>SDK: Return new token
+    end
+    SDK->>FluentAPI: GraphQL request (Bearer token)
+    FluentAPI-->>SDK: Response
+```
+### API Communication Patterns
+```mermaid
+graph TB
+    SDK[SDK] --> DECIDE{Operation Type}
+    DECIDE -->|Batch| BATCH[Batch API]
+    DECIDE -->|Query| GRAPHQL[GraphQL API]
+    DECIDE -->|Mutation| GRAPHQL
+    BATCH --> JOB[Create Job]
+    JOB --> SEND[Send Batches]
+    SEND --> POLL[Poll Status]
+    GRAPHQL --> BUILD[Build Query/Mutation]
+    BUILD --> EXECUTE[Execute Request]
+    EXECUTE --> PAGINATE{More Pages?}
+    PAGINATE -->|Yes| CURSOR[Fetch Next Cursor]
+    PAGINATE -->|No| COMPLETE[Return Results]
+    CURSOR --> EXECUTE
+    style SDK fill:#e3f2fd
+    style BATCH fill:#fff3e0
+    style GRAPHQL fill:#f3e5f5
+```
+**Batch API Flow:**
+```typescript
+// 1. Create job
+const job = await client.createJob({
+  name: 'inventory-sync',
+  retailerId: 'my-retailer',
+});
+// 2. Send batches
+const batch = await client.sendBatch(job.id, {
+  entities: inventoryData,
+});
+// 3. Poll status
+const status = await client.getBatchStatus(job.id, batch.id);
+```
+**GraphQL Flow:**
+```typescript
+// Query with pagination (auto-enabled for queries that include pagination vars)
+const results = await client.graphql({
+  query: `
+    query GetInventory($first: Int!, $after: String) {
+      inventories(first: $first, after: $after) {
+        edges {
+          cursor
+          node { id ref productRef locationRef qty }
+        }
+        pageInfo { hasNextPage }
+      }
+    }
+  `,
+  variables: { first: 100 },
+  // To opt out: pagination: { enabled: false }
+});
+```
+### Error Handling & Retry Logic
+**Retry Configuration:**
+```typescript
+const client = await createClient({
+  config: {
+    baseUrl: 'https://api.fluentcommerce.com',
+    clientId: process.env.FLUENT_CLIENT_ID!,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
+    retryAttempts: 3,
+    retryDelay: 1000,
+  },
+});
+```
+## Amazon S3 Integration
+### Dual S3 Implementation
+The SDK supports two S3 integration methods:
+```mermaid
+graph TB
+    SDK[SDK] --> CHOOSE{S3 Method}
+    CHOOSE -->|Default| PRESIGN[Presigned URLs<br/>No AWS credentials needed]
+    CHOOSE -->|Optional| AWS_SDK[AWS SDK v3<br/>Direct S3 access]
+    PRESIGN --> SIG_SERVICE[S3PresignService]
+    SIG_SERVICE --> GENERATE[Generate SigV4 URL]
+    GENERATE --> HTTP[Standard HTTP Upload/Download]
+    AWS_SDK --> S3_CLIENT[S3Client]
+    S3_CLIENT --> PUT[PutObject]
+    S3_CLIENT --> GET[GetObject]
+    S3_CLIENT --> LIST[ListObjects]
+    HTTP --> WORKS[Works Everywhere<br/>Node.js, Deno, Versori]
+    PUT --> WORKS2[Requires AWS SDK<br/>More features]
+    style PRESIGN fill:#e8f5e9
+    style AWS_SDK fill:#fff3e0
+    style WORKS fill:#e3f2fd
+```
+### Presigned URL Flow
+```mermaid
+sequenceDiagram
+    participant SDK
+    participant PresignService
+    participant S3
+    Note over SDK,S3: Upload Flow with Presigned URLs
+    SDK->>PresignService: getPresignedUploadUrl(bucket, key, expiry)
+    PresignService->>PresignService: Generate SigV4 signature
+    PresignService-->>SDK: Presigned PUT URL (valid 1 hour)
+    SDK->>S3: PUT {presigned_url} (file data)
+    S3-->>SDK: 200 OK
+    Note over SDK,S3: Download Flow with Presigned URLs
+    SDK->>PresignService: getPresignedDownloadUrl(bucket, key, expiry)
+    PresignService->>PresignService: Generate SigV4 signature
+    PresignService-->>SDK: Presigned GET URL (valid 1 hour)
+    SDK->>S3: GET {presigned_url}
+    S3-->>SDK: File data
+```
+**Presigned URL Benefits:**
+- No AWS credentials needed in application
+- Works in restricted environments (Versori, browser)
+- Temporary, secure access
+- Cross-account and cross-region support
+**Example:**
+```typescript
+const presignService = new S3PresignService({
+  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+  region: 'us-east-1',
+});
+// Upload
+const uploadUrl = await presignService.getPresignedUploadUrl(
+  'my-bucket',
+  'path/to/file.csv',
+  3600 // 1 hour
+);
+await fetch(uploadUrl, {
+  method: 'PUT',
+  body: fileData,
+});
+// Download
+const downloadUrl = await presignService.getPresignedDownloadUrl(
+  'my-bucket',
+  'path/to/file.csv',
+  3600
+);
+const response = await fetch(downloadUrl);
+const data = await response.text();
+```
+### S3 Operations
+```mermaid
+graph TB
+    S3_SOURCE[S3DataSource]
+    S3_SOURCE --> LIST[List Files]
+    S3_SOURCE --> DOWNLOAD[Download]
+    S3_SOURCE --> UPLOAD[Upload]
+    S3_SOURCE --> ARCHIVE[Archive]
+    LIST --> PREFIX[Prefix Filtering<br/>folder/]
+    style S3_SOURCE fill:#e3f2fd
+    style LIST fill:#fff3e0
+    style DOWNLOAD fill:#f3e5f5
+    style UPLOAD fill:#e8f5e9
+    style ARCHIVE fill:#fce4ec
+```
+- Note: S3 supports prefix filtering (e.g., `prefix: 'folder/'`). Wildcard patterns (e.g., `*.csv`) are not supported on S3.
+## SFTP Integration
+### Connection Management
+```mermaid
+sequenceDiagram
+    participant SDK
+    participant SftpDataSource
+    participant SSHClient
+    participant SftpServer
+    SDK->>SftpDataSource: Initialize connection
+    SftpDataSource->>SftpDataSource: Load SSH key or password
+    SftpDataSource->>SSHClient: Connect
+    SSHClient->>SftpServer: SSH handshake
+    alt Key-based Auth
+        SSHClient->>SftpServer: Authenticate with SSH key
+    else Password Auth
+        SSHClient->>SftpServer: Authenticate with password
+    end
+    SftpServer-->>SSHClient: Authentication success
+    SSHClient-->>SftpDataSource: Connection established
+    SDK->>SftpDataSource: listFiles(pattern)
+    SftpDataSource->>SftpServer: List directory
+    SftpServer-->>SftpDataSource: File list
+    SftpDataSource-->>SDK: Matching files
+    SDK->>SftpDataSource: downloadFile(path)
+    SftpDataSource->>SftpServer: Get file
+    SftpServer-->>SftpDataSource: File stream
+    SftpDataSource-->>SDK: File data
+    SDK->>SftpDataSource: Disconnect
+    SftpDataSource->>SSHClient: Close connection
+```
+**Configuration:**
+```typescript
+const sftp = new SftpDataSource(
+  {
+    /* host, auth, path, filePattern */
+  },
+  logger
+);
+// List files with wildcards
+const files = await sftp.listFiles({ remotePath: '/incoming', filePattern: '*.csv' });
+// Download file (Buffer or string depending on options)
+const content = await sftp.downloadFile('/incoming/file.csv');
+```
+### SFTP Operations
+```mermaid
+graph TB
+    SFTP[SftpDataSource]
+    SFTP --> CONN[Connection Pool]
+    SFTP --> LIST[List Files]
+    SFTP --> DOWN[Download]
+    SFTP --> UP[Upload]
+    SFTP --> DEL[Delete]
+    CONN --> REUSE[Reuse Connections]
+    CONN --> TIMEOUT[Auto-disconnect on Idle]
+    LIST --> FILTER[Pattern Filtering<br/>*.csv, *.xml]
+    DOWN --> STREAM[Streaming Support]
+    UP --> RETRY[Retry on Failure]
+    DEL --> SAFE[Safe Delete with Verification]
+    style SFTP fill:#e3f2fd
+    style CONN fill:#fff3e0
+    style LIST fill:#f3e5f5
+    style DOWN fill:#e8f5e9
+```
+## Versori Platform Integration
+### Platform Context Detection
+```mermaid
+flowchart TD
+    START[SDK Initialization] --> DETECT{Detect Context}
+    DETECT -->|Has 'activation'| VERSORI[Versori Platform]
+    DETECT -->|No 'activation'| STANDALONE[Standalone Runtime]
+    VERSORI --> PLATFORM_SERVICES[Platform Services]
+    STANDALONE --> MANUAL_CONFIG[Manual Configuration]
+    PLATFORM_SERVICES --> KV[KV Storage]
+    PLATFORM_SERVICES --> LOGGER[Platform Logger]
+    PLATFORM_SERVICES --> FETCH[Authenticated Fetch]
+    PLATFORM_SERVICES --> CONN[Connections]
+    MANUAL_CONFIG --> ENV[Environment Variables]
+    MANUAL_CONFIG --> CONFIG[Config Objects]
+    style START fill:#e3f2fd
+    style VERSORI fill:#e8f5e9
+    style STANDALONE fill:#fff3e0
+```
+### Versori Services Integration
+```mermaid
+graph TB
+    subgraph "Versori Platform"
+        HTTP[HTTP Webhook]
+        SCHEDULE[Scheduled Workflow]
+        KV_STORE[KV Storage]
+        CONNECTIONS[Connections]
+        LOGGING[Platform Logging]
+    end
+    subgraph "SDK Adapters"
+        KV_ADAPTER[VersoriKVAdapter]
+        FILE_TRACKER[VersoriFileTracker]
+        CLIENT[FluentVersoriClient]
+    end
+    HTTP --> CLIENT
+    SCHEDULE --> CLIENT
+    KV_STORE --> KV_ADAPTER
+    KV_ADAPTER --> FILE_TRACKER
+    CONNECTIONS --> CLIENT
+    LOGGING --> CLIENT
+    CLIENT --> FLUENT[Fluent Commerce]
+    style HTTP fill:#e3f2fd
+    style KV_STORE fill:#fff3e0
+    style CONNECTIONS fill:#f3e5f5
+    style CLIENT fill:#e8f5e9
+```
+### KV Storage Adapter
+```mermaid
+sequenceDiagram
+    participant SDK
+    participant VersoriKVAdapter
+    participant VersoriKV
+    SDK->>VersoriKVAdapter: Initialize with openKv()
+    VersoriKVAdapter->>VersoriKV: Open KV store
+    SDK->>VersoriKVAdapter: get('state')
+    VersoriKVAdapter->>VersoriKV: Read key
+    VersoriKV-->>VersoriKVAdapter: Value
+    VersoriKVAdapter-->>SDK: Parsed state object
+    SDK->>VersoriKVAdapter: set('state', newState)
+    VersoriKVAdapter->>VersoriKV: Write key
+    VersoriKV-->>VersoriKVAdapter: Confirmation
+    VersoriKVAdapter-->>SDK: Success
+    SDK->>VersoriKVAdapter: delete('old-key')
+    VersoriKVAdapter->>VersoriKV: Delete key
+    VersoriKV-->>VersoriKVAdapter: Deleted
+    VersoriKVAdapter-->>SDK: Success
+```
+**Usage:**
+```typescript
+import { VersoriKVAdapter, StateService } from '@fluentcommerce/fc-connect-sdk';
+// ✅ CORRECT: Access openKv from Versori context
+// import { openKv } from '@versori/run'; // ❌ WRONG - Not a direct export
+// In Versori workflow handler:
+const { openKv } = ctx;
+const kvAdapter = new VersoriKVAdapter(openKv(':project:'));
+const stateService = new StateService(logger); // logger is optional
+// Use state service for deduplication (kv is first parameter)
+const processed = await stateService.isFileProcessed(kvAdapter, 'file-key', 'my-workflow');
+```
+### Webhook Integration
+```mermaid
+sequenceDiagram
+    participant External
+    participant Versori
+    participant SDK
+    participant FluentAPI
+    External->>Versori: POST /webhook (data)
+    Versori->>Versori: Validate signature
+    Versori->>SDK: Trigger workflow
+    SDK->>SDK: Parse webhook payload
+    SDK->>SDK: Apply field mappings
+    SDK->>SDK: Build GraphQL mutation
+    SDK->>FluentAPI: Execute mutation
+    FluentAPI-->>SDK: Result
+    SDK-->>Versori: Response
+    Versori-->>External: 200 OK
+```
+**Webhook Example:**
+```typescript
+import { webhook } from '@versori/run';
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+export const processOrder = webhook('process-order', async ctx => {
+  const client = await createClient(ctx);
+  // Parse webhook payload
+  const orderData = await ctx.request.json();
+  // Transform and send to Fluent via GraphQL
+  const result = await client.graphql({
+    query: `mutation($input: CreateOrderInput!) { createOrder(input: $input) { id } }`,
+    variables: { input: orderData },
+  });
+  return { success: true, orderId: (result.data as any)?.createOrder?.id };
+});
+```
+## Cross-System Data Flows
+### Ingestion Flow (S3 → Fluent)
+```mermaid
+sequenceDiagram
+    participant S3
+    participant SDK
+    participant State
+    participant Fluent
+    SDK->>S3: List files
+    S3-->>SDK: [file1.csv, file2.csv]
+    loop For Each File
+        SDK->>State: isFileProcessed(file)
+        State-->>SDK: false
+        SDK->>S3: Download file
+        S3-->>SDK: File data
+        SDK->>SDK: Parse CSV
+        SDK->>SDK: Map fields
+        SDK->>Fluent: Create job
+        Fluent-->>SDK: Job ID
+        SDK->>Fluent: Send batches
+        Fluent-->>SDK: Batch IDs
+        SDK->>State: updateSyncState(kv, processedFiles)
+        SDK->>S3: Archive file
+    end
+```
+### Extraction Flow (Fluent → S3)
+```mermaid
+sequenceDiagram
+    participant SDK
+    participant Fluent
+    participant S3
+    SDK->>Fluent: Execute query (page 1)
+    Fluent-->>SDK: Results + cursor
+    loop Pagination
+        SDK->>Fluent: Fetch next page
+        Fluent-->>SDK: Results + cursor
+    end
+    SDK->>SDK: Transform data
+    SDK->>SDK: Apply mappings
+    SDK->>S3: Upload JSON
+    S3-->>SDK: Upload confirmation
+```
+### Transformation Flow (External → Fluent)
+```mermaid
+sequenceDiagram
+    participant External
+    participant SDK
+    participant Fluent
+    External->>SDK: POST webhook (XML/JSON)
+    SDK->>SDK: Parse input
+    SDK->>SDK: Load mapping config
+    SDK->>SDK: Build GraphQL mutation
+    SDK->>Fluent: Execute mutation
+    Fluent-->>SDK: Created entity
+    SDK-->>External: 200 OK (entity ID)
+```
+## Security Considerations
+### Authentication Security
+```mermaid
+graph TB
+    SECURITY[Security Measures]
+    SECURITY --> TOKEN[Token Management]
+    SECURITY --> CREDS[Credential Storage]
+    SECURITY --> NETWORK[Network Security]
+    TOKEN --> CACHE[In-memory Cache Only]
+    TOKEN --> REFRESH[Auto-refresh Before Expiry]
+    TOKEN --> EXPIRE[Short-lived Tokens]
+    CREDS --> ENV[Environment Variables]
+    CREDS --> VAULT[Secret Vaults]
+    CREDS --> NEVER[Never Hardcode]
+    NETWORK --> HTTPS[HTTPS Only]
+    NETWORK --> VALIDATE[Validate Certificates]
+    NETWORK --> TIMEOUT[Request Timeouts]
+    style SECURITY fill:#ffebee
+    style TOKEN fill:#fff3e0
+    style CREDS fill:#f3e5f5
+    style NETWORK fill:#e8f5e9
+```
+**Best Practices:**
+- Store credentials in environment variables or secret managers
+- Use short-lived OAuth2 tokens (auto-refreshed)
+- Enable HTTPS for all API communication
+- Implement request timeouts
+- Validate webhook signatures (Versori)
+- Use presigned URLs with expiration for S3
+**Versori Client Features:**
+- GraphQL auto-pagination parity with standalone client
+- Excludes internal fields from responses
+- Aligned endpoints (sendBatch uses POST, status endpoints)
+- Automatic retry/backoff for 5xx errors
+- Exposes `validateWebhook()` for signature validation
+## Next Steps
+- [State Management](./architecture-06-state-management.md) - Distributed state and deduplication
+- [Client Architecture](./architecture-02-client-architecture.md) - Client layer details
+- [Data Flow](./architecture-03-data-flow.md) - Data processing patterns