npm - @fluentcommerce/fc-connect-sdk - Versions diffs - 0.1.54 → 0.1.56 - Mend

@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (476) hide show

package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-inventory-positions-to-sftp-xml.md CHANGED Viewed

@@ -1,2529 +1,2529 @@
----
-template_id: tpl-extract-inventory-positions-to-sftp-xml
-canonical_filename: template-extraction-inventory-positions-to-sftp-xml.md
-version: 2.0.0
-sdk_version: ^0.1.39
-runtime: versori
-direction: extraction
-source: fluent-graphql
-destination: sftp-xml
-entity: inventoryPositions
-format: xml
-logging: versori
-status: stable
-features:
-  - memory-management
-  - enhanced-logging
-  - pagination-progress
-  - dispose-finally
----
-# Template: Extraction - Inventory Positions to SFTP XML
-**Template Version:** 2.0.0
-**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
-**Last Updated:** 2025-01-24
-**Deployment Target:** Versori Platform
-**🆕 Version 2.0.0 Enhancements:**
-- ✅ **Memory Management** - Clear large result sets after processing batches
-- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
-- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
-- ✅ **Resource Cleanup** - SFTP dispose in finally blocks prevents connection leaks
----
-## ?? STEP 1: Load These Docs Into Your AI (Human Checklist)
-1. REQUIRED (load all)
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-Copy-paste list (open these):
-fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
----
-## 📋 STEP 2: Tell Your AI (Prompt)
-Copy/paste this prompt into your AI tool after loading the documentation above:
-```
-I need a Versori scheduled extractor that:
-1) Queries Fluent Commerce GraphQL for Inventory Positions with auto-pagination
-2) Supports incremental runs via KV state (with an overlap buffer)
-3) Transforms results using UniversalMapper per mapping JSON
-4) Generates pretty-printed XML and uploads to SFTP
-5) Tracks progress with JobTracker and exposes a job-status webhook
-6) Uses native Versori log (LoggingService removed - use native log) and disposes SFTP
-Use the loaded docs to fill in SDK specifics and best practices.
-Keep the structure identical to the template; only adapt where needed.
-```
----
-## 📋 Template Overview
-This connector runs on the Versori platform. Most operational settings (Fluent account/connection, SFTP connection, schedule, page size/limits) are configured via activation variables. Data shape and logic (mapping JSON, XML structure, GraphQL selection set/filters, validators/resolvers) are adjusted in code as needed. It extracts inventory positions from Fluent Commerce via GraphQL, transforms the data into XML, and uploads the result to SFTP.
-### What This Template Does
-```
-�������������������������������������������������������������������
-�                    EXTRACTION WORKFLOW                           �
-�������������������������������������������������������������������
-1. TRIGGER
-   �� Scheduled (Cron): Runs automatically every hour
-   �� Ad hoc (Webhook): Manual trigger with optional date override
-   �� Status Query (Webhook): Check job progress
-2. EXTRACT (ExtractionOrchestrator)
-   �� Query Fluent GraphQL API for inventory positions
-   �� Auto-pagination (handles large datasets)
-   �� Apply date filters (incremental or manual range)
-   �� Validate each record (optional)
-3. TRANSFORM (UniversalMapper)
-   �� Map GraphQL fields to XML schema
-   �� Apply SDK resolvers (trim, uppercase, parseInt, etc.)
-   �� Extract nested data (catalogue)
-   �� Handle transformation errors
-4. GENERATE XML (XMLBuilder)
-   �� Convert transformed records to XML
-   �� Auto-escape special characters
-   �� Apply pretty-print formatting
-   �� Generate timestamped filename
-5. UPLOAD (SftpDataSource)
-   �� Connect to SFTP server
-   �� Upload XML file with retry logic
-   �� Verify upload success
-   �� Always dispose connection (finally block)
-6. TRACK JOB (JobTracker)
-   �� Create job with unique ID
-   �� Update status at each step
-   �� Store job result in KV
-   �� Enable status queries via webhook
-7. UPDATE STATE (VersoriKVAdapter)
-   �� Calculate max updatedOn from records
-   �� Store timestamp for next incremental run
-   �� Apply overlap buffer (prevent missed records)
-   �� Skip update if manual date override
-```
-### Key Features
-- Job tracking with status queries
-- Execution modes: scheduled, ad hoc, status query
-- Uses ExtractionOrchestrator, UniversalMapper, JobTracker
-- Error handling, retry logic, and SFTP cleanup
-- Reusable services suitable for similar use cases
-Note: JobTracker persists stage/status to Versori KV for visibility, job-status webhooks, and auditing. Recommended for production multi-step flows; can be skipped for trivial single-step utilities.
-### 📦 Package Information
-**SDK:** [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
-**Version:** Check [npm](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk) for latest version
-```bash
-npm install @fluentcommerce/fc-connect-sdk
-```
----
-**Templates are designed for direct deployment; customize via activation variables.**
----
-## 📦 SDK Imports (Verified - Versori Optimized)
-```typescript
-// ? VERIFIED IMPORTS - These match actual SDK exports
-import { Buffer } from 'node:buffer';
-import {
-  createClient, // Universal client factory (auto-detects Versori/Node/Deno)
-  ExtractionOrchestrator, // High-level extraction with auto-pagination
-  JobTracker, // Job status tracking in KV store
-  UniversalMapper, // Field mapping with SDK resolvers
-  XMLBuilder, // XML generation with auto-escaping (from SDK)
-  SftpDataSource, // SFTP operations with retry logic
-} from '@fluentcommerce/fc-connect-sdk';
-// Versori platform imports
-import { schedule, webhook, http, fn } from '@versori/run';
-```
-**CRITICAL:** `openKv` is NOT imported directly - it's accessed from the context parameter:
-```typescript
-// ✅ CORRECT - From context parameter
-export const myWorkflow = schedule('task', '0 * * * *', async (ctx) => {
-  const { openKv, log } = ctx;  // Access from context
-  const kv = openKv(':project:');
-});
-```
-**Note:** `openKv` is NOT exported from `@versori/run` - attempting to import it directly will cause a runtime error.
-**Note:** All imports are from actual SDK exports - this code compiles and runs as-is.
-**? VERSORI PLATFORM - Use Native Logs:**
-- Use `log` from context: `const { log } = ctx;`
-- Don't import or use LoggingService for Versori connectors
-- Native Versori logs are simpler and automatically integrated with platform monitoring
----
-## ??� Configuration
-### SFTP Connection Setup
-**? RECOMMENDED APPROACH:** Store SFTP credentials in a Versori connection for security and reusability.
-Versori provides **three methods** to access connection credentials. Choose based on your use case:
-#### Method 1: `activation.connections` (Recommended - Simplest)
-**When to use:** Most workflows (scheduled, webhooks without connection param)
-**Setup:**
-1. Create connection named `versori_ftp_server` with Basic Auth
-2. Access via `activation.connections`:
-```typescript
-// ========================================
-    // SFTP CREDENTIAL RETRIEVAL
-    // ========================================
-    /**
-     * Retrieve SFTP credentials from connection configuration
-     *
-     * This approach retrieves credentials stored in the Versori connection settings.
-     * The connection must be configured in the UI with Basic Authentication.
-     *
-     * Steps:
-     * 1. Call ctx.credentials().getAccessToken('SFTP') to get base64-encoded credentials
-     * 2. Decode the accessToken from base64 to get "username:password" string
-     * 3. Split on ':' to extract username and password
-     *
-     * Connection Name: 'SFTP' (must match the connection name in Versori UI)
-     * Auth Type: Basic Authentication (username + password)
-     *
-     * This method provides:
-     * - Centralized credential management through Versori UI
-     * - Better security (credentials not stored in integration variables)
-     * - Easier credential rotation and updates
-     */
-    log.info('Retrieving SFTP credentials from connection configuration');
-    let sftpUsername: string;
-    let sftpPassword: string;
-    try {
-      // Retrieve credentials from the 'SFTP' connection
-      const sftpCred = await ctx.credentials().getAccessToken('SFTP');
-      if (!sftpCred?.accessToken) {
-        throw new Error('No SFTP credentials found in connection configuration');
-      }
-      // Decode base64 accessToken to get "username:password"
-      const rawBasicAuth = Buffer.from(sftpCred.accessToken, 'base64').toString('utf-8');
-      // Split on ':' to extract username and password
-      const parts = rawBasicAuth.split(':');
-      if (parts.length !== 2) {
-        throw new Error('Invalid SFTP credential format - expected username:password');
-      }
-      sftpUsername = parts[0];
-      sftpPassword = parts[1];
-      log.info('SFTP credentials retrieved successfully', {
-        hasUsername: !!sftpUsername,
-        hasPassword: !!sftpPassword,
-        usernameLength: sftpUsername.length,
-        passwordLength: sftpPassword.length,
-      });
-    } catch (error: any) {
-      log.error('Failed to retrieve SFTP credentials', {
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',,
-      });
-      return {
-        success: false,
-        error: 'Failed to retrieve SFTP credentials from connection configuration',
-        details: error?.message,
-        recommendation: 'Please ensure the SFTP connection is configured in the Connections section with Basic Authentication (username and password)',
-      };
-    }
-```
-**Pros:**
-- ✅ Dynamic connection name from config/environment
-- ✅ Explicit error handling for missing connections
-- ✅ Works in all workflow types
-**Cons:**
-- ⚠️ Async (requires `await`)
-- ⚠️ More verbose than `activation.connections`
-#### Method 3: `connections` from Context (Workflow-Scoped)
-**When to use:** Webhook with `connection` parameter (connection passed to specific step)
-**Note:** This method is different from `activation.connections` - it only works when the workflow step explicitly receives a `connection` parameter.
-**Setup:**
-1. Create connection named `versori_ftp_server`
-2. Pass to workflow step:
-```typescript
-export const myWebhook = webhook('my-webhook', { connection: 'versori_ftp_server' }).then(
-  http('process', { connection: 'versori_ftp_server' }, async ctx => {
-    // Get SFTP credentials from Versori connection (Basic Auth)
-    // RECOMMENDED: Use activation.connections (already decoded)
-    const allConnections = ctx.activation.connections || [];
-    const sftpConn = allConnections.find(c => c.name === 'versori_ftp_server');
-    if (!sftpConn) {
-      throw new Error('SFTP connection "versori_ftp_server" not found');
-    }
-    const credential = sftpConn.credentials[0]?.credential;
-    if (!credential?.data?.basicAuth) {
-      throw new Error('SFTP connection not configured with Basic Authentication');
-    }
-    const { username, password } = credential.data.basicAuth;
-    // ? Already decoded - no Buffer.from() needed!
-  })
-);
-```
-**Pros:**
-- ✅ Explicit connection passing to specific steps
-- ✅ Connection validation at workflow definition
-**Cons:**
-- ⚠️ Only works with `connection` parameter in workflow definition
-- ⚠️ Not available in scheduled workflows or webhooks without connection param
-#### Quick Decision Guide
-| Scenario                            | Method                     | Example                                                             |
-| ----------------------------------- | -------------------------- | ------------------------------------------------------------------- |
-| **Scheduled workflow**              | `activation.connections`   | `schedule().then(http(...))`                                        |
-| **Webhook (no connection param)**   | `activation.connections`   | `webhook().then(fn(...))`                                           |
-| **Webhook (with connection param)** | `connections` from context | `webhook({ connection: 'x' }).then(http({ connection: 'x' }, ...))` |
-| **Dynamic connection name**         | `credentials().get()`      | `await credentials().get(connName)`                                 |
-| **Need error handling**             | `credentials().get()`      | Explicit null check                                                 |
-**Connection Configuration:**
-1. In Versori platform, create connection named `versori_ftp_server`
-2. Set **Authentication Type**: `Basic Auth`
-3. Enter **Username**: Your SFTP username
-4. Enter **Password**: Your SFTP password
-5. Connection auto-encodes credentials as base64 Basic Auth
-**Why use connections instead of activation variables?**
-- ✅ Credentials stored securely in Versori vault
-- ✅ Connection can be reused across workflows
-- ✅ No need to manage sensitive data in activation variables
-- ✅ Easier credential rotation
-- ✅ Better separation of concerns (config vs secrets)
-- ✅ Follows industry security best practices
-**?? Complete Guide:** See `docs/02-CORE-GUIDES/data-sources/sftp-credential-access-security.md` for:
-- Detailed comparison of all three methods
-- Security best practices
-- Common pitfalls and solutions
-- Migration examples
-- Troubleshooting connection issues
-### Activation Variables
-**Configuration is driven by activation variables - modify these instead of code:**
-```json
-{
-  "sftpHost": "sftp.partner.com",
-  "sftpPort": 22,
-  "sftpPrivateKey": "",
-  "sftpPath": "/incoming/inventory/",
-  "fileNamePrefix": "inventorypositions",
-  "pageSize": 200,
-  "maxRecords": 100000,
-  "overlapBufferSeconds": 60,
-  "requireAbsolutePaths": "true"
-}
-```
-> **Note:** `sftpUsername` and `sftpPassword` are fetched from the `versori_ftp_server` connection (see above).
-Note: For webhook security, set `webhookApiKey` (activation variable) and provide it via `X-API-Key` header on webhook calls.
-### Variable Explanations
-| Variable                    | Purpose                      | Default                | Customization Hints                                              |
-| --------------------------- | ---------------------------- | ---------------------- | ---------------------------------------------------------------- |
-| **SFTP Credentials**        | _From Connection_            |                        | _See connection setup above_                                     |
-| `sftpHost`                  | SFTP server hostname         | -                      | Required - your SFTP server                                      |
-| `sftpPort`                  | SFTP server port             | `22`                   | Usually 22, sometimes 2222                                       |
-| `sftpPrivateKey`            | SSH private key (optional)   | -                      | Alternative to password auth                                     |
-| `sftpPath`                  | Remote directory path        | `/incoming/inventory/` | Where to upload XML files                                        |
-| `fileNamePrefix`            | XML filename prefix          | `inventorypositions`   | Customize naming convention                                      |
-| `pageSize`                  | Records per GraphQL page     | `200`                  | Increase for fewer API calls                                     |
-| `maxRecords`                | Total extraction limit       | `100000`               | Safety limit - adjust for volume                                 |
-| `overlapBufferSeconds`      | Incremental safety window    | `60`                   | Prevents missed records                                          |
-| `requireAbsolutePaths`      | Require absolute SFTP paths  | `"true"`               | `"true"` for AWS Transfer Family, `"false"` for standard OpenSSH |
-| `validateConnectionOnStart` | Fail-fast connection testing | `"false"`              | `"true"` to validate auth before extraction (optional)           |
-**🆕 New Activation Variables (v1.1.0):**
-- **`validateConnectionOnStart`**: When set to `"true"`, validates Fluent Commerce connection before extraction starts. Executes `query { me { ref } }` to verify authentication. Default: `"false"` (validation happens on first API call). Use for fail-fast behavior in production environments.
----
-### ?? State Management & Incremental Sync
-**How incremental sync works:**
-1. **First Run:** Uses `DEFAULT_FALLBACK` date (2024-01-01T00:00:00Z)
-2. **Subsequent Runs:** Uses `lastInventoryPositionSync` timestamp from KV store
-3. **Overlap Buffer:** Subtracts 60 seconds to catch late-arriving records
-4. **State Update:** After successful upload, stores max `updatedOn` for next run
-**Incremental vs Manual Modes:**
-| Mode                        | When to Use          | State Update | Payload Example                                                |
-| --------------------------- | -------------------- | ------------ | -------------------------------------------------------------- |
-| **Incremental**             | Daily scheduled sync | ? Yes       | `{}` (empty - uses last sync)                                  |
-| **Manual Range**            | Historical backfill  |  No       | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": false }` |
-| **Manual Range with State** | One-time catch-up    | ? Yes       | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": true }`  |
-**Why overlap buffer?**
-Records updated near the sync time might not appear in the query due to:
-- Clock drift between systems
-- Transaction timing in the database
-- GraphQL query execution timing
-The 60-second buffer ensures these edge-case records are captured in the next run, preventing data loss.
-**When to skip state update (`updateState: false`):**
-- Historical backfills (don't affect ongoing incremental sync)
-- Testing/debugging specific date ranges
-- Reprocessing old data without changing the sync pointer
----
-### 📁 SFTP Path Configuration
-**Note:** Folder paths and filename patterns are configured separately.
-- **Folder Path:** Configured via `sftpPath` activation variable (e.g., `/incoming/inventory/`)
-- **Filename Pattern:** Configured via `fileNamePrefix` activation variable (e.g., `inventorypositions`)
-**The workflow generates files like:** `{sftpPath}/{fileNamePrefix}-{timestamp}.xml`
-**Example:** With `sftpPath="/incoming/inventory/"` and `fileNamePrefix="inventorypositions"`, a generated file will look like:
-```
-/incoming/inventory/inventorypositions-2025-10-27T18-30-45Z.xml
-```
-Format may vary slightly (colons replaced; includes Z).
-**Note:** To change the upload folder, modify the `sftpPath` activation variable (not in code).
----
-### Auto-pagination and limits (ExtractionOrchestrator)
-**What:** ExtractionOrchestrator handles GraphQL Relay cursor-based pagination automatically.
-**Why:** Prevents manual pagination loop code, handles large datasets efficiently.
-**How:** You configure `pageSize` and `maxRecords`; the orchestrator injects `$first` and `$after` variables automatically and loops until `pageInfo.hasNextPage === false` or `maxRecords` is reached.
-**Critical:** Your query MUST include `edges { cursor }` and `pageInfo { hasNextPage }` fields, or pagination will fail.
-#### How Auto-Pagination Works (Step-by-Step)
-Note: Orchestrator injects $first/$after automatically, flattens via resultPath, and continues while pageInfo.hasNextPage and total < maxRecords.
-#### Configuration Parameters
-| Parameter        | Purpose                        | Example                           | Effect                     |
-| ---------------- | ------------------------------ | --------------------------------- | -------------------------- |
-| `pageSize`       | Records per GraphQL request    | `200`                             | Controls `first` variable  |
-| `maxRecords`     | Total extraction limit         | `100000`                          | Hard stop across all pages |
-| `resultPath`     | Where records live in response | `"inventoryPositions.edges.node"` | Flattening path            |
-| `validateItem()` | Optional record filter         | `(item) => !!item.ref`            | Skips invalid records      |
-#### Example Configuration
-```typescript
-const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
-const maxRecords = parseInt(activation.getVariable('maxRecords') || '100000', 10);
-const extractionResult = await orchestrator.extract({
-  query: INVENTORY_POSITIONS_EXTRACTION_QUERY,
-  resultPath: 'inventoryPositions.edges.node',
-  variables: {
-    dateRangeFilter,
-    // Note: Don't include 'first' or 'after' here; orchestrator injects them based on pageSize.
-  },
-  pageSize,
-  maxRecords,
-  validateItem: item => !!(item.ref && item.productRef && typeof item.onHand === 'number'),
-});
-// Stats available after extraction (Versori)
-log.info('Extraction stats', {
-  totalRecords: extractionResult.stats.totalRecords,
-  totalPages: extractionResult.stats.totalPages,
-  validRecords: extractionResult.stats.validRecords ?? extractionResult.data.length,
-  failedValidations: extractionResult.stats.failedValidations,
-});
-```
-#### GraphQL Query Requirements
-**Your query MUST include these pagination fields:**
-```graphql
-query GetInventoryPositions(
-  $dateRangeFilter: DateRange
-  $first: Int! # ?� Orchestrator injects this
-  $after: String # ?� Orchestrator injects this
-) {
-  inventoryPositions(
-    updatedOn: $dateRangeFilter
-    first: $first # ?� Pagination page size
-    after: $after # ?� Pagination cursor
-  ) {
-    edges {
-      # ?� REQUIRED: Relay connection structure
-      node {
-        # ?� REQUIRED: Actual records here
-        id
-        ref
-        # ... your fields
-      }
-      cursor # ?� REQUIRED: Orchestrator uses this for next page
-    }
-    pageInfo {
-      # ?� REQUIRED: Orchestrator checks this
-      hasNextPage # ?� REQUIRED: Tells orchestrator to continue
-    }
-  }
-}
-```
----
-## 📄 Mapping Configuration
-**File:** `config/inventory-positions.export.xml.json`
-```json
-{
-  "name": "inventory-positions.export.xml",
-  "version": "1.0.0",
-  "description": "Inventory Positions ? XML Export Mapping",
-  "fields": {
-    "position_ref": {
-      "source": "ref",
-      "required": true,
-      "resolver": "sdk.trim"
-    },
-    "product_ref": {
-      "source": "productRef",
-      "required": true,
-      "resolver": "sdk.trim"
-    },
-    "location_ref": {
-      "source": "locationRef",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "on_hand": {
-      "source": "onHand",
-      "required": true,
-      "resolver": "sdk.parseInt"
-    },
-    "position_type": {
-      "source": "type",
-      "required": true,
-      "resolver": "sdk.uppercase"
-    },
-    "status": {
-      "source": "status",
-      "required": false,
-      "resolver": "sdk.uppercase"
-    },
-    "catalogue_ref": {
-      "source": "catalogue.ref",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "catalogue_name": {
-      "source": "catalogue.name",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "created_on": {
-      "source": "createdOn",
-      "required": true,
-      "resolver": "sdk.toString"
-    },
-    "updated_on": {
-      "source": "updatedOn",
-      "required": true,
-      "resolver": "sdk.toString"
-    }
-  }
-}
-```
-**AI Customization Hints:**
-- Add fields: Copy existing field config, change `source` path
-- Remove fields: Delete field from config
-- Change resolvers: Replace `sdk.trim` with `sdk.uppercase`, etc.
-- Nested fields: Use dot notation like `catalogue.ref`
-Note: Customize mapping by editing the JSON above; prefer built-in resolvers. See SDK Universal Mapping guide for advanced usage.
----
-## Versori Workflows Structure
-**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
-**Trigger Types:**
-- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
-- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
-- **`workflow()`** → Durable workflows (advanced, rarely used)
-**Execution Steps (chained to triggers):**
-- **`http()`** → External API calls (chained from schedule/webhook)
-- **`fn()`** → Internal processing (chained from schedule/webhook)
-### Recommended Project Structure
-```
-inventory-positions-extraction/
-├── index.ts                              # Entry point - exports all workflows
-└── src/
-    ├── workflows/
-    │   ├── scheduled/
-    │   │   └── daily-inventory-positions-extraction.ts         # Scheduled workflow
-    │   └── webhook/
-    │       ├── adhoc-inventory-positions-extraction.ts         # Webhook: Manual trigger
-    │       └── job-status-check.ts                             # Webhook: Status query
-    │
-    ├── services/
-    │   └── inventory-positions-extraction.service.ts    # Shared orchestration logic (reusable)
-    │
-    └── config/
-        └── inventory-positions.export.xml.json          # Mapping configuration
-```
----
-### 1. Entry Point (`index.ts`)
-**Purpose**: Register all workflows with Versori platform
-**Pattern**: Use simple re-exports (NOT `MemoryInterpreter` pattern)
-```typescript
-/**
- * Entry point - Export all workflows for Versori platform
- *
- * This file exports all workflows to be registered with Versori.
- * Each workflow is defined in its own file for better organization.
- */
-// Scheduled workflows
-export { scheduledInventoryPositionsExtraction } from './src/workflows/scheduled/daily-inventory-positions-extraction';
-// Webhook workflows
-export { adhocInventoryPositionsExtraction } from './src/workflows/webhook/adhoc-inventory-positions-extraction';
-export { inventoryPositionsJobStatus } from './src/workflows/webhook/job-status-check';
-```
-**IMPORTANT**: Do NOT use `MemoryInterpreter` or complex patterns in `index.ts`. Simple re-exports are the Versori standard.
-### 2. Workflow Definitions
-The code examples below show the three workflow types that go in separate files:
-- `src/workflows/scheduled/daily-inventory-positions-extraction.ts` - Scheduled workflow
-- `src/workflows/webhook/adhoc-inventory-positions-extraction.ts` - Webhook: Manual trigger
-- `src/workflows/webhook/job-status-check.ts` - Webhook: Status query
-### 3. Main Orchestration Service (`src/services/inventory-positions-extraction.service.ts`)
-**Purpose**: Shared orchestration logic reused by all workflows
-**Note:** This service coordinates the complete extraction workflow:
-- Initialize clients and services
-- Determine date range (incremental vs manual)
-- Extract data using ExtractionOrchestrator
-- Transform using UniversalMapper
-- Generate XML using XMLBuilder
-- Upload to SFTP
-- Track job progress with JobTracker
-- Update state for next run
-**NAMING PATTERN** (consistent across all use cases):
-- Interface: `{Entity}ExtractionParams` (e.g., `InventoryPositionExtractionParams`)
-- Function: `execute{Entity}Extraction` (e.g., `executeInventoryPositionExtraction`)
-- Service file: `{entity}-extraction.service.ts` (e.g., `inventory-positions-extraction.service.ts`)
-```typescript
-/**
- * Workflows - Defines 3 execution patterns for inventory positions extraction
- *
- * WORKFLOW 1: Scheduled (Cron) - Runs automatically every hour
- * WORKFLOW 2: Ad hoc (Webhook) - Manual trigger with optional date override
- * WORKFLOW 3: Job Status (Webhook) - Query job progress
- *
- * AI CUSTOMIZATION HINTS:
- * - Change schedule: Modify cron expression in schedule()
- * - Add filtering: Pass additional params to executeInventoryPositionExtraction()
- * - Change response format: Modify return object structure
- */
-import { schedule, webhook, http, fn } from '@versori/run';
-import { executeInventoryPositionExtraction, getJobStatus } from '../services/extraction-orchestration';
-import { generateJobId, extractFileName } from '../utils/job-id-generator';  // We'll create this
-/**
- * WORKFLOW 1: Scheduled Extraction
- *
- * Purpose: Automated hourly extraction for incremental sync
- * Trigger: Cron schedule (every hour at minute 0)
- * State Update: Always updates lastSync timestamp
- *
- * AI CUSTOMIZATION:
- * - Change schedule: Replace '0 * * * *' with your cron expression
- *   Examples:
- *   - Every 30 min: '*/30 * * * *'
- *   - Daily at 2 AM: '0 2 * * *'
- *   - Every 15 min: '*/15 * * * *'
- */
-export const scheduledInventoryPositionsExtraction = schedule('scheduled-inventory-positions-extraction', '0 * * * *')  // ? CUSTOMIZE: Cron expression
-.then(
-  http('execute-scheduled-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
-    const { log } = ctx;
-    // Generate unique job ID for tracking
-    // Format: SCHEDULED_IP_YYYYMMDD_HHMMSS_random
-    const jobId = generateJobId('SCHEDULED', 'INVENTORY_POSITIONS');
-    log.info('Scheduled extraction triggered', { jobId });
-    try {
-      // Execute main workflow (extraction ? transform ? upload)
-      const result = await executeInventoryPositionExtraction(ctx, {
-        jobId,
-        triggeredBy: 'schedule',
-        updateState: true,        // Always update state for scheduled runs
-        // positionTypes: ['DEFAULT', 'SEASONAL'],
-        // locationRefs: ['LOC-001'],
-      });
-      log.info('Scheduled extraction completed', {
-        jobId,
-        recordCount: result.recordsExtracted,
-        fileName: result.fileName
-      });
-      return result;
-    } catch (error: any) {
-      log.error('Scheduled extraction failed', {
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error'
-      });
-      throw error;
-    }
-  })
-);
-/**
- * WORKFLOW 2: Ad hoc Extraction (Manual Trigger)
- *
- * Purpose: Manual extraction with optional date range override
- * Trigger: Webhook POST to /webhooks/inventory-positions-adhoc
- * State Update: Optional (controlled by request payload)
- *
- * WEBHOOK PAYLOAD EXAMPLES:
- *
- * 1. Incremental (use last sync timestamp):
- *    {}
- *
- * 2. Date range (manual override):
- *    {
- *      "fromDate": "2025-01-01T00:00:00Z",
- *      "toDate": "2025-01-31T23:59:59Z",
- *      "updateState": false
- *    }
- *
- * 3. Specific time window:
- *    {
- *      "fromDate": "2025-01-15T08:00:00Z",
- *      "toDate": "2025-01-15T17:00:00Z",
- *      "updateState": false
- *    }
- *
- * AI CUSTOMIZATION:
- * - Add request validation
- * - Add authentication check
- * - Add custom filters from payload
- */
-export const adhocInventoryPositionsExtraction = webhook(
-  'inventory-positions-adhoc',
-  { connection: 'inventory-positions-adhoc', response: { mode: 'sync' } }
-)
-.then(
-  http('execute-adhoc-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
-    const { data, log } = ctx;
-    // Generate unique job ID
-    const jobId = generateJobId('ADHOC', 'INVENTORY_POSITIONS');
-    // Security handled by Versori connection
-    // Extract optional date override from webhook payload
-    const fromDate = data.fromDate as string | undefined;
-    const toDate = data.toDate as string | undefined;
-    const updateState = data.updateState === true;  // Default false; advance state only if explicitly true
-    log.info('Ad hoc extraction triggered via webhook', {
-      jobId,
-      hasDateOverride: !!fromDate,
-      fromDate: fromDate || 'not specified',
-      toDate: toDate || 'not specified',
-      updateState
-    });
-    try {
-      // Execute main workflow with optional overrides
-      const result = await executeInventoryPositionExtraction(ctx, {
-        jobId,
-        triggeredBy: 'webhook',
-        fromDate,              // Optional: override start date
-        toDate,                // Optional: override end date
-        updateState,           // Optional: skip state update for historical queries
-        // positionTypes: data.positionTypes,
-        // locationRefs: data.locationRefs,
-      });
-      log.info('Ad hoc extraction completed', {
-        jobId,
-        recordCount: result.recordsExtracted,
-        fileName: result.fileName,
-        isManualOverride: !!fromDate,
-        stateUpdated: result.stateUpdated
-      });
-      // Return success with job details
-      return {
-        success: true,
-        jobId,
-        recordsExtracted: result.recordsExtracted,
-        fileName: result.fileName,
-        sftpPath: result.sftpPath,
-        statusUrl: `/webhooks/inventory-positions-job-status?jobId=${jobId}`,
-        dateRange: fromDate ? {
-          from: fromDate,
-          to: toDate || 'not specified',
-          updateState
-        } : undefined
-      };
-    } catch (error: any) {
-      log.error('Ad hoc extraction failed', {
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error'
-      });
-      return {
-        success: false,
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',
-      };
-    }
-  })
-);
-/**
- * WORKFLOW 3: Job Status Query
- *
- * Purpose: Check job progress and status
- * Trigger: Webhook GET/POST to /webhooks/inventory-positions-job-status?jobId=xxx
- * Returns: Current job status, stage, progress
- *
- * QUERY EXAMPLES:
- *
- * 1. HTTP GET:
- *    GET /webhooks/inventory-positions-job-status?jobId=ADHOC_IP_20251027_183045_abc123
- *
- * 2. HTTP POST:
- *    POST /webhooks/inventory-positions-job-status
- *    { "jobId": "ADHOC_IP_20251027_183045_abc123" }
- *
- * RESPONSE EXAMPLE:
- * {
- *   "success": true,
- *   "jobId": "ADHOC_IP_20251027_183045_abc123",
- *   "status": "processing",
- *   "stage": "transforming",
- *   "message": "Transforming 15000 records",
- *   "createdAt": "2025-10-27T18:30:45.000Z",
- *   "startedAt": "2025-10-27T18:30:46.000Z"
- * }
- *
- * AI CUSTOMIZATION:
- * - Add detailed progress percentage
- * - Add estimated time remaining
- * - Add historical job queries
- */
-export const inventoryPositionsJobStatus = webhook(
-  'inventory-positions-job-status',
-  { connection: 'inventory-positions-job-status', response: { mode: 'sync' } }
-)
-.then(
-  fn('query-job-status', async (ctx) => {
-    const { data, log, openKv } = ctx;
-    // Get jobId from query param or POST body
-    const jobId = data.jobId as string;
-    if (!jobId) {
-      log.error('Job ID not provided in request');
-      return {
-        success: false,
-        error: 'Job ID is required. Provide jobId in query param or request body.'
-      };
-    }
-    log.info('Querying job status', { jobId });
-    try {
-      // Query job status from KV store
-      const status = await getJobStatus(openKv(':project:'), jobId, log);
-      if (!status) {
-        log.info('Job not found', { jobId });
-        return {
-          success: false,
-          error: 'Job not found',
-          jobId
-        };
-      }
-      log.info('Job status retrieved', { jobId, status: status.status });
-      return {
-        success: true,
-        jobId,
-        ...status
-      };
-    } catch (error: any) {
-      log.error('Failed to query job status', {
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',
-      });
-      return {
-        success: false,
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',
-      };
-    }
-  })
-);
-```
----
-# Production Template: Service Implementation
-## 3. Main Orchestration Service (src/services/extraction-orchestration.ts)
-```typescript
-/**
- * MAIN ORCHESTRATION SERVICE
- *
- * This is the heart of the extraction workflow. It coordinates all steps:
- * 1. Initialize clients and services
- * 2. Determine date range (incremental vs manual)
- * 3. Extract data using ExtractionOrchestrator
- * 4. Transform using UniversalMapper
- * 5. Generate XML using XMLBuilder
- * 6. Upload to SFTP
- * 7. Track job progress with JobTracker
- * 8. Update state for next run
- *
- * NAMING PATTERN (consistent across all use cases):
- * - Interface: {Entity}ExtractionParams (e.g., InventoryPositionExtractionParams)
- * - Result: {Entity}ExtractionResult (e.g., InventoryPositionExtractionResult)
- * - Main function: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
- *
- * AI CUSTOMIZATION HINTS:
- * - Change entity: Replace "InventoryPosition" with "VirtualPosition", "Order", etc.
- * - Change output: Replace XMLBuilder with CSVParserService.stringify()
- * - Change destination: Replace SftpDataSource with S3DataSource
- * - Add steps: Insert new service calls between existing steps
- */
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  XMLBuilder,
-  SftpDataSource,
-  type ExtractionOptions,
-  type ExtractionResult,
-  type JobStatus,
-} from '@fluentcommerce/fc-connect-sdk';
-import mappingConfig from '../../config/inventory-positions.export.xml.json' with { type: 'json' };
-import { extractFileName } from '../../utils/job-id-generator';
-// ? VERSORI PLATFORM: Use native log from context, not LoggingService
-/**
- * Parameters for extraction workflow
- *
- * NAMING: {Entity}ExtractionParams
- */
-export interface InventoryPositionExtractionParams {
-  jobId: string;
-  triggeredBy: 'schedule' | 'webhook';
-  fromDate?: string; // Optional: manual date override
-  toDate?: string; // Optional: manual date override
-  updateState: boolean; // Whether to update lastSync timestamp
-  // AI CUSTOMIZATION: Add filters specific to entity
-  positionTypes?: string[]; // e.g., ['DEFAULT', 'SEASONAL']
-  locationRefs?: string[]; // e.g., ['LOC-001']
-}
-/**
- * Result from extraction workflow
- *
- * NAMING: {Entity}ExtractionResult
- */
-export interface InventoryPositionExtractionResult {
-  success: boolean;
-  jobId: string;
-  recordsExtracted: number;
-  fileName?: string;
-  sftpPath?: string;
-  error?: string;
-  errors?: any[];
-  isManualOverride?: boolean;
-  stateUpdated?: boolean;
-  newTimestamp?: string;
-}
-/**
- * GraphQL Query for Inventory Positions
- *
- * NAMING: {ENTITY}_EXTRACTION_QUERY (uppercase constant)
- *
- * Replace entity/fields as needed but keep pagination structure (edges, node, pageInfo)
- */
-const INVENTORY_POSITIONS_EXTRACTION_QUERY = `
-  query GetInventoryPositions(
-    $dateRangeFilter: DateRange
-    $productRefs: [String!]
-    $types: [String!]
-    $locationRefs: [String]
-    $first: Int!
-    $after: String
-  ) {
-    inventoryPositions(
-      updatedOn: $dateRangeFilter
-      productRef: $productRefs
-      type: $types
-      locationRef: $locationRefs
-      first: $first
-      after: $after
-    ) {
-      edges {
-        node {
-          id
-          ref
-          productRef
-          locationRef
-          onHand
-          type
-          status
-          catalogue {
-            ref
-            name
-          }
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-/**
- * Query job status from KV store
- *
- * NAMING: get{Entity}JobStatus or just getJobStatus (generic)
- *
- * ? VERSORI PLATFORM: Uses native log from context (passed as parameter)
- */
-export async function getJobStatus(
-  kv: any, // ? Versori KV (compatible with JobTracker's KVAdapter interface)
-  jobId: string,
-  log: any // ? Native Versori log from context
-): Promise<JobStatus | undefined> {
-  try {
-    const tracker = new JobTracker(kv, log);
-    return await tracker.getJob(jobId); // ? Use getJob() not getJobStatus()
-  } catch (error: any) {
-    log.error('Failed to get job status', { jobId, message: error instanceof Error ? error.message : String(error),
- stack: error instanceof Error ? error.stack : undefined,
- errorType: error instanceof Error ? error.constructor.name : 'Error', });
-    return undefined;
-  }
-}
-/**
- * MAIN ORCHESTRATION FUNCTION
- *
- * NAMING: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
- *
- * This function implements the complete workflow in steps.
- * Each step is clearly commented for AI understanding.
- */
-export async function executeInventoryPositionExtraction(
-  ctx: any,
-  params: InventoryPositionExtractionParams
-): Promise<InventoryPositionExtractionResult> {
-  // ? VERSORI PLATFORM: Extract native log from context
-  const { log, openKv, activation, credentials } = ctx;
-  const { jobId, triggeredBy, fromDate, toDate, updateState } = params;
-  // ⏱️ Track total execution time
-  const startTime = Date.now();
-  // Open KV store for state management and job tracking
-  // ? Pass raw Versori KV directly - it matches KVAdapter interface
-  // ? Pass native log to JobTracker
-  const kv = openKv(':project:');
-  const tracker = new JobTracker(kv, log);
-  try {
-    // �����������������������������������������������������������
-    // STEP 1/8: Initialize Job Tracking
-    // �����������������������������������������������������������
-    log.info('📊 [STEP 1/8] Initializing job tracking', { jobId });
-    await tracker.createJob(jobId, {
-      triggeredBy,
-      hasDateOverride: !!fromDate,
-      fromDate,
-      toDate,
-      updateStateAfterRun: updateState,
-    });
-    // �����������������������������������������������������������
-    // STEP 2/8: Initialize Fluent Client
-    // �����������������������������������������������������������
-    log.info('🔌 [STEP 2/8] Initializing Fluent Commerce client', { jobId });
-    // ✅ Optional: Validate connection immediately (fail-fast mode)
-    // Set activation variable 'validateConnectionOnStart' = 'true' to enable
-    // When enabled: Executes query { me { ref } } to verify authentication
-    // When disabled: Fast creation, validation happens on first API call (default)
-    const validateConnection = activation.getVariable('validateConnectionOnStart') === 'true';
-    const client = await createClient(ctx, validateConnection ? { validateConnection: true } : undefined);
-    if (!client) {
-      throw new Error('Failed to create Fluent Commerce client');
-    }
-    if (validateConnection) {
-      log.info('✅ Connection validated successfully - authentication confirmed', { jobId });
-    }
-    // �����������������������������������������������������������
-    // STEP 3/8: Determine Date Range (WITH overlap buffer)
-    // �����������������������������������������������������������
-    log.info('📅 [STEP 3/8] Determining date range for extraction', { jobId });
-    // State key for incremental sync tracking
-    // NAMING: last{Entity}Sync (e.g., lastInventoryPositionSync)
-    const STATE_KEY = 'lastInventoryPositionSync';
-    const DEFAULT_FALLBACK = '2024-01-01T00:00:00Z';
-    const OVERLAP_BUFFER_SECONDS = parseInt(
-      activation.getVariable('overlapBufferSeconds') || '60',
-      10
-    );
-    let dateRangeFilter: { from?: string; to?: string } | null = null;
-    const isManualOverride = !!fromDate;
-    if (isManualOverride) {
-      // Manual date override from webhook
-      dateRangeFilter = { from: fromDate, to: toDate };
-      log.info('Using manual date override', { fromDate, toDate });
-    } else {
-      // Incremental sync - get last sync timestamp
-      const rawLastRunTime = (await kv.get<string>(STATE_KEY)) || DEFAULT_FALLBACK;
-      // Apply overlap buffer (prevents missed records)
-      const bufferedLastRunTime = new Date(
-        new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_SECONDS * 1000
-      ).toISOString();
-      const effectiveEndTime = toDate || new Date().toISOString();
-      dateRangeFilter = {
-        from: bufferedLastRunTime,
-        to: effectiveEndTime, // End of extraction window
-      };
-      log.info('Using incremental sync with overlap buffer', {
-        rawLastRunTime,
-        bufferedLastRunTime,
-        effectiveEndTime,
-        overlapBufferSeconds: OVERLAP_BUFFER_SECONDS,
-      });
-    }
-    // �����������������������������������������������������������
-    // STEP 4/8: Extract Data (ExtractionOrchestrator)
-    // �����������������������������������������������������������
-    log.info('📥 [STEP 4/8] Extracting data from Fluent Commerce', { jobId });
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 'extraction',
-      message: 'Extracting data with auto-pagination',
-    });
-    // Configure extraction
-    const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
-    const maxRecords = parseInt(activation.getVariable('maxRecords') || '100000', 10);
-    // Memory management: maxRecords prevents unbounded memory; for very large outputs, split files or use S3.
-    // Performance note: pageSize=200 is a balanced default; split output if records > ~50k for optimal performance.
-    // ? Enhanced: Extract context for progress logging
-    const dateRangeInfo = {
-      start: dateRangeFilter?.from || 'N/A',
-      end: dateRangeFilter?.to || 'N/A',
-      types: params.positionTypes?.join(', ') || 'all',
-      locationRefs: params.locationRefs?.slice(0, 3).join(', ') || 'all'
-    };
-    // ? Enhanced: Start logging with context
-    log.info(`🔍 [ExtractionOrchestrator] Starting extraction`, {
-     query: 'inventoryPositions',
-     pageSize,
-     maxRecords,
-     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
-     positionTypes: dateRangeInfo.types,
-     sampleLocations: dateRangeInfo.locationRefs,
-     jobId
-    });
-    // Initialize ExtractionOrchestrator
-    const orchestrator = new ExtractionOrchestrator(client, log);
-    // Execute extraction with auto-pagination
-    const extractionResult: ExtractionResult<any> = await orchestrator.extract({
-      query: INVENTORY_POSITIONS_EXTRACTION_QUERY,
-      resultPath: 'inventoryPositions.edges.node',
-      variables: {
-        dateRangeFilter,
-        types: params.positionTypes,
-        locationRefs: params.locationRefs,
-        // Note: Don't include 'first' or 'after' here; orchestrator injects them based on pageSize below
-      },
-      pageSize,
-      maxRecords,
-      // Optional: validate each record
-      validateItem: (item: any) => {
-        return !!(item.ref && item.productRef && typeof item.onHand === 'number');
-      },
-    });
-    const records = extractionResult.data || [];
-    log.info('Extraction complete', {
-      totalRecords: extractionResult.stats.totalRecords,
-      totalPages: extractionResult.stats.totalPages,
-      validRecords: extractionResult.stats.validRecords ?? records.length,
-      failedValidations: extractionResult.stats.failedValidations,
-      truncated: extractionResult.stats.truncated,
-      truncationReason: extractionResult.stats.truncationReason,
-      errors: extractionResult.errors ? extractionResult.errors.length : 0,
-    });
-    // ? Enhanced: Completion logging with summary
-    log.info(`✅ [ExtractionOrchestrator] Extraction completed`, {
-     totalRecords: extractionResult.stats.totalRecords,
-     totalPages: extractionResult.stats.totalPages,
-     validRecords: extractionResult.stats.validRecords ?? records.length,
-     failedValidations: extractionResult.stats.failedValidations,
-     truncated: extractionResult.stats.truncated,
-     truncationReason: extractionResult.stats.truncationReason,
-     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
-     jobId
-    });
-    if (extractionResult.errors && extractionResult.errors.length > 0) {
-      log.warn('Non-fatal extraction errors encountered', {
-        errorCount: extractionResult.errors.length,
-        sampleErrors: extractionResult.errors.slice(0, 3),
-      });
-    }
-    // Handle empty result
-    if (records.length === 0) {
-      log.info('No records to process');
-      // Update state even with no records (prevents re-querying empty window)
-      if (updateState && !isManualOverride) {
-        await kv.set(STATE_KEY, new Date().toISOString());
-      }
-      await tracker.markCompleted(jobId, {
-        recordCount: 0,
-        message: 'No records to extract',
-      });
-      return {
-        success: true,
-        jobId,
-        recordsExtracted: 0,
-      };
-    }
-    // �����������������������������������������������������������
-    // STEP 5/8: Transform Data (UniversalMapper)
-    // �����������������������������������������������������������
-    log.info('🔄 [STEP 5/8] Transforming data with UniversalMapper', {
-      jobId,
-      recordCount: records.length,
-    });
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 'transformation',
-      message: `Transforming ${records.length} records`,
-    });
-    const mapper = new UniversalMapper(mappingConfig);
-    const mappingResult = await mapper.map(records);
-    const mappingErrors = mappingResult.errors || [];
-    if (!mappingResult.success) {
-      log.error('Mapping failed - terminating job', {
-        jobId,
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-      await tracker.markFailed(
-        jobId,
-        mappingErrors[0] || 'UniversalMapper returned unsuccessful result',
-        {
-          failedCount: mappingErrors.length,
-          errors: mappingErrors,
-        }
-      );
-      return {
-        success: false,
-        jobId,
-        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
-        errors: mappingErrors,
-      };
-    }
-    const transformedRecords = Array.isArray(mappingResult.data)
-      ? (mappingResult.data as any[])
-      : [];
-    if (mappingErrors.length > 0) {
-      log.warn('Some records failed transformation', {
-        jobId,
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-    }
-    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
-      log.info('ℹ️ [MAPPING] Optional fields skipped (undefined values)', {
-        jobId,
-        skippedFields: mappingResult.skippedFields,
-        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
-      });
-    }
-    if (transformedRecords.length === 0) {
-      await tracker.markFailed(jobId, 'All records failed mapping', {
-        failedCount: mappingErrors.length,
-        errors: mappingErrors,
-      });
-      return {
-        success: false,
-        jobId,
-        error: 'All records failed mapping',
-        errors: mappingErrors,
-      };
-    }
-    log.info('Transformation complete', {
-      successful: transformedRecords.length,
-      failed: mappingErrors.length,
-      skippedRecords: records.length - transformedRecords.length,
-    });
-    // �����������������������������������������������������������
-    // STEP 6/8: Generate XML (XMLBuilder)
-    // �����������������������������������������������������������
-    log.info('📝 [STEP 6/8] Generating XML file', { jobId });
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 'xml_generation',
-      message: `Generating XML for ${transformedRecords.length} records`,
-    });
-    // Initialize XMLBuilder
-    const xmlBuilder = new XMLBuilder({
-      ignoreAttributes: false,
-      format: true,
-      indentBy: '  ',
-      suppressEmptyNode: true,
-    });
-    // Build XML structure
-    const xmlData = {
-      '?xml': {
-        '@_version': '1.0',
-        '@_encoding': 'UTF-8',
-      },
-      InventoryPositions: {
-        Position: transformedRecords,
-      },
-    };
-    const xmlContent = xmlBuilder.build(xmlData);
-    // Generate timestamped filename using extractFileName helper
-    const fileNamePrefix = activation.getVariable('fileNamePrefix') || 'inventorypositions';
-    const fileName = extractFileName(fileNamePrefix, 'xml');
-    log.info('XML file generated', {
-      fileName,
-      sizeBytes: xmlContent.length,
-      recordCount: transformedRecords.length,
-    });
-    // �����������������������������������������������������������
-    // STEP 7/8: Upload to SFTP (SftpDataSource)
-    // �����������������������������������������������������������
-    log.info('📤 [STEP 7/8] Uploading to SFTP', { jobId, fileName });
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 'sftp_upload',
-      message: `Uploading ${fileName} to SFTP`,
-    });
-    // Get SFTP credentials from Versori connection (Basic Auth)
-    // RECOMMENDED: Use activation.connections (already decoded)
-    const allConnections = ctx.activation.connections || [];
-    const sftpConn = allConnections.find(c => c.name === 'versori_ftp_server');
-    if (!sftpConn) {
-      throw new Error('SFTP connection "versori_ftp_server" not found');
-    }
-    const credential = sftpConn.credentials[0]?.credential;
-    if (!credential?.data?.basicAuth) {
-      throw new Error('SFTP connection not configured with Basic Authentication');
-    }
-    const { username, password } = credential.data.basicAuth;
-    // ? Already decoded - no Buffer.from() needed!
-    // Get SFTP configuration from activation variables
-    const sftpConfig = {
-      host: activation.getVariable('sftpHost'),
-      port: parseInt(activation.getVariable('sftpPort') || '22', 10),
-      username: sftpUsername, // From connection
-      password: sftpPassword, // From connection
-      privateKey: activation.getVariable('sftpPrivateKey'),
-      remotePath: activation.getVariable('sftpPath') || '/incoming/',
-    };
-    // Validate SFTP config
-    if (!sftpConfig.host || !username) {
-      throw new Error('SFTP configuration incomplete: missing host or username (from connection)');
-    }
-    if (!sftpConfig.password && !sftpConfig.privateKey) {
-      throw new Error('SFTP configuration incomplete: missing password or privateKey');
-    }
-    // Initialize SFTP data source
-    // ? VERSORI PLATFORM: Pass native log from context
-    const sftp = new SftpDataSource(
-      {
-        type: 'SFTP_XML',
-        connectionId: 'inventory-positions-sftp',
-        name: 'Inventory Positions SFTP Upload',
-        settings: {
-          host: sftpConfig.host,
-          port: sftpConfig.port,
-          username: sftpConfig.username,
-          password: sftpConfig.password,
-          privateKey: sftpConfig.privateKey,
-          remotePath: sftpConfig.remotePath,
-          filePattern: '*.xml',
-        },
-      },
-      log
-    );
-    try {
-      // Ensure remote directory exists (recursive)
-      await sftp.createDirectory(sftpConfig.remotePath || '/incoming/', true);
-      // Get SFTP path configuration
-      const requireAbsolutePaths = activation.getVariable('requireAbsolutePaths') === 'true';
-      /**
-       * Helper: Join SFTP path segments safely
-       *
-       * Handles different SFTP server path requirements:
-       * - AWS Transfer Family: Requires absolute paths (leading /)
-       * - Standard OpenSSH: Supports relative paths
-       *
-       * @param requireAbsolutePath - true for AWS Transfer Family, false for standard OpenSSH
-       * @param parts - Path segments to join
-       * @returns Properly formatted SFTP path
-       */
-      function joinSftpPath(requireAbsolutePath: boolean, ...parts: string[]): string {
-        // Clean each segment (remove leading/trailing slashes)
-        const cleaned = parts
-          .filter(Boolean)
-          .map(p => String(p).replace(/^\/+|\/+$/g, ''))
-          .join('/');
-        // Add leading slash if required (AWS Transfer Family)
-        return requireAbsolutePath && !cleaned.startsWith('/') ? `/${cleaned}` : cleaned;
-      }
-      // Construct full SFTP path safely
-      const fullPath = joinSftpPath(
-        requireAbsolutePaths,
-        sftpConfig.remotePath || '/incoming/',
-        fileName
-      );
-      // Upload with retry logic (built into SftpDataSource)
-      // ? Use createDirectories: true to ensure remote path exists
-      // Upload failure: If SFTP upload fails, job is failed and state is not advanced; next run retries same window.
-      await sftp.uploadFile(fullPath, Buffer.from(xmlContent, 'utf8'), {
-        createDirectories: true,
-      });
-      log.info('SFTP upload successful', { fileName, remotePath: fullPath });
-      // Optional: upload a transformation error report alongside the XML
-      if (mappingErrors.length > 0) {
-        const baseName = fileName.replace(/\.xml$/i, '');
-        const report = {
-          fileName,
-          processedAt: new Date().toISOString(),
-          errorStage: 'transform',
-          totals: {
-            totalRecords: records.length,
-            recordsTransformed: transformedRecords.length,
-            transformErrors: mappingErrors.length,
-          },
-          errors: mappingErrors,
-        };
-        const reportPath = joinSftpPath(
-          requireAbsolutePaths,
-          sftpConfig.remotePath || '/incoming/',
-          `${baseName}-errors.json`
-        );
-        await sftp.uploadFile(reportPath, Buffer.from(JSON.stringify(report, null, 2), 'utf8'), {
-          createDirectories: true,
-          overwrite: true,
-        });
-        log.info('Uploaded transformation error report', { reportPath });
-      }
-      // Optional: For very large outputs, split transformedRecords into chunks
-      // and upload multiple XML files to avoid size/memory limits.
-      // �����������������������������������������������������������
-      // STEP 8/8: Update State & Complete Job (WITHOUT buffer)
-      // �����������������������������������������������������������
-      log.info('✅ [STEP 8/8] Updating state and completing job', { jobId });
-      // Calculate new timestamp for next incremental run
-      let newTimestamp: string | undefined;
-      if (updateState && !isManualOverride) {
-        // Find max updatedOn from extracted records
-        const maxUpdatedOn = records.reduce(
-          (max, record) => {
-            const recordTime = new Date(record.updatedOn).getTime();
-            return recordTime > max ? recordTime : max;
-          },
-          new Date(dateRangeFilter?.from || DEFAULT_FALLBACK).getTime()
-        );
-        newTimestamp = new Date(maxUpdatedOn).toISOString();
-        // Store new timestamp (WITHOUT buffer - buffer only applied on read)
-        await kv.set(STATE_KEY, newTimestamp);
-        log.info('State updated', {
-          oldTimestamp: dateRangeFilter?.from,
-          newTimestamp,
-        });
-      }
-      // Mark job as completed
-      await tracker.markCompleted(jobId, {
-        recordCount: transformedRecords.length,
-        fileName,
-        remotePath: fullPath,
-        errorCount: mappingErrors.length,
-        errors: mappingErrors,
-        isManualOverride,
-        stateUpdated: updateState && !isManualOverride,
-        newTimestamp,
-      });
-      // ⏱️ Calculate total execution time
-      const totalDuration = Date.now() - startTime;
-      log.info('✅ Extraction workflow completed successfully', {
-        jobId,
-        totalDuration,
-        recordsExtracted: transformedRecords.length,
-        fileName,
-      });
-      return {
-        success: true,
-        jobId,
-        recordsExtracted: transformedRecords.length,
-        fileName,
-        sftpPath: fullPath,
-        isManualOverride,
-        stateUpdated: updateState && !isManualOverride,
-        newTimestamp,
-        duration: totalDuration,
-        errors: mappingErrors.length > 0 ? mappingErrors : undefined,
-      };
-    } finally {
-      // ?��️ CRITICAL: Always dispose SFTP connection
-      await sftp.dispose();
-      log.info('SFTP connection disposed');
-    }
-  } catch (error: any) {
-    const totalDuration = Date.now() - startTime;
-    log.error('❌ Extraction workflow failed', {
-      jobId,
-      duration: totalDuration,
-      message: error instanceof Error ? error.message : String(error),
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'Error',
-      recommendation: getErrorRecommendation(error),
-    });
-    // Mark job as failed
-    await tracker.markFailed(jobId, error);
-    return {
-      success: false,
-      jobId,
-      recordsExtracted: 0,
-      duration: totalDuration,
-      message: error instanceof Error ? error.message : String(error),
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'Error',
-      recommendation: getErrorRecommendation(error),
-    };
-  }
-}
-/**
- * Get actionable recommendation based on error type
- */
-function getErrorRecommendation(error: any): string {
-  const message = error instanceof Error ? error.message : String(error);
-  if (message.includes('SFTP') || message.includes('connection')) {
-    return 'Check SFTP credentials and network connectivity. Verify the SFTP connection is configured in Versori Connections with Basic Authentication.';
-  }
-  if (message.includes('GraphQL') || message.includes('query')) {
-    return 'Verify GraphQL query syntax and schema compatibility. Check Fluent Commerce connection credentials.';
-  }
-  if (message.includes('mapping') || message.includes('transform')) {
-    return 'Review mapping configuration. Run schema validation: npx fc-connect validate-schema --mapping config.json --schema schema.json';
-  }
-  if (message.includes('authentication') || message.includes('401')) {
-    return 'Check Fluent Commerce OAuth2 credentials. Verify clientId and clientSecret in connection configuration.';
-  }
-  if (message.includes('timeout')) {
-    return 'Reduce maxRecords or pageSize to process smaller batches. Check network latency and API response times.';
-  }
-  return 'Review error details above. Check activation variables and connection configuration. Consult documentation for troubleshooting steps.';
-}
-```
-Note: Customize mapping by editing the JSON above; prefer built-in resolvers. See SDK Universal Mapping guide for advanced usage.
----
-## 4. Utility Functions (src/utils/)
-### Job ID Generator (src/utils/job-id-generator.ts)
-```typescript
-/**
- * Job ID Generator
- *
- * Generates unique job IDs for tracking extraction workflows
- *
- * FORMAT: {TYPE}_{ENTITY}_{YYYYMMDD}_{HHMMSS}_{RANDOM}
- * Example: SCHEDULED_IP_20251027_183045_a1b2c3
- *
- * NAMING: generate{Entity}JobId or generateJobId (generic)
- */
-/**
- * Generate unique job ID
- *
- * @param type - Job trigger type (SCHEDULED, ADHOC, MANUAL)
- * @param entity - Entity abbreviation (IP, VP, ORD, PRD)
- * @returns Unique job ID string
- */
-export function generateJobId(type: string, entity: string): string {
-  const now = new Date();
-  // Format: YYYYMMDD
-  const dateStr = now.toISOString().slice(0, 10).replace(/-/g, '');
-  // Format: HHMMSS
-  const timeStr = now.toISOString().slice(11, 19).replace(/:/g, '');
-  // Random suffix (6 chars)
-  const randomStr = Math.random().toString(36).substring(2, 8);
-  return `${type}_${entity}_${dateStr}_${timeStr}_${randomStr}`;
-}
-/**
- * Generate timestamped filename for extraction output
- *
- * @param prefix - Filename prefix (from activation variable)
- * @param extension - File extension (xml, csv, json)
- * @returns Timestamped filename
- *
- * Example: extractFileName('inventorypositions', 'xml')
- * Returns: 'inventorypositions-2025-11-04T14-30-45-123Z.xml'
- */
-export function extractFileName(prefix: string, extension: string = 'xml'): string {
-  const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
-  return `${prefix}-${timestamp}.${extension}`;
-}
-/**
- * Parse job ID components
- *
- * @param jobId - Job ID to parse
- * @returns Parsed components or null if invalid
- */
-export function parseJobId(jobId: string): {
-  type: string;
-  entity: string;
-  date: string;
-  time: string;
-  random: string;
-} | null {
-  const parts = jobId.split('_');
-  if (parts.length !== 5) {
-    return null;
-  }
-  return {
-    type: parts[0],
-    entity: parts[1],
-    date: parts[2],
-    time: parts[3],
-    random: parts[4],
-  };
-}
-```
----
-## 5. Package Configuration
-### package.json
-```json
-{
-  "name": "inventory-positions-to-sftp-xml",
-  "version": "1.0.0",
-  "description": "Extract inventory positions from Fluent Commerce and export to SFTP as XML",
-  "type": "module",
-  "main": "src/index.ts",
-  "scripts": {
-    "dev": "versori dev",
-    "build": "versori build",
-    "deploy": "versori deploy"
-  },
-  "dependencies": {
-    "@fluentcommerce/fc-connect-sdk": "^0.1.39",
-    "@versori/run": "latest",
-    "fast-xml-parser": "^5.2.5"
-  },
-  "devDependencies": {
-    "@types/node": "^20.0.0",
-    "typescript": "^5.0.0"
-  }
-}
-```
-### tsconfig.json
-```json
-{
-  "compilerOptions": {
-    "module": "ES2022",
-    "target": "ES2024",
-    "moduleResolution": "node"
-  }
-}
-```
----
-## 6. Deployment Instructions
-### Deploy to Versori
-```bash
-# 1. Install dependencies
-npm install
-# 2. Test locally (if using Versori CLI)
-npm run dev
-# 3. Deploy to Versori platform
-npm run deploy
-```
-### Configure Activation Variables
-In Versori platform settings, configure:
-```json
-{
-  "sftpHost": "sftp.partner.com",
-  "sftpPort": 22,
-  "sftpUsername": "export_user",
-  "sftpPassword": "********",
-  "sftpPath": "/incoming/inventory/",
-  "fileNamePrefix": "inventorypositions",
-  "pageSize": 200,
-  "maxRecords": 100000,
-  "overlapBufferSeconds": 60
-}
-```
----
-## 7. Performance Notes
-- **Pagination:** Default pageSize=200 is optimized for most use cases; increase to 500 if network latency is high
-- **Large outputs:** For extractions exceeding ~50k records, consider splitting into multiple files or streaming to S3
-- **Memory limits:** maxRecords parameter prevents unbounded memory growth; adjust based on available memory
----
-## 8. Testing
-### Test Scheduled Extraction
-The scheduled workflow runs automatically based on cron schedule.
-**Check logs:**
-```
-[STEP 1/8] Initializing job tracking
-[STEP 2/8] Initializing Fluent Commerce client
-[STEP 3/8] Determining date range for extraction
-[STEP 4/8] Extracting data from Fluent Commerce
-[STEP 5/8] Transforming data with UniversalMapper
-[STEP 6/8] Generating XML file
-[STEP 7/8] Uploading to SFTP
-[STEP 8/8] Updating state and completing job
-```
-### Test Ad hoc Extraction
-```bash
-# Incremental (uses last sync timestamp)
-curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
-  -H "Content-Type: application/json" \
-  -H "X-API-Key: your-api-key" \
-  -d '{}'
-# Date range override
-curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
-  -H "Content-Type: application/json" \
-  -H "X-API-Key: your-api-key" \
-  -d '{
-    "fromDate": "2025-01-01T00:00:00Z",
-    "toDate": "2025-01-31T23:59:59Z",
-    "updateState": false
-  }'
-```
-### Test Job Status Query
-```bash
-curl -X POST https://api.versori.com/webhooks/inventory-positions-job-status \
-  -H "Content-Type: application/json" \
-  -H "X-API-Key: your-api-key" \
-  -d '{
-    "jobId": "ADHOC_IP_20251027_183045_abc123"
-  }'
-```
-**Response:**
-```json
-{
-  "success": true,
-  "jobId": "ADHOC_IP_20251027_183045_abc123",
-  "status": "processing",
-  "stage": "transformation",
-  "message": "Transforming 15000 records",
-  "createdAt": "2025-10-27T18:30:45.000Z",
-  "startedAt": "2025-10-27T18:30:46.000Z"
-}
-```
-### Monitoring
-Use Versori logs filtered by `jobId` or `stage`; optional status webhook for dashboards.
----
-### Schema requirements
-Ensure your GraphQL query includes Relay pagination fields and variables ($first, $after), and that `resultPath` matches the edges.node path. The orchestrator injects pagination variables automatically (don't include them in your variables object).
----
-## ?? Troubleshooting (quick)
-- No records extracted: Check dateRange (manual override vs incremental).
-- SFTP upload failed: Job fails; state not advanced. Next run retries same window.
-- GraphQL pagination error: Ensure edges.cursor and pageInfo.hasNextPage are in the query.
-- Memory pressure: Lower pageSize or maxRecords; split output or stream to S3.
-## 🗺️ Mapping tips (compact)
-- Required fields: mark in mapping JSON; invalid records are skipped and reported.
-- Nested paths: use dot notation (e.g., catalogue.ref).
-- Arrays: map arrays directly; one record in, one mapped object out.
-- Resolvers: prefer built-ins (trim, uppercase, parseInt); keep output types aligned.
----
-## 9. Replication Checklist
-**To replicate this template for other entities/formats:**
-1. **File Naming:** Replace `inventory-positions`, `IP`, `InventoryPosition` with your entity name across all files
-2. **GraphQL Query:** Update query constant and field selection to match your entity schema
-3. **Mapping Config:** Create new mapping file in `config/` with correct field paths
-4. **Workflows:** Rename workflow exports to match entity (e.g., `scheduledVirtualPositionsExtraction`)
-5. **Service Function:** Rename main function (e.g., `executeVirtualPositionExtraction`)
-6. **State Key:** Update KV key (e.g., `lastVirtualPositionSync`)
-7. **Output Format:** For CSV use `CSVParserService`, for JSON use `JSON.stringify()`, for XML use `XMLBuilder`
-8. **Upload Destination:** For S3 replace `SftpDataSource` with `S3DataSource`
----
-### Pattern 8: Backward Pagination (Optional - Advanced)
-**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
-**When to Use**:
-- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
-- ✅ Time-bounded reverse traversal for auditing
-- ✅ Display newest-first in UI/reports
-- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
-**GraphQL Query Requirements**:
-Your query must support backward pagination by including `$last` and `$before`:
-```graphql
-query GetData(
-  $retailerId: ID!
-  $first: Int # For forward pagination
-  $after: String # For forward pagination
-  $last: Int # For backward pagination
-  $before: String # For backward pagination
-) {
-  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
-    edges {
-      cursor # ? REQUIRED
-      node {
-        id
-        createdAt
-        # ... other fields
-      }
-    }
-    pageInfo {
-      hasNextPage # For forward
-      hasPreviousPage # ? REQUIRED for backward
-    }
-  }
-}
-```
-**Implementation**:
-```typescript
-// Backward pagination - newest records first
-const result = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    retailerId,
-    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
-    //  Don't include last/before - orchestrator injects them
-  },
-  pageSize: 200,
-  direction: 'backward', // ? Enable reverse pagination
-  maxRecords: 10000,
-});
-// Records are returned in reverse chronological order
-console.log(result.data[0].createdAt); // Newest
-console.log(result.data[result.data.length - 1].createdAt); // Oldest (within range)
-```
-**Key Differences from Forward Pagination**:
-| Aspect                 | Forward (Default)                | Backward                |
-| ---------------------- | -------------------------------- | ----------------------- |
-| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
-| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
-| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
-| **Cursor Source**      | Last edge of page                | First edge of page      |
-| **Record Order**       | Oldest ? Newest               | Newest ? Oldest      |
-**Important Notes**:
-1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
-2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
-3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
-4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
-**Example: Extract Latest 1000 Orders**
-```typescript
-const latestOrders = await orchestrator.extract({
-  query: ORDERS_QUERY,
-  resultPath: 'orders.edges.node',
-  variables: {
-    retailerId,
-    statuses: ['BOOKED', 'ALLOCATED'],
-  },
-  direction: 'backward', // Start from newest
-  maxRecords: 1000, // Stop after 1000 records
-  pageSize: 100, // 100 per page = 10 pages
-});
-// latestOrders.data[0] is the newest order
-// latestOrders.data[999] is the 1000th newest order
-```
-**When to Use Forward vs Backward**:
-```typescript
-// ? Forward (default) - For incremental sync
-const incrementalData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    dateRangeFilter: { from: lastSyncTime, to: now },
-  },
-  // direction defaults to 'forward'
-  // Processes oldest ? newest for proper sequencing
-});
-// ? Backward - For "latest N records" use cases
-const latestData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  direction: 'backward',
-  maxRecords: 100, // Just get latest 100
-  // Gets newest ? oldest
-});
-```
-**Pagination Variables Reference**:
-| Variable | Forward      | Backward     | Injected By  | Notes                    |
-| -------- | ------------ | ------------ | ------------ | ------------------------ |
-| `first`  | ? Used      |  Not used | Orchestrator | From `pageSize`          |
-| `after`  | ? Used      |  Not used | Orchestrator | From cursor (last edge)  |
-| `last`   |  Not used | ? Used      | Orchestrator | From `pageSize`          |
-| `before` |  Not used | ? Used      | Orchestrator | From cursor (first edge) |
-**Common Mistakes to Avoid**:
-```typescript
-//  WRONG - Don't pass pagination variables
-const result = await orchestrator.extract({
-  variables: {
-    last: 200, //  Orchestrator will override this
-    before: cursor, //  Orchestrator manages cursor
-  },
-  direction: 'backward',
-});
-// ? CORRECT - Let orchestrator inject pagination
-const result = await orchestrator.extract({
-  variables: {
-    retailerId, // ? Your business variables only
-  },
-  pageSize: 200, // ? Orchestrator uses this for last/before
-  direction: 'backward',
-});
-```
-#### Optional: Reverse Pagination
-- Default remains forward pagination ($first/$after) using pageInfo.hasNextPage.
-- To paginate backward, define $last/$before and add pageInfo.hasPreviousPage; set direction='backward' in the orchestrator.
-Snippet:
-```graphql
-query GetInventoryPositionsBackward($last: Int!, $before: String) {
-  inventoryPositions(last: $last, before: $before) {
-    edges {
-      cursor
-      node {
-        id
-        ref
-        updatedOn
-      }
-    }
-    pageInfo {
-      hasPreviousPage
-    }
-  }
-}
-```
-SDK:
-```typescript
-await orchestrator.extract({
-  query: INVENTORY_POSITIONS_BACKWARD_QUERY,
-  resultPath: 'inventoryPositions.edges.node',
-  variables: { dateRangeFilter },
-  pageSize,
-  direction: 'backward',
-});
-```
----
-## Testing Checklist
-**Before production deployment:**
-### 1. Schema Validation
-- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
-- [ ] Run `npx fc-connect validate-schema --mapping ./config/inventory-positions.export.xml.json --schema ./fluent-schema.json`
-- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/inventory-positions.export.xml.json --schema ./fluent-schema.json`
-- [ ] Verify all `source` paths in mapping exist in GraphQL schema
-- [ ] Verify query structure matches schema (fields, types, filters)
-### 2. Extraction Testing
-- [ ] Test with small dataset first (maxRecords=10)
-- [ ] Verify ExtractionOrchestrator pagination works correctly
-- [ ] Test with multiple pages of data (verify cursor handling)
-- [ ] Verify date range filtering (updatedOn filter)
-- [ ] Test empty result handling (no records in date range)
-- [ ] Verify extraction stops at maxRecords limit
-### 3. Mapping Testing
-- [ ] Verify required fields are populated
-- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
-- [ ] Test custom resolvers with edge cases (if any)
-- [ ] Verify nested field extraction
-- [ ] Test with null/missing fields
-- [ ] Verify mapping error collection works
-### 4. XML Generation Testing
-- [ ] Verify XML structure matches expected format
-- [ ] Test XML validation against XSD schema (if applicable)
-- [ ] Verify special character escaping in XML
-- [ ] Test with large datasets (>1000 records)
-- [ ] Verify UTF-8 encoding
-- [ ] Test XML namespace handling (if applicable)
-### 5. SFTP Upload Testing
-- [ ] Test SFTP connection and authentication
-- [ ] Verify file upload to correct path
-- [ ] Test file naming convention (timestamp format)
-- [ ] Verify file permissions on SFTP server
-- [ ] Test upload retry logic (simulate network failure)
-- [ ] Verify SFTP connection disposal (no connection leaks)
-### 6. State Management Testing
-- [ ] Verify overlap buffer prevents missed records (60-second default)
-- [ ] Test state recovery after extraction failure
-- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
-- [ ] Test first run with no previous state (uses fallbackStartDate)
-- [ ] Verify state update only happens on successful upload
-- [ ] Test manual date override (doesn't update state)
-### 7. Job Tracking Testing
-- [ ] Test job creation with JobTracker
-- [ ] Verify job status updates at each stage
-- [ ] Test job completion with metadata
-- [ ] Test job failure handling
-- [ ] Query job status via webhook endpoint
-- [ ] Verify job status persists in KV store
-### 8. Error Handling Testing
-- [ ] Test with invalid GraphQL query
-- [ ] Test with mapping errors (invalid field paths)
-- [ ] Test with SFTP connection failures
-- [ ] Test with authentication failures
-- [ ] Test with network timeouts
-- [ ] Verify error logging includes context (jobId, stage, error details)
-- [ ] Test error threshold logic (if applicable)
-### 9. Staging Environment Testing
-- [ ] Run full extraction in staging environment
-- [ ] Verify XML file format with downstream system
-- [ ] Monitor extraction duration and resource usage
-- [ ] Test with production-like data volumes
-- [ ] Verify no performance degradation over time
-### 10. Integration Testing
-- [ ] Test scheduled workflow (cron trigger)
-- [ ] Test ad hoc webhook trigger
-- [ ] Test job status query webhook
-- [ ] Verify activation variables are read correctly
-- [ ] Test with different extraction modes (incremental, date range)
-- [ ] End-to-end test: trigger ? extract ? transform ? upload ? verify file
----
-## Monitoring & Alerting
-### Success Response Example
-```json
-{
-  "success": true,
-  "jobId": "SCHEDULED_IP_20251102_140000_abc123",
-  "recordsExtracted": 1523,
-  "fileName": "inventory-positions-2025-11-02T14-00-00-000Z.xml",
-  "sftpPath": "/outbound/inventory-positions/inventory-positions-2025-11-02T14-00-00-000Z.xml",
-  "metrics": {
-    "extractionDurationMs": 12543,
-    "totalPages": 8,
-    "pageSize": 200,
-    "mappingErrors": 0,
-    "fileSizeBytes": 524288,
-    "uploadDurationMs": 1234
-  },
-  "timestamps": {
-    "extractionStart": "2025-11-02T14:00:00.000Z",
-    "extractionEnd": "2025-11-02T14:00:12.543Z",
-    "uploadComplete": "2025-11-02T14:00:13.777Z"
-  },
-  "state": {
-    "previousTimestamp": "2025-11-02T13:00:00.000Z",
-    "newTimestamp": "2025-11-02T13:59:58.123Z",
-    "stateUpdated": true,
-    "overlapBufferSeconds": 60
-  }
-}
-```
-### Error Response Example
-```json
-{
-  "success": false,
-  "jobId": "ADHOC_IP_20251102_140500_xyz789",
-  "error": "SFTP upload failed: Connection timeout",
-  "errorCategory": "NETWORK",
-  "recordsExtracted": 0,
-  "stage": "sftp_upload",
-  "details": {
-    "message": "Failed to upload file after 3 retry attempts",
-    "retryAttempts": 3,
-    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
-  },
-  "state": {
-    "stateUpdated": false,
-    "willRetryNextRun": true,
-    "note": "State not advanced - next extraction will retry same time window"
-  }
-}
-```
-### Key Metrics to Track
-```typescript
-const METRICS = {
-  // Extraction Performance
-  extractionDurationMs: Date.now() - extractionStart,
-  recordCount: records.length,
-  pageCount: extractionResult.stats.totalPages,
-  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
-  // Transformation Performance
-  transformedCount: transformedRecords.length,
-  failedCount: mappingErrors.length,
-  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
-  // File Generation
-  fileSizeMB: (xmlContent.length / (1024 * 1024)).toFixed(2),
-  // Upload Performance
-  uploadDurationMs: uploadEnd - uploadStart,
-  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
-  // State Management
-  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
-  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
-};
-log.info('Extraction metrics', metrics);
-```
-### Alert Thresholds
-```typescript
-const ALERT_THRESHOLDS = {
-  // Duration Alerts
-  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
-  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
-  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
-  // Error Rate Alerts
-  MAX_ERROR_RATE: 0.05, // 5% mapping errors
-  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
-  // Volume Alerts
-  MAX_RECORDS_PER_RUN: 100000,
-  MIN_RECORDS_WARNING: 0, // Alert if no records found
-  MAX_FILE_SIZE_MB: 150, // 150MB
-  // State Alerts
-  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
-  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
-};
-// Check thresholds
-if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
-  log.warn('Extraction duration exceeded threshold', {
-    duration: metrics.extractionDurationMs,
-    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
-    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
-  });
-}
-```
-### Monitoring Dashboard Queries
-**Versori Platform Logs Query:**
-```
-# Successful extractions
-log_level:info AND message:"Extraction complete" AND jobId:*
-# Failed extractions
-log_level:error AND message:"Extraction workflow failed" AND jobId:*
-# Performance issues
-extractionDurationMs:>300000 OR uploadDurationMs:>120000
-# High error rates
-errorRate:>5
-# State management issues
-stateUpdated:false AND success:true
-```
-### Common Issues and Solutions
-**Issue**: "Extraction timeout after 10 minutes"
-- **Cause**: Too many records in single extraction
-- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
-- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
-**Issue**: "Mapping errors for 50% of records"
-- **Cause**: Schema mismatch between GraphQL response and mapping config
-- **Fix**: Run schema validation, update mapping config paths
-- **Prevention**: Use `npx fc-connect validate-schema` before deployment
-**Issue**: "SFTP connection timeout"
-- **Cause**: Network issues, firewall, or connection pool exhaustion
-- **Fix**: Check SFTP credentials, verify network connectivity
-- **Prevention**: Implement connection health checks, monitor connection status
-**Issue**: "State not updating after successful extraction"
-- **Cause**: KV write failure or intentional retry logic
-- **Fix**: Check KV logs, verify state update code executed
-- **Prevention**: Add KV write verification, log state updates explicitly
-**Issue**: "First run exceeds record limits"
-- **Cause**: No previous timestamp, fetches all historical records
-- **Fix**: Set fallbackStartDate close to current date, apply additional filters
-- **Prevention**: Use appropriate fallbackStartDate for initial runs
-**Issue**: "Excessive duplicate records in output"
-- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
-- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
-- **Prevention**: Monitor duplicate rates, verify state update logic
----
-## Troubleshooting Quick Reference
-| Error Message | Likely Cause | Solution |
-|--------------|--------------|----------|
-| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
-| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
-| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
-| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
-| "SFTP authentication failed" | Invalid credentials | Verify SFTP credentials in activation variables |
-| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
-| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
-| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
-| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
-| "XML generation failed" | Format-specific error | Check XML generation logic, validate output |
----
+---
+template_id: tpl-extract-inventory-positions-to-sftp-xml
+canonical_filename: template-extraction-inventory-positions-to-sftp-xml.md
+version: 2.0.0
+sdk_version: ^0.1.39
+runtime: versori
+direction: extraction
+source: fluent-graphql
+destination: sftp-xml
+entity: inventoryPositions
+format: xml
+logging: versori
+status: stable
+features:
+  - memory-management
+  - enhanced-logging
+  - pagination-progress
+  - dispose-finally
+---
+# Template: Extraction - Inventory Positions to SFTP XML
+**Template Version:** 2.0.0
+**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
+**Last Updated:** 2025-01-24
+**Deployment Target:** Versori Platform
+**🆕 Version 2.0.0 Enhancements:**
+- ✅ **Memory Management** - Clear large result sets after processing batches
+- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
+- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
+- ✅ **Resource Cleanup** - SFTP dispose in finally blocks prevents connection leaks
+---
+## ?? STEP 1: Load These Docs Into Your AI (Human Checklist)
+1. REQUIRED (load all)
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+Copy-paste list (open these):
+fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+---
+## 📋 STEP 2: Tell Your AI (Prompt)
+Copy/paste this prompt into your AI tool after loading the documentation above:
+```
+I need a Versori scheduled extractor that:
+1) Queries Fluent Commerce GraphQL for Inventory Positions with auto-pagination
+2) Supports incremental runs via KV state (with an overlap buffer)
+3) Transforms results using UniversalMapper per mapping JSON
+4) Generates pretty-printed XML and uploads to SFTP
+5) Tracks progress with JobTracker and exposes a job-status webhook
+6) Uses native Versori log (LoggingService removed - use native log) and disposes SFTP
+Use the loaded docs to fill in SDK specifics and best practices.
+Keep the structure identical to the template; only adapt where needed.
+```
+---
+## 📋 Template Overview
+This connector runs on the Versori platform. Most operational settings (Fluent account/connection, SFTP connection, schedule, page size/limits) are configured via activation variables. Data shape and logic (mapping JSON, XML structure, GraphQL selection set/filters, validators/resolvers) are adjusted in code as needed. It extracts inventory positions from Fluent Commerce via GraphQL, transforms the data into XML, and uploads the result to SFTP.
+### What This Template Does
+```
+�������������������������������������������������������������������
+�                    EXTRACTION WORKFLOW                           �
+�������������������������������������������������������������������
+1. TRIGGER
+   �� Scheduled (Cron): Runs automatically every hour
+   �� Ad hoc (Webhook): Manual trigger with optional date override
+   �� Status Query (Webhook): Check job progress
+2. EXTRACT (ExtractionOrchestrator)
+   �� Query Fluent GraphQL API for inventory positions
+   �� Auto-pagination (handles large datasets)
+   �� Apply date filters (incremental or manual range)
+   �� Validate each record (optional)
+3. TRANSFORM (UniversalMapper)
+   �� Map GraphQL fields to XML schema
+   �� Apply SDK resolvers (trim, uppercase, parseInt, etc.)
+   �� Extract nested data (catalogue)
+   �� Handle transformation errors
+4. GENERATE XML (XMLBuilder)
+   �� Convert transformed records to XML
+   �� Auto-escape special characters
+   �� Apply pretty-print formatting
+   �� Generate timestamped filename
+5. UPLOAD (SftpDataSource)
+   �� Connect to SFTP server
+   �� Upload XML file with retry logic
+   �� Verify upload success
+   �� Always dispose connection (finally block)
+6. TRACK JOB (JobTracker)
+   �� Create job with unique ID
+   �� Update status at each step
+   �� Store job result in KV
+   �� Enable status queries via webhook
+7. UPDATE STATE (VersoriKVAdapter)
+   �� Calculate max updatedOn from records
+   �� Store timestamp for next incremental run
+   �� Apply overlap buffer (prevent missed records)
+   �� Skip update if manual date override
+```
+### Key Features
+- Job tracking with status queries
+- Execution modes: scheduled, ad hoc, status query
+- Uses ExtractionOrchestrator, UniversalMapper, JobTracker
+- Error handling, retry logic, and SFTP cleanup
+- Reusable services suitable for similar use cases
+Note: JobTracker persists stage/status to Versori KV for visibility, job-status webhooks, and auditing. Recommended for production multi-step flows; can be skipped for trivial single-step utilities.
+### 📦 Package Information
+**SDK:** [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
+**Version:** Check [npm](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk) for latest version
+```bash
+npm install @fluentcommerce/fc-connect-sdk
+```
+---
+**Templates are designed for direct deployment; customize via activation variables.**
+---
+## 📦 SDK Imports (Verified - Versori Optimized)
+```typescript
+// ? VERIFIED IMPORTS - These match actual SDK exports
+import { Buffer } from 'node:buffer';
+import {
+  createClient, // Universal client factory (auto-detects Versori/Node/Deno)
+  ExtractionOrchestrator, // High-level extraction with auto-pagination
+  JobTracker, // Job status tracking in KV store
+  UniversalMapper, // Field mapping with SDK resolvers
+  XMLBuilder, // XML generation with auto-escaping (from SDK)
+  SftpDataSource, // SFTP operations with retry logic
+} from '@fluentcommerce/fc-connect-sdk';
+// Versori platform imports
+import { schedule, webhook, http, fn } from '@versori/run';
+```
+**CRITICAL:** `openKv` is NOT imported directly - it's accessed from the context parameter:
+```typescript
+// ✅ CORRECT - From context parameter
+export const myWorkflow = schedule('task', '0 * * * *', async (ctx) => {
+  const { openKv, log } = ctx;  // Access from context
+  const kv = openKv(':project:');
+});
+```
+**Note:** `openKv` is NOT exported from `@versori/run` - attempting to import it directly will cause a runtime error.
+**Note:** All imports are from actual SDK exports - this code compiles and runs as-is.
+**? VERSORI PLATFORM - Use Native Logs:**
+- Use `log` from context: `const { log } = ctx;`
+- Don't import or use LoggingService for Versori connectors
+- Native Versori logs are simpler and automatically integrated with platform monitoring
+---
+## ??� Configuration
+### SFTP Connection Setup
+**? RECOMMENDED APPROACH:** Store SFTP credentials in a Versori connection for security and reusability.
+Versori provides **three methods** to access connection credentials. Choose based on your use case:
+#### Method 1: `activation.connections` (Recommended - Simplest)
+**When to use:** Most workflows (scheduled, webhooks without connection param)
+**Setup:**
+1. Create connection named `versori_ftp_server` with Basic Auth
+2. Access via `activation.connections`:
+```typescript
+// ========================================
+    // SFTP CREDENTIAL RETRIEVAL
+    // ========================================
+    /**
+     * Retrieve SFTP credentials from connection configuration
+     *
+     * This approach retrieves credentials stored in the Versori connection settings.
+     * The connection must be configured in the UI with Basic Authentication.
+     *
+     * Steps:
+     * 1. Call ctx.credentials().getAccessToken('SFTP') to get base64-encoded credentials
+     * 2. Decode the accessToken from base64 to get "username:password" string
+     * 3. Split on ':' to extract username and password
+     *
+     * Connection Name: 'SFTP' (must match the connection name in Versori UI)
+     * Auth Type: Basic Authentication (username + password)
+     *
+     * This method provides:
+     * - Centralized credential management through Versori UI
+     * - Better security (credentials not stored in integration variables)
+     * - Easier credential rotation and updates
+     */
+    log.info('Retrieving SFTP credentials from connection configuration');
+    let sftpUsername: string;
+    let sftpPassword: string;
+    try {
+      // Retrieve credentials from the 'SFTP' connection
+      const sftpCred = await ctx.credentials().getAccessToken('SFTP');
+      if (!sftpCred?.accessToken) {
+        throw new Error('No SFTP credentials found in connection configuration');
+      }
+      // Decode base64 accessToken to get "username:password"
+      const rawBasicAuth = Buffer.from(sftpCred.accessToken, 'base64').toString('utf-8');
+      // Split on ':' to extract username and password
+      const parts = rawBasicAuth.split(':');
+      if (parts.length !== 2) {
+        throw new Error('Invalid SFTP credential format - expected username:password');
+      }
+      sftpUsername = parts[0];
+      sftpPassword = parts[1];
+      log.info('SFTP credentials retrieved successfully', {
+        hasUsername: !!sftpUsername,
+        hasPassword: !!sftpPassword,
+        usernameLength: sftpUsername.length,
+        passwordLength: sftpPassword.length,
+      });
+    } catch (error: any) {
+      log.error('Failed to retrieve SFTP credentials', {
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',,
+      });
+      return {
+        success: false,
+        error: 'Failed to retrieve SFTP credentials from connection configuration',
+        details: error?.message,
+        recommendation: 'Please ensure the SFTP connection is configured in the Connections section with Basic Authentication (username and password)',
+      };
+    }
+```
+**Pros:**
+- ✅ Dynamic connection name from config/environment
+- ✅ Explicit error handling for missing connections
+- ✅ Works in all workflow types
+**Cons:**
+- ⚠️ Async (requires `await`)
+- ⚠️ More verbose than `activation.connections`
+#### Method 3: `connections` from Context (Workflow-Scoped)
+**When to use:** Webhook with `connection` parameter (connection passed to specific step)
+**Note:** This method is different from `activation.connections` - it only works when the workflow step explicitly receives a `connection` parameter.
+**Setup:**
+1. Create connection named `versori_ftp_server`
+2. Pass to workflow step:
+```typescript
+export const myWebhook = webhook('my-webhook', { connection: 'versori_ftp_server' }).then(
+  http('process', { connection: 'versori_ftp_server' }, async ctx => {
+    // Get SFTP credentials from Versori connection (Basic Auth)
+    // RECOMMENDED: Use activation.connections (already decoded)
+    const allConnections = ctx.activation.connections || [];
+    const sftpConn = allConnections.find(c => c.name === 'versori_ftp_server');
+    if (!sftpConn) {
+      throw new Error('SFTP connection "versori_ftp_server" not found');
+    }
+    const credential = sftpConn.credentials[0]?.credential;
+    if (!credential?.data?.basicAuth) {
+      throw new Error('SFTP connection not configured with Basic Authentication');
+    }
+    const { username, password } = credential.data.basicAuth;
+    // ? Already decoded - no Buffer.from() needed!
+  })
+);
+```
+**Pros:**
+- ✅ Explicit connection passing to specific steps
+- ✅ Connection validation at workflow definition
+**Cons:**
+- ⚠️ Only works with `connection` parameter in workflow definition
+- ⚠️ Not available in scheduled workflows or webhooks without connection param
+#### Quick Decision Guide
+| Scenario                            | Method                     | Example                                                             |
+| ----------------------------------- | -------------------------- | ------------------------------------------------------------------- |
+| **Scheduled workflow**              | `activation.connections`   | `schedule().then(http(...))`                                        |
+| **Webhook (no connection param)**   | `activation.connections`   | `webhook().then(fn(...))`                                           |
+| **Webhook (with connection param)** | `connections` from context | `webhook({ connection: 'x' }).then(http({ connection: 'x' }, ...))` |
+| **Dynamic connection name**         | `credentials().get()`      | `await credentials().get(connName)`                                 |
+| **Need error handling**             | `credentials().get()`      | Explicit null check                                                 |
+**Connection Configuration:**
+1. In Versori platform, create connection named `versori_ftp_server`
+2. Set **Authentication Type**: `Basic Auth`
+3. Enter **Username**: Your SFTP username
+4. Enter **Password**: Your SFTP password
+5. Connection auto-encodes credentials as base64 Basic Auth
+**Why use connections instead of activation variables?**
+- ✅ Credentials stored securely in Versori vault
+- ✅ Connection can be reused across workflows
+- ✅ No need to manage sensitive data in activation variables
+- ✅ Easier credential rotation
+- ✅ Better separation of concerns (config vs secrets)
+- ✅ Follows industry security best practices
+**?? Complete Guide:** See `docs/02-CORE-GUIDES/data-sources/sftp-credential-access-security.md` for:
+- Detailed comparison of all three methods
+- Security best practices
+- Common pitfalls and solutions
+- Migration examples
+- Troubleshooting connection issues
+### Activation Variables
+**Configuration is driven by activation variables - modify these instead of code:**
+```json
+{
+  "sftpHost": "sftp.partner.com",
+  "sftpPort": 22,
+  "sftpPrivateKey": "",
+  "sftpPath": "/incoming/inventory/",
+  "fileNamePrefix": "inventorypositions",
+  "pageSize": 200,
+  "maxRecords": 100000,
+  "overlapBufferSeconds": 60,
+  "requireAbsolutePaths": "true"
+}
+```
+> **Note:** `sftpUsername` and `sftpPassword` are fetched from the `versori_ftp_server` connection (see above).
+Note: For webhook security, set `webhookApiKey` (activation variable) and provide it via `X-API-Key` header on webhook calls.
+### Variable Explanations
+| Variable                    | Purpose                      | Default                | Customization Hints                                              |
+| --------------------------- | ---------------------------- | ---------------------- | ---------------------------------------------------------------- |
+| **SFTP Credentials**        | _From Connection_            |                        | _See connection setup above_                                     |
+| `sftpHost`                  | SFTP server hostname         | -                      | Required - your SFTP server                                      |
+| `sftpPort`                  | SFTP server port             | `22`                   | Usually 22, sometimes 2222                                       |
+| `sftpPrivateKey`            | SSH private key (optional)   | -                      | Alternative to password auth                                     |
+| `sftpPath`                  | Remote directory path        | `/incoming/inventory/` | Where to upload XML files                                        |
+| `fileNamePrefix`            | XML filename prefix          | `inventorypositions`   | Customize naming convention                                      |
+| `pageSize`                  | Records per GraphQL page     | `200`                  | Increase for fewer API calls                                     |
+| `maxRecords`                | Total extraction limit       | `100000`               | Safety limit - adjust for volume                                 |
+| `overlapBufferSeconds`      | Incremental safety window    | `60`                   | Prevents missed records                                          |
+| `requireAbsolutePaths`      | Require absolute SFTP paths  | `"true"`               | `"true"` for AWS Transfer Family, `"false"` for standard OpenSSH |
+| `validateConnectionOnStart` | Fail-fast connection testing | `"false"`              | `"true"` to validate auth before extraction (optional)           |
+**🆕 New Activation Variables (v1.1.0):**
+- **`validateConnectionOnStart`**: When set to `"true"`, validates Fluent Commerce connection before extraction starts. Executes `query { me { ref } }` to verify authentication. Default: `"false"` (validation happens on first API call). Use for fail-fast behavior in production environments.
+---
+### ?? State Management & Incremental Sync
+**How incremental sync works:**
+1. **First Run:** Uses `DEFAULT_FALLBACK` date (2024-01-01T00:00:00Z)
+2. **Subsequent Runs:** Uses `lastInventoryPositionSync` timestamp from KV store
+3. **Overlap Buffer:** Subtracts 60 seconds to catch late-arriving records
+4. **State Update:** After successful upload, stores max `updatedOn` for next run
+**Incremental vs Manual Modes:**
+| Mode                        | When to Use          | State Update | Payload Example                                                |
+| --------------------------- | -------------------- | ------------ | -------------------------------------------------------------- |
+| **Incremental**             | Daily scheduled sync | ? Yes       | `{}` (empty - uses last sync)                                  |
+| **Manual Range**            | Historical backfill  |  No       | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": false }` |
+| **Manual Range with State** | One-time catch-up    | ? Yes       | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": true }`  |
+**Why overlap buffer?**
+Records updated near the sync time might not appear in the query due to:
+- Clock drift between systems
+- Transaction timing in the database
+- GraphQL query execution timing
+The 60-second buffer ensures these edge-case records are captured in the next run, preventing data loss.
+**When to skip state update (`updateState: false`):**
+- Historical backfills (don't affect ongoing incremental sync)
+- Testing/debugging specific date ranges
+- Reprocessing old data without changing the sync pointer
+---
+### 📁 SFTP Path Configuration
+**Note:** Folder paths and filename patterns are configured separately.
+- **Folder Path:** Configured via `sftpPath` activation variable (e.g., `/incoming/inventory/`)
+- **Filename Pattern:** Configured via `fileNamePrefix` activation variable (e.g., `inventorypositions`)
+**The workflow generates files like:** `{sftpPath}/{fileNamePrefix}-{timestamp}.xml`
+**Example:** With `sftpPath="/incoming/inventory/"` and `fileNamePrefix="inventorypositions"`, a generated file will look like:
+```
+/incoming/inventory/inventorypositions-2025-10-27T18-30-45Z.xml
+```
+Format may vary slightly (colons replaced; includes Z).
+**Note:** To change the upload folder, modify the `sftpPath` activation variable (not in code).
+---
+### Auto-pagination and limits (ExtractionOrchestrator)
+**What:** ExtractionOrchestrator handles GraphQL Relay cursor-based pagination automatically.
+**Why:** Prevents manual pagination loop code, handles large datasets efficiently.
+**How:** You configure `pageSize` and `maxRecords`; the orchestrator injects `$first` and `$after` variables automatically and loops until `pageInfo.hasNextPage === false` or `maxRecords` is reached.
+**Critical:** Your query MUST include `edges { cursor }` and `pageInfo { hasNextPage }` fields, or pagination will fail.
+#### How Auto-Pagination Works (Step-by-Step)
+Note: Orchestrator injects $first/$after automatically, flattens via resultPath, and continues while pageInfo.hasNextPage and total < maxRecords.
+#### Configuration Parameters
+| Parameter        | Purpose                        | Example                           | Effect                     |
+| ---------------- | ------------------------------ | --------------------------------- | -------------------------- |
+| `pageSize`       | Records per GraphQL request    | `200`                             | Controls `first` variable  |
+| `maxRecords`     | Total extraction limit         | `100000`                          | Hard stop across all pages |
+| `resultPath`     | Where records live in response | `"inventoryPositions.edges.node"` | Flattening path            |
+| `validateItem()` | Optional record filter         | `(item) => !!item.ref`            | Skips invalid records      |
+#### Example Configuration
+```typescript
+const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
+const maxRecords = parseInt(activation.getVariable('maxRecords') || '100000', 10);
+const extractionResult = await orchestrator.extract({
+  query: INVENTORY_POSITIONS_EXTRACTION_QUERY,
+  resultPath: 'inventoryPositions.edges.node',
+  variables: {
+    dateRangeFilter,
+    // Note: Don't include 'first' or 'after' here; orchestrator injects them based on pageSize.
+  },
+  pageSize,
+  maxRecords,
+  validateItem: item => !!(item.ref && item.productRef && typeof item.onHand === 'number'),
+});
+// Stats available after extraction (Versori)
+log.info('Extraction stats', {
+  totalRecords: extractionResult.stats.totalRecords,
+  totalPages: extractionResult.stats.totalPages,
+  validRecords: extractionResult.stats.validRecords ?? extractionResult.data.length,
+  failedValidations: extractionResult.stats.failedValidations,
+});
+```
+#### GraphQL Query Requirements
+**Your query MUST include these pagination fields:**
+```graphql
+query GetInventoryPositions(
+  $dateRangeFilter: DateRange
+  $first: Int! # ?� Orchestrator injects this
+  $after: String # ?� Orchestrator injects this
+) {
+  inventoryPositions(
+    updatedOn: $dateRangeFilter
+    first: $first # ?� Pagination page size
+    after: $after # ?� Pagination cursor
+  ) {
+    edges {
+      # ?� REQUIRED: Relay connection structure
+      node {
+        # ?� REQUIRED: Actual records here
+        id
+        ref
+        # ... your fields
+      }
+      cursor # ?� REQUIRED: Orchestrator uses this for next page
+    }
+    pageInfo {
+      # ?� REQUIRED: Orchestrator checks this
+      hasNextPage # ?� REQUIRED: Tells orchestrator to continue
+    }
+  }
+}
+```
+---
+## 📄 Mapping Configuration
+**File:** `config/inventory-positions.export.xml.json`
+```json
+{
+  "name": "inventory-positions.export.xml",
+  "version": "1.0.0",
+  "description": "Inventory Positions ? XML Export Mapping",
+  "fields": {
+    "position_ref": {
+      "source": "ref",
+      "required": true,
+      "resolver": "sdk.trim"
+    },
+    "product_ref": {
+      "source": "productRef",
+      "required": true,
+      "resolver": "sdk.trim"
+    },
+    "location_ref": {
+      "source": "locationRef",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "on_hand": {
+      "source": "onHand",
+      "required": true,
+      "resolver": "sdk.parseInt"
+    },
+    "position_type": {
+      "source": "type",
+      "required": true,
+      "resolver": "sdk.uppercase"
+    },
+    "status": {
+      "source": "status",
+      "required": false,
+      "resolver": "sdk.uppercase"
+    },
+    "catalogue_ref": {
+      "source": "catalogue.ref",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "catalogue_name": {
+      "source": "catalogue.name",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "created_on": {
+      "source": "createdOn",
+      "required": true,
+      "resolver": "sdk.toString"
+    },
+    "updated_on": {
+      "source": "updatedOn",
+      "required": true,
+      "resolver": "sdk.toString"
+    }
+  }
+}
+```
+**AI Customization Hints:**
+- Add fields: Copy existing field config, change `source` path
+- Remove fields: Delete field from config
+- Change resolvers: Replace `sdk.trim` with `sdk.uppercase`, etc.
+- Nested fields: Use dot notation like `catalogue.ref`
+Note: Customize mapping by editing the JSON above; prefer built-in resolvers. See SDK Universal Mapping guide for advanced usage.
+---
+## Versori Workflows Structure
+**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
+**Trigger Types:**
+- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
+- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
+- **`workflow()`** → Durable workflows (advanced, rarely used)
+**Execution Steps (chained to triggers):**
+- **`http()`** → External API calls (chained from schedule/webhook)
+- **`fn()`** → Internal processing (chained from schedule/webhook)
+### Recommended Project Structure
+```
+inventory-positions-extraction/
+├── index.ts                              # Entry point - exports all workflows
+└── src/
+    ├── workflows/
+    │   ├── scheduled/
+    │   │   └── daily-inventory-positions-extraction.ts         # Scheduled workflow
+    │   └── webhook/
+    │       ├── adhoc-inventory-positions-extraction.ts         # Webhook: Manual trigger
+    │       └── job-status-check.ts                             # Webhook: Status query
+    │
+    ├── services/
+    │   └── inventory-positions-extraction.service.ts    # Shared orchestration logic (reusable)
+    │
+    └── config/
+        └── inventory-positions.export.xml.json          # Mapping configuration
+```
+---
+### 1. Entry Point (`index.ts`)
+**Purpose**: Register all workflows with Versori platform
+**Pattern**: Use simple re-exports (NOT `MemoryInterpreter` pattern)
+```typescript
+/**
+ * Entry point - Export all workflows for Versori platform
+ *
+ * This file exports all workflows to be registered with Versori.
+ * Each workflow is defined in its own file for better organization.
+ */
+// Scheduled workflows
+export { scheduledInventoryPositionsExtraction } from './src/workflows/scheduled/daily-inventory-positions-extraction';
+// Webhook workflows
+export { adhocInventoryPositionsExtraction } from './src/workflows/webhook/adhoc-inventory-positions-extraction';
+export { inventoryPositionsJobStatus } from './src/workflows/webhook/job-status-check';
+```
+**IMPORTANT**: Do NOT use `MemoryInterpreter` or complex patterns in `index.ts`. Simple re-exports are the Versori standard.
+### 2. Workflow Definitions
+The code examples below show the three workflow types that go in separate files:
+- `src/workflows/scheduled/daily-inventory-positions-extraction.ts` - Scheduled workflow
+- `src/workflows/webhook/adhoc-inventory-positions-extraction.ts` - Webhook: Manual trigger
+- `src/workflows/webhook/job-status-check.ts` - Webhook: Status query
+### 3. Main Orchestration Service (`src/services/inventory-positions-extraction.service.ts`)
+**Purpose**: Shared orchestration logic reused by all workflows
+**Note:** This service coordinates the complete extraction workflow:
+- Initialize clients and services
+- Determine date range (incremental vs manual)
+- Extract data using ExtractionOrchestrator
+- Transform using UniversalMapper
+- Generate XML using XMLBuilder
+- Upload to SFTP
+- Track job progress with JobTracker
+- Update state for next run
+**NAMING PATTERN** (consistent across all use cases):
+- Interface: `{Entity}ExtractionParams` (e.g., `InventoryPositionExtractionParams`)
+- Function: `execute{Entity}Extraction` (e.g., `executeInventoryPositionExtraction`)
+- Service file: `{entity}-extraction.service.ts` (e.g., `inventory-positions-extraction.service.ts`)
+```typescript
+/**
+ * Workflows - Defines 3 execution patterns for inventory positions extraction
+ *
+ * WORKFLOW 1: Scheduled (Cron) - Runs automatically every hour
+ * WORKFLOW 2: Ad hoc (Webhook) - Manual trigger with optional date override
+ * WORKFLOW 3: Job Status (Webhook) - Query job progress
+ *
+ * AI CUSTOMIZATION HINTS:
+ * - Change schedule: Modify cron expression in schedule()
+ * - Add filtering: Pass additional params to executeInventoryPositionExtraction()
+ * - Change response format: Modify return object structure
+ */
+import { schedule, webhook, http, fn } from '@versori/run';
+import { executeInventoryPositionExtraction, getJobStatus } from '../services/extraction-orchestration';
+import { generateJobId, extractFileName } from '../utils/job-id-generator';  // We'll create this
+/**
+ * WORKFLOW 1: Scheduled Extraction
+ *
+ * Purpose: Automated hourly extraction for incremental sync
+ * Trigger: Cron schedule (every hour at minute 0)
+ * State Update: Always updates lastSync timestamp
+ *
+ * AI CUSTOMIZATION:
+ * - Change schedule: Replace '0 * * * *' with your cron expression
+ *   Examples:
+ *   - Every 30 min: '*/30 * * * *'
+ *   - Daily at 2 AM: '0 2 * * *'
+ *   - Every 15 min: '*/15 * * * *'
+ */
+export const scheduledInventoryPositionsExtraction = schedule('scheduled-inventory-positions-extraction', '0 * * * *')  // ? CUSTOMIZE: Cron expression
+.then(
+  http('execute-scheduled-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
+    const { log } = ctx;
+    // Generate unique job ID for tracking
+    // Format: SCHEDULED_IP_YYYYMMDD_HHMMSS_random
+    const jobId = generateJobId('SCHEDULED', 'INVENTORY_POSITIONS');
+    log.info('Scheduled extraction triggered', { jobId });
+    try {
+      // Execute main workflow (extraction ? transform ? upload)
+      const result = await executeInventoryPositionExtraction(ctx, {
+        jobId,
+        triggeredBy: 'schedule',
+        updateState: true,        // Always update state for scheduled runs
+        // positionTypes: ['DEFAULT', 'SEASONAL'],
+        // locationRefs: ['LOC-001'],
+      });
+      log.info('Scheduled extraction completed', {
+        jobId,
+        recordCount: result.recordsExtracted,
+        fileName: result.fileName
+      });
+      return result;
+    } catch (error: any) {
+      log.error('Scheduled extraction failed', {
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error'
+      });
+      throw error;
+    }
+  })
+);
+/**
+ * WORKFLOW 2: Ad hoc Extraction (Manual Trigger)
+ *
+ * Purpose: Manual extraction with optional date range override
+ * Trigger: Webhook POST to /webhooks/inventory-positions-adhoc
+ * State Update: Optional (controlled by request payload)
+ *
+ * WEBHOOK PAYLOAD EXAMPLES:
+ *
+ * 1. Incremental (use last sync timestamp):
+ *    {}
+ *
+ * 2. Date range (manual override):
+ *    {
+ *      "fromDate": "2025-01-01T00:00:00Z",
+ *      "toDate": "2025-01-31T23:59:59Z",
+ *      "updateState": false
+ *    }
+ *
+ * 3. Specific time window:
+ *    {
+ *      "fromDate": "2025-01-15T08:00:00Z",
+ *      "toDate": "2025-01-15T17:00:00Z",
+ *      "updateState": false
+ *    }
+ *
+ * AI CUSTOMIZATION:
+ * - Add request validation
+ * - Add authentication check
+ * - Add custom filters from payload
+ */
+export const adhocInventoryPositionsExtraction = webhook(
+  'inventory-positions-adhoc',
+  { connection: 'inventory-positions-adhoc', response: { mode: 'sync' } }
+)
+.then(
+  http('execute-adhoc-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
+    const { data, log } = ctx;
+    // Generate unique job ID
+    const jobId = generateJobId('ADHOC', 'INVENTORY_POSITIONS');
+    // Security handled by Versori connection
+    // Extract optional date override from webhook payload
+    const fromDate = data.fromDate as string | undefined;
+    const toDate = data.toDate as string | undefined;
+    const updateState = data.updateState === true;  // Default false; advance state only if explicitly true
+    log.info('Ad hoc extraction triggered via webhook', {
+      jobId,
+      hasDateOverride: !!fromDate,
+      fromDate: fromDate || 'not specified',
+      toDate: toDate || 'not specified',
+      updateState
+    });
+    try {
+      // Execute main workflow with optional overrides
+      const result = await executeInventoryPositionExtraction(ctx, {
+        jobId,
+        triggeredBy: 'webhook',
+        fromDate,              // Optional: override start date
+        toDate,                // Optional: override end date
+        updateState,           // Optional: skip state update for historical queries
+        // positionTypes: data.positionTypes,
+        // locationRefs: data.locationRefs,
+      });
+      log.info('Ad hoc extraction completed', {
+        jobId,
+        recordCount: result.recordsExtracted,
+        fileName: result.fileName,
+        isManualOverride: !!fromDate,
+        stateUpdated: result.stateUpdated
+      });
+      // Return success with job details
+      return {
+        success: true,
+        jobId,
+        recordsExtracted: result.recordsExtracted,
+        fileName: result.fileName,
+        sftpPath: result.sftpPath,
+        statusUrl: `/webhooks/inventory-positions-job-status?jobId=${jobId}`,
+        dateRange: fromDate ? {
+          from: fromDate,
+          to: toDate || 'not specified',
+          updateState
+        } : undefined
+      };
+    } catch (error: any) {
+      log.error('Ad hoc extraction failed', {
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error'
+      });
+      return {
+        success: false,
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',
+      };
+    }
+  })
+);
+/**
+ * WORKFLOW 3: Job Status Query
+ *
+ * Purpose: Check job progress and status
+ * Trigger: Webhook GET/POST to /webhooks/inventory-positions-job-status?jobId=xxx
+ * Returns: Current job status, stage, progress
+ *
+ * QUERY EXAMPLES:
+ *
+ * 1. HTTP GET:
+ *    GET /webhooks/inventory-positions-job-status?jobId=ADHOC_IP_20251027_183045_abc123
+ *
+ * 2. HTTP POST:
+ *    POST /webhooks/inventory-positions-job-status
+ *    { "jobId": "ADHOC_IP_20251027_183045_abc123" }
+ *
+ * RESPONSE EXAMPLE:
+ * {
+ *   "success": true,
+ *   "jobId": "ADHOC_IP_20251027_183045_abc123",
+ *   "status": "processing",
+ *   "stage": "transforming",
+ *   "message": "Transforming 15000 records",
+ *   "createdAt": "2025-10-27T18:30:45.000Z",
+ *   "startedAt": "2025-10-27T18:30:46.000Z"
+ * }
+ *
+ * AI CUSTOMIZATION:
+ * - Add detailed progress percentage
+ * - Add estimated time remaining
+ * - Add historical job queries
+ */
+export const inventoryPositionsJobStatus = webhook(
+  'inventory-positions-job-status',
+  { connection: 'inventory-positions-job-status', response: { mode: 'sync' } }
+)
+.then(
+  fn('query-job-status', async (ctx) => {
+    const { data, log, openKv } = ctx;
+    // Get jobId from query param or POST body
+    const jobId = data.jobId as string;
+    if (!jobId) {
+      log.error('Job ID not provided in request');
+      return {
+        success: false,
+        error: 'Job ID is required. Provide jobId in query param or request body.'
+      };
+    }
+    log.info('Querying job status', { jobId });
+    try {
+      // Query job status from KV store
+      const status = await getJobStatus(openKv(':project:'), jobId, log);
+      if (!status) {
+        log.info('Job not found', { jobId });
+        return {
+          success: false,
+          error: 'Job not found',
+          jobId
+        };
+      }
+      log.info('Job status retrieved', { jobId, status: status.status });
+      return {
+        success: true,
+        jobId,
+        ...status
+      };
+    } catch (error: any) {
+      log.error('Failed to query job status', {
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',
+      });
+      return {
+        success: false,
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',
+      };
+    }
+  })
+);
+```
+---
+# Production Template: Service Implementation
+## 3. Main Orchestration Service (src/services/extraction-orchestration.ts)
+```typescript
+/**
+ * MAIN ORCHESTRATION SERVICE
+ *
+ * This is the heart of the extraction workflow. It coordinates all steps:
+ * 1. Initialize clients and services
+ * 2. Determine date range (incremental vs manual)
+ * 3. Extract data using ExtractionOrchestrator
+ * 4. Transform using UniversalMapper
+ * 5. Generate XML using XMLBuilder
+ * 6. Upload to SFTP
+ * 7. Track job progress with JobTracker
+ * 8. Update state for next run
+ *
+ * NAMING PATTERN (consistent across all use cases):
+ * - Interface: {Entity}ExtractionParams (e.g., InventoryPositionExtractionParams)
+ * - Result: {Entity}ExtractionResult (e.g., InventoryPositionExtractionResult)
+ * - Main function: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
+ *
+ * AI CUSTOMIZATION HINTS:
+ * - Change entity: Replace "InventoryPosition" with "VirtualPosition", "Order", etc.
+ * - Change output: Replace XMLBuilder with CSVParserService.stringify()
+ * - Change destination: Replace SftpDataSource with S3DataSource
+ * - Add steps: Insert new service calls between existing steps
+ */
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  XMLBuilder,
+  SftpDataSource,
+  type ExtractionOptions,
+  type ExtractionResult,
+  type JobStatus,
+} from '@fluentcommerce/fc-connect-sdk';
+import mappingConfig from '../../config/inventory-positions.export.xml.json' with { type: 'json' };
+import { extractFileName } from '../../utils/job-id-generator';
+// ? VERSORI PLATFORM: Use native log from context, not LoggingService
+/**
+ * Parameters for extraction workflow
+ *
+ * NAMING: {Entity}ExtractionParams
+ */
+export interface InventoryPositionExtractionParams {
+  jobId: string;
+  triggeredBy: 'schedule' | 'webhook';
+  fromDate?: string; // Optional: manual date override
+  toDate?: string; // Optional: manual date override
+  updateState: boolean; // Whether to update lastSync timestamp
+  // AI CUSTOMIZATION: Add filters specific to entity
+  positionTypes?: string[]; // e.g., ['DEFAULT', 'SEASONAL']
+  locationRefs?: string[]; // e.g., ['LOC-001']
+}
+/**
+ * Result from extraction workflow
+ *
+ * NAMING: {Entity}ExtractionResult
+ */
+export interface InventoryPositionExtractionResult {
+  success: boolean;
+  jobId: string;
+  recordsExtracted: number;
+  fileName?: string;
+  sftpPath?: string;
+  error?: string;
+  errors?: any[];
+  isManualOverride?: boolean;
+  stateUpdated?: boolean;
+  newTimestamp?: string;
+}
+/**
+ * GraphQL Query for Inventory Positions
+ *
+ * NAMING: {ENTITY}_EXTRACTION_QUERY (uppercase constant)
+ *
+ * Replace entity/fields as needed but keep pagination structure (edges, node, pageInfo)
+ */
+const INVENTORY_POSITIONS_EXTRACTION_QUERY = `
+  query GetInventoryPositions(
+    $dateRangeFilter: DateRange
+    $productRefs: [String!]
+    $types: [String!]
+    $locationRefs: [String]
+    $first: Int!
+    $after: String
+  ) {
+    inventoryPositions(
+      updatedOn: $dateRangeFilter
+      productRef: $productRefs
+      type: $types
+      locationRef: $locationRefs
+      first: $first
+      after: $after
+    ) {
+      edges {
+        node {
+          id
+          ref
+          productRef
+          locationRef
+          onHand
+          type
+          status
+          catalogue {
+            ref
+            name
+          }
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+/**
+ * Query job status from KV store
+ *
+ * NAMING: get{Entity}JobStatus or just getJobStatus (generic)
+ *
+ * ? VERSORI PLATFORM: Uses native log from context (passed as parameter)
+ */
+export async function getJobStatus(
+  kv: any, // ? Versori KV (compatible with JobTracker's KVAdapter interface)
+  jobId: string,
+  log: any // ? Native Versori log from context
+): Promise<JobStatus | undefined> {
+  try {
+    const tracker = new JobTracker(kv, log);
+    return await tracker.getJob(jobId); // ? Use getJob() not getJobStatus()
+  } catch (error: any) {
+    log.error('Failed to get job status', { jobId, message: error instanceof Error ? error.message : String(error),
+ stack: error instanceof Error ? error.stack : undefined,
+ errorType: error instanceof Error ? error.constructor.name : 'Error', });
+    return undefined;
+  }
+}
+/**
+ * MAIN ORCHESTRATION FUNCTION
+ *
+ * NAMING: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
+ *
+ * This function implements the complete workflow in steps.
+ * Each step is clearly commented for AI understanding.
+ */
+export async function executeInventoryPositionExtraction(
+  ctx: any,
+  params: InventoryPositionExtractionParams
+): Promise<InventoryPositionExtractionResult> {
+  // ? VERSORI PLATFORM: Extract native log from context
+  const { log, openKv, activation, credentials } = ctx;
+  const { jobId, triggeredBy, fromDate, toDate, updateState } = params;
+  // ⏱️ Track total execution time
+  const startTime = Date.now();
+  // Open KV store for state management and job tracking
+  // ? Pass raw Versori KV directly - it matches KVAdapter interface
+  // ? Pass native log to JobTracker
+  const kv = openKv(':project:');
+  const tracker = new JobTracker(kv, log);
+  try {
+    // �����������������������������������������������������������
+    // STEP 1/8: Initialize Job Tracking
+    // �����������������������������������������������������������
+    log.info('📊 [STEP 1/8] Initializing job tracking', { jobId });
+    await tracker.createJob(jobId, {
+      triggeredBy,
+      hasDateOverride: !!fromDate,
+      fromDate,
+      toDate,
+      updateStateAfterRun: updateState,
+    });
+    // �����������������������������������������������������������
+    // STEP 2/8: Initialize Fluent Client
+    // �����������������������������������������������������������
+    log.info('🔌 [STEP 2/8] Initializing Fluent Commerce client', { jobId });
+    // ✅ Optional: Validate connection immediately (fail-fast mode)
+    // Set activation variable 'validateConnectionOnStart' = 'true' to enable
+    // When enabled: Executes query { me { ref } } to verify authentication
+    // When disabled: Fast creation, validation happens on first API call (default)
+    const validateConnection = activation.getVariable('validateConnectionOnStart') === 'true';
+    const client = await createClient(ctx, validateConnection ? { validateConnection: true } : undefined);
+    if (!client) {
+      throw new Error('Failed to create Fluent Commerce client');
+    }
+    if (validateConnection) {
+      log.info('✅ Connection validated successfully - authentication confirmed', { jobId });
+    }
+    // �����������������������������������������������������������
+    // STEP 3/8: Determine Date Range (WITH overlap buffer)
+    // �����������������������������������������������������������
+    log.info('📅 [STEP 3/8] Determining date range for extraction', { jobId });
+    // State key for incremental sync tracking
+    // NAMING: last{Entity}Sync (e.g., lastInventoryPositionSync)
+    const STATE_KEY = 'lastInventoryPositionSync';
+    const DEFAULT_FALLBACK = '2024-01-01T00:00:00Z';
+    const OVERLAP_BUFFER_SECONDS = parseInt(
+      activation.getVariable('overlapBufferSeconds') || '60',
+      10
+    );
+    let dateRangeFilter: { from?: string; to?: string } | null = null;
+    const isManualOverride = !!fromDate;
+    if (isManualOverride) {
+      // Manual date override from webhook
+      dateRangeFilter = { from: fromDate, to: toDate };
+      log.info('Using manual date override', { fromDate, toDate });
+    } else {
+      // Incremental sync - get last sync timestamp
+      const rawLastRunTime = (await kv.get<string>(STATE_KEY)) || DEFAULT_FALLBACK;
+      // Apply overlap buffer (prevents missed records)
+      const bufferedLastRunTime = new Date(
+        new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_SECONDS * 1000
+      ).toISOString();
+      const effectiveEndTime = toDate || new Date().toISOString();
+      dateRangeFilter = {
+        from: bufferedLastRunTime,
+        to: effectiveEndTime, // End of extraction window
+      };
+      log.info('Using incremental sync with overlap buffer', {
+        rawLastRunTime,
+        bufferedLastRunTime,
+        effectiveEndTime,
+        overlapBufferSeconds: OVERLAP_BUFFER_SECONDS,
+      });
+    }
+    // �����������������������������������������������������������
+    // STEP 4/8: Extract Data (ExtractionOrchestrator)
+    // �����������������������������������������������������������
+    log.info('📥 [STEP 4/8] Extracting data from Fluent Commerce', { jobId });
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 'extraction',
+      message: 'Extracting data with auto-pagination',
+    });
+    // Configure extraction
+    const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
+    const maxRecords = parseInt(activation.getVariable('maxRecords') || '100000', 10);
+    // Memory management: maxRecords prevents unbounded memory; for very large outputs, split files or use S3.
+    // Performance note: pageSize=200 is a balanced default; split output if records > ~50k for optimal performance.
+    // ? Enhanced: Extract context for progress logging
+    const dateRangeInfo = {
+      start: dateRangeFilter?.from || 'N/A',
+      end: dateRangeFilter?.to || 'N/A',
+      types: params.positionTypes?.join(', ') || 'all',
+      locationRefs: params.locationRefs?.slice(0, 3).join(', ') || 'all'
+    };
+    // ? Enhanced: Start logging with context
+    log.info(`🔍 [ExtractionOrchestrator] Starting extraction`, {
+     query: 'inventoryPositions',
+     pageSize,
+     maxRecords,
+     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
+     positionTypes: dateRangeInfo.types,
+     sampleLocations: dateRangeInfo.locationRefs,
+     jobId
+    });
+    // Initialize ExtractionOrchestrator
+    const orchestrator = new ExtractionOrchestrator(client, log);
+    // Execute extraction with auto-pagination
+    const extractionResult: ExtractionResult<any> = await orchestrator.extract({
+      query: INVENTORY_POSITIONS_EXTRACTION_QUERY,
+      resultPath: 'inventoryPositions.edges.node',
+      variables: {
+        dateRangeFilter,
+        types: params.positionTypes,
+        locationRefs: params.locationRefs,
+        // Note: Don't include 'first' or 'after' here; orchestrator injects them based on pageSize below
+      },
+      pageSize,
+      maxRecords,
+      // Optional: validate each record
+      validateItem: (item: any) => {
+        return !!(item.ref && item.productRef && typeof item.onHand === 'number');
+      },
+    });
+    const records = extractionResult.data || [];
+    log.info('Extraction complete', {
+      totalRecords: extractionResult.stats.totalRecords,
+      totalPages: extractionResult.stats.totalPages,
+      validRecords: extractionResult.stats.validRecords ?? records.length,
+      failedValidations: extractionResult.stats.failedValidations,
+      truncated: extractionResult.stats.truncated,
+      truncationReason: extractionResult.stats.truncationReason,
+      errors: extractionResult.errors ? extractionResult.errors.length : 0,
+    });
+    // ? Enhanced: Completion logging with summary
+    log.info(`✅ [ExtractionOrchestrator] Extraction completed`, {
+     totalRecords: extractionResult.stats.totalRecords,
+     totalPages: extractionResult.stats.totalPages,
+     validRecords: extractionResult.stats.validRecords ?? records.length,
+     failedValidations: extractionResult.stats.failedValidations,
+     truncated: extractionResult.stats.truncated,
+     truncationReason: extractionResult.stats.truncationReason,
+     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
+     jobId
+    });
+    if (extractionResult.errors && extractionResult.errors.length > 0) {
+      log.warn('Non-fatal extraction errors encountered', {
+        errorCount: extractionResult.errors.length,
+        sampleErrors: extractionResult.errors.slice(0, 3),
+      });
+    }
+    // Handle empty result
+    if (records.length === 0) {
+      log.info('No records to process');
+      // Update state even with no records (prevents re-querying empty window)
+      if (updateState && !isManualOverride) {
+        await kv.set(STATE_KEY, new Date().toISOString());
+      }
+      await tracker.markCompleted(jobId, {
+        recordCount: 0,
+        message: 'No records to extract',
+      });
+      return {
+        success: true,
+        jobId,
+        recordsExtracted: 0,
+      };
+    }
+    // �����������������������������������������������������������
+    // STEP 5/8: Transform Data (UniversalMapper)
+    // �����������������������������������������������������������
+    log.info('🔄 [STEP 5/8] Transforming data with UniversalMapper', {
+      jobId,
+      recordCount: records.length,
+    });
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 'transformation',
+      message: `Transforming ${records.length} records`,
+    });
+    const mapper = new UniversalMapper(mappingConfig);
+    const mappingResult = await mapper.map(records);
+    const mappingErrors = mappingResult.errors || [];
+    if (!mappingResult.success) {
+      log.error('Mapping failed - terminating job', {
+        jobId,
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+      await tracker.markFailed(
+        jobId,
+        mappingErrors[0] || 'UniversalMapper returned unsuccessful result',
+        {
+          failedCount: mappingErrors.length,
+          errors: mappingErrors,
+        }
+      );
+      return {
+        success: false,
+        jobId,
+        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
+        errors: mappingErrors,
+      };
+    }
+    const transformedRecords = Array.isArray(mappingResult.data)
+      ? (mappingResult.data as any[])
+      : [];
+    if (mappingErrors.length > 0) {
+      log.warn('Some records failed transformation', {
+        jobId,
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+    }
+    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
+      log.info('ℹ️ [MAPPING] Optional fields skipped (undefined values)', {
+        jobId,
+        skippedFields: mappingResult.skippedFields,
+        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
+      });
+    }
+    if (transformedRecords.length === 0) {
+      await tracker.markFailed(jobId, 'All records failed mapping', {
+        failedCount: mappingErrors.length,
+        errors: mappingErrors,
+      });
+      return {
+        success: false,
+        jobId,
+        error: 'All records failed mapping',
+        errors: mappingErrors,
+      };
+    }
+    log.info('Transformation complete', {
+      successful: transformedRecords.length,
+      failed: mappingErrors.length,
+      skippedRecords: records.length - transformedRecords.length,
+    });
+    // �����������������������������������������������������������
+    // STEP 6/8: Generate XML (XMLBuilder)
+    // �����������������������������������������������������������
+    log.info('📝 [STEP 6/8] Generating XML file', { jobId });
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 'xml_generation',
+      message: `Generating XML for ${transformedRecords.length} records`,
+    });
+    // Initialize XMLBuilder
+    const xmlBuilder = new XMLBuilder({
+      ignoreAttributes: false,
+      format: true,
+      indentBy: '  ',
+      suppressEmptyNode: true,
+    });
+    // Build XML structure
+    const xmlData = {
+      '?xml': {
+        '@_version': '1.0',
+        '@_encoding': 'UTF-8',
+      },
+      InventoryPositions: {
+        Position: transformedRecords,
+      },
+    };
+    const xmlContent = xmlBuilder.build(xmlData);
+    // Generate timestamped filename using extractFileName helper
+    const fileNamePrefix = activation.getVariable('fileNamePrefix') || 'inventorypositions';
+    const fileName = extractFileName(fileNamePrefix, 'xml');
+    log.info('XML file generated', {
+      fileName,
+      sizeBytes: xmlContent.length,
+      recordCount: transformedRecords.length,
+    });
+    // �����������������������������������������������������������
+    // STEP 7/8: Upload to SFTP (SftpDataSource)
+    // �����������������������������������������������������������
+    log.info('📤 [STEP 7/8] Uploading to SFTP', { jobId, fileName });
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 'sftp_upload',
+      message: `Uploading ${fileName} to SFTP`,
+    });
+    // Get SFTP credentials from Versori connection (Basic Auth)
+    // RECOMMENDED: Use activation.connections (already decoded)
+    const allConnections = ctx.activation.connections || [];
+    const sftpConn = allConnections.find(c => c.name === 'versori_ftp_server');
+    if (!sftpConn) {
+      throw new Error('SFTP connection "versori_ftp_server" not found');
+    }
+    const credential = sftpConn.credentials[0]?.credential;
+    if (!credential?.data?.basicAuth) {
+      throw new Error('SFTP connection not configured with Basic Authentication');
+    }
+    const { username, password } = credential.data.basicAuth;
+    // ? Already decoded - no Buffer.from() needed!
+    // Get SFTP configuration from activation variables
+    const sftpConfig = {
+      host: activation.getVariable('sftpHost'),
+      port: parseInt(activation.getVariable('sftpPort') || '22', 10),
+      username: sftpUsername, // From connection
+      password: sftpPassword, // From connection
+      privateKey: activation.getVariable('sftpPrivateKey'),
+      remotePath: activation.getVariable('sftpPath') || '/incoming/',
+    };
+    // Validate SFTP config
+    if (!sftpConfig.host || !username) {
+      throw new Error('SFTP configuration incomplete: missing host or username (from connection)');
+    }
+    if (!sftpConfig.password && !sftpConfig.privateKey) {
+      throw new Error('SFTP configuration incomplete: missing password or privateKey');
+    }
+    // Initialize SFTP data source
+    // ? VERSORI PLATFORM: Pass native log from context
+    const sftp = new SftpDataSource(
+      {
+        type: 'SFTP_XML',
+        connectionId: 'inventory-positions-sftp',
+        name: 'Inventory Positions SFTP Upload',
+        settings: {
+          host: sftpConfig.host,
+          port: sftpConfig.port,
+          username: sftpConfig.username,
+          password: sftpConfig.password,
+          privateKey: sftpConfig.privateKey,
+          remotePath: sftpConfig.remotePath,
+          filePattern: '*.xml',
+        },
+      },
+      log
+    );
+    try {
+      // Ensure remote directory exists (recursive)
+      await sftp.createDirectory(sftpConfig.remotePath || '/incoming/', true);
+      // Get SFTP path configuration
+      const requireAbsolutePaths = activation.getVariable('requireAbsolutePaths') === 'true';
+      /**
+       * Helper: Join SFTP path segments safely
+       *
+       * Handles different SFTP server path requirements:
+       * - AWS Transfer Family: Requires absolute paths (leading /)
+       * - Standard OpenSSH: Supports relative paths
+       *
+       * @param requireAbsolutePath - true for AWS Transfer Family, false for standard OpenSSH
+       * @param parts - Path segments to join
+       * @returns Properly formatted SFTP path
+       */
+      function joinSftpPath(requireAbsolutePath: boolean, ...parts: string[]): string {
+        // Clean each segment (remove leading/trailing slashes)
+        const cleaned = parts
+          .filter(Boolean)
+          .map(p => String(p).replace(/^\/+|\/+$/g, ''))
+          .join('/');
+        // Add leading slash if required (AWS Transfer Family)
+        return requireAbsolutePath && !cleaned.startsWith('/') ? `/${cleaned}` : cleaned;
+      }
+      // Construct full SFTP path safely
+      const fullPath = joinSftpPath(
+        requireAbsolutePaths,
+        sftpConfig.remotePath || '/incoming/',
+        fileName
+      );
+      // Upload with retry logic (built into SftpDataSource)
+      // ? Use createDirectories: true to ensure remote path exists
+      // Upload failure: If SFTP upload fails, job is failed and state is not advanced; next run retries same window.
+      await sftp.uploadFile(fullPath, Buffer.from(xmlContent, 'utf8'), {
+        createDirectories: true,
+      });
+      log.info('SFTP upload successful', { fileName, remotePath: fullPath });
+      // Optional: upload a transformation error report alongside the XML
+      if (mappingErrors.length > 0) {
+        const baseName = fileName.replace(/\.xml$/i, '');
+        const report = {
+          fileName,
+          processedAt: new Date().toISOString(),
+          errorStage: 'transform',
+          totals: {
+            totalRecords: records.length,
+            recordsTransformed: transformedRecords.length,
+            transformErrors: mappingErrors.length,
+          },
+          errors: mappingErrors,
+        };
+        const reportPath = joinSftpPath(
+          requireAbsolutePaths,
+          sftpConfig.remotePath || '/incoming/',
+          `${baseName}-errors.json`
+        );
+        await sftp.uploadFile(reportPath, Buffer.from(JSON.stringify(report, null, 2), 'utf8'), {
+          createDirectories: true,
+          overwrite: true,
+        });
+        log.info('Uploaded transformation error report', { reportPath });
+      }
+      // Optional: For very large outputs, split transformedRecords into chunks
+      // and upload multiple XML files to avoid size/memory limits.
+      // �����������������������������������������������������������
+      // STEP 8/8: Update State & Complete Job (WITHOUT buffer)
+      // �����������������������������������������������������������
+      log.info('✅ [STEP 8/8] Updating state and completing job', { jobId });
+      // Calculate new timestamp for next incremental run
+      let newTimestamp: string | undefined;
+      if (updateState && !isManualOverride) {
+        // Find max updatedOn from extracted records
+        const maxUpdatedOn = records.reduce(
+          (max, record) => {
+            const recordTime = new Date(record.updatedOn).getTime();
+            return recordTime > max ? recordTime : max;
+          },
+          new Date(dateRangeFilter?.from || DEFAULT_FALLBACK).getTime()
+        );
+        newTimestamp = new Date(maxUpdatedOn).toISOString();
+        // Store new timestamp (WITHOUT buffer - buffer only applied on read)
+        await kv.set(STATE_KEY, newTimestamp);
+        log.info('State updated', {
+          oldTimestamp: dateRangeFilter?.from,
+          newTimestamp,
+        });
+      }
+      // Mark job as completed
+      await tracker.markCompleted(jobId, {
+        recordCount: transformedRecords.length,
+        fileName,
+        remotePath: fullPath,
+        errorCount: mappingErrors.length,
+        errors: mappingErrors,
+        isManualOverride,
+        stateUpdated: updateState && !isManualOverride,
+        newTimestamp,
+      });
+      // ⏱️ Calculate total execution time
+      const totalDuration = Date.now() - startTime;
+      log.info('✅ Extraction workflow completed successfully', {
+        jobId,
+        totalDuration,
+        recordsExtracted: transformedRecords.length,
+        fileName,
+      });
+      return {
+        success: true,
+        jobId,
+        recordsExtracted: transformedRecords.length,
+        fileName,
+        sftpPath: fullPath,
+        isManualOverride,
+        stateUpdated: updateState && !isManualOverride,
+        newTimestamp,
+        duration: totalDuration,
+        errors: mappingErrors.length > 0 ? mappingErrors : undefined,
+      };
+    } finally {
+      // ?��️ CRITICAL: Always dispose SFTP connection
+      await sftp.dispose();
+      log.info('SFTP connection disposed');
+    }
+  } catch (error: any) {
+    const totalDuration = Date.now() - startTime;
+    log.error('❌ Extraction workflow failed', {
+      jobId,
+      duration: totalDuration,
+      message: error instanceof Error ? error.message : String(error),
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'Error',
+      recommendation: getErrorRecommendation(error),
+    });
+    // Mark job as failed
+    await tracker.markFailed(jobId, error);
+    return {
+      success: false,
+      jobId,
+      recordsExtracted: 0,
+      duration: totalDuration,
+      message: error instanceof Error ? error.message : String(error),
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'Error',
+      recommendation: getErrorRecommendation(error),
+    };
+  }
+}
+/**
+ * Get actionable recommendation based on error type
+ */
+function getErrorRecommendation(error: any): string {
+  const message = error instanceof Error ? error.message : String(error);
+  if (message.includes('SFTP') || message.includes('connection')) {
+    return 'Check SFTP credentials and network connectivity. Verify the SFTP connection is configured in Versori Connections with Basic Authentication.';
+  }
+  if (message.includes('GraphQL') || message.includes('query')) {
+    return 'Verify GraphQL query syntax and schema compatibility. Check Fluent Commerce connection credentials.';
+  }
+  if (message.includes('mapping') || message.includes('transform')) {
+    return 'Review mapping configuration. Run schema validation: npx fc-connect validate-schema --mapping config.json --schema schema.json';
+  }
+  if (message.includes('authentication') || message.includes('401')) {
+    return 'Check Fluent Commerce OAuth2 credentials. Verify clientId and clientSecret in connection configuration.';
+  }
+  if (message.includes('timeout')) {
+    return 'Reduce maxRecords or pageSize to process smaller batches. Check network latency and API response times.';
+  }
+  return 'Review error details above. Check activation variables and connection configuration. Consult documentation for troubleshooting steps.';
+}
+```
+Note: Customize mapping by editing the JSON above; prefer built-in resolvers. See SDK Universal Mapping guide for advanced usage.
+---
+## 4. Utility Functions (src/utils/)
+### Job ID Generator (src/utils/job-id-generator.ts)
+```typescript
+/**
+ * Job ID Generator
+ *
+ * Generates unique job IDs for tracking extraction workflows
+ *
+ * FORMAT: {TYPE}_{ENTITY}_{YYYYMMDD}_{HHMMSS}_{RANDOM}
+ * Example: SCHEDULED_IP_20251027_183045_a1b2c3
+ *
+ * NAMING: generate{Entity}JobId or generateJobId (generic)
+ */
+/**
+ * Generate unique job ID
+ *
+ * @param type - Job trigger type (SCHEDULED, ADHOC, MANUAL)
+ * @param entity - Entity abbreviation (IP, VP, ORD, PRD)
+ * @returns Unique job ID string
+ */
+export function generateJobId(type: string, entity: string): string {
+  const now = new Date();
+  // Format: YYYYMMDD
+  const dateStr = now.toISOString().slice(0, 10).replace(/-/g, '');
+  // Format: HHMMSS
+  const timeStr = now.toISOString().slice(11, 19).replace(/:/g, '');
+  // Random suffix (6 chars)
+  const randomStr = Math.random().toString(36).substring(2, 8);
+  return `${type}_${entity}_${dateStr}_${timeStr}_${randomStr}`;
+}
+/**
+ * Generate timestamped filename for extraction output
+ *
+ * @param prefix - Filename prefix (from activation variable)
+ * @param extension - File extension (xml, csv, json)
+ * @returns Timestamped filename
+ *
+ * Example: extractFileName('inventorypositions', 'xml')
+ * Returns: 'inventorypositions-2025-11-04T14-30-45-123Z.xml'
+ */
+export function extractFileName(prefix: string, extension: string = 'xml'): string {
+  const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+  return `${prefix}-${timestamp}.${extension}`;
+}
+/**
+ * Parse job ID components
+ *
+ * @param jobId - Job ID to parse
+ * @returns Parsed components or null if invalid
+ */
+export function parseJobId(jobId: string): {
+  type: string;
+  entity: string;
+  date: string;
+  time: string;
+  random: string;
+} | null {
+  const parts = jobId.split('_');
+  if (parts.length !== 5) {
+    return null;
+  }
+  return {
+    type: parts[0],
+    entity: parts[1],
+    date: parts[2],
+    time: parts[3],
+    random: parts[4],
+  };
+}
+```
+---
+## 5. Package Configuration
+### package.json
+```json
+{
+  "name": "inventory-positions-to-sftp-xml",
+  "version": "1.0.0",
+  "description": "Extract inventory positions from Fluent Commerce and export to SFTP as XML",
+  "type": "module",
+  "main": "src/index.ts",
+  "scripts": {
+    "dev": "versori dev",
+    "build": "versori build",
+    "deploy": "versori deploy"
+  },
+  "dependencies": {
+    "@fluentcommerce/fc-connect-sdk": "^0.1.39",
+    "@versori/run": "latest",
+    "fast-xml-parser": "^5.2.5"
+  },
+  "devDependencies": {
+    "@types/node": "^20.0.0",
+    "typescript": "^5.0.0"
+  }
+}
+```
+### tsconfig.json
+```json
+{
+  "compilerOptions": {
+    "module": "ES2022",
+    "target": "ES2024",
+    "moduleResolution": "node"
+  }
+}
+```
+---
+## 6. Deployment Instructions
+### Deploy to Versori
+```bash
+# 1. Install dependencies
+npm install
+# 2. Test locally (if using Versori CLI)
+npm run dev
+# 3. Deploy to Versori platform
+npm run deploy
+```
+### Configure Activation Variables
+In Versori platform settings, configure:
+```json
+{
+  "sftpHost": "sftp.partner.com",
+  "sftpPort": 22,
+  "sftpUsername": "export_user",
+  "sftpPassword": "********",
+  "sftpPath": "/incoming/inventory/",
+  "fileNamePrefix": "inventorypositions",
+  "pageSize": 200,
+  "maxRecords": 100000,
+  "overlapBufferSeconds": 60
+}
+```
+---
+## 7. Performance Notes
+- **Pagination:** Default pageSize=200 is optimized for most use cases; increase to 500 if network latency is high
+- **Large outputs:** For extractions exceeding ~50k records, consider splitting into multiple files or streaming to S3
+- **Memory limits:** maxRecords parameter prevents unbounded memory growth; adjust based on available memory
+---
+## 8. Testing
+### Test Scheduled Extraction
+The scheduled workflow runs automatically based on cron schedule.
+**Check logs:**
+```
+[STEP 1/8] Initializing job tracking
+[STEP 2/8] Initializing Fluent Commerce client
+[STEP 3/8] Determining date range for extraction
+[STEP 4/8] Extracting data from Fluent Commerce
+[STEP 5/8] Transforming data with UniversalMapper
+[STEP 6/8] Generating XML file
+[STEP 7/8] Uploading to SFTP
+[STEP 8/8] Updating state and completing job
+```
+### Test Ad hoc Extraction
+```bash
+# Incremental (uses last sync timestamp)
+curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
+  -H "Content-Type: application/json" \
+  -H "X-API-Key: your-api-key" \
+  -d '{}'
+# Date range override
+curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
+  -H "Content-Type: application/json" \
+  -H "X-API-Key: your-api-key" \
+  -d '{
+    "fromDate": "2025-01-01T00:00:00Z",
+    "toDate": "2025-01-31T23:59:59Z",
+    "updateState": false
+  }'
+```
+### Test Job Status Query
+```bash
+curl -X POST https://api.versori.com/webhooks/inventory-positions-job-status \
+  -H "Content-Type: application/json" \
+  -H "X-API-Key: your-api-key" \
+  -d '{
+    "jobId": "ADHOC_IP_20251027_183045_abc123"
+  }'
+```
+**Response:**
+```json
+{
+  "success": true,
+  "jobId": "ADHOC_IP_20251027_183045_abc123",
+  "status": "processing",
+  "stage": "transformation",
+  "message": "Transforming 15000 records",
+  "createdAt": "2025-10-27T18:30:45.000Z",
+  "startedAt": "2025-10-27T18:30:46.000Z"
+}
+```
+### Monitoring
+Use Versori logs filtered by `jobId` or `stage`; optional status webhook for dashboards.
+---
+### Schema requirements
+Ensure your GraphQL query includes Relay pagination fields and variables ($first, $after), and that `resultPath` matches the edges.node path. The orchestrator injects pagination variables automatically (don't include them in your variables object).
+---
+## ?? Troubleshooting (quick)
+- No records extracted: Check dateRange (manual override vs incremental).
+- SFTP upload failed: Job fails; state not advanced. Next run retries same window.
+- GraphQL pagination error: Ensure edges.cursor and pageInfo.hasNextPage are in the query.
+- Memory pressure: Lower pageSize or maxRecords; split output or stream to S3.
+## 🗺️ Mapping tips (compact)
+- Required fields: mark in mapping JSON; invalid records are skipped and reported.
+- Nested paths: use dot notation (e.g., catalogue.ref).
+- Arrays: map arrays directly; one record in, one mapped object out.
+- Resolvers: prefer built-ins (trim, uppercase, parseInt); keep output types aligned.
+---
+## 9. Replication Checklist
+**To replicate this template for other entities/formats:**
+1. **File Naming:** Replace `inventory-positions`, `IP`, `InventoryPosition` with your entity name across all files
+2. **GraphQL Query:** Update query constant and field selection to match your entity schema
+3. **Mapping Config:** Create new mapping file in `config/` with correct field paths
+4. **Workflows:** Rename workflow exports to match entity (e.g., `scheduledVirtualPositionsExtraction`)
+5. **Service Function:** Rename main function (e.g., `executeVirtualPositionExtraction`)
+6. **State Key:** Update KV key (e.g., `lastVirtualPositionSync`)
+7. **Output Format:** For CSV use `CSVParserService`, for JSON use `JSON.stringify()`, for XML use `XMLBuilder`
+8. **Upload Destination:** For S3 replace `SftpDataSource` with `S3DataSource`
+---
+### Pattern 8: Backward Pagination (Optional - Advanced)
+**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
+**When to Use**:
+- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
+- ✅ Time-bounded reverse traversal for auditing
+- ✅ Display newest-first in UI/reports
+- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
+**GraphQL Query Requirements**:
+Your query must support backward pagination by including `$last` and `$before`:
+```graphql
+query GetData(
+  $retailerId: ID!
+  $first: Int # For forward pagination
+  $after: String # For forward pagination
+  $last: Int # For backward pagination
+  $before: String # For backward pagination
+) {
+  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
+    edges {
+      cursor # ? REQUIRED
+      node {
+        id
+        createdAt
+        # ... other fields
+      }
+    }
+    pageInfo {
+      hasNextPage # For forward
+      hasPreviousPage # ? REQUIRED for backward
+    }
+  }
+}
+```
+**Implementation**:
+```typescript
+// Backward pagination - newest records first
+const result = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    retailerId,
+    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
+    //  Don't include last/before - orchestrator injects them
+  },
+  pageSize: 200,
+  direction: 'backward', // ? Enable reverse pagination
+  maxRecords: 10000,
+});
+// Records are returned in reverse chronological order
+console.log(result.data[0].createdAt); // Newest
+console.log(result.data[result.data.length - 1].createdAt); // Oldest (within range)
+```
+**Key Differences from Forward Pagination**:
+| Aspect                 | Forward (Default)                | Backward                |
+| ---------------------- | -------------------------------- | ----------------------- |
+| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
+| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
+| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
+| **Cursor Source**      | Last edge of page                | First edge of page      |
+| **Record Order**       | Oldest ? Newest               | Newest ? Oldest      |
+**Important Notes**:
+1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
+2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
+3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
+4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
+**Example: Extract Latest 1000 Orders**
+```typescript
+const latestOrders = await orchestrator.extract({
+  query: ORDERS_QUERY,
+  resultPath: 'orders.edges.node',
+  variables: {
+    retailerId,
+    statuses: ['BOOKED', 'ALLOCATED'],
+  },
+  direction: 'backward', // Start from newest
+  maxRecords: 1000, // Stop after 1000 records
+  pageSize: 100, // 100 per page = 10 pages
+});
+// latestOrders.data[0] is the newest order
+// latestOrders.data[999] is the 1000th newest order
+```
+**When to Use Forward vs Backward**:
+```typescript
+// ? Forward (default) - For incremental sync
+const incrementalData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    dateRangeFilter: { from: lastSyncTime, to: now },
+  },
+  // direction defaults to 'forward'
+  // Processes oldest ? newest for proper sequencing
+});
+// ? Backward - For "latest N records" use cases
+const latestData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  direction: 'backward',
+  maxRecords: 100, // Just get latest 100
+  // Gets newest ? oldest
+});
+```
+**Pagination Variables Reference**:
+| Variable | Forward      | Backward     | Injected By  | Notes                    |
+| -------- | ------------ | ------------ | ------------ | ------------------------ |
+| `first`  | ? Used      |  Not used | Orchestrator | From `pageSize`          |
+| `after`  | ? Used      |  Not used | Orchestrator | From cursor (last edge)  |
+| `last`   |  Not used | ? Used      | Orchestrator | From `pageSize`          |
+| `before` |  Not used | ? Used      | Orchestrator | From cursor (first edge) |
+**Common Mistakes to Avoid**:
+```typescript
+//  WRONG - Don't pass pagination variables
+const result = await orchestrator.extract({
+  variables: {
+    last: 200, //  Orchestrator will override this
+    before: cursor, //  Orchestrator manages cursor
+  },
+  direction: 'backward',
+});
+// ? CORRECT - Let orchestrator inject pagination
+const result = await orchestrator.extract({
+  variables: {
+    retailerId, // ? Your business variables only
+  },
+  pageSize: 200, // ? Orchestrator uses this for last/before
+  direction: 'backward',
+});
+```
+#### Optional: Reverse Pagination
+- Default remains forward pagination ($first/$after) using pageInfo.hasNextPage.
+- To paginate backward, define $last/$before and add pageInfo.hasPreviousPage; set direction='backward' in the orchestrator.
+Snippet:
+```graphql
+query GetInventoryPositionsBackward($last: Int!, $before: String) {
+  inventoryPositions(last: $last, before: $before) {
+    edges {
+      cursor
+      node {
+        id
+        ref
+        updatedOn
+      }
+    }
+    pageInfo {
+      hasPreviousPage
+    }
+  }
+}
+```
+SDK:
+```typescript
+await orchestrator.extract({
+  query: INVENTORY_POSITIONS_BACKWARD_QUERY,
+  resultPath: 'inventoryPositions.edges.node',
+  variables: { dateRangeFilter },
+  pageSize,
+  direction: 'backward',
+});
+```
+---
+## Testing Checklist
+**Before production deployment:**
+### 1. Schema Validation
+- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
+- [ ] Run `npx fc-connect validate-schema --mapping ./config/inventory-positions.export.xml.json --schema ./fluent-schema.json`
+- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/inventory-positions.export.xml.json --schema ./fluent-schema.json`
+- [ ] Verify all `source` paths in mapping exist in GraphQL schema
+- [ ] Verify query structure matches schema (fields, types, filters)
+### 2. Extraction Testing
+- [ ] Test with small dataset first (maxRecords=10)
+- [ ] Verify ExtractionOrchestrator pagination works correctly
+- [ ] Test with multiple pages of data (verify cursor handling)
+- [ ] Verify date range filtering (updatedOn filter)
+- [ ] Test empty result handling (no records in date range)
+- [ ] Verify extraction stops at maxRecords limit
+### 3. Mapping Testing
+- [ ] Verify required fields are populated
+- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
+- [ ] Test custom resolvers with edge cases (if any)
+- [ ] Verify nested field extraction
+- [ ] Test with null/missing fields
+- [ ] Verify mapping error collection works
+### 4. XML Generation Testing
+- [ ] Verify XML structure matches expected format
+- [ ] Test XML validation against XSD schema (if applicable)
+- [ ] Verify special character escaping in XML
+- [ ] Test with large datasets (>1000 records)
+- [ ] Verify UTF-8 encoding
+- [ ] Test XML namespace handling (if applicable)
+### 5. SFTP Upload Testing
+- [ ] Test SFTP connection and authentication
+- [ ] Verify file upload to correct path
+- [ ] Test file naming convention (timestamp format)
+- [ ] Verify file permissions on SFTP server
+- [ ] Test upload retry logic (simulate network failure)
+- [ ] Verify SFTP connection disposal (no connection leaks)
+### 6. State Management Testing
+- [ ] Verify overlap buffer prevents missed records (60-second default)
+- [ ] Test state recovery after extraction failure
+- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
+- [ ] Test first run with no previous state (uses fallbackStartDate)
+- [ ] Verify state update only happens on successful upload
+- [ ] Test manual date override (doesn't update state)
+### 7. Job Tracking Testing
+- [ ] Test job creation with JobTracker
+- [ ] Verify job status updates at each stage
+- [ ] Test job completion with metadata
+- [ ] Test job failure handling
+- [ ] Query job status via webhook endpoint
+- [ ] Verify job status persists in KV store
+### 8. Error Handling Testing
+- [ ] Test with invalid GraphQL query
+- [ ] Test with mapping errors (invalid field paths)
+- [ ] Test with SFTP connection failures
+- [ ] Test with authentication failures
+- [ ] Test with network timeouts
+- [ ] Verify error logging includes context (jobId, stage, error details)
+- [ ] Test error threshold logic (if applicable)
+### 9. Staging Environment Testing
+- [ ] Run full extraction in staging environment
+- [ ] Verify XML file format with downstream system
+- [ ] Monitor extraction duration and resource usage
+- [ ] Test with production-like data volumes
+- [ ] Verify no performance degradation over time
+### 10. Integration Testing
+- [ ] Test scheduled workflow (cron trigger)
+- [ ] Test ad hoc webhook trigger
+- [ ] Test job status query webhook
+- [ ] Verify activation variables are read correctly
+- [ ] Test with different extraction modes (incremental, date range)
+- [ ] End-to-end test: trigger ? extract ? transform ? upload ? verify file
+---
+## Monitoring & Alerting
+### Success Response Example
+```json
+{
+  "success": true,
+  "jobId": "SCHEDULED_IP_20251102_140000_abc123",
+  "recordsExtracted": 1523,
+  "fileName": "inventory-positions-2025-11-02T14-00-00-000Z.xml",
+  "sftpPath": "/outbound/inventory-positions/inventory-positions-2025-11-02T14-00-00-000Z.xml",
+  "metrics": {
+    "extractionDurationMs": 12543,
+    "totalPages": 8,
+    "pageSize": 200,
+    "mappingErrors": 0,
+    "fileSizeBytes": 524288,
+    "uploadDurationMs": 1234
+  },
+  "timestamps": {
+    "extractionStart": "2025-11-02T14:00:00.000Z",
+    "extractionEnd": "2025-11-02T14:00:12.543Z",
+    "uploadComplete": "2025-11-02T14:00:13.777Z"
+  },
+  "state": {
+    "previousTimestamp": "2025-11-02T13:00:00.000Z",
+    "newTimestamp": "2025-11-02T13:59:58.123Z",
+    "stateUpdated": true,
+    "overlapBufferSeconds": 60
+  }
+}
+```
+### Error Response Example
+```json
+{
+  "success": false,
+  "jobId": "ADHOC_IP_20251102_140500_xyz789",
+  "error": "SFTP upload failed: Connection timeout",
+  "errorCategory": "NETWORK",
+  "recordsExtracted": 0,
+  "stage": "sftp_upload",
+  "details": {
+    "message": "Failed to upload file after 3 retry attempts",
+    "retryAttempts": 3,
+    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
+  },
+  "state": {
+    "stateUpdated": false,
+    "willRetryNextRun": true,
+    "note": "State not advanced - next extraction will retry same time window"
+  }
+}
+```
+### Key Metrics to Track
+```typescript
+const METRICS = {
+  // Extraction Performance
+  extractionDurationMs: Date.now() - extractionStart,
+  recordCount: records.length,
+  pageCount: extractionResult.stats.totalPages,
+  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
+  // Transformation Performance
+  transformedCount: transformedRecords.length,
+  failedCount: mappingErrors.length,
+  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
+  // File Generation
+  fileSizeMB: (xmlContent.length / (1024 * 1024)).toFixed(2),
+  // Upload Performance
+  uploadDurationMs: uploadEnd - uploadStart,
+  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
+  // State Management
+  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
+  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
+};
+log.info('Extraction metrics', metrics);
+```
+### Alert Thresholds
+```typescript
+const ALERT_THRESHOLDS = {
+  // Duration Alerts
+  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
+  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
+  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
+  // Error Rate Alerts
+  MAX_ERROR_RATE: 0.05, // 5% mapping errors
+  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
+  // Volume Alerts
+  MAX_RECORDS_PER_RUN: 100000,
+  MIN_RECORDS_WARNING: 0, // Alert if no records found
+  MAX_FILE_SIZE_MB: 150, // 150MB
+  // State Alerts
+  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
+  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
+};
+// Check thresholds
+if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
+  log.warn('Extraction duration exceeded threshold', {
+    duration: metrics.extractionDurationMs,
+    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
+    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
+  });
+}
+```
+### Monitoring Dashboard Queries
+**Versori Platform Logs Query:**
+```
+# Successful extractions
+log_level:info AND message:"Extraction complete" AND jobId:*
+# Failed extractions
+log_level:error AND message:"Extraction workflow failed" AND jobId:*
+# Performance issues
+extractionDurationMs:>300000 OR uploadDurationMs:>120000
+# High error rates
+errorRate:>5
+# State management issues
+stateUpdated:false AND success:true
+```
+### Common Issues and Solutions
+**Issue**: "Extraction timeout after 10 minutes"
+- **Cause**: Too many records in single extraction
+- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
+- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
+**Issue**: "Mapping errors for 50% of records"
+- **Cause**: Schema mismatch between GraphQL response and mapping config
+- **Fix**: Run schema validation, update mapping config paths
+- **Prevention**: Use `npx fc-connect validate-schema` before deployment
+**Issue**: "SFTP connection timeout"
+- **Cause**: Network issues, firewall, or connection pool exhaustion
+- **Fix**: Check SFTP credentials, verify network connectivity
+- **Prevention**: Implement connection health checks, monitor connection status
+**Issue**: "State not updating after successful extraction"
+- **Cause**: KV write failure or intentional retry logic
+- **Fix**: Check KV logs, verify state update code executed
+- **Prevention**: Add KV write verification, log state updates explicitly
+**Issue**: "First run exceeds record limits"
+- **Cause**: No previous timestamp, fetches all historical records
+- **Fix**: Set fallbackStartDate close to current date, apply additional filters
+- **Prevention**: Use appropriate fallbackStartDate for initial runs
+**Issue**: "Excessive duplicate records in output"
+- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
+- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
+- **Prevention**: Monitor duplicate rates, verify state update logic
+---
+## Troubleshooting Quick Reference
+| Error Message | Likely Cause | Solution |
+|--------------|--------------|----------|
+| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
+| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
+| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
+| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
+| "SFTP authentication failed" | Invalid credentials | Verify SFTP credentials in activation variables |
+| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
+| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
+| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
+| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
+| "XML generation failed" | Format-specific error | Check XML generation logic, validate output |
+---