npm - @fluentcommerce/fc-connect-sdk - Versions diffs - 0.1.53 → 0.1.55 - Mend

@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (495) hide show

package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-fulfillments-to-sftp-xml.md CHANGED Viewed

@@ -1,2294 +1,2294 @@
----
-template_id: tpl-extract-fulfillments-to-sftp-xml
-canonical_filename: template-extraction-fulfillments-to-sftp-xml.md
-version: 2.0.0
-sdk_version: ^0.1.39
-runtime: versori
-direction: extraction
-source: fluent-graphql
-destination: sftp-xml
-entity: fulfillments
-format: xml
-logging: versori
-status: stable
-features:
-  - memory-management
-  - enhanced-logging
-  - pagination-progress
-  - dispose-finally
----
-# Template: Extraction - Fulfillments to SFTP XML
-**Template Version:** 2.0.0
-**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
-**Last Updated:** 2025-01-24
-**Deployment Target:** Versori Platform
-**🆕 Version 2.0.0 Enhancements:**
-- ✅ **Memory Management** - Clear large result sets after processing batches
-- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
-- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
-- ✅ **Resource Cleanup** - SFTP dispose in finally blocks prevents connection leaks
-## Installation
-```bash
-npm install @fluentcommerce/fc-connect-sdk@^0.1.39
-```
-Use version `^0.1.39` to ensure compatibility with this template.
----
-## 📚 STEP 1: Load These Docs (Human Checklist)
-1. REQUIRED (load all)
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-Copy-paste list (open these):
-fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
----
-## 📋 STEP 2: Tell Your AI (Prompt)
-Copy/paste this prompt after loading the documentation above:
-```
-Create a Versori scheduled extractor for fulfillments that uses ExtractionOrchestrator + JobTracker, incremental updatedOn with a 60s overlap buffer, transforms via UniversalMapper, generates XML with fast-xml-parser XMLBuilder, uploads to SFTP using SftpDataSource with dispose(). Include 3 workflows: scheduled, ad-hoc webhook, and job-status query with native Versori logging.
-```
----
-## 📦 SDK Imports (Verified - Versori Optimized)
-```typescript
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  XMLBuilder,
-  SftpDataSource,
-  VersoriKVAdapter,
-} from '@fluentcommerce/fc-connect-sdk';
-import { schedule, webhook, http, fn } from '@versori/run';
-```
----
-# Versori Scheduled: Fulfillments Extraction to SFTP XML (Shipment Tracking)
-**FC Connect SDK Use Case Guide**
-> SDK: [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
-> Version: Use ^0.1.39 - `npm install @fluentcommerce/fc-connect-sdk@^0.1.39`
-Context: Scheduled Versori workflow that extracts fulfillment data (shipment tracking, delivery information) from Fluent Commerce via GraphQL query with **ExtractionOrchestrator**, **JobTracker**, and **incremental timestamp tracking**, transforms with `UniversalMapper`, and writes **XML files** to SFTP for logistics coordination and delivery tracking.
-**Pattern**: EXTRACTION (Fluent → SFTP XML)
-**Complexity**: High | Runtime: Versori Platform (Scheduled)
----
-## ⚠️ IMPORTANT: Production-Ready Base Template
-> **📋 BASE TEMPLATE - Ready for Production (Customize for Your Needs)**
->
-> This is a **production-ready base template** demonstrating FC Connect SDK best practices for fulfillment extraction workflows with XML output.
->
-> **✅ INCLUDED FEATURES:**
->
-> - ✅ Comprehensive error handling with retry logic
-> - ✅ SFTP upload with exponential backoff (3 attempts)
-> - ✅ State management with overlap buffer (prevents missed records)
-> - ✅ Job tracking with lifecycle management
-> - ✅ Security (credential masking in logs)
-> - ✅ UTC time enforcement (prevents timezone bugs)
-> - ✅ Incremental extraction (safe, efficient, production-ready)
-> - ✅ Natural rate limiting via timestamps
->
-> **📝 BEFORE DEPLOYING:**
->
-> 1. Review and customize activation variables for your environment
-> 2. Test with sample data in your Versori workspace
-> 3. Adjust safety limits (pageSize, maxRecords) if needed
-> 4. Configure monitoring alerts for extraction failures
-> 5. Verify SFTP credentials and paths
->
-> **This base template follows SDK best practices - tweak specific to your needs.**
----
-## 🚀 Quick Deploy to Versori
-1. **Create Integration:** Versori Dashboard → New Integration → Name it "Fulfillments Extraction"
-2. **Copy Code:** Copy all workflow code from this template into Versori editor
-3. **Configure Connections:**
-   - `fluent_commerce` - Fluent Commerce API (OAuth2)
-   - `sftp_export_server` - SFTP destination (Basic Auth)
-   - `fulfillments-adhoc` - Ad-hoc webhook auth (API Key)
-   - `fulfillments-job-status` - Status webhook auth (API Key)
-4. **Set Activation Variables:** Configure all variables listed in Activation Variables section
-5. **Test:** Use ad-hoc webhook to test extraction before enabling schedule
-6. **Enable Schedule:** Activate scheduled workflow after successful test
-**Time to Deploy:** ~15 minutes
----
-## 🔧 Common Customizations
-**Most Common Changes:**
-| What to Change         | Where                                     | Why                                |
-| ---------------------- | ----------------------------------------- | ---------------------------------- |
-| **Status filters**     | Activation variable `fulfillmentStatuses` | Target specific fulfillment states |
-| **Schedule frequency** | Workflow: `'0 */4 * * *'`                 | Change from 4-hourly to your needs |
-| **XML field mapping**  | `config/fulfillments.export.xml.json`     | Add/remove/modify output fields    |
-| **Date filters**       | GraphQL query variables                   | Filter by creation/update dates    |
-| **Page size**          | Activation variable `pageSize`            | Tune for performance (50-1000)     |
-| **Safety limits**      | Activation variable `maxRecords`          | Adjust based on dataset size       |
-**Less Common Changes:**
-| What to Change          | Where                                      | Default                 | When to Change               |
-| ----------------------- | ------------------------------------------ | ----------------------- | ---------------------------- |
-| **Overlap buffer**      | Activation variable `overlapBufferSeconds` | 60s                     | If seeing missed records     |
-| **SFTP retry attempts** | Code: `uploadWithRetry(..., 3)`            | 3                       | For unreliable networks      |
-| **Fallback start date** | Activation variable `fallbackStartDate`    | 2024-01-01              | For historical extraction    |
-| **Remote path**         | Activation variable `sftpRemotePath`       | /incoming/fulfillments/ | Match partner's requirements |
-**What NOT to Change (Unless You Know Why):**
-- 10-step workflow structure
-- Error taxonomy (CONFIG/API/SFTP/NO_DATA)
-- State management logic
-- UTC time enforcement
-- Connection cleanup (`sftp.dispose()`)
----
-## What You'll Build
-- **Incremental extraction** using `updatedOn >= (lastRunTime - buffer)` with **overlap buffer**
-- **ExtractionOrchestrator** for auto-pagination and path-based extraction
-- **JobTracker** for lifecycle management
-- **State management** with VersoriKV to track last successful run
-- **Safety buffer** (60 seconds) to handle clock skew and race conditions
-- GraphQL query with status-based filtering and date range support
-- UniversalMapper transformation with nested order data
-- **XML file generation** with fulfillment/shipment data
-- **SFTP upload** to partner systems with proper connection cleanup
-- **3 workflow patterns**: scheduled, ad-hoc webhook, job status query
-- **Failure recovery** with timestamp tracking
-## Business Use Case
-**4-hourly shipment tracking extraction for logistics:**
-- Extract shipment and delivery information for carrier integration
-- Track fulfillment status changes (SHIPPED, DELIVERED)
-- Monitor delivery exceptions and returns (RMA)
-- Support logistics partner feeds
-- Enable shipment tracking updates
-- Run every 4 hours for timely status updates
-- Deliver XML format to enterprise systems via SFTP
-## Fulfillments Explained
-**Fulfillment** = Shipment execution record
-- Represents order fulfillment process
-- Tracks shipment status and delivery progress
-- Links to orders, items, and locations
-- Used for: Shipment tracking, delivery coordination, logistics integration
-**Key Fields:**
-- `ref` - Unique fulfillment reference
-- `status` - Fulfillment state (SHIPPED, DELIVERED, etc.)
-- `deliveryType` - Shipping method
-- `order.ref` - Associated order reference
-- `fromLocation` - Shipping origin
-- `items` - Fulfilled order items
-- `createdOn`, `updatedOn` - Timestamps for tracking
-## SDK Methods Used
-```typescript
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  XMLBuilder,
-  SftpDataSource,
-  VersoriKVAdapter,
-} from '@fluentcommerce/fc-connect-sdk';
-await createClient(ctx); // Versori-aware client
-const orchestrator = new ExtractionOrchestrator(client, log); // Auto-pagination
-const tracker = new JobTracker(kv, log); // Job lifecycle tracking
-await orchestrator.extract({ query, resultPath, variables, pageSize, maxRecords }); // Extract
-new VersoriKVAdapter(ctx.openKv(':project:')); // State management
-new UniversalMapper(exportMapping); // Field transformation
-new XMLBuilder({ prettyPrint: true, xmlDeclaration: true }); // XML generation
-const xml = xmlBuilder.build(data, 'RootElement'); // Build XML string
-await sftp.uploadFile(remotePath, buffer, options); // SFTP upload
-await sftp.dispose(); // CRITICAL: Connection cleanup
-```
-## Activation Variables
-**Configuration is driven by activation variables - modify these instead of code:**
-```json
-{
-  "fulfillmentStatuses": "SHIPPED,DELIVERED",
-  "sftpHost": "sftp.partner.com",
-  "sftpPort": 22,
-  "sftpPrivateKey": "",
-  "sftpRemotePath": "/incoming/fulfillments/",
-  "fileNamePrefix": "fulfillments",
-  "requireAbsolutePaths": "false",
-  "pageSize": 200,
-  "maxRecords": 50000,
-  "fallbackStartDate": "2024-01-01T00:00:00Z",
-  "overlapBufferSeconds": "60"
-}
-```
-> **Note:** `sftpUsername` and `sftpPassword` are fetched from the `sftp_export_server` connection (see SFTP Credential Management section below).
-## SFTP Credential Management
-Fetch credentials securely from Versori connection vault using `ctx.credentials().getAccessToken()`:
-```typescript
-import { Buffer } from 'node:buffer'; // Required for Versori/Deno
-// ========================================
-// SFTP CREDENTIAL RETRIEVAL
-// ========================================
-/**
- * Retrieve SFTP credentials from connection configuration
- *
- * This approach retrieves credentials stored in the Versori connection settings.
- * The connection must be configured in the UI with Basic Authentication.
- *
- * Steps:
- * 1. Call ctx.credentials().getAccessToken('SFTP') to get base64-encoded credentials
- * 2. Decode the accessToken from base64 to get "username:password" string
- * 3. Split on ':' to extract username and password
- *
- * Connection Name: 'sftp_export_server' (must match the connection name in Versori UI)
- * Auth Type: Basic Authentication (username + password)
- *
- * This method provides:
- * - Centralized credential management through Versori UI
- * - Better security (credentials not stored in integration variables)
- * - Easier credential rotation and updates
- */
-log.info('Retrieving SFTP credentials from connection configuration');
-let sftpUsername: string;
-let sftpPassword: string;
-try {
-  // Retrieve credentials from the 'sftp_export_server' connection
-  const sftpCred = await ctx.credentials().getAccessToken('sftp_export_server');
-  if (!sftpCred?.accessToken) {
-    throw new Error('No SFTP credentials found in connection configuration');
-  }
-  // Decode base64 accessToken to get "username:password"
-  const rawBasicAuth = Buffer.from(sftpCred.accessToken, 'base64').toString('utf-8');
-  // Split on ':' to extract username and password
-  const parts = rawBasicAuth.split(':');
-  if (parts.length !== 2) {
-    throw new Error('Invalid SFTP credential format - expected username:password');
-  }
-  sftpUsername = parts[0];
-  sftpPassword = parts[1];
-  log.info('SFTP credentials retrieved successfully', {
-    hasUsername: !!sftpUsername,
-    hasPassword: !!sftpPassword,
-    usernameLength: sftpUsername.length,
-    passwordLength: sftpPassword.length,
-  });
-} catch (error: any) {
-  log.error('Failed to retrieve SFTP credentials', {
-    message: error instanceof Error ? error.message : String(error),
-    stack: error instanceof Error ? error.stack : undefined,
-    errorType: error instanceof Error ? error.constructor.name : 'Error',
-  });
-  return {
-    success: false,
-    error: 'Failed to retrieve SFTP credentials from connection configuration',
-    details: error instanceof Error ? error.message : String(error),
-    recommendation: 'Please ensure the SFTP connection is configured in the Connections section with Basic Authentication (username and password)',
-  };
-}
-// Initialize SFTP with connection credentials
-const sftp = new SftpDataSource(
-  {
-    type: 'SFTP_XML',
-    connectionId: 'sftp_export_server',
-    name: 'Export SFTP',
-    settings: {
-      host: ctx.activation.getVariable('sftpHost'),
-      port: parseInt(ctx.activation.getVariable('sftpPort') || '22'),
-      username: sftpUsername,
-      password: sftpPassword,
-      remotePath: ctx.activation.getVariable('sftpRemotePath') || '/exports',
-      encoding: 'utf8',
-    },
-  },
-  log
-);
-```
-**Benefits:**
-- ✅ Credentials stored securely in Versori vault
-- ✅ Centralized management - update in one place
-- ✅ Access control enforced by platform
-- ✅ No credentials in activation variables
-- ✅ Explicit error handling with validation
-### Setup Steps
-1. In Versori Dashboard → Connections → Add Connection
-2. Name: `sftp_export_server`
-3. Type: Basic Auth
-4. Enter SFTP username and password
-5. Save connection
-6. Reference in code: `ctx.credentials().getAccessToken('sftp_export_server')`
----
-## Webhook Security Configuration
-**⚠️ IMPORTANT:** Webhook security is now handled at the **Versori connection level** (not in code).
-### Required Connections
-Create these connections in Versori Dashboard before deploying:
-1. **`fulfillments-adhoc`** (Ad-hoc webhook auth)
-   - Configure authentication (e.g., API Key) in the connection settings
-2. **`fulfillments-job-status`** (Job status query auth)
-   - Configure authentication (e.g., API Key) in the connection settings
-### How to Create Webhook Connections
-1. **Navigate to Versori Dashboard** → Connections
-2. **Click "Add Connection"**
-3. **Configure:**
-   - Name: `fulfillments-adhoc`
-   - Authentication: API Key (or preferred method)
-   - Provide header name/value in the connection UI
-4. **Save Connection**
-5. **Repeat** for `fulfillments-job-status` connection
-### Generate Secure API Keys
-```bash
-# Node.js - Generate cryptographically secure 64-character key
-node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
-# Example output:
-# 7f3e9a2b1c8d6e4f5a0b9c8d7e6f5a4b3c2d1e0f9a8b7c6d5e4f3a2b1c0d9e8f
-```
-### How It Works
-```
-┌────────────────┐
-│  External       │
-│  System         │
-└──────┬─────────┘
-       │
-       │ POST /webhook/fulfillments-adhoc
-       │ Headers: (as configured on the connection)
-       ▼
- ┌─────────────────────────┐
- │  Versori Platform        │
- │  ✓ Validates API key     │  ← Authentication happens HERE
- │  ✓ Checks header         │
- │  ✗ Rejects if invalid    │
- └──────┬──────────────────┘
-        │
-        │ ✅ Only reaches workflow if valid
-        ▼
- ┌─────────────────────────┐
- │  Your Workflow Code      │
- │  (No auth code needed!)  │
- └──────────────────────────┘
-```
-**Benefits:**
-- ✅ Security at platform level (before code runs)
-- ✅ No manual validation code needed
-- ✅ Consistent across all webhooks
-- ✅ Easy to rotate keys (update connection, not code)
-- ✅ Failed auth never hits your workflow (saves resources)
-## Export Mapping Configuration
-Create file: `config/fulfillments.export.xml.json`
-```json
-{
-  "name": "fulfillments.export.xml",
-  "version": "1.0.0",
-  "description": "Fluent Fulfillments → SFTP XML Export (shipment tracking)",
-  "fields": {
-    "fulfillment_ref": { "source": "ref", "required": true, "resolver": "sdk.trim" },
-    "order_ref": { "source": "order.ref", "required": true, "resolver": "sdk.trim" },
-    "status": { "source": "status", "required": true, "resolver": "sdk.uppercase" },
-    "delivery_type": { "source": "deliveryType", "required": false, "resolver": "sdk.trim" },
-    "from_location_ref": {
-      "source": "fromLocation.ref",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "from_location_name": {
-      "source": "fromLocation.name",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "created_on": { "source": "createdOn", "required": true, "resolver": "sdk.toString" },
-    "updated_on": { "source": "updatedOn", "required": true, "resolver": "sdk.toString" }
-  }
-}
-```
-**Note**: The mapping uses nested paths like `order.ref` and `fromLocation.ref` to access related entity fields.
-## Mapping & Resolvers Explained
-This section explains how the SDK transforms raw GraphQL data into your XML export format using **UniversalMapper** and **SDK resolvers**.
-### SDK Resolvers Used
-| Field                | Resolver        | Why?                         | Example Transformation                  |
-| -------------------- | --------------- | ---------------------------- | --------------------------------------- |
-| `fulfillment_ref`    | `sdk.trim`      | Clean fulfillment references | `" FF-001 "` → `"FF-001"`               |
-| `order_ref`          | `sdk.trim`      | Clean order references       | `" ORD-001 "` → `"ORD-001"`             |
-| `status`             | `sdk.uppercase` | Normalize status codes       | `"shipped"` → `"SHIPPED"`               |
-| `delivery_type`      | `sdk.trim`      | Clean delivery method        | `" STANDARD "` → `"STANDARD"`           |
-| `from_location_ref`  | `sdk.trim`      | Clean location references    | `" LOC-001 "` → `"LOC-001"`             |
-| `from_location_name` | `sdk.trim`      | Clean location names         | `" NYC Warehouse "` → `"NYC Warehouse"` |
-| `created_on`         | `sdk.toString`  | Preserve ISO 8601 timestamp  | `"2025-01-15T10:00:00.000Z"` → same     |
-| `updated_on`         | `sdk.toString`  | Preserve ISO 8601 timestamp  | `"2025-01-22T08:30:00.000Z"` → same     |
-### Transformation Flow
-```typescript
-// 1. GraphQL Response (raw data from Fluent Commerce)
-const rawFulfillment = {
-  ref: ' FF-001 ',
-  status: 'shipped',
-  deliveryType: ' STANDARD ',
-  order: {
-    ref: ' ORD-001 ',
-  },
-  fromLocation: {
-    ref: ' LOC-001 ',
-    name: ' NYC Warehouse ',
-  },
-  items: {
-    edges: [
-      {
-        node: {
-          productRef: 'PROD-001',
-          requestedQuantity: 2,
-          filledQuantity: 2,
-        },
-      },
-    ],
-  },
-  createdOn: '2025-01-15T10:00:00.000Z',
-  updatedOn: '2025-01-22T08:30:00.000Z',
-};
-// 2. UniversalMapper applies SDK resolvers (handles nested paths automatically)
-const mapper = new UniversalMapper(fulfillmentsExportMapping);
-const result = await mapper.map(rawFulfillment);
-// 3. Transformed Output (clean, normalized for XML)
-const transformedFulfillment = {
-  fulfillment_ref: 'FF-001',
-  order_ref: 'ORD-001',
-  status: 'SHIPPED',
-  delivery_type: 'STANDARD',
-  from_location_ref: 'LOC-001',
-  from_location_name: 'NYC Warehouse',
-  created_on: '2025-01-15T10:00:00.000Z',
-  updated_on: '2025-01-22T08:30:00.000Z',
-};
-```
-### Nested Object Mapping
-Fulfillments include **nested objects** that require special path syntax:
-```typescript
-// ✅ CORRECT - Access nested object fields with dot notation
-{
-  "order_ref": { "source": "order.ref" },
-  "from_location_ref": { "source": "fromLocation.ref" },
-  "from_location_name": { "source": "fromLocation.name" }
-}
-```
-**GraphQL must fetch the nested objects:**
-```graphql
-node {
-  ref                    # Direct field
-  status
-  order {                # Nested Order object
-    ref                  # Access via order.ref
-  }
-  fromLocation {         # Nested Location object
-    ref                  # Access via fromLocation.ref
-    name                 # Access via fromLocation.name
-  }
-  items {                # Nested items collection
-    edges {
-      node {
-        productRef
-        requestedQuantity
-        filledQuantity
-      }
-    }
-  }
-}
-```
-**UniversalMapper handles nested paths automatically** - just use dot notation in the mapping configuration.
-### Available SDK Resolvers
-The SDK provides these built-in resolvers (no custom code needed):
-**String Transformations:**
-- `sdk.trim` - Remove leading/trailing whitespace
-- `sdk.uppercase` - Convert to uppercase
-- `sdk.lowercase` - Convert to lowercase
-- `sdk.toString` - Convert to string
-**Number Parsing:**
-- `sdk.parseInt` - Parse as integer
-- `sdk.parseFloat` - Parse as decimal
-- `sdk.number` - Parse as number (auto-detect int/float)
-**Date Formatting:**
-- `sdk.formatDate` - ISO 8601 date only (YYYY-MM-DD)
-- `sdk.formatDateShort` - Short date format
-- `sdk.parseDate` - Parse various date formats
-**Type Conversions:**
-- `sdk.boolean` - Convert to boolean
-- `sdk.parseJson` - Parse JSON strings
-- `sdk.toJson` - Convert to JSON string
-**Utilities:**
-- `sdk.identity` - Return value unchanged
-- `sdk.coalesce` - Return first non-null value
-See [Universal Mapping Guide](../../../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) for complete resolver documentation.
-## GraphQL Query
-**Note**: This query is verified against Fluent Commerce schema introspection.
-```graphql
-query GetFulfillments(
-  $statuses: [String!]
-  $updatedFrom: DateTime
-  $updatedTo: DateTime
-  $first: Int!
-  $after: String
-) {
-  fulfillments(
-    status: $statuses
-    updatedOn: { from: $updatedFrom, to: $updatedTo }
-    first: $first
-    after: $after
-  ) {
-    edges {
-      node {
-        id
-        ref
-        status
-        deliveryType
-        order {
-          ref
-        }
-        fromLocation {
-          ref
-          name
-        }
-        items {
-          edges {
-            node {
-              productRef
-              requestedQuantity
-              filledQuantity
-            }
-          }
-        }
-        createdOn
-        updatedOn
-      }
-      cursor
-    }
-    pageInfo {
-      hasNextPage
-    }
-  }
-}
-```
----
-## Versori Workflows Structure
-**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
-**Trigger Types:**
-- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
-- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
-- **`workflow()`** → Durable workflows (advanced, rarely used)
-**Execution Steps (chained to triggers):**
-- **`http()`** → External API calls (chained from schedule/webhook)
-- **`fn()`** → Internal processing (chained from schedule/webhook)
-### Recommended Project Structure
-**This template demonstrates a complete workflow implementation in a single file.** In production, consider organizing your code into this modular structure:
-```
-fulfillments-extraction/
-├── index.ts                              # Entry point - exports all workflows
-└── src/
-    ├── workflows/
-    │   └── fulfillments-extraction.ts            # All workflows (scheduled + webhooks)
-    │
-    ├── services/
-    │   └── extraction-orchestration.ts           # Shared orchestration logic (reusable)
-    │
-    └── config/
-        ├── queries.ts                            # GraphQL queries
-        └── fulfillments.export.xml.json          # Mapping configuration
-```
-**Alternative Modular Structure** (for larger projects with multiple workflows):
-```
-fulfillments-extraction/
-├── index.ts
-└── src/
-    ├── workflows/
-    │   ├── scheduled/
-    │   │   └── daily-fulfillments-extraction.ts  # Scheduled extraction only
-    │   └── webhook/
-    │       ├── adhoc-fulfillments-extraction.ts  # Ad-hoc webhook
-    │       └── job-status-check.ts               # Status query webhook
-    ├── services/
-    │   └── extraction-orchestration.ts
-    └── config/
-        ├── queries.ts
-        └── fulfillments.export.xml.json
-```
----
-## Expected XML Output
-```xml
-<?xml version="1.0" encoding="UTF-8"?>
-<Fulfillments>
-  <Fulfillment>
-    <fulfillment_ref>FF-001</fulfillment_ref>
-    <order_ref>ORD-001</order_ref>
-    <status>SHIPPED</status>
-    <delivery_type>STANDARD</delivery_type>
-    <from_location_ref>LOC-001</from_location_ref>
-    <from_location_name>NYC Warehouse</from_location_name>
-    <created_on>2025-01-15T10:00:00Z</created_on>
-    <updated_on>2025-01-22T08:30:00Z</updated_on>
-  </Fulfillment>
-  <Fulfillment>
-    <fulfillment_ref>FF-002</fulfillment_ref>
-    <order_ref>ORD-002</order_ref>
-    <status>DELIVERED</status>
-    <delivery_type>EXPRESS</delivery_type>
-    <from_location_ref>LOC-002</from_location_ref>
-    <from_location_name>LA Warehouse</from_location_name>
-    <created_on>2025-01-16T11:00:00Z</created_on>
-    <updated_on>2025-01-22T09:15:00Z</updated_on>
-  </Fulfillment>
-</Fulfillments>
-```
-## Production Safety & Guardrails
-### Overview
-Fulfillments require safeguards even in incremental mode:
-- **Status changes**: Fulfillments can update frequently during processing
-- **Delivery tracking**: Real-time status updates create rapid data changes
-- **Order volume**: High-volume retailers process thousands of fulfillments daily
-- **Partner feeds**: Logistics systems need predictable file sizes
-### Hard Limits
-```typescript
-const SAFETY_LIMITS = {
-  MAX_RECORDS_PER_RUN: 100000, // 100k fulfillments per run
-  MAX_RECORDS_PER_FILE: 25000, // 25k per XML file
-  MAX_FILE_SIZE_MB: 75, // 75MB per file
-  MAX_XML_SIZE_MB: 150, // Total extraction size
-  CHUNK_SIZE: 5000, // Process in chunks
-  ESTIMATED_BYTES_PER_RECORD: 600, // Conservative estimate for XML
-};
-```
-**Why these limits?**
-- Fulfillments have nested items (larger than simple entities)
-- Shipment tracking needs fast processing for timely updates
-- Multiple status changes can accumulate quickly
-- Partner systems require predictable batch sizes
-- XML is more verbose than CSV (~25% larger)
-### Runtime Validation Function
-```typescript
-/**
- * Validate extraction safety limits before processing
- */
-function validateExtractionLimits(recordCount: number, estimatedSizeMB: number) {
-  const MAX_RECORDS_PER_RUN = 100000;
-  const MAX_XML_SIZE_MB = 150;
-  if (recordCount > MAX_RECORDS_PER_RUN) {
-    return {
-      valid: false,
-      error: `Extraction limit exceeded: ${recordCount} records (max: ${MAX_RECORDS_PER_RUN})`,
-      recommendation: `Fulfillments dataset too large. Consider:
-        1. Increase extraction frequency (4-hourly → hourly)
-        2. Filter by status (extract SHIPPED and DELIVERED separately)
-        3. Filter by date range (last 24 hours only)
-        4. Split by location (extract by fulfillment center)`,
-      recordCount,
-      maxAllowed: MAX_RECORDS_PER_RUN,
-    };
-  }
-  if (estimatedSizeMB > MAX_XML_SIZE_MB) {
-    return {
-      valid: false,
-      error: `XML size limit exceeded: ${estimatedSizeMB}MB (max: ${MAX_XML_SIZE_MB}MB)`,
-      recommendation: 'File splitting required. Filter by status or date range.',
-      estimatedSizeMB,
-      maxAllowed: MAX_XML_SIZE_MB,
-    };
-  }
-  return { valid: true };
-}
-```
----
-## Production Capabilities
-Built-in safeguards and best practices:
-| Capability              | Implementation                             | Benefit                         |
-| ----------------------- | ------------------------------------------ | ------------------------------- |
-| **Config Validation**   | Fail-fast with detailed error messages     | Catches issues before execution |
-| **SFTP Retry**          | 3 attempts with exponential backoff        | Resilient to network issues     |
-| **UTC Enforcement**     | Normalize all dates to UTC                 | Prevents timezone bugs          |
-| **Window Persistence**  | Save effective time window in job metadata | Complete execution history      |
-| **Error Taxonomy**      | Categorize errors: CONFIG/API/SFTP/NO_DATA | Better support triage           |
-| **Extended Metrics**    | Track pages, skipped records, XML size     | Enhanced observability          |
-| **Input Normalization** | Lowercase, trim, dedupe list inputs        | Consistent filtering            |
-| **Secrets Masking**     | Mask credentials in error logs             | Security compliance             |
----
-## Complete Workflow Code
-### File Structure
-**This template demonstrates a single-file implementation** (all workflows in one file for easier understanding). The actual code structure matches the "Recommended Project Structure" shown earlier.
-```
-fulfillments-extraction/
-├── src/
-│   ├── index.ts                              # Entry point (exports workflows)
-│   ├── workflows/
-│   │   └── fulfillments-extraction.ts        # All 3 workflows (scheduled + 2 webhooks)
-│   ├── services/
-│   │   └── extraction-orchestration.ts       # Main orchestration logic (shared)
-│   └── config/
-│       └── queries.ts                        # GraphQL queries & mapping
-├── config/
-│   └── fulfillments.export.xml.json          # Field mapping config
-└── package.json
-```
-**Why Single File?**
-- ✅ Easier to understand workflow relationships
-- ✅ All patterns visible in one place
-- ✅ Simpler to maintain for small projects
-- ⚠️ For larger projects, split into `scheduled/` and `webhook/` subdirectories (see "Alternative Modular Structure" above)
-**Entry Point (`src/index.ts`):**
-```typescript
-// Export all workflows for Versori to discover
-export {
-  scheduledFulfillmentsExtraction,
-  adhocFulfillmentsExtraction,
-  fulfillmentsJobStatus,
-} from './workflows/fulfillments-extraction';
-// Export services for testing or direct use
-export {
-  executeFulfillmentExtraction,
-  queryJobStatus,
-  generateJobId,
-} from './services/extraction-orchestration';
-// Export configuration for reference
-export { FULFILLMENTS_QUERY, FULFILLMENTS_EXPORT_MAPPING } from './config/queries';
-```
-**How Versori Discovers Workflows:**
-Versori scans exported workflow objects (created with `schedule()`, `webhook()`) from your entry point and automatically registers them.
----
-## 📄 src/workflows/fulfillments-extraction.ts
-```typescript
-// ============================================================================
-// FULFILLMENTS EXTRACTION WORKFLOWS
-// ============================================================================
-// Purpose: Extract shipment tracking and delivery data from Fluent Commerce
-//          and export as XML to SFTP for logistics coordination
-//
-// Workflows:
-//   1. Scheduled: 4-hourly incremental extraction with state tracking
-//   2. Ad-hoc: Manual trigger with custom date ranges
-//   3. Job Status: Query extraction job progress and results
-// ============================================================================
-import { schedule, webhook, http, fn } from '@versori/run';
-import type { Context } from '@versori/run';
-import {
-  executeFulfillmentExtraction,
-  queryJobStatus,
-  generateJobId,
-} from '../services/extraction-orchestration';
-// ============================================================================
-// SECTION 1: TYPE DEFINITIONS
-// ============================================================================
-/**
- * Error categories for classification and triage
- */
-type ErrorCategory = 'CONFIG' | 'API' | 'SFTP' | 'NO_DATA';
-/**
- * Extraction time window with buffer tracking
- */
-interface TimeWindow {
-  from: string; // Original start time (ISO 8601 UTC)
-  to: string; // End time (ISO 8601 UTC)
-  bufferedFrom: string; // Actual query start with overlap buffer applied
-}
-/**
- * Detailed extraction metrics for observability
- */
-interface ExtractionMetrics {
-  totalPages: number; // GraphQL pages retrieved
-  validRecords: number; // Successfully transformed records
-  skippedRecords: number; // Records that failed mapping
-  xmlSizeKb: number; // Final XML file size
-  window: TimeWindow; // Time window used for extraction
-}
-/**
- * Result structure returned by extraction workflows
- */
-interface ExtractionResult {
-  success: boolean;
-  jobId: string;
-  recordCount?: number;
-  fileName?: string;
-  message?: string;
-  error?: string;
-  errorCategory?: ErrorCategory;
-  metrics?: ExtractionMetrics;
-}
-/**
- * Webhook payload for ad-hoc extraction
- */
-interface AdhocExtractionPayload {
-  fromDate?: string; // ISO 8601 UTC timestamp - extraction start
-  toDate?: string; // ISO 8601 UTC timestamp - extraction end
-  updateState?: boolean; // Whether to update last run timestamp
-}
-/**
- * Webhook payload for job status query
- */
-interface JobStatusPayload {
-  jobId: string; // Job ID to query
-}
-// ============================================================================
-// SECTION 2: WORKFLOW EXPORTS
-// ============================================================================
-/**
- * WORKFLOW 1: Scheduled Extraction
- *
- * Runs: Every 4 hours (0 */4 * * *)
- * Purpose: Incremental extraction of updated fulfillments
- * State: Updates lastRunTime after successful extraction
- *
- * Behavior:
- * - Loads last run timestamp from KV storage
- * - Applies 60-second overlap buffer for clock skew
- * - Extracts records with updatedOn >= (lastRunTime - buffer)
- * - Generates XML and uploads to SFTP with retry
- * - Updates state with current timestamp
- */
-export const scheduledFulfillmentsExtraction = schedule(
-  'fulfillments-extract-xml-4hourly',
-  '0 */4 * * *' // Every 4 hours at minute 0
-).then(
-  http('execute-scheduled-extraction', { connection: 'fluent_commerce' }, handleScheduledExtraction)
-);
-/**
- * WORKFLOW 2: Ad-hoc Extraction
- *
- * Trigger: Webhook POST to /webhooks/fulfillments-adhoc
- * Purpose: On-demand extraction with custom date ranges
- * State: Optional - only updates if updateState: true in payload
- *
- * Payload Examples:
- *
- * 1. Extract last 7 days (no state update):
- *    {
- *      "fromDate": "2025-01-22T00:00:00Z",
- *      "toDate": "2025-01-29T00:00:00Z"
- *    }
- *
- * 2. Full re-extraction with state reset:
- *    {
- *      "fromDate": "2024-01-01T00:00:00Z",
- *      "updateState": true
- *    }
- *
- * Note: fromDate and toDate MUST be ISO 8601 UTC timestamps
- */
-export const adhocFulfillmentsExtraction = webhook('fulfillments-adhoc', {
-  connection: 'fulfillments-adhoc', // Security via Versori connection
-  response: { mode: 'sync' }, // ✅ Sync mode: response sent when handler returns
-}).then(http('execute-adhoc-extraction', { connection: 'fluent_commerce' }, handleAdhocExtraction));
-/**
- * WORKFLOW 3: Job Status Query
- *
- * Trigger: Webhook GET/POST to /webhooks/fulfillments-job-status
- * Purpose: Check extraction job progress and results
- *
- * Query Payload:
- * {
- *   "jobId": "SCHEDULED_FULFILLMENTS_20250129_120000_abc123"
- * }
- *
- * Response:
- * {
- *   "success": true,
- *   "jobId": "...",
- *   "status": "completed",
- *   "stage": "upload_complete",
- *   "recordCount": 1250,
- *   "fileName": "fulfillments_20250129_120000.xml",
- *   "metrics": { ... }
- * }
- */
-export const fulfillmentsJobStatus = webhook('fulfillments-job-status', {
-  connection: 'fulfillments-job-status', // Security via Versori connection
-  response: { mode: 'sync' },
-}).then(fn('query-job-status', handleJobStatusQuery));
-// ============================================================================
-// SECTION 3: WORKFLOW HANDLERS
-// ============================================================================
-/**
- * Handle scheduled extraction workflow
- *
- * Always updates state for next incremental run
- */
-async function handleScheduledExtraction(ctx: Context): Promise<ExtractionResult> {
-  const { log } = ctx;
-  const jobId = generateJobId('SCHEDULED', 'FULFILLMENTS');
-  log.info('Scheduled extraction started', { jobId });
-  try {
-    const result = await executeFulfillmentExtraction(ctx, {
-      jobId,
-      triggeredBy: 'schedule',
-      updateState: true, // Always update state for scheduled runs
-    });
-    log.info('Scheduled extraction completed', {
-      jobId,
-      success: result.success,
-      recordCount: result.recordCount,
-      errorCategory: result.errorCategory,
-    });
-    return result;
-  } catch (error: any) {
-    log.error('Scheduled extraction failed', {
-      jobId,
-      message: error instanceof Error ? error.message : String(error),
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'Error',
-    });
-    throw error;
-  }
-}
-/**
- * Handle ad-hoc extraction workflow
- *
- * Allows custom date ranges and optional state updates
- */
-async function handleAdhocExtraction(ctx: Context): Promise<{ success: boolean; jobId: string; message: string; statusEndpoint: string; note: string }> {
-  const { data, log } = ctx;
-  const payload = data as AdhocExtractionPayload;
-  const jobId = generateJobId('ADHOC', 'FULFILLMENTS');
-  log.info('🚀 [WEBHOOK] Adhoc fulfillments extraction triggered', {
-    jobId,
-    fromDate: payload.fromDate || 'lastRunTime',
-    toDate: payload.toDate || 'now',
-    updateState: payload.updateState || false,
-  });
-  // ✅ Fire-and-forget: Start background processing WITHOUT await
-  // The promise continues execution after we return the response
-  executeFulfillmentExtraction(ctx, {
-      jobId,
-      triggeredBy: 'webhook',
-      fromDate: payload.fromDate,
-      toDate: payload.toDate,
-      updateState: payload.updateState === true,
-  })
-    .then((result) => {
-      log.info('✅ [BACKGROUND] Fulfillments extraction completed successfully', {
-      jobId,
-      recordCount: result.recordCount,
-        fileName: result.fileName,
-      errorCategory: result.errorCategory,
-    });
-    })
-    .catch((error: unknown) => {
-      const errorMessage = error instanceof Error ? error.message : String(error);
-      log.error('❌ [BACKGROUND] Fulfillments extraction failed', {
-      jobId,
-        error: errorMessage,
-      stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : typeof error,
-      });
-    });
-  // Return immediately with jobId (response sent with this return value)
-    return {
-    success: true,
-      jobId,
-    message: 'Fulfillments extraction started in background',
-    statusEndpoint: `https://{workspace}.versori.run/fulfillments-job-status`,
-    note: 'Poll the status endpoint with jobId to check progress',
-    };
-}
-/**
- * Handle job status query workflow
- *
- * Returns current job state from KV storage
- */
-async function handleJobStatusQuery(ctx: Context): Promise<any> {
-  const { data, log, openKv } = ctx;
-  const payload = data as JobStatusPayload;
-  if (!payload.jobId) {
-    return {
-      success: false,
-      error: 'Job ID required',
-      message: 'Provide jobId in request payload',
-    };
-  }
-  log.info('Job status query', { jobId: payload.jobId });
-  const status = await queryJobStatus(openKv(':project:'), payload.jobId, log);
-  if (!status) {
-    return {
-      success: false,
-      error: 'Job not found',
-      jobId: payload.jobId,
-      message: 'Job may have been deleted or never existed',
-    };
-  }
-  return {
-    success: true,
-    jobId: payload.jobId,
-    ...status,
-  };
-}
-```
----
-## 📄 src/services/extraction-orchestration.ts
-```typescript
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  XMLBuilder,
-  SftpDataSource,
-} from '@fluentcommerce/fc-connect-sdk';
-import mappingConfig from '../../config/fulfillments.export.xml.json' with { type: 'json' };
-import { FULFILLMENTS_QUERY } from '../config/queries';
-type ErrorCategory = 'CONFIG' | 'API' | 'SFTP' | 'NO_DATA';
-interface TimeWindow {
-  from: string;
-  to: string;
-  bufferedFrom: string;
-}
-interface ExtractionMetrics {
-  totalPages: number;
-  validRecords: number;
-  skippedRecords: number;
-  xmlSizeKb: number;
-  window: TimeWindow;
-}
-export interface FulfillmentExtractionParams {
-  jobId: string;
-  triggeredBy: 'schedule' | 'webhook';
-  fromDate?: string;
-  toDate?: string;
-  updateState: boolean;
-  fulfillmentStatuses?: string[];
-}
-export interface FulfillmentExtractionResult {
-  success: boolean;
-  jobId: string;
-  recordCount: number;
-  fileName?: string;
-  sftpPath?: string;
-  message?: string;
-  error?: string;
-  errorCategory?: ErrorCategory;
-  metrics?: ExtractionMetrics;
-}
-interface FulfillmentLineItem {
-  product_ref: string;
-  requested_quantity: number | string;
-  filled_quantity: number | string;
-}
-const STATE_KEY = 'lastFulfillmentSync';
-const DEFAULT_FALLBACK = '2024-01-01T00:00:00Z';
-const STATUS_DELIMITER = ',';
-function normalizeStatuses(input?: string[] | string | null): string[] | undefined {
-  if (!input) {
-    return undefined;
-  }
-  const raw = Array.isArray(input) ? input : input.split(STATUS_DELIMITER);
-  const cleaned = raw.map(status => status.trim()).filter(Boolean);
-  return cleaned.length > 0 ? cleaned : undefined;
-}
-function joinSftpPath(
-  requireAbsolutePath: boolean,
-  ...segments: Array<string | undefined>
-): string {
-  const sanitized = segments
-    .filter(Boolean)
-    .map(segment => String(segment).replace(/^\/+|\/+$/g, ''))
-    .join('/');
-  if (!sanitized) {
-    return requireAbsolutePath ? '/' : '';
-  }
-  return requireAbsolutePath && !sanitized.startsWith('/') ? `/${sanitized}` : sanitized;
-}
-export async function queryJobStatus(kv: any, jobId: string, log: any): Promise<any | undefined> {
-  try {
-    const tracker = new JobTracker(kv, log);
-    return await tracker.getJob(jobId);
-  } catch (error: any) {
-    log.error('Failed to get job status', {
-      jobId,
-      message: error instanceof Error ? error.message : String(error),
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'Error',
-    });
-    return undefined;
-  }
-}
-export async function executeFulfillmentExtraction(
-  ctx: any,
-  params: FulfillmentExtractionParams
-): Promise<FulfillmentExtractionResult> {
-  const { log, openKv, activation } = ctx;
-  const { jobId, triggeredBy, fromDate, toDate, updateState } = params;
-  const kv = openKv(':project:');
-  const tracker = new JobTracker(kv, log);
-  // Track execution duration
-  const executionStartTime = Date.now();
-  const overlapBufferSeconds = parseInt(activation.getVariable('overlapBufferSeconds') || '60', 10);
-  const fallbackStartDate = activation.getVariable('fallbackStartDate') || DEFAULT_FALLBACK;
-  const isManualOverride = Boolean(fromDate);
-  const shouldUpdateState = updateState === true;
-  try {
-    log.info('🚀 [STEP 1/8] Initializing job tracking', { jobId, triggeredBy });
-    await tracker.createJob(jobId, {
-      triggeredBy,
-      hasDateOverride: isManualOverride,
-      updateStateAfterRun: shouldUpdateState,
-      fromDate,
-      toDate,
-    });
-    log.info('📡 [STEP 2/8] Initializing Fluent Commerce client', { jobId });
-    const client = await createClient(ctx, { validateConnection: true });
-    if (!client) {
-      throw new Error('Failed to create Fluent Commerce client');
-    }
-    log.info('📅 [STEP 3/8] Determining date range for extraction', { jobId });
-    let rawState = fallbackStartDate;
-    if (!isManualOverride) {
-      try {
-        const stored = await kv.get(STATE_KEY);
-        if (typeof stored === 'string') {
-          rawState = stored;
-        } else if (stored?.timestamp) {
-          rawState = stored.timestamp;
-        }
-      } catch {
-        rawState = fallbackStartDate;
-      }
-    }
-    const overlapMs = overlapBufferSeconds * 1000;
-    const queryFrom = isManualOverride
-      ? fromDate!
-      : new Date(Math.max(0, new Date(rawState).getTime() - overlapMs)).toISOString();
-    const windowStart = isManualOverride ? (fromDate ?? queryFrom) : rawState;
-    const windowEnd = toDate || new Date().toISOString();
-    const statusesInput =
-      params.fulfillmentStatuses ?? activation.getVariable('fulfillmentStatuses');
-    const fulfillmentStatuses = normalizeStatuses(statusesInput);
-    await tracker.updateJob(jobId, {
-      stage: 'extraction',
-      message: 'Extracting fulfillments with auto-pagination',
-      status: 'processing',
-      statuses: fulfillmentStatuses,
-    });
-    const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
-    const maxRecords = parseInt(activation.getVariable('maxRecords') || '50000', 10);
-    log.info('📊 [STEP 4/8] Extracting fulfillments via GraphQL', {
-      jobId,
-      pageSize,
-      maxRecords,
-      queryFrom,
-      windowEnd,
-      statuses: fulfillmentStatuses,
-    });
-    const orchestrator = new ExtractionOrchestrator(client, log);
-    // Extract context for progress logging
-    const dateRangeInfo = {
-      start: queryFrom || 'N/A',
-      end: windowEnd || 'N/A',
-      statuses: fulfillmentStatuses?.join(', ') || 'all'
-    };
-    // Start logging with context
-    log.info(`📥 [ExtractionOrchestrator] Starting extraction`, {
-     query: 'fulfillments',
-     pageSize,
-     maxRecords,
-     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
-     statuses: dateRangeInfo.statuses,
-     jobId
-    });
-    const extractionResult = await orchestrator.extract({
-      query: FULFILLMENTS_QUERY,
-      resultPath: 'fulfillments.edges.node',
-      variables: {
-        statuses: fulfillmentStatuses,
-        updatedFrom: queryFrom,
-        updatedTo: windowEnd,
-      },
-      pageSize,
-      maxRecords,
-      validateItem: (item: any) => Boolean(item?.ref && item?.order?.ref),
-    });
-    const rawRecords = extractionResult.data || [];
-    const extractionErrors = extractionResult.errors || [];
-    const totalPages = extractionResult.stats.totalPages ?? 0;
-    log.info('✅ [STEP 4/8] Extraction complete', {
-      jobId,
-      totalPages,
-      totalRecords: extractionResult.stats.totalRecords,
-      validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
-      truncated: extractionResult.stats.truncated ?? false,
-      errorCount: extractionErrors.length,
-    });
-    // Completion logging with summary
-    log.info(`✅ [ExtractionOrchestrator] Extraction completed`, {
-     totalRecords: extractionResult.stats.totalRecords,
-     totalPages,
-     validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
-     failedValidations: extractionResult.stats.failedValidations,
-     truncated: extractionResult.stats.truncated,
-     truncationReason: extractionResult.stats.truncationReason,
-     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
-     jobId
-    });
-    if (extractionErrors.length > 0) {
-      log.warn('[STEP 4/8] Non-fatal extraction errors detected', {
-        jobId,
-        sampleErrors: extractionErrors.slice(0, 3),
-      });
-    }
-    if (rawRecords.length === 0) {
-      log.info('No fulfillments to process', { jobId });
-      if (shouldUpdateState) {
-        await kv.set(STATE_KEY, {
-          timestamp: windowEnd,
-          updatedAt: new Date().toISOString(),
-          recordCount: 0,
-          triggeredBy,
-        });
-      }
-      await tracker.markCompleted(jobId, {
-        recordCount: 0,
-        message: 'No fulfillments to extract',
-        stateUpdated: shouldUpdateState,
-        window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
-      });
-      return {
-        success: true,
-        jobId,
-        recordCount: 0,
-        message: 'No fulfillments to extract',
-        metrics: {
-          totalPages,
-          validRecords: 0,
-          skippedRecords: 0,
-          xmlSizeKb: 0,
-          window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
-        },
-      };
-    }
-    log.info('🔄 [STEP 5/8] Transforming data with UniversalMapper', {
-      jobId,
-      recordCount: rawRecords.length,
-    });
-    await tracker.updateJob(jobId, {
-      stage: 'transformation',
-      message: `Transforming ${rawRecords.length} fulfillments`,
-      status: 'processing',
-    });
-    const mapper = new UniversalMapper(mappingConfig);
-    const mappingResult = await mapper.map(rawRecords);
-    const mappingErrors = mappingResult.errors || [];
-    if (!mappingResult.success) {
-      log.error('[STEP 5/8] Transformation failed', {
-        jobId,
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-      await tracker.markFailed(jobId, 'UniversalMapper returned unsuccessful result', {
-        failedCount: mappingErrors.length,
-        errors: mappingErrors,
-      });
-      return {
-        success: false,
-        jobId,
-        recordCount: 0,
-        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
-        errorCategory: 'CONFIG',
-      };
-    }
-    const headerRecords: any[] = Array.isArray(mappingResult.data) ? mappingResult.data : [];
-    const mappingErrors = mappingResult.errors || [];
-    if (mappingErrors.length > 0) {
-      log.warn('[STEP 5/8] Some records failed transformation', {
-        jobId,
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-    }
-    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
-      log.info('ℹ️ [MAPPING] Optional fields skipped (undefined values)', {
-        jobId,
-        skippedFields: mappingResult.skippedFields,
-        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
-      });
-    }
-    const transformedRecords = headerRecords.map((record, index) => {
-      const raw = rawRecords[index] ?? {};
-      const lineItems: FulfillmentLineItem[] = ((raw.items?.edges ?? []) as any[])
-        .map(edge => edge?.node)
-        .filter(Boolean)
-        .map((item: any) => ({
-          product_ref: item.productRef ?? '',
-          requested_quantity: item.requestedQuantity ?? '',
-          filled_quantity: item.filledQuantity ?? '',
-        }));
-      return {
-        ...record,
-        line_items: lineItems,
-      };
-    });
-    if (transformedRecords.length === 0) {
-      await tracker.markFailed(jobId, 'All fulfillments failed mapping', {
-        failedCount: mappingErrors.length,
-        errors: mappingErrors,
-      });
-      return {
-        success: false,
-        jobId,
-        recordCount: 0,
-        error: 'All records failed mapping',
-        errorCategory: 'CONFIG',
-        metrics: {
-          totalPages,
-          validRecords: 0,
-          skippedRecords: rawRecords.length,
-          xmlSizeKb: 0,
-          window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
-        },
-      };
-    }
-    if (mappingErrors.length > 0) {
-      log.warn('[STEP 5/8] Some fulfillments failed transformation', {
-        jobId,
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-    }
-    const totalLineItems = transformedRecords.reduce(
-      (sum, record) => sum + record.line_items.length,
-      0
-    );
-    log.info('📝 [STEP 6/8] Generating XML document', {
-      jobId,
-      fulfillmentCount: transformedRecords.length,
-      lineItemCount: totalLineItems,
-    });
-    await tracker.updateJob(jobId, {
-      stage: 'xml_generation',
-      message: `Generating XML for ${transformedRecords.length} fulfillments`,
-      status: 'processing',
-    });
-    const xmlBuilder = new XMLBuilder({
-      prettyPrint: true,
-      indent: '  ',
-      xmlDeclaration: true,
-      encoding: 'UTF-8',
-      attributePrefix: '@',
-      textKey: '#text',
-    });
-    const xmlData = {
-      Fulfillments: {
-        Fulfillment: transformedRecords.map(record => {
-          const lineItemNodes =
-            record.line_items.length > 0
-              ? {
-                  LineItem: record.line_items.map(item => ({
-                    product_ref: item.product_ref,
-                    requested_quantity: String(item.requested_quantity ?? ''),
-                    filled_quantity: String(item.filled_quantity ?? ''),
-                  })),
-                }
-              : undefined;
-          return {
-            fulfillment_ref: record.fulfillment_ref,
-            order_ref: record.order_ref,
-            status: record.status,
-            delivery_type: record.delivery_type ?? '',
-            from_location_ref: record.from_location_ref ?? '',
-            from_location_name: record.from_location_name ?? '',
-            created_on: record.created_on,
-            updated_on: record.updated_on,
-            ...(lineItemNodes ? { LineItems: lineItemNodes } : {}),
-          };
-        }),
-      },
-    };
-    const xmlString = xmlBuilder.build(xmlData, 'Fulfillments');
-    const xmlBuffer = Buffer.from(xmlString, 'utf8');
-    const xmlSizeKb = Math.round(xmlBuffer.length / 1024);
-    const fileNamePrefix = activation.getVariable('fileNamePrefix') || 'fulfillments';
-    const timestampSuffix = new Date().toISOString().replace(/[:.]/g, '-');
-    const fileName = `${fileNamePrefix}-${timestampSuffix}.xml`;
-    log.info('📤 [STEP 7/8] Preparing SFTP upload', { jobId, fileName });
-    await tracker.updateJob(jobId, {
-      stage: 'sftp_upload',
-      message: `Uploading ${fileName} to SFTP`,
-      status: 'processing',
-    });
-    const sftpHost = activation.getVariable('sftpHost');
-    const sftpPort = parseInt(activation.getVariable('sftpPort') || '22', 10);
-    const sftpRemotePath = activation.getVariable('sftpRemotePath') || '/incoming/fulfillments/';
-    const sftpPrivateKey = activation.getVariable('sftpPrivateKey');
-    const requireAbsolutePaths = activation.getVariable('requireAbsolutePaths') === 'true';
-    if (!sftpHost) {
-      throw new Error('SFTP configuration incomplete: sftpHost activation variable missing');
-    }
-    const allConnections = activation.connections || [];
-    const sftpConnection = allConnections.find((conn: any) => conn.name === 'sftp_export_server');
-    if (!sftpConnection) {
-      throw new Error('SFTP connection "sftp_export_server" not found');
-    }
-    const credential = sftpConnection.credentials?.[0]?.credential;
-    if (!credential?.data?.basicAuth) {
-      throw new Error('SFTP connection not configured with Basic Authentication');
-    }
-    const { username, password } = credential.data.basicAuth;
-    const sftp = new SftpDataSource(
-      {
-        type: 'SFTP_XML',
-        connectionId: 'sftp_export_server',
-        name: 'Fulfillments XML Export',
-        settings: {
-          host: sftpHost,
-          port: sftpPort,
-          username,
-          password,
-          privateKey: sftpPrivateKey,
-          remotePath: sftpRemotePath,
-          filePattern: '*.xml',
-          requireAbsolutePaths,
-        },
-      },
-      log
-    );
-    const fullPath = joinSftpPath(requireAbsolutePaths, sftpRemotePath, fileName);
-    try {
-      await sftp.uploadFile(fullPath, xmlBuffer, {
-        createDirectories: true,
-        overwrite: true,
-      });
-      if (mappingErrors.length > 0) {
-        const reportPayload = {
-          fileName,
-          processedAt: new Date().toISOString(),
-          failedRecords: mappingErrors.length,
-          sampleErrors: mappingErrors.slice(0, 10),
-        };
-        const reportPath = joinSftpPath(
-          requireAbsolutePaths,
-          sftpRemotePath,
-          `${fileName.replace(/\.xml$/i, '')}-errors.json`
-        );
-        await sftp.uploadFile(
-          reportPath,
-          Buffer.from(JSON.stringify(reportPayload, null, 2), 'utf8'),
-          {
-            createDirectories: true,
-            overwrite: true,
-          }
-        );
-        log.warn('Uploaded transformation error report', { jobId, reportPath });
-      }
-    } finally {
-      await sftp.dispose();
-      log.info('SFTP connection disposed', { jobId });
-    }
-    log.info('💾 [STEP 8/8] Updating state and completing job', { jobId });
-    const newTimestamp = (() => {
-      const maxUpdated = rawRecords.reduce((max, record) => {
-        const current = new Date(record.updatedOn || windowEnd).getTime();
-        return current > max ? current : max;
-      }, new Date(windowStart).getTime());
-      return new Date(maxUpdated).toISOString();
-    })();
-    if (shouldUpdateState) {
-      await kv.set(STATE_KEY, {
-        timestamp: newTimestamp,
-        updatedAt: new Date().toISOString(),
-        recordCount: transformedRecords.length,
-        lineItemCount: totalLineItems,
-        fileName,
-        sftpPath: fullPath,
-        statuses: fulfillmentStatuses ?? [],
-      });
-    }
-    const executionDurationMs = Date.now() - executionStartTime;
-    await tracker.markCompleted(jobId, {
-      recordCount: transformedRecords.length,
-      lineItemCount: totalLineItems,
-      fileName,
-      remotePath: fullPath,
-      errorCount: mappingErrors.length,
-      errors: mappingErrors,
-      stateUpdated: shouldUpdateState,
-      window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
-      durationMs: executionDurationMs,
-    });
-    log.info('✅ Extraction completed successfully', {
-      jobId,
-      recordCount: transformedRecords.length,
-      durationMs: executionDurationMs,
-      durationSeconds: (executionDurationMs / 1000).toFixed(2),
-    });
-    return {
-      success: true,
-      jobId,
-      recordCount: transformedRecords.length,
-      fileName,
-      sftpPath: fullPath,
-      message: `Uploaded ${transformedRecords.length} fulfillments`,
-      metrics: {
-        totalPages,
-        validRecords: transformedRecords.length,
-        skippedRecords: mappingErrors.length,
-        xmlSizeKb,
-        window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
-      },
-    };
-  } catch (error: any) {
-    const errorMessage = error instanceof Error ? error.message : String(error);
-    log.error('❌ Fulfillment extraction failed', {
-      jobId,
-      message: errorMessage,
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'Error',
-    });
-    // Determine error category and recommendation
-    let errorCategory: ErrorCategory = 'API';
-    let recommendation = '';
-    if (errorMessage.includes('SFTP') || errorMessage.includes('connection')) {
-      errorCategory = 'SFTP';
-      recommendation = 'Check SFTP credentials and network connectivity. Verify host, port, and authentication settings in activation variables.';
-    } else if (errorMessage.includes('mapping') || errorMessage.includes('transformation')) {
-      errorCategory = 'CONFIG';
-      recommendation = 'Review mapping configuration in config/fulfillments.export.xml.json. Verify all source paths match GraphQL response structure.';
-    } else if (errorMessage.includes('GraphQL') || errorMessage.includes('query')) {
-      errorCategory = 'API';
-      recommendation = 'Verify Fluent Commerce API credentials and GraphQL query syntax. Check if the query matches the current schema.';
-    } else if (errorMessage.includes('No records') || errorMessage.includes('empty')) {
-      errorCategory = 'NO_DATA';
-      recommendation = 'No fulfillments found in the specified date range. This may be expected if there are no recent updates.';
-    }
-    await tracker.markFailed(jobId, {
-      error: errorMessage,
-      errorCategory,
-      recommendation,
-      triggeredBy,
-    });
-    return {
-      success: false,
-      jobId,
-      recordCount: 0,
-      error: errorMessage,
-      errorCategory,
-      details: {
-        recommendation,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',
-      },
-    };
-  }
-}
-```
----
-## 📄 src/config/queries.ts
-```typescript
-// ============================================================================
-// GRAPHQL QUERIES AND MAPPING CONFIGURATION
-// ============================================================================
-import mappingConfig from '../../config/fulfillments.export.xml.json' with { type: 'json' };
-/**
- * GraphQL query for fulfillments extraction
- *
- * Supports:
- * - Pagination (first, after)
- * - Date range filtering (updatedOn) - UTC timestamps required
- * - Status filtering (statuses)
- */
-export const FULFILLMENTS_QUERY = `
-  query GetFulfillments(
-    $statuses: [String!]
-    $updatedFrom: DateTime
-    $updatedTo: DateTime
-    $first: Int!
-    $after: String
-  ) {
-    fulfillments(
-      status: $statuses
-      updatedOn: { from: $updatedFrom, to: $updatedTo }
-      first: $first
-      after: $after
-    ) {
-      edges {
-        node {
-          id
-          ref
-          status
-          deliveryType
-          order {
-            ref
-          }
-          fromLocation {
-            ref
-            name
-          }
-          items {
-            edges {
-              node {
-                productRef
-                requestedQuantity
-                filledQuantity
-              }
-            }
-          }
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-        # Note: Fluent doesn't return endCursor - cursors are in edges[].cursor
-      }
-    }
-  }
-`;
-/**
- * Field mapping configuration loaded from JSON file
- *
- * Location: config/fulfillments.export.xml.json
- */
-export const FULFILLMENTS_EXPORT_MAPPING = mappingConfig;
-```
----
-## Testing
-### Test Scheduled Extraction
-The scheduled workflow runs automatically every 4 hours.
-**Check logs:**
-```
-[STEP 1/10] Loading and validating configuration
-[STEP 2/10] Initializing job tracker
-[STEP 3/10] Calculating extraction time window
-[STEP 4/10] Initializing SDK services
-[STEP 5/10] Extracting fulfillments from Fluent
-[STEP 6/10] Transforming records with UniversalMapper
-[STEP 7/10] Generating XML file
-[STEP 8/10] Uploading XML to SFTP
-[STEP 9/10] Updating extraction state
-[STEP 10/10] Finalizing job
-```
-### Test Ad-hoc Extraction
-```bash
-# Incremental (uses last sync timestamp)
-curl -X POST https://api.versori.com/webhooks/fulfillments-adhoc \
-  -H "Content-Type: application/json" \
-  -d '{}'
-# Date range override
-curl -X POST https://api.versori.com/webhooks/fulfillments-adhoc \
-  -H "Content-Type: application/json" \
-  -d '{
-    "fromDate": "2025-01-01T00:00:00Z",
-    "toDate": "2025-01-31T23:59:59Z",
-    "updateState": false
-  }'
-```
-### Test Job Status Query
-```bash
-curl -X POST https://api.versori.com/webhooks/fulfillments-job-status \
-  -H "Content-Type: application/json" \
-  -d '{
-    "jobId": "ADHOC_FULFILLMENTS_20251027_183045_abc123"
-  }'
-```
-**Response:**
-```json
-{
-  "success": true,
-  "jobId": "ADHOC_FULFILLMENTS_20251027_183045_abc123",
-  "status": "processing",
-  "stage": "transformation",
-  "message": "Transforming 150 records",
-  "createdAt": "2025-10-27T18:30:45.000Z",
-  "startedAt": "2025-10-27T18:30:46.000Z"
-}
-```
----
-## Next Steps
-1. **Review Configuration:** Verify all activation variables match your environment
-2. **Test with Sample Data:** Run ad-hoc extraction with small date range
-3. **Validate XML Output:** Check SFTP server for correctly formatted files
-4. **Enable Schedule:** Activate 4-hourly schedule after successful test
-5. **Monitor Jobs:** Use job status webhook to track extraction progress
-6. **Adjust Filters:** Customize status filters and date ranges as needed
-7. **Scale:** Increase page size and max records based on data volume
----
-## Testing Checklist
-**Before production deployment:**
-### 1. Schema Validation
-- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
-- [ ] Run `npx fc-connect validate-schema --mapping ./config/fulfillments.export.xml.json --schema ./fluent-schema.json`
-- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/fulfillments.export.xml.json --schema ./fluent-schema.json`
-- [ ] Verify all `source` paths in mapping exist in GraphQL schema
-- [ ] Verify query structure matches schema (fields, types, filters)
-### 2. Extraction Testing
-- [ ] Test with small dataset first (maxRecords=10)
-- [ ] Verify ExtractionOrchestrator pagination works correctly
-- [ ] Test with multiple pages of data (verify cursor handling)
-- [ ] Verify date range filtering (updatedOn filter)
-- [ ] Test empty result handling (no records in date range)
-- [ ] Verify extraction stops at maxRecords limit
-### 3. Mapping Testing
-- [ ] Verify required fields are populated
-- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
-- [ ] Test custom resolvers with edge cases (if any)
-- [ ] Verify nested field extraction
-- [ ] Test with null/missing fields
-- [ ] Verify mapping error collection works
-### 4. XML Generation Testing
-- [ ] Verify XML structure matches expected format
-- [ ] Test XML validation against XSD schema (if applicable)
-- [ ] Verify special character escaping in XML
-- [ ] Test with large datasets (>1000 records)
-- [ ] Verify UTF-8 encoding
-- [ ] Test XML namespace handling (if applicable)
-### 5. SFTP Upload Testing
-- [ ] Test SFTP connection and authentication
-- [ ] Verify file upload to correct path
-- [ ] Test file naming convention (timestamp format)
-- [ ] Verify file permissions on SFTP server
-- [ ] Test upload retry logic (simulate network failure)
-- [ ] Verify SFTP connection disposal (no connection leaks)
-### 6. State Management Testing
-- [ ] Verify overlap buffer prevents missed records (60-second default)
-- [ ] Test state recovery after extraction failure
-- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
-- [ ] Test first run with no previous state (uses fallbackStartDate)
-- [ ] Verify state update only happens on successful upload
-- [ ] Test manual date override (doesn't update state)
-### 7. Job Tracking Testing
-- [ ] Test job creation with JobTracker
-- [ ] Verify job status updates at each stage
-- [ ] Test job completion with metadata
-- [ ] Test job failure handling
-- [ ] Query job status via webhook endpoint
-- [ ] Verify job status persists in KV store
-### 8. Error Handling Testing
-- [ ] Test with invalid GraphQL query
-- [ ] Test with mapping errors (invalid field paths)
-- [ ] Test with SFTP connection failures
-- [ ] Test with authentication failures
-- [ ] Test with network timeouts
-- [ ] Verify error logging includes context (jobId, stage, error details)
-- [ ] Test error threshold logic (if applicable)
-### 9. Staging Environment Testing
-- [ ] Run full extraction in staging environment
-- [ ] Verify XML file format with downstream system
-- [ ] Monitor extraction duration and resource usage
-- [ ] Test with production-like data volumes
-- [ ] Verify no performance degradation over time
-### 10. Integration Testing
-- [ ] Test scheduled workflow (cron trigger)
-- [ ] Test ad hoc webhook trigger
-- [ ] Test job status query webhook
-- [ ] Verify activation variables are read correctly
-- [ ] Test with different extraction modes (incremental, date range)
-- [ ] End-to-end test: trigger → extract → transform → upload → verify file
----
-## Monitoring & Alerting
-### Success Response Example
-```json
-{
-  "success": true,
-  "jobId": "SCHEDULED_FUL_20251102_140000_abc123",
-  "recordsExtracted": 1523,
-  "fileName": "fulfillments-2025-11-02T14-00-00-000Z.xml",
-  "sftpPath": "/outbound/fulfillments/fulfillments-2025-11-02T14-00-00-000Z.xml",
-  "metrics": {
-    "extractionDurationMs": 12543,
-    "totalPages": 8,
-    "pageSize": 200,
-    "mappingErrors": 0,
-    "fileSizeBytes": 524288,
-    "uploadDurationMs": 1234
-  },
-  "timestamps": {
-    "extractionStart": "2025-11-02T14:00:00.000Z",
-    "extractionEnd": "2025-11-02T14:00:12.543Z",
-    "uploadComplete": "2025-11-02T14:00:13.777Z"
-  },
-  "state": {
-    "previousTimestamp": "2025-11-02T13:00:00.000Z",
-    "newTimestamp": "2025-11-02T13:59:58.123Z",
-    "stateUpdated": true,
-    "overlapBufferSeconds": 60
-  }
-}
-```
-### Error Response Example
-```json
-{
-  "success": false,
-  "jobId": "ADHOC_FUL_20251102_140500_xyz789",
-  "error": "SFTP upload failed: Connection timeout",
-  "errorCategory": "NETWORK",
-  "recordsExtracted": 0,
-  "stage": "sftp_upload",
-  "details": {
-    "message": "Failed to upload file after 3 retry attempts",
-    "retryAttempts": 3,
-    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
-  },
-  "state": {
-    "stateUpdated": false,
-    "willRetryNextRun": true,
-    "note": "State not advanced - next extraction will retry same time window"
-  }
-}
-```
-### Key Metrics to Track
-```typescript
-const METRICS = {
-  // Extraction Performance
-  extractionDurationMs: Date.now() - extractionStart,
-  recordCount: records.length,
-  pageCount: extractionResult.stats.totalPages,
-  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
-  // Transformation Performance
-  transformedCount: transformedRecords.length,
-  failedCount: mappingErrors.length,
-  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
-  // File Generation
-  fileSizeMB: (xmlContent.length / (1024 * 1024)).toFixed(2),
-  // Upload Performance
-  uploadDurationMs: uploadEnd - uploadStart,
-  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
-  // State Management
-  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
-  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
-};
-log.info('Extraction metrics', metrics);
-```
-### Alert Thresholds
-```typescript
-const ALERT_THRESHOLDS = {
-  // Duration Alerts
-  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
-  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
-  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
-  // Error Rate Alerts
-  MAX_ERROR_RATE: 0.05, // 5% mapping errors
-  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
-  // Volume Alerts
-  MAX_RECORDS_PER_RUN: 100000,
-  MIN_RECORDS_WARNING: 0, // Alert if no records found
-  MAX_FILE_SIZE_MB: 150, // 150MB
-  // State Alerts
-  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
-  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
-};
-// Check thresholds
-if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
-  log.warn('Extraction duration exceeded threshold', {
-    duration: metrics.extractionDurationMs,
-    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
-    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
-  });
-}
-```
-### Monitoring Dashboard Queries
-**Versori Platform Logs Query:**
-```
-# Successful extractions
-log_level:info AND message:"Extraction complete" AND jobId:*
-# Failed extractions
-log_level:error AND message:"Extraction workflow failed" AND jobId:*
-# Performance issues
-extractionDurationMs:>300000 OR uploadDurationMs:>120000
-# High error rates
-errorRate:>5
-# State management issues
-stateUpdated:false AND success:true
-```
-### Common Issues and Solutions
-**Issue**: "Extraction timeout after 10 minutes"
-- **Cause**: Too many records in single extraction
-- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
-- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
-**Issue**: "Mapping errors for 50% of records"
-- **Cause**: Schema mismatch between GraphQL response and mapping config
-- **Fix**: Run schema validation, update mapping config paths
-- **Prevention**: Use `npx fc-connect validate-schema` before deployment
-**Issue**: "SFTP connection timeout"
-- **Cause**: Network issues, firewall, or connection pool exhaustion
-- **Fix**: Check SFTP credentials, verify network connectivity
-- **Prevention**: Implement connection health checks, monitor connection status
-**Issue**: "State not updating after successful extraction"
-- **Cause**: KV write failure or intentional retry logic
-- **Fix**: Check KV logs, verify state update code executed
-- **Prevention**: Add KV write verification, log state updates explicitly
-**Issue**: "First run exceeds record limits"
-- **Cause**: No previous timestamp, fetches all historical records
-- **Fix**: Set fallbackStartDate close to current date, apply additional filters
-- **Prevention**: Use appropriate fallbackStartDate for initial runs
-**Issue**: "Excessive duplicate records in output"
-- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
-- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
-- **Prevention**: Monitor duplicate rates, verify state update logic
----
-## Troubleshooting Quick Reference
-| Error Message | Likely Cause | Solution |
-|--------------|--------------|----------|
-| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
-| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
-| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
-| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
-| "SFTP authentication failed" | Invalid credentials | Verify SFTP credentials in activation variables |
-| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
-| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
-| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
-| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
-| "XML generation failed" | Format-specific error | Check XML generation logic, validate output |
----
+---
+template_id: tpl-extract-fulfillments-to-sftp-xml
+canonical_filename: template-extraction-fulfillments-to-sftp-xml.md
+version: 2.0.0
+sdk_version: ^0.1.39
+runtime: versori
+direction: extraction
+source: fluent-graphql
+destination: sftp-xml
+entity: fulfillments
+format: xml
+logging: versori
+status: stable
+features:
+  - memory-management
+  - enhanced-logging
+  - pagination-progress
+  - dispose-finally
+---
+# Template: Extraction - Fulfillments to SFTP XML
+**Template Version:** 2.0.0
+**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
+**Last Updated:** 2025-01-24
+**Deployment Target:** Versori Platform
+**🆕 Version 2.0.0 Enhancements:**
+- ✅ **Memory Management** - Clear large result sets after processing batches
+- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
+- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
+- ✅ **Resource Cleanup** - SFTP dispose in finally blocks prevents connection leaks
+## Installation
+```bash
+npm install @fluentcommerce/fc-connect-sdk@^0.1.39
+```
+Use version `^0.1.39` to ensure compatibility with this template.
+---
+## 📚 STEP 1: Load These Docs (Human Checklist)
+1. REQUIRED (load all)
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+Copy-paste list (open these):
+fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+---
+## 📋 STEP 2: Tell Your AI (Prompt)
+Copy/paste this prompt after loading the documentation above:
+```
+Create a Versori scheduled extractor for fulfillments that uses ExtractionOrchestrator + JobTracker, incremental updatedOn with a 60s overlap buffer, transforms via UniversalMapper, generates XML with fast-xml-parser XMLBuilder, uploads to SFTP using SftpDataSource with dispose(). Include 3 workflows: scheduled, ad-hoc webhook, and job-status query with native Versori logging.
+```
+---
+## 📦 SDK Imports (Verified - Versori Optimized)
+```typescript
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  XMLBuilder,
+  SftpDataSource,
+  VersoriKVAdapter,
+} from '@fluentcommerce/fc-connect-sdk';
+import { schedule, webhook, http, fn } from '@versori/run';
+```
+---
+# Versori Scheduled: Fulfillments Extraction to SFTP XML (Shipment Tracking)
+**FC Connect SDK Use Case Guide**
+> SDK: [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
+> Version: Use ^0.1.39 - `npm install @fluentcommerce/fc-connect-sdk@^0.1.39`
+Context: Scheduled Versori workflow that extracts fulfillment data (shipment tracking, delivery information) from Fluent Commerce via GraphQL query with **ExtractionOrchestrator**, **JobTracker**, and **incremental timestamp tracking**, transforms with `UniversalMapper`, and writes **XML files** to SFTP for logistics coordination and delivery tracking.
+**Pattern**: EXTRACTION (Fluent → SFTP XML)
+**Complexity**: High | Runtime: Versori Platform (Scheduled)
+---
+## ⚠️ IMPORTANT: Production-Ready Base Template
+> **📋 BASE TEMPLATE - Ready for Production (Customize for Your Needs)**
+>
+> This is a **production-ready base template** demonstrating FC Connect SDK best practices for fulfillment extraction workflows with XML output.
+>
+> **✅ INCLUDED FEATURES:**
+>
+> - ✅ Comprehensive error handling with retry logic
+> - ✅ SFTP upload with exponential backoff (3 attempts)
+> - ✅ State management with overlap buffer (prevents missed records)
+> - ✅ Job tracking with lifecycle management
+> - ✅ Security (credential masking in logs)
+> - ✅ UTC time enforcement (prevents timezone bugs)
+> - ✅ Incremental extraction (safe, efficient, production-ready)
+> - ✅ Natural rate limiting via timestamps
+>
+> **📝 BEFORE DEPLOYING:**
+>
+> 1. Review and customize activation variables for your environment
+> 2. Test with sample data in your Versori workspace
+> 3. Adjust safety limits (pageSize, maxRecords) if needed
+> 4. Configure monitoring alerts for extraction failures
+> 5. Verify SFTP credentials and paths
+>
+> **This base template follows SDK best practices - tweak specific to your needs.**
+---
+## 🚀 Quick Deploy to Versori
+1. **Create Integration:** Versori Dashboard → New Integration → Name it "Fulfillments Extraction"
+2. **Copy Code:** Copy all workflow code from this template into Versori editor
+3. **Configure Connections:**
+   - `fluent_commerce` - Fluent Commerce API (OAuth2)
+   - `sftp_export_server` - SFTP destination (Basic Auth)
+   - `fulfillments-adhoc` - Ad-hoc webhook auth (API Key)
+   - `fulfillments-job-status` - Status webhook auth (API Key)
+4. **Set Activation Variables:** Configure all variables listed in Activation Variables section
+5. **Test:** Use ad-hoc webhook to test extraction before enabling schedule
+6. **Enable Schedule:** Activate scheduled workflow after successful test
+**Time to Deploy:** ~15 minutes
+---
+## 🔧 Common Customizations
+**Most Common Changes:**
+| What to Change         | Where                                     | Why                                |
+| ---------------------- | ----------------------------------------- | ---------------------------------- |
+| **Status filters**     | Activation variable `fulfillmentStatuses` | Target specific fulfillment states |
+| **Schedule frequency** | Workflow: `'0 */4 * * *'`                 | Change from 4-hourly to your needs |
+| **XML field mapping**  | `config/fulfillments.export.xml.json`     | Add/remove/modify output fields    |
+| **Date filters**       | GraphQL query variables                   | Filter by creation/update dates    |
+| **Page size**          | Activation variable `pageSize`            | Tune for performance (50-1000)     |
+| **Safety limits**      | Activation variable `maxRecords`          | Adjust based on dataset size       |
+**Less Common Changes:**
+| What to Change          | Where                                      | Default                 | When to Change               |
+| ----------------------- | ------------------------------------------ | ----------------------- | ---------------------------- |
+| **Overlap buffer**      | Activation variable `overlapBufferSeconds` | 60s                     | If seeing missed records     |
+| **SFTP retry attempts** | Code: `uploadWithRetry(..., 3)`            | 3                       | For unreliable networks      |
+| **Fallback start date** | Activation variable `fallbackStartDate`    | 2024-01-01              | For historical extraction    |
+| **Remote path**         | Activation variable `sftpRemotePath`       | /incoming/fulfillments/ | Match partner's requirements |
+**What NOT to Change (Unless You Know Why):**
+- 10-step workflow structure
+- Error taxonomy (CONFIG/API/SFTP/NO_DATA)
+- State management logic
+- UTC time enforcement
+- Connection cleanup (`sftp.dispose()`)
+---
+## What You'll Build
+- **Incremental extraction** using `updatedOn >= (lastRunTime - buffer)` with **overlap buffer**
+- **ExtractionOrchestrator** for auto-pagination and path-based extraction
+- **JobTracker** for lifecycle management
+- **State management** with VersoriKV to track last successful run
+- **Safety buffer** (60 seconds) to handle clock skew and race conditions
+- GraphQL query with status-based filtering and date range support
+- UniversalMapper transformation with nested order data
+- **XML file generation** with fulfillment/shipment data
+- **SFTP upload** to partner systems with proper connection cleanup
+- **3 workflow patterns**: scheduled, ad-hoc webhook, job status query
+- **Failure recovery** with timestamp tracking
+## Business Use Case
+**4-hourly shipment tracking extraction for logistics:**
+- Extract shipment and delivery information for carrier integration
+- Track fulfillment status changes (SHIPPED, DELIVERED)
+- Monitor delivery exceptions and returns (RMA)
+- Support logistics partner feeds
+- Enable shipment tracking updates
+- Run every 4 hours for timely status updates
+- Deliver XML format to enterprise systems via SFTP
+## Fulfillments Explained
+**Fulfillment** = Shipment execution record
+- Represents order fulfillment process
+- Tracks shipment status and delivery progress
+- Links to orders, items, and locations
+- Used for: Shipment tracking, delivery coordination, logistics integration
+**Key Fields:**
+- `ref` - Unique fulfillment reference
+- `status` - Fulfillment state (SHIPPED, DELIVERED, etc.)
+- `deliveryType` - Shipping method
+- `order.ref` - Associated order reference
+- `fromLocation` - Shipping origin
+- `items` - Fulfilled order items
+- `createdOn`, `updatedOn` - Timestamps for tracking
+## SDK Methods Used
+```typescript
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  XMLBuilder,
+  SftpDataSource,
+  VersoriKVAdapter,
+} from '@fluentcommerce/fc-connect-sdk';
+await createClient(ctx); // Versori-aware client
+const orchestrator = new ExtractionOrchestrator(client, log); // Auto-pagination
+const tracker = new JobTracker(kv, log); // Job lifecycle tracking
+await orchestrator.extract({ query, resultPath, variables, pageSize, maxRecords }); // Extract
+new VersoriKVAdapter(ctx.openKv(':project:')); // State management
+new UniversalMapper(exportMapping); // Field transformation
+new XMLBuilder({ prettyPrint: true, xmlDeclaration: true }); // XML generation
+const xml = xmlBuilder.build(data, 'RootElement'); // Build XML string
+await sftp.uploadFile(remotePath, buffer, options); // SFTP upload
+await sftp.dispose(); // CRITICAL: Connection cleanup
+```
+## Activation Variables
+**Configuration is driven by activation variables - modify these instead of code:**
+```json
+{
+  "fulfillmentStatuses": "SHIPPED,DELIVERED",
+  "sftpHost": "sftp.partner.com",
+  "sftpPort": 22,
+  "sftpPrivateKey": "",
+  "sftpRemotePath": "/incoming/fulfillments/",
+  "fileNamePrefix": "fulfillments",
+  "requireAbsolutePaths": "false",
+  "pageSize": 200,
+  "maxRecords": 50000,
+  "fallbackStartDate": "2024-01-01T00:00:00Z",
+  "overlapBufferSeconds": "60"
+}
+```
+> **Note:** `sftpUsername` and `sftpPassword` are fetched from the `sftp_export_server` connection (see SFTP Credential Management section below).
+## SFTP Credential Management
+Fetch credentials securely from Versori connection vault using `ctx.credentials().getAccessToken()`:
+```typescript
+import { Buffer } from 'node:buffer'; // Required for Versori/Deno
+// ========================================
+// SFTP CREDENTIAL RETRIEVAL
+// ========================================
+/**
+ * Retrieve SFTP credentials from connection configuration
+ *
+ * This approach retrieves credentials stored in the Versori connection settings.
+ * The connection must be configured in the UI with Basic Authentication.
+ *
+ * Steps:
+ * 1. Call ctx.credentials().getAccessToken('SFTP') to get base64-encoded credentials
+ * 2. Decode the accessToken from base64 to get "username:password" string
+ * 3. Split on ':' to extract username and password
+ *
+ * Connection Name: 'sftp_export_server' (must match the connection name in Versori UI)
+ * Auth Type: Basic Authentication (username + password)
+ *
+ * This method provides:
+ * - Centralized credential management through Versori UI
+ * - Better security (credentials not stored in integration variables)
+ * - Easier credential rotation and updates
+ */
+log.info('Retrieving SFTP credentials from connection configuration');
+let sftpUsername: string;
+let sftpPassword: string;
+try {
+  // Retrieve credentials from the 'sftp_export_server' connection
+  const sftpCred = await ctx.credentials().getAccessToken('sftp_export_server');
+  if (!sftpCred?.accessToken) {
+    throw new Error('No SFTP credentials found in connection configuration');
+  }
+  // Decode base64 accessToken to get "username:password"
+  const rawBasicAuth = Buffer.from(sftpCred.accessToken, 'base64').toString('utf-8');
+  // Split on ':' to extract username and password
+  const parts = rawBasicAuth.split(':');
+  if (parts.length !== 2) {
+    throw new Error('Invalid SFTP credential format - expected username:password');
+  }
+  sftpUsername = parts[0];
+  sftpPassword = parts[1];
+  log.info('SFTP credentials retrieved successfully', {
+    hasUsername: !!sftpUsername,
+    hasPassword: !!sftpPassword,
+    usernameLength: sftpUsername.length,
+    passwordLength: sftpPassword.length,
+  });
+} catch (error: any) {
+  log.error('Failed to retrieve SFTP credentials', {
+    message: error instanceof Error ? error.message : String(error),
+    stack: error instanceof Error ? error.stack : undefined,
+    errorType: error instanceof Error ? error.constructor.name : 'Error',
+  });
+  return {
+    success: false,
+    error: 'Failed to retrieve SFTP credentials from connection configuration',
+    details: error instanceof Error ? error.message : String(error),
+    recommendation: 'Please ensure the SFTP connection is configured in the Connections section with Basic Authentication (username and password)',
+  };
+}
+// Initialize SFTP with connection credentials
+const sftp = new SftpDataSource(
+  {
+    type: 'SFTP_XML',
+    connectionId: 'sftp_export_server',
+    name: 'Export SFTP',
+    settings: {
+      host: ctx.activation.getVariable('sftpHost'),
+      port: parseInt(ctx.activation.getVariable('sftpPort') || '22'),
+      username: sftpUsername,
+      password: sftpPassword,
+      remotePath: ctx.activation.getVariable('sftpRemotePath') || '/exports',
+      encoding: 'utf8',
+    },
+  },
+  log
+);
+```
+**Benefits:**
+- ✅ Credentials stored securely in Versori vault
+- ✅ Centralized management - update in one place
+- ✅ Access control enforced by platform
+- ✅ No credentials in activation variables
+- ✅ Explicit error handling with validation
+### Setup Steps
+1. In Versori Dashboard → Connections → Add Connection
+2. Name: `sftp_export_server`
+3. Type: Basic Auth
+4. Enter SFTP username and password
+5. Save connection
+6. Reference in code: `ctx.credentials().getAccessToken('sftp_export_server')`
+---
+## Webhook Security Configuration
+**⚠️ IMPORTANT:** Webhook security is now handled at the **Versori connection level** (not in code).
+### Required Connections
+Create these connections in Versori Dashboard before deploying:
+1. **`fulfillments-adhoc`** (Ad-hoc webhook auth)
+   - Configure authentication (e.g., API Key) in the connection settings
+2. **`fulfillments-job-status`** (Job status query auth)
+   - Configure authentication (e.g., API Key) in the connection settings
+### How to Create Webhook Connections
+1. **Navigate to Versori Dashboard** → Connections
+2. **Click "Add Connection"**
+3. **Configure:**
+   - Name: `fulfillments-adhoc`
+   - Authentication: API Key (or preferred method)
+   - Provide header name/value in the connection UI
+4. **Save Connection**
+5. **Repeat** for `fulfillments-job-status` connection
+### Generate Secure API Keys
+```bash
+# Node.js - Generate cryptographically secure 64-character key
+node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
+# Example output:
+# 7f3e9a2b1c8d6e4f5a0b9c8d7e6f5a4b3c2d1e0f9a8b7c6d5e4f3a2b1c0d9e8f
+```
+### How It Works
+```
+┌────────────────┐
+│  External       │
+│  System         │
+└──────┬─────────┘
+       │
+       │ POST /webhook/fulfillments-adhoc
+       │ Headers: (as configured on the connection)
+       ▼
+ ┌─────────────────────────┐
+ │  Versori Platform        │
+ │  ✓ Validates API key     │  ← Authentication happens HERE
+ │  ✓ Checks header         │
+ │  ✗ Rejects if invalid    │
+ └──────┬──────────────────┘
+        │
+        │ ✅ Only reaches workflow if valid
+        ▼
+ ┌─────────────────────────┐
+ │  Your Workflow Code      │
+ │  (No auth code needed!)  │
+ └──────────────────────────┘
+```
+**Benefits:**
+- ✅ Security at platform level (before code runs)
+- ✅ No manual validation code needed
+- ✅ Consistent across all webhooks
+- ✅ Easy to rotate keys (update connection, not code)
+- ✅ Failed auth never hits your workflow (saves resources)
+## Export Mapping Configuration
+Create file: `config/fulfillments.export.xml.json`
+```json
+{
+  "name": "fulfillments.export.xml",
+  "version": "1.0.0",
+  "description": "Fluent Fulfillments → SFTP XML Export (shipment tracking)",
+  "fields": {
+    "fulfillment_ref": { "source": "ref", "required": true, "resolver": "sdk.trim" },
+    "order_ref": { "source": "order.ref", "required": true, "resolver": "sdk.trim" },
+    "status": { "source": "status", "required": true, "resolver": "sdk.uppercase" },
+    "delivery_type": { "source": "deliveryType", "required": false, "resolver": "sdk.trim" },
+    "from_location_ref": {
+      "source": "fromLocation.ref",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "from_location_name": {
+      "source": "fromLocation.name",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "created_on": { "source": "createdOn", "required": true, "resolver": "sdk.toString" },
+    "updated_on": { "source": "updatedOn", "required": true, "resolver": "sdk.toString" }
+  }
+}
+```
+**Note**: The mapping uses nested paths like `order.ref` and `fromLocation.ref` to access related entity fields.
+## Mapping & Resolvers Explained
+This section explains how the SDK transforms raw GraphQL data into your XML export format using **UniversalMapper** and **SDK resolvers**.
+### SDK Resolvers Used
+| Field                | Resolver        | Why?                         | Example Transformation                  |
+| -------------------- | --------------- | ---------------------------- | --------------------------------------- |
+| `fulfillment_ref`    | `sdk.trim`      | Clean fulfillment references | `" FF-001 "` → `"FF-001"`               |
+| `order_ref`          | `sdk.trim`      | Clean order references       | `" ORD-001 "` → `"ORD-001"`             |
+| `status`             | `sdk.uppercase` | Normalize status codes       | `"shipped"` → `"SHIPPED"`               |
+| `delivery_type`      | `sdk.trim`      | Clean delivery method        | `" STANDARD "` → `"STANDARD"`           |
+| `from_location_ref`  | `sdk.trim`      | Clean location references    | `" LOC-001 "` → `"LOC-001"`             |
+| `from_location_name` | `sdk.trim`      | Clean location names         | `" NYC Warehouse "` → `"NYC Warehouse"` |
+| `created_on`         | `sdk.toString`  | Preserve ISO 8601 timestamp  | `"2025-01-15T10:00:00.000Z"` → same     |
+| `updated_on`         | `sdk.toString`  | Preserve ISO 8601 timestamp  | `"2025-01-22T08:30:00.000Z"` → same     |
+### Transformation Flow
+```typescript
+// 1. GraphQL Response (raw data from Fluent Commerce)
+const rawFulfillment = {
+  ref: ' FF-001 ',
+  status: 'shipped',
+  deliveryType: ' STANDARD ',
+  order: {
+    ref: ' ORD-001 ',
+  },
+  fromLocation: {
+    ref: ' LOC-001 ',
+    name: ' NYC Warehouse ',
+  },
+  items: {
+    edges: [
+      {
+        node: {
+          productRef: 'PROD-001',
+          requestedQuantity: 2,
+          filledQuantity: 2,
+        },
+      },
+    ],
+  },
+  createdOn: '2025-01-15T10:00:00.000Z',
+  updatedOn: '2025-01-22T08:30:00.000Z',
+};
+// 2. UniversalMapper applies SDK resolvers (handles nested paths automatically)
+const mapper = new UniversalMapper(fulfillmentsExportMapping);
+const result = await mapper.map(rawFulfillment);
+// 3. Transformed Output (clean, normalized for XML)
+const transformedFulfillment = {
+  fulfillment_ref: 'FF-001',
+  order_ref: 'ORD-001',
+  status: 'SHIPPED',
+  delivery_type: 'STANDARD',
+  from_location_ref: 'LOC-001',
+  from_location_name: 'NYC Warehouse',
+  created_on: '2025-01-15T10:00:00.000Z',
+  updated_on: '2025-01-22T08:30:00.000Z',
+};
+```
+### Nested Object Mapping
+Fulfillments include **nested objects** that require special path syntax:
+```typescript
+// ✅ CORRECT - Access nested object fields with dot notation
+{
+  "order_ref": { "source": "order.ref" },
+  "from_location_ref": { "source": "fromLocation.ref" },
+  "from_location_name": { "source": "fromLocation.name" }
+}
+```
+**GraphQL must fetch the nested objects:**
+```graphql
+node {
+  ref                    # Direct field
+  status
+  order {                # Nested Order object
+    ref                  # Access via order.ref
+  }
+  fromLocation {         # Nested Location object
+    ref                  # Access via fromLocation.ref
+    name                 # Access via fromLocation.name
+  }
+  items {                # Nested items collection
+    edges {
+      node {
+        productRef
+        requestedQuantity
+        filledQuantity
+      }
+    }
+  }
+}
+```
+**UniversalMapper handles nested paths automatically** - just use dot notation in the mapping configuration.
+### Available SDK Resolvers
+The SDK provides these built-in resolvers (no custom code needed):
+**String Transformations:**
+- `sdk.trim` - Remove leading/trailing whitespace
+- `sdk.uppercase` - Convert to uppercase
+- `sdk.lowercase` - Convert to lowercase
+- `sdk.toString` - Convert to string
+**Number Parsing:**
+- `sdk.parseInt` - Parse as integer
+- `sdk.parseFloat` - Parse as decimal
+- `sdk.number` - Parse as number (auto-detect int/float)
+**Date Formatting:**
+- `sdk.formatDate` - ISO 8601 date only (YYYY-MM-DD)
+- `sdk.formatDateShort` - Short date format
+- `sdk.parseDate` - Parse various date formats
+**Type Conversions:**
+- `sdk.boolean` - Convert to boolean
+- `sdk.parseJson` - Parse JSON strings
+- `sdk.toJson` - Convert to JSON string
+**Utilities:**
+- `sdk.identity` - Return value unchanged
+- `sdk.coalesce` - Return first non-null value
+See [Universal Mapping Guide](../../../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) for complete resolver documentation.
+## GraphQL Query
+**Note**: This query is verified against Fluent Commerce schema introspection.
+```graphql
+query GetFulfillments(
+  $statuses: [String!]
+  $updatedFrom: DateTime
+  $updatedTo: DateTime
+  $first: Int!
+  $after: String
+) {
+  fulfillments(
+    status: $statuses
+    updatedOn: { from: $updatedFrom, to: $updatedTo }
+    first: $first
+    after: $after
+  ) {
+    edges {
+      node {
+        id
+        ref
+        status
+        deliveryType
+        order {
+          ref
+        }
+        fromLocation {
+          ref
+          name
+        }
+        items {
+          edges {
+            node {
+              productRef
+              requestedQuantity
+              filledQuantity
+            }
+          }
+        }
+        createdOn
+        updatedOn
+      }
+      cursor
+    }
+    pageInfo {
+      hasNextPage
+    }
+  }
+}
+```
+---
+## Versori Workflows Structure
+**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
+**Trigger Types:**
+- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
+- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
+- **`workflow()`** → Durable workflows (advanced, rarely used)
+**Execution Steps (chained to triggers):**
+- **`http()`** → External API calls (chained from schedule/webhook)
+- **`fn()`** → Internal processing (chained from schedule/webhook)
+### Recommended Project Structure
+**This template demonstrates a complete workflow implementation in a single file.** In production, consider organizing your code into this modular structure:
+```
+fulfillments-extraction/
+├── index.ts                              # Entry point - exports all workflows
+└── src/
+    ├── workflows/
+    │   └── fulfillments-extraction.ts            # All workflows (scheduled + webhooks)
+    │
+    ├── services/
+    │   └── extraction-orchestration.ts           # Shared orchestration logic (reusable)
+    │
+    └── config/
+        ├── queries.ts                            # GraphQL queries
+        └── fulfillments.export.xml.json          # Mapping configuration
+```
+**Alternative Modular Structure** (for larger projects with multiple workflows):
+```
+fulfillments-extraction/
+├── index.ts
+└── src/
+    ├── workflows/
+    │   ├── scheduled/
+    │   │   └── daily-fulfillments-extraction.ts  # Scheduled extraction only
+    │   └── webhook/
+    │       ├── adhoc-fulfillments-extraction.ts  # Ad-hoc webhook
+    │       └── job-status-check.ts               # Status query webhook
+    ├── services/
+    │   └── extraction-orchestration.ts
+    └── config/
+        ├── queries.ts
+        └── fulfillments.export.xml.json
+```
+---
+## Expected XML Output
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<Fulfillments>
+  <Fulfillment>
+    <fulfillment_ref>FF-001</fulfillment_ref>
+    <order_ref>ORD-001</order_ref>
+    <status>SHIPPED</status>
+    <delivery_type>STANDARD</delivery_type>
+    <from_location_ref>LOC-001</from_location_ref>
+    <from_location_name>NYC Warehouse</from_location_name>
+    <created_on>2025-01-15T10:00:00Z</created_on>
+    <updated_on>2025-01-22T08:30:00Z</updated_on>
+  </Fulfillment>
+  <Fulfillment>
+    <fulfillment_ref>FF-002</fulfillment_ref>
+    <order_ref>ORD-002</order_ref>
+    <status>DELIVERED</status>
+    <delivery_type>EXPRESS</delivery_type>
+    <from_location_ref>LOC-002</from_location_ref>
+    <from_location_name>LA Warehouse</from_location_name>
+    <created_on>2025-01-16T11:00:00Z</created_on>
+    <updated_on>2025-01-22T09:15:00Z</updated_on>
+  </Fulfillment>
+</Fulfillments>
+```
+## Production Safety & Guardrails
+### Overview
+Fulfillments require safeguards even in incremental mode:
+- **Status changes**: Fulfillments can update frequently during processing
+- **Delivery tracking**: Real-time status updates create rapid data changes
+- **Order volume**: High-volume retailers process thousands of fulfillments daily
+- **Partner feeds**: Logistics systems need predictable file sizes
+### Hard Limits
+```typescript
+const SAFETY_LIMITS = {
+  MAX_RECORDS_PER_RUN: 100000, // 100k fulfillments per run
+  MAX_RECORDS_PER_FILE: 25000, // 25k per XML file
+  MAX_FILE_SIZE_MB: 75, // 75MB per file
+  MAX_XML_SIZE_MB: 150, // Total extraction size
+  CHUNK_SIZE: 5000, // Process in chunks
+  ESTIMATED_BYTES_PER_RECORD: 600, // Conservative estimate for XML
+};
+```
+**Why these limits?**
+- Fulfillments have nested items (larger than simple entities)
+- Shipment tracking needs fast processing for timely updates
+- Multiple status changes can accumulate quickly
+- Partner systems require predictable batch sizes
+- XML is more verbose than CSV (~25% larger)
+### Runtime Validation Function
+```typescript
+/**
+ * Validate extraction safety limits before processing
+ */
+function validateExtractionLimits(recordCount: number, estimatedSizeMB: number) {
+  const MAX_RECORDS_PER_RUN = 100000;
+  const MAX_XML_SIZE_MB = 150;
+  if (recordCount > MAX_RECORDS_PER_RUN) {
+    return {
+      valid: false,
+      error: `Extraction limit exceeded: ${recordCount} records (max: ${MAX_RECORDS_PER_RUN})`,
+      recommendation: `Fulfillments dataset too large. Consider:
+        1. Increase extraction frequency (4-hourly → hourly)
+        2. Filter by status (extract SHIPPED and DELIVERED separately)
+        3. Filter by date range (last 24 hours only)
+        4. Split by location (extract by fulfillment center)`,
+      recordCount,
+      maxAllowed: MAX_RECORDS_PER_RUN,
+    };
+  }
+  if (estimatedSizeMB > MAX_XML_SIZE_MB) {
+    return {
+      valid: false,
+      error: `XML size limit exceeded: ${estimatedSizeMB}MB (max: ${MAX_XML_SIZE_MB}MB)`,
+      recommendation: 'File splitting required. Filter by status or date range.',
+      estimatedSizeMB,
+      maxAllowed: MAX_XML_SIZE_MB,
+    };
+  }
+  return { valid: true };
+}
+```
+---
+## Production Capabilities
+Built-in safeguards and best practices:
+| Capability              | Implementation                             | Benefit                         |
+| ----------------------- | ------------------------------------------ | ------------------------------- |
+| **Config Validation**   | Fail-fast with detailed error messages     | Catches issues before execution |
+| **SFTP Retry**          | 3 attempts with exponential backoff        | Resilient to network issues     |
+| **UTC Enforcement**     | Normalize all dates to UTC                 | Prevents timezone bugs          |
+| **Window Persistence**  | Save effective time window in job metadata | Complete execution history      |
+| **Error Taxonomy**      | Categorize errors: CONFIG/API/SFTP/NO_DATA | Better support triage           |
+| **Extended Metrics**    | Track pages, skipped records, XML size     | Enhanced observability          |
+| **Input Normalization** | Lowercase, trim, dedupe list inputs        | Consistent filtering            |
+| **Secrets Masking**     | Mask credentials in error logs             | Security compliance             |
+---
+## Complete Workflow Code
+### File Structure
+**This template demonstrates a single-file implementation** (all workflows in one file for easier understanding). The actual code structure matches the "Recommended Project Structure" shown earlier.
+```
+fulfillments-extraction/
+├── src/
+│   ├── index.ts                              # Entry point (exports workflows)
+│   ├── workflows/
+│   │   └── fulfillments-extraction.ts        # All 3 workflows (scheduled + 2 webhooks)
+│   ├── services/
+│   │   └── extraction-orchestration.ts       # Main orchestration logic (shared)
+│   └── config/
+│       └── queries.ts                        # GraphQL queries & mapping
+├── config/
+│   └── fulfillments.export.xml.json          # Field mapping config
+└── package.json
+```
+**Why Single File?**
+- ✅ Easier to understand workflow relationships
+- ✅ All patterns visible in one place
+- ✅ Simpler to maintain for small projects
+- ⚠️ For larger projects, split into `scheduled/` and `webhook/` subdirectories (see "Alternative Modular Structure" above)
+**Entry Point (`src/index.ts`):**
+```typescript
+// Export all workflows for Versori to discover
+export {
+  scheduledFulfillmentsExtraction,
+  adhocFulfillmentsExtraction,
+  fulfillmentsJobStatus,
+} from './workflows/fulfillments-extraction';
+// Export services for testing or direct use
+export {
+  executeFulfillmentExtraction,
+  queryJobStatus,
+  generateJobId,
+} from './services/extraction-orchestration';
+// Export configuration for reference
+export { FULFILLMENTS_QUERY, FULFILLMENTS_EXPORT_MAPPING } from './config/queries';
+```
+**How Versori Discovers Workflows:**
+Versori scans exported workflow objects (created with `schedule()`, `webhook()`) from your entry point and automatically registers them.
+---
+## 📄 src/workflows/fulfillments-extraction.ts
+```typescript
+// ============================================================================
+// FULFILLMENTS EXTRACTION WORKFLOWS
+// ============================================================================
+// Purpose: Extract shipment tracking and delivery data from Fluent Commerce
+//          and export as XML to SFTP for logistics coordination
+//
+// Workflows:
+//   1. Scheduled: 4-hourly incremental extraction with state tracking
+//   2. Ad-hoc: Manual trigger with custom date ranges
+//   3. Job Status: Query extraction job progress and results
+// ============================================================================
+import { schedule, webhook, http, fn } from '@versori/run';
+import type { Context } from '@versori/run';
+import {
+  executeFulfillmentExtraction,
+  queryJobStatus,
+  generateJobId,
+} from '../services/extraction-orchestration';
+// ============================================================================
+// SECTION 1: TYPE DEFINITIONS
+// ============================================================================
+/**
+ * Error categories for classification and triage
+ */
+type ErrorCategory = 'CONFIG' | 'API' | 'SFTP' | 'NO_DATA';
+/**
+ * Extraction time window with buffer tracking
+ */
+interface TimeWindow {
+  from: string; // Original start time (ISO 8601 UTC)
+  to: string; // End time (ISO 8601 UTC)
+  bufferedFrom: string; // Actual query start with overlap buffer applied
+}
+/**
+ * Detailed extraction metrics for observability
+ */
+interface ExtractionMetrics {
+  totalPages: number; // GraphQL pages retrieved
+  validRecords: number; // Successfully transformed records
+  skippedRecords: number; // Records that failed mapping
+  xmlSizeKb: number; // Final XML file size
+  window: TimeWindow; // Time window used for extraction
+}
+/**
+ * Result structure returned by extraction workflows
+ */
+interface ExtractionResult {
+  success: boolean;
+  jobId: string;
+  recordCount?: number;
+  fileName?: string;
+  message?: string;
+  error?: string;
+  errorCategory?: ErrorCategory;
+  metrics?: ExtractionMetrics;
+}
+/**
+ * Webhook payload for ad-hoc extraction
+ */
+interface AdhocExtractionPayload {
+  fromDate?: string; // ISO 8601 UTC timestamp - extraction start
+  toDate?: string; // ISO 8601 UTC timestamp - extraction end
+  updateState?: boolean; // Whether to update last run timestamp
+}
+/**
+ * Webhook payload for job status query
+ */
+interface JobStatusPayload {
+  jobId: string; // Job ID to query
+}
+// ============================================================================
+// SECTION 2: WORKFLOW EXPORTS
+// ============================================================================
+/**
+ * WORKFLOW 1: Scheduled Extraction
+ *
+ * Runs: Every 4 hours (0 */4 * * *)
+ * Purpose: Incremental extraction of updated fulfillments
+ * State: Updates lastRunTime after successful extraction
+ *
+ * Behavior:
+ * - Loads last run timestamp from KV storage
+ * - Applies 60-second overlap buffer for clock skew
+ * - Extracts records with updatedOn >= (lastRunTime - buffer)
+ * - Generates XML and uploads to SFTP with retry
+ * - Updates state with current timestamp
+ */
+export const scheduledFulfillmentsExtraction = schedule(
+  'fulfillments-extract-xml-4hourly',
+  '0 */4 * * *' // Every 4 hours at minute 0
+).then(
+  http('execute-scheduled-extraction', { connection: 'fluent_commerce' }, handleScheduledExtraction)
+);
+/**
+ * WORKFLOW 2: Ad-hoc Extraction
+ *
+ * Trigger: Webhook POST to /webhooks/fulfillments-adhoc
+ * Purpose: On-demand extraction with custom date ranges
+ * State: Optional - only updates if updateState: true in payload
+ *
+ * Payload Examples:
+ *
+ * 1. Extract last 7 days (no state update):
+ *    {
+ *      "fromDate": "2025-01-22T00:00:00Z",
+ *      "toDate": "2025-01-29T00:00:00Z"
+ *    }
+ *
+ * 2. Full re-extraction with state reset:
+ *    {
+ *      "fromDate": "2024-01-01T00:00:00Z",
+ *      "updateState": true
+ *    }
+ *
+ * Note: fromDate and toDate MUST be ISO 8601 UTC timestamps
+ */
+export const adhocFulfillmentsExtraction = webhook('fulfillments-adhoc', {
+  connection: 'fulfillments-adhoc', // Security via Versori connection
+  response: { mode: 'sync' }, // ✅ Sync mode: response sent when handler returns
+}).then(http('execute-adhoc-extraction', { connection: 'fluent_commerce' }, handleAdhocExtraction));
+/**
+ * WORKFLOW 3: Job Status Query
+ *
+ * Trigger: Webhook GET/POST to /webhooks/fulfillments-job-status
+ * Purpose: Check extraction job progress and results
+ *
+ * Query Payload:
+ * {
+ *   "jobId": "SCHEDULED_FULFILLMENTS_20250129_120000_abc123"
+ * }
+ *
+ * Response:
+ * {
+ *   "success": true,
+ *   "jobId": "...",
+ *   "status": "completed",
+ *   "stage": "upload_complete",
+ *   "recordCount": 1250,
+ *   "fileName": "fulfillments_20250129_120000.xml",
+ *   "metrics": { ... }
+ * }
+ */
+export const fulfillmentsJobStatus = webhook('fulfillments-job-status', {
+  connection: 'fulfillments-job-status', // Security via Versori connection
+  response: { mode: 'sync' },
+}).then(fn('query-job-status', handleJobStatusQuery));
+// ============================================================================
+// SECTION 3: WORKFLOW HANDLERS
+// ============================================================================
+/**
+ * Handle scheduled extraction workflow
+ *
+ * Always updates state for next incremental run
+ */
+async function handleScheduledExtraction(ctx: Context): Promise<ExtractionResult> {
+  const { log } = ctx;
+  const jobId = generateJobId('SCHEDULED', 'FULFILLMENTS');
+  log.info('Scheduled extraction started', { jobId });
+  try {
+    const result = await executeFulfillmentExtraction(ctx, {
+      jobId,
+      triggeredBy: 'schedule',
+      updateState: true, // Always update state for scheduled runs
+    });
+    log.info('Scheduled extraction completed', {
+      jobId,
+      success: result.success,
+      recordCount: result.recordCount,
+      errorCategory: result.errorCategory,
+    });
+    return result;
+  } catch (error: any) {
+    log.error('Scheduled extraction failed', {
+      jobId,
+      message: error instanceof Error ? error.message : String(error),
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'Error',
+    });
+    throw error;
+  }
+}
+/**
+ * Handle ad-hoc extraction workflow
+ *
+ * Allows custom date ranges and optional state updates
+ */
+async function handleAdhocExtraction(ctx: Context): Promise<{ success: boolean; jobId: string; message: string; statusEndpoint: string; note: string }> {
+  const { data, log } = ctx;
+  const payload = data as AdhocExtractionPayload;
+  const jobId = generateJobId('ADHOC', 'FULFILLMENTS');
+  log.info('🚀 [WEBHOOK] Adhoc fulfillments extraction triggered', {
+    jobId,
+    fromDate: payload.fromDate || 'lastRunTime',
+    toDate: payload.toDate || 'now',
+    updateState: payload.updateState || false,
+  });
+  // ✅ Fire-and-forget: Start background processing WITHOUT await
+  // The promise continues execution after we return the response
+  executeFulfillmentExtraction(ctx, {
+      jobId,
+      triggeredBy: 'webhook',
+      fromDate: payload.fromDate,
+      toDate: payload.toDate,
+      updateState: payload.updateState === true,
+  })
+    .then((result) => {
+      log.info('✅ [BACKGROUND] Fulfillments extraction completed successfully', {
+      jobId,
+      recordCount: result.recordCount,
+        fileName: result.fileName,
+      errorCategory: result.errorCategory,
+    });
+    })
+    .catch((error: unknown) => {
+      const errorMessage = error instanceof Error ? error.message : String(error);
+      log.error('❌ [BACKGROUND] Fulfillments extraction failed', {
+      jobId,
+        error: errorMessage,
+      stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : typeof error,
+      });
+    });
+  // Return immediately with jobId (response sent with this return value)
+    return {
+    success: true,
+      jobId,
+    message: 'Fulfillments extraction started in background',
+    statusEndpoint: `https://{workspace}.versori.run/fulfillments-job-status`,
+    note: 'Poll the status endpoint with jobId to check progress',
+    };
+}
+/**
+ * Handle job status query workflow
+ *
+ * Returns current job state from KV storage
+ */
+async function handleJobStatusQuery(ctx: Context): Promise<any> {
+  const { data, log, openKv } = ctx;
+  const payload = data as JobStatusPayload;
+  if (!payload.jobId) {
+    return {
+      success: false,
+      error: 'Job ID required',
+      message: 'Provide jobId in request payload',
+    };
+  }
+  log.info('Job status query', { jobId: payload.jobId });
+  const status = await queryJobStatus(openKv(':project:'), payload.jobId, log);
+  if (!status) {
+    return {
+      success: false,
+      error: 'Job not found',
+      jobId: payload.jobId,
+      message: 'Job may have been deleted or never existed',
+    };
+  }
+  return {
+    success: true,
+    jobId: payload.jobId,
+    ...status,
+  };
+}
+```
+---
+## 📄 src/services/extraction-orchestration.ts
+```typescript
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  XMLBuilder,
+  SftpDataSource,
+} from '@fluentcommerce/fc-connect-sdk';
+import mappingConfig from '../../config/fulfillments.export.xml.json' with { type: 'json' };
+import { FULFILLMENTS_QUERY } from '../config/queries';
+type ErrorCategory = 'CONFIG' | 'API' | 'SFTP' | 'NO_DATA';
+interface TimeWindow {
+  from: string;
+  to: string;
+  bufferedFrom: string;
+}
+interface ExtractionMetrics {
+  totalPages: number;
+  validRecords: number;
+  skippedRecords: number;
+  xmlSizeKb: number;
+  window: TimeWindow;
+}
+export interface FulfillmentExtractionParams {
+  jobId: string;
+  triggeredBy: 'schedule' | 'webhook';
+  fromDate?: string;
+  toDate?: string;
+  updateState: boolean;
+  fulfillmentStatuses?: string[];
+}
+export interface FulfillmentExtractionResult {
+  success: boolean;
+  jobId: string;
+  recordCount: number;
+  fileName?: string;
+  sftpPath?: string;
+  message?: string;
+  error?: string;
+  errorCategory?: ErrorCategory;
+  metrics?: ExtractionMetrics;
+}
+interface FulfillmentLineItem {
+  product_ref: string;
+  requested_quantity: number | string;
+  filled_quantity: number | string;
+}
+const STATE_KEY = 'lastFulfillmentSync';
+const DEFAULT_FALLBACK = '2024-01-01T00:00:00Z';
+const STATUS_DELIMITER = ',';
+function normalizeStatuses(input?: string[] | string | null): string[] | undefined {
+  if (!input) {
+    return undefined;
+  }
+  const raw = Array.isArray(input) ? input : input.split(STATUS_DELIMITER);
+  const cleaned = raw.map(status => status.trim()).filter(Boolean);
+  return cleaned.length > 0 ? cleaned : undefined;
+}
+function joinSftpPath(
+  requireAbsolutePath: boolean,
+  ...segments: Array<string | undefined>
+): string {
+  const sanitized = segments
+    .filter(Boolean)
+    .map(segment => String(segment).replace(/^\/+|\/+$/g, ''))
+    .join('/');
+  if (!sanitized) {
+    return requireAbsolutePath ? '/' : '';
+  }
+  return requireAbsolutePath && !sanitized.startsWith('/') ? `/${sanitized}` : sanitized;
+}
+export async function queryJobStatus(kv: any, jobId: string, log: any): Promise<any | undefined> {
+  try {
+    const tracker = new JobTracker(kv, log);
+    return await tracker.getJob(jobId);
+  } catch (error: any) {
+    log.error('Failed to get job status', {
+      jobId,
+      message: error instanceof Error ? error.message : String(error),
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'Error',
+    });
+    return undefined;
+  }
+}
+export async function executeFulfillmentExtraction(
+  ctx: any,
+  params: FulfillmentExtractionParams
+): Promise<FulfillmentExtractionResult> {
+  const { log, openKv, activation } = ctx;
+  const { jobId, triggeredBy, fromDate, toDate, updateState } = params;
+  const kv = openKv(':project:');
+  const tracker = new JobTracker(kv, log);
+  // Track execution duration
+  const executionStartTime = Date.now();
+  const overlapBufferSeconds = parseInt(activation.getVariable('overlapBufferSeconds') || '60', 10);
+  const fallbackStartDate = activation.getVariable('fallbackStartDate') || DEFAULT_FALLBACK;
+  const isManualOverride = Boolean(fromDate);
+  const shouldUpdateState = updateState === true;
+  try {
+    log.info('🚀 [STEP 1/8] Initializing job tracking', { jobId, triggeredBy });
+    await tracker.createJob(jobId, {
+      triggeredBy,
+      hasDateOverride: isManualOverride,
+      updateStateAfterRun: shouldUpdateState,
+      fromDate,
+      toDate,
+    });
+    log.info('📡 [STEP 2/8] Initializing Fluent Commerce client', { jobId });
+    const client = await createClient(ctx, { validateConnection: true });
+    if (!client) {
+      throw new Error('Failed to create Fluent Commerce client');
+    }
+    log.info('📅 [STEP 3/8] Determining date range for extraction', { jobId });
+    let rawState = fallbackStartDate;
+    if (!isManualOverride) {
+      try {
+        const stored = await kv.get(STATE_KEY);
+        if (typeof stored === 'string') {
+          rawState = stored;
+        } else if (stored?.timestamp) {
+          rawState = stored.timestamp;
+        }
+      } catch {
+        rawState = fallbackStartDate;
+      }
+    }
+    const overlapMs = overlapBufferSeconds * 1000;
+    const queryFrom = isManualOverride
+      ? fromDate!
+      : new Date(Math.max(0, new Date(rawState).getTime() - overlapMs)).toISOString();
+    const windowStart = isManualOverride ? (fromDate ?? queryFrom) : rawState;
+    const windowEnd = toDate || new Date().toISOString();
+    const statusesInput =
+      params.fulfillmentStatuses ?? activation.getVariable('fulfillmentStatuses');
+    const fulfillmentStatuses = normalizeStatuses(statusesInput);
+    await tracker.updateJob(jobId, {
+      stage: 'extraction',
+      message: 'Extracting fulfillments with auto-pagination',
+      status: 'processing',
+      statuses: fulfillmentStatuses,
+    });
+    const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
+    const maxRecords = parseInt(activation.getVariable('maxRecords') || '50000', 10);
+    log.info('📊 [STEP 4/8] Extracting fulfillments via GraphQL', {
+      jobId,
+      pageSize,
+      maxRecords,
+      queryFrom,
+      windowEnd,
+      statuses: fulfillmentStatuses,
+    });
+    const orchestrator = new ExtractionOrchestrator(client, log);
+    // Extract context for progress logging
+    const dateRangeInfo = {
+      start: queryFrom || 'N/A',
+      end: windowEnd || 'N/A',
+      statuses: fulfillmentStatuses?.join(', ') || 'all'
+    };
+    // Start logging with context
+    log.info(`📥 [ExtractionOrchestrator] Starting extraction`, {
+     query: 'fulfillments',
+     pageSize,
+     maxRecords,
+     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
+     statuses: dateRangeInfo.statuses,
+     jobId
+    });
+    const extractionResult = await orchestrator.extract({
+      query: FULFILLMENTS_QUERY,
+      resultPath: 'fulfillments.edges.node',
+      variables: {
+        statuses: fulfillmentStatuses,
+        updatedFrom: queryFrom,
+        updatedTo: windowEnd,
+      },
+      pageSize,
+      maxRecords,
+      validateItem: (item: any) => Boolean(item?.ref && item?.order?.ref),
+    });
+    const rawRecords = extractionResult.data || [];
+    const extractionErrors = extractionResult.errors || [];
+    const totalPages = extractionResult.stats.totalPages ?? 0;
+    log.info('✅ [STEP 4/8] Extraction complete', {
+      jobId,
+      totalPages,
+      totalRecords: extractionResult.stats.totalRecords,
+      validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
+      truncated: extractionResult.stats.truncated ?? false,
+      errorCount: extractionErrors.length,
+    });
+    // Completion logging with summary
+    log.info(`✅ [ExtractionOrchestrator] Extraction completed`, {
+     totalRecords: extractionResult.stats.totalRecords,
+     totalPages,
+     validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
+     failedValidations: extractionResult.stats.failedValidations,
+     truncated: extractionResult.stats.truncated,
+     truncationReason: extractionResult.stats.truncationReason,
+     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
+     jobId
+    });
+    if (extractionErrors.length > 0) {
+      log.warn('[STEP 4/8] Non-fatal extraction errors detected', {
+        jobId,
+        sampleErrors: extractionErrors.slice(0, 3),
+      });
+    }
+    if (rawRecords.length === 0) {
+      log.info('No fulfillments to process', { jobId });
+      if (shouldUpdateState) {
+        await kv.set(STATE_KEY, {
+          timestamp: windowEnd,
+          updatedAt: new Date().toISOString(),
+          recordCount: 0,
+          triggeredBy,
+        });
+      }
+      await tracker.markCompleted(jobId, {
+        recordCount: 0,
+        message: 'No fulfillments to extract',
+        stateUpdated: shouldUpdateState,
+        window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
+      });
+      return {
+        success: true,
+        jobId,
+        recordCount: 0,
+        message: 'No fulfillments to extract',
+        metrics: {
+          totalPages,
+          validRecords: 0,
+          skippedRecords: 0,
+          xmlSizeKb: 0,
+          window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
+        },
+      };
+    }
+    log.info('🔄 [STEP 5/8] Transforming data with UniversalMapper', {
+      jobId,
+      recordCount: rawRecords.length,
+    });
+    await tracker.updateJob(jobId, {
+      stage: 'transformation',
+      message: `Transforming ${rawRecords.length} fulfillments`,
+      status: 'processing',
+    });
+    const mapper = new UniversalMapper(mappingConfig);
+    const mappingResult = await mapper.map(rawRecords);
+    const mappingErrors = mappingResult.errors || [];
+    if (!mappingResult.success) {
+      log.error('[STEP 5/8] Transformation failed', {
+        jobId,
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+      await tracker.markFailed(jobId, 'UniversalMapper returned unsuccessful result', {
+        failedCount: mappingErrors.length,
+        errors: mappingErrors,
+      });
+      return {
+        success: false,
+        jobId,
+        recordCount: 0,
+        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
+        errorCategory: 'CONFIG',
+      };
+    }
+    const headerRecords: any[] = Array.isArray(mappingResult.data) ? mappingResult.data : [];
+    const mappingErrors = mappingResult.errors || [];
+    if (mappingErrors.length > 0) {
+      log.warn('[STEP 5/8] Some records failed transformation', {
+        jobId,
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+    }
+    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
+      log.info('ℹ️ [MAPPING] Optional fields skipped (undefined values)', {
+        jobId,
+        skippedFields: mappingResult.skippedFields,
+        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
+      });
+    }
+    const transformedRecords = headerRecords.map((record, index) => {
+      const raw = rawRecords[index] ?? {};
+      const lineItems: FulfillmentLineItem[] = ((raw.items?.edges ?? []) as any[])
+        .map(edge => edge?.node)
+        .filter(Boolean)
+        .map((item: any) => ({
+          product_ref: item.productRef ?? '',
+          requested_quantity: item.requestedQuantity ?? '',
+          filled_quantity: item.filledQuantity ?? '',
+        }));
+      return {
+        ...record,
+        line_items: lineItems,
+      };
+    });
+    if (transformedRecords.length === 0) {
+      await tracker.markFailed(jobId, 'All fulfillments failed mapping', {
+        failedCount: mappingErrors.length,
+        errors: mappingErrors,
+      });
+      return {
+        success: false,
+        jobId,
+        recordCount: 0,
+        error: 'All records failed mapping',
+        errorCategory: 'CONFIG',
+        metrics: {
+          totalPages,
+          validRecords: 0,
+          skippedRecords: rawRecords.length,
+          xmlSizeKb: 0,
+          window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
+        },
+      };
+    }
+    if (mappingErrors.length > 0) {
+      log.warn('[STEP 5/8] Some fulfillments failed transformation', {
+        jobId,
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+    }
+    const totalLineItems = transformedRecords.reduce(
+      (sum, record) => sum + record.line_items.length,
+      0
+    );
+    log.info('📝 [STEP 6/8] Generating XML document', {
+      jobId,
+      fulfillmentCount: transformedRecords.length,
+      lineItemCount: totalLineItems,
+    });
+    await tracker.updateJob(jobId, {
+      stage: 'xml_generation',
+      message: `Generating XML for ${transformedRecords.length} fulfillments`,
+      status: 'processing',
+    });
+    const xmlBuilder = new XMLBuilder({
+      prettyPrint: true,
+      indent: '  ',
+      xmlDeclaration: true,
+      encoding: 'UTF-8',
+      attributePrefix: '@',
+      textKey: '#text',
+    });
+    const xmlData = {
+      Fulfillments: {
+        Fulfillment: transformedRecords.map(record => {
+          const lineItemNodes =
+            record.line_items.length > 0
+              ? {
+                  LineItem: record.line_items.map(item => ({
+                    product_ref: item.product_ref,
+                    requested_quantity: String(item.requested_quantity ?? ''),
+                    filled_quantity: String(item.filled_quantity ?? ''),
+                  })),
+                }
+              : undefined;
+          return {
+            fulfillment_ref: record.fulfillment_ref,
+            order_ref: record.order_ref,
+            status: record.status,
+            delivery_type: record.delivery_type ?? '',
+            from_location_ref: record.from_location_ref ?? '',
+            from_location_name: record.from_location_name ?? '',
+            created_on: record.created_on,
+            updated_on: record.updated_on,
+            ...(lineItemNodes ? { LineItems: lineItemNodes } : {}),
+          };
+        }),
+      },
+    };
+    const xmlString = xmlBuilder.build(xmlData, 'Fulfillments');
+    const xmlBuffer = Buffer.from(xmlString, 'utf8');
+    const xmlSizeKb = Math.round(xmlBuffer.length / 1024);
+    const fileNamePrefix = activation.getVariable('fileNamePrefix') || 'fulfillments';
+    const timestampSuffix = new Date().toISOString().replace(/[:.]/g, '-');
+    const fileName = `${fileNamePrefix}-${timestampSuffix}.xml`;
+    log.info('📤 [STEP 7/8] Preparing SFTP upload', { jobId, fileName });
+    await tracker.updateJob(jobId, {
+      stage: 'sftp_upload',
+      message: `Uploading ${fileName} to SFTP`,
+      status: 'processing',
+    });
+    const sftpHost = activation.getVariable('sftpHost');
+    const sftpPort = parseInt(activation.getVariable('sftpPort') || '22', 10);
+    const sftpRemotePath = activation.getVariable('sftpRemotePath') || '/incoming/fulfillments/';
+    const sftpPrivateKey = activation.getVariable('sftpPrivateKey');
+    const requireAbsolutePaths = activation.getVariable('requireAbsolutePaths') === 'true';
+    if (!sftpHost) {
+      throw new Error('SFTP configuration incomplete: sftpHost activation variable missing');
+    }
+    const allConnections = activation.connections || [];
+    const sftpConnection = allConnections.find((conn: any) => conn.name === 'sftp_export_server');
+    if (!sftpConnection) {
+      throw new Error('SFTP connection "sftp_export_server" not found');
+    }
+    const credential = sftpConnection.credentials?.[0]?.credential;
+    if (!credential?.data?.basicAuth) {
+      throw new Error('SFTP connection not configured with Basic Authentication');
+    }
+    const { username, password } = credential.data.basicAuth;
+    const sftp = new SftpDataSource(
+      {
+        type: 'SFTP_XML',
+        connectionId: 'sftp_export_server',
+        name: 'Fulfillments XML Export',
+        settings: {
+          host: sftpHost,
+          port: sftpPort,
+          username,
+          password,
+          privateKey: sftpPrivateKey,
+          remotePath: sftpRemotePath,
+          filePattern: '*.xml',
+          requireAbsolutePaths,
+        },
+      },
+      log
+    );
+    const fullPath = joinSftpPath(requireAbsolutePaths, sftpRemotePath, fileName);
+    try {
+      await sftp.uploadFile(fullPath, xmlBuffer, {
+        createDirectories: true,
+        overwrite: true,
+      });
+      if (mappingErrors.length > 0) {
+        const reportPayload = {
+          fileName,
+          processedAt: new Date().toISOString(),
+          failedRecords: mappingErrors.length,
+          sampleErrors: mappingErrors.slice(0, 10),
+        };
+        const reportPath = joinSftpPath(
+          requireAbsolutePaths,
+          sftpRemotePath,
+          `${fileName.replace(/\.xml$/i, '')}-errors.json`
+        );
+        await sftp.uploadFile(
+          reportPath,
+          Buffer.from(JSON.stringify(reportPayload, null, 2), 'utf8'),
+          {
+            createDirectories: true,
+            overwrite: true,
+          }
+        );
+        log.warn('Uploaded transformation error report', { jobId, reportPath });
+      }
+    } finally {
+      await sftp.dispose();
+      log.info('SFTP connection disposed', { jobId });
+    }
+    log.info('💾 [STEP 8/8] Updating state and completing job', { jobId });
+    const newTimestamp = (() => {
+      const maxUpdated = rawRecords.reduce((max, record) => {
+        const current = new Date(record.updatedOn || windowEnd).getTime();
+        return current > max ? current : max;
+      }, new Date(windowStart).getTime());
+      return new Date(maxUpdated).toISOString();
+    })();
+    if (shouldUpdateState) {
+      await kv.set(STATE_KEY, {
+        timestamp: newTimestamp,
+        updatedAt: new Date().toISOString(),
+        recordCount: transformedRecords.length,
+        lineItemCount: totalLineItems,
+        fileName,
+        sftpPath: fullPath,
+        statuses: fulfillmentStatuses ?? [],
+      });
+    }
+    const executionDurationMs = Date.now() - executionStartTime;
+    await tracker.markCompleted(jobId, {
+      recordCount: transformedRecords.length,
+      lineItemCount: totalLineItems,
+      fileName,
+      remotePath: fullPath,
+      errorCount: mappingErrors.length,
+      errors: mappingErrors,
+      stateUpdated: shouldUpdateState,
+      window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
+      durationMs: executionDurationMs,
+    });
+    log.info('✅ Extraction completed successfully', {
+      jobId,
+      recordCount: transformedRecords.length,
+      durationMs: executionDurationMs,
+      durationSeconds: (executionDurationMs / 1000).toFixed(2),
+    });
+    return {
+      success: true,
+      jobId,
+      recordCount: transformedRecords.length,
+      fileName,
+      sftpPath: fullPath,
+      message: `Uploaded ${transformedRecords.length} fulfillments`,
+      metrics: {
+        totalPages,
+        validRecords: transformedRecords.length,
+        skippedRecords: mappingErrors.length,
+        xmlSizeKb,
+        window: { from: windowStart, to: windowEnd, bufferedFrom: queryFrom },
+      },
+    };
+  } catch (error: any) {
+    const errorMessage = error instanceof Error ? error.message : String(error);
+    log.error('❌ Fulfillment extraction failed', {
+      jobId,
+      message: errorMessage,
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'Error',
+    });
+    // Determine error category and recommendation
+    let errorCategory: ErrorCategory = 'API';
+    let recommendation = '';
+    if (errorMessage.includes('SFTP') || errorMessage.includes('connection')) {
+      errorCategory = 'SFTP';
+      recommendation = 'Check SFTP credentials and network connectivity. Verify host, port, and authentication settings in activation variables.';
+    } else if (errorMessage.includes('mapping') || errorMessage.includes('transformation')) {
+      errorCategory = 'CONFIG';
+      recommendation = 'Review mapping configuration in config/fulfillments.export.xml.json. Verify all source paths match GraphQL response structure.';
+    } else if (errorMessage.includes('GraphQL') || errorMessage.includes('query')) {
+      errorCategory = 'API';
+      recommendation = 'Verify Fluent Commerce API credentials and GraphQL query syntax. Check if the query matches the current schema.';
+    } else if (errorMessage.includes('No records') || errorMessage.includes('empty')) {
+      errorCategory = 'NO_DATA';
+      recommendation = 'No fulfillments found in the specified date range. This may be expected if there are no recent updates.';
+    }
+    await tracker.markFailed(jobId, {
+      error: errorMessage,
+      errorCategory,
+      recommendation,
+      triggeredBy,
+    });
+    return {
+      success: false,
+      jobId,
+      recordCount: 0,
+      error: errorMessage,
+      errorCategory,
+      details: {
+        recommendation,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',
+      },
+    };
+  }
+}
+```
+---
+## 📄 src/config/queries.ts
+```typescript
+// ============================================================================
+// GRAPHQL QUERIES AND MAPPING CONFIGURATION
+// ============================================================================
+import mappingConfig from '../../config/fulfillments.export.xml.json' with { type: 'json' };
+/**
+ * GraphQL query for fulfillments extraction
+ *
+ * Supports:
+ * - Pagination (first, after)
+ * - Date range filtering (updatedOn) - UTC timestamps required
+ * - Status filtering (statuses)
+ */
+export const FULFILLMENTS_QUERY = `
+  query GetFulfillments(
+    $statuses: [String!]
+    $updatedFrom: DateTime
+    $updatedTo: DateTime
+    $first: Int!
+    $after: String
+  ) {
+    fulfillments(
+      status: $statuses
+      updatedOn: { from: $updatedFrom, to: $updatedTo }
+      first: $first
+      after: $after
+    ) {
+      edges {
+        node {
+          id
+          ref
+          status
+          deliveryType
+          order {
+            ref
+          }
+          fromLocation {
+            ref
+            name
+          }
+          items {
+            edges {
+              node {
+                productRef
+                requestedQuantity
+                filledQuantity
+              }
+            }
+          }
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+        # Note: Fluent doesn't return endCursor - cursors are in edges[].cursor
+      }
+    }
+  }
+`;
+/**
+ * Field mapping configuration loaded from JSON file
+ *
+ * Location: config/fulfillments.export.xml.json
+ */
+export const FULFILLMENTS_EXPORT_MAPPING = mappingConfig;
+```
+---
+## Testing
+### Test Scheduled Extraction
+The scheduled workflow runs automatically every 4 hours.
+**Check logs:**
+```
+[STEP 1/10] Loading and validating configuration
+[STEP 2/10] Initializing job tracker
+[STEP 3/10] Calculating extraction time window
+[STEP 4/10] Initializing SDK services
+[STEP 5/10] Extracting fulfillments from Fluent
+[STEP 6/10] Transforming records with UniversalMapper
+[STEP 7/10] Generating XML file
+[STEP 8/10] Uploading XML to SFTP
+[STEP 9/10] Updating extraction state
+[STEP 10/10] Finalizing job
+```
+### Test Ad-hoc Extraction
+```bash
+# Incremental (uses last sync timestamp)
+curl -X POST https://api.versori.com/webhooks/fulfillments-adhoc \
+  -H "Content-Type: application/json" \
+  -d '{}'
+# Date range override
+curl -X POST https://api.versori.com/webhooks/fulfillments-adhoc \
+  -H "Content-Type: application/json" \
+  -d '{
+    "fromDate": "2025-01-01T00:00:00Z",
+    "toDate": "2025-01-31T23:59:59Z",
+    "updateState": false
+  }'
+```
+### Test Job Status Query
+```bash
+curl -X POST https://api.versori.com/webhooks/fulfillments-job-status \
+  -H "Content-Type: application/json" \
+  -d '{
+    "jobId": "ADHOC_FULFILLMENTS_20251027_183045_abc123"
+  }'
+```
+**Response:**
+```json
+{
+  "success": true,
+  "jobId": "ADHOC_FULFILLMENTS_20251027_183045_abc123",
+  "status": "processing",
+  "stage": "transformation",
+  "message": "Transforming 150 records",
+  "createdAt": "2025-10-27T18:30:45.000Z",
+  "startedAt": "2025-10-27T18:30:46.000Z"
+}
+```
+---
+## Next Steps
+1. **Review Configuration:** Verify all activation variables match your environment
+2. **Test with Sample Data:** Run ad-hoc extraction with small date range
+3. **Validate XML Output:** Check SFTP server for correctly formatted files
+4. **Enable Schedule:** Activate 4-hourly schedule after successful test
+5. **Monitor Jobs:** Use job status webhook to track extraction progress
+6. **Adjust Filters:** Customize status filters and date ranges as needed
+7. **Scale:** Increase page size and max records based on data volume
+---
+## Testing Checklist
+**Before production deployment:**
+### 1. Schema Validation
+- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
+- [ ] Run `npx fc-connect validate-schema --mapping ./config/fulfillments.export.xml.json --schema ./fluent-schema.json`
+- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/fulfillments.export.xml.json --schema ./fluent-schema.json`
+- [ ] Verify all `source` paths in mapping exist in GraphQL schema
+- [ ] Verify query structure matches schema (fields, types, filters)
+### 2. Extraction Testing
+- [ ] Test with small dataset first (maxRecords=10)
+- [ ] Verify ExtractionOrchestrator pagination works correctly
+- [ ] Test with multiple pages of data (verify cursor handling)
+- [ ] Verify date range filtering (updatedOn filter)
+- [ ] Test empty result handling (no records in date range)
+- [ ] Verify extraction stops at maxRecords limit
+### 3. Mapping Testing
+- [ ] Verify required fields are populated
+- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
+- [ ] Test custom resolvers with edge cases (if any)
+- [ ] Verify nested field extraction
+- [ ] Test with null/missing fields
+- [ ] Verify mapping error collection works
+### 4. XML Generation Testing
+- [ ] Verify XML structure matches expected format
+- [ ] Test XML validation against XSD schema (if applicable)
+- [ ] Verify special character escaping in XML
+- [ ] Test with large datasets (>1000 records)
+- [ ] Verify UTF-8 encoding
+- [ ] Test XML namespace handling (if applicable)
+### 5. SFTP Upload Testing
+- [ ] Test SFTP connection and authentication
+- [ ] Verify file upload to correct path
+- [ ] Test file naming convention (timestamp format)
+- [ ] Verify file permissions on SFTP server
+- [ ] Test upload retry logic (simulate network failure)
+- [ ] Verify SFTP connection disposal (no connection leaks)
+### 6. State Management Testing
+- [ ] Verify overlap buffer prevents missed records (60-second default)
+- [ ] Test state recovery after extraction failure
+- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
+- [ ] Test first run with no previous state (uses fallbackStartDate)
+- [ ] Verify state update only happens on successful upload
+- [ ] Test manual date override (doesn't update state)
+### 7. Job Tracking Testing
+- [ ] Test job creation with JobTracker
+- [ ] Verify job status updates at each stage
+- [ ] Test job completion with metadata
+- [ ] Test job failure handling
+- [ ] Query job status via webhook endpoint
+- [ ] Verify job status persists in KV store
+### 8. Error Handling Testing
+- [ ] Test with invalid GraphQL query
+- [ ] Test with mapping errors (invalid field paths)
+- [ ] Test with SFTP connection failures
+- [ ] Test with authentication failures
+- [ ] Test with network timeouts
+- [ ] Verify error logging includes context (jobId, stage, error details)
+- [ ] Test error threshold logic (if applicable)
+### 9. Staging Environment Testing
+- [ ] Run full extraction in staging environment
+- [ ] Verify XML file format with downstream system
+- [ ] Monitor extraction duration and resource usage
+- [ ] Test with production-like data volumes
+- [ ] Verify no performance degradation over time
+### 10. Integration Testing
+- [ ] Test scheduled workflow (cron trigger)
+- [ ] Test ad hoc webhook trigger
+- [ ] Test job status query webhook
+- [ ] Verify activation variables are read correctly
+- [ ] Test with different extraction modes (incremental, date range)
+- [ ] End-to-end test: trigger → extract → transform → upload → verify file
+---
+## Monitoring & Alerting
+### Success Response Example
+```json
+{
+  "success": true,
+  "jobId": "SCHEDULED_FUL_20251102_140000_abc123",
+  "recordsExtracted": 1523,
+  "fileName": "fulfillments-2025-11-02T14-00-00-000Z.xml",
+  "sftpPath": "/outbound/fulfillments/fulfillments-2025-11-02T14-00-00-000Z.xml",
+  "metrics": {
+    "extractionDurationMs": 12543,
+    "totalPages": 8,
+    "pageSize": 200,
+    "mappingErrors": 0,
+    "fileSizeBytes": 524288,
+    "uploadDurationMs": 1234
+  },
+  "timestamps": {
+    "extractionStart": "2025-11-02T14:00:00.000Z",
+    "extractionEnd": "2025-11-02T14:00:12.543Z",
+    "uploadComplete": "2025-11-02T14:00:13.777Z"
+  },
+  "state": {
+    "previousTimestamp": "2025-11-02T13:00:00.000Z",
+    "newTimestamp": "2025-11-02T13:59:58.123Z",
+    "stateUpdated": true,
+    "overlapBufferSeconds": 60
+  }
+}
+```
+### Error Response Example
+```json
+{
+  "success": false,
+  "jobId": "ADHOC_FUL_20251102_140500_xyz789",
+  "error": "SFTP upload failed: Connection timeout",
+  "errorCategory": "NETWORK",
+  "recordsExtracted": 0,
+  "stage": "sftp_upload",
+  "details": {
+    "message": "Failed to upload file after 3 retry attempts",
+    "retryAttempts": 3,
+    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
+  },
+  "state": {
+    "stateUpdated": false,
+    "willRetryNextRun": true,
+    "note": "State not advanced - next extraction will retry same time window"
+  }
+}
+```
+### Key Metrics to Track
+```typescript
+const METRICS = {
+  // Extraction Performance
+  extractionDurationMs: Date.now() - extractionStart,
+  recordCount: records.length,
+  pageCount: extractionResult.stats.totalPages,
+  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
+  // Transformation Performance
+  transformedCount: transformedRecords.length,
+  failedCount: mappingErrors.length,
+  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
+  // File Generation
+  fileSizeMB: (xmlContent.length / (1024 * 1024)).toFixed(2),
+  // Upload Performance
+  uploadDurationMs: uploadEnd - uploadStart,
+  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
+  // State Management
+  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
+  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
+};
+log.info('Extraction metrics', metrics);
+```
+### Alert Thresholds
+```typescript
+const ALERT_THRESHOLDS = {
+  // Duration Alerts
+  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
+  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
+  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
+  // Error Rate Alerts
+  MAX_ERROR_RATE: 0.05, // 5% mapping errors
+  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
+  // Volume Alerts
+  MAX_RECORDS_PER_RUN: 100000,
+  MIN_RECORDS_WARNING: 0, // Alert if no records found
+  MAX_FILE_SIZE_MB: 150, // 150MB
+  // State Alerts
+  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
+  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
+};
+// Check thresholds
+if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
+  log.warn('Extraction duration exceeded threshold', {
+    duration: metrics.extractionDurationMs,
+    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
+    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
+  });
+}
+```
+### Monitoring Dashboard Queries
+**Versori Platform Logs Query:**
+```
+# Successful extractions
+log_level:info AND message:"Extraction complete" AND jobId:*
+# Failed extractions
+log_level:error AND message:"Extraction workflow failed" AND jobId:*
+# Performance issues
+extractionDurationMs:>300000 OR uploadDurationMs:>120000
+# High error rates
+errorRate:>5
+# State management issues
+stateUpdated:false AND success:true
+```
+### Common Issues and Solutions
+**Issue**: "Extraction timeout after 10 minutes"
+- **Cause**: Too many records in single extraction
+- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
+- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
+**Issue**: "Mapping errors for 50% of records"
+- **Cause**: Schema mismatch between GraphQL response and mapping config
+- **Fix**: Run schema validation, update mapping config paths
+- **Prevention**: Use `npx fc-connect validate-schema` before deployment
+**Issue**: "SFTP connection timeout"
+- **Cause**: Network issues, firewall, or connection pool exhaustion
+- **Fix**: Check SFTP credentials, verify network connectivity
+- **Prevention**: Implement connection health checks, monitor connection status
+**Issue**: "State not updating after successful extraction"
+- **Cause**: KV write failure or intentional retry logic
+- **Fix**: Check KV logs, verify state update code executed
+- **Prevention**: Add KV write verification, log state updates explicitly
+**Issue**: "First run exceeds record limits"
+- **Cause**: No previous timestamp, fetches all historical records
+- **Fix**: Set fallbackStartDate close to current date, apply additional filters
+- **Prevention**: Use appropriate fallbackStartDate for initial runs
+**Issue**: "Excessive duplicate records in output"
+- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
+- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
+- **Prevention**: Monitor duplicate rates, verify state update logic
+---
+## Troubleshooting Quick Reference
+| Error Message | Likely Cause | Solution |
+|--------------|--------------|----------|
+| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
+| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
+| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
+| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
+| "SFTP authentication failed" | Invalid credentials | Verify SFTP credentials in activation variables |
+| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
+| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
+| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
+| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
+| "XML generation failed" | Format-specific error | Check XML generation logic, validate output |
+---