@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-inventory-positions-to-s3-csv.md

@@ -1,2461 +1,2461 @@
----
-template_id: tpl-extract-inventory-positions-to-s3-csv
-canonical_filename: template-extraction-inventory-positions-to-s3-csv.md
-version: 2.0.0
-sdk_version: ^0.1.39
-runtime: versori
-direction: extraction
-source: fluent-graphql
-destination: s3-csv
-entity: inventory-positions
-format: csv
-logging: versori
-status: stable
-features:
-  - memory-management
-  - enhanced-logging
-  - pagination-progress
----
-
-# Template: Extraction - Inventory Positions to S3 CSV
-
-**Template Version:** 2.0.0
-**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
-**Last Updated:** 2025-01-24
-**Deployment Target:** Versori Platform
-
-**🆕 Version 2.0.0 Enhancements:**
-- ✅ **Memory Management** - Clear large result sets after processing batches
-- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
-- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
-
----
-
-## 📚 STEP 1: Load These Docs (Human Checklist)
-
-1. REQUIRED (load all)
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-
-Copy-paste list (open these):
-fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-
----
-
-## 📋 STEP 2: Tell Your AI (Prompt)
-
-Copy/paste this prompt into your AI tool after loading the documentation above:
-
-```
-I need a Versori scheduled extractor that:
-
-1) Queries Fluent Commerce GraphQL for Inventory Positions with auto-pagination
-2) Supports incremental runs via KV state (with an overlap buffer)
-3) Transforms results using UniversalMapper per mapping JSON
-4) Generates CSV and uploads to S3
-5) Tracks progress with JobTracker and exposes a job-status webhook
-6) Uses native Versori log (LoggingService removed - use native log)
-
-Use the loaded docs to fill in SDK specifics and best practices.
-Keep the structure identical to the template; only adapt where needed.
-```
-
----
-
-## 📋 Template Overview
-
-This connector runs on the Versori platform. Most operational settings (Fluent account/connection, S3 credentials, schedule, page size/limits) are configured via activation variables. Data shape and logic (mapping JSON, CSV structure, GraphQL selection set/filters, validators/resolvers) are adjusted in code as needed. It extracts inventory positions from Fluent Commerce via GraphQL, transforms the data into CSV, and uploads the result to S3.
-
-### What This Template Does
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│                    EXTRACTION WORKFLOW                           │
-└─────────────────────────────────────────────────────────────────┘
-
-1. TRIGGER
-   ├─ Scheduled (Cron): Runs automatically every hour
-   ├─ Ad hoc (Webhook): Manual trigger with optional date override
-   └─ Status Query (Webhook): Check job progress
-
-2. EXTRACT (ExtractionOrchestrator)
-   ├─ Query Fluent GraphQL API for inventory positions
-   ├─ Auto-pagination (handles large datasets)
-   ├─ Apply date filters (incremental or manual range)
-   └─ Validate each record (optional)
-
-3. TRANSFORM (UniversalMapper)
-   ├─ Map GraphQL fields to CSV schema
-   ├─ Apply SDK resolvers (trim, uppercase, parseInt, etc.)
-   ├─ Extract nested data (catalogue)
-   └─ Handle transformation errors
-
-4. GENERATE CSV (CSVParserService)
-   ├─ Convert transformed records to CSV
-   ├─ Include headers
-   ├─ Handle special characters
-   └─ Generate timestamped filename
-
-5. UPLOAD (S3DataSource)
-   ├─ Connect to S3
-   ├─ Upload CSV file with retry logic
-   ├─ Set content type and metadata
-   └─ Verify upload success
-
-6. TRACK JOB (JobTracker)
-   ├─ Create job with unique ID
-   ├─ Update status at each step
-   ├─ Store job result in KV
-   └─ Enable status queries via webhook
-
-7. UPDATE STATE (VersoriKVAdapter)
-   ├─ Calculate max updatedOn from records
-   ├─ Store timestamp for next incremental run
-   ├─ Apply overlap buffer (prevent missed records)
-   └─ Skip update if manual date override
-```
-
-### Key Features
-
-- Job tracking with status queries
-- Execution modes: scheduled, ad hoc, status query
-- Uses ExtractionOrchestrator, UniversalMapper, JobTracker, CSVParserService
-- Error handling, retry logic
-- Reusable services suitable for similar use cases
-
-Note: JobTracker persists stage/status to Versori KV for visibility, job-status webhooks, and auditing. Recommended for production multi-step flows; can be skipped for trivial single-step utilities.
-
-### 📦 Package Information
-
-**SDK:** [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
-
-**Latest Version:** See [npm package page](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk) for current version
-
-```bash
-npm install @fluentcommerce/fc-connect-sdk
-```
-
----
-
-**Templates are designed for direct deployment; customize via activation variables.**
-
----
-
-## 📦 SDK Imports (Verified - Versori Optimized)
-
-```typescript
-// ✅ VERIFIED IMPORTS - These match actual SDK exports
-import { Buffer } from 'node:buffer';
-import {
-  createClient, // Universal client factory (auto-detects Versori/Node/Deno)
-  ExtractionOrchestrator, // High-level extraction with auto-pagination
-  JobTracker, // Job status tracking in KV store
-  UniversalMapper, // Field mapping with SDK resolvers
-  CSVParserService, // CSV generation (NOT CSVBuilder!)
-  S3DataSource, // S3 operations with retry logic
-} from '@fluentcommerce/fc-connect-sdk';
-
-// Versori platform imports
-import { schedule, webhook, http, fn } from '@versori/run';
-```
-
-**Note:** All imports are from actual SDK exports - this code compiles and runs as-is.
-
-**✅ VERSORI PLATFORM - Use Native Logs:**
-
-- Use `log` from context: `const { log } = ctx;`
-- Don't import or use LoggingService for Versori connectors
-- Native Versori logs are simpler and automatically integrated with platform monitoring
-
----
-
-## ⚙️ Activation Variables
-
-**Configuration is driven by activation variables - modify these instead of code:**
-
-```json
-{
-  "catalogueRef": "DEFAULT_CATALOGUE",
-  "s3BucketName": "inventory-analytics-exports",
-  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
-  "awsSecretAccessKey": "********",
-  "awsRegion": "us-east-1",
-  "s3Prefix": "inventory-positions/daily/",
-  "fileNamePrefix": "inventorypositions",
-  "pageSize": 200,
-  "maxRecords": 100000,
-  "overlapBufferSeconds": 60
-}
-```
-
-Note: Webhook security is enforced via Versori connections. Configure auth on the connection; reference it in `webhook({ connection: '...' })`.
-
-### Variable Explanations
-
-| Variable               | Purpose                       | Default                      | Customization Hints              |
-| ---------------------- | ----------------------------- | ---------------------------- | -------------------------------- |
-| `catalogueRef`         | Filter by inventory catalogue | `DEFAULT_CATALOGUE`          | Change to filter by catalogue    |
-| `s3BucketName`         | Target S3 bucket              | -                            | Required - your analytics bucket |
-| `awsAccessKeyId`       | AWS access key                | -                            | Required - AWS credential        |
-| `awsSecretAccessKey`   | AWS secret key                | -                            | Required - AWS credential        |
-| `awsRegion`            | AWS region                    | `us-east-1`                  | Match bucket region              |
-| `s3Prefix`             | S3 key prefix                 | `inventory-positions/daily/` | Customize folder structure       |
-| `fileNamePrefix`       | CSV filename prefix           | `inventorypositions`         | Customize naming convention      |
-| `pageSize`             | Records per GraphQL page      | `200`                        | Increase for fewer API calls     |
-| `maxRecords`           | Total extraction limit        | `100000`                     | Safety limit - adjust for volume |
-| `overlapBufferSeconds` | Incremental safety window     | `60`                         | Prevents missed records          |
-
----
-
-### 🔄 State Management & Incremental Sync
-
-**How incremental sync works:**
-
-1. **First Run:** Uses `DEFAULT_FALLBACK` date (2024-01-01T00:00:00Z)
-2. **Subsequent Runs:** Uses `lastInventoryPositionSync` timestamp from KV store
-3. **Overlap Buffer:** Subtracts 60 seconds to catch late-arriving records
-4. **State Update:** After successful upload, stores max `updatedOn` for next run
-
-**Incremental vs Manual Modes:**
-
-| Mode                        | When to Use          | State Update | Payload Example                                                |
-| --------------------------- | -------------------- | ------------ | -------------------------------------------------------------- |
-| **Incremental**             | Daily scheduled sync | ✅ Yes       | `{}` (empty - uses last sync)                                  |
-| **Manual Range**            | Historical backfill  | ❌ No        | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": false }` |
-| **Manual Range with State** | One-time catch-up    | ✅ Yes       | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": true }`  |
-
-**Why overlap buffer?**
-
-Records updated near the sync time might not appear in the query due to:
-
-- Clock drift between systems
-- Transaction timing in the database
-- GraphQL query execution timing
-
-The 60-second buffer ensures these edge-case records are captured in the next run, preventing data loss.
-
-**When to skip state update (`updateState: false`):**
-
-- Historical backfills (don't affect ongoing incremental sync)
-- Testing/debugging specific date ranges
-- Reprocessing old data without changing the sync pointer
-
----
-
-### 📁 S3 Path Configuration
-
-**Note:** Folder paths and filename patterns are configured separately.
-
-- **Folder Path:** Configured via `s3Prefix` activation variable (e.g., `inventory-positions/daily/`)
-- **Filename Pattern:** Configured via `fileNamePrefix` activation variable (e.g., `inventorypositions`)
-
-**The workflow generates files like:** `{s3Prefix}{fileNamePrefix}-{timestamp}.csv`
-
-**Example:** With `s3Prefix="inventory-positions/daily/"` and `fileNamePrefix="inventorypositions"`, a generated file will look like:
-
-```
-inventory-positions/daily/inventorypositions-2025-10-27T18-30-45Z.csv
-```
-
-Format may vary slightly (colons replaced; includes Z).
-
-**Note:** To change the upload folder, modify the `s3Prefix` activation variable (not in code).
-
----
-
-### Auto-pagination and limits (ExtractionOrchestrator)
-
-**What:** ExtractionOrchestrator handles GraphQL Relay cursor-based pagination automatically.
-
-**Why:** Prevents manual pagination loop code, handles large datasets efficiently.
-
-**How:** You configure `pageSize` and `maxRecords`; the orchestrator injects `$first` and `$after` variables automatically and loops until `pageInfo.hasNextPage === false` or `maxRecords` is reached.
-
-**Critical:** Your query MUST include `edges { cursor }` and `pageInfo { hasNextPage }` fields, or pagination will fail.
-
-#### Configuration Parameters
-
-| Parameter        | Purpose                        | Example                           | Effect                     |
-| ---------------- | ------------------------------ | --------------------------------- | -------------------------- |
-| `pageSize`       | Records per GraphQL request    | `200`                             | Controls `first` variable  |
-| `maxRecords`     | Total extraction limit         | `100000`                          | Hard stop across all pages |
-| `resultPath`     | Where records live in response | `"inventoryPositions.edges.node"` | Flattening path            |
-| `validateItem()` | Optional record filter         | `(item) => !!item.ref`            | Skips invalid records      |
-
-#### GraphQL Query Requirements
-
-**Your query MUST include these pagination fields:**
-
-```graphql
-query GetInventoryPositions(
-  $catalogues: [InventoryCatalogueKey]
-  $dateRangeFilter: DateRange
-  $first: Int! # ← Orchestrator injects this
-  $after: String # ← Orchestrator injects this
-) {
-  inventoryPositions(
-    catalogues: $catalogues
-    updatedOn: $dateRangeFilter
-    first: $first # ← Pagination page size
-    after: $after # ← Pagination cursor
-  ) {
-    edges {
-      # ← REQUIRED: Relay connection structure
-      node {
-        # ← REQUIRED: Actual records here
-        id
-        ref
-        # ... your fields
-      }
-      cursor # ← REQUIRED: Orchestrator uses this for next page
-    }
-    pageInfo {
-      # ← REQUIRED: Orchestrator checks this
-      hasNextPage # ← REQUIRED: Tells orchestrator to continue
-    }
-  }
-}
-```
-
----
-
-## 📄 Mapping Configuration
-
-**File:** `config/inventory-positions.export.csv.json`
-
-```json
-{
-  "name": "inventory-positions.export.csv",
-  "version": "1.0.0",
-  "description": "Inventory Positions → CSV Export Mapping",
-  "fields": {
-    "position_ref": {
-      "source": "ref",
-      "required": true,
-      "resolver": "sdk.trim"
-    },
-    "product_ref": {
-      "source": "productRef",
-      "required": true,
-      "resolver": "sdk.trim"
-    },
-    "location_ref": {
-      "source": "locationRef",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "on_hand": {
-      "source": "onHand",
-      "required": true,
-      "resolver": "sdk.parseInt"
-    },
-    "position_type": {
-      "source": "type",
-      "required": true,
-      "resolver": "sdk.uppercase"
-    },
-    "status": {
-      "source": "status",
-      "required": false,
-      "resolver": "sdk.uppercase"
-    },
-    "catalogue_ref": {
-      "source": "catalogue.ref",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "catalogue_name": {
-      "source": "catalogue.name",
-      "required": false,
-      "resolver": "sdk.trim"
-    },
-    "created_on": {
-      "source": "createdOn",
-      "required": true,
-      "resolver": "sdk.toString"
-    },
-    "updated_on": {
-      "source": "updatedOn",
-      "required": true,
-      "resolver": "sdk.toString"
-    }
-  }
-}
-```
-
-**AI Customization Hints:**
-
-- Add fields: Copy existing field config, change `source` path
-- Remove fields: Delete field from config
-- Change resolvers: Replace `sdk.trim` with `sdk.uppercase`, etc.
-- Nested fields: Use dot notation like `catalogue.ref`
-
-Note: Customize mapping by editing the JSON above; prefer built-in resolvers. See SDK Universal Mapping guide for advanced usage.
-
-## 🔧 Complete Production Code
-
-### File Structure
-
-**Production-ready modular structure:**
-
-```
-inventory-positions-to-s3-csv/
-├── src/
-│   ├── index.ts                             # Register workflows
-│   ├── workflows/
-│   │   └── inventory-positions-extraction.ts # ALL 3 workflows (scheduled + 2 webhooks)
-│   ├── services/
-│   │   └── extraction-orchestration.ts      # Main orchestration logic
-│   └── utils/
-│       └── job-id-generator.ts              # Job ID utility
-├── config/
-│   └── inventory-positions.export.csv.json  # Mapping configuration
-├── package.json
-└── tsconfig.json
-```
-
----
-
-### 1. Entry Point (src/index.ts)
-
-```typescript
-/**
- * Entry Point - Export all workflows for Versori platform
- *
- * This file exports all workflows to be registered with Versori.
- * Each workflow is defined in its own file for better organization.
- *
- * DELEGATION PATTERN:
- * - index.ts exports workflows (no business logic)
- * - Workflows orchestrate execution (minimal logic)
- * - Services contain all business logic (maximum reuse)
- *
- * AI CUSTOMIZATION:
- * - Add new workflows by importing and exporting them
- * - Remove workflows by commenting out exports
- * - Change workflow names in import statements
- */
-
-// Scheduled workflows
-export { scheduledInventoryPositionsExtraction } from './workflows/scheduled/inventory-positions-extraction';
-
-// Webhook workflows
-export { adhocInventoryPositionsExtraction } from './workflows/webhook/adhoc-inventory-positions-extraction';
-export { inventoryPositionsJobStatus } from './workflows/webhook/job-status-check';
-```
-
----
-
-### 2. Workflows (src/workflows/inventory-positions-extraction.ts)
-
-```typescript
-/**
- * Workflows - Defines 3 execution patterns for inventory positions extraction
- *
- * WORKFLOW 1: Scheduled (Cron) - Runs automatically every hour
- * WORKFLOW 2: Ad hoc (Webhook) - Manual trigger with optional date override
- * WORKFLOW 3: Job Status (Webhook) - Query job progress
- *
- * AI CUSTOMIZATION HINTS:
- * - Change schedule: Modify cron expression in schedule()
- * - Add filtering: Pass additional params to executeInventoryPositionExtraction()
- * - Change response format: Modify return object structure
- */
-
-import { schedule, webhook, http, fn } from '@versori/run';
-import { executeInventoryPositionExtraction, getJobStatus } from '../services/extraction-orchestration';
-import { generateJobId } from '../utils/job-id-generator';
-
-/**
- * WORKFLOW 1: Scheduled Extraction
- *
- * Purpose: Automated hourly extraction for incremental sync
- * Trigger: Cron schedule (every hour at minute 0)
- * State Update: Always updates lastSync timestamp
- *
- * AI CUSTOMIZATION:
- * - Change schedule: Replace '0 * * * *' with your cron expression
- *   Examples:
- *   - Every 30 min: '*/30 * * * *'
- *   - Daily at 2 AM: '0 2 * * *'
- *   - Every 15 min: '*/15 * * * *'
- */
-export const scheduledInventoryPositionsExtraction = schedule(
-  'scheduled-inventory-positions-extraction',
-  '0 * * * *',  // ← CUSTOMIZE: Cron expression
-  http('execute-scheduled-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
-    const { log } = ctx;
-    const executionStartTime = Date.now();
-
-    // Generate unique job ID for tracking
-    // Format: SCHEDULED_IP_YYYYMMDD_HHMMSS_random
-    const jobId = generateJobId('SCHEDULED', 'INVENTORY_POSITIONS');
-
-    log.info('🚀 [WORKFLOW] Scheduled extraction triggered', { jobId });
-
-    try {
-      // Execute main workflow (extraction → transform → upload)
-      const result = await executeInventoryPositionExtraction(ctx, {
-        jobId,
-        triggeredBy: 'schedule',
-        updateState: true,        // Always update state for scheduled runs
-      });
-
-      const executionDuration = Date.now() - executionStartTime;
-
-      log.info('✅ [WORKFLOW] Scheduled extraction completed', {
-        jobId,
-        recordCount: result.recordsExtracted,
-        fileName: result.fileName,
-        duration: executionDuration,
-      });
-
-      return { ...result, duration: executionDuration };
-
-    } catch (error: any) {
-      const executionDuration = Date.now() - executionStartTime;
-
-      log.error('❌ [WORKFLOW] Scheduled extraction failed', {
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',
-        duration: executionDuration,
-      });
-
-      return {
-        success: false,
-        jobId,
-        error: error.message,
-        duration: executionDuration,
-        recommendation: getErrorRecommendation(error),
-      };
-    }
-  }));
-
-/**
- * WORKFLOW 2: Ad hoc Extraction (Manual Trigger)
- *
- * Purpose: Manual extraction with optional date range override
- * Trigger: Webhook POST to /webhooks/inventory-positions-adhoc
- * State Update: Optional (controlled by request payload)
- *
- * WEBHOOK PAYLOAD EXAMPLES:
- *
- * 1. Incremental (use last sync timestamp):
- *    {}
- *
- * 2. Date range (manual override):
- *    {
- *      "fromDate": "2025-01-01T00:00:00Z",
- *      "toDate": "2025-01-31T23:59:59Z",
- *      "updateState": false
- *    }
- *
- * AI CUSTOMIZATION:
- * - Add request validation
- * - Add authentication check
- * - Add custom filters from payload
- */
-export const adhocInventoryPositionsExtraction = webhook(
-  'inventory-positions-adhoc',
-  { connection: 'inventory-positions-adhoc', response: { mode: 'sync' } }
-)
-.then(
-  http('execute-adhoc-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
-    const { data, log } = ctx;
-    const executionStartTime = Date.now();
-
-    // Generate unique job ID
-    const jobId = generateJobId('ADHOC', 'INVENTORY_POSITIONS');
-
-    // SECURITY: Authentication is enforced by Versori connection configuration
-    // Configure auth on the connection and reference it in webhook({ connection: '...' })
-
-    // Extract optional date override from webhook payload
-    const fromDate = data.fromDate as string | undefined;
-    const toDate = data.toDate as string | undefined;
-    const updateState = data.updateState === true;  // Default false; advance state only if explicitly true
-
-    log.info('🔔 [WORKFLOW] Ad hoc extraction triggered via webhook', {
-      jobId,
-      hasDateOverride: !!fromDate,
-      fromDate: fromDate || 'not specified',
-      toDate: toDate || 'not specified',
-      updateState
-    });
-
-    try {
-      // Execute main workflow with optional overrides
-      const result = await executeInventoryPositionExtraction(ctx, {
-        jobId,
-        triggeredBy: 'webhook',
-        fromDate,              // Optional: override start date
-        toDate,                // Optional: override end date
-        updateState,           // Optional: skip state update for historical queries
-      });
-
-      const executionDuration = Date.now() - executionStartTime;
-
-      log.info('✅ [WORKFLOW] Ad hoc extraction completed', {
-        jobId,
-        recordCount: result.recordsExtracted,
-        fileName: result.fileName,
-        isManualOverride: !!fromDate,
-        stateUpdated: result.stateUpdated,
-        duration: executionDuration,
-      });
-
-      // Return success with job details
-      return {
-        success: true,
-        jobId,
-        recordsExtracted: result.recordsExtracted,
-        fileName: result.fileName,
-        s3Path: result.s3Path,
-        statusUrl: `/webhooks/inventory-positions-job-status?jobId=${jobId}`,
-        duration: executionDuration,
-        dateRange: fromDate ? {
-          from: fromDate,
-          to: toDate || 'not specified',
-          updateState
-        } : undefined
-      };
-
-    } catch (error: any) {
-      const executionDuration = Date.now() - executionStartTime;
-
-      log.error('❌ [WORKFLOW] Ad hoc extraction failed', {
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',
-        duration: executionDuration,
-      });
-
-      return {
-        success: false,
-        jobId,
-        error: error.message,
-        duration: executionDuration,
-        recommendation: getErrorRecommendation(error),
-      };
-    }
-  }));
-
-/**
- * WORKFLOW 3: Job Status Query
- *
- * Purpose: Check job progress and status
- * Trigger: Webhook GET/POST to /webhooks/inventory-positions-job-status?jobId=xxx
- * Returns: Current job status, stage, progress
- *
- * QUERY EXAMPLES:
- *
- * 1. HTTP GET:
- *    GET /webhooks/inventory-positions-job-status?jobId=ADHOC_IP_20251027_183045_abc123
- *
- * 2. HTTP POST:
- *    POST /webhooks/inventory-positions-job-status
- *    { "jobId": "ADHOC_IP_20251027_183045_abc123" }
- */
-export const inventoryPositionsJobStatus = webhook(
-  'inventory-positions-job-status',
-  { connection: 'inventory-positions-job-status', response: { mode: 'sync' } }
-)
-.then(
-  fn('query-job-status', async (ctx) => {
-    const { data, log, openKv } = ctx;
-    // SECURITY: Authentication is enforced by Versori connection configuration
-    // Configure auth on the connection and reference it in webhook({ connection: '...' })
-
-    // Get jobId from query param or POST body
-    const jobId = data.jobId as string;
-
-    if (!jobId) {
-      log.error('❌ [WORKFLOW] Job ID not provided in request');
-      return {
-        success: false,
-        error: 'Job ID is required. Provide jobId in query param or request body.',
-        recommendation: 'Include jobId in your request: { "jobId": "ADHOC_IP_20251027_183045_abc123" }'
-      };
-    }
-
-    log.info('🔍 [WORKFLOW] Querying job status', { jobId });
-
-    try {
-      // Query job status from KV store
-      const status = await getJobStatus(openKv(':project:'), jobId, log);
-
-      if (!status) {
-        log.info('⚠️ [WORKFLOW] Job not found', { jobId });
-        return {
-          success: false,
-          error: 'Job not found',
-          jobId,
-          recommendation: 'Verify the jobId is correct. Jobs may expire after 7 days.'
-        };
-      }
-
-      log.info('✅ [WORKFLOW] Job status retrieved', { jobId, status: status.status });
-
-      return {
-        success: true,
-        jobId,
-        ...status
-      };
-
-    } catch (error: any) {
-      log.error('❌ [WORKFLOW] Failed to query job status', {
-        jobId,
-        message: error instanceof Error ? error.message : String(error),
-        stack: error instanceof Error ? error.stack : undefined,
-        errorType: error instanceof Error ? error.constructor.name : 'Error',
-      });
-
-      return {
-        success: false,
-        jobId,
-        error: error.message,
-        recommendation: getErrorRecommendation(error),
-      };
-    }
-  }));
-
-/**
- * Helper: Get error-specific recommendations
- */
-function getErrorRecommendation(error: any): string {
-  const message = error instanceof Error ? error.message : String(error);
-
-  if (message.includes('ECONNREFUSED') || message.includes('ETIMEDOUT')) {
-    return 'Network connectivity issue. Check firewall rules and network configuration.';
-  }
-  if (message.includes('authentication') || message.includes('401')) {
-    return 'Authentication failed. Verify OAuth2 credentials in connection configuration.';
-  }
-  if (message.includes('S3') || message.includes('bucket')) {
-    return 'S3 operation failed. Check bucket name, region, and AWS credentials.';
-  }
-  if (message.includes('GraphQL') || message.includes('query')) {
-    return 'GraphQL query failed. Verify query syntax and field availability in schema.';
-  }
-  if (message.includes('mapping') || message.includes('transform')) {
-    return 'Data transformation failed. Check mapping configuration and source data structure.';
-  }
-  if (message.includes('KV') || message.includes('state')) {
-    return 'State management failed. Check Versori KV availability and permissions.';
-  }
-
-  return 'Check logs for detailed error information and stack trace.';
-}
-```
-
----
-
-## 3. Main Orchestration Service (src/services/extraction-orchestration.ts)
-
-```typescript
-/**
- * MAIN ORCHESTRATION SERVICE
- *
- * This is the heart of the extraction workflow. It coordinates all steps:
- * 1. Initialize clients and services
- * 2. Determine date range (incremental vs manual)
- * 3. Extract data using ExtractionOrchestrator
- * 4. Transform using UniversalMapper
- * 5. Generate CSV using CSVParserService
- * 6. Upload to S3
- * 7. Track job progress with JobTracker
- * 8. Update state for next run
- *
- * NAMING PATTERN (consistent across all use cases):
- * - Interface: {Entity}ExtractionParams (e.g., InventoryPositionExtractionParams)
- * - Result: {Entity}ExtractionResult (e.g., InventoryPositionExtractionResult)
- * - Main function: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
- *
- * AI CUSTOMIZATION HINTS:
- * - Change entity: Replace "InventoryPosition" with "Order", "Product", etc.
- * - Change output: Replace CSVParserService with XMLBuilder
- * - Change destination: Replace S3DataSource with SftpDataSource
- * - Add steps: Insert new service calls between existing steps
- */
-
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  CSVParserService,
-  S3DataSource,
-} from '@fluentcommerce/fc-connect-sdk';
-
-import mappingConfig from '../../config/inventory-positions.export.csv.json' with { type: 'json' };
-
-// ✅ VERSORI PLATFORM: Use native log from context, not LoggingService
-
-/**
- * Parameters for extraction workflow
- *
- * NAMING: {Entity}ExtractionParams
- */
-export interface InventoryPositionExtractionParams {
-  jobId: string;
-  triggeredBy: 'schedule' | 'webhook';
-  fromDate?: string; // Optional: manual date override
-  toDate?: string; // Optional: manual date override
-  updateState: boolean; // Whether to update lastSync timestamp
-
-  // AI CUSTOMIZATION: Add filters specific to entity
-  positionTypes?: string[]; // e.g., ['DEFAULT', 'SEASONAL']
-  catalogueRef?: string; // e.g., 'DEFAULT_CATALOGUE'
-}
-
-/**
- * Result from extraction workflow
- *
- * NAMING: {Entity}ExtractionResult
- */
-export interface InventoryPositionExtractionResult {
-  success: boolean;
-  jobId: string;
-  recordsExtracted: number;
-  fileName?: string;
-  s3Path?: string;
-  error?: string;
-  errors?: any[];
-  isManualOverride?: boolean;
-  stateUpdated?: boolean;
-  newTimestamp?: string;
-}
-
-/**
- * GraphQL Query for Inventory Positions
- *
- * NAMING: {ENTITY}_EXTRACTION_QUERY (uppercase constant)
- */
-const INVENTORY_POSITIONS_EXTRACTION_QUERY = `
-  query GetInventoryPositions(
-    $catalogues: [InventoryCatalogueKey]
-    $dateRangeFilter: DateRange
-    $productRefs: [String!]
-    $types: [String!]
-    $first: Int!
-    $after: String
-  ) {
-    inventoryPositions(
-      catalogues: $catalogues
-      updatedOn: $dateRangeFilter
-      productRef: $productRefs
-      type: $types
-      first: $first
-      after: $after
-    ) {
-      edges {
-        node {
-          id
-          ref
-          productRef
-          locationRef
-          onHand
-          type
-          status
-          catalogue {
-            ref
-            name
-          }
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-
-/**
- * Query job status from KV store
- *
- * ✅ VERSORI PLATFORM: Uses native log from context (passed as parameter)
- */
-export async function getJobStatus(
-  kv: any, // ✅ Versori KV (compatible with JobTracker's KVAdapter interface)
-  jobId: string,
-  log: any // ✅ Native Versori log from context
-): Promise<any | undefined> {
-  try {
-    const tracker = new JobTracker(kv, log);
-    return await tracker.getJob(jobId);
-  } catch (error: any) {
-    log.error('Failed to get job status', {
-      jobId,
-      message: error instanceof Error ? error.message : String(error),
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'Error',
-    });
-    return undefined;
-  }
-}
-
-/**
- * MAIN ORCHESTRATION FUNCTION
- *
- * NAMING: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
- *
- * This function implements the complete workflow in steps.
- * Each step is clearly commented for AI understanding.
- */
-export async function executeInventoryPositionExtraction(
-  ctx: any,
-  params: InventoryPositionExtractionParams
-): Promise<InventoryPositionExtractionResult> {
-  // ✅ VERSORI PLATFORM: Extract native log from context
-  const { log, openKv, activation } = ctx;
-  const { jobId, triggeredBy, fromDate, toDate, updateState } = params;
-
-  // Open KV store for state management and job tracking
-  // ✅ Pass raw Versori KV directly - it matches KVAdapter interface
-  // ✅ Pass native log to JobTracker
-  const kv = openKv(':project:');
-  const tracker = new JobTracker(kv, log);
-
-  try {
-    // ═══════════════════════════════════════════════════════════
-    // STEP 1/8: Initialize Job Tracking
-    // ═══════════════════════════════════════════════════════════
-    const step1Start = Date.now();
-    log.info('📋 [STEP 1/8] Initializing job tracking', { jobId });
-
-    await tracker.createJob(jobId, {
-      triggeredBy,
-      hasDateOverride: !!fromDate,
-      fromDate,
-      toDate,
-      updateStateAfterRun: updateState,
-    });
-
-    log.info('✅ [STEP 1/8] Job tracking initialized', {
-      jobId,
-      duration: Date.now() - step1Start,
-    });
-
-    // ═══════════════════════════════════════════════════════════
-    // STEP 2/8: Initialize Fluent Client
-    // ═══════════════════════════════════════════════════════════
-    const step2Start = Date.now();
-    log.info('🔌 [STEP 2/8] Initializing Fluent Commerce client', { jobId });
-
-    const client = await createClient(ctx);
-
-    if (!client) {
-      throw new Error('Failed to create Fluent Commerce client - check connection configuration');
-    }
-
-    log.info('✅ [STEP 2/8] Client initialized', {
-      jobId,
-      duration: Date.now() - step2Start,
-    });
-
-    // ═══════════════════════════════════════════════════════════
-    // STEP 3/8: Determine Date Range
-    // ═══════════════════════════════════════════════════════════
-    const step3Start = Date.now();
-    log.info('📅 [STEP 3/8] Determining date range for extraction', { jobId });
-
-    // State key for incremental sync tracking
-    // NAMING: last{Entity}Sync (e.g., lastInventoryPositionSync)
-    const STATE_KEY = 'lastInventoryPositionSync';
-    const DEFAULT_FALLBACK = '2024-01-01T00:00:00Z';
-    const OVERLAP_BUFFER_SECONDS = parseInt(
-      activation.getVariable('overlapBufferSeconds') || '60',
-      10
-    );
-
-    let dateRangeFilter: { from?: string; to?: string } | null = null;
-    const isManualOverride = !!fromDate;
-
-    if (isManualOverride) {
-      // Manual date override from webhook
-      dateRangeFilter = { from: fromDate, to: toDate };
-      log.info('Using manual date override', { fromDate, toDate });
-    } else {
-      // Incremental sync - get last sync timestamp
-      const rawLastRunTime = (await kv.get<string>(STATE_KEY)) || DEFAULT_FALLBACK;
-
-      // Apply overlap buffer (prevents missed records)
-      const bufferedLastRunTime = new Date(
-        new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_SECONDS * 1000
-      ).toISOString();
-
-      const effectiveEndTime = toDate || new Date().toISOString();
-
-      dateRangeFilter = {
-        from: bufferedLastRunTime,
-        to: effectiveEndTime, // End of extraction window
-      };
-
-      log.info('Using incremental sync with overlap buffer', {
-        rawLastRunTime,
-        bufferedLastRunTime,
-        effectiveEndTime,
-        overlapBufferSeconds: OVERLAP_BUFFER_SECONDS,
-      });
-    }
-
-    log.info('✅ [STEP 3/8] Date range determined', {
-      jobId,
-      fromDate: dateRangeFilter?.from,
-      toDate: dateRangeFilter?.to,
-      isManualOverride,
-      duration: Date.now() - step3Start,
-    });
-
-    // ═══════════════════════════════════════════════════════════
-    // STEP 4/8: Extract Data (ExtractionOrchestrator)
-    // ═══════════════════════════════════════════════════════════
-    const step4Start = Date.now();
-    log.info('📥 [STEP 4/8] Extracting data from Fluent Commerce', { jobId });
-
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 'extraction',
-      message: 'Extracting data with auto-pagination',
-    });
-
-    // Build catalogues array from config
-    const catalogueRef = params.catalogueRef || activation.getVariable('catalogueRef');
-    const catalogues = catalogueRef ? [{ ref: catalogueRef }] : [];
-
-    // Configure extraction
-    const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
-    const maxRecords = parseInt(activation.getVariable('maxRecords') || '100000', 10);
-
-    // Initialize ExtractionOrchestrator
-    const orchestrator = new ExtractionOrchestrator(client, log);
-
-    // ? Enhanced: Extract context for progress logging
-    const dateRangeInfo = {
-      start: dateRangeFilter?.from || 'N/A',
-      end: dateRangeFilter?.to || 'N/A',
-      catalogues: catalogues.map((c: any) => c.ref).join(', ') || 'all',
-      types: params.positionTypes?.join(', ') || 'all'
-    };
-
-    // ? Enhanced: Start logging with context
-    log.info(`🔍 [ExtractionOrchestrator] Starting extraction`, {
-     query: 'inventoryPositions',
-     pageSize,
-     maxRecords,
-     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
-     catalogues: dateRangeInfo.catalogues,
-     positionTypes: dateRangeInfo.types,
-     jobId
-    });
-
-    // Execute extraction with auto-pagination
-    const extractionResult = await orchestrator.extract({
-      query: INVENTORY_POSITIONS_EXTRACTION_QUERY,
-      resultPath: 'inventoryPositions.edges.node',
-      variables: {
-        catalogues,
-        dateRangeFilter,
-        types: params.positionTypes,
-        // Note: Don't include 'first' or 'after' here; orchestrator injects them
-      },
-      pageSize,
-      maxRecords,
-      // Optional: validate each record
-      validateItem: (item: any) => {
-        return !!(item.ref && item.productRef);
-      },
-    });
-
-    const rawRecords = extractionResult.data;
-
-    log.info('✅ [STEP 4/8] Extraction completed', {
-      jobId,
-      totalRecords: extractionResult.stats.totalRecords,
-      totalPages: extractionResult.stats.totalPages,
-      validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
-      errors: extractionResult.errors ? extractionResult.errors.length : 0,
-      duration: Date.now() - step4Start,
-    });
-
-    // ? Enhanced: Completion logging with summary
-    log.info(`✅ [ExtractionOrchestrator] Extraction completed`, {
-     totalRecords: extractionResult.stats.totalRecords,
-     totalPages: extractionResult.stats.totalPages,
-     validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
-     failedValidations: extractionResult.stats.failedValidations,
-     truncated: extractionResult.stats.truncated,
-     truncationReason: extractionResult.stats.truncationReason,
-     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
-     jobId
-    });
-
-    if (extractionResult.errors && extractionResult.errors.length > 0) {
-      log.warn('Non-fatal extraction errors encountered', {
-        jobId,
-        errorCount: extractionResult.errors.length,
-        sampleErrors: extractionResult.errors.slice(0, 3),
-      });
-    }
-
-    // Handle empty result
-    if (rawRecords.length === 0) {
-      log.info('No records to process');
-
-      // Update state even with no records (prevents re-querying empty window)
-      if (updateState) {
-        await kv.set(STATE_KEY, new Date().toISOString());
-      }
-
-      await tracker.markCompleted(jobId, {
-        recordCount: 0,
-        message: 'No records to extract',
-      });
-
-      return {
-        success: true,
-        jobId,
-        recordsExtracted: 0,
-      };
-    }
-
-    // ═══════════════════════════════════════════════════════════
-    // STEP 5/8: Transform Data (UniversalMapper)
-    // ═══════════════════════════════════════════════════════════
-    const step5Start = Date.now();
-    log.info('🔄 [STEP 5/8] Transforming data with UniversalMapper', {
-      jobId,
-      recordCount: rawRecords.length,
-    });
-
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 'transformation',
-      message: `Transforming ${rawRecords.length} records`,
-    });
-
-    const mapper = new UniversalMapper(mappingConfig);
-    const mappingResult = await mapper.map(rawRecords);
-
-    if (!mappingResult.success) {
-      const mappingErrors = mappingResult.errors || ['Unknown mapping failure'];
-      log.error('[STEP 5/8] Mapping failed - terminating job', {
-        jobId,
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-
-      await tracker.markFailed(
-        jobId,
-        new Error(mappingErrors[0] || 'UniversalMapper returned unsuccessful result')
-      );
-
-      return {
-        success: false,
-        jobId,
-        recordsExtracted: 0,
-        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
-      };
-    }
-
-    const transformedRecords = Array.isArray(mappingResult.data) ? mappingResult.data : [];
-    const mappingErrors = mappingResult.errors || [];
-
-    if (mappingErrors.length > 0) {
-      log.warn('[STEP 5/8] Some records failed transformation', {
-        jobId,
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-    }
-
-    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
-      log.info('ℹ️ [MAPPING] Optional fields skipped (undefined values)', {
-        jobId,
-        skippedFields: mappingResult.skippedFields,
-        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
-      });
-    }
-
-    if (transformedRecords.length === 0) {
-      throw new Error('All records failed transformation');
-    }
-
-    log.info('✅ [STEP 5/8] Transformation complete', {
-      jobId,
-      successful: transformedRecords.length,
-      skippedRecords: rawRecords.length - transformedRecords.length,
-      duration: Date.now() - step5Start,
-    });
-
-    // ═══════════════════════════════════════════════════════════
-    // STEP 6/8: Generate CSV (CSVParserService)
-    // ═══════════════════════════════════════════════════════════
-    const step6Start = Date.now();
-    log.info('📄 [STEP 6/8] Generating CSV file', { jobId });
-
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 'csv_generation',
-      message: `Generating CSV for ${transformedRecords.length} records`,
-    });
-
-    // Initialize CSVParserService
-    const csvParser = new CSVParserService();
-
-    // Generate CSV content (synchronous, not async)
-    const csvContent = csvParser.stringify(transformedRecords, { headers: true });
-
-    // Generate filename
-    const fileNamePrefix = activation.getVariable('fileNamePrefix') || 'inventorypositions';
-    const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
-    const fileName = `${fileNamePrefix}-${timestamp}.csv`;
-
-    log.info('✅ [STEP 6/8] CSV file generated', {
-      fileName,
-      sizeBytes: csvContent.length,
-      recordCount: transformedRecords.length,
-      duration: Date.now() - step6Start,
-    });
-
-    // ═══════════════════════════════════════════════════════════
-    // STEP 7/8: Upload to S3 (S3DataSource)
-    // ═══════════════════════════════════════════════════════════
-    const step7Start = Date.now();
-    log.info('☁️ [STEP 7/8] Uploading to S3', { jobId, fileName });
-
-    await tracker.updateJob(jobId, {
-      status: 'processing',
-      stage: 's3_upload',
-      message: `Uploading ${fileName} to S3`,
-    });
-
-    // Get S3 configuration from activation variables
-    const s3Config = {
-      bucket: activation.getVariable('s3BucketName'),
-      region: activation.getVariable('awsRegion') || 'us-east-1',
-      accessKeyId: activation.getVariable('awsAccessKeyId'),
-      secretAccessKey: activation.getVariable('awsSecretAccessKey'),
-    };
-    const s3Prefix = activation.getVariable('s3Prefix') || 'inventory-positions/daily/';
-
-    // Validate S3 config
-    if (!s3Config.bucket || !s3Config.accessKeyId || !s3Config.secretAccessKey) {
-      const missingFields = [];
-      if (!s3Config.bucket) missingFields.push('s3BucketName');
-      if (!s3Config.accessKeyId) missingFields.push('awsAccessKeyId');
-      if (!s3Config.secretAccessKey) missingFields.push('awsSecretAccessKey');
-
-      throw new Error(
-        `S3 configuration incomplete - missing activation variables: ${missingFields.join(', ')}`
-      );
-    }
-
-    // Initialize S3 data source
-    // ✅ VERSORI PLATFORM: Pass native log from context
-    const s3 = new S3DataSource(
-      {
-        type: 'S3_CSV',
-        connectionId: 'inventory-positions-s3',
-        name: 'Inventory Positions S3 Upload',
-        s3Config,
-      },
-      log
-    );
-
-    // Validate S3 connection before upload
-    try {
-      await s3.validateConnection();
-      log.info('✅ S3 connection validated', { bucket: s3Config.bucket });
-    } catch (validateError: any) {
-      log.error('❌ S3 connection validation failed', {
-        bucket: s3Config.bucket,
-        region: s3Config.region,
-        error: validateError.message,
-      });
-      throw new Error(
-        `S3 connection validation failed: ${validateError.message}. ` +
-        `Check bucket name, region, and AWS credentials.`
-      );
-    }
-
-    // Construct S3 key
-    const s3Key = `${s3Prefix}${fileName}`;
-
-    // Upload with retry logic (built into S3DataSource)
-    await s3.upload(s3Key, Buffer.from(csvContent, 'utf-8'), {
-      contentType: 'text/csv',
-      metadata: {
-        recordCount: String(transformedRecords.length),
-        extractedAt: new Date().toISOString(),
-        jobId,
-      },
-    });
-
-    log.info('✅ [STEP 7/8] S3 upload successful', {
-      fileName,
-      s3Key,
-      bucket: s3Config.bucket,
-      sizeBytes: csvContent.length,
-      duration: Date.now() - step7Start,
-    });
-
-    // ═══════════════════════════════════════════════════════════
-    // STEP 8/8: Update State & Complete Job
-    // ═══════════════════════════════════════════════════════════
-    const step8Start = Date.now();
-    log.info('💾 [STEP 8/8] Updating state and completing job', { jobId });
-
-    // Calculate new timestamp for next incremental run
-    let newTimestamp: string | undefined;
-
-    if (updateState) {
-      // Find max updatedOn from extracted records
-      const maxUpdatedOn = rawRecords.reduce(
-        (max, record) => {
-          const recordTime = new Date(record.updatedOn).getTime();
-          return recordTime > max ? recordTime : max;
-        },
-        new Date(dateRangeFilter?.from || DEFAULT_FALLBACK).getTime()
-      );
-
-      newTimestamp = new Date(maxUpdatedOn).toISOString();
-
-      // Store new timestamp (WITHOUT buffer - buffer only applied on read)
-      await kv.set(STATE_KEY, newTimestamp);
-
-      log.info('State updated', {
-        oldTimestamp: dateRangeFilter?.from,
-        newTimestamp,
-      });
-    }
-
-    // Mark job as completed
-    await tracker.markCompleted(jobId, {
-      recordCount: transformedRecords.length,
-      fileName,
-      s3Key,
-      errorCount: mappingErrors.length,
-      isManualOverride,
-      stateUpdated: updateState,
-      newTimestamp,
-    });
-
-    log.info('✅ [STEP 8/8] State updated and job completed', {
-      jobId,
-      stateUpdated: updateState,
-      newTimestamp,
-      duration: Date.now() - step8Start,
-    });
-
-    return {
-      success: true,
-      jobId,
-      recordsExtracted: transformedRecords.length,
-      fileName,
-      s3Path: s3Key,
-      isManualOverride,
-      stateUpdated: updateState,
-      newTimestamp,
-      errors: mappingErrors.length > 0 ? mappingErrors : undefined,
-    };
-  } catch (error: any) {
-    log.error('❌ [ORCHESTRATION] Extraction workflow failed', {
-      jobId,
-      message: error instanceof Error ? error.message : String(error),
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'Error',
-    });
-
-    // Mark job as failed
-    await tracker.markFailed(jobId, error);
-
-    return {
-      success: false,
-      jobId,
-      recordsExtracted: 0,
-      error: error.message,
-      recommendation: getErrorRecommendation(error),
-    };
-  }
-}
-
-/**
- * Helper: Get error-specific recommendations
- * (Same helper function used in workflows - can be shared via utils)
- */
-function getErrorRecommendation(error: any): string {
-  const message = error instanceof Error ? error.message : String(error);
-
-  if (message.includes('ECONNREFUSED') || message.includes('ETIMEDOUT')) {
-    return 'Network connectivity issue. Check firewall rules and network configuration.';
-  }
-  if (message.includes('authentication') || message.includes('401')) {
-    return 'Authentication failed. Verify OAuth2 credentials in connection configuration.';
-  }
-  if (message.includes('S3') || message.includes('bucket')) {
-    return 'S3 operation failed. Check bucket name, region, and AWS credentials.';
-  }
-  if (message.includes('GraphQL') || message.includes('query')) {
-    return 'GraphQL query failed. Verify query syntax and field availability in schema.';
-  }
-  if (message.includes('mapping') || message.includes('transform')) {
-    return 'Data transformation failed. Check mapping configuration and source data structure.';
-  }
-  if (message.includes('KV') || message.includes('state')) {
-    return 'State management failed. Check Versori KV availability and permissions.';
-  }
-
-  return 'Check logs for detailed error information and stack trace.';
-}
-```
-
----
-
-## 4. Utility Functions (src/utils/job-id-generator.ts)
-
-```typescript
-/**
- * Job ID Generator
- *
- * Generates unique job IDs for tracking extraction workflows
- *
- * FORMAT: {TYPE}_{ENTITY}_{YYYYMMDD}_{HHMMSS}_{RANDOM}
- * Example: SCHEDULED_IP_20251027_183045_a1b2c3
- */
-
-/**
- * Generate unique job ID
- *
- * @param type - Job trigger type (SCHEDULED, ADHOC, MANUAL)
- * @param entity - Entity abbreviation (IP=Inventory Positions, VP, ORD, PRD)
- * @returns Unique job ID string
- */
-export function generateJobId(type: string, entity: string): string {
-  const now = new Date();
-
-  // Format: YYYYMMDD
-  const dateStr = now.toISOString().slice(0, 10).replace(/-/g, '');
-
-  // Format: HHMMSS
-  const timeStr = now.toISOString().slice(11, 19).replace(/:/g, '');
-
-  // Random suffix (6 chars)
-  const randomStr = Math.random().toString(36).substring(2, 8);
-
-  return `${type}_${entity}_${dateStr}_${timeStr}_${randomStr}`;
-}
-
-/**
- * Parse job ID components
- */
-export function parseJobId(jobId: string): {
-  type: string;
-  entity: string;
-  date: string;
-  time: string;
-  random: string;
-} | null {
-  const parts = jobId.split('_');
-
-  if (parts.length !== 5) {
-    return null;
-  }
-
-  return {
-    type: parts[0],
-    entity: parts[1],
-    date: parts[2],
-    time: parts[3],
-    random: parts[4],
-  };
-}
-```
-
----
-
-## 5. Package Configuration
-
-### package.json
-
-```json
-{
-  "name": "inventory-positions-to-s3-csv",
-  "version": "1.0.0",
-  "description": "Extract inventory positions from Fluent Commerce and export to S3 as CSV",
-  "type": "module",
-  "main": "src/index.ts",
-  "scripts": {
-    "dev": "versori dev",
-    "build": "versori build",
-    "deploy": "versori deploy"
-  },
-  "dependencies": {
-    "@fluentcommerce/fc-connect-sdk": "^0.1.39",
-    "@versori/run": "latest"
-  },
-  "devDependencies": {
-    "@types/node": "^20.0.0",
-    "typescript": "^5.0.0"
-  }
-}
-```
-
-### tsconfig.json
-
-```json
-{
-  "compilerOptions": {
-    "module": "ES2022",
-    "target": "ES2024",
-    "moduleResolution": "node"
-  }
-}
-```
-
----
-
-## 6. Deployment Instructions
-
-### Deploy to Versori
-
-```bash
-# 1. Install dependencies
-npm install
-
-# 2. Test locally (if using Versori CLI)
-npm run dev
-
-# 3. Deploy to Versori platform
-npm run deploy
-```
-
-### Configure Activation Variables
-
-In Versori platform settings, configure:
-
-```json
-{
-  "catalogueRef": "DEFAULT_CATALOGUE",
-  "s3BucketName": "inventory-analytics-exports",
-  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
-  "awsSecretAccessKey": "********",
-  "awsRegion": "us-east-1",
-  "s3Prefix": "inventory-positions/daily/",
-  "fileNamePrefix": "inventorypositions",
-  "pageSize": 200,
-  "maxRecords": 100000,
-  "overlapBufferSeconds": 60
-}
-```
-
----
-
-## 7. Testing
-
-### Test Scheduled Extraction
-
-The scheduled workflow runs automatically based on cron schedule.
-
-**Check logs:**
-
-```
-[STEP 1/8] Initializing job tracking
-[STEP 2/8] Initializing Fluent Commerce client
-[STEP 3/8] Determining date range for extraction
-[STEP 4/8] Extracting data from Fluent Commerce
-[STEP 5/8] Transforming data with UniversalMapper
-[STEP 6/8] Generating CSV file
-[STEP 7/8] Uploading to S3
-[STEP 8/8] Updating state and completing job
-```
-
-### Test Ad hoc Extraction
-
-```bash
-# Incremental (uses last sync timestamp)
-curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
-  -H "Content-Type: application/json" \
-  -d '{}'
-
-# Date range override
-curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
-  -H "Content-Type: application/json" \
-  -d '{
-    "fromDate": "2025-01-01T00:00:00Z",
-    "toDate": "2025-01-31T23:59:59Z",
-    "updateState": false
-  }'
-```
-
-### Test Job Status Query
-
-```bash
-curl -X POST https://api.versori.com/webhooks/inventory-positions-job-status \
-  -H "Content-Type: application/json" \
-  -d '{
-    "jobId": "ADHOC_IP_20251027_183045_abc123"
-  }'
-```
-
-**Response:**
-
-```json
-{
-  "success": true,
-  "jobId": "ADHOC_IP_20251027_183045_abc123",
-  "status": "processing",
-  "stage": "transformation",
-  "message": "Transforming 15000 records",
-  "createdAt": "2025-10-27T18:30:45.000Z",
-  "startedAt": "2025-10-27T18:30:46.000Z"
-}
-```
-
----
-
----
-
-### Pattern 7: State Management & Date Overrides
-
-**Use Case**: Understand how state management works with scheduled and ad-hoc extractions.
-
-**How it works**:
-
-VersoriKV stores the last successful extraction timestamp to enable incremental sync:
-
-```typescript
-interface ExtractionState {
-  timestamp: string; // Last run timestamp (WITHOUT overlap buffer)
-  recordCount: number; // Number of records extracted
-  extractedAt: string; // When extraction completed
-  fileName?: string; // Generated filename
-  s3Key?: string; // S3 upload path
-  overlapBufferSeconds?: number; // Buffer configuration
-}
-```
-
-**State Priority Chain** (highest to lowest):
-
-1. **`fromDate` override** (manual date in webhook payload) - Highest priority
-2. **Stored state** (`await kv.get(stateKey)`) - Normal incremental mode
-3. **`fallbackStartDate`** (activation variable) - First run fallback
-
-**Three Scenarios**:
-
-#### Scenario 1: Normal Scheduled Runs (Incremental)
-
-```typescript
-// Payload: {} (empty - no overrides)
-
-// Behavior:
-// 1. Load last timestamp from KV: "2025-01-22T10:00:00Z"
-// 2. Apply overlap buffer: "2025-01-22T09:59:00Z" (query WITH buffer)
-// 3. Extract records updated since buffered time
-// 4. Calculate MAX(updatedOn) from results: "2025-01-22T14:30:00Z"
-// 5. Save new timestamp WITHOUT buffer: "2025-01-22T14:30:00Z"
-// 6. Next run starts from "2025-01-22T14:29:00Z" (with buffer)
-```
-
-**Test**:
-
-```bash
-# Trigger scheduled run (no payload needed)
-# State advances automatically
-curl -X POST https://workspace.versori.run/inventory-positions-scheduled
-```
-
-#### Scenario 2: Ad-hoc Extraction WITH State Update
-
-```typescript
-// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": true }
-
-// Behavior:
-// 1. Ignore stored state
-// 2. Use fromDate: "2025-01-01T00:00:00Z" (no buffer applied to manual dates)
-// 3. Extract all records since 2025-01-01
-// 4. Calculate MAX(updatedOn): "2025-01-22T14:30:00Z"
-// 5. Save new timestamp: "2025-01-22T14:30:00Z" (updates state!)
-// 6. Next scheduled run starts from this new timestamp
-```
-
-**Use Case**: One-time catch-up extraction that advances the state pointer.
-
-**Test**:
-
-```bash
-curl -X POST https://workspace.versori.run/inventory-positions-adhoc \
-  -H "Content-Type: application/json" \
-  -d '{
-    "fromDate": "2025-01-01T00:00:00Z",
-    "updateState": true
-  }'
-```
-
-#### Scenario 3: Ad-hoc Extraction WITHOUT State Update
-
-```typescript
-// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": false }
-
-// Behavior:
-// 1. Ignore stored state
-// 2. Use fromDate: "2025-01-01T00:00:00Z"
-// 3. Extract all records since 2025-01-01
-// 4. DO NOT update state
-// 5. Next scheduled run uses previous timestamp (unaffected)
-```
-
-**Use Case**: Historical backfill or testing without affecting incremental sync.
-
-**Test**:
-
-```bash
-curl -X POST https://workspace.versori.run/inventory-positions-adhoc \
-  -H "Content-Type: application/json" \
-  -d '{
-    "fromDate": "2025-01-01T00:00:00Z",
-    "toDate": "2025-01-31T23:59:59Z",
-    "updateState": false
-  }'
-```
-
-**Why this matters**:
-
-- **Incremental sync** relies on state continuity
-- **Manual overrides** allow catch-up without breaking incremental flow
-- **Overlap buffer** prevents missed records at time boundaries
-- **State isolation** lets you test/backfill without affecting production sync
-
----
-
-### Pattern 8: Optional GraphQL Query Logging
-
-**Use Case**: Debug extraction issues by logging the exact GraphQL query sent to Fluent Commerce API.
-
-**When to use**:
-
-- ✅ Debugging pagination issues
-- ✅ Verifying query variables (dates, filters, limits)
-- ✅ Development and testing
-- ❌ Production (verbose logs, potential secrets in variables)
-
-**How to enable**:
-
-Set `DEBUG_GRAPHQL=true` environment variable in Versori activation settings.
-
-**Implementation**:
-
-```typescript
-// In your extraction workflow
-const DEBUG_GRAPHQL = activation?.getVariable('DEBUG_GRAPHQL') === 'true';
-
-if (DEBUG_GRAPHQL) {
-  log.info('GraphQL Query Debug', {
-    query: INVENTORY_POSITIONS_QUERY,
-    variables: {
-      catalogueRef,
-      dateRangeFilter,
-      first: pageSize,
-      after: null, // First page
-    },
-    pagination: {
-      pageSize,
-      maxRecords,
-      currentPage: 1,
-    },
-  });
-}
-
-const extractionResult = await orchestrator.extract({
-  query: INVENTORY_POSITIONS_QUERY,
-  resultPath: 'inventoryPositions.edges.node',
-  variables: {
-    catalogueRef,
-    dateRangeFilter,
-  },
-  pageSize,
-  maxRecords,
-});
-
-if (DEBUG_GRAPHQL) {
-  log.info('GraphQL Response Debug', {
-    totalRecords: extractionResult.stats.totalRecords,
-    totalPages: extractionResult.stats.totalPages,
-    truncated: extractionResult.stats.truncated,
-    truncationReason: extractionResult.stats.truncationReason,
-    firstRecordId: extractionResult.data[0]?.id,
-    lastRecordId: extractionResult.data[extractionResult.data.length - 1]?.id,
-  });
-}
-```
-
-**What gets logged**:
-
-```json
-{
-  "level": "info",
-  "message": "GraphQL Query Debug",
-  "query": "query GetInventoryPositions($catalogueRef: String, $dateRangeFilter: DateRange, ...)",
-  "variables": {
-    "catalogueRef": "DEFAULT_CATALOGUE",
-    "dateRangeFilter": "2025-01-22T09:59:00Z",
-    "first": 200,
-    "after": null
-  },
-  "pagination": {
-    "pageSize": 200,
-    "maxRecords": 100000,
-    "currentPage": 1
-  }
-}
-```
-
-**Versori Environment Variables**:
-
-Add to activation settings:
-
-```json
-{
-  "DEBUG_GRAPHQL": "true"
-}
-```
-
-**Testing**:
-
-```bash
-# Enable debug logging
-curl -X POST https://workspace.versori.run/inventory-positions-scheduled
-
-# Check Versori logs for "GraphQL Query Debug" entries
-# Verify query structure and variables are correct
-```
-
-**Sample Debug Output**:
-
-```
-[INFO] GraphQL Query Debug
-  query: "query GetInventoryPositions($catalogueRef: String, $dateRangeFilter: DateRange, ...)"
-  variables: { catalogueRef: "DEFAULT_CATALOGUE", dateRangeFilter: "2025-01-22T09:59:00Z", first: 200, after: null }
-  pagination: { pageSize: 200, maxRecords: 100000, currentPage: 1 }
-
-[INFO] Extraction complete
-  totalRecords: 1500
-  totalPages: 8
-  truncated: false
-
-[INFO] GraphQL Response Debug
-  totalRecords: 1500
-  totalPages: 8
-  truncated: false
-  truncationReason: undefined
-  firstRecordId: "ip_001"
-  lastRecordId: "ip_1500"
-```
-
-**Key Benefits**:
-
-- Quickly identify pagination configuration issues
-- Verify date filters are applied correctly
-- Debug "no records found" scenarios
-- Validate ExtractionOrchestrator variable injection
-
-**Production Best Practice**: Disable `DEBUG_GRAPHQL` in production to reduce log volume and avoid logging sensitive data.
-
----
-
-## 8. Troubleshooting
-
-**Issue**: "No records extracted"
-
-- Check dateRange (manual override vs incremental)
-- Check catalogueRef filter
-
-**Issue**: "S3 upload failed"
-
-- Job fails; state not advanced
-- Next run retries same window
-- Check S3 credentials and bucket permissions
-
-**Issue**: "GraphQL pagination error"
-
-- Ensure edges.cursor and pageInfo.hasNextPage are in query
-
-**Issue**: "Memory pressure"
-
-- Lower pageSize or maxRecords
-- Consider file splitting for large extractions
-
----
-
-## 9. Replication Checklist
-
-**To replicate this template for other entities/formats:**
-
-1. **File Naming:** Replace `inventory-positions`, `IP`, `InventoryPosition` with your entity name
-2. **GraphQL Query:** Update query constant and field selection to match your entity schema
-3. **Mapping Config:** Create new mapping file in `config/` with correct field paths
-4. **Workflows:** Rename workflow exports to match entity (e.g., `scheduledOrdersExtraction`)
-5. **Service Function:** Rename main function (e.g., `executeOrderExtraction`)
-6. **State Key:** Update KV key (e.g., `lastOrderSync`)
-7. **Output Format:** For XML use `XMLBuilder`, for JSON use `JSON.stringify()`, for CSV use `CSVParserService`
-8. **Upload Destination:** For SFTP replace `S3DataSource` with `SftpDataSource` (and add `dispose()` in finally block)
-9. **Job ID Entity Code:** Update entity abbreviation in generateJobId() (e.g., 'ORD' for orders)
-
----
-
-**Pattern**: Enterprise incremental extraction with overlap buffer for inventory positions
-**Key Learning**: Use `catalogues` (plural, array) input for filtering
-**Critical**: Apply 60-second overlap buffer to prevent missed records due to clock skew
-**Buffer Pattern**: Query WITH buffer (`updatedOn >= lastRunTime - 60s`), save WITHOUT buffer (`MAX(updatedOn)`)
-**SDK Services**: ExtractionOrchestrator, UniversalMapper, CSVParserService, S3DataSource, JobTracker
-
----
-
-### Pattern 8: Backward Pagination (Optional - Advanced)
-
-**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
-
-**When to Use**:
-
-- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
-- ✅ Time-bounded reverse traversal for auditing
-- ✅ Display newest-first in UI/reports
-- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
-
-**GraphQL Query Requirements**:
-
-Your query must support backward pagination by including `$last` and `$before`:
-
-```graphql
-query GetData(
-  $retailerId: ID!
-  $first: Int # For forward pagination
-  $after: String # For forward pagination
-  $last: Int # For backward pagination
-  $before: String # For backward pagination
-) {
-  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
-    edges {
-      cursor # ✅ REQUIRED
-      node {
-        id
-        createdAt
-        # ... other fields
-      }
-    }
-    pageInfo {
-      hasNextPage # For forward
-      hasPreviousPage # ✅ REQUIRED for backward
-    }
-  }
-}
-```
-
-**Implementation**:
-
-```typescript
-// Backward pagination - newest records first
-const result = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    retailerId,
-    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
-    // ❌ Don't include last/before - orchestrator injects them
-  },
-  pageSize: 200,
-  direction: 'backward', // ✅ Enable reverse pagination
-  maxRecords: 10000,
-});
-
-// Records are returned in reverse chronological order
-console.log(result.data[0].createdAt); // Newest
-console.log(result.data[result.data.length - 1].createdAt); // Oldest (within range)
-```
-
-**Key Differences from Forward Pagination**:
-
-| Aspect                 | Forward (Default)                | Backward                |
-| ---------------------- | -------------------------------- | ----------------------- |
-| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
-| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
-| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
-| **Cursor Source**      | Last edge of page                | First edge of page      |
-| **Record Order**       | Oldest → Newest                  | Newest → Oldest         |
-
-**Important Notes**:
-
-1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
-
-2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
-
-3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
-
-4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
-
-**Example: Extract Latest 1000 Orders**
-
-```typescript
-const latestOrders = await orchestrator.extract({
-  query: ORDERS_QUERY,
-  resultPath: 'orders.edges.node',
-  variables: {
-    retailerId,
-    statuses: ['BOOKED', 'ALLOCATED'],
-  },
-  direction: 'backward', // Start from newest
-  maxRecords: 1000, // Stop after 1000 records
-  pageSize: 100, // 100 per page = 10 pages
-});
-
-// latestOrders.data[0] is the newest order
-// latestOrders.data[999] is the 1000th newest order
-```
-
-**When to Use Forward vs Backward**:
-
-```typescript
-// ✅ Forward (default) - For incremental sync
-const incrementalData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    dateRangeFilter: { from: lastSyncTime, to: now },
-  },
-  // direction defaults to 'forward'
-  // Processes oldest → newest for proper sequencing
-});
-
-// ✅ Backward - For "latest N records" use cases
-const latestData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  direction: 'backward',
-  maxRecords: 100, // Just get latest 100
-  // Gets newest → oldest
-});
-```
-
-**Pagination Variables Reference**:
-
-| Variable | Forward     | Backward    | Injected By  | Notes                    |
-| -------- | ----------- | ----------- | ------------ | ------------------------ |
-| `first`  | ✅ Used     | ❌ Not used | Orchestrator | From `pageSize`          |
-| `after`  | ✅ Used     | ❌ Not used | Orchestrator | From cursor (last edge)  |
-| `last`   | ❌ Not used | ✅ Used     | Orchestrator | From `pageSize`          |
-| `before` | ❌ Not used | ✅ Used     | Orchestrator | From cursor (first edge) |
-
-**Common Mistakes to Avoid**:
-
-```typescript
-// ❌ WRONG - Don't pass pagination variables
-const result = await orchestrator.extract({
-  variables: {
-    last: 200, // ❌ Orchestrator will override this
-    before: cursor, // ❌ Orchestrator manages cursor
-  },
-  direction: 'backward',
-});
-
-// ✅ CORRECT - Let orchestrator inject pagination
-const result = await orchestrator.extract({
-  variables: {
-    retailerId, // ✅ Your business variables only
-  },
-  pageSize: 200, // ✅ Orchestrator uses this for last/before
-  direction: 'backward',
-});
-```
-
-#### Optional: Reverse Pagination
-
-- Default: forward ($first/$after) + pageInfo.hasNextPage.
-- Reverse: use $last/$before and pageInfo.hasPreviousPage in the query, and set direction='backward' in the orchestrator call.
-
-GraphQL:
-
-```graphql
-query GetInventoryPositionsBackward(
-  $catalogues: [InventoryCatalogueKey]
-  $dateRangeFilter: DateRange
-  $last: Int!
-  $before: String
-) {
-  inventoryPositions(
-    catalogues: $catalogues
-    updatedOn: $dateRangeFilter
-    last: $last
-    before: $before
-  ) {
-    edges {
-      cursor
-      node {
-        id
-        ref
-        updatedOn
-      }
-    }
-    pageInfo {
-      hasPreviousPage
-    }
-  }
-}
-```
-
-SDK:
-
-```typescript
-await orchestrator.extract({
-  query: INVENTORY_POSITIONS_BACKWARD_QUERY,
-  resultPath: 'inventoryPositions.edges.node',
-  variables: { catalogues, dateRangeFilter },
-  pageSize,
-  direction: 'backward',
-});
-```
-
----
-
-## Testing Checklist
-
-**Before production deployment:**
-
-### 1. Schema Validation
-
-- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
-- [ ] Run `npx fc-connect validate-schema --mapping ./config/inventory-positions.export.csv.json --schema ./fluent-schema.json`
-- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/inventory-positions.export.csv.json --schema ./fluent-schema.json`
-- [ ] Verify all `source` paths in mapping exist in GraphQL schema
-- [ ] Verify query structure matches schema (fields, types, filters)
-
-### 2. Extraction Testing
-
-- [ ] Test with small dataset first (maxRecords=10)
-- [ ] Verify ExtractionOrchestrator pagination works correctly
-- [ ] Test with multiple pages of data (verify cursor handling)
-- [ ] Verify date range filtering (updatedOn filter)
-- [ ] Test empty result handling (no records in date range)
-- [ ] Verify extraction stops at maxRecords limit
-
-### 3. Mapping Testing
-
-- [ ] Verify required fields are populated
-- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
-- [ ] Test custom resolvers with edge cases (if any)
-- [ ] Verify nested field extraction
-- [ ] Test with null/missing fields
-- [ ] Verify mapping error collection works
-
-### 4. CSV Generation Testing
-
-- [ ] Verify CSV structure matches expected format
-- [ ] Test CSV validation against schema (if applicable)
-- [ ] Verify header row is present and correct
-- [ ] Test with large datasets (>1000 records)
-- [ ] Verify UTF-8 encoding
-- [ ] Test special character handling (commas, quotes, newlines)
-
-### 5. S3 Upload Testing
-
-- [ ] Test S3 connection and authentication
-- [ ] Verify file upload to correct bucket and path
-- [ ] Test file naming convention (timestamp format)
-- [ ] Verify S3 object metadata
-- [ ] Test upload retry logic (simulate network failure)
-- [ ] Verify file permissions and ACLs
-
-### 6. State Management Testing
-
-- [ ] Verify overlap buffer prevents missed records (60-second default)
-- [ ] Test state recovery after extraction failure
-- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
-- [ ] Test first run with no previous state (uses fallbackStartDate)
-- [ ] Verify state update only happens on successful upload
-- [ ] Test manual date override (doesn't update state)
-
-### 7. Job Tracking Testing
-
-- [ ] Test job creation with JobTracker
-- [ ] Verify job status updates at each stage
-- [ ] Test job completion with metadata
-- [ ] Test job failure handling
-- [ ] Query job status via webhook endpoint
-- [ ] Verify job status persists in KV store
-
-### 8. Error Handling Testing
-
-- [ ] Test with invalid GraphQL query
-- [ ] Test with mapping errors (invalid field paths)
-- [ ] Test with S3 connection failures
-- [ ] Test with authentication failures
-- [ ] Test with network timeouts
-- [ ] Verify error logging includes context (jobId, stage, error details)
-- [ ] Test error threshold logic (if applicable)
-
-### 9. Staging Environment Testing
-
-- [ ] Run full extraction in staging environment
-- [ ] Verify CSV file format with downstream system
-- [ ] Monitor extraction duration and resource usage
-- [ ] Test with production-like data volumes
-- [ ] Verify no performance degradation over time
-
-### 10. Integration Testing
-
-- [ ] Test scheduled workflow (cron trigger)
-- [ ] Test ad hoc webhook trigger
-- [ ] Test job status query webhook
-- [ ] Verify activation variables are read correctly
-- [ ] Test with different extraction modes (incremental, date range)
-- [ ] End-to-end test: trigger → extract → transform → upload → verify file
-
----
-
-## Recommended Project Structure
-
-**Organize workflows by trigger type in separate files:**
-
-```
-inventory-positions-extraction/
-├── index.ts                              # Entry point - exports all workflows
-└── src/
-    ├── workflows/
-    │   ├── scheduled/
-    │   │   └── daily-inventory-positions-extraction.ts  # Scheduled: Daily extraction
-    │   │
-    │   └── webhook/
-    │       ├── adhoc-inventory-positions-extraction.ts  # Webhook: Manual trigger
-    │       └── job-status-check.ts                     # Webhook: Status query
-    │
-    ├── services/
-    │   └── inventory-positions-extraction.service.ts    # Shared orchestration logic (reusable)
-    │
-    └── config/
-        └── inventory-positions.export.csv.json         # Mapping configuration
-```
-
-**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
-
-**Trigger Types:**
-- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
-- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
-- **`workflow()`** → Durable workflows (advanced, rarely used)
-
-**Execution Steps (chained to triggers):**
-- **`http()`** → External API calls (chained from schedule/webhook)
-- **`fn()`** → Internal processing (chained from schedule/webhook)
-
----
-
-## Monitoring & Alerting
-
-### Success Response Example
-
-```json
-{
-  "success": true,
-  "jobId": "SCHEDULED_IP_20251102_140000_abc123",
-  "recordsExtracted": 1523,
-  "fileName": "inventory-positions-2025-11-02T14-00-00-000Z.csv",
-  "s3Path": "s3://bucket/inventory-positions/inventory-positions-2025-11-02T14-00-00-000Z.csv",
-  "metrics": {
-    "extractionDurationMs": 12543,
-    "totalPages": 8,
-    "pageSize": 200,
-    "mappingErrors": 0,
-    "fileSizeBytes": 524288,
-    "uploadDurationMs": 1234
-  },
-  "timestamps": {
-    "extractionStart": "2025-11-02T14:00:00.000Z",
-    "extractionEnd": "2025-11-02T14:00:12.543Z",
-    "uploadComplete": "2025-11-02T14:00:13.777Z"
-  },
-  "state": {
-    "previousTimestamp": "2025-11-02T13:00:00.000Z",
-    "newTimestamp": "2025-11-02T13:59:58.123Z",
-    "stateUpdated": true,
-    "overlapBufferSeconds": 60
-  }
-}
-```
-
-### Error Response Example
-
-```json
-{
-  "success": false,
-  "jobId": "ADHOC_IP_20251102_140500_xyz789",
-  "error": "S3 upload failed: Connection timeout",
-  "errorCategory": "NETWORK",
-  "recordsExtracted": 0,
-  "stage": "s3_upload",
-  "details": {
-    "message": "Failed to upload file after 3 retry attempts",
-    "retryAttempts": 3,
-    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
-  },
-  "state": {
-    "stateUpdated": false,
-    "willRetryNextRun": true,
-    "note": "State not advanced - next extraction will retry same time window"
-  }
-}
-```
-
-### Key Metrics to Track
-
-```typescript
-const METRICS = {
-  // Extraction Performance
-  extractionDurationMs: Date.now() - extractionStart,
-  recordCount: records.length,
-  pageCount: extractionResult.stats.totalPages,
-  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
-
-  // Transformation Performance
-  transformedCount: transformedRecords.length,
-  failedCount: mappingErrors.length,
-  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
-
-  // File Generation
-  fileSizeMB: (csvContent.length / (1024 * 1024)).toFixed(2),
-
-  // Upload Performance
-  uploadDurationMs: uploadEnd - uploadStart,
-  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
-
-  // State Management
-  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
-  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
-};
-
-log.info('Extraction metrics', metrics);
-```
-
-### Alert Thresholds
-
-```typescript
-const ALERT_THRESHOLDS = {
-  // Duration Alerts
-  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
-  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
-  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
-
-  // Error Rate Alerts
-  MAX_ERROR_RATE: 0.05, // 5% mapping errors
-  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
-
-  // Volume Alerts
-  MAX_RECORDS_PER_RUN: 100000,
-  MIN_RECORDS_WARNING: 0, // Alert if no records found
-  MAX_FILE_SIZE_MB: 150, // 150MB
-
-  // State Alerts
-  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
-  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
-};
-
-// Check thresholds
-if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
-  log.warn('Extraction duration exceeded threshold', {
-    duration: metrics.extractionDurationMs,
-    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
-    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
-  });
-}
-```
-
-### Monitoring Dashboard Queries
-
-**Versori Platform Logs Query:**
-
-```
-# Successful extractions
-log_level:info AND message:"Extraction complete" AND jobId:*
-
-# Failed extractions
-log_level:error AND message:"Extraction workflow failed" AND jobId:*
-
-# Performance issues
-extractionDurationMs:>300000 OR uploadDurationMs:>120000
-
-# High error rates
-errorRate:>5
-
-# State management issues
-stateUpdated:false AND success:true
-```
-
-### Common Issues and Solutions
-
-**Issue**: "Extraction timeout after 10 minutes"
-
-- **Cause**: Too many records in single extraction
-- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
-- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
-
-**Issue**: "Mapping errors for 50% of records"
-
-- **Cause**: Schema mismatch between GraphQL response and mapping config
-- **Fix**: Run schema validation, update mapping config paths
-- **Prevention**: Use `npx fc-connect validate-schema` before deployment
-
-**Issue**: "S3 connection timeout"
-
-- **Cause**: Network issues, firewall, or connection pool exhaustion
-- **Fix**: Check S3 credentials, verify network connectivity
-- **Prevention**: Implement connection health checks, monitor connection status
-
-**Issue**: "State not updating after successful extraction"
-
-- **Cause**: KV write failure or intentional retry logic
-- **Fix**: Check KV logs, verify state update code executed
-- **Prevention**: Add KV write verification, log state updates explicitly
-
-**Issue**: "First run exceeds record limits"
-
-- **Cause**: No previous timestamp, fetches all historical records
-- **Fix**: Set fallbackStartDate close to current date, apply additional filters
-- **Prevention**: Use appropriate fallbackStartDate for initial runs
-
-**Issue**: "Excessive duplicate records in output"
-
-- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
-- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
-- **Prevention**: Monitor duplicate rates, verify state update logic
-
----
-
-## Troubleshooting Quick Reference
-
-| Error Message | Likely Cause | Solution |
-|--------------|--------------|----------|
-| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
-| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
-| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
-| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
-| "S3 authentication failed" | Invalid credentials | Verify S3 credentials in activation variables |
-| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
-| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
-| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
-| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
-| "CSV generation failed" | Format-specific error | Check CSV generation logic, validate output |
-
----
+---
+template_id: tpl-extract-inventory-positions-to-s3-csv
+canonical_filename: template-extraction-inventory-positions-to-s3-csv.md
+version: 2.0.0
+sdk_version: ^0.1.39
+runtime: versori
+direction: extraction
+source: fluent-graphql
+destination: s3-csv
+entity: inventory-positions
+format: csv
+logging: versori
+status: stable
+features:
+  - memory-management
+  - enhanced-logging
+  - pagination-progress
+---
+
+# Template: Extraction - Inventory Positions to S3 CSV
+
+**Template Version:** 2.0.0
+**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
+**Last Updated:** 2025-01-24
+**Deployment Target:** Versori Platform
+
+**🆕 Version 2.0.0 Enhancements:**
+- ✅ **Memory Management** - Clear large result sets after processing batches
+- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
+- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
+
+---
+
+## 📚 STEP 1: Load These Docs (Human Checklist)
+
+1. REQUIRED (load all)
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+
+Copy-paste list (open these):
+fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+
+---
+
+## 📋 STEP 2: Tell Your AI (Prompt)
+
+Copy/paste this prompt into your AI tool after loading the documentation above:
+
+```
+I need a Versori scheduled extractor that:
+
+1) Queries Fluent Commerce GraphQL for Inventory Positions with auto-pagination
+2) Supports incremental runs via KV state (with an overlap buffer)
+3) Transforms results using UniversalMapper per mapping JSON
+4) Generates CSV and uploads to S3
+5) Tracks progress with JobTracker and exposes a job-status webhook
+6) Uses native Versori log (LoggingService removed - use native log)
+
+Use the loaded docs to fill in SDK specifics and best practices.
+Keep the structure identical to the template; only adapt where needed.
+```
+
+---
+
+## 📋 Template Overview
+
+This connector runs on the Versori platform. Most operational settings (Fluent account/connection, S3 credentials, schedule, page size/limits) are configured via activation variables. Data shape and logic (mapping JSON, CSV structure, GraphQL selection set/filters, validators/resolvers) are adjusted in code as needed. It extracts inventory positions from Fluent Commerce via GraphQL, transforms the data into CSV, and uploads the result to S3.
+
+### What This Template Does
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    EXTRACTION WORKFLOW                           │
+└─────────────────────────────────────────────────────────────────┘
+
+1. TRIGGER
+   ├─ Scheduled (Cron): Runs automatically every hour
+   ├─ Ad hoc (Webhook): Manual trigger with optional date override
+   └─ Status Query (Webhook): Check job progress
+
+2. EXTRACT (ExtractionOrchestrator)
+   ├─ Query Fluent GraphQL API for inventory positions
+   ├─ Auto-pagination (handles large datasets)
+   ├─ Apply date filters (incremental or manual range)
+   └─ Validate each record (optional)
+
+3. TRANSFORM (UniversalMapper)
+   ├─ Map GraphQL fields to CSV schema
+   ├─ Apply SDK resolvers (trim, uppercase, parseInt, etc.)
+   ├─ Extract nested data (catalogue)
+   └─ Handle transformation errors
+
+4. GENERATE CSV (CSVParserService)
+   ├─ Convert transformed records to CSV
+   ├─ Include headers
+   ├─ Handle special characters
+   └─ Generate timestamped filename
+
+5. UPLOAD (S3DataSource)
+   ├─ Connect to S3
+   ├─ Upload CSV file with retry logic
+   ├─ Set content type and metadata
+   └─ Verify upload success
+
+6. TRACK JOB (JobTracker)
+   ├─ Create job with unique ID
+   ├─ Update status at each step
+   ├─ Store job result in KV
+   └─ Enable status queries via webhook
+
+7. UPDATE STATE (VersoriKVAdapter)
+   ├─ Calculate max updatedOn from records
+   ├─ Store timestamp for next incremental run
+   ├─ Apply overlap buffer (prevent missed records)
+   └─ Skip update if manual date override
+```
+
+### Key Features
+
+- Job tracking with status queries
+- Execution modes: scheduled, ad hoc, status query
+- Uses ExtractionOrchestrator, UniversalMapper, JobTracker, CSVParserService
+- Error handling, retry logic
+- Reusable services suitable for similar use cases
+
+Note: JobTracker persists stage/status to Versori KV for visibility, job-status webhooks, and auditing. Recommended for production multi-step flows; can be skipped for trivial single-step utilities.
+
+### 📦 Package Information
+
+**SDK:** [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
+
+**Latest Version:** See [npm package page](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk) for current version
+
+```bash
+npm install @fluentcommerce/fc-connect-sdk
+```
+
+---
+
+**Templates are designed for direct deployment; customize via activation variables.**
+
+---
+
+## 📦 SDK Imports (Verified - Versori Optimized)
+
+```typescript
+// ✅ VERIFIED IMPORTS - These match actual SDK exports
+import { Buffer } from 'node:buffer';
+import {
+  createClient, // Universal client factory (auto-detects Versori/Node/Deno)
+  ExtractionOrchestrator, // High-level extraction with auto-pagination
+  JobTracker, // Job status tracking in KV store
+  UniversalMapper, // Field mapping with SDK resolvers
+  CSVParserService, // CSV generation (NOT CSVBuilder!)
+  S3DataSource, // S3 operations with retry logic
+} from '@fluentcommerce/fc-connect-sdk';
+
+// Versori platform imports
+import { schedule, webhook, http, fn } from '@versori/run';
+```
+
+**Note:** All imports are from actual SDK exports - this code compiles and runs as-is.
+
+**✅ VERSORI PLATFORM - Use Native Logs:**
+
+- Use `log` from context: `const { log } = ctx;`
+- Don't import or use LoggingService for Versori connectors
+- Native Versori logs are simpler and automatically integrated with platform monitoring
+
+---
+
+## ⚙️ Activation Variables
+
+**Configuration is driven by activation variables - modify these instead of code:**
+
+```json
+{
+  "catalogueRef": "DEFAULT_CATALOGUE",
+  "s3BucketName": "inventory-analytics-exports",
+  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
+  "awsSecretAccessKey": "********",
+  "awsRegion": "us-east-1",
+  "s3Prefix": "inventory-positions/daily/",
+  "fileNamePrefix": "inventorypositions",
+  "pageSize": 200,
+  "maxRecords": 100000,
+  "overlapBufferSeconds": 60
+}
+```
+
+Note: Webhook security is enforced via Versori connections. Configure auth on the connection; reference it in `webhook({ connection: '...' })`.
+
+### Variable Explanations
+
+| Variable               | Purpose                       | Default                      | Customization Hints              |
+| ---------------------- | ----------------------------- | ---------------------------- | -------------------------------- |
+| `catalogueRef`         | Filter by inventory catalogue | `DEFAULT_CATALOGUE`          | Change to filter by catalogue    |
+| `s3BucketName`         | Target S3 bucket              | -                            | Required - your analytics bucket |
+| `awsAccessKeyId`       | AWS access key                | -                            | Required - AWS credential        |
+| `awsSecretAccessKey`   | AWS secret key                | -                            | Required - AWS credential        |
+| `awsRegion`            | AWS region                    | `us-east-1`                  | Match bucket region              |
+| `s3Prefix`             | S3 key prefix                 | `inventory-positions/daily/` | Customize folder structure       |
+| `fileNamePrefix`       | CSV filename prefix           | `inventorypositions`         | Customize naming convention      |
+| `pageSize`             | Records per GraphQL page      | `200`                        | Increase for fewer API calls     |
+| `maxRecords`           | Total extraction limit        | `100000`                     | Safety limit - adjust for volume |
+| `overlapBufferSeconds` | Incremental safety window     | `60`                         | Prevents missed records          |
+
+---
+
+### 🔄 State Management & Incremental Sync
+
+**How incremental sync works:**
+
+1. **First Run:** Uses `DEFAULT_FALLBACK` date (2024-01-01T00:00:00Z)
+2. **Subsequent Runs:** Uses `lastInventoryPositionSync` timestamp from KV store
+3. **Overlap Buffer:** Subtracts 60 seconds to catch late-arriving records
+4. **State Update:** After successful upload, stores max `updatedOn` for next run
+
+**Incremental vs Manual Modes:**
+
+| Mode                        | When to Use          | State Update | Payload Example                                                |
+| --------------------------- | -------------------- | ------------ | -------------------------------------------------------------- |
+| **Incremental**             | Daily scheduled sync | ✅ Yes       | `{}` (empty - uses last sync)                                  |
+| **Manual Range**            | Historical backfill  | ❌ No        | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": false }` |
+| **Manual Range with State** | One-time catch-up    | ✅ Yes       | `{ "fromDate": "2024-01-01T00:00:00Z", "updateState": true }`  |
+
+**Why overlap buffer?**
+
+Records updated near the sync time might not appear in the query due to:
+
+- Clock drift between systems
+- Transaction timing in the database
+- GraphQL query execution timing
+
+The 60-second buffer ensures these edge-case records are captured in the next run, preventing data loss.
+
+**When to skip state update (`updateState: false`):**
+
+- Historical backfills (don't affect ongoing incremental sync)
+- Testing/debugging specific date ranges
+- Reprocessing old data without changing the sync pointer
+
+---
+
+### 📁 S3 Path Configuration
+
+**Note:** Folder paths and filename patterns are configured separately.
+
+- **Folder Path:** Configured via `s3Prefix` activation variable (e.g., `inventory-positions/daily/`)
+- **Filename Pattern:** Configured via `fileNamePrefix` activation variable (e.g., `inventorypositions`)
+
+**The workflow generates files like:** `{s3Prefix}{fileNamePrefix}-{timestamp}.csv`
+
+**Example:** With `s3Prefix="inventory-positions/daily/"` and `fileNamePrefix="inventorypositions"`, a generated file will look like:
+
+```
+inventory-positions/daily/inventorypositions-2025-10-27T18-30-45Z.csv
+```
+
+Format may vary slightly (colons replaced; includes Z).
+
+**Note:** To change the upload folder, modify the `s3Prefix` activation variable (not in code).
+
+---
+
+### Auto-pagination and limits (ExtractionOrchestrator)
+
+**What:** ExtractionOrchestrator handles GraphQL Relay cursor-based pagination automatically.
+
+**Why:** Prevents manual pagination loop code, handles large datasets efficiently.
+
+**How:** You configure `pageSize` and `maxRecords`; the orchestrator injects `$first` and `$after` variables automatically and loops until `pageInfo.hasNextPage === false` or `maxRecords` is reached.
+
+**Critical:** Your query MUST include `edges { cursor }` and `pageInfo { hasNextPage }` fields, or pagination will fail.
+
+#### Configuration Parameters
+
+| Parameter        | Purpose                        | Example                           | Effect                     |
+| ---------------- | ------------------------------ | --------------------------------- | -------------------------- |
+| `pageSize`       | Records per GraphQL request    | `200`                             | Controls `first` variable  |
+| `maxRecords`     | Total extraction limit         | `100000`                          | Hard stop across all pages |
+| `resultPath`     | Where records live in response | `"inventoryPositions.edges.node"` | Flattening path            |
+| `validateItem()` | Optional record filter         | `(item) => !!item.ref`            | Skips invalid records      |
+
+#### GraphQL Query Requirements
+
+**Your query MUST include these pagination fields:**
+
+```graphql
+query GetInventoryPositions(
+  $catalogues: [InventoryCatalogueKey]
+  $dateRangeFilter: DateRange
+  $first: Int! # ← Orchestrator injects this
+  $after: String # ← Orchestrator injects this
+) {
+  inventoryPositions(
+    catalogues: $catalogues
+    updatedOn: $dateRangeFilter
+    first: $first # ← Pagination page size
+    after: $after # ← Pagination cursor
+  ) {
+    edges {
+      # ← REQUIRED: Relay connection structure
+      node {
+        # ← REQUIRED: Actual records here
+        id
+        ref
+        # ... your fields
+      }
+      cursor # ← REQUIRED: Orchestrator uses this for next page
+    }
+    pageInfo {
+      # ← REQUIRED: Orchestrator checks this
+      hasNextPage # ← REQUIRED: Tells orchestrator to continue
+    }
+  }
+}
+```
+
+---
+
+## 📄 Mapping Configuration
+
+**File:** `config/inventory-positions.export.csv.json`
+
+```json
+{
+  "name": "inventory-positions.export.csv",
+  "version": "1.0.0",
+  "description": "Inventory Positions → CSV Export Mapping",
+  "fields": {
+    "position_ref": {
+      "source": "ref",
+      "required": true,
+      "resolver": "sdk.trim"
+    },
+    "product_ref": {
+      "source": "productRef",
+      "required": true,
+      "resolver": "sdk.trim"
+    },
+    "location_ref": {
+      "source": "locationRef",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "on_hand": {
+      "source": "onHand",
+      "required": true,
+      "resolver": "sdk.parseInt"
+    },
+    "position_type": {
+      "source": "type",
+      "required": true,
+      "resolver": "sdk.uppercase"
+    },
+    "status": {
+      "source": "status",
+      "required": false,
+      "resolver": "sdk.uppercase"
+    },
+    "catalogue_ref": {
+      "source": "catalogue.ref",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "catalogue_name": {
+      "source": "catalogue.name",
+      "required": false,
+      "resolver": "sdk.trim"
+    },
+    "created_on": {
+      "source": "createdOn",
+      "required": true,
+      "resolver": "sdk.toString"
+    },
+    "updated_on": {
+      "source": "updatedOn",
+      "required": true,
+      "resolver": "sdk.toString"
+    }
+  }
+}
+```
+
+**AI Customization Hints:**
+
+- Add fields: Copy existing field config, change `source` path
+- Remove fields: Delete field from config
+- Change resolvers: Replace `sdk.trim` with `sdk.uppercase`, etc.
+- Nested fields: Use dot notation like `catalogue.ref`
+
+Note: Customize mapping by editing the JSON above; prefer built-in resolvers. See SDK Universal Mapping guide for advanced usage.
+
+## 🔧 Complete Production Code
+
+### File Structure
+
+**Production-ready modular structure:**
+
+```
+inventory-positions-to-s3-csv/
+├── src/
+│   ├── index.ts                             # Register workflows
+│   ├── workflows/
+│   │   └── inventory-positions-extraction.ts # ALL 3 workflows (scheduled + 2 webhooks)
+│   ├── services/
+│   │   └── extraction-orchestration.ts      # Main orchestration logic
+│   └── utils/
+│       └── job-id-generator.ts              # Job ID utility
+├── config/
+│   └── inventory-positions.export.csv.json  # Mapping configuration
+├── package.json
+└── tsconfig.json
+```
+
+---
+
+### 1. Entry Point (src/index.ts)
+
+```typescript
+/**
+ * Entry Point - Export all workflows for Versori platform
+ *
+ * This file exports all workflows to be registered with Versori.
+ * Each workflow is defined in its own file for better organization.
+ *
+ * DELEGATION PATTERN:
+ * - index.ts exports workflows (no business logic)
+ * - Workflows orchestrate execution (minimal logic)
+ * - Services contain all business logic (maximum reuse)
+ *
+ * AI CUSTOMIZATION:
+ * - Add new workflows by importing and exporting them
+ * - Remove workflows by commenting out exports
+ * - Change workflow names in import statements
+ */
+
+// Scheduled workflows
+export { scheduledInventoryPositionsExtraction } from './workflows/scheduled/inventory-positions-extraction';
+
+// Webhook workflows
+export { adhocInventoryPositionsExtraction } from './workflows/webhook/adhoc-inventory-positions-extraction';
+export { inventoryPositionsJobStatus } from './workflows/webhook/job-status-check';
+```
+
+---
+
+### 2. Workflows (src/workflows/inventory-positions-extraction.ts)
+
+```typescript
+/**
+ * Workflows - Defines 3 execution patterns for inventory positions extraction
+ *
+ * WORKFLOW 1: Scheduled (Cron) - Runs automatically every hour
+ * WORKFLOW 2: Ad hoc (Webhook) - Manual trigger with optional date override
+ * WORKFLOW 3: Job Status (Webhook) - Query job progress
+ *
+ * AI CUSTOMIZATION HINTS:
+ * - Change schedule: Modify cron expression in schedule()
+ * - Add filtering: Pass additional params to executeInventoryPositionExtraction()
+ * - Change response format: Modify return object structure
+ */
+
+import { schedule, webhook, http, fn } from '@versori/run';
+import { executeInventoryPositionExtraction, getJobStatus } from '../services/extraction-orchestration';
+import { generateJobId } from '../utils/job-id-generator';
+
+/**
+ * WORKFLOW 1: Scheduled Extraction
+ *
+ * Purpose: Automated hourly extraction for incremental sync
+ * Trigger: Cron schedule (every hour at minute 0)
+ * State Update: Always updates lastSync timestamp
+ *
+ * AI CUSTOMIZATION:
+ * - Change schedule: Replace '0 * * * *' with your cron expression
+ *   Examples:
+ *   - Every 30 min: '*/30 * * * *'
+ *   - Daily at 2 AM: '0 2 * * *'
+ *   - Every 15 min: '*/15 * * * *'
+ */
+export const scheduledInventoryPositionsExtraction = schedule(
+  'scheduled-inventory-positions-extraction',
+  '0 * * * *',  // ← CUSTOMIZE: Cron expression
+  http('execute-scheduled-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
+    const { log } = ctx;
+    const executionStartTime = Date.now();
+
+    // Generate unique job ID for tracking
+    // Format: SCHEDULED_IP_YYYYMMDD_HHMMSS_random
+    const jobId = generateJobId('SCHEDULED', 'INVENTORY_POSITIONS');
+
+    log.info('🚀 [WORKFLOW] Scheduled extraction triggered', { jobId });
+
+    try {
+      // Execute main workflow (extraction → transform → upload)
+      const result = await executeInventoryPositionExtraction(ctx, {
+        jobId,
+        triggeredBy: 'schedule',
+        updateState: true,        // Always update state for scheduled runs
+      });
+
+      const executionDuration = Date.now() - executionStartTime;
+
+      log.info('✅ [WORKFLOW] Scheduled extraction completed', {
+        jobId,
+        recordCount: result.recordsExtracted,
+        fileName: result.fileName,
+        duration: executionDuration,
+      });
+
+      return { ...result, duration: executionDuration };
+
+    } catch (error: any) {
+      const executionDuration = Date.now() - executionStartTime;
+
+      log.error('❌ [WORKFLOW] Scheduled extraction failed', {
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',
+        duration: executionDuration,
+      });
+
+      return {
+        success: false,
+        jobId,
+        error: error.message,
+        duration: executionDuration,
+        recommendation: getErrorRecommendation(error),
+      };
+    }
+  }));
+
+/**
+ * WORKFLOW 2: Ad hoc Extraction (Manual Trigger)
+ *
+ * Purpose: Manual extraction with optional date range override
+ * Trigger: Webhook POST to /webhooks/inventory-positions-adhoc
+ * State Update: Optional (controlled by request payload)
+ *
+ * WEBHOOK PAYLOAD EXAMPLES:
+ *
+ * 1. Incremental (use last sync timestamp):
+ *    {}
+ *
+ * 2. Date range (manual override):
+ *    {
+ *      "fromDate": "2025-01-01T00:00:00Z",
+ *      "toDate": "2025-01-31T23:59:59Z",
+ *      "updateState": false
+ *    }
+ *
+ * AI CUSTOMIZATION:
+ * - Add request validation
+ * - Add authentication check
+ * - Add custom filters from payload
+ */
+export const adhocInventoryPositionsExtraction = webhook(
+  'inventory-positions-adhoc',
+  { connection: 'inventory-positions-adhoc', response: { mode: 'sync' } }
+)
+.then(
+  http('execute-adhoc-extraction', { connection: 'fluent_commerce' }, async (ctx) => {
+    const { data, log } = ctx;
+    const executionStartTime = Date.now();
+
+    // Generate unique job ID
+    const jobId = generateJobId('ADHOC', 'INVENTORY_POSITIONS');
+
+    // SECURITY: Authentication is enforced by Versori connection configuration
+    // Configure auth on the connection and reference it in webhook({ connection: '...' })
+
+    // Extract optional date override from webhook payload
+    const fromDate = data.fromDate as string | undefined;
+    const toDate = data.toDate as string | undefined;
+    const updateState = data.updateState === true;  // Default false; advance state only if explicitly true
+
+    log.info('🔔 [WORKFLOW] Ad hoc extraction triggered via webhook', {
+      jobId,
+      hasDateOverride: !!fromDate,
+      fromDate: fromDate || 'not specified',
+      toDate: toDate || 'not specified',
+      updateState
+    });
+
+    try {
+      // Execute main workflow with optional overrides
+      const result = await executeInventoryPositionExtraction(ctx, {
+        jobId,
+        triggeredBy: 'webhook',
+        fromDate,              // Optional: override start date
+        toDate,                // Optional: override end date
+        updateState,           // Optional: skip state update for historical queries
+      });
+
+      const executionDuration = Date.now() - executionStartTime;
+
+      log.info('✅ [WORKFLOW] Ad hoc extraction completed', {
+        jobId,
+        recordCount: result.recordsExtracted,
+        fileName: result.fileName,
+        isManualOverride: !!fromDate,
+        stateUpdated: result.stateUpdated,
+        duration: executionDuration,
+      });
+
+      // Return success with job details
+      return {
+        success: true,
+        jobId,
+        recordsExtracted: result.recordsExtracted,
+        fileName: result.fileName,
+        s3Path: result.s3Path,
+        statusUrl: `/webhooks/inventory-positions-job-status?jobId=${jobId}`,
+        duration: executionDuration,
+        dateRange: fromDate ? {
+          from: fromDate,
+          to: toDate || 'not specified',
+          updateState
+        } : undefined
+      };
+
+    } catch (error: any) {
+      const executionDuration = Date.now() - executionStartTime;
+
+      log.error('❌ [WORKFLOW] Ad hoc extraction failed', {
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',
+        duration: executionDuration,
+      });
+
+      return {
+        success: false,
+        jobId,
+        error: error.message,
+        duration: executionDuration,
+        recommendation: getErrorRecommendation(error),
+      };
+    }
+  }));
+
+/**
+ * WORKFLOW 3: Job Status Query
+ *
+ * Purpose: Check job progress and status
+ * Trigger: Webhook GET/POST to /webhooks/inventory-positions-job-status?jobId=xxx
+ * Returns: Current job status, stage, progress
+ *
+ * QUERY EXAMPLES:
+ *
+ * 1. HTTP GET:
+ *    GET /webhooks/inventory-positions-job-status?jobId=ADHOC_IP_20251027_183045_abc123
+ *
+ * 2. HTTP POST:
+ *    POST /webhooks/inventory-positions-job-status
+ *    { "jobId": "ADHOC_IP_20251027_183045_abc123" }
+ */
+export const inventoryPositionsJobStatus = webhook(
+  'inventory-positions-job-status',
+  { connection: 'inventory-positions-job-status', response: { mode: 'sync' } }
+)
+.then(
+  fn('query-job-status', async (ctx) => {
+    const { data, log, openKv } = ctx;
+    // SECURITY: Authentication is enforced by Versori connection configuration
+    // Configure auth on the connection and reference it in webhook({ connection: '...' })
+
+    // Get jobId from query param or POST body
+    const jobId = data.jobId as string;
+
+    if (!jobId) {
+      log.error('❌ [WORKFLOW] Job ID not provided in request');
+      return {
+        success: false,
+        error: 'Job ID is required. Provide jobId in query param or request body.',
+        recommendation: 'Include jobId in your request: { "jobId": "ADHOC_IP_20251027_183045_abc123" }'
+      };
+    }
+
+    log.info('🔍 [WORKFLOW] Querying job status', { jobId });
+
+    try {
+      // Query job status from KV store
+      const status = await getJobStatus(openKv(':project:'), jobId, log);
+
+      if (!status) {
+        log.info('⚠️ [WORKFLOW] Job not found', { jobId });
+        return {
+          success: false,
+          error: 'Job not found',
+          jobId,
+          recommendation: 'Verify the jobId is correct. Jobs may expire after 7 days.'
+        };
+      }
+
+      log.info('✅ [WORKFLOW] Job status retrieved', { jobId, status: status.status });
+
+      return {
+        success: true,
+        jobId,
+        ...status
+      };
+
+    } catch (error: any) {
+      log.error('❌ [WORKFLOW] Failed to query job status', {
+        jobId,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+        errorType: error instanceof Error ? error.constructor.name : 'Error',
+      });
+
+      return {
+        success: false,
+        jobId,
+        error: error.message,
+        recommendation: getErrorRecommendation(error),
+      };
+    }
+  }));
+
+/**
+ * Helper: Get error-specific recommendations
+ */
+function getErrorRecommendation(error: any): string {
+  const message = error instanceof Error ? error.message : String(error);
+
+  if (message.includes('ECONNREFUSED') || message.includes('ETIMEDOUT')) {
+    return 'Network connectivity issue. Check firewall rules and network configuration.';
+  }
+  if (message.includes('authentication') || message.includes('401')) {
+    return 'Authentication failed. Verify OAuth2 credentials in connection configuration.';
+  }
+  if (message.includes('S3') || message.includes('bucket')) {
+    return 'S3 operation failed. Check bucket name, region, and AWS credentials.';
+  }
+  if (message.includes('GraphQL') || message.includes('query')) {
+    return 'GraphQL query failed. Verify query syntax and field availability in schema.';
+  }
+  if (message.includes('mapping') || message.includes('transform')) {
+    return 'Data transformation failed. Check mapping configuration and source data structure.';
+  }
+  if (message.includes('KV') || message.includes('state')) {
+    return 'State management failed. Check Versori KV availability and permissions.';
+  }
+
+  return 'Check logs for detailed error information and stack trace.';
+}
+```
+
+---
+
+## 3. Main Orchestration Service (src/services/extraction-orchestration.ts)
+
+```typescript
+/**
+ * MAIN ORCHESTRATION SERVICE
+ *
+ * This is the heart of the extraction workflow. It coordinates all steps:
+ * 1. Initialize clients and services
+ * 2. Determine date range (incremental vs manual)
+ * 3. Extract data using ExtractionOrchestrator
+ * 4. Transform using UniversalMapper
+ * 5. Generate CSV using CSVParserService
+ * 6. Upload to S3
+ * 7. Track job progress with JobTracker
+ * 8. Update state for next run
+ *
+ * NAMING PATTERN (consistent across all use cases):
+ * - Interface: {Entity}ExtractionParams (e.g., InventoryPositionExtractionParams)
+ * - Result: {Entity}ExtractionResult (e.g., InventoryPositionExtractionResult)
+ * - Main function: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
+ *
+ * AI CUSTOMIZATION HINTS:
+ * - Change entity: Replace "InventoryPosition" with "Order", "Product", etc.
+ * - Change output: Replace CSVParserService with XMLBuilder
+ * - Change destination: Replace S3DataSource with SftpDataSource
+ * - Add steps: Insert new service calls between existing steps
+ */
+
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  CSVParserService,
+  S3DataSource,
+} from '@fluentcommerce/fc-connect-sdk';
+
+import mappingConfig from '../../config/inventory-positions.export.csv.json' with { type: 'json' };
+
+// ✅ VERSORI PLATFORM: Use native log from context, not LoggingService
+
+/**
+ * Parameters for extraction workflow
+ *
+ * NAMING: {Entity}ExtractionParams
+ */
+export interface InventoryPositionExtractionParams {
+  jobId: string;
+  triggeredBy: 'schedule' | 'webhook';
+  fromDate?: string; // Optional: manual date override
+  toDate?: string; // Optional: manual date override
+  updateState: boolean; // Whether to update lastSync timestamp
+
+  // AI CUSTOMIZATION: Add filters specific to entity
+  positionTypes?: string[]; // e.g., ['DEFAULT', 'SEASONAL']
+  catalogueRef?: string; // e.g., 'DEFAULT_CATALOGUE'
+}
+
+/**
+ * Result from extraction workflow
+ *
+ * NAMING: {Entity}ExtractionResult
+ */
+export interface InventoryPositionExtractionResult {
+  success: boolean;
+  jobId: string;
+  recordsExtracted: number;
+  fileName?: string;
+  s3Path?: string;
+  error?: string;
+  errors?: any[];
+  isManualOverride?: boolean;
+  stateUpdated?: boolean;
+  newTimestamp?: string;
+}
+
+/**
+ * GraphQL Query for Inventory Positions
+ *
+ * NAMING: {ENTITY}_EXTRACTION_QUERY (uppercase constant)
+ */
+const INVENTORY_POSITIONS_EXTRACTION_QUERY = `
+  query GetInventoryPositions(
+    $catalogues: [InventoryCatalogueKey]
+    $dateRangeFilter: DateRange
+    $productRefs: [String!]
+    $types: [String!]
+    $first: Int!
+    $after: String
+  ) {
+    inventoryPositions(
+      catalogues: $catalogues
+      updatedOn: $dateRangeFilter
+      productRef: $productRefs
+      type: $types
+      first: $first
+      after: $after
+    ) {
+      edges {
+        node {
+          id
+          ref
+          productRef
+          locationRef
+          onHand
+          type
+          status
+          catalogue {
+            ref
+            name
+          }
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+
+/**
+ * Query job status from KV store
+ *
+ * ✅ VERSORI PLATFORM: Uses native log from context (passed as parameter)
+ */
+export async function getJobStatus(
+  kv: any, // ✅ Versori KV (compatible with JobTracker's KVAdapter interface)
+  jobId: string,
+  log: any // ✅ Native Versori log from context
+): Promise<any | undefined> {
+  try {
+    const tracker = new JobTracker(kv, log);
+    return await tracker.getJob(jobId);
+  } catch (error: any) {
+    log.error('Failed to get job status', {
+      jobId,
+      message: error instanceof Error ? error.message : String(error),
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'Error',
+    });
+    return undefined;
+  }
+}
+
+/**
+ * MAIN ORCHESTRATION FUNCTION
+ *
+ * NAMING: execute{Entity}Extraction (e.g., executeInventoryPositionExtraction)
+ *
+ * This function implements the complete workflow in steps.
+ * Each step is clearly commented for AI understanding.
+ */
+export async function executeInventoryPositionExtraction(
+  ctx: any,
+  params: InventoryPositionExtractionParams
+): Promise<InventoryPositionExtractionResult> {
+  // ✅ VERSORI PLATFORM: Extract native log from context
+  const { log, openKv, activation } = ctx;
+  const { jobId, triggeredBy, fromDate, toDate, updateState } = params;
+
+  // Open KV store for state management and job tracking
+  // ✅ Pass raw Versori KV directly - it matches KVAdapter interface
+  // ✅ Pass native log to JobTracker
+  const kv = openKv(':project:');
+  const tracker = new JobTracker(kv, log);
+
+  try {
+    // ═══════════════════════════════════════════════════════════
+    // STEP 1/8: Initialize Job Tracking
+    // ═══════════════════════════════════════════════════════════
+    const step1Start = Date.now();
+    log.info('📋 [STEP 1/8] Initializing job tracking', { jobId });
+
+    await tracker.createJob(jobId, {
+      triggeredBy,
+      hasDateOverride: !!fromDate,
+      fromDate,
+      toDate,
+      updateStateAfterRun: updateState,
+    });
+
+    log.info('✅ [STEP 1/8] Job tracking initialized', {
+      jobId,
+      duration: Date.now() - step1Start,
+    });
+
+    // ═══════════════════════════════════════════════════════════
+    // STEP 2/8: Initialize Fluent Client
+    // ═══════════════════════════════════════════════════════════
+    const step2Start = Date.now();
+    log.info('🔌 [STEP 2/8] Initializing Fluent Commerce client', { jobId });
+
+    const client = await createClient(ctx);
+
+    if (!client) {
+      throw new Error('Failed to create Fluent Commerce client - check connection configuration');
+    }
+
+    log.info('✅ [STEP 2/8] Client initialized', {
+      jobId,
+      duration: Date.now() - step2Start,
+    });
+
+    // ═══════════════════════════════════════════════════════════
+    // STEP 3/8: Determine Date Range
+    // ═══════════════════════════════════════════════════════════
+    const step3Start = Date.now();
+    log.info('📅 [STEP 3/8] Determining date range for extraction', { jobId });
+
+    // State key for incremental sync tracking
+    // NAMING: last{Entity}Sync (e.g., lastInventoryPositionSync)
+    const STATE_KEY = 'lastInventoryPositionSync';
+    const DEFAULT_FALLBACK = '2024-01-01T00:00:00Z';
+    const OVERLAP_BUFFER_SECONDS = parseInt(
+      activation.getVariable('overlapBufferSeconds') || '60',
+      10
+    );
+
+    let dateRangeFilter: { from?: string; to?: string } | null = null;
+    const isManualOverride = !!fromDate;
+
+    if (isManualOverride) {
+      // Manual date override from webhook
+      dateRangeFilter = { from: fromDate, to: toDate };
+      log.info('Using manual date override', { fromDate, toDate });
+    } else {
+      // Incremental sync - get last sync timestamp
+      const rawLastRunTime = (await kv.get<string>(STATE_KEY)) || DEFAULT_FALLBACK;
+
+      // Apply overlap buffer (prevents missed records)
+      const bufferedLastRunTime = new Date(
+        new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_SECONDS * 1000
+      ).toISOString();
+
+      const effectiveEndTime = toDate || new Date().toISOString();
+
+      dateRangeFilter = {
+        from: bufferedLastRunTime,
+        to: effectiveEndTime, // End of extraction window
+      };
+
+      log.info('Using incremental sync with overlap buffer', {
+        rawLastRunTime,
+        bufferedLastRunTime,
+        effectiveEndTime,
+        overlapBufferSeconds: OVERLAP_BUFFER_SECONDS,
+      });
+    }
+
+    log.info('✅ [STEP 3/8] Date range determined', {
+      jobId,
+      fromDate: dateRangeFilter?.from,
+      toDate: dateRangeFilter?.to,
+      isManualOverride,
+      duration: Date.now() - step3Start,
+    });
+
+    // ═══════════════════════════════════════════════════════════
+    // STEP 4/8: Extract Data (ExtractionOrchestrator)
+    // ═══════════════════════════════════════════════════════════
+    const step4Start = Date.now();
+    log.info('📥 [STEP 4/8] Extracting data from Fluent Commerce', { jobId });
+
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 'extraction',
+      message: 'Extracting data with auto-pagination',
+    });
+
+    // Build catalogues array from config
+    const catalogueRef = params.catalogueRef || activation.getVariable('catalogueRef');
+    const catalogues = catalogueRef ? [{ ref: catalogueRef }] : [];
+
+    // Configure extraction
+    const pageSize = parseInt(activation.getVariable('pageSize') || '200', 10);
+    const maxRecords = parseInt(activation.getVariable('maxRecords') || '100000', 10);
+
+    // Initialize ExtractionOrchestrator
+    const orchestrator = new ExtractionOrchestrator(client, log);
+
+    // ? Enhanced: Extract context for progress logging
+    const dateRangeInfo = {
+      start: dateRangeFilter?.from || 'N/A',
+      end: dateRangeFilter?.to || 'N/A',
+      catalogues: catalogues.map((c: any) => c.ref).join(', ') || 'all',
+      types: params.positionTypes?.join(', ') || 'all'
+    };
+
+    // ? Enhanced: Start logging with context
+    log.info(`🔍 [ExtractionOrchestrator] Starting extraction`, {
+     query: 'inventoryPositions',
+     pageSize,
+     maxRecords,
+     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
+     catalogues: dateRangeInfo.catalogues,
+     positionTypes: dateRangeInfo.types,
+     jobId
+    });
+
+    // Execute extraction with auto-pagination
+    const extractionResult = await orchestrator.extract({
+      query: INVENTORY_POSITIONS_EXTRACTION_QUERY,
+      resultPath: 'inventoryPositions.edges.node',
+      variables: {
+        catalogues,
+        dateRangeFilter,
+        types: params.positionTypes,
+        // Note: Don't include 'first' or 'after' here; orchestrator injects them
+      },
+      pageSize,
+      maxRecords,
+      // Optional: validate each record
+      validateItem: (item: any) => {
+        return !!(item.ref && item.productRef);
+      },
+    });
+
+    const rawRecords = extractionResult.data;
+
+    log.info('✅ [STEP 4/8] Extraction completed', {
+      jobId,
+      totalRecords: extractionResult.stats.totalRecords,
+      totalPages: extractionResult.stats.totalPages,
+      validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
+      errors: extractionResult.errors ? extractionResult.errors.length : 0,
+      duration: Date.now() - step4Start,
+    });
+
+    // ? Enhanced: Completion logging with summary
+    log.info(`✅ [ExtractionOrchestrator] Extraction completed`, {
+     totalRecords: extractionResult.stats.totalRecords,
+     totalPages: extractionResult.stats.totalPages,
+     validRecords: extractionResult.stats.validRecords ?? rawRecords.length,
+     failedValidations: extractionResult.stats.failedValidations,
+     truncated: extractionResult.stats.truncated,
+     truncationReason: extractionResult.stats.truncationReason,
+     dateRange: `${dateRangeInfo.start} to ${dateRangeInfo.end}`,
+     jobId
+    });
+
+    if (extractionResult.errors && extractionResult.errors.length > 0) {
+      log.warn('Non-fatal extraction errors encountered', {
+        jobId,
+        errorCount: extractionResult.errors.length,
+        sampleErrors: extractionResult.errors.slice(0, 3),
+      });
+    }
+
+    // Handle empty result
+    if (rawRecords.length === 0) {
+      log.info('No records to process');
+
+      // Update state even with no records (prevents re-querying empty window)
+      if (updateState) {
+        await kv.set(STATE_KEY, new Date().toISOString());
+      }
+
+      await tracker.markCompleted(jobId, {
+        recordCount: 0,
+        message: 'No records to extract',
+      });
+
+      return {
+        success: true,
+        jobId,
+        recordsExtracted: 0,
+      };
+    }
+
+    // ═══════════════════════════════════════════════════════════
+    // STEP 5/8: Transform Data (UniversalMapper)
+    // ═══════════════════════════════════════════════════════════
+    const step5Start = Date.now();
+    log.info('🔄 [STEP 5/8] Transforming data with UniversalMapper', {
+      jobId,
+      recordCount: rawRecords.length,
+    });
+
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 'transformation',
+      message: `Transforming ${rawRecords.length} records`,
+    });
+
+    const mapper = new UniversalMapper(mappingConfig);
+    const mappingResult = await mapper.map(rawRecords);
+
+    if (!mappingResult.success) {
+      const mappingErrors = mappingResult.errors || ['Unknown mapping failure'];
+      log.error('[STEP 5/8] Mapping failed - terminating job', {
+        jobId,
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+
+      await tracker.markFailed(
+        jobId,
+        new Error(mappingErrors[0] || 'UniversalMapper returned unsuccessful result')
+      );
+
+      return {
+        success: false,
+        jobId,
+        recordsExtracted: 0,
+        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
+      };
+    }
+
+    const transformedRecords = Array.isArray(mappingResult.data) ? mappingResult.data : [];
+    const mappingErrors = mappingResult.errors || [];
+
+    if (mappingErrors.length > 0) {
+      log.warn('[STEP 5/8] Some records failed transformation', {
+        jobId,
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+    }
+
+    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
+      log.info('ℹ️ [MAPPING] Optional fields skipped (undefined values)', {
+        jobId,
+        skippedFields: mappingResult.skippedFields,
+        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
+      });
+    }
+
+    if (transformedRecords.length === 0) {
+      throw new Error('All records failed transformation');
+    }
+
+    log.info('✅ [STEP 5/8] Transformation complete', {
+      jobId,
+      successful: transformedRecords.length,
+      skippedRecords: rawRecords.length - transformedRecords.length,
+      duration: Date.now() - step5Start,
+    });
+
+    // ═══════════════════════════════════════════════════════════
+    // STEP 6/8: Generate CSV (CSVParserService)
+    // ═══════════════════════════════════════════════════════════
+    const step6Start = Date.now();
+    log.info('📄 [STEP 6/8] Generating CSV file', { jobId });
+
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 'csv_generation',
+      message: `Generating CSV for ${transformedRecords.length} records`,
+    });
+
+    // Initialize CSVParserService
+    const csvParser = new CSVParserService();
+
+    // Generate CSV content (synchronous, not async)
+    const csvContent = csvParser.stringify(transformedRecords, { headers: true });
+
+    // Generate filename
+    const fileNamePrefix = activation.getVariable('fileNamePrefix') || 'inventorypositions';
+    const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+    const fileName = `${fileNamePrefix}-${timestamp}.csv`;
+
+    log.info('✅ [STEP 6/8] CSV file generated', {
+      fileName,
+      sizeBytes: csvContent.length,
+      recordCount: transformedRecords.length,
+      duration: Date.now() - step6Start,
+    });
+
+    // ═══════════════════════════════════════════════════════════
+    // STEP 7/8: Upload to S3 (S3DataSource)
+    // ═══════════════════════════════════════════════════════════
+    const step7Start = Date.now();
+    log.info('☁️ [STEP 7/8] Uploading to S3', { jobId, fileName });
+
+    await tracker.updateJob(jobId, {
+      status: 'processing',
+      stage: 's3_upload',
+      message: `Uploading ${fileName} to S3`,
+    });
+
+    // Get S3 configuration from activation variables
+    const s3Config = {
+      bucket: activation.getVariable('s3BucketName'),
+      region: activation.getVariable('awsRegion') || 'us-east-1',
+      accessKeyId: activation.getVariable('awsAccessKeyId'),
+      secretAccessKey: activation.getVariable('awsSecretAccessKey'),
+    };
+    const s3Prefix = activation.getVariable('s3Prefix') || 'inventory-positions/daily/';
+
+    // Validate S3 config
+    if (!s3Config.bucket || !s3Config.accessKeyId || !s3Config.secretAccessKey) {
+      const missingFields = [];
+      if (!s3Config.bucket) missingFields.push('s3BucketName');
+      if (!s3Config.accessKeyId) missingFields.push('awsAccessKeyId');
+      if (!s3Config.secretAccessKey) missingFields.push('awsSecretAccessKey');
+
+      throw new Error(
+        `S3 configuration incomplete - missing activation variables: ${missingFields.join(', ')}`
+      );
+    }
+
+    // Initialize S3 data source
+    // ✅ VERSORI PLATFORM: Pass native log from context
+    const s3 = new S3DataSource(
+      {
+        type: 'S3_CSV',
+        connectionId: 'inventory-positions-s3',
+        name: 'Inventory Positions S3 Upload',
+        s3Config,
+      },
+      log
+    );
+
+    // Validate S3 connection before upload
+    try {
+      await s3.validateConnection();
+      log.info('✅ S3 connection validated', { bucket: s3Config.bucket });
+    } catch (validateError: any) {
+      log.error('❌ S3 connection validation failed', {
+        bucket: s3Config.bucket,
+        region: s3Config.region,
+        error: validateError.message,
+      });
+      throw new Error(
+        `S3 connection validation failed: ${validateError.message}. ` +
+        `Check bucket name, region, and AWS credentials.`
+      );
+    }
+
+    // Construct S3 key
+    const s3Key = `${s3Prefix}${fileName}`;
+
+    // Upload with retry logic (built into S3DataSource)
+    await s3.upload(s3Key, Buffer.from(csvContent, 'utf-8'), {
+      contentType: 'text/csv',
+      metadata: {
+        recordCount: String(transformedRecords.length),
+        extractedAt: new Date().toISOString(),
+        jobId,
+      },
+    });
+
+    log.info('✅ [STEP 7/8] S3 upload successful', {
+      fileName,
+      s3Key,
+      bucket: s3Config.bucket,
+      sizeBytes: csvContent.length,
+      duration: Date.now() - step7Start,
+    });
+
+    // ═══════════════════════════════════════════════════════════
+    // STEP 8/8: Update State & Complete Job
+    // ═══════════════════════════════════════════════════════════
+    const step8Start = Date.now();
+    log.info('💾 [STEP 8/8] Updating state and completing job', { jobId });
+
+    // Calculate new timestamp for next incremental run
+    let newTimestamp: string | undefined;
+
+    if (updateState) {
+      // Find max updatedOn from extracted records
+      const maxUpdatedOn = rawRecords.reduce(
+        (max, record) => {
+          const recordTime = new Date(record.updatedOn).getTime();
+          return recordTime > max ? recordTime : max;
+        },
+        new Date(dateRangeFilter?.from || DEFAULT_FALLBACK).getTime()
+      );
+
+      newTimestamp = new Date(maxUpdatedOn).toISOString();
+
+      // Store new timestamp (WITHOUT buffer - buffer only applied on read)
+      await kv.set(STATE_KEY, newTimestamp);
+
+      log.info('State updated', {
+        oldTimestamp: dateRangeFilter?.from,
+        newTimestamp,
+      });
+    }
+
+    // Mark job as completed
+    await tracker.markCompleted(jobId, {
+      recordCount: transformedRecords.length,
+      fileName,
+      s3Key,
+      errorCount: mappingErrors.length,
+      isManualOverride,
+      stateUpdated: updateState,
+      newTimestamp,
+    });
+
+    log.info('✅ [STEP 8/8] State updated and job completed', {
+      jobId,
+      stateUpdated: updateState,
+      newTimestamp,
+      duration: Date.now() - step8Start,
+    });
+
+    return {
+      success: true,
+      jobId,
+      recordsExtracted: transformedRecords.length,
+      fileName,
+      s3Path: s3Key,
+      isManualOverride,
+      stateUpdated: updateState,
+      newTimestamp,
+      errors: mappingErrors.length > 0 ? mappingErrors : undefined,
+    };
+  } catch (error: any) {
+    log.error('❌ [ORCHESTRATION] Extraction workflow failed', {
+      jobId,
+      message: error instanceof Error ? error.message : String(error),
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'Error',
+    });
+
+    // Mark job as failed
+    await tracker.markFailed(jobId, error);
+
+    return {
+      success: false,
+      jobId,
+      recordsExtracted: 0,
+      error: error.message,
+      recommendation: getErrorRecommendation(error),
+    };
+  }
+}
+
+/**
+ * Helper: Get error-specific recommendations
+ * (Same helper function used in workflows - can be shared via utils)
+ */
+function getErrorRecommendation(error: any): string {
+  const message = error instanceof Error ? error.message : String(error);
+
+  if (message.includes('ECONNREFUSED') || message.includes('ETIMEDOUT')) {
+    return 'Network connectivity issue. Check firewall rules and network configuration.';
+  }
+  if (message.includes('authentication') || message.includes('401')) {
+    return 'Authentication failed. Verify OAuth2 credentials in connection configuration.';
+  }
+  if (message.includes('S3') || message.includes('bucket')) {
+    return 'S3 operation failed. Check bucket name, region, and AWS credentials.';
+  }
+  if (message.includes('GraphQL') || message.includes('query')) {
+    return 'GraphQL query failed. Verify query syntax and field availability in schema.';
+  }
+  if (message.includes('mapping') || message.includes('transform')) {
+    return 'Data transformation failed. Check mapping configuration and source data structure.';
+  }
+  if (message.includes('KV') || message.includes('state')) {
+    return 'State management failed. Check Versori KV availability and permissions.';
+  }
+
+  return 'Check logs for detailed error information and stack trace.';
+}
+```
+
+---
+
+## 4. Utility Functions (src/utils/job-id-generator.ts)
+
+```typescript
+/**
+ * Job ID Generator
+ *
+ * Generates unique job IDs for tracking extraction workflows
+ *
+ * FORMAT: {TYPE}_{ENTITY}_{YYYYMMDD}_{HHMMSS}_{RANDOM}
+ * Example: SCHEDULED_IP_20251027_183045_a1b2c3
+ */
+
+/**
+ * Generate unique job ID
+ *
+ * @param type - Job trigger type (SCHEDULED, ADHOC, MANUAL)
+ * @param entity - Entity abbreviation (IP=Inventory Positions, VP, ORD, PRD)
+ * @returns Unique job ID string
+ */
+export function generateJobId(type: string, entity: string): string {
+  const now = new Date();
+
+  // Format: YYYYMMDD
+  const dateStr = now.toISOString().slice(0, 10).replace(/-/g, '');
+
+  // Format: HHMMSS
+  const timeStr = now.toISOString().slice(11, 19).replace(/:/g, '');
+
+  // Random suffix (6 chars)
+  const randomStr = Math.random().toString(36).substring(2, 8);
+
+  return `${type}_${entity}_${dateStr}_${timeStr}_${randomStr}`;
+}
+
+/**
+ * Parse job ID components
+ */
+export function parseJobId(jobId: string): {
+  type: string;
+  entity: string;
+  date: string;
+  time: string;
+  random: string;
+} | null {
+  const parts = jobId.split('_');
+
+  if (parts.length !== 5) {
+    return null;
+  }
+
+  return {
+    type: parts[0],
+    entity: parts[1],
+    date: parts[2],
+    time: parts[3],
+    random: parts[4],
+  };
+}
+```
+
+---
+
+## 5. Package Configuration
+
+### package.json
+
+```json
+{
+  "name": "inventory-positions-to-s3-csv",
+  "version": "1.0.0",
+  "description": "Extract inventory positions from Fluent Commerce and export to S3 as CSV",
+  "type": "module",
+  "main": "src/index.ts",
+  "scripts": {
+    "dev": "versori dev",
+    "build": "versori build",
+    "deploy": "versori deploy"
+  },
+  "dependencies": {
+    "@fluentcommerce/fc-connect-sdk": "^0.1.39",
+    "@versori/run": "latest"
+  },
+  "devDependencies": {
+    "@types/node": "^20.0.0",
+    "typescript": "^5.0.0"
+  }
+}
+```
+
+### tsconfig.json
+
+```json
+{
+  "compilerOptions": {
+    "module": "ES2022",
+    "target": "ES2024",
+    "moduleResolution": "node"
+  }
+}
+```
+
+---
+
+## 6. Deployment Instructions
+
+### Deploy to Versori
+
+```bash
+# 1. Install dependencies
+npm install
+
+# 2. Test locally (if using Versori CLI)
+npm run dev
+
+# 3. Deploy to Versori platform
+npm run deploy
+```
+
+### Configure Activation Variables
+
+In Versori platform settings, configure:
+
+```json
+{
+  "catalogueRef": "DEFAULT_CATALOGUE",
+  "s3BucketName": "inventory-analytics-exports",
+  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
+  "awsSecretAccessKey": "********",
+  "awsRegion": "us-east-1",
+  "s3Prefix": "inventory-positions/daily/",
+  "fileNamePrefix": "inventorypositions",
+  "pageSize": 200,
+  "maxRecords": 100000,
+  "overlapBufferSeconds": 60
+}
+```
+
+---
+
+## 7. Testing
+
+### Test Scheduled Extraction
+
+The scheduled workflow runs automatically based on cron schedule.
+
+**Check logs:**
+
+```
+[STEP 1/8] Initializing job tracking
+[STEP 2/8] Initializing Fluent Commerce client
+[STEP 3/8] Determining date range for extraction
+[STEP 4/8] Extracting data from Fluent Commerce
+[STEP 5/8] Transforming data with UniversalMapper
+[STEP 6/8] Generating CSV file
+[STEP 7/8] Uploading to S3
+[STEP 8/8] Updating state and completing job
+```
+
+### Test Ad hoc Extraction
+
+```bash
+# Incremental (uses last sync timestamp)
+curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
+  -H "Content-Type: application/json" \
+  -d '{}'
+
+# Date range override
+curl -X POST https://api.versori.com/webhooks/inventory-positions-adhoc \
+  -H "Content-Type: application/json" \
+  -d '{
+    "fromDate": "2025-01-01T00:00:00Z",
+    "toDate": "2025-01-31T23:59:59Z",
+    "updateState": false
+  }'
+```
+
+### Test Job Status Query
+
+```bash
+curl -X POST https://api.versori.com/webhooks/inventory-positions-job-status \
+  -H "Content-Type: application/json" \
+  -d '{
+    "jobId": "ADHOC_IP_20251027_183045_abc123"
+  }'
+```
+
+**Response:**
+
+```json
+{
+  "success": true,
+  "jobId": "ADHOC_IP_20251027_183045_abc123",
+  "status": "processing",
+  "stage": "transformation",
+  "message": "Transforming 15000 records",
+  "createdAt": "2025-10-27T18:30:45.000Z",
+  "startedAt": "2025-10-27T18:30:46.000Z"
+}
+```
+
+---
+
+---
+
+### Pattern 7: State Management & Date Overrides
+
+**Use Case**: Understand how state management works with scheduled and ad-hoc extractions.
+
+**How it works**:
+
+VersoriKV stores the last successful extraction timestamp to enable incremental sync:
+
+```typescript
+interface ExtractionState {
+  timestamp: string; // Last run timestamp (WITHOUT overlap buffer)
+  recordCount: number; // Number of records extracted
+  extractedAt: string; // When extraction completed
+  fileName?: string; // Generated filename
+  s3Key?: string; // S3 upload path
+  overlapBufferSeconds?: number; // Buffer configuration
+}
+```
+
+**State Priority Chain** (highest to lowest):
+
+1. **`fromDate` override** (manual date in webhook payload) - Highest priority
+2. **Stored state** (`await kv.get(stateKey)`) - Normal incremental mode
+3. **`fallbackStartDate`** (activation variable) - First run fallback
+
+**Three Scenarios**:
+
+#### Scenario 1: Normal Scheduled Runs (Incremental)
+
+```typescript
+// Payload: {} (empty - no overrides)
+
+// Behavior:
+// 1. Load last timestamp from KV: "2025-01-22T10:00:00Z"
+// 2. Apply overlap buffer: "2025-01-22T09:59:00Z" (query WITH buffer)
+// 3. Extract records updated since buffered time
+// 4. Calculate MAX(updatedOn) from results: "2025-01-22T14:30:00Z"
+// 5. Save new timestamp WITHOUT buffer: "2025-01-22T14:30:00Z"
+// 6. Next run starts from "2025-01-22T14:29:00Z" (with buffer)
+```
+
+**Test**:
+
+```bash
+# Trigger scheduled run (no payload needed)
+# State advances automatically
+curl -X POST https://workspace.versori.run/inventory-positions-scheduled
+```
+
+#### Scenario 2: Ad-hoc Extraction WITH State Update
+
+```typescript
+// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": true }
+
+// Behavior:
+// 1. Ignore stored state
+// 2. Use fromDate: "2025-01-01T00:00:00Z" (no buffer applied to manual dates)
+// 3. Extract all records since 2025-01-01
+// 4. Calculate MAX(updatedOn): "2025-01-22T14:30:00Z"
+// 5. Save new timestamp: "2025-01-22T14:30:00Z" (updates state!)
+// 6. Next scheduled run starts from this new timestamp
+```
+
+**Use Case**: One-time catch-up extraction that advances the state pointer.
+
+**Test**:
+
+```bash
+curl -X POST https://workspace.versori.run/inventory-positions-adhoc \
+  -H "Content-Type: application/json" \
+  -d '{
+    "fromDate": "2025-01-01T00:00:00Z",
+    "updateState": true
+  }'
+```
+
+#### Scenario 3: Ad-hoc Extraction WITHOUT State Update
+
+```typescript
+// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": false }
+
+// Behavior:
+// 1. Ignore stored state
+// 2. Use fromDate: "2025-01-01T00:00:00Z"
+// 3. Extract all records since 2025-01-01
+// 4. DO NOT update state
+// 5. Next scheduled run uses previous timestamp (unaffected)
+```
+
+**Use Case**: Historical backfill or testing without affecting incremental sync.
+
+**Test**:
+
+```bash
+curl -X POST https://workspace.versori.run/inventory-positions-adhoc \
+  -H "Content-Type: application/json" \
+  -d '{
+    "fromDate": "2025-01-01T00:00:00Z",
+    "toDate": "2025-01-31T23:59:59Z",
+    "updateState": false
+  }'
+```
+
+**Why this matters**:
+
+- **Incremental sync** relies on state continuity
+- **Manual overrides** allow catch-up without breaking incremental flow
+- **Overlap buffer** prevents missed records at time boundaries
+- **State isolation** lets you test/backfill without affecting production sync
+
+---
+
+### Pattern 8: Optional GraphQL Query Logging
+
+**Use Case**: Debug extraction issues by logging the exact GraphQL query sent to Fluent Commerce API.
+
+**When to use**:
+
+- ✅ Debugging pagination issues
+- ✅ Verifying query variables (dates, filters, limits)
+- ✅ Development and testing
+- ❌ Production (verbose logs, potential secrets in variables)
+
+**How to enable**:
+
+Set `DEBUG_GRAPHQL=true` environment variable in Versori activation settings.
+
+**Implementation**:
+
+```typescript
+// In your extraction workflow
+const DEBUG_GRAPHQL = activation?.getVariable('DEBUG_GRAPHQL') === 'true';
+
+if (DEBUG_GRAPHQL) {
+  log.info('GraphQL Query Debug', {
+    query: INVENTORY_POSITIONS_QUERY,
+    variables: {
+      catalogueRef,
+      dateRangeFilter,
+      first: pageSize,
+      after: null, // First page
+    },
+    pagination: {
+      pageSize,
+      maxRecords,
+      currentPage: 1,
+    },
+  });
+}
+
+const extractionResult = await orchestrator.extract({
+  query: INVENTORY_POSITIONS_QUERY,
+  resultPath: 'inventoryPositions.edges.node',
+  variables: {
+    catalogueRef,
+    dateRangeFilter,
+  },
+  pageSize,
+  maxRecords,
+});
+
+if (DEBUG_GRAPHQL) {
+  log.info('GraphQL Response Debug', {
+    totalRecords: extractionResult.stats.totalRecords,
+    totalPages: extractionResult.stats.totalPages,
+    truncated: extractionResult.stats.truncated,
+    truncationReason: extractionResult.stats.truncationReason,
+    firstRecordId: extractionResult.data[0]?.id,
+    lastRecordId: extractionResult.data[extractionResult.data.length - 1]?.id,
+  });
+}
+```
+
+**What gets logged**:
+
+```json
+{
+  "level": "info",
+  "message": "GraphQL Query Debug",
+  "query": "query GetInventoryPositions($catalogueRef: String, $dateRangeFilter: DateRange, ...)",
+  "variables": {
+    "catalogueRef": "DEFAULT_CATALOGUE",
+    "dateRangeFilter": "2025-01-22T09:59:00Z",
+    "first": 200,
+    "after": null
+  },
+  "pagination": {
+    "pageSize": 200,
+    "maxRecords": 100000,
+    "currentPage": 1
+  }
+}
+```
+
+**Versori Environment Variables**:
+
+Add to activation settings:
+
+```json
+{
+  "DEBUG_GRAPHQL": "true"
+}
+```
+
+**Testing**:
+
+```bash
+# Enable debug logging
+curl -X POST https://workspace.versori.run/inventory-positions-scheduled
+
+# Check Versori logs for "GraphQL Query Debug" entries
+# Verify query structure and variables are correct
+```
+
+**Sample Debug Output**:
+
+```
+[INFO] GraphQL Query Debug
+  query: "query GetInventoryPositions($catalogueRef: String, $dateRangeFilter: DateRange, ...)"
+  variables: { catalogueRef: "DEFAULT_CATALOGUE", dateRangeFilter: "2025-01-22T09:59:00Z", first: 200, after: null }
+  pagination: { pageSize: 200, maxRecords: 100000, currentPage: 1 }
+
+[INFO] Extraction complete
+  totalRecords: 1500
+  totalPages: 8
+  truncated: false
+
+[INFO] GraphQL Response Debug
+  totalRecords: 1500
+  totalPages: 8
+  truncated: false
+  truncationReason: undefined
+  firstRecordId: "ip_001"
+  lastRecordId: "ip_1500"
+```
+
+**Key Benefits**:
+
+- Quickly identify pagination configuration issues
+- Verify date filters are applied correctly
+- Debug "no records found" scenarios
+- Validate ExtractionOrchestrator variable injection
+
+**Production Best Practice**: Disable `DEBUG_GRAPHQL` in production to reduce log volume and avoid logging sensitive data.
+
+---
+
+## 8. Troubleshooting
+
+**Issue**: "No records extracted"
+
+- Check dateRange (manual override vs incremental)
+- Check catalogueRef filter
+
+**Issue**: "S3 upload failed"
+
+- Job fails; state not advanced
+- Next run retries same window
+- Check S3 credentials and bucket permissions
+
+**Issue**: "GraphQL pagination error"
+
+- Ensure edges.cursor and pageInfo.hasNextPage are in query
+
+**Issue**: "Memory pressure"
+
+- Lower pageSize or maxRecords
+- Consider file splitting for large extractions
+
+---
+
+## 9. Replication Checklist
+
+**To replicate this template for other entities/formats:**
+
+1. **File Naming:** Replace `inventory-positions`, `IP`, `InventoryPosition` with your entity name
+2. **GraphQL Query:** Update query constant and field selection to match your entity schema
+3. **Mapping Config:** Create new mapping file in `config/` with correct field paths
+4. **Workflows:** Rename workflow exports to match entity (e.g., `scheduledOrdersExtraction`)
+5. **Service Function:** Rename main function (e.g., `executeOrderExtraction`)
+6. **State Key:** Update KV key (e.g., `lastOrderSync`)
+7. **Output Format:** For XML use `XMLBuilder`, for JSON use `JSON.stringify()`, for CSV use `CSVParserService`
+8. **Upload Destination:** For SFTP replace `S3DataSource` with `SftpDataSource` (and add `dispose()` in finally block)
+9. **Job ID Entity Code:** Update entity abbreviation in generateJobId() (e.g., 'ORD' for orders)
+
+---
+
+**Pattern**: Enterprise incremental extraction with overlap buffer for inventory positions
+**Key Learning**: Use `catalogues` (plural, array) input for filtering
+**Critical**: Apply 60-second overlap buffer to prevent missed records due to clock skew
+**Buffer Pattern**: Query WITH buffer (`updatedOn >= lastRunTime - 60s`), save WITHOUT buffer (`MAX(updatedOn)`)
+**SDK Services**: ExtractionOrchestrator, UniversalMapper, CSVParserService, S3DataSource, JobTracker
+
+---
+
+### Pattern 8: Backward Pagination (Optional - Advanced)
+
+**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
+
+**When to Use**:
+
+- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
+- ✅ Time-bounded reverse traversal for auditing
+- ✅ Display newest-first in UI/reports
+- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
+
+**GraphQL Query Requirements**:
+
+Your query must support backward pagination by including `$last` and `$before`:
+
+```graphql
+query GetData(
+  $retailerId: ID!
+  $first: Int # For forward pagination
+  $after: String # For forward pagination
+  $last: Int # For backward pagination
+  $before: String # For backward pagination
+) {
+  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
+    edges {
+      cursor # ✅ REQUIRED
+      node {
+        id
+        createdAt
+        # ... other fields
+      }
+    }
+    pageInfo {
+      hasNextPage # For forward
+      hasPreviousPage # ✅ REQUIRED for backward
+    }
+  }
+}
+```
+
+**Implementation**:
+
+```typescript
+// Backward pagination - newest records first
+const result = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    retailerId,
+    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
+    // ❌ Don't include last/before - orchestrator injects them
+  },
+  pageSize: 200,
+  direction: 'backward', // ✅ Enable reverse pagination
+  maxRecords: 10000,
+});
+
+// Records are returned in reverse chronological order
+console.log(result.data[0].createdAt); // Newest
+console.log(result.data[result.data.length - 1].createdAt); // Oldest (within range)
+```
+
+**Key Differences from Forward Pagination**:
+
+| Aspect                 | Forward (Default)                | Backward                |
+| ---------------------- | -------------------------------- | ----------------------- |
+| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
+| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
+| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
+| **Cursor Source**      | Last edge of page                | First edge of page      |
+| **Record Order**       | Oldest → Newest                  | Newest → Oldest         |
+
+**Important Notes**:
+
+1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
+
+2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
+
+3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
+
+4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
+
+**Example: Extract Latest 1000 Orders**
+
+```typescript
+const latestOrders = await orchestrator.extract({
+  query: ORDERS_QUERY,
+  resultPath: 'orders.edges.node',
+  variables: {
+    retailerId,
+    statuses: ['BOOKED', 'ALLOCATED'],
+  },
+  direction: 'backward', // Start from newest
+  maxRecords: 1000, // Stop after 1000 records
+  pageSize: 100, // 100 per page = 10 pages
+});
+
+// latestOrders.data[0] is the newest order
+// latestOrders.data[999] is the 1000th newest order
+```
+
+**When to Use Forward vs Backward**:
+
+```typescript
+// ✅ Forward (default) - For incremental sync
+const incrementalData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    dateRangeFilter: { from: lastSyncTime, to: now },
+  },
+  // direction defaults to 'forward'
+  // Processes oldest → newest for proper sequencing
+});
+
+// ✅ Backward - For "latest N records" use cases
+const latestData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  direction: 'backward',
+  maxRecords: 100, // Just get latest 100
+  // Gets newest → oldest
+});
+```
+
+**Pagination Variables Reference**:
+
+| Variable | Forward     | Backward    | Injected By  | Notes                    |
+| -------- | ----------- | ----------- | ------------ | ------------------------ |
+| `first`  | ✅ Used     | ❌ Not used | Orchestrator | From `pageSize`          |
+| `after`  | ✅ Used     | ❌ Not used | Orchestrator | From cursor (last edge)  |
+| `last`   | ❌ Not used | ✅ Used     | Orchestrator | From `pageSize`          |
+| `before` | ❌ Not used | ✅ Used     | Orchestrator | From cursor (first edge) |
+
+**Common Mistakes to Avoid**:
+
+```typescript
+// ❌ WRONG - Don't pass pagination variables
+const result = await orchestrator.extract({
+  variables: {
+    last: 200, // ❌ Orchestrator will override this
+    before: cursor, // ❌ Orchestrator manages cursor
+  },
+  direction: 'backward',
+});
+
+// ✅ CORRECT - Let orchestrator inject pagination
+const result = await orchestrator.extract({
+  variables: {
+    retailerId, // ✅ Your business variables only
+  },
+  pageSize: 200, // ✅ Orchestrator uses this for last/before
+  direction: 'backward',
+});
+```
+
+#### Optional: Reverse Pagination
+
+- Default: forward ($first/$after) + pageInfo.hasNextPage.
+- Reverse: use $last/$before and pageInfo.hasPreviousPage in the query, and set direction='backward' in the orchestrator call.
+
+GraphQL:
+
+```graphql
+query GetInventoryPositionsBackward(
+  $catalogues: [InventoryCatalogueKey]
+  $dateRangeFilter: DateRange
+  $last: Int!
+  $before: String
+) {
+  inventoryPositions(
+    catalogues: $catalogues
+    updatedOn: $dateRangeFilter
+    last: $last
+    before: $before
+  ) {
+    edges {
+      cursor
+      node {
+        id
+        ref
+        updatedOn
+      }
+    }
+    pageInfo {
+      hasPreviousPage
+    }
+  }
+}
+```
+
+SDK:
+
+```typescript
+await orchestrator.extract({
+  query: INVENTORY_POSITIONS_BACKWARD_QUERY,
+  resultPath: 'inventoryPositions.edges.node',
+  variables: { catalogues, dateRangeFilter },
+  pageSize,
+  direction: 'backward',
+});
+```
+
+---
+
+## Testing Checklist
+
+**Before production deployment:**
+
+### 1. Schema Validation
+
+- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
+- [ ] Run `npx fc-connect validate-schema --mapping ./config/inventory-positions.export.csv.json --schema ./fluent-schema.json`
+- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/inventory-positions.export.csv.json --schema ./fluent-schema.json`
+- [ ] Verify all `source` paths in mapping exist in GraphQL schema
+- [ ] Verify query structure matches schema (fields, types, filters)
+
+### 2. Extraction Testing
+
+- [ ] Test with small dataset first (maxRecords=10)
+- [ ] Verify ExtractionOrchestrator pagination works correctly
+- [ ] Test with multiple pages of data (verify cursor handling)
+- [ ] Verify date range filtering (updatedOn filter)
+- [ ] Test empty result handling (no records in date range)
+- [ ] Verify extraction stops at maxRecords limit
+
+### 3. Mapping Testing
+
+- [ ] Verify required fields are populated
+- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
+- [ ] Test custom resolvers with edge cases (if any)
+- [ ] Verify nested field extraction
+- [ ] Test with null/missing fields
+- [ ] Verify mapping error collection works
+
+### 4. CSV Generation Testing
+
+- [ ] Verify CSV structure matches expected format
+- [ ] Test CSV validation against schema (if applicable)
+- [ ] Verify header row is present and correct
+- [ ] Test with large datasets (>1000 records)
+- [ ] Verify UTF-8 encoding
+- [ ] Test special character handling (commas, quotes, newlines)
+
+### 5. S3 Upload Testing
+
+- [ ] Test S3 connection and authentication
+- [ ] Verify file upload to correct bucket and path
+- [ ] Test file naming convention (timestamp format)
+- [ ] Verify S3 object metadata
+- [ ] Test upload retry logic (simulate network failure)
+- [ ] Verify file permissions and ACLs
+
+### 6. State Management Testing
+
+- [ ] Verify overlap buffer prevents missed records (60-second default)
+- [ ] Test state recovery after extraction failure
+- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
+- [ ] Test first run with no previous state (uses fallbackStartDate)
+- [ ] Verify state update only happens on successful upload
+- [ ] Test manual date override (doesn't update state)
+
+### 7. Job Tracking Testing
+
+- [ ] Test job creation with JobTracker
+- [ ] Verify job status updates at each stage
+- [ ] Test job completion with metadata
+- [ ] Test job failure handling
+- [ ] Query job status via webhook endpoint
+- [ ] Verify job status persists in KV store
+
+### 8. Error Handling Testing
+
+- [ ] Test with invalid GraphQL query
+- [ ] Test with mapping errors (invalid field paths)
+- [ ] Test with S3 connection failures
+- [ ] Test with authentication failures
+- [ ] Test with network timeouts
+- [ ] Verify error logging includes context (jobId, stage, error details)
+- [ ] Test error threshold logic (if applicable)
+
+### 9. Staging Environment Testing
+
+- [ ] Run full extraction in staging environment
+- [ ] Verify CSV file format with downstream system
+- [ ] Monitor extraction duration and resource usage
+- [ ] Test with production-like data volumes
+- [ ] Verify no performance degradation over time
+
+### 10. Integration Testing
+
+- [ ] Test scheduled workflow (cron trigger)
+- [ ] Test ad hoc webhook trigger
+- [ ] Test job status query webhook
+- [ ] Verify activation variables are read correctly
+- [ ] Test with different extraction modes (incremental, date range)
+- [ ] End-to-end test: trigger → extract → transform → upload → verify file
+
+---
+
+## Recommended Project Structure
+
+**Organize workflows by trigger type in separate files:**
+
+```
+inventory-positions-extraction/
+├── index.ts                              # Entry point - exports all workflows
+└── src/
+    ├── workflows/
+    │   ├── scheduled/
+    │   │   └── daily-inventory-positions-extraction.ts  # Scheduled: Daily extraction
+    │   │
+    │   └── webhook/
+    │       ├── adhoc-inventory-positions-extraction.ts  # Webhook: Manual trigger
+    │       └── job-status-check.ts                     # Webhook: Status query
+    │
+    ├── services/
+    │   └── inventory-positions-extraction.service.ts    # Shared orchestration logic (reusable)
+    │
+    └── config/
+        └── inventory-positions.export.csv.json         # Mapping configuration
+```
+
+**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
+
+**Trigger Types:**
+- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
+- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
+- **`workflow()`** → Durable workflows (advanced, rarely used)
+
+**Execution Steps (chained to triggers):**
+- **`http()`** → External API calls (chained from schedule/webhook)
+- **`fn()`** → Internal processing (chained from schedule/webhook)
+
+---
+
+## Monitoring & Alerting
+
+### Success Response Example
+
+```json
+{
+  "success": true,
+  "jobId": "SCHEDULED_IP_20251102_140000_abc123",
+  "recordsExtracted": 1523,
+  "fileName": "inventory-positions-2025-11-02T14-00-00-000Z.csv",
+  "s3Path": "s3://bucket/inventory-positions/inventory-positions-2025-11-02T14-00-00-000Z.csv",
+  "metrics": {
+    "extractionDurationMs": 12543,
+    "totalPages": 8,
+    "pageSize": 200,
+    "mappingErrors": 0,
+    "fileSizeBytes": 524288,
+    "uploadDurationMs": 1234
+  },
+  "timestamps": {
+    "extractionStart": "2025-11-02T14:00:00.000Z",
+    "extractionEnd": "2025-11-02T14:00:12.543Z",
+    "uploadComplete": "2025-11-02T14:00:13.777Z"
+  },
+  "state": {
+    "previousTimestamp": "2025-11-02T13:00:00.000Z",
+    "newTimestamp": "2025-11-02T13:59:58.123Z",
+    "stateUpdated": true,
+    "overlapBufferSeconds": 60
+  }
+}
+```
+
+### Error Response Example
+
+```json
+{
+  "success": false,
+  "jobId": "ADHOC_IP_20251102_140500_xyz789",
+  "error": "S3 upload failed: Connection timeout",
+  "errorCategory": "NETWORK",
+  "recordsExtracted": 0,
+  "stage": "s3_upload",
+  "details": {
+    "message": "Failed to upload file after 3 retry attempts",
+    "retryAttempts": 3,
+    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
+  },
+  "state": {
+    "stateUpdated": false,
+    "willRetryNextRun": true,
+    "note": "State not advanced - next extraction will retry same time window"
+  }
+}
+```
+
+### Key Metrics to Track
+
+```typescript
+const METRICS = {
+  // Extraction Performance
+  extractionDurationMs: Date.now() - extractionStart,
+  recordCount: records.length,
+  pageCount: extractionResult.stats.totalPages,
+  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
+
+  // Transformation Performance
+  transformedCount: transformedRecords.length,
+  failedCount: mappingErrors.length,
+  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
+
+  // File Generation
+  fileSizeMB: (csvContent.length / (1024 * 1024)).toFixed(2),
+
+  // Upload Performance
+  uploadDurationMs: uploadEnd - uploadStart,
+  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
+
+  // State Management
+  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
+  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
+};
+
+log.info('Extraction metrics', metrics);
+```
+
+### Alert Thresholds
+
+```typescript
+const ALERT_THRESHOLDS = {
+  // Duration Alerts
+  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
+  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
+  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
+
+  // Error Rate Alerts
+  MAX_ERROR_RATE: 0.05, // 5% mapping errors
+  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
+
+  // Volume Alerts
+  MAX_RECORDS_PER_RUN: 100000,
+  MIN_RECORDS_WARNING: 0, // Alert if no records found
+  MAX_FILE_SIZE_MB: 150, // 150MB
+
+  // State Alerts
+  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
+  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
+};
+
+// Check thresholds
+if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
+  log.warn('Extraction duration exceeded threshold', {
+    duration: metrics.extractionDurationMs,
+    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
+    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
+  });
+}
+```
+
+### Monitoring Dashboard Queries
+
+**Versori Platform Logs Query:**
+
+```
+# Successful extractions
+log_level:info AND message:"Extraction complete" AND jobId:*
+
+# Failed extractions
+log_level:error AND message:"Extraction workflow failed" AND jobId:*
+
+# Performance issues
+extractionDurationMs:>300000 OR uploadDurationMs:>120000
+
+# High error rates
+errorRate:>5
+
+# State management issues
+stateUpdated:false AND success:true
+```
+
+### Common Issues and Solutions
+
+**Issue**: "Extraction timeout after 10 minutes"
+
+- **Cause**: Too many records in single extraction
+- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
+- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
+
+**Issue**: "Mapping errors for 50% of records"
+
+- **Cause**: Schema mismatch between GraphQL response and mapping config
+- **Fix**: Run schema validation, update mapping config paths
+- **Prevention**: Use `npx fc-connect validate-schema` before deployment
+
+**Issue**: "S3 connection timeout"
+
+- **Cause**: Network issues, firewall, or connection pool exhaustion
+- **Fix**: Check S3 credentials, verify network connectivity
+- **Prevention**: Implement connection health checks, monitor connection status
+
+**Issue**: "State not updating after successful extraction"
+
+- **Cause**: KV write failure or intentional retry logic
+- **Fix**: Check KV logs, verify state update code executed
+- **Prevention**: Add KV write verification, log state updates explicitly
+
+**Issue**: "First run exceeds record limits"
+
+- **Cause**: No previous timestamp, fetches all historical records
+- **Fix**: Set fallbackStartDate close to current date, apply additional filters
+- **Prevention**: Use appropriate fallbackStartDate for initial runs
+
+**Issue**: "Excessive duplicate records in output"
+
+- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
+- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
+- **Prevention**: Monitor duplicate rates, verify state update logic
+
+---
+
+## Troubleshooting Quick Reference
+
+| Error Message | Likely Cause | Solution |
+|--------------|--------------|----------|
+| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
+| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
+| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
+| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
+| "S3 authentication failed" | Invalid credentials | Verify S3 credentials in activation variables |
+| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
+| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
+| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
+| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
+| "CSV generation failed" | Format-specific error | Check CSV generation logic, validate output |
+
+---