@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-inventory-quantities-to-s3-json.md

@@ -1,1959 +1,1959 @@
----
-template_id: tpl-extract-inventory-quantities-graphql-to-s3-json
-canonical_filename: template-extraction-inventory-quantities-to-s3-json.md
-version: 2.0.0
-sdk_version: ^0.1.39
-runtime: versori
-direction: extraction
-source: fluent-graphql
-destination: s3-json
-entity: inventory-quantities
-format: json
-logging: versori
-status: stable
-features:
-  - memory-management
-  - enhanced-logging
-  - pagination-progress
----
-
-# Template: Extraction - Inventory Quantities GraphQL to S3 JSON
-
-**Template Version:** 2.0.0
-**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
-**Last Updated:** 2025-01-24
-**Deployment Target:** Versori Platform
-
-**🆕 Version 2.0.0 Enhancements:**
-- ✅ **Memory Management** - Clear large result sets after processing batches
-- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
-- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
-
----
-
-## 📚 STEP 1: Load These Docs (Human Checklist)
-
-1. REQUIRED (load all)
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-
-Copy-paste list (open these):
-fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-
----
-
-## 📋 STEP 2: Tell Your AI (Prompt)
-
-Copy/paste the standardized prompt from `docs/template-loading-matrix.md#prompts`.
-
----
-
-## 💻 STEP 3: Implementation (Verified Imports)
-
-```ts
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  UniversalMapper,
-  S3DataSource,
-  JSONBuilder,
-  VersoriKVAdapter,
-  ExtractionOrchestrator,
-  JobTracker,
-} from '@fluentcommerce/fc-connect-sdk';
-```
-
-These are the only SDK imports required for this template. Keep type-only imports out of code samples. Prefer the orchestrator-based flow shown below for pagination and stats.
-
-Note on ad hoc runs: Use `fromDate`/`toDate` in webhook payloads. Default `updateState` is `false` for ad hoc; set to `true` only when you want the run to advance the saved timestamp used by scheduled runs.
-
----
-
-# Versori Scheduled: Inventory Quantities Extraction to S3 JSON (Detailed Records)
-
-**FC Connect SDK Use Case Guide**
-
-> SDK: [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
-> Version: ^0.1.39
-
-Context: Scheduled Versori workflow that extracts inventory quantities (detailed quantity records) from Fluent Commerce via GraphQL query with **incremental timestamp tracking**, transforms with `UniversalMapper`, and writes JSON files to S3 for API consumption and audit trail systems.
-
-**Pattern**: EXTRACTION (Fluent → S3 JSON)
-**Complexity**: Medium | Runtime: Versori Platform (Scheduled)
-
----
-
-## ⚠️ IMPORTANT: Sample Code for SDK Demonstration Only
-
-> **🔴 PRODUCTION RECOMMENDATION**
->
-> This guide demonstrates FC Connect SDK capabilities for **extraction and mapping workflows**. This is a **Versori sample connector** showing how to build inventory quantity extraction workflows using the SDK.
->
-> **✅ FOR PRODUCTION IMPLEMENTATIONS:**
->
-> - **ONLY use INCREMENTAL mode with scheduled runs** (e.g., every 15 minutes for real-time)
-> - Incremental mode is safe, efficient, and production-ready
-> - Uses overlap buffer to prevent missed records
-> - Natural rate limiting via timestamps
-> - JSON format ideal for API consumption
->
-> **🎯 RECOMMENDED SCHEDULE:**
->
-> - **Every 15 minutes** for real-time inventory APIs
-> - **Hourly** for standard inventory feeds
-> - **Daily** for analytics/reporting systems
->
-> **⚠️ NOTE:** This sample shows incremental mode implementation. For your production Versori connector, adapt this pattern with proper error handling, monitoring, and testing for your specific use case.
->
-> **This is a reference implementation showing HOW to use the SDK - adapt for your needs.**
-
----
-
-## What You'll Build
-
-- **Three workflows**: Scheduled extraction, ad hoc extraction, job status lookup
-- **Incremental extraction** using `updatedOn > lastRunTime` filter
-- **State management** with VersoriKVAdapter to track last successful run
-- **JobTracker integration** for KV-backed job tracking
-- GraphQL query with auto-pagination
-- UniversalMapper transformation for export schema
-- JSON file generation with proper structure
-- S3 upload to target bucket
-- **Failure recovery** - maintains last successful timestamp on errors
-
-## Business Use Case
-
-**Real-time inventory API for external systems:**
-
-- Extract inventory quantity changes every 15 minutes
-- Export as JSON for REST API consumption
-- Support for webhooks/polling by downstream systems
-- Lightweight incremental updates
-- Detailed audit trail with quantity types (AVAILABLE, RESERVED, EXPECTED, etc.)
-- Feed to order management and ecommerce platforms
-
-## Inventory Quantities Explained
-
-**InventoryQuantity** = Detailed quantity record with type breakdown
-
-- Individual quantity records by type (AVAILABLE, RESERVED, EXPECTED, etc.)
-- SKU + Location + Type combination
-- Retailer-defined types for specific tracking needs
-- Used for: Audit trails, detailed inventory tracking, compliance
-
-**vs InventoryPosition** = Physical on-hand calculation
-
-- Aggregated stock in warehouse
-- Used for: Stock reporting
-
-**vs VirtualPosition** = ATP (Available To Promise) calculation
-
-- Calculated quantity available for orders
-- Used for: Order promising
-
----
-
-## SDK Methods Used
-
-```typescript
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  UniversalMapper,
-  S3DataSource,
-  JSONBuilder,
-  VersoriKVAdapter,
-  ExtractionOrchestrator,
-  JobTracker,
-} from '@fluentcommerce/fc-connect-sdk';
-
-// STEP 1: Create client
-const client = await createClient(ctx);
-
-// STEP 2: Setup job tracking
-const kv = new VersoriKVAdapter(ctx.openKv(':project:'));
-const tracker = new JobTracker(kv, log);
-
-// STEP 3: Create extraction job
-const job = await tracker.createJob({
-  type: 'extraction',
-  entity: 'inventoryQuantities',
-  config: { extractionMode: 'incremental' },
-});
-
-// STEP 4: Initialize orchestrator
-const orchestrator = new ExtractionOrchestrator(client, log);
-
-// STEP 5: Execute extraction with auto-pagination
-const result = await orchestrator.extract({
-  query: INVENTORY_QUANTITIES_QUERY,
-  resultPath: 'inventoryQuantities.edges.node',
-  variables: { retailerId, updatedAfter },
-  maxRecords: 20000,
-});
-
-// STEP 6: Transform with UniversalMapper
-const mapper = new UniversalMapper(exportMapping);
-const transformed = result.data.map(r => mapper.map(r));
-
-// STEP 7: Build JSON and upload to S3
-const jsonBuilder = new JSONBuilder({
-  prettyPrint: true,
-  indent: 2,
-});
-const jsonOutput = {
-  metadata: { extractedAt: new Date().toISOString(), recordCount: transformed.length },
-  data: transformed,
-};
-const jsonContent = jsonBuilder.build(jsonOutput);
-const s3 = new S3DataSource(config, log);
-await s3.upload({
-  key: s3Key,
-  body: Buffer.from(jsonContent, 'utf8'),
-  contentType: 'application/json'
-});
-
-// STEP 8: Update job status
-await tracker.markCompleted(job.id, { recordsExtracted: transformed.length, s3Key });
-```
-
----
-
-## Activation Variables
-
-Configure these variables in your Versori connector's Activation Variables section:
-
-```json
-{
-  "retailerId": "your-retailer-id",
-  "s3BucketName": "api-inventory-exports",
-  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
-  "awsSecretAccessKey": "********",
-  "awsRegion": "us-east-1",
-  "s3Prefix": "inventory/quantities/",
-  "pageSize": 500,
-  "maxRecords": 20000,
-  "fallbackStartDate": "2024-01-01T00:00:00Z",
-  "overlapBufferSeconds": "60",
-  "prettyPrint": "true"
-}
-```
-
-**Variable Descriptions:**
-
-| Variable | Required | Description | Default | Example |
-|----------|----------|-------------|---------|---------|
-| `retailerId` | ✅ Yes | Fluent Commerce retailer ID for GraphQL queries | - | `"ACME_CORP"` |
-| `s3BucketName` | ✅ Yes | Target S3 bucket for JSON exports | - | `"api-inventory-exports"` |
-| `awsAccessKeyId` | ✅ Yes | AWS access key ID for S3 uploads | - | `"AKIAIOSFODNN7EXAMPLE"` |
-| `awsSecretAccessKey` | ✅ Yes | AWS secret access key for S3 uploads | - | `"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"` |
-| `awsRegion` | No | AWS region for S3 bucket | `"us-east-1"` | `"us-west-2"` |
-| `s3Prefix` | No | S3 key prefix for file organization | `"inventory/quantities/"` | `"exports/inventory/"` |
-| `pageSize` | No | GraphQL pagination page size | `500` | `200` |
-| `maxRecords` | No | Maximum records per extraction run | `20000` | `50000` |
-| `fallbackStartDate` | No | Initial timestamp for first run | `"2024-01-01T00:00:00Z"` | `"2025-01-01T00:00:00Z"` |
-| `overlapBufferSeconds` | No | Safety buffer to prevent missed records (seconds) | `60` | `120` |
-| `prettyPrint` | No | Pretty-print JSON output (`"true"` or `"false"`) | `"true"` | `"false"` |
-
-**🔒 Security Notes:**
-- Store credentials securely in Versori's encrypted variable storage
-- Never commit credentials to source control
-- Rotate AWS keys regularly
-- Use IAM roles with minimal required permissions
-
-**⚙️ Performance Tuning:**
-- **pageSize**: Lower values (100-200) for slower networks, higher values (500-1000) for faster connections
-- **maxRecords**: Adjust based on expected change volume - 20K is safe for 15-minute incremental runs
-- **overlapBufferSeconds**: Increase if you see missed records, decrease if seeing too many duplicates
-
----
-
-## Export Mapping Configuration
-
-Create file: `./config/inventory-quantities.export.json`
-
-```json
-{
-  "name": "inventory-quantities.export",
-  "version": "1.0.0",
-  "description": "Fluent Inventory Quantities → JSON API Export Mapping",
-  "fields": {
-    "ref": { "source": "ref", "required": true, "resolver": "sdk.trim" },
-    "location": { "source": "locationRef", "required": true, "resolver": "sdk.trim" },
-    "sku": { "source": "articleRef", "required": true, "resolver": "sdk.trim" },
-    "quantity": { "source": "qty", "required": true, "resolver": "sdk.parseInt" },
-    "type": { "source": "type", "required": true, "resolver": "sdk.uppercase" },
-    "condition": { "source": "condition", "required": false, "resolver": "sdk.uppercase" },
-    "status": { "source": "status", "required": true, "resolver": "sdk.uppercase" },
-    "storageArea": { "source": "storageAreaRef", "required": false, "resolver": "sdk.trim" },
-    "catalogueRef": { "source": "catalogue.ref", "required": false, "resolver": "sdk.trim" },
-    "catalogueName": { "source": "catalogue.name", "required": false, "resolver": "sdk.trim" },
-    "expectedDate": { "source": "expectedOn", "required": false, "resolver": "sdk.formatDate" },
-    "createdOn": { "source": "createdOn", "required": true, "resolver": "sdk.toString" },
-    "updatedOn": { "source": "updatedOn", "required": true, "resolver": "sdk.toString" }
-  }
-}
-```
-
----
-
-## Mapping & Resolvers Explained
-
-This section explains how the SDK transforms raw GraphQL data into your JSON export format using **UniversalMapper** and **SDK resolvers**.
-
-### SDK Resolvers Used
-
-| Field           | Resolver         | Why?                                          | Example Transformation                                      |
-| --------------- | ---------------- | --------------------------------------------- | ----------------------------------------------------------- |
-| `ref`           | `sdk.trim`       | Clean quantity record references              | `" QTY-001 "` → `"QTY-001"`                                 |
-| `location`      | `sdk.trim`       | Clean location references                     | `" DC01 "` → `"DC01"`                                       |
-| `sku`           | `sdk.trim`       | Clean SKU references                          | `" SKU-001 "` → `"SKU-001"`                                 |
-| `quantity`      | `sdk.parseInt`   | Parse quantity as integer for JSON APIs       | `"150"` → `150`                                             |
-| `type`          | `sdk.uppercase`  | Normalize type codes for consistency          | `"available"` → `"AVAILABLE"`                               |
-| `condition`     | `sdk.uppercase`  | Normalize condition codes                     | `"good"` → `"GOOD"`                                         |
-| `status`        | `sdk.uppercase`  | Normalize status codes                        | `"active"` → `"ACTIVE"`                                     |
-| `storageArea`   | `sdk.trim`       | Clean storage area references                 | `" ZONE-A "` → `"ZONE-A"`                                   |
-| `catalogueRef`  | `sdk.trim`       | Clean catalogue references from nested object | `" DEFAULT_CATALOGUE "` → `"DEFAULT_CATALOGUE"`             |
-| `catalogueName` | `sdk.trim`       | Clean catalogue names from nested object      | `" Default Inventory "` → `"Default Inventory"`             |
-| `expectedDate`  | `sdk.formatDate` | Format dates for JSON APIs                    | `"2025-01-30T00:00:00.000Z"` → `"2025-01-30"`               |
-| `createdOn`     | `sdk.toString`   | Preserve ISO 8601 timestamp string            | `"2025-01-22T10:00:00.000Z"` → `"2025-01-22T10:00:00.000Z"` |
-| `updatedOn`     | `sdk.toString`   | Preserve ISO 8601 timestamp for tracking      | `"2025-01-22T10:30:00.000Z"` → `"2025-01-22T10:30:00.000Z"` |
-
-### Transformation Flow
-
-```typescript
-// 1. GraphQL Response (raw data from Fluent Commerce)
-const rawQuantity = {
-  ref: ' QTY-001 ',
-  locationRef: ' DC01 ',
-  articleRef: ' SKU-001 ',
-  qty: '150',
-  type: 'available',
-  condition: 'good',
-  status: 'active',
-  storageAreaRef: ' ZONE-A ',
-  catalogue: {
-    ref: ' DEFAULT_CATALOGUE ',
-    name: ' Default Inventory ',
-  },
-  expectedOn: '2025-01-25T00:00:00.000Z',
-  createdOn: '2025-01-20T10:00:00.000Z',
-  updatedOn: '2025-01-22T10:30:00.000Z',
-};
-
-// 2. UniversalMapper applies SDK resolvers
-const mapper = new UniversalMapper(inventoryQuantitiesExportMapping);
-const result = await mapper.map(rawQuantity);
-
-// 3. Transformed Output (JSON-friendly camelCase)
-const transformedQuantity = {
-  ref: 'QTY-001',
-  location: 'DC01',
-  sku: 'SKU-001',
-  quantity: 150, // Parsed as number for JSON
-  type: 'AVAILABLE',
-  condition: 'GOOD',
-  status: 'ACTIVE',
-  storageArea: 'ZONE-A',
-  catalogueRef: 'DEFAULT_CATALOGUE',
-  catalogueName: 'Default Inventory',
-  expectedDate: '2025-01-25',
-  createdOn: '2025-01-20T10:00:00.000Z',
-  updatedOn: '2025-01-22T10:30:00.000Z', // ISO 8601 for APIs
-};
-
-// 4. Final JSON Structure with Metadata
-const jsonOutput = {
-  metadata: {
-    extractedAt: '2025-01-22T14:30:00.000Z',
-    recordCount: 1,
-    incrementalFrom: '2025-01-22T10:00:00.000Z',
-    incrementalTo: '2025-01-22T14:30:00.000Z',
-  },
-  data: [transformedQuantity],
-};
-```
-
-### Custom Resolvers for Inventory Quantity-Specific Logic
-
-While the mapping above uses built-in SDK resolvers, you can extend with custom business logic for API enrichment:
-
-```typescript
-const customResolvers = {
-  /**
-   * Validate and normalize quantity values for API consumption
-   */
-  'custom.normalizeQuantity': (qty: any) => {
-    const parsed = parseInt(qty) || 0;
-    return Math.max(0, parsed); // Ensure non-negative for APIs
-  },
-
-  /**
-   * Add human-readable type descriptions for API responses
-   */
-  'custom.enrichQuantityType': (type: string, sourceData: any) => {
-    const typeDescriptions: Record<string, string> = {
-      AVAILABLE: 'Available for Sale',
-      RESERVED: 'Reserved by Order',
-      EXPECTED: 'Expected Arrival',
-      ADJUSTMENT: 'Inventory Adjustment',
-      DAMAGED: 'Damaged/Unsellable',
-      IN_TRANSIT: 'In Transit to Location',
-    };
-    return {
-      code: type.toUpperCase(),
-      description: typeDescriptions[type.toUpperCase()] || type,
-      isAvailableForSale: type.toUpperCase() === 'AVAILABLE',
-    };
-  },
-
-  /**
-   * Calculate days until expected arrival for EXPECTED quantities
-   */
-  'custom.calculateExpectedArrival': (expectedOn: string | null) => {
-    if (!expectedOn) return null;
-
-    const expected = new Date(expectedOn);
-    const today = new Date();
-    const diffMs = expected.getTime() - today.getTime();
-    const daysUntil = Math.ceil(diffMs / (1000 * 60 * 60 * 24));
-
-    return {
-      date: expectedOn,
-      daysUntil: daysUntil,
-      isOverdue: daysUntil < 0,
-      isPending: daysUntil >= 0,
-    };
-  },
-
-  /**
-   * Generate API-friendly status summary
-   */
-  'custom.generateStatusSummary': (quantity: any) => {
-    const type = (quantity.type || '').toUpperCase();
-    const status = (quantity.status || '').toUpperCase();
-    const qty = parseInt(quantity.qty) || 0;
-
-    return {
-      quantity: qty,
-      type: type,
-      status: status,
-      isActive: status === 'ACTIVE',
-      isAvailable: type === 'AVAILABLE' && status === 'ACTIVE',
-      needsAction: type === 'EXPECTED' && !quantity.expectedOn,
-      stockLevel: qty === 0 ? 'OUT_OF_STOCK' : qty < 10 ? 'LOW_STOCK' : 'IN_STOCK',
-    };
-  },
-
-  /**
-   * Format for real-time inventory API responses
-   */
-  'custom.formatForRealtimeAPI': (quantity: any) => {
-    return {
-      location: quantity.locationRef?.trim(),
-      sku: quantity.articleRef?.trim(),
-      available: quantity.type === 'AVAILABLE' ? parseInt(quantity.qty) || 0 : 0,
-      reserved: quantity.type === 'RESERVED' ? parseInt(quantity.qty) || 0 : 0,
-      expected: quantity.type === 'EXPECTED' ? parseInt(quantity.qty) || 0 : 0,
-      expectedDate: quantity.expectedOn || null,
-      updatedOn: quantity.updatedOn,
-      isActive: quantity.status === 'ACTIVE',
-    };
-  },
-};
-
-// Use custom resolvers with UniversalMapper
-const mapper = new UniversalMapper(inventoryQuantitiesExportMapping, {
-  customResolvers,
-});
-```
-
-### JSON-Specific Mapping Considerations
-
-**1. camelCase vs snake_case:**
-
-```typescript
-// CSV export: snake_case for compatibility
-{ "location_code": "DC01", "last_updated": "..." }
-
-// JSON export: camelCase for APIs
-{ "location": "DC01", "updatedOn": "..." }
-```
-
-**2. Number Types:**
-
-```typescript
-// JSON preserves number types (no quotes)
-{ "quantity": 150 }  // ✓ Correct for JSON APIs
-
-// CSV requires strings
-"150"  // ✓ Correct for CSV
-```
-
-**3. Null Handling:**
-
-```typescript
-// JSON can use null
-{ "expectedDate": null }  // ✓ Valid JSON
-
-// CSV uses empty string
-""  // ✓ Valid CSV
-```
-
-**4. Metadata Wrapper:**
-
-```json
-{
-  "metadata": {
-    "extractedAt": "2025-01-22T14:30:00.000Z",
-    "recordCount": 2,
-    "incrementalFrom": "...",
-    "incrementalTo": "..."
-  },
-  "data": [
-    /* transformed records */
-  ]
-}
-```
-
-### Available SDK Resolvers
-
-The SDK provides these built-in resolvers (no custom code needed):
-
-**String Transformations:**
-
-- `sdk.trim` - Remove leading/trailing whitespace
-- `sdk.uppercase` - Convert to uppercase
-- `sdk.lowercase` - Convert to lowercase
-- `sdk.toString` - Convert to string
-
-**Number Parsing:**
-
-- `sdk.parseInt` - Parse as integer (ideal for JSON)
-- `sdk.parseFloat` - Parse as decimal
-- `sdk.number` - Parse as number (auto-detect int/float)
-
-**Date Formatting:**
-
-- `sdk.formatDate` - ISO 8601 date only (YYYY-MM-DD)
-- `sdk.formatDateShort` - Short date format
-- `sdk.parseDate` - Parse various date formats
-
-**Type Conversions:**
-
-- `sdk.boolean` - Convert to boolean
-- `sdk.parseJson` - Parse JSON strings
-- `sdk.toJson` - Convert to JSON string
-
-**Utilities:**
-
-- `sdk.identity` - Return value unchanged
-- `sdk.coalesce` - Return first non-null value
-
-See [Universal Mapping Guide](../../../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) for complete resolver documentation.
-
----
-
-## GraphQL Query
-
-**Note**: This query is verified against Fluent Commerce schema introspection.
-
-```graphql
-query GetInventoryQuantities(
-  $retailerId: ID!
-  $updatedAfter: DateTime!
-  $first: Int!
-  $after: String
-) {
-  inventoryQuantities(
-    retailerId: $retailerId
-    updatedOn: { after: $updatedAfter }
-    first: $first
-    after: $after
-  ) {
-    edges {
-      node {
-        id
-        ref
-        locationRef
-        articleRef
-        qty
-        type
-        condition
-        status
-        storageAreaRef
-        catalogue {
-          ref
-          name
-        }
-        expectedOn
-        createdOn
-        updatedOn
-      }
-      cursor
-    }
-    pageInfo {
-      hasNextPage
-      # Note: Fluent doesn't return endCursor - cursors are in edges[].cursor
-    }
-  }
-}
-```
-
----
-
-## Versori Workflows Structure
-
-**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
-
-**Trigger Types:**
-- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
-- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
-- **`workflow()`** → Durable workflows (advanced, rarely used)
-
-**Execution Steps (chained to triggers):**
-- **`http()`** → External API calls (chained from schedule/webhook)
-- **`fn()`** → Internal processing (chained from schedule/webhook)
-
-### Recommended Project Structure
-
-This template demonstrates a modular project structure with separate files for better organization and maintainability:
-
-```
-inventory-quantities-extraction/
-├── index.ts                              # Entry point - exports all workflows
-└── src/
-    ├── workflows/
-    │   ├── scheduled/
-    │   │   └── daily-inventory-quantities-extraction.ts  # Scheduled: Daily extraction
-    │   │
-    │   └── webhook/
-    │       ├── adhoc-inventory-quantities-extraction.ts  # Webhook: Manual trigger
-    │       └── job-status-check.ts                      # Webhook: Status query
-    │
-    ├── services/
-    │   └── inventory-quantities-extraction.service.ts    # Shared orchestration logic (reusable)
-    │
-    └── config/
-        └── inventory-quantities.export.json             # Mapping configuration
-```
-
-### Entry Point: index.ts
-
-**File:** `index.ts`
-
-The entry point uses the **MemoryInterpreter pattern** to export all workflows for Versori:
-
-```typescript
-/**
- * Entry point - Export all workflows for Versori platform
- *
- * VERSORI MEMORY INTERPRETER PATTERN:
- * This file exports all workflows to be registered with Versori.
- * Each workflow is defined in its own file for better organization.
- *
- * Import and re-export pattern allows Versori to discover and register
- * all workflows without manual configuration.
- */
-
-// Scheduled workflows
-export { inventoryQuantitiesExtractionJson } from './src/workflows/scheduled/daily-inventory-quantities-extraction';
-
-// Webhook workflows
-export { inventoryQuantitiesAdHocExtraction } from './src/workflows/webhook/adhoc-inventory-quantities-extraction';
-export { inventoryQuantitiesJobStatus } from './src/workflows/webhook/job-status-check';
-```
-
-**Key Points:**
-- ✅ Simple re-export pattern - Versori auto-discovers workflows
-- ✅ Each workflow in separate file for maintainability
-- ✅ Clear naming: `export { workflowName } from './path/to/workflow'`
-- ❌ No configuration needed - Versori reads exports directly
-
----
-
-## Example Output
-
-The extraction generates JSON files with this structure:
-
-```json
-{
-  "metadata": {
-    "extractedAt": "2025-01-22T14:30:00.000Z",
-    "recordCount": 2,
-    "incrementalFrom": "2025-01-22T10:00:00.000Z",
-    "incrementalTo": "2025-01-22T14:30:00.000Z"
-  },
-  "data": [
-    {
-      "ref": "QTY-001",
-      "location": "DC01",
-      "sku": "SKU-001",
-      "quantity": 150,
-      "type": "AVAILABLE",
-      "condition": "GOOD",
-      "status": "ACTIVE",
-      "storageArea": "ZONE-A",
-      "catalogueRef": "DEFAULT_CATALOGUE",
-      "catalogueName": "Default Inventory",
-      "expectedDate": null,
-      "createdOn": "2025-01-20T10:00:00Z",
-      "updatedOn": "2025-01-22T10:30:00Z"
-    },
-    {
-      "ref": "QTY-002",
-      "location": "DC02",
-      "sku": "SKU-002",
-      "quantity": 200,
-      "type": "RESERVED",
-      "condition": "GOOD",
-      "status": "ACTIVE",
-      "storageArea": "ZONE-B",
-      "catalogueRef": "DEFAULT_CATALOGUE",
-      "catalogueName": "Default Inventory",
-      "expectedDate": null,
-      "createdOn": "2025-01-21T11:00:00Z",
-      "updatedOn": "2025-01-22T11:15:00Z"
-    }
-  ]
-}
-```
-
----
-
-## Production Safety & Guardrails
-
-### Overview
-
-Even with **incremental-only** extraction, inventory quantities in JSON format need safeguards:
-
-- **JSON parsing memory**: API consumers parse entire JSON before processing
-- **Real-time APIs**: Inventory feeds power order promising, need fast processing
-- **High-frequency updates**: Every 15 minutes accumulates large datasets
-- **Nested structure**: JSON metadata wrapper adds overhead vs CSV
-
-### Hard Limits
-
-```typescript
-const SAFETY_LIMITS = {
-  MAX_RECORDS_PER_RUN: 300000, // 300k quantity records per run
-  MAX_RECORDS_PER_FILE: 50000, // 50k per JSON file (lower than CSV)
-  MAX_FILE_SIZE_MB: 100, // 100MB per file
-  MAX_JSON_SIZE_MB: 200, // Total extraction size
-  CHUNK_SIZE: 10000, // Process in chunks
-  ESTIMATED_BYTES_PER_RECORD: 400, // JSON with metadata
-};
-```
-
-**Why JSON has different limits than CSV?**
-
-- JSON includes field names in every record (overhead)
-- JSON metadata wrapper adds size
-- JSON.parse() is memory-intensive
-- API consumers need predictable payload sizes
-
----
-
-## Complete Workflow Implementation
-
-The code examples below demonstrate the implementation of each workflow component. In your project, these would be organized into separate files following the modular structure shown above.
-
-### Workflow 1: Scheduled Extraction (Incremental)
-
-**File:** `src/workflows/scheduled/daily-inventory-quantities-extraction.ts`
-
-```typescript
-import { schedule, http } from '@versori/run';
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  UniversalMapper,
-  S3DataSource,
-  JSONBuilder,
-  VersoriKVAdapter,
-  ExtractionOrchestrator,
-  JobTracker,
-} from '@fluentcommerce/fc-connect-sdk';
-import inventoryQuantitiesExportMapping from './config/inventory-quantities.export.json' with { type: 'json' };
-
-// GraphQL query
-const INVENTORY_QUANTITIES_QUERY = `
-  query GetInventoryQuantities(
-    $retailerId: ID!
-    $updatedAfter: DateTime!
-    $first: Int!
-    $after: String
-  ) {
-    inventoryQuantities(
-      retailerId: $retailerId
-      updatedOn: { after: $updatedAfter }
-      first: $first
-      after: $after
-    ) {
-      edges {
-        node {
-          id
-          ref
-          locationRef
-          articleRef
-          qty
-          type
-          condition
-          status
-          storageAreaRef
-          catalogue {
-            ref
-            name
-          }
-          expectedOn
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-        # Note: Fluent doesn't return endCursor - cursors are in edges[].cursor
-      }
-    }
-  }
-`;
-
-export const inventoryQuantitiesExtractionJson = schedule(
-  'inventory-quantities-extract-json-15min',
-  '*/15 * * * *',
-  http('extract-inventory-quantities-json', { connection: 'fluent_commerce', validateConnection: true }, async ctx => {
-    const { log, activation, openKv } = ctx;
-    const executionStartTime = Date.now();
-
-    // ========================================
-    // EXECUTION BOUNDARY: Workflow Start
-    // ========================================
-    log.info('🚀 [WORKFLOW] Inventory Quantities Extraction - Started', {
-      trigger: 'schedule',
-      schedule: '*/15 * * * *'
-    });
-
-    const retailerId = activation?.getVariable('retailerId');
-    const pageSize = parseInt(activation?.getVariable('pageSize') || '500', 10);
-    const maxRecords = parseInt(activation?.getVariable('maxRecords') || '20000', 10);
-    const fallbackStartDate =
-      activation?.getVariable('fallbackStartDate') || '2024-01-01T00:00:00Z';
-    const prettyPrint = activation?.getVariable('prettyPrint') === 'true';
-    const overlapBufferSeconds = parseInt(
-      activation?.getVariable('overlapBufferSeconds') || '60',
-      10
-    );
-    const OVERLAP_BUFFER_MS = overlapBufferSeconds * 1000;
-
-    const s3Config = {
-      bucket: activation?.getVariable('s3BucketName'),
-      region: activation?.getVariable('awsRegion') || 'us-east-1',
-      accessKeyId: activation?.getVariable('awsAccessKeyId'),
-      secretAccessKey: activation?.getVariable('awsSecretAccessKey'),
-    };
-    const s3Prefix = activation?.getVariable('s3Prefix') || 'inventory/quantities/';
-
-    // Validate required variables
-    const missing: string[] = [];
-    if (!retailerId) missing.push('retailerId');
-    if (!s3Config.bucket) missing.push('s3BucketName');
-    if (!s3Config.accessKeyId) missing.push('awsAccessKeyId');
-    if (!s3Config.secretAccessKey) missing.push('awsSecretAccessKey');
-    if (missing.length) {
-      log.error('❌ [VALIDATION] Missing required activation variables', { missing });
-      return {
-        success: false,
-        error: `Missing required variables: ${missing.join(', ')}`,
-        recommendation: 'Configure these variables in the Activation Variables section of your Versori connector'
-      };
-    }
-
-    try {
-      // STEP 1/8: Initialize KV state and job tracking
-      log.info('⚙️ [STEP 1/8] Initializing KV state and job tracking');
-      const kv = new VersoriKVAdapter(openKv(':project:'));
-      const tracker = new JobTracker(kv, log);
-      const stateKey = ['extraction', 'inventory-quantities-json', 'lastRunTime'];
-
-      // STEP 2/8: Create extraction job
-      log.info('⚙️ [STEP 2/8] Creating extraction job');
-      const job = await tracker.createJob({
-        type: 'extraction',
-        entity: 'inventoryQuantities',
-        config: {
-          extractionMode: 'incremental',
-          retailerId,
-          s3Bucket: s3Config.bucket,
-          s3Prefix,
-        },
-      });
-
-      log.info('✅ [JOB] Extraction job created', { jobId: job.id });
-
-      // STEP 3/8: Load last successful extraction timestamp
-      log.info('⚙️ [STEP 3/8] Loading last extraction timestamp');
-      const lastRunState = await kv.get(stateKey);
-      const rawLastRunTime = lastRunState?.value?.timestamp || fallbackStartDate;
-
-      // Apply overlap buffer for query (safety window)
-      const bufferedLastRunTime = new Date(
-        new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_MS
-      ).toISOString();
-
-      log.info('✅ [STATE] Incremental extraction window configured', {
-        jobId: job.id,
-        rawLastRunTime,
-        bufferedLastRunTime,
-        overlapBufferSeconds,
-      });
-
-      // STEP 4/8: Initialize Fluent client and orchestrator
-      log.info('⚙️ [STEP 4/8] Initializing Fluent client and orchestrator');
-      const client = await createClient(ctx);
-      const orchestrator = new ExtractionOrchestrator(client, log);
-
-      // STEP 5/8: Execute GraphQL query with auto-pagination
-      log.info('🔍 [STEP 5/8] Executing GraphQL extraction', {
-        query: 'inventoryQuantities',
-        pageSize,
-        maxRecords,
-        dateRange: `from ${bufferedLastRunTime}`,
-        retailerId,
-        jobId: job.id
-      });
-
-      const extractionStartTime = Date.now();
-      const result = await orchestrator.extract({
-        query: INVENTORY_QUANTITIES_QUERY,
-        resultPath: 'inventoryQuantities.edges.node',
-        variables: {
-          retailerId,
-          updatedAfter: bufferedLastRunTime,
-          // SDK adds 'first' automatically based on pageSize
-        },
-        pageSize,
-        maxRecords,
-      });
-      const extractionDuration = Date.now() - extractionStartTime;
-
-      const edges = result.data || [];
-
-      log.info('✅ [EXTRACTION] GraphQL extraction completed', {
-        totalRecords: result.stats.totalRecords,
-        totalPages: result.stats.totalPages,
-        validRecords: result.stats.validRecords ?? edges.length,
-        failedValidations: result.stats.failedValidations,
-        truncated: result.stats.truncated,
-        truncationReason: result.stats.truncationReason,
-        durationMs: extractionDuration,
-        jobId: job.id
-      });
-
-      if (edges.length === 0) {
-        log.info('ℹ️ [RESULT] No new inventory quantity records to extract');
-        const totalDuration = Date.now() - executionStartTime;
-        await tracker.markCompleted(job.id, {
-          recordsExtracted: 0,
-          message: 'No new records',
-          durationMs: totalDuration
-        });
-        await kv.set(stateKey, {
-          timestamp: new Date().toISOString(),
-          recordCount: 0,
-          extractedAt: new Date().toISOString(),
-        });
-
-        log.info('✅ [WORKFLOW] Inventory Quantities Extraction - Completed (No Records)', {
-          jobId: job.id,
-          durationMs: totalDuration
-        });
-
-        return {
-          success: true,
-          message: 'No new records to extract',
-          jobId: job.id,
-          rawLastRunTime,
-          durationMs: totalDuration
-        };
-      }
-
-      log.info('✅ [DATA] Inventory quantity records retrieved', { count: edges.length, jobId: job.id });
-
-      // STEP 6/8: Transform with UniversalMapper
-      log.info('⚙️ [STEP 6/8] Transforming records with UniversalMapper', { recordCount: edges.length });
-      const mappingStartTime = Date.now();
-      const mapper = new UniversalMapper(inventoryQuantitiesExportMapping);
-      const transformedRecords: any[] = [];
-      const errors: any[] = [];
-
-      for (const edge of edges) {
-        const mapped = await mapper.map(edge.node);
-        if (mapped.success) {
-          transformedRecords.push(mapped.data);
-        } else {
-          errors.push({
-            record: edge.node.id,
-            errors: mapped.errors,
-          });
-        }
-      }
-
-      const mappingDuration = Date.now() - mappingStartTime;
-
-      if (transformedRecords.length === 0) {
-        log.error('❌ [MAPPING] All records failed mapping validation', {
-          totalRecords: edges.length,
-          errorCount: errors.length,
-          jobId: job.id
-        });
-        await tracker.markFailed(job.id, {
-          error: 'All records failed mapping',
-          errors,
-        });
-        return {
-          success: false,
-          error: 'All records failed mapping',
-          jobId: job.id,
-          errors,
-          recommendation: 'Check mapping configuration in config/inventory-quantities.export.json and verify source data format'
-        };
-      }
-
-      log.info('✅ [MAPPING] Records transformed successfully', {
-        successful: transformedRecords.length,
-        failed: errors.length,
-        durationMs: mappingDuration,
-        jobId: job.id,
-      });
-
-      // Calculate max updatedOn for next run (without buffer)
-      const maxUpdatedOn = transformedRecords.reduce((max, record) => {
-        const recordTime = new Date(record.updatedOn).getTime();
-        return recordTime > max ? recordTime : max;
-      }, new Date(rawLastRunTime).getTime());
-
-      const newTimestamp = new Date(maxUpdatedOn).toISOString();
-
-      // STEP 7/8: Build JSON with metadata and upload to S3
-      log.info('⚙️ [STEP 7/8] Building JSON and uploading to S3');
-      const jsonBuildStartTime = Date.now();
-      const jsonOutput = {
-        metadata: {
-          extractedAt: new Date().toISOString(),
-          recordCount: transformedRecords.length,
-          incrementalFrom: rawLastRunTime,
-          incrementalTo: newTimestamp,
-          jobId: job.id,
-        },
-        data: transformedRecords,
-      };
-
-      // Use JSONBuilder for consistent JSON generation
-      const jsonBuilder = new JSONBuilder({
-        prettyPrint,
-        indent: 2,
-      });
-      const jsonContent = jsonBuilder.build(jsonOutput);
-      const jsonBuildDuration = Date.now() - jsonBuildStartTime;
-
-      const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
-      const fileName = `inventory-quantities-${timestamp}.json`;
-      const s3Key = `${s3Prefix}${fileName}`;
-
-      log.info('✅ [JSON] JSON file generated', {
-        fileName,
-        sizeBytes: jsonContent.length,
-        sizeMB: (jsonContent.length / (1024 * 1024)).toFixed(2),
-        recordCount: transformedRecords.length,
-        durationMs: jsonBuildDuration,
-        jobId: job.id,
-      });
-
-      const s3UploadStartTime = Date.now();
-      const s3 = new S3DataSource(
-        {
-          type: 'S3_JSON',
-          connectionId: 's3-inventory-quantities-json-export',
-          name: 'S3 Inventory Quantities JSON Export',
-          s3Config,
-        },
-        log
-      );
-
-      await s3.upload({
-        key: s3Key,
-        body: Buffer.from(jsonContent, 'utf8'),
-        contentType: 'application/json',
-        metadata: {
-          recordCount: String(transformedRecords.length),
-          extractedAt: new Date().toISOString(),
-          incrementalFrom: rawLastRunTime,
-          incrementalTo: newTimestamp,
-          jobId: job.id,
-        },
-      });
-      const s3UploadDuration = Date.now() - s3UploadStartTime;
-
-      log.info('✅ [S3] JSON file uploaded successfully', {
-        s3Key,
-        bucket: s3Config.bucket,
-        durationMs: s3UploadDuration,
-        jobId: job.id
-      });
-
-      // STEP 8/8: Update state and complete job
-      log.info('⚙️ [STEP 8/8] Updating state and completing job');
-      await kv.set(stateKey, {
-        timestamp: newTimestamp, // ← NO buffer applied
-        recordCount: transformedRecords.length,
-        extractedAt: new Date().toISOString(),
-        overlapBufferSeconds,
-        fileName,
-        s3Key,
-        jobId: job.id,
-        errors: errors.length > 0 ? errors : undefined,
-      });
-
-      const totalDuration = Date.now() - executionStartTime;
-      await tracker.markCompleted(job.id, {
-        recordsExtracted: transformedRecords.length,
-        recordsFailed: errors.length,
-        fileName,
-        s3Key,
-        newTimestamp,
-        durationMs: totalDuration,
-        performance: {
-          extractionMs: extractionDuration,
-          mappingMs: mappingDuration,
-          jsonBuildMs: jsonBuildDuration,
-          s3UploadMs: s3UploadDuration,
-          totalMs: totalDuration
-        }
-      });
-
-      // ========================================
-      // EXECUTION BOUNDARY: Workflow Success
-      // ========================================
-      log.info('✅ [WORKFLOW] Inventory Quantities Extraction - Completed Successfully', {
-        jobId: job.id,
-        recordsExtracted: transformedRecords.length,
-        recordsFailed: errors.length,
-        fileName,
-        s3Key,
-        durationMs: totalDuration,
-        performance: {
-          extractionMs: extractionDuration,
-          mappingMs: mappingDuration,
-          jsonBuildMs: jsonBuildDuration,
-          s3UploadMs: s3UploadDuration
-        }
-      });
-
-      return {
-        success: true,
-        jobId: job.id,
-        recordsExtracted: transformedRecords.length,
-        recordsFailed: errors.length,
-        fileName,
-        s3Key,
-        rawLastRunTime,
-        newTimestamp,
-        durationMs: totalDuration,
-        performance: {
-          extractionMs: extractionDuration,
-          mappingMs: mappingDuration,
-          jsonBuildMs: jsonBuildDuration,
-          s3UploadMs: s3UploadDuration,
-          totalMs: totalDuration
-        },
-        errors: errors.length > 0 ? errors : undefined,
-      };
-    } catch (error: any) {
-      const totalDuration = Date.now() - executionStartTime;
-      const errorMessage = error instanceof Error ? error.message : String(error);
-      const errorType = error instanceof Error ? error.constructor.name : 'UnknownError';
-
-      // ========================================
-      // EXECUTION BOUNDARY: Workflow Failure
-      // ========================================
-      log.error('❌ [WORKFLOW] Inventory Quantities Extraction - Failed', {
-        error: errorMessage,
-        errorType,
-        durationMs: totalDuration,
-        stack: error instanceof Error ? error.stack : undefined,
-      });
-
-      // Determine error recommendation based on error type
-      let recommendation = 'Check logs for details and verify configuration';
-      if (errorMessage.includes('authentication') || errorMessage.includes('credentials')) {
-        recommendation = 'Verify Fluent Commerce and S3 credentials in Activation Variables';
-      } else if (errorMessage.includes('network') || errorMessage.includes('timeout')) {
-        recommendation = 'Check network connectivity and increase timeout settings if needed';
-      } else if (errorMessage.includes('GraphQL') || errorMessage.includes('query')) {
-        recommendation = 'Verify GraphQL query syntax and ensure retailerId has access to inventory data';
-      } else if (errorMessage.includes('S3') || errorMessage.includes('upload')) {
-        recommendation = 'Verify S3 bucket exists, credentials are valid, and bucket permissions allow uploads';
-      } else if (errorMessage.includes('mapping') || errorMessage.includes('transform')) {
-        recommendation = 'Check mapping configuration in config/inventory-quantities.export.json';
-      }
-
-      return {
-        success: false,
-        error: errorMessage,
-        errorType,
-        durationMs: totalDuration,
-        stack: error instanceof Error ? error.stack : undefined,
-        recommendation
-      };
-    }
-  }));
-```
-
-### Workflow 2: Ad Hoc Extraction (HTTP Endpoint)
-
-**File:** `src/workflows/webhook/adhoc-inventory-quantities-extraction.ts`
-
-```typescript
-export const inventoryQuantitiesAdHocExtraction = webhook('inventory-quantities-adhoc-extract', {
-  connection: 'inventory-quantities-adhoc',
-}).then(http('execute-adhoc-extraction', { connection: 'fluent_commerce', validateConnection: true }, async ctx => {
-    const { log, openKv, activation } = ctx;
-    const executionStartTime = Date.now();
-
-    log.info('🚀 [WEBHOOK] Ad Hoc Extraction - Started', { trigger: 'webhook' });
-
-    const { startDate, endDate, retailerId } = ctx.request.body;
-
-    if (!startDate || !endDate || !retailerId) {
-      log.error('❌ [VALIDATION] Missing required webhook payload fields', {
-        hasStartDate: !!startDate,
-        hasEndDate: !!endDate,
-        hasRetailerId: !!retailerId
-      });
-      return {
-        success: false,
-        error: 'Missing required fields: startDate, endDate, retailerId',
-        recommendation: 'Provide all required fields in the webhook payload: { startDate, endDate, retailerId }'
-      };
-    }
-
-    try {
-      const kv = new VersoriKVAdapter(openKv(':project:'));
-      const tracker = new JobTracker(kv, log);
-
-      // Create ad hoc job
-      log.info('⚙️ Creating ad hoc extraction job');
-      const job = await tracker.createJob({
-        type: 'extraction',
-        entity: 'inventoryQuantities',
-        config: {
-          extractionMode: 'dateRange',
-          retailerId,
-          startDate,
-          endDate,
-        },
-      });
-
-      log.info('✅ [JOB] Ad hoc extraction job created', { jobId: job.id, startDate, endDate });
-
-      // Execute extraction (similar to scheduled workflow)
-      // ... (extraction logic here - implement full extraction as in scheduled workflow)
-
-      const totalDuration = Date.now() - executionStartTime;
-      log.info('✅ [WEBHOOK] Ad Hoc Extraction - Completed', {
-        jobId: job.id,
-        durationMs: totalDuration
-      });
-
-      return {
-        success: true,
-        jobId: job.id,
-        message: 'Ad hoc extraction started',
-        durationMs: totalDuration
-      };
-    } catch (error: any) {
-      const totalDuration = Date.now() - executionStartTime;
-      const errorMessage = error instanceof Error ? error.message : String(error);
-
-      log.error('❌ [WEBHOOK] Ad Hoc Extraction - Failed', {
-        error: errorMessage,
-        durationMs: totalDuration
-      });
-
-      return {
-        success: false,
-        error: errorMessage,
-        durationMs: totalDuration,
-        recommendation: 'Check logs for details and verify webhook payload format'
-      };
-    }
-  }
-);
-```
-
-### Workflow 3: Job Status Lookup
-
-**File:** `src/workflows/webhook/job-status-check.ts`
-
-```typescript
-export const inventoryQuantitiesJobStatus = webhook('inventory-quantities-job-status', {
-  connection: 'inventory-quantities-job-status',
-}).then(
-  http('query-job-status', { validateConnection: true }, async ctx => {
-    const { log, openKv } = ctx;
-    const executionStartTime = Date.now();
-
-    log.info('🚀 [WEBHOOK] Job Status Query - Started', { trigger: 'webhook' });
-
-    const { jobId } = ctx.request.body;
-
-    if (!jobId) {
-      log.error('❌ [VALIDATION] Missing required field: jobId');
-      return {
-        success: false,
-        error: 'Missing required field: jobId',
-        recommendation: 'Provide jobId in webhook payload: { jobId: "your-job-id" }'
-      };
-    }
-
-    try {
-      log.info('⚙️ Querying job status', { jobId });
-      const kv = new VersoriKVAdapter(openKv(':project:'));
-      const tracker = new JobTracker(kv, log);
-
-      const job = await tracker.getJob(jobId);
-
-      if (!job) {
-        log.warn('⚠️ Job not found', { jobId });
-        return {
-          success: false,
-          error: `Job not found: ${jobId}`,
-          recommendation: 'Verify the jobId is correct and the job was created successfully'
-        };
-      }
-
-      const totalDuration = Date.now() - executionStartTime;
-      log.info('✅ [WEBHOOK] Job Status Query - Completed', {
-        jobId,
-        status: job.status,
-        durationMs: totalDuration
-      });
-
-      return {
-        success: true,
-        job: {
-          id: job.id,
-          type: job.type,
-          entity: job.entity,
-        status: job.status,
-        createdAt: job.createdAt,
-        completedAt: job.completedAt,
-        result: job.result,
-        error: job.error,
-      },
-        durationMs: totalDuration
-      };
-    } catch (error: any) {
-      const totalDuration = Date.now() - executionStartTime;
-      const errorMessage = error instanceof Error ? error.message : String(error);
-
-      log.error('❌ [WEBHOOK] Job Status Query - Failed', {
-        error: errorMessage,
-        durationMs: totalDuration
-      });
-
-      return {
-        success: false,
-        error: errorMessage,
-        durationMs: totalDuration,
-        recommendation: 'Check logs for details and verify KV storage is accessible'
-      };
-    }
-  })
-);
-```
-
----
-
-## Key Differences from CSV
-
-1. **JSON Structure with Metadata:**
-
-   ```json
-   {
-     "metadata": { ... },  // ← Extraction context
-     "data": [ ... ]        // ← Actual records
-   }
-   ```
-
-2. **Pretty Printing Option:**
-   - Set `prettyPrint: true` for human-readable JSON
-   - Set `prettyPrint: false` for compact/production JSON
-
-3. **Content Type:**
-   - CSV: `text/csv`
-   - JSON: `application/json`
-
-4. **Field Names:**
-   - CSV: `location_code` (snake_case for compatibility)
-   - JSON: `location` (camelCase for APIs)
-
----
-
-## Use Cases
-
-**1. REST API Polling:**
-
-```bash
-# External system polls S3 for latest file
-aws s3 ls s3://api-inventory-exports/inventory/quantities/ --recursive | sort | tail -n 1
-```
-
-**2. S3 Event Trigger:**
-
-```typescript
-// Lambda triggered on S3 PUT event
-export const handler = async (event: S3Event) => {
-  const key = event.Records[0].s3.object.key;
-  const data = await s3.getObject({ Bucket: bucket, Key: key });
-  const json = JSON.parse(data.Body.toString());
-  await processInventoryQuantities(json.data);
-};
-```
-
-**3. API Gateway Integration:**
-
-```typescript
-// Return latest extraction file via API
-// GET /api/inventory/quantities/latest
-export async function handler(event: APIGatewayEvent) {
-  const latestFile = await getLatestS3File('inventory/quantities/');
-  const data = await s3.getObject({ Key: latestFile });
-  return {
-    statusCode: 200,
-    headers: { 'Content-Type': 'application/json' },
-    body: data.Body.toString(),
-  };
-}
-```
-
----
-
-## Production Checklist
-
-- [ ] Set appropriate extraction frequency (15min, hourly, daily)
-- [ ] Configure `maxRecords` based on expected change volume
-- [ ] Enable/disable pretty printing based on use case
-- [ ] Set up S3 lifecycle policy to archive old files
-- [ ] Document JSON schema for API consumers
-- [ ] Test with real-time incremental changes
-- [ ] Verify error handling for partial failures
-- [ ] Set up CloudFront CDN if serving via HTTP
-- [ ] Configure CORS if accessed from browser
-- [ ] Set up job tracking and monitoring
-- [ ] Test ad hoc extraction workflow
-- [ ] Validate job status lookup
-
----
-
-**Pattern**: Incremental extraction with overlap buffer and JobTracker integration
-**Key Learning**: Include metadata for API consumers, use 3-workflow pattern
-**Use Case**: Real-time inventory quantity sync for external APIs with job tracking
-**Buffer Pattern**: Query WITH buffer (`updatedOn >= lastRunTime - 60s`), save WITHOUT buffer (`MAX(updatedOn)`)
-
----
-
-### Pattern 8: Backward Pagination (Optional - Advanced)
-
-**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
-
-**When to Use**:
-
-- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
-- ✅ Time-bounded reverse traversal for auditing
-- ✅ Display newest-first in UI/reports
-- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
-
-**GraphQL Query Requirements**:
-
-Your query must support backward pagination by including `$last` and `$before`:
-
-```graphql
-query GetData(
-  $retailerId: ID!
-  $first: Int # For forward pagination
-  $after: String # For forward pagination
-  $last: Int # For backward pagination
-  $before: String # For backward pagination
-) {
-  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
-    edges {
-      cursor # ✅ REQUIRED
-      node {
-        id
-        createdAt
-        # ... other fields
-      }
-    }
-    pageInfo {
-      hasNextPage # For forward
-      hasPreviousPage # ✅ REQUIRED for backward
-    }
-  }
-}
-```
-
-**Implementation**:
-
-```typescript
-// Backward pagination - newest records first
-const result = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    retailerId,
-    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
-    // ❌ Don't include last/before - orchestrator injects them
-  },
-  pageSize: 200,
-  direction: 'backward', // ✅ Enable reverse pagination
-  maxRecords: 10000,
-});
-
-// Records are returned in reverse chronological order
-log.info('Newest record', { createdAt: result.data[0].createdAt });
-log.info('Oldest record', { createdAt: result.data[result.data.length - 1].createdAt });
-```
-
-**Key Differences from Forward Pagination**:
-
-| Aspect                 | Forward (Default)                | Backward                |
-| ---------------------- | -------------------------------- | ----------------------- |
-| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
-| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
-| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
-| **Cursor Source**      | Last edge of page                | First edge of page      |
-| **Record Order**       | Oldest → Newest                  | Newest → Oldest         |
-
-**Important Notes**:
-
-1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
-
-2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
-
-3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
-
-4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
-
-**Example: Extract Latest 1000 Orders**
-
-```typescript
-const latestOrders = await orchestrator.extract({
-  query: ORDERS_QUERY,
-  resultPath: 'orders.edges.node',
-  variables: {
-    retailerId,
-    statuses: ['BOOKED', 'ALLOCATED'],
-  },
-  direction: 'backward', // Start from newest
-  maxRecords: 1000, // Stop after 1000 records
-  pageSize: 100, // 100 per page = 10 pages
-});
-
-// latestOrders.data[0] is the newest order
-// latestOrders.data[999] is the 1000th newest order
-```
-
-**When to Use Forward vs Backward**:
-
-```typescript
-// ✅ Forward (default) - For incremental sync
-const incrementalData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    dateRangeFilter: { from: lastSyncTime, to: now },
-  },
-  // direction defaults to 'forward'
-  // Processes oldest → newest for proper sequencing
-});
-
-// ✅ Backward - For "latest N records" use cases
-const latestData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  direction: 'backward',
-  maxRecords: 100, // Just get latest 100
-  // Gets newest → oldest
-});
-```
-
-**Pagination Variables Reference**:
-
-| Variable | Forward     | Backward    | Injected By  | Notes                    |
-| -------- | ----------- | ----------- | ------------ | ------------------------ |
-| `first`  | ✅ Used     | ❌ Not used | Orchestrator | From `pageSize`          |
-| `after`  | ✅ Used     | ❌ Not used | Orchestrator | From cursor (last edge)  |
-| `last`   | ❌ Not used | ✅ Used     | Orchestrator | From `pageSize`          |
-| `before` | ❌ Not used | ✅ Used     | Orchestrator | From cursor (first edge) |
-
-**Common Mistakes to Avoid**:
-
-```typescript
-// ❌ WRONG - Don't pass pagination variables
-const result = await orchestrator.extract({
-  variables: {
-    last: 200, // ❌ Orchestrator will override this
-    before: cursor, // ❌ Orchestrator manages cursor
-  },
-  direction: 'backward',
-});
-
-// ✅ CORRECT - Let orchestrator inject pagination
-const result = await orchestrator.extract({
-  variables: {
-    retailerId, // ✅ Your business variables only
-  },
-  pageSize: 200, // ✅ Orchestrator uses this for last/before
-  direction: 'backward',
-});
-```
-
-#### Optional: Reverse Pagination
-
-- Forward remains default. Reverse requires $last/$before and pageInfo.hasPreviousPage.
-
-GraphQL:
-
-```graphql
-query GetInventoryQuantitiesBackward($retailerId: ID!, $last: Int!, $before: String) {
-  inventoryQuantities(retailerId: $retailerId, last: $last, before: $before) {
-    edges {
-      cursor
-      node {
-        id
-        ref
-        updatedOn
-      }
-    }
-    pageInfo {
-      hasPreviousPage
-    }
-  }
-}
-```
-
-SDK:
-
-```typescript
-await orchestrator.extract({
-  query: INVENTORY_QUANTITIES_BACKWARD_QUERY,
-  resultPath: 'inventoryQuantities.edges.node',
-  variables: { retailerId },
-  pageSize,
-  direction: 'backward',
-});
-```
-
----
-
-## Testing Checklist
-
-**Before production deployment:**
-
-### 1. Schema Validation
-
-- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
-- [ ] Run `npx fc-connect validate-schema --mapping ./config/inventory-quantities.export.json --schema ./fluent-schema.json`
-- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/inventory-quantities.export.json --schema ./fluent-schema.json`
-- [ ] Verify all `source` paths in mapping exist in GraphQL schema
-- [ ] Verify query structure matches schema (fields, types, filters)
-
-### 2. Extraction Testing
-
-- [ ] Test with small dataset first (maxRecords=10)
-- [ ] Verify ExtractionOrchestrator pagination works correctly
-- [ ] Test with multiple pages of data (verify cursor handling)
-- [ ] Verify date range filtering (updatedOn filter)
-- [ ] Test empty result handling (no records in date range)
-- [ ] Verify extraction stops at maxRecords limit
-
-### 3. Mapping Testing
-
-- [ ] Verify required fields are populated
-- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
-- [ ] Test custom resolvers with edge cases (if any)
-- [ ] Verify nested field extraction
-- [ ] Test with null/missing fields
-- [ ] Verify mapping error collection works
-
-### 4. JSON Generation Testing
-
-- [ ] Verify JSON structure matches expected format
-- [ ] Test JSON validation against schema (if applicable)
-- [ ] Verify proper nesting and structure
-- [ ] Test with large datasets (>1000 records)
-- [ ] Verify UTF-8 encoding
-- [ ] Test special character escaping
-
-### 5. S3 Upload Testing
-
-- [ ] Test S3 connection and authentication
-- [ ] Verify file upload to correct bucket and path
-- [ ] Test file naming convention (timestamp format)
-- [ ] Verify S3 object metadata
-- [ ] Test upload retry logic (simulate network failure)
-- [ ] Verify file permissions and ACLs
-
-### 6. State Management Testing
-
-- [ ] Verify overlap buffer prevents missed records (60-second default)
-- [ ] Test state recovery after extraction failure
-- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
-- [ ] Test first run with no previous state (uses fallbackStartDate)
-- [ ] Verify state update only happens on successful upload
-- [ ] Test manual date override (doesn't update state)
-
-### 7. Job Tracking Testing
-
-- [ ] Test job creation with JobTracker
-- [ ] Verify job status updates at each stage
-- [ ] Test job completion with metadata
-- [ ] Test job failure handling
-- [ ] Query job status via webhook endpoint
-- [ ] Verify job status persists in KV store
-
-### 8. Error Handling Testing
-
-- [ ] Test with invalid GraphQL query
-- [ ] Test with mapping errors (invalid field paths)
-- [ ] Test with S3 connection failures
-- [ ] Test with authentication failures
-- [ ] Test with network timeouts
-- [ ] Verify error logging includes context (jobId, stage, error details)
-- [ ] Test error threshold logic (if applicable)
-
-### 9. Staging Environment Testing
-
-- [ ] Run full extraction in staging environment
-- [ ] Verify JSON file format with downstream system
-- [ ] Monitor extraction duration and resource usage
-- [ ] Test with production-like data volumes
-- [ ] Verify no performance degradation over time
-
-### 10. Integration Testing
-
-- [ ] Test scheduled workflow (cron trigger)
-- [ ] Test ad hoc webhook trigger
-- [ ] Test job status query webhook
-- [ ] Verify activation variables are read correctly
-- [ ] Test with different extraction modes (incremental, date range)
-- [ ] End-to-end test: trigger → extract → transform → upload → verify file
-
----
-## Monitoring & Alerting
-
-### Success Response Example
-
-```json
-{
-  "success": true,
-  "jobId": "SCHEDULED_IQ_20251102_140000_abc123",
-  "recordsExtracted": 1523,
-  "fileName": "inventory-quantities-2025-11-02T14-00-00-000Z.json",
-  "s3Path": "s3://bucket/inventory-quantities/inventory-quantities-2025-11-02T14-00-00-000Z.json",
-  "metrics": {
-    "extractionDurationMs": 12543,
-    "totalPages": 8,
-    "pageSize": 200,
-    "mappingErrors": 0,
-    "fileSizeBytes": 524288,
-    "uploadDurationMs": 1234
-  },
-  "timestamps": {
-    "extractionStart": "2025-11-02T14:00:00.000Z",
-    "extractionEnd": "2025-11-02T14:00:12.543Z",
-    "uploadComplete": "2025-11-02T14:00:13.777Z"
-  },
-  "state": {
-    "previousTimestamp": "2025-11-02T13:00:00.000Z",
-    "newTimestamp": "2025-11-02T13:59:58.123Z",
-    "stateUpdated": true,
-    "overlapBufferSeconds": 60
-  }
-}
-```
-
-### Error Response Example
-
-```json
-{
-  "success": false,
-  "jobId": "ADHOC_IQ_20251102_140500_xyz789",
-  "error": "S3 upload failed: Connection timeout",
-  "errorCategory": "NETWORK",
-  "recordsExtracted": 0,
-  "stage": "s3_upload",
-  "details": {
-    "message": "Failed to upload file after 3 retry attempts",
-    "retryAttempts": 3,
-    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
-  },
-  "state": {
-    "stateUpdated": false,
-    "willRetryNextRun": true,
-    "note": "State not advanced - next extraction will retry same time window"
-  }
-}
-```
-
-### Key Metrics to Track
-
-```typescript
-const METRICS = {
-  // Extraction Performance
-  extractionDurationMs: Date.now() - extractionStart,
-  recordCount: records.length,
-  pageCount: extractionResult.stats.totalPages,
-  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
-
-  // Transformation Performance
-  transformedCount: transformedRecords.length,
-  failedCount: mappingErrors.length,
-  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
-
-  // File Generation
-  fileSizeMB: (jsonContent.length / (1024 * 1024)).toFixed(2),
-
-  // Upload Performance
-  uploadDurationMs: uploadEnd - uploadStart,
-  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
-
-  // State Management
-  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
-  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
-};
-
-log.info('Extraction metrics', metrics);
-```
-
-### Alert Thresholds
-
-```typescript
-const ALERT_THRESHOLDS = {
-  // Duration Alerts
-  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
-  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
-  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
-
-  // Error Rate Alerts
-  MAX_ERROR_RATE: 0.05, // 5% mapping errors
-  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
-
-  // Volume Alerts
-  MAX_RECORDS_PER_RUN: 100000,
-  MIN_RECORDS_WARNING: 0, // Alert if no records found
-  MAX_FILE_SIZE_MB: 150, // 150MB
-
-  // State Alerts
-  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
-  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
-};
-
-// Check thresholds
-if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
-  log.warn('Extraction duration exceeded threshold', {
-    duration: metrics.extractionDurationMs,
-    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
-    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
-  });
-}
-```
-
-### Monitoring Dashboard Queries
-
-**Versori Platform Logs Query:**
-
-```
-# Successful extractions
-log_level:info AND message:"Extraction complete" AND jobId:*
-
-# Failed extractions
-log_level:error AND message:"Extraction workflow failed" AND jobId:*
-
-# Performance issues
-extractionDurationMs:>300000 OR uploadDurationMs:>120000
-
-# High error rates
-errorRate:>5
-
-# State management issues
-stateUpdated:false AND success:true
-```
-
-### Common Issues and Solutions
-
-**Issue**: "Extraction timeout after 10 minutes"
-
-- **Cause**: Too many records in single extraction
-- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
-- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
-
-**Issue**: "Mapping errors for 50% of records"
-
-- **Cause**: Schema mismatch between GraphQL response and mapping config
-- **Fix**: Run schema validation, update mapping config paths
-- **Prevention**: Use `npx fc-connect validate-schema` before deployment
-
-**Issue**: "S3 connection timeout"
-
-- **Cause**: Network issues, firewall, or connection pool exhaustion
-- **Fix**: Check S3 credentials, verify network connectivity
-- **Prevention**: Implement connection health checks, monitor connection status
-
-**Issue**: "State not updating after successful extraction"
-
-- **Cause**: KV write failure or intentional retry logic
-- **Fix**: Check KV logs, verify state update code executed
-- **Prevention**: Add KV write verification, log state updates explicitly
-
-**Issue**: "First run exceeds record limits"
-
-- **Cause**: No previous timestamp, fetches all historical records
-- **Fix**: Set fallbackStartDate close to current date, apply additional filters
-- **Prevention**: Use appropriate fallbackStartDate for initial runs
-
-**Issue**: "Excessive duplicate records in output"
-
-- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
-- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
-- **Prevention**: Monitor duplicate rates, verify state update logic
-
----
-
-## Troubleshooting Quick Reference
-
-| Error Message | Likely Cause | Solution |
-|--------------|--------------|----------|
-| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
-| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
-| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
-| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
-| "S3 authentication failed" | Invalid credentials | Verify S3 credentials in activation variables |
-| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
-| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
-| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
-| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
-| "JSON generation failed" | Format-specific error | Check JSON generation logic, validate output |
-
----
+---
+template_id: tpl-extract-inventory-quantities-graphql-to-s3-json
+canonical_filename: template-extraction-inventory-quantities-to-s3-json.md
+version: 2.0.0
+sdk_version: ^0.1.39
+runtime: versori
+direction: extraction
+source: fluent-graphql
+destination: s3-json
+entity: inventory-quantities
+format: json
+logging: versori
+status: stable
+features:
+  - memory-management
+  - enhanced-logging
+  - pagination-progress
+---
+
+# Template: Extraction - Inventory Quantities GraphQL to S3 JSON
+
+**Template Version:** 2.0.0
+**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
+**Last Updated:** 2025-01-24
+**Deployment Target:** Versori Platform
+
+**🆕 Version 2.0.0 Enhancements:**
+- ✅ **Memory Management** - Clear large result sets after processing batches
+- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
+- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
+
+---
+
+## 📚 STEP 1: Load These Docs (Human Checklist)
+
+1. REQUIRED (load all)
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+
+Copy-paste list (open these):
+fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+
+---
+
+## 📋 STEP 2: Tell Your AI (Prompt)
+
+Copy/paste the standardized prompt from `docs/template-loading-matrix.md#prompts`.
+
+---
+
+## 💻 STEP 3: Implementation (Verified Imports)
+
+```ts
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  UniversalMapper,
+  S3DataSource,
+  JSONBuilder,
+  VersoriKVAdapter,
+  ExtractionOrchestrator,
+  JobTracker,
+} from '@fluentcommerce/fc-connect-sdk';
+```
+
+These are the only SDK imports required for this template. Keep type-only imports out of code samples. Prefer the orchestrator-based flow shown below for pagination and stats.
+
+Note on ad hoc runs: Use `fromDate`/`toDate` in webhook payloads. Default `updateState` is `false` for ad hoc; set to `true` only when you want the run to advance the saved timestamp used by scheduled runs.
+
+---
+
+# Versori Scheduled: Inventory Quantities Extraction to S3 JSON (Detailed Records)
+
+**FC Connect SDK Use Case Guide**
+
+> SDK: [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
+> Version: ^0.1.39
+
+Context: Scheduled Versori workflow that extracts inventory quantities (detailed quantity records) from Fluent Commerce via GraphQL query with **incremental timestamp tracking**, transforms with `UniversalMapper`, and writes JSON files to S3 for API consumption and audit trail systems.
+
+**Pattern**: EXTRACTION (Fluent → S3 JSON)
+**Complexity**: Medium | Runtime: Versori Platform (Scheduled)
+
+---
+
+## ⚠️ IMPORTANT: Sample Code for SDK Demonstration Only
+
+> **🔴 PRODUCTION RECOMMENDATION**
+>
+> This guide demonstrates FC Connect SDK capabilities for **extraction and mapping workflows**. This is a **Versori sample connector** showing how to build inventory quantity extraction workflows using the SDK.
+>
+> **✅ FOR PRODUCTION IMPLEMENTATIONS:**
+>
+> - **ONLY use INCREMENTAL mode with scheduled runs** (e.g., every 15 minutes for real-time)
+> - Incremental mode is safe, efficient, and production-ready
+> - Uses overlap buffer to prevent missed records
+> - Natural rate limiting via timestamps
+> - JSON format ideal for API consumption
+>
+> **🎯 RECOMMENDED SCHEDULE:**
+>
+> - **Every 15 minutes** for real-time inventory APIs
+> - **Hourly** for standard inventory feeds
+> - **Daily** for analytics/reporting systems
+>
+> **⚠️ NOTE:** This sample shows incremental mode implementation. For your production Versori connector, adapt this pattern with proper error handling, monitoring, and testing for your specific use case.
+>
+> **This is a reference implementation showing HOW to use the SDK - adapt for your needs.**
+
+---
+
+## What You'll Build
+
+- **Three workflows**: Scheduled extraction, ad hoc extraction, job status lookup
+- **Incremental extraction** using `updatedOn > lastRunTime` filter
+- **State management** with VersoriKVAdapter to track last successful run
+- **JobTracker integration** for KV-backed job tracking
+- GraphQL query with auto-pagination
+- UniversalMapper transformation for export schema
+- JSON file generation with proper structure
+- S3 upload to target bucket
+- **Failure recovery** - maintains last successful timestamp on errors
+
+## Business Use Case
+
+**Real-time inventory API for external systems:**
+
+- Extract inventory quantity changes every 15 minutes
+- Export as JSON for REST API consumption
+- Support for webhooks/polling by downstream systems
+- Lightweight incremental updates
+- Detailed audit trail with quantity types (AVAILABLE, RESERVED, EXPECTED, etc.)
+- Feed to order management and ecommerce platforms
+
+## Inventory Quantities Explained
+
+**InventoryQuantity** = Detailed quantity record with type breakdown
+
+- Individual quantity records by type (AVAILABLE, RESERVED, EXPECTED, etc.)
+- SKU + Location + Type combination
+- Retailer-defined types for specific tracking needs
+- Used for: Audit trails, detailed inventory tracking, compliance
+
+**vs InventoryPosition** = Physical on-hand calculation
+
+- Aggregated stock in warehouse
+- Used for: Stock reporting
+
+**vs VirtualPosition** = ATP (Available To Promise) calculation
+
+- Calculated quantity available for orders
+- Used for: Order promising
+
+---
+
+## SDK Methods Used
+
+```typescript
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  UniversalMapper,
+  S3DataSource,
+  JSONBuilder,
+  VersoriKVAdapter,
+  ExtractionOrchestrator,
+  JobTracker,
+} from '@fluentcommerce/fc-connect-sdk';
+
+// STEP 1: Create client
+const client = await createClient(ctx);
+
+// STEP 2: Setup job tracking
+const kv = new VersoriKVAdapter(ctx.openKv(':project:'));
+const tracker = new JobTracker(kv, log);
+
+// STEP 3: Create extraction job
+const job = await tracker.createJob({
+  type: 'extraction',
+  entity: 'inventoryQuantities',
+  config: { extractionMode: 'incremental' },
+});
+
+// STEP 4: Initialize orchestrator
+const orchestrator = new ExtractionOrchestrator(client, log);
+
+// STEP 5: Execute extraction with auto-pagination
+const result = await orchestrator.extract({
+  query: INVENTORY_QUANTITIES_QUERY,
+  resultPath: 'inventoryQuantities.edges.node',
+  variables: { retailerId, updatedAfter },
+  maxRecords: 20000,
+});
+
+// STEP 6: Transform with UniversalMapper
+const mapper = new UniversalMapper(exportMapping);
+const transformed = result.data.map(r => mapper.map(r));
+
+// STEP 7: Build JSON and upload to S3
+const jsonBuilder = new JSONBuilder({
+  prettyPrint: true,
+  indent: 2,
+});
+const jsonOutput = {
+  metadata: { extractedAt: new Date().toISOString(), recordCount: transformed.length },
+  data: transformed,
+};
+const jsonContent = jsonBuilder.build(jsonOutput);
+const s3 = new S3DataSource(config, log);
+await s3.upload({
+  key: s3Key,
+  body: Buffer.from(jsonContent, 'utf8'),
+  contentType: 'application/json'
+});
+
+// STEP 8: Update job status
+await tracker.markCompleted(job.id, { recordsExtracted: transformed.length, s3Key });
+```
+
+---
+
+## Activation Variables
+
+Configure these variables in your Versori connector's Activation Variables section:
+
+```json
+{
+  "retailerId": "your-retailer-id",
+  "s3BucketName": "api-inventory-exports",
+  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
+  "awsSecretAccessKey": "********",
+  "awsRegion": "us-east-1",
+  "s3Prefix": "inventory/quantities/",
+  "pageSize": 500,
+  "maxRecords": 20000,
+  "fallbackStartDate": "2024-01-01T00:00:00Z",
+  "overlapBufferSeconds": "60",
+  "prettyPrint": "true"
+}
+```
+
+**Variable Descriptions:**
+
+| Variable | Required | Description | Default | Example |
+|----------|----------|-------------|---------|---------|
+| `retailerId` | ✅ Yes | Fluent Commerce retailer ID for GraphQL queries | - | `"ACME_CORP"` |
+| `s3BucketName` | ✅ Yes | Target S3 bucket for JSON exports | - | `"api-inventory-exports"` |
+| `awsAccessKeyId` | ✅ Yes | AWS access key ID for S3 uploads | - | `"AKIAIOSFODNN7EXAMPLE"` |
+| `awsSecretAccessKey` | ✅ Yes | AWS secret access key for S3 uploads | - | `"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"` |
+| `awsRegion` | No | AWS region for S3 bucket | `"us-east-1"` | `"us-west-2"` |
+| `s3Prefix` | No | S3 key prefix for file organization | `"inventory/quantities/"` | `"exports/inventory/"` |
+| `pageSize` | No | GraphQL pagination page size | `500` | `200` |
+| `maxRecords` | No | Maximum records per extraction run | `20000` | `50000` |
+| `fallbackStartDate` | No | Initial timestamp for first run | `"2024-01-01T00:00:00Z"` | `"2025-01-01T00:00:00Z"` |
+| `overlapBufferSeconds` | No | Safety buffer to prevent missed records (seconds) | `60` | `120` |
+| `prettyPrint` | No | Pretty-print JSON output (`"true"` or `"false"`) | `"true"` | `"false"` |
+
+**🔒 Security Notes:**
+- Store credentials securely in Versori's encrypted variable storage
+- Never commit credentials to source control
+- Rotate AWS keys regularly
+- Use IAM roles with minimal required permissions
+
+**⚙️ Performance Tuning:**
+- **pageSize**: Lower values (100-200) for slower networks, higher values (500-1000) for faster connections
+- **maxRecords**: Adjust based on expected change volume - 20K is safe for 15-minute incremental runs
+- **overlapBufferSeconds**: Increase if you see missed records, decrease if seeing too many duplicates
+
+---
+
+## Export Mapping Configuration
+
+Create file: `./config/inventory-quantities.export.json`
+
+```json
+{
+  "name": "inventory-quantities.export",
+  "version": "1.0.0",
+  "description": "Fluent Inventory Quantities → JSON API Export Mapping",
+  "fields": {
+    "ref": { "source": "ref", "required": true, "resolver": "sdk.trim" },
+    "location": { "source": "locationRef", "required": true, "resolver": "sdk.trim" },
+    "sku": { "source": "articleRef", "required": true, "resolver": "sdk.trim" },
+    "quantity": { "source": "qty", "required": true, "resolver": "sdk.parseInt" },
+    "type": { "source": "type", "required": true, "resolver": "sdk.uppercase" },
+    "condition": { "source": "condition", "required": false, "resolver": "sdk.uppercase" },
+    "status": { "source": "status", "required": true, "resolver": "sdk.uppercase" },
+    "storageArea": { "source": "storageAreaRef", "required": false, "resolver": "sdk.trim" },
+    "catalogueRef": { "source": "catalogue.ref", "required": false, "resolver": "sdk.trim" },
+    "catalogueName": { "source": "catalogue.name", "required": false, "resolver": "sdk.trim" },
+    "expectedDate": { "source": "expectedOn", "required": false, "resolver": "sdk.formatDate" },
+    "createdOn": { "source": "createdOn", "required": true, "resolver": "sdk.toString" },
+    "updatedOn": { "source": "updatedOn", "required": true, "resolver": "sdk.toString" }
+  }
+}
+```
+
+---
+
+## Mapping & Resolvers Explained
+
+This section explains how the SDK transforms raw GraphQL data into your JSON export format using **UniversalMapper** and **SDK resolvers**.
+
+### SDK Resolvers Used
+
+| Field           | Resolver         | Why?                                          | Example Transformation                                      |
+| --------------- | ---------------- | --------------------------------------------- | ----------------------------------------------------------- |
+| `ref`           | `sdk.trim`       | Clean quantity record references              | `" QTY-001 "` → `"QTY-001"`                                 |
+| `location`      | `sdk.trim`       | Clean location references                     | `" DC01 "` → `"DC01"`                                       |
+| `sku`           | `sdk.trim`       | Clean SKU references                          | `" SKU-001 "` → `"SKU-001"`                                 |
+| `quantity`      | `sdk.parseInt`   | Parse quantity as integer for JSON APIs       | `"150"` → `150`                                             |
+| `type`          | `sdk.uppercase`  | Normalize type codes for consistency          | `"available"` → `"AVAILABLE"`                               |
+| `condition`     | `sdk.uppercase`  | Normalize condition codes                     | `"good"` → `"GOOD"`                                         |
+| `status`        | `sdk.uppercase`  | Normalize status codes                        | `"active"` → `"ACTIVE"`                                     |
+| `storageArea`   | `sdk.trim`       | Clean storage area references                 | `" ZONE-A "` → `"ZONE-A"`                                   |
+| `catalogueRef`  | `sdk.trim`       | Clean catalogue references from nested object | `" DEFAULT_CATALOGUE "` → `"DEFAULT_CATALOGUE"`             |
+| `catalogueName` | `sdk.trim`       | Clean catalogue names from nested object      | `" Default Inventory "` → `"Default Inventory"`             |
+| `expectedDate`  | `sdk.formatDate` | Format dates for JSON APIs                    | `"2025-01-30T00:00:00.000Z"` → `"2025-01-30"`               |
+| `createdOn`     | `sdk.toString`   | Preserve ISO 8601 timestamp string            | `"2025-01-22T10:00:00.000Z"` → `"2025-01-22T10:00:00.000Z"` |
+| `updatedOn`     | `sdk.toString`   | Preserve ISO 8601 timestamp for tracking      | `"2025-01-22T10:30:00.000Z"` → `"2025-01-22T10:30:00.000Z"` |
+
+### Transformation Flow
+
+```typescript
+// 1. GraphQL Response (raw data from Fluent Commerce)
+const rawQuantity = {
+  ref: ' QTY-001 ',
+  locationRef: ' DC01 ',
+  articleRef: ' SKU-001 ',
+  qty: '150',
+  type: 'available',
+  condition: 'good',
+  status: 'active',
+  storageAreaRef: ' ZONE-A ',
+  catalogue: {
+    ref: ' DEFAULT_CATALOGUE ',
+    name: ' Default Inventory ',
+  },
+  expectedOn: '2025-01-25T00:00:00.000Z',
+  createdOn: '2025-01-20T10:00:00.000Z',
+  updatedOn: '2025-01-22T10:30:00.000Z',
+};
+
+// 2. UniversalMapper applies SDK resolvers
+const mapper = new UniversalMapper(inventoryQuantitiesExportMapping);
+const result = await mapper.map(rawQuantity);
+
+// 3. Transformed Output (JSON-friendly camelCase)
+const transformedQuantity = {
+  ref: 'QTY-001',
+  location: 'DC01',
+  sku: 'SKU-001',
+  quantity: 150, // Parsed as number for JSON
+  type: 'AVAILABLE',
+  condition: 'GOOD',
+  status: 'ACTIVE',
+  storageArea: 'ZONE-A',
+  catalogueRef: 'DEFAULT_CATALOGUE',
+  catalogueName: 'Default Inventory',
+  expectedDate: '2025-01-25',
+  createdOn: '2025-01-20T10:00:00.000Z',
+  updatedOn: '2025-01-22T10:30:00.000Z', // ISO 8601 for APIs
+};
+
+// 4. Final JSON Structure with Metadata
+const jsonOutput = {
+  metadata: {
+    extractedAt: '2025-01-22T14:30:00.000Z',
+    recordCount: 1,
+    incrementalFrom: '2025-01-22T10:00:00.000Z',
+    incrementalTo: '2025-01-22T14:30:00.000Z',
+  },
+  data: [transformedQuantity],
+};
+```
+
+### Custom Resolvers for Inventory Quantity-Specific Logic
+
+While the mapping above uses built-in SDK resolvers, you can extend with custom business logic for API enrichment:
+
+```typescript
+const customResolvers = {
+  /**
+   * Validate and normalize quantity values for API consumption
+   */
+  'custom.normalizeQuantity': (qty: any) => {
+    const parsed = parseInt(qty) || 0;
+    return Math.max(0, parsed); // Ensure non-negative for APIs
+  },
+
+  /**
+   * Add human-readable type descriptions for API responses
+   */
+  'custom.enrichQuantityType': (type: string, sourceData: any) => {
+    const typeDescriptions: Record<string, string> = {
+      AVAILABLE: 'Available for Sale',
+      RESERVED: 'Reserved by Order',
+      EXPECTED: 'Expected Arrival',
+      ADJUSTMENT: 'Inventory Adjustment',
+      DAMAGED: 'Damaged/Unsellable',
+      IN_TRANSIT: 'In Transit to Location',
+    };
+    return {
+      code: type.toUpperCase(),
+      description: typeDescriptions[type.toUpperCase()] || type,
+      isAvailableForSale: type.toUpperCase() === 'AVAILABLE',
+    };
+  },
+
+  /**
+   * Calculate days until expected arrival for EXPECTED quantities
+   */
+  'custom.calculateExpectedArrival': (expectedOn: string | null) => {
+    if (!expectedOn) return null;
+
+    const expected = new Date(expectedOn);
+    const today = new Date();
+    const diffMs = expected.getTime() - today.getTime();
+    const daysUntil = Math.ceil(diffMs / (1000 * 60 * 60 * 24));
+
+    return {
+      date: expectedOn,
+      daysUntil: daysUntil,
+      isOverdue: daysUntil < 0,
+      isPending: daysUntil >= 0,
+    };
+  },
+
+  /**
+   * Generate API-friendly status summary
+   */
+  'custom.generateStatusSummary': (quantity: any) => {
+    const type = (quantity.type || '').toUpperCase();
+    const status = (quantity.status || '').toUpperCase();
+    const qty = parseInt(quantity.qty) || 0;
+
+    return {
+      quantity: qty,
+      type: type,
+      status: status,
+      isActive: status === 'ACTIVE',
+      isAvailable: type === 'AVAILABLE' && status === 'ACTIVE',
+      needsAction: type === 'EXPECTED' && !quantity.expectedOn,
+      stockLevel: qty === 0 ? 'OUT_OF_STOCK' : qty < 10 ? 'LOW_STOCK' : 'IN_STOCK',
+    };
+  },
+
+  /**
+   * Format for real-time inventory API responses
+   */
+  'custom.formatForRealtimeAPI': (quantity: any) => {
+    return {
+      location: quantity.locationRef?.trim(),
+      sku: quantity.articleRef?.trim(),
+      available: quantity.type === 'AVAILABLE' ? parseInt(quantity.qty) || 0 : 0,
+      reserved: quantity.type === 'RESERVED' ? parseInt(quantity.qty) || 0 : 0,
+      expected: quantity.type === 'EXPECTED' ? parseInt(quantity.qty) || 0 : 0,
+      expectedDate: quantity.expectedOn || null,
+      updatedOn: quantity.updatedOn,
+      isActive: quantity.status === 'ACTIVE',
+    };
+  },
+};
+
+// Use custom resolvers with UniversalMapper
+const mapper = new UniversalMapper(inventoryQuantitiesExportMapping, {
+  customResolvers,
+});
+```
+
+### JSON-Specific Mapping Considerations
+
+**1. camelCase vs snake_case:**
+
+```typescript
+// CSV export: snake_case for compatibility
+{ "location_code": "DC01", "last_updated": "..." }
+
+// JSON export: camelCase for APIs
+{ "location": "DC01", "updatedOn": "..." }
+```
+
+**2. Number Types:**
+
+```typescript
+// JSON preserves number types (no quotes)
+{ "quantity": 150 }  // ✓ Correct for JSON APIs
+
+// CSV requires strings
+"150"  // ✓ Correct for CSV
+```
+
+**3. Null Handling:**
+
+```typescript
+// JSON can use null
+{ "expectedDate": null }  // ✓ Valid JSON
+
+// CSV uses empty string
+""  // ✓ Valid CSV
+```
+
+**4. Metadata Wrapper:**
+
+```json
+{
+  "metadata": {
+    "extractedAt": "2025-01-22T14:30:00.000Z",
+    "recordCount": 2,
+    "incrementalFrom": "...",
+    "incrementalTo": "..."
+  },
+  "data": [
+    /* transformed records */
+  ]
+}
+```
+
+### Available SDK Resolvers
+
+The SDK provides these built-in resolvers (no custom code needed):
+
+**String Transformations:**
+
+- `sdk.trim` - Remove leading/trailing whitespace
+- `sdk.uppercase` - Convert to uppercase
+- `sdk.lowercase` - Convert to lowercase
+- `sdk.toString` - Convert to string
+
+**Number Parsing:**
+
+- `sdk.parseInt` - Parse as integer (ideal for JSON)
+- `sdk.parseFloat` - Parse as decimal
+- `sdk.number` - Parse as number (auto-detect int/float)
+
+**Date Formatting:**
+
+- `sdk.formatDate` - ISO 8601 date only (YYYY-MM-DD)
+- `sdk.formatDateShort` - Short date format
+- `sdk.parseDate` - Parse various date formats
+
+**Type Conversions:**
+
+- `sdk.boolean` - Convert to boolean
+- `sdk.parseJson` - Parse JSON strings
+- `sdk.toJson` - Convert to JSON string
+
+**Utilities:**
+
+- `sdk.identity` - Return value unchanged
+- `sdk.coalesce` - Return first non-null value
+
+See [Universal Mapping Guide](../../../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) for complete resolver documentation.
+
+---
+
+## GraphQL Query
+
+**Note**: This query is verified against Fluent Commerce schema introspection.
+
+```graphql
+query GetInventoryQuantities(
+  $retailerId: ID!
+  $updatedAfter: DateTime!
+  $first: Int!
+  $after: String
+) {
+  inventoryQuantities(
+    retailerId: $retailerId
+    updatedOn: { after: $updatedAfter }
+    first: $first
+    after: $after
+  ) {
+    edges {
+      node {
+        id
+        ref
+        locationRef
+        articleRef
+        qty
+        type
+        condition
+        status
+        storageAreaRef
+        catalogue {
+          ref
+          name
+        }
+        expectedOn
+        createdOn
+        updatedOn
+      }
+      cursor
+    }
+    pageInfo {
+      hasNextPage
+      # Note: Fluent doesn't return endCursor - cursors are in edges[].cursor
+    }
+  }
+}
+```
+
+---
+
+## Versori Workflows Structure
+
+**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
+
+**Trigger Types:**
+- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
+- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
+- **`workflow()`** → Durable workflows (advanced, rarely used)
+
+**Execution Steps (chained to triggers):**
+- **`http()`** → External API calls (chained from schedule/webhook)
+- **`fn()`** → Internal processing (chained from schedule/webhook)
+
+### Recommended Project Structure
+
+This template demonstrates a modular project structure with separate files for better organization and maintainability:
+
+```
+inventory-quantities-extraction/
+├── index.ts                              # Entry point - exports all workflows
+└── src/
+    ├── workflows/
+    │   ├── scheduled/
+    │   │   └── daily-inventory-quantities-extraction.ts  # Scheduled: Daily extraction
+    │   │
+    │   └── webhook/
+    │       ├── adhoc-inventory-quantities-extraction.ts  # Webhook: Manual trigger
+    │       └── job-status-check.ts                      # Webhook: Status query
+    │
+    ├── services/
+    │   └── inventory-quantities-extraction.service.ts    # Shared orchestration logic (reusable)
+    │
+    └── config/
+        └── inventory-quantities.export.json             # Mapping configuration
+```
+
+### Entry Point: index.ts
+
+**File:** `index.ts`
+
+The entry point uses the **MemoryInterpreter pattern** to export all workflows for Versori:
+
+```typescript
+/**
+ * Entry point - Export all workflows for Versori platform
+ *
+ * VERSORI MEMORY INTERPRETER PATTERN:
+ * This file exports all workflows to be registered with Versori.
+ * Each workflow is defined in its own file for better organization.
+ *
+ * Import and re-export pattern allows Versori to discover and register
+ * all workflows without manual configuration.
+ */
+
+// Scheduled workflows
+export { inventoryQuantitiesExtractionJson } from './src/workflows/scheduled/daily-inventory-quantities-extraction';
+
+// Webhook workflows
+export { inventoryQuantitiesAdHocExtraction } from './src/workflows/webhook/adhoc-inventory-quantities-extraction';
+export { inventoryQuantitiesJobStatus } from './src/workflows/webhook/job-status-check';
+```
+
+**Key Points:**
+- ✅ Simple re-export pattern - Versori auto-discovers workflows
+- ✅ Each workflow in separate file for maintainability
+- ✅ Clear naming: `export { workflowName } from './path/to/workflow'`
+- ❌ No configuration needed - Versori reads exports directly
+
+---
+
+## Example Output
+
+The extraction generates JSON files with this structure:
+
+```json
+{
+  "metadata": {
+    "extractedAt": "2025-01-22T14:30:00.000Z",
+    "recordCount": 2,
+    "incrementalFrom": "2025-01-22T10:00:00.000Z",
+    "incrementalTo": "2025-01-22T14:30:00.000Z"
+  },
+  "data": [
+    {
+      "ref": "QTY-001",
+      "location": "DC01",
+      "sku": "SKU-001",
+      "quantity": 150,
+      "type": "AVAILABLE",
+      "condition": "GOOD",
+      "status": "ACTIVE",
+      "storageArea": "ZONE-A",
+      "catalogueRef": "DEFAULT_CATALOGUE",
+      "catalogueName": "Default Inventory",
+      "expectedDate": null,
+      "createdOn": "2025-01-20T10:00:00Z",
+      "updatedOn": "2025-01-22T10:30:00Z"
+    },
+    {
+      "ref": "QTY-002",
+      "location": "DC02",
+      "sku": "SKU-002",
+      "quantity": 200,
+      "type": "RESERVED",
+      "condition": "GOOD",
+      "status": "ACTIVE",
+      "storageArea": "ZONE-B",
+      "catalogueRef": "DEFAULT_CATALOGUE",
+      "catalogueName": "Default Inventory",
+      "expectedDate": null,
+      "createdOn": "2025-01-21T11:00:00Z",
+      "updatedOn": "2025-01-22T11:15:00Z"
+    }
+  ]
+}
+```
+
+---
+
+## Production Safety & Guardrails
+
+### Overview
+
+Even with **incremental-only** extraction, inventory quantities in JSON format need safeguards:
+
+- **JSON parsing memory**: API consumers parse entire JSON before processing
+- **Real-time APIs**: Inventory feeds power order promising, need fast processing
+- **High-frequency updates**: Every 15 minutes accumulates large datasets
+- **Nested structure**: JSON metadata wrapper adds overhead vs CSV
+
+### Hard Limits
+
+```typescript
+const SAFETY_LIMITS = {
+  MAX_RECORDS_PER_RUN: 300000, // 300k quantity records per run
+  MAX_RECORDS_PER_FILE: 50000, // 50k per JSON file (lower than CSV)
+  MAX_FILE_SIZE_MB: 100, // 100MB per file
+  MAX_JSON_SIZE_MB: 200, // Total extraction size
+  CHUNK_SIZE: 10000, // Process in chunks
+  ESTIMATED_BYTES_PER_RECORD: 400, // JSON with metadata
+};
+```
+
+**Why JSON has different limits than CSV?**
+
+- JSON includes field names in every record (overhead)
+- JSON metadata wrapper adds size
+- JSON.parse() is memory-intensive
+- API consumers need predictable payload sizes
+
+---
+
+## Complete Workflow Implementation
+
+The code examples below demonstrate the implementation of each workflow component. In your project, these would be organized into separate files following the modular structure shown above.
+
+### Workflow 1: Scheduled Extraction (Incremental)
+
+**File:** `src/workflows/scheduled/daily-inventory-quantities-extraction.ts`
+
+```typescript
+import { schedule, http } from '@versori/run';
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  UniversalMapper,
+  S3DataSource,
+  JSONBuilder,
+  VersoriKVAdapter,
+  ExtractionOrchestrator,
+  JobTracker,
+} from '@fluentcommerce/fc-connect-sdk';
+import inventoryQuantitiesExportMapping from './config/inventory-quantities.export.json' with { type: 'json' };
+
+// GraphQL query
+const INVENTORY_QUANTITIES_QUERY = `
+  query GetInventoryQuantities(
+    $retailerId: ID!
+    $updatedAfter: DateTime!
+    $first: Int!
+    $after: String
+  ) {
+    inventoryQuantities(
+      retailerId: $retailerId
+      updatedOn: { after: $updatedAfter }
+      first: $first
+      after: $after
+    ) {
+      edges {
+        node {
+          id
+          ref
+          locationRef
+          articleRef
+          qty
+          type
+          condition
+          status
+          storageAreaRef
+          catalogue {
+            ref
+            name
+          }
+          expectedOn
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+        # Note: Fluent doesn't return endCursor - cursors are in edges[].cursor
+      }
+    }
+  }
+`;
+
+export const inventoryQuantitiesExtractionJson = schedule(
+  'inventory-quantities-extract-json-15min',
+  '*/15 * * * *',
+  http('extract-inventory-quantities-json', { connection: 'fluent_commerce', validateConnection: true }, async ctx => {
+    const { log, activation, openKv } = ctx;
+    const executionStartTime = Date.now();
+
+    // ========================================
+    // EXECUTION BOUNDARY: Workflow Start
+    // ========================================
+    log.info('🚀 [WORKFLOW] Inventory Quantities Extraction - Started', {
+      trigger: 'schedule',
+      schedule: '*/15 * * * *'
+    });
+
+    const retailerId = activation?.getVariable('retailerId');
+    const pageSize = parseInt(activation?.getVariable('pageSize') || '500', 10);
+    const maxRecords = parseInt(activation?.getVariable('maxRecords') || '20000', 10);
+    const fallbackStartDate =
+      activation?.getVariable('fallbackStartDate') || '2024-01-01T00:00:00Z';
+    const prettyPrint = activation?.getVariable('prettyPrint') === 'true';
+    const overlapBufferSeconds = parseInt(
+      activation?.getVariable('overlapBufferSeconds') || '60',
+      10
+    );
+    const OVERLAP_BUFFER_MS = overlapBufferSeconds * 1000;
+
+    const s3Config = {
+      bucket: activation?.getVariable('s3BucketName'),
+      region: activation?.getVariable('awsRegion') || 'us-east-1',
+      accessKeyId: activation?.getVariable('awsAccessKeyId'),
+      secretAccessKey: activation?.getVariable('awsSecretAccessKey'),
+    };
+    const s3Prefix = activation?.getVariable('s3Prefix') || 'inventory/quantities/';
+
+    // Validate required variables
+    const missing: string[] = [];
+    if (!retailerId) missing.push('retailerId');
+    if (!s3Config.bucket) missing.push('s3BucketName');
+    if (!s3Config.accessKeyId) missing.push('awsAccessKeyId');
+    if (!s3Config.secretAccessKey) missing.push('awsSecretAccessKey');
+    if (missing.length) {
+      log.error('❌ [VALIDATION] Missing required activation variables', { missing });
+      return {
+        success: false,
+        error: `Missing required variables: ${missing.join(', ')}`,
+        recommendation: 'Configure these variables in the Activation Variables section of your Versori connector'
+      };
+    }
+
+    try {
+      // STEP 1/8: Initialize KV state and job tracking
+      log.info('⚙️ [STEP 1/8] Initializing KV state and job tracking');
+      const kv = new VersoriKVAdapter(openKv(':project:'));
+      const tracker = new JobTracker(kv, log);
+      const stateKey = ['extraction', 'inventory-quantities-json', 'lastRunTime'];
+
+      // STEP 2/8: Create extraction job
+      log.info('⚙️ [STEP 2/8] Creating extraction job');
+      const job = await tracker.createJob({
+        type: 'extraction',
+        entity: 'inventoryQuantities',
+        config: {
+          extractionMode: 'incremental',
+          retailerId,
+          s3Bucket: s3Config.bucket,
+          s3Prefix,
+        },
+      });
+
+      log.info('✅ [JOB] Extraction job created', { jobId: job.id });
+
+      // STEP 3/8: Load last successful extraction timestamp
+      log.info('⚙️ [STEP 3/8] Loading last extraction timestamp');
+      const lastRunState = await kv.get(stateKey);
+      const rawLastRunTime = lastRunState?.value?.timestamp || fallbackStartDate;
+
+      // Apply overlap buffer for query (safety window)
+      const bufferedLastRunTime = new Date(
+        new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_MS
+      ).toISOString();
+
+      log.info('✅ [STATE] Incremental extraction window configured', {
+        jobId: job.id,
+        rawLastRunTime,
+        bufferedLastRunTime,
+        overlapBufferSeconds,
+      });
+
+      // STEP 4/8: Initialize Fluent client and orchestrator
+      log.info('⚙️ [STEP 4/8] Initializing Fluent client and orchestrator');
+      const client = await createClient(ctx);
+      const orchestrator = new ExtractionOrchestrator(client, log);
+
+      // STEP 5/8: Execute GraphQL query with auto-pagination
+      log.info('🔍 [STEP 5/8] Executing GraphQL extraction', {
+        query: 'inventoryQuantities',
+        pageSize,
+        maxRecords,
+        dateRange: `from ${bufferedLastRunTime}`,
+        retailerId,
+        jobId: job.id
+      });
+
+      const extractionStartTime = Date.now();
+      const result = await orchestrator.extract({
+        query: INVENTORY_QUANTITIES_QUERY,
+        resultPath: 'inventoryQuantities.edges.node',
+        variables: {
+          retailerId,
+          updatedAfter: bufferedLastRunTime,
+          // SDK adds 'first' automatically based on pageSize
+        },
+        pageSize,
+        maxRecords,
+      });
+      const extractionDuration = Date.now() - extractionStartTime;
+
+      const edges = result.data || [];
+
+      log.info('✅ [EXTRACTION] GraphQL extraction completed', {
+        totalRecords: result.stats.totalRecords,
+        totalPages: result.stats.totalPages,
+        validRecords: result.stats.validRecords ?? edges.length,
+        failedValidations: result.stats.failedValidations,
+        truncated: result.stats.truncated,
+        truncationReason: result.stats.truncationReason,
+        durationMs: extractionDuration,
+        jobId: job.id
+      });
+
+      if (edges.length === 0) {
+        log.info('ℹ️ [RESULT] No new inventory quantity records to extract');
+        const totalDuration = Date.now() - executionStartTime;
+        await tracker.markCompleted(job.id, {
+          recordsExtracted: 0,
+          message: 'No new records',
+          durationMs: totalDuration
+        });
+        await kv.set(stateKey, {
+          timestamp: new Date().toISOString(),
+          recordCount: 0,
+          extractedAt: new Date().toISOString(),
+        });
+
+        log.info('✅ [WORKFLOW] Inventory Quantities Extraction - Completed (No Records)', {
+          jobId: job.id,
+          durationMs: totalDuration
+        });
+
+        return {
+          success: true,
+          message: 'No new records to extract',
+          jobId: job.id,
+          rawLastRunTime,
+          durationMs: totalDuration
+        };
+      }
+
+      log.info('✅ [DATA] Inventory quantity records retrieved', { count: edges.length, jobId: job.id });
+
+      // STEP 6/8: Transform with UniversalMapper
+      log.info('⚙️ [STEP 6/8] Transforming records with UniversalMapper', { recordCount: edges.length });
+      const mappingStartTime = Date.now();
+      const mapper = new UniversalMapper(inventoryQuantitiesExportMapping);
+      const transformedRecords: any[] = [];
+      const errors: any[] = [];
+
+      for (const edge of edges) {
+        const mapped = await mapper.map(edge.node);
+        if (mapped.success) {
+          transformedRecords.push(mapped.data);
+        } else {
+          errors.push({
+            record: edge.node.id,
+            errors: mapped.errors,
+          });
+        }
+      }
+
+      const mappingDuration = Date.now() - mappingStartTime;
+
+      if (transformedRecords.length === 0) {
+        log.error('❌ [MAPPING] All records failed mapping validation', {
+          totalRecords: edges.length,
+          errorCount: errors.length,
+          jobId: job.id
+        });
+        await tracker.markFailed(job.id, {
+          error: 'All records failed mapping',
+          errors,
+        });
+        return {
+          success: false,
+          error: 'All records failed mapping',
+          jobId: job.id,
+          errors,
+          recommendation: 'Check mapping configuration in config/inventory-quantities.export.json and verify source data format'
+        };
+      }
+
+      log.info('✅ [MAPPING] Records transformed successfully', {
+        successful: transformedRecords.length,
+        failed: errors.length,
+        durationMs: mappingDuration,
+        jobId: job.id,
+      });
+
+      // Calculate max updatedOn for next run (without buffer)
+      const maxUpdatedOn = transformedRecords.reduce((max, record) => {
+        const recordTime = new Date(record.updatedOn).getTime();
+        return recordTime > max ? recordTime : max;
+      }, new Date(rawLastRunTime).getTime());
+
+      const newTimestamp = new Date(maxUpdatedOn).toISOString();
+
+      // STEP 7/8: Build JSON with metadata and upload to S3
+      log.info('⚙️ [STEP 7/8] Building JSON and uploading to S3');
+      const jsonBuildStartTime = Date.now();
+      const jsonOutput = {
+        metadata: {
+          extractedAt: new Date().toISOString(),
+          recordCount: transformedRecords.length,
+          incrementalFrom: rawLastRunTime,
+          incrementalTo: newTimestamp,
+          jobId: job.id,
+        },
+        data: transformedRecords,
+      };
+
+      // Use JSONBuilder for consistent JSON generation
+      const jsonBuilder = new JSONBuilder({
+        prettyPrint,
+        indent: 2,
+      });
+      const jsonContent = jsonBuilder.build(jsonOutput);
+      const jsonBuildDuration = Date.now() - jsonBuildStartTime;
+
+      const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+      const fileName = `inventory-quantities-${timestamp}.json`;
+      const s3Key = `${s3Prefix}${fileName}`;
+
+      log.info('✅ [JSON] JSON file generated', {
+        fileName,
+        sizeBytes: jsonContent.length,
+        sizeMB: (jsonContent.length / (1024 * 1024)).toFixed(2),
+        recordCount: transformedRecords.length,
+        durationMs: jsonBuildDuration,
+        jobId: job.id,
+      });
+
+      const s3UploadStartTime = Date.now();
+      const s3 = new S3DataSource(
+        {
+          type: 'S3_JSON',
+          connectionId: 's3-inventory-quantities-json-export',
+          name: 'S3 Inventory Quantities JSON Export',
+          s3Config,
+        },
+        log
+      );
+
+      await s3.upload({
+        key: s3Key,
+        body: Buffer.from(jsonContent, 'utf8'),
+        contentType: 'application/json',
+        metadata: {
+          recordCount: String(transformedRecords.length),
+          extractedAt: new Date().toISOString(),
+          incrementalFrom: rawLastRunTime,
+          incrementalTo: newTimestamp,
+          jobId: job.id,
+        },
+      });
+      const s3UploadDuration = Date.now() - s3UploadStartTime;
+
+      log.info('✅ [S3] JSON file uploaded successfully', {
+        s3Key,
+        bucket: s3Config.bucket,
+        durationMs: s3UploadDuration,
+        jobId: job.id
+      });
+
+      // STEP 8/8: Update state and complete job
+      log.info('⚙️ [STEP 8/8] Updating state and completing job');
+      await kv.set(stateKey, {
+        timestamp: newTimestamp, // ← NO buffer applied
+        recordCount: transformedRecords.length,
+        extractedAt: new Date().toISOString(),
+        overlapBufferSeconds,
+        fileName,
+        s3Key,
+        jobId: job.id,
+        errors: errors.length > 0 ? errors : undefined,
+      });
+
+      const totalDuration = Date.now() - executionStartTime;
+      await tracker.markCompleted(job.id, {
+        recordsExtracted: transformedRecords.length,
+        recordsFailed: errors.length,
+        fileName,
+        s3Key,
+        newTimestamp,
+        durationMs: totalDuration,
+        performance: {
+          extractionMs: extractionDuration,
+          mappingMs: mappingDuration,
+          jsonBuildMs: jsonBuildDuration,
+          s3UploadMs: s3UploadDuration,
+          totalMs: totalDuration
+        }
+      });
+
+      // ========================================
+      // EXECUTION BOUNDARY: Workflow Success
+      // ========================================
+      log.info('✅ [WORKFLOW] Inventory Quantities Extraction - Completed Successfully', {
+        jobId: job.id,
+        recordsExtracted: transformedRecords.length,
+        recordsFailed: errors.length,
+        fileName,
+        s3Key,
+        durationMs: totalDuration,
+        performance: {
+          extractionMs: extractionDuration,
+          mappingMs: mappingDuration,
+          jsonBuildMs: jsonBuildDuration,
+          s3UploadMs: s3UploadDuration
+        }
+      });
+
+      return {
+        success: true,
+        jobId: job.id,
+        recordsExtracted: transformedRecords.length,
+        recordsFailed: errors.length,
+        fileName,
+        s3Key,
+        rawLastRunTime,
+        newTimestamp,
+        durationMs: totalDuration,
+        performance: {
+          extractionMs: extractionDuration,
+          mappingMs: mappingDuration,
+          jsonBuildMs: jsonBuildDuration,
+          s3UploadMs: s3UploadDuration,
+          totalMs: totalDuration
+        },
+        errors: errors.length > 0 ? errors : undefined,
+      };
+    } catch (error: any) {
+      const totalDuration = Date.now() - executionStartTime;
+      const errorMessage = error instanceof Error ? error.message : String(error);
+      const errorType = error instanceof Error ? error.constructor.name : 'UnknownError';
+
+      // ========================================
+      // EXECUTION BOUNDARY: Workflow Failure
+      // ========================================
+      log.error('❌ [WORKFLOW] Inventory Quantities Extraction - Failed', {
+        error: errorMessage,
+        errorType,
+        durationMs: totalDuration,
+        stack: error instanceof Error ? error.stack : undefined,
+      });
+
+      // Determine error recommendation based on error type
+      let recommendation = 'Check logs for details and verify configuration';
+      if (errorMessage.includes('authentication') || errorMessage.includes('credentials')) {
+        recommendation = 'Verify Fluent Commerce and S3 credentials in Activation Variables';
+      } else if (errorMessage.includes('network') || errorMessage.includes('timeout')) {
+        recommendation = 'Check network connectivity and increase timeout settings if needed';
+      } else if (errorMessage.includes('GraphQL') || errorMessage.includes('query')) {
+        recommendation = 'Verify GraphQL query syntax and ensure retailerId has access to inventory data';
+      } else if (errorMessage.includes('S3') || errorMessage.includes('upload')) {
+        recommendation = 'Verify S3 bucket exists, credentials are valid, and bucket permissions allow uploads';
+      } else if (errorMessage.includes('mapping') || errorMessage.includes('transform')) {
+        recommendation = 'Check mapping configuration in config/inventory-quantities.export.json';
+      }
+
+      return {
+        success: false,
+        error: errorMessage,
+        errorType,
+        durationMs: totalDuration,
+        stack: error instanceof Error ? error.stack : undefined,
+        recommendation
+      };
+    }
+  }));
+```
+
+### Workflow 2: Ad Hoc Extraction (HTTP Endpoint)
+
+**File:** `src/workflows/webhook/adhoc-inventory-quantities-extraction.ts`
+
+```typescript
+export const inventoryQuantitiesAdHocExtraction = webhook('inventory-quantities-adhoc-extract', {
+  connection: 'inventory-quantities-adhoc',
+}).then(http('execute-adhoc-extraction', { connection: 'fluent_commerce', validateConnection: true }, async ctx => {
+    const { log, openKv, activation } = ctx;
+    const executionStartTime = Date.now();
+
+    log.info('🚀 [WEBHOOK] Ad Hoc Extraction - Started', { trigger: 'webhook' });
+
+    const { startDate, endDate, retailerId } = ctx.request.body;
+
+    if (!startDate || !endDate || !retailerId) {
+      log.error('❌ [VALIDATION] Missing required webhook payload fields', {
+        hasStartDate: !!startDate,
+        hasEndDate: !!endDate,
+        hasRetailerId: !!retailerId
+      });
+      return {
+        success: false,
+        error: 'Missing required fields: startDate, endDate, retailerId',
+        recommendation: 'Provide all required fields in the webhook payload: { startDate, endDate, retailerId }'
+      };
+    }
+
+    try {
+      const kv = new VersoriKVAdapter(openKv(':project:'));
+      const tracker = new JobTracker(kv, log);
+
+      // Create ad hoc job
+      log.info('⚙️ Creating ad hoc extraction job');
+      const job = await tracker.createJob({
+        type: 'extraction',
+        entity: 'inventoryQuantities',
+        config: {
+          extractionMode: 'dateRange',
+          retailerId,
+          startDate,
+          endDate,
+        },
+      });
+
+      log.info('✅ [JOB] Ad hoc extraction job created', { jobId: job.id, startDate, endDate });
+
+      // Execute extraction (similar to scheduled workflow)
+      // ... (extraction logic here - implement full extraction as in scheduled workflow)
+
+      const totalDuration = Date.now() - executionStartTime;
+      log.info('✅ [WEBHOOK] Ad Hoc Extraction - Completed', {
+        jobId: job.id,
+        durationMs: totalDuration
+      });
+
+      return {
+        success: true,
+        jobId: job.id,
+        message: 'Ad hoc extraction started',
+        durationMs: totalDuration
+      };
+    } catch (error: any) {
+      const totalDuration = Date.now() - executionStartTime;
+      const errorMessage = error instanceof Error ? error.message : String(error);
+
+      log.error('❌ [WEBHOOK] Ad Hoc Extraction - Failed', {
+        error: errorMessage,
+        durationMs: totalDuration
+      });
+
+      return {
+        success: false,
+        error: errorMessage,
+        durationMs: totalDuration,
+        recommendation: 'Check logs for details and verify webhook payload format'
+      };
+    }
+  }
+);
+```
+
+### Workflow 3: Job Status Lookup
+
+**File:** `src/workflows/webhook/job-status-check.ts`
+
+```typescript
+export const inventoryQuantitiesJobStatus = webhook('inventory-quantities-job-status', {
+  connection: 'inventory-quantities-job-status',
+}).then(
+  http('query-job-status', { validateConnection: true }, async ctx => {
+    const { log, openKv } = ctx;
+    const executionStartTime = Date.now();
+
+    log.info('🚀 [WEBHOOK] Job Status Query - Started', { trigger: 'webhook' });
+
+    const { jobId } = ctx.request.body;
+
+    if (!jobId) {
+      log.error('❌ [VALIDATION] Missing required field: jobId');
+      return {
+        success: false,
+        error: 'Missing required field: jobId',
+        recommendation: 'Provide jobId in webhook payload: { jobId: "your-job-id" }'
+      };
+    }
+
+    try {
+      log.info('⚙️ Querying job status', { jobId });
+      const kv = new VersoriKVAdapter(openKv(':project:'));
+      const tracker = new JobTracker(kv, log);
+
+      const job = await tracker.getJob(jobId);
+
+      if (!job) {
+        log.warn('⚠️ Job not found', { jobId });
+        return {
+          success: false,
+          error: `Job not found: ${jobId}`,
+          recommendation: 'Verify the jobId is correct and the job was created successfully'
+        };
+      }
+
+      const totalDuration = Date.now() - executionStartTime;
+      log.info('✅ [WEBHOOK] Job Status Query - Completed', {
+        jobId,
+        status: job.status,
+        durationMs: totalDuration
+      });
+
+      return {
+        success: true,
+        job: {
+          id: job.id,
+          type: job.type,
+          entity: job.entity,
+        status: job.status,
+        createdAt: job.createdAt,
+        completedAt: job.completedAt,
+        result: job.result,
+        error: job.error,
+      },
+        durationMs: totalDuration
+      };
+    } catch (error: any) {
+      const totalDuration = Date.now() - executionStartTime;
+      const errorMessage = error instanceof Error ? error.message : String(error);
+
+      log.error('❌ [WEBHOOK] Job Status Query - Failed', {
+        error: errorMessage,
+        durationMs: totalDuration
+      });
+
+      return {
+        success: false,
+        error: errorMessage,
+        durationMs: totalDuration,
+        recommendation: 'Check logs for details and verify KV storage is accessible'
+      };
+    }
+  })
+);
+```
+
+---
+
+## Key Differences from CSV
+
+1. **JSON Structure with Metadata:**
+
+   ```json
+   {
+     "metadata": { ... },  // ← Extraction context
+     "data": [ ... ]        // ← Actual records
+   }
+   ```
+
+2. **Pretty Printing Option:**
+   - Set `prettyPrint: true` for human-readable JSON
+   - Set `prettyPrint: false` for compact/production JSON
+
+3. **Content Type:**
+   - CSV: `text/csv`
+   - JSON: `application/json`
+
+4. **Field Names:**
+   - CSV: `location_code` (snake_case for compatibility)
+   - JSON: `location` (camelCase for APIs)
+
+---
+
+## Use Cases
+
+**1. REST API Polling:**
+
+```bash
+# External system polls S3 for latest file
+aws s3 ls s3://api-inventory-exports/inventory/quantities/ --recursive | sort | tail -n 1
+```
+
+**2. S3 Event Trigger:**
+
+```typescript
+// Lambda triggered on S3 PUT event
+export const handler = async (event: S3Event) => {
+  const key = event.Records[0].s3.object.key;
+  const data = await s3.getObject({ Bucket: bucket, Key: key });
+  const json = JSON.parse(data.Body.toString());
+  await processInventoryQuantities(json.data);
+};
+```
+
+**3. API Gateway Integration:**
+
+```typescript
+// Return latest extraction file via API
+// GET /api/inventory/quantities/latest
+export async function handler(event: APIGatewayEvent) {
+  const latestFile = await getLatestS3File('inventory/quantities/');
+  const data = await s3.getObject({ Key: latestFile });
+  return {
+    statusCode: 200,
+    headers: { 'Content-Type': 'application/json' },
+    body: data.Body.toString(),
+  };
+}
+```
+
+---
+
+## Production Checklist
+
+- [ ] Set appropriate extraction frequency (15min, hourly, daily)
+- [ ] Configure `maxRecords` based on expected change volume
+- [ ] Enable/disable pretty printing based on use case
+- [ ] Set up S3 lifecycle policy to archive old files
+- [ ] Document JSON schema for API consumers
+- [ ] Test with real-time incremental changes
+- [ ] Verify error handling for partial failures
+- [ ] Set up CloudFront CDN if serving via HTTP
+- [ ] Configure CORS if accessed from browser
+- [ ] Set up job tracking and monitoring
+- [ ] Test ad hoc extraction workflow
+- [ ] Validate job status lookup
+
+---
+
+**Pattern**: Incremental extraction with overlap buffer and JobTracker integration
+**Key Learning**: Include metadata for API consumers, use 3-workflow pattern
+**Use Case**: Real-time inventory quantity sync for external APIs with job tracking
+**Buffer Pattern**: Query WITH buffer (`updatedOn >= lastRunTime - 60s`), save WITHOUT buffer (`MAX(updatedOn)`)
+
+---
+
+### Pattern 8: Backward Pagination (Optional - Advanced)
+
+**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
+
+**When to Use**:
+
+- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
+- ✅ Time-bounded reverse traversal for auditing
+- ✅ Display newest-first in UI/reports
+- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
+
+**GraphQL Query Requirements**:
+
+Your query must support backward pagination by including `$last` and `$before`:
+
+```graphql
+query GetData(
+  $retailerId: ID!
+  $first: Int # For forward pagination
+  $after: String # For forward pagination
+  $last: Int # For backward pagination
+  $before: String # For backward pagination
+) {
+  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
+    edges {
+      cursor # ✅ REQUIRED
+      node {
+        id
+        createdAt
+        # ... other fields
+      }
+    }
+    pageInfo {
+      hasNextPage # For forward
+      hasPreviousPage # ✅ REQUIRED for backward
+    }
+  }
+}
+```
+
+**Implementation**:
+
+```typescript
+// Backward pagination - newest records first
+const result = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    retailerId,
+    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
+    // ❌ Don't include last/before - orchestrator injects them
+  },
+  pageSize: 200,
+  direction: 'backward', // ✅ Enable reverse pagination
+  maxRecords: 10000,
+});
+
+// Records are returned in reverse chronological order
+log.info('Newest record', { createdAt: result.data[0].createdAt });
+log.info('Oldest record', { createdAt: result.data[result.data.length - 1].createdAt });
+```
+
+**Key Differences from Forward Pagination**:
+
+| Aspect                 | Forward (Default)                | Backward                |
+| ---------------------- | -------------------------------- | ----------------------- |
+| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
+| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
+| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
+| **Cursor Source**      | Last edge of page                | First edge of page      |
+| **Record Order**       | Oldest → Newest                  | Newest → Oldest         |
+
+**Important Notes**:
+
+1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
+
+2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
+
+3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
+
+4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
+
+**Example: Extract Latest 1000 Orders**
+
+```typescript
+const latestOrders = await orchestrator.extract({
+  query: ORDERS_QUERY,
+  resultPath: 'orders.edges.node',
+  variables: {
+    retailerId,
+    statuses: ['BOOKED', 'ALLOCATED'],
+  },
+  direction: 'backward', // Start from newest
+  maxRecords: 1000, // Stop after 1000 records
+  pageSize: 100, // 100 per page = 10 pages
+});
+
+// latestOrders.data[0] is the newest order
+// latestOrders.data[999] is the 1000th newest order
+```
+
+**When to Use Forward vs Backward**:
+
+```typescript
+// ✅ Forward (default) - For incremental sync
+const incrementalData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    dateRangeFilter: { from: lastSyncTime, to: now },
+  },
+  // direction defaults to 'forward'
+  // Processes oldest → newest for proper sequencing
+});
+
+// ✅ Backward - For "latest N records" use cases
+const latestData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  direction: 'backward',
+  maxRecords: 100, // Just get latest 100
+  // Gets newest → oldest
+});
+```
+
+**Pagination Variables Reference**:
+
+| Variable | Forward     | Backward    | Injected By  | Notes                    |
+| -------- | ----------- | ----------- | ------------ | ------------------------ |
+| `first`  | ✅ Used     | ❌ Not used | Orchestrator | From `pageSize`          |
+| `after`  | ✅ Used     | ❌ Not used | Orchestrator | From cursor (last edge)  |
+| `last`   | ❌ Not used | ✅ Used     | Orchestrator | From `pageSize`          |
+| `before` | ❌ Not used | ✅ Used     | Orchestrator | From cursor (first edge) |
+
+**Common Mistakes to Avoid**:
+
+```typescript
+// ❌ WRONG - Don't pass pagination variables
+const result = await orchestrator.extract({
+  variables: {
+    last: 200, // ❌ Orchestrator will override this
+    before: cursor, // ❌ Orchestrator manages cursor
+  },
+  direction: 'backward',
+});
+
+// ✅ CORRECT - Let orchestrator inject pagination
+const result = await orchestrator.extract({
+  variables: {
+    retailerId, // ✅ Your business variables only
+  },
+  pageSize: 200, // ✅ Orchestrator uses this for last/before
+  direction: 'backward',
+});
+```
+
+#### Optional: Reverse Pagination
+
+- Forward remains default. Reverse requires $last/$before and pageInfo.hasPreviousPage.
+
+GraphQL:
+
+```graphql
+query GetInventoryQuantitiesBackward($retailerId: ID!, $last: Int!, $before: String) {
+  inventoryQuantities(retailerId: $retailerId, last: $last, before: $before) {
+    edges {
+      cursor
+      node {
+        id
+        ref
+        updatedOn
+      }
+    }
+    pageInfo {
+      hasPreviousPage
+    }
+  }
+}
+```
+
+SDK:
+
+```typescript
+await orchestrator.extract({
+  query: INVENTORY_QUANTITIES_BACKWARD_QUERY,
+  resultPath: 'inventoryQuantities.edges.node',
+  variables: { retailerId },
+  pageSize,
+  direction: 'backward',
+});
+```
+
+---
+
+## Testing Checklist
+
+**Before production deployment:**
+
+### 1. Schema Validation
+
+- [ ] Run `npx fc-connect introspect-schema --url <your-graphql-url>`
+- [ ] Run `npx fc-connect validate-schema --mapping ./config/inventory-quantities.export.json --schema ./fluent-schema.json`
+- [ ] Run `npx fc-connect analyze-coverage --mapping ./config/inventory-quantities.export.json --schema ./fluent-schema.json`
+- [ ] Verify all `source` paths in mapping exist in GraphQL schema
+- [ ] Verify query structure matches schema (fields, types, filters)
+
+### 2. Extraction Testing
+
+- [ ] Test with small dataset first (maxRecords=10)
+- [ ] Verify ExtractionOrchestrator pagination works correctly
+- [ ] Test with multiple pages of data (verify cursor handling)
+- [ ] Verify date range filtering (updatedOn filter)
+- [ ] Test empty result handling (no records in date range)
+- [ ] Verify extraction stops at maxRecords limit
+
+### 3. Mapping Testing
+
+- [ ] Verify required fields are populated
+- [ ] Verify SDK resolvers work correctly (sdk.trim, sdk.parseInt, sdk.formatDate, etc.)
+- [ ] Test custom resolvers with edge cases (if any)
+- [ ] Verify nested field extraction
+- [ ] Test with null/missing fields
+- [ ] Verify mapping error collection works
+
+### 4. JSON Generation Testing
+
+- [ ] Verify JSON structure matches expected format
+- [ ] Test JSON validation against schema (if applicable)
+- [ ] Verify proper nesting and structure
+- [ ] Test with large datasets (>1000 records)
+- [ ] Verify UTF-8 encoding
+- [ ] Test special character escaping
+
+### 5. S3 Upload Testing
+
+- [ ] Test S3 connection and authentication
+- [ ] Verify file upload to correct bucket and path
+- [ ] Test file naming convention (timestamp format)
+- [ ] Verify S3 object metadata
+- [ ] Test upload retry logic (simulate network failure)
+- [ ] Verify file permissions and ACLs
+
+### 6. State Management Testing
+
+- [ ] Verify overlap buffer prevents missed records (60-second default)
+- [ ] Test state recovery after extraction failure
+- [ ] Verify timestamp saved WITHOUT buffer (MAX(updatedOn))
+- [ ] Test first run with no previous state (uses fallbackStartDate)
+- [ ] Verify state update only happens on successful upload
+- [ ] Test manual date override (doesn't update state)
+
+### 7. Job Tracking Testing
+
+- [ ] Test job creation with JobTracker
+- [ ] Verify job status updates at each stage
+- [ ] Test job completion with metadata
+- [ ] Test job failure handling
+- [ ] Query job status via webhook endpoint
+- [ ] Verify job status persists in KV store
+
+### 8. Error Handling Testing
+
+- [ ] Test with invalid GraphQL query
+- [ ] Test with mapping errors (invalid field paths)
+- [ ] Test with S3 connection failures
+- [ ] Test with authentication failures
+- [ ] Test with network timeouts
+- [ ] Verify error logging includes context (jobId, stage, error details)
+- [ ] Test error threshold logic (if applicable)
+
+### 9. Staging Environment Testing
+
+- [ ] Run full extraction in staging environment
+- [ ] Verify JSON file format with downstream system
+- [ ] Monitor extraction duration and resource usage
+- [ ] Test with production-like data volumes
+- [ ] Verify no performance degradation over time
+
+### 10. Integration Testing
+
+- [ ] Test scheduled workflow (cron trigger)
+- [ ] Test ad hoc webhook trigger
+- [ ] Test job status query webhook
+- [ ] Verify activation variables are read correctly
+- [ ] Test with different extraction modes (incremental, date range)
+- [ ] End-to-end test: trigger → extract → transform → upload → verify file
+
+---
+## Monitoring & Alerting
+
+### Success Response Example
+
+```json
+{
+  "success": true,
+  "jobId": "SCHEDULED_IQ_20251102_140000_abc123",
+  "recordsExtracted": 1523,
+  "fileName": "inventory-quantities-2025-11-02T14-00-00-000Z.json",
+  "s3Path": "s3://bucket/inventory-quantities/inventory-quantities-2025-11-02T14-00-00-000Z.json",
+  "metrics": {
+    "extractionDurationMs": 12543,
+    "totalPages": 8,
+    "pageSize": 200,
+    "mappingErrors": 0,
+    "fileSizeBytes": 524288,
+    "uploadDurationMs": 1234
+  },
+  "timestamps": {
+    "extractionStart": "2025-11-02T14:00:00.000Z",
+    "extractionEnd": "2025-11-02T14:00:12.543Z",
+    "uploadComplete": "2025-11-02T14:00:13.777Z"
+  },
+  "state": {
+    "previousTimestamp": "2025-11-02T13:00:00.000Z",
+    "newTimestamp": "2025-11-02T13:59:58.123Z",
+    "stateUpdated": true,
+    "overlapBufferSeconds": 60
+  }
+}
+```
+
+### Error Response Example
+
+```json
+{
+  "success": false,
+  "jobId": "ADHOC_IQ_20251102_140500_xyz789",
+  "error": "S3 upload failed: Connection timeout",
+  "errorCategory": "NETWORK",
+  "recordsExtracted": 0,
+  "stage": "s3_upload",
+  "details": {
+    "message": "Failed to upload file after 3 retry attempts",
+    "retryAttempts": 3,
+    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
+  },
+  "state": {
+    "stateUpdated": false,
+    "willRetryNextRun": true,
+    "note": "State not advanced - next extraction will retry same time window"
+  }
+}
+```
+
+### Key Metrics to Track
+
+```typescript
+const METRICS = {
+  // Extraction Performance
+  extractionDurationMs: Date.now() - extractionStart,
+  recordCount: records.length,
+  pageCount: extractionResult.stats.totalPages,
+  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
+
+  // Transformation Performance
+  transformedCount: transformedRecords.length,
+  failedCount: mappingErrors.length,
+  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
+
+  // File Generation
+  fileSizeMB: (jsonContent.length / (1024 * 1024)).toFixed(2),
+
+  // Upload Performance
+  uploadDurationMs: uploadEnd - uploadStart,
+  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
+
+  // State Management
+  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
+  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
+};
+
+log.info('Extraction metrics', metrics);
+```
+
+### Alert Thresholds
+
+```typescript
+const ALERT_THRESHOLDS = {
+  // Duration Alerts
+  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
+  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
+  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
+
+  // Error Rate Alerts
+  MAX_ERROR_RATE: 0.05, // 5% mapping errors
+  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
+
+  // Volume Alerts
+  MAX_RECORDS_PER_RUN: 100000,
+  MIN_RECORDS_WARNING: 0, // Alert if no records found
+  MAX_FILE_SIZE_MB: 150, // 150MB
+
+  // State Alerts
+  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
+  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
+};
+
+// Check thresholds
+if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
+  log.warn('Extraction duration exceeded threshold', {
+    duration: metrics.extractionDurationMs,
+    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
+    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
+  });
+}
+```
+
+### Monitoring Dashboard Queries
+
+**Versori Platform Logs Query:**
+
+```
+# Successful extractions
+log_level:info AND message:"Extraction complete" AND jobId:*
+
+# Failed extractions
+log_level:error AND message:"Extraction workflow failed" AND jobId:*
+
+# Performance issues
+extractionDurationMs:>300000 OR uploadDurationMs:>120000
+
+# High error rates
+errorRate:>5
+
+# State management issues
+stateUpdated:false AND success:true
+```
+
+### Common Issues and Solutions
+
+**Issue**: "Extraction timeout after 10 minutes"
+
+- **Cause**: Too many records in single extraction
+- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
+- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
+
+**Issue**: "Mapping errors for 50% of records"
+
+- **Cause**: Schema mismatch between GraphQL response and mapping config
+- **Fix**: Run schema validation, update mapping config paths
+- **Prevention**: Use `npx fc-connect validate-schema` before deployment
+
+**Issue**: "S3 connection timeout"
+
+- **Cause**: Network issues, firewall, or connection pool exhaustion
+- **Fix**: Check S3 credentials, verify network connectivity
+- **Prevention**: Implement connection health checks, monitor connection status
+
+**Issue**: "State not updating after successful extraction"
+
+- **Cause**: KV write failure or intentional retry logic
+- **Fix**: Check KV logs, verify state update code executed
+- **Prevention**: Add KV write verification, log state updates explicitly
+
+**Issue**: "First run exceeds record limits"
+
+- **Cause**: No previous timestamp, fetches all historical records
+- **Fix**: Set fallbackStartDate close to current date, apply additional filters
+- **Prevention**: Use appropriate fallbackStartDate for initial runs
+
+**Issue**: "Excessive duplicate records in output"
+
+- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
+- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
+- **Prevention**: Monitor duplicate rates, verify state update logic
+
+---
+
+## Troubleshooting Quick Reference
+
+| Error Message | Likely Cause | Solution |
+|--------------|--------------|----------|
+| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
+| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
+| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
+| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
+| "S3 authentication failed" | Invalid credentials | Verify S3 credentials in activation variables |
+| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
+| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
+| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
+| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
+| "JSON generation failed" | Format-specific error | Check JSON generation logic, validate output |
+
+---