@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/02-CORE-GUIDES/extraction/readme.md

@@ -1,151 +1,151 @@
-# Data Extraction Learning Path
-
-> **Complete guide to extracting and transforming data from Fluent Commerce**
-> Progressive learning modules with real schema examples and production-ready patterns
-
-## Quick Start
-
-```typescript
-import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
-
-const client = await createClient({ config: {/*...*/} });
-const orchestrator = new ExtractionOrchestrator(client, logger);
-
-const result = await orchestrator.extract({
-  query: `
-    query ExtractInventory($first: Int!, $after: String) {
-      inventoryPositions(first: $first, after: $after) {
-        edges {
-          node { id ref productRef locationRef qty }
-          cursor
-        }
-        pageInfo { hasNextPage }
-      }
-    }
-  `,
-  resultPath: 'inventoryPositions.edges.node',
-  pageSize: 200
-});
-```
-
-## 📚 What You'll Learn
-
-This guide teaches you how to extract data from Fluent Commerce and transform it for external systems using the FC Connect SDK. By the end, you'll understand:
-
-- ✅ How to extract data from Fluent Commerce using validated GraphQL queries
-- ✅ How to process Parquet files from S3 with the SDK's ParquetParserService
-- ✅ How to enrich data with complex GraphQL operations
-- ✅ How to transform data using UniversalMapper and custom transformers
-- ✅ How to optimize extraction for performance and reliability
-- ✅ How to export data to multiple formats (JSON, XML, Parquet, JSONL)
-
-## 🎯 Learning Path Overview
-
-This guide is organized into **8 progressive modules**. Follow them in order for the best learning experience:
-
-| Module | Topic | Level |
-|--------|-------|-------|
-| [Module 1](../auto-pagination/modules/auto-pagination-01-foundations.md) | **Foundations** - Extraction concepts, workflow overview | Beginner |
-| [Module 2](./modules/02-core-guides-extraction-02-basic-extraction.md) | **Basic Extraction** - Simple GraphQL queries with pagination | Beginner |
-| [Module 3](./modules/02-core-guides-extraction-03-parquet-processing.md) | **Parquet Processing** - Reading and writing Parquet files | Intermediate |
-| [Module 4](./modules/02-core-guides-extraction-04-data-enrichment.md) | **Data Enrichment** - GraphQL operations, batch enrichment | Intermediate |
-| [Module 5](./modules/02-core-guides-extraction-05-transformation.md) | **Data Transformation** - Field mapping, custom transformers | Advanced |
-| [Module 6](./modules/02-core-guides-extraction-06-export-formats.md) | **Export Formats** - Output to JSON, XML, Parquet, S3, SFTP | Advanced |
-| [Module 7](./modules/02-core-guides-extraction-07-optimization.md) | **Optimization & Best Practices** - Performance, caching, monitoring | Advanced |
-| [Module 8](./modules/02-core-guides-extraction-08-extraction-orchestrator.md) | **ExtractionOrchestrator** - Simplified extraction with high-level service | Intermediate |
-
-## 🚀 Quick Navigation
-
-### By Use Case
-
-- **First time extracting data?** → Start with [Module 1: Foundations](../auto-pagination/modules/auto-pagination-01-foundations.md)
-- **Need the simplest approach?** → Use [Module 8: ExtractionOrchestrator](./modules/02-core-guides-extraction-08-extraction-orchestrator.md)
-- **Need a working example now?** → Jump to [Module 2: Basic Extraction](./modules/02-core-guides-extraction-02-basic-extraction.md)
-- **Processing Parquet files?** → See [Module 3: Parquet Processing](./modules/02-core-guides-extraction-03-parquet-processing.md)
-- **Need data enrichment?** → Check [Module 4: Data Enrichment](./modules/02-core-guides-extraction-04-data-enrichment.md)
-- **Want to transform data?** → See [Module 5: Data Transformation](./modules/02-core-guides-extraction-05-transformation.md)
-- **Experiencing issues?** → Browse [Module 7: Optimization](./modules/02-core-guides-extraction-07-optimization.md)
-- **Want code examples?** → Browse [Examples Directory](./examples/)
-
-### By Entity Type
-
-Different Fluent entities require different extraction strategies:
-
-| Entity Type | Complexity | Fields | Example |
-|-------------|------------|--------|---------|
-| `inventoryPositions` | Simple | Basic inventory fields | [Example](./examples/extract-inventory-basic.ts) |
-| `virtualPositions` | Simple | Virtual inventory | [Example](./examples/common-patterns.ts) (see virtual positions pattern) |
-| `products` (with relations) | Medium | Product with catalogue | [Example](./examples/common-patterns.ts) (see products pattern) |
-| `orders` (with items) | Complex | Orders with nested items | [Example](./examples/common-patterns.ts) (see orders pattern) |
-| Multi-entity enrichment | Complex | Multiple related entities | [Example](./examples/common-patterns.ts) (see multi-entity pattern) |
-
-> **⚠️ CRITICAL: VirtualPosition vs InventoryPosition Field Differences**
->
-> These two inventory entity types have **different field names**:
->
-> | Field | VirtualPosition | InventoryPosition |
-> |-------|----------------|-------------------|
-> | Quantity | `quantity` (Int!) | `qty` (Int!), `onHand` (Int!) |
-> | Reference | `ref` (String!) | `ref` (String!) |
-> | Product | `productRef` (String!) | `productRef` (String!) |
-> | Location | `groupRef` (String) | `locationRef` (String!) |
->
-> **Always verify field names in the GraphQL schema before extraction!**
-
-### By Output Format
-
-| Format | Use Case | Complexity | Example |
-|--------|----------|------------|---------|
-| JSON | REST APIs, web applications | Simple | [Example](./examples/common-patterns.ts) (see JSON export pattern) |
-| JSONL | Streaming, big data tools | Simple | [Example](./examples/common-patterns.ts) (see JSONL export pattern) |
-| CSV | Spreadsheets, BI tools | Medium | [Example](./examples/common-patterns.ts) (see CSV export pattern) |
-| Parquet | Data lakes, analytics | Medium | [Example](./examples/common-patterns.ts) (see Parquet export pattern) |
-| XML | Enterprise systems | Medium | [Example](./examples/common-patterns.ts) (see XML export pattern) |
-
-## 📖 Quick Reference
-
-For quick lookups, see:
-- [Quick Reference Cheat Sheet](./extraction-quick-reference.md) - All patterns on one page
-- [Common Patterns](./examples/common-patterns.ts) - Copy-paste ready snippets
-- [Schema Validation Guide](../auto-pagination/modules/auto-pagination-01-foundations.md#schema-validation-critical-first-step) - Verify queries before use
-
-## 🎓 Prerequisites
-
-Before starting this guide, you should:
-
-1. ✅ Have the SDK installed: `npm install @fluentcommerce/fc-connect-sdk`
-2. ✅ Have Fluent API credentials (clientId, clientSecret, retailerId)
-3. ✅ Understand basic GraphQL concepts (queries, variables, types)
-4. ✅ Be familiar with cursor-based pagination (see [Auto-Pagination Guide](../auto-pagination/auto-pagination-readme.md))
-
-**New to the SDK?** Start with the [Getting Started Guide](../../00-START-HERE/getting-started.md) first.
-
-## 🔗 Related Documentation
-
-- [Auto-Pagination Guide](../auto-pagination/auto-pagination-readme.md) - Automatic pagination for large datasets
-- [Universal Mapping Guide](../mapping/readme.md) - Transform extracted data
-- [GraphQL Mutation Mapping](../mapping/graphql-mutation-mapping/graphql-mutation-mapping-readme.md) - Create mutations from data
-- [Resolvers Guide](../mapping/resolvers/mapping-resolvers-readme.md) - Custom transformation resolvers
-- [SDK Architecture](../../04-REFERENCE/architecture/) - Understanding SDK design
-
-## 💡 Tips for Best Results
-
-1. **Always validate queries** - Use schema introspection to verify field names
-2. **Start with small datasets** - Test with `first: 10` before extracting thousands
-3. **Use auto-pagination** - Let the SDK handle cursor management
-4. **Monitor memory usage** - Use streaming for large datasets
-5. **Implement error recovery** - Add retry logic for network issues
-6. **Cache frequently accessed data** - Reduce API calls
-7. **Test transformations** - Validate field mappings with sample data
-
-## 🚦 Getting Started
-
-Ready to begin? Start with [Module 1: Foundations →](../auto-pagination/modules/auto-pagination-01-foundations.md)
-
----
-
-**Need Help?**
-- 📖 Browse [Examples](./examples/)
-- 🔍 Check [Module 7: Optimization](./modules/02-core-guides-extraction-07-optimization.md)
-- 📧 Contact support@fluentcommerce.com
-- 🐛 Report issues at [GitHub](https://github.com/fluentcommerce/fc-connect-sdk/issues)
+# Data Extraction Learning Path
+
+> **Complete guide to extracting and transforming data from Fluent Commerce**
+> Progressive learning modules with real schema examples and production-ready patterns
+
+## Quick Start
+
+```typescript
+import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
+
+const client = await createClient({ config: {/*...*/} });
+const orchestrator = new ExtractionOrchestrator(client, logger);
+
+const result = await orchestrator.extract({
+  query: `
+    query ExtractInventory($first: Int!, $after: String) {
+      inventoryPositions(first: $first, after: $after) {
+        edges {
+          node { id ref productRef locationRef qty }
+          cursor
+        }
+        pageInfo { hasNextPage }
+      }
+    }
+  `,
+  resultPath: 'inventoryPositions.edges.node',
+  pageSize: 200
+});
+```
+
+## 📚 What You'll Learn
+
+This guide teaches you how to extract data from Fluent Commerce and transform it for external systems using the FC Connect SDK. By the end, you'll understand:
+
+- ✅ How to extract data from Fluent Commerce using validated GraphQL queries
+- ✅ How to process Parquet files from S3 with the SDK's ParquetParserService
+- ✅ How to enrich data with complex GraphQL operations
+- ✅ How to transform data using UniversalMapper and custom transformers
+- ✅ How to optimize extraction for performance and reliability
+- ✅ How to export data to multiple formats (JSON, XML, Parquet, JSONL)
+
+## 🎯 Learning Path Overview
+
+This guide is organized into **8 progressive modules**. Follow them in order for the best learning experience:
+
+| Module | Topic | Level |
+|--------|-------|-------|
+| [Module 1](../auto-pagination/modules/auto-pagination-01-foundations.md) | **Foundations** - Extraction concepts, workflow overview | Beginner |
+| [Module 2](./modules/02-core-guides-extraction-02-basic-extraction.md) | **Basic Extraction** - Simple GraphQL queries with pagination | Beginner |
+| [Module 3](./modules/02-core-guides-extraction-03-parquet-processing.md) | **Parquet Processing** - Reading and writing Parquet files | Intermediate |
+| [Module 4](./modules/02-core-guides-extraction-04-data-enrichment.md) | **Data Enrichment** - GraphQL operations, batch enrichment | Intermediate |
+| [Module 5](./modules/02-core-guides-extraction-05-transformation.md) | **Data Transformation** - Field mapping, custom transformers | Advanced |
+| [Module 6](./modules/02-core-guides-extraction-06-export-formats.md) | **Export Formats** - Output to JSON, XML, Parquet, S3, SFTP | Advanced |
+| [Module 7](./modules/02-core-guides-extraction-07-optimization.md) | **Optimization & Best Practices** - Performance, caching, monitoring | Advanced |
+| [Module 8](./modules/02-core-guides-extraction-08-extraction-orchestrator.md) | **ExtractionOrchestrator** - Simplified extraction with high-level service | Intermediate |
+
+## 🚀 Quick Navigation
+
+### By Use Case
+
+- **First time extracting data?** → Start with [Module 1: Foundations](../auto-pagination/modules/auto-pagination-01-foundations.md)
+- **Need the simplest approach?** → Use [Module 8: ExtractionOrchestrator](./modules/02-core-guides-extraction-08-extraction-orchestrator.md)
+- **Need a working example now?** → Jump to [Module 2: Basic Extraction](./modules/02-core-guides-extraction-02-basic-extraction.md)
+- **Processing Parquet files?** → See [Module 3: Parquet Processing](./modules/02-core-guides-extraction-03-parquet-processing.md)
+- **Need data enrichment?** → Check [Module 4: Data Enrichment](./modules/02-core-guides-extraction-04-data-enrichment.md)
+- **Want to transform data?** → See [Module 5: Data Transformation](./modules/02-core-guides-extraction-05-transformation.md)
+- **Experiencing issues?** → Browse [Module 7: Optimization](./modules/02-core-guides-extraction-07-optimization.md)
+- **Want code examples?** → Browse [Examples Directory](./examples/)
+
+### By Entity Type
+
+Different Fluent entities require different extraction strategies:
+
+| Entity Type | Complexity | Fields | Example |
+|-------------|------------|--------|---------|
+| `inventoryPositions` | Simple | Basic inventory fields | [Example](./examples/extract-inventory-basic.ts) |
+| `virtualPositions` | Simple | Virtual inventory | [Example](./examples/common-patterns.ts) (see virtual positions pattern) |
+| `products` (with relations) | Medium | Product with catalogue | [Example](./examples/common-patterns.ts) (see products pattern) |
+| `orders` (with items) | Complex | Orders with nested items | [Example](./examples/common-patterns.ts) (see orders pattern) |
+| Multi-entity enrichment | Complex | Multiple related entities | [Example](./examples/common-patterns.ts) (see multi-entity pattern) |
+
+> **⚠️ CRITICAL: VirtualPosition vs InventoryPosition Field Differences**
+>
+> These two inventory entity types have **different field names**:
+>
+> | Field | VirtualPosition | InventoryPosition |
+> |-------|----------------|-------------------|
+> | Quantity | `quantity` (Int!) | `qty` (Int!), `onHand` (Int!) |
+> | Reference | `ref` (String!) | `ref` (String!) |
+> | Product | `productRef` (String!) | `productRef` (String!) |
+> | Location | `groupRef` (String) | `locationRef` (String!) |
+>
+> **Always verify field names in the GraphQL schema before extraction!**
+
+### By Output Format
+
+| Format | Use Case | Complexity | Example |
+|--------|----------|------------|---------|
+| JSON | REST APIs, web applications | Simple | [Example](./examples/common-patterns.ts) (see JSON export pattern) |
+| JSONL | Streaming, big data tools | Simple | [Example](./examples/common-patterns.ts) (see JSONL export pattern) |
+| CSV | Spreadsheets, BI tools | Medium | [Example](./examples/common-patterns.ts) (see CSV export pattern) |
+| Parquet | Data lakes, analytics | Medium | [Example](./examples/common-patterns.ts) (see Parquet export pattern) |
+| XML | Enterprise systems | Medium | [Example](./examples/common-patterns.ts) (see XML export pattern) |
+
+## 📖 Quick Reference
+
+For quick lookups, see:
+- [Quick Reference Cheat Sheet](./extraction-quick-reference.md) - All patterns on one page
+- [Common Patterns](./examples/common-patterns.ts) - Copy-paste ready snippets
+- [Schema Validation Guide](../auto-pagination/modules/auto-pagination-01-foundations.md#schema-validation-critical-first-step) - Verify queries before use
+
+## 🎓 Prerequisites
+
+Before starting this guide, you should:
+
+1. ✅ Have the SDK installed: `npm install @fluentcommerce/fc-connect-sdk`
+2. ✅ Have Fluent API credentials (clientId, clientSecret, retailerId)
+3. ✅ Understand basic GraphQL concepts (queries, variables, types)
+4. ✅ Be familiar with cursor-based pagination (see [Auto-Pagination Guide](../auto-pagination/auto-pagination-readme.md))
+
+**New to the SDK?** Start with the [Getting Started Guide](../../00-START-HERE/getting-started.md) first.
+
+## 🔗 Related Documentation
+
+- [Auto-Pagination Guide](../auto-pagination/auto-pagination-readme.md) - Automatic pagination for large datasets
+- [Universal Mapping Guide](../mapping/readme.md) - Transform extracted data
+- [GraphQL Mutation Mapping](../mapping/graphql-mutation-mapping/graphql-mutation-mapping-readme.md) - Create mutations from data
+- [Resolvers Guide](../mapping/resolvers/mapping-resolvers-readme.md) - Custom transformation resolvers
+- [SDK Architecture](../../04-REFERENCE/architecture/) - Understanding SDK design
+
+## 💡 Tips for Best Results
+
+1. **Always validate queries** - Use schema introspection to verify field names
+2. **Start with small datasets** - Test with `first: 10` before extracting thousands
+3. **Use auto-pagination** - Let the SDK handle cursor management
+4. **Monitor memory usage** - Use streaming for large datasets
+5. **Implement error recovery** - Add retry logic for network issues
+6. **Cache frequently accessed data** - Reduce API calls
+7. **Test transformations** - Validate field mappings with sample data
+
+## 🚦 Getting Started
+
+Ready to begin? Start with [Module 1: Foundations →](../auto-pagination/modules/auto-pagination-01-foundations.md)
+
+---
+
+**Need Help?**
+- 📖 Browse [Examples](./examples/)
+- 🔍 Check [Module 7: Optimization](./modules/02-core-guides-extraction-07-optimization.md)
+- 📧 Contact support@fluentcommerce.com
+- 🐛 Report issues at [GitHub](https://github.com/fluentcommerce/fc-connect-sdk/issues)

package/docs/02-CORE-GUIDES/ingestion/examples/_simple-kv-store.ts

@@ -1,40 +1,40 @@
-/**
- * Simple In-Memory KV Store Implementation
- *
- * This is a minimal implementation of the KVStore interface for standalone/testing.
- * On Versori platform, use VersoriKVAdapter instead.
- *
- * For production standalone deployments, consider using:
- * - Redis with ioredis
- * - DynamoDB
- * - MongoDB
- * - PostgreSQL with JSON columns
- */
-
-export class SimpleKVStore {
-  private store = new Map<string, any>();
-
-  async get(keys: string[]): Promise<{ value: any } | null> {
-    const key = keys.join(':');
-    const value = this.store.get(key);
-    return value !== undefined ? { value } : null;
-  }
-
-  async set(keys: string[], value: any): Promise<void> {
-    this.store.set(keys.join(':'), value);
-  }
-
-  async delete(keys: string[]): Promise<void> {
-    this.store.delete(keys.join(':'));
-  }
-
-  atomic() {
-    return {
-      check: () => {},
-      set: (key: string[], value: any) => {
-        this.store.set(key.join(':'), value);
-      },
-      commit: async () => true,
-    };
-  }
-}
+/**
+ * Simple In-Memory KV Store Implementation
+ *
+ * This is a minimal implementation of the KVStore interface for standalone/testing.
+ * On Versori platform, use VersoriKVAdapter instead.
+ *
+ * For production standalone deployments, consider using:
+ * - Redis with ioredis
+ * - DynamoDB
+ * - MongoDB
+ * - PostgreSQL with JSON columns
+ */
+
+export class SimpleKVStore {
+  private store = new Map<string, any>();
+
+  async get(keys: string[]): Promise<{ value: any } | null> {
+    const key = keys.join(':');
+    const value = this.store.get(key);
+    return value !== undefined ? { value } : null;
+  }
+
+  async set(keys: string[], value: any): Promise<void> {
+    this.store.set(keys.join(':'), value);
+  }
+
+  async delete(keys: string[]): Promise<void> {
+    this.store.delete(keys.join(':'));
+  }
+
+  atomic() {
+    return {
+      check: () => {},
+      set: (key: string[], value: any) => {
+        this.store.set(key.join(':'), value);
+      },
+      commit: async () => true,
+    };
+  }
+}