@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

package/docs/02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-01-foundations.md

@@ -1,355 +1,355 @@
-# Module 1: Parser Foundations
-
-**Level:** Beginner
-**Estimated Time:** 15 minutes
-
-## Overview
-
-This module introduces the four parser services available in the Fluent Connect SDK and helps you understand when to use each one. You'll learn about data format characteristics, parser capabilities, and how to make the right choice for your integration.
-
-## Learning Objectives
-
-By the end of this module, you will:
-- ✅ Understand the four parser types (CSV, JSON, XML, Parquet)
-- ✅ Know which parser to use for different data sources
-- ✅ Understand basic parser architecture and design patterns
-- ✅ Recognize common use cases for each format
-
-## What is a Parser?
-
-A **parser** is a service that reads data in a specific format and converts it into structured JavaScript objects that you can work with programmatically.
-
-```typescript
-// Input: Raw data string in some format
-const csvData = "sku,qty\nSKU001,100\nSKU002,50";
-
-// Parser: Converts format → JavaScript objects
-const parser = new CSVParserService();
-const records = await parser.parse(csvData);
-
-// Output: Structured data
-console.log(records);
-// [
-//   { sku: 'SKU001', qty: '100' },
-//   { sku: 'SKU002', qty: '50' }
-// ]
-```
-
-## The Four SDK Parsers
-
-### 1. CSV Parser (`CSVParserService`)
-
-**Format:** Comma-Separated Values (tabular data)
-
-**Characteristics:**
-- ✅ Simple, universal format
-- ✅ Row-based structure (each line is a record)
-- ✅ Headers define column names
-- ✅ Best for flat, tabular data
-
-**Example:**
-```csv
-sku,name,qty,location
-SKU001,Wireless Mouse,100,WH-01
-SKU002,Keyboard,50,WH-01
-```
-
-**When to use:**
-- Inventory feeds from warehouses
-- Spreadsheet exports (Excel → CSV)
-- Simple bulk data imports
-- Cross-system compatibility needed
-
-### 2. JSON Parser (`JSONParserService`)
-
-**Format:** JavaScript Object Notation (nested structures)
-
-**Characteristics:**
-- ✅ Native JavaScript format
-- ✅ Supports nested objects and arrays
-- ✅ Type-aware (strings, numbers, booleans)
-- ✅ Standard for APIs and modern systems
-
-**Example:**
-```json
-{
-  "products": [
-    {
-      "sku": "SKU001",
-      "name": "Wireless Mouse",
-      "stock": {
-        "qty": 100,
-        "locations": ["WH-01", "WH-02"]
-      }
-    }
-  ]
-}
-```
-
-**When to use:**
-- REST API responses
-- Webhook payloads from Fluent or other systems
-- Configuration files
-- Modern e-commerce platform exports
-
-### 3. XML Parser (`XMLParserService`)
-
-**Format:** eXtensible Markup Language (hierarchical)
-
-**Characteristics:**
-- ✅ Hierarchical, tag-based structure
-- ✅ Supports attributes and namespaces
-- ✅ Self-describing with schemas
-- ✅ Common in B2B and enterprise systems
-
-**Example:**
-```xml
-<?xml version="1.0" encoding="UTF-8"?>
-<products>
-  <product id="P001">
-    <sku>SKU001</sku>
-    <name>Wireless Mouse</name>
-    <stock qty="100" location="WH-01"/>
-  </product>
-</products>
-```
-
-**When to use:**
-- EDI/B2B integrations
-- Enterprise system exports
-- SOAP API responses
-- SFCC/Adobe Commerce order feeds
-
-### 4. Parquet Parser (`ParquetParserService`)
-
-**Format:** Apache Parquet (columnar binary)
-
-**Characteristics:**
-- ✅ Columnar storage format
-- ✅ Highly compressed
-- ✅ Optimized for analytics
-- ✅ Efficient for large datasets
-
-**Example:** (Binary format - not human-readable)
-```
-[Parquet file with columnar data]
-Columns: sku, qty, location
-Compressed: ~10:1 ratio
-```
-
-**When to use:**
-- Large-scale data extraction to S3
-- Analytics and reporting
-- Data warehouse ingestion
-- Memory-efficient processing of huge datasets
-
-## Parser Comparison Table
-
-| Feature | CSV | JSON | XML | Parquet |
-|---------|-----|------|-----|---------|
-| **Complexity** | Simple | Medium | Medium | Advanced |
-| **Structure** | Flat (rows) | Nested | Hierarchical | Columnar |
-| **Human-readable** | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Binary |
-| **File size** | Medium | Large | Large | Small (compressed) |
-| **Parsing speed** | Fast | Fast | Medium | Fast (streaming) |
-| **Nested data** | ❌ No | ✅ Yes | ✅ Yes | ✅ Yes |
-| **Attributes** | ❌ No | N/A | ✅ Yes | N/A |
-| **Streaming** | ✅ Yes | ✅ Yes (JSONL) | ✅ Yes | ✅ Yes |
-| **Common sources** | Warehouses, Excel | APIs, Webhooks | EDI, B2B | Analytics, S3 |
-
-## Decision Tree: Choosing the Right Parser
-
-```
-What is your data source?
-│
-├─ Modern API or webhook
-│  └─ Use JSON Parser (JSONParserService)
-│
-├─ Spreadsheet or simple tabular file
-│  └─ Use CSV Parser (CSVParserService)
-│
-├─ Enterprise system, EDI, or SFCC
-│  └─ Use XML Parser (XMLParserService)
-│
-└─ Large-scale extraction or analytics
-   └─ Use Parquet Parser (ParquetParserService)
-```
-
-### Real-World Scenarios
-
-**Scenario 1: Warehouse sends daily inventory file**
-- Format: CSV (simple, universal)
-- Parser: `CSVParserService`
-- Why: Warehouse systems typically export flat CSV files
-
-**Scenario 2: Webhook from Fluent Commerce**
-- Format: JSON (API standard)
-- Parser: `JSONParserService`
-- Why: Modern APIs use JSON for structured data
-
-**Scenario 3: SFCC order feed**
-- Format: XML (industry standard)
-- Parser: `XMLParserService`
-- Why: SFCC uses XML for order exports
-
-**Scenario 4: Extract 1M+ inventory positions to S3**
-- Format: Parquet (efficient storage)
-- Parser: `ParquetParserService`
-- Why: Compressed columnar format saves storage and processing time
-
-## Common Parser Architecture
-
-All SDK parsers follow a consistent pattern:
-
-### 1. Import the Parser
-
-```typescript
-import { CSVParserService, JSONParserService, XMLParserService, ParquetParserService } from '@fluentcommerce/fc-connect-sdk';
-```
-
-### 2. Create Parser Instance
-
-```typescript
-import { createConsoleLogger } from '@fluentcommerce/fc-connect-sdk';
-
-const logger = createConsoleLogger();
-
-const csvParser = new CSVParserService();
-const jsonParser = new JSONParserService();
-const xmlParser = new XMLParserService();
-const parquetParser = new ParquetParserService(logger);  // Logger required
-```
-
-### 3. Parse Data
-
-```typescript
-// All parsers expose a parse() method
-const records = await csvParser.parse(csvContent);
-const data = await jsonParser.parse(jsonContent);
-const parsed = await xmlParser.parse(xmlContent);
-const rows = await parquetParser.parse(parquetBuffer);
-```
-
-### 4. Process Results
-
-```typescript
-// Results are always JavaScript objects/arrays
-for (const record of records) {
-  console.log(record.sku, record.qty);
-}
-```
-
-## Design Principles
-
-The SDK parsers are designed with these principles:
-
-### 1. Consistency
-
-All parsers expose similar methods:
-- `parse()` - Parse data into memory
-- `parseStreaming()` - Stream large files
-- `validate()` - Validate data structure
-
-### 2. Safety
-
-Parsers include built-in protections:
-- ✅ Type validation
-- ✅ Error handling
-- ✅ Memory limits
-- ✅ Schema validation
-
-### 3. Performance
-
-Parsers are optimized for:
-- ✅ Streaming large files
-- ✅ Batch processing
-- ✅ Lazy loading (Parquet)
-- ✅ Memory efficiency
-
-### 4. Integration
-
-Parsers integrate seamlessly with:
-- ✅ `UniversalMapper` for field transformations
-- ✅ Batch API for Fluent ingestion
-- ✅ S3 services for file operations
-- ✅ State management for tracking
-
-## Common Workflow Pattern
-
-Typical parser usage in SDK integrations:
-
-```typescript
-import {
-  createClient,
-  CSVParserService,
-  UniversalMapper
-} from '@fluentcommerce/fc-connect-sdk';
-
-async function processInventoryFile(csvContent: string) {
-  // 1. Parse data
-  const parser = new CSVParserService();
-  const records = await parser.parse(csvContent);
-
-  // 2. Transform data
-  const mapper = new UniversalMapper(mappingConfig);
-  const result = await mapper.map(records);
-
-  // 3. Validate transformation
-  if (!result.success) {
-    throw new Error(`Mapping failed: ${result.errors.join(', ')}`);
-  }
-
-  // 4. Send to Fluent Commerce
-  const client = await createClient({ config });
-  const job = await client.createJob({
-    name: 'inventory-ingestion',
-    retailerId: config.config.retailerId
-  });
-
-  await client.sendBatch(job.id, {
-    action: 'UPSERT',
-    entityType: 'INVENTORY',
-    source: 'csv-import',
-    event: 'inventory.update',
-    entities: result.data
-  });
-}
-```
-
-## Key Takeaways
-
-- 🎯 **CSV Parser** - Simple, universal, best for flat tabular data
-- 🎯 **JSON Parser** - Modern, nested, best for APIs and webhooks
-- 🎯 **XML Parser** - Hierarchical, attributes, best for B2B and EDI
-- 🎯 **Parquet Parser** - Columnar, compressed, best for large-scale extraction
-- 🎯 All parsers follow consistent API patterns
-- 🎯 Parsers integrate with UniversalMapper for transformations
-- 🎯 Choose parser based on data source and structure
-
-## Practice Exercise
-
-**Challenge:** Identify which parser to use for these scenarios:
-
-1. Your warehouse sends a daily `inventory.csv` file with columns: sku, qty, location
-2. Shopify sends a webhook with JSON order data
-3. Your enterprise WMS exports `orders.xml` with nested items
-4. You need to extract 5 million inventory records to S3 for analytics
-
-**Answers:**
-1. CSV Parser - Simple tabular format
-2. JSON Parser - Modern API webhook
-3. XML Parser - Enterprise hierarchical format
-4. Parquet Parser - Large-scale efficient extraction
-
-## Next Steps
-
-Now that you understand parser fundamentals, continue to:
-- [Module 2: CSV Parser](./02-core-guides-parsers-02-csv-parser.md) - Learn CSV parsing in depth
-- [Module 3: JSON Parser](./02-core-guides-parsers-03-json-parser.md) - Master JSON and JSONL
-- [Module 4: XML Parser](./02-core-guides-parsers-04-xml-parser.md) - Handle XML transformations
-- [Module 5: Parquet Parser](./02-core-guides-parsers-05-parquet-parser.md) - Work with columnar data
-
----
-
-**Module 1 Complete!** ✅
-Continue to [Module 2: CSV Parser →](./02-core-guides-parsers-02-csv-parser.md)
+# Module 1: Parser Foundations
+
+**Level:** Beginner
+**Estimated Time:** 15 minutes
+
+## Overview
+
+This module introduces the four parser services available in the Fluent Connect SDK and helps you understand when to use each one. You'll learn about data format characteristics, parser capabilities, and how to make the right choice for your integration.
+
+## Learning Objectives
+
+By the end of this module, you will:
+- ✅ Understand the four parser types (CSV, JSON, XML, Parquet)
+- ✅ Know which parser to use for different data sources
+- ✅ Understand basic parser architecture and design patterns
+- ✅ Recognize common use cases for each format
+
+## What is a Parser?
+
+A **parser** is a service that reads data in a specific format and converts it into structured JavaScript objects that you can work with programmatically.
+
+```typescript
+// Input: Raw data string in some format
+const csvData = "sku,qty\nSKU001,100\nSKU002,50";
+
+// Parser: Converts format → JavaScript objects
+const parser = new CSVParserService();
+const records = await parser.parse(csvData);
+
+// Output: Structured data
+console.log(records);
+// [
+//   { sku: 'SKU001', qty: '100' },
+//   { sku: 'SKU002', qty: '50' }
+// ]
+```
+
+## The Four SDK Parsers
+
+### 1. CSV Parser (`CSVParserService`)
+
+**Format:** Comma-Separated Values (tabular data)
+
+**Characteristics:**
+- ✅ Simple, universal format
+- ✅ Row-based structure (each line is a record)
+- ✅ Headers define column names
+- ✅ Best for flat, tabular data
+
+**Example:**
+```csv
+sku,name,qty,location
+SKU001,Wireless Mouse,100,WH-01
+SKU002,Keyboard,50,WH-01
+```
+
+**When to use:**
+- Inventory feeds from warehouses
+- Spreadsheet exports (Excel → CSV)
+- Simple bulk data imports
+- Cross-system compatibility needed
+
+### 2. JSON Parser (`JSONParserService`)
+
+**Format:** JavaScript Object Notation (nested structures)
+
+**Characteristics:**
+- ✅ Native JavaScript format
+- ✅ Supports nested objects and arrays
+- ✅ Type-aware (strings, numbers, booleans)
+- ✅ Standard for APIs and modern systems
+
+**Example:**
+```json
+{
+  "products": [
+    {
+      "sku": "SKU001",
+      "name": "Wireless Mouse",
+      "stock": {
+        "qty": 100,
+        "locations": ["WH-01", "WH-02"]
+      }
+    }
+  ]
+}
+```
+
+**When to use:**
+- REST API responses
+- Webhook payloads from Fluent or other systems
+- Configuration files
+- Modern e-commerce platform exports
+
+### 3. XML Parser (`XMLParserService`)
+
+**Format:** eXtensible Markup Language (hierarchical)
+
+**Characteristics:**
+- ✅ Hierarchical, tag-based structure
+- ✅ Supports attributes and namespaces
+- ✅ Self-describing with schemas
+- ✅ Common in B2B and enterprise systems
+
+**Example:**
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<products>
+  <product id="P001">
+    <sku>SKU001</sku>
+    <name>Wireless Mouse</name>
+    <stock qty="100" location="WH-01"/>
+  </product>
+</products>
+```
+
+**When to use:**
+- EDI/B2B integrations
+- Enterprise system exports
+- SOAP API responses
+- SFCC/Adobe Commerce order feeds
+
+### 4. Parquet Parser (`ParquetParserService`)
+
+**Format:** Apache Parquet (columnar binary)
+
+**Characteristics:**
+- ✅ Columnar storage format
+- ✅ Highly compressed
+- ✅ Optimized for analytics
+- ✅ Efficient for large datasets
+
+**Example:** (Binary format - not human-readable)
+```
+[Parquet file with columnar data]
+Columns: sku, qty, location
+Compressed: ~10:1 ratio
+```
+
+**When to use:**
+- Large-scale data extraction to S3
+- Analytics and reporting
+- Data warehouse ingestion
+- Memory-efficient processing of huge datasets
+
+## Parser Comparison Table
+
+| Feature | CSV | JSON | XML | Parquet |
+|---------|-----|------|-----|---------|
+| **Complexity** | Simple | Medium | Medium | Advanced |
+| **Structure** | Flat (rows) | Nested | Hierarchical | Columnar |
+| **Human-readable** | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Binary |
+| **File size** | Medium | Large | Large | Small (compressed) |
+| **Parsing speed** | Fast | Fast | Medium | Fast (streaming) |
+| **Nested data** | ❌ No | ✅ Yes | ✅ Yes | ✅ Yes |
+| **Attributes** | ❌ No | N/A | ✅ Yes | N/A |
+| **Streaming** | ✅ Yes | ✅ Yes (JSONL) | ✅ Yes | ✅ Yes |
+| **Common sources** | Warehouses, Excel | APIs, Webhooks | EDI, B2B | Analytics, S3 |
+
+## Decision Tree: Choosing the Right Parser
+
+```
+What is your data source?
+│
+├─ Modern API or webhook
+│  └─ Use JSON Parser (JSONParserService)
+│
+├─ Spreadsheet or simple tabular file
+│  └─ Use CSV Parser (CSVParserService)
+│
+├─ Enterprise system, EDI, or SFCC
+│  └─ Use XML Parser (XMLParserService)
+│
+└─ Large-scale extraction or analytics
+   └─ Use Parquet Parser (ParquetParserService)
+```
+
+### Real-World Scenarios
+
+**Scenario 1: Warehouse sends daily inventory file**
+- Format: CSV (simple, universal)
+- Parser: `CSVParserService`
+- Why: Warehouse systems typically export flat CSV files
+
+**Scenario 2: Webhook from Fluent Commerce**
+- Format: JSON (API standard)
+- Parser: `JSONParserService`
+- Why: Modern APIs use JSON for structured data
+
+**Scenario 3: SFCC order feed**
+- Format: XML (industry standard)
+- Parser: `XMLParserService`
+- Why: SFCC uses XML for order exports
+
+**Scenario 4: Extract 1M+ inventory positions to S3**
+- Format: Parquet (efficient storage)
+- Parser: `ParquetParserService`
+- Why: Compressed columnar format saves storage and processing time
+
+## Common Parser Architecture
+
+All SDK parsers follow a consistent pattern:
+
+### 1. Import the Parser
+
+```typescript
+import { CSVParserService, JSONParserService, XMLParserService, ParquetParserService } from '@fluentcommerce/fc-connect-sdk';
+```
+
+### 2. Create Parser Instance
+
+```typescript
+import { createConsoleLogger } from '@fluentcommerce/fc-connect-sdk';
+
+const logger = createConsoleLogger();
+
+const csvParser = new CSVParserService();
+const jsonParser = new JSONParserService();
+const xmlParser = new XMLParserService();
+const parquetParser = new ParquetParserService(logger);  // Logger required
+```
+
+### 3. Parse Data
+
+```typescript
+// All parsers expose a parse() method
+const records = await csvParser.parse(csvContent);
+const data = await jsonParser.parse(jsonContent);
+const parsed = await xmlParser.parse(xmlContent);
+const rows = await parquetParser.parse(parquetBuffer);
+```
+
+### 4. Process Results
+
+```typescript
+// Results are always JavaScript objects/arrays
+for (const record of records) {
+  console.log(record.sku, record.qty);
+}
+```
+
+## Design Principles
+
+The SDK parsers are designed with these principles:
+
+### 1. Consistency
+
+All parsers expose similar methods:
+- `parse()` - Parse data into memory
+- `parseStreaming()` - Stream large files
+- `validate()` - Validate data structure
+
+### 2. Safety
+
+Parsers include built-in protections:
+- ✅ Type validation
+- ✅ Error handling
+- ✅ Memory limits
+- ✅ Schema validation
+
+### 3. Performance
+
+Parsers are optimized for:
+- ✅ Streaming large files
+- ✅ Batch processing
+- ✅ Lazy loading (Parquet)
+- ✅ Memory efficiency
+
+### 4. Integration
+
+Parsers integrate seamlessly with:
+- ✅ `UniversalMapper` for field transformations
+- ✅ Batch API for Fluent ingestion
+- ✅ S3 services for file operations
+- ✅ State management for tracking
+
+## Common Workflow Pattern
+
+Typical parser usage in SDK integrations:
+
+```typescript
+import {
+  createClient,
+  CSVParserService,
+  UniversalMapper
+} from '@fluentcommerce/fc-connect-sdk';
+
+async function processInventoryFile(csvContent: string) {
+  // 1. Parse data
+  const parser = new CSVParserService();
+  const records = await parser.parse(csvContent);
+
+  // 2. Transform data
+  const mapper = new UniversalMapper(mappingConfig);
+  const result = await mapper.map(records);
+
+  // 3. Validate transformation
+  if (!result.success) {
+    throw new Error(`Mapping failed: ${result.errors.join(', ')}`);
+  }
+
+  // 4. Send to Fluent Commerce
+  const client = await createClient({ config });
+  const job = await client.createJob({
+    name: 'inventory-ingestion',
+    retailerId: config.config.retailerId
+  });
+
+  await client.sendBatch(job.id, {
+    action: 'UPSERT',
+    entityType: 'INVENTORY',
+    source: 'csv-import',
+    event: 'inventory.update',
+    entities: result.data
+  });
+}
+```
+
+## Key Takeaways
+
+- 🎯 **CSV Parser** - Simple, universal, best for flat tabular data
+- 🎯 **JSON Parser** - Modern, nested, best for APIs and webhooks
+- 🎯 **XML Parser** - Hierarchical, attributes, best for B2B and EDI
+- 🎯 **Parquet Parser** - Columnar, compressed, best for large-scale extraction
+- 🎯 All parsers follow consistent API patterns
+- 🎯 Parsers integrate with UniversalMapper for transformations
+- 🎯 Choose parser based on data source and structure
+
+## Practice Exercise
+
+**Challenge:** Identify which parser to use for these scenarios:
+
+1. Your warehouse sends a daily `inventory.csv` file with columns: sku, qty, location
+2. Shopify sends a webhook with JSON order data
+3. Your enterprise WMS exports `orders.xml` with nested items
+4. You need to extract 5 million inventory records to S3 for analytics
+
+**Answers:**
+1. CSV Parser - Simple tabular format
+2. JSON Parser - Modern API webhook
+3. XML Parser - Enterprise hierarchical format
+4. Parquet Parser - Large-scale efficient extraction
+
+## Next Steps
+
+Now that you understand parser fundamentals, continue to:
+- [Module 2: CSV Parser](./02-core-guides-parsers-02-csv-parser.md) - Learn CSV parsing in depth
+- [Module 3: JSON Parser](./02-core-guides-parsers-03-json-parser.md) - Master JSON and JSONL
+- [Module 4: XML Parser](./02-core-guides-parsers-04-xml-parser.md) - Handle XML transformations
+- [Module 5: Parquet Parser](./02-core-guides-parsers-05-parquet-parser.md) - Work with columnar data
+
+---
+
+**Module 1 Complete!** ✅
+Continue to [Module 2: CSV Parser →](./02-core-guides-parsers-02-csv-parser.md)