@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/03-PATTERN-GUIDES/parquet/parquet-readme.md

@@ -1,248 +1,248 @@
-# Parquet File Processing Learning Path
-
-> **Complete guide to Parquet file operations in the Fluent Connect SDK**
-> Progressive learning modules with real schema examples, performance optimization, and verified patterns
-
-## 📚 What You'll Learn
-
-This guide teaches you how to efficiently process Parquet files for **Inventory Management**, **Product Catalog**, and **Order Management** operations. By the end, you'll understand:
-
-- ✅ When and why to use Parquet format for data exchange
-- ✅ Reading and writing Parquet files with validation
-- ✅ Extracting data from Fluent Commerce GraphQL → Parquet
-- ✅ S3 integration for data lake operations
-- ✅ Performance optimization for large-scale data processing
-- ✅ Production-ready patterns with error handling
-- ✅ Memory-efficient streaming for massive datasets
-
-## 🎯 Learning Path Overview
-
-This guide is organized into **8 progressive modules**. Follow them in order for the best learning experience:
-
-| Module | Topic | Level | Time |
-|--------|-------|-------|------|
-| [Module 1](./modules/03-pattern-guides-parquet-01-introduction.md) | **Introduction** - What is Parquet, when to use it, key features | Beginner | 10 min |
-| [Module 2](./modules/03-pattern-guides-parquet-02-quick-start.md) | **Quick Start** - Installation, basic parsing, first examples | Beginner | 15 min |
-| [Module 3](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | **Reading Parquet** - Streaming, filtering, column selection | Intermediate | 20 min |
-| [Module 4](./modules/03-pattern-guides-parquet-04-writing-parquet.md) | **Writing Parquet** - Schema definition, compression, data types | Intermediate | 20 min |
-| [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | **GraphQL Extraction** - Extract from Fluent → Parquet | Intermediate | 25 min |
-| [Module 6](./modules/03-pattern-guides-parquet-06-s3-integration.md) | **S3 Integration** - S3 operations, presigned URLs, streaming | Advanced | 20 min |
-| [Module 7](./modules/03-pattern-guides-parquet-07-performance-optimization.md) | **Performance** - Partitioning, compression, parallel processing | Advanced | 25 min |
-| [Module 8](./modules/03-pattern-guides-parquet-08-best-practices.md) | **Best Practices** - Error handling, monitoring, production patterns | All Levels | 20 min |
-
-**Total Learning Time**: ~2.5 hours
-
----
-
-## 🚀 Quick Navigation
-
-### By Use Case
-
-- **First time using Parquet?** → Start with [Module 1: Introduction](./modules/03-pattern-guides-parquet-01-introduction.md)
-- **Need a working example now?** → Jump to [Module 2: Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md)
-- **Extracting from Fluent Commerce?** → See [Module 5: GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md)
-- **Processing large files?** → Check [Module 7: Performance Optimization](./modules/03-pattern-guides-parquet-07-performance-optimization.md)
-- **Experiencing issues?** → See [Module 8: Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md)
-- **Need S3 integration?** → Check [Module 6: S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md)
-
-### By Business Domain
-
-| Domain | Use Case | Complexity | Module | Example Size |
-|--------|----------|------------|--------|--------------|
-| **Inventory Management** | Daily inventory extracts | Medium | [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | ~1M records |
-| **Product Catalog** | Product data sync | Low | [Module 3](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | ~100K products |
-| **Order Management** | Historical order analysis | High | [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | ~500K orders |
-| **Data Lake Integration** | ETL pipelines | High | [Module 7](./modules/03-pattern-guides-parquet-07-performance-optimization.md) | Multi-million |
-| **Business Intelligence** | Analytics reporting | Medium | [Module 6](./modules/03-pattern-guides-parquet-06-s3-integration.md) | Partitioned |
-
-### By Task
-
-| **Goal** | **Start Here** | **Then See** |
-|----------|----------------|--------------|
-| **Parse a Parquet file** | [Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md) | [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) |
-| **Extract from GraphQL** | [GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) |
-| **Process large files** | [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | [Performance](./modules/03-pattern-guides-parquet-07-performance-optimization.md) |
-| **Write Parquet files** | [Writing Parquet](./modules/03-pattern-guides-parquet-04-writing-parquet.md) | [Performance](./modules/03-pattern-guides-parquet-07-performance-optimization.md) |
-| **Upload to S3** | [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) | [Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) |
-| **Handle errors** | [Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) | [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) |
-
----
-
-## 📖 Quick Reference
-
-For quick lookups, see:
-- **[Module 2: Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md)** - All basic operations on one page
-- **[Module 8: Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md)** - Common patterns and troubleshooting
-- **[Performance Metrics](./modules/03-pattern-guides-parquet-07-performance-optimization.md#performance-benchmarks)** - Expected throughput
-
----
-
-## 🎓 Prerequisites
-
-Before starting this guide, you should:
-
-1. ✅ Have the SDK installed: `npm install @fluentcommerce/fc-connect-sdk`
-2. ✅ Have Fluent API credentials (clientId, clientSecret, retailerId)
-3. ✅ Understand basic GraphQL concepts (queries, mutations, pagination)
-4. ✅ Have AWS S3 credentials (for S3 integration modules)
-
-**New to the SDK?** Start with the [Getting Started Guide](../../00-START-HERE/getting-started.md) first.
-
----
-
-## 🔍 Key Concepts at a Glance
-
-### What is Parquet?
-
-**Parquet** is a columnar storage format optimized for analytics workloads:
-
-```typescript
-// Traditional row-based format (CSV, JSON)
-// Reads entire records even when you need one column
-[
-  { sku: "ABC", location: "WH1", qty: 100, status: "AVAILABLE" },
-  { sku: "DEF", location: "WH2", qty: 50, status: "RESERVED" }
-]
-
-// Columnar format (Parquet)
-// Reads only columns you need - much faster for analytics
-{
-  sku:      ["ABC", "DEF"],
-  location: ["WH1", "WH2"],
-  qty:      [100, 50],
-  status:   ["AVAILABLE", "RESERVED"]
-}
-```
-
-**Benefits**:
-- 🚀 **10-100x faster** for analytical queries
-- 💾 **50-80% smaller** files with compression
-- 📊 **Query only columns needed** - reduced I/O
-- 🔧 **Schema enforcement** - data integrity
-
-### SDK ParquetParserService
-
-The SDK provides **fully functional** Parquet processing:
-
-```typescript
-import { ParquetParserService } from '@fluentcommerce/fc-connect-sdk';
-
-// Parse a Parquet file
-const parser = new ParquetParserService(logger);
-const { records, stats } = await parser.parse(arrayBuffer, 'inventory.parquet');
-
-console.log(`Processed ${stats.totalRecords} records in ${stats.processingTimeMs}ms`);
-// Throughput: 50,000 - 500,000 records/sec depending on file size
-```
-
-**Key Features**:
-- ✅ Powered by high-performance `hyparquet` library
-- ✅ Streaming support for memory-efficient processing
-- ✅ Built-in validation with customizable rules
-- ✅ Column statistics for data profiling
-- ✅ Batch processing with configurable sizes
-- ✅ Performance metrics tracking
-
----
-
-## 🎯 Learning Paths by Role
-
-### Data Engineer
-1. [Introduction](./modules/03-pattern-guides-parquet-01-introduction.md) - Understand Parquet format
-2. [GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) - Extract from Fluent Commerce
-3. [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) - Build ETL pipelines
-4. [Performance Optimization](./modules/03-pattern-guides-parquet-07-performance-optimization.md) - Scale to millions of records
-
-### Integration Developer
-1. [Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md) - Basic operations
-2. [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) - Process incoming files
-3. [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) - S3 workflows
-4. [Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) - Production patterns
-
-### Analytics Developer
-1. [Introduction](./modules/03-pattern-guides-parquet-01-introduction.md) - Why Parquet for analytics
-2. [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) - Query optimization
-3. [Performance Optimization](./modules/03-pattern-guides-parquet-07-performance-optimization.md) - Partitioning strategies
-4. [GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) - Data source integration
-
----
-
-## 📊 Documentation Structure
-
-```
-parquet/
-├── readme.md (this file - navigation hub)
-└── modules/
-    ├── 01-introduction.md          # What is Parquet, when to use
-    ├── 02-quick-start.md            # Installation, basic examples
-    ├── 03-reading-parquet.md        # Streaming, filtering, validation
-    ├── 04-writing-parquet.md        # Schema, compression, data types
-    ├── 05-graphql-extraction.md/     # Fluent Commerce → Parquet
-    ├── 06-s3-integration.md         # S3 operations, presigned URLs
-    ├── 07-performance-optimization.md # Partitioning, parallel processing
-    └── 08-best-practices.md         # Error handling, monitoring
-```
-
----
-
-## 🔗 Related Documentation
-
-### GraphQL Operations
-- **[Data Extraction Guide](../../02-CORE-GUIDES/extraction/extraction-readme.md)** - Complete GraphQL extraction workflows
-- **[Auto-Pagination Guide](../../02-CORE-GUIDES/auto-pagination/auto-pagination-readme.md)** - Automatic pagination for large datasets
-- **[Schema Validation](../../04-REFERENCE/schema/fluent-commerce-schema.json)** - Valid GraphQL fields reference
-
-### S3 Operations
-- **[S3 Data Source Guide](../../02-CORE-GUIDES/data-sources/data-sources-readme.md)** - S3 upload/download patterns
-- **[S3 Presigned URLs](../../02-CORE-GUIDES/data-sources/data-sources-readme.md)** - Secure file access
-
-### Data Transformation
-- **[Universal Mapping Guide](../../02-CORE-GUIDES/mapping/)** - Field mapping and transformations
-- **[Ingestion Guide](../../02-CORE-GUIDES/ingestion/ingestion-readme.md)** - Inverse operation (Parquet → Fluent)
-
-### Test Data
-- **[Canonical Inventory](../examples/test-data/canonical-inventory.json)** - Sample inventory data
-- **[Canonical Order](../examples/test-data/canonical-order.json)** - Sample order data
-
----
-
-## 🚨 Critical Notes
-
-### Schema Validation
-
-**⚠️ IMPORTANT**: Before using Parquet extraction, validate all GraphQL queries against your Fluent Commerce schema:
-
-```bash
-# Step 1: Introspect your schema
-npx @fluentcommerce/fc-connect-sdk introspect-schema \
-  --url https://YOUR_ACCOUNT.api.fluentcommerce.com/graphql \
-  --output schema.json
-
-# Step 2: Verify extraction query fields
-# All queries in this guide are validated against docs/schema/fluent-commerce-schema.json
-```
-
-### PageInfo Pattern
-
-**⚠️ FLUENT COMMERCE DEVIATION**: Fluent's PageInfo **does NOT** return `startCursor`/`endCursor`:
-
-```typescript
-// ❌ WRONG - Fluent doesn't return endCursor
-const cursor = result.data.inventoryPositions.pageInfo.endCursor;
-
-// ✅ CORRECT - Use last edge cursor
-const edges = result.data.inventoryPositions.edges;
-const cursor = edges.length > 0 ? edges[edges.length - 1].cursor : null;
-```
-
-See [Auto-Pagination Guide](../../02-CORE-GUIDES/auto-pagination/auto-pagination-readme.md) for complete details.
-
----
-
-## 🎓 Getting Started
-
-Ready to start learning? Begin with [Module 1: Introduction](./modules/03-pattern-guides-parquet-01-introduction.md) to understand Parquet fundamentals, or jump directly to [Module 2: Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md) for hands-on examples.
-
----
-
-**Need Help?** See [Module 8: Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) for troubleshooting and common patterns.
+# Parquet File Processing Learning Path
+
+> **Complete guide to Parquet file operations in the Fluent Connect SDK**
+> Progressive learning modules with real schema examples, performance optimization, and verified patterns
+
+## 📚 What You'll Learn
+
+This guide teaches you how to efficiently process Parquet files for **Inventory Management**, **Product Catalog**, and **Order Management** operations. By the end, you'll understand:
+
+- ✅ When and why to use Parquet format for data exchange
+- ✅ Reading and writing Parquet files with validation
+- ✅ Extracting data from Fluent Commerce GraphQL → Parquet
+- ✅ S3 integration for data lake operations
+- ✅ Performance optimization for large-scale data processing
+- ✅ Production-ready patterns with error handling
+- ✅ Memory-efficient streaming for massive datasets
+
+## 🎯 Learning Path Overview
+
+This guide is organized into **8 progressive modules**. Follow them in order for the best learning experience:
+
+| Module | Topic | Level | Time |
+|--------|-------|-------|------|
+| [Module 1](./modules/03-pattern-guides-parquet-01-introduction.md) | **Introduction** - What is Parquet, when to use it, key features | Beginner | 10 min |
+| [Module 2](./modules/03-pattern-guides-parquet-02-quick-start.md) | **Quick Start** - Installation, basic parsing, first examples | Beginner | 15 min |
+| [Module 3](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | **Reading Parquet** - Streaming, filtering, column selection | Intermediate | 20 min |
+| [Module 4](./modules/03-pattern-guides-parquet-04-writing-parquet.md) | **Writing Parquet** - Schema definition, compression, data types | Intermediate | 20 min |
+| [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | **GraphQL Extraction** - Extract from Fluent → Parquet | Intermediate | 25 min |
+| [Module 6](./modules/03-pattern-guides-parquet-06-s3-integration.md) | **S3 Integration** - S3 operations, presigned URLs, streaming | Advanced | 20 min |
+| [Module 7](./modules/03-pattern-guides-parquet-07-performance-optimization.md) | **Performance** - Partitioning, compression, parallel processing | Advanced | 25 min |
+| [Module 8](./modules/03-pattern-guides-parquet-08-best-practices.md) | **Best Practices** - Error handling, monitoring, production patterns | All Levels | 20 min |
+
+**Total Learning Time**: ~2.5 hours
+
+---
+
+## 🚀 Quick Navigation
+
+### By Use Case
+
+- **First time using Parquet?** → Start with [Module 1: Introduction](./modules/03-pattern-guides-parquet-01-introduction.md)
+- **Need a working example now?** → Jump to [Module 2: Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md)
+- **Extracting from Fluent Commerce?** → See [Module 5: GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md)
+- **Processing large files?** → Check [Module 7: Performance Optimization](./modules/03-pattern-guides-parquet-07-performance-optimization.md)
+- **Experiencing issues?** → See [Module 8: Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md)
+- **Need S3 integration?** → Check [Module 6: S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md)
+
+### By Business Domain
+
+| Domain | Use Case | Complexity | Module | Example Size |
+|--------|----------|------------|--------|--------------|
+| **Inventory Management** | Daily inventory extracts | Medium | [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | ~1M records |
+| **Product Catalog** | Product data sync | Low | [Module 3](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | ~100K products |
+| **Order Management** | Historical order analysis | High | [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | ~500K orders |
+| **Data Lake Integration** | ETL pipelines | High | [Module 7](./modules/03-pattern-guides-parquet-07-performance-optimization.md) | Multi-million |
+| **Business Intelligence** | Analytics reporting | Medium | [Module 6](./modules/03-pattern-guides-parquet-06-s3-integration.md) | Partitioned |
+
+### By Task
+
+| **Goal** | **Start Here** | **Then See** |
+|----------|----------------|--------------|
+| **Parse a Parquet file** | [Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md) | [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) |
+| **Extract from GraphQL** | [GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) |
+| **Process large files** | [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | [Performance](./modules/03-pattern-guides-parquet-07-performance-optimization.md) |
+| **Write Parquet files** | [Writing Parquet](./modules/03-pattern-guides-parquet-04-writing-parquet.md) | [Performance](./modules/03-pattern-guides-parquet-07-performance-optimization.md) |
+| **Upload to S3** | [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) | [Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) |
+| **Handle errors** | [Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) | [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) |
+
+---
+
+## 📖 Quick Reference
+
+For quick lookups, see:
+- **[Module 2: Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md)** - All basic operations on one page
+- **[Module 8: Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md)** - Common patterns and troubleshooting
+- **[Performance Metrics](./modules/03-pattern-guides-parquet-07-performance-optimization.md#performance-benchmarks)** - Expected throughput
+
+---
+
+## 🎓 Prerequisites
+
+Before starting this guide, you should:
+
+1. ✅ Have the SDK installed: `npm install @fluentcommerce/fc-connect-sdk`
+2. ✅ Have Fluent API credentials (clientId, clientSecret, retailerId)
+3. ✅ Understand basic GraphQL concepts (queries, mutations, pagination)
+4. ✅ Have AWS S3 credentials (for S3 integration modules)
+
+**New to the SDK?** Start with the [Getting Started Guide](../../00-START-HERE/getting-started.md) first.
+
+---
+
+## 🔍 Key Concepts at a Glance
+
+### What is Parquet?
+
+**Parquet** is a columnar storage format optimized for analytics workloads:
+
+```typescript
+// Traditional row-based format (CSV, JSON)
+// Reads entire records even when you need one column
+[
+  { sku: "ABC", location: "WH1", qty: 100, status: "AVAILABLE" },
+  { sku: "DEF", location: "WH2", qty: 50, status: "RESERVED" }
+]
+
+// Columnar format (Parquet)
+// Reads only columns you need - much faster for analytics
+{
+  sku:      ["ABC", "DEF"],
+  location: ["WH1", "WH2"],
+  qty:      [100, 50],
+  status:   ["AVAILABLE", "RESERVED"]
+}
+```
+
+**Benefits**:
+- 🚀 **10-100x faster** for analytical queries
+- 💾 **50-80% smaller** files with compression
+- 📊 **Query only columns needed** - reduced I/O
+- 🔧 **Schema enforcement** - data integrity
+
+### SDK ParquetParserService
+
+The SDK provides **fully functional** Parquet processing:
+
+```typescript
+import { ParquetParserService } from '@fluentcommerce/fc-connect-sdk';
+
+// Parse a Parquet file
+const parser = new ParquetParserService(logger);
+const { records, stats } = await parser.parse(arrayBuffer, 'inventory.parquet');
+
+console.log(`Processed ${stats.totalRecords} records in ${stats.processingTimeMs}ms`);
+// Throughput: 50,000 - 500,000 records/sec depending on file size
+```
+
+**Key Features**:
+- ✅ Powered by high-performance `hyparquet` library
+- ✅ Streaming support for memory-efficient processing
+- ✅ Built-in validation with customizable rules
+- ✅ Column statistics for data profiling
+- ✅ Batch processing with configurable sizes
+- ✅ Performance metrics tracking
+
+---
+
+## 🎯 Learning Paths by Role
+
+### Data Engineer
+1. [Introduction](./modules/03-pattern-guides-parquet-01-introduction.md) - Understand Parquet format
+2. [GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) - Extract from Fluent Commerce
+3. [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) - Build ETL pipelines
+4. [Performance Optimization](./modules/03-pattern-guides-parquet-07-performance-optimization.md) - Scale to millions of records
+
+### Integration Developer
+1. [Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md) - Basic operations
+2. [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) - Process incoming files
+3. [S3 Integration](./modules/03-pattern-guides-parquet-06-s3-integration.md) - S3 workflows
+4. [Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) - Production patterns
+
+### Analytics Developer
+1. [Introduction](./modules/03-pattern-guides-parquet-01-introduction.md) - Why Parquet for analytics
+2. [Reading Parquet](./modules/03-pattern-guides-parquet-03-reading-parquet.md) - Query optimization
+3. [Performance Optimization](./modules/03-pattern-guides-parquet-07-performance-optimization.md) - Partitioning strategies
+4. [GraphQL Extraction](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) - Data source integration
+
+---
+
+## 📊 Documentation Structure
+
+```
+parquet/
+├── readme.md (this file - navigation hub)
+└── modules/
+    ├── 01-introduction.md          # What is Parquet, when to use
+    ├── 02-quick-start.md            # Installation, basic examples
+    ├── 03-reading-parquet.md        # Streaming, filtering, validation
+    ├── 04-writing-parquet.md        # Schema, compression, data types
+    ├── 05-graphql-extraction.md/     # Fluent Commerce → Parquet
+    ├── 06-s3-integration.md         # S3 operations, presigned URLs
+    ├── 07-performance-optimization.md # Partitioning, parallel processing
+    └── 08-best-practices.md         # Error handling, monitoring
+```
+
+---
+
+## 🔗 Related Documentation
+
+### GraphQL Operations
+- **[Data Extraction Guide](../../02-CORE-GUIDES/extraction/extraction-readme.md)** - Complete GraphQL extraction workflows
+- **[Auto-Pagination Guide](../../02-CORE-GUIDES/auto-pagination/auto-pagination-readme.md)** - Automatic pagination for large datasets
+- **[Schema Validation](../../04-REFERENCE/schema/fluent-commerce-schema.json)** - Valid GraphQL fields reference
+
+### S3 Operations
+- **[S3 Data Source Guide](../../02-CORE-GUIDES/data-sources/data-sources-readme.md)** - S3 upload/download patterns
+- **[S3 Presigned URLs](../../02-CORE-GUIDES/data-sources/data-sources-readme.md)** - Secure file access
+
+### Data Transformation
+- **[Universal Mapping Guide](../../02-CORE-GUIDES/mapping/)** - Field mapping and transformations
+- **[Ingestion Guide](../../02-CORE-GUIDES/ingestion/ingestion-readme.md)** - Inverse operation (Parquet → Fluent)
+
+### Test Data
+- **[Canonical Inventory](../examples/test-data/canonical-inventory.json)** - Sample inventory data
+- **[Canonical Order](../examples/test-data/canonical-order.json)** - Sample order data
+
+---
+
+## 🚨 Critical Notes
+
+### Schema Validation
+
+**⚠️ IMPORTANT**: Before using Parquet extraction, validate all GraphQL queries against your Fluent Commerce schema:
+
+```bash
+# Step 1: Introspect your schema
+npx @fluentcommerce/fc-connect-sdk introspect-schema \
+  --url https://YOUR_ACCOUNT.api.fluentcommerce.com/graphql \
+  --output schema.json
+
+# Step 2: Verify extraction query fields
+# All queries in this guide are validated against docs/schema/fluent-commerce-schema.json
+```
+
+### PageInfo Pattern
+
+**⚠️ FLUENT COMMERCE DEVIATION**: Fluent's PageInfo **does NOT** return `startCursor`/`endCursor`:
+
+```typescript
+// ❌ WRONG - Fluent doesn't return endCursor
+const cursor = result.data.inventoryPositions.pageInfo.endCursor;
+
+// ✅ CORRECT - Use last edge cursor
+const edges = result.data.inventoryPositions.edges;
+const cursor = edges.length > 0 ? edges[edges.length - 1].cursor : null;
+```
+
+See [Auto-Pagination Guide](../../02-CORE-GUIDES/auto-pagination/auto-pagination-readme.md) for complete details.
+
+---
+
+## 🎓 Getting Started
+
+Ready to start learning? Begin with [Module 1: Introduction](./modules/03-pattern-guides-parquet-01-introduction.md) to understand Parquet fundamentals, or jump directly to [Module 2: Quick Start](./modules/03-pattern-guides-parquet-02-quick-start.md) for hands-on examples.
+
+---
+
+**Need Help?** See [Module 8: Best Practices](./modules/03-pattern-guides-parquet-08-best-practices.md) for troubleshooting and common patterns.

package/docs/03-PATTERN-GUIDES/parquet/readme.md

@@ -1,32 +1,32 @@
-# Parquet Processing Guide
-
-> **Parquet file processing patterns**
-> Read and write Parquet files efficiently
-
-For complete documentation, see [parquet-readme.md](./parquet-readme.md)
-
-## Overview
-
-This guide covers Parquet file processing for analytics and data lake integrations, including reading, writing, partitioning, and S3 integration.
-
-## Navigation
-
-- [Complete Documentation](./parquet-readme.md) - Full guide
-- [Quick Reference](./parquet-quick-reference.md) - Cheat sheet
-- [Modules](./modules/) - Progressive learning modules
-- [Examples](./examples/) - Code examples
-
-## Learning Path
-
-| Module | Topic | Level |
-|--------|-------|-------|
-| [Module 1](./modules/03-pattern-guides-parquet-01-introduction.md) | Introduction | Beginner |
-| [Module 2](./modules/03-pattern-guides-parquet-02-quick-start.md) | Quick Start | Beginner |
-| [Module 3](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | Reading Parquet | Intermediate |
-| [Module 4](./modules/03-pattern-guides-parquet-04-writing-parquet.md) | Writing Parquet | Intermediate |
-| [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | GraphQL Extraction | Advanced |
-| [Module 6](./modules/03-pattern-guides-parquet-06-s3-integration.md) | S3 Integration | Advanced |
-| [Module 7](./modules/03-pattern-guides-parquet-07-performance-optimization.md) | Performance Optimization | Advanced |
-| [Module 8](./modules/03-pattern-guides-parquet-08-best-practices.md) | Best Practices | All Levels |
-
-See [parquet-readme.md](./parquet-readme.md) for complete details.
+# Parquet Processing Guide
+
+> **Parquet file processing patterns**
+> Read and write Parquet files efficiently
+
+For complete documentation, see [parquet-readme.md](./parquet-readme.md)
+
+## Overview
+
+This guide covers Parquet file processing for analytics and data lake integrations, including reading, writing, partitioning, and S3 integration.
+
+## Navigation
+
+- [Complete Documentation](./parquet-readme.md) - Full guide
+- [Quick Reference](./parquet-quick-reference.md) - Cheat sheet
+- [Modules](./modules/) - Progressive learning modules
+- [Examples](./examples/) - Code examples
+
+## Learning Path
+
+| Module | Topic | Level |
+|--------|-------|-------|
+| [Module 1](./modules/03-pattern-guides-parquet-01-introduction.md) | Introduction | Beginner |
+| [Module 2](./modules/03-pattern-guides-parquet-02-quick-start.md) | Quick Start | Beginner |
+| [Module 3](./modules/03-pattern-guides-parquet-03-reading-parquet.md) | Reading Parquet | Intermediate |
+| [Module 4](./modules/03-pattern-guides-parquet-04-writing-parquet.md) | Writing Parquet | Intermediate |
+| [Module 5](./modules/03-pattern-guides-parquet-05-graphql-extraction.md) | GraphQL Extraction | Advanced |
+| [Module 6](./modules/03-pattern-guides-parquet-06-s3-integration.md) | S3 Integration | Advanced |
+| [Module 7](./modules/03-pattern-guides-parquet-07-performance-optimization.md) | Performance Optimization | Advanced |
+| [Module 8](./modules/03-pattern-guides-parquet-08-best-practices.md) | Best Practices | All Levels |
+
+See [parquet-readme.md](./parquet-readme.md) for complete details.

package/docs/03-PATTERN-GUIDES/parsers/parsers-readme.md

@@ -1,12 +1,12 @@
-# Parser Patterns
-
-Parser documentation has been moved to:
-
-**📍 [Parsers Guide](../../02-CORE-GUIDES/parsers/parsers-readme.md)**
-
-This guide covers:
-- CSV parsing
-- XML parsing
-- JSON parsing
-- Parquet parsing
-- Streaming patterns
+# Parser Patterns
+
+Parser documentation has been moved to:
+
+**📍 [Parsers Guide](../../02-CORE-GUIDES/parsers/parsers-readme.md)**
+
+This guide covers:
+- CSV parsing
+- XML parsing
+- JSON parsing
+- Parquet parsing
+- Streaming patterns

package/docs/03-PATTERN-GUIDES/parsers/readme.md

@@ -1,24 +1,24 @@
-# Parser Patterns
-
-> **Parser implementation patterns**
-> Best practices for CSV, XML, JSON, Parquet parsing
-
-For complete documentation, see [parsers-readme.md](./parsers-readme.md)
-
-## Overview
-
-This guide provides parser implementation patterns for various data formats. Includes error handling, streaming, and performance optimization.
-
-## Quick Start
-
-See [Parsers Guide](../../02-CORE-GUIDES/parsers/) for complete parser documentation.
-
-This pattern guide focuses on advanced parser patterns and custom implementations.
-
-## Related Documentation
-
-- [Parsers Guide](../../02-CORE-GUIDES/parsers/) - Core parser documentation
-- [File Operations](../file-operations/) - File handling patterns
-- [Error Handling](../error-handling/) - Parser error handling
-
-See [parsers-readme.md](./parsers-readme.md) for complete details.
+# Parser Patterns
+
+> **Parser implementation patterns**
+> Best practices for CSV, XML, JSON, Parquet parsing
+
+For complete documentation, see [parsers-readme.md](./parsers-readme.md)
+
+## Overview
+
+This guide provides parser implementation patterns for various data formats. Includes error handling, streaming, and performance optimization.
+
+## Quick Start
+
+See [Parsers Guide](../../02-CORE-GUIDES/parsers/) for complete parser documentation.
+
+This pattern guide focuses on advanced parser patterns and custom implementations.
+
+## Related Documentation
+
+- [Parsers Guide](../../02-CORE-GUIDES/parsers/) - Core parser documentation
+- [File Operations](../file-operations/) - File handling patterns
+- [Error Handling](../error-handling/) - Parser error handling
+
+See [parsers-readme.md](./parsers-readme.md) for complete details.