@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

package/docs/02-CORE-GUIDES/auto-pagination/modules/auto-pagination-06-troubleshooting.md

@@ -1,359 +1,359 @@
-# Module 6: Troubleshooting & Optimization
-
-> **Learning Objective:** Diagnose and fix common pagination issues, optimize performance, and handle edge cases.
->
-> **Level:** Intermediate
-
-## Common Errors & Solutions
-
-### Error 1: "Cannot read property 'edges' of undefined"
-
-**Cause:** Wrong `connectionPath` or connection doesn't exist in response.
-
-```typescript
-// ❌ ERROR
-pagination: {
-  connectionPath: 'order.items';
-}
-// Query returns: { data: { orderById: { items: {...} } } }
-
-// ✅ FIX: Match actual response structure
-pagination: {
-  connectionPath: 'orderById.items';
-}
-```
-
-### Error 2: "Auto-pagination not activating"
-
-**Cause:** Missing pagination variables in query.
-
-```typescript
-// ❌ WRONG: No pagination variables
-query GetItems {
-  items(first: 100) { ... }
-}
-
-// ✅ CORRECT: Include $first and $after
-query GetItems($first: Int!, $after: String) {
-  items(first: $first, after: $after) { ... }
-}
-```
-
-### Error 3: Memory exhaustion (OOM)
-
-**Cause:** Fetching too many records at once.
-
-```typescript
-// ❌ RISKY: 100,000+ records in memory
-pagination: {
-  maxRecords: 100000;
-}
-
-// ✅ FIX: Use chunked processing
-pagination: {
-  maxRecords: 5000;
-}
-// Process in multiple batches instead
-```
-
-### Error 4: "Timeout waiting for response"
-
-**Cause:** Query too complex or page size too large.
-
-```typescript
-// ❌ TOO COMPLEX
-variables: {
-  first: 250;
-} // Max page size with nested data
-
-// ✅ FIX: Reduce page size for complex queries
-variables: {
-  first: 20;
-}
-pagination: {
-  delayMs: 100;
-} // Add delays
-```
-
-### Error 5: "Field 'X' doesn't exist on type 'Y'"
-
-**Cause:** Using wrong field names (not from schema).
-
-```typescript
-// ❌ WRONG: Guessed field
-node { name }  // 'name' doesn't exist on VirtualPosition
-
-// ✅ FIX: Use schema introspection
-// Run: npx fc-connect introspect-schema
-node { productRef }  // Real field
-```
-
-## Performance Optimization
-
-### Optimize Page Size
-
-```typescript
-// Match page size to entity complexity
-const configs = {
-  simple: { first: 150, maxRecords: 15000 }, // virtualPositions, locations
-  medium: { first: 75, maxRecords: 5000 }, // products, categories
-  complex: { first: 20, maxRecords: 1000 }, // orders with items
-};
-```
-
-### Use Progress Monitoring
-
-```typescript
-pagination: {
-  onProgress: (page, records) => {
-    if (page % 10 === 0) {
-      console.log(`Checkpoint: ${records} records fetched`);
-    }
-  };
-}
-```
-
-### Implement Abort on Errors
-
-```typescript
-const controller = new AbortController();
-
-setTimeout(() => controller.abort(), 60000); // 1 minute timeout
-
-try {
-  const result = await client.graphql({
-    query,
-    variables,
-    pagination: { abortSignal: controller.signal },
-  });
-} catch (error) {
-  if (error.name === 'AbortError') {
-    console.log('Pagination aborted due to timeout');
-  }
-}
-```
-
-## Edge Cases
-
-### Empty Result Sets
-
-```typescript
-const result = await client.graphql({
-  query,
-  variables: { first: 100, productRef: 'NONEXISTENT' },
-});
-
-// Handle empty results
-if (result.data.virtualPositions.edges.length === 0) {
-  console.log('No records found');
-} else {
-  console.log(`Found ${result.extensions.autoPagination.totalRecords} records`);
-}
-```
-
-### Handling Duplicates
-
-SDK automatically deduplicates by node.id, but if needed:
-
-```typescript
-const result = await client.graphql({ query, variables, pagination });
-const edges = result.data.items.edges;
-
-// Additional deduplication by custom field
-const uniqueByRef = Array.from(new Map(edges.map(e => [e.node.ref, e])).values());
-```
-
-### Partial Results (Truncation)
-
-```typescript
-const result = await client.graphql({
-  query,
-  variables,
-  pagination: { maxRecords: 5000 },
-});
-
-if (result.extensions.autoPagination.truncated) {
-  console.warn(`⚠️ Results truncated at ${result.extensions.autoPagination.totalRecords} records`);
-  console.warn(`Reason: ${result.extensions.autoPagination.truncationReason}`);
-
-  // Handle truncation
-  if (result.extensions.autoPagination.truncationReason === 'maxRecords') {
-    console.log('Consider increasing maxRecords or processing in batches');
-  }
-}
-```
-
-## When NOT to Use Auto-Pagination
-
-### Scenario 1: Extremely Large Datasets (50K+ records)
-
-```typescript
-// ❌ DON'T: Auto-pagination for 100K records
-// This loads all 100K into memory at once
-
-// ✅ DO: Manual pagination with chunked processing
-let cursor = null;
-let processedCount = 0;
-
-while (true) {
-  const result = await client.graphql({
-    query,
-    variables: { first: 1000, after: cursor },
-    pagination: { enabled: false }, // Disable auto-pagination
-  });
-
-  const edges = result.data.items.edges;
-  if (edges.length === 0) break;
-
-  // Process this chunk
-  await processChunk(edges);
-  processedCount += edges.length;
-
-  // Get cursor for next page
-  cursor = edges[edges.length - 1].cursor;
-
-  if (!result.data.items.pageInfo.hasNextPage) break;
-}
-```
-
-### Scenario 2: Streaming ETL Pipelines
-
-```typescript
-// ❌ DON'T: Auto-pagination when streaming to destination
-// Would fetch ALL pages before starting writes
-
-// ✅ DO: Manual pagination with streaming writes
-let cursor = null;
-
-while (true) {
-  const page = await client.graphql({
-    query,
-    variables: { first: 500, after: cursor },
-    pagination: { enabled: false },
-  });
-
-  const edges = page.data.items.edges;
-  if (edges.length === 0) break;
-
-  // Write this page immediately
-  await writeToDestination(edges);
-
-  cursor = edges[edges.length - 1].cursor;
-  if (!page.data.items.pageInfo.hasNextPage) break;
-}
-```
-
-### Scenario 3: Fine-Grained Error Recovery
-
-```typescript
-// ✅ Manual pagination for per-page error handling
-let cursor = null;
-let pageNumber = 0;
-
-while (true) {
-  try {
-    const page = await client.graphql({
-      query,
-      variables: { first: 100, after: cursor },
-      pagination: { enabled: false },
-    });
-
-    const edges = page.data.items.edges;
-    if (edges.length === 0) break;
-
-    await processPage(edges);
-    pageNumber++;
-
-    cursor = edges[edges.length - 1].cursor;
-    if (!page.data.items.pageInfo.hasNextPage) break;
-  } catch (error) {
-    console.error(`Error on page ${pageNumber}:`, error);
-    // Retry this specific page or skip it
-    await delay(5000); // Wait before retry
-  }
-}
-```
-
-## Debugging Tips
-
-### Enable Debug Logging
-
-```typescript
-const client = await createClient({
-  config: {
-    baseUrl: process.env.FLUENT_BASE_URL!,
-    clientId: process.env.FLUENT_CLIENT_ID!,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
-    retailerId: process.env.FLUENT_RETAILER_ID!,
-  },
-  logger: console, // Enable debug logging
-});
-
-// Logger will output pagination progress automatically
-```
-
-### Check Pagination Metadata
-
-```typescript
-const result = await client.graphql({ query, variables, pagination });
-
-console.log('Pagination Stats:', {
-  totalPages: result.extensions.autoPagination.totalPages,
-  totalRecords: result.extensions.autoPagination.totalRecords,
-  truncated: result.extensions.autoPagination.truncated,
-  truncationReason: result.extensions.autoPagination.truncationReason,
-  direction: result.extensions.autoPagination.direction,
-});
-
-// Calculate duration yourself if needed:
-// const start = Date.now();
-// const result = await client.graphql(...);
-// const duration = Date.now() - start;
-```
-
-### Test with Small Limits First
-
-```typescript
-// Start with very small limits to test quickly
-pagination: {
-  maxPages: 2,
-  maxRecords: 200
-}
-
-// Then scale up once working:
-pagination: {
-  maxPages: 100,
-  maxRecords: 10000
-}
-```
-
-## Performance Benchmarks
-
-Typical performance by entity type (approximate):
-
-| Entity Type         | Records/Second | Page Size | Notes             |
-| ------------------- | -------------- | --------- | ----------------- |
-| virtualPositions    | 500-800        | 150       | Simple structure  |
-| locations           | 600-900        | 100       | Simple structure  |
-| products            | 200-400        | 75        | Medium complexity |
-| orders (with items) | 50-150         | 20        | Complex nested    |
-
-Factors affecting speed:
-
-- Network latency
-- Query complexity
-- Field count
-- Nested connections
-- API server load
-
-## Key Takeaways
-
-✅ **Common errors**: Wrong connectionPath, missing variables, memory issues
-✅ **Optimization**: Match page size to entity complexity
-✅ **Edge cases**: Handle empty results, duplicates, truncation
-✅ **When to avoid**: Very large datasets, streaming, fine-grained error handling
-✅ **Debugging**: Use logging, check metadata, test with small limits first
-
----
-
-[← Back to Index](../../advanced-services/advanced-services-readme.md) | [← Previous: SDK Integration](./auto-pagination-05-sdk-integration.md) | [Next: API Reference →](./auto-pagination-07-api-reference.md)
+# Module 6: Troubleshooting & Optimization
+
+> **Learning Objective:** Diagnose and fix common pagination issues, optimize performance, and handle edge cases.
+>
+> **Level:** Intermediate
+
+## Common Errors & Solutions
+
+### Error 1: "Cannot read property 'edges' of undefined"
+
+**Cause:** Wrong `connectionPath` or connection doesn't exist in response.
+
+```typescript
+// ❌ ERROR
+pagination: {
+  connectionPath: 'order.items';
+}
+// Query returns: { data: { orderById: { items: {...} } } }
+
+// ✅ FIX: Match actual response structure
+pagination: {
+  connectionPath: 'orderById.items';
+}
+```
+
+### Error 2: "Auto-pagination not activating"
+
+**Cause:** Missing pagination variables in query.
+
+```typescript
+// ❌ WRONG: No pagination variables
+query GetItems {
+  items(first: 100) { ... }
+}
+
+// ✅ CORRECT: Include $first and $after
+query GetItems($first: Int!, $after: String) {
+  items(first: $first, after: $after) { ... }
+}
+```
+
+### Error 3: Memory exhaustion (OOM)
+
+**Cause:** Fetching too many records at once.
+
+```typescript
+// ❌ RISKY: 100,000+ records in memory
+pagination: {
+  maxRecords: 100000;
+}
+
+// ✅ FIX: Use chunked processing
+pagination: {
+  maxRecords: 5000;
+}
+// Process in multiple batches instead
+```
+
+### Error 4: "Timeout waiting for response"
+
+**Cause:** Query too complex or page size too large.
+
+```typescript
+// ❌ TOO COMPLEX
+variables: {
+  first: 250;
+} // Max page size with nested data
+
+// ✅ FIX: Reduce page size for complex queries
+variables: {
+  first: 20;
+}
+pagination: {
+  delayMs: 100;
+} // Add delays
+```
+
+### Error 5: "Field 'X' doesn't exist on type 'Y'"
+
+**Cause:** Using wrong field names (not from schema).
+
+```typescript
+// ❌ WRONG: Guessed field
+node { name }  // 'name' doesn't exist on VirtualPosition
+
+// ✅ FIX: Use schema introspection
+// Run: npx fc-connect introspect-schema
+node { productRef }  // Real field
+```
+
+## Performance Optimization
+
+### Optimize Page Size
+
+```typescript
+// Match page size to entity complexity
+const configs = {
+  simple: { first: 150, maxRecords: 15000 }, // virtualPositions, locations
+  medium: { first: 75, maxRecords: 5000 }, // products, categories
+  complex: { first: 20, maxRecords: 1000 }, // orders with items
+};
+```
+
+### Use Progress Monitoring
+
+```typescript
+pagination: {
+  onProgress: (page, records) => {
+    if (page % 10 === 0) {
+      console.log(`Checkpoint: ${records} records fetched`);
+    }
+  };
+}
+```
+
+### Implement Abort on Errors
+
+```typescript
+const controller = new AbortController();
+
+setTimeout(() => controller.abort(), 60000); // 1 minute timeout
+
+try {
+  const result = await client.graphql({
+    query,
+    variables,
+    pagination: { abortSignal: controller.signal },
+  });
+} catch (error) {
+  if (error.name === 'AbortError') {
+    console.log('Pagination aborted due to timeout');
+  }
+}
+```
+
+## Edge Cases
+
+### Empty Result Sets
+
+```typescript
+const result = await client.graphql({
+  query,
+  variables: { first: 100, productRef: 'NONEXISTENT' },
+});
+
+// Handle empty results
+if (result.data.virtualPositions.edges.length === 0) {
+  console.log('No records found');
+} else {
+  console.log(`Found ${result.extensions.autoPagination.totalRecords} records`);
+}
+```
+
+### Handling Duplicates
+
+SDK automatically deduplicates by node.id, but if needed:
+
+```typescript
+const result = await client.graphql({ query, variables, pagination });
+const edges = result.data.items.edges;
+
+// Additional deduplication by custom field
+const uniqueByRef = Array.from(new Map(edges.map(e => [e.node.ref, e])).values());
+```
+
+### Partial Results (Truncation)
+
+```typescript
+const result = await client.graphql({
+  query,
+  variables,
+  pagination: { maxRecords: 5000 },
+});
+
+if (result.extensions.autoPagination.truncated) {
+  console.warn(`⚠️ Results truncated at ${result.extensions.autoPagination.totalRecords} records`);
+  console.warn(`Reason: ${result.extensions.autoPagination.truncationReason}`);
+
+  // Handle truncation
+  if (result.extensions.autoPagination.truncationReason === 'maxRecords') {
+    console.log('Consider increasing maxRecords or processing in batches');
+  }
+}
+```
+
+## When NOT to Use Auto-Pagination
+
+### Scenario 1: Extremely Large Datasets (50K+ records)
+
+```typescript
+// ❌ DON'T: Auto-pagination for 100K records
+// This loads all 100K into memory at once
+
+// ✅ DO: Manual pagination with chunked processing
+let cursor = null;
+let processedCount = 0;
+
+while (true) {
+  const result = await client.graphql({
+    query,
+    variables: { first: 1000, after: cursor },
+    pagination: { enabled: false }, // Disable auto-pagination
+  });
+
+  const edges = result.data.items.edges;
+  if (edges.length === 0) break;
+
+  // Process this chunk
+  await processChunk(edges);
+  processedCount += edges.length;
+
+  // Get cursor for next page
+  cursor = edges[edges.length - 1].cursor;
+
+  if (!result.data.items.pageInfo.hasNextPage) break;
+}
+```
+
+### Scenario 2: Streaming ETL Pipelines
+
+```typescript
+// ❌ DON'T: Auto-pagination when streaming to destination
+// Would fetch ALL pages before starting writes
+
+// ✅ DO: Manual pagination with streaming writes
+let cursor = null;
+
+while (true) {
+  const page = await client.graphql({
+    query,
+    variables: { first: 500, after: cursor },
+    pagination: { enabled: false },
+  });
+
+  const edges = page.data.items.edges;
+  if (edges.length === 0) break;
+
+  // Write this page immediately
+  await writeToDestination(edges);
+
+  cursor = edges[edges.length - 1].cursor;
+  if (!page.data.items.pageInfo.hasNextPage) break;
+}
+```
+
+### Scenario 3: Fine-Grained Error Recovery
+
+```typescript
+// ✅ Manual pagination for per-page error handling
+let cursor = null;
+let pageNumber = 0;
+
+while (true) {
+  try {
+    const page = await client.graphql({
+      query,
+      variables: { first: 100, after: cursor },
+      pagination: { enabled: false },
+    });
+
+    const edges = page.data.items.edges;
+    if (edges.length === 0) break;
+
+    await processPage(edges);
+    pageNumber++;
+
+    cursor = edges[edges.length - 1].cursor;
+    if (!page.data.items.pageInfo.hasNextPage) break;
+  } catch (error) {
+    console.error(`Error on page ${pageNumber}:`, error);
+    // Retry this specific page or skip it
+    await delay(5000); // Wait before retry
+  }
+}
+```
+
+## Debugging Tips
+
+### Enable Debug Logging
+
+```typescript
+const client = await createClient({
+  config: {
+    baseUrl: process.env.FLUENT_BASE_URL!,
+    clientId: process.env.FLUENT_CLIENT_ID!,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
+    retailerId: process.env.FLUENT_RETAILER_ID!,
+  },
+  logger: console, // Enable debug logging
+});
+
+// Logger will output pagination progress automatically
+```
+
+### Check Pagination Metadata
+
+```typescript
+const result = await client.graphql({ query, variables, pagination });
+
+console.log('Pagination Stats:', {
+  totalPages: result.extensions.autoPagination.totalPages,
+  totalRecords: result.extensions.autoPagination.totalRecords,
+  truncated: result.extensions.autoPagination.truncated,
+  truncationReason: result.extensions.autoPagination.truncationReason,
+  direction: result.extensions.autoPagination.direction,
+});
+
+// Calculate duration yourself if needed:
+// const start = Date.now();
+// const result = await client.graphql(...);
+// const duration = Date.now() - start;
+```
+
+### Test with Small Limits First
+
+```typescript
+// Start with very small limits to test quickly
+pagination: {
+  maxPages: 2,
+  maxRecords: 200
+}
+
+// Then scale up once working:
+pagination: {
+  maxPages: 100,
+  maxRecords: 10000
+}
+```
+
+## Performance Benchmarks
+
+Typical performance by entity type (approximate):
+
+| Entity Type         | Records/Second | Page Size | Notes             |
+| ------------------- | -------------- | --------- | ----------------- |
+| virtualPositions    | 500-800        | 150       | Simple structure  |
+| locations           | 600-900        | 100       | Simple structure  |
+| products            | 200-400        | 75        | Medium complexity |
+| orders (with items) | 50-150         | 20        | Complex nested    |
+
+Factors affecting speed:
+
+- Network latency
+- Query complexity
+- Field count
+- Nested connections
+- API server load
+
+## Key Takeaways
+
+✅ **Common errors**: Wrong connectionPath, missing variables, memory issues
+✅ **Optimization**: Match page size to entity complexity
+✅ **Edge cases**: Handle empty results, duplicates, truncation
+✅ **When to avoid**: Very large datasets, streaming, fine-grained error handling
+✅ **Debugging**: Use logging, check metadata, test with small limits first
+
+---
+
+[← Back to Index](../../advanced-services/advanced-services-readme.md) | [← Previous: SDK Integration](./auto-pagination-05-sdk-integration.md) | [Next: API Reference →](./auto-pagination-07-api-reference.md)