npm - @fluentcommerce/fc-connect-sdk - Versions diffs - 0.1.54 → 0.1.55 - Mend

@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (475) hide show

package/docs/03-PATTERN-GUIDES/connector-scenarios/connector-scenarios-quick-reference.md CHANGED Viewed

@@ -1,498 +1,498 @@
-# Connector Scenarios Quick Reference
-> **One-page cheat sheet for building connectors with the FC Connect SDK**
-## Connector Architecture
-```
-Data Source → Parser → Validator → Mapper → Fluent Client → State Tracker
-   (SFTP/S3)  (CSV/JSON) (Schema)  (Transform)  (Batch/GraphQL)  (Dedup)
-```
-## Common Imports
-```typescript
-import {
-  createClient,
-  SftpDataSource,
-  S3DataSource,
-  CSVParserService,
-  ParquetParserService,
-  JSONParserService,
-  XMLParserService,
-  UniversalMapper,
-  SchemaValidationService,
-  StateService,
-  BatchAPIClient,
-  ExtractionOrchestrator,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-```
-## Quick Start Template
-```typescript
-// 1. Initialize services
-const logger = toStructuredLogger(createConsoleLogger(), {
-  service: 'connector-name',
-  module: 'workflow-name'
-});
-// StateService constructor takes only logger (optional)
-const stateService = new StateService(logger);
-// 2. Create data source
-const dataSource = new SftpDataSource(
-  {
-    type: 'SFTP_CSV',
-    connectionId: 'my-source',
-    name: 'My Source',
-    settings: { host, username, password, remotePath },
-  },
-  logger
-);
-// 3. Create field mapper
-const mapper = new UniversalMapper({
-  fields: {
-    ref: { source: 'sku', resolver: 'sdk.uppercase' },
-    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
-  },
-});
-// 4. Create Fluent client
-const client = await createClient({
-  config: {
-    baseUrl: process.env.FLUENT_BASE_URL!,
-    clientId: process.env.FLUENT_CLIENT_ID!,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
-    retailerId: process.env.FLUENT_RETAILER_ID!,
-  },
-});
-// 5. Process files
-const files = await dataSource.listFiles({ filePattern: '*.csv' });
-for (const file of files) {
-  const content = await dataSource.downloadFile(file.name);
-  const parser = new CSVParserService();
-  const records = await parser.parse(content as string);
-  const transformed = await mapper.mapBatch(records);
-  await client.sendBatch(jobId, { entities: transformed });
-}
-```
-## Data Source Configurations
-### SFTP CSV
-```typescript
-new SftpDataSource(
-  {
-    type: 'SFTP_CSV',
-    connectionId: 'sftp-csv',
-    name: 'SFTP CSV Source',
-    settings: {
-      host: 'sftp.example.com',
-      port: 22,
-      username: 'user',
-      password: 'pass',
-      remotePath: '/data',
-      filePattern: '*.csv',
-      csvDelimiter: ',',
-      csvHeaders: true,
-      csvSkipEmptyLines: true,
-    },
-  },
-  logger
-);
-```
-### S3 Parquet
-```typescript
-new S3DataSource(
-  {
-    type: 'S3_PARQUET',
-    connectionId: 's3-parquet',
-    name: 'S3 Parquet Source',
-    settings: {
-      bucketName: 'my-bucket',
-      region: 'us-east-1',
-      accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
-      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
-      prefix: 'data/',
-      filePattern: '*.parquet',
-    },
-  },
-  logger
-);
-```
-### SFTP JSON
-```typescript
-new SftpDataSource(
-  {
-    type: 'SFTP_JSON',
-    connectionId: 'sftp-json',
-    name: 'SFTP JSON Source',
-    settings: {
-      host: 'sftp.example.com',
-      username: 'user',
-      password: 'pass',
-      remotePath: '/orders',
-      filePattern: 'ORDER_*.json',
-      jsonValidate: true,
-    },
-  },
-  logger
-);
-```
-## Field Mapping Patterns
-### Basic Mapping
-```typescript
-const mapper = new UniversalMapper({
-  fields: {
-    ref: { source: 'sku' },
-    locationRef: { source: 'warehouse_code' },
-    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
-    status: { source: 'status', resolver: 'sdk.uppercase' },
-  },
-});
-```
-### Custom Resolver
-```typescript
-{
-  status: {
-    source: 'status_code',
-    resolver: (value) => {
-      const map = { '1': 'ACTIVE', '0': 'INACTIVE' };
-      return map[value] || 'INACTIVE';
-    }
-  }
-}
-```
-### Nested Field Mapping
-```typescript
-{
-  'attributes.weight': {
-    source: 'weight_kg',
-    resolver: (value) => ({
-      value: parseFloat(value),
-      unit: 'kg'
-    })
-  }
-}
-```
-### Multi-Source Mapping
-```typescript
-{
-  availableQty: {
-    source: ['on_hand', 'allocated', 'damaged'],
-    resolver: ([onHand, allocated, damaged]) => {
-      return Number(onHand) - Number(allocated) - Number(damaged);
-    }
-  }
-}
-```
-## Built-in Resolvers
-| Resolver         | Purpose              | Example                     |
-| ---------------- | -------------------- | --------------------------- |
-| `sdk.uppercase`  | Convert to uppercase | `'abc'` → `'ABC'`           |
-| `sdk.lowercase`  | Convert to lowercase | `'ABC'` → `'abc'`           |
-| `sdk.trim`       | Remove whitespace    | `' abc '` → `'abc'`         |
-| `sdk.parseInt`   | Parse as integer     | `'123'` → `123`             |
-| `sdk.parseFloat` | Parse as float       | `'12.34'` → `12.34`         |
-| `sdk.formatDate` | Format ISO date      | `'2025-01-15'` → ISO string |
-| `sdk.boolean`    | Parse as boolean     | `'true'` → `true`           |
-| `sdk.toJson`     | Stringify JSON       | `{a:1}` → `'{"a":1}'`       |
-| `sdk.parseJson`  | Parse JSON string    | `'{"a":1}'` → `{a:1}`       |
-## Validation Patterns
-### Required Fields
-```typescript
-if (!record.sku || !record.quantity) {
-  return { valid: false, error: 'Missing required fields' };
-}
-```
-### Type Validation
-```typescript
-if (isNaN(Number(record.quantity))) {
-  return { valid: false, error: 'Quantity must be a number' };
-}
-```
-### Format Validation
-```typescript
-if (!/^[A-Z0-9-]+$/i.test(record.sku)) {
-  return { valid: false, error: 'Invalid SKU format' };
-}
-```
-### Schema Validation
-```typescript
-// SchemaValidationService validates GraphQL payloads against Fluent schema
-const validator = new SchemaValidationService(client);
-// Fetch and validate mutation input against schema
-const result = await validator.validateMutation(
-  'updateInventoryQuantity',
-  'UpdateInventoryQuantityInput',
-  payload
-);
-if (!result.valid) {
-  console.error('Validation errors:', result.errors);
-}
-```
-## Sending Data to Fluent
-### Batch API (Inventory Only)
-```typescript
-const job = await client.createJob({
-  name: 'Inventory Update',
-  retailerId: process.env.FLUENT_RETAILER_ID!,
-});
-await client.sendBatch(job.id, {
-  action: 'UPSERT',
-  entityType: 'INVENTORY',
-  entities: records.map(r => ({
-    ref: r.ref,
-    productRef: r.ref,
-    locationRef: r.locationRef,
-    qty: r.qty,
-    status: r.status,
-  })),
-});
-```
-### GraphQL Mutation (All Entity Types)
-```typescript
-const mutation = `
-  mutation CreateOrder($input: CreateOrderInput!) {
-    createOrder(input: $input) {
-      id
-      ref
-      status
-    }
-  }
-`;
-const result = await client.graphql({ query: mutation, variables: { input: orderData } });
-```
-## State Management
-⚠️ **IMPORTANT:** StateService requires KV adapter passed to EVERY method call. The constructor takes only logger (optional).
-### File-Level Tracking
-```typescript
-// StateService constructor takes only logger (optional)
-const stateService = new StateService(logger);
-// KV adapter is required for ALL state operations
-const kvAdapter = new VersoriKVAdapter(openKv());
-const workflowId = 'my-workflow';
-// KV adapter MUST be passed to every method
-if (await stateService.isFileProcessed(kvAdapter, file.name, workflowId)) {
-  return; // Skip
-}
-// Process file...
-// KV adapter MUST be passed to every method
-await stateService.markFileProcessed(kvAdapter, file.name, {
-  recordsProcessed: 1000,
-}, workflowId);
-```
-### Record-Level Checkpoints
-```typescript
-// Use KV directly for checkpoints (not StateService)
-const kv = openKv();
-// Save checkpoint
-await kv.set(['checkpoint', 'job-123'], {
-  file: currentFile,
-  lastRecordId: record.id,
-  recordsProcessed: count,
-  timestamp: new Date().toISOString(),
-});
-// Resume from checkpoint
-const checkpoint = await kv.get(['checkpoint', 'job-123']);
-if (checkpoint?.value) {
-  startFrom = checkpoint.value.lastRecordId;
-}
-```
-## Error Handling
-### Three-Level Strategy
-```typescript
-// 1. Record-level: Skip invalid records
-const valid = records.filter(r => validate(r));
-// 2. File-level: Quarantine failed files
-try {
-  await processFile(file);
-} catch (error) {
-  await moveToErrorFolder(file);
-}
-// 3. Job-level: Fail fast on critical errors
-try {
-  await runJob();
-} catch (error) {
-  await notifyAdmins(error);
-  throw error;
-}
-```
-### Retry with Backoff
-```typescript
-async function retryWithBackoff(fn, maxRetries = 3) {
-  for (let i = 0; i < maxRetries; i++) {
-    try {
-      return await fn();
-    } catch (error) {
-      if (i === maxRetries - 1) throw error;
-      await sleep(1000 * Math.pow(2, i));
-    }
-  }
-}
-```
-## Extraction from Fluent
-### Query with Auto-Pagination
-```typescript
-const query = `
-  query GetOrders($retailerId: ID!, $first: Int!, $after: String) {
-    orders(retailerId: $retailerId, first: $first, after: $after) {
-      edges {
-        node { id ref status }
-        cursor
-      }
-      pageInfo { hasNextPage }
-    }
-  }
-`;
-const result = await client.graphql({
-  query,
-  variables: { retailerId, first: 100 },
-  pagination: { maxPages: 50 },
-});
-const orders = result.data.orders.edges.map(e => e.node);
-```
-## File Operations
-### Download File
-```typescript
-const content = await dataSource.downloadFile('inventory.csv');
-```
-### Upload File
-```typescript
-await dataSource.uploadFile('export.json', jsonContent);
-```
-### Move/Archive File
-```typescript
-await dataSource.moveFile('inventory.csv', 'archive/2025-01-15/inventory.csv');
-```
-### Stream Large File
-```typescript
-const stream = await s3Source.getFileStream('large.csv');
-stream.pipe(csvParser.createParseStream()).on('data', record => {
-  /* process */
-});
-```
-## Performance Tips
-| File Size        | Batch Size | Streaming | Checkpoints |
-| ---------------- | ---------- | --------- | ----------- |
-| < 1K records     | 500        | No        | Optional    |
-| 1K - 10K records | 200-500    | Optional  | Recommended |
-| > 10K records    | 100-200    | Yes       | Required    |
-## Environment Variables Template
-```bash
-# Fluent Commerce
-FLUENT_BASE_URL=https://api.fluentcommerce.com
-FLUENT_CLIENT_ID=client-id
-FLUENT_CLIENT_SECRET=***
-FLUENT_RETAILER_ID=1
-# SFTP
-SFTP_HOST=sftp.example.com
-SFTP_USERNAME=user
-SFTP_PASSWORD=***
-# S3
-S3_BUCKET=bucket-name
-AWS_REGION=us-east-1
-AWS_ACCESS_KEY_ID=***
-AWS_SECRET_ACCESS_KEY=***
-# Settings
-LOG_LEVEL=info
-BATCH_SIZE=100
-```
-## Common Scenarios Summary
-| Scenario                 | Source  | Format   | Components                                  | Complexity |
-| ------------------------ | ------- | -------- | ------------------------------------------- | ---------- |
-| Inventory Update         | SFTP    | CSV      | SftpDataSource, CSVParserService, Mapper    | Low        |
-| Product Sync             | S3      | Parquet  | S3DataSource, ParquetParserService, Mapper  | Medium     |
-| Order Creation           | SFTP    | JSON     | SftpDataSource, JSONParserService, Validator| Medium     |
-| Large File Processing    | S3      | CSV      | S3DataSource, StateService, Streaming       | High       |
-| Multi-Source Aggregation | SFTP+S3 | Multiple | Multiple sources, Reconciliation            | High       |
-| Data Export              | Fluent  | JSON/CSV | FluentClient, ExtractionOrchestrator        | Medium     |
-## Quick Links
-- [Full Guide](./connector-scenarios-readme.md)
-- [Module 1: Foundations](./modules/connector-scenarios-01-foundations.md)
-- [Module 2: Simple Scenarios](./modules/connector-scenarios-02-simple-scenarios.md)
-- [Module 7: Reference](./modules/connector-scenarios-07-reference.md)
-- [Examples](./examples/)
+# Connector Scenarios Quick Reference
+> **One-page cheat sheet for building connectors with the FC Connect SDK**
+## Connector Architecture
+```
+Data Source → Parser → Validator → Mapper → Fluent Client → State Tracker
+   (SFTP/S3)  (CSV/JSON) (Schema)  (Transform)  (Batch/GraphQL)  (Dedup)
+```
+## Common Imports
+```typescript
+import {
+  createClient,
+  SftpDataSource,
+  S3DataSource,
+  CSVParserService,
+  ParquetParserService,
+  JSONParserService,
+  XMLParserService,
+  UniversalMapper,
+  SchemaValidationService,
+  StateService,
+  BatchAPIClient,
+  ExtractionOrchestrator,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+```
+## Quick Start Template
+```typescript
+// 1. Initialize services
+const logger = toStructuredLogger(createConsoleLogger(), {
+  service: 'connector-name',
+  module: 'workflow-name'
+});
+// StateService constructor takes only logger (optional)
+const stateService = new StateService(logger);
+// 2. Create data source
+const dataSource = new SftpDataSource(
+  {
+    type: 'SFTP_CSV',
+    connectionId: 'my-source',
+    name: 'My Source',
+    settings: { host, username, password, remotePath },
+  },
+  logger
+);
+// 3. Create field mapper
+const mapper = new UniversalMapper({
+  fields: {
+    ref: { source: 'sku', resolver: 'sdk.uppercase' },
+    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
+  },
+});
+// 4. Create Fluent client
+const client = await createClient({
+  config: {
+    baseUrl: process.env.FLUENT_BASE_URL!,
+    clientId: process.env.FLUENT_CLIENT_ID!,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
+    retailerId: process.env.FLUENT_RETAILER_ID!,
+  },
+});
+// 5. Process files
+const files = await dataSource.listFiles({ filePattern: '*.csv' });
+for (const file of files) {
+  const content = await dataSource.downloadFile(file.name);
+  const parser = new CSVParserService();
+  const records = await parser.parse(content as string);
+  const transformed = await mapper.mapBatch(records);
+  await client.sendBatch(jobId, { entities: transformed });
+}
+```
+## Data Source Configurations
+### SFTP CSV
+```typescript
+new SftpDataSource(
+  {
+    type: 'SFTP_CSV',
+    connectionId: 'sftp-csv',
+    name: 'SFTP CSV Source',
+    settings: {
+      host: 'sftp.example.com',
+      port: 22,
+      username: 'user',
+      password: 'pass',
+      remotePath: '/data',
+      filePattern: '*.csv',
+      csvDelimiter: ',',
+      csvHeaders: true,
+      csvSkipEmptyLines: true,
+    },
+  },
+  logger
+);
+```
+### S3 Parquet
+```typescript
+new S3DataSource(
+  {
+    type: 'S3_PARQUET',
+    connectionId: 's3-parquet',
+    name: 'S3 Parquet Source',
+    settings: {
+      bucketName: 'my-bucket',
+      region: 'us-east-1',
+      accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
+      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
+      prefix: 'data/',
+      filePattern: '*.parquet',
+    },
+  },
+  logger
+);
+```
+### SFTP JSON
+```typescript
+new SftpDataSource(
+  {
+    type: 'SFTP_JSON',
+    connectionId: 'sftp-json',
+    name: 'SFTP JSON Source',
+    settings: {
+      host: 'sftp.example.com',
+      username: 'user',
+      password: 'pass',
+      remotePath: '/orders',
+      filePattern: 'ORDER_*.json',
+      jsonValidate: true,
+    },
+  },
+  logger
+);
+```
+## Field Mapping Patterns
+### Basic Mapping
+```typescript
+const mapper = new UniversalMapper({
+  fields: {
+    ref: { source: 'sku' },
+    locationRef: { source: 'warehouse_code' },
+    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
+    status: { source: 'status', resolver: 'sdk.uppercase' },
+  },
+});
+```
+### Custom Resolver
+```typescript
+{
+  status: {
+    source: 'status_code',
+    resolver: (value) => {
+      const map = { '1': 'ACTIVE', '0': 'INACTIVE' };
+      return map[value] || 'INACTIVE';
+    }
+  }
+}
+```
+### Nested Field Mapping
+```typescript
+{
+  'attributes.weight': {
+    source: 'weight_kg',
+    resolver: (value) => ({
+      value: parseFloat(value),
+      unit: 'kg'
+    })
+  }
+}
+```
+### Multi-Source Mapping
+```typescript
+{
+  availableQty: {
+    source: ['on_hand', 'allocated', 'damaged'],
+    resolver: ([onHand, allocated, damaged]) => {
+      return Number(onHand) - Number(allocated) - Number(damaged);
+    }
+  }
+}
+```
+## Built-in Resolvers
+| Resolver         | Purpose              | Example                     |
+| ---------------- | -------------------- | --------------------------- |
+| `sdk.uppercase`  | Convert to uppercase | `'abc'` → `'ABC'`           |
+| `sdk.lowercase`  | Convert to lowercase | `'ABC'` → `'abc'`           |
+| `sdk.trim`       | Remove whitespace    | `' abc '` → `'abc'`         |
+| `sdk.parseInt`   | Parse as integer     | `'123'` → `123`             |
+| `sdk.parseFloat` | Parse as float       | `'12.34'` → `12.34`         |
+| `sdk.formatDate` | Format ISO date      | `'2025-01-15'` → ISO string |
+| `sdk.boolean`    | Parse as boolean     | `'true'` → `true`           |
+| `sdk.toJson`     | Stringify JSON       | `{a:1}` → `'{"a":1}'`       |
+| `sdk.parseJson`  | Parse JSON string    | `'{"a":1}'` → `{a:1}`       |
+## Validation Patterns
+### Required Fields
+```typescript
+if (!record.sku || !record.quantity) {
+  return { valid: false, error: 'Missing required fields' };
+}
+```
+### Type Validation
+```typescript
+if (isNaN(Number(record.quantity))) {
+  return { valid: false, error: 'Quantity must be a number' };
+}
+```
+### Format Validation
+```typescript
+if (!/^[A-Z0-9-]+$/i.test(record.sku)) {
+  return { valid: false, error: 'Invalid SKU format' };
+}
+```
+### Schema Validation
+```typescript
+// SchemaValidationService validates GraphQL payloads against Fluent schema
+const validator = new SchemaValidationService(client);
+// Fetch and validate mutation input against schema
+const result = await validator.validateMutation(
+  'updateInventoryQuantity',
+  'UpdateInventoryQuantityInput',
+  payload
+);
+if (!result.valid) {
+  console.error('Validation errors:', result.errors);
+}
+```
+## Sending Data to Fluent
+### Batch API (Inventory Only)
+```typescript
+const job = await client.createJob({
+  name: 'Inventory Update',
+  retailerId: process.env.FLUENT_RETAILER_ID!,
+});
+await client.sendBatch(job.id, {
+  action: 'UPSERT',
+  entityType: 'INVENTORY',
+  entities: records.map(r => ({
+    ref: r.ref,
+    productRef: r.ref,
+    locationRef: r.locationRef,
+    qty: r.qty,
+    status: r.status,
+  })),
+});
+```
+### GraphQL Mutation (All Entity Types)
+```typescript
+const mutation = `
+  mutation CreateOrder($input: CreateOrderInput!) {
+    createOrder(input: $input) {
+      id
+      ref
+      status
+    }
+  }
+`;
+const result = await client.graphql({ query: mutation, variables: { input: orderData } });
+```
+## State Management
+⚠️ **IMPORTANT:** StateService requires KV adapter passed to EVERY method call. The constructor takes only logger (optional).
+### File-Level Tracking
+```typescript
+// StateService constructor takes only logger (optional)
+const stateService = new StateService(logger);
+// KV adapter is required for ALL state operations
+const kvAdapter = new VersoriKVAdapter(openKv());
+const workflowId = 'my-workflow';
+// KV adapter MUST be passed to every method
+if (await stateService.isFileProcessed(kvAdapter, file.name, workflowId)) {
+  return; // Skip
+}
+// Process file...
+// KV adapter MUST be passed to every method
+await stateService.markFileProcessed(kvAdapter, file.name, {
+  recordsProcessed: 1000,
+}, workflowId);
+```
+### Record-Level Checkpoints
+```typescript
+// Use KV directly for checkpoints (not StateService)
+const kv = openKv();
+// Save checkpoint
+await kv.set(['checkpoint', 'job-123'], {
+  file: currentFile,
+  lastRecordId: record.id,
+  recordsProcessed: count,
+  timestamp: new Date().toISOString(),
+});
+// Resume from checkpoint
+const checkpoint = await kv.get(['checkpoint', 'job-123']);
+if (checkpoint?.value) {
+  startFrom = checkpoint.value.lastRecordId;
+}
+```
+## Error Handling
+### Three-Level Strategy
+```typescript
+// 1. Record-level: Skip invalid records
+const valid = records.filter(r => validate(r));
+// 2. File-level: Quarantine failed files
+try {
+  await processFile(file);
+} catch (error) {
+  await moveToErrorFolder(file);
+}
+// 3. Job-level: Fail fast on critical errors
+try {
+  await runJob();
+} catch (error) {
+  await notifyAdmins(error);
+  throw error;
+}
+```
+### Retry with Backoff
+```typescript
+async function retryWithBackoff(fn, maxRetries = 3) {
+  for (let i = 0; i < maxRetries; i++) {
+    try {
+      return await fn();
+    } catch (error) {
+      if (i === maxRetries - 1) throw error;
+      await sleep(1000 * Math.pow(2, i));
+    }
+  }
+}
+```
+## Extraction from Fluent
+### Query with Auto-Pagination
+```typescript
+const query = `
+  query GetOrders($retailerId: ID!, $first: Int!, $after: String) {
+    orders(retailerId: $retailerId, first: $first, after: $after) {
+      edges {
+        node { id ref status }
+        cursor
+      }
+      pageInfo { hasNextPage }
+    }
+  }
+`;
+const result = await client.graphql({
+  query,
+  variables: { retailerId, first: 100 },
+  pagination: { maxPages: 50 },
+});
+const orders = result.data.orders.edges.map(e => e.node);
+```
+## File Operations
+### Download File
+```typescript
+const content = await dataSource.downloadFile('inventory.csv');
+```
+### Upload File
+```typescript
+await dataSource.uploadFile('export.json', jsonContent);
+```
+### Move/Archive File
+```typescript
+await dataSource.moveFile('inventory.csv', 'archive/2025-01-15/inventory.csv');
+```
+### Stream Large File
+```typescript
+const stream = await s3Source.getFileStream('large.csv');
+stream.pipe(csvParser.createParseStream()).on('data', record => {
+  /* process */
+});
+```
+## Performance Tips
+| File Size        | Batch Size | Streaming | Checkpoints |
+| ---------------- | ---------- | --------- | ----------- |
+| < 1K records     | 500        | No        | Optional    |
+| 1K - 10K records | 200-500    | Optional  | Recommended |
+| > 10K records    | 100-200    | Yes       | Required    |
+## Environment Variables Template
+```bash
+# Fluent Commerce
+FLUENT_BASE_URL=https://api.fluentcommerce.com
+FLUENT_CLIENT_ID=client-id
+FLUENT_CLIENT_SECRET=***
+FLUENT_RETAILER_ID=1
+# SFTP
+SFTP_HOST=sftp.example.com
+SFTP_USERNAME=user
+SFTP_PASSWORD=***
+# S3
+S3_BUCKET=bucket-name
+AWS_REGION=us-east-1
+AWS_ACCESS_KEY_ID=***
+AWS_SECRET_ACCESS_KEY=***
+# Settings
+LOG_LEVEL=info
+BATCH_SIZE=100
+```
+## Common Scenarios Summary
+| Scenario                 | Source  | Format   | Components                                  | Complexity |
+| ------------------------ | ------- | -------- | ------------------------------------------- | ---------- |
+| Inventory Update         | SFTP    | CSV      | SftpDataSource, CSVParserService, Mapper    | Low        |
+| Product Sync             | S3      | Parquet  | S3DataSource, ParquetParserService, Mapper  | Medium     |
+| Order Creation           | SFTP    | JSON     | SftpDataSource, JSONParserService, Validator| Medium     |
+| Large File Processing    | S3      | CSV      | S3DataSource, StateService, Streaming       | High       |
+| Multi-Source Aggregation | SFTP+S3 | Multiple | Multiple sources, Reconciliation            | High       |
+| Data Export              | Fluent  | JSON/CSV | FluentClient, ExtractionOrchestrator        | Medium     |
+## Quick Links
+- [Full Guide](./connector-scenarios-readme.md)
+- [Module 1: Foundations](./modules/connector-scenarios-01-foundations.md)
+- [Module 2: Simple Scenarios](./modules/connector-scenarios-02-simple-scenarios.md)
+- [Module 7: Reference](./modules/connector-scenarios-07-reference.md)
+- [Examples](./examples/)