@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

package/docs/03-PATTERN-GUIDES/file-operations/modules/file-operations-04-sftp-operations.md

@@ -1,963 +1,963 @@
-# Module 4: SFTP Operations
-
-**Level:** Intermediate
-**Estimated Time:** 25 minutes
-
-## Overview
-
-Complete guide to SFTP-specific file operations, including connection management, directory operations, and SFTP-specific features.
-
-## Learning Objectives
-
-By the end of this module, you will:
-
-- ✅ Master all SFTP file operations
-- ✅ Understand connection pooling and lifecycle
-- ✅ Learn directory creation and path handling
-- ✅ Implement secure authentication (password and key-based)
-- ✅ Handle SFTP-specific errors
-
-## SFTP Concepts
-
-### Connection Lifecycle
-
-```typescript
-import {
-  SftpDataSource,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-
-const logger = createConsoleLogger();
-
-// Create data source (connection established on first operation)
-const sftp = new SftpDataSource(
-  {
-    settings: {
-      remotePath: '/data/incoming',
-      filePattern: '*.csv',
-      host: process.env.SFTP_HOST!,
-      port: 22,
-      username: process.env.SFTP_USERNAME!,
-      password: process.env.SFTP_PASSWORD!,
-    },
-  },
-  logger
-);
-
-try {
-  // Use the connection
-  const files = await sftp.listFiles();
-} finally {
-  // CRITICAL: Always dispose to close connection
-  await sftp.dispose();
-}
-```
-
-### Path Conventions
-
-```typescript
-// ✅ CORRECT: Absolute paths with leading slash
-const file = await sftp.downloadFile('/data/incoming/orders.csv');
-
-// ❌ WRONG: Relative paths without leading slash
-const file = await sftp.downloadFile('data/incoming/orders.csv'); // May fail
-
-// remotePath in config provides base path
-settings: {
-  remotePath: '/data/incoming',  // Base path
-  filePattern: '*.csv'
-}
-```
-
-## All SFTP Operations
-
-### List Files
-
-```typescript
-const sftp = new SftpDataSource(
-  {
-    settings: {
-      remotePath: '/data/incoming',
-      filePattern: '*.csv', // Pattern specified in config
-      host: process.env.SFTP_HOST!,
-      username: process.env.SFTP_USERNAME!,
-      password: process.env.SFTP_PASSWORD!,
-    },
-  },
-  logger
-);
-
-try {
-  // List files matching pattern from remotePath
-  const files = await sftp.listFiles();
-
-  files.forEach(file => {
-    console.log(`Name: ${file.name}`);
-    console.log(`Path: ${file.path}`);
-    console.log(`Size: ${file.size} bytes`);
-  });
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Download Files
-
-```typescript
-try {
-  const content = await sftp.downloadFile('/data/incoming/orders.csv');
-  console.log(`Downloaded ${content.length} bytes`);
-
-  // Process content
-  const lines = content.split('\n');
-  console.log(`File has ${lines.length} lines`);
-} catch (error: any) {
-  if (error.message.includes('No such file')) {
-    console.error('File does not exist');
-  } else {
-    console.error('Download failed:', error);
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Upload Files
-
-```typescript
-try {
-  const data = JSON.stringify({ status: 'processed' });
-
-  await sftp.uploadFile('/data/outgoing/results.json', data);
-  console.log('Upload successful');
-
-  // Upload buffer
-  const buffer = Buffer.from('Hello, SFTP!');
-  await sftp.uploadFile('/data/messages/greeting.txt', buffer);
-} finally {
-  await sftp.dispose();
-}
-```
-
-**Options parameter** (optional):
-```typescript
-await sftp.uploadFile('/data/outgoing/results.json', data, {
-  overwrite: true,           // Overwrite existing file (default: false)
-  createDirectories: true,   // Create parent directories (default: false)
-  permissions: '0644',       // File permissions (default: server default)
-  encoding: 'utf8'          // Content encoding (default: 'utf8')
-});
-```
-
-### Move Files
-
-```typescript
-try {
-  // Move/rename file (default: overwrite = false)
-  await sftp.moveFile('/data/incoming/orders.csv', '/data/processed/orders.csv');
-
-  // Move with overwrite flag
-  await sftp.moveFile(
-    '/data/incoming/inventory.csv',
-    '/archive/2025/01/15/inventory.csv',
-    true // Overwrite if destination exists
-  );
-
-  console.log('Files moved successfully');
-} finally {
-  await sftp.dispose();
-}
-````
-
-### Copy Files
-
-```typescript
-try {
-  // Copy file (default: overwrite = false)
-  await sftp.copyFile('/data/important/config.json', '/backup/config.json');
-
-  // Copy with overwrite
-  await sftp.copyFile('/data/source.json', '/backup/target.json', true);
-
-  console.log('File copied successfully');
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Delete Files
-
-```typescript
-try {
-  // Delete file
-  await sftp.deleteFile('/data/temp/old-file.csv');
-
-  // Delete with existence check
-  if (await sftp.fileExists('/data/archive/old-data.csv')) {
-    await sftp.deleteFile('/data/archive/old-data.csv');
-    console.log('File deleted');
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Check File Existence
-
-```typescript
-try {
-  const exists = await sftp.fileExists('/data/incoming/orders.csv');
-  console.log(`File exists: ${exists}`);
-
-  // Conditional processing
-  if (exists) {
-    const content = await sftp.downloadFile('/data/incoming/orders.csv');
-    // Process file
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-## Authentication Methods
-
-SFTP supports multiple authentication methods. How you access credentials depends on your deployment context.
-
-### Credential Access by Runtime
-
-| Runtime | Method | Location |
-|---------|--------|----------|
-| **Versori** | `activation.connections` (recommended) | Connection UI |
-| **Versori** | `credentials().get()` (alternative) | Connection UI |
-| **Standalone** | Environment variables | `.env` file |
-
-### Versori Platform: Connection-Based Auth (Recommended)
-
-**Best practice for Versori platform** - credentials are already decoded and type-safe:
-
-```typescript
-import { schedule, http } from '@versori/run';
-import { SftpDataSource } from '@fluentcommerce/fc-connect-sdk';
-
-export const sftpWorkflow = schedule("sftp-sync", "0 * * * *")
-  .then(http("process", { connection: "fluent_commerce" }, async (ctx) => {
-    const { activation, log } = ctx;
-
-    // Access ALL connections from activation
-    const allConnections = activation.connections || [];
-
-    // Find SFTP connection by name
-    const sftpConnection = allConnections.find(c => c.name === 'versori_ftp_server');
-
-    if (!sftpConnection) {
-      throw new Error(
-        `SFTP connection not found. Available: ${allConnections.map(c => c.name).join(', ')}`
-      );
-    }
-
-    // Extract credentials (already decoded - no Buffer needed!)
-    const sftpCred = sftpConnection.credentials[0]?.credential;
-
-    if (!sftpCred?.data?.basicAuth) {
-      throw new Error('SFTP connection missing Basic Authentication configuration');
-    }
-
-    const sftpUsername = sftpCred.data.basicAuth.username;
-    const sftpPassword = sftpCred.data.basicAuth.password;
-    const sftpHost = sftpConnection.baseUrl || 'sftp.example.com';
-
-    // Create SFTP data source
-    const sftp = new SftpDataSource(
-      {
-        settings: {
-          host: sftpHost,
-          port: 22,
-          username: sftpUsername,
-          password: sftpPassword,
-          remotePath: '/data',
-        },
-      },
-      log
-    );
-
-    // Use SFTP
-    const files = await sftp.listFiles();
-    return { success: true, fileCount: files.length };
-  }));
-```
-
-**Why this is recommended:**
-- ✅ Credentials already decoded (no Buffer.from needed)
-- ✅ Works in any task type (fn, http, webhook)
-- ✅ Type-safe access
-- ✅ Better error messages
-- ✅ Supports multiple connections
-
-### Versori Platform: credentials().get() (Alternative)
-
-**Use when working in fn() tasks** where connectionVariables aren't available:
-
-```typescript
-import { schedule, fn } from '@versori/run';
-import { Buffer } from 'node:buffer';  // Required for Deno runtime!
-import { SftpDataSource } from '@fluentcommerce/fc-connect-sdk';
-
-export const sftpWorkflow = schedule("sftp-sync", "0 * * * *")
-  .then(fn("get-files", async (ctx) => {
-    const { log } = ctx;
-
-    // Retrieve credentials from connection configuration
-    const sftpCred = await ctx.credentials().getAccessToken('SFTP');
-
-    if (!sftpCred?.accessToken) {
-      throw new Error('No SFTP credentials found in connection configuration');
-    }
-
-    // Decode base64 accessToken to get "username:password"
-    const rawBasicAuth = Buffer.from(sftpCred.accessToken, 'base64').toString('utf-8');
-
-    // Split on ':' to extract username and password
-    const [sftpUsername, sftpPassword] = rawBasicAuth.split(':');
-
-    if (!sftpUsername || !sftpPassword) {
-      throw new Error('Invalid SFTP credential format - expected username:password');
-    }
-
-    // Create SFTP data source
-    const sftp = new SftpDataSource(
-      {
-        settings: {
-          host: 'sftp.example.com',
-          port: 22,
-          username: sftpUsername,
-          password: sftpPassword,
-          remotePath: '/data',
-        },
-      },
-      log
-    );
-
-    const files = await sftp.listFiles();
-    return { success: true, fileCount: files.length };
-  }));
-```
-
-**Limitations:**
-- ⚠️ Requires manual base64 decoding
-- ⚠️ Only works in fn() tasks
-- ⚠️ More error-prone (string splitting)
-
-### Standalone: Password Authentication
-
-**For Node.js/Deno standalone scripts:**
-
-```typescript
-import { SftpDataSource, createConsoleLogger } from '@fluentcommerce/fc-connect-sdk';
-
-const logger = createConsoleLogger();
-
-const sftp = new SftpDataSource(
-  {
-    settings: {
-      host: process.env.SFTP_HOST!,
-      port: 22,
-      username: process.env.SFTP_USERNAME!,
-      password: process.env.SFTP_PASSWORD!,
-      remotePath: '/data',
-    },
-  },
-  logger
-);
-
-try {
-  const files = await sftp.listFiles();
-  console.log(`Found ${files.length} files`);
-} finally {
-  await sftp.dispose();
-}
-```
-
-**Environment variables (.env):**
-```bash
-SFTP_HOST=sftp.example.com
-SFTP_USERNAME=myuser
-SFTP_PASSWORD=mypassword
-```
-
-### Standalone: Private Key Authentication
-
-**For key-based authentication in standalone scripts:**
-
-```typescript
-import * as fs from 'fs';
-import { SftpDataSource, createConsoleLogger } from '@fluentcommerce/fc-connect-sdk';
-
-const logger = createConsoleLogger();
-const privateKey = fs.readFileSync('/path/to/private-key.pem', 'utf-8');
-
-const sftp = new SftpDataSource(
-  {
-    settings: {
-      host: 'sftp.example.com',
-      port: 22,
-      username: 'myuser',
-      privateKey: privateKey,
-      passphrase: process.env.KEY_PASSPHRASE, // Optional if key is encrypted
-      remotePath: '/data',
-    },
-  },
-  logger
-);
-```
-
-### Security Best Practices
-
-**For all authentication methods:**
-
-```typescript
-// ✅ GOOD: Never log credentials
-log.info('SFTP configuration', {
-  host: sftpHost,
-  username: sftpUsername,
-  hasPassword: !!sftpPassword,
-  passwordLength: sftpPassword.length  // Log length, not value
-});
-
-// ❌ BAD: Logging credentials
-log.info('SFTP config', { username, password }); // Don't do this!
-
-// ✅ GOOD: Validate credentials before use
-if (!sftpUsername || !sftpPassword) {
-  throw new Error('Missing SFTP credentials');
-}
-
-if (sftpPassword.length < 8) {
-  log.warn('SFTP password seems short', { length: sftpPassword.length });
-}
-
-// ✅ GOOD: Centralized credential management
-// Store credentials in Versori Connections (platform) or .env files (standalone)
-// Never hardcode credentials in code
-
-// ❌ BAD: Hardcoded credentials
-const username = 'hardcoded-user';  // Don't do this!
-const password = 'hardcoded-pass';  // Don't do this!
-```
-
-**See also:** [SFTP Credential Access & Security Guide](../../../02-CORE-GUIDES/data-sources/data-sources-sftp-credential-access-security.md) for complete credential retrieval patterns.
-
-## SFTP-Specific Patterns
-
-### Multi-Folder Processing
-
-```typescript
-const folders = ['orders', 'inventory', 'products'];
-
-for (const folder of folders) {
-  const sftp = new SftpDataSource(
-    {
-      settings: {
-        remotePath: `/data/${folder}`,
-        filePattern: '*.csv',
-        host: process.env.SFTP_HOST!,
-        username: process.env.SFTP_USERNAME!,
-        password: process.env.SFTP_PASSWORD!,
-      },
-    },
-    logger
-  );
-
-  try {
-    const files = await sftp.listFiles();
-    console.log(`Processing ${files.length} files from ${folder}`);
-
-    for (const file of files) {
-      const content = await sftp.downloadFile(file.path);
-      await processByType(folder, content);
-
-      const archivePath = `/archive/${folder}/${file.name}`;
-      await sftp.moveFile(file.path, archivePath);
-    }
-  } finally {
-    await sftp.dispose();
-  }
-}
-```
-
-### Date-Based Archive
-
-```typescript
-function getDateBasedPath(filename: string, basePath: string): string {
-  const now = new Date();
-  const year = now.getFullYear();
-  const month = String(now.getMonth() + 1).padStart(2, '0');
-  const day = String(now.getDate()).padStart(2, '0');
-
-  return `${basePath}/${year}/${month}/${day}/${filename}`;
-}
-
-try {
-  const files = await sftp.listFiles();
-
-  for (const file of files) {
-    const content = await sftp.downloadFile(file.path);
-    await processData(content);
-
-    const archivePath = getDateBasedPath(file.name, '/archive');
-    await sftp.moveFile(file.path, archivePath);
-    // Result: /archive/2025/01/15/orders.csv
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Batch Processing with Connection Reuse
-
-```typescript
-const sftp = new SftpDataSource(
-  {
-    settings: {
-      remotePath: '/data/incoming',
-      filePattern: '*.csv',
-      host: process.env.SFTP_HOST!,
-      username: process.env.SFTP_USERNAME!,
-      password: process.env.SFTP_PASSWORD!,
-    },
-  },
-  logger
-);
-
-try {
-  const files = await sftp.listFiles();
-
-  // Process all files using same connection
-  for (const file of files) {
-    try {
-      const content = await sftp.downloadFile(file.path);
-      const result = await processData(content);
-
-      await sftp.uploadFile(`/data/outgoing/${file.name.replace('.csv', '.json', JSON.stringify(result))}`
-      );
-
-      const today = new Date().toISOString().split('T')[0];
-      await sftp.moveFile(file.path, `/archive/${today}/${file.name}`);
-
-      console.log(`Processed ${file.name}`);
-    } catch (error) {
-      console.error(`Failed to process ${file.name}:`, error);
-      // Continue with next file
-    }
-  }
-} finally {
-  // Connection closed once after processing all files
-  await sftp.dispose();
-}
-```
-
-## Error Handling
-
-### Common SFTP Errors
-
-```typescript
-try {
-  await sftp.downloadFile('/data/missing-file.csv');
-} catch (error: any) {
-  const message = error.message.toLowerCase();
-
-  if (message.includes('no such file')) {
-    console.error('File does not exist');
-  } else if (message.includes('permission denied')) {
-    console.error('Permission denied - check SFTP permissions');
-  } else if (message.includes('connection')) {
-    console.error('Connection error - check host/port/credentials');
-  } else if (message.includes('authentication')) {
-    console.error('Authentication failed - check username/password');
-  } else if (message.includes('timeout')) {
-    console.error('Operation timed out - check network connectivity');
-  } else {
-    console.error('SFTP error:', error.message);
-  }
-}
-```
-
-### Connection Retry Logic
-
-```typescript
-async function createSftpWithRetry(
-  config: any,
-  logger: Logger,
-  maxRetries = 3
-): Promise<SftpDataSource> {
-  for (let attempt = 1; attempt <= maxRetries; attempt++) {
-    try {
-      const sftp = new SftpDataSource(config, logger);
-
-      // Test connection
-      await sftp.fileExists('/'); // Simple operation to verify connection
-
-      return sftp;
-    } catch (error: any) {
-      if (attempt === maxRetries) throw error;
-
-      const delay = Math.pow(2, attempt) * 1000;
-      console.log(`Connection retry ${attempt}/${maxRetries} after ${delay}ms`);
-      await new Promise(resolve => setTimeout(resolve, delay));
-    }
-  }
-  throw new Error('Should not reach here');
-}
-```
-
-## Performance Tips
-
-### Connection Pooling
-
-The SDK handles connection pooling automatically, but you should:
-
-```typescript
-// ✅ GOOD: Reuse connection for multiple operations
-const sftp = new SftpDataSource(config, logger);
-try {
-  await sftp.downloadFile('/file1.csv');
-  await sftp.downloadFile('/file2.csv');
-  await sftp.downloadFile('/file3.csv');
-} finally {
-  await sftp.dispose(); // Close once
-}
-
-// ❌ BAD: Create/dispose for each operation
-for (const file of files) {
-  const sftp = new SftpDataSource(config, logger);
-  await sftp.downloadFile(file.path);
-  await sftp.dispose(); // Too many connections!
-}
-```
-
-### Parallel Operations
-
-```typescript
-// Process files in parallel (with connection limit)
-const sftp = new SftpDataSource(config, logger);
-
-try {
-  const files = await sftp.listFiles();
-
-  // Limit concurrency to avoid overwhelming server
-  const concurrencyLimit = 5;
-
-  for (let i = 0; i < files.length; i += concurrencyLimit) {
-    const batch = files.slice(i, i + concurrencyLimit);
-
-    await Promise.all(
-      batch.map(async file => {
-        const content = await sftp.downloadFile(file.path);
-        return processData(content);
-      })
-    );
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-## Directory Operations
-
-### Create Directory
-
-```typescript
-try {
-  // Create directory (non-recursive)
-  await sftp.createDirectory('/data/incoming');
-
-  // Create directory recursively (creates parent directories if needed)
-  await sftp.createDirectory('/data/archive/2025/01/15', true);
-
-  // Create with specific permissions
-  await sftp.createDirectory('/data/public', true, '0755');
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Check Directory Existence
-
-```typescript
-try {
-  const exists = await sftp.directoryExists('/data/incoming');
-  
-  if (!exists) {
-    await sftp.createDirectory('/data/incoming', true);
-    console.log('Directory created');
-  } else {
-    console.log('Directory already exists');
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Directory Operations Pattern
-
-```typescript
-async function ensureDirectoryExists(sftp: SftpDataSource, path: string) {
-  if (!(await sftp.directoryExists(path))) {
-    await sftp.createDirectory(path, true); // Recursive
-    console.log(`Created directory: ${path}`);
-  }
-}
-
-try {
-  // Ensure archive directory exists before moving files
-  await ensureDirectoryExists(sftp, '/archive/2025/01/15');
-  
-  const files = await sftp.listFiles();
-  for (const file of files) {
-    const archivePath = `/archive/2025/01/15/${file.name}`;
-    await sftp.moveFile(file.path, archivePath);
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-## Built-in Content Parsing
-
-SFTP data source provides built-in parsing methods for CSV and JSON content, eliminating the need for separate parser services.
-
-### Parse CSV Content
-
-```typescript
-try {
-  // Download file
-  const csvContent = await sftp.downloadFile('/data/incoming/orders.csv');
-  
-  // Parse CSV content directly (no need for CSVParserService)
-  const fileMetadata = {
-    path: '/data/incoming/orders.csv',
-    name: 'orders.csv',
-    size: csvContent.length,
-    source: 'SFTP'
-  };
-  
-  const records = sftp.parseCsvContent(csvContent as string, fileMetadata);
-  
-  console.log(`Parsed ${records.length} CSV records`);
-  records.forEach(record => {
-    console.log(`Order: ${record.orderId}, Total: ${record.total}`);
-  });
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Parse JSON Content
-
-```typescript
-try {
-  // Download JSON file
-  const jsonContent = await sftp.downloadFile('/data/incoming/orders.json');
-  
-  // Parse JSON content (supports both JSON and JSONL formats)
-  const fileMetadata = {
-    path: '/data/incoming/orders.json',
-    name: 'orders.json',
-    size: jsonContent.length,
-    source: 'SFTP'
-  };
-  
-  // Auto-detect format (JSON array or JSONL)
-  const records = sftp.parseJsonContent(jsonContent as string, fileMetadata);
-  
-  // Or specify format explicitly
-  const jsonRecords = sftp.parseJsonContent(jsonContent as string, fileMetadata, {
-    format: 'json',  // or 'jsonl'
-    validate: true,  // Throw on invalid JSON
-    maxDepth: 10     // Limit nesting depth
-  });
-  
-  console.log(`Parsed ${records.length} JSON records`);
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Write CSV Content
-
-```typescript
-try {
-  // Prepare data
-  const records = [
-    { sku: 'SKU-001', quantity: 10, location: 'WAREHOUSE-A' },
-    { sku: 'SKU-002', quantity: 25, location: 'WAREHOUSE-B' }
-  ];
-  
-  // Generate CSV content
-  const csvContent = sftp.writeCsvContent(records, {
-    headers: ['sku', 'quantity', 'location'],
-    delimiter: ',',
-    includeHeaders: true
-  });
-  
-  // Upload CSV file
-  await sftp.uploadFile('/data/outgoing/inventory.csv', csvContent);
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Write JSON Content
-
-```typescript
-try {
-  // Prepare data
-  const orders = [
-    { id: 'ORD-001', total: 99.99, status: 'pending' },
-    { id: 'ORD-002', total: 149.99, status: 'shipped' }
-  ];
-  
-  // Generate JSON content (compact or pretty)
-  const jsonContent = sftp.writeJsonContent(orders, {
-    format: 'json',  // or 'jsonl' for JSON Lines
-    pretty: true     // Pretty-print JSON
-  });
-  
-  // Upload JSON file
-  await sftp.uploadFile('/data/outgoing/orders.json', jsonContent);
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Complete Parsing Workflow
-
-```typescript
-try {
-  // List CSV files
-  const csvFiles = await sftp.listFiles({ filePattern: '*.csv' });
-  
-  for (const file of csvFiles) {
-    // Download and parse in one step
-    const content = await sftp.downloadFile(file.path);
-    const records = sftp.parseCsvContent(content as string, file);
-    
-    // Transform data
-    const transformed = records.map(record => ({
-      ref: record.sku,
-      qty: parseInt(record.quantity),
-      locationRef: record.location
-    }));
-    
-    // Write transformed data as JSON
-    const jsonContent = sftp.writeJsonContent(transformed, { pretty: true });
-    const outputPath = file.path.replace('.csv', '.json');
-    await sftp.uploadFile(outputPath, jsonContent);
-    
-    // Archive original
-    await sftp.moveFile(file.path, `/archive/${file.name}`);
-  }
-} finally {
-  await sftp.dispose();
-}
-```
-
-## Parquet Support
-
-SFTP data source supports writing Parquet files for efficient data storage and transfer.
-
-### Write Parquet Content
-
-```typescript
-try {
-  // Prepare data records
-  const records = [
-    { sku: 'SKU-001', quantity: 10, price: 99.99, available: true },
-    { sku: 'SKU-002', quantity: 25, price: 149.99, available: true }
-  ];
-  
-  // Generate Parquet content as Buffer
-  const parquetBuffer = await sftp.writeParquetContent(records, {
-    compression: 'GZIP',      // Options: 'UNCOMPRESSED', 'GZIP', 'SNAPPY', 'LZO', 'BROTLI'
-    rowGroupSize: 5000,       // Number of rows per row group
-    schema: {                 // Optional: explicit schema (auto-detected if not provided)
-      sku: { type: 'UTF8' },
-      quantity: { type: 'INT64' },
-      price: { type: 'DOUBLE' },
-      available: { type: 'BOOLEAN' }
-    }
-  });
-  
-  // Upload Parquet file
-  await sftp.uploadFile('/data/outgoing/inventory.parquet', parquetBuffer);
-  
-  console.log(`Generated Parquet file: ${parquetBuffer.length} bytes`);
-} finally {
-  await sftp.dispose();
-}
-```
-
-### Parquet with Compression
-
-```typescript
-try {
-  const records = /* ... large dataset ... */;
-  
-  // Use SNAPPY compression for faster processing
-  const compressed = await sftp.writeParquetContent(records, {
-    compression: 'SNAPPY',
-    rowGroupSize: 10000
-  });
-  
-  // Use GZIP for smaller file size
-  const gzipped = await sftp.writeParquetContent(records, {
-    compression: 'GZIP',
-    rowGroupSize: 10000
-  });
-  
-  console.log(`SNAPPY: ${compressed.length} bytes`);
-  console.log(`GZIP: ${gzipped.length} bytes`);
-  
-  // Upload smaller file
-  await sftp.uploadFile('/data/outgoing/data.parquet', gzipped);
-} finally {
-  await sftp.dispose();
-}
-```
-
-**Note:** For reading Parquet files, use `ParquetParserService` from the SDK:
-
-```typescript
-import { ParquetParserService } from '@fluentcommerce/fc-connect-sdk';
-
-const parser = new ParquetParserService();
-const content = await sftp.downloadFile('/data/incoming/inventory.parquet', {
-  asBuffer: true
-});
-const records = await parser.parse(content as Buffer);
-```
-
-## Key Takeaways
-
-- 🎯 Always call `dispose()` in `finally` blocks to close connections
-- 🎯 Use absolute paths with leading slashes (`/data/file.csv`)
-- 🎯 SFTP has native move operation (faster than S3's copy + delete)
-- 🎯 Use `createDirectory()` with `recursive: true` to create parent directories
-- 🎯 Use `directoryExists()` to check before creating directories
-- 🎯 Built-in `parseCsvContent()` and `parseJsonContent()` eliminate need for separate parsers
-- 🎯 Built-in `writeCsvContent()` and `writeJsonContent()` for generating content
-- 🎯 `writeParquetContent()` generates compressed Parquet files efficiently
-- 🎯 Reuse connections for batch operations to improve performance
-- 🎯 Handle authentication errors, connection timeouts, and permission issues
-
-## Next Steps
-
-Continue to [Module 5: Streaming & Performance](./file-operations-05-streaming-performance.md) for handling large files.
-
----
-
-**Related Resources:**
-
-- [Quick Reference](../../../02-CORE-GUIDES/advanced-services/advanced-services-quick-reference.md) - SFTP operations cheat sheet
-- [Common Patterns](../examples/common-patterns.ts) - Copy-paste ready snippets
-- [Testing Guide](./file-operations-07-testing-troubleshooting.md) - SFTP test coverage
+# Module 4: SFTP Operations
+
+**Level:** Intermediate
+**Estimated Time:** 25 minutes
+
+## Overview
+
+Complete guide to SFTP-specific file operations, including connection management, directory operations, and SFTP-specific features.
+
+## Learning Objectives
+
+By the end of this module, you will:
+
+- ✅ Master all SFTP file operations
+- ✅ Understand connection pooling and lifecycle
+- ✅ Learn directory creation and path handling
+- ✅ Implement secure authentication (password and key-based)
+- ✅ Handle SFTP-specific errors
+
+## SFTP Concepts
+
+### Connection Lifecycle
+
+```typescript
+import {
+  SftpDataSource,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+
+const logger = createConsoleLogger();
+
+// Create data source (connection established on first operation)
+const sftp = new SftpDataSource(
+  {
+    settings: {
+      remotePath: '/data/incoming',
+      filePattern: '*.csv',
+      host: process.env.SFTP_HOST!,
+      port: 22,
+      username: process.env.SFTP_USERNAME!,
+      password: process.env.SFTP_PASSWORD!,
+    },
+  },
+  logger
+);
+
+try {
+  // Use the connection
+  const files = await sftp.listFiles();
+} finally {
+  // CRITICAL: Always dispose to close connection
+  await sftp.dispose();
+}
+```
+
+### Path Conventions
+
+```typescript
+// ✅ CORRECT: Absolute paths with leading slash
+const file = await sftp.downloadFile('/data/incoming/orders.csv');
+
+// ❌ WRONG: Relative paths without leading slash
+const file = await sftp.downloadFile('data/incoming/orders.csv'); // May fail
+
+// remotePath in config provides base path
+settings: {
+  remotePath: '/data/incoming',  // Base path
+  filePattern: '*.csv'
+}
+```
+
+## All SFTP Operations
+
+### List Files
+
+```typescript
+const sftp = new SftpDataSource(
+  {
+    settings: {
+      remotePath: '/data/incoming',
+      filePattern: '*.csv', // Pattern specified in config
+      host: process.env.SFTP_HOST!,
+      username: process.env.SFTP_USERNAME!,
+      password: process.env.SFTP_PASSWORD!,
+    },
+  },
+  logger
+);
+
+try {
+  // List files matching pattern from remotePath
+  const files = await sftp.listFiles();
+
+  files.forEach(file => {
+    console.log(`Name: ${file.name}`);
+    console.log(`Path: ${file.path}`);
+    console.log(`Size: ${file.size} bytes`);
+  });
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Download Files
+
+```typescript
+try {
+  const content = await sftp.downloadFile('/data/incoming/orders.csv');
+  console.log(`Downloaded ${content.length} bytes`);
+
+  // Process content
+  const lines = content.split('\n');
+  console.log(`File has ${lines.length} lines`);
+} catch (error: any) {
+  if (error.message.includes('No such file')) {
+    console.error('File does not exist');
+  } else {
+    console.error('Download failed:', error);
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Upload Files
+
+```typescript
+try {
+  const data = JSON.stringify({ status: 'processed' });
+
+  await sftp.uploadFile('/data/outgoing/results.json', data);
+  console.log('Upload successful');
+
+  // Upload buffer
+  const buffer = Buffer.from('Hello, SFTP!');
+  await sftp.uploadFile('/data/messages/greeting.txt', buffer);
+} finally {
+  await sftp.dispose();
+}
+```
+
+**Options parameter** (optional):
+```typescript
+await sftp.uploadFile('/data/outgoing/results.json', data, {
+  overwrite: true,           // Overwrite existing file (default: false)
+  createDirectories: true,   // Create parent directories (default: false)
+  permissions: '0644',       // File permissions (default: server default)
+  encoding: 'utf8'          // Content encoding (default: 'utf8')
+});
+```
+
+### Move Files
+
+```typescript
+try {
+  // Move/rename file (default: overwrite = false)
+  await sftp.moveFile('/data/incoming/orders.csv', '/data/processed/orders.csv');
+
+  // Move with overwrite flag
+  await sftp.moveFile(
+    '/data/incoming/inventory.csv',
+    '/archive/2025/01/15/inventory.csv',
+    true // Overwrite if destination exists
+  );
+
+  console.log('Files moved successfully');
+} finally {
+  await sftp.dispose();
+}
+````
+
+### Copy Files
+
+```typescript
+try {
+  // Copy file (default: overwrite = false)
+  await sftp.copyFile('/data/important/config.json', '/backup/config.json');
+
+  // Copy with overwrite
+  await sftp.copyFile('/data/source.json', '/backup/target.json', true);
+
+  console.log('File copied successfully');
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Delete Files
+
+```typescript
+try {
+  // Delete file
+  await sftp.deleteFile('/data/temp/old-file.csv');
+
+  // Delete with existence check
+  if (await sftp.fileExists('/data/archive/old-data.csv')) {
+    await sftp.deleteFile('/data/archive/old-data.csv');
+    console.log('File deleted');
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Check File Existence
+
+```typescript
+try {
+  const exists = await sftp.fileExists('/data/incoming/orders.csv');
+  console.log(`File exists: ${exists}`);
+
+  // Conditional processing
+  if (exists) {
+    const content = await sftp.downloadFile('/data/incoming/orders.csv');
+    // Process file
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+## Authentication Methods
+
+SFTP supports multiple authentication methods. How you access credentials depends on your deployment context.
+
+### Credential Access by Runtime
+
+| Runtime | Method | Location |
+|---------|--------|----------|
+| **Versori** | `activation.connections` (recommended) | Connection UI |
+| **Versori** | `credentials().get()` (alternative) | Connection UI |
+| **Standalone** | Environment variables | `.env` file |
+
+### Versori Platform: Connection-Based Auth (Recommended)
+
+**Best practice for Versori platform** - credentials are already decoded and type-safe:
+
+```typescript
+import { schedule, http } from '@versori/run';
+import { SftpDataSource } from '@fluentcommerce/fc-connect-sdk';
+
+export const sftpWorkflow = schedule("sftp-sync", "0 * * * *")
+  .then(http("process", { connection: "fluent_commerce" }, async (ctx) => {
+    const { activation, log } = ctx;
+
+    // Access ALL connections from activation
+    const allConnections = activation.connections || [];
+
+    // Find SFTP connection by name
+    const sftpConnection = allConnections.find(c => c.name === 'versori_ftp_server');
+
+    if (!sftpConnection) {
+      throw new Error(
+        `SFTP connection not found. Available: ${allConnections.map(c => c.name).join(', ')}`
+      );
+    }
+
+    // Extract credentials (already decoded - no Buffer needed!)
+    const sftpCred = sftpConnection.credentials[0]?.credential;
+
+    if (!sftpCred?.data?.basicAuth) {
+      throw new Error('SFTP connection missing Basic Authentication configuration');
+    }
+
+    const sftpUsername = sftpCred.data.basicAuth.username;
+    const sftpPassword = sftpCred.data.basicAuth.password;
+    const sftpHost = sftpConnection.baseUrl || 'sftp.example.com';
+
+    // Create SFTP data source
+    const sftp = new SftpDataSource(
+      {
+        settings: {
+          host: sftpHost,
+          port: 22,
+          username: sftpUsername,
+          password: sftpPassword,
+          remotePath: '/data',
+        },
+      },
+      log
+    );
+
+    // Use SFTP
+    const files = await sftp.listFiles();
+    return { success: true, fileCount: files.length };
+  }));
+```
+
+**Why this is recommended:**
+- ✅ Credentials already decoded (no Buffer.from needed)
+- ✅ Works in any task type (fn, http, webhook)
+- ✅ Type-safe access
+- ✅ Better error messages
+- ✅ Supports multiple connections
+
+### Versori Platform: credentials().get() (Alternative)
+
+**Use when working in fn() tasks** where connectionVariables aren't available:
+
+```typescript
+import { schedule, fn } from '@versori/run';
+import { Buffer } from 'node:buffer';  // Required for Deno runtime!
+import { SftpDataSource } from '@fluentcommerce/fc-connect-sdk';
+
+export const sftpWorkflow = schedule("sftp-sync", "0 * * * *")
+  .then(fn("get-files", async (ctx) => {
+    const { log } = ctx;
+
+    // Retrieve credentials from connection configuration
+    const sftpCred = await ctx.credentials().getAccessToken('SFTP');
+
+    if (!sftpCred?.accessToken) {
+      throw new Error('No SFTP credentials found in connection configuration');
+    }
+
+    // Decode base64 accessToken to get "username:password"
+    const rawBasicAuth = Buffer.from(sftpCred.accessToken, 'base64').toString('utf-8');
+
+    // Split on ':' to extract username and password
+    const [sftpUsername, sftpPassword] = rawBasicAuth.split(':');
+
+    if (!sftpUsername || !sftpPassword) {
+      throw new Error('Invalid SFTP credential format - expected username:password');
+    }
+
+    // Create SFTP data source
+    const sftp = new SftpDataSource(
+      {
+        settings: {
+          host: 'sftp.example.com',
+          port: 22,
+          username: sftpUsername,
+          password: sftpPassword,
+          remotePath: '/data',
+        },
+      },
+      log
+    );
+
+    const files = await sftp.listFiles();
+    return { success: true, fileCount: files.length };
+  }));
+```
+
+**Limitations:**
+- ⚠️ Requires manual base64 decoding
+- ⚠️ Only works in fn() tasks
+- ⚠️ More error-prone (string splitting)
+
+### Standalone: Password Authentication
+
+**For Node.js/Deno standalone scripts:**
+
+```typescript
+import { SftpDataSource, createConsoleLogger } from '@fluentcommerce/fc-connect-sdk';
+
+const logger = createConsoleLogger();
+
+const sftp = new SftpDataSource(
+  {
+    settings: {
+      host: process.env.SFTP_HOST!,
+      port: 22,
+      username: process.env.SFTP_USERNAME!,
+      password: process.env.SFTP_PASSWORD!,
+      remotePath: '/data',
+    },
+  },
+  logger
+);
+
+try {
+  const files = await sftp.listFiles();
+  console.log(`Found ${files.length} files`);
+} finally {
+  await sftp.dispose();
+}
+```
+
+**Environment variables (.env):**
+```bash
+SFTP_HOST=sftp.example.com
+SFTP_USERNAME=myuser
+SFTP_PASSWORD=mypassword
+```
+
+### Standalone: Private Key Authentication
+
+**For key-based authentication in standalone scripts:**
+
+```typescript
+import * as fs from 'fs';
+import { SftpDataSource, createConsoleLogger } from '@fluentcommerce/fc-connect-sdk';
+
+const logger = createConsoleLogger();
+const privateKey = fs.readFileSync('/path/to/private-key.pem', 'utf-8');
+
+const sftp = new SftpDataSource(
+  {
+    settings: {
+      host: 'sftp.example.com',
+      port: 22,
+      username: 'myuser',
+      privateKey: privateKey,
+      passphrase: process.env.KEY_PASSPHRASE, // Optional if key is encrypted
+      remotePath: '/data',
+    },
+  },
+  logger
+);
+```
+
+### Security Best Practices
+
+**For all authentication methods:**
+
+```typescript
+// ✅ GOOD: Never log credentials
+log.info('SFTP configuration', {
+  host: sftpHost,
+  username: sftpUsername,
+  hasPassword: !!sftpPassword,
+  passwordLength: sftpPassword.length  // Log length, not value
+});
+
+// ❌ BAD: Logging credentials
+log.info('SFTP config', { username, password }); // Don't do this!
+
+// ✅ GOOD: Validate credentials before use
+if (!sftpUsername || !sftpPassword) {
+  throw new Error('Missing SFTP credentials');
+}
+
+if (sftpPassword.length < 8) {
+  log.warn('SFTP password seems short', { length: sftpPassword.length });
+}
+
+// ✅ GOOD: Centralized credential management
+// Store credentials in Versori Connections (platform) or .env files (standalone)
+// Never hardcode credentials in code
+
+// ❌ BAD: Hardcoded credentials
+const username = 'hardcoded-user';  // Don't do this!
+const password = 'hardcoded-pass';  // Don't do this!
+```
+
+**See also:** [SFTP Credential Access & Security Guide](../../../02-CORE-GUIDES/data-sources/data-sources-sftp-credential-access-security.md) for complete credential retrieval patterns.
+
+## SFTP-Specific Patterns
+
+### Multi-Folder Processing
+
+```typescript
+const folders = ['orders', 'inventory', 'products'];
+
+for (const folder of folders) {
+  const sftp = new SftpDataSource(
+    {
+      settings: {
+        remotePath: `/data/${folder}`,
+        filePattern: '*.csv',
+        host: process.env.SFTP_HOST!,
+        username: process.env.SFTP_USERNAME!,
+        password: process.env.SFTP_PASSWORD!,
+      },
+    },
+    logger
+  );
+
+  try {
+    const files = await sftp.listFiles();
+    console.log(`Processing ${files.length} files from ${folder}`);
+
+    for (const file of files) {
+      const content = await sftp.downloadFile(file.path);
+      await processByType(folder, content);
+
+      const archivePath = `/archive/${folder}/${file.name}`;
+      await sftp.moveFile(file.path, archivePath);
+    }
+  } finally {
+    await sftp.dispose();
+  }
+}
+```
+
+### Date-Based Archive
+
+```typescript
+function getDateBasedPath(filename: string, basePath: string): string {
+  const now = new Date();
+  const year = now.getFullYear();
+  const month = String(now.getMonth() + 1).padStart(2, '0');
+  const day = String(now.getDate()).padStart(2, '0');
+
+  return `${basePath}/${year}/${month}/${day}/${filename}`;
+}
+
+try {
+  const files = await sftp.listFiles();
+
+  for (const file of files) {
+    const content = await sftp.downloadFile(file.path);
+    await processData(content);
+
+    const archivePath = getDateBasedPath(file.name, '/archive');
+    await sftp.moveFile(file.path, archivePath);
+    // Result: /archive/2025/01/15/orders.csv
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Batch Processing with Connection Reuse
+
+```typescript
+const sftp = new SftpDataSource(
+  {
+    settings: {
+      remotePath: '/data/incoming',
+      filePattern: '*.csv',
+      host: process.env.SFTP_HOST!,
+      username: process.env.SFTP_USERNAME!,
+      password: process.env.SFTP_PASSWORD!,
+    },
+  },
+  logger
+);
+
+try {
+  const files = await sftp.listFiles();
+
+  // Process all files using same connection
+  for (const file of files) {
+    try {
+      const content = await sftp.downloadFile(file.path);
+      const result = await processData(content);
+
+      await sftp.uploadFile(`/data/outgoing/${file.name.replace('.csv', '.json', JSON.stringify(result))}`
+      );
+
+      const today = new Date().toISOString().split('T')[0];
+      await sftp.moveFile(file.path, `/archive/${today}/${file.name}`);
+
+      console.log(`Processed ${file.name}`);
+    } catch (error) {
+      console.error(`Failed to process ${file.name}:`, error);
+      // Continue with next file
+    }
+  }
+} finally {
+  // Connection closed once after processing all files
+  await sftp.dispose();
+}
+```
+
+## Error Handling
+
+### Common SFTP Errors
+
+```typescript
+try {
+  await sftp.downloadFile('/data/missing-file.csv');
+} catch (error: any) {
+  const message = error.message.toLowerCase();
+
+  if (message.includes('no such file')) {
+    console.error('File does not exist');
+  } else if (message.includes('permission denied')) {
+    console.error('Permission denied - check SFTP permissions');
+  } else if (message.includes('connection')) {
+    console.error('Connection error - check host/port/credentials');
+  } else if (message.includes('authentication')) {
+    console.error('Authentication failed - check username/password');
+  } else if (message.includes('timeout')) {
+    console.error('Operation timed out - check network connectivity');
+  } else {
+    console.error('SFTP error:', error.message);
+  }
+}
+```
+
+### Connection Retry Logic
+
+```typescript
+async function createSftpWithRetry(
+  config: any,
+  logger: Logger,
+  maxRetries = 3
+): Promise<SftpDataSource> {
+  for (let attempt = 1; attempt <= maxRetries; attempt++) {
+    try {
+      const sftp = new SftpDataSource(config, logger);
+
+      // Test connection
+      await sftp.fileExists('/'); // Simple operation to verify connection
+
+      return sftp;
+    } catch (error: any) {
+      if (attempt === maxRetries) throw error;
+
+      const delay = Math.pow(2, attempt) * 1000;
+      console.log(`Connection retry ${attempt}/${maxRetries} after ${delay}ms`);
+      await new Promise(resolve => setTimeout(resolve, delay));
+    }
+  }
+  throw new Error('Should not reach here');
+}
+```
+
+## Performance Tips
+
+### Connection Pooling
+
+The SDK handles connection pooling automatically, but you should:
+
+```typescript
+// ✅ GOOD: Reuse connection for multiple operations
+const sftp = new SftpDataSource(config, logger);
+try {
+  await sftp.downloadFile('/file1.csv');
+  await sftp.downloadFile('/file2.csv');
+  await sftp.downloadFile('/file3.csv');
+} finally {
+  await sftp.dispose(); // Close once
+}
+
+// ❌ BAD: Create/dispose for each operation
+for (const file of files) {
+  const sftp = new SftpDataSource(config, logger);
+  await sftp.downloadFile(file.path);
+  await sftp.dispose(); // Too many connections!
+}
+```
+
+### Parallel Operations
+
+```typescript
+// Process files in parallel (with connection limit)
+const sftp = new SftpDataSource(config, logger);
+
+try {
+  const files = await sftp.listFiles();
+
+  // Limit concurrency to avoid overwhelming server
+  const concurrencyLimit = 5;
+
+  for (let i = 0; i < files.length; i += concurrencyLimit) {
+    const batch = files.slice(i, i + concurrencyLimit);
+
+    await Promise.all(
+      batch.map(async file => {
+        const content = await sftp.downloadFile(file.path);
+        return processData(content);
+      })
+    );
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+## Directory Operations
+
+### Create Directory
+
+```typescript
+try {
+  // Create directory (non-recursive)
+  await sftp.createDirectory('/data/incoming');
+
+  // Create directory recursively (creates parent directories if needed)
+  await sftp.createDirectory('/data/archive/2025/01/15', true);
+
+  // Create with specific permissions
+  await sftp.createDirectory('/data/public', true, '0755');
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Check Directory Existence
+
+```typescript
+try {
+  const exists = await sftp.directoryExists('/data/incoming');
+  
+  if (!exists) {
+    await sftp.createDirectory('/data/incoming', true);
+    console.log('Directory created');
+  } else {
+    console.log('Directory already exists');
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Directory Operations Pattern
+
+```typescript
+async function ensureDirectoryExists(sftp: SftpDataSource, path: string) {
+  if (!(await sftp.directoryExists(path))) {
+    await sftp.createDirectory(path, true); // Recursive
+    console.log(`Created directory: ${path}`);
+  }
+}
+
+try {
+  // Ensure archive directory exists before moving files
+  await ensureDirectoryExists(sftp, '/archive/2025/01/15');
+  
+  const files = await sftp.listFiles();
+  for (const file of files) {
+    const archivePath = `/archive/2025/01/15/${file.name}`;
+    await sftp.moveFile(file.path, archivePath);
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+## Built-in Content Parsing
+
+SFTP data source provides built-in parsing methods for CSV and JSON content, eliminating the need for separate parser services.
+
+### Parse CSV Content
+
+```typescript
+try {
+  // Download file
+  const csvContent = await sftp.downloadFile('/data/incoming/orders.csv');
+  
+  // Parse CSV content directly (no need for CSVParserService)
+  const fileMetadata = {
+    path: '/data/incoming/orders.csv',
+    name: 'orders.csv',
+    size: csvContent.length,
+    source: 'SFTP'
+  };
+  
+  const records = sftp.parseCsvContent(csvContent as string, fileMetadata);
+  
+  console.log(`Parsed ${records.length} CSV records`);
+  records.forEach(record => {
+    console.log(`Order: ${record.orderId}, Total: ${record.total}`);
+  });
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Parse JSON Content
+
+```typescript
+try {
+  // Download JSON file
+  const jsonContent = await sftp.downloadFile('/data/incoming/orders.json');
+  
+  // Parse JSON content (supports both JSON and JSONL formats)
+  const fileMetadata = {
+    path: '/data/incoming/orders.json',
+    name: 'orders.json',
+    size: jsonContent.length,
+    source: 'SFTP'
+  };
+  
+  // Auto-detect format (JSON array or JSONL)
+  const records = sftp.parseJsonContent(jsonContent as string, fileMetadata);
+  
+  // Or specify format explicitly
+  const jsonRecords = sftp.parseJsonContent(jsonContent as string, fileMetadata, {
+    format: 'json',  // or 'jsonl'
+    validate: true,  // Throw on invalid JSON
+    maxDepth: 10     // Limit nesting depth
+  });
+  
+  console.log(`Parsed ${records.length} JSON records`);
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Write CSV Content
+
+```typescript
+try {
+  // Prepare data
+  const records = [
+    { sku: 'SKU-001', quantity: 10, location: 'WAREHOUSE-A' },
+    { sku: 'SKU-002', quantity: 25, location: 'WAREHOUSE-B' }
+  ];
+  
+  // Generate CSV content
+  const csvContent = sftp.writeCsvContent(records, {
+    headers: ['sku', 'quantity', 'location'],
+    delimiter: ',',
+    includeHeaders: true
+  });
+  
+  // Upload CSV file
+  await sftp.uploadFile('/data/outgoing/inventory.csv', csvContent);
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Write JSON Content
+
+```typescript
+try {
+  // Prepare data
+  const orders = [
+    { id: 'ORD-001', total: 99.99, status: 'pending' },
+    { id: 'ORD-002', total: 149.99, status: 'shipped' }
+  ];
+  
+  // Generate JSON content (compact or pretty)
+  const jsonContent = sftp.writeJsonContent(orders, {
+    format: 'json',  // or 'jsonl' for JSON Lines
+    pretty: true     // Pretty-print JSON
+  });
+  
+  // Upload JSON file
+  await sftp.uploadFile('/data/outgoing/orders.json', jsonContent);
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Complete Parsing Workflow
+
+```typescript
+try {
+  // List CSV files
+  const csvFiles = await sftp.listFiles({ filePattern: '*.csv' });
+  
+  for (const file of csvFiles) {
+    // Download and parse in one step
+    const content = await sftp.downloadFile(file.path);
+    const records = sftp.parseCsvContent(content as string, file);
+    
+    // Transform data
+    const transformed = records.map(record => ({
+      ref: record.sku,
+      qty: parseInt(record.quantity),
+      locationRef: record.location
+    }));
+    
+    // Write transformed data as JSON
+    const jsonContent = sftp.writeJsonContent(transformed, { pretty: true });
+    const outputPath = file.path.replace('.csv', '.json');
+    await sftp.uploadFile(outputPath, jsonContent);
+    
+    // Archive original
+    await sftp.moveFile(file.path, `/archive/${file.name}`);
+  }
+} finally {
+  await sftp.dispose();
+}
+```
+
+## Parquet Support
+
+SFTP data source supports writing Parquet files for efficient data storage and transfer.
+
+### Write Parquet Content
+
+```typescript
+try {
+  // Prepare data records
+  const records = [
+    { sku: 'SKU-001', quantity: 10, price: 99.99, available: true },
+    { sku: 'SKU-002', quantity: 25, price: 149.99, available: true }
+  ];
+  
+  // Generate Parquet content as Buffer
+  const parquetBuffer = await sftp.writeParquetContent(records, {
+    compression: 'GZIP',      // Options: 'UNCOMPRESSED', 'GZIP', 'SNAPPY', 'LZO', 'BROTLI'
+    rowGroupSize: 5000,       // Number of rows per row group
+    schema: {                 // Optional: explicit schema (auto-detected if not provided)
+      sku: { type: 'UTF8' },
+      quantity: { type: 'INT64' },
+      price: { type: 'DOUBLE' },
+      available: { type: 'BOOLEAN' }
+    }
+  });
+  
+  // Upload Parquet file
+  await sftp.uploadFile('/data/outgoing/inventory.parquet', parquetBuffer);
+  
+  console.log(`Generated Parquet file: ${parquetBuffer.length} bytes`);
+} finally {
+  await sftp.dispose();
+}
+```
+
+### Parquet with Compression
+
+```typescript
+try {
+  const records = /* ... large dataset ... */;
+  
+  // Use SNAPPY compression for faster processing
+  const compressed = await sftp.writeParquetContent(records, {
+    compression: 'SNAPPY',
+    rowGroupSize: 10000
+  });
+  
+  // Use GZIP for smaller file size
+  const gzipped = await sftp.writeParquetContent(records, {
+    compression: 'GZIP',
+    rowGroupSize: 10000
+  });
+  
+  console.log(`SNAPPY: ${compressed.length} bytes`);
+  console.log(`GZIP: ${gzipped.length} bytes`);
+  
+  // Upload smaller file
+  await sftp.uploadFile('/data/outgoing/data.parquet', gzipped);
+} finally {
+  await sftp.dispose();
+}
+```
+
+**Note:** For reading Parquet files, use `ParquetParserService` from the SDK:
+
+```typescript
+import { ParquetParserService } from '@fluentcommerce/fc-connect-sdk';
+
+const parser = new ParquetParserService();
+const content = await sftp.downloadFile('/data/incoming/inventory.parquet', {
+  asBuffer: true
+});
+const records = await parser.parse(content as Buffer);
+```
+
+## Key Takeaways
+
+- 🎯 Always call `dispose()` in `finally` blocks to close connections
+- 🎯 Use absolute paths with leading slashes (`/data/file.csv`)
+- 🎯 SFTP has native move operation (faster than S3's copy + delete)
+- 🎯 Use `createDirectory()` with `recursive: true` to create parent directories
+- 🎯 Use `directoryExists()` to check before creating directories
+- 🎯 Built-in `parseCsvContent()` and `parseJsonContent()` eliminate need for separate parsers
+- 🎯 Built-in `writeCsvContent()` and `writeJsonContent()` for generating content
+- 🎯 `writeParquetContent()` generates compressed Parquet files efficiently
+- 🎯 Reuse connections for batch operations to improve performance
+- 🎯 Handle authentication errors, connection timeouts, and permission issues
+
+## Next Steps
+
+Continue to [Module 5: Streaming & Performance](./file-operations-05-streaming-performance.md) for handling large files.
+
+---
+
+**Related Resources:**
+
+- [Quick Reference](../../../02-CORE-GUIDES/advanced-services/advanced-services-quick-reference.md) - SFTP operations cheat sheet
+- [Common Patterns](../examples/common-patterns.ts) - Copy-paste ready snippets
+- [Testing Guide](./file-operations-07-testing-troubleshooting.md) - SFTP test coverage