@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/04-REFERENCE/architecture/readme.md

@@ -1,279 +1,279 @@
-# FC Connect SDK - Architecture Documentation
-
-Welcome to the FC Connect SDK architecture documentation. This guide explains how the SDK works internally and helps you understand the design decisions behind its implementation.
-
-## Documentation Structure
-
-### 1. [Architecture Overview](./architecture-01-overview.md)
-**Start here!** Get a high-level understanding of the SDK architecture.
-
-- What is the FC Connect SDK?
-- High-level system architecture
-- Core concepts and design principles
-- Common use cases with diagrams
-- Multi-runtime support
-
-**Best for:** First-time readers, architects, technical decision-makers
-
----
-
-### 2. [Client Architecture](./architecture-02-client-architecture.md)
-Deep dive into client creation, authentication, and runtime detection.
-
-- Universal client factory pattern
-- FluentClient vs FluentVersoriClient
-- OAuth2 authentication flow
-- Platform auth integration
-- Context detection mechanism
-- Shared service layer
-
-**Best for:** Developers implementing custom integrations, understanding authentication
-
----
-
-### 3. [Data Flow Architecture](./architecture-03-data-flow.md)
-Understand how data flows through the SDK for different operations.
-
-- **Ingestion flow:** External files → Fluent Commerce
-- **Extraction flow:** Fluent Commerce → External storage
-- **Transformation flow:** External data → GraphQL mutations
-- Universal mapping engine
-- State management integration
-- Error handling strategies
-
-**Best for:** Integration developers, data engineers, workflow designers
-
----
-
-### 4. [Service Layer Architecture](./architecture-04-service-layer.md)
-Explore the organization and responsibilities of SDK services.
-
-- Service module organization (12 modules)
-- Batch, Core, Mapping, Transformation services
-- Resolver, Integration, and S3 services
-- Orchestration and Validation layers
-- Data sources and parsers
-- Service communication patterns
-
-**Best for:** SDK contributors, advanced integration developers
-
----
-
-### 5. [Integration Architecture](./architecture-05-integration-architecture.md)
-Learn how the SDK integrates with external systems.
-
-- **Fluent Commerce:** OAuth2, GraphQL, Batch API
-- **Amazon S3:** Presigned URLs, AWS SDK, streaming
-- **SFTP:** SSH connections, file operations
-- **Versori Platform:** KV storage, webhooks, connections
-- Cross-system data flows
-- Security considerations
-
-**Best for:** DevOps engineers, integration specialists, security teams
-
----
-
-### 6. [State Management Architecture](./architecture-06-state-management.md)
-Understand distributed state management and file deduplication.
-
-- File deduplication system
-- Job caching for DAILY strategy
-- Batch counting for BATCHES_PER_JOB
-- Distributed locking mechanism
-- KV adapter interface
-- State versioning and migration
-
-**Best for:** Workflow developers, reliability engineers
-
----
-
-## Quick Reference Diagrams
-
-### System Architecture
-
-```mermaid
-graph TB
-    APP[Your Application] --> SDK[FC Connect SDK]
-
-    SDK --> CLIENT[Client Layer]
-    SDK --> SERVICES[Service Layer]
-    SDK --> DATA[Data Layer]
-
-    CLIENT --> FLUENT[Fluent Commerce]
-    SERVICES --> S3[Amazon S3]
-    DATA --> SFTP[SFTP Servers]
-    SDK --> VERSORI[Versori Platform]
-
-    style APP fill:#e3f2fd
-    style SDK fill:#fff4e6
-    style FLUENT fill:#e8f5e9
-```
-
-### Core Design Patterns
-
-```mermaid
-graph LR
-    FACTORY[Universal<br/>Client Factory] --> CONTEXT[Context<br/>Detection]
-    CONTEXT --> STATELESS[Stateless<br/>Services]
-    STATELESS --> MAPPER[Universal<br/>Mapping]
-    MAPPER --> STREAM[Streaming<br/>Processing]
-    STREAM --> STATE[Distributed<br/>State]
-
-    style FACTORY fill:#e3f2fd
-    style MAPPER fill:#fff3e0
-    style STREAM fill:#e8f5e9
-```
-
-## Architecture Principles
-
-### 1. Universal Compatibility
-- **Single codebase** works across Node.js, Deno, and Versori
-- **Context detection** automatically adapts to runtime
-- **Consistent API** regardless of environment
-
-### 2. Layered Design
-- Clear separation between client, service, and data layers
-- Services can be used independently or orchestrated
-- Easy to test, maintain, and extend
-
-### 3. Stateless & Parallel-Safe
-- All operations are stateless and idempotent
-- Safe to call methods in parallel (`Promise.all`)
-- Distributed state management via KV storage
-
-### 4. Streaming-First
-- Handle large files without memory issues
-- Process data incrementally
-- Constant memory footprint
-
-### 5. Universal Mapping
-- One mapping configuration for all transformations
-- CSV ingestion, XML transformation, JSON extraction
-- Consistent field resolution priority
-
-### 6. Smart Error Handling
-- 4xx errors: Fail fast, no retry
-- 5xx errors: Exponential backoff
-- Job expiration: Auto-recreation
-- Configurable error strategies
-
-## Architecture Decision Records (ADRs)
-
-### Why Universal Client Factory?
-
-**Problem:** Different runtimes (Node.js, Deno, Versori) require different authentication and configuration.
-
-**Solution:** Single `createClient()` factory detects context and returns appropriate client implementation.
-
-**Benefits:**
-- Write once, run anywhere
-- No runtime-specific code in applications
-- Easy to add new runtime support
-
-### Why Universal Mapping?
-
-**Problem:** Different data transformations (CSV ingestion, XML mutation, extraction) used separate mapping systems.
-
-**Solution:** Single mapping configuration format with consistent field resolution.
-
-**Benefits:**
-- Learn once, use everywhere
-- Reduced complexity
-- Easier to maintain and test
-
-### Why Presigned URLs for S3?
-
-**Problem:** AWS SDK doesn't work in all environments (browsers, Versori platform).
-
-**Solution:** Generate presigned URLs for S3 operations using SigV4 signing.
-
-**Benefits:**
-- Works in restricted environments
-- No AWS SDK dependency required
-- Temporary, secure access
-- Cross-account support
-
-### Why KV-Based State Management?
-
-**Problem:** Multiple workflow executions need to coordinate and avoid duplicate processing.
-
-**Solution:** Key-value storage for distributed state with file deduplication and locking.
-
-**Benefits:**
-- Works across platforms (Versori KV, Redis, custom)
-- Prevents duplicate file processing
-- Enables job caching and batch counting
-- Distributed locking with stale detection
-
-## Performance Characteristics
-
-### Memory Usage
-- **Constant:** Streaming parsers maintain constant memory
-- **Low:** Batch operations chunk data into manageable sizes
-- **Configurable:** Adjust batch sizes based on environment
-
-### Throughput
-- **Parallel:** Stateless design enables parallel API calls
-- **Optimized:** Single-page optimization for small datasets
-- **Streaming:** Process large files without loading entirely
-
-### Latency
-- **Minimal:** Direct API communication without middleware
-- **Cached:** OAuth2 tokens cached to avoid repeated auth
-- **Retry:** Smart retry logic for transient failures
-
-## Testing Strategy
-
-### Unit Tests
-- Test individual services in isolation
-- Mock external dependencies
-- Fast feedback loop
-
-### Integration Tests
-- Test service interactions
-- Use MockFluentClient for API simulation
-- Validate end-to-end flows
-
-### Runtime Tests
-- Deno compatibility validation
-- Versori platform integration
-- Cross-runtime consistency
-
-## Contributing to Architecture
-
-When proposing architectural changes:
-
-1. **Document the problem** clearly
-2. **Propose solution** with diagrams
-3. **Consider impact** on existing integrations
-4. **Add tests** for new patterns
-5. **Update documentation** including architecture docs
-
-## Further Reading
-
-### Guides
-- [Getting Started](../../00-START-HERE/getting-started.md) - Quick start guide
-- [Universal Mapping Guide](../../02-CORE-GUIDES/mapping/mapping-readme.md) - Field mapping patterns
-- [GraphQL Mutation Mapping](../../02-CORE-GUIDES/mapping/graphql-mutation-mapping/) - XML/JSON to GraphQL
-
-### Use Cases
-- [Inventory Ingestion](../../01-TEMPLATES/versori/workflows/readme.md) - Ingestion templates
-- [Data Extraction](../../01-TEMPLATES/versori/workflows/readme.md) - Extraction templates
-- [Order Transformation](../../01-TEMPLATES/versori/webhooks/)
-
-### API Reference
-- [API Documentation](../../02-CORE-GUIDES/api-reference/) - Complete API documentation
-
-## Feedback
-
-Have questions or suggestions about the architecture?
-
-- **GitHub Issues:** [Report issues or suggest improvements](https://github.com/fluentcommerce/fc-connect-sdk/issues)
-- **Documentation:** Found something unclear? Let us know!
-
----
-
-**Next Steps:**
-1. Start with [Architecture Overview](./architecture-01-overview.md) for the big picture
-2. Dive into [Client Architecture](./architecture-02-client-architecture.md) for authentication details
-3. Explore [Data Flow](./architecture-03-data-flow.md) for integration patterns
+# FC Connect SDK - Architecture Documentation
+
+Welcome to the FC Connect SDK architecture documentation. This guide explains how the SDK works internally and helps you understand the design decisions behind its implementation.
+
+## Documentation Structure
+
+### 1. [Architecture Overview](./architecture-01-overview.md)
+**Start here!** Get a high-level understanding of the SDK architecture.
+
+- What is the FC Connect SDK?
+- High-level system architecture
+- Core concepts and design principles
+- Common use cases with diagrams
+- Multi-runtime support
+
+**Best for:** First-time readers, architects, technical decision-makers
+
+---
+
+### 2. [Client Architecture](./architecture-02-client-architecture.md)
+Deep dive into client creation, authentication, and runtime detection.
+
+- Universal client factory pattern
+- FluentClient vs FluentVersoriClient
+- OAuth2 authentication flow
+- Platform auth integration
+- Context detection mechanism
+- Shared service layer
+
+**Best for:** Developers implementing custom integrations, understanding authentication
+
+---
+
+### 3. [Data Flow Architecture](./architecture-03-data-flow.md)
+Understand how data flows through the SDK for different operations.
+
+- **Ingestion flow:** External files → Fluent Commerce
+- **Extraction flow:** Fluent Commerce → External storage
+- **Transformation flow:** External data → GraphQL mutations
+- Universal mapping engine
+- State management integration
+- Error handling strategies
+
+**Best for:** Integration developers, data engineers, workflow designers
+
+---
+
+### 4. [Service Layer Architecture](./architecture-04-service-layer.md)
+Explore the organization and responsibilities of SDK services.
+
+- Service module organization (12 modules)
+- Batch, Core, Mapping, Transformation services
+- Resolver, Integration, and S3 services
+- Orchestration and Validation layers
+- Data sources and parsers
+- Service communication patterns
+
+**Best for:** SDK contributors, advanced integration developers
+
+---
+
+### 5. [Integration Architecture](./architecture-05-integration-architecture.md)
+Learn how the SDK integrates with external systems.
+
+- **Fluent Commerce:** OAuth2, GraphQL, Batch API
+- **Amazon S3:** Presigned URLs, AWS SDK, streaming
+- **SFTP:** SSH connections, file operations
+- **Versori Platform:** KV storage, webhooks, connections
+- Cross-system data flows
+- Security considerations
+
+**Best for:** DevOps engineers, integration specialists, security teams
+
+---
+
+### 6. [State Management Architecture](./architecture-06-state-management.md)
+Understand distributed state management and file deduplication.
+
+- File deduplication system
+- Job caching for DAILY strategy
+- Batch counting for BATCHES_PER_JOB
+- Distributed locking mechanism
+- KV adapter interface
+- State versioning and migration
+
+**Best for:** Workflow developers, reliability engineers
+
+---
+
+## Quick Reference Diagrams
+
+### System Architecture
+
+```mermaid
+graph TB
+    APP[Your Application] --> SDK[FC Connect SDK]
+
+    SDK --> CLIENT[Client Layer]
+    SDK --> SERVICES[Service Layer]
+    SDK --> DATA[Data Layer]
+
+    CLIENT --> FLUENT[Fluent Commerce]
+    SERVICES --> S3[Amazon S3]
+    DATA --> SFTP[SFTP Servers]
+    SDK --> VERSORI[Versori Platform]
+
+    style APP fill:#e3f2fd
+    style SDK fill:#fff4e6
+    style FLUENT fill:#e8f5e9
+```
+
+### Core Design Patterns
+
+```mermaid
+graph LR
+    FACTORY[Universal<br/>Client Factory] --> CONTEXT[Context<br/>Detection]
+    CONTEXT --> STATELESS[Stateless<br/>Services]
+    STATELESS --> MAPPER[Universal<br/>Mapping]
+    MAPPER --> STREAM[Streaming<br/>Processing]
+    STREAM --> STATE[Distributed<br/>State]
+
+    style FACTORY fill:#e3f2fd
+    style MAPPER fill:#fff3e0
+    style STREAM fill:#e8f5e9
+```
+
+## Architecture Principles
+
+### 1. Universal Compatibility
+- **Single codebase** works across Node.js, Deno, and Versori
+- **Context detection** automatically adapts to runtime
+- **Consistent API** regardless of environment
+
+### 2. Layered Design
+- Clear separation between client, service, and data layers
+- Services can be used independently or orchestrated
+- Easy to test, maintain, and extend
+
+### 3. Stateless & Parallel-Safe
+- All operations are stateless and idempotent
+- Safe to call methods in parallel (`Promise.all`)
+- Distributed state management via KV storage
+
+### 4. Streaming-First
+- Handle large files without memory issues
+- Process data incrementally
+- Constant memory footprint
+
+### 5. Universal Mapping
+- One mapping configuration for all transformations
+- CSV ingestion, XML transformation, JSON extraction
+- Consistent field resolution priority
+
+### 6. Smart Error Handling
+- 4xx errors: Fail fast, no retry
+- 5xx errors: Exponential backoff
+- Job expiration: Auto-recreation
+- Configurable error strategies
+
+## Architecture Decision Records (ADRs)
+
+### Why Universal Client Factory?
+
+**Problem:** Different runtimes (Node.js, Deno, Versori) require different authentication and configuration.
+
+**Solution:** Single `createClient()` factory detects context and returns appropriate client implementation.
+
+**Benefits:**
+- Write once, run anywhere
+- No runtime-specific code in applications
+- Easy to add new runtime support
+
+### Why Universal Mapping?
+
+**Problem:** Different data transformations (CSV ingestion, XML mutation, extraction) used separate mapping systems.
+
+**Solution:** Single mapping configuration format with consistent field resolution.
+
+**Benefits:**
+- Learn once, use everywhere
+- Reduced complexity
+- Easier to maintain and test
+
+### Why Presigned URLs for S3?
+
+**Problem:** AWS SDK doesn't work in all environments (browsers, Versori platform).
+
+**Solution:** Generate presigned URLs for S3 operations using SigV4 signing.
+
+**Benefits:**
+- Works in restricted environments
+- No AWS SDK dependency required
+- Temporary, secure access
+- Cross-account support
+
+### Why KV-Based State Management?
+
+**Problem:** Multiple workflow executions need to coordinate and avoid duplicate processing.
+
+**Solution:** Key-value storage for distributed state with file deduplication and locking.
+
+**Benefits:**
+- Works across platforms (Versori KV, Redis, custom)
+- Prevents duplicate file processing
+- Enables job caching and batch counting
+- Distributed locking with stale detection
+
+## Performance Characteristics
+
+### Memory Usage
+- **Constant:** Streaming parsers maintain constant memory
+- **Low:** Batch operations chunk data into manageable sizes
+- **Configurable:** Adjust batch sizes based on environment
+
+### Throughput
+- **Parallel:** Stateless design enables parallel API calls
+- **Optimized:** Single-page optimization for small datasets
+- **Streaming:** Process large files without loading entirely
+
+### Latency
+- **Minimal:** Direct API communication without middleware
+- **Cached:** OAuth2 tokens cached to avoid repeated auth
+- **Retry:** Smart retry logic for transient failures
+
+## Testing Strategy
+
+### Unit Tests
+- Test individual services in isolation
+- Mock external dependencies
+- Fast feedback loop
+
+### Integration Tests
+- Test service interactions
+- Use MockFluentClient for API simulation
+- Validate end-to-end flows
+
+### Runtime Tests
+- Deno compatibility validation
+- Versori platform integration
+- Cross-runtime consistency
+
+## Contributing to Architecture
+
+When proposing architectural changes:
+
+1. **Document the problem** clearly
+2. **Propose solution** with diagrams
+3. **Consider impact** on existing integrations
+4. **Add tests** for new patterns
+5. **Update documentation** including architecture docs
+
+## Further Reading
+
+### Guides
+- [Getting Started](../../00-START-HERE/getting-started.md) - Quick start guide
+- [Universal Mapping Guide](../../02-CORE-GUIDES/mapping/mapping-readme.md) - Field mapping patterns
+- [GraphQL Mutation Mapping](../../02-CORE-GUIDES/mapping/graphql-mutation-mapping/) - XML/JSON to GraphQL
+
+### Use Cases
+- [Inventory Ingestion](../../01-TEMPLATES/versori/workflows/readme.md) - Ingestion templates
+- [Data Extraction](../../01-TEMPLATES/versori/workflows/readme.md) - Extraction templates
+- [Order Transformation](../../01-TEMPLATES/versori/webhooks/)
+
+### API Reference
+- [API Documentation](../../02-CORE-GUIDES/api-reference/) - Complete API documentation
+
+## Feedback
+
+Have questions or suggestions about the architecture?
+
+- **GitHub Issues:** [Report issues or suggest improvements](https://github.com/fluentcommerce/fc-connect-sdk/issues)
+- **Documentation:** Found something unclear? Let us know!
+
+---
+
+**Next Steps:**
+1. Start with [Architecture Overview](./architecture-01-overview.md) for the big picture
+2. Dive into [Client Architecture](./architecture-02-client-architecture.md) for authentication details
+3. Explore [Data Flow](./architecture-03-data-flow.md) for integration patterns