simple_flow 0.1.0

data/examples/README.md

@@ -0,0 +1,374 @@
+# SimpleFlow Examples
+
+This directory contains comprehensive examples demonstrating the capabilities of the SimpleFlow gem.
+
+## Running the Examples
+
+All examples are executable Ruby scripts. Make sure you've installed the gem dependencies first:
+
+```bash
+bundle install
+```
+
+Then run any example:
+
+```bash
+ruby examples/01_basic_pipeline.rb
+ruby examples/02_error_handling.rb
+# ... etc
+```
+
+## Examples Overview
+
+### 1. Basic Pipeline (`01_basic_pipeline.rb`)
+
+**Demonstrates:**
+- Sequential step execution
+- Data transformation
+- Context propagation through steps
+- Simple computational pipelines
+
+**Key concepts:**
+- Using `step` to define pipeline stages
+- `result.continue(value)` to pass data forward
+- `result.with_context(key, value)` to track metadata
+
+**Run time:** ~2 seconds
+
+---
+
+### 2. Error Handling (`02_error_handling.rb`)
+
+**Demonstrates:**
+- Input validation with `halt`
+- Error accumulation across steps
+- Conditional flow control
+- Role-based processing
+
+**Key concepts:**
+- `result.halt` to stop execution
+- `result.with_error(key, message)` to track errors
+- Checking `result.continue?` to see if pipeline should proceed
+- Early termination prevents later steps from running
+
+**Run time:** ~1 second
+
+---
+
+### 3. Middleware (`03_middleware.rb`)
+
+**Demonstrates:**
+- Built-in logging middleware
+- Built-in instrumentation middleware
+- Stacking multiple middleware
+- Custom middleware (retry logic, authentication)
+
+**Key concepts:**
+- `use_middleware ClassName, options` to add middleware
+- Middleware wraps all steps in the pipeline
+- Custom middleware by implementing `.new(callable, **options)` and `#call(result)`
+- Middleware execution order (reverse of declaration)
+
+**Run time:** ~3 seconds
+
+---
+
+### 4. Automatic Parallel Discovery (`04_parallel_automatic.rb`)
+
+**Demonstrates:**
+- Named steps with dependency declarations
+- Automatic dependency graph construction
+- Parallel execution of independent steps
+- Complex multi-level dependency graphs
+- Dependency graph visualization
+
+**Key concepts:**
+- `step :name, callable, depends_on: [...]` for named steps
+- `call_parallel` to execute with automatic parallelism
+- Steps with satisfied dependencies run concurrently
+- Context and error merging from parallel steps
+
+**Run time:** ~1-2 seconds (with async), ~3-4 seconds (without)
+
+---
+
+### 5. Explicit Parallel Blocks (`05_parallel_explicit.rb`)
+
+**Demonstrates:**
+- Explicit `parallel do...end` blocks
+- Multiple parallel blocks in one pipeline
+- Mixing sequential and parallel execution
+- Error handling in parallel blocks
+- Performance comparison vs sequential
+
+**Key concepts:**
+- `parallel do ... end` to define concurrent execution
+- All steps in a parallel block run simultaneously
+- Halting in any parallel step stops the pipeline
+- Contexts from all parallel steps are merged
+
+**Run time:** ~1 second (with async), ~2 seconds (without)
+
+---
+
+### 6. Real-World: E-commerce Order Processing (`06_real_world_ecommerce.rb`)
+
+**Demonstrates:**
+- Complete e-commerce order pipeline
+- Integration with multiple services (inventory, payment, shipping)
+- Parallel validation and data fetching
+- Error handling at each stage
+- Notifications in parallel
+
+**Pipeline flow:**
+1. Validate order
+2. Check inventory + Calculate shipping (parallel)
+3. Calculate totals
+4. Process payment
+5. Reserve inventory
+6. Create shipment
+7. Send email + SMS (parallel)
+8. Finalize order
+
+**Key concepts:**
+- Real-world service integration patterns
+- Dependency management for complex workflows
+- Transaction-like behavior with halt on failure
+- Parallel execution for independent operations
+
+**Run time:** ~1-2 seconds
+
+---
+
+### 7. Real-World: Data ETL Pipeline (`07_real_world_etl.rb`)
+
+**Demonstrates:**
+- Extract, Transform, Load (ETL) pattern
+- Multi-source data extraction in parallel
+- Data transformation and normalization
+- Data aggregation and analytics
+- Data quality validation
+- Output preparation
+
+**Pipeline flow:**
+1. **Extract:** Fetch users + orders + products (parallel)
+2. **Transform:** Clean/normalize all data (parallel)
+3. **Aggregate:** Compute statistics (parallel)
+4. **Validate:** Check data quality
+5. **Load:** Prepare final output
+
+**Key concepts:**
+- Parallel data loading from multiple sources
+- Independent transformation pipelines
+- Analytics computation from joined data
+- Validation before output
+- Metadata tracking
+
+**Run time:** ~1-2 seconds
+
+**Optional:** Run with `--save` flag to export JSON output:
+```bash
+ruby examples/07_real_world_etl.rb --save
+```
+
+---
+
+### 8. Graph Visualization (`08_graph_visualization.rb`)
+
+**Demonstrates:**
+- Visualizing dependency graphs
+- ASCII art terminal output
+- Exporting to Graphviz DOT format
+- Exporting to Mermaid diagram format
+- Generating interactive HTML visualizations
+- Execution plan analysis
+- Graph analytics and statistics
+
+**Visualization formats:**
+- **ASCII** - Terminal-friendly text representation
+- **DOT** - Graphviz format for PNG/SVG/PDF generation
+- **Mermaid** - Modern diagram syntax for web
+- **HTML** - Interactive browser-based visualization
+- **Execution Plan** - Detailed performance analysis
+
+**Key concepts:**
+- Understanding dependency relationships
+- Identifying parallel execution opportunities
+- Visualizing pipeline structure
+- Analyzing graph performance characteristics
+- Exporting for documentation
+
+**Run time:** ~1 second
+
+**Generates files:**
+- `ecommerce_graph.dot` - For Graphviz
+- `ecommerce_graph.mmd` - For Mermaid
+- `ecommerce_graph.html` - Interactive visualization
+
+**To generate images:**
+```bash
+# Install Graphviz first (brew install graphviz / apt-get install graphviz)
+dot -Tpng ecommerce_graph.dot -o ecommerce_graph.png
+dot -Tsvg ecommerce_graph.dot -o ecommerce_graph.svg
+```
+
+---
+
+### 9. Pipeline Visualization (Direct) (`09_pipeline_visualization.rb`) ⭐ RECOMMENDED
+
+**Demonstrates:**
+- **Direct visualization from pipelines** (no manual graph creation!)
+- Calling `pipeline.visualize_ascii` directly
+- Exporting with `pipeline.visualize_dot`, `pipeline.visualize_mermaid`
+- Getting execution plan with `pipeline.execution_plan`
+- Checking if pipeline can be visualized
+- Comparing different pipeline structures
+
+**Key advantage:**
+This example shows the **recommended approach** - visualizing pipelines directly without manually recreating dependency structures. The pipeline already knows its dependencies, so you can simply call visualization methods on it.
+
+**API methods:**
+- `pipeline.visualize_ascii()` - Terminal visualization
+- `pipeline.visualize_dot()` - Graphviz export
+- `pipeline.visualize_mermaid()` - Mermaid export
+- `pipeline.execution_plan()` - Performance analysis
+- `pipeline.dependency_graph()` - Get graph object
+- `pipeline.visualize()` - Get visualizer object
+
+**Run time:** ~1 second
+
+**Note:** Only works with pipelines that have named steps (using `step :name, callable, depends_on: [...]` or `depends_on: :none`). Unnamed steps cannot be auto-visualized.
+
+---
+
+### 10. Concurrency Control (`10_concurrency_control.rb`)
+
+**Demonstrates:**
+- Per-pipeline concurrency model selection
+- Forcing threads vs async for different pipelines
+- Mixing concurrency models in the same application
+- Auto-detection behavior
+- Error handling for unavailable concurrency models
+
+**Key concepts:**
+- `Pipeline.new(concurrency: :threads)` - Force thread-based execution
+- `Pipeline.new(concurrency: :async)` - Require async gem (raises if unavailable)
+- `Pipeline.new(concurrency: :auto)` - Auto-detect (default)
+- Different pipelines can use different models in the same app
+
+**Run time:** ~1-2 seconds
+
+---
+
+### 11. Sequential Dependencies (`11_sequential_dependencies.rb`)
+
+**Demonstrates:**
+- Sequential step execution with automatic dependencies
+- Pipeline short-circuiting when a step halts
+- How unnamed steps depend on previous step's success
+- Error propagation through the pipeline
+- Comparison with parallel execution
+
+**Key concepts:**
+- Unnamed steps automatically depend on previous step
+- `result.halt` stops the entire pipeline immediately
+- Subsequent steps are never executed after a halt
+- Errors accumulate and propagate through the result
+
+**Run time:** ~1 second
+
+---
+
+### 12. Reserved Dependency Symbols (`12_none_constant.rb`)
+
+**Demonstrates:**
+- Using `:none` symbol for cleaner "no dependencies" syntax
+- Using `:nothing` as an alternative
+- Comparison with empty array `[]` syntax
+- Multiple independent root steps
+- Filtering reserved symbols from dependency arrays
+
+**Key concepts:**
+- `depends_on: :none` is equivalent to `depends_on: []`
+- More readable and semantic than empty array
+- `:none` and `:nothing` are reserved symbols (cannot be step names)
+- Symbols are automatically filtered from dependency arrays
+
+**Run time:** ~1 second
+
+---
+
+## Async Gem Availability
+
+All parallel examples will automatically use the `async` gem if available for true concurrent execution. If not available, they fall back to sequential execution.
+
+Check async availability:
+```ruby
+pipeline = SimpleFlow::Pipeline.new
+puts pipeline.async_available?  # => true or false
+```
+
+## Learning Path
+
+Recommended order for learning:
+
+1. Start with `01_basic_pipeline.rb` to understand core concepts
+2. Move to `02_error_handling.rb` for flow control
+3. Explore `03_middleware.rb` for cross-cutting concerns
+4. Learn parallel execution with `04_parallel_automatic.rb` and `05_parallel_explicit.rb`
+5. See real-world applications in `06_real_world_ecommerce.rb` and `07_real_world_etl.rb`
+
+## Customization
+
+All examples are designed to be modified. Try:
+
+- Adding your own steps to pipelines
+- Creating custom middleware
+- Changing dependency graphs
+- Adding more error handling
+- Integrating with real external services
+
+## Performance Notes
+
+- **Parallel execution** is most beneficial for I/O-bound operations (API calls, DB queries, file operations)
+- **Sequential execution** may be faster for CPU-bound tasks due to Ruby's GIL
+- Run examples multiple times to see consistent timing
+- Actual speedup depends on system resources and async gem availability
+
+## Troubleshooting
+
+**Async gem not loading:**
+```bash
+bundle install
+gem install async
+```
+
+**Permission errors on example files:**
+```bash
+chmod +x examples/*.rb
+```
+
+**Examples run sequentially despite async gem:**
+- Check that async gem is properly installed
+- Verify with `pipeline.async_available?`
+
+## Additional Resources
+
+- [SimpleFlow README](../README.md) - Full gem documentation
+- [Test Suite](../test/) - More usage examples
+- [Source Code](../lib/simple_flow/) - Implementation details
+
+## Contributing Examples
+
+Have a great use case? Consider contributing an example:
+
+1. Follow the existing naming convention (`NN_description.rb`)
+2. Include clear comments and output
+3. Demonstrate a specific feature or pattern
+4. Keep examples self-contained and runnable
+5. Add an entry to this README
+
+---
+
+**Questions or Issues?**
+Open an issue at: https://github.com/MadBomber/simple_flow/issues

data/examples/regression_test/01_basic_pipeline.txt

@@ -0,0 +1,38 @@
+============================================================
+Basic Pipeline Example
+============================================================
+
+Example 1: Data Transformation Pipeline
+------------------------------------------------------------
+  Step 1: Trimming whitespace
+  Step 2: Converting to uppercase
+  Step 3: Adding greeting
+Input:  '  world  '
+Output: 'Hello, WORLD!'
+
+
+Example 2: Numerical Computation Pipeline
+------------------------------------------------------------
+  Step 1: Add 10
+  Step 2: Multiply by 2
+  Step 3: Subtract 5
+Input:  5
+Output: 25
+Formula: (5 + 10) * 2 - 5 = 25
+
+
+Example 3: Context Propagation
+------------------------------------------------------------
+  Step 1: Recording start time
+  Step 2: Processing data
+  Step 3: Recording completion
+Result: PROCESSING
+Context:
+  started_at: 2001-09-11 07:00:00 -0500
+  processed_at: 2001-09-11 07:00:00 -0500
+  completed_at: 2001-09-11 07:00:00 -0500
+  steps_executed: 3
+
+============================================================
+Basic pipeline examples completed!
+============================================================

data/examples/regression_test/02_error_handling.txt

@@ -0,0 +1,92 @@
+============================================================
+Error Handling and Flow Control
+============================================================
+
+Example 1: Input Validation with Halt
+------------------------------------------------------------
+
+Test 1: Valid age (21)
+  Step 1: Validating age is numeric
+  Step 2: Validating age is positive
+  Step 3: Checking minimum age
+  Step 4: Processing valid age
+Continue? true
+Result: Approved for age 21
+Errors: {}
+
+Test 2: Invalid age (15)
+  Step 1: Validating age is numeric
+  Step 2: Validating age is positive
+  Step 3: Checking minimum age
+Continue? false
+Result: 15
+Errors: {validation: ["Must be 18 or older"]}
+
+Test 3: Negative age (-5)
+  Step 1: Validating age is numeric
+  Step 2: Validating age is positive
+Continue? false
+Result: -5
+Errors: {validation: ["Age cannot be negative"]}
+
+
+Example 2: Error Accumulation
+------------------------------------------------------------
+
+Test 1: Weak password
+  Step 1: Checking password length
+  Step 2: Checking for uppercase letters
+  Step 3: Checking for numbers
+  Step 4: Final validation
+Continue? false
+Result: {username: "john", password: "weak"}
+Errors: {password: ["Password must be at least 8 characters", "Password must contain uppercase letters", "Password must contain numbers"]}
+
+Test 2: Strong password
+  Step 1: Checking password length
+  Step 2: Checking for uppercase letters
+  Step 3: Checking for numbers
+  Step 4: Final validation
+  Step 5: Creating account (only runs if valid)
+Continue? true
+Result: Account created for jane
+Errors: {}
+
+
+Example 3: Conditional Processing
+------------------------------------------------------------
+
+Testing role: admin
+  Step 1: Checking user role
+  Step 2: Role-based processing
+  Step 3: Generating access token
+  Continue? true
+  Result: Token granted with permissions: read, write, delete
+  Permissions: [:read, :write, :delete]
+
+Testing role: editor
+  Step 1: Checking user role
+  Step 2: Role-based processing
+  Step 3: Generating access token
+  Continue? true
+  Result: Token granted with permissions: read, write
+  Permissions: [:read, :write]
+
+Testing role: viewer
+  Step 1: Checking user role
+  Step 2: Role-based processing
+  Step 3: Generating access token
+  Continue? true
+  Result: Token granted with permissions: read
+  Permissions: [:read]
+
+Testing role: unknown
+  Step 1: Checking user role
+  Step 2: Role-based processing
+  Continue? false
+  Result: {role: :unknown}
+  Errors: {auth: ["Unknown role: unknown"]}
+
+============================================================
+Error handling examples completed!
+============================================================

data/examples/regression_test/03_middleware.txt

@@ -0,0 +1,61 @@
+============================================================
+Middleware Examples
+============================================================
+
+Example 1: Logging Middleware
+------------------------------------------------------------
+Executing pipeline with logging middleware:
+I, [2001-09-11T07:00:00.000051 #62312]  INFO -- : Before call
+I, [2001-09-11T07:00:00.000083 #62312]  INFO -- : After call
+I, [2001-09-11T07:00:00.000099 #62312]  INFO -- : Before call
+I, [2001-09-11T07:00:00.000109 #62312]  INFO -- : After call
+Final result: 20
+
+
+Example 2: Instrumentation Middleware
+------------------------------------------------------------
+Executing pipeline with instrumentation middleware:
+Instrumentation: demo-key-123 took 0.012575s
+Instrumentation: demo-key-123 took 0.022543s
+Final result: Processed: DATA
+
+
+Example 3: Stacked Middleware
+------------------------------------------------------------
+Executing pipeline with multiple middleware:
+I, [2001-09-11T07:00:00.035360 #62312]  INFO -- : Before call
+I, [2001-09-11T07:00:00.035377 #62312]  INFO -- : After call
+Instrumentation: stacked-demo took 4.3e-05s
+I, [2001-09-11T07:00:00.035405 #62312]  INFO -- : Before call
+I, [2001-09-11T07:00:00.035414 #62312]  INFO -- : After call
+Instrumentation: stacked-demo took 2.4e-05s
+Final result: 45
+
+
+Example 4: Custom Retry Middleware
+------------------------------------------------------------
+Executing pipeline with retry middleware:
+    Attempt 1 of 3
+    Failed (Temporary failure), retrying...
+    Attempt 2 of 3
+Final result: Success on attempt 2
+
+
+Example 5: Custom Authentication Middleware
+------------------------------------------------------------
+
+Test 1: Admin user
+    Access granted for admin
+Continue? true
+Result: Sensitive admin operation completed
+Errors: {}
+
+Test 2: Regular user
+    Access denied: requires admin, got user
+Continue? false
+Result: {role: :user}
+Errors: {auth: ["Unauthorized: requires admin role"]}
+
+============================================================
+Middleware examples completed!
+============================================================

data/examples/regression_test/04_parallel_automatic.txt

@@ -0,0 +1,86 @@
+============================================================
+Automatic Parallel Discovery
+============================================================
+
+✓ Async gem is available - will use fiber-based concurrency
+
+Example 1: Basic Parallel Execution
+------------------------------------------------------------
+
+  [07:00:00.000] Fetching user...
+  [07:00:00.105] Fetching orders...
+  [07:00:00.105] Fetching products...
+  [07:00:00.206] Calculating total...
+
+Result: Total: 3 orders, 3 products
+User: {id: 123, name: "John Doe"}
+Orders: [1, 2, 3]
+Products: [:a, :b, :c]
+Execution time: 206.3ms
+(Should be ~200ms with parallel, ~400ms sequential)
+
+
+Example 2: Complex Dependency Graph
+------------------------------------------------------------
+
+Dependency graph structure:
+  Level 1: validate_input
+  Level 2: check_inventory, check_pricing, check_shipping (parallel)
+  Level 3: calculate_discount
+  Level 4: finalize_order
+
+  [Level 1] Validating input...
+  [Level 2] Checking inventory...
+  [Level 2] Checking pricing...
+  [Level 2] Checking shipping...
+  [Level 3] Calculating discount...
+  [Level 4] Finalizing order...
+
+Result: Order total: $100.0
+Context: {validated: true, inventory: :available, price: 100, shipping: 10, discount: 10.0}
+Execution time: 160.32ms
+
+
+Example 3: Dependency Graph Analysis
+------------------------------------------------------------
+
+Dependencies:
+  fetch_user: (none)
+  fetch_orders: fetch_user
+  fetch_products: fetch_user
+  fetch_reviews: fetch_user
+  calculate_stats: fetch_orders, fetch_products
+  generate_report: calculate_stats, fetch_reviews
+
+Sequential order:
+  fetch_user → fetch_orders → fetch_products → fetch_reviews → calculate_stats → generate_report
+
+Parallel execution groups:
+  Group 1: fetch_user
+  Group 2: fetch_orders, fetch_products, fetch_reviews
+  Group 3: calculate_stats
+  Group 4: generate_report
+
+Execution strategy:
+  • fetch_user runs first (no dependencies)
+  • fetch_orders, fetch_products, fetch_reviews run in parallel
+  • calculate_stats waits for orders and products
+  • generate_report waits for stats and reviews
+
+
+Example 4: Error Handling in Parallel Execution
+------------------------------------------------------------
+
+  Task A: Processing...
+  Task B: Processing...
+  Task C: Processing...
+
+Result:
+  Continue? false
+  Errors: {task_b: ["Task B encountered an error"]}
+  Context: {}
+  Note: Pipeline halted when task_b failed, preventing final_step
+
+============================================================
+Automatic parallel discovery examples completed!
+============================================================