fractor 0.1.6 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (172) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop_todo.yml +227 -102
  3. data/README.adoc +113 -1940
  4. data/docs/.lycheeignore +16 -0
  5. data/docs/Gemfile +24 -0
  6. data/docs/README.md +157 -0
  7. data/docs/_config.yml +151 -0
  8. data/docs/_features/error-handling.adoc +1192 -0
  9. data/docs/_features/index.adoc +80 -0
  10. data/docs/_features/monitoring.adoc +589 -0
  11. data/docs/_features/signal-handling.adoc +202 -0
  12. data/docs/_features/workflows.adoc +1235 -0
  13. data/docs/_guides/continuous-mode.adoc +736 -0
  14. data/docs/_guides/cookbook.adoc +1133 -0
  15. data/docs/_guides/index.adoc +55 -0
  16. data/docs/_guides/pipeline-mode.adoc +730 -0
  17. data/docs/_guides/troubleshooting.adoc +358 -0
  18. data/docs/_pages/architecture.adoc +1390 -0
  19. data/docs/_pages/core-concepts.adoc +1392 -0
  20. data/docs/_pages/design-principles.adoc +862 -0
  21. data/docs/_pages/getting-started.adoc +290 -0
  22. data/docs/_pages/installation.adoc +143 -0
  23. data/docs/_reference/api.adoc +1080 -0
  24. data/docs/_reference/error-reporting.adoc +670 -0
  25. data/docs/_reference/examples.adoc +181 -0
  26. data/docs/_reference/index.adoc +96 -0
  27. data/docs/_reference/troubleshooting.adoc +862 -0
  28. data/docs/_tutorials/complex-workflows.adoc +1022 -0
  29. data/docs/_tutorials/data-processing-pipeline.adoc +740 -0
  30. data/docs/_tutorials/first-application.adoc +384 -0
  31. data/docs/_tutorials/index.adoc +48 -0
  32. data/docs/_tutorials/long-running-services.adoc +931 -0
  33. data/docs/assets/images/favicon-16.png +0 -0
  34. data/docs/assets/images/favicon-32.png +0 -0
  35. data/docs/assets/images/favicon-48.png +0 -0
  36. data/docs/assets/images/favicon.ico +0 -0
  37. data/docs/assets/images/favicon.png +0 -0
  38. data/docs/assets/images/favicon.svg +45 -0
  39. data/docs/assets/images/fractor-icon.svg +49 -0
  40. data/docs/assets/images/fractor-logo.svg +61 -0
  41. data/docs/index.adoc +131 -0
  42. data/docs/lychee.toml +39 -0
  43. data/examples/api_aggregator/README.adoc +627 -0
  44. data/examples/api_aggregator/api_aggregator.rb +376 -0
  45. data/examples/auto_detection/README.adoc +407 -29
  46. data/examples/continuous_chat_common/message_protocol.rb +1 -1
  47. data/examples/error_reporting.rb +207 -0
  48. data/examples/file_processor/README.adoc +170 -0
  49. data/examples/file_processor/file_processor.rb +615 -0
  50. data/examples/file_processor/sample_files/invalid.csv +1 -0
  51. data/examples/file_processor/sample_files/orders.xml +24 -0
  52. data/examples/file_processor/sample_files/products.json +23 -0
  53. data/examples/file_processor/sample_files/users.csv +6 -0
  54. data/examples/hierarchical_hasher/README.adoc +629 -41
  55. data/examples/image_processor/README.adoc +610 -0
  56. data/examples/image_processor/image_processor.rb +349 -0
  57. data/examples/image_processor/processed_images/sample_10_processed.jpg.json +12 -0
  58. data/examples/image_processor/processed_images/sample_1_processed.jpg.json +12 -0
  59. data/examples/image_processor/processed_images/sample_2_processed.jpg.json +12 -0
  60. data/examples/image_processor/processed_images/sample_3_processed.jpg.json +12 -0
  61. data/examples/image_processor/processed_images/sample_4_processed.jpg.json +12 -0
  62. data/examples/image_processor/processed_images/sample_5_processed.jpg.json +12 -0
  63. data/examples/image_processor/processed_images/sample_6_processed.jpg.json +12 -0
  64. data/examples/image_processor/processed_images/sample_7_processed.jpg.json +12 -0
  65. data/examples/image_processor/processed_images/sample_8_processed.jpg.json +12 -0
  66. data/examples/image_processor/processed_images/sample_9_processed.jpg.json +12 -0
  67. data/examples/image_processor/test_images/sample_1.png +1 -0
  68. data/examples/image_processor/test_images/sample_10.png +1 -0
  69. data/examples/image_processor/test_images/sample_2.png +1 -0
  70. data/examples/image_processor/test_images/sample_3.png +1 -0
  71. data/examples/image_processor/test_images/sample_4.png +1 -0
  72. data/examples/image_processor/test_images/sample_5.png +1 -0
  73. data/examples/image_processor/test_images/sample_6.png +1 -0
  74. data/examples/image_processor/test_images/sample_7.png +1 -0
  75. data/examples/image_processor/test_images/sample_8.png +1 -0
  76. data/examples/image_processor/test_images/sample_9.png +1 -0
  77. data/examples/log_analyzer/README.adoc +662 -0
  78. data/examples/log_analyzer/log_analyzer.rb +579 -0
  79. data/examples/log_analyzer/sample_logs/apache.log +20 -0
  80. data/examples/log_analyzer/sample_logs/json.log +15 -0
  81. data/examples/log_analyzer/sample_logs/nginx.log +15 -0
  82. data/examples/log_analyzer/sample_logs/rails.log +29 -0
  83. data/examples/multi_work_type/README.adoc +576 -26
  84. data/examples/performance_monitoring.rb +120 -0
  85. data/examples/pipeline_processing/README.adoc +740 -26
  86. data/examples/pipeline_processing/pipeline_processing.rb +2 -2
  87. data/examples/priority_work_example.rb +155 -0
  88. data/examples/producer_subscriber/README.adoc +889 -46
  89. data/examples/scatter_gather/README.adoc +829 -27
  90. data/examples/simple/README.adoc +347 -0
  91. data/examples/specialized_workers/README.adoc +622 -26
  92. data/examples/specialized_workers/specialized_workers.rb +44 -8
  93. data/examples/stream_processor/README.adoc +206 -0
  94. data/examples/stream_processor/stream_processor.rb +284 -0
  95. data/examples/web_scraper/README.adoc +625 -0
  96. data/examples/web_scraper/web_scraper.rb +285 -0
  97. data/examples/workflow/README.adoc +406 -0
  98. data/examples/workflow/circuit_breaker/README.adoc +360 -0
  99. data/examples/workflow/circuit_breaker/circuit_breaker_workflow.rb +225 -0
  100. data/examples/workflow/conditional/README.adoc +483 -0
  101. data/examples/workflow/conditional/conditional_workflow.rb +215 -0
  102. data/examples/workflow/dead_letter_queue/README.adoc +374 -0
  103. data/examples/workflow/dead_letter_queue/dead_letter_queue_workflow.rb +217 -0
  104. data/examples/workflow/fan_out/README.adoc +381 -0
  105. data/examples/workflow/fan_out/fan_out_workflow.rb +202 -0
  106. data/examples/workflow/retry/README.adoc +248 -0
  107. data/examples/workflow/retry/retry_workflow.rb +195 -0
  108. data/examples/workflow/simple_linear/README.adoc +267 -0
  109. data/examples/workflow/simple_linear/simple_linear_workflow.rb +175 -0
  110. data/examples/workflow/simplified/README.adoc +329 -0
  111. data/examples/workflow/simplified/simplified_workflow.rb +222 -0
  112. data/exe/fractor +10 -0
  113. data/lib/fractor/cli.rb +288 -0
  114. data/lib/fractor/configuration.rb +307 -0
  115. data/lib/fractor/continuous_server.rb +60 -65
  116. data/lib/fractor/error_formatter.rb +72 -0
  117. data/lib/fractor/error_report_generator.rb +152 -0
  118. data/lib/fractor/error_reporter.rb +244 -0
  119. data/lib/fractor/error_statistics.rb +147 -0
  120. data/lib/fractor/execution_tracer.rb +162 -0
  121. data/lib/fractor/logger.rb +230 -0
  122. data/lib/fractor/main_loop_handler.rb +406 -0
  123. data/lib/fractor/main_loop_handler3.rb +135 -0
  124. data/lib/fractor/main_loop_handler4.rb +299 -0
  125. data/lib/fractor/performance_metrics_collector.rb +181 -0
  126. data/lib/fractor/performance_monitor.rb +215 -0
  127. data/lib/fractor/performance_report_generator.rb +202 -0
  128. data/lib/fractor/priority_work.rb +93 -0
  129. data/lib/fractor/priority_work_queue.rb +189 -0
  130. data/lib/fractor/result_aggregator.rb +32 -0
  131. data/lib/fractor/shutdown_handler.rb +168 -0
  132. data/lib/fractor/signal_handler.rb +80 -0
  133. data/lib/fractor/supervisor.rb +382 -269
  134. data/lib/fractor/supervisor_logger.rb +88 -0
  135. data/lib/fractor/version.rb +1 -1
  136. data/lib/fractor/work.rb +12 -0
  137. data/lib/fractor/work_distribution_manager.rb +151 -0
  138. data/lib/fractor/work_queue.rb +20 -0
  139. data/lib/fractor/work_result.rb +181 -9
  140. data/lib/fractor/worker.rb +73 -0
  141. data/lib/fractor/workflow/builder.rb +210 -0
  142. data/lib/fractor/workflow/chain_builder.rb +169 -0
  143. data/lib/fractor/workflow/circuit_breaker.rb +183 -0
  144. data/lib/fractor/workflow/circuit_breaker_orchestrator.rb +208 -0
  145. data/lib/fractor/workflow/circuit_breaker_registry.rb +112 -0
  146. data/lib/fractor/workflow/dead_letter_queue.rb +334 -0
  147. data/lib/fractor/workflow/execution_hooks.rb +39 -0
  148. data/lib/fractor/workflow/execution_strategy.rb +225 -0
  149. data/lib/fractor/workflow/execution_trace.rb +134 -0
  150. data/lib/fractor/workflow/helpers.rb +191 -0
  151. data/lib/fractor/workflow/job.rb +290 -0
  152. data/lib/fractor/workflow/job_dependency_validator.rb +120 -0
  153. data/lib/fractor/workflow/logger.rb +110 -0
  154. data/lib/fractor/workflow/pre_execution_context.rb +193 -0
  155. data/lib/fractor/workflow/retry_config.rb +156 -0
  156. data/lib/fractor/workflow/retry_orchestrator.rb +184 -0
  157. data/lib/fractor/workflow/retry_strategy.rb +93 -0
  158. data/lib/fractor/workflow/structured_logger.rb +30 -0
  159. data/lib/fractor/workflow/type_compatibility_validator.rb +222 -0
  160. data/lib/fractor/workflow/visualizer.rb +211 -0
  161. data/lib/fractor/workflow/workflow_context.rb +132 -0
  162. data/lib/fractor/workflow/workflow_executor.rb +669 -0
  163. data/lib/fractor/workflow/workflow_result.rb +55 -0
  164. data/lib/fractor/workflow/workflow_validator.rb +295 -0
  165. data/lib/fractor/workflow.rb +333 -0
  166. data/lib/fractor/wrapped_ractor.rb +66 -101
  167. data/lib/fractor/wrapped_ractor3.rb +161 -0
  168. data/lib/fractor/wrapped_ractor4.rb +242 -0
  169. data/lib/fractor.rb +92 -4
  170. metadata +179 -6
  171. data/tests/sample.rb.bak +0 -309
  172. data/tests/sample_working.rb.bak +0 -209
@@ -0,0 +1,181 @@
1
+ ---
2
+ layout: default
3
+ title: Examples
4
+ nav_order: 3
5
+ ---
6
+ = Examples
7
+
8
+ == Running a basic example
9
+
10
+ . Install the gem as described in the link:../installation/[Installation] section.
11
+
12
+ . Create a new Ruby file (e.g., `my_fractor_example.rb`) with your implementation:
13
+ +
14
+ [source,ruby]
15
+ ----
16
+ require 'fractor'
17
+
18
+ # Define your Work class
19
+ class MyWork < Fractor::Work
20
+ def to_s
21
+ "MyWork: #{@input}"
22
+ end
23
+ end
24
+
25
+ # Define your Worker class
26
+ class MyWorker < Fractor::Worker
27
+ def process(work)
28
+ if work.input == 5
29
+ # Return a Fractor::WorkResult for errors
30
+ return Fractor::WorkResult.new(
31
+ error: "Error processing work #{work.input}",
32
+ work: work
33
+ )
34
+ end
35
+
36
+ calculated = work.input * 2
37
+ # Return a Fractor::WorkResult for success
38
+ Fractor::WorkResult.new(result: calculated, work: work)
39
+ end
40
+ end
41
+
42
+ # Create supervisor with a worker pool
43
+ supervisor = Fractor::Supervisor.new(
44
+ worker_pools: [
45
+ { worker_class: MyWorker, num_workers: 2 }
46
+ ]
47
+ )
48
+
49
+ # Create Work objects
50
+ work_items = (1..10).map { |i| MyWork.new(i) }
51
+
52
+ # Add work items
53
+ supervisor.add_work_items(work_items)
54
+
55
+ # Run processing
56
+ supervisor.run
57
+
58
+ # Display results
59
+ puts "Results: #{supervisor.results.results.map(&:result).join(', ')}"
60
+ puts "Errors: #{supervisor.results.errors.map { |e| e.work.input }.join(', ')}"
61
+ ----
62
+
63
+ . Run the example from your terminal:
64
+ +
65
+ [source,sh]
66
+ ----
67
+ ruby my_fractor_example.rb
68
+ ----
69
+ +
70
+ You will see output showing Ractors starting, receiving work, processing it, and the final aggregated results, including any errors encountered. Press `Ctrl+C` during execution to test the graceful shutdown.
71
+
72
+ == Examples by category
73
+
74
+ Fractor includes comprehensive examples demonstrating various patterns and use cases:
75
+
76
+ === Pipeline mode examples
77
+
78
+ For batch processing and one-time computations, see:
79
+
80
+ * link:../guides/pipeline-mode.adoc#pipeline-mode-examples[Pipeline Mode Examples]
81
+ ** Simple example - Basic framework usage
82
+ ** Auto-detection example - Worker configuration
83
+ ** Hierarchical hasher - File chunking and parallel hashing
84
+ ** Multi-work type - Polymorphic workers
85
+ ** Pipeline processing - Multi-stage transformations
86
+ ** Producer/subscriber - Hierarchical processing
87
+ ** Scatter/gather - Task decomposition
88
+ ** Specialized workers - Domain-specific workers
89
+
90
+ === Continuous mode examples
91
+
92
+ For long-running servers and stream processing, see:
93
+
94
+ * link:../guides/continuous-mode.adoc#continuous-mode-examples[Continuous Mode Examples]
95
+ ** Plain socket implementation - Baseline comparison
96
+ ** Fractor-based chat server - Production-ready server with high-level primitives
97
+
98
+ === Workflow examples
99
+
100
+ For complex data processing pipelines, see:
101
+
102
+ * link:../features/workflows.adoc#workflow-examples[Workflow Examples]
103
+ ** Simple linear workflow - Sequential job pipeline
104
+ ** Fan-out workflow - Parallel job execution with aggregation
105
+ ** Conditional workflow - Runtime conditional execution
106
+ ** Simplified workflow - Reduced-boilerplate syntax
107
+ ** YAML workflows - Configuration-driven definitions
108
+ ** Builder API - Programmatic workflow construction
109
+
110
+ === Monitoring and analytics examples
111
+
112
+ For production monitoring and error analysis, see:
113
+
114
+ * link:../features/error-handling/[Error Handling] - Comprehensive error analytics
115
+ ** Error aggregation by category and job
116
+ ** Real-time error handlers and alerts
117
+ ** Trending error detection
118
+ ** Multiple export formats (text, Prometheus, JSON)
119
+ ** Critical error highlighting
120
+
121
+ The error reporting example demonstrates:
122
+
123
+ [source,ruby]
124
+ ----
125
+ # Set up error reporter
126
+ reporter = Fractor::ErrorReporter.new
127
+
128
+ # Register critical error alerts
129
+ reporter.on_error do |work_result, job_name|
130
+ if work_result.critical?
131
+ PagerDuty.alert(work_result, job_name)
132
+ end
133
+ end
134
+
135
+ # Record work results
136
+ reporter.record(work_result, job_name: "process_data")
137
+
138
+ # Generate reports
139
+ puts reporter.formatted_report
140
+ File.write("metrics.txt", reporter.to_prometheus)
141
+ ----
142
+
143
+ See `examples/error_reporting.rb` for the complete example.
144
+
145
+ == Example directory structure
146
+
147
+ All examples are located in the `examples/` directory of the gem repository:
148
+
149
+ [source]
150
+ ----
151
+ examples/
152
+ ├── simple/ # Basic usage
153
+ ├── auto_detection/ # Worker auto-detection
154
+ ├── hierarchical_hasher/ # File processing
155
+ ├── multi_work_type/ # Multiple work types
156
+ ├── pipeline_processing/ # Multi-stage pipelines
157
+ ├── producer_subscriber/ # Hierarchical processing
158
+ ├── scatter_gather/ # Task decomposition
159
+ ├── specialized_workers/ # Domain-specific workers
160
+ ├── continuous_chat_server/ # Plain socket server
161
+ ├── continuous_chat_fractor/ # Fractor-based server
162
+ └── workflow/ # Workflow examples
163
+ ├── simple_linear/
164
+ ├── fan_out/
165
+ ├── conditional/
166
+ └── simplified/
167
+ ----
168
+
169
+ Each example directory contains:
170
+
171
+ * Source code implementing the pattern
172
+ * `README.adoc` with detailed explanation
173
+ * RSpec tests demonstrating usage
174
+
175
+ == Next steps
176
+
177
+ * Start with link:../pages/getting-started/[Getting Started] for quick start guides
178
+ * Learn about link:../guides/pipeline-mode/[Pipeline Mode] for batch processing
179
+ * Learn about link:../guides/continuous-mode/[Continuous Mode] for long-running servers
180
+ * Explore link:../features/workflows/[Workflows] for complex processing graphs
181
+ * Understand link:../pages/core-concepts/[Core Concepts] of the framework
@@ -0,0 +1,96 @@
1
+ ---
2
+ layout: default
3
+ title: Overview
4
+ nav_order: 1
5
+ ---
6
+
7
+ == Overview
8
+
9
+ Comprehensive API documentation, design patterns, and advanced topics.
10
+
11
+ The reference section provides in-depth technical documentation for advanced
12
+ users and contributors. This includes complete API specifications, design
13
+ patterns, performance optimization techniques, and production deployment
14
+ guidelines.
15
+
16
+ === Available References
17
+
18
+ ==== API Documentation
19
+
20
+ * *link:../api/[API Reference]* - Complete API documentation for all classes and methods
21
+ // * *link:../configuration/[Configuration Options]* - All configuration parameters and their effects
22
+
23
+ ==== Design Patterns & Examples
24
+
25
+ * *link:../examples/[Examples]* - Real-world use cases with complete code
26
+ // * *link:../patterns/[Design Patterns]* - Common architectural patterns and best practices
27
+ // * *link:../performance/[Performance Tuning]* - Optimization strategies and benchmarking
28
+
29
+ ==== Advanced Topics
30
+
31
+ // * *link:../production/[Production Deployment]* - Guidelines for production environments
32
+ // * *link:best-practices/[Best Practices]* - Recommendations from production usage
33
+ * *link:../troubleshooting/[Troubleshooting]* - Common issues and solutions
34
+ // * *link:../contributing/[Contributing]* - How to contribute to Fractor
35
+
36
+ === Who Should Use This Section
37
+
38
+ The reference documentation is for:
39
+
40
+ * *Advanced Users* - Looking to optimize or customize Fractor
41
+ * *Library Developers* - Building on top of Fractor
42
+ * *Contributors* - Understanding internals for contributions
43
+ * *Production Engineers* - Deploying and maintaining Fractor in production
44
+
45
+ === Prerequisites
46
+
47
+ Before diving into reference documentation, you should:
48
+
49
+ * Complete the link:../../tutorials/[Tutorials] section
50
+ * Understand link:../../core-concepts/[Core Concepts]
51
+ * Have experience with production Ruby applications
52
+ * Understand parallel processing fundamentals
53
+
54
+ === Quick Navigation
55
+
56
+ ==== By Use Case
57
+
58
+ *I want to...*
59
+
60
+ * *Optimize performance* → See link:../performance/[Performance Tuning]
61
+ * *Handle production errors* → See link:best-practices/[Best Practices] and link:../troubleshooting/[Troubleshooting]
62
+ * *Understand all options* → See link:../api/[API Reference] and link:configuration/[Configuration]
63
+ * *Learn patterns* → See link:../patterns/[Design Patterns] and link:examples/[Examples]
64
+ * *Deploy to production* → See link:../production/[Production Deployment]
65
+ * *Contribute code* → See link:../contributing/[Contributing]
66
+
67
+ ==== By Experience Level
68
+
69
+ *Advanced Users*
70
+
71
+ . link:../performance/[Performance Tuning]
72
+ . link:../patterns/[Design Patterns]
73
+ . link:best-practices/[Best Practices]
74
+
75
+ *Library Developers*
76
+
77
+ . link:../api/[API Reference]
78
+ . link:../examples/[Examples]
79
+ . link:../contributing/[Contributing]
80
+
81
+ *Production Engineers*
82
+
83
+ . link:../production/[Production Deployment]
84
+ . link:best-practices/[Best Practices]
85
+ . link:../troubleshooting/[Troubleshooting]
86
+
87
+ === Related Resources
88
+
89
+ * link:../tutorials/[Tutorials] - Beginner-friendly introductions
90
+ * link:../guides/[Guides] - In-depth feature documentation
91
+ * https://github.com/metanorma/fractor[GitHub Repository] - Source code and issues
92
+ * https://rubygems.org/gems/fractor[RubyGems] - Gem releases
93
+
94
+ === Contributing to Documentation
95
+
96
+ Found an error or want to improve the documentation? See link:../contributing/[Contributing] for guidelines on submitting documentation improvements.