fractor 0.1.6 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (172) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop_todo.yml +227 -102
  3. data/README.adoc +113 -1940
  4. data/docs/.lycheeignore +16 -0
  5. data/docs/Gemfile +24 -0
  6. data/docs/README.md +157 -0
  7. data/docs/_config.yml +151 -0
  8. data/docs/_features/error-handling.adoc +1192 -0
  9. data/docs/_features/index.adoc +80 -0
  10. data/docs/_features/monitoring.adoc +589 -0
  11. data/docs/_features/signal-handling.adoc +202 -0
  12. data/docs/_features/workflows.adoc +1235 -0
  13. data/docs/_guides/continuous-mode.adoc +736 -0
  14. data/docs/_guides/cookbook.adoc +1133 -0
  15. data/docs/_guides/index.adoc +55 -0
  16. data/docs/_guides/pipeline-mode.adoc +730 -0
  17. data/docs/_guides/troubleshooting.adoc +358 -0
  18. data/docs/_pages/architecture.adoc +1390 -0
  19. data/docs/_pages/core-concepts.adoc +1392 -0
  20. data/docs/_pages/design-principles.adoc +862 -0
  21. data/docs/_pages/getting-started.adoc +290 -0
  22. data/docs/_pages/installation.adoc +143 -0
  23. data/docs/_reference/api.adoc +1080 -0
  24. data/docs/_reference/error-reporting.adoc +670 -0
  25. data/docs/_reference/examples.adoc +181 -0
  26. data/docs/_reference/index.adoc +96 -0
  27. data/docs/_reference/troubleshooting.adoc +862 -0
  28. data/docs/_tutorials/complex-workflows.adoc +1022 -0
  29. data/docs/_tutorials/data-processing-pipeline.adoc +740 -0
  30. data/docs/_tutorials/first-application.adoc +384 -0
  31. data/docs/_tutorials/index.adoc +48 -0
  32. data/docs/_tutorials/long-running-services.adoc +931 -0
  33. data/docs/assets/images/favicon-16.png +0 -0
  34. data/docs/assets/images/favicon-32.png +0 -0
  35. data/docs/assets/images/favicon-48.png +0 -0
  36. data/docs/assets/images/favicon.ico +0 -0
  37. data/docs/assets/images/favicon.png +0 -0
  38. data/docs/assets/images/favicon.svg +45 -0
  39. data/docs/assets/images/fractor-icon.svg +49 -0
  40. data/docs/assets/images/fractor-logo.svg +61 -0
  41. data/docs/index.adoc +131 -0
  42. data/docs/lychee.toml +39 -0
  43. data/examples/api_aggregator/README.adoc +627 -0
  44. data/examples/api_aggregator/api_aggregator.rb +376 -0
  45. data/examples/auto_detection/README.adoc +407 -29
  46. data/examples/continuous_chat_common/message_protocol.rb +1 -1
  47. data/examples/error_reporting.rb +207 -0
  48. data/examples/file_processor/README.adoc +170 -0
  49. data/examples/file_processor/file_processor.rb +615 -0
  50. data/examples/file_processor/sample_files/invalid.csv +1 -0
  51. data/examples/file_processor/sample_files/orders.xml +24 -0
  52. data/examples/file_processor/sample_files/products.json +23 -0
  53. data/examples/file_processor/sample_files/users.csv +6 -0
  54. data/examples/hierarchical_hasher/README.adoc +629 -41
  55. data/examples/image_processor/README.adoc +610 -0
  56. data/examples/image_processor/image_processor.rb +349 -0
  57. data/examples/image_processor/processed_images/sample_10_processed.jpg.json +12 -0
  58. data/examples/image_processor/processed_images/sample_1_processed.jpg.json +12 -0
  59. data/examples/image_processor/processed_images/sample_2_processed.jpg.json +12 -0
  60. data/examples/image_processor/processed_images/sample_3_processed.jpg.json +12 -0
  61. data/examples/image_processor/processed_images/sample_4_processed.jpg.json +12 -0
  62. data/examples/image_processor/processed_images/sample_5_processed.jpg.json +12 -0
  63. data/examples/image_processor/processed_images/sample_6_processed.jpg.json +12 -0
  64. data/examples/image_processor/processed_images/sample_7_processed.jpg.json +12 -0
  65. data/examples/image_processor/processed_images/sample_8_processed.jpg.json +12 -0
  66. data/examples/image_processor/processed_images/sample_9_processed.jpg.json +12 -0
  67. data/examples/image_processor/test_images/sample_1.png +1 -0
  68. data/examples/image_processor/test_images/sample_10.png +1 -0
  69. data/examples/image_processor/test_images/sample_2.png +1 -0
  70. data/examples/image_processor/test_images/sample_3.png +1 -0
  71. data/examples/image_processor/test_images/sample_4.png +1 -0
  72. data/examples/image_processor/test_images/sample_5.png +1 -0
  73. data/examples/image_processor/test_images/sample_6.png +1 -0
  74. data/examples/image_processor/test_images/sample_7.png +1 -0
  75. data/examples/image_processor/test_images/sample_8.png +1 -0
  76. data/examples/image_processor/test_images/sample_9.png +1 -0
  77. data/examples/log_analyzer/README.adoc +662 -0
  78. data/examples/log_analyzer/log_analyzer.rb +579 -0
  79. data/examples/log_analyzer/sample_logs/apache.log +20 -0
  80. data/examples/log_analyzer/sample_logs/json.log +15 -0
  81. data/examples/log_analyzer/sample_logs/nginx.log +15 -0
  82. data/examples/log_analyzer/sample_logs/rails.log +29 -0
  83. data/examples/multi_work_type/README.adoc +576 -26
  84. data/examples/performance_monitoring.rb +120 -0
  85. data/examples/pipeline_processing/README.adoc +740 -26
  86. data/examples/pipeline_processing/pipeline_processing.rb +2 -2
  87. data/examples/priority_work_example.rb +155 -0
  88. data/examples/producer_subscriber/README.adoc +889 -46
  89. data/examples/scatter_gather/README.adoc +829 -27
  90. data/examples/simple/README.adoc +347 -0
  91. data/examples/specialized_workers/README.adoc +622 -26
  92. data/examples/specialized_workers/specialized_workers.rb +44 -8
  93. data/examples/stream_processor/README.adoc +206 -0
  94. data/examples/stream_processor/stream_processor.rb +284 -0
  95. data/examples/web_scraper/README.adoc +625 -0
  96. data/examples/web_scraper/web_scraper.rb +285 -0
  97. data/examples/workflow/README.adoc +406 -0
  98. data/examples/workflow/circuit_breaker/README.adoc +360 -0
  99. data/examples/workflow/circuit_breaker/circuit_breaker_workflow.rb +225 -0
  100. data/examples/workflow/conditional/README.adoc +483 -0
  101. data/examples/workflow/conditional/conditional_workflow.rb +215 -0
  102. data/examples/workflow/dead_letter_queue/README.adoc +374 -0
  103. data/examples/workflow/dead_letter_queue/dead_letter_queue_workflow.rb +217 -0
  104. data/examples/workflow/fan_out/README.adoc +381 -0
  105. data/examples/workflow/fan_out/fan_out_workflow.rb +202 -0
  106. data/examples/workflow/retry/README.adoc +248 -0
  107. data/examples/workflow/retry/retry_workflow.rb +195 -0
  108. data/examples/workflow/simple_linear/README.adoc +267 -0
  109. data/examples/workflow/simple_linear/simple_linear_workflow.rb +175 -0
  110. data/examples/workflow/simplified/README.adoc +329 -0
  111. data/examples/workflow/simplified/simplified_workflow.rb +222 -0
  112. data/exe/fractor +10 -0
  113. data/lib/fractor/cli.rb +288 -0
  114. data/lib/fractor/configuration.rb +307 -0
  115. data/lib/fractor/continuous_server.rb +60 -65
  116. data/lib/fractor/error_formatter.rb +72 -0
  117. data/lib/fractor/error_report_generator.rb +152 -0
  118. data/lib/fractor/error_reporter.rb +244 -0
  119. data/lib/fractor/error_statistics.rb +147 -0
  120. data/lib/fractor/execution_tracer.rb +162 -0
  121. data/lib/fractor/logger.rb +230 -0
  122. data/lib/fractor/main_loop_handler.rb +406 -0
  123. data/lib/fractor/main_loop_handler3.rb +135 -0
  124. data/lib/fractor/main_loop_handler4.rb +299 -0
  125. data/lib/fractor/performance_metrics_collector.rb +181 -0
  126. data/lib/fractor/performance_monitor.rb +215 -0
  127. data/lib/fractor/performance_report_generator.rb +202 -0
  128. data/lib/fractor/priority_work.rb +93 -0
  129. data/lib/fractor/priority_work_queue.rb +189 -0
  130. data/lib/fractor/result_aggregator.rb +32 -0
  131. data/lib/fractor/shutdown_handler.rb +168 -0
  132. data/lib/fractor/signal_handler.rb +80 -0
  133. data/lib/fractor/supervisor.rb +382 -269
  134. data/lib/fractor/supervisor_logger.rb +88 -0
  135. data/lib/fractor/version.rb +1 -1
  136. data/lib/fractor/work.rb +12 -0
  137. data/lib/fractor/work_distribution_manager.rb +151 -0
  138. data/lib/fractor/work_queue.rb +20 -0
  139. data/lib/fractor/work_result.rb +181 -9
  140. data/lib/fractor/worker.rb +73 -0
  141. data/lib/fractor/workflow/builder.rb +210 -0
  142. data/lib/fractor/workflow/chain_builder.rb +169 -0
  143. data/lib/fractor/workflow/circuit_breaker.rb +183 -0
  144. data/lib/fractor/workflow/circuit_breaker_orchestrator.rb +208 -0
  145. data/lib/fractor/workflow/circuit_breaker_registry.rb +112 -0
  146. data/lib/fractor/workflow/dead_letter_queue.rb +334 -0
  147. data/lib/fractor/workflow/execution_hooks.rb +39 -0
  148. data/lib/fractor/workflow/execution_strategy.rb +225 -0
  149. data/lib/fractor/workflow/execution_trace.rb +134 -0
  150. data/lib/fractor/workflow/helpers.rb +191 -0
  151. data/lib/fractor/workflow/job.rb +290 -0
  152. data/lib/fractor/workflow/job_dependency_validator.rb +120 -0
  153. data/lib/fractor/workflow/logger.rb +110 -0
  154. data/lib/fractor/workflow/pre_execution_context.rb +193 -0
  155. data/lib/fractor/workflow/retry_config.rb +156 -0
  156. data/lib/fractor/workflow/retry_orchestrator.rb +184 -0
  157. data/lib/fractor/workflow/retry_strategy.rb +93 -0
  158. data/lib/fractor/workflow/structured_logger.rb +30 -0
  159. data/lib/fractor/workflow/type_compatibility_validator.rb +222 -0
  160. data/lib/fractor/workflow/visualizer.rb +211 -0
  161. data/lib/fractor/workflow/workflow_context.rb +132 -0
  162. data/lib/fractor/workflow/workflow_executor.rb +669 -0
  163. data/lib/fractor/workflow/workflow_result.rb +55 -0
  164. data/lib/fractor/workflow/workflow_validator.rb +295 -0
  165. data/lib/fractor/workflow.rb +333 -0
  166. data/lib/fractor/wrapped_ractor.rb +66 -101
  167. data/lib/fractor/wrapped_ractor3.rb +161 -0
  168. data/lib/fractor/wrapped_ractor4.rb +242 -0
  169. data/lib/fractor.rb +92 -4
  170. metadata +179 -6
  171. data/tests/sample.rb.bak +0 -309
  172. data/tests/sample_working.rb.bak +0 -209
data/README.adoc CHANGED
@@ -1,2049 +1,222 @@
1
1
  = Fractor: Function-driven Ractors framework
2
2
 
3
- Fractor is a lightweight Ruby framework designed to simplify the process of
4
- distributing computational work across multiple Ractors.
3
+ image:https://img.shields.io/gem/v/fractor.svg[RubyGems Version, link=https://rubygems.org/gems/fractor]
4
+ image:https://img.shields.io/github/license/metanorma/fractor.svg[License, link=https://github.com/metanorma/fractor/blob/main/LICENSE]
5
+ image:https://github.com/metanorma/fractor/actions/workflows/rake.yml/badge.svg[Build Status, link=https://github.com/metanorma/fractor/actions/workflows/rake.yml]
6
+ image:https://img.shields.io/badge/ruby-3.0%20%E2%86%92%204.0%2B-ruby.svg[Ruby 3.0 to 4.0+, link=https://www.ruby-lang.org]
5
7
 
6
- == Introduction
8
+ Fractor is a lightweight Ruby framework for parallel processing using Ractors
9
+ (Ruby Actors).
7
10
 
8
- Fractor stands for *Function-driven Ractors framework*. It is a lightweight
9
- Ruby framework designed to simplify the process of distributing computational
10
- work across multiple Ractors (Ruby's actor-like concurrency model).
11
+ It provides a structured way to distribute computational work across multiple
12
+ Ractors with minimal boilerplate.
11
13
 
12
- The primary goal of Fractor is to provide a structured way to define work,
13
- process it in parallel using Ractors, and aggregate the results, while
14
- abstracting away much of the boilerplate code involved in Ractor management and
15
- communication.
14
+ == Ruby version support
16
15
 
17
- == Installation
16
+ Fractor fully supports both **Ruby 3.x** and **Ruby 4.0+**:
18
17
 
19
- === Using RubyGems
18
+ * *Ruby 3.0+*: Uses `Ractor.yield` for message passing from workers
20
19
 
21
- [source,sh]
22
- ----
23
- gem install fractor
24
- ----
20
+ * *Ruby 4.0+*: Uses `Ractor::Port` for more efficient communication patterns
25
21
 
26
- === Using Bundler
22
+ Fractor automatically detects your Ruby version and uses the appropriate
23
+ internal implementation. The user-facing API is identical across versions --
24
+ write your code once, and Fractor handles the differences internally.
27
25
 
28
- Add this line to your application's Gemfile:
26
+ See
27
+ link:docs/_pages/architecture.adoc#ruby-version-compatibility[Architecture: Ruby Version Compatibility]
28
+ for details on the internal differences.
29
29
 
30
- [source,ruby]
31
- ----
32
- gem 'fractor'
33
- ----
30
+ == Quick start
34
31
 
35
- And then execute:
32
+ === Installation
36
33
 
37
34
  [source,sh]
38
35
  ----
39
- bundle install
36
+ gem install fractor
40
37
  ----
41
38
 
39
+ See link:docs/_pages/installation.adoc[Installation Guide] for more options.
42
40
 
43
- === Key concepts
44
-
45
- * *Function-driven:* You define the core processing logic by subclassing
46
- `Fractor::Worker` and implementing the `process` method.
47
-
48
- * *Parallel execution:* Work items are automatically distributed to available
49
- worker Ractors for concurrent processing.
50
-
51
- * *Result aggregation:* The framework collects both successful results and
52
- errors from the workers.
53
-
54
- * *Separation of concerns:* Keeps the framework logic (`fractor.rb`) separate
55
- from the client's specific implementation (`sample.rb`).
56
-
57
- == Scope
58
-
59
- This document describes the design, implementation, and usage of the Fractor
60
- framework. It provides detailed information about the framework's components,
61
- their interactions, and how to use them to implement parallel processing in Ruby
62
- applications.
63
-
64
- [bibliography]
65
- == Normative references
66
-
67
- * [[[ruby-ractor,Ruby Ractor Documentation]]], https://docs.ruby-lang.org/en/master/Ractor.html
68
-
69
- == Terms and definitions
70
-
71
- === ractor
72
-
73
- concurrent programming abstraction in Ruby that enables parallel execution
74
- with thread safety
75
-
76
- [.source]
77
- <<ruby>>
78
-
79
- === worker
80
-
81
- component that processes work items to produce work results
82
-
83
- === work item
84
-
85
- unit of computation to be processed by a ractor
86
-
87
- === work result
88
-
89
- result of processing a work item, either successful or an error
90
-
91
- === work item class
92
-
93
- class that represents a work item, typically subclassing `Fractor::Work`
94
-
95
- === worker class
96
-
97
- class that represents a worker, typically subclassing `Fractor::Worker`
98
-
99
- === wrapped ractor
100
-
101
- component that manages a single ractor and its associated worker
102
-
103
- === supervisor
104
-
105
- component that manages the pool of workers and distributes work items
106
-
107
- === result aggregator
108
-
109
- component that collects and organizes work results from workers
110
-
111
- === pipeline mode
112
-
113
- operating mode where Fractor processes a defined set of work items and then
114
- stops
115
-
116
- === continuous mode
117
-
118
- operating mode where Fractor runs indefinitely, processing work items as they
119
- arrive
120
-
121
-
122
-
123
-
124
- == Understanding Fractor operating modes
125
-
126
- === General
127
-
128
- Fractor supports two distinct operating modes, each optimized for different use
129
- cases. Understanding these modes is essential for choosing the right approach
130
- for your application.
131
-
132
- === Pipeline mode (batch processing)
133
-
134
- Pipeline mode is designed for processing a defined set of work items with a
135
- clear beginning and end.
136
-
137
- Characteristics:
138
-
139
- * Processes a predetermined batch of work items
140
- * Stops automatically when all work is completed
141
- * Results are collected and accessed after processing completes
142
- * Ideal for one-time computations or periodic batch jobs
143
-
144
- Common use cases:
145
-
146
- * Processing a file or dataset
147
- * Batch data transformations
148
- * One-time parallel computations
149
- * Scheduled batch jobs
150
- * Hierarchical or multi-stage processing
151
-
152
- === Continuous mode (long-running servers)
153
-
154
- Continuous mode is designed for applications that need to run indefinitely,
155
- processing work items as they arrive.
156
-
157
- Characteristics:
158
-
159
- * Runs continuously without a predetermined end
160
- * Processes work items dynamically as they become available
161
- * Workers idle efficiently when no work is available
162
- * Results are processed via callbacks, not batch collection
163
- * Supports graceful shutdown and runtime monitoring
164
-
165
- Common use cases:
166
-
167
- * Chat servers and messaging systems
168
- * Background job processors
169
- * Real-time data stream processing
170
- * Web servers handling concurrent requests
171
- * Monitoring and alerting systems
172
- * Event-driven architectures
173
-
174
- === Comparison
175
-
176
- [cols="1,2,2",options="header"]
177
- |===
178
- |Aspect |Pipeline Mode |Continuous Mode
179
-
180
- |Duration
181
- |Finite (stops when done)
182
- |Indefinite (runs until stopped)
183
-
184
- |Work arrival
185
- |All work known upfront
186
- |Work arrives dynamically
187
-
188
- |Result handling
189
- |Batch collection after completion
190
- |Callback-based processing
191
-
192
- |Typical lifetime
193
- |Seconds to minutes
194
- |Hours to days/weeks
195
-
196
- |Shutdown
197
- |Automatic on completion
198
- |Manual or signal-based
199
-
200
- |Best for
201
- |Batch jobs, file processing
202
- |Servers, streams, job queues
203
- |===
204
-
205
- === Decision guide
206
-
207
- Choose *Pipeline mode* when:
208
-
209
- * You have a complete dataset to process
210
- * Processing has a clear start and end
211
- * You need all results aggregated after completion
212
- * The task is one-time or scheduled periodically
213
-
214
- Choose *Continuous mode* when:
215
-
216
- * Work arrives over time from external sources
217
- * Your application runs as a long-lived server
218
- * You need to process items as they arrive
219
- * Results should be handled immediately via callbacks
220
-
221
-
222
-
223
-
224
- == Quick start: Pipeline mode
225
-
226
- === General
227
-
228
- This quick start guide shows the minimum steps needed to get parallel batch
229
- processing working with Fractor.
230
-
231
- === Step 1: Create a minimal Work class
232
-
233
- The Work class represents a unit of work to be processed by a Worker. It
234
- encapsulates the input data needed for processing.
41
+ === 30-second example
235
42
 
236
43
  [source,ruby]
237
44
  ----
238
45
  require 'fractor'
239
46
 
47
+ # Define your work
240
48
  class MyWork < Fractor::Work
241
- # Store all properties in the input hash
242
49
  def initialize(value)
243
50
  super({ value: value })
244
51
  end
245
52
 
246
- # Accessor method for the stored value
247
53
  def value
248
54
  input[:value]
249
55
  end
250
-
251
- def to_s
252
- "MyWork: #{value}"
253
- end
254
56
  end
255
- ----
256
-
257
- A Work is instantiated with the input data it will process this way:
258
-
259
- [source,ruby]
260
- ----
261
- work_item = MyWork.new(42)
262
- puts work_item.to_s # Output: MyWork: 42
263
- ----
264
-
265
-
266
- === Step 2: Create a minimal Worker class
267
57
 
268
- The Worker class defines the processing logic for work items. Each Worker
269
- instance runs within its own Ractor and processes Work objects sent to it.
270
-
271
- It must implement the `process(work)` method, which takes a Work object as
272
- input and returns a `Fractor::WorkResult` object.
273
-
274
- The `process` method should handle both successful processing and error
275
- conditions.
276
-
277
- [source,ruby]
278
- ----
58
+ # Define your worker
279
59
  class MyWorker < Fractor::Worker
280
60
  def process(work)
281
- # Your processing logic here
282
- result = work.input * 2
283
-
284
- # Return a success result
61
+ result = work.value * 2
285
62
  Fractor::WorkResult.new(result: result, work: work)
286
63
  rescue => e
287
- # Return an error result if something goes wrong
288
64
  Fractor::WorkResult.new(error: e.message, work: work)
289
65
  end
290
66
  end
291
- ----
292
-
293
- The `process` method can perform any computation you need. In this example, it
294
- multiplies the input by 2. If an error occurs, it catches the exception and
295
- returns an error result.
296
67
 
297
- === Step 3: Set up and run the Supervisor
298
-
299
- The Supervisor class orchestrates the entire framework, managing worker Ractors,
300
- distributing work, and collecting results.
301
-
302
- [source,ruby]
303
- ----
304
- # Create the supervisor with auto-detected number of workers
68
+ # Create supervisor and process work
305
69
  supervisor = Fractor::Supervisor.new(
306
- worker_pools: [
307
- { worker_class: MyWorker } # Number of workers auto-detected
308
- ]
70
+ worker_pools: [{ worker_class: MyWorker }]
309
71
  )
310
72
 
311
- # Add work items (instances of Work subclasses)
312
73
  supervisor.add_work_items([
313
74
  MyWork.new(1),
314
75
  MyWork.new(2),
315
- MyWork.new(3),
316
- MyWork.new(4),
317
- MyWork.new(5)
76
+ MyWork.new(3)
318
77
  ])
319
78
 
320
- # Run the processing
321
79
  supervisor.run
322
80
 
323
- # Access results after completion
324
81
  puts "Results: #{supervisor.results.results.map(&:result)}"
325
- puts "Errors: #{supervisor.results.errors.size}"
82
+ # => Results: [2, 4, 6]
326
83
  ----
327
84
 
328
- That's it! With these three simple steps, you have a working parallel processing
329
- system using Fractor in pipeline mode.
85
+ == Key features
330
86
 
87
+ * *Function-driven*: Define processing logic by subclassing `Fractor::Worker`
88
+ * *Parallel execution*: Work automatically distributed across Ractor workers
89
+ * *Two operating modes*:
90
+ ** **Pipeline mode** for batch processing
91
+ ** **Continuous mode** for long-running servers
92
+ * *Workflow system*: GitHub Actions-style declarative workflows
93
+ * *Error handling*: Retry logic, circuit breakers, dead letter queues, error reporting
94
+ * *Production-ready*: Signal handling, logging, monitoring, graceful shutdown
95
+ * *Performance tools*: Built-in monitoring, benchmarking, and error analytics
96
+ * *High-level primitives*: WorkQueue and ContinuousServer eliminate boilerplate
331
97
 
98
+ == Documentation
332
99
 
100
+ === Getting started
333
101
 
334
- == Quick start: Continuous mode
102
+ * link:docs/_pages/installation.adoc[Installation] - System requirements and installation methods
103
+ * link:docs/_pages/getting-started.adoc[Getting Started] - Quick start guides for both modes
104
+ * link:docs/_pages/core-concepts.adoc[Core Concepts] - Understanding Fractor components
335
105
 
336
- === General
106
+ === Operating modes
337
107
 
338
- This quick start guide shows how to build a long-running server using Fractor's
339
- high-level primitives for continuous mode. These primitives eliminate boilerplate
340
- code for thread management, queuing, and results processing.
108
+ * link:docs/_guides/pipeline-mode.adoc[Pipeline Mode] - Batch processing with predefined work
109
+ * link:docs/_guides/continuous-mode.adoc[Continuous Mode] - Long-running servers and streaming
341
110
 
342
- === Step 1: Create Work and Worker classes
111
+ === Advanced features
343
112
 
344
- Just like pipeline mode, you need Work and Worker classes:
113
+ * link:docs/_features/workflows.adoc[Workflows] - Declarative workflow system for complex pipelines
114
+ * link:docs/_features/error-handling.adoc[Error Handling] - Retry logic, circuit breakers, and dead letter queues
115
+ * link:docs/_features/monitoring.adoc[Monitoring] - Performance monitoring and metrics
116
+ * link:docs/_features/signal-handling.adoc[Signal Handling] - Process monitoring and graceful shutdown
345
117
 
346
- [source,ruby]
347
- ----
348
- require 'fractor'
349
-
350
- class MessageWork < Fractor::Work
351
- def initialize(client_id, message)
352
- super({ client_id: client_id, message: message })
353
- end
118
+ === Reference
354
119
 
355
- def client_id
356
- input[:client_id]
357
- end
120
+ * link:docs/_reference/api.adoc[API Reference] - Complete API documentation
121
+ * link:docs/_reference/examples.adoc[Examples] - Complete examples for all patterns
122
+ * link:docs/_reference/troubleshooting.adoc[Troubleshooting] - Common issues and solutions
358
123
 
359
- def message
360
- input[:message]
361
- end
362
- end
124
+ == Operating modes
363
125
 
364
- class MessageWorker < Fractor::Worker
365
- def process(work)
366
- # Process the message
367
- processed = "Echo: #{work.message}"
126
+ Fractor supports two distinct modes:
368
127
 
369
- Fractor::WorkResult.new(
370
- result: { client_id: work.client_id, response: processed },
371
- work: work
372
- )
373
- rescue => e
374
- Fractor::WorkResult.new(error: e.message, work: work)
375
- end
376
- end
377
- ----
128
+ [cols="1,2,2",options="header"]
129
+ |===
130
+ |Mode |Best for |Example use cases
378
131
 
379
- === Step 2: Set up WorkQueue
132
+ |*Pipeline Mode*
133
+ |Batch processing, one-time jobs
134
+ |File processing, ETL pipelines, data transformations
380
135
 
381
- Create a thread-safe work queue that will hold incoming work items:
136
+ |*Continuous Mode*
137
+ |Long-running servers, streaming
138
+ |Chat servers, job processors, event streams
139
+ |===
382
140
 
383
- [source,ruby]
384
- ----
385
- # Create a thread-safe work queue
386
- work_queue = Fractor::WorkQueue.new
387
- ----
141
+ See link:docs/getting-started.adoc#choosing-your-mode[Choosing Your Mode] for detailed guidance.
388
142
 
389
- === Step 3: Set up ContinuousServer with callbacks
143
+ == Example applications
390
144
 
391
- The ContinuousServer handles all the boilerplate: thread management, signal
392
- handling, and results processing.
145
+ === Pipeline mode
393
146
 
394
147
  [source,ruby]
395
148
  ----
396
- # Create the continuous server
397
- server = Fractor::ContinuousServer.new(
398
- worker_pools: [
399
- { worker_class: MessageWorker, num_workers: 4 }
400
- ],
401
- work_queue: work_queue, # Auto-registers as work source
402
- log_file: 'logs/server.log' # Optional logging
149
+ supervisor = Fractor::Supervisor.new(
150
+ worker_pools: [{ worker_class: DataWorker }]
403
151
  )
404
152
 
405
- # Define how to handle successful results
406
- server.on_result do |result|
407
- client_id = result.result[:client_id]
408
- response = result.result[:response]
409
- puts "Sending to client #{client_id}: #{response}"
410
- # Send response to client here
411
- end
412
-
413
- # Define how to handle errors
414
- server.on_error do |error_result|
415
- puts "Error processing work: #{error_result.error}"
416
- end
417
- ----
418
-
419
- === Step 4: Run and add work dynamically
420
-
421
- Start the server and add work items as they arrive:
422
-
423
- [source,ruby]
424
- ----
425
- # Start the server in a background thread
426
- server_thread = Thread.new { server.run }
427
-
428
- # Your application can now push work items dynamically
429
- # For example, when a client sends a message:
430
- work_queue << MessageWork.new(client_id: 1, message: "Hello")
431
- work_queue << MessageWork.new(client_id: 2, message: "World")
432
-
433
- # The server runs indefinitely, processing work as it arrives
434
- # Use Ctrl+C or send SIGTERM for graceful shutdown
435
-
436
- # Or stop programmatically
437
- sleep 10
438
- server.stop
439
- server_thread.join
440
- ----
441
-
442
- That's it! The ContinuousServer handles all thread management, signal handling,
443
- and graceful shutdown automatically.
444
-
445
-
446
-
447
-
448
- == Core components
449
-
450
- === General
451
-
452
- The Fractor framework consists of the following main classes, all residing
453
- within the `Fractor` module. These core components are used by both pipeline
454
- mode and continuous mode.
455
-
456
-
457
- === Fractor::Worker
458
-
459
- The abstract base class for defining how work should be processed.
460
-
461
- Client code must subclass this and implement the `process(work)` method.
462
-
463
- The `process` method receives a `Fractor::Work` object (or a subclass) and
464
- should return a `Fractor::WorkResult` object.
465
-
466
- === Fractor::Work
467
-
468
- The abstract base class for representing a unit of work.
469
-
470
- Typically holds the input data needed by the `Worker`.
471
-
472
- Client code should subclass this to define specific types of work items.
473
-
474
- === Fractor::WorkResult
475
-
476
- A container object returned by the `Worker#process` method.
477
-
478
- Holds either the successful `:result` of the computation or an `:error`
479
- message if processing failed.
480
-
481
- Includes a reference back to the original `:work` item.
482
-
483
- Provides a `success?` method.
484
-
485
- === Fractor::ResultAggregator
486
-
487
- Collects and stores all `WorkResult` objects generated by the workers.
488
-
489
- Separates results into `results` (successful) and `errors` arrays.
490
-
491
- === Fractor::WrappedRactor
492
-
493
- Manages an individual Ruby `Ractor`.
494
-
495
- Instantiates the client-provided `Worker` subclass within the Ractor.
496
-
497
- Handles receiving `Work` items, calling the `Worker#process` method, and
498
- yielding `WorkResult` objects (or errors) back to the `Supervisor`.
499
-
500
- === Fractor::Supervisor
501
-
502
- The main orchestrator of the framework.
503
-
504
- Initializes and manages a pool of `WrappedRactor` instances.
505
-
506
- Manages a `work_queue` of input data.
507
-
508
- Distributes work items (wrapped in the client's `Work` subclass) to available
509
- Ractors.
510
-
511
- Listens for results and errors from Ractors using `Ractor.select`.
512
-
513
- Uses `ResultAggregator` to store outcomes.
514
-
515
- Handles graceful shutdown on `SIGINT` (Ctrl+C).
516
-
517
-
518
-
519
-
520
- == Pipeline mode components
521
-
522
- === General
523
-
524
- This section describes the components and their detailed usage specifically for
525
- pipeline mode (batch processing). For continuous mode, see the Continuous mode
526
- components section.
527
-
528
- Pipeline mode uses only the core components without any additional primitives.
529
-
530
- === Work class
531
-
532
- ==== Purpose and responsibilities
533
-
534
- The `Fractor::Work` class represents a unit of work to be processed by a Worker.
535
- Its primary responsibility is to encapsulate the input data needed for
536
- processing.
537
-
538
- ==== Implementation requirements
539
-
540
- At minimum, your Work subclass should:
541
-
542
- . Inherit from `Fractor::Work`
543
- . Pass the input data to the superclass constructor
153
+ supervisor.add_work_items(dataset.map { |item| DataWork.new(item) })
154
+ supervisor.run
544
155
 
545
- [source,ruby]
546
- ----
547
- class MyWork < Fractor::Work
548
- def initialize(input)
549
- super(input) # This stores input in @input
550
- # Add any additional initialization if needed
551
- end
552
- end
156
+ puts "Processed #{supervisor.results.results.size} items"
553
157
  ----
554
158
 
555
- ==== Advanced usage
159
+ See link:examples/simple/[Simple Example] and link:docs/examples.adoc#pipeline-mode-examples[more examples].
556
160
 
557
- You can extend your Work class to include additional data or methods:
161
+ === Continuous mode
558
162
 
559
163
  [source,ruby]
560
164
  ----
561
- class ComplexWork < Fractor::Work
562
- attr_reader :options
563
-
564
- def initialize(input, options = {})
565
- super(input)
566
- @options = options
567
- end
568
-
569
- def high_priority?
570
- @options[:priority] == :high
571
- end
572
-
573
- def to_s
574
- "ComplexWork: #{@input} (#{@options[:priority]} priority)"
575
- end
576
- end
577
- ----
578
-
579
- [TIP]
580
- ====
581
- * Keep Work objects lightweight and serializable since they will be passed
582
- between Ractors
583
- * Implement a meaningful `to_s` method for better debugging
584
- * Consider adding validation in the initializer to catch issues early
585
- ====
586
-
587
- === Worker class
588
-
589
- ==== Purpose and responsibilities
590
-
591
- The `Fractor::Worker` class defines the processing logic for work items. Each
592
- Worker instance runs within its own Ractor and processes Work objects sent to
593
- it.
594
-
595
- ==== Implementation requirements
596
-
597
- Your Worker subclass must:
598
-
599
- . Inherit from `Fractor::Worker`
600
- . Implement the `process(work)` method
601
- . Return a `Fractor::WorkResult` object from the `process` method
602
- . Handle both successful processing and error conditions
165
+ work_queue = Fractor::WorkQueue.new
603
166
 
604
- [source,ruby]
605
- ----
606
- class MyWorker < Fractor::Worker
607
- def process(work)
608
- # Process the work
167
+ server = Fractor::ContinuousServer.new(
168
+ worker_pools: [{ worker_class: MessageWorker, num_workers: 4 }],
169
+ work_queue: work_queue
170
+ )
609
171
 
610
- if work.input < 0
611
- return Fractor::WorkResult.new(
612
- error: "Cannot process negative numbers",
613
- work: work
614
- )
615
- end
172
+ server.on_result { |result| puts "Processed: #{result.result}" }
173
+ server.on_error { |error| puts "Error: #{error.error}" }
616
174
 
617
- # Normal processing...
618
- result = work.input * 2
175
+ Thread.new { server.run }
619
176
 
620
- # Return a WorkResult
621
- Fractor::WorkResult.new(result: result, work: work)
622
- end
623
- end
177
+ # Add work dynamically
178
+ work_queue << MessageWork.new(client_id: 1, message: "Hello")
624
179
  ----
625
180
 
181
+ See link:examples/continuous_chat_fractor/[Chat Server Example] and link:docs/examples.adoc#continuous-mode-examples[more examples].
626
182
 
627
- ==== Error handling
628
-
629
- The Worker class should handle two types of errors.
630
-
631
-
632
- ===== Handled errors
633
-
634
- These are expected error conditions that your code explicitly checks for.
183
+ === Workflows
635
184
 
636
185
  [source,ruby]
637
186
  ----
638
- def process(work)
639
- if work.input < 0
640
- return Fractor::WorkResult.new(
641
- error: "Cannot process negative numbers",
642
- work: work
643
- )
644
- end
645
-
646
- # Normal processing...
647
- Fractor::WorkResult.new(result: calculated_value, work: work)
187
+ # Define workflow with simplified syntax
188
+ workflow = Fractor::Workflow.define("data-pipeline") do
189
+ job :extract, ExtractWorker
190
+ job :transform, TransformWorker, needs: :extract
191
+ job :load, LoadWorker, needs: :transform
648
192
  end
649
- ----
650
-
651
- ===== Unexpected errors caught by rescue
652
193
 
653
- These are unexpected exceptions that may occur during processing. You should
654
- catch these and convert them into error results.
655
-
656
- [source,ruby]
194
+ # Execute workflow
195
+ result = workflow.new.execute(input_data)
657
196
  ----
658
- def process(work)
659
- # Processing that might raise exceptions
660
- result = complex_calculation(work.input)
661
-
662
- Fractor::WorkResult.new(result: result, work: work)
663
- rescue StandardError => e
664
- # Catch and convert any unexpected exceptions to error results
665
- Fractor::WorkResult.new(
666
- error: "An unexpected error occurred: #{e.message}",
667
- work: work
668
- )
669
- end
670
- ----
671
-
672
- [TIP]
673
- ====
674
- * Keep the `process` method focused on a single responsibility
675
- * Use meaningful error messages that help diagnose issues
676
- * Consider adding logging within the `process` method for debugging
677
- * Ensure all paths return a valid `WorkResult` object
678
- ====
679
-
680
- === Supervisor class for pipeline mode
681
197
 
682
- ==== Purpose and responsibilities
198
+ See link:examples/workflow/simplified/README.adoc[Simplified Workflows] and link:docs/workflows.adoc[Workflow Guide].
683
199
 
684
- The `Fractor::Supervisor` class orchestrates the entire framework, managing
685
- worker Ractors, distributing work, and collecting results.
200
+ == Production deployment
686
201
 
687
- ==== Configuration options
202
+ Fractor includes production-ready features:
688
203
 
689
- When creating a Supervisor for pipeline mode, configure worker pools:
204
+ * **Signal handling**: SIGTERM, SIGINT, SIGUSR1/SIGBREAK
205
+ * **Graceful shutdown**: Complete in-progress work before exit
206
+ * **Process monitoring**: Runtime status via signals
207
+ * **Structured logging**: JSON logging with correlation IDs
208
+ * **Workflow visualization**: Mermaid, DOT, ASCII diagrams
690
209
 
691
- [source,ruby]
692
- ----
693
- supervisor = Fractor::Supervisor.new(
694
- worker_pools: [
695
- # Pool 1 - for general data processing
696
- { worker_class: MyWorker, num_workers: 4 },
697
-
698
- # Pool 2 - for specialized image processing
699
- { worker_class: ImageWorker, num_workers: 2 }
700
- ]
701
- # Note: continuous_mode defaults to false for pipeline mode
702
- )
703
- ----
704
-
705
- ==== Worker auto-detection
210
+ See link:docs/signal-handling.adoc[Signal Handling Guide] for deployment patterns.
706
211
 
707
- Fractor automatically detects the number of available processors on your system
708
- and uses that value when `num_workers` is not specified. This provides optimal
709
- resource utilization across different deployment environments without requiring
710
- manual configuration.
711
-
712
- [source,ruby]
713
- ----
714
- # Auto-detect number of workers (recommended for most cases)
715
- supervisor = Fractor::Supervisor.new(
716
- worker_pools: [
717
- { worker_class: MyWorker } # Will use number of available processors
718
- ]
719
- )
212
+ == Contributing
720
213
 
721
- # Explicitly set number of workers (useful for specific requirements)
722
- supervisor = Fractor::Supervisor.new(
723
- worker_pools: [
724
- { worker_class: MyWorker, num_workers: 4 } # Always use exactly 4 workers
725
- ]
726
- )
214
+ Bug reports and pull requests are welcome on GitHub at https://github.com/metanorma/fractor.
727
215
 
728
- # Mix auto-detection and explicit configuration
729
- supervisor = Fractor::Supervisor.new(
730
- worker_pools: [
731
- { worker_class: FastWorker }, # Auto-detected
732
- { worker_class: HeavyWorker, num_workers: 2 } # Explicitly 2 workers
733
- ]
734
- )
735
- ----
216
+ == License
736
217
 
737
- The auto-detection uses Ruby's `Etc.nprocessors` which returns the number of
738
- available processors. If detection fails for any reason, it falls back to 2
739
- workers.
218
+ The gem is available as open source under the terms of the Ribose BSD 2-Clause License.
740
219
 
741
- [TIP]
742
- ====
743
- * Use auto-detection for portable code that adapts to different environments
744
- * Explicitly set `num_workers` when you need precise control over resource usage
745
- * Consider system load and other factors when choosing explicit values
746
- ====
747
-
748
- ==== Adding work
749
-
750
- You can add work items individually or in batches:
751
-
752
- [source,ruby]
753
- ----
754
- # Add a single item
755
- supervisor.add_work_item(MyWork.new(42))
756
-
757
- # Add multiple items
758
- supervisor.add_work_items([
759
- MyWork.new(1),
760
- MyWork.new(2),
761
- MyWork.new(3),
762
- MyWork.new(4),
763
- MyWork.new(5)
764
- ])
765
-
766
- # Add items of different work types
767
- supervisor.add_work_items([
768
- TextWork.new("Process this text"),
769
- ImageWork.new({ width: 800, height: 600 })
770
- ])
771
- ----
772
-
773
- The Supervisor can handle any Work object that inherits from Fractor::Work.
774
- Workers must check the type of Work they receive and process it accordingly.
775
-
776
- ==== Running and monitoring
777
-
778
- To start processing:
779
-
780
- [source,ruby]
781
- ----
782
- # Start processing and block until complete
783
- supervisor.run
784
- ----
785
-
786
- The Supervisor automatically handles:
787
-
788
- * Starting the worker Ractors
789
- * Distributing work items to available workers
790
- * Collecting results and errors
791
- * Graceful shutdown on completion or interruption (Ctrl+C)
792
-
793
- === ResultAggregator for pipeline mode
794
-
795
- ==== Purpose and responsibilities
796
-
797
- The `Fractor::ResultAggregator` collects and organizes all results from the
798
- workers, separating successful results from errors.
799
-
800
- In pipeline mode, results are collected throughout processing and accessed
801
- after the supervisor finishes running.
802
-
803
- ==== Accessing results
804
-
805
- After processing completes:
806
-
807
- [source,ruby]
808
- ----
809
- # Get the ResultAggregator
810
- aggregator = supervisor.results
811
-
812
- # Check counts
813
- puts "Processed #{aggregator.results.size} items successfully"
814
- puts "Encountered #{aggregator.errors.size} errors"
815
-
816
- # Access successful results
817
- aggregator.results.each do |result|
818
- puts "Work item #{result.work.input} produced #{result.result}"
819
- end
820
-
821
- # Access errors
822
- aggregator.errors.each do |error_result|
823
- puts "Work item #{error_result.work.input} failed: #{error_result.error}"
824
- end
825
- ----
826
-
827
- To access successful results:
828
-
829
- [source,ruby]
830
- ----
831
- # Get all successful results
832
- successful_results = supervisor.results.results
833
-
834
- # Extract just the result values
835
- result_values = successful_results.map(&:result)
836
- ----
837
-
838
- To access errors:
839
-
840
- [source,ruby]
841
- ----
842
- # Get all error results
843
- error_results = supervisor.results.errors
844
-
845
- # Extract error messages
846
- error_messages = error_results.map(&:error)
847
-
848
- # Get the work items that failed
849
- failed_work_items = error_results.map(&:work)
850
- ----
851
-
852
-
853
- [TIP]
854
- ====
855
- * Check both successful results and errors after processing completes
856
- * Consider implementing custom reporting based on the aggregated results
857
- ====
858
-
859
-
860
-
861
-
862
- == Pipeline mode patterns
863
-
864
- === Custom work distribution
865
-
866
- For more complex scenarios, you might want to prioritize certain work items:
867
-
868
- [source,ruby]
869
- ----
870
- # Create Work objects for high priority items
871
- high_priority_works = high_priority_items.map { |item| MyWork.new(item) }
872
-
873
- # Add high-priority items first
874
- supervisor.add_work_items(high_priority_works)
875
-
876
- # Run with just enough workers for high-priority items
877
- supervisor.run
878
-
879
- # Create Work objects for lower priority items
880
- low_priority_works = low_priority_items.map { |item| MyWork.new(item) }
881
-
882
- # Add and process lower-priority items
883
- supervisor.add_work_items(low_priority_works)
884
- supervisor.run
885
- ----
886
-
887
- === Handling large datasets
888
-
889
- For very large datasets, consider processing in batches:
890
-
891
- [source,ruby]
892
- ----
893
- large_dataset.each_slice(1000) do |batch|
894
- # Convert batch items to Work objects
895
- work_batch = batch.map { |item| MyWork.new(item) }
896
-
897
- supervisor.add_work_items(work_batch)
898
- supervisor.run
899
-
900
- # Process this batch's results before continuing
901
- process_batch_results(supervisor.results)
902
- end
903
- ----
904
-
905
- === Multi-work type processing
906
-
907
- The Multi-Work Type pattern demonstrates how a single supervisor and worker can
908
- handle multiple types of work items.
909
-
910
- [source,ruby]
911
- ----
912
- class UniversalWorker < Fractor::Worker
913
- def process(work)
914
- case work
915
- when TextWork
916
- process_text(work)
917
- when ImageWork
918
- process_image(work)
919
- else
920
- Fractor::WorkResult.new(
921
- error: "Unknown work type: #{work.class}",
922
- work: work
923
- )
924
- end
925
- end
926
-
927
- private
928
-
929
- def process_text(work)
930
- result = work.text.upcase
931
- Fractor::WorkResult.new(result: result, work: work)
932
- end
933
-
934
- def process_image(work)
935
- result = { width: work.width * 2, height: work.height * 2 }
936
- Fractor::WorkResult.new(result: result, work: work)
937
- end
938
- end
939
-
940
- # Add different types of work
941
- supervisor.add_work_items([
942
- TextWork.new("hello"),
943
- ImageWork.new(width: 100, height: 100),
944
- TextWork.new("world")
945
- ])
946
- ----
947
-
948
- === Hierarchical work processing
949
-
950
- The Producer/Subscriber pattern showcases processing that generates sub-work:
951
-
952
- [source,ruby]
953
- ----
954
- # First pass: Process documents
955
- supervisor.add_work_items(documents.map { |doc| DocumentWork.new(doc) })
956
- supervisor.run
957
-
958
- # Collect sections generated from documents
959
- sections = supervisor.results.results.flat_map do |result|
960
- result.result[:sections]
961
- end
962
-
963
- # Second pass: Process sections
964
- supervisor.add_work_items(sections.map { |section| SectionWork.new(section) })
965
- supervisor.run
966
- ----
967
-
968
- === Pipeline stages
969
-
970
- The Pipeline Processing pattern implements multi-stage transformation:
971
-
972
- [source,ruby]
973
- ----
974
- # Stage 1: Extract data
975
- supervisor1 = Fractor::Supervisor.new(
976
- worker_pools: [{ worker_class: ExtractionWorker }]
977
- )
978
- supervisor1.add_work_items(raw_data.map { |d| ExtractionWork.new(d) })
979
- supervisor1.run
980
- extracted = supervisor1.results.results.map(&:result)
981
-
982
- # Stage 2: Transform data
983
- supervisor2 = Fractor::Supervisor.new(
984
- worker_pools: [{ worker_class: TransformWorker }]
985
- )
986
- supervisor2.add_work_items(extracted.map { |e| TransformWork.new(e) })
987
- supervisor2.run
988
- transformed = supervisor2.results.results.map(&:result)
989
-
990
- # Stage 3: Load data
991
- supervisor3 = Fractor::Supervisor.new(
992
- worker_pools: [{ worker_class: LoadWorker }]
993
- )
994
- supervisor3.add_work_items(transformed.map { |t| LoadWork.new(t) })
995
- supervisor3.run
996
- ----
997
-
998
-
999
-
1000
-
1001
- == Continuous mode components
1002
-
1003
- === General
1004
-
1005
- This section describes the components and their detailed usage specifically for
1006
- continuous mode (long-running servers). For pipeline mode, see the Pipeline mode
1007
- components section.
1008
-
1009
- Continuous mode offers two approaches: a low-level API for manual control, and
1010
- high-level primitives that eliminate boilerplate code.
1011
-
1012
- === Low-level components
1013
-
1014
- ==== General
1015
-
1016
- The low-level API provides manual control over continuous mode operation. This
1017
- approach is useful when you need fine-grained control over threading, work
1018
- sources, or results processing.
1019
-
1020
- Use the low-level API when:
1021
-
1022
- * You need custom thread management
1023
- * Your work source logic is complex
1024
- * You require precise control over the supervisor lifecycle
1025
- * You're integrating with existing thread pools or event loops
1026
-
1027
- For most applications, the high-level primitives (described in the next section)
1028
- are recommended as they eliminate significant boilerplate code.
1029
-
1030
- ==== Supervisor with continuous_mode: true
1031
-
1032
- To enable continuous mode, set the `continuous_mode` option:
1033
-
1034
- [source,ruby]
1035
- ----
1036
- supervisor = Fractor::Supervisor.new(
1037
- worker_pools: [
1038
- { worker_class: MyWorker, num_workers: 2 }
1039
- ],
1040
- continuous_mode: true # Enable continuous mode
1041
- )
1042
- ----
1043
-
1044
- ==== Work source callbacks
1045
-
1046
- Register a callback that provides new work on demand:
1047
-
1048
- [source,ruby]
1049
- ----
1050
- supervisor.register_work_source do
1051
- # Return nil or empty array if no work is available
1052
- # Return a work item or array of work items when available
1053
- items = get_next_work_items
1054
- if items && !items.empty?
1055
- # Convert to Work objects if needed
1056
- items.map { |item| MyWork.new(item) }
1057
- else
1058
- nil
1059
- end
1060
- end
1061
- ----
1062
-
1063
- The callback is polled every 100ms by an internal timer thread.
1064
-
1065
- ==== Manual thread management
1066
-
1067
- You must manually manage threads and results processing:
1068
-
1069
- [source,ruby]
1070
- ----
1071
- # Start supervisor in a background thread
1072
- supervisor_thread = Thread.new { supervisor.run }
1073
-
1074
- # Start results processing thread
1075
- results_thread = Thread.new do
1076
- loop do
1077
- # Process results
1078
- while (result = supervisor.results.results.shift)
1079
- handle_result(result)
1080
- end
1081
-
1082
- # Process errors
1083
- while (error = supervisor.results.errors.shift)
1084
- handle_error(error)
1085
- end
1086
-
1087
- sleep 0.1
1088
- end
1089
- end
1090
-
1091
- # Ensure cleanup on shutdown
1092
- begin
1093
- supervisor_thread.join
1094
- rescue Interrupt
1095
- supervisor.stop
1096
- ensure
1097
- results_thread.kill
1098
- supervisor_thread.join
1099
- end
1100
- ----
1101
-
1102
- === High-level components
1103
-
1104
- ==== General
1105
-
1106
- Fractor provides high-level primitives that dramatically simplify continuous
1107
- mode applications by eliminating boilerplate code.
1108
-
1109
- These primitives solve common problems:
1110
-
1111
- * *Thread management*: Automatic supervisor and results processing threads
1112
- * *Queue synchronization*: Thread-safe work queue with automatic integration
1113
- * *Results processing*: Callback-based handling instead of manual loops
1114
- * *Signal handling*: Built-in support for SIGINT, SIGTERM, SIGUSR1/SIGBREAK
1115
- * *Graceful shutdown*: Coordinated cleanup across all threads
1116
-
1117
- Real-world benefits:
1118
-
1119
- * The chat server example reduced from 279 lines to 167 lines (40% reduction)
1120
- * Eliminates ~112 lines of thread, queue, and signal handling boilerplate
1121
- * Simpler, more maintainable code with fewer error-prone details
1122
-
1123
- ==== Fractor::WorkQueue
1124
-
1125
- ===== Purpose and responsibilities
1126
-
1127
- `Fractor::WorkQueue` provides a thread-safe queue for continuous mode
1128
- applications. It handles work item storage and integrates automatically with the
1129
- supervisor's work source mechanism.
1130
-
1131
- ===== Thread-safety
1132
-
1133
- The WorkQueue is *thread-safe* but not *Ractor-safe*:
1134
-
1135
- * *Thread-safe*: Multiple threads can safely push work items concurrently
1136
- * *Not Ractor-safe*: The queue lives in the main process and cannot be shared
1137
- across Ractor boundaries
1138
-
1139
- This design is intentional. The WorkQueue operates in the main process where
1140
- your application code runs. Work items are retrieved by the Supervisor (also in
1141
- the main process) and then sent to worker Ractors.
1142
-
1143
- .WorkQueue architecture
1144
- [source]
1145
- ----
1146
- Main Process
1147
- ├─→ Your application threads (push to WorkQueue)
1148
- ├─→ WorkQueue (thread-safe, lives here)
1149
- ├─→ Supervisor (polls WorkQueue)
1150
- │ └─→ Sends work to Worker Ractors
1151
- └─→ Worker Ractors (receive frozen/shareable work items)
1152
- ----
1153
-
1154
- ===== Creating a WorkQueue
1155
-
1156
- [source,ruby]
1157
- ----
1158
- work_queue = Fractor::WorkQueue.new
1159
- ----
1160
-
1161
- ===== Adding work items
1162
-
1163
- Use the `<<` operator for thread-safe push operations:
1164
-
1165
- [source,ruby]
1166
- ----
1167
- # From any thread in your application
1168
- work_queue << MyWork.new(data)
1169
-
1170
- # Thread-safe even from multiple threads
1171
- threads = 10.times.map do |i|
1172
- Thread.new do
1173
- 100.times do |j|
1174
- work_queue << MyWork.new("thread-#{i}-item-#{j}")
1175
- end
1176
- end
1177
- end
1178
- threads.each(&:join)
1179
- ----
1180
-
1181
- ===== Checking queue status
1182
-
1183
- [source,ruby]
1184
- ----
1185
- # Check if queue is empty
1186
- if work_queue.empty?
1187
- puts "No work available"
1188
- end
1189
-
1190
- # Get current queue size
1191
- puts "Queue has #{work_queue.size} items"
1192
- ----
1193
-
1194
- ===== Integration with Supervisor
1195
-
1196
- The WorkQueue integrates automatically with ContinuousServer (see next section).
1197
- For manual integration with a Supervisor:
1198
-
1199
- [source,ruby]
1200
- ----
1201
- supervisor = Fractor::Supervisor.new(
1202
- worker_pools: [{ worker_class: MyWorker }],
1203
- continuous_mode: true
1204
- )
1205
-
1206
- # Register the work queue as a work source
1207
- work_queue.register_with_supervisor(supervisor)
1208
-
1209
- # Now the supervisor will automatically poll the queue for work
1210
- ----
1211
-
1212
- ==== Fractor::ContinuousServer
1213
-
1214
- ===== Purpose and responsibilities
1215
-
1216
- `Fractor::ContinuousServer` is a high-level wrapper that handles all the
1217
- complexity of running a continuous mode application. It manages:
1218
-
1219
- * Supervisor thread lifecycle
1220
- * Results processing thread with callback system
1221
- * Signal handling (SIGINT, SIGTERM, SIGUSR1/SIGBREAK)
1222
- * Graceful shutdown coordination
1223
- * Optional logging
1224
-
1225
- ===== Creating a ContinuousServer
1226
-
1227
- [source,ruby]
1228
- ----
1229
- server = Fractor::ContinuousServer.new(
1230
- worker_pools: [
1231
- { worker_class: MessageWorker, num_workers: 4 }
1232
- ],
1233
- work_queue: work_queue, # Optional, auto-registers if provided
1234
- log_file: 'logs/server.log' # Optional
1235
- )
1236
- ----
1237
-
1238
- Parameters:
1239
-
1240
- * `worker_pools` (required): Array of worker pool configurations
1241
- * `work_queue` (optional): A Fractor::WorkQueue instance to auto-register
1242
- * `log_file` (optional): Path for log output
1243
-
1244
- ===== Registering callbacks
1245
-
1246
- Define how to handle results and errors:
1247
-
1248
- [source,ruby]
1249
- ----
1250
- # Handle successful results
1251
- server.on_result do |result|
1252
- # result is a Fractor::WorkResult with result.result containing your data
1253
- puts "Success: #{result.result}"
1254
- # Send response to client, update database, etc.
1255
- end
1256
-
1257
- # Handle errors
1258
- server.on_error do |error_result|
1259
- # error_result is a Fractor::WorkResult with error_result.error containing the message
1260
- puts "Error: #{error_result.error}"
1261
- # Log error, send notification, etc.
1262
- end
1263
- ----
1264
-
1265
- ===== Running the server
1266
-
1267
- [source,ruby]
1268
- ----
1269
- # Blocking: Run the server (blocks until shutdown signal)
1270
- server.run
1271
-
1272
- # Non-blocking: Run in background thread
1273
- server_thread = Thread.new { server.run }
1274
-
1275
- # Your application continues here...
1276
- # Add work to queue as needed
1277
- work_queue << MyWork.new(data)
1278
-
1279
- # Later, stop the server
1280
- server.stop
1281
- server_thread.join
1282
- ----
1283
-
1284
- ===== Signal handling
1285
-
1286
- The ContinuousServer automatically handles:
1287
-
1288
- * *SIGINT* (Ctrl+C): Graceful shutdown
1289
- * *SIGTERM*: Graceful shutdown (production deployment)
1290
- * *SIGUSR1* (Unix) / *SIGBREAK* (Windows): Status output
1291
-
1292
- No additional code needed - signals work automatically.
1293
-
1294
- ===== Graceful shutdown
1295
-
1296
- When a shutdown signal is received:
1297
-
1298
- . Stops accepting new work from the work queue
1299
- . Allows in-progress work to complete (within ~2 seconds)
1300
- . Processes remaining results through callbacks
1301
- . Cleans up all threads and resources
1302
- . Returns from the `run` method
1303
-
1304
- ===== Programmatic shutdown
1305
-
1306
- [source,ruby]
1307
- ----
1308
- # Stop the server programmatically
1309
- server.stop
1310
-
1311
- # The run method will return shortly after
1312
- ----
1313
-
1314
- ==== Integration architecture
1315
-
1316
- The high-level components work together seamlessly:
1317
-
1318
- .Complete architecture diagram
1319
- [source]
1320
- ----
1321
- ┌───────────────────────────────────────────────────────────┐
1322
- │ Main Process │
1323
- │ │
1324
- │ ┌──────────────┐ ┌──────────────────────────────┐ │
1325
- │ │ Your App │────>│ WorkQueue (thread-safe) │ │
1326
- │ │ (any thread) │ │ - Thread::Queue internally │ │
1327
- │ └──────────────┘ └──────────────────────────────┘ │
1328
- │ │ │
1329
- │ │ polled every 100ms │
1330
- │ ▼ │
1331
- │ ┌────────────────────────────────────────────────────┐ │
1332
- │ │ ContinuousServer │ │
1333
- │ │ ┌─────────────────────────────────────────────┐ │ │
1334
- │ │ │ Supervisor Thread │ │ │
1335
- │ │ │ - Manages worker Ractors │ │ │
1336
- │ │ │ - Distributes work │ │ │
1337
- │ │ │ - Coordinates shutdown │ │ │
1338
- │ │ └─────────────────────────────────────────────┘ │ │
1339
- │ │ │ │ │
1340
- │ │ ▼ │ │
1341
- │ │ ┌─────────────────────────────────────────────┐ │ │
1342
- │ │ │ Worker Ractors (parallel execution) │ │ │
1343
- │ │ │ - Ractor 1: WorkerInstance.process(work) │ │ │
1344
- │ │ │ - Ractor 2: WorkerInstance.process(work) │ │ │
1345
- │ │ │ - Ractor N: WorkerInstance.process(work) │ │ │
1346
- │ │ └─────────────────────────────────────────────┘ │ │
1347
- │ │ │ │ │
1348
- │ │ ▼ (WorkResults) │ │
1349
- │ │ ┌─────────────────────────────────────────────┐ │ │
1350
- │ │ │ Results Processing Thread │ │ │
1351
- │ │ │ - on_result callback for successes │ │ │
1352
- │ │ │ - on_error callback for failures │ │ │
1353
- │ │ └─────────────────────────────────────────────┘ │ │
1354
- │ │ │ │
1355
- │ │ ┌─────────────────────────────────────────────┐ │ │
1356
- │ │ │ Signal Handler Thread │ │ │
1357
- │ │ │ - SIGINT/SIGTERM: Shutdown │ │ │
1358
- │ │ │ - SIGUSR1/SIGBREAK: Status │ │ │
1359
- │ │ └─────────────────────────────────────────────┘ │ │
1360
- │ └────────────────────────────────────────────────────┘ │
1361
- └───────────────────────────────────────────────────────────┘
1362
- ----
1363
-
1364
- Key points:
1365
-
1366
- * WorkQueue lives in main process (thread-safe, not Ractor-safe)
1367
- * Supervisor polls WorkQueue and distributes to Ractors
1368
- * Work items must be frozen/shareable to cross Ractor boundary
1369
- * Results come back through callbacks, not batch collection
1370
- * All thread management is automatic
1371
-
1372
-
1373
-
1374
-
1375
- == Continuous mode patterns
1376
-
1377
- === Basic server with callbacks
1378
-
1379
- The most common pattern uses WorkQueue + ContinuousServer:
1380
-
1381
- [source,ruby]
1382
- ----
1383
- require 'fractor'
1384
-
1385
- # Define work and worker
1386
- class RequestWork < Fractor::Work
1387
- def initialize(request_id, data)
1388
- super({ request_id: request_id, data: data })
1389
- end
1390
- end
1391
-
1392
- class RequestWorker < Fractor::Worker
1393
- def process(work)
1394
- # Process the request
1395
- result = perform_computation(work.input[:data])
1396
-
1397
- Fractor::WorkResult.new(
1398
- result: { request_id: work.input[:request_id], response: result },
1399
- work: work
1400
- )
1401
- rescue => e
1402
- Fractor::WorkResult.new(error: e.message, work: work)
1403
- end
1404
-
1405
- private
1406
-
1407
- def perform_computation(data)
1408
- # Your business logic here
1409
- data.upcase
1410
- end
1411
- end
1412
-
1413
- # Set up server
1414
- work_queue = Fractor::WorkQueue.new
1415
-
1416
- server = Fractor::ContinuousServer.new(
1417
- worker_pools: [{ worker_class: RequestWorker, num_workers: 4 }],
1418
- work_queue: work_queue
1419
- )
1420
-
1421
- server.on_result { |result| puts "Success: #{result.result}" }
1422
- server.on_error { |error| puts "Error: #{error.error}" }
1423
-
1424
- # Run server (blocks until shutdown)
1425
- Thread.new { server.run }
1426
-
1427
- # Application logic adds work as needed
1428
- work_queue << RequestWork.new(1, "hello")
1429
- work_queue << RequestWork.new(2, "world")
1430
-
1431
- sleep # Keep main thread alive
1432
- ----
1433
-
1434
- === Event-driven processing
1435
-
1436
- Process events from external sources as they arrive:
1437
-
1438
- [source,ruby]
1439
- ----
1440
- # Event source (could be webhooks, message queue, etc.)
1441
- event_source = EventSource.new
1442
-
1443
- # Set up work queue and server
1444
- work_queue = Fractor::WorkQueue.new
1445
- server = Fractor::ContinuousServer.new(
1446
- worker_pools: [{ worker_class: EventWorker, num_workers: 8 }],
1447
- work_queue: work_queue
1448
- )
1449
-
1450
- server.on_result do |result|
1451
- # Publish result to subscribers
1452
- publish_event(result.result)
1453
- end
1454
-
1455
- # Event loop adds work to queue
1456
- event_source.on_event do |event|
1457
- work_queue << EventWork.new(event)
1458
- end
1459
-
1460
- # Start server
1461
- server.run
1462
- ----
1463
-
1464
- === Dynamic work sources
1465
-
1466
- Combine multiple work sources:
1467
-
1468
- [source,ruby]
1469
- ----
1470
- work_queue = Fractor::WorkQueue.new
1471
-
1472
- # Source 1: HTTP requests
1473
- http_server.on_request do |request|
1474
- work_queue << HttpWork.new(request)
1475
- end
1476
-
1477
- # Source 2: Message queue
1478
- message_queue.subscribe do |message|
1479
- work_queue << MessageWork.new(message)
1480
- end
1481
-
1482
- # Source 3: Scheduled tasks
1483
- scheduler.every('1m') do
1484
- work_queue << ScheduledWork.new(Time.now)
1485
- end
1486
-
1487
- # Single server processes all work types
1488
- server = Fractor::ContinuousServer.new(
1489
- worker_pools: [
1490
- { worker_class: HttpWorker, num_workers: 4 },
1491
- { worker_class: MessageWorker, num_workers: 2 },
1492
- { worker_class: ScheduledWorker, num_workers: 1 }
1493
- ],
1494
- work_queue: work_queue
1495
- )
1496
-
1497
- server.run
1498
- ----
1499
-
1500
- === Graceful shutdown strategies
1501
-
1502
- ==== Signal-based shutdown (production)
1503
-
1504
- [source,ruby]
1505
- ----
1506
- # Server automatically handles SIGTERM
1507
- server = Fractor::ContinuousServer.new(
1508
- worker_pools: [{ worker_class: MyWorker }],
1509
- work_queue: work_queue,
1510
- log_file: '/var/log/myapp/server.log'
1511
- )
1512
-
1513
- # Just run the server - signals handled automatically
1514
- server.run
1515
-
1516
- # In production:
1517
- # systemctl stop myapp # Sends SIGTERM
1518
- # docker stop container # Sends SIGTERM
1519
- # kill -TERM <pid> # Manual SIGTERM
1520
- ----
1521
-
1522
- ==== Time-based shutdown
1523
-
1524
- [source,ruby]
1525
- ----
1526
- server_thread = Thread.new { server.run }
1527
-
1528
- # Run for specific duration
1529
- sleep 3600 # Run for 1 hour
1530
- server.stop
1531
- server_thread.join
1532
- ----
1533
-
1534
- ==== Condition-based shutdown
1535
-
1536
- [source,ruby]
1537
- ----
1538
- server_thread = Thread.new { server.run }
1539
-
1540
- # Monitor thread checks conditions
1541
- monitor = Thread.new do
1542
- loop do
1543
- if should_shutdown?
1544
- server.stop
1545
- break
1546
- end
1547
- sleep 10
1548
- end
1549
- end
1550
-
1551
- server_thread.join
1552
- monitor.kill
1553
- ----
1554
-
1555
- === Before/after comparison
1556
-
1557
- The chat server example demonstrates the real-world impact of using the
1558
- high-level primitives.
1559
-
1560
- ==== Before: Low-level API (279 lines)
1561
-
1562
- Required manual management of:
1563
-
1564
- * Supervisor thread creation and lifecycle (~15 lines)
1565
- * Results processing thread with loops (~50 lines)
1566
- * Queue creation and synchronization (~10 lines)
1567
- * Signal handling setup (~15 lines)
1568
- * Thread coordination and shutdown (~20 lines)
1569
- * IO.select event loop (~110 lines)
1570
- * Manual error handling throughout (~59 lines)
1571
-
1572
- ==== After: High-level primitives (167 lines)
1573
-
1574
- Eliminated boilerplate:
1575
-
1576
- * WorkQueue handles queue and synchronization (automatic)
1577
- * ContinuousServer manages all threads (automatic)
1578
- * Callbacks replace manual results loops (automatic)
1579
- * Signal handling built-in (automatic)
1580
- * Graceful shutdown coordinated (automatic)
1581
-
1582
- Result: **40% code reduction** (112 fewer lines), simpler architecture, fewer
1583
- error-prone details.
1584
-
1585
- See link:examples/continuous_chat_fractor/chat_server.rb[the refactored chat
1586
- server] for the complete example.
1587
-
1588
-
1589
-
1590
-
1591
- == Process monitoring and logging
1592
-
1593
- === Status monitoring and health checks
1594
-
1595
- The signals SIGUSR1 (or SIGBREAK on Windows) can be used for health checks.
1596
-
1597
- When the signal is received, the supervisor prints its current status to
1598
- standard output.
1599
-
1600
- [example]
1601
- Sending the signal:
1602
-
1603
- Unix:
1604
-
1605
- [source,sh]
1606
- ----
1607
- # Send SIGUSR1 to the supervisor process
1608
- kill -USR1 <pid>
1609
- ----
1610
-
1611
- Windows:
1612
-
1613
- [source,sh]
1614
- ----
1615
- # Send SIGBREAK to the supervisor process
1616
- kill -BREAK <pid>
1617
- ----
1618
-
1619
- Output:
1620
-
1621
- [source]
1622
- ----
1623
- === Fractor Supervisor Status ===
1624
- Mode: Continuous
1625
- Running: true
1626
- Workers: 4
1627
- Idle workers: 2
1628
- Queue size: 15
1629
- Results: 127
1630
- Errors: 3
1631
- ----
1632
-
1633
- === Logging
1634
-
1635
- Fractor supports logging of its operations to a specified log file.
1636
-
1637
- For ContinuousServer, pass the `log_file` parameter:
1638
-
1639
- [source,ruby]
1640
- ----
1641
- server = Fractor::ContinuousServer.new(
1642
- worker_pools: [{ worker_class: MyWorker }],
1643
- work_queue: work_queue,
1644
- log_file: 'logs/server.log'
1645
- )
1646
- ----
1647
-
1648
- For manual Supervisor usage, set the `FRACTOR_LOG_FILE` environment variable
1649
- before starting your application:
1650
-
1651
- [source,sh]
1652
- ----
1653
- export FRACTOR_LOG_FILE=/path/to/logs/server.log
1654
- ruby my_fractor_app.rb
1655
- ----
1656
-
1657
- The log file will contain detailed information about the supervisor's
1658
- operations, including worker activity, work distribution, results, and errors.
1659
-
1660
- .Examples of accessing logs
1661
- [example]
1662
- [source,sh]
1663
- ----
1664
- # Check if server is responsive (Unix/Linux/macOS)
1665
- kill -USR1 <pid> && tail -f /path/to/logs/server.log
1666
-
1667
- # Monitor with systemd
1668
- systemctl status fractor-server
1669
- journalctl -u fractor-server -f
1670
-
1671
- # Monitor with Docker
1672
- docker logs -f <container_id>
1673
- ----
1674
-
1675
-
1676
-
1677
-
1678
- == Signal handling
1679
-
1680
- === General
1681
-
1682
- Fractor provides production-ready signal handling for process control and
1683
- monitoring. The framework supports different signals depending on the operating
1684
- system, enabling graceful shutdown and runtime status monitoring.
1685
-
1686
- === Unix signals (Linux, macOS, Unix)
1687
-
1688
- ==== SIGINT (Ctrl+C)
1689
-
1690
- Interactive interrupt signal for graceful shutdown.
1691
-
1692
- Usage:
1693
-
1694
- * Press `Ctrl+C` in the terminal running Fractor
1695
- * Behavior depends on mode:
1696
- ** *Batch mode*: Stops immediately after current work completes
1697
- ** *Continuous mode*: Initiates graceful shutdown
1698
-
1699
- ==== SIGTERM
1700
-
1701
- Standard Unix termination signal, preferred for production deployments.
1702
-
1703
- This ensures a graceful shutdown of the Fractor supervisor and its workers.
1704
-
1705
- Usage:
1706
-
1707
- [source,sh]
1708
- ----
1709
- kill -TERM <pid>
1710
- # or simply
1711
- kill <pid> # SIGTERM is the default
1712
- ----
1713
-
1714
- Typical signals from service managers:
1715
-
1716
- * Systemd sends SIGTERM on `systemctl stop`
1717
- * Docker sends SIGTERM on `docker stop`
1718
- * Kubernetes sends SIGTERM during pod termination
1719
-
1720
- [source,ini]
1721
- ----
1722
- # Example systemd service
1723
- [Service]
1724
- ExecStart=/usr/bin/ruby /path/to/fractor_server.rb
1725
- KillMode=process
1726
- KillSignal=SIGTERM
1727
- TimeoutStopSec=30
1728
- ----
1729
-
1730
- ==== SIGUSR1
1731
-
1732
- Real-time status monitoring without stopping the process.
1733
-
1734
- Usage:
1735
-
1736
- [source,sh]
1737
- ----
1738
- kill -USR1 <pid>
1739
- ----
1740
-
1741
- Output example:
1742
-
1743
- [example]
1744
- [source]
1745
- ----
1746
- === Fractor Supervisor Status ===
1747
- Mode: Continuous
1748
- Running: true
1749
- Workers: 4
1750
- Idle workers: 2
1751
- Queue size: 15
1752
- Results: 127
1753
- Errors: 3
1754
- ----
1755
-
1756
- === Windows signals
1757
-
1758
- ==== SIGBREAK (Ctrl+Break)
1759
-
1760
- Windows alternative to SIGUSR1 for status monitoring.
1761
-
1762
- Usage:
1763
-
1764
- * Press `Ctrl+Break` in the terminal running Fractor
1765
- * Same output as SIGUSR1 on Unix
1766
-
1767
- [NOTE]
1768
- SIGUSR1 is not available on Windows. Use `Ctrl+Break` instead for status
1769
- monitoring on Windows platforms.
1770
-
1771
-
1772
- === Signal behavior by mode
1773
-
1774
- ==== Batch mode
1775
-
1776
- In batch processing mode:
1777
-
1778
- * SIGINT/SIGTERM: Stops immediately after current work completes
1779
- * SIGUSR1/SIGBREAK: Displays current status
1780
-
1781
- ==== Continuous mode
1782
-
1783
- In continuous mode (long-running servers):
1784
-
1785
- * SIGINT/SIGTERM: Graceful shutdown within ~2 seconds
1786
- ** Stops accepting new work
1787
- ** Completes in-progress work
1788
- ** Cleans up resources
1789
- * SIGUSR1/SIGBREAK: Displays current status
1790
-
1791
-
1792
-
1793
-
1794
- == Running a basic example
1795
-
1796
- . Install the gem as described in the Installation section.
1797
-
1798
- . Create a new Ruby file (e.g., `my_fractor_example.rb`) with your
1799
- implementation:
1800
-
1801
- [source,ruby]
1802
- ----
1803
- require 'fractor'
1804
-
1805
- # Define your Work class
1806
- class MyWork < Fractor::Work
1807
- def to_s
1808
- "MyWork: #{@input}"
1809
- end
1810
- end
1811
-
1812
- # Define your Worker class
1813
- class MyWorker < Fractor::Worker
1814
- def process(work)
1815
- if work.input == 5
1816
- # Return a Fractor::WorkResult for errors
1817
- return Fractor::WorkResult.new(
1818
- error: "Error processing work #{work.input}",
1819
- work: work
1820
- )
1821
- end
1822
-
1823
- calculated = work.input * 2
1824
- # Return a Fractor::WorkResult for success
1825
- Fractor::WorkResult.new(result: calculated, work: work)
1826
- end
1827
- end
1828
-
1829
- # Create supervisor with a worker pool
1830
- supervisor = Fractor::Supervisor.new(
1831
- worker_pools: [
1832
- { worker_class: MyWorker, num_workers: 2 }
1833
- ]
1834
- )
1835
-
1836
- # Create Work objects
1837
- work_items = (1..10).map { |i| MyWork.new(i) }
1838
-
1839
- # Add work items
1840
- supervisor.add_work_items(work_items)
1841
-
1842
- # Run processing
1843
- supervisor.run
1844
-
1845
- # Display results
1846
- puts "Results: #{supervisor.results.results.map(&:result).join(', ')}"
1847
- puts "Errors: #{supervisor.results.errors.map { |e| e.work.input }.join(', ')}"
1848
- ----
1849
-
1850
- . Run the example from your terminal:
1851
-
1852
- [source,sh]
1853
- ----
1854
- ruby my_fractor_example.rb
1855
- ----
1856
-
1857
- You will see output showing Ractors starting, receiving work, processing it, and
1858
- the final aggregated results, including any errors encountered. Press `Ctrl+C`
1859
- during execution to test the graceful shutdown.
1860
-
1861
-
1862
-
1863
-
1864
- == Example applications
1865
-
1866
- === General
1867
-
1868
- The Fractor gem comes with several example applications that demonstrate various
1869
- patterns and use cases. Each example can be found in the `examples` directory of
1870
- the gem repository. Detailed descriptions for these are provided below.
1871
-
1872
- === Pipeline mode examples
1873
-
1874
- ==== Simple example
1875
-
1876
- The Simple Example (link:examples/simple/[examples/simple/]) demonstrates the
1877
- basic usage of the Fractor framework. It shows how to create a simple Work
1878
- class, a Worker class, and a Supervisor to manage the processing of work items
1879
- in parallel. This example serves as a starting point for understanding how to
1880
- use Fractor.
1881
-
1882
- Key features:
1883
-
1884
- * Basic Work and Worker class implementation
1885
- * Simple Supervisor setup
1886
- * Parallel processing of work items
1887
- * Error handling and result aggregation
1888
- * Auto-detection of available processors
1889
- * Graceful shutdown on completion
1890
-
1891
- ==== Auto-detection example
1892
-
1893
- The Auto-Detection Example
1894
- (link:examples/auto_detection/[examples/auto_detection/]) demonstrates
1895
- Fractor's automatic worker detection feature. It shows how to use
1896
- auto-detection, explicit configuration, and mixed approaches for controlling
1897
- the number of workers.
1898
-
1899
- Key features:
1900
-
1901
- * Automatic detection of available processors
1902
- * Comparison of auto-detection vs explicit configuration
1903
- * Mixed configuration with multiple worker pools
1904
- * Best practices for worker configuration
1905
- * Portable code that adapts to different environments
1906
-
1907
- ==== Hierarchical hasher
1908
-
1909
- The Hierarchical Hasher example
1910
- (link:examples/hierarchical_hasher/[examples/hierarchical_hasher/]) demonstrates
1911
- how to use the Fractor framework to process a file in parallel by breaking it
1912
- into chunks, hashing each chunk independently, and then combining the results
1913
- into a final hash. This approach is useful for processing large files
1914
- efficiently.
1915
-
1916
- Key features:
1917
-
1918
- * Parallel data chunking for large files
1919
- * Independent processing of data segments
1920
- * Aggregation of results to form a final output
1921
-
1922
- ==== Multi-work type
1923
-
1924
- The Multi-Work Type example
1925
- (link:examples/multi_work_type/[examples/multi_work_type/]) demonstrates how a
1926
- single Fractor supervisor and worker can handle multiple types of work items
1927
- (e.g., `TextWork` and `ImageWork`). The worker intelligently adapts its
1928
- processing strategy based on the class of the incoming work item.
1929
-
1930
- Key features:
1931
-
1932
- * Support for multiple `Fractor::Work` subclasses
1933
- * Polymorphic worker processing based on work type
1934
- * Unified workflow for diverse tasks
1935
-
1936
- ==== Pipeline processing
1937
-
1938
- The Pipeline Processing example
1939
- (link:examples/pipeline_processing/[examples/pipeline_processing/]) implements a
1940
- multi-stage processing pipeline where data flows sequentially through a series
1941
- of transformations. The output of one stage becomes the input for the next, and
1942
- different stages can operate concurrently on different data items.
1943
-
1944
- Key features:
1945
-
1946
- * Sequential data flow through multiple processing stages
1947
- * Concurrent execution of different pipeline stages
1948
- * Data transformation at each step of the pipeline
1949
-
1950
- ==== Producer/subscriber
1951
-
1952
- The Producer/Subscriber example
1953
- (link:examples/producer_subscriber/[examples/producer_subscriber/]) showcases a
1954
- multi-stage document processing system where initial work (processing a
1955
- document) can generate additional sub-work items (processing sections of the
1956
- document). This creates a hierarchical processing pattern.
1957
-
1958
- Key features:
1959
-
1960
- * Implementation of producer-consumer patterns
1961
- * Dynamic generation of sub-work based on initial processing
1962
- * Construction of hierarchical result structures
1963
-
1964
- ==== Scatter/gather
1965
-
1966
- The Scatter/Gather example
1967
- (link:examples/scatter_gather/[examples/scatter_gather/]) illustrates how a
1968
- large task or dataset is broken down (scattered) into smaller, independent
1969
- subtasks. These subtasks are processed in parallel by multiple workers, and
1970
- their results are then collected (gathered) and combined to produce the final
1971
- output.
1972
-
1973
- Key features:
1974
-
1975
- * Distribution of a large task into smaller, parallelizable subtasks
1976
- * Concurrent processing of subtasks
1977
- * Aggregation of partial results into a final result
1978
-
1979
- ==== Specialized workers
1980
-
1981
- The Specialized Workers example
1982
- (link:examples/specialized_workers/[examples/specialized_workers/]) demonstrates
1983
- creating distinct worker types, each tailored to handle specific kinds of tasks
1984
- (e.g., `ComputeWorker` for CPU-intensive operations and `DatabaseWorker` for
1985
- I/O-bound database interactions). This allows for optimized resource utilization
1986
- and domain-specific logic.
1987
-
1988
- Key features:
1989
-
1990
- * Creation of worker classes for specific processing domains
1991
- * Routing of work items to appropriately specialized workers
1992
- * Optimization of resources and logic per task type
1993
-
1994
- === Continuous mode examples
1995
-
1996
- ==== Plain socket implementation
1997
-
1998
- The plain socket implementation
1999
- (link:examples/continuous_chat_server/[examples/continuous_chat_server/])
2000
- provides a baseline chat server using plain TCP sockets without Fractor. This
2001
- serves as a comparison point to understand the benefits of using Fractor for
2002
- continuous processing.
2003
-
2004
- ==== Fractor-based implementation
2005
-
2006
- The Fractor-based implementation
2007
- (link:examples/continuous_chat_fractor/[examples/continuous_chat_fractor/])
2008
- demonstrates how to build a production-ready chat server using Fractor's
2009
- continuous mode with high-level primitives.
2010
-
2011
- Key features:
2012
-
2013
- * *Continuous mode operation*: Server runs indefinitely processing messages as
2014
- they arrive
2015
- * *High-level primitives*: Uses WorkQueue and ContinuousServer to eliminate
2016
- boilerplate
2017
- * *Graceful shutdown*: Production-ready signal handling (SIGINT, SIGTERM,
2018
- SIGUSR1/SIGBREAK)
2019
- * *Callback-based results*: Clean separation of concerns with on_result and
2020
- on_error callbacks
2021
- * *Cross-platform support*: Works on Unix/Linux/macOS and Windows
2022
- * *Process monitoring*: Runtime status checking via signals
2023
- * *40% code reduction*: 167 lines vs 279 lines with low-level API
2024
-
2025
- The implementation includes:
2026
-
2027
- * `chat_common.rb`: Work and Worker class definitions for chat message
2028
- processing
2029
- * `chat_server.rb`: Main server using high-level primitives
2030
- * `simulate.rb`: Test client simulator
2031
-
2032
- This example demonstrates production deployment patterns including:
2033
-
2034
- * Systemd service integration
2035
- * Docker container deployment
2036
- * Process monitoring and health checks
2037
- * Graceful restart procedures
2038
-
2039
- See link:examples/continuous_chat_fractor/README.adoc[the chat server README]
2040
- for detailed implementation documentation.
2041
-
2042
-
2043
-
2044
-
2045
- == Copyright and license
220
+ == Copyright
2046
221
 
2047
222
  Copyright Ribose.
2048
-
2049
- Licensed under the Ribose BSD 2-Clause License.