fractor 0.1.4 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (189) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop-https---raw-githubusercontent-com-riboseinc-oss-guides-main-ci-rubocop-yml +552 -0
  3. data/.rubocop.yml +14 -8
  4. data/.rubocop_todo.yml +284 -43
  5. data/README.adoc +111 -950
  6. data/docs/.lycheeignore +16 -0
  7. data/docs/Gemfile +24 -0
  8. data/docs/README.md +157 -0
  9. data/docs/_config.yml +151 -0
  10. data/docs/_features/error-handling.adoc +1192 -0
  11. data/docs/_features/index.adoc +80 -0
  12. data/docs/_features/monitoring.adoc +589 -0
  13. data/docs/_features/signal-handling.adoc +202 -0
  14. data/docs/_features/workflows.adoc +1235 -0
  15. data/docs/_guides/continuous-mode.adoc +736 -0
  16. data/docs/_guides/cookbook.adoc +1133 -0
  17. data/docs/_guides/index.adoc +55 -0
  18. data/docs/_guides/pipeline-mode.adoc +730 -0
  19. data/docs/_guides/troubleshooting.adoc +358 -0
  20. data/docs/_pages/architecture.adoc +1390 -0
  21. data/docs/_pages/core-concepts.adoc +1392 -0
  22. data/docs/_pages/design-principles.adoc +862 -0
  23. data/docs/_pages/getting-started.adoc +290 -0
  24. data/docs/_pages/installation.adoc +143 -0
  25. data/docs/_reference/api.adoc +1080 -0
  26. data/docs/_reference/error-reporting.adoc +670 -0
  27. data/docs/_reference/examples.adoc +181 -0
  28. data/docs/_reference/index.adoc +96 -0
  29. data/docs/_reference/troubleshooting.adoc +862 -0
  30. data/docs/_tutorials/complex-workflows.adoc +1022 -0
  31. data/docs/_tutorials/data-processing-pipeline.adoc +740 -0
  32. data/docs/_tutorials/first-application.adoc +384 -0
  33. data/docs/_tutorials/index.adoc +48 -0
  34. data/docs/_tutorials/long-running-services.adoc +931 -0
  35. data/docs/assets/images/favicon-16.png +0 -0
  36. data/docs/assets/images/favicon-32.png +0 -0
  37. data/docs/assets/images/favicon-48.png +0 -0
  38. data/docs/assets/images/favicon.ico +0 -0
  39. data/docs/assets/images/favicon.png +0 -0
  40. data/docs/assets/images/favicon.svg +45 -0
  41. data/docs/assets/images/fractor-icon.svg +49 -0
  42. data/docs/assets/images/fractor-logo.svg +61 -0
  43. data/docs/index.adoc +131 -0
  44. data/docs/lychee.toml +39 -0
  45. data/examples/api_aggregator/README.adoc +627 -0
  46. data/examples/api_aggregator/api_aggregator.rb +376 -0
  47. data/examples/auto_detection/README.adoc +407 -29
  48. data/examples/auto_detection/auto_detection.rb +9 -9
  49. data/examples/continuous_chat_common/message_protocol.rb +53 -0
  50. data/examples/continuous_chat_fractor/README.adoc +217 -0
  51. data/examples/continuous_chat_fractor/chat_client.rb +303 -0
  52. data/examples/continuous_chat_fractor/chat_common.rb +83 -0
  53. data/examples/continuous_chat_fractor/chat_server.rb +167 -0
  54. data/examples/continuous_chat_fractor/simulate.rb +345 -0
  55. data/examples/continuous_chat_server/README.adoc +135 -0
  56. data/examples/continuous_chat_server/chat_client.rb +303 -0
  57. data/examples/continuous_chat_server/chat_server.rb +359 -0
  58. data/examples/continuous_chat_server/simulate.rb +343 -0
  59. data/examples/error_reporting.rb +207 -0
  60. data/examples/file_processor/README.adoc +170 -0
  61. data/examples/file_processor/file_processor.rb +615 -0
  62. data/examples/file_processor/sample_files/invalid.csv +1 -0
  63. data/examples/file_processor/sample_files/orders.xml +24 -0
  64. data/examples/file_processor/sample_files/products.json +23 -0
  65. data/examples/file_processor/sample_files/users.csv +6 -0
  66. data/examples/hierarchical_hasher/README.adoc +629 -41
  67. data/examples/hierarchical_hasher/hierarchical_hasher.rb +12 -8
  68. data/examples/image_processor/README.adoc +610 -0
  69. data/examples/image_processor/image_processor.rb +349 -0
  70. data/examples/image_processor/processed_images/sample_10_processed.jpg.json +12 -0
  71. data/examples/image_processor/processed_images/sample_1_processed.jpg.json +12 -0
  72. data/examples/image_processor/processed_images/sample_2_processed.jpg.json +12 -0
  73. data/examples/image_processor/processed_images/sample_3_processed.jpg.json +12 -0
  74. data/examples/image_processor/processed_images/sample_4_processed.jpg.json +12 -0
  75. data/examples/image_processor/processed_images/sample_5_processed.jpg.json +12 -0
  76. data/examples/image_processor/processed_images/sample_6_processed.jpg.json +12 -0
  77. data/examples/image_processor/processed_images/sample_7_processed.jpg.json +12 -0
  78. data/examples/image_processor/processed_images/sample_8_processed.jpg.json +12 -0
  79. data/examples/image_processor/processed_images/sample_9_processed.jpg.json +12 -0
  80. data/examples/image_processor/test_images/sample_1.png +1 -0
  81. data/examples/image_processor/test_images/sample_10.png +1 -0
  82. data/examples/image_processor/test_images/sample_2.png +1 -0
  83. data/examples/image_processor/test_images/sample_3.png +1 -0
  84. data/examples/image_processor/test_images/sample_4.png +1 -0
  85. data/examples/image_processor/test_images/sample_5.png +1 -0
  86. data/examples/image_processor/test_images/sample_6.png +1 -0
  87. data/examples/image_processor/test_images/sample_7.png +1 -0
  88. data/examples/image_processor/test_images/sample_8.png +1 -0
  89. data/examples/image_processor/test_images/sample_9.png +1 -0
  90. data/examples/log_analyzer/README.adoc +662 -0
  91. data/examples/log_analyzer/log_analyzer.rb +579 -0
  92. data/examples/log_analyzer/sample_logs/apache.log +20 -0
  93. data/examples/log_analyzer/sample_logs/json.log +15 -0
  94. data/examples/log_analyzer/sample_logs/nginx.log +15 -0
  95. data/examples/log_analyzer/sample_logs/rails.log +29 -0
  96. data/examples/multi_work_type/README.adoc +576 -26
  97. data/examples/multi_work_type/multi_work_type.rb +30 -29
  98. data/examples/performance_monitoring.rb +120 -0
  99. data/examples/pipeline_processing/README.adoc +740 -26
  100. data/examples/pipeline_processing/pipeline_processing.rb +16 -16
  101. data/examples/priority_work_example.rb +155 -0
  102. data/examples/producer_subscriber/README.adoc +889 -46
  103. data/examples/producer_subscriber/producer_subscriber.rb +20 -16
  104. data/examples/scatter_gather/README.adoc +829 -27
  105. data/examples/scatter_gather/scatter_gather.rb +29 -28
  106. data/examples/simple/README.adoc +347 -0
  107. data/examples/simple/sample.rb +5 -5
  108. data/examples/specialized_workers/README.adoc +622 -26
  109. data/examples/specialized_workers/specialized_workers.rb +88 -45
  110. data/examples/stream_processor/README.adoc +206 -0
  111. data/examples/stream_processor/stream_processor.rb +284 -0
  112. data/examples/web_scraper/README.adoc +625 -0
  113. data/examples/web_scraper/web_scraper.rb +285 -0
  114. data/examples/workflow/README.adoc +406 -0
  115. data/examples/workflow/circuit_breaker/README.adoc +360 -0
  116. data/examples/workflow/circuit_breaker/circuit_breaker_workflow.rb +225 -0
  117. data/examples/workflow/conditional/README.adoc +483 -0
  118. data/examples/workflow/conditional/conditional_workflow.rb +215 -0
  119. data/examples/workflow/dead_letter_queue/README.adoc +374 -0
  120. data/examples/workflow/dead_letter_queue/dead_letter_queue_workflow.rb +217 -0
  121. data/examples/workflow/fan_out/README.adoc +381 -0
  122. data/examples/workflow/fan_out/fan_out_workflow.rb +202 -0
  123. data/examples/workflow/retry/README.adoc +248 -0
  124. data/examples/workflow/retry/retry_workflow.rb +195 -0
  125. data/examples/workflow/simple_linear/README.adoc +267 -0
  126. data/examples/workflow/simple_linear/simple_linear_workflow.rb +175 -0
  127. data/examples/workflow/simplified/README.adoc +329 -0
  128. data/examples/workflow/simplified/simplified_workflow.rb +222 -0
  129. data/exe/fractor +10 -0
  130. data/lib/fractor/cli.rb +288 -0
  131. data/lib/fractor/configuration.rb +307 -0
  132. data/lib/fractor/continuous_server.rb +183 -0
  133. data/lib/fractor/error_formatter.rb +72 -0
  134. data/lib/fractor/error_report_generator.rb +152 -0
  135. data/lib/fractor/error_reporter.rb +244 -0
  136. data/lib/fractor/error_statistics.rb +147 -0
  137. data/lib/fractor/execution_tracer.rb +162 -0
  138. data/lib/fractor/logger.rb +230 -0
  139. data/lib/fractor/main_loop_handler.rb +406 -0
  140. data/lib/fractor/main_loop_handler3.rb +135 -0
  141. data/lib/fractor/main_loop_handler4.rb +299 -0
  142. data/lib/fractor/performance_metrics_collector.rb +181 -0
  143. data/lib/fractor/performance_monitor.rb +215 -0
  144. data/lib/fractor/performance_report_generator.rb +202 -0
  145. data/lib/fractor/priority_work.rb +93 -0
  146. data/lib/fractor/priority_work_queue.rb +189 -0
  147. data/lib/fractor/result_aggregator.rb +33 -1
  148. data/lib/fractor/shutdown_handler.rb +168 -0
  149. data/lib/fractor/signal_handler.rb +80 -0
  150. data/lib/fractor/supervisor.rb +430 -144
  151. data/lib/fractor/supervisor_logger.rb +88 -0
  152. data/lib/fractor/version.rb +1 -1
  153. data/lib/fractor/work.rb +12 -0
  154. data/lib/fractor/work_distribution_manager.rb +151 -0
  155. data/lib/fractor/work_queue.rb +88 -0
  156. data/lib/fractor/work_result.rb +181 -9
  157. data/lib/fractor/worker.rb +75 -1
  158. data/lib/fractor/workflow/builder.rb +210 -0
  159. data/lib/fractor/workflow/chain_builder.rb +169 -0
  160. data/lib/fractor/workflow/circuit_breaker.rb +183 -0
  161. data/lib/fractor/workflow/circuit_breaker_orchestrator.rb +208 -0
  162. data/lib/fractor/workflow/circuit_breaker_registry.rb +112 -0
  163. data/lib/fractor/workflow/dead_letter_queue.rb +334 -0
  164. data/lib/fractor/workflow/execution_hooks.rb +39 -0
  165. data/lib/fractor/workflow/execution_strategy.rb +225 -0
  166. data/lib/fractor/workflow/execution_trace.rb +134 -0
  167. data/lib/fractor/workflow/helpers.rb +191 -0
  168. data/lib/fractor/workflow/job.rb +290 -0
  169. data/lib/fractor/workflow/job_dependency_validator.rb +120 -0
  170. data/lib/fractor/workflow/logger.rb +110 -0
  171. data/lib/fractor/workflow/pre_execution_context.rb +193 -0
  172. data/lib/fractor/workflow/retry_config.rb +156 -0
  173. data/lib/fractor/workflow/retry_orchestrator.rb +184 -0
  174. data/lib/fractor/workflow/retry_strategy.rb +93 -0
  175. data/lib/fractor/workflow/structured_logger.rb +30 -0
  176. data/lib/fractor/workflow/type_compatibility_validator.rb +222 -0
  177. data/lib/fractor/workflow/visualizer.rb +211 -0
  178. data/lib/fractor/workflow/workflow_context.rb +132 -0
  179. data/lib/fractor/workflow/workflow_executor.rb +669 -0
  180. data/lib/fractor/workflow/workflow_result.rb +55 -0
  181. data/lib/fractor/workflow/workflow_validator.rb +295 -0
  182. data/lib/fractor/workflow.rb +333 -0
  183. data/lib/fractor/wrapped_ractor.rb +66 -91
  184. data/lib/fractor/wrapped_ractor3.rb +161 -0
  185. data/lib/fractor/wrapped_ractor4.rb +242 -0
  186. data/lib/fractor.rb +93 -3
  187. metadata +192 -6
  188. data/tests/sample.rb.bak +0 -309
  189. data/tests/sample_working.rb.bak +0 -209
data/README.adoc CHANGED
@@ -1,1061 +1,222 @@
1
1
  = Fractor: Function-driven Ractors framework
2
2
 
3
- Fractor is a lightweight Ruby framework designed to simplify the process of
4
- distributing computational work across multiple Ractors.
3
+ image:https://img.shields.io/gem/v/fractor.svg[RubyGems Version, link=https://rubygems.org/gems/fractor]
4
+ image:https://img.shields.io/github/license/metanorma/fractor.svg[License, link=https://github.com/metanorma/fractor/blob/main/LICENSE]
5
+ image:https://github.com/metanorma/fractor/actions/workflows/rake.yml/badge.svg[Build Status, link=https://github.com/metanorma/fractor/actions/workflows/rake.yml]
6
+ image:https://img.shields.io/badge/ruby-3.0%20%E2%86%92%204.0%2B-ruby.svg[Ruby 3.0 to 4.0+, link=https://www.ruby-lang.org]
5
7
 
6
- == Introduction
8
+ Fractor is a lightweight Ruby framework for parallel processing using Ractors
9
+ (Ruby Actors).
7
10
 
8
- Fractor stands for *Function-driven Ractors framework*. It is a lightweight Ruby
9
- framework designed to simplify the process of distributing computational work
10
- across multiple Ractors (Ruby's actor-like concurrency model).
11
+ It provides a structured way to distribute computational work across multiple
12
+ Ractors with minimal boilerplate.
11
13
 
12
- The primary goal of Fractor is to provide a structured way to define work,
13
- process it in parallel using Ractors, and aggregate the results, while
14
- abstracting away much of the boilerplate code involved in Ractor management and
15
- communication.
14
+ == Ruby version support
16
15
 
17
- == Installation
16
+ Fractor fully supports both **Ruby 3.x** and **Ruby 4.0+**:
18
17
 
19
- === Using RubyGems
18
+ * *Ruby 3.0+*: Uses `Ractor.yield` for message passing from workers
20
19
 
21
- [source,sh]
22
- ----
23
- gem install fractor
24
- ----
20
+ * *Ruby 4.0+*: Uses `Ractor::Port` for more efficient communication patterns
25
21
 
26
- === Using Bundler
22
+ Fractor automatically detects your Ruby version and uses the appropriate
23
+ internal implementation. The user-facing API is identical across versions --
24
+ write your code once, and Fractor handles the differences internally.
27
25
 
28
- Add this line to your application's Gemfile:
26
+ See
27
+ link:docs/_pages/architecture.adoc#ruby-version-compatibility[Architecture: Ruby Version Compatibility]
28
+ for details on the internal differences.
29
29
 
30
- [source,ruby]
31
- ----
32
- gem 'fractor'
33
- ----
30
+ == Quick start
34
31
 
35
- And then execute:
32
+ === Installation
36
33
 
37
34
  [source,sh]
38
35
  ----
39
- bundle install
36
+ gem install fractor
40
37
  ----
41
38
 
39
+ See link:docs/_pages/installation.adoc[Installation Guide] for more options.
42
40
 
43
- === Key concepts
44
-
45
- * *Function-driven:* You define the core processing logic by subclassing
46
- `Fractor::Worker` and implementing the `process` method.
47
-
48
- * *Parallel execution:* Work items are automatically distributed to available
49
- worker Ractors for concurrent processing.
50
-
51
- * *Result aggregation:* The framework collects both successful results and
52
- errors from the workers.
53
-
54
- * *Separation of concerns:* Keeps the framework logic (`fractor.rb`) separate
55
- from the client's specific implementation (`sample.rb`).
56
-
57
- == Scope
58
-
59
- This document describes the design, implementation, and usage of the Fractor
60
- framework. It provides detailed information about the framework's components,
61
- their interactions, and how to use them to implement parallel processing in Ruby
62
- applications.
63
-
64
- [bibliography]
65
- == Normative references
66
-
67
- * [[[ruby-ractor,Ruby Ractor Documentation]]], https://docs.ruby-lang.org/en/master/Ractor.html
68
-
69
- == Terms and definitions
70
-
71
- === ractor
72
-
73
- concurrent programming abstraction in Ruby that enables parallel execution
74
- with thread safety
75
-
76
- [.source]
77
- <<ruby>>
78
-
79
- === worker
80
-
81
- component that processes work items to produce work results
82
-
83
- === work item
84
-
85
- unit of computation to be processed by a ractor
86
-
87
- === work result
88
-
89
- result of processing a work item, either successful or an error
90
-
91
- === work item class
92
-
93
- class that represents a work item, typically subclassing `Fractor::Work`
94
-
95
- === worker class
96
-
97
- class that represents a worker, typically subclassing `Fractor::Worker`
98
-
99
- === wrapped ractor
100
-
101
- component that manages a single ractor and its associated worker
102
-
103
- === supervisor
104
-
105
- component that manages the pool of workers and distributes work items
106
-
107
- === result aggregator
108
-
109
- component that collects and organizes work results from workers
110
-
111
-
112
-
113
-
114
- == Core components
115
-
116
- === General
117
-
118
- The Fractor framework consists of the following main classes, all residing
119
- within the `Fractor` module.
120
-
121
-
122
- === Fractor::Worker
123
-
124
- The abstract base class for defining how work should be processed.
125
-
126
- Client code must subclass this and implement the `process(work)` method.
127
-
128
- The `process` method receives a `Fractor::Work` object (or a subclass) and
129
- should return a `Fractor::WorkResult` object.
130
-
131
- === Fractor::Work
132
-
133
- The abstract base class for representing a unit of work.
134
-
135
- Typically holds the input data needed by the `Worker`.
136
-
137
- Client code should subclass this to define specific types of work items.
138
-
139
- === Fractor::WorkResult
140
-
141
- A container object returned by the `Worker#process` method.
142
-
143
- Holds either the successful `:result` of the computation or an `:error`
144
- message if processing failed.
145
-
146
- Includes a reference back to the original `:work` item.
147
-
148
- Provides a `success?` method.
149
-
150
- === Fractor::ResultAggregator
151
-
152
- Collects and stores all `WorkResult` objects generated by the workers.
153
-
154
- Separates results into `results` (successful) and `errors` arrays.
155
-
156
- === Fractor::WrappedRactor
157
-
158
- Manages an individual Ruby `Ractor`.
159
-
160
- Instantiates the client-provided `Worker` subclass within the Ractor.
161
-
162
- Handles receiving `Work` items, calling the `Worker#process` method, and
163
- yielding `WorkResult` objects (or errors) back to the `Supervisor`.
164
-
165
- === Fractor::Supervisor
166
-
167
- The main orchestrator of the framework.
168
-
169
- Initializes and manages a pool of `WrappedRactor` instances.
170
-
171
- Manages a `work_queue` of input data.
172
-
173
- Distributes work items (wrapped in the client's `Work` subclass) to available
174
- Ractors.
175
-
176
- Listens for results and errors from Ractors using `Ractor.select`.
177
-
178
- Uses `ResultAggregator` to store outcomes.
179
-
180
- Handles graceful shutdown on `SIGINT` (Ctrl+C).
181
-
182
-
183
-
184
- == Quick start
185
-
186
- === General
187
-
188
- This quick start guide shows the minimum steps needed to get a simple parallel
189
- execution working with Fractor.
190
-
191
- === Step 1: Create a minimal Work class
192
-
193
- The Work class represents a unit of work to be processed by a Worker. It
194
- encapsulates the input data needed for processing.
41
+ === 30-second example
195
42
 
196
43
  [source,ruby]
197
44
  ----
198
45
  require 'fractor'
199
46
 
47
+ # Define your work
200
48
  class MyWork < Fractor::Work
201
- # Store all properties in the input hash
202
49
  def initialize(value)
203
50
  super({ value: value })
204
51
  end
205
52
 
206
- # Accessor method for the stored value
207
53
  def value
208
54
  input[:value]
209
55
  end
210
-
211
- def to_s
212
- "MyWork: #{value}"
213
- end
214
56
  end
215
- ----
216
-
217
- A Work is instantiated with the input data it will process this way:
218
57
 
219
- [source,ruby]
220
- ----
221
- work_item = MyWork.new(42)
222
- puts work_item.to_s # Output: MyWork: 42
223
- ----
224
-
225
-
226
- === Step 2: Create a minimal Worker class
227
-
228
- The Worker class defines the processing logic for work items. Each Worker
229
- instance runs within its own Ractor and processes Work objects sent to it.
230
-
231
- It must implement the `process(work)` method, which takes a Work object as
232
- input and returns a `Fractor::WorkResult` object.
233
-
234
- The `process` method should handle both successful processing and error
235
- conditions.
236
-
237
- [source,ruby]
238
- ----
58
+ # Define your worker
239
59
  class MyWorker < Fractor::Worker
240
60
  def process(work)
241
- # Your processing logic here
242
- result = work.input * 2
243
-
244
- # Return a success result
61
+ result = work.value * 2
245
62
  Fractor::WorkResult.new(result: result, work: work)
246
63
  rescue => e
247
- # Return an error result if something goes wrong
248
64
  Fractor::WorkResult.new(error: e.message, work: work)
249
65
  end
250
66
  end
251
- ----
252
-
253
- The `process` method can perform any computation you need. In this example, it
254
- multiplies the input by 2. If an error occurs, it catches the exception and
255
- returns an error result.
256
-
257
- === Step 3: Set up and run the Supervisor
258
-
259
- The Supervisor class orchestrates the entire framework, managing worker Ractors,
260
- distributing work, and collecting results.
261
-
262
- It initializes pools of Ractors, each running an instance of a Worker
263
- class. The Supervisor handles the communication between the main thread and
264
- the Ractors, including sending work items and receiving results.
265
-
266
- The Supervisor also manages the work queue and the ResultAggregator, which
267
- collects and organizes all results from the workers.
268
-
269
- To set up the Supervisor, you specify worker pools, each containing a Worker class
270
- and optionally the number of workers to create. If you don't specify `num_workers`,
271
- Fractor will automatically detect the number of available processors on your system
272
- and use that value. You can create multiple worker pools with different worker types
273
- to handle different kinds of work. Each worker pool can process any type of Work
274
- object that inherits from Fractor::Work.
275
-
276
- [source,ruby]
277
- ----
278
- # Create the supervisor with auto-detected number of workers
279
- supervisor = Fractor::Supervisor.new(
280
- worker_pools: [
281
- { worker_class: MyWorker } # Number of workers auto-detected
282
- ]
283
- )
284
67
 
285
- # Or explicitly specify the number of workers
68
+ # Create supervisor and process work
286
69
  supervisor = Fractor::Supervisor.new(
287
- worker_pools: [
288
- { worker_class: MyWorker, num_workers: 4 } # Explicitly use 4 workers
289
- ]
70
+ worker_pools: [{ worker_class: MyWorker }]
290
71
  )
291
72
 
292
- # Add individual work items (instances of Work subclasses)
293
- supervisor.add_work_item(MyWork.new(1))
294
-
295
- # Add multiple work items
296
73
  supervisor.add_work_items([
74
+ MyWork.new(1),
297
75
  MyWork.new(2),
298
- MyWork.new(3),
299
- MyWork.new(4),
300
- MyWork.new(5)
301
- ])
302
-
303
- # You can add different types of Work objects to the same supervisor
304
- supervisor.add_work_items([
305
- MyWork.new(6),
306
- OtherWork.new("data")
76
+ MyWork.new(3)
307
77
  ])
308
78
 
309
- # Run the processing
310
79
  supervisor.run
311
80
 
312
- # Access results
313
81
  puts "Results: #{supervisor.results.results.map(&:result)}"
314
- puts "Errors: #{supervisor.results.errors.size}"
315
- ----
316
-
317
- That's it! With these three simple steps, you have a working parallel processing
318
- system using Fractor.
319
-
320
-
321
- == Usage
322
-
323
- === Work class
324
-
325
- ==== Purpose and responsibilities
326
-
327
- The `Fractor::Work` class represents a unit of work to be processed by a Worker.
328
- Its primary responsibility is to encapsulate the input data needed for
329
- processing.
330
-
331
- ==== Implementation requirements
332
-
333
- At minimum, your Work subclass should:
334
-
335
- . Inherit from `Fractor::Work`
336
- . Pass the input data to the superclass constructor
337
-
338
- [source,ruby]
339
- ----
340
- class MyWork < Fractor::Work
341
- def initialize(input)
342
- super(input) # This stores input in @input
343
- # Add any additional initialization if needed
344
- end
345
- end
346
- ----
347
-
348
- ==== Advanced usage
349
-
350
- You can extend your Work class to include additional data or methods:
351
-
352
- [source,ruby]
353
- ----
354
- class ComplexWork < Fractor::Work
355
- attr_reader :options
356
-
357
- def initialize(input, options = {})
358
- super(input)
359
- @options = options
360
- end
361
-
362
- def high_priority?
363
- @options[:priority] == :high
364
- end
365
-
366
- def to_s
367
- "ComplexWork: #{@input} (#{@options[:priority]} priority)"
368
- end
369
- end
370
- ----
371
-
372
- [TIP]
373
- ====
374
- ====
375
- * Keep Work objects lightweight and serializable since they will be passed
376
- between Ractors
377
- * Implement a meaningful `to_s` method for better debugging
378
- ====
379
- * Consider adding validation in the initializer to catch issues early
380
- ====
381
-
382
- === Worker class
383
-
384
- ==== Purpose and responsibilities
385
-
386
- The `Fractor::Worker` class defines the processing logic for work items. Each
387
- Worker instance runs within its own Ractor and processes Work objects sent to
388
- it.
389
-
390
- ==== Implementation requirements
391
-
392
- Your Worker subclass must:
393
-
394
- . Inherit from `Fractor::Worker`
395
- . Implement the `process(work)` method
396
- . Return a `Fractor::WorkResult` object from the `process` method
397
- . Handle both successful processing and error conditions
398
-
399
- [source,ruby]
400
- ----
401
- class MyWorker < Fractor::Worker
402
- def process(work)
403
- # Process the work
404
-
405
- if work.input < 0
406
- return Fractor::WorkResult.new(
407
- error: "Cannot process negative numbers",
408
- work: work
409
- )
410
- end
411
-
412
- # Normal processing...
413
- result = work.input * 2
414
-
415
- # Return a WorkResult
416
- Fractor::WorkResult.new(result: result, work: work)
417
- end
418
- end
419
- ----
420
-
421
-
422
- ==== Error handling
423
-
424
- The Worker class should handle two types of errors.
425
-
426
-
427
- ===== Handled errors
428
-
429
- These are expected error conditions that your code explicitly checks for.
430
-
431
- [source,ruby]
432
- ----
433
- def process(work)
434
- if work.input < 0
435
- return Fractor::WorkResult.new(
436
- error: "Cannot process negative numbers",
437
- work: work
438
- )
439
- end
440
-
441
- # Normal processing...
442
- Fractor::WorkResult.new(result: calculated_value, work: work)
443
- end
82
+ # => Results: [2, 4, 6]
444
83
  ----
445
84
 
446
- ===== Unexpected errors caught by rescue
85
+ == Key features
447
86
 
448
- These are unexpected exceptions that may occur during processing. You should
449
- catch these and convert them into error results.
87
+ * *Function-driven*: Define processing logic by subclassing `Fractor::Worker`
88
+ * *Parallel execution*: Work automatically distributed across Ractor workers
89
+ * *Two operating modes*:
90
+ ** **Pipeline mode** for batch processing
91
+ ** **Continuous mode** for long-running servers
92
+ * *Workflow system*: GitHub Actions-style declarative workflows
93
+ * *Error handling*: Retry logic, circuit breakers, dead letter queues, error reporting
94
+ * *Production-ready*: Signal handling, logging, monitoring, graceful shutdown
95
+ * *Performance tools*: Built-in monitoring, benchmarking, and error analytics
96
+ * *High-level primitives*: WorkQueue and ContinuousServer eliminate boilerplate
450
97
 
451
- [source,ruby]
452
- ----
453
- def process(work)
454
- # Processing that might raise exceptions
455
- result = complex_calculation(work.input)
456
-
457
- Fractor::WorkResult.new(result: result, work: work)
458
- rescue StandardError => e
459
- # Catch and convert any unexpected exceptions to error results
460
- Fractor::WorkResult.new(error: "An unexpected error occurred: #{e.message}", work: work)
461
- end
462
- ----
463
-
464
- [TIP]
465
- ====
466
- * Keep the `process` method focused on a single responsibility
467
- * Use meaningful error messages that help diagnose issues
468
- * Consider adding logging within the `process` method for debugging
469
- * Ensure all paths return a valid `WorkResult` object
470
- ====
98
+ == Documentation
471
99
 
472
- === WorkResult class
100
+ === Getting started
473
101
 
474
- ==== Purpose and responsibilities
102
+ * link:docs/_pages/installation.adoc[Installation] - System requirements and installation methods
103
+ * link:docs/_pages/getting-started.adoc[Getting Started] - Quick start guides for both modes
104
+ * link:docs/_pages/core-concepts.adoc[Core Concepts] - Understanding Fractor components
475
105
 
476
- The `Fractor::WorkResult` class is a container that holds either the successful
477
- result of processing or an error message, along with a reference to the original
478
- work item.
106
+ === Operating modes
479
107
 
480
- ==== Creating results
108
+ * link:docs/_guides/pipeline-mode.adoc[Pipeline Mode] - Batch processing with predefined work
109
+ * link:docs/_guides/continuous-mode.adoc[Continuous Mode] - Long-running servers and streaming
481
110
 
482
- To create a successful result:
111
+ === Advanced features
483
112
 
484
- [source,ruby]
485
- ----
486
- # For successful processing
487
- Fractor::WorkResult.new(result: calculated_value, work: work_object)
488
- ----
113
+ * link:docs/_features/workflows.adoc[Workflows] - Declarative workflow system for complex pipelines
114
+ * link:docs/_features/error-handling.adoc[Error Handling] - Retry logic, circuit breakers, and dead letter queues
115
+ * link:docs/_features/monitoring.adoc[Monitoring] - Performance monitoring and metrics
116
+ * link:docs/_features/signal-handling.adoc[Signal Handling] - Process monitoring and graceful shutdown
489
117
 
490
- To create an error result:
118
+ === Reference
491
119
 
492
- [source,ruby]
493
- ----
494
- # For error conditions
495
- Fractor::WorkResult.new(error: "Error message", work: work_object)
496
- ----
120
+ * link:docs/_reference/api.adoc[API Reference] - Complete API documentation
121
+ * link:docs/_reference/examples.adoc[Examples] - Complete examples for all patterns
122
+ * link:docs/_reference/troubleshooting.adoc[Troubleshooting] - Common issues and solutions
497
123
 
498
- ==== Checking result status
124
+ == Operating modes
499
125
 
500
- You can check if a result was successful:
126
+ Fractor supports two distinct modes:
501
127
 
502
- [source,ruby]
503
- ----
504
- if work_result.success?
505
- # Handle successful result
506
- processed_value = work_result.result
507
- else
508
- # Handle error
509
- error_message = work_result.error
510
- end
511
- ----
128
+ [cols="1,2,2",options="header"]
129
+ |===
130
+ |Mode |Best for |Example use cases
512
131
 
513
- ==== Accessing original work
132
+ |*Pipeline Mode*
133
+ |Batch processing, one-time jobs
134
+ |File processing, ETL pipelines, data transformations
514
135
 
515
- The original work item is always available:
136
+ |*Continuous Mode*
137
+ |Long-running servers, streaming
138
+ |Chat servers, job processors, event streams
139
+ |===
516
140
 
517
- [source,ruby]
518
- ----
519
- original_work = work_result.work
520
- input_value = original_work.input
521
- ----
141
+ See link:docs/getting-started.adoc#choosing-your-mode[Choosing Your Mode] for detailed guidance.
522
142
 
523
- === ResultAggregator class
524
-
525
- ==== Purpose and responsibilities
526
-
527
- The `Fractor::ResultAggregator` collects and organizes all results from the
528
- workers, separating successful results from errors.
529
-
530
- Completed work results may be order independent or order dependent.
531
-
532
- * For order independent results, the results may be utilized (popped) as they
533
- are received.
534
-
535
- * For order dependent results, the results are aggregated in the order they
536
- are received. The order of results is important for re-assembly or
537
- further processing.
538
-
539
- * For results that require aggregation, the `ResultsAggregator` is used to determine
540
- whether the results are completed, which signify that all work items have
541
- been processed and ready for further processing.
542
-
543
-
544
- ==== Accessing results
545
-
546
- To access successful results:
547
-
548
- [source,ruby]
549
- ----
550
- # Get all successful results
551
- successful_results = supervisor.results.results
552
-
553
- # Extract just the result values
554
- result_values = successful_results.map(&:result)
555
- ----
556
-
557
- To access errors:
558
-
559
- [source,ruby]
560
- ----
561
- # Get all error results
562
- error_results = supervisor.results.errors
563
-
564
- # Extract error messages
565
- error_messages = error_results.map(&:error)
566
-
567
- # Get the work items that failed
568
- failed_work_items = error_results.map(&:work)
569
- ----
570
-
571
-
572
- [TIP]
573
- ====
574
- * Check both successful results and errors after processing completes
575
- * Consider implementing custom reporting based on the aggregated results
576
- ====
577
-
578
-
579
- === WrappedRactor class
580
-
581
- ==== Purpose and responsibilities
582
-
583
- The `Fractor::WrappedRactor` class manages an individual Ruby Ractor, handling
584
- the communication between the Supervisor and the Worker instance running inside
585
- the Ractor.
586
-
587
- ==== Usage notes
588
-
589
- This class is primarily used internally by the Supervisor, but understanding its
590
- role helps with debugging:
591
-
592
- * Each WrappedRactor creates and manages one Ractor
593
- * The Worker instance lives inside the Ractor
594
- * Work items are sent to the Ractor via the WrappedRactor's `send` method
595
- * Results are yielded back to the Supervisor
596
-
597
- ==== Error propagation
598
-
599
- The WrappedRactor handles error propagation in two ways:
600
-
601
- . Errors from the Worker's `process` method are wrapped in a WorkResult and
602
- yielded back
603
- . Unexpected errors in the Ractor itself are caught and logged
604
-
605
-
606
- === Supervisor class
607
-
608
- ==== Purpose and responsibilities
609
-
610
- The `Fractor::Supervisor` class orchestrates the entire framework, managing
611
- worker Ractors, distributing work, and collecting results.
612
-
613
- ==== Configuration options
614
-
615
- When creating a Supervisor, you can configure:
616
-
617
- [source,ruby]
618
- ----
619
- supervisor = Fractor::Supervisor.new(
620
- worker_pools: [
621
- # Pool 1 - for general data processing
622
- { worker_class: MyWorker, num_workers: 4 },
623
-
624
- # Pool 2 - for specialized image processing
625
- { worker_class: ImageWorker, num_workers: 2 }
626
- ],
627
- continuous_mode: false # Optional: Run in continuous mode (default: false)
628
- )
629
- ----
630
-
631
- ==== Worker auto-detection
143
+ == Example applications
632
144
 
633
- Fractor automatically detects the number of available processors on your system
634
- and uses that value when `num_workers` is not specified. This provides optimal
635
- resource utilization across different deployment environments without requiring
636
- manual configuration.
145
+ === Pipeline mode
637
146
 
638
147
  [source,ruby]
639
148
  ----
640
- # Auto-detect number of workers (recommended for most cases)
641
- supervisor = Fractor::Supervisor.new(
642
- worker_pools: [
643
- { worker_class: MyWorker } # Will use number of available processors
644
- ]
645
- )
646
-
647
- # Explicitly set number of workers (useful for specific requirements)
648
- supervisor = Fractor::Supervisor.new(
649
- worker_pools: [
650
- { worker_class: MyWorker, num_workers: 4 } # Always use exactly 4 workers
651
- ]
652
- )
653
-
654
- # Mix auto-detection and explicit configuration
655
149
  supervisor = Fractor::Supervisor.new(
656
- worker_pools: [
657
- { worker_class: FastWorker }, # Auto-detected
658
- { worker_class: HeavyWorker, num_workers: 2 } # Explicitly 2 workers
659
- ]
150
+ worker_pools: [{ worker_class: DataWorker }]
660
151
  )
661
- ----
662
-
663
- The auto-detection uses Ruby's `Etc.nprocessors` which returns the number of
664
- available processors. If detection fails for any reason, it falls back to 2
665
- workers.
666
-
667
- [TIP]
668
- * Use auto-detection for portable code that adapts to different environments
669
- * Explicitly set `num_workers` when you need precise control over resource usage
670
- * Consider system load and other factors when choosing explicit values
671
-
672
- ==== Adding work
673
-
674
- You can add work items individually or in batches:
675
-
676
- [source,ruby]
677
- ----
678
- # Add a single item
679
- supervisor.add_work_item(MyWork.new(42))
680
-
681
- # Add multiple items
682
- supervisor.add_work_items([
683
- MyWork.new(1),
684
- MyWork.new(2),
685
- MyWork.new(3),
686
- MyWork.new(4),
687
- MyWork.new(5)
688
- ])
689
-
690
- # Add items of different work types
691
- supervisor.add_work_items([
692
- TextWork.new("Process this text"),
693
- ImageWork.new({ width: 800, height: 600 })
694
- ])
695
- ----
696
-
697
- The Supervisor can handle any Work object that inherits from Fractor::Work.
698
- Workers must check the type of Work they receive and process it accordingly.
699
-
700
- ==== Running and monitoring
701
152
 
702
- To start processing:
703
-
704
- [source,ruby]
705
- ----
706
- # Start processing and block until complete
153
+ supervisor.add_work_items(dataset.map { |item| DataWork.new(item) })
707
154
  supervisor.run
708
- ----
709
-
710
- The Supervisor automatically handles:
711
-
712
- * Starting the worker Ractors
713
- * Distributing work items to available workers
714
- * Collecting results and errors
715
- * Graceful shutdown on completion or interruption (Ctrl+C)
716
-
717
-
718
- ==== Accessing results
719
-
720
- After processing completes:
721
-
722
- [source,ruby]
723
- ----
724
- # Get the ResultAggregator
725
- aggregator = supervisor.results
726
155
 
727
- # Check counts
728
- puts "Processed #{aggregator.results.size} items successfully"
729
- puts "Encountered #{aggregator.errors.size} errors"
730
-
731
- # Access successful results
732
- aggregator.results.each do |result|
733
- puts "Work item #{result.work.input} produced #{result.result}"
734
- end
735
-
736
- # Access errors
737
- aggregator.errors.each do |error_result|
738
- puts "Work item #{error_result.work.input} failed: #{error_result.error}"
739
- end
156
+ puts "Processed #{supervisor.results.results.size} items"
740
157
  ----
741
158
 
742
- == Advanced usage patterns
743
-
744
- === Custom work distribution
159
+ See link:examples/simple/[Simple Example] and link:docs/examples.adoc#pipeline-mode-examples[more examples].
745
160
 
746
- For more complex scenarios, you might want to prioritize certain work items:
161
+ === Continuous mode
747
162
 
748
163
  [source,ruby]
749
164
  ----
750
- # Create Work objects for high priority items
751
- high_priority_works = high_priority_items.map { |item| MyWork.new(item) }
752
-
753
- # Add high-priority items first
754
- supervisor.add_work_items(high_priority_works)
755
-
756
- # Run with just enough workers for high-priority items
757
- supervisor.run
758
-
759
- # Create Work objects for lower priority items
760
- low_priority_works = low_priority_items.map { |item| MyWork.new(item) }
761
-
762
- # Add and process lower-priority items
763
- supervisor.add_work_items(low_priority_works)
764
- supervisor.run
765
- ----
165
+ work_queue = Fractor::WorkQueue.new
766
166
 
767
- === Handling large datasets
768
-
769
- For very large datasets, consider processing in batches:
770
-
771
- [source,ruby]
772
- ----
773
- large_dataset.each_slice(1000) do |batch|
774
- # Convert batch items to Work objects
775
- work_batch = batch.map { |item| MyWork.new(item) }
776
-
777
- supervisor.add_work_items(work_batch)
778
- supervisor.run
779
-
780
- # Process this batch's results before continuing
781
- process_batch_results(supervisor.results)
782
- end
783
- ----
784
-
785
-
786
- == Running a basic example
787
-
788
- . Install the gem as described in the Installation section.
789
-
790
- . Create a new Ruby file (e.g., `my_fractor_example.rb`) with your
791
- implementation:
792
-
793
- [source,ruby]
794
- ----
795
- require 'fractor'
796
-
797
- # Define your Work class
798
- class MyWork < Fractor::Work
799
- def to_s
800
- "MyWork: #{@input}"
801
- end
802
- end
803
-
804
- # Define your Worker class
805
- class MyWorker < Fractor::Worker
806
- def process(work)
807
- if work.input == 5
808
- # Return a Fractor::WorkResult for errors
809
- return Fractor::WorkResult.new(error: "Error processing work #{work.input}", work: work)
810
- end
811
-
812
- calculated = work.input * 2
813
- # Return a Fractor::WorkResult for success
814
- Fractor::WorkResult.new(result: calculated, work: work)
815
- end
816
- end
817
-
818
- # Create supervisor with a worker pool
819
- supervisor = Fractor::Supervisor.new(
820
- worker_pools: [
821
- { worker_class: MyWorker, num_workers: 2 }
822
- ]
167
+ server = Fractor::ContinuousServer.new(
168
+ worker_pools: [{ worker_class: MessageWorker, num_workers: 4 }],
169
+ work_queue: work_queue
823
170
  )
824
171
 
825
- # Create Work objects
826
- work_items = (1..10).map { |i| MyWork.new(i) }
827
-
828
- # Add work items
829
- supervisor.add_work_items(work_items)
830
-
831
- # Run processing
832
- supervisor.run
833
-
834
- # Display results
835
- puts "Results: #{supervisor.results.results.map(&:result).join(', ')}"
836
- puts "Errors: #{supervisor.results.errors.map { |e| e.work.input }.join(', ')}"
837
- ----
172
+ server.on_result { |result| puts "Processed: #{result.result}" }
173
+ server.on_error { |error| puts "Error: #{error.error}" }
838
174
 
839
- . Run the example from your terminal:
175
+ Thread.new { server.run }
840
176
 
841
- [source,sh]
177
+ # Add work dynamically
178
+ work_queue << MessageWork.new(client_id: 1, message: "Hello")
842
179
  ----
843
- ruby my_fractor_example.rb
844
- ----
845
-
846
- You will see output showing Ractors starting, receiving work, processing it, and
847
- the final aggregated results, including any errors encountered. Press `Ctrl+C`
848
- during execution to test the graceful shutdown.
849
-
850
180
 
851
- == Continuous mode
181
+ See link:examples/continuous_chat_fractor/[Chat Server Example] and link:docs/examples.adoc#continuous-mode-examples[more examples].
852
182
 
853
- === General
854
-
855
- Fractor provides a powerful feature called "continuous mode" that allows
856
- supervisors to run indefinitely, processing work items as they arrive without
857
- stopping after the initial work queue is empty.
858
-
859
- === Features
860
-
861
- * *Non-stopping Execution*: Supervisors run indefinitely until explicitly stopped
862
- * *On-demand Work*: Workers only process work when it's available
863
- * *Resource Efficiency*: Workers idle when no work is available, without consuming excessive resources
864
- * *Dynamic Work Addition*: New work can be added at any time through the work source callback
865
- * *Graceful Shutdown*: Resources are properly cleaned up when the supervisor is stopped
866
-
867
- Continuous mode is particularly useful for:
868
-
869
- * *Chat servers*: Processing incoming messages as they arrive
870
- * *Background job processors*: Handling tasks from a job queue
871
- * *Real-time data processing*: Analyzing data streams as they come in
872
- * *Web servers*: Responding to incoming requests in parallel
873
- * *Monitoring systems*: Continuously checking system statuses
874
-
875
- See the Chat Server example in the examples directory for a complete implementation of continuous mode.
876
-
877
-
878
- === Using continuous mode
879
-
880
- ==== Step 1. Create a supervisor with the `continuous_mode: true` option
183
+ === Workflows
881
184
 
882
185
  [source,ruby]
883
186
  ----
884
- supervisor = Fractor::Supervisor.new(
885
- worker_pools: [
886
- { worker_class: MyWorker, num_workers: 2 }
887
- ],
888
- continuous_mode: true # Enable continuous mode
889
- )
890
- ----
891
-
892
- ==== Step 2. Register a work source callback that provides new work on demand
893
-
894
- [source,ruby]
895
- ----
896
- supervisor.register_work_source do
897
- # Return nil or empty array if no work is available
898
- # Return a work item or array of work items when available
899
- items = get_next_work_items
900
- if items && !items.empty?
901
- # Convert to Work objects if needed
902
- items.map { |item| MyWork.new(item) }
903
- else
904
- nil
905
- end
187
+ # Define workflow with simplified syntax
188
+ workflow = Fractor::Workflow.define("data-pipeline") do
189
+ job :extract, ExtractWorker
190
+ job :transform, TransformWorker, needs: :extract
191
+ job :load, LoadWorker, needs: :transform
906
192
  end
907
- ----
908
-
909
- ==== Step 4. Run the supervisor in a non-blocking way
910
-
911
- Typically in a background thread.
912
-
913
- [source,ruby]
914
- ----
915
- supervisor_thread = Thread.new { supervisor.run }
916
- ----
917
193
 
918
- ==== Step 4. Explicitly call `stop` on the supervisor to stop processing
919
-
920
- [source,ruby]
921
- ----
922
- supervisor.stop
923
- supervisor_thread.join # Wait for the supervisor thread to finish
194
+ # Execute workflow
195
+ result = workflow.new.execute(input_data)
924
196
  ----
925
197
 
198
+ See link:examples/workflow/simplified/README.adoc[Simplified Workflows] and link:docs/workflows.adoc[Workflow Guide].
926
199
 
200
+ == Production deployment
927
201
 
928
- == Example applications
929
-
930
- === General
931
-
932
- The Fractor gem comes with several example applications that demonstrate various
933
- patterns and use cases. Each example can be found in the `examples` directory of
934
- the gem repository. Detailed descriptions for these are provided below.
935
-
936
- === Simple example
937
-
938
- The Simple Example (link:examples/simple/[examples/simple/]) demonstrates the
939
- basic usage of the Fractor framework. It shows how to create a simple Work
940
- class, a Worker class, and a Supervisor to manage the processing of work items
941
- in parallel. This example serves as a starting point for understanding how to
942
- use Fractor.
943
-
944
- Key features:
945
-
946
- * Basic Work and Worker class implementation
947
- * Simple Supervisor setup
948
- * Parallel processing of work items
949
- * Error handling and result aggregation
950
- * Auto-detection of available processors
951
- * Graceful shutdown on completion
952
-
953
- === Auto-detection example
954
-
955
- The Auto-Detection Example (link:examples/auto_detection/[examples/auto_detection/])
956
- demonstrates Fractor's automatic worker detection feature. It shows how to use
957
- auto-detection, explicit configuration, and mixed approaches for controlling
958
- the number of workers.
202
+ Fractor includes production-ready features:
959
203
 
960
- Key features:
204
+ * **Signal handling**: SIGTERM, SIGINT, SIGUSR1/SIGBREAK
205
+ * **Graceful shutdown**: Complete in-progress work before exit
206
+ * **Process monitoring**: Runtime status via signals
207
+ * **Structured logging**: JSON logging with correlation IDs
208
+ * **Workflow visualization**: Mermaid, DOT, ASCII diagrams
961
209
 
962
- * Automatic detection of available processors
963
- * Comparison of auto-detection vs explicit configuration
964
- * Mixed configuration with multiple worker pools
965
- * Best practices for worker configuration
966
- * Portable code that adapts to different environments
210
+ See link:docs/signal-handling.adoc[Signal Handling Guide] for deployment patterns.
967
211
 
968
- === Hierarchical hasher
212
+ == Contributing
969
213
 
970
- The Hierarchical Hasher example
971
- (link:examples/hierarchical_hasher/[examples/hierarchical_hasher/]) demonstrates
972
- how to use the Fractor framework to process a file in parallel by breaking it
973
- into chunks, hashing each chunk independently, and then combining the results
974
- into a final hash. This approach is useful for processing large files
975
- efficiently.
214
+ Bug reports and pull requests are welcome on GitHub at https://github.com/metanorma/fractor.
976
215
 
977
- Key features:
216
+ == License
978
217
 
979
- * Parallel data chunking for large files
980
- * Independent processing of data segments
981
- * Aggregation of results to form a final output
218
+ The gem is available as open source under the terms of the Ribose BSD 2-Clause License.
982
219
 
983
- === Multi-work type
984
-
985
- The Multi-Work Type example
986
- (link:examples/multi_work_type/[examples/multi_work_type/]) demonstrates how a
987
- single Fractor supervisor and worker can handle multiple types of work items
988
- (e.g., `TextWork` and `ImageWork`). The worker intelligently adapts its
989
- processing strategy based on the class of the incoming work item.
990
-
991
- Key features:
992
-
993
- * Support for multiple `Fractor::Work` subclasses
994
- * Polymorphic worker processing based on work type
995
- * Unified workflow for diverse tasks
996
-
997
- === Pipeline processing
998
-
999
- The Pipeline Processing example
1000
- (link:examples/pipeline_processing/[examples/pipeline_processing/]) implements a
1001
- multi-stage processing pipeline where data flows sequentially through a series
1002
- of transformations. The output of one stage becomes the input for the next, and
1003
- different stages can operate concurrently on different data items.
1004
-
1005
- Key features:
1006
-
1007
- * Sequential data flow through multiple processing stages
1008
- * Concurrent execution of different pipeline stages
1009
- * Data transformation at each step of the pipeline
1010
-
1011
- === Producer/subscriber
1012
-
1013
- The Producer/Subscriber example
1014
- (link:examples/producer_subscriber/[examples/producer_subscriber/]) showcases a
1015
- multi-stage document processing system where initial work (processing a
1016
- document) can generate additional sub-work items (processing sections of the
1017
- document). This creates a hierarchical processing pattern.
1018
-
1019
- Key features:
1020
-
1021
- * Implementation of producer-consumer patterns
1022
- * Dynamic generation of sub-work based on initial processing
1023
- * Construction of hierarchical result structures
1024
-
1025
- === Scatter/gather
1026
-
1027
- The Scatter/Gather example
1028
- (link:examples/scatter_gather/[examples/scatter_gather/]) illustrates how a
1029
- large task or dataset is broken down (scattered) into smaller, independent
1030
- subtasks. These subtasks are processed in parallel by multiple workers, and
1031
- their results are then collected (gathered) and combined to produce the final
1032
- output.
1033
-
1034
- Key features:
1035
-
1036
- * Distribution of a large task into smaller, parallelizable subtasks
1037
- * Concurrent processing of subtasks
1038
- * Aggregation of partial results into a final result
1039
-
1040
- === Specialized workers
1041
-
1042
- The Specialized Workers example
1043
- (link:examples/specialized_workers/[examples/specialized_workers/]) demonstrates
1044
- creating distinct worker types, each tailored to handle specific kinds of tasks
1045
- (e.g., `ComputeWorker` for CPU-intensive operations and `DatabaseWorker` for
1046
- I/O-bound database interactions). This allows for optimized resource utilization
1047
- and domain-specific logic.
1048
-
1049
- Key features:
1050
-
1051
- * Creation of worker classes for specific processing domains
1052
- * Routing of work items to appropriately specialized workers
1053
- * Optimization of resources and logic per task type
1054
-
1055
-
1056
-
1057
- == Copyright and license
220
+ == Copyright
1058
221
 
1059
222
  Copyright Ribose.
1060
-
1061
- Licensed under the MIT License.