miner_mover 0.0.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/README.md +321 -0
- data/Rakefile +20 -0
- data/VERSION +1 -0
- data/demo/config.rb +3 -0
- data/demo/fiber.rb +65 -0
- data/demo/fiber_scheduler.rb +117 -0
- data/demo/process.rb +109 -0
- data/demo/ractor.rb +117 -0
- data/demo/serial.rb +50 -0
- data/demo/thread.rb +91 -0
- data/lib/miner_mover/config.rb +78 -0
- data/lib/miner_mover/run.rb +67 -0
- data/lib/miner_mover/worker.rb +164 -0
- data/lib/miner_mover.rb +46 -0
- data/miner_mover.gemspec +30 -0
- data/test/miner_mover.rb +33 -0
- metadata +176 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: d6bf5c39cc32d1d7f8b7bf4528223eb5be897f59a62851e8875f396a43a4197f
|
4
|
+
data.tar.gz: 297267fba1f86e49871742687d5df26a730804cb2dc67026647bdc2dc7f49855
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: c7b446c0a36b0636cfd237b1b2adcbd0bbf9d2078a0eb0dc0246f678a9a46bfce0f7c1295e2c38a543dae192695dc23b87926bb3217f8c0ccaf878d57834702f
|
7
|
+
data.tar.gz: 9829ccf199d42f8d2a85a2aa392e0e89ef34a4f17d5de6bcd3d4adf5018216e7f80dc69fd7c4cd05384d20a0f25e961ce379ec320fadf536a2ae9452051e0a58
|
data/README.md
ADDED
@@ -0,0 +1,321 @@
|
|
1
|
+
[](https://github.com/rickhull/miner_mover/actions/workflows/test.yaml)
|
2
|
+
|
3
|
+
# Miner Mover
|
4
|
+
|
5
|
+
This project provides a basic concurrency problem useful for exploring
|
6
|
+
different multitasking paradigms available in Ruby. Fundamentally, we have a
|
7
|
+
set of *miners* and a set of *movers.* A *miner* takes some amount of time to
|
8
|
+
mine ore, which is given to a *mover*. When a *move*r has enough ore for a full
|
9
|
+
batch, the delivery takes some amount of time before more ore can be
|
10
|
+
loaded.
|
11
|
+
|
12
|
+
## Mining
|
13
|
+
|
14
|
+
A miner is given some depth (e.g. 1 to 100) to mine down to, which will
|
15
|
+
take an increasing amount of time with depth. More depth provides greater ore
|
16
|
+
results as well. Ore is gathered at each depth; either a fixed amount or
|
17
|
+
randomized, based on depth. The amount of time spent mining each level is
|
18
|
+
independent and may be randomized.
|
19
|
+
|
20
|
+
https://github.com/rickhull/miner_mover/blob/bd76ea400944aab8eab9e3ffcac85d1e28353eff/lib/miner_mover/worker.rb#L85-L99
|
21
|
+
|
22
|
+
In this case, miners are rewarded by calculating `fibonacci(depth)`, using
|
23
|
+
classic, inefficient fibonacci. 10M ore represents `fibonacci(35)`, which
|
24
|
+
takes around 0.75 seconds on my local VM.
|
25
|
+
|
26
|
+
## Moving
|
27
|
+
|
28
|
+
A mover has a batch size, say 10. As the mover accumulates ore over time,
|
29
|
+
once the batch size is reached, the mover delivers the ore to the destination.
|
30
|
+
Larger batches take longer. The delivery time can be randomized.
|
31
|
+
|
32
|
+
https://github.com/rickhull/miner_mover/blob/bd76ea400944aab8eab9e3ffcac85d1e28353eff/lib/miner_mover/worker.rb#L135-L162
|
33
|
+
|
34
|
+
The time and work spent delivering ore can be simulated three ways,
|
35
|
+
configured via `:work_type`
|
36
|
+
|
37
|
+
* `:wait` - represents waiting on IO; calls `sleep(duration)`
|
38
|
+
* `:cpu` - busy work; calls `fibonacci(30)` until `duration` is reached
|
39
|
+
* `:instant` - useful for testing; returns immediately
|
40
|
+
|
41
|
+
# Usage
|
42
|
+
|
43
|
+
## Install
|
44
|
+
|
45
|
+
You'll want to use Ruby 3.x to make the most of Fibers.
|
46
|
+
|
47
|
+
Clone the repo, then install dependencies:
|
48
|
+
|
49
|
+
* rake
|
50
|
+
* minitest
|
51
|
+
* compsci
|
52
|
+
* dotcfg
|
53
|
+
* fiber_scheduler
|
54
|
+
|
55
|
+
`gem install rake minitest compsci dotcfg fiber_scheduler`
|
56
|
+
|
57
|
+
## Satisfy `LOAD_PATH`
|
58
|
+
|
59
|
+
Execute scripts and irb sessions from the project root, e.g. `~/miner_mover`.
|
60
|
+
Use `-I lib` as a flag to `ruby` or `irb` to add e.g. `~/miner_mover/lib`
|
61
|
+
to `LOAD_PATH` so that `require 'miner_mover'` will work.
|
62
|
+
This project does not use `require_relative`.
|
63
|
+
|
64
|
+
## Exploration in `irb`
|
65
|
+
|
66
|
+
`$ irb -Ilib -rminer_mover/worker`
|
67
|
+
|
68
|
+
```
|
69
|
+
irb(main):001:0> include MinerMover
|
70
|
+
=> Object
|
71
|
+
|
72
|
+
irb(main):002:0> miner = Miner.new
|
73
|
+
=>
|
74
|
+
#<MinerMover::Miner:0x00007fdd649754e8
|
75
|
+
...
|
76
|
+
|
77
|
+
irb(main):003:0> miner.mine_ore
|
78
|
+
=> 0
|
79
|
+
|
80
|
+
irb(main):004:0> miner.mine_ore 5
|
81
|
+
=> 3
|
82
|
+
|
83
|
+
irb(main):005:0> miner.mine_ore 20
|
84
|
+
=> 6483
|
85
|
+
|
86
|
+
irb(main):006:0> mover = Mover.new
|
87
|
+
=>
|
88
|
+
#<MinerMover::Mover:0x00007fdd64979930
|
89
|
+
...
|
90
|
+
|
91
|
+
irb(main):007:0> mover.load_ore 6483
|
92
|
+
=> 6483
|
93
|
+
|
94
|
+
irb(main):008:0> mover.status
|
95
|
+
=> "Batch 6483 / 10M 0% | Moved 0x (0M)"
|
96
|
+
```
|
97
|
+
|
98
|
+
## Included scripts
|
99
|
+
|
100
|
+
These scripts implement a full miner mover simulation using different
|
101
|
+
multitasking paradigms in Ruby.
|
102
|
+
|
103
|
+
* [`demo/serial.rb`](demo/serial.rb)
|
104
|
+
* [`demo/fiber.rb`](demo/fiber.rb)
|
105
|
+
* [`demo/fiber_scheduler.rb`](demo/fiber_scheduler.rb)
|
106
|
+
* [`demo/thread.rb`](demo/thread.rb)
|
107
|
+
* [`demo/ractor.rb`](demo/ractor.rb)
|
108
|
+
|
109
|
+
See [config/example.cfg](config/example.cfg) for configuration.
|
110
|
+
It will be loaded by default.
|
111
|
+
Note that serial.rb and fiber.rb have no concurrency and cannot use multiple
|
112
|
+
miners or movers.
|
113
|
+
|
114
|
+
Execute via e.g. `ruby -Ilib demo/ractor.rb`
|
115
|
+
|
116
|
+
# Multitasking
|
117
|
+
|
118
|
+
*Multitasking* here means "the most general sense of performing several tasks
|
119
|
+
or actions *at the same time*". *At the same time* can mean fast switching
|
120
|
+
between tasks, or left and right hands operating truly in parallel.
|
121
|
+
|
122
|
+
## Concurrency
|
123
|
+
|
124
|
+
In the broadest sense, two tasks are *concurrent* if they happen *at the
|
125
|
+
same time*, as above. When I tell Siri to call home while I drive, I perform
|
126
|
+
these tasks concurrently.
|
127
|
+
|
128
|
+
## Parallelism
|
129
|
+
|
130
|
+
In the strictest sense of parallelism, one executes several *identical* tasks
|
131
|
+
using multiple *facilities* that operate independently and in parallel.
|
132
|
+
Multiple lanes on a highway offer parallelism for the task of driving from
|
133
|
+
A to B.
|
134
|
+
|
135
|
+
If there is a bucket brigade to put out a fire, all members of the brigade are
|
136
|
+
operating in parallel. The last brigade member is dousing the fire instead of
|
137
|
+
handing the bucket to the next member. While this might not meet the most
|
138
|
+
strict definition of parallelism, it is broadly accepted as parallel. It is
|
139
|
+
certainly concurrent. Often though, *concurrent* means *merely concurrent*,
|
140
|
+
where there is only one *facility* switching between tasks rather than multiple
|
141
|
+
devices operating in parallel.
|
142
|
+
|
143
|
+
## Multitasking from the perspective of the OS (Linux, Windows, MacOS)
|
144
|
+
|
145
|
+
* A modern OS executes _threads_ within a _process_
|
146
|
+
* Processes are mostly about accounting and containment
|
147
|
+
- Organization and safety from other processes and users
|
148
|
+
* By default, a process has a single thread of execution
|
149
|
+
* A single-threaded process cannot (easily) perform two tasks concurrently
|
150
|
+
- Maybe it implements green threads or coroutines?
|
151
|
+
* A process can (easily) create additional threads for multitasking
|
152
|
+
- Either within this process or via spawning a child process
|
153
|
+
* Process spawning implies more overhead than thread creation
|
154
|
+
- Threads can only share memory within a process
|
155
|
+
- fork() / CoW can provide thread-like efficiency
|
156
|
+
* Child processes are managed differently than threads
|
157
|
+
- Memory protection
|
158
|
+
- OS integration / init system
|
159
|
+
|
160
|
+
# Multitasking in Ruby
|
161
|
+
|
162
|
+
The default Ruby runtime is known as CRuby, named for its implementation in
|
163
|
+
the C language, also known as MRI (Matz Ruby Interpreter), named for its
|
164
|
+
creator Yukihiro Matsumoto. Some history:
|
165
|
+
|
166
|
+
## Before YARV (up to Ruby 1.9):
|
167
|
+
|
168
|
+
* Execute-as-we-interpret
|
169
|
+
* Ruby code executes as the main thread of the main process
|
170
|
+
* Green threads implemented and scheduled by the runtime (not OS threads)
|
171
|
+
* GIL (Global Interpreter Lock) implies threads cannot execute in parallel
|
172
|
+
* Occasional concurrency, when a waiting thread is scheduled out in favor of a
|
173
|
+
running thread
|
174
|
+
- `schedule(waiting, running) YES`
|
175
|
+
- `schedule(waiting, waiting) NO`
|
176
|
+
- `schedule(running, running) NO`
|
177
|
+
- `schedule(running, waiting) OH DEAR`
|
178
|
+
|
179
|
+
## YARV (Ruby 1.9 through 3.x):
|
180
|
+
|
181
|
+
* Interpret to bytecode, then execute
|
182
|
+
* YARV (Yet Another Ruby VM) is introduced, providing a runtime virtual
|
183
|
+
machine for executing bytecode
|
184
|
+
* Fiber is introduced for cooperative multitasking, lighter than threads
|
185
|
+
* Ruby code executes as the main fiber of the main thread of the main process
|
186
|
+
* Ruby threads are implemented as OS threads, scheduled by the OS
|
187
|
+
* YARV is single threaded (not threadsafe) thus requring a Global VM Lock (GVL)
|
188
|
+
- GVL is more fine grained than GIL
|
189
|
+
- Threads explicitly give up the execution lock when waiting (IO, sleep, etc)
|
190
|
+
* YARV typically achieves 2-4x concurrency with multiple threads
|
191
|
+
- Less concurrency when threads are CPU bound (thus waiting on GVL)
|
192
|
+
- More concurrency when threads are IO bound (thus yielding GVL)
|
193
|
+
- Less concurrency when not enough threads (GVL is underutilized)
|
194
|
+
- More waiting (BAD!) when too many threads (GVL is under contention)
|
195
|
+
- Thus, tuning is required, with many pathological cases
|
196
|
+
- Thread pools (where most threads are idle) make tuning easier but still
|
197
|
+
required
|
198
|
+
* As before, processes can be spawned for more more true parallelism
|
199
|
+
- typically via `fork` with Copy-on-write for efficiency
|
200
|
+
- management of child process lifecycles can be more difficult than
|
201
|
+
multithreading
|
202
|
+
- multiprocessing and multithreading can be combined, often with differing
|
203
|
+
task shapes
|
204
|
+
* Fibers offer even lighter weight concurrency primitives
|
205
|
+
|
206
|
+
### Fibers
|
207
|
+
|
208
|
+
```
|
209
|
+
Fiber.yield(arg) # call within a Fiber to suspend execution and yield a value
|
210
|
+
Fiber#resume # tell a Fiber to proceed and return the next yielded value
|
211
|
+
```
|
212
|
+
|
213
|
+
```ruby
|
214
|
+
fiber = Fiber.new do
|
215
|
+
Fiber.yield 1
|
216
|
+
2
|
217
|
+
end
|
218
|
+
|
219
|
+
fiber.resume
|
220
|
+
#=> 1
|
221
|
+
|
222
|
+
fiber.resume
|
223
|
+
#=> 2
|
224
|
+
|
225
|
+
fiber.resume
|
226
|
+
# FiberError: attempt to resume a terminated fiber
|
227
|
+
```
|
228
|
+
|
229
|
+
Any argument(s) passed to `Fiber#resume` on its first call (to start the Fiber)
|
230
|
+
will be passed to the `Fiber.new` block:
|
231
|
+
|
232
|
+
```ruby
|
233
|
+
fiber = Fiber.new do |arg1, arg2|
|
234
|
+
Fiber.yield arg1
|
235
|
+
arg2
|
236
|
+
end
|
237
|
+
|
238
|
+
fiber.resume(:x, :y)
|
239
|
+
#=> :x
|
240
|
+
|
241
|
+
fiber.resume
|
242
|
+
#=> :y
|
243
|
+
```
|
244
|
+
|
245
|
+
## YARV with `Fiber::Scheduler` (Ruby 3.x)
|
246
|
+
|
247
|
+
* Non-blocking fibers are introduced
|
248
|
+
- any waits that would cause a fiber to block will cause the fiber to suspend
|
249
|
+
- `Fiber::Scheduler` is introduced to manage non-blocking fibers
|
250
|
+
|
251
|
+
### Non-blocking Fibers
|
252
|
+
|
253
|
+
The concept of non-blocking fiber was introduced in Ruby 3.0. A non-blocking
|
254
|
+
fiber, when reaching a operation that would normally block the fiber (like
|
255
|
+
sleep, or wait for another process or I/O) will yield control to other fibers
|
256
|
+
and allow the scheduler to handle blocking and waking up (resuming) this fiber
|
257
|
+
when it can proceed.
|
258
|
+
|
259
|
+
For a Fiber to behave as non-blocking, it need to be created in `Fiber.new`
|
260
|
+
with `blocking: false` (which is the default), and `Fiber.scheduler` should be
|
261
|
+
set with `Fiber.set_scheduler`. If `Fiber.scheduler` is not set in the current
|
262
|
+
thread, blocking and non-blocking fibers’ behavior is identical.
|
263
|
+
|
264
|
+
Thus, any fiber without a scheduler is a blocking fiber. If a fiber is created
|
265
|
+
with `blocking: true`, it is a blocking fiber. Otherwise, if it has a
|
266
|
+
scheduler, it is non-blocking.
|
267
|
+
|
268
|
+
### Fiber scheduling
|
269
|
+
|
270
|
+
```
|
271
|
+
Fiber.scheduler # get the current scheduler
|
272
|
+
Fiber.set_scheduler # set the current scheduler
|
273
|
+
Fiber.schedule # perform a given block in a non-blocking manner
|
274
|
+
Fiber::Scheduler # scheduler interface
|
275
|
+
```
|
276
|
+
|
277
|
+
### `Fiber::Scheduler`
|
278
|
+
|
279
|
+
* `Fiber::Scheduler` is **not an implementation** but an **interface**
|
280
|
+
* The implementation is provided by a library / gem / user
|
281
|
+
|
282
|
+
## YARV with Ractors (Ruby 3.x, experimental)
|
283
|
+
|
284
|
+
* YARV allows multiple threads but locks areas where multiple threads have
|
285
|
+
access to the same data
|
286
|
+
* Ractors are introduced, with no shared data, requiring messages to be passed
|
287
|
+
between Ractors
|
288
|
+
* Ruby code executes as the main Fiber of the main Thread of the main Ractor
|
289
|
+
of the main Process
|
290
|
+
* The default thread within each Ractor has its own OS thread, with as much
|
291
|
+
parallelism as the host OS provides
|
292
|
+
* Additional threads spawned by a Ractor are normal OS threads but they must
|
293
|
+
contend for the Ractor Lock (RL) to execute on YARV
|
294
|
+
|
295
|
+
### Ractors
|
296
|
+
|
297
|
+
Ractors are an abstraction and a container for threads. Threads within a
|
298
|
+
Ractor can share memory. Threads must use message passaging to communicate
|
299
|
+
across Ractors. Also, Ractors hold the execution lock on YARV, so threads
|
300
|
+
in different Ractors have zero contention.
|
301
|
+
|
302
|
+
```ruby
|
303
|
+
# get the current Ractor object
|
304
|
+
r = Ractor.current
|
305
|
+
|
306
|
+
# create a new Ractor (block will execute in parallel via thread creation)
|
307
|
+
Ractor.new(arg1, arg2, etc) { |arg1, arg2, etc|
|
308
|
+
# now use arg1 and arg2 from outside
|
309
|
+
}
|
310
|
+
```
|
311
|
+
|
312
|
+
* Ractors communicate via messages
|
313
|
+
* send via outgoing port
|
314
|
+
* receive via incoming port (infinite storage, FIFO)
|
315
|
+
|
316
|
+
```
|
317
|
+
Ractor#send - puts a message at the incoming port of a Ractor
|
318
|
+
Ractor.receive - returns a message from the current Ractor's incoming port
|
319
|
+
Ractor.yield - current Ractor sends a message on the outgoing port
|
320
|
+
Ractor#take - returns the next outgoing message from a Ractor
|
321
|
+
```
|
data/Rakefile
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
require 'rake/testtask'
|
2
|
+
|
3
|
+
Rake::TestTask.new :test do |t|
|
4
|
+
t.pattern = "test/*.rb"
|
5
|
+
t.warning = true
|
6
|
+
end
|
7
|
+
|
8
|
+
task default: :test
|
9
|
+
|
10
|
+
begin
|
11
|
+
require 'buildar'
|
12
|
+
|
13
|
+
Buildar.new do |b|
|
14
|
+
b.gemspec_file = 'miner_mover.gemspec'
|
15
|
+
b.version_file = 'VERSION'
|
16
|
+
b.use_git = true
|
17
|
+
end
|
18
|
+
rescue LoadError
|
19
|
+
warn "buildar tasks unavailable"
|
20
|
+
end
|
data/VERSION
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
0.0.0.3
|
data/demo/config.rb
ADDED
data/demo/fiber.rb
ADDED
@@ -0,0 +1,65 @@
|
|
1
|
+
require 'miner_mover/run'
|
2
|
+
|
3
|
+
include MinerMover
|
4
|
+
|
5
|
+
run = Run.new.cfg_banner!(duration: 1)
|
6
|
+
run.timer.timestamp!
|
7
|
+
run.log "Starting"
|
8
|
+
|
9
|
+
stop_mining = false
|
10
|
+
Signal.trap("INT") {
|
11
|
+
run.timer.timestamp!
|
12
|
+
run.log " *** SIGINT *** Stop Mining"
|
13
|
+
stop_mining = true
|
14
|
+
}
|
15
|
+
|
16
|
+
# miner runs in its own Fiber
|
17
|
+
miner = Fiber.new(blocking: true) {
|
18
|
+
run.log "MINE Mining operation started [ctrl-c] to stop"
|
19
|
+
m = run.new_miner
|
20
|
+
|
21
|
+
ore_mined = 0
|
22
|
+
|
23
|
+
# miner waits for the SIGINT signal to quit
|
24
|
+
while !stop_mining
|
25
|
+
ore = m.mine_ore
|
26
|
+
|
27
|
+
# send any ore mined to the mover
|
28
|
+
Fiber.yield ore if ore > 0
|
29
|
+
ore_mined += ore
|
30
|
+
|
31
|
+
# stop mining after a while
|
32
|
+
if run.time_limit? or run.ore_limit?(ore_mined)
|
33
|
+
run.timer.timestamp!
|
34
|
+
m.log format("Mining limit reached: %s", Ore.display(ore_mined))
|
35
|
+
stop_mining = true
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
m.log format("MINE Miner finished after mining %s", Ore.display(ore_mined))
|
40
|
+
Fiber.yield :quit
|
41
|
+
ore_mined
|
42
|
+
}
|
43
|
+
|
44
|
+
mover = run.new_mover
|
45
|
+
run.log "MOVE Moving operation started"
|
46
|
+
run.log "WAIT Waiting for ore ..."
|
47
|
+
|
48
|
+
loop {
|
49
|
+
# pick up ore yielded by the miner
|
50
|
+
ore = miner.resume
|
51
|
+
break if ore == :quit
|
52
|
+
|
53
|
+
# load (and possibly move) the ore
|
54
|
+
mover.load_ore ore if ore > 0
|
55
|
+
}
|
56
|
+
|
57
|
+
# miner has quit; move any remaining ore and quit
|
58
|
+
mover.move_batch while mover.batch > 0
|
59
|
+
run.log "QUIT #{mover.status}"
|
60
|
+
|
61
|
+
ore_mined = miner.resume
|
62
|
+
ore_moved = mover.ore_moved
|
63
|
+
run.log format("MINE %s mined (%i)", Ore.display(ore_mined), ore_mined)
|
64
|
+
run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
|
65
|
+
run.timer.timestamp!
|
@@ -0,0 +1,117 @@
|
|
1
|
+
require 'miner_mover/run'
|
2
|
+
require 'fiber_scheduler'
|
3
|
+
|
4
|
+
include MinerMover
|
5
|
+
|
6
|
+
run = Run.new.cfg_banner!(duration: 1)
|
7
|
+
run.timer.timestamp!
|
8
|
+
run.log "Starting"
|
9
|
+
|
10
|
+
stop_mining = false
|
11
|
+
Signal.trap("INT") {
|
12
|
+
run.timer.timestamp!
|
13
|
+
run.log " *** SIGINT *** Stop Mining"
|
14
|
+
stop_mining = true
|
15
|
+
}
|
16
|
+
|
17
|
+
# for moving ore
|
18
|
+
queue = Thread::Queue.new
|
19
|
+
|
20
|
+
# for signalling between miners and supervisor
|
21
|
+
mutex = Mutex.new
|
22
|
+
miner_quit = ConditionVariable.new
|
23
|
+
|
24
|
+
# for getting results from scheduled fibers
|
25
|
+
mined = Thread::Queue.new
|
26
|
+
moved = Thread::Queue.new
|
27
|
+
|
28
|
+
# follow the rabbit
|
29
|
+
FiberScheduler do
|
30
|
+
|
31
|
+
# several miners, stored in an array
|
32
|
+
miners = Array.new(run.num_miners) { |i|
|
33
|
+
|
34
|
+
# each miner gets a fiber
|
35
|
+
Fiber.schedule do
|
36
|
+
m = run.new_miner
|
37
|
+
m.log "MINE Miner #{i} started"
|
38
|
+
|
39
|
+
ore_mined = 0
|
40
|
+
|
41
|
+
# miner waits for the SIGINT signal to quit
|
42
|
+
while !stop_mining
|
43
|
+
ore = m.mine_ore
|
44
|
+
|
45
|
+
# send any ore mined to the mover
|
46
|
+
queue.push(ore) if ore > 0
|
47
|
+
ore_mined += ore
|
48
|
+
|
49
|
+
# stop mining after a while
|
50
|
+
if run.time_limit? or run.ore_limit?(ore_mined)
|
51
|
+
run.timer.timestamp!
|
52
|
+
m.log format("Mining limit reached: %s", Ore.display(ore_mined))
|
53
|
+
stop_mining = true
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
m.log format("MINE Miner #{i} finished after mining %s",
|
58
|
+
Ore.display(ore_mined))
|
59
|
+
|
60
|
+
# register the ore mined (scheduled fiber can't return a value)
|
61
|
+
mined.push ore_mined
|
62
|
+
|
63
|
+
# signal to the supervisor that a miner is done
|
64
|
+
mutex.synchronize { miner_quit.signal }
|
65
|
+
end
|
66
|
+
}
|
67
|
+
|
68
|
+
# several movers, no need to store
|
69
|
+
run.num_movers.times { |i|
|
70
|
+
|
71
|
+
# each mover gets a fiber
|
72
|
+
Fiber.schedule do
|
73
|
+
m = run.new_mover
|
74
|
+
m.log "MOVE Mover #{i} started"
|
75
|
+
|
76
|
+
loop {
|
77
|
+
# pick up ore from the miner until we get a :quit message
|
78
|
+
ore = queue.pop
|
79
|
+
break if ore == :quit
|
80
|
+
|
81
|
+
# load (and possibly move) the ore
|
82
|
+
m.load_ore ore if ore > 0
|
83
|
+
}
|
84
|
+
|
85
|
+
# miners have quit; move any remaining ore and quit
|
86
|
+
m.move_batch while m.batch > 0
|
87
|
+
m.log "QUIT #{m.status}"
|
88
|
+
|
89
|
+
# register the ore moved (scheduled fiber can't return a value)
|
90
|
+
moved.push m.ore_moved
|
91
|
+
end
|
92
|
+
}
|
93
|
+
|
94
|
+
# supervisor waits for the miners to quit
|
95
|
+
# and signals the mover to quit by pushing :quit onto the queue
|
96
|
+
Fiber.schedule do
|
97
|
+
# every time a miner quits, check if any are left
|
98
|
+
mutex.synchronize { miner_quit.wait(mutex) while miners.any?(&:alive?) }
|
99
|
+
|
100
|
+
# tell every mover to quit
|
101
|
+
run.num_movers.times { queue.push(:quit) }
|
102
|
+
|
103
|
+
# queue closes once it is empty
|
104
|
+
# should helpfully cause errors if something is out of sync
|
105
|
+
queue.close
|
106
|
+
end
|
107
|
+
end
|
108
|
+
|
109
|
+
total_mined = 0
|
110
|
+
total_mined += mined.pop until mined.empty?
|
111
|
+
|
112
|
+
total_moved = 0
|
113
|
+
total_moved += moved.pop until moved.empty?
|
114
|
+
|
115
|
+
run.log format("MINE %s mined (%i)", Ore.display(total_mined), total_mined)
|
116
|
+
run.log format("MOVE %s moved (%i)", Ore.display(total_moved), total_moved)
|
117
|
+
run.timer.timestamp!
|
data/demo/process.rb
ADDED
@@ -0,0 +1,109 @@
|
|
1
|
+
require 'miner_mover/run'
|
2
|
+
require 'thread'
|
3
|
+
|
4
|
+
include MinerMover
|
5
|
+
|
6
|
+
run = Run.new.cfg_banner!(duration: 1)
|
7
|
+
run.timer.timestamp!
|
8
|
+
run.log "Starting"
|
9
|
+
|
10
|
+
stop_mining = false
|
11
|
+
Signal.trap("INT") {
|
12
|
+
run.timer.timestamp!
|
13
|
+
run.log " *** SIGINT *** Stop Mining"
|
14
|
+
stop_mining = true
|
15
|
+
}
|
16
|
+
|
17
|
+
mover = Process.fork {
|
18
|
+
run.log "MOVE Moving operation started"
|
19
|
+
|
20
|
+
# mover queue
|
21
|
+
queue = Thread::Queue.new
|
22
|
+
|
23
|
+
# store the mover threads in an array
|
24
|
+
movers = Array.new(run.num_movers) { |i|
|
25
|
+
Thread.new {
|
26
|
+
m = run.new_mover
|
27
|
+
m.log "MOVE Mover #{i} started"
|
28
|
+
|
29
|
+
loop {
|
30
|
+
# a mover picks up ore from the queue
|
31
|
+
run.debug && m.log("POP ")
|
32
|
+
ore = queue.pop
|
33
|
+
run.debug && m.log("POPD #{ore}")
|
34
|
+
|
35
|
+
break if ore == :quit
|
36
|
+
|
37
|
+
# load (and possibly move) the ore
|
38
|
+
m.load_ore ore
|
39
|
+
}
|
40
|
+
|
41
|
+
# move any remaining ore and quit
|
42
|
+
m.move_batch while m.batch > 0
|
43
|
+
m.log "QUIT #{m.status}"
|
44
|
+
m
|
45
|
+
}
|
46
|
+
}
|
47
|
+
|
48
|
+
run.log "WAIT Waiting for ore ..."
|
49
|
+
|
50
|
+
# TODO: pipe or unix socket for ore
|
51
|
+
|
52
|
+
TODO = :notyet
|
53
|
+
|
54
|
+
loop {
|
55
|
+
# pull from the pipe / socket
|
56
|
+
ore = TODO
|
57
|
+
break if ore == :quit
|
58
|
+
queue.push ore
|
59
|
+
}
|
60
|
+
}
|
61
|
+
|
62
|
+
# our mining operation executes in the main Process, here
|
63
|
+
run.log "MINE Mining operation started [ctrl-c] to stop"
|
64
|
+
|
65
|
+
# store the miner threads in an array
|
66
|
+
miners = Array.new(run.num_miners) { |i|
|
67
|
+
Thread.new {
|
68
|
+
m = run.new_miner
|
69
|
+
m.log "MINE Miner #{i} started"
|
70
|
+
ore_mined = 0
|
71
|
+
|
72
|
+
# miners wait for the SIGINT signal to quit
|
73
|
+
while !stop_mining
|
74
|
+
ore = m.mine_ore
|
75
|
+
|
76
|
+
# send any ore mined to the mover Ractor
|
77
|
+
if ore > 0
|
78
|
+
run.debug && m.log("SEND #{ore}")
|
79
|
+
mover.send ore
|
80
|
+
run.debug && m.log("SENT #{ore}")
|
81
|
+
end
|
82
|
+
|
83
|
+
ore_mined += ore
|
84
|
+
|
85
|
+
# stop mining after a while
|
86
|
+
if run.time_limit? or run.ore_limit?(ore_mined)
|
87
|
+
run.timer.timestamp!
|
88
|
+
m.log format("Mining limit reached: %s", Ore.display(ore_mined))
|
89
|
+
stop_mining = true
|
90
|
+
end
|
91
|
+
end
|
92
|
+
|
93
|
+
m.log format("MINE Miner %i finished after mining %s",
|
94
|
+
i, Ore.display(ore_mined))
|
95
|
+
ore_mined
|
96
|
+
}
|
97
|
+
}
|
98
|
+
|
99
|
+
# wait on all mining threads to stop
|
100
|
+
ore_mined = miners.map { |thr| thr.value }.sum
|
101
|
+
run.log format("MINE %s mined (%i)", Ore.display(ore_mined), ore_mined)
|
102
|
+
|
103
|
+
# tell mover to quit
|
104
|
+
mover.send :quit
|
105
|
+
|
106
|
+
# wait for results
|
107
|
+
ore_moved = mover.take
|
108
|
+
run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
|
109
|
+
run.timer.timestamp!
|