miner_mover 0.0.0.4 → 0.1.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3292fa885539a3a1c4aca0940e71dfb03ca69aeff10526ec643c6cc86f2829f6
-  data.tar.gz: 59bab2d1982bf27221b722407fe15a61d25d9e3da3e24ceec31349ef31a154a5
+  metadata.gz: 7d6d5e0e85b066dc78e4e6c7b86d751245f1e2b3809ab176f9b69e7047c2d330
+  data.tar.gz: cd0f57ea360871fefdb7927eddef3651a277c8e8116d41a2da085dcf02a3ed73
 SHA512:
-  metadata.gz: 174fdfc1a8b30977acf4a5934f087c89da67cd12e1d6ea70f863d7d3a93c58d984b63449c4f48a5e2e53fd8f477eb493c79abfa7c805c038474312c1c6b294e5
-  data.tar.gz: 6322cb259e9e65d5c57a59a3ab4ea20d7d0e1ef7c5414be691faae6ae7fd19bd0bb5007cd3dd9af87624b5081ddb1fa05d676bb63fcb317ebd40e8964e169729
+  metadata.gz: 1a8d2759692483f004c0d512a17b2b4ec6a10e6d26b3e66b8785b28a0510cc3eaf7ad029b95f0a11a37b0264dc028e3c65a5cbccffbc64e5446365cdaf9ae2ce
+  data.tar.gz: d88946c92b88073fc01f4b57ae0f13ba157ca7a72c85ae6ad405a0569b311e5420689d76014c5cda88c709b2dfbfa2dbed6b090e2c001894fdf4267686cbd87d

data/README.md

@@ -1,4 +1,5 @@
 [![Test Status](https://github.com/rickhull/miner_mover/actions/workflows/test.yaml/badge.svg)](https://github.com/rickhull/miner_mover/actions/workflows/test.yaml)
+[![Demo Status](https://github.com/rickhull/miner_mover/actions/workflows/demo.yaml/badge.svg)](https://github.com/rickhull/miner_mover/actions/workflows/demo.yaml)
 
 # Miner Mover
 
@@ -20,8 +21,9 @@ independent and may be randomized.
 https://github.com/rickhull/miner_mover/blob/bd76ea400944aab8eab9e3ffcac85d1e28353eff/lib/miner_mover/worker.rb#L85-L99
 
 In this case, miners are rewarded by calculating `fibonacci(depth)`, using
-classic, inefficient fibonacci.  10M ore represents `fibonacci(35)`, which
-takes around 0.75 seconds on my local VM.
+classic, inefficient fibonacci.
+`fibonacci(35)` yields around 10M ore, while `fibonacci(30)` yields under
+1M ore.
 
 ## Moving
 
@@ -42,24 +44,72 @@ configured via `:work_type`
 
 ## Install
 
-You'll want to use Ruby 3.x to make the most of Fibers.
+You'll want to use **Ruby 3.x** to make the most of Fibers.
 
-Clone the repo, then install dependencies:
+### Dependencies
 
 * rake
 * minitest
-* compsci
 * dotcfg
 * fiber_scheduler
 
-`gem install rake minitest compsci dotcfg fiber_scheduler`
+`gem install miner_mover` will take care of the following:
+
+`gem install rake minitest dotcfg fiber_scheduler`
+
+### Clone
+
+```
+git clone https://github.com/rickhull/miner_mover
+cd miner_mover
+```
+
+### Try rake
+
+Try: `rake -T` to see available [Rake tasks](Rakefile)
+
+```
+$ rake -T
+
+rake alt_demo         # run all demos minus fiber_scheduler / ractor / process
+rake config           # run demo/config
+rake default          # rake test
+rake demo             # run all demos
+rake fiber            # run demo/fiber
+rake fiber_scheduler  # run demo/fiber_scheduler
+rake process_pipe     # run demo/process_pipe
+rake process_socket   # run demo/process_socket
+rake ractor           # run demo/ractor
+rake serial           # run demo/serial
+rake test             # Run tests
+rake thread           # run demo/thread
+```
+
+## Rake Tasks
+
+Included demonstration scripts can be executed via Rake tasks.
+The following order is recommended:
+
+* `rake config`
+* `rake serial`
+* `rake fiber`
+* `rake fiber_scheduler`
+* `rake thread`
+* `rake process_pipe`
+* `rake process_socket`
+
+Try each task; there will be about 6 seconds worth of many lines of output
+logging.  These rake tasks correspond to the scripts within [`demo/`](demo/).
 
 ## Satisfy `LOAD_PATH`
 
-Execute scripts and irb sessions from the project root, e.g. `~/miner_mover`.
-Use `-I lib` as a flag to `ruby` or `irb` to add e.g. `~/miner_mover/lib`
-to `LOAD_PATH` so that `require 'miner_mover'` will work.
-This project does not use `require_relative`.
+Rake tasks take care of `LOAD_PATH`, so the following is
+**only necessary when *not* using rake tasks**:
+
+* Execute scripts and irb sessions from the project root, e.g. `~/miner_mover`
+* Use `-I lib` as a flag to `ruby` or `irb` to update `LOAD_PATH` so that
+  `require 'miner_mover'` will work.
+* This project does not use `require_relative`
 
 ## Exploration in `irb`
 
@@ -105,13 +155,15 @@ multitasking paradigms in Ruby.
 * [`demo/fiber_scheduler.rb`](demo/fiber_scheduler.rb)
 * [`demo/thread.rb`](demo/thread.rb)
 * [`demo/ractor.rb`](demo/ractor.rb)
+* [`demo/process_pipe.rb`](demo/process_pipe.rb)
+* [`demo/process_socket.rb`](demo/process_socket.rb)
 
 See [config/example.cfg](config/example.cfg) for configuration.
 It will be loaded by default.
-Note that serial.rb and fiber.rb have no concurrency and cannot use multiple
-miners or movers.
+Note that `serial.rb` and `fiber.rb` have no concurrency and cannot use
+multiple miners or movers.
 
-Execute via e.g. `ruby -Ilib demo/ractor.rb`
+Execute via e.g. `ruby -Ilib demo/thread.rb`
 
 # Multitasking
 
@@ -319,3 +371,19 @@ Ractor.receive - returns a message from the current Ractor's incoming port
 Ractor.yield   - current Ractor sends a message on the outgoing port
 Ractor#take    - returns the next outgoing message from a Ractor
 ```
+
+## Processes
+
+There are many ways to create a process in Ruby, some more useful than
+others.  My favorites:
+
+* `Process.fork` - when called with a block, the block is only executed in the
+  child subprocess
+* `Process.spawn` - extensive options, nonblocking, call Process.wait(pid)
+  to get the result
+* `Open3.popen3` - for access to STDIN, STDOUT, STDERR
+
+### IPC
+
+* Pipes - streaming bytes: `IO.pipe`
+* Unix sockets - messages with datagrams: `Socket.pair(:UNIX, :DGRAM, 0)`

data/Rakefile

@@ -5,16 +5,31 @@ Rake::TestTask.new :test do |t|
   t.warning = true
 end
 
+desc "rake test"
 task default: :test
 
+Dir['demo/*.rb'].each { |demo_path|
+  name = File.basename(demo_path, '.rb')
+  desc "run demo/#{name}"
+  task(name) { sh "ruby -Ilib #{demo_path}" }
+}
+
+# jruby / truffleruby lack fiber_scheduler, Ractor, and Process#fork
+desc "run all demos minus fiber_scheduler / ractor / process"
+task jvm_demo: [:serial, :fiber, :thread]
+
+desc "run all demos"
+task demo: [:serial, :fiber, :fiber_scheduler,
+            :thread, :ractor, :process_pipe, :process_socket]
+
 begin
   require 'buildar'
 
-    Buildar.new do |b|
+  Buildar.new do |b|
     b.gemspec_file = 'miner_mover.gemspec'
     b.version_file = 'VERSION'
     b.use_git = true
   end
 rescue LoadError
-  warn "buildar tasks unavailable"
+  # warn "buildar tasks unavailable"
 end

data/VERSION

@@ -1 +1 @@
-0.0.0.4
+0.1.0.1

data/demo/fiber.rb

@@ -2,13 +2,13 @@ require 'miner_mover/run'
 
 include MinerMover
 
-run = Run.new.cfg_banner!(duration: 1)
-run.timer.timestamp!
-run.log "Starting"
+run = Run.new.cfg_banner!(duration: 1).start!
+run.timestamp!
+run.log "Starting #{__FILE__}"
 
 stop_mining = false
 Signal.trap("INT") {
-  run.timer.timestamp!
+  run.timestamp!
   run.log " *** SIGINT ***  Stop Mining"
   stop_mining = true
 }
@@ -17,20 +17,16 @@ Signal.trap("INT") {
 miner = Fiber.new(blocking: true) {
   run.log "MINE Mining operation started  [ctrl-c] to stop"
   m = run.new_miner
-
   ore_mined = 0
 
-  # miner waits for the SIGINT signal to quit
-  while !stop_mining
+  while !stop_mining # SIGINT will trigger stop_mining = true
     ore = m.mine_ore
-
-    # send any ore mined to the mover
-    Fiber.yield ore if ore > 0
     ore_mined += ore
+    Fiber.yield ore if ore > 0
 
     # stop mining after a while
     if run.time_limit? or run.ore_limit?(ore_mined)
-      run.timer.timestamp!
+      run.timestamp!
       m.log format("Mining limit reached: %s", Ore.display(ore_mined))
       stop_mining = true
     end
@@ -45,16 +41,14 @@ mover = run.new_mover
 run.log "MOVE Moving operation started"
 run.log "WAIT Waiting for ore ..."
 
+# mover pulls from the queue, loads the ore, and moves it
 loop {
-  # pick up ore yielded by the miner
   ore = miner.resume
   break if ore == :quit
-
-  # load (and possibly move) the ore
-  mover.load_ore ore if ore > 0
+  mover.load_ore ore if ore > 0 # move_batch happens when a batch is full
 }
 
-# miner has quit; move any remaining ore and quit
+# move any remaining ore and quit
 mover.move_batch while mover.batch > 0
 run.log "QUIT #{mover.status}"
 
@@ -62,4 +56,4 @@ ore_mined = miner.resume
 ore_moved = mover.ore_moved
 run.log format("MINE %s mined (%i)", Ore.display(ore_mined), ore_mined)
 run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
-run.timer.timestamp!
+run.timestamp!

data/demo/fiber_scheduler.rb

@@ -3,13 +3,13 @@ require 'fiber_scheduler'
 
 include MinerMover
 
-run = Run.new.cfg_banner!(duration: 1)
-run.timer.timestamp!
-run.log "Starting"
+run = Run.new.cfg_banner!(duration: 1).start!
+run.timestamp!
+run.log "Starting #{__FILE__}"
 
 stop_mining = false
 Signal.trap("INT") {
-  run.timer.timestamp!
+  run.timestamp!
   run.log " *** SIGINT ***  Stop Mining"
   stop_mining = true
 }
@@ -35,20 +35,16 @@ FiberScheduler do
     Fiber.schedule do
       m = run.new_miner
       m.log "MINE Miner #{i} started"
-
       ore_mined = 0
 
-      # miner waits for the SIGINT signal to quit
-      while !stop_mining
+      while !stop_mining # SIGINT will trigger stop_mining = true
         ore = m.mine_ore
-
-        # send any ore mined to the mover
-        queue.push(ore) if ore > 0
         ore_mined += ore
+        queue.push(ore) if ore > 0
 
         # stop mining after a while
         if run.time_limit? or run.ore_limit?(ore_mined)
-          run.timer.timestamp!
+          run.timestamp!
           m.log format("Mining limit reached: %s", Ore.display(ore_mined))
           stop_mining = true
         end
@@ -73,16 +69,14 @@ FiberScheduler do
       m = run.new_mover
       m.log "MOVE Mover #{i} started"
 
+      # movers pull from the queue, load the ore, and move it
       loop {
-        # pick up ore from the miner until we get a :quit message
         ore = queue.pop
         break if ore == :quit
-
-        # load (and possibly move) the ore
-        m.load_ore ore if ore > 0
+        m.load_ore ore if ore > 0 # move_batch happens when a batch is full
       }
 
-      # miners have quit; move any remaining ore and quit
+      # move any remaining ore and quit
       m.move_batch while m.batch > 0
       m.log "QUIT #{m.status}"
 
@@ -100,9 +94,8 @@ FiberScheduler do
     # tell every mover to quit
     run.num_movers.times { queue.push(:quit) }
 
-    # queue closes once it is empty
-    # should helpfully cause errors if something is out of sync
-    queue.close
+    # queue only closes once it is empty
+    queue.close # should helpfully cause errors if something is out of sync
   end
 end
 
@@ -114,4 +107,4 @@ total_moved += moved.pop until moved.empty?
 
 run.log format("MINE %s mined (%i)", Ore.display(total_mined), total_mined)
 run.log format("MOVE %s moved (%i)", Ore.display(total_moved), total_moved)
-run.timer.timestamp!
+run.timestamp!

data/demo/{process.rb → process_pipe.rb}

@@ -3,22 +3,32 @@ require 'thread'
 
 include MinerMover
 
-run = Run.new.cfg_banner!(duration: 1)
-run.timer.timestamp!
-run.log "Starting"
+run = Run.new.cfg_banner!(duration: 1).start!
+run.timestamp!
+run.log "Starting #{__FILE__}"
 
 stop_mining = false
 Signal.trap("INT") {
-  run.timer.timestamp!
+  run.timestamp!
   run.log " *** SIGINT ***  Stop Mining"
   stop_mining = true
 }
 
+pipe_reader, pipe_writer = IO.pipe
+
+def pipe_reader.pop
+  Ore.decode self.read(Ore::WORD_LENGTH)
+end
+
+def pipe_writer.push amt
+  self.write(Ore.encode(amt))
+end
+
+# the moving operation executes in its own Process
 mover = Process.fork {
   run.log "MOVE Moving operation started"
-
-  # mover queue
-  queue = Thread::Queue.new
+  pipe_writer.close # we're only using pipe_reader in this process
+  queue = Thread::Queue.new # distribute incoming ore to mover threads
 
   # store the mover threads in an array
   movers = Array.new(run.num_movers) { |i|
@@ -26,16 +36,11 @@ mover = Process.fork {
       m = run.new_mover
       m.log "MOVE Mover #{i} started"
 
+      # movers pull from the queue, load the ore, and move it
       loop {
-        # a mover picks up ore from the queue
-        run.debug && m.log("POP ")
         ore = queue.pop
-        run.debug && m.log("POPD #{ore}")
-
         break if ore == :quit
-
-        # load (and possibly move) the ore
-        m.load_ore ore
+        m.load_ore ore # move_batch happens when a batch is full
       }
 
       # move any remaining ore and quit
@@ -45,22 +50,25 @@ mover = Process.fork {
     }
   }
 
+  # Miners feed this Process with ore
+  # Pass the ore into a queue for the movers
+  # When the miners say to quit, tell the movers to quit
   run.log "WAIT Waiting for ore ..."
-
-  # TODO: pipe or unix socket for ore
-
-  TODO = :notyet
-
   loop {
-    # pull from the pipe / socket
-    ore = TODO
-    break if ore == :quit
+    ore = pipe_reader.pop
+    break if ore == 0 # signal to quit
     queue.push ore
   }
+
+  # tell all the movers to quit and gather their results
+  run.num_movers.times { queue.push :quit }
+  ore_moved = movers.map { |thr| thr.value.ore_moved }.sum
+  run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
 }
 
-# our mining operation executes in the main Process, here
+# our mining operation executes in the main process, here
 run.log "MINE Mining operation started  [ctrl-c] to stop"
+pipe_reader.close # we're only using pipe_writer in this process
 
 # store the miner threads in an array
 miners = Array.new(run.num_miners) { |i|
@@ -69,22 +77,14 @@ miners = Array.new(run.num_miners) { |i|
     m.log "MINE Miner #{i} started"
     ore_mined = 0
 
-    # miners wait for the SIGINT signal to quit
-    while !stop_mining
+    while !stop_mining # SIGINT will trigger stop_mining = true
       ore = m.mine_ore
-
-      # send any ore mined to the mover Ractor
-      if ore > 0
-        run.debug && m.log("SEND #{ore}")
-        mover.send ore
-        run.debug && m.log("SENT #{ore}")
-      end
-
       ore_mined += ore
+      pipe_writer.push(ore) if ore > 0
 
       # stop mining after a while
       if run.time_limit? or run.ore_limit?(ore_mined)
-        run.timer.timestamp!
+        run.timestamp!
         m.log format("Mining limit reached: %s", Ore.display(ore_mined))
         stop_mining = true
       end
@@ -101,9 +101,8 @@ ore_mined = miners.map { |thr| thr.value }.sum
 run.log format("MINE %s mined (%i)", Ore.display(ore_mined), ore_mined)
 
 # tell mover to quit
-mover.send :quit
+pipe_writer.push 0
 
 # wait for results
-ore_moved = mover.take
-run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
-run.timer.timestamp!
+Process.wait
+run.timestamp!

data/demo/process_socket.rb

@@ -0,0 +1,111 @@
+require 'miner_mover/run'
+require 'thread'
+require 'socket'
+
+include MinerMover
+
+run = Run.new.cfg_banner!(duration: 1).start!
+run.timestamp!
+run.log "Starting #{__FILE__}"
+
+stop_mining = false
+Signal.trap("INT") {
+  run.timestamp!
+  run.log " *** SIGINT ***  Stop Mining"
+  stop_mining = true
+}
+
+socktype = [:DGRAM, :STREAM, :RAW].sample
+run.log "SOCK #{socktype}"
+csock, psock = UNIXSocket.pair(socktype)
+
+def csock.pop
+  Ore.decode self.recv(Ore::WORD_LENGTH)
+end
+
+def psock.push amt
+  self.send(Ore.encode(amt), 0)
+end
+
+# the moving operation executes in its own Process
+mover = Process.fork {
+  run.log "MOVE Moving operation started"
+  psock.close # we're only using csock in this process
+  queue = Thread::Queue.new # distribute incoming ore to mover threads
+
+  # store the mover threads in an array
+  movers = Array.new(run.num_movers) { |i|
+    Thread.new {
+      m = run.new_mover
+      m.log "MOVE Mover #{i} started"
+
+      # movers pull from the queue, load the ore, and move it
+      loop {
+        ore = queue.pop
+        break if ore == :quit
+        m.load_ore ore # move_batch happens when a batch is full
+      }
+
+      # move any remaining ore and quit
+      m.move_batch while m.batch > 0
+      m.log "QUIT #{m.status}"
+      m
+    }
+  }
+
+  # Miners feed this Process with ore
+  # Pass the ore into a queue for the movers
+  # When the miners say to quit, tell the movers to quit
+  run.log "WAIT Waiting for ore ..."
+  loop {
+    ore = csock.pop
+    break if ore == 0 # signal to quit
+    queue.push ore
+  }
+
+  # tell all the movers to quit and gather their results
+  run.num_movers.times { queue.push :quit }
+  ore_moved = movers.map { |thr| thr.value.ore_moved }.sum
+  run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
+}
+
+# our mining operation executes in the main process, here
+run.log "MINE Mining operation started  [ctrl-c] to stop"
+csock.close # we're only using psock in this process
+
+# store the miner threads in an array
+miners = Array.new(run.num_miners) { |i|
+  Thread.new {
+    m = run.new_miner
+    m.log "MINE Miner #{i} started"
+    ore_mined = 0
+
+    while !stop_mining # SIGINT will trigger stop_mining = true
+      ore = m.mine_ore
+      ore_mined += ore
+      psock.push(ore) if ore > 0
+
+      # stop mining after a while
+      if run.time_limit? or run.ore_limit?(ore_mined)
+        run.timestamp!
+        m.log format("Mining limit reached: %s", Ore.display(ore_mined))
+        stop_mining = true
+      end
+    end
+
+    m.log format("MINE Miner %i finished after mining %s",
+                 i, Ore.display(ore_mined))
+    ore_mined
+  }
+}
+
+# wait on all mining threads to stop
+ore_mined = miners.map { |thr| thr.value }.sum
+run.log format("MINE %s mined (%i)", Ore.display(ore_mined), ore_mined)
+
+# tell mover to quit
+psock.push 0
+
+# wait for results
+Process.wait
+run.timestamp!

data/demo/ractor.rb

@@ -3,13 +3,13 @@ require 'thread'
 
 include MinerMover
 
-run = Run.new.cfg_banner!(duration: 1)
-run.timer.timestamp!
-run.log "Starting"
+run = Run.new.cfg_banner!(duration: 1).start!
+run.timestamp!
+run.log "Starting #{__FILE__}"
 
 stop_mining = false
 Signal.trap("INT") {
-  run.timer.timestamp!
+  run.timestamp!
   run.log " *** SIGINT ***  Stop Mining"
   stop_mining = true
 }
@@ -27,16 +27,11 @@ mover = Ractor.new(run) { |r|
       m = r.new_mover
       m.log "MOVE Mover #{i} started"
 
+      # movers pull from the queue, load the ore, and move it
       loop {
-        # a mover picks up ore from the queue
-        r.debug && m.log("POP ")
         ore = queue.pop
-        r.debug && m.log("POPD #{ore}")
-
         break if ore == :quit
-
-        # load (and possibly move) the ore
-        m.load_ore ore
+        m.load_ore ore # move_batch happens when a batch is full
       }
 
       # move any remaining ore and quit
@@ -53,13 +48,8 @@ mover = Ractor.new(run) { |r|
   loop {
     # when the Ractor gets ore, push it into the queue
     ore = Ractor.recv
-    r.debug && r.log("RECV #{ore}")
-
     break if ore == :quit
-
-    r.debug && r.log("PUSH #{ore}")
     queue.push ore
-    r.debug && r.log("PSHD #{ore}")
   }
 
   # tell all the movers to quit and gather their results
@@ -77,22 +67,14 @@ miners = Array.new(run.num_miners) { |i|
     m.log "MINE Miner #{i} started"
     ore_mined = 0
 
-    # miners wait for the SIGINT signal to quit
-    while !stop_mining
+    while !stop_mining # SIGINT will trigger stop_mining = true
       ore = m.mine_ore
-
-      # send any ore mined to the mover Ractor
-      if ore > 0
-        run.debug && m.log("SEND #{ore}")
-        mover.send ore
-        run.debug && m.log("SENT #{ore}")
-      end
-
       ore_mined += ore
+      mover.send ore if ore > 0 # send any ore mined to the mover Ractor
 
       # stop mining after a while
       if run.time_limit? or run.ore_limit?(ore_mined)
-        run.timer.timestamp!
+        run.timestamp!
         m.log format("Mining limit reached: %s", Ore.display(ore_mined))
         stop_mining = true
       end
@@ -114,4 +96,4 @@ mover.send :quit
 # wait for results
 ore_moved = mover.take
 run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
-run.timer.timestamp!
+run.timestamp!

data/demo/serial.rb

@@ -2,39 +2,32 @@ require 'miner_mover/run'
 
 include MinerMover
 
-run = Run.new.cfg_banner!(duration: 1)
-run.timer.timestamp!
-run.log "Starting"
+run = Run.new.cfg_banner!(duration: 1).start!
+run.timestamp!
+run.log "Starting #{__FILE__}"
 
 stop_mining = false
 Signal.trap("INT") {
-  run.timer.timestamp!
+  run.timestamp!
   run.log " *** SIGINT ***  Stop Mining"
   stop_mining = true
 }
 
-# system 'cpulimit', "--pid=#{Process.pid}", '--limit=1', '--background'
-
 miner = run.new_miner
 run.log "MINE Mining operation initialized  [ctrl-c] to stop"
 
 mover = run.new_mover
 run.log "MOVE Moving operation initialized"
-
 ore_mined = 0
 
-# miner waits for the SIGINT signal to quit
-while !stop_mining
-  # mine the ore
+while !stop_mining # SIGINT will trigger stop_mining = true
   ore = miner.mine_ore
   ore_mined += ore
-
-  # load (and possibly move) the ore
-  mover.load_ore ore if ore > 0
+  mover.load_ore ore if ore > 0 # move_batch happens when a batch is full
 
   # stop mining after a while
   if run.time_limit? or run.ore_limit?(ore_mined)
-    run.timer.timestamp!
+    run.timestamp!
     miner.log format("Mining limit reached: %s", Ore.display(ore_mined))
     stop_mining = true
   end
@@ -47,4 +40,4 @@ run.log "QUIT #{mover.status}"
 ore_moved = mover.ore_moved
 run.log format("MINE %s mined (%i)", Ore.display(ore_mined), ore_mined)
 run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
-run.timer.timestamp!
+run.timestamp!

data/demo/thread.rb

@@ -3,13 +3,13 @@ require 'thread'
 
 include MinerMover
 
-run = Run.new.cfg_banner!(duration: 1)
-run.timer.timestamp!
-run.log "Starting"
+run = Run.new.cfg_banner!(duration: 1).start!
+run.timestamp!
+run.log "Starting #{__FILE__}"
 
 stop_mining = false
 Signal.trap("INT") {
-  run.timer.timestamp!
+  run.timestamp!
   run.log " *** SIGINT ***  Stop Mining"
   stop_mining = true
 }
@@ -24,16 +24,11 @@ movers = Array.new(run.num_movers) { |i|
     m = run.new_mover
     run.log "MOVE Mover #{i} started"
 
+    # movers pull from the queue, load the ore, and move it
     loop {
-      # a mover picks up mined ore from the queue
-      run.debug && m.log("POP ")
       ore = queue.pop
-      run.debug && m.log("POPD #{ore}")
-
       break if ore == :quit
-
-      # load (and possibly move) the ore
-      m.load_ore ore
+      m.load_ore ore # moving of ore possibly happens here (on a full batch)
     }
 
     # move any remaining ore and quit
@@ -45,6 +40,7 @@ movers = Array.new(run.num_movers) { |i|
 
 
 run.log "MINE Mining operation started  [ctrl-c] to stop"
+
 # store the miner threads in an array
 miners = Array.new(run.num_miners) { |i|
   Thread.new {
@@ -55,19 +51,12 @@ miners = Array.new(run.num_miners) { |i|
     # miners wait for the SIGINT signal to quit
     while !stop_mining
       ore = m.mine_ore
-
-      # send any ore mined to the movers
-      if ore > 0
-        run.debug && m.log("PUSH #{ore}")
-        queue.push ore
-        run.debug && m.log("PSHD #{ore}")
-      end
-
       ore_mined += ore
+      queue.push ore if ore > 0 # send any ore mined to the movers
 
       # stop mining after a while
       if run.time_limit? or run.ore_limit?(ore_mined)
-        run.timer.timestamp!
+        run.timestamp!
         m.log format("Mining limit reached: %s", Ore.display(ore_mined))
         stop_mining = true
       end
@@ -86,6 +75,7 @@ run.log format("MINE %s mined (%i)", Ore.display(ore_mined), ore_mined)
 # tell all the movers to quit; gather their results
 run.num_movers.times { queue.push :quit }
 
+# wait for results
 ore_moved = movers.map { |thr| thr.value.ore_moved }.sum
 run.log format("MOVE %s moved (%i)", Ore.display(ore_moved), ore_moved)
-run.timer.timestamp!
+run.timestamp!

data/lib/miner_mover/config.rb

@@ -14,19 +14,22 @@ module MinerMover
         time_limit: 5,
         ore_limit: 100,
         logging: true,
+        debugging: false,
       }.freeze,
       miner: {
         depth: 30,
         partial_reward: false,
         variance: 0,
         logging: true,
+        debugging: false,
       }.freeze,
       mover: {
-        batch_size: 10,
-        rate: 2,
+        batch_size: 8,
+        rate: 4,
         work_type: :wait,
         variance: 0,
         logging: true,
+        debugging: false,
       }.freeze,
     }.freeze
 
@@ -60,7 +63,7 @@ module MinerMover
         mover = cfg[:mover] || {}
         main  = cfg[:main]  || {}
       else
-        raise(Error, "couldn't find miner, mover, or main in #{file}")
+        miner, mover, main = {}, {}, {}
       end
       { miner: DEFAULT[:miner].merge(miner),
         mover: DEFAULT[:mover].merge(mover),

data/lib/miner_mover/run.rb

@@ -1,5 +1,6 @@
 require 'miner_mover/worker'
 require 'miner_mover/config'
+require 'miner_mover/timer'
 
 module MinerMover
   class Run
@@ -15,35 +16,47 @@ module MinerMover
       f
     end
 
-    attr_accessor :debug, :logging
+    attr_accessor :debugging, :logging
     attr_accessor :num_miners, :num_movers
     attr_accessor :cfg_file, :cfg, :miner, :mover, :timer
     attr_accessor :time_limit, :ore_limit
 
-    def initialize(cfg_file: nil, timer: nil, debug: false)
+    def initialize(cfg_file: nil, timer: nil)
       @cfg_file = self.class.cfg_file(cfg_file)
       @cfg = Config.process @cfg_file
       main  = @cfg.fetch :main
       @miner = @cfg.fetch :miner
       @mover = @cfg.fetch :mover
 
+      @num_miners = main.fetch :num_miners
+      @num_movers = main.fetch :num_movers
+
       @time_limit = main.fetch :time_limit
       @ore_limit  = main.fetch :ore_limit
       @logging    = main.fetch :logging
-      @num_miners = main.fetch :num_miners
-      @num_movers = main.fetch :num_movers
+      @debugging  = main.fetch :debugging
 
-      @timer = timer || CompSci::Timer.new
-      @debug = debug
+      @timer = timer || Timer.new
     end
 
     def cfg_banner!(duration: 0)
-      log "USING: #{@cfg_file}"
+      MinerMover.puts "USING: #{@cfg_file}"
       pp @cfg
       sleep duration if duration > 0
       self
     end
 
+    def start!
+      @timer.restart
+      self
+    end
+
+    def timestamp!
+      dash = '-' * 70
+      str = [dash, Timer.timestamp, dash].join(MinerMover::LINE_SEP)
+      MinerMover.puts str
+    end
+
     def new_miner
       Miner.new(**@miner)
     end
@@ -61,7 +74,11 @@ module MinerMover
     end
 
     def log msg
-      @logging and puts(MinerMover.log_fmt(@timer, ' (main) ', msg))
+      @logging and MinerMover.log @timer, ' (main) ', msg
+    end
+
+    def debug msg
+      @debugging and MinerMover.log @timer, '(debug) ', msg
     end
   end
 end

data/lib/miner_mover/timer.rb

@@ -0,0 +1,67 @@
+module MinerMover
+  class Timer
+    SECS_PER_MIN = 60
+    MINS_PER_HOUR = 60
+    SECS_PER_HOUR = SECS_PER_MIN * MINS_PER_HOUR
+
+    # returns a float representing seconds since epoch
+    if defined? Process::CLOCK_MONOTONIC
+      def self.now
+        Process.clock_gettime Process::CLOCK_MONOTONIC
+      end
+    else
+      def self.now
+        Time.now.to_f
+      end
+    end
+
+    def self.since f
+      self.now - f
+    end
+
+    def self.elapsed &work
+      f = self.now
+      return yield, self.since(f)
+    end
+
+    # HH::MM::SS.mmm.uuuuuuuu
+    def self.elapsed_display(elapsed_ms, show_micro: false)
+      elapsed_s, ms = elapsed_ms.divmod 1000
+      ms_only, ms_fraction = ms.round(8).divmod 1
+
+      h = elapsed_s / SECS_PER_HOUR
+      elapsed_s -= h * SECS_PER_HOUR
+      m, s = elapsed_s.divmod SECS_PER_MIN
+
+      hmsms = [[h, m, s].map { |i| i.to_s.rjust(2, '0') }.join(':'),
+               ms_only.to_s.rjust(3, '0')]
+      hmsms << (ms_fraction * 10 ** 8).round.to_s.ljust(8, '0') if show_micro
+      hmsms.join('.')
+    end
+
+    # YYYY-MM-DD HH::MM::SS.mmm
+    def self.timestamp(t = Time.now)
+      t.strftime "%Y-%m-%d %H:%M:%S.%L"
+    end
+
+    def restart(t = Time.now)
+      @start = t
+      self
+    end
+    alias_method :initialize, :restart
+
+    def elapsed(t = Time.now)
+      t - @start
+    end
+
+    def elapsed_ms(t = Time.now)
+      elapsed(t) * 1000
+    end
+
+    def elapsed_display(t = Time.now)
+      Timer.elapsed_display(elapsed_ms(t))
+    end
+    alias_method :to_s, :elapsed_display
+    alias_method :inspect, :elapsed_display
+  end
+end

data/lib/miner_mover/worker.rb

@@ -1,16 +1,15 @@
 require 'miner_mover'
-require 'compsci/timer'
-require 'compsci/fibonacci'
+require 'miner_mover/timer'
 
 module MinerMover
-  def self.work(duration, type = :wait, fib = 30)
+  def self.work(duration, type = :wait, fib_target = 30)
     case type
     when :wait
       sleep duration
       duration
     when :cpu
-      t = CompSci::Timer.new
-      CompSci::Fibonacci.classic(fib) while t.elapsed < duration
+      t = Timer.new
+      self.fib(fib_target) while t.elapsed < duration
       t.elapsed
     when :instant
       0
@@ -19,14 +18,19 @@ module MinerMover
     end
   end
 
+  def self.fib n
+    (n < 2) ?  n  :  fib(n-1) + fib(n-2)
+  end
+
   class Worker
-    attr_accessor :variance, :logging
+    attr_accessor :variance, :logging, :debugging
     attr_reader :timer
 
-    def initialize(variance: 0, logging: false, timer: nil)
+    def initialize(variance: 0, logging: false, debugging: false, timer: nil)
       @variance = variance
       @logging = logging
-      @timer = timer || CompSci::Timer.new
+      @debugging = debugging
+      @timer = timer || Timer.new
     end
 
     def id
@@ -36,6 +40,7 @@ module MinerMover
     def state
       { id: self.id,
         logging: @logging,
+        debugging: @debugging,
         timer: @timer.elapsed_ms.round,
         variance: @variance }
     end
@@ -45,7 +50,11 @@ module MinerMover
     end
 
     def log msg
-      @logging && puts(MinerMover.log_fmt(@timer, self.id, msg))
+      @logging and MinerMover.log @timer, self.id, msg
+    end
+
+    def debug msg
+      @debugging and MinerMover.log @timer, self.id, msg
     end
 
     # 4 levels:
@@ -72,10 +81,12 @@ module MinerMover
                    partial_reward: true,
                    variance: 0,
                    logging: false,
+                   debugging: false,
                    timer: nil)
       @partial_reward = partial_reward
       @depth = depth
-      super(variance: variance, logging: logging, timer: timer)
+      super(variance: variance, logging: logging,
+            debugging: debugging, timer: timer)
     end
 
     def state
@@ -84,11 +95,11 @@ module MinerMover
 
     def mine_ore(depth = @depth)
       log format("MINE Depth %i", depth)
-      ores, elapsed = CompSci::Timer.elapsed {
+      ores, elapsed = Timer.elapsed {
         # every new depth is a new mining operation
         Array.new(depth) { |d|
           # mine ore by calculating fibonacci for that depth
-          mined = CompSci::Fibonacci.classic(self.varied(d).round)
+          mined = MinerMover.fib self.varied(d).round
           @partial_reward ? rand(1 + mined) : mined
         }
       }
@@ -107,12 +118,14 @@ module MinerMover
                    work_type: :cpu,
                    variance: 0,
                    logging: false,
+                   debugging: false,
                    timer: nil)
       @batch_size = batch_size * Ore::BLOCK
       @rate = rate.to_f * Ore::BLOCK
       @work_type = work_type
       @batch, @batches, @ore_moved = 0, 0, 0
-      super(variance: variance, logging: logging, timer: timer)
+      super(variance: variance, logging: logging,
+            debugging: debugging, timer: timer)
     end
 
     def state
@@ -149,7 +162,7 @@ module MinerMover
       duration = self.varied(amt / @rate)
 
       log format("MOVE %s (%.2f s)", Ore.display(amt), duration)
-      _, elapsed = CompSci::Timer.elapsed { self.move(duration) }
+      _, elapsed = Timer.elapsed { self.move(duration) }
       log format("MOVD %s (%.2f s)", Ore.display(amt), elapsed)
 
       # accounting

data/lib/miner_mover.rb

@@ -1,7 +1,30 @@
 module MinerMover
-  # called by Worker instances, available for general use
+  LINE_SEP = $/.freeze
+
+  # $/ is the default line separator: "\n"
+  def self.puts(str, io = $stdout, separator = LINE_SEP)
+    self.write_nonblock(io, str + separator)
+  end
+
+  def self.write_nonblock(io, str)
+    begin
+      # nonblocking write attempt; ensure the full string is written
+      size = str.bytesize
+      num_bytes = io.write_nonblock(str)
+      # blocking write if nonblocking write comes up short
+      io.write str.byteslice(num_bytes, size - num_bytes) if num_bytes < size
+    rescue IO::WaitWritable, Errno::EINTR
+      IO.select([], [io]) # wait until writable
+      retry
+    end
+  end
+
   def self.log_fmt(timer, id, msg)
-    format("%s %s %s", timer.elapsed_display, id, msg)
+    format("%s %s %s", timer, id, msg)
+  end
+
+  def self.log(timer, id, msg)
+    self.puts self.log_fmt(timer, id, msg)
   end
 
   # i +- 50% at squeeze 0
@@ -18,7 +41,28 @@ module MinerMover
 
   # ore is handled in blocks of 1M
   module Ore
-    BLOCK = 1_000_000
+    BLOCK = 1_000_000  # between fib(30) and fib(31)
+    WORD_LENGTH = 4    # bytes
+    WORD_MAX = 256 ** WORD_LENGTH      # up to 4.3B ore
+    HEX_UNPACK = "H#{WORD_LENGTH * 2}" # 4 bytes is 8 hex digits
+
+    # return 4 bytes, unsigned 32 bit integer, network order
+    def self.encode(ore)
+      raise "WORD_MAX overflow: #{self.block(ore)}M ore" if ore > WORD_MAX
+      [ore].pack('N')
+    end
+
+    # return an integer from the first 4 bytes
+    def self.decode(word)
+      raise "unexpected size: #{word.bytesize}" unless word.bytesize == 4
+      word.unpack('N').first
+    end
+
+    # return "0x01020304" for "\x01\x02\x03\x04"
+    def self.hex(word)
+      raise "unexpected size: #{word.bytesize}" unless word.bytesize == 4
+      "0x" + word.unpack(HEX_UNPACK).first
+    end
 
     # raw ore in, blocks out
     def self.block(ore, size = BLOCK)

data/miner_mover.gemspec

@@ -18,13 +18,16 @@ EOF
   s.files += Dir['test/**/*.rb']
   s.files += Dir['demo/**/*.rb']
 
-  s.add_dependency "compsci", "~> 0.3"
   s.add_dependency "dotcfg", "~> 1.0"
-  s.add_dependency "fiber_scheduler", "~> 0.13"
 
-  s.add_development_dependency "buildar", "~> 3.0"
-  s.add_development_dependency "minitest", "~> 5.0"
+  s.requirements << "For all features and best performance:"
+  s.requirements << "gem install rake minitest fiber_scheduler"
+  s.requirements << "For dev tools:"
+  s.requirements << "gem install buildar flog flay"
+
   s.add_development_dependency "rake", "~> 13.0" # CVE-2020-8130
+  s.add_development_dependency "minitest", "~> 5.0"
+  s.add_development_dependency "buildar", "~> 3.0"
   s.add_development_dependency "flog", "~> 0"
   s.add_development_dependency "flay", "~> 0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: miner_mover
 version: !ruby/object:Gem::Version
-  version: 0.0.0.4
+  version: 0.1.0.1
 platform: ruby
 authors:
 - Rick Hull
@@ -10,20 +10,6 @@ bindir: bin
 cert_chain: []
 date: 1980-01-01 00:00:00.000000000 Z
 dependencies:
-- !ruby/object:Gem::Dependency
-  name: compsci
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.3'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.3'
 - !ruby/object:Gem::Dependency
   name: dotcfg
   requirement: !ruby/object:Gem::Requirement
@@ -39,33 +25,19 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '1.0'
 - !ruby/object:Gem::Dependency
-  name: fiber_scheduler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.13'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '0.13'
-- !ruby/object:Gem::Dependency
-  name: buildar
+  name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '13.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '13.0'
 - !ruby/object:Gem::Dependency
   name: minitest
   requirement: !ruby/object:Gem::Requirement
@@ -81,19 +53,19 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '5.0'
 - !ruby/object:Gem::Dependency
-  name: rake
+  name: buildar
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '13.0'
+        version: '3.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '13.0'
+        version: '3.0'
 - !ruby/object:Gem::Dependency
   name: flog
   requirement: !ruby/object:Gem::Requirement
@@ -139,13 +111,15 @@ files:
 - demo/config.rb
 - demo/fiber.rb
 - demo/fiber_scheduler.rb
-- demo/process.rb
+- demo/process_pipe.rb
+- demo/process_socket.rb
 - demo/ractor.rb
 - demo/serial.rb
 - demo/thread.rb
 - lib/miner_mover.rb
 - lib/miner_mover/config.rb
 - lib/miner_mover/run.rb
+- lib/miner_mover/timer.rb
 - lib/miner_mover/worker.rb
 - miner_mover.gemspec
 - test/miner_mover.rb
@@ -167,7 +141,11 @@ required_rubygems_version: !ruby/object:Gem::Requirement
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
-requirements: []
+requirements:
+- 'For all features and best performance:'
+- gem install rake minitest fiber_scheduler
+- 'For dev tools:'
+- gem install buildar flog flay
 rubygems_version: 3.3.20
 signing_key:
 specification_version: 4