tupelo 0.7 → 0.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 74456cde00385b5340494724ee0affc7cf6e2534
-  data.tar.gz: 331f3970d1fd41a702ada650f5701616de24b184
+  metadata.gz: 1a19d73a0cfead5fa7d4078521ac079bc69b6654
+  data.tar.gz: 24d0e5206aa04688072768f90c630d95bff54636
 SHA512:
-  metadata.gz: bcfe41b674e798ba15761f6740bc74a11963a793538a725ec393fefd4328eb9053e019d95c0f4701cb2ca680dc52c2d2e8dce48921be7cd68532c639b4c61d90
-  data.tar.gz: 82b13b6aa0675069946428033e5d032ca4418409c5901da00cac76cf139d7cc34afa428c4e666e42370b69241c797cabb55b375cfd34e5d8bd5bc83f2e6bf9a9
+  metadata.gz: c06d448f481f6e18b2d944e2961af9fb3007a2e56eaf69b65fa82975023c622abf5beca87142f3e844af0fa1e4e7eba2905b8c6c83b06c4193309d33345fac21
+  data.tar.gz: 97163417cc8366be36adb87b3ee3b1b1cbfcedf007dc817d579782231790c0cdaf0177c4104abde51fd4034c40ff88d2e90ef836442a66944b4c9a00cee7f9d4

data/README.md

@@ -11,12 +11,16 @@ Tupelo differs from other spaces in several ways:
 
 * minimal central computation: just counter increment, message dispatch, and connection management (and it never unpacks serialized tuples)
 
-* clients do all the tuple work: registering and checking waiters, matching, searching, notifying, storing, inserting, deleting, persisting, etc. Each client is free to to decide how to do these things (application code is insulated from this, however). Special-purpose clients may use specialized algorithms and stores for the subspaces they manage.
+* clients do all the tuple work: registering and checking waiters, matching, searching, notifying, storing, inserting, deleting, persisting, etc. Each client is free to to decide how to do these things (application code is insulated from this, however). Special-purpose clients (known as *tuplets*) may use specialized algorithms and stores for the subspaces they manage.
 
-* transactions, in addition to the classic operators.
+* transactions, in addition to the classic operators (and transactions execute client-side, reducing bottleneck and increasing expressiveness).
 
 * replication is inherent in the design (in fact it is unavoidable), for better or worse.
 
+Documentation
+============
+
+* [FAQ](doc/faq.md)
 
 Getting started
 ==========
@@ -58,11 +62,12 @@ Getting started
 
   Pulse without waiting:
 
-        pulse_nowait ...
+        pulse_nowait <tuple>,...
 
   Read tuple matching a template, waiting for a match to exist:
 
         r <template>
+        read <template>
         read_wait <template>
 
   Read tuple matching a template and return it, without waiting for a match to exist (returning nil in that case):
@@ -78,7 +83,7 @@ Getting started
 
         ra Hash
 
-  reads all hash tuples (and ignore array tuples), and
+  reads all hash tuples (and ignores array tuples), and
 
         ra proc {|t| t.size==2}
 
@@ -119,6 +124,22 @@ Getting started
           end
 
   Note that the block may execute more than once, if there is competition for the tuples that you are trying to #take or #read. When the block exits, however, the transaction is final and universally accepted by all clients.
+  
+  You can timeout a transaction:
+  
+        transaction timeout: 1 do
+          read ["does not exist"]
+        end
+
+  This uses tupelo's internal lightweight scheduler, rather than ruby's heavyweight (one thread per timeout) Timeout, though the latter works with tupelo as well.
+  
+  You can also abort a transaction while inside it by calling `#abort` on it:
+
+        write [1]
+        transaction {take [1]; abort}
+        read_all # => [[1]]
+
+  Another thread can abort a transaction in progress (to the extent possible) by calling `#cancel` on it. See [example/cancel.rb](example/cancel.rb).
 
 4. Run tup with a server file so that two sessions can interact. Do this in two terminals in the same dir:
 
@@ -134,7 +155,13 @@ Getting started
 
   Note that all bin and example programs accept blob type (e.g., --msgpack, --json) on command line (it only needs to be specified for server -- the clients discover it). Also, all these programs accept log level on command line. The default is --warn. The --info level is a good way to get an idea of what is happening, without the verbosity of --debug.
 
-6. Debugging: in addition to the --info switch on all bin and example programs, bin/tspy is also really useful. There is also the similar --trace switch that is available to all bin and example programs. This switch diagnostic output for each transaction. For example:
+6. Debugging: in addition to the --info switch on all bin and example programs, bin/tspy is also really useful; it shows all tuplespace events in sequence that they occur. For example, run
+
+        $ tspy svr
+
+  in another terminal after running `tup svr`. The output shows the clock tick, sending client, operation, and operation status (success or failure).
+
+  There is also the similar --trace switch that is available to all bin and example programs. This turns on diagnostic output for each transaction. For example:
 
   ```
     tick    cid status operation
@@ -143,7 +170,7 @@ Getting started
        3      3        atomic take ["x", 1], ["y", 2]
   ```
 
-  The `Tupelo.application` command, provided by `tupelo/app`, is the source of all these options and is available to your programs. It's a kind of lightweight process deployment and control framework; however it is not necessary to use tupelo.
+  The `Tupelo.application` command, provided by `tupelo/app`, is the source of all these options and is available to your programs. It's a kind of lightweight process deployment and control framework; however `Tupelo.application` is not necessary to use tupelo.
 
 
 What is a tuplespace?
@@ -151,7 +178,7 @@ What is a tuplespace?
 
 A tuplespace is a service for coordination, configuration, and control of concurrent and distributed systems. The model it provides to processes is a shared space that they can use to communicate in a deterministic and sequential manner. (Deterministic in that all clients see the same, consistent view of the data.) The space contains tuples. The operations on the space are few, but powerful. It's not a database, but it might be a front-end for one or more databases.
 
-See https://en.wikipedia.org/wiki/Tuple_space for general information and history. This project is strongly influenced by Masatoshi Seki's Rinda implementation, part of the Ruby standard library.
+See https://en.wikipedia.org/wiki/Tuple_space for general information and history. This project is strongly influenced by Masatoshi Seki's Rinda implementation, part of the Ruby standard library. See http://pragprog.com/book/sidruby/the-druby-book for a good introduction to rinda and druby.
 
 What is a tuple?
 ----------------
@@ -183,6 +210,12 @@ In other words, a tuple is a fairly general object, though this depends on the s
 
 It's kind of like a "JSON object", except that, when using the json serializer, the hash keys can only be strings. In the msgpack case, keys have no special limitations. In the case of the marshal and yaml modes, tuples can contain many other kinds of objects.
 
+One other thing to keep in mind: in the array case, the order of the elements is significant. In the hash case, the order is not significant. So these are both true:
+
+    [1,2] != [2,1]
+    {a:1, b:2} == {b:2, a:1}
+
+
 What is a template?
 -------------------
 
@@ -255,9 +288,11 @@ If you prefer classical tuplespace locking, you can simply use certain tuples as
 
 If an optimistic transaction fails (for example, it is trying to take a tuple, but the tuple has just been taken by another transaction), then the transaction block is re-executed, possibly waiting for new matches to the templates. Application code must be aware of the possible re-execution of the block. This is better explained in the examples...
 
-Transactions have a significant disadvantage compared to lock tuples: a transaction can protect only resources that are represented in the tuplespace, whereas a lock can protect anything: a file, a device, a service, etc. This is because a transaction begins and ends within a single instant of logical (tuplespace) time, whereas a lock tuple can be taken out for an arbitrary duration of real time. Furthermore, the instant of logical time in which a transaction takes effect may occur at different wall-clock times on different processes, even on the same host.
+Transactions have a significant disadvantage compared to using take/write to lock/unlock tuples: a transaction can protect only resources that are represented in the tuplespace, whereas a lock can protect anything: a file, a device, a service, etc. This is because a transaction begins and ends within a single instant of logical (tuplespace) time, whereas a lock tuple can be taken out for an arbitrary duration of real (and logical) time. Furthermore, the instant of logical time in which a transaction takes effect may occur at different wall-clock times on different processes, even on the same host.
 
-Tupelo transactions are ACID in the following sense. They are Atomic and Isolated -- this is enforced by the transaction processing in each client. Consistency is enforced by the underlying message sequencer: each client's copy of the space is the deterministic result of the same sequence of operations. Durability is optional, but can be provided by the archiver (to be implemented) or other clients.
+Transactions do have an advantage over using take/write to lock/unlock tuples: there is no possibility of deadlock. See [example/deadlock.rb](example/deadlock.rb) and [example/parallel.rb](example/parallel.rb).
+
+Tupelo transactions are ACID in the following sense. They are Atomic and Isolated -- this is enforced by the transaction processing in each client. Consistency is enforced by the underlying message sequencer: each client's copy of the space is the deterministic result of the same sequence of operations. Durability is optional, but can be provided by the persistent archiver or other clients.
 
 On the CAP spectrum, tupelo tends towards consistency: for all clients, write and take operations are applied in the same order, so the state of the entire system up through a given tick of discrete time is universally agreed upon. Of course, because of the difficulties of distributed systems, one client may not yet have seen the same range of ticks as another.
 
@@ -318,7 +353,7 @@ Future
 
 - Investigate nio4r for faster networking, especially with many clients.
 
-- Interoperable client and server implementations in C, Python, Go, ....
+- Interoperable client and server implementations in C, Python, Go, .... Elixir?
 
 - UDP multicast to further reduce the bottleneck in the message sequencer.
 
@@ -363,11 +398,13 @@ To compare
 
 * resque
 
-* zookeeper -- totally ordered updates
+* zookeeper -- totally ordered updates; tupelo trades availability for lower latency (?)
 
 * chubby
 
-* doozer
+* doozer, etcd
+
+* arakoon
 
 * hazelcast
 
@@ -375,12 +412,17 @@ To compare
 
 * datomic -- similar distribution of "facts", but not tuplespace; similar use of pluggable storage managers
 
-* job queues: sidekiq, resque, delayedjob, http://queues.io
+* job queues: sidekiq, resque, delayedjob, http://queues.io, https://github.com/factual/skuld
 
 * pubsubs: kafka
 
 * spark, storm
 
+* tibco and gigaspace
+
+* gridgain
+
+
 Architecture
 ============
 

data/bin/tup

@@ -47,6 +47,8 @@ if ARGV.delete("-h") or ARGV.delete("--help")
       
       -v          verbose mode (include time and pid in log messages)
       
+      --trace     enable trace output
+      
       --pubsub    publish/subscribe mode; does not keep local tuple store:
       
                     * read only works in blocking mode (waiting for new tuple)
@@ -57,107 +59,53 @@ if ARGV.delete("-h") or ARGV.delete("--help")
       --yaml
       --json
       --msgpack   <-- default
+      
+      --persist-dir DIR
+                  load and save tuplespace to DIR
 
   END
   exit
 end
 
-require 'easy-serve'
+require 'tupelo/app'
 
-log_level = case
-  when ARGV.delete("--debug"); Logger::DEBUG
-  when ARGV.delete("--info");  Logger::INFO
-  when ARGV.delete("--warn");  Logger::WARN
-  when ARGV.delete("--error"); Logger::ERROR
-  when ARGV.delete("--fatal"); Logger::FATAL
-  else Logger::WARN
-end
-verbose = ARGV.delete("-v")
-pubsub = ARGV.delete("--pubsub")
+argv, tupelo_opts = Tupelo.parse_args(ARGV)
 
-blob_type = nil
-%w{--marshal --yaml --json --msgpack}.each do |switch|
-  s = ARGV.delete(switch) and
-    blob_type ||= s.delete("--")
-end
+pubsub = argv.delete("--pubsub") # not a standard tupelo opt
 
-ez_opts = {
-  servers_file: ARGV.shift,
-  interactive: $stdin.isatty
-}
+servers_file = argv.shift
+addr = argv.shift(3)
 
-addr = ARGV.shift(3)
+Tupelo.application(
+  argv: argv,
+  **tupelo_opts,
+  servers_file: servers_file,
+  seqd_addr: addr,
+  cseqd_addr: addr, # using same addr causes autoincrement of port/filename
+  arcd_addr: addr) do
 
-EasyServe.start ez_opts do |ez|
-  log = ez.log
-  log.level = log_level
-  log.formatter = nil if verbose
-  log.progname = File.basename($0)
-    
-  ez.start_servers do
-    arc_to_seq_sock, seq_to_arc_sock = UNIXSocket.pair
-    arc_to_cseq_sock, cseq_to_arc_sock = UNIXSocket.pair
-    
-    ez.server :seqd, *addr do |svr|
-      require 'funl/message-sequencer'
-      seq_opts = {}
-      seq_opts[:blob_type] = blob_type if blob_type
-      seq = Funl::MessageSequencer.new svr, seq_to_arc_sock, log: log,
-        **seq_opts
-      seq.start ## thwait? or can easy-serve do that?
-    end
-    
-    ez.server :cseqd, *addr do |svr|
-      require 'funl/client-sequencer'
-      cseq = Funl::ClientSequencer.new svr, cseq_to_arc_sock, log: log
-      cseq.start
-    end
-
-    ez.server :arcd, *addr do |svr|
-      require 'tupelo/archiver'
-      arc = Tupelo::Archiver.new svr, seq: arc_to_seq_sock,
-              cseq: arc_to_cseq_sock, log: log
-      arc.start
-    end
+  class TupClient < Tupelo::Client
+    alias w write_wait
+    alias pl pulse_wait
+    alias t take
+    alias r read_wait
+    alias ra read_all
+    alias tr transaction
+    CMD_ALIASES = %w{ w pl t r ra tr }
+    private *CMD_ALIASES
   end
-  
-  ez.local :seqd, :cseqd, :arcd do |seqd, cseqd, arcd|
-    log.progname = "client <starting in #{log.progname}>"
-
-    require 'tupelo/client'
-    class TupClient < Tupelo::Client
-      alias w write_wait
-      alias pl pulse_wait
-      alias t take
-      alias r read_wait
-      alias ra read_all
-      alias tr transaction
-      CMD_ALIASES = %w{ w pl t r ra tr }
-      private *CMD_ALIASES
-    end
-
-    client_opts = {seq: seqd, cseq: cseqd, log: log}
-    if pubsub
-      client_opts[:arc] = nil
-      client_opts[:tuplespace] = TupClient::NullTuplespace
-    else
-      client_opts[:arc] = arcd
-    end
-
-    client = TupClient.new client_opts
-    client.start do
-      log.progname = "client #{client.client_id}"
-    end
-    log.info {
-      "cpu time: %.2fs" % Process.times.inject {|s,x|s+x}
-    }
-    log.info {
-      "starting shell. Commands: #{TupClient::CMD_ALIASES.join(", ")}"
-    }
 
-    require 'tupelo/app/irb-shell'
-    IRB.start_session(client)
+  client_opts = {}
+  if pubsub
+    client_opts[:arc] = nil
+    client_opts[:tuplespace] = TupClient::NullTuplespace
+  end
 
-    client.stop
+  local TupClient, **client_opts do
+    log.info {"cpu time: %.2fs" % Process.times.inject {|s,x|s+x}}
+    log.info {"starting shell. Commands: #{TupClient::CMD_ALIASES.join(", ")}"}
+
+    require 'tupelo/app/irb-shell'
+    IRB.start_session(self)
   end
 end

data/example/child-of-child.rb

@@ -0,0 +1,34 @@
+require 'tupelo/app'
+
+### need a programmatic way to start up clients
+
+Tupelo.application do |app|
+
+  app.child do ## local still hangs
+    3.times do |i|
+      app.child do
+        write [i]
+        log "wrote #{i}"
+      end
+    end
+
+    3.times do
+      log take [nil]
+    end
+  end
+end
+
+__END__
+
+this hangs sometimes but not always:
+
+  tick    cid status operation
+A: client 3: wrote 0
+A: client 4: wrote 1
+     1      3        batch write [0]
+     2      4        batch write [1]
+A: client 2: [0]
+     3      2        atomic take [0]
+     4      2        atomic take [1]
+A: client 2: [1]
+A: client 5: wrote 2

data/example/deadlock.rb

@@ -0,0 +1,66 @@
+# See the README reference to this file.
+# Run with --trace to see what's happening.
+
+require 'tupelo/app'
+
+def observe_deadlock
+  done = false
+  at_exit do
+       # for a passive client, exit is forced when there are no
+       # more non-passive clients
+    if done
+      log "done (should not happen)"
+    else
+      log "stopped in deadlock (as expected)"
+    end
+  end
+
+  yield  
+
+  done = true
+end
+
+Tupelo.application do
+  local do
+    write [1], [2], [3], [4]
+  end
+  
+  child passive: true do
+    observe_deadlock do
+      take [1]
+      sleep 1
+      take [2]
+      write [1], [2]
+    end
+  end
+
+  child passive: true do
+    observe_deadlock do
+      sleep 0.5
+      take [2]
+      take [1]
+      write [1], [2]
+    end
+  end
+
+  child do
+    transaction do
+      take [3]
+      sleep 1
+      take [4]
+      write [3], [4]
+      log "done"
+    end
+  end
+
+  child do
+    transaction do
+      sleep 0.5
+      take [4]
+      take [3]
+      write [3], [4]
+      log "done"
+    end
+  end
+
+end

data/example/lease.rb

@@ -0,0 +1,103 @@
+require 'tupelo/app'
+
+N_WORKERS = 3
+N_TASKS = 10
+N_SLEEPS = 2
+
+Tupelo.application do
+  N_WORKERS.times do |w_i|
+    child passive: true do
+      loop do
+        task_id = task_data = nil
+        
+        transaction do
+          _, task_id, task_data = take ["task", nil, nil]
+          write ["lease", client_id, task_id, task_data]
+          write ["alive", client_id, task_id, (Time.now + 1).to_f]
+        end
+
+        N_SLEEPS.times do
+          sleep 1 # pretend to be working
+          write ["alive", client_id, task_id, (Time.now + 1).to_f]
+
+          # randomly exit or oversleep the lease deadline
+          if w_i == 1
+            log "bad worker exiting"
+            exit
+          elsif w_i == 2
+            log "bad worker oversleeping"
+            sleep 3
+          end
+        end
+
+        result = task_data * 1000
+
+        transaction do
+          if take_nowait ["lease", client_id, task_id, nil]
+            write ["result", task_id, result]
+              # write the result only if this client still has lease --
+              # otherwise, some other client has been assigned to this task.
+          else
+            log.warn "I lost my lease because I didn't finish task in time!"
+          end
+        end
+      end
+    end
+  end
+  
+  # Lease manager. Ensures that, for each input tuple ["task", i, ...],
+  # there is exactly one output tuple ["result", i, ...]. It does not
+  # attempt to stop / start processes. So it can fail if all the workers die,
+  # or if the lease manager itself dies. But it will succeed if it and at least
+  # one worker lives. This demonstrates how to recover from worker failure
+  # and prevent "lost tuples".
+  child passive: true do
+    require 'tupelo/client/atdo'
+
+    scheduler = make_scheduler
+    alive_until = Hash.new(0)
+
+    loop do
+      _, lease_client_id, task_id, time = take ["alive", nil, nil, nil]
+      t = alive_until[[lease_client_id, task_id]]
+      alive_until[[lease_client_id, task_id]] = [t, time].max
+
+      scheduler.at time + 0.2 do # allow for network latency etc.
+        t = alive_until[[lease_client_id, task_id]]
+        if t < Time.now.to_f # expired
+          task_data = nil
+          transaction do
+            _,_,_,task_data =
+              take_nowait ["lease", lease_client_id, task_id, nil]
+              # if lease is gone, ok!
+            if task_data
+              write ["task", task_id, task_data] # for someone else to work on
+            end
+          end
+          if task_data
+            log.warn "took lease from #{lease_client_id} on #{task_id}"
+          end
+        end
+      end
+    end
+  end
+  
+  # Task requestor.
+  child do
+    N_TASKS.times do |task_id|
+      task_data = task_id # for simplicity
+      write ["task", task_id, task_data]
+    end
+
+    N_TASKS.times do |task_id|
+      log take ["result", task_id, nil]
+    end
+    
+    extra_results = read_all ["result", nil, nil]
+    if extra_results.empty?
+      log "results look ok!"
+    else
+      log.error "extra results = #{extra_results}"
+    end
+  end
+end

data/example/parallel.rb

@@ -1 +1,100 @@
-# like gnu parallel
+# a bit like gnu parallel
+# see also https://github.com/grosser/parallel
+
+require 'tupelo/app/remote'
+
+show_steps = !!ARGV.delete("--show-steps")
+
+hosts = ARGV.shift
+map = ARGV.slice!(0,3)
+reduce = ARGV.slice!(0,4)
+
+abort <<END unless hosts and
+  map[0] == "map" and reduce[0] == "reduce" and reduce[3]
+
+  usage: #$0 <ssh-host>,... map <var> <expr> reduce <var> <var> <expr> [<infile> ...]
+  
+  Input can be provided on standard input or as the contents of the files
+  specified in the infile arguments. Writes the result of the last
+  reduction to standard output.
+  
+  If --show-steps is set then intermediate reductions are printed as they
+  are computed. If input is stdin at the terminal, then you can see these
+  outputs even before you type the EOF character.
+  
+  Caution: very little argument checking!
+  Caution: no robustness guarantees (but see comments)!
+  
+  Example:
+  
+    ruby #$0 localhost,localhost map s s.length reduce l1 l2 l1+l2
+  
+  Use `s.split.length` to get word count instead of char count.
+
+END
+
+hosts = hosts.split(",")
+
+map_str = <<END
+  proc do |#{map[1]}|
+    #{map[2]}
+  end
+END
+
+reducer = eval <<END
+  proc do |#{reduce[1]}, #{reduce[2]}|
+    #{reduce[3]}
+  end
+END
+
+Tupelo.tcp_application do
+  hosts.each do |host|
+    remote host: host, passive: true, log: true, eval: %{
+      mapper = #{map_str}
+      loop do
+        s = take(line: String)["line"]
+        output = mapper[s]
+        log(mapped: output) if #{show_steps}
+        write output: output
+      end
+    }
+  end
+  
+  child passive: true do
+    loop do
+      m1, m2 = transaction do # transaction avoids deadlock!
+        [take(output: nil)["output"],
+         take(output: nil)["output"]]
+      end
+
+      # Fragile! A crash after the transaction above means the whole app
+      # can't finish. You could fix this with lease tuples--see lease.rb.
+
+      output = reducer[m1, m2]
+      log reduced: output if show_steps
+      
+      transaction do
+        count = take(count: nil)["count"]
+        write count: count - 1
+        write output: output
+      end
+    end
+  end
+
+  local do
+    write count: 0
+
+    ARGF.each do |line|
+      transaction do
+        write line: line.chomp
+        count = take(count: nil)["count"]
+        write count: count + 1
+      end
+    end
+    
+    read count: 1
+    result = take output: nil
+    log result if show_steps
+    puts result["output"]
+  end
+end

data/example/remote-map-reduce.rb

@@ -1,3 +1,5 @@
+# see also parallel.rb
+
 require 'tupelo/app/remote'
 
 hosts = ARGV.shift or abort "usage: #$0 <ssh-hostname>,<ssh-hostname>,..."
@@ -22,6 +24,5 @@ Tupelo.tcp_application do
       sum += take([Numeric])[0]
     end
     log "sum = #{sum}, correct sum = #{input.flatten.join.size}"
-    sleep 2
   end
 end