tupelo 0.11 → 0.12

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: be780826d017e0ea442fa49d2a6403cc44cfe263
-  data.tar.gz: 133fa6c3218dd1fc4b06a8ea407a35ca3bdea0ac
+  metadata.gz: dc4bce707ea05097c67e685d20435968d326daaf
+  data.tar.gz: 02a62f5ed3513d411b0e3bbd4b6d21af08b06c58
 SHA512:
-  metadata.gz: 3327df2fc6fd2587dd244b99a4ed64827aeafd3314f7d3f34360e69f5598f294cd1c31010f8729c7ba913f1d1f489ce624735090074976de5aca19f88db5a646
-  data.tar.gz: 9b8c852daa50910f9c0823601a85618000404259893de1bc187393b8f0faeb99321ed06cbf78fb7ad488e9725326efa43202f201ae81c5a6d1e832cb9660cd3b
+  metadata.gz: 843a155e6e411c6ff9da110b83cfae4c69fa53b06abd0b34b0aa4f469bd97447e58e08091a3009b49d914615547dee9c5f3f3a84e1e0c89481b6184c8f429581
+  data.tar.gz: 74fd680cd05b27977afc4958eeea19311dcf11120e1d3c2232371e8abda0f275ffc3f9204cb3197412cbb08664c9fc37a789b43c7341a716b00a1f0fc8261fe3

data/README.md

@@ -1,3 +1,5 @@
+**NEWS**: Come hear a talk on Tupelo on December 11 in San Francisco at the [SF Distributed Computing meetup](http://www.meetup.com/San-Francisco-Distributed-Computing). Location is TBD. Abstract: [doc/sfdc.txt](doc/sfdc.txt).
+
 tupelo
 ==
 
@@ -308,7 +310,14 @@ Transactions have a significant disadvantage compared to using take/write to loc
 
 Transactions do have an advantage over using take/write to lock/unlock tuples: there is no possibility of deadlock. See [example/deadlock.rb](example/deadlock.rb) and [example/parallel.rb](example/parallel.rb).
 
-Another advantage of tranactions is that it is possible to guarantee continuous existence of a time-series of tuples. For example, suppose that tuples matching `{step: Numeric}` indicate the progress of some activity. With transactions, you can guarantee that there is exactly one matching tuple at any time. So any client which reads this template will find a match without blocking.
+Another advantage of tranactions is that it is possible to guarantee continuous existence of a time-series of tuples. For example, suppose that tuples matching `{step: Numeric}` indicate the progress of some activity. With transactions, you can guarantee that there is exactly one matching tuple at any time, and that no client ever sees in intermediate or inconsistent state of the counter:
+
+    transaction do
+      step = take(step: nil)["step"]
+      write step: step + 1
+    end
+
+Any client which reads this template will find a (unique) match without blocking.
 
 Another use of transactions: forcing a retry when something changes:
 
@@ -479,6 +488,14 @@ General application clients:
 
 * client threads construct transactions and wait for results (communicating with the worker thread over queues); they may also use asynchronous transactions
 
+Some design principles:
+
+* Once a transaction has been sent from a client to the message sequencer, it references only tuples, not templates. This makes it faster and simpler for each receiving client to apply or reject the transaction. Also, clients that do not support local template searching (such as archivers) can store tuples using especially efficient data structures that only support tuple-insert, tuple-delete, and iterate/export operations.
+
+* Use non-blocking protocols. For example, transactions can be evaluated in one client without waiting for information from other clients. Even at the level of reading messages over sockets, tupelo uses (via funl and object-stream) non-blocking constructs. At the application level, you can use transactions to optimistically modify shared state (but applications are free to use locking if high contention demands it).
+
+* Do the hard work on the client side. For example, all pattern matching happens in the client that requested an operation that has a template argument, not on the server or other clients.
+
 Protocol
 --------
 

data/bin/tspy

@@ -28,18 +28,21 @@ if ARGV.delete("-h") or ARGV.delete("--help")
 end
 
 require 'tupelo/app'
+require 'tupelo/archiver/tuplespace'
 
-Tupelo.application do |app|
-  app.local do |client|
+Tupelo.application do
+  # Use hash-and-count-based storage, for efficiency (this client never
+  # does take or read).
+  local tuplespace: [Tupelo::Archiver::Tuplespace, zero_tolerance: 1000] do
     trap :INT do
       exit!
     end
 
-    note = client.notifier
-    client.log "%10s %10s %10s %s" % %w{ tick client status operation }
+    note = notifier
+    log "%10s %10s %10s %s" % %w{ tick client status operation }
     loop do
       status, tick, cid, op = note.wait
-      client.log "%10d %10d %10s %p" % [tick, cid, status, op]
+      log "%10d %10d %10s %p" % [tick, cid, status, op]
     end
   end
 end

data/example/chat/chat.rb

@@ -0,0 +1,62 @@
+# Accepts usual tupelo switches (such as --trace, --debug), plus one argument: a
+# user name to be shared with other chat clients. New clients see a brief
+# history of the chat, as well as new messages from other clients.
+#
+# You can run several instances of chat.rb. The first will set up all needed
+# services. The rest will connect by referring to a yaml file in the same dir.
+# Copy that file to remote hosts (and modify hostnames as needed) for remote
+# access. If the first instance is run with "--persist-dir <dir>", messages
+# will persist across service shutdown.
+#
+# Compare: https://github.com/bloom-lang/bud/blob/master/examples/chat.
+#
+# To do: use a subspace with a sorted data structure, like rbtree or in-memory
+# sqlite, for the messages.
+
+require 'tupelo/app'
+
+svr = "chat.yaml"
+history_period = 60 # seconds -- discard _my_ messages older than this
+
+Thread.abort_on_exception = true
+
+def display_message msg
+  from, time, line = msg.values_at(*%w{from time line})
+  time_str = Time.at(time).strftime("%I:%M.%S")
+  puts "#{from}@#{time_str}> #{line}"
+end
+
+Tupelo.tcp_application servers_file: svr do
+  me = argv.shift
+
+  local do
+    require 'readline'
+
+    Thread.new do
+      seen_at_start = {}
+      read_all(from: nil, line: nil, time: nil).
+        sort_by {|msg| msg["time"]}.
+        each {|msg| display_message msg; seen_at_start[msg] = true}
+      
+      read from: nil, line: nil, time: nil do |msg|
+        next if msg["from"] == me or seen_at_start[msg]
+        print "\r"; display_message msg
+        Readline.redisplay ### why not u work?
+      end
+    end
+    
+    Thread.new do
+      loop do
+        begin
+          t = Time.now.to_f - history_period
+          take({from: me, line: nil, time: 0..t}, timeout: 10)
+        rescue TimeoutError
+        end
+      end
+    end
+
+    while line = Readline.readline("#{me}> ", true)
+      write from: me, line: line, time: Time.now.to_f
+    end
+  end
+end

data/example/fish.rb

@@ -0,0 +1,44 @@
+require 'tupelo/app'
+
+Tupelo.application do
+  2.times do
+    child passive: true do # these could be threads in the next child
+      loop do
+        transaction do
+          fish, = take [String]
+          write [1, fish]
+        end
+      end
+    end
+  end
+
+  2.times do
+    child passive: true do
+      loop do
+        transaction do
+          n1, fish  = take([Integer, String]) ## need to iterate on this search
+          n2, _     = take([Integer, fish])   ## if this take blocks
+          write [n1 + n2, fish]
+        end
+      end
+    end
+  end
+
+  local do
+    seed = 3
+    srand seed
+    log "seed = #{seed}"
+    
+    fishes = %w{ trout marlin char salmon }
+    a = fishes * 10
+    a.shuffle!
+    a.each do |fish|
+      write [fish]
+      sleep rand % 0.1
+    end
+
+    fishes.each do |fish|
+      log take [10, fish]
+    end
+  end
+end

data/example/fish0.rb

@@ -0,0 +1,39 @@
+# This works, but requires pre-initialization of all counters.
+
+require 'tupelo/app'
+
+Tupelo.application do
+  2.times do
+    child passive: true do
+      loop do
+        transaction do
+          fish, _ = take([String])
+          n, _ = take([Integer, fish])
+          write [n + 1, fish]
+        end
+      end
+    end
+  end
+
+  local do
+    seed = 3
+    srand seed
+    log "seed = #{seed}"
+    
+    fishes = %w{ trout marlin char salmon }
+    fishes.each do |fish|
+      write [0, fish]
+    end
+
+    a = fishes * 10
+    a.shuffle!
+    a.each do |fish|
+      write [fish]
+      sleep rand % 0.1
+    end
+
+    fishes.each do |fish|
+      log take [10, fish]
+    end
+  end
+end

data/example/multi-tier/multi-sinatras.rb

@@ -0,0 +1,85 @@
+# A tupelo cluster may expose its services using some other protocol so that
+# process need not be tupelo-aware to use them. This example expands on the
+# http.rb example and shows how to use tupelo to coordinate multiple sinatra
+# instances.
+#
+# Depends on the sinatra and http gems.
+
+PORTS = [9001, 9002 , 9003]
+
+fork do
+  require 'tupelo/app'
+  
+  Tupelo.application do
+    PORTS.each do |port|
+      child do |client|
+        require 'sinatra/base'
+
+        Class.new(Sinatra::Base).class_eval do
+          get '/' do
+            "hello, world\n"
+          end
+
+          post '/send' do
+            text = params["text"]
+            dest = params["dest"]
+            client.write ["message", dest, text]
+              ## should use subspaces and a data structure that keeps
+              ## messages in order
+          end
+          
+          get '/recv' do
+            dest = params["dest"]
+            _, _, text = client.take ["message", dest, String]
+            text
+          end
+
+          get '/exit' do
+            Thread.new {sleep 1; exit}
+            "bye\n"
+          end
+
+          set :port, port
+          run!
+        end
+      end
+    end
+  end
+end
+
+fork do # No tupelo in this process.
+  require 'http'
+
+  # For simplicity, one http client per http server
+  http_clients = PORTS.map.with_index do |port, i|
+    {
+      server_url: "http://localhost:#{port}",
+      id:         i
+    }
+  end
+
+  http_clients.each_with_index do |http_client|
+    fork do
+      url = http_client[:server_url]
+      print "trying server at #{url}"
+      begin
+        print "."
+        HTTP.get url
+      rescue Errno::ECONNREFUSED
+        sleep 0.2
+        retry
+      end
+
+      other = (http_client[:id] + 1) % http_clients.size
+      me = http_client[:id]
+
+      puts
+      HTTP.post "#{url}/send?dest=#{other}&text=hello_from_#{me}"
+      text = HTTP.get "#{url}/recv?dest=#{me}"
+      puts "http client #{me} got: #{text}\n"
+      HTTP.get "#{url}/exit"
+    end
+  end
+end
+
+Process.waitall

data/example/pregel/distributed.rb

@@ -1,4 +1,5 @@
-# Distributed version of pagerank.rb.
+# Distributed version of pagerank.rb. This uses only a single host. To use
+# many hosts, see remote.rb.
 
 # TODO
 #
@@ -7,42 +8,16 @@
 # And the subspaces could be defined by consistent hashing and smarter
 # partitioning.
 # Also, need to handle crashed process and lost tuple (as lease.rb maybe).
-# Would be nice to have remote option as well.
 # Abstract out domain-specific code from generic framework code.
 # Option to compare result with that of pagerank.rb using same seed.
 
 require 'tupelo/app'
+require_relative 'update'
 
 NUM_WORKERS = 4
 NUM_VERTICES = 10
 PRNG_SEED = 1234
 
-def update vertex, incoming_messages, vs_dst
-  vertex = vertex.dup
-  incoming_messages ||= []
-  outgoing_messages = []
-  v_me = vertex["id"]
-  rank = vertex["rank"]
-  step = vertex["step"]
-  active = true
-
-  if step < 50
-    rank = 0.15 / NUM_VERTICES + 0.85 * incoming_messages.inject(0.0) {|sum, m|
-      sum + m["rank"]}
-    outgoing_rank = rank / vs_dst.size
-    outgoing_messages = vs_dst.map {|v_dst|
-      {src: v_me, dst: v_dst, step: step + 1, rank: outgoing_rank}}
-  else
-    active = false
-  end
-
-  vertex["rank"] = rank
-  vertex["active"] = active
-  vertex["step"] += 1
-
-  [vertex, outgoing_messages]
-end
-
 Tupelo.application do
   
   NUM_WORKERS.times do |i|

data/example/pregel/remote.rb

@@ -2,61 +2,29 @@
 
 # TODO
 #
-# Improvements noted in the article.
-# Scale better with subspaces and sqlite or other data structure.
-# And the subspaces could be defined by consistent hashing and smarter
-# partitioning.
-# Also, need to handle crashed process and lost tuple (as lease.rb maybe).
-# Would be nice to have remote option as well.
-# Abstract out domain-specific code from generic framework code.
-# Option to compare result with that of pagerank.rb using same seed.
+# Improvements listed in pagerank.rb.
 
 require 'tupelo/app'
+require 'tupelo/app/remote'
+require_relative 'update'
 
-NUM_WORKERS = 4
+NUM_WORKERS = 8
 NUM_VERTICES = 10
 PRNG_SEED = 1234
 
-def update vertex, incoming_messages, vs_dst
-  vertex = vertex.dup
-  incoming_messages ||= []
-  outgoing_messages = []
-  v_me = vertex["id"]
-  rank = vertex["rank"]
-  step = vertex["step"]
-  active = true
-
-  if step < 50
-    rank = 0.15 / NUM_VERTICES + 0.85 * incoming_messages.inject(0.0) {|sum, m|
-      sum + m["rank"]}
-    outgoing_rank = rank / vs_dst.size
-    outgoing_messages = vs_dst.map {|v_dst|
-      {src: v_me, dst: v_dst, step: step + 1, rank: outgoing_rank}}
-  else
-    active = false
-  end
-
-  vertex["rank"] = rank
-  vertex["active"] = active
-  vertex["step"] += 1
-
-  [vertex, outgoing_messages]
-end
-
-require 'tupelo/app/remote'
 def host i
-  case i % 2
-  when 0; "od1"
-  when 1; "od2"
+  if ARGV.empty?
+    abort "Usage: #{$0} [usual opts] host1 host2 host3 ..."
   end
+  ARGV[i % ARGV.size]
 end
 
 Tupelo.tcp_application do
-  
   NUM_WORKERS.times do |i|
-#    child passive: true do
+    ## need better support from EasyServe:
+    ##  remote: ..., load: 'update.rb' OR rsync option and then require
     remote host: host(i), log: true, passive: true, eval: %{
-    log "hello"
+      log "starting worker #{i} on host #{host(i)}"
       def update vertex, incoming_messages, vs_dst
         vertex = vertex.dup
         incoming_messages ||= []

data/example/pregel/update.rb

@@ -0,0 +1,25 @@
+def update vertex, incoming_messages, vs_dst
+  vertex = vertex.dup
+  incoming_messages ||= []
+  outgoing_messages = []
+  v_me = vertex["id"]
+  rank = vertex["rank"]
+  step = vertex["step"]
+  active = true
+
+  if step < 50
+    rank = 0.15 / NUM_VERTICES + 0.85 * incoming_messages.inject(0.0) {|sum, m|
+      sum + m["rank"]}
+    outgoing_rank = rank / vs_dst.size
+    outgoing_messages = vs_dst.map {|v_dst|
+      {src: v_me, dst: v_dst, step: step + 1, rank: outgoing_rank}}
+  else
+    active = false
+  end
+
+  vertex["rank"] = rank
+  vertex["active"] = active
+  vertex["step"] += 1
+
+  [vertex, outgoing_messages]
+end

data/example/socket-broker.rb

@@ -0,0 +1,42 @@
+# Tuples are not well suited to streaming data or very large data, but they
+# can be used to coordinate access to such data.
+
+require 'tupelo/app'
+
+Tupelo.application do
+  child passive: true do
+    loop do
+      _, key = take ["session-request", nil]
+
+      serv = TCPServer.new 0
+      host = serv.addr[2]
+      port = serv.addr[1]
+
+      fork do
+        sock = serv.accept
+        sock.send "lots of data at #{Time.now}", 0
+        sleep 1
+        sock.send "lots more data at #{Time.now}", 0
+      end
+
+      write_wait ["session-response", key, host, port]
+    end
+  end
+  
+  2.times do
+    child do
+      key = client_id
+        # use client_id here just because we know it is unique to this client
+
+      write ["session-request", key]
+      _, _, host, port = take ["session-response", key, nil, nil]
+
+      sock = TCPSocket.new host, port
+      loop do
+        msg = sock.recv(1000)
+        break if msg.size == 0
+        log msg
+      end
+    end
+  end
+end

data/example/subspaces/shop/shop-v1.rb

@@ -0,0 +1,53 @@
+# The shop has products, customers, and shopping carts. Customers move
+# products to their carts, and the app has to prevent two customers getting
+# the same instance of a product.
+#
+# It works, but every process has to handle every transaction. Not good. See
+# shop-v2.rb.
+
+require 'tupelo/app'
+
+PRODUCT_IDS = 1..10
+CUSTOMER_IDS = 1..10
+
+Tupelo.application do
+  local do
+    PRODUCT_IDS.each do |product_id|
+      count = 10
+      write ["product", product_id, count]
+    end
+  end
+  
+  CUSTOMER_IDS.each do |customer_id|
+    child passive: true do
+      loop do
+        sleep rand % 0.1
+        transaction do
+          # buy the first product we see:
+          _, product_id, count = take ["product", nil, 1..Float::INFINITY]
+          write ["product", product_id, count-1]
+          write ["cart", customer_id, product_id]
+        end
+      end
+    end
+  end
+  
+  local do
+    PRODUCT_IDS.each do |product_id|
+      read ["product", product_id, 0] # wait until sold out
+    end
+    
+    CUSTOMER_IDS.each do |customer_id|
+      h = Hash.new(0)
+      transaction do
+        while t=take_nowait(["cart", customer_id, nil])
+          h[t[2]] += 1
+        end
+      end
+      puts "Customer #{customer_id} bought:"
+      h.keys.sort.each do |product_id|
+        printf "%10d of product %3d.\n", h[product_id], product_id
+      end
+    end
+  end
+end

data/example/subspaces/shop/shop-v2.rb

@@ -0,0 +1,66 @@
+# Makes the shop a bit more efficient using subspaces. Each customer needs to
+# subscribe only to the inventory tuples, not to all the tuples in customer
+# carts. Run with --trace to see tuples assigned to subspaces and also note how
+# much contention there is.
+
+require 'tupelo/app'
+
+PRODUCT_IDS = 1..10
+CUSTOMER_IDS = 1..10
+
+Tupelo.application do
+  local do
+    use_subspaces!
+
+    define_subspace(
+      tag:          "inventory",
+      template:     [
+        {value: "product"},
+        nil,                  # product_id
+        {type:  "number"}     # count
+      ]
+    )
+
+    PRODUCT_IDS.each do |product_id|
+      count = 10
+      write ["product", product_id, count]
+    end
+  end
+
+  CUSTOMER_IDS.each do |customer_id|
+    child subscribe: "inventory", passive: true do
+      loop do
+        sleep rand % 0.1
+        transaction do
+          # buy the first product we see:
+          _, product_id, count = take ["product", nil, 1..Float::INFINITY]
+          write ["product", product_id, count-1]
+          write ["cart", customer_id, product_id]
+          # Note that the transaction *takes* from inventory and *writes*
+          # outside inventory. To support this, the client must subscribe
+          # to inventory. It doesn't matter whether it subscribes to the
+          # rest of the tuplespace. See the [subspace doc](doc/subspace.md).
+        end
+      end
+    end
+  end
+  
+  local subscribe: :all do
+    PRODUCT_IDS.each do |product_id|
+      read ["product", product_id, 0] # wait until sold out
+    end
+    
+    CUSTOMER_IDS.each do |customer_id|
+      h = Hash.new(0)
+      transaction do
+        while t=take_nowait(["cart", customer_id, nil])
+          h[t[2]] += 1
+        end
+      end
+      puts "Customer #{customer_id} bought:"
+      h.keys.sort.each do |product_id|
+        printf "%10d of product %3d.\n", h[product_id], product_id
+      end
+    end
+  end
+end

data/example/{subspace.rb → subspaces/simple.rb}

@@ -1,3 +1,6 @@
+# Minimal example of using subspaces to limit which tuples each client sees.
+# Run with --trace to see assignment to subspaces.
+
 require 'tupelo/app'
 
 Tupelo.application do

data/example/take-nowait-caution.rb

@@ -0,0 +1,36 @@
+# The #take and #take_nowait methods behave the same if a match is found in
+# the local tuple store: they both send a transaction that takes the tuple.
+# If no match is found, #take blocks, but #take_nowait returns nil.
+#
+# In a transaction, #take_nowait has the same behavior. But keep in mind that
+# things may change by the time the transaction commit is successful.
+# Some other process may write a matching tuple. So, the return value of
+# nil is not a guarantee that, when the transaction finishes, there is no match.
+# This example demonstrates this point.
+#
+# See these examples for interesting uses of #take_nowait in a transaction:
+#
+# broker-optimistic.rb
+# broker-optimistic-v2.rb
+# lease.rb
+# pregel/distributed.rb
+
+require 'tupelo/app'
+
+Tupelo.application do
+  child do
+    results = transaction do
+      r1 = take_nowait [1]
+      sleep 1
+      r2 = take_nowait [2]
+      [r1, r2]
+    end
+    
+    log results
+  end
+
+  child do
+    sleep 0.5
+    write [1], [2]
+  end
+end

data/example/take-nowait.rb

@@ -10,7 +10,7 @@ Tupelo.application do
     child do
       read ["ready"]
       r = take_nowait [1]
-      log "winner! result = #{r.inspect}" if r
+      log "winner! result = #{r}" if r
     end
   end
 

data/lib/tupelo/app/builder.rb

@@ -0,0 +1,73 @@
+module Tupelo
+  # Not an essential part of the library, but used to build up groups of
+  # processes for use in examples, tests, benchmarks, etc.
+  class AppBuilder
+    attr_reader :ez
+
+    # Does this app own (as child processes) the seq, cseq, and arc servers?
+    attr_reader :owns_servers
+
+    # Arguments available to application after tupelo has parsed out switches
+    # and args that it recognizes.
+    attr_reader :argv
+
+    def initialize ez, owns_servers: nil, argv: argv
+      @ez = ez
+      @owns_servers = owns_servers
+      @argv = argv
+    end
+
+    def log
+      ez.log
+    end
+
+    # Yields a client that runs in this process.
+    def local client_class = Client, **opts, &block
+      ez.local :seqd, :cseqd, :arcd do |seqd, cseqd, arcd|
+        opts = {seq: seqd, cseq: cseqd, arc: arcd, log: log}.merge(opts)
+        run_client client_class, **opts do |client|
+          if block
+            if block.arity == 0
+              client.instance_eval &block
+            else
+              yield client
+            end
+          end
+        end
+      end
+    end
+
+    # Yields a client that runs in a subprocess.
+    #
+    # A passive client will be forced to stop after all active clients exit. Use
+    # the passive flag for processes that wait for tuples and respond in some
+    # way. Then you do not have to manually interrupt the whole application when
+    # the active processes are done. See examples.
+    def child client_class = Client, passive: false, **opts, &block
+      ez.child :seqd, :cseqd, :arcd, passive: passive do |seqd, cseqd, arcd|
+        opts = {seq: seqd, cseq: cseqd, arc: arcd, log: log}.merge(opts)
+        run_client client_class, **opts do |client|
+          if block
+            if block.arity == 0
+              client.instance_eval &block
+            else
+              yield client
+            end
+          end
+        end
+      end
+    end
+
+    def run_client client_class, opts
+      log = opts[:log]
+      log.progname = "client <starting in #{log.progname}>"
+      client = client_class.new opts
+      client.start do
+        log.progname = "client #{client.client_id}"
+      end
+      yield client
+    ensure
+      client.stop if client # gracefully exit the tuplespace management thread
+    end
+  end
+end

data/lib/tupelo/app/trace.rb

@@ -12,10 +12,14 @@ class Tupelo::AppBuilder
       note = client.notifier
       log << ( "%6s %6s %6s %s\n" % %w{ tick cid status operation } )
       loop do
-        status, tick, cid, op = note.wait
+        status, tick, cid, op, tags = note.wait
         unless status == :attempt
           s = status == :failure ? "FAIL" : ""
-          log << ( "%6d %6d %6s %p\n" % [tick, cid, s, op] )
+          if tags and not tags.empty?
+            log << ( "%6d %6d %6s %p to %p\n" % [tick, cid, s, op, tags] )
+          else
+            log << ( "%6d %6d %6s %p\n" % [tick, cid, s, op] )
+          end
         end
       end
     end

data/lib/tupelo/app.rb

@@ -1,74 +1,8 @@
 require 'easy-serve'
 require 'tupelo/client'
+require 'tupelo/app/builder'
 
 module Tupelo
-  # Not an essential part of the library, but used to build up groups of
-  # processes for use in examples, tests, benchmarks, etc.
-  class AppBuilder
-    attr_reader :ez
-
-    # Does this app own (as child processes) the seq, cseq, and arc servers?
-    attr_reader :owns_servers
-
-    def initialize ez, owns_servers: nil
-      @ez = ez
-      @owns_servers = owns_servers
-    end
-
-    def log
-      ez.log
-    end
-
-    # Yields a client that runs in this process.
-    def local client_class = Client, **opts, &block
-      ez.local :seqd, :cseqd, :arcd do |seqd, cseqd, arcd|
-        opts = {seq: seqd, cseq: cseqd, arc: arcd, log: log}.merge(opts)
-        run_client client_class, **opts do |client|
-          if block
-            if block.arity == 0
-              client.instance_eval &block
-            else
-              yield client
-            end
-          end
-        end
-      end
-    end
-
-    # Yields a client that runs in a subprocess.
-    #
-    # A passive client will be forced to stop after all active clients exit. Use
-    # the passive flag for processes that wait for tuples and respond in some
-    # way. Then you do not have to manually interrupt the whole application when
-    # the active processes are done. See examples.
-    def child client_class = Client, passive: false, **opts, &block
-      ez.child :seqd, :cseqd, :arcd, passive: passive do |seqd, cseqd, arcd|
-        opts = {seq: seqd, cseq: cseqd, arc: arcd, log: log}.merge(opts)
-        run_client client_class, **opts do |client|
-          if block
-            if block.arity == 0
-              client.instance_eval &block
-            else
-              yield client
-            end
-          end
-        end
-      end
-    end
-
-    def run_client client_class, opts
-      log = opts[:log]
-      log.progname = "client <starting in #{log.progname}>"
-      client = client_class.new opts
-      client.start do
-        log.progname = "client #{client.client_id}"
-      end
-      yield client
-    ensure
-      client.stop if client # gracefully exit the tuplespace management thread
-    end
-  end
-  
   # Returns [argv, opts], leaving orig_argv unmodified. The opts hash contains
   # switches (and their arguments, if any) recognized by tupelo. The argv array
   # contains all unrecognized arguments.
@@ -181,7 +115,7 @@ module Tupelo
         end
       end
 
-      app = AppBuilder.new(ez, owns_servers: owns_servers)
+      app = AppBuilder.new(ez, owns_servers: owns_servers, argv: argv.dup)
       
       if enable_trace
         require 'tupelo/app/trace'

data/lib/tupelo/client/reader.rb

@@ -1,14 +1,23 @@
 require 'tupelo/client/common'
 
 class Tupelo::Client
-  # include into class that defines #worker and #log
+  # Include into class that defines #worker and #log.
   module Api
+    # If block given, yield matching tuple to the block if one is found
+    # locally and then yield each new tuple as it arrives.
+    # Otherwise, return one matching tuple, blocking if necessary.
     def read_wait template
-      waiter = Waiter.new(worker.make_template(template), self)
+      waiter = Waiter.new(worker.make_template(template), self, !block_given?)
       worker << waiter
-      result = waiter.wait
-      waiter = nil
-      result
+      if block_given?
+        loop do
+          yield waiter.wait
+        end
+      else
+        result = waiter.wait
+        waiter = nil
+        result
+      end
     ensure
       worker << Unwaiter.new(waiter) if waiter
     end
@@ -40,17 +49,18 @@ class Tupelo::Client
   class WaiterBase
     attr_reader :template
     attr_reader :queue
+    attr_reader :once
 
-    def initialize template, client
+    def initialize template, client, once = true
       @template = template
       @queue = client.make_queue
       @client = client
+      @once = once
     end
 
     def gloms tuple
       if template === tuple
         peek tuple
-        true
       else
         false
       end
@@ -58,6 +68,7 @@ class Tupelo::Client
     
     def peek tuple
       queue << tuple
+      once
     end
 
     def wait

data/lib/tupelo/client/worker.rb

@@ -328,7 +328,7 @@ class Tupelo::Client
       log.debug {"applying #{op} from client #{msg.client_id}"}
 
       notify_waiters.each do |waiter|
-        waiter << [:attempt, msg.global_tick, msg.client_id, op]
+        waiter << [:attempt, msg.global_tick, msg.client_id, op, msg.tags]
       end
 
       granted_tuples = tuplespace.find_distinct_matches_for(op.takes)
@@ -370,7 +370,7 @@ class Tupelo::Client
       notify_waiters.each do |waiter|
         waiter << [
           succeeded ?  :success : :failure,
-          msg.global_tick, msg.client_id, op]
+          msg.global_tick, msg.client_id, op, msg.tags]
       end
 
       trans = nil
@@ -509,7 +509,10 @@ class Tupelo::Client
     def handle_waiter waiter
       tuple = tuplespace.find_match_for waiter.template
       if tuple
-        waiter.peek tuple
+        once = waiter.peek tuple
+        unless once
+          read_waiters << waiter
+        end
       else
         read_waiters << waiter
       end

data/lib/tupelo/version.rb

@@ -1,3 +1,3 @@
 module Tupelo
-  VERSION = "0.11"
+  VERSION = "0.12"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: tupelo
 version: !ruby/object:Gem::Version
-  version: '0.11'
+  version: '0.12'
 platform: ruby
 authors:
 - Joel VanderWerf
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-11-22 00:00:00.000000000 Z
+date: 2013-12-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: atdo
@@ -82,6 +82,7 @@ files:
 - lib/tupelo/app/remote.rb
 - lib/tupelo/app/irb-shell.rb
 - lib/tupelo/app/trace.rb
+- lib/tupelo/app/builder.rb
 - lib/tupelo/client.rb
 - lib/tupelo/tuplets/persistent-archiver.rb
 - lib/tupelo/tuplets/persistent-archiver/worker.rb
@@ -118,6 +119,7 @@ files:
 - example/pregel/pagerank.rb
 - example/pregel/pregel.rb
 - example/pregel/distributed.rb
+- example/pregel/update.rb
 - example/take-nowait.rb
 - example/boolean-match.rb
 - example/lease.rb
@@ -127,10 +129,12 @@ files:
 - example/small-simplified.rb
 - example/small.rb
 - example/lock-mgr.rb
+- example/take-nowait-caution.rb
 - example/concurrent-transactions.rb
 - example/tcp.rb
 - example/notify.rb
 - example/pulse.rb
+- example/chat/chat.rb
 - example/hash-tuples.rb
 - example/balance-xfer-locking.rb
 - example/balance-xfer-retry.rb
@@ -138,17 +142,22 @@ files:
 - example/app-and-tup.rb
 - example/multi-tier/memo2.rb
 - example/multi-tier/http.rb
+- example/multi-tier/multi-sinatras.rb
 - example/multi-tier/kvspace.rb
 - example/multi-tier/memo.rb
 - example/multi-tier/drb.rb
 - example/take-many.rb
+- example/subspaces/simple.rb
+- example/subspaces/shop/shop-v2.rb
+- example/subspaces/shop/shop-v1.rb
 - example/subspaces/pubsub.rb
 - example/dphil-optimistic.rb
 - example/async-transaction.rb
 - example/wait-interrupt.rb
+- example/fish0.rb
 - example/zk/lock.rb
-- example/subspace.rb
 - example/deadlock.rb
+- example/fish.rb
 - example/add.rb
 - example/dphil-optimistic-v2.rb
 - example/parallel.rb
@@ -159,6 +168,7 @@ files:
 - example/lock-mgr-with-queue.rb
 - example/balance-xfer.rb
 - example/cancel.rb
+- example/socket-broker.rb
 - example/timeout-trans.rb
 - example/optimist.rb
 - example/tiny-server.rb