bud 0.1.0.pre1 → 0.9.0

data/History.txt

@@ -0,0 +1,23 @@
+== 0.9.0 / 2012-03-20
+
+* Major performance enhancements
+  * Much, much faster: rewritten runtime that now uses a push-based dataflow
+  * Operator state is cached; only deltas are updated across ticks in many cases
+  * Joins that use collection keys can use collection storage for improved
+    performance
+* Improved compatibility: Bud now works with MRI 1.9 (as well as 1.8.7)
+  * Switched from ParseTree to ruby_parser
+  * Rewritten Bloom module system
+* Tuples are now represented as Ruby Structs, rather than Arrays
+  * This avoids the need to define column accessor methods by hand
+* Tests now use MiniTest rather than Test::Unit
+* Observe the following incompatibilities:
+  * Support for "semi-structured" collections have been removed. That is,
+    previously you could store extra field values into a tuple; those values
+    would be collapsed into a single array that was tacked onto the end of the
+    tuple.
+  * Support for Bloom-based signal handling has been removed
+  * Support for the "with" syntax has been removed
+  * The Bloom-based "deployment" framework has been removed
+  * Support for Tokyo Cabinet-based collections has been removed
+

data/{README → README.md}

@@ -14,23 +14,27 @@ Main deficiencies at this point are:
   available, including mutable state.  This allows programmers to get outside
   the Bloom framework and lose cleanliness.
 
-- Compatibility: Bud only works with Ruby (MRI) 1.8 and 1.9. JRuby and other
+- Compatibility: Bud only works with Ruby (MRI) 1.8.7 and 1.9. JRuby and other
   Ruby implementations are currently not supported.
 
 ## Installation
 
 To install the latest release:
+
     % gem install bud
 
 To build and install a new gem from the current development sources:
+
     % gem build bud.gemspec ; gem install bud*.gem
 
-Note that GraphViz must be installed.
+Note that [GraphViz](http://www.graphviz.org/) must be installed.
 
 Simple example programs can be found in examples. A much larger set of example
 programs and libraries can be found in the bud-sandbox repository.
 
 To run the unit tests:
+
+    % gem install minitest      # unless already installed
     % cd test; ruby ts_bud.rb
 
 ## Optional Dependencies

data/docs/cheat.md

@@ -101,13 +101,6 @@ Statements with stdio on lhs must use async merge (`<~`).<br>
 Using `stdio` on the lhs of an async merge results in writing to the `IO` object specified by the `:stdout` Bud option (`$stdout` by default).<br>
 To use `stdio` on rhs, instantiate Bud with `:stdin` option set to an `IO` object (e.g., `$stdin`).<br>
 
-### signals ###
-Built-in read-only scratch collection for receiving OS signals.<br>
-System-provided attributes: `[:key] => []`
-
-Currently catches only SIGINT ("INT") and SIGTERM ("TERM").  If Bud option `:signal_handling=>:bloom` is set, the signal is trapped and Bloom rules
-are responsible to deal with the content of `signals`.
-
 ### halt ###
 Built-in scratch collection to be used on the lhs of a rule; permanently halts the Bud instance upon first insertion.  
 
@@ -310,7 +303,7 @@ Like `pairs`, but implicitly includes a block that projects down to the right it
 `outer(`*hash pairs*`)`:<br>
 Left Outer Join.  Like `pairs`, but items in the first collection will be produced nil-padded if they have no match in the second collection.
 
-## Temp Collections and With Blocks ##
+## Temp Collections ##
 `temp`<br>
 Temp collections are scratches defined within a `bloom` block:
 

data/docs/intro.md

@@ -29,7 +29,7 @@ This alpha is targeted at "friends and family", and at developers who'd like to
 ## Getting Started ##
 We're shipping Bud with a [sandbox](http://github.com/bloom-lang/bud-sandbox) of libraries and example applications for distributed systems.  These illustrate the language and how it can be used, and also can serve as mixins for new code you might want to write.  You may be surprised at how short the provided Bud code is, but don't be fooled.
 
-To get you started with Bud, we've provided a [quick-start tutorial](getstarted.md), instructions for [deploying distributed Bud](deploy.md) programs on Amazon's EC2 cloud, and a number of other docs you can find linked from the [README](README.md).
+To get you started with Bud, we've provided a [quick-start tutorial](getstarted.md) and a number of other docs you can find linked from the [README](README.md).
 
 We welcome both constructive criticism and (hopefully occasional) smoke-out-your-ears, hair-tearing shouts of frustration.  Please point your feedback cannon at the [Bloom mailing list](http://groups.google.com/group/bloom-lang).
 

data/lib/bud/aggs.rb

@@ -4,7 +4,7 @@ module Bud
     def init(val)
       val
     end
-    
+
     # In order to support argagg, trans must return a pair:
     #  1. the running aggregate state
     #  2. a flag to indicate what the caller should do with the input tuple for argaggs
@@ -40,16 +40,16 @@ module Bud
 
   class Min < ArgExemplary #:nodoc: all
     def trans(the_state, val)
-      if the_state < val 
+      if the_state < val
         return the_state, :ignore
       elsif the_state == val
         return the_state, :keep
-      else 
+      else
         return val, :replace
       end
     end
   end
-  # exemplary aggregate method to be used in Bud::BudCollection.group.  
+  # exemplary aggregate method to be used in Bud::BudCollection.group.
   # computes minimum of x entries aggregated.
   def min(x)
     [Min.new, x]
@@ -64,7 +64,7 @@ module Bud
       end
     end
   end
-  # exemplary aggregate method to be used in Bud::BudCollection.group.  
+  # exemplary aggregate method to be used in Bud::BudCollection.group.
   # computes maximum of x entries aggregated.
   def max(x)
     [Max.new, x]
@@ -83,7 +83,7 @@ module Bud
     end
   end
 
-  # exemplary aggregate method to be used in Bud::BudCollection.group.  
+  # exemplary aggregate method to be used in Bud::BudCollection.group.
   # arbitrarily but deterministically chooses among x entries being aggregated.
   def choose(x)
     [Choose.new, x]
@@ -93,7 +93,7 @@ module Bud
     def init(x=nil) # Vitter's reservoir sampling, sample size = 1
       the_state = {:cnt => 1, :val => x}
     end
-    
+
     def trans(the_state, val)
       the_state[:cnt] += 1
       j = rand(the_state[:cnt])
@@ -113,7 +113,7 @@ module Bud
     end
   end
 
-  # exemplary aggregate method to be used in Bud::BudCollection.group.  
+  # exemplary aggregate method to be used in Bud::BudCollection.group.
   # randomly chooses among x entries being aggregated.
   def choose_rand(x=nil)
     [ChooseOneRand.new, x]
@@ -124,8 +124,8 @@ module Bud
       return the_state + val, nil
     end
   end
-  
-  # aggregate method to be used in Bud::BudCollection.group.  
+
+  # aggregate method to be used in Bud::BudCollection.group.
   # computes sum of x entries aggregated.
   def sum(x)
     [Sum.new, x]
@@ -139,8 +139,8 @@ module Bud
       return the_state + 1, nil
     end
   end
-  
-  # aggregate method to be used in Bud::BudCollection.group.  
+
+  # aggregate method to be used in Bud::BudCollection.group.
   # counts number of entries aggregated.  argument is ignored.
   def count(x=nil)
     [Count.new]
@@ -159,8 +159,8 @@ module Bud
       the_state[0]*1.0 / the_state[1]
     end
   end
-  
-  # aggregate method to be used in Bud::BudCollection.group.  
+
+  # aggregate method to be used in Bud::BudCollection.group.
   # computes average of a multiset of x values
   def avg(x)
     [Avg.new, x]
@@ -175,8 +175,8 @@ module Bud
       return the_state, nil
     end
   end
-  
-  # aggregate method to be used in Bud::BudCollection.group.  
+
+  # aggregate method to be used in Bud::BudCollection.group.
   # accumulates all x inputs into an array. note that the order of the elements
   # in the resulting array is undefined.
   def accum(x)

data/lib/bud/bud_meta.rb

@@ -37,7 +37,6 @@ class BudMeta #:nodoc: all
       # Cleanup
       stratified_rules = stratified_rules.reject{|r| r.empty?}
       dump_rewrite(stratified_rules) if @bud_instance.options[:dump_rewrite]
-
     end
     return stratified_rules
   end
@@ -140,7 +139,7 @@ class BudMeta #:nodoc: all
       end
 
       next if i == 1 and n.sexp_type == :nil # a block got rewritten to an empty block
-        
+
       # Check for a common case
       if n.sexp_type == :lasgn
         return [n, "illegal operator: '='"]
@@ -192,7 +191,7 @@ class BudMeta #:nodoc: all
       lhs = (nodes[d.lhs.to_s] ||= Node.new(d.lhs.to_s, :init, 0, [], true, false, false, false))
       lhs.in_lhs = true
       body = (nodes[d.body.to_s] ||= Node.new(d.body.to_s, :init, 0, [], false, true, false, false))
-      temporal = d.op != "<=" 
+      temporal = d.op != "<="
       lhs.edges << Edge.new(body, d.op, d.nm, temporal)
       body.in_body = true
     end
@@ -200,7 +199,7 @@ class BudMeta #:nodoc: all
     nodes.values.each {|n| calc_stratum(n, false, false, [n.name])}
     # Normalize stratum numbers because they may not be 0-based or consecutive
     remap = {}
-    # if the nodes stratum numbers are [2, 3, 2, 4], remap = {2 => 0, 3 => 1, 4 => 2} 
+    # if the nodes stratum numbers are [2, 3, 2, 4], remap = {2 => 0, 3 => 1, 4 => 2}
     nodes.values.map {|n| n.stratum}.uniq.sort.each_with_index{|num, i|
       remap[num] = i
     }
@@ -239,25 +238,19 @@ class BudMeta #:nodoc: all
 
 
   def analyze_dependencies(nodes)  # nodes = {node name => node}
-    bud = @bud_instance
-                                                      
-    preds_in_lhs = nodes.inject(Set.new) {|preds, name_n| preds.add(name_n[0]) if name_n[1].in_lhs; preds}
-    preds_in_body = nodes.inject(Set.new) {|preds, name_n| preds.add(name_n[0]) if name_n[1].in_body; preds}
+    preds_in_lhs = nodes.select {|_, node| node.in_lhs}.map {|name, _| name}.to_set
+    preds_in_body = nodes.select {|_, node| node.in_body}.map {|name, _| name}.to_set
 
+    bud = @bud_instance
     bud.t_provides.each do |p|
       pred, input = p.interface, p.input
       if input
-        # an interface pred is a source if it is an input and it is not in any rule's lhs
-        #bud.sources << [pred] unless (preds_in_lhs.include? pred)
         unless preds_in_body.include? pred.to_s
           # input interface is underspecified if not used in any rule body
           bud.t_underspecified << [pred, true] # true indicates input mode
           puts "Warning: input interface #{pred} not used"
         end
       else
-        # an interface pred is a sink if it is not an input and it is not in any rule's body
-        #(if it is in the body, then it is an intermediate node feeding some lhs)
-        #bud.sinks << [pred] unless (preds_in_body.include? pred)
         unless preds_in_lhs.include? pred.to_s
           # output interface underspecified if not in any rule's lhs
           bud.t_underspecified << [pred, false]  #false indicates output mode.
@@ -287,8 +280,8 @@ class BudMeta #:nodoc: all
     fout.puts "Declarations:"
 
     strata.each_with_index do |rules, i|
-      fout.print "=================================\n"
-      fout.print "Stratum #{i}\n"
+      fout.puts "================================="
+      fout.puts "Stratum #{i}"
       rules.each do |r|
         fout.puts "#{r.bud_obj.class}##{r.bud_obj.object_id} #{r.rule_id}"
         fout.puts "\tsrc:      #{r.src}"