bud 0.9.4 → 0.9.9
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/History.txt +106 -0
- data/README.md +6 -4
- data/Rakefile +91 -0
- data/bin/budlabel +63 -0
- data/bin/budplot +18 -8
- data/bin/budtimelines +2 -2
- data/bin/budvis +7 -1
- data/docs/README.md +8 -17
- data/docs/cheat.md +112 -13
- data/docs/getstarted.md +97 -84
- data/docs/operational.md +3 -3
- data/examples/basics/paths.rb +2 -2
- data/examples/chat/README.md +2 -0
- data/examples/chat/chat.rb +3 -2
- data/examples/chat/chat_protocol.rb +2 -2
- data/examples/chat/chat_server.rb +3 -2
- data/lib/bud.rb +229 -114
- data/lib/bud/aggs.rb +20 -4
- data/lib/bud/bud_meta.rb +83 -73
- data/lib/bud/collections.rb +306 -120
- data/lib/bud/depanalysis.rb +3 -4
- data/lib/bud/executor/README.rescan +2 -1
- data/lib/bud/executor/elements.rb +96 -95
- data/lib/bud/executor/group.rb +35 -32
- data/lib/bud/executor/join.rb +164 -183
- data/lib/bud/graphs.rb +3 -3
- data/lib/bud/labeling/bloomgraph.rb +47 -0
- data/lib/bud/labeling/budplot_style.rb +53 -0
- data/lib/bud/labeling/labeling.rb +288 -0
- data/lib/bud/lattice-core.rb +595 -0
- data/lib/bud/lattice-lib.rb +422 -0
- data/lib/bud/monkeypatch.rb +68 -32
- data/lib/bud/rebl.rb +28 -10
- data/lib/bud/rewrite.rb +361 -152
- data/lib/bud/server.rb +16 -8
- data/lib/bud/source.rb +21 -18
- data/lib/bud/state.rb +93 -4
- data/lib/bud/storage/zookeeper.rb +45 -33
- data/lib/bud/version.rb +3 -0
- data/lib/bud/viz.rb +10 -12
- data/lib/bud/viz_util.rb +8 -3
- metadata +107 -108
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 628133a838208bab16e5ec51d166da3b35803cc1
|
4
|
+
data.tar.gz: afeab2b5aaa75b379aa5561559fb7acc89cfb50a
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: cd64cbf997cd2592e90f1be28e9818c074ff8bbfbb0cc997dfe0d4af455484e98959f72408ab70487ba29de32cfd10fdf7d662bca702c8ff92ddf82862d47941
|
7
|
+
data.tar.gz: 541c9f257e269de04137d6217e8960c680e1864cc9065cd12eb6a04eecf869dc6080ea184ac12c8d45e0f24ae589339aaddf99ea47c4020b2d3060a0669a7b00
|
data/History.txt
CHANGED
@@ -1,3 +1,109 @@
|
|
1
|
+
== 0.9.8 / ???
|
2
|
+
|
3
|
+
* Fix bug in the stratification algorithm
|
4
|
+
* Fix bug with insertions into stdio inside bootstrap blocks
|
5
|
+
* Improve error message when <~ applied to a collection type that doesn't
|
6
|
+
support it (#316)
|
7
|
+
* Fix rescan logic for Zookeeper-backed collections (#317)
|
8
|
+
* Fix rescan logic for chained negation operators in a single rule
|
9
|
+
* Support + operator for concatenating tuples
|
10
|
+
* More consistent behavior for #==, #eql?, and #hash methods on Bud tuple
|
11
|
+
objects (TupleStruct)
|
12
|
+
* Fix intermittent EM::ConnectionNotBound exceptions during the Bud shutdown
|
13
|
+
sequence
|
14
|
+
* Improve stratification algorithm to be slightly more aggressive; that is, we
|
15
|
+
can place certain rules in an earlier strata than we were previously
|
16
|
+
(partially fixes #277)
|
17
|
+
* Add Rakefile to simplify common development operations (Joel VanderWerf, #324)
|
18
|
+
|
19
|
+
== 0.9.7 / 2013-04-22
|
20
|
+
|
21
|
+
* Avoid raising an exception when Bud is used with Ruby 2.0. There isn't
|
22
|
+
anything special that Bud itself needs to do to support Ruby 2.0, but
|
23
|
+
RubyParser doesn't properly support 2.0 yet. So using 2.0-only syntax in Bud
|
24
|
+
rules is unlikely to work (until RubyParser is updated), but no other problems
|
25
|
+
have been observed.
|
26
|
+
* Reject <= on collections from outside Bloom rules (#289, Aaron Davidson). The
|
27
|
+
previous behavior was error-prone: inserting into a scratch collection via <=
|
28
|
+
was not rejected but the inserted data would immediately be discarded at the
|
29
|
+
beginning of the next tick. Hence, it is simpler to require <+ or <~ for all
|
30
|
+
insertions from outside Bloom rules.
|
31
|
+
* Fix bug in join operators whose qualifiers reference collections defined
|
32
|
+
inside imported modules (#301)
|
33
|
+
* Fix bug in join operators when the same table and column name appears on the
|
34
|
+
LHS of multiple join predicates (#313)
|
35
|
+
* Fix bug in outer joins with multiple predicates (#315)
|
36
|
+
* Fix several bugs in rescan/invalidation logic for tables in the presence of
|
37
|
+
upstream deletions (#303)
|
38
|
+
* Fix regression in file_reader collections (#304)
|
39
|
+
* Improve chaining of notin operators
|
40
|
+
* Optimize join evaluation with multiple predicates
|
41
|
+
* Optimize notin self joins
|
42
|
+
* Improve error reporting for syntax errors on temp collections (#310)
|
43
|
+
* Add Travis CI integration hooks (#300, Josh Rosen)
|
44
|
+
|
45
|
+
Lattices:
|
46
|
+
|
47
|
+
* Rename lmap#apply_monotone to lmap#apply. Also, allow #apply to take either
|
48
|
+
monotone functions or morphisms, and improve error reporting.
|
49
|
+
* Add lmap#filter. This takes an lmap whose values are elements of the lbool
|
50
|
+
lattice and returns an lmap containing only those k/v pairs where the value is
|
51
|
+
true.
|
52
|
+
* More intuitive lattice equi-join syntax (lset#eqjoin). Rather than specifying
|
53
|
+
the join predicate(s) using array indexes, instead allow them to be specified
|
54
|
+
using a hash of field names, as in traditional Bloom joins. This only works
|
55
|
+
when the lset contains Structs.
|
56
|
+
* Remove lset#project, and allow lset#eqjoin to take zero predicates; in the
|
57
|
+
latter case, eqjoin computes the Cartesian product.
|
58
|
+
* Support deletion rules (<-) with lattice values on the RHS
|
59
|
+
|
60
|
+
== 0.9.6 / 2013-02-25
|
61
|
+
|
62
|
+
* Support syntax sugar for initializing lattices (#294). For example, rather
|
63
|
+
than writing "foo <= Bud::MaxLattice.new(2)", you can now just write "foo <=
|
64
|
+
2". Note that, due to a bug in the superator gem, you cannot use this syntax
|
65
|
+
with the <+ operator.
|
66
|
+
* Allow nested lattice values to be sent over channels (#295, patch from Josh
|
67
|
+
Rosen). Lattice values could previously be sent as a field value in a tuple
|
68
|
+
delivered over a channel, but they could not be embedded arbitrarily deeply
|
69
|
+
within tuples (e.g., lattices nested within an array could not previously be
|
70
|
+
sent as a field value).
|
71
|
+
* Support "Comparable" for Bud::Lattice values
|
72
|
+
* Add support for lattices to rebl
|
73
|
+
* Reject attempts to insert into stdio via <+ (#288)
|
74
|
+
* Restore functionality of reading from stdio (i.e., stdin) with MRI 1.9
|
75
|
+
* Improve MRI 1.8 compatibility
|
76
|
+
* Require ruby_parser >= 3.1.0
|
77
|
+
* Fix bug in bootstrap blocks for lattice values
|
78
|
+
* Fix bug in rescan/invalidation for rules that reference lattice values and use
|
79
|
+
the <+ or <~ operators (#290)
|
80
|
+
* Fix bug in rescan logic for notin operator (#291)
|
81
|
+
* Fix bug in notin operators involving self joins (#298)
|
82
|
+
* Fix error when output from a notin operator was fed into a lattice
|
83
|
+
|
84
|
+
== 0.9.5 / 2012-11-24
|
85
|
+
|
86
|
+
* Lattice branch (Bloom^L) merged
|
87
|
+
* Compatibility with recent versions of ruby_parser (3.0.2+) and ruby2ruby
|
88
|
+
(2.0.1+). Older versions of these two gems are no longer supported
|
89
|
+
* Add support for aggregate functions that take multiple input columns
|
90
|
+
* Add built-in aggregate function accum_pair(x, y), which produces a Set of
|
91
|
+
pairs (two-element arrays [x,y])
|
92
|
+
* Support user-specified code blocks in payloads(), argagg(), argmin() and
|
93
|
+
argmax()
|
94
|
+
* Change behavior of BudChannel#payloads for channels with two
|
95
|
+
columns. Previously we returned a single *column* (scalar) value in this case;
|
96
|
+
now we always return a tuple with k-1 columns
|
97
|
+
* More consistent behavior for BudCollection#sort when used outside Bloom
|
98
|
+
programs
|
99
|
+
* Restore support for each_with_index() over Bud collections
|
100
|
+
* Restore functionality of Zookeeper-backed Bud collections and fix
|
101
|
+
incompatibility with recent (> 0.4.4) versions of the Zookeeper gem
|
102
|
+
* Optimize parsing of Bloom statements, particularly for large Bloom programs
|
103
|
+
* Fix bug in argagg state materialization
|
104
|
+
* Fix bug in chaining argmin() or argmax() expressions
|
105
|
+
* Fix bug in chaining notin() expressions
|
106
|
+
|
1
107
|
== 0.9.4 / 2012-09-06
|
2
108
|
|
3
109
|
* Optimize grouping performance
|
data/README.md
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
[![Build Status](https://travis-ci.org/bloom-lang/bud.svg?branch=v0.9.8)](https://travis-ci.org/bloom-lang/bud)
|
1
2
|
# Bud
|
2
3
|
|
3
4
|
This is Bud, a.k.a. "Bloom Under Development". It is an initial cut at a Bloom
|
@@ -11,11 +12,12 @@ documentation.
|
|
11
12
|
Main deficiencies at this point are:
|
12
13
|
|
13
14
|
- No Ruby constraints: Within Bloom programs the full power of Ruby is also
|
14
|
-
available, including mutable state.
|
15
|
-
|
15
|
+
available, including mutable state. This allows programmers to get outside the
|
16
|
+
Bloom framework and lose cleanliness.
|
16
17
|
|
17
|
-
- Compatibility: Bud only works with Ruby (MRI) 1.8.7 and 1.9.
|
18
|
-
Ruby
|
18
|
+
- Compatibility: Bud only works with Ruby (MRI) 1.8.7 and 1.9. Bud also has
|
19
|
+
experimental support for Ruby 2.0. JRuby and other Ruby implementations are
|
20
|
+
currently not supported.
|
19
21
|
|
20
22
|
## Installation
|
21
23
|
|
data/Rakefile
ADDED
@@ -0,0 +1,91 @@
|
|
1
|
+
require 'rake'
|
2
|
+
require 'rake/testtask'
|
3
|
+
|
4
|
+
PRJ = "bud"
|
5
|
+
|
6
|
+
def version
|
7
|
+
@version ||= begin
|
8
|
+
$LOAD_PATH.unshift 'lib'
|
9
|
+
require 'bud/version'
|
10
|
+
warn "Bud::VERSION not a string" unless Bud::VERSION.kind_of? String
|
11
|
+
Bud::VERSION
|
12
|
+
end
|
13
|
+
end
|
14
|
+
|
15
|
+
def tag
|
16
|
+
@tag ||= "v#{version}"
|
17
|
+
end
|
18
|
+
|
19
|
+
desc "Run all tests"
|
20
|
+
task :test => "test:unit"
|
21
|
+
|
22
|
+
TESTS = FileList["test/tc_*.rb"]
|
23
|
+
SLOW_TESTS = %w{ test/tc_execmodes.rb }
|
24
|
+
|
25
|
+
namespace :test do
|
26
|
+
desc "Run unit tests"
|
27
|
+
Rake::TestTask.new :unit do |t|
|
28
|
+
t.libs << "lib"
|
29
|
+
t.ruby_opts = %w{ -C test }
|
30
|
+
t.test_files = TESTS.sub('test/', '')
|
31
|
+
### it would be better to make each tc_*.rb not depend on pwd
|
32
|
+
end
|
33
|
+
|
34
|
+
desc "Run quick unit tests"
|
35
|
+
Rake::TestTask.new :quick do |t|
|
36
|
+
t.libs << "lib"
|
37
|
+
t.ruby_opts = %w{ -C test }
|
38
|
+
t.test_files = TESTS.exclude(*SLOW_TESTS).sub('test/', '')
|
39
|
+
end
|
40
|
+
|
41
|
+
desc "Run quick non-zk unit tests"
|
42
|
+
Rake::TestTask.new :quick_no_zk do |t|
|
43
|
+
t.libs << "lib"
|
44
|
+
t.ruby_opts = %w{ -C test }
|
45
|
+
t.test_files = TESTS.
|
46
|
+
exclude('test/tc_zookeeper.rb').
|
47
|
+
exclude(*SLOW_TESTS).
|
48
|
+
sub('test/', '')
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
desc "Commit, tag, and push repo; build and push gem"
|
53
|
+
task :release => "release:is_new_version" do
|
54
|
+
require 'tempfile'
|
55
|
+
|
56
|
+
sh "gem build #{PRJ}.gemspec"
|
57
|
+
|
58
|
+
file = Tempfile.new "template"
|
59
|
+
begin
|
60
|
+
file.puts "release #{version}"
|
61
|
+
file.close
|
62
|
+
sh "git commit --allow-empty -a -v -t #{file.path}"
|
63
|
+
ensure
|
64
|
+
file.close unless file.closed?
|
65
|
+
file.unlink
|
66
|
+
end
|
67
|
+
|
68
|
+
sh "git tag #{tag}"
|
69
|
+
sh "git push"
|
70
|
+
sh "git push --tags"
|
71
|
+
|
72
|
+
sh "gem push #{tag}.gem"
|
73
|
+
end
|
74
|
+
|
75
|
+
namespace :release do
|
76
|
+
desc "Diff to latest release"
|
77
|
+
task :diff do
|
78
|
+
latest = `git describe --abbrev=0 --tags --match 'v*'`.chomp
|
79
|
+
sh "git diff #{latest}"
|
80
|
+
end
|
81
|
+
|
82
|
+
desc "Log to latest release"
|
83
|
+
task :log do
|
84
|
+
latest = `git describe --abbrev=0 --tags --match 'v*'`.chomp
|
85
|
+
sh "git log #{latest}.."
|
86
|
+
end
|
87
|
+
|
88
|
+
task :is_new_version do
|
89
|
+
abort "#{tag} exists; update version!" unless `git tag -l #{tag}`.empty?
|
90
|
+
end
|
91
|
+
end
|
data/bin/budlabel
ADDED
@@ -0,0 +1,63 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
require 'rubygems'
|
4
|
+
require 'bud'
|
5
|
+
require 'getopt/std'
|
6
|
+
require 'bud/labeling/labeling'
|
7
|
+
require 'bud/labeling/bloomgraph'
|
8
|
+
require 'bud/labeling/budplot_style'
|
9
|
+
|
10
|
+
$LOAD_PATH.unshift(".")
|
11
|
+
|
12
|
+
@opts = Getopt::Std.getopts("r:i:p:O:CP")
|
13
|
+
|
14
|
+
unless @opts["r"] and @opts["i"]
|
15
|
+
puts "USAGE:"
|
16
|
+
puts "-r REQUIRE"
|
17
|
+
puts "-i INCLUDE"
|
18
|
+
puts "[-p INCLUDE PATH]"
|
19
|
+
puts "[-O <FMT> Output a graphviz representation of the module in FMT format (pdf if not specified)."
|
20
|
+
puts "-C Concise output -- Associate a single label with each output interface"
|
21
|
+
puts "-P Path-based output -- For each output interface, attribute a label to paths from each input interface"
|
22
|
+
exit
|
23
|
+
end
|
24
|
+
|
25
|
+
hreadable = {
|
26
|
+
"D" => "Diffluent: Nondeterministic output contents.",
|
27
|
+
"A" => "Asynchronous. Nondeterministic output orders.",
|
28
|
+
"N" => "Nonmonotonic. Output contents are sensitive to input orders.",
|
29
|
+
"Bot" => "Monotonic. Order-insensitive and retraction-free."
|
30
|
+
}
|
31
|
+
|
32
|
+
if @opts["p"]
|
33
|
+
$LOAD_PATH.unshift @opts["p"]
|
34
|
+
end
|
35
|
+
|
36
|
+
require @opts["r"]
|
37
|
+
c = Label.new(@opts["i"])
|
38
|
+
|
39
|
+
puts "--- Report for module #{@opts["i"]} ---"
|
40
|
+
|
41
|
+
if @opts["C"]
|
42
|
+
puts "---------------"
|
43
|
+
puts "Output\t\tLabel"
|
44
|
+
puts "---------------"
|
45
|
+
c.output_report.each_pair do |k, v|
|
46
|
+
puts [k, hreadable[v]].join("\t")
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
if @opts["P"]
|
51
|
+
c.path_report.each_pair do |output, inpaths|
|
52
|
+
puts ""
|
53
|
+
puts "--------------------"
|
54
|
+
puts "Output\tInput\tLabel"
|
55
|
+
puts "--------------------"
|
56
|
+
puts output
|
57
|
+
inpaths.each_pair do |inp, lbl|
|
58
|
+
puts "\t#{inp}\t#{hreadable[lbl]}"
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
62
|
+
|
63
|
+
c.write_graph(@opts["O"]) if @opts["O"]
|
data/bin/budplot
CHANGED
@@ -21,6 +21,10 @@ def make_instance(mods)
|
|
21
21
|
# If we're given a single identifier that names a class, just return an
|
22
22
|
# instance of that class. Otherwise, define a bogus class that includes all
|
23
23
|
# the module names specified by the user and return an instance.
|
24
|
+
tmpserver = TCPServer.new('127.0.0.1', 0) # get a free port
|
25
|
+
default_params = {:dbm_dir => "/tmp/budplot_dbm_" + SecureRandom.uuid.to_s, :port => tmpserver.addr[1]}
|
26
|
+
|
27
|
+
|
24
28
|
mods.each do |m|
|
25
29
|
unless is_constant? m
|
26
30
|
puts "Error: unable to find definition for module or class \"#{m}\""
|
@@ -30,7 +34,7 @@ def make_instance(mods)
|
|
30
34
|
mod_klass = eval m
|
31
35
|
if mod_klass.class == Class
|
32
36
|
if mods.length == 1
|
33
|
-
return mod_klass.new
|
37
|
+
return mod_klass.new(default_params)
|
34
38
|
else
|
35
39
|
puts "Error: cannot intermix class \"#{mod_klass}\" with modules"
|
36
40
|
exit
|
@@ -50,7 +54,7 @@ def make_instance(mods)
|
|
50
54
|
]
|
51
55
|
class_def = def_lines.flatten.join("\n")
|
52
56
|
eval(class_def)
|
53
|
-
f =FooBar.new
|
57
|
+
f = FooBar.new(default_params)
|
54
58
|
3.times{ f.tick }
|
55
59
|
f
|
56
60
|
end
|
@@ -77,8 +81,8 @@ def process(mods)
|
|
77
81
|
d = make_instance(mods)
|
78
82
|
|
79
83
|
interfaces = {}
|
80
|
-
d.t_provides.each do |
|
81
|
-
interfaces[
|
84
|
+
d.t_provides.to_a.each do |prov|
|
85
|
+
interfaces[prov.interface] = prov.input
|
82
86
|
end
|
83
87
|
|
84
88
|
tabinf = {}
|
@@ -201,13 +205,18 @@ def get_trace_data
|
|
201
205
|
end
|
202
206
|
|
203
207
|
if ARGV.length < 2
|
204
|
-
puts "Usage: budplot LIST_OF_FILES LIST_OF_MODULES_OR_CLASSES"
|
208
|
+
puts "Usage: budplot [-I PATH_TO_RUBY_INCLUDES] LIST_OF_FILES LIST_OF_MODULES_OR_CLASSES"
|
205
209
|
exit
|
206
210
|
end
|
207
211
|
|
208
|
-
@opts = Getopt::Std.getopts("t:")
|
209
|
-
|
210
|
-
|
212
|
+
@opts = Getopt::Std.getopts("I:t:")
|
213
|
+
unless @opts["I"].nil?
|
214
|
+
if @opts["I"].class == Array
|
215
|
+
@opts["I"].each{|i| $:.unshift i}
|
216
|
+
else
|
217
|
+
$:.unshift @opts["I"]
|
218
|
+
end
|
219
|
+
end
|
211
220
|
|
212
221
|
@data = get_trace_data
|
213
222
|
`mkdir bud_doc`
|
@@ -215,6 +224,7 @@ end
|
|
215
224
|
modules = []
|
216
225
|
ARGV.each do |arg|
|
217
226
|
if File.exists? arg
|
227
|
+
arg = File.expand_path arg
|
218
228
|
require arg
|
219
229
|
else
|
220
230
|
modules << arg
|
data/bin/budtimelines
CHANGED
@@ -10,7 +10,7 @@ include VizUtil
|
|
10
10
|
|
11
11
|
module Depends
|
12
12
|
state do
|
13
|
-
table :depends, [:rid, :lhs, :op, :rhs, :nm]
|
13
|
+
table :depends, [:bud_obj, :rid, :lhs, :op, :rhs, :nm, :in_body]
|
14
14
|
end
|
15
15
|
end
|
16
16
|
|
@@ -49,7 +49,7 @@ end
|
|
49
49
|
module DeltaLogic
|
50
50
|
include TPSchema
|
51
51
|
bloom do
|
52
|
-
zerod_cards <= cardinalities{|c| c + [c.bud_time-1]}
|
52
|
+
zerod_cards <= cardinalities{|c| c.to_a + [c.bud_time-1]}
|
53
53
|
zerod_cards <= (times * depends).pairs do |t, d|
|
54
54
|
unless cardinalities{|c| c[1] if c[0] == t.bud_time}.include? d[1]
|
55
55
|
[t.bud_time, d[1], 0, t.bud_time - 1]
|
data/bin/budvis
CHANGED
@@ -28,7 +28,13 @@ usage unless ARGV[0]
|
|
28
28
|
usage if ARGV[0] == '--help'
|
29
29
|
|
30
30
|
meta, data = get_meta2(BUD_DBM_DIR)
|
31
|
-
|
31
|
+
|
32
|
+
# prune outbufs from tabinf
|
33
|
+
tabinf = meta[:tabinf].find_all do |k|
|
34
|
+
!(k[1] == "Bud::BudChannel" and k[0] =~ /_snd\z/)
|
35
|
+
end
|
36
|
+
|
37
|
+
vh = VizHelper.new(tabinf, meta[:cycle], meta[:depends], meta[:rules], ARGV[0], meta[:provides])
|
32
38
|
data.each do |d|
|
33
39
|
vh.full_info << d
|
34
40
|
end
|
data/docs/README.md
CHANGED
@@ -6,28 +6,19 @@ Welcome to the documentation for *Bud*, a prototype of Bloom under development.
|
|
6
6
|
The documents here are organized to be read in any order, but you might like to
|
7
7
|
try the following:
|
8
8
|
|
9
|
-
* [intro.md
|
10
|
-
* [getstarted.md
|
9
|
+
* [intro](intro.md): A brief introduction to Bud and Bloom.
|
10
|
+
* [getstarted](getstarted.md): A quickstart to teach you basic Bloom
|
11
11
|
concepts, the use of `rebl` interactive terminal, and the embedding of Bloom
|
12
12
|
code in Ruby via the `Bud` module.
|
13
|
-
* [operational.md
|
13
|
+
* [operational](operational.md): An operational view of Bloom, to provide
|
14
14
|
a more detailed model of how Bloom code is evaluated by Bud.
|
15
|
-
* [cheat.md
|
16
|
-
* [modules.md
|
17
|
-
* [ruby\_hooks.md
|
15
|
+
* [cheat](cheat.md): Full documentation of the language constructs in a concise "cheat sheet" style.
|
16
|
+
* [modules](modules.md): An overview of Bloom's modularity features.
|
17
|
+
* [ruby_hooks](ruby\_hooks.md): Bud module methods that allow you to
|
18
18
|
interact with the Bud evaluator from other Ruby threads.
|
19
|
-
* [visualizations.md
|
19
|
+
* [visualizations](visualizations.md): Overview of the `budvis` and
|
20
20
|
`budplot` tools for visualizing Bloom program analyses.
|
21
|
-
* [bfs.md
|
22
|
-
|
23
|
-
[intro]: /bloom-lang/bud/blob/master/docs/intro.md
|
24
|
-
[getstarted]: /bloom-lang/bud/blob/master/docs/getstarted.md
|
25
|
-
[operational]: /bloom-lang/bud/blob/master/docs/operational.md
|
26
|
-
[cheat]: /bloom-lang/bud/blob/master/docs/cheat.md
|
27
|
-
[modules]: /bloom-lang/bud/blob/master/docs/modules.md
|
28
|
-
[ruby_hooks]: /bloom-lang/bud/blob/master/docs/ruby_hooks.md
|
29
|
-
[visualizations]: /bloom-lang/bud/blob/master/docs/visualizations.md
|
30
|
-
[bfs]: /bloom-lang/bud/blob/master/docs/bfs.md
|
21
|
+
* [bfs](bfs.md): A walkthrough of the Bloom distributed filesystem.
|
31
22
|
|
32
23
|
In addition, the [bud-sandbox](http://github.com/bloom-lang/bud-sandbox) GitHub
|
33
24
|
repository contains lots of useful libraries and example programs built using
|
data/docs/cheat.md
CHANGED
@@ -21,15 +21,23 @@ As in Ruby, backslash is used to escape a newline.<br>
|
|
21
21
|
end
|
22
22
|
|
23
23
|
## State Declarations ##
|
24
|
-
|
25
|
-
|
26
|
-
|
24
|
+
|
25
|
+
A `state` block contains definitions of two kinds of program state:
|
26
|
+
*collections* and *lattices*. A Bud collection is a *set* of *facts*; each fact
|
27
|
+
is an array of Ruby values. Note that collections do not contain duplicates
|
28
|
+
(inserting a duplicate fact into a collection is ignored).
|
27
29
|
|
28
30
|
Like a table in a relational database, a subset of the columns in a collection
|
29
31
|
makeup the collection's _key_. Attempting to insert two facts into a collection
|
30
32
|
that agree on the key columns (but are not duplicates) results in a runtime
|
31
33
|
exception.
|
32
34
|
|
35
|
+
A lattice represents a value that *grows* over time, where the definition of
|
36
|
+
"growth" depends on the kind of lattice in question. For example, an `lset`
|
37
|
+
lattice contains a set of facts that grows over time (similar to a traditional
|
38
|
+
Bud collection), whereas an `lmax` lattice holds an increasing integer
|
39
|
+
value. For more information on lattices, see the section below.
|
40
|
+
|
33
41
|
### Default Declaration Syntax ###
|
34
42
|
*BudCollection :name, [keys] => [values]*
|
35
43
|
|
@@ -186,7 +194,7 @@ implicit map:
|
|
186
194
|
end
|
187
195
|
|
188
196
|
## BudCollection-Specific Methods ##
|
189
|
-
`bc.schema`: returns the schema of `bc` (Hash of key column names => non-key column names). Note that for channels, this omits the location specifier (<tt>@</tt>).<br>
|
197
|
+
`bc.schema`: returns the schema of `bc` (Hash of key column names => non-key column names; if no non-key columns, just an Array of key column names). Note that for channels, this omits the location specifier (<tt>@</tt>).<br>
|
190
198
|
|
191
199
|
`bc.cols`: returns the column names in `bc` as an Array<br>
|
192
200
|
|
@@ -224,14 +232,14 @@ Output the facts in `bc` that do not appear in `bc2`, as follows. First, we form
|
|
224
232
|
:attr2`).
|
225
233
|
|
226
234
|
2. If a code block is specified, invoke the block on every pair of matching
|
227
|
-
tuples in the join result. Any matches for which the block returns `
|
235
|
+
tuples in the join result. Any matches for which the block returns `false`
|
228
236
|
are removed from `t`.
|
229
237
|
|
230
238
|
Finally, we output every tuple of `bc` that does *not* appear in `t`.
|
231
239
|
|
232
240
|
# output items from foo if (a) there is no matching key in bar, or
|
233
241
|
# (b) all matching keys in bar have a smaller value
|
234
|
-
stdio <~ foo.notin(bar, :key=>:key) {|f, b|
|
242
|
+
stdio <~ foo.notin(bar, :key=>:key) {|f, b| f.val <= b.val}
|
235
243
|
|
236
244
|
|
237
245
|
## SQL-style grouping/aggregation (and then some) ##
|
@@ -239,18 +247,12 @@ Finally, we output every tuple of `bc` that does *not* appear in `t`.
|
|
239
247
|
* `bc.group([:col1, :col2], min(:col3))`. *akin to min(col3) GROUP BY col1,col2*
|
240
248
|
* exemplary aggs: `min`, `max`, `bool_and`, `bool_or`, `choose`
|
241
249
|
* summary aggs: `sum`, `avg`, `count`
|
242
|
-
* structural aggs: `accum` *accumulates inputs into a Set*
|
250
|
+
* structural aggs: `accum`, `accum_pair` *accumulates inputs into a Set; accum_pair takes two inputs and accumulates a Set of pairs (two element arrays)*
|
243
251
|
* `bc.argmax([:attr1], :attr2)` *returns the bc items per attr1 that have highest attr2*
|
244
252
|
* `bc.argmin([:attr1], :attr2)`
|
245
253
|
* `bc.argagg(:exemplary_agg_name, [:attr1], :attr2))`. *generalizes argmin/max: returns the bc items per attr1 that are chosen by the exemplary
|
246
254
|
aggregate named*
|
247
255
|
|
248
|
-
### Built-in Aggregates: ###
|
249
|
-
|
250
|
-
* Exemplary aggs: `min`, `max`, `choose`
|
251
|
-
* Summary aggs: `count`, `sum`, `avg`
|
252
|
-
* Structural aggs: `accum`
|
253
|
-
|
254
256
|
Note that custom aggregation can be written using `reduce`.
|
255
257
|
|
256
258
|
## Collection Combination (Join) ###
|
@@ -313,6 +315,103 @@ The schema of a temp collection in inherited from the rhs; if the rhs has no
|
|
313
315
|
schema, a simple one is manufactured to suit the data found in the rhs at
|
314
316
|
runtime: `[c0, c1, ...]`.
|
315
317
|
|
318
|
+
## Lattices ##
|
319
|
+
|
320
|
+
In addition to traditional Bud collections and relational-style statements that
|
321
|
+
operate over collections, Bud also supports lattices and rules that operate over
|
322
|
+
lattices. Lattices provide a way to represent values that *grow over time*,
|
323
|
+
where the definition of "growth" depends on the kind of lattice. The following
|
324
|
+
built-in lattice types are currently supported:
|
325
|
+
|
326
|
+
<table>
|
327
|
+
<tr>
|
328
|
+
<td><b>Name</b></td>
|
329
|
+
<td><b>Description</b></td>
|
330
|
+
<td><b>Initial Value</b></td>
|
331
|
+
<td><b>Monotone Functions</b></td>
|
332
|
+
</tr>
|
333
|
+
|
334
|
+
<tr>
|
335
|
+
<td><code>lbool</code></td>
|
336
|
+
<td>Threshold test (<code>false</code> => <code>true</code> conditional)</td>
|
337
|
+
<td>false</td>
|
338
|
+
<td>when_true</td>
|
339
|
+
</tr>
|
340
|
+
|
341
|
+
<tr>
|
342
|
+
<td><code>lmax</code></td>
|
343
|
+
<td>Increasing numeric value</td>
|
344
|
+
<td>-∞</td>
|
345
|
+
<td>gt(n), gt_eq(n), +(n), -(n)</td>
|
346
|
+
</tr>
|
347
|
+
|
348
|
+
<tr>
|
349
|
+
<td><code>lmin</code></td>
|
350
|
+
<td>Decreasing numeric value</td>
|
351
|
+
<td>+∞</td>
|
352
|
+
<td>lt(n), lt_eq(n), +(n), -(n)</td>
|
353
|
+
</tr>
|
354
|
+
|
355
|
+
<tr>
|
356
|
+
<td><code>lset</code></td>
|
357
|
+
<td>Growing set of values</td>
|
358
|
+
<td>empty set</td>
|
359
|
+
<td>contains?, eqjoin, filter, intersect, product, project, size</td>
|
360
|
+
</tr>
|
361
|
+
|
362
|
+
<tr>
|
363
|
+
<td><code>lpset</code></td>
|
364
|
+
<td>Growing set of non-negative numeric values</td>
|
365
|
+
<td>empty set</td>
|
366
|
+
<td>contains?, eqjoin, filter, intersect, product, project, size, sum</td>
|
367
|
+
</tr>
|
368
|
+
|
369
|
+
<tr>
|
370
|
+
<td><code>lbag</code></td>
|
371
|
+
<td>Growing multiset of values</td>
|
372
|
+
<td>empty multiset</td>
|
373
|
+
<td>contains?, multiplicity, intersect, product, project, size</td>
|
374
|
+
</tr>
|
375
|
+
|
376
|
+
<tr>
|
377
|
+
<td><code>lmap</code></td>
|
378
|
+
<td>Map from keys to lattice values</td>
|
379
|
+
<td>empty map</td>
|
380
|
+
<td>at, intersect, key?, key_set, project, size</td>
|
381
|
+
</tr>
|
382
|
+
</table>
|
383
|
+
|
384
|
+
Lattices can be declared in `state` blocks in a similar manner to traditional
|
385
|
+
Bud collections. Similarly, Bloom rules can invoke functions on lattice
|
386
|
+
values. A simple Bloom program that uses lattices to compute a quorum vote is as
|
387
|
+
follows:
|
388
|
+
|
389
|
+
```ruby
|
390
|
+
QUORUM_SIZE = 5
|
391
|
+
RESULT_ADDR = "example.org"
|
392
|
+
|
393
|
+
class QuorumVote
|
394
|
+
include Bud
|
395
|
+
|
396
|
+
state do
|
397
|
+
channel :vote_chn, [:@addr, :voter_id]
|
398
|
+
channel :result_chn, [:@addr]
|
399
|
+
lset :votes
|
400
|
+
lmax :vote_cnt
|
401
|
+
lbool :vote_done
|
402
|
+
end
|
403
|
+
|
404
|
+
bloom do
|
405
|
+
votes <= vote_chn {|v| v.voter_id}
|
406
|
+
vote_cnt <= votes.size
|
407
|
+
got_quorum <= vote_cnt.gt_eq(QUORUM_SIZE)
|
408
|
+
result_chn <~ got_quorum.when_true { [RESULT_ADDR] }
|
409
|
+
end
|
410
|
+
end
|
411
|
+
```
|
412
|
+
|
413
|
+
For more information on lattice support in Bloom, see this [recent paper](http://db.cs.berkeley.edu/papers/socc12-blooml.pdf).
|
414
|
+
|
316
415
|
## Bud Modules ##
|
317
416
|
A Bud module combines state (collections) and logic (Bloom rules). Using modules allows your program to be decomposed into a collection of smaller units.
|
318
417
|
|