bud 0.9.4 → 0.9.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/History.txt +106 -0
- data/README.md +6 -4
- data/Rakefile +91 -0
- data/bin/budlabel +63 -0
- data/bin/budplot +18 -8
- data/bin/budtimelines +2 -2
- data/bin/budvis +7 -1
- data/docs/README.md +8 -17
- data/docs/cheat.md +112 -13
- data/docs/getstarted.md +97 -84
- data/docs/operational.md +3 -3
- data/examples/basics/paths.rb +2 -2
- data/examples/chat/README.md +2 -0
- data/examples/chat/chat.rb +3 -2
- data/examples/chat/chat_protocol.rb +2 -2
- data/examples/chat/chat_server.rb +3 -2
- data/lib/bud.rb +229 -114
- data/lib/bud/aggs.rb +20 -4
- data/lib/bud/bud_meta.rb +83 -73
- data/lib/bud/collections.rb +306 -120
- data/lib/bud/depanalysis.rb +3 -4
- data/lib/bud/executor/README.rescan +2 -1
- data/lib/bud/executor/elements.rb +96 -95
- data/lib/bud/executor/group.rb +35 -32
- data/lib/bud/executor/join.rb +164 -183
- data/lib/bud/graphs.rb +3 -3
- data/lib/bud/labeling/bloomgraph.rb +47 -0
- data/lib/bud/labeling/budplot_style.rb +53 -0
- data/lib/bud/labeling/labeling.rb +288 -0
- data/lib/bud/lattice-core.rb +595 -0
- data/lib/bud/lattice-lib.rb +422 -0
- data/lib/bud/monkeypatch.rb +68 -32
- data/lib/bud/rebl.rb +28 -10
- data/lib/bud/rewrite.rb +361 -152
- data/lib/bud/server.rb +16 -8
- data/lib/bud/source.rb +21 -18
- data/lib/bud/state.rb +93 -4
- data/lib/bud/storage/zookeeper.rb +45 -33
- data/lib/bud/version.rb +3 -0
- data/lib/bud/viz.rb +10 -12
- data/lib/bud/viz_util.rb +8 -3
- metadata +107 -108
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 628133a838208bab16e5ec51d166da3b35803cc1
|
4
|
+
data.tar.gz: afeab2b5aaa75b379aa5561559fb7acc89cfb50a
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: cd64cbf997cd2592e90f1be28e9818c074ff8bbfbb0cc997dfe0d4af455484e98959f72408ab70487ba29de32cfd10fdf7d662bca702c8ff92ddf82862d47941
|
7
|
+
data.tar.gz: 541c9f257e269de04137d6217e8960c680e1864cc9065cd12eb6a04eecf869dc6080ea184ac12c8d45e0f24ae589339aaddf99ea47c4020b2d3060a0669a7b00
|
data/History.txt
CHANGED
@@ -1,3 +1,109 @@
|
|
1
|
+
== 0.9.8 / ???
|
2
|
+
|
3
|
+
* Fix bug in the stratification algorithm
|
4
|
+
* Fix bug with insertions into stdio inside bootstrap blocks
|
5
|
+
* Improve error message when <~ applied to a collection type that doesn't
|
6
|
+
support it (#316)
|
7
|
+
* Fix rescan logic for Zookeeper-backed collections (#317)
|
8
|
+
* Fix rescan logic for chained negation operators in a single rule
|
9
|
+
* Support + operator for concatenating tuples
|
10
|
+
* More consistent behavior for #==, #eql?, and #hash methods on Bud tuple
|
11
|
+
objects (TupleStruct)
|
12
|
+
* Fix intermittent EM::ConnectionNotBound exceptions during the Bud shutdown
|
13
|
+
sequence
|
14
|
+
* Improve stratification algorithm to be slightly more aggressive; that is, we
|
15
|
+
can place certain rules in an earlier strata than we were previously
|
16
|
+
(partially fixes #277)
|
17
|
+
* Add Rakefile to simplify common development operations (Joel VanderWerf, #324)
|
18
|
+
|
19
|
+
== 0.9.7 / 2013-04-22
|
20
|
+
|
21
|
+
* Avoid raising an exception when Bud is used with Ruby 2.0. There isn't
|
22
|
+
anything special that Bud itself needs to do to support Ruby 2.0, but
|
23
|
+
RubyParser doesn't properly support 2.0 yet. So using 2.0-only syntax in Bud
|
24
|
+
rules is unlikely to work (until RubyParser is updated), but no other problems
|
25
|
+
have been observed.
|
26
|
+
* Reject <= on collections from outside Bloom rules (#289, Aaron Davidson). The
|
27
|
+
previous behavior was error-prone: inserting into a scratch collection via <=
|
28
|
+
was not rejected but the inserted data would immediately be discarded at the
|
29
|
+
beginning of the next tick. Hence, it is simpler to require <+ or <~ for all
|
30
|
+
insertions from outside Bloom rules.
|
31
|
+
* Fix bug in join operators whose qualifiers reference collections defined
|
32
|
+
inside imported modules (#301)
|
33
|
+
* Fix bug in join operators when the same table and column name appears on the
|
34
|
+
LHS of multiple join predicates (#313)
|
35
|
+
* Fix bug in outer joins with multiple predicates (#315)
|
36
|
+
* Fix several bugs in rescan/invalidation logic for tables in the presence of
|
37
|
+
upstream deletions (#303)
|
38
|
+
* Fix regression in file_reader collections (#304)
|
39
|
+
* Improve chaining of notin operators
|
40
|
+
* Optimize join evaluation with multiple predicates
|
41
|
+
* Optimize notin self joins
|
42
|
+
* Improve error reporting for syntax errors on temp collections (#310)
|
43
|
+
* Add Travis CI integration hooks (#300, Josh Rosen)
|
44
|
+
|
45
|
+
Lattices:
|
46
|
+
|
47
|
+
* Rename lmap#apply_monotone to lmap#apply. Also, allow #apply to take either
|
48
|
+
monotone functions or morphisms, and improve error reporting.
|
49
|
+
* Add lmap#filter. This takes an lmap whose values are elements of the lbool
|
50
|
+
lattice and returns an lmap containing only those k/v pairs where the value is
|
51
|
+
true.
|
52
|
+
* More intuitive lattice equi-join syntax (lset#eqjoin). Rather than specifying
|
53
|
+
the join predicate(s) using array indexes, instead allow them to be specified
|
54
|
+
using a hash of field names, as in traditional Bloom joins. This only works
|
55
|
+
when the lset contains Structs.
|
56
|
+
* Remove lset#project, and allow lset#eqjoin to take zero predicates; in the
|
57
|
+
latter case, eqjoin computes the Cartesian product.
|
58
|
+
* Support deletion rules (<-) with lattice values on the RHS
|
59
|
+
|
60
|
+
== 0.9.6 / 2013-02-25
|
61
|
+
|
62
|
+
* Support syntax sugar for initializing lattices (#294). For example, rather
|
63
|
+
than writing "foo <= Bud::MaxLattice.new(2)", you can now just write "foo <=
|
64
|
+
2". Note that, due to a bug in the superator gem, you cannot use this syntax
|
65
|
+
with the <+ operator.
|
66
|
+
* Allow nested lattice values to be sent over channels (#295, patch from Josh
|
67
|
+
Rosen). Lattice values could previously be sent as a field value in a tuple
|
68
|
+
delivered over a channel, but they could not be embedded arbitrarily deeply
|
69
|
+
within tuples (e.g., lattices nested within an array could not previously be
|
70
|
+
sent as a field value).
|
71
|
+
* Support "Comparable" for Bud::Lattice values
|
72
|
+
* Add support for lattices to rebl
|
73
|
+
* Reject attempts to insert into stdio via <+ (#288)
|
74
|
+
* Restore functionality of reading from stdio (i.e., stdin) with MRI 1.9
|
75
|
+
* Improve MRI 1.8 compatibility
|
76
|
+
* Require ruby_parser >= 3.1.0
|
77
|
+
* Fix bug in bootstrap blocks for lattice values
|
78
|
+
* Fix bug in rescan/invalidation for rules that reference lattice values and use
|
79
|
+
the <+ or <~ operators (#290)
|
80
|
+
* Fix bug in rescan logic for notin operator (#291)
|
81
|
+
* Fix bug in notin operators involving self joins (#298)
|
82
|
+
* Fix error when output from a notin operator was fed into a lattice
|
83
|
+
|
84
|
+
== 0.9.5 / 2012-11-24
|
85
|
+
|
86
|
+
* Lattice branch (Bloom^L) merged
|
87
|
+
* Compatibility with recent versions of ruby_parser (3.0.2+) and ruby2ruby
|
88
|
+
(2.0.1+). Older versions of these two gems are no longer supported
|
89
|
+
* Add support for aggregate functions that take multiple input columns
|
90
|
+
* Add built-in aggregate function accum_pair(x, y), which produces a Set of
|
91
|
+
pairs (two-element arrays [x,y])
|
92
|
+
* Support user-specified code blocks in payloads(), argagg(), argmin() and
|
93
|
+
argmax()
|
94
|
+
* Change behavior of BudChannel#payloads for channels with two
|
95
|
+
columns. Previously we returned a single *column* (scalar) value in this case;
|
96
|
+
now we always return a tuple with k-1 columns
|
97
|
+
* More consistent behavior for BudCollection#sort when used outside Bloom
|
98
|
+
programs
|
99
|
+
* Restore support for each_with_index() over Bud collections
|
100
|
+
* Restore functionality of Zookeeper-backed Bud collections and fix
|
101
|
+
incompatibility with recent (> 0.4.4) versions of the Zookeeper gem
|
102
|
+
* Optimize parsing of Bloom statements, particularly for large Bloom programs
|
103
|
+
* Fix bug in argagg state materialization
|
104
|
+
* Fix bug in chaining argmin() or argmax() expressions
|
105
|
+
* Fix bug in chaining notin() expressions
|
106
|
+
|
1
107
|
== 0.9.4 / 2012-09-06
|
2
108
|
|
3
109
|
* Optimize grouping performance
|
data/README.md
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
[](https://travis-ci.org/bloom-lang/bud)
|
1
2
|
# Bud
|
2
3
|
|
3
4
|
This is Bud, a.k.a. "Bloom Under Development". It is an initial cut at a Bloom
|
@@ -11,11 +12,12 @@ documentation.
|
|
11
12
|
Main deficiencies at this point are:
|
12
13
|
|
13
14
|
- No Ruby constraints: Within Bloom programs the full power of Ruby is also
|
14
|
-
available, including mutable state.
|
15
|
-
|
15
|
+
available, including mutable state. This allows programmers to get outside the
|
16
|
+
Bloom framework and lose cleanliness.
|
16
17
|
|
17
|
-
- Compatibility: Bud only works with Ruby (MRI) 1.8.7 and 1.9.
|
18
|
-
Ruby
|
18
|
+
- Compatibility: Bud only works with Ruby (MRI) 1.8.7 and 1.9. Bud also has
|
19
|
+
experimental support for Ruby 2.0. JRuby and other Ruby implementations are
|
20
|
+
currently not supported.
|
19
21
|
|
20
22
|
## Installation
|
21
23
|
|
data/Rakefile
ADDED
@@ -0,0 +1,91 @@
|
|
1
|
+
require 'rake'
|
2
|
+
require 'rake/testtask'
|
3
|
+
|
4
|
+
PRJ = "bud"
|
5
|
+
|
6
|
+
def version
|
7
|
+
@version ||= begin
|
8
|
+
$LOAD_PATH.unshift 'lib'
|
9
|
+
require 'bud/version'
|
10
|
+
warn "Bud::VERSION not a string" unless Bud::VERSION.kind_of? String
|
11
|
+
Bud::VERSION
|
12
|
+
end
|
13
|
+
end
|
14
|
+
|
15
|
+
def tag
|
16
|
+
@tag ||= "v#{version}"
|
17
|
+
end
|
18
|
+
|
19
|
+
desc "Run all tests"
|
20
|
+
task :test => "test:unit"
|
21
|
+
|
22
|
+
TESTS = FileList["test/tc_*.rb"]
|
23
|
+
SLOW_TESTS = %w{ test/tc_execmodes.rb }
|
24
|
+
|
25
|
+
namespace :test do
|
26
|
+
desc "Run unit tests"
|
27
|
+
Rake::TestTask.new :unit do |t|
|
28
|
+
t.libs << "lib"
|
29
|
+
t.ruby_opts = %w{ -C test }
|
30
|
+
t.test_files = TESTS.sub('test/', '')
|
31
|
+
### it would be better to make each tc_*.rb not depend on pwd
|
32
|
+
end
|
33
|
+
|
34
|
+
desc "Run quick unit tests"
|
35
|
+
Rake::TestTask.new :quick do |t|
|
36
|
+
t.libs << "lib"
|
37
|
+
t.ruby_opts = %w{ -C test }
|
38
|
+
t.test_files = TESTS.exclude(*SLOW_TESTS).sub('test/', '')
|
39
|
+
end
|
40
|
+
|
41
|
+
desc "Run quick non-zk unit tests"
|
42
|
+
Rake::TestTask.new :quick_no_zk do |t|
|
43
|
+
t.libs << "lib"
|
44
|
+
t.ruby_opts = %w{ -C test }
|
45
|
+
t.test_files = TESTS.
|
46
|
+
exclude('test/tc_zookeeper.rb').
|
47
|
+
exclude(*SLOW_TESTS).
|
48
|
+
sub('test/', '')
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
desc "Commit, tag, and push repo; build and push gem"
|
53
|
+
task :release => "release:is_new_version" do
|
54
|
+
require 'tempfile'
|
55
|
+
|
56
|
+
sh "gem build #{PRJ}.gemspec"
|
57
|
+
|
58
|
+
file = Tempfile.new "template"
|
59
|
+
begin
|
60
|
+
file.puts "release #{version}"
|
61
|
+
file.close
|
62
|
+
sh "git commit --allow-empty -a -v -t #{file.path}"
|
63
|
+
ensure
|
64
|
+
file.close unless file.closed?
|
65
|
+
file.unlink
|
66
|
+
end
|
67
|
+
|
68
|
+
sh "git tag #{tag}"
|
69
|
+
sh "git push"
|
70
|
+
sh "git push --tags"
|
71
|
+
|
72
|
+
sh "gem push #{tag}.gem"
|
73
|
+
end
|
74
|
+
|
75
|
+
namespace :release do
|
76
|
+
desc "Diff to latest release"
|
77
|
+
task :diff do
|
78
|
+
latest = `git describe --abbrev=0 --tags --match 'v*'`.chomp
|
79
|
+
sh "git diff #{latest}"
|
80
|
+
end
|
81
|
+
|
82
|
+
desc "Log to latest release"
|
83
|
+
task :log do
|
84
|
+
latest = `git describe --abbrev=0 --tags --match 'v*'`.chomp
|
85
|
+
sh "git log #{latest}.."
|
86
|
+
end
|
87
|
+
|
88
|
+
task :is_new_version do
|
89
|
+
abort "#{tag} exists; update version!" unless `git tag -l #{tag}`.empty?
|
90
|
+
end
|
91
|
+
end
|
data/bin/budlabel
ADDED
@@ -0,0 +1,63 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
require 'rubygems'
|
4
|
+
require 'bud'
|
5
|
+
require 'getopt/std'
|
6
|
+
require 'bud/labeling/labeling'
|
7
|
+
require 'bud/labeling/bloomgraph'
|
8
|
+
require 'bud/labeling/budplot_style'
|
9
|
+
|
10
|
+
$LOAD_PATH.unshift(".")
|
11
|
+
|
12
|
+
@opts = Getopt::Std.getopts("r:i:p:O:CP")
|
13
|
+
|
14
|
+
unless @opts["r"] and @opts["i"]
|
15
|
+
puts "USAGE:"
|
16
|
+
puts "-r REQUIRE"
|
17
|
+
puts "-i INCLUDE"
|
18
|
+
puts "[-p INCLUDE PATH]"
|
19
|
+
puts "[-O <FMT> Output a graphviz representation of the module in FMT format (pdf if not specified)."
|
20
|
+
puts "-C Concise output -- Associate a single label with each output interface"
|
21
|
+
puts "-P Path-based output -- For each output interface, attribute a label to paths from each input interface"
|
22
|
+
exit
|
23
|
+
end
|
24
|
+
|
25
|
+
hreadable = {
|
26
|
+
"D" => "Diffluent: Nondeterministic output contents.",
|
27
|
+
"A" => "Asynchronous. Nondeterministic output orders.",
|
28
|
+
"N" => "Nonmonotonic. Output contents are sensitive to input orders.",
|
29
|
+
"Bot" => "Monotonic. Order-insensitive and retraction-free."
|
30
|
+
}
|
31
|
+
|
32
|
+
if @opts["p"]
|
33
|
+
$LOAD_PATH.unshift @opts["p"]
|
34
|
+
end
|
35
|
+
|
36
|
+
require @opts["r"]
|
37
|
+
c = Label.new(@opts["i"])
|
38
|
+
|
39
|
+
puts "--- Report for module #{@opts["i"]} ---"
|
40
|
+
|
41
|
+
if @opts["C"]
|
42
|
+
puts "---------------"
|
43
|
+
puts "Output\t\tLabel"
|
44
|
+
puts "---------------"
|
45
|
+
c.output_report.each_pair do |k, v|
|
46
|
+
puts [k, hreadable[v]].join("\t")
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
if @opts["P"]
|
51
|
+
c.path_report.each_pair do |output, inpaths|
|
52
|
+
puts ""
|
53
|
+
puts "--------------------"
|
54
|
+
puts "Output\tInput\tLabel"
|
55
|
+
puts "--------------------"
|
56
|
+
puts output
|
57
|
+
inpaths.each_pair do |inp, lbl|
|
58
|
+
puts "\t#{inp}\t#{hreadable[lbl]}"
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
62
|
+
|
63
|
+
c.write_graph(@opts["O"]) if @opts["O"]
|
data/bin/budplot
CHANGED
@@ -21,6 +21,10 @@ def make_instance(mods)
|
|
21
21
|
# If we're given a single identifier that names a class, just return an
|
22
22
|
# instance of that class. Otherwise, define a bogus class that includes all
|
23
23
|
# the module names specified by the user and return an instance.
|
24
|
+
tmpserver = TCPServer.new('127.0.0.1', 0) # get a free port
|
25
|
+
default_params = {:dbm_dir => "/tmp/budplot_dbm_" + SecureRandom.uuid.to_s, :port => tmpserver.addr[1]}
|
26
|
+
|
27
|
+
|
24
28
|
mods.each do |m|
|
25
29
|
unless is_constant? m
|
26
30
|
puts "Error: unable to find definition for module or class \"#{m}\""
|
@@ -30,7 +34,7 @@ def make_instance(mods)
|
|
30
34
|
mod_klass = eval m
|
31
35
|
if mod_klass.class == Class
|
32
36
|
if mods.length == 1
|
33
|
-
return mod_klass.new
|
37
|
+
return mod_klass.new(default_params)
|
34
38
|
else
|
35
39
|
puts "Error: cannot intermix class \"#{mod_klass}\" with modules"
|
36
40
|
exit
|
@@ -50,7 +54,7 @@ def make_instance(mods)
|
|
50
54
|
]
|
51
55
|
class_def = def_lines.flatten.join("\n")
|
52
56
|
eval(class_def)
|
53
|
-
f =FooBar.new
|
57
|
+
f = FooBar.new(default_params)
|
54
58
|
3.times{ f.tick }
|
55
59
|
f
|
56
60
|
end
|
@@ -77,8 +81,8 @@ def process(mods)
|
|
77
81
|
d = make_instance(mods)
|
78
82
|
|
79
83
|
interfaces = {}
|
80
|
-
d.t_provides.each do |
|
81
|
-
interfaces[
|
84
|
+
d.t_provides.to_a.each do |prov|
|
85
|
+
interfaces[prov.interface] = prov.input
|
82
86
|
end
|
83
87
|
|
84
88
|
tabinf = {}
|
@@ -201,13 +205,18 @@ def get_trace_data
|
|
201
205
|
end
|
202
206
|
|
203
207
|
if ARGV.length < 2
|
204
|
-
puts "Usage: budplot LIST_OF_FILES LIST_OF_MODULES_OR_CLASSES"
|
208
|
+
puts "Usage: budplot [-I PATH_TO_RUBY_INCLUDES] LIST_OF_FILES LIST_OF_MODULES_OR_CLASSES"
|
205
209
|
exit
|
206
210
|
end
|
207
211
|
|
208
|
-
@opts = Getopt::Std.getopts("t:")
|
209
|
-
|
210
|
-
|
212
|
+
@opts = Getopt::Std.getopts("I:t:")
|
213
|
+
unless @opts["I"].nil?
|
214
|
+
if @opts["I"].class == Array
|
215
|
+
@opts["I"].each{|i| $:.unshift i}
|
216
|
+
else
|
217
|
+
$:.unshift @opts["I"]
|
218
|
+
end
|
219
|
+
end
|
211
220
|
|
212
221
|
@data = get_trace_data
|
213
222
|
`mkdir bud_doc`
|
@@ -215,6 +224,7 @@ end
|
|
215
224
|
modules = []
|
216
225
|
ARGV.each do |arg|
|
217
226
|
if File.exists? arg
|
227
|
+
arg = File.expand_path arg
|
218
228
|
require arg
|
219
229
|
else
|
220
230
|
modules << arg
|
data/bin/budtimelines
CHANGED
@@ -10,7 +10,7 @@ include VizUtil
|
|
10
10
|
|
11
11
|
module Depends
|
12
12
|
state do
|
13
|
-
table :depends, [:rid, :lhs, :op, :rhs, :nm]
|
13
|
+
table :depends, [:bud_obj, :rid, :lhs, :op, :rhs, :nm, :in_body]
|
14
14
|
end
|
15
15
|
end
|
16
16
|
|
@@ -49,7 +49,7 @@ end
|
|
49
49
|
module DeltaLogic
|
50
50
|
include TPSchema
|
51
51
|
bloom do
|
52
|
-
zerod_cards <= cardinalities{|c| c + [c.bud_time-1]}
|
52
|
+
zerod_cards <= cardinalities{|c| c.to_a + [c.bud_time-1]}
|
53
53
|
zerod_cards <= (times * depends).pairs do |t, d|
|
54
54
|
unless cardinalities{|c| c[1] if c[0] == t.bud_time}.include? d[1]
|
55
55
|
[t.bud_time, d[1], 0, t.bud_time - 1]
|
data/bin/budvis
CHANGED
@@ -28,7 +28,13 @@ usage unless ARGV[0]
|
|
28
28
|
usage if ARGV[0] == '--help'
|
29
29
|
|
30
30
|
meta, data = get_meta2(BUD_DBM_DIR)
|
31
|
-
|
31
|
+
|
32
|
+
# prune outbufs from tabinf
|
33
|
+
tabinf = meta[:tabinf].find_all do |k|
|
34
|
+
!(k[1] == "Bud::BudChannel" and k[0] =~ /_snd\z/)
|
35
|
+
end
|
36
|
+
|
37
|
+
vh = VizHelper.new(tabinf, meta[:cycle], meta[:depends], meta[:rules], ARGV[0], meta[:provides])
|
32
38
|
data.each do |d|
|
33
39
|
vh.full_info << d
|
34
40
|
end
|
data/docs/README.md
CHANGED
@@ -6,28 +6,19 @@ Welcome to the documentation for *Bud*, a prototype of Bloom under development.
|
|
6
6
|
The documents here are organized to be read in any order, but you might like to
|
7
7
|
try the following:
|
8
8
|
|
9
|
-
* [intro.md
|
10
|
-
* [getstarted.md
|
9
|
+
* [intro](intro.md): A brief introduction to Bud and Bloom.
|
10
|
+
* [getstarted](getstarted.md): A quickstart to teach you basic Bloom
|
11
11
|
concepts, the use of `rebl` interactive terminal, and the embedding of Bloom
|
12
12
|
code in Ruby via the `Bud` module.
|
13
|
-
* [operational.md
|
13
|
+
* [operational](operational.md): An operational view of Bloom, to provide
|
14
14
|
a more detailed model of how Bloom code is evaluated by Bud.
|
15
|
-
* [cheat.md
|
16
|
-
* [modules.md
|
17
|
-
* [ruby\_hooks.md
|
15
|
+
* [cheat](cheat.md): Full documentation of the language constructs in a concise "cheat sheet" style.
|
16
|
+
* [modules](modules.md): An overview of Bloom's modularity features.
|
17
|
+
* [ruby_hooks](ruby\_hooks.md): Bud module methods that allow you to
|
18
18
|
interact with the Bud evaluator from other Ruby threads.
|
19
|
-
* [visualizations.md
|
19
|
+
* [visualizations](visualizations.md): Overview of the `budvis` and
|
20
20
|
`budplot` tools for visualizing Bloom program analyses.
|
21
|
-
* [bfs.md
|
22
|
-
|
23
|
-
[intro]: /bloom-lang/bud/blob/master/docs/intro.md
|
24
|
-
[getstarted]: /bloom-lang/bud/blob/master/docs/getstarted.md
|
25
|
-
[operational]: /bloom-lang/bud/blob/master/docs/operational.md
|
26
|
-
[cheat]: /bloom-lang/bud/blob/master/docs/cheat.md
|
27
|
-
[modules]: /bloom-lang/bud/blob/master/docs/modules.md
|
28
|
-
[ruby_hooks]: /bloom-lang/bud/blob/master/docs/ruby_hooks.md
|
29
|
-
[visualizations]: /bloom-lang/bud/blob/master/docs/visualizations.md
|
30
|
-
[bfs]: /bloom-lang/bud/blob/master/docs/bfs.md
|
21
|
+
* [bfs](bfs.md): A walkthrough of the Bloom distributed filesystem.
|
31
22
|
|
32
23
|
In addition, the [bud-sandbox](http://github.com/bloom-lang/bud-sandbox) GitHub
|
33
24
|
repository contains lots of useful libraries and example programs built using
|
data/docs/cheat.md
CHANGED
@@ -21,15 +21,23 @@ As in Ruby, backslash is used to escape a newline.<br>
|
|
21
21
|
end
|
22
22
|
|
23
23
|
## State Declarations ##
|
24
|
-
|
25
|
-
|
26
|
-
|
24
|
+
|
25
|
+
A `state` block contains definitions of two kinds of program state:
|
26
|
+
*collections* and *lattices*. A Bud collection is a *set* of *facts*; each fact
|
27
|
+
is an array of Ruby values. Note that collections do not contain duplicates
|
28
|
+
(inserting a duplicate fact into a collection is ignored).
|
27
29
|
|
28
30
|
Like a table in a relational database, a subset of the columns in a collection
|
29
31
|
makeup the collection's _key_. Attempting to insert two facts into a collection
|
30
32
|
that agree on the key columns (but are not duplicates) results in a runtime
|
31
33
|
exception.
|
32
34
|
|
35
|
+
A lattice represents a value that *grows* over time, where the definition of
|
36
|
+
"growth" depends on the kind of lattice in question. For example, an `lset`
|
37
|
+
lattice contains a set of facts that grows over time (similar to a traditional
|
38
|
+
Bud collection), whereas an `lmax` lattice holds an increasing integer
|
39
|
+
value. For more information on lattices, see the section below.
|
40
|
+
|
33
41
|
### Default Declaration Syntax ###
|
34
42
|
*BudCollection :name, [keys] => [values]*
|
35
43
|
|
@@ -186,7 +194,7 @@ implicit map:
|
|
186
194
|
end
|
187
195
|
|
188
196
|
## BudCollection-Specific Methods ##
|
189
|
-
`bc.schema`: returns the schema of `bc` (Hash of key column names => non-key column names). Note that for channels, this omits the location specifier (<tt>@</tt>).<br>
|
197
|
+
`bc.schema`: returns the schema of `bc` (Hash of key column names => non-key column names; if no non-key columns, just an Array of key column names). Note that for channels, this omits the location specifier (<tt>@</tt>).<br>
|
190
198
|
|
191
199
|
`bc.cols`: returns the column names in `bc` as an Array<br>
|
192
200
|
|
@@ -224,14 +232,14 @@ Output the facts in `bc` that do not appear in `bc2`, as follows. First, we form
|
|
224
232
|
:attr2`).
|
225
233
|
|
226
234
|
2. If a code block is specified, invoke the block on every pair of matching
|
227
|
-
tuples in the join result. Any matches for which the block returns `
|
235
|
+
tuples in the join result. Any matches for which the block returns `false`
|
228
236
|
are removed from `t`.
|
229
237
|
|
230
238
|
Finally, we output every tuple of `bc` that does *not* appear in `t`.
|
231
239
|
|
232
240
|
# output items from foo if (a) there is no matching key in bar, or
|
233
241
|
# (b) all matching keys in bar have a smaller value
|
234
|
-
stdio <~ foo.notin(bar, :key=>:key) {|f, b|
|
242
|
+
stdio <~ foo.notin(bar, :key=>:key) {|f, b| f.val <= b.val}
|
235
243
|
|
236
244
|
|
237
245
|
## SQL-style grouping/aggregation (and then some) ##
|
@@ -239,18 +247,12 @@ Finally, we output every tuple of `bc` that does *not* appear in `t`.
|
|
239
247
|
* `bc.group([:col1, :col2], min(:col3))`. *akin to min(col3) GROUP BY col1,col2*
|
240
248
|
* exemplary aggs: `min`, `max`, `bool_and`, `bool_or`, `choose`
|
241
249
|
* summary aggs: `sum`, `avg`, `count`
|
242
|
-
* structural aggs: `accum` *accumulates inputs into a Set*
|
250
|
+
* structural aggs: `accum`, `accum_pair` *accumulates inputs into a Set; accum_pair takes two inputs and accumulates a Set of pairs (two element arrays)*
|
243
251
|
* `bc.argmax([:attr1], :attr2)` *returns the bc items per attr1 that have highest attr2*
|
244
252
|
* `bc.argmin([:attr1], :attr2)`
|
245
253
|
* `bc.argagg(:exemplary_agg_name, [:attr1], :attr2))`. *generalizes argmin/max: returns the bc items per attr1 that are chosen by the exemplary
|
246
254
|
aggregate named*
|
247
255
|
|
248
|
-
### Built-in Aggregates: ###
|
249
|
-
|
250
|
-
* Exemplary aggs: `min`, `max`, `choose`
|
251
|
-
* Summary aggs: `count`, `sum`, `avg`
|
252
|
-
* Structural aggs: `accum`
|
253
|
-
|
254
256
|
Note that custom aggregation can be written using `reduce`.
|
255
257
|
|
256
258
|
## Collection Combination (Join) ###
|
@@ -313,6 +315,103 @@ The schema of a temp collection in inherited from the rhs; if the rhs has no
|
|
313
315
|
schema, a simple one is manufactured to suit the data found in the rhs at
|
314
316
|
runtime: `[c0, c1, ...]`.
|
315
317
|
|
318
|
+
## Lattices ##
|
319
|
+
|
320
|
+
In addition to traditional Bud collections and relational-style statements that
|
321
|
+
operate over collections, Bud also supports lattices and rules that operate over
|
322
|
+
lattices. Lattices provide a way to represent values that *grow over time*,
|
323
|
+
where the definition of "growth" depends on the kind of lattice. The following
|
324
|
+
built-in lattice types are currently supported:
|
325
|
+
|
326
|
+
<table>
|
327
|
+
<tr>
|
328
|
+
<td><b>Name</b></td>
|
329
|
+
<td><b>Description</b></td>
|
330
|
+
<td><b>Initial Value</b></td>
|
331
|
+
<td><b>Monotone Functions</b></td>
|
332
|
+
</tr>
|
333
|
+
|
334
|
+
<tr>
|
335
|
+
<td><code>lbool</code></td>
|
336
|
+
<td>Threshold test (<code>false</code> => <code>true</code> conditional)</td>
|
337
|
+
<td>false</td>
|
338
|
+
<td>when_true</td>
|
339
|
+
</tr>
|
340
|
+
|
341
|
+
<tr>
|
342
|
+
<td><code>lmax</code></td>
|
343
|
+
<td>Increasing numeric value</td>
|
344
|
+
<td>-∞</td>
|
345
|
+
<td>gt(n), gt_eq(n), +(n), -(n)</td>
|
346
|
+
</tr>
|
347
|
+
|
348
|
+
<tr>
|
349
|
+
<td><code>lmin</code></td>
|
350
|
+
<td>Decreasing numeric value</td>
|
351
|
+
<td>+∞</td>
|
352
|
+
<td>lt(n), lt_eq(n), +(n), -(n)</td>
|
353
|
+
</tr>
|
354
|
+
|
355
|
+
<tr>
|
356
|
+
<td><code>lset</code></td>
|
357
|
+
<td>Growing set of values</td>
|
358
|
+
<td>empty set</td>
|
359
|
+
<td>contains?, eqjoin, filter, intersect, product, project, size</td>
|
360
|
+
</tr>
|
361
|
+
|
362
|
+
<tr>
|
363
|
+
<td><code>lpset</code></td>
|
364
|
+
<td>Growing set of non-negative numeric values</td>
|
365
|
+
<td>empty set</td>
|
366
|
+
<td>contains?, eqjoin, filter, intersect, product, project, size, sum</td>
|
367
|
+
</tr>
|
368
|
+
|
369
|
+
<tr>
|
370
|
+
<td><code>lbag</code></td>
|
371
|
+
<td>Growing multiset of values</td>
|
372
|
+
<td>empty multiset</td>
|
373
|
+
<td>contains?, multiplicity, intersect, product, project, size</td>
|
374
|
+
</tr>
|
375
|
+
|
376
|
+
<tr>
|
377
|
+
<td><code>lmap</code></td>
|
378
|
+
<td>Map from keys to lattice values</td>
|
379
|
+
<td>empty map</td>
|
380
|
+
<td>at, intersect, key?, key_set, project, size</td>
|
381
|
+
</tr>
|
382
|
+
</table>
|
383
|
+
|
384
|
+
Lattices can be declared in `state` blocks in a similar manner to traditional
|
385
|
+
Bud collections. Similarly, Bloom rules can invoke functions on lattice
|
386
|
+
values. A simple Bloom program that uses lattices to compute a quorum vote is as
|
387
|
+
follows:
|
388
|
+
|
389
|
+
```ruby
|
390
|
+
QUORUM_SIZE = 5
|
391
|
+
RESULT_ADDR = "example.org"
|
392
|
+
|
393
|
+
class QuorumVote
|
394
|
+
include Bud
|
395
|
+
|
396
|
+
state do
|
397
|
+
channel :vote_chn, [:@addr, :voter_id]
|
398
|
+
channel :result_chn, [:@addr]
|
399
|
+
lset :votes
|
400
|
+
lmax :vote_cnt
|
401
|
+
lbool :vote_done
|
402
|
+
end
|
403
|
+
|
404
|
+
bloom do
|
405
|
+
votes <= vote_chn {|v| v.voter_id}
|
406
|
+
vote_cnt <= votes.size
|
407
|
+
got_quorum <= vote_cnt.gt_eq(QUORUM_SIZE)
|
408
|
+
result_chn <~ got_quorum.when_true { [RESULT_ADDR] }
|
409
|
+
end
|
410
|
+
end
|
411
|
+
```
|
412
|
+
|
413
|
+
For more information on lattice support in Bloom, see this [recent paper](http://db.cs.berkeley.edu/papers/socc12-blooml.pdf).
|
414
|
+
|
316
415
|
## Bud Modules ##
|
317
416
|
A Bud module combines state (collections) and logic (Bloom rules). Using modules allows your program to be decomposed into a collection of smaller units.
|
318
417
|
|