gorillib 0.4.2pre → 0.4.2

Sign up to get free protection for your applications and to get access to all the features.
data/notes/HOWTO.md DELETED
@@ -1,22 +0,0 @@
1
- # Walkthrough
2
-
3
- **note (2012-06): you must use the dev branch, `version_1` of Gorillib** -- https://github.com/infochimps-labs/gorillib/tree/version_1/
4
-
5
- 1. Here's a set of examples of Gorillib records in action.
6
- * [examples of `Gorillib::Model`](https://github.com/infochimps-labs/gorillib/tree/version_1/examples/model)
7
- * see example gorillib records in the /examples directories
8
-
9
- __________________________________________________________________________
10
-
11
- 2. Walkthrough dataflow for a new customer:
12
- * models to represent records in a dataflow (gorillib/examples)
13
- * describe the macro dataflow (wukong/examples)
14
- * simulate the dataflow at commandline
15
- * project the dataflow using Flume
16
- * export the dataflow using graphviz
17
-
18
- https://github.com/infochimps-labs/wukong/wiki
19
- https://github.com/infochimps-labs/gorillib/wiki
20
- Plus source files (or you can instead see the Yard Docs)
21
-
22
- 3. flume event model shim
data/notes/bucket.md DELETED
@@ -1,155 +0,0 @@
1
- # gorillib/bucket (WIP -- NOT READY YET) -- freeform storage with definable access
2
-
3
- * A `Go::Bucket` fulfills the contract of `Go::Model`
4
-
5
- ## Arbitrary read/writes with `[]/[]=`, defined access with `.foo/.foo=`
6
-
7
- Overriding `method_missing` is uncouth. It screws up your stack traces, muddies your interface and leads to unassertive code:
8
-
9
- ```ruby
10
- Settings.defcon = 3
11
- # ... elsewhere ...
12
- if Settings.def_con.to_i <= 1
13
- WOPR.launch_nukes!
14
- end
15
- ```
16
-
17
- Here, mispelling `def_con` as `defcon` (along with being wussy about the attribute's type and negligent about prescribing a default value) leads to the destruction of humanity.
18
-
19
- Nonetheless, for several reasons (prudent laziness, handling configuration of an external component, etc) it's important to handle arbitrary attributes uniformly.
20
-
21
- So the rule is that you can get or set anything you like using `[]` and `[]=` respectively, no need to define it first:
22
-
23
- Settings[:whatever] = 3
24
- Settings[:whatever] #=> 3
25
-
26
- You don't get any magic, and the ugly accessor leaves you in no doubt that you're being lazy (ain't judging, just sayin'). When you define a setting you get accessors for that attribute and all associated magic:
27
-
28
- Settings.option :defcon, Integer, :doc => 'Current NORAD defense condition', :default => 5, :validates => { :in => 1..5 }
29
- Settings.defcon #=> 5
30
- Settings.defcon = 0
31
- Settings.validate! # raises a validation exception
32
-
33
- Defined fields' magic works whichever form of access you use -- here, type-converting the value on assignment:
34
-
35
- Settings[:defcon] = '5' #=> 5
36
- Settings.defcon = '5' #=> 5
37
- Settings.receive_defcon('5') #=> 5
38
-
39
- ## Keys
40
-
41
- Keys be lower-cased identifiers: they should match `/\A([a-z][a-z0-9\_]*)\z/`.
42
-
43
- ## Deep Hash
44
-
45
- Hash.new {|h,k| h[k] = Hash.new(&h.default_proc)}
46
-
47
- ### Dot addressing
48
-
49
- Can retrieve keys as 'x.y.z' meaning 'foo[x][y][z]'.
50
-
51
- * On a `get`, will
52
- - return the value, if `foo[:x][:y][:z]` exists
53
- - return nil, if either `foo[:x]` or `foo[:x][:y]` is *unset*. It will not create `foo[:x]` or `foo[:x][:y]`.
54
- - raise an error if either `foo[:x]` or `foo[:x][:y]` is set to a value that does not respond to `[]`
55
-
56
- * On a `set`, will
57
- - raise an error if either `foo[:x]` or `foo[:x][:y]` is set to a value that doesn't respond to `[]`
58
- - set and return the value; any intermediate buckets (`foo[:x]` and `foo[:x][:y]`) will be created if they don't exist.
59
-
60
-
61
- __________________________________________________________________________
62
-
63
-
64
- ## Examples
65
-
66
- ### how hash-like is this?
67
-
68
- * obj.deep_get(:foo, :bar, :baz) #
69
- * obj.deep_set(:foo, :bar, :baz, val) # sets foo bar baz.
70
- - where property has a type, it uses that type
71
- - otherwise it uses Mash, and calls [] on it
72
-
73
- * TD votes NO on magic recursive hash behavior
74
-
75
-
76
- {
77
- :buck1 => {
78
- :buck2 => { :k3 => 33, :ehsh => {}, :arr => [11, 12, 13] },
79
- :cars => [{ :model => 'ford', :cylinders => 8 }, { :model => 'buick', :cylinders => 6 }],
80
- }
81
- :buck3 => {
82
- :buck4 => { :k4 => 44 }
83
- :k5 => nil,
84
- }
85
- :k6 => 69
86
- }
87
-
88
- * `obj[:buck3]` # b{ :buck4=>b{ :k4=>44 },:k5=>nil } -- it's a bucket
89
- * `obj[:buck5] = Hash.new`
90
- * `obj[:buck5].class` # Bucket -- it's converted to bucket
91
- * `obj[:xxx]` # nil -- it's not there
92
- * `obj[:xxx][:yyy][:zzz]` # fails -- it doesn't try to index into the nil `obj[:xxx]` cell.
93
- * `obj[:xxx]` # nil -- it didn't create the `obj[:xxx]` object when we tried to read on the previous line.
94
-
95
- c.options_for 'wheels', 'interior.fabric', 'interior.carpeting'
96
-
97
- c[:wheels] # c{ }
98
- c[:wheels][:whitewall] # nil
99
- c[:wheels][:whitewall] = true # true
100
-
101
- c.interior.fabric.color
102
-
103
-
104
- * `obj[:k6][:bomb]` # raises ArgumentError; `obj[:k6]` is not a bucket.
105
-
106
- * `obj[:'hello-there'] # undefined; `'hello-there'` is not a valid identifier.
107
-
108
- * `obj[:buck3][:buck4][:k5] = 55`
109
- `obj[:buck3]` # b{ :buck4=>b{ :k4=>44 },:k5 => 55}
110
- * `obj[:f][:g][:h] = 7`
111
- `obj[:f]` # b{ :g => b{ :h => 7 } }
112
-
113
- * `obj[:foo][:bar][:baz] ||= 1` -- **hard** ?I think?
114
-
115
- * `obj[:foo]` -- nil
116
- * `obj[:foo][:bar]` -- raise
117
- * `obj[:foo][:bar] = 3` --
118
- - now obj[:foo][:bar] is 3
119
- - and obj[:foo] is a ?dsl_object?? but
120
-
121
- `obj[:foo][:bar]`
122
-
123
- Suppose `obj[:foo]` is set to a
124
-
125
- Seems clear these should do the right thing:
126
-
127
- * `obj.merge`
128
- * `obj.reverse_merge`
129
- * `obj.keys`
130
- * `obj.values`
131
- * ... and a few more
132
-
133
- Also:
134
-
135
- * `obj.to_a`?
136
- * `obj.each`?
137
- * other crazy Enumerable properties?
138
-
139
- ### TODO
140
-
141
- * figure out the method structure for
142
- - read/write/unset of attributes when Hash vs Accessors vs Instance Variables
143
- - reader/writer: raw vs. hooks, dirty, etc.
144
-
145
- ## Configliere Settings
146
-
147
-
148
- Configliere lets you define arbitrary attributes of a param, notably:
149
-
150
- [:description] Documentation for the param, used in the --help message
151
- [:default] Sets a default value (applied immediately)
152
- [:env_var] Environment variable to adopt (applied immediately, and after +:default+)
153
- [:type] Converts param's value to the given type, just before the finally block is called
154
- [:finally] Block of code to postprocess settings or handle complex configuration.
155
- [:required] Raises an error if, at the end of calling resolve!, the param's value is nil.
data/notes/builder.md DELETED
@@ -1,170 +0,0 @@
1
- # gorillib/builder -- construct elegant ruby Domain-Specific Languages (DSLs).
2
-
3
- `Gorillib::Builder` provides foundation models for elegant ruby DSLs (Domain-Specific Languages). A builder block's relaxed ruby syntax enables highly-readable specification of complex behavior:
4
-
5
- ```ruby
6
- garage do
7
- car :ford_tudor do
8
- manufacturer 'Ford Motor Company'
9
- car_model 'Tudor'
10
- year 1939
11
- doors 2
12
- style :sedan
13
- engine do
14
- volume 350
15
- cylinders 8
16
- end
17
- end
18
-
19
- car :wildcat do
20
- manufacturer 'Buick'
21
- car_model 'Wildcat'
22
- year 1968
23
- doors 2
24
- style :convertible
25
- engine do
26
- volume 455
27
- cylinders 8
28
- end
29
- end
30
- end
31
- ```
32
-
33
- ### Defining a builder
34
-
35
- To make a class be a builder, simply `include Gorillib::Builder`:
36
-
37
-
38
- ```ruby
39
- class Garage
40
- include Gorillib::Builder
41
- collection :car
42
- end
43
-
44
- class Car
45
- include Gorillib::Builder
46
- field :name, String
47
- field :manufacturer, String
48
- field :car_model, String
49
- field :year, Integer
50
- field :doors, Integer
51
- field :style, Symbol, :validates => { :inclusion_in => [:sedan, :coupe, :convertible] }
52
- member :engine
53
- belongs_to :garage
54
- end
55
-
56
- class Engine
57
- include Gorillib::Builder
58
- field :volume, Integer
59
- field :cylinders, Integer
60
- belongs_to :car
61
- end
62
- ```
63
-
64
- ### getset accessors
65
-
66
- Fields of a Builder class create a single "getset" accessor (and not the familiar 'foo/foo=' pair):
67
-
68
- * `car_model.foo` -- returns value of foo
69
- * `car_model.foo(val)` -- sets foo to `val`, which can be any value, even `nil`.
70
-
71
- ### member fields
72
-
73
- A builder class can have `member` fields; the type of a member field should be a builder class itself
74
-
75
- With no arguments, the accessor returns the value of engine attribute, creating if necessary:
76
-
77
- ```ruby
78
- car.engine #=> #<Engine volume=~ cylinders=~>
79
- ```
80
-
81
- If you pass in a hash of values, the engine is created if necessary and then asked to `receive!` the hash.
82
-
83
- ```ruby
84
- car.engine #=> #<Engine volume=~ cylinders=~>
85
- car.engine(:cylinders => 8) #=> #<Engine volume=~ cylinders=8>
86
- car.engine.cylinders #=> 8
87
- ```
88
-
89
- If you provide a no-args block, it is `instance_eval`ed in the context of the member object:
90
-
91
- ```ruby
92
- car :ford_tudor do
93
- engine(:cylinders => 8) do
94
- self #=> #<Engine volume=~ cylinders=8>
95
- volume 455
96
- cylinders self.cylinders - 2
97
- self #=> #<Engine volume=455 cylinders=6>
98
- end
99
- engine #=> #<Engine volume=455 cylinders=6>
100
- end
101
- ```
102
-
103
- Some people disapprove of `instance_eval`, as they consider it unseemly to mess around with `self`. If you instead provide a one-arg block, the member object is passed in:
104
-
105
- ```ruby
106
- car :ford_tudor do |c|
107
- c.engine(:cylinders => 8) do |eng|
108
- self #=> #<Car ...>
109
- eng.volume 455
110
- eng.cylinders eng.cylinders - 2
111
- eng #=> #<Engine volume=455 cylinders=6>
112
- end
113
- c.engine #=> #<Engine volume=455 cylinders=6>
114
- end
115
- ```
116
-
117
- ### collections
118
-
119
- A builder class can also have `collection` fields, to contain named builder objects.
120
-
121
- ```ruby
122
- garage do
123
- car(:ford_tudor) #=> #<Car name="ford tudor" ...>
124
- cars #=> { :ford_tudor => #<Car ...>, :wildcat => #<Car ...> }
125
- end
126
- ```
127
-
128
- The collected items must respond to `id` (FIXME:). The singular accessor accepts arguments just like a `member` accessor:
129
-
130
- ```ruby
131
- garage do
132
- car(:ford_tudor, :year => 1939)
133
- #=> #<Car name=`<:ford_tudor year=1939 doors=~ ...>
134
- car(:ford_tudor, :year => 1939) do
135
- doors 2
136
- style :convertible
137
- end
138
- #=> #<Car name="ford tudor" year=1939 doors=2 ...>
139
- car(:wildcat, :year => 1968) do |c|
140
- c.doors 2
141
- c.style :convertible
142
- end
143
- #=> #<Car name= year=1939 doors=2 ...>
144
-
145
-
146
-
147
-
148
- ### Model methods
149
-
150
- builders have the following:
151
-
152
- * `Model::Defaults`
153
- * `Model::Naming`
154
- * `Model::Conversion`
155
-
156
-
157
-
158
- ### SubclassRegistry
159
-
160
- When a subclass happens, decorates class with `saucepot(...)`, equivalent to
161
-
162
- update_or_create
163
- registers
164
-
165
- * At class level, `registry_for(CookingUtensil)` gives
166
- - `add_cooking_utensil()` and so forth for adding and getting
167
- - Protected `cooking_utensils` hash class attribute
168
- * Enlisting a resource class (Kitchen.register(FryingPan) and gives you a magic `frying_pan` factory method with signature `frying_pan(name, *args, &block)`
169
- - If resource does not exist, calls `FryingPan.new` with full set of params and block. Add it to `cooking_utensils` registry.
170
- - If resource with that name exists, retrieve it. call `merge()` with the parameters, and `run_in_scope` with the block.
data/notes/collection.md DELETED
@@ -1,81 +0,0 @@
1
- # gorillib/collection -- associative arrays of keyed objects
2
-
3
- Collection flexibly exchanges `{name => obj}` and `[ obj_with_name, obj_with_name, ... ]` -- this makes Mongo and JSON happy.
4
-
5
- ### Collection type
6
-
7
- Defining
8
-
9
- ```ruby
10
- class Continent
11
- include Gorillib::Record
12
- # ...
13
- field :countries, Collection, :of => Geo::Country, :key_method => :country_id
14
- end
15
- ```
16
-
17
- Lets it serialize as
18
-
19
- ```yaml
20
- - name: Africa
21
- countries:
22
- - name: Rwanda
23
- country_id: rw
24
- capitol_city: Kigali
25
- - name: Djibouti
26
- country_id: dj
27
- capitol_city: Djibouti
28
- ```
29
-
30
- or naturally pivot to be
31
-
32
- ```ruby
33
- { name: "Africa",
34
- countries: {
35
- rw: {
36
- name: "Rwanda",
37
- country_id: "rw",
38
- capitol_city: "Kigali" },
39
- dj: {
40
- name: "Djibouti",
41
- country_id: "dj",
42
- capitol_city: "Djibouti" },
43
- }
44
- }
45
- ```
46
-
47
- Calling `Collection.receive` works on either representation, and calling `to_a` or `to_hash` does what you'd think:
48
-
49
- ```ruby
50
- africa.countries.to_a # [ <Geo::Country name="Rwanda"...>, <Geo::Country name="Djibouti"...> ]
51
- africa.countries.to_hash # { :rw => <Geo::Country name="Rwanda"...>, :dj => <Geo::Country name="Djibouti"...> }
52
- ```
53
-
54
- `Collection` proxies a small set of methods to an internal Hash. We purposefully keep you from calling methods that work differently on Hash and Array -- most notably, there is no `each` method, and it is currently *not* an Enumerable.
55
-
56
- * `each_pair` iterates over the key/value pairs
57
- * `each_value` iterates over the values
58
- * `to_a` gives the list of values
59
- * `to_hash` gives a duplicate of the proxied hash
60
- * `inspect`, `to_s`, `as_json` and `to_json` show a list of values -- it serializes like an array
61
-
62
- * `merge!`, `concat` or `receive!` add values in-place, and `merge` to a duplicate, as follows:
63
- - if the given collection responds_to `to_hash`, it is merged into the internal collection; each hash key *must* match the id of its value or results are undefined.
64
- - otherwise, it merges a hash generates from the id/value pairs of each object in the given collection.
65
-
66
- * `[]=` adds a single value with a specified key; that key *must* match the value's id or it is undefined behavior.
67
- * `<<` adds a single value at the right key.
68
-
69
- * `build(attributes={})`
70
-
71
- ## advanced use
72
-
73
- * `key_method` (read-only class attribute, a Symbol) defines how to find the id; by default, `:id`
74
- * `factory` (read-only class attribute) defines how to create a new item given its raw material. This is handed to `Gorillib::Factory` for conversion the first time it's read.
75
-
76
- ## Decisions
77
-
78
- ## Mysteries
79
-
80
- * ?? no `each` method
81
- * ?? `Enumerable`
data/notes/factories.md DELETED
@@ -1,86 +0,0 @@
1
- # gorillib/factories (WIP -- NOT READY YET) -- efficient, predictable type conversion
2
-
3
- * Is a missing value unset? or nil?
4
-
5
- * (Boolean)
6
- * String Symbol
7
- * Regexp
8
- * Bignum Integer Numeric
9
- * Float (Double)
10
- * Complex Rational
11
- * Random
12
- * Time
13
- * Date
14
-
15
- * Dir File Pathname
16
- * Whatever -- alias for IdentityFactory
17
-
18
- * Method, Proc
19
- * Range
20
-
21
- * Hash
22
- * Array
23
-
24
- * Object
25
- * Class Module FalseClass TrueClass NilClass
26
-
27
- These don't have factories because I can't really think of a reasonable use case
28
- * DateTime Struct MatchData UnboundMethod IO Enumerator Fixnum Object File::Stat StringIO
29
-
30
- These are *not* decorated because it's not a good idea
31
- * BasicObject
32
- * Exception Interrupt SignalException SystemExit
33
- * Encoding Data Fiber Mutex ThreadGroup Thread Binding Queue
34
-
35
- # name produces convert receivable? converted? nil "" [] {} false true
36
- :Integer, Fixnum, :to_i, :to_i, is_a?(Float), nil nil, err, err
37
- :Bignum
38
- :Float, Float, :to_f, :to_f, is_a?(Fixnum), nil nil, err, err
39
- :Complex
40
- :Rational
41
-
42
- # :Random
43
- # :Long
44
- # :Double
45
- # :Numeric
46
-
47
- :String, String, :to_s, :to_s, is_a?(String), nil, "", ??, ??
48
- :Binary, Binary, :to_s, :to_s, is_a?(String), nil, "", ??, ??
49
- :Symbol, Symbol, :to_s, :to_s, is_a?(String), nil, nil, ??, ??
50
- :Regexp,
51
- :Time
52
- # :Date
53
- # :Dir
54
- # :File
55
- # :Pathname
56
-
57
- :Identity
58
- :Whatever
59
-
60
- :Class
61
- :Module
62
- :TrueClass, true,
63
- :FalseClass, false,
64
- :NilClass, nil,
65
- :Boolean, [true,false], ??, [true, false], [true, false], nil, nil, ??, ??
66
-
67
- :Hash
68
- :Array
69
- :Range
70
- :Set
71
-
72
- :Method
73
- :Proc
74
-
75
- INTERNAL_CLASSES_RE = /(^(Encoding|Struct|Gem)|Error\b|Errno|#<|fatal)/)
76
- ADDABLE_CLASSES = [Identity, Whatever, Itself, Boolean, Long, Double, Pathname, Set]
77
- INTERESTING_CLASSES = [String, Symbol, Regexp, Integer, Bignum, Float, Numeric, Complex, Rational, Time, Date, Dir, File, Hash, Array, Range, Method, Proc, Random, Class, Module, NilClass, TrueClass, FalseClass, ]
78
- BORING_CLASSES = [Object, File::Stat, StringIO, DateTime, Struct, MatchData, UnboundMethod, IO, Enumerator, Fixnum, BasicObject, Exception, Interrupt, SignalException, SystemExit, Encoding, Data, Fiber, Mutex, ThreadGroup, Thread, Binding, ]
79
-
80
- ObjectSpace.each_object(Class).
81
- reject{|kl| (kl.to_s =~ INTERNAL_CLASSES_RE }.map(&:to_s).sort - (ADDED_CLASSES + INTERESTING_CLASSES + BORING_CLASSES)
82
-
83
-
84
- [:to_ary, :to_io, :to_str, :to_hash, :to_sym, :to_path, :to_proc, :to_int, :to_f, :to_a, :to_r, :to_c, :to_i, :to_s, :to_enum]
85
-
86
-