gorillib 0.4.0pre → 0.4.1pre

data/notes/builder.md

@@ -0,0 +1,170 @@
+# gorillib/builder -- construct elegant ruby Domain-Specific Languages (DSLs).
+
+`Gorillib::Builder` provides foundation models for elegant ruby DSLs (Domain-Specific Languages). A builder block's relaxed ruby syntax enables highly-readable specification of complex behavior:
+
+    ```ruby
+    garage do
+      car :ford_tudor do
+        manufacturer    'Ford Motor Company'
+        car_model       'Tudor'
+        year            1939
+        doors           2
+        style           :sedan
+        engine do
+          volume  350
+          cylinders     8
+        end
+      end
+
+      car :wildcat do
+        manufacturer    'Buick'
+        car_model       'Wildcat'
+        year            1968
+        doors           2
+        style           :convertible
+        engine do
+          volume  455
+          cylinders     8
+        end
+      end
+    end
+    ```
+
+### Defining a builder
+
+To make a class be a builder, simply `include Gorillib::Builder`:
+
+
+      ```ruby
+      class Garage
+        include Gorillib::Builder
+        collection :car
+      end
+
+      class Car
+        include Gorillib::Builder
+        field      :name,         String
+        field      :manufacturer, String
+        field      :car_model,    String
+        field      :year,         Integer
+        field      :doors,        Integer
+        field      :style,        Symbol, :validates => { :inclusion_in => [:sedan, :coupe, :convertible] }
+        member     :engine
+        belongs_to :garage
+      end
+
+      class Engine
+        include Gorillib::Builder
+        field      :volume, Integer
+        field      :cylinders,    Integer
+        belongs_to :car
+      end
+      ```
+
+### getset accessors
+
+Fields of a Builder class create a single "getset" accessor (and not the familiar 'foo/foo=' pair):
+
+* `car_model.foo`                -- returns value of foo
+* `car_model.foo(val)`           -- sets foo to `val`, which can be any value, even `nil`.
+
+### member fields
+
+A builder class can have `member` fields; the type of a member field should be a builder class itself
+
+With no arguments, the accessor returns the value of engine attribute, creating if necessary:
+
+      ```ruby
+      car.engine                    #=> #<Engine volume=~ cylinders=~>
+      ```
+
+If you pass in a hash of values, the engine is created if necessary and then asked to `receive!` the hash.
+
+      ```ruby
+      car.engine                    #=> #<Engine volume=~ cylinders=~>
+      car.engine(:cylinders => 8)   #=> #<Engine volume=~ cylinders=8>
+      car.engine.cylinders          #=> 8
+      ```
+
+If you provide a no-args block, it is `instance_eval`ed in the context of the member object:
+
+      ```ruby
+      car :ford_tudor do
+        engine(:cylinders => 8) do
+          self                      #=> #<Engine volume=~ cylinders=8>
+          volume  455
+          cylinders     self.cylinders - 2
+          self                      #=> #<Engine volume=455 cylinders=6>
+        end
+        engine                      #=> #<Engine volume=455 cylinders=6>
+      end
+      ```
+
+Some people disapprove of `instance_eval`, as they consider it unseemly to mess around with `self`. If you instead provide a one-arg block, the member object is passed in:
+
+      ```ruby
+      car :ford_tudor do |c|
+        c.engine(:cylinders => 8) do |eng|
+          self                      #=> #<Car ...>
+          eng.volume 455
+          eng.cylinders    eng.cylinders - 2
+          eng                       #=> #<Engine volume=455 cylinders=6>
+        end
+        c.engine                    #=> #<Engine volume=455 cylinders=6>
+      end
+      ```
+
+### collections
+
+A builder class can also have `collection` fields, to contain named builder objects.
+
+      ```ruby
+      garage do
+        car(:ford_tudor)            #=> #<Car name="ford tudor" ...>
+        cars                        #=> { :ford_tudor => #<Car ...>, :wildcat => #<Car ...> }
+      end
+      ```
+
+The collected items must respond to `id` (FIXME:). The singular accessor accepts arguments just like a `member` accessor:
+
+      ```ruby
+      garage do
+        car(:ford_tudor, :year => 1939)
+        #=> #<Car name=`<:ford_tudor year=1939 doors=~ ...>
+        car(:ford_tudor, :year => 1939) do
+          doors           2
+          style           :convertible
+        end
+        #=> #<Car name="ford tudor" year=1939 doors=2 ...>
+        car(:wildcat, :year => 1968) do |c|
+          c.doors         2
+          c.style         :convertible
+        end
+        #=> #<Car name= year=1939 doors=2 ...>
+
+
+
+
+### Model methods
+
+builders have the following:
+
+* `Model::Defaults`
+* `Model::Naming`
+* `Model::Conversion`
+
+
+
+### SubclassRegistry
+
+When a subclass happens, decorates class with `saucepot(...)`, equivalent to
+
+        update_or_create
+        registers
+
+* At class level, `registry_for(CookingUtensil)` gives
+  - `add_cooking_utensil()` and so forth for adding and getting
+  - Protected `cooking_utensils` hash class attribute
+* Enlisting a resource class (Kitchen.register(FryingPan) and gives you a magic `frying_pan` factory method with signature `frying_pan(name, *args, &block)`
+  - If resource does not exist, calls `FryingPan.new` with full set of params and block. Add it to `cooking_utensils` registry.
+  - If resource with that name exists, retrieve it. call `merge()` with the parameters, and `run_in_scope` with the block.

data/notes/collection.md

@@ -0,0 +1,81 @@
+# gorillib/collection -- associative arrays of keyed objects
+
+Collection flexibly exchanges `{name => obj}` and `[ obj_with_name, obj_with_name, ... ]` -- this makes Mongo and JSON happy.
+
+### Collection type
+
+Defining
+
+```ruby
+  class Continent
+    include Gorillib::Record
+    # ...
+    field :countries, Collection, :of => Geo::Country, :key_method => :country_id
+  end
+```
+
+Lets it serialize as
+
+```yaml
+- name:             Africa
+  countries:
+    - name:           Rwanda
+      country_id:     rw
+      capitol_city:   Kigali
+    - name:           Djibouti
+      country_id:     dj
+      capitol_city:   Djibouti
+```
+
+or naturally pivot to be
+
+```ruby
+{ name:               "Africa",
+  countries: {
+    rw: {
+      name:           "Rwanda",
+      country_id:     "rw",
+      capitol_city:   "Kigali" },
+    dj: {
+      name:           "Djibouti",
+      country_id:     "dj",
+      capitol_city:   "Djibouti" },
+  }
+}
+```
+
+Calling `Collection.receive` works on either representation, and calling `to_a` or `to_hash` does what you'd think:
+
+```ruby
+   africa.countries.to_a    # [ <Geo::Country name="Rwanda"...>, <Geo::Country name="Djibouti"...> ]
+   africa.countries.to_hash # { :rw => <Geo::Country name="Rwanda"...>, :dj => <Geo::Country name="Djibouti"...> }
+```
+
+`Collection` proxies a small set of methods to an internal Hash. We purposefully keep you from calling methods that work differently on Hash and Array -- most notably, there is no `each` method, and it is currently *not* an Enumerable. 
+
+* `each_pair`  iterates over the key/value pairs
+* `each_value` iterates over the values
+* `to_a`       gives the list of values
+* `to_hash`    gives a duplicate of the proxied hash
+* `inspect`, `to_s`, `as_json` and `to_json` show a list of values -- it serializes like an array
+
+* `merge!`, `concat` or `receive!` add values in-place, and `merge` to a duplicate, as follows:
+  - if the given collection responds_to `to_hash`, it is merged into the internal collection; each hash key *must* match the id of its value or results are undefined.
+  - otherwise, it merges a hash generates from the id/value pairs of each object in the given collection.
+
+* `[]=` adds a single value with a specified key; that key *must* match the value's id or it is undefined behavior.
+* `<<` adds a single value at the right key.
+
+* `build(attributes={})`
+
+## advanced use
+
+* `key_method` (read-only class attribute, a Symbol) defines how to find the id; by default, `:id`
+* `factory` (read-only class attribute) defines how to create a new item given its raw material. This is handed to `Gorillib::Factory` for conversion the first time it's read.
+
+## Decisions
+
+## Mysteries
+
+* ?? no `each` method
+* ?? `Enumerable`

data/notes/factories.md

@@ -0,0 +1,86 @@
+# gorillib/factories (WIP -- NOT READY YET) -- efficient, predictable type conversion
+
+* Is a missing value unset? or nil?
+
+* (Boolean)
+* String Symbol
+* Regexp
+* Bignum Integer Numeric
+* Float (Double)
+* Complex Rational
+* Random
+* Time
+* Date
+
+* Dir File Pathname
+* Whatever -- alias for IdentityFactory
+
+* Method, Proc
+* Range
+
+* Hash
+* Array
+
+* Object
+* Class Module FalseClass TrueClass NilClass
+
+These don't have factories because I can't really think of a reasonable use case
+* DateTime Struct MatchData UnboundMethod IO Enumerator Fixnum Object File::Stat StringIO
+
+These are *not* decorated because it's not a good idea
+* BasicObject
+* Exception Interrupt SignalException SystemExit
+* Encoding Data Fiber Mutex ThreadGroup Thread Binding Queue
+
+    # name    produces      convert  receivable?     converted?      nil  ""   []   {}  false  true
+    :Integer, Fixnum,       :to_i,   :to_i,          is_a?(Float),   nil  nil, err, err
+    :Bignum
+    :Float,   Float,        :to_f,   :to_f,          is_a?(Fixnum),  nil  nil, err, err
+    :Complex
+    :Rational
+
+    # :Random
+    # :Long
+    # :Double
+    # :Numeric
+
+    :String,  String,       :to_s,   :to_s,          is_a?(String),  nil, "",  ??,  ??
+    :Binary,  Binary,       :to_s,   :to_s,          is_a?(String),  nil, "",  ??,  ??
+    :Symbol,  Symbol,       :to_s,   :to_s,          is_a?(String),  nil, nil, ??,  ??
+    :Regexp,
+    :Time
+    # :Date
+    # :Dir
+    # :File
+    # :Pathname
+
+    :Identity
+    :Whatever
+
+    :Class
+    :Module
+    :TrueClass, true,
+    :FalseClass, false,
+    :NilClass,  nil,
+    :Boolean, [true,false], ??,      [true, false],  [true, false],  nil, nil, ??,  ??
+
+    :Hash
+    :Array
+    :Range
+    :Set
+
+    :Method
+    :Proc
+
+    INTERNAL_CLASSES_RE = /(^(Encoding|Struct|Gem)|Error\b|Errno|#<|fatal)/)
+    ADDABLE_CLASSES     = [Identity, Whatever, Itself, Boolean, Long, Double, Pathname, Set]
+    INTERESTING_CLASSES = [String, Symbol, Regexp, Integer, Bignum, Float, Numeric, Complex, Rational, Time, Date, Dir, File, Hash, Array, Range, Method, Proc, Random, Class, Module, NilClass, TrueClass, FalseClass, ]
+    BORING_CLASSES      = [Object, File::Stat, StringIO, DateTime, Struct, MatchData, UnboundMethod, IO, Enumerator, Fixnum, BasicObject, Exception, Interrupt, SignalException, SystemExit, Encoding, Data, Fiber, Mutex, ThreadGroup, Thread, Binding, ] 
+
+    ObjectSpace.each_object(Class).
+      reject{|kl| (kl.to_s =~ INTERNAL_CLASSES_RE }.map(&:to_s).sort - (ADDED_CLASSES + INTERESTING_CLASSES + BORING_CLASSES)
+
+
+[:to_ary, :to_io, :to_str, :to_hash, :to_sym, :to_path, :to_proc, :to_int, :to_f, :to_a, :to_r, :to_c, :to_i, :to_s, :to_enum]
+
+

data/notes/model-overlay.md

@@ -0,0 +1,209 @@
+# gorillib/record/overlay (WIP -- NOT READY YET) -- Progressively overlayed configuration
+
+`Record::Overlay` lets you specify an ordered stack of attributes, each overriding the following. For example, you may wih to load your program's configuration using
+
+     (wins) value set directly on self
+            commandline parameters
+            environment variables
+            per-user config file
+            machine-wide config file
+    (loses) application defaults
+
+In the Ironfan toolset, servers are organized into hierarchical groups -- 'provider', then 'cluster', then 'facet'. Its configuration builder lets you describe a cloud server as follows:
+
+     (wins) server-specific
+            facet
+            cluster
+            provider
+    (loses) default
+
+### Higher layers override lower layers
+
+In the following example, we set a value for the `environment` attribute on the cluster, then override it in the facet:
+
+    cluster :gibbon do |c|
+      c.environment  :prod
+
+      c.facet :worker do |f|
+        # since no value has been set on the facet, a read pokes through to the cluster's value
+        f.environment                 #=> :prod
+        # setting a value on the facet overrides the value from the cluster:
+        f.environment :dev
+        f.environment                 #=> :dev
+      end
+    end
+
+This diagram might help, for a hypothetical 'owner' setting in confilgiere:
+
+     Settings:
+                                   +-- reads come in from the top
+     _______________________       |
+     | .owner               |     \|/
+     +----------------------+------------------+
+     | self                 |                  |
+     | overlay(:commandline)  |                  |
+     | overlay(:env_vars)     |    'bob'         | <-- writes go in from the side
+     | overlay(:user_conf)    |                  |
+     | overlay(:system_conf)  |    'alice'       | <--
+     | default, if any      |                  |
+     +----------------------+------------------+
+
+In this diagram, values have been set on the objects at the 'env_vars' and 'system_conf' layers, so when you read the final settings value with `Settings.owner` it returns `"bob"`. If the user had specified `--owner="charlie"` on the commandline, it would have set a value in the 'commandline' row and taken precedence; `Settings.owner` would return `"charlie"` instead.
+
+### Late Resolution means overlays are dynamic
+
+Overlay layers are late-resolved -- they return whatever value is appropriate at time of read:
+
+    cluster(:gibbon) do |c|
+      c.environment(:prod)
+      # since no value has been set on the facet, a read pokes through to the cluster's value
+      c.facet(:worker).environment      #=> :prod
+
+      # changing the cluster's value changes the facet's effective value as well:
+      c.environment(:test)
+      c.facet(:worker).environment      #=> :test
+    end
+
+## Defining an Overlay
+
+You can apply overlay behavior to any `Gorillib::Record`: the `Builder`, `Model` and `Bucket` classes, or any diabolical little variant you've cooked up.
+
+An Overlay'ed object's `read_unset_attribute` calls `super`, so if there is a default value but neither the object nor any overlay have a value set,
+
+## Cascading attributes
+
+I want to be able to define what's uniform across a collection at one layer, and customize only what's special at another. In this example, our overlay stack is `cluster > facet`; each has a `volumes` collection field. The `:master` facet overrides the size and tags applied to the volume; in the latter case this means modifying the value read from the cluster layer and applying it to the facet layer:
+
+    ```ruby
+    cluster :gibbon do
+      volumes(:hadoop_data) do
+        size            300
+        mount_point     '/hadoop_data'
+        tags( :hadoop_data => true, :persistent => true, :bulk => true)
+      end
+      # the master uses a smaller volume and only stores data for the namenode
+      facet :master do
+        volumes(:hadoop_data) do
+          size            50
+          tags            tags.except(:hadoop_data).merge(:hadoop_namenode_data => true)
+        end
+      end
+      # workers use the volume just as described
+      facet :worker
+        # ...
+      end
+    end
+    ```
+
+_note_: This does not feel elegant at all.
+
+    class Facet
+      attr_accessor  :cluster
+      overlays       :cluster
+      collection     :volumes, Volume, :members_overlay => :cluster
+    end
+
+When a collection element is built on the
+
+
+## Deep Merge
+
+
+
+## Dynamic Resolution Rules
+
+As you've seen, the core resolution rule is 'first encountered wins'. Simple and predictable, but sometimes insufficient.
+
+Define your field with a `:combine_with` block (or anything that responds to `#call`) to specify a custom resolution rule:
+
+    Settings.option :flavors, Array, :of => String, :combine_with => ->(obj,attr,layer_vals){ ... }
+
+Whenever there's a conflict, the block will be called with these arguments:
+
+* the host object,
+* the field name in question,
+* a list of the layer values in this order:
+  - field default (if set)
+  - lowest layer's value (if set)
+  - ...
+  - highest layer's value (if set)
+  - object's own value (if set)
+
+If you pass a symbol to `:combine_with`, the corresponding method will be invoked on each layer, starting with the lowest-priority:
+
+    Settings.option :flavors, Array, :of => String, :combine_with => :concat
+    # effectively performs
+    # layer(:charlie).flavors.
+    #   concat( layer(:bob).flavors ).
+    #   concat( layer(:alice).flavors )
+
+    Settings.option :flavors, Hash,  :of => String, :combine_with => :merge
+    # effectively performs
+    # layer(:charlie).flavors.
+    #   merge( layer(:bob).flavors ).
+    #   merge( layer(:alice).flavors )
+
+Notes:
+* a field's default counts as a layer: the combiner is invoked even if just the default and a value are present.
+* normal (first-encountered-wins) resolution uses `read_unset_attribute`
+* dynamic (explicit-block) resolution uses `read_attribute`
+* there's not a lot of sugar here, because my guess is that concat'ing arrays and merging or deep-merging hashes covers all the repeatable cases.
+
+__________________________________________________________________________
+__________________________________________________________________________
+__________________________________________________________________________
+
+
+## Decisions
+
+* Layer *does not* catch `Gorillib::UnknownFieldError` exceptions -- a layered field must be defined on each part of the layer stack.
+* a field's default counts as a layer: the combiner is invoked even if just the default and a value are present
+
+__________________________________________________________________________
+
+**Provisional**:
+
+## LayeredObject -- push some properties through to parent; earn predictable deep merge characteristics
+
+* holds a stack of objects
+* properties 'poke through' to get value with well-defined resultion rules
+* you can define a custom resolution rule if you define a property, or when you set its value
+
+* Properties are resolved in a first-one-wins
+* Defined properties can have resolution rules attached
+* You can put yourself in an explicit scope; higher properties are set, lower ones appear unset
+
+Server overlays facet
+Facet overlays cluster
+Cluster overlays provider
+
+don't prescribe *any* resolution semantics except the following:
+
+* layers are evaluated in order to create a composite, `dup`ing as you go.
+* while building a composite, when a later layer and the current layer collide:
+  - if layer has merge logic, hand it off.
+  - simple + any:      clobber
+  - array + array:     layer value appended to composite value
+  - hash  + hash:      recurse
+  - otherwise:         error
+
+###  no complicated resolution rules allowed
+
+This is the key to the whole thing.
+
+* You can very easily
+* You can adorn an object with merge logic if it's more complicated than that
+
+(?mystery - duping: I'm not certain of the always-`dup` rule. We'll find out).
+
+How do volumes overlay?
+
+Which direction is overlay declared?
+
+Configliere:
+
+* Default
+* File
+* Env var
+* Cmdline
+* Setting it directly

data/notes/model.md

@@ -0,0 +1,135 @@
+# gorillib/record -- construct lightweight structured data classes
+
+## Goals
+
+* light, predictable magic; you can define records without requiring everything & the kitchen sink.
+* No magic in normal operation: you are left with regular instance-variable attrs, control over your initializer, and in almost every respect can do anything you'd like to do with a regular ruby class.
+* Compatible with the [Avro schema format](http://avro.apache.org/)'s terminology & conceptual model
+* Upwards compatible with ActiveSupport / ActiveModel
+* All four obey the basic contract of a Gorillib::Record
+* Encourages assertive code -- no `method_missing` or complex proxy soup.
+
+___________________________________________________________________________
+
+## Gorillib::Record
+
+### Defining a record
+
+To make a class a record, simply `include Gorillib::Record`.
+
+      ```ruby
+      class Place
+        include Gorillib::Record
+        field :name, String
+        field :geo, GeoCoordinates, :doc => 'geographic location of the place'
+      end
+      ```
+
+### Field
+
+A record has `field`s that describe its attributes. The simplest definition just requires a field name and a type: `field :name, String`. (Use `Object` as the type to accept any value as-is). Specify optional attributes using keyword arguments -- for example, `doc` describes the field:
+
+      ```ruby
+        field :geo, GeoCoordinates, :doc => 'geographic location of the place'
+      ```
+
+You can list the fields, the field names (in the order they were defined), and ask if a field has been defined:
+
+      ```ruby
+      Place.fields             #=> {:name=>field(:name, Integer), :geo=>field(:geo, Geocoordinates)}
+      Place.field_names        #=> [:name, :geo]
+      Place.has_field?(:name)  #=> true
+      ```
+
+Subclasses inherit their parent's fields, just as you'd expect:
+
+      ```ruby
+      class Stadium < Place
+        field :capacity, Integer, :doc => 'quantity of seats'
+      end
+      Stadium.field_names #=> [:name, :geo, :capacity]
+
+      # Add a field to the parent and it shows up on the children, no sweat:
+      Place.field :country_id, String
+      Place.field_names   #=> [:name, :geo, :country_id]
+      Stadium.field_names #=> [:name, :geo, :capacity, :country_id]
+      ```
+
+### Reading, writing and unsetting values
+
+Defining a field defines accessor methods:
+
+      ```ruby
+      lunch_spot = Place.receive({ :name => "Torchy's Tacos", :country_id => "us",
+        :geo => { :latitude => "30.295", :longitude => "-97.745" }})
+
+### Attributes
+
+(A class defines `fields`; instances receive `value`s for those fields; the collection of an instance's `values` form its `attributes`)
+
+## Contract
+
+Every record responds to and guarantees uniform behavior for these methods:
+
+* Class methods from `Gorillib::Record`     -- `field`, `fields`, `field_names`, `has_field?`, `metamodel`
+* Instance methods from `Gorillib::Record`  -- `read_attribute`, `write_attribute`, `unset_attribute`, `attribute_set?`, `read_unset_attribute`, `attributes`
+
+Records generally respond to the following, but are allowed to get fancy as long as they fulfill your basic expectations (and can mark accessors private/protected or omit them):
+
+* Class methods from `Gorillib::Record`     -- `receive`, `inspect`
+* Instance methods from `Gorillib::Record`  -- `receive!`, `update`, `inspect`, `to_s`, `==`
+* Metamodel methods (eg, field named 'foo') -- `receive_foo`, `foo=`, `foo`
+
+These are the only
+
+* `Object#blank?`, `Hash#symbolize_keys`, `Object#try`
+
+### `read_attribute`, `write_attribute` and friends
+
+All normal access to attributes goes through `read_attribute`, `write_attribute`, `unset_attribute` and `attribute_set?`. All 'fixup' access goes through each field's `receive_XXX` method, which calls `write_attribute` in turn. This provides a consistent attachment point for advanced magic.
+
+     external methods             fixup gate          accessor gate
+
+     Klass.receive           => receive_foo(val) => write_attribute(:foo, val)
+     receive!(:foo => val)   => receive_foo(val) => write_attribute(:foo, val)
+                                receive_foo(val) => write_attribute(:foo, val)
+     update_attributes(:foo => val)              => write_attribute(:foo, val)
+     foo=(val)                                   => write_attribute(:foo, val)
+
+     attributes                                  => read_attribute(:foo)
+     foo                                         => read_attribute(:foo)
+
+                                                    attribute_set?(:foo)
+                                                    unset_attribute(:foo)
+
+If you are writing library code to extend `Gorillib::Record`, you *must* call the `xx_attribute` methods -- do not assume any behavior from (or even the existence of) accessors or anything else. By default, the core `xx_attribute` methods get/set/remove instance variables, but we've deliberately left them open to be implemented as hash values, by delegation, as a passthrough to database access, or things as-yet undreamt of. That's just for library code, though -- your class knows how it's built and can naturally leverage all its amenities.
+
+If you call `read_attribute` on an unset value, it in turn calls `read_unset_attribute`; the mixins that provide defaults, lazy access, or layered configuration hook in here.
+
+
+### method visibility
+
+You can mark a field's methods (`:reader`, `:writer`, `:receiver`) as public, private or protected, or even prevent its creation in the first place:
+
+    field :monogram, String, :writer => false, :receiver => :protected # a read-only field
+
+Visibility can be `:public`, `:protected`, `:private`, `true` (meaning `:public`) or `false` (in which case no method is manufactured at all).
+
+### extra_attributes
+
+Extra attributes passed to `receive!` are collected in `@extra_attributes`, but nothing is done with them.
+
+
+## Record::Default
+
+
+* default values
+  - nil by default
+  - simple value is returned directly
+  - Proc is `call`ed: `->{ Time.now }`
+    - to return a block as default, just wrap it in a dummy block: `->{ ->(obj,fn){  } }`
+    - The block may store an attribute value if desired, but must do so explicitly; otherwise, the block will be invoked on every access while the value is unset.
+  - how to invoke?
+    - foo_default
+    - attribute_default(:foo)
+    - field(:foo).default(self)