rubadana 0.0.1 → 0.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 0a631bf26519d087f84658df366b3e8215ac07d8
-  data.tar.gz: c3e5b87d59a2e137288e54bea5c6f488f23d6f98
+  metadata.gz: f1d90348a8965bea4b82b2ce6fedb26bd7a1f148
+  data.tar.gz: 4dc9622fc7400d63e8bade76e05825f460d835f3
 SHA512:
-  metadata.gz: ce23a984a846411b53048cfd7288082a3431ea4f653fb98e308e79b9a2f55ddc4914b3538b31cdc7cf02437269c1b455af784d8674bf726af519417881b79e28
-  data.tar.gz: fd437f3e95811f6f78746b093152aa02c13b35745a1c983dce38cd5f9e8333360132fe62188e0daee13c665ba3cb0b9e56a08d6f38f6d312cdfc6dbd4e1f581b
+  metadata.gz: e01e9dceaed7d60c22432b6fd3834b69493ebb683d0c62c981050419c1d4e0e236699a3f6a24bcc736c33608b8870e90f618cb4f216c5d9852da33906e80def4
+  data.tar.gz: 77bc8d0c7241055b3d7bccf603a4b0f82490b5eaea2115c8fccf3f041f8498cf07e2125644d31c512d6b52c9182b6b7a5aa33ac46ed4b8a7ebb985651c7f9f3e

data/README.md

@@ -2,6 +2,9 @@
 
 Rubadana is an elementary ruby data-analysis package. It works with plain old ruby objects, not sql or databases or anything fancy like that.
 
+The aim is to create a summary overview from a list of objects by basically running a group-by/map/reduce operation on your list. The input, grouping,
+mapping, reducing, and display of the result are all independently variable.
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -20,33 +23,72 @@ Or install it yourself as:
 
 ## Usage
 
-See spec for some examples. The basic idea:
+Here's a trivial example, returning the number of requests per user:
+
+```ruby
+  program  = Rubadana::Registry.build groupings: [:user], mappers: [:self], reducers: [:count]
+  analysis = program.run Request.all
+```
+
+In this example, `:user`, `:self`, and `:count` are plugins you will have provided to rubadana for extracting and manipulating your data.
+
+`analyser.run` returns a list of `Rubadana::Analysis` instances, with, for this example, the following attributes:
 
-1. Create a Registry for your `Dimension` and `Accumulator` instances
+|`key`    | a `Hash` instance with keys `:user`                                                         |
+|`list`   | the subset of `Request.all` with the corresponding value for `:user`                        |
+|`mapped` | in this case, the same as `list` (assuming the `:identity` mapper returns the thing itself) |
+|`reduced`| a list of one integers, equal to the size of the list                                       |
+
+
+In ordinary ruby, you would write `Request.all.group_by(:user).map {|user, requests| [user, requests.count] }` to get the same information.
+
+Here's a richer example which returns the sum of debits, credits, and account balances from a set of accounting transactions:
+
+```ruby
+  program = Rubadana::Analyser.new group: [:month, :account_number], map: [:debits, :credits, :balance], reduce: [:sum, :sum, :sum]
+  analysis = program.run AccountingTransaction.all
+```
+
+In this example, `:month`, `:account_number`, `:debits` and so on, are plugins you will have provided to rubadana for extracting and manipulating your data.
+
+`analyser.run` returns a list of `Rubadana::Analysis` instances, with the following attributes:
+
+|`key`    | a `Hash` instance with keys `:month` and `:account_number`                                                                                       |
+|`list`   | the subset of `AccountingTransaction.all` having the corresponding values for `:month` and `:account_number`                                     |
+|`mapped` | the output of the `map` operations on `list`. This is a list of n-tuples, where `n` is the number of operations specified by the `map` parameter |
+|`reduced`| the output of the `reduce` operations on `mapped`. This is a list of n values, one for each operation specified by the `reduce` parameter.       |
+
+In this example, `reduced` gives us the sum of all debits, the sum of all credits, and the sum of all balances, per account-number
+
+See spec for some examples.
+
+## Steps
+
+1. Create a Registry for your mappers and your reducers
 
 my_registry = Rubadana::Registry.new
 
-2. Create and register some `Dimension` instances:
+2. Create and register some mappers
 
 ```ruby
-  class SaleYear < Rubadana::Dimension
-    def name                  ; "yearly"                                        ; end
-    def group_value_for thing ; thing.date.year                                 ; end
-    def value_label_for value ; value                                           ; end
+  class SaleYear
+    def name        ; :yearly         ; end
+    def run  thing  ; thing.date.year ; end
+    def label value ; value           ; end
   end
 
-  my_registry.register_dimension SaleYear.new
+  my_registry.register_mapper SaleYear.new
 ```
 
-3. Create and register some `Accumulator` instances:
+3. Create and register some reducers:
 
 ```ruby
-  class SumSaleAmount < Rubadana::Summation
-    def name            ; "sum-sale-amount"         ; end
-    def value_for thing ; thing.sale_amount         ; end
+  class Sum
+    def name          ; :sum             ; end
+    def reduce things ; things.reduce :+ ; end
   end
 
-  my_registry.register_accumulator SumSaleAmount.new
+  my_registry.register_reducer Sum.new
 ```
 
 4. Build an analysis program and run it:
@@ -54,21 +96,16 @@ my_registry = Rubadana::Registry.new
 ```ruby
   # this is a program to analyse invoices by year and product, giving the
   # number of sales, the sum of sales and the average sale in each case
-  my_program = register.build ["yearly", "invoice-product"], ["count", "sum-sale-amount", "avg-sale-amount"]
+  my_program = register.build group: %i{ yearly }, map: %i{ self sale_amount sale_amount }, reduce: %i{ count sum average }
 
   data = my_program.run(invoices)
 ```
 
-`#run` returns an array of `DataSet` each with the following attributes:
-
-* `analyser`    - a `Dimension` instance
-* `group_value` - the common value of this dimension for all objects in this data-set
-* `data`        - either an accumulated value given by an accumulator, or a nested array of `DataSet` instances
-
+`#run` returns an array of `Rubadana::Analysis` as described above.
 
 ## Contributing
 
-1. Fork it ( https://github.com/[my-github-username]/rubadana/fork )
+1. Fork it ( https://github.com/conanite/rubadana/fork )
 2. Create your feature branch (`git checkout -b my-new-feature`)
 3. Commit your changes (`git commit -am 'Add some feature'`)
 4. Push to the branch (`git push origin my-new-feature`)

data/lib/rubadana.rb

@@ -3,75 +3,71 @@ require "rubadana/version"
 module Rubadana
   class Registry
     def initialize
-      @dimensions   = Hash.new
-      @accumulators = Hash.new
+      @mappers    = Hash.new
+      @reducers   = Hash.new
     end
 
-    def register_dimension    d ; @dimensions[d.name.to_sym]   = d                             ; end
-    def register_accumulator  a ; @accumulators[a.name.to_sym] = a                             ; end
-    def dimensions              ; @dimensions.values                                           ; end
-    def accumulators            ; @accumulators.values                                         ; end
-    def not_nil attr, hsh, name ; hsh[name.to_sym] || raise("unknown #{attr} #{name.inspect}") ; end
-    def dimension          name ; not_nil "dimension"  , @dimensions  , name                   ; end
-    def accumulator        name ; not_nil "accumulator", @accumulators, name                   ; end
-
-    def build dnames, anames
-      dd = dnames.compact.map { |n| dimension   n }
-      aa = anames.compact.map { |n| accumulator n }
-      Program.new(dd + aa)
-    end
+    def register_mapper       m ; @mappers[m.name.to_sym]  = m                                       ; end
+    def register_reducer      r ; @reducers[r.name.to_sym] = r                                       ; end
+    def mapper             name ; @mappers[name.to_sym]  || raise("unknown mapper #{name.inspect}")  ; end
+    def reducer            name ; @reducers[name.to_sym] || raise("unknown reducer #{name.inspect}") ; end
+    def mappers           names ; names.map { |n| mapper n }                                         ; end
+    def reducers          names ; names.map { |n| reducer n }                                        ; end
+    def build params            ; Programmer.new(params).build(self)                                 ; end
   end
 
-  class DataSet < Aduki::Initializable
-    attr_accessor :analyser, :group_value, :data
-    def value_label ; analyser.value_label_for group_value ; end
+  class Self
+    def name           ; :self  ; end
+    def run      thing ; thing  ; end
   end
 
-  class Accumulator
-    def name              ; raise "implement this and return a unique name for this accumulator"        ; end
-    def accumulate things ; raise "implement this and return a value extracted from #things"            ; end
-    def run things, after ; [DataSet.new(analyser: self, data: accumulate(things))] + after.run(things) ; end
+  class Sum
+    def name               ; :sum              ; end
+    def reduce      things ; things.reduce(:+) ; end
   end
 
-
-  class Summation < Accumulator
-    def value_for   thing ; raise "implement this and return a value extracted from #thing"       ; end
-    def accumulate things ; things.map { |thing| value_for(thing) }.reduce :+                     ; end
+  class Count
+    def name          ; :count       ; end
+    def reduce things ; things.count ; end
   end
 
-  class Counter < Accumulator
-    def name              ; "count"           ; end
-    def accumulate things ; things.uniq.count ; end
+  class CountUnique
+    def name          ; :count_unique     ; end
+    def reduce things ; things.uniq.count ; end
   end
 
-  class Average < Summation
-    def accumulate things ; super / (1.0 * things.count) ; end
+  class Average
+    def name          ; :average                                 ; end
+    def reduce things ; things.reduce(:+) / (1.0 * things.count) ; end
   end
 
-  class Dimension
-    def name                  ; raise "implement this and return a unique name for this dimension"    ; end
-    def group_value_for thing ; raise "implement this and return a value extracted from #thing"       ; end
-    def value_label_for value ; raise "implement this to return a display value for #{value.inspect}" ; end
+  class Analysis < Aduki::Initializable
+    attr_accessor :program, :key, :list, :mapped, :reduced
+    def key_labels ; key_str = program.group.zip(key).map { |g,k| g.label k } ; end
+    def to_s       ; "#{key_labels.join ", "} : #{reduced.join ", "}"            ; end
+  end
 
-    def run objects, after
-      objects.group_by { |obj| group_value_for obj }.map { |value, list|
-        DataSet.new analyser: self, group_value: value, data: after.run(list)
-      }
+  class Programmer < Aduki::Initializable
+    attr_accessor :group, :map, :reduce
+    def build reg
+      Program.new group: reg.mappers(group), map: reg.mappers(map), reduce: reg.reducers(reduce)
     end
   end
 
-  class Program
-    attr_accessor :dimension, :after
+  class Program < Aduki::Initializable
+    attr_accessor :group, :map, :reduce, :groups
 
-    def initialize dimensions
-      if dimensions
-        self.dimension = dimensions.first
-        self.after = Program.new dimensions[1..-1]
-      end
-    end
+    def run things
+      self.groups = Hash.new { |h, k| h[k] = [] }
+      things.each { |thing|
+        groups[group.map { |g| g.run thing }] << thing
+      }
 
-    def run objects
-      dimension ? dimension.run(objects, after) : []
+      groups.map { |key, things|
+        mapped  = map.map  { |m| things.map  { |thing| m.run thing } }
+        reduced = reduce.zip(mapped).map { |r, m| r.reduce m           }
+        Analysis.new(program: self, key: key, list: things, mapped: mapped, reduced: reduced )
+      }
     end
   end
 end

data/lib/rubadana/version.rb

@@ -1,3 +1,3 @@
 module Rubadana
-  VERSION = "0.0.1"
+  VERSION = "0.0.2"
 end

data/rubadana.gemspec

@@ -1,4 +1,3 @@
-# coding: utf-8
 lib = File.expand_path('../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 require 'rubadana/version'
@@ -22,4 +21,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "bundler", "~> 1.7"
   spec.add_development_dependency "rake", "~> 10.0"
   spec.add_development_dependency 'rspec'
+  spec.add_development_dependency 'rspec_numbering_formatter'
 end

data/spec/analyse_invoices_spec.rb

@@ -7,38 +7,34 @@ describe "analyse invoices" do
     attr_accessor :type, :date, :amount
   end
 
-  class InvoiceMonth < Rubadana::Dimension
-    def name                  ; "monthly"                                       ; end
-    def group_value_for thing ; Date.new(thing.date.year, thing.date.month, 1)  ; end # rails just use #beginning_of_month
-    def value_label_for value ; value.strftime "%B %Y"                          ; end # better with I18n
+  class InvoiceMonth
+    def name        ; "monthly"                                       ; end
+    def run   thing ; Date.new(thing.date.year, thing.date.month, 1)  ; end # rails just use #beginning_of_month
+    def label value ; value.strftime "%B %Y"                          ; end # better with I18n
   end
 
-  class InvoiceYear < Rubadana::Dimension
-    def name                  ; "yearly"                                        ; end
-    def group_value_for thing ; thing.date.year                                 ; end
-    def value_label_for value ; value                                           ; end
+  class InvoiceYear
+    def name        ; "yearly"                                        ; end
+    def run   thing ; thing.date.year                                 ; end
+    def label value ; value                                           ; end
   end
 
-  class InvoiceType < Rubadana::Dimension
-    def name                  ; "type"                                          ; end
-    def group_value_for thing ; thing.type                                      ; end
-    def value_label_for value ; value.to_s                                      ; end
+  class InvoiceType
+    def name        ; "type"                                          ; end
+    def run   thing ; thing.type                                      ; end
+    def label value ; value.to_s                                      ; end
   end
 
-  class InvoiceScale < Rubadana::Dimension
-    def name                  ; "scale"                                         ; end
-    def group_value_for thing ; Math.log(thing.amount, 10).to_i                 ; end
-    def value_label_for value ; value                                           ; end
+  class InvoiceScale
+    def name        ; "scale"                                         ; end
+    def run   thing ; Math.log(thing.amount, 10).to_i                 ; end
+    def label value ; value                                           ; end
   end
 
-  class InvoiceSum < Rubadana::Summation
-    def name            ; "sum-amount" ; end
-    def value_for thing ; thing.amount ; end
-  end
-
-  class InvoiceAvg < Rubadana::Average
-    def name            ; "avg-amount" ; end
-    def value_for thing ; thing.amount ; end
+  class InvoiceAmount
+    def name        ; :invoice_amount                                 ; end
+    def run   thing ; thing.amount                                    ; end
+    def label value ; value.to_s                                      ; end
   end
 
   let(:i00) { Invoice.new type: "SalesInvoice"      , date: date("2020-02-01"), amount:     53 }
@@ -62,19 +58,21 @@ describe "analyse invoices" do
   let(:register) { Rubadana::Registry.new }
 
   before {
-    register.register_dimension   InvoiceYear.new
-    register.register_dimension   InvoiceMonth.new
-    register.register_dimension   InvoiceType.new
-    register.register_dimension   InvoiceScale.new
-    register.register_accumulator InvoiceSum.new
-    register.register_accumulator InvoiceAvg.new
-    register.register_accumulator Rubadana::Counter.new
+    register.register_mapper  InvoiceYear.new
+    register.register_mapper  InvoiceMonth.new
+    register.register_mapper  InvoiceType.new
+    register.register_mapper  InvoiceScale.new
+    register.register_mapper  InvoiceAmount.new
+    register.register_mapper  Rubadana::Self.new
+    register.register_reducer Rubadana::Sum.new
+    register.register_reducer Rubadana::Average.new
+    register.register_reducer Rubadana::Count.new
   }
 
   it "groups items by month and counts them" do
-    program  = register.build ["monthly"], ["count"]
+    program  = register.build group: %i{ monthly }, map: %i{ self }, reduce: %i{ count }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).map { |d| [d.value_label] +  d.data.map(&:data) }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
     expected = [
                 ["February 2020", 3],
                 ["May 2020"     , 3],
@@ -88,9 +86,10 @@ describe "analyse invoices" do
   end
 
   it "groups items by year and sums them" do
-    program  = register.build ["yearly"], ["sum-amount"]
+    program  = register.build group: %i{ yearly }, map: %i{ invoice_amount }, reduce: %i{ sum }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).map { |d| [d.value_label] +  d.data.map(&:data) }
+    # data.sort_by(&:key).each { |d| puts d }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
     expected = [
                 [2020, 175166],
                 [2021,   4369],
@@ -100,9 +99,9 @@ describe "analyse invoices" do
   end
 
   it "groups items by year and counts them" do
-    program  = register.build ["yearly"], ["count"]
+    program  = register.build group: %i{ yearly }, map: %i{ self }, reduce: %i{ count }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).map { |d| [d.value_label] +  d.data.map(&:data) }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
     expected = [
                 [2020, 8],
                 [2021, 4],
@@ -112,9 +111,9 @@ describe "analyse invoices" do
   end
 
   it "groups items by year and averages them" do
-    program  = register.build ["yearly"], ["avg-amount"]
+    program  = register.build group: %i{ yearly }, map: %i{ invoice_amount }, reduce: %i{ average }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).map { |d| [d.value_label] +  d.data.map(&:data) }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
     expected = [
                 [2020, 21895.75],
                 [2021,  1092.25],
@@ -124,9 +123,9 @@ describe "analyse invoices" do
   end
 
   it "groups items by year and gives the count, sum, and average" do
-    program  = register.build ["yearly"], ["count", "sum-amount", "avg-amount"]
+    program  = register.build group: %i{ yearly }, map: %i{ invoice_amount invoice_amount invoice_amount }, reduce: %i{ count sum average }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).map { |d| [d.value_label] + d.data.map(&:data) }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
     expected = [
                 [2020, 8, 175166, 21895.75],
                 [2021, 4,   4369,  1092.25],
@@ -135,14 +134,10 @@ describe "analyse invoices" do
     expect(actual).to eq expected
   end
 
-  it "groups items by year and by type and by scale and counts them" do
-    program  = register.build ["yearly", "type"], ["count"]
+  it "groups items by year and by type and counts them" do
+    program  = register.build group: %i{ yearly type }, map: %i{ self }, reduce: %i{ count }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).inject([]) { |arr, d|
-      d.data.sort_by(&:group_value).each { |s|
-        arr << [d.value_label, s.value_label] + s.data.map(&:data) }
-      arr
-    }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
 
     expected = [
                 [2020 , "PurchaseCreditNote" , 2 ],
@@ -160,13 +155,9 @@ describe "analyse invoices" do
   end
 
   it "groups items by scale and by type and sums them" do
-    program  = register.build ["scale", "type"], ["sum-amount"]
+    program  = register.build group: %i{ scale type }, map: %i{ invoice_amount }, reduce: %i{ sum }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).inject([]) { |arr, d|
-      d.data.sort_by(&:group_value).each { |s|
-        arr << [d.value_label, s.value_label] +  s.data.map(&:data) }
-      arr
-    }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
 
     expected = [
                 [1 , "Order"              , 59     ] ,
@@ -187,15 +178,9 @@ describe "analyse invoices" do
   end
 
   it "groups items by year and by type and by scale and counts them" do
-    program  = register.build ["yearly", "type", "scale"], ["sum-amount"]
+    program  = register.build group: %i{ yearly type scale }, map: %i{ invoice_amount }, reduce: %i{ sum }
     data     = program.run(invoices)
-    actual   = data.sort_by(&:group_value).inject([]) { |arr, d|
-      d.data.sort_by(&:group_value).each { |s|
-        s.data.sort_by(&:group_value).each { |z|
-          arr << [d.value_label, s.value_label, z.value_label] + z.data.map(&:data) }
-      }
-      arr
-    }
+    actual   = data.sort_by(&:key).map { |d| d.key_labels + d.reduced }
 
     expected = [
                 [2020 , "PurchaseCreditNote" , 1 , 23.0     ],

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rubadana
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.0.2
 platform: ruby
 authors:
 - Conan Dalton
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-05-22 00:00:00.000000000 Z
+date: 2016-11-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: aduki
@@ -66,6 +66,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec_numbering_formatter
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: " Simple data grouping and calculations. Bring your own extractors. "
 email:
 - conan@conandalton.net