rails-data-explorer 1.0.1 → 1.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d359492c5e290bf9416c9040224f2c248cb8cad9
-  data.tar.gz: c245c7979079b276ac307427c4345fe987a303a5
+  metadata.gz: 20fe7f28eccad4493c4587b62d14409ead5fe178
+  data.tar.gz: dcadfa97805704be5521ba0ba4985de64a29d981
 SHA512:
-  metadata.gz: e7ef42c4035f467cbf96b0bfdd6fad6725139a091fd2f3191c55042e23d3fbf0290850dae6abe6f651706f8175d0b10bf8d36d8ce9e0692e10fe2e7170ffc665
-  data.tar.gz: 5d5e8ecdf816c24a95b10189391e75abd207454f81ed4219ad378046f7fa19c369b66ef403b6c495faf35c9f409c5b37e04ee87985e4d567a05869d604a3009c
+  metadata.gz: de90bf34ebed7445860e907e633d455b758addc56f9ae0e26ae47f2bb71895170ef86af84aa685312a00c1786d6e0aad26157eb2d0893a3dd2ac38cde3b84ef3
+  data.tar.gz: 267a938ca270d284756f5eb7b807887f3aaabdfafd74a7483c335fd31a12191485cf67d87b3b147ca1ff242ab573ac15e6d1a7af59a59f7b53df45449481ad43

data/CHANGELOG.md

@@ -3,6 +3,11 @@ TODO:
 * sampling for large ActiveRecord tables
 * automatic clustering of records (using each row as a feature vector)
 
+
+### 1.0.2
+
+* Updated specs, documentation, and Gem dependencies.
+
 ### 1.0.1
 
 * Added :note option to data series specs
@@ -12,7 +17,7 @@ TODO:
 
 # 1.0.0
 
-* New chart type: StackedBarChartCategorical to go with its percentage cousing.
+* New chart type: StackedBarChartCategorical to go with its percentage cousin.
 * Improved display of table of contents.
 * Improved display of box plot groups: Auto scale charts to make best use of
   screen space when there are extreme outliers.

data/README.md

@@ -1,17 +1,97 @@
 # rails-data-explorer
 
-Rails-data-explorer is a Rails Engine plugin that makes it easy to explore the
-data in your app using charts and statistics.
+rails-data-explorer (*RDE*) is a Rails engine plugin that makes it magically
+easy to explore your app's data using charts and statistics.
 
-All you have to do is to throw your data in table form at rails-data-explorer.
-Rails-data-explorer then analyzes your data and automatically renders suitable
-charts and descriptive statistics for each column. It offers tools for uni-, bi-
-and multi-variate analysis. It can even find correlations in your categorical data
-using Pearson's Chi Squared test.
+The purpose of *RDE* is to help you develop an intuition for your app's data
+so that you can find patterns, correlations and anomalies in the data that
+you can then further investigate and exploit.
 
-See below for a screenshot of what you get.
 
-### Installation
+
+## Examples
+
+***
+
+<table>
+  <tr>
+    <td>
+      <a href="https://raw.githubusercontent.com/jhund/rails-data-explorer/master/doc/rails-data-explorer-screenshot-all-univariate.png"><img src="https://github.com/jhund/rails-data-explorer/blob/master/doc/rails-data-explorer-screenshot-all-univariate.png" width="200" ></a>
+      <p>
+        The default charts rendered by rails-data-explorer: Univariate charts and
+        statistics for every column in your data.
+        <br/>
+        (Click image for full size screenshot)
+      </p>
+    </td>
+    <td>
+      <a href="https://raw.githubusercontent.com/jhund/rails-data-explorer/master/doc/rails-data-explorer-screenshot-plan-bivariate.png"><img src="https://github.com/jhund/rails-data-explorer/blob/master/doc/rails-data-explorer-screenshot-plan-bivariate.png" width="200" ></a>
+      <p>
+        All bivariate charts and statistics for the "Plan" column.
+        <br/>
+        (Click image for full size screenshot)
+      </p>
+    </td>
+  <tr/>
+<table>
+
+***
+
+
+
+## Features
+
+* **Optimized for simplicity and convenience**: All you have to do is provide
+  your data in tabular form. *RDE* then decides how to best visualize and
+  present your data.
+* **Batteries included**: *RDE* includes everything required to render stunning
+  charts and visualizations. It is tightly integrated with Rails and doesn't
+  require knowledge of charting libraries, statistics, or data engineering.
+* **Comprehensive data visualization and analysis**: Out of the box *RDE*
+  gives you univariate charts and statistics for each column of your data. It
+  also gives you single click access to all bivariate combinations
+  of your data columns. With a single config param, it will even generate
+  arbitrary multivariate charts for you.
+* **Works with small- and medium-sized data**: *RDE* works great for up to ~100K records.
+  I'm thinking about ways to make it work for larger data series using sampling.
+  If you know how to do this correctly in ActiveRecord, please let me know.
+
+Detailed features
+
+* Integrates with ActiveRecord.
+* Renders beautiful charts (using Vega/Nvd3 and d3.js).
+* Uses best charting practices (using Vega visualization grammar).
+* Automatically detects and handles categorical and quantitative data types.
+  (integer, decimal, and temporal)
+* Automatically renders visualizations and statistics for univariate, bivariate
+  and multivariate analysis.
+* Provides single click access to univarate and all possible combinations of
+  bivariate analyses.
+* Automatically picks linear vs. logarithmic axes.
+* Automatically finds top N values for categorical data to make charts easier
+  to read.
+* Automatically sorts axis labels for maximum clarity.
+* Performs Pearson's Chi squared test on bivariate analysis of categorical data.
+  This is super useful when analyzing and interpreting A/B test data, or to find
+  correlations in your data.
+* Works well with the Filterrific gem for filtering data before it is fed to
+  *RDE*.
+* Provides statistical utility methods, e.g., to bin or quantize your data.
+
+*RDE* provides the following chart types:
+
+* Bar chart (simple, stacked, percent distribution)
+* Pie chart
+* Histogram
+* Box plot (single and group)
+* Scatterplot
+* Parallel coordinates
+* Parallel set
+* Contingency table with pearson's chi squared test (good to interpret A/B test data)
+
+
+
+## Installation
 
 `gem install rails-data-explorer`
 
@@ -20,16 +100,16 @@ or with bundler in your Gemfile:
 `gem 'rails-data-explorer'`
 
 
-### Usage
 
-Let's say you want to explore your app's `User` signup data. Create a route and action
-named 'signups' in `app/controllers/users_controller.rb`:
+## Usage
+
+Let's say you want to explore your app's `User` signup data. Create a route and
+action for `users#signups`:
 
 ~~~ ruby
-def signups
-  # DANGER! Make sure not to load too much data into memory!
-  user_data = User.all.to_a
+# app/controllers/users_controller.rb
 
+def signups
   c_binner = RailsDataExplorer::Utils::DataBinner.new(
     '0' => 0,
     '1' => 1,
@@ -39,8 +119,8 @@ def signups
     '101..1,000' => 1000,
     '1,001..10,000' => 10000
   )
-  @rails_data_explorer = RailsDataExplorer.new(
-    user_data,
+  @rde = RailsDataExplorer.new(
+    User.all.to_a, # DANGER! Make sure not to load too much data into memory!
     [
       {
         name: "Session duration [Minutes]",
@@ -51,7 +131,6 @@ def signups
       {
         name: "Country",
         data_method: Proc.new { |row| row.country },
-        multivariate: ['All'],
       },
       {
         name: "Sign in count",
@@ -60,12 +139,10 @@ def signups
       {
         name: "Language",
         data_method: Proc.new { |row| row.language },
-        multivariate: ['All'],
       },
       {
         name: "Plan",
         data_method: Proc.new { |row| row.plan },
-        multivariate: ['All'],
       },
       {
         name: "Sign up date",
@@ -79,7 +156,8 @@ def signups
           "#{ year } / Q#{ quarter }"
         },
       },
-    ]
+    ],
+    params[:rde]
   )
 end
 ~~~
@@ -89,62 +167,55 @@ Then create a view at `app/views/users/signups.html.erb`:
 ~~~erb
 <div class="rails-data-explorer">
   <h1>User signup data</h1>
-  <%= @rails_data_explorer.render %>
+  <%= rails_data_explorer(@rde) %>
 </div>
 ~~~
 
-With just a few lines of code you get comprehensive statistics and charts for your data:
+With just a few lines of code you get comprehensive statistics and charts for
+your data (see screenshots above).
 
-***
-
-![Rails data exploration](https://github.com/jhund/rails-data-explorer/blob/master/doc/rails-data-explorer-screenshot.png)
-
-***
 
 
-### Ways to shoot yourself in the foot
+## Ways to shoot yourself in the foot
 
 * **Loading too many DB rows at once**: Remember that you are loading ActiveRecord
   objects, and they can use a lot of ram. It's a cartesian product of number of
   rows times columns per record. As a rule of thumb, for a medium sized model with
-  30 columns, you can load up to 10,000 rows.
+  10 columns, you can load up to ~100K rows.
 * **Using expensive operations in the `:data_method` option for a given data series**:
   As a rule of thumb, it should be ok to run simple methods that don't require
   DB access. Examples: `#.to_s`, `if` and `case`, and math operations should all
   be fine.
-* **Declaring too many charts**: The charts rendered are the sum of the following:
-    * univariate: one or more charts for each data series
-    * bivariate: the cartesian product of all data series in a bivariate group
-    * multivariate: one or more charts for each multivariate group
-  I have tested it with 70 charts on a single page. More are probably ok, I
-  just haven't tested it.
 * **Drowning in detail**: rails-data-explorer makes it easy to generate a large
   number of charts. Make sure you don't miss the important data in the noise.
 
 
-### Dependencies
 
-* ActionView >= 3.0
-* ActiveSupport >= 3.0
+## Dependencies
+
+* rails >= 3.1
 * Asset pipeline (for batteries included, otherwise you'll have to pull in a number of assets manually)
-* jQuery
 
-### Resources
 
+
+## Resources
+
+* [API documentation](http://www.rubydoc.info/gems/rails-data-explorer/)
 * [Changelog](https://github.com/jhund/rails-data-explorer/blob/master/CHANGELOG.md)
 * [Source code (github)](https://github.com/jhund/rails-data-explorer)
 * [Issues](https://github.com/jhund/rails-data-explorer/issues)
 * [Rubygems.org](http://rubygems.org/gems/rails-data-explorer)
-* [API documentation](http://www.rubydoc.info/gems/rails-data-explorer/)
 
 [![Build Status](https://travis-ci.org/jhund/rails-data-explorer.svg?branch=master)](https://travis-ci.org/jhund/rails-data-explorer)
 
-### License
+
+
+## License
 
 [MIT licensed](https://github.com/jhund/rails-data-explorer/blob/master/MIT-LICENSE).
 
 
 
-### Copyright
+## Copyright
 
-Copyright (c) 2015 Jo Hund. See [(MIT) LICENSE](https://github.com/jhund/rails-data-explorer/blob/master/MIT-LICENSE) for details.
+Copyright (c) 2014 - 2015 Jo Hund. See [(MIT) LICENSE](https://github.com/jhund/rails-data-explorer/blob/master/MIT-LICENSE) for details.

data/lib/rails_data_explorer.rb

@@ -13,11 +13,31 @@ class RailsDataExplorer
   attr_reader :explorations
   attr_reader :data_series_names
 
-  # Top level initialization. This is what you normally should use.
+  # Top level initialization. This is what you use to explore your data.
   # @param data_collection [Array<Array>] Outer array is the container, inner
   #                                       array represents each record (row of data).
   # @param data_series_specs [Array<Hash>] One Hash for each data series.
-  # @param explorations_to_render [Hash]
+  # @option data_series_specs [String] :name the name of the data series
+  # @option data_series_specs [Proc] :data_method a proc that will return the
+  #                                   data for this column. Valid types are
+  #                                   String, Numeric or Date/Time.
+  # @option data_series_specs [String] :note any comment you want to make for
+  #                                   this data series. Will be printed with
+  #                                   each chart that uses this data series.
+  # @option data_series_specs [Boolean, String] :univariate override to always
+  #                                   render univariate chart and statistics for
+  #                                   this data series. Defaults to true.
+  # @option data_series_specs [Boolean, String] :bivariate override to always
+  #                                   render bivariate chart and statistics for
+  #                                   this data series. Defaults to false.
+  # @option data_series_specs [Boolean, String] :multivariate override to always
+  #                                   render multivariate chart and statistics for
+  #                                   this data series. Defaults to false.
+  # @option data_series_specs [Integer] :max_num_distinct_values override the
+  #                                   max number of distinct values for
+  #                                   categorical data. Default: 20.
+  # @param explorations_to_render [Hash] A hash to specify which explorations
+  #     you want rendered. Example data structure:
   #        {
   #          "univariate" => {
   #            "1" => ["Hour of day"],
@@ -27,6 +47,7 @@ class RailsDataExplorer
   #            "2" => ["Year", "Timezone"],
   #          }
   #        }
+  #
   def initialize(data_collection, data_series_specs, explorations_to_render)
     @explorations = []
     charts = {
@@ -115,18 +136,22 @@ class RailsDataExplorer
     }
   end
 
+  # @return [Array<Exploration>]
   def explorations_with_charts_available
     explorations.find_all { |e| e.charts.any? }
   end
 
+  # @return [Array<Exploration>]
   def explorations_with_charts_to_render
     explorations_with_charts_available.find_all { |e| e.render_charts? }
   end
 
+  # @return [Array<Exploration>]
   def explorations_with_no_charts_available
     explorations.find_all { |e| e.charts.empty? }
   end
 
+  # @return [Integer]
   def number_of_values
     explorations.first.number_of_values
   end

data/rails-data-explorer.gemspec

@@ -3,7 +3,7 @@ $:.push File.expand_path('../lib', __FILE__)
 
 Gem::Specification.new do |gem|
   gem.name = 'rails-data-explorer'
-  gem.version = '1.0.1'
+  gem.version = '1.0.2'
   gem.platform = Gem::Platform::RUBY
 
   gem.authors = ['Jo Hund']
@@ -26,4 +26,5 @@ Gem::Specification.new do |gem|
   gem.add_development_dependency 'minitest', '>= 0'
   gem.add_development_dependency 'minitest-spec-expect', '>= 0'
   gem.add_development_dependency 'rake', '>= 0'
+  gem.add_development_dependency 'yui-compressor', '>= 0'
 end

data/spec/rails_data_explorer/exploration_spec.rb

@@ -3,53 +3,64 @@ require_relative '../helper_no_rails'
 class RailsDataExplorer
   describe Exploration do
 
-    describe "integration test" do
-
-      def check_render_expectations(rendered_string, options)
-        r = true
-
-        if options[:has_charts]
-          options[:has_charts].each { |chart_name|
-            # TODO: use Nokogiri to test that it is a div class
-            rendered_string.must_match(
-              Regexp.new(Regexp.escape("rde-#{ chart_name }"))
-            )
-          }
-        end
-
-        r
-      end
-
-      [
-        [
-          ['Univariate Integer data', [1, 2, 3]],
-          { has_charts: ['histogram-quantitative', 'descriptive-statistics-table'] }
-        ],
-        [
-          ['Univariate Decimal data', [1.0, 2.0, 3.0]],
-          { has_charts: ['histogram-quantitative', 'descriptive-statistics-table'] }
-        ],
-        [
-          ['Univariate Temporal data', [DateTime.new(2015,2,7)]],
-          { has_charts: ['histogram-temporal', 'descriptive-statistics-table'] }
-        ],
-        [
-          ['Univariate Categorical data', ['a', 'b', 'c']],
-          { has_charts: ['histogram-categorical', 'pie-chart', 'descriptive-statistics-table'] }
-        ],
-      ].each { |(args, xpect_options)|
-        title, data_set_or_array, chart_specs = args
-
-        it "renders #{ title } correctly" do
-          check_render_expectations(
-            Exploration.new(*args, true).render,
-            xpect_options
-          )
-        end
-
-      }
-
+    let(:exploration) {
+      Exploration.new(
+        'the title',
+        [{ name: 'ds1', values: [1,2,3] }],
+        true
+      )
+    }
+     it 'computes a dom id' do
+      exploration.dom_id.must_match /rde-exploration-./
     end
 
+    # describe "integration test" do
+
+    #   def check_render_expectations(rendered_string, options)
+    #     r = true
+
+    #     if options[:has_charts]
+    #       options[:has_charts].each { |chart_name|
+    #         # TODO: use Nokogiri to test that it is a div class
+    #         rendered_string.must_match(
+    #           Regexp.new(Regexp.escape("rde-#{ chart_name }"))
+    #         )
+    #       }
+    #     end
+
+    #     r
+    #   end
+
+    #   [
+    #     [
+    #       ['Univariate Integer data', [1, 2, 3]],
+    #       { has_charts: ['histogram-quantitative', 'descriptive-statistics-table'] }
+    #     ],
+    #     [
+    #       ['Univariate Decimal data', [1.0, 2.0, 3.0]],
+    #       { has_charts: ['histogram-quantitative', 'descriptive-statistics-table'] }
+    #     ],
+    #     [
+    #       ['Univariate Temporal data', [DateTime.new(2015,2,7)]],
+    #       { has_charts: ['histogram-temporal', 'descriptive-statistics-table'] }
+    #     ],
+    #     [
+    #       ['Univariate Categorical data', ['a', 'b', 'c']],
+    #       { has_charts: ['histogram-categorical', 'pie-chart', 'descriptive-statistics-table'] }
+    #     ],
+    #   ].each { |(args, xpect_options)|
+    #     title, data_set_or_array, chart_specs = args
+
+    #     it "renders #{ title } correctly" do
+    #       check_render_expectations(
+    #         Exploration.new(*args, true).render,
+    #         xpect_options
+    #       )
+    #     end
+
+    #   }
+
+    # end
+
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rails-data-explorer
 version: !ruby/object:Gem::Version
-  version: 1.0.1
+  version: 1.0.2
 platform: ruby
 authors:
 - Jo Hund
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-02-09 00:00:00.000000000 Z
+date: 2015-02-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -166,6 +166,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: yui-compressor
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: rails-data-explorer is a Rails Engine plugin that makes it easy to explore
   the data in your app using charts and statistics.
 email: jhund@clearcove.ca
@@ -183,6 +197,8 @@ files:
 - app/assets/images/rails-data-explorer/multiple_bivariate_small.png
 - doc/how_to/release.md
 - doc/how_to/trouble_when_packaging_assets.md
+- doc/rails-data-explorer-screenshot-all-univariate.png
+- doc/rails-data-explorer-screenshot-plan-bivariate.png
 - doc/rails-data-explorer-screenshot.png
 - lib/rails-data-explorer-no-rails.rb
 - lib/rails-data-explorer.rb