mikeg-vanity 1.3.0

data/CHANGELOG

@@ -0,0 +1,153 @@
+== 1.3.0
+This release adds support for Google Analytics and AdWords.
+
+To view Google Analytics metrics from within Vanity, first make sure you are using Garb.  For example, in your Gemfile:
+
+  gem "vanity", "1.3.0"
+  gem "garb",   "0.5.0" 
+
+Next, authenticate using your account credentials.  For example, in your config/environments/production.rb:
+
+  require "garb"
+	Garb::Session.login('..email..', '..password..', account_type: "GOOGLE") rescue nil
+
+Last, define Vanity metrics that tap to Google Analytics metrics.  For example:
+
+  metric "Acquisition: Visitors" do
+    description "Unique visitors on any given page, as tracked by Google Analytics"
+    google_analytics "UA-10087469-2", :visitors
+  end
+
+* Added: Support for Google Analytics metrics, thanks to Tony Pitale's Garb and blog post: http://www.viget.com/extend/user-goal-tracking-in-rails-with-vanity-and-google-analytics/
+* Added: Vanity query parameter that you can use to choose a particular alternative, e.g. to tie an advertisement banner with content of the site.
+* Added: Command line "vanity list" catalogs all ongoing experiments, their alternatives (and fingerprints) and all metrics.
+* Changed: Rails integration now separates use_vanity method, filters and helpers.
+* Changed: Explicit vanity_context_filter and vanity_reload_filter so you can skip them, or order filters relative to them.
+* Fixed: if metric cannot be loaded (e.g. offline, no db access) show error message for that metric but don't break dashboard.
+* Removed: Vanity.playground.define is deprecated. Bad choice for a method name. If you need this feature, make a suggestion and let's create a better API.
+
+== 1.2.0 (2009-12-14)
+This release introduces metrics backed by ActiveRecord.  Use them when your model is already tracking a metric, and you get instant historical data.
+
+Example, track sign ups using User model:
+
+  metric "Signups" do
+    model Account
+  end
+
+Example, track satisfaction using Survey model:
+  metric "Satisfaction" do
+    model Survey, :average=>:rating
+  end
+
+Example, track only high ratings:
+  metric "High ratings" do
+    model Rating, :conditions=>["stars >= 4"]
+  end
+
+There's no need to call track! on these metrics.
+
+* Added: Metrics backed by ActiveRecord.
+* Added: track! and ab_test methods now available from Object (i.e. everywhere).
+* Added: Playground.load!. Now loading all metrics and experiments from Rails initializer.
+* Changed: Decoupled metric name from identifier.  You can now define a metric with more descriptive name, e.g. "Cheers per second (user satisfaction)" and keep their ID simple.  Identifier is matched against the file name (for metrics loaded from experiments/metrics).
+* Changed: Metrics no longer defined on-demand, i.e. calling playground.metric either returns existing metric or raises exception.
+* Changed: Playground.experiments returns hash instead of array.
+* Changed: All dates in report are UTC, since we don't know which locale to use.
+* Removed: Object.experiment is deprecated, please call Vanity.playground.experiment directly.
+* Fixed: Playground no longer changes logging level on supplied logger.
+
+== 1.1.1 (2009-12-4)
+* Fixed: Binding issue that shows up on 1.8.6/7.
+
+== 1.1.0 (2009-12-4)
+This release introduces metrics.  Metrics are the gateway drug to better software.
+
+It’s as simple as defining a metric:
+
+	metric "Cheers" do
+		description "They love us, don't they?"
+	end
+
+Tracking it from your code:
+
+	track! :cheers
+
+And watching the graph from the Dashboard.
+
+You can (should) also use metrics with your A/B tests, for example:
+
+  ab_test "Pricing options" do
+	  metrics :signup
+		alternatives 15, 25, 29
+	end
+
+This new usage may become requirement in a future release.
+
+Much thanks to Ian Sefferman for fixing issues with Ruby 1.8.7 and Rails support.
+
+* Added: Metrics.
+* Added: Use Vanity.playground.mock! when running tests and you'd rather not access a live Redis server.
+* Changed: A/B tests now using metrics for tracking.
+* Changed: Now throwing NameError instead of LoadError when failing to load experiment/metric.  NameError can be rescued on same line. 
+* Changed: New, easier URL mapping for Dashboard: map.vanity "/vanity", :controller=>:vanity.
+* Changed: All tests are green on Ruby 1.8.6, 1.8.7 and 1.9.1.
+* Changed: Switched to redis-rb from http://github.com/ezmobius/redis-rb.
+* Deprecated: Please call experiment method with experiment identifier (a symbol) and not experiment name.
+
+== 1.0.0 (2009-11-19)
+This release changes the way you define a new experiment.  You can use a method suitable for the type of experiment you want to define, or call the generic define method (previously: experiment method).  For example:
+
+  ab_test "My A/B test" do
+	  alternatives :a, :b
+	end
+
+The experiment method is no longer overloaded: it looks up an experiment (loading its definition if necessary), but does not define an experiment.  The ab_test method is overloaded, though this may change in the future.
+
+In addition, the ab_goal! method is now track!.  This method may be used for other tests in the future.
+
+* Added: A/B test report now showing number of participants.
+* Added: AbTest.score method accepts minimum probability (default 90), and
+* Removed: Experiment.reset! method. Destroy and save have the same effect.
+* Changed: Playground.define now requires an experiment type, ab_test is not the default any more.
+* Changed: When you run Vanity in development mode (configuration.cache_classes = false), it will reload experiments on each request. You can also Vanity.playground.reload!.
+* Changed: Fancy AJAX trickery in Rails console.
+* Changed: You can break long experiment descriptions into multiple paragraphs using two consecutive newlines.
+* Changed: AbTest confidence becomes probability; only returns choice alternative with probability equal or higher than that.
+* Changed: ab_goal! becomes track!.
+* Changed: Console becomes Dashboard, which is less confusing with rails console (script/console).
+
+== 0.3.1 (2009-11-13)
+* Changed: Redis 1.0 is now vendored into Vanity. This means one less dependecy ... actually two, since Redis brings with it RSpec.
+
+== 0.3.0 (2009-11-13)
+* Added: score now includes least performing alternatives, names and values.
+* Added: shiny reports.
+* Added: Rails console shows current experiments status and also allows you to choose which alternative you want to see.
+* Changed: letters instead of numbers for options (option 1 => option A).
+* Changed: experiment.alternatives is now an immutable snapshot.
+* Changed: experiment.score returns populated alternative objects instead of structs.
+* Changed: experiment.chooses uses Redis to store state, better for (when we get to) browser integration.
+* Changed: experiment.chooses skips recording participant or conversion.
+* Changed: to MIT license.
+
+== 0.2.2 (2009-11-12)
+* Added: vanity binary, with single command for generating a report.
+* Added: return alternative by value from experiment.alternative(val) method.
+* Added: reset an experiment by calling reset!.
+* Added: experiment alternative name (option 1, option 2, etc).
+* Added: new scoring algorithm: use experiment.score instead of alternative.z_score/confidence.
+* Added: experiment.conclusion for plain English results.
+
+== 0.2.1 (2009-11-11)
+* Added: z-score and confidence level for A/B test alternatives.
+* Added: test auto-completion and auto-outcome (complete_it, outcome_is).
+* Changed: default alternatives are now false/true, so if can't decide outcome, fall back on false.
+
+== 0.2.0 (2009-11-10)
+* Added: experiment method on object, used to define and access experiments.
+* Added: playground configuration (Vanity.playground.namespace = , etc).
+* Added: use_vanity now accepts block instead of symbol.
+* Changed: Vanity::Helpers becomes Vanity::Rails.
+* Changed: A/B test experiments alternatives now handled using Alternatives object.
+* Removed: A/B test measure method no longer in use.

data/MIT-LICENSE

@@ -0,0 +1,21 @@
+Copyright (c) 2009 Assaf Arkin
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+

data/README.rdoc

@@ -0,0 +1,83 @@
+Vanity is an Experiment Driven Development framework for Rails.
+
+* All about Vanity: http://vanity.labnotes.org
+* On github: http://github.com/assaf/vanity
+* Vanity requires Redis 1.0 or later.
+
+http://farm3.static.flickr.com/2540/4099665871_497f274f68_o.jpg
+
+
+== A/B Testing With Rails (In 5 Easy Steps)
+
+<b>Step 1:</b> Start using Vanity in your Rails application:
+
+  gem.config "vanity"
+
+And:
+
+  class ApplicationController < ActionController::Base
+    use_vanity :current_user
+  end
+
+<b>Step 2:</b> Define your first A/B test.  This experiment goes in the file <code>experiments/price_options.rb</code>:
+
+  ab_test "Price options" do
+    description "Mirror, mirror on the wall, who's the better price of all?"
+    alternatives 19, 25, 29
+    metrics :signups
+  end
+
+<b>Step 3:</b> Present the different options to your users:
+
+  <h2>Get started for only $<%= ab_test :price_options %> a month!</h2>
+
+<b>Step 4:</b> Measure conversion:
+
+  class SignupController < ApplicationController
+    def signup
+      @account = Account.new(params[:account])
+      if @account.save
+        track! :signups
+        redirect_to @acccount
+      else
+        render action: :offer
+      end
+    end
+  end
+
+<b>Step 5:</b> Check the report:
+
+  vanity --output vanity.html
+
+
+== Building From Source
+
+To run the test suite for the first time:
+
+  $ gem install rails mocha timecop
+  $ rake
+
+You can also +rake test+ if you insist on being explicit.
+
+To build the documentation:
+ 
+  $ gem install yardoc jekyll
+  $ rake docs
+  $ open html/index.html
+
+To clean up after yourself:
+
+  $ rake clobber
+
+To package Vanity as a gem and install on your machine:
+
+  $ rake install
+
+
+== Credits/License
+
+Original code, copyright of Assaf Arkin, released under the MIT license.  
+
+Documentation available under the Creative Commons Attribution license.
+
+For full list of credits and licenses: http://vanity.labnotes.org/credits.html.

data/bin/vanity

@@ -0,0 +1,53 @@
+#!/usr/bin/env ruby
+path = File.expand_path("../lib", File.dirname(__FILE__))
+$LOAD_PATH.unshift path unless $LOAD_PATH.include?(path)
+
+require "vanity"
+require "optparse"
+
+playground = Vanity.playground
+options = Struct.new(:output).new
+opts = OptionParser.new("", 24, "  ") do |opts|
+  opts.banner = "Usage: #{File.basename($0)} [options] command\n"
+  opts.banner << "Commands:\n"
+  opts.banner << "  report  Report on all running experiments/metrics"
+  opts.banner << "  list    List all experiments and metrics"
+
+  opts.separator ""
+  opts.separator "General options:"
+  opts.on("--path PATH", "Path to experiments directory (default: #{playground.load_path})")  { |v| playground.load_path = v }
+  opts.on("--output FILE", "Write report to this file (default: stdout)")                     { |v| options.output = v }
+
+  opts.separator ""
+  opts.separator "Redis options:"
+  opts.on("--host HOST", "Redis server host (default: #{playground.host})")       { |v| playground.host = v }
+  opts.on("--port PORT", "Redis server port (default: #{playground.port})")       { |v| playground.port = v }
+  opts.on("--db DB", "Redis database (default: #{playground.db})")                { |v| playground.db = v }
+  opts.on("--password PWD", "Redis database password")                            { |v| playground.password = v }
+  opts.on("--namespace NS", "Redis namespace (default: #{playground.namespace})") { |v| playground.namespace = v }
+
+  opts.separator ""
+  opts.separator "Common options:"
+  opts.on_tail "-h", "-H", "--help", "Show this message" do
+    puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '')
+    exit
+  end
+  opts.on_tail "-v", "--version", "Show version"  do
+    puts "Vanity #{Vanity::Version::STRING}"
+    exit
+  end
+end
+
+opts.parse!(ARGV)
+if ARGV.empty?
+  puts opts.banner
+  exit
+end
+
+ARGV.each do |cmd|
+  case cmd
+  when "report";  Vanity::Commands.report options.output
+  when "list";    Vanity::Commands.list
+  else fail "No such command: #{cmd}"
+  end
+end

data/lib/vanity.rb

@@ -0,0 +1,38 @@
+require "date"
+require "time"
+require "logger"
+
+$LOAD_PATH << File.join(File.dirname(__FILE__), "../vendor/redis-rb/lib")
+autoload :Redis, "redis"
+
+# All the cool stuff happens in other places.
+# @see Vanity::Helper
+# @see Vanity::Rails
+# @see Vanity::Playground
+# @see Vanity::Metric
+# @see Vanity::Experiment
+module Vanity
+  # Version number.
+  module Version
+    version = Gem::Specification.load(File.expand_path("../vanity.gemspec", File.dirname(__FILE__))).version.to_s.split(".").map { |i| i.to_i }
+    MAJOR = version[0]
+    MINOR = version[1]
+    PATCH = version[2]
+    STRING = "#{MAJOR}.#{MINOR}.#{PATCH}"
+  end
+end
+
+require "vanity/backport" if RUBY_VERSION < "1.9"
+# Metrics.
+require "vanity/metric/base"
+require "vanity/metric/active_record"
+require "vanity/metric/google_analytics" if defined?(Garb)
+# Experiments.
+require "vanity/experiment/base"
+require "vanity/experiment/ab_test"
+# Playground.
+require "vanity/playground"
+require "vanity/helpers"
+Vanity.autoload :MockRedis, "vanity/mock_redis"
+Vanity.autoload :Commands, "vanity/commands"
+require "vanity/rails" if defined?(Rails)

data/lib/vanity/backport.rb

@@ -0,0 +1,43 @@
+class Time
+  unless method_defined?(:to_date)
+    # Backported from Ruby 1.9.
+    def to_date
+      jd = Date.__send__(:civil_to_jd, year, mon, mday, Date::ITALY)
+      Date.new!(Date.__send__(:jd_to_ajd, jd, 0, 0), 0, Date::ITALY)
+    end
+  end
+end
+
+class Date
+  unless method_defined?(:to_date)
+    # Backported from Ruby 1.9.
+    def to_date
+      self
+    end
+  end
+
+  unless method_defined?(:to_time)
+    # Backported from Ruby 1.9.
+    def to_time
+      Time.local(year, mon, mday) 
+    end
+  end
+end
+
+class Symbol
+  unless method_defined?(:to_proc)
+    # Backported from Ruby 1.9.
+    def to_proc
+      Proc.new { |*args| args.shift.__send__(self, *args) }
+    end
+  end
+end
+
+class Array
+  unless method_defined?(:minmax)
+    # Backported from Ruby 1.9.
+    def minmax
+      [min, max]
+    end
+  end
+end 

data/lib/vanity/commands.rb

@@ -0,0 +1,2 @@
+require "vanity/commands/report"
+require "vanity/commands/list"

data/lib/vanity/commands/list.rb

@@ -0,0 +1,21 @@
+module Vanity
+  module Commands
+    class << self
+      # Lists all experiments and metrics.
+      def list
+        Vanity.playground.experiments.each do |id, experiment|
+          puts "experiment :%-.20s (%-.40s)" % [id, experiment.name]
+          if experiment.respond_to?(:alternatives)
+            experiment.alternatives.each do |alt|
+              hash = experiment.fingerprint(alt)
+              puts "  %s: %-40.40s  (%s)" % [alt.name, alt.value, hash]
+            end
+          end
+        end
+        Vanity.playground.metrics.each do |id, metric|
+          puts "metric :%-.20s (%-.40s)" % [id, metric.name]
+        end
+      end
+    end
+  end
+end

data/lib/vanity/commands/report.rb

@@ -0,0 +1,60 @@
+require "erb"
+require "cgi"
+
+module Vanity
+  
+  # Render method available to templates (when used by Vanity command line,
+  # outside Rails).
+  module Render
+    
+    # Render the named template.  Used for reporting and the dashboard.
+    def render(path, locals = {})
+      locals[:playground] = self
+      keys = locals.keys
+      struct = Struct.new(*keys)
+      struct.send :include, Render
+      locals = struct.new(*locals.values_at(*keys))
+      dir, base = File.split(path)
+      path = File.join(dir, "_#{base}")
+      erb = ERB.new(File.read(path), nil, '<>')
+      erb.filename = path
+      erb.result(locals.instance_eval { binding })
+    end
+
+    # Escape HTML.
+    def h(html)
+      CGI.escapeHTML(html)
+    end
+
+    # Dumbed down from Rails' simple_format.
+    def simple_format(text, options={})
+      open = "<p #{options.map { |k,v| "#{k}=\"#{CGI.escapeHTML v}\"" }.join(" ")}>"
+      text = open + text.gsub(/\r\n?/, "\n").   # \r\n and \r -> \n
+        gsub(/\n\n+/, "</p>\n\n#{open}").       # 2+ newline  -> paragraph
+        gsub(/([^\n]\n)(?=[^\n])/, '\1<br />') + # 1 newline   -> br
+        "</p>"
+    end
+  end
+
+  # Commands available when running Vanity from the command line (see bin/vanity).
+  module Commands
+    class << self
+      include Render
+
+      # Generate an HTML report.  Outputs to the named file, or stdout with no
+      # arguments.
+      def report(output = nil)
+        html = render(Vanity.template("report"))
+        if output
+          File.open output, 'w' do |file|
+            file.write html
+          end
+          puts "New report available in #{output}"
+        else
+          $stdout.write html
+        end
+      end
+
+    end
+  end
+end