vanity 1.3.0 → 1.4.0.beta

data/CHANGELOG

@@ -1,12 +1,70 @@
+== 1.4.0 (Pending)
+
+Note: Run this command to upgrade your database to 1.4, or you will not have
+access to collected metrics and experiment data:
+
+  vanity upgrade
+
+
+Connection adapters! We have a new way for managing connections which extends
+to multiple adapters (not just Redis). The easiest is to use the configuration
+file config/vanity.yml. For example:
+
+  production:
+	  adapter: redis
+	worker:
+	  adapter: mongodb
+
+Redis is carried forward and we get MongoDB adapter. Instead of using mock
+adapter or calling playground.test!, turn collection off when running outside
+production. Under Rails, collection is turned off in all environment except
+production. You can control it using:
+
+  Vanity.playground.collection = true/false.
+
+
+This release switches from older vendored version of the Redis gem to the new
+2.0.x gem that is now required as a separate dependency. If your own code
+relies on the Redis gem, watch out for subtle incompatibilities introduced in
+2.0.
+
+Now using RVM, gemsets and Bundler to test Vanity in different configurations.
+To run the full set of tests in all the supported versions of Ruby:
+  rake test:rubies
+To test specific version of Ruby:
+  rake test:rubies[1.8.7]
+To switch around:
+  rvm 1.9.2@vanity
+
+* Added: Adapter API, see Vanity::Adapters::AbstractAdapter and
+Vanity::Adapters::RedisAdapter.
+* Added: MongoDB support.
+* Added: Upgrade command.
+* Added: Metric.last_update_at.
+* Added: Vanity.playground.collecting. You want this to be true only in
+production environment. When false, disables collecting of metric and
+experiment data.
+* Added: Remote metrics. Push data to remote service.
+* Added: Partial support for multi-series metrics. Laying the ground for the
+future.
+* Change: Vanity.playground.redis and redis= methods are deprecated, use
+connection and establish_connection instead.
+* Removed: Metric.created_at, derived from experiment and never used.
+
+
 == 1.3.0 (2010-03-01)
-This release adds support for Google Analytics, AdWords and forking servers (Passenger, Unicorn).
 
-To view Google Analytics metrics from within Vanity, first make sure you are using Garb.  For example, in your Gemfile:
+This release adds support for Google Analytics, AdWords and forking servers
+(Passenger, Unicorn).
+
+To view Google Analytics metrics from within Vanity, first make sure you are
+using Garb.  For example, in your Gemfile:
 
   gem "vanity", "1.3.0"
   gem "garb",   "0.5.0" 
 
-Next, authenticate using your account credentials.  For example, in your config/environments/production.rb:
+Next, authenticate using your account credentials.  For example, in your
+config/environments/production.rb:
 
   require "garb"
 	Garb::Session.login('..email..', '..password..', account_type: "GOOGLE") rescue nil

data/Gemfile

@@ -1,16 +1,24 @@
 source "http://rubygems.org/"
-gem "autotest"
-gem "autotest-fsevent"
-gem "autotest-growl"
-gem "bundler"
 gem "garb"
-gem "jekyll"
-gem "mocha"
-gem "passenger"
-gem "rails", "2.3.5"
-gem "rack", "1.0.1"
-gem "rubygems-update"
-gem "shoulda"
-gem "sqlite3-ruby"
-gem "timecop"
-gem "yard"
+gem "redis"
+gem "redis-namespace"
+
+group :development do
+  gem "jekyll"
+  gem "rake"
+  gem "RedCloth"
+  gem "yard"
+end
+
+group :test do
+  gem "bson_ext"
+  gem "mocha"
+  gem "mongo"
+  gem "passenger"
+  gem "rails", "2.3.8"
+  gem "rack", "1.1.0"
+  gem "shoulda"
+  gem "sqlite3-ruby", "1.2.5" # 1.3.0 doesn't like Ruby 1.9.1
+  gem "timecop"
+  gem "webmock"
+end

data/README.rdoc

@@ -2,7 +2,6 @@ Vanity is an Experiment Driven Development framework for Rails.
 
 * All about Vanity: http://vanity.labnotes.org
 * On github: http://github.com/assaf/vanity
-* Vanity requires Redis 1.0 or later.
 
 http://farm3.static.flickr.com/2540/4099665871_497f274f68_o.jpg
 
@@ -60,10 +59,16 @@ And:
 
 To run the test suite for the first time:
 
-  $ bundle install
-  $ bundle exec rake
+  $ rake test:setup test
 
-You can also +rake test+ if you insist on being explicit.
+Vanity is tested against multiple Ruby implementations. To run the full test
+suite against all Ruby implementations and bundled adapters:
+
+  $ rake test:rubies
+  $ # only ruby 1.9.2
+  $ rake test:rubies[1.9.2]
+  $ # only redis
+  $ rake test:adapters[redis]
 
 To build the documentation:
  

data/Rakefile

@@ -1,9 +1,11 @@
 require "rake/testtask"
 
+# -- Building stuff --
+
 spec = Gem::Specification.load(File.expand_path("vanity.gemspec", File.dirname(__FILE__)))
 
 desc "Build the Gem"
-task :build=>:test do
+task :build do
   sh "gem build #{spec.name}.gemspec"
 end
 
@@ -14,7 +16,7 @@ task :install=>:build do
 end
 
 desc "Push new release to gemcutter and git tag"
-task :push=>:build do
+task :push=>["test:rubies", "build"] do
   sh "git push"
   puts "Tagging version #{spec.version} .."
   sh "git tag v#{spec.version}"
@@ -24,10 +26,64 @@ task :push=>:build do
 end
 
 
+# -- Testing stuff --
+
+# Ruby versions we're testing with.
+RUBIES = %w{1.8.7 1.9.1 1.9.2}
+
+# Use rake test:rubies to run all combination of tests (see test:adapters) using
+# all the versions of Ruby specified in RUBIES. Or to test a specific version of
+# Ruby, rake test:rubies[1.8.7].
+#
+# This task uses RVM to install all the Ruby versions it needs, and creates a
+# vanity gemset in each one that includes Bundler and all the gems specified in
+# Gemfile. If anything goes south you can always wipe these gemsets or uninstall
+# these Rubies and start over.
+desc "Test using multiple versions of Ruby"
+task "test:rubies", :ruby do |t, args|
+  rubies = args.ruby ? [args.ruby] : RUBIES
+  rubies.each do |ruby|
+    puts "** Setup #{ruby}"
+    sh "env rvm_install_on_use_flag=1 rvm_gemset_create_on_use_flag=1 rvm use #{ruby}@vanity"
+    sh "rvm #{ruby}@vanity rake test:setup"
+    puts
+    puts "** Test using #{ruby}"
+    sh "rvm #{ruby}@vanity -S bundle exec rake test:adapters #{'--trace' if Rake.application.options.trace}"
+  end
+end
+
+task "test:setup" do
+  # Intended to be used from test:rubies, within specific RVM context.
+  begin # Make sure we got Bundler installed.
+    require "bundler"
+  rescue LoadError
+    sh "gem install bundler"
+  end
+  begin # Make sure we got all the dependencies
+    sh "bundle exec ruby -e puts > /dev/null"
+  rescue
+    sh "bundle install"
+  end
+end
+
+# These are all the adapters we're going to test with.
+ADAPTERS = %w{redis mongodb}
+
+desc "Test using different back-ends"
+task "test:adapters", :adapter do |t, args|
+  adapters = args.adapter ? [args.adapter] : ADAPTERS
+  adapters.each do |adapter|
+    puts "** Testing #{adapter} adapter"
+    sh "rake test ADAPTER=#{adapter} #{'--trace' if Rake.application.options.trace}"
+  end
+end
+
+# Run the test suit.
+
 task :default=>:test
-desc "Run all tests using Redis mock (also default task)"
+desc "Run all tests"
 Rake::TestTask.new do |task|
-  task.test_files = FileList['test/*_test.rb']
+  task.test_files = FileList['test/**/*_test.rb']
   if Rake.application.options.trace
     #task.warning = true
     task.verbose = true
@@ -36,17 +92,14 @@ Rake::TestTask.new do |task|
   else
     task.verbose = true
   end
-end
-
-desc "Run all tests using live redis server"
-task "test:redis" do
-  ENV["REDIS"] = "true"
-  task(:test).invoke
+    task.ruby_opts << "-I."
 end
 
 task(:clobber) { rm_rf "tmp" }
 
 
+# -- Documenting stuff --
+
 begin
   require "yard"
   YARD::Rake::YardocTask.new(:yardoc) do |task|
@@ -59,10 +112,15 @@ end
 desc "Jekyll generates the main documentation (sans API)"
 task(:jekyll) { sh "jekyll", "doc", "html" }
 
+file "html/vanity-api-#{spec.version}.zip"=>:yardoc do |t|
+  Dir.chdir "html" do
+    sh "zip vanity-api-#{spec.version}.zip -r api"
+  end
+end
 desc "Create documentation in docs directory (including API)"
-task :docs=>[:jekyll, :yardoc]
+task :docs=>[:jekyll, :yardoc, "html/vanity-api-#{spec.version}.zip"]
 desc "Remove temporary files and directories"
-task(:clobber) { rm_rf "html" }
+task(:clobber) { rm_rf "html" ; rm_rf ".yardoc" }
 
 desc "Publish documentation to vanity.labnotes.org"
 task :publish=>[:clobber, :docs] do
@@ -70,6 +128,8 @@ task :publish=>[:clobber, :docs] do
 end
 
 
+# -- Misc --
+
 task :report do
   $LOAD_PATH.unshift "lib"
   require "vanity"

data/bin/vanity

@@ -17,6 +17,7 @@ opts = OptionParser.new("", 24, "  ") do |opts|
   opts.banner << "Commands:\n"
   opts.banner << "  list    List all experiments and metrics\n"
   opts.banner << "  report  Report on all running experiments/metrics\n"
+  opts.banner << "  upgrade Upgrade your database when deploying new release\n"
 
   opts.separator ""
   opts.separator "Reporting options:"
@@ -29,8 +30,12 @@ opts = OptionParser.new("", 24, "  ") do |opts|
   opts.on "--load_path PATH", "Path to experiments directory (default: #{playground.load_path})" do |path|
     playground.load_path = path
   end
-  opts.on "--redis HOST:PORT:DB", "Redis server host (default: localhost:6379)" do |redis|
-    playground.redis = redis
+  opts.on "-d", "--database url", "Database connection URL (e.g. redis:/localhost:6379)" do |conn|
+    playground.establish_connection conn
+  end
+  opts.on "--redis HOST:PORT:DB", "DEPRECATED: Redis server host (default: localhost:6379)" do |redis|
+    host, port, db = redis.split(":")
+    playground.establish_connection "redis:/#{host}:#{port}/#{db}"
   end
   opts.on_tail "-h", "--help", "Show this message" do
     puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '')
@@ -50,8 +55,15 @@ end
 
 ARGV.each do |cmd|
   case cmd
-  when "report";  Vanity::Commands.report options.output
-  when "list";    Vanity::Commands.list
+  when "report"
+    require "vanity/commands/report"
+    Vanity::Commands.report options.output
+  when "list"
+    require "vanity/commands/list"
+    Vanity::Commands.list
+  when "upgrade"
+    require "vanity/commands/upgrade"
+    Vanity::Commands.upgrade
   else fail "No such command: #{cmd}"
   end
 end

data/lib/vanity.rb

@@ -1,9 +1,7 @@
 require "date"
 require "time"
 require "logger"
-
-$LOAD_PATH << File.join(File.dirname(__FILE__), "../vendor/redis-rb/lib")
-autoload :Redis, "redis"
+require "cgi"
 
 # All the cool stuff happens in other places.
 # @see Vanity::Helper
@@ -28,13 +26,17 @@ require "vanity/backport" if RUBY_VERSION < "1.9"
 require "vanity/metric/base"
 require "vanity/metric/active_record"
 require "vanity/metric/google_analytics"
+require "vanity/metric/remote"
 # Experiments.
 require "vanity/experiment/base"
 require "vanity/experiment/ab_test"
+# Database adapters
+require "vanity/adapters/abstract_adapter"
+require "vanity/adapters/redis_adapter"
+require "vanity/adapters/mongo_adapter"
+require "vanity/adapters/mock_adapter"
 # Playground.
 require "vanity/playground"
 require "vanity/helpers"
-Vanity.autoload :MockRedis, "vanity/mock_redis"
-Vanity.autoload :Commands, "vanity/commands"
 # Integration with various frameworks.
 require "vanity/frameworks/rails" if defined?(Rails)

data/lib/vanity/adapters/abstract_adapter.rb

@@ -0,0 +1,135 @@
+module Vanity
+  module Adapters
+
+    class << self
+      # Creates new connection to underlying datastore and returns suitable
+      # adapter (adapter object extends AbstractAdapter and wraps the
+      # connection). Vanity.playgroup.establish_connection uses this.
+      #
+      # @since 1.4.0
+      def establish_connection(spec)
+        adapter_method = "#{spec[:adapter]}_connection"
+        send adapter_method, spec
+      end
+    end
+
+    # Base class for all adapters. Adapters wrap underlying connection to a
+    # datastore and implement an API that Vanity can use to store/access
+    # metrics, experiments, etc.
+    class AbstractAdapter
+      # Returns true if connected.
+      def active?
+        false
+      end
+
+      # Close connection, release any resources.
+      def disconnect!
+      end
+
+      # Close and reopen connection.
+      def reconnect!
+      end
+
+      # Empty the database. This is used during tests.
+      def flushdb
+      end
+      
+
+      # -- Metrics --
+     
+      # Return when metric was last updated.
+      def get_metric_last_update_at(metric)
+        fail "Not implemented"
+      end
+  
+      # Track metric data.
+      def metric_track(metric, timestamp, identity, values)
+        fail "Not implemented"
+      end
+
+      # Returns all the metric values between from and to time instances
+      # (inclusive). Returns pairs of date and total count for that date.
+      def metric_values(metric, from, to)
+        fail "Not implemented"
+      end
+
+      # Deletes all information about this metric.
+      def destroy_metric(metric)
+        fail "Not implemented"
+      end
+
+
+      # -- Experiments --
+
+      # Store when experiment was created (do not write over existing value). 
+      def set_experiment_created_at(experiment, time)
+        fail "Not implemented"
+      end
+
+      # Return when experiment was created.
+      def get_experiment_created_at(experiment)
+        fail "Not implemented"
+      end
+     
+      # Returns true if experiment completed. 
+      def is_experiment_completed?(experiment)
+        fail "Not implemented"
+      end
+
+      # Returns counts for given A/B experiment and alternative (by index).
+      # Returns hash with values for the keys :participants, :converted and
+      # :conversions.
+      def ab_counts(experiment, alternative)
+        fail "Not implemented"
+      end 
+
+      # Pick particular alternative (by index) to show to this particular
+      # participant (by identity).
+      def ab_show(experiment, identity, alternative)
+        fail "Not implemented"
+      end
+
+      # Indicates which alternative to show to this participant. See #ab_show.
+      def ab_showing(experiment, identity)
+        fail "Not implemented"
+      end
+
+      # Cancels previously set association between identity and alternative. See
+      # #ab_show.
+      def ab_not_showing(experiment, identity)
+        fail "Not implemented"
+      end
+
+      # Records a participant in this experiment for the given alternative.
+      def ab_add_participant(experiment, alternative, identity)
+        fail "Not implemented"
+      end
+
+      # Records a conversion in this experiment for the given alternative.
+      # Associates a value with the conversion (default to 1). If implicit is
+      # true, add particpant if not already recorded for this experiment. If
+      # implicit is false (default), only add conversion is participant
+      # previously recorded as participating in this experiment.
+      def ab_add_conversion(experiment, alternative, identity, count = 1, implicit = false)
+        fail "Not implemented"
+      end
+
+      # Returns the outcome of this expriment (if set), the index of a
+      # particular alternative.
+      def ab_get_outcome(experiment)
+        fail "Not implemented"
+      end
+
+      # Sets the outcome of this experiment to a particular alternative.
+      def ab_set_outcome(experiment, alternative = 0)
+        fail "Not implemented"
+      end
+
+      # Deletes all information about this experiment.
+      def destroy_experiment(experiment)
+        fail "Not implemented"
+      end
+
+    end
+  end
+end