split 0.5.0 → 0.6.0

data/.travis.yml

@@ -10,8 +10,6 @@ rvm:
   - jruby-head
   - rbx-18mode
   - rbx-19mode
-  - ree
 matrix:
   allow_failures:
-    - rvm: ruby-head
-    - rvm: 2.0.0
+    - rvm: ruby-head

data/CHANGELOG.mdown

@@ -1,4 +1,28 @@
-## 0.5.0 (January 28, 2012)
+## 0.6.0 (April 4, 2013)
+
+Features:
+
+  - Support for Ruby 2.0.0 (@phoet, #142)
+  - Multiple Goals (@liujin, #109)
+  - Ignoring IPs using Regular Expressions (@waynemoore, #119)
+  - Added ability to add more bots to the default list (@themgt, #140)
+  - Allow custom configuration of user blocking logic (@phoet , #148)
+
+Bugfixes:
+
+  - Fixed regression in handling of config files (@iangreenleaf, #115)
+  - Fixed completion rate increases for experiments users aren't participating in (@philnash, #67)
+  - Handle exceptions from invalid JSON in cookies (@iangreenleaf, #126)
+
+Misc:
+
+  - updated minimum json version requirement
+  - Refactor Yaml Configuration (@rtwomey, #124)
+  - Refactoring of Experiments (@iangreenleaf @tamird, #117 #118)
+  - Added more known Bots, including Pingdom, Bing, YandexBot (@julesie, @zinkkrysty)
+  - Improved Readme (@iangreenleaf @phoet)
+
+## 0.5.0 (January 28, 2013)
 
 Features:
 
@@ -11,11 +35,6 @@ Bugfixes:
   - Fixed negative number of non-finished rates (@philnash, #83)
   - Fixed behaviour of finished(:reset => false) (@philnash, #88)
   - Only take into consideration positive z-scores (@thomasmaas, #96)
-
-## 0.4.7 (November 1, 2012)
-
-Bugfixes:
-
   - Amended ab_test method to raise ArgumentError if passed integers or symbols as
     alternatives (@buddhamagnet, #81)
 

data/Gemfile

@@ -1,3 +1,3 @@
-source "http://rubygems.org"
+source "https://rubygems.org"
 
 gemspec

data/README.mdown

@@ -82,7 +82,7 @@ end
 
 ## Usage
 
-To begin your ab test use the `ab_test` method, naming your experiment with the first argument and then the different variants which you wish to test on as the other arguments.
+To begin your ab test use the `ab_test` method, naming your experiment with the first argument and then the different alternatives which you wish to test on as the other arguments.
 
 `ab_test` returns one of the alternatives, if a user has already seen that test they will get the same alternative as before, which you can use to split your code on.
 
@@ -92,7 +92,7 @@ It can be used to render different templates, show different text or any other c
 
 Example: View
 
-```ruby
+```erb
 <% ab_test("login_button", "/images/button1.jpg", "/images/button2.jpg") do |button_file| %>
   <%= img_tag(button_file, :alt => "Login!") %>
 <% end %>
@@ -118,8 +118,8 @@ end
 
 Example: Conversion tracking (in a view)
 
-```ruby
-Thanks for signing up, dude! <% finished("signup_page_redesign") >
+```erb
+Thanks for signing up, dude! <% finished("signup_page_redesign") %>
 ```
 
 You can find more examples, tutorials and guides on the [wiki](https://github.com/andrew/split/wiki).
@@ -234,7 +234,7 @@ Then adding this to config/routes.rb
 mount Split::Dashboard, :at => 'split'
 ```
 
-You may want to password protect that page, you can do so with `Rack::Auth::Basic`
+You may want to password protect that page, you can do so with `Rack::Auth::Basic` (in your split initializer file)
 
 ```ruby
 Split::Dashboard.use Rack::Auth::Basic do |username, password|
@@ -242,14 +242,16 @@ Split::Dashboard.use Rack::Auth::Basic do |username, password|
 end
 ```
 
+### Screenshot
+
+![split_screenshot](https://f.cloud.github.com/assets/78887/306152/99c64650-9670-11e2-93f8-197f49495d02.png)
+
 ## Configuration
 
 You can override the default configuration options of Split like so:
 
 ```ruby
 Split.configure do |config|
-  config.robot_regex = /my_custom_robot_regex/
-  config.ignore_ip_addresses << '81.19.48.130'
   config.db_failover = true # handle redis errors gracefully
   config.db_failover_on_db_error = proc{|error| Rails.logger.error(error.message) }
   config.allow_multiple_experiments = true
@@ -258,41 +260,84 @@ Split.configure do |config|
 end
 ```
 
+### Filtering
+
+In most scenarios you don't want to have AB-Testing enabled for web spiders, robots or special groups of users.
+Split provides functionality to filter this based on a predefined, extensible list of bots, IP-lists or custom exclude logic.
+
+```ruby
+Split.configure do |config|
+  # bot config
+  config.robot_regex = /my_custom_robot_regex/ # or
+  config.bots['newbot'] = "Description for bot with 'newbot' user agent, which will be added to config.robot_regex for exclusion"
+
+  # IP config
+  config.ignore_ip_addresses << '81.19.48.130' # or regex: /81\.19\.48\.[0-9]+/
+
+  # or provide your own filter functionality, the default is proc{ |request| is_robot? || is_ignored_ip_address? }
+  config.ignore_filter = proc{ |request| CustomExcludeLogic.excludes?(request) }
+end
+```
+
 ### Experiment configuration
 
 Instead of providing the experiment options inline, you can store them
-in a hash or a configuration file:
-
-    Split.configure do |config|
-      config.experiments = YAML.load_file "config/experiments.yml"
-    end
-
-This hash can control your experiment's variants, weights, algorithm and if the
-experiment resets once finished:
-
-    Split.configure do |config|
-      config.experiments = {
-        :my_first_experiment => {
-          :variants => ["a", "b"],
-          :resettable => false,
-        },
-        :my_second_experiment => {
-          :algorithm => 'Split::Algorithms::Whiplash',
-          :variants => [
-            { :name => "a", :percent => 67 },
-            { :name => "b", :percent => 33 },
-          ]
-        }
-      }
-    end
+in a hash. This hash can control your experiment's alternatives, weights,
+algorithm and if the experiment resets once finished:
+
+```ruby
+Split.configure do |config|
+  config.experiments = {
+    "my_first_experiment" => {
+      :alternatives => ["a", "b"],
+      :resettable => false
+    },
+    "my_second_experiment" => {
+      :algorithm => 'Split::Algorithms::Whiplash',
+      :alternatives => [
+        { :name => "a", :percent => 67 },
+        { :name => "b", :percent => 33 }
+      ]
+    }
+  }
+end
+```
+
+You can also store your experiments in a YAML file:
+
+```ruby
+Split.configure do |config|
+  config.experiments = YAML.load_file "config/experiments.yml"
+end
+```
+
+You can then define the YAML file like:
+
+```yaml
+my_first_experiment:
+  alternatives:
+    - a
+    - b
+my_second_experiment:
+  alternatives:
+    - name: a
+      percent: 67
+    - name: b
+      percent: 33
+  resettable: false
+```
 
 This simplifies the calls from your code:
 
-    ab_test(:my_first_experiment)
+```ruby
+ab_test("my_first_experiment")
+```
 
 and:
 
-    finished(:my_first_experiment)
+```ruby
+finished("my_first_experiment")
+```
 
 #### Metrics
 
@@ -301,28 +346,62 @@ those to complete multiple different experiments without adding more to
 your code. You can use the configuration hash to do this, thanks to
 the `:metric` option.
 
-    Split.configure do |config|
-      config.experiments = {
-        :my_first_experiment => {
-          :variants => ["a", "b"],
-          :metric => :conversion,
-        }
-      }
-    end
+```ruby
+Split.configure do |config|
+  config.experiments = {
+    "my_first_experiment" => {
+      :alternatives => ["a", "b"],
+      :metric => :my_metric,
+    }
+  }
+end
+```
 
 Your code may then track a completion using the metric instead of
 the experiment name:
 
-    finished(:conversion)
-    
-You can also create a new metric by instantiating and saving a new Metric object. 
+```ruby
+finished(:my_metric)
+```
+
+You can also create a new metric by instantiating and saving a new Metric object.
+
+```ruby
+Split::Metric.new(:my_metric)
+Split::Metric.save
+```
+
+#### Goals
 
-    Split::Metric.new(:conversion)
-    Split::Metric.save
+You might wish to allow an experiment to have multiple, distinguishable goals.
+The API to define goals for an experiment is this:
+
+```ruby
+ab_test({"link_color" => ["purchase", "refund"]}, "red", "blue")
+```
+
+or you can you can define them in a configuration file:
+
+```ruby
+Split.configure do |config|
+  config.experiments = {
+    "link_color" => {
+      :alternatives => ["red", "blue"],
+      :goals => ["purchase", "refund"]
+    }
+  }
+end
+```
+
+To complete a goal conversion, you do it like:
+
+```ruby
+finished("link_color" => "purchase")
+```
 
 ### DB failover solution
 
-Due to the fact that Redis has no autom. failover mechanism, it's
+Due to the fact that Redis has no automatic failover mechanism, it's
 possible to switch on the `db_failover` config option, so that `ab_test`
 and `finished` will not crash in case of a db failure. `ab_test` always
 delivers alternative A (the first one) in that case.
@@ -373,7 +452,7 @@ If you're running multiple, separate instances of Split you may want
 to namespace the keyspaces so they do not overlap. This is not unlike
 the approach taken by many memcached clients.
 
-This feature is provided by the [redis-namespace][rs] library, which
+This feature is provided by the [redis-namespace](https://github.com/defunkt/redis-namespace) library, which
 Split uses by default to separate the keys it manages from other keys
 in your Redis server.
 
@@ -388,7 +467,7 @@ is configured.
 
 ## Outside of a Web Session
 
-Split provides the Helper module to facilitate running experiments inside web sessions. 
+Split provides the Helper module to facilitate running experiments inside web sessions.
 
 Alternatively, you can access the underlying Metric, Trial, Experiment and Alternative objects to
 conduct experiments that are not tied to a web session.
@@ -412,16 +491,17 @@ end
 
 ## Algorithms
 
-By default, Split ships with an algorithm that randomly selects from possible alternatives for a traditional a/b test. 
+By default, Split ships with an algorithm that randomly selects from possible alternatives for a traditional a/b test.
 
-An implementation of a bandit algorithm is also provided. 
+An implementation of a bandit algorithm is also provided.
 
-Users may also write their own algorithms. The default algorithm may be specified globally in the configuration file, or on a per experiment basis using the experiments hash of the configuration file. 
+Users may also write their own algorithms. The default algorithm may be specified globally in the configuration file, or on a per experiment basis using the experiments hash of the configuration file.
 
 ## Extensions
 
   - [Split::Export](http://github.com/andrew/split-export) - easily export ab test data out of Split
   - [Split::Analytics](http://github.com/andrew/split-analytics) - push test data to google analytics
+  - [Split::Mongoid](https://github.com/MongoHQ/split-mongoid) - store data in mongoid instead of redis
 
 ## Screencast
 
@@ -436,6 +516,11 @@ Special thanks to the following people for submitting patches:
 * Andrew Appleton
 * Phil Nash
 * Dave Goodchild
+* Ian Young
+* Nathan Woodhull
+* Ville Lautanala
+* Liu Jin
+* Peter Schröder
 
 ## Development
 

data/lib/split.rb

@@ -48,7 +48,7 @@ module Split
   #
   # @example
   #   Split.configure do |config|
-  #     config.ignore_ips = '192.168.2.1'
+  #     config.ignore_ip_addresses = '192.168.2.1'
   #   end
   def configure
     self.configuration ||= Configuration.new

data/lib/split/alternative.rb

@@ -19,50 +19,71 @@ module Split
       name
     end
 
+    def goals
+      self.experiment.goals
+    end
+
     def participant_count
-      @participant_count ||= Split.redis.hget(key, 'participant_count').to_i
+      Split.redis.hget(key, 'participant_count').to_i
     end
 
     def participant_count=(count)
-      @participant_count = count
       Split.redis.hset(key, 'participant_count', count.to_i)
     end
 
-    def completed_count
-      @completed_count ||= Split.redis.hget(key, 'completed_count').to_i
+    def completed_count(goal = nil)
+      field = set_field(goal)
+      Split.redis.hget(key, field).to_i
     end
-    
+
+    def all_completed_count
+      if goals.empty?
+        completed_count
+      else
+        goals.inject(completed_count) do |sum, g|
+          sum + completed_count(g)
+        end
+      end
+    end
+
     def unfinished_count
-      participant_count - completed_count
+      participant_count - all_completed_count
+    end
+
+    def set_field(goal)
+      field = "completed_count"
+      field += ":" + goal unless goal.nil?
+      return field
     end
 
-    def completed_count=(count)
-      @completed_count = count
-      Split.redis.hset(key, 'completed_count', count.to_i)
+    def set_completed_count (count, goal = nil)
+      field = set_field(goal)
+      Split.redis.hset(key, field, count.to_i)
     end
 
     def increment_participation
-      @participant_count = Split.redis.hincrby key, 'participant_count', 1
+      Split.redis.hincrby key, 'participant_count', 1
     end
 
-    def increment_completion
-       @completed_count = Split.redis.hincrby key, 'completed_count', 1
+    def increment_completion(goal = nil)
+      field = set_field(goal)
+      Split.redis.hincrby(key, field, 1)
     end
 
     def control?
       experiment.control.name == self.name
     end
 
-    def conversion_rate
+    def conversion_rate(goal = nil)
       return 0 if participant_count.zero?
-      (completed_count.to_f/participant_count.to_f)
+      (completed_count(goal).to_f)/participant_count.to_f
     end
 
     def experiment
       Split::Experiment.find(experiment_name)
     end
 
-    def z_score
+    def z_score(goal = nil)
       # CTR_E = the CTR within the experiment split
       # CTR_C = the CTR within the control split
       # E = the number of impressions within the experiment split
@@ -74,8 +95,9 @@ module Split
 
       return 'N/A' if control.name == alternative.name
 
-      ctr_e = alternative.conversion_rate
-      ctr_c = control.conversion_rate
+      ctr_e = alternative.conversion_rate(goal)
+      ctr_c = control.conversion_rate(goal)
+
 
       e = alternative.participant_count
       c = control.participant_count
@@ -92,28 +114,34 @@ module Split
       Split.redis.hsetnx key, 'completed_count', 0
     end
 
+    def validate!
+      unless String === @name || hash_with_correct_values?(@name)
+        raise ArgumentError, 'Alternative must be a string'
+      end
+    end
+
     def reset
-      @participant_count = nil
-      @completed_count = nil
       Split.redis.hmset key, 'participant_count', 0, 'completed_count', 0
+      unless goals.empty?
+        goals.each do |g|
+          field = "completed_count:#{g}"
+          Split.redis.hset key, field, 0
+        end
+      end
     end
 
     def delete
       Split.redis.del(key)
     end
 
-    def self.valid?(name)
-      String === name || hash_with_correct_values?(name)
-    end
+    private
 
-    def self.hash_with_correct_values?(name)
+    def hash_with_correct_values?(name)
       Hash === name && String === name.keys.first && Float(name.values.first) rescue false
     end
 
-    private
-
     def key
       "#{experiment_name}:#{name}"
     end
   end
-end
+end