split 1.4.3 → 1.4.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 86dd69d253b8dd5ddb9296729f89a04d98a482d4
-  data.tar.gz: 561287a2dfc015d8ec5b1751c78ef7852b626edb
+  metadata.gz: 3272611f8e1b67b852c92022c86d10c79d60adfa
+  data.tar.gz: 560dc5662f892e2dd5909c1d1588954bedf11feb
 SHA512:
-  metadata.gz: c34f1b8d4893ada8c64a27deed0536aaa0f0922531abe9180d6c3eb96742dea570b8dedeef341961184b9cd2bc9ef3d0ee439cd18f945d371069006abee3a530
-  data.tar.gz: 7b63e41dc1d105ce2e1e00fd1488d7d149db7b44f5df6b4de6a835ed17532d8511593829babedab310b5258cf7861840a80e0d9bc50c333f04ce91acf997712d
+  metadata.gz: 17cdb89c56142fccc390986303f156d1a71a6c65e1e2759c1ad96394dc8620ccf2515e9d3ecc8fad0f6db7503381c5cb056ac985b2ba235d6130a171f7d82441
+  data.tar.gz: 330ae49fc82430ec36f810f6b07e0a6b78802f9610b531d86995955284e4252fcc37d7d7ed63c69056f6a4c069d125492ceab2f40183ae11864b00a15cf86a4f

data/CHANGELOG.md

@@ -1,3 +1,16 @@
+## 1.4.4 (May 9th, 2016)
+
+Bugfixes:
+
+- Increment participation if store override is true and no experiment key exists (@spheric, #380)
+
+Misc:
+
+- Deprecated `finished` method in favour of `ab_finished` (@andreibondarev, #389)
+- Added minimum version requirement to simple-random
+- Clarify finished with first option being a hash in Readme (@henrik, #382)
+- Refactoring the User abstraction (@andreibondarev, #384)
+
 ## 1.4.3 (April 28th, 2016)
 
 Features:

data/README.md

@@ -62,7 +62,7 @@ To begin your ab test use the `ab_test` method, naming your experiment with the
 
 It can be used to render different templates, show different text or any other case based logic.
 
-`finished` is used to make a completion of an experiment, or conversion.
+`ab_finished` is used to make a completion of an experiment, or conversion.
 
 Example: View
 
@@ -86,14 +86,14 @@ Example: Conversion tracking (in a controller!)
 ```ruby
 def buy_new_points
   # some business logic
-  finished(:new_user_free_points)
+  ab_finished(:new_user_free_points)
 end
 ```
 
 Example: Conversion tracking (in a view)
 
 ```erb
-Thanks for signing up, dude! <% finished(:signup_page_redesign) %>
+Thanks for signing up, dude! <% ab_finished(:signup_page_redesign) %>
 ```
 
 You can find more examples, tutorials and guides on the [wiki](https://github.com/splitrb/split/wiki).
@@ -157,10 +157,10 @@ After choosing this option tests won't be started right after deploy, but after
 
 When a user completes a test their session is reset so that they may start the test again in the future.
 
-To stop this behaviour you can pass the following option to the `finished` method:
+To stop this behaviour you can pass the following option to the `ab_finished` method:
 
 ```ruby
-finished(:experiment_name, reset: false)
+ab_finished(:experiment_name, reset: false)
 ```
 
 The user will then always see the alternative they started with.
@@ -445,7 +445,7 @@ ab_test(:my_first_experiment)
 and:
 
 ```ruby
-finished(:my_first_experiment)
+ab_finished(:my_first_experiment)
 ```
 
 You can also add meta data for each experiment, very useful when you need more than an alternative name to change behaviour:
@@ -515,7 +515,7 @@ Your code may then track a completion using the metric instead of
 the experiment name:
 
 ```ruby
-finished(:my_metric)
+ab_finished(:my_metric)
 ```
 
 You can also create a new metric by instantiating and saving a new Metric object.
@@ -550,20 +550,30 @@ end
 To complete a goal conversion, you do it like:
 
 ```ruby
-finished(link_color: "purchase")
+ab_finished(link_color: "purchase")
 ```
 
-**NOTE:** This does not mean that a single experiment can have/complete progressive goals.
+Note that if you pass additional options, that should be a separate hash:
+
+```ruby
+ab_finished({ link_color: "purchase" }, reset: false)
+```
+
+**NOTE:** This does not mean that a single experiment can complete more than one goal.
+
+Once you finish one of the goals, the test is considered to be completed, and finishing the other goal will no longer register. (Assuming the test runs with `reset: false`.)
 
 **Good Example**: Test if listing Plan A first result in more conversions to Plan A (goal: "plana_conversion") or Plan B (goal: "planb_conversion").
 
 **Bad Example**: Test if button color increases conversion rate through multiple steps of a funnel. THIS WILL NOT WORK.
 
+**Bad Example**: Test both how button color affects signup *and* how it affects login, at the same time. THIS WILL NOT WORK.
+
 ### DB failover solution
 
 Due to the fact that Redis has no automatic failover mechanism, it's
 possible to switch on the `db_failover` config option, so that `ab_test`
-and `finished` will not crash in case of a db failure. `ab_test` always
+and `ab_finished` will not crash in case of a db failure. `ab_test` always
 delivers alternative A (the first one) in that case.
 
 It's also possible to set a `db_failover_on_db_error` callback (proc)

data/lib/split/encapsulated_helper.rb

@@ -4,7 +4,7 @@
 #
 # This module exposes only two methods:
 #  - ab_test()
-#  - finished()
+#  - ab_finished()
 # that can safely be mixed into any class.
 #
 # Passes the instance of the class that it's mixed into to the
@@ -15,7 +15,7 @@ module Split
 
     class ContextShim
       include Split::Helper
-      public :ab_test, :finished
+      public :ab_test, :finished, :ab_finished
 
       def initialize(context)
         @context = context
@@ -44,7 +44,7 @@ module Split
     end
 
     def ab_test_finished(*arguments)
-      warn 'DEPRECATION WARNING: ab_test_finished is deprecated and will be removed from Split 1.5.0'
+      warn 'DEPRECATION WARNING: ab_test_finished is deprecated and will be removed from Split 2.0.0'
       split_context_shim.finished *arguments
     end
 

data/lib/split/helper.rb

@@ -59,7 +59,7 @@ module Split
       end
     end
 
-    def finished(metric_descriptor, options = {:reset => true})
+    def ab_finished(metric_descriptor, options = {:reset => true})
       return if exclude_visitor? || Split.configuration.disabled?
       metric_descriptor, goals = normalize_metric(metric_descriptor)
       experiments = Metric.possible_experiments(metric_descriptor)
@@ -74,6 +74,11 @@ module Split
       Split.configuration.db_failover_on_db_error.call(e)
     end
 
+    def finished(metric_descriptor, options = {:reset => true})
+      warn 'DEPRECATION WARNING: finished method was renamed to ab_finished and will be removed in Split 2.0.0'
+      ab_finished(metric_descriptor, options)
+    end
+
     def override_present?(experiment_name)
       defined?(params) && params[experiment_name]
     end
@@ -87,7 +92,7 @@ module Split
     end
 
     def begin_experiment(experiment, alternative_name = nil)
-      warn 'DEPRECATION WARNING: begin_experiment is deprecated and will be removed from Split 1.5.0'
+      warn 'DEPRECATION WARNING: begin_experiment is deprecated and will be removed from Split 2.0.0'
       alternative_name ||= experiment.control.name
       ab_user[experiment.key] = alternative_name
       alternative_name
@@ -115,16 +120,7 @@ module Split
     end
 
     def active_experiments
-      experiment_pairs = {}
-      ab_user.keys.each do |key|
-        key_without_version = key.split(/\:\d(?!\:)/)[0]
-        Metric.possible_experiments(key_without_version).each do |experiment|
-          if !experiment.has_winner?
-            experiment_pairs[key_without_version] = ab_user[key]
-          end
-        end
-      end
-      return experiment_pairs
+      ab_user.active_experiments
     end
 
     def normalize_metric(metric_descriptor)

data/lib/split/trial.rb

@@ -49,12 +49,15 @@ module Split
     # method is guaranteed to only run once, and will skip the alternative choosing process if run
     # a second time.
     def choose!(context = nil)
-      @user.cleanup_old_experiments
+      @user.cleanup_old_experiments!
       # Only run the process once
       return alternative if @alternative_choosen
 
       if override_is_alternative?
         self.alternative = @options[:override]
+        if should_store_alternative? && !@user[@experiment.key]
+          self.alternative.increment_participation
+        end
       elsif @options[:disabled] || Split.configuration.disabled?
         self.alternative = @experiment.control
       elsif @experiment.has_winner?
@@ -102,22 +105,12 @@ module Split
 
     def cleanup_old_versions
       if @experiment.version > 0
-        keys = @user.keys.select { |k| k.match(Regexp.new(@experiment.name)) }
-        keys_without_experiment(keys).each { |key| @user.delete(key) }
+        @user.cleanup_old_versions!(@experiment)
       end
     end
 
     def exclude_user?
-      @options[:exclude] || @experiment.start_time.nil? || max_experiments_reached?
-    end
-
-    def max_experiments_reached?
-      !Split.configuration.allow_multiple_experiments &&
-          keys_without_experiment(@user.keys).length > 0
-    end
-
-    def keys_without_experiment(keys)
-      keys.reject { |k| k.match(Regexp.new("^#{@experiment.key}(:finished)?$")) }
+      @options[:exclude] || @experiment.start_time.nil? || @user.max_experiments_reached?(@experiment.key)
     end
   end
 end

data/lib/split/user.rb

@@ -8,7 +8,7 @@ module Split
       @user = Split::Persistence.adapter.new(context)
     end
 
-    def cleanup_old_experiments
+    def cleanup_old_experiments!
       user.keys.each do |key|
         experiment = ExperimentCatalog.find key_without_version(key)
         if experiment.nil? || experiment.has_winner? || experiment.start_time.nil?
@@ -17,6 +17,34 @@ module Split
       end
     end
 
+    def max_experiments_reached?(experiment_key)
+      !Split.configuration.allow_multiple_experiments &&
+        keys_without_experiment(user.keys, experiment_key).length > 0
+    end
+
+    def cleanup_old_versions!(experiment)
+      keys = user.keys.select { |k| k.match(Regexp.new(experiment.name)) }
+      keys_without_experiment(keys, experiment.key).each { |key| user.delete(key) }
+    end
+
+    def active_experiments
+      experiment_pairs = {}
+      user.keys.each do |key|
+        Metric.possible_experiments(key_without_version(key)).each do |experiment|
+          if !experiment.has_winner?
+            experiment_pairs[key_without_version(key)] = user[key]
+          end
+        end
+      end
+      experiment_pairs
+    end
+
+    private
+
+    def keys_without_experiment(keys, experiment_key)
+      keys.reject { |k| k.match(Regexp.new("^#{experiment_key}(:finished)?$")) }
+    end
+
     def key_without_version(key)
       key.split(/\:\d(?!\:)/)[0]
     end

data/lib/split/version.rb

@@ -2,6 +2,6 @@
 module Split
   MAJOR   = 1
   MINOR   = 4
-  PATCH   = 3
+  PATCH   = 4
   VERSION = [MAJOR, MINOR, PATCH].join('.')
 end

data/spec/helper_spec.rb

@@ -231,7 +231,7 @@ describe Split::Helper do
     end
   end
 
-  describe 'finished' do
+  describe 'ab_finished' do
     before(:each) do
       @experiment_name = 'link_color'
       @alternatives = ['blue', 'red']
@@ -241,19 +241,19 @@ describe Split::Helper do
     end
 
     it 'should increment the counter for the completed alternative' do
-      finished(@experiment_name)
+      ab_finished(@experiment_name)
       new_completion_count = Split::Alternative.new(@alternative_name, @experiment_name).completed_count
       expect(new_completion_count).to eq(@previous_completion_count + 1)
     end
 
     it "should set experiment's finished key if reset is false" do
-      finished(@experiment_name, {:reset => false})
+      ab_finished(@experiment_name, {:reset => false})
       expect(ab_user[@experiment.key]).to eq(@alternative_name)
       expect(ab_user[@experiment.finished_key]).to eq(true)
     end
 
     it 'should not increment the counter if reset is false and the experiment has been already finished' do
-      2.times { finished(@experiment_name, {:reset => false}) }
+      2.times { ab_finished(@experiment_name, {:reset => false}) }
       new_completion_count = Split::Alternative.new(@alternative_name, @experiment_name).completed_count
       expect(new_completion_count).to eq(@previous_completion_count + 1)
     end
@@ -266,7 +266,7 @@ describe Split::Helper do
       # the button size experiment, finishing it should not increase the
       # completion count for that alternative.
       expect(lambda {
-        finished('button_size')
+        ab_finished('button_size')
       }).not_to change { Split::Alternative.new('small', 'button_size').completed_count }
     end
 
@@ -276,26 +276,26 @@ describe Split::Helper do
       a = ab_test('button_size', 'small', 'big')
       expect(a).to eq('small')
       expect(lambda {
-        finished('button_size')
+        ab_finished('button_size')
       }).not_to change { Split::Alternative.new(a, 'button_size').completed_count }
     end
 
     it "should clear out the user's participation from their session" do
       expect(ab_user[@experiment.key]).to eq(@alternative_name)
-      finished(@experiment_name)
+      ab_finished(@experiment_name)
       expect(ab_user.keys).to be_empty
     end
 
     it "should not clear out the users session if reset is false" do
       expect(ab_user[@experiment.key]).to eq(@alternative_name)
-      finished(@experiment_name, {:reset => false})
+      ab_finished(@experiment_name, {:reset => false})
       expect(ab_user[@experiment.key]).to eq(@alternative_name)
       expect(ab_user[@experiment.finished_key]).to eq(true)
     end
 
     it "should reset the users session when experiment is not versioned" do
       expect(ab_user[@experiment.key]).to eq(@alternative_name)
-      finished(@experiment_name)
+      ab_finished(@experiment_name)
       expect(ab_user.keys).to be_empty
     end
 
@@ -304,26 +304,26 @@ describe Split::Helper do
       @alternative_name = ab_test(@experiment_name, *@alternatives)
 
       expect(ab_user[@experiment.key]).to eq(@alternative_name)
-      finished(@experiment_name)
+      ab_finished(@experiment_name)
       expect(ab_user.keys).to be_empty
     end
 
     it "should do nothing where the experiment was not started by this user" do
       ab_user = nil
-      expect(lambda { finished('some_experiment_not_started_by_the_user') }).not_to raise_exception
+      expect(lambda { ab_finished('some_experiment_not_started_by_the_user') }).not_to raise_exception
     end
 
     context "when on_trial_complete is set" do
       before { Split.configuration.on_trial_complete = :some_method }
       it "should call the method" do
         expect(self).to receive(:some_method)
-        finished(@experiment_name)
+        ab_finished(@experiment_name)
       end
 
       it "should not call the method without alternative" do
         ab_user[@experiment.key] = nil
         expect(self).not_to receive(:some_method)
-        finished(@experiment_name)
+        ab_finished(@experiment_name)
       end
     end
   end
@@ -339,7 +339,7 @@ describe Split::Helper do
       alternative = ab_test(:my_experiment)
       experiment = Split::ExperimentCatalog.find :my_experiment
 
-      finished :my_experiment
+      ab_finished :my_experiment
       expect(ab_user[experiment.key]).to eq(alternative)
       expect(ab_user[experiment.finished_key]).to eq(true)
     end
@@ -369,7 +369,7 @@ describe Split::Helper do
         :metric => :my_metric
       }
       should_finish_experiment :my_experiment
-      finished :my_metric
+      ab_finished :my_metric
     end
 
     it "completes all relevant tests" do
@@ -390,7 +390,7 @@ describe Split::Helper do
       should_finish_experiment :exp_1
       should_finish_experiment :exp_2, false
       should_finish_experiment :exp_3
-      finished :my_metric
+      ab_finished :my_metric
     end
 
     it "passes reset option" do
@@ -404,7 +404,7 @@ describe Split::Helper do
       alternative_name = ab_test(:my_exp)
       exp = Split::ExperimentCatalog.find :my_exp
 
-      finished :my_metric
+      ab_finished :my_metric
       expect(ab_user[exp.key]).to eq(alternative_name)
       expect(ab_user[exp.finished_key]).to be_truthy
     end
@@ -419,7 +419,7 @@ describe Split::Helper do
       alternative_name = ab_test(:my_exp)
       exp = Split::ExperimentCatalog.find :my_exp
 
-      finished :my_metric, :reset => false
+      ab_finished :my_metric, :reset => false
       expect(ab_user[exp.key]).to eq(alternative_name)
       expect(ab_user[exp.finished_key]).to be_truthy
     end
@@ -432,7 +432,7 @@ describe Split::Helper do
       previous_convertion_rate = Split::Alternative.new(alternative_name, 'link_color').conversion_rate
       expect(previous_convertion_rate).to eq(0.0)
 
-      finished('link_color')
+      ab_finished('link_color')
 
       new_convertion_rate = Split::Alternative.new(alternative_name, 'link_color').conversion_rate
       expect(new_convertion_rate).to eq(1.0)
@@ -449,7 +449,7 @@ describe Split::Helper do
 
     it 'should show a finished test' do
       alternative = ab_test('def', '4', '5', '6')
-      finished('def', {:reset => false})
+      ab_finished('def', {:reset => false})
       expect(active_experiments.count).to eq 1
       expect(active_experiments.first[0]).to eq "def"
       expect(active_experiments.first[1]).to eq alternative
@@ -519,7 +519,7 @@ describe Split::Helper do
 
         previous_completion_count = Split::Alternative.new(alternative_name, 'link_color').completed_count
 
-        finished('link_color')
+        ab_finished('link_color')
 
         new_completion_count = Split::Alternative.new(alternative_name, 'link_color').completed_count
 
@@ -573,7 +573,7 @@ describe Split::Helper do
 
         previous_completion_count = Split::Alternative.new(alternative_name, 'link_color').completed_count
 
-        finished('link_color')
+        ab_finished('link_color')
 
         new_completion_count = Split::Alternative.new(alternative_name, 'link_color').completed_count
 
@@ -681,7 +681,7 @@ describe Split::Helper do
       experiment.reset
       expect(experiment.version).to eq(1)
 
-      finished('link_color')
+      ab_finished('link_color')
       alternative = Split::Alternative.new(alternative_name, 'link_color')
       expect(alternative.completed_count).to eq(0)
     end
@@ -707,7 +707,7 @@ describe Split::Helper do
 
       describe 'finished' do
         it 'should raise an exception' do
-          expect(lambda { finished('link_color') }).to raise_error(Errno::ECONNREFUSED)
+          expect(lambda { ab_finished('link_color') }).to raise_error(Errno::ECONNREFUSED)
         end
       end
 
@@ -724,7 +724,7 @@ describe Split::Helper do
 
         it "should return control variable" do
           expect(ab_test('link_color', 'blue', 'red')).to eq('blue')
-          expect(lambda { finished('link_color') }).not_to raise_error
+          expect(lambda { ab_finished('link_color') }).not_to raise_error
         end
       end
     end
@@ -797,7 +797,7 @@ describe Split::Helper do
 
       describe 'finished' do
         it 'should not raise an exception' do
-          expect(lambda { finished('link_color') }).not_to raise_error
+          expect(lambda { ab_finished('link_color') }).not_to raise_error
         end
 
         it 'should call db_failover_on_db_error proc with error as parameter' do
@@ -808,7 +808,7 @@ describe Split::Helper do
           end
 
           expect(Split.configuration.db_failover_on_db_error).to receive(:call)
-          finished('link_color')
+          ab_finished('link_color')
         end
       end
     end
@@ -929,7 +929,7 @@ describe Split::Helper do
     experiment2 = Split::ExperimentCatalog.find_or_create('link_color2', 'blue', 'red')
     alternative_name = ab_test('link_color', 'blue', 'red')
     alternative_name2 = ab_test('link_color2', 'blue', 'red')
-    finished('link_color2')
+    ab_finished('link_color2')
 
     experiment2.alternatives.each do |alt|
       expect(alt.unfinished_count).to eq(0)
@@ -958,7 +958,7 @@ describe Split::Helper do
       end
     end
 
-    describe "finished" do
+    describe "ab_finished" do
       before do
         @alternative_name = ab_test(@experiment, *@alternatives)
       end
@@ -966,7 +966,7 @@ describe Split::Helper do
       it "should increment the counter for the specified-goal completed alternative" do
         expect(lambda {
           expect(lambda {
-            finished({"link_color" => ["purchase"]})
+            ab_finished({"link_color" => ["purchase"]})
           }).not_to change {
             Split::Alternative.new(@alternative_name, @experiment_name).completed_count(@goal2)
           }

data/spec/trial_spec.rb

@@ -239,6 +239,7 @@ describe Split::Trial do
         Split.configuration.store_override = true
         expect(user).to receive("[]=")
         trial.choose!
+        expect(trial.alternative.participant_count).to eq(1)
       end
 
       it "does not store when store_override is false" do

data/spec/user_spec.rb

@@ -4,9 +4,9 @@ require 'split/experiment'
 require 'split/user'
 
 describe Split::User do
-  let(:context) do
-    double(:session => { split: { 'link_color' => 'blue' } })
-  end
+  let(:user_keys) { { 'link_color' => 'blue' } }
+  let(:context) { double(:session => { split:  user_keys }) }
+  let(:experiment) { Split::Experiment.new('link_color') }
 
   before(:each) do
     @subject = described_class.new(context)
@@ -16,11 +16,18 @@ describe Split::User do
     expect(@subject['link_color']).to eq(@subject.user['link_color'])
   end
 
-  context '#cleanup_old_experiments' do
-    let(:experiment) { Split::Experiment.new('link_color') }
+  context '#cleanup_old_versions!' do
+    let(:user_keys) { { 'link_color:1' => 'blue' } }
+
+    it 'removes key if old experiment is found' do
+      @subject.cleanup_old_versions!(experiment)
+      expect(@subject.keys).to be_empty
+    end
+  end 
 
+  context '#cleanup_old_experiments!' do
     it 'removes key if experiment is not found' do
-      @subject.cleanup_old_experiments
+      @subject.cleanup_old_experiments!
       expect(@subject.keys).to be_empty
     end
 
@@ -28,14 +35,14 @@ describe Split::User do
       allow(Split::ExperimentCatalog).to receive(:find).with('link_color').and_return(experiment)
       allow(experiment).to receive(:start_time).and_return(Date.today)
       allow(experiment).to receive(:has_winner?).and_return(true)
-      @subject.cleanup_old_experiments
+      @subject.cleanup_old_experiments!
       expect(@subject.keys).to be_empty
     end
 
     it 'removes key if experiment has not started yet' do
       allow(Split::ExperimentCatalog).to receive(:find).with('link_color').and_return(experiment)
       allow(experiment).to receive(:has_winner?).and_return(false)
-      @subject.cleanup_old_experiments
+      @subject.cleanup_old_experiments!
       expect(@subject.keys).to be_empty
     end    
   end

data/split.gemspec

@@ -23,7 +23,7 @@ Gem::Specification.new do |s|
   s.add_dependency 'redis',           '>= 2.1'
   s.add_dependency 'redis-namespace', '>= 1.1.0'
   s.add_dependency 'sinatra',         '>= 1.2.6'
-  s.add_dependency 'simple-random'
+  s.add_dependency 'simple-random',   '>= 0.9.3'
 
   s.add_development_dependency 'bundler',     '~> 1.10'
   s.add_development_dependency 'coveralls',   '~> 0.8'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: split
 version: !ruby/object:Gem::Version
-  version: 1.4.3
+  version: 1.4.4
 platform: ruby
 authors:
 - Andrew Nesbitt
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-04-28 00:00:00.000000000 Z
+date: 2016-05-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: redis
@@ -58,14 +58,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '0'
+        version: 0.9.3
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '0'
+        version: 0.9.3
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -247,7 +247,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: split
-rubygems_version: 2.5.1
+rubygems_version: 2.6.4
 signing_key: 
 specification_version: 4
 summary: Rack based split testing framework