split 0.6.6 → 0.7.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 2d56d14f269b46f4409460cd7f8805c423d2ba92
4
+ data.tar.gz: 574c8c51ccf9a72efad3aace656e80e95301bd3b
5
+ SHA512:
6
+ metadata.gz: 9be1ddfbab6be25f4458ff893a272523e15b8109727b2eddae3d508db5afed4b810693d61e9c5931b7e5d0ebf36610d442bb08a22869b29c8583ca07e144722f
7
+ data.tar.gz: 77b36b084453e5aaa46be4d9d1b0551e22c51547a4df02a642dd9ebf94c23c5fdb6fa8a01a2fe7be8f5052dce3cc5003a7c29c7b178d4c54ee5c57b098fd0240
data/.gitignore CHANGED
@@ -4,4 +4,7 @@ Gemfile.lock
4
4
  pkg/*
5
5
  *.rbc
6
6
  .idea
7
- coverage
7
+ coverage
8
+ issues.rtf
9
+ dump.rdb
10
+ .gitignore
@@ -1,25 +1,20 @@
1
1
  language: ruby
2
2
  bundler_args: ''
3
3
  rvm:
4
- - 1.8.7
5
4
  - 1.9.2
6
5
  - 1.9.3
7
6
  - 2.0.0
8
7
  - ruby-head
9
- - jruby-18mode
10
8
  - jruby-19mode
11
9
  - jruby-head
12
- - rbx-18mode
10
+
13
11
  - rbx-19mode
14
12
  matrix:
15
13
  allow_failures:
16
14
  - rvm: ruby-head
17
- - rvm: rbx-18mode
18
15
  - rvm: rbx-19mode
19
16
  - rvm: jruby-head
20
17
  exclude:
21
- - rvm: 1.8.7
22
- gemfile: gemfiles/4.0.gemfile
23
18
  - rvm: 1.9.2
24
19
  gemfile: gemfiles/4.0.gemfile
25
20
  - rvm: jruby-18mode
@@ -30,4 +25,7 @@ gemfile:
30
25
  - gemfiles/3.0.gemfile
31
26
  - gemfiles/3.1.gemfile
32
27
  - gemfiles/3.2.gemfile
33
- - gemfiles/4.0.gemfile
28
+ - gemfiles/4.0.gemfile
29
+
30
+ services:
31
+ - redis-server
@@ -1,3 +1,19 @@
1
+ ## 0.7.0 (December 26th, 2013)
2
+
3
+ Features:
4
+
5
+ - Significantly improved z-score algorithm (@caser ,#221)
6
+ - Better sorting of Experiments on dashboard (@wadako111, #218)
7
+
8
+ Bugfixes:
9
+
10
+ - Fixed start button not being displayed in some cases (@vigosan, #219)
11
+
12
+ Misc
13
+
14
+ - Experiment#initialize refactoring (@nberger, #224)
15
+ - Extract ExperimentStore into a seperate class (@nberger, #225)
16
+
1
17
  ## 0.6.6 (October 15th, 2013)
2
18
 
3
19
  Features:
@@ -134,6 +134,20 @@ Thanks for signing up, dude! <% finished("signup_page_redesign") %>
134
134
 
135
135
  You can find more examples, tutorials and guides on the [wiki](https://github.com/andrew/split/wiki).
136
136
 
137
+ ## Statistical Validity
138
+
139
+ Split uses a z test (n>30) of the difference between your control and alternative conversion rates to calculate statistical significance.
140
+
141
+ This means that Split will tell you whether an alternative is better or worse than your control, but it will not distinguish between which alternative is the best in an experiment with multiple alternatives. To find that out, run a new experiment with one of the prior alternatives as the control.
142
+
143
+ Also, as per this [blog post](http://www.evanmiller.org/how-not-to-run-an-ab-test.html) on the pitfalls of A/B testing, it is highly recommended that you determine your requisite sample size for each branch before running the experiment. Otherwise, you'll have an increased rate of false positives (experiments which show a significant effect where really there is none).
144
+
145
+ [Here](http://www.evanmiller.org/ab-testing/sample-size.html) is a sample size calculator for your convenience.
146
+
147
+ Finally, two things should be noted about the dashboard:
148
+ * Split will only tell if you if your experiment is 90%, 95%, or 99% significant. For levels of lesser significance, Split will simply show "insufficient significance."
149
+ * If you have less than 30 participants or 5 conversions for a branch, Split will not calculate significance, as you have not yet gathered enough data.
150
+
137
151
  ## Extras
138
152
 
139
153
  ### Weighted alternatives
@@ -270,6 +284,22 @@ def log_trial_complete(trial)
270
284
  end
271
285
  ```
272
286
 
287
+ #### Views
288
+
289
+ If you are running `ab_test` from a view, you must define your event
290
+ hook callback as a
291
+ [helper_method](http://apidock.com/rails/AbstractController/Helpers/ClassMethods/helper_method)
292
+ in the controller:
293
+
294
+ ``` ruby
295
+ helper_method :log_trial_choose
296
+
297
+ def log_trial_choose(trial)
298
+ logger.info "experiment=%s alternative=%s user=%s" %
299
+ [ trial.experiment.name, trial.alternative, current_user.id ]
300
+ end
301
+ ```
302
+
273
303
  ### Experiment Hooks
274
304
 
275
305
  You can assign a proc that will be called when an experiment is reset or deleted. You can use these hooks to call methods within your application to keep data related to experiments in sync with Split.
@@ -606,6 +636,12 @@ end
606
636
 
607
637
  Ryan bates has produced an excellent 10 minute screencast about split on the Railscasts site: [A/B Testing with Split](http://railscasts.com/episodes/331-a-b-testing-with-split)
608
638
 
639
+ ## Blogposts
640
+
641
+ * [A/B Testing with Split in Ruby on Rails](http://grinnick.com/posts/a-b-testing-with-split-in-ruby-on-rails)
642
+ * [Recipe: A/B testing with KISSMetrics and the split gem](http://robots.thoughtbot.com/post/9595887299/recipe-a-b-testing-with-kissmetrics-and-the-split-gem)
643
+ * [Rails A/B testing with Split on Heroku](http://blog.nathanhumbert.com/2012/02/rails-ab-testing-with-split-on-heroku.html)
644
+
609
645
  ## Contributors
610
646
 
611
647
  Special thanks to the following people for submitting patches:
@@ -3,13 +3,15 @@
3
3
  configuration
4
4
  exceptions
5
5
  experiment
6
+ experiment_catalog
6
7
  extensions
7
8
  helper
8
9
  metric
9
10
  persistence
10
11
  encapsulated_helper
11
12
  trial
12
- version].each do |f|
13
+ version
14
+ zscore].each do |f|
13
15
  require "split/#{f}"
14
16
  end
15
17
 
@@ -1,9 +1,15 @@
1
+ require 'split/zscore'
2
+
3
+ # TODO - take out require and implement using file paths?
4
+
1
5
  module Split
2
6
  class Alternative
3
7
  attr_accessor :name
4
8
  attr_accessor :experiment_name
5
9
  attr_accessor :weight
6
10
 
11
+ include Zscore
12
+
7
13
  def initialize(name, experiment_name)
8
14
  @experiment_name = experiment_name
9
15
  if Hash === name
@@ -84,29 +90,23 @@ module Split
84
90
  end
85
91
 
86
92
  def z_score(goal = nil)
87
- # CTR_E = the CTR within the experiment split
88
- # CTR_C = the CTR within the control split
89
- # E = the number of impressions within the experiment split
90
- # C = the number of impressions within the control split
93
+ # p_a = Pa = proportion of users who converted within the experiment split (conversion rate)
94
+ # p_c = Pc = proportion of users who converted within the control split (conversion rate)
95
+ # n_a = Na = the number of impressions within the experiment split
96
+ # n_c = Nc = the number of impressions within the control split
91
97
 
92
98
  control = experiment.control
93
-
94
99
  alternative = self
95
100
 
96
101
  return 'N/A' if control.name == alternative.name
97
102
 
98
- ctr_e = alternative.conversion_rate(goal)
99
- ctr_c = control.conversion_rate(goal)
100
-
101
-
102
- e = alternative.participant_count
103
- c = control.participant_count
104
-
105
- return 0 if ctr_c.zero?
103
+ p_a = alternative.conversion_rate(goal)
104
+ p_c = control.conversion_rate(goal)
106
105
 
107
- standard_deviation = ((ctr_e / ctr_c**3) * ((e*ctr_e)+(c*ctr_c)-(ctr_c*ctr_e)*(c+e))/(c*e)) ** 0.5
106
+ n_a = alternative.participant_count
107
+ n_c = control.participant_count
108
108
 
109
- z_score = ((ctr_e / ctr_c) - 1) / standard_deviation
109
+ z_score = Split::Zscore.calculate(p_a, n_a, p_c, n_c)
110
110
  end
111
111
 
112
112
  def save
@@ -21,17 +21,16 @@ module Split
21
21
 
22
22
  z = round(z_score.to_s.to_f, 3).abs
23
23
 
24
- if z == 0.0
25
- 'No Change'
26
- elsif z < 1.645
27
- 'no confidence'
28
- elsif z < 1.96
29
- '95% confidence'
30
- elsif z < 2.57
24
+ if z >= 2.58
31
25
  '99% confidence'
26
+ elsif z >= 1.96
27
+ '95% confidence'
28
+ elsif z >= 1.65
29
+ '90% confidence'
32
30
  else
33
- '99.9% confidence'
31
+ 'Insufficient confidence'
34
32
  end
33
+
35
34
  end
36
35
  end
37
36
  end
@@ -0,0 +1,13 @@
1
+ <% if experiment.start_time %>
2
+ <form action="<%= url "/reset/#{experiment.name}" %>" method='post' onclick="return confirmReset()">
3
+ <input type="submit" value="Reset Data">
4
+ </form>
5
+ <% else%>
6
+ <form action="<%= url "/start/#{experiment.name}" %>" method='post'>
7
+ <input type="submit" value="Start">
8
+ </form>
9
+ <% end %>
10
+ <form action="<%= url "/#{experiment.name}" %>" method='post' onclick="return confirmDelete()">
11
+ <input type="hidden" name="_method" value="delete"/>
12
+ <input type="submit" value="Delete" class="red">
13
+ </form>
@@ -14,19 +14,7 @@
14
14
  <% if goal.nil? %>
15
15
  <div class='inline-controls'>
16
16
  <small><%= experiment.start_time ? experiment.start_time.strftime('%Y-%m-%d') : 'Unknown' %></small>
17
- <% if experiment.start_time %>
18
- <form action="<%= url "/reset/#{experiment.name}" %>" method='post' onclick="return confirmReset()">
19
- <input type="submit" value="Reset Data">
20
- </form>
21
- <% else%>
22
- <form action="<%= url "/start/#{experiment.name}" %>" method='post'>
23
- <input type="submit" value="Start">
24
- </form>
25
- <% end %>
26
- <form action="<%= url "/#{experiment.name}" %>" method='post' onclick="return confirmDelete()">
27
- <input type="hidden" name="_method" value="delete"/>
28
- <input type="submit" value="Delete" class="red">
29
- </form>
17
+ <%= erb :_controls, :locals => {:experiment => experiment} %>
30
18
  </div>
31
19
  <% end %>
32
20
  </div>
@@ -2,13 +2,7 @@
2
2
  <div class="experiment-header">
3
3
  <div class='inline-controls'>
4
4
  <small><%= experiment.start_time ? experiment.start_time.strftime('%Y-%m-%d') : 'Unknown' %></small>
5
- <form action="<%= url "/reset/#{experiment.name}" %>" method='post' onclick="return confirmReset()">
6
- <input type="submit" value="Reset Data">
7
- </form>
8
- <form action="<%= url "/#{experiment.name}" %>" method='post' onclick="return confirmDelete()">
9
- <input type="hidden" name="_method" value="delete"/>
10
- <input type="submit" value="Delete" class="red">
11
- </form>
5
+ <%= erb :_controls, :locals => {:experiment => experiment} %>
12
6
  </div>
13
7
  </div>
14
- </div>
8
+ </div>
@@ -6,13 +6,42 @@ module Split
6
6
  attr_accessor :goals
7
7
  attr_accessor :alternatives
8
8
 
9
+ DEFAULT_OPTIONS = {
10
+ :resettable => true,
11
+ }
12
+
9
13
  def initialize(name, options = {})
10
- options = {
11
- :resettable => true,
12
- }.merge(options)
14
+ options = DEFAULT_OPTIONS.merge(options)
13
15
 
14
16
  @name = name.to_s
15
17
 
18
+ alternatives = extract_alternatives_from_options(options)
19
+
20
+ if alternatives.empty? && (exp_config = Split.configuration.experiment_for(name))
21
+ set_alternatives_and_options(
22
+ alternatives: load_alternatives_from_configuration,
23
+ goals: load_goals_from_configuration,
24
+ resettable: exp_config[:resettable],
25
+ algorithm: exp_config[:algorithm]
26
+ )
27
+ else
28
+ set_alternatives_and_options(
29
+ alternatives: alternatives,
30
+ goals: options[:goals],
31
+ resettable: options[:resettable],
32
+ algorithm: options[:algorithm]
33
+ )
34
+ end
35
+ end
36
+
37
+ def set_alternatives_and_options(options)
38
+ self.alternatives = options[:alternatives]
39
+ self.goals = options[:goals]
40
+ self.resettable = options[:resettable]
41
+ self.algorithm = options[:algorithm]
42
+ end
43
+
44
+ def extract_alternatives_from_options(options)
16
45
  alts = options[:alternatives] || []
17
46
 
18
47
  if alts.length == 1
@@ -21,48 +50,24 @@ module Split
21
50
  end
22
51
  end
23
52
 
24
- if alts.empty?
25
- exp_config = Split.configuration.experiment_for(name)
26
- if exp_config
27
- alts = load_alternatives_from_configuration
28
- options[:goals] = load_goals_from_configuration
29
- options[:resettable] = exp_config[:resettable]
30
- options[:algorithm] = exp_config[:algorithm]
31
- end
32
- end
33
-
34
- self.alternatives = alts
35
- self.goals = options[:goals]
36
- self.algorithm = options[:algorithm]
37
- self.resettable = options[:resettable]
53
+ alts
38
54
  end
39
55
 
40
56
  def self.all
41
- Split.redis.smembers(:experiments).map {|e| find(e)}
57
+ ExperimentCatalog.all
42
58
  end
43
59
 
44
60
  # Return experiments without a winner (considered "active") first
45
61
  def self.all_active_first
46
- all.sort_by{|e| e.winner ? 1 : 0} # sort_by hack since true/false isn't sortable
62
+ ExperimentCatalog.all_active_first
47
63
  end
48
64
 
49
65
  def self.find(name)
50
- if Split.redis.exists(name)
51
- obj = self.new name
52
- obj.load_from_redis
53
- else
54
- obj = nil
55
- end
56
- obj
66
+ ExperimentCatalog.find(name)
57
67
  end
58
68
 
59
69
  def self.find_or_create(label, *alternatives)
60
- experiment_name_with_version, goals = normalize_experiment(label)
61
- name = experiment_name_with_version.to_s.split(':')[0]
62
-
63
- exp = self.new name, :alternatives => alternatives, :goals => goals
64
- exp.save
65
- exp
70
+ ExperimentCatalog.find_or_create(label, *alternatives)
66
71
  end
67
72
 
68
73
  def save
@@ -247,17 +252,6 @@ module Split
247
252
 
248
253
  protected
249
254
 
250
- def self.normalize_experiment(label)
251
- if Hash === label
252
- experiment_name = label.keys.first
253
- goals = label.values.first
254
- else
255
- experiment_name = label
256
- goals = []
257
- end
258
- return experiment_name, goals
259
- end
260
-
261
255
  def experiment_config_key
262
256
  "experiment_configurations/#{@name}"
263
257
  end
@@ -0,0 +1,45 @@
1
+ module Split
2
+ class ExperimentCatalog
3
+ def self.all
4
+ Split.redis.smembers(:experiments).map {|e| find(e)}
5
+ end
6
+
7
+ # Return experiments without a winner (considered "active") first
8
+ def self.all_active_first
9
+ all.partition{|e| not e.winner}.map{|es| es.sort_by(&:name)}.flatten
10
+ end
11
+
12
+ def self.find(name)
13
+ if Split.redis.exists(name)
14
+ obj = Experiment.new name
15
+ obj.load_from_redis
16
+ else
17
+ obj = nil
18
+ end
19
+ obj
20
+ end
21
+
22
+ def self.find_or_create(label, *alternatives)
23
+ experiment_name_with_version, goals = normalize_experiment(label)
24
+ name = experiment_name_with_version.to_s.split(':')[0]
25
+
26
+ exp = Experiment.new name, :alternatives => alternatives, :goals => goals
27
+ exp.save
28
+ exp
29
+ end
30
+
31
+ private
32
+
33
+ def self.normalize_experiment(label)
34
+ if Hash === label
35
+ experiment_name = label.keys.first
36
+ goals = label.values.first
37
+ else
38
+ experiment_name = label
39
+ goals = []
40
+ end
41
+ return experiment_name, goals
42
+ end
43
+
44
+ end
45
+ end
@@ -1,6 +1,6 @@
1
1
  module Split
2
2
  MAJOR = 0
3
- MINOR = 6
4
- PATCH = 6
3
+ MINOR = 7
4
+ PATCH = 0
5
5
  VERSION = [MAJOR, MINOR, PATCH].join('.')
6
6
  end
@@ -0,0 +1,56 @@
1
+ module Split
2
+ module Zscore
3
+
4
+ include Math
5
+
6
+ def self.calculate(p1, n1, p2, n2)
7
+ # p_1 = Pa = proportion of users who converted within the experiment split (conversion rate)
8
+ # p_2 = Pc = proportion of users who converted within the control split (conversion rate)
9
+ # n_1 = Na = the number of impressions within the experiment split
10
+ # n_2 = Nc = the number of impressions within the control split
11
+ # s_1 = SEa = standard error of p_1, the estiamte of the mean
12
+ # s_2 = SEc = standard error of p_2, the estimate of the control
13
+ # s_p = SEp = standard error of p_1 - p_2, assuming a pooled variance
14
+ # s_unp = SEunp = standard error of p_1 - p_2, assuming unpooled variance
15
+
16
+ p_1 = p1.to_f
17
+ p_2 = p2.to_f
18
+
19
+ n_1 = n1.to_f
20
+ n_2 = n2.to_f
21
+
22
+ # Perform checks on data to make sure we can validly run our confidence tests
23
+ if n_1 < 30 || n_2 < 30
24
+ error = "Needs 30+ participants."
25
+ return error
26
+ elsif p_1 * n_1 < 5 || p_2 * n_2 < 5
27
+ error = "Needs 5+ conversions."
28
+ return error
29
+ end
30
+
31
+ # Formula for standard error: root(pq/n) = root(p(1-p)/n)
32
+ s_1 = Math.sqrt((p_1)*(1-p_1)/(n_1))
33
+ s_2 = Math.sqrt((p_2)*(1-p_2)/(n_2))
34
+
35
+ # Formula for pooled error of the difference of the means: root(π*(1-π)*(1/na+1/nc)
36
+ # π = (xa + xc) / (na + nc)
37
+ pi = (p_1*n_1 + p_2*n_2)/(n_1 + n_2)
38
+ s_p = Math.sqrt(pi*(1-pi)*(1/n_1 + 1/n_2))
39
+
40
+ # Formula for unpooled error of the difference of the means: root(sa**2/na + sc**2/nc)
41
+ s_unp = Math.sqrt(s_1**2 + s_2**2)
42
+
43
+ # Boolean variable decides whether we can pool our variances
44
+ pooled = s_1/s_2 < 2 && s_2/s_1 < 2
45
+
46
+ # Assign standard error either the pooled or unpooled variance
47
+ se = pooled ? s_p : s_unp
48
+
49
+ # Calculate z-score
50
+ z_score = (p_1 - p_2)/(se)
51
+
52
+ return z_score
53
+
54
+ end
55
+ end
56
+ end
@@ -211,10 +211,45 @@ describe Split::Alternative do
211
211
  end
212
212
 
213
213
  describe 'z score' do
214
- it 'should be zero when the control has no conversions' do
215
- alternative2.z_score.should eql(0)
216
- alternative2.z_score(goal1).should eql(0)
217
- alternative2.z_score(goal2).should eql(0)
214
+
215
+ it "should return an error string when the control has 0 people" do
216
+ alternative2.z_score.should eql("Needs 30+ participants.")
217
+ alternative2.z_score(goal1).should eql("Needs 30+ participants.")
218
+ alternative2.z_score(goal2).should eql("Needs 30+ participants.")
219
+ end
220
+
221
+ it "should return an error string when the data is skewed or incomplete as per the np > 5 test" do
222
+ control = experiment.control
223
+ control.participant_count = 100
224
+ control.set_completed_count(50)
225
+
226
+ alternative2.participant_count = 50
227
+ alternative2.set_completed_count(1)
228
+
229
+ alternative2.z_score.should eql("Needs 5+ conversions.")
230
+ end
231
+
232
+ it "should return a float for a z_score given proper data" do
233
+ control = experiment.control
234
+ control.participant_count = 120
235
+ control.set_completed_count(20)
236
+
237
+ alternative2.participant_count = 100
238
+ alternative2.set_completed_count(25)
239
+
240
+ alternative2.z_score.should be_kind_of(Float)
241
+ alternative2.z_score.should_not eql(0)
242
+ end
243
+
244
+ it "should correctly calculate a z_score given proper data" do
245
+ control = experiment.control
246
+ control.participant_count = 126
247
+ control.set_completed_count(89)
248
+
249
+ alternative2.participant_count = 142
250
+ alternative2.set_completed_count(119)
251
+
252
+ alternative2.z_score.round(2).should eql(2.58)
218
253
  end
219
254
 
220
255
  it "should be N/A for the control" do
@@ -6,12 +6,22 @@ include Split::DashboardHelpers
6
6
  describe Split::DashboardHelpers do
7
7
  describe 'confidence_level' do
8
8
  it 'should handle very small numbers' do
9
- confidence_level(Complex(2e-18, -0.03)).should eql('No Change')
9
+ confidence_level(Complex(2e-18, -0.03)).should eql('Insufficient confidence')
10
10
  end
11
11
 
12
- it "should consider a z-score of 1.645 < z < 1.96 as 95% confident" do
13
- confidence_level(1.80).should eql('95% confidence')
12
+ it "should consider a z-score of 1.65 <= z < 1.96 as 90% confident" do
13
+ confidence_level(1.65).should eql('90% confidence')
14
+ confidence_level(1.80).should eql('90% confidence')
14
15
  end
15
16
 
17
+ it "should consider a z-score of 1.96 <= z < 2.58 as 95% confident" do
18
+ confidence_level(1.96).should eql('95% confidence')
19
+ confidence_level(2.00).should eql('95% confidence')
20
+ end
21
+
22
+ it "should consider a z-score of z >= 2.58 as 95% confident" do
23
+ confidence_level(2.58).should eql('99% confidence')
24
+ confidence_level(3.00).should eql('99% confidence')
25
+ end
16
26
  end
17
27
  end
@@ -14,7 +14,11 @@ describe Split::Dashboard do
14
14
  end
15
15
 
16
16
  let(:experiment) {
17
- Split::Experiment.find_or_create('link_color', 'blue', 'red')
17
+ Split::Experiment.find_or_create("link_color", "blue", "red")
18
+ }
19
+
20
+ let(:experiment_with_goals) {
21
+ Split::Experiment.find_or_create({"link_color" => ["goal_1", "goal_2"]}, "blue", "red")
18
22
  }
19
23
 
20
24
  let(:red_link) { link("red") }
@@ -25,15 +29,34 @@ describe Split::Dashboard do
25
29
  last_response.should be_ok
26
30
  end
27
31
 
28
- it "should start experiment" do
29
- Split.configuration.start_manually = true
30
- experiment
31
- get '/'
32
- last_response.body.should include('Start')
33
-
34
- post "/start/#{experiment.name}"
35
- get '/'
36
- last_response.body.should include('Reset Data')
32
+ context "start experiment manually" do
33
+ before do
34
+ Split.configuration.start_manually = true
35
+ end
36
+
37
+ context "experiment without goals" do
38
+ it "should display a Start button" do
39
+ experiment
40
+ get '/'
41
+ last_response.body.should include('Start')
42
+
43
+ post "/start/#{experiment.name}"
44
+ get '/'
45
+ last_response.body.should include('Reset Data')
46
+ end
47
+ end
48
+
49
+ context "with goals" do
50
+ it "should display a Start button" do
51
+ experiment_with_goals
52
+ get '/'
53
+ last_response.body.should include('Start')
54
+
55
+ post "/start/#{experiment.name}"
56
+ get '/'
57
+ last_response.body.should include('Reset Data')
58
+ end
59
+ end
37
60
  end
38
61
 
39
62
  it "should reset an experiment" do
@@ -437,7 +437,7 @@ describe Split::Helper do
437
437
 
438
438
  before(:each) do
439
439
  Split.configure do |c|
440
- c.ignore_filter = proc{|request| !!"i_am_going_to_be_disabled" }
440
+ c.ignore_filter = proc{|request| true } # ignore everything
441
441
  end
442
442
  end
443
443
 
metadata CHANGED
@@ -1,100 +1,88 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: split
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.6
5
- prerelease:
4
+ version: 0.7.0
6
5
  platform: ruby
7
6
  authors:
8
7
  - Andrew Nesbitt
9
8
  autorequire:
10
9
  bindir: bin
11
10
  cert_chain: []
12
- date: 2013-10-14 00:00:00.000000000 Z
11
+ date: 2013-12-26 00:00:00.000000000 Z
13
12
  dependencies:
14
13
  - !ruby/object:Gem::Dependency
15
14
  name: redis
16
15
  requirement: !ruby/object:Gem::Requirement
17
- none: false
18
16
  requirements:
19
- - - ! '>='
17
+ - - '>='
20
18
  - !ruby/object:Gem::Version
21
19
  version: '2.1'
22
20
  type: :runtime
23
21
  prerelease: false
24
22
  version_requirements: !ruby/object:Gem::Requirement
25
- none: false
26
23
  requirements:
27
- - - ! '>='
24
+ - - '>='
28
25
  - !ruby/object:Gem::Version
29
26
  version: '2.1'
30
27
  - !ruby/object:Gem::Dependency
31
28
  name: redis-namespace
32
29
  requirement: !ruby/object:Gem::Requirement
33
- none: false
34
30
  requirements:
35
- - - ! '>='
31
+ - - '>='
36
32
  - !ruby/object:Gem::Version
37
33
  version: 1.1.0
38
34
  type: :runtime
39
35
  prerelease: false
40
36
  version_requirements: !ruby/object:Gem::Requirement
41
- none: false
42
37
  requirements:
43
- - - ! '>='
38
+ - - '>='
44
39
  - !ruby/object:Gem::Version
45
40
  version: 1.1.0
46
41
  - !ruby/object:Gem::Dependency
47
42
  name: sinatra
48
43
  requirement: !ruby/object:Gem::Requirement
49
- none: false
50
44
  requirements:
51
- - - ! '>='
45
+ - - '>='
52
46
  - !ruby/object:Gem::Version
53
47
  version: 1.2.6
54
48
  type: :runtime
55
49
  prerelease: false
56
50
  version_requirements: !ruby/object:Gem::Requirement
57
- none: false
58
51
  requirements:
59
- - - ! '>='
52
+ - - '>='
60
53
  - !ruby/object:Gem::Version
61
54
  version: 1.2.6
62
55
  - !ruby/object:Gem::Dependency
63
56
  name: simple-random
64
57
  requirement: !ruby/object:Gem::Requirement
65
- none: false
66
58
  requirements:
67
- - - ! '>='
59
+ - - '>='
68
60
  - !ruby/object:Gem::Version
69
61
  version: '0'
70
62
  type: :runtime
71
63
  prerelease: false
72
64
  version_requirements: !ruby/object:Gem::Requirement
73
- none: false
74
65
  requirements:
75
- - - ! '>='
66
+ - - '>='
76
67
  - !ruby/object:Gem::Version
77
68
  version: '0'
78
69
  - !ruby/object:Gem::Dependency
79
70
  name: rake
80
71
  requirement: !ruby/object:Gem::Requirement
81
- none: false
82
72
  requirements:
83
- - - ! '>='
73
+ - - '>='
84
74
  - !ruby/object:Gem::Version
85
75
  version: '0'
86
76
  type: :development
87
77
  prerelease: false
88
78
  version_requirements: !ruby/object:Gem::Requirement
89
- none: false
90
79
  requirements:
91
- - - ! '>='
80
+ - - '>='
92
81
  - !ruby/object:Gem::Version
93
82
  version: '0'
94
83
  - !ruby/object:Gem::Dependency
95
84
  name: bundler
96
85
  requirement: !ruby/object:Gem::Requirement
97
- none: false
98
86
  requirements:
99
87
  - - ~>
100
88
  - !ruby/object:Gem::Version
@@ -102,7 +90,6 @@ dependencies:
102
90
  type: :development
103
91
  prerelease: false
104
92
  version_requirements: !ruby/object:Gem::Requirement
105
- none: false
106
93
  requirements:
107
94
  - - ~>
108
95
  - !ruby/object:Gem::Version
@@ -110,7 +97,6 @@ dependencies:
110
97
  - !ruby/object:Gem::Dependency
111
98
  name: rspec
112
99
  requirement: !ruby/object:Gem::Requirement
113
- none: false
114
100
  requirements:
115
101
  - - ~>
116
102
  - !ruby/object:Gem::Version
@@ -118,7 +104,6 @@ dependencies:
118
104
  type: :development
119
105
  prerelease: false
120
106
  version_requirements: !ruby/object:Gem::Requirement
121
- none: false
122
107
  requirements:
123
108
  - - ~>
124
109
  - !ruby/object:Gem::Version
@@ -126,33 +111,29 @@ dependencies:
126
111
  - !ruby/object:Gem::Dependency
127
112
  name: rack-test
128
113
  requirement: !ruby/object:Gem::Requirement
129
- none: false
130
114
  requirements:
131
- - - ! '>='
115
+ - - '>='
132
116
  - !ruby/object:Gem::Version
133
117
  version: 0.5.7
134
118
  type: :development
135
119
  prerelease: false
136
120
  version_requirements: !ruby/object:Gem::Requirement
137
- none: false
138
121
  requirements:
139
- - - ! '>='
122
+ - - '>='
140
123
  - !ruby/object:Gem::Version
141
124
  version: 0.5.7
142
125
  - !ruby/object:Gem::Dependency
143
126
  name: coveralls
144
127
  requirement: !ruby/object:Gem::Requirement
145
- none: false
146
128
  requirements:
147
- - - ! '>='
129
+ - - '>='
148
130
  - !ruby/object:Gem::Version
149
131
  version: '0'
150
132
  type: :development
151
133
  prerelease: false
152
134
  version_requirements: !ruby/object:Gem::Requirement
153
- none: false
154
135
  requirements:
155
- - - ! '>='
136
+ - - '>='
156
137
  - !ruby/object:Gem::Version
157
138
  version: '0'
158
139
  description:
@@ -190,6 +171,7 @@ files:
190
171
  - lib/split/dashboard/public/dashboard.js
191
172
  - lib/split/dashboard/public/reset.css
192
173
  - lib/split/dashboard/public/style.css
174
+ - lib/split/dashboard/views/_controls.erb
193
175
  - lib/split/dashboard/views/_experiment.erb
194
176
  - lib/split/dashboard/views/_experiment_with_goal_header.erb
195
177
  - lib/split/dashboard/views/index.erb
@@ -198,6 +180,7 @@ files:
198
180
  - lib/split/engine.rb
199
181
  - lib/split/exceptions.rb
200
182
  - lib/split/experiment.rb
183
+ - lib/split/experiment_catalog.rb
201
184
  - lib/split/extensions.rb
202
185
  - lib/split/extensions/array.rb
203
186
  - lib/split/extensions/string.rb
@@ -209,6 +192,7 @@ files:
209
192
  - lib/split/persistence/session_adapter.rb
210
193
  - lib/split/trial.rb
211
194
  - lib/split/version.rb
195
+ - lib/split/zscore.rb
212
196
  - spec/algorithms/weighted_sample_spec.rb
213
197
  - spec/algorithms/whiplash_spec.rb
214
198
  - spec/alternative_spec.rb
@@ -229,27 +213,26 @@ files:
229
213
  homepage: https://github.com/andrew/split
230
214
  licenses:
231
215
  - MIT
216
+ metadata: {}
232
217
  post_install_message:
233
218
  rdoc_options: []
234
219
  require_paths:
235
220
  - lib
236
221
  required_ruby_version: !ruby/object:Gem::Requirement
237
- none: false
238
222
  requirements:
239
- - - ! '>='
223
+ - - '>='
240
224
  - !ruby/object:Gem::Version
241
225
  version: '0'
242
226
  required_rubygems_version: !ruby/object:Gem::Requirement
243
- none: false
244
227
  requirements:
245
- - - ! '>='
228
+ - - '>='
246
229
  - !ruby/object:Gem::Version
247
230
  version: '0'
248
231
  requirements: []
249
232
  rubyforge_project: split
250
- rubygems_version: 1.8.23
233
+ rubygems_version: 2.0.3
251
234
  signing_key:
252
- specification_version: 3
235
+ specification_version: 4
253
236
  summary: Rack based split testing framework
254
237
  test_files:
255
238
  - spec/algorithms/weighted_sample_spec.rb
@@ -268,4 +251,3 @@ test_files:
268
251
  - spec/spec_helper.rb
269
252
  - spec/support/cookies_mock.rb
270
253
  - spec/trial_spec.rb
271
- has_rdoc: