split 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,4 @@
1
+ *.gem
2
+ .bundle
3
+ Gemfile.lock
4
+ pkg/*
data/Gemfile ADDED
@@ -0,0 +1,3 @@
1
+ source "http://rubygems.org"
2
+
3
+ gemspec
@@ -0,0 +1,177 @@
1
+ # Split
2
+ ## Rack based split testing framework
3
+
4
+ Split is a rack based ab testing framework designed to work with Rails, Sinatra or any other rack based app.
5
+
6
+ Split is heavily inspired by the Abingo and Vanity rails ab testing plugins and Resque in its use of Redis.
7
+
8
+ ## Requirements
9
+
10
+ Split uses redis as a datastore.
11
+
12
+ If you're on OS X, Homebrew is the simplest way to install Redis:
13
+
14
+ $ brew install redis
15
+ $ redis-server /usr/local/etc/redis.conf
16
+
17
+ You now have a Redis daemon running on 6379.
18
+
19
+ ## Setup
20
+
21
+ If you are using bundler add split to your Gemfile:
22
+
23
+ gem 'split'
24
+
25
+ Then run:
26
+
27
+ bundle install
28
+
29
+ Otherwise install the gem:
30
+
31
+ gem install split
32
+
33
+ and require it in your project:
34
+
35
+ require 'split'
36
+
37
+ ### Rails
38
+
39
+ Split is autoloaded when rails starts up, as long as you've configured redis it will 'just work'.
40
+
41
+ ### Sinatra
42
+
43
+ To configure sinatra with Split you need to enable sessions and mix in the helper methods. Add the following lines at the top of your sinatra app:
44
+
45
+ class MySinatraApp < Sinatra::Base
46
+ enable :sessions
47
+ helpers Split::Helper
48
+
49
+ get '/' do
50
+ ...
51
+ end
52
+
53
+ ## Usage
54
+
55
+ To begin your ab test use the `ab_test` method, naming your experiment with the first argument and then the different variants which you wish to test on as the other arguments.
56
+
57
+ `ab_test` returns one of the alternatives, if a user has already seen that test they will get the same alternative as before, which you can use to split your code on.
58
+
59
+ It can be used to render different templates, show different text or any other case based logic.
60
+
61
+ `finished` is used to make a completion of an experiment, or conversion.
62
+
63
+ Example: View
64
+
65
+ <% ab_test("login_button", "/images/button1.jpg", "/images/button2.jpg") do |button_file| %>
66
+ <%= img_tag(button_file, :alt => "Login!") %>
67
+ <% end %>
68
+
69
+ Example: Controller
70
+
71
+ def register_new_user
72
+ #See what level of free points maximizes users' decision to buy replacement points.
73
+ @starter_points = ab_test("new_user_free_points", 100, 200, 300)
74
+ end
75
+
76
+ Example: Conversion tracking (in a controller!)
77
+
78
+ def buy_new_points
79
+ #some business logic
80
+ finished("buy_new_points") #Either a conversion named with :conversion or a test name.
81
+ end
82
+
83
+ Example: Conversion tracking (in a view)
84
+
85
+ Thanks for signing up, dude! <% finished("signup_page_redesign") >
86
+
87
+ ## Web Interface
88
+
89
+ Split comes with a Sinatra-based front end to get an overview of how your experiments are doing.
90
+
91
+ You can mount this inside your app using Rack::URLMap in your `config.ru`
92
+
93
+ require 'split/dashboard'
94
+
95
+ run Rack::URLMap.new \
96
+ "/" => Your::App.new,
97
+ "/split" => Split::Dashboard.new
98
+
99
+ You may want to password protect that page, you can do so with `Rack::Auth::Basic`
100
+
101
+ Split::Dashboard.use Rack::Auth::Basic do |username, password|
102
+ username == 'admin' && password == 'p4s5w0rd'
103
+ end
104
+
105
+ ## Configuration
106
+
107
+ You may want to change the Redis host and port Split connects to, or
108
+ set various other options at startup.
109
+
110
+ Split has a `redis` setter which can be given a string or a Redis
111
+ object. This means if you're already using Redis in your app, Split
112
+ can re-use the existing connection.
113
+
114
+ String: `Split.redis = 'localhost:6379'`
115
+
116
+ Redis: `Split.redis = $redis`
117
+
118
+ For our rails app we have a `config/initializers/split.rb` file where
119
+ we load `config/split.yml` by hand and set the Redis information
120
+ appropriately.
121
+
122
+ Here's our `config/split.yml`:
123
+
124
+ development: localhost:6379
125
+ test: localhost:6379
126
+ staging: redis1.example.com:6379
127
+ fi: localhost:6379
128
+ production: redis1.example.com:6379
129
+
130
+ And our initializer:
131
+
132
+ rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
133
+ rails_env = ENV['RAILS_ENV'] || 'development'
134
+
135
+ split_config = YAML.load_file(rails_root + '/config/split.yml')
136
+ Split.redis = split_config[rails_env]
137
+
138
+ ## Namespaces
139
+
140
+ If you're running multiple, separate instances of Split you may want
141
+ to namespace the keyspaces so they do not overlap. This is not unlike
142
+ the approach taken by many memcached clients.
143
+
144
+ This feature is provided by the [redis-namespace][rs] library, which
145
+ Split uses by default to separate the keys it manages from other keys
146
+ in your Redis server.
147
+
148
+ Simply use the `Split.redis.namespace` accessor:
149
+
150
+ Split.redis.namespace = "resque:GitHub"
151
+
152
+ We recommend sticking this in your initializer somewhere after Redis
153
+ is configured.
154
+
155
+ ## Contributors
156
+
157
+ Special thanks to the following people for submitting patches:
158
+
159
+ * Lloyd Pick
160
+ * Jeffery Chupp
161
+
162
+ ## Note on Patches/Pull Requests
163
+
164
+ * Fork the project.
165
+ * Make your feature addition or bug fix.
166
+ * Add tests for it. This is important so I don't break it in a
167
+ future version unintentionally.
168
+ * Commit, do not mess with rakefile, version, or history.
169
+ (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
170
+ * Send me a pull request. Bonus points for topic branches.
171
+
172
+ ## Copyright
173
+
174
+ Copyright (c) 2011 Andrew Nesbitt. See LICENSE for details.
175
+
176
+
177
+ n.b don't pass the same alternative twice!
@@ -0,0 +1,8 @@
1
+ require 'bundler'
2
+ Bundler::GemHelper.install_tasks
3
+
4
+ require 'rspec/core/rake_task'
5
+
6
+ RSpec::Core::RakeTask.new('spec')
7
+
8
+ task :default => :spec
@@ -0,0 +1,50 @@
1
+ require 'rubygems'
2
+ require 'split/experiment'
3
+ require 'split/alternative'
4
+ require 'split/helper'
5
+ require 'redis/namespace'
6
+
7
+ module Split
8
+ extend self
9
+ # Accepts:
10
+ # 1. A 'hostname:port' string
11
+ # 2. A 'hostname:port:db' string (to select the Redis db)
12
+ # 3. A 'hostname:port/namespace' string (to set the Redis namespace)
13
+ # 4. A redis URL string 'redis://host:port'
14
+ # 5. An instance of `Redis`, `Redis::Client`, `Redis::DistRedis`,
15
+ # or `Redis::Namespace`.
16
+ def redis=(server)
17
+ if server.respond_to? :split
18
+ if server =~ /redis\:\/\//
19
+ redis = Redis.connect(:url => server, :thread_safe => true)
20
+ else
21
+ server, namespace = server.split('/', 2)
22
+ host, port, db = server.split(':')
23
+ redis = Redis.new(:host => host, :port => port,
24
+ :thread_safe => true, :db => db)
25
+ end
26
+ namespace ||= :split
27
+
28
+ @redis = Redis::Namespace.new(namespace, :redis => redis)
29
+ elsif server.respond_to? :namespace=
30
+ @redis = server
31
+ else
32
+ @redis = Redis::Namespace.new(:split, :redis => server)
33
+ end
34
+ end
35
+
36
+ # Returns the current Redis connection. If none has been created, will
37
+ # create a new one.
38
+ def redis
39
+ return @redis if @redis
40
+ self.redis = 'localhost:6379'
41
+ self.redis
42
+ end
43
+ end
44
+
45
+ if defined?(Rails)
46
+ class ActionController::Base
47
+ ActionController::Base.send :include, Split::Helper
48
+ ActionController::Base.helper Split::Helper
49
+ end
50
+ end
@@ -0,0 +1,77 @@
1
+ module Split
2
+ class Alternative
3
+ attr_accessor :name
4
+ attr_accessor :participant_count
5
+ attr_accessor :completed_count
6
+ attr_accessor :experiment_name
7
+
8
+ def initialize(name, experiment_name, counters = {})
9
+ @experiment_name = experiment_name
10
+ @name = name
11
+ @participant_count = counters['participant_count'].to_i
12
+ @completed_count = counters['completed_count'].to_i
13
+ end
14
+
15
+ def increment_participation
16
+ @participant_count +=1
17
+ self.save
18
+ end
19
+
20
+ def increment_completion
21
+ @completed_count +=1
22
+ self.save
23
+ end
24
+
25
+ def conversion_rate
26
+ return 0 if participant_count.zero?
27
+ (completed_count.to_f/participant_count.to_f)
28
+ end
29
+
30
+ def z_score
31
+ # CTR_E = the CTR within the experiment split
32
+ # CTR_C = the CTR within the control split
33
+ # E = the number of impressions within the experiment split
34
+ # C = the number of impressions within the control split
35
+
36
+ experiment = Split::Experiment.find(@experiment_name)
37
+ control = experiment.alternatives[0]
38
+ alternative = self
39
+
40
+ return 'N/A' if control.name == alternative.name
41
+
42
+ ctr_e = alternative.conversion_rate
43
+ ctr_c = control.conversion_rate
44
+
45
+ e = alternative.participant_count
46
+ c = control.participant_count
47
+
48
+ standard_deviation = ((ctr_e / ctr_c**3) * ((e*ctr_e)+(c*ctr_c)-(ctr_c*ctr_e)*(c+e))/(c*e)) ** 0.5
49
+
50
+ z_score = ((ctr_e / ctr_c) - 1) / standard_deviation
51
+ end
52
+
53
+ def save
54
+ if Split.redis.hgetall("#{experiment_name}:#{name}")
55
+ Split.redis.hset "#{experiment_name}:#{name}", 'participant_count', @participant_count
56
+ Split.redis.hset "#{experiment_name}:#{name}", 'completed_count', @completed_count
57
+ else
58
+ Split.redis.hmset "#{experiment_name}:#{name}", 'participant_count', 'completed_count', @participant_count, @completed_count
59
+ end
60
+ end
61
+
62
+ def self.find(name, experiment_name)
63
+ counters = Split.redis.hgetall "#{experiment_name}:#{name}"
64
+ self.new(name, experiment_name, counters)
65
+ end
66
+
67
+ def self.find_or_create(name, experiment_name)
68
+ self.find(name, experiment_name) || self.create(name, experiment_name)
69
+ end
70
+
71
+ def self.create(name, experiment_name)
72
+ alt = self.new(name, experiment_name)
73
+ alt.save
74
+ alt
75
+ end
76
+ end
77
+ end
@@ -0,0 +1,35 @@
1
+ require 'sinatra/base'
2
+ require 'split'
3
+
4
+ module Split
5
+ class Dashboard < Sinatra::Base
6
+ dir = File.dirname(File.expand_path(__FILE__))
7
+
8
+ set :views, "#{dir}/dashboard/views"
9
+ set :public, "#{dir}/dashboard/public"
10
+ set :static, true
11
+
12
+ helpers do
13
+ def url(*path_parts)
14
+ [ path_prefix, path_parts ].join("/").squeeze('/')
15
+ end
16
+
17
+ def path_prefix
18
+ request.env['SCRIPT_NAME']
19
+ end
20
+ end
21
+
22
+ get '/' do
23
+ @experiments = Split::Experiment.all
24
+ erb :index
25
+ end
26
+
27
+ post '/:experiment' do
28
+ @experiment = Split::Experiment.find(params[:experiment])
29
+ @alternative = Split::Alternative.find(params[:alternative], params[:experiment])
30
+ @experiment.winner = @alternative.name
31
+ @experiment.save
32
+ redirect url('/')
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,48 @@
1
+ html, body, div, span, applet, object, iframe,
2
+ h1, h2, h3, h4, h5, h6, p, blockquote, pre,
3
+ a, abbr, acronym, address, big, cite, code,
4
+ del, dfn, em, font, img, ins, kbd, q, s, samp,
5
+ small, strike, strong, sub, sup, tt, var,
6
+ dl, dt, dd, ul, li,
7
+ form, label, legend,
8
+ table, caption, tbody, tfoot, thead, tr, th, td {
9
+ margin: 0;
10
+ padding: 0;
11
+ border: 0;
12
+ outline: 0;
13
+ font-weight: inherit;
14
+ font-style: normal;
15
+ font-size: 100%;
16
+ font-family: inherit;
17
+ }
18
+
19
+ :focus {
20
+ outline: 0;
21
+ }
22
+
23
+ body {
24
+ line-height: 1;
25
+ }
26
+
27
+ ul {
28
+ list-style: none;
29
+ }
30
+
31
+ table {
32
+ border-collapse: collapse;
33
+ border-spacing: 0;
34
+ }
35
+
36
+ caption, th, td {
37
+ text-align: left;
38
+ font-weight: normal;
39
+ }
40
+
41
+ blockquote:before, blockquote:after,
42
+ q:before, q:after {
43
+ content: "";
44
+ }
45
+
46
+ blockquote, q {
47
+ quotes: "" "";
48
+ }
@@ -0,0 +1,34 @@
1
+ html { background:#efefef; font-family:Arial, Verdana, sans-serif; font-size:13px; }
2
+ body { padding:0; margin:0; }
3
+
4
+ .header { background:#000; padding:8px 5% 0 5%; border-bottom:1px solid #444;border-bottom:5px solid #0080FF;}
5
+ .header h1 { color:#333; font-size:90%; font-weight:bold; margin-bottom:6px;}
6
+ .header ul li { display:inline;}
7
+ .header ul li a { color:#fff; text-decoration:none; margin-right:10px; display:inline-block; padding:8px; -webkit-border-top-right-radius:6px; -webkit-border-top-left-radius:6px; -moz-border-radius-topleft:6px; -moz-border-radius-topright:6px; }
8
+ .header ul li a:hover { background:#333;}
9
+ .header ul li.current a { background:#0080FF; font-weight:bold; color:#fff;}
10
+
11
+ #main { padding:10px 5%; background:#fff; overflow:hidden; }
12
+ #main .logo { float:right; margin:10px;}
13
+ #main span.hl { background:#efefef; padding:2px;}
14
+ #main h1 { margin:10px 0; font-size:190%; font-weight:bold; color:#0080FF;}
15
+ #main h2 { margin:10px 0; font-size:130%;}
16
+ #main table { width:100%; margin:10px 0;}
17
+ #main table tr td, #main table tr th { border:1px solid #ccc; padding:6px;}
18
+ #main table tr th { background:#efefef; color:#888; font-size:80%; font-weight:bold;}
19
+ #main table tr td.no-data { text-align:center; padding:40px 0; color:#999; font-style:italic; font-size:130%;}
20
+ #main a { color:#111;}
21
+ #main p { margin:5px 0;}
22
+ #main p.intro { margin-bottom:15px; font-size:85%; color:#999; margin-top:0; line-height:1.3;}
23
+ #main h1.wi { margin-bottom:5px;}
24
+ #main p.sub { font-size:95%; color:#999;}
25
+
26
+ #main table.queues { width:40%;}
27
+
28
+ #main table .totals td{ background:#eee; font-weight:bold; }
29
+
30
+ #footer { padding:10px 5%; background:#efefef; color:#999; font-size:85%; line-height:1.5; border-top:5px solid #ccc; padding-top:10px;}
31
+ #footer p a { color:#999;}
32
+
33
+
34
+
@@ -0,0 +1,56 @@
1
+ <h1>Split Dashboard</h1>
2
+ <p class="intro">The list below contains all the registered experiments along with the number of test participants, completed and conversion rate currently in the system.</p>
3
+
4
+ <% @experiments.each do |experiment| %>
5
+ <h2><%= experiment.name %></h2>
6
+ <table class="queues">
7
+ <tr>
8
+ <th>Alternative Name</th>
9
+ <th>Participants</th>
10
+ <th>Non-finished</th>
11
+ <th>Completed</th>
12
+ <th>Conversion Rate</th>
13
+ <th>Z-Score</th>
14
+ <th>Winner</th>
15
+ </tr>
16
+
17
+ <% total_participants = total_completed = 0 %>
18
+ <% experiment.alternatives.each do |alternative| %>
19
+ <tr>
20
+ <td><%= alternative.name %></td>
21
+ <td><%= alternative.participant_count %></td>
22
+ <td><%= alternative.participant_count - alternative.completed_count %></td>
23
+ <td><%= alternative.completed_count %></td>
24
+ <td><%= (alternative.conversion_rate * 100).round(2) %>%</td>
25
+ <td><%= alternative.z_score %></td>
26
+ <td>
27
+ <% if experiment.winner %>
28
+ <% if experiment.winner.name == alternative.name %>
29
+ Winner
30
+ <% else %>
31
+ Loser
32
+ <% end %>
33
+ <% else %>
34
+ <form action="<%= url experiment.name %>" method='post'>
35
+ <input type='hidden' name='alternative' value='<%= alternative.name %>'>
36
+ <input type="submit" value="Use this">
37
+ </form>
38
+ <% end %>
39
+ </td>
40
+ </tr>
41
+
42
+ <% total_participants += alternative.participant_count %>
43
+ <% total_completed += alternative.completed_count %>
44
+ <% end %>
45
+
46
+ <tr class="totals">
47
+ <td>Totals</td>
48
+ <td><%= total_participants %></td>
49
+ <td><%= total_participants - total_completed %></td>
50
+ <td><%= total_completed %></td>
51
+ <td>N/A</td>
52
+ <td>N/A</td>
53
+ <td>N/A</td>
54
+ </tr>
55
+ </table>
56
+ <% end %>
@@ -0,0 +1,22 @@
1
+ <!DOCTYPE html>
2
+ <html>
3
+ <head>
4
+ <meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
5
+ <link href="<%= url 'reset.css' %>" media="screen" rel="stylesheet" type="text/css">
6
+ <link href="<%= url 'style.css' %>" media="screen" rel="stylesheet" type="text/css">
7
+
8
+ <title>Split</title>
9
+
10
+ </head>
11
+ <body>
12
+ <div class="header"></div>
13
+
14
+ <div id="main">
15
+ <%= yield %>
16
+ </div>
17
+
18
+ <div id="footer">
19
+ <p>Powered by <a href="http://github.com/andrew/split">Split</a> v<%=Split::VERSION %></p>
20
+ </div>
21
+ </body>
22
+ </html>
@@ -0,0 +1,59 @@
1
+ module Split
2
+ class Experiment
3
+ attr_accessor :name
4
+ attr_accessor :alternatives
5
+ attr_accessor :winner
6
+
7
+ def initialize(name, *alternatives)
8
+ @name = name.to_s
9
+ @alternatives = alternatives
10
+ end
11
+
12
+ def winner
13
+ if w = Split.redis.hget(:experiment_winner, name)
14
+ return Split::Alternative.find(w, name)
15
+ else
16
+ nil
17
+ end
18
+ end
19
+
20
+ def winner=(winner_name)
21
+ Split.redis.hset(:experiment_winner, name, winner_name.to_s)
22
+ end
23
+
24
+ def alternatives
25
+ @alternatives.map {|a| Split::Alternative.find_or_create(a, name)}
26
+ end
27
+
28
+ def next_alternative
29
+ winner || alternatives.sort_by{|a| a.participant_count + rand}.first
30
+ end
31
+
32
+ def save
33
+ Split.redis.sadd(:experiments, name)
34
+ @alternatives.each {|a| Split.redis.sadd(name, a) }
35
+ end
36
+
37
+ def self.all
38
+ Array(Split.redis.smembers(:experiments)).map {|e| find(e)}
39
+ end
40
+
41
+ def self.find(name)
42
+ if Split.redis.exists(name)
43
+ self.new(name, *Split.redis.smembers(name))
44
+ else
45
+ raise 'Experiment not found'
46
+ end
47
+ end
48
+
49
+ def self.find_or_create(name, *alternatives)
50
+ if Split.redis.exists(name)
51
+ return self.new(name, *Split.redis.smembers(name))
52
+ else
53
+ experiment = self.new(name, *alternatives)
54
+ experiment.save
55
+ return experiment
56
+ end
57
+ end
58
+ end
59
+ end
@@ -0,0 +1,28 @@
1
+ module Split
2
+ module Helper
3
+ def ab_test(experiment_name, *alternatives)
4
+ experiment = Split::Experiment.find_or_create(experiment_name, *alternatives)
5
+ return experiment.winner.name if experiment.winner
6
+
7
+ if ab_user[experiment_name]
8
+ return ab_user[experiment_name]
9
+ else
10
+ alternative = experiment.next_alternative
11
+ alternative.increment_participation
12
+ ab_user[experiment_name] = alternative.name
13
+ return alternative.name
14
+ end
15
+ end
16
+
17
+ def finished(experiment_name)
18
+ alternative_name = ab_user[experiment_name]
19
+ alternative = Split::Alternative.find(alternative_name, experiment_name)
20
+ alternative.increment_completion
21
+ session[:split].delete(experiment_name)
22
+ end
23
+
24
+ def ab_user
25
+ session[:split] ||= {}
26
+ end
27
+ end
28
+ end
@@ -0,0 +1,3 @@
1
+ module Split
2
+ VERSION = "0.1.0"
3
+ end
@@ -0,0 +1,67 @@
1
+ require 'spec_helper'
2
+ require 'split/experiment'
3
+
4
+ describe Split::Experiment do
5
+ before(:each) { Split.redis.flushall }
6
+
7
+ it "should have a name" do
8
+ experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
9
+ experiment.name.should eql('basket_text')
10
+ end
11
+
12
+ it "should have alternatives" do
13
+ experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
14
+ experiment.alternatives.length.should be 2
15
+ end
16
+
17
+ it "should save to redis" do
18
+ experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
19
+ experiment.save
20
+ Split.redis.exists('basket_text').should be true
21
+ end
22
+
23
+ it "should return an existing experiment" do
24
+ experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
25
+ experiment.save
26
+ Split::Experiment.find('basket_text').name.should eql('basket_text')
27
+ end
28
+
29
+ describe 'winner' do
30
+ it "should have no winner initially" do
31
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
32
+ experiment.winner.should be_nil
33
+ end
34
+
35
+ it "should allow you to specify a winner" do
36
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
37
+ experiment.winner = 'red'
38
+
39
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
40
+ experiment.winner.name.should == 'red'
41
+ end
42
+ end
43
+
44
+ describe 'next_alternative' do
45
+ it "should return a random alternative from those with the least participants" do
46
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red', 'green')
47
+
48
+ Split::Alternative.find('blue', 'link_color').increment_participation
49
+ Split::Alternative.find('red', 'link_color').increment_participation
50
+
51
+ experiment.next_alternative.name.should == 'green'
52
+ end
53
+
54
+ it "should always return the winner if one exists" do
55
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red', 'green')
56
+ green = Split::Alternative.find('green', 'link_color')
57
+ experiment.winner = 'green'
58
+
59
+ experiment.next_alternative.name.should == 'green'
60
+ green.increment_participation
61
+
62
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red', 'green')
63
+ experiment.next_alternative.name.should == 'green'
64
+ end
65
+ end
66
+ end
67
+
@@ -0,0 +1,86 @@
1
+ require 'spec_helper'
2
+
3
+ describe Split::Helper do
4
+ include Split::Helper
5
+
6
+ before(:each) do
7
+ Split.redis.flushall
8
+ @session = {}
9
+ end
10
+
11
+ describe "ab_test" do
12
+ it "should assign a random alternative to a new user when there are an equal number of alternatives assigned" do
13
+ ab_test('link_color', 'blue', 'red')
14
+ ['red', 'blue'].should include(ab_user['link_color'])
15
+ end
16
+
17
+ it "should increment the participation counter after assignment to a new user" do
18
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
19
+
20
+ previous_red_count = Split::Alternative.find('red', 'link_color').participant_count
21
+ previous_blue_count = Split::Alternative.find('blue', 'link_color').participant_count
22
+
23
+ ab_test('link_color', 'blue', 'red')
24
+
25
+ new_red_count = Split::Alternative.find('red', 'link_color').participant_count
26
+ new_blue_count = Split::Alternative.find('blue', 'link_color').participant_count
27
+
28
+ (new_red_count + new_blue_count).should eql(previous_red_count + previous_blue_count + 1)
29
+ end
30
+
31
+ it "should return the given alternative for an existing user" do
32
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
33
+ alternative = ab_test('link_color', 'blue', 'red')
34
+ repeat_alternative = ab_test('link_color', 'blue', 'red')
35
+ alternative.should eql repeat_alternative
36
+ end
37
+
38
+ it 'should always return the winner if one is present' do
39
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
40
+ experiment.winner = "orange"
41
+
42
+ ab_test('link_color', 'blue', 'red').should == 'orange'
43
+ end
44
+ end
45
+
46
+ describe 'finished' do
47
+ it 'should increment the counter for the completed alternative' do
48
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
49
+ alternative_name = ab_test('link_color', 'blue', 'red')
50
+
51
+ previous_completion_count = Split::Alternative.find(alternative_name, 'link_color').completed_count
52
+
53
+ finished('link_color')
54
+
55
+ new_completion_count = Split::Alternative.find(alternative_name, 'link_color').completed_count
56
+
57
+ new_completion_count.should eql(previous_completion_count + 1)
58
+ end
59
+
60
+ it "should clear out the user's participation from their session" do
61
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
62
+ alternative_name = ab_test('link_color', 'blue', 'red')
63
+
64
+ previous_completion_count = Split::Alternative.find(alternative_name, 'link_color').completed_count
65
+
66
+ session[:split].should == {"link_color" => alternative_name}
67
+ finished('link_color')
68
+ session[:split].should == {}
69
+ end
70
+ end
71
+
72
+ describe 'conversions' do
73
+ it 'should return a conversion rate for an alternative' do
74
+ experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
75
+ alternative_name = ab_test('link_color', 'blue', 'red')
76
+
77
+ previous_convertion_rate = Split::Alternative.find(alternative_name, 'link_color').conversion_rate
78
+ previous_convertion_rate.should eql(0.0)
79
+
80
+ finished('link_color')
81
+
82
+ new_convertion_rate = Split::Alternative.find(alternative_name, 'link_color').conversion_rate
83
+ new_convertion_rate.should eql(1.0)
84
+ end
85
+ end
86
+ end
@@ -0,0 +1,7 @@
1
+ require 'rubygems'
2
+ require 'bundler/setup'
3
+ require 'split'
4
+
5
+ def session
6
+ @session ||= {}
7
+ end
@@ -0,0 +1,27 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "split/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "split"
7
+ s.version = Split::VERSION
8
+ s.platform = Gem::Platform::RUBY
9
+ s.authors = ["Andrew Nesbitt"]
10
+ s.email = ["andrewnez@gmail.com"]
11
+ s.homepage = ""
12
+ s.summary = %q{Rack based split testing framework}
13
+
14
+ s.rubyforge_project = "split"
15
+
16
+ s.files = `git ls-files`.split("\n")
17
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
18
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
19
+ s.require_paths = ["lib"]
20
+
21
+ s.add_dependency(%q<redis>, ["~> 2.1"])
22
+ s.add_dependency(%q<redis-namespace>, ["~> 0.10.0"])
23
+ s.add_dependency(%q<sinatra>, ["~> 1.2.6"])
24
+
25
+ # Development Dependencies
26
+ s.add_development_dependency(%q<rspec>, ["~> 2.6"])
27
+ end
metadata ADDED
@@ -0,0 +1,141 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: split
3
+ version: !ruby/object:Gem::Version
4
+ prerelease: false
5
+ segments:
6
+ - 0
7
+ - 1
8
+ - 0
9
+ version: 0.1.0
10
+ platform: ruby
11
+ authors:
12
+ - Andrew Nesbitt
13
+ autorequire:
14
+ bindir: bin
15
+ cert_chain: []
16
+
17
+ date: 2011-05-17 00:00:00 -04:00
18
+ default_executable:
19
+ dependencies:
20
+ - !ruby/object:Gem::Dependency
21
+ name: redis
22
+ prerelease: false
23
+ requirement: &id001 !ruby/object:Gem::Requirement
24
+ none: false
25
+ requirements:
26
+ - - ~>
27
+ - !ruby/object:Gem::Version
28
+ segments:
29
+ - 2
30
+ - 1
31
+ version: "2.1"
32
+ type: :runtime
33
+ version_requirements: *id001
34
+ - !ruby/object:Gem::Dependency
35
+ name: redis-namespace
36
+ prerelease: false
37
+ requirement: &id002 !ruby/object:Gem::Requirement
38
+ none: false
39
+ requirements:
40
+ - - ~>
41
+ - !ruby/object:Gem::Version
42
+ segments:
43
+ - 0
44
+ - 10
45
+ - 0
46
+ version: 0.10.0
47
+ type: :runtime
48
+ version_requirements: *id002
49
+ - !ruby/object:Gem::Dependency
50
+ name: sinatra
51
+ prerelease: false
52
+ requirement: &id003 !ruby/object:Gem::Requirement
53
+ none: false
54
+ requirements:
55
+ - - ~>
56
+ - !ruby/object:Gem::Version
57
+ segments:
58
+ - 1
59
+ - 2
60
+ - 6
61
+ version: 1.2.6
62
+ type: :runtime
63
+ version_requirements: *id003
64
+ - !ruby/object:Gem::Dependency
65
+ name: rspec
66
+ prerelease: false
67
+ requirement: &id004 !ruby/object:Gem::Requirement
68
+ none: false
69
+ requirements:
70
+ - - ~>
71
+ - !ruby/object:Gem::Version
72
+ segments:
73
+ - 2
74
+ - 6
75
+ version: "2.6"
76
+ type: :development
77
+ version_requirements: *id004
78
+ description:
79
+ email:
80
+ - andrewnez@gmail.com
81
+ executables: []
82
+
83
+ extensions: []
84
+
85
+ extra_rdoc_files: []
86
+
87
+ files:
88
+ - .gitignore
89
+ - Gemfile
90
+ - README.mdown
91
+ - Rakefile
92
+ - lib/split.rb
93
+ - lib/split/alternative.rb
94
+ - lib/split/dashboard.rb
95
+ - lib/split/dashboard/public/reset.css
96
+ - lib/split/dashboard/public/style.css
97
+ - lib/split/dashboard/views/index.erb
98
+ - lib/split/dashboard/views/layout.erb
99
+ - lib/split/experiment.rb
100
+ - lib/split/helper.rb
101
+ - lib/split/version.rb
102
+ - spec/experiment_spec.rb
103
+ - spec/helper_spec.rb
104
+ - spec/spec_helper.rb
105
+ - split.gemspec
106
+ has_rdoc: true
107
+ homepage: ""
108
+ licenses: []
109
+
110
+ post_install_message:
111
+ rdoc_options: []
112
+
113
+ require_paths:
114
+ - lib
115
+ required_ruby_version: !ruby/object:Gem::Requirement
116
+ none: false
117
+ requirements:
118
+ - - ">="
119
+ - !ruby/object:Gem::Version
120
+ segments:
121
+ - 0
122
+ version: "0"
123
+ required_rubygems_version: !ruby/object:Gem::Requirement
124
+ none: false
125
+ requirements:
126
+ - - ">="
127
+ - !ruby/object:Gem::Version
128
+ segments:
129
+ - 0
130
+ version: "0"
131
+ requirements: []
132
+
133
+ rubyforge_project: split
134
+ rubygems_version: 1.3.7
135
+ signing_key:
136
+ specification_version: 3
137
+ summary: Rack based split testing framework
138
+ test_files:
139
+ - spec/experiment_spec.rb
140
+ - spec/helper_spec.rb
141
+ - spec/spec_helper.rb