split 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +4 -0
- data/Gemfile +3 -0
- data/README.mdown +177 -0
- data/Rakefile +8 -0
- data/lib/split.rb +50 -0
- data/lib/split/alternative.rb +77 -0
- data/lib/split/dashboard.rb +35 -0
- data/lib/split/dashboard/public/reset.css +48 -0
- data/lib/split/dashboard/public/style.css +34 -0
- data/lib/split/dashboard/views/index.erb +56 -0
- data/lib/split/dashboard/views/layout.erb +22 -0
- data/lib/split/experiment.rb +59 -0
- data/lib/split/helper.rb +28 -0
- data/lib/split/version.rb +3 -0
- data/spec/experiment_spec.rb +67 -0
- data/spec/helper_spec.rb +86 -0
- data/spec/spec_helper.rb +7 -0
- data/split.gemspec +27 -0
- metadata +141 -0
data/.gitignore
ADDED
data/Gemfile
ADDED
data/README.mdown
ADDED
@@ -0,0 +1,177 @@
|
|
1
|
+
# Split
|
2
|
+
## Rack based split testing framework
|
3
|
+
|
4
|
+
Split is a rack based ab testing framework designed to work with Rails, Sinatra or any other rack based app.
|
5
|
+
|
6
|
+
Split is heavily inspired by the Abingo and Vanity rails ab testing plugins and Resque in its use of Redis.
|
7
|
+
|
8
|
+
## Requirements
|
9
|
+
|
10
|
+
Split uses redis as a datastore.
|
11
|
+
|
12
|
+
If you're on OS X, Homebrew is the simplest way to install Redis:
|
13
|
+
|
14
|
+
$ brew install redis
|
15
|
+
$ redis-server /usr/local/etc/redis.conf
|
16
|
+
|
17
|
+
You now have a Redis daemon running on 6379.
|
18
|
+
|
19
|
+
## Setup
|
20
|
+
|
21
|
+
If you are using bundler add split to your Gemfile:
|
22
|
+
|
23
|
+
gem 'split'
|
24
|
+
|
25
|
+
Then run:
|
26
|
+
|
27
|
+
bundle install
|
28
|
+
|
29
|
+
Otherwise install the gem:
|
30
|
+
|
31
|
+
gem install split
|
32
|
+
|
33
|
+
and require it in your project:
|
34
|
+
|
35
|
+
require 'split'
|
36
|
+
|
37
|
+
### Rails
|
38
|
+
|
39
|
+
Split is autoloaded when rails starts up, as long as you've configured redis it will 'just work'.
|
40
|
+
|
41
|
+
### Sinatra
|
42
|
+
|
43
|
+
To configure sinatra with Split you need to enable sessions and mix in the helper methods. Add the following lines at the top of your sinatra app:
|
44
|
+
|
45
|
+
class MySinatraApp < Sinatra::Base
|
46
|
+
enable :sessions
|
47
|
+
helpers Split::Helper
|
48
|
+
|
49
|
+
get '/' do
|
50
|
+
...
|
51
|
+
end
|
52
|
+
|
53
|
+
## Usage
|
54
|
+
|
55
|
+
To begin your ab test use the `ab_test` method, naming your experiment with the first argument and then the different variants which you wish to test on as the other arguments.
|
56
|
+
|
57
|
+
`ab_test` returns one of the alternatives, if a user has already seen that test they will get the same alternative as before, which you can use to split your code on.
|
58
|
+
|
59
|
+
It can be used to render different templates, show different text or any other case based logic.
|
60
|
+
|
61
|
+
`finished` is used to make a completion of an experiment, or conversion.
|
62
|
+
|
63
|
+
Example: View
|
64
|
+
|
65
|
+
<% ab_test("login_button", "/images/button1.jpg", "/images/button2.jpg") do |button_file| %>
|
66
|
+
<%= img_tag(button_file, :alt => "Login!") %>
|
67
|
+
<% end %>
|
68
|
+
|
69
|
+
Example: Controller
|
70
|
+
|
71
|
+
def register_new_user
|
72
|
+
#See what level of free points maximizes users' decision to buy replacement points.
|
73
|
+
@starter_points = ab_test("new_user_free_points", 100, 200, 300)
|
74
|
+
end
|
75
|
+
|
76
|
+
Example: Conversion tracking (in a controller!)
|
77
|
+
|
78
|
+
def buy_new_points
|
79
|
+
#some business logic
|
80
|
+
finished("buy_new_points") #Either a conversion named with :conversion or a test name.
|
81
|
+
end
|
82
|
+
|
83
|
+
Example: Conversion tracking (in a view)
|
84
|
+
|
85
|
+
Thanks for signing up, dude! <% finished("signup_page_redesign") >
|
86
|
+
|
87
|
+
## Web Interface
|
88
|
+
|
89
|
+
Split comes with a Sinatra-based front end to get an overview of how your experiments are doing.
|
90
|
+
|
91
|
+
You can mount this inside your app using Rack::URLMap in your `config.ru`
|
92
|
+
|
93
|
+
require 'split/dashboard'
|
94
|
+
|
95
|
+
run Rack::URLMap.new \
|
96
|
+
"/" => Your::App.new,
|
97
|
+
"/split" => Split::Dashboard.new
|
98
|
+
|
99
|
+
You may want to password protect that page, you can do so with `Rack::Auth::Basic`
|
100
|
+
|
101
|
+
Split::Dashboard.use Rack::Auth::Basic do |username, password|
|
102
|
+
username == 'admin' && password == 'p4s5w0rd'
|
103
|
+
end
|
104
|
+
|
105
|
+
## Configuration
|
106
|
+
|
107
|
+
You may want to change the Redis host and port Split connects to, or
|
108
|
+
set various other options at startup.
|
109
|
+
|
110
|
+
Split has a `redis` setter which can be given a string or a Redis
|
111
|
+
object. This means if you're already using Redis in your app, Split
|
112
|
+
can re-use the existing connection.
|
113
|
+
|
114
|
+
String: `Split.redis = 'localhost:6379'`
|
115
|
+
|
116
|
+
Redis: `Split.redis = $redis`
|
117
|
+
|
118
|
+
For our rails app we have a `config/initializers/split.rb` file where
|
119
|
+
we load `config/split.yml` by hand and set the Redis information
|
120
|
+
appropriately.
|
121
|
+
|
122
|
+
Here's our `config/split.yml`:
|
123
|
+
|
124
|
+
development: localhost:6379
|
125
|
+
test: localhost:6379
|
126
|
+
staging: redis1.example.com:6379
|
127
|
+
fi: localhost:6379
|
128
|
+
production: redis1.example.com:6379
|
129
|
+
|
130
|
+
And our initializer:
|
131
|
+
|
132
|
+
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
|
133
|
+
rails_env = ENV['RAILS_ENV'] || 'development'
|
134
|
+
|
135
|
+
split_config = YAML.load_file(rails_root + '/config/split.yml')
|
136
|
+
Split.redis = split_config[rails_env]
|
137
|
+
|
138
|
+
## Namespaces
|
139
|
+
|
140
|
+
If you're running multiple, separate instances of Split you may want
|
141
|
+
to namespace the keyspaces so they do not overlap. This is not unlike
|
142
|
+
the approach taken by many memcached clients.
|
143
|
+
|
144
|
+
This feature is provided by the [redis-namespace][rs] library, which
|
145
|
+
Split uses by default to separate the keys it manages from other keys
|
146
|
+
in your Redis server.
|
147
|
+
|
148
|
+
Simply use the `Split.redis.namespace` accessor:
|
149
|
+
|
150
|
+
Split.redis.namespace = "resque:GitHub"
|
151
|
+
|
152
|
+
We recommend sticking this in your initializer somewhere after Redis
|
153
|
+
is configured.
|
154
|
+
|
155
|
+
## Contributors
|
156
|
+
|
157
|
+
Special thanks to the following people for submitting patches:
|
158
|
+
|
159
|
+
* Lloyd Pick
|
160
|
+
* Jeffery Chupp
|
161
|
+
|
162
|
+
## Note on Patches/Pull Requests
|
163
|
+
|
164
|
+
* Fork the project.
|
165
|
+
* Make your feature addition or bug fix.
|
166
|
+
* Add tests for it. This is important so I don't break it in a
|
167
|
+
future version unintentionally.
|
168
|
+
* Commit, do not mess with rakefile, version, or history.
|
169
|
+
(if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
|
170
|
+
* Send me a pull request. Bonus points for topic branches.
|
171
|
+
|
172
|
+
## Copyright
|
173
|
+
|
174
|
+
Copyright (c) 2011 Andrew Nesbitt. See LICENSE for details.
|
175
|
+
|
176
|
+
|
177
|
+
n.b don't pass the same alternative twice!
|
data/Rakefile
ADDED
data/lib/split.rb
ADDED
@@ -0,0 +1,50 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require 'split/experiment'
|
3
|
+
require 'split/alternative'
|
4
|
+
require 'split/helper'
|
5
|
+
require 'redis/namespace'
|
6
|
+
|
7
|
+
module Split
|
8
|
+
extend self
|
9
|
+
# Accepts:
|
10
|
+
# 1. A 'hostname:port' string
|
11
|
+
# 2. A 'hostname:port:db' string (to select the Redis db)
|
12
|
+
# 3. A 'hostname:port/namespace' string (to set the Redis namespace)
|
13
|
+
# 4. A redis URL string 'redis://host:port'
|
14
|
+
# 5. An instance of `Redis`, `Redis::Client`, `Redis::DistRedis`,
|
15
|
+
# or `Redis::Namespace`.
|
16
|
+
def redis=(server)
|
17
|
+
if server.respond_to? :split
|
18
|
+
if server =~ /redis\:\/\//
|
19
|
+
redis = Redis.connect(:url => server, :thread_safe => true)
|
20
|
+
else
|
21
|
+
server, namespace = server.split('/', 2)
|
22
|
+
host, port, db = server.split(':')
|
23
|
+
redis = Redis.new(:host => host, :port => port,
|
24
|
+
:thread_safe => true, :db => db)
|
25
|
+
end
|
26
|
+
namespace ||= :split
|
27
|
+
|
28
|
+
@redis = Redis::Namespace.new(namespace, :redis => redis)
|
29
|
+
elsif server.respond_to? :namespace=
|
30
|
+
@redis = server
|
31
|
+
else
|
32
|
+
@redis = Redis::Namespace.new(:split, :redis => server)
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
# Returns the current Redis connection. If none has been created, will
|
37
|
+
# create a new one.
|
38
|
+
def redis
|
39
|
+
return @redis if @redis
|
40
|
+
self.redis = 'localhost:6379'
|
41
|
+
self.redis
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
if defined?(Rails)
|
46
|
+
class ActionController::Base
|
47
|
+
ActionController::Base.send :include, Split::Helper
|
48
|
+
ActionController::Base.helper Split::Helper
|
49
|
+
end
|
50
|
+
end
|
@@ -0,0 +1,77 @@
|
|
1
|
+
module Split
|
2
|
+
class Alternative
|
3
|
+
attr_accessor :name
|
4
|
+
attr_accessor :participant_count
|
5
|
+
attr_accessor :completed_count
|
6
|
+
attr_accessor :experiment_name
|
7
|
+
|
8
|
+
def initialize(name, experiment_name, counters = {})
|
9
|
+
@experiment_name = experiment_name
|
10
|
+
@name = name
|
11
|
+
@participant_count = counters['participant_count'].to_i
|
12
|
+
@completed_count = counters['completed_count'].to_i
|
13
|
+
end
|
14
|
+
|
15
|
+
def increment_participation
|
16
|
+
@participant_count +=1
|
17
|
+
self.save
|
18
|
+
end
|
19
|
+
|
20
|
+
def increment_completion
|
21
|
+
@completed_count +=1
|
22
|
+
self.save
|
23
|
+
end
|
24
|
+
|
25
|
+
def conversion_rate
|
26
|
+
return 0 if participant_count.zero?
|
27
|
+
(completed_count.to_f/participant_count.to_f)
|
28
|
+
end
|
29
|
+
|
30
|
+
def z_score
|
31
|
+
# CTR_E = the CTR within the experiment split
|
32
|
+
# CTR_C = the CTR within the control split
|
33
|
+
# E = the number of impressions within the experiment split
|
34
|
+
# C = the number of impressions within the control split
|
35
|
+
|
36
|
+
experiment = Split::Experiment.find(@experiment_name)
|
37
|
+
control = experiment.alternatives[0]
|
38
|
+
alternative = self
|
39
|
+
|
40
|
+
return 'N/A' if control.name == alternative.name
|
41
|
+
|
42
|
+
ctr_e = alternative.conversion_rate
|
43
|
+
ctr_c = control.conversion_rate
|
44
|
+
|
45
|
+
e = alternative.participant_count
|
46
|
+
c = control.participant_count
|
47
|
+
|
48
|
+
standard_deviation = ((ctr_e / ctr_c**3) * ((e*ctr_e)+(c*ctr_c)-(ctr_c*ctr_e)*(c+e))/(c*e)) ** 0.5
|
49
|
+
|
50
|
+
z_score = ((ctr_e / ctr_c) - 1) / standard_deviation
|
51
|
+
end
|
52
|
+
|
53
|
+
def save
|
54
|
+
if Split.redis.hgetall("#{experiment_name}:#{name}")
|
55
|
+
Split.redis.hset "#{experiment_name}:#{name}", 'participant_count', @participant_count
|
56
|
+
Split.redis.hset "#{experiment_name}:#{name}", 'completed_count', @completed_count
|
57
|
+
else
|
58
|
+
Split.redis.hmset "#{experiment_name}:#{name}", 'participant_count', 'completed_count', @participant_count, @completed_count
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
def self.find(name, experiment_name)
|
63
|
+
counters = Split.redis.hgetall "#{experiment_name}:#{name}"
|
64
|
+
self.new(name, experiment_name, counters)
|
65
|
+
end
|
66
|
+
|
67
|
+
def self.find_or_create(name, experiment_name)
|
68
|
+
self.find(name, experiment_name) || self.create(name, experiment_name)
|
69
|
+
end
|
70
|
+
|
71
|
+
def self.create(name, experiment_name)
|
72
|
+
alt = self.new(name, experiment_name)
|
73
|
+
alt.save
|
74
|
+
alt
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
@@ -0,0 +1,35 @@
|
|
1
|
+
require 'sinatra/base'
|
2
|
+
require 'split'
|
3
|
+
|
4
|
+
module Split
|
5
|
+
class Dashboard < Sinatra::Base
|
6
|
+
dir = File.dirname(File.expand_path(__FILE__))
|
7
|
+
|
8
|
+
set :views, "#{dir}/dashboard/views"
|
9
|
+
set :public, "#{dir}/dashboard/public"
|
10
|
+
set :static, true
|
11
|
+
|
12
|
+
helpers do
|
13
|
+
def url(*path_parts)
|
14
|
+
[ path_prefix, path_parts ].join("/").squeeze('/')
|
15
|
+
end
|
16
|
+
|
17
|
+
def path_prefix
|
18
|
+
request.env['SCRIPT_NAME']
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
get '/' do
|
23
|
+
@experiments = Split::Experiment.all
|
24
|
+
erb :index
|
25
|
+
end
|
26
|
+
|
27
|
+
post '/:experiment' do
|
28
|
+
@experiment = Split::Experiment.find(params[:experiment])
|
29
|
+
@alternative = Split::Alternative.find(params[:alternative], params[:experiment])
|
30
|
+
@experiment.winner = @alternative.name
|
31
|
+
@experiment.save
|
32
|
+
redirect url('/')
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
@@ -0,0 +1,48 @@
|
|
1
|
+
html, body, div, span, applet, object, iframe,
|
2
|
+
h1, h2, h3, h4, h5, h6, p, blockquote, pre,
|
3
|
+
a, abbr, acronym, address, big, cite, code,
|
4
|
+
del, dfn, em, font, img, ins, kbd, q, s, samp,
|
5
|
+
small, strike, strong, sub, sup, tt, var,
|
6
|
+
dl, dt, dd, ul, li,
|
7
|
+
form, label, legend,
|
8
|
+
table, caption, tbody, tfoot, thead, tr, th, td {
|
9
|
+
margin: 0;
|
10
|
+
padding: 0;
|
11
|
+
border: 0;
|
12
|
+
outline: 0;
|
13
|
+
font-weight: inherit;
|
14
|
+
font-style: normal;
|
15
|
+
font-size: 100%;
|
16
|
+
font-family: inherit;
|
17
|
+
}
|
18
|
+
|
19
|
+
:focus {
|
20
|
+
outline: 0;
|
21
|
+
}
|
22
|
+
|
23
|
+
body {
|
24
|
+
line-height: 1;
|
25
|
+
}
|
26
|
+
|
27
|
+
ul {
|
28
|
+
list-style: none;
|
29
|
+
}
|
30
|
+
|
31
|
+
table {
|
32
|
+
border-collapse: collapse;
|
33
|
+
border-spacing: 0;
|
34
|
+
}
|
35
|
+
|
36
|
+
caption, th, td {
|
37
|
+
text-align: left;
|
38
|
+
font-weight: normal;
|
39
|
+
}
|
40
|
+
|
41
|
+
blockquote:before, blockquote:after,
|
42
|
+
q:before, q:after {
|
43
|
+
content: "";
|
44
|
+
}
|
45
|
+
|
46
|
+
blockquote, q {
|
47
|
+
quotes: "" "";
|
48
|
+
}
|
@@ -0,0 +1,34 @@
|
|
1
|
+
html { background:#efefef; font-family:Arial, Verdana, sans-serif; font-size:13px; }
|
2
|
+
body { padding:0; margin:0; }
|
3
|
+
|
4
|
+
.header { background:#000; padding:8px 5% 0 5%; border-bottom:1px solid #444;border-bottom:5px solid #0080FF;}
|
5
|
+
.header h1 { color:#333; font-size:90%; font-weight:bold; margin-bottom:6px;}
|
6
|
+
.header ul li { display:inline;}
|
7
|
+
.header ul li a { color:#fff; text-decoration:none; margin-right:10px; display:inline-block; padding:8px; -webkit-border-top-right-radius:6px; -webkit-border-top-left-radius:6px; -moz-border-radius-topleft:6px; -moz-border-radius-topright:6px; }
|
8
|
+
.header ul li a:hover { background:#333;}
|
9
|
+
.header ul li.current a { background:#0080FF; font-weight:bold; color:#fff;}
|
10
|
+
|
11
|
+
#main { padding:10px 5%; background:#fff; overflow:hidden; }
|
12
|
+
#main .logo { float:right; margin:10px;}
|
13
|
+
#main span.hl { background:#efefef; padding:2px;}
|
14
|
+
#main h1 { margin:10px 0; font-size:190%; font-weight:bold; color:#0080FF;}
|
15
|
+
#main h2 { margin:10px 0; font-size:130%;}
|
16
|
+
#main table { width:100%; margin:10px 0;}
|
17
|
+
#main table tr td, #main table tr th { border:1px solid #ccc; padding:6px;}
|
18
|
+
#main table tr th { background:#efefef; color:#888; font-size:80%; font-weight:bold;}
|
19
|
+
#main table tr td.no-data { text-align:center; padding:40px 0; color:#999; font-style:italic; font-size:130%;}
|
20
|
+
#main a { color:#111;}
|
21
|
+
#main p { margin:5px 0;}
|
22
|
+
#main p.intro { margin-bottom:15px; font-size:85%; color:#999; margin-top:0; line-height:1.3;}
|
23
|
+
#main h1.wi { margin-bottom:5px;}
|
24
|
+
#main p.sub { font-size:95%; color:#999;}
|
25
|
+
|
26
|
+
#main table.queues { width:40%;}
|
27
|
+
|
28
|
+
#main table .totals td{ background:#eee; font-weight:bold; }
|
29
|
+
|
30
|
+
#footer { padding:10px 5%; background:#efefef; color:#999; font-size:85%; line-height:1.5; border-top:5px solid #ccc; padding-top:10px;}
|
31
|
+
#footer p a { color:#999;}
|
32
|
+
|
33
|
+
|
34
|
+
|
@@ -0,0 +1,56 @@
|
|
1
|
+
<h1>Split Dashboard</h1>
|
2
|
+
<p class="intro">The list below contains all the registered experiments along with the number of test participants, completed and conversion rate currently in the system.</p>
|
3
|
+
|
4
|
+
<% @experiments.each do |experiment| %>
|
5
|
+
<h2><%= experiment.name %></h2>
|
6
|
+
<table class="queues">
|
7
|
+
<tr>
|
8
|
+
<th>Alternative Name</th>
|
9
|
+
<th>Participants</th>
|
10
|
+
<th>Non-finished</th>
|
11
|
+
<th>Completed</th>
|
12
|
+
<th>Conversion Rate</th>
|
13
|
+
<th>Z-Score</th>
|
14
|
+
<th>Winner</th>
|
15
|
+
</tr>
|
16
|
+
|
17
|
+
<% total_participants = total_completed = 0 %>
|
18
|
+
<% experiment.alternatives.each do |alternative| %>
|
19
|
+
<tr>
|
20
|
+
<td><%= alternative.name %></td>
|
21
|
+
<td><%= alternative.participant_count %></td>
|
22
|
+
<td><%= alternative.participant_count - alternative.completed_count %></td>
|
23
|
+
<td><%= alternative.completed_count %></td>
|
24
|
+
<td><%= (alternative.conversion_rate * 100).round(2) %>%</td>
|
25
|
+
<td><%= alternative.z_score %></td>
|
26
|
+
<td>
|
27
|
+
<% if experiment.winner %>
|
28
|
+
<% if experiment.winner.name == alternative.name %>
|
29
|
+
Winner
|
30
|
+
<% else %>
|
31
|
+
Loser
|
32
|
+
<% end %>
|
33
|
+
<% else %>
|
34
|
+
<form action="<%= url experiment.name %>" method='post'>
|
35
|
+
<input type='hidden' name='alternative' value='<%= alternative.name %>'>
|
36
|
+
<input type="submit" value="Use this">
|
37
|
+
</form>
|
38
|
+
<% end %>
|
39
|
+
</td>
|
40
|
+
</tr>
|
41
|
+
|
42
|
+
<% total_participants += alternative.participant_count %>
|
43
|
+
<% total_completed += alternative.completed_count %>
|
44
|
+
<% end %>
|
45
|
+
|
46
|
+
<tr class="totals">
|
47
|
+
<td>Totals</td>
|
48
|
+
<td><%= total_participants %></td>
|
49
|
+
<td><%= total_participants - total_completed %></td>
|
50
|
+
<td><%= total_completed %></td>
|
51
|
+
<td>N/A</td>
|
52
|
+
<td>N/A</td>
|
53
|
+
<td>N/A</td>
|
54
|
+
</tr>
|
55
|
+
</table>
|
56
|
+
<% end %>
|
@@ -0,0 +1,22 @@
|
|
1
|
+
<!DOCTYPE html>
|
2
|
+
<html>
|
3
|
+
<head>
|
4
|
+
<meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
|
5
|
+
<link href="<%= url 'reset.css' %>" media="screen" rel="stylesheet" type="text/css">
|
6
|
+
<link href="<%= url 'style.css' %>" media="screen" rel="stylesheet" type="text/css">
|
7
|
+
|
8
|
+
<title>Split</title>
|
9
|
+
|
10
|
+
</head>
|
11
|
+
<body>
|
12
|
+
<div class="header"></div>
|
13
|
+
|
14
|
+
<div id="main">
|
15
|
+
<%= yield %>
|
16
|
+
</div>
|
17
|
+
|
18
|
+
<div id="footer">
|
19
|
+
<p>Powered by <a href="http://github.com/andrew/split">Split</a> v<%=Split::VERSION %></p>
|
20
|
+
</div>
|
21
|
+
</body>
|
22
|
+
</html>
|
@@ -0,0 +1,59 @@
|
|
1
|
+
module Split
|
2
|
+
class Experiment
|
3
|
+
attr_accessor :name
|
4
|
+
attr_accessor :alternatives
|
5
|
+
attr_accessor :winner
|
6
|
+
|
7
|
+
def initialize(name, *alternatives)
|
8
|
+
@name = name.to_s
|
9
|
+
@alternatives = alternatives
|
10
|
+
end
|
11
|
+
|
12
|
+
def winner
|
13
|
+
if w = Split.redis.hget(:experiment_winner, name)
|
14
|
+
return Split::Alternative.find(w, name)
|
15
|
+
else
|
16
|
+
nil
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
def winner=(winner_name)
|
21
|
+
Split.redis.hset(:experiment_winner, name, winner_name.to_s)
|
22
|
+
end
|
23
|
+
|
24
|
+
def alternatives
|
25
|
+
@alternatives.map {|a| Split::Alternative.find_or_create(a, name)}
|
26
|
+
end
|
27
|
+
|
28
|
+
def next_alternative
|
29
|
+
winner || alternatives.sort_by{|a| a.participant_count + rand}.first
|
30
|
+
end
|
31
|
+
|
32
|
+
def save
|
33
|
+
Split.redis.sadd(:experiments, name)
|
34
|
+
@alternatives.each {|a| Split.redis.sadd(name, a) }
|
35
|
+
end
|
36
|
+
|
37
|
+
def self.all
|
38
|
+
Array(Split.redis.smembers(:experiments)).map {|e| find(e)}
|
39
|
+
end
|
40
|
+
|
41
|
+
def self.find(name)
|
42
|
+
if Split.redis.exists(name)
|
43
|
+
self.new(name, *Split.redis.smembers(name))
|
44
|
+
else
|
45
|
+
raise 'Experiment not found'
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
def self.find_or_create(name, *alternatives)
|
50
|
+
if Split.redis.exists(name)
|
51
|
+
return self.new(name, *Split.redis.smembers(name))
|
52
|
+
else
|
53
|
+
experiment = self.new(name, *alternatives)
|
54
|
+
experiment.save
|
55
|
+
return experiment
|
56
|
+
end
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
data/lib/split/helper.rb
ADDED
@@ -0,0 +1,28 @@
|
|
1
|
+
module Split
|
2
|
+
module Helper
|
3
|
+
def ab_test(experiment_name, *alternatives)
|
4
|
+
experiment = Split::Experiment.find_or_create(experiment_name, *alternatives)
|
5
|
+
return experiment.winner.name if experiment.winner
|
6
|
+
|
7
|
+
if ab_user[experiment_name]
|
8
|
+
return ab_user[experiment_name]
|
9
|
+
else
|
10
|
+
alternative = experiment.next_alternative
|
11
|
+
alternative.increment_participation
|
12
|
+
ab_user[experiment_name] = alternative.name
|
13
|
+
return alternative.name
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
def finished(experiment_name)
|
18
|
+
alternative_name = ab_user[experiment_name]
|
19
|
+
alternative = Split::Alternative.find(alternative_name, experiment_name)
|
20
|
+
alternative.increment_completion
|
21
|
+
session[:split].delete(experiment_name)
|
22
|
+
end
|
23
|
+
|
24
|
+
def ab_user
|
25
|
+
session[:split] ||= {}
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
@@ -0,0 +1,67 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'split/experiment'
|
3
|
+
|
4
|
+
describe Split::Experiment do
|
5
|
+
before(:each) { Split.redis.flushall }
|
6
|
+
|
7
|
+
it "should have a name" do
|
8
|
+
experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
|
9
|
+
experiment.name.should eql('basket_text')
|
10
|
+
end
|
11
|
+
|
12
|
+
it "should have alternatives" do
|
13
|
+
experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
|
14
|
+
experiment.alternatives.length.should be 2
|
15
|
+
end
|
16
|
+
|
17
|
+
it "should save to redis" do
|
18
|
+
experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
|
19
|
+
experiment.save
|
20
|
+
Split.redis.exists('basket_text').should be true
|
21
|
+
end
|
22
|
+
|
23
|
+
it "should return an existing experiment" do
|
24
|
+
experiment = Split::Experiment.new('basket_text', 'Basket', "Cart")
|
25
|
+
experiment.save
|
26
|
+
Split::Experiment.find('basket_text').name.should eql('basket_text')
|
27
|
+
end
|
28
|
+
|
29
|
+
describe 'winner' do
|
30
|
+
it "should have no winner initially" do
|
31
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
32
|
+
experiment.winner.should be_nil
|
33
|
+
end
|
34
|
+
|
35
|
+
it "should allow you to specify a winner" do
|
36
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
37
|
+
experiment.winner = 'red'
|
38
|
+
|
39
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
40
|
+
experiment.winner.name.should == 'red'
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
describe 'next_alternative' do
|
45
|
+
it "should return a random alternative from those with the least participants" do
|
46
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red', 'green')
|
47
|
+
|
48
|
+
Split::Alternative.find('blue', 'link_color').increment_participation
|
49
|
+
Split::Alternative.find('red', 'link_color').increment_participation
|
50
|
+
|
51
|
+
experiment.next_alternative.name.should == 'green'
|
52
|
+
end
|
53
|
+
|
54
|
+
it "should always return the winner if one exists" do
|
55
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red', 'green')
|
56
|
+
green = Split::Alternative.find('green', 'link_color')
|
57
|
+
experiment.winner = 'green'
|
58
|
+
|
59
|
+
experiment.next_alternative.name.should == 'green'
|
60
|
+
green.increment_participation
|
61
|
+
|
62
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red', 'green')
|
63
|
+
experiment.next_alternative.name.should == 'green'
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
67
|
+
|
data/spec/helper_spec.rb
ADDED
@@ -0,0 +1,86 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Split::Helper do
|
4
|
+
include Split::Helper
|
5
|
+
|
6
|
+
before(:each) do
|
7
|
+
Split.redis.flushall
|
8
|
+
@session = {}
|
9
|
+
end
|
10
|
+
|
11
|
+
describe "ab_test" do
|
12
|
+
it "should assign a random alternative to a new user when there are an equal number of alternatives assigned" do
|
13
|
+
ab_test('link_color', 'blue', 'red')
|
14
|
+
['red', 'blue'].should include(ab_user['link_color'])
|
15
|
+
end
|
16
|
+
|
17
|
+
it "should increment the participation counter after assignment to a new user" do
|
18
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
19
|
+
|
20
|
+
previous_red_count = Split::Alternative.find('red', 'link_color').participant_count
|
21
|
+
previous_blue_count = Split::Alternative.find('blue', 'link_color').participant_count
|
22
|
+
|
23
|
+
ab_test('link_color', 'blue', 'red')
|
24
|
+
|
25
|
+
new_red_count = Split::Alternative.find('red', 'link_color').participant_count
|
26
|
+
new_blue_count = Split::Alternative.find('blue', 'link_color').participant_count
|
27
|
+
|
28
|
+
(new_red_count + new_blue_count).should eql(previous_red_count + previous_blue_count + 1)
|
29
|
+
end
|
30
|
+
|
31
|
+
it "should return the given alternative for an existing user" do
|
32
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
33
|
+
alternative = ab_test('link_color', 'blue', 'red')
|
34
|
+
repeat_alternative = ab_test('link_color', 'blue', 'red')
|
35
|
+
alternative.should eql repeat_alternative
|
36
|
+
end
|
37
|
+
|
38
|
+
it 'should always return the winner if one is present' do
|
39
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
40
|
+
experiment.winner = "orange"
|
41
|
+
|
42
|
+
ab_test('link_color', 'blue', 'red').should == 'orange'
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
describe 'finished' do
|
47
|
+
it 'should increment the counter for the completed alternative' do
|
48
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
49
|
+
alternative_name = ab_test('link_color', 'blue', 'red')
|
50
|
+
|
51
|
+
previous_completion_count = Split::Alternative.find(alternative_name, 'link_color').completed_count
|
52
|
+
|
53
|
+
finished('link_color')
|
54
|
+
|
55
|
+
new_completion_count = Split::Alternative.find(alternative_name, 'link_color').completed_count
|
56
|
+
|
57
|
+
new_completion_count.should eql(previous_completion_count + 1)
|
58
|
+
end
|
59
|
+
|
60
|
+
it "should clear out the user's participation from their session" do
|
61
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
62
|
+
alternative_name = ab_test('link_color', 'blue', 'red')
|
63
|
+
|
64
|
+
previous_completion_count = Split::Alternative.find(alternative_name, 'link_color').completed_count
|
65
|
+
|
66
|
+
session[:split].should == {"link_color" => alternative_name}
|
67
|
+
finished('link_color')
|
68
|
+
session[:split].should == {}
|
69
|
+
end
|
70
|
+
end
|
71
|
+
|
72
|
+
describe 'conversions' do
|
73
|
+
it 'should return a conversion rate for an alternative' do
|
74
|
+
experiment = Split::Experiment.find_or_create('link_color', 'blue', 'red')
|
75
|
+
alternative_name = ab_test('link_color', 'blue', 'red')
|
76
|
+
|
77
|
+
previous_convertion_rate = Split::Alternative.find(alternative_name, 'link_color').conversion_rate
|
78
|
+
previous_convertion_rate.should eql(0.0)
|
79
|
+
|
80
|
+
finished('link_color')
|
81
|
+
|
82
|
+
new_convertion_rate = Split::Alternative.find(alternative_name, 'link_color').conversion_rate
|
83
|
+
new_convertion_rate.should eql(1.0)
|
84
|
+
end
|
85
|
+
end
|
86
|
+
end
|
data/spec/spec_helper.rb
ADDED
data/split.gemspec
ADDED
@@ -0,0 +1,27 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
$:.push File.expand_path("../lib", __FILE__)
|
3
|
+
require "split/version"
|
4
|
+
|
5
|
+
Gem::Specification.new do |s|
|
6
|
+
s.name = "split"
|
7
|
+
s.version = Split::VERSION
|
8
|
+
s.platform = Gem::Platform::RUBY
|
9
|
+
s.authors = ["Andrew Nesbitt"]
|
10
|
+
s.email = ["andrewnez@gmail.com"]
|
11
|
+
s.homepage = ""
|
12
|
+
s.summary = %q{Rack based split testing framework}
|
13
|
+
|
14
|
+
s.rubyforge_project = "split"
|
15
|
+
|
16
|
+
s.files = `git ls-files`.split("\n")
|
17
|
+
s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
|
18
|
+
s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
|
19
|
+
s.require_paths = ["lib"]
|
20
|
+
|
21
|
+
s.add_dependency(%q<redis>, ["~> 2.1"])
|
22
|
+
s.add_dependency(%q<redis-namespace>, ["~> 0.10.0"])
|
23
|
+
s.add_dependency(%q<sinatra>, ["~> 1.2.6"])
|
24
|
+
|
25
|
+
# Development Dependencies
|
26
|
+
s.add_development_dependency(%q<rspec>, ["~> 2.6"])
|
27
|
+
end
|
metadata
ADDED
@@ -0,0 +1,141 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: split
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
prerelease: false
|
5
|
+
segments:
|
6
|
+
- 0
|
7
|
+
- 1
|
8
|
+
- 0
|
9
|
+
version: 0.1.0
|
10
|
+
platform: ruby
|
11
|
+
authors:
|
12
|
+
- Andrew Nesbitt
|
13
|
+
autorequire:
|
14
|
+
bindir: bin
|
15
|
+
cert_chain: []
|
16
|
+
|
17
|
+
date: 2011-05-17 00:00:00 -04:00
|
18
|
+
default_executable:
|
19
|
+
dependencies:
|
20
|
+
- !ruby/object:Gem::Dependency
|
21
|
+
name: redis
|
22
|
+
prerelease: false
|
23
|
+
requirement: &id001 !ruby/object:Gem::Requirement
|
24
|
+
none: false
|
25
|
+
requirements:
|
26
|
+
- - ~>
|
27
|
+
- !ruby/object:Gem::Version
|
28
|
+
segments:
|
29
|
+
- 2
|
30
|
+
- 1
|
31
|
+
version: "2.1"
|
32
|
+
type: :runtime
|
33
|
+
version_requirements: *id001
|
34
|
+
- !ruby/object:Gem::Dependency
|
35
|
+
name: redis-namespace
|
36
|
+
prerelease: false
|
37
|
+
requirement: &id002 !ruby/object:Gem::Requirement
|
38
|
+
none: false
|
39
|
+
requirements:
|
40
|
+
- - ~>
|
41
|
+
- !ruby/object:Gem::Version
|
42
|
+
segments:
|
43
|
+
- 0
|
44
|
+
- 10
|
45
|
+
- 0
|
46
|
+
version: 0.10.0
|
47
|
+
type: :runtime
|
48
|
+
version_requirements: *id002
|
49
|
+
- !ruby/object:Gem::Dependency
|
50
|
+
name: sinatra
|
51
|
+
prerelease: false
|
52
|
+
requirement: &id003 !ruby/object:Gem::Requirement
|
53
|
+
none: false
|
54
|
+
requirements:
|
55
|
+
- - ~>
|
56
|
+
- !ruby/object:Gem::Version
|
57
|
+
segments:
|
58
|
+
- 1
|
59
|
+
- 2
|
60
|
+
- 6
|
61
|
+
version: 1.2.6
|
62
|
+
type: :runtime
|
63
|
+
version_requirements: *id003
|
64
|
+
- !ruby/object:Gem::Dependency
|
65
|
+
name: rspec
|
66
|
+
prerelease: false
|
67
|
+
requirement: &id004 !ruby/object:Gem::Requirement
|
68
|
+
none: false
|
69
|
+
requirements:
|
70
|
+
- - ~>
|
71
|
+
- !ruby/object:Gem::Version
|
72
|
+
segments:
|
73
|
+
- 2
|
74
|
+
- 6
|
75
|
+
version: "2.6"
|
76
|
+
type: :development
|
77
|
+
version_requirements: *id004
|
78
|
+
description:
|
79
|
+
email:
|
80
|
+
- andrewnez@gmail.com
|
81
|
+
executables: []
|
82
|
+
|
83
|
+
extensions: []
|
84
|
+
|
85
|
+
extra_rdoc_files: []
|
86
|
+
|
87
|
+
files:
|
88
|
+
- .gitignore
|
89
|
+
- Gemfile
|
90
|
+
- README.mdown
|
91
|
+
- Rakefile
|
92
|
+
- lib/split.rb
|
93
|
+
- lib/split/alternative.rb
|
94
|
+
- lib/split/dashboard.rb
|
95
|
+
- lib/split/dashboard/public/reset.css
|
96
|
+
- lib/split/dashboard/public/style.css
|
97
|
+
- lib/split/dashboard/views/index.erb
|
98
|
+
- lib/split/dashboard/views/layout.erb
|
99
|
+
- lib/split/experiment.rb
|
100
|
+
- lib/split/helper.rb
|
101
|
+
- lib/split/version.rb
|
102
|
+
- spec/experiment_spec.rb
|
103
|
+
- spec/helper_spec.rb
|
104
|
+
- spec/spec_helper.rb
|
105
|
+
- split.gemspec
|
106
|
+
has_rdoc: true
|
107
|
+
homepage: ""
|
108
|
+
licenses: []
|
109
|
+
|
110
|
+
post_install_message:
|
111
|
+
rdoc_options: []
|
112
|
+
|
113
|
+
require_paths:
|
114
|
+
- lib
|
115
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
116
|
+
none: false
|
117
|
+
requirements:
|
118
|
+
- - ">="
|
119
|
+
- !ruby/object:Gem::Version
|
120
|
+
segments:
|
121
|
+
- 0
|
122
|
+
version: "0"
|
123
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
124
|
+
none: false
|
125
|
+
requirements:
|
126
|
+
- - ">="
|
127
|
+
- !ruby/object:Gem::Version
|
128
|
+
segments:
|
129
|
+
- 0
|
130
|
+
version: "0"
|
131
|
+
requirements: []
|
132
|
+
|
133
|
+
rubyforge_project: split
|
134
|
+
rubygems_version: 1.3.7
|
135
|
+
signing_key:
|
136
|
+
specification_version: 3
|
137
|
+
summary: Rack based split testing framework
|
138
|
+
test_files:
|
139
|
+
- spec/experiment_spec.rb
|
140
|
+
- spec/helper_spec.rb
|
141
|
+
- spec/spec_helper.rb
|