split 0.4.6 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.travis.yml +11 -3
- data/CHANGELOG.mdown +22 -1
- data/CONTRIBUTING.md +10 -0
- data/LICENSE +1 -1
- data/README.mdown +235 -60
- data/lib/split.rb +8 -9
- data/lib/split/algorithms.rb +3 -0
- data/lib/split/algorithms/weighted_sample.rb +17 -0
- data/lib/split/algorithms/whiplash.rb +35 -0
- data/lib/split/alternative.rb +12 -4
- data/lib/split/configuration.rb +91 -1
- data/lib/split/dashboard/helpers.rb +3 -3
- data/lib/split/dashboard/views/_experiment.erb +1 -1
- data/lib/split/exceptions.rb +4 -0
- data/lib/split/experiment.rb +112 -24
- data/lib/split/extensions.rb +3 -0
- data/lib/split/extensions/array.rb +4 -0
- data/lib/split/extensions/string.rb +15 -0
- data/lib/split/helper.rb +87 -55
- data/lib/split/metric.rb +68 -0
- data/lib/split/persistence.rb +28 -0
- data/lib/split/persistence/cookie_adapter.rb +44 -0
- data/lib/split/persistence/session_adapter.rb +28 -0
- data/lib/split/trial.rb +43 -0
- data/lib/split/version.rb +3 -3
- data/spec/algorithms/weighted_sample_spec.rb +18 -0
- data/spec/algorithms/whiplash_spec.rb +23 -0
- data/spec/alternative_spec.rb +81 -9
- data/spec/configuration_spec.rb +61 -9
- data/spec/dashboard_helpers_spec.rb +2 -5
- data/spec/dashboard_spec.rb +0 -2
- data/spec/experiment_spec.rb +144 -74
- data/spec/helper_spec.rb +234 -29
- data/spec/metric_spec.rb +30 -0
- data/spec/persistence/cookie_adapter_spec.rb +31 -0
- data/spec/persistence/session_adapter_spec.rb +31 -0
- data/spec/persistence_spec.rb +33 -0
- data/spec/spec_helper.rb +12 -0
- data/spec/support/cookies_mock.rb +19 -0
- data/spec/trial_spec.rb +59 -0
- data/split.gemspec +7 -3
- metadata +58 -29
- data/Guardfile +0 -5
data/.travis.yml
CHANGED
@@ -3,7 +3,15 @@ rvm:
|
|
3
3
|
- 1.8.7
|
4
4
|
- 1.9.2
|
5
5
|
- 1.9.3
|
6
|
-
-
|
7
|
-
-
|
6
|
+
- 2.0.0
|
7
|
+
- ruby-head
|
8
|
+
- jruby-18mode
|
9
|
+
- jruby-19mode
|
10
|
+
- jruby-head
|
8
11
|
- rbx-18mode
|
9
|
-
- rbx-19mode
|
12
|
+
- rbx-19mode
|
13
|
+
- ree
|
14
|
+
matrix:
|
15
|
+
allow_failures:
|
16
|
+
- rvm: ruby-head
|
17
|
+
- rvm: 2.0.0
|
data/CHANGELOG.mdown
CHANGED
@@ -1,3 +1,24 @@
|
|
1
|
+
## 0.5.0 (January 28, 2012)
|
2
|
+
|
3
|
+
Features:
|
4
|
+
|
5
|
+
- Persistence Adapters: Cookies and Session (@patbenatar, #98)
|
6
|
+
- Configure experiments from a hash (@iangreenleaf, #97)
|
7
|
+
- Pluggable sampling algorithms (@woodhull, #105)
|
8
|
+
|
9
|
+
Bugfixes:
|
10
|
+
|
11
|
+
- Fixed negative number of non-finished rates (@philnash, #83)
|
12
|
+
- Fixed behaviour of finished(:reset => false) (@philnash, #88)
|
13
|
+
- Only take into consideration positive z-scores (@thomasmaas, #96)
|
14
|
+
|
15
|
+
## 0.4.7 (November 1, 2012)
|
16
|
+
|
17
|
+
Bugfixes:
|
18
|
+
|
19
|
+
- Amended ab_test method to raise ArgumentError if passed integers or symbols as
|
20
|
+
alternatives (@buddhamagnet, #81)
|
21
|
+
|
1
22
|
## 0.4.6 (October 28, 2012)
|
2
23
|
|
3
24
|
Features:
|
@@ -176,4 +197,4 @@ Bugfixes:
|
|
176
197
|
|
177
198
|
## 0.1.0 (May 17, 2011)
|
178
199
|
|
179
|
-
Initial Release
|
200
|
+
Initial Release
|
data/CONTRIBUTING.md
ADDED
@@ -0,0 +1,10 @@
|
|
1
|
+
## Contributing
|
2
|
+
* Fork the project.
|
3
|
+
* Make your feature addition or bug fix.
|
4
|
+
* Add tests for it. This is important so I don't break it in a
|
5
|
+
future version unintentionally.
|
6
|
+
* Add documentation if necessary.
|
7
|
+
* Commit, do not mess with rakefile, version, or history.
|
8
|
+
(if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
|
9
|
+
* Send a pull request. Bonus points for topic branches.
|
10
|
+
* Discussion at the [Google Group](https://groups.google.com/d/forum/split-ruby)
|
data/LICENSE
CHANGED
data/README.mdown
CHANGED
@@ -6,7 +6,10 @@ Split is heavily inspired by the Abingo and Vanity rails ab testing plugins and
|
|
6
6
|
|
7
7
|
Split is designed to be hacker friendly, allowing for maximum customisation and extensibility.
|
8
8
|
|
9
|
-
[](http://badge.fury.io/rb/split)
|
10
|
+
[](http://travis-ci.org/andrew/split)
|
11
|
+
[](https://gemnasium.com/andrew/split)
|
12
|
+
[](https://codeclimate.com/github/andrew/split)
|
10
13
|
|
11
14
|
## Requirements
|
12
15
|
|
@@ -16,8 +19,10 @@ Split only supports redis 2.0 or greater.
|
|
16
19
|
|
17
20
|
If you're on OS X, Homebrew is the simplest way to install Redis:
|
18
21
|
|
19
|
-
|
20
|
-
|
22
|
+
```bash
|
23
|
+
$ brew install redis
|
24
|
+
$ redis-server /usr/local/etc/redis.conf
|
25
|
+
```
|
21
26
|
|
22
27
|
You now have a Redis daemon running on 6379.
|
23
28
|
|
@@ -25,19 +30,27 @@ You now have a Redis daemon running on 6379.
|
|
25
30
|
|
26
31
|
If you are using bundler add split to your Gemfile:
|
27
32
|
|
28
|
-
|
33
|
+
``` ruby
|
34
|
+
gem 'split'
|
35
|
+
```
|
29
36
|
|
30
37
|
Then run:
|
31
38
|
|
32
|
-
|
39
|
+
```bash
|
40
|
+
$ bundle install
|
41
|
+
```
|
33
42
|
|
34
43
|
Otherwise install the gem:
|
35
44
|
|
36
|
-
|
45
|
+
```bash
|
46
|
+
$ gem install split
|
47
|
+
```
|
37
48
|
|
38
49
|
and require it in your project:
|
39
50
|
|
40
|
-
|
51
|
+
```ruby
|
52
|
+
require 'split'
|
53
|
+
```
|
41
54
|
|
42
55
|
### SystemTimer
|
43
56
|
|
@@ -45,7 +58,9 @@ If you are using Redis on Ruby 1.8.x then you will likely want to also use the S
|
|
45
58
|
|
46
59
|
Put the following in your gemfile as well:
|
47
60
|
|
48
|
-
|
61
|
+
```
|
62
|
+
gem 'SystemTimer'
|
63
|
+
```
|
49
64
|
|
50
65
|
### Rails
|
51
66
|
|
@@ -55,13 +70,15 @@ Split is autoloaded when rails starts up, as long as you've configured redis it
|
|
55
70
|
|
56
71
|
To configure sinatra with Split you need to enable sessions and mix in the helper methods. Add the following lines at the top of your sinatra app:
|
57
72
|
|
58
|
-
|
59
|
-
|
60
|
-
|
73
|
+
```ruby
|
74
|
+
class MySinatraApp < Sinatra::Base
|
75
|
+
enable :sessions
|
76
|
+
helpers Split::Helper
|
61
77
|
|
62
|
-
|
63
|
-
|
64
|
-
|
78
|
+
get '/' do
|
79
|
+
...
|
80
|
+
end
|
81
|
+
```
|
65
82
|
|
66
83
|
## Usage
|
67
84
|
|
@@ -75,27 +92,35 @@ It can be used to render different templates, show different text or any other c
|
|
75
92
|
|
76
93
|
Example: View
|
77
94
|
|
78
|
-
|
79
|
-
|
80
|
-
|
95
|
+
```ruby
|
96
|
+
<% ab_test("login_button", "/images/button1.jpg", "/images/button2.jpg") do |button_file| %>
|
97
|
+
<%= img_tag(button_file, :alt => "Login!") %>
|
98
|
+
<% end %>
|
99
|
+
```
|
81
100
|
|
82
101
|
Example: Controller
|
83
102
|
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
|
103
|
+
```ruby
|
104
|
+
def register_new_user
|
105
|
+
# See what level of free points maximizes users' decision to buy replacement points.
|
106
|
+
@starter_points = ab_test("new_user_free_points", '100', '200', '300')
|
107
|
+
end
|
108
|
+
```
|
88
109
|
|
89
110
|
Example: Conversion tracking (in a controller!)
|
90
111
|
|
91
|
-
|
92
|
-
|
93
|
-
|
94
|
-
|
112
|
+
```ruby
|
113
|
+
def buy_new_points
|
114
|
+
# some business logic
|
115
|
+
finished("new_user_free_points")
|
116
|
+
end
|
117
|
+
```
|
95
118
|
|
96
119
|
Example: Conversion tracking (in a view)
|
97
120
|
|
98
|
-
|
121
|
+
```ruby
|
122
|
+
Thanks for signing up, dude! <% finished("signup_page_redesign") >
|
123
|
+
```
|
99
124
|
|
100
125
|
You can find more examples, tutorials and guides on the [wiki](https://github.com/andrew/split/wiki).
|
101
126
|
|
@@ -107,11 +132,13 @@ Perhaps you only want to show an alternative to 10% of your visitors because it
|
|
107
132
|
|
108
133
|
To do this you can pass a weight with each alternative in the following ways:
|
109
134
|
|
110
|
-
|
135
|
+
```ruby
|
136
|
+
ab_test('homepage design', {'Old' => 20}, {'New' => 2})
|
111
137
|
|
112
|
-
|
138
|
+
ab_test('homepage design', 'Old', {'New' => 0.1})
|
113
139
|
|
114
|
-
|
140
|
+
ab_test('homepage design', {'Old' => 10}, 'New')
|
141
|
+
```
|
115
142
|
|
116
143
|
Note: If using ruby 1.8.x and weighted alternatives you should always pass the control alternative through as the second argument with any other alternatives as a third argument because the order of the hash is not preserved in ruby 1.8, ruby 1.9.1+ users are not affected by this bug.
|
117
144
|
|
@@ -134,7 +161,9 @@ When a user completes a test their session is reset so that they may start the t
|
|
134
161
|
|
135
162
|
To stop this behaviour you can pass the following option to the `finished` method:
|
136
163
|
|
137
|
-
|
164
|
+
```ruby
|
165
|
+
finished('experiment_name', :reset => false)
|
166
|
+
```
|
138
167
|
|
139
168
|
The user will then always see the alternative they started with.
|
140
169
|
|
@@ -144,9 +173,40 @@ By default Split will avoid users participating in multiple experiments at once.
|
|
144
173
|
|
145
174
|
To stop this behaviour and allow users to participate in multiple experiments at once enable the `allow_multiple_experiments` config option like so:
|
146
175
|
|
147
|
-
|
148
|
-
|
149
|
-
|
176
|
+
```ruby
|
177
|
+
Split.configure do |config|
|
178
|
+
config.allow_multiple_experiments = true
|
179
|
+
end
|
180
|
+
```
|
181
|
+
|
182
|
+
### Experiment Persistence
|
183
|
+
|
184
|
+
Split comes with two built-in persistence adapters for storing users and the alternatives they've been given for each experiment.
|
185
|
+
|
186
|
+
By default Split will store the tests for each user in the session.
|
187
|
+
|
188
|
+
You can optionally configure Split to use a cookie or any custom adapter of your choosing.
|
189
|
+
|
190
|
+
#### Cookies
|
191
|
+
|
192
|
+
```ruby
|
193
|
+
Split.configure do |config|
|
194
|
+
config.persistence = :cookie
|
195
|
+
end
|
196
|
+
```
|
197
|
+
|
198
|
+
__Note:__ Using cookies depends on `ActionDispatch::Cookies` or any identical API
|
199
|
+
|
200
|
+
#### Custom Adapter
|
201
|
+
|
202
|
+
Your custom adapter needs to implement the same API as existing adapters.
|
203
|
+
See `Split::Persistance::CookieAdapter` or `Split::Persistence::SessionAdapter` for a starting point.
|
204
|
+
|
205
|
+
```ruby
|
206
|
+
Split.configure do |config|
|
207
|
+
config.persistence = YourCustomAdapterClass
|
208
|
+
end
|
209
|
+
```
|
150
210
|
|
151
211
|
## Web Interface
|
152
212
|
|
@@ -154,39 +214,112 @@ Split comes with a Sinatra-based front end to get an overview of how your experi
|
|
154
214
|
|
155
215
|
If you are running Rails 2: You can mount this inside your app using Rack::URLMap in your `config.ru`
|
156
216
|
|
157
|
-
|
217
|
+
```ruby
|
218
|
+
require 'split/dashboard'
|
158
219
|
|
159
|
-
|
160
|
-
|
161
|
-
|
220
|
+
run Rack::URLMap.new \
|
221
|
+
"/" => Your::App.new,
|
222
|
+
"/split" => Split::Dashboard.new
|
223
|
+
```
|
162
224
|
|
163
225
|
However, if you are using Rails 3: You can mount this inside your app routes by first adding this to the Gemfile:
|
164
226
|
|
165
|
-
|
227
|
+
```ruby
|
228
|
+
gem 'split', :require => 'split/dashboard'
|
229
|
+
```
|
166
230
|
|
167
231
|
Then adding this to config/routes.rb
|
168
232
|
|
169
|
-
|
233
|
+
```ruby
|
234
|
+
mount Split::Dashboard, :at => 'split'
|
235
|
+
```
|
170
236
|
|
171
237
|
You may want to password protect that page, you can do so with `Rack::Auth::Basic`
|
172
238
|
|
173
|
-
|
174
|
-
|
175
|
-
|
239
|
+
```ruby
|
240
|
+
Split::Dashboard.use Rack::Auth::Basic do |username, password|
|
241
|
+
username == 'admin' && password == 'p4s5w0rd'
|
242
|
+
end
|
243
|
+
```
|
176
244
|
|
177
245
|
## Configuration
|
178
246
|
|
179
247
|
You can override the default configuration options of Split like so:
|
180
248
|
|
249
|
+
```ruby
|
250
|
+
Split.configure do |config|
|
251
|
+
config.robot_regex = /my_custom_robot_regex/
|
252
|
+
config.ignore_ip_addresses << '81.19.48.130'
|
253
|
+
config.db_failover = true # handle redis errors gracefully
|
254
|
+
config.db_failover_on_db_error = proc{|error| Rails.logger.error(error.message) }
|
255
|
+
config.allow_multiple_experiments = true
|
256
|
+
config.enabled = true
|
257
|
+
config.persistence = Split::Persistence::SessionAdapter
|
258
|
+
end
|
259
|
+
```
|
260
|
+
|
261
|
+
### Experiment configuration
|
262
|
+
|
263
|
+
Instead of providing the experiment options inline, you can store them
|
264
|
+
in a hash or a configuration file:
|
265
|
+
|
266
|
+
Split.configure do |config|
|
267
|
+
config.experiments = YAML.load_file "config/experiments.yml"
|
268
|
+
end
|
269
|
+
|
270
|
+
This hash can control your experiment's variants, weights, algorithm and if the
|
271
|
+
experiment resets once finished:
|
272
|
+
|
181
273
|
Split.configure do |config|
|
182
|
-
config.
|
183
|
-
|
184
|
-
|
185
|
-
|
186
|
-
|
187
|
-
|
274
|
+
config.experiments = {
|
275
|
+
:my_first_experiment => {
|
276
|
+
:variants => ["a", "b"],
|
277
|
+
:resettable => false,
|
278
|
+
},
|
279
|
+
:my_second_experiment => {
|
280
|
+
:algorithm => 'Split::Algorithms::Whiplash',
|
281
|
+
:variants => [
|
282
|
+
{ :name => "a", :percent => 67 },
|
283
|
+
{ :name => "b", :percent => 33 },
|
284
|
+
]
|
285
|
+
}
|
286
|
+
}
|
188
287
|
end
|
189
288
|
|
289
|
+
This simplifies the calls from your code:
|
290
|
+
|
291
|
+
ab_test(:my_first_experiment)
|
292
|
+
|
293
|
+
and:
|
294
|
+
|
295
|
+
finished(:my_first_experiment)
|
296
|
+
|
297
|
+
#### Metrics
|
298
|
+
|
299
|
+
You might wish to track generic metrics, such as conversions, and use
|
300
|
+
those to complete multiple different experiments without adding more to
|
301
|
+
your code. You can use the configuration hash to do this, thanks to
|
302
|
+
the `:metric` option.
|
303
|
+
|
304
|
+
Split.configure do |config|
|
305
|
+
config.experiments = {
|
306
|
+
:my_first_experiment => {
|
307
|
+
:variants => ["a", "b"],
|
308
|
+
:metric => :conversion,
|
309
|
+
}
|
310
|
+
}
|
311
|
+
end
|
312
|
+
|
313
|
+
Your code may then track a completion using the metric instead of
|
314
|
+
the experiment name:
|
315
|
+
|
316
|
+
finished(:conversion)
|
317
|
+
|
318
|
+
You can also create a new metric by instantiating and saving a new Metric object.
|
319
|
+
|
320
|
+
Split::Metric.new(:conversion)
|
321
|
+
Split::Metric.save
|
322
|
+
|
190
323
|
### DB failover solution
|
191
324
|
|
192
325
|
Due to the fact that Redis has no autom. failover mechanism, it's
|
@@ -216,19 +349,23 @@ appropriately.
|
|
216
349
|
|
217
350
|
Here's our `config/split.yml`:
|
218
351
|
|
219
|
-
|
220
|
-
|
221
|
-
|
222
|
-
|
223
|
-
|
352
|
+
```yml
|
353
|
+
development: localhost:6379
|
354
|
+
test: localhost:6379
|
355
|
+
staging: redis1.example.com:6379
|
356
|
+
fi: localhost:6379
|
357
|
+
production: redis1.example.com:6379
|
358
|
+
```
|
224
359
|
|
225
360
|
And our initializer:
|
226
361
|
|
227
|
-
|
228
|
-
|
362
|
+
```ruby
|
363
|
+
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
|
364
|
+
rails_env = ENV['RAILS_ENV'] || 'development'
|
229
365
|
|
230
|
-
|
231
|
-
|
366
|
+
split_config = YAML.load_file(rails_root + '/config/split.yml')
|
367
|
+
Split.redis = split_config[rails_env]
|
368
|
+
```
|
232
369
|
|
233
370
|
## Namespaces
|
234
371
|
|
@@ -242,11 +379,45 @@ in your Redis server.
|
|
242
379
|
|
243
380
|
Simply use the `Split.redis.namespace` accessor:
|
244
381
|
|
245
|
-
|
382
|
+
```ruby
|
383
|
+
Split.redis.namespace = "split:blog"
|
384
|
+
```
|
246
385
|
|
247
386
|
We recommend sticking this in your initializer somewhere after Redis
|
248
387
|
is configured.
|
249
388
|
|
389
|
+
## Outside of a Web Session
|
390
|
+
|
391
|
+
Split provides the Helper module to facilitate running experiments inside web sessions.
|
392
|
+
|
393
|
+
Alternatively, you can access the underlying Metric, Trial, Experiment and Alternative objects to
|
394
|
+
conduct experiments that are not tied to a web session.
|
395
|
+
|
396
|
+
```ruby
|
397
|
+
# create a new experiment
|
398
|
+
experiment = Split::Experiment.find_or_create('color', 'red', 'blue')
|
399
|
+
# create a new trial
|
400
|
+
trial = Trial.new(:experiment => experiment)
|
401
|
+
# run trial
|
402
|
+
trial.choose!
|
403
|
+
# get the result, returns either red or blue
|
404
|
+
trial.alternative.name
|
405
|
+
|
406
|
+
# if the goal has been achieved, increment the successful completions for this alternative.
|
407
|
+
if goal_acheived?
|
408
|
+
trial.complete!
|
409
|
+
end
|
410
|
+
|
411
|
+
```
|
412
|
+
|
413
|
+
## Algorithms
|
414
|
+
|
415
|
+
By default, Split ships with an algorithm that randomly selects from possible alternatives for a traditional a/b test.
|
416
|
+
|
417
|
+
An implementation of a bandit algorithm is also provided.
|
418
|
+
|
419
|
+
Users may also write their own algorithms. The default algorithm may be specified globally in the configuration file, or on a per experiment basis using the experiments hash of the configuration file.
|
420
|
+
|
250
421
|
## Extensions
|
251
422
|
|
252
423
|
- [Split::Export](http://github.com/andrew/split-export) - easily export ab test data out of Split
|
@@ -263,11 +434,14 @@ Special thanks to the following people for submitting patches:
|
|
263
434
|
* Lloyd Pick
|
264
435
|
* Jeffery Chupp
|
265
436
|
* Andrew Appleton
|
437
|
+
* Phil Nash
|
438
|
+
* Dave Goodchild
|
266
439
|
|
267
440
|
## Development
|
268
441
|
|
269
442
|
Source hosted at [GitHub](http://github.com/andrew/split).
|
270
443
|
Report Issues/Feature requests on [GitHub Issues](http://github.com/andrew/split/issues).
|
444
|
+
Discussion at [Google Groups](https://groups.google.com/d/forum/split-ruby)
|
271
445
|
|
272
446
|
Tests can be ran with `rake spec`
|
273
447
|
|
@@ -277,10 +451,11 @@ Tests can be ran with `rake spec`
|
|
277
451
|
* Make your feature addition or bug fix.
|
278
452
|
* Add tests for it. This is important so I don't break it in a
|
279
453
|
future version unintentionally.
|
454
|
+
* Add documentation if necessary.
|
280
455
|
* Commit, do not mess with rakefile, version, or history.
|
281
456
|
(if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
|
282
|
-
* Send
|
457
|
+
* Send a pull request. Bonus points for topic branches.
|
283
458
|
|
284
459
|
## Copyright
|
285
460
|
|
286
|
-
Copyright (c)
|
461
|
+
Copyright (c) 2013 Andrew Nesbitt. See [LICENSE](https://github.com/andrew/split/blob/master/LICENSE) for details.
|