ab-split 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (93) hide show
  1. checksums.yaml +7 -0
  2. data/.codeclimate.yml +30 -0
  3. data/.csslintrc +2 -0
  4. data/.eslintignore +1 -0
  5. data/.eslintrc +213 -0
  6. data/.github/FUNDING.yml +1 -0
  7. data/.github/ISSUE_TEMPLATE/bug_report.md +24 -0
  8. data/.rspec +1 -0
  9. data/.rubocop.yml +7 -0
  10. data/.rubocop_todo.yml +679 -0
  11. data/.travis.yml +60 -0
  12. data/Appraisals +19 -0
  13. data/CHANGELOG.md +696 -0
  14. data/CODE_OF_CONDUCT.md +74 -0
  15. data/CONTRIBUTING.md +62 -0
  16. data/Gemfile +7 -0
  17. data/LICENSE +22 -0
  18. data/README.md +955 -0
  19. data/Rakefile +9 -0
  20. data/ab-split.gemspec +44 -0
  21. data/gemfiles/4.2.gemfile +9 -0
  22. data/gemfiles/5.0.gemfile +9 -0
  23. data/gemfiles/5.1.gemfile +9 -0
  24. data/gemfiles/5.2.gemfile +9 -0
  25. data/gemfiles/6.0.gemfile +9 -0
  26. data/lib/split.rb +76 -0
  27. data/lib/split/algorithms/block_randomization.rb +23 -0
  28. data/lib/split/algorithms/weighted_sample.rb +18 -0
  29. data/lib/split/algorithms/whiplash.rb +38 -0
  30. data/lib/split/alternative.rb +191 -0
  31. data/lib/split/combined_experiments_helper.rb +37 -0
  32. data/lib/split/configuration.rb +255 -0
  33. data/lib/split/dashboard.rb +74 -0
  34. data/lib/split/dashboard/helpers.rb +45 -0
  35. data/lib/split/dashboard/pagination_helpers.rb +86 -0
  36. data/lib/split/dashboard/paginator.rb +16 -0
  37. data/lib/split/dashboard/public/dashboard-filtering.js +43 -0
  38. data/lib/split/dashboard/public/dashboard.js +24 -0
  39. data/lib/split/dashboard/public/jquery-1.11.1.min.js +4 -0
  40. data/lib/split/dashboard/public/reset.css +48 -0
  41. data/lib/split/dashboard/public/style.css +328 -0
  42. data/lib/split/dashboard/views/_controls.erb +18 -0
  43. data/lib/split/dashboard/views/_experiment.erb +155 -0
  44. data/lib/split/dashboard/views/_experiment_with_goal_header.erb +8 -0
  45. data/lib/split/dashboard/views/index.erb +26 -0
  46. data/lib/split/dashboard/views/layout.erb +27 -0
  47. data/lib/split/encapsulated_helper.rb +42 -0
  48. data/lib/split/engine.rb +15 -0
  49. data/lib/split/exceptions.rb +6 -0
  50. data/lib/split/experiment.rb +486 -0
  51. data/lib/split/experiment_catalog.rb +51 -0
  52. data/lib/split/extensions/string.rb +16 -0
  53. data/lib/split/goals_collection.rb +45 -0
  54. data/lib/split/helper.rb +165 -0
  55. data/lib/split/metric.rb +101 -0
  56. data/lib/split/persistence.rb +28 -0
  57. data/lib/split/persistence/cookie_adapter.rb +94 -0
  58. data/lib/split/persistence/dual_adapter.rb +85 -0
  59. data/lib/split/persistence/redis_adapter.rb +57 -0
  60. data/lib/split/persistence/session_adapter.rb +29 -0
  61. data/lib/split/redis_interface.rb +50 -0
  62. data/lib/split/trial.rb +117 -0
  63. data/lib/split/user.rb +69 -0
  64. data/lib/split/version.rb +7 -0
  65. data/lib/split/zscore.rb +57 -0
  66. data/spec/algorithms/block_randomization_spec.rb +32 -0
  67. data/spec/algorithms/weighted_sample_spec.rb +19 -0
  68. data/spec/algorithms/whiplash_spec.rb +24 -0
  69. data/spec/alternative_spec.rb +320 -0
  70. data/spec/combined_experiments_helper_spec.rb +57 -0
  71. data/spec/configuration_spec.rb +258 -0
  72. data/spec/dashboard/pagination_helpers_spec.rb +200 -0
  73. data/spec/dashboard/paginator_spec.rb +37 -0
  74. data/spec/dashboard_helpers_spec.rb +42 -0
  75. data/spec/dashboard_spec.rb +210 -0
  76. data/spec/encapsulated_helper_spec.rb +52 -0
  77. data/spec/experiment_catalog_spec.rb +53 -0
  78. data/spec/experiment_spec.rb +533 -0
  79. data/spec/goals_collection_spec.rb +80 -0
  80. data/spec/helper_spec.rb +1111 -0
  81. data/spec/metric_spec.rb +31 -0
  82. data/spec/persistence/cookie_adapter_spec.rb +106 -0
  83. data/spec/persistence/dual_adapter_spec.rb +194 -0
  84. data/spec/persistence/redis_adapter_spec.rb +90 -0
  85. data/spec/persistence/session_adapter_spec.rb +32 -0
  86. data/spec/persistence_spec.rb +34 -0
  87. data/spec/redis_interface_spec.rb +111 -0
  88. data/spec/spec_helper.rb +52 -0
  89. data/spec/split_spec.rb +43 -0
  90. data/spec/support/cookies_mock.rb +20 -0
  91. data/spec/trial_spec.rb +299 -0
  92. data/spec/user_spec.rb +87 -0
  93. metadata +322 -0
@@ -0,0 +1,74 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, gender identity and expression, level of experience,
9
+ nationality, personal appearance, race, religion, or sexual identity and
10
+ orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at andrewnez@gmail.com. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at [https://contributor-covenant.org/version/1/4][version]
72
+
73
+ [homepage]: https://contributor-covenant.org
74
+ [version]: https://contributor-covenant.org/version/1/4/
data/CONTRIBUTING.md ADDED
@@ -0,0 +1,62 @@
1
+ # Contributing to Split
2
+
3
+ Want to contribute to Split? That's great! Here are a couple of guidelines that will help you contribute. Before we get started: Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md) to ensure that this project is a welcoming place for **everyone** to contribute to. By participating in this project you agree to abide by its terms.
4
+
5
+ #### Overview
6
+
7
+ * [Contribution workflow](#contribution-workflow)
8
+ * [Setup instructions](#setup-instructions)
9
+ * [Reporting a bug](#reporting-a-bug)
10
+ * [Contributing to an existing issue](#contributing-to-an-existing-issue)
11
+ * [Our labels](#our-labels)
12
+ * [Additional info](#additional-info)
13
+
14
+ ## Contribution workflow
15
+
16
+ * Fork the project.
17
+ * Make your feature addition or bug fix.
18
+ * Add tests for it. This is important so I don't break it in a
19
+ future version unintentionally.
20
+ * Add documentation if necessary.
21
+ * Commit. Do not mess with the Rakefile, version, or history.
22
+ (If you want to have your own version, that is fine. But bump version in a commit by itself I can ignore when I pull.)
23
+ * Send a pull request. Bonus points for topic branches.
24
+ * Discussion at the [Google Group](https://groups.google.com/d/forum/split-ruby)
25
+
26
+ ## Setup instructions
27
+
28
+ You can find in-depth instructions to install in our [README](https://github.com/splitrb/split/blob/master/README.md).
29
+
30
+ *Note*: Split requires Ruby 1.9.2 or higher.
31
+
32
+ ## Reporting a bug
33
+
34
+ So you've found a bug, and want to help us fix it? Before filing a bug report, please double-check the bug hasn't already been reported. You can do so [on our issue tracker](https://github.com/splitrb/split/issues?q=is%3Aissue+is%3Aopen+label%3Abug). If something hasn't been raised, you can go ahead and create a new issue with the following information:
35
+
36
+ * When did the error happen?
37
+ * How can the error be reproduced?
38
+ * If possible, please also provide an error message or a screenshot to illustrate the problem.
39
+
40
+ If you want to be really thorough, there is a great overview on Stack Overflow of [what you should consider when reporting a bug](http://stackoverflow.com/questions/240323/how-to-report-bugs-the-smart-way).
41
+
42
+ It goes without saying that you're welcome to help investigate further and/or find a fix for the bug. If you want to do so, just mention it in your bug report and offer your help!
43
+
44
+ ## Contributing to an existing issue
45
+
46
+ ### Finding an issue to work on
47
+
48
+ We've got a few open issues and are always glad to get help on that front. You can view the list of issues [here](https://github.com/splitrb/split/issues). Most of the issues are labelled, so you can use the labels to get an idea of which issue could be a good fit for you. (Here's [a good article](https://medium.freecodecamp.com/finding-your-first-open-source-project-or-bug-to-work-on-1712f651e5ba) on how to find your first bug to fix).
49
+
50
+ Before getting to work, take a look at the issue and at the conversation around it. Has someone already offered to work on the issue? Has someone been assigned to the issue? If so, you might want to check with them to see whether they're still actively working on it.
51
+
52
+ If the issue is a few months old, it might be a good idea to write a short comment to double-check that the issue or feature is still a valid one to jump on.
53
+
54
+ Feel free to ask for more detail on what is expected: are there any more details or specifications you need to know?
55
+
56
+ And if at any point you get stuck: don't hesitate to ask for help.
57
+
58
+ ### Making your contribution
59
+
60
+ We've outlined the contribution workflow [here](#contribution-workflow). If you're a first-timer, don't worry! GitHub has a ton of guides to help you through your first pull request: You can find out more about pull requests [here](https://help.github.com/articles/about-pull-requests/) and about creating a pull request [here](https://help.github.com/articles/creating-a-pull-request/).
61
+
62
+ Especially if you're a newcomer to Open Source and you've found some little bumps along the way while contributing, we recommend you write about them. [Here](https://medium.freecodecamp.com/new-contributors-to-open-source-please-blog-more-920af14cffd)'s a great article about why writing about your experience is important; this will encourage other beginners to try their luck at Open Source, too!
data/Gemfile ADDED
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+ source "https://rubygems.org"
3
+
4
+ gemspec
5
+
6
+ gem "appraisal"
7
+ gem "codeclimate-test-reporter"
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2017 Andrew Nesbitt
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,955 @@
1
+ # [Split](https://libraries.io/rubygems/split)
2
+
3
+ [![Gem Version](https://badge.fury.io/rb/split.svg)](http://badge.fury.io/rb/split)
4
+ [![Build Status](https://secure.travis-ci.org/splitrb/split.svg?branch=master)](https://travis-ci.org/splitrb/split)
5
+ [![Code Climate](https://codeclimate.com/github/splitrb/split/badges/gpa.svg)](https://codeclimate.com/github/splitrb/split)
6
+ [![Test Coverage](https://codeclimate.com/github/splitrb/split/badges/coverage.svg)](https://codeclimate.com/github/splitrb/split/coverage)
7
+ [![standard-readme compliant](https://img.shields.io/badge/readme%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme)
8
+ [![Open Source Helpers](https://www.codetriage.com/splitrb/split/badges/users.svg)](https://www.codetriage.com/splitrb/split)
9
+
10
+ > 📈 The Rack Based A/B testing framework https://libraries.io/rubygems/split
11
+
12
+ Split is a rack based A/B testing framework designed to work with Rails, Sinatra or any other rack based app.
13
+
14
+ Split is heavily inspired by the [Abingo](https://github.com/ryanb/abingo) and [Vanity](https://github.com/assaf/vanity) Rails A/B testing plugins and [Resque](https://github.com/resque/resque) in its use of Redis.
15
+
16
+ Split is designed to be hacker friendly, allowing for maximum customisation and extensibility.
17
+
18
+ ## Install
19
+
20
+ ### Requirements
21
+
22
+ Split currently requires Ruby 1.9.3 or higher. If your project requires compatibility with Ruby 1.8.x and Rails 2.3, please use v0.8.0.
23
+
24
+ Split uses Redis as a datastore.
25
+
26
+ Split only supports Redis 2.0 or greater.
27
+
28
+ If you're on OS X, Homebrew is the simplest way to install Redis:
29
+
30
+ ```bash
31
+ brew install redis
32
+ redis-server /usr/local/etc/redis.conf
33
+ ```
34
+
35
+ You now have a Redis daemon running on port `6379`.
36
+
37
+ ### Setup
38
+
39
+ ```bash
40
+ gem install split
41
+ ```
42
+
43
+ #### Rails
44
+
45
+ Adding `gem 'split'` to your Gemfile will autoload it when rails starts up, as long as you've configured Redis it will 'just work'.
46
+
47
+ #### Sinatra
48
+
49
+ To configure Sinatra with Split you need to enable sessions and mix in the helper methods. Add the following lines at the top of your Sinatra app:
50
+
51
+ ```ruby
52
+ require 'split'
53
+
54
+ class MySinatraApp < Sinatra::Base
55
+ enable :sessions
56
+ helpers Split::Helper
57
+
58
+ get '/' do
59
+ ...
60
+ end
61
+ ```
62
+
63
+ ## Usage
64
+
65
+ To begin your A/B test use the `ab_test` method, naming your experiment with the first argument and then the different alternatives which you wish to test on as the other arguments.
66
+
67
+ `ab_test` returns one of the alternatives, if a user has already seen that test they will get the same alternative as before, which you can use to split your code on.
68
+
69
+ It can be used to render different templates, show different text or any other case based logic.
70
+
71
+ `ab_finished` is used to make a completion of an experiment, or conversion.
72
+
73
+ Example: View
74
+
75
+ ```erb
76
+ <% ab_test(:login_button, "/images/button1.jpg", "/images/button2.jpg") do |button_file| %>
77
+ <%= image_tag(button_file, alt: "Login!") %>
78
+ <% end %>
79
+ ```
80
+
81
+ Example: Controller
82
+
83
+ ```ruby
84
+ def register_new_user
85
+ # See what level of free points maximizes users' decision to buy replacement points.
86
+ @starter_points = ab_test(:new_user_free_points, '100', '200', '300')
87
+ end
88
+ ```
89
+
90
+ Example: Conversion tracking (in a controller!)
91
+
92
+ ```ruby
93
+ def buy_new_points
94
+ # some business logic
95
+ ab_finished(:new_user_free_points)
96
+ end
97
+ ```
98
+
99
+ Example: Conversion tracking (in a view)
100
+
101
+ ```erb
102
+ Thanks for signing up, dude! <% ab_finished(:signup_page_redesign) %>
103
+ ```
104
+
105
+ You can find more examples, tutorials and guides on the [wiki](https://github.com/splitrb/split/wiki).
106
+
107
+ ## Statistical Validity
108
+
109
+ Split has two options for you to use to determine which alternative is the best.
110
+
111
+ The first option (default on the dashboard) uses a z test (n>30) for the difference between your control and alternative conversion rates to calculate statistical significance. This test will tell you whether an alternative is better or worse than your control, but it will not distinguish between which alternative is the best in an experiment with multiple alternatives. Split will only tell you if your experiment is 90%, 95%, or 99% significant, and this test only works if you have more than 30 participants and 5 conversions for each branch.
112
+
113
+ As per this [blog post](https://www.evanmiller.org/how-not-to-run-an-ab-test.html) on the pitfalls of A/B testing, it is highly recommended that you determine your requisite sample size for each branch before running the experiment. Otherwise, you'll have an increased rate of false positives (experiments which show a significant effect where really there is none).
114
+
115
+ [Here](https://www.evanmiller.org/ab-testing/sample-size.html) is a sample size calculator for your convenience.
116
+
117
+ The second option uses simulations from a beta distribution to determine the probability that the given alternative is the winner compared to all other alternatives. You can view these probabilities by clicking on the drop-down menu labeled "Confidence." This option should be used when the experiment has more than just 1 control and 1 alternative. It can also be used for a simple, 2-alternative A/B test.
118
+
119
+ Calculating the beta-distribution simulations for a large number of experiments can be slow, so the results are cached. You can specify how often they should be recalculated (the default is once per day).
120
+
121
+ ```ruby
122
+ Split.configure do |config|
123
+ config.winning_alternative_recalculation_interval = 3600 # 1 hour
124
+ end
125
+ ```
126
+
127
+ ## Extras
128
+
129
+ ### Weighted alternatives
130
+
131
+ Perhaps you only want to show an alternative to 10% of your visitors because it is very experimental or not yet fully load tested.
132
+
133
+ To do this you can pass a weight with each alternative in the following ways:
134
+
135
+ ```ruby
136
+ ab_test(:homepage_design, {'Old' => 18}, {'New' => 2})
137
+
138
+ ab_test(:homepage_design, 'Old', {'New' => 1.0/9})
139
+
140
+ ab_test(:homepage_design, {'Old' => 9}, 'New')
141
+ ```
142
+
143
+ This will only show the new alternative to visitors 1 in 10 times, the default weight for an alternative is 1.
144
+
145
+ ### Overriding alternatives
146
+
147
+ For development and testing, you may wish to force your app to always return an alternative.
148
+ You can do this by passing it as a parameter in the url.
149
+
150
+ If you have an experiment called `button_color` with alternatives called `red` and `blue` used on your homepage, a url such as:
151
+
152
+ http://myawesomesite.com?ab_test[button_color]=red
153
+
154
+ will always have red buttons. This won't be stored in your session or count towards to results, unless you set the `store_override` configuration option.
155
+
156
+ In the event you want to disable all tests without having to know the individual experiment names, add a `SPLIT_DISABLE` query parameter.
157
+
158
+ http://myawesomesite.com?SPLIT_DISABLE=true
159
+
160
+ It is not required to send `SPLIT_DISABLE=false` to activate Split.
161
+
162
+
163
+ ### Rspec Helper
164
+ To aid testing with RSpec, write `spec/support/split_helper.rb` and call `use_ab_test(alternatives_by_experiment)` in your specs as instructed below:
165
+
166
+ ```ruby
167
+ # Create a file with these contents at 'spec/support/split_helper.rb'
168
+ # and ensure it is `require`d in your rails_helper.rb or spec_helper.rb
169
+ module SplitHelper
170
+
171
+ # Force a specific experiment alternative to always be returned:
172
+ # use_ab_test(signup_form: "single_page")
173
+ #
174
+ # Force alternatives for multiple experiments:
175
+ # use_ab_test(signup_form: "single_page", pricing: "show_enterprise_prices")
176
+ #
177
+ def use_ab_test(alternatives_by_experiment)
178
+ allow_any_instance_of(Split::Helper).to receive(:ab_test) do |_receiver, experiment|
179
+ alternatives_by_experiment.fetch(experiment) { |key| raise "Unknown experiment '#{key}'" }
180
+ end
181
+ end
182
+ end
183
+
184
+ # Make the `use_ab_test` method available to all specs:
185
+ RSpec.configure do |config|
186
+ config.include SplitHelper
187
+ end
188
+ ```
189
+
190
+ Now you can call `use_ab_test(alternatives_by_experiment)` in your specs, for example:
191
+ ```ruby
192
+ it "registers using experimental signup" do
193
+ use_ab_test experiment_name: "alternative_name"
194
+ post "/signups"
195
+ ...
196
+ end
197
+ ```
198
+
199
+
200
+ ### Starting experiments manually
201
+
202
+ By default new A/B tests will be active right after deployment. In case you would like to start new test a while after
203
+ the deploy, you can do it by setting the `start_manually` configuration option to `true`.
204
+
205
+ After choosing this option tests won't be started right after deploy, but after pressing the `Start` button in Split admin dashboard. If a test is deleted from the Split dashboard, then it can only be started after pressing the `Start` button whenever being re-initialized.
206
+
207
+ ### Reset after completion
208
+
209
+ When a user completes a test their session is reset so that they may start the test again in the future.
210
+
211
+ To stop this behaviour you can pass the following option to the `ab_finished` method:
212
+
213
+ ```ruby
214
+ ab_finished(:experiment_name, reset: false)
215
+ ```
216
+
217
+ The user will then always see the alternative they started with.
218
+
219
+ Any old unfinished experiment key will be deleted from the user's data storage if the experiment had been removed or is over and a winner had been chosen. This allows a user to enroll into any new experiment in cases when the `allow_multiple_experiments` config option is set to `false`.
220
+
221
+ ### Reset experiments manually
222
+
223
+ By default Split automatically resets the experiment whenever it detects the configuration for an experiment has changed (e.g. you call `ab_test` with different alternatives). You can prevent this by setting the option `reset_manually` to `true`.
224
+
225
+ You may want to do this when you want to change something, like the variants' names, the metadata about an experiment, etc. without resetting everything.
226
+
227
+ ### Multiple experiments at once
228
+
229
+ By default Split will avoid users participating in multiple experiments at once. This means you are less likely to skew results by adding in more variation to your tests.
230
+
231
+ To stop this behaviour and allow users to participate in multiple experiments at once set the `allow_multiple_experiments` config option to true like so:
232
+
233
+ ```ruby
234
+ Split.configure do |config|
235
+ config.allow_multiple_experiments = true
236
+ end
237
+ ```
238
+
239
+ This will allow the user to participate in any number of experiments and belong to any alternative in each experiment. This has the possible downside of a variation in one experiment influencing the outcome of another.
240
+
241
+ To address this, setting the `allow_multiple_experiments` config option to 'control' like so:
242
+ ```ruby
243
+ Split.configure do |config|
244
+ config.allow_multiple_experiments = 'control'
245
+ end
246
+ ```
247
+
248
+ For this to work, each and every experiment you define must have an alternative named 'control'. This will allow the user to participate in multiple experiments as long as the user belongs to the alternative 'control' in each experiment. As soon as the user belongs to an alternative named something other than 'control' the user may not participate in any more experiments. Calling ab_test(<other experiments>) will always return the first alternative without adding the user to that experiment.
249
+
250
+ ### Experiment Persistence
251
+
252
+ Split comes with three built-in persistence adapters for storing users and the alternatives they've been given for each experiment.
253
+
254
+ By default Split will store the tests for each user in the session.
255
+
256
+ You can optionally configure Split to use a cookie, Redis, or any custom adapter of your choosing.
257
+
258
+ #### Cookies
259
+
260
+ ```ruby
261
+ Split.configure do |config|
262
+ config.persistence = :cookie
263
+ end
264
+ ```
265
+
266
+ By default, cookies will expire in 1 year. To change that, set the `persistence_cookie_length` in the configuration (unit of time in seconds).
267
+
268
+ ```ruby
269
+ Split.configure do |config|
270
+ config.persistence = :cookie
271
+ config.persistence_cookie_length = 2592000 # 30 days
272
+ end
273
+ ```
274
+
275
+ __Note:__ Using cookies depends on `ActionDispatch::Cookies` or any identical API
276
+
277
+ #### Redis
278
+
279
+ Using Redis will allow ab_users to persist across sessions or machines.
280
+
281
+ ```ruby
282
+ Split.configure do |config|
283
+ config.persistence = Split::Persistence::RedisAdapter.with_config(lookup_by: -> (context) { context.current_user_id })
284
+ # Equivalent
285
+ # config.persistence = Split::Persistence::RedisAdapter.with_config(lookup_by: :current_user_id)
286
+ end
287
+ ```
288
+
289
+ Options:
290
+ * `lookup_by`: method to invoke per request for uniquely identifying ab_users (mandatory configuration)
291
+ * `namespace`: separate namespace to store these persisted values (default "persistence")
292
+ * `expire_seconds`: sets TTL for user key. (if a user is in multiple experiments most recent update will reset TTL for all their assignments)
293
+
294
+ #### Dual Adapter
295
+
296
+ The Dual Adapter allows the use of different persistence adapters for logged-in and logged-out users. A common use case is to use Redis for logged-in users and Cookies for logged-out users.
297
+
298
+ ```ruby
299
+ cookie_adapter = Split::Persistence::CookieAdapter
300
+ redis_adapter = Split::Persistence::RedisAdapter.with_config(
301
+ lookup_by: -> (context) { context.send(:current_user).try(:id) },
302
+ expire_seconds: 2592000)
303
+
304
+ Split.configure do |config|
305
+ config.persistence = Split::Persistence::DualAdapter.with_config(
306
+ logged_in: -> (context) { !context.send(:current_user).try(:id).nil? },
307
+ logged_in_adapter: redis_adapter,
308
+ logged_out_adapter: cookie_adapter)
309
+ config.persistence_cookie_length = 2592000 # 30 days
310
+ end
311
+ ```
312
+
313
+ #### Custom Adapter
314
+
315
+ Your custom adapter needs to implement the same API as existing adapters.
316
+ See `Split::Persistence::CookieAdapter` or `Split::Persistence::SessionAdapter` for a starting point.
317
+
318
+ ```ruby
319
+ Split.configure do |config|
320
+ config.persistence = YourCustomAdapterClass
321
+ end
322
+ ```
323
+
324
+ ### Trial Event Hooks
325
+
326
+ You can define methods that will be called at the same time as experiment
327
+ alternative participation and goal completion.
328
+
329
+ For example:
330
+
331
+ ``` ruby
332
+ Split.configure do |config|
333
+ config.on_trial = :log_trial # run on every trial
334
+ config.on_trial_choose = :log_trial_choose # run on trials with new users only
335
+ config.on_trial_complete = :log_trial_complete
336
+ end
337
+ ```
338
+
339
+ Set these attributes to a method name available in the same context as the
340
+ `ab_test` method. These methods should accept one argument, a `Trial` instance.
341
+
342
+ ``` ruby
343
+ def log_trial(trial)
344
+ logger.info "experiment=%s alternative=%s user=%s" %
345
+ [ trial.experiment.name, trial.alternative, current_user.id ]
346
+ end
347
+
348
+ def log_trial_choose(trial)
349
+ logger.info "[new user] experiment=%s alternative=%s user=%s" %
350
+ [ trial.experiment.name, trial.alternative, current_user.id ]
351
+ end
352
+
353
+ def log_trial_complete(trial)
354
+ logger.info "experiment=%s alternative=%s user=%s complete=true" %
355
+ [ trial.experiment.name, trial.alternative, current_user.id ]
356
+ end
357
+ ```
358
+
359
+ #### Views
360
+
361
+ If you are running `ab_test` from a view, you must define your event
362
+ hook callback as a
363
+ [helper_method](https://apidock.com/rails/AbstractController/Helpers/ClassMethods/helper_method)
364
+ in the controller:
365
+
366
+ ``` ruby
367
+ helper_method :log_trial_choose
368
+
369
+ def log_trial_choose(trial)
370
+ logger.info "experiment=%s alternative=%s user=%s" %
371
+ [ trial.experiment.name, trial.alternative, current_user.id ]
372
+ end
373
+ ```
374
+
375
+ ### Experiment Hooks
376
+
377
+ You can assign a proc that will be called when an experiment is reset or deleted. You can use these hooks to call methods within your application to keep data related to experiments in sync with Split.
378
+
379
+ For example:
380
+
381
+ ``` ruby
382
+ Split.configure do |config|
383
+ # after experiment reset or deleted
384
+ config.on_experiment_reset = -> (example) { # Do something on reset }
385
+ config.on_experiment_delete = -> (experiment) { # Do something else on delete }
386
+ # before experiment reset or deleted
387
+ config.on_before_experiment_reset = -> (example) { # Do something on reset }
388
+ config.on_before_experiment_delete = -> (experiment) { # Do something else on delete }
389
+ end
390
+ ```
391
+
392
+ ## Web Interface
393
+
394
+ Split comes with a Sinatra-based front end to get an overview of how your experiments are doing.
395
+
396
+ If you are running Rails 2: You can mount this inside your app using Rack::URLMap in your `config.ru`
397
+
398
+ ```ruby
399
+ require 'split/dashboard'
400
+
401
+ run Rack::URLMap.new \
402
+ "/" => Your::App.new,
403
+ "/split" => Split::Dashboard.new
404
+ ```
405
+
406
+ However, if you are using Rails 3 or higher: You can mount this inside your app routes by first adding this to the Gemfile:
407
+
408
+ ```ruby
409
+ gem 'split', require: 'split/dashboard'
410
+ ```
411
+
412
+ Then adding this to config/routes.rb
413
+
414
+ ```ruby
415
+ mount Split::Dashboard, at: 'split'
416
+ ```
417
+
418
+ You may want to password protect that page, you can do so with `Rack::Auth::Basic` (in your split initializer file)
419
+
420
+ ```ruby
421
+ # Rails apps or apps that already depend on activesupport
422
+ Split::Dashboard.use Rack::Auth::Basic do |username, password|
423
+ # Protect against timing attacks:
424
+ # - Use & (do not use &&) so that it doesn't short circuit.
425
+ # - Use digests to stop length information leaking
426
+ ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(username), ::Digest::SHA256.hexdigest(ENV["SPLIT_USERNAME"])) &
427
+ ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(password), ::Digest::SHA256.hexdigest(ENV["SPLIT_PASSWORD"]))
428
+ end
429
+
430
+ # Apps without activesupport
431
+ Split::Dashboard.use Rack::Auth::Basic do |username, password|
432
+ # Protect against timing attacks:
433
+ # - Use & (do not use &&) so that it doesn't short circuit.
434
+ # - Use digests to stop length information leaking
435
+ Rack::Utils.secure_compare(::Digest::SHA256.hexdigest(username), ::Digest::SHA256.hexdigest(ENV["SPLIT_USERNAME"])) &
436
+ Rack::Utils.secure_compare(::Digest::SHA256.hexdigest(password), ::Digest::SHA256.hexdigest(ENV["SPLIT_PASSWORD"]))
437
+ end
438
+ ```
439
+
440
+ You can even use Devise or any other Warden-based authentication method to authorize users. Just replace `mount Split::Dashboard, :at => 'split'` in `config/routes.rb` with the following:
441
+ ```ruby
442
+ match "/split" => Split::Dashboard, anchor: false, via: [:get, :post, :delete], constraints: -> (request) do
443
+ request.env['warden'].authenticated? # are we authenticated?
444
+ request.env['warden'].authenticate! # authenticate if not already
445
+ # or even check any other condition such as request.env['warden'].user.is_admin?
446
+ end
447
+ ```
448
+
449
+ More information on this [here](https://steve.dynedge.co.uk/2011/12/09/controlling-access-to-routes-and-rack-apps-in-rails-3-with-devise-and-warden/)
450
+
451
+ ### Screenshot
452
+
453
+ ![split_screenshot](https://raw.githubusercontent.com/caser/caser.github.io/master/dashboard.png)
454
+
455
+ ## Configuration
456
+
457
+ You can override the default configuration options of Split like so:
458
+
459
+ ```ruby
460
+ Split.configure do |config|
461
+ config.db_failover = true # handle Redis errors gracefully
462
+ config.db_failover_on_db_error = -> (error) { Rails.logger.error(error.message) }
463
+ config.allow_multiple_experiments = true
464
+ config.enabled = true
465
+ config.persistence = Split::Persistence::SessionAdapter
466
+ #config.start_manually = false ## new test will have to be started manually from the admin panel. default false
467
+ #config.reset_manually = false ## if true, it never resets the experiment data, even if the configuration changes
468
+ config.include_rails_helper = true
469
+ config.redis = "redis://custom.redis.url:6380"
470
+ end
471
+ ```
472
+
473
+ Split looks for the Redis host in the environment variable `REDIS_URL` then
474
+ defaults to `redis://localhost:6379` if not specified by configure block.
475
+
476
+ On platforms like Heroku, Split will use the value of `REDIS_PROVIDER` to
477
+ determine which env variable key to use when retrieving the host config. This
478
+ defaults to `REDIS_URL`.
479
+
480
+ ### Filtering
481
+
482
+ In most scenarios you don't want to have AB-Testing enabled for web spiders, robots or special groups of users.
483
+ Split provides functionality to filter this based on a predefined, extensible list of bots, IP-lists or custom exclude logic.
484
+
485
+ ```ruby
486
+ Split.configure do |config|
487
+ # bot config
488
+ config.robot_regex = /my_custom_robot_regex/ # or
489
+ config.bots['newbot'] = "Description for bot with 'newbot' user agent, which will be added to config.robot_regex for exclusion"
490
+
491
+ # IP config
492
+ config.ignore_ip_addresses << '81.19.48.130' # or regex: /81\.19\.48\.[0-9]+/
493
+
494
+ # or provide your own filter functionality, the default is proc{ |request| is_robot? || is_ignored_ip_address? || is_preview? }
495
+ config.ignore_filter = -> (request) { CustomExcludeLogic.excludes?(request) }
496
+ end
497
+ ```
498
+
499
+ ### Experiment configuration
500
+
501
+ Instead of providing the experiment options inline, you can store them
502
+ in a hash. This hash can control your experiment's alternatives, weights,
503
+ algorithm and if the experiment resets once finished:
504
+
505
+ ```ruby
506
+ Split.configure do |config|
507
+ config.experiments = {
508
+ my_first_experiment: {
509
+ alternatives: ["a", "b"],
510
+ resettable: false
511
+ },
512
+ :my_second_experiment => {
513
+ algorithm: 'Split::Algorithms::Whiplash',
514
+ alternatives: [
515
+ { name: "a", percent: 67 },
516
+ { name: "b", percent: 33 }
517
+ ]
518
+ }
519
+ }
520
+ end
521
+ ```
522
+
523
+ You can also store your experiments in a YAML file:
524
+
525
+ ```ruby
526
+ Split.configure do |config|
527
+ config.experiments = YAML.load_file "config/experiments.yml"
528
+ end
529
+ ```
530
+
531
+ You can then define the YAML file like:
532
+
533
+ ```yaml
534
+ my_first_experiment:
535
+ alternatives:
536
+ - a
537
+ - b
538
+ my_second_experiment:
539
+ alternatives:
540
+ - name: a
541
+ percent: 67
542
+ - name: b
543
+ percent: 33
544
+ resettable: false
545
+ ```
546
+
547
+ This simplifies the calls from your code:
548
+
549
+ ```ruby
550
+ ab_test(:my_first_experiment)
551
+ ```
552
+
553
+ and:
554
+
555
+ ```ruby
556
+ ab_finished(:my_first_experiment)
557
+ ```
558
+
559
+ You can also add meta data for each experiment, which is very useful when you need more than an alternative name to change behaviour:
560
+
561
+ ```ruby
562
+ Split.configure do |config|
563
+ config.experiments = {
564
+ my_first_experiment: {
565
+ alternatives: ["a", "b"],
566
+ metadata: {
567
+ "a" => {"text" => "Have a fantastic day"},
568
+ "b" => {"text" => "Don't get hit by a bus"}
569
+ }
570
+ }
571
+ }
572
+ end
573
+ ```
574
+
575
+ ```yaml
576
+ my_first_experiment:
577
+ alternatives:
578
+ - a
579
+ - b
580
+ metadata:
581
+ a:
582
+ text: "Have a fantastic day"
583
+ b:
584
+ text: "Don't get hit by a bus"
585
+ ```
586
+
587
+ This allows for some advanced experiment configuration using methods like:
588
+
589
+ ```ruby
590
+ trial.alternative.name # => "a"
591
+
592
+ trial.metadata['text'] # => "Have a fantastic day"
593
+ ```
594
+
595
+ or in views:
596
+
597
+ ```erb
598
+ <% ab_test("my_first_experiment") do |alternative, meta| %>
599
+ <%= alternative %>
600
+ <small><%= meta['text'] %></small>
601
+ <% end %>
602
+ ```
603
+
604
+ The keys used in meta data should be Strings
605
+
606
+ #### Metrics
607
+
608
+ You might wish to track generic metrics, such as conversions, and use
609
+ those to complete multiple different experiments without adding more to
610
+ your code. You can use the configuration hash to do this, thanks to
611
+ the `:metric` option.
612
+
613
+ ```ruby
614
+ Split.configure do |config|
615
+ config.experiments = {
616
+ my_first_experiment: {
617
+ alternatives: ["a", "b"],
618
+ metric: :my_metric
619
+ }
620
+ }
621
+ end
622
+ ```
623
+
624
+ Your code may then track a completion using the metric instead of
625
+ the experiment name:
626
+
627
+ ```ruby
628
+ ab_finished(:my_metric)
629
+ ```
630
+
631
+ You can also create a new metric by instantiating and saving a new Metric object.
632
+
633
+ ```ruby
634
+ Split::Metric.new(:my_metric)
635
+ Split::Metric.save
636
+ ```
637
+
638
+ #### Goals
639
+
640
+ You might wish to allow an experiment to have multiple, distinguishable goals.
641
+ The API to define goals for an experiment is this:
642
+
643
+ ```ruby
644
+ ab_test({link_color: ["purchase", "refund"]}, "red", "blue")
645
+ ```
646
+
647
+ or you can you can define them in a configuration file:
648
+
649
+ ```ruby
650
+ Split.configure do |config|
651
+ config.experiments = {
652
+ link_color: {
653
+ alternatives: ["red", "blue"],
654
+ goals: ["purchase", "refund"]
655
+ }
656
+ }
657
+ end
658
+ ```
659
+
660
+ To complete a goal conversion, you do it like:
661
+
662
+ ```ruby
663
+ ab_finished(link_color: "purchase")
664
+ ```
665
+
666
+ Note that if you pass additional options, that should be a separate hash:
667
+
668
+ ```ruby
669
+ ab_finished({ link_color: "purchase" }, reset: false)
670
+ ```
671
+
672
+ **NOTE:** This does not mean that a single experiment can complete more than one goal.
673
+
674
+ Once you finish one of the goals, the test is considered to be completed, and finishing the other goal will no longer register. (Assuming the test runs with `reset: false`.)
675
+
676
+ **Good Example**: Test if listing Plan A first result in more conversions to Plan A (goal: "plana_conversion") or Plan B (goal: "planb_conversion").
677
+
678
+ **Bad Example**: Test if button color increases conversion rate through multiple steps of a funnel. THIS WILL NOT WORK.
679
+
680
+ **Bad Example**: Test both how button color affects signup *and* how it affects login, at the same time. THIS WILL NOT WORK.
681
+
682
+ #### Combined Experiments
683
+ If you want to test how button color affects signup *and* how it affects login at the same time, use combined experiments.
684
+ Configure like so:
685
+ ```ruby
686
+ Split.configuration.experiments = {
687
+ :button_color_experiment => {
688
+ :alternatives => ["blue", "green"],
689
+ :combined_experiments => ["button_color_on_signup", "button_color_on_login"]
690
+ }
691
+ }
692
+ ```
693
+
694
+ Starting the combined test starts all combined experiments
695
+ ```ruby
696
+ ab_combined_test(:button_color_experiment)
697
+ ```
698
+ Finish each combined test as normal
699
+
700
+ ```ruby
701
+ ab_finished(:button_color_on_login)
702
+ ab_finished(:button_color_on_signup)
703
+ ```
704
+
705
+ **Additional Configuration**:
706
+ * Be sure to enable `allow_multiple_experiments`
707
+ * In Sinatra include the CombinedExperimentsHelper
708
+ ```
709
+ helpers Split::CombinedExperimentsHelper
710
+ ```
711
+ ### DB failover solution
712
+
713
+ Due to the fact that Redis has no automatic failover mechanism, it's
714
+ possible to switch on the `db_failover` config option, so that `ab_test`
715
+ and `ab_finished` will not crash in case of a db failure. `ab_test` always
716
+ delivers alternative A (the first one) in that case.
717
+
718
+ It's also possible to set a `db_failover_on_db_error` callback (proc)
719
+ for example to log these errors via Rails.logger.
720
+
721
+ ### Redis
722
+
723
+ You may want to change the Redis host and port Split connects to, or
724
+ set various other options at startup.
725
+
726
+ Split has a `redis` setter which can be given a string or a Redis
727
+ object. This means if you're already using Redis in your app, Split
728
+ can re-use the existing connection.
729
+
730
+ String: `Split.redis = 'redis://localhost:6379'`
731
+
732
+ Redis: `Split.redis = $redis`
733
+
734
+ For our rails app we have a `config/initializers/split.rb` file where
735
+ we load `config/split.yml` by hand and set the Redis information
736
+ appropriately.
737
+
738
+ Here's our `config/split.yml`:
739
+
740
+ ```yml
741
+ development: redis://localhost:6379
742
+ test: redis://localhost:6379
743
+ staging: redis://redis1.example.com:6379
744
+ fi: redis://localhost:6379
745
+ production: redis://redis1.example.com:6379
746
+ ```
747
+
748
+ And our initializer:
749
+
750
+ ```ruby
751
+ split_config = YAML.load_file(Rails.root.join('config', 'split.yml'))
752
+ Split.redis = split_config[Rails.env]
753
+ ```
754
+
755
+ ## Namespaces
756
+
757
+ If you're running multiple, separate instances of Split you may want
758
+ to namespace the keyspaces so they do not overlap. This is not unlike
759
+ the approach taken by many memcached clients.
760
+
761
+ This feature can be provided by the [redis-namespace](https://github.com/defunkt/redis-namespace)
762
+ library. To configure Split to use `Redis::Namespace`, do the following:
763
+
764
+ 1. Add `redis-namespace` to your Gemfile:
765
+
766
+ ```ruby
767
+ gem 'redis-namespace'
768
+ ```
769
+
770
+ 2. Configure `Split.redis` to use a `Redis::Namespace` instance (possible in an
771
+ intializer):
772
+
773
+ ```ruby
774
+ redis = Redis.new(url: ENV['REDIS_URL']) # or whatever config you want
775
+ Split.redis = Redis::Namespace.new(:your_namespace, redis: redis)
776
+ ```
777
+
778
+ ## Outside of a Web Session
779
+
780
+ Split provides the Helper module to facilitate running experiments inside web sessions.
781
+
782
+ Alternatively, you can access the underlying Metric, Trial, Experiment and Alternative objects to
783
+ conduct experiments that are not tied to a web session.
784
+
785
+ ```ruby
786
+ # create a new experiment
787
+ experiment = Split::ExperimentCatalog.find_or_create('color', 'red', 'blue')
788
+ # create a new trial
789
+ trial = Split::Trial.new(:experiment => experiment)
790
+ # run trial
791
+ trial.choose!
792
+ # get the result, returns either red or blue
793
+ trial.alternative.name
794
+
795
+ # if the goal has been achieved, increment the successful completions for this alternative.
796
+ if goal_achieved?
797
+ trial.complete!
798
+ end
799
+
800
+ ```
801
+
802
+ ## Algorithms
803
+
804
+ By default, Split ships with `Split::Algorithms::WeightedSample` that randomly selects from possible alternatives for a traditional a/b test.
805
+ It is possible to specify static weights to favor certain alternatives.
806
+
807
+ `Split::Algorithms::Whiplash` is an implementation of a [multi-armed bandit algorithm](http://stevehanov.ca/blog/index.php?id=132).
808
+ This algorithm will automatically weight the alternatives based on their relative performance,
809
+ choosing the better-performing ones more often as trials are completed.
810
+
811
+ `Split::Algorithms::BlockRandomization` is an algorithm that ensures equal
812
+ participation across all alternatives. This algorithm will choose the alternative
813
+ with the fewest participants. In the event of multiple minimum participant alternatives
814
+ (i.e. starting a new "Block") the algorithm will choose a random alternative from
815
+ those minimum participant alternatives.
816
+
817
+ Users may also write their own algorithms. The default algorithm may be specified globally in the configuration file, or on a per experiment basis using the experiments hash of the configuration file.
818
+
819
+ To change the algorithm globally for all experiments, use the following in your initializer:
820
+
821
+ ```ruby
822
+ Split.configure do |config|
823
+ config.algorithm = Split::Algorithms::Whiplash
824
+ end
825
+ ```
826
+
827
+ ## Extensions
828
+
829
+ - [Split::Export](https://github.com/splitrb/split-export) - Easily export A/B test data out of Split.
830
+ - [Split::Analytics](https://github.com/splitrb/split-analytics) - Push test data to Google Analytics.
831
+ - [Split::Mongoid](https://github.com/MongoHQ/split-mongoid) - Store experiment data in mongoid (still uses redis).
832
+ - [Split::Cacheable](https://github.com/harrystech/split_cacheable) - Automatically create cache buckets per test.
833
+ - [Split::Counters](https://github.com/bernardkroes/split-counters) - Add counters per experiment and alternative.
834
+ - [Split::Cli](https://github.com/craigmcnamara/split-cli) - A CLI to trigger Split A/B tests.
835
+
836
+ ## Screencast
837
+
838
+ Ryan bates has produced an excellent 10 minute screencast about split on the Railscasts site: [A/B Testing with Split](http://railscasts.com/episodes/331-a-b-testing-with-split)
839
+
840
+ ## Blogposts
841
+
842
+ * [Recipe: A/B testing with KISSMetrics and the split gem](https://robots.thoughtbot.com/post/9595887299/recipe-a-b-testing-with-kissmetrics-and-the-split-gem)
843
+ * [Rails A/B testing with Split on Heroku](http://blog.nathanhumbert.com/2012/02/rails-ab-testing-with-split-on-heroku.html)
844
+
845
+ ## Backers
846
+
847
+ Support us with a monthly donation and help us continue our activities. [[Become a backer](https://opencollective.com/split#backer)]
848
+
849
+ <a href="https://opencollective.com/split/backer/0/website" target="_blank"><img src="https://opencollective.com/split/backer/0/avatar.svg"></a>
850
+ <a href="https://opencollective.com/split/backer/1/website" target="_blank"><img src="https://opencollective.com/split/backer/1/avatar.svg"></a>
851
+ <a href="https://opencollective.com/split/backer/2/website" target="_blank"><img src="https://opencollective.com/split/backer/2/avatar.svg"></a>
852
+ <a href="https://opencollective.com/split/backer/3/website" target="_blank"><img src="https://opencollective.com/split/backer/3/avatar.svg"></a>
853
+ <a href="https://opencollective.com/split/backer/4/website" target="_blank"><img src="https://opencollective.com/split/backer/4/avatar.svg"></a>
854
+ <a href="https://opencollective.com/split/backer/5/website" target="_blank"><img src="https://opencollective.com/split/backer/5/avatar.svg"></a>
855
+ <a href="https://opencollective.com/split/backer/6/website" target="_blank"><img src="https://opencollective.com/split/backer/6/avatar.svg"></a>
856
+ <a href="https://opencollective.com/split/backer/7/website" target="_blank"><img src="https://opencollective.com/split/backer/7/avatar.svg"></a>
857
+ <a href="https://opencollective.com/split/backer/8/website" target="_blank"><img src="https://opencollective.com/split/backer/8/avatar.svg"></a>
858
+ <a href="https://opencollective.com/split/backer/9/website" target="_blank"><img src="https://opencollective.com/split/backer/9/avatar.svg"></a>
859
+ <a href="https://opencollective.com/split/backer/10/website" target="_blank"><img src="https://opencollective.com/split/backer/10/avatar.svg"></a>
860
+ <a href="https://opencollective.com/split/backer/11/website" target="_blank"><img src="https://opencollective.com/split/backer/11/avatar.svg"></a>
861
+ <a href="https://opencollective.com/split/backer/12/website" target="_blank"><img src="https://opencollective.com/split/backer/12/avatar.svg"></a>
862
+ <a href="https://opencollective.com/split/backer/13/website" target="_blank"><img src="https://opencollective.com/split/backer/13/avatar.svg"></a>
863
+ <a href="https://opencollective.com/split/backer/14/website" target="_blank"><img src="https://opencollective.com/split/backer/14/avatar.svg"></a>
864
+ <a href="https://opencollective.com/split/backer/15/website" target="_blank"><img src="https://opencollective.com/split/backer/15/avatar.svg"></a>
865
+ <a href="https://opencollective.com/split/backer/16/website" target="_blank"><img src="https://opencollective.com/split/backer/16/avatar.svg"></a>
866
+ <a href="https://opencollective.com/split/backer/17/website" target="_blank"><img src="https://opencollective.com/split/backer/17/avatar.svg"></a>
867
+ <a href="https://opencollective.com/split/backer/18/website" target="_blank"><img src="https://opencollective.com/split/backer/18/avatar.svg"></a>
868
+ <a href="https://opencollective.com/split/backer/19/website" target="_blank"><img src="https://opencollective.com/split/backer/19/avatar.svg"></a>
869
+ <a href="https://opencollective.com/split/backer/20/website" target="_blank"><img src="https://opencollective.com/split/backer/20/avatar.svg"></a>
870
+ <a href="https://opencollective.com/split/backer/21/website" target="_blank"><img src="https://opencollective.com/split/backer/21/avatar.svg"></a>
871
+ <a href="https://opencollective.com/split/backer/22/website" target="_blank"><img src="https://opencollective.com/split/backer/22/avatar.svg"></a>
872
+ <a href="https://opencollective.com/split/backer/23/website" target="_blank"><img src="https://opencollective.com/split/backer/23/avatar.svg"></a>
873
+ <a href="https://opencollective.com/split/backer/24/website" target="_blank"><img src="https://opencollective.com/split/backer/24/avatar.svg"></a>
874
+ <a href="https://opencollective.com/split/backer/25/website" target="_blank"><img src="https://opencollective.com/split/backer/25/avatar.svg"></a>
875
+ <a href="https://opencollective.com/split/backer/26/website" target="_blank"><img src="https://opencollective.com/split/backer/26/avatar.svg"></a>
876
+ <a href="https://opencollective.com/split/backer/27/website" target="_blank"><img src="https://opencollective.com/split/backer/27/avatar.svg"></a>
877
+ <a href="https://opencollective.com/split/backer/28/website" target="_blank"><img src="https://opencollective.com/split/backer/28/avatar.svg"></a>
878
+ <a href="https://opencollective.com/split/backer/29/website" target="_blank"><img src="https://opencollective.com/split/backer/29/avatar.svg"></a>
879
+
880
+
881
+ ## Sponsors
882
+
883
+ Become a sponsor and get your logo on our README on Github with a link to your site. [[Become a sponsor](https://opencollective.com/split#sponsor)]
884
+
885
+ <a href="https://opencollective.com/split/sponsor/0/website" target="_blank"><img src="https://opencollective.com/split/sponsor/0/avatar.svg"></a>
886
+ <a href="https://opencollective.com/split/sponsor/1/website" target="_blank"><img src="https://opencollective.com/split/sponsor/1/avatar.svg"></a>
887
+ <a href="https://opencollective.com/split/sponsor/2/website" target="_blank"><img src="https://opencollective.com/split/sponsor/2/avatar.svg"></a>
888
+ <a href="https://opencollective.com/split/sponsor/3/website" target="_blank"><img src="https://opencollective.com/split/sponsor/3/avatar.svg"></a>
889
+ <a href="https://opencollective.com/split/sponsor/4/website" target="_blank"><img src="https://opencollective.com/split/sponsor/4/avatar.svg"></a>
890
+ <a href="https://opencollective.com/split/sponsor/5/website" target="_blank"><img src="https://opencollective.com/split/sponsor/5/avatar.svg"></a>
891
+ <a href="https://opencollective.com/split/sponsor/6/website" target="_blank"><img src="https://opencollective.com/split/sponsor/6/avatar.svg"></a>
892
+ <a href="https://opencollective.com/split/sponsor/7/website" target="_blank"><img src="https://opencollective.com/split/sponsor/7/avatar.svg"></a>
893
+ <a href="https://opencollective.com/split/sponsor/8/website" target="_blank"><img src="https://opencollective.com/split/sponsor/8/avatar.svg"></a>
894
+ <a href="https://opencollective.com/split/sponsor/9/website" target="_blank"><img src="https://opencollective.com/split/sponsor/9/avatar.svg"></a>
895
+ <a href="https://opencollective.com/split/sponsor/10/website" target="_blank"><img src="https://opencollective.com/split/sponsor/10/avatar.svg"></a>
896
+ <a href="https://opencollective.com/split/sponsor/11/website" target="_blank"><img src="https://opencollective.com/split/sponsor/11/avatar.svg"></a>
897
+ <a href="https://opencollective.com/split/sponsor/12/website" target="_blank"><img src="https://opencollective.com/split/sponsor/12/avatar.svg"></a>
898
+ <a href="https://opencollective.com/split/sponsor/13/website" target="_blank"><img src="https://opencollective.com/split/sponsor/13/avatar.svg"></a>
899
+ <a href="https://opencollective.com/split/sponsor/14/website" target="_blank"><img src="https://opencollective.com/split/sponsor/14/avatar.svg"></a>
900
+ <a href="https://opencollective.com/split/sponsor/15/website" target="_blank"><img src="https://opencollective.com/split/sponsor/15/avatar.svg"></a>
901
+ <a href="https://opencollective.com/split/sponsor/16/website" target="_blank"><img src="https://opencollective.com/split/sponsor/16/avatar.svg"></a>
902
+ <a href="https://opencollective.com/split/sponsor/17/website" target="_blank"><img src="https://opencollective.com/split/sponsor/17/avatar.svg"></a>
903
+ <a href="https://opencollective.com/split/sponsor/18/website" target="_blank"><img src="https://opencollective.com/split/sponsor/18/avatar.svg"></a>
904
+ <a href="https://opencollective.com/split/sponsor/19/website" target="_blank"><img src="https://opencollective.com/split/sponsor/19/avatar.svg"></a>
905
+ <a href="https://opencollective.com/split/sponsor/20/website" target="_blank"><img src="https://opencollective.com/split/sponsor/20/avatar.svg"></a>
906
+ <a href="https://opencollective.com/split/sponsor/21/website" target="_blank"><img src="https://opencollective.com/split/sponsor/21/avatar.svg"></a>
907
+ <a href="https://opencollective.com/split/sponsor/22/website" target="_blank"><img src="https://opencollective.com/split/sponsor/22/avatar.svg"></a>
908
+ <a href="https://opencollective.com/split/sponsor/23/website" target="_blank"><img src="https://opencollective.com/split/sponsor/23/avatar.svg"></a>
909
+ <a href="https://opencollective.com/split/sponsor/24/website" target="_blank"><img src="https://opencollective.com/split/sponsor/24/avatar.svg"></a>
910
+ <a href="https://opencollective.com/split/sponsor/25/website" target="_blank"><img src="https://opencollective.com/split/sponsor/25/avatar.svg"></a>
911
+ <a href="https://opencollective.com/split/sponsor/26/website" target="_blank"><img src="https://opencollective.com/split/sponsor/26/avatar.svg"></a>
912
+ <a href="https://opencollective.com/split/sponsor/27/website" target="_blank"><img src="https://opencollective.com/split/sponsor/27/avatar.svg"></a>
913
+ <a href="https://opencollective.com/split/sponsor/28/website" target="_blank"><img src="https://opencollective.com/split/sponsor/28/avatar.svg"></a>
914
+ <a href="https://opencollective.com/split/sponsor/29/website" target="_blank"><img src="https://opencollective.com/split/sponsor/29/avatar.svg"></a>
915
+
916
+ ## Contribute
917
+
918
+ Please do! Over 70 different people have contributed to the project, you can see them all here: https://github.com/splitrb/split/graphs/contributors.
919
+
920
+ ### Development
921
+
922
+ The source code is hosted at [GitHub](https://github.com/splitrb/split).
923
+
924
+ Report issues and feature requests on [GitHub Issues](https://github.com/splitrb/split/issues).
925
+
926
+ You can find a discussion form on [Google Groups](https://groups.google.com/d/forum/split-ruby).
927
+
928
+ ### Tests
929
+
930
+ Run the tests like this:
931
+
932
+ # Start a Redis server in another tab.
933
+ redis-server
934
+
935
+ bundle
936
+ rake spec
937
+
938
+ ### A Note on Patches and Pull Requests
939
+
940
+ * Fork the project.
941
+ * Make your feature addition or bug fix.
942
+ * Add tests for it. This is important so I don't break it in a
943
+ future version unintentionally.
944
+ * Add documentation if necessary.
945
+ * Commit. Do not mess with the rakefile, version, or history.
946
+ (If you want to have your own version, that is fine. But bump the version in a commit by itself, which I can ignore when I pull.)
947
+ * Send a pull request. Bonus points for topic branches.
948
+
949
+ ### Code of Conduct
950
+
951
+ Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.
952
+
953
+ ## Copyright
954
+
955
+ [MIT License](LICENSE) © 2019 [Andrew Nesbitt](https://github.com/andrew).