field_test 0.1.2 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of field_test might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -0
- data/README.md +33 -33
- data/app/controllers/field_test/{home_controller.rb → base_controller.rb} +1 -5
- data/app/controllers/field_test/experiments_controller.rb +18 -0
- data/app/controllers/field_test/memberships_controller.rb +15 -0
- data/app/controllers/field_test/participants_controller.rb +7 -0
- data/app/views/field_test/{home → experiments}/_experiments.html.erb +16 -3
- data/app/views/field_test/{home → experiments}/index.html.erb +0 -0
- data/app/views/field_test/experiments/show.html.erb +52 -0
- data/app/views/field_test/participants/show.html.erb +47 -0
- data/app/views/layouts/field_test/application.html.erb +29 -1
- data/config/routes.rb +4 -1
- data/field_test.gemspec +1 -0
- data/lib/field_test.rb +7 -1
- data/lib/field_test/calculations.rb +59 -0
- data/lib/field_test/experiment.rb +35 -34
- data/lib/field_test/helpers.rb +10 -1
- data/lib/field_test/version.rb +1 -1
- data/lib/generators/field_test/templates/memberships.rb +1 -0
- metadata +25 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 715ea396faed10c5b142f003eb76405d8ba34f01
|
4
|
+
data.tar.gz: 43a830f9996b974a917734f39641d8783b9c33c9
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: fee3d99177f6e2044bc38bba2a557e4832b9a402b4743400a8c6a47cff272885417100caf7522ef48ef62f2641a71790e1107cc4885d079d5624673a26178e18
|
7
|
+
data.tar.gz: 07a3450cd3e1ad4da4a909841ea7532c7683b65a366907345f186bd261bac53721d1a4b603f97c0f8e541773dc39d2e7398a41655c0811effeece190b29a9741
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
@@ -3,8 +3,8 @@
|
|
3
3
|
:maple_leaf: A/B testing for Rails
|
4
4
|
|
5
5
|
- Designed for web and email
|
6
|
+
- Comes with a [nice dashboard](https://fieldtest.dokkuapp.com/)
|
6
7
|
- Seamlessly handles the transition from anonymous visitor to logged in user
|
7
|
-
- Results are stored in your database
|
8
8
|
|
9
9
|
Uses [Bayesian methods](http://www.evanmiller.org/bayesian-ab-testing.html) to evaluate results so you don’t need to choose a sample size ahead of time.
|
10
10
|
|
@@ -30,7 +30,7 @@ mount FieldTest::Engine, at: "field_test"
|
|
30
30
|
|
31
31
|
Be sure to [secure the dashboard](#security) in production.
|
32
32
|
|
33
|
-
![Screenshot](https://ankane.github.io/field_test/
|
33
|
+
![Screenshot](https://ankane.github.io/field_test/screenshot6.png)
|
34
34
|
|
35
35
|
## Getting Started
|
36
36
|
|
@@ -57,13 +57,6 @@ When someone converts, record it with:
|
|
57
57
|
field_test_converted(:button_color)
|
58
58
|
```
|
59
59
|
|
60
|
-
Get the results with:
|
61
|
-
|
62
|
-
```ruby
|
63
|
-
experiment = FieldTest::Experiment.find(:button_color)
|
64
|
-
experiment.results
|
65
|
-
```
|
66
|
-
|
67
60
|
When an experiment is over, specify a winner:
|
68
61
|
|
69
62
|
```yml
|
@@ -86,19 +79,7 @@ Assign a specific variant to a user with:
|
|
86
79
|
|
87
80
|
```ruby
|
88
81
|
experiment = FieldTest::Experiment.find(:button_color)
|
89
|
-
experiment.variant(
|
90
|
-
```
|
91
|
-
|
92
|
-
Specify a participant with:
|
93
|
-
|
94
|
-
```ruby
|
95
|
-
field_test(:button_color, participant: "test@example.org")
|
96
|
-
```
|
97
|
-
|
98
|
-
You can pass an object as well.
|
99
|
-
|
100
|
-
```ruby
|
101
|
-
field_test(:button_color, participant: user)
|
82
|
+
experiment.variant(participant, variant: "red")
|
102
83
|
```
|
103
84
|
|
104
85
|
## Config
|
@@ -110,13 +91,24 @@ exclude:
|
|
110
91
|
bots: false
|
111
92
|
```
|
112
93
|
|
113
|
-
Keep track of when experiments started and ended.
|
94
|
+
Keep track of when experiments started and ended. Use any format `Time.parse` accepts.
|
95
|
+
|
96
|
+
```yml
|
97
|
+
experiments:
|
98
|
+
button_color:
|
99
|
+
started_at: Dec 1, 2016 8 am PST
|
100
|
+
ended_at: Dec 8, 2016 2 pm PST
|
101
|
+
```
|
102
|
+
|
103
|
+
Add a friendlier name and description with:
|
114
104
|
|
115
105
|
```yml
|
116
106
|
experiments:
|
117
107
|
button_color:
|
118
|
-
|
119
|
-
|
108
|
+
name: Buttons!
|
109
|
+
description: >
|
110
|
+
Different button colors
|
111
|
+
for the landing page.
|
120
112
|
```
|
121
113
|
|
122
114
|
By default, variants are given the same probability of being selected. Change this with:
|
@@ -132,6 +124,14 @@ experiments:
|
|
132
124
|
- 10
|
133
125
|
```
|
134
126
|
|
127
|
+
If the dashboard gets slow, you can make it faster with:
|
128
|
+
|
129
|
+
```yml
|
130
|
+
cache: true
|
131
|
+
```
|
132
|
+
|
133
|
+
This will use the Rails cache to speed up winning probability calculations.
|
134
|
+
|
135
135
|
## Funnels
|
136
136
|
|
137
137
|
For advanced funnels, we recommend an analytics platform like [Ahoy](https://github.com/ankane/ahoy) or [Mixpanel](https://mixpanel.com/).
|
@@ -146,6 +146,14 @@ to get all experiments and variants for a participant, and pass them as properti
|
|
146
146
|
|
147
147
|
## Security
|
148
148
|
|
149
|
+
#### Devise
|
150
|
+
|
151
|
+
```ruby
|
152
|
+
authenticate :user, -> (user) { user.admin? } do
|
153
|
+
mount FieldTest::Engine, at: "field_test"
|
154
|
+
end
|
155
|
+
```
|
156
|
+
|
149
157
|
#### Basic Authentication
|
150
158
|
|
151
159
|
Set the following variables in your environment or an initializer.
|
@@ -155,14 +163,6 @@ ENV["FIELD_TEST_USERNAME"] = "moonrise"
|
|
155
163
|
ENV["FIELD_TEST_PASSWORD"] = "kingdom"
|
156
164
|
```
|
157
165
|
|
158
|
-
#### Devise
|
159
|
-
|
160
|
-
```ruby
|
161
|
-
authenticate :user, -> (user) { user.admin? } do
|
162
|
-
mount FieldTest::Engine, at: "field_test"
|
163
|
-
end
|
164
|
-
```
|
165
|
-
|
166
166
|
## Credits
|
167
167
|
|
168
168
|
A huge thanks to [Evan Miller](http://www.evanmiller.org/) for deriving the Bayesian formulas.
|
@@ -1,13 +1,9 @@
|
|
1
1
|
module FieldTest
|
2
|
-
class
|
2
|
+
class BaseController < ActionController::Base
|
3
3
|
layout "field_test/application"
|
4
4
|
|
5
5
|
protect_from_forgery
|
6
6
|
|
7
7
|
http_basic_authenticate_with name: ENV["FIELD_TEST_USERNAME"], password: ENV["FIELD_TEST_PASSWORD"] if ENV["FIELD_TEST_PASSWORD"]
|
8
|
-
|
9
|
-
def index
|
10
|
-
@active_experiments, @completed_experiments = FieldTest::Experiment.all.sort_by(&:id).partition { |e| e.active? }
|
11
|
-
end
|
12
8
|
end
|
13
9
|
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
module FieldTest
|
2
|
+
class ExperimentsController < BaseController
|
3
|
+
def index
|
4
|
+
@active_experiments, @completed_experiments = FieldTest::Experiment.all.sort_by(&:id).partition { |e| e.active? }
|
5
|
+
end
|
6
|
+
|
7
|
+
def show
|
8
|
+
@experiment = FieldTest::Experiment.find(params[:id])
|
9
|
+
|
10
|
+
@per_page = 200
|
11
|
+
@page = [1, params[:page].to_i].max
|
12
|
+
offset = (@page - 1) * @per_page
|
13
|
+
@memberships = @experiment.memberships.order(created_at: :desc).limit(@per_page).offset(offset).to_a
|
14
|
+
rescue FieldTest::ExperimentNotFound
|
15
|
+
raise ActionController::RoutingError, "Experiment not found"
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
module FieldTest
|
2
|
+
class MembershipsController < BaseController
|
3
|
+
def update
|
4
|
+
membership = FieldTest::Membership.find(params[:id])
|
5
|
+
membership.update_attributes(membership_params)
|
6
|
+
redirect_to participant_path(membership.participant)
|
7
|
+
end
|
8
|
+
|
9
|
+
private
|
10
|
+
|
11
|
+
def membership_params
|
12
|
+
params.require(:membership).permit(:variant)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
@@ -1,7 +1,16 @@
|
|
1
1
|
<% experiments.each do |experiment| %>
|
2
2
|
<% results = experiment.results %>
|
3
3
|
|
4
|
-
<h2
|
4
|
+
<h2>
|
5
|
+
<%= experiment.name %>
|
6
|
+
<small><%= link_to "Details", experiment_path(experiment.id) %></small>
|
7
|
+
</h2>
|
8
|
+
|
9
|
+
|
10
|
+
<% if experiment.description %>
|
11
|
+
<p class="description"><%= experiment.description %></p>
|
12
|
+
<% end %>
|
13
|
+
|
5
14
|
<table>
|
6
15
|
<thead>
|
7
16
|
<tr>
|
@@ -18,7 +27,7 @@
|
|
18
27
|
<td>
|
19
28
|
<%= variant %>
|
20
29
|
<% if variant == experiment.winner %>
|
21
|
-
<span
|
30
|
+
<span class="check">✓</span>
|
22
31
|
<% end %>
|
23
32
|
</td>
|
24
33
|
<td><%= result[:participated] %></td>
|
@@ -32,7 +41,11 @@
|
|
32
41
|
</td>
|
33
42
|
<td>
|
34
43
|
<% if result[:prob_winning] %>
|
35
|
-
|
44
|
+
<% if result[:prob_winning] < 0.01 %>
|
45
|
+
< 1%
|
46
|
+
<% else %>
|
47
|
+
<%= (100.0 * result[:prob_winning]).round %>%
|
48
|
+
<% end %>
|
36
49
|
<% end %>
|
37
50
|
</td>
|
38
51
|
</tr>
|
File without changes
|
@@ -0,0 +1,52 @@
|
|
1
|
+
<p>
|
2
|
+
<%= link_to root_path do %>
|
3
|
+
« Experiments
|
4
|
+
<% end %>
|
5
|
+
</p>
|
6
|
+
|
7
|
+
<h1><%= @experiment.name %></h1>
|
8
|
+
|
9
|
+
<table>
|
10
|
+
<thead>
|
11
|
+
<tr>
|
12
|
+
<th>Participant</th>
|
13
|
+
<th style="width: 20%;">Variant</th>
|
14
|
+
<th style="width: 20%;">Converted</th>
|
15
|
+
<th style="width: 20%;">Started</th>
|
16
|
+
</tr>
|
17
|
+
</thead>
|
18
|
+
<tbody>
|
19
|
+
<% @memberships.each do |membership| %>
|
20
|
+
<tr>
|
21
|
+
<td><%= link_to membership.participant, participant_path(membership.participant) %></td>
|
22
|
+
<td><%= membership.variant %></td>
|
23
|
+
<td>
|
24
|
+
<% if membership.converted %>
|
25
|
+
<span class="check">✓</span>
|
26
|
+
<% end %>
|
27
|
+
</td>
|
28
|
+
<td>
|
29
|
+
<% if membership.created_at > 1.day.ago %>
|
30
|
+
<%= time_ago_in_words(membership.created_at, include_seconds: true) %> ago
|
31
|
+
<% else %>
|
32
|
+
<%= membership.created_at.to_formatted_s(:short) %>
|
33
|
+
<% end %>
|
34
|
+
</td>
|
35
|
+
</tr>
|
36
|
+
<% end %>
|
37
|
+
</tbody>
|
38
|
+
</table>
|
39
|
+
|
40
|
+
<p class="pagination">
|
41
|
+
<% unless @page == 1 %>
|
42
|
+
<%= link_to experiment_path(@experiment.id, page: @page - 1) do %>
|
43
|
+
« Prev
|
44
|
+
<% end %>
|
45
|
+
<% end %>
|
46
|
+
<!-- there may not be a next page, but don't want another DB query -->
|
47
|
+
<% if @memberships.size == @per_page %>
|
48
|
+
<%= link_to experiment_path(@experiment.id, page: @page + 1) do %>
|
49
|
+
Next »
|
50
|
+
<% end %>
|
51
|
+
<% end %>
|
52
|
+
</p>
|
@@ -0,0 +1,47 @@
|
|
1
|
+
<p>
|
2
|
+
<%= link_to root_path do %>
|
3
|
+
« Experiments
|
4
|
+
<% end %>
|
5
|
+
</p>
|
6
|
+
|
7
|
+
<h1><%= @participant %></h1>
|
8
|
+
|
9
|
+
<table>
|
10
|
+
<thead>
|
11
|
+
<tr>
|
12
|
+
<th>Experiment</th>
|
13
|
+
<th style="width: 20%;">Variant</th>
|
14
|
+
<th style="width: 20%;">Converted</th>
|
15
|
+
<th style="width: 20%;">Started</th>
|
16
|
+
</tr>
|
17
|
+
</thead>
|
18
|
+
<tbody>
|
19
|
+
<% FieldTest::Membership.where(participant: @participant).each do |membership| %>
|
20
|
+
<tr>
|
21
|
+
<td><%= link_to membership.experiment, experiment_path(membership.experiment) %></td>
|
22
|
+
<td>
|
23
|
+
<% experiment = FieldTest::Experiment.find(membership.experiment) rescue nil %>
|
24
|
+
<% if experiment %>
|
25
|
+
<%= form_for membership do |f| %>
|
26
|
+
<%= f.select "variant", options_for_select(FieldTest::Experiment.find(membership.experiment).variants.map { |v| [v, v] }, membership.variant), {}, onchange: "this.form.submit()" %>
|
27
|
+
<% end %>
|
28
|
+
<% else %>
|
29
|
+
<%= membership.variant %>
|
30
|
+
<% end %>
|
31
|
+
</td>
|
32
|
+
<td>
|
33
|
+
<% if membership.converted %>
|
34
|
+
<span class="check">✓</span>
|
35
|
+
<% end %>
|
36
|
+
</td>
|
37
|
+
<td>
|
38
|
+
<% if membership.created_at > 1.day.ago %>
|
39
|
+
<%= time_ago_in_words(membership.created_at, include_seconds: true) %> ago
|
40
|
+
<% else %>
|
41
|
+
<%= membership.created_at.to_formatted_s(:short) %>
|
42
|
+
<% end %>
|
43
|
+
</td>
|
44
|
+
</tr>
|
45
|
+
<% end %>
|
46
|
+
</tbody>
|
47
|
+
</table>
|
@@ -7,15 +7,21 @@
|
|
7
7
|
margin: 0;
|
8
8
|
padding: 20px;
|
9
9
|
font-family: "Helvetica Neue", Arial, Helvetica, sans-serif;
|
10
|
-
font-size:
|
10
|
+
font-size: 16px;
|
11
11
|
line-height: 1.4;
|
12
12
|
color: #333;
|
13
13
|
}
|
14
14
|
|
15
|
+
a, a:visited, a:active {
|
16
|
+
color: #08c;
|
17
|
+
text-decoration: none;
|
18
|
+
}
|
19
|
+
|
15
20
|
table {
|
16
21
|
width: 100%;
|
17
22
|
border-collapse: collapse;
|
18
23
|
border-spacing: 0;
|
24
|
+
font-size: 16px;
|
19
25
|
margin-bottom: 20px;
|
20
26
|
}
|
21
27
|
|
@@ -31,6 +37,28 @@
|
|
31
37
|
td {
|
32
38
|
border-top: solid 1px #ddd;
|
33
39
|
}
|
40
|
+
|
41
|
+
h2 small {
|
42
|
+
font-size: 16px;
|
43
|
+
font-weight: normal;
|
44
|
+
}
|
45
|
+
|
46
|
+
.description {
|
47
|
+
color: #999;
|
48
|
+
}
|
49
|
+
|
50
|
+
.check {
|
51
|
+
color: #5cb85c;
|
52
|
+
}
|
53
|
+
|
54
|
+
.pagination {
|
55
|
+
text-align: center;
|
56
|
+
}
|
57
|
+
|
58
|
+
.pagination a {
|
59
|
+
padding-left: 10px;
|
60
|
+
padding-right: 10px;
|
61
|
+
}
|
34
62
|
</style>
|
35
63
|
</head>
|
36
64
|
<body>
|
data/config/routes.rb
CHANGED
data/field_test.gemspec
CHANGED
data/lib/field_test.rb
CHANGED
@@ -1,7 +1,9 @@
|
|
1
1
|
require "distribution/math_extension"
|
2
2
|
require "browser"
|
3
|
+
require "active_support"
|
4
|
+
require "field_test/calculations"
|
3
5
|
require "field_test/experiment"
|
4
|
-
require "field_test/engine"
|
6
|
+
require "field_test/engine" if defined?(Rails)
|
5
7
|
require "field_test/helpers"
|
6
8
|
require "field_test/participant"
|
7
9
|
require "field_test/version"
|
@@ -22,6 +24,10 @@ module FieldTest
|
|
22
24
|
config = self.config # dev performance
|
23
25
|
config["exclude"] && config["exclude"]["bots"]
|
24
26
|
end
|
27
|
+
|
28
|
+
def self.cache
|
29
|
+
config["cache"]
|
30
|
+
end
|
25
31
|
end
|
26
32
|
|
27
33
|
ActiveSupport.on_load(:action_controller) do
|
@@ -0,0 +1,59 @@
|
|
1
|
+
# formulas from
|
2
|
+
# http://www.evanmiller.org/bayesian-ab-testing.html
|
3
|
+
|
4
|
+
module FieldTest
|
5
|
+
module Calculations
|
6
|
+
def self.prob_b_beats_a(alpha_a, beta_a, alpha_b, beta_b)
|
7
|
+
total = 0.0
|
8
|
+
|
9
|
+
# for performance
|
10
|
+
logbeta_aa_ba = Math.logbeta(alpha_a, beta_a)
|
11
|
+
beta_ba = beta_b + beta_a
|
12
|
+
|
13
|
+
0.upto(alpha_b - 1) do |i|
|
14
|
+
total += Math.exp(Math.logbeta(alpha_a + i, beta_ba) -
|
15
|
+
Math.log(beta_b + i) - Math.logbeta(1 + i, beta_b) -
|
16
|
+
logbeta_aa_ba)
|
17
|
+
end
|
18
|
+
|
19
|
+
total
|
20
|
+
end
|
21
|
+
|
22
|
+
def self.prob_c_beats_a_and_b(alpha_a, beta_a, alpha_b, beta_b, alpha_c, beta_c)
|
23
|
+
total = 0.0
|
24
|
+
|
25
|
+
# for performance
|
26
|
+
logbeta_ac_bc = Math.logbeta(alpha_c, beta_c)
|
27
|
+
abc = beta_a + beta_b + beta_c
|
28
|
+
log_bb_j = []
|
29
|
+
logbeta_j_bb = []
|
30
|
+
0.upto(alpha_b - 1) do |j|
|
31
|
+
log_bb_j[j] = Math.log(beta_b + j)
|
32
|
+
logbeta_j_bb[j] = Math.logbeta(1 + j, beta_b)
|
33
|
+
end
|
34
|
+
|
35
|
+
logbeta_ac_i_j = []
|
36
|
+
0.upto(alpha_a - 1) do |i|
|
37
|
+
0.upto(alpha_b - 1) do |j|
|
38
|
+
logbeta_ac_i_j[i + j] ||= Math.logbeta(alpha_c + i + j, abc)
|
39
|
+
end
|
40
|
+
end
|
41
|
+
|
42
|
+
0.upto(alpha_a - 1) do |i|
|
43
|
+
# for performance
|
44
|
+
log_ba_i = Math.log(beta_a + i)
|
45
|
+
logbeta_i_ba = Math.logbeta(1 + i, beta_a)
|
46
|
+
|
47
|
+
0.upto(alpha_b - 1) do |j|
|
48
|
+
total += Math.exp(logbeta_ac_i_j[i + j] -
|
49
|
+
log_ba_i - log_bb_j[j] -
|
50
|
+
logbeta_i_ba - logbeta_j_bb[j] -
|
51
|
+
logbeta_ac_bc)
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
1 - prob_b_beats_a(alpha_c, beta_c, alpha_a, beta_a) -
|
56
|
+
prob_b_beats_a(alpha_c, beta_c, alpha_b, beta_b) + total
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
@@ -1,11 +1,12 @@
|
|
1
1
|
module FieldTest
|
2
2
|
class Experiment
|
3
|
-
attr_reader :id, :name, :variants, :weights, :winner, :started_at, :ended_at
|
3
|
+
attr_reader :id, :name, :description, :variants, :weights, :winner, :started_at, :ended_at
|
4
4
|
|
5
5
|
def initialize(attributes)
|
6
6
|
attributes = attributes.symbolize_keys
|
7
7
|
@id = attributes[:id]
|
8
8
|
@name = attributes[:name] || @id.to_s.titleize
|
9
|
+
@description = attributes[:description]
|
9
10
|
@variants = attributes[:variants]
|
10
11
|
@weights = @variants.size.times.map { |i| attributes[:weights].to_a[i] || 1 }
|
11
12
|
@winner = attributes[:winner]
|
@@ -33,6 +34,17 @@ module FieldTest
|
|
33
34
|
if membership.changed?
|
34
35
|
begin
|
35
36
|
membership.save!
|
37
|
+
|
38
|
+
# log it!
|
39
|
+
info = {
|
40
|
+
experiment: id,
|
41
|
+
variant: membership.variant,
|
42
|
+
participant: membership.participant
|
43
|
+
}.merge(options.slice(:ip, :user_agent))
|
44
|
+
|
45
|
+
# sorta logfmt :)
|
46
|
+
info = info.map { |k, v| v = "\"#{v}\"" if k == :user_agent; "#{k}=#{v}" }.join(" ")
|
47
|
+
Rails.logger.info "[field test] #{info}"
|
36
48
|
rescue ActiveRecord::RecordNotUnique
|
37
49
|
membership = memberships.find_by(participant: participants.first)
|
38
50
|
end
|
@@ -75,10 +87,10 @@ module FieldTest
|
|
75
87
|
}
|
76
88
|
end
|
77
89
|
case variants.size
|
78
|
-
when 1
|
79
|
-
|
80
|
-
|
81
|
-
variants.size.times do |i|
|
90
|
+
when 1, 2, 3
|
91
|
+
total = 0.0
|
92
|
+
|
93
|
+
(variants.size - 1).times do |i|
|
82
94
|
c = results.values[i]
|
83
95
|
b = results.values[(i + 1) % variants.size]
|
84
96
|
a = results.values[(i + 2) % variants.size]
|
@@ -90,13 +102,23 @@ module FieldTest
|
|
90
102
|
alpha_c = 1 + c[:converted]
|
91
103
|
beta_c = 1 + c[:participated] - c[:converted]
|
92
104
|
|
93
|
-
results
|
105
|
+
# TODO calculate this incrementally by caching intermediate results
|
106
|
+
prob_winning =
|
94
107
|
if variants.size == 2
|
95
|
-
prob_b_beats_a
|
108
|
+
cache_fetch ["field_test", "prob_b_beats_a", alpha_b, beta_b, alpha_c, beta_c] do
|
109
|
+
Calculations.prob_b_beats_a(alpha_b, beta_b, alpha_c, beta_c)
|
110
|
+
end
|
96
111
|
else
|
97
|
-
prob_c_beats_a_and_b
|
112
|
+
cache_fetch ["field_test", "prob_c_beats_a_and_b", alpha_a, beta_a, alpha_b, beta_b, alpha_c, beta_c] do
|
113
|
+
Calculations.prob_c_beats_a_and_b(alpha_a, beta_a, alpha_b, beta_b, alpha_c, beta_c)
|
114
|
+
end
|
98
115
|
end
|
116
|
+
|
117
|
+
results[variants[i]][:prob_winning] = prob_winning
|
118
|
+
total += prob_winning
|
99
119
|
end
|
120
|
+
|
121
|
+
results[variants.last][:prob_winning] = 1 - total
|
100
122
|
end
|
101
123
|
results
|
102
124
|
end
|
@@ -140,33 +162,12 @@ module FieldTest
|
|
140
162
|
variants.last
|
141
163
|
end
|
142
164
|
|
143
|
-
|
144
|
-
|
145
|
-
|
146
|
-
|
147
|
-
|
148
|
-
0.upto(alpha_b - 1) do |i|
|
149
|
-
total += Math.exp(Math.logbeta(alpha_a + i, beta_b + beta_a) -
|
150
|
-
Math.log(beta_b + i) - Math.logbeta(1 + i, beta_b) -
|
151
|
-
Math.logbeta(alpha_a, beta_a))
|
152
|
-
end
|
153
|
-
|
154
|
-
total
|
155
|
-
end
|
156
|
-
|
157
|
-
def prob_c_beats_a_and_b(alpha_a, beta_a, alpha_b, beta_b, alpha_c, beta_c)
|
158
|
-
total = 0.0
|
159
|
-
0.upto(alpha_a - 1) do |i|
|
160
|
-
0.upto(alpha_b - 1) do |j|
|
161
|
-
total += Math.exp(Math.logbeta(alpha_c + i + j, beta_a + beta_b + beta_c) -
|
162
|
-
Math.log(beta_a + i) - Math.log(beta_b + j) -
|
163
|
-
Math.logbeta(1 + i, beta_a) - Math.logbeta(1 + j, beta_b) -
|
164
|
-
Math.logbeta(alpha_c, beta_c))
|
165
|
-
end
|
165
|
+
def cache_fetch(key)
|
166
|
+
if FieldTest.cache
|
167
|
+
Rails.cache.fetch(key.join("/")) { yield }
|
168
|
+
else
|
169
|
+
yield
|
166
170
|
end
|
167
|
-
|
168
|
-
1 - prob_b_beats_a(alpha_c, beta_c, alpha_a, beta_a) -
|
169
|
-
prob_b_beats_a(alpha_c, beta_c, alpha_b, beta_b) + total
|
170
171
|
end
|
171
172
|
end
|
172
173
|
end
|
data/lib/field_test/helpers.rb
CHANGED
@@ -13,6 +13,9 @@ module FieldTest
|
|
13
13
|
if FieldTest.exclude_bots?
|
14
14
|
options[:exclude] = Browser.new(request.user_agent).bot?
|
15
15
|
end
|
16
|
+
|
17
|
+
options[:ip] = request.remote_ip
|
18
|
+
options[:user_agent] = request.user_agent
|
16
19
|
end
|
17
20
|
|
18
21
|
# cache results for request
|
@@ -54,13 +57,19 @@ module FieldTest
|
|
54
57
|
if try(:request)
|
55
58
|
# use cookie
|
56
59
|
cookie_key = "field_test"
|
60
|
+
|
57
61
|
token = cookies[cookie_key]
|
62
|
+
token = token.gsub(/[^a-z0-9\-]/i, "") if token
|
63
|
+
|
58
64
|
if participants.empty? && !token
|
59
65
|
token = SecureRandom.uuid
|
60
66
|
cookies[cookie_key] = {value: token, expires: 30.days.from_now}
|
61
67
|
end
|
62
68
|
if token
|
63
|
-
participants <<
|
69
|
+
participants << token
|
70
|
+
|
71
|
+
# backwards compatibility
|
72
|
+
participants << "cookie:#{token}"
|
64
73
|
end
|
65
74
|
end
|
66
75
|
|
data/lib/field_test/version.rb
CHANGED
@@ -10,5 +10,6 @@ class <%= migration_class_name %> < ActiveRecord::Migration
|
|
10
10
|
|
11
11
|
add_index :field_test_memberships, [:experiment, :participant], unique: true
|
12
12
|
add_index :field_test_memberships, :participant
|
13
|
+
add_index :field_test_memberships, [:experiment, :created_at]
|
13
14
|
end
|
14
15
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: field_test
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2016-12-
|
11
|
+
date: 2016-12-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: railties
|
@@ -94,6 +94,20 @@ dependencies:
|
|
94
94
|
- - ">="
|
95
95
|
- !ruby/object:Gem::Version
|
96
96
|
version: '0'
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: minitest
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - ">="
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '0'
|
104
|
+
type: :development
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - ">="
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '0'
|
97
111
|
description:
|
98
112
|
email:
|
99
113
|
- andrew@chartkick.com
|
@@ -107,14 +121,20 @@ files:
|
|
107
121
|
- LICENSE.txt
|
108
122
|
- README.md
|
109
123
|
- Rakefile
|
110
|
-
- app/controllers/field_test/
|
124
|
+
- app/controllers/field_test/base_controller.rb
|
125
|
+
- app/controllers/field_test/experiments_controller.rb
|
126
|
+
- app/controllers/field_test/memberships_controller.rb
|
127
|
+
- app/controllers/field_test/participants_controller.rb
|
111
128
|
- app/models/field_test/membership.rb
|
112
|
-
- app/views/field_test/
|
113
|
-
- app/views/field_test/
|
129
|
+
- app/views/field_test/experiments/_experiments.html.erb
|
130
|
+
- app/views/field_test/experiments/index.html.erb
|
131
|
+
- app/views/field_test/experiments/show.html.erb
|
132
|
+
- app/views/field_test/participants/show.html.erb
|
114
133
|
- app/views/layouts/field_test/application.html.erb
|
115
134
|
- config/routes.rb
|
116
135
|
- field_test.gemspec
|
117
136
|
- lib/field_test.rb
|
137
|
+
- lib/field_test/calculations.rb
|
118
138
|
- lib/field_test/engine.rb
|
119
139
|
- lib/field_test/experiment.rb
|
120
140
|
- lib/field_test/helpers.rb
|