field_test 0.2.4 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of field_test might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +14 -0
- data/README.md +184 -21
- data/app/controllers/field_test/memberships_controller.rb +2 -2
- data/app/controllers/field_test/participants_controller.rb +10 -2
- data/app/helpers/field_test/base_helper.rb +12 -0
- data/app/models/field_test/membership.rb +2 -1
- data/app/views/field_test/experiments/_experiments.html.erb +2 -2
- data/app/views/field_test/experiments/show.html.erb +1 -1
- data/config/routes.rb +2 -1
- data/lib/field_test.rb +44 -13
- data/lib/field_test/controller.rb +57 -0
- data/lib/field_test/experiment.rb +56 -43
- data/lib/field_test/helpers.rb +10 -48
- data/lib/field_test/mailer.rb +20 -0
- data/lib/field_test/participant.rb +33 -2
- data/lib/field_test/version.rb +1 -1
- data/lib/generators/field_test/events_generator.rb +1 -3
- data/lib/generators/field_test/install_generator.rb +1 -3
- data/lib/generators/field_test/templates/events.rb.tt +1 -3
- data/lib/generators/field_test/templates/memberships.rb.tt +4 -3
- metadata +57 -17
- data/.gitignore +0 -9
- data/Gemfile +0 -4
- data/Rakefile +0 -10
- data/field_test.gemspec +0 -30
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5303485194bd6850672facf548ba671a2e661e242edfc4a4e3ad90beeed7485d
|
4
|
+
data.tar.gz: c694d25c71464323daf17e7cd9844d8aa8f51eb3d65e0f788e92e0f5998f18b5
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: df92ba63ed13eaec202546d497b2408288b9b4c8901e3a8a25fff96fbb098fa4c3f6acbb9edcd713f94b4ff95540c9542a3c026b4842ca519b76b7f31df2888b
|
7
|
+
data.tar.gz: b9407161a151713987f2db017200ad673cff0eba08f5c114d5787bb322dc4eb87f431fac708b90b42653bed90d34a71a595763cbd32cb107a2a631bcfb7f198c
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,17 @@
|
|
1
|
+
## 0.3.0
|
2
|
+
|
3
|
+
- Added support for native apps
|
4
|
+
- Added `cookies` option
|
5
|
+
- Added `precision` option
|
6
|
+
- Fixed bug in results with multiple goals
|
7
|
+
- Fixed issue where metrics disappeared from dashboard when moving to multiple goals
|
8
|
+
- Dropped support for Rails < 5
|
9
|
+
|
10
|
+
Breaking changes
|
11
|
+
|
12
|
+
- Split out participant id and type
|
13
|
+
- Changed participant logic for emails
|
14
|
+
|
1
15
|
## 0.2.4
|
2
16
|
|
3
17
|
- Fixed `PG::AmbiguousColumn` error
|
data/README.md
CHANGED
@@ -3,11 +3,13 @@
|
|
3
3
|
:maple_leaf: A/B testing for Rails
|
4
4
|
|
5
5
|
- Designed for web and email
|
6
|
-
- Comes with a [
|
6
|
+
- Comes with a [dashboard](https://fieldtest.dokkuapp.com/) to view results and update variants
|
7
7
|
- Seamlessly handles the transition from anonymous visitor to logged in user
|
8
8
|
|
9
9
|
Uses [Bayesian statistics](https://www.evanmiller.org/bayesian-ab-testing.html) to evaluate results so you don’t need to choose a sample size ahead of time.
|
10
10
|
|
11
|
+
[![Build Status](https://travis-ci.org/ankane/field_test.svg?branch=master)](https://travis-ci.org/ankane/field_test)
|
12
|
+
|
11
13
|
## Installation
|
12
14
|
|
13
15
|
Add this line to your application’s Gemfile:
|
@@ -19,7 +21,8 @@ gem "field_test"
|
|
19
21
|
Run:
|
20
22
|
|
21
23
|
```sh
|
22
|
-
rails
|
24
|
+
rails generate field_test:install
|
25
|
+
rails db:migrate
|
23
26
|
```
|
24
27
|
|
25
28
|
And mount the dashboard in your `config/routes.rb`:
|
@@ -28,7 +31,7 @@ And mount the dashboard in your `config/routes.rb`:
|
|
28
31
|
mount FieldTest::Engine, at: "field_test"
|
29
32
|
```
|
30
33
|
|
31
|
-
Be sure to [secure the dashboard](#security) in production.
|
34
|
+
Be sure to [secure the dashboard](#dashboard-security) in production.
|
32
35
|
|
33
36
|
![Screenshot](https://ankane.github.io/field_test/screenshot6.png)
|
34
37
|
|
@@ -45,12 +48,18 @@ experiments:
|
|
45
48
|
- blue
|
46
49
|
```
|
47
50
|
|
48
|
-
Refer to it in
|
51
|
+
Refer to it in controllers, views, and mailers.
|
49
52
|
|
50
53
|
```ruby
|
51
54
|
button_color = field_test(:button_color)
|
52
55
|
```
|
53
56
|
|
57
|
+
To make testing easier, you can specify a variant with query parameters
|
58
|
+
|
59
|
+
```
|
60
|
+
http://localhost:3000/?field_test[button_color]=green
|
61
|
+
```
|
62
|
+
|
54
63
|
When someone converts, record it with:
|
55
64
|
|
56
65
|
```ruby
|
@@ -67,22 +76,60 @@ experiments:
|
|
67
76
|
|
68
77
|
All calls to `field_test` will now return the winner, and metrics will stop being recorded.
|
69
78
|
|
70
|
-
|
79
|
+
You can get the list of experiments and variants for a user with:
|
80
|
+
|
81
|
+
```ruby
|
82
|
+
field_test_experiments
|
83
|
+
```
|
71
84
|
|
72
|
-
|
85
|
+
## JavaScript and Native Apps
|
73
86
|
|
87
|
+
For JavaScript and native apps, add calls to your normal endpoints.
|
88
|
+
|
89
|
+
```ruby
|
90
|
+
class CheckoutController < ActionController::API
|
91
|
+
def start
|
92
|
+
render json: {button_color: field_test(:button_color)}
|
93
|
+
end
|
94
|
+
|
95
|
+
def finish
|
96
|
+
field_test_converted(:button_color)
|
97
|
+
# ...
|
98
|
+
end
|
99
|
+
end
|
74
100
|
```
|
75
|
-
|
101
|
+
|
102
|
+
For anonymous visitors in native apps, pass a `Field-Test-Visitor` header with a unique identifier.
|
103
|
+
|
104
|
+
## Participants
|
105
|
+
|
106
|
+
Any model or string can be a participant in an experiment.
|
107
|
+
|
108
|
+
For web requests, it uses `current_user` (if it exists) and an anonymous visitor id to determine the participant. Set your own with:
|
109
|
+
|
110
|
+
```ruby
|
111
|
+
class ApplicationController < ActionController::Base
|
112
|
+
def field_test_participant
|
113
|
+
current_company
|
114
|
+
end
|
115
|
+
end
|
76
116
|
```
|
77
117
|
|
78
|
-
|
118
|
+
For mailers, it tries `@user` then `params[:user]` to determine the participant. Set your own with:
|
79
119
|
|
80
120
|
```ruby
|
81
|
-
|
82
|
-
|
121
|
+
class ApplicationMailer < ActionMailer::Base
|
122
|
+
def field_test_participant
|
123
|
+
@company
|
124
|
+
end
|
125
|
+
end
|
83
126
|
```
|
84
127
|
|
85
|
-
You can also
|
128
|
+
You can also manually pass a participant with:
|
129
|
+
|
130
|
+
```ruby
|
131
|
+
field_test(:button_color, participant: company)
|
132
|
+
```
|
86
133
|
|
87
134
|
## Config
|
88
135
|
|
@@ -126,6 +173,14 @@ experiments:
|
|
126
173
|
- 15
|
127
174
|
```
|
128
175
|
|
176
|
+
To help with GDPR compliance, you can switch from cookies to [anonymity sets](https://privacypatterns.org/patterns/Anonymity-set) for anonymous visitors. Visitors with the same IP mask and user agent are grouped together.
|
177
|
+
|
178
|
+
```yml
|
179
|
+
cookies: false
|
180
|
+
```
|
181
|
+
|
182
|
+
## Dashboard Config
|
183
|
+
|
129
184
|
If the dashboard gets slow, you can make it faster with:
|
130
185
|
|
131
186
|
```yml
|
@@ -134,12 +189,19 @@ cache: true
|
|
134
189
|
|
135
190
|
This will use the Rails cache to speed up winning probability calculations.
|
136
191
|
|
137
|
-
|
192
|
+
If you need more precision, set:
|
193
|
+
|
194
|
+
```yml
|
195
|
+
precision: 1
|
196
|
+
```
|
197
|
+
|
198
|
+
## Multiple Goals
|
138
199
|
|
139
200
|
You can set multiple goals for an experiment to track conversions at different parts of the funnel. First, run:
|
140
201
|
|
141
202
|
```sh
|
142
|
-
rails
|
203
|
+
rails generate field_test:events
|
204
|
+
rails db:migrate
|
143
205
|
```
|
144
206
|
|
145
207
|
And add to your config:
|
@@ -162,20 +224,30 @@ The results for all goals will appear on the dashboard.
|
|
162
224
|
|
163
225
|
## Analytics Platforms
|
164
226
|
|
165
|
-
You
|
227
|
+
You may also want to send experiment data as properties to other analytics platforms like [Segment](https://segment.com), [Amplitude](https://amplitude.com), and [Ahoy](https://github.com/ankane/ahoy). Get the list of experiments and variants with:
|
166
228
|
|
167
229
|
```ruby
|
168
230
|
field_test_experiments
|
169
231
|
```
|
170
232
|
|
171
|
-
|
233
|
+
### Ahoy
|
234
|
+
|
235
|
+
You can configure Field Test to use Ahoy’s visitor token instead of creating its own:
|
236
|
+
|
237
|
+
```ruby
|
238
|
+
class ApplicationController < ActionController::Base
|
239
|
+
def field_test_participant
|
240
|
+
[ahoy.user, ahoy.visitor_token]
|
241
|
+
end
|
242
|
+
end
|
243
|
+
```
|
172
244
|
|
173
|
-
## Security
|
245
|
+
## Dashboard Security
|
174
246
|
|
175
247
|
#### Devise
|
176
248
|
|
177
249
|
```ruby
|
178
|
-
authenticate :user, ->
|
250
|
+
authenticate :user, ->(user) { user.admin? } do
|
179
251
|
mount FieldTest::Engine, at: "field_test"
|
180
252
|
end
|
181
253
|
```
|
@@ -189,13 +261,104 @@ ENV["FIELD_TEST_USERNAME"] = "moonrise"
|
|
189
261
|
ENV["FIELD_TEST_PASSWORD"] = "kingdom"
|
190
262
|
```
|
191
263
|
|
192
|
-
##
|
264
|
+
## Reference
|
193
265
|
|
194
|
-
|
266
|
+
Assign a specific variant to a user with:
|
267
|
+
|
268
|
+
```ruby
|
269
|
+
experiment = FieldTest::Experiment.find(:button_color)
|
270
|
+
experiment.variant(participant, variant: "green")
|
271
|
+
```
|
272
|
+
|
273
|
+
You can also change a user’s variant from the dashboard.
|
274
|
+
|
275
|
+
To associate models with field test memberships, use:
|
276
|
+
|
277
|
+
```ruby
|
278
|
+
class User < ApplicationRecord
|
279
|
+
has_many :field_test_memberships, class_name: "FieldTest::Membership", as: :participant
|
280
|
+
end
|
281
|
+
```
|
282
|
+
|
283
|
+
Now you can do:
|
284
|
+
|
285
|
+
```ruby
|
286
|
+
user.field_test_memberships
|
287
|
+
```
|
288
|
+
|
289
|
+
## Upgrading
|
290
|
+
|
291
|
+
### 0.3.0
|
292
|
+
|
293
|
+
Upgrade the gem and add to `config/field_test.yml`:
|
294
|
+
|
295
|
+
```yml
|
296
|
+
legacy_participants: true
|
297
|
+
```
|
298
|
+
|
299
|
+
Also, if you use Field Test in emails, know that the default way participants are determined has changed. Restore the previous way with:
|
300
|
+
|
301
|
+
```ruby
|
302
|
+
class ApplicationMailer < ActionMailer::Base
|
303
|
+
def field_test_participant
|
304
|
+
message.to.first
|
305
|
+
end
|
306
|
+
end
|
307
|
+
```
|
308
|
+
|
309
|
+
We also recommend upgrading participants when you have time.
|
310
|
+
|
311
|
+
#### Upgrading Participants
|
312
|
+
|
313
|
+
Field Test 0.3.0 splits the `field_test_memberships.participant` column into `participant_type` and `participant_id`.
|
314
|
+
|
315
|
+
To upgrade without downtime, create a migration:
|
195
316
|
|
196
|
-
|
317
|
+
```sh
|
318
|
+
rails generate migration upgrade_field_test_participants
|
319
|
+
```
|
320
|
+
|
321
|
+
with:
|
197
322
|
|
198
|
-
|
323
|
+
```ruby
|
324
|
+
class UpgradeFieldTestParticipants < ActiveRecord::Migration[5.2]
|
325
|
+
def change
|
326
|
+
add_column :field_test_memberships, :participant_type, :string
|
327
|
+
add_column :field_test_memberships, :participant_id, :string
|
328
|
+
|
329
|
+
add_index :field_test_memberships, [:participant_type, :participant_id, :experiment],
|
330
|
+
unique: true, name: "index_field_test_memberships_on_participant_and_experiment"
|
331
|
+
end
|
332
|
+
end
|
333
|
+
```
|
334
|
+
|
335
|
+
After you run it, writes will go to both the old and new sets of columns.
|
336
|
+
|
337
|
+
Next, backfill data:
|
338
|
+
|
339
|
+
```ruby
|
340
|
+
FieldTest::Membership.where(participant_id: nil).find_each do |membership|
|
341
|
+
participant = membership.participant
|
342
|
+
|
343
|
+
if participant.include?(":")
|
344
|
+
participant_type, _, participant_id = participant.rpartition(":")
|
345
|
+
participant_type = nil if participant_type == "cookie" # legacy
|
346
|
+
else
|
347
|
+
participant_id = participant
|
348
|
+
end
|
349
|
+
|
350
|
+
membership.update!(
|
351
|
+
participant_type: participant_type,
|
352
|
+
participant_id: participant_id
|
353
|
+
)
|
354
|
+
end
|
355
|
+
```
|
356
|
+
|
357
|
+
Finally, remove `legacy_participants: true` from the config file. Once you confirm it’s working, you can drop the `participant` column (you can rename it first just to be extra safe).
|
358
|
+
|
359
|
+
## Credits
|
360
|
+
|
361
|
+
A huge thanks to [Evan Miller](https://www.evanmiller.org/) for deriving the Bayesian formulas.
|
199
362
|
|
200
363
|
## History
|
201
364
|
|
@@ -2,8 +2,8 @@ module FieldTest
|
|
2
2
|
class MembershipsController < BaseController
|
3
3
|
def update
|
4
4
|
membership = FieldTest::Membership.find(params[:id])
|
5
|
-
membership.
|
6
|
-
|
5
|
+
membership.update!(membership_params)
|
6
|
+
redirect_back(fallback_location: root_path)
|
7
7
|
end
|
8
8
|
|
9
9
|
private
|
@@ -1,9 +1,17 @@
|
|
1
1
|
module FieldTest
|
2
2
|
class ParticipantsController < BaseController
|
3
3
|
def show
|
4
|
-
@participant = params[:id]
|
5
4
|
# TODO better ordering
|
6
|
-
@memberships =
|
5
|
+
@memberships =
|
6
|
+
if FieldTest.legacy_participants
|
7
|
+
@participant = params[:id]
|
8
|
+
FieldTest::Membership.where(participant: @participant).order(:id)
|
9
|
+
else
|
10
|
+
id = params[:id]
|
11
|
+
type = params[:type]
|
12
|
+
@participant = [type, id].compact.join(" ")
|
13
|
+
FieldTest::Membership.where(participant_type: type, participant_id: id).order(:id)
|
14
|
+
end
|
7
15
|
|
8
16
|
@events =
|
9
17
|
if FieldTest.events_supported?
|
@@ -0,0 +1,12 @@
|
|
1
|
+
module FieldTest
|
2
|
+
module BaseHelper
|
3
|
+
def field_test_participant_link(membership)
|
4
|
+
if FieldTest.legacy_participants
|
5
|
+
link_to membership.participant, legacy_participant_path(membership.participant)
|
6
|
+
else
|
7
|
+
text = [membership.participant_type, membership.participant_id].compact.join(" ")
|
8
|
+
link_to text, participant_path(type: membership.participant_type, id: membership.participant_id)
|
9
|
+
end
|
10
|
+
end
|
11
|
+
end
|
12
|
+
end
|
@@ -4,7 +4,8 @@ module FieldTest
|
|
4
4
|
|
5
5
|
has_many :events, class_name: "FieldTest::Event"
|
6
6
|
|
7
|
-
validates :participant, presence: true
|
7
|
+
validates :participant, presence: true, if: -> { FieldTest.legacy_participants }
|
8
|
+
validates :participant_id, presence: true, if: -> { !FieldTest.legacy_participants }
|
8
9
|
validates :experiment, presence: true
|
9
10
|
validates :variant, presence: true
|
10
11
|
end
|
@@ -38,7 +38,7 @@
|
|
38
38
|
<td><%= result[:converted] %></td>
|
39
39
|
<td>
|
40
40
|
<% if result[:conversion_rate] %>
|
41
|
-
<%= (100.0 * result[:conversion_rate]).round %>%
|
41
|
+
<%= (100.0 * result[:conversion_rate]).round(FieldTest.precision) %>%
|
42
42
|
<% else %>
|
43
43
|
-
|
44
44
|
<% end %>
|
@@ -48,7 +48,7 @@
|
|
48
48
|
<% if result[:prob_winning] < 0.01 %>
|
49
49
|
< 1%
|
50
50
|
<% else %>
|
51
|
-
<%= (100.0 * result[:prob_winning]).round %>%
|
51
|
+
<%= (100.0 * result[:prob_winning]).round(FieldTest.precision) %>%
|
52
52
|
<% end %>
|
53
53
|
<% end %>
|
54
54
|
</td>
|
@@ -22,7 +22,7 @@
|
|
22
22
|
<tbody>
|
23
23
|
<% @memberships.each do |membership| %>
|
24
24
|
<tr>
|
25
|
-
<td><%=
|
25
|
+
<td><%= field_test_participant_link(membership) %></td>
|
26
26
|
<td><%= membership.variant %></td>
|
27
27
|
<td>
|
28
28
|
<% converted = false %>
|
data/config/routes.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
FieldTest::Engine.routes.draw do
|
2
2
|
resources :experiments, only: [:show]
|
3
3
|
resources :memberships, only: [:update]
|
4
|
-
get "participants/:id", to: "participants#show", constraints: {id: /.+/}, as: :
|
4
|
+
get "participants/:id", to: "participants#show", constraints: {id: /.+/}, as: :legacy_participant
|
5
|
+
get "participants", to: "participants#show", as: :participant
|
5
6
|
root "experiments#index"
|
6
7
|
end
|
data/lib/field_test.rb
CHANGED
@@ -1,26 +1,36 @@
|
|
1
|
-
|
1
|
+
# dependencies
|
2
2
|
require "active_support"
|
3
|
+
require "browser"
|
4
|
+
require "ipaddr"
|
5
|
+
|
6
|
+
# modules
|
3
7
|
require "field_test/calculations"
|
4
8
|
require "field_test/experiment"
|
5
|
-
require "field_test/engine" if defined?(Rails)
|
6
9
|
require "field_test/helpers"
|
7
10
|
require "field_test/participant"
|
8
11
|
require "field_test/version"
|
9
12
|
|
13
|
+
# integrations
|
14
|
+
require "field_test/engine" if defined?(Rails)
|
15
|
+
|
10
16
|
module FieldTest
|
11
17
|
class Error < StandardError; end
|
12
18
|
class ExperimentNotFound < Error; end
|
13
19
|
class UnknownParticipant < Error; end
|
14
20
|
|
15
|
-
|
16
|
-
|
17
|
-
@config = nil if Rails.env.development?
|
21
|
+
# same as ahoy
|
22
|
+
UUID_NAMESPACE = "a82ae811-5011-45ab-a728-569df7499c5f"
|
18
23
|
|
19
|
-
|
24
|
+
def self.config_path
|
25
|
+
path = defined?(Rails) ? Rails.root : File
|
26
|
+
path.join("config", "field_test.yml")
|
27
|
+
end
|
28
|
+
|
29
|
+
def self.config
|
30
|
+
@config ||= YAML.load(ERB.new(File.read(config_path)).result)
|
20
31
|
end
|
21
32
|
|
22
33
|
def self.exclude_bots?
|
23
|
-
config = self.config # dev performance
|
24
34
|
config["exclude"] && config["exclude"]["bots"]
|
25
35
|
end
|
26
36
|
|
@@ -28,6 +38,18 @@ module FieldTest
|
|
28
38
|
config["cache"]
|
29
39
|
end
|
30
40
|
|
41
|
+
def self.cookies
|
42
|
+
config.key?("cookies") ? config["cookies"] : true
|
43
|
+
end
|
44
|
+
|
45
|
+
def self.legacy_participants
|
46
|
+
config["legacy_participants"]
|
47
|
+
end
|
48
|
+
|
49
|
+
def self.precision
|
50
|
+
config["precision"] || 0
|
51
|
+
end
|
52
|
+
|
31
53
|
def self.events_supported?
|
32
54
|
unless defined?(@events_supported)
|
33
55
|
connection = FieldTest::Membership.connection
|
@@ -41,16 +63,25 @@ module FieldTest
|
|
41
63
|
end
|
42
64
|
@events_supported
|
43
65
|
end
|
44
|
-
end
|
45
66
|
|
46
|
-
|
47
|
-
|
67
|
+
def self.mask_ip(ip)
|
68
|
+
addr = IPAddr.new(ip)
|
69
|
+
if addr.ipv4?
|
70
|
+
# set last octet to 0
|
71
|
+
addr.mask(24).to_s
|
72
|
+
else
|
73
|
+
# set last 80 bits to zeros
|
74
|
+
addr.mask(48).to_s
|
75
|
+
end
|
76
|
+
end
|
48
77
|
end
|
49
78
|
|
50
|
-
ActiveSupport.on_load(:
|
51
|
-
|
79
|
+
ActiveSupport.on_load(:action_controller) do
|
80
|
+
require "field_test/controller"
|
81
|
+
include FieldTest::Controller
|
52
82
|
end
|
53
83
|
|
54
84
|
ActiveSupport.on_load(:action_mailer) do
|
55
|
-
|
85
|
+
require "field_test/mailer"
|
86
|
+
include FieldTest::Mailer
|
56
87
|
end
|
@@ -0,0 +1,57 @@
|
|
1
|
+
module FieldTest
|
2
|
+
module Controller
|
3
|
+
extend ActiveSupport::Concern
|
4
|
+
include Helpers
|
5
|
+
|
6
|
+
included do
|
7
|
+
if respond_to?(:helper_method)
|
8
|
+
helper_method :field_test
|
9
|
+
helper_method :field_test_converted
|
10
|
+
helper_method :field_test_experiments
|
11
|
+
end
|
12
|
+
end
|
13
|
+
|
14
|
+
private
|
15
|
+
|
16
|
+
def field_test_participant
|
17
|
+
participants = []
|
18
|
+
|
19
|
+
if respond_to?(:current_user, true)
|
20
|
+
user = send(:current_user)
|
21
|
+
participants << user if user
|
22
|
+
end
|
23
|
+
|
24
|
+
cookie_key = "field_test"
|
25
|
+
|
26
|
+
if request.headers["Field-Test-Visitor"]
|
27
|
+
token = request.headers["Field-Test-Visitor"]
|
28
|
+
elsif FieldTest.cookies
|
29
|
+
token = cookies[cookie_key]
|
30
|
+
|
31
|
+
if participants.empty? && !token
|
32
|
+
token = SecureRandom.uuid
|
33
|
+
cookies[cookie_key] = {value: token, expires: 30.days.from_now}
|
34
|
+
end
|
35
|
+
else
|
36
|
+
# anonymity set
|
37
|
+
# note: hashing does not conceal input
|
38
|
+
token = Digest::UUID.uuid_v5(FieldTest::UUID_NAMESPACE, ["visitor", FieldTest.mask_ip(request.remote_ip), request.user_agent].join("/"))
|
39
|
+
|
40
|
+
# delete cookie if present
|
41
|
+
cookies.delete(cookie_key) if cookies[cookie_key]
|
42
|
+
end
|
43
|
+
|
44
|
+
# sanitize tokens
|
45
|
+
token = token.gsub(/[^a-z0-9\-]/i, "") if token
|
46
|
+
|
47
|
+
if token.present?
|
48
|
+
participants << token
|
49
|
+
|
50
|
+
# backwards compatibility
|
51
|
+
participants << "cookie:#{token}" if FieldTest.legacy_participants
|
52
|
+
end
|
53
|
+
|
54
|
+
participants
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
@@ -13,6 +13,7 @@ module FieldTest
|
|
13
13
|
@started_at = Time.zone.parse(attributes[:started_at].to_s) if attributes[:started_at]
|
14
14
|
@ended_at = Time.zone.parse(attributes[:ended_at].to_s) if attributes[:ended_at]
|
15
15
|
@goals = attributes[:goals] || ["conversion"]
|
16
|
+
@goals_defined = !attributes[:goals].nil?
|
16
17
|
@use_events = attributes[:use_events]
|
17
18
|
end
|
18
19
|
|
@@ -30,25 +31,18 @@ module FieldTest
|
|
30
31
|
membership.variant ||= weighted_variant
|
31
32
|
end
|
32
33
|
|
34
|
+
participant = participants.first
|
35
|
+
|
33
36
|
# upgrade to preferred participant
|
34
|
-
membership.participant =
|
37
|
+
membership.participant = participant.participant if membership.respond_to?(:participant=)
|
38
|
+
membership.participant_type = participant.type if membership.respond_to?(:participant_type=)
|
39
|
+
membership.participant_id = participant.id if membership.respond_to?(:participant_id=)
|
35
40
|
|
36
41
|
if membership.changed?
|
37
42
|
begin
|
38
43
|
membership.save!
|
39
|
-
|
40
|
-
# log it!
|
41
|
-
info = {
|
42
|
-
experiment: id,
|
43
|
-
variant: membership.variant,
|
44
|
-
participant: membership.participant
|
45
|
-
}.merge(options.slice(:ip, :user_agent))
|
46
|
-
|
47
|
-
# sorta logfmt :)
|
48
|
-
info = info.map { |k, v| v = "\"#{v}\"" if k == :user_agent; "#{k}=#{v}" }.join(" ")
|
49
|
-
Rails.logger.info "[field test] #{info}"
|
50
44
|
rescue ActiveRecord::RecordNotUnique
|
51
|
-
membership = memberships.find_by(participant
|
45
|
+
membership = memberships.find_by(participant.where_values)
|
52
46
|
end
|
53
47
|
end
|
54
48
|
|
@@ -100,16 +94,29 @@ module FieldTest
|
|
100
94
|
relation = relation.where("field_test_memberships.created_at >= ?", started_at) if started_at
|
101
95
|
relation = relation.where("field_test_memberships.created_at <= ?", ended_at) if ended_at
|
102
96
|
|
103
|
-
if use_events?
|
97
|
+
if use_events? && @goals_defined
|
104
98
|
data = {}
|
105
|
-
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
|
99
|
+
|
100
|
+
participated = relation.count
|
101
|
+
|
102
|
+
adapter_name = relation.connection.adapter_name
|
103
|
+
column =
|
104
|
+
if FieldTest.legacy_participants
|
105
|
+
:participant
|
106
|
+
elsif adapter_name =~ /postg/i # postgres
|
107
|
+
"(participant_type, participant_id)"
|
108
|
+
elsif adapter_name =~ /mysql/i
|
109
|
+
"participant_type, participant_id"
|
110
|
+
else
|
111
|
+
# not perfect, but it'll do
|
112
|
+
"COALESCE(participant_type, '') || ':' || participant_id"
|
113
|
+
end
|
114
|
+
|
115
|
+
converted = events.merge(relation).where(field_test_events: {name: goal}).distinct.count(column)
|
116
|
+
|
117
|
+
(participated.keys + converted.keys).uniq.each do |variant|
|
118
|
+
data[[variant, true]] = converted[variant].to_i
|
119
|
+
data[[variant, false]] = participated[variant].to_i - converted[variant].to_i
|
113
120
|
end
|
114
121
|
else
|
115
122
|
data = relation.group(:converted).count
|
@@ -125,6 +132,7 @@ module FieldTest
|
|
125
132
|
conversion_rate: participated > 0 ? converted.to_f / participated : nil
|
126
133
|
}
|
127
134
|
end
|
135
|
+
|
128
136
|
case variants.size
|
129
137
|
when 1, 2, 3
|
130
138
|
total = 0.0
|
@@ -189,32 +197,37 @@ module FieldTest
|
|
189
197
|
|
190
198
|
private
|
191
199
|
|
192
|
-
|
193
|
-
|
194
|
-
|
200
|
+
def check_participants(participants)
|
201
|
+
raise FieldTest::UnknownParticipant, "Use the :participant option to specify a participant" if participants.empty?
|
202
|
+
end
|
195
203
|
|
196
|
-
|
197
|
-
|
198
|
-
|
204
|
+
# TODO fetch in single query
|
205
|
+
def membership_for(participants)
|
206
|
+
membership = nil
|
207
|
+
participants.each do |participant|
|
208
|
+
membership = self.memberships.find_by(participant.where_values)
|
209
|
+
break if membership
|
199
210
|
end
|
211
|
+
membership
|
212
|
+
end
|
200
213
|
|
201
|
-
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
208
|
-
end
|
209
|
-
variants.last
|
214
|
+
def weighted_variant
|
215
|
+
total = weights.sum.to_f
|
216
|
+
pick = rand
|
217
|
+
n = 0
|
218
|
+
weights.map { |w| w / total }.each_with_index do |w, i|
|
219
|
+
n += w
|
220
|
+
return variants[i] if n >= pick
|
210
221
|
end
|
222
|
+
variants.last
|
223
|
+
end
|
211
224
|
|
212
|
-
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
end
|
225
|
+
def cache_fetch(key)
|
226
|
+
if FieldTest.cache
|
227
|
+
Rails.cache.fetch(key.join("/")) { yield }
|
228
|
+
else
|
229
|
+
yield
|
218
230
|
end
|
231
|
+
end
|
219
232
|
end
|
220
233
|
end
|
data/lib/field_test/helpers.rb
CHANGED
@@ -3,11 +3,11 @@ module FieldTest
|
|
3
3
|
def field_test(experiment, options = {})
|
4
4
|
exp = FieldTest::Experiment.find(experiment)
|
5
5
|
|
6
|
-
participants =
|
6
|
+
participants = FieldTest::Participant.standardize(field_test_participant, options)
|
7
7
|
|
8
8
|
if try(:request)
|
9
|
-
if params[:field_test] && params[:field_test][experiment]
|
10
|
-
|
9
|
+
if !options[:variant] && params[:field_test] && params[:field_test][experiment]
|
10
|
+
params_variant = params[:field_test][experiment]
|
11
11
|
end
|
12
12
|
|
13
13
|
if FieldTest.exclude_bots?
|
@@ -20,67 +20,29 @@ module FieldTest
|
|
20
20
|
|
21
21
|
# cache results for request
|
22
22
|
@field_test_cache ||= {}
|
23
|
-
|
23
|
+
|
24
|
+
# don't update variant when passed via params
|
25
|
+
@field_test_cache[experiment] ||= params_variant || exp.variant(participants, options)
|
24
26
|
end
|
25
27
|
|
26
28
|
def field_test_converted(experiment, options = {})
|
27
29
|
exp = FieldTest::Experiment.find(experiment)
|
28
30
|
|
29
|
-
participants =
|
31
|
+
participants = FieldTest::Participant.standardize(field_test_participant, options)
|
30
32
|
|
31
33
|
exp.convert(participants, goal: options[:goal])
|
32
34
|
end
|
33
35
|
|
36
|
+
# TODO fetch in single query
|
34
37
|
def field_test_experiments(options = {})
|
35
|
-
participants =
|
36
|
-
memberships = FieldTest::Membership.where(participant: participants).group_by(&:participant)
|
38
|
+
participants = FieldTest::Participant.standardize(field_test_participant, options)
|
37
39
|
experiments = {}
|
38
40
|
participants.each do |participant|
|
39
|
-
|
41
|
+
FieldTest::Membership.where(participant.where_values).each do |membership|
|
40
42
|
experiments[membership.experiment] ||= membership.variant
|
41
43
|
end
|
42
44
|
end
|
43
45
|
experiments
|
44
46
|
end
|
45
|
-
|
46
|
-
def field_test_participants(options = {})
|
47
|
-
participants = []
|
48
|
-
|
49
|
-
if options[:participant]
|
50
|
-
participants << options[:participant]
|
51
|
-
else
|
52
|
-
if respond_to?(:current_user, true) && current_user
|
53
|
-
participants << current_user
|
54
|
-
end
|
55
|
-
|
56
|
-
# controllers and views
|
57
|
-
if try(:request)
|
58
|
-
# use cookie
|
59
|
-
cookie_key = "field_test"
|
60
|
-
|
61
|
-
token = cookies[cookie_key]
|
62
|
-
token = token.gsub(/[^a-z0-9\-]/i, "") if token
|
63
|
-
|
64
|
-
if participants.empty? && !token
|
65
|
-
token = SecureRandom.uuid
|
66
|
-
cookies[cookie_key] = {value: token, expires: 30.days.from_now}
|
67
|
-
end
|
68
|
-
if token
|
69
|
-
participants << token
|
70
|
-
|
71
|
-
# backwards compatibility
|
72
|
-
participants << "cookie:#{token}"
|
73
|
-
end
|
74
|
-
end
|
75
|
-
|
76
|
-
# mailers
|
77
|
-
to = try(:message).try(:to).try(:first)
|
78
|
-
if to
|
79
|
-
participants << to
|
80
|
-
end
|
81
|
-
end
|
82
|
-
|
83
|
-
FieldTest::Participant.standardize(participants)
|
84
|
-
end
|
85
47
|
end
|
86
48
|
end
|
@@ -0,0 +1,20 @@
|
|
1
|
+
module FieldTest
|
2
|
+
module Mailer
|
3
|
+
extend ActiveSupport::Concern
|
4
|
+
include Helpers
|
5
|
+
|
6
|
+
included do
|
7
|
+
helper_method :field_test
|
8
|
+
helper_method :field_test_converted
|
9
|
+
helper_method :field_test_experiments
|
10
|
+
end
|
11
|
+
|
12
|
+
def field_test_participant
|
13
|
+
if @user
|
14
|
+
@user
|
15
|
+
elsif respond_to?(:params) && params
|
16
|
+
params[:user]
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
20
|
+
end
|
@@ -1,7 +1,38 @@
|
|
1
1
|
module FieldTest
|
2
2
|
class Participant
|
3
|
-
|
4
|
-
|
3
|
+
attr_reader :type, :id
|
4
|
+
|
5
|
+
def initialize(object)
|
6
|
+
if object.is_a?(FieldTest::Participant)
|
7
|
+
@type = object.type
|
8
|
+
@id = object.id
|
9
|
+
elsif object.respond_to?(:model_name)
|
10
|
+
@type = object.model_name.name
|
11
|
+
@id = object.id.to_s
|
12
|
+
else
|
13
|
+
@id = object.to_s
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
def participant
|
18
|
+
[type, id].compact.join(":")
|
19
|
+
end
|
20
|
+
|
21
|
+
def where_values
|
22
|
+
if FieldTest.legacy_participants
|
23
|
+
{
|
24
|
+
participant: participant
|
25
|
+
}
|
26
|
+
else
|
27
|
+
{
|
28
|
+
participant_type: type,
|
29
|
+
participant_id: id
|
30
|
+
}
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
def self.standardize(participants, options = {})
|
35
|
+
Array(options[:participant] || participants).compact.map { |v| FieldTest::Participant.new(v) }
|
5
36
|
end
|
6
37
|
end
|
7
38
|
end
|
data/lib/field_test/version.rb
CHANGED
@@ -24,9 +24,7 @@ module FieldTest
|
|
24
24
|
end
|
25
25
|
|
26
26
|
def migration_version
|
27
|
-
|
28
|
-
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
29
|
-
end
|
27
|
+
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
30
28
|
end
|
31
29
|
end
|
32
30
|
end
|
@@ -28,9 +28,7 @@ module FieldTest
|
|
28
28
|
end
|
29
29
|
|
30
30
|
def migration_version
|
31
|
-
|
32
|
-
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
33
|
-
end
|
31
|
+
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
34
32
|
end
|
35
33
|
end
|
36
34
|
end
|
@@ -1,11 +1,9 @@
|
|
1
1
|
class <%= migration_class_name %> < ActiveRecord::Migration<%= migration_version %>
|
2
2
|
def change
|
3
3
|
create_table :field_test_events do |t|
|
4
|
-
t.
|
4
|
+
t.references :field_test_membership
|
5
5
|
t.string :name
|
6
6
|
t.timestamp :created_at
|
7
7
|
end
|
8
|
-
|
9
|
-
add_index :field_test_events, :field_test_membership_id
|
10
8
|
end
|
11
9
|
end
|
@@ -1,15 +1,16 @@
|
|
1
1
|
class <%= migration_class_name %> < ActiveRecord::Migration<%= migration_version %>
|
2
2
|
def change
|
3
3
|
create_table :field_test_memberships do |t|
|
4
|
-
t.string :
|
4
|
+
t.string :participant_type
|
5
|
+
t.string :participant_id
|
5
6
|
t.string :experiment
|
6
7
|
t.string :variant
|
7
8
|
t.timestamp :created_at
|
8
9
|
t.boolean :converted, default: false
|
9
10
|
end
|
10
11
|
|
11
|
-
add_index :field_test_memberships, [:
|
12
|
-
|
12
|
+
add_index :field_test_memberships, [:participant_type, :participant_id, :experiment],
|
13
|
+
unique: true, name: "index_field_test_memberships_on_participant"
|
13
14
|
add_index :field_test_memberships, [:experiment, :created_at]
|
14
15
|
end
|
15
16
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: field_test
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
|
-
bindir:
|
9
|
+
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-06-02 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: railties
|
@@ -16,28 +16,28 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '
|
19
|
+
version: '5'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '
|
26
|
+
version: '5'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: activerecord
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '
|
33
|
+
version: '5'
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version: '
|
40
|
+
version: '5'
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: distribution
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -108,23 +108,62 @@ dependencies:
|
|
108
108
|
- - ">="
|
109
109
|
- !ruby/object:Gem::Version
|
110
110
|
version: '0'
|
111
|
+
- !ruby/object:Gem::Dependency
|
112
|
+
name: combustion
|
113
|
+
requirement: !ruby/object:Gem::Requirement
|
114
|
+
requirements:
|
115
|
+
- - ">="
|
116
|
+
- !ruby/object:Gem::Version
|
117
|
+
version: '0'
|
118
|
+
type: :development
|
119
|
+
prerelease: false
|
120
|
+
version_requirements: !ruby/object:Gem::Requirement
|
121
|
+
requirements:
|
122
|
+
- - ">="
|
123
|
+
- !ruby/object:Gem::Version
|
124
|
+
version: '0'
|
125
|
+
- !ruby/object:Gem::Dependency
|
126
|
+
name: rails
|
127
|
+
requirement: !ruby/object:Gem::Requirement
|
128
|
+
requirements:
|
129
|
+
- - ">="
|
130
|
+
- !ruby/object:Gem::Version
|
131
|
+
version: '0'
|
132
|
+
type: :development
|
133
|
+
prerelease: false
|
134
|
+
version_requirements: !ruby/object:Gem::Requirement
|
135
|
+
requirements:
|
136
|
+
- - ">="
|
137
|
+
- !ruby/object:Gem::Version
|
138
|
+
version: '0'
|
139
|
+
- !ruby/object:Gem::Dependency
|
140
|
+
name: sqlite3
|
141
|
+
requirement: !ruby/object:Gem::Requirement
|
142
|
+
requirements:
|
143
|
+
- - ">="
|
144
|
+
- !ruby/object:Gem::Version
|
145
|
+
version: '0'
|
146
|
+
type: :development
|
147
|
+
prerelease: false
|
148
|
+
version_requirements: !ruby/object:Gem::Requirement
|
149
|
+
requirements:
|
150
|
+
- - ">="
|
151
|
+
- !ruby/object:Gem::Version
|
152
|
+
version: '0'
|
111
153
|
description:
|
112
|
-
email:
|
113
|
-
- andrew@chartkick.com
|
154
|
+
email: andrew@chartkick.com
|
114
155
|
executables: []
|
115
156
|
extensions: []
|
116
157
|
extra_rdoc_files: []
|
117
158
|
files:
|
118
|
-
- ".gitignore"
|
119
159
|
- CHANGELOG.md
|
120
|
-
- Gemfile
|
121
160
|
- LICENSE.txt
|
122
161
|
- README.md
|
123
|
-
- Rakefile
|
124
162
|
- app/controllers/field_test/base_controller.rb
|
125
163
|
- app/controllers/field_test/experiments_controller.rb
|
126
164
|
- app/controllers/field_test/memberships_controller.rb
|
127
165
|
- app/controllers/field_test/participants_controller.rb
|
166
|
+
- app/helpers/field_test/base_helper.rb
|
128
167
|
- app/models/field_test/event.rb
|
129
168
|
- app/models/field_test/membership.rb
|
130
169
|
- app/views/field_test/experiments/_experiments.html.erb
|
@@ -133,12 +172,13 @@ files:
|
|
133
172
|
- app/views/field_test/participants/show.html.erb
|
134
173
|
- app/views/layouts/field_test/application.html.erb
|
135
174
|
- config/routes.rb
|
136
|
-
- field_test.gemspec
|
137
175
|
- lib/field_test.rb
|
138
176
|
- lib/field_test/calculations.rb
|
177
|
+
- lib/field_test/controller.rb
|
139
178
|
- lib/field_test/engine.rb
|
140
179
|
- lib/field_test/experiment.rb
|
141
180
|
- lib/field_test/helpers.rb
|
181
|
+
- lib/field_test/mailer.rb
|
142
182
|
- lib/field_test/participant.rb
|
143
183
|
- lib/field_test/version.rb
|
144
184
|
- lib/generators/field_test/events_generator.rb
|
@@ -147,7 +187,8 @@ files:
|
|
147
187
|
- lib/generators/field_test/templates/events.rb.tt
|
148
188
|
- lib/generators/field_test/templates/memberships.rb.tt
|
149
189
|
homepage: https://github.com/ankane/field_test
|
150
|
-
licenses:
|
190
|
+
licenses:
|
191
|
+
- MIT
|
151
192
|
metadata: {}
|
152
193
|
post_install_message:
|
153
194
|
rdoc_options: []
|
@@ -157,15 +198,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
157
198
|
requirements:
|
158
199
|
- - ">="
|
159
200
|
- !ruby/object:Gem::Version
|
160
|
-
version: '
|
201
|
+
version: '2.4'
|
161
202
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
162
203
|
requirements:
|
163
204
|
- - ">="
|
164
205
|
- !ruby/object:Gem::Version
|
165
206
|
version: '0'
|
166
207
|
requirements: []
|
167
|
-
|
168
|
-
rubygems_version: 2.7.6
|
208
|
+
rubygems_version: 3.0.3
|
169
209
|
signing_key:
|
170
210
|
specification_version: 4
|
171
211
|
summary: A/B testing for Rails
|
data/.gitignore
DELETED
data/Gemfile
DELETED
data/Rakefile
DELETED
data/field_test.gemspec
DELETED
@@ -1,30 +0,0 @@
|
|
1
|
-
# coding: utf-8
|
2
|
-
lib = File.expand_path("../lib", __FILE__)
|
3
|
-
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
-
require "field_test/version"
|
5
|
-
|
6
|
-
Gem::Specification.new do |spec|
|
7
|
-
spec.name = "field_test"
|
8
|
-
spec.version = FieldTest::VERSION
|
9
|
-
spec.authors = ["Andrew Kane"]
|
10
|
-
spec.email = ["andrew@chartkick.com"]
|
11
|
-
|
12
|
-
spec.summary = "A/B testing for Rails"
|
13
|
-
spec.homepage = "https://github.com/ankane/field_test"
|
14
|
-
|
15
|
-
spec.files = `git ls-files -z`.split("\x0").reject do |f|
|
16
|
-
f.match(%r{^(test|spec|features)/})
|
17
|
-
end
|
18
|
-
spec.bindir = "exe"
|
19
|
-
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
|
-
spec.require_paths = ["lib"]
|
21
|
-
|
22
|
-
spec.add_dependency "railties"
|
23
|
-
spec.add_dependency "activerecord"
|
24
|
-
spec.add_dependency "distribution"
|
25
|
-
spec.add_dependency "browser", "~> 2.0"
|
26
|
-
|
27
|
-
spec.add_development_dependency "bundler"
|
28
|
-
spec.add_development_dependency "rake"
|
29
|
-
spec.add_development_dependency "minitest"
|
30
|
-
end
|