field_test 0.2.4 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of field_test might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +14 -0
- data/README.md +184 -21
- data/app/controllers/field_test/memberships_controller.rb +2 -2
- data/app/controllers/field_test/participants_controller.rb +10 -2
- data/app/helpers/field_test/base_helper.rb +12 -0
- data/app/models/field_test/membership.rb +2 -1
- data/app/views/field_test/experiments/_experiments.html.erb +2 -2
- data/app/views/field_test/experiments/show.html.erb +1 -1
- data/config/routes.rb +2 -1
- data/lib/field_test.rb +44 -13
- data/lib/field_test/controller.rb +57 -0
- data/lib/field_test/experiment.rb +56 -43
- data/lib/field_test/helpers.rb +10 -48
- data/lib/field_test/mailer.rb +20 -0
- data/lib/field_test/participant.rb +33 -2
- data/lib/field_test/version.rb +1 -1
- data/lib/generators/field_test/events_generator.rb +1 -3
- data/lib/generators/field_test/install_generator.rb +1 -3
- data/lib/generators/field_test/templates/events.rb.tt +1 -3
- data/lib/generators/field_test/templates/memberships.rb.tt +4 -3
- metadata +57 -17
- data/.gitignore +0 -9
- data/Gemfile +0 -4
- data/Rakefile +0 -10
- data/field_test.gemspec +0 -30
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5303485194bd6850672facf548ba671a2e661e242edfc4a4e3ad90beeed7485d
|
4
|
+
data.tar.gz: c694d25c71464323daf17e7cd9844d8aa8f51eb3d65e0f788e92e0f5998f18b5
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: df92ba63ed13eaec202546d497b2408288b9b4c8901e3a8a25fff96fbb098fa4c3f6acbb9edcd713f94b4ff95540c9542a3c026b4842ca519b76b7f31df2888b
|
7
|
+
data.tar.gz: b9407161a151713987f2db017200ad673cff0eba08f5c114d5787bb322dc4eb87f431fac708b90b42653bed90d34a71a595763cbd32cb107a2a631bcfb7f198c
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,17 @@
|
|
1
|
+
## 0.3.0
|
2
|
+
|
3
|
+
- Added support for native apps
|
4
|
+
- Added `cookies` option
|
5
|
+
- Added `precision` option
|
6
|
+
- Fixed bug in results with multiple goals
|
7
|
+
- Fixed issue where metrics disappeared from dashboard when moving to multiple goals
|
8
|
+
- Dropped support for Rails < 5
|
9
|
+
|
10
|
+
Breaking changes
|
11
|
+
|
12
|
+
- Split out participant id and type
|
13
|
+
- Changed participant logic for emails
|
14
|
+
|
1
15
|
## 0.2.4
|
2
16
|
|
3
17
|
- Fixed `PG::AmbiguousColumn` error
|
data/README.md
CHANGED
@@ -3,11 +3,13 @@
|
|
3
3
|
:maple_leaf: A/B testing for Rails
|
4
4
|
|
5
5
|
- Designed for web and email
|
6
|
-
- Comes with a [
|
6
|
+
- Comes with a [dashboard](https://fieldtest.dokkuapp.com/) to view results and update variants
|
7
7
|
- Seamlessly handles the transition from anonymous visitor to logged in user
|
8
8
|
|
9
9
|
Uses [Bayesian statistics](https://www.evanmiller.org/bayesian-ab-testing.html) to evaluate results so you don’t need to choose a sample size ahead of time.
|
10
10
|
|
11
|
+
[](https://travis-ci.org/ankane/field_test)
|
12
|
+
|
11
13
|
## Installation
|
12
14
|
|
13
15
|
Add this line to your application’s Gemfile:
|
@@ -19,7 +21,8 @@ gem "field_test"
|
|
19
21
|
Run:
|
20
22
|
|
21
23
|
```sh
|
22
|
-
rails
|
24
|
+
rails generate field_test:install
|
25
|
+
rails db:migrate
|
23
26
|
```
|
24
27
|
|
25
28
|
And mount the dashboard in your `config/routes.rb`:
|
@@ -28,7 +31,7 @@ And mount the dashboard in your `config/routes.rb`:
|
|
28
31
|
mount FieldTest::Engine, at: "field_test"
|
29
32
|
```
|
30
33
|
|
31
|
-
Be sure to [secure the dashboard](#security) in production.
|
34
|
+
Be sure to [secure the dashboard](#dashboard-security) in production.
|
32
35
|
|
33
36
|

|
34
37
|
|
@@ -45,12 +48,18 @@ experiments:
|
|
45
48
|
- blue
|
46
49
|
```
|
47
50
|
|
48
|
-
Refer to it in
|
51
|
+
Refer to it in controllers, views, and mailers.
|
49
52
|
|
50
53
|
```ruby
|
51
54
|
button_color = field_test(:button_color)
|
52
55
|
```
|
53
56
|
|
57
|
+
To make testing easier, you can specify a variant with query parameters
|
58
|
+
|
59
|
+
```
|
60
|
+
http://localhost:3000/?field_test[button_color]=green
|
61
|
+
```
|
62
|
+
|
54
63
|
When someone converts, record it with:
|
55
64
|
|
56
65
|
```ruby
|
@@ -67,22 +76,60 @@ experiments:
|
|
67
76
|
|
68
77
|
All calls to `field_test` will now return the winner, and metrics will stop being recorded.
|
69
78
|
|
70
|
-
|
79
|
+
You can get the list of experiments and variants for a user with:
|
80
|
+
|
81
|
+
```ruby
|
82
|
+
field_test_experiments
|
83
|
+
```
|
71
84
|
|
72
|
-
|
85
|
+
## JavaScript and Native Apps
|
73
86
|
|
87
|
+
For JavaScript and native apps, add calls to your normal endpoints.
|
88
|
+
|
89
|
+
```ruby
|
90
|
+
class CheckoutController < ActionController::API
|
91
|
+
def start
|
92
|
+
render json: {button_color: field_test(:button_color)}
|
93
|
+
end
|
94
|
+
|
95
|
+
def finish
|
96
|
+
field_test_converted(:button_color)
|
97
|
+
# ...
|
98
|
+
end
|
99
|
+
end
|
74
100
|
```
|
75
|
-
|
101
|
+
|
102
|
+
For anonymous visitors in native apps, pass a `Field-Test-Visitor` header with a unique identifier.
|
103
|
+
|
104
|
+
## Participants
|
105
|
+
|
106
|
+
Any model or string can be a participant in an experiment.
|
107
|
+
|
108
|
+
For web requests, it uses `current_user` (if it exists) and an anonymous visitor id to determine the participant. Set your own with:
|
109
|
+
|
110
|
+
```ruby
|
111
|
+
class ApplicationController < ActionController::Base
|
112
|
+
def field_test_participant
|
113
|
+
current_company
|
114
|
+
end
|
115
|
+
end
|
76
116
|
```
|
77
117
|
|
78
|
-
|
118
|
+
For mailers, it tries `@user` then `params[:user]` to determine the participant. Set your own with:
|
79
119
|
|
80
120
|
```ruby
|
81
|
-
|
82
|
-
|
121
|
+
class ApplicationMailer < ActionMailer::Base
|
122
|
+
def field_test_participant
|
123
|
+
@company
|
124
|
+
end
|
125
|
+
end
|
83
126
|
```
|
84
127
|
|
85
|
-
You can also
|
128
|
+
You can also manually pass a participant with:
|
129
|
+
|
130
|
+
```ruby
|
131
|
+
field_test(:button_color, participant: company)
|
132
|
+
```
|
86
133
|
|
87
134
|
## Config
|
88
135
|
|
@@ -126,6 +173,14 @@ experiments:
|
|
126
173
|
- 15
|
127
174
|
```
|
128
175
|
|
176
|
+
To help with GDPR compliance, you can switch from cookies to [anonymity sets](https://privacypatterns.org/patterns/Anonymity-set) for anonymous visitors. Visitors with the same IP mask and user agent are grouped together.
|
177
|
+
|
178
|
+
```yml
|
179
|
+
cookies: false
|
180
|
+
```
|
181
|
+
|
182
|
+
## Dashboard Config
|
183
|
+
|
129
184
|
If the dashboard gets slow, you can make it faster with:
|
130
185
|
|
131
186
|
```yml
|
@@ -134,12 +189,19 @@ cache: true
|
|
134
189
|
|
135
190
|
This will use the Rails cache to speed up winning probability calculations.
|
136
191
|
|
137
|
-
|
192
|
+
If you need more precision, set:
|
193
|
+
|
194
|
+
```yml
|
195
|
+
precision: 1
|
196
|
+
```
|
197
|
+
|
198
|
+
## Multiple Goals
|
138
199
|
|
139
200
|
You can set multiple goals for an experiment to track conversions at different parts of the funnel. First, run:
|
140
201
|
|
141
202
|
```sh
|
142
|
-
rails
|
203
|
+
rails generate field_test:events
|
204
|
+
rails db:migrate
|
143
205
|
```
|
144
206
|
|
145
207
|
And add to your config:
|
@@ -162,20 +224,30 @@ The results for all goals will appear on the dashboard.
|
|
162
224
|
|
163
225
|
## Analytics Platforms
|
164
226
|
|
165
|
-
You
|
227
|
+
You may also want to send experiment data as properties to other analytics platforms like [Segment](https://segment.com), [Amplitude](https://amplitude.com), and [Ahoy](https://github.com/ankane/ahoy). Get the list of experiments and variants with:
|
166
228
|
|
167
229
|
```ruby
|
168
230
|
field_test_experiments
|
169
231
|
```
|
170
232
|
|
171
|
-
|
233
|
+
### Ahoy
|
234
|
+
|
235
|
+
You can configure Field Test to use Ahoy’s visitor token instead of creating its own:
|
236
|
+
|
237
|
+
```ruby
|
238
|
+
class ApplicationController < ActionController::Base
|
239
|
+
def field_test_participant
|
240
|
+
[ahoy.user, ahoy.visitor_token]
|
241
|
+
end
|
242
|
+
end
|
243
|
+
```
|
172
244
|
|
173
|
-
## Security
|
245
|
+
## Dashboard Security
|
174
246
|
|
175
247
|
#### Devise
|
176
248
|
|
177
249
|
```ruby
|
178
|
-
authenticate :user, ->
|
250
|
+
authenticate :user, ->(user) { user.admin? } do
|
179
251
|
mount FieldTest::Engine, at: "field_test"
|
180
252
|
end
|
181
253
|
```
|
@@ -189,13 +261,104 @@ ENV["FIELD_TEST_USERNAME"] = "moonrise"
|
|
189
261
|
ENV["FIELD_TEST_PASSWORD"] = "kingdom"
|
190
262
|
```
|
191
263
|
|
192
|
-
##
|
264
|
+
## Reference
|
193
265
|
|
194
|
-
|
266
|
+
Assign a specific variant to a user with:
|
267
|
+
|
268
|
+
```ruby
|
269
|
+
experiment = FieldTest::Experiment.find(:button_color)
|
270
|
+
experiment.variant(participant, variant: "green")
|
271
|
+
```
|
272
|
+
|
273
|
+
You can also change a user’s variant from the dashboard.
|
274
|
+
|
275
|
+
To associate models with field test memberships, use:
|
276
|
+
|
277
|
+
```ruby
|
278
|
+
class User < ApplicationRecord
|
279
|
+
has_many :field_test_memberships, class_name: "FieldTest::Membership", as: :participant
|
280
|
+
end
|
281
|
+
```
|
282
|
+
|
283
|
+
Now you can do:
|
284
|
+
|
285
|
+
```ruby
|
286
|
+
user.field_test_memberships
|
287
|
+
```
|
288
|
+
|
289
|
+
## Upgrading
|
290
|
+
|
291
|
+
### 0.3.0
|
292
|
+
|
293
|
+
Upgrade the gem and add to `config/field_test.yml`:
|
294
|
+
|
295
|
+
```yml
|
296
|
+
legacy_participants: true
|
297
|
+
```
|
298
|
+
|
299
|
+
Also, if you use Field Test in emails, know that the default way participants are determined has changed. Restore the previous way with:
|
300
|
+
|
301
|
+
```ruby
|
302
|
+
class ApplicationMailer < ActionMailer::Base
|
303
|
+
def field_test_participant
|
304
|
+
message.to.first
|
305
|
+
end
|
306
|
+
end
|
307
|
+
```
|
308
|
+
|
309
|
+
We also recommend upgrading participants when you have time.
|
310
|
+
|
311
|
+
#### Upgrading Participants
|
312
|
+
|
313
|
+
Field Test 0.3.0 splits the `field_test_memberships.participant` column into `participant_type` and `participant_id`.
|
314
|
+
|
315
|
+
To upgrade without downtime, create a migration:
|
195
316
|
|
196
|
-
|
317
|
+
```sh
|
318
|
+
rails generate migration upgrade_field_test_participants
|
319
|
+
```
|
320
|
+
|
321
|
+
with:
|
197
322
|
|
198
|
-
|
323
|
+
```ruby
|
324
|
+
class UpgradeFieldTestParticipants < ActiveRecord::Migration[5.2]
|
325
|
+
def change
|
326
|
+
add_column :field_test_memberships, :participant_type, :string
|
327
|
+
add_column :field_test_memberships, :participant_id, :string
|
328
|
+
|
329
|
+
add_index :field_test_memberships, [:participant_type, :participant_id, :experiment],
|
330
|
+
unique: true, name: "index_field_test_memberships_on_participant_and_experiment"
|
331
|
+
end
|
332
|
+
end
|
333
|
+
```
|
334
|
+
|
335
|
+
After you run it, writes will go to both the old and new sets of columns.
|
336
|
+
|
337
|
+
Next, backfill data:
|
338
|
+
|
339
|
+
```ruby
|
340
|
+
FieldTest::Membership.where(participant_id: nil).find_each do |membership|
|
341
|
+
participant = membership.participant
|
342
|
+
|
343
|
+
if participant.include?(":")
|
344
|
+
participant_type, _, participant_id = participant.rpartition(":")
|
345
|
+
participant_type = nil if participant_type == "cookie" # legacy
|
346
|
+
else
|
347
|
+
participant_id = participant
|
348
|
+
end
|
349
|
+
|
350
|
+
membership.update!(
|
351
|
+
participant_type: participant_type,
|
352
|
+
participant_id: participant_id
|
353
|
+
)
|
354
|
+
end
|
355
|
+
```
|
356
|
+
|
357
|
+
Finally, remove `legacy_participants: true` from the config file. Once you confirm it’s working, you can drop the `participant` column (you can rename it first just to be extra safe).
|
358
|
+
|
359
|
+
## Credits
|
360
|
+
|
361
|
+
A huge thanks to [Evan Miller](https://www.evanmiller.org/) for deriving the Bayesian formulas.
|
199
362
|
|
200
363
|
## History
|
201
364
|
|
@@ -2,8 +2,8 @@ module FieldTest
|
|
2
2
|
class MembershipsController < BaseController
|
3
3
|
def update
|
4
4
|
membership = FieldTest::Membership.find(params[:id])
|
5
|
-
membership.
|
6
|
-
|
5
|
+
membership.update!(membership_params)
|
6
|
+
redirect_back(fallback_location: root_path)
|
7
7
|
end
|
8
8
|
|
9
9
|
private
|
@@ -1,9 +1,17 @@
|
|
1
1
|
module FieldTest
|
2
2
|
class ParticipantsController < BaseController
|
3
3
|
def show
|
4
|
-
@participant = params[:id]
|
5
4
|
# TODO better ordering
|
6
|
-
@memberships =
|
5
|
+
@memberships =
|
6
|
+
if FieldTest.legacy_participants
|
7
|
+
@participant = params[:id]
|
8
|
+
FieldTest::Membership.where(participant: @participant).order(:id)
|
9
|
+
else
|
10
|
+
id = params[:id]
|
11
|
+
type = params[:type]
|
12
|
+
@participant = [type, id].compact.join(" ")
|
13
|
+
FieldTest::Membership.where(participant_type: type, participant_id: id).order(:id)
|
14
|
+
end
|
7
15
|
|
8
16
|
@events =
|
9
17
|
if FieldTest.events_supported?
|
@@ -0,0 +1,12 @@
|
|
1
|
+
module FieldTest
|
2
|
+
module BaseHelper
|
3
|
+
def field_test_participant_link(membership)
|
4
|
+
if FieldTest.legacy_participants
|
5
|
+
link_to membership.participant, legacy_participant_path(membership.participant)
|
6
|
+
else
|
7
|
+
text = [membership.participant_type, membership.participant_id].compact.join(" ")
|
8
|
+
link_to text, participant_path(type: membership.participant_type, id: membership.participant_id)
|
9
|
+
end
|
10
|
+
end
|
11
|
+
end
|
12
|
+
end
|
@@ -4,7 +4,8 @@ module FieldTest
|
|
4
4
|
|
5
5
|
has_many :events, class_name: "FieldTest::Event"
|
6
6
|
|
7
|
-
validates :participant, presence: true
|
7
|
+
validates :participant, presence: true, if: -> { FieldTest.legacy_participants }
|
8
|
+
validates :participant_id, presence: true, if: -> { !FieldTest.legacy_participants }
|
8
9
|
validates :experiment, presence: true
|
9
10
|
validates :variant, presence: true
|
10
11
|
end
|
@@ -38,7 +38,7 @@
|
|
38
38
|
<td><%= result[:converted] %></td>
|
39
39
|
<td>
|
40
40
|
<% if result[:conversion_rate] %>
|
41
|
-
<%= (100.0 * result[:conversion_rate]).round %>%
|
41
|
+
<%= (100.0 * result[:conversion_rate]).round(FieldTest.precision) %>%
|
42
42
|
<% else %>
|
43
43
|
-
|
44
44
|
<% end %>
|
@@ -48,7 +48,7 @@
|
|
48
48
|
<% if result[:prob_winning] < 0.01 %>
|
49
49
|
< 1%
|
50
50
|
<% else %>
|
51
|
-
<%= (100.0 * result[:prob_winning]).round %>%
|
51
|
+
<%= (100.0 * result[:prob_winning]).round(FieldTest.precision) %>%
|
52
52
|
<% end %>
|
53
53
|
<% end %>
|
54
54
|
</td>
|
@@ -22,7 +22,7 @@
|
|
22
22
|
<tbody>
|
23
23
|
<% @memberships.each do |membership| %>
|
24
24
|
<tr>
|
25
|
-
<td><%=
|
25
|
+
<td><%= field_test_participant_link(membership) %></td>
|
26
26
|
<td><%= membership.variant %></td>
|
27
27
|
<td>
|
28
28
|
<% converted = false %>
|
data/config/routes.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
FieldTest::Engine.routes.draw do
|
2
2
|
resources :experiments, only: [:show]
|
3
3
|
resources :memberships, only: [:update]
|
4
|
-
get "participants/:id", to: "participants#show", constraints: {id: /.+/}, as: :
|
4
|
+
get "participants/:id", to: "participants#show", constraints: {id: /.+/}, as: :legacy_participant
|
5
|
+
get "participants", to: "participants#show", as: :participant
|
5
6
|
root "experiments#index"
|
6
7
|
end
|
data/lib/field_test.rb
CHANGED
@@ -1,26 +1,36 @@
|
|
1
|
-
|
1
|
+
# dependencies
|
2
2
|
require "active_support"
|
3
|
+
require "browser"
|
4
|
+
require "ipaddr"
|
5
|
+
|
6
|
+
# modules
|
3
7
|
require "field_test/calculations"
|
4
8
|
require "field_test/experiment"
|
5
|
-
require "field_test/engine" if defined?(Rails)
|
6
9
|
require "field_test/helpers"
|
7
10
|
require "field_test/participant"
|
8
11
|
require "field_test/version"
|
9
12
|
|
13
|
+
# integrations
|
14
|
+
require "field_test/engine" if defined?(Rails)
|
15
|
+
|
10
16
|
module FieldTest
|
11
17
|
class Error < StandardError; end
|
12
18
|
class ExperimentNotFound < Error; end
|
13
19
|
class UnknownParticipant < Error; end
|
14
20
|
|
15
|
-
|
16
|
-
|
17
|
-
@config = nil if Rails.env.development?
|
21
|
+
# same as ahoy
|
22
|
+
UUID_NAMESPACE = "a82ae811-5011-45ab-a728-569df7499c5f"
|
18
23
|
|
19
|
-
|
24
|
+
def self.config_path
|
25
|
+
path = defined?(Rails) ? Rails.root : File
|
26
|
+
path.join("config", "field_test.yml")
|
27
|
+
end
|
28
|
+
|
29
|
+
def self.config
|
30
|
+
@config ||= YAML.load(ERB.new(File.read(config_path)).result)
|
20
31
|
end
|
21
32
|
|
22
33
|
def self.exclude_bots?
|
23
|
-
config = self.config # dev performance
|
24
34
|
config["exclude"] && config["exclude"]["bots"]
|
25
35
|
end
|
26
36
|
|
@@ -28,6 +38,18 @@ module FieldTest
|
|
28
38
|
config["cache"]
|
29
39
|
end
|
30
40
|
|
41
|
+
def self.cookies
|
42
|
+
config.key?("cookies") ? config["cookies"] : true
|
43
|
+
end
|
44
|
+
|
45
|
+
def self.legacy_participants
|
46
|
+
config["legacy_participants"]
|
47
|
+
end
|
48
|
+
|
49
|
+
def self.precision
|
50
|
+
config["precision"] || 0
|
51
|
+
end
|
52
|
+
|
31
53
|
def self.events_supported?
|
32
54
|
unless defined?(@events_supported)
|
33
55
|
connection = FieldTest::Membership.connection
|
@@ -41,16 +63,25 @@ module FieldTest
|
|
41
63
|
end
|
42
64
|
@events_supported
|
43
65
|
end
|
44
|
-
end
|
45
66
|
|
46
|
-
|
47
|
-
|
67
|
+
def self.mask_ip(ip)
|
68
|
+
addr = IPAddr.new(ip)
|
69
|
+
if addr.ipv4?
|
70
|
+
# set last octet to 0
|
71
|
+
addr.mask(24).to_s
|
72
|
+
else
|
73
|
+
# set last 80 bits to zeros
|
74
|
+
addr.mask(48).to_s
|
75
|
+
end
|
76
|
+
end
|
48
77
|
end
|
49
78
|
|
50
|
-
ActiveSupport.on_load(:
|
51
|
-
|
79
|
+
ActiveSupport.on_load(:action_controller) do
|
80
|
+
require "field_test/controller"
|
81
|
+
include FieldTest::Controller
|
52
82
|
end
|
53
83
|
|
54
84
|
ActiveSupport.on_load(:action_mailer) do
|
55
|
-
|
85
|
+
require "field_test/mailer"
|
86
|
+
include FieldTest::Mailer
|
56
87
|
end
|
@@ -0,0 +1,57 @@
|
|
1
|
+
module FieldTest
|
2
|
+
module Controller
|
3
|
+
extend ActiveSupport::Concern
|
4
|
+
include Helpers
|
5
|
+
|
6
|
+
included do
|
7
|
+
if respond_to?(:helper_method)
|
8
|
+
helper_method :field_test
|
9
|
+
helper_method :field_test_converted
|
10
|
+
helper_method :field_test_experiments
|
11
|
+
end
|
12
|
+
end
|
13
|
+
|
14
|
+
private
|
15
|
+
|
16
|
+
def field_test_participant
|
17
|
+
participants = []
|
18
|
+
|
19
|
+
if respond_to?(:current_user, true)
|
20
|
+
user = send(:current_user)
|
21
|
+
participants << user if user
|
22
|
+
end
|
23
|
+
|
24
|
+
cookie_key = "field_test"
|
25
|
+
|
26
|
+
if request.headers["Field-Test-Visitor"]
|
27
|
+
token = request.headers["Field-Test-Visitor"]
|
28
|
+
elsif FieldTest.cookies
|
29
|
+
token = cookies[cookie_key]
|
30
|
+
|
31
|
+
if participants.empty? && !token
|
32
|
+
token = SecureRandom.uuid
|
33
|
+
cookies[cookie_key] = {value: token, expires: 30.days.from_now}
|
34
|
+
end
|
35
|
+
else
|
36
|
+
# anonymity set
|
37
|
+
# note: hashing does not conceal input
|
38
|
+
token = Digest::UUID.uuid_v5(FieldTest::UUID_NAMESPACE, ["visitor", FieldTest.mask_ip(request.remote_ip), request.user_agent].join("/"))
|
39
|
+
|
40
|
+
# delete cookie if present
|
41
|
+
cookies.delete(cookie_key) if cookies[cookie_key]
|
42
|
+
end
|
43
|
+
|
44
|
+
# sanitize tokens
|
45
|
+
token = token.gsub(/[^a-z0-9\-]/i, "") if token
|
46
|
+
|
47
|
+
if token.present?
|
48
|
+
participants << token
|
49
|
+
|
50
|
+
# backwards compatibility
|
51
|
+
participants << "cookie:#{token}" if FieldTest.legacy_participants
|
52
|
+
end
|
53
|
+
|
54
|
+
participants
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
@@ -13,6 +13,7 @@ module FieldTest
|
|
13
13
|
@started_at = Time.zone.parse(attributes[:started_at].to_s) if attributes[:started_at]
|
14
14
|
@ended_at = Time.zone.parse(attributes[:ended_at].to_s) if attributes[:ended_at]
|
15
15
|
@goals = attributes[:goals] || ["conversion"]
|
16
|
+
@goals_defined = !attributes[:goals].nil?
|
16
17
|
@use_events = attributes[:use_events]
|
17
18
|
end
|
18
19
|
|
@@ -30,25 +31,18 @@ module FieldTest
|
|
30
31
|
membership.variant ||= weighted_variant
|
31
32
|
end
|
32
33
|
|
34
|
+
participant = participants.first
|
35
|
+
|
33
36
|
# upgrade to preferred participant
|
34
|
-
membership.participant =
|
37
|
+
membership.participant = participant.participant if membership.respond_to?(:participant=)
|
38
|
+
membership.participant_type = participant.type if membership.respond_to?(:participant_type=)
|
39
|
+
membership.participant_id = participant.id if membership.respond_to?(:participant_id=)
|
35
40
|
|
36
41
|
if membership.changed?
|
37
42
|
begin
|
38
43
|
membership.save!
|
39
|
-
|
40
|
-
# log it!
|
41
|
-
info = {
|
42
|
-
experiment: id,
|
43
|
-
variant: membership.variant,
|
44
|
-
participant: membership.participant
|
45
|
-
}.merge(options.slice(:ip, :user_agent))
|
46
|
-
|
47
|
-
# sorta logfmt :)
|
48
|
-
info = info.map { |k, v| v = "\"#{v}\"" if k == :user_agent; "#{k}=#{v}" }.join(" ")
|
49
|
-
Rails.logger.info "[field test] #{info}"
|
50
44
|
rescue ActiveRecord::RecordNotUnique
|
51
|
-
membership = memberships.find_by(participant
|
45
|
+
membership = memberships.find_by(participant.where_values)
|
52
46
|
end
|
53
47
|
end
|
54
48
|
|
@@ -100,16 +94,29 @@ module FieldTest
|
|
100
94
|
relation = relation.where("field_test_memberships.created_at >= ?", started_at) if started_at
|
101
95
|
relation = relation.where("field_test_memberships.created_at <= ?", ended_at) if ended_at
|
102
96
|
|
103
|
-
if use_events?
|
97
|
+
if use_events? && @goals_defined
|
104
98
|
data = {}
|
105
|
-
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
|
99
|
+
|
100
|
+
participated = relation.count
|
101
|
+
|
102
|
+
adapter_name = relation.connection.adapter_name
|
103
|
+
column =
|
104
|
+
if FieldTest.legacy_participants
|
105
|
+
:participant
|
106
|
+
elsif adapter_name =~ /postg/i # postgres
|
107
|
+
"(participant_type, participant_id)"
|
108
|
+
elsif adapter_name =~ /mysql/i
|
109
|
+
"participant_type, participant_id"
|
110
|
+
else
|
111
|
+
# not perfect, but it'll do
|
112
|
+
"COALESCE(participant_type, '') || ':' || participant_id"
|
113
|
+
end
|
114
|
+
|
115
|
+
converted = events.merge(relation).where(field_test_events: {name: goal}).distinct.count(column)
|
116
|
+
|
117
|
+
(participated.keys + converted.keys).uniq.each do |variant|
|
118
|
+
data[[variant, true]] = converted[variant].to_i
|
119
|
+
data[[variant, false]] = participated[variant].to_i - converted[variant].to_i
|
113
120
|
end
|
114
121
|
else
|
115
122
|
data = relation.group(:converted).count
|
@@ -125,6 +132,7 @@ module FieldTest
|
|
125
132
|
conversion_rate: participated > 0 ? converted.to_f / participated : nil
|
126
133
|
}
|
127
134
|
end
|
135
|
+
|
128
136
|
case variants.size
|
129
137
|
when 1, 2, 3
|
130
138
|
total = 0.0
|
@@ -189,32 +197,37 @@ module FieldTest
|
|
189
197
|
|
190
198
|
private
|
191
199
|
|
192
|
-
|
193
|
-
|
194
|
-
|
200
|
+
def check_participants(participants)
|
201
|
+
raise FieldTest::UnknownParticipant, "Use the :participant option to specify a participant" if participants.empty?
|
202
|
+
end
|
195
203
|
|
196
|
-
|
197
|
-
|
198
|
-
|
204
|
+
# TODO fetch in single query
|
205
|
+
def membership_for(participants)
|
206
|
+
membership = nil
|
207
|
+
participants.each do |participant|
|
208
|
+
membership = self.memberships.find_by(participant.where_values)
|
209
|
+
break if membership
|
199
210
|
end
|
211
|
+
membership
|
212
|
+
end
|
200
213
|
|
201
|
-
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
208
|
-
end
|
209
|
-
variants.last
|
214
|
+
def weighted_variant
|
215
|
+
total = weights.sum.to_f
|
216
|
+
pick = rand
|
217
|
+
n = 0
|
218
|
+
weights.map { |w| w / total }.each_with_index do |w, i|
|
219
|
+
n += w
|
220
|
+
return variants[i] if n >= pick
|
210
221
|
end
|
222
|
+
variants.last
|
223
|
+
end
|
211
224
|
|
212
|
-
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
end
|
225
|
+
def cache_fetch(key)
|
226
|
+
if FieldTest.cache
|
227
|
+
Rails.cache.fetch(key.join("/")) { yield }
|
228
|
+
else
|
229
|
+
yield
|
218
230
|
end
|
231
|
+
end
|
219
232
|
end
|
220
233
|
end
|
data/lib/field_test/helpers.rb
CHANGED
@@ -3,11 +3,11 @@ module FieldTest
|
|
3
3
|
def field_test(experiment, options = {})
|
4
4
|
exp = FieldTest::Experiment.find(experiment)
|
5
5
|
|
6
|
-
participants =
|
6
|
+
participants = FieldTest::Participant.standardize(field_test_participant, options)
|
7
7
|
|
8
8
|
if try(:request)
|
9
|
-
if params[:field_test] && params[:field_test][experiment]
|
10
|
-
|
9
|
+
if !options[:variant] && params[:field_test] && params[:field_test][experiment]
|
10
|
+
params_variant = params[:field_test][experiment]
|
11
11
|
end
|
12
12
|
|
13
13
|
if FieldTest.exclude_bots?
|
@@ -20,67 +20,29 @@ module FieldTest
|
|
20
20
|
|
21
21
|
# cache results for request
|
22
22
|
@field_test_cache ||= {}
|
23
|
-
|
23
|
+
|
24
|
+
# don't update variant when passed via params
|
25
|
+
@field_test_cache[experiment] ||= params_variant || exp.variant(participants, options)
|
24
26
|
end
|
25
27
|
|
26
28
|
def field_test_converted(experiment, options = {})
|
27
29
|
exp = FieldTest::Experiment.find(experiment)
|
28
30
|
|
29
|
-
participants =
|
31
|
+
participants = FieldTest::Participant.standardize(field_test_participant, options)
|
30
32
|
|
31
33
|
exp.convert(participants, goal: options[:goal])
|
32
34
|
end
|
33
35
|
|
36
|
+
# TODO fetch in single query
|
34
37
|
def field_test_experiments(options = {})
|
35
|
-
participants =
|
36
|
-
memberships = FieldTest::Membership.where(participant: participants).group_by(&:participant)
|
38
|
+
participants = FieldTest::Participant.standardize(field_test_participant, options)
|
37
39
|
experiments = {}
|
38
40
|
participants.each do |participant|
|
39
|
-
|
41
|
+
FieldTest::Membership.where(participant.where_values).each do |membership|
|
40
42
|
experiments[membership.experiment] ||= membership.variant
|
41
43
|
end
|
42
44
|
end
|
43
45
|
experiments
|
44
46
|
end
|
45
|
-
|
46
|
-
def field_test_participants(options = {})
|
47
|
-
participants = []
|
48
|
-
|
49
|
-
if options[:participant]
|
50
|
-
participants << options[:participant]
|
51
|
-
else
|
52
|
-
if respond_to?(:current_user, true) && current_user
|
53
|
-
participants << current_user
|
54
|
-
end
|
55
|
-
|
56
|
-
# controllers and views
|
57
|
-
if try(:request)
|
58
|
-
# use cookie
|
59
|
-
cookie_key = "field_test"
|
60
|
-
|
61
|
-
token = cookies[cookie_key]
|
62
|
-
token = token.gsub(/[^a-z0-9\-]/i, "") if token
|
63
|
-
|
64
|
-
if participants.empty? && !token
|
65
|
-
token = SecureRandom.uuid
|
66
|
-
cookies[cookie_key] = {value: token, expires: 30.days.from_now}
|
67
|
-
end
|
68
|
-
if token
|
69
|
-
participants << token
|
70
|
-
|
71
|
-
# backwards compatibility
|
72
|
-
participants << "cookie:#{token}"
|
73
|
-
end
|
74
|
-
end
|
75
|
-
|
76
|
-
# mailers
|
77
|
-
to = try(:message).try(:to).try(:first)
|
78
|
-
if to
|
79
|
-
participants << to
|
80
|
-
end
|
81
|
-
end
|
82
|
-
|
83
|
-
FieldTest::Participant.standardize(participants)
|
84
|
-
end
|
85
47
|
end
|
86
48
|
end
|
@@ -0,0 +1,20 @@
|
|
1
|
+
module FieldTest
|
2
|
+
module Mailer
|
3
|
+
extend ActiveSupport::Concern
|
4
|
+
include Helpers
|
5
|
+
|
6
|
+
included do
|
7
|
+
helper_method :field_test
|
8
|
+
helper_method :field_test_converted
|
9
|
+
helper_method :field_test_experiments
|
10
|
+
end
|
11
|
+
|
12
|
+
def field_test_participant
|
13
|
+
if @user
|
14
|
+
@user
|
15
|
+
elsif respond_to?(:params) && params
|
16
|
+
params[:user]
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
20
|
+
end
|
@@ -1,7 +1,38 @@
|
|
1
1
|
module FieldTest
|
2
2
|
class Participant
|
3
|
-
|
4
|
-
|
3
|
+
attr_reader :type, :id
|
4
|
+
|
5
|
+
def initialize(object)
|
6
|
+
if object.is_a?(FieldTest::Participant)
|
7
|
+
@type = object.type
|
8
|
+
@id = object.id
|
9
|
+
elsif object.respond_to?(:model_name)
|
10
|
+
@type = object.model_name.name
|
11
|
+
@id = object.id.to_s
|
12
|
+
else
|
13
|
+
@id = object.to_s
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
def participant
|
18
|
+
[type, id].compact.join(":")
|
19
|
+
end
|
20
|
+
|
21
|
+
def where_values
|
22
|
+
if FieldTest.legacy_participants
|
23
|
+
{
|
24
|
+
participant: participant
|
25
|
+
}
|
26
|
+
else
|
27
|
+
{
|
28
|
+
participant_type: type,
|
29
|
+
participant_id: id
|
30
|
+
}
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
def self.standardize(participants, options = {})
|
35
|
+
Array(options[:participant] || participants).compact.map { |v| FieldTest::Participant.new(v) }
|
5
36
|
end
|
6
37
|
end
|
7
38
|
end
|
data/lib/field_test/version.rb
CHANGED
@@ -24,9 +24,7 @@ module FieldTest
|
|
24
24
|
end
|
25
25
|
|
26
26
|
def migration_version
|
27
|
-
|
28
|
-
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
29
|
-
end
|
27
|
+
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
30
28
|
end
|
31
29
|
end
|
32
30
|
end
|
@@ -28,9 +28,7 @@ module FieldTest
|
|
28
28
|
end
|
29
29
|
|
30
30
|
def migration_version
|
31
|
-
|
32
|
-
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
33
|
-
end
|
31
|
+
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
34
32
|
end
|
35
33
|
end
|
36
34
|
end
|
@@ -1,11 +1,9 @@
|
|
1
1
|
class <%= migration_class_name %> < ActiveRecord::Migration<%= migration_version %>
|
2
2
|
def change
|
3
3
|
create_table :field_test_events do |t|
|
4
|
-
t.
|
4
|
+
t.references :field_test_membership
|
5
5
|
t.string :name
|
6
6
|
t.timestamp :created_at
|
7
7
|
end
|
8
|
-
|
9
|
-
add_index :field_test_events, :field_test_membership_id
|
10
8
|
end
|
11
9
|
end
|
@@ -1,15 +1,16 @@
|
|
1
1
|
class <%= migration_class_name %> < ActiveRecord::Migration<%= migration_version %>
|
2
2
|
def change
|
3
3
|
create_table :field_test_memberships do |t|
|
4
|
-
t.string :
|
4
|
+
t.string :participant_type
|
5
|
+
t.string :participant_id
|
5
6
|
t.string :experiment
|
6
7
|
t.string :variant
|
7
8
|
t.timestamp :created_at
|
8
9
|
t.boolean :converted, default: false
|
9
10
|
end
|
10
11
|
|
11
|
-
add_index :field_test_memberships, [:
|
12
|
-
|
12
|
+
add_index :field_test_memberships, [:participant_type, :participant_id, :experiment],
|
13
|
+
unique: true, name: "index_field_test_memberships_on_participant"
|
13
14
|
add_index :field_test_memberships, [:experiment, :created_at]
|
14
15
|
end
|
15
16
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: field_test
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
|
-
bindir:
|
9
|
+
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-06-02 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: railties
|
@@ -16,28 +16,28 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '
|
19
|
+
version: '5'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '
|
26
|
+
version: '5'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: activerecord
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '
|
33
|
+
version: '5'
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version: '
|
40
|
+
version: '5'
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: distribution
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -108,23 +108,62 @@ dependencies:
|
|
108
108
|
- - ">="
|
109
109
|
- !ruby/object:Gem::Version
|
110
110
|
version: '0'
|
111
|
+
- !ruby/object:Gem::Dependency
|
112
|
+
name: combustion
|
113
|
+
requirement: !ruby/object:Gem::Requirement
|
114
|
+
requirements:
|
115
|
+
- - ">="
|
116
|
+
- !ruby/object:Gem::Version
|
117
|
+
version: '0'
|
118
|
+
type: :development
|
119
|
+
prerelease: false
|
120
|
+
version_requirements: !ruby/object:Gem::Requirement
|
121
|
+
requirements:
|
122
|
+
- - ">="
|
123
|
+
- !ruby/object:Gem::Version
|
124
|
+
version: '0'
|
125
|
+
- !ruby/object:Gem::Dependency
|
126
|
+
name: rails
|
127
|
+
requirement: !ruby/object:Gem::Requirement
|
128
|
+
requirements:
|
129
|
+
- - ">="
|
130
|
+
- !ruby/object:Gem::Version
|
131
|
+
version: '0'
|
132
|
+
type: :development
|
133
|
+
prerelease: false
|
134
|
+
version_requirements: !ruby/object:Gem::Requirement
|
135
|
+
requirements:
|
136
|
+
- - ">="
|
137
|
+
- !ruby/object:Gem::Version
|
138
|
+
version: '0'
|
139
|
+
- !ruby/object:Gem::Dependency
|
140
|
+
name: sqlite3
|
141
|
+
requirement: !ruby/object:Gem::Requirement
|
142
|
+
requirements:
|
143
|
+
- - ">="
|
144
|
+
- !ruby/object:Gem::Version
|
145
|
+
version: '0'
|
146
|
+
type: :development
|
147
|
+
prerelease: false
|
148
|
+
version_requirements: !ruby/object:Gem::Requirement
|
149
|
+
requirements:
|
150
|
+
- - ">="
|
151
|
+
- !ruby/object:Gem::Version
|
152
|
+
version: '0'
|
111
153
|
description:
|
112
|
-
email:
|
113
|
-
- andrew@chartkick.com
|
154
|
+
email: andrew@chartkick.com
|
114
155
|
executables: []
|
115
156
|
extensions: []
|
116
157
|
extra_rdoc_files: []
|
117
158
|
files:
|
118
|
-
- ".gitignore"
|
119
159
|
- CHANGELOG.md
|
120
|
-
- Gemfile
|
121
160
|
- LICENSE.txt
|
122
161
|
- README.md
|
123
|
-
- Rakefile
|
124
162
|
- app/controllers/field_test/base_controller.rb
|
125
163
|
- app/controllers/field_test/experiments_controller.rb
|
126
164
|
- app/controllers/field_test/memberships_controller.rb
|
127
165
|
- app/controllers/field_test/participants_controller.rb
|
166
|
+
- app/helpers/field_test/base_helper.rb
|
128
167
|
- app/models/field_test/event.rb
|
129
168
|
- app/models/field_test/membership.rb
|
130
169
|
- app/views/field_test/experiments/_experiments.html.erb
|
@@ -133,12 +172,13 @@ files:
|
|
133
172
|
- app/views/field_test/participants/show.html.erb
|
134
173
|
- app/views/layouts/field_test/application.html.erb
|
135
174
|
- config/routes.rb
|
136
|
-
- field_test.gemspec
|
137
175
|
- lib/field_test.rb
|
138
176
|
- lib/field_test/calculations.rb
|
177
|
+
- lib/field_test/controller.rb
|
139
178
|
- lib/field_test/engine.rb
|
140
179
|
- lib/field_test/experiment.rb
|
141
180
|
- lib/field_test/helpers.rb
|
181
|
+
- lib/field_test/mailer.rb
|
142
182
|
- lib/field_test/participant.rb
|
143
183
|
- lib/field_test/version.rb
|
144
184
|
- lib/generators/field_test/events_generator.rb
|
@@ -147,7 +187,8 @@ files:
|
|
147
187
|
- lib/generators/field_test/templates/events.rb.tt
|
148
188
|
- lib/generators/field_test/templates/memberships.rb.tt
|
149
189
|
homepage: https://github.com/ankane/field_test
|
150
|
-
licenses:
|
190
|
+
licenses:
|
191
|
+
- MIT
|
151
192
|
metadata: {}
|
152
193
|
post_install_message:
|
153
194
|
rdoc_options: []
|
@@ -157,15 +198,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
157
198
|
requirements:
|
158
199
|
- - ">="
|
159
200
|
- !ruby/object:Gem::Version
|
160
|
-
version: '
|
201
|
+
version: '2.4'
|
161
202
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
162
203
|
requirements:
|
163
204
|
- - ">="
|
164
205
|
- !ruby/object:Gem::Version
|
165
206
|
version: '0'
|
166
207
|
requirements: []
|
167
|
-
|
168
|
-
rubygems_version: 2.7.6
|
208
|
+
rubygems_version: 3.0.3
|
169
209
|
signing_key:
|
170
210
|
specification_version: 4
|
171
211
|
summary: A/B testing for Rails
|
data/.gitignore
DELETED
data/Gemfile
DELETED
data/Rakefile
DELETED
data/field_test.gemspec
DELETED
@@ -1,30 +0,0 @@
|
|
1
|
-
# coding: utf-8
|
2
|
-
lib = File.expand_path("../lib", __FILE__)
|
3
|
-
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
-
require "field_test/version"
|
5
|
-
|
6
|
-
Gem::Specification.new do |spec|
|
7
|
-
spec.name = "field_test"
|
8
|
-
spec.version = FieldTest::VERSION
|
9
|
-
spec.authors = ["Andrew Kane"]
|
10
|
-
spec.email = ["andrew@chartkick.com"]
|
11
|
-
|
12
|
-
spec.summary = "A/B testing for Rails"
|
13
|
-
spec.homepage = "https://github.com/ankane/field_test"
|
14
|
-
|
15
|
-
spec.files = `git ls-files -z`.split("\x0").reject do |f|
|
16
|
-
f.match(%r{^(test|spec|features)/})
|
17
|
-
end
|
18
|
-
spec.bindir = "exe"
|
19
|
-
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
|
-
spec.require_paths = ["lib"]
|
21
|
-
|
22
|
-
spec.add_dependency "railties"
|
23
|
-
spec.add_dependency "activerecord"
|
24
|
-
spec.add_dependency "distribution"
|
25
|
-
spec.add_dependency "browser", "~> 2.0"
|
26
|
-
|
27
|
-
spec.add_development_dependency "bundler"
|
28
|
-
spec.add_development_dependency "rake"
|
29
|
-
spec.add_development_dependency "minitest"
|
30
|
-
end
|