field_test 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 177ccc428012c8862f9d798b6cf58629d41d54bb
4
- data.tar.gz: 5b57269ef51ea55f5f10ff280d3c8aac895db01d
3
+ metadata.gz: f8bbd74bb28099c186bacc0ea7b195c7d89a7e35
4
+ data.tar.gz: e87355a520385d34192ab200b80411a4bbc70310
5
5
  SHA512:
6
- metadata.gz: e014861b6912b47d5320511ebfbc0a5a8abd9470fc6a2f8b4c25f0c7d5cd07ff217836cb3d6f6bcda44a1d615fb3bf782ebdfb3161c49a292df251b581717ed4
7
- data.tar.gz: d12c7f9d920bd6aa0df5c5aa15876b31c9693cca38f3fd628013ffa3f72e2323852405ea50d3fc9719509a78a8abfdaa2b4a4316e899fb37b692c7149acdfaf4
6
+ metadata.gz: 6f717922757f35cd62c835beb58bde3bce74fb326ca1965ddb5bbdd03811d1433fd99f1e677738b52d12f3c9158919faf832f99427d507ce5f23423b03757ccc
7
+ data.tar.gz: 5912035f52bfb111970e1fcf3f6bf73a9c4dbe55334e04b39a1ba9e456481ffe23f5838974989f45f560a16960b6f2a835bce2a0603c34991167fe39caa75230
data/CHANGELOG.md CHANGED
@@ -1,3 +1,8 @@
1
+ ## 0.1.2
2
+
3
+ - Exclude bots
4
+ - Mailer improvements
5
+
1
6
  ## 0.1.1
2
7
 
3
8
  - Added basic web UI
data/README.md CHANGED
@@ -16,7 +16,7 @@ Add this line to your application’s Gemfile:
16
16
  gem 'field_test'
17
17
  ```
18
18
 
19
- And run:
19
+ Run:
20
20
 
21
21
  ```sh
22
22
  rails g field_test:install
@@ -30,6 +30,8 @@ mount FieldTest::Engine, at: "field_test"
30
30
 
31
31
  Be sure to [secure the dashboard](#security) in production.
32
32
 
33
+ ![Screenshot](https://ankane.github.io/field_test/screenshot5.png)
34
+
33
35
  ## Getting Started
34
36
 
35
37
  Add an experiment to `config/field_test.yml`.
@@ -70,7 +72,7 @@ experiments:
70
72
  winner: red
71
73
  ```
72
74
 
73
- All calls to `field_test` will now return the winner.
75
+ All calls to `field_test` will now return the winner, and metrics will stop being recorded.
74
76
 
75
77
  ## Features
76
78
 
@@ -80,26 +82,67 @@ You can specify a variant with query parameters to make testing easier
80
82
  http://localhost:3000/?field_test[button_color]=red
81
83
  ```
82
84
 
83
- For mailers, you need to specify a participant:
85
+ Assign a specific variant to a user with:
86
+
87
+ ```ruby
88
+ experiment = FieldTest::Experiment.find(:button_color)
89
+ experiment.variant(user, variant: "red")
90
+ ```
91
+
92
+ Specify a participant with:
84
93
 
85
94
  ```ruby
86
95
  field_test(:button_color, participant: "test@example.org")
87
96
  ```
88
97
 
98
+ You can pass an object as well.
99
+
100
+ ```ruby
101
+ field_test(:button_color, participant: user)
102
+ ```
103
+
104
+ ## Config
105
+
106
+ By default, bots are returned the first variant and excluded from metrics. Change this with:
107
+
108
+ ```yml
109
+ exclude:
110
+ bots: false
111
+ ```
112
+
89
113
  Keep track of when experiments started and ended.
90
114
 
91
115
  ```yml
92
116
  experiments:
93
- button_colors:
117
+ button_color:
94
118
  started_at: 2016-12-01 14:00:00
95
119
  ended_at: 2016-12-08 14:00:00
96
120
  ```
97
121
 
122
+ By default, variants are given the same probability of being selected. Change this with:
123
+
124
+ ```yml
125
+ experiments:
126
+ button_color:
127
+ variants:
128
+ - red
129
+ - blue
130
+ weights:
131
+ - 90
132
+ - 10
133
+ ```
134
+
98
135
  ## Funnels
99
136
 
100
137
  For advanced funnels, we recommend an analytics platform like [Ahoy](https://github.com/ankane/ahoy) or [Mixpanel](https://mixpanel.com/).
101
138
 
102
- You can pass experiments and variants as properties.
139
+ You can use:
140
+
141
+ ```ruby
142
+ field_test_experiments
143
+ ```
144
+
145
+ to get all experiments and variants for a participant, and pass them as properties.
103
146
 
104
147
  ## Security
105
148
 
@@ -108,8 +151,8 @@ You can pass experiments and variants as properties.
108
151
  Set the following variables in your environment or an initializer.
109
152
 
110
153
  ```ruby
111
- ENV["FIELD_TEST_USERNAME"] = "link"
112
- ENV["FIELD_TEST_PASSWORD"] = "hyrule"
154
+ ENV["FIELD_TEST_USERNAME"] = "moonrise"
155
+ ENV["FIELD_TEST_PASSWORD"] = "kingdom"
113
156
  ```
114
157
 
115
158
  #### Devise
@@ -124,10 +167,6 @@ end
124
167
 
125
168
  A huge thanks to [Evan Miller](http://www.evanmiller.org/) for deriving the Bayesian formulas.
126
169
 
127
- ## TODO
128
-
129
- - Exclude bots
130
-
131
170
  ## History
132
171
 
133
172
  View the [changelog](https://github.com/ankane/field_test/blob/master/CHANGELOG.md)
@@ -5,7 +5,7 @@
5
5
  <table>
6
6
  <thead>
7
7
  <tr>
8
- <th>Name</th>
8
+ <th>Variant</th>
9
9
  <th style="width: 20%;">Participants</th>
10
10
  <th style="width: 20%;">Conversions</th>
11
11
  <th style="width: 20%;">Conversion Rate</th>
data/field_test.gemspec CHANGED
@@ -22,6 +22,7 @@ Gem::Specification.new do |spec|
22
22
  spec.add_dependency "railties"
23
23
  spec.add_dependency "activerecord"
24
24
  spec.add_dependency "distribution"
25
+ spec.add_dependency "browser", "~> 2.0"
25
26
 
26
27
  spec.add_development_dependency "bundler"
27
28
  spec.add_development_dependency "rake"
data/lib/field_test.rb CHANGED
@@ -1,4 +1,5 @@
1
1
  require "distribution/math_extension"
2
+ require "browser"
2
3
  require "field_test/experiment"
3
4
  require "field_test/engine"
4
5
  require "field_test/helpers"
@@ -7,14 +8,20 @@ require "field_test/version"
7
8
 
8
9
  module FieldTest
9
10
  class Error < StandardError; end
10
- class ExperimentNotFound < Error; end
11
+ class ExperimentNotFound < Error; end
12
+ class UnknownParticipant < Error; end
11
13
 
12
- def self.config
14
+ def self.config
13
15
  # reload in dev
14
16
  @config = nil if Rails.env.development?
15
17
 
16
18
  @config ||= YAML.load(ERB.new(File.read("config/field_test.yml")).result)
17
19
  end
20
+
21
+ def self.exclude_bots?
22
+ config = self.config # dev performance
23
+ config["exclude"] && config["exclude"]["bots"]
24
+ end
18
25
  end
19
26
 
20
27
  ActiveSupport.on_load(:action_controller) do
@@ -1,12 +1,13 @@
1
1
  module FieldTest
2
2
  class Experiment
3
- attr_reader :id, :name, :variants, :winner, :started_at, :ended_at
3
+ attr_reader :id, :name, :variants, :weights, :winner, :started_at, :ended_at
4
4
 
5
5
  def initialize(attributes)
6
6
  attributes = attributes.symbolize_keys
7
7
  @id = attributes[:id]
8
8
  @name = attributes[:name] || @id.to_s.titleize
9
9
  @variants = attributes[:variants]
10
+ @weights = @variants.size.times.map { |i| attributes[:weights].to_a[i] || 1 }
10
11
  @winner = attributes[:winner]
11
12
  @started_at = Time.zone.parse(attributes[:started_at].to_s) if attributes[:started_at]
12
13
  @ended_at = Time.zone.parse(attributes[:ended_at].to_s) if attributes[:ended_at]
@@ -17,12 +18,13 @@ module FieldTest
17
18
  return variants.first if options[:exclude]
18
19
 
19
20
  participants = FieldTest::Participant.standardize(participants)
21
+ check_participants(participants)
20
22
  membership = membership_for(participants) || FieldTest::Membership.new(experiment: id)
21
23
 
22
24
  if options[:variant] && variants.include?(options[:variant])
23
25
  membership.variant = options[:variant]
24
26
  else
25
- membership.variant ||= variants.sample
27
+ membership.variant ||= weighted_variant
26
28
  end
27
29
 
28
30
  # upgrade to preferred participant
@@ -41,6 +43,7 @@ module FieldTest
41
43
 
42
44
  def convert(participants)
43
45
  participants = FieldTest::Participant.standardize(participants)
46
+ check_participants(participants)
44
47
  membership = membership_for(participants)
45
48
 
46
49
  if membership
@@ -117,11 +120,26 @@ module FieldTest
117
120
 
118
121
  private
119
122
 
123
+ def check_participants(participants)
124
+ raise FieldTest::UnknownParticipant, "Use the :participant option to specify a participant" if participants.empty?
125
+ end
126
+
120
127
  def membership_for(participants)
121
128
  memberships = self.memberships.where(participant: participants).index_by(&:participant)
122
129
  participants.map { |part| memberships[part] }.compact.first
123
130
  end
124
131
 
132
+ def weighted_variant
133
+ total = weights.sum.to_f
134
+ pick = rand
135
+ n = 0
136
+ weights.map { |w| w / total }.each_with_index do |w, i|
137
+ n += w
138
+ return variants[i] if n >= pick
139
+ end
140
+ variants.last
141
+ end
142
+
125
143
  # formula from
126
144
  # http://www.evanmiller.org/bayesian-ab-testing.html
127
145
  def prob_b_beats_a(alpha_a, beta_a, alpha_b, beta_b)
@@ -9,6 +9,10 @@ module FieldTest
9
9
  if params[:field_test] && params[:field_test][experiment]
10
10
  options[:variant] ||= params[:field_test][experiment]
11
11
  end
12
+
13
+ if FieldTest.exclude_bots?
14
+ options[:exclude] = Browser.new(request.user_agent).bot?
15
+ end
12
16
  end
13
17
 
14
18
  # cache results for request
@@ -29,7 +33,7 @@ module FieldTest
29
33
  memberships = FieldTest::Membership.where(participant: participants).group_by(&:participant)
30
34
  experiments = {}
31
35
  participants.each do |participant|
32
- memberships[participant].each do |membership|
36
+ memberships[participant].to_a.each do |membership|
33
37
  experiments[membership.experiment] ||= membership.variant
34
38
  end
35
39
  end
@@ -46,6 +50,7 @@ module FieldTest
46
50
  participants << current_user
47
51
  end
48
52
 
53
+ # controllers and views
49
54
  if try(:request)
50
55
  # use cookie
51
56
  cookie_key = "field_test"
@@ -58,6 +63,12 @@ module FieldTest
58
63
  participants << "cookie:#{token.gsub(/[^a-z0-9\-]/i, "")}"
59
64
  end
60
65
  end
66
+
67
+ # mailers
68
+ to = try(:message).try(:to).try(:first)
69
+ if to
70
+ participants << to
71
+ end
61
72
  end
62
73
 
63
74
  FieldTest::Participant.standardize(participants)
@@ -1,3 +1,3 @@
1
1
  module FieldTest
2
- VERSION = "0.1.1"
2
+ VERSION = "0.1.2"
3
3
  end
@@ -5,3 +5,6 @@ experiments:
5
5
  - green
6
6
  - blue
7
7
  # winner: green
8
+
9
+ exclude:
10
+ bots: true
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: field_test
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.1.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew Kane
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2016-12-15 00:00:00.000000000 Z
11
+ date: 2016-12-17 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: railties
@@ -52,6 +52,20 @@ dependencies:
52
52
  - - ">="
53
53
  - !ruby/object:Gem::Version
54
54
  version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: browser
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '2.0'
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '2.0'
55
69
  - !ruby/object:Gem::Dependency
56
70
  name: bundler
57
71
  requirement: !ruby/object:Gem::Requirement