verdict 0.5.0 → 0.6.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: e5d867e9231108358536850b04548a9a43594d79
4
- data.tar.gz: 272e5fb808e0083732fd5516660812ce92ade4c9
3
+ metadata.gz: 5ae06081f05919820bf8606d93ea3b673dfaf8a7
4
+ data.tar.gz: da4b5f7de2203ecb7c5a99d1f3140b7e596dac1d
5
5
  SHA512:
6
- metadata.gz: 4883a16667083e18a41955708c0624abf6caa0360df42b7a58968b528a4a04cef3aee013733f4e046ce37994b437834a14182429324c454313f3061e68cf6767
7
- data.tar.gz: a62aba796b913ca660d039881149edfa5cbda21f240683b9fe66e28441058e4f0ce6e0c49dcd2e7af02f224005824d91c5b72731963cda6f6e3f52871c56402d
6
+ metadata.gz: d0b692f5c3bd768152e9e09d09ff0ede4eaa01788fd452f762531d8682660a197211b5c3f9e49dc988344d776f7f57c86978d6715a351c4605db9d521ca79dae
7
+ data.tar.gz: 4ae1d2cd72f869b3b423427791a37fc8316e7d68462f6f051f42631fc208fbc59ee988cdc770974a8efec506f5d5e89f49eb4dd73514232f3951a595c1190c55
@@ -3,12 +3,11 @@ cache: bundler
3
3
  before_install: gem update bundler
4
4
  script: bundle exec rake
5
5
  rvm:
6
- - 1.9.3
7
- - 2.0.0
8
- - 2.1.1
6
+ - 2.0
7
+ - 2.1
8
+ - 2.3.3
9
9
  - ruby-head
10
- - rbx-2
11
- - jruby-19mode
10
+ - jruby
12
11
  matrix:
13
12
  allow_failures:
14
13
  - rvm: ruby-head
@@ -0,0 +1,39 @@
1
+ ## v0.6.0
2
+ **This version has breaking changes**
3
+
4
+ ### Verdict::Experiment
5
+ * replaced following public methods that took a `subject_identifier` with an equivalent method which takes a `subject`
6
+
7
+ | old method | new method |
8
+ | --------------------------------------------------------------- | ---------------------------------------- |
9
+ | `lookup_assignment_for_identifier(subject_identifier)` | `lookup(subject)` |
10
+ | `assign_manually_by_identifier(subject_identifier, group)` | `assign_manually(subject, group)` |
11
+ | `disqualify_manually_by_identifier(subject_identifier)` | `disqualify_manually(subject)` |
12
+ | `remove_subject_assignment_by_identifier(subject_identifier)` | `remove_subject_assignment(subject)` |
13
+ | `fetch_assignment(subject_identifier)` | `lookup(subject)` |
14
+
15
+ * Changed the following methods to take a `subject` instead of `subject_identifier`
16
+ * `subject_assignment(subject, group, originally_created_at = nil, temporary = false)`
17
+ * `subject_conversion(subject, goal, created_at = Time.now.utc)`
18
+
19
+ #### Improved Testability
20
+ `Verdict::Experiment#subject_qualifies?(subject, context = nil)` is now public, so it's easier to test
21
+ the qualification logic for your experiments.
22
+
23
+ ### Verdict::BaseStorage
24
+ * `BaseStorage`'s public methods now take a `subject`, instead of a `subject_identifier`. They fetch the `subject_identifier` using the `Experiment#subject_identifier(subject)` method. Existing storages **will still work normally**.
25
+ * The basic `#get`,`#set`, and `#remove` methods are now protected.
26
+
27
+ ### Verdict::Assignment
28
+ * `#initialize` now takes a `subject` instead of a `subject_identifier`, the new signature is `initialize(experiment, subject, group, originally_created_at, temporary = false)`
29
+
30
+ ### Verdict::Conversion
31
+ * `#initialize` now takes a `subject` instead of a `subject_identifier`, the new signature is `initialize(experiment, subject, goal, created_at = Time.now.utc)`
32
+
33
+ ### Rake Tasks
34
+ * In order to use the included helper Rake Tasks, you must implement `fetch_subject(subject_identifier)` in `Experiment`.
35
+
36
+ ### Unsupported Ruby Versions
37
+ Support has been removed for the following Ruby versions:
38
+ - 1.9.X
39
+ - Rubinius
data/README.md CHANGED
@@ -76,13 +76,14 @@ You can set up storage for your experiment by calling the `storage` method with
76
76
  an object that responds to the following methods:
77
77
 
78
78
  * `store_assignment(assignment)`
79
- * `retrieve_assignment(experiment, subject_identifier)`
80
- * `remove_assignment(experiment, subject_identifier)`
79
+ * `retrieve_assignment(experiment, subject)`
80
+ * `remove_assignment(experiment, subject)`
81
81
  * `retrieve_start_timestamp(experiment)`
82
82
  * `store_start_timestamp(experiment, timestamp)`
83
83
 
84
- Regarding the method signatures above, `experiment` is the Experiment instance, `subject_identifier` is a string that uniquely identifies the subject, and `assignment` is a `Verdict::Assignment` instance.
84
+ Regarding the method signatures above, `experiment` is the Experiment instance, `subject` is the Subject instance, and `assignment` is a `Verdict::Assignment` instance.
85
85
 
86
+ The `subject` instance will be identified internally by its `subject_identifier`
86
87
  By default it will use `subject.id.to_s` as `subject_identifier`, but you can change that by overriding `def subject_identifier(subject)` on the experiment.
87
88
 
88
89
  Storage providers simply store subject assignments and require quick lookups of subject identifiers. They allow for complex (high CPU) assignments, and for assignments that might not always put the same subject in the same group by storing the assignment for later use.
@@ -1,19 +1,15 @@
1
1
  class Verdict::Assignment
2
- attr_reader :experiment, :subject_identifier, :group, :created_at
2
+ attr_reader :experiment, :subject, :group, :created_at
3
3
 
4
- def initialize(experiment, subject_identifier, group, originally_created_at, temporary = false)
4
+ def initialize(experiment, subject, group, originally_created_at, temporary = false)
5
5
  @experiment = experiment
6
- @subject_identifier = subject_identifier
6
+ @subject = subject
7
7
  @group = group
8
8
  @first = originally_created_at.nil? || experiment.manual_assignment_timestamps?
9
9
  @created_at = originally_created_at || Time.now.utc
10
10
  @temporary = temporary
11
11
  end
12
12
 
13
- def subject
14
- @subject ||= experiment.fetch_subject(subject_identifier)
15
- end
16
-
17
13
  def qualified?
18
14
  !group.nil?
19
15
  end
@@ -27,20 +23,24 @@ class Verdict::Assignment
27
23
  end
28
24
 
29
25
  def returning
30
- self.class.new(@experiment, @subject_identifier, @group, @created_at)
26
+ self.class.new(@experiment, @subject, @group, @created_at)
31
27
  end
32
28
 
33
29
  def returning?
34
30
  @first.nil?
35
31
  end
36
32
 
33
+ def subject_identifier
34
+ experiment.retrieve_subject_identifier(subject)
35
+ end
36
+
37
37
  def handle
38
38
  qualified? ? group.handle : nil
39
39
  end
40
40
 
41
41
  def to_sym
42
42
  qualified? ? group.to_sym : nil
43
- end
43
+ end
44
44
 
45
45
  def as_json(options = {})
46
46
  {
@@ -1,20 +1,20 @@
1
1
  class Verdict::Conversion
2
2
 
3
- attr_reader :experiment, :subject_identifier, :goal, :created_at
3
+ attr_reader :experiment, :subject, :goal, :created_at
4
4
 
5
- def initialize(experiment, subject_identifier, goal, created_at = Time.now.utc)
5
+ def initialize(experiment, subject, goal, created_at = Time.now.utc)
6
6
  @experiment = experiment
7
- @subject_identifier = subject_identifier
7
+ @subject = subject
8
8
  @goal = goal
9
9
  @created_at = created_at
10
10
  end
11
11
 
12
- def subject
13
- experiment.fetch_subject(subject_identifier)
12
+ def subject_identifier
13
+ experiment.retrieve_subject_identifier(subject)
14
14
  end
15
15
 
16
16
  def assignment
17
- experiment.fetch_assignment(subject_identifier)
17
+ experiment.lookup(subject)
18
18
  end
19
19
 
20
20
  def as_json(options = {})
@@ -93,17 +93,17 @@ class Verdict::Experiment
93
93
  segmenter.groups.keys
94
94
  end
95
95
 
96
- def subject_assignment(subject_identifier, group, originally_created_at = nil, temporary = false)
97
- Verdict::Assignment.new(self, subject_identifier, group, originally_created_at, temporary)
96
+ def subject_assignment(subject, group, originally_created_at = nil, temporary = false)
97
+ Verdict::Assignment.new(self, subject, group, originally_created_at, temporary)
98
98
  end
99
99
 
100
- def subject_conversion(subject_identifier, goal, created_at = Time.now.utc)
101
- Verdict::Conversion.new(self, subject_identifier, goal, created_at)
100
+ def subject_conversion(subject, goal, created_at = Time.now.utc)
101
+ Verdict::Conversion.new(self, subject, goal, created_at)
102
102
  end
103
103
 
104
104
  def convert(subject, goal)
105
105
  identifier = retrieve_subject_identifier(subject)
106
- conversion = subject_conversion(identifier, goal)
106
+ conversion = subject_conversion(subject, goal)
107
107
  event_logger.log_conversion(conversion)
108
108
  segmenter.conversion_feedback(identifier, subject, conversion)
109
109
  conversion
@@ -121,22 +121,17 @@ class Verdict::Experiment
121
121
 
122
122
  store_assignment(assignment)
123
123
  rescue Verdict::StorageError
124
- subject_assignment(identifier, nil, nil)
124
+ nil_assignment(subject)
125
125
  rescue Verdict::EmptySubjectIdentifier
126
126
  if disqualify_empty_identifier?
127
- subject_assignment(identifier, nil, nil)
127
+ nil_assignment(subject)
128
128
  else
129
129
  raise
130
130
  end
131
131
  end
132
132
 
133
133
  def assign_manually(subject, group)
134
- identifier = retrieve_subject_identifier(subject)
135
- assign_manually_by_identifier(identifier, group)
136
- end
137
-
138
- def assign_manually_by_identifier(subject_identifier, group)
139
- assignment = subject_assignment(subject_identifier, group)
134
+ assignment = subject_assignment(subject, group)
140
135
  if !assignment.qualified? && !store_unqualified?
141
136
  raise Verdict::Error, "Unqualified subject assignments are not stored for this experiment, so manual disqualification is impossible. Consider setting :store_unqualified to true for this experiment."
142
137
  end
@@ -149,10 +144,6 @@ class Verdict::Experiment
149
144
  assign_manually(subject, nil)
150
145
  end
151
146
 
152
- def disqualify_manually_by_identifier(subject_identifier)
153
- assign_manually_by_identifier(subject_identifier, nil)
154
- end
155
-
156
147
  def store_assignment(assignment)
157
148
  @storage.store_assignment(assignment) if should_store_assignment?(assignment)
158
149
  event_logger.log_assignment(assignment)
@@ -160,11 +151,7 @@ class Verdict::Experiment
160
151
  end
161
152
 
162
153
  def remove_subject_assignment(subject)
163
- remove_subject_assignment_by_identifier(retrieve_subject_identifier(subject))
164
- end
165
-
166
- def remove_subject_assignment_by_identifier(subject_identifier)
167
- @storage.remove_assignment(self, subject_identifier)
154
+ @storage.remove_assignment(self, subject)
168
155
  end
169
156
 
170
157
  def switch(subject, context = nil)
@@ -172,11 +159,7 @@ class Verdict::Experiment
172
159
  end
173
160
 
174
161
  def lookup(subject)
175
- lookup_assignment_for_identifier(retrieve_subject_identifier(subject))
176
- end
177
-
178
- def lookup_assignment_for_identifier(subject_identifier)
179
- fetch_assignment(subject_identifier)
162
+ @storage.retrieve_assignment(self, subject)
180
163
  end
181
164
 
182
165
  def retrieve_subject_identifier(subject)
@@ -212,17 +195,18 @@ class Verdict::Experiment
212
195
  end
213
196
 
214
197
  def fetch_subject(subject_identifier)
215
- raise NotImplementedError, "Fetching subjects based in identifier is not implemented for eperiment @{handle.inspect}."
216
- end
217
-
218
- def fetch_assignment(subject_identifier)
219
- @storage.retrieve_assignment(self, subject_identifier)
198
+ raise NotImplementedError, "Fetching subjects based in identifier is not implemented for experiment @{handle.inspect}."
220
199
  end
221
200
 
222
201
  def disqualify_empty_identifier?
223
202
  @disqualify_empty_identifier
224
203
  end
225
204
 
205
+ def subject_qualifies?(subject, context = nil)
206
+ ensure_experiment_has_started
207
+ everybody_qualifies? || @qualifier.call(subject, context)
208
+ end
209
+
226
210
  protected
227
211
 
228
212
  def default_options
@@ -234,24 +218,24 @@ class Verdict::Experiment
234
218
  end
235
219
 
236
220
  def assignment_with_unqualified_persistence(subject_identifier, subject, context)
237
- previous_assignment = fetch_assignment(subject_identifier)
221
+ previous_assignment = lookup(subject)
238
222
  return previous_assignment unless previous_assignment.nil?
239
223
  if subject_qualifies?(subject, context)
240
224
  group = segmenter.assign(subject_identifier, subject, context)
241
- subject_assignment(subject_identifier, group, nil, group.nil?)
225
+ subject_assignment(subject, group, nil, group.nil?)
242
226
  else
243
- subject_assignment(subject_identifier, nil, nil)
227
+ nil_assignment(subject)
244
228
  end
245
229
  end
246
230
 
247
231
  def assignment_without_unqualified_persistence(subject_identifier, subject, context)
248
232
  if subject_qualifies?(subject, context)
249
- previous_assignment = fetch_assignment(subject_identifier)
233
+ previous_assignment = lookup(subject)
250
234
  return previous_assignment unless previous_assignment.nil?
251
235
  group = segmenter.assign(subject_identifier, subject, context)
252
- subject_assignment(subject_identifier, group, nil, group.nil?)
236
+ subject_assignment(subject, group, nil, group.nil?)
253
237
  else
254
- subject_assignment(subject_identifier, nil, nil)
238
+ nil_assignment(subject)
255
239
  end
256
240
  end
257
241
 
@@ -259,11 +243,6 @@ class Verdict::Experiment
259
243
  subject.respond_to?(:id) ? subject.id : subject.to_s
260
244
  end
261
245
 
262
- def subject_qualifies?(subject, context = nil)
263
- ensure_experiment_has_started
264
- everybody_qualifies? || @qualifier.call(subject, context)
265
- end
266
-
267
246
  def set_start_timestamp
268
247
  @storage.store_start_timestamp(self, started_now = Time.now.utc)
269
248
  started_now
@@ -274,4 +253,8 @@ class Verdict::Experiment
274
253
  rescue Verdict::StorageError
275
254
  @started_at ||= Time.now.utc
276
255
  end
256
+
257
+ def nil_assignment(subject)
258
+ subject_assignment(subject, nil, nil)
259
+ end
277
260
  end
@@ -15,11 +15,12 @@ module Verdict
15
15
  # Should do a fast lookup of an assignment of the subject for the given experiment.
16
16
  # - Should return nil if not found in store
17
17
  # - Should return an Assignment instance otherwise.
18
- def retrieve_assignment(experiment, subject_identifier)
18
+ def retrieve_assignment(experiment, subject)
19
+ subject_identifier = experiment.retrieve_subject_identifier(subject)
19
20
  if value = get(experiment.handle.to_s, "assignment_#{subject_identifier}")
20
21
  hash = JSON.parse(value)
21
22
  experiment.subject_assignment(
22
- subject_identifier,
23
+ subject,
23
24
  experiment.group(hash['group']),
24
25
  Time.xmlschema(hash['created_at'])
25
26
  )
@@ -27,7 +28,8 @@ module Verdict
27
28
  end
28
29
 
29
30
  # Should remove the subject from storage, so it will be reassigned later.
30
- def remove_assignment(experiment, subject_identifier)
31
+ def remove_assignment(experiment, subject)
32
+ subject_identifier = experiment.retrieve_subject_identifier(subject)
31
33
  remove(experiment.handle.to_s, "assignment_#{subject_identifier}")
32
34
  end
33
35
 
@@ -43,7 +45,7 @@ module Verdict
43
45
  set(experiment.handle.to_s, 'started_at', timestamp.utc.strftime('%FT%TZ'))
44
46
  end
45
47
 
46
-
48
+ protected
47
49
  # Retrieves a key in a given scope from storage.
48
50
  # - The scope and key are both provided as string.
49
51
  # - Should return a string value if the key is found in the scope, nil otherwise.
@@ -27,6 +27,10 @@ module Verdict
27
27
  def self.subject_identifier
28
28
  Verdict::Rake.require_env('subject')
29
29
  end
30
+
31
+ def self.subject
32
+ experiment.fetch_subject(subject_identifier)
33
+ end
30
34
  end
31
35
  end
32
36
 
@@ -46,8 +50,9 @@ namespace :verdict do
46
50
  task :lookup_assignment => 'environment' do
47
51
  experiment = Verdict::Rake.experiment
48
52
  subject_identifier = Verdict::Rake.subject_identifier
53
+ subject = Verdict::Rake.subject
49
54
 
50
- assignment = experiment.lookup_assignment_for_identifier(subject_identifier)
55
+ assignment = experiment.lookup(subject)
51
56
  if assignment.nil?
52
57
  Verdict::Rake.stdout.puts "Subject `#{subject_identifier}` is not assigned to experiment `#{experiment.handle}` yet."
53
58
  elsif assignment.qualified?
@@ -62,8 +67,9 @@ namespace :verdict do
62
67
  experiment = Verdict::Rake.experiment
63
68
  group = Verdict::Rake.group
64
69
  subject_identifier = Verdict::Rake.subject_identifier
70
+ subject = Verdict::Rake.subject
65
71
 
66
- experiment.assign_manually_by_identifier(Verdict::Rake.require_env('subject'), group)
72
+ experiment.assign_manually(subject, group)
67
73
  Verdict::Rake.stdout.puts "Subject `#{subject_identifier}` has been assigned to group `#{group.handle}` of experiment `#{experiment.handle}`."
68
74
  end
69
75
 
@@ -71,8 +77,9 @@ namespace :verdict do
71
77
  task :disqualify_manually => 'environment' do
72
78
  experiment = Verdict::Rake.experiment
73
79
  subject_identifier = Verdict::Rake.subject_identifier
80
+ subject = Verdict::Rake.subject
74
81
 
75
- experiment.disqualify_manually_by_identifier(subject_identifier)
82
+ experiment.disqualify_manually(subject)
76
83
  Verdict::Rake.stdout.puts "Subject `#{subject_identifier}` has been disqualified from experiment `#{experiment.handle}`."
77
84
  end
78
85
 
@@ -80,8 +87,9 @@ namespace :verdict do
80
87
  task :remove_assignment => 'environment' do
81
88
  experiment = Verdict::Rake.experiment
82
89
  subject_identifier = Verdict::Rake.subject_identifier
90
+ subject = Verdict::Rake.subject
83
91
 
84
- experiment.remove_subject_assignment_by_identifier(subject_identifier)
92
+ experiment.remove_subject_assignment(subject)
85
93
  Verdict::Rake.stdout.puts "Removed assignment of subject with identifier `#{subject_identifier}`."
86
94
  Verdict::Rake.stdout.puts "The subject will be reasigned when it encounters the experiment `#{experiment.handle}` again."
87
95
  end
@@ -1,3 +1,3 @@
1
1
  module Verdict
2
- VERSION = "0.5.0"
2
+ VERSION = "0.6.0"
3
3
  end
@@ -28,13 +28,12 @@ class AssignmentTest < Minitest::Test
28
28
  assert_kind_of Time, assignment.created_at
29
29
  end
30
30
 
31
- def test_subject_lookup
32
- assignment = Verdict::Assignment.new(@experiment, 'test_subject_id', nil, Time.now.utc)
33
- assert_raises(NotImplementedError) { assignment.subject }
31
+ def test_subject_identifier_lookup
32
+ klass = Struct.new(:id)
33
+ subject = klass.new(123)
34
34
 
35
- @experiment.expects(:fetch_subject).with('test_subject_id').returns(subject = mock('subject'))
36
- assignment = Verdict::Assignment.new(@experiment, 'test_subject_id', nil, Time.now.utc)
37
- assert_equal subject, assignment.subject
35
+ assignment = Verdict::Assignment.new(@experiment, subject, nil, Time.now.utc)
36
+ assert_equal '123', assignment.subject_identifier
38
37
  end
39
38
 
40
39
  def test_triple_equals
@@ -9,13 +9,12 @@ class ConversionTest < Minitest::Test
9
9
  end
10
10
  end
11
11
 
12
- def test_subject_lookup
13
- conversion = Verdict::Conversion.new(@experiment, 'test_subject_id', :test_goal)
14
- assert_raises(NotImplementedError) { conversion.subject }
12
+ def test_subject_identifier_lookup
13
+ klass = Struct.new(:id)
14
+ subject = klass.new(123)
15
15
 
16
- @experiment.expects(:fetch_subject).with('test_subject_id').returns(subject = mock('subject'))
17
- conversion = Verdict::Conversion.new(@experiment, 'test_subject_id', :test_goal)
18
- assert_equal subject, conversion.subject
16
+ conversion = Verdict::Conversion.new(@experiment, subject, :test_goal)
17
+ assert_equal '123', conversion.subject_identifier
19
18
  end
20
19
 
21
20
  def test_assignment_lookup
@@ -3,12 +3,19 @@ require 'stringio'
3
3
 
4
4
  class RakeTasksTest < Minitest::Test
5
5
 
6
+ TestSubjectClass = Struct.new(:id)
7
+ class TestExperiment < Verdict::Experiment
8
+ def fetch_subject(subject_identifier)
9
+ TestSubjectClass.new(subject_identifier.to_i)
10
+ end
11
+ end
12
+
6
13
  def setup
7
14
  require 'rake' unless defined?(Rake)
8
15
  Rake::Task.define_task(:environment)
9
16
  Rake.application.rake_require('verdict/tasks')
10
-
11
- @experiment = Verdict::Experiment.define(:rake, store_unqualified: true) do
17
+
18
+ @experiment = TestExperiment.define(:rake, store_unqualified: true) do
12
19
  groups do
13
20
  group :a, 50
14
21
  group :b, 50
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: verdict
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.0
4
+ version: 0.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Shopify
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2016-02-01 00:00:00.000000000 Z
11
+ date: 2017-02-21 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: minitest
@@ -90,6 +90,7 @@ extra_rdoc_files: []
90
90
  files:
91
91
  - ".gitignore"
92
92
  - ".travis.yml"
93
+ - CHANGELOG.md
93
94
  - Gemfile
94
95
  - LICENSE
95
96
  - README.md
@@ -154,7 +155,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
154
155
  version: '0'
155
156
  requirements: []
156
157
  rubyforge_project:
157
- rubygems_version: 2.2.3
158
+ rubygems_version: 2.5.2
158
159
  signing_key:
159
160
  specification_version: 4
160
161
  summary: A library to centrally define experiments for your application, and collect