verdict 0.11.0 → 0.15.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.travis.yml +1 -0
- data/CHANGELOG.md +22 -0
- data/CONTRIBUTING.md +7 -0
- data/README.md +10 -17
- data/docs/concepts.md +35 -0
- data/lib/verdict.rb +1 -0
- data/lib/verdict/experiment.rb +65 -21
- data/lib/verdict/railtie.rb +8 -3
- data/lib/verdict/storage.rb +1 -0
- data/lib/verdict/storage/base_storage.rb +22 -4
- data/lib/verdict/storage/cookie_storage.rb +105 -0
- data/lib/verdict/storage/memory_storage.rb +2 -0
- data/lib/verdict/storage/mock_storage.rb +5 -3
- data/lib/verdict/storage/redis_storage.rb +6 -4
- data/lib/verdict/version.rb +1 -1
- data/test/experiment_test.rb +230 -1
- data/test/experiments_repository_test.rb +1 -0
- data/test/fake_app.rb +1 -1
- data/test/storage/cookie_storage_test.rb +81 -0
- data/test/storage/memory_storage_test.rb +2 -2
- data/test/storage/redis_storage_test.rb +13 -1
- data/test/verdict_rails_test.rb +8 -1
- data/verdict.gemspec +2 -0
- metadata +9 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 27eea30b628bf29627fe68762f18dea365ebc77d0ac34e8b1b63f24e29814276
|
4
|
+
data.tar.gz: 65a0cb4ae9247c75bdcff094a7546e673cda0d05f8a922c48ed7f448deb931b7
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 813097404fecc5c3f44121d68b5da2cb0213fcbadfa4b5b20c451286577b6d7e35900e636582fe3d4b1d6b95c6df3916da9e89ded291d042f11f963a8a58e895
|
7
|
+
data.tar.gz: ea1e6b44880d6a6dd085173abf4155f5e8605272d6d679c8dea8a71259d5b7b8f86b62adc39afb49e40126ef8cd547efda8904214a3d51f5881f96030498e028
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,25 @@
|
|
1
|
+
## v0.15.1
|
2
|
+
* Make the `dynamic_qualifies` parameter in `Verdict::Experiment#subject_qualifies?` optional. This fixes a bug where users that were previously calling this method directly experienced issues after v0.15.0
|
3
|
+
|
4
|
+
## v0.15.0
|
5
|
+
* Add optional `qualifiers` parameter to the `Verdict::Experiment#switch` method. This parameter accepts an array of procs and is used as additional qualifiers. The purpose of this parameter is to allow users to define qualification logic outside of the experiment definition.
|
6
|
+
|
7
|
+
## v0.14.0
|
8
|
+
* Add optional experiment definition method `schedule_stop_new_assignment_timestamp` to support limiting experiment's assignment lifetime with another pre-determined time interval. It allows users to have an assignment cooldown period for stable analysis of the experiment results. Experiment's lifetime now becomes: start experiment -> stop new assignments -> end experiment.
|
9
|
+
|
10
|
+
## v0.13.0
|
11
|
+
|
12
|
+
* Add optional experiment definition methods `schedule_start_timestamp` and `schedule_end_timestamp` to support limiting experiment's lifetime in a pre-determined time interval.
|
13
|
+
* Support eager loading from within a Rails app using Zeitwerk.
|
14
|
+
* Add `CookieStorage` storage backend. This backend is a distributed store for Verdict and does not support experiment timestamps. It is designed to be used with Rails applications and requires that `.cookies` be set to the `CookieJar` instance before use.
|
15
|
+
|
16
|
+
## v0.12.0
|
17
|
+
|
18
|
+
* Allow options to be passed to `Experiment#cleanup` so they can be forwarded to storage.
|
19
|
+
|
20
|
+
* Changed `Experiment#cleanup` to accept an argument of type `Verdict::Experiment`.
|
21
|
+
Passing a `String`/`Symbol` argument is still supported, but will log a deprecation warning.
|
22
|
+
|
1
23
|
## v0.11.0
|
2
24
|
|
3
25
|
* Automatic eager loading when inside a Rails app.
|
data/CONTRIBUTING.md
ADDED
data/README.md
CHANGED
@@ -19,6 +19,8 @@ Add this line to your application's Gemfile, and run `bundle install`:
|
|
19
19
|
|
20
20
|
If you're using Rails, the Railtie will handle setting the logger to `Rails.logger` and the experiments directory to `app/experiments`. It will also load the rake tasks for you (run `bundle exec rake -T | grep experiments:` for options).
|
21
21
|
|
22
|
+
You may find the [Concepts](docs/concepts.md) documentation a good place to familiarise yourself with Verdict's nomenclature.
|
23
|
+
|
22
24
|
## Usage
|
23
25
|
|
24
26
|
The `Verdict::Experiment` class is used to create an experiment, define control and experiment groups, and to qualify subjects.
|
@@ -42,8 +44,7 @@ Verdict::Experiment.define :my_experiment do
|
|
42
44
|
end
|
43
45
|
```
|
44
46
|
|
45
|
-
Usually you'll want to place this in a file called **my_experiment.rb** in the
|
46
|
-
**/app/experiments** folder.
|
47
|
+
Usually you'll want to place this in a file called **my_experiment.rb** in the **/app/experiments** folder.
|
47
48
|
|
48
49
|
_We recommend that you subclass `Verdict::Experiment` to set some default options for your app's environment, and call `define` on that class instead._
|
49
50
|
|
@@ -71,9 +72,9 @@ Verdict uses a very simple interface for storing experiment assignments. Out of
|
|
71
72
|
|
72
73
|
* Memory
|
73
74
|
* Redis
|
75
|
+
* Cookies
|
74
76
|
|
75
|
-
You can set up storage for your experiment by calling the `storage` method with
|
76
|
-
an object that responds to the following methods:
|
77
|
+
You can set up storage for your experiment by calling the `storage` method with an object that responds to the following methods:
|
77
78
|
|
78
79
|
* `store_assignment(assignment)`
|
79
80
|
* `retrieve_assignment(experiment, subject)`
|
@@ -83,14 +84,15 @@ an object that responds to the following methods:
|
|
83
84
|
|
84
85
|
Regarding the method signatures above, `experiment` is the Experiment instance, `subject` is the Subject instance, and `assignment` is a `Verdict::Assignment` instance.
|
85
86
|
|
86
|
-
The `subject` instance will be identified internally by its `subject_identifier`
|
87
|
-
By default it will use `subject.id.to_s` as `subject_identifier`, but you can change that by overriding `def subject_identifier(subject)` on the experiment.
|
87
|
+
The `subject` instance will be identified internally by its `subject_identifier`. By default it will use `subject.id.to_s` as `subject_identifier`, but you can change that by overriding `def subject_identifier(subject)` on the experiment.
|
88
88
|
|
89
89
|
Storage providers simply store subject assignments and require quick lookups of subject identifiers. They allow for complex (high CPU) assignments, and for assignments that might not always put the same subject in the same group by storing the assignment for later use.
|
90
90
|
|
91
|
-
Storage providers are intended for operational use and should not be used for data analysis. For data analysis, you should use the logger.
|
91
|
+
Storage providers are intended for operational use and should not be used for data analysis. For data analysis, you should use [the logger](#logging).
|
92
|
+
|
93
|
+
When removing old experiments you might want to clean up corresponding experiment assignments, to reduce the amount of data stored and loaded. By using the logger, this data removal doesn't impact historic data or data analysis.
|
92
94
|
|
93
|
-
For more details about these methods, check out the source code for [Verdict::Storage::MockStorage](lib/verdict/storage/mock_storage.rb)
|
95
|
+
For more details about these methods, check out the source code for [`Verdict::Storage::MockStorage`](lib/verdict/storage/mock_storage.rb)
|
94
96
|
|
95
97
|
## Logging
|
96
98
|
|
@@ -101,12 +103,3 @@ You can override the logging by overriding the `def log_assignment(assignment)`
|
|
101
103
|
Logging (as opposed to storage) should be used for data analysis. The logger requires a write-only / forward-only stream to write to, e.g. a log file, Kafka, or an insert-only database table.
|
102
104
|
|
103
105
|
It's possible to run an experiment without defining any storage, though this comes with several drawbacks. Logging on the other hand is required in order to analyze the results.
|
104
|
-
|
105
|
-
|
106
|
-
## Contributing
|
107
|
-
|
108
|
-
1. Fork it
|
109
|
-
2. Create your feature branch (`git checkout -b my-new-feature`)
|
110
|
-
3. Commit your changes, including tests (`git commit -am 'Added some feature'`)
|
111
|
-
4. Push to the branch (`git push origin my-new-feature`)
|
112
|
-
5. Create new Pull Request, and mention @wvanbergen.
|
data/docs/concepts.md
ADDED
@@ -0,0 +1,35 @@
|
|
1
|
+
# Verdict Concepts
|
2
|
+
|
3
|
+
Understanding the following Verdict concepts can help you implement experiments and discuss them with peers.
|
4
|
+
|
5
|
+
- [Experiment](#experiment)
|
6
|
+
- [Qualifiers](#qualifiers)
|
7
|
+
- [Subject](#subject)
|
8
|
+
- [Group](#group)
|
9
|
+
- [Assignment](#assignment)
|
10
|
+
|
11
|
+
### Experiment
|
12
|
+
|
13
|
+
`Experiment`s are defined in `app/experiments` and encapsulate the details of the test including metadata such as the name, description or owners, as well as any [qualifiers](#qualifiers) or the [groups](#groups) available in the test.
|
14
|
+
|
15
|
+
### Qualifiers
|
16
|
+
|
17
|
+
Qualifiers are methods that take two parameters, `subject` and `context`. They return `true` if the `subject` can take this experiment, or `false` if the `subject` is not eligible. Experiments can have multiple qualifiers which are evaluated in succession. If a qualifier returns `false`, no further qualifiers are checked.
|
18
|
+
|
19
|
+
Examples of qualifiers might be subjects that:
|
20
|
+
|
21
|
+
- Originate from certain browsers or devices
|
22
|
+
- Have a specific `Accept-Language` header
|
23
|
+
- Have added items to their cart.
|
24
|
+
|
25
|
+
### Subject
|
26
|
+
|
27
|
+
A subject is the thing that will be [qualified](#qualifiers), and if successfull, [assigned](#assignment) to a particular [group](#group). For example, a subject might represent a user.
|
28
|
+
|
29
|
+
### Group
|
30
|
+
|
31
|
+
Each [experiment](#experiment) will have two or more groups, usually a control group and a test group. Qualified [subjects](#subject) are assigned to one of the groups in the experiment.
|
32
|
+
|
33
|
+
### Assignment
|
34
|
+
|
35
|
+
An assignment occurs when a qualified [subject](#subject) is placed into a [group](#group). Assignments are persisted so repeat visits by the subject skip [qualification](#qualifiers) and re-assignment and simply returns the existing assignment.
|
data/lib/verdict.rb
CHANGED
data/lib/verdict/experiment.rb
CHANGED
@@ -2,7 +2,7 @@ class Verdict::Experiment
|
|
2
2
|
|
3
3
|
include Verdict::Metadata
|
4
4
|
|
5
|
-
attr_reader :handle, :qualifiers, :
|
5
|
+
attr_reader :handle, :qualifiers, :event_logger
|
6
6
|
|
7
7
|
def self.define(handle, *args, &block)
|
8
8
|
experiment = self.new(handle, *args, &block)
|
@@ -11,6 +11,7 @@ class Verdict::Experiment
|
|
11
11
|
end
|
12
12
|
|
13
13
|
def initialize(handle, options = {}, &block)
|
14
|
+
@started_at = nil
|
14
15
|
@handle = handle.to_s
|
15
16
|
|
16
17
|
options = default_options.merge(options)
|
@@ -51,6 +52,29 @@ class Verdict::Experiment
|
|
51
52
|
return self
|
52
53
|
end
|
53
54
|
|
55
|
+
# Optional: Together with the "end_timestamp" and "stop_new_assignment_timestamp", limits the experiment run timeline within
|
56
|
+
# the given time interval.
|
57
|
+
#
|
58
|
+
# Timestamps definitions:
|
59
|
+
# start_timestamp: Experiment's start time. No assignments are made i.e. switch will return nil before this timestamp.
|
60
|
+
# stop_new_assignment_timestamp: Experiment's new assignment stop time. No new assignments are made
|
61
|
+
# i.e. switch returns nil for new assignments but the existing assignments are preserved.
|
62
|
+
# end_timestamp: Experiment's end time. No assignments are made i.e. switch returns nil after this timestamp.
|
63
|
+
#
|
64
|
+
# Experiment run timeline:
|
65
|
+
# start_timestamp -> (new assignments occur) -> stop_new_assignment_timestamp -> (no new assignments occur) -> end_timestamp
|
66
|
+
def schedule_start_timestamp(timestamp)
|
67
|
+
@schedule_start_timestamp = timestamp
|
68
|
+
end
|
69
|
+
|
70
|
+
def schedule_end_timestamp(timestamp)
|
71
|
+
@schedule_end_timestamp = timestamp
|
72
|
+
end
|
73
|
+
|
74
|
+
def schedule_stop_new_assignment_timestamp(timestamp)
|
75
|
+
@schedule_stop_new_assignment_timestamp = timestamp
|
76
|
+
end
|
77
|
+
|
54
78
|
def rollout_percentage(percentage, rollout_group_name = :enabled)
|
55
79
|
groups(Verdict::Segmenters::RolloutSegmenter) do
|
56
80
|
group rollout_group_name, percentage
|
@@ -88,7 +112,7 @@ class Verdict::Experiment
|
|
88
112
|
|
89
113
|
def started_at
|
90
114
|
@started_at ||= @storage.retrieve_start_timestamp(self)
|
91
|
-
rescue Verdict::StorageError
|
115
|
+
rescue Verdict::StorageError
|
92
116
|
nil
|
93
117
|
end
|
94
118
|
|
@@ -118,18 +142,18 @@ class Verdict::Experiment
|
|
118
142
|
raise unless disqualify_empty_identifier?
|
119
143
|
end
|
120
144
|
|
121
|
-
def assign(subject, context = nil)
|
145
|
+
def assign(subject, context = nil, dynamic_qualifiers: [])
|
122
146
|
previous_assignment = lookup(subject)
|
123
147
|
|
124
148
|
subject_identifier = retrieve_subject_identifier(subject)
|
125
149
|
assignment = if previous_assignment
|
126
|
-
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
|
132
|
-
|
150
|
+
previous_assignment
|
151
|
+
elsif subject_qualifies?(subject, dynamic_qualifiers, context) && is_make_new_assignments?
|
152
|
+
group = segmenter.assign(subject_identifier, subject, context)
|
153
|
+
subject_assignment(subject, group, nil, group.nil?)
|
154
|
+
else
|
155
|
+
nil_assignment(subject)
|
156
|
+
end
|
133
157
|
|
134
158
|
store_assignment(assignment)
|
135
159
|
rescue Verdict::StorageError
|
@@ -162,16 +186,19 @@ class Verdict::Experiment
|
|
162
186
|
assignment
|
163
187
|
end
|
164
188
|
|
165
|
-
def cleanup
|
166
|
-
@storage.cleanup(
|
189
|
+
def cleanup(options = {})
|
190
|
+
@storage.cleanup(self, options)
|
167
191
|
end
|
168
192
|
|
169
193
|
def remove_subject_assignment(subject)
|
170
194
|
@storage.remove_assignment(self, subject)
|
171
195
|
end
|
172
196
|
|
173
|
-
|
174
|
-
|
197
|
+
# The qualifiers param accepts an array of procs.
|
198
|
+
# This is intended for qualification logic that cannot be defined in the experiment definition
|
199
|
+
def switch(subject, context = nil, qualifiers: [])
|
200
|
+
return unless is_scheduled?
|
201
|
+
assign(subject, context, dynamic_qualifiers: qualifiers).to_sym
|
175
202
|
end
|
176
203
|
|
177
204
|
def lookup(subject)
|
@@ -193,15 +220,13 @@ class Verdict::Experiment
|
|
193
220
|
end
|
194
221
|
|
195
222
|
def as_json(options = {})
|
196
|
-
|
223
|
+
{
|
197
224
|
handle: handle,
|
198
225
|
has_qualifier: has_qualifier?,
|
199
|
-
groups: segmenter.groups.values.map { |
|
226
|
+
groups: segmenter.groups.values.map { |group| group.as_json(options) },
|
200
227
|
metadata: metadata,
|
201
228
|
started_at: started_at.nil? ? nil : started_at.utc.strftime('%FT%TZ')
|
202
|
-
}
|
203
|
-
|
204
|
-
data.tap do |data|
|
229
|
+
}.tap do |data|
|
205
230
|
data[:subject_type] = subject_type.to_s unless subject_type.nil?
|
206
231
|
end
|
207
232
|
end
|
@@ -218,8 +243,9 @@ class Verdict::Experiment
|
|
218
243
|
@disqualify_empty_identifier
|
219
244
|
end
|
220
245
|
|
221
|
-
def subject_qualifies?(subject, context = nil)
|
246
|
+
def subject_qualifies?(subject, dynamic_qualifiers = [], context = nil)
|
222
247
|
ensure_experiment_has_started
|
248
|
+
return false unless dynamic_qualifiers.all? { |qualifier| qualifier.call(subject) }
|
223
249
|
everybody_qualifies? || @qualifiers.all? { |qualifier| qualifier.call(subject, context) }
|
224
250
|
end
|
225
251
|
|
@@ -240,10 +266,12 @@ class Verdict::Experiment
|
|
240
266
|
def set_start_timestamp
|
241
267
|
@storage.store_start_timestamp(self, started_now = Time.now.utc)
|
242
268
|
started_now
|
269
|
+
rescue NotImplementedError
|
270
|
+
nil
|
243
271
|
end
|
244
272
|
|
245
273
|
def ensure_experiment_has_started
|
246
|
-
@started_at ||=
|
274
|
+
@started_at ||= started_at || set_start_timestamp
|
247
275
|
rescue Verdict::StorageError
|
248
276
|
@started_at ||= Time.now.utc
|
249
277
|
end
|
@@ -251,4 +279,20 @@ class Verdict::Experiment
|
|
251
279
|
def nil_assignment(subject)
|
252
280
|
subject_assignment(subject, nil, nil)
|
253
281
|
end
|
282
|
+
|
283
|
+
private
|
284
|
+
|
285
|
+
def is_scheduled?
|
286
|
+
if @schedule_start_timestamp && @schedule_start_timestamp > Time.now
|
287
|
+
return false
|
288
|
+
end
|
289
|
+
if @schedule_end_timestamp && @schedule_end_timestamp <= Time.now
|
290
|
+
return false
|
291
|
+
end
|
292
|
+
return true
|
293
|
+
end
|
294
|
+
|
295
|
+
def is_make_new_assignments?
|
296
|
+
return !(@schedule_stop_new_assignment_timestamp && @schedule_stop_new_assignment_timestamp <= Time.now)
|
297
|
+
end
|
254
298
|
end
|
data/lib/verdict/railtie.rb
CHANGED
@@ -5,10 +5,15 @@ class Verdict::Railtie < Rails::Railtie
|
|
5
5
|
Verdict.default_logger = Rails.logger
|
6
6
|
|
7
7
|
Verdict.directory ||= Rails.root.join('app', 'experiments')
|
8
|
-
app.config.eager_load_paths -= Dir[Verdict.directory.to_s]
|
9
8
|
|
10
|
-
|
11
|
-
|
9
|
+
if Rails.gem_version >= Gem::Version.new('6.0.0') && Rails.autoloaders.zeitwerk_enabled?
|
10
|
+
Rails.autoloaders.main.ignore(Verdict.directory)
|
11
|
+
else
|
12
|
+
app.config.eager_load_paths -= Dir[Verdict.directory.to_s]
|
13
|
+
|
14
|
+
# Re-freeze eager load paths to ensure they blow up if modified at runtime, as Rails does
|
15
|
+
app.config.eager_load_paths.freeze
|
16
|
+
end
|
12
17
|
end
|
13
18
|
|
14
19
|
config.to_prepare do
|
data/lib/verdict/storage.rb
CHANGED
@@ -45,11 +45,21 @@ module Verdict
|
|
45
45
|
set(experiment.handle.to_s, 'started_at', timestamp.utc.strftime('%FT%TZ'))
|
46
46
|
end
|
47
47
|
|
48
|
-
|
49
|
-
|
48
|
+
# Deletes all assignments (and any other stored data) for the given experiment
|
49
|
+
def cleanup(experiment_or_scope, options = {})
|
50
|
+
if experiment_or_scope.is_a?(Symbol) || experiment_or_scope.is_a?(String)
|
51
|
+
Verdict.default_logger.warn(
|
52
|
+
"Passing a scope string/symbol to #{self.class}#cleanup is deprecated, " \
|
53
|
+
'pass a Verdict::Experiment instance instead.'
|
54
|
+
)
|
55
|
+
clear(experiment_or_scope, options)
|
56
|
+
else
|
57
|
+
clear(experiment_or_scope.handle.to_s, options)
|
58
|
+
end
|
50
59
|
end
|
51
60
|
|
52
61
|
protected
|
62
|
+
|
53
63
|
# Retrieves a key in a given scope from storage.
|
54
64
|
# - The scope and key are both provided as string.
|
55
65
|
# - Should return a string value if the key is found in the scope, nil otherwise.
|
@@ -58,7 +68,7 @@ module Verdict
|
|
58
68
|
raise NotImplementedError
|
59
69
|
end
|
60
70
|
|
61
|
-
#
|
71
|
+
# Sets the value of a key in a given scope from storage.
|
62
72
|
# - The scope, key, and value are all provided as string.
|
63
73
|
# - Should return true if the item was successfully stored.
|
64
74
|
# - Should raise Verdict::StorageError if anything goes wrong.
|
@@ -66,13 +76,21 @@ module Verdict
|
|
66
76
|
raise NotImplementedError
|
67
77
|
end
|
68
78
|
|
69
|
-
#
|
79
|
+
# Removes a key in a given scope from storage.
|
70
80
|
# - The scope and key are both provided as string.
|
71
81
|
# - Should return true if the item was successfully removed from storage.
|
72
82
|
# - Should raise Verdict::StorageError if anything goes wrong.
|
73
83
|
def remove(scope, key)
|
74
84
|
raise NotImplementedError
|
75
85
|
end
|
86
|
+
|
87
|
+
# Removes all keys in a given scope from storage.
|
88
|
+
# - The scope is provided as string.
|
89
|
+
# - Should return true if all items were successfully removed from storage.
|
90
|
+
# - Should raise Verdict::StorageError if anything goes wrong.
|
91
|
+
def clear(scope, options)
|
92
|
+
raise NotImplementedError
|
93
|
+
end
|
76
94
|
end
|
77
95
|
end
|
78
96
|
end
|
@@ -0,0 +1,105 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Verdict
|
4
|
+
module Storage
|
5
|
+
# CookieStorage, unlike other Verdict storage classes, is distributed and stored on the client.
|
6
|
+
# Because cookies are opaque to users, we obsfucate information about the test such as the
|
7
|
+
# human readable names for the experiment or assignment group. This means this class assumes
|
8
|
+
# that `name` will be an obsfucated value, or one that we comfortable being "public".
|
9
|
+
class CookieStorage < BaseStorage
|
10
|
+
DEFAULT_COOKIE_LIFESPAN_SECONDS = 15778476 # 6 months
|
11
|
+
KEY = :assignment
|
12
|
+
|
13
|
+
attr_accessor :cookies
|
14
|
+
attr_reader :cookie_lifespan
|
15
|
+
|
16
|
+
def initialize(cookie_lifespan: DEFAULT_COOKIE_LIFESPAN_SECONDS)
|
17
|
+
@cookies = nil
|
18
|
+
@cookie_lifespan = cookie_lifespan
|
19
|
+
end
|
20
|
+
|
21
|
+
def store_assignment(assignment)
|
22
|
+
data = {
|
23
|
+
group: digest_of(assignment.group.to_s),
|
24
|
+
created_at: assignment.created_at.strftime('%FT%TZ')
|
25
|
+
}
|
26
|
+
|
27
|
+
set(assignment.experiment.handle.to_s, KEY, JSON.dump(data))
|
28
|
+
end
|
29
|
+
|
30
|
+
def retrieve_assignment(experiment, subject)
|
31
|
+
if (value = get(experiment.handle.to_s, KEY))
|
32
|
+
data = parse_cookie_value(value)
|
33
|
+
group = find_group_by_digest(experiment, data['group'])
|
34
|
+
|
35
|
+
if group.nil?
|
36
|
+
experiment.remove_subject_assignment(subject)
|
37
|
+
return nil
|
38
|
+
end
|
39
|
+
|
40
|
+
experiment.subject_assignment(subject, group, Time.xmlschema(data['created_at']))
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
def remove_assignment(experiment, _subject)
|
45
|
+
remove(experiment.handle.to_s, KEY)
|
46
|
+
end
|
47
|
+
|
48
|
+
def retrieve_start_timestamp(_experiment)
|
49
|
+
nil
|
50
|
+
end
|
51
|
+
|
52
|
+
def store_start_timestamp(_experiment, _timestamp)
|
53
|
+
raise NotImplementedError
|
54
|
+
end
|
55
|
+
|
56
|
+
protected
|
57
|
+
|
58
|
+
def get(scope, key)
|
59
|
+
ensure_cookies_is_set
|
60
|
+
cookies[scope_key(scope, key)]
|
61
|
+
end
|
62
|
+
|
63
|
+
def set(scope, key, value)
|
64
|
+
ensure_cookies_is_set
|
65
|
+
cookies[scope_key(scope, key)] = {
|
66
|
+
value: value,
|
67
|
+
expires: Time.now.utc.advance(seconds: cookie_lifespan),
|
68
|
+
}
|
69
|
+
end
|
70
|
+
|
71
|
+
def remove(scope, key)
|
72
|
+
ensure_cookies_is_set
|
73
|
+
cookies.delete(scope_key(scope, key))
|
74
|
+
end
|
75
|
+
|
76
|
+
private
|
77
|
+
|
78
|
+
def ensure_cookies_is_set
|
79
|
+
raise Verdict::StorageError, 'cookies must be an instance of ActionDispatch::Cookies::CookieJar' if cookies.nil?
|
80
|
+
end
|
81
|
+
|
82
|
+
def digest_of(value)
|
83
|
+
Digest::MD5.hexdigest(value)
|
84
|
+
end
|
85
|
+
|
86
|
+
def find_group_by_digest(experiment, digest)
|
87
|
+
experiment.groups.values.find do |group|
|
88
|
+
digest_of(group.to_s) == digest
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
def parse_cookie_value(value)
|
93
|
+
value = value[:value] if value.is_a?(Hash)
|
94
|
+
|
95
|
+
JSON.parse(value)
|
96
|
+
rescue JSON::ParserError, TypeError
|
97
|
+
{}
|
98
|
+
end
|
99
|
+
|
100
|
+
def scope_key(scope, key)
|
101
|
+
"#{digest_of(scope)}_#{key}"
|
102
|
+
end
|
103
|
+
end
|
104
|
+
end
|
105
|
+
end
|
@@ -1,14 +1,16 @@
|
|
1
1
|
module Verdict
|
2
2
|
module Storage
|
3
3
|
class MockStorage < BaseStorage
|
4
|
-
|
5
|
-
false
|
6
|
-
end
|
4
|
+
protected
|
7
5
|
|
8
6
|
def get(scope, key)
|
9
7
|
nil
|
10
8
|
end
|
11
9
|
|
10
|
+
def set(scope, key, value)
|
11
|
+
false
|
12
|
+
end
|
13
|
+
|
12
14
|
def remove(scope, key)
|
13
15
|
end
|
14
16
|
end
|
@@ -10,6 +10,8 @@ module Verdict
|
|
10
10
|
@key_prefix = options[:key_prefix] || 'experiments/'
|
11
11
|
end
|
12
12
|
|
13
|
+
protected
|
14
|
+
|
13
15
|
def get(scope, key)
|
14
16
|
redis.hget(scope_key(scope), key)
|
15
17
|
rescue ::Redis::BaseError => e
|
@@ -28,8 +30,8 @@ module Verdict
|
|
28
30
|
raise Verdict::StorageError, "Redis error: #{e.message}"
|
29
31
|
end
|
30
32
|
|
31
|
-
def
|
32
|
-
|
33
|
+
def clear(scope, options)
|
34
|
+
scrub(scope)
|
33
35
|
redis.del(scope_key(scope))
|
34
36
|
rescue ::Redis::BaseError => e
|
35
37
|
raise Verdict::StorageError, "Redis error: #{e.message}"
|
@@ -41,12 +43,12 @@ module Verdict
|
|
41
43
|
"#{@key_prefix}#{scope}"
|
42
44
|
end
|
43
45
|
|
44
|
-
def
|
46
|
+
def scrub(scope, cursor: 0)
|
45
47
|
cursor, results = redis.hscan(scope_key(scope), cursor, count: PAGE_SIZE)
|
46
48
|
results.map(&:first).each do |key|
|
47
49
|
remove(scope, key)
|
48
50
|
end
|
49
|
-
|
51
|
+
scrub(scope, cursor: cursor) unless cursor.to_i.zero?
|
50
52
|
end
|
51
53
|
end
|
52
54
|
end
|
data/lib/verdict/version.rb
CHANGED
data/test/experiment_test.rb
CHANGED
@@ -414,6 +414,16 @@ class ExperimentTest < Minitest::Test
|
|
414
414
|
assert e.started?, "The experiment should have started after the first assignment"
|
415
415
|
end
|
416
416
|
|
417
|
+
def test_experiment_set_start_timestamp_handles_storage_that_does_not_implement_timestamps
|
418
|
+
e = Verdict::Experiment.new('starting_test') do
|
419
|
+
groups { group :all, 100 }
|
420
|
+
end
|
421
|
+
|
422
|
+
e.storage.expects(:store_start_timestamp).raises(NotImplementedError)
|
423
|
+
|
424
|
+
assert_nil e.send(:set_start_timestamp)
|
425
|
+
end
|
426
|
+
|
417
427
|
def test_no_storage
|
418
428
|
e = Verdict::Experiment.new('starting_test') do
|
419
429
|
groups { group :all, 100 }
|
@@ -424,7 +434,6 @@ class ExperimentTest < Minitest::Test
|
|
424
434
|
end
|
425
435
|
|
426
436
|
def test_cleanup
|
427
|
-
redis = ::Redis.new(host: REDIS_HOST, port: REDIS_PORT)
|
428
437
|
storage = Verdict::Storage::RedisStorage.new(redis)
|
429
438
|
experiment = Verdict::Experiment.new(:cleanup_test) do
|
430
439
|
groups { group :all, 100 }
|
@@ -440,6 +449,16 @@ class ExperimentTest < Minitest::Test
|
|
440
449
|
redis.del("experiments/cleanup_test")
|
441
450
|
end
|
442
451
|
|
452
|
+
def test_cleanup_options
|
453
|
+
experiment = Verdict::Experiment.new(:cleanup_test) do
|
454
|
+
groups { group :all, 100 }
|
455
|
+
end
|
456
|
+
|
457
|
+
experiment.storage.expects(:clear).with(experiment.handle, some: :thing)
|
458
|
+
experiment.assign("something")
|
459
|
+
experiment.cleanup(some: :thing)
|
460
|
+
end
|
461
|
+
|
443
462
|
def test_cleanup_without_redis
|
444
463
|
experiment = Verdict::Experiment.new(:cleanup_test) do
|
445
464
|
groups { group :all, 100 }
|
@@ -450,4 +469,214 @@ class ExperimentTest < Minitest::Test
|
|
450
469
|
experiment.cleanup
|
451
470
|
end
|
452
471
|
end
|
472
|
+
|
473
|
+
def test_is_scheduled
|
474
|
+
e = Verdict::Experiment.new(:json) do
|
475
|
+
groups do
|
476
|
+
group :a, :half
|
477
|
+
group :b, :rest
|
478
|
+
end
|
479
|
+
schedule_start_timestamp Time.new(2020, 1, 1)
|
480
|
+
schedule_end_timestamp Time.new(2020, 1, 3)
|
481
|
+
end
|
482
|
+
|
483
|
+
# Within the interval
|
484
|
+
Timecop.freeze(Time.new(2020, 1, 2)) do
|
485
|
+
assert e.send(:is_scheduled?)
|
486
|
+
end
|
487
|
+
# Too early
|
488
|
+
Timecop.freeze(Time.new(2019, 12, 30)) do
|
489
|
+
assert !e.send(:is_scheduled?)
|
490
|
+
end
|
491
|
+
# Too late
|
492
|
+
Timecop.freeze(Time.new(2020, 1, 4)) do
|
493
|
+
assert !e.send(:is_scheduled?)
|
494
|
+
end
|
495
|
+
end
|
496
|
+
|
497
|
+
def test_is_scheduled_no_end_timestamp
|
498
|
+
e = Verdict::Experiment.new(:json) do
|
499
|
+
groups do
|
500
|
+
group :a, :half
|
501
|
+
group :b, :rest
|
502
|
+
end
|
503
|
+
schedule_start_timestamp Time.new(2020, 1, 1)
|
504
|
+
end
|
505
|
+
|
506
|
+
# Within the interval because there is no end date
|
507
|
+
Timecop.freeze(Time.new(2030, 1, 1)) do
|
508
|
+
assert e.send(:is_scheduled?)
|
509
|
+
end
|
510
|
+
# Too early
|
511
|
+
Timecop.freeze(Time.new(2019, 12, 30)) do
|
512
|
+
assert !e.send(:is_scheduled?)
|
513
|
+
end
|
514
|
+
end
|
515
|
+
|
516
|
+
def test_is_scheduled_no_start_timestamp
|
517
|
+
e = Verdict::Experiment.new(:json) do
|
518
|
+
groups do
|
519
|
+
group :a, :half
|
520
|
+
group :b, :rest
|
521
|
+
end
|
522
|
+
schedule_end_timestamp Time.new(2020, 1, 3)
|
523
|
+
end
|
524
|
+
|
525
|
+
# Within the interval because there is no start date
|
526
|
+
Timecop.freeze(Time.new(2019, 12, 30)) do
|
527
|
+
assert e.send(:is_scheduled?)
|
528
|
+
end
|
529
|
+
# Too late
|
530
|
+
Timecop.freeze(Time.new(2020, 1, 4)) do
|
531
|
+
assert !e.send(:is_scheduled?)
|
532
|
+
end
|
533
|
+
end
|
534
|
+
|
535
|
+
def test_switch_respects_time_schedule
|
536
|
+
e = Verdict::Experiment.new('test') do
|
537
|
+
groups do
|
538
|
+
group :a, :half
|
539
|
+
group :b, :rest
|
540
|
+
end
|
541
|
+
schedule_start_timestamp Time.new(2020, 1, 1)
|
542
|
+
schedule_end_timestamp Time.new(2020, 1, 2)
|
543
|
+
end
|
544
|
+
|
545
|
+
Timecop.freeze(Time.new(2020, 1, 3)) do
|
546
|
+
assert_nil e.switch(1)
|
547
|
+
end
|
548
|
+
end
|
549
|
+
|
550
|
+
def test_switch_respects_time_schedule_even_after_assignment
|
551
|
+
e = Verdict::Experiment.new('test') do
|
552
|
+
groups do
|
553
|
+
group :a, :half
|
554
|
+
group :b, :rest
|
555
|
+
end
|
556
|
+
end
|
557
|
+
|
558
|
+
assert_equal :a, e.switch(1)
|
559
|
+
|
560
|
+
e.schedule_start_timestamp Time.new(2020, 1, 1)
|
561
|
+
e.schedule_end_timestamp Time.new(2020, 1, 2)
|
562
|
+
|
563
|
+
Timecop.freeze(Time.new(2020, 1, 3)) do
|
564
|
+
assert_nil e.switch(1)
|
565
|
+
end
|
566
|
+
end
|
567
|
+
|
568
|
+
def test_is_stop_new_assignments
|
569
|
+
e = Verdict::Experiment.new('test') do
|
570
|
+
groups do
|
571
|
+
group :a, :half
|
572
|
+
group :b, :half
|
573
|
+
end
|
574
|
+
schedule_stop_new_assignment_timestamp Time.new(2020, 1, 15)
|
575
|
+
end
|
576
|
+
|
577
|
+
# new assignments stopped after the stop timestamp
|
578
|
+
Timecop.freeze(Time.new(2020, 1, 16)) do
|
579
|
+
assert !e.send(:is_make_new_assignments?)
|
580
|
+
assert_nil e.switch(1)
|
581
|
+
end
|
582
|
+
# new assignments didn't stop before the stop timestamp
|
583
|
+
Timecop.freeze(Time.new(2020, 1, 3)) do
|
584
|
+
assert e.send(:is_make_new_assignments?)
|
585
|
+
assert :a, e.switch(2)
|
586
|
+
end
|
587
|
+
end
|
588
|
+
|
589
|
+
def test_switch_preserves_old_assignments_after_stop_new_assignments_timestamp
|
590
|
+
e = Verdict::Experiment.new('test') do
|
591
|
+
groups do
|
592
|
+
group :a, :half
|
593
|
+
group :b, :half
|
594
|
+
end
|
595
|
+
end
|
596
|
+
|
597
|
+
assert_equal :a, e.switch(1)
|
598
|
+
|
599
|
+
e.schedule_stop_new_assignment_timestamp Time.new(2020, 4, 15)
|
600
|
+
|
601
|
+
# switch respects to stop new assignment timestamp, old assignment preserves, new assignment returns nil
|
602
|
+
Timecop.freeze(Time.new(2020, 4, 16)) do
|
603
|
+
assert !e.send(:is_make_new_assignments?)
|
604
|
+
# old assignment stay the same
|
605
|
+
assert_equal :a, e.switch(1)
|
606
|
+
# new assignment returns nil
|
607
|
+
assert_nil e.switch(2)
|
608
|
+
end
|
609
|
+
end
|
610
|
+
|
611
|
+
def test_schedule_start_timestamp_and_stop_new_assignemnt_timestamp_are_inclusive_but_end_timestamp_is_exclusive
|
612
|
+
e = Verdict::Experiment.new('test') do
|
613
|
+
groups do
|
614
|
+
group :a, :half
|
615
|
+
group :b, :half
|
616
|
+
end
|
617
|
+
|
618
|
+
schedule_start_timestamp Time.new(2020, 1, 1)
|
619
|
+
schedule_stop_new_assignment_timestamp Time.new(2020, 1, 15)
|
620
|
+
schedule_end_timestamp Time.new(2020, 1, 31)
|
621
|
+
end
|
622
|
+
|
623
|
+
# start_timestamp is included
|
624
|
+
Timecop.freeze(Time.new(2020, 1, 1)) do
|
625
|
+
assert e.send(:is_scheduled?)
|
626
|
+
assert_equal :a, e.switch(1)
|
627
|
+
end
|
628
|
+
|
629
|
+
# stop_new_assignment_timestamp is included
|
630
|
+
Timecop.freeze(Time.new(2020, 1, 15)) do
|
631
|
+
assert !e.send(:is_make_new_assignments?)
|
632
|
+
# old assignment preserved
|
633
|
+
assert_equal :a, e.switch(1)
|
634
|
+
# new assignment returns nil
|
635
|
+
assert_nil e.switch(2)
|
636
|
+
end
|
637
|
+
|
638
|
+
# end_timestamp is excluded
|
639
|
+
Timecop.freeze(Time.new(2020, 1, 31)) do
|
640
|
+
assert !e.send(:is_scheduled?)
|
641
|
+
assert_nil e.switch(1)
|
642
|
+
end
|
643
|
+
end
|
644
|
+
|
645
|
+
def test_custom_qualifiers_success
|
646
|
+
e = Verdict::Experiment.new('test') do
|
647
|
+
groups do
|
648
|
+
group :all, 100
|
649
|
+
end
|
650
|
+
end
|
651
|
+
|
652
|
+
subject = 2
|
653
|
+
custom_qualifier_a = Proc.new { |subject| subject.even? }
|
654
|
+
custom_qualifier_b = Proc.new { |subject| subject > 0 }
|
655
|
+
|
656
|
+
group = e.switch(subject, qualifiers: [custom_qualifier_a, custom_qualifier_b])
|
657
|
+
assert_equal e.group(:all).to_sym, group
|
658
|
+
end
|
659
|
+
|
660
|
+
def test_custom_qualifiers_failure
|
661
|
+
e = Verdict::Experiment.new('test') do
|
662
|
+
groups do
|
663
|
+
group :all, 100
|
664
|
+
end
|
665
|
+
end
|
666
|
+
|
667
|
+
subject = 3
|
668
|
+
custom_qualifier_a = Proc.new { |subject| subject.even? }
|
669
|
+
custom_qualifier_b = Proc.new { |subject| subject > 0 }
|
670
|
+
|
671
|
+
e.switch(subject, qualifiers: [custom_qualifier_a, custom_qualifier_b])
|
672
|
+
|
673
|
+
group = e.switch(subject, qualifiers: [custom_qualifier_a, custom_qualifier_b])
|
674
|
+
assert_nil group
|
675
|
+
end
|
676
|
+
|
677
|
+
private
|
678
|
+
|
679
|
+
def redis
|
680
|
+
@redis ||= ::Redis.new(host: REDIS_HOST, port: REDIS_PORT)
|
681
|
+
end
|
453
682
|
end
|
data/test/fake_app.rb
CHANGED
@@ -0,0 +1,81 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'test_helper'
|
4
|
+
require 'fake_app'
|
5
|
+
|
6
|
+
class CookieStorageTest < Minitest::Test
|
7
|
+
def setup
|
8
|
+
@storage = Verdict::Storage::CookieStorage.new.tap do |s|
|
9
|
+
s.cookies = ActionDispatch::Cookies::CookieJar.new(nil)
|
10
|
+
end
|
11
|
+
@experiment = Verdict::Experiment.new(:cookie_storage_test) do
|
12
|
+
groups { group :all, 100 }
|
13
|
+
storage @storage, store_unqualified: true
|
14
|
+
end
|
15
|
+
@subject = stub(id: 'bob')
|
16
|
+
@assignment = Verdict::Assignment.new(@experiment, @subject, @experiment.group(:all), nil)
|
17
|
+
end
|
18
|
+
|
19
|
+
def test_cookie_lifespan_has_a_default
|
20
|
+
cookie_lifespan = Verdict::Storage::CookieStorage.new.cookie_lifespan
|
21
|
+
|
22
|
+
assert_equal Verdict::Storage::CookieStorage::DEFAULT_COOKIE_LIFESPAN_SECONDS, cookie_lifespan
|
23
|
+
end
|
24
|
+
|
25
|
+
def test_cookie_lifespan_can_be_configured
|
26
|
+
storage = Verdict::Storage::CookieStorage.new(cookie_lifespan: 60)
|
27
|
+
|
28
|
+
assert_equal 60, storage.cookie_lifespan
|
29
|
+
end
|
30
|
+
|
31
|
+
def test_raises_storage_error_when_cookies_is_nil
|
32
|
+
storage = Verdict::Storage::CookieStorage.new
|
33
|
+
|
34
|
+
assert_raises(Verdict::StorageError) { storage.store_assignment(@assignment) }
|
35
|
+
assert_raises(Verdict::StorageError) { storage.retrieve_assignment(@experiment, @subject) }
|
36
|
+
assert_raises(Verdict::StorageError) { storage.remove_assignment(@experiment, @subject) }
|
37
|
+
end
|
38
|
+
|
39
|
+
def test_store_assignment_returns_true_when_an_assignment_is_stored
|
40
|
+
assert @storage.store_assignment(@assignment)
|
41
|
+
refute_nil @storage.retrieve_assignment(@experiment, @subject)
|
42
|
+
end
|
43
|
+
|
44
|
+
def test_retrieve_assignment_returns_an_assignment
|
45
|
+
@storage.store_assignment(@assignment)
|
46
|
+
assignment = @storage.retrieve_assignment(@experiment, @subject)
|
47
|
+
|
48
|
+
assert assignment.returning?
|
49
|
+
assert_equal :all, assignment.handle.to_sym
|
50
|
+
assert_equal @experiment, assignment.experiment
|
51
|
+
assert_equal @subject, assignment.subject
|
52
|
+
end
|
53
|
+
|
54
|
+
def test_retrieve_assignment_returns_nil_when_an_assignment_does_not_exist
|
55
|
+
assert_nil @storage.retrieve_assignment(@experiment, @subject)
|
56
|
+
end
|
57
|
+
|
58
|
+
def test_retrieve_assignment_returns_nil_when_the_assignment_group_is_invalid
|
59
|
+
invalid_group = Verdict::Group.new(@experiment, :invalid)
|
60
|
+
assignment = Verdict::Assignment.new(@experiment, @subject, invalid_group, nil)
|
61
|
+
|
62
|
+
@storage.store_assignment(assignment)
|
63
|
+
|
64
|
+
assert_nil @storage.retrieve_assignment(@experiment, @subject)
|
65
|
+
end
|
66
|
+
|
67
|
+
def test_remove_assignment_returns_true_when_removing_an_assignment
|
68
|
+
@storage.store_assignment(@assignment)
|
69
|
+
|
70
|
+
assert @storage.remove_assignment(@experiment, nil)
|
71
|
+
assert_nil @storage.retrieve_assignment(@experiment, @subject)
|
72
|
+
end
|
73
|
+
|
74
|
+
def test_retrieve_start_timestamp_always_returns_nil
|
75
|
+
assert_nil @storage.retrieve_start_timestamp(nil)
|
76
|
+
end
|
77
|
+
|
78
|
+
def test_store_start_timestamp_raises_not_implemented_error
|
79
|
+
assert_raises(NotImplementedError) { @storage.store_start_timestamp(nil, nil) }
|
80
|
+
end
|
81
|
+
end
|
@@ -35,9 +35,9 @@ class MemoryStorageTest < Minitest::Test
|
|
35
35
|
end
|
36
36
|
|
37
37
|
def test_started_at
|
38
|
-
assert @storage.get
|
38
|
+
assert @storage.send(:get, @experiment.handle.to_s, 'started_at').nil?
|
39
39
|
@experiment.send(:ensure_experiment_has_started)
|
40
|
-
refute @storage.get
|
40
|
+
refute @storage.send(:get, @experiment.handle.to_s, 'started_at').nil?
|
41
41
|
assert_instance_of Time, @experiment.started_at
|
42
42
|
end
|
43
43
|
end
|
@@ -71,17 +71,29 @@ class RedisStorageTest < Minitest::Test
|
|
71
71
|
assert_equal a, @experiment.started_at
|
72
72
|
end
|
73
73
|
|
74
|
-
def
|
74
|
+
def test_cleanup_with_scope_argument
|
75
75
|
1000.times do |n|
|
76
76
|
@experiment.assign("something_#{n}")
|
77
77
|
end
|
78
78
|
|
79
79
|
assert_operator @redis, :exists, experiment_key
|
80
80
|
|
81
|
+
Verdict.default_logger.expects(:warn).with(regexp_matches(/deprecated/))
|
81
82
|
@storage.cleanup(:redis_storage)
|
82
83
|
refute_operator @redis, :exists, experiment_key
|
83
84
|
end
|
84
85
|
|
86
|
+
def test_cleanup
|
87
|
+
1000.times do |n|
|
88
|
+
@experiment.assign("something_#{n}")
|
89
|
+
end
|
90
|
+
|
91
|
+
assert_operator @redis, :exists, experiment_key
|
92
|
+
|
93
|
+
@storage.cleanup(@experiment)
|
94
|
+
refute_operator @redis, :exists, experiment_key
|
95
|
+
end
|
96
|
+
|
85
97
|
private
|
86
98
|
|
87
99
|
def experiment_key
|
data/test/verdict_rails_test.rb
CHANGED
@@ -5,9 +5,16 @@ class VerdictRailsTest < Minitest::Test
|
|
5
5
|
def setup
|
6
6
|
Verdict.clear_repository_cache
|
7
7
|
new_rails_app = Dummy::Application.new
|
8
|
+
new_rails_app.config.eager_load = false
|
8
9
|
new_rails_app.initialize!
|
9
10
|
end
|
10
11
|
|
12
|
+
def teardown
|
13
|
+
Verdict.default_logger = Logger.new("/dev/null")
|
14
|
+
Verdict.directory = nil
|
15
|
+
Verdict.clear_repository_cache
|
16
|
+
end
|
17
|
+
|
11
18
|
def test_verdict_railtie_should_find_directory_path
|
12
19
|
assert_equal Verdict.directory, Rails.root.join('app', 'experiments')
|
13
20
|
end
|
@@ -16,4 +23,4 @@ class VerdictRailsTest < Minitest::Test
|
|
16
23
|
expected_experiment = Verdict.instance_variable_get('@repository')
|
17
24
|
assert expected_experiment.include?("test_rails_app_experiment")
|
18
25
|
end
|
19
|
-
end
|
26
|
+
end
|
data/verdict.gemspec
CHANGED
@@ -17,6 +17,8 @@ Gem::Specification.new do |gem|
|
|
17
17
|
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
18
18
|
gem.require_paths = ["lib"]
|
19
19
|
|
20
|
+
gem.metadata["allowed_push_host"] = "https://rubygems.org"
|
21
|
+
|
20
22
|
gem.add_development_dependency("minitest", '~> 5.2')
|
21
23
|
gem.add_development_dependency("rake")
|
22
24
|
gem.add_development_dependency("mocha")
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: verdict
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.15.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Shopify
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2020-07-08 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: minitest
|
@@ -106,10 +106,12 @@ files:
|
|
106
106
|
- ".gitignore"
|
107
107
|
- ".travis.yml"
|
108
108
|
- CHANGELOG.md
|
109
|
+
- CONTRIBUTING.md
|
109
110
|
- Gemfile
|
110
111
|
- LICENSE
|
111
112
|
- README.md
|
112
113
|
- Rakefile
|
114
|
+
- docs/concepts.md
|
113
115
|
- lib/verdict.rb
|
114
116
|
- lib/verdict/assignment.rb
|
115
117
|
- lib/verdict/conversion.rb
|
@@ -126,6 +128,7 @@ files:
|
|
126
128
|
- lib/verdict/segmenters/static_segmenter.rb
|
127
129
|
- lib/verdict/storage.rb
|
128
130
|
- lib/verdict/storage/base_storage.rb
|
131
|
+
- lib/verdict/storage/cookie_storage.rb
|
129
132
|
- lib/verdict/storage/legacy_redis_storage.rb
|
130
133
|
- lib/verdict/storage/memory_storage.rb
|
131
134
|
- lib/verdict/storage/mock_storage.rb
|
@@ -148,6 +151,7 @@ files:
|
|
148
151
|
- test/segmenters/rollout_segmenter_test.rb
|
149
152
|
- test/segmenters/static_segmenter_test.rb
|
150
153
|
- test/storage/base_storage_test.rb
|
154
|
+
- test/storage/cookie_storage_test.rb
|
151
155
|
- test/storage/legacy_redis_storage_test.rb
|
152
156
|
- test/storage/memory_storage_test.rb
|
153
157
|
- test/storage/redis_storage_test.rb
|
@@ -156,7 +160,8 @@ files:
|
|
156
160
|
- verdict.gemspec
|
157
161
|
homepage: http://github.com/Shopify/verdict
|
158
162
|
licenses: []
|
159
|
-
metadata:
|
163
|
+
metadata:
|
164
|
+
allowed_push_host: https://rubygems.org
|
160
165
|
post_install_message:
|
161
166
|
rdoc_options: []
|
162
167
|
require_paths:
|
@@ -193,6 +198,7 @@ test_files:
|
|
193
198
|
- test/segmenters/rollout_segmenter_test.rb
|
194
199
|
- test/segmenters/static_segmenter_test.rb
|
195
200
|
- test/storage/base_storage_test.rb
|
201
|
+
- test/storage/cookie_storage_test.rb
|
196
202
|
- test/storage/legacy_redis_storage_test.rb
|
197
203
|
- test/storage/memory_storage_test.rb
|
198
204
|
- test/storage/redis_storage_test.rb
|