vinted-ab 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +11 -7
- data/ab.gemspec +2 -0
- data/lib/ab/assigned_test.rb +8 -0
- data/lib/ab/null_test.rb +19 -8
- data/lib/ab/test.rb +6 -3
- data/lib/ab/tests.rb +1 -0
- data/lib/ab/version.rb +1 -1
- data/spec/integration_spec.rb +1 -0
- data/spec/null_test_spec.rb +2 -0
- data/spec/tests_spec.rb +1 -1
- metadata +42 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 67031305b2c5c180288a616ff7d931d4918604dc
|
4
|
+
data.tar.gz: de313c446acea2eda841eea20930c82938628c0c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8cd252fbb99403b6405401584879f9a410450adbaf2b8efa1c942d113e6046089fea519c90c7d4a9ae449e3e45a236acc1fc2ec0d9762071dcde2e2b88e07712
|
7
|
+
data.tar.gz: e0f21c86e0a21c40359da87572cfcfc9a23b955ea4d61c0730b547ea80d16a565e85c9a4c75da3551fbabda8f7c76703ae07c8cb7fa35f3bf3edeb8dc90fe9bd
|
data/README.md
CHANGED
@@ -4,9 +4,9 @@
|
|
4
4
|
[](http://badge.fury.io/rb/vinted-ab)
|
5
5
|
[](https://gemnasium.com/vinted/ab)
|
6
6
|
|
7
|
-
|
7
|
+
If you didn't guess it from the name, this library is meant for ab testing. But it doesn't cover everything associated with it, it lacks configuration and management parts. [vinted/ab](https://github.com/vinted/ab) is only used to determine which variant should be applied for a user. Two inputs are expected - [configuration](#configuration) and identifier. Identifier, at least in Vinted's case, represents users, but other scenarios are certainly possible.
|
8
8
|
|
9
|
-
|
9
|
+
Each identifier is assigned to a bucket, using a hashing function. Buckets can then be assigned to tests. That allows isolation control, when we don't want clashing and creation of biases. Each test also has a seed, which is used to randomise how identifiers are divided among test variants. You can find algorithm description [here](#algorithm) if you want more detail.
|
10
10
|
|
11
11
|

|
12
12
|
|
@@ -19,8 +19,7 @@ ab = Ab::Tests.new(configuration, identifier)
|
|
19
19
|
Ab::Tests.before_picking_variant { |test| puts "picking variant for #{test}" }
|
20
20
|
Ab::Tests.after_picking_variant { |test, variant| puts "#{variant_name}" }
|
21
21
|
|
22
|
-
# ab.test never returns nil
|
23
|
-
# but if you don't belong to any of the buckets, variant will be nil
|
22
|
+
# ab.test never returns nil, but #variant can
|
24
23
|
case ab.test.variant
|
25
24
|
when 'red_button'
|
26
25
|
red_button
|
@@ -32,6 +31,9 @@ end
|
|
32
31
|
|
33
32
|
# calls #variant underneath, results of that call are cached
|
34
33
|
puts 'red button' if ab.test.red_button?
|
34
|
+
|
35
|
+
# both start_at and end_at dates are accessible
|
36
|
+
puts 'newbie button' if user.created_at > ab.test.start_at && ab.test.for_newbies?
|
35
37
|
```
|
36
38
|
|
37
39
|
## Configuration
|
@@ -91,7 +93,7 @@ Short explanation for a couple of config parameters:
|
|
91
93
|
|
92
94
|
`ab_tests.variants`: tests can have multiple variants, each with a name and a weight.
|
93
95
|
|
94
|
-
More examples can be found in [spec/examples](https://github.com/vinted/ab/tree/master/spec/examples). Those examples are part of the test suite, which is run using [this code](https://github.com/vinted/ab/blob/master/spec/integration_spec.rb). We strongly recommend using those examples if you're reimplementing this library in another language.
|
96
|
+
More examples can be found in [spec/examples](https://github.com/vinted/ab/tree/master/spec/examples). Those examples are part of the test suite, which is run using [this code](https://github.com/vinted/ab/blob/master/spec/integration_spec.rb). `input.json` is configuration json and `output.json` gives expectations - which identifiers should fall to which variant. We strongly recommend using those examples if you're reimplementing this library in another language.
|
95
97
|
|
96
98
|
## Algorithm
|
97
99
|
|
@@ -100,12 +102,14 @@ Most of the logic, is in `AssignedTest` class, which can be used as an [example
|
|
100
102
|
Here's some procedural pseudo code to serve as a reference:
|
101
103
|
|
102
104
|
```pseudo
|
103
|
-
|
105
|
+
salted_identifier = salt + identifier.to_string
|
106
|
+
bucket_id = SHA256.hexdigest(salted_identifier).to_int % bucket_count
|
104
107
|
|
105
108
|
return if not (test.all_buckets? or test.buckets.include?(bucket_id))
|
106
109
|
return if not DateTime.now.between?(test.start_at, test.end_at)
|
107
110
|
|
108
111
|
chance_weight_sum = chance_weight_sum > 0 ? test.chance_weight_sum : 1
|
109
|
-
|
112
|
+
seeded_identifier = test.seed + identifier.to_string
|
113
|
+
weight_id = SHA256.hexdigest(seeded_identifier).to_int % chance_weight_sum
|
110
114
|
test.variants.find { |variant| variant.accumulated_chance_weight > weight_id }
|
111
115
|
```
|
data/ab.gemspec
CHANGED
@@ -24,4 +24,6 @@ Gem::Specification.new do |spec|
|
|
24
24
|
spec.add_development_dependency 'rake', '~> 10.1', '>= 10.1.0'
|
25
25
|
spec.add_development_dependency 'rspec', '~> 2.14', '>= 2.14.0'
|
26
26
|
spec.add_development_dependency 'json-schema', '~> 2.2.2', '>= 2.2.2'
|
27
|
+
spec.add_development_dependency 'ruby-prof', '~> 0.15.0', '>= 0.15.0'
|
28
|
+
spec.add_development_dependency 'pry', '~> 0.10.0', '>= 0.10.0'
|
27
29
|
end
|
data/lib/ab/assigned_test.rb
CHANGED
data/lib/ab/null_test.rb
CHANGED
@@ -1,12 +1,23 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
|
1
|
+
module Ab
|
2
|
+
class NullTest
|
3
|
+
def variant
|
4
|
+
end
|
4
5
|
|
5
|
-
|
6
|
-
|
7
|
-
|
6
|
+
def start_at
|
7
|
+
Test::DEFAULT_START_AT
|
8
|
+
end
|
9
|
+
|
10
|
+
def end_at
|
11
|
+
Test::DEFAULT_END_AT
|
12
|
+
end
|
13
|
+
|
14
|
+
|
15
|
+
def method_missing(meth, *args, &block)
|
16
|
+
meth.to_s.end_with?('?') ? false : super
|
17
|
+
end
|
8
18
|
|
9
|
-
|
10
|
-
|
19
|
+
def respond_to?(meth)
|
20
|
+
meth.to_s.end_with?('?') ? true : super
|
21
|
+
end
|
11
22
|
end
|
12
23
|
end
|
data/lib/ab/test.rb
CHANGED
@@ -1,5 +1,8 @@
|
|
1
1
|
module Ab
|
2
2
|
class Test < Struct.new(:hash, :salt, :bucket_count)
|
3
|
+
DEFAULT_START_AT = DateTime.new(0)
|
4
|
+
DEFAULT_END_AT = DateTime.new(3000)
|
5
|
+
|
3
6
|
def buckets
|
4
7
|
hash['buckets']
|
5
8
|
end
|
@@ -26,11 +29,11 @@ module Ab
|
|
26
29
|
end
|
27
30
|
|
28
31
|
def start_at
|
29
|
-
@start_at ||= parse_time('start_at',
|
32
|
+
@start_at ||= parse_time('start_at', DEFAULT_START_AT)
|
30
33
|
end
|
31
34
|
|
32
35
|
def end_at
|
33
|
-
@end_at ||= parse_time('end_at',
|
36
|
+
@end_at ||= parse_time('end_at', DEFAULT_END_AT)
|
34
37
|
end
|
35
38
|
|
36
39
|
def weight_sum
|
@@ -41,7 +44,7 @@ module Ab
|
|
41
44
|
|
42
45
|
def parse_time(name, default)
|
43
46
|
value = hash[name]
|
44
|
-
value.nil? ?
|
47
|
+
value.nil? ? default : DateTime.parse(value)
|
45
48
|
end
|
46
49
|
end
|
47
50
|
end
|
data/lib/ab/tests.rb
CHANGED
data/lib/ab/version.rb
CHANGED
data/spec/integration_spec.rb
CHANGED
data/spec/null_test_spec.rb
CHANGED
data/spec/tests_spec.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: vinted-ab
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mindaugas Mozūras
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-06-
|
12
|
+
date: 2014-06-19 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: hooks
|
@@ -105,6 +105,46 @@ dependencies:
|
|
105
105
|
- - '>='
|
106
106
|
- !ruby/object:Gem::Version
|
107
107
|
version: 2.2.2
|
108
|
+
- !ruby/object:Gem::Dependency
|
109
|
+
name: ruby-prof
|
110
|
+
requirement: !ruby/object:Gem::Requirement
|
111
|
+
requirements:
|
112
|
+
- - ~>
|
113
|
+
- !ruby/object:Gem::Version
|
114
|
+
version: 0.15.0
|
115
|
+
- - '>='
|
116
|
+
- !ruby/object:Gem::Version
|
117
|
+
version: 0.15.0
|
118
|
+
type: :development
|
119
|
+
prerelease: false
|
120
|
+
version_requirements: !ruby/object:Gem::Requirement
|
121
|
+
requirements:
|
122
|
+
- - ~>
|
123
|
+
- !ruby/object:Gem::Version
|
124
|
+
version: 0.15.0
|
125
|
+
- - '>='
|
126
|
+
- !ruby/object:Gem::Version
|
127
|
+
version: 0.15.0
|
128
|
+
- !ruby/object:Gem::Dependency
|
129
|
+
name: pry
|
130
|
+
requirement: !ruby/object:Gem::Requirement
|
131
|
+
requirements:
|
132
|
+
- - ~>
|
133
|
+
- !ruby/object:Gem::Version
|
134
|
+
version: 0.10.0
|
135
|
+
- - '>='
|
136
|
+
- !ruby/object:Gem::Version
|
137
|
+
version: 0.10.0
|
138
|
+
type: :development
|
139
|
+
prerelease: false
|
140
|
+
version_requirements: !ruby/object:Gem::Requirement
|
141
|
+
requirements:
|
142
|
+
- - ~>
|
143
|
+
- !ruby/object:Gem::Version
|
144
|
+
version: 0.10.0
|
145
|
+
- - '>='
|
146
|
+
- !ruby/object:Gem::Version
|
147
|
+
version: 0.10.0
|
108
148
|
description:
|
109
149
|
email:
|
110
150
|
- mindaugas.mozuras@gmail.com
|