determinator 0.10.0 → 0.11.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -0
- data/README.md +13 -8
- data/docs/background.md +42 -0
- data/docs/local_development.md +72 -0
- data/examples/determinator-rails/app/controllers/application_controller.rb +6 -1
- data/lib/determinator.rb +10 -2
- data/lib/determinator/control.rb +10 -2
- data/lib/determinator/retrieve/routemaster.rb +3 -4
- data/lib/determinator/version.rb +1 -1
- data/lib/rspec/determinator.rb +1 -1
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e34898ef5ebd55998c28389b7644266fff00ca90
|
4
|
+
data.tar.gz: 01497458ea7daf49b59fb70c9214e7a2ae4c14cf
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8ec544cbdd744dc830e2432ee98af864361427ea543883b4db6227d72fcdf61ab2345bba6abd36df5abccacd67f99f2f4524a85338e5eea8fac5dbca32f2afe5
|
7
|
+
data.tar.gz: c68d088da9eba87eba25e3ecc89ae20122801a9be93c183c18c96c2e4b71b0081fc2d2e1b4c450400dd8945d4f1f54c47b3c00a32f60ad31af04a8d7da6b5b22
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,10 @@
|
|
1
|
+
# v0.11.0 (2017-10-13)
|
2
|
+
|
3
|
+
Bug fix:
|
4
|
+
- Ensure constraints and properties are string-keyed so that they match regardless of which are used (#33)
|
5
|
+
- Be more permissive with the situations where `Rspec::Determinator` can be used (#34)
|
6
|
+
- Swallow not found errors from routemaster so determinator isn't too shouty, allow them to be tracked. (#35)
|
7
|
+
|
1
8
|
# v0.10.0 (2017-09-15)
|
2
9
|
|
3
10
|
Feature:
|
data/README.md
CHANGED
@@ -1,9 +1,15 @@
|
|
1
1
|
# Determinator
|
2
2
|
|
3
|
-
A gem that works with
|
3
|
+
A gem that works with _Florence_ to deterministically calculate whether an **actor** should have a feature flag turned on or off, or which variant they should see in an experiment.
|
4
4
|
|
5
5
|
![Determinator](docs/img/determinator.jpg)
|
6
6
|
|
7
|
+
Useful documentation:
|
8
|
+
|
9
|
+
- [Terminology and Background](docs/background.md)
|
10
|
+
- [Local development](docs/local_development.md)
|
11
|
+
- [Example implemention in Rails](examples/determinator-rails)
|
12
|
+
|
7
13
|
## Usage
|
8
14
|
|
9
15
|
Once [set up](#installation), determinator can be used to determine whether a **feature flag** or **experiment** is on or off for the current user and, for experiments, which **variant** they should see.
|
@@ -25,16 +31,14 @@ when 'velociraptors'
|
|
25
31
|
end
|
26
32
|
```
|
27
33
|
|
28
|
-
Feature flags and
|
29
|
-
|
30
|
-
Constraints must be strings, what matches and doesn't is configurable after-the-fact within Florence.
|
34
|
+
Feature flags and experiments can be targeted to specific actors by specifying actor properties (which must match the constraints defined in the feature).
|
31
35
|
|
32
36
|
```ruby
|
33
|
-
#
|
37
|
+
# Targeting specific actors
|
34
38
|
variant = determinator.which_variant(
|
35
39
|
:my_experiment_name,
|
36
40
|
properties: {
|
37
|
-
|
41
|
+
employee: current_user.employee?
|
38
42
|
}
|
39
43
|
)
|
40
44
|
```
|
@@ -53,10 +57,11 @@ Check the example Rails app in `examples` for more information on how to make us
|
|
53
57
|
require 'determinator/retrieve/routemaster'
|
54
58
|
Determinator.configure(
|
55
59
|
retrieval: Determinator::Retrieve::Routemaster.new(
|
56
|
-
discovery_url: 'https://
|
60
|
+
discovery_url: 'https://flo.dev/'
|
57
61
|
retrieval_cache: ActiveSupport::Cache::MemoryStore.new(expires_in: 1.minute)
|
58
62
|
),
|
59
|
-
errors: -> error { NewRelic::Agent.notice_error(error) }
|
63
|
+
errors: -> error { NewRelic::Agent.notice_error(error) },
|
64
|
+
missing_features: -> feature_name { STATSD.increment 'determinator.missing_feature', tags: ["feature:#{name}"] }
|
60
65
|
)
|
61
66
|
```
|
62
67
|
|
data/docs/background.md
ADDED
@@ -0,0 +1,42 @@
|
|
1
|
+
# Terminology & Background
|
2
|
+
|
3
|
+
**Florence** is a suite of tools which help run experiments and feature flags (_collectively called **features**_), **Determinator** is the client-side component which implements the algorithm for figuring out what to show to whom.
|
4
|
+
|
5
|
+
**Feature flags** are used as a way to switch on and off functionality for specific actors across an entire ecosystem, where an **actor** might be a customer, a rider, or any identifiable agent using those systems.
|
6
|
+
|
7
|
+
**Experiments** are, at this stage, really just [A/B tests](https://en.wikipedia.org/wiki/A/B_testing) where an actor is repeatably shown either one **variant** of the product or another. The activity of large numbers of actors can be analysed to determine if one variant was, statistically speaking, better than the other for a given metric.
|
8
|
+
|
9
|
+
## Targeting actors
|
10
|
+
|
11
|
+
Florence also provides a very flexible way to target specific actors for new features or experiments. Every feature has one or more **target groups** associated with it each for which specifies a _rollout_ fraction and any number of _constraints_.
|
12
|
+
|
13
|
+
For a given feature an actor is part of a target group if the actor's **properties** are a match with the target group's **constraints** (ie. the feature's constraints are a subset of the actor's properties). For example, a customer may have a property `employee: 'false'`; this actor would _not_ be part of a target group with the constraint `employee: 'true'`, but _would_ be part of a target group with no constraints.
|
14
|
+
|
15
|
+
If an actor is in no target groups, then the feature (whether it is a feature flag or an experiment) will be off for that actor. If an actor is in more than one target group, then the most permissive target group is chosen.
|
16
|
+
|
17
|
+
Target groups also have a **rollout** fraction which represents how many actors should be included or excluded from the feature. For example this allows a feature to be on for 95% of people using it, or to be rolled out to 100% of employees, but only 5% of non-employees.
|
18
|
+
|
19
|
+
## Experiments vs. Feature Flags
|
20
|
+
|
21
|
+
Experiments in Florence are Feature flags which also allocate an experimental **variant** to the actors invovled. The variants chosen are also chosen deterministically, so the same actor will always see the same experiment.
|
22
|
+
|
23
|
+
For example an experiment which tested whether the 🎉 or the 🙌 emoji was better in a given situation by showing 80% of all non-employees one or the other (in a 50/50 split) would have:
|
24
|
+
|
25
|
+
- One target group, with a rollout of 80% and a single constraint that `employee` must be `false`
|
26
|
+
- Two variants, one called `party popper` and one called `high ten`, with the same **weight** as each other (so the split is equal)
|
27
|
+
|
28
|
+
## Determinism
|
29
|
+
|
30
|
+
Whether an actor is rolled out to or not and which variant they see is calculated [deterministically](https://en.wikipedia.org/wiki/Deterministic_system). This is so that two isolated systems, if they have the same list of experiments and feature flags, will have _the same outcomes_ for every actor, feature flag and experiment.
|
31
|
+
|
32
|
+
In order to ensure that the same actor sees the same things _every time_ a [cryptographically secure hashing function](https://en.wikipedia.org/wiki/Cryptographic_hash_function) is applied to a combination of an actor's identifier (eg. their ID or their anonymous ID) and the feature's name and the resulting information is used to determine what the specified actor will see.
|
33
|
+
|
34
|
+
This is so that there is no need for a centralised database of which actors should be shown which variants, and which actors should be in hold out groups for experiments (this removes the need for [locks](https://en.wikipedia.org/wiki/Lock_%28computer_science%29) which become very complex in distributed environments).
|
35
|
+
|
36
|
+
This algorithm is also organised so that, when increasing and lowering rollout fractions, the same actors will be included and excluded at each fraction.
|
37
|
+
|
38
|
+
## Overrides
|
39
|
+
|
40
|
+
The determination algorithm also allows **overrides** which allow specified actors to receive the given outcome even if the algorithm would normally give them another. This is particularly helpful for testing in production environments, where a product manager might have a feature turned on for only them, or to switch themselves between the two variants of an experiment to ensure both work as expected.
|
41
|
+
|
42
|
+
Overrides should be used sparingly and only for temporary changes; for situations where even a small group of actors should see a specific feature consider whether the actor has an attribute which defines whether they should see it or not, and instead deliver that as a property so that a target group can specify just them. A simple example of this is 'VIPs', rather than specifying them as `override: true` for a feature flag, they should instead have the property `vip: true`, with an equivalent constraint on a 100% rollout target group.
|
@@ -0,0 +1,72 @@
|
|
1
|
+
# Local development
|
2
|
+
|
3
|
+
Because Determinator depends on features defined elsewhere, local development is supported by two pieces of software.
|
4
|
+
|
5
|
+
## `RSpec::Determinator` for automated testing
|
6
|
+
|
7
|
+
When writing tests for your code that makes use of Determinator you can use some RSpec helpers:
|
8
|
+
|
9
|
+
```ruby
|
10
|
+
require 'rspec/determinator'
|
11
|
+
|
12
|
+
RSpec.describe YourClass, :determinator_support do
|
13
|
+
|
14
|
+
context "when the actor is in variant_a" do
|
15
|
+
# This allows testing of the experiment being in a specific variant
|
16
|
+
forced_determination(:experiment_name, 'variant_a')
|
17
|
+
|
18
|
+
it "should respond in a way that is defined by variant_a"
|
19
|
+
end
|
20
|
+
|
21
|
+
context "when the actor is not in the experiment" do
|
22
|
+
# This allows testing of the experiment being off
|
23
|
+
forced_determination(:experiment_name, false)
|
24
|
+
|
25
|
+
it "should respond in a way that is defined by being out of the experiment"
|
26
|
+
end
|
27
|
+
|
28
|
+
context "when the actor is not from France" do
|
29
|
+
before { ensure_the_actor_is_not_from_france }
|
30
|
+
# This allows testing of target group constraint functionality
|
31
|
+
forced_determination(:experiment_name, 'variant_b', only_for: { country: 'fr' })
|
32
|
+
|
33
|
+
it "should respond in a way that is defined by being out of the experiment"
|
34
|
+
end
|
35
|
+
|
36
|
+
context "when the actor has a specified id" do
|
37
|
+
before { ensure_the_actor_has_id_123 }
|
38
|
+
# This allows testing of override functionality
|
39
|
+
forced_determination(:experiment_name, 'variant_b', only_for: { id: '123' })
|
40
|
+
|
41
|
+
it "should respond in a way that is defined by variant_b"
|
42
|
+
end
|
43
|
+
end
|
44
|
+
```
|
45
|
+
|
46
|
+
## Fake Florence for local execution
|
47
|
+
|
48
|
+
[Fake Florence](https://github.com/deliveroo/fake_florence) is a command line utility which operates a determinator compatible server and provides tooling for easy editing of feature flags and experiments.
|
49
|
+
|
50
|
+
```bash
|
51
|
+
$ gem install fake_florence
|
52
|
+
Fake Florence has been installed. Run `flo help` for more information.
|
53
|
+
1 gem installed
|
54
|
+
|
55
|
+
$ flo start
|
56
|
+
Flo is now running at https://flo.dev
|
57
|
+
Use other commands to create or edit Feature flags and Experiments.
|
58
|
+
See `flo help` for more information
|
59
|
+
|
60
|
+
$ flo create my_experiment
|
61
|
+
create ~/.flo/features/my_experiment.yaml
|
62
|
+
my_experiment created and opened for editing
|
63
|
+
```
|
64
|
+
|
65
|
+
Then in your service, configured with `discovery_url: 'https://flo.dev'`, experiments and feature flags will retrieved and posted from Fake Florence:
|
66
|
+
|
67
|
+
```ruby
|
68
|
+
determinator.which_variant(:my_experiment, id: 123)
|
69
|
+
"anchovy"
|
70
|
+
```
|
71
|
+
|
72
|
+
More information can be found on the [Fake Florence](https://github.com/deliveroo/fake_florence) project page.
|
@@ -14,7 +14,12 @@ class ApplicationController < ActionController::API
|
|
14
14
|
# which allows simple use throughout the app
|
15
15
|
@_determinator ||= Determinator.instance.for_actor(
|
16
16
|
id: current_user && current_user.id || nil,
|
17
|
-
guid: guid
|
17
|
+
guid: guid,
|
18
|
+
default_properties: {
|
19
|
+
# Clearly this would return real information about whether the
|
20
|
+
# user is an employee.
|
21
|
+
employee: false
|
22
|
+
}
|
18
23
|
)
|
19
24
|
end
|
20
25
|
end
|
data/lib/determinator.rb
CHANGED
@@ -7,9 +7,11 @@ require 'determinator/retrieve/null_retriever'
|
|
7
7
|
|
8
8
|
module Determinator
|
9
9
|
# @param :retrieval [Determinator::Retrieve::Routemaster] A retrieval instance for Features
|
10
|
-
# @param :errors [
|
11
|
-
|
10
|
+
# @param :errors [#call, nil] a proc, accepting an error, which will be called with any errors which occur while determinating
|
11
|
+
# @param :missing_feature [#call, nil] a proc, accepting a feature name, which will be called any time a feature is requested but isn't available
|
12
|
+
def self.configure(retrieval:, errors: nil, missing_feature: nil)
|
12
13
|
@error_logger = errors if errors.respond_to?(:call)
|
14
|
+
@missing_feature_logger = missing_feature if missing_feature.respond_to?(:call)
|
13
15
|
@instance = Control.new(retrieval: retrieval)
|
14
16
|
end
|
15
17
|
|
@@ -23,4 +25,10 @@ module Determinator
|
|
23
25
|
|
24
26
|
@error_logger.call(error)
|
25
27
|
end
|
28
|
+
|
29
|
+
def self.missing_feature(name)
|
30
|
+
return unless @missing_feature_logger
|
31
|
+
|
32
|
+
@missing_feature_logger.call(name)
|
33
|
+
end
|
26
34
|
end
|
data/lib/determinator/control.rb
CHANGED
@@ -57,7 +57,10 @@ module Determinator
|
|
57
57
|
|
58
58
|
def determinate(name, id:, guid:, properties:)
|
59
59
|
feature = retrieval.retrieve(name)
|
60
|
-
|
60
|
+
if feature.nil?
|
61
|
+
Determinator.missing_feature(name)
|
62
|
+
return false
|
63
|
+
end
|
61
64
|
|
62
65
|
# Calling method can place constraints on the feature, eg. experiment only
|
63
66
|
return false if block_given? && !yield(feature)
|
@@ -86,9 +89,14 @@ module Determinator
|
|
86
89
|
end
|
87
90
|
|
88
91
|
def choose_target_group(feature, properties)
|
92
|
+
# Keys must be strings
|
93
|
+
normalised_properties = properties.each_with_object({}) do |(k, v), h|
|
94
|
+
h[k.to_s] = v
|
95
|
+
end
|
96
|
+
|
89
97
|
feature.target_groups.select { |tg|
|
90
98
|
tg.constraints.reduce(true) do |fit, (scope, *required)|
|
91
|
-
present = [*
|
99
|
+
present = [*normalised_properties[scope.to_s]]
|
92
100
|
fit && (required.flatten & present.flatten).any?
|
93
101
|
end
|
94
102
|
# Must choose target group deterministically, if more than one match
|
@@ -31,6 +31,9 @@ module Determinator
|
|
31
31
|
cached_feature_lookup(feature_id) do
|
32
32
|
@actor_service.feature.show(feature_id)
|
33
33
|
end
|
34
|
+
rescue ::Routemaster::Errors::ResourceNotFound
|
35
|
+
# Don't be noisy
|
36
|
+
nil
|
34
37
|
rescue => e
|
35
38
|
Determinator.notice_error(e)
|
36
39
|
nil
|
@@ -47,10 +50,6 @@ module Determinator
|
|
47
50
|
@routemaster_app ||= ::Routemaster::Drain::Caching.new
|
48
51
|
end
|
49
52
|
|
50
|
-
def self.index_cache_key(feature_name)
|
51
|
-
"determinator_index:#{feature_name}"
|
52
|
-
end
|
53
|
-
|
54
53
|
def self.lookup_cache_key(feature_name)
|
55
54
|
"determinator_cache:#{feature_name}"
|
56
55
|
end
|
data/lib/determinator/version.rb
CHANGED
data/lib/rspec/determinator.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: determinator
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.11.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- JP Hastings-Spital
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-
|
11
|
+
date: 2017-10-13 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: routemaster-drain
|
@@ -143,7 +143,9 @@ files:
|
|
143
143
|
- bin/setup
|
144
144
|
- circle.yml
|
145
145
|
- determinator.gemspec
|
146
|
+
- docs/background.md
|
146
147
|
- docs/img/determinator.jpg
|
148
|
+
- docs/local_development.md
|
147
149
|
- examples/determinator-rails/.env
|
148
150
|
- examples/determinator-rails/.gitignore
|
149
151
|
- examples/determinator-rails/Gemfile
|