determinator 0.10.0 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: ecd2a7d802e9ad2562fbdf9708ff34a84242b91e
-  data.tar.gz: 5b1918806a440a52f4c337286164b79376059fef
+  metadata.gz: e34898ef5ebd55998c28389b7644266fff00ca90
+  data.tar.gz: 01497458ea7daf49b59fb70c9214e7a2ae4c14cf
 SHA512:
-  metadata.gz: f1a2cf800a27d2b5bf007d50507bc8a1394c596cc0563b4b73fa6cb5dabe3ea43f8558b64840df9baad51ed571c2837409f20aef17e3a5ac4c895264326bb56f
-  data.tar.gz: 64035bfedb2ed42cb72f27bf48cc09d875a1764f5ecec945f8a409a95c8aab09ac66893b0ef9432d4852689093ccc1c3d137e248285e00dfc27826d0479f4116
+  metadata.gz: 8ec544cbdd744dc830e2432ee98af864361427ea543883b4db6227d72fcdf61ab2345bba6abd36df5abccacd67f99f2f4524a85338e5eea8fac5dbca32f2afe5
+  data.tar.gz: c68d088da9eba87eba25e3ecc89ae20122801a9be93c183c18c96c2e4b71b0081fc2d2e1b4c450400dd8945d4f1f54c47b3c00a32f60ad31af04a8d7da6b5b22

data/CHANGELOG.md

@@ -1,3 +1,10 @@
+# v0.11.0 (2017-10-13)
+
+Bug fix:
+- Ensure constraints and properties are string-keyed so that they match regardless of which are used (#33)
+- Be more permissive with the situations where `Rspec::Determinator` can be used (#34)
+- Swallow not found errors from routemaster so determinator isn't too shouty, allow them to be tracked. (#35)
+
 # v0.10.0 (2017-09-15)
 
 Feature:

data/README.md

@@ -1,9 +1,15 @@
 # Determinator
 
-A gem that works with [Florence](https://github.com/deliveroo/actor-tracking) to deterministically calculate whether an actor (a customer, rider, restaurant or employee) should have a feature flag turned on or off, or which variant they should see in an experiment.
+A gem that works with _Florence_ to deterministically calculate whether an **actor** should have a feature flag turned on or off, or which variant they should see in an experiment.
 
 ![Determinator](docs/img/determinator.jpg)
 
+Useful documentation:
+
+- [Terminology and Background](docs/background.md)
+- [Local development](docs/local_development.md)
+- [Example implemention in Rails](examples/determinator-rails)
+
 ## Usage
 
 Once [set up](#installation), determinator can be used to determine whether a **feature flag** or **experiment** is on or off for the current user and, for experiments, which **variant** they should see.
@@ -25,16 +31,14 @@ when 'velociraptors'
 end
 ```
 
-Feature flags and Experiments can be configured to have string based constraints. When the experiment's _constraints_ do not match the given actor's _properties_, the flag or experiment will always be off. When they match the rollout specified by the feature will be applied.
-
-Constraints must be strings, what matches and doesn't is configurable after-the-fact within Florence.
+Feature flags and experiments can be targeted to specific actors by specifying actor properties (which must match the constraints defined in the feature).
 
 ```ruby
-# Constraints
+# Targeting specific actors
 variant = determinator.which_variant(
   :my_experiment_name,
   properties: {
-    country_of_first_order: current_user.orders.first.country.tld,
+    employee: current_user.employee?
   }
 )
 ```
@@ -53,10 +57,11 @@ Check the example Rails app in `examples` for more information on how to make us
 require 'determinator/retrieve/routemaster'
 Determinator.configure(
   retrieval: Determinator::Retrieve::Routemaster.new(
-    discovery_url: 'https://florence.dev/'
+    discovery_url: 'https://flo.dev/'
     retrieval_cache: ActiveSupport::Cache::MemoryStore.new(expires_in: 1.minute)
   ),
-  errors: -> error { NewRelic::Agent.notice_error(error) }
+  errors: -> error { NewRelic::Agent.notice_error(error) },
+  missing_features: -> feature_name { STATSD.increment 'determinator.missing_feature', tags: ["feature:#{name}"] }
 )
 ```
 

data/docs/background.md

@@ -0,0 +1,42 @@
+# Terminology & Background
+
+**Florence** is a suite of tools which help run experiments and feature flags (_collectively called **features**_), **Determinator** is the client-side component which implements the algorithm for figuring out what to show to whom.
+
+**Feature flags** are used as a way to switch on and off functionality for specific actors across an entire ecosystem, where an **actor** might be a customer, a rider, or any identifiable agent using those systems.
+
+**Experiments** are, at this stage, really just [A/B tests](https://en.wikipedia.org/wiki/A/B_testing) where an actor is repeatably shown either one **variant** of the product or another. The activity of large numbers of actors can be analysed to determine if one variant was, statistically speaking, better than the other for a given metric.
+
+## Targeting actors
+
+Florence also provides a very flexible way to target specific actors for new features or experiments. Every feature has one or more **target groups** associated with it each for which specifies a _rollout_ fraction and any number of _constraints_.
+
+For a given feature an actor is part of a target group if the actor's **properties** are a match with the target group's **constraints** (ie. the feature's constraints are a subset of the actor's properties). For example, a customer may have a property `employee: 'false'`; this actor would _not_ be part of a target group with the constraint `employee: 'true'`, but _would_ be part of a target group with no constraints.
+
+If an actor is in no target groups, then the feature (whether it is a feature flag or an experiment) will be off for that actor. If an actor is in more than one target group, then the most permissive target group is chosen.
+
+Target groups also have a **rollout** fraction which represents how many actors should be included or excluded from the feature. For example this allows a feature to be on for 95% of people using it, or to be rolled out to 100% of employees, but only 5% of non-employees.
+
+## Experiments vs. Feature Flags
+
+Experiments in Florence are Feature flags which also allocate an experimental **variant** to the actors invovled. The variants chosen are also chosen deterministically, so the same actor will always see the same experiment.
+
+For example an experiment which tested whether the 🎉 or the 🙌 emoji was better in a given situation by showing 80% of all non-employees one or the other (in a 50/50 split) would have:
+
+- One target group, with a rollout of 80% and a single constraint that `employee` must be `false`
+- Two variants, one called `party popper` and one called `high ten`, with the same **weight** as each other (so the split is equal)
+
+## Determinism
+
+Whether an actor is rolled out to or not and which variant they see is calculated [deterministically](https://en.wikipedia.org/wiki/Deterministic_system). This is so that two isolated systems, if they have the same list of experiments and feature flags, will have _the same outcomes_ for every actor, feature flag and experiment.
+
+In order to ensure that the same actor sees the same things _every time_ a [cryptographically secure hashing function](https://en.wikipedia.org/wiki/Cryptographic_hash_function) is applied to a combination of an actor's identifier (eg. their ID or their anonymous ID) and the feature's name and the resulting information is used to determine what the specified actor will see.
+
+This is so that there is no need for a centralised database of which actors should be shown which variants, and which actors should be in hold out groups for experiments (this removes the need for [locks](https://en.wikipedia.org/wiki/Lock_%28computer_science%29) which become very complex in distributed environments).
+
+This algorithm is also organised so that, when increasing and lowering rollout fractions, the same actors will be included and excluded at each fraction.
+
+## Overrides
+
+The determination algorithm also allows **overrides** which allow specified actors to receive the given outcome even if the algorithm would normally give them another. This is particularly helpful for testing in production environments, where a product manager might have a feature turned on for only them, or to switch themselves between the two variants of an experiment to ensure both work as expected.
+
+Overrides should be used sparingly and only for temporary changes; for situations where even a small group of actors should see a specific feature consider whether the actor has an attribute which defines whether they should see it or not, and instead deliver that as a property so that a target group can specify just them. A simple example of this is 'VIPs', rather than specifying them as `override: true` for a feature flag, they should instead have the property `vip: true`, with an equivalent constraint on a 100% rollout target group.

data/docs/local_development.md

@@ -0,0 +1,72 @@
+# Local development
+
+Because Determinator depends on features defined elsewhere, local development is supported by two pieces of software.
+
+## `RSpec::Determinator` for automated testing
+
+When writing tests for your code that makes use of Determinator you can use some RSpec helpers:
+
+```ruby
+require 'rspec/determinator'
+
+RSpec.describe YourClass, :determinator_support do
+
+  context "when the actor is in variant_a" do
+    # This allows testing of the experiment being in a specific variant
+    forced_determination(:experiment_name, 'variant_a')
+
+    it "should respond in a way that is defined by variant_a"
+  end
+
+  context "when the actor is not in the experiment" do
+    # This allows testing of the experiment being off
+    forced_determination(:experiment_name, false)
+
+    it "should respond in a way that is defined by being out of the experiment"
+  end
+
+  context "when the actor is not from France" do
+    before { ensure_the_actor_is_not_from_france }
+    # This allows testing of target group constraint functionality
+    forced_determination(:experiment_name, 'variant_b', only_for: { country: 'fr' })
+
+    it "should respond in a way that is defined by being out of the experiment"
+  end
+
+  context "when the actor has a specified id" do
+    before { ensure_the_actor_has_id_123 }
+    # This allows testing of override functionality
+    forced_determination(:experiment_name, 'variant_b', only_for: { id: '123' })
+
+    it "should respond in a way that is defined by variant_b"
+  end
+end
+```
+
+## Fake Florence for local execution
+
+[Fake Florence](https://github.com/deliveroo/fake_florence) is a command line utility which operates a determinator compatible server and provides tooling for easy editing of feature flags and experiments.
+
+```bash
+$ gem install fake_florence
+Fake Florence has been installed. Run `flo help` for more information.
+1 gem installed
+
+$ flo start
+Flo is now running at https://flo.dev
+Use other commands to create or edit Feature flags and Experiments.
+See `flo help` for more information
+
+$ flo create my_experiment
+      create  ~/.flo/features/my_experiment.yaml
+my_experiment created and opened for editing
+```
+
+Then in your service, configured with `discovery_url: 'https://flo.dev'`, experiments and feature flags will retrieved and posted from Fake Florence:
+
+```ruby
+determinator.which_variant(:my_experiment, id: 123)
+"anchovy"
+```
+
+More information can be found on the [Fake Florence](https://github.com/deliveroo/fake_florence) project page.

data/examples/determinator-rails/app/controllers/application_controller.rb

@@ -14,7 +14,12 @@ class ApplicationController < ActionController::API
     # which allows simple use throughout the app
     @_determinator ||= Determinator.instance.for_actor(
       id: current_user && current_user.id || nil,
-      guid: guid
+      guid: guid,
+      default_properties: {
+        # Clearly this would return real information about whether the
+        # user is an employee.
+        employee: false
+      }
     )
   end
 end

data/lib/determinator.rb

@@ -7,9 +7,11 @@ require 'determinator/retrieve/null_retriever'
 
 module Determinator
   # @param :retrieval [Determinator::Retrieve::Routemaster] A retrieval instance for Features
-  # @param :errors [Proc, nil] a proc, accepting an error, which will be called with any errors which occur while determinating
-  def self.configure(retrieval:, errors: nil)
+  # @param :errors [#call, nil] a proc, accepting an error, which will be called with any errors which occur while determinating
+  # @param :missing_feature [#call, nil] a proc, accepting a feature name, which will be called any time a feature is requested but isn't available
+  def self.configure(retrieval:, errors: nil, missing_feature: nil)
     @error_logger = errors if errors.respond_to?(:call)
+    @missing_feature_logger = missing_feature if missing_feature.respond_to?(:call)
     @instance = Control.new(retrieval: retrieval)
   end
 
@@ -23,4 +25,10 @@ module Determinator
 
     @error_logger.call(error)
   end
+
+  def self.missing_feature(name)
+    return unless @missing_feature_logger
+
+    @missing_feature_logger.call(name)
+  end
 end

data/lib/determinator/control.rb

@@ -57,7 +57,10 @@ module Determinator
 
     def determinate(name, id:, guid:, properties:)
       feature = retrieval.retrieve(name)
-      return false unless feature
+      if feature.nil?
+        Determinator.missing_feature(name)
+        return false
+      end
 
       # Calling method can place constraints on the feature, eg. experiment only
       return false if block_given? && !yield(feature)
@@ -86,9 +89,14 @@ module Determinator
     end
 
     def choose_target_group(feature, properties)
+      # Keys must be strings
+      normalised_properties = properties.each_with_object({}) do |(k, v), h|
+        h[k.to_s] = v
+      end
+
       feature.target_groups.select { |tg|
         tg.constraints.reduce(true) do |fit, (scope, *required)|
-          present = [*properties[scope]]
+          present = [*normalised_properties[scope.to_s]]
           fit && (required.flatten & present.flatten).any?
         end
       # Must choose target group deterministically, if more than one match

data/lib/determinator/retrieve/routemaster.rb

@@ -31,6 +31,9 @@ module Determinator
         cached_feature_lookup(feature_id) do
           @actor_service.feature.show(feature_id)
         end
+      rescue ::Routemaster::Errors::ResourceNotFound
+        # Don't be noisy
+        nil
       rescue => e
         Determinator.notice_error(e)
         nil
@@ -47,10 +50,6 @@ module Determinator
         @routemaster_app ||= ::Routemaster::Drain::Caching.new
       end
 
-      def self.index_cache_key(feature_name)
-        "determinator_index:#{feature_name}"
-      end
-
       def self.lookup_cache_key(feature_name)
         "determinator_cache:#{feature_name}"
       end

data/lib/determinator/version.rb

@@ -1,3 +1,3 @@
 module Determinator
-  VERSION = '0.10.0'
+  VERSION = '0.11.0'
 end

data/lib/rspec/determinator.rb

@@ -7,7 +7,7 @@ module RSpec
 
       by.let(:fake_determinator) { FakeControl.new }
       by.before do
-        allow(::Determinator::Control).to receive(:new).and_return(fake_determinator)
+        allow(::Determinator).to receive(:instance).and_return(fake_determinator)
       end
     end
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: determinator
 version: !ruby/object:Gem::Version
-  version: 0.10.0
+  version: 0.11.0
 platform: ruby
 authors:
 - JP Hastings-Spital
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-09-15 00:00:00.000000000 Z
+date: 2017-10-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: routemaster-drain
@@ -143,7 +143,9 @@ files:
 - bin/setup
 - circle.yml
 - determinator.gemspec
+- docs/background.md
 - docs/img/determinator.jpg
+- docs/local_development.md
 - examples/determinator-rails/.env
 - examples/determinator-rails/.gitignore
 - examples/determinator-rails/Gemfile