gitlab-experiment 0.2.4 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a6284349854da93da979695a6493df95c0247616e10d5adc047d56d5dbdbf324
-  data.tar.gz: 1e4456945313fce9fbf03250299e3a0eb1b406035e4cce0d0559eee1fb3cfe53
+  metadata.gz: 6134f703a49eb7411ff59e1fa7778421890498a537419cd765a99c5e07ff1523
+  data.tar.gz: ea3736188b46c9527818109dc87839c33937e0c9086bae4464992046f425874a
 SHA512:
-  metadata.gz: '08d5c04e817c2679d115a845df588a4b81358e1b7ac7ddc8282a4cdaa2a878f6a4111b318fa3155671fdd05c0cb950ab076210518142e5f8a04db89e7b164104'
-  data.tar.gz: 118de1ae6b9995edee4058a1b756030ee6659199bf19a29cf2c1446ddb87a1a8ff29f479cf9ff20f7af47ff7706bbc97256929ed800f803fc381450bcedbf82c
+  metadata.gz: ec048634699257e018d7e67f290dfa2dd5e2712d58b1ef25eeb1c9353c5a23ad2f7a8ec0484493e6bccaabc57801aa22ceb8632a5a8c355652ae4a5541a43286
+  data.tar.gz: 51f63fb0256c49f393887451723483233927a0a983c51ca80f98494634e156dab28d0f87f21319a4df750392bb8329923e94ab23091929b3b26904edb25ce5b3

data/README.md

@@ -38,7 +38,7 @@ In our control (current world) we show a simple toggle interface labeled, "Notif
 
 The behavior will be the same, but the interface will be different and may involve more or fewer steps.
 
-Our hypothesis is that this will make the action more clear and will help the user in making a choice about if that's what they really want to do.
+Our hypothesis is that this will make the action more clear and will help in making a choice about if that's what the user really wants to do.
 
 We'll name our experiment `notification_toggle`. This name is prefixed based on configuration. If you've set `config.name_prefix = 'gitlab'`, the experiment name would be `gitlab_notification_toggle` elsewhere.
 
@@ -51,7 +51,7 @@ Now in our experiment we're going to render one of two views: the control will b
 ```ruby
 class SubscriptionsController < ApplicationController
   def show
-    experiment(:notification_toggle, user_id: user.id) do |e|
+    experiment(:notification_toggle, actor: user) do |e|
       e.use { render_toggle } # control
       e.try { render_button } # candidate
     end
@@ -64,7 +64,7 @@ You can define the experiment using simple control/candidate paths, or provide n
 Handling multi-variant experiments is up to the configuration you provide around resolving variants. But in our example we may want to try with and without the confirmation. We can run any number of variations in our experiments this way.
 
 ```ruby
-experiment(:notification_toggle, user_id: user.id) do |e|
+experiment(:notification_toggle, actor: user) do |e|
   e.use { render_toggle } # control
   e.try(:variant_one) { render_button(confirmation: true) }
   e.try(:variant_two) { render_button(confirmation: false) }
@@ -76,7 +76,7 @@ Understanding how an experiment can change behavior is important in evaluating i
 To this end, we track events that are important by calling the same experiment elsewhere in code. By using the same context, you'll have consistent behavior and the ability to track events to it.
 
 ```ruby
-experiment(:notification_toggle, user_id: user.id).track(:clicked_button)
+experiment(:notification_toggle, actor: user).track(:clicked_button)
 ```
 
 <details>
@@ -85,10 +85,10 @@ experiment(:notification_toggle, user_id: user.id).track(:clicked_button)
 ### Class level interface using `.run`
 
 ```ruby
-exp = Gitlab::Experiment.run(:notification_toggle, user_id: user.id) do |e|
+exp = Gitlab::Experiment.run(:notification_toggle, actor: user) do |e|
   # Context may be passed in the block, but must be finalized before calling
   # run or track.
-  e.context(project_id: project.id) # add the project id to the context
+  e.context(project: project) # add the project to the context
 
   # Define the control and candidate variant.
   e.use { render_toggle } # control
@@ -102,10 +102,10 @@ exp.track(:clicked_button)
 ### Instance level interface
 
 ```ruby
-exp = Gitlab::Experiment.new(:notification_toggle, user_id: user.id)
+exp = Gitlab::Experiment.new(:notification_toggle, actor: user)
 # Additional context may be provided to the instance (exp) but must be
 # finalized before calling run or track.
-exp.context(project_id: project.id) # add the project id to the context
+exp.context(project: project) # add the project id to the context
 
 # Define the control and candidate variant.
 exp.use { render_toggle } # control
@@ -136,10 +136,10 @@ class NotificationExperiment < Gitlab::Experiment
   end
 end
 
-exp = NotificationExperiment.new(user_id: user.id) do |e|
+exp = NotificationExperiment.new(actor: user) do |e|
   # Context may be provided within the block or to the instance (exp) but must
   # be finalized before calling run or track.
-  e.context(project_id: project.id) # add the project id to the context
+  e.context(project: project) # add the project id to the context
 end
 
 # Run the experiment -- returning the result.
@@ -159,7 +159,7 @@ exp.track(:clicked_button)
 You can hardcode the variant if you want. It's important to know what this might do to your data during rollout, so use this with consideration.
 
 ```ruby
-experiment(:notification_toggle, :no_interface, user_id: user.id) do |e|
+experiment(:notification_toggle, :no_interface, actor: user) do |e|
   e.use { render_toggle } # control
   e.try { render_button } # candidate
   e.try(:no_interface) { no_interface! } # variant
@@ -169,7 +169,7 @@ end
 Or you can set the variant within the block. This allows using unique segmentation logic or variant resolution if you need it.
 
 ```ruby
-experiment(:notification_toggle, user_id: user.id) do |e|
+experiment(:notification_toggle, actor: user) do |e|
   # Variant selection must be done before calling run or track.
   e.variant(:no_interface) # set the variant
   # ...
@@ -179,7 +179,7 @@ end
 Or it can be specified in the call to run if you call it from within the block.
 
 ```ruby
-experiment(:notification_toggle, user_id: user.id) do |e|
+experiment(:notification_toggle, actor: user) do |e|
   # ...
   # Variant selection can be specified when calling run.
   e.run(:no_interface)
@@ -209,13 +209,13 @@ Some experiments may extend outside of those layers, so you may want to include
 Note: In a lot of these contexts you may not have a reference to the request (unless you pass it in, or provide access to it) which may be needed if you want to enable cookie behaviors and track that through to user conversion.
 
 ```ruby
-class UserMailer < ApplicationMailer
+class WelcomeMailer < ApplicationMailer
   include Gitlab::Experiment::Dsl # include the `experiment` method
 
   def welcome
     @user = params[:user]
 
-    ex = experiment(:project_suggestions, user_id: @user.id) do |e|
+    ex = experiment(:project_suggestions, actor: @user) do |e|
       e.use { 'welcome' }
       e.try { 'welcome_with_project_suggestions' }
     end
@@ -234,8 +234,8 @@ Take for instance, that you might be using `version: 1` in your context currentl
 In providing the context migration data, we can resolve an experience and its events all the way back. This can also help in keeping our cache relevant.
 
 ```ruby
-# Migrate just the `:version` portion of the previous context, `{ user_id: 42, version: 1 }`:
-experiment(:my_experiment, user_id: 42, version: 2, migrated_with: { version: 1 })
+# Migrate just the `:version` portion of the previous context, `{ actor: project, version: 1 }`:
+experiment(:my_experiment, actor: project, version: 2, migrated_with: { version: 1 })
 ```
 
 You can add or remove context by providing a `migrated_from` option. This approach expects a full context replacement -- i.e. what it was before you added or removed the new context key.
@@ -243,8 +243,8 @@ You can add or remove context by providing a `migrated_from` option. This approa
 If you wanted to introduce a `version` to your context, provide the full previous context.
 
 ```ruby
-# Migrate the full context from `{ user_id: 42 }` to `{ user_id: 42, version: 1 }`:
-experiment(:my_experiment, user_id: 42, version: 1, migrated_from: { user_id: 42 })
+# Migrate the full context from `{ actor: project }` to `{ actor: project, version: 1 }`:
+experiment(:my_experiment, actor: project, version: 1, migrated_from: { actor: project })
 ```
 
 This can impact an experience if you:
@@ -252,36 +252,36 @@ This can impact an experience if you:
 1. haven't implemented the concept of migrations in your variant resolver
 1. haven't enabled a reasonable caching mechanism
 
-### When there isn't a user (cookies)
+### When there isn't an actor (cookie fallback)
 
-When there isn't an identifying key in the context (this is `user_id` by default), we fall back to cookies to provide a consistent experience.
+When there isn't an identifying key in the context (this is `actor` by default), we fall back to cookies to provide a consistent experience for the client viewing them.
 
-Once we assign a certain variant to a context, we need to always provide the same experience. We achieve this by setting a cookie for the experiment in question.
+Once we assign a certain variant to a context, we need to always provide the same experience. We achieve this by setting a cookie for the experiment in question, but only when needed.
 
-This cookie is a randomized uuid and isn't associated with the user. When we can finally provide an identifying key, the context is auto migrated from the cookie to that identifying key. The cookie is a temporary value, and isn't used for tracking.
+This cookie is a temporary, randomized uuid and isn't associated with a user. When we can finally provide an actor, the context is auto migrated from the cookie to that actor.
 
 To read and write cookies, we provide the `request` from within the controller and views. The cookie migration will happen automatically if the experiment is within those layers.
 
 You'll need to provide the `request` as an option to the experiment if it's outside of the controller and views.
 
 ```ruby
-experiment(:my_experiment, user_id: user&.id, request: request)
+experiment(:my_experiment, actor: user, request: request)
 ```
 
-The cookie isn't set if the identifying key isn't present at all in the context. Using the default identifying key, when there is no `user_id` key provided, the cookie will not be set.
+The cookie isn't set if the `actor` key isn't present at all in the context. Meaning that when no `actor` key is provided, the cookie will not be set.
 
 ```ruby
-# no user_id context key is present, so no cookie is set
-experiment(:my_experiment, project_id: @project.id)
+# actor is not present, so no cookie is set
+experiment(:my_experiment, project: project)
 
-# user_id context key is present, but is set to nil, so the cookie is set & used instead
-experiment(:my_experiment, user_id: nil, project_id: @project.id)
+# actor is present and is nil, so the cookie is set and used
+experiment(:my_experiment, actor: nil, project: project)
 
-# user_id context key is present and set to a value, so no cookie is set
-experiment(:my_experiment, user_id: @user.id, project_id: @project.id)
+# actor is present and set to a value, so no cookie is set
+experiment(:my_experiment, actor: user, project: project)
 ```
 
-For edge cases, you can pass the cookie through by assigning it yourself -- e.g. `user_id: request.cookie_jar.signed['my_experiment_id']`. The cookie name is the full experiment name (including any configured prefix) with `_id` appended -- e.g. `gitlab_notification_toggle_id` for the `:notification_toggle` experiment key with a configured prefix of `gitlab`.
+For edge cases, you can pass the cookie through by assigning it yourself -- e.g. `actor: request.cookie_jar.signed['my_experiment_actor']`. The cookie name is the full experiment name (including any configured prefix) with `_actor` appended -- e.g. `gitlab_notification_toggle_actor` for the `:notification_toggle` experiment key with a configured prefix of `gitlab`.
 
 ## Configuration
 

data/lib/gitlab/experiment.rb

@@ -4,6 +4,7 @@ require 'scientist'
 
 require 'gitlab/experiment/caching'
 require 'gitlab/experiment/configuration'
+require 'gitlab/experiment/cookies'
 require 'gitlab/experiment/context'
 require 'gitlab/experiment/dsl'
 require 'gitlab/experiment/variant'
@@ -28,16 +29,13 @@ module Gitlab
       end
     end
 
-    delegate :signature, :cache_strategy, to: :context
-
     def initialize(name, variant_name = nil, **context)
       @name = name
       @variant_name = variant_name
-      @context = Context.new(self)
-
-      context(context)
+      @excluded = []
+      @context = Context.new(self, context)
 
-      ignore { true }
+      exclude { !@context.trackable? }
       compare { false }
 
       yield self if block_given?
@@ -57,10 +55,17 @@ module Gitlab
       result.respond_to?(:name) ? result : Variant.new(name: result.to_s)
     end
 
+    def exclude(&block)
+      @excluded << block
+    end
+
     def run(variant_name = nil)
-      @variant_name = variant_name unless variant_name.nil?
+      @result ||= begin
+        @variant_name = variant_name unless variant_name.nil?
+        @variant_name ||= :control if excluded?
 
-      @result ||= super(cache { variant.name })
+        super(cache { variant.name })
+      end
     end
 
     def publish(result)
@@ -68,6 +73,8 @@ module Gitlab
     end
 
     def track(action, **event_args)
+      return if excluded?
+
       instance_exec(action, event_args, &Configuration.tracking_behavior)
     end
 
@@ -76,19 +83,19 @@ module Gitlab
     end
 
     def variant_names
-      @variant_names ||= behaviors.keys.tap { |keys| keys.delete('control') }.map(&:to_sym)
+      @variant_names ||= behaviors.keys.map(&:to_sym) - [:control]
     end
 
-    def enabled?
-      true
+    def signature
+      { variant: variant.name, experiment: name }.merge(context.signature)
     end
 
-    def identifying_key
-      :user_id
+    def enabled?
+      true
     end
 
-    def cache_key_for(key, migration: false)
-      "#{name}:#{key}"
+    def excluded?
+      @excluded.any? { |exclude| exclude.call(self) }
     end
 
     protected

data/lib/gitlab/experiment/caching.rb

@@ -10,6 +10,15 @@ module Gitlab
         migrated_cache(cache, migrations || [], key) or cache.fetch(key, &block)
       end
 
+      private
+
+      def cache_strategy
+        [
+          "#{name}:#{signature[:key]}",
+          signature[:migration_keys]&.map { |key| "#{name}:#{key}" }
+        ]
+      end
+
       def migrated_cache(cache, migrations, new_key)
         migrations.find do |old_key|
           next unless (value = cache.read(old_key))

data/lib/gitlab/experiment/context.rb

@@ -3,92 +3,65 @@
 module Gitlab
   class Experiment
     class Context
+      include Cookies
+
       DNT_REGEXP = /^(true|t|yes|y|1|on)$/i.freeze
 
-      def initialize(experiment)
+      def initialize(experiment, **initial_value)
         @experiment = experiment
         @value = {}
-        @migrations_from = []
-        @migrations_with = []
+        @migrations = { merged: [], unmerged: [] }
+
+        value(initial_value)
+      end
+
+      def reinitialize(request)
+        @signature = nil # clear memoization
+        @request = request if request.respond_to?(:headers) && request.respond_to?(:cookie_jar)
       end
 
       def value(value = nil)
         return @value if value.nil?
 
         value = value.dup # dup so we don't mutate
-        @signature = @cache_strategy = nil # clear memoization
+        reinitialize(value.delete(:request))
 
-        @migrations_from << value.delete(:migrated_from) if value[:migrated_from]
-        @migrations_with << value.delete(:migrated_with) if value[:migrated_with]
-        @value.merge!(auto_migrate_cookie(value, value.delete(:request)))
+        @value.merge!(process_migrations(value))
       end
 
-      def freeze
-        cache_strategy # ensure we memoize before freezing
-        super
+      def trackable?
+        !(@request && @request.headers['DNT'].to_s.match?(DNT_REGEXP))
       end
 
-      def cache_strategy
-        @cache_strategy ||= [
-          @experiment.cache_key_for(signature[:key]),
-          signature[:migration_keys]&.map do |key|
-            @experiment.cache_key_for(key, migration: true)
-          end
-        ]
+      def freeze
+        signature # finalize before freezing
+        super
       end
 
       def signature
-        @signature ||= {
-          key: key_for(@value),
-          migration_keys: migration_keys,
-          variant: @experiment.variant.name
-        }.compact
+        @signature ||= { key: key_for(@value), migration_keys: migration_keys }.compact
       end
 
       private
 
-      def auto_migrate_cookie(hash, request)
-        return hash unless request&.respond_to?(:headers) && request&.respond_to?(:cookie_jar)
-        return hash if request.headers['DNT'].to_s.match?(DNT_REGEXP)
+      def process_migrations(value)
+        add_migration(value.delete(:migrated_from))
+        add_migration(value.delete(:migrated_with), merge: true)
 
-        resolver = cookie_resolver(request.cookie_jar, hash)
-        resolve_cookie(*resolver) or generate_cookie(*resolver)
+        migrate_cookie(value, "#{@experiment.name}_id")
       end
 
-      def cookie_resolver(jar, hash)
-        [jar, hash, @experiment.identifying_key, jar.signed[cookie_name]].compact
-      end
-
-      def cookie_name
-        @cookie_name ||= [@experiment.name, 'id'].join('_')
-      end
-
-      def resolve_cookie(jar, hash, key, cookie = nil)
-        return if cookie.blank? && hash[key].blank?
-        return hash if cookie.blank?
-        return hash.merge(key => cookie) if hash[key].blank?
-
-        @migrations_with << { key => cookie }
-        jar.delete(cookie_name, domain: :all)
-
-        hash
-      end
-
-      def generate_cookie(jar, hash, key, cookie = SecureRandom.uuid)
-        return hash unless hash.key?(key)
-
-        jar.permanent.signed[cookie_name] = {
-          value: cookie, secure: true, domain: :all, httponly: true
-        }
+      def add_migration(value, merge: false)
+        return unless value.is_a?(Hash)
 
-        hash.merge(key => cookie)
+        @migrations[merge ? :merged : :unmerged] << value
       end
 
       def migration_keys
-        return nil if @migrations_from.empty? && @migrations_with.empty?
+        return nil if @migrations[:unmerged].empty? && @migrations[:merged].empty?
 
-        @migrations_from.map { |m| key_for(m) } +
-          @migrations_with.map { |m| key_for(@value.merge(m)) }
+        @migrations[:unmerged].map { |m| key_for(m) } +
+          @migrations[:merged].map { |m| key_for(@value.merge(m)) }
       end
 
       def key_for(context)

data/lib/gitlab/experiment/cookies.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+require 'securerandom'
+
+module Gitlab
+  class Experiment
+    module Cookies
+      private
+
+      def migrate_cookie(hash, cookie_name)
+        return hash if cookie_jar.nil?
+
+        resolver = [hash, :actor, cookie_name, cookie_jar.signed[cookie_name]]
+        resolve_cookie(*resolver) or generate_cookie(*resolver)
+      end
+
+      def cookie_jar
+        @request&.cookie_jar
+      end
+
+      def resolve_cookie(hash, key, cookie_name, cookie)
+        return if cookie.to_s.empty? && hash[key].nil?
+        return hash if cookie.to_s.empty?
+        return hash.merge(key => cookie) if hash[key].nil?
+
+        add_migration(key => cookie)
+        cookie_jar.delete(cookie_name, domain: :all)
+
+        hash
+      end
+
+      def generate_cookie(hash, key, cookie_name, cookie)
+        return hash unless hash.key?(key)
+
+        cookie ||= SecureRandom.uuid
+        cookie_jar.permanent.signed[cookie_name] = {
+          value: cookie, secure: true, domain: :all, httponly: true
+        }
+
+        hash.merge(key => cookie)
+      end
+    end
+  end
+end

data/lib/gitlab/experiment/version.rb

@@ -2,6 +2,6 @@
 
 module Gitlab
   class Experiment
-    VERSION = '0.2.4'
+    VERSION = '0.3.0'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: gitlab-experiment
 version: !ruby/object:Gem::Version
-  version: 0.2.4
+  version: 0.3.0
 platform: ruby
 authors:
 - GitLab
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-10-06 00:00:00.000000000 Z
+date: 2020-10-21 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: scientist
@@ -46,6 +46,7 @@ files:
 - lib/gitlab/experiment/caching.rb
 - lib/gitlab/experiment/configuration.rb
 - lib/gitlab/experiment/context.rb
+- lib/gitlab/experiment/cookies.rb
 - lib/gitlab/experiment/dsl.rb
 - lib/gitlab/experiment/engine.rb
 - lib/gitlab/experiment/variant.rb