gitlab-experiment 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fae9519317810c50decdff810249899d5c1855881544a20a31a2219cf9bda715
-  data.tar.gz: 19987d9c24486e99424aa6cfcade34d3123dbe1fc4a6afb8dc6c4c6ad4f5a99b
+  metadata.gz: 8b64d15c51bec74dc0b9be36e3b1541c3509c0f62a2259595c71366ff66f35e3
+  data.tar.gz: 5c6862e1a9d44335ebe1d6b6e3974da4e0e368297d006eb91398ebf9d3004681
 SHA512:
-  metadata.gz: afe30984c7b7f96bb4ad1122c5a4f5ff48a45f8c7cfca5b1d2d04368ba3eef25c84776e3768588cde6cd717ee918b25e8158746ea9ea3e9bb20fe9ef39d20377
-  data.tar.gz: 2492d43cbc76cfa20c1b785033fdc635ca75874d1052270c950a86acff17d04011c406bf1879bc5f21b06fb604c5ed259c149aecbea33efe2adae59ab12f6164
+  metadata.gz: 0cea1580efdbe7daa88f76fb93de94d4ea04059bbc1bd25a15edf4a501e139ec9102c6abf9c2002e797ecfc9bb6ffc3115dcf4e817b94b5f6499ed09ca14de75
+  data.tar.gz: 20490eec566060797379ecea83e07fe74a00f2725817bc396b6bea7cfc07947cd120562d433fb33d6db5b4094c58fd50b2ff1cb9455867c6300443fc4c2c6509

data/README.md

@@ -1,36 +1,48 @@
-# Experiment Guide
+GitLab Experiment
+=================
 
-Experiments can be conducted by any GitLab team, but are most often conducted by the teams from the [Growth Sub-department](https://about.gitlab.com/handbook/engineering/development/growth/).
+Here at GitLab, experiments are run as A/B/n tests and are evaluated by the data the experiment generates. The team reviews the data, determines the variant that performed most effectively, and promotes that variant as the new default code path (or reverts back to the control).
 
-Experiments are run as A/B/n tests and are evaluated by the data the experiment generates. The team reviews the data, determines the variant that performed most effectively, and promotes that variant as the new default code path (or reverts back to the control). In all instances, an experiment will be cleaned up after it has generated enough data to determine performance and be considered resolved.
+When we discuss the behavior of this gem, we'll use terms like experiment, control, candidate, and variant. It's worth defining these terms so they're more understood.
 
-## Process and tracking issue
+- `experiment` is any deviation of code paths we want to run sometimes and not others.
+- `control` is the default, or "original" code path.
+- `candidate` is used if there's one experimental code path.
+- `variant(s)` is used when more than one experimental code path exists.
 
-Each experiment should have a related [Experiment tracking issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20tracking) created, which is intended to track the experiment from deployment and rollout through to resolution and cleanup.
+Candidate and variant are referencing the same concept, but can simplify how we speak about the available code paths of a given experiment.
 
-At the time an experiment is rolled out, the due date of the tracking issue should be specified. The timeline depends on the experiment and can be up to several weeks in the future.
+## Installation
 
-With experiment resolution, the outcome of the experiment should be posted to the tracking issue with the reasoning for the decision. Any and all non-relevant experiment code should be removed and review concerns should be addressed during cleanup. After cleanup, the tracking issue can be closed.
+Add the gem to your Gemfile and then bundle install.
 
-## Implementing an experiment
+```ruby
+gem 'gitlab-experiment'
+```
 
-For the sake of our example, let's say we want to run an experiment for how to cancel a subscription. In our control (current world) we show a toggle that reads "Auto-renew", and in our experiment candidate we want to show a "Cancel subscription" button with a confirmation. Ultimately the behavior is the same, but the interface will be considerably different.
+If you're using Rails, you can install an initializer that provides basic configuration by running the install generator.  
 
-### Defining the feature flag
+```shell
+$ rails generate gitlab-experiment:install
+```
 
-Let's name our experiment `subscription_cancellation`. It's important to understand how this name is prefixed as `growth_experiment_subscription_cancellation` in our Unleash feature detection and in our Snowplow tracking calls. We prefix our experiments so we can consistently identify them as experiments and clean them up over time.
+## Implementing an experiment
 
-This means that you'll need to go to the [Feature Flags interface](https://gitlab.com/gitlab-org/gitlab/-/feature_flags) interface (for the project you're working on) and add a `growth_experiment_subscription_cancellation` feature flag and define which environment(s) it should be rolled out on for initial deployment, for instance, `gitlab-staging` or `customers-staging`.
+For the sake of our example, let's say we want to run an experiment for the interface we render for subscription cancellation.
 
-You can include yourself and others in a list and assign that list using the `User List` rollout strategy, or you can add your user id to the experiment manually. You can use this to include or exclude yourself from an experiment for verification before rolling it out to others.
+In our control (current world) we show a simple toggle interface that reads "Auto-renew", and in our experiment candidate we want to show a "Cancel subscription" button with a confirmation dialog. Ultimately the behavior will be the same, but the interface will be considerably different and may involve more or less steps.
+ 
+We hypothesize that making it more clear what the action is will help people in making a choice about taking that action or not. 
 
-### Implementation
+We'll name our experiment `subscription_cancellation`. It's important to understand this name may be prefixed based on your configuration, so if you've set `config.name_prefix = "gitlab"` it would be `gitlab_subscription_cancellation`.   
 
-When you implement an experiment in code you'll need to provide the name that you've given it in the feature flags interface, and a context -- which usually will include something like a user / user id, but may also include several other aspects.
+When you implement and run an experiment you'll need to provide the name you've given it, and a context hash. The context hash is used to determine which variant to provide, and is expected to be consistent between calls to your experiment, so a context hash key can be generated and cached -- which is used to consistently render the same experience given the same context.
 
+In our experiment we're going to render one of two views. Control will be our current one, and candidate will be the new view with a cancel button and javascript confirm call.
+ 
 ```ruby
 class SubscriptionsController < ApplicationController
-  include Gitlab::GrowthExperiment::Interface
+  include Gitlab::Experiment::Dsl
 
   def show
     experiment(:subscription_cancellation, user_id: user.id) do |e|
@@ -41,7 +53,7 @@ class SubscriptionsController < ApplicationController
 end
 ```
 
-You can also provide different variants for the experience if you've defined variants in the Feature Flag interface (not yet available).
+You can also provide different variant names if you want to. In our case we might want to include the confirmation dialog step or not, to see which one behaves better within our wider experiment.  
 
 ```ruby
 experiment(:subscription_cancellation, user_id: user.id) do |e|
@@ -51,7 +63,7 @@ experiment(:subscription_cancellation, user_id: user.id) do |e|
 end
 ```
 
-Later, and elsewhere in code you can use the same `experiment` call to track events on the experiment. The important detail here is to use the same context between your calls to `experiment`. If the context is the same, we're able to consistently track the event in a way that associates it to which variant is being presented -- which may be based on the user we're presenting it to.
+Later, and elsewhere in code you can use the same `experiment` call to track events on the experiment. You can use this consistent event concept to build out funnels from the data that's being tracked. The important detail here is to use the same context between your calls to `experiment`. If the context is the same, we're able to consistently track the event in a way that associates it to which variant is being presented.
 
 ```ruby
 exp = experiment(:subscription_cancellation, user_id: user.id)
@@ -59,38 +71,41 @@ exp.track('clicked_button')
 ```
 
 <details>
-  <summary>You can use the more low level class or instance interfaces...</summary>
+  <summary>You can also use the more low level class or instance interfaces...</summary>
 
 ### Class level interface using `.run`
 
 ```ruby
-exp = Gitlab::GrowthExperiment.run(:subscription_cancellation, user_id: user.id) do |e|
-  # context can be passed to `experiment`, `.run`, `new`, or after the fact like here.
-  # context must be added before `#run` or `#track` calls.
-  e.context(project_id: project.id)
-
+exp = Gitlab::Experiment.run(:subscription_cancellation, user_id: user.id) do |e|
+  # Context may be passed in the block, but must be finalized before calling 
+  # run or track.
+  e.context(project_id: project.id) # add the project id to the context
+  e.variant(:candidate) # always run the candidate
+  # Define the control and candidate variant.
   e.use { toggle_button_interface } # control
   e.try { cancel_button_interface } # candidate
 end
 
-# track an event on the experiment we've defined.
+# Track an event on the experiment we've defined.
 exp.track(:clicked_button)
 ```
 
-While `Gitlab::GrowthExperiment.run` is what we document, you can also use `Gitlab::GrowthExperiment.experiment`.
-
 ### Instance level interface
 
 ```ruby
-exp = Gitlab::GrowthExperiment.new(:subscription_cancellation, user_id: user.id)
-# context can be passed to `.new`, or after the fact like here.
-# context must be added before `#run` or `#track` calls.
-exp.context(project_id: project.id)
+exp = Gitlab::Experiment.new(:subscription_cancellation, user_id: user.id)
+# Context may be passed in the block, but must be finalized before calling 
+# run or track.
+exp.context(project_id: project.id) # add the project id to the context
+
+# Define the control and candidate variant.
 exp.use { toggle_button_interface } # control
 exp.try { cancel_button_interface } # candidate
+
+# Run the experiment -- returning the result.
 exp.run
 
-# track an event on the experiment we've defined.
+# Track an event on the experiment we've defined.
 exp.track(:clicked_button)
 ```
 
@@ -102,29 +117,33 @@ exp.track(:clicked_button)
 ### Custom class
 
 ```ruby
-class CancellationExperiment < Gitlab::GrowthExperiment
+class CancellationExperiment < Gitlab::Experiment
   def initialize(variant_name = nil, **context, &block)
     super(:subscription_cancellation, variant_name, **context, &block)
+
+    # Define the control and candidate variant.
+    use { toggle_button_interface } # control
+    try { cancel_button_interface } # candidate
   end
 end
 
-exp = CancellationExperiment.run(user_id: user.id) do |e|
-  # context can be passed to `.run`, or after the fact like here.
-  # context must be added before `#run` or `#track` calls.
-  e.context(project_id: project.id)
-
-  e.use { toggle_button_interface } # control
-  e.try { cancel_button_interface } # candidate
+exp = CancellationExperiment.new(user_id: user.id) do |e|
+  # Context may be passed in the block, but must be finalized before calling 
+  # run or track.
+  e.context(project_id: project.id) # add the project id to the context
 end
 
-# track an event on the experiment we've defined.
+# Run the experiment -- returning the result.
+exp.run
+
+# Track an event on the experiment we've defined.
 exp.track(:clicked_button)
 ```
 
 </details>
 
 <details>
-  <summary>You can hard specify the variant to use...</summary>
+  <summary>You can also hard specify the variant to use...</summary>
 
 ### Specifying which variant to use
 
@@ -138,45 +157,62 @@ experiment(:subscription_cancellation, :no_interface, user_id: user.id) do |e|
 end
 ```
 
-Or you can set the variant within the block.
+Or you can set the variant within the block -- potentially using some unique or different segmentation strategy that you've written specifically for the experiment at hand.
 
 ```ruby
 experiment(:subscription_cancellation, user_id: user.id) do |e|
-  e.variant(:variant) # set the variant
+  e.variant(:no_interface) # set the variant
   # ...
 end
 ```
 
 </details>
 
-The `experiment` method, and the underlying `Gitlab::GrowthExperiment` is an implementation on top of [Scientist](https://github.com/github/scientist). Generally speaking you can use the DSL that Scientist defines, but for experiments we use `experiment` instead of `science`, and specify the variant on initialization (or via `#variant` and not in the call to `#run`. The interface is otherwise the same, even though not every aspect of Scientist makes sense for experiments.
+The `experiment` method, and the underlying `Gitlab::Experiment` instance is an implementation on top of [Scientist](https://github.com/github/scientist). Generally speaking you can use the DSL that Scientist defines, but for experiments we use `experiment` instead of `science`, and specify the variant on initialization (or via `#variant` and not in the call to `#run`. The interface is otherwise the same, even though not every aspect of Scientist makes sense for experiments.
 
 ### Context migrations
 
-There are times when we may need to add new values or change something that we're providing in context while an experiment is running. We make this possible by passing the `migrated_from` context key.
+There are times when we may need to change something that we're providing in the context while an experiment is running, or even add or remove contexts. We make this possible by passing the migration data to the experiment.
 
-Take for instance, that you might be using `version: 1` in your context. If you want to migrate this to `version: 2`, you just need to provide the context that you provided prior, in a `migrated_from` context key. In doing this, a given experiment experience can be resolved back through any number of migrations.
+Take for instance, that you might be using `version: 1` in your context currently. If you want to migrate this to `version: 2`, you just need to provide the context that you want to change using a `migrated_with` option. In doing this, a given experience (variant rendered/events tracked) can be resolved back through any number of migrations, and can be cached/resolved by using the context key value that was already already generated.
 
 ```ruby
-experiment(:my_experiment, version: 2, migrated_from: { version: 1 })
+experiment(:my_experiment, user_id: 42, version: 2, migrated_with: { version: 1 })
 ```
 
-It's important to understand that this can bucket a user in a new experience (depending on the rollout strategy being used and what is changing in the context), so you should investigate how this might impact your experiment before using it.
+If you're adding or removing a new a value from the context, you'll need to use `migrated_from`, which expects a full context replacement -- e.g. what it was before you added or removed the new context key. For instance, if you wanted to introduce the `version: 1` concept to your context, you would need to use something like the following. 
+ 
+```ruby
+experiment(:my_experiment, user_id: 42, version: 1, migrated_from: { user_id: 42 })
+```
 
-### When there isn't a user
+It's important to understand that this can bucket a user in a new experience (depending on the logic in determining in the variant), so you should investigate how this might impact your experiment before using it and that your experiment is implemented in a way that supports migrations.
 
-When there isn't a user, we typically have to fall back to another concept to provide a consistent experiment experience. What this means, is that once we bucket someone in a certain bucket, we always bucket them in that bucket.
+### When there isn't a user (cookies)
 
-We do this by using cookies.... [document more]
+When there isn't a user we typically have to fall back to another concept to provide a consistent experience. What this means, is that once we assign someone a certain variant, we want to always give them the same experience, and we do this by setting a cookie for the experiment we're currently checking on. This cookie value is a random uuid, which we auto migrate to a user_id when our experiment context is finally provided that information. This means that you really only need to provide the request as an option if you're wanting to have an experiment that flows from not-signed-in (or not registered) users to eventually signed-in users.
 
-## Tracking, anonymity and GDPR
+This is considered a temporary cookie value, and isn't used for tracking purposes other than to give a given "user" (in this case it's actually the browser), a consistent experience after we've assigned one.
 
-We intentionally don't, and shouldn't, track things like user ids. What we can and do track is what we consider an "experiment experience" key. This key is generated by the context we pass to the experiment implementation. If we consistently pass the same context to an experiment, we're able to consistently track events generated in that experience. A context can contain things like user, or project -- so, if you only included a user in the context that user would get the same experience across all projects they view, but if you include the currently viewed project in the context the user would potentially have a different experience on each of their projects. Each can be desirable given the objectives of the experiment.
+To read and write cookies, we allow for passing the `request` as an option. This allows us to read, write, and even clean up a cookie when appropriate. We've provided this by default in the ActionController interface though, so this is primarily useful if you're trying to have an experiment span controller and model layers. 
 
-## Code quality expectations
+```ruby
+experiment(:subscription_cancellation, user_id: user.id, request: request) do |e|
+  e.use { render_toggle_button } # control
+  e.try { render_cancel_button } # candidate
+end
+``` 
+
+If needed, for edge cases (like in a background job), you can manually pass in the cookie value. Passing it in as `user_id: request.cookie_jar.signed['subscription_cancellation_id']`. The cookie name is the experiment name (prefixed if configured to be) with `_id` appended.
+
+## Configuration
 
-Since experimental code is inherently short lived, an intentionally stated goal is to iterate quickly to generate and evaluate performance data.
+The gem is meant to be configured before being used. The default configuration will always render the control behavior, so it's important to implement your own logic for this or you will always get the control of an experiment.
 
-This goal prioritizes iteration and resolution over code quality, which means that experiment code may not always meet our code standards guidelines, but should also not negatively impact the availability of GitLab nor contribute to bad data. Even though experiments will be deployed to a minority of users, we still expect a flawless experience for those users, therefore, good test coverage is still required.
+The most important aspect of the gem, determining which variant to render and when, is up to you, and you may want to consider using [Unleash](https://github.com/Unleash/unleash-client-ruby) (which has the concept of multi-variants built in), or [Flipper](https://github.com/jnunemaker/flipper) in helping with this.
+
+Examples for configuration are available in the provided install generator, or in the source code configuration.rb file itself.
+
+## Tracking, anonymity and GDPR
 
-Reviewers and maintainers are encouraged to note when code doesn't meet our code standards guidelines. Please mention your concerns and include or link to them on the experiment tracking issue. The experiment author(s) are responsible for addressing these concerns when the experiment is resolved.
+We intentionally don't, and shouldn't, track things like user ids on our experiments. What we can and do track is what we consider an "experiment experience" key. This key is generated from the context we pass to the experiment and has the concept of migrating through different versions of context. If we consistently pass the same context to an experiment, we're able to consistently track events generated in that experience. A context can contain things like user, or project -- so, if you only included a user in the context that user would get the same experience across all projects they view, but if you include the currently viewed project in the context the user would potentially have a different experience on each of their projects. Each can be desirable given the objectives of the experiment.

data/lib/generators/gitlab_experiment/install/install_generator.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+require 'rails/generators'
+
+module GitlabExperiment
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      source_root File.expand_path(__dir__)
+
+      desc 'Installs the Gitlab Experiment initializer into your application.'
+
+      def copy_initializers
+        copy_file 'templates/initializer.rb', 'config/initializers/gitlab_experiment.rb'
+      end
+
+      def display_post_install
+        readme 'POST_INSTALL' if behavior == :invoke
+      end
+    end
+  end
+end

data/lib/generators/gitlab_experiment/install/templates/initializer.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+Gitlab::Experiment.configure do |config|
+  # Prefix all experiment names with a given value. Use `nil` for none.
+  config.name_prefix = nil
+
+  # The logger is used to log various details of the experiments.
+  config.logger = Logger.new(STDOUT)
+
+  # Logic this project uses to resolve a variant for a given experiment.
+  #
+  # This can return an instance of any object that responds to `name`, or can
+  # return a variant name as a string, in which case the build in variant
+  # class will be used.
+  #
+  # This block will be executed within the scope of the experiment instance,
+  # so can easily access experiment methods, like getting the name or context.
+  config.variant_resolver = lambda do |requested_variant|
+    # An example of running the control, unless a variant was requested:
+    requested_variant || 'control'
+
+    # Always run candidate, unless a variant was requested, with fallback:
+    # variant_name || variant_names.first || 'control'
+
+    # Using unleash to determine the variant:
+    # TODO: this isn't entirely accurate.
+    # fallback = Unleash::Variant.new(name: requested_variant || 'control', enabled: true)
+    # UNLEASH.get_variant(name, context.value, fallback)
+
+    # Using Flipper to determine the variant:
+    # TODO: provide example.
+    # Variant.new(name: resolved)
+  end
+
+  # Tracking behavior can be implemented to link an event to an experiment.
+  #
+  # Similar to the variant resolver, this is called within the scope of the
+  # experiment instance and so can access any methods on the experiment.
+  config.tracking_behavior = lambda do |action, event_args|
+    # An example of using a generic logger to track events:
+    (event_args[:context] ||= []) << context.signature.merge(group: variant.name)
+    config.logger.info "Gitlab::Experiment[#{name}] #{action}: #{event_args}"
+
+    # Using something like snowplow to track events:
+    # (event_args[:context] ||= []) << SnowplowTracker::SelfDescribingJson.new(
+    #   'iglu:com.gitlab/gitlab_experiment/jsonschema/1-0-0',
+    #   experiment: context.signature.merge(group: variant.name)
+    # )
+    #
+    # Tracking.event(name, action, **event_args)
+  end
+
+  # Called at the end of every experiment run, with the result.
+  #
+  # You may want to track that you've assigned a variant to a given context,
+  # or push the experiment into the client or publish results elsewhere, like
+  # into redis.
+  config.publishing_behavior = lambda do |_result|
+    track(:assignment) # this will call the config.tracking_behavior
+
+    # Log the results, so we can inspect them if we wanted to later.
+    # LogRage.log(result: _result)
+
+    # Push the experiment knowledge into the client using Gon.
+    # Gon.push(experiment: { name => signature })
+  end
+
+  # Algorithm that consistently generates a hash key for a given hash map.
+  #
+  # Given a specific context hash map, we need to generate a consistent hash
+  # key. The logic in here will be used for generating cache keys, and may also
+  # be used when determining which variant may be presented.
+  config.context_hash_strategy = lambda do |context|
+    values = context.values.map { |v| (v.respond_to?(:to_global_id) ? v.to_global_id : v).to_s }
+    Digest::MD5.hexdigest((context.keys + values).join('|'))
+  end
+end

data/lib/gitlab/experiment.rb

@@ -7,6 +7,7 @@ require 'gitlab/experiment/context'
 require 'gitlab/experiment/dsl'
 require 'gitlab/experiment/variant'
 require 'gitlab/experiment/version'
+require 'gitlab/experiment/engine' if defined?(Rails::Engine)
 
 module Gitlab
   class Experiment
@@ -25,6 +26,8 @@ module Gitlab
       end
     end
 
+    delegate :signature, to: :context
+
     def initialize(name, variant_name = nil, **context)
       @name = name
       @variant_name = variant_name
@@ -47,15 +50,16 @@ module Gitlab
 
     def variant(value = nil)
       @variant_name = value unless value.nil?
-      instance_exec(@variant_name, &Configuration.variant_resolver)
+      result = instance_exec(@variant_name, &Configuration.variant_resolver)
+      result.respond_to?(:name) ? result : Variant.new(name: result.to_s)
     end
 
     def run
       @result ||= super(variant.name)
     end
 
-    def publish(_result)
-      track(:assignment)
+    def publish(result)
+      instance_exec(result, &Configuration.publishing_behavior)
     end
 
     def track(action, **event_args)

data/lib/gitlab/experiment/configuration.rb

@@ -10,41 +10,25 @@ module Gitlab
       include Singleton
 
       # Prefix all experiment names with a given value. Use `nil` for none.
-      @name_prefix = 'gitlab_experiment'
+      @name_prefix = nil
 
       # The logger is used to log various details of the experiments.
       @logger = Logger.new(STDOUT)
 
       # Logic this project uses to resolve a variant for a given experiment.
       @variant_resolver = lambda do |requested_variant|
-        # An example of running the control, unless a variant was requested:
-        Variant.new(name: requested_variant || 'control')
-
-        # Always run candidate, unless a variant was requested, with fallback:
-        # Variant.new(name: variant_name || variant_names.first || 'control')
-
-        # Using unleash to determine the variant:
-        # fallback = Unleash::Variant.new(name: 'control', enabled: true)
-        # Unleash.get_variant(name, context.value, fallback)
-
-        # Using Flipper to determine the variant:
-        # TODO: provide example
-        # Variant.new(name: resolved)
+        requested_variant || 'control'
       end
 
       # Tracking behavior can be implemented to link an event to an experiment.
       @tracking_behavior = lambda do |action, event_args|
-        # An example of using a generic logger to track events:
-        (event_args[:context] ||= []) << context.signature.merge(group: variant.name)
-        Configuration.logger.info "EXPERIMENT[#{name}] #{action}: #{event_args}"
-
-        # Using snowplow to track events:
-        # (event_args[:context] ||= []) << SnowplowTracker::SelfDescribingJson.new(
-        #   'iglu:com.gitlab/gitlab_experiment/jsonschema/1-0-0',
-        #   experiment: context.signature.merge(group: variant.name)
-        # )
-        #
-        # Tracking.event(name, action, **event_args)
+        Configuration.logger.info "Gitlab::Experiment[#{name}] #{action}: #{event_args.merge(signature: signature)}"
+      end
+
+      # Called at the end of every experiment run, with the results. You may
+      # want to push the experiment into the client or push results elsewhere.
+      @publishing_behavior = lambda do |_result|
+        track(:assignment) # this will call the config.tracking_behavior
       end
 
       # Algorithm that consistently generates a hash key for a given hash map.
@@ -55,7 +39,7 @@ module Gitlab
 
       class << self
         attr_accessor :name_prefix, :logger
-        attr_accessor :variant_resolver, :tracking_behavior, :context_hash_strategy
+        attr_accessor :variant_resolver, :tracking_behavior, :publishing_behavior, :context_hash_strategy
       end
     end
   end

data/lib/gitlab/experiment/context.rb

@@ -3,6 +3,8 @@
 module Gitlab
   class Experiment
     class Context
+      DNT_REGEXP = /^(true|t|yes|y|1|on)$/i.freeze
+
       def initialize(experiment)
         @experiment = experiment
         @value = {}
@@ -27,32 +29,45 @@ module Gitlab
       end
 
       def signature
-        @signature ||= { key: key_for(@value), migration_keys: migration_keys }.compact
+        @signature ||= {
+          key: key_for(@value),
+          migration_keys: migration_keys,
+          variant: @experiment.variant.name
+        }.compact
       end
 
       private
 
-      def migrate_cookie_to_user_id(value)
-        return value unless (request = value.delete(:request))
-        return value if request.headers['DNT'].to_s.match?(/^(true|t|yes|y|1|on)$/i)
-
-        cookie_name = [@experiment.name, 'id'].join('_')
-        cookie_value = request.cookie_jar.signed[cookie_name]
-
-        if value[:user_id].blank? && cookie_value.blank?
-          request.cookie_jar.permanent.signed[cookie_name] = {
-            value: value[:user_id] = SecureRandom.uuid, secure: true, domain: :all, httponly: true
-          }
-        elsif cookie_value # we know via the cookie
-          if value[:user_id].blank?
-            value[:user_id] = cookie_value
-          else
-            @migrations_with << { user_id: cookie_value }
-            request.cookie_jar.delete(cookie_name, domain: :all)
-          end
-        end
-
-        value
+      def migrate_cookie_to_user_id(hash)
+        return hash unless (request = hash.delete(:request))
+        return hash unless request.respond_to?(:headers) && request.respond_to?(:cookie_jar)
+        return hash if request.headers['DNT'].to_s.match?(DNT_REGEXP)
+
+        jar = request.cookie_jar
+        resolver = [jar, hash, :user_id, jar.signed[cookie_name]].compact
+        resolve_cookie(*resolver) or generate_cookie(*resolver)
+      end
+
+      def cookie_name
+        @cookie_name ||= [@experiment.name, 'id'].join('_')
+      end
+
+      def resolve_cookie(jar, hash, key, cookie = nil)
+        return if cookie.blank? && hash[key].blank?
+        return hash.merge(key => cookie) if hash[key].blank?
+
+        @migrations_with << { user_id: cookie }
+        jar.delete(cookie_name, domain: :all)
+
+        hash
+      end
+
+      def generate_cookie(jar, hash, key, cookie = SecureRandom.uuid)
+        jar.permanent.signed[cookie_name] = {
+          value: cookie, secure: true, domain: :all, httponly: true
+        }
+
+        hash.merge(key => cookie)
       end
 
       def migration_keys

data/lib/gitlab/experiment/dsl.rb

@@ -4,6 +4,7 @@ module Gitlab
   class Experiment
     module Dsl
       def experiment(name, variant_name = nil, **context, &block)
+        context[:request] ||= request if respond_to?(:request)
         Experiment.run(name, variant_name, **context, &block)
       end
     end

data/lib/gitlab/experiment/engine.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+module Gitlab
+  class Experiment
+    class Engine < ::Rails::Engine
+      config.after_initialize do
+        # Add out experiment method to the base controller.
+        ActionController::Base.include(Dsl) if defined?(ActionController)
+      end
+    end
+  end
+end

data/lib/gitlab/experiment/version.rb

@@ -2,6 +2,6 @@
 
 module Gitlab
   class Experiment
-    VERSION = '0.2.0'
+    VERSION = '0.2.1'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: gitlab-experiment
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.1
 platform: ruby
 authors:
 - GitLab
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-09-20 00:00:00.000000000 Z
+date: 2020-09-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: scientist
@@ -39,10 +39,14 @@ extra_rdoc_files: []
 files:
 - LICENSE.txt
 - README.md
+- lib/generators/gitlab_experiment/install/POST_INSTALL
+- lib/generators/gitlab_experiment/install/install_generator.rb
+- lib/generators/gitlab_experiment/install/templates/initializer.rb
 - lib/gitlab/experiment.rb
 - lib/gitlab/experiment/configuration.rb
 - lib/gitlab/experiment/context.rb
 - lib/gitlab/experiment/dsl.rb
+- lib/gitlab/experiment/engine.rb
 - lib/gitlab/experiment/variant.rb
 - lib/gitlab/experiment/version.rb
 homepage: https://gitlab.com/gitlab-org/gitlab-experiment