gitlab-experiment 0.4.5 → 0.4.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 81476752521c6ead83308324fdcf360db7b8182bab340c9ffe7ad4eec21b998e
-  data.tar.gz: 006d94ff92104bef820be7d6c7c9638b20a6e0a4a533439fca056b4cba31a0bc
+  metadata.gz: a1cbae46377202f37922a880d0c1cde571de29e96dd14fbe879e05718ba6d8ea
+  data.tar.gz: 48258191a1de1e9ca5de40428d481466c3a68272422783610dbfbd112a834ea7
 SHA512:
-  metadata.gz: a5644f3cf6798ddc27034b8857cf03bb77904b3b5c8a80b84c398134ad143b919e16ddc23746a58e63b71d0464c92ec5cce682ddb2039e471b55cfeb42e3cd4a
-  data.tar.gz: 2dd2adf62427f96a9745d9681ee87b5621acf43fc4dae33ffefdadfad5af6ec3a0ad1e79e601a4d6260e61fa64faf635af9ae753e3fb474e033af7d2d7d55381
+  metadata.gz: 8b7d5b2e1725e988bc55653baa1ece89c2d4b31f9cafc7e4d563bc6c093071abdf8fea89abf758658740f34168eb185a01beada94d331e85bf6c9b5cad78a345
+  data.tar.gz: cb295e0547616c5c7b47bcec319e27693e7017bbf14a0026b02afb6324715922a01620485db0878cf806f3ac89306f22e812cd44b812da871e3dc77157aa2386

data/README.md

@@ -15,7 +15,7 @@ When we discuss the behavior of this gem, we'll use terms like experiment, conte
 - `candidate` defines that there's one experimental code path.
 - `variant(s)` is used when more than one experimental code path exists.
 
-Candidate and variant are the same concept, but simplify how we speak about experimental paths.<br clear="all">
+Candidate and variant are the same concept, but simplify how we speak about experimental paths -- this concept is also widely referred to as the "experiment group".<br clear="all">
 
 [[_TOC_]]
 
@@ -100,26 +100,33 @@ Here are some examples of what you can introduce once you have a custom experime
 
 ```ruby
 class NotificationToggleExperiment < ApplicationExperiment
-  # Segment any account less than 2 weeks old into the candidate, without
+  # Exclude any users that aren't me.
+  exclude :users_named_richard
+
+  # Segment any account older than 2 weeks into the candidate, without
   # asking the variant resolver to decide which variant to provide.
-  segment :account_age, variant: :candidate
+  segment :old_account?, variant: :candidate
 
   # Define the default control behavior, which can be overridden at
   # experiment time.
   def control_behavior
-    render_toggle
+    # render_toggle
   end
 
   # Define the default candidate behavior, which can be overridden
   # at experiment time.
   def candidate_behavior
-    render_button
+    # render_button
   end
 
   private
 
-  def account_age
-    context.actor && context.actor.created_at < 2.weeks.ago
+  def users_named_richard
+    context.actor.first_name == 'Richard'
+  end
+
+  def old_account?
+    context.actor.created_at < 2.weeks.ago
   end
 end
 
@@ -177,7 +184,7 @@ Generally, defining segmentation rules is a better way to approach routing into
 experiment(:notification_toggle, :no_interface, actor: user) do |e|
   e.use { render_toggle } # control
   e.try { render_button } # candidate
-  e.try(:no_interface) { no_interface! } # variant
+  e.try(:no_interface) { no_interface! } # no_interface variant
 end
 ```
 
@@ -210,17 +217,53 @@ This library comes with the capability to segment contexts into a specific varia
 Segmentation can be achieved by using a custom experiment class and specifying the segmentation rules at a class level.
 
 ```ruby
-class NotificationToggleExperiment < ApplicationExperiment
-  segment(variant: :variant_one) { context.actor.username == 'jejacks0n' }
-  segment(variant: :variant_two) { context.actor.created_at < 2.weeks.ago }
+class ExampleExperiment < ApplicationExperiment
+  segment(variant: :variant_one) { context.actor.first_name == 'Richard' }
+  segment :old_account?, variant: :variant_two
+  
+  private
+
+  def old_account?
+    context.actor.created_at < 2.weeks.ago
+  end
 end
 ```
 
-In the previous examples, any user with the username `'jejacks0n'` would always receive the experience defined in "variant_one". As well, any account less than 2 weeks old would get the alternate experience defined in "variant_two".
+In the previous examples, any user named `'Richard'` would always receive the experience defined in "variant_one". As well, any account older than 2 weeks old would get the alternate experience defined in "variant_two".
 
 When an experiment is run, the segmentation rules are executed in the order they're defined. The first segmentation rule to produce a truthy result is the one which gets used to assign the variant. The remaining segmentation rules are skipped.
 
-This means that any user with the name `'jejacks0n'`, regardless of account age, will always be provided the experience as defined in "variant_one". 
+This means that any user named `'Richard'`, regardless of account age, will always be provided the experience as defined in "variant_one". If you wanted the opposite logic, you can flip the order.
+
+### Exclusion rules
+
+Exclusion rules are similar to segmentation rules, but are intended to determine if a context should even be considered as something we should track events towards. Exclusion means we don't care about the events in relation to the given context.
+
+```ruby
+class ExampleExperiment < ApplicationExperiment
+  exclude :old_account?, ->{ context.actor.first_name == 'Richard' }
+  
+  private
+
+  def old_account? 
+    context.actor.created_at < 2.weeks.ago
+  end
+end
+```
+
+The previous examples will exclude all users named `'Richard'` as well as any account older than 2 weeks old. Not only will they be given the control behavior, but no events will be tracked in these cases as well. 
+
+You may need to check exclusion in custom tracking logic by calling `should_track?`:
+
+```ruby
+def expensive_tracking_logic
+  return unless should_track?
+  
+  track(:my_event, value: expensive_method_call)
+end
+```
+
+Note: Exclusion rules aren't the best way to determine if an experiment is enabled. There's an `enabled?` method that can be overridden to have a high-level way of determining if an experiment should be running and tracking at all. This `enabled?` check should be as efficient as possible because it's the first early opt out path an experiment can implement.
 
 ### Return value
 
@@ -269,7 +312,7 @@ In providing the context migration data, we can resolve an experience and its ev
 
 ```ruby
 # Migrate just the `:version` portion of the previous context, `{ actor: project, version: 1 }`:
-experiment(:my_experiment, actor: project, version: 2, migrated_with: { version: 1 })
+experiment(:example, actor: project, version: 2, migrated_with: { version: 1 })
 ```
 
 You can add or remove context by providing a `migrated_from` option. This approach expects a full context replacement -- i.e. what it was before you added or removed the new context key.
@@ -278,7 +321,7 @@ If you wanted to introduce a `version` to your context, provide the full previou
 
 ```ruby
 # Migrate the full context from `{ actor: project }` to `{ actor: project, version: 1 }`:
-experiment(:my_experiment, actor: project, version: 1, migrated_from: { actor: project })
+experiment(:example, actor: project, version: 1, migrated_from: { actor: project })
 ```
 
 This can impact an experience if you:
@@ -299,23 +342,23 @@ To read and write cookies, we provide the `request` from within the controller a
 You'll need to provide the `request` as an option to the experiment if it's outside of the controller and views.
 
 ```ruby
-experiment(:my_experiment, actor: user, request: request)
+experiment(:example, actor: user, request: request)
 ```
 
 The cookie isn't set if the `actor` key isn't present at all in the context. Meaning that when no `actor` key is provided, the cookie will not be set.
 
 ```ruby
 # actor is not present, so no cookie is set
-experiment(:my_experiment, project: project)
+experiment(:example, project: project)
 
 # actor is present and is nil, so the cookie is set and used
-experiment(:my_experiment, actor: nil, project: project)
+experiment(:example, actor: nil, project: project)
 
 # actor is present and set to a value, so no cookie is set
-experiment(:my_experiment, actor: user, project: project)
+experiment(:example, actor: user, project: project)
 ```
 
-For edge cases, you can pass the cookie through by assigning it yourself -- e.g. `actor: request.cookie_jar.signed['my_experiment_actor']`. The cookie name is the full experiment name (including any configured prefix) with `_actor` appended -- e.g. `gitlab_notification_toggle_actor` for the `:notification_toggle` experiment key with a configured prefix of `gitlab`.
+For edge cases, you can pass the cookie through by assigning it yourself -- e.g. `actor: request.cookie_jar.signed['example_actor']`. The cookie name is the full experiment name (including any configured prefix) with `_actor` appended -- e.g. `gitlab_notification_toggle_actor` for the `:notification_toggle` experiment key with a configured prefix of `gitlab`.
 
 ## Configuration
 
@@ -373,6 +416,99 @@ Gitlab::Experiment.configure do |config|
 end
 ```
 
+## Testing (rspec support)
+
+This gem comes with some rspec helpers and custom matchers. These are in flux at the time of writing.
+
+First, require the rspec support file:
+
+```ruby
+require 'gitlab/experiment/rspec'
+```
+
+This mixes in some of the basics, but the matchers and other aspects need to be included. This happens automatically for files in `spec/experiments`, but for other files and specs you want to include it in, you can specify the `:experiment` type:
+
+```ruby
+it "tests", :experiment do
+end
+```
+
+### Stub helpers
+
+You can stub experiments using `stub_experiments`. Pass it a hash using experiment names as the keys and the variants you want each to resolve to as the values:
+
+```ruby
+# Ensures the experiments named `:example` & `:example2` are both
+# "enabled" and that each will resolve to the given variant
+# (`:my_variant` & `:control` respectively).
+stub_experiments(example: :my_variant, example2: :control)
+
+experiment(:example) do |e|
+  e.enabled? # => true
+  e.variant.name # => 'my_variant'
+end
+
+experiment(:example2) do |e|
+  e.enabled? # => true
+  e.variant.name # => 'control'
+end
+```
+
+### Exclusion and segmentation matchers
+
+You can also easily test the exclusion and segmentation matchers.
+
+```ruby
+class ExampleExperiment < ApplicationExperiment
+  exclude { context.actor.first_name == 'Richard' }
+  segment(variant: :candidate) { context.actor.username == 'jejacks0n' }
+end
+
+excluded = double(username: 'rdiggitty', first_name: 'Richard')
+segmented = double(username: 'jejacks0n', first_name: 'Jeremy')
+
+# exclude matcher
+expect(experiment(:example)).to exclude(actor: excluded)
+expect(experiment(:example)).not_to exclude(actor: segmented)
+
+# segment matcher
+expect(experiment(:example)).to segment(actor: segmented).into(:candidate)
+expect(experiment(:example)).not_to segment(actor: excluded)
+```
+
+### Tracking matcher
+
+Tracking events is a major aspect of experimentation, and because of this we try to provide a flexible way to ensure your tracking calls are covered.
+
+You can do this on the instance level or at an "any instance" level. At an instance level this is pretty straight forward:
+
+```ruby
+subject = experiment(:example)
+
+expect(subject).to track(:my_event)
+
+subject.track(:my_event)
+```
+
+You can use the `on_any_instance` chain method to specify that it could happen on any instance of the experiment. This can be useful if you're calling `experiment(:example).track` downstream:
+
+```ruby
+expect(experiment(:example)).to track(:my_event).on_any_instance
+
+experiment(:example).track(:my_event)
+```
+
+And here's a full example of the methods that can be chained onto the `track` matcher:
+
+```ruby
+expect(experiment(:example)).to track(:my_event, value: 1, property: '_property_')
+  .on_any_instance
+  .with_context(foo: :bar)
+  .for(:variant_name)
+
+experiment(:example, :variant_name, foo: :bar).track(:my_event, value: 1, property: '_property_')
+```
+
 ## Tracking, anonymity and GDPR
 
 We generally try not to track things like user identifying values in our experimentation. What we can and do track is the "experiment experience" (a.k.a. the context key).

data/lib/gitlab/experiment.rb

@@ -2,10 +2,11 @@
 
 require 'scientist'
 require 'active_support/callbacks'
+require 'active_support/cache'
 require 'active_support/core_ext/object/blank'
 require 'active_support/core_ext/string/inflections'
 
-require 'gitlab/experiment/caching'
+require 'gitlab/experiment/cache'
 require 'gitlab/experiment/callbacks'
 require 'gitlab/experiment/configuration'
 require 'gitlab/experiment/cookies'
@@ -18,7 +19,7 @@ require 'gitlab/experiment/engine' if defined?(Rails::Engine)
 module Gitlab
   class Experiment
     include Scientist::Experiment
-    include Caching
+    include Cache
     include Callbacks
 
     class << self
@@ -58,16 +59,18 @@ module Gitlab
       raise ArgumentError, 'name is required' if name.blank? && self.class.base?
 
       @name = self.class.experiment_name(name, suffix: false)
-      @excluded = []
       @context = Context.new(self, **context)
       @variant_name = cache_variant(variant_name) { nil } if variant_name.present?
 
-      exclude { !@context.trackable? }
       compare { false }
 
       yield self if block_given?
     end
 
+    def inspect
+      "#<#{self.class.name || 'AnonymousClass'}:#{format('0x%016X', __id__)} @name=#{name} @context=#{context.value}>"
+    end
+
     def context(value = nil)
       return @context if value.blank?
 
@@ -81,27 +84,25 @@ module Gitlab
       end
 
       @variant_name = value unless value.blank?
-      @variant_name ||= :control if excluded?
 
-      @resolving_variant = true
-      resolved = :control
-      if (result = cache_variant(@variant_name) { resolve_variant_name }).present?
-        @variant_name = resolved = result.to_sym
+      if enabled?
+        @variant_name ||= :control if excluded?
+
+        @resolving_variant = true
+        if (result = cache_variant(@variant_name) { resolve_variant_name }).present?
+          @variant_name = result.to_sym
+        end
       end
 
-      Variant.new(name: resolved.to_s)
+      Variant.new(name: (@variant_name || :control).to_s)
     ensure
       @resolving_variant = false
     end
 
-    def exclude(&block)
-      @excluded << block
-    end
-
     def run(variant_name = nil)
       @result ||= begin
         variant_name = variant(variant_name).name
-        run_callbacks(variant_assigned? ? :unsegmented_run : :segmented_run) do
+        run_callbacks(run_with_segmenting? ? :segmented_run : :unsegmented_run) do
           super(@variant_name ||= variant_name)
         end
       end
@@ -112,7 +113,7 @@ module Gitlab
     end
 
     def track(action, **event_args)
-      return if excluded?
+      return unless should_track?
 
       instance_exec(action, event_args, &Configuration.tracking_behavior)
     end
@@ -143,25 +144,26 @@ module Gitlab
       { variant: variant.name, experiment: name }.merge(context.signature)
     end
 
-    def enabled?
-      true
+    def id
+      "#{name}:#{key_for(context.value)}"
     end
+    alias_method :session_id, :id
 
-    def excluded?
-      @excluded.any? { |exclude| exclude.call(self) }
+    def flipper_id
+      "Experiment;#{id}"
     end
 
-    def variant_assigned?
-      !@variant_name.nil?
+    def enabled?
+      true
     end
 
-    def id
-      "#{name}:#{key_for(context.value)}"
+    def excluded?
+      @excluded ||= !@context.trackable? || # adhere to DNT headers
+        !run_callbacks(:exclusion_check) { :not_excluded } # didn't pass exclusion check
     end
-    alias_method :session_id, :id
 
-    def flipper_id
-      "Experiment;#{id}"
+    def should_track?
+      enabled? && !excluded?
     end
 
     def key_for(hash)
@@ -170,6 +172,14 @@ module Gitlab
 
     protected
 
+    def run_with_segmenting?
+      !variant_assigned? && enabled? && !excluded?
+    end
+
+    def variant_assigned?
+      !@variant_name.nil?
+    end
+
     def resolve_variant_name
       instance_exec(@variant_name, &Configuration.variant_resolver)
     end

data/lib/gitlab/experiment/cache.rb

@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+module Gitlab
+  class Experiment
+    module Cache
+      autoload :RedisHashStore, 'gitlab/experiment/cache/redis_hash_store.rb'
+
+      class Interface
+        attr_reader :store, :key
+
+        def initialize(experiment, store)
+          @experiment = experiment
+          @store = store
+          @key = experiment.cache_key
+        end
+
+        def read
+          store.read(key)
+        end
+
+        def write(value = nil)
+          store.write(key, value || @experiment.variant.name)
+        end
+
+        def delete
+          store.delete(key)
+        end
+
+        def attr_get(name)
+          store.read(@experiment.cache_key(name, suffix: :attrs))
+        end
+
+        def attr_set(name, value)
+          store.write(@experiment.cache_key(name, suffix: :attrs), value)
+        end
+
+        def attr_inc(name, amount = 1)
+          store.increment(@experiment.cache_key(name, suffix: :attrs), amount)
+        end
+      end
+
+      def cache
+        @cache ||= Interface.new(self, Configuration.cache)
+      end
+
+      def cache_variant(specified = nil, &block)
+        return (specified.presence || yield) unless cache.store
+
+        result = migrated_cache_fetch(cache.store, &block)
+        return result unless specified.present?
+
+        cache.write(specified) if result != specified
+        specified
+      end
+
+      def cache_key(key = nil, suffix: nil)
+        "#{[name, suffix].compact.join('_')}:#{key || context.signature[:key]}"
+      end
+
+      private
+
+      def migrated_cache_fetch(store, &block)
+        migrations = context.signature[:migration_keys]&.map { |key| cache_key(key) } || []
+        migrations.find do |old_key|
+          next unless (value = store.read(old_key))
+
+          store.write(cache_key, value)
+          store.delete(old_key)
+          return value
+        end
+
+        store.fetch(cache_key, &block)
+      end
+    end
+  end
+end

data/lib/gitlab/experiment/cache/redis_hash_store.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+require 'active_support/notifications'
+
+# This cache strategy is an implementation on top of the redis hash data type,
+# that also adheres to the ActiveSupport::Cache::Store interface. It's a good
+# example of how to build a custom caching strategy for Gitlab::Experiment, and
+# is intended to be a long lived cache -- until the experiment is cleaned up.
+#
+# The data structure:
+#   key: experiment.name
+#   fields: context key => variant name
+#
+# Gitlab::Experiment::Configuration.cache = Gitlab::Experiment::Cache::RedisHashStore.new(
+#   pool: -> { Gitlab::Redis::SharedState.with { |redis| yield redis } }
+# )
+module Gitlab
+  class Experiment
+    module Cache
+      class RedisHashStore < ActiveSupport::Cache::Store
+        # Clears the entire cache for a given experiment. Be careful with this
+        # since it would reset all resolved variants for the entire experiment.
+        def clear(key:)
+          key = hkey(key)[0] # extract only the first part of the key
+          pool do |redis|
+            case redis.type(key)
+            when 'hash', 'none'
+              redis.del(key) # delete the primary experiment key
+              redis.del("#{key}_attrs") # delete the experiment attributes key
+            else raise ArgumentError, 'invalid call to clear a non-hash cache key'
+            end
+          end
+        end
+
+        def increment(key, amount = 1)
+          pool { |redis| redis.hincrby(*hkey(key), amount) }
+        end
+
+        private
+
+        def pool(&block)
+          raise ArgumentError, 'missing block' unless block.present?
+
+          @options[:pool].call(&block)
+        end
+
+        def hkey(key)
+          key.to_s.split(':') # this assumes the default strategy in gitlab-experiment
+        end
+
+        def read_entry(key, **options)
+          value = pool { |redis| redis.hget(*hkey(key)) }
+          value.nil? ? nil : ActiveSupport::Cache::Entry.new(value)
+        end
+
+        def write_entry(key, entry, **options)
+          return false if entry.value.blank? # don't cache any empty values
+
+          pool { |redis| redis.hset(*hkey(key), entry.value) }
+        end
+
+        def delete_entry(key, **options)
+          pool { |redis| redis.hdel(*hkey(key)) }
+        end
+      end
+    end
+  end
+end

data/lib/gitlab/experiment/callbacks.rb

@@ -9,9 +9,23 @@ module Gitlab
       included do
         define_callbacks(:unsegmented_run)
         define_callbacks(:segmented_run)
+        define_callbacks(:exclusion_check, skip_after_callbacks_if_terminated: true)
       end
 
       class_methods do
+        def exclude(*filter_list, **options, &block)
+          filters = filter_list.unshift(block).compact.map do |filter|
+            result_lambda = ActiveSupport::Callbacks::CallTemplate.build(filter, self).make_lambda
+            lambda do |target|
+              throw(:abort) if target.instance_variable_get(:'@excluded') || result_lambda.call(target, nil) == true
+            end
+          end
+
+          raise ArgumentError, 'no filters provided' if filters.empty?
+
+          set_callback(:exclusion_check, :before, *filters, options)
+        end
+
         def segment(*filter_list, variant:, **options, &block)
           filters = filter_list.unshift(block).compact.map do |filter|
             result_lambda = ActiveSupport::Callbacks::CallTemplate.build(filter, self).make_lambda

data/lib/gitlab/experiment/context.rb

@@ -42,6 +42,14 @@ module Gitlab
         @signature ||= { key: @experiment.key_for(@value), migration_keys: migration_keys }.compact
       end
 
+      def method_missing(method_name, *)
+        @value.include?(method_name.to_sym) ? @value[method_name.to_sym] : super
+      end
+
+      def respond_to_missing?(method_name, *)
+        @value.include?(method_name.to_sym) ? true : super
+      end
+
       private
 
       def process_migrations(value)

data/lib/gitlab/experiment/engine.rb

@@ -4,13 +4,18 @@ module Gitlab
   class Experiment
     class Engine < ::Rails::Engine
       def self.include_dsl
-        ActionController::Base.include(Dsl)
-        ActionController::Base.helper_method(:experiment)
-      end
+        if defined?(ActionController)
+          ActionController::Base.include(Dsl)
+          ActionController::Base.helper_method(:experiment)
+        end
+
+        return unless defined?(ActionMailer)
 
-      config.after_initialize do
-        include_dsl if defined?(ActionController)
+        ActionMailer::Base.include(Dsl)
+        ActionMailer::Base.helper_method(:experiment)
       end
+
+      config.after_initialize { include_dsl }
     end
   end
 end

data/lib/gitlab/experiment/rspec.rb

@@ -0,0 +1,193 @@
+# frozen_string_literal: true
+
+module Gitlab
+  class Experiment
+    module RSpecHelpers
+      def stub_experiments(experiments)
+        experiments.each do |name, variant|
+          variant = :control if variant == false
+          raise ArgumentError, 'variant must be a symbol or false' unless variant.is_a?(Symbol)
+
+          klass = Gitlab::Experiment.send(:constantize, name) # rubocop:disable GitlabSecurity/PublicSend
+
+          # We have to use this high level any_instance behavior as there's
+          # not an alternative that allows multiple wrappings of `new`.
+          allow_any_instance_of(klass).to receive(:enabled?).and_return(true)
+          allow_any_instance_of(klass).to receive(:resolve_variant_name).and_return(variant.to_s)
+        end
+      end
+
+      def wrapped_experiment(experiment, shallow: false, failure: nil, &block)
+        if shallow
+          yield experiment if block.present?
+          return experiment
+        end
+
+        receive_wrapped_new = receive(:new).and_wrap_original do |new, *new_args, &new_block|
+          instance = new.call(*new_args)
+          instance.tap(&block) if block.present?
+          instance.tap(&new_block) if new_block.present?
+          instance
+        end
+
+        klass = experiment.class == Class ? experiment : experiment.class
+        if failure
+          expect(klass).to receive_wrapped_new, failure
+        else
+          allow(klass).to receive_wrapped_new
+        end
+      end
+    end
+
+    module RSpecMatchers
+      extend RSpec::Matchers::DSL
+
+      def require_experiment(experiment, matcher_name, classes: false)
+        klass = experiment.class == Class ? experiment : experiment.class
+        unless klass <= Gitlab::Experiment
+          raise(
+            ArgumentError,
+            "#{matcher_name} matcher is limited to experiment instances#{classes ? ' and classes' : ''}"
+          )
+        end
+
+        if experiment == klass && !classes
+          raise ArgumentError, "#{matcher_name} matcher requires an instance of an experiment"
+        end
+
+        experiment != klass
+      end
+
+      matcher :exclude do |context|
+        ivar = :'@excluded'
+
+        match do |experiment|
+          require_experiment(experiment, 'exclude')
+          experiment.context(context)
+
+          experiment.instance_variable_set(ivar, nil)
+          !experiment.run_callbacks(:exclusion_check) { :not_excluded }
+        end
+
+        failure_message do
+          %(expected #{context} to be excluded)
+        end
+
+        failure_message_when_negated do
+          %(expected #{context} not to be excluded)
+        end
+      end
+
+      matcher :segment do |context|
+        ivar = :'@variant_name'
+
+        match do |experiment|
+          require_experiment(experiment, 'segment')
+          experiment.context(context)
+
+          experiment.instance_variable_set(ivar, nil)
+          experiment.run_callbacks(:segmented_run)
+
+          @actual = experiment.instance_variable_get(ivar)
+          @expected ? @actual.to_s == @expected.to_s : @actual.present?
+        end
+
+        chain :into do |expected|
+          raise ArgumentError, 'variant name must be provided' if expected.blank?
+
+          @expected = expected.to_s
+        end
+
+        failure_message do
+          %(expected #{context} to be segmented#{message_details})
+        end
+
+        failure_message_when_negated do
+          %(expected #{context} not to be segmented#{message_details})
+        end
+
+        def message_details
+          message = ''
+          message += %( into variant\n    expected variant: #{@expected}) if @expected
+          message += %(\n      actual variant: #{@actual}) if @actual
+          message
+        end
+      end
+
+      matcher :track do |event, *event_args|
+        match do |experiment|
+          expect_tracking_on(experiment, false, event, *event_args)
+        end
+
+        match_when_negated do |experiment|
+          expect_tracking_on(experiment, true, event, *event_args)
+        end
+
+        chain :for do |expected_variant|
+          raise ArgumentError, 'variant name must be provided' if expected.blank?
+
+          @expected_variant = expected_variant.to_s
+        end
+
+        chain(:with_context) { |expected_context| @expected_context = expected_context }
+        chain(:on_any_instance) { @on_self = false }
+
+        def expect_tracking_on(experiment, negated, event, *event_args)
+          @experiment = experiment
+          @on_self = true if require_experiment(experiment, 'track', classes: !@on_self) && @on_self.nil?
+          wrapped_experiment(experiment, shallow: @on_self, failure: failure_message(:no_new, event)) do |instance|
+            @experiment = instance
+            allow(@experiment).to receive(:track)
+
+            if negated
+              expect(@experiment).not_to receive_tracking_call_for(event, *event_args)
+            else
+              expect(@experiment).to receive_tracking_call_for(event, *event_args)
+            end
+          end
+        end
+
+        def receive_tracking_call_for(event, *event_args)
+          receive(:track).with(*[event, *event_args]) do # rubocop:disable CodeReuse/ActiveRecord
+            if @expected_variant
+              expect(@experiment.variant.name).to eq(@expected_variant), failure_message(:variant, event)
+            end
+
+            if @expected_context
+              expect(@experiment.context.value).to include(@expected_context), failure_message(:context, event)
+            end
+          end
+        end
+
+        def failure_message(failure_type, event)
+          case failure_type
+          when :variant
+            <<~MESSAGE.strip
+              expected #{@experiment.inspect} to have tracked #{event.inspect} for variant
+                  expected variant: #{@expected_variant}
+                    actual variant: #{@experiment.variant.name}
+            MESSAGE
+          when :context
+            <<~MESSAGE.strip
+              expected #{@experiment.inspect} to have tracked #{event.inspect} with context
+                  expected context: #{@expected_context}
+                    actual context: #{@experiment.context.value}
+            MESSAGE
+          when :no_new
+            %(expected #{@experiment.inspect} to have tracked #{event.inspect}, but no new instances were created)
+          end
+        end
+      end
+    end
+  end
+end
+
+RSpec.configure do |config|
+  config.include Gitlab::Experiment::RSpecHelpers
+  config.include Gitlab::Experiment::Dsl
+
+  config.include Gitlab::Experiment::RSpecMatchers, :experiment
+  config.define_derived_metadata(file_path: Regexp.new('/spec/experiments/')) do |metadata|
+    metadata[:type] = :experiment
+  end
+end

data/lib/gitlab/experiment/version.rb

@@ -2,6 +2,6 @@
 
 module Gitlab
   class Experiment
-    VERSION = '0.4.5'
+    VERSION = '0.4.10'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: gitlab-experiment
 version: !ruby/object:Gem::Version
-  version: 0.4.5
+  version: 0.4.10
 platform: ruby
 authors:
 - GitLab
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2021-01-12 00:00:00.000000000 Z
+date: 2021-02-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -65,13 +65,15 @@ files:
 - lib/generators/test_unit/experiment/experiment_generator.rb
 - lib/generators/test_unit/experiment/templates/experiment_test.rb.tt
 - lib/gitlab/experiment.rb
-- lib/gitlab/experiment/caching.rb
+- lib/gitlab/experiment/cache.rb
+- lib/gitlab/experiment/cache/redis_hash_store.rb
 - lib/gitlab/experiment/callbacks.rb
 - lib/gitlab/experiment/configuration.rb
 - lib/gitlab/experiment/context.rb
 - lib/gitlab/experiment/cookies.rb
 - lib/gitlab/experiment/dsl.rb
 - lib/gitlab/experiment/engine.rb
+- lib/gitlab/experiment/rspec.rb
 - lib/gitlab/experiment/variant.rb
 - lib/gitlab/experiment/version.rb
 homepage: https://gitlab.com/gitlab-org/gitlab-experiment

data/lib/gitlab/experiment/caching.rb

@@ -1,39 +0,0 @@
-# frozen_string_literal: true
-
-module Gitlab
-  class Experiment
-    module Caching
-      def cache_variant(specified = nil, &block)
-        cache = Configuration.cache
-        return (specified.presence || yield) unless cache
-
-        key, migrations = cache_strategy
-        result = migrated_cache(cache, migrations || [], key) || cache.fetch(key, &block)
-        return result unless specified.present?
-
-        cache.write(cache_key, specified) if result != specified
-        specified
-      end
-
-      def cache_key(key = nil)
-        "#{name}:#{key || context.signature[:key]}"
-      end
-
-      private
-
-      def cache_strategy
-        [cache_key, context.signature[:migration_keys]&.map { |key| cache_key(key) }]
-      end
-
-      def migrated_cache(cache, migrations, new_key)
-        migrations.find do |old_key|
-          next unless (value = cache.read(old_key))
-
-          cache.write(new_key, value)
-          cache.delete(old_key)
-          break value
-        end
-      end
-    end
-  end
-end