trailguide 0.1.17 → 0.1.18

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e4bc166a7ac69aa3e1c4eb9d72e43004431b7263ee49f29959bcbb50bce260fa
4
- data.tar.gz: 4371c7fd6844f62602bc71884855ada79c9d2cf1ff78a511ef139432ec98cced
3
+ metadata.gz: 4bfe3a4830dd35f2af7fe1849c9d92254a0f422db41a153e3360077fb7f6cde7
4
+ data.tar.gz: bbeed95f2e497288e37e99b000b193a907e176a73ce596cdd81110f381abba27
5
5
  SHA512:
6
- metadata.gz: 3d57bc137ce240bf6090a72a3f3fb7031b2488919c0bfb44b20d9389a918ca4be02b79c5dc3d20e7af68bfcba69b81eaddbce4b954fbd22cde21b51038f85398
7
- data.tar.gz: b7691b7ae52dd19c7d2a71d7417a2626a8dcfed7c3c8d544a5431580e24d5aac09378ee1d883471e206860bf219f234bb795e105c9141b38edfa6611259f1e14
6
+ metadata.gz: b6fe56eb7a87037e2c081d455ac304325071294a6180aedb0c4cda9dafd0382c4e9290e906b331f50c7e04bff01dfea342d5a265bddc38fd729d3635d6432d5e
7
+ data.tar.gz: 5ecc3ccb399e13216f3dd8e88c2ba9884cca993c930cfeb300ac27643e077abd0c151691e29533a6d9998d2323811a3875366f0032901755d4506ccdaae3a478
data/README.md CHANGED
@@ -1,28 +1,181 @@
1
1
  # TrailGuide
2
- Short description and motivation.
3
2
 
4
- ## Usage
5
- How to use my plugin.
3
+ TrailGuide is a rails engine providing a framework for running user experiments and A/B tests in rails apps.
6
4
 
7
- ## Installation
8
- Add this line to your application's Gemfile:
5
+ ## Acknowledgements
9
6
 
10
- ```ruby
7
+ This gem is heavily inspired by the [split gem](https://github.com/splitrb/split). I've used split many times in the past and am a fan. It's an excellent alternative to trailguide, and really your best bet if you're not using rails. If you've used split in the past, you'll probably see a lot of familiar concepts and similarly named configuration variables. Parts of this project are even loosely modeled after some of the more brilliant patterns in split - like the user adapters for persistence.
8
+
9
+ ### Motivation
10
+
11
+ While working on a project to more deeply integrate custom experiments into a rails app, I found myself digging into the split internals. Split has been the go-to for A/B testing in ruby for a while. It's grown and evolved over the years, but as I explored the codebase and the github repo it became clear I wouldn't be able to do a lot of what was required for the project without overriding much of the existing behavior. Additionally, there are some differing opinions and approaches taken here that directly conflicted with split's defaults - for example the way "combined experiments" work, or how split allows defining and running experiments directly inline, while trailguide requires configuration.
12
+
13
+ After spending so much time with split and struggling with some of the implementation, I saw what I thought was a clear model and path forward for a more customizable and extensible rails-focused framework.
14
+
15
+ ## Getting Started
16
+
17
+ ### Requirements
18
+
19
+ Currently only rails 5.x is officially supported, and trailguide requires redis as a datastore for experiment metadata.
20
+
21
+ `docker-compose` is a great way to run redis in development. Take a look at the `docker-compose.yml` in the root of this repo for an example.
22
+
23
+ ### Installation
24
+
25
+ Add this line to your Gemfile:
26
+
27
+ ```
11
28
  gem 'trailguide'
12
29
  ```
13
30
 
14
- And then execute:
15
- ```bash
16
- $ bundle
31
+ Then run `bundle install`.
32
+
33
+ ## Configuration
34
+
35
+ The core engine and base experiment class have a number of configuration flags available to customize behavior and hook into various pieces of functionality. The preferred way to configure trailguide is via a config initializer, and the gem sets it's config defaults via it's own initializer.
36
+
37
+ ```ruby
38
+ # config/initializers/trailguide.rb
39
+
40
+ TrailGuide.configure do |config|
41
+ config.redis = Redis.new(url: ENV['REDIS_URL'])
42
+ # ...
43
+ end
44
+
45
+ TrailGuide::Experiment.configure do |config|
46
+ config.algorithm = :weighted
47
+ # ...
48
+ end
49
+ ```
50
+
51
+ Take a look at `config/initializers/trailguide.rb` in this for a full list of defaults and examples of available configuration.
52
+
53
+ ### Defining Experiments
54
+
55
+ Before you can start running experiments in your app, you'll need to define and configure them. There are a few options for defining experiments - YAML files, a ruby DSL, or custom classes - and they all inherit the base `TrailGuide::Experiment.configuration` for defaults, which can be overridden per-experiment.
56
+
57
+ #### YAML
58
+
59
+ YAML files are an easy way to configure simple experiments. They can be put in `config/experiments.yml` or `config/experiments/**/*.yml`:
60
+
61
+ ```yaml
62
+ # config/experiments.yml
63
+
64
+ simple_ab:
65
+ variants:
66
+ - 'option_a'
67
+ - 'option_b'
68
+ ```
69
+
70
+ ```yaml
71
+ # config/experiments/search/widget.yml
72
+
73
+ search_widget:
74
+ start_manually: true
75
+ algorithm: 'distributed'
76
+ variants:
77
+ - 'basic'
78
+ - 'simple'
79
+ - 'advanced'
80
+ ```
81
+
82
+ #### Ruby DSL
83
+
84
+ The ruby DSL provides a more dynamic and flexible way to configure your experiments, and allows you to define custom behavior via callbacks and options. You can put these experiments in `config/experiments.rb` or `config/experiments/**/*.rb`:
85
+
86
+ ```ruby
87
+ # config/experiments.rb
88
+
89
+ experiment :search_widget do |config|
90
+ config.start_manually = true
91
+ config.algorithm = :distributed
92
+ config.allow_multiple_goals = true
93
+
94
+ variant :basic
95
+ variant :simple, control: true
96
+ variant :advanced
97
+
98
+ goal :interacted
99
+ goal :searched
100
+
101
+ on_choose do |experiment, variant, metadata|
102
+ # ... send a track to some third party service ...
103
+ end
104
+
105
+ on_convert do |experiment, variant, goal, metadata|
106
+ # ... send a track to some third party service ...
107
+ end
108
+ end
17
109
  ```
18
110
 
19
- Or install it yourself as:
20
- ```bash
21
- $ gem install trailguide
111
+ #### Custom Classes
112
+
113
+ You can also take it a step further and define your own custom experiment classes, inheriting from `TrailGuide::Experiment`. This allows you to add or override all sorts of additional behavior on top of all the standard configuration provided by the DSL. In fact, the YAML and ruby DSL configs both use this to parse experiments into anonmymous classes extending `TrailGuide::Experiment`.
114
+
115
+ You can put these classes anywhere rails will autoload them (or require them yourself), but I recommend `app/experiments/**/*.rb`:
116
+
117
+ ```ruby
118
+ # app/experiments/my_complex_experiment.rb
119
+
120
+ class MyComplexExperiment < TrailGuide::Experiment
121
+
122
+ # all standard experiment config goes in the `configure` block
123
+ configure do |config|
124
+ config.reset_manually = true
125
+
126
+ control :option_a
127
+ variant :option_b
128
+ variant :option_c
129
+ variant :option_d
130
+
131
+ on_start do |experiment|
132
+ # ... do some custom stuff when the experiment is started ...
133
+ end
134
+ end
135
+
136
+ # override the experiment `choose!` method, and maybe do some custom stuff
137
+ # depending on custom options you pass in
138
+ def choose!(**opts)
139
+ if opts[:foo] == :bar
140
+ return control
141
+ else
142
+ super(**opts)
143
+ end
144
+ end
145
+
146
+ def foobar
147
+ # ... you can define whatever other custom methods, mixins and behaviors ...
148
+ end
149
+
150
+ end
22
151
  ```
23
152
 
153
+ You can also use inheritance to setup base experiments and inherit configuration:
154
+
155
+ ```ruby
156
+
157
+ class ApplicationExperiment < TrailGuide::Experiment
158
+ configure do |config|
159
+ # ... config, variants, etc.
160
+ end
161
+ # ... custom behavior, etc.
162
+ end
163
+
164
+ class MyAppExperiment < ApplicationExperiment
165
+ # inherits config from ApplicationExperiment
166
+ end
167
+
168
+ class MyDefaultExperiment < TrailGuide::Experiment
169
+ # inherits from configured trailguide defaults
170
+ end
171
+ ```
172
+
173
+ ## Usage
174
+
24
175
  ## Contributing
176
+
25
177
  Contribution directions go here.
26
178
 
27
179
  ## License
180
+
28
181
  The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
@@ -1,5 +1,10 @@
1
- <div class="footer bg-light">
2
- <div class="text-right">
3
- <%= link_to "TrailGuide v#{TrailGuide::Version::VERSION}", "https://github.com/markrebec/trailguide", target: :blank, class: "text-muted" %>
1
+ <div class="footer container-fluid bg-light">
2
+ <div class="row">
3
+ <div class="col-sm text-left">
4
+ <%= link_to "TrailGuide v#{TrailGuide::Version::VERSION}", "https://github.com/markrebec/trailguide", target: :blank, class: "text-muted" %>
5
+ </div>
6
+ <div class="col-sm text-right">
7
+ <small class="text-muted"><%= TrailGuide.redis._client.id %><%= "/#{TrailGuide.redis.namespace}" if TrailGuide.redis.respond_to?(:namespace) %></small>
8
+ </div>
4
9
  </div>
5
10
  </div>
@@ -3,6 +3,12 @@
3
3
  <%= image_tag "trail_guide/trailguide.png" %>
4
4
  <%= TrailGuide::Admin.configuration.title %>
5
5
  <% end %>
6
+ <div class="col-sm text-center">
7
+ <strong class="total"><%= TrailGuide.catalog.to_a.count %></strong> experiments
8
+ <span>/</span>
9
+ <strong class="running"><%= TrailGuide.catalog.running.count %></strong> running
10
+ <span>/</span>
11
+ <strong class="stopped"><%= TrailGuide.catalog.stopped.count %></strong> stopped
12
+ </div>
6
13
  <span class="navbar-brand"><small class="text-muted"><%= TrailGuide::Admin.configuration.subtitle %></small></span>
7
14
  </nav>
8
-
@@ -0,0 +1,205 @@
1
+ # top-level trailguide rails engine configuration
2
+ #
3
+ TrailGuide.configure do |config|
4
+ # url string or initialized Redis object
5
+ config.redis = ENV['REDIS_URL']
6
+
7
+ # globally disable trailguide - returns control everywhere
8
+ config.disabled = false
9
+
10
+ # request param for overriding/previewing variants - allows previewing
11
+ # variants with request params
12
+ config.override_parameter = :experiment
13
+
14
+ # whether or not participants are allowed to enter variant groups in multiple
15
+ # experiments
16
+ #
17
+ # true participants are entered into any experiments they encounter
18
+ #
19
+ # false as soon as a participant enters an experiment, they are prevented
20
+ # from entering any future experiments (they will only ever be in a
21
+ # single experiment)
22
+ #
23
+ # :control participants can enter any number of experiments as long as they
24
+ # are in the control groups, but as soon as they enter a non-control
25
+ # variant in any experiment, they will be prevented from entering
26
+ # any future experiments (they may be in multiple experiments, but
27
+ # in the control group for all except potentially one)
28
+ config.allow_multiple_experiments = false
29
+
30
+ # the participant adapter for users
31
+ #
32
+ # :redis uses redis to persist user participation
33
+ # :cookie uses a cookie to persist user participation
34
+ # :session uses the rails session to persist user participation
35
+ # :anonymous does not persist, can only convert in the same script/request
36
+ # execution while holding onto a reference to the object
37
+ # :multi allows using multiple adapters based on logic you define - i.e.
38
+ # use :redis if :current_user is present or use :cookie if not
39
+ # :unity a custom adapter for unity that helps track experiments across
40
+ # logged in/out sessions and across devices
41
+ config.adapter = :cookie
42
+
43
+ # callback when your participant adapter fails to initialize, and trailguide
44
+ # falls back to the anonymous adapter
45
+ config.on_adapter_failover = -> (adapter, error) do
46
+ Rails.logger.error("#{error.class.name}: #{error.message}")
47
+ end
48
+
49
+ # list of user agents used by the default request filter proc below when
50
+ # provided, can be an array or a proc that returns an array
51
+ #
52
+ # if using a proc, make sure it returns an array:
53
+ # -> { return [...] }
54
+ #
55
+ # config.filtered_user_agents = []
56
+
57
+ # list of ip addresses used by the default request filter proc below when
58
+ # provided, can be an array or a proc that returns an array
59
+ #
60
+ # if using a proc, make sure it returns an array:
61
+ # -> { return [...] }
62
+ #
63
+ # config.filtered_ip_addresses = []
64
+
65
+ # default request filter logic uses the configured filtered ip and user
66
+ # agents above, all requests matching this filter will be excluded from being
67
+ # entered into experiments - used to block bots, crawlers, scrapers, etc.
68
+ config.request_filter = -> (context) do
69
+ is_preview? ||
70
+ is_filtered_user_agent? ||
71
+ is_filtered_ip_address?
72
+ end
73
+ end
74
+
75
+ # base experiment configuration
76
+ #
77
+ TrailGuide::Experiment.configure do |config|
78
+ # the default algorithm to use for new experiments
79
+ config.algorithm = :weighted
80
+
81
+ # whether or not experiments must be started manually
82
+ #
83
+ # true experiments must be started manually via the admin UI or console
84
+ # false experiments will start the first time they're encountered by a user
85
+ config.start_manually = true
86
+
87
+ # whether or not participants will be reset upon conversion
88
+ #
89
+ # true participants will only be entered into the experiment once, and the
90
+ # variant group they belong to is sticky for the duration
91
+ # false participants will be reset upon conversion and be able to re-enter
92
+ # the experiment if they encounter it again
93
+ config.reset_manually = true
94
+
95
+ # whether or not to enter participants into a variant when using the override
96
+ # parameter to preview variants
97
+ #
98
+ # true using overrides to preview experiments will enter participants into
99
+ # that variant group
100
+ # false using overrides to preview experiments will not enter participants
101
+ # into the experment (won't persist their variant group)
102
+ config.store_override = false
103
+
104
+ # whether or not we track participants when using the override parameter to
105
+ # preview variants
106
+ #
107
+ # true using overrides to preview experiments will increment the
108
+ # participant count for the override variant
109
+ # false using overrides to preview experiments will not increment the
110
+ # participant count for the override variant
111
+ config.track_override = false
112
+
113
+ # whether or not to continue tracking conversions after a winner has been
114
+ # selected in order to continue monitoring performance of the variant
115
+ #
116
+ # true continues to track conversions after a winner has been selected (as
117
+ # long as the experiment is still running)
118
+ # false all conversion and participation tracking stops once a winner has
119
+ # been selected
120
+ config.track_winner_conversions = false
121
+
122
+ # whether or not to allow multiple conversions of the same goal, or default
123
+ # conversion if no goals are defined
124
+ #
125
+ # true tracks multiple participant conversions for the same goal as long
126
+ # as they haven't been reset (see config.reset_manually)
127
+ # false prevents tracking multiple conversions for a single participant
128
+ config.allow_multiple_conversions = false
129
+
130
+ # whether or not to allow participants to convert for multiple defined goals
131
+ #
132
+ # true allows participants to convert more than one goal as long as they
133
+ # haven't been reset (see config.reset_manually)
134
+ # false prevents converting to multiple goals for a single participant
135
+ config.allow_multiple_goals = false
136
+
137
+ # whether or not to skip the request filtering for this experiment - can be
138
+ # useful when defining content-based experiments with custom algorithms which
139
+ # bucket participants strictly based on additional content metadata and you
140
+ # want to expose those variants to crawlers and bots
141
+ #
142
+ # true requests that would otherwise be filtered based on your
143
+ # TrailGuide.configuration.request_filter config will instead be
144
+ # allowed through to this experiment
145
+ # false default behavior, requests will be filtered based on your config
146
+ config.skip_request_filter = false
147
+
148
+ # callback when connecting to redis fails and trailguide falls back to always
149
+ # returning control variants
150
+ config.on_redis_failover = -> (experiment, error) do
151
+ Rails.logger.error("#{error.class.name}: #{error.message}")
152
+ end
153
+
154
+ # callback on experiment start, either manually via UI/console or
155
+ # automatically depending on config.start_manually, can be used for logging,
156
+ # tracking, etc.
157
+ #
158
+ # config.on_start = -> (experiment) { ... }
159
+
160
+ # callback on experiment stop manually via UI/console, can be used for
161
+ # logging, tracking, etc.
162
+ #
163
+ # config.on_stop = -> (experiment) { ... }
164
+
165
+ # callback on experiment resume manually via UI/console, can be used for
166
+ # logging, tracking, etc.
167
+ #
168
+ # config.on_resume = -> (experiment) { ... }
169
+
170
+ # callback on experiment reset manually via UI/console, can be used for
171
+ # logging, tracking, etc.
172
+ #
173
+ # config.on_reset = -> (experiment) { ... }
174
+
175
+ # callback when a winner is selected manually via UI/console, can be used for
176
+ # logging, tracking, etc.
177
+ #
178
+ # config.on_winner = -> (experiment, winner) { ... }
179
+
180
+
181
+ # callback when a participant is entered into a variant for the first time,
182
+ # can be used for logging, tracking, etc.
183
+ #
184
+ # config.on_choose = -> (experiment, variant, metadata) { ... }
185
+
186
+ # callback every time a participant is returned a variant in the experiment,
187
+ # can be used for logging, tracking, etc.
188
+ #
189
+ # config.on_use = -> (experiment, variant, metadata) { ... }
190
+
191
+ # callback when a participant converts for a variant in the experiment, can be
192
+ # used for logging, tracking, etc.
193
+ #
194
+ # config.on_convert = -> (experiment, variant, checkpoint, metadata) { ... }
195
+
196
+
197
+ # callback that can be used to modify the rollout of a selected winner - for
198
+ # example you could use a custom algorithm or even something like the flipper
199
+ # gem to do a "feature rollout" from your control variant to your winner for
200
+ # all users
201
+ #
202
+ # must return an experiment variant
203
+ #
204
+ # config.rollout_winner = -> (experiment, winner) { ... return variant }
205
+ end
@@ -1,14 +1,5 @@
1
1
  module TrailGuide
2
2
  class Catalog
3
- class DSL
4
- def self.experiment(name, &block)
5
- Class.new(TrailGuide::Experiment) do
6
- configure name: name
7
- configure &block
8
- end
9
- end
10
- end
11
-
12
3
  include Enumerable
13
4
 
14
5
  class << self
@@ -70,7 +61,7 @@ module TrailGuide
70
61
  name: name.to_s.underscore.to_sym,
71
62
  parent: combined,
72
63
  combined: [],
73
- variants: combined.configuration.variants.map { |var| Variant.new(experiment, var.name, metadata: var.metadata, weight: var.weight, control: var.control?) },
64
+ variants: combined.configuration.variants.map { |var| var.dup(experiment) }
74
65
  # TODO also map goals once they're separate classes
75
66
  })
76
67
  experiment
@@ -98,6 +89,18 @@ module TrailGuide
98
89
  end.flatten
99
90
  end
100
91
 
92
+ def started
93
+ to_a.select(&:started?)
94
+ end
95
+
96
+ def running
97
+ to_a.select(&:running?)
98
+ end
99
+
100
+ def stopped
101
+ to_a.select(&:stopped?)
102
+ end
103
+
101
104
  def find(name)
102
105
  if name.is_a?(Class)
103
106
  experiments.find { |exp| exp == name }
@@ -152,5 +155,18 @@ module TrailGuide
152
155
  def respond_to_missing?(meth, include_private=false)
153
156
  experiments.respond_to?(meth, include_private)
154
157
  end
158
+
159
+ class DSL
160
+ def self.experiment(name, &block)
161
+ Class.new(TrailGuide::Experiment) do
162
+ configure name: name
163
+ configure &block
164
+ end
165
+ end
166
+ end
167
+ end
168
+
169
+ def self.catalog
170
+ TrailGuide::Catalog.catalog
155
171
  end
156
172
  end
@@ -0,0 +1,46 @@
1
+ module TrailGuide
2
+ class Config < Canfig::Config
3
+ DEFAULT_KEYS = [
4
+ :redis, :disabled, :override_parameter, :allow_multiple_experiments,
5
+ :adapter, :on_adapter_failover, :filtered_ip_addresses,
6
+ :filtered_user_agents, :request_filter
7
+ ].freeze
8
+
9
+ def initialize(*args, **opts, &block)
10
+ args = args.concat(DEFAULT_KEYS)
11
+ super(*args, **opts, &block)
12
+ end
13
+
14
+ def configure(*args, &block)
15
+ super(*args) do |config|
16
+ yield(config, TrailGuide::Experiment.configuration) if block_given?
17
+ end
18
+ end
19
+
20
+ def redis
21
+ @redis ||= begin
22
+ if ['Redis', 'Redis::Namespace'].include?(self[:redis].class.name)
23
+ self[:redis]
24
+ else
25
+ Redis.new(url: self[:redis])
26
+ end
27
+ end
28
+ end
29
+
30
+ def filtered_user_agents
31
+ @filtered_user_agents ||= begin
32
+ uas = self[:filtered_user_agents]
33
+ uas = uas.call if uas.respond_to?(:call)
34
+ uas || []
35
+ end
36
+ end
37
+
38
+ def filtered_ip_addresses
39
+ @filtered_ip_addresses ||= begin
40
+ ips = self[:filtered_ip_addresses]
41
+ ips = ips.call if ips.respond_to?(:call)
42
+ ips || []
43
+ end
44
+ end
45
+ end
46
+ end
@@ -2,7 +2,9 @@ require "trail_guide/experiments/base"
2
2
 
3
3
  module TrailGuide
4
4
  class Experiment < Experiments::Base
5
+
5
6
  def self.inherited(child)
7
+ child.instance_variable_set :@configuration, Experiments::Config.new(child, inherit: self.configuration)
6
8
  TrailGuide.catalog.register(child)
7
9
  end
8
10
  end
@@ -6,7 +6,8 @@ module TrailGuide
6
6
  class << self
7
7
  delegate :metric, :algorithm, :control, :goals, :callbacks, :combined,
8
8
  :combined?, :allow_multiple_conversions?, :allow_multiple_goals?,
9
- :track_winner_conversions?, to: :configuration
9
+ :track_winner_conversions?, :start_manually?, :reset_manually?,
10
+ to: :configuration
10
11
  alias_method :funnels, :goals
11
12
 
12
13
  def configuration
@@ -17,10 +18,6 @@ module TrailGuide
17
18
  configuration.configure(*args, &block)
18
19
  end
19
20
 
20
- def resettable?
21
- !configuration.reset_manually
22
- end
23
-
24
21
  def experiment_name
25
22
  configuration.name
26
23
  end
@@ -35,7 +32,7 @@ module TrailGuide
35
32
 
36
33
  def run_callbacks(hook, *args)
37
34
  return unless callbacks[hook]
38
- return args[0] if hook == :return_winner
35
+ return args[0] if hook == :rollout_winner
39
36
  args.unshift(self)
40
37
  callbacks[hook].each do |callback|
41
38
  if callback.respond_to?(:call)
@@ -140,7 +137,8 @@ module TrailGuide
140
137
  algorithm: algorithm.name,
141
138
  variants: variants.as_json,
142
139
  goals: goals.as_json,
143
- resettable: resettable?,
140
+ start_manually: start_manually?,
141
+ reset_manually: reset_manually?,
144
142
  allow_multiple_conversions: allow_multiple_conversions?,
145
143
  allow_multiple_goals: allow_multiple_goals?
146
144
  },
@@ -159,9 +157,9 @@ module TrailGuide
159
157
 
160
158
  attr_reader :participant
161
159
  delegate :configuration, :experiment_name, :variants, :control, :goals,
162
- :storage_key, :running?, :started?, :started_at, :start!, :resettable?,
163
- :winner?, :allow_multiple_conversions?, :allow_multiple_goals?,
164
- :track_winner_conversions?, :callbacks, to: :class
160
+ :storage_key, :running?, :started?, :started_at, :start!,
161
+ :start_manually?, :reset_manually?, :winner?, :allow_multiple_conversions?,
162
+ :allow_multiple_goals?, :track_winner_conversions?, :callbacks, to: :class
165
163
 
166
164
  def initialize(participant)
167
165
  @participant = participant
@@ -172,7 +170,7 @@ module TrailGuide
172
170
  end
173
171
 
174
172
  def winner
175
- run_callbacks(:return_winner, self.class.winner)
173
+ run_callbacks(:rollout_winner, self.class.winner)
176
174
  end
177
175
 
178
176
  def choose!(override: nil, metadata: nil, **opts)
@@ -203,7 +201,7 @@ module TrailGuide
203
201
  start! unless started?
204
202
  return control unless running?
205
203
  return variants.find { |var| var == participant[storage_key] } if participating?
206
- return control unless TrailGuide.configuration.allow_multiple_experiments == true || !participant.participating_in_active_experiments?(TrailGuide.configuration.allow_multiple_experiments == false)
204
+ return control unless TrailGuide.configuration.allow_multiple_experiments == true || !participant.participating_in_active_experiments?(TrailGuide.configuration.allow_multiple_experiments == false)
207
205
 
208
206
  variant = algorithm_choose!(metadata: metadata)
209
207
  end
@@ -232,7 +230,7 @@ module TrailGuide
232
230
 
233
231
  variant = variants.find { |var| var == participant[storage_key] }
234
232
  # TODO eventually only reset if we're at the final goal in a funnel
235
- participant.converted!(variant, checkpoint, reset: resettable?)
233
+ participant.converted!(variant, checkpoint, reset: !reset_manually?)
236
234
  variant.increment_conversion!(checkpoint)
237
235
  run_callbacks(:on_convert, variant, checkpoint, metadata)
238
236
  variant
@@ -251,7 +249,7 @@ module TrailGuide
251
249
 
252
250
  def run_callbacks(hook, *args)
253
251
  return unless callbacks[hook]
254
- if hook == :return_winner
252
+ if hook == :rollout_winner
255
253
  callbacks[hook].reduce(args[0]) do |winner, callback|
256
254
  if callback.respond_to?(:call)
257
255
  callback.call(self, winner)
@@ -1,71 +1,76 @@
1
1
  module TrailGuide
2
2
  module Experiments
3
3
  class Config < Canfig::Config
4
- ENGINE_CONFIG_KEYS = [
4
+ DEFAULT_KEYS = [
5
+ :name, :summary, :preview_url, :algorithm, :metric, :variants, :goals,
5
6
  :start_manually, :reset_manually, :store_override, :track_override,
6
- :algorithm, :allow_multiple_conversions, :allow_multiple_goals,
7
- :track_winner_conversions
7
+ :combined, :allow_multiple_conversions, :allow_multiple_goals,
8
+ :track_winner_conversions, :skip_request_filter
8
9
  ].freeze
9
10
 
10
- def self.engine_config
11
- ENGINE_CONFIG_KEYS.map do |key|
12
- [key, TrailGuide.configuration.send(key.to_sym)]
13
- end.to_h
14
- end
11
+ CALLBACK_KEYS = [
12
+ :on_start, :on_stop, :on_resume, :on_winner, :on_reset, :on_delete,
13
+ :on_choose, :on_use, :on_convert,
14
+ :on_redis_failover,
15
+ :rollout_winner
16
+ ].freeze
15
17
 
16
- def self.default_config
17
- { name: nil,
18
- metric: nil,
18
+ def default_config
19
+ DEFAULT_KEYS.map do |key|
20
+ [key, nil]
21
+ end.to_h.merge({
19
22
  variants: [],
20
23
  goals: [],
21
- combined: [],
22
- summary: nil,
23
- preview_url: nil,
24
- }
25
- end
26
-
27
- def self.callbacks_config
28
- {
29
- callbacks: {
30
- on_choose: [TrailGuide.configuration.on_experiment_choose].flatten.compact,
31
- on_use: [TrailGuide.configuration.on_experiment_use].flatten.compact,
32
- on_convert: [TrailGuide.configuration.on_experiment_convert].flatten.compact,
33
- on_start: [TrailGuide.configuration.on_experiment_start].flatten.compact,
34
- on_stop: [TrailGuide.configuration.on_experiment_stop].flatten.compact,
35
- on_resume: [TrailGuide.configuration.on_experiment_resume].flatten.compact,
36
- on_winner: [TrailGuide.configuration.on_experiment_winner].flatten.compact,
37
- on_reset: [TrailGuide.configuration.on_experiment_reset].flatten.compact,
38
- on_delete: [TrailGuide.configuration.on_experiment_delete].flatten.compact,
39
- on_redis_failover: [TrailGuide.configuration.on_redis_failover].flatten.compact,
40
- return_winner: [TrailGuide.configuration.return_experiment_winner].flatten.compact,
41
- }
42
- }
24
+ combined: []
25
+ }).merge(callback_config)
26
+ end
27
+
28
+ def callback_config
29
+ CALLBACK_KEYS.map do |key|
30
+ [key, []]
31
+ end.to_h
43
32
  end
44
33
 
45
34
  attr_reader :experiment
46
35
 
47
36
  def initialize(experiment, *args, **opts, &block)
48
37
  @experiment = experiment
49
- opts = opts.merge(self.class.engine_config)
50
- opts = opts.merge(self.class.default_config)
51
- opts = opts.merge(self.class.callbacks_config)
38
+ opts = opts.merge(default_config)
39
+ ancestor = opts.delete(:inherit)
40
+ if ancestor.present?
41
+ keys = opts.keys.dup.concat(args).concat(DEFAULT_KEYS).concat(CALLBACK_KEYS).uniq
42
+ opts = opts.merge(ancestor.to_h.slice(*keys))
43
+ opts[:name] = nil
44
+ opts[:goals] = ancestor.goals.dup
45
+ opts[:combined] = ancestor.combined.dup
46
+ opts[:variants] = ancestor.variants.map { |var| var.dup(experiment) }
47
+ opts = opts.merge(ancestor.callbacks.map { |k,v| [k,[v].flatten.compact] }.to_h)
48
+ end
52
49
  super(*args, **opts, &block)
53
50
  end
54
51
 
55
- def resettable?
56
- !reset_manually
52
+ def start_manually?
53
+ !!start_manually
54
+ end
55
+
56
+ def reset_manually?
57
+ !!reset_manually
57
58
  end
58
59
 
59
60
  def allow_multiple_conversions?
60
- allow_multiple_conversions
61
+ !!allow_multiple_conversions
61
62
  end
62
63
 
63
64
  def allow_multiple_goals?
64
- allow_multiple_goals
65
+ !!allow_multiple_goals
65
66
  end
66
67
 
67
68
  def track_winner_conversions?
68
- track_winner_conversions
69
+ !!track_winner_conversions
70
+ end
71
+
72
+ def skip_request_filter?
73
+ !!skip_request_filter
69
74
  end
70
75
 
71
76
  def name
@@ -126,48 +131,63 @@ module TrailGuide
126
131
  !!preview_url
127
132
  end
128
133
 
134
+ def callbacks
135
+ to_h.slice(*CALLBACK_KEYS).map { |k,v| [k, [v].flatten.compact] }.to_h
136
+ end
137
+
129
138
  def on_choose(meth=nil, &block)
130
- callbacks[:on_choose] << (meth || block)
139
+ self[:on_choose] ||= []
140
+ self[:on_choose] << (meth || block)
131
141
  end
132
142
 
133
143
  def on_use(meth=nil, &block)
134
- callbacks[:on_use] << (meth || block)
144
+ self[:on_use] ||= []
145
+ self[:on_use] << (meth || block)
135
146
  end
136
147
 
137
148
  def on_convert(meth=nil, &block)
138
- callbacks[:on_convert] << (meth || block)
149
+ self[:on_convert] ||= []
150
+ self[:on_convert] << (meth || block)
139
151
  end
140
152
 
141
153
  def on_start(meth=nil, &block)
142
- callbacks[:on_start] << (meth || block)
154
+ self[:on_start] ||= []
155
+ self[:on_start] << (meth || block)
143
156
  end
144
157
 
145
158
  def on_stop(meth=nil, &block)
146
- callbacks[:on_stop] << (meth || block)
159
+ self[:on_stop] ||= []
160
+ self[:on_stop] << (meth || block)
147
161
  end
148
162
 
149
163
  def on_resume(meth=nil, &block)
150
- callbacks[:on_resume] << (meth || block)
164
+ self[:on_resume] ||= []
165
+ self[:on_resume] << (meth || block)
151
166
  end
152
167
 
153
168
  def on_winner(meth=nil, &block)
154
- callbacks[:on_winner] << (meth || block)
169
+ self[:on_winner] ||= []
170
+ self[:on_winner] << (meth || block)
155
171
  end
156
172
 
157
173
  def on_reset(meth=nil, &block)
158
- callbacks[:on_reset] << (meth || block)
174
+ self[:on_reset] ||= []
175
+ self[:on_reset] << (meth || block)
159
176
  end
160
177
 
161
178
  def on_delete(meth=nil, &block)
162
- callbacks[:on_delete] << (meth || block)
179
+ self[:on_delete] ||= []
180
+ self[:on_delete] << (meth || block)
163
181
  end
164
182
 
165
183
  def on_redis_failover(meth=nil, &block)
166
- callbacks[:on_redis_failover] << (meth || block)
184
+ self[:on_redis_failover] ||= []
185
+ self[:on_redis_failover] << (meth || block)
167
186
  end
168
187
 
169
- def return_winner(meth=nil, &block)
170
- callbacks[:return_winner] << (meth || block)
188
+ def rollout_winner(meth=nil, &block)
189
+ self[:rollout_winner] ||= []
190
+ self[:rollout_winner] << (meth || block)
171
191
  end
172
192
  end
173
193
  end
@@ -142,6 +142,7 @@ module TrailGuide
142
142
  end
143
143
 
144
144
  def exclude_visitor?
145
+ return false if experiment.configuration.skip_request_filter?
145
146
  instance_exec(context, &TrailGuide.configuration.request_filter)
146
147
  end
147
148
 
@@ -2,6 +2,10 @@ module TrailGuide
2
2
  class Variant
3
3
  attr_reader :experiment, :name, :metadata, :weight
4
4
 
5
+ def dup(experiment)
6
+ self.class.new(experiment, name, metadata: metadata, weight: weight, control: control?)
7
+ end
8
+
5
9
  def initialize(experiment, name, metadata: {}, weight: 1, control: false)
6
10
  @experiment = experiment
7
11
  @name = name.to_s.underscore.to_sym
@@ -2,7 +2,7 @@ module TrailGuide
2
2
  module Version
3
3
  MAJOR = 0
4
4
  MINOR = 1
5
- PATCH = 17
5
+ PATCH = 18
6
6
  VERSION = "#{MAJOR}.#{MINOR}.#{PATCH}"
7
7
 
8
8
  class << self
data/lib/trailguide.rb CHANGED
@@ -1,5 +1,6 @@
1
1
  require "canfig"
2
2
  require "redis"
3
+ require "trail_guide/config"
3
4
  require "trail_guide/errors"
4
5
  require "trail_guide/adapters"
5
6
  require "trail_guide/algorithms"
@@ -14,74 +15,9 @@ require "trail_guide/version"
14
15
 
15
16
  module TrailGuide
16
17
  include Canfig::Module
18
+ @@configuration = TrailGuide::Config.new
17
19
 
18
- configure do |config|
19
- config.redis = ENV['REDIS_URL']
20
- config.disabled = false
21
- config.start_manually = true
22
- config.reset_manually = true
23
- config.store_override = false
24
- config.track_override = false
25
- config.override_parameter = :experiment
26
- config.algorithm = :weighted
27
- config.adapter = :multi
28
- config.allow_multiple_experiments = true # false / :control
29
- config.track_winner_conversions = false
30
- config.allow_multiple_conversions = false
31
- config.allow_multiple_goals = false
32
-
33
- config.on_redis_failover = nil # -> (experiment, error) { ... }
34
- config.on_adapter_failover = nil # -> (adapter, error) { ... }
35
-
36
- config.on_experiment_choose = nil # -> (experiment, variant, metadata) { ... }
37
- config.on_experiment_use = nil # -> (experiment, variant, metadata) { ... }
38
- config.on_experiment_convert = nil # -> (experiment, variant, checkpoint, metadata) { ... }
39
-
40
- config.on_experiment_start = nil # -> (experiment) { ... }
41
- config.on_experiment_stop = nil # -> (experiment) { ... }
42
- config.on_experiment_resume = nil # -> (experiment) { ... }
43
- config.on_experiment_reset = nil # -> (experiment) { ... }
44
- config.on_experiment_delete = nil # -> (experiment) { ... }
45
- config.on_experiment_winner = nil # -> (experiment, winner) { ... }
46
-
47
- config.return_experiment_winner = nil # -> (experiment, winner) { ... return variant }
48
-
49
- config.filtered_user_agents = []
50
- config.filtered_ip_addresses = []
51
- config.request_filter = -> (context) do
52
- is_preview? ||
53
- is_filtered_user_agent? ||
54
- is_filtered_ip_address?
55
- end
56
-
57
- def filtered_user_agents
58
- @filtered_user_agents ||= begin
59
- uas = @state[:filtered_user_agents]
60
- uas = uas.call if uas.respond_to?(:call)
61
- uas
62
- end
63
- end
64
-
65
- def filtered_ip_addresses
66
- @filtered_ip_addresses ||= begin
67
- ips = @state[:filtered_ip_addresses]
68
- ips = ips.call if ips.respond_to?(:call)
69
- ips
70
- end
71
- end
72
- end
73
-
74
- def self.catalog
75
- TrailGuide::Catalog.catalog
76
- end
77
-
78
- def self.redis
79
- @redis ||= begin
80
- if ['Redis', 'Redis::Namespace'].include?(configuration.redis.class.name)
81
- configuration.redis
82
- else
83
- Redis.new(url: configuration.redis)
84
- end
85
- end
20
+ class << self
21
+ delegate :redis, to: :configuration
86
22
  end
87
23
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: trailguide
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.17
4
+ version: 0.1.18
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mark Rebec
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-03-08 00:00:00.000000000 Z
11
+ date: 2019-03-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rails
@@ -80,6 +80,20 @@ dependencies:
80
80
  - - "~>"
81
81
  - !ruby/object:Gem::Version
82
82
  version: 1.3.6
83
+ - !ruby/object:Gem::Dependency
84
+ name: redis-namespace
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
83
97
  - !ruby/object:Gem::Dependency
84
98
  name: rspec-rails
85
99
  requirement: !ruby/object:Gem::Requirement
@@ -119,6 +133,7 @@ files:
119
133
  - app/views/trail_guide/admin/experiments/_experiment.html.erb
120
134
  - app/views/trail_guide/admin/experiments/index.html.erb
121
135
  - config/initializers/assets.rb
136
+ - config/initializers/trailguide.rb
122
137
  - config/routes.rb
123
138
  - lib/trail_guide/adapters.rb
124
139
  - lib/trail_guide/adapters/participants.rb
@@ -137,6 +152,7 @@ files:
137
152
  - lib/trail_guide/algorithms/weighted.rb
138
153
  - lib/trail_guide/catalog.rb
139
154
  - lib/trail_guide/combined_experiment.rb
155
+ - lib/trail_guide/config.rb
140
156
  - lib/trail_guide/engine.rb
141
157
  - lib/trail_guide/errors.rb
142
158
  - lib/trail_guide/experiment.rb