gitlab-experiment 1.6.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +16 -30
- data/lib/generators/gitlab/experiment/install/templates/initializer.rb.tt +0 -14
- data/lib/gitlab/experiment/configuration.rb +0 -29
- data/lib/gitlab/experiment/errors.rb +0 -1
- data/lib/gitlab/experiment/rollout/random.rb +6 -11
- data/lib/gitlab/experiment/rollout.rb +3 -21
- data/lib/gitlab/experiment/version.rb +1 -1
- data/lib/gitlab/experiment.rb +0 -7
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b69f6d4aa4d4bbba5b9ef15f5a1d72c84726df5a6ec648cd09761cdc9c997d5b
|
|
4
|
+
data.tar.gz: ea6f508020bf5941ee7e74647c43052aedf55fa969d8c7dea9d42f86e9b03c8b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 62690c224599c2c0cbb78a42bdffee6104d5c811faa43027e92a2fd219006e4d0dda7664ee22bdfbb850ad0aa171fa23a82b944a0e667f3b680f2355119c2ae8
|
|
7
|
+
data.tar.gz: 9b7cfe7e5fcdfce2cfd2826da2974b64c5d61d4989fc87725483e45a9f2547840847ee0f7847d5ebd2e4bcfda13dfe78873404d8c6efd5b60e962b8d6413f56a
|
data/README.md
CHANGED
|
@@ -685,11 +685,10 @@ end
|
|
|
685
685
|
- ✅ Experiment reach stays controlled
|
|
686
686
|
- ✅ Perfect for multi-page experimental experiences
|
|
687
687
|
|
|
688
|
-
> **Note:** `only_assigned: true` relies on cached assignments.
|
|
689
|
-
>
|
|
690
|
-
>
|
|
691
|
-
>
|
|
692
|
-
> excluded from tracking there.
|
|
688
|
+
> **Note:** `only_assigned: true` relies on cached assignments. All rollout
|
|
689
|
+
> strategies cache control variants (see
|
|
690
|
+
> [Advanced: Caching Configuration](#advanced-caching-configuration)) so that bucketed
|
|
691
|
+
> control users are included in `only_assigned` tracking.
|
|
693
692
|
|
|
694
693
|
**Real-world use cases:**
|
|
695
694
|
- **Post-signup experiences**: Assign at signup, show features throughout the app
|
|
@@ -704,7 +703,7 @@ needs.
|
|
|
704
703
|
|
|
705
704
|
**Built-in strategies:**
|
|
706
705
|
- [`Percent`](lib/gitlab/experiment/rollout/percent.rb) - Consistent percentage-based assignment (default, recommended)
|
|
707
|
-
- [`Random`](lib/gitlab/experiment/rollout/random.rb) -
|
|
706
|
+
- [`Random`](lib/gitlab/experiment/rollout/random.rb) - Random assignment on first roll (useful for load testing)
|
|
708
707
|
- [`RoundRobin`](lib/gitlab/experiment/rollout/round_robin.rb) - Cycle through variants (requires caching)
|
|
709
708
|
- [`Base`](lib/gitlab/experiment/rollout.rb) - Useful for building custom rollout strategies
|
|
710
709
|
|
|
@@ -885,13 +884,13 @@ graph TD
|
|
|
885
884
|
Excluded? -->|No| Segmented?
|
|
886
885
|
Segmented? -->|Yes / Cached| VariantA
|
|
887
886
|
Segmented? -->|No| Rollout[Rollout Resolve]
|
|
888
|
-
Rollout
|
|
887
|
+
Rollout -->|Cached| Control
|
|
889
888
|
Rollout -->|Cached| VariantA
|
|
890
889
|
Rollout -->|Cached| VariantB
|
|
891
890
|
Rollout -->|Cached| VariantN
|
|
892
891
|
|
|
893
|
-
class ForcedVariant,VariantA,VariantB,VariantN included
|
|
894
|
-
class
|
|
892
|
+
class ForcedVariant,VariantA,VariantB,VariantN,Control included
|
|
893
|
+
class Excluded excluded
|
|
895
894
|
class Cached cached
|
|
896
895
|
```
|
|
897
896
|
|
|
@@ -901,6 +900,7 @@ class Cached cached
|
|
|
901
900
|
3. Cache provides consistency across calls
|
|
902
901
|
4. Segmentation takes priority over rollout
|
|
903
902
|
5. `only_assigned: true` exits early if no cache hit
|
|
903
|
+
6. Control is cached like any other variant, so the assignment stays stable across calls for every rollout strategy
|
|
904
904
|
|
|
905
905
|
### Experiment Context and Stickiness
|
|
906
906
|
|
|
@@ -1326,31 +1326,17 @@ documented in its implementation.
|
|
|
1326
1326
|
|
|
1327
1327
|
**Control variant caching:**
|
|
1328
1328
|
|
|
1329
|
-
|
|
1329
|
+
The gem always caches `:control` assignments alongside variant assignments. This is what
|
|
1330
1330
|
makes `only_assigned: true` lookups include bucketed control users -- without it, control
|
|
1331
1331
|
users would appear unassigned at secondary tracking sites and would be silently excluded.
|
|
1332
1332
|
|
|
1333
|
-
|
|
1334
|
-
# Disable globally (don't cache :control)
|
|
1335
|
-
Gitlab::Experiment.configure do |config|
|
|
1336
|
-
config.cache_control = false
|
|
1337
|
-
end
|
|
1338
|
-
|
|
1339
|
-
# Or override per-rollout
|
|
1340
|
-
class MyExperiment < ApplicationExperiment
|
|
1341
|
-
default_rollout :percent, cache_control: false
|
|
1342
|
-
end
|
|
1343
|
-
```
|
|
1344
|
-
|
|
1345
|
-
The `Random` rollout strategy intentionally opts out of caching `:control` regardless of this
|
|
1346
|
-
setting, because its convergence model depends on re-rolling control assignments on every
|
|
1347
|
-
visit until a non-control variant is picked.
|
|
1333
|
+
This applies to every rollout strategy.
|
|
1348
1334
|
|
|
1349
|
-
**Operational note:**
|
|
1350
|
-
|
|
1351
|
-
|
|
1352
|
-
|
|
1353
|
-
`
|
|
1335
|
+
**Operational note:** Because control is cached, cache hashes grow with every bucketed
|
|
1336
|
+
user, not just users assigned to a candidate variant. For long-running experiments the
|
|
1337
|
+
cache will be roughly `1 / candidate_percentage` the size it would have been before. Plan
|
|
1338
|
+
periodic cleanup using your store's deletion API -- for the `RedisHashStore`, that's
|
|
1339
|
+
`experiment.cache.store.clear(key: experiment.name)`.
|
|
1354
1340
|
|
|
1355
1341
|
### Advanced: Custom Rollout Strategies
|
|
1356
1342
|
|
|
@@ -22,20 +22,6 @@ Gitlab::Experiment.configure do |config|
|
|
|
22
22
|
# Use `nil` for no caching.
|
|
23
23
|
config.cache = nil
|
|
24
24
|
|
|
25
|
-
# Cache `:control` assignments alongside variant assignments.
|
|
26
|
-
#
|
|
27
|
-
# When set to true (the default), bucketed control users are cached and
|
|
28
|
-
# therefore visible to `only_assigned: true` tracking calls. Set to false
|
|
29
|
-
# to skip caching control -- control users will be treated as unassigned
|
|
30
|
-
# at `only_assigned` sites and excluded from tracking there.
|
|
31
|
-
#
|
|
32
|
-
# Can be overridden per-rollout:
|
|
33
|
-
#
|
|
34
|
-
# class ExampleExperiment < ApplicationExperiment
|
|
35
|
-
# default_rollout :percent, cache_control: false
|
|
36
|
-
# end
|
|
37
|
-
config.cache_control = true
|
|
38
|
-
|
|
39
25
|
# The domain to use on cookies.
|
|
40
26
|
#
|
|
41
27
|
# When not set, it uses the current host. If you want to provide specific
|
|
@@ -30,21 +30,6 @@ module Gitlab
|
|
|
30
30
|
# Use `nil` for no caching.
|
|
31
31
|
@cache = nil
|
|
32
32
|
|
|
33
|
-
# Whether `:control` assignments should be written to the cache.
|
|
34
|
-
#
|
|
35
|
-
# When true (default), control variants are cached alongside other
|
|
36
|
-
# variants, which is required for `only_assigned: true` tracking to
|
|
37
|
-
# work correctly for the control arm of an experiment.
|
|
38
|
-
#
|
|
39
|
-
# When false, control variants are not cached. This preserves the
|
|
40
|
-
# behavior of versions prior to this option being introduced. Any
|
|
41
|
-
# experiment using `only_assigned: true` will silently exclude
|
|
42
|
-
# control users from tracking.
|
|
43
|
-
#
|
|
44
|
-
# Individual rollouts can override this via the `cache_control:`
|
|
45
|
-
# option, e.g. `default_rollout :percent, cache_control: false`.
|
|
46
|
-
@cache_control = true
|
|
47
|
-
|
|
48
33
|
# The domain to use on cookies.
|
|
49
34
|
#
|
|
50
35
|
# When not set, it uses the current host. If you want to provide specific
|
|
@@ -208,7 +193,6 @@ module Gitlab
|
|
|
208
193
|
:base_class,
|
|
209
194
|
:strict_registration,
|
|
210
195
|
:cache,
|
|
211
|
-
:cache_control,
|
|
212
196
|
:cookie_domain,
|
|
213
197
|
:cookie_name,
|
|
214
198
|
:secure_cookie,
|
|
@@ -229,19 +213,6 @@ module Gitlab
|
|
|
229
213
|
@default_rollout = Rollout.resolve(*args)
|
|
230
214
|
end
|
|
231
215
|
|
|
232
|
-
def cache_control=(value) # rubocop:disable Lint/DuplicateMethods
|
|
233
|
-
if value != true
|
|
234
|
-
deprecated(
|
|
235
|
-
:cache_control,
|
|
236
|
-
"setting `cache_control` to a non-true value is deprecated and will be removed in version 2.0. " \
|
|
237
|
-
"Control variants will always be cached.",
|
|
238
|
-
version: '1.4'
|
|
239
|
-
)
|
|
240
|
-
end
|
|
241
|
-
|
|
242
|
-
@cache_control = value
|
|
243
|
-
end
|
|
244
|
-
|
|
245
216
|
# Internal warning helpers.
|
|
246
217
|
|
|
247
218
|
def deprecated(*args, version:, stack: 0)
|
|
@@ -7,7 +7,6 @@ module Gitlab
|
|
|
7
7
|
UnregisteredExperiment = Class.new(Error)
|
|
8
8
|
ExistingBehaviorError = Class.new(Error)
|
|
9
9
|
BehaviorMissingError = Class.new(Error)
|
|
10
|
-
ControlNotCachedError = Class.new(Error)
|
|
11
10
|
|
|
12
11
|
class NestingError < Error
|
|
13
12
|
def initialize(experiment:, nested_experiment:)
|
|
@@ -1,14 +1,12 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
# The random rollout strategy
|
|
4
|
-
# group.
|
|
3
|
+
# The random rollout strategy picks a variant uniformly at random the first time a context is assigned into the
|
|
4
|
+
# experiment group.
|
|
5
5
|
#
|
|
6
|
-
# If caching is enabled
|
|
7
|
-
#
|
|
8
|
-
#
|
|
9
|
-
# assignment is cached and stable. Without caching, assignment is random on every call.
|
|
6
|
+
# If caching is enabled, that first roll is persisted and the context gets the same variant on every subsequent visit,
|
|
7
|
+
# so the assignment is stable/sticky per context -- including :control, which is cached like any other variant. Without
|
|
8
|
+
# caching, assignment is random on every call.
|
|
10
9
|
#
|
|
11
|
-
# This is the only rollout that opts out of caching control: every other rollout caches whatever variant it resolves to.
|
|
12
10
|
# Example configuration usage:
|
|
13
11
|
#
|
|
14
12
|
# config.default_rollout = Gitlab::Experiment::Rollout::Random.new
|
|
@@ -31,10 +29,7 @@ module Gitlab
|
|
|
31
29
|
protected
|
|
32
30
|
|
|
33
31
|
def execute_assignment
|
|
34
|
-
|
|
35
|
-
# Drop :control so it's never cached, even if cache_control is true.
|
|
36
|
-
# The context will be re-rolled on the next visit and eventually converge on a non-control variant.
|
|
37
|
-
result == :control ? nil : result
|
|
32
|
+
behavior_names.sample # pick a random variant; cached on first roll so it stays stable
|
|
38
33
|
end
|
|
39
34
|
end
|
|
40
35
|
end
|
|
@@ -31,15 +31,6 @@ module Gitlab
|
|
|
31
31
|
|
|
32
32
|
@experiment = experiment
|
|
33
33
|
@options = options
|
|
34
|
-
|
|
35
|
-
return if !options.key?(:cache_control) || options[:cache_control] == true
|
|
36
|
-
|
|
37
|
-
Configuration.deprecated(
|
|
38
|
-
:cache_control,
|
|
39
|
-
"setting `cache_control` to a non-true value on a rollout is deprecated and " \
|
|
40
|
-
"will be removed in version 2.0. Control variants will always be cached.",
|
|
41
|
-
version: '1.4'
|
|
42
|
-
)
|
|
43
34
|
end
|
|
44
35
|
|
|
45
36
|
def enabled?
|
|
@@ -49,18 +40,9 @@ module Gitlab
|
|
|
49
40
|
def resolve
|
|
50
41
|
validate! # allow the rollout strategy to validate itself
|
|
51
42
|
|
|
52
|
-
|
|
53
|
-
#
|
|
54
|
-
|
|
55
|
-
return assignment if cache_control_enabled?
|
|
56
|
-
|
|
57
|
-
assignment == :control ? nil : assignment
|
|
58
|
-
end
|
|
59
|
-
|
|
60
|
-
# Returns true when this rollout caches `:control` assignments. Per-rollout
|
|
61
|
-
# `cache_control:` option takes precedence over the global default.
|
|
62
|
-
def cache_control_enabled?
|
|
63
|
-
options.fetch(:cache_control, Configuration.cache_control)
|
|
43
|
+
# Control variants are cached alongside other variants, which is required
|
|
44
|
+
# for `only_assigned: true` tracking to work for the control arm.
|
|
45
|
+
execute_assignment
|
|
64
46
|
end
|
|
65
47
|
|
|
66
48
|
private
|
data/lib/gitlab/experiment.rb
CHANGED
|
@@ -168,13 +168,6 @@ module Gitlab
|
|
|
168
168
|
def only_assigned?
|
|
169
169
|
return false unless context.only_assigned
|
|
170
170
|
|
|
171
|
-
unless rollout.cache_control_enabled?
|
|
172
|
-
raise ControlNotCachedError,
|
|
173
|
-
"`only_assigned: true` requires `cache_control: true`, but the rollout for " \
|
|
174
|
-
"experiment `#{name}` has it disabled. Control users would be silently excluded " \
|
|
175
|
-
"from tracking. Enable cache_control or remove `only_assigned: true`."
|
|
176
|
-
end
|
|
177
|
-
|
|
178
171
|
find_variant.blank?
|
|
179
172
|
end
|
|
180
173
|
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: gitlab-experiment
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version:
|
|
4
|
+
version: 2.0.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- GitLab
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-07-22 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activesupport
|