lab_coat 0.1.3 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 35fcc39652cc2ab43879a6a513214962c139d6b5751f5a1d4b4e8dba7b72b5e3
-  data.tar.gz: 5b54b1fb3fcabe26a80b2cdd619308f51b7fc44c6b0b93572550eb3e9a50aedd
+  metadata.gz: 0c1c5580cec0ec677137fcbf23a90291fab80564c955a1d8b10fb48903f1f43b
+  data.tar.gz: 15540d166b6a0d6098c07979efc8b780ecebb26d5059ac3db1217489e2448e06
 SHA512:
-  metadata.gz: 48e1e6af3c47d799f6eecd76dcac57a168da9c1729fc29091742b79ae114d36741a9d72bac6348db0db43a868ee5f75087ced725d270ae55074221a2dc152cd5
-  data.tar.gz: 6b98f3702d8a8cc8494bc2c9eb5e09e1ecc6aff21d204be93d54485eba4754fa60aa89aa832b2644910e42d6ac42c4cdc683db0ac4d0b49b7c3fe2025b417488
+  metadata.gz: c84bde55abd4974f068a802f181552111d847c3eeca07e28046fe3492919ba25a12fdefb10ac547bc22fbce90c1785fa01d73084db769f51ebe55993d52b2cc9
+  data.tar.gz: 128f1a14f2a2ec4f370d40d5c5b147d35b26abd75f558d5f19755fb71e82cfe8aad4de8673835a82c9b12f21088642ea346aa3bba4d29cdcc7d334a15b4622db

data/CHANGELOG.md

@@ -1,3 +1,7 @@
+## [0.1.4] - Unreleased
+- Remove the arity check, it's not very intuitive
+- Adds a `@context` that gets set at runtime and reset after each run. This is a much simpler way for methods to access a shared runtime context that can be set per `run!`.
+
 ## [0.1.3] - 2024-04-17
 - `Experiment` now enforces arity at runtime for the `#enabled?`, `control`, and `candidate` methods.
 

data/README.md

@@ -57,7 +57,7 @@ See the [`Experiment`](lib/lab_coat/experiment.rb) class for more details.
 |`publish!`|This is not _technically_ required, but `Experiments` are not useful unless you can analyze the results. Override this method to record the `Result` however you wish.|
 
 > [!IMPORTANT]
-> The `#run!` method accepts arbitrary arguments and forwards them to `enabled?`, `control`, and `candidate` in case you need to provide data at runtime. This means the [arity](https://en.wikipedia.org/wiki/Arity) of the three methods needs to be the same. This is enforced by `LabCoat` at runtime.
+> The `#run!` method accepts arbitrary key word arguments and stores them in an instance variable called `@context` in case you need to provide data at runtime. You can access the runtime context via `@context` or `context`. The runtime context **is reset** after each run.
 
 #### Additional methods
 
@@ -88,13 +88,16 @@ You might want to `publish!` all experiments in a consistent way so that you can
 # application_experiment.rb
 class ApplicationExperiment < LabCoat::Experiment
   def publish!(result)
-    payload = result.to_h.merge(user_id: @user.id)
+    payload = result.to_h.merge(
+      user_id: @user.id, # e.g. something from the `Experiment` state
+      build_number: context.version # e.g. something from the runtime context
+    )
     YourO11yService.track_experiment_result(payload)
   end
 end
 ```
 
-You might have a common way to enable experiments such as a feature flag system and/or common guards you want to enforce application wide. These might come from a mix of services and the `Experiment`'s state.
+You might have a common way to enable experiments such as a feature flag system and/or common guards you want to enforce application wide. These might come from a mix of services, the `Experiment`'s state, or the runtime `context`.
 
 ```ruby
 # application_experiment.rb
@@ -123,6 +126,8 @@ end
 
 You don't have to create an `Observation` yourself; that happens automatically when you call `Experiment#run!`. The control and candidate `Observations` are packaged into a `Result` and [passed to `Experiment#publish!`](#publish-the-result).
 
+The `run!` method accepts arbitrary keyword arguments, to allow you to set runtime context for the specific run of the experiment. You can access this `Hash` via the `context` reader method, or directly via the `@context` instance variable.
+
 |Attribute|Description|
 |---|---|
 |`duration`|The duration of the run represented as a `Benchmark::Tms` object.|
@@ -180,14 +185,14 @@ A `Result` represents a single run of an `Experiment`.
 |`matched?`|Whether or not the `control` and `candidate` match, as defined by `Experiment#compare`|
 |`to_h`|A hash representation of the `Result`. Useful for publishing and/or reporting.|
 
-The `Result` is passed to your implementation of `#publish!` when an `Experiment` is finished running. The `to_h` method on a Result is a good place to start and might be sufficient for most experiments.
+The `Result` is passed to your implementation of `#publish!` when an `Experiment` is finished running. The `to_h` method on a Result is a good place to start and might be sufficient for most experiments. You might want to `merge` additional data such as the runtime `context` or other state if you find that relevant for analysis.
 
 ```ruby
 # your_experiment.rb
 def publish!(result)
   return if result.ignored?
 
-  puts result.to_h
+  puts result.to_h.merge(run_context: context)
 end
 ```
 
@@ -276,7 +281,7 @@ end
 ```
 
 > [!WARNING]
-> Be careful when using `Observation` instances without an `Experiment` set. Some methods like `#publishable_value` and `#slug` depend on an `experiment` and may raise an error when called.
+> Be careful when using `Observation` instances without an `Experiment` set. Some methods like `#publishable_value` and `#slug` depend on an `experiment` and may raise an error or return unexpected values when called without one.
 
 ## Development
 

data/lib/lab_coat/experiment.rb

@@ -3,10 +3,11 @@
 module LabCoat
   # A base experiment class meant to be subclassed to define various experiments.
   class Experiment
-    attr_reader :name
+    attr_reader :name, :context
 
     def initialize(name)
       @name = name
+      @context = {}
     end
 
     # Override this method to control whether or not the experiment runs.
@@ -18,13 +19,13 @@ module LabCoat
     # Override this method to define the existing aka "control" behavior. This method is always run, even when
     # `enabled?` is false.
     # @return [Object] Anything.
-    def control(...)
+    def control
       raise InvalidExperimentError, "`#control` must be implemented in your Experiment class."
     end
 
     # Override this method to define the new aka "candidate" behavior. Only run if the experiment is enabled.
     # @return [Object] Anything.
-    def candidate(...)
+    def candidate
       raise InvalidExperimentError, "`#candidate` must be implemented in your Experiment class."
     end
 
@@ -59,25 +60,25 @@ module LabCoat
 
     # Runs the control and candidate and publishes the result. Always returns the result of `control`.
     # @param context [Hash] Any data needed at runtime.
-    def run!(...) # rubocop:disable Metrics/MethodLength
-      enforce_arity!
+    def run!(**context)
+      # Set the context for this run.
+      @context = context
 
       # Run the control and exit early if the experiment is not enabled.
-      control_obs = Observation.new("control", self) do
-        control(...)
-      end
+      control_obs = Observation.new("control", self) { control }
       raised(control_obs) if control_obs.raised?
-      return control_obs.value unless enabled?(...)
+      return control_obs.value unless enabled?
 
-      candidate_obs = Observation.new("candidate", self) do
-        candidate(...)
-      end
+      candidate_obs = Observation.new("candidate", self) { candidate }
       raised(candidate_obs) if candidate_obs.raised?
 
       # Compare and publish the results.
       result = Result.new(self, control_obs, candidate_obs)
       publish!(result)
 
+      # Reset the context for this run.
+      @context = {}
+
       # Always return the control.
       control_obs.value
     end

data/lib/lab_coat/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LabCoat
-  VERSION = "0.1.3"
+  VERSION = "0.1.4"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: lab_coat
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.1.4
 platform: ruby
 authors:
 - Omkar Moghe
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2024-04-18 00:00:00.000000000 Z
+date: 2024-04-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest