lab_coat 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d66afab349e98ec1ce140293e4372772633d9d8565ff66543f502769274614fb
4
- data.tar.gz: 683b940631dc537a41049af5c49006f56c80b833d8644957c6cbb8c6b28cff2f
3
+ metadata.gz: d819fd05745d82590e7d0f8c2e05f415b3ec6bea1cdc5def244fcfee17b969c7
4
+ data.tar.gz: 27819f2af7099608845196b00ae89a0ba59e9bdb8224814585a659601a261e76
5
5
  SHA512:
6
- metadata.gz: 1c492775bf3f0b3c9418934764891a82331761cc507e41e7de7c9085481b2860cc6fe3b5c9d048e008b15519cad72a1226ec9476248f75365ec581d634e59317
7
- data.tar.gz: ead0b6e3125f239444654e492ed5414073ba9b56c33e365226b7218548c429b08599c5df4475dbf76eb9375add204126fa7f000dcd59eb1c69dce6cf65234a5b
6
+ metadata.gz: 711d1510f5d1f4d59a57830567988d74fad50bfe7c4e548cd76e67d3f46bb2dfbbff503914e6129f7023032f7986c2153a1359e600d1f7d2e6aa443d98f27c0d
7
+ data.tar.gz: 66ed5879e1e803045806464c6b708b1a355c0d997965e1a6ea43e42b789d4f2904b7c39223b60101c08a1857c35dba72003d9463102962da27d9f39b181c1636
data/CHANGELOG.md CHANGED
@@ -1,2 +1,5 @@
1
+ ## [0.1.1] - 2024-04-08
2
+ - add `#slug` method to `Observation`
3
+
1
4
  ## [0.1.0] - 2024-04-08
2
5
  - Initial release
data/README.md CHANGED
@@ -7,7 +7,7 @@ A simple experiment library to safely test new code paths.
7
7
  This library is heavily inspired by [Scientist](https://github.com/github/scientist), with some key differences:
8
8
  - `Experiments` are `classes`, not `modules` which means they are stateful by default.
9
9
  - There is no app wide default experiment that gets magically set.
10
- - The `Result` only supports one comparison at a time, i.e. only 1 `candidate` is allowed.
10
+ - The `Result` only supports one comparison at a time, i.e. only 1 `candidate` is allowed per run.
11
11
 
12
12
  ## Installation
13
13
 
@@ -23,7 +23,26 @@ If bundler is not being used to manage dependencies, install the gem by executin
23
23
 
24
24
  ### Create an `Experiment`
25
25
 
26
- To do some science, i.e. test out a new code path, start by defining an `Experiment`. An experiment is any class that inherits from `LabCoat::Experiment` and implements the required methods.
26
+ To do some science, i.e. test out a new code path, start by defining an `Experiment`. An experiment is any class that inherits from `LabCoat::Experiment` and implements the [required methods](#required-methods).
27
+
28
+ ```ruby
29
+ # your_experiment.rb
30
+ class YourExperiment < LabCoat::Experiment
31
+ def control
32
+ expensive_query.first
33
+ end
34
+
35
+ def candidate
36
+ refactored_version_of_the_query.first
37
+ end
38
+
39
+ def enabled?
40
+ true
41
+ end
42
+ end
43
+ ```
44
+
45
+ The base initializer for an `Experiment` requires a `name` argument; it's a good idea to name your experiments.
27
46
 
28
47
  #### Required methods
29
48
 
@@ -31,11 +50,14 @@ See the [`Experiment`](lib/lab_coat/experiment.rb) class for more details.
31
50
 
32
51
  |Method|Description|
33
52
  |---|---|
34
- |`enabled?`|Returns a `Boolean` that controls whether or not the experiment runs.|
35
- |`control`|The existing or default behavior. This will always be returned from `#run!`.|
36
53
  |`candidate`|The new behavior you want to test.|
54
+ |`control`|The existing or default behavior. This will always be returned from `#run!`.|
55
+ |`enabled?`|Returns a `Boolean` that controls whether or not the experiment runs.|
37
56
  |`publish!`|This is not _technically_ required, but `Experiments` are not useful unless you can analyze the results. Override this method to record the `Result` however you wish.|
38
57
 
58
+ > [!TIP]
59
+ > The `#run!` method accepts arbitrary arguments and forwards them to `enabled?`, `control`, and `candidate` in case you need to provide data at runtime.
60
+
39
61
  #### Additional methods
40
62
 
41
63
  |Method|Description|
@@ -44,13 +66,8 @@ See the [`Experiment`](lib/lab_coat/experiment.rb) class for more details.
44
66
  |`ignore?`|Whether or not the result should be ignored. Ignored `Results` are still passed to `#publish!`|
45
67
  |`raised`|Callback method that's called when an `Observation` raises.|
46
68
 
47
- Consider creating a shared base class(es) to create consistency across experiments within your app.
48
-
49
- ```ruby
50
- # application_experiment.rb
51
- class ApplicationExperiment < LabCoat::Experiment
52
- end
53
- ```
69
+ > [!TIP]
70
+ > You should create a shared base class(es) to maintain consistency across experiments within your app.
54
71
 
55
72
  You may want to give your experiment some context, or state. You can do this via an initializer or writer methods just like any other Ruby class.
56
73
 
@@ -66,7 +83,7 @@ class ApplicationExperiment < LabCoat::Experiment
66
83
  end
67
84
  ```
68
85
 
69
- You likely want to `publish!` all experiments in a uniform way, so that you can analyze the data and make decisions.
86
+ You likely want to `publish!` all experiments in a consistent way, so that you can analyze the data and make decisions. New `Experiment` authors should not have to redo the "plumbing" between your experimentation framework (e.g. `LabCoat`) and your observability (o11y) process.
70
87
 
71
88
  ```ruby
72
89
  # application_experiment.rb
@@ -84,7 +101,7 @@ class ApplicationExperiment < LabCoat::Experiment
84
101
  end
85
102
  ```
86
103
 
87
- You might also have a common way to enable experiments such as a feature flag system and/or common guards you want to enforce application wide.
104
+ You might also have a common way to enable experiments such as a feature flag system and/or common guards you want to enforce application wide. These might come from a mix of services and the `Experiment`'s state.
88
105
 
89
106
  ```ruby
90
107
  # application_experiment.rb
@@ -97,17 +114,18 @@ end
97
114
 
98
115
  ### Make some `Observations` via `run!`
99
116
 
100
- You don't have to create an `Observation` yourself; that happens automatically when you call `Experiment#run!`.
117
+ You don't have to create an `Observation` yourself; that happens automatically when you call `Experiment#run!`. The control and candidate `Observations` are packaged into a `Result` and [passed to `Experiment#publish!`](#publish-the-result).
101
118
 
102
119
  |Attribute|Description|
103
120
  |---|---|
104
- |`name`|Either `"control"` or `"candidate"`.|
105
- |`experiment`|The `Experiment` instance this `Result` is for.|
106
121
  |`duration`|The duration of the run in `float` seconds.|
107
- |`value`|The return value of the observed code path.|
122
+ |`error`|If the code path raised, the thrown exception is stored here.|
123
+ |`experiment`|The `Experiment` instance this `Result` is for.|
124
+ |`name`|Either `"control"` or `"candidate"`.|
108
125
  |`publishable_value`|A publishable representation of the `value`, as defined by `Experiment#publishable_value`.|
109
126
  |`raised?`|Whether or not the code path raised.|
110
- |`error`|If the code path raised, the thrown exception is stored here.|
127
+ |`slug`|A combination of the `Experiment#name` and `Observation#name`, e.g. `"experiment_name.control"`|
128
+ |`value`|The return value of the observed code path.|
111
129
 
112
130
  `Observation` instances are passed to many of the `Experiment` methods that you may override.
113
131
 
@@ -151,11 +169,11 @@ A `Result` represents a single run of an `Experiment`.
151
169
 
152
170
  |Attribute|Description|
153
171
  |---|---|
154
- |`experiment`|The `Experiment` instance this `Result` is for.|
155
- |`control`|An `Observation` instance representing the `Experiment#control` behavior|
156
172
  |`candidate`|An `Observation` instance representing the `Experiment#candidate` behavior|
157
- |`matched?`|Whether or not the `control` and `candidate` match, as defined by `Experiment#compare`|
173
+ |`control`|An `Observation` instance representing the `Experiment#control` behavior|
174
+ |`experiment`|The `Experiment` instance this `Result` is for.|
158
175
  |`ignored?`|Whether or not the result should be ignored, as defined by `Experiment#ignore?`|
176
+ |`matched?`|Whether or not the `control` and `candidate` match, as defined by `Experiment#compare`|
159
177
 
160
178
  The `Result` is passed to your implementation of `#publish!` when an `Experiment` is finished running.
161
179
 
@@ -169,22 +187,65 @@ def publish!(result)
169
187
  if result.matched?
170
188
  puts "😎"
171
189
  else
190
+ control = result.control
191
+ candidate = result.candidate
172
192
  puts <<~MISMATCH
173
193
  😮
174
194
 
175
- [Control]
176
- Value: #{result.control.publishable_value}
177
- Duration: #{result.control.duration}
178
- Error: #{result.control.error&.message}
195
+ #{control.slug}
196
+ Value: #{control.publishable_value}
197
+ Duration: #{control.duration}
198
+ Error: #{control.error&.message}
179
199
 
180
- [Candidate]
181
- Value: #{result.candidate.publishable_value}
182
- Duration: #{result.candidate.duration}
183
- Error: #{result.candidate.error&.message}
200
+ #{candidate.slug}
201
+ Value: #{candidate.publishable_value}
202
+ Duration: #{candidate.duration}
203
+ Error: #{candidate.error&.message}
184
204
  MISMATCH
185
205
  end
186
206
  end
187
207
  ```
208
+ Running a mismatched experiment with this implementation of `publish!` would produce:
209
+
210
+ ```
211
+ 😮
212
+
213
+ my_experiment.control
214
+ Value: 420
215
+ Duration: 12.934
216
+ Error:
217
+
218
+ my_experiment.candidate
219
+ Value: 69
220
+ Duration: 9.702
221
+ Error:
222
+ ```
223
+
224
+ ### Standalone `Observations`
225
+
226
+ The `Observation` class can be used as a standalone wrapper for any code that you want to experiment with. Instantiating an `Observation` automatically:
227
+ - measures the duration of the code block
228
+ - captures the return value of the code block
229
+ - rescues and stores any errors raised by the code block
230
+
231
+ ```ruby
232
+ 10.times do |i|
233
+ observation = Observation.new("test-#{i}", nil) do
234
+ some_code_path
235
+ end
236
+
237
+ puts "#{observation.name} results:"
238
+ if observation.raised?
239
+ puts "error: #{observation.error.message}"
240
+ else
241
+ puts "duration: #{observation.duration}"
242
+ puts "succeeded: #{!observation.raised?}"
243
+ end
244
+ end
245
+ ```
246
+
247
+ > [!WARNING]
248
+ > Be careful when using `Observation` instances without an `Experiment` set. Some methods like `#publishable_value` and `#slug` depend on an `experiment` and may raise an error when called.
188
249
 
189
250
  ## Development
190
251
 
@@ -194,7 +255,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
194
255
 
195
256
  ## Contributing
196
257
 
197
- Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/lab_coat.
258
+ Bug reports and pull requests are welcome on GitHub at https://github.com/omkarmoghe/lab_coat.
198
259
 
199
260
  ## License
200
261
 
@@ -27,5 +27,10 @@ module LabCoat
27
27
  def raised?
28
28
  !error.nil?
29
29
  end
30
+
31
+ # @return [String] String representing this Observation.
32
+ def slug
33
+ "#{experiment.name}.#{name}"
34
+ end
30
35
  end
31
36
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module LabCoat
4
- VERSION = "0.1.0"
4
+ VERSION = "0.1.1"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: lab_coat
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Omkar Moghe
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2024-04-08 00:00:00.000000000 Z
11
+ date: 2024-04-09 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: minitest