dat-analysis 1.2.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +4 -0
- data/Gemfile +2 -0
- data/LICENSE.txt +22 -0
- data/README.md +423 -0
- data/dat-analysis.gemspec +17 -0
- data/lib/dat/analysis.rb +446 -0
- data/lib/dat/analysis/library.rb +30 -0
- data/lib/dat/analysis/matcher.rb +43 -0
- data/lib/dat/analysis/registry.rb +50 -0
- data/lib/dat/analysis/result.rb +78 -0
- data/lib/dat/analysis/tally.rb +59 -0
- data/script/bootstrap +9 -0
- data/script/release +38 -0
- data/script/test +9 -0
- data/test/dat_analysis_subclassing_test.rb +119 -0
- data/test/dat_analysis_test.rb +822 -0
- data/test/fixtures/analysis/test-suite-experiment/matcher.rb +7 -0
- data/test/fixtures/experiment-with-classes/matcher_a.rb +5 -0
- data/test/fixtures/experiment-with-classes/matcher_b.rb +11 -0
- data/test/fixtures/experiment-with-classes/wrapper_a.rb +5 -0
- data/test/fixtures/experiment-with-classes/wrapper_b.rb +11 -0
- data/test/fixtures/experiment-with-good-and-extraneous-classes/matcher_w.rb +5 -0
- data/test/fixtures/experiment-with-good-and-extraneous-classes/matcher_y.rb +11 -0
- data/test/fixtures/experiment-with-good-and-extraneous-classes/matcher_z.rb +11 -0
- data/test/fixtures/experiment-with-good-and-extraneous-classes/wrapper_w.rb +5 -0
- data/test/fixtures/experiment-with-good-and-extraneous-classes/wrapper_y.rb +11 -0
- data/test/fixtures/experiment-with-good-and-extraneous-classes/wrapper_z.rb +11 -0
- data/test/fixtures/initialize-classes/matcher_m.rb +5 -0
- data/test/fixtures/initialize-classes/matcher_n.rb +11 -0
- data/test/fixtures/initialize-classes/wrapper_m.rb +5 -0
- data/test/fixtures/initialize-classes/wrapper_n.rb +11 -0
- data/test/fixtures/invalid-matcher/matcher.rb +1 -0
- metadata +128 -0
data/.gitignore
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2013 GitHub, Inc.
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
19
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
20
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
21
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
22
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,423 @@
|
|
1
|
+
# Dat-analysis
|
2
|
+
|
3
|
+
A Ruby library for analyzing the results of [dat-science][dsc] experiments. For
|
4
|
+
the motivation behind this library, and documentation on setting up experiments,
|
5
|
+
go check out [dat-science][dsc]'s documentation.
|
6
|
+
|
7
|
+
[dsc]: https://github.com/github/dat-science/
|
8
|
+
|
9
|
+
## What do I do with all these experiment results?
|
10
|
+
|
11
|
+
Once you've started a `dat-science` experiment and published some results,
|
12
|
+
you'll want to analyze the mismatches from your experiment. In `dat-analysis`
|
13
|
+
you'll find an analysis toolkit to help understand experiment results.
|
14
|
+
|
15
|
+
We designed the analysis tools to be run from your ruby console (`irb` or
|
16
|
+
`script/console` if you're doing science on a Rails app). You create an analyzer
|
17
|
+
and then interactively fetch experiment results and study them to determine the
|
18
|
+
reason the control method's results differ from the candidate method's results.
|
19
|
+
|
20
|
+
### Your very own analyzer
|
21
|
+
|
22
|
+
The `Dat::Analysis` base class provides a number of tools for analysis. Since
|
23
|
+
the process of retrieving your experiment results depends on how you used
|
24
|
+
`publish` in your experiment, you'll need to create a subclass of `Dat::Analysis`
|
25
|
+
which implements methods to handle reading and processing results.
|
26
|
+
|
27
|
+
You will need to define `read` and `count` to return the next published experiment
|
28
|
+
result, and the count of remaining published experiment results, respectively.
|
29
|
+
You can optionally define `cook` to do any decoding, un-marshalling, or whatever
|
30
|
+
other pre-processing you desire on the raw experiment result returned by `read`.
|
31
|
+
|
32
|
+
|
33
|
+
``` ruby
|
34
|
+
require 'dat/analysis'
|
35
|
+
|
36
|
+
module MyApp
|
37
|
+
# Public: Perform dat analysis on a dat-science experiment.
|
38
|
+
#
|
39
|
+
# This is a subclass of Dat::Analysis which provides the concrete implementation
|
40
|
+
# of the `#read`, `#count`, and `#cook` methods to interact with our Redis data
|
41
|
+
# store, and decodes our science mismatch results from JSON.
|
42
|
+
class Analysis < Dat::Analysis
|
43
|
+
# Public: Read the next available science mismatch result.
|
44
|
+
#
|
45
|
+
# Returns the next raw science mismatch result from Redis.
|
46
|
+
def read
|
47
|
+
Redis.rpop "dat-science.#{experiment_name}.results"
|
48
|
+
end
|
49
|
+
|
50
|
+
# Public: Get the number of pending science mismatch results.
|
51
|
+
#
|
52
|
+
# Returns the number of pending science mismatch results from redis.
|
53
|
+
def count
|
54
|
+
Redis.llen "dat-science.#{experiment_name}.results"
|
55
|
+
end
|
56
|
+
|
57
|
+
# Public: "Cook" a raw science mismatch result.
|
58
|
+
#
|
59
|
+
# raw_result - a raw science mismatch result
|
60
|
+
#
|
61
|
+
# Returns nil if raw_result is nil.
|
62
|
+
# Returns the JSON-parsed raw_result.
|
63
|
+
def cook(raw_result)
|
64
|
+
return nil unless raw_result
|
65
|
+
JSON.parse(raw_result)
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
69
|
+
|
70
|
+
```
|
71
|
+
|
72
|
+
#### Instantiating the analyzer
|
73
|
+
|
74
|
+
This analyzer can be used with many experiments, so you'll need to instantiate an
|
75
|
+
analyzer instance for your current experiment:
|
76
|
+
|
77
|
+
``` ruby
|
78
|
+
irb> a = MyApp::Analysis.new('widget-permissions')
|
79
|
+
=> #<MyApp::Analysis:0x007fae4a0101f8 ...>
|
80
|
+
```
|
81
|
+
|
82
|
+
### Working with individual results
|
83
|
+
|
84
|
+
First, let's look at how you can work with single experiment mismatch results.
|
85
|
+
The `#result` method (also available as `#current`) will show you the most
|
86
|
+
recently fetched experiment result. Before you've fetched any results, this
|
87
|
+
will be empty:
|
88
|
+
|
89
|
+
``` ruby
|
90
|
+
irb> a.result
|
91
|
+
=> nil
|
92
|
+
irb> a.current
|
93
|
+
=> nil
|
94
|
+
```
|
95
|
+
|
96
|
+
We can use the `#more?` predicate method to see if there are experiment results
|
97
|
+
pending, and `#count` to see just how many results are available:
|
98
|
+
|
99
|
+
``` ruby
|
100
|
+
irb> a.more?
|
101
|
+
=> true
|
102
|
+
irb> a.count
|
103
|
+
=> 103
|
104
|
+
```
|
105
|
+
|
106
|
+
Let's fetch a result:
|
107
|
+
|
108
|
+
``` ruby
|
109
|
+
irb> a.fetch
|
110
|
+
=> {"experiment"=>"widget-permissions", "user"=>{ ... } .... }
|
111
|
+
irb> a.result
|
112
|
+
=> {"experiment"=>"widget-permissions", "user"=>{ ... } .... }
|
113
|
+
irb> a.result.keys
|
114
|
+
=> ["experiment", "user", "timestamp", "candidate", "control", "first"]
|
115
|
+
irb> a.result.experiment_name
|
116
|
+
=> "widget-permissions"
|
117
|
+
irb> a.result['first']
|
118
|
+
=> "candidate"
|
119
|
+
irb> a.result.first
|
120
|
+
=> "candidate"
|
121
|
+
irb> a.result['control']
|
122
|
+
=> {"duration"=>12.307, "exception"=>nil, "value"=>false}
|
123
|
+
irb> a.result.control
|
124
|
+
=> {"duration"=>12.307, "exception"=>nil, "value"=>false}
|
125
|
+
irb> a.result['candidate']
|
126
|
+
=> {"duration"=>12.366999999999999, "exception"=>nil, "value"=>true}
|
127
|
+
irb> a.result.candidate
|
128
|
+
=> {"duration"=>12.366999999999999, "exception"=>nil, "value"=>true}
|
129
|
+
irb> a.result['first']
|
130
|
+
=> "control"
|
131
|
+
irb> a.result['timestamp']
|
132
|
+
=> "2013-04-22T13:31:32-05:00"
|
133
|
+
irb> a.result.timestamp
|
134
|
+
=> 2013-04-22 13:31:32 -0500
|
135
|
+
irb> a.result.timestamp.class
|
136
|
+
=> Time
|
137
|
+
irb> a.result.timestamp.to_i
|
138
|
+
=> 1366655492
|
139
|
+
irb> a.result['user']
|
140
|
+
=> {"login"=>"somed00d", ... }
|
141
|
+
```
|
142
|
+
|
143
|
+
Results will contain entries for the duration (in milliseconds), exceptions,
|
144
|
+
and values returned by both the candidate and control methods for the experiment;
|
145
|
+
the time when the result was recorded; whether the candidate or the control method
|
146
|
+
was run first; and an entry for every object saved via a `context` call during
|
147
|
+
the experiment.
|
148
|
+
|
149
|
+
Note that the `#result` method will continue to return the previously fetched
|
150
|
+
result, until we overwrite it with another `#fetch`, `#skip`, or `#analyze`
|
151
|
+
(see below).
|
152
|
+
|
153
|
+
#### Skipping results
|
154
|
+
|
155
|
+
Sometimes we make changes to the code we're running experiments against, and
|
156
|
+
sometimes those changes cause experiment results to be out of date -- if we've
|
157
|
+
fixed a bug we found via science, it's not much point in looking at results
|
158
|
+
generated while our code still had that bug. To jump past a batch of results,
|
159
|
+
use `#skip`, giving it a block to test for the condition we want to skip
|
160
|
+
past:
|
161
|
+
|
162
|
+
``` ruby
|
163
|
+
irb> a.skip {|r| 5.minutes.ago < a.result.timestamp }
|
164
|
+
=> 43
|
165
|
+
irb> a.skip {|r| true }
|
166
|
+
=> nil
|
167
|
+
```
|
168
|
+
|
169
|
+
### Batch analysis of results
|
170
|
+
|
171
|
+
After sifting through a handful of results from an experiment, it usually
|
172
|
+
becomes obvious that a single behavior in our studied code is often responsible
|
173
|
+
for many results published in an experiment. If a behavior difference can be
|
174
|
+
easily fixed by improving the candidate code, and your production release cycle
|
175
|
+
is short, then you just update the candidate method and continuing running your
|
176
|
+
experiment.
|
177
|
+
|
178
|
+
It's often the case that the relevant code can't be changed that quickly.
|
179
|
+
Perhaps the assumptions made when writing the candidate code were wrong in a way
|
180
|
+
that requires deeper consideration and discussion with your team. It could be
|
181
|
+
that the experiment results actually turn up bugs in the implementation of the
|
182
|
+
control method -- in which case there will likely be even more discussion
|
183
|
+
needed, and possibly a fairly long cycle to get production behaving properly.
|
184
|
+
|
185
|
+
That doesn't mean that analysis can't continue, but it could well be that a
|
186
|
+
majority of the experimental results to analyze are already examples of already
|
187
|
+
known behaviors. In this case, it's useful to be able to identify these results
|
188
|
+
and skip over them, to find results which can't be accounted for by any
|
189
|
+
currently known explanation.
|
190
|
+
|
191
|
+
The `#analyze` method, in conjunction with "matcher classes", makes this possible.
|
192
|
+
|
193
|
+
### `#analyze`
|
194
|
+
|
195
|
+
You can run `#analyze` to automate the fetching of pending results. If a result
|
196
|
+
is identifiable by a matcher class, then a summary of the identified result will
|
197
|
+
be printed and that result will skipped. This process continues until either an
|
198
|
+
unidentifiable result is found, or there are no more results available. When an
|
199
|
+
unidentifiable result is found, a summary of the identified results is output,
|
200
|
+
and then the first unidentified result is displayed in detail.
|
201
|
+
|
202
|
+
```
|
203
|
+
irb> a.analyze
|
204
|
+
User [somed00d] is staff (see http://github.com/our/project/issues/123)
|
205
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
206
|
+
User [somed00d] is staff (see http://github.com/our/project/issues/123)
|
207
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
208
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
209
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
210
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
211
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
212
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
213
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
214
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
215
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
216
|
+
User [somed00d] is staff (see http://github.com/our/project/issues/123)
|
217
|
+
User [somed00d] is staff (see http://github.com/our/project/issues/123)
|
218
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
219
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
220
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
221
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
222
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
223
|
+
User [somed00d] is staff (see http://github.com/our/project/issues/123)
|
224
|
+
User [somed00d] is staff (see http://github.com/our/project/issues/123)
|
225
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
226
|
+
User [0th3rd00d] is staff (see http://github.com/our/project/issues/123)
|
227
|
+
Permission [totesadmin] is obsolete (see http://github.com/dat/thing/issues/5234)
|
228
|
+
|
229
|
+
Summary of identified results:
|
230
|
+
|
231
|
+
StaffFunninessMatcher: 14
|
232
|
+
ZOMGIssue5423Matcher: 10
|
233
|
+
TOTAL: 24
|
234
|
+
|
235
|
+
First unidentifiable result:
|
236
|
+
|
237
|
+
Experiment [widget-permissions] first: candidate @ 2013-04-19T18:55:23-05:00
|
238
|
+
Duration: control ( 0.01) | candidate ( 1.36)
|
239
|
+
|
240
|
+
Control value: [false]
|
241
|
+
Candidate value: [true]
|
242
|
+
|
243
|
+
user => {
|
244
|
+
id => 1234876
|
245
|
+
login => "somed00d"
|
246
|
+
[...]
|
247
|
+
}
|
248
|
+
=> 32
|
249
|
+
```
|
250
|
+
|
251
|
+
Note that the number of pending results is returned as the result of the
|
252
|
+
analysis.
|
253
|
+
|
254
|
+
|
255
|
+
### Matcher classes
|
256
|
+
|
257
|
+
The purpose of a matcher class is to identify a behavior which results in
|
258
|
+
mismatches in your experiment. For example, if permissions for staff users are
|
259
|
+
not implemented properly by your candidate code, you might create a matcher that
|
260
|
+
recognizes when the user involved is a staff user.
|
261
|
+
|
262
|
+
You create a matcher class by subclassing `Dat::Analysis::Matcher` and writing a
|
263
|
+
`#match?` method that returns true if the experiment result (available as
|
264
|
+
`result`) is an example of the behavior we know about:
|
265
|
+
|
266
|
+
``` ruby
|
267
|
+
class StaffFunninessMatcher < Dat::Analysis::Matcher
|
268
|
+
# our staff role permissions are just soooo busted
|
269
|
+
def match?
|
270
|
+
User.find_by_login(result['user']['login']).staff?
|
271
|
+
end
|
272
|
+
|
273
|
+
def readable
|
274
|
+
"User [#{result['user']['login']}] is staff (see http://github.com/our/project/issues/123)"
|
275
|
+
end
|
276
|
+
end
|
277
|
+
```
|
278
|
+
|
279
|
+
If you create a matcher class in the console, use `#add_matcher` to let your
|
280
|
+
analyzer know about it:
|
281
|
+
|
282
|
+
``` ruby
|
283
|
+
irb> a.add_matcher StaffFunninessMatcher
|
284
|
+
Loading matcher class [StaffFunninessMatcher]
|
285
|
+
=> [StaffFunninessMatcher]
|
286
|
+
```
|
287
|
+
|
288
|
+
Now, when you run `#analyze`, all the results with staff users recorded in the
|
289
|
+
`user` context will be tallied and skipped.
|
290
|
+
|
291
|
+
See "Maintaining a library of matchers and wrappers" below for a more durable
|
292
|
+
way to let your analyzers keep track of your helper classes.
|
293
|
+
|
294
|
+
#### Getting a summary of an identified result
|
295
|
+
|
296
|
+
The `#summary` method on the analyzer will return a readable version of the
|
297
|
+
current result. This is by default a fairly voluminous output (it's what you saw
|
298
|
+
at the end of an `#analyze` run above), but if your matcher defines a
|
299
|
+
`#readable` method.
|
300
|
+
|
301
|
+
``` ruby
|
302
|
+
irb> a.summary
|
303
|
+
=> "User [somed00d] is staff (see http://github.com/our/project/issues/123)"
|
304
|
+
```
|
305
|
+
|
306
|
+
The `#analyze` method uses these `#readable` methods to produce a more succinct
|
307
|
+
summary of identified results, like we showed above.
|
308
|
+
|
309
|
+
**Define a `#readable` method for cleaner `#analyze` output!**
|
310
|
+
|
311
|
+
### Adding methods to results (wrappers)
|
312
|
+
|
313
|
+
For many experiments there is information in the results which is used often
|
314
|
+
enough that you'll get tired of doing repetitive lookups in the results hash.
|
315
|
+
When this happens, you can create result wrapper classes for your experiment
|
316
|
+
which can add methods to every result returned. Simply subclass
|
317
|
+
`Dat::Analysis::Result` and define the instance methods you want:
|
318
|
+
|
319
|
+
``` ruby
|
320
|
+
class PermissionsWrapper < Dat::Analysis::Result
|
321
|
+
def user
|
322
|
+
User.find_by_login!(result['user']['login'])
|
323
|
+
rescue
|
324
|
+
"Could not find user, id=[#{result['actor']['id']}]"
|
325
|
+
end
|
326
|
+
|
327
|
+
def permission
|
328
|
+
Permission.find_by_handle!(result['permission']['handle'])
|
329
|
+
rescue
|
330
|
+
"Could not find permission, handle=[#{result['permission']['handle']}]"
|
331
|
+
end
|
332
|
+
alias_method :perm, :permission
|
333
|
+
end
|
334
|
+
```
|
335
|
+
|
336
|
+
Then, add the wrapper to your analyzer:
|
337
|
+
|
338
|
+
``` ruby
|
339
|
+
irb> a.add_wrapper(PermissionsWrapper)
|
340
|
+
=> [PermissionsWrapper]
|
341
|
+
irb> a.result.user
|
342
|
+
=> #<User id: 1234876, login: "somed00d", ...>
|
343
|
+
```
|
344
|
+
|
345
|
+
These wrappers can also be used in your matchers classes:
|
346
|
+
|
347
|
+
``` ruby
|
348
|
+
class StaffFunninessMatcher < Dat::Analysis::Matcher
|
349
|
+
# our staff role permissions are just soooo busted
|
350
|
+
def match?
|
351
|
+
result.user.staff?
|
352
|
+
end
|
353
|
+
|
354
|
+
def readable
|
355
|
+
"User [#{result.user.login}] is staff (see http://github.com/our/project/issues/123)"
|
356
|
+
end
|
357
|
+
end
|
358
|
+
```
|
359
|
+
|
360
|
+
#### Skipping class naming
|
361
|
+
|
362
|
+
Inventing new non-conflicting class names for matcher and wrapper classes is a
|
363
|
+
bit of a pain. Often we just declare an anonymous class and skip the naming
|
364
|
+
altogether. If you do this, you'll probably want to define a readable `.name`
|
365
|
+
method for your class, so that `#analyze` summaries are readable:
|
366
|
+
|
367
|
+
``` ruby
|
368
|
+
Class.new(Dat::Analysis::Matcher) do
|
369
|
+
def self.name
|
370
|
+
"Staff Permission Silliness"
|
371
|
+
end
|
372
|
+
|
373
|
+
def match?
|
374
|
+
result.user.staff?
|
375
|
+
end
|
376
|
+
|
377
|
+
def readable
|
378
|
+
"User [#{result.user.login}] is staff (see http://github.com/our/project/issues/123)"
|
379
|
+
end
|
380
|
+
end
|
381
|
+
```
|
382
|
+
|
383
|
+
### Maintaining a library of matchers and result wrappers
|
384
|
+
|
385
|
+
Being able to add matchers and result wrappers to an analyzer during a console
|
386
|
+
session is a fast way to iteratively identify problems and work through a batch of
|
387
|
+
results. Keeping those matchers around for the next session is usually in order.
|
388
|
+
Your `Dat::Analysis` subclass can define a `#path` instance method, which points
|
389
|
+
to the place on the filesystem where your matcher and wrapper classes live. The
|
390
|
+
analyzer will look here, in a sub-directory named for your experiment, and load
|
391
|
+
any ruby files it finds there:
|
392
|
+
|
393
|
+
``` ruby
|
394
|
+
require 'dat/analysis'
|
395
|
+
|
396
|
+
module MyApp
|
397
|
+
# Public: Perform dat analysis on a dat-science experiment.
|
398
|
+
#
|
399
|
+
# This is a subclass of Dat::Analysis which provides the concrete implementation
|
400
|
+
# of the `#read`, `#count`, and `#cook` methods to interact with our Redis data
|
401
|
+
# store, and decodes our science mismatch results from JSON.
|
402
|
+
class Analysis < Dat::Analysis
|
403
|
+
def path
|
404
|
+
'/path/to/dat-science/experiments/'
|
405
|
+
end
|
406
|
+
end
|
407
|
+
end
|
408
|
+
```
|
409
|
+
|
410
|
+
In this example, the analyzer for the `widget-permissions` experiment will look
|
411
|
+
in `/path/to/dat-science/experiments/widget-permissions/` for matcher and
|
412
|
+
wrapper classes.
|
413
|
+
|
414
|
+
## Hacking on dat-analysis
|
415
|
+
|
416
|
+
Be on a Unixy box. Make sure a modern Bundler is available. `script/test` runs
|
417
|
+
the unit tests. All development dependencies will be installed automatically if
|
418
|
+
they're not available. Dat science happens primarily on Ruby 1.9.3 and 1.8.7,
|
419
|
+
but science should be universal.
|
420
|
+
|
421
|
+
## Maintainers
|
422
|
+
|
423
|
+
[@jbarnette](https://github.com/jbarnette) and [@rick](https://github.com/rick)
|