scientist 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c0c4363232ca282aeab9af989a6499a4aaa8eea8
-  data.tar.gz: 781901ba02fe3c35bc0ab79ed9a969998834a92a
+  metadata.gz: 5c22232b66f7eeb5a35e4f6a308042bdbed6a42b
+  data.tar.gz: 18765e089ded1443d66ece9a606bea76df53fc51
 SHA512:
-  metadata.gz: 5d27e2fc8dc16c73253dcf66f22c21c73a589cdfd6f8cbf88337f88f8ae6cd40fa047e701c16440f20eca36aa644ca940fe6f321269a62d5e0808fb3e79bc43f
-  data.tar.gz: 063961175f72b277982ce4eb1bd9060a65a902a75c4a4735689202030534fe0fedeede28e6e67815c8d7e8a69af7bd979c7fcb41758790683045a73da21d90de
+  metadata.gz: fe1306268ecef4790dc0acc9c9d8783c3ba4088d0bc7aabda95725cf805cbee06ac058cbbbd2783e5f785236e60288146091ac6794e2ba7b912c3695ac3415b4
+  data.tar.gz: 262e07ce96dee85a475c962ba3de7e5f2eafb3199ba036d7a6cd727693baf27271cef7f84b310bcaad1d187a98317edcc2aeee9026dae6b1dc8028b8b6aeecc3

data/.gitignore

@@ -2,3 +2,4 @@
 /.bundle
 /.ruby-version
 /Gemfile.lock
+/coverage

data/.travis.yml

@@ -7,4 +7,10 @@ rvm:
   - 2.0.0
   - 2.1.8
   - 2.2.4
+  - 2.3.0
+  - 2.4.0
 before_install: gem install bundler
+addons:
+ apt:
+   packages:
+     - libgmp-dev

data/README.md

@@ -1,6 +1,6 @@
 # Scientist!
 
-A Ruby library for carefully refactoring critical paths. [![Build Status](https://travis-ci.org/github/scientist.svg?branch=master)](https://travis-ci.org/github/scientist)
+A Ruby library for carefully refactoring critical paths. [![Build Status](https://travis-ci.org/github/scientist.svg?branch=master)](https://travis-ci.org/github/scientist) [![Coverage Status](https://coveralls.io/repos/github/github/scientist/badge.svg?branch=master)](https://coveralls.io/github/github/scientist?branch=master)
 
 ## How do I science?
 
@@ -40,9 +40,9 @@ class MyWidget
   include Scientist
 
   def allows?(user)
-    science "widget-permissions" do |e|
-      e.use { model.check_user(user).valid? } # old way
-      e.try { user.can?(:read, model) } # new way
+    science "widget-permissions" do |experiment|
+      experiment.use { model.check_user(user).valid? } # old way
+      experiment.try { user.can?(:read, model) } # new way
     end # returns the control value
   end
 end
@@ -55,22 +55,25 @@ If you don't declare any `try` blocks, none of the Scientist machinery is invoke
 The examples above will run, but they're not really *doing* anything. The `try` blocks run every time and none of the results get published. Replace the default experiment implementation to control execution and reporting:
 
 ```ruby
-require "scientist"
+require "scientist/experiment"
 
 class MyExperiment
-  include ActiveModel::Model
   include Scientist::Experiment
 
   attr_accessor :name
 
+  def initialize(name:)
+    @name = name
+  end
+
   def enabled?
     # see "Ramping up experiments" below
-    super
+    true
   end
 
   def publish(result)
     # see "Publishing results" below
-    super
+    p result
   end
 end
 
@@ -191,9 +194,11 @@ end
 And this cleaned value is available in observations in the final published result:
 
 ```ruby
-class MyExperiment < ActiveRecord::Base
+class MyExperiment
   include Scientist::Experiment
 
+  # ...
+
   def publish(result)
     result.control.value         # [<User alice>, <User bob>, <User carol>]
     result.control.cleaned_value # ["alice", "bob", "carol"]
@@ -244,11 +249,22 @@ end
 As a scientist, you know it's always important to be able to turn your experiment off, lest it run amok and result in villagers with pitchforks on your doorstep. In order to control whether or not an experiment is enabled, you must include the `enabled?` method in your `Scientist::Experiment` implementation.
 
 ```ruby
-class MyExperiment < ActiveRecord::Base
+class MyExperiment
   include Scientist::Experiment
+
+  attr_accessor :name, :percent_enabled
+
+  def initialize(name:)
+    @name = name
+    @percent_enabled = 100
+  end
+
   def enabled?
     percent_enabled > 0 && rand(100) < percent_enabled
   end
+
+  # ...
+
 end
 ```
 
@@ -338,7 +354,18 @@ Scientist will raise a `Scientist::Experiment::MismatchError` exception if any o
 
 ### Handling errors
 
-If an exception is raised within any of scientist's internal helpers, like `publish`, `compare`, or `clean`, the `raised` method is called with the symbol name of the internal operation that failed and the exception that was raised. The default behavior of `Scientist::Default` is to simply re-raise the exception. Since this halts the experiment entirely, it's often a better idea to handle this error and continue so the experiment as a whole isn't canceled entirely:
+#### In candidate code
+
+Scientist rescues and tracks _all_ exceptions raised in a `try` or `use` block, including some where rescuing may cause unexpected behavior (like `SystemExit` or `ScriptError`). To rescue a more restrictive set of exceptions, modify the `RESCUES` list:
+
+```ruby
+# default is [Exception]
+Scientist::Observation::RESCUES.replace [StandardError]
+```
+
+#### In a Scientist callback
+
+If an exception is raised within any of Scientist's internal helpers, like `publish`, `compare`, or `clean`, the `raised` method is called with the symbol name of the internal operation that failed and the exception that was raised. The default behavior of `Scientist::Default` is to simply re-raise the exception. Since this halts the experiment entirely, it's often a better idea to handle this error and continue so the experiment as a whole isn't canceled entirely:
 
 ```ruby
 class MyExperiment
@@ -435,6 +462,23 @@ end
 
 Be on a Unixy box. Make sure a modern Bundler is available. `script/test` runs the unit tests. All development dependencies are installed automatically. Science requires Ruby 1.9 or newer.
 
+## Alternatives
+
+- [daylerees/scientist](https://github.com/daylerees/scientist) (PHP)
+- [github/scientist.net](https://github.com/github/scientist.net) (.NET)
+- [joealcorn/laboratory](https://github.com/joealcorn/laboratory) (Python)
+- [rawls238/Scientist4J](https://github.com/rawls238/Scientist4J) (Java)
+- [tomiaijo/scientist](https://github.com/tomiaijo/scientist) (C++)
+- [trello/scientist](https://github.com/trello/scientist) (node.js)
+- [ziyasal/scientist.js](https://github.com/ziyasal/scientist.js) (node.js, ES6)
+- [yeller/laboratory](https://github.com/yeller/laboratory) (Clojure)
+- [lancew/Scientist](https://github.com/lancew/Scientist) (Perl 5)
+- [lancew/ScientistP6](https://github.com/lancew/ScientistP6) (Perl 6)
+- [MadcapJake/Test-Lab](https://github.com/MadcapJake/Test-Lab) (Perl 6)
+- [cwbriones/scientist](https://github.com/cwbriones/scientist) (Elixir)
+- [calavera/go-scientist](https://github.com/calavera/go-scientist) (Go)
+
+
 ## Maintainers
 
 [@jbarnette](https://github.com/jbarnette),

data/doc/changelog.md

@@ -0,0 +1,10 @@
+# Changes
+
+## v1.1.0 (29 August 2017)
+
+- New: [Specify which exception types to rescue](https://github.com/github/scientist#in-candidate-code)
+- New: List [alternative implementations](https://github.com/github/scientist#alternatives) in `README.md`
+- New: Code coverage via [coveralls.io](https://coveralls.io/github/github/scientist)
+- New: Run CI on Ruby 2.3 and 2.4
+- Fix: `README` typos, examples, and lies
+- Fix: `false` results are now passed to the cleaner

data/lib/scientist/errors.rb

@@ -23,7 +23,7 @@ module Scientist
   class BehaviorNotUnique < BadBehavior
     def initialize(experiment, name)
       super experiment, name,
-        "#{experiment.name} alread has #{name} behavior"
+        "#{experiment.name} already has #{name} behavior"
     end
   end
 

data/lib/scientist/experiment.rb

@@ -124,7 +124,7 @@ module Scientist::Experiment
   # A Symbol-keyed Hash of extra experiment data.
   def context(context = nil)
     @_scientist_context ||= {}
-    @_scientist_context.merge!(context) if !context.nil?
+    @_scientist_context.merge!(context) unless context.nil?
     @_scientist_context
   end
 

data/lib/scientist/observation.rb

@@ -1,6 +1,10 @@
 # What happened when this named behavior was executed? Immutable.
 class Scientist::Observation
 
+  # An Array of Exception types to rescue when initializing an observation.
+  # NOTE: This Array will change to `[StandardError]` in the next major release.
+  RESCUES = [Exception]
+
   # The experiment this observation is for
   attr_reader :experiment
 
@@ -26,7 +30,7 @@ class Scientist::Observation
 
     begin
       @value = block.call
-    rescue Object => e
+    rescue *RESCUES => e
       @exception = e
     end
 
@@ -38,9 +42,7 @@ class Scientist::Observation
   # Return a cleaned value suitable for publishing. Uses the experiment's
   # defined cleaner block to clean the observed value.
   def cleaned_value
-    if value
-      experiment.clean_value value
-    end
+    experiment.clean_value value unless value.nil?
   end
 
   # Is this observation equivalent to another?
@@ -61,25 +63,16 @@ class Scientist::Observation
   def equivalent_to?(other, &comparator)
     return false unless other.is_a?(Scientist::Observation)
 
-    values_are_equal = false
-    both_raised      = other.raised? && raised?
-    neither_raised   = !other.raised? && !raised?
-
-    if neither_raised
-      if block_given?
-        values_are_equal = yield value, other.value
-      else
-        values_are_equal = value == other.value
-      end
+    if raised? || other.raised?
+      return other.exception.class == exception.class &&
+        other.exception.message == exception.message
     end
 
-    exceptions_are_equivalent = # backtraces will differ, natch
-      both_raised &&
-        other.exception.class == exception.class &&
-          other.exception.message == exception.message
-
-    (neither_raised && values_are_equal) ||
-      (both_raised && exceptions_are_equivalent)
+    if comparator
+      comparator.call(value, other.value)
+    else
+      value == other.value
+    end
   end
 
   def hash

data/lib/scientist/version.rb

@@ -1,3 +1,3 @@
 module Scientist
-  VERSION = "1.0.0"
+  VERSION = "1.1.0"
 end

data/scientist.gemspec

@@ -16,4 +16,5 @@ Gem::Specification.new do |gem|
   gem.require_paths = ["lib"]
 
   gem.add_development_dependency "minitest", "~> 5.8"
+  gem.add_development_dependency "coveralls", "~> 0.8"
 end

data/script/test

@@ -6,6 +6,7 @@ set -e
 cd $(dirname "$0")/..
   script/bootstrap && ruby -I lib \
     -e 'require "bundler/setup"' \
+    -e 'require "coveralls"; Coveralls.wear!{ add_filter ".bundle" }' \
     -e 'require "minitest/autorun"' \
     -e 'require "scientist"' \
     -e '(ARGV.empty? ? Dir["test/**/*_test.rb"] : ARGV).each { |f| load f }' -- "$@"

data/test/scientist/observation_test.rb

@@ -25,6 +25,37 @@ describe Scientist::Observation do
     assert_nil ob.value
   end
 
+  describe "::RESCUES" do
+    before do
+      @original = Scientist::Observation::RESCUES.dup
+    end
+
+    after do
+      Scientist::Observation::RESCUES.replace(@original)
+    end
+
+    it "includes all exception types by default" do
+      ob = Scientist::Observation.new("test", @experiment) do
+        raise Exception.new("not a StandardError")
+      end
+
+      assert ob.raised?
+      assert_instance_of Exception, ob.exception
+    end
+
+    it "can customize rescued types" do
+      Scientist::Observation::RESCUES.replace [StandardError]
+
+      ex = assert_raises Exception do
+        Scientist::Observation.new("test", @experiment) do
+          raise Exception.new("not a StandardError")
+        end
+      end
+
+      assert_equal "not a StandardError", ex.message
+    end
+  end
+
   it "compares values" do
     a = Scientist::Observation.new("test", @experiment) { 1 }
     b = Scientist::Observation.new("test", @experiment) { 1 }
@@ -88,6 +119,23 @@ describe Scientist::Observation do
       a = Scientist::Observation.new("test", @experiment) { "test" }
       assert_equal "TEST", a.cleaned_value
     end
+
+    it "doesn't clean nil values" do
+      @experiment.clean { |value| "foo" }
+      a = Scientist::Observation.new("test", @experiment) { nil }
+      assert_nil a.cleaned_value
+    end
+
+    it "returns false boolean values" do
+      a = Scientist::Observation.new("test", @experiment) { false }
+      assert_equal false, a.cleaned_value
+    end
+
+    it "cleans false values" do
+      @experiment.clean { |value| value.to_s.upcase }
+      a = Scientist::Observation.new("test", @experiment) { false }
+      assert_equal "FALSE", a.cleaned_value
+    end
   end
 
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: scientist
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - GitHub Open Source
@@ -12,22 +12,36 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-02-03 00:00:00.000000000 Z
+date: 2017-08-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ~>
       - !ruby/object:Gem::Version
         version: '5.8'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ~>
       - !ruby/object:Gem::Version
         version: '5.8'
+- !ruby/object:Gem::Dependency
+  name: coveralls
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '0.8'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '0.8'
 description: A Ruby library for carefully refactoring critical paths
 email:
 - opensource+scientist@github.com
@@ -39,11 +53,12 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".gitignore"
-- ".travis.yml"
+- .gitignore
+- .travis.yml
 - Gemfile
 - LICENSE.txt
 - README.md
+- doc/changelog.md
 - lib/scientist.rb
 - lib/scientist/default.rb
 - lib/scientist/errors.rb
@@ -70,17 +85,17 @@ require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - '>='
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - '>='
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.2.3
+rubygems_version: 2.0.14.1
 signing_key: 
 specification_version: 4
 summary: Carefully test, measure, and track refactored code.