suture 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 65850d2b933384f85c0cd13ec67fe92efcb08320
-  data.tar.gz: ba005e2f0e87b693f89948102559dfccfba7720b
+  metadata.gz: 03b931eb888e1ce5e2d66f44c82063095b2d77b7
+  data.tar.gz: 395821b0b2c7075bfebddec65debbe8dc1860f4b
 SHA512:
-  metadata.gz: 7132a9cfa0499d89c786c7ccea1c7ed3da896ec786d3249b1aa3d00433eb0387d3be8917e3bb589f917f888624ab4bab71dc3c2f751a599a100422c09b717f2f
-  data.tar.gz: cd7f1fea872d8887a51f8b65d03cd5bff5189d9d4032141c0862c02265aec1c2acd4552e24e77e611fbb10e57ebd62796d2a956edf754c6dcde57df43438f0ff
+  metadata.gz: f909ce361c7a0b62da87a7332b022725d1c38e02d52c6aa0b4e7f884be841750e2d8b763f8b805cc48f6eff09ebe639d26d330254ef6a04933bba81076e3a335
+  data.tar.gz: f054acf5f1bb4a577b67973df7f093d26a93258cd2435c86c93885c7e6f4f4c5e7b17c5b5d5a516c4756cbf7816e2cdef990a4fd04191da8f41d519de16cc938

data/.codeclimate.yml

@@ -26,3 +26,4 @@ ratings:
 exclude_paths:
 - test/
 - safe/fixtures/
+- example/

data/CHANGELOG.md

@@ -1,8 +1,25 @@
 # Change Log
 
-## [Unreleased](https://github.com/testdouble/suture/tree/HEAD)
+## [v1.0.0](https://github.com/testdouble/suture/tree/v1.0.0) (2016-09-07)
+[Full Changelog](https://github.com/testdouble/suture/compare/v0.5.0...v1.0.0)
 
-[Full Changelog](https://github.com/testdouble/suture/compare/v0.4.0...HEAD)
+**Implemented enhancements:**
+
+- API to drop Suture's database [\#31](https://github.com/testdouble/suture/issues/31)
+
+**Closed issues:**
+
+- Document expected\_error\_types feature [\#55](https://github.com/testdouble/suture/issues/55)
+- Document rails env-specific settings [\#44](https://github.com/testdouble/suture/issues/44)
+- Rails project [\#35](https://github.com/testdouble/suture/issues/35)
+- Add sensible defaults to active record objects [\#30](https://github.com/testdouble/suture/issues/30)
+- Document how to use Coverage  [\#26](https://github.com/testdouble/suture/issues/26)
+- Support awesome\_print [\#23](https://github.com/testdouble/suture/issues/23)
+- Document Value-ifying methods with side effects [\#18](https://github.com/testdouble/suture/issues/18)
+- Document configuration  [\#17](https://github.com/testdouble/suture/issues/17)
+
+## [v0.5.0](https://github.com/testdouble/suture/tree/v0.5.0) (2016-09-05)
+[Full Changelog](https://github.com/testdouble/suture/compare/v0.4.0...v0.5.0)
 
 **Implemented enhancements:**
 

data/README.md

@@ -1,4 +1,4 @@
-# Suture
+# Suture 🏥
 
 [![Build Status](https://travis-ci.org/testdouble/suture.svg?branch=master)](https://travis-ci.org/testdouble/suture) [![Code Climate](https://codeclimate.com/github/testdouble/suture/badges/gpa.svg)](https://codeclimate.com/github/testdouble/suture) [![Test Coverage](https://codeclimate.com/github/testdouble/suture/badges/coverage.svg)](https://codeclimate.com/github/testdouble/suture/coverage)
 
@@ -65,7 +65,7 @@ much easier to verify return values.
 Since any changes to the code while it's untested are very dangerous, it's
 important to minimize changes made for the sake of creating a clear seam.
 
-### 2. Create our suture
+### 2. Create our seam
 
 Next, we introduce Suture to the call site so we can start analyzing its
 behavior:
@@ -392,7 +392,9 @@ In general, most configuration options can be set in several places:
 to an expected ENV var named `SUTURE_RECORD_CALLS` and can be set from the
 command line like so: `SUTURE_RECORD_CALLS=true bundle exec rails server`, to
 tell Suture to record all your interactions with your seams without touching the
-source code.
+source code. (Note: this is really only appropriate if your codebase only has one
+Suture seam in progress at a time, since using a global env var configuration
+for one seam's sake will erroneously impact the other.)
 
 * Globally, via the top-level `Suture.config` method. Most variables can be set
 via this top-level configuration, like
@@ -430,28 +432,35 @@ the `new` path at the exclusion of the `old` path (unless a mode flag like
 current working directory to the Sqlite3 database Suture uses to record and
 playback calls
 
-* _record_calls_ - (Default: false) - when set to true, the `old` path is called
+* _record_calls_ - (Default: `false`) - when set to true, the `old` path is called
 (regardless of whether `new` is set) and its arguments and result (be it a return
 value or an expected raised error) is recorded into the Suture database for the
 purpose of more coverage for calls to `Suture.verify`. [Read
 more](#3-record-the-current-behavior)
 
-* _call_both_ - (Default: false) - when set to true, the `new` path is invoked,
+* _call_both_ - (Default: `false`) - when set to true, the `new` path is invoked,
 then the `old` path is invoked, each with the seam's `args`. The return value
 from each is compared with the `comparator`, and if they are not equivalent, then
 a `Suture::Error::ResultMismatch` is raised. Intended after the `new` path is
 initially developed and to be run in pre-production environments. [Read
 more](#staging)
 
-* _fallback_on_error_ - (Default: false) - designed to be run in production after
+* _fallback_on_error_ - (Default: `false`) - designed to be run in production after
 the initial development of the new code path, when set to true, Suture will
 invoke the `new` code path. If `new` raises an error that isn't an
 `expected_error_type`, then Suture will invoke the `old` path with the same args
 in an attempt to recover a working state for the user. [Read more](#production)
 
-* _raise_on_result_mismatch_ - (Default: true) - when set to true, the
+* _raise_on_result_mismatch_ - (Default: `true`) - when set to true, the
 `call_both` mode will merely log incidents of result mismatches, as opposed to
-raising `Suture::Error::ResultMismatch`.
+raising `Suture::Error::ResultMismatch`
+
+* _return_old_on_result_mismatch_ - (Default: `false`) - when set to true, the
+`call_both` mode will return the result of the `old` code path instead of the
+`new` code path. This is useful when you want to log mismatches in production
+(i.e. when you're very confident it's safe and fast enough to use `call_both` in
+production), but want to fallback to the `old` path in the case of a mismatch
+to minimize disruption to your users
 
 * _comparator_ - (Default: `Suture::Comparator.new`) - determines how return
 values from the Suture are compared when invoking `Suture.verify` or when
@@ -471,16 +480,14 @@ certain cases, setting `:expected_error_types => [WidgetError]` will result in:
   * `Suture.create`, when `fallback_on_error` is enabled, will allow expected
     errors raised by the `new` path to propogate, as opposed to logging &
     rescuing them before calling the `old` path as a fallback
-  * Additionally, `Suture.verify` can be passed `expected_error_types` to squelch
-    warning logs that result from unexpectedly raised errors
 
-* _disable_ - (Default: false) - when enabled, Suture will attempt to revert to
+* _disable_ - (Default: `false`) - when enabled, Suture will attempt to revert to
 the original behavior of the `old` path and take no special action. Useful in
 cases where a bug is discovered in a deployed environment and you simply want
 to hit the brakes on any new code path experiments by setting
 `SUTURE_DISABLE=true` globally
 
-* _dup_args_ - (Default: false) - when enabled, Suture will call `dup` on each
+* _dup_args_ - (Default: `false`) - when enabled, Suture will call `dup` on each
 of the args passed to the `old` and/or `new` code paths. Useful when the code
 path(s) mutate the arguments in such a way as to prevent `call_both` or
 `fallback_on_error` from being effective
@@ -499,9 +506,13 @@ unexpected error (see `expected_error_types`).
 
 #### Suture.verify
 
-Many of the settings for `Suture.verify` are analogous to the same settings in
-`Suture.create` and are generally expected to be configured in the same way, as
-if symmetrically with the `Suture.create` call of the seam under test:
+`Suture.verify(name, [options hash])`
+
+Many of the settings for `Suture.verify` mirror the settings available to
+`Suture.create`. In general, the two methods' common options should be configured
+identically for a given seam; this is necessary, because the `Suture.verify` call
+site doesn't depend on (or know about) any `Suture.create` call site of the same
+name; the only resource they share is the recorded calls in Suture's database.
 
 * _name_ - (Required) - should be the same name as a seam for which some number
 of recorded calls exist
@@ -511,27 +522,31 @@ set of `args` and have its result compared to that of each recording. This is
 used in lieu of `old` or `new`, since the subject of a `Suture.verify` test might
 be either (or neither!)
 
-* _verify_only_ - (Default: nil) - when set to an ID, Suture.verify` will only
+* _database_path_ - (Default: `"db/suture.sqlite3"`) - as with `Suture.create`, a
+custom database path can be set for almost any invocation of Suture, and
+`Suture.verify is no exception`
+
+* _verify_only_ - (Default: `nil`) - when set to an ID, Suture.verify` will only
 run against recorded calls for the matching ID. This option is meant to be used
 to focus work on resolving a single verification failure
 
-* _fail_fast_ - (Default: false) - `Suture.verify` will, by default, run against
+* _fail_fast_ - (Default: `false`) - `Suture.verify` will, by default, run against
 every single recording, aggregating and reporting on all errors (just like, say,
 RSpec or Minitest would). However, if the seam is slow to invoke or if you
 confidently expect all of the recordings to pass verification, `fail_fast` is an
 appropriate option to set.
 
-* _call_limit_ - (Default: nil) - when set to a number, Suture will only verify
+* _call_limit_ - (Default: `nil`) - when set to a number, Suture will only verify
 up to the set number of recorded calls. Because Suture randomizes the order of
 verifications by default, you can see this as setting Suture.verify to sample a
 random smattering of `call_limit` recordings as a smell test. Potentially useful
 when a seam is very slow
 
-* _time_limit_ - (Default: nil) - when set to a number (in seconds), Suture will
+* _time_limit_ - (Default: `nil`) - when set to a number (in seconds), Suture will
 stop running verifications against recordings once `time_limit` seconds has
 elapsed. Useful when a seam is very slow to invoke
 
-* _error_message_limit_ - (Default: nil) - when set to a number, Suture will only
+* _error_message_limit_ - (Default: `nil`) - when set to a number, Suture will only
 print up to `error_message_limit` failure messages. That way, if you currently
 have hundreds of verifications failing, your console isn't overwhelmed by them on
 each run of `Suture.verify`
@@ -548,9 +563,10 @@ used by `Suture.verify` to ensure the results are comparable. [Read
 more](#creating-a-custom-comparator) on creating custom comparators
 )
 
-* _database_path_ - (Default: `"db/suture.sqlite3"`) - as with `Suture.create`, a
-custom database path can be set for almost any invocation of Suture, and
-`Suture.verify is no exception`
+* _expected_error_types_ - (Default: `[]`) - this option has little impact on
+`Suture.verify` (since each recording will either verify a return value or an
+error in its own right), however it can be set to squelch log messages warning
+that errors were raised when invoking the `subject`
 
 * _after_subject_ - a `call`-able hook that runs after `subject` is invoked. If
 `subject` raises an error, it is not invoked

data/Rakefile

@@ -47,7 +47,7 @@ if Gem.ruby_version >= Gem::Version.new("2.2.2")
     puts "-------> #{cmd}"
     system cmd
   end
-  Rake::Task["release:source_control_push"].enhance([:changelog, :changelog_commit])
+  Rake::Task["release:rubygem_push"].enhance([:changelog, :changelog_commit])
 end
 
 

data/lib/suture/create/validates_plan.rb

@@ -26,32 +26,81 @@ module Suture
     CONFLICTS = [
       lambda { |plan|
         if plan.record_calls && !plan.database_path
-          ":record_calls is enabled, but :database_path is nil, so Suture doesn't know where to record calls to the seam."
+          <<-MSG.gsub(/^ {12}/,'')
+            :record_calls is enabled, but :database_path is nil, so Suture
+              doesn't know where to record calls to the seam.
+          MSG
         end
       },
       lambda { |plan|
         if plan.record_calls && plan.call_both
-          ":record_calls & :call_both are both enabled and conflict with one another. :record_calls will only invoke the old code path (intended for characterization of the old code path and initial development of the new code path), whereas :call_both will invoke the new path and the old to compare their results after development of the new code path is initially complete (typically in a pre-production environment to validate the behavior of the new code path is consistent with the old). If you're still actively developing the new code path and need more recordings to feed Suture.verify, disable :call_both; otherwise, it's likely time to turn off :record_calls on this seam."
+          <<-MSG.gsub(/^ {12}/,'')
+            :record_calls & :call_both are both enabled and conflict with one
+              another. :record_calls will only invoke the old code path (intended
+              for characterization of the old code path and initial development
+              of the new code path), whereas :call_both will invoke the new path
+              and the old to compare their results after development of the new
+              code path is initially complete (typically in a pre-production
+              environment to validate the behavior of the new code path is
+              consistent with the old). If you're still actively developing the
+              new code path and need more recordings to feed Suture.verify,
+              disable :call_both; otherwise, it's likely time to turn off
+              :record_calls on this seam.
+          MSG
         end
       },
       lambda { |plan|
         if plan.record_calls && plan.fallback_on_error
-          ":record_calls & :fallback_on_error are both enabled and conflict with one another. :record_calls will only invoke the old code path (intended for characterization of the old code path and initial development of the new code path), whereas :fallback_on_error will call the new code path unless an error is raised, in which case it will fall back on the old code path."
+          <<-MSG.gsub(/^ {12}/,'')
+            :record_calls & :fallback_on_error are both enabled and conflict with
+              one another. :record_calls will only invoke the old code path
+              (intended for characterization of the old code path and initial
+              development of the new code path), whereas :fallback_on_error will
+              call the new code path unless an error is raised, in which case it
+              will fall back on the old code path.
+          MSG
         end
       },
       lambda { |plan|
         if plan.call_both && plan.fallback_on_error
-          ":call_both & :fallback_on_error are both enabled and conflict with one another. :call_both is designed for pre-production environments and will call both the old and new code paths to compare their results, whereas :fallback_on_error is designed for production environments where it is safe to call the old code path in the event that the new code path fails unexpectedly"
+          <<-MSG.gsub(/^ {12}/,'')
+            :call_both & :fallback_on_error are both enabled and conflict with
+              one another. :call_both is designed for pre-production environments
+              and will call both the old and new code paths to compare their
+              results, whereas :fallback_on_error is designed for production
+              environments where it is safe to call the old code path in the
+              event that the new code path fails unexpectedly
+          MSG
         end
       },
       lambda { |plan|
         if plan.call_both && !plan.new.respond_to?(:call)
-          ":call_both is set but :new is either not set or is not callable. In order to call both code paths, both :old and :new must be set and callable."
+          <<-MSG.gsub(/^ {12}/,'')
+            :call_both is set but :new is either not set or is not callable. In
+              order to call both code paths, both :old and :new must be set and
+              callable
+          MSG
         end
       },
       lambda { |plan|
         if plan.fallback_on_error && !plan.new.respond_to?(:call)
-          ":fallback_on_error is set but :new is either not set or is not callable. This mode is designed for after the :new code path has been developed and run in production-like environments, where :old is only kept around as a fallback to retry in the event that :new raises an unexpected error. Either specify a :new code path or disable :fallback_on_error."
+          <<-MSG.gsub(/^ {12}/,'')
+            :fallback_on_error is set but :new is either not set or is not
+              callable. This mode is designed for after the :new code path has
+              been developed and run in production-like environments, where :old
+              is only kept around as a fallback to retry in the event that
+              :new raises an unexpected error. Either specify a :new code path or
+              disable :fallback_on_error.
+          MSG
+        end
+      },
+      lambda { |plan|
+        if !plan.raise_on_result_mismatch && !plan.call_both
+          <<-MSG.gsub(/^ {12}/,'')
+            :raise_on_result_mismatch was disabled but :call_both is not enabled.
+              This option only applies to the :call_both mode, and will have no
+              impact when set for other modes
+          MSG
         end
       }
     ]

data/lib/suture/surgeon/auditor.rb

@@ -10,20 +10,35 @@ module Suture::Surgeon
       new_result = scalpel.cut(plan, :new)
       old_result = scalpel.cut(plan, :old)
       if !plan.comparator.call(old_result, new_result)
-        log_warn <<-MSG.gsub(/^ {10}/,'')
-          Seam #{plan.name.inspect} is set to :call_both the :new and :old code
-          paths, but they did not match. The new result was: ```
-            #{new_result.inspect}
-          ```
-          The old result was: ```
-            #{old_result.inspect}
-          ```
-        MSG
-        if plan.raise_on_result_mismatch
-          raise Suture::Error::ResultMismatch.new(plan, new_result, old_result)
-        end
+        handle_mismatch(plan, old_result, new_result)
+      else
+        new_result
       end
-      new_result
+    end
+
+    private
+
+    def handle_mismatch(plan, old_result, new_result)
+      log_warning(plan, old_result, new_result)
+      if plan.raise_on_result_mismatch
+        raise Suture::Error::ResultMismatch.new(plan, new_result, old_result)
+      elsif plan.return_old_on_result_mismatch
+        old_result
+      else
+        new_result
+      end
+    end
+
+    def log_warning(plan, old_result, new_result)
+      log_warn <<-MSG.gsub(/^ {8}/,'')
+        Seam #{plan.name.inspect} is set to :call_both the :new and :old code
+        paths, but they did not match. The new result was: ```
+          #{new_result.inspect}
+        ```
+        The old result was: ```
+          #{old_result.inspect}
+        ```
+      MSG
     end
   end
 end

data/lib/suture/value/plan.rb

@@ -2,7 +2,8 @@ module Suture::Value
   class Plan
     attr_reader :name, :old, :new, :args, :after_new, :after_old, :on_new_error,
                 :on_old_error, :database_path, :record_calls, :comparator,
-                :call_both, :raise_on_result_mismatch, :fallback_on_error,
+                :call_both, :raise_on_result_mismatch,
+                :return_old_on_result_mismatch, :fallback_on_error,
                 :expected_error_types, :disable, :dup_args
 
     def initialize(attrs = {})
@@ -19,6 +20,7 @@ module Suture::Value
       @comparator = attrs[:comparator]
       @call_both = !!attrs[:call_both]
       @raise_on_result_mismatch = !!attrs[:raise_on_result_mismatch]
+      @return_old_on_result_mismatch = !!attrs[:return_old_on_result_mismatch]
       @fallback_on_error = !!attrs[:fallback_on_error]
       @expected_error_types = attrs[:expected_error_types] || []
       @disable = !!attrs[:disable]

data/lib/suture/version.rb

@@ -1,3 +1,3 @@
 module Suture
-  VERSION = "1.0.0"
+  VERSION = "1.1.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: suture
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Justin Searls
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-09-07 00:00:00.000000000 Z
+date: 2016-09-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sqlite3