suture 0.4.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b9c19b3ad957fa4dee235e314fcacde3aeac88c3
-  data.tar.gz: de6d831eaaecf0ff8b7edde1397b7c1304a486c2
+  metadata.gz: 9ca2d3a0376378b6ead8f55b0f92e1ef20b73991
+  data.tar.gz: b506cf09899bfd1e372c088aaba8203a0f0c20aa
 SHA512:
-  metadata.gz: e35674ad68833373e8d926437c0c272f7f9b216c821b150aec70fc74250e29a188f698d277bdca8dcf48f67baa534bbdcd355652ff56b9d5f991c38802955608
-  data.tar.gz: 4fa715fc19bbb2911c3e028ab360cf557b49b87d564764c4cb4d64109362d5bc00683c294a9d3e6d2ecfc72de6ebbf879540363898e0e60ee41f52f063b90efe
+  metadata.gz: 30813362842d2a559181b248b08904187d96f78d6feac8b3662ef4a2e7dfa35a2cdda38f5b589184e531266574f4c1e7b23fb7fd68b749bd90d36f73f3a9d652
+  data.tar.gz: 94f1c2b749cadc2a1dc80ac962bc7bfee7d735cea0eae1940a49c09543466f167e25faa809915ae8077e70f34d93590cd1e9bee1d32278be0cd0c77c47ee1122

data/.gitignore

@@ -10,3 +10,4 @@
 /db/
 /log/
 /ignore/
+.ruby-version

data/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Change Log
 
+## [Unreleased](https://github.com/testdouble/suture/tree/HEAD)
+
+[Full Changelog](https://github.com/testdouble/suture/compare/v0.3.3...HEAD)
+
+**Closed issues:**
+
+- common test parent for unit tests [\#49](https://github.com/testdouble/suture/issues/49)
+- rake test -\> rake unit w/ test as default [\#48](https://github.com/testdouble/suture/issues/48)
+- Override flag to disable the seam / revert to old path [\#47](https://github.com/testdouble/suture/issues/47)
+- Suture.verify :after\_subject hook [\#46](https://github.com/testdouble/suture/issues/46)
+- Suture.create :call\_old\_on\_error flag [\#43](https://github.com/testdouble/suture/issues/43)
+- Suture.create :on\_{new,old,subject}\_error handler [\#42](https://github.com/testdouble/suture/issues/42)
+- Implement Suture.create :call\_both [\#41](https://github.com/testdouble/suture/issues/41)
+- Suture.create :after\_old and :after\_new hooks [\#40](https://github.com/testdouble/suture/issues/40)
+- Suture.create :raise\_on\_result\_mismatch [\#39](https://github.com/testdouble/suture/issues/39)
+- Add code coverage / code climate [\#38](https://github.com/testdouble/suture/issues/38)
+- implement changelog generator [\#37](https://github.com/testdouble/suture/issues/37)
+- Validate plans [\#15](https://github.com/testdouble/suture/issues/15)
+
+## [v0.3.3](https://github.com/testdouble/suture/tree/v0.3.3) (2016-08-23)
+[Full Changelog](https://github.com/testdouble/suture/compare/v0.3.2...v0.3.3)
+
 ## [v0.3.2](https://github.com/testdouble/suture/tree/v0.3.2) (2016-08-23)
 [Full Changelog](https://github.com/testdouble/suture/compare/v0.3.1...v0.3.2)
 

data/README.md

@@ -34,7 +34,7 @@ you'd like to change. A good seam is:
 
 * easy to invoke in isolation
 * takes arguments, returns a value
-* eliminates (or at least minimizes) side effects
+* eliminates (or at least minimizes) side effects (for more on side effects, see [this tutorial](https://semaphoreci.com/community/tutorials/isolate-side-effects-in-ruby))
 
 Then, to create a seam, typically we create a new unit to house the code that we
 excise from its original site, and then we call it. This adds a level of
@@ -150,8 +150,8 @@ the test environment has the data needed to behave the same way as when it was
 recorded (it may be appropriate to take a snapshot of the database before you
 start recording and load it before you run your tests)
 
-* by generating a code coverage report (
-  [simplecov](https://github.com/colszowka/simplecov) is a good one to start
+* by generating a code coverage report
+  ([simplecov](https://github.com/colszowka/simplecov) is a good one to start
   with) from running this test in isolation, we can see what `LegacyWorker` is
   actually calling, in an attempt to do two things:
   * maximize coverage for code in the LegacyWorker (and for code that's
@@ -317,6 +317,7 @@ is a good choice).
 Suture.config({
   :log_level => "WARN", #<-- defaults to "INFO"
   :log_stdout => false, #<-- defaults to true
+  :log_io => StringIO.new,      #<-- defaults to nil
   :log_file => "log/suture.log" #<-- defaults to nil
 })
 ```
@@ -346,7 +347,7 @@ end
 ### Retrying failures
 
 Since the legacy code path hasn't been deleted yet, there's no reason to leave
-users hanging if the new code path explodes. By setting the `:call_old_on_error`
+users hanging if the new code path explodes. By setting the `:fallback_on_error`
 entry to `true`, Suture will rescue any errors raised from the new code path and
 attempt to invoke the legacy code path instead.
 
@@ -357,7 +358,7 @@ class MyWorker
       old: LegacyWorker.new,
       new: NewWorker.new,
       args: [id],
-      call_old_on_error: true
+      fallback_on_error: true
     }))
   end
 end
@@ -368,6 +369,15 @@ path will go unnoticed, so it's best used in conjunction with Suture's logging
 feature. Before ultimately deciding to finally delete the legacy code path,
 double-check that the logs aren't full of rescued errors!
 
+## Public API Summary
+
+* `Suture.create(name, opts)` - Creates a seam in your production source code
+* `Suture.verify(name, opts)` - Verifies a callable subject can recreate recorded calls
+* `Suture.config(config)` - Sets logging options, as well global defaults for other properties
+* `Suture.reset!` - Resets all Suture configuration
+* `Suture.delete!(id)` - Deletes a recorded call by `id`
+* `Suture.delete_all!(name)` - Deletes all recorded calls for a given seam `name`
+
 ## Configuration
 
 Legacy code is, necessarily, complex and hard-to-wrangle. That's why Suture comes
@@ -479,4 +489,3 @@ seeing false negatives:
     dropping Suture's database (which is, by default, stored in
     `db/suture.sqlite3`) or by observing the ID of the recording from an error
     message and invoking `Suture.delete!(42)`
-

data/Rakefile

@@ -1,7 +1,7 @@
 require "bundler/gem_tasks"
 require "rake/testtask"
 
-Rake::TestTask.new(:test) do |t|
+Rake::TestTask.new(:unit) do |t|
   t.libs << "test"
   t.libs << "lib"
   t.test_files = FileList['test/helper.rb', 'test/**/*_test.rb']
@@ -13,7 +13,7 @@ Rake::TestTask.new(:safe) do |t|
   t.test_files = FileList['safe/helper.rb', 'safe/**/*_test.rb']
 end
 
-Rake::TestTask.new(:everything) do |t|
+Rake::TestTask.new(:test) do |t|
   t.libs << "test"
   t.libs << "safe"
   t.libs << "lib"
@@ -39,4 +39,4 @@ if Gem.ruby_version >= Gem::Version.new("2.2.2")
 end
 
 
-task :default => :everything
+task :default => :test

data/lib/suture/adapter/dictaphone.rb

@@ -1,7 +1,9 @@
 require "suture/wrap/sqlite"
 require "suture/adapter/log"
 require "suture/value/observation"
+require "suture/value/result"
 require "suture/error/observation_conflict"
+require "suture/util/compares_results"
 
 module Suture::Adapter
   class Dictaphone
@@ -10,24 +12,20 @@ module Suture::Adapter
     def initialize(plan)
       @db = Suture::Wrap::Sqlite.init(plan.database_path)
       @name = plan.name
-      @comparator = plan.comparator
+      @compares_results = Suture::Util::ComparesResults.new(plan.comparator)
       if plan.respond_to?(:args) # does not apply to TestPlan objects
         @args_inspect = plan.args.inspect
         @args_dump = Marshal.dump(plan.args)
       end
     end
 
-    def record(result)
-      Suture::Wrap::Sqlite.insert(@db, :observations, [:name, :args, :result],
-                                  [@name.to_s, @args_dump, Marshal.dump(result)])
-      log_info("recorded call for seam #{@name.inspect} with args `#{@args_inspect}` and result `#{result.inspect}`")
-    rescue SQLite3::ConstraintException
-      old_observation = known_observation
-      if @comparator.call(old_observation.result, result)
-        log_debug("skipped recording of duplicate call for seam #{@name.inspect} with args `#{@args_inspect}` and result `#{result.inspect}`")
-      else
-        raise Suture::Error::ObservationConflict.new(@name, @args_inspect, result, old_observation)
-      end
+
+    def record(returned_value)
+      record_result(Suture::Value::Result.returned(returned_value))
+    end
+
+    def record_error(error)
+      record_result(Suture::Value::Result.errored(error))
     end
 
     def play(only_id = nil)
@@ -40,19 +38,42 @@ module Suture::Adapter
       rows.map { |row| row_to_observation(row) }
     end
 
-    def delete!(id)
+    def delete_by_id!(id)
       log_info("deleting call with ID: #{id}")
       Suture::Wrap::Sqlite.delete(@db, :observations, "where id = ?", [id])
     end
 
+    def delete_by_name!(name)
+      log_info("deleting calls for seam named: #{name}")
+      Suture::Wrap::Sqlite.delete(@db, :observations, "where name = ?", [name.to_s])
+    end
+
   private
 
+    def record_result(result)
+      Suture::Wrap::Sqlite.insert(
+        @db,
+        :observations,
+        [:name, :args, result.errored? ? :error : :result],
+        [@name.to_s, @args_dump, Marshal.dump(result.value)]
+      )
+      log_info("recorded call for seam #{@name.inspect} with args `#{@args_inspect}` and result `#{result.value.inspect}`")
+    rescue SQLite3::ConstraintException
+      old_observation = known_observation
+      if @compares_results.compare(old_observation.result, result)
+        log_debug("skipped recording of duplicate call for seam #{@name.inspect} with args `#{@args_inspect}` and result `#{result.value.inspect}`")
+      else
+        raise Suture::Error::ObservationConflict.new(@name, @args_inspect, result, old_observation)
+      end
+    end
+
     def row_to_observation(row)
       Suture::Value::Observation.new(
-        row[0],
-        row[1].to_sym,
-        Marshal.load(row[2]),
-        Marshal.load(row[3])
+        :id => row[0],
+        :name => row[1].to_sym,
+        :args => Marshal.load(row[2]),
+        :return => row[3] ? Marshal.load(row[3]) : nil,
+        :error => row[4] ? Marshal.load(row[4]) : nil
       )
     end
 

data/lib/suture/adapter/log.rb

@@ -27,5 +27,9 @@ module Suture::Adapter
     def log_warn(*args, &blk)
       Log.logger.warn(*args, &blk)
     end
+
+    def log_error(*args, &blk)
+      Log.logger.error(*args, &blk)
+    end
   end
 end

data/lib/suture/adapter/progress_bar.rb

@@ -0,0 +1,13 @@
+require "bar-of-progress"
+
+module Suture::Adapter
+  class ProgressBar
+    def initialize(attrs = {})
+      @options = {:length => 60}.merge(attrs)
+    end
+
+    def progress(numerator, denominator)
+      BarOfProgress.new(@options.merge(:total => denominator)).progress(numerator)
+    end
+  end
+end

data/lib/suture/config.rb

@@ -5,8 +5,9 @@ module Suture
     :database_path => "db/suture.sqlite3",
     :comparator => Comparator.new,
     :log_level => "INFO",
-    :log_file => nil,
     :log_stdout => true,
+    :log_io => nil,
+    :log_file => nil,
     :raise_on_result_mismatch => true
   }
 

data/lib/suture/create/chooses_surgeon.rb

@@ -21,7 +21,7 @@ module Suture
         Surgeon::Observer.new
       elsif plan.call_both
         Surgeon::Auditor.new
-      elsif plan.call_old_on_error
+      elsif plan.fallback_on_error
         Surgeon::Remediator.new
       else
         Surgeon::NoOp.new

data/lib/suture/create/validates_plan.rb

@@ -35,13 +35,13 @@ module Suture
         end
       },
       lambda { |plan|
-        if plan.record_calls && plan.call_old_on_error
-          ":record_calls & :call_old_on_error are both enabled and conflict with one another. :record_calls will only invoke the old code path (intended for characterization of the old code path and initial development of the new code path), whereas :call_old_on_error will call the new code path unless an error is raised, in which case it will fall back on the old code path."
+        if plan.record_calls && plan.fallback_on_error
+          ":record_calls & :fallback_on_error are both enabled and conflict with one another. :record_calls will only invoke the old code path (intended for characterization of the old code path and initial development of the new code path), whereas :fallback_on_error will call the new code path unless an error is raised, in which case it will fall back on the old code path."
         end
       },
       lambda { |plan|
-        if plan.call_both && plan.call_old_on_error
-          ":call_both & :call_old_on_error are both enabled and conflict with one another. :call_both is designed for pre-production environments and will call both the old and new code paths to compare their results, whereas :call_old_on_error is designed for production environments where it is safe to call the old code path in the event that the new code path fails unexpectedly"
+        if plan.call_both && plan.fallback_on_error
+          ":call_both & :fallback_on_error are both enabled and conflict with one another. :call_both is designed for pre-production environments and will call both the old and new code paths to compare their results, whereas :fallback_on_error is designed for production environments where it is safe to call the old code path in the event that the new code path fails unexpectedly"
         end
       },
       lambda { |plan|
@@ -50,8 +50,8 @@ module Suture
         end
       },
       lambda { |plan|
-        if plan.call_old_on_error && !plan.new.respond_to?(:call)
-          ":call_old_on_error is set but :new is either not set or is not callable. This mode is designed for after the :new code path has been developed and run in production-like environments, where :old is only kept around as a fallback to retry in the event that :new raises an unexpected error. Either specify a :new code path or disable :call_old_on_error."
+        if plan.fallback_on_error && !plan.new.respond_to?(:call)
+          ":fallback_on_error is set but :new is either not set or is not callable. This mode is designed for after the :new code path has been developed and run in production-like environments, where :old is only kept around as a fallback to retry in the event that :new raises an unexpected error. Either specify a :new code path or disable :fallback_on_error."
         end
       }
     ]

data/lib/suture/delete.rb

@@ -4,6 +4,11 @@ require "suture/adapter/dictaphone"
 module Suture
   def self.delete!(id, options = {})
     plan = BuildsPlan.new.build(:name_not_used_here, options)
-    Adapter::Dictaphone.new(plan).delete!(id)
+    Adapter::Dictaphone.new(plan).delete_by_id!(id)
+  end
+
+  def self.delete_all!(name, options = {})
+    plan = BuildsPlan.new.build(:name_not_used_here, options)
+    Adapter::Dictaphone.new(plan).delete_by_name!(name)
   end
 end

data/lib/suture/error/observation_conflict.rb

@@ -17,10 +17,10 @@ module Suture::Error
           #{@args_inspect}
         ```
         Previously-observed return value: ```
-          #{@old_result.inspect}
+          #{@old_result.value.inspect}
         ```
         Newly-observed return value: ```
-          #{@new_result.inspect}
+          #{@new_result.value.inspect}
         ```
 
         That's not good! Here are a few ideas of what may have happened:

data/lib/suture/error/verification_failed.rb

@@ -1,3 +1,5 @@
+require "suture/adapter/progress_bar"
+
 module Suture::Error
   class VerificationFailed < StandardError
     def initialize(plan, results)
@@ -10,8 +12,10 @@ module Suture::Error
         intro,
         describe_failures(@results.failed, @plan),
         configuration(@plan),
-        summarize(@results)
-      ].join("\n")
+        summarize(@results),
+        progress(@plan, @results)
+      ].compact.join("\n").tap do
+      end
     end
 
     private
@@ -37,6 +41,23 @@ module Suture::Error
       MSG
     end
 
+    def progress(plan, results)
+      return if plan.fail_fast
+      bar = Suture::Adapter::ProgressBar.new.progress(
+        results.passed.size,
+        results.all.size
+      )
+      <<-MSG.gsub(/^ {8}/,'')
+        ## Progress
+
+        Here's what your progress to initial completion looks like so far:
+
+        #{bar}
+
+        Of #{results.all.size} recorded interactions, #{results.passed.size} are currently passing.
+      MSG
+    end
+
     def configuration(plan)
       <<-MSG.gsub(/^ {8}/,'')
         # Configuration
@@ -88,24 +109,24 @@ module Suture::Error
       return <<-MSG.gsub(/^ {8}/,'')
         #{index + 1}.) Recorded call for seam #{expected.name.inspect} (ID: #{expected.id}) ran and #{failure[:error] ? "raised an error" : "failed comparison"}.
 
-           Arguments: ```
-             #{expected.args.inspect}
-           ```
-           Expected result: ```
-             #{expected.result.inspect}
-           ```
-           #{failure[:error] ? "Error raised" : "Actual result"}: ```
-             #{if failure[:error]
-                 stringify_error(failure[:error])
-               else
-                 failure[:new_result].inspect
-               end
-             }
-           ```
-
-           Ideas to fix this:
-             * Focus on this test by setting ENV var `SUTURE_VERIFY_ONLY=#{expected.id}`
-             * Is the recording wrong? Delete it! `Suture.delete!(#{expected.id})`
+          Arguments: ```
+            #{expected.args.inspect}
+          ```
+          Expected #{expected.result.errored? ? "error raised" : "returned value"}: ```
+            #{expected.result.value.inspect}
+          ```
+          Actual #{(failure[:error] || failure[:new_result].errored?) ? "error raised" : "returned value"}: ```
+            #{if failure[:error]
+                stringify_error(failure[:error])
+              else
+                failure[:new_result].value.inspect
+              end
+            }
+          ```
+
+          Ideas to fix this:
+            * Focus on this test by setting ENV var `SUTURE_VERIFY_ONLY=#{expected.id}`
+            * Is the recording wrong? Delete it! `Suture.delete!(#{expected.id})`
       MSG
     end
 

data/lib/suture/surgeon/no_op.rb

@@ -3,8 +3,19 @@ require "suture/util/scalpel"
 module Suture::Surgeon
   class NoOp
     def operate(plan)
-      return unless plan.old
-      Suture::Util::Scalpel.new.cut(plan, :old)
+      if code_path = code_path_for(plan)
+        Suture::Util::Scalpel.new.cut(plan, code_path)
+      end
+    end
+
+    private
+
+    def code_path_for(plan)
+      if plan.new && !plan.disable
+        :new
+      elsif plan.old
+        :old
+      end
     end
   end
 end

data/lib/suture/surgeon/observer.rb

@@ -5,8 +5,15 @@ module Suture::Surgeon
   class Observer
     def operate(plan)
       dictaphone = Suture::Adapter::Dictaphone.new(plan)
-      Suture::Util::Scalpel.new.cut(plan, :old).tap do |result|
-        dictaphone.record(result)
+      begin
+        Suture::Util::Scalpel.new.cut(plan, :old).tap do |result|
+          dictaphone.record(result)
+        end
+      rescue StandardError => error
+        if plan.expected_error_types.any? {|e| error.kind_of?(e) }
+          dictaphone.record_error(error)
+        end
+        raise error
       end
     end
   end

data/lib/suture/util/compares_results.rb

@@ -0,0 +1,18 @@
+module Suture::Util
+  class ComparesResults
+    def initialize(comparator)
+      @comparator = comparator
+    end
+
+    def compare(expected, actual)
+      if expected.errored? != actual.errored?
+        false
+      elsif expected.errored?
+        actual.value.kind_of?(expected.value.class) &&
+          expected.value.message == actual.value.message
+      else
+        @comparator.call(expected.value, actual.value)
+      end
+    end
+  end
+end

data/lib/suture/util/scalpel.rb

@@ -1,12 +1,17 @@
+require "suture/adapter/log"
+
 module Suture::Util
   class Scalpel
+    include Suture::Adapter::Log
+
     def cut(plan, location, args_override = nil)
-      args = args(plan, args_override)
+      args = args_for(plan, args_override)
       begin
         plan.send(location).call(*args).tap do |result|
           call_after_hook(plan, location, args, result)
         end
       rescue StandardError => error
+        log_error_details(plan, location, args, error)
         call_error_hook(plan, location, args, error)
         raise error
       end
@@ -16,16 +21,37 @@ module Suture::Util
 
     def call_after_hook(plan, location, args, result)
       return unless after_hook = try(plan, "after_#{location}")
-      after_hook.call(plan.name, args, result)
+      after_hook.call(args, result)
+    end
+
+    def log_error_details(plan, location, args, error)
+      return if expected_error?(plan, error)
+      log_error <<-MSG.gsub(/^ {8}/,'')
+        Suture invoked the #{plan.name.inspect} seam's #{location.inspect} code path with args: ```
+          #{args.inspect}
+        ```
+        which raised a #{error.class} with message: ```
+          #{error.message}
+        ```
+      MSG
     end
 
     def call_error_hook(plan, location, args, error)
-      return if plan.expected_error_types.any? {|e| error.kind_of?(e) }
+      return if expected_error?(plan, error)
       return unless error_hook = try(plan, "on_#{location}_error")
       error_hook.call(plan.name, args, error)
     end
 
-    def args(plan, args_override)
+    def expected_error?(plan, error)
+      plan.expected_error_types.any? {|e| error.kind_of?(e) }
+    end
+
+    def args_for(plan, args_override)
+      args = pick_args(plan, args_override)
+      try(plan, :dup_args) ? Marshal.load(Marshal.dump(args)) : args
+    end
+
+    def pick_args(plan, args_override)
       return args_override if args_override
       if plan.respond_to?(:args)
         plan.args

data/lib/suture/value/observation.rb

@@ -1,11 +1,19 @@
+require "suture/value/result"
+
 module Suture::Value
   class Observation
-    attr_reader :id, :name, :args, :result
-    def initialize(id, name, args, result)
-      @id = id
-      @name = name
-      @args = args
-      @result = result
+    attr_reader :id, :name, :args
+
+    def initialize(attrs)
+      @id = attrs[:id]
+      @name = attrs[:name]
+      @args = attrs[:args]
+      @return_value = attrs[:return]
+      @error = attrs[:error]
+    end
+
+    def result
+      @expectation ||= @error ? Result.errored(@error) : Result.returned(@return_value)
     end
   end
 end

data/lib/suture/value/plan.rb

@@ -2,8 +2,8 @@ module Suture::Value
   class Plan
     attr_reader :name, :old, :new, :args, :after_new, :after_old, :on_new_error,
                 :on_old_error, :database_path, :record_calls, :comparator,
-                :call_both, :raise_on_result_mismatch, :call_old_on_error,
-                :expected_error_types, :disable
+                :call_both, :raise_on_result_mismatch, :fallback_on_error,
+                :expected_error_types, :disable, :dup_args
 
     def initialize(attrs = {})
       @name = attrs[:name]
@@ -19,9 +19,10 @@ module Suture::Value
       @comparator = attrs[:comparator]
       @call_both = !!attrs[:call_both]
       @raise_on_result_mismatch = !!attrs[:raise_on_result_mismatch]
-      @call_old_on_error = !!attrs[:call_old_on_error]
+      @fallback_on_error = !!attrs[:fallback_on_error]
       @expected_error_types = attrs[:expected_error_types] || []
       @disable = !!attrs[:disable]
+      @dup_args = !!attrs[:dup_args]
     end
   end
 end

data/lib/suture/value/result.rb

@@ -0,0 +1,35 @@
+module Suture::Value
+  class Result
+    def self.errored(error)
+      new(:error, error)
+    end
+
+    def self.returned(return_value)
+      new(:return_value, return_value)
+    end
+
+    attr_reader :value
+    def initialize(result_type, value)
+      @value = value
+      @errored = result_type == :error
+    end
+
+    def errored?
+      @errored
+    end
+
+    def ==(other)
+      other.kind_of?(self.class) && other.state == state
+    end
+
+    def hash
+      state.hash
+    end
+
+    protected
+
+    def state
+      [@value, @errored]
+    end
+  end
+end

data/lib/suture/value/test_plan.rb

@@ -7,11 +7,12 @@ module Suture::Value
                   :expected_error_types
 
     def initialize(attrs = {})
-      assign_simple_ivars!(attrs, :name, :subject, :fail_fast, :comparator,
+      assign_simple_ivars!(attrs, :name, :subject, :comparator,
                                   :database_path, :after_subject,
                                   :on_subject_error)
       assign_integral_ivars!(attrs, :verify_only, :call_limit, :time_limit,
                                    :error_message_limit)
+      @fail_fast = !!attrs[:fail_fast]
       @expected_error_types = attrs[:expected_error_types] || []
       @random_seed = determine_random_seed(attrs)
     end

data/lib/suture/value/test_results.rb

@@ -1,7 +1,5 @@
 module Suture::Value
   class TestResults
-    attr_reader :results
-
     def initialize(results)
       @results = results
     end
@@ -22,6 +20,14 @@ module Suture::Value
       @results.count { |r| r[:passed] }
     end
 
+    def all
+      @results
+    end
+
+    def passed
+      @results.select { |r| r[:passed] }
+    end
+
     def failed
       @results.select { |r| !r[:passed] && r[:ran] }
     end

data/lib/suture/verify/administers_test.rb

@@ -0,0 +1,32 @@
+require "suture/util/compares_results"
+require "suture/util/scalpel"
+require "suture/value/result"
+
+module Suture
+  class AdministersTest
+    def initialize
+      @scalpel = Util::Scalpel.new
+    end
+
+    def administer(test_plan, observation)
+      compares_results = Util::ComparesResults.new(test_plan.comparator)
+      begin
+        result = Value::Result.returned(@scalpel.cut(test_plan, :subject, observation.args))
+        {
+          :new_result => result,
+          :passed => compares_results.compare(observation.result, result)
+        }
+      rescue StandardError => error
+        if observation.result.errored?
+          result = Value::Result.errored(error)
+          {
+            :new_result => result,
+            :passed => compares_results.compare(observation.result, result)
+          }
+        else
+          { :error => error, :passed => false }
+        end
+      end
+    end
+  end
+end

data/lib/suture/verify/tests_patient.rb

@@ -1,14 +1,17 @@
+require "suture/verify/administers_test"
+require "suture/adapter/log"
 require "suture/adapter/dictaphone"
 require "suture/value/test_results"
-require "suture/util/scalpel"
 require "suture/util/shuffle"
 require "suture/util/timer"
 require "backports/1.9.2/random"
 
 module Suture
   class TestsPatient
+    include Suture::Adapter::Log
+
     def initialize
-      @scalpel = Suture::Util::Scalpel.new
+      @administers_test = AdministersTest.new
     end
 
     def test(test_plan)
@@ -16,15 +19,13 @@ module Suture
       timer = Suture::Util::Timer.new(test_plan.time_limit) unless test_plan.time_limit.nil?
       test_cases = build_test_cases(test_plan)
       Value::TestResults.new(test_cases.each_with_index.map { |observation, i|
-        if (test_plan.fail_fast && experienced_failure_in_life) ||
-            (test_plan.call_limit && i >= test_plan.call_limit) ||
-            (timer && timer.time_up?)
+        if should_skip?(test_plan, experienced_failure_in_life, i, timer)
           {
             :observation => observation,
             :ran => false
           }
         else
-          invoke(test_plan, observation).merge({
+          @administers_test.administer(test_plan, observation).merge({
             :observation => observation,
             :ran => true
           }).tap { |r| experienced_failure_in_life = true unless r[:passed] }
@@ -34,30 +35,31 @@ module Suture
 
     private
 
+    def should_skip?(test_plan, failed_fast, call_count, timer)
+      (test_plan.fail_fast && failed_fast) ||
+        (test_plan.call_limit && call_count >= test_plan.call_limit) ||
+        (timer && timer.time_up?)
+    end
+
     def build_test_cases(test_plan)
       dictaphone = Suture::Adapter::Dictaphone.new(test_plan)
       shuffle(
         dictaphone.play(test_plan.verify_only),
         test_plan.random_seed
-      )
+      ).tap do |test_cases|
+        next if test_cases.size > 0
+        log_warn <<-MSG.gsub(/^ {10}/,'')
+          Suture.verify found no recorded calls for seam #{test_plan.name.inspect}.
+          As a result, verify will have no effect and cannot provide any assurance
+          that the subject is working as expected.
+        MSG
+      end
     end
 
     def shuffle(rows, random_seed)
       return rows unless random_seed
       Suture::Util::Shuffle.shuffle(Random.new(random_seed), rows)
     end
-
-    def invoke(test_plan, observation)
-      {}.tap do |result|
-        begin
-          result[:new_result] = @scalpel.cut(test_plan, :subject, observation.args)
-          result[:passed] = test_plan.comparator.call(observation.result, result[:new_result])
-        rescue StandardError => e
-          result[:passed] = false
-          result[:error] = e
-        end
-      end
-    end
   end
 end
 

data/lib/suture/version.rb

@@ -1,3 +1,3 @@
 module Suture
-  VERSION = "0.4.0"
+  VERSION = "0.5.0"
 end

data/lib/suture/wrap/logger.rb

@@ -3,14 +3,12 @@ require "logger"
 module Suture::Wrap
   module Logger
     def self.init(options)
-      if options[:log_file]
+      @logger = if options[:log_file]
         full_path = File.join(Dir.getwd, options[:log_file])
         FileUtils.mkdir_p(File.dirname(full_path))
-        @logger = ::Logger.new(full_path)
-      elsif options[:log_stdout]
-        @logger = ::Logger.new(STDOUT)
+        ::Logger.new(full_path)
       else
-        @logger = NullLogger.new
+        ::Logger.new(NullIO.new)
       end
 
       @logger.level = if options[:log_level]
@@ -22,19 +20,24 @@ module Suture::Wrap
       @logger.formatter = proc { |_, time , _, msg|
         formatted_time = time.strftime("%Y-%m-%dT%H:%M:%S.%6N")
         "[#{formatted_time}] Suture: #{msg}\n".tap { |out|
-          puts out if options[:log_file] && options[:log_stdout]
+          puts out if options[:log_stdout]
+          options[:log_io].write(out) if options[:log_io]
         }
       }
 
       return @logger
     end
 
-    class NullLogger
-      def formatter=(*args); end
-      def level=(*args); end
-      def debug(*args); end
-      def info(*args); end
-      def warn(*args); end
+    class NullIO
+      def gets; end
+      def each; end
+      def read(count=nil,buffer=nil); (count && count > 0) ? nil : ""; end
+      def rewind; 0; end
+      def close; end
+      def size; 0; end
+      def sync=(*args); end
+      def puts(*args); end
+      def write(*args); end
     end
   end
 end

data/lib/suture/wrap/sqlite.rb

@@ -4,18 +4,18 @@ require "suture/error/schema_version"
 
 module Suture::Wrap
   module Sqlite
-    SCHEMA_VERSION=1
+    SCHEMA_VERSION=2
     def self.init(location)
       full_path = File.join(Dir.getwd, location)
       FileUtils.mkdir_p(File.dirname(full_path))
       SQLite3::Database.new(full_path).tap do |db|
         db.execute <<-SQL
-          create table if not exists schema_info (
+          create table if not exists suture_schema_info (
             version integer unique
           );
         SQL
-        db.execute("insert or ignore into schema_info values (?)", [SCHEMA_VERSION])
-        actual_schema_version = db.execute("select * from schema_info").first[0]
+        db.execute("insert or ignore into suture_schema_info values (?)", [SCHEMA_VERSION])
+        actual_schema_version = db.execute("select * from suture_schema_info").first[0]
         if SCHEMA_VERSION != actual_schema_version
           raise Suture::Error::SchemaVersion.new(SCHEMA_VERSION, actual_schema_version)
         end
@@ -23,9 +23,10 @@ module Suture::Wrap
         db.execute <<-SQL
           create table if not exists observations (
             id integer primary key,
-            name varchar(255),
-            args clob,
+            name varchar(255) not null,
+            args clob not null,
             result clob,
+            error clob,
             unique(name, args)
           );
         SQL

data/suture.gemspec

@@ -13,13 +13,14 @@ Gem::Specification.new do |spec|
   spec.description   = %q{Provides tools to record calls to legacy code and verify new implementations still work}
   spec.homepage      = "https://github.com/testdouble/suture"
 
-  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|db|log|safe|spec|features)/}) }
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(example|test|db|log|safe|spec|features)/}) }
   spec.bindir        = "exe"
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
   spec.add_dependency "sqlite3"
   spec.add_dependency "backports"
+  spec.add_dependency "bar-of-progress", ">= 0.1.3"
 
   spec.add_development_dependency "bundler", "~> 1.12"
   spec.add_development_dependency "rake", "~> 10.0"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: suture
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.5.0
 platform: ruby
 authors:
 - Justin Searls
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-08-29 00:00:00.000000000 Z
+date: 2016-09-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sqlite3
@@ -38,6 +38,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: bar-of-progress
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.1.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.1.3
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -173,6 +187,7 @@ files:
 - lib/suture.rb
 - lib/suture/adapter/dictaphone.rb
 - lib/suture/adapter/log.rb
+- lib/suture/adapter/progress_bar.rb
 - lib/suture/comparator.rb
 - lib/suture/config.rb
 - lib/suture/create.rb
@@ -191,15 +206,18 @@ files:
 - lib/suture/surgeon/no_op.rb
 - lib/suture/surgeon/observer.rb
 - lib/suture/surgeon/remediator.rb
+- lib/suture/util/compares_results.rb
 - lib/suture/util/env.rb
 - lib/suture/util/scalpel.rb
 - lib/suture/util/shuffle.rb
 - lib/suture/util/timer.rb
 - lib/suture/value/observation.rb
 - lib/suture/value/plan.rb
+- lib/suture/value/result.rb
 - lib/suture/value/test_plan.rb
 - lib/suture/value/test_results.rb
 - lib/suture/verify.rb
+- lib/suture/verify/administers_test.rb
 - lib/suture/verify/interprets_results.rb
 - lib/suture/verify/prescribes_test_plan.rb
 - lib/suture/verify/tests_patient.rb