regtest 2.0.0.pre → 2.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 9a917267b567e320d9f372ad942f4602282c42b4
-  data.tar.gz: d2e5e0da3b36b4190db29449c188b31ff6072f23
+SHA256:
+  metadata.gz: b91c5f1f5b30d309f77eaf7c3c218529bd82321a7af578b37c52a4bed09d53e4
+  data.tar.gz: f66632f3eb0f2844ab43068f83c30e62aca2805dad460543436b9df0620c3598
 SHA512:
-  metadata.gz: 2b439502a95aa227425959e0de2be46d09d71233a3fc70b62a01220650464584d5a8ff1498ffcd41b9197679ad1b69e441d477b91a648a0ccb1910ec28b9cd5c
-  data.tar.gz: 6e3b8462a54c051a5d1ae795572c6ba5871f07df53359c21c34c2eada095b23eab01fc0176e1afa9b17bc6efcdeca48fb9c8e2bddeca03b9911b91cc5b150528
+  metadata.gz: 38257004b8aab5bbad388e430b85362e80b5759be4ca1729756ce0e2704a6cc85f12e3899c17b28df32afc1380251228321f750fb7f8bc4bd951a56e1ca27690
+  data.tar.gz: 2a2f5f808848caccb56db4a4d90a9e34babf70a79d30bf49182317c77081985bcae89289c77acab5ba4a029483330ba3533a37d91be6fbdfc7b63b3cfd397948

data/.aspell.pws

@@ -1,25 +1,38 @@
-personal_ws-1.1 en 24 
-NOREGTESTRC
-dir
+personal_ws-1.1 en 26 
 Bugfix
-ostruct
 ENV
-filename
+Friedrich
 Gemfile
+NOREGTESTRC
+README
+Regtest
+SCM
+SemVer
+SemVerTag
+YAML
 basename
-Rakefile
+bitbucket
+dir
 exiftool
-yml
-yaml
-workflow
-refactoring
+gettime
+github
+md
+metatest
 metatests
-catched
-regtest
 rb
 rc
-Friedrich
-SCM
+regtest
 regtestrc
-metatest
+scripref
+yml
+arg
+Matz
+ary
+Matz
 OpenStruct
+Rakefile
+arg
+ary
+catched
+ostruct
+yaml

data/Changelog

@@ -1,4 +1,24 @@
-2.0.0.pre Major Release
+2.2.1
+Adapt git output to work with different git versions.
+Simplify determination of filenames for outputs.
+Add a test and Fix documentation.
+
+2.2.0
+Add a new method Regtest.log to allow the logging of temporary values which
+could be of interest for different reasons (i. e. debugging).
+With this version you can use `extend Regtest` to have the methods sample, log
+and combinations as top level methods. This is experimental and therefore not
+yet mentioned at README.
+
+2.1.1
+Adapt remaining examples of regtest/colorize to regtest/colors.
+
+2.1.0
+Use Process.clock_gettime for time measurements.
+Eliminate dependency to colorize gem. So Regtest has no longer any
+dependency.
+
+2.0.0 Major Release
 
 Incompatibilities to regtest 1.x
 1. The methods sample and combinations are not longer defined as instance
@@ -15,13 +35,18 @@ will not be loaded.
 A simple plugin mechanism is implemented and two plugins are part of regtest:
 regtest/colorize and regtest/git. Plugins are usual activated by require in
 one of the .regtestrc files.
+It is possible to change the exit codes for different return states of
+Regtest.check_results. This is done with changing Regtest.exit_codes see
+README.md for an example.
 
 Further Changes
-Reporting of statistics for writing files is removed because it's not useful.
-Now the full filename for results files are reported when running regtest.
+Reporting of statistics was reduced and the full filename for results files
+are now reported when running regtest.
 Some metatests are added.
 Version constant Regtest::VERSION is also added.
-A lot of internal refactoring and improving documentation.
+Regtest.sample returns now the hash with result information. This could be
+useful for plugins which overload this method.
+A lot of internal refactoring and improving documentation was done.
 
 
 1.1.0

data/LICENSE

@@ -1,3 +1,3 @@
-Copyright 2014-2017 by Jan Friedrich <janfri26@gmail.com>
+Copyright 2014, 2015, 2017-2019 by Jan Friedrich <janfri26@gmail.com>
 
 Regtest is licensed under the same terms as Ruby itself.

data/README.md

@@ -146,6 +146,79 @@ older runs of the samples is done by your SCM. So the sample files and their
 corresponding results files should be taken under version control.
 
 
+## Logging
+
+The key idea behind regtest is to produce values that are invariant and check
+if this assumption is true at another (mostly later) state of code. But often
+there are temporary or specific values which changes or could change at each
+run of regtest. This could be for example an id for a created record or the
+version of a used external service or some time-relevant values. Sometimes it
+is useful, to know the actual value of one of these.
+
+In such cases the method ```Regtest.log``` could be handy. It writes a line of
+the given object to a log file which is named with the same name as the calling
+Ruby script but has as extension ```.log```. It could be called inside as well
+as outside of a regtest sample. Per default this file is overwritten by the
+first call of ```Regtest.log``` of each run of regtest and per file. And each
+further call of ```Regtest.log``` appends then to the file.  So you get a
+complete log for each run. But this behaviour could be changed with the
+```mode``` keyword arg of the method. Let's see an example:
+
+```ruby
+Regtest.log RUBY_VERSION
+
+def create_record **values
+  # do stuff to create the record and return the id of the generated record
+end
+
+def delete_record id
+  # delete an existing record or raise an exception if record with id doesn't
+  # exists
+end
+
+Regtest.sample 'create a record and delete it' do
+  id = create_record language: 'Ruby', creator: 'Matz'
+  Regtest.log "generated record with id #{id}"
+  delete_record id
+  # the generated id should not be part of the result, because it changes every
+  # time you create a new record
+  'Record created and deleted'
+end
+
+Regtest.sample 'try to delete a non existing record' do
+  delete_record -1
+end
+```
+
+If you want to have a log that is not truncated at each run of regtest, you can
+use ```mode: 'a'```at the first call of ```Regtest.log``` in the corresponding
+ruby script.
+
+```ruby
+# ...
+# the first call of Regtest.log in this Ruby file
+Regtest.log Time.now, mode: 'a'
+```
+
+On the other hand, you can use ```mode: 'w'``` to truncate the log file even at
+a later call of ```Regtest.log```.
+
+```ruby
+max_time = 0
+ary.each do |e|
+  t = get_time_to_do_some_stuff
+  if t > max_time
+    max_time = t
+    Regtest.log max_time, mode: 'w'
+    # the content of the corresponding log file is now one line with max_time
+  end
+end
+```
+
+Because the log files contains only temporary stuff they should normally not
+checked in the SCM.
+
+
 ## Configuration and Plugins
 
 You can adapt the behaviour of regtest with plugins. To configure this and
@@ -159,10 +232,10 @@ For example the following is a good default when you want colorized output and
 use git as your SCM:
 
 ```ruby
-require 'regtest/colorize'
+require 'regtest/colors'
 # adapt some colorizing if wanted
-Regtest::Colorize.mapping[:filename] = :blue
-Regtest::Colorize.mapping[:statistics] = {mode: :italic}
+Regtest::Colors.mapping[:filename] = :cyan
+Regtest::Colors.mapping[:statistics] = %i(blue italic)
 
 require 'regtest/git'
 ```
@@ -174,7 +247,8 @@ script is accordingly set. The standard values are: 0 for success, 1 for an
 unknown result (normally a new results file) and 2 for failure. If you use
 plain regtest without a SCM plugin the exit code is 1 (= unknown result).
 
-You can change the exit codes for the states with `Regtest.exit_codes` for example:
+You can change the exit codes for the states with `Regtest.exit_codes`. The
+following example changes the behaviour to the same as in regtest version 1.x.
 
 ```ruby
 Regtest.exit_codes[:unknown_result] = 0 
@@ -224,6 +298,13 @@ The code is hosted on [github](https://github.com/janfri/regtest) and
 [bitbucket](https://bitbucket.org/janfri/regtest) Change it to your needs.
 Release a fork. It is open source.
 
+
+## Versioning
+
+Regtest follows [Semantic Versioning](https://semver.org/), both SemVer and
+SemVerTag.
+
+
 ## Author
 
 Jan Friedrich <janfri26@gmail.com>

data/Rakefile

@@ -34,7 +34,12 @@ Rim.setup do |p|
   END
   p.homepage = 'https://github.com/janfri/regtest'
   p.ruby_version = '>=2.1.0'
-  p.dependencies << %w(colorize ~>0.8)
+end
+
+# JRuby does not support escaping of filenames with spaces in Open3.capture3
+# therefore ignore metatest files when running on JRuby
+if RUBY_ENGINE == 'jruby'
+  REGTEST_FILES_RB.reject! {|fn| fn =~ /metatest/}
 end
 
 task :test => :regtest

data/lib/regtest.rb

@@ -3,83 +3,112 @@
 #
 # Regtest - Simple Regression Testing With Ruby
 #
-# Copyright 2014, 2015, 2017 by Jan Friedrich <janfri26@gmail.com>
+# Copyright 2014, 2015, 2017-2019 by Jan Friedrich <janfri26@gmail.com>
 # License: Regtest is licensed under the same terms as Ruby itself.
 #
 
 require 'ostruct'
 require 'regtest/version'
+require 'set'
 require 'yaml'
 
 module Regtest
 
-  @start = Time.now
+  extend self
+
+  @start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
   @results = {}
-  @statistics = []
+  @log_filenames = Set.new
   @exit_codes = Hash.new(1)
   @exit_codes.merge!({success: 0, unknown_result: 1, fail: 2})
 
-  class << self
+  # Define a sample
+  def sample name
+    h = {}
+    name = name.to_s if name.kind_of?(Symbol)
+    h['sample'] = name
+    begin
+      h['result'] = yield
+    rescue Exception => e
+      h['exception'] = e.message
+    end
+    output_filename = Regtest.determine_filename_from_caller('.yml')
+    unless Regtest.results[output_filename]
+      Regtest.report "\n", type: :filename unless Regtest.results.empty?
+      Regtest.report output_filename, type: :filename
+      Regtest.results[output_filename] = []
+    end
+    Regtest.results[output_filename] << h
+    print '.'; $stdout.flush
+    h
+  end
 
-    attr_reader :exit_codes, :results, :statistics, :start
-
-    # Define a sample
-    def sample name, &_
-      start = Time.now
-      h = {}
-      name = name.to_s if name.kind_of?(Symbol)
-      h['sample'] = name
-      begin
-        h['result'] = yield
-      rescue Exception => e
-        h['exception'] = e.message
-      end
-      output_filename = caller.first.split(/:\d+:/).first.sub(/\.rb/, '') << '.yml'
-      unless Regtest.results[output_filename]
-        Regtest.report "\n", type: :filename unless Regtest.results.empty?
-        Regtest.report output_filename, type: :filename
-        Regtest.results[output_filename] = []
+  # Build all combinations of a Hash-like object with arrays as values.
+  # Return value is an array of OpenStruct instances.
+  #
+  # Example:
+  #   require 'ostruct'
+  #   require 'regtest'
+  #
+  #   o = OpenStruct.new
+  #   o.a = [1,2,3]
+  #   o.b = [:x, :y]
+  #   Regtest.combinations(o)
+  #   # => [#<OpenStruct a=1, b=:x>, #<OpenStruct a=1, b=:y>,
+  #   #     #<OpenStruct a=2, b=:x>, #<OpenStruct a=2, b=:y>,
+  #   #     #<OpenStruct a=3, b=:x>, #<OpenStruct a=3, b=:y>]
+  def combinations hashy
+    h = hashy.to_h
+    a = h.values[0].product(*h.values[1..-1])
+    res = []
+    a.each do |e|
+      o = OpenStruct.new
+      h.keys.zip(e) do |k, v|
+        o[k] = v
       end
-      Regtest.results[output_filename] << h
-      print '.'; $stdout.flush
-      stop = Time.now
-      Regtest.statistics << OpenStruct.new(filename: output_filename, sample: name, time: stop - start)
-      name
+      res << o
     end
+    res
+  end
 
-    # Build all combinations of a Hash-like object with arrays as values.
-    # Return value is an array of OpenStruct instances.
-    #
-    # Example:
-    #   require 'ostruct'
-    #   require 'regtest'
-    #
-    #   o = OpenStruct.new
-    #   o.a = [1,2,3]
-    #   o.b = [:x, :y]
-    #   Regtest.combinations(o)
-    #   # => [#<OpenStruct a=1, b=:x>, #<OpenStruct a=1, b=:y>,
-    #   #     #<OpenStruct a=2, b=:x>, #<OpenStruct a=2, b=:y>,
-    #   #     #<OpenStruct a=3, b=:x>, #<OpenStruct a=3, b=:y>]
-    def combinations hashy
-      h = hashy.to_h
-      a = h.values[0].product(*h.values[1..-1])
-      res = []
-      a.each do |e|
-        o = OpenStruct.new
-        h.keys.zip(e) do |k, v|
-          o[k] = v
-        end
-        res << o
-      end
-      res
+  # Write (temporary) informations to a log file
+  # By default the log file is truncated at the first call of Regtest.log for
+  # each run of regtest, and all following calls appends to the log file. So
+  # you have a log for one run of regtest. You can use mode ('a' or 'w') to
+  # change this behaviour for each call of Regtest.log.
+  def log s, mode: nil
+    log_filename = Regtest.determine_filename_from_caller('.log')
+    case mode
+    when nil
+      mode = Regtest.log_filenames.include?(log_filename) ? 'a' : 'w'
+    when 'a', 'w'
+      # ok
+    else
+      raise ArgumentError.new(format('Mode %s is not allowed.', mode))
+    end
+    Regtest.log_filenames << log_filename
+    File.open log_filename, mode do |f|
+      f.puts s
+    end
+  end
+
+  class << self
+
+    attr_reader :exit_codes, :log_filenames, :results, :start
+
+    # Determine a filename which is derived from the filename of the "real"
+    # caller of the calling method
+    # @param ext new extension (i.e. '.yml')
+    def determine_filename_from_caller ext
+      caller_locations(2, 1).first.path.sub(/\.rb$/, '') << ext.to_s
     end
 
     # Report some statistics, could be overwritten by plugins.
     def report_statistics
-      time = Time.now - start
-      sample_count = statistics.size
-      report format("\n\n%d samples executed in %.2f s (%.2f samples/s)", sample_count, time, sample_count / time), type: :statistics
+      now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      time = now - start
+      sample_count = results.values.map(&:size).reduce(0, &:+)
+      report format("\n\n%d samples executed in %.2f s (%d samples/s)", sample_count, time, sample_count / time), type: :statistics
     end
 
     # Save all results to the corresponding files.
@@ -102,7 +131,7 @@ module Regtest
     end
 
     # Report text to output with possible type, could be overwritten
-    # by plugins e.g. regtest/colorize.
+    # by plugins e.g. regtest/colors.
     def report *args, type: nil
       puts *args
     end

data/lib/regtest/colorize.rb

@@ -1,47 +1,8 @@
 # encoding: utf-8
 # frozen_string_literal: true
 
-begin
-  module Regtest
+require 'regtest/colors'
 
-    # Regtest plugin to have colorized reports.
-    # It uses the great colorize gem
-    # (see https://rubygems.org/gems/colorize)
-    module Colorize
+Regtest::Colorize = Regtest::Colors
 
-      @mapping = (%i(success fail unknown_result filename statistics).zip %i(green red yellow cyan default)).to_h
-
-      class << self
-        # Color mapping (all parameters of ColorizedString.colorize are supported
-        # (see https://github.com/fazibear/colorize#usage)
-        # Examples:
-        #   Regtest::Colorize.mapping[:filename] = :light_blue
-        #   Regtest::Colorize.mapping[:statistics] = {color: :cyan, mode: :italic}
-        attr_accessor :mapping
-      end
-
-      require 'colorized_string'
-      # If loading of colorized_string (gem colorize) fails the accessor above is
-      # available in .regtestrc files but method redefinition of Regtest.report and
-      # prepending of Colorize is skipped.
-
-      # Redefine Regtest.report.
-      def report *args, type: nil
-        color = Colorize.mapping[type]
-        if color && $stdout.tty?
-          args = args.map {|a| ColorizedString[a.to_s].colorize(color)}
-        end
-        puts *args
-      end
-
-    end
-
-    class << self
-      prepend Colorize
-    end
-
-  end
-rescue LoadError
-  warn 'gem colorize not found'
-  warn 'plugin regtest/colorize disabled'
-end
+warn 'regtest/colorize is deprecated. use regtest/colors instead.'