regtest 1.1.0 → 2.0.0.pre

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 0e25e74bc6bf4d1abf0c84fe8f7602b0bc355385
-  data.tar.gz: 80178a4963af2aebd6cb87d9c9e57fc7dc359dd8
+  metadata.gz: 9a917267b567e320d9f372ad942f4602282c42b4
+  data.tar.gz: d2e5e0da3b36b4190db29449c188b31ff6072f23
 SHA512:
-  metadata.gz: 32e29f778fa7128b743fab14bd29834d9a52c2f226e13a46d473e538c64962965df1ef9b622bf35462a60c4e568cf6d88b6aa282c34302debc05b5af87058e87
-  data.tar.gz: 1dfba5fe46becc517109a06670ccebcbabb930fc032570e67f34195649495a3d7b0f32f329887e8e7ed92dc4d492da507df275bcd4f13b8a4351f63b4dedb41e
+  metadata.gz: 2b439502a95aa227425959e0de2be46d09d71233a3fc70b62a01220650464584d5a8ff1498ffcd41b9197679ad1b69e441d477b91a648a0ccb1910ec28b9cd5c
+  data.tar.gz: 6e3b8462a54c051a5d1ae795572c6ba5871f07df53359c21c34c2eada095b23eab01fc0176e1afa9b17bc6efcdeca48fb9c8e2bddeca03b9911b91cc5b150528

data/.aspell.pws

@@ -0,0 +1,25 @@
+personal_ws-1.1 en 24 
+NOREGTESTRC
+dir
+Bugfix
+ostruct
+ENV
+filename
+Gemfile
+basename
+Rakefile
+exiftool
+yml
+yaml
+workflow
+refactoring
+metatests
+catched
+regtest
+rb
+rc
+Friedrich
+SCM
+regtestrc
+metatest
+OpenStruct

data/Changelog

@@ -1,3 +1,29 @@
+2.0.0.pre Major Release
+
+Incompatibilities to regtest 1.x
+1. The methods sample and combinations are not longer defined as instance
+methods but as as module methods only. Therefore you can't use it as mixed in
+methods via include.
+2. The exit code of regtest is now by default 1 if there are no plugins used which
+check the results like regtest/git.
+3. Regtest needs Ruby version 2.1 or higher.
+
+New Features
+Regtest can now be customized with a .regtestrc file in the home directory
+and/or actual directory. If ENV['NOREGTESTRC'] is set then .regtestrc files
+will not be loaded.
+A simple plugin mechanism is implemented and two plugins are part of regtest:
+regtest/colorize and regtest/git. Plugins are usual activated by require in
+one of the .regtestrc files.
+
+Further Changes
+Reporting of statistics for writing files is removed because it's not useful.
+Now the full filename for results files are reported when running regtest.
+Some metatests are added.
+Version constant Regtest::VERSION is also added.
+A lot of internal refactoring and improving documentation.
+
+
 1.1.0
 Remove output of statistics for writing files: It's not useful.
 Report filename of generated results not only the basename of it.

data/README.md

@@ -1,13 +1,16 @@
-# Regtest - Simple Regression Testing For Ruby Projects
+# Regtest - Simple Regression Testing With Ruby
 
 ## Description
 
-This library support a very simple way to do regression testing in Ruby
-projects.  You write Ruby scripts with samples. Run these and get the sample
-results as YAML output besides your scripts. Check both the scripts and the YAML
-files with the results in you Source Code Control System.  When you run the
-scrips on a later (or even previous) version of your code a simple diff show you
-if and how the changes in your code impact the results of your samples.
+This library supports a very simple way to do regression testing with Ruby. It
+is not limited to Ruby projects you can use it also in other contexts where you
+can extract data with Ruby.
+
+You write Ruby scripts with samples. Run these and get the sample results as
+results files besides your scripts. Check both the scripts and the results
+files in you Source Code Management System (SCM). When you run the scrips on a
+later (or even previous) version of your code a simple diff show you if and how
+the changes in your code or environment impact the results of your samples.
 
 This is not a replacement for unit testing but a complement: You can produce a
 lot of samples with a small amount of Ruby code (e.g. a large number of
@@ -27,7 +30,7 @@ or add
 ```ruby
 gem 'regtest'
 ```
-to your Gemfile or put a copy of `regtest.rb` in your project.
+to your Gemfile.
 
 
 ## Using
@@ -35,9 +38,11 @@ to your Gemfile or put a copy of `regtest.rb` in your project.
 The idea behind regtest is the following workflow:
   1. Writing samples
   2. Running samples
-  3. Checking results (differences between the actual results and the results of
-     a previous run of your samples using the diff functionality of your Source
-     Code Control System)
+  3. Commit samples and result files to SCM.
+  4. Change your code and / or external environment.
+  5. Rerun your samples
+  6. Check sample results for changes (this is normally automatically done with
+     a regtest plugin like regtest/git).
 
 
 ### Writing Samples
@@ -62,15 +67,13 @@ end
 ```
 
 The name of the sample (parameter of the `Regtest.sample` method) and the
-results of the samples (return value of the block) are stored in YAML format. So
-it should be a YAML friendly value as `String`, `Number`, `Boolean value`,
-`Symbol`.  Results could also be an `Array` or `Hash` with such values.
-
-
-#### Helpers
+results of the samples (return value of the block) are stored in YAML format.
+So it should be a YAML-friendly value as `String`, `Number`, `Boolean value`,
+`Symbol`. Results could also be an `Array` or `Hash` with such values.
 
-There is a method `Regtest.combinations` to generate a lot of combinations the
-easy way. An example:
+In many cases you want to generate a lot of combinations of input data in your
+sample code. For this there is a method `Regtest.combinations` to generate a
+lot of combinations the easy way. An example:
 
 ```ruby
 require 'ostruct'
@@ -79,23 +82,14 @@ require 'regtest'
 o = OpenStruct.new
 o.a = [1,2,3]
 o.b = [:x, :y]
-Regtest.combinations(o).map(&:to_h)
-# => [{:a=>1, :b=>:x}, {:a=>1, :b=>:y}, {:a=>2, :b=>:x}, {:a=>2, :b=>:y}, {:a=>3, :b=>:x}, {:a=>3, :b=>:y}]
+Regtest.combinations(o)
+# => [#<OpenStruct a=1, b=:x>, #<OpenStruct a=1, b=:y>,
+#     #<OpenStruct a=2, b=:x>, #<OpenStruct a=2, b=:y>,
+#     #<OpenStruct a=3, b=:x>, #<OpenStruct a=3, b=:y>] 
 ```
 
 See also the combinations example in the `regtest` folder.
 
-You can also include `Regtest` to have the `sample` and `combinations method at top level.
-
-```ruby
-require 'regtest'
-include Regtest
-
-sample :x do
-  :x
-end
-```
-
 By convention sample files are stored in a directory `regtest` in your Ruby application.
 
 
@@ -119,7 +113,7 @@ to your `Rakefile` and you can run your samples with `rake regtest`.
 ### Checking Results
 
 The results of each samples file are stored as a collection of YAML documents in
-a corresponding results file (YAML) per samples file.  For example for the
+a corresponding results file per samples file. For example for the
 samples files
 
 ```sh
@@ -145,14 +139,69 @@ sample: Division by zero
 exception: divided by 0
 ```
 
-Note: Each sample is represented by a YAML document in the corresponding YAML
-file.
-
-Each time you run one ore more samples file the corresponding results file will
+Each time you run one ore more samples file the corresponding results files will
 be overwritten (or generated if not yet existent) with the actual result values
-of your samples.  So Source Code Version Control program is the tool to
-determine changes between older runs of the samples.  Therefore the samples file
-and their corresponding results files should be taken under version control.
+of your samples. The determination of changes between the results of actual and
+older runs of the samples is done by your SCM. So the sample files and their
+corresponding results files should be taken under version control.
+
+
+## Configuration and Plugins
+
+You can adapt the behaviour of regtest with plugins. To configure this and
+maybe other things regtest support a simple rc file mechanism. While loading
+regtest via `require 'regtest'` it looks for a file `.regtestrc` first in your
+home directory and then in the local directory. So you can do global
+configurations in the first one and project specific configurations in the
+latter.
+
+For example the following is a good default when you want colorized output and
+use git as your SCM:
+
+```ruby
+require 'regtest/colorize'
+# adapt some colorizing if wanted
+Regtest::Colorize.mapping[:filename] = :blue
+Regtest::Colorize.mapping[:statistics] = {mode: :italic}
+
+require 'regtest/git'
+```
+
+Normally the check of changes in results is done automatically by a regtest
+plugin like regtest/git (see example for `.regtestrc` above). In this case the
+report will show you if there are changes or not and the exit code of the
+script is accordingly set. The standard values are: 0 for success, 1 for an
+unknown result (normally a new results file) and 2 for failure. If you use
+plain regtest without a SCM plugin the exit code is 1 (= unknown result).
+
+You can change the exit codes for the states with `Regtest.exit_codes` for example:
+
+```ruby
+Regtest.exit_codes[:unknown_result] = 0 
+Regtest.exit_codes[:fail] = 0 
+```
+
+This also should be don in a `.regtest` file and not in the sample files.
+
+Because in a `.regtestrc` file are individual configuration aspects of your
+workflow and environment it should not be checked into your SCM.
+
+## Rake task
+
+Regtest includes a Rake task `regtest`. Per default it runs any `.rb` files
+under the `regtest` directory and includes all files under the `regtest`
+directory to the files packaged with gem.
+You can change these defaults like this
+
+```ruby
+require 'regtest/task'
+
+REGTEST_FILES_RB.clear << 'my_regtest_file.rb'
+REGTEST_FILES.clear << 'my_regtest_file.rb' << 'my_regtest_file.yml' << 'other_file'
+```
+
+It's a little bit old school like `CLEAN` and `CLOBBER` but I like the simple
+approach to use constants.
 
 
 ## Further information

data/Rakefile

@@ -1,35 +1,41 @@
 # encoding: utf-8
 # frozen_string_literal: true
 
+$:.unshift 'lib'
+
 require 'rim/tire'
 require 'rim/regtest'
 require 'rim/version'
 
-$:.unshift 'lib'
 require 'regtest/task'
+require 'regtest/version'
 
 Rim.setup do |p|
   p.name = 'regtest'
-  p.version = '1.1.0'
+  p.version = Regtest::VERSION
   p.authors = 'Jan Friedrich'
   p.email = 'janfri26@gmail.com'
-  p.summary = 'Simple regression testing for Ruby projects.'
+  p.summary = 'Simple regression testing with Ruby.'
   p.license = 'Ruby'
   p.description = <<-END.gsub(/^ +/, '')
-    This library support a very simple way to do regression testing in Ruby
-    projects. You write Ruby scripts with samples. Run these and get the sample
-    results as YAML output besides your scripts. Check both the scripts and the YAML
-    files with the results in you Source Code Control System. When you run the
-    scrips on a later (or even previous) version of your code a simple diff show you
-    if and how the changes in your code impact the results of your samples.
+    This library supports a very simple way to do regression testing with Ruby. It
+    is not limited to Ruby projects you can use it also in other contexts where you
+    can extract data with Ruby.
+
+    You write Ruby scripts with samples. Run these and get the sample results as
+    results files besides your scripts. Check both the scripts and the results
+    files in you Source Code Management System (SCM). When you run the scrips on a
+    later (or even previous) version of your code a simple diff show you if and how
+    the changes in your code or environment impact the results of your samples.
 
     This is not a replacement for unit testing but a complement: You can produce a
     lot of samples with a small amount of Ruby code (e.g. a large number of
     combinations of data).
   END
   p.homepage = 'https://github.com/janfri/regtest'
+  p.ruby_version = '>=2.1.0'
+  p.dependencies << %w(colorize ~>0.8)
 end
 
-task :test do
-  # nothing
-end
+task :test => :regtest
+task :default => :test

data/lib/regtest.rb

@@ -1,73 +1,88 @@
 # encoding: utf-8
 # frozen_string_literal: true
 #
-# Regtest - Simple Regression Testing For Ruby Projects
+# Regtest - Simple Regression Testing With Ruby
 #
 # Copyright 2014, 2015, 2017 by Jan Friedrich <janfri26@gmail.com>
 # License: Regtest is licensed under the same terms as Ruby itself.
 #
 
 require 'ostruct'
+require 'regtest/version'
 require 'yaml'
 
 module Regtest
 
-  @count = 0
-  @results = {}
   @start = Time.now
+  @results = {}
+  @statistics = []
+  @exit_codes = Hash.new(1)
+  @exit_codes.merge!({success: 0, unknown_result: 1, fail: 2})
 
-  def sample name
-    h = {}
-    name = name.to_s if name.kind_of?(Symbol)
-    h['sample'] = name
-    begin
-      h['result'] = yield
-    rescue Exception => e
-      h['exception'] = e.message
-    end
-    output_filename = caller.first.split(/:/).first.sub(/\.rb/, '') << '.yml'
-    unless Regtest.results[output_filename]
-      puts unless Regtest.results.empty?
-      puts output_filename
-      Regtest.results[output_filename] = []
-    end
-    Regtest.results[output_filename] << h
-    Regtest.count += 1
-    print '.'; $stdout.flush
-  end
+  class << self
+
+    attr_reader :exit_codes, :results, :statistics, :start
 
-  # Build all combinations of a Hash-like object with arrays as values.
-  # Return value is an array of OpenStruct instances.
-  #
-  # Example:
-  #   require 'ostruct'
-  #   o = OpenStruct.new
-  #   o.a = [1,2,3]
-  #   o.b = [:x, :y]
-  #   Regtest.combinations(o).map(&:to_h) # => [{:a=>1, :b=>:x}, {:a=>1, :b=>:y}, {:a=>2, :b=>:x}, {:a=>2, :b=>:y}, {:a=>3, :b=>:x}, {:a=>3, :b=>:y}]
-  #
-  def combinations hashy
-    h = hashy.to_h
-    a = h.values[0].product(*h.values[1..-1])
-    res = []
-    a.each do |e|
-      o = OpenStruct.new
-      h.keys.zip(e) do |k, v|
-        o[k] = v
+    # Define a sample
+    def sample name, &_
+      start = Time.now
+      h = {}
+      name = name.to_s if name.kind_of?(Symbol)
+      h['sample'] = name
+      begin
+        h['result'] = yield
+      rescue Exception => e
+        h['exception'] = e.message
       end
-      res << o
+      output_filename = caller.first.split(/:\d+:/).first.sub(/\.rb/, '') << '.yml'
+      unless Regtest.results[output_filename]
+        Regtest.report "\n", type: :filename unless Regtest.results.empty?
+        Regtest.report output_filename, type: :filename
+        Regtest.results[output_filename] = []
+      end
+      Regtest.results[output_filename] << h
+      print '.'; $stdout.flush
+      stop = Time.now
+      Regtest.statistics << OpenStruct.new(filename: output_filename, sample: name, time: stop - start)
+      name
     end
-    res
-  end
 
-  class << self
-    attr_accessor :count, :results, :start
+    # Build all combinations of a Hash-like object with arrays as values.
+    # Return value is an array of OpenStruct instances.
+    #
+    # Example:
+    #   require 'ostruct'
+    #   require 'regtest'
+    #
+    #   o = OpenStruct.new
+    #   o.a = [1,2,3]
+    #   o.b = [:x, :y]
+    #   Regtest.combinations(o)
+    #   # => [#<OpenStruct a=1, b=:x>, #<OpenStruct a=1, b=:y>,
+    #   #     #<OpenStruct a=2, b=:x>, #<OpenStruct a=2, b=:y>,
+    #   #     #<OpenStruct a=3, b=:x>, #<OpenStruct a=3, b=:y>]
+    def combinations hashy
+      h = hashy.to_h
+      a = h.values[0].product(*h.values[1..-1])
+      res = []
+      a.each do |e|
+        o = OpenStruct.new
+        h.keys.zip(e) do |k, v|
+          o[k] = v
+        end
+        res << o
+      end
+      res
+    end
 
-    def report
+    # Report some statistics, could be overwritten by plugins.
+    def report_statistics
       time = Time.now - start
-      puts format("\n\n%d samples executed in %.2f s (%.2f samples/s)", count, time, count / time)
+      sample_count = statistics.size
+      report format("\n\n%d samples executed in %.2f s (%.2f samples/s)", sample_count, time, sample_count / time), type: :statistics
     end
 
+    # Save all results to the corresponding files.
     def save
       results.each_pair do |filename, arr|
         File.open(filename, 'w') do |f|
@@ -77,14 +92,34 @@ module Regtest
         end
       end
     end
+
+    # Checking results, should be overwritten by SCM plugins
+    # e.g. regtest/git
+    # @return Symbol with check result (one of :success, :unknown_result or :fail)
+    def check_results
+      report "\nPlease check result files manually. Regtest isn't able to do that.", type: :unknown_result
+      :unknown_result
+    end
+
+    # Report text to output with possible type, could be overwritten
+    # by plugins e.g. regtest/colorize.
+    def report *args, type: nil
+      puts *args
+    end
   end
 
-  module_function :sample, :combinations
+end
 
+# Load .regtestrc from home directory and actual directory if exists
+# and ENV['NOREGTESTRC'] is not set.
+unless ENV['NOREGTESTRC']
+  Dir[File.join(Dir.home, '.regtestrc'), '.regtestrc'].each {|fn| load fn}
 end
 
 at_exit do
   ARGV.each {|a| load a}
-  Regtest.report
   Regtest.save
+  Regtest.report_statistics
+  check_state = Regtest.check_results
+  exit Regtest.exit_codes[check_state]
 end

data/lib/regtest/colorize.rb

@@ -0,0 +1,47 @@
+# encoding: utf-8
+# frozen_string_literal: true
+
+begin
+  module Regtest
+
+    # Regtest plugin to have colorized reports.
+    # It uses the great colorize gem
+    # (see https://rubygems.org/gems/colorize)
+    module Colorize
+
+      @mapping = (%i(success fail unknown_result filename statistics).zip %i(green red yellow cyan default)).to_h
+
+      class << self
+        # Color mapping (all parameters of ColorizedString.colorize are supported
+        # (see https://github.com/fazibear/colorize#usage)
+        # Examples:
+        #   Regtest::Colorize.mapping[:filename] = :light_blue
+        #   Regtest::Colorize.mapping[:statistics] = {color: :cyan, mode: :italic}
+        attr_accessor :mapping
+      end
+
+      require 'colorized_string'
+      # If loading of colorized_string (gem colorize) fails the accessor above is
+      # available in .regtestrc files but method redefinition of Regtest.report and
+      # prepending of Colorize is skipped.
+
+      # Redefine Regtest.report.
+      def report *args, type: nil
+        color = Colorize.mapping[type]
+        if color && $stdout.tty?
+          args = args.map {|a| ColorizedString[a.to_s].colorize(color)}
+        end
+        puts *args
+      end
+
+    end
+
+    class << self
+      prepend Colorize
+    end
+
+  end
+rescue LoadError
+  warn 'gem colorize not found'
+  warn 'plugin regtest/colorize disabled'
+end

data/lib/regtest/git.rb

@@ -0,0 +1,57 @@
+# encoding: utf-8
+# frozen_string_literal: true
+
+require 'open3'
+
+# Check if git is available and current directory is part of a git repository.
+begin
+  _, ps = Open3.capture2e('git status --porcelain')
+  if ps.exitstatus == 0
+    module Regtest
+
+      # Regtest plugin for git (results are checked automatically when running
+      # regtest)
+      module Git
+
+        # Redefine Regtest.check_results.
+        def check_results
+          output_files = Regtest.results.keys
+          if output_files.empty?
+            report "\nNothing to do.", type: :success
+            return :success
+          end
+          git_stat, _, _ = Open3.capture3(*%w(git status --porcelain --), *output_files)
+          case git_stat
+          when /^.M/ # at least one modified file
+            report "\nThere are changes in your sample results!", type: :fail
+            system *%w(git status -s --), *output_files
+            return :fail
+          when /^.\?/ # at least one unknown file
+            report "\nThere is at least one new sample result file.", type: :unknown_result
+            system *%w(git status -s --), *output_files
+            return :unknown_result
+          when '', /^. / # no changes in (maybe staged) files
+            report "\nLooks good. :)", type: :success
+            system *%w(git status -s --), *output_files
+            return :success
+          else
+            report "\nYour sample results are in a bad condition!", type: :fail
+            system *%w(git status -s --), *output_files
+            return :fail
+          end
+        end
+
+      end
+
+      class << self
+        prepend Git
+      end
+
+    end
+  else
+    warn 'current directory is not part of a git repository:'
+    warn 'plugin regtest/git disabled'
+  end
+rescue Errno::ENOENT
+  warn 'git command not found: plugin regtest/git disabled'
+end

data/lib/regtest/task.rb

@@ -1,15 +1,17 @@
 # encoding: utf-8
 # frozen_string_literal: true
 #
-# Copyright 2014, 2016 by Jan Friedrich <janfri26@gmail.com>
+# Copyright 2014, 2016, 2017 by Jan Friedrich <janfri26@gmail.com>
 # License: Regtest is licensed under the same terms as Ruby itself.
 #
 
+# Regtest sample files (= Ruby files)
 REGTEST_FILES_RB = FileList.new('regtest/**/*.rb')
-REGTEST_FILES_YML = FileList.new('regtest/**/*.yml')
-REGTEST_FILES = REGTEST_FILES_RB + REGTEST_FILES_YML
+
+# ALL Regtest files (sample files, results files and maybe other files)
+REGTEST_FILES = FileList.new('regtest/**/*').select {|fn| File.file?(fn)}
 
 desc 'Run regression tests'
 task :regtest do
-  sh "ruby -I lib:regtest #{REGTEST_FILES_RB}"
+  sh *%w(ruby -I lib:regtest), *REGTEST_FILES_RB
 end

data/lib/regtest/version.rb

@@ -0,0 +1,6 @@
+# encoding: utf-8
+# frozen_string_literal: true
+
+module Regtest
+  VERSION = '2.0.0.pre'
+end

data/regtest.gemspec

@@ -1,5 +1,5 @@
 # -*- encoding: utf-8 -*-
-# stub: regtest 1.1.0 ruby lib
+# stub: regtest 2.0.0.pre ruby lib
 #
 # This file is automatically generated by rim.
 # PLEASE DO NOT EDIT IT DIRECTLY!
@@ -7,35 +7,39 @@
 
 Gem::Specification.new do |s|
   s.name = "regtest"
-  s.version = "1.1.0"
+  s.version = "2.0.0.pre"
 
-  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.required_rubygems_version = Gem::Requirement.new("> 1.3.1") if s.respond_to? :required_rubygems_version=
   s.require_paths = ["lib"]
   s.authors = ["Jan Friedrich"]
-  s.date = "2017-08-29"
-  s.description = "This library support a very simple way to do regression testing in Ruby\nprojects. You write Ruby scripts with samples. Run these and get the sample\nresults as YAML output besides your scripts. Check both the scripts and the YAML\nfiles with the results in you Source Code Control System. When you run the\nscrips on a later (or even previous) version of your code a simple diff show you\nif and how the changes in your code impact the results of your samples.\n\nThis is not a replacement for unit testing but a complement: You can produce a\nlot of samples with a small amount of Ruby code (e.g. a large number of\ncombinations of data).\n"
+  s.date = "2017-09-12"
+  s.description = "This library supports a very simple way to do regression testing with Ruby. It\nis not limited to Ruby projects you can use it also in other contexts where you\ncan extract data with Ruby.\n\nYou write Ruby scripts with samples. Run these and get the sample results as\nresults files besides your scripts. Check both the scripts and the results\nfiles in you Source Code Management System (SCM). When you run the scrips on a\nlater (or even previous) version of your code a simple diff show you if and how\nthe changes in your code or environment impact the results of your samples.\n\nThis is not a replacement for unit testing but a complement: You can produce a\nlot of samples with a small amount of Ruby code (e.g. a large number of\ncombinations of data).\n"
   s.email = "janfri26@gmail.com"
-  s.files = ["Changelog", "Gemfile", "LICENSE", "README.md", "Rakefile", "lib/regtest", "lib/regtest.rb", "lib/regtest/task.rb", "regtest.gemspec", "regtest/combinations.rb", "regtest/combinations.yml", "regtest/example.rb", "regtest/example.yml", "regtest/metatest.rb", "regtest/metatest.yml", "regtest/no_samples.rb"]
+  s.files = ["./.aspell.pws", "Changelog", "Gemfile", "LICENSE", "README.md", "Rakefile", "lib/regtest", "lib/regtest.rb", "lib/regtest/colorize.rb", "lib/regtest/git.rb", "lib/regtest/task.rb", "lib/regtest/version.rb", "regtest.gemspec", "regtest/combinations.rb", "regtest/combinations.yml", "regtest/example.rb", "regtest/example.yml", "regtest/filename with spaces.rb", "regtest/filename with spaces.yml", "regtest/metatest.rb", "regtest/metatest.yml", "regtest/metatest_git.rb", "regtest/metatest_git.yml", "regtest/no_samples.rb"]
   s.homepage = "https://github.com/janfri/regtest"
   s.licenses = ["Ruby"]
-  s.rubygems_version = "2.6.12"
-  s.summary = "Simple regression testing for Ruby projects."
+  s.required_ruby_version = Gem::Requirement.new(">= 2.1.0")
+  s.rubygems_version = "2.6.13"
+  s.summary = "Simple regression testing with Ruby."
 
   if s.respond_to? :specification_version then
     s.specification_version = 4
 
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
-      s.add_development_dependency(%q<rake>, ["< 100", "> 0"])
-      s.add_development_dependency(%q<rim>, ["~> 2.12"])
+      s.add_runtime_dependency(%q<colorize>, ["~> 0.8"])
+      s.add_development_dependency(%q<rake>, [">= 0"])
+      s.add_development_dependency(%q<rim>, ["~> 2.15"])
       s.add_development_dependency(%q<regtest>, ["~> 1.0"])
     else
-      s.add_dependency(%q<rake>, ["< 100", "> 0"])
-      s.add_dependency(%q<rim>, ["~> 2.12"])
+      s.add_dependency(%q<colorize>, ["~> 0.8"])
+      s.add_dependency(%q<rake>, [">= 0"])
+      s.add_dependency(%q<rim>, ["~> 2.15"])
       s.add_dependency(%q<regtest>, ["~> 1.0"])
     end
   else
-    s.add_dependency(%q<rake>, ["< 100", "> 0"])
-    s.add_dependency(%q<rim>, ["~> 2.12"])
+    s.add_dependency(%q<colorize>, ["~> 0.8"])
+    s.add_dependency(%q<rake>, [">= 0"])
+    s.add_dependency(%q<rim>, ["~> 2.15"])
     s.add_dependency(%q<regtest>, ["~> 1.0"])
   end
 end

data/regtest/example.rb

@@ -5,13 +5,13 @@ require 'regtest'
 
 Regtest.sample 'String result' do
   # Doing something to get the result of the sample
-  # and make sure it is the result of the block
+  # and make sure it is the result of the block.
   'some text'
 end
 
 Regtest.sample 'Division by zero' do
   # If an exception occurs while execution of the
   # block it is catched and its message is used as
-  # exception value for the sample
+  # exception value for the sample.
   2 / 0
 end

data/regtest/filename with spaces.rb

@@ -0,0 +1,8 @@
+# encoding: utf-8
+# frozen_string_literal: true
+
+require 'regtest'
+
+Regtest.sample 'spaces in filenames' do
+  "If you can read this in the results file #{File.basename(__FILE__.sub(/\.rb/, '.yml')).inspect} than regtest can handle filenames with spaces. :)"
+end

data/regtest/filename with spaces.yml

@@ -0,0 +1,4 @@
+---
+sample: spaces in filenames
+result: If you can read this in the results file "filename with spaces.yml" than regtest
+  can handle filenames with spaces. :)

data/regtest/metatest.rb

@@ -9,17 +9,19 @@ require 'tmpdir'
 Dir.mktmpdir('regtest') do |tmpdir|
   Dir.chdir __dir__ do
     Dir['*.rb'].each do |filename|
-      # beware of endless recursion
+      # Beware of endless recursion.
       next if filename =~ /metatest/
       FileUtils.cp filename, tmpdir
     end
   end
   Dir.chdir tmpdir do
-    _, o, e = Open3.popen3("ruby -I #{File.join(__dir__, '../lib')} #{Dir['*.rb'].sort.join(' ')}")
+    lib_dir = File.join(__dir__, '../lib')
+    o, e, ps = Open3.capture3(*[{'NOREGTESTRC' => 'true'}, 'ruby', '-I', lib_dir], *Dir['*.rb'].sort)
     Regtest.sample 'metatest' do
-      res = {'stdout' => o.read, 'stderr' => e.read}
-      # levelling runtime specific values
+      res = {'stdout' => o, 'stderr' => e}
+      # Levelling out runtime specific differences.
       res['stdout'].gsub!(/\d+\.\d+/, 'x.xx')
+      res['exitstatus'] = ps.exitstatus
       res
     end
   end

data/regtest/metatest.yml

@@ -6,6 +6,11 @@ result:
     ......
     example.yml
     ..
+    filename with spaces.yml
+    .
 
-    8 samples executed in x.xx s (x.xx samples/s)
+    9 samples executed in x.xx s (x.xx samples/s)
+
+    Please check result files manually. Regtest isn't able to do that.
   stderr: ''
+  exitstatus: 1

data/regtest/metatest_git.rb

@@ -0,0 +1,63 @@
+# encoding: utf-8
+# frozen_string_literal: true
+
+# If git isn't available skip this file
+begin
+  Open3.capture2e('git --version')
+
+  require 'fileutils'
+  require 'open3'
+  require 'regtest/git'
+  require 'tmpdir'
+
+  def create_sample name
+    lib_dir = File.join(__dir__, '../lib')
+    o, e, ps = Open3.capture3(*[{'NOREGTESTRC' => 'true'}, 'ruby', '-I', lib_dir, '-r', 'regtest/git'], *Dir['*.rb'].sort)
+    Regtest.sample name do
+      res = {'stdout' => o, 'stderr' => e}
+      # Levelling out runtime specific differences.
+      res['stdout'].gsub!(/\d+\.\d+/, 'x.xx')
+      res['stdout'].gsub!(%r(\d+ samples/s), 'x samples/s')
+      res['exitstatus'] = ps.exitstatus
+      res
+    end
+  end
+
+  def execute cmd
+    Open3.capture2e cmd
+  end
+
+  NEW_SAMPLE = <<-END
+
+  Regtest.sample 'new sample' do
+    'new sample'
+  end
+  END
+
+  Dir.mktmpdir('regtest') do |tmpdir|
+    Dir.chdir __dir__ do
+      Dir['*.rb'].each do |filename|
+        # Beware of endless recursion.
+        next if filename =~ /metatest/
+        FileUtils.cp filename, tmpdir
+      end
+    end
+    Dir.chdir tmpdir do
+      execute 'git init'
+      create_sample 'all new'
+      execute 'git add *.rb example.yml combinations.yml'
+      create_sample 'only one new'
+      example_rb = File.read('example.rb')
+      example_rb << NEW_SAMPLE
+      File.write('example.rb', example_rb)
+      create_sample 'one new one modified to index'
+      execute 'git commit -m "commit"'
+      create_sample 'one new one modified'
+      execute 'git add *.yml'
+      create_sample 'all in index'
+      execute 'git commit -m "commit"'
+      create_sample 'all commited'
+    end
+  end
+rescue Errno::ENOENT
+end

data/regtest/metatest_git.yml

@@ -0,0 +1,109 @@
+---
+sample: all new
+result:
+  stdout: |
+    combinations.yml
+    ......
+    example.yml
+    ..
+    filename with spaces.yml
+    .
+
+    9 samples executed in x.xx s (x.xx samples/s)
+
+    There is at least one new sample result file.
+    ?? combinations.yml
+    ?? example.yml
+    ?? filename with spaces.yml
+  stderr: ''
+  exitstatus: 1
+---
+sample: only one new
+result:
+  stdout: |
+    combinations.yml
+    ......
+    example.yml
+    ..
+    filename with spaces.yml
+    .
+
+    9 samples executed in x.xx s (x.xx samples/s)
+
+    There is at least one new sample result file.
+    A  combinations.yml
+    A  example.yml
+    ?? filename with spaces.yml
+  stderr: ''
+  exitstatus: 1
+---
+sample: one new one modified to index
+result:
+  stdout: |
+    combinations.yml
+    ......
+    example.yml
+    ...
+    filename with spaces.yml
+    .
+
+    10 samples executed in x.xx s (x.xx samples/s)
+
+    There are changes in your sample results!
+    A  combinations.yml
+    AM example.yml
+    ?? filename with spaces.yml
+  stderr: ''
+  exitstatus: 2
+---
+sample: one new one modified
+result:
+  stdout: |
+    combinations.yml
+    ......
+    example.yml
+    ...
+    filename with spaces.yml
+    .
+
+    10 samples executed in x.xx s (x.xx samples/s)
+
+    There are changes in your sample results!
+     M example.yml
+    ?? filename with spaces.yml
+  stderr: ''
+  exitstatus: 2
+---
+sample: all in index
+result:
+  stdout: |
+    combinations.yml
+    ......
+    example.yml
+    ...
+    filename with spaces.yml
+    .
+
+    10 samples executed in x.xx s (x.xx samples/s)
+
+    Looks good. :)
+    M  example.yml
+    A  "filename with spaces.yml"
+  stderr: ''
+  exitstatus: 0
+---
+sample: all commited
+result:
+  stdout: |
+    combinations.yml
+    ......
+    example.yml
+    ...
+    filename with spaces.yml
+    .
+
+    10 samples executed in x.xx s (x.xx samples/s)
+
+    Looks good. :)
+  stderr: ''
+  exitstatus: 0

data/regtest/no_samples.rb

@@ -1,3 +1,8 @@
+# encoding: utf-8
+# frozen_string_literal: true
+
+require 'regtest'
+
 # If a Ruby file doesn't contain any sample regtest shouldn't
 # generate a correspondig YAML file.
 2 + 2

metadata

@@ -1,33 +1,41 @@
 --- !ruby/object:Gem::Specification
 name: regtest
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 2.0.0.pre
 platform: ruby
 authors:
 - Jan Friedrich
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-08-29 00:00:00.000000000 Z
+date: 2017-09-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: rake
+  name: colorize
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "<"
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.8'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '100'
-    - - ">"
+        version: '0.8'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '100'
-    - - ">"
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
@@ -36,14 +44,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.12'
+        version: '2.15'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.12'
+        version: '2.15'
 - !ruby/object:Gem::Dependency
   name: regtest
   requirement: !ruby/object:Gem::Requirement
@@ -59,12 +67,15 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '1.0'
 description: |
-  This library support a very simple way to do regression testing in Ruby
-  projects. You write Ruby scripts with samples. Run these and get the sample
-  results as YAML output besides your scripts. Check both the scripts and the YAML
-  files with the results in you Source Code Control System. When you run the
-  scrips on a later (or even previous) version of your code a simple diff show you
-  if and how the changes in your code impact the results of your samples.
+  This library supports a very simple way to do regression testing with Ruby. It
+  is not limited to Ruby projects you can use it also in other contexts where you
+  can extract data with Ruby.
+
+  You write Ruby scripts with samples. Run these and get the sample results as
+  results files besides your scripts. Check both the scripts and the results
+  files in you Source Code Management System (SCM). When you run the scrips on a
+  later (or even previous) version of your code a simple diff show you if and how
+  the changes in your code or environment impact the results of your samples.
 
   This is not a replacement for unit testing but a complement: You can produce a
   lot of samples with a small amount of Ruby code (e.g. a large number of
@@ -74,20 +85,28 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- "./.aspell.pws"
 - Changelog
 - Gemfile
 - LICENSE
 - README.md
 - Rakefile
 - lib/regtest.rb
+- lib/regtest/colorize.rb
+- lib/regtest/git.rb
 - lib/regtest/task.rb
+- lib/regtest/version.rb
 - regtest.gemspec
 - regtest/combinations.rb
 - regtest/combinations.yml
 - regtest/example.rb
 - regtest/example.yml
+- regtest/filename with spaces.rb
+- regtest/filename with spaces.yml
 - regtest/metatest.rb
 - regtest/metatest.yml
+- regtest/metatest_git.rb
+- regtest/metatest_git.yml
 - regtest/no_samples.rb
 homepage: https://github.com/janfri/regtest
 licenses:
@@ -101,16 +120,16 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 2.1.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.12
+rubygems_version: 2.6.13
 signing_key: 
 specification_version: 4
-summary: Simple regression testing for Ruby projects.
+summary: Simple regression testing with Ruby.
 test_files: []