regtest 2.4.1 → 2.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a2c3eae05189bc9397839b3116acbfcfc8669d049a25fe5041f10e2c0d10c710
-  data.tar.gz: 32d1023dcd6cc66eea0bb47ede52a474edc026d8e87cefc2a79a50e3c95f07ef
+  metadata.gz: 187d2488712423e101a963409b2765cbb4a3d1b9c8a0da0a08fa80dd5737ff32
+  data.tar.gz: 653bcfb237f6f075dd95eaaa8f60b119206cb22543d4ec402fba51bb929b26d8
 SHA512:
-  metadata.gz: 8c47c65c0b8fb2e68b77f742d669f870a665b4777a3466da43c03bee7dc450d1875e67f1fce45098ca2e36d00f30cb7da85043af4c9393095d4d985f85ce0b57
-  data.tar.gz: 8d6afb80fe0584f38783a87b129272c48732d6362a404316afc38c40df6c6b1d8d90c428bebb20e5c881d7766b941f90e982935de2da71953b482443a8451ca5
+  metadata.gz: 81a5a53f3ce35af1fbbef97a6fb02d8dc5a9d5318754926a9617d95262d4c84f7b037639481f4d225491aa13d3af1039f758946f87182e7c8b488bf13d8a239a
+  data.tar.gz: 9f32c01f32d675a86ace1ffea8758a716418b7a80fe5d68eeae48a3a9f6ba4021f6e152582ad88a3fd2ab74b7dcc0b48bd5a27eb6f5934ed426a199305d72a9b

data/Changelog

@@ -1,3 +1,7 @@
+2.5.0
+Update README.
+Use rim 3.0 for development.
+
 2.4.1
 Fix a bug when using Regtest::Git.C or Regtest::Git.git_dir with relative
 paths for filenames.

data/README.md

@@ -3,20 +3,23 @@
 ## Description
 
 This library supports a very simple way to do regression testing with Ruby. It
-is not limited to Ruby projects you can use it also in other contexts where you
-can extract data with Ruby.
+is not limited to Ruby projects; you can use it also in other contexts where
+you can extract data with Ruby.
+
+The core idea is to test results against the results of an earlier run of the
+tests, not against defined results of a specification.
 
 You write Ruby scripts with samples. Run these and get the sample results as
-results files besides your scripts. Check both the scripts and the results
-files in you Source Code Management System (SCM). When you run the scrips on a
-later (or even previous) version of your code a simple diff show you if and how
-the changes in your code or environment impact the results of your samples.
+result files beside your scripts. Check both the scripts and the result files
+in your Source Code Management System (SCM). When you run the scripts on a
+later (or even previous) version of your code, a simple diff will show you if
+and how the changes in your code or environment impact the results of your
+samples.
 
 This is not a replacement for unit testing but a complement: You can produce a
-lot of samples with a small amount of Ruby code (e.g. a large number of
+lot of samples with a small amount of Ruby code (e.g., a large number of
 combinations of data).
 
-
 ## Installation
 
 Installing the gem on the command line with
@@ -38,16 +41,19 @@ to your Gemfile.
 The idea behind regtest is the following workflow:
   1. Writing samples
   2. Running samples
-  3. Commit samples and result files to SCM.
-  4. Change your code and / or external environment.
-  5. Rerun your samples
-  6. Check sample results for changes (this is normally automatically done with
-     a regtest plugin like regtest/git).
+  3. Committing samples and result files to SCM
+  4. Changing code and/or external environment
+  5. Rerunning the samples
+  6. Checking sample results for changes (this is normally automatically done with
+     a regtest plugin like regtest/git)
+  7. If there are changed that are indented (new samples, corrected behavior)
+     committing the files to the SCM - if there are unintended changes, fix the
+     causes
 
 
 ### Writing Samples
 
-A samples file is a simple Ruby script with one ore more samples, for example
+A samples file is a simple Ruby script with one or more samples. Let's see an example
 
 ```ruby
 require 'regtest'
@@ -69,11 +75,11 @@ end
 The name of the sample (parameter of the `Regtest.sample` method) and the
 results of the samples (return value of the block) are stored in YAML format.
 So it should be a YAML-friendly value as `String`, `Number`, `Boolean value`,
-`Symbol`. Results could also be an `Array` or `Hash` with such values.
+`Symbol`. Also `arrays` and `hashes` are commonly used.
 
-In many cases you want to generate a lot of combinations of input data in your
+In some cases you want to generate a lot of combinations of input data in your
 sample code. For this there is a method `Regtest.combinations` to generate a
-lot of combinations the easy way. An example:
+lot of combinations in an easy way. An example:
 
 ```ruby
 require 'ostruct'
@@ -85,12 +91,13 @@ o.b = [:x, :y]
 Regtest.combinations(o)
 # => [#<OpenStruct a=1, b=:x>, #<OpenStruct a=1, b=:y>,
 #     #<OpenStruct a=2, b=:x>, #<OpenStruct a=2, b=:y>,
-#     #<OpenStruct a=3, b=:x>, #<OpenStruct a=3, b=:y>] 
+#     #<OpenStruct a=3, b=:x>, #<OpenStruct a=3, b=:y>]
 ```
 
-See also the combinations example in the `regtest` folder.
+See also the example `combinations` in directory `regtest` of this repository.
 
-By convention sample files are stored in a directory `regtest` in your Ruby application.
+By convention samples files are stored in a directory names `regtest` in your
+Ruby application.
 
 
 ### Running Samples
@@ -101,13 +108,13 @@ Whether you run your examples manually
 ruby -I lib regtest/*.rb
 ```
 
-or using the Rake task of regtest and add
+or using the Rake task of regtest by adding
 
 ```ruby
 require 'regtest/task'
 ```
 
-to your `Rakefile` and you can run your samples with `rake regtest`.
+to your `Rakefile`. Then you can run your samples with `rake regtest`.
 
 
 ### Checking Results
@@ -128,7 +135,7 @@ regtest/foo.yml
 regtest/bar.yml
 ```
 
-So the content of the results file of the example above is
+So the content of the results file of the first example above is
 
 ```yaml
 ---
@@ -139,30 +146,31 @@ sample: Division by zero
 exception: divided by 0
 ```
 
-Each time you run one ore more samples file the corresponding results files will
-be overwritten (or generated if not yet existent) with the actual result values
-of your samples. The determination of changes between the results of actual and
-older runs of the samples is done by your SCM. So the sample files and their
-corresponding results files should be taken under version control.
+Each time you run one or more samples files, the corresponding results files
+will be overwritten (or generated if they do not yet exist) with the actual
+results values of your samples. The determination of changes between the
+results of actual and older runs of the samples is done by your SCM. Therefore
+the sample files and their corresponding results files should be taken under
+version control.
 
 
 ## Logging
 
 The key idea behind regtest is to produce values that are invariant and check
 if this assumption is true at another (mostly later) state of code. But often
-there are temporary or specific values which changes or could change at each
-run of regtest. This could be for example an id of a created record or the
+there are temporary or specific values which change or could change at each
+run of regtest. This could be, for example, an id of a created record or the
 version of a used external service or some time-relevant values. Sometimes it
-is useful, to know the actual value of one of these.
+is useful, to know the actual values of some of these.
 
-In such cases the method ```Regtest.log``` could be handy. It writes a line of
-the given object to a log file which is named with the same name as the calling
-Ruby script but has as extension ```.log```. It could be called inside as well
-as outside of a regtest sample. Per default this file is overwritten by the
-first call of ```Regtest.log``` of each run of regtest and per file. And each
-further call of ```Regtest.log``` appends then to the file.  So you get a
-complete log for each run. But this behaviour could be changed with the
-```mode``` keyword arg of the method. Let's see an example:
+In such cases the method `Regtest.log` could be handy. It writes a line of the
+given object to a log file which is named with the same name as the calling
+Ruby script but has the extension `.log`. It could be called inside as well as
+outside of a regtest sample. As default this file is overwritten by the first
+call of `Regtest.log` of each run of regtest and per file. And each further
+call of `Regtest.log` appends then to the file.  So you get a complete log for
+each run. This behavior could be changed with the `mode` keyword argument of
+the method. Let's see an example:
 
 ```ruby
 Regtest.log RUBY_VERSION
@@ -191,7 +199,7 @@ end
 ```
 
 If you want to have a log that is not truncated at each run of regtest, you can
-use ```mode: 'a'```at the first call of ```Regtest.log``` in the corresponding
+use `mode: 'a'` at the first call of `Regtest.log` in the corresponding
 ruby script.
 
 ```ruby
@@ -200,8 +208,8 @@ ruby script.
 Regtest.log Time.now, mode: 'a'
 ```
 
-On the other hand, you can use ```mode: 'w'``` to truncate the log file even at
-a later call of ```Regtest.log```.
+On the other hand, you can use `mode: 'w'` to truncate the log file even at a
+later call of `Regtest.log`.
 
 ```ruby
 max_time = 0
@@ -215,91 +223,105 @@ ary.each do |e|
 end
 ```
 
-Because the log files contains only temporary stuff they should normally not
-checked in the SCM.
+Because the log files contain only temporary information, they should normally
+not be checked into the SCM.
 
 
-## Exceptions an backtraces
+## Configuration and Plugins
 
-If there is an exception raised inside a regtest sample its message is a part
-of the result of the sample. This is intentional because exceptions are
-possible results you want to check.
+You can adapt the behavior of regtest with plugins. To configure this and maybe
+other things regtest supports a simple rc file mechanism. While loading regtest
+via `require 'regtest'` it looks for a file `.regtestrc` first in your home
+directory and then in the local directory. So you can do global configurations
+in the first one and project-specific configurations in the latter.
 
-But sometimes an exception occur inside of a sample that was not the intention
-of the sample code. In such situation it would be helpful to have the full
-exception message with the backtrace, to find the code location where the error
-occurred.
+Normally, the check of changes in results is done automatically by a regtest
+plugin like regtest/git (see below). In this case, the report will show you if
+there are changes or not, and the exit code of the script is accordingly set.
+The standard values are: `0` for success, `1` for an unknown result (normally a
+new results file), and `2` for failure. If you use plain regtest without a SCM
+plugin, the exit code is `1` (= unknown result).
 
-You can do this with setting `Regtest.show_exceptions = true` (normally in a
-local `.regtestrc` file, see below) temporarily. Then the exception and
-backtrace is written to STDERR.
+You can change the exit codes for the states with `Regtest.exit_codes`. The
+following example changes the behavior to the same as in regtest version 1.x.
 
+```ruby
+Regtest.exit_codes[:unknown_result] = 0
+Regtest.exit_codes[:fail] = 0
+```
 
-## Configuration and Plugins
+This should be done in a `.regtestrc` file and not in sample files.
 
-You can adapt the behaviour of regtest with plugins. To configure this and
-maybe other things regtest support a simple rc file mechanism. While loading
-regtest via `require 'regtest'` it looks for a file `.regtestrc` first in your
-home directory and then in the local directory. So you can do global
-configurations in the first one and project specific configurations in the
-latter.
+Because in a `.regtestrc` file are individual configuration aspects of your
+workflow and environment, it should not be checked into your SCM.
 
-For example the following is a good default when you want colorized output and
-use git as your SCM:
 
-```ruby
-require 'regtest/colors'
-# adapt some colorizing if wanted
-Regtest::Colors.mapping[:filename] = :cyan
-Regtest::Colors.mapping[:statistics] = %i(blue italic)
+### Plugin regtest/colors
 
-require 'regtest/git'
-```
+When using the regtest/colors plugin (`require 'regtest/colors'`) it is
+possible to adapt the colors of the output of different types of messages. The
+following mappings are possible:
 
-Normally the check of changes in results is done automatically by a regtest
-plugin like regtest/git (see example for `.regtestrc` above). In this case the
-report will show you if there are changes or not and the exit code of the
-script is accordingly set. The standard values are: 0 for success, 1 for an
-unknown result (normally a new results file) and 2 for failure. If you use
-plain regtest without a SCM plugin the exit code is 1 (= unknown result).
+* `:filename`
+* `:statistics`
+* `:success`
+* `:unknown_result`
+* `:fail`
 
-You can change the exit codes for the states with `Regtest.exit_codes`. The
-following example changes the behaviour to the same as in regtest version 1.x.
+The configuration is done as the following example shows:
 
 ```ruby
-Regtest.exit_codes[:unknown_result] = 0 
-Regtest.exit_codes[:fail] = 0 
+require 'regtest/colors'
+# adapt some colorizing if wanted
+Regtest::Colors.mapping[:filename] = :cyan
+Regtest::Colors.mapping[:statistics] = %i(blue italic)
+Regtest::Colors.mapping[:fail] = %i(white @red)
 ```
 
-This also should be done in a `.regtest` file and not in the sample files.
-
-Because in a `.regtestrc` file are individual configuration aspects of your
-workflow and environment it should not be checked into your SCM.
+As you can see there are colors and modifiers (such as `[:blue, :italic]`)
+possible. Color codes with prefix `@` are background colors. Run
+`Regtest::Colors.codes` to get a list of possible color codes.
 
 
-## Plugin regtest/git
+### Plugin regtest/git
 
-If you use the git plugin (`require 'regtest/git'`) there will be two options
-you can set:
+If you use the git plugin (`require 'regtest/git'`), there are the following
+options available:
 
 * `Regtest::Git.C`
 * `Regtest::Git.git_dir`
 * `Regtest::Git.work_tree`
 
 which corresponds to the git parameters `-C`, `--git-dir` and `--work-tree`.
-Which could be helpful if you run `regtest` from inside some other git
+These could be helpful if you run `regtest` from inside some other git
 repository than your regtest files. Have a look at the git documentation for
 more details.
 
+Be aware when using these options. It is possible to get unwanted results.
+
 As said above: this should also be done in a local `.regtestrc` file.
 
 
+### Example of a .regtestrc file
+
+The following is a good default when you want colorized output and
+use git as your SCM:
+
+```ruby
+require 'regtest/colors'
+require 'regtest/git'
+```
+
+See above for more information about the plugins `regtest/colors` and
+`regtest/git`.
+
+
 ## Rake task
 
-Regtest includes a Rake task `regtest`. Per default it runs any `.rb` files
+Regtest includes a Rake task `regtest`. By default it runs any `.rb` files
 under the `regtest` directory and includes all files under the `regtest`
-directory to the files packaged with gem.
-You can change these defaults like this
+directory to the files packaged with gem.  You can change these defaults like
+this:
 
 ```ruby
 require 'regtest/task'
@@ -308,14 +330,100 @@ REGTEST_FILES_RB.clear << 'my_regtest_file.rb'
 REGTEST_FILES.clear << 'my_regtest_file.rb' << 'my_regtest_file.yml' << 'other_file'
 ```
 
-It's a little bit old school like `CLEAN` and `CLOBBER` but I like the simple
-approach to use constants.
+It's a little bit old school like `CLEAN` and `CLOBBER` in `rake/clean` but I
+like the simple approach to use constants.
+
+
+## Best practise
+
+### Use Ruby
+
+As said above, the sample files are plain Ruby scripts. Yes, Ruby is a
+scripting language. So use the power of Ruby to get the things done you need.
+Some food for thought:
+
+* Generate samples inside of loops.
+* Use `String#gsub` to level out runtime specific values. For example, if you
+get a response of a web service "Record 4711 created" and do a
+`sub(/\d+/, '#')` the `4711` is eliminated and the string is invariant between
+different runs of regtest.
+* Use compare operators to get invariant values for sample results.
+
+
+### Know your SCM
+
+I use git as SCM for my work. Here are some hints how I check changes in the
+results files:
+
+```shell
+git diff --ignore-all-space -- regtest/*.yml
+git diff --color-words --ignore-all-space -- regtest/*.yml
+git diff --color-words=. --ignore-all-space
+git diff --color-words=\\w+ --ignore-all-space
+git diff --color-moved
+```
+
+
+### Unexpected exceptions
+
+If there is an exception raised inside a regtest sample its message is a part
+of the result of the sample. This is intentional because exceptions are
+possible results you want to check.
+
+But sometimes an exception occur inside of a sample that was not the intention
+of the sample code. In such situation it would be helpful to have the full
+exception message with the backtrace, to find the code location where the error
+occurred.
+
+You can do this with setting `Regtest.show_exceptions = true` Then the
+exception and backtrace is written to STDERR.
+
+Example
+
+```Ruby
+Regtest.sample 'something' do
+  # ...
+end
+
+Regtest.sample 'something goes wrong' do
+  Regtest.show_exceptions = true
+  # the code that causes the problem
+  Regtest.show_exceptions = false
+end
+
+Regtest.sample 'another thing' do
+  # ...
+end
+```
+
+Alternatively the use of the Ruby debugger (`require 'debug'`) could be helpful (Know Ruby):
+
+```ruby
+Regtest.sample 'something' do
+  # ...
+end
+
+Regtest.sample 'something goes wrong' do
+  IRB.debugger
+  # the code that causes the problem
+end
+
+Regtest.sample 'another thing' do
+  # ...
+end
+```
+
+
+### Regtest is not thread safe
+
+The methods `Regtest.sample` and `Regtest.log` are not thread safe. So you have
+to ensure they are executed in a sequential order to avoid unexpected results.
 
 
 ## Further information
 
 I use `regtest` in my project [scripref](https://github.com/janfri/scripref) to
-generate a lot of malformed input data to check regressions in the behaviour of
+generate a lot of malformed input data to check regressions in the behavior of
 the parser and text processor.
 
 A little different is the usage in my projects

data/lib/regtest/version.rb

@@ -2,5 +2,5 @@
 # frozen_string_literal: true
 
 module Regtest
-  VERSION = '2.4.1'
+  VERSION = '2.5.0'
 end

metadata

@@ -1,13 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: regtest
 version: !ruby/object:Gem::Version
-  version: 2.4.1
+  version: 2.5.0
 platform: ruby
+original_platform: ''
 authors:
 - Jan Friedrich
 bindir: bin
 cert_chain: []
-date: 2024-09-14 00:00:00.000000000 Z
+date: 2024-11-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -29,14 +30,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.17'
+        version: '3.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.17'
+        version: '3.0'
 description: |
   This library supports a very simple way to do regression testing with Ruby. It
   is not limited to Ruby projects you can use it also in other contexts where you
@@ -56,18 +57,14 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- "./.aspell.pws"
 - Changelog
-- Gemfile
 - LICENSE
 - README.md
-- Rakefile
 - lib/regtest.rb
 - lib/regtest/colors.rb
 - lib/regtest/git.rb
 - lib/regtest/task.rb
 - lib/regtest/version.rb
-- regtest.gemspec
 homepage: https://github.com/janfri/regtest
 licenses:
 - Ruby

data/.aspell.pws

@@ -1,38 +0,0 @@
-personal_ws-1.1 en 37 
-Bugfix
-ENV
-Friedrich
-Gemfile
-Matz
-NOREGTESTRC
-OpenStruct
-README
-Rakefile
-Regtest
-SCM
-STDERR
-SemVer
-SemVerTag
-YAML
-arg
-ary
-backtrace
-backtraces
-basename
-bitbucket
-catched
-dir
-exiftool
-gettime
-github
-md
-metatest
-metatests
-ostruct
-rb
-rc
-regtest
-regtestrc
-scripref
-yaml
-yml

data/Gemfile

@@ -1,3 +0,0 @@
-source 'https://rubygems.org'
-
-gemspec

data/Rakefile

@@ -1,53 +0,0 @@
-# encoding: utf-8
-# frozen_string_literal: true
-
-$:.unshift 'lib'
-
-require 'rim/tire'
-require 'rim/version'
-
-require 'regtest/task'
-require 'regtest/version'
-
-Rim.setup do |p|
-  p.name = 'regtest'
-  p.version = Regtest::VERSION
-  p.authors = 'Jan Friedrich'
-  p.email = 'janfri26@gmail.com'
-  p.summary = 'Simple regression testing with Ruby.'
-  p.license = 'Ruby'
-  p.description = <<-END.gsub(/^ +/, '')
-    This library supports a very simple way to do regression testing with Ruby. It
-    is not limited to Ruby projects you can use it also in other contexts where you
-    can extract data with Ruby.
-
-    You write Ruby scripts with samples. Run these and get the sample results as
-    results files besides your scripts. Check both the scripts and the results
-    files in you Source Code Management System (SCM). When you run the scrips on a
-    later (or even previous) version of your code a simple diff show you if and how
-    the changes in your code or environment impact the results of your samples.
-
-    This is not a replacement for unit testing but a complement: You can produce a
-    lot of samples with a small amount of Ruby code (e.g. a large number of
-    combinations of data).
-  END
-  p.homepage = 'https://github.com/janfri/regtest'
-  p.ruby_version = '>=2.1.0'
-end
-
-# JRuby does not support escaping of filenames with spaces in Open3.capture3
-# therefore ignore metatest files when running on JRuby
-if RUBY_ENGINE == 'jruby'
-  REGTEST_FILES_RB.reject! {|fn| fn =~ /metatest/}
-end
-
-task :before_regtest do
-  verbose false do
-    rm_rf 'regtest/*.log'
-    rm_rf 'regtest/*.yml'
-  end
-end
-
-task :regtest => :before_regtest
-task :test => :regtest
-task :default => :test

data/regtest.gemspec

@@ -1,29 +0,0 @@
-# -*- encoding: utf-8 -*-
-# stub: regtest 2.4.1 ruby lib
-#
-# This file is automatically generated by rim.
-# PLEASE DO NOT EDIT IT DIRECTLY!
-# Change the values in Rim.setup in Rakefile instead.
-
-Gem::Specification.new do |s|
-  s.name = "regtest"
-  s.version = "2.4.1"
-
-  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
-  s.require_paths = ["lib"]
-  s.authors = ["Jan Friedrich"]
-  s.date = "2024-09-14"
-  s.description = "This library supports a very simple way to do regression testing with Ruby. It\nis not limited to Ruby projects you can use it also in other contexts where you\ncan extract data with Ruby.\n\nYou write Ruby scripts with samples. Run these and get the sample results as\nresults files besides your scripts. Check both the scripts and the results\nfiles in you Source Code Management System (SCM). When you run the scrips on a\nlater (or even previous) version of your code a simple diff show you if and how\nthe changes in your code or environment impact the results of your samples.\n\nThis is not a replacement for unit testing but a complement: You can produce a\nlot of samples with a small amount of Ruby code (e.g. a large number of\ncombinations of data).\n"
-  s.email = "janfri26@gmail.com"
-  s.files = ["./.aspell.pws", "Changelog", "Gemfile", "LICENSE", "README.md", "Rakefile", "lib/regtest", "lib/regtest.rb", "lib/regtest/colors.rb", "lib/regtest/git.rb", "lib/regtest/task.rb", "lib/regtest/version.rb", "regtest.gemspec"]
-  s.homepage = "https://github.com/janfri/regtest"
-  s.licenses = ["Ruby"]
-  s.required_ruby_version = Gem::Requirement.new(">= 2.1.0")
-  s.rubygems_version = "3.6.0.dev"
-  s.summary = "Simple regression testing with Ruby."
-
-  s.specification_version = 4
-
-  s.add_development_dependency(%q<rake>, [">= 0"])
-  s.add_development_dependency(%q<rim>, ["~> 2.17"])
-end