simmer 2.0.0.pre.alpha.1 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 81ea16fc455aa4484da9ddde45699539a50a9d26fc38f8dd73e7a596bd9358f9
4
- data.tar.gz: cddf6abdb4a6369c490d60ae9313a62c52d2253bf3f1a04be63ac85f784c1739
3
+ metadata.gz: fcaddd54b4d50b90701d91f9a62fdfd163a249565d88c850386caf22135a0af3
4
+ data.tar.gz: 143b6629e67a14e365d6e16a0d795b607f1f4f29aadc270be6458e715a7b68b3
5
5
  SHA512:
6
- metadata.gz: ae83fea49a7f342e3032d0f8e7ba01d10e29269a94c0c045e36cbada63f6fa5c168ead31d3aa6a0bcb58c75dbfd8e02ba6a5c4362f2ec5bd96252fe2d203bf26
7
- data.tar.gz: a678ead18314a24df043ea56f9c90de8093daf1ef56c2006efb5ed2c52eda056fa0d612c2a84a8a94f32d9434dfdd01b35ea4e908509c364e716728d5c2c6d8c
6
+ metadata.gz: 6645db0960863e2490964b56228969bd3b4a061cc8ad61b069a239dee8920d2861bdbeb125b758861426079529b3c7faed1bb645c67bb3ebbbbeb7a5a9fc11bf
7
+ data.tar.gz: 257ab72c85cca10bc54a6e23d451b3da17bd056c8d0a2b96a5b493c0f58a2d38f98bd7e6a4fcd14bafe190a8cafecfabf70a713be37777b86befef88e34e7220
@@ -13,7 +13,7 @@ Metrics/BlockLength:
13
13
  - define
14
14
 
15
15
  Metrics/MethodLength:
16
- Max: 25
16
+ Max: 30
17
17
 
18
18
  AllCops:
19
19
  TargetRubyVersion: 2.5
@@ -23,3 +23,9 @@ Metrics/AbcSize:
23
23
 
24
24
  Metrics/ClassLength:
25
25
  Max: 125
26
+
27
+ Style/TrailingCommaInHashLiteral:
28
+ Enabled: false
29
+
30
+ Style/TrailingCommaInArrayLiteral:
31
+ Enabled: false
@@ -1 +1 @@
1
- 2.6.5
1
+ 2.6.6
@@ -6,9 +6,9 @@ services:
6
6
  - mysql
7
7
  rvm:
8
8
  # Build on the latest stable of all supported Rubies (https://www.ruby-lang.org/en/downloads/):
9
- - 2.5.5
10
- - 2.6.5
11
- - 2.7.0
9
+ - 2.5.8
10
+ - 2.6.6
11
+ - 2.7.1
12
12
  cache: bundler
13
13
  before_script:
14
14
  - curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
@@ -1,5 +1,48 @@
1
- # 2.0.0 (May 11th, 2020)
1
+ # Simmer Change Log
2
+
3
+ ## 4.0.0 (TBD, 2020)
4
+
5
+ Additions:
6
+
7
+ * Support for custom test lifecycle hooks to be run before and after a every test or the entire suite.
8
+ * Simmer can now be configured to re-run tests which have failed due to a timeout a custom number of times.
9
+
10
+ Breaking Changes:
11
+
12
+ * The `callback_configuration` and `make_runner` methods have been removed from the `Simmer` module.
13
+ * `Simmer::Runner::Result#errors` now contains error/exception instances instead of strings.
14
+ * `Simmer::Suite.new` and `Simmer::Runner.run` now takes a `Simmer::Configuration` instance instead of a hash of configuration values.
15
+ * The config keyword parameter of `Simmer::Runner.run` is now mandatory.
16
+
17
+ ## 3.0.0 (June 8th, 2020)
18
+
19
+ Breaking Changes:
20
+
21
+ * `Simmer::Runner` now accepts a `Simmer::Suite::OutputRouter` instead of an `IO` instance as its 'out' parameter.
22
+ * The `execution_output` and `execution_result` methods have been removed from `Simmer::Runner::Result`.
23
+
24
+ Additions:
25
+
26
+ * pdi_out.txt is written to throughout test execution instead of at the end.
27
+
28
+ Fixes:
29
+
30
+ * Fixtures now handle identifiers which are MySQL reserved words.
31
+
32
+ ## 2.1.0 (May 13th, 2020)
33
+
34
+ Additions:
35
+
36
+ * Do not make missing fixtures short-circuit the rest of the test suite.
37
+ * Do not make PDI timeouts short-circuit the rest of the test suite.
38
+ * Report PDI's exit code and execution time to the console.
39
+
40
+ ## 2.0.0 (May 11th, 2020)
2
41
 
3
42
  Breaking Changes:
4
43
 
5
44
  * Do not emit error file, standard error has been combined with the standard output within the underlying pdi library.
45
+
46
+ Additions:
47
+
48
+ * Enhanced Simmer configuration for `spoon_client` to account for `timeout_in_seconds` option.
data/README.md CHANGED
@@ -53,6 +53,9 @@ The configuration file contains information about external systems, such as:
53
53
  Copy this configuration template into your project's root to: `config/simmer.yaml`:
54
54
 
55
55
  ````yaml
56
+ # Automatically retry a test when it has failed this many times due to a timeout error:
57
+ timeout_failure_retry_count: 0
58
+
56
59
  mysql_database:
57
60
  database:
58
61
  username:
@@ -270,6 +273,34 @@ You can also omit the path altogether to execute all specs:
270
273
  bundle exec simmer
271
274
  ````
272
275
 
276
+ ## Custom Configuration
277
+
278
+ It is possible to define custom test lifecycle hooks. These are very similar to [Rspec](https://relishapp.com/rspec/rspec-core/v/3-9/docs/hooks/before-and-after-hooks). Here is an example of how to ensure that code called before and after the entire suite:
279
+
280
+ ````ruby
281
+ Simmer.configure do |config|
282
+ config.before(:suite) { puts 'about to run the entire suite' }
283
+ config.after(:suite) do |result|
284
+ result_msg = result.passed? ? 'passed' : 'failed'
285
+ puts "The suite #{result_msg}."
286
+ end
287
+ end
288
+ ````
289
+
290
+ Not that after callbacks taken an optional parameter which is the result object.
291
+
292
+ It is also possible to specify custom code which runs before and after each individual specification.
293
+
294
+ ````ruby
295
+ Simmer.configure do |config|
296
+ config.before(:each) { puts 'I will run before each test' }
297
+ config.after(:each) do |result|
298
+ result_msg = result.passed? ? 'passed' : 'failed'
299
+ puts "The specification #{result_msg}."
300
+ end
301
+ end
302
+ ````
303
+
273
304
  ## Contributing
274
305
 
275
306
  ### Development Environment Configuration
@@ -281,6 +312,15 @@ Basic steps to take to get this repository compiling:
281
312
  3. Clone the repository (git clone git@github.com:bluemarblepayroll/simmer.git)
282
313
  4. Navigate to the root folder (cd simmer)
283
314
  5. Install dependencies (bundle)
315
+ 6. Create the 'simmer_test' MySQL database as defined in `spec/db/tables.sql`.
316
+ 7. Add the tables from `spec/db/tables.sql` to this database.
317
+ 8. Configure your test simmer.yaml:
318
+
319
+ ````bash
320
+ cp spec/config/simmer.yaml.ci spec/config/simmer.yaml
321
+ ```
322
+
323
+ 9. Edit `spec/config/simmer.yaml` so that it can connect to the database created in step seven.
284
324
 
285
325
  ### Running Tests
286
326
 
@@ -22,7 +22,7 @@ require 'set'
22
22
  require 'stringento'
23
23
  require 'yaml'
24
24
 
25
- # Monkey-patching core libaries
25
+ # Monkey-patching core libraries
26
26
  require_relative 'simmer/core_ext/hash'
27
27
  Hash.include Simmer::CoreExt::Hash
28
28
 
@@ -30,10 +30,12 @@ Hash.include Simmer::CoreExt::Hash
30
30
  require_relative 'simmer/util'
31
31
 
32
32
  # Core code
33
+ require_relative 'simmer/bootstrap'
33
34
  require_relative 'simmer/configuration'
34
35
  require_relative 'simmer/database'
35
36
  require_relative 'simmer/externals'
36
37
  require_relative 'simmer/runner'
38
+ require_relative 'simmer/re_runner'
37
39
  require_relative 'simmer/specification'
38
40
  require_relative 'simmer/suite'
39
41
 
@@ -49,112 +51,28 @@ module Simmer
49
51
  out: $stdout,
50
52
  simmer_dir: DEFAULT_SIMMER_DIR
51
53
  )
52
- configuration = make_configuration(config_path: config_path, simmer_dir: simmer_dir)
53
- specs = make_specifications(path, configuration.tests_dir)
54
- runner = make_runner(configuration, out)
55
- suite = make_suite(configuration, out, runner)
56
-
57
- suite.run(specs)
54
+ Bootstrap.new(
55
+ spec_path: path,
56
+ config_path: config_path,
57
+ simmer_dir: simmer_dir,
58
+ callback_configuration: callback_configuration,
59
+ console_out: out
60
+ ).run_suite
58
61
  end
59
62
 
60
- def make_configuration(
61
- config_path: DEFAULT_CONFIG_PATH,
62
- simmer_dir: DEFAULT_SIMMER_DIR
63
- )
64
- raw_config = yaml_reader.smash(config_path)
65
-
66
- Configuration.new(raw_config, simmer_dir)
63
+ def configuration(config_path: DEFAULT_CONFIG_PATH, simmer_dir: DEFAULT_SIMMER_DIR)
64
+ Bootstrap.new(config_path: config_path, simmer_dir: simmer_dir).configuration
67
65
  end
66
+ alias make_configuration configuration
68
67
 
69
- def make_runner(configuration, out)
70
- database = make_mysql_database(configuration)
71
- file_system = make_file_system(configuration)
72
- fixture_set = make_fixture_set(configuration)
73
- spoon_client = make_spoon_client(configuration)
74
-
75
- Runner.new(
76
- database: database,
77
- file_system: file_system,
78
- fixture_set: fixture_set,
79
- out: out,
80
- spoon_client: spoon_client
81
- )
68
+ def configure
69
+ yield callback_configuration
82
70
  end
83
71
 
84
72
  private
85
73
 
86
- def yaml_reader
87
- Util::YamlReader.new
88
- end
89
-
90
- def make_specifications(path, tests_dir)
91
- path = path.to_s.empty? ? tests_dir : path
92
-
93
- Util::YamlReader.new.read(path).map do |file|
94
- config = (file.data || {}).merge(path: file.path)
95
-
96
- Specification.make(config)
97
- end
98
- end
99
-
100
- def make_fixture_set(configuration)
101
- config = Util::YamlReader.new.smash(configuration.fixtures_dir)
102
-
103
- Database::FixtureSet.new(config)
104
- end
105
-
106
- def make_mysql_database(configuration)
107
- config = configuration.mysql_database_config.symbolize_keys
108
- client = Mysql2::Client.new(config)
109
- exclude_tables = config[:exclude_tables]
110
-
111
- Externals::MysqlDatabase.new(client, exclude_tables)
112
- end
113
-
114
- def make_file_system(configuration)
115
- if configuration.aws_file_system?
116
- make_aws_file_system(configuration)
117
- elsif configuration.local_file_system?
118
- make_local_file_system(configuration)
119
- else
120
- raise ArgumentError, 'cannot determine file system'
121
- end
122
- end
123
-
124
- def make_aws_file_system(configuration)
125
- config = configuration.aws_file_system_config.symbolize_keys
126
- client_args = config.slice(:access_key_id, :secret_access_key, :region)
127
- client = Aws::S3::Client.new(client_args)
128
-
129
- Externals::AwsFileSystem.new(
130
- client,
131
- config[:bucket],
132
- config[:encryption],
133
- configuration.files_dir
134
- )
135
- end
136
-
137
- def make_local_file_system(configuration)
138
- config = configuration.local_file_system_config.symbolize_keys
139
-
140
- Externals::LocalFileSystem.new(config[:dir], configuration.files_dir)
141
- end
142
-
143
- def make_spoon_client(configuration)
144
- config = (configuration.spoon_client_config || {}).symbolize_keys
145
- spoon_args = config.slice(:args, :dir, :kitchen, :pan, :timeout_in_seconds)
146
- spoon = Pdi::Spoon.new(spoon_args)
147
-
148
- Externals::SpoonClient.new(configuration.files_dir, spoon)
149
- end
150
-
151
- def make_suite(configuration, out, runner)
152
- Suite.new(
153
- config: configuration.config,
154
- out: out,
155
- results_dir: configuration.results_dir,
156
- runner: runner
157
- )
74
+ def callback_configuration
75
+ @callback_configuration ||= Configuration::CallbackDsl.new
158
76
  end
159
77
  end
160
78
  end
@@ -0,0 +1,133 @@
1
+ # frozen_string_literal: true
2
+
3
+ #
4
+ # Copyright (c) 2020-present, Blue Marble Payroll, LLC
5
+ #
6
+ # This source code is licensed under the MIT license found in the
7
+ # LICENSE file in the root directory of this source tree.
8
+ #
9
+
10
+ module Simmer
11
+ # :nodoc:
12
+ # Responsible for creating core objects needed to run tests.
13
+ class Bootstrap
14
+ attr_reader :callback_configuration, :configuration, :console_out, :spec_path
15
+
16
+ def initialize(
17
+ config_path:,
18
+ simmer_dir:,
19
+ spec_path: nil,
20
+ console_out: $stdout,
21
+ callback_configuration: Configuration::CallbackDsl.new
22
+ )
23
+ @spec_path = spec_path.to_s
24
+ raw_config = yaml_reader.smash(config_path)
25
+ @configuration = Configuration.new(raw_config, simmer_dir, callbacks: callback_configuration)
26
+ @console_out = console_out
27
+ @callback_configuration = callback_configuration
28
+
29
+ freeze
30
+ end
31
+
32
+ def run_suite
33
+ suite.run(specs)
34
+ end
35
+
36
+ private
37
+
38
+ def specs
39
+ path = spec_path.empty? ? configuration.tests_dir : spec_path
40
+
41
+ Util::YamlReader.new.read(path).map do |file|
42
+ config = (file.data || {}).merge(path: file.path)
43
+
44
+ Specification.make(config)
45
+ end
46
+ end
47
+
48
+ def suite
49
+ Suite.new(
50
+ config: configuration,
51
+ out: console_out,
52
+ results_dir: configuration.results_dir,
53
+ runner: runner
54
+ )
55
+ end
56
+
57
+ def runner
58
+ runner = Runner.new(
59
+ database: mysql_database,
60
+ file_system: file_system,
61
+ fixture_set: fixture_set,
62
+ out: output_router,
63
+ spoon_client: spoon_client
64
+ )
65
+
66
+ ReRunner.new(
67
+ runner,
68
+ output_router,
69
+ timeout_failure_retry_count: configuration.timeout_failure_retry_count
70
+ )
71
+ end
72
+
73
+ def yaml_reader
74
+ Util::YamlReader.new
75
+ end
76
+
77
+ def fixture_set
78
+ config = Util::YamlReader.new.smash(configuration.fixtures_dir)
79
+
80
+ Database::FixtureSet.new(config)
81
+ end
82
+
83
+ def mysql_database
84
+ config = configuration.mysql_database_config.symbolize_keys
85
+ client = Mysql2::Client.new(config)
86
+ exclude_tables = config[:exclude_tables]
87
+
88
+ Externals::MysqlDatabase.new(client, exclude_tables)
89
+ end
90
+
91
+ def file_system
92
+ if configuration.aws_file_system?
93
+ aws_file_system
94
+ elsif configuration.local_file_system?
95
+ local_file_system
96
+ else
97
+ raise ArgumentError, 'cannot determine file system'
98
+ end
99
+ end
100
+
101
+ def aws_file_system
102
+ config = configuration.aws_file_system_config.symbolize_keys
103
+ client_args = config.slice(:access_key_id, :secret_access_key, :region)
104
+ client = Aws::S3::Client.new(client_args)
105
+
106
+ Externals::AwsFileSystem.new(
107
+ client,
108
+ config[:bucket],
109
+ config[:encryption],
110
+ configuration.files_dir
111
+ )
112
+ end
113
+
114
+ def local_file_system
115
+ config = configuration.local_file_system_config.symbolize_keys
116
+
117
+ Externals::LocalFileSystem.new(config[:dir], configuration.files_dir)
118
+ end
119
+
120
+ def spoon_client
121
+ config = (configuration.spoon_client_config || {}).symbolize_keys
122
+ spoon_args = config.slice(:args, :dir, :kitchen, :pan, :timeout_in_seconds)
123
+ spoon = Pdi::Spoon.new(spoon_args)
124
+
125
+ Externals::SpoonClient.new(configuration.files_dir, spoon)
126
+ end
127
+
128
+ def output_router
129
+ pdi_out = Suite::PdiOutputWriter.new(configuration.results_dir)
130
+ Simmer::Suite::OutputRouter.new(console_out, pdi_out)
131
+ end
132
+ end
133
+ end
@@ -7,14 +7,19 @@
7
7
  # LICENSE file in the root directory of this source tree.
8
8
  #
9
9
 
10
+ require_relative 'configuration/callback_dsl'
11
+
10
12
  module Simmer
11
13
  # Reads in the Simmer configuration file and options and provides it to the rest of the
12
14
  # Simmer implementation.
13
15
  class Configuration
16
+ extend Forwardable
17
+
14
18
  # Configuration Keys
15
19
  AWS_FILE_SYSTEM_KEY = :aws_file_system
16
20
  LOCAL_FILE_SYSTEM_KEY = :local_file_system
17
21
  MYSQL_DATABASE_KEY = :mysql_database
22
+ TIMEOUT_RETRY_KEY = :timeout_failure_retry_count
18
23
  SPOON_CLIENT_KEY = :spoon_client
19
24
 
20
25
  # Paths
@@ -32,10 +37,14 @@ module Simmer
32
37
 
33
38
  attr_reader :config
34
39
 
35
- def initialize(config, simmer_dir, resolver: Objectable.resolver)
40
+ # :nodoc:
41
+ def_delegators :callbacks, :run_single_test_with_callbacks, :run_suite_with_callbacks
42
+
43
+ def initialize(config, simmer_dir, resolver: Objectable.resolver, callbacks: nil)
36
44
  @config = config || {}
37
45
  @resolver = resolver
38
46
  @simmer_dir = simmer_dir
47
+ @callbacks = callbacks
39
48
 
40
49
  freeze
41
50
  end
@@ -60,6 +69,10 @@ module Simmer
60
69
  config.key?(LOCAL_FILE_SYSTEM_KEY.to_s)
61
70
  end
62
71
 
72
+ def timeout_failure_retry_count
73
+ get(TIMEOUT_RETRY_KEY) || 0
74
+ end
75
+
63
76
  def spoon_client_config
64
77
  get(SPOON_CLIENT_KEY) || {}
65
78
  end
@@ -82,7 +95,8 @@ module Simmer
82
95
 
83
96
  private
84
97
 
85
- attr_reader :resolver,
98
+ attr_reader :callbacks,
99
+ :resolver,
86
100
  :simmer_dir,
87
101
  :yaml_reader
88
102