backspin 0.4.5 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: dced94847519f3f0f72a83bf4faf924d1a87271ba04891082f4e403124eda48c
4
- data.tar.gz: d92e797ff52c989b6e1fd4910397d38770fe91b8e5b14aef4924412c11e8a804
3
+ metadata.gz: d9c7a5cfbb0da377abb251f0f53d3bef24f131e72664fde1de6372f02024a506
4
+ data.tar.gz: 3835b7201b3d9ca7d4b399b7f7e58fdfa8e81838e22a45b1f4d7225da2fca09b
5
5
  SHA512:
6
- metadata.gz: d1dc52cdfbc6835548b8cdac5e0ef7ea26b4714a168870fb480f2e67c9d54b1305f5ef93a0942909a685c74b53a40323beb81bc2b79a2c46ad274ed1054f98a1
7
- data.tar.gz: d3fc481d4e52eb5871c1deace41707a654981336c6570ffc07ae847e3d10a94f3ff602c09b764d0e3591841e334f8c8eed53028db217a3c5f1082e50b5493d9a
6
+ metadata.gz: e533519b22a5e4a83a3511eb3ffbfd0429432b75179988435806d006ff57aa7a91e7c3f73c3bb6d1fe2b587f90bef97febdc5952a82f69e46c1b6f7bc26f6087
7
+ data.tar.gz: ddffab372459f5c8ff6f8246cc5ffd988c3a28c5041d56f9961e13f7d5bd0a2c9bfde3bf6d844792e708b332e5acd836afd4d71d041ec1435e7a1644716d2156
data/Gemfile CHANGED
@@ -6,8 +6,8 @@ source "https://rubygems.org"
6
6
  gemspec
7
7
 
8
8
  group :development do
9
- gem "rake", "~> 13.0"
10
- gem "rspec", "~> 3.0"
9
+ gem "rake", "~> 13"
10
+ gem "rspec", "~> 3"
11
11
  gem "standard"
12
12
  gem "timecop", "~> 0.9"
13
13
  end
data/Gemfile.lock CHANGED
@@ -1,20 +1,19 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- backspin (0.4.5)
5
- ostruct (~> 0.5.0)
6
- rspec-mocks (~> 3.0)
4
+ backspin (0.5.0)
5
+ ostruct
6
+ rspec-mocks (~> 3)
7
7
 
8
8
  GEM
9
9
  remote: https://rubygems.org/
10
10
  specs:
11
11
  ast (2.4.3)
12
12
  diff-lcs (1.6.2)
13
- gem-release (2.2.4)
14
13
  json (2.12.2)
15
14
  language_server-protocol (3.17.0.5)
16
15
  lint_roller (1.1.0)
17
- ostruct (0.5.5)
16
+ ostruct (0.6.1)
18
17
  parallel (1.27.0)
19
18
  parser (3.3.8.0)
20
19
  ast (~> 2.4.1)
@@ -48,7 +47,7 @@ GEM
48
47
  rubocop-ast (>= 1.44.0, < 2.0)
49
48
  ruby-progressbar (~> 1.7)
50
49
  unicode-display_width (>= 2.4.0, < 4.0)
51
- rubocop-ast (1.44.1)
50
+ rubocop-ast (1.45.1)
52
51
  parser (>= 3.3.7.2)
53
52
  prism (~> 1.4)
54
53
  rubocop-performance (1.25.0)
@@ -79,9 +78,8 @@ PLATFORMS
79
78
 
80
79
  DEPENDENCIES
81
80
  backspin!
82
- gem-release (~> 2)
83
- rake (~> 13.0)
84
- rspec (~> 3.0)
81
+ rake (~> 13)
82
+ rspec (~> 3)
85
83
  standard
86
84
  timecop (~> 0.9)
87
85
 
data/MATCHERS.md ADDED
@@ -0,0 +1,234 @@
1
+ # Custom Matcher Usage Guide
2
+
3
+ Backspin supports custom matchers for flexible verification of command outputs. This is useful when dealing with dynamic content like timestamps, process IDs, or version numbers.
4
+
5
+ ## Matcher Behavior
6
+
7
+ **Important**: Matchers work as overrides - only the fields you specify will be checked. Fields without matchers are ignored completely.
8
+
9
+ ## Matcher Formats
10
+
11
+ ### 1. Simple Proc Matcher
12
+
13
+ A proc matcher receives full command hashes and can check any combination of fields:
14
+
15
+ ```ruby
16
+ # Check that output starts with expected prefix
17
+ result = Backspin.run("version_test",
18
+ matcher: ->(recorded, actual) {
19
+ recorded["stdout"].start_with?("v") &&
20
+ actual["stdout"].start_with?("v")
21
+ }) do
22
+ Open3.capture3("node --version")
23
+ end
24
+ ```
25
+
26
+ ### 2. Field-Specific Hash Matchers
27
+
28
+ Use a hash to specify matchers for individual fields. Only specified fields are checked:
29
+
30
+ ```ruby
31
+ # Only check stdout - stderr and status are ignored
32
+ result = Backspin.run("timestamp_test",
33
+ matcher: {
34
+ stdout: ->(recorded, actual) {
35
+ # Both should contain a timestamp
36
+ recorded.match?(/\d{4}-\d{2}-\d{2}/) &&
37
+ actual.match?(/\d{4}-\d{2}-\d{2}/)
38
+ }
39
+ }) do
40
+ Open3.capture3("date")
41
+ end
42
+
43
+ # Check multiple specific fields
44
+ result = Backspin.run("api_test",
45
+ matcher: {
46
+ stdout: ->(recorded, actual) {
47
+ # Check JSON structure exists
48
+ recorded.include?("\"data\":") &&
49
+ actual.include?("\"data\":")
50
+ },
51
+ status: ->(recorded, actual) {
52
+ # Both should succeed
53
+ recorded == 0 && actual == 0
54
+ }
55
+ # Note: stderr is NOT checked
56
+ }) do
57
+ Open3.capture3("curl", "-s", "https://api.example.com")
58
+ end
59
+ ```
60
+
61
+ ### 3. The :all Matcher
62
+
63
+ The `:all` matcher receives complete command hashes with all fields:
64
+
65
+ ```ruby
66
+ # Cross-field validation
67
+ result = Backspin.run("build_test",
68
+ matcher: {
69
+ all: ->(recorded, actual) {
70
+ # Check overall success: stdout has "BUILD SUCCESSFUL" AND status is 0
71
+ actual["stdout"].include?("BUILD SUCCESSFUL") &&
72
+ actual["status"] == 0
73
+ }
74
+ }) do
75
+ Open3.capture3("./build.sh")
76
+ end
77
+ ```
78
+
79
+ ### 4. Combining :all with Field Matchers
80
+
81
+ When both `:all` and field matchers are present, all must pass:
82
+
83
+ ```ruby
84
+ result = Backspin.run("complex_test",
85
+ matcher: {
86
+ all: ->(recorded, actual) {
87
+ # Overall check: command succeeded
88
+ actual["status"] == 0
89
+ },
90
+ stdout: ->(recorded, actual) {
91
+ # Specific check: output contains result
92
+ actual.include?("Result:")
93
+ },
94
+ stderr: ->(recorded, actual) {
95
+ # No errors expected
96
+ actual.empty?
97
+ }
98
+ }) do
99
+ Open3.capture3("./process_data.sh")
100
+ end
101
+ ```
102
+
103
+ ## Matcher Proc Arguments
104
+
105
+ Field-specific matchers receive field values:
106
+ - For `:stdout`, `:stderr` - String values
107
+ - For `:status` - Integer exit code
108
+
109
+ The `:all` matcher receives full hashes with these keys:
110
+ - `"stdout"` - String output
111
+ - `"stderr"` - String error output
112
+ - `"status"` - Integer exit code
113
+ - `"command_type"` - String like "Open3::Capture3"
114
+ - `"args"` - Array of command arguments
115
+ - `"recorded_at"` - Timestamp string
116
+
117
+ ## Examples
118
+
119
+ ### Matching Version Numbers
120
+
121
+ ```ruby
122
+ # Match major version only
123
+ matcher: {
124
+ stdout: ->(recorded, actual) {
125
+ recorded_major = recorded[/(\d+)\./, 1]
126
+ actual_major = actual[/(\d+)\./, 1]
127
+ recorded_major == actual_major
128
+ }
129
+ }
130
+ ```
131
+
132
+ ### Ignoring Timestamps
133
+
134
+ ```ruby
135
+ # Strip timestamps before comparing
136
+ matcher: {
137
+ stdout: ->(recorded, actual) {
138
+ pattern = /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/
139
+ recorded.gsub(pattern, '[TIME]') == actual.gsub(pattern, '[TIME]')
140
+ }
141
+ }
142
+ ```
143
+
144
+ ### JSON Response Validation
145
+
146
+ ```ruby
147
+ # Validate JSON structure, ignore values
148
+ matcher: {
149
+ stdout: ->(recorded, actual) {
150
+ begin
151
+ recorded_data = JSON.parse(recorded)
152
+ actual_data = JSON.parse(actual)
153
+
154
+ # Same keys at top level
155
+ recorded_data.keys.sort == actual_data.keys.sort
156
+ rescue JSON::ParserError
157
+ false
158
+ end
159
+ }
160
+ }
161
+ ```
162
+
163
+ ### Logging/Debugging with :all
164
+
165
+ ```ruby
166
+ # Use :all for side effects while other matchers do validation
167
+ logged_commands = []
168
+
169
+ result = Backspin.run("debug_test",
170
+ matcher: {
171
+ all: ->(recorded, actual) {
172
+ # Log for debugging
173
+ logged_commands << {
174
+ args: actual["args"],
175
+ exit: actual["status"],
176
+ output_size: actual["stdout"].size
177
+ }
178
+ true # Always pass
179
+ },
180
+ stdout: ->(recorded, actual) {
181
+ # Actual validation
182
+ actual.include?("SUCCESS")
183
+ }
184
+ }) do
185
+ Open3.capture3("./test.sh")
186
+ end
187
+
188
+ # Can inspect logged_commands after run
189
+ ```
190
+
191
+ ## Using with run!
192
+
193
+ The `run!` method automatically fails tests when matchers return false:
194
+
195
+ ```ruby
196
+ # This will raise an error if the matcher fails
197
+ Backspin.run!("critical_test",
198
+ matcher: {
199
+ stdout: ->(r, a) { a.include?("OK") },
200
+ status: ->(r, a) { a == 0 }
201
+ }) do
202
+ Open3.capture3("./health_check.sh")
203
+ end
204
+ ```
205
+
206
+ ## Migration from match_on
207
+
208
+ The `match_on` option is deprecated. To migrate:
209
+
210
+ ```ruby
211
+ # Old match_on style:
212
+ Backspin.run("test",
213
+ match_on: [:stdout, ->(r, a) { ... }])
214
+
215
+ # New matcher style:
216
+ Backspin.run("test",
217
+ matcher: { stdout: ->(r, a) { ... } })
218
+
219
+ # Old match_on with multiple fields:
220
+ Backspin.run("test",
221
+ match_on: [
222
+ [:stdout, ->(r, a) { ... }],
223
+ [:stderr, ->(r, a) { ... }]
224
+ ])
225
+
226
+ # New matcher style:
227
+ Backspin.run("test",
228
+ matcher: {
229
+ stdout: ->(r, a) { ... },
230
+ stderr: ->(r, a) { ... }
231
+ })
232
+ ```
233
+
234
+ Key difference: `match_on` required other fields to match exactly, while the new `matcher` hash only checks specified fields.
data/README.md CHANGED
@@ -5,11 +5,11 @@
5
5
  [![CircleCI](https://img.shields.io/circleci/build/github/rsanheim/backspin/main)](https://circleci.com/gh/rsanheim/backspin)
6
6
  [![Last Commit](https://img.shields.io/github/last-commit/rsanheim/backspin/main)](https://github.com/rsanheim/backspin/commits/main)
7
7
 
8
- Backspin records and replays CLI interactions in Ruby for easy snapshot testing of command-line interfaces. Currently supports `Open3.capture3` and `system` and requires `rspec-mocks`. More system calls and test integrations are welcome - send a PR!
8
+ Backspin records and replays CLI interactions in Ruby for easy snapshot testing of command-line interfaces. Currently supports `Open3.capture3` and `system` and requires `rspec`, as it uses `rspec-mocks` under the hood.
9
9
 
10
- **NOTE:** Backspin should be considered alpha quality software while pre v1.0. It is in heavy development, and you can expect the API to change. It is being developed in conjunction with production CLI apps, so the API will be refined and improved as we get to 1.0.
10
+ **NOTE:** Backspin should be considered alpha while pre version 1.0. It is in heavy development along-side some real-world CLI apps, so expect things to change and mature.
11
11
 
12
- Inspired by [VCR](https://github.com/vcr/vcr) and other [golden master](https://en.wikipedia.org/wiki/Golden_master_(software_development)) libraries.
12
+ Inspired by [VCR](https://github.com/vcr/vcr) and other [golden master](https://en.wikipedia.org/wiki/Golden_master_(software_development)) testing libraries.
13
13
 
14
14
  ## Overview
15
15
 
@@ -17,6 +17,8 @@ Backspin is a Ruby library for snapshot testing (or characterization testing) of
17
17
 
18
18
  ## Installation
19
19
 
20
+ Requires Ruby 3+ and will use rspec-mocks under the hood...Backspin has not been tested in other test frameworks.
21
+
20
22
  Add this line to your application's Gemfile in the `:test` group:
21
23
 
22
24
  ```ruby
@@ -95,20 +97,47 @@ end
95
97
 
96
98
  ### Custom matchers
97
99
 
98
- For cases where exact matching isn't suitable, you can provide custom verification logic:
100
+ For cases where full matching isn't suitable, you can override via `matcher:`. **NOTE**: If you provide
101
+ custom matchers, that is the only matching that will be done. Default matching is skipped if user-provided
102
+ matchers are present.
103
+
104
+ You can override the full match logic with a proc:
99
105
 
100
106
  ```ruby
101
- # Use custom logic to verify output
102
- result = Backspin.run("version_check",
103
- matcher: ->(recorded, actual) {
104
- # Just check that both start with "ruby"
105
- recorded["stdout"].start_with?("ruby") &&
106
- actual["stdout"].start_with?("ruby")
107
- }) do
108
- Open3.capture3("ruby --version")
107
+ # Match stdout and status, ignore stderr
108
+ my_matcher = ->(recorded, actual) {
109
+ recorded["stdout"] == actual["stdout"] && recorded["status"] != actual["status"]
110
+ }
111
+
112
+ result = Backspin.run("my_test", matcher: { all: my_matcher }) do
113
+ Open3.capture3("echo hello")
109
114
  end
110
115
  ```
111
116
 
117
+ Or you can override specific fields:
118
+
119
+ ```ruby
120
+ # Match dynamic timestamps in stdout
121
+ timestamp_matcher = ->(recorded, actual) {
122
+ recorded.match?(/\d{4}-\d{2}-\d{2}/) && actual.match?(/\d{4}-\d{2}-\d{2}/)
123
+ }
124
+
125
+ result = Backspin.run("timestamp_test", matcher: { stdout: timestamp_matcher }) do
126
+ Open3.capture3("date")
127
+ end
128
+
129
+ # Match version numbers in stderr
130
+ version_matcher = ->(recorded, actual) {
131
+ recorded[/v(\d+)\./, 1] == actual[/v(\d+)\./, 1]
132
+ }
133
+
134
+ result = Backspin.run("version_check", matcher: { stderr: version_matcher }) do
135
+ Open3.capture3("node --version")
136
+ end
137
+ ```
138
+
139
+ For more matcher examples and detailed documentation, see [MATCHERS.md](MATCHERS.md).
140
+
112
141
  ### Working with the Result Object
113
142
 
114
143
  The API returns a `RecordResult` object with helpful methods:
@@ -174,10 +203,10 @@ When verifying multiple commands, Backspin ensures all commands match in the exa
174
203
 
175
204
  If the CLI interaction you are recording contains sensitive data in stdout or stderr, you should be careful to make sure it is not recorded to yaml!
176
205
 
177
- By default, Backspin automatically scrubs [common credential patterns](https://github.com/rsanheim/backspin/blob/f8661f084aad0ae759cd971c4af31ccf9bdc6bba/lib/backspin.rb#L46-L65) from records, but this will only handle some common cases.
206
+ By default, Backspin automatically tries to scrub [common credential patterns](https://github.com/rsanheim/backspin/blob/f8661f084aad0ae759cd971c4af31ccf9bdc6bba/lib/backspin.rb#L46-L65) from records, but this will only handle some common cases.
178
207
  Always review your record files before commiting them to source control.
179
208
 
180
- A tool like [trufflehog](https://github.com/trufflesecurity/trufflehog) or [gitleaks](https://github.com/gitleaks/gitleaks) run via a pre-commit to catch any sensitive data before commit.
209
+ Use a tool like [trufflehog](https://github.com/trufflesecurity/trufflehog) or [gitleaks](https://github.com/gitleaks/gitleaks) run via a pre-commit to catch any sensitive data before commit.
181
210
 
182
211
  ```ruby
183
212
  # This will automatically scrub AWS keys, API tokens, passwords, etc.
@@ -194,7 +223,6 @@ end
194
223
  Backspin.configure do |config|
195
224
  config.scrub_credentials = false
196
225
  end
197
-
198
226
  ```
199
227
 
200
228
  Automatic scrubbing includes:
@@ -203,16 +231,6 @@ Automatic scrubbing includes:
203
231
  - Generic API keys, auth tokens, and passwords
204
232
  - Private keys (RSA, etc.)
205
233
 
206
- ## Features
207
-
208
- - **Simple recording**: Capture stdout, stderr, and exit status
209
- - **Flexible verification**: Strict matching, playback mode, or custom matchers
210
- - **Auto-naming**: Automatically generate record names from RSpec examples
211
- - **Multiple commands**: Record sequences of commands in a single record
212
- - **RSpec integration**: Works seamlessly with RSpec's mocking framework
213
- - **Human-readable**: YAML records are easy to read and edit
214
- - **Credential scrubbing**: Automatically removes sensitive data like API keys and passwords
215
-
216
234
  ## Development
217
235
 
218
236
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests.
data/backspin.gemspec CHANGED
@@ -25,8 +25,6 @@ Gem::Specification.new do |spec|
25
25
  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
26
26
  spec.require_paths = ["lib"]
27
27
 
28
- spec.add_dependency "ostruct", "~> 0.5.0"
29
- spec.add_dependency "rspec-mocks", "~> 3.0"
30
-
31
- spec.add_development_dependency "gem-release", "~> 2"
28
+ spec.add_dependency "ostruct"
29
+ spec.add_dependency "rspec-mocks", "~> 3"
32
30
  end
@@ -5,7 +5,7 @@ commands:
5
5
  - command_type: Open3::Capture3
6
6
  args:
7
7
  - date
8
- stdout: 'Tue Jun 10 22:50:14 CDT 2025
8
+ stdout: 'Wed Jun 11 02:48:13 CDT 2025
9
9
 
10
10
  '
11
11
  stderr: ''
@@ -9,23 +9,18 @@ module Backspin
9
9
  def initialize(recorded_command:, actual_command:, matcher: nil)
10
10
  @recorded_command = recorded_command
11
11
  @actual_command = actual_command
12
- @matcher = normalize_matcher(matcher)
12
+ @matcher = Matcher.new(
13
+ config: matcher,
14
+ recorded_command: recorded_command,
15
+ actual_command: actual_command
16
+ )
13
17
  end
14
18
 
15
19
  # @return [Boolean] true if the command output matches
16
20
  def verified?
17
- # First check if method classes match
18
21
  return false unless method_classes_match?
19
22
 
20
- if matcher.nil?
21
- recorded_command.result == actual_command.result
22
- elsif matcher.is_a?(Proc) # basic all matcher: lambda { |recorded, actual| ...}
23
- matcher.call(recorded_command.to_h, actual_command.to_h)
24
- elsif matcher.is_a?(Hash) # matcher: {all: lambda { |recorded, actual| ...}, stdout: lambda { |recorded, actual| ...}}
25
- verify_with_hash_matcher
26
- else
27
- raise ArgumentError, "Invalid matcher type: #{matcher.class}"
28
- end
23
+ @matcher.match?
29
24
  end
30
25
 
31
26
  # @return [String, nil] Human-readable diff if not verified
@@ -34,7 +29,6 @@ module Backspin
34
29
 
35
30
  parts = []
36
31
 
37
- # Check method class mismatch first
38
32
  unless method_classes_match?
39
33
  parts << "Command type mismatch: expected #{recorded_command.method_class.name}, got #{actual_command.method_class.name}"
40
34
  end
@@ -65,78 +59,12 @@ module Backspin
65
59
  recorded_command.method_class == actual_command.method_class
66
60
  end
67
61
 
68
- def normalize_matcher(matcher)
69
- return nil if matcher.nil?
70
- return matcher if matcher.is_a?(Proc)
71
-
72
- raise ArgumentError, "Matcher must be a Proc or Hash, got #{matcher.class}" unless matcher.is_a?(Hash)
73
-
74
- # Validate hash keys and values
75
- matcher.each do |key, value|
76
- unless %i[all stdout stderr status].include?(key)
77
- raise ArgumentError, "Invalid matcher key: #{key}. Must be one of: :all, :stdout, :stderr, :status"
78
- end
79
- raise ArgumentError, "Matcher for #{key} must be callable (Proc/Lambda)" unless value.respond_to?(:call)
80
- end
81
- matcher
82
- end
83
-
84
- def verify_with_hash_matcher
85
- recorded_hash = recorded_command.to_h
86
- actual_hash = actual_command.to_h
87
-
88
- all_passed = matcher[:all].nil? || matcher[:all].call(recorded_hash, actual_hash)
89
-
90
- fields_passed = %w[stdout stderr status].all? do |field|
91
- field_sym = field.to_sym
92
- if matcher[field_sym]
93
- matcher[field_sym].call(recorded_hash[field], actual_hash[field])
94
- else
95
- recorded_hash[field] == actual_hash[field]
96
- end
97
- end
98
-
99
- all_passed && fields_passed
100
- end
101
-
102
62
  def failure_reason
103
- reasons = []
104
-
105
- # Check method class first
106
63
  unless method_classes_match?
107
- reasons << "command type mismatch"
108
- return reasons.join(", ")
109
- end
110
-
111
- if matcher.nil?
112
- reasons << "stdout differs" if recorded_command.stdout != actual_command.stdout
113
- reasons << "stderr differs" if recorded_command.stderr != actual_command.stderr
114
- reasons << "exit status differs" if recorded_command.status != actual_command.status
115
- elsif matcher.is_a?(Hash)
116
- recorded_hash = recorded_command.to_h
117
- actual_hash = actual_command.to_h
118
-
119
- # Check :all matcher first
120
- reasons << ":all matcher failed" if matcher[:all] && !matcher[:all].call(recorded_hash, actual_hash)
121
-
122
- # Check field-specific matchers
123
- %w[stdout stderr status].each do |field|
124
- field_sym = field.to_sym
125
- if matcher[field_sym]
126
- unless matcher[field_sym].call(recorded_hash[field], actual_hash[field])
127
- reasons << "#{field} custom matcher failed"
128
- end
129
- elsif recorded_hash[field] != actual_hash[field]
130
- # Always check exact equality for fields without matchers
131
- reasons << "#{field} differs"
132
- end
133
- end
134
- else
135
- # Proc matcher
136
- reasons << "custom matcher failed"
64
+ return "command type mismatch"
137
65
  end
138
66
 
139
- reasons.join(", ")
67
+ @matcher.failure_reason
140
68
  end
141
69
 
142
70
  def stdout_diff
@@ -0,0 +1,57 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pathname"
4
+
5
+ module Backspin
6
+ # Configuration for Backspin
7
+ class Configuration
8
+ attr_accessor :scrub_credentials
9
+ # The directory where backspin will store its files - defaults to fixtures/backspin
10
+ attr_accessor :backspin_dir
11
+ # Regex patterns to scrub from saved output
12
+ attr_reader :credential_patterns
13
+
14
+ def initialize
15
+ @scrub_credentials = true
16
+ @credential_patterns = default_credential_patterns
17
+ @backspin_dir = Pathname(Dir.pwd).join("fixtures", "backspin")
18
+ end
19
+
20
+ def add_credential_pattern(pattern)
21
+ @credential_patterns << pattern
22
+ end
23
+
24
+ def clear_credential_patterns
25
+ @credential_patterns = []
26
+ end
27
+
28
+ def reset_credential_patterns
29
+ @credential_patterns = default_credential_patterns
30
+ end
31
+
32
+ private
33
+
34
+ # Some default patterns for common credential types
35
+ def default_credential_patterns
36
+ [
37
+ # AWS credentials
38
+ /AKIA[0-9A-Z]{16}/, # AWS Access Key ID
39
+ %r{aws_secret_access_key\s*[:=]\s*["']?([A-Za-z0-9/+=]{40})["']?}i, # AWS Secret Key
40
+ %r{aws_session_token\s*[:=]\s*["']?([A-Za-z0-9/+=]+)["']?}i, # AWS Session Token
41
+
42
+ # Google Cloud credentials
43
+ /AIza[0-9A-Za-z\-_]{35}/, # Google API Key
44
+ /[0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com/, # Google OAuth2 client ID
45
+ /-----BEGIN (RSA )?PRIVATE KEY-----/, # Private keys
46
+
47
+ # Generic patterns
48
+ /api[_-]?key\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i, # Generic API keys
49
+ /auth[_-]?token\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i, # Auth tokens
50
+ /Bearer\s+([A-Za-z0-9\-_]+)/, # Bearer tokens
51
+ /password\s*[:=]\s*["']?([^"'\s]{8,})["']?/i, # Passwords
52
+ /-p([^"'\s]{8,})/, # MySQL-style password args
53
+ /secret\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i # Generic secrets
54
+ ]
55
+ end
56
+ end
57
+ end
@@ -0,0 +1,110 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Backspin
4
+ # Handles matching logic between recorded and actual commands
5
+ class Matcher
6
+ attr_reader :config, :recorded_command, :actual_command
7
+
8
+ def initialize(config:, recorded_command:, actual_command:)
9
+ @config = normalize_config(config)
10
+ @recorded_command = recorded_command
11
+ @actual_command = actual_command
12
+ end
13
+
14
+ # @return [Boolean] true if commands match according to the configured matcher
15
+ def match?
16
+ if config.nil?
17
+ # Default behavior: check all fields for equality
18
+ default_matcher.call(recorded_command.to_h, actual_command.to_h)
19
+ elsif config.is_a?(Proc)
20
+ config.call(recorded_command.to_h, actual_command.to_h)
21
+ elsif config.is_a?(Hash)
22
+ verify_with_hash_matcher
23
+ else
24
+ raise ArgumentError, "Invalid matcher type: #{config.class}"
25
+ end
26
+ end
27
+
28
+ # @return [String] reason why matching failed
29
+ def failure_reason
30
+ reasons = []
31
+
32
+ if config.nil?
33
+ # Default matcher checks all fields
34
+ recorded_hash = recorded_command.to_h
35
+ actual_hash = actual_command.to_h
36
+
37
+ reasons << "stdout differs" if recorded_hash["stdout"] != actual_hash["stdout"]
38
+ reasons << "stderr differs" if recorded_hash["stderr"] != actual_hash["stderr"]
39
+ reasons << "exit status differs" if recorded_hash["status"] != actual_hash["status"]
40
+ elsif config.is_a?(Hash)
41
+ recorded_hash = recorded_command.to_h
42
+ actual_hash = actual_command.to_h
43
+
44
+ # Only check matchers that were provided
45
+ config.each do |field, matcher_proc|
46
+ case field
47
+ when :all
48
+ reasons << ":all matcher failed" unless matcher_proc.call(recorded_hash, actual_hash)
49
+ when :stdout, :stderr, :status
50
+ unless matcher_proc.call(recorded_hash[field.to_s], actual_hash[field.to_s])
51
+ reasons << "#{field} custom matcher failed"
52
+ end
53
+ end
54
+ end
55
+ else
56
+ # Proc matcher
57
+ reasons << "custom matcher failed"
58
+ end
59
+
60
+ reasons.join(", ")
61
+ end
62
+
63
+ private
64
+
65
+ def normalize_config(config)
66
+ return nil if config.nil?
67
+ return config if config.is_a?(Proc)
68
+
69
+ raise ArgumentError, "Matcher must be a Proc or Hash, got #{config.class}" unless config.is_a?(Hash)
70
+
71
+ # Validate hash keys and values
72
+ config.each do |key, value|
73
+ unless %i[all stdout stderr status].include?(key)
74
+ raise ArgumentError, "Invalid matcher key: #{key}. Must be one of: :all, :stdout, :stderr, :status"
75
+ end
76
+ raise ArgumentError, "Matcher for #{key} must be callable (Proc/Lambda)" unless value.respond_to?(:call)
77
+ end
78
+ config
79
+ end
80
+
81
+ def verify_with_hash_matcher
82
+ recorded_hash = recorded_command.to_h
83
+ actual_hash = actual_command.to_h
84
+
85
+ # Override-based: only run matchers that are explicitly provided
86
+ # Use map to ensure all matchers run, then check if all passed
87
+ results = config.map do |field, matcher_proc|
88
+ case field
89
+ when :all
90
+ matcher_proc.call(recorded_hash, actual_hash)
91
+ when :stdout, :stderr, :status
92
+ matcher_proc.call(recorded_hash[field.to_s], actual_hash[field.to_s])
93
+ else
94
+ # This should never happen due to normalize_config validation
95
+ raise ArgumentError, "Unknown field: #{field}"
96
+ end
97
+ end
98
+
99
+ results.all?
100
+ end
101
+
102
+ def default_matcher
103
+ @default_matcher ||= lambda do |recorded, actual|
104
+ recorded["stdout"] == actual["stdout"] &&
105
+ recorded["stderr"] == actual["stderr"] &&
106
+ recorded["status"] == actual["status"]
107
+ end
108
+ end
109
+ end
110
+ end
@@ -12,12 +12,13 @@ module Backspin
12
12
  include RSpec::Mocks::ExampleMethods
13
13
  SUPPORTED_COMMAND_TYPES = %i[capture3 system].freeze
14
14
 
15
- attr_reader :commands, :mode, :record, :options
15
+ attr_reader :commands, :mode, :record, :matcher, :filter
16
16
 
17
- def initialize(mode: :record, record: nil, options: {})
17
+ def initialize(mode: :record, record: nil, matcher: nil, filter: nil)
18
18
  @mode = mode
19
19
  @record = record
20
- @options = options
20
+ @matcher = matcher
21
+ @filter = filter
21
22
  @commands = []
22
23
  @playback_index = 0
23
24
  @command_diffs = []
@@ -45,14 +46,14 @@ module Backspin
45
46
  # Records registered commands, adds them to the record, saves the record, and returns the overall RecordResult
46
47
  def perform_recording
47
48
  result = yield
48
- record.save(filter: options[:filter])
49
+ record.save(filter: @filter)
49
50
  RecordResult.new(output: result, mode: :record, record: record)
50
51
  end
51
52
 
52
53
  # Performs verification by executing commands and comparing with recorded values
53
54
  def perform_verification
54
55
  raise RecordNotFoundError, "Record not found: #{record.path}" unless record.exists?
55
- raise RecordNotFoundError, "No commands found in record" if record.empty?
56
+ raise RecordNotFoundError, "No commands found in record #{record.path}" if record.empty?
56
57
 
57
58
  # Initialize tracking variables
58
59
  @command_diffs = []
@@ -76,7 +77,7 @@ module Backspin
76
77
  status: status.exitstatus
77
78
  )
78
79
 
79
- @command_diffs << CommandDiff.new(recorded_command: recorded_command, actual_command: actual_command, matcher: options[:matcher])
80
+ @command_diffs << CommandDiff.new(recorded_command: recorded_command, actual_command: actual_command, matcher: @matcher)
80
81
  @command_index += 1
81
82
  [stdout, stderr, status]
82
83
  end
@@ -96,7 +97,7 @@ module Backspin
96
97
  )
97
98
 
98
99
  # Create CommandDiff to track the comparison
99
- @command_diffs << CommandDiff.new(recorded_command: recorded_command, actual_command: actual_command, matcher: options[:matcher])
100
+ @command_diffs << CommandDiff.new(recorded_command: recorded_command, actual_command: actual_command, matcher: @matcher)
100
101
 
101
102
  @command_index += 1
102
103
  result
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Backspin
4
- VERSION = "0.4.5"
4
+ VERSION = "0.5.0"
5
5
  end
data/lib/backspin.rb CHANGED
@@ -7,8 +7,10 @@ require "pathname"
7
7
  require "ostruct"
8
8
  require "rspec/mocks"
9
9
  require "backspin/version"
10
+ require "backspin/configuration"
10
11
  require "backspin/command_result"
11
12
  require "backspin/command"
13
+ require "backspin/matcher"
12
14
  require "backspin/command_diff"
13
15
  require "backspin/record"
14
16
  require "backspin/recorder"
@@ -20,58 +22,6 @@ module Backspin
20
22
  # Include RSpec mocks methods
21
23
  extend RSpec::Mocks::ExampleMethods
22
24
 
23
- # Configuration for Backspin
24
- class Configuration
25
- attr_accessor :scrub_credentials
26
- # The directory where backspin will store its files - defaults to fixtures/backspin
27
- attr_accessor :backspin_dir
28
- # Regex patterns to scrub from saved output
29
- attr_reader :credential_patterns
30
-
31
- def initialize
32
- @scrub_credentials = true
33
- @credential_patterns = default_credential_patterns
34
- @backspin_dir = Pathname(Dir.pwd).join("fixtures", "backspin")
35
- end
36
-
37
- def add_credential_pattern(pattern)
38
- @credential_patterns << pattern
39
- end
40
-
41
- def clear_credential_patterns
42
- @credential_patterns = []
43
- end
44
-
45
- def reset_credential_patterns
46
- @credential_patterns = default_credential_patterns
47
- end
48
-
49
- private
50
-
51
- # Some default patterns for common credential types
52
- def default_credential_patterns
53
- [
54
- # AWS credentials
55
- /AKIA[0-9A-Z]{16}/, # AWS Access Key ID
56
- %r{aws_secret_access_key\s*[:=]\s*["']?([A-Za-z0-9/+=]{40})["']?}i, # AWS Secret Key
57
- %r{aws_session_token\s*[:=]\s*["']?([A-Za-z0-9/+=]+)["']?}i, # AWS Session Token
58
-
59
- # Google Cloud credentials
60
- /AIza[0-9A-Za-z\-_]{35}/, # Google API Key
61
- /[0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com/, # Google OAuth2 client ID
62
- /-----BEGIN (RSA )?PRIVATE KEY-----/, # Private keys
63
-
64
- # Generic patterns
65
- /api[_-]?key\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i, # Generic API keys
66
- /auth[_-]?token\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i, # Auth tokens
67
- /Bearer\s+([A-Za-z0-9\-_]+)/, # Bearer tokens
68
- /password\s*[:=]\s*["']?([^"'\s]{8,})["']?/i, # Passwords
69
- /-p([^"'\s]{8,})/, # MySQL-style password args
70
- /secret\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i # Generic secrets
71
- ]
72
- end
73
- end
74
-
75
25
  class << self
76
26
  def configuration
77
27
  return @configuration if @configuration
@@ -102,22 +52,21 @@ module Backspin
102
52
  # Primary API - records on first run, verifies on subsequent runs
103
53
  #
104
54
  # @param record_name [String] Name for the record file
105
- # @param options [Hash] Options for recording/verification
106
- # @option options [Symbol] :mode (:auto) Recording mode - :auto, :record, :verify, :playback
107
- # @option options [Proc] :filter Custom filter for recorded data
108
- # @option options [Proc, Hash] :matcher Custom matcher for verification
55
+ # @param mode [Symbol] Recording mode - :auto, :record, :verify, :playback
56
+ # @param matcher [Proc, Hash] Custom matcher for verification
109
57
  # - Proc: ->(recorded, actual) { ... } for full command matching
110
58
  # - Hash: { stdout: ->(recorded, actual) { ... }, stderr: ->(recorded, actual) { ... } } for field-specific matching
111
- # - Hash with :all key: { all: ->(recorded, actual) { ... }, stdout: ->(recorded, actual) { ... } } for combined matching
112
- # When both :all and field matchers are present, both must pass for verification to succeed.
113
- # Fields without specific matchers always use exact equality, regardless of :all presence.
59
+ # Only specified fields are checked - fields without matchers are ignored
60
+ # - Hash with :all key: { all: ->(recorded, actual) { ... } } receives full command hashes
61
+ # Can be combined with field matchers - all specified matchers must pass
62
+ # @param filter [Proc] Custom filter for recorded data
114
63
  # @return [RecordResult] Result object with output and status
115
- def run(record_name, options = {}, &block)
64
+ def run(record_name, mode: :auto, matcher: nil, filter: nil, &block)
116
65
  raise ArgumentError, "record_name is required" if record_name.nil? || record_name.empty?
117
66
  raise ArgumentError, "block is required" unless block_given?
118
67
 
119
68
  record_path = Record.build_record_path(record_name)
120
- mode = determine_mode(options[:mode], record_path)
69
+ mode = determine_mode(mode, record_path)
121
70
 
122
71
  # Create or load the record based on mode
123
72
  record = if mode == :record
@@ -127,7 +76,7 @@ module Backspin
127
76
  end
128
77
 
129
78
  # Create recorder with all needed context
130
- recorder = Recorder.new(record: record, options: options, mode: mode)
79
+ recorder = Recorder.new(record: record, mode: mode, matcher: matcher, filter: filter)
131
80
 
132
81
  # Execute the appropriate mode
133
82
  case mode
@@ -146,11 +95,13 @@ module Backspin
146
95
  # Strict version of run that raises on verification failure
147
96
  #
148
97
  # @param record_name [String] Name for the record file
149
- # @param options [Hash] Options for recording/verification
98
+ # @param mode [Symbol] Recording mode - :auto, :record, :verify, :playback
99
+ # @param matcher [Proc, Hash] Custom matcher for verification
100
+ # @param filter [Proc] Custom filter for recorded data
150
101
  # @return [RecordResult] Result object with output and status
151
102
  # @raise [RSpec::Expectations::ExpectationNotMetError] If verification fails
152
- def run!(record_name, options = {}, &block)
153
- result = run(record_name, options, &block)
103
+ def run!(record_name, mode: :auto, matcher: nil, filter: nil, &block)
104
+ result = run(record_name, mode: mode, matcher: matcher, filter: filter, &block)
154
105
 
155
106
  if result.verified? == false
156
107
  error_message = "Backspin verification failed!\n"
data/release.rake CHANGED
@@ -23,12 +23,13 @@ namespace :release do
23
23
  sh "git commit -am 'Bump version to #{new_version}'"
24
24
  sh "git push"
25
25
 
26
- sh "gem release --tag --github --push"
26
+ sh "gem release --tag --push"
27
+ Rake::Task["release:github"].invoke(new_version)
27
28
  end
28
29
 
29
- desc "Create GitHub release for current version"
30
- task :github do
31
- version = Backspin::VERSION
30
+ desc "Create GitHub release for specified version or current version"
31
+ task :github, [:version] do |t, args|
32
+ version = args[:version] || Backspin::VERSION
32
33
 
33
34
  if system("which gh > /dev/null 2>&1")
34
35
  puts "\nCreating GitHub release for v#{version}..."
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: backspin
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.5
4
+ version: 0.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Rob Sanheim
@@ -13,44 +13,30 @@ dependencies:
13
13
  name: ostruct
14
14
  requirement: !ruby/object:Gem::Requirement
15
15
  requirements:
16
- - - "~>"
16
+ - - ">="
17
17
  - !ruby/object:Gem::Version
18
- version: 0.5.0
18
+ version: '0'
19
19
  type: :runtime
20
20
  prerelease: false
21
21
  version_requirements: !ruby/object:Gem::Requirement
22
22
  requirements:
23
- - - "~>"
23
+ - - ">="
24
24
  - !ruby/object:Gem::Version
25
- version: 0.5.0
25
+ version: '0'
26
26
  - !ruby/object:Gem::Dependency
27
27
  name: rspec-mocks
28
28
  requirement: !ruby/object:Gem::Requirement
29
29
  requirements:
30
30
  - - "~>"
31
31
  - !ruby/object:Gem::Version
32
- version: '3.0'
32
+ version: '3'
33
33
  type: :runtime
34
34
  prerelease: false
35
35
  version_requirements: !ruby/object:Gem::Requirement
36
36
  requirements:
37
37
  - - "~>"
38
38
  - !ruby/object:Gem::Version
39
- version: '3.0'
40
- - !ruby/object:Gem::Dependency
41
- name: gem-release
42
- requirement: !ruby/object:Gem::Requirement
43
- requirements:
44
- - - "~>"
45
- - !ruby/object:Gem::Version
46
- version: '2'
47
- type: :development
48
- prerelease: false
49
- version_requirements: !ruby/object:Gem::Requirement
50
- requirements:
51
- - - "~>"
52
- - !ruby/object:Gem::Version
53
- version: '2'
39
+ version: '3'
54
40
  description: Backspin is a Ruby library for characterization testing of command-line
55
41
  interfaces. Inspired by VCR's cassette-based approach, it records and replays CLI
56
42
  interactions to make testing faster and more deterministic.
@@ -71,7 +57,7 @@ files:
71
57
  - Gemfile
72
58
  - Gemfile.lock
73
59
  - LICENSE.txt
74
- - MATCH_ON_USAGE.md
60
+ - MATCHERS.md
75
61
  - README.md
76
62
  - Rakefile
77
63
  - backspin.gemspec
@@ -138,6 +124,8 @@ files:
138
124
  - lib/backspin/command.rb
139
125
  - lib/backspin/command_diff.rb
140
126
  - lib/backspin/command_result.rb
127
+ - lib/backspin/configuration.rb
128
+ - lib/backspin/matcher.rb
141
129
  - lib/backspin/record.rb
142
130
  - lib/backspin/record_result.rb
143
131
  - lib/backspin/recorder.rb
data/MATCH_ON_USAGE.md DELETED
@@ -1,110 +0,0 @@
1
- # Using match_on for Field-Specific Verification
2
-
3
- The `match_on` option allows you to use custom matchers for specific fields while maintaining exact equality checks for all other fields. This is useful when dealing with dynamic content like timestamps, process IDs, or version numbers.
4
-
5
- ## Basic Usage
6
-
7
- ### Single Field Matcher
8
-
9
- ```ruby
10
- # Record a command with a timestamp
11
- Backspin.run("timestamp_test") do
12
- Open3.capture3("date")
13
- end
14
-
15
- # Verify with a custom matcher for stdout
16
- result = Backspin.run("timestamp_test",
17
- match_on: [:stdout, ->(recorded, actual) {
18
- # Both should contain a day of the week
19
- recorded.match?(/Mon|Tue|Wed|Thu|Fri|Sat|Sun/) &&
20
- actual.match?(/Mon|Tue|Wed|Thu|Fri|Sat|Sun/)
21
- }]) do
22
- Open3.capture3("date")
23
- end
24
- ```
25
-
26
- ### Multiple Field Matchers
27
-
28
- ```ruby
29
- # Match different fields with different rules
30
- result = Backspin.run("multi_field_test",
31
- match_on: [
32
- [:stdout, ->(recorded, actual) {
33
- # Match process ID format
34
- recorded.match?(/PID: \d+/) && actual.match?(/PID: \d+/)
35
- }],
36
- [:stderr, ->(recorded, actual) {
37
- # Match error type, ignore details
38
- recorded.include?("Error:") && actual.include?("Error:")
39
- }]
40
- ]) do
41
- Open3.capture3("./my_script.sh")
42
- end
43
- ```
44
-
45
- ## Matcher Format
46
-
47
- The `match_on` option accepts two formats:
48
-
49
- 1. **Single field**: `[:field_name, matcher_proc]`
50
- 2. **Multiple fields**: `[[:field1, matcher1], [:field2, matcher2], ...]`
51
-
52
- Valid field names are:
53
- - `:stdout` - Standard output
54
- - `:stderr` - Standard error
55
- - `:status` - Exit status code
56
-
57
- ## Matcher Proc
58
-
59
- The matcher proc receives two arguments:
60
- - `recorded_value` - The value from the saved recording
61
- - `actual_value` - The value from the current execution
62
-
63
- It should return `true` if the values match according to your criteria, `false` otherwise.
64
-
65
- ## Examples
66
-
67
- ### Matching Version Numbers
68
-
69
- ```ruby
70
- # Match major version only
71
- match_on: [:stdout, ->(recorded, actual) {
72
- recorded.match(/Version: (\d+)\./) &&
73
- actual.match(/Version: (\d+)\./) &&
74
- $1 == $1 # Major versions match
75
- }]
76
- ```
77
-
78
- ### Ignoring Timestamps
79
-
80
- ```ruby
81
- # Match log format but ignore timestamp
82
- match_on: [:stdout, ->(recorded, actual) {
83
- # Remove timestamps before comparing
84
- recorded.gsub(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/, '') ==
85
- actual.gsub(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/, '')
86
- }]
87
- ```
88
-
89
- ### Handling Dynamic IDs
90
-
91
- ```ruby
92
- # Match API response structure, ignore dynamic IDs
93
- match_on: [:stdout, ->(recorded, actual) {
94
- recorded_json = JSON.parse(recorded)
95
- actual_json = JSON.parse(actual)
96
-
97
- # Compare structure, not values
98
- recorded_json.keys.sort == actual_json.keys.sort
99
- }]
100
- ```
101
-
102
- ## Important Notes
103
-
104
- 1. **Other fields must match exactly**: When using `match_on`, all fields not specified in the matcher list must match exactly. If stdout has a custom matcher but stderr doesn't, stderr must be identical to pass verification.
105
-
106
- 2. **Precedence**: If both `matcher` and `match_on` options are provided, `matcher` takes precedence (for backward compatibility).
107
-
108
- 3. **Error messages**: When verification fails with `match_on`, the error will indicate which fields failed and whether they failed exact matching or custom matching.
109
-
110
- 4. **Works with run!**: The `match_on` option works with both `run` and `run!` methods.