backspin 0.7.1 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.circleci/config.yml +18 -6
- data/.ruby-version +1 -0
- data/CHANGELOG.md +8 -1
- data/CLAUDE.md +8 -9
- data/CONTRIBUTING.md +5 -12
- data/Gemfile.lock +3 -6
- data/MATCHERS.md +28 -136
- data/README.md +56 -120
- data/backspin.gemspec +0 -3
- data/examples/match_on_example.rb +42 -71
- data/lib/backspin/command.rb +32 -21
- data/lib/backspin/command_diff.rb +14 -8
- data/lib/backspin/configuration.rb +1 -1
- data/lib/backspin/record.rb +9 -18
- data/lib/backspin/record_result.rb +0 -6
- data/lib/backspin/recorder.rb +46 -301
- data/lib/backspin/version.rb +1 -1
- data/lib/backspin.rb +117 -87
- metadata +4 -31
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 3937af0a8fc0d808e6cc2f8788481ee479a46b6c21353a7c73820711995f9b96
|
|
4
|
+
data.tar.gz: a30e4ebd5608ad6718fc0d5cbda3685adc49b97cbb5af188db604f3b1b0de3be
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1343b731c00281871a50f4d61e276068d4aa18d54ba60f80f7ad780f88191c06f6af5168d53c7559f82c9d722e67521b8ca56aeb7998a2f61f410b925caff1aa
|
|
7
|
+
data.tar.gz: a3e496f886dae99cc6f624399655955b18b131991f332b2470ecd57ed74fb7e4c8e58a8cf89237f2faa9e10456ab67566ebc18da3c6fdc253716a8abb4728061
|
data/.circleci/config.yml
CHANGED
|
@@ -6,26 +6,38 @@ orbs:
|
|
|
6
6
|
ruby: circleci/ruby@2.5.3
|
|
7
7
|
|
|
8
8
|
jobs:
|
|
9
|
-
|
|
9
|
+
test:
|
|
10
10
|
resource_class: medium
|
|
11
11
|
parameters:
|
|
12
12
|
ruby-version:
|
|
13
13
|
type: string
|
|
14
14
|
docker:
|
|
15
15
|
- image: cimg/ruby:<< parameters.ruby-version >>
|
|
16
|
-
|
|
17
16
|
steps:
|
|
18
17
|
- checkout
|
|
19
18
|
- ruby/install-deps:
|
|
20
19
|
key: gems-v1-ruby<< parameters.ruby-version >>
|
|
21
20
|
- run:
|
|
22
|
-
name: Run specs
|
|
23
|
-
command: bundle exec rake
|
|
21
|
+
name: Run specs
|
|
22
|
+
command: bundle exec rake spec
|
|
23
|
+
|
|
24
|
+
lint:
|
|
25
|
+
resource_class: medium
|
|
26
|
+
docker:
|
|
27
|
+
- image: cimg/ruby:4.0
|
|
28
|
+
steps:
|
|
29
|
+
- checkout
|
|
30
|
+
- ruby/install-deps:
|
|
31
|
+
key: gems-v1-ruby4.0
|
|
32
|
+
- run:
|
|
33
|
+
name: Run Standard Ruby linter
|
|
34
|
+
command: bundle exec rake standard
|
|
24
35
|
|
|
25
36
|
workflows:
|
|
26
37
|
build:
|
|
27
38
|
jobs:
|
|
28
|
-
-
|
|
39
|
+
- test:
|
|
29
40
|
matrix:
|
|
30
41
|
parameters:
|
|
31
|
-
ruby-version: ["3.2", "3.3", "3.4"]
|
|
42
|
+
ruby-version: ["3.2", "3.3", "3.4", "4.0"]
|
|
43
|
+
- lint
|
data/.ruby-version
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
4.0.0
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,12 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.8.0 - 2026-02-05
|
|
4
|
+
* Breaking: new `Backspin.run("command", name:, env:)` command API plus block capture via `Backspin.run(name:) { ... }` and `Backspin.capture("name") { ... }`
|
|
5
|
+
* Breaking: remove `run!` and `:playback`
|
|
6
|
+
* Breaking: drop RSpec dependency; verification failures raise `Backspin::VerificationError`
|
|
7
|
+
* Breaking: record format bumped to 3.0 and only `Open3::Capture3` / `Backspin::Capturer` records are accepted
|
|
8
|
+
* Scrub credential patterns apply to stdout, stderr, args, and env values
|
|
9
|
+
|
|
3
10
|
## 0.7.1 - 2025-12-02
|
|
4
11
|
* Include result object on VerificationError to make it easier for callers to debug verification errors
|
|
5
12
|
|
|
@@ -36,4 +43,4 @@ Simpler, unified API: `Backspin.run` and `Backspin run!` methods that automatica
|
|
|
36
43
|
- `use_cassette` method for VCR-style record/replay
|
|
37
44
|
- Support for multiple verification modes (strict, playback, custom matcher)
|
|
38
45
|
- Multi-command recording support
|
|
39
|
-
- RSpec integration using RSpec's mocking framework
|
|
46
|
+
- RSpec integration using RSpec's mocking framework
|
data/CLAUDE.md
CHANGED
|
@@ -41,13 +41,11 @@ bin/rake standard # Alternative: Run via Rake task
|
|
|
41
41
|
### Core Components
|
|
42
42
|
|
|
43
43
|
**Backspin Module** (`lib/backspin.rb`)
|
|
44
|
-
- Main API: `run
|
|
45
|
-
- Capture API: `capture` (raises `VerificationError` on verification failure by default)
|
|
46
|
-
- Legacy API: `call`, `verify`, `verify!`, `use_record`
|
|
44
|
+
- Main API: `run` (direct command execution and block capture), `capture` (alias for block form)
|
|
47
45
|
- Credential scrubbing logic
|
|
48
|
-
- Configuration management (including `raise_on_verification_failure` which defaults to `true`
|
|
46
|
+
- Configuration management (including `raise_on_verification_failure` which defaults to `true`)
|
|
49
47
|
|
|
50
|
-
**Command Class** (`lib/backspin.rb`)
|
|
48
|
+
**Command Class** (`lib/backspin/command.rb`)
|
|
51
49
|
- Represents a single CLI execution
|
|
52
50
|
- Stores: args, stdout, stderr, status, recorded_at
|
|
53
51
|
|
|
@@ -60,10 +58,11 @@ bin/rake standard # Alternative: Run via Rake task
|
|
|
60
58
|
|
|
61
59
|
### Key Design Patterns
|
|
62
60
|
|
|
63
|
-
-
|
|
64
|
-
-
|
|
65
|
-
-
|
|
66
|
-
-
|
|
61
|
+
- Direct `Open3.capture3` execution for command runs
|
|
62
|
+
- Tempfile-based FD capture for block forms
|
|
63
|
+
- Single-command records stored as YAML
|
|
64
|
+
- Automatic credential scrubbing for security (AWS keys, API tokens, passwords, env values)
|
|
65
|
+
- Recording modes: `:auto`, `:record`, `:verify`
|
|
67
66
|
|
|
68
67
|
### Testing Approach
|
|
69
68
|
|
data/CONTRIBUTING.md
CHANGED
|
@@ -21,7 +21,7 @@ Backspin is a Ruby gem for characterization testing of command-line interfaces.
|
|
|
21
21
|
|
|
22
22
|
### Prerequisites
|
|
23
23
|
|
|
24
|
-
- Ruby 3.2, 3.3, or
|
|
24
|
+
- Ruby 3.2, 3.3, 3.4, or 4.0
|
|
25
25
|
- Bundler
|
|
26
26
|
- Git
|
|
27
27
|
|
|
@@ -71,7 +71,7 @@ Backspin is a Ruby gem for characterization testing of command-line interfaces.
|
|
|
71
71
|
**Core Components:**
|
|
72
72
|
|
|
73
73
|
- **Backspin Module** (`lib/backspin.rb`)
|
|
74
|
-
- Main API: `
|
|
74
|
+
- Main API: `run` (direct command execution and block capture), `capture` (alias for block form)
|
|
75
75
|
- Credential scrubbing logic
|
|
76
76
|
- Configuration management
|
|
77
77
|
|
|
@@ -132,15 +132,8 @@ Example test structure:
|
|
|
132
132
|
```ruby
|
|
133
133
|
RSpec.describe "Feature name" do
|
|
134
134
|
it "does something specific" do
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
# Exercise
|
|
139
|
-
result = Backspin.call(record_name) do
|
|
140
|
-
Open3.capture3("echo", "hello")
|
|
141
|
-
end
|
|
142
|
-
|
|
143
|
-
# Verify
|
|
135
|
+
result = Backspin.run(["echo", "hello"], name: "my_test_record")
|
|
136
|
+
|
|
144
137
|
expect(result.stdout).to eq("hello\n")
|
|
145
138
|
end
|
|
146
139
|
end
|
|
@@ -162,7 +155,7 @@ end
|
|
|
162
155
|
- Keep changes focused and atomic
|
|
163
156
|
- Include tests for new functionality
|
|
164
157
|
- Update examples in README.md if changing public APIs
|
|
165
|
-
- Ensure CI passes (tests against Ruby 3.2, 3.3, and
|
|
158
|
+
- Ensure CI passes (tests against Ruby 3.2, 3.3, 3.4, and 4.0)
|
|
166
159
|
|
|
167
160
|
## Code Style
|
|
168
161
|
|
data/Gemfile.lock
CHANGED
|
@@ -1,9 +1,7 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
backspin (0.
|
|
5
|
-
ostruct
|
|
6
|
-
rspec-mocks (~> 3)
|
|
4
|
+
backspin (0.8.0)
|
|
7
5
|
|
|
8
6
|
GEM
|
|
9
7
|
remote: https://rubygems.org/
|
|
@@ -13,7 +11,6 @@ GEM
|
|
|
13
11
|
json (2.16.0)
|
|
14
12
|
language_server-protocol (3.17.0.5)
|
|
15
13
|
lint_roller (1.1.0)
|
|
16
|
-
ostruct (0.6.1)
|
|
17
14
|
parallel (1.27.0)
|
|
18
15
|
parser (3.3.10.0)
|
|
19
16
|
ast (~> 2.4.1)
|
|
@@ -70,7 +67,7 @@ GEM
|
|
|
70
67
|
timecop (0.9.10)
|
|
71
68
|
unicode-display_width (3.2.0)
|
|
72
69
|
unicode-emoji (~> 4.1)
|
|
73
|
-
unicode-emoji (4.
|
|
70
|
+
unicode-emoji (4.2.0)
|
|
74
71
|
|
|
75
72
|
PLATFORMS
|
|
76
73
|
arm64-darwin-24
|
|
@@ -84,4 +81,4 @@ DEPENDENCIES
|
|
|
84
81
|
timecop (~> 0.9)
|
|
85
82
|
|
|
86
83
|
BUNDLED WITH
|
|
87
|
-
4.0.
|
|
84
|
+
4.0.5
|
data/MATCHERS.md
CHANGED
|
@@ -13,14 +13,11 @@ Backspin supports custom matchers for flexible verification of command outputs.
|
|
|
13
13
|
A proc matcher receives full command hashes and can check any combination of fields:
|
|
14
14
|
|
|
15
15
|
```ruby
|
|
16
|
-
|
|
17
|
-
result = Backspin.run("version_test",
|
|
16
|
+
result = Backspin.run(["node", "--version"], name: "version_test",
|
|
18
17
|
matcher: ->(recorded, actual) {
|
|
19
|
-
recorded["stdout"].start_with?("v") &&
|
|
18
|
+
recorded["stdout"].start_with?("v") &&
|
|
20
19
|
actual["stdout"].start_with?("v")
|
|
21
|
-
})
|
|
22
|
-
Open3.capture3("node --version")
|
|
23
|
-
end
|
|
20
|
+
})
|
|
24
21
|
```
|
|
25
22
|
|
|
26
23
|
### 2. Field-Specific Hash Matchers
|
|
@@ -29,33 +26,13 @@ Use a hash to specify matchers for individual fields. Only specified fields are
|
|
|
29
26
|
|
|
30
27
|
```ruby
|
|
31
28
|
# Only check stdout - stderr and status are ignored
|
|
32
|
-
result = Backspin.run("timestamp_test",
|
|
29
|
+
result = Backspin.run(["date"], name: "timestamp_test",
|
|
33
30
|
matcher: {
|
|
34
31
|
stdout: ->(recorded, actual) {
|
|
35
|
-
# Both should contain a timestamp
|
|
36
32
|
recorded.match?(/\d{4}-\d{2}-\d{2}/) &&
|
|
37
33
|
actual.match?(/\d{4}-\d{2}-\d{2}/)
|
|
38
34
|
}
|
|
39
|
-
})
|
|
40
|
-
Open3.capture3("date")
|
|
41
|
-
end
|
|
42
|
-
|
|
43
|
-
# Check multiple specific fields
|
|
44
|
-
result = Backspin.run("api_test",
|
|
45
|
-
matcher: {
|
|
46
|
-
stdout: ->(recorded, actual) {
|
|
47
|
-
# Check JSON structure exists
|
|
48
|
-
recorded.include?("\"data\":") &&
|
|
49
|
-
actual.include?("\"data\":")
|
|
50
|
-
},
|
|
51
|
-
status: ->(recorded, actual) {
|
|
52
|
-
# Both should succeed
|
|
53
|
-
recorded == 0 && actual == 0
|
|
54
|
-
}
|
|
55
|
-
# Note: stderr is NOT checked
|
|
56
|
-
}) do
|
|
57
|
-
Open3.capture3("curl", "-s", "https://api.example.com")
|
|
58
|
-
end
|
|
35
|
+
})
|
|
59
36
|
```
|
|
60
37
|
|
|
61
38
|
### 3. The :all Matcher
|
|
@@ -63,17 +40,13 @@ end
|
|
|
63
40
|
The `:all` matcher receives complete command hashes with all fields:
|
|
64
41
|
|
|
65
42
|
```ruby
|
|
66
|
-
|
|
67
|
-
result = Backspin.run("build_test",
|
|
43
|
+
result = Backspin.run(["./build.sh"], name: "build_test",
|
|
68
44
|
matcher: {
|
|
69
45
|
all: ->(recorded, actual) {
|
|
70
|
-
# Check overall success: stdout has "BUILD SUCCESSFUL" AND status is 0
|
|
71
46
|
actual["stdout"].include?("BUILD SUCCESSFUL") &&
|
|
72
47
|
actual["status"] == 0
|
|
73
48
|
}
|
|
74
|
-
})
|
|
75
|
-
Open3.capture3("./build.sh")
|
|
76
|
-
end
|
|
49
|
+
})
|
|
77
50
|
```
|
|
78
51
|
|
|
79
52
|
### 4. Combining :all with Field Matchers
|
|
@@ -81,23 +54,12 @@ end
|
|
|
81
54
|
When both `:all` and field matchers are present, all must pass:
|
|
82
55
|
|
|
83
56
|
```ruby
|
|
84
|
-
result = Backspin.run("complex_test",
|
|
57
|
+
result = Backspin.run(["./process_data.sh"], name: "complex_test",
|
|
85
58
|
matcher: {
|
|
86
|
-
all: ->(recorded, actual) {
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
stdout: ->(recorded, actual) {
|
|
91
|
-
# Specific check: output contains result
|
|
92
|
-
actual.include?("Result:")
|
|
93
|
-
},
|
|
94
|
-
stderr: ->(recorded, actual) {
|
|
95
|
-
# No errors expected
|
|
96
|
-
actual.empty?
|
|
97
|
-
}
|
|
98
|
-
}) do
|
|
99
|
-
Open3.capture3("./process_data.sh")
|
|
100
|
-
end
|
|
59
|
+
all: ->(recorded, actual) { actual["status"] == 0 },
|
|
60
|
+
stdout: ->(recorded, actual) { actual.include?("Result:") },
|
|
61
|
+
stderr: ->(recorded, actual) { actual.empty? }
|
|
62
|
+
})
|
|
101
63
|
```
|
|
102
64
|
|
|
103
65
|
## Matcher Proc Arguments
|
|
@@ -108,10 +70,11 @@ Field-specific matchers receive field values:
|
|
|
108
70
|
|
|
109
71
|
The `:all` matcher receives full hashes with these keys:
|
|
110
72
|
- `"stdout"` - String output
|
|
111
|
-
- `"stderr"` - String error output
|
|
112
|
-
- `"status"` - Integer exit code
|
|
113
|
-
- `"command_type"` - String like "Open3::Capture3"
|
|
114
|
-
- `"args"` - Array of command arguments
|
|
73
|
+
- `"stderr"` - String error output
|
|
74
|
+
- `"status"` - Integer exit code (placeholder `0` for block capture)
|
|
75
|
+
- `"command_type"` - String like "Open3::Capture3" or "Backspin::Capturer"
|
|
76
|
+
- `"args"` - String or Array of command arguments
|
|
77
|
+
- `"env"` - Optional Hash of env vars (command runs only)
|
|
115
78
|
- `"recorded_at"` - Timestamp string
|
|
116
79
|
|
|
117
80
|
## Examples
|
|
@@ -119,116 +82,45 @@ The `:all` matcher receives full hashes with these keys:
|
|
|
119
82
|
### Matching Version Numbers
|
|
120
83
|
|
|
121
84
|
```ruby
|
|
122
|
-
|
|
123
|
-
matcher: {
|
|
85
|
+
matcher = {
|
|
124
86
|
stdout: ->(recorded, actual) {
|
|
125
|
-
recorded_major = recorded[
|
|
126
|
-
actual_major = actual[
|
|
87
|
+
recorded_major = recorded[/\d+/, 0]
|
|
88
|
+
actual_major = actual[/\d+/, 0]
|
|
127
89
|
recorded_major == actual_major
|
|
128
90
|
}
|
|
129
91
|
}
|
|
92
|
+
|
|
93
|
+
Backspin.run(["ruby", "--version"], name: "ruby_version", matcher: matcher)
|
|
130
94
|
```
|
|
131
95
|
|
|
132
96
|
### Ignoring Timestamps
|
|
133
97
|
|
|
134
98
|
```ruby
|
|
135
|
-
|
|
136
|
-
matcher: {
|
|
99
|
+
matcher = {
|
|
137
100
|
stdout: ->(recorded, actual) {
|
|
138
101
|
pattern = /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/
|
|
139
|
-
recorded.gsub(pattern,
|
|
102
|
+
recorded.gsub(pattern, "[TIME]") == actual.gsub(pattern, "[TIME]")
|
|
140
103
|
}
|
|
141
104
|
}
|
|
142
|
-
```
|
|
143
105
|
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
```ruby
|
|
147
|
-
# Validate JSON structure, ignore values
|
|
148
|
-
matcher: {
|
|
149
|
-
stdout: ->(recorded, actual) {
|
|
150
|
-
begin
|
|
151
|
-
recorded_data = JSON.parse(recorded)
|
|
152
|
-
actual_data = JSON.parse(actual)
|
|
153
|
-
|
|
154
|
-
# Same keys at top level
|
|
155
|
-
recorded_data.keys.sort == actual_data.keys.sort
|
|
156
|
-
rescue JSON::ParserError
|
|
157
|
-
false
|
|
158
|
-
end
|
|
159
|
-
}
|
|
160
|
-
}
|
|
106
|
+
Backspin.run(["date"], name: "timestamp_test", matcher: matcher)
|
|
161
107
|
```
|
|
162
108
|
|
|
163
109
|
### Logging/Debugging with :all
|
|
164
110
|
|
|
165
111
|
```ruby
|
|
166
|
-
# Use :all for side effects while other matchers do validation
|
|
167
112
|
logged_commands = []
|
|
168
113
|
|
|
169
|
-
|
|
114
|
+
Backspin.run(["./test.sh"], name: "debug_test",
|
|
170
115
|
matcher: {
|
|
171
116
|
all: ->(recorded, actual) {
|
|
172
|
-
# Log for debugging
|
|
173
117
|
logged_commands << {
|
|
174
118
|
args: actual["args"],
|
|
175
119
|
exit: actual["status"],
|
|
176
120
|
output_size: actual["stdout"].size
|
|
177
121
|
}
|
|
178
|
-
true
|
|
122
|
+
true
|
|
179
123
|
},
|
|
180
|
-
stdout: ->(recorded, actual) {
|
|
181
|
-
# Actual validation
|
|
182
|
-
actual.include?("SUCCESS")
|
|
183
|
-
}
|
|
184
|
-
}) do
|
|
185
|
-
Open3.capture3("./test.sh")
|
|
186
|
-
end
|
|
187
|
-
|
|
188
|
-
# Can inspect logged_commands after run
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
## Using with run!
|
|
192
|
-
|
|
193
|
-
The `run!` method automatically fails tests when matchers return false:
|
|
194
|
-
|
|
195
|
-
```ruby
|
|
196
|
-
# This will raise an error if the matcher fails
|
|
197
|
-
Backspin.run!("critical_test",
|
|
198
|
-
matcher: {
|
|
199
|
-
stdout: ->(r, a) { a.include?("OK") },
|
|
200
|
-
status: ->(r, a) { a == 0 }
|
|
201
|
-
}) do
|
|
202
|
-
Open3.capture3("./health_check.sh")
|
|
203
|
-
end
|
|
204
|
-
```
|
|
205
|
-
|
|
206
|
-
## Migration from match_on
|
|
207
|
-
|
|
208
|
-
The `match_on` option is deprecated. To migrate:
|
|
209
|
-
|
|
210
|
-
```ruby
|
|
211
|
-
# Old match_on style:
|
|
212
|
-
Backspin.run("test",
|
|
213
|
-
match_on: [:stdout, ->(r, a) { ... }])
|
|
214
|
-
|
|
215
|
-
# New matcher style:
|
|
216
|
-
Backspin.run("test",
|
|
217
|
-
matcher: { stdout: ->(r, a) { ... } })
|
|
218
|
-
|
|
219
|
-
# Old match_on with multiple fields:
|
|
220
|
-
Backspin.run("test",
|
|
221
|
-
match_on: [
|
|
222
|
-
[:stdout, ->(r, a) { ... }],
|
|
223
|
-
[:stderr, ->(r, a) { ... }]
|
|
224
|
-
])
|
|
225
|
-
|
|
226
|
-
# New matcher style:
|
|
227
|
-
Backspin.run("test",
|
|
228
|
-
matcher: {
|
|
229
|
-
stdout: ->(r, a) { ... },
|
|
230
|
-
stderr: ->(r, a) { ... }
|
|
124
|
+
stdout: ->(recorded, actual) { actual.include?("SUCCESS") }
|
|
231
125
|
})
|
|
232
126
|
```
|
|
233
|
-
|
|
234
|
-
Key difference: `match_on` required other fields to match exactly, while the new `matcher` hash only checks specified fields.
|