backspin 0.14.0 → 0.15.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.circleci/config.yml +1 -1
- data/.github/workflows/release.yml +1 -1
- data/.gitignore +0 -1
- data/CHANGELOG.md +7 -0
- data/CLAUDE.md +2 -1
- data/CONTRIBUTING.md +13 -216
- data/Gemfile.lock +14 -12
- data/README.md +34 -7
- data/fixtures/projects/dummy_cli_gem/Gemfile.lock +2 -2
- data/lib/backspin/snapshot.rb +15 -11
- data/lib/backspin/version.rb +1 -1
- data/lib/backspin.rb +29 -15
- data/mise.toml +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 075a4844c3bb93c3123497426d6f95e2cbc28b82b608f7bf5e6ff4968b4a5ff2
|
|
4
|
+
data.tar.gz: 60cfc30a385d234e330339d5cd70873b277af40a1c305b0781713f804b559190
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 294564bdac99b664bf8a1ed078f67736db43ab389b314d7b9c482884826d5ef6021a359d1c4d3d31a137dd0b3df44616f61dddf5cdb47478fe79c128583e36e0
|
|
7
|
+
data.tar.gz: eaa2e5d9d5c121fe6672cf5c49eee8644c6462cf86fe49fbabca69d0bbb29ae5bac368b9fa0ae20f56f2437f0fc01e13c3b7e13662b1fdfb41530e5df3bbf546
|
data/.circleci/config.yml
CHANGED
data/.gitignore
CHANGED
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,12 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.15.1
|
|
4
|
+
* Drop ruby 3.2 support; housekeeping and dependency clean up;
|
|
5
|
+
## 0.15.0
|
|
6
|
+
* Added `stdin:` to `Backspin.run` and `Backspin.compare` to pipe a String into the command, replacing `sh -c` wrappers. For `compare`, give a side as `{command:, stdin:}`. Stdin is saved on every snapshot like stdout and stderr (empty when not given), so re-recorded files gain a `stdin` line, but it is not compared.
|
|
7
|
+
* Changed `Backspin.compare` to skip credential scrubbing, so filters, matchers, and diffs see raw output and scrubbing can no longer hide mismatches (generic patterns could turn different paths into identical star runs). `run` and `capture` still scrub.
|
|
8
|
+
* Added `env:` to `Backspin.compare` side Hashes (`{command:, env:, stdin:}`), so each command can get its own environment. A side's env replaces the top-level `env:` for that side, and each snapshot records the env it ran with.
|
|
9
|
+
|
|
3
10
|
## 0.14.0
|
|
4
11
|
* Improved verification errors with per-field summaries for stdout, stderr, and exit status, plus three lines of context around output changes. Long diffs are limited to 50 lines; set `BACKSPIN_FULL_DIFF=1` to show the full diff.
|
|
5
12
|
* Preserved custom matcher failure reasons and made missing final newlines visible in output diffs.
|
data/CLAUDE.md
CHANGED
|
@@ -27,7 +27,8 @@ bin/rake install # Install gem locally for testing
|
|
|
27
27
|
bin/rake build # Build the gem locally
|
|
28
28
|
```
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
Use the [shared release guide](https://github.com/rsanheim/.github#readme) and
|
|
31
|
+
[Backspin quick reference](CONTRIBUTING.md#release-maintainer-setup) for releases.
|
|
31
32
|
|
|
32
33
|
### Code Quality
|
|
33
34
|
```bash
|
data/CONTRIBUTING.md
CHANGED
|
@@ -1,199 +1,22 @@
|
|
|
1
|
-
# Contributing
|
|
1
|
+
# Contributing
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
Note that Backspin is in early development and the API _will_ change before stabilizing at 1.0.
|
|
6
|
-
|
|
7
|
-
## Table of Contents
|
|
8
|
-
|
|
9
|
-
- [Getting Started](#getting-started)
|
|
10
|
-
- [Development Setup](#development-setup)
|
|
11
|
-
- [Making Changes](#making-changes)
|
|
12
|
-
- [Testing](#testing)
|
|
13
|
-
- [Submitting Changes](#submitting-changes)
|
|
14
|
-
- [Release Maintainer Setup](#release-maintainer-setup)
|
|
15
|
-
- [Code Style](#code-style)
|
|
16
|
-
- [Reporting Issues](#reporting-issues)
|
|
17
|
-
- [Feature Requests](#feature-requests)
|
|
18
|
-
|
|
19
|
-
## Getting Started
|
|
20
|
-
|
|
21
|
-
Backspin is a Ruby gem for characterization testing of command-line interfaces. It records and verifies CLI interactions by capturing stdout, stderr, and exit status from shell commands, similar to how VCR works for HTTP interactions.
|
|
22
|
-
|
|
23
|
-
### Prerequisites
|
|
24
|
-
|
|
25
|
-
- Ruby 3.2, 3.3, 3.4, or 4.0
|
|
26
|
-
- Bundler
|
|
27
|
-
- Git
|
|
3
|
+
Ideas welcome. PRs cool. Just ensure your
|
|
4
|
+
Backspin is in early development and the API _will_ change before stabilizing at 1.0.
|
|
28
5
|
|
|
29
6
|
## Development Setup
|
|
30
7
|
|
|
31
|
-
1. Fork and clone the repository:
|
|
32
|
-
```bash
|
|
33
|
-
git clone https://github.com/rsanheim/backspin.git
|
|
34
|
-
cd backspin
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
2. Run the setup script:
|
|
38
|
-
```bash
|
|
39
|
-
bin/setup
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
3. Run the full build (specs & standardrb linting) to ensure everything is working:
|
|
43
|
-
```bash
|
|
44
|
-
bin/rake spec
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
## Making Changes
|
|
48
|
-
|
|
49
|
-
### Development Workflow
|
|
50
|
-
|
|
51
|
-
1. Create a feature branch:
|
|
52
|
-
```bash
|
|
53
|
-
git checkout -b feature/your-feature-name
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
2. Make your changes following the project conventions
|
|
57
|
-
|
|
58
|
-
3. Run tests frequently:
|
|
59
|
-
```bash
|
|
60
|
-
bin/rake spec
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
4. Run the linter:
|
|
64
|
-
```bash
|
|
65
|
-
bin/rake standard
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
5. Commit your changes with clear, descriptive messages
|
|
69
|
-
|
|
70
|
-
### Architecture Overview
|
|
71
|
-
|
|
72
|
-
**Core Components:**
|
|
73
|
-
|
|
74
|
-
- **Backspin Module** (`lib/backspin.rb`)
|
|
75
|
-
- Main API: `run` (direct command execution and block capture), `capture` (alias for block form)
|
|
76
|
-
- Credential scrubbing logic
|
|
77
|
-
- Configuration management
|
|
78
|
-
|
|
79
|
-
- **Snapshot Class** (`lib/backspin/snapshot.rb`)
|
|
80
|
-
- Represents a single execution snapshot
|
|
81
|
-
- Stores: command type, args, env, stdout, stderr, status, recorded_at
|
|
82
|
-
|
|
83
|
-
- **BackspinResult Class** (`lib/backspin/backspin_result.rb`)
|
|
84
|
-
- Return object from `Backspin.run` / `Backspin.capture`
|
|
85
|
-
- Exposes `actual` and `expected` snapshots plus verify details
|
|
86
|
-
|
|
87
|
-
- **Record Class** (`lib/backspin/record.rb`)
|
|
88
|
-
- Manages YAML record files
|
|
89
|
-
- Handles record/verify sequencing
|
|
90
|
-
|
|
91
|
-
### Common Development Tasks
|
|
92
|
-
|
|
93
|
-
#### Adding New Features
|
|
94
|
-
|
|
95
|
-
1. Write integration tests in `spec/backspin/`
|
|
96
|
-
2. Implement in appropriate module (usually `lib/backspin.rb`)
|
|
97
|
-
3. Update README.md if adding public API
|
|
98
|
-
4. Run the the full build
|
|
99
|
-
|
|
100
|
-
#### Updating Credential Patterns
|
|
101
|
-
|
|
102
|
-
- Add patterns to `DEFAULT_CREDENTIAL_PATTERNS` in `lib/backspin.rb`
|
|
103
|
-
- Test with appropriate fixtures in specs
|
|
104
|
-
|
|
105
|
-
#### Debugging Tests
|
|
106
|
-
|
|
107
|
-
- Records are saved to `fixtures/backspin/` by default
|
|
108
|
-
- Check YAML files to see recorded command outputs
|
|
109
|
-
|
|
110
|
-
## Testing
|
|
111
|
-
|
|
112
|
-
### Running Tests
|
|
113
|
-
|
|
114
8
|
```bash
|
|
115
|
-
|
|
9
|
+
git clone https://github.com/rsanheim/backspin.git
|
|
10
|
+
cd backspin
|
|
11
|
+
bin/setup
|
|
12
|
+
# run full build
|
|
116
13
|
bin/rake spec
|
|
117
|
-
|
|
118
|
-
# Run specific test file
|
|
119
|
-
bundle exec rspec spec/backspin/record_spec.rb
|
|
120
|
-
|
|
121
|
-
# Run specific test
|
|
122
|
-
bundle exec rspec spec/backspin/record_spec.rb:42
|
|
123
14
|
```
|
|
124
15
|
|
|
125
|
-
### Writing Tests
|
|
126
|
-
|
|
127
|
-
Backspin uses integration-focused tests that exercise the full stack. When writing tests:
|
|
128
|
-
|
|
129
|
-
- Keep specs self-contained with their own setup, expectations, and cleanup
|
|
130
|
-
- Avoid shared contexts or helpers that hide important test details
|
|
131
|
-
- Use real shell commands (`echo`, `date`, etc.) for testing
|
|
132
|
-
- Ensure configuration is reset between tests to avoid side effects
|
|
133
|
-
- Verify new or updated test records in `fixtures/backspin/`
|
|
134
|
-
|
|
135
|
-
Example test structure:
|
|
136
|
-
|
|
137
|
-
```ruby
|
|
138
|
-
RSpec.describe "Feature name" do
|
|
139
|
-
it "does something specific" do
|
|
140
|
-
result = Backspin.run(["echo", "hello"], name: "my_test_record")
|
|
141
|
-
|
|
142
|
-
expect(result.actual.stdout).to eq("hello\n")
|
|
143
|
-
end
|
|
144
|
-
end
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
## Submitting Changes
|
|
148
|
-
|
|
149
|
-
### Pull Request Process
|
|
150
|
-
|
|
151
|
-
1. Ensure all tests pass
|
|
152
|
-
2. Update documentation if needed
|
|
153
|
-
3. Add entries to CHANGELOG.md following the existing format
|
|
154
|
-
4. Push your branch and create a pull request
|
|
155
|
-
5. Provide a clear description of your changes
|
|
156
|
-
6. Link any related issues
|
|
157
|
-
|
|
158
|
-
### Pull Request Guidelines
|
|
159
|
-
|
|
160
|
-
- Keep changes focused and atomic
|
|
161
|
-
- Include tests for new functionality
|
|
162
|
-
- Update examples in README.md if changing public APIs
|
|
163
|
-
- Ensure CI passes (tests against Ruby 3.2, 3.3, 3.4, and 4.0)
|
|
164
|
-
|
|
165
16
|
## Release Maintainer Setup
|
|
166
17
|
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
- Repository owner: `rsanheim`
|
|
170
|
-
- Repository name: `backspin`
|
|
171
|
-
- Workflow filename: `gem-release.yml`
|
|
172
|
-
- Workflow Repository Owner: `rsanheim`
|
|
173
|
-
- Workflow Repository Name: `.github`
|
|
174
|
-
- Environment: `release`
|
|
175
|
-
|
|
176
|
-
The workflow filename identifies the shared reusable workflow. The caller is `.github/workflows/release.yml`, which grants GitHub OIDC permission (`id-token: write`). No long-lived RubyGems API key is needed.
|
|
177
|
-
|
|
178
|
-
Release pull requests update `lib/backspin/version.rb` and add a nonempty `## VERSION` section to `CHANGELOG.md`; the section body becomes the GitHub release notes. Keep the version change and its notes in the same pull request. Use the normal premerge CI checks to establish release readiness. Merging to `main` starts the release workflow, which rechecks the version and changelog before publishing; invalid release notes prevent publication. Ordinary merges do not publish anything.
|
|
179
|
-
|
|
180
|
-
If publication fails before the gem is uploaded, fix the cause and manually run the Release workflow on `main`, setting `previous-ref` to the tag before the version increase. Confirm the target version is still unpublished before retrying.
|
|
181
|
-
|
|
182
|
-
The shared release workflow is pinned in `.github/workflows/release.yml`. Update that commit deliberately when adopting shared release changes. Bundler's standard `build`, `install`, and `release` tasks remain available; the automated workflow is the normal publishing path.
|
|
183
|
-
|
|
184
|
-
## Code Style
|
|
185
|
-
|
|
186
|
-
Backspin uses [Standard Ruby](https://github.com/standardrb/standard) for code formatting. Run the linter before committing:
|
|
187
|
-
|
|
188
|
-
```bash
|
|
189
|
-
bin/rake standard
|
|
190
|
-
```
|
|
191
|
-
|
|
192
|
-
To automatically fix issues:
|
|
193
|
-
|
|
194
|
-
```bash
|
|
195
|
-
bin/rake standard:fix
|
|
196
|
-
```
|
|
18
|
+
Setup, trusted publishing, workflow updates, verification, and recovery live in
|
|
19
|
+
[the shared release guide](https://github.com/rsanheim/.github#readme).
|
|
197
20
|
|
|
198
21
|
## Reporting Issues
|
|
199
22
|
|
|
@@ -201,39 +24,13 @@ bin/rake standard:fix
|
|
|
201
24
|
|
|
202
25
|
When reporting bugs, please include *full logs* including:
|
|
203
26
|
|
|
204
|
-
1.
|
|
27
|
+
1. full ruby version (`ruby -v`)
|
|
205
28
|
2. Backspin version
|
|
206
|
-
3.
|
|
29
|
+
3. reproduction code
|
|
207
30
|
4. Expected behavior vs actual behavior
|
|
208
31
|
5. Full error messages and stack traces
|
|
209
|
-
6. Relevant YAML record files (sanitized of sensitive data)
|
|
210
|
-
|
|
211
|
-
### Security Issues
|
|
212
|
-
|
|
213
|
-
For security vulnerabilities, please email the maintainers directly rather than opening a public issue.
|
|
214
|
-
|
|
215
|
-
## Feature Requests
|
|
216
|
-
|
|
217
|
-
We welcome feature requests! When proposing new features:
|
|
218
|
-
|
|
219
|
-
1. Check existing issues to avoid duplicates
|
|
220
|
-
2. Describe the use case and motivation
|
|
221
|
-
3. Provide code examples of how the feature would work
|
|
222
|
-
4. Be open to discussion and alternative approaches
|
|
223
|
-
|
|
224
|
-
## Additional Resources
|
|
225
|
-
|
|
226
|
-
- [Project README](README.md)
|
|
227
|
-
- [CLAUDE.md](CLAUDE.md) - AI assistant guidance
|
|
228
|
-
- [RSpec Documentation](https://rspec.info/)
|
|
229
|
-
- [Standardrb linting](https://github.com/standardrb/standard)
|
|
32
|
+
6. Relevant YAML record files if applicable (sanitized of sensitive data)
|
|
230
33
|
|
|
231
34
|
## Questions?
|
|
232
35
|
|
|
233
|
-
If you have questions about contributing, feel free to
|
|
234
|
-
|
|
235
|
-
- Open an issue for discussion
|
|
236
|
-
- Check existing issues and pull requests
|
|
237
|
-
- Review the test suite for examples
|
|
238
|
-
|
|
239
|
-
Thank you for contributing to Backspin!
|
|
36
|
+
If you have questions about contributing, feel free to open an issue and holler
|
data/Gemfile.lock
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
backspin (0.
|
|
4
|
+
backspin (0.15.1)
|
|
5
5
|
logger
|
|
6
6
|
|
|
7
7
|
GEM
|
|
@@ -9,11 +9,11 @@ GEM
|
|
|
9
9
|
specs:
|
|
10
10
|
ast (2.4.3)
|
|
11
11
|
diff-lcs (1.6.2)
|
|
12
|
-
json (
|
|
12
|
+
json (3.0.2)
|
|
13
13
|
language_server-protocol (3.17.0.6)
|
|
14
14
|
lint_roller (1.1.0)
|
|
15
15
|
logger (1.7.0)
|
|
16
|
-
parallel (
|
|
16
|
+
parallel (2.3.0)
|
|
17
17
|
parser (3.3.12.0)
|
|
18
18
|
ast (~> 2.4.1)
|
|
19
19
|
racc
|
|
@@ -21,7 +21,7 @@ GEM
|
|
|
21
21
|
racc (1.8.1)
|
|
22
22
|
rainbow (3.1.1)
|
|
23
23
|
rake (13.4.2)
|
|
24
|
-
regexp_parser (2.
|
|
24
|
+
regexp_parser (2.13.1)
|
|
25
25
|
rspec (3.13.2)
|
|
26
26
|
rspec-core (~> 3.13.0)
|
|
27
27
|
rspec-expectations (~> 3.13.0)
|
|
@@ -35,8 +35,8 @@ GEM
|
|
|
35
35
|
diff-lcs (>= 1.2.0, < 2.0)
|
|
36
36
|
rspec-support (~> 3.13.0)
|
|
37
37
|
rspec-support (3.13.7)
|
|
38
|
-
rubocop (1.
|
|
39
|
-
json (
|
|
38
|
+
rubocop (1.91.0)
|
|
39
|
+
json (>= 2.3)
|
|
40
40
|
language_server-protocol (~> 3.17.0.2)
|
|
41
41
|
lint_roller (~> 1.1.0)
|
|
42
42
|
parallel (>= 1.10)
|
|
@@ -54,10 +54,10 @@ GEM
|
|
|
54
54
|
rubocop (>= 1.75.0, < 2.0)
|
|
55
55
|
rubocop-ast (>= 1.47.1, < 2.0)
|
|
56
56
|
ruby-progressbar (1.13.0)
|
|
57
|
-
standard (1.
|
|
57
|
+
standard (1.57.0)
|
|
58
58
|
language_server-protocol (~> 3.17.0.2)
|
|
59
59
|
lint_roller (~> 1.0)
|
|
60
|
-
rubocop (~> 1.
|
|
60
|
+
rubocop (~> 1.91.0)
|
|
61
61
|
standard-custom (~> 1.0.0)
|
|
62
62
|
standard-performance (~> 1.8)
|
|
63
63
|
standard-custom (1.0.2)
|
|
@@ -67,13 +67,15 @@ GEM
|
|
|
67
67
|
lint_roller (~> 1.1)
|
|
68
68
|
rubocop-performance (~> 1.26.0)
|
|
69
69
|
timecop (0.9.11)
|
|
70
|
-
unicode-display_width (3.
|
|
71
|
-
unicode-emoji (~> 4.
|
|
72
|
-
unicode-emoji (4.
|
|
70
|
+
unicode-display_width (3.3.0)
|
|
71
|
+
unicode-emoji (~> 4.3)
|
|
72
|
+
unicode-emoji (4.3.0)
|
|
73
73
|
|
|
74
74
|
PLATFORMS
|
|
75
|
+
aarch64-linux
|
|
75
76
|
arm64-darwin-24
|
|
76
77
|
ruby
|
|
78
|
+
x86_64-linux
|
|
77
79
|
|
|
78
80
|
DEPENDENCIES
|
|
79
81
|
backspin!
|
|
@@ -83,4 +85,4 @@ DEPENDENCIES
|
|
|
83
85
|
timecop (~> 0.9)
|
|
84
86
|
|
|
85
87
|
BUNDLED WITH
|
|
86
|
-
4.0.
|
|
88
|
+
4.0.22
|
data/README.md
CHANGED
|
@@ -92,6 +92,26 @@ result.verified? # true when they match
|
|
|
92
92
|
|
|
93
93
|
Only stdout, stderr, and exit status are compared, so the two commands can differ in argv and environment without any normalization. `filter` and `matcher` work exactly as they do for `Backspin.run`, and a mismatch raises `Backspin::VerificationError` unless `raise_on_verification_failure` is disabled.
|
|
94
94
|
|
|
95
|
+
To pipe data into a command's stdin, give that side as a Hash with `command:` and `stdin:`:
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
Backspin.compare(
|
|
99
|
+
reference: {command: ["jq", "."], stdin: '{"a": 1}'},
|
|
100
|
+
actual: {command: ["my-jq", "."], stdin: '{"a": 1}'}
|
|
101
|
+
)
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
A side Hash can also take `env:`, which replaces the top-level `env:` for that side:
|
|
105
|
+
|
|
106
|
+
```ruby
|
|
107
|
+
Backspin.compare(
|
|
108
|
+
reference: {command: ["jq", "."], env: {"HOME" => reference_home}},
|
|
109
|
+
actual: {command: ["my-jq", "."], env: {"HOME" => actual_home}}
|
|
110
|
+
)
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
`compare` records nothing, so it does not scrub credentials. Filters, matchers, and failure diffs see raw output.
|
|
114
|
+
|
|
95
115
|
If the reference command produces no output at all - usually a sign it failed to start rather than that its output is genuinely empty - `compare` raises `Backspin::ReferenceCommandError` instead of comparing empty to empty.
|
|
96
116
|
|
|
97
117
|
### Recording Modes
|
|
@@ -143,6 +163,7 @@ record_count: 3 # increments on each write
|
|
|
143
163
|
snapshot:
|
|
144
164
|
command_type: "Open3::Capture3"
|
|
145
165
|
args: ["echo", "hello"]
|
|
166
|
+
stdin: ""
|
|
146
167
|
stdout: "hello\n"
|
|
147
168
|
stderr: ""
|
|
148
169
|
status: 0
|
|
@@ -164,6 +185,14 @@ Backspin.run(
|
|
|
164
185
|
|
|
165
186
|
If `env:` is not provided, it is not passed to `Open3.capture3` and is not recorded.
|
|
166
187
|
|
|
188
|
+
### Standard Input
|
|
189
|
+
|
|
190
|
+
```ruby
|
|
191
|
+
Backspin.run(["cat"], name: "with_stdin", stdin: "hello from stdin")
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
`stdin:` is passed to `Open3.capture3` as `stdin_data:` and saved in the record, but only stdout, stderr, and exit status are verified. `stdin:` is not supported with the block form.
|
|
195
|
+
|
|
167
196
|
### Custom Matchers
|
|
168
197
|
|
|
169
198
|
For cases where full matching isn't suitable, you can override via `matcher:`. **NOTE**: If you provide
|
|
@@ -278,7 +307,7 @@ You can configure Backspin's behavior globally:
|
|
|
278
307
|
Backspin.configure do |config|
|
|
279
308
|
config.raise_on_verification_failure = false # default is true
|
|
280
309
|
config.backspin_dir = "spec/fixtures/cli_records" # default is "fixtures/backspin"
|
|
281
|
-
config.scrub_credentials = false # default is true
|
|
310
|
+
config.scrub_credentials = false # default is true; scrubs run/capture records
|
|
282
311
|
end
|
|
283
312
|
```
|
|
284
313
|
|
|
@@ -331,7 +360,7 @@ end
|
|
|
331
360
|
|
|
332
361
|
If the CLI interaction you are recording contains sensitive data in stdout/stderr, you should be careful to make sure it is not recorded to YAML.
|
|
333
362
|
|
|
334
|
-
|
|
363
|
+
Backspin scrubs credentials from everything it records: by default it tries to scrub common credential patterns from stdout, stderr, args, env, and stdin before `run` and `capture` save them. `compare` records nothing, so it does not scrub. Always review your record files before commiting them to source control.
|
|
335
364
|
|
|
336
365
|
```ruby
|
|
337
366
|
# This will automatically scrub AWS keys, API tokens, passwords, etc.
|
|
@@ -374,11 +403,9 @@ To install this gem onto your local machine, run `bundle exec rake install`.
|
|
|
374
403
|
|
|
375
404
|
### Releasing
|
|
376
405
|
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
The release implementation lives in [`rsanheim/.github`](https://github.com/rsanheim/.github) and is pinned to a commit in `.github/workflows/release.yml`. See [the maintainer setup](CONTRIBUTING.md#release-maintainer-setup) before enabling releases.
|
|
406
|
+
See the [shared release guide](https://github.com/rsanheim/.github#readme) for
|
|
407
|
+
setup and operation, and the [Backspin quick reference](CONTRIBUTING.md#release-maintainer-setup)
|
|
408
|
+
for the files to update in a release PR.
|
|
382
409
|
|
|
383
410
|
## Contributing
|
|
384
411
|
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: ../../..
|
|
3
3
|
specs:
|
|
4
|
-
backspin (0.
|
|
4
|
+
backspin (0.15.0)
|
|
5
5
|
logger
|
|
6
6
|
|
|
7
7
|
PATH
|
|
@@ -40,7 +40,7 @@ DEPENDENCIES
|
|
|
40
40
|
rspec (~> 3)
|
|
41
41
|
|
|
42
42
|
CHECKSUMS
|
|
43
|
-
backspin (0.
|
|
43
|
+
backspin (0.15.0)
|
|
44
44
|
diff-lcs (1.6.2) sha256=9ae0d2cba7d4df3075fe8cd8602a8604993efc0dfa934cff568969efb1909962
|
|
45
45
|
dummy_cli_gem (0.1.0)
|
|
46
46
|
logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203
|
data/lib/backspin/snapshot.rb
CHANGED
|
@@ -3,14 +3,16 @@
|
|
|
3
3
|
module Backspin
|
|
4
4
|
# Represents a single captured execution snapshot.
|
|
5
5
|
class Snapshot
|
|
6
|
-
attr_reader :command_type, :args, :env, :stdout, :stderr, :status, :recorded_at
|
|
6
|
+
attr_reader :command_type, :args, :env, :stdin, :stdout, :stderr, :status, :recorded_at
|
|
7
7
|
|
|
8
|
-
def initialize(command_type:, args:, env: nil, stdout: "", stderr: "", status: 0, recorded_at: nil)
|
|
8
|
+
def initialize(command_type:, args:, env: nil, stdin: "", stdout: "", stderr: "", status: 0, recorded_at: nil, scrub: Backspin.configuration.scrub_credentials)
|
|
9
|
+
@scrub = scrub
|
|
9
10
|
@command_type = command_type
|
|
10
11
|
@args = sanitize_args(args)
|
|
11
12
|
@env = env.nil? ? nil : sanitize_env(env)
|
|
12
|
-
@
|
|
13
|
-
@
|
|
13
|
+
@stdin = scrub_string((stdin || "").dup).freeze
|
|
14
|
+
@stdout = scrub_string((stdout || "").dup).freeze
|
|
15
|
+
@stderr = scrub_string((stderr || "").dup).freeze
|
|
14
16
|
@status = status || 0
|
|
15
17
|
@recorded_at = recorded_at.nil? ? nil : recorded_at.dup.freeze
|
|
16
18
|
@serialized_hash = build_serialized_hash
|
|
@@ -42,6 +44,7 @@ module Backspin
|
|
|
42
44
|
command_type: command_type,
|
|
43
45
|
args: data["args"],
|
|
44
46
|
env: data["env"],
|
|
47
|
+
stdin: data["stdin"],
|
|
45
48
|
stdout: data["stdout"],
|
|
46
49
|
stderr: data["stderr"],
|
|
47
50
|
status: data["status"],
|
|
@@ -55,6 +58,7 @@ module Backspin
|
|
|
55
58
|
data = {
|
|
56
59
|
"command_type" => command_type.name,
|
|
57
60
|
"args" => args,
|
|
61
|
+
"stdin" => stdin,
|
|
58
62
|
"stdout" => stdout,
|
|
59
63
|
"stderr" => stderr,
|
|
60
64
|
"status" => status,
|
|
@@ -64,25 +68,25 @@ module Backspin
|
|
|
64
68
|
deep_freeze(data)
|
|
65
69
|
end
|
|
66
70
|
|
|
67
|
-
def
|
|
68
|
-
|
|
71
|
+
def scrub_string(text)
|
|
72
|
+
@scrub ? Backspin.scrub_text(text) : text
|
|
73
|
+
end
|
|
69
74
|
|
|
75
|
+
def scrub_args(value)
|
|
70
76
|
case value
|
|
71
77
|
when String
|
|
72
|
-
|
|
78
|
+
scrub_string(value)
|
|
73
79
|
when Array
|
|
74
80
|
value.map { |entry| scrub_args(entry) }
|
|
75
81
|
when Hash
|
|
76
|
-
value.transform_values { |entry| entry.is_a?(String) ?
|
|
82
|
+
value.transform_values { |entry| entry.is_a?(String) ? scrub_string(entry) : entry }
|
|
77
83
|
else
|
|
78
84
|
value
|
|
79
85
|
end
|
|
80
86
|
end
|
|
81
87
|
|
|
82
88
|
def scrub_env(value)
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
value.transform_values { |entry| entry.is_a?(String) ? Backspin.scrub_text(entry) : entry }
|
|
89
|
+
value.transform_values { |entry| entry.is_a?(String) ? scrub_string(entry) : entry }
|
|
86
90
|
end
|
|
87
91
|
|
|
88
92
|
def sanitize_args(value)
|
data/lib/backspin/version.rb
CHANGED
data/lib/backspin.rb
CHANGED
|
@@ -76,17 +76,19 @@ module Backspin
|
|
|
76
76
|
# @param command [String, Array] Command to execute via Open3.capture3
|
|
77
77
|
# @param name [String] Name for the record file
|
|
78
78
|
# @param env [Hash] Environment variables to pass to Open3.capture3
|
|
79
|
+
# @param stdin [String] Data written to the command's stdin (recorded, not compared)
|
|
79
80
|
# @param mode [Symbol] Recording mode - :auto, :record, :verify
|
|
80
81
|
# @param matcher [Proc, Hash] Custom matcher for verification
|
|
81
82
|
# @param filter [Proc] Custom filter for recorded data/canonicalization
|
|
82
83
|
# @param filter_on [Symbol] Filter application mode - :both (default), :record
|
|
83
84
|
# @return [BackspinResult] Aggregate result for this run
|
|
84
|
-
def run(command = nil, name:, env: nil, mode: :auto, matcher: nil, filter: nil, filter_on: :both, &block)
|
|
85
|
+
def run(command = nil, name:, env: nil, stdin: nil, mode: :auto, matcher: nil, filter: nil, filter_on: :both, &block)
|
|
85
86
|
validate_filter_on!(filter_on)
|
|
86
87
|
|
|
87
88
|
if block_given?
|
|
88
89
|
raise ArgumentError, "command must be omitted when using a block" unless command.nil?
|
|
89
90
|
raise ArgumentError, "env is not supported when using a block" unless env.nil?
|
|
91
|
+
raise ArgumentError, "stdin is not supported when using a block" unless stdin.nil?
|
|
90
92
|
|
|
91
93
|
return perform_capture(name, mode: mode, matcher: matcher, filter: filter, filter_on: filter_on, &block)
|
|
92
94
|
end
|
|
@@ -97,6 +99,7 @@ module Backspin
|
|
|
97
99
|
command,
|
|
98
100
|
name: name,
|
|
99
101
|
env: env,
|
|
102
|
+
stdin: stdin,
|
|
100
103
|
mode: mode,
|
|
101
104
|
matcher: matcher,
|
|
102
105
|
filter: filter,
|
|
@@ -126,22 +129,31 @@ module Backspin
|
|
|
126
129
|
# Only stdout, stderr, and status are compared, so the two commands may
|
|
127
130
|
# differ in argv and env without any normalization.
|
|
128
131
|
#
|
|
129
|
-
#
|
|
130
|
-
#
|
|
132
|
+
# Either side may be a Hash of {command:, env:, stdin:} to give that command
|
|
133
|
+
# its own env or stdin. A side's env replaces the top-level env for that
|
|
134
|
+
# side. Stdin is recorded on each snapshot, not compared.
|
|
135
|
+
#
|
|
136
|
+
# Credentials are not scrubbed, so the filter, matcher, and diff see raw output.
|
|
137
|
+
#
|
|
138
|
+
# @param reference [String, Array, Hash] Command that defines the correct output
|
|
139
|
+
# @param actual [String, Array, Hash] Command under test
|
|
131
140
|
# @param env [Hash] Environment variables passed to both commands
|
|
132
141
|
# @param matcher [Proc, Hash] Custom matcher for verification
|
|
133
142
|
# @param filter [Proc] Custom filter applied to both sides before comparing
|
|
134
143
|
# @return [BackspinResult] expected = reference, actual = command under test
|
|
135
144
|
def compare(reference:, actual:, env: nil, matcher: nil, filter: nil)
|
|
136
|
-
|
|
145
|
+
reference_command, reference_env, reference_stdin = reference.is_a?(Hash) ? reference.values_at(:command, :env, :stdin) : [reference, nil, nil]
|
|
146
|
+
actual_command, actual_env, actual_stdin = actual.is_a?(Hash) ? actual.values_at(:command, :env, :stdin) : [actual, nil, nil]
|
|
147
|
+
reference_env = normalize_env(reference_env || env)
|
|
148
|
+
actual_env = normalize_env(actual_env || env)
|
|
137
149
|
|
|
138
|
-
expected_snapshot, = capture_command_snapshot(
|
|
150
|
+
expected_snapshot, = capture_command_snapshot(reference_command, reference_env, reference_stdin, scrub: false)
|
|
139
151
|
if expected_snapshot.stdout.empty? && expected_snapshot.stderr.empty?
|
|
140
152
|
raise ReferenceCommandError,
|
|
141
153
|
"Reference command produced no output (exit status #{expected_snapshot.status}): #{reference.inspect}"
|
|
142
154
|
end
|
|
143
155
|
|
|
144
|
-
actual_snapshot, = capture_command_snapshot(
|
|
156
|
+
actual_snapshot, = capture_command_snapshot(actual_command, actual_env, actual_stdin, scrub: false)
|
|
145
157
|
|
|
146
158
|
command_diff = CommandDiff.new(
|
|
147
159
|
expected: expected_snapshot,
|
|
@@ -188,7 +200,7 @@ module Backspin
|
|
|
188
200
|
result
|
|
189
201
|
end
|
|
190
202
|
|
|
191
|
-
def perform_command_run(command, name:, env:, mode:, matcher:, filter:, filter_on:)
|
|
203
|
+
def perform_command_run(command, name:, env:, stdin:, mode:, matcher:, filter:, filter_on:)
|
|
192
204
|
record_path = Record.build_record_path(name)
|
|
193
205
|
mode = determine_mode(mode, record_path)
|
|
194
206
|
validate_mode!(mode)
|
|
@@ -199,7 +211,7 @@ module Backspin
|
|
|
199
211
|
|
|
200
212
|
result = case mode
|
|
201
213
|
when :record
|
|
202
|
-
actual_snapshot, output = capture_command_snapshot(command, normalized_env)
|
|
214
|
+
actual_snapshot, output = capture_command_snapshot(command, normalized_env, stdin)
|
|
203
215
|
record.set_snapshot(actual_snapshot)
|
|
204
216
|
record.save(filter: filter)
|
|
205
217
|
BackspinResult.new(
|
|
@@ -217,7 +229,7 @@ module Backspin
|
|
|
217
229
|
raise RecordFormatError, "Invalid record format: expected Open3::Capture3 for run"
|
|
218
230
|
end
|
|
219
231
|
|
|
220
|
-
actual_snapshot, output = capture_command_snapshot(command, normalized_env)
|
|
232
|
+
actual_snapshot, output = capture_command_snapshot(command, normalized_env, stdin)
|
|
221
233
|
command_diff = CommandDiff.new(
|
|
222
234
|
expected: expected_snapshot,
|
|
223
235
|
actual: actual_snapshot,
|
|
@@ -243,16 +255,18 @@ module Backspin
|
|
|
243
255
|
result
|
|
244
256
|
end
|
|
245
257
|
|
|
246
|
-
def capture_command_snapshot(command, env)
|
|
247
|
-
stdout, stderr, status = execute_command(command, env)
|
|
258
|
+
def capture_command_snapshot(command, env, stdin, scrub: configuration.scrub_credentials)
|
|
259
|
+
stdout, stderr, status = execute_command(command, env, stdin)
|
|
248
260
|
snapshot = Snapshot.new(
|
|
249
261
|
command_type: Open3::Capture3,
|
|
250
262
|
args: command,
|
|
251
263
|
env: env,
|
|
264
|
+
stdin: stdin,
|
|
252
265
|
stdout: stdout,
|
|
253
266
|
stderr: stderr,
|
|
254
267
|
status: status.exitstatus,
|
|
255
|
-
recorded_at: Time.now.utc.iso8601
|
|
268
|
+
recorded_at: Time.now.utc.iso8601,
|
|
269
|
+
scrub: scrub
|
|
256
270
|
)
|
|
257
271
|
[snapshot, [stdout, stderr, status]]
|
|
258
272
|
end
|
|
@@ -264,13 +278,13 @@ module Backspin
|
|
|
264
278
|
env.empty? ? nil : env
|
|
265
279
|
end
|
|
266
280
|
|
|
267
|
-
def execute_command(command, env)
|
|
281
|
+
def execute_command(command, env, stdin)
|
|
268
282
|
case command
|
|
269
283
|
when String
|
|
270
|
-
env ? Open3.capture3(env, command) : Open3.capture3(command)
|
|
284
|
+
env ? Open3.capture3(env, command, stdin_data: stdin) : Open3.capture3(command, stdin_data: stdin)
|
|
271
285
|
when Array
|
|
272
286
|
raise ArgumentError, "command array cannot be empty" if command.empty?
|
|
273
|
-
env ? Open3.capture3(env, *command) : Open3.capture3(*command)
|
|
287
|
+
env ? Open3.capture3(env, *command, stdin_data: stdin) : Open3.capture3(*command, stdin_data: stdin)
|
|
274
288
|
else
|
|
275
289
|
raise ArgumentError, "command must be a String or Array"
|
|
276
290
|
end
|
data/mise.toml
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: backspin
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.15.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Rob Sanheim
|
|
@@ -106,7 +106,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
106
106
|
- !ruby/object:Gem::Version
|
|
107
107
|
version: '0'
|
|
108
108
|
requirements: []
|
|
109
|
-
rubygems_version: 4.0.
|
|
109
|
+
rubygems_version: 4.0.20
|
|
110
110
|
specification_version: 4
|
|
111
111
|
summary: Record and replay CLI interactions for testing
|
|
112
112
|
test_files: []
|