backspin 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: a1f998c2134e48ab2c3eee38e86e64ec1a4325ba8d81dca4e7c8e042513dde13
4
+ data.tar.gz: 43aa6c53243fce4cab5911a6635944f25cb339ee5c58dbeb421f15bd30960a7a
5
+ SHA512:
6
+ metadata.gz: 15ded2144dfe4db263a8cd54f449aaf591d6f90da25675e77de3a22b7dd0384d8d226f44ed7a7ccfd96e7edb5eae72ecdf95ae6152b38d288153a001b085cb71
7
+ data.tar.gz: 0bb9f49eb95c197403e8a7d7869cd5b33bb45323c6d9461e07af23df3f24deaa87af5571fd8475759eff5b87d2dc4de94ff3ff898a71e8cc3154e1b1e874122e
data/.gitignore ADDED
@@ -0,0 +1,19 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /_yardoc/
4
+ /coverage/
5
+ /doc/
6
+ /pkg/
7
+ /spec/reports/
8
+ /tmp/
9
+ /.claude
10
+ /.cursor
11
+
12
+ # rspec failure tracking
13
+ .rspec_status
14
+
15
+ # Backspin cassettes
16
+ /tmp/backspin/
17
+
18
+ # Bundler
19
+ Gemfile.lock
data/.rspec ADDED
@@ -0,0 +1,3 @@
1
+ --format documentation
2
+ --color
3
+ --require spec_helper
data/.standard.yml ADDED
@@ -0,0 +1,7 @@
1
+ # StandardRB configuration for Backspin
2
+ parallel: true
3
+ format: progress
4
+
5
+ ignore:
6
+ - 'tmp/**/*'
7
+ - 'vendor/**/*'
data/CHANGELOG.md ADDED
@@ -0,0 +1,19 @@
1
+ # Changelog
2
+
3
+
4
+ ## [0.2.0] - 2025-06-05
5
+ - First public release of Backspin, extracteed from `name-TBD` CLI tool
6
+
7
+ ## [0.2.1] - 2025-06-04
8
+ - major refactoring, add support for `system` calls
9
+
10
+ ## [0.1.0] - 2025-06-02
11
+
12
+ ### Added
13
+ - Initial (internal) release of Backspin
14
+ - `record` method to capture CLI command outputs
15
+ - `verify` and `verify!` methods for output verification
16
+ - `use_cassette` method for VCR-style record/replay
17
+ - Support for multiple verification modes (strict, playback, custom matcher)
18
+ - Multi-command recording support
19
+ - RSpec integration using RSpec's mocking framework
data/CLAUDE.md ADDED
@@ -0,0 +1,85 @@
1
+ # CLAUDE.md
2
+
3
+ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+ ## Project Overview
6
+
7
+ Backspin is a Ruby gem for characterization testing of command-line interfaces. It records and replays CLI interactions by capturing stdout, stderr, and exit status from shell commands - similar to how VCR works for HTTP interactions. Backspin uses "records" (YAML files) to store recorded command outputs.
8
+
9
+ ## Development Commands
10
+
11
+ ### Setup
12
+ ```bash
13
+ bundle install
14
+ bin/setup
15
+ ```
16
+
17
+ ### Testing
18
+ ```bash
19
+ bin/rake spec # Run all tests
20
+ rspec spec/[file] # Run specific test file
21
+ rspec spec/[file]:[line] # Run specific test
22
+ ```
23
+
24
+ ### Building and Releasing
25
+ ```bash
26
+ bundle exec rake install # Install gem locally for testing
27
+ bundle exec rake release # Release to RubyGems (updates version, tags, pushes)
28
+ ```
29
+
30
+ ### Code Quality
31
+ ```bash
32
+ bin/rake standard # Run Standard Ruby linter
33
+ ```
34
+
35
+ ## Architecture
36
+
37
+ ### Core Components
38
+
39
+ **Backspin Module** (`lib/backspin.rb`)
40
+ - Main API: `call`, `verify`, `verify!`, `use_record`
41
+ - Credential scrubbing logic
42
+ - Configuration management
43
+
44
+ **Command Class** (`lib/backspin.rb`)
45
+ - Represents a single CLI execution
46
+ - Stores: args, stdout, stderr, status, recorded_at
47
+
48
+ **Record Class** (`lib/backspin/record.rb`)
49
+ - Manages YAML record files
50
+ - Handles recording/playback sequencing
51
+
52
+ **RSpecMetadata** (`lib/backspin/rspec_metadata.rb`)
53
+ - Auto-generates record names from RSpec context
54
+
55
+ ### Key Design Patterns
56
+
57
+ - Uses RSpec mocking to intercept `Open3.capture3` calls
58
+ - Records are stored as YAML arrays to support multiple commands
59
+ - Automatic credential scrubbing for security (AWS keys, API tokens, passwords)
60
+ - VCR-style recording modes: `:once`, `:all`, `:none`, `:new_episodes`
61
+
62
+ ### Testing Approach
63
+
64
+ - Integration-focused tests that exercise the full stack
65
+ - Default record directory is `spec/backspin_data` (can be configured)
66
+ - Tests use real shell commands (`echo`, `date`, etc.)
67
+ - Configuration is reset between tests to avoid side effects
68
+ - **Important**: Backspin specs MUST be as local and un-DRY as possible. Each spec should be self-contained with its own setup, expectations, and cleanup if needed. Avoid shared contexts or helpers that hide important test details.
69
+
70
+ ## Common Development Tasks
71
+
72
+ ### Adding New Features
73
+ 1. Write integration tests in `spec/backspin/`
74
+ 2. Implement in appropriate module (usually `lib/backspin.rb`)
75
+ 3. Update README.md if adding public API
76
+ 4. Run tests with `rake spec`
77
+
78
+ ### Debugging Tests
79
+ - Records are saved to `spec/backspin_data/` by default
80
+ - Check YAML files to see recorded command outputs
81
+ - Use `VERBOSE=1` for additional output during tests
82
+
83
+ ### Updating Credential Patterns
84
+ - Add patterns to `DEFAULT_CREDENTIAL_PATTERNS` in `lib/backspin.rb`
85
+ - Test with appropriate fixtures in specs
data/Gemfile ADDED
@@ -0,0 +1,11 @@
1
+ source "https://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in backspin.gemspec
4
+ gemspec
5
+
6
+ group :development do
7
+ gem "rake", "~> 13.0"
8
+ gem "rspec", "~> 3.0"
9
+ gem "timecop", "~> 0.9"
10
+ gem "standard", "~> 1.0"
11
+ end
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2025 Backspin contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,149 @@
1
+ # Backspin
2
+
3
+ Backspin records and replays CLI interactions in Ruby for easy snapshot testing of command-line interfaces. Currently supports `Open3.capture3` and `system` and requires `rspec-mocks`. More system calls and flexible test integration are welcome - PRs welcome!
4
+
5
+ **NOTE:** Backspin is in early development (version 0.2.x), and you can expect the API to change. It is being developed along-side in-production CLI apps, so the API will be refined and improved as we get to 1.0.
6
+
7
+ Inspired by [VCR](https://github.com/vcr/vcr) and other [golden master](https://en.wikipedia.org/wiki/Golden_master_(software_development)) libraries.
8
+
9
+ ## Overview
10
+
11
+ Backspin is a Ruby library for snapshot testing (or characterization testing) of command-line interfaces. While VCR records and replays HTTP interactions, Backspin records and replays CLI interactions - capturing stdout, stderr, and exit status from shell commands.
12
+
13
+ ## Installation
14
+
15
+ Add this line to your application's Gemfile in the `:test` group:
16
+
17
+ ```ruby
18
+ group :test do
19
+ gem "backspin"
20
+ end
21
+ ```
22
+
23
+ And then run `bundle install`.
24
+
25
+ ## Usage
26
+
27
+ ### Recording CLI interactions
28
+
29
+ ```ruby
30
+ require "backspin"
31
+
32
+ # Record a command's output
33
+ result = Backspin.call("echo_hello") do
34
+ stdout, stderr, status = Open3.capture3("echo hello")
35
+ # This will save the output to `spec/backspin_data/echo_hello.yaml`.
36
+ end
37
+
38
+ ```
39
+
40
+ ### Verifying CLI output
41
+
42
+ ```ruby
43
+ # Verify that a command produces the expected output
44
+ result = Backspin.verify("echo_hello") do
45
+ Open3.capture3("echo hello")
46
+ end
47
+
48
+ expect(result.verified?).to be true
49
+ ```
50
+
51
+ ### Using verify! for automatic test failures
52
+
53
+ ```ruby
54
+ # Automatically fail the test if output doesn't match
55
+ Backspin.verify!("echo_hello") do
56
+ Open3.capture3("echo hello")
57
+ end
58
+ ```
59
+
60
+ ### Playback mode for fast tests
61
+
62
+ ```ruby
63
+ # Return recorded output without running the command
64
+ result = Backspin.verify("slow_command", mode: :playback) do
65
+ Open3.capture3("slow_command") # Not executed - will playback from the record yaml (assuming it exists)
66
+ end
67
+ ```
68
+
69
+ ### Custom matchers
70
+
71
+ ```ruby
72
+ # Use custom logic to verify output
73
+ Backspin.verify("version_check",
74
+ matcher: ->(recorded, actual) {
75
+ # Just check that both start with "ruby"
76
+ recorded["stdout"].start_with?("ruby") &&
77
+ actual["stdout"].start_with?("ruby")
78
+ }) do
79
+ Open3.capture3("ruby --version")
80
+ end
81
+ ```
82
+
83
+ ### VCR-style use_record
84
+
85
+ _The plan is to make something like this the main entry point API for ease of use_
86
+
87
+ ```ruby
88
+ # Record on first run, replay on subsequent runs
89
+ Backspin.use_record("my_command", record: :once) do
90
+ Open3.capture3("echo hello")
91
+ end
92
+ ```
93
+
94
+ ### Credential Scrubbing
95
+
96
+ If the CLI interaction you are recording contains sensitive data in stdout or stderr, you should be careful to make sure it is not recorded to yaml!
97
+
98
+ By default, Backspin automatically scrubs [common credential patterns](https://github.com/rsanheim/backspin/blob/f8661f084aad0ae759cd971c4af31ccf9bdc6bba/lib/backspin.rb#L46-L65) from records, but this will only handle some common cases.
99
+ Always review your record files before commiting them to source control.
100
+
101
+ A tool like [trufflehog](https://github.com/trufflesecurity/trufflehog) or [gitleaks](https://github.com/gitleaks/gitleaks) run via a pre-commit to catch any sensitive data before commit.
102
+
103
+ ```ruby
104
+ # This will automatically scrub AWS keys, API tokens, passwords, etc.
105
+ Backspin.call("aws_command") do
106
+ Open3.capture3("aws s3 ls")
107
+ end
108
+
109
+ # Add custom patterns to scrub
110
+ Backspin.configure do |config|
111
+ config.add_credential_pattern(/MY_SECRET_[A-Z0-9]+/)
112
+ end
113
+
114
+ # Disable credential scrubbing - use with caution!
115
+ Backspin.configure do |config|
116
+ config.scrub_credentials = false
117
+ end
118
+
119
+ ```
120
+
121
+ Automatic scrubbing includes:
122
+ - AWS access keys, secret keys, and session tokens
123
+ - Google API keys and OAuth client IDs
124
+ - Generic API keys, auth tokens, and passwords
125
+ - Private keys (RSA, etc.)
126
+
127
+ ## Features
128
+
129
+ - **Simple recording**: Capture stdout, stderr, and exit status
130
+ - **Flexible verification**: Strict matching, playback mode, or custom matchers
131
+ - **Auto-naming**: Automatically generate record names from RSpec examples
132
+ - **Multiple commands**: Record sequences of commands in a single record
133
+ - **RSpec integration**: Works seamlessly with RSpec's mocking framework
134
+ - **Human-readable**: YAML records are easy to read and edit
135
+ - **Credential scrubbing**: Automatically removes sensitive data like API keys and passwords
136
+
137
+ ## Development
138
+
139
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests.
140
+
141
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
142
+
143
+ ## Contributing
144
+
145
+ Bug reports and pull requests are welcome on GitHub at https://github.com/rsanheim/backspin.
146
+
147
+ ## License
148
+
149
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,7 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+ require "standard/rake"
4
+
5
+ RSpec::Core::RakeTask.new(:spec)
6
+
7
+ task default: [:spec, :standard]
data/backspin.gemspec ADDED
@@ -0,0 +1,29 @@
1
+ require_relative "lib/backspin/version"
2
+
3
+ Gem::Specification.new do |spec|
4
+ spec.name = "backspin"
5
+ spec.version = Backspin::VERSION
6
+ spec.authors = ["Rob Sanheim"]
7
+ spec.email = ["rsanheim@gmail.com"]
8
+
9
+ spec.summary = "Record and replay CLI interactions for testing"
10
+ spec.description = "Backspin is a Ruby library for characterization testing of command-line interfaces. Inspired by VCR's cassette-based approach, it records and replays CLI interactions to make testing faster and more deterministic."
11
+ spec.homepage = "https://github.com/rsanheim/backspin"
12
+ spec.license = "MIT"
13
+ spec.required_ruby_version = Gem::Requirement.new(">= 2.5.0")
14
+
15
+ spec.metadata["homepage_uri"] = spec.homepage
16
+ spec.metadata["source_code_uri"] = spec.homepage
17
+ spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/main/CHANGELOG.md"
18
+
19
+ # Specify which files should be added to the gem when it is released.
20
+ spec.files = Dir.chdir(File.expand_path("..", __FILE__)) do
21
+ `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
22
+ end
23
+ spec.bindir = "bin"
24
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
25
+ spec.require_paths = ["lib"]
26
+
27
+ spec.add_dependency "rspec-mocks", "~> 3.0"
28
+ spec.add_dependency "ostruct"
29
+ end
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
@@ -0,0 +1,66 @@
1
+ module Backspin
2
+ class Command
3
+ attr_reader :args, :stdout, :stderr, :status, :recorded_at, :method_class
4
+
5
+ def initialize(method_class:, args:, stdout: nil, stderr: nil, status: nil, recorded_at: nil)
6
+ @method_class = method_class
7
+ @args = args
8
+ @stdout = stdout
9
+ @stderr = stderr
10
+ @status = status
11
+ @recorded_at = recorded_at
12
+ end
13
+
14
+ # Convert to hash for YAML serialization
15
+ def to_h(filter: nil)
16
+ data = {
17
+ "command_type" => @method_class.name,
18
+ "args" => @args,
19
+ "stdout" => Backspin.scrub_text(@stdout),
20
+ "stderr" => Backspin.scrub_text(@stderr),
21
+ "status" => @status,
22
+ "recorded_at" => @recorded_at
23
+ }
24
+
25
+ # Apply filter if provided
26
+ if filter
27
+ data = filter.call(data)
28
+ end
29
+
30
+ data
31
+ end
32
+
33
+ # Create from hash (for loading from YAML)
34
+ def self.from_h(data)
35
+ # Determine method class from command_type
36
+ method_class = case data["command_type"]
37
+ when "Open3::Capture3"
38
+ Open3::Capture3
39
+ when "Kernel::System"
40
+ ::Kernel::System
41
+ else
42
+ # Default to capture3 for backwards compatibility
43
+ Open3::Capture3
44
+ end
45
+
46
+ new(
47
+ method_class: method_class,
48
+ args: data["args"],
49
+ stdout: data["stdout"],
50
+ stderr: data["stderr"],
51
+ status: data["status"],
52
+ recorded_at: data["recorded_at"]
53
+ )
54
+ end
55
+ end
56
+ end
57
+
58
+ # Define the Open3::Capture3 class for identification
59
+ module Open3
60
+ class Capture3; end
61
+ end
62
+
63
+ # Define the Kernel::System class for identification
64
+ module ::Kernel
65
+ class System; end
66
+ end
@@ -0,0 +1,87 @@
1
+ module Backspin
2
+ class RecordFormatError < StandardError; end
3
+
4
+ class NoMoreRecordingsError < StandardError; end
5
+
6
+ class Record
7
+ attr_reader :path, :commands, :first_recorded_at
8
+
9
+ def initialize(path)
10
+ @path = path
11
+ @commands = []
12
+ @first_recorded_at = nil
13
+ @playback_index = 0
14
+ load_from_file if File.exist?(@path)
15
+ end
16
+
17
+ def add_command(command)
18
+ @commands << command
19
+ @first_recorded_at ||= command.recorded_at
20
+ self
21
+ end
22
+
23
+ def save(filter: nil)
24
+ FileUtils.mkdir_p(File.dirname(@path))
25
+ # New format: top-level metadata with commands array
26
+ record_data = {
27
+ "first_recorded_at" => @first_recorded_at,
28
+ "format_version" => "2.0",
29
+ "commands" => @commands.map { |cmd| cmd.to_h(filter: filter) }
30
+ }
31
+ File.write(@path, record_data.to_yaml)
32
+ end
33
+
34
+ def reload
35
+ @commands = []
36
+ @playback_index = 0
37
+ load_from_file if File.exist?(@path)
38
+ @playback_index = 0 # Reset again after loading to ensure it's at 0
39
+ end
40
+
41
+ def exists?
42
+ File.exist?(@path)
43
+ end
44
+
45
+ def empty?
46
+ @commands.empty?
47
+ end
48
+
49
+ def size
50
+ @commands.size
51
+ end
52
+
53
+ def next_command
54
+ if @playback_index >= @commands.size
55
+ raise NoMoreRecordingsError, "No more recordings available for replay"
56
+ end
57
+
58
+ command = @commands[@playback_index]
59
+ @playback_index += 1
60
+ command
61
+ end
62
+
63
+ def clear
64
+ @commands = []
65
+ @playback_index = 0
66
+ end
67
+
68
+ def self.load_or_create(path)
69
+ new(path)
70
+ end
71
+
72
+ private
73
+
74
+ def load_from_file
75
+ data = YAML.load_file(@path.to_s)
76
+
77
+ unless data.is_a?(Hash) && data["format_version"] == "2.0"
78
+ raise RecordFormatError, "Invalid record format: expected format version 2.0"
79
+ end
80
+
81
+ @first_recorded_at = data["first_recorded_at"]
82
+ @commands = data["commands"].map { |command_data| Command.from_h(command_data) }
83
+ rescue Psych::SyntaxError => e
84
+ raise RecordFormatError, "Invalid record format: #{e.message}"
85
+ end
86
+ end
87
+ end
@@ -0,0 +1,193 @@
1
+ require "open3"
2
+ require "ostruct"
3
+ require "rspec/mocks"
4
+
5
+ module Backspin
6
+ # Handles stubbing and recording of command executions
7
+ class Recorder
8
+ include RSpec::Mocks::ExampleMethods
9
+
10
+ attr_reader :commands, :verification_data, :mode, :record
11
+
12
+ def initialize(mode: :record, record: nil)
13
+ @mode = mode
14
+ @record = record
15
+ @commands = []
16
+ @verification_data = {}
17
+ end
18
+
19
+ def record_calls(*command_types)
20
+ command_types = [:capture3, :system] if command_types.empty?
21
+
22
+ command_types.each do |command_type|
23
+ record_call(command_type)
24
+ end
25
+ end
26
+
27
+ def record_call(command_type)
28
+ case command_type
29
+ when :system
30
+ setup_system_call_stub
31
+ when :capture3
32
+ setup_capture3_call_stub
33
+ else
34
+ raise ArgumentError, "Unknown command type: #{command_type}"
35
+ end
36
+ end
37
+
38
+ # Setup stubs for playback mode - just return recorded values
39
+ def setup_playback_stub(command)
40
+ if command.method_class == Open3::Capture3
41
+ actual_status = OpenStruct.new(exitstatus: command.status)
42
+ allow(Open3).to receive(:capture3).and_return([command.stdout, command.stderr, actual_status])
43
+ elsif command.method_class == ::Kernel::System
44
+ # For system, return true if exit status was 0
45
+ allow_any_instance_of(Object).to receive(:system).and_return(command.status == 0)
46
+ end
47
+ end
48
+
49
+ # Setup stubs for verification - capture actual output
50
+ def setup_verification_stub(command)
51
+ @verification_data = {}
52
+
53
+ if command.method_class == Open3::Capture3
54
+ allow(Open3).to receive(:capture3).and_wrap_original do |original_method, *args|
55
+ stdout, stderr, status = original_method.call(*args)
56
+ @verification_data["stdout"] = stdout
57
+ @verification_data["stderr"] = stderr
58
+ @verification_data["status"] = status.exitstatus
59
+ [stdout, stderr, status]
60
+ end
61
+ elsif command.method_class == ::Kernel::System
62
+ allow_any_instance_of(Object).to receive(:system).and_wrap_original do |original_method, receiver, *args|
63
+ # Execute the real system call
64
+ result = original_method.call(receiver, *args)
65
+
66
+ # For system calls, we only track the exit status
67
+ @verification_data["stdout"] = ""
68
+ @verification_data["stderr"] = ""
69
+ # Derive exit status from result: true = 0, false = non-zero
70
+ @verification_data["status"] = result ? 0 : 1
71
+
72
+ result
73
+ end
74
+ end
75
+ end
76
+
77
+ # Setup stubs for replay mode - returns recorded values for multiple commands
78
+ def setup_replay_stubs
79
+ raise ArgumentError, "Record required for replay mode" unless @record
80
+
81
+ setup_capture3_replay_stub
82
+ setup_system_replay_stub
83
+ end
84
+
85
+ private
86
+
87
+ def setup_capture3_replay_stub
88
+ allow(Open3).to receive(:capture3) do |*args|
89
+ command = @record.next_command
90
+
91
+ # Make sure this is a capture3 command
92
+ unless command.method_class == Open3::Capture3
93
+ raise RecordNotFoundError, "Expected Open3::Capture3 command but got #{command.method_class.name}"
94
+ end
95
+
96
+ recorded_stdout = command.stdout
97
+ recorded_stderr = command.stderr
98
+ recorded_status = OpenStruct.new(exitstatus: command.status)
99
+
100
+ [recorded_stdout, recorded_stderr, recorded_status]
101
+ rescue NoMoreRecordingsError => e
102
+ raise RecordNotFoundError, e.message
103
+ end
104
+ end
105
+
106
+ def setup_system_replay_stub
107
+ allow_any_instance_of(Object).to receive(:system) do |receiver, *args|
108
+ command = @record.next_command
109
+
110
+ # Make sure this is a system command
111
+ unless command.method_class == ::Kernel::System
112
+ raise RecordNotFoundError, "Expected Kernel::System command but got #{command.method_class.name}"
113
+ end
114
+
115
+ # Return true if exit status was 0, false otherwise
116
+ command.status == 0
117
+ rescue NoMoreRecordingsError => e
118
+ raise RecordNotFoundError, e.message
119
+ end
120
+ end
121
+
122
+ def setup_capture3_call_stub
123
+ allow(Open3).to receive(:capture3).and_wrap_original do |original_method, *args|
124
+ # Execute the real command
125
+ stdout, stderr, status = original_method.call(*args)
126
+
127
+ # Parse command args
128
+ cmd_args = if args.length == 1 && args.first.is_a?(String)
129
+ args.first.split(" ")
130
+ else
131
+ args
132
+ end
133
+
134
+ # Create command with interaction data
135
+ command = Command.new(
136
+ method_class: Open3::Capture3,
137
+ args: cmd_args,
138
+ stdout: stdout,
139
+ stderr: stderr,
140
+ status: status.exitstatus,
141
+ recorded_at: Time.now.iso8601
142
+ )
143
+ @commands << command
144
+
145
+ # Store output for later access (last one wins)
146
+ Backspin.last_output = stdout
147
+
148
+ # Return original result
149
+ [stdout, stderr, status]
150
+ end
151
+ end
152
+
153
+ def setup_system_call_stub
154
+ allow_any_instance_of(Object).to receive(:system).and_wrap_original do |original_method, receiver, *args|
155
+ # Execute the real system call
156
+ result = original_method.call(receiver, *args)
157
+
158
+ # Parse command args based on how system was called
159
+ parsed_args = if args.empty? && receiver.is_a?(String)
160
+ # Single string form - split the command string
161
+ receiver.split(" ")
162
+ else
163
+ # Multi-arg form - already an array
164
+ args
165
+ end
166
+
167
+ # For system calls, stdout and stderr are not captured
168
+ # The caller of system() doesn't have access to them
169
+ stdout = ""
170
+ stderr = ""
171
+ # Derive exit status from result: true = 0, false = non-zero, nil = command failed
172
+ status = result ? 0 : 1
173
+
174
+ # Create command with interaction data
175
+ command = Command.new(
176
+ method_class: ::Kernel::System,
177
+ args: parsed_args,
178
+ stdout: stdout,
179
+ stderr: stderr,
180
+ status: status,
181
+ recorded_at: Time.now.iso8601
182
+ )
183
+ @commands << command
184
+
185
+ # Store output for later access (for consistency with capture3)
186
+ Backspin.last_output = stdout
187
+
188
+ # Return the original result (true/false/nil)
189
+ result
190
+ end
191
+ end
192
+ end
193
+ end
@@ -0,0 +1,3 @@
1
+ module Backspin
2
+ VERSION = "0.2.1"
3
+ end
data/lib/backspin.rb ADDED
@@ -0,0 +1,431 @@
1
+ require "yaml"
2
+ require "fileutils"
3
+ require "open3"
4
+ require "pathname"
5
+ require "ostruct"
6
+ require "rspec/mocks"
7
+ require "backspin/version"
8
+ require "backspin/command"
9
+ require "backspin/record"
10
+ require "backspin/recorder"
11
+
12
+ module Backspin
13
+ class RecordNotFoundError < StandardError; end
14
+
15
+ # Include RSpec mocks methods
16
+ extend RSpec::Mocks::ExampleMethods
17
+
18
+ # Configuration for Backspin
19
+ class Configuration
20
+ attr_accessor :scrub_credentials
21
+ # The directory where backspin will store its files - defaults to spec/backspin_data
22
+ attr_accessor :backspin_dir
23
+ # Regex patterns to scrub from saved output
24
+ attr_reader :credential_patterns
25
+
26
+ def initialize
27
+ @scrub_credentials = true
28
+ @credential_patterns = default_credential_patterns
29
+ @backspin_dir = Pathname(Dir.pwd).join("spec", "backspin_data")
30
+ end
31
+
32
+ def add_credential_pattern(pattern)
33
+ @credential_patterns << pattern
34
+ end
35
+
36
+ def clear_credential_patterns
37
+ @credential_patterns = []
38
+ end
39
+
40
+ def reset_credential_patterns
41
+ @credential_patterns = default_credential_patterns
42
+ end
43
+
44
+ private
45
+
46
+ def default_credential_patterns
47
+ [
48
+ # AWS credentials
49
+ /AKIA[0-9A-Z]{16}/, # AWS Access Key ID
50
+ /aws_secret_access_key\s*[:=]\s*["']?([A-Za-z0-9\/+=]{40})["']?/i, # AWS Secret Key
51
+ /aws_session_token\s*[:=]\s*["']?([A-Za-z0-9\/+=]+)["']?/i, # AWS Session Token
52
+
53
+ # Google Cloud credentials
54
+ /AIza[0-9A-Za-z\-_]{35}/, # Google API Key
55
+ /[0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com/, # Google OAuth2 client ID
56
+ /-----BEGIN (RSA )?PRIVATE KEY-----/, # Private keys
57
+
58
+ # Generic patterns
59
+ /api[_-]?key\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i, # Generic API keys
60
+ /auth[_-]?token\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i, # Auth tokens
61
+ /password\s*[:=]\s*["']?([^"'\s]{8,})["']?/i, # Passwords
62
+ /secret\s*[:=]\s*["']?([A-Za-z0-9\-_]{20,})["']?/i # Generic secrets
63
+ ]
64
+ end
65
+ end
66
+
67
+ class << self
68
+ def configuration
69
+ @configuration ||= Configuration.new
70
+ end
71
+
72
+ def configure
73
+ yield(configuration)
74
+ end
75
+
76
+ def reset_configuration!
77
+ @configuration = Configuration.new
78
+ end
79
+ end
80
+
81
+ class Result
82
+ attr_reader :commands, :record_path
83
+
84
+ def initialize(commands:, record_path:)
85
+ @commands = commands
86
+ @record_path = record_path
87
+ end
88
+ end
89
+
90
+ class VerifyResult
91
+ attr_reader :record_path, :expected_output, :actual_output, :diff, :stderr_diff
92
+
93
+ def initialize(verified:, record_path:, expected_output: nil, actual_output: nil,
94
+ expected_stderr: nil, actual_stderr: nil, expected_status: nil, actual_status: nil,
95
+ command_executed: true)
96
+ @verified = verified
97
+ @record_path = record_path
98
+ @expected_output = expected_output
99
+ @actual_output = actual_output
100
+ @expected_stderr = expected_stderr
101
+ @actual_stderr = actual_stderr
102
+ @expected_status = expected_status
103
+ @actual_status = actual_status
104
+ @command_executed = command_executed
105
+
106
+ if !verified && expected_output && actual_output
107
+ @diff = generate_diff(expected_output, actual_output)
108
+ end
109
+
110
+ if !verified && expected_stderr && actual_stderr && expected_stderr != actual_stderr
111
+ @stderr_diff = generate_diff(expected_stderr, actual_stderr)
112
+ end
113
+ end
114
+
115
+ def verified?
116
+ @verified
117
+ end
118
+
119
+ def output
120
+ @actual_output
121
+ end
122
+
123
+ def error_message
124
+ return nil if verified?
125
+
126
+ "Output verification failed\nExpected: #{@expected_output.chomp}\nActual: #{@actual_output.chomp}"
127
+ end
128
+
129
+ def command_executed?
130
+ @command_executed
131
+ end
132
+
133
+ private
134
+
135
+ def generate_diff(expected, actual)
136
+ expected_lines = expected.lines
137
+ actual_lines = actual.lines
138
+ diff = []
139
+
140
+ # Simple diff for now
141
+ expected_lines.each do |line|
142
+ unless actual_lines.include?(line)
143
+ diff << "-#{line.chomp}"
144
+ end
145
+ end
146
+
147
+ actual_lines.each do |line|
148
+ unless expected_lines.include?(line)
149
+ diff << "+#{line.chomp}"
150
+ end
151
+ end
152
+
153
+ diff.join("\n")
154
+ end
155
+ end
156
+
157
+ class << self
158
+ attr_accessor :last_output
159
+
160
+ def scrub_text(text)
161
+ return text unless configuration.scrub_credentials && text
162
+
163
+ scrubbed = text.dup
164
+ configuration.credential_patterns.each do |pattern|
165
+ scrubbed.gsub!(pattern) do |match|
166
+ # Replace with asterisks of the same length
167
+ "*" * match.length
168
+ end
169
+ end
170
+ scrubbed
171
+ end
172
+
173
+ def call(record_name, filter: nil)
174
+ raise ArgumentError, "record_name is required" if record_name.nil? || record_name.empty?
175
+
176
+ record_path = build_record_path(record_name)
177
+
178
+ # Create recorder to handle stubbing and command recording
179
+ recorder = Recorder.new
180
+ recorder.record_calls(:capture3, :system)
181
+
182
+ yield
183
+
184
+ # Save commands using new format
185
+ FileUtils.mkdir_p(File.dirname(record_path))
186
+ # Don't load existing data when creating new record
187
+ record = Record.new(record_path)
188
+ record.clear # Clear any loaded data
189
+ recorder.commands.each { |cmd| record.add_command(cmd) }
190
+ record.save(filter: filter)
191
+
192
+ Result.new(commands: recorder.commands, record_path: Pathname.new(record_path))
193
+ end
194
+
195
+ def output
196
+ last_output
197
+ end
198
+
199
+ def use_record(record_name, options = {}, &block)
200
+ raise ArgumentError, "record_name is required" if record_name.nil? || record_name.empty?
201
+
202
+ record_path = build_record_path(record_name)
203
+ record_mode = options[:record] || :once
204
+ filter = options[:filter]
205
+
206
+ case record_mode
207
+ when :none
208
+ # Never record, only replay
209
+ unless File.exist?(record_path)
210
+ raise RecordNotFoundError, "Record not found: #{record_path}"
211
+ end
212
+ replay_record(record_path, &block)
213
+ when :all
214
+ # Always record
215
+ record_and_save_record(record_path, filter: filter, &block)
216
+ when :once
217
+ # Record if doesn't exist, replay if exists
218
+ if File.exist?(record_path)
219
+ replay_record(record_path, &block)
220
+ else
221
+ record_and_save_record(record_path, filter: filter, &block)
222
+ end
223
+ when :new_episodes
224
+ # Record new commands not in record
225
+ # For now, simplified: just append new recordings
226
+ record_new_episode(record_path, filter: filter, &block)
227
+ else
228
+ raise ArgumentError, "Unknown record mode: #{record_mode}"
229
+ end
230
+ end
231
+
232
+ def verify(record_name, mode: :strict, matcher: nil, &block)
233
+ record_path = build_record_path(record_name)
234
+
235
+ record = Record.load_or_create(record_path)
236
+ unless record.exists?
237
+ raise RecordNotFoundError, "Record not found: #{record_path}"
238
+ end
239
+
240
+ if record.empty?
241
+ raise RecordNotFoundError, "No commands found in record"
242
+ end
243
+
244
+ # For verify, we only handle single command verification for now
245
+ # Use the first command
246
+ command = record.commands.first
247
+
248
+ # Create recorder for verification
249
+ recorder = Recorder.new
250
+
251
+ if mode == :playback
252
+ # Playback mode: return recorded output without running command
253
+ recorder.setup_playback_stub(command)
254
+
255
+ yield
256
+
257
+ # In playback mode, always verified
258
+ VerifyResult.new(
259
+ verified: true,
260
+ record_path: Pathname.new(record_path),
261
+ expected_output: command.stdout,
262
+ actual_output: command.stdout,
263
+ expected_stderr: command.stderr,
264
+ actual_stderr: command.stderr,
265
+ expected_status: command.status,
266
+ actual_status: command.status,
267
+ command_executed: false
268
+ )
269
+ elsif matcher
270
+ # Custom matcher verification
271
+ recorder.setup_verification_stub(command)
272
+
273
+ yield
274
+
275
+ # Call custom matcher - convert command back to hash format for matcher
276
+ recorded_data = command.to_h
277
+ verified = matcher.call(recorded_data, recorder.verification_data)
278
+
279
+ VerifyResult.new(
280
+ verified: verified,
281
+ record_path: Pathname.new(record_path),
282
+ expected_output: command.stdout,
283
+ actual_output: recorder.verification_data["stdout"],
284
+ expected_stderr: command.stderr,
285
+ actual_stderr: recorder.verification_data["stderr"],
286
+ expected_status: command.status,
287
+ actual_status: recorder.verification_data["status"]
288
+ )
289
+ else
290
+ # Default strict mode
291
+ recorder.setup_verification_stub(command)
292
+
293
+ yield
294
+
295
+ # Compare outputs
296
+ actual_stdout = recorder.verification_data["stdout"]
297
+ actual_stderr = recorder.verification_data["stderr"]
298
+ actual_status = recorder.verification_data["status"]
299
+
300
+ verified =
301
+ command.stdout == actual_stdout &&
302
+ command.stderr == actual_stderr &&
303
+ command.status == actual_status
304
+
305
+ VerifyResult.new(
306
+ verified: verified,
307
+ record_path: Pathname.new(record_path),
308
+ expected_output: command.stdout,
309
+ actual_output: actual_stdout,
310
+ expected_stderr: command.stderr,
311
+ actual_stderr: actual_stderr,
312
+ expected_status: command.status,
313
+ actual_status: actual_status
314
+ )
315
+ end
316
+ end
317
+
318
+ def verify!(record_name, mode: :strict, matcher: nil, &block)
319
+ result = verify(record_name, mode: mode, matcher: matcher, &block)
320
+
321
+ unless result.verified?
322
+ error_message = "Backspin verification failed!\n"
323
+ error_message += "Record: #{result.record_path}\n"
324
+ error_message += "Expected output:\n#{result.expected_output}\n"
325
+ error_message += "Actual output:\n#{result.actual_output}\n"
326
+
327
+ if result.diff && !result.diff.empty?
328
+ error_message += "Diff:\n#{result.diff}\n"
329
+ end
330
+
331
+ if result.stderr_diff && !result.stderr_diff.empty?
332
+ error_message += "Stderr diff:\n#{result.stderr_diff}\n"
333
+ end
334
+
335
+ # Raise RSpec's expectation failure for proper integration
336
+ raise RSpec::Expectations::ExpectationNotMetError, error_message
337
+ end
338
+
339
+ result
340
+ end
341
+
342
+ private
343
+
344
+ def replay_record(record_path, &block)
345
+ record = Record.load_or_create(record_path)
346
+ unless record.exists?
347
+ raise RecordNotFoundError, "Record not found: #{record_path}"
348
+ end
349
+
350
+ if record.empty?
351
+ raise RecordNotFoundError, "No commands found in record"
352
+ end
353
+
354
+ # Create recorder in replay mode
355
+ recorder = Recorder.new(mode: :replay, record: record)
356
+ recorder.setup_replay_stubs
357
+
358
+ block_return_value = yield
359
+
360
+ # Return stdout, stderr, status if the block returned capture3 results
361
+ # Otherwise return the block's return value
362
+ if block_return_value.is_a?(Array) && block_return_value.size == 3 &&
363
+ block_return_value[0].is_a?(String) && block_return_value[1].is_a?(String)
364
+ # Convert status to integer for consistency
365
+ stdout, stderr, status = block_return_value
366
+ status_int = status.respond_to?(:exitstatus) ? status.exitstatus : status
367
+ [stdout, stderr, status_int]
368
+ else
369
+ block_return_value
370
+ end
371
+ end
372
+
373
+ def record_and_save_record(record_path, filter: nil, &block)
374
+ # Create recorder to handle stubbing and command recording
375
+ recorder = Recorder.new
376
+ recorder.record_calls(:capture3, :system)
377
+
378
+ block_return_value = yield
379
+
380
+ # Save commands using new format
381
+ FileUtils.mkdir_p(File.dirname(record_path))
382
+ # Don't load existing data when creating new record
383
+ record = Record.new(record_path)
384
+ record.clear # Clear any loaded data
385
+ recorder.commands.each { |cmd| record.add_command(cmd) }
386
+ record.save(filter: filter)
387
+
388
+ # Return appropriate value
389
+ if block_return_value.is_a?(Array) && block_return_value.size == 3
390
+ # Return stdout, stderr, status as integers
391
+ stdout, stderr, status = block_return_value
392
+ [stdout, stderr, status.respond_to?(:exitstatus) ? status.exitstatus : status]
393
+ else
394
+ block_return_value
395
+ end
396
+ end
397
+
398
+ def record_new_episode(record_path, filter: nil, &block)
399
+ # For new_episodes mode, we'd need to track which commands have been seen
400
+ # For now, simplified implementation that just appends
401
+ record = Record.load_or_create(record_path)
402
+
403
+ # Create recorder to handle stubbing and command recording
404
+ recorder = Recorder.new
405
+ recorder.record_calls(:capture3, :system)
406
+
407
+ result = yield
408
+
409
+ # Save all recordings (existing + new)
410
+ if recorder.commands.any?
411
+ recorder.commands.each { |cmd| record.add_command(cmd) }
412
+ record.save(filter: filter)
413
+ end
414
+
415
+ # Return appropriate value
416
+ if result.is_a?(Array) && result.size == 3
417
+ stdout, stderr, status = result
418
+ [stdout, stderr, status.respond_to?(:exitstatus) ? status.exitstatus : status]
419
+ else
420
+ result
421
+ end
422
+ end
423
+
424
+ def build_record_path(name)
425
+ backspin_dir = configuration.backspin_dir
426
+ backspin_dir.mkpath
427
+
428
+ File.join(backspin_dir, "#{name}.yaml")
429
+ end
430
+ end
431
+ end
metadata ADDED
@@ -0,0 +1,90 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: backspin
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.2.1
5
+ platform: ruby
6
+ authors:
7
+ - Rob Sanheim
8
+ bindir: bin
9
+ cert_chain: []
10
+ date: 1980-01-02 00:00:00.000000000 Z
11
+ dependencies:
12
+ - !ruby/object:Gem::Dependency
13
+ name: rspec-mocks
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - "~>"
17
+ - !ruby/object:Gem::Version
18
+ version: '3.0'
19
+ type: :runtime
20
+ prerelease: false
21
+ version_requirements: !ruby/object:Gem::Requirement
22
+ requirements:
23
+ - - "~>"
24
+ - !ruby/object:Gem::Version
25
+ version: '3.0'
26
+ - !ruby/object:Gem::Dependency
27
+ name: ostruct
28
+ requirement: !ruby/object:Gem::Requirement
29
+ requirements:
30
+ - - ">="
31
+ - !ruby/object:Gem::Version
32
+ version: '0'
33
+ type: :runtime
34
+ prerelease: false
35
+ version_requirements: !ruby/object:Gem::Requirement
36
+ requirements:
37
+ - - ">="
38
+ - !ruby/object:Gem::Version
39
+ version: '0'
40
+ description: Backspin is a Ruby library for characterization testing of command-line
41
+ interfaces. Inspired by VCR's cassette-based approach, it records and replays CLI
42
+ interactions to make testing faster and more deterministic.
43
+ email:
44
+ - rsanheim@gmail.com
45
+ executables:
46
+ - setup
47
+ extensions: []
48
+ extra_rdoc_files: []
49
+ files:
50
+ - ".gitignore"
51
+ - ".rspec"
52
+ - ".standard.yml"
53
+ - CHANGELOG.md
54
+ - CLAUDE.md
55
+ - Gemfile
56
+ - LICENSE.txt
57
+ - README.md
58
+ - Rakefile
59
+ - backspin.gemspec
60
+ - bin/setup
61
+ - lib/backspin.rb
62
+ - lib/backspin/command.rb
63
+ - lib/backspin/record.rb
64
+ - lib/backspin/recorder.rb
65
+ - lib/backspin/version.rb
66
+ homepage: https://github.com/rsanheim/backspin
67
+ licenses:
68
+ - MIT
69
+ metadata:
70
+ homepage_uri: https://github.com/rsanheim/backspin
71
+ source_code_uri: https://github.com/rsanheim/backspin
72
+ changelog_uri: https://github.com/rsanheim/backspin/blob/main/CHANGELOG.md
73
+ rdoc_options: []
74
+ require_paths:
75
+ - lib
76
+ required_ruby_version: !ruby/object:Gem::Requirement
77
+ requirements:
78
+ - - ">="
79
+ - !ruby/object:Gem::Version
80
+ version: 2.5.0
81
+ required_rubygems_version: !ruby/object:Gem::Requirement
82
+ requirements:
83
+ - - ">="
84
+ - !ruby/object:Gem::Version
85
+ version: '0'
86
+ requirements: []
87
+ rubygems_version: 3.6.7
88
+ specification_version: 4
89
+ summary: Record and replay CLI interactions for testing
90
+ test_files: []