backspin 0.7.0 → 0.8.0

data/README.md

@@ -5,19 +5,19 @@
 [![CircleCI](https://img.shields.io/circleci/build/github/rsanheim/backspin/main)](https://circleci.com/gh/rsanheim/backspin)
 [![Last Commit](https://img.shields.io/github/last-commit/rsanheim/backspin/main)](https://github.com/rsanheim/backspin/commits/main)
 
-Backspin records and replays CLI interactions in Ruby for easy snapshot testing of command-line interfaces. Currently supports `Open3.capture3` and `system` and requires `rspec`, as it uses `rspec-mocks` under the hood.
+Backspin records command output and block output in Ruby for easy snapshot testing of command-line interfaces. It supports direct command runs via `Open3.capture3` and block capture for more complex scenarios.
 
 **NOTE:** Backspin should be considered alpha while pre version 1.0. It is in heavy development along-side some real-world CLI apps, so expect things to change and mature.
 
-Inspired by [VCR](https://github.com/vcr/vcr) and other [golden master](https://en.wikipedia.org/wiki/Golden_master_(software_development)) testing libraries.
+Inspired by [VCR](https://github.com/vcr/vcr) and other [characterization (aka golden master)](https://en.wikipedia.org/wiki/Characterization_test) testing libraries.
 
 ## Overview
 
-Backspin is a Ruby library for snapshot testing (or characterization testing) of command-line interfaces. While VCR records and replays HTTP interactions, Backspin records and replays CLI interactions - capturing stdout, stderr, and exit status from shell commands. 
+Backspin is a Ruby library for snapshot testing (or characterization testing) of command-line interfaces. While VCR records and replays HTTP interactions, Backspin records stdout, stderr, and exit status from shell commands, or captures all output from a block.
 
 ## Installation
 
-Requires Ruby 3+ and will use rspec-mocks under the hood...Backspin has not been tested in other test frameworks.
+Requires Ruby 3+.
 
 Add this line to your application's Gemfile in the `:test` group:
 
@@ -31,7 +31,7 @@ And then run `bundle install`.
 
 ## Usage
 
-### Quick Start
+### Quick Start (Command Runs)
 
 The simplest way to use Backspin is with the `run` method, which automatically records on the first execution and verifies on subsequent runs.
 
@@ -39,23 +39,41 @@ The simplest way to use Backspin is with the `run` method, which automatically r
 require "backspin"
 
 # First run: records the output
-result = Backspin.run("my_command") do
-  Open3.capture3("echo hello world")
-end
+result = Backspin.run(["echo", "hello world"], name: "my_command")
 
 # Subsequent runs: verifies the output matches and raises on mismatch
-Backspin.run("my_command") do
-  Open3.capture3("echo hello world")  # Passes - output matches
-end
+Backspin.run(["echo", "hello world"], name: "my_command")
 
 # This will raise an error automatically
-Backspin.run("my_command") do
-  Open3.capture3("echo hello mars")  
+Backspin.run(["echo", "hello mars"], name: "my_command")
+# Raises Backspin::VerificationError because output doesn't match
+```
+
+You can also pass a string command (which invokes a shell):
+
+```ruby
+Backspin.run("echo hello", name: "string_command")
+```
+
+### Block Capture
+
+Use block capture when you need to run multiple commands or use APIs that already write to stdout/stderr:
+
+```ruby
+# Capture all output from the block
+result = Backspin.run(name: "block_capture") do
+  system("echo from system")
+  puts "from puts"
+  `echo from backticks`
+end
+
+# Alias form
+Backspin.capture("block_capture") do
+  puts "from capture"
 end
-# Raises RSpec::Expectations::ExpectationNotMetError because output doesn't match
 ```
 
-By default, `Backspin.run` will raise an exception when verification fails, making your tests fail automatically. This is the recommended approach for most scenarios.
+Block capture records a single combined stdout/stderr snapshot. Exit status is a placeholder (`0`) in this mode.
 
 ### Recording Modes
 
@@ -63,48 +81,29 @@ Backspin supports different modes for controlling how commands are recorded and
 
 ```ruby
 # Auto mode (default): Record on first run, verify on subsequent runs
-result = Backspin.run("my_command") do
-  Open3.capture3("echo hello")
-end
+Backspin.run(["echo", "hello"], name: "my_command")
 
 # Explicit record mode: Always record, overwriting existing recordings
-result = Backspin.run("echo_test", mode: :record) do
-  Open3.capture3("echo hello")
-end
-# This will save the output to `fixtures/backspin/echo_test.yml`.
+Backspin.run(["echo", "hello"], name: "echo_test", mode: :record)
 
 # Explicit verify mode: Always verify against existing recording
-result = Backspin.run("echo_test", mode: :verify) do
-  Open3.capture3("echo hello")
-end
+result = Backspin.run(["echo", "hello"], name: "echo_test", mode: :verify)
 expect(result.verified?).to be true
-
-# Playback mode: Return recorded output without running the command
-result = Backspin.run("slow_command", mode: :playback) do
-  Open3.capture3("slow_command")  # Not executed - returns recorded output
-end
 ```
 
-### The run! method
-
-**NOTE:** This method is deprecated and will be removed soon.
-
-The `run!` method is maintained for backwards compatibility and works identically to `run`. Since `run` now raises on verification failure by default, both methods behave the same way:
+### Environment Variables
 
 ```ruby
-# Both of these are equivalent and will raise on verification failure
-Backspin.run("echo_test") do
-  Open3.capture3("echo hello")
-end
-
-Backspin.run!("echo_test") do
-  Open3.capture3("echo hello")
-end
+Backspin.run(
+  ["ruby", "-e", "print ENV.fetch('MY_ENV_VAR')"],
+  name: "with_env",
+  env: {"MY_ENV_VAR" => "value"}
+)
 ```
 
-For new code, we recommend using `run` as it's the primary API method.
+If `env:` is not provided, it is not passed to `Open3.capture3` and is not recorded.
 
-### Custom matchers
+### Custom Matchers
 
 For cases where full matching isn't suitable, you can override via `matcher:`. **NOTE**: If you provide
 custom matchers, that is the only matching that will be done. Default matching is skipped if user-provided
@@ -113,14 +112,11 @@ matchers are present.
 You can override the full match logic with a proc:
 
 ```ruby
-# Match stdout and status, ignore stderr
 my_matcher = ->(recorded, actual) {
-  recorded["stdout"] == actual["stdout"] && recorded["status"] != actual["status"]
+  recorded["stdout"] == actual["stdout"] && recorded["status"] == actual["status"]
 }
 
-result = Backspin.run("my_test", matcher: { all: my_matcher }) do
-  Open3.capture3("echo hello")
-end
+result = Backspin.run(["echo", "hello"], name: "my_test", matcher: {all: my_matcher})
 ```
 
 Or you can override specific fields:
@@ -131,18 +127,7 @@ timestamp_matcher = ->(recorded, actual) {
   recorded.match?(/\d{4}-\d{2}-\d{2}/) && actual.match?(/\d{4}-\d{2}-\d{2}/)
 }
 
-result = Backspin.run("timestamp_test", matcher: { stdout: timestamp_matcher }) do
-  Open3.capture3("date")
-end
-
-# Match version numbers in stderr
-version_matcher = ->(recorded, actual) {
-  recorded[/v(\d+)\./, 1] == actual[/v(\d+)\./, 1]
-}
-
-result = Backspin.run("version_check", matcher: { stderr: version_matcher }) do
-  Open3.capture3("node --version")
-end
+result = Backspin.run(["date"], name: "timestamp_test", matcher: {stdout: timestamp_matcher})
 ```
 
 For more matcher examples and detailed documentation, see [MATCHERS.md](MATCHERS.md).
@@ -152,21 +137,18 @@ For more matcher examples and detailed documentation, see [MATCHERS.md](MATCHERS
 The API returns a `RecordResult` object with helpful methods:
 
 ```ruby
-result = Backspin.run("my_test") do
-  Open3.capture3("echo out; echo err >&2; exit 42")
-end
+result = Backspin.run(["sh", "-c", "echo out; echo err >&2; exit 42"], name: "my_test")
 
 # Check the mode
 result.recorded?  # true on first run
 result.verified?  # true/false on subsequent runs, nil when recording
-result.playback?  # true in playback mode
 
-# Access output (first command for single commands)
+# Access output (first command)
 result.stdout     # "out\n"
-result.stderr     # "err\n" 
+result.stderr     # "err\n"
 result.status     # 42
 result.success?   # false (non-zero exit)
-result.output     # The raw return value from the block
+result.output     # [stdout, stderr, status] for command runs
 
 # Debug information
 result.record_path  # Path to the YAML file
@@ -174,47 +156,12 @@ result.error_message  # Human-readable error if verification failed
 result.diff  # Diff between expected and actual output
 ```
 
-### Multiple Commands
-
-Backspin automatically records and verifies all commands executed in a block:
-
-```ruby
-result = Backspin.run("multi_command_test") do
-  # All of these commands will be recorded
-  version, = Open3.capture3("ruby --version")
-  files, = Open3.capture3("ls -la")
-  system("echo 'Processing...'")  # Note: system doesn't capture output
-  data, stderr, = Open3.capture3("curl https://api.example.com/data")
-  
-  # Return whatever you need
-  { version: version.strip, file_count: files.lines.count, data: data }
-end
-
-# Access individual command results
-result.commands.size       # 4
-result.multiple_commands?  # true
-
-# For multiple commands, use these accessors
-result.all_stdout  # Array of stdout from each command
-result.all_stderr  # Array of stderr from each command
-result.all_status  # Array of exit statuses
-
-# Or access specific commands
-result.commands[0].stdout  # Ruby version output
-result.commands[1].stdout  # ls output
-result.commands[2].status  # system call exit status (stdout is empty)
-result.commands[3].stderr  # curl errors if any
-```
-
-When verifying multiple commands, Backspin ensures all commands match in the exact order they were recorded. If any command differs, you'll get a detailed error showing which commands failed.
-
 ### Configuration
 
 You can configure Backspin's behavior globally:
 
 ```ruby
 Backspin.configure do |config|
-  # Both run and capture methods will raise on verification failure by default
   config.raise_on_verification_failure = false # default is true
   config.backspin_dir = "spec/fixtures/cli_records" # default is "fixtures/backspin"
   config.scrub_credentials = false # default is true
@@ -222,42 +169,31 @@ end
 ```
 
 The `raise_on_verification_failure` setting affects both `Backspin.run` and `Backspin.capture`:
-- When `true` (default): Both methods raise exceptions on verification failure
-  - `run` raises `RSpec::Expectations::ExpectationNotMetError`
-  - `capture` raises `Backspin::VerificationError` (framework-agnostic)
+- When `true` (default): Both methods raise `Backspin::VerificationError` on verification failure
 - When `false`: Both methods return a result with `verified?` set to false
 
 If you need to disable the raising behavior for a specific test, you can temporarily configure it:
 
 ```ruby
-# Temporarily disable raising for this block
 Backspin.configure do |config|
   config.raise_on_verification_failure = false
 end
 
-result = Backspin.run("my_test") do
-  Open3.capture3("echo different")
-end
+result = Backspin.run(["echo", "different"], name: "my_test")
 # result.verified? will be false but won't raise
 
-# Reset configuration
 Backspin.reset_configuration!
 ```
 
 ### Credential Scrubbing
 
-If the CLI interaction you are recording contains sensitive data in stdout or stderr, you should be careful to make sure it is not recorded to yaml!
+If the CLI interaction you are recording contains sensitive data in stdout/stderr, you should be careful to make sure it is not recorded to YAML.
 
-By default, Backspin automatically tries to scrub [common credential patterns](https://github.com/rsanheim/backspin/blob/f8661f084aad0ae759cd971c4af31ccf9bdc6bba/lib/backspin.rb#L46-L65) from records, but this will only handle some common cases.
-Always review your record files before commiting them to source control. 
-
-Use a tool like [trufflehog](https://github.com/trufflesecurity/trufflehog) or [gitleaks](https://github.com/gitleaks/gitleaks) run via a pre-commit to catch any sensitive data before commit. 
+By default, Backspin automatically tries to scrub common credential patterns from recorded stdout, stderr, args, and env values. Always review your record files before commiting them to source control.
 
 ```ruby
 # This will automatically scrub AWS keys, API tokens, passwords, etc.
-Backspin.run("aws_command") do
-  Open3.capture3("aws s3 ls")
-end
+Backspin.run(["aws", "s3", "ls"], name: "aws_command")
 
 # Add custom patterns to scrub
 Backspin.configure do |config|

data/backspin.gemspec

@@ -24,7 +24,4 @@ Gem::Specification.new do |spec|
   spec.bindir = "exe"
   spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
-
-  spec.add_dependency "ostruct"
-  spec.add_dependency "rspec-mocks", "~> 3"
 end

data/examples/match_on_example.rb

@@ -2,7 +2,8 @@
 # frozen_string_literal: true
 
 require "bundler/inline"
-require "open3"
+
+# Example usage of Backspin matchers
 
 gemfile do
   source "https://rubygems.org"
@@ -13,24 +14,18 @@ end
 puts "Example 1: Matching timestamps with custom matcher"
 puts "-" * 50
 
-# First run: Record the output
-result = Backspin.run("timestamp_example") do
-  Open3.capture3("date '+%Y-%m-%d %H:%M:%S'")
-end
+result = Backspin.run(["date", "+%Y-%m-%d %H:%M:%S"], name: "timestamp_example")
 puts "Recorded: #{result.stdout.chomp}"
 
-# Sleep to ensure different timestamp
 sleep 1
 
-# Second run: Verify with custom matcher
-result = Backspin.run("timestamp_example",
-  match_on: [:stdout, lambda { |recorded, actual|
-    # Both should have the same date format
-    recorded.match?(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/) &&
-    actual.match?(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/)
-  }]) do
-  Open3.capture3("date '+%Y-%m-%d %H:%M:%S'")
-end
+result = Backspin.run(["date", "+%Y-%m-%d %H:%M:%S"], name: "timestamp_example",
+  matcher: {
+    stdout: ->(recorded, actual) {
+      recorded.match?(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/) &&
+        actual.match?(/\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/)
+    }
+  })
 
 puts "Current:  #{result.stdout.chomp}"
 puts "Verified: #{result.verified?}"
@@ -40,35 +35,23 @@ puts
 puts "Example 2: Matching multiple fields with different patterns"
 puts "-" * 50
 
-# Record a command with dynamic content
-Backspin.run("multi_field_example") do
-  script = <<~BASH
-    echo "PID: $$"
-    echo "Error: Timeout at $(date '+%H:%M:%S')" >&2
-    exit 1
-  BASH
-  Open3.capture3("bash", "-c", script)
-end
+script = <<~SH
+  echo "PID: $$"
+  echo "Error: Timeout at $(date '+%H:%M:%S')" >&2
+  exit 1
+SH
+
+Backspin.run(["sh", "-c", script], name: "multi_field_example")
 
-# Verify with different PID and timestamp
-result = Backspin.run("multi_field_example",
-  match_on: [
-    [:stdout, lambda { |recorded, actual|
-      # Both should have PID format
+result = Backspin.run(["sh", "-c", script], name: "multi_field_example",
+  matcher: {
+    stdout: ->(recorded, actual) {
       recorded.match?(/PID: \d+/) && actual.match?(/PID: \d+/)
-    }],
-    [:stderr, lambda { |recorded, actual|
-      # Both should have timeout error, ignore timestamp
+    },
+    stderr: ->(recorded, actual) {
       recorded.match?(/Error: Timeout at/) && actual.match?(/Error: Timeout at/)
-    }]
-  ]) do
-  script = <<~BASH
-    echo "PID: $$"
-    echo "Error: Timeout at $(date '+%H:%M:%S')" >&2
-    exit 1
-  BASH
-  Open3.capture3("bash", "-c", script)
-end
+    }
+  })
 
 puts "Stdout:   #{result.stdout.chomp}"
 puts "Stderr:   #{result.stderr.chomp}"
@@ -80,37 +63,25 @@ puts
 puts "Example 3: Mixed field matching"
 puts "-" * 50
 
-# Record with specific values
-Backspin.run("mixed_matching") do
-  script = <<~BASH
-    echo "Version: 1.2.3"
-    echo "Build: $(date +%s)"
-    echo "Status: OK"
-  BASH
-  Open3.capture3("bash", "-c", script)
-end
+script = <<~SH
+  echo "Version: 1.2.3"
+  echo "Build: $(date +%s)"
+  echo "Status: OK"
+SH
 
-# Verify - stdout uses custom matcher, stderr must match exactly
-result = Backspin.run("mixed_matching",
-  match_on: [:stdout, lambda { |recorded, actual|
-    # Version and Status must match, Build can differ
-    recorded_lines = recorded.lines
-    actual_lines = actual.lines
-
-    recorded_lines[0] == actual_lines[0] && # Version line must match
-    recorded_lines[1].start_with?("Build:") && actual_lines[1].start_with?("Build:") && # Build line format
-    recorded_lines[2] == actual_lines[2] # Status line must match
-  }]) do
-  script = <<~BASH
-    echo "Version: 1.2.3"
-    echo "Build: $(date +%s)"
-    echo "Status: OK"
-  BASH
-  Open3.capture3("bash", "-c", script)
-end
+Backspin.run(["sh", "-c", script], name: "mixed_matching")
+
+result = Backspin.run(["sh", "-c", script], name: "mixed_matching",
+  matcher: {
+    stdout: ->(recorded, actual) {
+      recorded_lines = recorded.lines
+      actual_lines = actual.lines
+
+      recorded_lines[0] == actual_lines[0] &&
+        recorded_lines[1].start_with?("Build:") && actual_lines[1].start_with?("Build:") &&
+        recorded_lines[2] == actual_lines[2]
+    }
+  })
 
 puts "Output:\n#{result.stdout}"
 puts "Verified: #{result.verified?}"
-
-# Cleanup
-FileUtils.rm_rf("fixtures/backspin")

data/lib/backspin/command.rb

@@ -4,11 +4,12 @@ require_relative "command_result"
 
 module Backspin
   class Command
-    attr_reader :args, :result, :recorded_at, :method_class
+    attr_reader :args, :env, :result, :recorded_at, :method_class
 
-    def initialize(method_class:, args:, stdout: nil, stderr: nil, status: nil, result: nil, recorded_at: nil)
+    def initialize(method_class:, args:, env: nil, stdout: nil, stderr: nil, status: nil, result: nil, recorded_at: nil)
       @method_class = method_class
       @args = args
+      @env = env
       @recorded_at = recorded_at
 
       # Accept either a CommandResult or individual stdout/stderr/status
@@ -38,6 +39,8 @@ module Backspin
         "recorded_at" => @recorded_at
       }
 
+      data["env"] = scrub_env(@env) if @env
+
       # Apply filter if provided
       data = filter.call(data) if filter
 
@@ -50,18 +53,16 @@ module Backspin
       method_class = case data["command_type"]
       when "Open3::Capture3"
         Open3::Capture3
-      when "Kernel::System"
-        ::Kernel::System
       when "Backspin::Capturer"
         Backspin::Capturer
       else
-        # Default to capture3 for backwards compatibility
-        Open3::Capture3
+        raise RecordFormatError, "Unknown command type: #{data["command_type"]}"
       end
 
       new(
         method_class: method_class,
         args: data["args"],
+        env: data["env"],
         stdout: data["stdout"],
         stderr: data["stderr"],
         status: data["status"],
@@ -74,19 +75,34 @@ module Backspin
     def scrub_args(args)
       return args unless Backspin.configuration.scrub_credentials && args
 
-      args.map do |arg|
-        case arg
-        when String
-          Backspin.scrub_text(arg)
-        when Array
-          scrub_args(arg)
-        when Hash
-          arg.transform_values { |v| v.is_a?(String) ? Backspin.scrub_text(v) : v }
-        else
-          arg
+      case args
+      when String
+        Backspin.scrub_text(args)
+      when Array
+        args.map do |arg|
+          case arg
+          when String
+            Backspin.scrub_text(arg)
+          when Array
+            scrub_args(arg)
+          when Hash
+            arg.transform_values { |v| v.is_a?(String) ? Backspin.scrub_text(v) : v }
+          else
+            arg
+          end
         end
+      when Hash
+        args.transform_values { |v| v.is_a?(String) ? Backspin.scrub_text(v) : v }
+      else
+        args
       end
     end
+
+    def scrub_env(env)
+      return env unless Backspin.configuration.scrub_credentials && env
+
+      env.transform_values { |value| value.is_a?(String) ? Backspin.scrub_text(value) : value }
+    end
   end
 end
 
@@ -95,11 +111,6 @@ module Open3
   class Capture3; end
 end
 
-# Define the Kernel::System class for identification
-module ::Kernel
-  class System; end
-end
-
 # Define the Backspin::Capturer class for identification
 module Backspin
   class Capturer; end

data/lib/backspin/command_diff.rb

@@ -28,17 +28,23 @@ module Backspin
       return nil if verified?
 
       parts = []
+      recorded_hash = recorded_command.to_h
+      actual_hash = actual_command.to_h
 
       unless method_classes_match?
         parts << "Command type mismatch: expected #{recorded_command.method_class.name}, got #{actual_command.method_class.name}"
       end
 
-      parts << stdout_diff if recorded_command.stdout != actual_command.stdout
+      if recorded_hash["stdout"] != actual_hash["stdout"]
+        parts << stdout_diff(recorded_hash["stdout"], actual_hash["stdout"])
+      end
 
-      parts << stderr_diff if recorded_command.stderr != actual_command.stderr
+      if recorded_hash["stderr"] != actual_hash["stderr"]
+        parts << stderr_diff(recorded_hash["stderr"], actual_hash["stderr"])
+      end
 
-      if recorded_command.status != actual_command.status
-        parts << "Exit status: expected #{recorded_command.status}, got #{actual_command.status}"
+      if recorded_hash["status"] != actual_hash["status"]
+        parts << "Exit status: expected #{recorded_hash["status"]}, got #{actual_hash["status"]}"
       end
 
       parts.join("\n\n")
@@ -67,12 +73,12 @@ module Backspin
       @matcher.failure_reason
     end
 
-    def stdout_diff
-      "stdout diff:\n#{generate_line_diff(recorded_command.stdout, actual_command.stdout)}"
+    def stdout_diff(expected, actual)
+      "[stdout]\n#{generate_line_diff(expected, actual)}"
     end
 
-    def stderr_diff
-      "stderr diff:\n#{generate_line_diff(recorded_command.stderr, actual_command.stderr)}"
+    def stderr_diff(expected, actual)
+      "[stderr]\n#{generate_line_diff(expected, actual)}"
     end
 
     def generate_line_diff(expected, actual)

data/lib/backspin/configuration.rb

@@ -8,7 +8,7 @@ module Backspin
     attr_accessor :scrub_credentials
     # The directory where backspin will store its files - defaults to fixtures/backspin
     attr_accessor :backspin_dir
-    # Whether to raise an exception when verification fails in `run` method - defaults to true
+    # Whether to raise an exception when verification fails in `run`/`capture` - defaults to true
     attr_accessor :raise_on_verification_failure
     # Regex patterns to scrub from saved output
     attr_reader :credential_patterns