backspin 0.8.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3937af0a8fc0d808e6cc2f8788481ee479a46b6c21353a7c73820711995f9b96
-  data.tar.gz: a30e4ebd5608ad6718fc0d5cbda3685adc49b97cbb5af188db604f3b1b0de3be
+  metadata.gz: c8a6c6a0ef97c99ace7fb068e7b85bc40c2f45594bc30ce5002920fd62fcd384
+  data.tar.gz: f62c15526c0a19a4ae876b8c737b172e2557c54a55f4d5045a0dbc499ccfcd51
 SHA512:
-  metadata.gz: 1343b731c00281871a50f4d61e276068d4aa18d54ba60f80f7ad780f88191c06f6af5168d53c7559f82c9d722e67521b8ca56aeb7998a2f61f410b925caff1aa
-  data.tar.gz: a3e496f886dae99cc6f624399655955b18b131991f332b2470ecd57ed74fb7e4c8e58a8cf89237f2faa9e10456ab67566ebc18da3c6fdc253716a8abb4728061
+  metadata.gz: 117a79e8e448bae03c68b44e8a6de23671d746d2ea664094e7dc82792d8ff323af7c6faf73f85a9aa020bb259127660abc8d7687cf861b627049b6e3b88ff41f
+  data.tar.gz: dfeabf728ef8448d01889dbe2d62d9d2a8236da296d9f6ee21619f6e3f976218cd0af3da2ac8b7f4ab1ff9cbc19dd4afaef1ca64ac777f3f9215180a1349c1c0

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.9.0 - 2026-02-11
+* Breaking: `Backspin.run` and `Backspin.capture` now return `Backspin::BackspinResult` with explicit `result.actual` / `result.expected` snapshots.
+* Breaking: result convenience accessors (`result.stdout`, `result.stderr`, `result.status`) were removed in favor of snapshot access.
+* Breaking: record format bumped to 4.0 and now persists a single `snapshot` object (v3 records are rejected).
+* Simplification: removed legacy `Command`, `CommandResult`, and `RecordResult` layers; matcher/diff now operate directly on snapshots.
+* Added focused coverage for the new result contract and capture stream restoration behavior.
+* Updated project docs to reflect the BackspinResult + Snapshot API surface.
+
 ## 0.8.0 - 2026-02-05
 * Breaking: new `Backspin.run("command", name:, env:)` command API plus block capture via `Backspin.run(name:) { ... }` and `Backspin.capture("name") { ... }`
 * Breaking: remove `run!` and `:playback`

data/CLAUDE.md

@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Project Overview
 
-Backspin is a Ruby gem for characterization testing of command-line interfaces. It records and replays CLI interactions by capturing stdout, stderr, and exit status from shell commands - similar to how VCR works for HTTP interactions. Backspin uses "records" (YAML files) to store recorded command outputs.
+Backspin is a Ruby gem for characterization testing of command-line interfaces. It records and verifies CLI interactions by capturing stdout, stderr, and exit status from shell commands, similar to how VCR works for HTTP interactions. Backspin uses YAML "records" to store snapshots.
 
 ## Development Commands
 
@@ -45,16 +45,21 @@ bin/rake standard                # Alternative: Run via Rake task
 - Credential scrubbing logic
 - Configuration management (including `raise_on_verification_failure` which defaults to `true`)
 
-**Command Class** (`lib/backspin/command.rb`)
-- Represents a single CLI execution
-- Stores: args, stdout, stderr, status, recorded_at
+**Snapshot Class** (`lib/backspin/snapshot.rb`)
+- Represents a single captured execution snapshot
+- Stores: command type, args, env, stdout, stderr, status, recorded_at
+
+**BackspinResult Class** (`lib/backspin/backspin_result.rb`)
+- Return object from `run` and `capture`
+- Exposes `actual` and `expected` snapshots plus verification metadata
 
 **Record Class** (`lib/backspin/record.rb`)
 - Manages YAML record files
-- Handles recording/playback sequencing
+- Handles record/verify sequencing
 
-**RSpecMetadata** (`lib/backspin/rspec_metadata.rb`)
-- Auto-generates record names from RSpec context
+**Recorder Class** (`lib/backspin/recorder.rb`)
+- Implements block capture recording and verification
+- Restores stdout/stderr streams safely after capture
 
 ### Key Design Patterns
 
@@ -86,4 +91,4 @@ bin/rake standard                # Alternative: Run via Rake task
 
 ### Updating Credential Patterns
 - Add patterns to `DEFAULT_CREDENTIAL_PATTERNS` in `lib/backspin.rb`
-- Test with appropriate fixtures in specs
+- Test with appropriate fixtures in specs

data/CONTRIBUTING.md

@@ -17,7 +17,7 @@ Note that Backspin is in early development and the API _will_ change before stab
 
 ## Getting Started
 
-Backspin is a Ruby gem for characterization testing of command-line interfaces. It records and replays CLI interactions by capturing stdout, stderr, and exit status from shell commands - similar to how VCR works for HTTP interactions.
+Backspin is a Ruby gem for characterization testing of command-line interfaces. It records and verifies CLI interactions by capturing stdout, stderr, and exit status from shell commands, similar to how VCR works for HTTP interactions.
 
 ### Prerequisites
 
@@ -75,13 +75,17 @@ Backspin is a Ruby gem for characterization testing of command-line interfaces.
   - Credential scrubbing logic
   - Configuration management
 
-- **Command Class** (`lib/backspin/command.rb`)
-  - Represents a single CLI execution
-  - Stores: args, stdout, stderr, status, recorded_at, etc
+- **Snapshot Class** (`lib/backspin/snapshot.rb`)
+  - Represents a single execution snapshot
+  - Stores: command type, args, env, stdout, stderr, status, recorded_at
+
+- **BackspinResult Class** (`lib/backspin/backspin_result.rb`)
+  - Return object from `Backspin.run` / `Backspin.capture`
+  - Exposes `actual` and `expected` snapshots plus verify details
 
 - **Record Class** (`lib/backspin/record.rb`)
   - Manages YAML record files
-  - Handles recording/playback sequencing
+  - Handles record/verify sequencing
 
 ### Common Development Tasks
 
@@ -134,7 +138,7 @@ RSpec.describe "Feature name" do
   it "does something specific" do
     result = Backspin.run(["echo", "hello"], name: "my_test_record")
 
-    expect(result.stdout).to eq("hello\n")
+    expect(result.actual.stdout).to eq("hello\n")
   end
 end
 ```
@@ -212,4 +216,4 @@ If you have questions about contributing, feel free to:
 - Check existing issues and pull requests
 - Review the test suite for examples
 
-Thank you for contributing to Backspin!
+Thank you for contributing to Backspin!

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    backspin (0.8.0)
+    backspin (0.9.0)
 
 GEM
   remote: https://rubygems.org/

data/README.md

@@ -134,7 +134,7 @@ For more matcher examples and detailed documentation, see [MATCHERS.md](MATCHERS
 
 ### Working with the Result Object
 
-The API returns a `RecordResult` object with helpful methods:
+The API returns a `Backspin::BackspinResult` object with helpful methods:
 
 ```ruby
 result = Backspin.run(["sh", "-c", "echo out; echo err >&2; exit 42"], name: "my_test")
@@ -143,11 +143,12 @@ result = Backspin.run(["sh", "-c", "echo out; echo err >&2; exit 42"], name: "my
 result.recorded?  # true on first run
 result.verified?  # true/false on subsequent runs, nil when recording
 
-# Access output (first command)
-result.stdout     # "out\n"
-result.stderr     # "err\n"
-result.status     # 42
-result.success?   # false (non-zero exit)
+# Access output snapshots
+result.actual.stdout   # "out\n"
+result.actual.stderr   # "err\n"
+result.actual.status   # 42
+result.expected        # nil in :record mode, populated in :verify mode
+result.success?        # false (non-zero exit)
 result.output     # [stdout, stderr, status] for command runs
 
 # Debug information

data/docs/backspin-result-api-sketch.md

@@ -0,0 +1,203 @@
+# Backspin Result API Sketch
+
+Date: 2026-02-11  
+Branch: `spike-backspin-result-api`
+
+## Goals
+
+- Keep the public API small and predictable.
+- Make runtime output and baseline output explicit.
+- Remove multi-command semantics from the result object.
+- Keep command run and block capture under one consistent return type.
+
+## Public API
+
+### Entry points
+
+```ruby
+Backspin.run(command = nil, name:, env: nil, mode: :auto, matcher: nil, filter: nil, &block)
+Backspin.capture(name, mode: :auto, matcher: nil, filter: nil, &block)
+```
+
+Both return `BackspinResult`.
+
+### `BackspinResult`
+
+Top-level aggregate with one responsibility: represent this run and its comparison.
+
+```ruby
+class BackspinResult
+  attr_reader :mode, :record_path, :actual, :expected
+
+  def recorded?; end
+  def verified?; end
+  def diff; end
+  def error_message; end
+  def success?; end
+  def failure?; end
+end
+```
+
+Rules:
+
+- `actual` is always present and represents what just ran.
+- `expected` is baseline snapshot when one exists.
+- In `:record` mode, `expected` is `nil`, `verified?` is `nil`.
+- In `:verify` mode, `expected` is present, `verified?` is boolean.
+
+### `Snapshot`
+
+Value object for one recorded/captured execution.
+
+```ruby
+class Snapshot
+  attr_reader :command_type, :args, :env, :stdout, :stderr, :status, :recorded_at
+
+  def success?; end
+  def failure?; end
+end
+```
+
+Notes:
+
+- `command_type` is `Open3::Capture3` for command runs.
+- `command_type` is `Backspin::Capturer` for block capture.
+- Capture status remains placeholder `0`.
+
+## Usage Examples
+
+### Command verify mismatch
+
+```ruby
+result = Backspin.run(["echo", "changed"], name: "echo_case", mode: :verify)
+
+result.actual.stdout      # "changed\n"
+result.expected.stdout    # "original\n"
+result.verified?          # false
+result.diff               # unified-ish stdout/stderr/status diff
+```
+
+### First record
+
+```ruby
+result = Backspin.run(["echo", "hello"], name: "hello_case")
+
+result.mode               # :record
+result.actual.stdout      # "hello\n"
+result.expected           # nil
+result.verified?          # nil
+```
+
+### Capture verify
+
+```ruby
+result = Backspin.capture("capture_case", mode: :verify) do
+  puts "runtime output"
+end
+
+result.actual.stdout
+result.expected.stdout
+```
+
+### Unix CLI examples
+
+```ruby
+# 1) Record + verify a simple command
+Backspin.run(["echo", "hello"], name: "echo_hello")
+result = Backspin.run(["echo", "hello"], name: "echo_hello")
+result.verified?          # true
+result.actual.stdout      # "hello\n"
+result.expected.stdout    # "hello\n"
+
+# 2) Verify mismatch with a common command
+Backspin.run(["date", "+%Y-%m-%d"], name: "today", mode: :record)
+result = Backspin.run(["date", "+%Y-%m-%d"], name: "today", mode: :verify)
+result.verified?          # true/false depending on day change
+
+# 3) Capture a small shell pipeline output
+result = Backspin.capture("grep_wc") do
+  system("printf 'alpha\\nbeta\\nalpha\\n' | grep alpha | wc -l")
+end
+result.actual.stdout
+
+# 4) Verify a directory listing snapshot
+Backspin.run(["ls", "-1"], name: "project_listing", mode: :record)
+result = Backspin.run(["ls", "-1"], name: "project_listing", mode: :verify)
+result.actual.stdout
+result.expected.stdout
+```
+
+## Matcher and Filter Semantics
+
+- `matcher:` applies only during verify and compares `expected` vs `actual`.
+- `filter:` applies only when writing snapshots to disk.
+- Default match still compares stdout/stderr/status only.
+
+## Error Semantics
+
+- `Backspin::VerificationError` still raised by default when verification fails.
+- Error message is generated from `BackspinResult#error_message`.
+- Do not duplicate `diff` content in exception formatting.
+
+## Record Format Sketch (v4)
+
+Single-snapshot format to match single-snapshot runtime model:
+
+```yaml
+---
+format_version: "4.0"
+recorded_at: "2026-02-11T00:00:00Z"
+snapshot:
+  command_type: "Open3::Capture3"
+  args: ["echo", "hello"]
+  env:
+    MY_VAR: value
+  stdout: "hello\n"
+  stderr: ""
+  status: 0
+```
+
+For capture snapshots:
+
+```yaml
+snapshot:
+  command_type: "Backspin::Capturer"
+  args: ["<captured block>"]
+  stdout: "..."
+  stderr: "..."
+  status: 0
+```
+
+## Implemented Simplifications
+
+- Unified all run/capture return values under `BackspinResult`.
+- Introduced `Snapshot` as the shared value object for `actual` and `expected`.
+- Removed multi-command result semantics from the public return API.
+- Kept `CommandDiff`, now operating directly on snapshots.
+- Simplified persistence to one snapshot per record file.
+
+## Current Status
+
+Status date: 2026-02-11
+
+1. `Snapshot` and `BackspinResult` classes are implemented and wired into runtime paths.
+2. `Backspin.run` and `Backspin.capture` now return `BackspinResult`.
+3. `Record` persistence moved to v4 single-snapshot format (`snapshot` key, no `commands` array).
+4. `Matcher` and `CommandDiff` now operate on expected/actual snapshots.
+5. Legacy result/command layering was removed from `lib/`.
+6. Specs have been migrated to the new result contract and v4 format.
+7. Validation is green: `66 examples, 0 failures` and Standard lint passes.
+8. Public docs now use `result.actual` / `result.expected` terminology.
+
+## Success Criteria
+
+1. `Backspin.run` and `Backspin.capture` always return `BackspinResult` with `actual` populated.
+2. In `:record` mode, `result.expected` is `nil` and `result.verified?` is `nil`.
+3. In `:verify` mode, `result.expected` is present, `result.verified?` is boolean, and mismatch cases populate `result.diff` plus `result.error_message`.
+4. No multi-command result API remains in the public result contract.
+5. Snapshot object exposes a stable single-command shape: `stdout`, `stderr`, `status`, `args`, `env`, `command_type`.
+6. Record format uses one snapshot (v4), not a commands array.
+7. Existing strict verification behavior remains: default raises `Backspin::VerificationError`, while `raise_on_verification_failure = false` returns a failed result without raising.
+8. End-to-end Unix command examples are covered in specs: `echo` record/verify, `ls -1` record/verify, `date` mismatch behavior (or matcher override), and captured `grep | wc` pipeline output via `Backspin.capture`.
+9. Matcher behavior is preserved: default matching remains stdout/stderr/status, and custom `matcher:` contract (Proc, hash fields, `:all`) continues to work for both run and capture verification.
+10. Credential scrubbing behavior is preserved: stdout/stderr/args/env are scrubbed on persistence, capture output is scrubbed, custom patterns still apply, and verification diffs/error messages do not re-expose scrubbed secrets.

data/lib/backspin/backspin_result.rb

@@ -0,0 +1,66 @@
+# frozen_string_literal: true
+
+module Backspin
+  # Aggregate result returned by Backspin.run/capture.
+  class BackspinResult
+    attr_reader :mode, :record_path, :actual, :expected, :output
+
+    def initialize(mode:, record_path:, actual:, expected: nil, verified: nil, command_diff: nil, output: nil)
+      @mode = mode
+      @record_path = record_path
+      @actual = actual
+      @expected = expected
+      @verified = verified
+      @command_diff = command_diff
+      @output = output
+    end
+
+    def recorded?
+      mode == :record
+    end
+
+    # true/false for verify mode, nil for record mode
+    def verified?
+      return nil if mode == :record
+
+      @verified
+    end
+
+    def diff
+      return nil unless verified? == false
+
+      @command_diff&.diff
+    end
+
+    def error_message
+      return nil unless verified? == false
+      return "Output verification failed" unless @command_diff
+
+      msg = "Output verification failed:\n\n"
+      msg += @command_diff.summary
+      msg += "\n#{@command_diff.diff}" if @command_diff.diff
+      msg
+    end
+
+    def success?
+      actual&.success? || false
+    end
+
+    def failure?
+      !success?
+    end
+
+    def to_h
+      hash = {
+        mode: mode,
+        record_path: record_path,
+        actual: actual&.to_h
+      }
+
+      hash[:expected] = expected&.to_h
+      hash[:verified] = verified? unless verified?.nil?
+      hash[:diff] = diff if diff
+      hash
+    end
+  end
+end

data/lib/backspin/command_diff.rb

@@ -1,24 +1,23 @@
 # frozen_string_literal: true
 
 module Backspin
-  # Represents the difference between a recorded command and actual execution
-  # Handles verification and diff generation for a single command
+  # Represents the difference between expected and actual snapshots.
   class CommandDiff
-    attr_reader :recorded_command, :actual_command, :matcher
+    attr_reader :expected, :actual, :matcher
 
-    def initialize(recorded_command:, actual_command:, matcher: nil)
-      @recorded_command = recorded_command
-      @actual_command = actual_command
+    def initialize(expected:, actual:, matcher: nil)
+      @expected = expected
+      @actual = actual
       @matcher = Matcher.new(
         config: matcher,
-        recorded_command: recorded_command,
-        actual_command: actual_command
+        expected: expected,
+        actual: actual
       )
     end
 
-    # @return [Boolean] true if the command output matches
+    # @return [Boolean] true if the snapshot output matches.
     def verified?
-      return false unless method_classes_match?
+      return false unless command_types_match?
 
       @matcher.match?
     end
@@ -28,23 +27,23 @@ module Backspin
       return nil if verified?
 
       parts = []
-      recorded_hash = recorded_command.to_h
-      actual_hash = actual_command.to_h
+      expected_hash = expected.to_h
+      actual_hash = actual.to_h
 
-      unless method_classes_match?
-        parts << "Command type mismatch: expected #{recorded_command.method_class.name}, got #{actual_command.method_class.name}"
+      unless command_types_match?
+        parts << "Command type mismatch: expected #{expected.command_type.name}, got #{actual.command_type.name}"
       end
 
-      if recorded_hash["stdout"] != actual_hash["stdout"]
-        parts << stdout_diff(recorded_hash["stdout"], actual_hash["stdout"])
+      if expected_hash["stdout"] != actual_hash["stdout"]
+        parts << stdout_diff(expected_hash["stdout"], actual_hash["stdout"])
       end
 
-      if recorded_hash["stderr"] != actual_hash["stderr"]
-        parts << stderr_diff(recorded_hash["stderr"], actual_hash["stderr"])
+      if expected_hash["stderr"] != actual_hash["stderr"]
+        parts << stderr_diff(expected_hash["stderr"], actual_hash["stderr"])
       end
 
-      if recorded_hash["status"] != actual_hash["status"]
-        parts << "Exit status: expected #{recorded_hash["status"]}, got #{actual_hash["status"]}"
+      if expected_hash["status"] != actual_hash["status"]
+        parts << "Exit status: expected #{expected_hash["status"]}, got #{actual_hash["status"]}"
       end
 
       parts.join("\n\n")
@@ -61,12 +60,12 @@ module Backspin
 
     private
 
-    def method_classes_match?
-      recorded_command.method_class == actual_command.method_class
+    def command_types_match?
+      expected.command_type == actual.command_type
     end
 
     def failure_reason
-      unless method_classes_match?
+      unless command_types_match?
         return "command type mismatch"
       end
 

data/lib/backspin/matcher.rb

@@ -1,23 +1,22 @@
 # frozen_string_literal: true
 
 module Backspin
-  # Handles matching logic between recorded and actual commands
+  # Handles matching logic between expected and actual snapshots.
   class Matcher
-    attr_reader :config, :recorded_command, :actual_command
+    attr_reader :config, :expected, :actual
 
-    def initialize(config:, recorded_command:, actual_command:)
+    def initialize(config:, expected:, actual:)
       @config = normalize_config(config)
-      @recorded_command = recorded_command
-      @actual_command = actual_command
+      @expected = expected
+      @actual = actual
     end
 
-    # @return [Boolean] true if commands match according to the configured matcher
+    # @return [Boolean] true if snapshots match according to configured matcher
     def match?
       if config.nil?
-        # Default behavior: check all fields for equality
-        default_matcher.call(recorded_command.to_h, actual_command.to_h)
+        default_matcher.call(expected.to_h, actual.to_h)
       elsif config.is_a?(Proc)
-        config.call(recorded_command.to_h, actual_command.to_h)
+        config.call(expected.to_h, actual.to_h)
       elsif config.is_a?(Hash)
         verify_with_hash_matcher
       else
@@ -30,24 +29,22 @@ module Backspin
       reasons = []
 
       if config.nil?
-        # Default matcher checks all fields
-        recorded_hash = recorded_command.to_h
-        actual_hash = actual_command.to_h
+        expected_hash = expected.to_h
+        actual_hash = actual.to_h
 
-        reasons << "stdout differs" if recorded_hash["stdout"] != actual_hash["stdout"]
-        reasons << "stderr differs" if recorded_hash["stderr"] != actual_hash["stderr"]
-        reasons << "exit status differs" if recorded_hash["status"] != actual_hash["status"]
+        reasons << "stdout differs" if expected_hash["stdout"] != actual_hash["stdout"]
+        reasons << "stderr differs" if expected_hash["stderr"] != actual_hash["stderr"]
+        reasons << "exit status differs" if expected_hash["status"] != actual_hash["status"]
       elsif config.is_a?(Hash)
-        recorded_hash = recorded_command.to_h
-        actual_hash = actual_command.to_h
+        expected_hash = expected.to_h
+        actual_hash = actual.to_h
 
-        # Only check matchers that were provided
         config.each do |field, matcher_proc|
           case field
           when :all
-            reasons << ":all matcher failed" unless matcher_proc.call(recorded_hash, actual_hash)
+            reasons << ":all matcher failed" unless matcher_proc.call(expected_hash, actual_hash)
           when :stdout, :stderr, :status
-            unless matcher_proc.call(recorded_hash[field.to_s], actual_hash[field.to_s])
+            unless matcher_proc.call(expected_hash[field.to_s], actual_hash[field.to_s])
               reasons << "#{field} custom matcher failed"
             end
           end
@@ -79,19 +76,16 @@ module Backspin
     end
 
     def verify_with_hash_matcher
-      recorded_hash = recorded_command.to_h
-      actual_hash = actual_command.to_h
+      expected_hash = expected.to_h
+      actual_hash = actual.to_h
 
-      # Override-based: only run matchers that are explicitly provided
-      # Use map to ensure all matchers run, then check if all passed
       results = config.map do |field, matcher_proc|
         case field
         when :all
-          matcher_proc.call(recorded_hash, actual_hash)
+          matcher_proc.call(expected_hash, actual_hash)
         when :stdout, :stderr, :status
-          matcher_proc.call(recorded_hash[field.to_s], actual_hash[field.to_s])
+          matcher_proc.call(expected_hash[field.to_s], actual_hash[field.to_s])
         else
-          # This should never happen due to normalize_config validation
           raise ArgumentError, "Unknown field: #{field}"
         end
       end