mock-suey 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b136abffc56541079a6df6b63167f4cd465283fe16e096749d18f5656adb4174
-  data.tar.gz: 9d76f5c3c086350b3c78750ecb2f0aabba698dc9899c4514ffb529e0e2d23be4
+  metadata.gz: 4631d070cd8722c45646c5a06453dd9eb80f34cd18dde0e0d6a5bfe0f085299f
+  data.tar.gz: 0a353b089256d5641fdfa426b4fd5922a06a6ca05e44fb196dbece77e9f58b62
 SHA512:
-  metadata.gz: 20e94c8af89c30a2a21038b2979063355bd1b6378bde3c66e6d6f0b9ae44d812b9c46d37a690dc98b8ac4ff942cf6891b7f30fd8520bab9de9beb62fbc735340
-  data.tar.gz: f756ba2ecdb919c253e7e482f54b1b7cfee0fb00098d870d2e8a4233725bf03500580c0d5f730b3e8ff6ed4c69347459f328b8bae1ade17f9b6fd8d1000a8512
+  metadata.gz: f676d7ebbdab9bc1874e2cda0633ba566868a8bcfc4c8319dda3bb662331a05539790e6324b105715893feaf9b071f37aa1cc27e12484f4eeea7dd0138b93efc
+  data.tar.gz: c7840271c1a2588100496034b06e4b0b5d1b5d62f6b36fc0813113aa8f0232a3c475ee5aa52cd1ebaad67123d36bb39ed64cfc13e5ce4805b0d9f8ae8ce824b1

data/CHANGELOG.md

@@ -2,4 +2,8 @@
 
 ## master
 
+## 0.1.0 (2022-11-25) 🎈
+
+- Initial release.
+
 [@palkan]: https://github.com/palkan

data/README.md

@@ -2,12 +2,27 @@
 
 # Mock Suey
 
-_Coming soon_
+<img align="right" height="168" width="120"
+     title="Mock Suey logo" src="./assets/logo.png">
 
-Utilities to keep mocks in line with real objects.
+A collection of tools to keep mocks in line with real objects.
 
 > Based on the RubyConf 2022 talk ["Weaving and seaming mocks"][the-talk]
 
+## Table of contents
+
+- [Installation](#installation)
+- [Typed doubles](#typed-doubles)
+  - [Using with RBS](#using-with-rbs)
+  - [Typed doubles limitations](#typed-doubles-limitations)
+- [Mock context](#mock-context)
+- [Auto-generated type signatures and post-run checks](#auto-generated-type-signatures-and-post-run-checks)
+- [Mock contracts verification](#mock-contracts-verification)
+- [Tracking stubbed method calls](#tracking-stubbed-method-calls)
+- [Tracking real method calls](#tracking-real-method-calls)
+- [Configuration](#configuration)
+- [Future development](#future-development)
+
 ## Installation
 
 ```ruby
@@ -17,9 +32,328 @@ group :test do
 end
 ```
 
-## Usage
+Then, drop `"require 'mock-suey'` to your `spec_helper.rb` / `rails_helper.rb` / `whatever_helper.rb`.
+
+## Typed doubles
+
+MockSuey enhances verified doubles by adding type-checking support: every mocked method call is checked against the corresponding method signature (if present), and an exception is raised if types mismatch.
+
+Consider an example:
+
+```ruby
+let(:array_double) { instance_double("Array") }
+
+specify "#take" do
+  allow(array_double).to receve(:take).and_return([1, 2, 3])
+
+  expect(array_double.take("three")).to eq([1, 2, 3])
+end
+```
+
+This test passes with plain RSpec, because from the verified double perspective everything is valid. However, calling `[].take("string")` raises a TypeError in runtime.
+
+With MockSuey and [RBS][rbs], we can make verified doubles stricter and ensure that the types we use in method stubs are correct.
+
+To enable typed verified doubles, you must explicitly configure a type checker.
+
+### Using with RBS
+
+To use MockSuey with RBS, configure it as follows:
+
+```ruby
+MockSuey.configure do |config|
+  config.type_check = :ruby
+  # Optional: specify signature directries to use ("sig" is used by default)
+  # config.signature_load_dirs = ["sig"]
+  # Optional: specify whether to raise an exception if no signature found
+  # config.raise_on_missing_types = false
+end
+```
+
+Make sure that `rbs` gem is present in the bundle (MockSuey doesn't require it as a runtime dependency).
+
+That's it! Now all mocked methods are type-checked.
+
+### Typed doubles limitations
+
+Typed doubles rely on the type signatures being defined. What if you don't have types (or don't want to add them)? There are two options:
+
+1) Adding type signatures only for the objects being mocked. You don't even need to type check your code or cover it with types. Instead, you can rely on runtime checks made in tests for real objects and use typed doubles for mocked objects.
+
+2) Auto-generating types on-the-fly from the real call traces (see below).
+
+## Mock context
+
+Mock context is a re-usable mocking/stubbing configuration. Keeping a _library of mocks_
+helps to keep fake objects under control. The idea is similar to data fixtures (and heavily inspired by the [fixturama][] gem).
+
+Technically, mock contexts are _shared contexts_ (in RSpec sense) that _know_ which objects and methods are being mocked at the boot time, not at the run time. We use this knowledge to collect calls made on _real_ objects (so we can use them for the mocked calls verification later).
+
+### Defining and including mock contexts
+
+The API is similar to shared contexts:
+
+```ruby
+# Define a context
+RSpec.mock_context "Anyway::Env" do
+  let(:testo_env) do
+    {
+      "a" => "x",
+      "data" => {
+        "key" => "value"
+      }
+    }
+  end
+
+  before do
+    env_double = instance_double("Anyway::Env")
+    allow(::Anyway::Env).to receive(:new).and_return(env_double)
+
+    allow(env_double).to receive(:fetch).with("UNKNOWN", any_args).and_return(Anyway::Env::Parsed.new({}, nil))
+    allow(env_double).to receive(:fetch).with("TESTO", any_args).and_return(Anyway::Env::Parsed.new(testo_env, nil))
+    allow(env_double).to receive(:fetch).with("", any_args).and_return(nil)
+  end
+end
+
+# Include in a test file
+describe Anyway::Loaders::Env do
+  include_mock_context "Anyway::Env"
+
+  # ...
+end
+```
+
+It's recommended to keep mock contexts under `spec/mocks` or `spec/fixtures/mocks` and load them in the RSpec configuration file:
+
+```ruby
+Dir["#{__dir__}/mocks/**/*.rb"].sort.each { |f| require f }
+```
+
+### Accessing mocked objects information
+
+You can get the registry of mocked objects and methods after all tests were loaded,
+for example, in the `before(:suite)` hook:
+
+```ruby
+RSpec.configure do |config|
+  config.before(:suite) do
+    MockSuey::RSpec::MockContext.registry.each do |klass, methods|
+      methods.each do |method_name, stub_calls|
+        # Stub calls is a method call object,
+        # containing the information about the stubbed call:
+        # - receiver_class == klass
+        # - method_name == method_name
+        # - arguments: expected arguments (empty if expects no arguments)
+        # - return_value: stubbed return value
+      end
+    end
+  end
+end
+```
+
+## Auto-generated type signatures and post-run checks
+
+We can combine typed doubles and mock contexts to provide type-checking capabilities to codebase not using type signatures. For that, we can generate type signatures automatically by tracing _real_ object calls.
+
+You must opt-in to use this feature:
+
+```ruby
+MockSuey.configure do |config|
+  # Make sure type checker is configured
+  config.type_check = :ruby
+  config.auto_type_check = true
+  # Choose the real objects tracing method
+  config.trace_real_calls_via = :prepend # or :trace_point
+  # Whether to raise if type is missing in a post-check
+  # (i.e., hasn't been generated)
+  config.raise_on_missing_auto_types = true
+end
+```
+
+Under the hood, we use the [Tracking real method calls](#tracking-real-method-calls) feature described below.
+
+**IMPORTANT**: Only objects declared within mock contexts could be type-checked.
+
+### Limitations
+
+Currently, this feature only works if both real objects and mock objects calls are made during the same test run. Thus, tests could fail when running tests in parallel.
+
+## Mock contracts verification
+
+Types drastically increase mocks/stubs stability (or consistency), but even they do not guarantee that mocks behave the same way as real objects. For example, if your method returns completely different results depending on the values (not types) of the input.
+
+The only way to provide ~100% confidence to mocks is enforcing a contract. One way to enforce mock contracts is to require having a unit/functional tests where a real object receives the same input and returns the same result as the mock. For example, consider the following tests:
+
+```ruby
+describe Accountant do
+  let(:calculator) { instance_double("TaxCalculator") }
+
+  # Declaring a mock == declaring a contract (input/output correspondance)
+  before do
+    allow(calculator).to receive(:tax_for_income).with(2020).and_return(202)
+    allow(calculator).to receive(:tax_for_income).with(0).and_return(0)
+  end
+
+  subject { described_class.new(calculator) }
+
+  specify "#after_taxes" do
+    # Assuming the #after_taxes method calls calculator.tax_for_income
+    expect(subject.after_taxes(2020)).to eq(1818)
+    expect(subject.after_taxes(0)).to be_nil
+  end
+end
+
+describe TaxCalculator do
+  subject { described_class.new }
+
+  # Adding a unit-test using the same input
+  # verifies the contract
+  specify "#tax_for_income" do
+    expect(subject.tax_for_income(2020)).to eq(202)
+    expect(subject.tax_for_income(0)).to eq(0)
+  end
+end
+```
+
+We need a way to enforce mock contract verification. In other words, if the dependency behaviour changes and the corresponding unit-test reflects this change, our mock should be marked as invalid and result into a test suit failure.
+
+One way to do this is to introduce explicit contract verification (via custom mocking mechanisms or DSL or whatever, see [bogus][] or [compact][], for example).
+
+Mock Suey chooses another way: automatically infer mock contracts (via mock contexts) and verify them by collecting real object calls during the test run. You can enable this feature via the following configuration options:
+
+```ruby
+MockSuey.configure do |config|
+  config.verify_mock_contracts = true
+  # Choose the real objects tracing method
+  config.trace_real_calls_via = :prepend # or :trace_point
+end
+```
+
+Each method stub represents a contract. For example:
+
+```ruby
+allow(calculator).to receive(:tax_for_income).with(2020).and_return(202)
+allow(calculator).to receive(:tax_for_income).with(0).and_return(0)
+
+#=> TaxCalculator#tax_for_income: (2020) -> Integer
+#=> TaxCalculator#tax_for_income: (0) -> Integer
+```
+
+If the method behaviours changes, running tests would result in a failure if mock doesn't reflect the change:
+
+```ruby
+# Assuming we decided to return nil for non-positive integers
+specify "#tax_for_income" do
+  expect(subject.tax_for_income(0)).to be_nil
+end
+```
+
+The test suite will fail with the following exception:
+
+```sh
+$ rspec accountant_spec.rb
+
+........
+
+1) Mock contract verification failed:
+   No matching call found for:
+     TaxCalculator#tax_for_income: (0) -> Integer
+   Captured calls:
+     (0) -> NilClass
+```
+
+The contract describes which explicit input values result in a particular output type (not value). Such verification can help to verify boundary conditions (e.g., when some inputs result in nil results or exceptions).
+
+### Limitations
+
+1. Currently, verification takes into account only primitive values (String, Number, Booleans, plain Ruby hashes and arrays, etc.). Custom classes are not yet supported.
+
+2. Similarly to auto-type checks, this feature does not yet support parallel tests execution.
+
+## Tracking stubbed method calls
+
+The core functionality of this gem is the ability to hook into mocked method invocations to perform custom checks.
+
+**NOTE:** Currently, only RSpec is supported.
+
+You can add an after mock call callback as follows:
+
+```ruby
+MockSuey.on_mocked_call do |call_obj|
+  # Do whatever you want with the method call object,
+  # containing the following fields:
+  # - receiver_class
+  # - method_name
+  # - arguments
+  # - return_value
+end
+```
+
+By default, MockSuey doesn't keep mocked calls, but if you want to analyze them
+at the end of the test suite run, you can configure MockSuey to keep all the calls and
+access them later:
+
+```ruby
+MockSuey.configure do |config|
+  config.store_mocked_calls = true
+end
+
+# Them, you can access them in the after(:suite) hook, for example
+RSpec.configure do |config|
+  config.after(:suite) do
+    p MockSuey.stored_mocked_calls
+  end
+end
+```
+
+## Tracking real method calls
+
+This gem provides a method call tracking functionality for non-double objects.
+You can use it separately (not as a part of auto-type-checking or contract verification features). For example:
+
+```ruby
+RSpec.configure do |config|
+  tracer = MockSuey::Tracer.new(via: :trace_point) # or via: :prepend
+
+  config.before(:suite) do
+    tracer.collect(SomeClass, %i[some_method another_method])
+    tracer.start!
+  end
+
+  config.after(:suite) do
+    calls = traces.stop
+    # where call is a call object
+    # similar to a mocked call object described above
+  end
+end
+```
+
+## Configuration
+
+Additional configuration options could be set:
+
+```ruby
+MockSuey.configure do |config|
+  # A logger instance
+  config.logger = Logger.new($stdout)
+  # You can also specify log level and whether to colorize logs
+  # config.log_level = :info
+  # config.color = ? # Depends on the logging device
+  # Debug mode is a shortcut to setup an STDOUT logger with the debug level
+  config.debug = ENV["MOCK_SUEY_DEBUG"] == "true" # or 1, y, yes, or t
+end
+```
+
+## Future development
+
+I'm interested in the following contributions/discussions:
 
-_Coming soon_
+- Figure out parallel builds
+- Sorbet support
+- Minitest support
+- Advanced mock contracts (custom rules, custom classes support, etc.)
+- Methods delegation (e.g., `.perform_asyn -> #perform`)
+- Exceptions support in contracts verification
 
 ## Contributing
 
@@ -34,3 +368,7 @@ This gem is generated via [new-gem-generator](https://github.com/palkan/new-gem-
 The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
 
 [the-talk]: https://evilmartians.com/events/weaving-and-seaming-mocks
+[rbs]: https://github.com/ruby/rbs
+[fixturama]: https://github.com/nepalez/fixturama
+[bogus]: https://github.com/psyho/bogus
+[compact]: https://github.com/robwold/compact

data/lib/.rbnext/3.0/mock_suey/core.rb

@@ -0,0 +1,246 @@
+# frozen_string_literal: true
+
+module MockSuey
+  class Configuration
+    # No freezing this const to allow third-party libraries
+    # to integrate with mock_suey
+    TYPE_CHECKERS = %w[ruby]
+
+    attr_accessor :debug,
+      :logger,
+      :log_level,
+      :color,
+      :store_mocked_calls,
+      :signature_load_dirs,
+      :raise_on_missing_types,
+      :raise_on_missing_auto_types,
+      :trace_real_calls,
+      :trace_real_calls_via
+
+    attr_reader :type_check, :auto_type_check, :verify_mock_contracts
+
+    def initialize
+      @debug = %w[1 y yes true t].include?(ENV["MOCK_SUEY_DEBUG"])
+      @log_level = debug ? :debug : :info
+      @color = nil
+      @store_mocked_calls = false
+      @type_check = nil
+      @signature_load_dirs = ["sig"]
+      @raise_on_missing_types = false
+      @raise_on_missing_auto_types = true
+      @trace_real_calls = false
+      @auto_type_check = false
+      @trace_real_calls_via = :prepend
+    end
+
+    def color?
+      return color unless color.nil?
+
+      logdev = logger.instance_variable_get(:@logdev)
+      return self.color = false unless logdev
+
+      output = logdev.instance_variable_get(:@dev)
+      return self.color = false unless output
+
+      self.color = output.is_a?(IO) && output.tty?
+    end
+
+    def type_check=(val)
+      if val.nil?
+        @type_check = nil
+        return
+      end
+
+      val = val.to_s
+      raise ArgumentError, "Unsupported type checker: #{val}. Supported: #{TYPE_CHECKERS.join(",")}" unless TYPE_CHECKERS.include?(val)
+
+      @type_check = val
+    end
+
+    def auto_type_check=(val)
+      if val
+        @trace_real_calls = true
+        @store_mocked_calls = true
+        @auto_type_check = true
+      else
+        @auto_type_check = val
+      end
+    end
+
+    def verify_mock_contracts=(val)
+      if val
+        @trace_real_calls = true
+        @verify_mock_contracts = true
+      else
+        @verify_mock_contracts = val
+      end
+    end
+  end
+
+  class << self
+    attr_reader :stored_mocked_calls, :tracer, :stored_real_calls
+    attr_accessor :type_checker
+
+    def config ;  @config ||= Configuration.new; end
+
+    def configure ;  yield config; end
+
+    def logger ;  config.logger; end
+
+    def on_mocked_call(&block)
+      on_mocked_callbacks << block
+    end
+
+    def handle_mocked_call(call_obj)
+      on_mocked_callbacks.each { _1.call(call_obj) }
+    end
+
+    def on_mocked_callbacks
+      @on_mocked_callbacks ||= []
+    end
+
+    # Load extensions and start tracing if required
+    def cook
+      setup_logger
+      setup_type_checker
+      setup_mocked_calls_collection if config.store_mocked_calls
+      setup_real_calls_collection if config.trace_real_calls
+    end
+
+    # Run post-suite checks
+    def eat
+      @stored_real_calls = tracer.stop if config.trace_real_calls
+
+      offenses = []
+
+      if config.store_mocked_calls
+        logger.debug { "Stored mocked calls:\n#{stored_mocked_calls.map { "  #{_1.inspect}" }.join("\n")}" }
+      end
+
+      if config.trace_real_calls
+        logger.debug { "Traced real calls:\n#{stored_real_calls.map { "  #{_1.inspect}" }.join("\n")}" }
+      end
+
+      if config.auto_type_check
+        perform_auto_type_check(offenses)
+      end
+
+      if config.verify_mock_contracts
+        perform_contracts_verification(offenses)
+      end
+
+      offenses
+    end
+
+    private
+
+    def setup_logger
+      if !config.logger || config.debug
+        config.logger = Logger.new($stdout)
+        config.logger.formatter = Logging::Formatter.new
+      end
+      config.logger.level = config.log_level
+    end
+
+    def setup_type_checker
+      return unless config.type_check
+
+      # Allow configuring type checher manually
+      unless type_checker
+        require "mock_suey/type_checks/#{config.type_check}"
+        const_name = config.type_check.split("_").map(&:capitalize).join
+
+        self.type_checker = MockSuey::TypeChecks.const_get(const_name)
+          .new(load_dirs: config.signature_load_dirs)
+
+        logger.info "Set up type checker: #{type_checker.class.name} (load_dirs: #{config.signature_load_dirs})"
+      end
+
+      raise_on_missing = config.raise_on_missing_types
+
+      on_mocked_call do |call_obj|
+        type_checker.typecheck!(call_obj, raise_on_missing: raise_on_missing)
+      end
+    end
+
+    def setup_mocked_calls_collection
+      logger.info "Collect mocked calls (MockSuey.stored_mocked_calls)"
+
+      @stored_mocked_calls = []
+
+      on_mocked_call { @stored_mocked_calls << _1 }
+    end
+
+    def setup_real_calls_collection
+      logger.info "Collect real calls via #{config.trace_real_calls_via} (MockSuey.stored_real_calls)"
+
+      @tracer = Tracer.new(via: config.trace_real_calls_via)
+
+      MockSuey::RSpec::MockContext.registry.each do |klass, methods|
+        logger.debug { "Trace #{klass} methods: #{methods.keys.join(", ")}" }
+        tracer.collect(klass, methods.keys)
+      end
+
+      tracer.start!
+    end
+
+    def perform_auto_type_check(offenses)
+      raise "No type checker configured" unless type_checker
+
+      # Generate signatures
+      type_checker.load_signatures_from_calls(stored_real_calls)
+
+      logger.info "Type-checking mocked calls against auto-generated signatures..."
+
+      was_offenses = offenses.size
+
+      # Verify stored mocked calls
+      raise_on_missing = config.raise_on_missing_auto_types
+
+      stored_mocked_calls.each do |call_obj|
+        type_checker.typecheck!(call_obj, raise_on_missing: raise_on_missing)
+      rescue RBS::Test::Tester::TypeError, TypeChecks::MissingSignature => err
+        call_obj.metadata[:error] = err
+        offenses << call_obj
+      end
+
+      failed_count = offenses.size - was_offenses
+      failed = failed_count > 0
+
+      if failed
+        logger.error "❌ Type-checking completed. Failed examples: #{failed_count}"
+      else
+        logger.info "✅ Type-checking completed. All good"
+      end
+    end
+
+    def perform_contracts_verification(offenses)
+      logger.info "Verifying mock contracts..."
+      real_calls_per_class_method = stored_real_calls.group_by(&:receiver_class).tap do |grouped|
+        grouped.transform_values! { _1.group_by(&:method_name) }
+      end
+
+      was_offenses = offenses.size
+
+      MockSuey::RSpec::MockContext.registry.each do |klass, methods|
+        methods.values.flatten.each do |stub_call|
+          contract = MockContract.from_stub(stub_call)
+          logger.debug { "Generated contract:\n  #{contract.inspect}\n    (from stub: #{stub_call.inspect})" }
+          contract.verify!(real_calls_per_class_method.dig(klass, stub_call.method_name))
+        rescue MockContract::Error => err
+          stub_call.metadata[:error] = err
+          offenses << stub_call
+        end
+      end
+
+      failed_count = offenses.size - was_offenses
+      failed = failed_count > 0
+
+      if failed
+        logger.error "❌ Verifying mock contracts completed. Failed contracts: #{failed_count}"
+      else
+        logger.info "✅ Verifying mock contracts completed. All good"
+      end
+    end
+  end
+end

data/lib/.rbnext/3.0/mock_suey/ext/instance_class.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module MockSuey
+  module Ext
+    module InstanceClass
+      refine Class do
+        def instance_class ;  self; end
+
+        def instance_class_name ;  name; end
+      end
+
+      refine Class.singleton_class do
+        def instance_class
+          # TODO: replace with const_get
+          eval(instance_class_name) # rubocop:disable Security/Eval
+        end
+
+        def instance_class_name ;  inspect.sub(%r{^#<Class:}, "").sub(/>$/, ""); end
+      end
+    end
+  end
+end

data/lib/.rbnext/3.0/mock_suey/ext/rspec.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+module MockSuey
+  module Ext
+    module RSpec
+      # Provide unified interface to access target class
+      # for different double/proxy types
+      refine ::RSpec::Mocks::TestDoubleProxy do
+        def target_class ;  nil; end
+      end
+
+      refine ::RSpec::Mocks::PartialDoubleProxy do
+        def target_class ;  object.class; end
+      end
+
+      refine ::RSpec::Mocks::VerifyingPartialDoubleProxy do
+        def target_class ;  object.class; end
+      end
+
+      refine ::RSpec::Mocks::PartialClassDoubleProxy do
+        def target_class ;  object.singleton_class; end
+      end
+
+      refine ::RSpec::Mocks::VerifyingPartialClassDoubleProxy do
+        def target_class ;  object.singleton_class; end
+      end
+
+      refine ::RSpec::Mocks::VerifyingProxy do
+        def target_class ;  @doubled_module.target; end
+      end
+
+      refine ::RSpec::Mocks::Proxy do
+        attr_reader :method_doubles
+      end
+    end
+  end
+end