openapi_first 2.7.4 → 2.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2d3e13bc24feccc8015701ca7360e77fbb8c44d8c7dad9e87310f52b04ddebe6
-  data.tar.gz: 8a5895ce7b3a5a7a2669e862f4821125538573a6e694db36e8d2687580c3e0c9
+  metadata.gz: d5a5311b4ca774b9a0cb80e85f29243f46e1c5c762241b015090834971373a7b
+  data.tar.gz: 640be1c8c02b8a86b5c9d562b7ac837b73881473c54cf9ea731779352e0cee2d
 SHA512:
-  metadata.gz: 3420becb820a470942b12e39d7e59197f9a50696a0f63c2af601ce26cad4e95c6b6e9d915e5b3f4528da78bf1f012993af9401d1863578e8e64c2059896c71bb
-  data.tar.gz: fe61cf2f437b9a725b377b7dcc5917ac130639ca67514450bee3ba919f289f083fa232b9a94770d175232da2546b5b999c5e544abe20d0dba060adc6d4eb6f07
+  metadata.gz: 42ab4483c694d921c68fcb30099ecd779e789c5bfc9163f97a70327f5ef1aacdc7b7abae66c70341e2edd4ff5821d4fe6e9ab3e0d65b081963685272d094e5e2
+  data.tar.gz: 7bbc02b4e061a012663b115758b1f43a6fd4642687d07711d90e7032a800fedb453e3b32a762658bd5185d9cea0f8859b2436d28f4bea12f0bebd4bb613aeeff

data/CHANGELOG.md

@@ -2,6 +2,17 @@
 
 ## Unreleased
 
+## 2.8.0
+
+### OpenapiFirst::Test is now stricter and more configurable
+
+Changes:
+- Changed OpenapiFirst::Test to raises an "invalid response" error if it sees an invalid response (https://github.com/ahx/openapi_first/issues/366).
+  You can change this back to the old behavior by setting `OpenapiFirst::Test::Configuration#response_raise_error = false` (but you shouldn't).
+- Added `Test.setup { it.observe(MyApp) }`, `Test.observe(App, api: :my_api)` and internal `Test::Callable[]` to inject request/response validation in rack app as an alternative to overwrite the `app` method in a test
+- Added `Test::Configuration#ignored_unknown_status` to configure response status(es) that do not have to be descriped in the API description. 404 statuses are ignored by default.
+- Changed `OpenapiFirst::Test` to make tests fail if API description is not covered by tests. You can adapt this behavior via `OpenapiFirst::Test.setup` / `skip_response_coverage` or deactivate coverage with `OpenapiFirst::Test::Configuration#report_coverage = false` or `report_coverage = :warn`
+
 ## 2.7.4
 
 - Return 400 if Rack cannot parse query string instead of raising an exception. Fixes https://github.com/ahx/openapi_first/issues/372

data/README.md

@@ -1,8 +1,24 @@
 # openapi_first
 
-openapi_first is a Ruby gem for request / response validation and contract-testing against an [OpenAPI](https://www.openapis.org/) 3.0 or 3.1 API description. It makes an APIFirst workflow easy and reliable.
+openapi_first is a Ruby gem for request / response validation and contract-testing against an [OpenAPI](https://www.openapis.org/) 3.0 or 3.1 Openapi API description (OAD). It makes an APIFirst workflow easy and reliable.
 
-You can use openapi_first on production for [request validation](#request-validation) and in your [tests](#contract-testing) to avoid API drift with it's request/response validation and coverage features.
+## Usage
+
+Use an OAD to validate incoming requests in production:
+```ruby
+use OpenapiFirst::Middlewares::RequestValidation, 'openapi/openapi.yaml'
+```
+
+Turn your request tests into contract tests against an OAD:
+```ruby
+# spec_helper.rb
+require 'openapi_first'
+OpenapiFirst::Test.setup do |config|
+  config.register('openapi/openapi.yaml')
+end
+require 'application' # Load Application code
+OpenapiFirst::Test.observe(Application)
+```
 
 ## Contents
 
@@ -35,27 +51,28 @@ Here is how to set it up:
   This should go at the top of your test helper file before loading your application code.
     ```ruby
     require 'openapi_first'
-    OpenapiFirst::Test.setup do |test|
-      test.register('openapi/openapi.yaml')
-      # Optional: Make tests fail if coverage is below minimum
-      test.minimum_coverage = 100
-      # Optional: Skip certain responses, which are described in your API description, but need no test coverage
-      test.skip_response_coverage { it.status == '500' } #
-    end
-    ```
-2. Add an `app` method to your tests by including a Module. This `app` method wraps your application with silent request / response validation. This validates all requests/responses in your test run. (✷1)
-    ```ruby
-    RSpec.configure do |config|
-      config.include OpenapiFirst::Test::Methods[MyApp], type: :request
-    end
-    ```
-    Or add the `app` method yourself:
-
-    ```ruby
-    def app
-      OpenapiFirst::Test.app(MyApp)
+    OpenapiFirst::Test.setup do |config|
+      config.register('openapi/openapi.yaml')
     end
     ```
+2. Observe your application. You can do this in one of two ways:
+    - Inject a Module to wrap (prepend) the `call` method of your Rack app Class.
+      ```ruby
+      OpenapiFirst::Test.observe(MyApplication)
+      ```
+    - Or add an `app` method to your tests, which wraps your application with silent request / response validation. (✷1)
+      ```ruby
+      RSpec.configure do |config|
+        config.include OpenapiFirst::Test::Methods[MyApp], type: :request
+      end
+      ```
+      Or add the `app` method yourself:
+
+      ```ruby
+      def app
+        OpenapiFirst::Test.app(MyApp)
+      end
+      ```
 3. Run your tests. The Coverage feature will tell you about missing or invalid requests/responses.
 
 (✷1): It does not matter what method of openapi_first you use to validate requests/responses. Instead of using `OpenapiFirstTest.app` to wrap your application, you could also use the [middlewares](#rack-middlewares) or [test assertion method](#test-assertions), but you would have to do that for all requests/responses defined in your API description to make coverage work.

data/lib/openapi_first/configuration.rb

@@ -30,6 +30,7 @@ module OpenapiFirst
     HOOKS.each do |hook|
       define_method(hook) do |&block|
         hooks[hook] << block
+        block
       end
     end
 

data/lib/openapi_first/definition.rb

@@ -86,14 +86,15 @@ module OpenapiFirst
 
       response_match = route.match_response(status: rack_response.status, content_type: rack_response.content_type)
       error = response_match.error
-      if error
-        ValidatedResponse.new(rack_response, error:)
-      else
-        response_match.response.validate(rack_response)
-      end.tap do |validated|
-        @config.hooks[:after_response_validation]&.each { |hook| hook.call(validated, rack_request, self) }
-        raise validated.error.exception(validated) if raise_error && validated.invalid?
-      end
+      validated = if error
+                    ValidatedResponse.new(rack_response, error:)
+                  else
+                    response_match.response.validate(rack_response)
+                  end
+      @config.hooks[:after_response_validation]&.each { |hook| hook.call(validated, rack_request, self) }
+      raise validated.error.exception(validated) if raise_error && validated.invalid?
+
+      validated
     end
 
     private

data/lib/openapi_first/response_body_parsers.rb

@@ -23,7 +23,9 @@ module OpenapiFirst
     register(/json/i, lambda do |body|
       JSON.parse(body)
     rescue JSON::ParserError
-      Failure.fail!(:invalid_response_body, message: 'Response body is invalid: Failed to parse response body as JSON')
+      return Failure.fail!(:invalid_response_body, message: 'JSON response body must not be empty') if body.empty?
+
+      Failure.fail!(:invalid_response_body, message: 'Failed to parse response body as JSON')
     end)
   end
 end

data/lib/openapi_first/test/callable.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module OpenapiFirst
+  module Test
+    # Return a Module with a call method that wrapps silent request/response validation to monitor a Rack app
+    # This is used by Openapi::Test.observe
+    module Callable
+      # Returns a Module with a `call(env)` method that wraps super inside silent request/response validation
+      # You can use this like `Application.prepend(OpenapiFirst::Test.app_module)` to monitor your app during testing.
+      def self.[](definition)
+        Module.new.tap do |mod|
+          mod.define_method(:call) do |env|
+            request = Rack::Request.new(env)
+
+            definition.validate_request(request, raise_error: false)
+            response = super(env)
+            status, headers, body = response
+            definition.validate_response(request, Rack::Response[status, headers, body], raise_error: false)
+            response
+          end
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test/configuration.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+module OpenapiFirst
+  module Test
+    # Helper class to setup tests
+    class Configuration
+      def initialize
+        @minimum_coverage = 100
+        @coverage_formatter = Coverage::TerminalFormatter
+        @coverage_formatter_options = {}
+        @skip_response_coverage = nil
+        @response_raise_error = true
+        @ignored_unknown_status = [404]
+        @report_coverage = true
+        @registry = {}
+        @apps = {}
+      end
+
+      # Register OADs, but don't load them just yet
+      # @param [OpenapiFirst::OAD] oad The OAD to register
+      # @param [Symbol] as The name to register the OAD under
+      def register(oad, as: :default)
+        @registry[as] = oad
+      end
+
+      # Observe a rack app
+      def observe(app, api: :default)
+        @apps[api] = app
+      end
+
+      attr_accessor :coverage_formatter_options, :coverage_formatter, :response_raise_error, :minimum_coverage
+      attr_reader :registry, :apps, :report_coverage, :ignored_unknown_status
+
+      # Configure report coverage
+      # @param [Boolean, :warn] value Whether to report coverage or just warn.
+      def report_coverage=(value)
+        allowed_values = [true, false, :warn]
+        unless allowed_values.include?(value)
+          raise ArgumentError, "'report_coverage' must be one of #{allowed_values}, but was #{value.inspect}"
+        end
+
+        @report_coverage = value
+      end
+
+      def skip_response_coverage(&block)
+        return @skip_response_coverage unless block_given?
+
+        @skip_response_coverage = block
+      end
+    end
+  end
+end

data/lib/openapi_first/test/coverage/terminal_formatter.rb

@@ -10,10 +10,14 @@ module OpenapiFirst
           @focused = focused && !verbose
         end
 
-        # This takes a list of Coverage::Plan instances and outputs a String
         def format(coverage_result)
+          coverage = coverage_result.coverage
           @out = StringIO.new
-          coverage_result.plans.each { |plan| format_plan(plan) }
+          if coverage.zero?
+            @out.puts 'API Coverage did not detect any API requests for the registered API descriptions. ' \
+                      'Make sure to observe your application using OpenapiFirst::Test.'
+          end
+          coverage_result.plans.each { |plan| format_plan(plan) } if coverage.positive?
           @out.string
         end
 
@@ -85,7 +89,9 @@ module OpenapiFirst
         def explain_unfinished_request(request)
           return 'No requests tracked!' unless request.requested?
 
-          "All requests invalid! (#{request.last_error_message.inspect})" unless request.any_valid_request?
+          return if request.any_valid_request?
+
+          "All requests invalid! (#{request.last_error_message.inspect})"
         end
 
         def response_label(response)

data/lib/openapi_first/test/coverage.rb

@@ -12,26 +12,12 @@ module OpenapiFirst
 
       Result = Data.define(:plans, :coverage)
 
+      @current_run = {}
+
       class << self
         attr_reader :current_run
 
-        def install
-          return if @installed
-
-          @after_request_validation = lambda do |validated_request, oad|
-            track_request(validated_request, oad)
-          end
-
-          @after_response_validation = lambda do |validated_response, request, oad|
-            track_response(validated_response, request, oad)
-          end
-
-          OpenapiFirst.configure do |config|
-            config.after_request_validation(&@after_request_validation)
-            config.after_response_validation(&@after_response_validation)
-          end
-          @installed = true
-        end
+        def install = Test.install
 
         def start(skip_response: nil)
           @current_run = Test.definitions.values.to_h do |oad|
@@ -40,16 +26,11 @@ module OpenapiFirst
           end
         end
 
-        def uninstall
-          configuration = OpenapiFirst.configuration
-          configuration.hooks[:after_request_validation].delete(@after_request_validation)
-          configuration.hooks[:after_response_validation].delete(@after_response_validation)
-          @installed = nil
-        end
+        def uninstall = Test.uninstall
 
         # Clear current coverage run
         def reset
-          @current_run = nil
+          @current_run = {}
         end
 
         def track_request(request, oad)
@@ -66,13 +47,13 @@ module OpenapiFirst
 
         # Returns all plans (Plan) that were registered for this run
         def plans
-          current_run&.values
+          current_run.values
         end
 
         private
 
         def coverage
-          return 0 unless plans
+          return 0 if plans.empty?
 
           plans.sum(&:coverage) / plans.length
         end

data/lib/openapi_first/test/observe.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module OpenapiFirst
+  module Test
+    class ObserveError < Error; end
+
+    # @visible private
+    module Observed; end
+
+    # Inject silent request/response validation to observe rack apps during testing
+    module Observe
+      def self.observe(app, api: :default)
+        unless app.instance_methods.include?(:call)
+          raise ObserveError, "Don't know how to observe #{app}, because it has no call instance method."
+        end
+
+        return if app.include?(Observed)
+
+        definition = OpenapiFirst::Test[api]
+        mod = OpenapiFirst::Test::Callable[definition]
+        app.prepend(mod)
+        app.include(Observed)
+      end
+    end
+  end
+end

data/lib/openapi_first/test/registry.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+module OpenapiFirst
+  module Test
+    class NotRegisteredError < Error; end
+    class AlreadyRegisteredError < Error; end
+
+    # @visibility private
+    module Registry
+      def definitions
+        @definitions ||= {}
+      end
+
+      # Register an OpenAPI definition for testing
+      # @param path_or_definition [String, Definition] Path to the OpenAPI file or a Definition object
+      # @param as [Symbol] Name to register the API definition as
+      def register(path_or_definition, as: :default)
+        if definitions.key?(as) && as == :default
+          raise(
+            AlreadyRegisteredError,
+            "#{definitions[as].filepath.inspect} is already registered " \
+            "as ':default' so you cannot register #{path_or_definition.inspect} without " \
+            'giving it a custom name. Please call register with a custom key like: ' \
+            "#{name}.register(#{path_or_definition.inspect}, as: :my_other_api)"
+          )
+        end
+
+        definition = OpenapiFirst.load(path_or_definition)
+        definitions[as] = definition
+        definition
+      end
+
+      def [](api)
+        definitions.fetch(api) do
+          option = api == :default ? '' : ", as: #{api.inspect}"
+          raise(NotRegisteredError,
+                "API description '#{api.inspect}' not found." \
+                "Please call #{name}.register('myopenapi.yaml'#{option}) " \
+                'once before running tests.')
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test.rb

@@ -1,10 +1,23 @@
 # frozen_string_literal: true
 
+require_relative 'test/configuration'
+require_relative 'test/registry'
+
 module OpenapiFirst
   # Test integration
   module Test
     autoload :Coverage, 'openapi_first/test/coverage'
     autoload :Methods, 'openapi_first/test/methods'
+    autoload :Callable, 'openapi_first/test/callable'
+    autoload :Observe, 'openapi_first/test/observe'
+    extend Registry
+
+    class CoverageError < Error; end
+
+    # Inject request/response validation in a rack app class
+    def self.observe(app, api: :default)
+      Observe.observe(app, api:)
+    end
 
     def self.minitest?(base)
       base.include?(::Minitest::Assertions)
@@ -12,58 +25,23 @@ module OpenapiFirst
       false
     end
 
-    # Helper class to setup tests
-    class Setup
-      def initialize
-        @minimum_coverage = 0
-        @coverage_formatter = Coverage::TerminalFormatter
-        @coverage_formatter_options = {}
-        @skip_response_coverage = nil
-        yield self
-      end
-
-      def register(oad, as: :default)
-        Test.register(oad, as:)
-      end
-
-      attr_accessor :minimum_coverage, :coverage_formatter_options, :coverage_formatter
-
-      def skip_response_coverage(&block)
-        return @skip_response_coverage unless block_given?
-
-        @skip_response_coverage = block
-      end
-
-      # This called at_exit
-      def handle_exit
-        coverage = Coverage.result.coverage
-        # :nocov:
-        puts 'API Coverage did not detect any API requests for the registered API descriptions' if coverage.zero?
-        if coverage.positive?
-          Test.report_coverage(
-            formatter: coverage_formatter,
-            **coverage_formatter_options
-          )
-        end
-        return unless minimum_coverage > coverage
-
-        puts "API Coverage fails with exit 2, because API coverage of #{coverage}% " \
-             "is below minimum of #{minimum_coverage}%!"
-        exit 2
-        # :nocov:
-      end
+    def self.configuration
+      @configuration ||= Configuration.new
     end
 
     # Sets up OpenAPI test coverage and OAD registration.
-    # @yieldparam [OpenapiFirst::Test::Setup] setup A setup for configuration
-    def self.setup(&)
+    # @yieldparam [OpenapiFirst::Test::Configuration] configuration A configuration to setup test integration
+    def self.setup
       unless block_given?
-        raise ArgumentError, "Please provide a block to #{self.class}.setup to register you API descriptions"
+        raise ArgumentError, "Please provide a block to #{self.class}.confgure to register you API descriptions"
       end
 
-      Coverage.install
-      setup = Setup.new(&)
-      Coverage.start(skip_response: setup.skip_response_coverage)
+      install
+      yield configuration
+
+      configuration.registry.each { |name, oad| register(oad, as: name) }
+      configuration.apps.each { |name, app| observe(app, api: name) }
+      Coverage.start(skip_response: configuration.skip_response_coverage)
 
       if definitions.empty?
         raise NotRegisteredError,
@@ -72,17 +50,37 @@ module OpenapiFirst
               "OpenapiFirst::Test.setup { |test| test.register('myopenapi.yaml') }"
       end
 
+      @exit_handler = method(:handle_exit)
+
       @setup ||= at_exit do
-        setup.handle_exit
+        # :nocov:
+        @exit_handler&.call
+        # :nocov:
       end
     end
 
+    def self.handle_exit
+      return unless configuration.report_coverage
+
+      report_coverage(
+        formatter: configuration.coverage_formatter,
+        **configuration.coverage_formatter_options
+      )
+      return unless configuration.report_coverage == true
+
+      coverage = Coverage.result.coverage
+      return if coverage >= configuration.minimum_coverage
+
+      puts "API Coverage fails with exit 2, because not all described requests and responses have been tested (#{coverage.round(4)}% covered)." # rubocop:disable Layout/LineLength
+
+      exit 2
+    end
+
     # Print the coverage report
     # @param formatter A formatter to define the report.
     # @output [IO] An output where to puts the report.
     def self.report_coverage(formatter: Coverage::TerminalFormatter, **)
-      coverage_result = Coverage.result
-      puts formatter.new(**).format(coverage_result)
+      puts formatter.new(**).format(Coverage.result)
     end
 
     # Returns the Rack app wrapped with silent request, response validation
@@ -97,42 +95,37 @@ module OpenapiFirst
       end
     end
 
-    class NotRegisteredError < StandardError; end
-    class AlreadyRegisteredError < StandardError; end
-
-    @definitions = {}
-
-    class << self
-      attr_reader :definitions
-
-      # Register an OpenAPI definition for testing
-      # @param path_or_definition [String, Definition] Path to the OpenAPI file or a Definition object
-      # @param as [Symbol] Name to register the API definition as
-      def register(path_or_definition, as: :default)
-        if definitions.key?(as) && as == :default
-          raise(
-            AlreadyRegisteredError,
-            "#{definitions[as].filepath.inspect} is already registered " \
-            "as ':default' so you cannot register #{path_or_definition.inspect} without " \
-            'giving it a custom name. Please call register with a custom key like: ' \
-            "OpenapiFirst::Test.register(#{path_or_definition.inspect}, as: :my_other_api)"
-          )
+    def self.install
+      return if @installed
+
+      OpenapiFirst.configure do |config|
+        @after_request_validation = config.after_request_validation do |validated_request, oad|
+          Coverage.track_request(validated_request, oad)
         end
 
-        definition = OpenapiFirst.load(path_or_definition)
-        definitions[as] = definition
-        definition
-      end
+        @after_response_validation = config.after_response_validation do |validated_response, rack_request, oad|
+          if validated_response.invalid? && raise_response_error?(validated_response)
+            raise validated_response.error.exception
+          end
 
-      def [](api)
-        definitions.fetch(api) do
-          option = api == :default ? '' : ", as: #{api.inspect}"
-          raise(NotRegisteredError,
-                "API description '#{api.inspect}' not found." \
-                "Please call OpenapiFirst::Test.register('myopenapi.yaml'#{option}) " \
-                'once before running tests.')
+          Coverage.track_response(validated_response, rack_request, oad)
         end
       end
+      @installed = true
+    end
+
+    def self.raise_response_error?(validated_response)
+      configuration.response_raise_error && !configuration.ignored_unknown_status.include?(validated_response.status)
+    end
+
+    def self.uninstall
+      configuration = OpenapiFirst.configuration
+      configuration.hooks[:after_request_validation].delete(@after_request_validation)
+      configuration.hooks[:after_response_validation].delete(@after_response_validation)
+      definitions.clear
+      @configuration = nil
+      @installed = nil
+      @exit_handler = nil
     end
   end
 end

data/lib/openapi_first/validators/request_body.rb

@@ -11,7 +11,7 @@ module OpenapiFirst
       def call(parsed_request)
         body = parsed_request.body
         if body.nil?
-          Failure.fail!(:invalid_body, message: 'Request body is not defined') if @required
+          Failure.fail!(:invalid_body, message: 'Request body must not be empty') if @required
           return
         end
 

data/lib/openapi_first/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module OpenapiFirst
-  VERSION = '2.7.4'
+  VERSION = '2.8.0'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: openapi_first
 version: !ruby/object:Gem::Version
-  version: 2.7.4
+  version: 2.8.0
 platform: ruby
 authors:
 - Andreas Haller
@@ -123,6 +123,8 @@ files:
 - lib/openapi_first/schema/validation_error.rb
 - lib/openapi_first/schema/validation_result.rb
 - lib/openapi_first/test.rb
+- lib/openapi_first/test/callable.rb
+- lib/openapi_first/test/configuration.rb
 - lib/openapi_first/test/coverage.rb
 - lib/openapi_first/test/coverage/plan.rb
 - lib/openapi_first/test/coverage/request_task.rb
@@ -131,7 +133,9 @@ files:
 - lib/openapi_first/test/coverage/terminal_formatter.rb
 - lib/openapi_first/test/methods.rb
 - lib/openapi_first/test/minitest_helpers.rb
+- lib/openapi_first/test/observe.rb
 - lib/openapi_first/test/plain_helpers.rb
+- lib/openapi_first/test/registry.rb
 - lib/openapi_first/validated_request.rb
 - lib/openapi_first/validated_response.rb
 - lib/openapi_first/validators/request_body.rb