openapi_first 2.2.4 → 2.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d85c4425401207c7ff52f921d34c77c7ee6c6ee5671ecfbac36ca5619685cdb6
-  data.tar.gz: afcb639314fd7f049b93d29dd89cbb88260abbcb66f1e5a20b8c1c01ade063b2
+  metadata.gz: 131c3511b96fc2a420b9c0a1b2e4deeb5b24d8602ad6210e5a60ed1a46fe6257
+  data.tar.gz: f6b7a44491e9aedf68498b2200a46e6048b9036360447a34c077c7476da8fc70
 SHA512:
-  metadata.gz: 8c9e2fadb4ba81ab68026b5b2f6c44daf9710d3e3fc27e32175f9691f885839f6fda6cd1f477c7e347b88ce0d27e933bd30642d1773665301613049df092e7a5
-  data.tar.gz: 493dcfa9384f1462b7ab713dcc8c3c9b7cb2c6f63a4977f0ed3485d65811a3cd0869c2ddc5f8dc9cd5efacaa773836b0290d2be7f88b4af20de9adad2aa7c7b6
+  metadata.gz: 498b7341aa473504c5fa13dd95e92d7d1e2317d690ddbd88fac36974c05b10cccd8ae8b9dccc025036151aab484304e14f053003668b576c547ac1c7c0716d4b
+  data.tar.gz: 5cb776022d6e0fb684b3c30e7dd507a6591e5bc0b34bc384347a28e2b5030f20c976fa6f92b9b3e81c8d8ec6912d46e6b0ee053189267de320ccf418e7d88f4f

data/CHANGELOG.md

@@ -2,6 +2,24 @@
 
 ## Unreleased
 
+## 2.4.0
+
+- Support less verbose test setup without the need to call `OpenapiFirst::Test.report_coverage`, which will be called `at_exit`:
+  ```ruby
+  OpenapiFirst::Test.setup do |test|
+    test.register('openapi/openapi.yaml')
+    test.minimum_coverage = 100 # Setting this will lead to an `exit 2` if coverage is below minimum
+  end
+  ```
+- Add `OpenapiFirst::Test::Setup#minimum_coverage=` to control exit behaviour (exit 2 if coverage is below minimum)
+- Add `verbose` option to `OpenapiFirst::Test.report_coverage(verbose: true)`
+  to see all passing requests/responses
+
+## 2.3.0
+
+### New feature
+- Add OpenapiFirst::Test::Coverage to track request/response coverage for your API descriptions. (https://github.com/ahx/openapi_first/pull/327)
+
 ## 2.2.4
 
 - Fix request validation file uploads in multipart/form-data requests with nested fields (https://github.com/ahx/openapi_first/issues/324)

data/README.md

@@ -1,21 +1,20 @@
 # openapi_first
 
-OpenapiFirst helps to implement HTTP APIs based on an [OpenAPI](https://www.openapis.org/) API description. It supports OpenAPI 3.0 and 3.1. It offers request and response validation and it ensures that your implementation follows exactly the API description.
+openapi_first is a Ruby gem for request / response validation and contract-testing against an [OpenAPI](https://www.openapis.org/) 3.0 or 3.1 API description. It makes an APIFirst workflow easy and reliable.
 
-[![Tests](https://github.com/ahx/openapi_first/actions/workflows/ruby.yml/badge.svg)](https://github.com/ahx/openapi_first/actions/workflows/ruby.yml)
-[![CodeQL](https://github.com/ahx/openapi_first/actions/workflows/codeql.yml/badge.svg)](https://github.com/ahx/openapi_first/blob/codeql/.github/workflows/codeql.yml)
+You can use openapi_first on production for [request validation](#request-validation) and in your tests to avoid API drift with it's request/response validation and coverage features.
 
 ## Contents
 
 <!-- TOC -->
 
-- [Manual use](#manual-use)
-  - [Validate request](#validate-request)
-  - [Validate response](#validate-response)
 - [Rack Middlewares](#rack-middlewares)
   - [Request validation](#request-validation)
   - [Response validation](#response-validation)
-- [Test assertions](#test-assertions)
+- [Contract testing](#contract-testing)
+  - [Coverage](#coverage)
+  - [Test assertions](#test-assertions)
+- [Manual use](#manual-use)
 - [Framework integration](#framework-integration)
 - [Configuration](#configuration)
 - [Hooks](#hooks)
@@ -26,59 +25,6 @@ OpenapiFirst helps to implement HTTP APIs based on an [OpenAPI](https://www.open
 
 <!-- /TOC -->
 
-## Manual use
-
-Load the API description:
-
-```ruby
-require 'openapi_first'
-
-definition = OpenapiFirst.load('openapi.yaml')
-```
-
-### Validate request
-
-```ruby
-validated_request = definition.validate_request(rack_request)
-
-# Inspect the request and access parsed parameters
-validated_request.valid?
-validated_request.invalid?
-validated_request.error # => Failure object or nil
-validated_request.parsed_body # => The parsed request body (Hash)
-validated_request.parsed_query # A Hash of query parameters that are defined in the API description, parsed exactly as described.
-validated_request.parsed_path_parameters
-validated_request.parsed_headers
-validated_request.parsed_cookies
-validated_request.parsed_params # Merged parsed path, query parameters and request body
-# Access the Openapi 3 Operation Object Hash
-validated_request.operation['x-foo']
-validated_request.operation['operationId'] => "getStuff"
-# or the whole request definition
-validated_request.request_definition.path # => "/pets/{petId}"
-validated_request.request_definition.operation_id # => "showPetById"
-
-# Or you can raise an exception if validation fails:
-definition.validate_request(rack_request, raise_error: true) # Raises OpenapiFirst::RequestInvalidError or OpenapiFirst::NotFoundError if request is invalid
-```
-
-### Validate response
-
-```ruby
-validated_response = definition.validate_response(rack_request, rack_response)
-
-# Inspect the response and access parsed parameters and
-validated_response.valid?
-validated_response.invalid?
-validated_response.error # => Failure object or nil
-validated_response.status # => 200
-validated_response.parsed_body
-validated_response.parsed_headers
-
-# Or you can raise an exception if validation fails:
-definition.validate_response(rack_request,rack_response, raise_error: true) # Raises OpenapiFirst::ResponseInvalidError or OpenapiFirst::ResponseNotFoundError
-```
-
 ## Rack Middlewares
 
 ### Request validation
@@ -87,15 +33,10 @@ The request validation middleware returns a 4xx if the request is invalid or not
 
 ```ruby
 use OpenapiFirst::Middlewares::RequestValidation, spec: 'openapi.yaml'
-```
 
-#### Options
-
-| Name              | Possible values                                                          | Description                                                                                                                         |
-| :---------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------- |
-| `spec:`           |                                                                          | The path to the spec file or spec loaded via `OpenapiFirst.load`                                                                    |
-| `raise_error:`    | `false` (default), `true`                                                | If set to true the middleware raises `OpenapiFirst::RequestInvalidError` or `OpenapiFirst::NotFoundError` instead of returning 4xx. |
-| `error_response:` | `:default` (default), `:jsonapi`, Your implementation of `ErrorResponse` or `false` to disable responding  |
+# Pass `raise_error: true` to raise an error if request is invalid:
+use OpenapiFirst::Middlewares::RequestValidation, raise_error: true, spec: 'openapi.yaml'
+```
 
 #### Error responses
 
@@ -129,7 +70,7 @@ content-type: "application/problem+json"
 }
 ```
 
-openapi_first offers a [JSON:API](https://jsonapi.org/) error response as well:
+openapi_first offers a [JSON:API](https://jsonapi.org/) error response by passing `error_response: :jsonapi`:
 
 ```ruby
 use OpenapiFirst::Middlewares::RequestValidation, spec: 'openapi.yaml, error_response: :jsonapi'
@@ -179,37 +120,56 @@ use OpenapiFirst::Middlewares::RequestValidation, spec: 'openapi.yaml, error_res
 You can build your own custom error response with `error_response: MyCustomClass` that implements `OpenapiFirst::ErrorResponse`.
 You can define custom error responses globally by including / implementing `OpenapiFirst::ErrorResponse` and register it via `OpenapiFirst.register_error_response(my_name, MyCustomErrorResponse)` and set `error_response: my_name`.
 
-#### readOnly / writeOnly properties
-
-Request validation fails if request includes a property with `readOnly: true`.
-
-Response validation fails if response body includes a property with `writeOnly: true`.
-
 ### Response validation
 
-This middleware is especially useful when testing. It raises an error by default if the response is not valid.
+This middleware raises an error by default if the response is not valid.
+This can be useful in a test or staging environment, especially if you are adopting OpenAPI for an existing implementation.
 
 ```ruby
 use OpenapiFirst::Middlewares::ResponseValidation, spec: 'openapi.yaml' if ENV['RACK_ENV'] == 'test'
+
+# Pass `raise_error: false` to not raise an error:
+use OpenapiFirst::Middlewares::ResponseValidation, raise_error: false, spec: 'openapi.yaml'
 ```
 
-#### Options
+If you are adopting OpenAPI you can use these options together with [hooks](#hooks) to get notified about requests/responses that do match your API description.
 
-| Name    | Possible values | Description                                                      |
-| :------ | --------------- | ---------------------------------------------------------------- |
-| `spec:` |                 | The path to the spec file or spec loaded via `OpenapiFirst.load` |
-| `raise_error:`    | `true` (default), `false`                                                | If set to true the middleware raises `OpenapiFirst::ResponseInvalidError` or `OpenapiFirst::ResonseNotFoundError` if the response does not match the API description. |
+## Contract Testing
 
-## Test assertions
+### Coverage
 
-openapi_first ships with a simple but powerful Test module to run request and response validation in your tests without using the middlewares. This is designed to be used with rack-test or Ruby on Rails integration tests or request specs.
+> [!NOTE]
+> This is a brand new feature. ✨ Your feedback is very welcome.
 
-Here is how to set it up for Rails integration tests:
+This feature tracks all requests/resposes that are validated via openapi_first and get return an overal coverage value. If all of your described requests/responses have been validated successfully at least once, your coverage is 100%.
+By checking your validation coverage you can avoid API drift where your API description describes requests/responses differently than your implemention works.
 
-```ruby
-# test_helper.rb
-OpenapiFirst::Test.register('openapi/v1.openapi.yaml')
-```
+Here is how to set it up for RSpec in your `spec/spec_helper.rb`:
+
+1. Register all OpenAPI documents to track coverage for and start tracking. This should go at the top of you test helper file before loading application code.
+  ```ruby
+  require 'openapi_first'
+  OpenapiFirst::Test.setup do |test|
+    test.register('openapi/openapi.yaml')
+    test.minimum_coverage = 100 # Setting this will lead to an `exit 2` if coverage is below minimum
+  end
+  ```
+2. Wrap your app with silent request / response validation. This validates all requets/responses you do during your test run. (✷1)
+  ```ruby
+  config.before type: :request do
+    def app
+      OpenapiFirst::Test.app(App)
+    end
+  end
+  ```
+
+(✷1): Instead of using `OpenapiFirstTest.app` to wrap your application, you can use the middlewares or [test assertion method](#test-assertions), but you would have to do that for all requests/responses defined in your API description to make coverage work.
+
+### Test assertions
+
+openapi_first ships with a simple but powerful Test method to run request and response validation in your tests without using the middlewares. This is designed to be used with rack-test or Ruby on Rails integration tests or request specs.
+
+Here is how to set it up for Rails integration tests:
 
 Inside your test:
 ```ruby
@@ -225,10 +185,64 @@ class TripsApiTest < ActionDispatch::IntegrationTest
                   date: '2024-07-02T09:00:00Z' }
 
     assert_api_conform(status: 200)
+    # assert_api_conform(status: 200, api: :v1) # Or this if you have multiple API descriptions
   end
 end
 ```
 
+## Manual use
+
+Load the API description:
+
+```ruby
+require 'openapi_first'
+
+definition = OpenapiFirst.load('openapi.yaml')
+```
+
+### Validate request
+
+```ruby
+validated_request = definition.validate_request(rack_request)
+
+# Inspect the request and access parsed parameters
+validated_request.valid?
+validated_request.invalid?
+validated_request.error # => Failure object or nil
+validated_request.parsed_body # => The parsed request body (Hash)
+validated_request.parsed_query # A Hash of query parameters that are defined in the API description, parsed exactly as described.
+validated_request.parsed_path_parameters
+validated_request.parsed_headers
+validated_request.parsed_cookies
+validated_request.parsed_params # Merged parsed path, query parameters and request body
+# Access the Openapi 3 Operation Object Hash
+validated_request.operation['x-foo']
+validated_request.operation['operationId'] => "getStuff"
+# or the whole request definition
+validated_request.request_definition.path # => "/pets/{petId}"
+validated_request.request_definition.operation_id # => "showPetById"
+
+# Or you can raise an exception if validation fails:
+definition.validate_request(rack_request, raise_error: true) # Raises OpenapiFirst::RequestInvalidError or OpenapiFirst::NotFoundError if request is invalid
+```
+
+### Validate response
+
+```ruby
+validated_response = definition.validate_response(rack_request, rack_response)
+
+# Inspect the response and access parsed parameters and
+validated_response.valid?
+validated_response.invalid?
+validated_response.error # => Failure object or nil
+validated_response.status # => 200
+validated_response.parsed_body
+validated_response.parsed_headers
+
+# Or you can raise an exception if validation fails:
+definition.validate_response(rack_request,rack_response, raise_error: true) # Raises OpenapiFirst::ResponseInvalidError or OpenapiFirst::ResponseNotFoundError
+```
+
 ## Configuration
 
 You can configure default options globally:
@@ -291,16 +305,6 @@ end
 Using rack middlewares is supported in probably all Ruby web frameworks.
 If you are using Ruby on Rails for example, you can add the request validation middleware globally in `config/application.rb` or inside specific controllers.
 
-When running integration tests (or request specs when using rspec), it makes sense to add the response validation middleware to `config/environments/test.rb`:
-
-```ruby
-config.middleware.use OpenapiFirst::Middlewares::ResponseValidation,
-  spec: 'api/openapi.yaml'
-```
-
-That way you don't have to call specific test assertions to make sure your API matches the OpenAPI document.
-There is no need to run response validation on production if your test coverage is decent.
-
 ## Alternatives
 
 This gem was inspired by [committe](https://github.com/interagent/committee) (Ruby) and [Connexion](https://github.com/spec-first/connexion) (Python).
@@ -328,6 +332,6 @@ bundle exec ruby benchmarks.rb
 
 ### Contributing
 
-If you have a question or an idea or found a bug don't hesitate to [create an issue](https://github.com/ahx/openapi_first/issues) or [start a discussion](https://github.com/ahx/openapi_first/discussions).
+If you have a question or an idea or found a bug, don't hesitate to create an issue [on Github](https://github.com/ahx/openapi_first) or [Codeberg](https://codeberg.org/ahx/openapi_first) or say hi on [Mastodon (ruby.social)](https://ruby.social/@ahx).
 
 Pull requests are very welcome as well, of course. Feel free to create a "draft" pull request early on, even if your change is still work in progress. 🤗

data/lib/openapi_first/builder.rb

@@ -22,7 +22,7 @@ module OpenapiFirst
       @schemer_configuration.insert_property_defaults = true
 
       @config = config
-      @contents = RefResolver.for(contents, dir: filepath && File.dirname(filepath))
+      @contents = RefResolver.for(contents, filepath:)
     end
 
     attr_reader :config
@@ -58,17 +58,18 @@ module OpenapiFirst
               request,
               request_method:,
               path:,
-              content_type: request.content_type
-            )
-          end
-          build_responses(responses: operation_object['responses']).each do |response|
-            router.add_response(
-              response,
-              request_method:,
-              path:,
-              status: response.status,
-              response_content_type: response.content_type
+              content_type: request.content_type,
+              allow_empty_content: request.allow_empty_content?
             )
+            build_responses(request:, responses: operation_object['responses']).each do |response|
+              router.add_response(
+                response,
+                request_method:,
+                path:,
+                status: response.status,
+                response_content_type: response.content_type
+              )
+            end
           end
         end
       end
@@ -106,28 +107,37 @@ module OpenapiFirst
     end
 
     def build_requests(path:, request_method:, operation_object:, parameters:)
+      content_objects = operation_object.dig('requestBody', 'content')
+      if content_objects.nil?
+        return [
+          request_without_body(path:, request_method:, parameters:, operation_object:)
+        ]
+      end
       required_body = operation_object['requestBody']&.resolved&.fetch('required', false) == true
-      result = operation_object.dig('requestBody', 'content')&.map do |content_type, content_object|
+      content_objects.map do |content_type, content_object|
         content_schema = content_object['schema'].schema(
           configuration: schemer_configuration,
           after_property_validation: config.hooks[:after_request_body_property_validation]
         )
-        Request.new(path:, request_method:,
+        Request.new(path:, request_method:, parameters:,
                     operation_object: operation_object.resolved,
-                    parameters:, content_type:,
+                    content_type:,
                     content_schema:,
-                    required_body:)
-      end || []
-      return result if required_body
-
-      result << Request.new(
-        path:, request_method:, operation_object: operation_object.resolved,
-        parameters:, content_type: nil, content_schema: nil,
-        required_body:
-      )
+                    required_body:,
+                    key: [path, request_method, content_type].join(':'))
+      end
+    end
+
+    def request_without_body(path:, request_method:, parameters:, operation_object:)
+      Request.new(path:, request_method:, parameters:,
+                  operation_object: operation_object.resolved,
+                  content_type: nil,
+                  content_schema: nil,
+                  required_body: false,
+                  key: [path, request_method, nil].join(':'))
     end
 
-    def build_responses(responses:)
+    def build_responses(responses:, request:)
       return [] unless responses
 
       responses.flat_map do |status, response_object|
@@ -142,9 +152,10 @@ module OpenapiFirst
                        headers:,
                        headers_schema:,
                        content_type:,
-                       content_schema:)
+                       content_schema:,
+                       key: [request.key, status, content_type].join(':'))
         end || Response.new(status:, headers:, headers_schema:, content_type: nil,
-                            content_schema: nil)
+                            content_schema: nil, key: [request.key, status, nil].join(':'))
       end
     end
 
@@ -198,5 +209,6 @@ module OpenapiFirst
 
     ParsedParameters = Data.define(:path, :query, :header, :cookie, :path_schema, :query_schema, :header_schema,
                                    :cookie_schema)
+    private_constant :ParsedParameters
   end
 end

data/lib/openapi_first/configuration.rb

@@ -14,7 +14,7 @@ module OpenapiFirst
       @request_validation_error_response = OpenapiFirst.find_error_response(:default)
       @request_validation_raise_error = false
       @response_validation_raise_error = true
-      @hooks = (HOOKS.map { [_1, []] }).to_h
+      @hooks = (HOOKS.map { [_1, Set.new] }).to_h
     end
 
     attr_reader :request_validation_error_response, :hooks

data/lib/openapi_first/definition.rb

@@ -63,7 +63,7 @@ module OpenapiFirst
     # Validates the response against the API description.
     # @param rack_request [Rack::Request] The Rack request object.
     # @param rack_response [Rack::Response] The Rack response object.
-    # @param raise_error [Boolean] Whether to raise an error if validation fails.
+    # @param raise_error [Boolean] Whethir to raise an error if validation fails.
     # @return [ValidatedResponse] The validated response object.
     def validate_response(rack_request, rack_response, raise_error: false)
       route = @router.match(rack_request.request_method, rack_request.path, content_type: rack_request.content_type)

data/lib/openapi_first/file_loader.rb

@@ -8,7 +8,7 @@ module OpenapiFirst
     module_function
 
     def load(file_path)
-      raise FileNotFoundError, "File not found #{file_path}" unless File.exist?(file_path)
+      raise FileNotFoundError, "File not found #{file_path.inspect}" unless File.exist?(file_path)
 
       body = File.read(file_path)
       extname = File.extname(file_path)

data/lib/openapi_first/ref_resolver.rb

@@ -6,17 +6,17 @@ module OpenapiFirst
   # This is here to give traverse an OAD while keeping $refs intact
   # @visibility private
   module RefResolver
-    def self.load(file_path)
-      contents = OpenapiFirst::FileLoader.load(file_path)
-      self.for(contents, dir: File.dirname(File.expand_path(file_path)))
+    def self.load(filepath)
+      contents = OpenapiFirst::FileLoader.load(filepath)
+      self.for(contents, filepath:)
     end
 
-    def self.for(value, context: value, dir: Dir.pwd)
+    def self.for(value, filepath: nil, context: value)
       case value
       when ::Hash
-        Hash.new(value, context:, dir:)
+        Hash.new(value, context:, filepath:)
       when ::Array
-        Array.new(value, context:, dir:)
+        Array.new(value, context:, filepath:)
       when ::NilClass
         nil
       else
@@ -37,9 +37,11 @@ module OpenapiFirst
 
     # @visibility private
     module Resolvable
-      def initialize(value, context: value, dir: nil)
+      def initialize(value, context: value, filepath: nil)
         @value = value
         @context = context
+        @filepath = filepath
+        dir = File.dirname(File.expand_path(filepath)) if filepath
         @dir = (dir && File.absolute_path(dir)) || Dir.pwd
       end
 
@@ -50,12 +52,14 @@ module OpenapiFirst
       # The object where this node was found in
       attr_reader :context
 
+      private attr_reader :filepath
+
       def resolve_ref(pointer)
         if pointer.start_with?('#')
           value = Hana::Pointer.new(pointer[1..]).eval(context)
           raise "Unknown reference #{pointer} in #{context}" unless value
 
-          return RefResolver.for(value, dir:, context:)
+          return RefResolver.for(value, filepath:, context:)
         end
 
         relative_path, file_pointer = pointer.split('#')
@@ -63,9 +67,12 @@ module OpenapiFirst
         return RefResolver.load(full_path) unless file_pointer
 
         file_contents = FileLoader.load(full_path)
-        new_dir = File.dirname(full_path)
         value = Hana::Pointer.new(file_pointer).eval(file_contents)
-        RefResolver.for(value, dir: new_dir, context: file_contents)
+        RefResolver.for(value, filepath: full_path, context: file_contents)
+      rescue OpenapiFirst::FileNotFoundError => e
+        message = "Problem with reference resolving #{pointer.inspect} in " \
+                  "file #{File.absolute_path(filepath).inspect}: #{e.message}"
+        raise OpenapiFirst::FileNotFoundError, message
       end
     end
 
@@ -91,18 +98,18 @@ module OpenapiFirst
       def [](key)
         return resolve_ref(@value['$ref'])[key] if !@value.key?(key) && @value.key?('$ref')
 
-        RefResolver.for(@value[key], dir:, context:)
+        RefResolver.for(@value[key], filepath:, context:)
       end
 
       def fetch(key)
         return resolve_ref(@value['$ref']).fetch(key) if !@value.key?(key) && @value.key?('$ref')
 
-        RefResolver.for(@value.fetch(key), dir:, context:)
+        RefResolver.for(@value.fetch(key), filepath:, context:)
       end
 
       def each
         resolved.each do |key, value|
-          yield key, RefResolver.for(value, dir:, context:)
+          yield key, RefResolver.for(value, filepath:, context:)
         end
       end
 
@@ -126,12 +133,12 @@ module OpenapiFirst
         item = @value[index]
         return resolve_ref(item['$ref']) if item.is_a?(::Hash) && item.key?('$ref')
 
-        RefResolver.for(item, dir:, context:)
+        RefResolver.for(item, filepath:, context:)
       end
 
       def each
         resolved.each do |item|
-          yield RefResolver.for(item, dir:, context:)
+          yield RefResolver.for(item, filepath:, context:)
         end
       end
 

data/lib/openapi_first/request.rb

@@ -10,13 +10,16 @@ module OpenapiFirst
   # An 3.x Operation object can accept multiple requests, because it can handle multiple content-types.
   # This class represents one of those requests.
   class Request
+    # rubocop:disable Metrics/MethodLength
     def initialize(path:, request_method:, operation_object:,
-                   parameters:, content_type:, content_schema:, required_body:)
+                   parameters:, content_type:, content_schema:, required_body:, key:)
       @path = path
       @request_method = request_method
       @content_type = content_type
       @content_schema = content_schema
       @operation = operation_object
+      @allow_empty_content = content_type.nil? || required_body == false
+      @key = key
       @request_parser = RequestParser.new(
         query_parameters: parameters.query,
         path_parameters: parameters.path,
@@ -33,8 +36,13 @@ module OpenapiFirst
         cookie_schema: parameters.cookie_schema
       )
     end
+    # rubocop:enable Metrics/MethodLength
 
-    attr_reader :content_type, :content_schema, :operation, :request_method, :path
+    attr_reader :content_type, :content_schema, :operation, :request_method, :path, :key
+
+    def allow_empty_content?
+      @allow_empty_content
+    end
 
     def validate(request, route_params:)
       parsed_request = nil

data/lib/openapi_first/response.rb

@@ -9,12 +9,13 @@ module OpenapiFirst
   # This is not a direct reflecton of the OpenAPI 3.X response definition, but a combination of
   # status, content type and content schema.
   class Response
-    def initialize(status:, headers:, headers_schema:, content_type:, content_schema:)
+    def initialize(status:, headers:, headers_schema:, content_type:, content_schema:, key:)
       @status = status
       @content_type = content_type
       @content_schema = content_schema
       @headers = headers
       @headers_schema = headers_schema
+      @key = key
       @parser = ResponseParser.new(headers:, content_type:)
       @validator = ResponseValidator.new(self)
     end
@@ -22,11 +23,15 @@ module OpenapiFirst
     # @attr_reader [Integer] status The HTTP status code of the response definition.
     # @attr_reader [String, nil] content_type Content type of this response.
     # @attr_reader [Schema, nil] content_schema the Schema of the response body.
-    attr_reader :status, :content_type, :content_schema, :headers, :headers_schema
+    attr_reader :status, :content_type, :content_schema, :headers, :headers_schema, :key
 
     def validate(response)
-      parsed_values = @parser.parse(response)
-      error = @validator.call(parsed_values)
+      parsed_values = nil
+      error = catch FAILURE do
+        parsed_values = @parser.parse(response)
+        nil
+      end
+      error ||= @validator.call(parsed_values)
       ValidatedResponse.new(response, parsed_values:, error:, response_definition: self)
     end
 

data/lib/openapi_first/router.rb

@@ -32,15 +32,18 @@ module OpenapiFirst
         request_methods.filter_map do |request_method, content|
           next if request_method == :template
 
-          Route.new(path:, request_method:, requests: content[:requests].each_value,
+          Route.new(path:, request_method:, requests: content[:requests].each_value.lazy.uniq,
                     responses: content[:responses].each_value.lazy.flat_map(&:values))
         end
       end
     end
 
     # Add a request definition
-    def add_request(request, request_method:, path:, content_type: nil)
-      route_at(path, request_method)[:requests][content_type] = request
+    def add_request(request, request_method:, path:, content_type: nil, allow_empty_content: false)
+      route = route_at(path, request_method)
+      requests = route[:requests]
+      requests[content_type] = request
+      requests[nil] = request if allow_empty_content
     end
 
     # Add a response definition

data/lib/openapi_first/test/coverage/plan.rb

@@ -0,0 +1,69 @@
+# frozen_string_literal: true
+
+require_relative 'route_task'
+require_relative 'response_task'
+require_relative 'request_task'
+
+module OpenapiFirst
+  module Test
+    module Coverage
+      # This stores the coverage data for one API description
+      # A plan can be #done? and has several #tasks which can be #finished?
+      class Plan
+        class UnknownRequestError < StandardError; end
+
+        def initialize(oad)
+          @oad = oad
+          @routes = []
+          @index = {}
+          @filepath = oad.filepath
+          oad.routes.each do |route|
+            add_route request_method: route.request_method,
+                      path: route.path,
+                      requests: route.requests,
+                      responses: route.responses
+          end
+        end
+
+        attr_reader :filepath, :oad, :routes
+        private attr_reader :index
+
+        def track_request(validated_request)
+          index[validated_request.request_definition.key].track(validated_request) if validated_request.known?
+        end
+
+        def track_response(validated_response)
+          index[validated_response.response_definition.key].track(validated_response) if validated_response.known?
+        end
+
+        def done?
+          tasks.all?(&:finished?)
+        end
+
+        def coverage
+          done = tasks.count(&:finished?)
+          return 0 if done.zero?
+
+          all = tasks.count
+          (done / (all.to_f / 100)).to_i
+        end
+
+        def tasks
+          index.values
+        end
+
+        private
+
+        def add_route(request_method:, path:, requests:, responses:)
+          request_tasks = requests.to_a.map do |request|
+            index[request.key] = RequestTask.new(request)
+          end
+          response_tasks = responses.to_a.map do |response|
+            index[response.key] = ResponseTask.new(response)
+          end
+          @routes << RouteTask.new(path:, request_method:, requests: request_tasks, responses: response_tasks)
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test/coverage/request_task.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+require 'forwardable'
+
+module OpenapiFirst
+  module Test
+    module Coverage
+      # @visibility private
+      class RequestTask
+        extend Forwardable
+
+        def_delegators :@request, :path, :request_method, :content_type
+
+        def initialize(request_definition)
+          @request = request_definition
+          @requested = false
+        end
+
+        attr_reader :request
+
+        def track(validated_request)
+          @requested = true
+          @valid ||= true if validated_request.valid?
+        end
+
+        def requested?
+          @requested == true
+        end
+
+        def any_valid_request?
+          @valid == true
+        end
+
+        def finished?
+          requested? && any_valid_request?
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test/coverage/response_task.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+require 'forwardable'
+
+module OpenapiFirst
+  module Test
+    module Coverage
+      # @visibility private
+      class ResponseTask
+        extend Forwardable
+
+        def_delegators :@response, :status, :content_type, :key
+
+        def initialize(response_definition)
+          @response = response_definition
+          @responded = false
+        end
+
+        attr_reader :response
+
+        def track(validated_response)
+          @responded = true
+          @valid ||= true if validated_response.valid?
+        end
+
+        def responded?
+          @responded == true
+        end
+
+        def any_valid_response?
+          @valid == true
+        end
+
+        def finished?
+          responded? && any_valid_response?
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test/coverage/route_task.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module OpenapiFirst
+  module Test
+    module Coverage
+      RouteTask = Data.define(:path, :request_method, :requests, :responses) do
+        def finished?
+          requests.all?(&:finished?) && responses.all?(&:finished?)
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test/coverage/terminal_formatter.rb

@@ -0,0 +1,105 @@
+# frozen_string_literal: true
+
+module OpenapiFirst
+  module Test
+    module Coverage
+      # This is the default formatter
+      class TerminalFormatter
+        def initialize(verbose: false)
+          @verbose = verbose
+        end
+
+        # This takes a list of Coverage::Plan instances and outputs a String
+        def format(coverage_result)
+          @out = StringIO.new
+          coverage_result.plans.each { |plan| format_plan(plan) }
+          @out.string
+        end
+
+        private attr_reader :out, :verbose
+
+        private
+
+        def puts(string)
+          @out.puts(string)
+        end
+
+        def print(string)
+          @out.print(string)
+        end
+
+        def format_plan(plan)
+          filepath = plan.filepath
+          puts ['', "API validation coverage for #{filepath}: #{plan.coverage}%"]
+          return if plan.done? && !verbose
+
+          plan.routes.each do |route|
+            next if route.finished? && !verbose
+
+            format_requests(route.requests)
+            next if route.requests.none?(&:requested?)
+
+            format_responses(route.responses)
+          end
+        end
+
+        def format_requests(requests)
+          requests.each do |request|
+            if request.finished?
+              puts green "✓ #{request_label(request)}"
+            else
+              puts red "❌ #{request_label(request)} – #{explain_unfinished_request(request)}"
+            end
+          end
+        end
+
+        def format_responses(responses)
+          responses.each do |response|
+            if response.finished?
+              puts green "  ✓  #{response_label(response)}" if verbose
+            else
+              puts red "  ❌ #{response_label(response)} – #{explain_unfinished_response(response)}"
+            end
+          end
+        end
+
+        def green(text)
+          "\e[32m#{text}\e[0m"
+        end
+
+        def red(text)
+          "\e[31m#{text}\e[0m"
+        end
+
+        def orange(text)
+          "\e[33m#{text}\e[0m"
+        end
+
+        def request_label(request)
+          name = "#{request.request_method.upcase} #{request.path}" # TODO: add required query parameters?
+          name << " (#{request.content_type})" if request.content_type
+          name
+        end
+
+        def explain_unfinished_request(request)
+          return 'No requests tracked!' unless request.requested?
+
+          'All requests invalid!' unless request.any_valid_request?
+        end
+
+        def response_label(response)
+          name = +''
+          name += response.status.to_s
+          name += "(#{response.content_type})" if response.content_type
+          name
+        end
+
+        def explain_unfinished_response(response)
+          return 'No responses tracked!' unless response.responded?
+
+          'All responses invalid!' unless response.any_valid_response?
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test/coverage.rb

@@ -0,0 +1,75 @@
+# frozen_string_literal: true
+
+require_relative 'coverage/plan'
+
+module OpenapiFirst
+  module Test
+    # The Coverage module is about tracking request and response validation
+    # to assess if all parts of the API description have been tested.
+    # Currently it does not care about unknown requests that are not part of any API description.
+    module Coverage
+      autoload :TerminalFormatter, 'openapi_first/test/coverage/terminal_formatter'
+
+      Result = Data.define(:plans, :coverage)
+
+      class << self
+        def start
+          @after_request_validation = lambda do |validated_request, oad|
+            track_request(validated_request, oad)
+          end
+
+          @after_response_validation = lambda do |validated_response, request, oad|
+            track_response(validated_response, request, oad)
+          end
+
+          OpenapiFirst.configure do |config|
+            config.after_request_validation(&@after_request_validation)
+            config.after_response_validation(&@after_response_validation)
+          end
+        end
+
+        def stop
+          configuration = OpenapiFirst.configuration
+          configuration.hooks[:after_request_validation].delete(@after_request_validation)
+          configuration.hooks[:after_response_validation].delete(@after_response_validation)
+        end
+
+        # Clear current coverage run
+        def reset
+          @current_run = nil
+        end
+
+        def track_request(request, oad)
+          current_run[oad.filepath].track_request(request)
+        end
+
+        def track_response(response, _request, oad)
+          current_run[oad.filepath].track_response(response)
+        end
+
+        def result
+          Result.new(plans:, coverage:)
+        end
+
+        private
+
+        # Returns all plans (Plan) that were registered for this run
+        def plans
+          current_run.values
+        end
+
+        def coverage
+          return 0 if plans.empty?
+
+          plans.sum(&:coverage) / plans.length
+        end
+
+        def current_run
+          @current_run ||= Test.definitions.values.to_h do |oad|
+            [oad.filepath, Plan.new(oad)]
+          end
+        end
+      end
+    end
+  end
+end

data/lib/openapi_first/test/methods.rb

@@ -4,7 +4,6 @@ require_relative 'minitest_helpers'
 require_relative 'plain_helpers'
 
 module OpenapiFirst
-  # Test integration
   module Test
     # Methods to use in integration tests
     module Methods

data/lib/openapi_first/test.rb

@@ -1,33 +1,116 @@
 # frozen_string_literal: true
 
-require_relative 'test/methods'
-
 module OpenapiFirst
   # Test integration
   module Test
+    autoload :Coverage, 'openapi_first/test/coverage'
+    autoload :Methods, 'openapi_first/test/methods'
+
     def self.minitest?(base)
       base.include?(::Minitest::Assertions)
     rescue NameError
       false
     end
 
-    class NotRegisteredError < StandardError; end
+    # Helper class to setup tests
+    class Setup
+      def initialize
+        @minimum_coverage = 0
+        yield self
+      end
 
-    DEFINITIONS = {} # rubocop:disable Style/MutableConstant
+      def register(*)
+        Test.register(*)
+      end
+
+      attr_accessor :minimum_coverage
 
-    def self.definitions = DEFINITIONS
+      # This called at_exit
+      def handle_exit
+        coverage = Coverage.result.coverage
+        # :nocov:
+        puts 'API Coverage did not detect any API requests for the registered API descriptions' if coverage.zero?
+        Test.report_coverage if coverage.positive?
+        return unless minimum_coverage > coverage
 
-    def self.register(path, as: :default)
-      definitions[as] = OpenapiFirst.load(path)
+        puts "API Coverage fails with exit 2, because API coverage of #{coverage}%" \
+             "is below minimum of #{minimum_coverage}%!"
+        exit 2
+        # :nocov:
+      end
     end
 
-    def self.[](api)
-      definitions.fetch(api) do
-        option = api == :default ? '' : ", as: #{api.inspect}"
-        raise(NotRegisteredError,
-              "API description '#{api.inspect}' not found." \
-              "Please call OpenapiFirst::Test.register('myopenapi.yaml'#{option}) " \
-              'once before calling assert_api_conform.')
+    def self.setup(&)
+      unless block_given?
+        raise ArgumentError, "Please provide a block to #{self.class}.setup to register you API descriptions"
+      end
+
+      Coverage.start
+      setup = Setup.new(&)
+
+      if definitions.empty?
+        raise NotRegisteredError,
+              'No API descriptions have been registered. ' \
+              'Please register your API description via ' \
+              "OpenapiFirst::Test.setup { |test| test.register('myopenapi.yaml') }"
+      end
+
+      @setup ||= at_exit do
+        setup.handle_exit
+      end
+    end
+
+    # Print the coverage report
+    # @param formatter A formatter to define the report.
+    # @output [IO] An output where to puts the report.
+    def self.report_coverage(formatter: Coverage::TerminalFormatter, **)
+      coverage_result = Coverage.result
+      puts formatter.new(**).format(coverage_result)
+      puts "The overal API validation coverage of this run is: #{coverage_result.coverage}%"
+    end
+
+    # Returns the Rack app wrapped with silent request, response validation
+    # You can use this if you want to track coverage via Test::Coverage, but don't want to use
+    # the middlewares or manual request, response validation.
+    def self.app(app, spec: nil, api: :default)
+      spec ||= self[api]
+      Rack::Builder.app do
+        use OpenapiFirst::Middlewares::ResponseValidation, spec:, raise_error: false
+        use OpenapiFirst::Middlewares::RequestValidation, spec:, raise_error: false, error_response: false
+        run app
+      end
+    end
+
+    class NotRegisteredError < StandardError; end
+    class AlreadyRegisteredError < StandardError; end
+
+    @definitions = {}
+
+    class << self
+      attr_reader :definitions
+
+      def register(path, as: :default)
+        if definitions.key?(:default)
+          raise(
+            AlreadyRegisteredError,
+            "#{definitions[as].filepath.inspect} is already registered " \
+            "as ':default' so you cannot register #{path.inspect} without " \
+            'giving it a custom name. Please call register with a custom key like: ' \
+            "OpenapiFirst::Test.register(#{path.inspect}, as: :my_other_api)"
+          )
+        end
+
+        definitions[as] = OpenapiFirst.load(path)
+      end
+
+      def [](api)
+        definitions.fetch(api) do
+          option = api == :default ? '' : ", as: #{api.inspect}"
+          raise(NotRegisteredError,
+                "API description '#{api.inspect}' not found." \
+                "Please call OpenapiFirst::Test.register('myopenapi.yaml'#{option}) " \
+                'once before running tests.')
+        end
       end
     end
   end

data/lib/openapi_first/validated_response.rb

@@ -39,6 +39,9 @@ module OpenapiFirst
       error.nil?
     end
 
+    # Returns true if the response is defined.
+    def known? = response_definition != nil
+
     # Checks if the response is invalid.
     # @return [Boolean]
     def invalid?

data/lib/openapi_first/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module OpenapiFirst
-  VERSION = '2.2.4'
+  VERSION = '2.4.0'
 end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: openapi_first
 version: !ruby/object:Gem::Version
-  version: 2.2.4
+  version: 2.4.0
 platform: ruby
 authors:
 - Andreas Haller
 bindir: bin
 cert_chain: []
-date: 2025-02-11 00:00:00.000000000 Z
+date: 2025-02-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: hana
@@ -105,7 +105,6 @@ files:
 - lib/openapi_first/json.rb
 - lib/openapi_first/middlewares/request_validation.rb
 - lib/openapi_first/middlewares/response_validation.rb
-- lib/openapi_first/rack/test.rb
 - lib/openapi_first/ref_resolver.rb
 - lib/openapi_first/request.rb
 - lib/openapi_first/request_body_parsers.rb
@@ -122,6 +121,12 @@ files:
 - lib/openapi_first/schema/validation_error.rb
 - lib/openapi_first/schema/validation_result.rb
 - lib/openapi_first/test.rb
+- lib/openapi_first/test/coverage.rb
+- lib/openapi_first/test/coverage/plan.rb
+- lib/openapi_first/test/coverage/request_task.rb
+- lib/openapi_first/test/coverage/response_task.rb
+- lib/openapi_first/test/coverage/route_task.rb
+- lib/openapi_first/test/coverage/terminal_formatter.rb
 - lib/openapi_first/test/methods.rb
 - lib/openapi_first/test/minitest_helpers.rb
 - lib/openapi_first/test/plain_helpers.rb
@@ -155,7 +160,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.2
+rubygems_version: 3.6.5
 specification_version: 4
 summary: Implement HTTP APIs based on OpenApi 3.x
 test_files: []

data/lib/openapi_first/rack/test.rb

@@ -1,5 +0,0 @@
-# frozen_string_literal: true
-
-# Integration for rack-test
-
-require 'rack/test'