pact_broker 2.109.1 → 2.110.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a4c49978b4312049fd2e394ce46bcf90f64125ff50556f9bb01f9610e1310546
-  data.tar.gz: 43f24ab98f5ce7bd757a52a085d653668d83395b6889921e8dc0d4aac5982b73
+  metadata.gz: 0747ca988604f56bec96bb0511242e8b3e9e2c18c823ec53742a1c58f80eee16
+  data.tar.gz: fbdc8b9be85cf443ccd8ddcb58704a1aece12759e267aea1b4f79ba0d05fbf9c
 SHA512:
-  metadata.gz: 3211f2cc4b2e9fbae7dd8a07bd1c49dc9d672fa46f6f4c83b6652b2266dc9498d4858a9a7714f91b09914d872e522affc5e96753cf3317e39228fecc620404c1
-  data.tar.gz: 36a5bc3f3ecc827cd6e24fa21b4d4bc9f943ec75bfdb3db8cdcdd1399533739bf0a2c15c68df816ae995fa58987fa43d4da45dcb72a2d68fff12f7594a69906a
+  metadata.gz: 416b29a9c6fe749e990ce9a051e526568d1a453c78db704d68e63002bb0ba1c17f8117b1d54ec540bea609168e26d2e7583bf087394aa249d9bacd6f0ad58ea5
+  data.tar.gz: b729576d06caac277699def3e48bfd1c75400467018da3b740fad3ff1ba84e628388369dddb60f964cf4b948d66d27096feca800f8e80314be762d327cdd1e54

data/CHANGELOG.md

@@ -1,3 +1,20 @@
+<a name="v2.110.0"></a>
+### v2.110.0 (2024-04-02)
+
+#### Features
+
+* reduce contention when updating the contract_data_updated_at field for integrations (#671)	 ([ff72d03c](/../../commit/ff72d03c))
+* support consumer version selector for all branches (#667)	 ([34334ca8](/../../commit/34334ca8))
+
+* **clean**
+  * use postgres advisory locks to ensure only one process can run a clean at a time (#672)	 ([637c25fa](/../../commit/637c25fa))
+
+#### Bug Fixes
+
+* use for_all_tag_heads instead of latest_by_consumer_tag when fetching wip by branch	 ([14148a34](/../../commit/14148a34))
+* optimise WIP pacts by using branch/tag heads (#668)	 ([871209e1](/../../commit/871209e1))
+* improve performance of WIP pacts by using branch heads instead of calculating latest pact for branch	 ([f9705583](/../../commit/f9705583))
+
 <a name="v2.109.1"></a>
 ### v2.109.1 (2024-02-21)
 

data/docs/developer/design_pattern_for_eager_loading_collections.md

@@ -0,0 +1,23 @@
+# Design pattern for eager loading collections
+
+For collection resources (eg. `/versions` ), associations included in the items (eg. branch versions) must be eager loaded for performance reasons.
+
+The responsiblities of each class used to render a collection resource are as follows:
+
+* collection decorator (eg. `VersionsDecorator`) - delegate each item in the collection to be rendered by the decorator for the individual item, render pagination links
+* item decorator (eg. `VersionDecorator`) - render the JSON for each item
+* resource (eg. `PactBroker::Api::Resources::Versions`) - coordinate between, and delegate to, the service and the decorator
+* service (eg. `PactBroker::Versions::Service`) - just delegate to repository, as there is no business logic required
+* repository (eg. `PactBroker::Versions::Repository`) - load the domain objects from the database
+
+If the associations for a model are not eager loaded, then each individual association will be lazy loaded when the decorator for the item calls the association method to render it. This results in at least `<number of items in the collection> * <number of associations to render>` calls to the database, and potentially more if any of the associations have their own associations that are required to render the item. This can cause significant performance issues.
+
+To efficiently render a collection resource, associations must be eager loaded when the collection items are loaded from the database in the repository. Since the repository method for loading the collection may be used in multiple places, and the eager loaded associations required for each of those places may be different (some may not require any associations to be eager loaded), we do not want to hard code the repository to load a fixed set of associations. The list of associations to eager load is therefore passed in to the repository finder method as an argument `eager_load_associations`.
+
+The decorator is the class that knows what associations are going to be called on the model to render the JSON, so following the design guideline of "put things together that change together", the best place for the declaration of "what associations should be eager loaded for this decorator" is in the decorator itself. The `PactBroker::Api::Decorators::BaseDecorator` has a default implementation of this method called `eager_load_associations` which attempts to automatically identify the required associations, but this can be overridden when necessary.
+
+We can therefore add the following responsiblities to our previous list:
+
+* item decorator - return a list of all the associations (including nested associations) that should be eager loaded in order to render its item
+* repository - eager load the associations that have been passed into it
+* resource - pass in the eager load associations to the repository from the decorator

data/docs/developer/matrix.md

@@ -0,0 +1,95 @@
+# The Matrix
+
+Read [these docs](https://docs.pact.io/pact_broker/advanced_topics/matrix_selectors) first for a introduction to matrix selectors and options from a user's perspective.
+
+## Terminology
+
+### Selectors
+
+#### Selector classes
+
+* `Unresolved selectors` - `PactBroker::Matrix::UnresolvedSelector` the class of the specified and ignore selectors, as they are created directly from the HTTP query. At this point, they are "unresolved" in that we do not know the IDs of the pacticipants/versions that they represent, and whether or not they even exist.
+
+* `Resolved selectors` - `PactBroker::Matrix::ResolvedSelector` the class of the object created for each selector, after its pacticipant and version IDs have been identified. If a selector represents multiple pacticipant versions (eg `{ branch: "main" }`) then one `ResolvedSelector` object will be returned for each pacticipant version found. The resolved selector keeps a reference to its original unresolved selector, as well as the pacticipant ID and the version ID. If the version or pacticipant specified does not actually exist, then a resolved selector is returned that indicates this.
+
+#### Selector collections
+
+* `Specified selectors` - the collection of objects that describe the application version(s) specified explicitly in the matrix query. eg. in `can-i-deploy --pacticipant Foo --version 1` the specified selector is `PactBroker::Matrix::UnresolvedSelector.new(pacticipant_name: "Foo", pacticipant_version_number: "1")`. There may be multiple specified selectors, for example, in the Matrix page for an integration Foo/Bar, the specified selectors would be `[ UnresolvedSelector.new(pacticipant_name: "Foo"), UnresolvedSelector.new(pacticipant_name: "Bar")]`. The selectors may use various combinations of pacticipant name, version number, branch, tag or environment to identify the pacticipant versions.
+
+* `Ignore selectors` - the collection of objects that describe the applications (or application versions) to ignore, if there are any missing or failed verifications between the versions for the specified selectors and the ignore selectors. eg. If we know that provider Dog is not ready and the verifications are failing, but the integration is feature toggled off in the Foo code, we would use the command `can-i-deploy --pacticipant Foo --version 1 --to-environment production --ignore Dog` to allow `can-i-deploy` to pass. An ignore selector can have a version number, but it's more common to just provide the name.
+
+* `Inferred selectors` - the collection of objects that describe the application versions(s) that already exist in the target environment/with the target tag/on the target branches. These are identified during the matrix query when the `can-i-deploy` query has a `--to TAG`/`--to-environment`/`--with-main-branches` option specified, and they allow us to find the full set of matrix rows that tell us whether or the application version we're about to deploy is compatible with its integrated applications. For example, given Foo v1 is a consumer of Bar, and Bar v8 is in production, and the `can-i-deploy` command is `can-i-deploy --pacticipant Foo --version 1 --to-environment production`, then an inferred unresolved selector is created for Bar in the production environment (`UnresolvedSelector.new(pacticipant_name: "Bar", environment_name: "production")`) which is then resolved to Bar v8.
+
+#### Notes on selector classes/collections
+
+Specified, ignore, and inferred selectors all start life as `UnresolvedSelector` objects, which then get resolved into `ResolvedSelector` objects.
+
+### Options
+
+TBC.
+
+## How the matrix query works
+
+1. Given that Foo v1 is a consumer of Bar, and Bar v8 is currently in production, take the pact-broker CLI command `can-i-deploy --pacticipant Foo --version 1 --to-environment production --ignore Dog`
+
+1. Turn that into a query string and make a request to the `/matrix` endpoint.
+
+1. Parse the HTTP query into the specified selectors, the ignore selectors and options.
+
+  * specified selectors - `PactBroker::Matrix::UnresolvedSelector.new(pacticipant_name: "Foo", pacticipant_version_number: "1")`
+  * ignore selectors - `PactBroker::Matrix::UnresolvedSelector.new(pacticipant_name: "Dog")`
+  * options: `{ to_environment: "production" }`
+
+1. Validate the selectors.
+
+  * Ensure conflicting fields are not used.
+  * Return an error if any of the pacticipants in the specified selectors or the environment do not exist.
+  * Do not check for the existance of version numbers or tags, or pacticipants in the ignore selectors.
+
+1. "Resolve" the ignore selectors (find the pacticipant and version IDs for the selectors).
+
+1. "Resolve" the specified selectors, passing in the ignore selectors to identify whether or not the resolved selector should be "ignored".
+
+1. Identify the integrations that the versions in the specified selectors are involved in.
+
+  * Identify the providers of the consumers: for the versions of each specified selector, find any pacts created for that version. If any exist, that means
+    the provider for each pact MUST be deployed to the target environment, and the deployed version must have a successful verification with the specified consumer version for the consumer to be safe to deploy. The providers must be identified for the specific consumer *version* of the selector, not just *consumer*, as different versions of a consumer may have different providers as integrations are created and removed over time. This is why we cannot just query the integrations table.
+
+    Represent the integration by creating a `PactBroker::Matrix::Integration` object for each consumer/provider pair, and mark it as `required: true`.
+
+  * Identify the consumers of the providers: for the pacticipant of each specified selector, find any integrations in the integrations table where
+    the specified pacticipant is the provider. Because specific provider versions don't have a dependency on any specific consumer versions being already present in an environment, we do not need to run this query at the pacticipant version level - we can just check the presence of any consumers at the integration level.
+
+    Represent the integration by creating a `PactBroker::Matrix::Integration` object for each consumer/provider pair, and mark it as `required: false`.
+
+1. Identify the inferred unresolved selectors (the selectors for the pacticipant versions that are already in the target scope)
+
+  * Collect all the pacticipant names from the integrations identified in the previous step. For every pacticipant name that does NOT already have a specified selector, create a new unresolved "inferred" selector for that pacticipant, and set the version selection attributes to be the target scope from the original `can-i-deploy` query. eg. Given we have identifed the required `Integration` for consumer Foo and provider Bar, and we are determining if we can deploy Foo to production, create an unresolved selector for Bar in the production environment (`UnresolvedSelector.new(pacticipant_name: "Bar", environment_name: "production"`).
+
+1. "Resolve" the inferred selectors (find the pacticipant and version IDs).
+
+1. Add the specified and inferred resolved selectors together to make the final collection of selectors that will be used to query the matrix.
+
+1. Peform the matrix query.
+
+  * When there are 2 or more total resolved selectors (the usual usecase):
+
+    * Create a collection of all the pacticipants in the selectors (let's call it `all_pacticipants_in_selectors`).
+
+    * Create the pact/consumer version dataset
+
+      * For each selector, find the pact publications where the consumer version is one of the versions described by the selector, and the provider is one of `all_pacticipants_in_selectors`.
+
+    * Create the verification/provider version dataset.
+
+      * For each selector, find the verifications where the provider version is one of the versions described by the selector, and the consumer is one of `all_pacticipants_in_selectors`.
+
+    * Join the pact/consumer verison dataset to the verification/provider version dataset.
+
+      * The two datasets are joined on the `pact_version_id` - this is the ID of the unique pact content that every pact publication and provider verification has a reference to. A left outer join is used, so that if there is a pact that doesn't have a verification, a row is present for the pact, with empty provider version columns. This allows us to identify that there is a missing verification.
+
+
+  * When there is only 1 total resolved selector: this is an unusual usecase, and cannot be done via the UI or the CLI, so I'm not going to spend time documenting it. Just know it is supported and theoretically possible.
+
+
+

data/docs/developer/rack.md

@@ -0,0 +1,11 @@
+# Rack
+
+https://medium.com/quick-code/rack-middleware-vs-rack-application-vs-rack-the-gem-vs-rack-the-architecture-912cd583ed24
+https://github.com/rack/rack/blob/main/SPEC.rdoc
+https://www.rubyguides.com/2018/09/rack-middleware/
+
+
+* Responds to `call`
+* Accepts a hash of parameters
+* Returns an array where the first item is the http status, the second is a hash of headers, and the third is an object that responds to `each` (or `call`) that provides the body (99% of the time it's an array of length 1 with a string)
+

data/lib/pact_broker/api/contracts/consumer_version_selector_contract.rb

@@ -13,7 +13,7 @@ module PactBroker
         json do
           optional(:mainBranch).filled(included_in?: [true])
           optional(:tag).filled(:str?)
-          optional(:branch).filled(:str?)
+          optional(:branch).filled { str? | eql?(true) }
           optional(:matchingBranch).filled(included_in?: [true])
           optional(:latest).filled(included_in?: [true, false])
           optional(:fallbackTag).filled(:str?)

data/lib/pact_broker/api/decorators/base_decorator.rb

@@ -34,7 +34,16 @@ module PactBroker
           end
         end
 
-        # Returns the names of the model associations to eager load for use with this decorator
+        # Returns the names of the model associations to eager load for use with this decorator.
+        # The default implementation attempts to do an "auto detect" of the associations.
+        # For single item decorators, it attempts to identify the attributes that are themselves models.
+        # For collection decorators, it delegates to the eager_load_associations
+        # method of the single item decorator used to decorate the collection.
+        #
+        # The "auto detect" logic can only go so far. It cannot identify when a child object needs its own
+        # child object(s) to render the attribute.
+        # This method should be overridden when the "auto detect" logic cannot identify the correct associations
+        # to load. eg VersionDecorator
         # @return [Array<Symbol>]
         def self.eager_load_associations
           if is_collection_resource?

data/lib/pact_broker/api/decorators/version_decorator.rb

@@ -17,6 +17,20 @@ module PactBroker
 
         include Timestamps
 
+        # Returns the list of associations that must be eager loaded to efficiently render a version
+        # when this decorator is used in a collection (eg. VersionsDecorator)
+        # The associations that need to be eager loaded for the VersionDecorator
+        # are hand coded
+        # @return <Array>
+        def self.eager_load_associations
+          [
+            :pacticipant,
+            :pact_publications,
+            { branch_versions: [:version, :branch_head, { branch: :pacticipant }] },
+            { tags: :head_tag }
+          ]
+        end
+
         link :self do | options |
           {
             title: "Version",

data/lib/pact_broker/api/resources/branch_versions.rb

@@ -31,7 +31,7 @@ module PactBroker
         end
 
         def versions
-          @versions ||= version_service.find_pacticipant_versions_in_reverse_order(pacticipant_name, { branch_name: identifier_from_path[:branch_name] }, pagination_options)
+          @versions ||= version_service.find_pacticipant_versions_in_reverse_order(pacticipant_name, { branch_name: identifier_from_path[:branch_name] }, pagination_options, decorator_class(:versions_decorator).eager_load_associations)
         end
 
         def policy_name

data/lib/pact_broker/api/resources/provider_pacts_for_verification.rb

@@ -15,6 +15,10 @@ module PactBroker
           [["application/hal+json", :to_json]]
         end
 
+        # TODO drop support for GET in next major version.
+        # GET was only used by the very first Ruby Pact clients that supported the 'pacts for verification'
+        # feature, until it became clear that the parameters for the request were going to get nested and complex,
+        # at which point the POST was added.
         def allowed_methods
           ["GET", "POST", "OPTIONS"]
         end
@@ -32,6 +36,7 @@ module PactBroker
           end
         end
 
+        # For this endoint, the POST is a "read" action (used for Pactflow)
         def read_methods
           super + %w{POST}
         end

data/lib/pact_broker/api/resources/versions.rb

@@ -31,7 +31,7 @@ module PactBroker
         end
 
         def versions
-          @versions ||= version_service.find_all_pacticipant_versions_in_reverse_order(pacticipant_name, pagination_options)
+          @versions ||= version_service.find_pacticipant_versions_in_reverse_order(pacticipant_name, {}, pagination_options, decorator_class(:versions_decorator).eager_load_associations)
         end
 
         def policy_name

data/lib/pact_broker/app.rb

@@ -27,6 +27,7 @@ require "pact_broker/config/basic_auth_configuration"
 require "pact_broker/api/authorization/resource_access_policy"
 require "pact_broker/api/middleware/http_debug_logs"
 require "pact_broker/application_context"
+require "pact_broker/db/advisory_lock"
 
 module PactBroker
 
@@ -101,22 +102,19 @@ module PactBroker
 
     def prepare_database
       logger.info "Database schema version is #{PactBroker::DB.version(configuration.database_connection)}"
+      lock = PactBroker::DB::AdvisoryLock.new(configuration.database_connection, :migrate, :pg_advisory_lock)
       if configuration.auto_migrate_db
-        migration_options = { allow_missing_migration_files: configuration.allow_missing_migration_files }
-        if PactBroker::DB.is_current?(configuration.database_connection, migration_options)
-          logger.info "Skipping database migrations as the latest migration has already been applied"
-        else
-          logger.info "Migrating database schema"
-          PactBroker::DB.run_migrations configuration.database_connection, migration_options
-          logger.info "Database schema version is now #{PactBroker::DB.version(configuration.database_connection)}"
+        lock.with_lock do
+          ensure_all_database_migrations_are_applied
         end
       else
         logger.info "Skipping database schema migrations as database auto migrate is disabled"
       end
 
       if configuration.auto_migrate_db_data
-        logger.info "Migrating data"
-        PactBroker::DB.run_data_migrations configuration.database_connection
+        lock.with_lock do
+          run_data_migrations
+        end
       else
         logger.info "Skipping data migrations"
       end
@@ -125,6 +123,23 @@ module PactBroker
       PactBroker::Webhooks::Service.fail_retrying_triggered_webhooks
     end
 
+    def ensure_all_database_migrations_are_applied
+      migration_options = { allow_missing_migration_files: configuration.allow_missing_migration_files }
+
+      if PactBroker::DB.is_current?(configuration.database_connection, migration_options)
+        logger.info "Skipping database migrations as the latest migration has already been applied"
+      else
+        logger.info "Migrating database schema"
+        PactBroker::DB.run_migrations(configuration.database_connection, migration_options)
+        logger.info "Database schema version is now #{PactBroker::DB.version(configuration.database_connection)}"
+      end
+    end
+
+    def run_data_migrations
+      logger.info "Migrating data"
+      PactBroker::DB.run_data_migrations(configuration.database_connection)
+    end
+
     def load_configuration_from_database
       configuration.load_from_database!
     end

data/lib/pact_broker/contracts/service.rb

@@ -35,7 +35,7 @@ module PactBroker
         version, version_notices = create_version(parsed_contracts)
         tags = create_tags(parsed_contracts, version)
         pacts, pact_notices = create_pacts(parsed_contracts, base_url)
-        update_integrations(pacts)
+        create_or_update_integrations(pacts)
         notices = version_notices + pact_notices
         ContractsPublicationResults.from_hash(
           pacticipant: version.pacticipant,
@@ -304,7 +304,11 @@ module PactBroker
         PactBroker::Api::PactBrokerUrls.triggered_webhook_logs_url(triggered_webhook, base_url)
       end
 
-      def update_integrations(pacts)
+      # Creating/updating the integrations all at once at the end of the transaction instead
+      # of one by one, as each pact is created, reduces the amount of time that
+      # a lock is held on the integrations table, therefore reducing contention
+      # and potential for deadlocks when there are many pacts being published at once.
+      def create_or_update_integrations(pacts)
         integration_service.handle_bulk_contract_data_published(pacts)
       end
 

data/lib/pact_broker/db/advisory_lock.rb

@@ -0,0 +1,58 @@
+require "pact_broker/logging"
+
+# Uses a Postgres advisory lock to ensure that a given block of code can only have ONE
+# thread in excution at a time against a given database.
+# When the database is not Postgres, the block will yield without any locks, allowing
+# this class to be used safely with other database types, but without the locking
+# functionality.
+#
+# This is a wrapper around the actual implementation code in the Sequel extension from https://github.com/yuryroot/sequel-pg_advisory_lock
+# which was copied into this codebase and modified for usage in this codebase.
+#
+# See https://www.postgresql.org/docs/16/functions-admin.html#FUNCTIONS-ADVISORY-LOCKS for docs on lock types
+#
+
+module PactBroker
+  module DB
+    class AdvisoryLock
+      include PactBroker::Logging
+
+      def initialize(database_connection, name, type = :pg_try_advisory_lock)
+        @database_connection = database_connection
+        @name = name
+        @type = type
+        @lock_obtained = false
+        register_advisory_lock if postgres?
+      end
+
+      def with_lock
+        if postgres?
+          @database_connection.with_advisory_lock(@name) do
+            logger.debug("Lock #{@name} obtained")
+            @lock_obtained = true
+            yield
+          end
+        else
+          logger.warn("Executing block without lock as this is not a Postgres database")
+          @lock_obtained = true
+          yield
+        end
+      end
+
+      def lock_obtained?
+        @lock_obtained
+      end
+
+      private
+
+      def postgres?
+        @database_connection.adapter_scheme.to_s =~ /postgres/
+      end
+
+      def register_advisory_lock
+        @database_connection.extension :pg_advisory_lock
+        @database_connection.register_advisory_lock(@name, @type)
+      end
+    end
+  end
+end

data/lib/pact_broker/domain/version.rb

@@ -4,9 +4,10 @@ require "pact_broker/versions/eager_loaders"
 
 module PactBroker
   module Domain
-    VERSION_COLUMNS = [:id, :number, :repository_ref, :pacticipant_id, :order, :created_at, :updated_at, :build_url]
+    class Version < Sequel::Model
+      VERSION_COLUMNS = Sequel::Model.db.schema(:versions).collect(&:first) - [:branch] # do not include the branch column, as we now have a branches table
+      set_dataset(Sequel::Model.db[:versions].select(*VERSION_COLUMNS.collect{ | column | Sequel.qualify(:versions, column) }))
 
-    class Version < Sequel::Model(Sequel::Model.db[:versions].select(*VERSION_COLUMNS.collect{ | column | Sequel.qualify(:versions, column) }))
       set_primary_key :id
 
       plugin :timestamps, update_on_create: true

data/lib/pact_broker/integrations/integration.rb

@@ -7,7 +7,17 @@ require "pact_broker/verifications/latest_verification_for_consumer_and_provider
 
 module PactBroker
   module Integrations
-    class Integration < Sequel::Model(Sequel::Model.db[:integrations].select(:id, :consumer_id, :provider_id, :contract_data_updated_at))
+    # The columns are explicitly specified for the Integration object so that the consumer_name and provider_name columns aren't included
+    # in the model.
+    # Those columns exist in the integrations table because the integrations table used to be an integrations view based on the
+    # pact_publications table, and those columns existed in the view.
+    # When the view was migrated to be a table (in db/migrations/20211102_create_table_temp_integrations.rb and the following migrations)
+    # the columns had to be maintained for backwards compatiblity.
+    # They are not used by the current code, however.
+    class Integration < Sequel::Model
+      INTEGRATION_COLUMNS = Sequel::Model.db.schema(:integrations).collect(&:first) - [:consumer_name, :provider_name]
+      set_dataset(Sequel::Model.db[:integrations].select(*INTEGRATION_COLUMNS))
+
       set_primary_key :id
       plugin :insert_ignore, identifying_columns: [:consumer_id, :provider_id]
       associate(:many_to_one, :consumer, :class => "PactBroker::Domain::Pacticipant", :key => :consumer_id, :primary_key => :id)

data/lib/pact_broker/integrations/repository.rb

@@ -28,6 +28,20 @@ module PactBroker
         nil
       end
 
+      # Ensure an Integration exists for each consumer/provider pair.
+      # Using SELECT ... INSERT IGNORE rather than just INSERT IGNORE so that we do not
+      # need to lock the table at all when the integrations already exist, which will
+      # be the most common use case. New integrations get created incredibly rarely.
+      # The INSERT IGNORE is used rather than just INSERT to handle race conditions
+      # when requests come in parallel.
+      # @param [Array<Object>] where each object has a consumer and a provider
+      def create_for_pacts(objects_with_consumer_and_provider)
+        published_integrations = objects_with_consumer_and_provider.collect{ |i| { consumer_id: i.consumer.id, provider_id: i.provider.id } }
+        existing_integrations = Sequel::Model.db[:integrations].select(:consumer_id, :provider_id).where(Sequel.|(*published_integrations) ).all
+        new_integrations = (published_integrations - existing_integrations).collect{ |i| i.merge(created_at: Sequel.datetime_class.now, contract_data_updated_at: Sequel.datetime_class.now) }
+        Integration.dataset.insert_ignore.multi_insert(new_integrations)
+      end
+
       def delete(consumer_id, provider_id)
         Integration.where(consumer_id: consumer_id, provider_id: provider_id).delete
       end
@@ -36,18 +50,43 @@ module PactBroker
       # @param [PactBroker::Domain::Pacticipant, nil] consumer the consumer for the integration, or nil if for a provider-only event (eg. Pactflow provider contract published)
       # @param [PactBroker::Domain::Pacticipant] provider the provider for the integration
       def set_contract_data_updated_at(consumer, provider)
-        Integration
-          .where({ consumer_id: consumer&.id, provider_id: provider.id }.compact )
-          .update(contract_data_updated_at: Sequel.datetime_class.now)
+        set_contract_data_updated_at_for_multiple_integrations([OpenStruct.new(consumer: consumer, provider: provider)])
       end
 
 
-      # Sets the contract_data_updated_at for the integrations as specified by an array of objects which each have a consumer and provider
-      # @param [Array<Object>] where each object has a consumer and a provider
+      # Sets the contract_data_updated_at for the integrations as specified by an array of objects which each have a consumer and provider.
+      #
+      # The contract_data_updated_at attribute is only ever used for ordering the list of integrations on the index page of the *Pact Broker* UI,
+      # so that the most recently updated integrations (the ones you're likely working on) are showed at the top of the first page.
+      # There is often contention around updating it however, which can cause deadlocks, and slow down API responses.
+      # Because it's not a critical field (eg. it won't change any can-i-deploy results), the easiest way to reduce this contention
+      # is to just not update it if the row is locked, because if it is locked, the value of contract_data_updated_at is already
+      # going to be a date from a few seconds ago, which is perfectly fine for the purposes for which we are using the value.
+      #
+      # Notes on SKIP LOCKED:
+      # SKIP LOCKED is only supported by Postgres.
+      # When executing SELECT ... FOR UPDATE SKIP LOCKED, the SELECT will run immediately, not waiting for any other transactions,
+      # and only return rows that are not already locked by another transaction.
+      # The FOR UPDATE is required to make it work this way - SKIP LOCKED on its own does not work.
+      #
+      # @param [Array<Object>] where each object MAY have a consumer and does have a provider (for Pactflow provider contract published there is no consumer)
       def set_contract_data_updated_at_for_multiple_integrations(objects_with_consumer_and_provider)
-        consumer_and_provider_ids = objects_with_consumer_and_provider.collect{ | object | [object.consumer.id, object.provider.id] }.uniq
+        consumer_and_provider_ids = objects_with_consumer_and_provider.collect{ | object | { consumer_id: object.consumer&.id, provider_id: object.provider.id }.compact }.uniq
+
+        # MySQL doesn't support an UPDATE with a subquery. FFS. Really need to do a major version release and delete the support code.
+        criteria =  if Integration.dataset.supports_skip_locked?
+                      integration_ids_to_update = Integration
+                                                  .select(:id)
+                                                  .where(Sequel.|(*consumer_and_provider_ids))
+                                                  .for_update
+                                                  .skip_locked
+                      { id: integration_ids_to_update }
+                    else
+                      Sequel.|(*consumer_and_provider_ids)
+                    end
+
         Integration
-          .where([:consumer_id, :provider_id] => consumer_and_provider_ids)
+          .where(criteria)
           .update(contract_data_updated_at: Sequel.datetime_class.now)
       end
     end

data/lib/pact_broker/integrations/service.rb

@@ -21,6 +21,7 @@ module PactBroker
       # @param [PactBroker::Domain::Pacticipant] consumer or nil
       # @param [PactBroker::Domain::Pacticipant] provider
       def self.handle_contract_data_published(consumer, provider)
+        integration_repository.create_for_pact(consumer.id, provider.id)
         integration_repository.set_contract_data_updated_at(consumer, provider)
       end
 
@@ -28,6 +29,7 @@ module PactBroker
       # Callback to invoke when a batch of contract data is published (eg. the publish contracts endpoint)
       # @param [Array<Object>] where each object has a consumer and a provider
       def self.handle_bulk_contract_data_published(objects_with_consumer_and_provider)
+        integration_repository.create_for_pacts(objects_with_consumer_and_provider)
         integration_repository.set_contract_data_updated_at_for_multiple_integrations(objects_with_consumer_and_provider)
       end
 

data/lib/pact_broker/pacts/pact_publication_dataset_module.rb

@@ -57,7 +57,13 @@ module PactBroker
         end
       end
 
-      # TODO use the branch heads here
+      # Returns the latest pact for each branch, returning a pact for every branch, even if
+      # the most recent version of that branch does not have a pact.
+      # This is different from for_all_branch_heads, which will find the branch head versions,
+      # and return the pacts associated with those versions.
+      # This method should not be used for 'pacts for verification', because it will return
+      # a pact for branches where that integration should no longer exist.
+      # @return [Dataset<PactBroker::Pacts::PactPublication>]
       def latest_by_consumer_branch
         branch_versions_join = {
           Sequel[:pact_publications][:consumer_version_id] => Sequel[:branch_versions][:version_id]
@@ -112,10 +118,29 @@ module PactBroker
           .limit(1)
       end
 
-      # Return the pacts (if they exist) for the branch heads.
+      # Returns the pacts (if they exist) for all the branch heads.
+      # If the version for the branch head does not have a pact, then no pact is returned,
+      # (unlike latest_by_consumer_branch)
+      # This is much more performant than latest_by_consumer_branch and should be used
+      # for the 'pacts for verification' response
+      # @return [Dataset<PactBroker::Pacts::PactPublication>]
+      def for_all_branch_heads
+        base_query = self
+        base_query = base_query.join(:branch_heads, { Sequel[:bh][:version_id] => Sequel[:pact_publications][:consumer_version_id] }, { table_alias: :bh })
+
+        if no_columns_selected?
+          base_query = base_query.select_all_qualified.select_append(Sequel[:bh][:branch_name].as(:branch_name))
+        end
+
+        base_query.remove_overridden_revisions
+      end
+
+      # Return the pacts (if they exist) for the branch heads of the given branch names
       # This uses the new logic of finding the branch head and returning any associated pacts,
       # rather than the old logic of returning the pact for the latest version
       # on the branch that had a pact.
+      # @param [String] branch_name
+      # @return [Sequel::Dataset<PactBroker::Pacts::PactPublication>]
       def for_branch_heads(branch_name)
         branch_head_join = {
           Sequel[:pact_publications][:consumer_version_id] => Sequel[:branch_heads][:version_id],
@@ -174,7 +199,10 @@ module PactBroker
       # The latest pact publication for each tag
       # This uses the old logic of "the latest pact for a version that has a tag" (which always returns a pact)
       # rather than "the pact for the latest version with a tag"
-      # Need to see about updating this.
+      #
+      # For 'pacts for verification' this has been replaced by for_all_tag_heads
+      # This should only be used for the UI
+      # @return [Sequel::Dataset<PactBroker::Pacts::PactPublication>]
       def latest_by_consumer_tag
         tags_join = {
           Sequel[:pact_publications][:consumer_version_id] => Sequel[:tags][:version_id],
@@ -203,6 +231,7 @@ module PactBroker
       # This uses the old logic of "the latest pact for a version that has a tag" (which always returns a pact)
       # rather than "the pact for the latest version with a tag"
       # Need to see about updating this.
+      # @return [Sequel::Dataset<PactBroker::Pacts::PactPublication>]
       def latest_for_consumer_tag(tag_name)
         tags_join = {
           Sequel[:pact_publications][:consumer_version_id] => Sequel[:tags][:version_id],
@@ -253,6 +282,30 @@ module PactBroker
          .remove_overridden_revisions_from_complete_query
       end
 
+      # The pacts for the latest versions for each tag.
+      # Will not return a pact if the pact is no longer published for a particular tag
+      # NEW LOGIC
+      # @return [Sequel::Dataset<PactBroker::Pacts::PactPublication>]
+      def for_all_tag_heads
+        head_tags = PactBroker::Domain::Tag
+                      .select_group(:pacticipant_id, :name)
+                      .select_append{ max(version_order).as(:latest_version_order) }
+
+        head_tags_join = {
+          Sequel[:pact_publications][:consumer_id] => Sequel[:head_tags][:pacticipant_id],
+          Sequel[:pact_publications][:consumer_version_order] => Sequel[:head_tags][:latest_version_order]
+        }
+
+        base_query = self
+        if no_columns_selected?
+          base_query = base_query.select_all_qualified.select_append(Sequel[:head_tags][:name].as(:tag_name))
+        end
+
+        base_query
+          .join(head_tags, head_tags_join, { table_alias: :head_tags })
+         .remove_overridden_revisions_from_complete_query
+      end
+
       def in_environments
         currently_deployed_join = {
           Sequel[:pact_publications][:consumer_version_id] => Sequel[:currently_deployed_version_ids][:version_id]

data/lib/pact_broker/pacts/pact_publication_selector_dataset_module.rb

@@ -15,6 +15,7 @@ module PactBroker
 
         # Do the "latest" logic last so that the provider/consumer criteria get included in the "latest" query before the join, rather than after
         query = query.latest_for_main_branches if selector.latest_for_main_branch?
+        query = query.for_all_branch_heads if selector.latest_for_each_branch?
         query = query.latest_for_consumer_branch(selector.branch) if selector.latest_for_branch?
         query = query.for_latest_consumer_versions_with_tag(selector.tag) if selector.latest_for_tag?
         query = query.overall_latest if selector.overall_latest?

data/lib/pact_broker/pacts/pacts_for_verification_repository.rb

@@ -64,7 +64,7 @@ module PactBroker
           provider_tags_names,
           wip_start_date,
           explicitly_specified_verifiable_pacts,
-          :latest_by_consumer_tag
+          :for_all_tag_heads
         )
 
         wip_by_consumer_branches = find_wip_pact_versions_for_provider_by_provider_tags(
@@ -72,7 +72,7 @@ module PactBroker
           provider_tags_names,
           wip_start_date,
           explicitly_specified_verifiable_pacts,
-          :latest_by_consumer_branch
+          :for_all_branch_heads
         )
 
         deduplicate_verifiable_pacts(wip_by_consumer_tags + wip_by_consumer_branches).sort
@@ -229,8 +229,8 @@ module PactBroker
         provider = pacticipant_repository.find_by_name(provider_name)
         wip_start_date = options.fetch(:include_wip_pacts_since)
 
-        potential_wip_by_consumer_branch = PactPublication.for_provider(provider).created_after(wip_start_date).latest_by_consumer_branch
-        potential_wip_by_consumer_tag = PactPublication.for_provider(provider).created_after(wip_start_date).latest_by_consumer_tag
+        potential_wip_by_consumer_branch = PactPublication.for_provider(provider).created_after(wip_start_date).for_all_branch_heads
+        potential_wip_by_consumer_tag = PactPublication.for_provider(provider).created_after(wip_start_date).for_all_tag_heads
 
         log_debug_for_wip do
           log_pact_publications_from_query("Potential WIP pacts for provider branch #{provider_version_branch} created after #{wip_start_date} by consumer branch", potential_wip_by_consumer_branch)

data/lib/pact_broker/pacts/repository.rb

@@ -33,8 +33,8 @@ module PactBroker
         scope
       end
 
-      def create params
-        integration_repository.create_for_pact(params.fetch(:consumer_id), params.fetch(:provider_id))
+      # @return [PactBroker::Domain::Pact]
+      def create(params)
         pact_version = find_or_create_pact_version(
           params.fetch(:consumer_id),
           params.fetch(:provider_id),

data/lib/pact_broker/pacts/selector.rb

@@ -31,6 +31,8 @@ module PactBroker
       def type
         if latest_for_main_branch?
           :latest_for_main_branch
+        elsif latest_for_each_branch?
+          :latest_for_each_branch
         elsif latest_for_branch?
           :latest_for_branch
         elsif matching_branch?
@@ -265,12 +267,16 @@ module PactBroker
       # Not sure if the fallback_tag logic is needed
       def latest_for_branch? potential_branch = nil
         if potential_branch
-          !!(latest && branch == potential_branch)
+          latest == true && branch == potential_branch
         else
-          !!(latest && !!branch)
+          latest == true && branch.is_a?(String)
         end
       end
 
+      def latest_for_each_branch?
+        latest == true && branch == true
+      end
+
       def all_for_tag_and_consumer?
         !!(tag && !latest? && consumer)
       end

data/lib/pact_broker/tasks/clean_task.rb

@@ -1,3 +1,8 @@
+# This task is used to clean up old data in a Pact Broker database
+# to stop performance issues from slowing down responses when there is
+# too much data.
+# See https://docs.pact.io/pact_broker/administration/maintenance
+
 module PactBroker
   module DB
     class CleanTask < ::Rake::TaskLib
@@ -7,11 +12,13 @@ module PactBroker
       attr_accessor :version_deletion_limit
       attr_accessor :logger
       attr_accessor :dry_run
+      attr_accessor :use_lock # allow disabling of postgres lock if it is causing problems
 
       def initialize &block
         require "pact_broker/db/clean_incremental"
         @version_deletion_limit = 1000
         @dry_run = false
+        @use_lock = true
         @keep_version_selectors = PactBroker::DB::CleanIncremental::DEFAULT_KEEP_SELECTORS
         rake_task(&block)
       end
@@ -28,42 +35,77 @@ module PactBroker
           namespace :db do
             desc "Clean unnecessary pacts and verifications from database"
             task :clean do | _t, _args |
-
               instance_eval(&block)
 
-              require "pact_broker/db/clean_incremental"
-              require "pact_broker/error"
-              require "yaml"
-              require "benchmark"
+              with_lock do
+                perform_clean
+              end
+            end
+          end
+        end
+      end
+
+      def perform_clean
+        require "pact_broker/db/clean_incremental"
+        require "pact_broker/error"
+        require "yaml"
+        require "benchmark"
 
-              raise PactBroker::Error.new("You must specify the version_deletion_limit") unless version_deletion_limit
+        raise PactBroker::Error.new("You must specify the version_deletion_limit") unless version_deletion_limit
 
-              prefix = dry_run ? "[DRY RUN] " : ""
+        if keep_version_selectors.nil? || keep_version_selectors.empty?
+          raise PactBroker::Error.new("You must specify which versions to keep")
+        else
+          add_defaults_to_keep_selectors
+          output "Deleting oldest #{version_deletion_limit} versions, keeping versions that match the configured selectors", keep_version_selectors.collect(&:to_hash)
+        end
 
-              if keep_version_selectors.nil? || keep_version_selectors.empty?
-                raise PactBroker::Error.new("You must specify which versions to keep")
-              else
-                add_defaults_to_keep_selectors
-                output "#{prefix}Deleting oldest #{version_deletion_limit} versions, keeping versions that match the configured selectors", keep_version_selectors.collect(&:to_hash)
-              end
+        start_time = Time.now
+        results = PactBroker::DB::CleanIncremental.call(database_connection,
+          keep: keep_version_selectors,
+          limit: version_deletion_limit,
+          logger: logger,
+          dry_run: dry_run
+        )
+        end_time = Time.now
+        elapsed_seconds = (end_time - start_time).to_i
+        output "Results (#{elapsed_seconds} seconds)", results
+      end
 
-              start_time = Time.now
-              results = PactBroker::DB::CleanIncremental.call(database_connection,
-                keep: keep_version_selectors,
-                limit: version_deletion_limit,
-                logger: logger,
-                dry_run: dry_run
-              )
-              end_time = Time.now
-              elapsed_seconds = (end_time - start_time).to_i
-              output "Results (#{elapsed_seconds} seconds)", results
-            end
+      # Use a Postgres advisory lock to ensure that only one clean can run at a time.
+      # This allows a cron schedule to be used on the Pact Broker Docker image when deployed
+      # on a multi-instance architecture, without all the instances stepping on each other's toes.
+      #
+      # Any tasks that attempt to run while a clean job is running will skip the clean
+      # and exit with a message and a success code.
+      #
+      # To test that the lock works, run:
+      #   script/docker/db-start.sh
+      #   script/docker/db-migrate.sh
+      #   for i in {0..3}; do PACT_BROKER_TEST_DATABASE_URL=postgres://postgres:postgres@localhost/postgres bundle exec rake pact_broker:db:clean &;  done;
+      #
+      # There will be 3 messages saying "Clean was not performed" and output from one thread showing the clean is being done.
+      def with_lock
+        if use_lock
+          require "pact_broker/db/advisory_lock"
+
+          lock = PactBroker::DB::AdvisoryLock.new(database_connection, :clean, :pg_try_advisory_lock)
+          results = lock.with_lock do
+            yield
+          end
+
+          if !lock.lock_obtained?
+            output("Clean was not performed as a clean is already in progress. Exiting.")
           end
+          results
+        else
+          yield
         end
       end
 
-      def output string, payload = {}
-        logger ? logger.info(string, payload) : puts("#{string} #{payload.to_json}")
+      def output(string, payload = {})
+        prefix = dry_run ? "[DRY RUN] " : ""
+        logger ? logger.info("#{prefix}#{string}") : puts("#{prefix}#{string} #{payload.to_json}")
       end
 
       def add_defaults_to_keep_selectors

data/lib/pact_broker/test/test_data_builder.rb

@@ -43,7 +43,6 @@ module PactBroker
       include PactBroker::Services
       using PactBroker::StringRefinements
 
-
       attr_reader :pacticipant
       attr_reader :consumer
       attr_reader :provider
@@ -192,8 +191,11 @@ module PactBroker
         self
       end
 
+      # Create an Integration object for the current consumer and provider
+      # @return [PactBroker::Test::TestDataBuilder]
       def create_integration
-        PactBroker::Integrations::Repository.new.create_for_pact(consumer.id, provider.id)
+        @integration = PactBroker::Integrations::Repository.new.create_for_pact(consumer.id, provider.id)
+        set_created_at_if_set(@now, :integrations, { consumer_id: consumer.id, provider_id: provider.id })
         self
       end
 
@@ -280,7 +282,9 @@ module PactBroker
         self
       end
 
-      def create_pact params = {}
+      # Creates a pact (and integration if one does not already exist) from the given params
+      # @return [PactBroker::Test::TestDataBuilder]
+      def create_pact(params = {})
         params.delete(:comment)
         json_content = params[:json_content] || default_json_content
         pact_version_sha = params[:pact_version_sha] || generate_pact_version_sha(json_content)
@@ -293,6 +297,7 @@ module PactBroker
           json_content: prepare_json_content(json_content),
           version: @consumer_version
         )
+        integration_service.handle_bulk_contract_data_published([@pact])
         pact_versions_count_after = PactBroker::Pacts::PactVersion.count
         set_created_at_if_set(params[:created_at], :pact_publications, id: @pact.id)
         set_created_at_if_set(params[:created_at], :pact_versions, sha: @pact.pact_version_sha) if pact_versions_count_after > pact_versions_count_before

data/lib/pact_broker/version.rb

@@ -1,3 +1,3 @@
 module PactBroker
-  VERSION = "2.109.1"
+  VERSION = "2.110.0"
 end

data/lib/pact_broker/versions/repository.rb

@@ -57,29 +57,11 @@ module PactBroker
           .single_record
       end
 
-      # The eager loaded relations are hardcoded here to support the PactBroker::Api::Decorators::VersionDecorator
-      # Newer "find all" implementations for other models pass the relations to eager load in
-      # from the decorator via the resource.
-      def find_all_pacticipant_versions_in_reverse_order name, pagination_options = {}
-        pacticipant = pacticipant_repository.find_by_name!(name)
-        query = PactBroker::Domain::Version
-                  .where(pacticipant: pacticipant)
-                  .eager(:pacticipant)
-                  .eager(branch_versions: [:version, :branch_head, { branch: :pacticipant }])
-                  .eager(tags: :head_tag)
-                  .eager(:pact_publications)
-                  .reverse_order(:order)
-        query.all_with_pagination_options(pagination_options)
-      end
-
-      def find_pacticipant_versions_in_reverse_order(pacticipant_name, options = {}, pagination_options = {})
+      def find_pacticipant_versions_in_reverse_order(pacticipant_name, options = {}, pagination_options = {}, eager_load_associations = [])
         pacticipant = pacticipant_repository.find_by_name!(pacticipant_name)
         query = PactBroker::Domain::Version
                   .where(pacticipant: pacticipant)
-                  .eager(:pacticipant)
-                  .eager(branch_versions: [:version, :branch_head, { branch: :pacticipant }])
-                  .eager(tags: :head_tag)
-                  .eager(:pact_publications)
+                  .eager(*eager_load_associations)
                   .reverse_order(:order)
 
         if options[:branch_name]

data/lib/pact_broker/versions/service.rb

@@ -26,12 +26,8 @@ module PactBroker
         version_repository.find_latest_by_pacticipant_name_and_branch_name(pacticipant_name, branch_name)
       end
 
-      def self.find_all_pacticipant_versions_in_reverse_order(name, pagination_options = {})
-        version_repository.find_all_pacticipant_versions_in_reverse_order(name, pagination_options)
-      end
-
-      def self.find_pacticipant_versions_in_reverse_order(pacticipant_name, options, pagination_options = {})
-        version_repository.find_pacticipant_versions_in_reverse_order(pacticipant_name, options, pagination_options)
+      def self.find_pacticipant_versions_in_reverse_order(pacticipant_name, options, pagination_options = {}, eager_load_associations = [])
+        version_repository.find_pacticipant_versions_in_reverse_order(pacticipant_name, options, pagination_options, eager_load_associations)
       end
 
       def self.create_or_overwrite(pacticipant_name, version_number, version)

data/lib/pact_broker/webhooks/execution.rb

@@ -2,14 +2,14 @@ require "pact_broker/dataset"
 
 module PactBroker
   module Webhooks
-    class Execution < Sequel::Model(
-      Sequel::Model.db[:webhook_executions].select(
-        Sequel[:webhook_executions][:id],
-        :triggered_webhook_id,
-        :success,
-        :logs,
-        Sequel[:webhook_executions][:created_at])
-      )
+    class Execution < Sequel::Model(:webhook_executions)
+      # Ignore the columns that were used before the TriggeredWebhook class existed.
+      # It used to go Webhook -> Execution, and now it goes Webhook -> TriggeredWebhook -> Execution
+      # If we ever release a major version where we drop unused columns, those columns could be deleted.
+      EXECUTION_COLUMNS = Sequel::Model.db.schema(:webhook_executions).collect(&:first) - [:webhook_id, :pact_publication_id, :consumer_id, :provider_id]
+
+      set_dataset(Sequel::Model.db[:webhook_executions].select(*EXECUTION_COLUMNS))
+
       set_primary_key :id
       plugin :timestamps
 

data/lib/sequel/extensions/pg_advisory_lock.rb

@@ -0,0 +1,101 @@
+# Copied with thanks from https://github.com/yuryroot/sequel-pg_advisory_lock/blob/d7509aa/lib/sequel/extensions/pg_advisory_lock.rb
+# The reason this is copy/pasted and modified is that I wanted to allow exact duplicate
+# locks to be registered because different threads running the same code
+# should not cause a Sequel::Error to be raised.
+# Also, I wanted it to use Concurrent::Hash for multi-threaded environments.
+
+require "sequel"
+require "zlib"
+require "concurrent/hash"
+
+module Sequel
+  module Postgres
+    module PgAdvisoryLock
+
+      SESSION_LEVEL_LOCKS = [
+        :pg_advisory_lock,
+        :pg_try_advisory_lock
+      ].freeze
+
+      TRANSACTION_LEVEL_LOCKS = [
+        :pg_advisory_xact_lock,
+        :pg_try_advisory_xact_lock
+      ].freeze
+
+      LOCK_FUNCTIONS = (SESSION_LEVEL_LOCKS + TRANSACTION_LEVEL_LOCKS).freeze
+
+      DEFAULT_LOCK_FUNCTION = :pg_advisory_lock
+      UNLOCK_FUNCTION = :pg_advisory_unlock
+
+      class LockAlreadyRegistered < Sequel::Error; end
+
+      def registered_advisory_locks
+        @registered_advisory_locks ||= Concurrent::Hash.new
+      end
+
+      def with_advisory_lock(name, id = nil)
+        options = registered_advisory_locks.fetch(name.to_sym)
+
+        lock_key = options.fetch(:key)
+        function_params = [lock_key, id].compact
+
+        lock_function = options.fetch(:lock_function)
+        transaction_level_lock = TRANSACTION_LEVEL_LOCKS.include?(lock_function)
+
+        if transaction_level_lock
+          # TODO: It's allowed to specify additional options (in particular, :server)
+          #       while opening database transaction.
+          #       That's why this check must be smarter.
+          unless in_transaction?
+            raise Error, "Transaction must be manually opened before using transaction level lock '#{lock_function}'"
+          end
+
+          if get(Sequel.function(lock_function, *function_params))
+            yield
+          end
+        else
+          synchronize do
+            if get(Sequel.function(lock_function, *function_params))
+              begin
+                result = yield
+              ensure
+                get(Sequel.function(UNLOCK_FUNCTION, *function_params))
+                result
+              end
+            end
+          end
+        end
+      end
+
+      # Beth: not sure how much extra value this registration provides.
+      # It turns the name into a number, and makes sure the name/number is unique,
+      # and that you don't try and use a different lock function with the same name.
+      def register_advisory_lock(name, lock_function = DEFAULT_LOCK_FUNCTION)
+        name = name.to_sym
+
+        if registered_advisory_locks.key?(name) && registered_advisory_locks[name][:lock_function] != lock_function
+          raise LockAlreadyRegistered, "Lock with name :#{name} is already registered with a different lock function (#{registered_advisory_locks[name][:lock_function]})"
+        end
+
+        key = advisory_lock_key_for(name)
+        name_for_key = registered_advisory_locks.keys.find { |n| registered_advisory_locks[n].fetch(:key) == key }
+        if name_for_key && name_for_key != name
+          raise Error, "Lock key #{key} is already taken"
+        end
+
+        function = lock_function.to_sym
+        unless LOCK_FUNCTIONS.include?(function)
+          raise Error, "Invalid lock function :#{function}"
+        end
+
+        registered_advisory_locks[name] = { key: key, lock_function: function }
+      end
+
+      def advisory_lock_key_for(lock_name)
+        Zlib.crc32(lock_name.to_s) % 2 ** 31
+      end
+    end
+  end
+
+  Database.register_extension(:pg_advisory_lock, Postgres::PgAdvisoryLock)
+end

data/pact_broker.gemspec

@@ -50,7 +50,7 @@ Gem::Specification.new do |gem|
   gem.license       = "MIT"
 
   gem.add_runtime_dependency "json", "~> 2.3"
-  gem.add_runtime_dependency "psych", "~> 4.0" # TODO identify breaking changes and see if we can use 5
+  gem.add_runtime_dependency "psych", "~> 5.0"
   gem.add_runtime_dependency "roar", "~> 1.1"
   gem.add_runtime_dependency "dry-validation", "~> 1.8"
   gem.add_runtime_dependency "reform", "~> 2.6"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pact_broker
 version: !ruby/object:Gem::Version
-  version: 2.109.1
+  version: 2.110.0
 platform: ruby
 authors:
 - Bethany Skurrie
@@ -10,7 +10,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-02-21 00:00:00.000000000 Z
+date: 2024-05-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: json
@@ -32,14 +32,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '4.0'
+        version: '5.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '4.0'
+        version: '5.0'
 - !ruby/object:Gem::Dependency
   name: roar
   requirement: !ruby/object:Gem::Requirement
@@ -582,6 +582,9 @@ files:
 - docs/api/PACTICIPANTS.md
 - docs/api/PAGINATION.md
 - docs/api/WEBHOOKS.md
+- docs/developer/design_pattern_for_eager_loading_collections.md
+- docs/developer/matrix.md
+- docs/developer/rack.md
 - lib/pact/doc/doc_file.rb
 - lib/pact/doc/generate.rb
 - lib/pact/doc/generator.rb
@@ -812,6 +815,7 @@ files:
 - lib/pact_broker/dataset/page.rb
 - lib/pact_broker/date_helper.rb
 - lib/pact_broker/db.rb
+- lib/pact_broker/db/advisory_lock.rb
 - lib/pact_broker/db/clean.rb
 - lib/pact_broker/db/clean/selector.rb
 - lib/pact_broker/db/clean_incremental.rb
@@ -1124,6 +1128,7 @@ files:
 - lib/rack/pact_broker/ui_request_filter.rb
 - lib/rack/pact_broker/use_when.rb
 - lib/semantic_logger/formatters/short.rb
+- lib/sequel/extensions/pg_advisory_lock.rb
 - lib/sequel/extensions/statement_timeout.rb
 - lib/sequel/plugins/age.rb
 - lib/sequel/plugins/insert_ignore.rb
@@ -1260,7 +1265,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.6
+rubygems_version: 3.5.10
 signing_key:
 specification_version: 4
 summary: See description