nulogy_message_bus_producer 3.2.1 → 3.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b81166f96e6dc18fa79dcd9a3d5fd56785e929866c9f6c2bf1571c3b7ca3965d
-  data.tar.gz: e0d7c148e751180d9f336146a17cb56b343d9459af5f436a452a4af4649904d3
+  metadata.gz: 42a5548fe112238674907a4b18853f42dcabbda48fb49dd711b824850b468c09
+  data.tar.gz: 7a33a695535255230f8ac46c6ddc75a0a343d8936d59f4b72e5d020b3b11b25a
 SHA512:
-  metadata.gz: 9aaf43cc6e873323874a31baa272fcb00ac8b4792b9bead305510ea3f52ec236e5654c3bdcbf5630a3af1466530e434e20958d4134cf75df5cbaf8c16d8a4416
-  data.tar.gz: 33e75bf74472c78810a433036a9658736e9f4b6cb8d506092ffb631bd3a9092ed914be9bb5196783e83d95dcfbabb34b8d45cbdb964154121f95f5caa8f75f14
+  metadata.gz: acf3750d5d987a1ecbec10258ea543095bd65b5a9e878eb7c73cc1e893a12e4ca14d13d3f3de0af97cc6796877f3995c11c9f99675f2b7437e4b91c0903f7de2
+  data.tar.gz: bc421953022e2682d89388d5b3ee02b4ee4194d9eba044b8b6ead775ebd166faee40927b2abe393904d26bcaa861eceae464c407a85541f1de5750227f6932e3

data/README.md

@@ -1,39 +1,93 @@
 # Nulogy Message Bus Producer
 
-Nulogy's code for producing to to the Message Bus
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+
+This engine contains the classes and validations for producing messages to the Message Bus.
 
 ## Installation
 
-1. Add the gem to your Gemfile `gem "nulogy_message_bus_producer"`
-2. Install the migration `rake railties:install:migrations`
+1. Add the gem to your Gemfile: `gem "nulogy_message_bus_producer"`
+2. Install the migrations: `rake railties:install:migrations`
+3. Use the [Nulogy GraphQL API](https://github.com/nulogy/nulogy_graphql_api)'s schema specs to [ensure schema changes are safe](https://github.com/nulogy/nulogy_graphql_api#schema-generation)
 
 ## Usage
 
-This engine contains the classes and validations for producing messages to the Message Bus.
+Subscriptions should inherit from `NulogyMessageBusProducer::BaseSubscription`.
+
+### Subscriptions
+
+Subscriptions are created by consumers to subscribe to certain events that the producer exposes.
+
+When an event is fired/triggered, subscriptions are used to generate events into the
+`NulogyMessageBusProducer.subscription_events_table_name` table.
+
+Debezium then reads this table from the PG replication log and pushes them into Kafka.
 
-In general, as long as a subscription inherits from `NulogyMessageBusProducer::BaseSubscription`, everything
-should be set up.
+There are two ways to create subscriptions:
 
-Subscriptions will be written to the `NulogyMessageBusProducer.subscriptions_table_name` while events generated from the subscriptions will be written to
-`NulogyMessageBusProducer.subscription_events_table_name`. Debezium then reads this table from the PG replication log and pushes them into Kafka.
+#### Configured
+
+Subscriptions provided by configuration in the producer app:
+
+```ruby
+NulogyMessageBusProducer.configure do |c|
+  c.register_schema(schema: "ExampleDomain::Schema", key: "exampleDomain")
+
+  c.add_subscription!(
+    schema: "ExampleDomain::Schema",
+    query: <<~QUERY
+      subscription {
+        domainEvent(subscriptionId: "uuid", subscriptionGroupId: "uuid2", topicName: "consumer-inbox") {
+          domainObject {
+            field
+          }
+        }
+      }
+    QUERY
+  )
+end
+```
+
+#### Self-Serve (soft-deprecated)
+
+You create these subscriptions via the public GraphQL API. For testing, the [Insomnia](https://insomnia.rest/) client
+is highly recommended.
+
+```graphql
+subscription {
+  workOrderUpdated(subscriptionId: "uuid", subscriptionGroupId: "uuid", topicName: "some-topic") {
+    workOrder {
+      id
+      code
+    }
+  }
+}
+```
+
+These subscriptions are written to the `NulogyMessageBusProducer.subscriptions_table_name` table.
+
+This style of subscriptions may be removed in a future release.
+
+There is a [guide to migrate from Self-Serve to Configured](docs/self_serve_to_configured_subscriptions.md)
 
 ## Configuration
 
-| Configuration                  | Default                         | Description                                                                                 |
-|--------------------------------|---------------------------------|---------------------------------------------------------------------------------------------|
-| context_for_subscription       | `->(_subscription) { {} }`        | A lambda used to inject any GraphQL Context to be used when producing subscription events |
-| producing_events_fails_with    | :raise                            | How the producer should handle errors when producing subscription events                  |
-| subscriptions_table_name       | "message_bus_subscriptions"       | The name of the table for the Subscription model                                          |
-| subscription_events_table_name | "message_bus_subscription_events" | The name of the table for the SubscriptionEvents model                                    |
+| Configuration                  | Default                           | Description                                                                                        |
+|--------------------------------|-----------------------------------|----------------------------------------------------------------------------------------------------|
+| context_for_subscription       | `->(_subscription) { {} }`        | A lambda used to inject any GraphQL Context to be used when producing subscription events          |
+| producing_events_fails_with    | :raise                            | How the producer should handle errors when producing subscription events                           |
+| subscriptions_table_name       | "message_bus_subscriptions"       | The name of the table for the SelfServeSubscription model                                          |
+| subscription_events_table_name | "message_bus_subscription_events" | The name of the table for the SubscriptionEvents model                                             |
+| add_subscription!              | [] (i.e. empty)                   | Add a subscription via configuration. It will supercede any configured via GraphQL for a given id. |
 
 ## Setup
 
 You should configure the gem with an initializer. A recommended setup:
 
 ```ruby
-# Table names are configurable (optional)
-NulogyMessageBusProducer.subscriptions_table_name = :message_bus_subscriptions
-NulogyMessageBusProducer.subscription_events_table_name = :message_bus_subscription_events
+# (optional) Table names are configurable
+# NulogyMessageBusProducer.subscriptions_table_name = :message_bus_subscriptions
+# NulogyMessageBusProducer.subscription_events_table_name = :message_bus_subscription_events
 
 # Register known schemas
 NulogyMessageBusProducer.configure do |config|
@@ -79,11 +133,6 @@ module ExampleDomainPublicApi
   class Schema < GraphQL::Schema
     use NulogyMessageBusProducer::Subscriptions::PostgresTransport
 
-    # You must use the Interpreter, as it will be the new default.
-    # See more here: https://graphql-ruby.org/queries/interpreter
-    use GraphQL::Analysis::AST
-    use GraphQL::Execution::Interpreter
-
     # This prevents subscriptions which contain arguments or expand lists form being registered.
     # Expanding unbounded lists inside of an event may severely impact the performance of the system.
     # Arguments are a source of variation that may not be appropriately tested.
@@ -223,3 +272,14 @@ docker-compose up
 rake db:setup
 rake
 ```
+
+# Gem Releases
+
+You must be listed as a gem owner [on rubygems](https://rubygems.org/gems/nulogy_message_bus_producer) to cut a release:
+
+* Merge your changes to `master`
+* Commit a version bump
+  * Update `lib/nulogy_message_bus_producer/version.rb`
+  * Run `bundle install`
+  * Update the CHANGELOG
+* Run `bundle exec rake release`

data/Rakefile

@@ -13,11 +13,9 @@ require "rspec/core/rake_task"
 
 RSpec::Core::RakeTask.new(:spec)
 
-require "rubocop/rake_task"
+require "standard/rake"
 
-RuboCop::RakeTask.new
-
-task default: [:spec, :rubocop]
+task default: %i[spec standard]
 
 require "rake/release"
 

data/db/migrate/20200611150212_create_public_subscriptions_and_events_tables.rb

@@ -5,7 +5,7 @@ class CreatePublicSubscriptionsAndEventsTables < ActiveRecord::Migration[5.2]
 
     create_table NulogyMessageBusProducer.subscriptions_table_name, id: :uuid, default: nil do |t|
       t.uuid :subscription_group_id, null: false
-      t.string :event_type, null: false, index: { name: "index_nulogy_mb_producer_subscriptions_on_event_type" }
+      t.string :event_type, null: false, index: {name: "index_nulogy_mb_producer_subscriptions_on_event_type"}
       t.string :topic_name, null: false
       t.string :query, null: false
       t.text :schema_key, null: false
@@ -19,7 +19,7 @@ class CreatePublicSubscriptionsAndEventsTables < ActiveRecord::Migration[5.2]
       t.string :topic_name, null: false
       t.uuid :tenant_id, null: false
       t.json :event_json, null: false
-      t.column :created_at, :datetime, index: { name: "index_nulogy_mb_producer_subscription_events_on_created_at" }
+      t.column :created_at, :datetime, index: {name: "index_nulogy_mb_producer_subscription_events_on_created_at"}
     end
   end
 end

data/lib/nulogy_message_bus_producer.rb

@@ -1,6 +1,8 @@
 require "graphql"
+require "pp"
 
 require "nulogy_message_bus_producer/config"
+require "nulogy_message_bus_producer/configuration/query_parser"
 require "nulogy_message_bus_producer/version"
 require "nulogy_message_bus_producer/engine"
 
@@ -14,11 +16,11 @@ module NulogyMessageBusProducer
   end
 
   def self.subscriptions_table_name=(table_name)
-    NulogyMessageBusProducer::Subscription.table_name = table_name
+    NulogyMessageBusProducer::SelfServeSubscription.table_name = table_name
   end
 
   def self.subscriptions_table_name
-    NulogyMessageBusProducer::Subscription.table_name
+    NulogyMessageBusProducer::SelfServeSubscription.table_name
   end
 
   def self.subscription_events_table_name=(table_name)
@@ -35,7 +37,7 @@ module NulogyMessageBusProducer
 
   def self.trigger_event(schema, event_type, root_object)
     schema_key = NulogyMessageBusProducer.resolve_schema_key(schema)
-    subscriptions = Subscription.where(event_type: event_type, schema_key: schema_key)
+    subscriptions = Subscriptions::Finder.new(config).for_schema_event(schema_key, event_type)
 
     subscriptions.each do |subscription|
       args = {
@@ -65,7 +67,9 @@ module NulogyMessageBusProducer
   end
 
   def self.resolve_schema_key(schema)
-    config.registered_schemas.invert.fetch(schema.name) do
+    schema_name = schema.respond_to?(:name) ? schema.name : schema
+
+    config.registered_schemas.invert.fetch(schema_name) do
       raise KeyError, <<~MESSAGE
         The schema registry did not contain an entry for the schema '#{schema.name}'.
 
@@ -80,7 +84,7 @@ module NulogyMessageBusProducer
   def self.validate_existing!
     return unless Db.exists? && Db.subscriptions_exist?
 
-    validator = NulogyMessageBusProducer::SubscriberGraphqlSchemaValidator.new
+    validator = NulogyMessageBusProducer::Subscriptions::QueryValidator.new
 
     return if validator.validate
 
@@ -136,10 +140,13 @@ end
 
 require "nulogy_message_bus_producer/application_record"
 require "nulogy_message_bus_producer/base_subscription"
-require "nulogy_message_bus_producer/subscription"
 require "nulogy_message_bus_producer/subscription_event"
 require "nulogy_message_bus_producer/repopulate_replication_slots"
-require "nulogy_message_bus_producer/subscriber_graphql_schema_validator"
-require "nulogy_message_bus_producer/subscriptions/postgres_transport"
+require "nulogy_message_bus_producer/subscriptions/finder"
 require "nulogy_message_bus_producer/subscriptions/no_variables"
+require "nulogy_message_bus_producer/subscriptions/postgres_transport"
+require "nulogy_message_bus_producer/subscriptions/query_validator"
+require "nulogy_message_bus_producer/subscriptions/valid_for_schema_validator"
+require "nulogy_message_bus_producer/subscriptions/configured_subscription"
+require "nulogy_message_bus_producer/self_serve_subscription"
 require "nulogy_message_bus_producer/subscriptions/risky_subscription_blocker"

data/lib/nulogy_message_bus_producer/base_subscription.rb

@@ -2,7 +2,7 @@ module NulogyMessageBusProducer
   # This base class contains the fields required to create a subscription.
   # For example, for a subscription to a model called AggregateRoot:
   #
-  #  class AggregateRootCreated < NulogyMessageBusProducer::BasePublicSubscription
+  #  class AggregateRootCreated < NulogyMessageBusProducer::BaseSubscription
   #    field :aggregate_root, Domain::Public::Types::AggregateRootType, null: true
   #  end
   class BaseSubscription < GraphQL::Schema::Subscription

data/lib/nulogy_message_bus_producer/config.rb

@@ -1,14 +1,16 @@
 module NulogyMessageBusProducer
   # Configuration for the gem
-  # This is a private class, so do not use it directly. Use the methods on NulogyMessageBusProducer.
   class Config
     FAILURE_MODES = [:raise, :soft_fail].freeze
 
     attr_reader :registered_schemas
+    attr_reader :configured_subscriptions
     attr_writer :context_for_subscription
 
     def initialize(options = {})
       @registered_schemas = {}
+      @configured_subscriptions = []
+
       producing_events_fails_with(:raise)
 
       update(options)
@@ -48,6 +50,28 @@ module NulogyMessageBusProducer
       options.each { |key, value| public_send("#{key}=", value) }
     end
 
+    def add_subscription!(schema:, query:)
+      @configured_subscriptions ||= []
+
+      schema_key = NulogyMessageBusProducer.resolve_schema_key(schema)
+      query_parser = Configuration::QueryParser.new(query)
+      subscription = Subscriptions::ConfiguredSubscription.new(
+        id: query_parser.subscription_id,
+        subscription_group_id: query_parser.subscription_group_id,
+        schema_key: schema_key,
+        event_type: query_parser.event_type,
+        topic_name: query_parser.topic,
+        query: query
+      )
+
+      if subscription.valid?
+        @configured_subscriptions << subscription
+        subscription
+      else
+        raise ArgumentError, subscription.errors.full_messages
+      end
+    end
+
     private
 
     def raise_handler(subscription_id:, context:, variables:, result:)

data/lib/nulogy_message_bus_producer/configuration/query_parser.rb

@@ -0,0 +1,71 @@
+module NulogyMessageBusProducer
+  module Configuration
+    class QueryParser
+      class ParseError < StandardError
+        attr_reader :source_error
+
+        def intialize(source_error)
+          @source_error = source_error
+        end
+      end
+
+      def initialize(query)
+        @query = query
+      end
+
+      def subscription_id
+        parse
+
+        argument("subscriptionId")
+      rescue => e
+        error("subscriptionId", e)
+      end
+
+      def subscription_group_id
+        parse
+
+        argument("subscriptionGroupId")
+      rescue => e
+        error("subscriptionGroupId", e)
+      end
+
+      def event_type
+        parse
+
+        first_selection.name
+      rescue => e
+        error("event type", e)
+      end
+
+      def topic
+        parse
+
+        argument("topicName")
+      rescue => e
+        error("topic", e)
+      end
+
+      private
+
+      def parse
+        @gql ||= GraphQL::Language::Parser.parse(@query)
+      end
+
+      def argument(field)
+        first_selection
+          .arguments.detect { |e| e.name == field }
+          .value
+      end
+
+      def first_selection
+        @gql
+          .definitions.first
+          .selections.first
+      end
+
+      def error(field, err)
+        raise ParseError.new(err), "Error extracting #{field} from the subscription"
+      end
+    end
+  end
+end

data/lib/nulogy_message_bus_producer/self_serve_subscription.rb

@@ -0,0 +1,18 @@
+module NulogyMessageBusProducer
+  # This model saves self-serve subscriptions to external systems.
+  # An external system can subscribe to events and specify the shape of data it would like to receive for the event.
+  class SelfServeSubscription < ApplicationRecord
+    self.table_name = :message_bus_subscriptions
+
+    validates :id, presence: true
+    validates :subscription_group_id, presence: true
+    validates :schema_key, :event_type, presence: true
+    validates :query, :presence => true, "nulogy_message_bus_producer/subscriptions/valid_for_schema" => true
+
+    def self.create_or_update(attrs)
+      find_or_initialize_by(id: attrs[:id]).tap do |model|
+        model.update!(attrs)
+      end
+    end
+  end
+end

data/lib/nulogy_message_bus_producer/subscriptions/configured_subscription.rb

@@ -0,0 +1,14 @@
+module NulogyMessageBusProducer
+  module Subscriptions
+    class ConfiguredSubscription
+      include ActiveModel::Model
+
+      attr_accessor :id, :subscription_group_id, :schema_key, :event_type, :query, :topic_name
+
+      validates :id, presence: true
+      validates :subscription_group_id, presence: true
+      validates :schema_key, :event_type, presence: true
+      validates :query, :presence => true, "nulogy_message_bus_producer/subscriptions/valid_for_schema" => true
+    end
+  end
+end

data/lib/nulogy_message_bus_producer/subscriptions/finder.rb

@@ -0,0 +1,40 @@
+module NulogyMessageBusProducer
+  module Subscriptions
+    # A facade to find subscriptions.
+    #
+    # It will retrieve subscriptions from these sources:
+    # - configured subscriptions via #add_subscription!
+    # - self-serve subscriptions via GraphQL API
+    #
+    # If a subscription exists with the same id in both sources, then
+    # the configured subscription will be returned.
+    class Finder
+      def initialize(config)
+        @config = config
+      end
+
+      # Note: raises like ActiveRecord#find
+      def find(id)
+        @config.configured_subscriptions.detect { |s| s.id == id } || SelfServeSubscription.find(id)
+      end
+
+      def for_schema_event(schema_key, event_type)
+        subscriptions = configured(schema_key, event_type) + self_serve(schema_key, event_type)
+
+        subscriptions.uniq(&:id)
+      end
+
+      private
+
+      def self_serve(schema_key, event_type)
+        SelfServeSubscription.where(event_type: event_type, schema_key: schema_key)
+      end
+
+      def configured(schema_key, event_type)
+        @config
+          .configured_subscriptions
+          .filter { |s| s.event_type == event_type && s.schema_key == schema_key }
+      end
+    end
+  end
+end