nulogy_message_bus_producer 3.3.0 → 3.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f25e6c8908723688660d57c0959e9a90148c3546a7f8015348fd610f05743711
-  data.tar.gz: 67440649f40183608436a823695d309dd7917c754aced8ac2c2e9e8d6279431f
+  metadata.gz: 53fdb223452262c8d9780b2e78fee69eb2a73d6e11d72a4041ef343813e17921
+  data.tar.gz: 25c21ecc26cf8843dfa4705a4051d653858a2ffae7ee934243bca4e7799c7713
 SHA512:
-  metadata.gz: 59e33bfcea9890adbb3d9814a7d64b22bb73c85273115a35c3f8915859e0cc382e8d3ad2f55a29e9e5ad46176a537ca7ecf187dedc483a006ab812c8103334bb
-  data.tar.gz: 3fc07ac8fc834bce6900cacbd1247ca069f9f83f678ec6865fc722e7d86d32e63718dc6306c55f046109803d9ab040fd9bf092782ac86d1a79c05d820d79f070
+  metadata.gz: c55aec7fbd31c27d479e9ae6f0f963ac074f76d159d9d7f54d07fc389092f4211a60415bdfeaa6f05114fa891f5c43b64893c732ba937255dcfb2bdfe32a70f4
+  data.tar.gz: 227c9de8ca1742b9fa9306d7e544b7df1964c09e98fffd889981ba5925742b128ba04e3b9c487d57dfced24625d9d575e866db9eec47d7d79b6bc6412302d37f

data/README.md

@@ -1,39 +1,89 @@
 # Nulogy Message Bus Producer
 
-Nulogy's code for producing to to the Message Bus
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+
+This engine contains the classes and validations for producing messages to the Message Bus.
 
 ## Installation
 
-1. Add the gem to your Gemfile `gem "nulogy_message_bus_producer"`
-2. Install the migration `rake railties:install:migrations`
+1. Add the gem to your Gemfile: `gem "nulogy_message_bus_producer"`
+2. Install the migrations: `rake railties:install:migrations`
 
 ## Usage
 
-This engine contains the classes and validations for producing messages to the Message Bus.
+Subscriptions should inherit from `NulogyMessageBusProducer::BaseSubscription`.
+
+### Subscriptions
+
+Subscriptions are created by consumers to subscribe to certain events that the producer exposes.
+
+When an event is fired/triggered, subscriptions are used to generate events into the
+`NulogyMessageBusProducer.subscription_events_table_name` table.
+
+Debezium then reads this table from the PG replication log and pushes them into Kafka.
+
+There are two ways to create subscriptions:
+
+#### Self-serve (deprecated)
+
+You create these subscriptions via the public GraphQL API. e.g.
 
-In general, as long as a subscription inherits from `NulogyMessageBusProducer::BaseSubscription`, everything
-should be set up.
+```graphql
+subscription {
+  workOrderUpdated(subscriptionId: "uuid", subscriptionGroupId: "uuid", topicName: "some-topic") {
+    workOrder {
+      id
+      code
+    }
+  }
+}
+```
+
+These subscriptions are written to the `NulogyMessageBusProducer.subscriptions_table_name` table.
+
+This style of subscriptions may be removed in a future release.
+
+#### Configured (new)
 
-Subscriptions will be written to the `NulogyMessageBusProducer.subscriptions_table_name` while events generated from the subscriptions will be written to
-`NulogyMessageBusProducer.subscription_events_table_name`. Debezium then reads this table from the PG replication log and pushes them into Kafka.
+Subscriptions provided by configuration in the producer app:
+
+```ruby
+NulogyMessageBusProducer.configure do |c|
+  c.register_schema(schema: "ExampleDomain::Schema", key: "exampleDomain")
+
+  c.add_subscription!(
+    schema: "ExampleDomain::Schema",
+    query: <<~QUERY
+      subscription {
+        domainEvent(subscriptionId: "uuid", subscriptionGroupId: "uuid2", topicName: "consumer-inbox") {
+          domainObject {
+            field
+          }
+        }
+      }
+    QUERY
+  )
+end
+```
 
 ## Configuration
 
-| Configuration                  | Default                         | Description                                                                                 |
-|--------------------------------|---------------------------------|---------------------------------------------------------------------------------------------|
-| context_for_subscription       | `->(_subscription) { {} }`        | A lambda used to inject any GraphQL Context to be used when producing subscription events |
-| producing_events_fails_with    | :raise                            | How the producer should handle errors when producing subscription events                  |
-| subscriptions_table_name       | "message_bus_subscriptions"       | The name of the table for the Subscription model                                          |
-| subscription_events_table_name | "message_bus_subscription_events" | The name of the table for the SubscriptionEvents model                                    |
+| Configuration                  | Default                           | Description                                                                                        |
+|--------------------------------|-----------------------------------|----------------------------------------------------------------------------------------------------|
+| context_for_subscription       | `->(_subscription) { {} }`        | A lambda used to inject any GraphQL Context to be used when producing subscription events          |
+| producing_events_fails_with    | :raise                            | How the producer should handle errors when producing subscription events                           |
+| subscriptions_table_name       | "message_bus_subscriptions"       | The name of the table for the Subscription model                                                   |
+| subscription_events_table_name | "message_bus_subscription_events" | The name of the table for the SubscriptionEvents model                                             |
+| add_subscription!              | [] (i.e. empty)                   | Add a subscription via configuration. It will supercede any configured via GraphQL for a given id. |
 
 ## Setup
 
 You should configure the gem with an initializer. A recommended setup:
 
 ```ruby
-# Table names are configurable (optional)
-NulogyMessageBusProducer.subscriptions_table_name = :message_bus_subscriptions
-NulogyMessageBusProducer.subscription_events_table_name = :message_bus_subscription_events
+# (optional) Table names are configurable
+# NulogyMessageBusProducer.subscriptions_table_name = :message_bus_subscriptions
+# NulogyMessageBusProducer.subscription_events_table_name = :message_bus_subscription_events
 
 # Register known schemas
 NulogyMessageBusProducer.configure do |config|

data/Rakefile

@@ -13,11 +13,9 @@ require "rspec/core/rake_task"
 
 RSpec::Core::RakeTask.new(:spec)
 
-require "rubocop/rake_task"
+require "standard/rake"
 
-RuboCop::RakeTask.new
-
-task default: [:spec, :rubocop]
+task default: %i[spec standard]
 
 require "rake/release"
 

data/db/migrate/20200611150212_create_public_subscriptions_and_events_tables.rb

@@ -5,7 +5,7 @@ class CreatePublicSubscriptionsAndEventsTables < ActiveRecord::Migration[5.2]
 
     create_table NulogyMessageBusProducer.subscriptions_table_name, id: :uuid, default: nil do |t|
       t.uuid :subscription_group_id, null: false
-      t.string :event_type, null: false, index: { name: "index_nulogy_mb_producer_subscriptions_on_event_type" }
+      t.string :event_type, null: false, index: {name: "index_nulogy_mb_producer_subscriptions_on_event_type"}
       t.string :topic_name, null: false
       t.string :query, null: false
       t.text :schema_key, null: false
@@ -19,7 +19,7 @@ class CreatePublicSubscriptionsAndEventsTables < ActiveRecord::Migration[5.2]
       t.string :topic_name, null: false
       t.uuid :tenant_id, null: false
       t.json :event_json, null: false
-      t.column :created_at, :datetime, index: { name: "index_nulogy_mb_producer_subscription_events_on_created_at" }
+      t.column :created_at, :datetime, index: {name: "index_nulogy_mb_producer_subscription_events_on_created_at"}
     end
   end
 end

data/lib/nulogy_message_bus_producer.rb

@@ -1,6 +1,7 @@
 require "graphql"
 
 require "nulogy_message_bus_producer/config"
+require "nulogy_message_bus_producer/configuration/query_parser"
 require "nulogy_message_bus_producer/version"
 require "nulogy_message_bus_producer/engine"
 
@@ -35,7 +36,7 @@ module NulogyMessageBusProducer
 
   def self.trigger_event(schema, event_type, root_object)
     schema_key = NulogyMessageBusProducer.resolve_schema_key(schema)
-    subscriptions = Subscription.where(event_type: event_type, schema_key: schema_key)
+    subscriptions = Subscriptions::Finder.new(config).for_schema_event(schema_key, event_type)
 
     subscriptions.each do |subscription|
       args = {
@@ -65,7 +66,9 @@ module NulogyMessageBusProducer
   end
 
   def self.resolve_schema_key(schema)
-    config.registered_schemas.invert.fetch(schema.name) do
+    schema_name = schema.respond_to?(:name) ? schema.name : schema
+
+    config.registered_schemas.invert.fetch(schema_name) do
       raise KeyError, <<~MESSAGE
         The schema registry did not contain an entry for the schema '#{schema.name}'.
 
@@ -80,7 +83,7 @@ module NulogyMessageBusProducer
   def self.validate_existing!
     return unless Db.exists? && Db.subscriptions_exist?
 
-    validator = NulogyMessageBusProducer::SubscriberGraphqlSchemaValidator.new
+    validator = NulogyMessageBusProducer::Subscriptions::QueryValidator.new
 
     return if validator.validate
 
@@ -139,7 +142,8 @@ require "nulogy_message_bus_producer/base_subscription"
 require "nulogy_message_bus_producer/subscription"
 require "nulogy_message_bus_producer/subscription_event"
 require "nulogy_message_bus_producer/repopulate_replication_slots"
-require "nulogy_message_bus_producer/subscriber_graphql_schema_validator"
+require "nulogy_message_bus_producer/subscriptions/finder"
 require "nulogy_message_bus_producer/subscriptions/postgres_transport"
+require "nulogy_message_bus_producer/subscriptions/query_validator"
 require "nulogy_message_bus_producer/subscriptions/no_variables"
 require "nulogy_message_bus_producer/subscriptions/risky_subscription_blocker"

data/lib/nulogy_message_bus_producer/base_subscription.rb

@@ -2,7 +2,7 @@ module NulogyMessageBusProducer
   # This base class contains the fields required to create a subscription.
   # For example, for a subscription to a model called AggregateRoot:
   #
-  #  class AggregateRootCreated < NulogyMessageBusProducer::BasePublicSubscription
+  #  class AggregateRootCreated < NulogyMessageBusProducer::BaseSubscription
   #    field :aggregate_root, Domain::Public::Types::AggregateRootType, null: true
   #  end
   class BaseSubscription < GraphQL::Schema::Subscription

data/lib/nulogy_message_bus_producer/config.rb

@@ -1,14 +1,16 @@
 module NulogyMessageBusProducer
   # Configuration for the gem
-  # This is a private class, so do not use it directly. Use the methods on NulogyMessageBusProducer.
   class Config
     FAILURE_MODES = [:raise, :soft_fail].freeze
 
     attr_reader :registered_schemas
+    attr_reader :configured_subscriptions
     attr_writer :context_for_subscription
 
     def initialize(options = {})
       @registered_schemas = {}
+      @configured_subscriptions = []
+
       producing_events_fails_with(:raise)
 
       update(options)
@@ -48,6 +50,28 @@ module NulogyMessageBusProducer
       options.each { |key, value| public_send("#{key}=", value) }
     end
 
+    def add_subscription!(schema:, query:)
+      @configured_subscriptions ||= []
+
+      schema_key = NulogyMessageBusProducer.resolve_schema_key(schema)
+      query_parser = Configuration::QueryParser.new(query)
+      subscription = NulogyMessageBusProducer::Subscription.new(
+        id: query_parser.subscription_id,
+        subscription_group_id: query_parser.subscription_group_id,
+        schema_key: schema_key,
+        event_type: query_parser.event_type,
+        topic_name: query_parser.topic,
+        query: query
+      )
+
+      if subscription.valid?
+        @configured_subscriptions << subscription
+        subscription
+      else
+        raise ArgumentError, subscription.errors.full_messages
+      end
+    end
+
     private
 
     def raise_handler(subscription_id:, context:, variables:, result:)

data/lib/nulogy_message_bus_producer/configuration/query_parser.rb

@@ -0,0 +1,71 @@
+module NulogyMessageBusProducer
+  module Configuration
+    class QueryParser
+      class ParseError < StandardError
+        attr_reader :source_error
+
+        def intialize(source_error)
+          @source_error = source_error
+        end
+      end
+
+      def initialize(query)
+        @query = query
+      end
+
+      def subscription_id
+        parse
+
+        argument("subscriptionId")
+      rescue => e
+        error("subscriptionId", e)
+      end
+
+      def subscription_group_id
+        parse
+
+        argument("subscriptionGroupId")
+      rescue => e
+        error("subscriptionGroupId", e)
+      end
+
+      def event_type
+        parse
+
+        first_selection.name
+      rescue => e
+        error("event type", e)
+      end
+
+      def topic
+        parse
+
+        argument("topicName")
+      rescue => e
+        error("topic", e)
+      end
+
+      private
+
+      def parse
+        @gql ||= GraphQL::Language::Parser.parse(@query)
+      end
+
+      def argument(field)
+        first_selection
+          .arguments.detect { |e| e.name == field }
+          .value
+      end
+
+      def first_selection
+        @gql
+          .definitions.first
+          .selections.first
+      end
+
+      def error(field, err)
+        raise ParseError.new(err), "Error extracting #{field} from the subscription"
+      end
+    end
+  end
+end

data/lib/nulogy_message_bus_producer/subscription.rb

@@ -9,13 +9,15 @@ module NulogyMessageBusProducer
       def validate_each(record, attribute, _value)
         return if record.schema_key.blank? || record.query.blank?
 
-        validator = NulogyMessageBusProducer::SubscriberGraphqlSchemaValidator.new
+        validator = NulogyMessageBusProducer::Subscriptions::QueryValidator.new
 
         validator.validate(record)
         validator.errors.each { |e| record.errors.add(attribute, e) }
       end
     end
 
+    validates :id, presence: true
+    validates :subscription_group_id, presence: true
     validates :schema_key, :event_type, presence: true
     validates :query, presence: true, valid_for_schema: true
 

data/lib/nulogy_message_bus_producer/subscriptions/finder.rb

@@ -0,0 +1,40 @@
+module NulogyMessageBusProducer
+  module Subscriptions
+    # A facade to find subscriptions.
+    #
+    # It will retrieve subscriptions from these sources:
+    # - configured subscriptions via #add_subscription!
+    # - self-serve subscriptions via GraphQL API
+    #
+    # If a subscription exists with the same id in both sources, then
+    # the configured subscription will be returned.
+    class Finder
+      def initialize(config)
+        @config = config
+      end
+
+      # Note: raises like ActiveRecord#find
+      def find(id)
+        @config.configured_subscriptions.detect { |s| s.id == id } || Subscription.find(id)
+      end
+
+      def for_schema_event(schema_key, event_type)
+        subscriptions = configured(schema_key, event_type) + self_serve(schema_key, event_type)
+
+        subscriptions.uniq(&:id)
+      end
+
+      private
+
+      def self_serve(schema_key, event_type)
+        Subscription.where(event_type: event_type, schema_key: schema_key)
+      end
+
+      def configured(schema_key, event_type)
+        @config
+          .configured_subscriptions
+          .filter { |s| s.event_type == event_type && s.schema_key == schema_key }
+      end
+    end
+  end
+end

data/lib/nulogy_message_bus_producer/subscriptions/postgres_transport.rb

@@ -24,7 +24,7 @@ module NulogyMessageBusProducer
       end
 
       def read_subscription(subscription_id)
-        subscription = Subscription.find(subscription_id)
+        subscription = find_subscription(subscription_id)
         context = NulogyMessageBusProducer.context_for_subscription(subscription)
 
         {
@@ -61,6 +61,7 @@ module NulogyMessageBusProducer
         end
       end
 
+      # TODO: how is this invoked?
       def delete_subscription(subscription_id)
         Subscription.find_by(id: subscription_id).destroy
       end
@@ -69,7 +70,7 @@ module NulogyMessageBusProducer
 
       def create_event(subscription_id, result)
         company_uuid = result.query.context.object[:company_uuid]
-        subscription = Subscription.find_by(id: subscription_id)
+        subscription = find_subscription(subscription_id)
 
         SubscriptionEvent.create_or_update(
           id: SecureRandom.uuid,
@@ -80,6 +81,12 @@ module NulogyMessageBusProducer
           topic_name: subscription.topic_name
         )
       end
+
+      def find_subscription(id)
+        Subscriptions::Finder
+          .new(NulogyMessageBusProducer.config)
+          .find(id)
+      end
     end
   end
 end

data/lib/nulogy_message_bus_producer/subscriptions/query_validator.rb

@@ -0,0 +1,47 @@
+module NulogyMessageBusProducer
+  module Subscriptions
+    # A custom validator that checks that the provided (or all) subscriptions have a query that is valid for its
+    # configured schema. Schemas must be registered with a `schema_key`, that is persisted in the database.
+    # It ties the subscription to a particular schema.
+    #
+    # This validator is run as part of an initializer as a last ditch effort to verify that the stored queries in the
+    # database are valid against the deployed schema, so that when events are generated in the system, they are always
+    # sucessfully created.
+    class QueryValidator
+      attr_reader :errors
+
+      def initialize
+        @errors = []
+      end
+
+      def validate(subscription_or_subscriptions = NulogyMessageBusProducer::Subscription.all)
+        Array(subscription_or_subscriptions).each do |subscription|
+          schema = find_schema(subscription)
+          next unless schema
+
+          gql_errors = schema.validate(subscription.query)
+          errors = gql_errors.map { |e| "#{e.message} #{display_id(subscription.id)}" }
+
+          @errors.concat(errors)
+        end
+
+        @errors.empty?
+      end
+
+      private
+
+      def find_schema(subscription)
+        NulogyMessageBusProducer.resolve_schema(subscription.schema_key) do
+          @errors << "Could not find a schema for schema_key '#{subscription.schema_key}' #{display_id(subscription.id)}"
+          nil
+        end
+      end
+
+      def display_id(id)
+        normalized = id.presence || "<new_record>"
+
+        "(id: #{normalized})"
+      end
+    end
+  end
+end