nulogy_message_bus_producer 3.2.0 → 4.0.0.alpha

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 54bcc4198672edd1d3366678b06118df437c95267fd274f1df2969d5e029abff
-  data.tar.gz: 5c4a2d3dd70682348e383a0398e48944845512fcb6a68478278e76b5f5ac3153
+  metadata.gz: ef6259b620567209e608c461298813e61646f25f8c7aa5adc10e4fa967069ed6
+  data.tar.gz: 61a9f10a879d6b2b05d25c1b3ad177f23948dcf79dd9a89766df29cc8f1c3bf9
 SHA512:
-  metadata.gz: 04707ec3c4da493d52a4d817da040a88fb32dc3017cb549098e7a1d38d47a8ba24a34d707382c7391822fe640c0bd2792f540acb826da8eb4300a5e3482caaca
-  data.tar.gz: a27eabba49d13fabae89628f1210232b7f46ce28c41514e1fc86d3ac344af8e8c650517dd76c7141a290f625474adcf34bb1ff11209a542d49d34194c5b08bcc
+  metadata.gz: d2b2a6558e97fe79eae7624d8bdaefc585b1a8747cd506cfde6d7757eea01d8f5b7153877efcdbbd424d5d4776a5c79cdfb3a61fdf3144e385a9c2010b7b88b0
+  data.tar.gz: 96f659ac685cef6807855baa4b3a4e633b0340f176fec2ac37dbf6c2cefbf20ec0f2d29a2b74196518cf6ee571bae5f56ffe08117cf86d72197125588ed14f8c

data/README.md

@@ -1,39 +1,183 @@
 # Nulogy Message Bus Producer
 
-Nulogy's code for producing to to the Message Bus
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/testdouble/standard)
+
+This engine contains the classes and validations for producing messages to the Message Bus.
 
 ## Installation
 
-1. Add the gem to your Gemfile `gem "nulogy_message_bus_producer"`
-2. Install the migration `rake railties:install:migrations`
+1. Add the gem to your Gemfile: `gem "nulogy_message_bus_producer"`
+2. Install the migrations: `rake railties:install:migrations`
+3. Use the [Nulogy GraphQL API](https://github.com/nulogy/nulogy_graphql_api)'s schema specs to [ensure schema changes are safe](https://github.com/nulogy/nulogy_graphql_api#schema-generation)
+
+## Overview
+
+The Producer gem supports two ways of producing messages:
+
+### Subscription
+
+A subscription (either self-serve or configured) is triggered via a domain event.
+The subscription `event_data` (TODO: column should also be renamed) will render the query against a public GraphQL Schema.
+
+### Publication
+
+aka the pub half of `pubsub`.
+
+A client publishes arbitrary `event_data` (serializable to JSON) to a `topic` under a `event_type`.
+
+### Comparison
+
+| Attribute             | Subscription                                         | Publication              |
+|-----------------------|------------------------------------------------------|--------------------------|
+| subscription_id       | consumer supplied                                    | random                   |
+| subscription_group_id | consumer supplied                                    | random                   |
+| topic                 | consumer supplied                                    | explicit (user supplied) |
+| type                  | computed (domain event / GraphQL subscription field) | explicit (user supplied) |
+| message ordering      | per `company_uuid` per `subscription_group_id`       | per `company_uuid`       |
 
 ## Usage
 
-This engine contains the classes and validations for producing messages to the Message Bus.
+### Subscriptions
+
+Subscriptions should inherit from `NulogyMessageBusProducer::BaseSubscription`.
+
+Subscriptions are created by consumers to subscribe to certain events that the producer exposes.
+
+When an event is fired/triggered, subscriptions are used to generate events into the
+`NulogyMessageBusProducer.subscription_events_table_name` table.
+
+Debezium then reads this table from the PG replication log and pushes them into Kafka.
+
+There are two ways to create subscriptions:
+
+#### Configured
+
+Subscriptions provided by configuration in the producer app:
+
+```ruby
+NulogyMessageBusProducer.configure do |c|
+  c.register_schema(schema: "ExampleDomain::Schema", key: "exampleDomain")
+
+  c.add_subscription!(
+    schema: "ExampleDomain::Schema",
+    query: <<~QUERY
+      subscription {
+        domainEvent(subscriptionId: "uuid", subscriptionGroupId: "uuid2", topicName: "consumer-inbox") {
+          domainObject {
+            field
+          }
+        }
+      }
+    QUERY
+  )
+end
+```
+
+These subscriptions are stored in the producer and require a deploy to update.
+
+#### Self-Serve (deprecated)
+
+You create these subscriptions via the public GraphQL API. e.g.
+
+```graphql
+subscription {
+  workOrderUpdated(subscriptionId: "uuid", subscriptionGroupId: "uuid", topicName: "some-topic") {
+    workOrder {
+      id
+      code
+    }
+  }
+}
+```
 
-In general, as long as a subscription inherits from `NulogyMessageBusProducer::BaseSubscription`, everything
-should be set up.
+These subscriptions are written to the `NulogyMessageBusProducer.subscriptions_table_name` table.
 
-Subscriptions will be written to the `NulogyMessageBusProducer.subscriptions_table_name` while events generated from the subscriptions will be written to
-`NulogyMessageBusProducer.subscription_events_table_name`. Debezium then reads this table from the PG replication log and pushes them into Kafka.
+This style of subscriptions may be removed in a future release.
+
+There is a [guide to migrate from Self-Serve to Configured](docs/self_serve_to_configured_subscriptions.md)
+
+### Publications
+
+The Message Bus has a unified `SubscriptionEvent` (TODO: rename) data structure that will create a Kafka message.
+Thus, the publication-style of producing messages must provide any required fields to produce a valid `SubscriptionEvent`.
+
+```ruby
+# publish
+# NulogyMessageBusProducer.publish(topic:, type:, company_uuid:, data:)
+
+NulogyMessageBusProducer.publish(
+  topic: "some-target-inbox",
+  type: "some-event-type",
+  company_uuid: company.uuid,
+  data: {message: "some data", value: "3", ...}
+)
+```
+
+Note that we do not support publishing to multiple topics at the moment.
+
+A producer may want to wrap these methods:
+
+```ruby
+class MetricsPublisher
+  def initialize(company, type: "some-kind-of-metric", topic: "metrics-stream")
+    @company_uuid = company.uuid
+    @topic = topic
+    @type = type
+  end
+
+  def publish(data)
+    NulogyMessageBusProducer.publish(
+      topic: @topic,
+      type: @type,
+      company_uuid: @company_uuid,
+      data: {message:"something", value: 3}
+    )
+  end
+end
+
+MetricsPublisher.new(Current.company).publish(metric_data)
+```
+
+#### Self-Serve (soft-deprecated)
+
+You create these subscriptions via the public GraphQL API. For testing, the [Insomnia](https://insomnia.rest/) client
+is highly recommended.
+
+```graphql
+subscription {
+  workOrderUpdated(subscriptionId: "uuid", subscriptionGroupId: "uuid", topicName: "some-topic") {
+    workOrder {
+      id
+      code
+    }
+  }
+}
+```
+
+These subscriptions are written to the `NulogyMessageBusProducer.subscriptions_table_name` table.
+
+This style of subscriptions may be removed in a future release.
+
+There is a [guide to migrate from Self-Serve to Configured](docs/self_serve_to_configured_subscriptions.md)
 
 ## Configuration
 
-| Configuration                  | Default                         | Description                                                                                 |
-|--------------------------------|---------------------------------|---------------------------------------------------------------------------------------------|
-| context_for_subscription       | `->(_subscription) { {} }`        | A lambda used to inject any GraphQL Context to be used when producing subscription events |
-| producing_events_fails_with    | :raise                            | How the producer should handle errors when producing subscription events                  |
-| subscriptions_table_name       | "message_bus_subscriptions"       | The name of the table for the Subscription model                                          |
-| subscription_events_table_name | "message_bus_subscription_events" | The name of the table for the SubscriptionEvents model                                    |
+| Configuration                  | Default                           | Description                                                                                        |
+|--------------------------------|-----------------------------------|----------------------------------------------------------------------------------------------------|
+| context_for_subscription       | `->(_subscription) { {} }`        | A lambda used to inject any GraphQL Context to be used when producing subscription events          |
+| producing_events_fails_with    | :raise                            | How the producer should handle errors when producing subscription events                           |
+| subscriptions_table_name       | "message_bus_subscriptions"       | The name of the table for the SelfServeSubscription model                                          |
+| subscription_events_table_name | "message_bus_subscription_events" | The name of the table for the SubscriptionEvents model                                             |
+| add_subscription!              | [] (i.e. empty)                   | Add a subscription via configuration. It will supercede any configured via GraphQL for a given id. |
 
 ## Setup
 
 You should configure the gem with an initializer. A recommended setup:
 
 ```ruby
-# Table names are configurable (optional)
-NulogyMessageBusProducer.subscriptions_table_name = :message_bus_subscriptions
-NulogyMessageBusProducer.subscription_events_table_name = :message_bus_subscription_events
+# (optional) Table names are configurable
+# NulogyMessageBusProducer.subscriptions_table_name = :message_bus_subscriptions
+# NulogyMessageBusProducer.subscription_events_table_name = :message_bus_subscription_events
 
 # Register known schemas
 NulogyMessageBusProducer.configure do |config|
@@ -90,6 +234,11 @@ module ExampleDomainPublicApi
     # For now, these are banned until we have a use-case which would require them.
     query_analyzer NulogyMessageBusProducer::Subscriptions::RiskySubscriptionBlocker
 
+    # This prevents creating subscriptions with variables.
+    # Since variables are not saved with the query,
+    # the Subscription would always fail when generating events.
+    query_analyzer NulogyMessageBusProducer::Subscriptions::NoVariables
+
     query Types::QueryType
     subscription Types::SubscriptionType
   end
@@ -171,7 +320,7 @@ class CreateSku
     if sku.save
       # This needs to be invoked within the same transaction as Sku.save
       # Otherwise, the sku could be saved without the event being triggered
-      trigger_subscription(:sku_created, sku)
+      trigger_subscription("skuCreated", sku)
     end
   end
 
@@ -204,17 +353,29 @@ rails message_bus_producer:repopulate_replication_slots
 
 ## Rake Tasks
 
-| Rake command | Description     |
-|--------------|-----------------|
-| (default)    | RSpec + RuboCop |
-| `spec`       | RSpec           |
-| `rubocop`    | RuboCop         |
+| Rake command     | Description      |
+|------------------|------------------|
+| (default)        | RSpec + Standard |
+| `spec`           | RSpec            |
+| `standardrb`     | Standard         |
+| `standardrb:fix` | Fix to Standard  |
 
 # Gem Development Setup
 
 If you are doing work on this gem you will want to:
 ```shell script
 docker-compose up
-rake db:setup
-rake
+bundle exec rake db:setup
+bundle exec rake
 ```
+
+# Gem Releases
+
+You must be listed as a gem owner [on rubygems](https://rubygems.org/gems/nulogy_message_bus_producer) to cut a release:
+
+* Merge your changes to `master`
+* Commit a version bump
+  * Update `lib/nulogy_message_bus_producer/version.rb`
+  * Run `bundle install`
+  * Update the CHANGELOG
+* Run `bundle exec rake release`

data/Rakefile

@@ -13,11 +13,9 @@ require "rspec/core/rake_task"
 
 RSpec::Core::RakeTask.new(:spec)
 
-require "rubocop/rake_task"
+require "standard/rake"
 
-RuboCop::RakeTask.new
-
-task default: [:spec, :rubocop]
+task default: %i[spec standard]
 
 require "rake/release"
 

data/db/migrate/20200611150212_create_public_subscriptions_and_events_tables.rb

@@ -5,7 +5,7 @@ class CreatePublicSubscriptionsAndEventsTables < ActiveRecord::Migration[5.2]
 
     create_table NulogyMessageBusProducer.subscriptions_table_name, id: :uuid, default: nil do |t|
       t.uuid :subscription_group_id, null: false
-      t.string :event_type, null: false, index: { name: "index_nulogy_mb_producer_subscriptions_on_event_type" }
+      t.string :event_type, null: false, index: {name: "index_nulogy_mb_producer_subscriptions_on_event_type"}
       t.string :topic_name, null: false
       t.string :query, null: false
       t.text :schema_key, null: false
@@ -19,7 +19,7 @@ class CreatePublicSubscriptionsAndEventsTables < ActiveRecord::Migration[5.2]
       t.string :topic_name, null: false
       t.uuid :tenant_id, null: false
       t.json :event_json, null: false
-      t.column :created_at, :datetime, index: { name: "index_nulogy_mb_producer_subscription_events_on_created_at" }
+      t.column :created_at, :datetime, index: {name: "index_nulogy_mb_producer_subscription_events_on_created_at"}
     end
   end
 end

data/db/migrate/20210330204121_add_event_type_to_subscription_events.rb

@@ -0,0 +1,9 @@
+class AddEventTypeToSubscriptionEvents < ActiveRecord::Migration[5.2]
+  def up
+    add_column :message_bus_subscription_events, :event_type, :text
+  end
+
+  def down
+    remove_column :message_bus_subscription_events, :event_type
+  end
+end

data/lib/nulogy_message_bus_producer.rb

@@ -1,6 +1,7 @@
 require "graphql"
 
 require "nulogy_message_bus_producer/config"
+require "nulogy_message_bus_producer/configuration/query_parser"
 require "nulogy_message_bus_producer/version"
 require "nulogy_message_bus_producer/engine"
 
@@ -14,11 +15,11 @@ module NulogyMessageBusProducer
   end
 
   def self.subscriptions_table_name=(table_name)
-    NulogyMessageBusProducer::Subscription.table_name = table_name
+    NulogyMessageBusProducer::SelfServeSubscription.table_name = table_name
   end
 
   def self.subscriptions_table_name
-    NulogyMessageBusProducer::Subscription.table_name
+    NulogyMessageBusProducer::SelfServeSubscription.table_name
   end
 
   def self.subscription_events_table_name=(table_name)
@@ -33,13 +34,26 @@ module NulogyMessageBusProducer
     config.context_for_subscription(subscription)
   end
 
-  def self.trigger_event(schema, event_type, root_object)
-    schema_key = NulogyMessageBusProducer.resolve_schema_key(schema)
+  def self.register_publication
+    raise "not implemented yet"
+  end
 
-    subscriptions = Subscription.where(
-      event_type: event_type,
-      schema_key: schema_key
+  def self.publish(topic:, type:, company_uuid:, data:)
+    SubscriptionEvent.create_or_update(
+      id: SecureRandom.uuid,
+      subscription_id: SecureRandom.uuid,
+      partition_key: company_uuid,
+
+      company_uuid: company_uuid,
+      event_type: type,
+      event_json: data,
+      topic_name: topic
     )
+  end
+
+  def self.trigger_event(schema, event_type, root_object)
+    schema_key = NulogyMessageBusProducer.resolve_schema_key(schema)
+    subscriptions = Subscriptions::Finder.new(config).for_schema_event(schema_key, event_type)
 
     subscriptions.each do |subscription|
       args = {
@@ -51,7 +65,7 @@ module NulogyMessageBusProducer
     end
   end
 
-  def self.resolve_schema(schema_key)
+  def self.resolve_schema(schema_key) # rubocop:disable Metrics/MethodLength
     if config.registered_schemas.key?(schema_key)
       config.registered_schemas[schema_key].constantize
     elsif block_given?
@@ -69,7 +83,9 @@ module NulogyMessageBusProducer
   end
 
   def self.resolve_schema_key(schema)
-    config.registered_schemas.invert.fetch(schema.name) do
+    schema_name = schema.respond_to?(:name) ? schema.name : schema
+
+    config.registered_schemas.invert.fetch(schema_name) do
       raise KeyError, <<~MESSAGE
         The schema registry did not contain an entry for the schema '#{schema.name}'.
 
@@ -84,7 +100,7 @@ module NulogyMessageBusProducer
   def self.validate_existing!
     return unless Db.exists? && Db.subscriptions_exist?
 
-    validator = NulogyMessageBusProducer::SubscriberGraphqlSchemaValidator.new
+    validator = NulogyMessageBusProducer::Subscriptions::QueryValidator.new
 
     return if validator.validate
 
@@ -140,9 +156,13 @@ end
 
 require "nulogy_message_bus_producer/application_record"
 require "nulogy_message_bus_producer/base_subscription"
-require "nulogy_message_bus_producer/subscription"
 require "nulogy_message_bus_producer/subscription_event"
 require "nulogy_message_bus_producer/repopulate_replication_slots"
-require "nulogy_message_bus_producer/subscriber_graphql_schema_validator"
+require "nulogy_message_bus_producer/subscriptions/finder"
+require "nulogy_message_bus_producer/subscriptions/no_variables"
 require "nulogy_message_bus_producer/subscriptions/postgres_transport"
+require "nulogy_message_bus_producer/subscriptions/query_validator"
+require "nulogy_message_bus_producer/subscriptions/valid_for_schema_validator"
+require "nulogy_message_bus_producer/subscriptions/configured_subscription"
+require "nulogy_message_bus_producer/self_serve_subscription"
 require "nulogy_message_bus_producer/subscriptions/risky_subscription_blocker"

data/lib/nulogy_message_bus_producer/base_subscription.rb

@@ -2,7 +2,7 @@ module NulogyMessageBusProducer
   # This base class contains the fields required to create a subscription.
   # For example, for a subscription to a model called AggregateRoot:
   #
-  #  class AggregateRootCreated < NulogyMessageBusProducer::BasePublicSubscription
+  #  class AggregateRootCreated < NulogyMessageBusProducer::BaseSubscription
   #    field :aggregate_root, Domain::Public::Types::AggregateRootType, null: true
   #  end
   class BaseSubscription < GraphQL::Schema::Subscription

data/lib/nulogy_message_bus_producer/config.rb

@@ -1,14 +1,16 @@
 module NulogyMessageBusProducer
   # Configuration for the gem
-  # This is a private class, so do not use it directly. Use the methods on NulogyMessageBusProducer.
   class Config
     FAILURE_MODES = [:raise, :soft_fail].freeze
 
     attr_reader :registered_schemas
+    attr_reader :configured_subscriptions
     attr_writer :context_for_subscription
 
     def initialize(options = {})
       @registered_schemas = {}
+      @configured_subscriptions = []
+
       producing_events_fails_with(:raise)
 
       update(options)
@@ -48,6 +50,28 @@ module NulogyMessageBusProducer
       options.each { |key, value| public_send("#{key}=", value) }
     end
 
+    def add_subscription!(schema:, query:)
+      @configured_subscriptions ||= []
+
+      schema_key = NulogyMessageBusProducer.resolve_schema_key(schema)
+      query_parser = Configuration::QueryParser.new(query)
+      subscription = Subscriptions::ConfiguredSubscription.new(
+        id: query_parser.subscription_id,
+        subscription_group_id: query_parser.subscription_group_id,
+        schema_key: schema_key,
+        event_type: query_parser.event_type,
+        topic_name: query_parser.topic,
+        query: query
+      )
+
+      if subscription.valid?
+        @configured_subscriptions << subscription
+        subscription
+      else
+        raise ArgumentError, subscription.errors.full_messages
+      end
+    end
+
     private
 
     def raise_handler(subscription_id:, context:, variables:, result:)