pg_eventstore 0.1.0

data/docs/reading_events.md

@@ -0,0 +1,161 @@
+# Reading Events
+
+## Reading from a specific stream
+
+The easiest way to read a stream forwards is to supply a `PgEventstore::Stream` object.
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.read(stream)
+# => [#<PgEventstore::Event 0x1>, #<PgEventstore::Event 0x1>, ...]
+```
+
+### max_count
+
+You can provide the `:max_count` option. This option determines how many records to return in a response. Default is `1000` and it can be changed with the `:max_count` configuration setting (see [**"Configuration"**](configuration.md) chapter):
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.read(stream, options: { max_count: 100 })
+```
+
+### resolve_link_tos
+
+When reading streams with projected events (links to other events) you can chose to resolve those links by setting `resolve_link_tos` to `true`, returning the original event instead of the "link" event.
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.read(stream, options: { resolve_link_tos: true })
+```
+
+### from_revision
+
+You can define from which revision number you would like to start to read events:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.read(stream, options: { from_revision: 2 })
+```
+
+### direction
+
+As well as being able to read a stream forwards you can also go backwards. This can be achieved by providing the `:direction` option:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.read(stream, options: { direction: 'Backwards' })
+```
+
+## Checking if stream exists
+
+In case a stream with given name does not exist, a `PgEventstore::StreamNotFoundError` error will be raised:
+
+```ruby
+begin
+  stream = PgEventstore::Stream.new(context: 'non-existing-context', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+  PgEventstore.client.read(stream)
+rescue PgEventstore::StreamNotFoundError => e
+  puts e.message # => Stream #<PgEventstore::Stream:0x01> does not exist.
+  puts e.stream # => #<PgEventstore::Stream:0x01>
+end
+```
+
+## Reading from the "all" stream
+
+"all" stream definition means that you don't scope your events when reading them from the database. To get the "all" `PgEventstore::Stream` instance you have to call the `all_stream` method: 
+
+```ruby
+PgEventstore::Stream.all_stream
+```
+
+Now you can use it to read from the "all" stream:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream)
+```
+
+You can read from a specific position of the "all" stream. This is very similar to reading from a specific revision of a specific stream, but instead of the `:from_revision` option you have to provide the `:from_position` option:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { from_position: 9023, direction: 'Backwards' })
+```
+
+## Middlewares
+
+If you would like to skip some of your registered middlewares from processing events after they being read from a stream - you should use the `:middlewares` argument which allows you to override the list of middlewares you would like to use.
+
+Let's say you have these registered middlewares:
+
+```ruby
+PgEventstore.configure do |config|
+  config.middlewares = { foo: FooMiddleware.new, bar: BarMiddleware.new, baz: BazMiddleware.new }
+end
+```
+
+And you want to skip `FooMiddleware` and `BazMiddleware`. You simply have to provide an array of corresponding middleware keys you would like to use:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, middlewares: %i[bar])
+```
+
+See [Writing middleware](writing_middleware.md) chapter for info about what is middleware and how to implement it.
+
+## Filtering
+
+When reading events, you can additionally filter the result. Available attributes for filtering depend on the type of stream you are reading from. Reading from the "all" stream supports filters by stream attributes and event types. Reading from a specific stream supports filters by event types only.  
+
+### Specific stream filtering
+
+Filtering events by their types:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'MYAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.read(stream, options: { filter: { event_types: %w[Foo Bar] } })
+```
+
+### "all" stream filtering
+
+**Warning** There is a restriction on a set of stream attributes that can be used when filtering an "all" stream result. Available combinations:
+
+- `:context`
+- `:context` and `:stream_name`
+- `:context`, `:stream_name` and `:stream_id`
+
+All other combinations, like providing only `:stream_name` or providing `:context` with `:stream_id` will be ignored.
+
+
+Filtering events by type:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { event_types: %w[Foo Bar] } })
+```
+
+Filtering events by context:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext' }] } })
+```
+
+Filtering events by context and name:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext', stream_name: 'User' }] } })
+```
+
+Filtering events by stream context, stream name and stream id:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae' }] } })
+```
+
+You can provide several sets of stream's attributes. The result will be a union of events that match those criteria. For example, next query will return all events that belong to streams with `AnotherContext` context and all events that belong to streams with `MyAwesomeContext` context and `User` stream name:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'AnotherContext' }, { context: 'MyAwesomeContext', stream_name: 'User' }] } })
+```
+
+You can also mix filtering by stream's attributes and event types. The result will be intersection of events matching stream's attributes and event's types. For example, next query will return events which type is either `Foo` or `Bar` and which belong to a stream with `MyAwesomeContext` context:
+
+```ruby
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext' }], event_types: %w[Foo Bar] } })
+```

data/docs/writing_middleware.md

@@ -0,0 +1,160 @@
+# Writing middleware
+
+Middlewares are objects that modify events before they are appended to a stream, or right after they are read from a stream. Middleware object must respond to `#serialize` and `#deserialize` methods. The `#serialize` method is called each time an event is going to be appended. The `#deserialize` method is called each time an event is read from a stream. There are two ways how you can define a middleware:
+
+- by defining your class and including `PgEventstore::Middleware` module in it. This way you can override only one of its methods, or both of them. Example:
+
+```ruby
+# Override #serialize only to define your custom logic
+class MyAwesomeSerializer
+  include PgEventstore::Middleware
+
+  def serialize(event)
+    do_something
+  end
+end
+
+# Override #deserialize only to define your custom logic
+class MyAwesomeDeserializer
+  include PgEventstore::Middleware
+
+  def deserialize(event)
+    do_something
+  end
+end
+
+# Override both #serialize and #deserialize methods
+class MyAwesomeMiddleware
+  include PgEventstore::Middleware
+
+  def serialize(event)
+    do_something
+  end
+
+  def deserialize(event)
+    do_something
+  end
+end
+
+# Configure your middlewares
+PgEventstore.configure do |config|
+  config.middlewares = { my_awesome_serializer: MyAwesomeSerializer.new, my_awesome_deserializer: MyAwesomeDeserializer.new, my_awesome_middleware: MyAwesomeMiddleware.new }
+end
+```
+
+- implement your own object that implements `#serialize` and `#deserialize` methods. Example: 
+
+```ruby
+require 'securerandom'
+
+# Provide some basic functionality for large payload extraction implementations. Every event in your app will be inherited from this class.
+class MyAppAbstractEvent < PgEventstore::Event
+  def self.payload_store_fields
+    []
+  end
+
+  def fields_with_large_payloads
+    data.slice(*self.class.payload_store_fields)
+  end
+end
+
+class DescriptionChangedEvent < MyAppAbstractEvent
+  def self.payload_store_fields
+    %w[description]
+  end
+end
+
+class ExtractLargePayload
+  class << self
+    def serialize(event)
+      return if event.fields_with_large_payloads.empty?
+
+      event.fields_with_large_payloads.each do |field, value|
+        # Extract fields with large payload asynchronously.
+        event.data[field] = extract_large_payload_async(field, value)
+      end
+    end
+
+    def deserialize(event)
+      # Load real values for large payload fields. You can use self.class.payload_store_fields here, but then you would require
+      # the event definition in each mircoservice you load this event
+      event.data.select { |k, v| v.start_with?('large-payload:') }.each do |field, value|
+        event.data[field] = resolve_large_payload(value.delete('large-payload:'))
+      end
+    end
+
+    private
+
+    def extract_large_payload_async(field_name, value)
+      payload_key = "large-payload:#{field_name}-#{Digest::MD5.hexdigest(value)}"
+      Thread.new do
+        Faraday.post(
+          "https://my.awesome.api/api/extract_large_payload",
+          { payload_key: payload_key, value: value }
+        )
+      end
+      payload_key
+    end
+
+    def resolve_large_payload(payload_key)
+      JSON.parse(Faraday.get("https://my.awesome.api/api/large_payload", { payload_key: payload_key }).body)['value']
+    end
+  end  
+end
+
+# Configure our middlewares
+PgEventstore.configure do |config|
+  config.middlewares = { extract_large_payload: ExtractLargePayload }
+end
+```
+
+Now you can use it as follows:
+
+```ruby
+event = DescriptionChangedEvent.new(data: { 'description' => 'some description' })
+stream = PgEventstore::Stream.new(context: 'ctx', stream_name: 'some-stream', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.append_to_stream(stream, event)
+# => #<DescriptionChangedEvent:0x0 @data={"description"=>"large-payload:description-7815696ecbf1c96e6894b779456d330e"}, ...>
+```
+
+But when you read it next time, it will automatically resolve the `description` value:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'ctx', stream_name: 'some-stream', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+PgEventstore.client.read(stream).last
+# => #<DescriptionChangedEvent:0x0 @data={"description"=>"some description"}, ...>
+```
+
+## Remarks
+
+It is important to know that `pg_eventstore` may retry commands. In that case `#serialize` and `#deserialize` methods may also be retried. You have to make sure that the implementation of `#serialize` and `#deserialize` always returns the same result for the same input, and it does not create duplications. Let's look at the `#serialize` implementation from the example above:
+
+```ruby
+def serialize(event)
+  return if event.fields_with_large_payloads.empty?
+
+  event.fields_with_large_payloads.each do |field, value|
+    # Extract fields with large payload asynchronously.
+    event.data[field] = extract_large_payload_async(field, value)
+  end
+end
+
+private
+
+def extract_large_payload_async(field_name, value)
+  payload_key = "large-payload:#{field_name}-#{Digest::MD5.hexdigest(value)}"
+  Thread.new do
+    Faraday.post(
+      "https://my.awesome.api/api/extract_large_payload",
+      { payload_key: payload_key, value: value }
+    )
+  end
+  payload_key
+end
+```
+
+Private method `#extract_large_payload_async` should return the same result when passing the same `field_name` and `value` arguments values, and `POST https://my.awesome.api/api/extract_large_payload` may not want to produce duplicates when called multiple times with the same payload.
+
+## Async vs Sync implementation
+
+You may notice that the extracting of a large payload is asynchronous in the example above. It is recommended approach of the implementation of `#serialize` method to increase overall performance. But if it hard for you to guarantee the persistence of a payload value - you can go with sync approach, thus not allowing event to be persisted if a payload extraction request fails.

data/lib/pg_eventstore/abstract_command.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+module PgEventstore
+  # @!visibility private
+  class AbstractCommand
+    attr_reader :queries
+    private :queries
+
+    # @param queries [PgEventstore::Queries]
+    def initialize(queries)
+      @queries = queries
+    end
+
+    def call(*, **)
+      raise NotImplementedError, "Implement #call in your child class."
+    end
+  end
+end

data/lib/pg_eventstore/client.rb

@@ -0,0 +1,133 @@
+# frozen_string_literal: true
+
+require_relative 'commands'
+require_relative 'event_serializer'
+require_relative 'pg_result_deserializer'
+require_relative 'queries'
+
+module PgEventstore
+  class Client
+    attr_reader :config
+    private :config
+
+    # @param config [PgEventstore::Config]
+    def initialize(config)
+      @config = config
+    end
+
+    # Append the event or multiple events to the stream. This operation is atomic, meaning that no other event can be
+    # appended by parallel process between the given events.
+    # @param stream [PgEventstore::Stream]
+    # @param events_or_event [PgEventstore::Event, Array<PgEventstore::Event>]
+    # @param options [Hash]
+    # @option options [Integer] :expected_revision provide your own revision number
+    # @option options [Symbol] :expected_revision provide one of next values: :any, :no_stream or :stream_exists
+    # @param middlewares [Array, nil] provide a list of middleware names to override a config's middlewares
+    # @return [PgEventstore::Event, Array<PgEventstore::Event>] persisted event(s)
+    # @raise [PgEventstore::WrongExpectedRevisionError]
+    def append_to_stream(stream, events_or_event, options: {}, middlewares: nil)
+      result =
+        Commands::Append.new(queries(middlewares(middlewares))).call(stream, *events_or_event, options: options)
+      events_or_event.is_a?(Array) ? result : result.first
+    end
+
+    # Allows you to make several different commands atomic by wrapping then into a block. Order of events, produced by
+    # multiple commands, belonging to different streams - is unbreakable. So, if you append event1 to stream1 and
+    # event2 to stream2 using this method, then thet appear in the same order in the "all" stream.
+    # Example:
+    #   PgEventstore.client.multiple do
+    #     PgEventstore.client.read(...)
+    #     PgEventstore.client.append_to_stream(...)
+    #     PgEventstore.client.append_to_stream(...)
+    #   end
+    #
+    # @return the result of the given block
+    def multiple(&blk)
+      Commands::Multiple.new(queries(middlewares)).call(&blk)
+    end
+
+    # Read events from the specific stream or from "all" stream.
+    # @param stream [PgEventstore::Stream]
+    # @param options [Hash] request options
+    # @option options [String] :direction read direction - 'Forwards' or 'Backwards'
+    # @option options [Integer] :from_revision a starting revision number. **Use this option when stream name is a
+    #   normal stream name**
+    # @option options [Integer, Symbol] :from_position a starting global position number. **Use this option when reading
+    #   from "all" stream**
+    # @option options [Integer] :max_count max number of events to return in one response. Defaults to config.max_count
+    # @option options [Boolean] :resolve_link_tos When using projections to create new events you
+    #   can set whether the generated events are pointers to existing events. Setting this option to true tells
+    #   PgEventstore to return the original event instead a link event.
+    # @option options [Hash] :filter provide it to filter events. You can filter by: stream and by event type. Filtering
+    #   by stream is only available when reading from "all" stream.
+    #   Examples:
+    #     # Filtering by stream's context. This will return all events which #context is 'User
+    #     PgEventstore.client.read(
+    #       PgEventstore::Stream.all_stream,
+    #       options: { filter: { streams: [{ context: 'User' }] } }
+    #     )
+    #
+    #     # Filtering by several stream's contexts. This will return all events which #context is either 'User' or
+    #     # 'Profile'
+    #     PgEventstore.client.read(
+    #       PgEventstore::Stream.all_stream,
+    #       options: { filter: { streams: [{ context: 'User' }, { context: 'Profile' }] } }
+    #     )
+    #
+    #     # Filtering by a mix of specific stream and a context. This will return all events which #context is 'User' or
+    #     # events belonging to the stream with { context: 'Profile', stream_name: 'ProfileFields', stream_id: '123' }
+    #     PgEventstore.client.read(
+    #       PgEventstore::Stream.all_stream,
+    #       options: {
+    #         filter: {
+    #           streams: [
+    #             { context: 'User' },
+    #             { context: 'Profile', stream_name: 'ProfileFields', stream_id: '123' }
+    #           ]
+    #         }
+    #       }
+    #     )
+    #
+    #     # Filtering the a mix of context and event type
+    #     PgEventstore.client.read(
+    #       PgEventstore::Stream.all_stream,
+    #       options: { filter: { streams: [{ context: 'User' }], event_types: ['MyAwesomeEvent'] } }
+    #     )
+    #
+    #     # Filtering by specific event when reading from the specific stream
+    #     PgEventstore.client.read(stream, options: { filter: { event_types: ['MyAwesomeEvent'] } })
+    # @param middlewares [Array, nil] provide a list of middleware names to override a config's middlewares
+    # @return [Array<PgEventstore::Event>]
+    # @raise [PgEventstore::StreamNotFoundError]
+    def read(stream, options: {}, middlewares: nil)
+      Commands::Read.
+        new(queries(middlewares(middlewares))).
+        call(stream, options: { max_count: config.max_count }.merge(options))
+    end
+
+    private
+
+    # @param middlewares [Array, nil]
+    # @return [Array<Object<#serialize, #deserialize>>]
+    def middlewares(middlewares = nil)
+      return config.middlewares.values unless middlewares
+
+      config.middlewares.slice(*middlewares).values
+    end
+
+    # @return [PgEventstore::Connection]
+    def connection
+      PgEventstore.connection(config.name)
+    end
+
+    # @param middlewares [Array<Object<#serialize, #deserialize>>]
+    # @return [PgEventstore::Queries]
+    def queries(middlewares)
+      Queries.new(
+        connection,
+        EventSerializer.new(middlewares),
+        PgResultDeserializer.new(middlewares, config.event_class_resolver)
+      )
+    end
+  end
+end

data/lib/pg_eventstore/commands/append.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+module PgEventstore
+  module Commands
+    # @!visibility private
+    class Append < AbstractCommand
+      # @param stream [PgEventstore::Stream]
+      # @param events [Array<PgEventstore::Event>]
+      # @param options [Hash]
+      # @option options [Integer] :expected_revision provide your own revision number
+      # @option options [Symbol] :expected_revision provide one of next values: :any, :no_stream or :stream_exists
+      # @return [Array<PgEventstore::Event>] persisted events
+      # @raise [PgEventstore::WrongExpectedRevisionError]
+      def call(stream, *events, options: {})
+        raise SystemStreamError, stream if stream.system?
+
+        queries.transaction do
+          stream = queries.find_or_create_stream(stream)
+          revision = stream.stream_revision
+          assert_expected_revision!(revision, options[:expected_revision]) if options[:expected_revision]
+          events.map.with_index(1) do |event, index|
+            queries.insert(stream, prepared_event(event, revision + index))
+          end.tap do
+            queries.update_stream_revision(stream, revision + events.size)
+          end
+        end
+      end
+
+      private
+
+      # @param event [PgEventstore::Event]
+      # @param revision [Integer]
+      # @return [PgEventstore::Event]
+      def prepared_event(event, revision)
+        event.class.new(
+          id: event.id, data: event.data, metadata: event.metadata, type: event.type, stream_revision: revision
+        )
+      end
+
+      # @param revision [Integer]
+      # @param expected_revision [Symbol, Integer]
+      # @raise [PgEventstore::WrongExpectedRevisionError] in case if revision does not satisfy expected revision
+      # @return [void]
+      def assert_expected_revision!(revision, expected_revision)
+        return if expected_revision == :any
+
+        case [revision, expected_revision]
+        in [Integer, Integer]
+          raise WrongExpectedRevisionError.new(revision, expected_revision) unless revision == expected_revision
+        in [Integer, Symbol]
+          if revision == Stream::INITIAL_STREAM_REVISION && expected_revision == :stream_exists
+            raise WrongExpectedRevisionError.new(revision, expected_revision)
+          end
+          if revision > Stream::INITIAL_STREAM_REVISION && expected_revision == :no_stream
+            raise WrongExpectedRevisionError.new(revision, expected_revision)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/pg_eventstore/commands/multiple.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+module PgEventstore
+  module Commands
+    # @!visibility private
+    class Multiple < AbstractCommand
+      def call(&blk)
+        queries.transaction do
+          yield
+        end
+      end
+    end
+  end
+end

data/lib/pg_eventstore/commands/read.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module PgEventstore
+  module Commands
+    # @!visibility private
+    class Read < AbstractCommand
+      # @param stream [PgEventstore::Stream]
+      # @param options [Hash] request options
+      # @option options [String] :direction read direction - 'Forwards' or 'Backwards'
+      # @option options [Integer, Symbol] :from_revision. **Use this option when stream name is a normal stream name**
+      # @option options [Integer, Symbol] :from_position. **Use this option when reading from "all" stream**
+      # @option options [Integer] :max_count
+      # @option options [Boolean] :resolve_link_tos
+      # @option options [Hash] :filter provide it to filter events
+      # @return [Array<PgEventstore::Event>]
+      # @raise [PgEventstore::StreamNotFoundError]
+      def call(stream, options: {})
+        stream = queries.find_stream(stream) || raise(StreamNotFoundError, stream) unless stream.all_stream?
+
+        queries.stream_events(stream, options)
+      end
+    end
+  end
+end

data/lib/pg_eventstore/commands.rb

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+require_relative 'abstract_command'
+require_relative 'commands/append'
+require_relative 'commands/read'
+require_relative 'commands/multiple'

data/lib/pg_eventstore/config.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module PgEventstore
+  class Config
+    include Extensions::OptionsExtension
+
+    attr_reader :name
+
+    # PostgreSQL connection URI docs https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS
+    option(:pg_uri) { 'postgresql://postgres:postgres@localhost:5432/eventstore' }
+    option(:max_count) { 1000 }
+    option(:middlewares) { {} }
+    # Object that responds to #call. Should accept a string and return a class
+    option(:event_class_resolver) { EventClassResolver.new }
+    option(:connection_pool_size) { 5 }
+    option(:connection_pool_timeout) { 5 } # seconds
+
+    # @param name [Symbol] config's name. Its value matches the appropriate key in PgEventstore.config hash
+    def initialize(name:, **options)
+      super
+      @name = name
+    end
+
+    # Computes a value for usage in PgEventstore::Connection
+    # @return [Hash]
+    def connection_options
+      { uri: pg_uri, pool_size: connection_pool_size, pool_timeout: connection_pool_timeout }
+    end
+  end
+end