pg_eventstore 2.0.0 → 3.0.0

data/docs/admin_ui.md

@@ -10,7 +10,9 @@
 
 ## Authorization
 
-Admin UI is implemented as a rack application. It doesn't have any built-in authentication/authorization mechanism - it is your responsibility to take care of it. Admin UI tries to look for `:admin_web_ui` config with a fallback to `:default` config. Thus, you can setup Admin UI-specific config, e.g. without some middlewares or so. Example:
+Admin UI is implemented as a rack application. It doesn't have any built-in authentication/authorization mechanism - it
+is your responsibility to take care of it. Admin UI tries to look for `:admin_web_ui` config with a fallback to
+`:default` config. Thus, you can setup Admin UI-specific config, e.g. without some middlewares or so. Example:
 
 ```ruby
 PgEventstore.configure(name: :admin_web_ui) do |config|

data/docs/appending_events.md

@@ -1,67 +1,51 @@
 # Appending Events
 
-## Append your first event
+## Append event
 
-The easiest way to append an event is to create an event object and a stream object and call the client's `#append_to_stream` method.
+The easiest way to append an event is to create an event object and a stream object and call the client's
+`#append_to_stream` method.
 
 ```ruby
-require 'securerandom'
-
 class SomethingHappened < PgEventstore::Event
 end
 
-event = SomethingHappened.new(data: { user_id: SecureRandom.uuid, title: "Something happened" })
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+event = SomethingHappened.new(data: { user_id: '1', title: "Something happened" })
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
 PgEventstore.client.append_to_stream(stream, event)
-# => #<SomethingHappened:0x0 @context="MyAwesomeContext", @created_at=2023-11-30 14:47:31.296229 UTC, @data={"title"=>"Something happened", "user_id"=>"be52a81c-ad5b-4cfd-a039-0b7276974e6b"}, @global_position=7, @id="0b01137b-bdd8-4f0d-8ccf-f8c959e3a324", @link_global_position=nil, @metadata={}, @stream_id="f37b82f2-4152-424d-ab6b-0cc6f0a53aae", @stream_name="SomeStream", @stream_revision=0, @type="SomethingHappened">
+# => #<SomethingHappened:0x000077c654b8f8c0 @readonly=Set[], @id="ecab1317-f2ae-4fc0-ad16-b820cd6fe053", @type="SomethingHappened", @global_position=133486960, @stream=#<PgEventstore::Stream:0x000077c6549e3990 @context="FooCtx", @stream_name="Foo", @stream_id="1", @stream_revision=nil, @starting_position=nil>, @stream_revision=16024, @data={"title" => "Something happened", "user_id" => "1"}, @markers=[], @metadata={}, @link_global_position=nil, @link_partition_id=nil, @link=nil, @created_at=2026-07-08 10:25:12.721906 UTC>
 ```
 
 ## Appending multiple events
 
-You can pass an array of events to the `#append_to_stream` method. This way events will be appended one-by-one. **This operation is atomic and it guarantees that events are added to the stream in the given order.**
+You can pass an array of events to the `#append_to_stream` method. This way events will be appended one-by-one. **This
+operation is atomic and it guarantees that events are added to the stream in the given order.**
 
 ```ruby
-require 'securerandom'
-
 class SomethingHappened < PgEventstore::Event
 end
 
-event1 = SomethingHappened.new(data: { user_id: SecureRandom.uuid, title: "Something happened 1" })
-event2 = SomethingHappened.new(data: { user_id: SecureRandom.uuid, title: "Something happened 2" })
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+event1 = SomethingHappened.new(data: { user_id: '1', title: "Something happened 1" })
+event2 = SomethingHappened.new(data: { user_id: '1', title: "Something happened 2" })
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
 PgEventstore.client.append_to_stream(stream, [event1, event2])
 ```
 
-### Duplicated event id
-
-If two events with the same id are appended to any stream - `pg_eventstore` will only append one event, and the second command will raise an error.
-
-```ruby
-class SomethingHappened < PgEventstore::Event
-end
-
-event = SomethingHappened.new(id: SecureRandom.uuid)
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
-PgEventstore.client.append_to_stream(stream, event)
-# Raises PG::UniqueViolation error
-PgEventstore.client.append_to_stream(stream, event)
-```
-
 ## Handling concurrency
 
-When appending events to a stream you can supply a stream state or stream revision. You can use this to tell `pg_eventstore` what state or version you expect the stream to be in when you append. If the stream isn't in that state then an exception will be thrown.
+When appending events to a stream you can supply a stream state or stream revision. You can use this to tell
+`pg_eventstore` what state or version you expect the stream to be in when you append. If the stream isn't in that state
+then an exception will be thrown.
 
-For example if we try to append two records expecting both times that the stream doesn't exist we will get an exception on the second:
+For example if we try to append two records expecting both times that the stream doesn't exist we will get an exception
+on the second:
 
 ```ruby
-require 'securerandom'
-
 class SomethingHappened < PgEventstore::Event
 end
 
 event1 = SomethingHappened.new(data: { foo: :bar })
 event2 = SomethingHappened.new(data: { bar: :baz })
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: SecureRandom.uuid)
+stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: '1')
 
 # Successfully appends an event
 PgEventstore.client.append_to_stream(stream, event1, options: { expected_revision: :no_stream })
@@ -76,15 +60,14 @@ Here are possible values of `:expected_revision` option:
 - `:stream_exists`. Expects a stream to be present when appending an event
 - a revision number(Integer). Expects a stream to be in the given revision.
 
-This check can be used to implement optimistic concurrency. When you retrieve a stream, you take note of the current version number, then when you save it back you can determine if somebody else has modified the record in the meantime.
+This check can be used to implement optimistic concurrency. When you retrieve a stream, you take note of the current
+version number, then when you save it back you can determine if somebody else has modified the record in the meantime.
 
 ```ruby
-require 'securerandom'
-
 class SomethingHappened < PgEventstore::Event
 end
 
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: SecureRandom.uuid)
+stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: '1')
 event1 = SomethingHappened.new(data: { foo: :bar })
 event2 = SomethingHappened.new(data: { bar: :baz })
 
@@ -99,20 +82,95 @@ PgEventstore.client.append_to_stream(stream, event2, options: { expected_revisio
 PgEventstore.client.append_to_stream(stream, event2, options: { expected_revision: revision })
 ```
 
-### What to do when a PgEventstore::WrongExpectedRevisionError error is risen?
+### Dynamic Consistency Boundaries support
+
+You can validate a revision of separate event type(s) when publishing an event. This allows you to enforce consistency
+of a specific event type(s) instead the consistency of the whole stream. To do so, you have to supply a
+<event type>-to-<event revision> map as a value of `:expected_revion` option. Available per-type expected revisions:
+
+- `:any`. Doesn't perform any checks. This is the default.
+- `:no_event`. Expects an event to be absent when appending an event
+- `:event_exists`. Expects an event to be present when appending an event
+- a revision number(Integer). Expects an event to be in the given revision.
+
+For example, next append command succeeds only if there is no `Foo` event, `Bar` event has revision `1` and `Baz` event
+exists with any revision:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+event = PgEventstore::Event.new(data: { foo: :bar })
+PgEventstore.client.append_to_stream(
+  stream, event, options: { expected_revision: { 'Foo' => :no_event, 'Bar' => 1, 'Baz' => :event_exists } }
+)
+```
+
+If event type(s) revision validation fails - `PgEventstore::WrongExpectedTypesRevisionError` exception is risen. Please
+note, that this is different exception class comparing to the one which is present when stream revision validation
+fails(`PgEventstore::WrongExpectedRevisionError`).
+
+## Event markers
+
+You can assign multiple markers(strings) to an event. This is multipurpose feature which may have application-specific
+meaning, like observability or can be used as an additional constraint to validate event type revision as a part of
+Dynamic Consistency Boundaries support.
+
+### Publishing event with markers
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+event = PgEventstore::Event.new(markers: %w[foo bar])
+PgEventstore.client.append_to_stream(stream, event)
+```
+
+Refer to [Read API docs](reading_events.md#filtering-events-by-markers) to find out how to filter events using markers.
+
+### Dynamic Consistency Boundaries and markers
+
+In addition to [per-event type validation](appending_events.md#dynamic-consistency-boundaries-support) - you can specify
+a set of markers the specific event type may contain in order to pass revision validation. The available per-type
+revisions are the same(`:any`, `:no_event`, `:event_exists` or number), but the syntax is a bit different. The example
+bellow succeeds if there is no `Foo` event yet with either `'foo'` **or** `'bar'` marker:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+event = PgEventstore::Event.new(type: 'Foo', markers: ['bar'])
+PgEventstore.client.append_to_stream(
+  stream, event, options: { expected_revision: { 'Foo' => { expected_revision: :no_event, markers: %w[foo bar] } } }
+)
+```
+
+Please note, there is no possibility to ensure the event has both - `'foo'` and `'bar'` markers. If you want to build
+the logic that requires to differentiate between events with `'foo'`, `'bar'` and both markers - you have to
+additionally mark such events with third marker. For example, you can have `'foobar'` marker which is used for events
+with both `'foo'` and `'bar'` markers:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+event = PgEventstore::Event.new(type: 'Foo', markers: %w[foo bar foobar])
+PgEventstore.client.append_to_stream(
+  stream, event, options: { expected_revision: { 'Foo' => { expected_revision: :no_event, markers: %w[foobar] } } }
+)
+```
+
+## What to do when a WrongExpectedRevisionError or WrongExpectedTypesRevisionError error is risen?
 
 Imagine the following scenario:
+
 1. You load events of a stream to build the state of your business object represented by the stream.
 2. You check your business rules to see if you can change that object's state the way you want to change it.
 3. If no business rules have been violated, you have the go to publish the event representing the state change.
-4. To make sure the new event will follow the last event you used to build your object state, you retrieve that last event's revision and increase it by one. You now have the expected revision for the event to be published.
-5. You publish the event but retrieve a `WrongExpectedRevisionError`. This means another process has appended an event to the same stream, after you were loading your business object, while you were checking your business rules.
-6. Now you need to repeat the process: load your business objects from the updated events stream, apply your business rules and if there is still no violation, try to append the event with the updated stream revision. You can do this procedure until the event is published or a maximum number of retries has been reached.
+4. To make sure the new event will follow the last event you used to build your object state, you retrieve that last
+   event's revision and increase it by one. You now have the expected revision for the event to be published.
+5. You publish the event but retrieve a `WrongExpectedRevisionError`. This means another process has appended an event
+   to the same stream, after you were loading your business object, while you were checking your business rules.
+6. Now you need to repeat the process: load your business objects from the updated events stream, apply your business
+   rules and if there is still no violation, try to append the event with the updated stream revision. You can do this
+   procedure until the event is published or a maximum number of retries has been reached.
 
-The following example shows the described retry procedure, with a simple business rule that does not allow adding an event after a `UserRemoved` event:
+The following example shows the described retry procedure, with a simple business rule that does not allow adding an
+event after a `UserRemoved` event:
 
 ```ruby
-require 'securerandom'
 class UserAboutMeChanged < PgEventstore::Event
 end
 
@@ -121,7 +179,7 @@ end
 
 def latest_event(stream)
   PgEventstore.client.read(stream, options: { max_count: 1, direction: 'Backwards' }).first
-rescue PgEventstore::StreamNotFoundError  
+rescue PgEventstore::StreamNotFoundError
 end
 
 def publish_event(stream, event)
@@ -138,10 +196,10 @@ def publish_event(stream, event)
     retries_count += 1
     raise if retries_count > 3
     retry
-  end  
+  end
 end
 
-stream = PgEventstore::Stream.new(context: 'UserProfile', stream_name: 'User', stream_id: SecureRandom.uuid)
+stream = PgEventstore::Stream.new(context: 'UserProfile', stream_name: 'User', stream_id: '1')
 event = UserAboutMeChanged.new(data: { user_id: '123', about_me: 'hi there!' })
 
 publish_event(stream, event)
@@ -149,7 +207,9 @@ publish_event(stream, event)
 
 ## Middlewares
 
-If you would like to skip some of your registered middlewares from processing events before they get appended to a stream - you should use the `:middlewares` argument which allows you to override the list of middlewares you would like to use.
+If you would like to skip some of your registered middlewares from processing events before they get appended to a
+stream - you should use the `:middlewares` argument which allows you to override the list of middlewares you would like
+to use.
 
 Let's say you have these registered middlewares:
 
@@ -159,11 +219,12 @@ PgEventstore.configure do |config|
 end
 ```
 
-And you want to skip `FooMiddleware` and `BazMiddleware`. You simply have to provide an array of corresponding middleware keys you would like to use:
+And you want to skip `FooMiddleware` and `BazMiddleware`. You simply have to provide an array of corresponding
+middleware keys you would like to use:
 
 ```ruby
 event = PgEventstore::Event.new
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'SomeStream', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
 PgEventstore.client.append_to_stream(stream, event, middlewares: %i[bar])
 ```
 

data/docs/configuration.md

@@ -2,22 +2,25 @@
 
 Configuration options:
 
-| name                                   | value          | default value                                                | description                                                                                                                                                                                                                                                                                                            |
-|----------------------------------------|----------------|--------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| pg_uri                                 | String         | `'postgresql://postgres:postgres@localhost:5432/eventstore'` | PostgreSQL connection string. See PostgreSQL [docs](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS) for more information.                                                                                                                                                            |
-| max_count                              | Integer        | `1000`                                                       | Number of events to return in one response when reading from a stream.                                                                                                                                                                                                                                                 |
-| middlewares                            | Array          | `{}`                                                         | A hash where a key is a name of your middleware and value is an object that respond to `#serialize` and `#deserialize` methods. See [**Writing middleware**](writing_middleware.md) chapter.                                                                                                                           |
-| event_class_resolver                   | `#call`        | `PgEventstore::EventClassResolver.new`                       | A `#call`-able object that accepts a string and returns an event's class. See **Resolving events classes** chapter bellow for more info.                                                                                                                                                                               |
-| connection_pool_size                   | Integer        | `5`                                                          | Max number of connections per ruby process. It must equal the number of threads of your application. When using subscriptions it is recommended to set it to the number of subscriptions divided by two or greater. See [**Picking max connections number**](#picking-max-connections-number) chapter of this section. |
-| connection_pool_timeout                | Integer        | `5`                                                          | Time in seconds to wait for a connection in the pool to be released. If no connections are available during this time - `ConnectionPool::TimeoutError` will be raised. See `connection_pool` gem [docs](https://github.com/mperham/connection_pool#usage) for more info.                                               |
-| subscription_pull_interval             | Float          | `1.0`                                                        | How often to pull new subscription events in seconds. The minimum meaningful value is `0.2`. Values less than `0.2` will act as it is `0.2`.                                                                                                                                                                           |
-| subscription_max_retries               | Integer        | `5`                                                          | Max number of retries of failed subscription.                                                                                                                                                                                                                                                                          |
-| subscription_retries_interval          | Integer        | `1`                                                          | Interval in seconds between retries of failed subscriptions.                                                                                                                                                                                                                                                           |
-| subscriptions_set_max_retries          | Integer        | `10`                                                         | Max number of retries for failed subscription sets.                                                                                                                                                                                                                                                                    |
-| subscriptions_set_retries_interval     | Integer        | `1`                                                          | Interval in seconds between retries of failed subscription sets.                                                                                                                                                                                                                                                       |
-| subscription_restart_terminator        | `#call`        | `nil`                                                        | A callable object that accepts `PgEventstore::Subscription` object to determine whether restarts should be stopped(true - stops restarts, false - continues restarts).                                                                                                                                                 |
-| failed_subscription_notifier           | `#call`        | `nil`                                                        | A callable object which is invoked with `PgEventstore::Subscription` instance and error instance after the related subscription died due to error and no longer can be automatically restarted due to max retries number reached. You can use this hook to send a notification about failed subscription.              |
-| subscription_graceful_shutdown_timeout | Integer, Float | `15`                                                         | The number of seconds to wait until force-shutdown the subscription during the stop process. If your subscription handler does not finish current event processing during this time(for example because of heavy-lifting task) - it will be force-shutdown.                                                            |
+| name                                         | value          | default value                                                | description                                                                                                                                                                                                                                                                                               |
+|----------------------------------------------|----------------|--------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| pg_uri                                       | String         | `'postgresql://postgres:postgres@localhost:5432/eventstore'` | PostgreSQL connection string. See PostgreSQL [docs](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS) for more information.                                                                                                                                               |
+| max_count                                    | Integer        | `1000`                                                       | Number of events to return in one response when reading from a stream.                                                                                                                                                                                                                                    |
+| middlewares                                  | Hash           | `{}`                                                         | A hash where a key is a name of your middleware and value is an object that respond to `#serialize` and `#deserialize` methods. See [**Writing middleware**](writing_middleware.md) chapter.                                                                                                              |
+| event_class_resolver                         | `#call`        | `PgEventstore::EventClassResolver.new`                       | A `#call`-able object that accepts a string and returns an event's class. See **Resolving events classes** chapter bellow for more info.                                                                                                                                                                  |
+| connection_pool_size                         | Integer        | `5`                                                          | Max number of connections per ruby process. See [**Picking max connections number**](#picking-max-connections-number) chapter of this section for more details.                                                                                                                                           |
+| connection_pool_timeout                      | Integer        | `5`                                                          | Time in seconds to wait for a connection in the pool to be released. If no connections are available during this time - `ConnectionPool::TimeoutError` will be raised. See `connection_pool` gem [docs](https://github.com/mperham/connection_pool#usage) for more info.                                  |
+| subscription_pull_interval                   | Float          | `1.0`                                                        | How often to pull new subscription events in seconds. The minimum meaningful value is `0.2`. Values less than `0.2` will act as it is `0.2`.                                                                                                                                                              |
+| subscription_max_retries                     | Integer        | `5`                                                          | Max number of retries of failed subscription.                                                                                                                                                                                                                                                             |
+| subscription_retries_interval                | Integer        | `1`                                                          | Interval in seconds between retries of failed subscriptions.                                                                                                                                                                                                                                              |
+| subscriptions_set_max_retries                | Integer        | `10`                                                         | Max number of retries for failed subscription sets.                                                                                                                                                                                                                                                       |
+| subscriptions_set_retries_interval           | Integer        | `1`                                                          | Interval in seconds between retries of failed subscription sets.                                                                                                                                                                                                                                          |
+| subscription_restart_terminator              | `#call`        | `nil`                                                        | A callable object that accepts `PgEventstore::Subscription` object to determine whether restarts should be stopped(true - stops restarts, false - continues restarts).                                                                                                                                    |
+| failed_subscription_notifier                 | `#call`        | `nil`                                                        | A callable object which is invoked with `PgEventstore::Subscription` instance and error instance after the related subscription died due to error and no longer can be automatically restarted due to max retries number reached. You can use this hook to send a notification about failed subscription. |
+| subscription_graceful_shutdown_timeout       | Integer, Float | `15`                                                         | The number of seconds to wait until force-shutdown the subscription during the stop process. If your subscription handler does not finish current event processing during this time(for example because of heavy-lifting task) - it will be force-shutdown.                                               |
+| events_subscription_position_update_interval | Integer, Float | `0.2`                                                        | Events subscription position update interval. See related docs [**Subscription position**](subscriptions.md#subscription-position)                                                                                                                                                                        |
+| eventstore_role                              | Symbol         | `:standalone`                                                | Sets the role of the current configuration. Read more about this config option [bellow](#eventstore-role)                                                                                                                                                                                                 |
+| max_events_to_replicate                      | Integer        | `10_000`                                                     | Max number of events to replicate at a time. Read more about this config option [bellow](#max-events-to-replicate)                                                                                                                                                                                        |
 
 ## Multiple configurations
 
@@ -93,31 +96,61 @@ end
 
 ## Picking max connections number
 
-A connection is hold from the connection pool to perform the request and it is released back to the connection pool once
-the request is finished. If you run into the (theoretical) edge case, when all your application's threads (or
+A connection is hold from the connection pool to perform the request, and it is released back to the connection pool
+once the request is finished. If you run into the (theoretical) edge case, when all your application's threads (or
 subscriptions) are performing `pg_eventstore` queries at the same time and all those queries take more
 than `connection_pool_timeout` seconds to complete, you have to have `connection_pool_size` set to the exact amount of
-your application's threads (or to the number of subscriptions when using subscriptions) to prevent timeout errors.
-Practically this is not the case, as all `pg_eventstore` queries are pretty fast. So, a good value for
-the `connection_pool_size` option is **half the number ** of your application's threads(or half the number of
-Subscriptions).
+your application's threads to prevent timeout errors. Practically this is not the case, as all `pg_eventstore` queries
+are pretty fast. So, a good value for the `connection_pool_size` option is the number of your application's threads.
 
-### Exception scenario
+### Connections number for subscriptions
 
-If you are using the [`#multiple`](multiple_commands.md) method - you have to take into account the execution time of
-the whole block you pass in it. This is because the connection will be released only after the block's execution is
-finished. So, for example, if you perform several commands within the block, as well as some API request, the connection
-will be release only after all those steps:
+Subscriptions implementation is more demanding on connections number. This is because it internally implements
+different services that support various aspects of the feature. Here is a breakdown of connections distribution:
+
+- 1 connection per subscription to manage each subscription itself(e.g. update its stats, positions, etc)
+- 1 connection per 10 subscriptions to pull new events
+- 1 connection to process remote commands, such as starting/stopping/restart subscriptions from admin web UI
+- 1 connection to assign subscription positions of newly created events
+- 1 connection to manage [SubscriptionsSet](subscriptions.md#pgeventstoresubscriptionsset)
+
+So, the formula to calculate the number of connection your subscriptions process should have is next:
 
 ```ruby
-PgEventstore.client.multiple do
-  # Connection is hold from the connection pool
-  PgEventstore.client.read(some_stream)
-  Stripe::Payment.create(some_attrs)
-  PgEventstore.client.append_to_stream(some_stream, some_event)
-  # Connection is released
-end
+number_of_subscriptions + (number_of_subscriptions / 10.0).ceil + 3
 ```
 
-Taking this into account you may want to increase `connection_pool_size` up to the number of your application's threads(
-or subscriptions).
+So, for example, for 10 subscriptions - the implementation may consume up to 14 connections at a time. On practice this
+number will be lower than the threshold most amount of time, because queries behind the implementation are near instant.
+Nevertheless, it is wise to always have the required number ready to be utilized.
+
+### Tips
+
+To be able to always have enough amount of available connections - it is recommended to use some connection pooler,
+such as [pgbouncer](https://www.pgbouncer.org/) in transaction mode.
+
+## Eventstore role
+
+Defines the role of your event store. Available values are:
+
+- `:standalone` Means you are running the only copy of your database. This is the default. No functional restrictions
+  are applied
+- `:primary` Means this is your primary node where events are get published
+- `:replica` Means this is a replica of your primary node. No events can be published
+
+Functional restrictions that apply to `:primary` and `:replica` roles:
+
+- `:primary` and `:replica` roles are not allowed to perform any maintenance operations, such as delete events or
+  delete streams
+- `:replica` role is not allowed to publish any events
+
+## Max events to replicate
+
+This option determines how many events you want to copy from your primary node into your replica node at a time. The
+larger this value - the more resources are needed. It is both - memory and CPU intensive as a replica subscription first
+loads events into memory, prepares them and then persists into the destination replica node. Thus, it may be wise to
+split your replica subscriptions from your application subscriptions and adjust the environment to handle higher loads.
+
+Please note, that this is an **upper limit** of events to replicate at a time. Subscriptions have their own measurement
+of how many events a subscription handler processes per second. Thus, the number of events actually processed by a
+subscription at a time may differ.

data/docs/event_tracing.md

@@ -0,0 +1,55 @@
+#  Event tracing
+
+pg_eventstore implements event tracing feature out of the box via `PgEventstore::Middleware::EventTracing` middleware
+class. It allows you to connect events between each other with markers and build causation dependency between them. The
+middleware is not configured by default. You can configure it as follows:
+
+```ruby
+PgEventstore.configure do |config|
+  config.middlewares = { event_tracing: PgEventstore::Middleware::EventTracing.new }
+end
+```
+
+You can now supply new event with an event, based on which current event is going to be published. Example:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'User', stream_name: 'RegistrationRequest', stream_id: '1')
+user_registration_confirmed = PgEventstore.client.append_to_stream(
+  stream, PgEventstore::Event.new(type: 'UserRegistrationConfirmed')
+)
+
+stream = PgEventstore::Stream.new(context: 'User', stream_name: 'User', stream_id: '1')
+user_created = PgEventstore.client.append_to_stream(
+  stream, PgEventstore::Event.new(caused_by: user_registration_confirmed, type: 'UserCreated')
+)
+```
+
+`UserCreated` event now has `#causation_id` value equal to `user_registration_confirmed.id`. Both of them have the same
+`#correlation_id`. You can use those values to find those events.
+
+Use `#correlation_id` to find all connected events:
+
+```ruby
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream, 
+  options: { filter: { event_types: [{ markers: [user_created.correlation_id] }] } }
+) # => returns both events
+```
+
+Use `#causation_id` to find all events, which were persisted, based on the given causation id:
+```ruby
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream, 
+  options: { filter: { event_types: [{ markers: [user_registration_confirmed.id] }] } }
+) # => returns both events
+```
+
+`#causation_id` and `#correlation_id` values are persisted into special keys of `#metadata` and are loaded back into 
+`#causation_id` and `#correlation_id` attributes during the deserialization process. Related constants are:
+- `PgEventstore::Middleware::EventTracing::CAUSATION_ID_KEY` for `#causation_id`
+- `PgEventstore::Middleware::EventTracing::CORRELATION_ID_KEY` for `#correlation_id`
+
+You can also find `#correlation_id` and `#causation_id` values in `Event#feature_markers` attribute, `Markers` section
+of extended event view(it is available by clicking on `JSON` link with eye icon near it on the admin dashboard page)
+under `Feature markers` chapter. It is only available when `PgEventstore::Middleware::EventTracing` is configured
+though, but filtering by markers is always there despite on configuration options.

data/docs/events_and_streams.md

@@ -9,17 +9,52 @@
 
 `PgEventstore::Event` has the following attributes:
 
-- `id` - String(UUIDv4, optional, not `nil`). If no provided - the value will be autogenerated.
-- `type` - String(optional, not `nil`). Default is an event's class name. Types which start from `$` indicate system events. It is not recommended to prefix your events types with `$` sign.
-- `global_position` - Integer(optional, read only). Event's global position in the eventstore, aka the "all" stream position (inspired by the popular EventstoreDB). Manually assigning this attribute has no effect. It is internally set when reading events from the database.
-- `stream` - PgEventstore::Stream(optional, read only). A Stream an event belongs to, see description below. Manually assigning this attribute has no effect. It is internally set when appending an event to the given stream or when reading events from the database.
-- `stream_revision` - Integer(optional, read only). A revision of an event inside its stream.
-- `data` - Hash(optional). Event's payload data. For example, if you have a `DescriptionChanged` event class, then you may want to have a description value in the event payload data. Example: `DescriptionChanged.new(data: { 'description' => 'Description of something', 'post_id' => SecureRandom.uuid })`
-- `metadata` - Hash(optional). Event metadata. Event meta information which is not part of an events data payload. Example: `{ published_by: publishing_user.id }`
-- `link_global_position` - Integer(optional, read only). If an event is a link event (link events are pointers to other events), this attribute contains a `global_position` of the original event. Manually assigning this attribute has no effect. It is internally set when linking an event to the given stream or when reading events from the database.
-- `link_partition_id` - Integer(optional, read only). If an event is a link event - this attribute contains a partition `id` of original event. Manually assigning this attribute has no effect. It is internally set when appending an event to the given stream or when reading events from the database.
-- `link` - PgEventstore::Event(optional, read only). When reading from a stream using `resolve_link_tos: true`, if an event is resolved from a link - this attribute contains a `PgEventstore::Event` object which corresponds to that link. Manually assigning this attribute has no effect. It is internally set when reading events from the database.
-- `created_at` - Time(optional, read only). Database's timestamp when an event was appended to a stream. You may want to put your own timestamp into a `metadata` attribute - it may be useful when migrating between different databases. Manually assigning this attribute has no effect. It is internally set when appending an event to the given stream or when reading events from the database.
+- `id` - `String`(UUIDv7, optional, not `nil`, default is `SecureRandom.uuid_v7`). Application-specific identifier, may
+  be used by some middlewares. Uniqueness is not guaranteed on the database level, but you can rely on
+  `SecureRandom.uuid_v7` as a source of unique values.
+- `type` - `String`(optional, not `nil`). Default is an event's class name. Types which start from `$` indicate system
+  events. It is not recommended to prefix your events types with `$` sign.
+- `global_position` - Integer(optional, read only). Event's global position in the eventstore, aka the "all" stream
+  position (inspired by the popular EventstoreDB). Manually assigning this attribute has no effect. It is internally set
+  when writing events into the database.
+- `stream` - `PgEventstore::Stream`(optional, read only). A Stream an event belongs to, see description below. Manually
+  assigning this attribute has no effect. It is internally set when appending an event to the given stream or when
+  reading events from the database.
+- `stream_revision` - `Integer`(optional, read only). Stream revision at the given event
+- `data` - Hash(optional). Event's payload data. For example, if you have a `DescriptionChanged` event class, then you
+  may want to have a description value in the event payload data. Example:
+  `DescriptionChanged.new(data: { 'description' => 'Description of something', 'post_id' => SecureRandom.uuid_v7 })`
+- `markers` - Array of strings(optional). Event markers. You can supply new events with markers list. Persisted events
+  contain assigned markers in this attribute. Example: `PgEventstore::Event.new(markers: %w[foo bar])`
+- `feature_markers` - `Array<PgEventstore::FeatureMarker>`(optional). Event markers, assigned by a middleware(or any
+  other external implementation), but is not going to be persisted along with the rest markers into the default markers
+  metadata key and won't be automatically available under `#markers` when reading events. The main reason for that is to
+  split event type-related markers and feature-specific markers. Your implementation is responsible for serializing and 
+  deserializing those markers. You can see the example of such implementation by inspecting
+  `PgEventstore::Middleware::EventTracing` middleware.
+- `metadata` - `Hash`(optional). Event metadata. Event meta information which is not part of an events data payload.
+  Example: `{ published_by: publishing_user.id }`
+- `link_global_position` - `Integer`(optional, read only). If an event is a link event (link events are pointers to
+  other events), this attribute contains a `global_position` of the original event. Manually assigning this attribute
+  has no effect. It is internally set when linking an event to the given stream or when reading events from the
+  database.
+- `link_partition_id` - `Integer`(optional, read only). If an event is a link event - this attribute contains a
+  partition `id` of original event. Manually assigning this attribute has no effect. It is internally set when appending
+  an event to the given stream or when reading events from the database.
+- `link` - `PgEventstore::Event`(optional, read only). When reading from a stream using `resolve_link_tos: true`, if an
+  event is resolved from a link - this attribute contains a `PgEventstore::Event` object which corresponds to that link.
+  Manually assigning this attribute has no effect. It is internally set when reading events from the database.
+- `created_at` - `Time`(optional, read only). Database's timestamp when an event was appended to a stream. You may want
+  to put your own timestamp into a `metadata` attribute - it may be useful when migrating between different databases.
+  Manually assigning this attribute has no effect. It is internally set when appending an event to the given stream or
+  when reading events from the database.
+- `caused_by` - `PgEventstore::Event`(optional). This is a part of event tracing feature(not configured by default).
+  Pass your persisted event to make your unpersisted event be marked as an event, caused by the given persisted event,
+  thus, creating causation dependency between them
+- `correlation_id` - `String`(UUIDv7, optional). This is a part of event tracing feature(not configured by default). All
+  connected events are marked with the same correlation id. Thus, you can fetch all connected events at once.
+- `causation_id` - `String`(UUIDv7, optional). This is a part of event tracing feature(not configured by default). The
+  id of an event caused current event be persisted.
 
 Example:
 
@@ -29,11 +64,14 @@ PgEventstore::Event.new(data: { 'foo' => 'bar' }, type: 'FooChanged')
 
 ## Stream object
 
-To be able to manipulate a stream, you have to compute a stream's object first. It can be achieved by using the `PgEventstore::Stream` class. Here is a description of its attributes:
+To be able to manipulate a stream, you have to compute a stream's object first. It can be achieved by using the
+`PgEventstore::Stream` class. Here is a description of its attributes:
 
-- `context` - String(required). A Bounded Context, read more [here](https://martinfowler.com/bliki/BoundedContext.html). Values which start from `$` sign are reserved by `pg_eventstore`. Such contexts can't be used to append events.
+- `context` - String(required). A Bounded Context, read more [here](https://martinfowler.com/bliki/BoundedContext.html).
 - `stream_name` - String(required). A stream name.
 - `stream_id` - String(required). A stream id.
+- `stream_revision` - Integer(optional, read only). Current stream revision
+- `starting_position` - Integer(optional, read only). Global position of the first event in the stream
 
 Example:
 
@@ -44,20 +82,6 @@ PgEventstore::Stream.new(context: 'Sales', stream_name: 'Customer', stream_id: '
 
 ### "all" stream
 
-There is a special stream, called the "all" stream. You can get this object by calling the `PgEventstore::Stream.all_stream` method. Read more about the "all" stream in the `Reading from the "all" stream` section of [Reading events](reading_events.md) chapter.
-
-### System streams
-
-System stream is a special stream, the representation of which is pre-defined by the gem. System stream object can be created in next way:
-
-```ruby
-PgEventstore::Stream.system_stream(stream_name)
-```
-
-Current list of system streams is:
-
-- `"$streams"`. Reading from this stream will return 0 revision events. This allows effectively loop through a list of streams. Read more in [Reading events](reading_events.md#streams-stream-filtering) chapter.
-
-## Important note
-
-Because the database is designed for Eventsourcing, some limitations should be met - a combination of `Event#type`, `Stream#context` and `Stream#stream_name` must have low cardinality(low unique values number). This means you should pre-defined values there. Otherwise it may lead to the performance degradation. See [How it works](how_it_works.md) chapter for the details.
+There is a special stream, called the "all" stream. You can get this object by calling the
+`PgEventstore::Stream.all_stream` method. Read more about the "all" stream in the `Reading from the "all" stream`
+section of [Reading events](reading_events.md) chapter.