pg_eventstore 2.0.0 → 3.0.0

data/docs/how_it_works.md

@@ -2,13 +2,24 @@
 
 ## Database architecture
 
-The database is designed specifically for Eventsourcing using Domain-Driven Design. `events` table is partitioned in next way:
+The database is designed specifically for Event Sourcing alongside Domain-Driven Design. Internally, it uses a
+Snowflake-based schema which consists of multiple tables:
+- `events_global_index` is the primary event index fact table
+- `event_markers_index` is a marker-specific factless bridge/index table, and is used as the primary access path for
+  marker-based event queries
+- `streams_global_index` is normalized stream-level dimension/reference table for `events_global_index`
+- `partitions` is a dictionary table for physical event partitions
+- `events` is a table that holds all information about an event - its payload, metadata, markers, etc
+
+`events` table is additionally partitioned in next way:
 
 - For each `Stream#context` there is a subpartition of `events` table. Those tables have `contexts_` prefix.
 - For each `Stream#stream_name` there is a subpartition of `contexts_` table. Those tables have `stream_names_` prefix.
 - For each `Event#type` there is a subpartition of `stream_names_` table. Those tables have `event_types_` prefix.
 
-To implement partitions - Declarative Partitioning is used. Partitioning means that you should not have any random values in the combination of `Stream#context`, `Stream#stream_name` and `Event#type`. A combination of those values must have low cardinality(low distinct values number) and must be pre-defined in your application. Otherwise it will lead to the performance degradation. More about PostgreSQL partitions is [here](https://www.postgresql.org/docs/current/ddl-partitioning.html).
+To implement partitions - Declarative Partitioning feature is used. While the implementation scales well with the number
+of partitions - it is recommended your application pre-defines all combinations of contexts, stream names and event
+types. More about PostgreSQL partitions is [here](https://www.postgresql.org/docs/current/ddl-partitioning.html).
 
 So, let's say you want to publish next event:
 
@@ -20,11 +31,15 @@ PgEventstore.client.append_to_stream(stream, event)
 
 To actually create `events` record next partitions will be created:
 
-- `contexts_81820a` table which is a subpartition of `events` table. It is needed to handle all events which comes to `"SomeCtx"` context
-- `stream_names_ecb803` table which is a subpartition of `contexts_81820a` table. It is needed to handle all events which comes to `"SomeStream"` stream name of `"SomeCtx"` context
-- `event_types_aeadd5` table which is a subpartition of `stream_names_ecb803` table. It is needed to handle all events which have `"SomethingChanged"` event type of `"SomeStream"` stream name of `"SomeCtx"` context
+- `contexts_81820a` table which is a subpartition of `events` table. It is needed to handle all events which comes to
+  `"SomeCtx"` context
+- `stream_names_ecb803` table which is a subpartition of `contexts_81820a` table. It is needed to handle all events
+  which comes to `"SomeStream"` stream name of `"SomeCtx"` context
+- `event_types_aeadd5` table which is a subpartition of `stream_names_ecb803` table. It is needed to handle all events
+  which have `"SomethingChanged"` event type of `"SomeStream"` stream name of `"SomeCtx"` context
 
-You can check all partitions and associated with them contexts, stream names and event types by querying `partitions` table. Example(based on the publish sample above):
+You can check all partitions and associated with them contexts, stream names and event types by querying special service
+table called `partitions`. Example(based on the sample above):
 
 ```ruby
 PgEventstore.connection.with do |conn|
@@ -38,26 +53,47 @@ end.to_a
 
 ### PostgreSQL settings
 
-The more partitions you have, the more locks are required for operations that affect multiple partitions. Especially it concerns the case when you are [reading events from "all" stream](reading_events.md#reading-from-the-all-stream) without providing any filters. It may lead to the next error:
+The more partitions you have, the more locks are required for operations that affect multiple partitions. It mainly
+concerns the case when you involve many different event types when using `Client#multiple`. It may lead to the next
+error:
 
 ```
 ERROR:  out of shared memory (PG::OutOfMemory)
-HINT:  You might need to increase max_locks_per_transaction.
+HINT:  You might need to increase max_pred_locks_per_transaction.
 ```
 
-PostgreSQL suggests to increase the `max_locks_per_transaction`(the description of it is [here](https://www.postgresql.org/docs/current/runtime-config-locks.html)). The default value is `64`. The good value of this setting really depends on your queries, the number of concurrent transactions, the values of `shared_buffers` and `work_mem` settings. In case you have several thousands of partitions - you may want to set it to `128` or event to `256` from the start. On the other hand - you may want to increase it even earlier(e.g. when having several hundreds of partitions) in case you involve high number of partitions into a single transaction(for example, when using [#multiple](multiple_commands.md)).
+PostgreSQL suggests to increase the `max_pred_locks_per_transaction`(the description of it
+is [here](https://www.postgresql.org/docs/current/runtime-config-locks.html)). The default value is `64`. In case you
+have several thousands of partitions - you may want to set it to `128` or even to `256`.
+
+You may also face similar error which refers to `max_locks_per_transaction` setting:
+
+```
+PG::OutOfMemory: ERROR:  out of shared memory (PG::OutOfMemory)
+HINT:  You might need to increase "max_locks_per_transaction".
+```
 
-Conclusion: monitor db logs, monitor exceptions and adjust your db settings accordingly.
+The reason of it to appear is the same - too many objects(partition tables, indexes, etc) are involved in a single
+transaction.
 
 ## Appending events and multiple commands
 
-You may want to get familiar with [Appending events](appending_events.md) and [multiple commands](multiple_commands.md) first.
+You may want to get familiar with [Appending events](appending_events.md) and [multiple commands](multiple_commands.md)
+first.
 
-`pg_eventstore` internally uses `Serializable` transaction isolation level(more about different transaction isolation levels in PostgreSQL is [here](https://www.postgresql.org/docs/current/transaction-iso.html)). On practice this means that any transaction may fail with serialization error, and the common approach is to restart this transaction. For ruby this means re-execution of the block of code. Which is why there is a warning regarding potential block re-execution when using `#multiple`. However current implementation allows to limit 99% of retries to the manipulations with one stream. For example, when two parallel processes changing the same stream. If different streams are being changed at the same time - it is less likely it would perform retry.
+`pg_eventstore` internally uses `Serializable` and `Repeatable read` transaction isolation levels depending on the API
+method(more about different transaction isolation levels in PostgreSQL is [here](https://www.postgresql.org/docs/current/transaction-iso.html)). On practice this means that any
+transaction may fail with serialization error, and the common approach is to restart this transaction. For ruby this
+means re-execution of the block of code. Which is why there is a warning regarding potential code re-execution when
+using `#multiple`. However, current implementation allows to limit 99% of retries to the manipulations with one stream.
+For example, when two parallel processes changing the same stream. If different streams are being changed at the same
+time - it is less likely it would perform retry.
 
 Examples:
 
-- if "process 1" and "process 2" perform the append command at the same time - one of the append commands will be retried:
+- if "process 1" and "process 2" perform the append command at the same time - one of the append commands will be
+  retried:
+
 ```ruby
 # process 1
 stream = PgEventstore::Stream.new(context: 'MyCtx', stream_name: 'MyStream', stream_id: '1')
@@ -70,7 +106,8 @@ event = PgEventstore::Event.new(type: 'SomethingElseChanged', data: { baz: :bar
 PgEventstore.client.append_to_stream(stream, event)
 ```
 
-- if "process 1" performs multiple commands at the same time "process 2" performs append command which involves the same stream from "process 1" - either block of `#multiple` or `#append_to_stream` will be retried:
+- if "process 1" performs multiple commands at the same time "process 2" performs append command which involves the same
+  stream from "process 1" - either block of `#multiple` or `#append_to_stream` will be retried:
 
 ```ruby
 # process 1
@@ -88,6 +125,10 @@ event = PgEventstore::Event.new(type: 'SomethingChanged', data: { foo: :bar })
 PgEventstore.client.append_to_stream(stream2, event)
 ```
 
-Retries also concern your potential implementation of [middlewares](writing_middleware.md). For example, `YourAwesomeMiddleware#serialize` can be executed several times when append the event. This is especially important when you involve your microservices here - they can receive the same payload several times.
+Retries also concern your potential implementation of [middlewares](writing_middleware.md). For example,
+`YourAwesomeMiddleware#serialize` can be executed several times when appending an event. This is especially important
+when you involve your microservices here - they can receive the same payload several times.
 
-Conclusion. When developing using `pg_eventstore` - always keep in mind that some parts of your implementation can be executed several times before successfully publishing an event, or event when reading events(`#deserializa` middleware method) if you perform reading withing `#multiple` block. 
+Conclusion. When developing using `pg_eventstore` - always keep in mind that some parts of your implementation can be
+executed several times before successfully publishing an event, or event when reading events(`#deserializa` middleware
+method) if you perform reading withing `#multiple` block. Thus, make sure your middlewares are idempotent.

data/docs/linking_events.md

@@ -54,6 +54,7 @@ Example:
 
 ```ruby
 require 'securerandom'
+
 class SomethingHappened < PgEventstore::Event
 end
 
@@ -61,7 +62,7 @@ event1 = SomethingHappened.new
 event2 = SomethingHappened.new
 
 events_stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'MyAwesomeStream', stream_id: '1')
-projection_stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'MyAwesomeProjection', stream_id: SecureRandom.uuid)
+projection_stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'MyAwesomeProjection', stream_id: SecureRandom.uuid_v7)
 
 event1, event2 = PgEventstore.client.append_to_stream(events_stream, [event1, event2])
 

data/docs/maintenance.md

@@ -9,14 +9,16 @@ stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'MyStream', st
 PgEventstore.client.append_to_stream(stream, PgEventstore::Event.new)
 PgEventstore.maintenance.delete_stream(stream) # => true
 ```
-deletes all events in the given stream. If the given stream does not exist - `false` is returned: 
+
+deletes all events in the given stream. If the given stream does not exist - `false` is returned:
 
 ```ruby
 stream = PgEventstore::Stream.new(context: 'NonExistingCtx', stream_name: 'NonExistingStream', stream_id: '1')
 PgEventstore.maintenance.delete_stream(stream) # => false
 ```
 
-**Please note that this operation does not automatically delete links to deleted events - you have to do delete them separately.**
+**Please note that this operation does not automatically delete links to deleted events - you have to do delete them
+separately.**
 
 ## Delete an event
 
@@ -27,6 +29,7 @@ PgEventstore.client.append_to_stream(stream, PgEventstore::Event.new)
 event = PgEventstore.client.read(stream, options: { max_count: 1 }).first
 PgEventstore.maintenance.delete_event(event) # => true
 ```
+
 deletes the given event. If the given event does not exist - `false` is returned:
 
 ```ruby
@@ -38,14 +41,20 @@ PgEventstore.maintenance.delete_event(event) # => true
 PgEventstore.maintenance.delete_event(event) # => false
 ```
 
-**Please note that this operation does not automatically delete links to the deleted event - you have to do delete them separately.**
+**Please note that this operation does not automatically delete links to the deleted event - you have to do delete them
+separately.**
 
 ### Deleting an event in a large stream
 
-There can be a situation when you would like to delete an event in a large stream. If an event is in the position where there are 1000+ events after it in a stream - a `PgEventstore::TooManyRecordsToLockError` error will be raised. This is because in addition to removing the event, we need to adjust the rest of the stream by updating the `#stream_revision` of all events that come after this event. To overcome this limitation - you can provide `force: true` flag:
+There can be a situation when you would like to delete an event in a large stream. If an event is in the position where
+there are 1000+ events after it in a stream - a `PgEventstore::TooManyRecordsToLockError` error will be raised. This is
+because in addition to removing the event, we need to adjust the rest of the stream by updating the `#stream_revision`
+of all events that come after this event. To overcome this limitation - you can provide `force: true` flag:
 
 ```ruby
 PgEventstore.maintenance.delete_event(event, force: true)
 ```
 
-Because the update operation can lock thousands of events - `#delete_event` can be slow and can slow down other processes which insert events in the same stream at the same moment. **Thus, it is recommended to avoid using `#delete_event` in your business logic.**
+Because the update operation can lock thousands of events - `#delete_event` can be slow and can slow down other
+processes which insert events in the same stream at the same moment. **Thus, it is recommended to avoid
+using `#delete_event` in your business logic.**

data/docs/multiple_commands.md

@@ -1,17 +1,19 @@
 # Multiple commands
 
-`pg_eventstore` implements the `#multiple` method to allow you to make several different commands atomic. Internally it executes the given block within PostgreSQL transaction. Example:
+`pg_eventstore` implements the `#multiple` method to allow you to make several different commands atomic. Internally it
+executes the given block within PostgreSQL transaction. Example:
 
 ```ruby
 PgEventstore.client.multiple do
   unless PgEventstore.client.read(stream3, options: { max_count: 1, direction: 'Backwards' }).last&.type == 'Removed'
     PgEventstore.client.append_to_stream(stream1, event1)
     PgEventstore.client.append_to_stream(stream2, event2)
-  end  
+  end
 end
 ```
 
-Optionally, you can provide `read_only: true` argument to run the transaction in read-only mode. This, however, will raise `PG::ReadOnlySqlTransaction` exception if any mutating query is executed within the block. Example:
+Optionally, you can provide `read_only: true` argument to run the transaction in read-only mode. This, however, will
+raise `PG::ReadOnlySqlTransaction` exception if any mutating query is executed within the block. Example:
 
 ```ruby
 # Good
@@ -25,10 +27,15 @@ PgEventstore.client.multiple(read_only: true) do
 end
 ```
 
+All commands inside a `multiple` block either all succeed or all fail. This allows you to easily implement complex
+business rules. However, it comes with a price of performance. The more you put in a single block, the higher the chance
+it will have conflicts with other commands run in parallel, increasing overall time to complete. **Because of this
+performance implications, do not put more events than needed in a `multple` block.** You may still want to use it though
+as it could simplify your implementation.
 
-All commands inside a `multiple` block either all succeed or all fail. This allows you to easily implement complex business rules. However, it comes with a price of performance. The more you put in a single block, the higher the chance it will have conflicts with other commands run in parallel, increasing overall time to complete. **Because of this performance implications, do not put more events than needed in a `multple` block.** You may still want to use it though as it could simplify your implementation.
-
-**Please take into account that due to concurrency of parallel commands, a block of code may be re-run several times before succeeding.** So, if you put any piece of code besides `pg_evenstore`'s commands - make sure it is ready for re-runs. A good and a bad examples:
+**Please take into account that due to concurrency of parallel commands, a block of code may be re-run several times
+before succeeding.** So, if you put any piece of code besides `pg_evenstore`'s commands - make sure it is ready for
+re-runs. A good and a bad examples:
 
 ## Bad
 
@@ -37,7 +44,7 @@ PgEventstore.client.multiple do
   old_email = PgEventstore.client.read(user_stream, options: { filter: { event_types: ['UserEmailChanged'] }, max_count: 1, direction: 'Backwards' }).first&.data&.dig('email')
   # Email hasn't changed - prevent publishing unnecessary changes
   next if old_email == user.email
-  
+
   PgEventstore.client.append_to_stream(user_stream, UserEmailChanged.new(data: { email: user.email }))
   # This is the mistake. UserMailer.notify_email_changed may be triggered several times
   UserMailer.notify_email_changed(user.id, old_email: old_email, new_email: user.email).deliver_later
@@ -62,4 +69,6 @@ UserMailer.notify_email_changed(user.id, old_email: old_email, new_email: user.e
 
 ## Side effect of internal implementation
 
-Please note that when publishing an event with a type as part of a `multiple` block that does not yet exist in the database, the block will run twice as the first attempt to publish will always fail due to the way `append_to_stream` is implemented. Consider this when writing expectations in your tests for example.
+Please note that when publishing an event with a type as part of a `multiple` block that does not yet exist in the
+database, the block will run twice as the first attempt to publish will always fail due to the way `append_to_stream` is
+implemented. Consider this when writing expectations in your tests for example.

data/docs/reading_events.md

@@ -12,7 +12,9 @@ PgEventstore.client.read(stream)
 
 ### max_count
 
-You can provide the `:max_count` option. This option determines how many records to return in a response. Default is `1000` and it can be changed with the `:max_count` configuration setting (see [**"Configuration"**](configuration.md) chapter):
+You can provide the `:max_count` option. This option determines how many records to return in a response. Default is
+`1000` and it can be changed with the `:max_count` configuration setting (see [**"Configuration"**](configuration.md)
+chapter):
 
 ```ruby
 stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
@@ -21,7 +23,8 @@ PgEventstore.client.read(stream, options: { max_count: 100 })
 
 ### resolve_link_tos
 
-When reading streams with projected events (links to other events) you can chose to resolve those links by setting `resolve_link_tos` to `true`, returning the original event instead of the "link" event.
+When reading streams with projected events (links to other events) you can chose to resolve those links by setting
+`resolve_link_tos` to `true`, returning the original event instead of the "link" event.
 
 ```ruby
 stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
@@ -39,7 +42,8 @@ PgEventstore.client.read(stream, options: { from_revision: 2 })
 
 ### direction
 
-As well as being able to read a stream forwards you can also go backwards. This can be achieved by providing the `:direction` option:
+As well as being able to read a stream forwards you can also go backwards. This can be achieved by providing the
+`:direction` option:
 
 ```ruby
 stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
@@ -62,7 +66,8 @@ end
 
 ## Reading from the "all" stream
 
-"all" stream definition means that you don't scope your events when reading them from the database. To get the "all" `PgEventstore::Stream` instance you have to call the `all_stream` method: 
+"all" stream definition means that you don't scope your events when reading them from the database. To get the "all"
+`PgEventstore::Stream` instance you have to call the `all_stream` method:
 
 ```ruby
 PgEventstore::Stream.all_stream
@@ -74,24 +79,18 @@ Now you can use it to read from the "all" stream:
 PgEventstore.client.read(PgEventstore::Stream.all_stream)
 ```
 
-You can read from a specific position of the "all" stream. This is very similar to reading from a specific revision of a specific stream, but instead of the `:from_revision` option you have to provide the `:from_position` option:
+You can read from a specific position of the "all" stream. This is very similar to reading from a specific revision of a
+specific stream, but instead of the `:from_revision` option you have to provide the `:from_position` option:
 
 ```ruby
 PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { from_position: 9023, direction: 'Backwards' })
 ```
 
-## Reading from "$streams" system stream
-
-`"$streams"` is a special stream which consists of events with `stream_revision == 0`. This allows you to effectively query all streams. Example:
-
-```ruby
-stream = PgEventstore::Stream.system_stream("$streams")
-PgEventstore.client.read(stream).map(&:stream) # => array of unique streams
-```
-
 ## Middlewares
 
-If you would like to skip some of your registered middlewares from processing events after they being read from a stream - you should use the `:middlewares` argument which allows you to override the list of middlewares you would like to use.
+If you would like to skip some of your registered middlewares from processing events after they being read from a
+stream - you should use the `:middlewares` argument which allows you to override the list of middlewares you would like
+to use.
 
 Let's say you have these registered middlewares:
 
@@ -101,7 +100,8 @@ PgEventstore.configure do |config|
 end
 ```
 
-And you want to skip `FooMiddleware` and `BazMiddleware`. You simply have to provide an array of corresponding middleware keys you would like to use:
+And you want to skip `FooMiddleware` and `BazMiddleware`. You simply have to provide an array of corresponding
+middleware keys you would like to use:
 
 ```ruby
 PgEventstore.client.read(PgEventstore::Stream.all_stream, middlewares: %i[bar])
@@ -111,78 +111,239 @@ See [Writing middleware](writing_middleware.md) chapter for info about what is m
 
 ## Filtering
 
-When reading events, you can additionally filter the result. Available attributes for filtering depend on the type of stream you are reading from. Reading from the "all" stream supports filters by stream attributes and event types. Reading from a specific stream supports filters by event types only.  
+When reading events, you can additionally filter the result. Available attributes for filtering depend on the type of
+stream you are reading from. Reading from the "all" stream supports filters by stream attributes, event types and event
+markers. Reading from a specific stream supports filters by event types and event markers only.
 
 ### Specific stream filtering
 
-Filtering events by their types:
+#### Filtering events by their types
 
+Read `Foo` and `Bar` events from the given stream:
 ```ruby
-stream = PgEventstore::Stream.new(context: 'MYAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
 PgEventstore.client.read(stream, options: { filter: { event_types: %w[Foo Bar] } })
 ```
 
+Also, you can provide event type values using `{ type: 'MyType' }` syntax. Example:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+PgEventstore.client.read(stream, options: { filter: { event_types: [{ type: 'Foo' }, { type: 'Bar' }] } })
+```
+
+#### Filtering events by markers
+
+Read events, marked as either `'foo'` **or** `'bar'`:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+PgEventstore.client.read(stream, options: { filter: { event_types: [{ markers: %w[foo bar] }] } })
+```
+
+##### Limitations
+
+Please note - there is no functional to filter events that are marked with both `'foo'` **and** `'bar'` markers. If you
+need a combination of both - you have to create another event marker - for example `'foobar'` - and additionally mark
+events using it that have both - `'foo'` and `'bar'` markers. Example:
+
+```ruby
+event1 = PgEventstore::Event.new(markers: ['foo'])
+event2 = PgEventstore::Event.new(markers: ['bar'])
+event3 = PgEventstore::Event.new(markers: %w[foo bar foobar])
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '3')
+PgEventstore.client.append_to_stream(stream, [event1, event2, event3])
+# Returns all three events
+PgEventstore.client.read(stream, options: { filter: { event_types: [{ markers: %w[foo bar] }] } })
+# Returns only the third event
+PgEventstore.client.read(stream, options: { filter: { event_types: [{ markers: ['foobar'] }] } })
+```
+
+#### Filtering events by event types with markers
+
+You can provide per event type markers list. The effect of this is that each event type is additionally restricted with
+the set of markers. In next example events with `'Foo'` type is only returned when they are additionally marked with
+`'foo'` or `'bar'` markers:
+
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+PgEventstore.client.read(stream, options: { filter: { event_types: [{ type: 'Foo', markers: %w[foo bar] }] } })
+```
+
+You can also combine event types filter, event type with markers filter and markers filter. Next query returns all
+events that are either:
+- marked with `'baz'`
+- or have type `'Foo'` that are marked with either `'foo'` or `'bar'`
+- or have type `'Bar'`
+```ruby
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
+PgEventstore.client.read(
+  stream,
+  options: {
+    filter: {
+      event_types: [
+        { markers: ['baz'] },
+        { type: 'Foo', markers: %w[foo bar] },
+        { type: 'Bar' }
+      ]
+    }
+  }
+)
+```
+
 ### "all" stream filtering
 
-**Warning** There is a restriction on a set of stream attributes that can be used when filtering an "all" stream result. Available combinations:
+#### Limitations
+
+There is a limitation on a set of stream attributes that can be used when filtering an "all" stream result.
+Available combinations:
 
 - `:context`
 - `:context` and `:stream_name`
 - `:context`, `:stream_name` and `:stream_id`
 
-All other combinations, like providing only `:stream_name` or providing `:context` with `:stream_id` will be ignored.
+So this is not allowed(filter is simply ignored):
 
+```ruby
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ stream_name: 'FooCtx' }] } }
+) # => Array of events like if there is no filter at all
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ stream_name: 'FooCtx', stream_id: '1' }] } }
+) # => Array of events like if there is no filter at all
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ stream_id: '1' }] } }
+) # => Array of events like if there is no filter at all
+```
 
-Filtering events by type:
+but this is allowed:
 
 ```ruby
-PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { event_types: %w[Foo Bar] } })
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'FooCtx' }] } }
+)
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'FooCtx', stream_name: 'Foo' }] } }
+)
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'FooCtx', stream_name: 'Foo', stream_id: '1' }] } }
+)
+```
+
+Also, you can't filter by markers only without a specific event type with only `:context` or `:context` and
+`:stream_name`. So, this is not allowed:
+
+```ruby
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'FooCtx' }], event_types: [{ markers: ['foo'] }] } }
+) # => exception
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'FooCtx', stream_name: 'Foo' }], event_types: [{ markers: ['foo'] }] } }
+) # => exception
+```
+
+but this is allowed:
+
+```ruby
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'FooCtx' }], event_types: [{ markers: ['foo'], type: 'Foo' }] } }
+) # => Array of events
 ```
 
-Filtering events by context:
+#### Filtering events by type and/or markers
+
+Please refer to the documentation of [Specific stream filtering](reading_events.md#reading-from-a-specific-stream) for
+more examples how to filter by event types and/or markers. The only thing which changes is stream argument. To filter
+"all" stream - you have to provide `PgEventstore::Stream.all_stream` instead of specific stream. Example:
 
 ```ruby
-PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext' }] } })
+PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { event_types: %w[Foo Bar] } })
 ```
 
-Filtering events by context and name:
+#### Filtering events by context
 
 ```ruby
-PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext', stream_name: 'User' }] } })
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'MyAwesomeContext' }] } }
+)
 ```
 
-Filtering events by stream context, stream name and stream id:
+#### Filtering events by context and name
 
 ```ruby
-PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae' }] } })
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: { filter: { streams: [{ context: 'MyAwesomeContext', stream_name: 'User' }] } }
+)
 ```
 
-You can provide several sets of stream's attributes. The result will be a union of events that match those criteria. For example, next query will return all events that belong to streams with `AnotherContext` context and all events that belong to streams with `MyAwesomeContext` context and `User` stream name:
+#### Filtering events by stream context, stream name and stream id
 
 ```ruby
-PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'AnotherContext' }, { context: 'MyAwesomeContext', stream_name: 'User' }] } })
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: {
+    filter: {
+      streams: [{ context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae' }]
+    }
+  }
+)
 ```
 
-You can also mix filtering by stream's attributes and event types. The result will be intersection of events matching stream's attributes and event's types. For example, next query will return events which type is either `Foo` or `Bar` and which belong to a stream with `MyAwesomeContext` context:
+You can provide several sets of stream's attributes. The result will be a union of events that match those criteria. For
+example, next query will return all events that belong to streams with `AnotherContext` context and all events that
+belong to streams with `MyAwesomeContext` context and `User` stream name:
 
 ```ruby
-PgEventstore.client.read(PgEventstore::Stream.all_stream, options: { filter: { streams: [{ context: 'MyAwesomeContext' }], event_types: %w[Foo Bar] } })
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: {
+    filter: {
+      streams: [{ context: 'AnotherContext' }, { context: 'MyAwesomeContext', stream_name: 'User' }] 
+    }
+  }
+)
 ```
 
-### "$streams" stream filtering
+#### Filtering events by stream attributes, event types and/or markers
+
+Please refer to the documentation of [Specific stream filtering](reading_events.md#reading-from-a-specific-stream) for
+more examples how to filter by event types and/or markers.
+
+You can also mix filtering by stream attributes, event types and markers. The result is union of combinations of 
+each stream filter and event types filter. For example, next query returns events with type either `Foo`, marked
+by `'foo'` marker, or `Bar` and belong to streams from `FooCtx` or `BarCtx` contexts:
 
-When reading from `"$streams"` same rules apply as when reading from "all" stream. For example, read all streams which have `context == "MyAwesomeContext"` and start from events with event type either `"Foo"` or `"Bar"`:
 ```ruby
-PgEventstore.client.read(PgEventstore::Stream.system_stream("$streams"), options: { filter: { streams: [{ context: 'MyAwesomeContext' }], event_types: %w[Foo Bar] } })
+PgEventstore.client.read(
+  PgEventstore::Stream.all_stream,
+  options: {
+    filter: {
+      streams: [{ context: 'FooCtx' }, { context: 'BarCtx' }],
+      event_types: [{ type: 'Foo', markers: ['foo'] }, { type: 'Bar' }]
+    }
+  }
+)
 ```
 
 ## Pagination
 
-You can use `#read_paginated` to iterate over all (filtered) events. It yields each batch of records that was found according to the filter options:
+You can use `#read_paginated` to iterate over all (filtered) events. It yields each batch of records that was found
+according to the filter options:
 
 ```ruby
 # Read from the specific stream
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
 PgEventstore.client.read_paginated(stream).each do |events|
   events.each do |event|
     # iterate through events
@@ -201,7 +362,7 @@ Options are the same as for `#read` method. Several examples:
 
 ```ruby
 # Read "Foo" events only from the specific stream
-stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'User', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae')
+stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')
 PgEventstore.client.read_paginated(stream, options: { filter: { event_types: ['Foo'] } }).each do |events|
   events.each do |event|
     # iterate through events
@@ -223,7 +384,7 @@ PgEventstore.client.read_paginated(PgEventstore::Stream.all_stream, options: { m
 end
 
 # Reading from projection stream
-projection_stream = PgEventstore::Stream.new(context: 'MyAwesomeContext', stream_name: 'MyAwesomeProjection', stream_id: 'f37b82f2-4152-424d-ab6b-0cc6f0a53aae') 
+projection_stream = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'FooProjection', stream_id: '1')
 PgEventstore.client.read_paginated(projection_stream, options: { resolve_link_tos: true }).each do |events|
   events.each do |event|
     # iterate through events
@@ -246,7 +407,8 @@ PgEventstore.client.append_to_stream(stream, [event1, event2, event3])
 PgEventstore.client.read_grouped(stream) # => returns event1 and event3
 ```
 
-API is very similar to the API of `#read`, but it ignores `:max_count` options as the result is always returns a set of event types in your stream.
+API is very similar to the API of `#read`, but it ignores `:max_count` options as the result is always returns a set of
+event types in your stream.
 
 Reading most recent events:
 
@@ -307,11 +469,14 @@ PgEventstore.client.read_grouped(PgEventstore::Stream.all_stream, options: { dir
 
 ### Event types list lookup
 
-If you do not provide event types filter - event types list will be determined based on the rest of arguments(a stream argument or a stream filters option).
+If you do not provide event types filter - event types list will be determined based on the rest of arguments(a stream
+argument or a stream filters option).
 
 ### Multiple events of same type in the result
 
-If same event type appear in different streams(different by `#context` and `#stream_name`) - those events will appear in the result. This is because even though `Event#type` value may be the same - its meaning may have different meaning in different `context`/`stream_name` couple. Example:
+If same event type appear in different streams(different by `#context` and `#stream_name`) - those events will appear in
+the result. This is because even though `Event#type` value may be the same - its meaning may have different meaning in
+different `context`/`stream_name` couple. Example:
 
 ```ruby
 stream1 = PgEventstore::Stream.new(context: 'FooCtx', stream_name: 'Foo', stream_id: '1')