pg_eventstore 3.0.1 → 3.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 24899ca6d10bac21341f41f3a8cfee09e10ce10bf81fb4525e9660f6c708d3af
-  data.tar.gz: 07cbbca86a3cb488d9123d663e77cb66cdbf850f93e12f1f15e46dc8f465879c
+  metadata.gz: b54fbf71f1dd082e5895ab8b1431f2a92442c0d0865e656b4fe65430503151b5
+  data.tar.gz: 45a0bfd8a606466495643cb41f503643bd6151f915d1e36cdf374eded7dab8b3
 SHA512:
-  metadata.gz: bc33cc4bcdda545ffa4476e80b55ca2541de4da036d715644f3957d1889170bbdf3b05b1391c7f981f28163da0ce36d06b0a8282ddeddb7560eb1892df43cfaa
-  data.tar.gz: 64b63413791d2c993315d4012acb1a5bec19b237eae6761c6b9da8725483e4284f5fbf3b956df5c4a750719460087f30cec6c9db1f176ce728ad985f2aefa213
+  metadata.gz: 59028bfa802198da04912ba0f339a165eb3d9c35e0cb940b9fb0437ae42909f3570b2f4bbda88d5234fb622420d4ff7425dd1fafa4a8951b4092879ca33fcf77
+  data.tar.gz: 671e391234a4f8b6a008b19bdbc533c94a7e3cc562fab9cdfd4dbef77968fd2ed03814f7ff132a8326aee859cc2e65db04ef63a8667e2e67b18615b102e08102

data/CHANGELOG.md

@@ -1,5 +1,22 @@
 ## [Unreleased]
 
+## [3.1.2]
+
+- Fix subscriptions skipping events when the estimated number of events to fetch drops to 0 during the pull
+
+## [3.1.1]
+
+- Prevent arbitrary order in replica queries
+
+## [3.1.0]
+
+- Hide "Delete stream" and "Delete" event buttons in admin UI for non-maintenance eventstore roles
+- New feature: Prometheus metrics endpoints for subscriptions observability (lag, health, throughput), served by the
+  standalone `PgEventstore::Web::Metrics::Application` rack application. Read more [here](docs/metrics.md)
+- Properly report internal errors via `failed_subscription_notifier` config option
+- Fix inaccuracy that could lead to mistakenly handle connection error as subscription handler error
+- Fix admin UI links to take into account mount path instead building a path from `'/'`
+
 ## [3.0.1]
 
 - Fix EventTracing bug: Respect user-supplied Event#correlation_id value when caused_by#correlation_id is absent
@@ -14,8 +31,10 @@
   middleware by default. Previously this wasn't the case - the deserialization phase was skipped. Such behavior was
   creating ambiguity about assumptions when deserialization happens. If you need old behavior - you have to create
   another middleware class which skips deserialization when publishing events and use it instead. Example:
-Let's say you have this middleware configured
+  Let's say you have this middleware configured
+
 ```ruby
+
 class MyMiddleware
   include PgEventstore::Middleware
 
@@ -36,9 +55,9 @@ end
 Now define another middleware that has empty `#deserialize` method
 
 ```ruby
+
 class MyMiddlewareWithoutDeserialize < MyMiddleware
-  def deserialize(event)
-  end
+  def deserialize(event) end
 end
 
 PgEventstore.configure do |config|
@@ -48,6 +67,7 @@ PgEventstore.configure do |config|
   }
 end
 ```
+
 and use only it when publishing events:
 
 ```ruby
@@ -76,12 +96,13 @@ PgEventstore.client(:write).append_to_stream(stream, event)
 - **Breaking change**: `Event#id` uniqueness is no longer guaranteed. It was dropped because there is not much usage of
   it internally. The default value was moved from the database(it was `gen_random_uuid()`) to the application level and
   is `SecureRandom.uuid_v7` now.
-- **Breaking change**: event types, stream attributes, event metadata keys, markers that start from `▒`(`"\u2592"` 
-Unicode character) character are now reserved by pg_eventstore. It is less likely you have any, but if you do - you have
-to adjust your implementation to no longer rely on it.
+- **Breaking change**: event types, stream attributes, event metadata keys, markers that start from `▒`(`"\u2592"`
+  Unicode character) character are now reserved by pg_eventstore. It is less likely you have any, but if you do - you
+  have
+  to adjust your implementation to no longer rely on it.
 - New feature: event markers. You can now mark an event and later use those markers to read events, build a projection
-using subscriptions or validate stream revision scoped to an event type and specific markers when publishing events as 
-a part of Dynamic Consistency Boundaries. Find more in [docs](docs/appending_events.md#event-markers)
+  using subscriptions or validate stream revision scoped to an event type and specific markers when publishing events as
+  a part of Dynamic Consistency Boundaries. Find more in [docs](docs/appending_events.md#event-markers)
 - New config option `config.events_subscription_position_update_interval`. See more
   in [Configuration](docs/configuration.md) docs
 - **Breaking change**: `:from_position` option of `SubscriptionManager#subscribe`, "Current position" column in admin

data/README.md

@@ -66,6 +66,7 @@ Documentation chapters:
 - [Writing middlewares](docs/writing_middleware.md)
 - [How to make multiple commands atomic](docs/multiple_commands.md)
 - [Admin UI](docs/admin_ui.md)
+- [Prometheus metrics](docs/metrics.md)
 - [Replication](docs/replication.md)
 
 To help your AI assistant to better recognize the capabilities of this gem - there is a crafted [instructions](docs/AGENTS.fragment.md)

data/docs/admin_ui.md

@@ -41,3 +41,8 @@ run PgEventstore::Web::Application
 ```
 
 Now you can use any web server to run it.
+
+## Prometheus metrics
+
+Subscriptions observability data in the Prometheus text format is served by a separate rack application, mounted
+independently of the Admin UI. See [Metrics](metrics.md).

data/docs/metrics.md

@@ -0,0 +1,233 @@
+# Prometheus metrics
+
+`pg_eventstore` ships rack endpoints which expose subscriptions observability data in the
+[Prometheus text exposition format](https://prometheus.io/docs/instrumenting/exposition_formats/). They answer the
+questions the [Admin UI](admin_ui.md) subscriptions page answers, but in a form Prometheus can scrape and Grafana can
+graph and alert on.
+
+## Endpoints
+
+Metrics are grouped into domains, and split per path within a domain so that each scrape runs only the query it
+needs and cheap families can be polled at a different interval than the expensive one. Paths below are relative to
+wherever the app is mounted - mounted at `/pg_eventstore/metrics` the first one is
+`/pg_eventstore/metrics/subscriptions/latency`.
+
+| Path | Metrics | Query cost |
+|---|---|---|
+| `/subscriptions/latency` | lag of every reported subscription + store positions | the only one looking at event positions (one index range scan per subscription) |
+| `/subscriptions/health` | state, lock, heartbeat age, restarts, last error age | single read of the `subscriptions` table |
+| `/subscriptions/throughput` | processed events counter, handler capacity | single read of the `subscriptions` table |
+| `/subscriptions` | all of the above | all of the above |
+
+The domain path (`/subscriptions`) is meant for humans and ad-hoc checks; point your scrape jobs at the split paths.
+
+**There is deliberately no route serving every domain at once.** `/metrics` is the Prometheus convention, so a route
+there would invite pointing scrape jobs at it by reflex - and every such scrape would pay for every query, including
+the expensive ones, which is exactly what the per-path split exists to avoid. Keeping aggregates per domain also
+keeps each response bounded as more domains are added.
+
+Every query is guarded by a `statement_timeout` of 5 seconds - a stuck scrape fails visibly instead of piling up on
+the database.
+
+### Which subscriptions are reported
+
+Every subscription in the queried database, unless you narrow it down with `set` params (see
+[Reporting only some subscription sets](#reporting-only-some-subscription-sets)). Note that the subscriptions registry
+never removes rows, so handlers that were renamed or removed keep theirs and are reported too.
+
+A subscription that died *without releasing its lock* is reported like any other - detecting it is what
+`pg_eventstore_subscription_heartbeat_age_seconds` is for, and it is the most useful thing to alert on: neither the
+state column nor the lock can be trusted to notice a process that went away.
+
+## Mounting
+
+`PgEventstore::Web::Metrics::Application` is a separate rack application designed to be a scrape target. In your
+`config/routes.rb`:
+
+```ruby
+require 'pg_eventstore/web'
+
+mount PgEventstore::Web::Metrics::Application, at: '/pg_eventstore/metrics'
+```
+
+Or as a standalone `config.ru`:
+
+```ruby
+require 'pg_eventstore/web'
+
+run PgEventstore::Web::Metrics::Application
+```
+
+### Authorization
+
+The application ships without authentication - how you protect the endpoint is up to you, exactly as it is for the
+[Admin UI](admin_ui.md#authorization). Wrap it in whatever middleware your setup already uses, for example:
+
+```ruby
+metrics_app = Rack::Builder.new do
+  use Rack::Auth::Basic do |_username, password|
+    Rack::Utils.secure_compare(ENV.fetch('PG_EVENTSTORE_METRICS_PASSWORD'), password)
+  end
+  run PgEventstore::Web::Metrics::Application
+end
+
+mount metrics_app, at: '/pg_eventstore/metrics'
+```
+
+### Choosing the database
+
+Which store is queried is decided per request by the `config` query param, so one mounted application can serve the
+metrics of every configured database:
+
+```ruby
+PgEventstore.configure(name: :db1) do |config|
+  config.pg_uri = ENV['DB1_URI']
+  config.connection_pool_size = 1
+end
+
+PgEventstore.configure(name: :db2) do |config|
+  config.pg_uri = ENV['DB2_URI']
+  config.connection_pool_size = 1
+end
+```
+
+```
+GET /pg_eventstore/metrics/subscriptions/latency?config=db1
+```
+
+A missing `config` means the default configuration. An unknown `config` is answered with `404` instead of being
+served from the default store, so a typo in the scrape config shows up as a failing target rather than as another
+database's numbers under the wrong labels.
+
+### Reporting only some subscription sets
+
+Rows of the subscriptions registry are never removed, so a database that has been running for a while also holds
+handlers that were renamed, removed, or never ran against it. Pass one or more `set` params to report only the
+subscriptions you care about:
+
+```
+GET /pg_eventstore/metrics/subscriptions/health?set=MyAppSet&set=MyOtherSet
+GET /pg_eventstore/metrics/subscriptions/health?set=MyAppSet,MyOtherSet
+```
+
+Both forms work - a repeated param (what Prometheus produces from `params: {set: [...]}`) and a comma separated list.
+
+Without a `set` param every subscription in the database is reported. `pg_eventstore` puts no constraints on how
+subscription sets are named, but it is convenient to name a set after the application that owns it - scoping the
+scrape by set is then the straightforward way to keep one application's dashboards to its own subscriptions.
+
+## Prometheus scrape config
+
+```yaml
+scrape_configs:
+  - job_name: 'pg-eventstore-subscriptions-latency'
+    metrics_path: /pg_eventstore/metrics/subscriptions/latency
+    scrape_interval: 30s
+    params:
+      config: ['db1']
+      set: ['MyAppSet']
+    static_configs:
+      - targets: ['your-app-host:port']
+  - job_name: 'pg-eventstore-subscriptions-health'
+    metrics_path: /pg_eventstore/metrics/subscriptions/health
+    scrape_interval: 30s
+    params:
+      config: ['db1']
+      set: ['MyAppSet']
+    static_configs:
+      - targets: ['your-app-host:port']
+  - job_name: 'pg-eventstore-subscriptions-throughput'
+    metrics_path: /pg_eventstore/metrics/subscriptions/throughput
+    scrape_interval: 60s
+    params:
+      config: ['db1']
+      set: ['MyAppSet']
+    static_configs:
+      - targets: ['your-app-host:port']
+```
+
+Add whatever credentials your chosen protection needs to these jobs - Prometheus supports `basic_auth`,
+`authorization` and `tls_config` per job.
+
+The split into three jobs is intentional - do not collapse them into a single `/subscriptions` scrape unless you
+are fine with every scrape paying the latency query.
+
+## Metrics reference
+
+All per-subscription metrics carry `set` and `name` labels.
+
+### Latency
+
+`pg_eventstore_subscription_lag_events` (gauge)
+
+How many events the subscription still has to catch up on before it reaches the edge of the `"all"` stream and starts
+processing newly appended events.
+
+Note on filters: a subscription's filter is accounted for by the store, so a caught-up subscription reports `0` no
+matter how narrow its filter is - traffic it does not care about never shows up as its lag. For a *lagging*
+subscription the value counts everything in the range it has not reached yet, matching its filter or not. Read it as
+staleness ("how far behind is this subscription"), not as the number of events its handler is about to run: that
+number is usually much smaller, because non-matching ranges are skipped without invoking the handler.
+
+`pg_eventstore_subscription_lag_seconds` (gauge)
+
+Age of the oldest event the subscription has not processed yet. `0` when fully caught up.
+
+The metric is absent for a subscription whose oldest unprocessed event no longer exists, which happens when that
+event or its stream was deleted. Reporting `0` there would read as "caught up", so nothing is reported instead and
+`lag_events` remains the source of truth for the backlog.
+
+`pg_eventstore_store_frontier_position` (gauge)
+
+Latest position assigned to an event, which is what subscription checkpoints are measured against. It only advances
+while at least one subscriptions process is running; when they are all down, lag stops growing - that situation shows
+up in the heartbeat metric below, not in the lag metrics.
+
+`pg_eventstore_store_head_global_position` (gauge)
+
+Global position of the newest event in the store. Contains gaps; do not compare subscription checkpoints against it.
+
+### Health
+
+`pg_eventstore_subscription_state` (gauge, extra `state` label, value is always 1)
+
+Last *recorded* state. A subscription killed without a graceful shutdown keeps `state="running"` and its lock
+forever - correlate with the heartbeat age below.
+
+`pg_eventstore_subscription_locked` (gauge, 0/1)
+
+Whether the subscription is currently locked by a subscriptions set.
+
+`pg_eventstore_subscription_heartbeat_age_seconds` (gauge)
+
+Seconds since the subscription row was last touched by its runner. Alive subscriptions update it about every 10
+seconds. **A locked subscription with a stale heartbeat is a dead process** - this is the signal to alert on for
+process death, since neither `state` nor the lock can be trusted for that.
+
+`pg_eventstore_subscription_restarts_total` (counter)
+
+Times the subscription was restarted after a failure.
+
+`pg_eventstore_subscription_last_error_age_seconds` (gauge)
+
+Seconds since the last error. Absent when the subscription never failed.
+
+### Throughput
+
+`pg_eventstore_subscription_processed_events_total` (counter)
+
+Total number of events processed. Use `rate()` over it for the actual current throughput - it correctly drops to 0
+when no matching events arrive.
+
+`pg_eventstore_subscription_capacity_events_per_second` (gauge)
+
+Derived from the average handler execution time of the last few processed events, *whenever they happened*. It
+answers "how fast can this handler go when it is fed" and is sticky while the subscription is idle - do not read it
+as current throughput. The ratio `rate(processed_events_total) / capacity` is a useful saturation signal: a
+subscription running close to its capacity has no headroom left, and a traffic spike will turn directly into lag.
+
+## Alerting suggestions
+
+- Lagging read models: `pg_eventstore_subscription_lag_seconds > 300 for 10m`
+- Dead subscription process: `pg_eventstore_subscription_locked == 1 and pg_eventstore_subscription_heartbeat_age_seconds > 60 for 10m`
+- No headroom: `rate(pg_eventstore_subscription_processed_events_total[5m]) / pg_eventstore_subscription_capacity_events_per_second > 0.8 for 15m`

data/lib/pg_eventstore/chunks/subscription_events_index_chunk.rb

@@ -64,9 +64,8 @@ module PgEventstore
 
       # @return [void]
       def resolve_indexes
-        indexes_to_resolve = @indexes.slice!(
-          Utils.range_to_slice(@indexes.map(&:event_type_partition_id), MAX_PARTITIONS_TO_RESOLVE_PER_CALL)
-        )
+        range = Utils.range_to_slice(@indexes.map(&:event_type_partition_id), MAX_PARTITIONS_TO_RESOLVE_PER_CALL)
+        indexes_to_resolve = @indexes[range]
         global_to_sub_position_map = indexes_to_resolve.to_h { [_1.global_position, _1.subscription_position] }
         raw_events = events_global_index_queries.resolve_indexes(
           indexes_to_resolve,
@@ -80,14 +79,10 @@ module PgEventstore
           )
         end
         raw_events = raw_events.sort_by(&:subscription_position)
+        # Remove indexes only after their events are resolved, so #size keeps counting them while the query runs
+        @indexes.slice!(range)
         @raw_events.push(*raw_events)
         @resolved = @indexes.empty?
-      rescue => exception
-        @indexes.unshift(*indexes_to_resolve)
-        @resolved = false
-        raise Utils.wrap_exception(
-          exception, global_positions: indexes_to_resolve.map(&:global_position)
-        )
       end
 
       # @return [PgEventstore::EventsGlobalIndexQueries]

data/lib/pg_eventstore/queries/replica_queries.rb

@@ -27,7 +27,10 @@ module PgEventstore
       return [] unless exists
 
       @query_strategy.exec_params(<<~SQL, [indexes.map(&:subscription_position)]).map { _1['subscription_position'] }
-        select subscription_position from event_subscription_positions where subscription_position = any($1::bigint[])
+        select subscription_position
+        from event_subscription_positions
+        where subscription_position = any($1::bigint[])
+        order by subscription_position
       SQL
     end
 
@@ -50,7 +53,7 @@ module PgEventstore
     # @return [Array<PgEventstore::EventGlobalIndex>]
     def load_events_global_index(indexes)
       result = @query_strategy.exec_params(<<~SQL, [indexes.map(&:global_position)])
-        select * from events_global_index where global_position = any($1::bigint[])
+        select * from events_global_index where global_position = any($1::bigint[]) order by global_position
       SQL
       result.map { EventGlobalIndex.new(**_1.transform_keys(&:to_sym)) }
     end
@@ -59,7 +62,7 @@ module PgEventstore
     # @return [Array<PgEventstore::StreamGlobalIndex>]
     def load_streams_global_index(indexes)
       result = @query_strategy.exec_params(<<~SQL, [indexes.map(&:streams_global_index_id)])
-        select * from streams_global_index where id = any($1::bigint[])
+        select * from streams_global_index where id = any($1::bigint[]) order by id
       SQL
       stream_revisions_map = indexes.group_by(&:streams_global_index_id).to_h do |stream_id, events_idx|
         [stream_id, events_idx.max_by(&:stream_revision).stream_revision]
@@ -75,7 +78,7 @@ module PgEventstore
     # @return [Array<PgEventstore::EventMarkerIndex>]
     def load_event_markers_index(indexes)
       result = @query_strategy.exec_params(<<~SQL, [indexes.map(&:global_position)])
-        select * from event_markers_index where global_position = any($1::bigint[])
+        select * from event_markers_index where global_position = any($1::bigint[]) order by global_position
       SQL
       result.map { EventMarkerIndex.new(**_1.transform_keys(&:to_sym)) }
     end
@@ -84,7 +87,7 @@ module PgEventstore
     # @return [Array<PgEventstore::EventMarker>]
     def load_markers(indexes)
       result = @query_strategy.exec_params(<<~SQL, [indexes.map(&:marker_id).uniq])
-        select * from event_markers where id = any($1::bigint[])
+        select * from event_markers where id = any($1::bigint[]) order by id
       SQL
       result.map { EventMarker.new(**_1.transform_keys(&:to_sym)) }
     end

data/lib/pg_eventstore/sql_builder.rb

@@ -215,7 +215,7 @@ module PgEventstore
 
     # @return [String]
     def from_sql
-      return @from_value if @from_value.is_a?(String)
+      return "#{@from_value} #{@table_alias if @table_alias != @from_value}" if @from_value.is_a?(String)
 
       sql = merge(@from_value)
       "(#{sql}) #{@table_alias}"

data/lib/pg_eventstore/subscriptions/callback_handlers/subscription_runner_handlers.rb

@@ -29,7 +29,7 @@ module PgEventstore
       end
 
       # @param subscription [PgEventstore::Subscription]
-      # @param error [PgEventstore::WrappedException]
+      # @param error [StandardError]
       # @return [void]
       def update_subscription_error(subscription, error)
         subscription.update(last_error: Utils.error_info(error), last_error_occurred_at: Time.now.utc)

data/lib/pg_eventstore/subscriptions/runner_recovery_strategies/report_subscription_unrecoverable_error.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module PgEventstore
+  module RunnerRecoveryStrategies
+    # Reports a subscription death that no other strategy handles, then leaves it dead.
+    # @!visibility private
+    class ReportSubscriptionUnrecoverableError
+      include RunnerRecoveryStrategy
+
+      # @param subscription [PgEventstore::Subscription]
+      # @param failed_subscription_notifier [#call, nil]
+      def initialize(subscription:, failed_subscription_notifier:)
+        @subscription = subscription
+        @failed_subscription_notifier = failed_subscription_notifier
+      end
+
+      # @param _error [StandardError]
+      # @return [true]
+      def recovers?(_error)
+        true
+      end
+
+      # @param error [StandardError]
+      # @return [false] the subscription stays dead - this strategy reports, it does not recover
+      def recover(error)
+        @failed_subscription_notifier&.call(@subscription.dup, error)
+        false
+      end
+    end
+  end
+end

data/lib/pg_eventstore/subscriptions/runner_recovery_strategies.rb

@@ -3,3 +3,4 @@
 require_relative 'runner_recovery_strategies/restore_connection'
 require_relative 'runner_recovery_strategies/restore_subscription_runner'
 require_relative 'runner_recovery_strategies/restore_subscription_feeder'
+require_relative 'runner_recovery_strategies/report_subscription_unrecoverable_error'

data/lib/pg_eventstore/subscriptions/subscription_feed_strategies/collection.rb

@@ -9,23 +9,25 @@ module PgEventstore
       DEFAULT_SUBSCRIPTIONS_NUM_PER_QUERY = 10
 
       class << self
-        # @param runners [Array<PgEventstore::SubscriptionRunner>]
+        # @param runners_query_options [Hash{PgEventstore::SubscriptionRunner => Hash}]
         # @param connection [PgEventstore::Connection]
         # @param query_strategy [PgEventstore::QueryStrategy]
         # @param subscriptions_per_query [Integer]
         # @return [PgEventstore::SubscriptionFeedStrategy::Collection]
-        def create(runners, connection, query_strategy, subscriptions_per_query: DEFAULT_SUBSCRIPTIONS_NUM_PER_QUERY)
+        def create(runners_query_options, connection, query_strategy,
+                   subscriptions_per_query: DEFAULT_SUBSCRIPTIONS_NUM_PER_QUERY)
           instance = new
-          replica_runners = runners.grep(ReplicaSubscriptionRunner)
-          common_runners = runners - replica_runners
-          common_runners.each_slice(subscriptions_per_query).each do |runners_slice|
+          replica_runners, common_runners = runners_query_options.partition do |runner, _|
+            runner.is_a?(ReplicaSubscriptionRunner)
+          end
+          common_runners.each_slice(subscriptions_per_query) do |runners_slice|
             strategy = IndexReadStrategy.new(connection, query_strategy)
-            strategy.add(*runners_slice)
+            runners_slice.each { |runner, query_options| strategy.add(runner, query_options) }
             instance.push(strategy)
           end
           replica_runners.each_slice(subscriptions_per_query) do |runners_slice|
             strategy = ReplicationStrategy.new(connection, query_strategy)
-            strategy.add(*runners_slice)
+            runners_slice.each { |runner, query_options| strategy.add(runner, query_options) }
             instance.push(strategy)
           end
           instance

data/lib/pg_eventstore/subscriptions/subscription_feed_strategies/index_read_strategy.rb

@@ -18,45 +18,45 @@ module PgEventstore
       def initialize(connection, query_strategy)
         @connection = connection
         @query_strategy = query_strategy
-        @runners = []
+        @runners_query_options = {}
       end
 
-      # @param runners [Array<PgEventstore::SubscriptionRunner>]
-      # @return [Array<PgEventstore::SubscriptionRunner>]
-      def add(*runners)
-        @runners.push(*runners)
+      # @param runner [PgEventstore::SubscriptionRunner]
+      # @param query_options [Hash]
+      # @return [void]
+      def add(runner, query_options)
+        @runners_query_options[runner] = query_options
       end
 
       # @return [Integer]
       def size
-        @runners.size
+        @runners_query_options.size
       end
 
       # @return [Boolean]
       def any?
-        @runners.any?
+        @runners_query_options.any?
       end
 
       # @return [void]
       def feed
-        runners_query_options = @runners.to_h do |runner|
-          next_chunk_query_opts = runner.next_chunk_query_opts
-          next_chunk_query_opts[:to_position] =
-            [next_chunk_query_opts[:from_position] + INDEX_LOOK_UP_DISTANCE, safe_position].min
-          [runner.id, next_chunk_query_opts]
+        @runners_query_options.each_value do |query_options|
+          query_options[:to_position] = [query_options[:from_position] + INDEX_LOOK_UP_DISTANCE, safe_position].min
         end
-        grouped_indexes = events_global_index_queries.fetch_indexes_for_subscriptions(runners_query_options)
-        @runners.each do |runner|
+        grouped_indexes = events_global_index_queries.fetch_indexes_for_subscriptions(
+          @runners_query_options.transform_keys(&:id)
+        )
+        @runners_query_options.each do |runner, query_options|
           if grouped_indexes[runner.id]
             chunk = Chunks::SubscriptionEventsIndexChunk.new(
               grouped_indexes[runner.id],
               @connection,
               QueryStrategy::Foreground.new(@connection),
-              runners_query_options[runner.id][:resolve_link_tos]
+              query_options[:resolve_link_tos]
             )
             runner.feed(chunk)
           else
-            runner.feed(Chunks::SubscriptionCheckpointChunk.new(runners_query_options[runner.id][:to_position]))
+            runner.feed(Chunks::SubscriptionCheckpointChunk.new(query_options[:to_position]))
           end
         end
       end

data/lib/pg_eventstore/subscriptions/subscription_feed_strategies/replication_strategy.rb

@@ -16,40 +16,40 @@ module PgEventstore
       def initialize(connection, query_strategy)
         @connection = connection
         @query_strategy = query_strategy
-        @runners = []
+        @runners_query_options = {}
       end
 
-      # @param runners [Array<PgEventstore::ReplicaSubscriptionRunner>]
-      # @return [Array<PgEventstore::ReplicaSubscriptionRunner>]
-      def add(*runners)
-        @runners.push(*runners)
+      # @param runner [PgEventstore::ReplicaSubscriptionRunner]
+      # @param query_options [Hash]
+      # @return [void]
+      def add(runner, query_options)
+        @runners_query_options[runner] = query_options
       end
 
       # @return [Integer]
       def size
-        @runners.size
+        @runners_query_options.size
       end
 
       # @return [Boolean]
       def any?
-        @runners.any?
+        @runners_query_options.any?
       end
 
       # @return [void]
       def feed
-        runners_query_options = @runners.to_h do |runner|
-          next_chunk_query_opts = runner.next_chunk_query_opts
-          next_chunk_query_opts[:to_position] =
-            [next_chunk_query_opts[:from_position] + INDEX_LOOK_UP_DISTANCE, safe_position].min
-          [runner.id, next_chunk_query_opts]
+        @runners_query_options.each_value do |query_options|
+          query_options[:to_position] = [query_options[:from_position] + INDEX_LOOK_UP_DISTANCE, safe_position].min
         end
-        grouped_indexes = events_global_index_queries.fetch_indexes_for_subscriptions(runners_query_options)
-        @runners.each do |runner|
+        grouped_indexes = events_global_index_queries.fetch_indexes_for_subscriptions(
+          @runners_query_options.transform_keys(&:id)
+        )
+        @runners_query_options.each do |runner, query_options|
           if grouped_indexes[runner.id]
             chunk = Chunks::ReplicaEventsIndexChunk.new(grouped_indexes[runner.id])
             runner.feed(chunk)
           else
-            runner.feed(Chunks::SubscriptionCheckpointChunk.new(runners_query_options[runner.id][:to_position]))
+            runner.feed(Chunks::SubscriptionCheckpointChunk.new(query_options[:to_position]))
           end
         end
       end

data/lib/pg_eventstore/subscriptions/subscription_feed_strategy.rb

@@ -3,9 +3,10 @@
 module PgEventstore
   # @!visibility private
   module SubscriptionFeedStrategy
-    # @param runners [PgEventstore::SubscriptionRunner, Array<PgEventstore::SubscriptionRunner>]
-    # @return [Array<PgEventstore::SubscriptionRunner>]
-    def add(*runners)
+    # @param runner [PgEventstore::SubscriptionRunner]
+    # @param query_options [Hash]
+    # @return [void]
+    def add(runner, query_options)
       raise NotImplementedError
     end
 

data/lib/pg_eventstore/subscriptions/subscription_runner.rb

@@ -50,11 +50,6 @@ module PgEventstore
       }
     end
 
-    # @return [Boolean]
-    def time_to_feed?
-      estimate_events_number > 0 && @subscription.last_chunk_fed_at + @subscription.chunk_query_interval <= Time.now.utc
-    end
-
     private
 
     # @return [Integer]