rails_event_viewer 0.1.2 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ef30fa1ce21c1dc5c6eaa4634707801f2d32b67ab561231865cc67c0eb2c557f
-  data.tar.gz: 9a423c1745dfb3a088137fbe4c912c069a1e65ca68afc0e4d1dac2961819baac
+  metadata.gz: 26f6b1798d95bf71759a16da1fd092f8d8296a4118378a8c5ce84e12848efe28
+  data.tar.gz: b290b0d5803b71aa2f8fda06ee160870672a4073c175c2bb6a0e0856f19b2dbc
 SHA512:
-  metadata.gz: 3b25d996bb14c0bada11737d07a98f93fbd10b8f4781ea3163b79185e31a2b6fe30a99b74e69f63e1d6e768a1fa137124bab9f3a47da07f5940466de1933e2ff
-  data.tar.gz: 828ddff25e0930d10bc34c75ee58e789f361f6dc14c3bf24c56b463e8b1327b454cf15ce7536f7d32969716f4c6db82bd9640c3bfcada4a8c9aed072132bc158
+  metadata.gz: ef7fc01c18016ff1f9d48de98fcc1f42b248ac1fea588806c6dc3a409b12f75291e4165ee59ff45220b93a024f6b816d7cb2dba4395c2b5c441dd6f323a8a17a
+  data.tar.gz: 896dd1e41a71c0c0bd25dbdd0a376ffd7c5c0c68b8d7d6cf445521127560d95ba0d351e1d9f9f4c6c5d73ed00ae6472dbd9096ceddf2d79cd8b1dbf4e3edb37f

data/CHANGELOG.md

@@ -1,5 +1,23 @@
 # Changelog
 
+## 0.2.0 (2026-10-01)
+
+- Pluggable buffers: `config.buffer` accepts any object implementing `RailsEventViewer::Buffer` (`push`, `drain`, `commit`, `revert`, `dead`, `size`, `after_fork`). `push` returns the new size. The in-memory buffer remains the default.
+- A failed batch is reverted and retried with exponential backoff. After three failures it is written one event at a time.
+- Events the adapter keeps rejecting, and events dropped by the buffer cap, are passed to `dead` instead of being lost silently.
+- A failed sync write, or a failed `push` to the buffer in async mode, is now logged and re-raised to `Rails.event`, which reports it to `Rails.error` (or raises, with `raise_on_error`), instead of being swallowed.
+- A banner warns when `sample_rate` is below 1.0, since timelines can have gaps.
+- README now marks the gem as experimental and not yet recommended for production.
+
+## 0.1.3 (2026-10-01)
+
+- The flusher thread starts on the first buffered event, not at boot, so consoles and rake tasks no longer run an idle thread.
+- Flusher start/stop messages log at debug level.
+- Events emitted after shutdown are written immediately instead of lost.
+- Web requests never write to storage. A full buffer wakes the flusher thread instead.
+- The buffer is capped at 10 times `buffer_size`. Overflow drops the oldest events and logs a warning.
+- Removed the unused ActionCable broadcast.
+
 ## 0.1.2 (2026-09-30)
 
 - Removed the `pagy` dependency. Pagination is handled by a small built-in class, so host apps on any Pagy version, or none, can install the gem.

data/README.md

@@ -2,6 +2,8 @@
 
 A Rails engine that captures events emitted with `Rails.event` and gives you a dashboard to browse, search, and chart them.
 
+> **Experimental.** This gem is in early development and not recommended for production yet. APIs, the database schema, and behavior may change between minor versions.
+
 ## Requirements
 
 - Rails 8.1+
@@ -61,8 +63,15 @@ RailsEventViewer.configure do |config|
 end
 ```
 
+With `async`, events are written by a background thread, never by your web requests. The buffer holds at most 10 times `buffer_size` events. If storage can't keep up, the oldest are dropped and a warning is logged. Events still in the buffer are written on a normal shutdown but lost if the process is killed.
+
 Internal Rails events such as `active_record.*` and `action_controller.*` are ignored by default. Remove a pattern from `ignored_events` to capture it.
 
+## Sync vs async
+
+- With `async = false`, each event is written inside your transaction, and a failed write is reported to `Rails.error`. In development and test, where Rails sets `raise_on_error`, it raises instead.
+- With `async = true`, events wait in an in-memory buffer and are lost if the process is killed. Set `config.buffer` to your own `RailsEventViewer::Buffer`, for example backed by Redis, to keep them.
+
 ## Authentication
 
 Without authentication, the dashboard is open in development and test, and returns `403` everywhere else.
@@ -83,27 +92,9 @@ Set `RAILS_ENV` on every server. Without it, Rails falls back to development and
 
 ## Storage adapters
 
-| Adapter | Use it for |
-|---|---|
-| `:active_record` | The default. Stores events in your database, with full analytics. |
-| `:redis` | High volume, short-lived storage. Keeps the newest `max_events`. |
-| `:memory` | Development and tests. Lost on restart. |
-| `:null` | Disables storage. |
-
-Redis needs the `redis` gem:
-
-```ruby
-config.storage_adapter = :redis
-config.adapter_options = {
-  redis_options: { url: ENV["REDIS_URL"] },
-  pool_size: 5,
-  max_events: 10_000
-}
-```
-
-Filtered queries on Redis scan every stored event, so keep `max_events` modest.
+Set `config.storage_adapter` to `:active_record` (default), `:redis`, `:memory`, or `:null`. Redis needs the `redis` gem and takes options through `config.adapter_options`.
 
-For a custom adapter, include `RailsEventViewer::Adapter` and implement the methods it lists.
+Filtered queries on Redis scan every stored event, so keep `max_events` (how many events Redis keeps, set in `adapter_options`) modest. For your own storage, include `RailsEventViewer::Adapter`.
 
 ## Rake tasks
 

data/app/views/layouts/rails_event_viewer/application.html.erb

@@ -33,6 +33,14 @@
           </div>
         <% end %>
 
+        <% if RailsEventViewer.sample_rate.to_f < 1.0 %>
+          <div class="mb-4 rounded-md bg-yellow-50 border border-yellow-200 p-4">
+            <p class="text-sm font-medium text-yellow-800">
+              Sampling is on: only <%= number_to_percentage(RailsEventViewer.sample_rate.to_f * 100, precision: 3, significant: true, strip_insignificant_zeros: true) %> of events are recorded, so timelines can have gaps.
+            </p>
+          </div>
+        <% end %>
+
         <%= yield %>
       </div>
     </main>

data/lib/generators/rails_event_viewer/install/templates/initializer.rb.erb

@@ -40,6 +40,11 @@ RailsEventViewer.configure do |config|
   # Flush interval in seconds (flushes even if buffer isn't full)
   # config.flush_interval = 2
 
+  # Buffer used when async is true. The default is in memory, so buffered events
+  # are lost if the process is killed. You can write your own buffers (see RailsEventViewer::Buffer)
+  # to persist them, for example in Redis.
+  # config.buffer = MyRedisBuffer.new
+
   # Sample rate: 1.0 = capture all events, 0.1 = capture 10% of events
   # Useful for high-volume production environments
   # config.sample_rate = 1.0

data/lib/rails_event_viewer/batch_writer.rb

@@ -0,0 +1,79 @@
+module RailsEventViewer
+  class BatchWriter
+    include Logging
+
+    MAX_ATTEMPTS = 3
+    LOGGED_EVENTS = 10
+
+    attr_reader :failed_attempts
+
+    def initialize(buffer)
+      @buffer = buffer
+      reset
+    end
+
+    def reset
+      @failed_attempts = 0
+      @strikes = Hash.new(0)
+    end
+
+    def write(events)
+      RailsEventViewer.adapter.write_events(events)
+      @buffer.commit(events)
+      forget(events)
+      @failed_attempts = 0
+      true
+    rescue => e
+      log_error("[RailsEventViewer] Failed to write #{events.size} events: #{e.message}")
+      @failed_attempts += 1
+      return write_individually(events) if @failed_attempts >= MAX_ATTEMPTS
+
+      @buffer.revert(events)
+      false
+    end
+
+    def forget(events)
+      events.each { |event| @strikes.delete(event) } unless @strikes.empty?
+    end
+
+    private
+
+    def write_individually(events)
+      written = []
+      rejected = []
+      events.each do |event|
+        (write_one(event) ? written : rejected) << event
+        break if written.empty? && rejected.size >= MAX_ATTEMPTS
+      end
+
+      if written.empty?
+        @buffer.revert(events)
+        return false
+      end
+
+      rejected.each { |event| @strikes[event] += 1 }
+      dead, retry_later = rejected.partition { |event| @strikes[event] >= MAX_ATTEMPTS }
+      forget(written + dead)
+
+      @buffer.commit(written)
+      @buffer.revert(retry_later) if retry_later.any?
+      give_up(dead) if dead.any?
+      @failed_attempts = retry_later.any? ? MAX_ATTEMPTS - 1 : 0
+      true
+    end
+
+    def give_up(events)
+      names = events.first(LOGGED_EVENTS).map { |event| event[:name] }.join(", ")
+      log_error("[RailsEventViewer] Gave up on #{events.size} events the adapter rejected #{MAX_ATTEMPTS} times (#{@last_rejection}): #{names}")
+      @buffer.dead(events)
+    end
+
+    def write_one(event)
+      RailsEventViewer.adapter.write_events([event])
+      true
+    rescue => e
+      @last_rejection = e.message
+      false
+    end
+  end
+end

data/lib/rails_event_viewer/buffer.rb

@@ -0,0 +1,28 @@
+module RailsEventViewer
+  module Buffer
+    def push(entry)
+      raise NotImplementedError, "#{self.class} must implement #push"
+    end
+
+    def drain(limit)
+      raise NotImplementedError, "#{self.class} must implement #drain"
+    end
+
+    def commit(entries)
+    end
+
+    def revert(entries)
+    end
+
+    def dead(entries)
+      commit(entries)
+    end
+
+    def size
+      raise NotImplementedError, "#{self.class} must implement #size"
+    end
+
+    def after_fork
+    end
+  end
+end

data/lib/rails_event_viewer/buffers/memory.rb

@@ -0,0 +1,36 @@
+module RailsEventViewer
+  module Buffers
+    class Memory
+      include Buffer
+
+      def initialize
+        @queue = Queue.new
+      end
+
+      def push(entry)
+        @queue << entry
+        @queue.size
+      end
+
+      def drain(limit)
+        entries = []
+        limit.times { entries << @queue.pop(true) }
+        entries
+      rescue ThreadError
+        entries
+      end
+
+      def revert(entries)
+        entries.each { |entry| @queue << entry }
+      end
+
+      def size
+        @queue.size
+      end
+
+      def after_fork
+        @queue = Queue.new
+      end
+    end
+  end
+end

data/lib/rails_event_viewer/flusher.rb

@@ -0,0 +1,150 @@
+module RailsEventViewer
+  class Flusher
+    include Logging
+
+    SHUTDOWN_TIMEOUT_SECONDS = 5
+    MAX_BUFFER_MULTIPLIER = 10
+
+    def initialize(buffer)
+      @buffer = buffer
+      @writer = BatchWriter.new(buffer)
+      @flush_requests = Queue.new
+      @mutex = Mutex.new
+      @thread = nil
+      @shutdown_requested = false
+      @dropped_count = 0
+      @pid = Process.pid
+    end
+
+    def push(entry)
+      handle_fork_if_needed
+
+      size = @buffer.push(entry)
+      return flush! if shutdown_requested?
+
+      ensure_thread!
+      drop_oldest_events if size > max_buffer_size
+      request_flush if size >= RailsEventViewer.buffer_size
+    end
+
+    def flush!
+      events = @buffer.drain([@buffer.size, max_buffer_size].min)
+      return true if events.empty?
+
+      written = @writer.write(events)
+      report_dropped_events if written
+      written
+    rescue => e
+      log_error("[RailsEventViewer] Failed to flush: #{e.message}")
+      false
+    end
+
+    def stop!
+      @mutex.synchronize { @shutdown_requested = true }
+
+      if @thread&.alive?
+        @flush_requests << true
+        @thread.join(SHUTDOWN_TIMEOUT_SECONDS)
+      end
+
+      @mutex.synchronize { @thread = nil }
+
+      loop do
+        remaining = @buffer.size
+        break if remaining.zero?
+
+        flush!
+        break if @buffer.size >= remaining
+      end
+    end
+
+    def running?
+      @thread&.alive? || false
+    end
+
+    def size
+      @buffer.size
+    end
+
+    private
+
+    def handle_fork_if_needed
+      @mutex.synchronize do
+        if @pid != Process.pid
+          @pid = Process.pid
+          @buffer.after_fork
+          @writer.reset
+          @flush_requests = Queue.new
+          @thread = nil
+          @shutdown_requested = false
+          @dropped_count = 0
+        end
+      end
+    end
+
+    def max_buffer_size
+      RailsEventViewer.buffer_size * MAX_BUFFER_MULTIPLIER
+    end
+
+    def drop_oldest_events
+      while @buffer.size > max_buffer_size
+        dropped = @buffer.drain(1)
+        break if dropped.empty?
+
+        @buffer.dead(dropped)
+        @writer.forget(dropped)
+        @mutex.synchronize { @dropped_count += 1 }
+      end
+    end
+
+    def report_dropped_events
+      dropped = @mutex.synchronize { @dropped_count.tap { @dropped_count = 0 } }
+      return if dropped.zero?
+
+      effective_logger&.warn("[RailsEventViewer] Dropped #{dropped} events because the buffer was full")
+    end
+
+    def request_flush
+      @flush_requests << true if @flush_requests.empty? && @writer.failed_attempts.zero?
+    end
+
+    def ensure_thread!
+      @mutex.synchronize do
+        return if @thread&.alive?
+        return if @shutdown_requested
+
+        @thread = Thread.new { run_loop }
+      end
+    end
+
+    def run_loop
+      effective_logger&.debug("[RailsEventViewer] Flusher thread started (PID: #{Process.pid})")
+
+      until shutdown_requested?
+        begin
+          wait_for_flush
+          loop do
+            break unless flush!
+            break if @buffer.size < RailsEventViewer.buffer_size || shutdown_requested?
+          end
+        rescue => e
+          log_error("[RailsEventViewer] Flusher error: #{e.message}")
+        end
+      end
+
+      effective_logger&.debug("[RailsEventViewer] Flusher thread stopped (PID: #{Process.pid})")
+    end
+
+    def wait_for_flush
+      backoff = 2**[@writer.failed_attempts, BatchWriter::MAX_ATTEMPTS].min
+      @flush_requests.clear unless @writer.failed_attempts.zero?
+      return if shutdown_requested?
+
+      @flush_requests.pop(timeout: RailsEventViewer.flush_interval * backoff)
+    end
+
+    def shutdown_requested?
+      @mutex.synchronize { @shutdown_requested }
+    end
+  end
+end

data/lib/rails_event_viewer/logging.rb

@@ -0,0 +1,19 @@
+module RailsEventViewer
+  module Logging
+    private
+
+    def log_error(message)
+      logger = effective_logger
+
+      if logger
+        logger.error(message)
+      else
+        warn message
+      end
+    end
+
+    def effective_logger
+      RailsEventViewer.logger || (defined?(Rails) && Rails.logger)
+    end
+  end
+end

data/lib/rails_event_viewer/subscriber.rb

@@ -2,156 +2,35 @@
 
 module RailsEventViewer
   class Subscriber
+    include Logging
+
     NANOSECONDS_PER_SECOND = 1_000_000_000.0
-    SHUTDOWN_TIMEOUT_SECONDS = 5
-    ACTION_CABLE_CHANNEL = "rails_event_viewer:events"
 
-    def initialize
-      @buffer = Queue.new
-      @mutex = Mutex.new
-      @flusher_thread = nil
-      @shutdown_requested = false
-      @pid = Process.pid
+    delegate :flush!, :stop!, :running?, to: :@flusher
+    delegate :size, to: :@flusher, prefix: :buffer
 
-      ensure_flusher_thread!
+    def initialize
+      @flusher = Flusher.new(RailsEventViewer.buffer)
     end
 
     def emit(event)
-      handle_fork_if_needed
-
       return unless should_capture?(event)
       return if sampled_out?
 
       entry = build_entry(event)
 
       if RailsEventViewer.async
-        buffer_event(entry)
+        @flusher.push(entry)
       else
-        write_immediately([entry])
+        RailsEventViewer.adapter.write_events([entry])
       end
     rescue => e
       handle_error(e, event)
-    end
-
-    def flush!
-      return if @buffer.empty?
-
-      events_to_write = drain_buffer
-
-      return if events_to_write.empty?
-
-      write_immediately(events_to_write)
-      broadcast_events(events_to_write)
-    end
-
-    def stop!
-      @mutex.synchronize do
-        @shutdown_requested = true
-      end
-
-      if @flusher_thread&.alive?
-        @flusher_thread.wakeup rescue nil
-        @flusher_thread.join(SHUTDOWN_TIMEOUT_SECONDS)
-      end
-
-      @mutex.synchronize do
-        @flusher_thread = nil
-      end
-
-      flush!
-    end
-
-    def running?
-      @flusher_thread&.alive? || false
-    end
-
-    def buffer_size
-      @buffer.size
+      raise
     end
 
     private
 
-    def handle_fork_if_needed
-      # Threads don't survive forks, so we need to restart the flusher
-      @mutex.synchronize do
-        if @pid != Process.pid
-          @pid = Process.pid
-          @buffer = Queue.new
-          @flusher_thread = nil
-          @shutdown_requested = false
-        end
-      end
-      ensure_flusher_thread!
-    end
-
-    def drain_buffer
-      events = []
-      loop { events << @buffer.pop(true) }
-    rescue ThreadError
-      events
-    end
-
-    def buffer_event(entry)
-      @buffer << entry
-      ensure_flusher_thread!
-
-      flush! if @buffer.size >= RailsEventViewer.buffer_size
-    end
-
-    def ensure_flusher_thread!
-      return unless RailsEventViewer.async
-
-      @mutex.synchronize do
-        return if @flusher_thread&.alive?
-        return if @shutdown_requested
-
-        @flusher_thread = Thread.new { run_flusher_loop }
-      end
-    end
-
-    def run_flusher_loop
-      log_info("[RailsEventViewer] Flusher thread started (PID: #{Process.pid})")
-
-      until shutdown_requested?
-        begin
-          sleep(RailsEventViewer.flush_interval)
-          flush! unless @buffer.empty?
-        rescue => e
-          log_error("[RailsEventViewer] Flusher error: #{e.message}")
-        end
-      end
-
-      log_info("[RailsEventViewer] Flusher thread stopped (PID: #{Process.pid})")
-    end
-
-    def shutdown_requested?
-      @mutex.synchronize { @shutdown_requested }
-    end
-
-    def write_immediately(events)
-      return if events.blank?
-
-      RailsEventViewer.adapter.write_events(events)
-    rescue => e
-      log_error("[RailsEventViewer] Failed to write #{events.size} events: #{e.message}")
-    end
-
-    def broadcast_events(events)
-      return unless defined?(ActionCable)
-      return unless events.any?
-
-      ActionCable.server.broadcast(
-        ACTION_CABLE_CHANNEL,
-        {
-          action: "new_events",
-          count: events.size,
-          events: events.map { |e| { name: e[:name], occurred_at: e[:occurred_at]&.iso8601 } }
-        }
-      )
-    rescue => e
-      log_error("[RailsEventViewer] Broadcast error: #{e.message}")
-    end
-
     def should_capture?(event)
       return false unless event.is_a?(Hash)
       return false if event[:name].blank?
@@ -233,23 +112,5 @@ module RailsEventViewer
     def handle_error(error, event)
       log_error("[RailsEventViewer] Failed to capture event '#{event[:name]}': #{error.message}")
     end
-
-    def log_info(message)
-      effective_logger&.info(message)
-    end
-
-    def log_error(message)
-      logger = effective_logger
-
-      if logger
-        logger.error(message)
-      else
-        warn message
-      end
-    end
-
-    def effective_logger
-      RailsEventViewer.logger || (defined?(Rails) && Rails.logger)
-    end
   end
 end

data/lib/rails_event_viewer/version.rb

@@ -1,3 +1,3 @@
 module RailsEventViewer
-  VERSION = "0.1.2"
+  VERSION = "0.2.0"
 end

data/lib/rails_event_viewer.rb

@@ -3,6 +3,8 @@ require "groupdate"
 
 require "rails_event_viewer/version"
 require "rails_event_viewer/engine"
+require "rails_event_viewer/buffer"
+require "rails_event_viewer/buffers/memory"
 
 module RailsEventViewer
   mattr_accessor :storage_adapter, default: :active_record
@@ -12,6 +14,7 @@ module RailsEventViewer
   mattr_accessor :buffer_size, default: 100 # Flush after N events
   mattr_accessor :flush_interval, default: 2 # Flush every N seconds
   mattr_accessor :sample_rate, default: 1.0 # 1.0 = 100%, 0.1 = 10%
+  mattr_accessor :buffer, default: Buffers::Memory.new
 
   mattr_accessor :retention_period, default: 7.days
   mattr_accessor :captured_events, default: []      # Empty = capture all
@@ -115,6 +118,9 @@ end
 
 require "rails_event_viewer/adapter"
 require "rails_event_viewer/events_relation"
+require "rails_event_viewer/logging"
+require "rails_event_viewer/batch_writer"
+require "rails_event_viewer/flusher"
 require "rails_event_viewer/subscriber"
 require "rails_event_viewer/json_query"
 require "rails_event_viewer/time_utils"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rails_event_viewer
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.2.0
 platform: ruby
 authors:
 - Keshav Biswa
@@ -107,9 +107,14 @@ files:
 - lib/rails_event_viewer/adapters/memory.rb
 - lib/rails_event_viewer/adapters/null.rb
 - lib/rails_event_viewer/adapters/redis.rb
+- lib/rails_event_viewer/batch_writer.rb
+- lib/rails_event_viewer/buffer.rb
+- lib/rails_event_viewer/buffers/memory.rb
 - lib/rails_event_viewer/engine.rb
 - lib/rails_event_viewer/events_relation.rb
+- lib/rails_event_viewer/flusher.rb
 - lib/rails_event_viewer/json_query.rb
+- lib/rails_event_viewer/logging.rb
 - lib/rails_event_viewer/subscriber.rb
 - lib/rails_event_viewer/time_utils.rb
 - lib/rails_event_viewer/version.rb