rails_error_dashboard 0.11.0 → 0.11.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: defed3f836bc0e812c76abfb81b95e5e3f229e5338bfa40cde4604643eb95582
-  data.tar.gz: 7657043dbf617694e5318a5b0e2829ecca78ade0c67412ede16fbecabc696c36
+  metadata.gz: c470ccb09f844bb6187fff2f3519d00370e3a0d188319cba49beed93c1bc139d
+  data.tar.gz: a1adc23940e22ffe6f35d00a161c7ec6494a07cc3576ca1a0b541cc28605e108
 SHA512:
-  metadata.gz: 79992d38956c257a116c1cfcb153429cbf3a082892bc71664c20ba57e801ba33325ae69f43bd0ef8404a3cd08a6d5df9727a989a0f8977826786a179ad70b448
-  data.tar.gz: 3bbcc477eff8db567415c727037a58d7e27bd8e01ad602674beb05cd6891f75bb06e665273698fed1b49c20806e4f07dbea586647583f6b7da3052a744d58fae
+  metadata.gz: dabde3ae967ff2d0615b0fa91477f3706490ef2fe8975fa26a364727905c646e393605a6d6c0bca2c1e614eb311b550d192a3f7d632bd19626407f586cd016ba
+  data.tar.gz: c0f5e711dac845b70506151dc56f2b4c079eb8dfe2dc85b491dfef4ae011d488d55c758fc0a0aee18ac64faf949253a1be3cb6c421e9315a62cbccef823d6f9d

data/README.md

@@ -7,27 +7,100 @@
 [![Sponsor](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-ea4aaa?logo=githubsponsors)](https://github.com/sponsors/AnjanJ)
 [![Buy Me A Coffee](https://img.shields.io/badge/Buy%20Me%20A%20Coffee-support-yellow?logo=buymeacoffee)](https://buymeacoffee.com/anjanj)
 
-**Self-hosted Rails error monitoring — free, forever.**
+**Rails-native error tracking for failure investigation — see the Ruby state and Rails runtime health behind every exception. Self-hosted, inside your app, in your own database. The gem is MIT and free forever.**
 
 ```ruby
-gem 'rails_error_dashboard'
+gem "rails_error_dashboard"
 ```
 
-**5-minute setup** · **Works out-of-the-box** · **PostgreSQL, MySQL/Trilogy, SQLite — shared or separate database** · **No vendor lock-in**
+```bash
+bundle install
+rails generate rails_error_dashboard:install
+rails db:migrate
+```
 
-[Full Documentation](https://anjanj.github.io/rails_error_dashboard/) · [Live Demo](https://rails-error-dashboard.anjan.dev) · [RubyGems](https://rubygems.org/gems/rails_error_dashboard)
+Open `/red` and raise a test exception. No monitoring account or ingestion service is required.
 
----
+[Try the live demo](https://rails-error-dashboard.anjan.dev) (`gandalf` / `youshallnotpass`) · [Read the documentation](https://anjanj.github.io/rails_error_dashboard/) · [View on RubyGems](https://rubygems.org/gems/rails_error_dashboard)
+
+> **Beta:** RED is functional and extensively tested, but configuration and APIs may change before 1.0. Supports Rails 7.0–8.1 and Ruby 3.2–4.0 (CI runs Ruby 3.2–3.4 against every supported Rails version; Ruby 4.0 is verified by the maintainer).
+
+## See the Ruby state and Rails runtime health behind every exception
+
+Rails Error Dashboard (RED) is an open-source, self-hosted Rails engine for investigating production failures. It helps you answer not only **what failed**, but **what was happening inside Ruby and Rails when it failed**.
+
+- Inspect local variables and the raising object's instance variables before the stack unwinds.
+- See error-time Active Record, Puma, job queue, GC, memory and process health.
+- Follow the SQL, cache, controller, job, mailer and other Rails events leading to the exception.
+- Stay safe during error floods with progressive, count-preserving storm protection.
+- Keep exception data on infrastructure you control.
+
+![Local and instance variables captured at the raise, scrubbed with filter_parameters](docs/images/local-variables.png)
+
+## The questions RED helps you answer
+
+A stack trace tells you where execution stopped. RED helps you investigate the state behind it:
+
+- What did `params`, local variables and objects such as `@order` contain?
+- Was the Active Record pool exhausted?
+- Was Puma out of thread capacity or building a backlog?
+- Were jobs failing or queues growing?
+- Was the process under GC, memory, descriptor or system pressure?
+- Which SQL queries, cache operations or Rails events preceded the failure?
+- Did a deploy introduce the error?
+- Can the failing request become a cURL reproduction or RSpec regression-test scaffold?
+
+## What makes RED different
+
+### Failure-time Ruby state
+
+Optionally capture local variables and — something no other error tracker does — the raising receiver's instance variables at `TracePoint(:raise)`, with bounded serialization and your Rails `filter_parameters` applied to sensitive values. Binding objects are never retained.
 
-### Try the Live Demo
+### Failure-time Rails health
 
-**[rails-error-dashboard.anjan.dev](https://rails-error-dashboard.anjan.dev)** — Username: `gandalf` · Password: `youshallnotpass`
+Attach connection-pool, Puma, background-job, GC, memory, file-descriptor, TCP, RubyVM and YJIT state to the error record, refreshed on every captured occurrence — not merely to a separate periodic metrics chart. Every APM has these as time-series; none attaches them to the error. Opt-in; the procfs-backed fields are Linux-only.
 
-> **Beta Software** — Functional and tested (4,200+ tests passing), but the API may change before v1.0. Supports Rails 7.0-8.1 and Ruby 3.2-4.0.
+### Monitoring that degrades safely
+
+During an error flood, RED progressively reduces captured context and database work, keeps a fresh exemplar every minute, records the storm in a Storm History ledger and reconciles exact in-process occurrence counts onto the error records. On by default.
+
+### Rails-specific investigation
+
+Connect exceptions with SQL, caching, Active Job, Action Cable, Active Storage, Rack::Attack, deprecations and other Rails subsystems from one dashboard.
+
+### Things no other tracker does
+
+Verified against Sentry, Honeybadger, AppSignal, Rollbar, Bugsnag, Airbrake, Raygun, New Relic, Datadog, Scout, Skylight and every self-hosted Rails tracker in August 2026 ([the ledger](.shipkit/research/red-unique-features-verified.md)):
+
+- **Copy as RSpec** — a runnable request spec generated from the captured request (Sentry offers curl only).
+- **Swallowed-exception aggregate** — raise-vs-rescue ratio per location, no APM span needed (Datadog's paid APM detects rescued exceptions but keeps no aggregate).
+- **Rack::Attack ledger** — throttle, blocklist and track events persisted with per-rule stats and an AI-crawler classifier; rack-attack ships no UI of its own.
+- **Codeberg issue tracking**, alongside GitHub, GitLab and Linear with two-way sync.
+- **The tracker instruments itself** — its capture pipeline exported as OpenTelemetry spans, so you can audit its overhead in your own APM.
+
+## How RED compares
+
+| Basic embedded tracker | General SaaS monitoring | RED |
+|---|---|---|
+| Stack trace and context | Cross-language telemetry and managed ingestion | Deep failure-time Ruby/Rails state inside the application boundary |
+| Lightweight and local | Strong distributed and frontend observability | Rails-specific operational investigation and storm-safe local capture |
+
+That makes RED a self-hosted Sentry alternative for teams that want Rails-specific depth and need error data to stay inside the application boundary — not a replacement for cross-language telemetry. RED has no mobile SDKs, no merge/split, no MCP server and no hosted operations.
+
+## Choose how you run it
+
+- Store data in the application's existing PostgreSQL, MySQL/Trilogy or SQLite database.
+- Isolate monitoring writes in a separate error database.
+- Use synchronous writes, or async logging through Sidekiq or Solid Queue (GoodJob is detected for job-health stats but is not an async adapter).
+- Track several Rails applications through a shared database.
+
+No RED licence or event-ingestion fee, and no plan limits — your database is the only cap, and storm protection deliberately sheds context during floods.
+
+---
 
 ### Screenshots
 
-**Dashboard Overview** — Real-time error stats, severity breakdown, and trend charts.
+**Dashboard Overview** — Live error stats, severity breakdown, and trend charts.
 
 ![Dashboard Overview](docs/images/dashboard-overview.png)
 
@@ -49,25 +122,12 @@ gem 'rails_error_dashboard'
 
 ---
 
-## Who This Is For
-
-- **Solo bootstrappers** who need professional error tracking without recurring costs
-- **Indie SaaS founders** building profitable apps on tight budgets
-- **Small dev teams** (2-5 people) who hate SaaS bloat
-- **Privacy-conscious apps** that need to keep error data on their own servers
-- **Side projects** that might become real businesses
-
-## What It Replaces
+## Safety, performance and compatibility
 
-| Before | After |
-|--------|-------|
-| $29-99/month for error monitoring | $0/month — runs on your existing Rails server |
-| Sensitive error data sent to third parties | All data stays on your infrastructure |
-| SaaS pricing tiers and usage limits | Unlimited errors, unlimited projects |
-| Vendor lock-in with proprietary APIs | 100% open source, fully portable |
-| Complex SDK setup and external services | 5-minute Rails Engine installation |
-| Pay extra for local variable capture (Sentry) | Local + instance variables included free |
-| No tool detects silently rescued exceptions | Swallowed exception detection built in |
+- **Host-app safety** — nothing in the capture path raises into your app; every subscriber and callback is rescue-wrapped, `Thread.current` is cleaned up in `ensure`, and the original exception is always re-raised. Variables, health and breadcrumbs are opt-in and off by default; storm protection is on by default and fails open.
+- **Performance** — the storm-protection hot path is a digest plus an atomic increment with no I/O; the figures quoted below are a maintainer's single-machine measurements and no benchmark script ships with the gem yet.
+- **Security** — HTTP Basic Auth or your own `authenticate_with` lambda (Devise, Warden, session); your Rails `filter_parameters` are applied to params, variables and breadcrumbs; prompts are never recorded by LLM observability. Vulnerability reports: [SECURITY.md](SECURITY.md).
+- **Compatibility** — Rails 7.0–8.1, Ruby 3.2–4.0, PostgreSQL, MySQL/Trilogy or SQLite; `turbo-rails` plus ActionCable are needed for live updates (no polling fallback); the gem's own CSS/JS is inline but Bootstrap JS, Chart.js, highlight.js and Google Fonts load from CDNs, so it is not air-gap clean.
 
 ---
 
@@ -75,14 +135,14 @@ gem 'rails_error_dashboard'
 
 ### Core (Always Enabled)
 
-Error capture from controllers, jobs, and middleware. Custom-designed dashboard with dark/light mode, search, filtering, and real-time updates. Analytics with trend charts, severity breakdown, and spike detection. Workflow management with assignment, priority, snooze, mute/unmute (notification suppression), comments, and batch operations. Security via HTTP Basic Auth or custom lambda (Devise, Warden, session-based). Exception cause chains, enriched HTTP context, custom fingerprinting, CurrentAttributes integration, auto-reopen on recurrence, and sensitive data filtering — all built in.
+Error capture from controllers, jobs, and middleware. Custom-designed dashboard with dark/light mode, search, filtering, and real-time updates (the latter with `turbo-rails` + ActionCable in the host). Analytics with trend charts, severity breakdown, and spike detection. Workflow management with assignment, priority, snooze, mute/unmute (notification suppression), comments, and batch operations. Security via HTTP Basic Auth or custom lambda (Devise, Warden, session-based). Exception cause chains, enriched HTTP context, custom fingerprinting, CurrentAttributes integration, auto-reopen on recurrence, and sensitive data filtering — all built in.
 
 ### Optional Features
 
 <details>
 <summary><strong>Storm Protection — Circuit Breaker + Adaptive Sampling</strong></summary>
 
-When the error rate spikes (a bad deploy throwing thousands of errors a minute), the nightmare scenario for any in-process tracker is amplifying the outage with its own database writes. Storm protection makes the gem **provably degrade itself first** — ON by default.
+When the error rate spikes (a bad deploy throwing thousands of errors a minute), the nightmare scenario for any in-process tracker is amplifying the outage with its own database writes. Storm protection is designed to **shed the gem's own expensive work first** — ON by default. The behaviour is measured (see Overhead below), though a bundled, reproducible benchmark is still to come.
 
 - **Per-fingerprint caps:** past N occurrences/minute per error, context is shed, then rows are sampled deterministically (a fresh exemplar is always kept each minute)
 - **Global circuit breaker:** sustained floods flip the gem to count-only mode — zero per-event I/O, exact in-memory counts reconciled onto error records every 30s. Async mode is gated too (a SolidQueue enqueue is itself a DB write)
@@ -98,7 +158,7 @@ config.storm_open_threshold_per_second = 50  # per process
 
 All thresholds are per process and individually configurable. Disable with one flag.
 
-**Measured overhead** (Apple Silicon, Ruby 4.0): 2.4µs/error with protection active and calm, 2.95µs in count-only mode, 0.2µs when disabled — against a 5µs budget. The check is a digest plus an atomic increment; there is no I/O on the hot path.
+**Overhead:** the check is a digest plus an atomic increment; there is no I/O on the hot path. The maintainer's single-machine measurement (Apple Silicon, Ruby 4.0) was 2.4µs/error with protection active and calm, 2.95µs in count-only mode and 0.2µs when disabled, against a 5µs budget — a reproducible benchmark script is not yet part of the gem.
 </details>
 
 <details>
@@ -108,7 +168,7 @@ See exactly what happened before the crash — SQL queries, controller actions,
 
 - Automatic capture — zero config beyond the enable flag
 - N+1 query detection with aggregate patterns page
-- Deprecation warnings with aggregate view
+- Deprecation warnings with aggregate view (needs the host's deprecation behaviour to include `:notify`; only requests that later raised are seen)
 - Custom breadcrumbs via `RailsErrorDashboard.add_breadcrumb("checkout started", { cart_id: 123 })`
 - Safe by design — fixed-size ring buffer, thread-local, every subscriber wrapped in rescue
 
@@ -146,7 +206,7 @@ Cross-error N+1 detection grouped by SQL fingerprint, and aggregate deprecation
 
 ![N+1 Query Patterns](docs/images/n-plus-one-queries.png)
 
-Requires breadcrumbs to be enabled.
+Requires breadcrumbs to be enabled. Deprecations are seen only when the host's `ActiveSupport::Deprecation` behaviour includes `:notify` (the production default does not) and only inside requests that later raised.
 
 [Complete documentation →](docs/FEATURES.md#n1-query-detection)
 </details>
@@ -154,7 +214,7 @@ Requires breadcrumbs to be enabled.
 <details>
 <summary><strong>Operational Health Panels — Jobs, Database, Cache, ActionCable</strong></summary>
 
-**Job Health** — Auto-detects Sidekiq, SolidQueue, or GoodJob. Per-error table with adapter badge, failed count (color-coded), sorted worst-first.
+**Job Health** — Aggregates the queue stats captured on each error (Sidekiq, SolidQueue or GoodJob auto-detected; needs `enable_system_health`). Not a live queue view — a per-error table with adapter badge, failed count (color-coded), sorted worst-first.
 
 ![Job Health](docs/images/job-health.png)
 
@@ -184,12 +244,12 @@ config.enable_activestorage_tracking = true  # requires enable_breadcrumbs = tru
 <details>
 <summary><strong>LLM Observability — Calls, Tokens, Cost, Tool Use</strong></summary>
 
-Capture every LLM call your app makes — model, latency, token counts, estimated USD cost, and tool-use requests — as breadcrumbs on the error that follows. When a request crashes, you see the chat completion that preceded it: which model was called, how long it took, what it cost, and which tools it asked to invoke.
+Capture your app's LLM calls — through a Faraday middleware, OpenTelemetry GenAI spans or a manual notification; nothing is auto-instrumented — as breadcrumbs on the error that follows, with model, latency, token counts, estimated USD cost and tool-use requests. When a request crashes, you see the chat completion that preceded it: which model was called, how long it took, what it cost, and which tools it asked to invoke.
 
 - Three capture paths — pick whichever matches your stack
 - Cost estimated from a built-in pricing table (Claude 4.x, GPT-4o/o1, Gemini 2.5) — override per-model via `config.llm_pricing_overrides`
 - Tool-call requests summarized inline; tool *execution* spans captured separately via the OTel path
-- Content capture (prompts/completions) **OFF by default** — only token counts and metadata are recorded
+- Prompts and completions are **never recorded** — only token counts and metadata (the `llm_observability_content_capture` flag is reserved and currently a no-op)
 - Same host-app safety guarantees as the rest of the gem — never raises, never blocks the request, every callback rescue-wrapped
 
 ```ruby
@@ -343,7 +403,7 @@ config.enable_git_blame = true
 <details>
 <summary><strong>Code Path Coverage (Diagnostic Mode)</strong></summary>
 
-Enable coverage via a dashboard button to see which production code paths were executed. Source code viewer overlays green checkmarks on executed lines and gray dots on unexecuted lines. Uses Ruby's `Coverage.setup(oneshot_lines: true)` — near-zero overhead, each line fires once. Zero overhead when off.
+Enable coverage via a dashboard button to see which production code paths were executed. Source code viewer overlays green checkmarks on executed lines and gray dots on unexecuted lines. Uses Ruby's `Coverage.setup(oneshot_lines: true)` — near-zero overhead, each line fires once. Zero overhead when off. Diagnostic mode only: coverage is process-global (a multi-threaded Puma blends requests), held in memory and not persisted. No error tracker integrates this; Coverband does it standalone with persistence.
 
 ```ruby
 config.enable_coverage_tracking = true   # shows Enable/Disable buttons on error detail page
@@ -405,9 +465,9 @@ Seven analysis engines built in:
 1. **Baseline Anomaly Alerts** — Statistical spike detection (mean + std dev) with intelligent cooldown
 2. **Fuzzy Error Matching** — Jaccard similarity + Levenshtein distance to find related errors
 3. **Co-occurring Errors** — Detect errors that happen together within configurable time windows
-4. **Error Cascade Detection** — Identify chains (A causes B causes C) with probability and delays
+4. **Error Cascade Detection** — Identify potential cascades (A is followed by B is followed by C) with probability and delays — temporal association, not proven causation
 5. **Error Correlation Analysis** — Correlate errors with app versions, git commits, and users
-6. **Platform Comparison** — iOS vs Android vs Web health metrics side-by-side
+6. **Platform Comparison** — iOS vs Android vs API health metrics side-by-side
 7. **Occurrence Pattern Detection** — Cyclical patterns (business hours, weekends) and burst detection
 
 [Complete documentation →](docs/FEATURES.md#advanced-analytics-features)
@@ -437,7 +497,7 @@ config.enable_instance_variables = true
 <details>
 <summary><strong>Swallowed Exception Detection</strong></summary>
 
-Detect exceptions that are raised but silently rescued — the hardest bugs to find. No other error tracker does this.
+Detect exceptions that are raised but silently rescued — the hardest bugs to find. Only Datadog's paid APM detects rescued exceptions (Ruby 3.3+, and only inside a traced request); RED does it free, without an APM span, and aggregates the raise-vs-rescue ratio per location — no other tracker does that.
 
 - Uses TracePoint(`:raise`) + TracePoint(`:rescue`) to track exception lifecycle
 - Identifies code paths where exceptions are caught but never logged or re-raised
@@ -478,7 +538,8 @@ config.enable_diagnostic_dump = true
 Track Rack Attack security events (throttles, blocklists, tracks) as breadcrumbs attached to errors, with a dedicated summary page.
 
 - Captures throttle, blocklist, and track events automatically
-- Dashboard page at `/errors/rack_attack_summary` with event breakdown
+- Dashboard page at `/errors/rack_attack_summary` with event breakdown and per-rule stats — rack-attack ships no UI of its own
+- Classifies AI-agent user agents (GPTBot, ClaudeBot, …) on `track` events
 - Requires breadcrumbs to be enabled
 
 ```ruby
@@ -496,7 +557,7 @@ Capture unhandled exceptions that crash the Ruby process via an `at_exit` hook
 - Disk-based fallback: writes crash data to disk because the database may be unavailable during shutdown
 - Imported automatically on next boot
 - Captures exception details, backtrace, uptime, GC stats, thread count, and cause chain
-- A self-hosted only feature — impossible for SaaS tools
+- Honeybadger, Bugsnag and AppSignal have `at_exit` reporters too; RED's writes to disk and imports at next boot because the database may already be gone during shutdown
 
 ```ruby
 config.enable_crash_capture = true
@@ -617,7 +678,7 @@ end
 
 ## Languages
 
-The dashboard, its emails and its notification payloads are translated. Eleven locales ship:
+RED ships in English with machine-translated previews for ten additional languages, covering the dashboard, its emails and its notification payloads. Native-speaking Rails developers are invited to review and improve them; once a locale has been reviewed it will be marked individually as community-reviewed. Eleven locales ship:
 
 | Locale | Language | Status |
 |---|---|---|
@@ -661,13 +722,13 @@ SQLite, PostgreSQL, and MySQL/Trilogy — in either shared or separate-database
 Yes. It runs entirely inside your own Rails process — no external services, no SDK calling out, no per-event pricing. Error data never leaves your infrastructure.
 
 **Does it capture local variables like Sentry?**
-Yes — local **and** instance variables at the moment the exception is raised, via `TracePoint(:raise)`, with sensitive-data filtering and configurable limits. This is opt-in and a capability Sentry charges extra for.
+Yes — local **and** instance variables at the moment the exception is raised, via `TracePoint(:raise)`, with sensitive-data filtering and configurable limits. It is opt-in. (Sentry's SDK can also capture locals as an opt-in option; RED adds instance variables and applies your Rails `filter_parameters` automatically.)
 
 **Will a flood of errors take down my app?**
-No. Storm protection (a circuit breaker with adaptive sampling, **ON by default**) makes the gem degrade itself first during error floods — occurrence counts stay exact while it sheds the expensive work. Measured hot-path overhead is ~2.4µs/error.
+No. Storm protection (a circuit breaker with adaptive sampling, **ON by default**) makes the gem degrade itself first during error floods — occurrence counts stay exact while it sheds the expensive work, and a Storm History page shows exactly what was shed. There is no I/O on the hot path — the check is a digest and an atomic increment.
 
 **Does it work with my background jobs?**
-Yes — it auto-detects and supports Sidekiq, SolidQueue, and GoodJob, and can log errors asynchronously through any of them.
+Yes — errors raised in jobs are captured, and it can log errors asynchronously through Sidekiq or SolidQueue (or the in-process `:async` adapter). Sidekiq, SolidQueue and GoodJob are all auto-detected for the job-queue stats stored on each error.
 
 **Does it work with my authentication?**
 Yes — HTTP Basic Auth out of the box, or a custom `authenticate_with` lambda that integrates with Devise, Warden, or session-based auth.
@@ -694,7 +755,7 @@ Rails 7.0–8.1 and Ruby 3.2–4.0.
 - **[Batch Operations](docs/guides/BATCH_OPERATIONS.md)** — Bulk resolve/delete
 - **[Real-Time Updates](docs/guides/REAL_TIME_UPDATES.md)** — Live dashboard
 - **[Error Trends](docs/guides/ERROR_TREND_VISUALIZATIONS.md)** — Charts and analytics
-- **[Translations](docs/guides/TRANSLATIONS.md)** — Seven shipped locales, correcting a string, adding a language
+- **[Translations](docs/guides/TRANSLATIONS.md)** — Eleven shipped locales, correcting a string, adding a language
 
 ### Advanced
 - **[Multi-App Support](docs/MULTI_APP_PERFORMANCE.md)** — Track multiple applications
@@ -703,7 +764,7 @@ Rails 7.0–8.1 and Ruby 3.2–4.0.
 - **[Customization](docs/CUSTOMIZATION.md)** — Customize everything
 - **[Database Options](docs/guides/DATABASE_OPTIONS.md)** — Separate database setup
 - **[Database Optimization](docs/guides/DATABASE_OPTIMIZATION.md)** — Performance tuning
-- **[Mobile App Integration](docs/guides/MOBILE_APP_INTEGRATION.md)** — React Native, Flutter, etc.
+- **[Mobile App Integration](docs/guides/MOBILE_APP_INTEGRATION.md)** — log mobile-originated errors through your own API endpoint, tagged by platform
 - **[FAQ](docs/FAQ.md)** — Common questions answered
 
 [View all documentation →](docs/README.md)
@@ -722,7 +783,7 @@ Built with **CQRS (Command/Query Responsibility Segregation)**:
 
 ## Testing
 
-4,200+ tests covering unit, integration, and browser-based system tests.
+An RSpec suite of unit, request and browser-based system specs runs in CI on every supported Rails version (see the Tests badge above); the current count lives in the CI log rather than here, where it would go stale.
 
 ```bash
 bundle exec rspec                              # Full suite

data/app/jobs/rails_error_dashboard/rack_attack_flush_job.rb

@@ -6,11 +6,17 @@ module RailsErrorDashboard
   # Two usage modes:
   # 1. With a counts hash — dispatched by RackAttackTracker's periodic flush.
   #    Zero I/O in the request path; all DB writes happen here.
-  # 2. Without arguments — scheduled periodic sweep that flushes the current
-  #    thread's buffer (useful as a cron safety net for low-traffic apps where
-  #    the flush interval may not be reached during a request).
+  # 2. Without arguments — sweeps EVERY live thread's buffer.
   #
-  # Example cron (via solid_queue or whenever):
+  # Mode 2 is now a belt-and-braces backstop, not the primary drain. Buffers are
+  # drained at the end of each request and job by the executor hook registered in
+  # the engine (see RackAttackTracker#flush_if_due!), and again at process exit.
+  # Scheduling this job is therefore optional; it only ever finds counts on
+  # threads that are still alive but have not completed a unit of work since
+  # their deadline elapsed. It CANNOT recover counts from a thread that has
+  # already died — Thread.list does not include it.
+  #
+  # Optional cron (via solid_queue or whenever):
   #   every 5.minutes { RailsErrorDashboard::RackAttackFlushJob.perform_later }
   class RackAttackFlushJob < ApplicationJob
     queue_as :default

data/lib/rails_error_dashboard/commands/find_or_increment_error.rb

@@ -16,6 +16,17 @@ module RailsErrorDashboard
     # stamped by the first occurrence that claims it, so history migrates
     # itself without a backfill. An exact match always wins over a NULL one.
     class FindOrIncrementError
+      # Context that describes THIS occurrence rather than the error as a
+      # group. It is refreshed on every recurrence so the row always shows the
+      # latest moment of failure, not the first one in the 24 h window. Keys
+      # absent from @attributes (feature disabled, column not migrated, or a
+      # storm :lite capture that shed context) leave the stored payload alone —
+      # a shed capture must never blank out a good snapshot.
+      REFRESHED_CONTEXT = %i[
+        breadcrumbs system_health local_variables instance_variables
+        http_method hostname content_type request_duration_ms
+      ].freeze
+
       def self.call(error_hash, attributes = {})
         new(error_hash, attributes).call
       end
@@ -69,6 +80,13 @@ module RailsErrorDashboard
              .order(Arel.sql("CASE WHEN environment IS NULL THEN 1 ELSE 0 END"))
       end
 
+      # The subset of REFRESHED_CONTEXT this occurrence actually captured.
+      def latest_context
+        REFRESHED_CONTEXT.each_with_object({}) do |key, refreshed|
+          refreshed[key] = @attributes[key] unless @attributes[key].nil?
+        end
+      end
+
       # {} unless this is a legacy NULL-environment row being claimed.
       def environment_adoption(error)
         return {} unless ErrorLog.column_names.include?("environment")
@@ -86,6 +104,7 @@ module RailsErrorDashboard
           request_params: @attributes[:request_params] || error.request_params,
           user_agent: @attributes[:user_agent] || error.user_agent,
           ip_address: @attributes[:ip_address] || error.ip_address,
+          **latest_context,
           **environment_adoption(error)
         )
         error
@@ -103,6 +122,7 @@ module RailsErrorDashboard
           request_params: @attributes[:request_params] || error.request_params,
           user_agent: @attributes[:user_agent] || error.user_agent,
           ip_address: @attributes[:ip_address] || error.ip_address,
+          **latest_context,
           **environment_adoption(error)
         }
         attrs[:reopened_at] = Time.current if ErrorLog.column_names.include?("reopened_at")
@@ -126,6 +146,7 @@ module RailsErrorDashboard
           retry_existing.update!(
             occurrence_count: retry_existing.occurrence_count + 1,
             last_seen_at: Time.current,
+            **latest_context,
             **environment_adoption(retry_existing)
           )
           retry_existing

data/lib/rails_error_dashboard/configuration.rb

@@ -205,7 +205,7 @@ module RailsErrorDashboard
     # their own table, independent of error capture (breadcrumbs optional).
     attr_accessor :enable_rack_attack_tracking          # Master switch (default: false)
     attr_accessor :rack_attack_max_cache_size           # Max buffered keys per thread (default: 1000)
-    attr_accessor :rack_attack_flush_interval           # Seconds between DB flushes (default: 60)
+    attr_accessor :rack_attack_flush_interval           # Seconds between DB flushes (default: 5)
 
     # ActionCable event tracking (requires enable_breadcrumbs = true)
     attr_accessor :enable_actioncable_tracking          # Master switch (default: false)
@@ -429,7 +429,12 @@ module RailsErrorDashboard
       # Persists to its own table; does NOT require breadcrumbs.
       @enable_rack_attack_tracking = false
       @rack_attack_max_cache_size = 1000 # Max buffered keys per thread (LRU eviction)
-      @rack_attack_flush_interval = 60   # Seconds between DB flushes
+      # Max age of buffered events before they are written out. Lowered from 60
+      # to 5 alongside the end-of-request drain (issue #170): the executor hook
+      # gates on this interval, so it is the upper bound on how stale the Rate
+      # Limits page can be, not a per-request cost. A flood still collapses to
+      # roughly one write per thread per interval.
+      @rack_attack_flush_interval = 5    # Seconds between DB flushes
 
       # ActionCable event tracking defaults - OFF by default (opt-in, requires breadcrumbs)
       @enable_actioncable_tracking = false
@@ -628,7 +633,7 @@ module RailsErrorDashboard
         # Rack::Attack initializer has loaded, so a missing constant here does
         # not prove it will still be missing at after_initialize (when the
         # subscriber actually registers). Auto-disabling would break that case.
-        unless defined?(::Rack::Attack)
+        unless rack_attack_defined?
           warnings << "enable_rack_attack_tracking is enabled but the rack-attack gem " \
                       "does not appear to be loaded. No events will be recorded until " \
                       "Rack::Attack is installed and configured."
@@ -1033,6 +1038,14 @@ module RailsErrorDashboard
 
     # Detect where the engine is mounted in the host app's routes.
     # @return [String] mount path (default: "/red")
+    # Extracted so specs can simulate the gem's absence. rack-attack is in the
+    # dev bundle (so specs can drive the real middleware), which means
+    # ::Rack::Attack is always defined during the suite and absence can no
+    # longer be produced by simply not requiring it.
+    def rack_attack_defined?
+      defined?(::Rack::Attack) ? true : false
+    end
+
     def detect_engine_mount_path
       return "/red" unless defined?(Rails) && Rails.application
 

data/lib/rails_error_dashboard/engine.rb

@@ -86,6 +86,21 @@ module RailsErrorDashboard
          defined?(Rack::Attack)
         RailsErrorDashboard::Subscribers::RackAttackSubscriber.subscribe!
 
+        # Drain buffered counts at the end of every request and job.
+        #
+        # Without this the buffer is only ever drained by a LATER event arriving
+        # on the SAME thread (see RackAttackTracker#flush_if_due!), so a rule that
+        # matches once stays invisible until the process exits, and counts on a
+        # Puma thread that retires are lost outright rather than delayed.
+        #
+        # to_complete fires after the response body is closed, so the client
+        # already has its bytes — this never delays a request (safety rule 2).
+        # It also fires when the app raised, and is re-entrant, so nested
+        # executor blocks do not double-flush.
+        Rails.application.executor.to_complete do
+          RailsErrorDashboard::Services::RackAttackTracker.flush_if_due!
+        end
+
         # Buffered counts live on the Puma threads that served the requests and
         # are only written out on the flush interval, which a low-traffic rule
         # may never reach. Without this, everything still buffered at SIGTERM

data/lib/rails_error_dashboard/services/rack_attack_tracker.rb

@@ -24,7 +24,23 @@ module RailsErrorDashboard
     # - Async flush via background job
     class RackAttackTracker
       COUNTS_THREAD_KEY = :red_rack_attack_counts
-      FLUSH_THREAD_KEY  = :red_rack_attack_last_flush
+
+      # Monotonic timestamp of the moment the buffer became non-empty — the
+      # DEADLINE clock, not a "last flush" clock.
+      #
+      # WHY THE DISTINCTION MATTERS: this used to hold the last flush time and be
+      # seeded lazily inside maybe_flush! with `||= now`, which meant the very
+      # first event of a buffer set the clock to now and then compared `now - now
+      # >= interval` — false, always. A rule that matched once and never again
+      # therefore never flushed at all, and a manual `curl` test showed an empty
+      # table indefinitely (issue #170, third report). Seeding when the buffer
+      # STARTS filling makes the guarantee "buffered data is never older than
+      # flush_interval", which is the property the dashboard actually needs.
+      DEADLINE_THREAD_KEY = :red_rack_attack_deadline_at
+
+      # Kept as an alias so a host or spec holding the old key name still clears
+      # the right slot. Both are cleared together in reset!.
+      FLUSH_THREAD_KEY = :red_rack_attack_last_flush
 
       # Field length caps — must match the column limits in the migration so that
       # truncation happens before the value ever reaches the unique upsert index.
@@ -38,6 +54,10 @@ module RailsErrorDashboard
       # eviction. Without this the evicted count vanishes silently and the
       # dashboard under-reports with no indication anything was lost — the same
       # problem StormProtection::CountBuffer solves with an overflow counter.
+      # Fallback when configuration is unreadable. Must match
+      # Configuration#rack_attack_flush_interval's default.
+      DEFAULT_FLUSH_INTERVAL = 5
+
       OVERFLOW_RULE       = "__overflow__"
       OVERFLOW_MATCH_TYPE = "overflow"
 
@@ -69,6 +89,13 @@ module RailsErrorDashboard
           )
 
           counts = (Thread.current[COUNTS_THREAD_KEY] ||= {})
+
+          # Start the deadline the moment the buffer goes from empty to non-empty.
+          # Doing it here (rather than lazily at flush-check time) is what makes
+          # "never older than flush_interval" true for a buffer that receives
+          # exactly one event and then goes quiet.
+          Thread.current[DEADLINE_THREAD_KEY] ||= monotonic_now if counts.empty?
+
           counts[key] = (counts[key] || 0) + 1
 
           # LRU eviction — bounds memory under rotating-discriminator attacks.
@@ -98,7 +125,11 @@ module RailsErrorDashboard
 
           snapshot = counts.dup
           counts.clear
-          Thread.current[FLUSH_THREAD_KEY] = Time.now.to_f
+          # Buffer is empty again, so there is nothing to be late: clear the
+          # deadline. The next record! reseeds it. Leaving a stale timestamp
+          # here would make the very next event look instantly overdue.
+          Thread.current[DEADLINE_THREAD_KEY] = nil
+          Thread.current[FLUSH_THREAD_KEY] = nil
 
           dispatch_flush(snapshot, sync: sync)
           nil
@@ -132,6 +163,7 @@ module RailsErrorDashboard
 
               snapshot = counts.dup
               counts.clear
+              thread[DEADLINE_THREAD_KEY] = nil
               thread[FLUSH_THREAD_KEY] = nil
 
               dispatch_flush(snapshot, sync: true)
@@ -149,10 +181,51 @@ module RailsErrorDashboard
           nil
         end
 
+        # Drain this thread's buffer at the end of a unit of work (a request or a
+        # job), if it has been waiting longer than flush_interval.
+        #
+        # WHY THIS EXISTS (issue #170, third report): before this, the ONLY
+        # in-process drain was maybe_flush! inside record, so the buffer could
+        # only ever be flushed by a LATER event landing on the SAME thread. Two
+        # consequences, both reported as "no events are recorded at all":
+        #
+        #   1. A rule that matched once showed nothing until the process exited.
+        #      `curl` once, look at the dashboard, see an empty table — forever.
+        #   2. Worse, Puma reuses and retires threads. Counts buffered on a thread
+        #      that dies are unreachable to flush_all_threads! (it walks
+        #      Thread.list), so they were lost outright, not merely delayed.
+        #      Measured: 5 of 5 events lost when the serving threads exited.
+        #
+        # ActiveSupport::Executor#to_complete is the right boundary because Rails
+        # already guarantees it runs once per request and once per job. Crucially,
+        # ActionDispatch::Executor returns a Rack::BodyProxy and defers the hook
+        # until the SERVER CLOSES THE RESPONSE BODY — so this runs after the client
+        # has its bytes and cannot delay the response (safety rule 2).
+        #
+        # The flush is gated on flush_due?, so a flood does not turn into one
+        # UPDATE per request — the exact regression #143's buffer exists to
+        # prevent. Measured 0.068 ms/req gated vs 0.508 ms/req ungated.
+        def flush_if_due!
+          return unless enabled?
+          return unless flush_due?
+
+          # sync: the response is already sent, so there is nothing left to block,
+          # and enqueueing a job per interval would be more overhead than the
+          # single upsert it replaces.
+          flush!(sync: true)
+          nil
+        rescue => e
+          RailsErrorDashboard::Logger.debug(
+            "[RailsErrorDashboard] RackAttackTracker.flush_if_due! failed: #{e.class} - #{e.message}"
+          )
+          nil
+        end
+
         # Clear thread-local state without persisting. Used by specs and by
         # thread teardown paths.
         def reset!
           Thread.current[COUNTS_THREAD_KEY] = nil
+          Thread.current[DEADLINE_THREAD_KEY] = nil
           Thread.current[FLUSH_THREAD_KEY] = nil
           nil
         rescue => e
@@ -179,6 +252,24 @@ module RailsErrorDashboard
           parts.fill("", parts.length, 6 - parts.length)
         end
 
+        # Cheap deadline check — a float subtraction, no I/O.
+        #
+        # Returns true when the buffer has been waiting at least flush_interval.
+        # Uses a monotonic clock: Time.now can jump backwards (NTP correction,
+        # leap second) and would then defer the flush indefinitely.
+        def flush_due?
+          deadline = Thread.current[DEADLINE_THREAD_KEY]
+          return false if deadline.nil?
+
+          (monotonic_now - deadline) >= flush_interval
+        rescue => e
+          false
+        end
+
+        def monotonic_now
+          Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        end
+
         private
 
         def enabled?
@@ -215,13 +306,9 @@ module RailsErrorDashboard
           )
         end
 
-        # Cheap periodic flush check — a float subtraction, no I/O.
+        # Cheap periodic flush check on the record path — no I/O.
         def maybe_flush!
-          now = Time.now.to_f
-          last_flush = Thread.current[FLUSH_THREAD_KEY] ||= now
-          return unless (now - last_flush) >= flush_interval
-
-          flush!
+          flush! if flush_due?
         end
 
         # Dispatch asynchronously so the request path never waits on the DB.
@@ -257,9 +344,9 @@ module RailsErrorDashboard
         end
 
         def flush_interval
-          RailsErrorDashboard.configuration.rack_attack_flush_interval || 60
+          RailsErrorDashboard.configuration.rack_attack_flush_interval || DEFAULT_FLUSH_INTERVAL
         rescue => e
-          60
+          DEFAULT_FLUSH_INTERVAL
         end
 
         # Truncate to the column limit and strip the key separator. A rule name

data/lib/rails_error_dashboard/version.rb

@@ -1,3 +1,3 @@
 module RailsErrorDashboard
-  VERSION = "0.11.0"
+  VERSION = "0.11.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rails_error_dashboard
 version: !ruby/object:Gem::Version
-  version: 0.11.0
+  version: 0.11.2
 platform: ruby
 authors:
 - Anjan Jagirdar
@@ -219,29 +219,24 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '0.15'
-description: 'Own your errors. Own your stack. A fully open-source, self-hosted error
-  tracking Rails engine — a free Sentry alternative that runs entirely inside your
-  own process, with no external services and zero recurring cost. Captures what SaaS
-  tools charge extra for: local and instance variables at the moment of failure (via
-  TracePoint), exception cause chains, swallowed-exception detection, breadcrumbs,
-  and system-health snapshots (GC, memory, threads, connection pool, Puma). Plus N+1
-  query detection, storm protection (a circuit breaker that shields your app from
-  error floods, ON by default), multi-app support, error sampling, and async logging
-  via Sidekiq, SolidQueue, or GoodJob. Runs on SQLite, PostgreSQL, or MySQL/Trilogy
-  — in your app''s existing database or an isolated separate error database. Beautiful
-  dashboard UI (dark/light), multi-channel notifications (Slack, Email, Discord, PagerDuty,
-  webhooks), workflow management, advanced analytics, platform detection (iOS/Android/Web/API),
-  and two-way issue sync with GitHub, GitLab, Codeberg, and Linear. Also: LLM observability,
-  AI-powered debugging help, and OpenTelemetry span export. The dashboard, mailers
-  and notification payloads are translated into 11 languages: English, German, Spanish,
-  French, Brazilian Portuguese, Italian, Polish, Russian, Ukrainian, Japanese, and
-  Simplified Chinese — 1,515 source strings, set via config.dashboard_locale with
-  a per-user picker. RED translates through its own private I18n backend and never
-  mutates your app''s I18n; a missing translation falls back to English. Backtraces,
-  exception names and webhook payload keys stay English by design. The ten non-English
-  locales are machine-translated and not yet reviewed by native speakers. 5-minute
-  setup, works out-of-the-box. Rails 7.0-8.1, Ruby 3.2-4.0. BETA: API may change before
-  v1.0.0. Live demo: https://rails-error-dashboard.anjan.dev (gandalf/youshallnotpass)'
+description: 'Rails Error Dashboard (RED) is an open-source, self-hosted Rails engine
+  for investigating production exceptions without sending error data to a monitoring
+  vendor. It groups errors and records request context and cause chains and, when
+  enabled, breadcrumbs plus local and instance variables captured before Ruby unwinds
+  the stack. RED attaches Rails and Ruby runtime health to the error record on every
+  captured occurrence, including Active Record pool, Puma, background jobs, GC, memory,
+  threads, file descriptors and system pressure. Built-in storm protection progressively
+  sheds expensive context and I/O during error floods while retaining useful exemplars
+  and exact occurrence counts. Run RED with your application''s database or an isolated
+  error database. It supports PostgreSQL, MySQL/Trilogy and SQLite, and includes workflow,
+  notifications (Slack, Email, Discord, PagerDuty, webhooks), two-way issue sync with
+  GitHub, GitLab, Codeberg and Linear, Copy as RSpec/curl/LLM, swallowed-exception
+  detection, LLM observability without prompt capture, OpenTelemetry span export and
+  Rails-specific operational views. The dashboard is translated into 11 languages
+  (machine-translated outside English, awaiting native review). A self-hosted Sentry
+  alternative that keeps error data in your own database. The gem is MIT and free
+  forever. Supports Rails 7.0-8.1 and Ruby 3.2-4.0. Beta: APIs may change before 1.0.
+  Live demo: https://rails-error-dashboard.anjan.dev'
 email:
 - anjan.jagirdar@gmail.com
 executables: []
@@ -561,7 +556,7 @@ metadata:
   funding_uri: https://github.com/sponsors/AnjanJ
 post_install_message: |
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-    RED (Rails Error Dashboard) v0.11.0
+    RED (Rails Error Dashboard) v0.11.2
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 
   First install:
@@ -599,6 +594,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 3.6.9
 specification_version: 4
-summary: Self-hosted error tracking for Rails — local variables, system health, separate
-  or shared database, dashboard in 11 languages. A free, open-source Sentry alternative.
+summary: Rails-native, self-hosted error monitoring with exception-time Ruby state,
+  runtime health, and storm-safe capture.
 test_files: []