solid_observer 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (65) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +73 -0
  3. data/README.md +198 -36
  4. data/app/assets/javascripts/solid_observer/live_poll.js +376 -0
  5. data/app/controllers/concerns/solid_observer/paginatable.rb +17 -0
  6. data/app/controllers/concerns/solid_observer/require_persistence_mode.rb +19 -0
  7. data/app/controllers/concerns/solid_observer/require_solid_queue.rb +19 -0
  8. data/app/controllers/solid_observer/application_controller.rb +69 -0
  9. data/app/controllers/solid_observer/dashboard_controller.rb +79 -0
  10. data/app/controllers/solid_observer/events_controller.rb +50 -0
  11. data/app/controllers/solid_observer/jobs_controller.rb +85 -0
  12. data/app/controllers/solid_observer/storages_controller.rb +12 -0
  13. data/app/helpers/solid_observer/application_helper.rb +95 -0
  14. data/app/helpers/solid_observer/dashboard_helper.rb +39 -0
  15. data/app/models/solid_observer/queue_event.rb +134 -0
  16. data/app/models/solid_observer/queue_metric.rb +1 -1
  17. data/app/presenters/solid_observer/execution_presenter.rb +50 -0
  18. data/app/views/layouts/solid_observer/application.html.erb +470 -0
  19. data/app/views/solid_observer/dashboard/_chart.html.erb +28 -0
  20. data/app/views/solid_observer/dashboard/_live_state.html.erb +20 -0
  21. data/app/views/solid_observer/dashboard/_queue_table.html.erb +34 -0
  22. data/app/views/solid_observer/dashboard/_right_now.html.erb +3 -0
  23. data/app/views/solid_observer/dashboard/_throughput.html.erb +32 -0
  24. data/app/views/solid_observer/dashboard/index.html.erb +113 -0
  25. data/app/views/solid_observer/errors/storage_unavailable.html.erb +27 -0
  26. data/app/views/solid_observer/events/index.html.erb +53 -0
  27. data/app/views/solid_observer/events/show.html.erb +47 -0
  28. data/app/views/solid_observer/jobs/index.html.erb +61 -0
  29. data/app/views/solid_observer/jobs/show.html.erb +71 -0
  30. data/app/views/solid_observer/shared/_empty_state.html.erb +5 -0
  31. data/app/views/solid_observer/shared/_pagination.html.erb +17 -0
  32. data/app/views/solid_observer/shared/_stat_card.html.erb +9 -0
  33. data/app/views/solid_observer/storages/show.html.erb +39 -0
  34. data/bin/quality_gate +95 -0
  35. data/config/routes.rb +17 -0
  36. data/db/migrate/20260424000001_add_composite_indexes_to_queue_events.rb +30 -0
  37. data/lib/generators/solid_observer/install_generator.rb +12 -25
  38. data/lib/generators/solid_observer/templates/initializer.rb.tt +5 -6
  39. data/lib/solid_observer/base_metric.rb +1 -1
  40. data/lib/solid_observer/chart_buffer.rb +83 -0
  41. data/lib/solid_observer/cli/base.rb +2 -2
  42. data/lib/solid_observer/cli/jobs.rb +2 -2
  43. data/lib/solid_observer/cli/status.rb +20 -2
  44. data/lib/solid_observer/cli/storage.rb +41 -32
  45. data/lib/solid_observer/configuration.rb +67 -34
  46. data/lib/solid_observer/correlation_id_resolver.rb +8 -6
  47. data/lib/solid_observer/engine.rb +75 -15
  48. data/lib/solid_observer/params/events_filter.rb +37 -0
  49. data/lib/solid_observer/params/jobs_filter.rb +35 -0
  50. data/lib/solid_observer/queries/events_query.rb +27 -0
  51. data/lib/solid_observer/queries/execution_finder.rb +42 -0
  52. data/lib/solid_observer/queries/job_executions_query.rb +73 -0
  53. data/lib/solid_observer/queue_event_buffer.rb +163 -22
  54. data/lib/solid_observer/queue_stats.rb +165 -19
  55. data/lib/solid_observer/services/cleanup_storage.rb +60 -42
  56. data/lib/solid_observer/services/database_size.rb +86 -0
  57. data/lib/solid_observer/services/flush_event_buffer.rb +31 -15
  58. data/lib/solid_observer/services/install_migrations.rb +49 -0
  59. data/lib/solid_observer/services/record_event.rb +53 -14
  60. data/lib/solid_observer/services/ui_auth_check.rb +65 -0
  61. data/lib/solid_observer/subscriber.rb +15 -8
  62. data/lib/solid_observer/version.rb +1 -1
  63. data/lib/solid_observer.rb +7 -0
  64. data/lib/tasks/solid_observer.rake +10 -2
  65. metadata +55 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 67f33210e5da882874417d97b040e5c1ea8a30b4cf5f628645fa76405bd38379
4
- data.tar.gz: ffa7843af6bdd14bc23762da2f42a245073bf4776ec6b2de1aece77852c41d9d
3
+ metadata.gz: a745bb3571f70b207e14f2e88f6162db6b2322a889d42f5ebd2a50e80c5b8014
4
+ data.tar.gz: d05f835df0503f04f88223c0879145a2dab13279cd5bfc59346545c0e8263f1c
5
5
  SHA512:
6
- metadata.gz: 3b8ca565611c5a343b55131fa63d84ff5ad8c47a954b9103914764d90a1d381b6dcd39923c71b9c467011c2f6f0bc17cb724a8b8a60e16eef32110142088b007
7
- data.tar.gz: 73e295dd624824b396bcd0ccca2c4d740f3f27287ce226b7816a92569193b996527e6cd5c8e0967c67a18b1b9c19fe100163ee49d5a5d3ebcb6e5b18624fa9bd
6
+ metadata.gz: 8417c8013a7c9ff03a9d52481af63889adaba093abf0c9b041842664af7fb15edde929f7c7b94c6df12cbac00ca41d2aeee0f488fd7ffe831abf9d5f873bff09
7
+ data.tar.gz: 92147e32675291ec743e055c55382bf719a9ab6dd10e2e507789d0163b703e06490ad97dbb785e0a9c0b51b8eb72eb7a849d95504ac52c8680e53a137b50e4e4
data/CHANGELOG.md CHANGED
@@ -1,3 +1,76 @@
1
+ ## [0.3.0] - 2026-05-25
2
+
3
+ > Note: there is no separate `0.2.0` gem release — work originally scoped for v0.2.0 (stability + refactoring, SO-040 through SO-049) was folded into this release.
4
+
5
+ Headline: **Web UI Dashboard** + stability hardening from pre-release review.
6
+
7
+ ### Added
8
+ - Web UI dashboard at `/solid_observer` — queue stats, jobs browser, events log, storage info, responsive layout, optional HTTP Basic Auth
9
+ - Web UI config: `ui_enabled`, `ui_username`, `ui_password`, `ui_base_controller`, `ui_refresh_interval`
10
+ - Dashboard "Live" toggle: when enabled, only the Right-Now card region (`so_right_now` Turbo Frame) reloads on the configured cadence (`SolidObserver.config.ui_refresh_interval`). Scoped cards stay frozen until the range selector changes. Toggle state lives in the `?live=on` URL param so it survives range navigations.
11
+ - Dashboard now supports a Time-Range selector (`15m`, `30m`, `1h`, `7h`, `1d`, `7d`, `14d`, default `1h`) that scopes the new "Last <range>" card region (Performed / Failed / Enqueue rate). The "Right Now" region (Ready / Scheduled / Claimed / Failed-awaiting-retry / Workers) is unaffected by the range and reflects current SolidQueue state.
12
+ - Retry / discard actions with confirmation dialogs and CSRF protection
13
+ - `QueueEventBuffer#metrics` and `#shutdown` (graceful drain on app exit)
14
+ - Configuration: `max_buffer_size` (default 10_000), `buffer_overflow_strategy` (`:drop_old` / `:drop_new`), `filter_cache_ttl`
15
+ - `Services::DatabaseSize` for cross-adapter table-size measurement; composite indexes and `distinct_job_classes` / `distinct_queue_names` scopes
16
+ - Dashboard now shows three throughput counters (Performed last hour, Failed last 24h, Enqueue rate last 5 min) sourced from the events table, alongside existing point-in-time counters. Throughput counters are persistence-mode only.
17
+ - Stat counter subtitles ("queued" / "future runs" / "in progress" / "awaiting retry" / "active processes") clarify SolidQueue lifecycle terminology.
18
+ - Conceptual hint banner on the dashboard explains the difference between Jobs tab (in-flight + failed) and Events tab (historical record).
19
+ - Jobs tab now has an empty state with a hint pointing to Events tab for completed-job history.
20
+ - Dashboard polled chart strip and unified card grid: a JSON-fed polling client refreshes six stat cards (Ready, Scheduled, Claimed, Workers, Failed, Enqueue Rate) and three sparklines (Performed/min, Ready depth, Failed/min) every `SolidObserver.config.ui_refresh_interval` seconds (default `30`, `0` disables polling). In realtime mode only the Ready sparkline renders and the Enqueue Rate card is omitted. Toggle in the dashboard top bar enables/disables polling; state lives in `?live=on` URL param. Hand-rolled inline-SVG sparklines, no JS framework, no external chart library.
21
+
22
+ ### Fixed
23
+ - **Duration was displayed off by 1000x.** `RecordEvent` stored `ActiveSupport::Notifications::Event#duration` (milliseconds) directly; `format_duration` interpreted it as seconds. Fixed by converting ms → seconds at write time. No data migration needed (fixed before v0.3.0 release); run `bin/rails solid_observer:storage:purge` to clear pre-fix local data.
24
+ - **Dashboard chart strip renders populated polylines on first paint** instead of empty placeholders. Server-side `spark_points` helper mirrors the JS `Sparkline.render` projection formula so the first HTML response ships with real data; the first JS poll appends one additional segment with no visible "empty → full" transition.
25
+ - **Live toggle cadence label stays in sync with toggle state.** The `.so-toggle__cadence` span now carries `aria-live="polite"` and is updated in the same synchronous tick as the `--on` class toggle. Server-side ERB hardcodes `"5s"` / `"off"` matching the JS literals.
26
+ - Jobs details page no longer crashes for `SolidQueue::FailedExecution`; Queue and Priority now fall back to underlying `SolidQueue::Job` values and show `N/A` when unavailable.
27
+ - Web UI now works on API-only Rails hosts. The engine ships its own Cookies / Session::CookieStore (`key: "_solid_observer_session"`) / Flash middleware stack, so requests routed to `/solid_observer/*` get the middleware they need regardless of whether the host app strips them via `config.api_only = true`. Previously, API-only hosts hit `NoMethodError: undefined method 'flash' for an instance of ActionDispatch::Request` rendering the dashboard layout.
28
+ - `bin/rails solid_observer:install:migrations` now respects `migrations_paths` from the `solid_observer_queue` connection in `config/database.yml`. When the host configures a dedicated migration folder (e.g. `db/solid_observer_migrate`), the install task copies migrations directly there instead of `db/migrate/`. Previously, operators in multi-database setups had to manually move the files after install to prevent cross-database migration contamination.
29
+ - Web UI now degrades gracefully when the `solid_observer_queue` database is missing or unreachable at request time. Previously, requests to `/solid_observer/*` raised `ActiveRecord::NoDatabaseError` / `ConnectionNotEstablished` with a raw 500 stack trace. The engine now renders a 503 "Storage unavailable" page using the dashboard layout, with an actionable hint to run migrations or check `database.yml`. Realtime mode is unaffected. Boot-time resilience (Engine activation skip) was already in place; this closes the equivalent gap at request time.
30
+ - Engine boot no longer requires a live DB (Docker/CI/K8s safe); table check uses `BaseEvent.connection_pool` (multi-DB safe); broader rescue covering adapter-specific connection errors
31
+ - `QueueEventBuffer` hard-caps at `max_buffer_size` with overflow strategy; uses a single persistent `Concurrent::TimerTask` instead of spawning a thread per flush
32
+ - Storage monitoring uses adapter-native size queries (SQLite / PostgreSQL / MySQL / Trilogy) — fixes `0 MB` readings off SQLite
33
+ - `EventsController` and `JobsController` filter dropdowns no longer full-table-scan on every request (cached via `filter_cache_ttl`)
34
+ - `RecordEvent` correctly handles real `ActiveJob::Base` payload objects (with hash fallback)
35
+
36
+ ### Security
37
+ - Job arguments removed from persisted event metadata and the jobs detail view (PII reduction)
38
+ - Web UI HTTP Basic Auth now requires **both** `ui_username` and `ui_password` to be configured. Previously, setting only `ui_username` (with `ui_password` missing or `nil`) would still trigger an auth challenge that any blank-password request would pass `secure_compare("", "")`, granting unauthenticated access. The README's "both must be set" guidance now matches the implementation; misconfigured auth now ships unauthenticated rather than allowing a bypass.
39
+ - Boot-time `Engine.check_ui_authentication` now warns on partial misconfiguration (exactly one of `ui_username` / `ui_password` set), naming the missing credential. Previously the check exited silently as soon as `ui_username.present?`, hiding the fail-open auth misconfiguration from operators.
40
+
41
+ ### Documentation
42
+ - Documented multi-adapter installation (PG host + SQLite observer DB) inline in Database Setup, with explicit `adapter:` override, `gem "sqlite3"` Bundler note, `migrations_paths` migration-isolation guidance, and cross-reference from the install steps.
43
+
44
+ ### Removed
45
+ - Removed the legacy implicit dashboard auto-refresh (`<meta http-equiv="refresh">` plus inline fetch/DOM-swap script for `.so-content`). Replaced by explicit, opt-in Live Mode targeting only the Right-Now frame.
46
+ - Removed the legacy `GET /solid_observer/right_now` HTML endpoint and its action template (it was the SO-060-era polling target that returned a partial-only HTML response wrapped in a turbo-frame). Replaced by `GET /solid_observer/poll_data` which returns JSON for the polling client. The `live_poll.js` script-delivery route is unchanged.
47
+ - **Removed `SolidObserver.config.ui_refresh_interval`** (was unreliable; cadence is now hardcoded at 5s). Upgrade note: remove the line from your initializer or boot will raise `NoMethodError`.
48
+
49
+ ### Changed
50
+ - Web UI controllers refactored to thin actions + query/param/presenter objects; Rails built-in number helpers replace custom ones
51
+ - Specs: `allow_any_instance_of` → `instance_double`; sleep-based timer specs use deterministic synchronisation; dead private-method `describe` blocks removed
52
+ - `.reek.yml` suppressions trimmed; hot-path services/buffer/engine methods refactored to pass `TooManyStatements` without new suppressions
53
+ - `SolidObserver.config.ui_refresh_interval` now controls Live Mode polling cadence instead of implicit full-page dashboard refresh. Default value is unchanged.
54
+ - Dashboard default range is `15m` (was `1h`); aligns with poll default.
55
+ - Live polling pauses while the tab is hidden and resumes on return (one immediate tick on visibility return so the user doesn't wait up to 5s for fresh data).
56
+ - Jobs tab default filter changed from `status=ready` to `status=all_active` (ready + scheduled + claimed + failed).
57
+ - Duration values on the Events index and detail pages now use per-event-type semantic context via `<abbr title="...">` tooltips so operators can distinguish enqueue call latency (`job_enqueued`) from perform-time duration (`job_completed` / `job_failed` / `job_discarded`).
58
+ - Refreshed the engine UI to a minimalist visual language: light surfaces, near-black text, restrained semantic colour accents reserved for badges/state, hairline separators, consistent rounding. Sidebar moves from dark slate to a light surface. No external CSS dependencies, no JS, no dark mode (single light theme).
59
+ - Dashboard: replaced the multi-line "Recent Failures" panel with a single-line **Stability** indicator (pill badge + summary + "View failures" link). Three states based on rolling failure counts: **Stable** (no failures in last 24h), **Degraded** (failures in last 24h but none in last hour), **Critical** (any failure in the last hour). Click-through targets the Events page filtered to `job_failed`.
60
+ - Dashboard: removed the "Jobs tab / Events tab" orientation banner. The distinction is discoverable via navigation; the dashboard is reserved for signal.
61
+ - Dashboard layout consolidated: dropped the previous two-frame split (Right-Now + Scoped) and the redundant Performed-in-range / Failed-in-range cards (those metrics now live in the polled sparkline strip).
62
+ - Live toggle restyled as a pill switch with cadence visible in-line ("Live · 2s" / "Live · off") and a pulsing dot when active.
63
+
64
+ ## [0.1.1] - 2026-02-10
65
+
66
+ ### Added
67
+ - **Real-time mode** (`storage_mode: :realtime`) — run SolidObserver without database migrations
68
+ - Queue status and job management CLI commands work without any SolidObserver database
69
+ - `storage_mode` configuration option (`:persistence` default, `:realtime` for no-DB operation)
70
+ - `persistence_mode?` and `realtime_mode?` configuration predicates
71
+ - Graceful `CLI::Storage` message when in real-time mode
72
+ - Event buffering, metric incrementing, and cleanup automatically disabled in real-time mode
73
+
1
74
  ## [0.1.0] - 2026-02-02
2
75
 
3
76
  ### Added
data/README.md CHANGED
@@ -2,33 +2,34 @@
2
2
  <picture>
3
3
  <source media="(prefers-color-scheme: dark)" srcset=".github/solid_logo_dark.svg">
4
4
  <source media="(prefers-color-scheme: light)" srcset=".github/solid_logo_light.svg">
5
- <img alt="SolidObserver" src=".github/solid_logo_light.svg" width="250">
5
+ <img alt="SolidObserver" src=".github/solid_logo_light.svg" width="220">
6
6
  </picture>
7
7
  </p>
8
8
 
9
9
  <p align="center">
10
- <strong>Observe your Solid Stack like a pro!</strong>
11
- </p>
12
-
13
- <p align="center">
14
- <a href="https://github.com/bart-oz/solid_observer/releases"><img src="https://img.shields.io/badge/version-0.1.0-blue.svg" alt="Version"></a>
10
+ <a href="https://github.com/bart-oz/solid_observer/releases"><img src="https://img.shields.io/badge/version-0.3.0-blue.svg" alt="Version"></a>
15
11
  <a href="LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
16
12
  <a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/tests-passing-brightgreen.svg" alt="Tests"></a>
17
- <a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/coverage-93.23%25-brightgreen.svg" alt="Coverage"></a>
13
+ <a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/coverage-96.98%25-brightgreen.svg" alt="Coverage"></a>
18
14
  </p>
19
15
 
20
16
  ---
17
+ <p align="center">
18
+ <img src=".github/assets/dash_1.png" alt="Dashboard overview" width="700">
19
+ </p>
21
20
 
22
- SolidObserver is a production-grade observability solution for Rails 8's Solid Stack. Starting with **Solid Queue** monitoring in v0.1.0, it provides unified visibility into your background job processing with CLI tools, metrics collection, and distributed tracing support.
21
+ SolidObserver is a production-grade observability solution for Rails 8's Solid Stack. Starting with **Solid Queue** monitoring in v0.3.0, it provides unified visibility into your background job processing with a Web UI dashboard, CLI tools, metrics collection, and distributed tracing support.
23
22
 
24
- ## Features (v0.1.0)
23
+ ## Features (v0.3.0)
25
24
 
25
+ - 🖥️ **Web UI Dashboard** — Live queue stats, job browser, event log, and storage info
26
26
  - 📊 **Real-time Queue Status** — Monitor jobs across all states (ready, scheduled, claimed, failed)
27
27
  - 🔍 **Job Management CLI** — List, inspect, retry, and discard failed jobs
28
28
  - 💾 **Storage Monitoring** — Track database size and event counts
29
29
  - 🔗 **Distributed Tracing** — Correlate jobs with APM tools (Datadog, Sentry, OpenTelemetry)
30
30
  - ⚡ **High Performance** — Buffered writes, configurable sampling, minimal overhead
31
- - 🛡️ **Production Ready** — Automatic cleanup, size limits, retention policies
31
+ - 🛡️ **Production Ready** — Docker/CI/K8s safe boot, automatic cleanup, size limits, retention policies
32
+ - 🚀 **Two Operating Modes** — Real-time (no migrations) or persistence (full event history)
32
33
 
33
34
  ## Requirements
34
35
 
@@ -46,14 +47,31 @@ Add to your Gemfile:
46
47
  gem "solid_observer"
47
48
  ```
48
49
 
49
- Run the installer:
50
-
51
50
  ```bash
52
51
  bundle install
53
52
  rails generate solid_observer:install
54
53
  ```
55
54
 
56
- Install and run migrations:
55
+ SolidObserver supports two operating modes. Choose the one that fits your needs:
56
+
57
+ ### Real-time Mode (no migrations needed)
58
+
59
+ Get queue monitoring and job management instantly — no database setup required. SolidObserver queries Solid Queue directly.
60
+
61
+ ```ruby
62
+ # config/initializers/solid_observer.rb
63
+ SolidObserver.configure do |config|
64
+ config.storage_mode = :realtime
65
+ end
66
+ ```
67
+
68
+ That's it. You now have access to queue status, job listing, retry, and discard commands.
69
+
70
+ ### Persistence Mode (default)
71
+
72
+ Store event history, metrics, and storage snapshots in a dedicated observer database. This gives you everything in real-time mode plus long-term event tracking, buffered writes, and retention-based cleanup. The install generator defaults to SQLite; the database can use any Rails-supported adapter for record persistence, and storage-size monitoring is implemented for SQLite, PostgreSQL/PostGIS, MySQL, and Trilogy. See [Database Setup](#database-setup-persistence-mode) below.
73
+
74
+ > If your host app uses a different adapter than SQLite (e.g. PostgreSQL or MySQL), see [Multi-adapter setup](#multi-adapter-setup) before running these commands.
57
75
 
58
76
  ```bash
59
77
  bin/rails solid_observer:install:migrations
@@ -61,6 +79,8 @@ bin/rails db:create
61
79
  bin/rails db:migrate
62
80
  ```
63
81
 
82
+ For SQLite-default apps, no further configuration is needed — persistence is the default `storage_mode`.
83
+
64
84
  ## Quick Start
65
85
 
66
86
  ### Check Queue Status
@@ -116,7 +136,16 @@ bin/rails "solid_observer:jobs:retry[JOB_ID]"
116
136
  bin/rails "solid_observer:jobs:discard[JOB_ID]"
117
137
  ```
118
138
 
119
- ### Check Storage
139
+ ### Events Page Filters (Web UI)
140
+
141
+ Filter dropdowns on `/solid_observer/events` (job class and queue) are cached for 1 minute by default and scoped to the configured retention window.
142
+ Tune the cache TTL with `config.filter_cache_ttl`.
143
+
144
+ ### Check Storage (Persistence Mode)
145
+
146
+ Storage monitoring is cross-adapter: SQLite, PostgreSQL, MySQL, and Trilogy are supported.
147
+ SolidObserver uses adapter-native SQL queries to measure size (not filesystem `File.size`),
148
+ so production deployments report real values even when the database is remote.
120
149
 
121
150
  ```bash
122
151
  bin/rails solid_observer:storage
@@ -136,24 +165,99 @@ Configuration:
136
165
  Warning: 80% threshold
137
166
  ```
138
167
 
168
+ ## Web UI Dashboard
169
+
170
+ SolidObserver ships with a zero-dependency Web UI (no asset pipeline, no JS framework) at `/solid_observer`.
171
+
172
+ <table>
173
+ <tr>
174
+ <td align="center"><a href=".github/assets/dash_1.png"><img src=".github/assets/dash_1.png" alt="Dashboard overview" width="250"></a></td>
175
+ <td align="center"><a href=".github/assets/dash_2.png"><img src=".github/assets/dash_2.png" alt="Dashboard jobs and events" width="250"></a></td>
176
+ <td align="center"><a href=".github/assets/dash_3.png"><img src=".github/assets/dash_3.png" alt="Dashboard storage and details" width="250"></a></td>
177
+ </tr>
178
+ </table>
179
+
180
+ ### Mount
181
+
182
+ The install generator mounts it for you. To mount manually:
183
+
184
+ ```ruby
185
+ # config/routes.rb
186
+ mount SolidObserver::Engine, at: "/solid_observer"
187
+ ```
188
+
189
+ ### Configuration
190
+
191
+ ```ruby
192
+ SolidObserver.configure do |config|
193
+ config.ui_enabled = !Rails.env.production? # default: true outside production
194
+ config.ui_username = "admin" # HTTP Basic Auth: BOTH username AND password must be set
195
+ config.ui_password = ENV["SOLID_OBSERVER_PASSWORD"]
196
+ config.ui_base_controller = "ApplicationController" # name of your host app's base controller (used for API-only detection)
197
+ end
198
+ ```
199
+
200
+ | Option | Default | Purpose |
201
+ |---|---|---|
202
+ | `ui_enabled` | `!Rails.env.production?` | Master switch for the Web UI |
203
+ | `ui_username` / `ui_password` | `nil` | HTTP Basic Auth credentials. Auth is enabled only when **both** are set; if either is missing or `nil`, the UI is unauthenticated |
204
+ | `ui_base_controller` | `"ApplicationController"` | Name of your host app's base controller. SolidObserver does **not** inherit from it; the value is used to detect API-only apps so the engine can include the rendering modules its dashboard needs |
205
+
206
+ Live polling cadence is hardcoded at 5s.
207
+
208
+ ### Production hardening (recommended)
209
+
210
+ The snippet above silently disables auth if `ENV["SOLID_OBSERVER_PASSWORD"]` is unset (fail-open with a boot warning — see Caveats). For production, prefer one of these patterns so a missing env var fails loudly at boot rather than shipping an unauthenticated UI:
211
+
212
+ ```ruby
213
+ # Option A: fail at boot if the password env var is missing
214
+ SolidObserver.configure do |config|
215
+ config.ui_username = "admin"
216
+ config.ui_password = ENV.fetch("SOLID_OBSERVER_PASSWORD") # raises KeyError if unset
217
+ end
218
+
219
+ # Option B: only enable auth when the env var is present (no auth otherwise)
220
+ SolidObserver.configure do |config|
221
+ if (password = ENV["SOLID_OBSERVER_PASSWORD"]).present?
222
+ config.ui_username = "admin"
223
+ config.ui_password = password
224
+ end
225
+ end
226
+ ```
227
+
228
+ Option A is best when the UI must always be authenticated in production (a missing env var crashes boot — you find out immediately). Option B is best when the UI is optional in some environments.
229
+
230
+ ### Caveats
231
+
232
+ - **Realtime mode** (`storage_mode: :realtime`) — Events and Storage navigation links are hidden. Direct visits to `/solid_observer/events` or `/solid_observer/storage` redirect to the dashboard with a flash alert (`This page is not available in real-time mode`).
233
+ - **API-only apps** — the Web UI works in API-only Rails apps without manual configuration. SolidObserver's engine ships its own Cookies/Session/Flash middleware stack scoped to `/solid_observer/*` requests, so the dashboard renders, flash messages display, and retry/discard CSRF forms work even when the host app has `config.api_only = true`. The host app's middleware stack is not modified.
234
+ - **Custom API base controller** — if your host app's API base controller is **not** named `ApplicationController`, set `config.ui_base_controller` to its name (e.g. `"Api::BaseController"`). The engine uses this only to detect `ActionController::API` ancestry; if detected, it includes `ActionView::Layouts`, `ActionView::Rendering`, and `ActionController::RequestForgeryProtection` so layouts and CSRF forms render.
235
+ - **Host-app callbacks are not inherited.** SolidObserver does not run your host app's `before_action`s or authentication. Use `ui_username` / `ui_password` for the engine's built-in HTTP Basic Auth.
236
+ - **Auth misconfiguration is fail-open, but loud.** If you set `ui_username` but `ui_password` resolves to `nil` (e.g. an unset ENV var), the UI ships unauthenticated rather than locking everyone out — and the engine logs a `WARNING` at boot naming the missing credential. Verify both are set in production.
237
+
139
238
  ## Configuration
140
239
 
141
240
  After installation, configure SolidObserver in `config/initializers/solid_observer.rb`:
142
241
 
143
242
  ```ruby
144
243
  SolidObserver.configure do |config|
244
+ # Storage Mode (:persistence or :realtime)
245
+ # :persistence — stores events, metrics, snapshots (requires migrations)
246
+ # :realtime — live monitoring only, no database needed
247
+ config.storage_mode = :persistence # default
248
+
145
249
  # Enable queue monitoring (default: true)
146
250
  config.observe_queue = true
147
251
 
148
- # Data Retention
252
+ # Data Retention (persistence mode only)
149
253
  config.event_retention = 30.days # Keep events for 30 days
150
254
  config.metrics_retention = 90.days # Keep metrics for 90 days
151
255
 
152
- # Database Limits
256
+ # Database Limits (persistence mode only)
153
257
  config.max_db_size = 1.gigabyte # Maximum database size
154
258
  config.warning_threshold = 0.8 # Warn at 80% capacity
155
259
 
156
- # Performance Tuning
260
+ # Performance Tuning (persistence mode only)
157
261
  config.buffer_size = 1000 # Buffer before flushing to DB
158
262
  config.flush_interval = 10.seconds # Flush interval
159
263
  config.sampling_rate = 1.0 # 1.0 = capture all events
@@ -192,6 +296,8 @@ When configured, all job events will include your correlation ID, allowing you t
192
296
 
193
297
  ## CLI Reference
194
298
 
299
+ **Available in both modes (real-time and persistence):**
300
+
195
301
  | Command | Description |
196
302
  |---------|-------------|
197
303
  | `solid_observer:status` | Show queue status overview |
@@ -199,13 +305,18 @@ When configured, all job events will include your correlation ID, allowing you t
199
305
  | `solid_observer:jobs:show[ID]` | Show job details |
200
306
  | `solid_observer:jobs:retry[ID]` | Retry a failed job |
201
307
  | `solid_observer:jobs:discard[ID]` | Discard a failed job |
308
+
309
+ **Persistence mode only:**
310
+
311
+ | Command | Description |
312
+ |---------|-------------|
202
313
  | `solid_observer:storage` | Show storage statistics |
203
314
  | `solid_observer:buffer:flush` | Force flush event buffer to database |
204
315
  | `solid_observer:buffer:clear` | Clear buffer without saving |
205
316
  | `solid_observer:storage:cleanup` | Run retention-based cleanup |
206
317
  | `solid_observer:storage:purge` | Delete ALL SolidObserver data |
207
318
 
208
- > **Note:** These commands manage **SolidObserver's storage** (event logs, metrics, snapshots) - not Solid Queue's jobs. To manage jobs, use `jobs:discard` or `jobs:retry`.
319
+ > **Note:** Storage commands manage **SolidObserver's storage** (event logs, metrics, snapshots) not Solid Queue's jobs. To manage jobs, use `jobs:discard` or `jobs:retry`.
209
320
 
210
321
  ### Jobs List Arguments
211
322
 
@@ -226,7 +337,7 @@ bin/rails "solid_observer:jobs:list[ready,mailers]" # Ready jobs in mai
226
337
  bin/rails "solid_observer:jobs:list[failed,,,50]" # 50 failed jobs (skip queue/class)
227
338
  ```
228
339
 
229
- ### Buffer & Storage Management
340
+ ### Buffer & Storage Management (Persistence Mode)
230
341
 
231
342
  ```bash
232
343
  # Flush in-memory buffer to database
@@ -244,21 +355,60 @@ bin/rails solid_observer:storage:purge
244
355
 
245
356
  > **Important:** `storage:purge` deletes SolidObserver's monitoring data, NOT your Solid Queue jobs. Your queued jobs remain safe.
246
357
 
247
- ## Database Setup
358
+ ## Database Setup (Persistence Mode)
359
+
360
+ > **Tip:** If you're using real-time mode (`storage_mode: :realtime`), you can skip this section entirely — no database setup is needed.
248
361
 
249
362
  **SolidObserver works with any main application database** — PostgreSQL, MySQL, or SQLite.
250
363
 
251
- For its own monitoring data, SolidObserver uses a **separate SQLite database**. This keeps monitoring isolated from your main app and provides simple file-based storage that requires no additional infrastructure.
364
+ For its own monitoring data, the install generator defaults to a **separate SQLite database** file-based, no extra infrastructure needed. The example below is what `rails generate solid_observer:install` produces:
252
365
 
253
366
  ```yaml
254
- # config/database.yml
367
+ # config/database.yml (generator default)
255
368
  solid_observer_queue:
256
369
  <<: *default
257
- adapter: sqlite3 # Always SQLite for SolidObserver storage
370
+ adapter: sqlite3
258
371
  database: storage/<%= Rails.env %>_solid_observer_queue.sqlite3
259
372
  ```
260
373
 
261
- > **Note:** Your main app's `primary` database can be PostgreSQL, MySQL, or any Rails-supported adapter. Only the `solid_observer_queue` database needs to be SQLite.
374
+ > **Note:** SQLite is the generator default, not a requirement. The `solid_observer_queue` database can use any Rails-supported adapter for record persistence. Adapter-native **storage-size monitoring** is currently implemented for SQLite, PostgreSQL/PostGIS, MySQL, and Trilogy. On other adapters, the size query returns `nil` and the engine logs a single `[SolidObserver] Unknown adapter for DatabaseSize: …` warning — record persistence still works, but the size column on the dashboard will be empty until adapter support is added.
375
+
376
+ ### Multi-adapter setup
377
+
378
+ If your host application uses one database adapter (e.g. PostgreSQL) and you want SolidObserver to use a different adapter (e.g. SQLite — for isolation, simpler ops, or to avoid loading observability traffic onto your primary DB), keep the `solid_observer_queue` block **self-contained** — do not rely on `<<: *default`. The generator default (shown above) uses `<<: *default` with an explicit `adapter: sqlite3` override; that is safe on SQLite-primary hosts where the anchor is also SQLite. On a PostgreSQL host, merging `<<: *default` without an explicit adapter override pulls the PG adapter into the observer connection and fails at `db:create` with `PG::SyntaxError` (PG treating a SQLite file path as a database name). For multi-adapter connections, omit the merge entirely, as shown below.
379
+
380
+ ```yaml
381
+ # config/database.yml
382
+ default: &default
383
+ adapter: postgresql # host's adapter
384
+ encoding: unicode
385
+ pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
386
+
387
+ development:
388
+ primary:
389
+ <<: *default
390
+ database: my_app_development
391
+ # ... cache / queue / cable on PG ...
392
+
393
+ solid_observer_queue:
394
+ adapter: sqlite3 # explicit override — do NOT merge *default
395
+ pool: 5
396
+ timeout: 5000
397
+ database: storage/development_solid_observer_queue.sqlite3
398
+ migrations_paths: db/solid_observer_migrate
399
+ ```
400
+
401
+ Apply the same pattern to `test:` and `production:`. For production with SQLite, ensure the `storage/` path is on persistent disk (not ephemeral container storage) — otherwise prefer PostgreSQL for the observer DB too.
402
+
403
+ **Bundler note (PG-only hosts):** if your `Gemfile` does not already include `sqlite3`, add it:
404
+
405
+ ```ruby
406
+ gem "sqlite3", "~> 2.0"
407
+ ```
408
+
409
+ then `bundle install`. SolidObserver does not declare `sqlite3` as a runtime dependency — adapter choice is yours.
410
+
411
+ **`migrations_paths` is recommended** to isolate SolidObserver's migrations from the host's primary `db/migrate/` folder. This prevents Rails from running SolidObserver's migrations against your primary database (which would create an unused `solid_observer_queue_events` table there). When `migrations_paths` is set on `solid_observer_queue`, `bin/rails solid_observer:install:migrations` copies migrations to that path automatically — no manual `mv` required.
262
412
 
263
413
  ## Roadmap
264
414
 
@@ -266,12 +416,13 @@ SolidObserver is actively developed. Here's what's coming:
266
416
 
267
417
  | Version | Focus | Status |
268
418
  |---------|-------|--------|
269
- | v0.1.0 | Solid Queue monitoring, CLI tools | ✅ Current |
270
- | v0.2.0 | Solid Cache monitoring | 🔜 Planned |
271
- | v0.3.0 | Solid Cable monitoring | 🔜 Planned |
272
- | v0.4.0 | Cross-component correlation, health scores | 🔜 Planned |
273
- | v0.5.0 | Alerting & notifications | 🔜 Planned |
274
- | v0.6.0 | Web UI dashboard | 🔜 Planned |
419
+ | v0.1.0 | Solid Queue monitoring, CLI tools | ✅ Released |
420
+ | v0.1.1 | Real-time mode (no migrations needed) | Released |
421
+ | v0.3.0 | Web UI dashboard + stability hardening | Current |
422
+ | v0.4.0 | Solid Cache monitoring | 🔜 Planned |
423
+ | v0.5.0 | Solid Cable monitoring | 🔜 Planned |
424
+ | v0.6.0 | Cross-component correlation, health scores | 🔜 Planned |
425
+ | v0.7.0 | Alerting & notifications | 🔜 Planned |
275
426
  | v1.0.0 | Production stable release | 🎯 Goal |
276
427
 
277
428
  See [GitHub Milestones](https://github.com/bart-oz/solid_observer/milestones) for detailed plans.
@@ -298,6 +449,19 @@ bundle exec reek
298
449
 
299
450
  ## Troubleshooting
300
451
 
452
+ ### Docker, CI, and Offline Boot
453
+
454
+ SolidObserver is designed to boot without a live database connection. During
455
+ `rails assets:precompile`, CI runs without a database service, or Kubernetes init
456
+ containers, the engine logs a single info message and skips subscriber activation:
457
+
458
+ ```text
459
+ [SolidObserver] Database not reachable at boot. Skipping subscriber activation.
460
+ ```
461
+
462
+ No monkey-patching or environment-variable workarounds are needed. Once the application
463
+ boots with a live database, the engine activates normally on the next restart.
464
+
301
465
  ### "no such table: solid_queue_ready_executions"
302
466
 
303
467
  This error means Solid Queue isn't configured to use the correct database in your environment.
@@ -314,7 +478,9 @@ config.solid_queue.connects_to = { database: { writing: :queue } }
314
478
 
315
479
  ### Multi-database setup
316
480
 
317
- SolidObserver works with Rails multi-database configurations. Here's an example with PostgreSQL as your primary database:
481
+ > **See also:** [Multi-adapter setup](#multi-adapter-setup) in the Database Setup section for the full pattern (explicit `adapter:` override, `gem "sqlite3"` Bundler note, `migrations_paths` rationale).
482
+
483
+ SolidObserver works with Rails multi-database configurations. Quick-reference example with PostgreSQL as your primary database and SQLite for SolidObserver's storage:
318
484
 
319
485
  ```yaml
320
486
  development:
@@ -322,14 +488,10 @@ development:
322
488
  adapter: postgresql
323
489
  database: myapp_development
324
490
  # ... PostgreSQL settings
325
- queue:
326
- <<: *default
327
- adapter: sqlite3
328
- database: storage/development_queue.sqlite3
329
- migrations_paths: db/queue_migrate
330
491
  solid_observer_queue:
331
492
  adapter: sqlite3
332
493
  database: storage/development_solid_observer_queue.sqlite3
494
+ migrations_paths: db/solid_observer_migrate
333
495
  ```
334
496
 
335
497
  ## Contributing