solid_observer 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +73 -0
- data/README.md +198 -36
- data/app/assets/javascripts/solid_observer/live_poll.js +376 -0
- data/app/controllers/concerns/solid_observer/paginatable.rb +17 -0
- data/app/controllers/concerns/solid_observer/require_persistence_mode.rb +19 -0
- data/app/controllers/concerns/solid_observer/require_solid_queue.rb +19 -0
- data/app/controllers/solid_observer/application_controller.rb +69 -0
- data/app/controllers/solid_observer/dashboard_controller.rb +79 -0
- data/app/controllers/solid_observer/events_controller.rb +50 -0
- data/app/controllers/solid_observer/jobs_controller.rb +85 -0
- data/app/controllers/solid_observer/storages_controller.rb +12 -0
- data/app/helpers/solid_observer/application_helper.rb +95 -0
- data/app/helpers/solid_observer/dashboard_helper.rb +39 -0
- data/app/models/solid_observer/queue_event.rb +134 -0
- data/app/models/solid_observer/queue_metric.rb +1 -1
- data/app/presenters/solid_observer/execution_presenter.rb +50 -0
- data/app/views/layouts/solid_observer/application.html.erb +470 -0
- data/app/views/solid_observer/dashboard/_chart.html.erb +28 -0
- data/app/views/solid_observer/dashboard/_live_state.html.erb +20 -0
- data/app/views/solid_observer/dashboard/_queue_table.html.erb +34 -0
- data/app/views/solid_observer/dashboard/_right_now.html.erb +3 -0
- data/app/views/solid_observer/dashboard/_throughput.html.erb +32 -0
- data/app/views/solid_observer/dashboard/index.html.erb +113 -0
- data/app/views/solid_observer/errors/storage_unavailable.html.erb +27 -0
- data/app/views/solid_observer/events/index.html.erb +53 -0
- data/app/views/solid_observer/events/show.html.erb +47 -0
- data/app/views/solid_observer/jobs/index.html.erb +61 -0
- data/app/views/solid_observer/jobs/show.html.erb +71 -0
- data/app/views/solid_observer/shared/_empty_state.html.erb +5 -0
- data/app/views/solid_observer/shared/_pagination.html.erb +17 -0
- data/app/views/solid_observer/shared/_stat_card.html.erb +9 -0
- data/app/views/solid_observer/storages/show.html.erb +39 -0
- data/bin/quality_gate +95 -0
- data/config/routes.rb +17 -0
- data/db/migrate/20260424000001_add_composite_indexes_to_queue_events.rb +30 -0
- data/lib/generators/solid_observer/install_generator.rb +12 -25
- data/lib/generators/solid_observer/templates/initializer.rb.tt +5 -6
- data/lib/solid_observer/base_metric.rb +1 -1
- data/lib/solid_observer/chart_buffer.rb +83 -0
- data/lib/solid_observer/cli/base.rb +2 -2
- data/lib/solid_observer/cli/jobs.rb +2 -2
- data/lib/solid_observer/cli/status.rb +20 -2
- data/lib/solid_observer/cli/storage.rb +41 -32
- data/lib/solid_observer/configuration.rb +67 -34
- data/lib/solid_observer/correlation_id_resolver.rb +8 -6
- data/lib/solid_observer/engine.rb +75 -15
- data/lib/solid_observer/params/events_filter.rb +37 -0
- data/lib/solid_observer/params/jobs_filter.rb +35 -0
- data/lib/solid_observer/queries/events_query.rb +27 -0
- data/lib/solid_observer/queries/execution_finder.rb +42 -0
- data/lib/solid_observer/queries/job_executions_query.rb +73 -0
- data/lib/solid_observer/queue_event_buffer.rb +163 -22
- data/lib/solid_observer/queue_stats.rb +165 -19
- data/lib/solid_observer/services/cleanup_storage.rb +60 -42
- data/lib/solid_observer/services/database_size.rb +86 -0
- data/lib/solid_observer/services/flush_event_buffer.rb +31 -15
- data/lib/solid_observer/services/install_migrations.rb +49 -0
- data/lib/solid_observer/services/record_event.rb +53 -14
- data/lib/solid_observer/services/ui_auth_check.rb +65 -0
- data/lib/solid_observer/subscriber.rb +15 -8
- data/lib/solid_observer/version.rb +1 -1
- data/lib/solid_observer.rb +7 -0
- data/lib/tasks/solid_observer.rake +10 -2
- metadata +55 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: a745bb3571f70b207e14f2e88f6162db6b2322a889d42f5ebd2a50e80c5b8014
|
|
4
|
+
data.tar.gz: d05f835df0503f04f88223c0879145a2dab13279cd5bfc59346545c0e8263f1c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8417c8013a7c9ff03a9d52481af63889adaba093abf0c9b041842664af7fb15edde929f7c7b94c6df12cbac00ca41d2aeee0f488fd7ffe831abf9d5f873bff09
|
|
7
|
+
data.tar.gz: 92147e32675291ec743e055c55382bf719a9ab6dd10e2e507789d0163b703e06490ad97dbb785e0a9c0b51b8eb72eb7a849d95504ac52c8680e53a137b50e4e4
|
data/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,76 @@
|
|
|
1
|
+
## [0.3.0] - 2026-05-25
|
|
2
|
+
|
|
3
|
+
> Note: there is no separate `0.2.0` gem release — work originally scoped for v0.2.0 (stability + refactoring, SO-040 through SO-049) was folded into this release.
|
|
4
|
+
|
|
5
|
+
Headline: **Web UI Dashboard** + stability hardening from pre-release review.
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
- Web UI dashboard at `/solid_observer` — queue stats, jobs browser, events log, storage info, responsive layout, optional HTTP Basic Auth
|
|
9
|
+
- Web UI config: `ui_enabled`, `ui_username`, `ui_password`, `ui_base_controller`, `ui_refresh_interval`
|
|
10
|
+
- Dashboard "Live" toggle: when enabled, only the Right-Now card region (`so_right_now` Turbo Frame) reloads on the configured cadence (`SolidObserver.config.ui_refresh_interval`). Scoped cards stay frozen until the range selector changes. Toggle state lives in the `?live=on` URL param so it survives range navigations.
|
|
11
|
+
- Dashboard now supports a Time-Range selector (`15m`, `30m`, `1h`, `7h`, `1d`, `7d`, `14d`, default `1h`) that scopes the new "Last <range>" card region (Performed / Failed / Enqueue rate). The "Right Now" region (Ready / Scheduled / Claimed / Failed-awaiting-retry / Workers) is unaffected by the range and reflects current SolidQueue state.
|
|
12
|
+
- Retry / discard actions with confirmation dialogs and CSRF protection
|
|
13
|
+
- `QueueEventBuffer#metrics` and `#shutdown` (graceful drain on app exit)
|
|
14
|
+
- Configuration: `max_buffer_size` (default 10_000), `buffer_overflow_strategy` (`:drop_old` / `:drop_new`), `filter_cache_ttl`
|
|
15
|
+
- `Services::DatabaseSize` for cross-adapter table-size measurement; composite indexes and `distinct_job_classes` / `distinct_queue_names` scopes
|
|
16
|
+
- Dashboard now shows three throughput counters (Performed last hour, Failed last 24h, Enqueue rate last 5 min) sourced from the events table, alongside existing point-in-time counters. Throughput counters are persistence-mode only.
|
|
17
|
+
- Stat counter subtitles ("queued" / "future runs" / "in progress" / "awaiting retry" / "active processes") clarify SolidQueue lifecycle terminology.
|
|
18
|
+
- Conceptual hint banner on the dashboard explains the difference between Jobs tab (in-flight + failed) and Events tab (historical record).
|
|
19
|
+
- Jobs tab now has an empty state with a hint pointing to Events tab for completed-job history.
|
|
20
|
+
- Dashboard polled chart strip and unified card grid: a JSON-fed polling client refreshes six stat cards (Ready, Scheduled, Claimed, Workers, Failed, Enqueue Rate) and three sparklines (Performed/min, Ready depth, Failed/min) every `SolidObserver.config.ui_refresh_interval` seconds (default `30`, `0` disables polling). In realtime mode only the Ready sparkline renders and the Enqueue Rate card is omitted. Toggle in the dashboard top bar enables/disables polling; state lives in `?live=on` URL param. Hand-rolled inline-SVG sparklines, no JS framework, no external chart library.
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
- **Duration was displayed off by 1000x.** `RecordEvent` stored `ActiveSupport::Notifications::Event#duration` (milliseconds) directly; `format_duration` interpreted it as seconds. Fixed by converting ms → seconds at write time. No data migration needed (fixed before v0.3.0 release); run `bin/rails solid_observer:storage:purge` to clear pre-fix local data.
|
|
24
|
+
- **Dashboard chart strip renders populated polylines on first paint** instead of empty placeholders. Server-side `spark_points` helper mirrors the JS `Sparkline.render` projection formula so the first HTML response ships with real data; the first JS poll appends one additional segment with no visible "empty → full" transition.
|
|
25
|
+
- **Live toggle cadence label stays in sync with toggle state.** The `.so-toggle__cadence` span now carries `aria-live="polite"` and is updated in the same synchronous tick as the `--on` class toggle. Server-side ERB hardcodes `"5s"` / `"off"` matching the JS literals.
|
|
26
|
+
- Jobs details page no longer crashes for `SolidQueue::FailedExecution`; Queue and Priority now fall back to underlying `SolidQueue::Job` values and show `N/A` when unavailable.
|
|
27
|
+
- Web UI now works on API-only Rails hosts. The engine ships its own Cookies / Session::CookieStore (`key: "_solid_observer_session"`) / Flash middleware stack, so requests routed to `/solid_observer/*` get the middleware they need regardless of whether the host app strips them via `config.api_only = true`. Previously, API-only hosts hit `NoMethodError: undefined method 'flash' for an instance of ActionDispatch::Request` rendering the dashboard layout.
|
|
28
|
+
- `bin/rails solid_observer:install:migrations` now respects `migrations_paths` from the `solid_observer_queue` connection in `config/database.yml`. When the host configures a dedicated migration folder (e.g. `db/solid_observer_migrate`), the install task copies migrations directly there instead of `db/migrate/`. Previously, operators in multi-database setups had to manually move the files after install to prevent cross-database migration contamination.
|
|
29
|
+
- Web UI now degrades gracefully when the `solid_observer_queue` database is missing or unreachable at request time. Previously, requests to `/solid_observer/*` raised `ActiveRecord::NoDatabaseError` / `ConnectionNotEstablished` with a raw 500 stack trace. The engine now renders a 503 "Storage unavailable" page using the dashboard layout, with an actionable hint to run migrations or check `database.yml`. Realtime mode is unaffected. Boot-time resilience (Engine activation skip) was already in place; this closes the equivalent gap at request time.
|
|
30
|
+
- Engine boot no longer requires a live DB (Docker/CI/K8s safe); table check uses `BaseEvent.connection_pool` (multi-DB safe); broader rescue covering adapter-specific connection errors
|
|
31
|
+
- `QueueEventBuffer` hard-caps at `max_buffer_size` with overflow strategy; uses a single persistent `Concurrent::TimerTask` instead of spawning a thread per flush
|
|
32
|
+
- Storage monitoring uses adapter-native size queries (SQLite / PostgreSQL / MySQL / Trilogy) — fixes `0 MB` readings off SQLite
|
|
33
|
+
- `EventsController` and `JobsController` filter dropdowns no longer full-table-scan on every request (cached via `filter_cache_ttl`)
|
|
34
|
+
- `RecordEvent` correctly handles real `ActiveJob::Base` payload objects (with hash fallback)
|
|
35
|
+
|
|
36
|
+
### Security
|
|
37
|
+
- Job arguments removed from persisted event metadata and the jobs detail view (PII reduction)
|
|
38
|
+
- Web UI HTTP Basic Auth now requires **both** `ui_username` and `ui_password` to be configured. Previously, setting only `ui_username` (with `ui_password` missing or `nil`) would still trigger an auth challenge that any blank-password request would pass `secure_compare("", "")`, granting unauthenticated access. The README's "both must be set" guidance now matches the implementation; misconfigured auth now ships unauthenticated rather than allowing a bypass.
|
|
39
|
+
- Boot-time `Engine.check_ui_authentication` now warns on partial misconfiguration (exactly one of `ui_username` / `ui_password` set), naming the missing credential. Previously the check exited silently as soon as `ui_username.present?`, hiding the fail-open auth misconfiguration from operators.
|
|
40
|
+
|
|
41
|
+
### Documentation
|
|
42
|
+
- Documented multi-adapter installation (PG host + SQLite observer DB) inline in Database Setup, with explicit `adapter:` override, `gem "sqlite3"` Bundler note, `migrations_paths` migration-isolation guidance, and cross-reference from the install steps.
|
|
43
|
+
|
|
44
|
+
### Removed
|
|
45
|
+
- Removed the legacy implicit dashboard auto-refresh (`<meta http-equiv="refresh">` plus inline fetch/DOM-swap script for `.so-content`). Replaced by explicit, opt-in Live Mode targeting only the Right-Now frame.
|
|
46
|
+
- Removed the legacy `GET /solid_observer/right_now` HTML endpoint and its action template (it was the SO-060-era polling target that returned a partial-only HTML response wrapped in a turbo-frame). Replaced by `GET /solid_observer/poll_data` which returns JSON for the polling client. The `live_poll.js` script-delivery route is unchanged.
|
|
47
|
+
- **Removed `SolidObserver.config.ui_refresh_interval`** (was unreliable; cadence is now hardcoded at 5s). Upgrade note: remove the line from your initializer or boot will raise `NoMethodError`.
|
|
48
|
+
|
|
49
|
+
### Changed
|
|
50
|
+
- Web UI controllers refactored to thin actions + query/param/presenter objects; Rails built-in number helpers replace custom ones
|
|
51
|
+
- Specs: `allow_any_instance_of` → `instance_double`; sleep-based timer specs use deterministic synchronisation; dead private-method `describe` blocks removed
|
|
52
|
+
- `.reek.yml` suppressions trimmed; hot-path services/buffer/engine methods refactored to pass `TooManyStatements` without new suppressions
|
|
53
|
+
- `SolidObserver.config.ui_refresh_interval` now controls Live Mode polling cadence instead of implicit full-page dashboard refresh. Default value is unchanged.
|
|
54
|
+
- Dashboard default range is `15m` (was `1h`); aligns with poll default.
|
|
55
|
+
- Live polling pauses while the tab is hidden and resumes on return (one immediate tick on visibility return so the user doesn't wait up to 5s for fresh data).
|
|
56
|
+
- Jobs tab default filter changed from `status=ready` to `status=all_active` (ready + scheduled + claimed + failed).
|
|
57
|
+
- Duration values on the Events index and detail pages now use per-event-type semantic context via `<abbr title="...">` tooltips so operators can distinguish enqueue call latency (`job_enqueued`) from perform-time duration (`job_completed` / `job_failed` / `job_discarded`).
|
|
58
|
+
- Refreshed the engine UI to a minimalist visual language: light surfaces, near-black text, restrained semantic colour accents reserved for badges/state, hairline separators, consistent rounding. Sidebar moves from dark slate to a light surface. No external CSS dependencies, no JS, no dark mode (single light theme).
|
|
59
|
+
- Dashboard: replaced the multi-line "Recent Failures" panel with a single-line **Stability** indicator (pill badge + summary + "View failures" link). Three states based on rolling failure counts: **Stable** (no failures in last 24h), **Degraded** (failures in last 24h but none in last hour), **Critical** (any failure in the last hour). Click-through targets the Events page filtered to `job_failed`.
|
|
60
|
+
- Dashboard: removed the "Jobs tab / Events tab" orientation banner. The distinction is discoverable via navigation; the dashboard is reserved for signal.
|
|
61
|
+
- Dashboard layout consolidated: dropped the previous two-frame split (Right-Now + Scoped) and the redundant Performed-in-range / Failed-in-range cards (those metrics now live in the polled sparkline strip).
|
|
62
|
+
- Live toggle restyled as a pill switch with cadence visible in-line ("Live · 2s" / "Live · off") and a pulsing dot when active.
|
|
63
|
+
|
|
64
|
+
## [0.1.1] - 2026-02-10
|
|
65
|
+
|
|
66
|
+
### Added
|
|
67
|
+
- **Real-time mode** (`storage_mode: :realtime`) — run SolidObserver without database migrations
|
|
68
|
+
- Queue status and job management CLI commands work without any SolidObserver database
|
|
69
|
+
- `storage_mode` configuration option (`:persistence` default, `:realtime` for no-DB operation)
|
|
70
|
+
- `persistence_mode?` and `realtime_mode?` configuration predicates
|
|
71
|
+
- Graceful `CLI::Storage` message when in real-time mode
|
|
72
|
+
- Event buffering, metric incrementing, and cleanup automatically disabled in real-time mode
|
|
73
|
+
|
|
1
74
|
## [0.1.0] - 2026-02-02
|
|
2
75
|
|
|
3
76
|
### Added
|
data/README.md
CHANGED
|
@@ -2,33 +2,34 @@
|
|
|
2
2
|
<picture>
|
|
3
3
|
<source media="(prefers-color-scheme: dark)" srcset=".github/solid_logo_dark.svg">
|
|
4
4
|
<source media="(prefers-color-scheme: light)" srcset=".github/solid_logo_light.svg">
|
|
5
|
-
<img alt="SolidObserver" src=".github/solid_logo_light.svg" width="
|
|
5
|
+
<img alt="SolidObserver" src=".github/solid_logo_light.svg" width="220">
|
|
6
6
|
</picture>
|
|
7
7
|
</p>
|
|
8
8
|
|
|
9
9
|
<p align="center">
|
|
10
|
-
<
|
|
11
|
-
</p>
|
|
12
|
-
|
|
13
|
-
<p align="center">
|
|
14
|
-
<a href="https://github.com/bart-oz/solid_observer/releases"><img src="https://img.shields.io/badge/version-0.1.0-blue.svg" alt="Version"></a>
|
|
10
|
+
<a href="https://github.com/bart-oz/solid_observer/releases"><img src="https://img.shields.io/badge/version-0.3.0-blue.svg" alt="Version"></a>
|
|
15
11
|
<a href="LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
|
|
16
12
|
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/tests-passing-brightgreen.svg" alt="Tests"></a>
|
|
17
|
-
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/coverage-
|
|
13
|
+
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/coverage-96.98%25-brightgreen.svg" alt="Coverage"></a>
|
|
18
14
|
</p>
|
|
19
15
|
|
|
20
16
|
---
|
|
17
|
+
<p align="center">
|
|
18
|
+
<img src=".github/assets/dash_1.png" alt="Dashboard overview" width="700">
|
|
19
|
+
</p>
|
|
21
20
|
|
|
22
|
-
SolidObserver is a production-grade observability solution for Rails 8's Solid Stack. Starting with **Solid Queue** monitoring in v0.
|
|
21
|
+
SolidObserver is a production-grade observability solution for Rails 8's Solid Stack. Starting with **Solid Queue** monitoring in v0.3.0, it provides unified visibility into your background job processing with a Web UI dashboard, CLI tools, metrics collection, and distributed tracing support.
|
|
23
22
|
|
|
24
|
-
## Features (v0.
|
|
23
|
+
## Features (v0.3.0)
|
|
25
24
|
|
|
25
|
+
- 🖥️ **Web UI Dashboard** — Live queue stats, job browser, event log, and storage info
|
|
26
26
|
- 📊 **Real-time Queue Status** — Monitor jobs across all states (ready, scheduled, claimed, failed)
|
|
27
27
|
- 🔍 **Job Management CLI** — List, inspect, retry, and discard failed jobs
|
|
28
28
|
- 💾 **Storage Monitoring** — Track database size and event counts
|
|
29
29
|
- 🔗 **Distributed Tracing** — Correlate jobs with APM tools (Datadog, Sentry, OpenTelemetry)
|
|
30
30
|
- ⚡ **High Performance** — Buffered writes, configurable sampling, minimal overhead
|
|
31
|
-
- 🛡️ **Production Ready** —
|
|
31
|
+
- 🛡️ **Production Ready** — Docker/CI/K8s safe boot, automatic cleanup, size limits, retention policies
|
|
32
|
+
- 🚀 **Two Operating Modes** — Real-time (no migrations) or persistence (full event history)
|
|
32
33
|
|
|
33
34
|
## Requirements
|
|
34
35
|
|
|
@@ -46,14 +47,31 @@ Add to your Gemfile:
|
|
|
46
47
|
gem "solid_observer"
|
|
47
48
|
```
|
|
48
49
|
|
|
49
|
-
Run the installer:
|
|
50
|
-
|
|
51
50
|
```bash
|
|
52
51
|
bundle install
|
|
53
52
|
rails generate solid_observer:install
|
|
54
53
|
```
|
|
55
54
|
|
|
56
|
-
|
|
55
|
+
SolidObserver supports two operating modes. Choose the one that fits your needs:
|
|
56
|
+
|
|
57
|
+
### Real-time Mode (no migrations needed)
|
|
58
|
+
|
|
59
|
+
Get queue monitoring and job management instantly — no database setup required. SolidObserver queries Solid Queue directly.
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
# config/initializers/solid_observer.rb
|
|
63
|
+
SolidObserver.configure do |config|
|
|
64
|
+
config.storage_mode = :realtime
|
|
65
|
+
end
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
That's it. You now have access to queue status, job listing, retry, and discard commands.
|
|
69
|
+
|
|
70
|
+
### Persistence Mode (default)
|
|
71
|
+
|
|
72
|
+
Store event history, metrics, and storage snapshots in a dedicated observer database. This gives you everything in real-time mode plus long-term event tracking, buffered writes, and retention-based cleanup. The install generator defaults to SQLite; the database can use any Rails-supported adapter for record persistence, and storage-size monitoring is implemented for SQLite, PostgreSQL/PostGIS, MySQL, and Trilogy. See [Database Setup](#database-setup-persistence-mode) below.
|
|
73
|
+
|
|
74
|
+
> If your host app uses a different adapter than SQLite (e.g. PostgreSQL or MySQL), see [Multi-adapter setup](#multi-adapter-setup) before running these commands.
|
|
57
75
|
|
|
58
76
|
```bash
|
|
59
77
|
bin/rails solid_observer:install:migrations
|
|
@@ -61,6 +79,8 @@ bin/rails db:create
|
|
|
61
79
|
bin/rails db:migrate
|
|
62
80
|
```
|
|
63
81
|
|
|
82
|
+
For SQLite-default apps, no further configuration is needed — persistence is the default `storage_mode`.
|
|
83
|
+
|
|
64
84
|
## Quick Start
|
|
65
85
|
|
|
66
86
|
### Check Queue Status
|
|
@@ -116,7 +136,16 @@ bin/rails "solid_observer:jobs:retry[JOB_ID]"
|
|
|
116
136
|
bin/rails "solid_observer:jobs:discard[JOB_ID]"
|
|
117
137
|
```
|
|
118
138
|
|
|
119
|
-
###
|
|
139
|
+
### Events Page Filters (Web UI)
|
|
140
|
+
|
|
141
|
+
Filter dropdowns on `/solid_observer/events` (job class and queue) are cached for 1 minute by default and scoped to the configured retention window.
|
|
142
|
+
Tune the cache TTL with `config.filter_cache_ttl`.
|
|
143
|
+
|
|
144
|
+
### Check Storage (Persistence Mode)
|
|
145
|
+
|
|
146
|
+
Storage monitoring is cross-adapter: SQLite, PostgreSQL, MySQL, and Trilogy are supported.
|
|
147
|
+
SolidObserver uses adapter-native SQL queries to measure size (not filesystem `File.size`),
|
|
148
|
+
so production deployments report real values even when the database is remote.
|
|
120
149
|
|
|
121
150
|
```bash
|
|
122
151
|
bin/rails solid_observer:storage
|
|
@@ -136,24 +165,99 @@ Configuration:
|
|
|
136
165
|
Warning: 80% threshold
|
|
137
166
|
```
|
|
138
167
|
|
|
168
|
+
## Web UI Dashboard
|
|
169
|
+
|
|
170
|
+
SolidObserver ships with a zero-dependency Web UI (no asset pipeline, no JS framework) at `/solid_observer`.
|
|
171
|
+
|
|
172
|
+
<table>
|
|
173
|
+
<tr>
|
|
174
|
+
<td align="center"><a href=".github/assets/dash_1.png"><img src=".github/assets/dash_1.png" alt="Dashboard overview" width="250"></a></td>
|
|
175
|
+
<td align="center"><a href=".github/assets/dash_2.png"><img src=".github/assets/dash_2.png" alt="Dashboard jobs and events" width="250"></a></td>
|
|
176
|
+
<td align="center"><a href=".github/assets/dash_3.png"><img src=".github/assets/dash_3.png" alt="Dashboard storage and details" width="250"></a></td>
|
|
177
|
+
</tr>
|
|
178
|
+
</table>
|
|
179
|
+
|
|
180
|
+
### Mount
|
|
181
|
+
|
|
182
|
+
The install generator mounts it for you. To mount manually:
|
|
183
|
+
|
|
184
|
+
```ruby
|
|
185
|
+
# config/routes.rb
|
|
186
|
+
mount SolidObserver::Engine, at: "/solid_observer"
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
### Configuration
|
|
190
|
+
|
|
191
|
+
```ruby
|
|
192
|
+
SolidObserver.configure do |config|
|
|
193
|
+
config.ui_enabled = !Rails.env.production? # default: true outside production
|
|
194
|
+
config.ui_username = "admin" # HTTP Basic Auth: BOTH username AND password must be set
|
|
195
|
+
config.ui_password = ENV["SOLID_OBSERVER_PASSWORD"]
|
|
196
|
+
config.ui_base_controller = "ApplicationController" # name of your host app's base controller (used for API-only detection)
|
|
197
|
+
end
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
| Option | Default | Purpose |
|
|
201
|
+
|---|---|---|
|
|
202
|
+
| `ui_enabled` | `!Rails.env.production?` | Master switch for the Web UI |
|
|
203
|
+
| `ui_username` / `ui_password` | `nil` | HTTP Basic Auth credentials. Auth is enabled only when **both** are set; if either is missing or `nil`, the UI is unauthenticated |
|
|
204
|
+
| `ui_base_controller` | `"ApplicationController"` | Name of your host app's base controller. SolidObserver does **not** inherit from it; the value is used to detect API-only apps so the engine can include the rendering modules its dashboard needs |
|
|
205
|
+
|
|
206
|
+
Live polling cadence is hardcoded at 5s.
|
|
207
|
+
|
|
208
|
+
### Production hardening (recommended)
|
|
209
|
+
|
|
210
|
+
The snippet above silently disables auth if `ENV["SOLID_OBSERVER_PASSWORD"]` is unset (fail-open with a boot warning — see Caveats). For production, prefer one of these patterns so a missing env var fails loudly at boot rather than shipping an unauthenticated UI:
|
|
211
|
+
|
|
212
|
+
```ruby
|
|
213
|
+
# Option A: fail at boot if the password env var is missing
|
|
214
|
+
SolidObserver.configure do |config|
|
|
215
|
+
config.ui_username = "admin"
|
|
216
|
+
config.ui_password = ENV.fetch("SOLID_OBSERVER_PASSWORD") # raises KeyError if unset
|
|
217
|
+
end
|
|
218
|
+
|
|
219
|
+
# Option B: only enable auth when the env var is present (no auth otherwise)
|
|
220
|
+
SolidObserver.configure do |config|
|
|
221
|
+
if (password = ENV["SOLID_OBSERVER_PASSWORD"]).present?
|
|
222
|
+
config.ui_username = "admin"
|
|
223
|
+
config.ui_password = password
|
|
224
|
+
end
|
|
225
|
+
end
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
Option A is best when the UI must always be authenticated in production (a missing env var crashes boot — you find out immediately). Option B is best when the UI is optional in some environments.
|
|
229
|
+
|
|
230
|
+
### Caveats
|
|
231
|
+
|
|
232
|
+
- **Realtime mode** (`storage_mode: :realtime`) — Events and Storage navigation links are hidden. Direct visits to `/solid_observer/events` or `/solid_observer/storage` redirect to the dashboard with a flash alert (`This page is not available in real-time mode`).
|
|
233
|
+
- **API-only apps** — the Web UI works in API-only Rails apps without manual configuration. SolidObserver's engine ships its own Cookies/Session/Flash middleware stack scoped to `/solid_observer/*` requests, so the dashboard renders, flash messages display, and retry/discard CSRF forms work even when the host app has `config.api_only = true`. The host app's middleware stack is not modified.
|
|
234
|
+
- **Custom API base controller** — if your host app's API base controller is **not** named `ApplicationController`, set `config.ui_base_controller` to its name (e.g. `"Api::BaseController"`). The engine uses this only to detect `ActionController::API` ancestry; if detected, it includes `ActionView::Layouts`, `ActionView::Rendering`, and `ActionController::RequestForgeryProtection` so layouts and CSRF forms render.
|
|
235
|
+
- **Host-app callbacks are not inherited.** SolidObserver does not run your host app's `before_action`s or authentication. Use `ui_username` / `ui_password` for the engine's built-in HTTP Basic Auth.
|
|
236
|
+
- **Auth misconfiguration is fail-open, but loud.** If you set `ui_username` but `ui_password` resolves to `nil` (e.g. an unset ENV var), the UI ships unauthenticated rather than locking everyone out — and the engine logs a `WARNING` at boot naming the missing credential. Verify both are set in production.
|
|
237
|
+
|
|
139
238
|
## Configuration
|
|
140
239
|
|
|
141
240
|
After installation, configure SolidObserver in `config/initializers/solid_observer.rb`:
|
|
142
241
|
|
|
143
242
|
```ruby
|
|
144
243
|
SolidObserver.configure do |config|
|
|
244
|
+
# Storage Mode (:persistence or :realtime)
|
|
245
|
+
# :persistence — stores events, metrics, snapshots (requires migrations)
|
|
246
|
+
# :realtime — live monitoring only, no database needed
|
|
247
|
+
config.storage_mode = :persistence # default
|
|
248
|
+
|
|
145
249
|
# Enable queue monitoring (default: true)
|
|
146
250
|
config.observe_queue = true
|
|
147
251
|
|
|
148
|
-
# Data Retention
|
|
252
|
+
# Data Retention (persistence mode only)
|
|
149
253
|
config.event_retention = 30.days # Keep events for 30 days
|
|
150
254
|
config.metrics_retention = 90.days # Keep metrics for 90 days
|
|
151
255
|
|
|
152
|
-
# Database Limits
|
|
256
|
+
# Database Limits (persistence mode only)
|
|
153
257
|
config.max_db_size = 1.gigabyte # Maximum database size
|
|
154
258
|
config.warning_threshold = 0.8 # Warn at 80% capacity
|
|
155
259
|
|
|
156
|
-
# Performance Tuning
|
|
260
|
+
# Performance Tuning (persistence mode only)
|
|
157
261
|
config.buffer_size = 1000 # Buffer before flushing to DB
|
|
158
262
|
config.flush_interval = 10.seconds # Flush interval
|
|
159
263
|
config.sampling_rate = 1.0 # 1.0 = capture all events
|
|
@@ -192,6 +296,8 @@ When configured, all job events will include your correlation ID, allowing you t
|
|
|
192
296
|
|
|
193
297
|
## CLI Reference
|
|
194
298
|
|
|
299
|
+
**Available in both modes (real-time and persistence):**
|
|
300
|
+
|
|
195
301
|
| Command | Description |
|
|
196
302
|
|---------|-------------|
|
|
197
303
|
| `solid_observer:status` | Show queue status overview |
|
|
@@ -199,13 +305,18 @@ When configured, all job events will include your correlation ID, allowing you t
|
|
|
199
305
|
| `solid_observer:jobs:show[ID]` | Show job details |
|
|
200
306
|
| `solid_observer:jobs:retry[ID]` | Retry a failed job |
|
|
201
307
|
| `solid_observer:jobs:discard[ID]` | Discard a failed job |
|
|
308
|
+
|
|
309
|
+
**Persistence mode only:**
|
|
310
|
+
|
|
311
|
+
| Command | Description |
|
|
312
|
+
|---------|-------------|
|
|
202
313
|
| `solid_observer:storage` | Show storage statistics |
|
|
203
314
|
| `solid_observer:buffer:flush` | Force flush event buffer to database |
|
|
204
315
|
| `solid_observer:buffer:clear` | Clear buffer without saving |
|
|
205
316
|
| `solid_observer:storage:cleanup` | Run retention-based cleanup |
|
|
206
317
|
| `solid_observer:storage:purge` | Delete ALL SolidObserver data |
|
|
207
318
|
|
|
208
|
-
> **Note:**
|
|
319
|
+
> **Note:** Storage commands manage **SolidObserver's storage** (event logs, metrics, snapshots) — not Solid Queue's jobs. To manage jobs, use `jobs:discard` or `jobs:retry`.
|
|
209
320
|
|
|
210
321
|
### Jobs List Arguments
|
|
211
322
|
|
|
@@ -226,7 +337,7 @@ bin/rails "solid_observer:jobs:list[ready,mailers]" # Ready jobs in mai
|
|
|
226
337
|
bin/rails "solid_observer:jobs:list[failed,,,50]" # 50 failed jobs (skip queue/class)
|
|
227
338
|
```
|
|
228
339
|
|
|
229
|
-
### Buffer & Storage Management
|
|
340
|
+
### Buffer & Storage Management (Persistence Mode)
|
|
230
341
|
|
|
231
342
|
```bash
|
|
232
343
|
# Flush in-memory buffer to database
|
|
@@ -244,21 +355,60 @@ bin/rails solid_observer:storage:purge
|
|
|
244
355
|
|
|
245
356
|
> **Important:** `storage:purge` deletes SolidObserver's monitoring data, NOT your Solid Queue jobs. Your queued jobs remain safe.
|
|
246
357
|
|
|
247
|
-
## Database Setup
|
|
358
|
+
## Database Setup (Persistence Mode)
|
|
359
|
+
|
|
360
|
+
> **Tip:** If you're using real-time mode (`storage_mode: :realtime`), you can skip this section entirely — no database setup is needed.
|
|
248
361
|
|
|
249
362
|
**SolidObserver works with any main application database** — PostgreSQL, MySQL, or SQLite.
|
|
250
363
|
|
|
251
|
-
For its own monitoring data,
|
|
364
|
+
For its own monitoring data, the install generator defaults to a **separate SQLite database** — file-based, no extra infrastructure needed. The example below is what `rails generate solid_observer:install` produces:
|
|
252
365
|
|
|
253
366
|
```yaml
|
|
254
|
-
# config/database.yml
|
|
367
|
+
# config/database.yml (generator default)
|
|
255
368
|
solid_observer_queue:
|
|
256
369
|
<<: *default
|
|
257
|
-
adapter: sqlite3
|
|
370
|
+
adapter: sqlite3
|
|
258
371
|
database: storage/<%= Rails.env %>_solid_observer_queue.sqlite3
|
|
259
372
|
```
|
|
260
373
|
|
|
261
|
-
> **Note:**
|
|
374
|
+
> **Note:** SQLite is the generator default, not a requirement. The `solid_observer_queue` database can use any Rails-supported adapter for record persistence. Adapter-native **storage-size monitoring** is currently implemented for SQLite, PostgreSQL/PostGIS, MySQL, and Trilogy. On other adapters, the size query returns `nil` and the engine logs a single `[SolidObserver] Unknown adapter for DatabaseSize: …` warning — record persistence still works, but the size column on the dashboard will be empty until adapter support is added.
|
|
375
|
+
|
|
376
|
+
### Multi-adapter setup
|
|
377
|
+
|
|
378
|
+
If your host application uses one database adapter (e.g. PostgreSQL) and you want SolidObserver to use a different adapter (e.g. SQLite — for isolation, simpler ops, or to avoid loading observability traffic onto your primary DB), keep the `solid_observer_queue` block **self-contained** — do not rely on `<<: *default`. The generator default (shown above) uses `<<: *default` with an explicit `adapter: sqlite3` override; that is safe on SQLite-primary hosts where the anchor is also SQLite. On a PostgreSQL host, merging `<<: *default` without an explicit adapter override pulls the PG adapter into the observer connection and fails at `db:create` with `PG::SyntaxError` (PG treating a SQLite file path as a database name). For multi-adapter connections, omit the merge entirely, as shown below.
|
|
379
|
+
|
|
380
|
+
```yaml
|
|
381
|
+
# config/database.yml
|
|
382
|
+
default: &default
|
|
383
|
+
adapter: postgresql # host's adapter
|
|
384
|
+
encoding: unicode
|
|
385
|
+
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
|
|
386
|
+
|
|
387
|
+
development:
|
|
388
|
+
primary:
|
|
389
|
+
<<: *default
|
|
390
|
+
database: my_app_development
|
|
391
|
+
# ... cache / queue / cable on PG ...
|
|
392
|
+
|
|
393
|
+
solid_observer_queue:
|
|
394
|
+
adapter: sqlite3 # explicit override — do NOT merge *default
|
|
395
|
+
pool: 5
|
|
396
|
+
timeout: 5000
|
|
397
|
+
database: storage/development_solid_observer_queue.sqlite3
|
|
398
|
+
migrations_paths: db/solid_observer_migrate
|
|
399
|
+
```
|
|
400
|
+
|
|
401
|
+
Apply the same pattern to `test:` and `production:`. For production with SQLite, ensure the `storage/` path is on persistent disk (not ephemeral container storage) — otherwise prefer PostgreSQL for the observer DB too.
|
|
402
|
+
|
|
403
|
+
**Bundler note (PG-only hosts):** if your `Gemfile` does not already include `sqlite3`, add it:
|
|
404
|
+
|
|
405
|
+
```ruby
|
|
406
|
+
gem "sqlite3", "~> 2.0"
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
then `bundle install`. SolidObserver does not declare `sqlite3` as a runtime dependency — adapter choice is yours.
|
|
410
|
+
|
|
411
|
+
**`migrations_paths` is recommended** to isolate SolidObserver's migrations from the host's primary `db/migrate/` folder. This prevents Rails from running SolidObserver's migrations against your primary database (which would create an unused `solid_observer_queue_events` table there). When `migrations_paths` is set on `solid_observer_queue`, `bin/rails solid_observer:install:migrations` copies migrations to that path automatically — no manual `mv` required.
|
|
262
412
|
|
|
263
413
|
## Roadmap
|
|
264
414
|
|
|
@@ -266,12 +416,13 @@ SolidObserver is actively developed. Here's what's coming:
|
|
|
266
416
|
|
|
267
417
|
| Version | Focus | Status |
|
|
268
418
|
|---------|-------|--------|
|
|
269
|
-
| v0.1.0 | Solid Queue monitoring, CLI tools | ✅
|
|
270
|
-
| v0.
|
|
271
|
-
| v0.3.0 |
|
|
272
|
-
| v0.4.0 |
|
|
273
|
-
| v0.5.0 |
|
|
274
|
-
| v0.6.0 |
|
|
419
|
+
| v0.1.0 | Solid Queue monitoring, CLI tools | ✅ Released |
|
|
420
|
+
| v0.1.1 | Real-time mode (no migrations needed) | ✅ Released |
|
|
421
|
+
| v0.3.0 | Web UI dashboard + stability hardening | ✅ Current |
|
|
422
|
+
| v0.4.0 | Solid Cache monitoring | 🔜 Planned |
|
|
423
|
+
| v0.5.0 | Solid Cable monitoring | 🔜 Planned |
|
|
424
|
+
| v0.6.0 | Cross-component correlation, health scores | 🔜 Planned |
|
|
425
|
+
| v0.7.0 | Alerting & notifications | 🔜 Planned |
|
|
275
426
|
| v1.0.0 | Production stable release | 🎯 Goal |
|
|
276
427
|
|
|
277
428
|
See [GitHub Milestones](https://github.com/bart-oz/solid_observer/milestones) for detailed plans.
|
|
@@ -298,6 +449,19 @@ bundle exec reek
|
|
|
298
449
|
|
|
299
450
|
## Troubleshooting
|
|
300
451
|
|
|
452
|
+
### Docker, CI, and Offline Boot
|
|
453
|
+
|
|
454
|
+
SolidObserver is designed to boot without a live database connection. During
|
|
455
|
+
`rails assets:precompile`, CI runs without a database service, or Kubernetes init
|
|
456
|
+
containers, the engine logs a single info message and skips subscriber activation:
|
|
457
|
+
|
|
458
|
+
```text
|
|
459
|
+
[SolidObserver] Database not reachable at boot. Skipping subscriber activation.
|
|
460
|
+
```
|
|
461
|
+
|
|
462
|
+
No monkey-patching or environment-variable workarounds are needed. Once the application
|
|
463
|
+
boots with a live database, the engine activates normally on the next restart.
|
|
464
|
+
|
|
301
465
|
### "no such table: solid_queue_ready_executions"
|
|
302
466
|
|
|
303
467
|
This error means Solid Queue isn't configured to use the correct database in your environment.
|
|
@@ -314,7 +478,9 @@ config.solid_queue.connects_to = { database: { writing: :queue } }
|
|
|
314
478
|
|
|
315
479
|
### Multi-database setup
|
|
316
480
|
|
|
317
|
-
|
|
481
|
+
> **See also:** [Multi-adapter setup](#multi-adapter-setup) in the Database Setup section for the full pattern (explicit `adapter:` override, `gem "sqlite3"` Bundler note, `migrations_paths` rationale).
|
|
482
|
+
|
|
483
|
+
SolidObserver works with Rails multi-database configurations. Quick-reference example with PostgreSQL as your primary database and SQLite for SolidObserver's storage:
|
|
318
484
|
|
|
319
485
|
```yaml
|
|
320
486
|
development:
|
|
@@ -322,14 +488,10 @@ development:
|
|
|
322
488
|
adapter: postgresql
|
|
323
489
|
database: myapp_development
|
|
324
490
|
# ... PostgreSQL settings
|
|
325
|
-
queue:
|
|
326
|
-
<<: *default
|
|
327
|
-
adapter: sqlite3
|
|
328
|
-
database: storage/development_queue.sqlite3
|
|
329
|
-
migrations_paths: db/queue_migrate
|
|
330
491
|
solid_observer_queue:
|
|
331
492
|
adapter: sqlite3
|
|
332
493
|
database: storage/development_solid_observer_queue.sqlite3
|
|
494
|
+
migrations_paths: db/solid_observer_migrate
|
|
333
495
|
```
|
|
334
496
|
|
|
335
497
|
## Contributing
|