solid_observer 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +10 -0
- data/README.md +52 -19
- data/lib/solid_observer/cli/storage.rb +8 -0
- data/lib/solid_observer/configuration.rb +23 -0
- data/lib/solid_observer/engine.rb +7 -2
- data/lib/solid_observer/queue_event_buffer.rb +4 -1
- data/lib/solid_observer/services/cleanup_storage.rb +2 -0
- data/lib/solid_observer/services/record_event.rb +2 -0
- data/lib/solid_observer/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c2e83e38f795537bcacb9a431fe39543476b615cb205904fd5af324a4a1886e8
|
|
4
|
+
data.tar.gz: cf501ff1b33535eb7a2b4f4858289bea3c93af7371af83bb867255b3f00de462
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 58fb350eda47268f65f13b71ac4f317fe7a9cd6274a05c636a4d7a91477ad5a6269a652a7bb537255cbce8e690231c6bd33507530baf6fb626e672b738bf175b
|
|
7
|
+
data.tar.gz: 9042759cdeca68bf9fd50a0c62493c6e9d057915b714cb8d68ef81f36028e6437c8f85dfeab8bac971419bc8471b147840051aa881a01871f720640f4abdfdb9
|
data/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,13 @@
|
|
|
1
|
+
## [0.1.1] - 2026-02-10
|
|
2
|
+
|
|
3
|
+
### Added
|
|
4
|
+
- **Real-time mode** (`storage_mode: :realtime`) — run SolidObserver without database migrations
|
|
5
|
+
- Queue status and job management CLI commands work without any SolidObserver database
|
|
6
|
+
- `storage_mode` configuration option (`:persistence` default, `:realtime` for no-DB operation)
|
|
7
|
+
- `persistence_mode?` and `realtime_mode?` configuration predicates
|
|
8
|
+
- Graceful `CLI::Storage` message when in real-time mode
|
|
9
|
+
- Event buffering, metric incrementing, and cleanup automatically disabled in real-time mode
|
|
10
|
+
|
|
1
11
|
## [0.1.0] - 2026-02-02
|
|
2
12
|
|
|
3
13
|
### Added
|
data/README.md
CHANGED
|
@@ -11,17 +11,17 @@
|
|
|
11
11
|
</p>
|
|
12
12
|
|
|
13
13
|
<p align="center">
|
|
14
|
-
<a href="https://github.com/bart-oz/solid_observer/releases"><img src="https://img.shields.io/badge/version-0.1.
|
|
14
|
+
<a href="https://github.com/bart-oz/solid_observer/releases"><img src="https://img.shields.io/badge/version-0.1.1-blue.svg" alt="Version"></a>
|
|
15
15
|
<a href="LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
|
|
16
16
|
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/tests-passing-brightgreen.svg" alt="Tests"></a>
|
|
17
|
-
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/coverage-93.
|
|
17
|
+
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/coverage-93.06%25-brightgreen.svg" alt="Coverage"></a>
|
|
18
18
|
</p>
|
|
19
19
|
|
|
20
20
|
---
|
|
21
21
|
|
|
22
22
|
SolidObserver is a production-grade observability solution for Rails 8's Solid Stack. Starting with **Solid Queue** monitoring in v0.1.0, it provides unified visibility into your background job processing with CLI tools, metrics collection, and distributed tracing support.
|
|
23
23
|
|
|
24
|
-
## Features (v0.1.
|
|
24
|
+
## Features (v0.1.1)
|
|
25
25
|
|
|
26
26
|
- 📊 **Real-time Queue Status** — Monitor jobs across all states (ready, scheduled, claimed, failed)
|
|
27
27
|
- 🔍 **Job Management CLI** — List, inspect, retry, and discard failed jobs
|
|
@@ -29,6 +29,7 @@ SolidObserver is a production-grade observability solution for Rails 8's Solid S
|
|
|
29
29
|
- 🔗 **Distributed Tracing** — Correlate jobs with APM tools (Datadog, Sentry, OpenTelemetry)
|
|
30
30
|
- ⚡ **High Performance** — Buffered writes, configurable sampling, minimal overhead
|
|
31
31
|
- 🛡️ **Production Ready** — Automatic cleanup, size limits, retention policies
|
|
32
|
+
- 🚀 **Two Operating Modes** — Real-time (no migrations) or persistence (full event history)
|
|
32
33
|
|
|
33
34
|
## Requirements
|
|
34
35
|
|
|
@@ -46,14 +47,29 @@ Add to your Gemfile:
|
|
|
46
47
|
gem "solid_observer"
|
|
47
48
|
```
|
|
48
49
|
|
|
49
|
-
Run the installer:
|
|
50
|
-
|
|
51
50
|
```bash
|
|
52
51
|
bundle install
|
|
53
52
|
rails generate solid_observer:install
|
|
54
53
|
```
|
|
55
54
|
|
|
56
|
-
|
|
55
|
+
SolidObserver supports two operating modes. Choose the one that fits your needs:
|
|
56
|
+
|
|
57
|
+
### Real-time Mode (no migrations needed)
|
|
58
|
+
|
|
59
|
+
Get queue monitoring and job management instantly — no database setup required. SolidObserver queries Solid Queue directly.
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
# config/initializers/solid_observer.rb
|
|
63
|
+
SolidObserver.configure do |config|
|
|
64
|
+
config.storage_mode = :realtime
|
|
65
|
+
end
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
That's it. You now have access to queue status, job listing, retry, and discard commands.
|
|
69
|
+
|
|
70
|
+
### Persistence Mode (default)
|
|
71
|
+
|
|
72
|
+
Store event history, metrics, and storage snapshots in a dedicated SQLite database. This gives you everything in real-time mode plus long-term event tracking, buffered writes, and retention-based cleanup.
|
|
57
73
|
|
|
58
74
|
```bash
|
|
59
75
|
bin/rails solid_observer:install:migrations
|
|
@@ -61,6 +77,8 @@ bin/rails db:create
|
|
|
61
77
|
bin/rails db:migrate
|
|
62
78
|
```
|
|
63
79
|
|
|
80
|
+
No additional configuration needed — persistence is the default `storage_mode`.
|
|
81
|
+
|
|
64
82
|
## Quick Start
|
|
65
83
|
|
|
66
84
|
### Check Queue Status
|
|
@@ -116,7 +134,7 @@ bin/rails "solid_observer:jobs:retry[JOB_ID]"
|
|
|
116
134
|
bin/rails "solid_observer:jobs:discard[JOB_ID]"
|
|
117
135
|
```
|
|
118
136
|
|
|
119
|
-
### Check Storage
|
|
137
|
+
### Check Storage (Persistence Mode)
|
|
120
138
|
|
|
121
139
|
```bash
|
|
122
140
|
bin/rails solid_observer:storage
|
|
@@ -142,18 +160,23 @@ After installation, configure SolidObserver in `config/initializers/solid_observ
|
|
|
142
160
|
|
|
143
161
|
```ruby
|
|
144
162
|
SolidObserver.configure do |config|
|
|
163
|
+
# Storage Mode (:persistence or :realtime)
|
|
164
|
+
# :persistence — stores events, metrics, snapshots (requires migrations)
|
|
165
|
+
# :realtime — live monitoring only, no database needed
|
|
166
|
+
config.storage_mode = :persistence # default
|
|
167
|
+
|
|
145
168
|
# Enable queue monitoring (default: true)
|
|
146
169
|
config.observe_queue = true
|
|
147
170
|
|
|
148
|
-
# Data Retention
|
|
171
|
+
# Data Retention (persistence mode only)
|
|
149
172
|
config.event_retention = 30.days # Keep events for 30 days
|
|
150
173
|
config.metrics_retention = 90.days # Keep metrics for 90 days
|
|
151
174
|
|
|
152
|
-
# Database Limits
|
|
175
|
+
# Database Limits (persistence mode only)
|
|
153
176
|
config.max_db_size = 1.gigabyte # Maximum database size
|
|
154
177
|
config.warning_threshold = 0.8 # Warn at 80% capacity
|
|
155
178
|
|
|
156
|
-
# Performance Tuning
|
|
179
|
+
# Performance Tuning (persistence mode only)
|
|
157
180
|
config.buffer_size = 1000 # Buffer before flushing to DB
|
|
158
181
|
config.flush_interval = 10.seconds # Flush interval
|
|
159
182
|
config.sampling_rate = 1.0 # 1.0 = capture all events
|
|
@@ -192,6 +215,8 @@ When configured, all job events will include your correlation ID, allowing you t
|
|
|
192
215
|
|
|
193
216
|
## CLI Reference
|
|
194
217
|
|
|
218
|
+
**Available in both modes (real-time and persistence):**
|
|
219
|
+
|
|
195
220
|
| Command | Description |
|
|
196
221
|
|---------|-------------|
|
|
197
222
|
| `solid_observer:status` | Show queue status overview |
|
|
@@ -199,13 +224,18 @@ When configured, all job events will include your correlation ID, allowing you t
|
|
|
199
224
|
| `solid_observer:jobs:show[ID]` | Show job details |
|
|
200
225
|
| `solid_observer:jobs:retry[ID]` | Retry a failed job |
|
|
201
226
|
| `solid_observer:jobs:discard[ID]` | Discard a failed job |
|
|
227
|
+
|
|
228
|
+
**Persistence mode only:**
|
|
229
|
+
|
|
230
|
+
| Command | Description |
|
|
231
|
+
|---------|-------------|
|
|
202
232
|
| `solid_observer:storage` | Show storage statistics |
|
|
203
233
|
| `solid_observer:buffer:flush` | Force flush event buffer to database |
|
|
204
234
|
| `solid_observer:buffer:clear` | Clear buffer without saving |
|
|
205
235
|
| `solid_observer:storage:cleanup` | Run retention-based cleanup |
|
|
206
236
|
| `solid_observer:storage:purge` | Delete ALL SolidObserver data |
|
|
207
237
|
|
|
208
|
-
> **Note:**
|
|
238
|
+
> **Note:** Storage commands manage **SolidObserver's storage** (event logs, metrics, snapshots) — not Solid Queue's jobs. To manage jobs, use `jobs:discard` or `jobs:retry`.
|
|
209
239
|
|
|
210
240
|
### Jobs List Arguments
|
|
211
241
|
|
|
@@ -226,7 +256,7 @@ bin/rails "solid_observer:jobs:list[ready,mailers]" # Ready jobs in mai
|
|
|
226
256
|
bin/rails "solid_observer:jobs:list[failed,,,50]" # 50 failed jobs (skip queue/class)
|
|
227
257
|
```
|
|
228
258
|
|
|
229
|
-
### Buffer & Storage Management
|
|
259
|
+
### Buffer & Storage Management (Persistence Mode)
|
|
230
260
|
|
|
231
261
|
```bash
|
|
232
262
|
# Flush in-memory buffer to database
|
|
@@ -244,7 +274,9 @@ bin/rails solid_observer:storage:purge
|
|
|
244
274
|
|
|
245
275
|
> **Important:** `storage:purge` deletes SolidObserver's monitoring data, NOT your Solid Queue jobs. Your queued jobs remain safe.
|
|
246
276
|
|
|
247
|
-
## Database Setup
|
|
277
|
+
## Database Setup (Persistence Mode)
|
|
278
|
+
|
|
279
|
+
> **Tip:** If you're using real-time mode (`storage_mode: :realtime`), you can skip this section entirely — no database setup is needed.
|
|
248
280
|
|
|
249
281
|
**SolidObserver works with any main application database** — PostgreSQL, MySQL, or SQLite.
|
|
250
282
|
|
|
@@ -266,12 +298,13 @@ SolidObserver is actively developed. Here's what's coming:
|
|
|
266
298
|
|
|
267
299
|
| Version | Focus | Status |
|
|
268
300
|
|---------|-------|--------|
|
|
269
|
-
| v0.1.0 | Solid Queue monitoring, CLI tools | ✅
|
|
270
|
-
| v0.
|
|
271
|
-
| v0.
|
|
272
|
-
| v0.
|
|
273
|
-
| v0.
|
|
274
|
-
| v0.
|
|
301
|
+
| v0.1.0 | Solid Queue monitoring, CLI tools | ✅ Released |
|
|
302
|
+
| v0.1.1 | Real-time mode (no migrations needed) | ✅ Current |
|
|
303
|
+
| v0.2.0 | Web UI dashboard (vanilla HTML/CSS) | 🔜 Planned |
|
|
304
|
+
| v0.3.0 | Solid Cache monitoring | 🔜 Planned |
|
|
305
|
+
| v0.4.0 | Solid Cable monitoring | 🔜 Planned |
|
|
306
|
+
| v0.5.0 | Cross-component correlation, health scores | 🔜 Planned |
|
|
307
|
+
| v0.6.0 | Alerting & notifications | 🔜 Planned |
|
|
275
308
|
| v1.0.0 | Production stable release | 🎯 Goal |
|
|
276
309
|
|
|
277
310
|
See [GitHub Milestones](https://github.com/bart-oz/solid_observer/milestones) for detailed plans.
|
|
@@ -4,6 +4,14 @@ module SolidObserver
|
|
|
4
4
|
module CLI
|
|
5
5
|
class Storage < Base
|
|
6
6
|
def call
|
|
7
|
+
if SolidObserver.config.realtime_mode?
|
|
8
|
+
print_section_header("💾 Storage Status")
|
|
9
|
+
info("Storage monitoring is not available in real-time mode.")
|
|
10
|
+
info("Switch to persistence mode for event history and storage tracking.")
|
|
11
|
+
output("")
|
|
12
|
+
return
|
|
13
|
+
end
|
|
14
|
+
|
|
7
15
|
print_section_header("💾 Storage Status")
|
|
8
16
|
|
|
9
17
|
current_stats = gather_storage_stats
|
|
@@ -38,6 +38,9 @@ module SolidObserver
|
|
|
38
38
|
# Storage Settings
|
|
39
39
|
attr_accessor :max_db_size
|
|
40
40
|
|
|
41
|
+
# Storage Mode
|
|
42
|
+
attr_reader :storage_mode
|
|
43
|
+
|
|
41
44
|
# Performance Settings (with validation)
|
|
42
45
|
attr_reader :sampling_rate,
|
|
43
46
|
:warning_threshold,
|
|
@@ -55,6 +58,9 @@ module SolidObserver
|
|
|
55
58
|
@http_basic_auth_user = nil
|
|
56
59
|
@http_basic_auth_password = nil
|
|
57
60
|
|
|
61
|
+
# Storage mode
|
|
62
|
+
@storage_mode = :persistence
|
|
63
|
+
|
|
58
64
|
# Observer defaults
|
|
59
65
|
@observe_queue = true
|
|
60
66
|
@observe_cache = false
|
|
@@ -78,6 +84,23 @@ module SolidObserver
|
|
|
78
84
|
@correlation_id_generator = nil
|
|
79
85
|
end
|
|
80
86
|
|
|
87
|
+
STORAGE_MODES = %i[persistence realtime].freeze
|
|
88
|
+
|
|
89
|
+
def storage_mode=(value)
|
|
90
|
+
value = value.to_sym
|
|
91
|
+
raise ArgumentError, "storage_mode must be :persistence or :realtime" unless STORAGE_MODES.include?(value)
|
|
92
|
+
|
|
93
|
+
@storage_mode = value
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
def persistence_mode?
|
|
97
|
+
@storage_mode == :persistence
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
def realtime_mode?
|
|
101
|
+
@storage_mode == :realtime
|
|
102
|
+
end
|
|
103
|
+
|
|
81
104
|
def sampling_rate=(value)
|
|
82
105
|
validate_rate!(:sampling_rate, value)
|
|
83
106
|
@sampling_rate = value
|
|
@@ -12,6 +12,8 @@ module SolidObserver
|
|
|
12
12
|
end
|
|
13
13
|
|
|
14
14
|
def configure_database_connection
|
|
15
|
+
return if SolidObserver.config.realtime_mode?
|
|
16
|
+
|
|
15
17
|
db_config = ActiveRecord::Base.configurations.configs_for(
|
|
16
18
|
env_name: Rails.env,
|
|
17
19
|
name: "solid_observer_queue"
|
|
@@ -30,12 +32,15 @@ module SolidObserver
|
|
|
30
32
|
def activate_subscribers
|
|
31
33
|
logger = Rails.logger
|
|
32
34
|
|
|
33
|
-
|
|
35
|
+
if SolidObserver.config.realtime_mode?
|
|
36
|
+
logger.info "[SolidObserver] Starting in real-time mode (no persistence)"
|
|
37
|
+
elsif !table_exists?("solid_observer_queue_events")
|
|
34
38
|
logger.info "[SolidObserver] Tables not found. Run: rails solid_observer:install:migrations && rails db:migrate"
|
|
35
39
|
return
|
|
40
|
+
else
|
|
41
|
+
logger.info "[SolidObserver] Activating event subscribers"
|
|
36
42
|
end
|
|
37
43
|
|
|
38
|
-
logger.info "[SolidObserver] Activating event subscribers"
|
|
39
44
|
Subscriber.subscribe!
|
|
40
45
|
rescue ActiveRecord::NoDatabaseError
|
|
41
46
|
logger.info "[SolidObserver] Database not ready yet. Skipping subscriber activation."
|
|
@@ -25,12 +25,15 @@ module SolidObserver
|
|
|
25
25
|
# @param event_data [Hash] Event data to buffer
|
|
26
26
|
# @return [void]
|
|
27
27
|
def push(event_data)
|
|
28
|
+
config = SolidObserver.config
|
|
29
|
+
return unless config.persistence_mode?
|
|
30
|
+
|
|
28
31
|
should_flush = false
|
|
29
32
|
|
|
30
33
|
@mutex.synchronize do
|
|
31
34
|
@buffer << event_data
|
|
32
35
|
schedule_flush unless @flush_scheduled
|
|
33
|
-
should_flush = @buffer.size >=
|
|
36
|
+
should_flush = @buffer.size >= config.buffer_size
|
|
34
37
|
end
|
|
35
38
|
|
|
36
39
|
flush! if should_flush
|