solid_observer 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +58 -0
- data/LICENSE.txt +21 -0
- data/README.md +347 -0
- data/app/jobs/solid_observer/cleanup_job.rb +12 -0
- data/app/models/solid_observer/queue_event.rb +23 -0
- data/app/models/solid_observer/queue_metric.rb +14 -0
- data/app/models/solid_observer/storage_info.rb +36 -0
- data/bin/console +11 -0
- data/bin/setup +8 -0
- data/db/migrate/20260115000001_create_solid_observer_queue_events.rb +21 -0
- data/db/migrate/20260115000002_create_solid_observer_metrics.rb +16 -0
- data/db/migrate/20260115000003_create_solid_observer_storage_info.rb +13 -0
- data/lib/generators/solid_observer/install_generator.rb +72 -0
- data/lib/generators/solid_observer/templates/initializer.rb.tt +57 -0
- data/lib/solid_observer/base_event.rb +10 -0
- data/lib/solid_observer/base_metric.rb +59 -0
- data/lib/solid_observer/cli/base.rb +98 -0
- data/lib/solid_observer/cli/jobs.rb +195 -0
- data/lib/solid_observer/cli/status.rb +59 -0
- data/lib/solid_observer/cli/storage.rb +114 -0
- data/lib/solid_observer/configuration.rb +125 -0
- data/lib/solid_observer/correlation_id_resolver.rb +62 -0
- data/lib/solid_observer/engine.rb +60 -0
- data/lib/solid_observer/queue_event_buffer.rb +80 -0
- data/lib/solid_observer/queue_stats.rb +89 -0
- data/lib/solid_observer/services/cleanup_storage.rb +94 -0
- data/lib/solid_observer/services/flush_event_buffer.rb +65 -0
- data/lib/solid_observer/services/record_event.rb +96 -0
- data/lib/solid_observer/subscriber.rb +96 -0
- data/lib/solid_observer/version.rb +7 -0
- data/lib/solid_observer.rb +40 -0
- data/lib/tasks/solid_observer.rake +155 -0
- metadata +93 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 67f33210e5da882874417d97b040e5c1ea8a30b4cf5f628645fa76405bd38379
|
|
4
|
+
data.tar.gz: ffa7843af6bdd14bc23762da2f42a245073bf4776ec6b2de1aece77852c41d9d
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 3b8ca565611c5a343b55131fa63d84ff5ad8c47a954b9103914764d90a1d381b6dcd39923c71b9c467011c2f6f0bc17cb724a8b8a60e16eef32110142088b007
|
|
7
|
+
data.tar.gz: 73e295dd624824b396bcd0ccca2c4d740f3f27287ce226b7816a92569193b996527e6cd5c8e0967c67a18b1b9c19fe100163ee49d5a5d3ebcb6e5b18624fa9bd
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
## [0.1.0] - 2026-02-02
|
|
2
|
+
|
|
3
|
+
### Added
|
|
4
|
+
|
|
5
|
+
#### Core Infrastructure
|
|
6
|
+
- Configuration DSL with 17 configurable attributes
|
|
7
|
+
- Rails Engine setup for seamless integration
|
|
8
|
+
- Separate database support for observability data
|
|
9
|
+
- Module-level `configure`, `config`, and `reset_configuration!` methods
|
|
10
|
+
- **QueueEvent validations** with `EVENT_TYPES`
|
|
11
|
+
- **Configuration validation**
|
|
12
|
+
- **Thread error handling** in `QueueEventBuffer#schedule_flush`
|
|
13
|
+
|
|
14
|
+
#### Queue Monitoring
|
|
15
|
+
- Real-time Solid Queue status monitoring (ready, scheduled, claimed, failed jobs)
|
|
16
|
+
- Queue depth tracking per queue name
|
|
17
|
+
- Worker count monitoring
|
|
18
|
+
- Event collection with buffered writes for performance
|
|
19
|
+
- **CleanupJob** inherits from `ActiveJob::Base` for better engine isolation
|
|
20
|
+
- **Database maintenance** commands are adapter-aware (SQLite VACUUM, PostgreSQL VACUUM ANALYZE, MySQL OPTIMIZE TABLE)
|
|
21
|
+
- **Subscriber** includes idempotency check to prevent duplicate subscriptions
|
|
22
|
+
- **RecordEvent** properly populates `job_class` and `queue_name` columns
|
|
23
|
+
|
|
24
|
+
#### CLI Tools
|
|
25
|
+
- `solid_observer:status` — Queue overview with formatted tables
|
|
26
|
+
- `solid_observer:jobs:list` — List jobs with filters (status, queue, job_class, limit)
|
|
27
|
+
- `solid_observer:jobs:show` — Detailed job inspection
|
|
28
|
+
- `solid_observer:jobs:retry` — Retry failed jobs with confirmation
|
|
29
|
+
- `solid_observer:jobs:discard` — Discard failed jobs with confirmation
|
|
30
|
+
- `solid_observer:storage` — Storage statistics and configuration
|
|
31
|
+
|
|
32
|
+
#### Database & Migrations
|
|
33
|
+
- Queue events migration with indexes for efficient queries
|
|
34
|
+
- Metrics migration for aggregated data
|
|
35
|
+
- Storage info migration for tracking database health
|
|
36
|
+
- Automatic cleanup job for data retention
|
|
37
|
+
|
|
38
|
+
#### APM Integration
|
|
39
|
+
- `correlation_id_generator` option for distributed tracing
|
|
40
|
+
- Built-in support for Datadog, Sentry, and OpenTelemetry
|
|
41
|
+
- Custom correlation ID generator support
|
|
42
|
+
|
|
43
|
+
#### Performance
|
|
44
|
+
- Buffered event writes (configurable buffer size and flush interval)
|
|
45
|
+
- Sampling rate configuration for high-traffic applications
|
|
46
|
+
- Insert batching with `insert_all!` for bulk operations
|
|
47
|
+
|
|
48
|
+
#### Developer Experience
|
|
49
|
+
- Rails generator: `rails generate solid_observer:install`
|
|
50
|
+
- Production-aware defaults (UI disabled in production)
|
|
51
|
+
- Rails idiomatic time helpers (30.days, 10.seconds)
|
|
52
|
+
- Comprehensive test suite (250+ tests)
|
|
53
|
+
|
|
54
|
+
#### Quality & CI
|
|
55
|
+
- Multi-version testing (Ruby 3.2-4.0, Rails 8.0-8.1)
|
|
56
|
+
- Code quality: StandardRB, Reek, SimpleCov (95%+ coverage)
|
|
57
|
+
- Security scanning: bundler-audit, brakeman
|
|
58
|
+
- Performance benchmarks
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 BartOz
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,347 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<picture>
|
|
3
|
+
<source media="(prefers-color-scheme: dark)" srcset=".github/solid_logo_dark.svg">
|
|
4
|
+
<source media="(prefers-color-scheme: light)" srcset=".github/solid_logo_light.svg">
|
|
5
|
+
<img alt="SolidObserver" src=".github/solid_logo_light.svg" width="250">
|
|
6
|
+
</picture>
|
|
7
|
+
</p>
|
|
8
|
+
|
|
9
|
+
<p align="center">
|
|
10
|
+
<strong>Observe your Solid Stack like a pro!</strong>
|
|
11
|
+
</p>
|
|
12
|
+
|
|
13
|
+
<p align="center">
|
|
14
|
+
<a href="https://github.com/bart-oz/solid_observer/releases"><img src="https://img.shields.io/badge/version-0.1.0-blue.svg" alt="Version"></a>
|
|
15
|
+
<a href="LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
|
|
16
|
+
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/tests-passing-brightgreen.svg" alt="Tests"></a>
|
|
17
|
+
<a href="https://github.com/bart-oz/solid_observer/actions"><img src="https://img.shields.io/badge/coverage-93.23%25-brightgreen.svg" alt="Coverage"></a>
|
|
18
|
+
</p>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
SolidObserver is a production-grade observability solution for Rails 8's Solid Stack. Starting with **Solid Queue** monitoring in v0.1.0, it provides unified visibility into your background job processing with CLI tools, metrics collection, and distributed tracing support.
|
|
23
|
+
|
|
24
|
+
## Features (v0.1.0)
|
|
25
|
+
|
|
26
|
+
- 📊 **Real-time Queue Status** — Monitor jobs across all states (ready, scheduled, claimed, failed)
|
|
27
|
+
- 🔍 **Job Management CLI** — List, inspect, retry, and discard failed jobs
|
|
28
|
+
- 💾 **Storage Monitoring** — Track database size and event counts
|
|
29
|
+
- 🔗 **Distributed Tracing** — Correlate jobs with APM tools (Datadog, Sentry, OpenTelemetry)
|
|
30
|
+
- ⚡ **High Performance** — Buffered writes, configurable sampling, minimal overhead
|
|
31
|
+
- 🛡️ **Production Ready** — Automatic cleanup, size limits, retention policies
|
|
32
|
+
|
|
33
|
+
## Requirements
|
|
34
|
+
|
|
35
|
+
- Ruby 3.2+
|
|
36
|
+
- Rails 8.0+
|
|
37
|
+
- Solid Queue (properly configured for all environments)
|
|
38
|
+
|
|
39
|
+
> **Note:** Ensure Solid Queue is configured with `connects_to` in all environments, not just production. See [Troubleshooting](#troubleshooting) if you encounter database connection issues.
|
|
40
|
+
|
|
41
|
+
## Installation
|
|
42
|
+
|
|
43
|
+
Add to your Gemfile:
|
|
44
|
+
|
|
45
|
+
```ruby
|
|
46
|
+
gem "solid_observer"
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Run the installer:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
bundle install
|
|
53
|
+
rails generate solid_observer:install
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Install and run migrations:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
bin/rails solid_observer:install:migrations
|
|
60
|
+
bin/rails db:create
|
|
61
|
+
bin/rails db:migrate
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Quick Start
|
|
65
|
+
|
|
66
|
+
### Check Queue Status
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
bin/rails solid_observer:status
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Output:
|
|
73
|
+
```
|
|
74
|
+
📊 SolidObserver Status
|
|
75
|
+
==================================================
|
|
76
|
+
|
|
77
|
+
🚀 Solid Queue
|
|
78
|
+
|
|
79
|
+
| Metric | Value |
|
|
80
|
+
|-----------|-------|
|
|
81
|
+
| Ready | 42 |
|
|
82
|
+
| Scheduled | 15 |
|
|
83
|
+
| Claimed | 3 |
|
|
84
|
+
| Failed | 2 |
|
|
85
|
+
| Workers | 4 |
|
|
86
|
+
|
|
87
|
+
📋 Queue Depths
|
|
88
|
+
|
|
89
|
+
| Queue | Jobs |
|
|
90
|
+
|------------|------|
|
|
91
|
+
| default | 38 |
|
|
92
|
+
| mailers | 12 |
|
|
93
|
+
| critical | 10 |
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
### Manage Jobs
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
# List jobs (defaults to ready jobs)
|
|
100
|
+
bin/rails solid_observer:jobs:list
|
|
101
|
+
|
|
102
|
+
# List failed jobs
|
|
103
|
+
bin/rails "solid_observer:jobs:list[failed]"
|
|
104
|
+
|
|
105
|
+
# Filter by status, queue, job class, and limit
|
|
106
|
+
bin/rails "solid_observer:jobs:list[failed,mailers]"
|
|
107
|
+
bin/rails "solid_observer:jobs:list[ready,default,UserNotificationJob,50]"
|
|
108
|
+
|
|
109
|
+
# Inspect a specific job
|
|
110
|
+
bin/rails "solid_observer:jobs:show[JOB_ID]"
|
|
111
|
+
|
|
112
|
+
# Retry a failed job
|
|
113
|
+
bin/rails "solid_observer:jobs:retry[JOB_ID]"
|
|
114
|
+
|
|
115
|
+
# Discard a failed job
|
|
116
|
+
bin/rails "solid_observer:jobs:discard[JOB_ID]"
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
### Check Storage
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
bin/rails solid_observer:storage
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Output:
|
|
126
|
+
```
|
|
127
|
+
💾 Storage Status
|
|
128
|
+
|
|
129
|
+
| Component | Size | Events | Usage | Status |
|
|
130
|
+
|-----------|---------|--------|-------|--------|
|
|
131
|
+
| Queue | 12.5 MB | 45,231 | 1.2% | ✓ |
|
|
132
|
+
|
|
133
|
+
Configuration:
|
|
134
|
+
Retention: 30 days
|
|
135
|
+
Max size: 1024.0 MB per database
|
|
136
|
+
Warning: 80% threshold
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
## Configuration
|
|
140
|
+
|
|
141
|
+
After installation, configure SolidObserver in `config/initializers/solid_observer.rb`:
|
|
142
|
+
|
|
143
|
+
```ruby
|
|
144
|
+
SolidObserver.configure do |config|
|
|
145
|
+
# Enable queue monitoring (default: true)
|
|
146
|
+
config.observe_queue = true
|
|
147
|
+
|
|
148
|
+
# Data Retention
|
|
149
|
+
config.event_retention = 30.days # Keep events for 30 days
|
|
150
|
+
config.metrics_retention = 90.days # Keep metrics for 90 days
|
|
151
|
+
|
|
152
|
+
# Database Limits
|
|
153
|
+
config.max_db_size = 1.gigabyte # Maximum database size
|
|
154
|
+
config.warning_threshold = 0.8 # Warn at 80% capacity
|
|
155
|
+
|
|
156
|
+
# Performance Tuning
|
|
157
|
+
config.buffer_size = 1000 # Buffer before flushing to DB
|
|
158
|
+
config.flush_interval = 10.seconds # Flush interval
|
|
159
|
+
config.sampling_rate = 1.0 # 1.0 = capture all events
|
|
160
|
+
end
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
### APM Integration
|
|
164
|
+
|
|
165
|
+
Connect SolidObserver with your Application Performance Monitoring tool for distributed tracing:
|
|
166
|
+
|
|
167
|
+
```ruby
|
|
168
|
+
SolidObserver.configure do |config|
|
|
169
|
+
# Datadog APM
|
|
170
|
+
config.correlation_id_generator = -> {
|
|
171
|
+
Datadog::Tracing.active_trace&.id
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
# Sentry
|
|
175
|
+
config.correlation_id_generator = -> {
|
|
176
|
+
Sentry.get_current_scope&.transaction&.trace_id
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
# OpenTelemetry
|
|
180
|
+
config.correlation_id_generator = -> {
|
|
181
|
+
OpenTelemetry::Trace.current_span&.context&.trace_id
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
# Custom implementation
|
|
185
|
+
config.correlation_id_generator = -> {
|
|
186
|
+
Thread.current[:request_id] || SecureRandom.uuid
|
|
187
|
+
}
|
|
188
|
+
end
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
When configured, all job events will include your correlation ID, allowing you to trace jobs back to the originating request.
|
|
192
|
+
|
|
193
|
+
## CLI Reference
|
|
194
|
+
|
|
195
|
+
| Command | Description |
|
|
196
|
+
|---------|-------------|
|
|
197
|
+
| `solid_observer:status` | Show queue status overview |
|
|
198
|
+
| `solid_observer:jobs:list[status,queue,class,limit]` | List jobs with optional filters |
|
|
199
|
+
| `solid_observer:jobs:show[ID]` | Show job details |
|
|
200
|
+
| `solid_observer:jobs:retry[ID]` | Retry a failed job |
|
|
201
|
+
| `solid_observer:jobs:discard[ID]` | Discard a failed job |
|
|
202
|
+
| `solid_observer:storage` | Show storage statistics |
|
|
203
|
+
| `solid_observer:buffer:flush` | Force flush event buffer to database |
|
|
204
|
+
| `solid_observer:buffer:clear` | Clear buffer without saving |
|
|
205
|
+
| `solid_observer:storage:cleanup` | Run retention-based cleanup |
|
|
206
|
+
| `solid_observer:storage:purge` | Delete ALL SolidObserver data |
|
|
207
|
+
|
|
208
|
+
> **Note:** These commands manage **SolidObserver's storage** (event logs, metrics, snapshots) - not Solid Queue's jobs. To manage jobs, use `jobs:discard` or `jobs:retry`.
|
|
209
|
+
|
|
210
|
+
### Jobs List Arguments
|
|
211
|
+
|
|
212
|
+
Arguments are positional: `[status, queue, job_class, limit]`
|
|
213
|
+
|
|
214
|
+
| Position | Description | Example |
|
|
215
|
+
|----------|-------------|---------|
|
|
216
|
+
| 1st | Filter by status | `failed`, `ready`, `scheduled` |
|
|
217
|
+
| 2nd | Filter by queue name | `default`, `mailers` |
|
|
218
|
+
| 3rd | Filter by job class | `UserNotificationJob` |
|
|
219
|
+
| 4th | Max results (default: 20) | `50` |
|
|
220
|
+
|
|
221
|
+
```bash
|
|
222
|
+
# Examples
|
|
223
|
+
bin/rails solid_observer:jobs:list # All ready jobs
|
|
224
|
+
bin/rails "solid_observer:jobs:list[failed]" # Failed jobs
|
|
225
|
+
bin/rails "solid_observer:jobs:list[ready,mailers]" # Ready jobs in mailers queue
|
|
226
|
+
bin/rails "solid_observer:jobs:list[failed,,,50]" # 50 failed jobs (skip queue/class)
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
### Buffer & Storage Management
|
|
230
|
+
|
|
231
|
+
```bash
|
|
232
|
+
# Flush in-memory buffer to database
|
|
233
|
+
bin/rails solid_observer:buffer:flush
|
|
234
|
+
|
|
235
|
+
# Clear buffer without saving (loses pending events!)
|
|
236
|
+
bin/rails solid_observer:buffer:clear
|
|
237
|
+
|
|
238
|
+
# Run cleanup based on retention policy (default: 30 days)
|
|
239
|
+
bin/rails solid_observer:storage:cleanup
|
|
240
|
+
|
|
241
|
+
# Delete ALL SolidObserver data (events + snapshots, interactive confirmation)
|
|
242
|
+
bin/rails solid_observer:storage:purge
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
> **Important:** `storage:purge` deletes SolidObserver's monitoring data, NOT your Solid Queue jobs. Your queued jobs remain safe.
|
|
246
|
+
|
|
247
|
+
## Database Setup
|
|
248
|
+
|
|
249
|
+
**SolidObserver works with any main application database** — PostgreSQL, MySQL, or SQLite.
|
|
250
|
+
|
|
251
|
+
For its own monitoring data, SolidObserver uses a **separate SQLite database**. This keeps monitoring isolated from your main app and provides simple file-based storage that requires no additional infrastructure.
|
|
252
|
+
|
|
253
|
+
```yaml
|
|
254
|
+
# config/database.yml
|
|
255
|
+
solid_observer_queue:
|
|
256
|
+
<<: *default
|
|
257
|
+
adapter: sqlite3 # Always SQLite for SolidObserver storage
|
|
258
|
+
database: storage/<%= Rails.env %>_solid_observer_queue.sqlite3
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
> **Note:** Your main app's `primary` database can be PostgreSQL, MySQL, or any Rails-supported adapter. Only the `solid_observer_queue` database needs to be SQLite.
|
|
262
|
+
|
|
263
|
+
## Roadmap
|
|
264
|
+
|
|
265
|
+
SolidObserver is actively developed. Here's what's coming:
|
|
266
|
+
|
|
267
|
+
| Version | Focus | Status |
|
|
268
|
+
|---------|-------|--------|
|
|
269
|
+
| v0.1.0 | Solid Queue monitoring, CLI tools | ✅ Current |
|
|
270
|
+
| v0.2.0 | Solid Cache monitoring | 🔜 Planned |
|
|
271
|
+
| v0.3.0 | Solid Cable monitoring | 🔜 Planned |
|
|
272
|
+
| v0.4.0 | Cross-component correlation, health scores | 🔜 Planned |
|
|
273
|
+
| v0.5.0 | Alerting & notifications | 🔜 Planned |
|
|
274
|
+
| v0.6.0 | Web UI dashboard | 🔜 Planned |
|
|
275
|
+
| v1.0.0 | Production stable release | 🎯 Goal |
|
|
276
|
+
|
|
277
|
+
See [GitHub Milestones](https://github.com/bart-oz/solid_observer/milestones) for detailed plans.
|
|
278
|
+
|
|
279
|
+
## Development
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
# Clone the repository
|
|
283
|
+
git clone https://github.com/bart-oz/solid_observer.git
|
|
284
|
+
cd solid_observer
|
|
285
|
+
|
|
286
|
+
# Install dependencies
|
|
287
|
+
bin/setup
|
|
288
|
+
|
|
289
|
+
# Run tests
|
|
290
|
+
bundle exec rspec
|
|
291
|
+
|
|
292
|
+
# Run linter
|
|
293
|
+
bundle exec standardrb
|
|
294
|
+
|
|
295
|
+
# Run code smell detector
|
|
296
|
+
bundle exec reek
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
## Troubleshooting
|
|
300
|
+
|
|
301
|
+
### "no such table: solid_queue_ready_executions"
|
|
302
|
+
|
|
303
|
+
This error means Solid Queue isn't configured to use the correct database in your environment.
|
|
304
|
+
|
|
305
|
+
**Solution:** Ensure `connects_to` is configured for all environments, not just production:
|
|
306
|
+
|
|
307
|
+
```ruby
|
|
308
|
+
# config/environments/development.rb
|
|
309
|
+
config.solid_queue.connects_to = { database: { writing: :queue } }
|
|
310
|
+
|
|
311
|
+
# config/environments/test.rb
|
|
312
|
+
config.solid_queue.connects_to = { database: { writing: :queue } }
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
### Multi-database setup
|
|
316
|
+
|
|
317
|
+
SolidObserver works with Rails multi-database configurations. Here's an example with PostgreSQL as your primary database:
|
|
318
|
+
|
|
319
|
+
```yaml
|
|
320
|
+
development:
|
|
321
|
+
primary:
|
|
322
|
+
adapter: postgresql
|
|
323
|
+
database: myapp_development
|
|
324
|
+
# ... PostgreSQL settings
|
|
325
|
+
queue:
|
|
326
|
+
<<: *default
|
|
327
|
+
adapter: sqlite3
|
|
328
|
+
database: storage/development_queue.sqlite3
|
|
329
|
+
migrations_paths: db/queue_migrate
|
|
330
|
+
solid_observer_queue:
|
|
331
|
+
adapter: sqlite3
|
|
332
|
+
database: storage/development_solid_observer_queue.sqlite3
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
## Contributing
|
|
336
|
+
|
|
337
|
+
Bug reports and pull requests are welcome on [GitHub](https://github.com/bart-oz/solid_observer).
|
|
338
|
+
|
|
339
|
+
Check out issues labeled:
|
|
340
|
+
- [good first issue](https://github.com/bart-oz/solid_observer/labels/good%20first%20issue) — Great for newcomers
|
|
341
|
+
- [help wanted](https://github.com/bart-oz/solid_observer/labels/help%20wanted) — We'd love your help
|
|
342
|
+
|
|
343
|
+
Please follow the [code of conduct](https://github.com/bart-oz/solid_observer/blob/main/CODE_OF_CONDUCT.md).
|
|
344
|
+
|
|
345
|
+
## License
|
|
346
|
+
|
|
347
|
+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SolidObserver
|
|
4
|
+
class QueueEvent < BaseEvent
|
|
5
|
+
self.table_name = "solid_observer_queue_events"
|
|
6
|
+
|
|
7
|
+
EVENT_TYPES = %w[
|
|
8
|
+
job_enqueued
|
|
9
|
+
job_completed
|
|
10
|
+
job_failed
|
|
11
|
+
job_discarded
|
|
12
|
+
].freeze
|
|
13
|
+
|
|
14
|
+
validates :event_type, presence: true, inclusion: {in: EVENT_TYPES}
|
|
15
|
+
validates :recorded_at, presence: true
|
|
16
|
+
|
|
17
|
+
scope :by_job_class, ->(job_class) { where(job_class: job_class) }
|
|
18
|
+
scope :by_queue, ->(queue_name) { where(queue_name: queue_name) }
|
|
19
|
+
scope :by_event_type, ->(event_type) { where(event_type: event_type) }
|
|
20
|
+
scope :since, ->(time) { where("recorded_at >= ?", time) }
|
|
21
|
+
scope :before, ->(time) { where("recorded_at < ?", time) }
|
|
22
|
+
end
|
|
23
|
+
end
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SolidObserver
|
|
4
|
+
# QueueMetric provides time-series metrics storage for queue statistics.
|
|
5
|
+
#
|
|
6
|
+
# NOTE: Metrics functionality is planned for v0.2.0. This class currently
|
|
7
|
+
# serves as a placeholder and inherits base functionality from BaseMetric.
|
|
8
|
+
# The database connection will be configured by the Engine when metrics
|
|
9
|
+
# are fully implemented.
|
|
10
|
+
#
|
|
11
|
+
# @see BaseMetric for available methods (increment, record)
|
|
12
|
+
class QueueMetric < BaseMetric
|
|
13
|
+
end
|
|
14
|
+
end
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SolidObserver
|
|
4
|
+
class StorageInfo < BaseEvent
|
|
5
|
+
self.table_name = "solid_observer_storage_info"
|
|
6
|
+
|
|
7
|
+
MB_TO_BYTES = 1_048_576
|
|
8
|
+
GB_TO_BYTES = 1_073_741_824
|
|
9
|
+
|
|
10
|
+
validates :db_size_bytes, presence: true, numericality: {only_integer: true, greater_than_or_equal_to: 0}
|
|
11
|
+
validates :event_count, presence: true, numericality: {only_integer: true, greater_than_or_equal_to: 0}
|
|
12
|
+
validates :recorded_at, presence: true
|
|
13
|
+
|
|
14
|
+
scope :recent, ->(limit = 10) { order(recorded_at: :desc).limit(limit) }
|
|
15
|
+
scope :since, ->(time) { where("recorded_at >= ?", time) }
|
|
16
|
+
|
|
17
|
+
def self.record_snapshot(db_size:, event_count:)
|
|
18
|
+
create!(
|
|
19
|
+
db_size_bytes: db_size || 0,
|
|
20
|
+
event_count: event_count,
|
|
21
|
+
recorded_at: Time.current
|
|
22
|
+
)
|
|
23
|
+
rescue ActiveRecord::RecordInvalid => e
|
|
24
|
+
Rails.logger.error "[SolidObserver] Failed to record storage snapshot: #{e.message}"
|
|
25
|
+
raise
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def db_size_mb
|
|
29
|
+
(db_size_bytes / MB_TO_BYTES.to_f).round(2)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def db_size_gb
|
|
33
|
+
(db_size_bytes / GB_TO_BYTES.to_f).round(2)
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
data/bin/console
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
require "bundler/setup"
|
|
5
|
+
require "solid_observer"
|
|
6
|
+
|
|
7
|
+
# You can add fixtures and/or initialization code here to make experimenting
|
|
8
|
+
# with your gem easier. You can also use a different console, if you like.
|
|
9
|
+
|
|
10
|
+
require "irb"
|
|
11
|
+
IRB.start(__FILE__)
|
data/bin/setup
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
class CreateSolidObserverQueueEvents < ActiveRecord::Migration[8.0]
|
|
4
|
+
def change
|
|
5
|
+
create_table :solid_observer_queue_events do |t|
|
|
6
|
+
t.string :event_type, null: false, limit: 50
|
|
7
|
+
t.string :job_class, limit: 100
|
|
8
|
+
t.string :queue_name, limit: 50
|
|
9
|
+
t.string :correlation_id, limit: 64
|
|
10
|
+
t.text :metadata
|
|
11
|
+
t.float :duration
|
|
12
|
+
t.datetime :recorded_at, null: false
|
|
13
|
+
|
|
14
|
+
t.index :recorded_at
|
|
15
|
+
t.index :correlation_id, where: "correlation_id IS NOT NULL"
|
|
16
|
+
t.index :event_type
|
|
17
|
+
t.index :job_class
|
|
18
|
+
t.index :queue_name
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
class CreateSolidObserverMetrics < ActiveRecord::Migration[8.0]
|
|
4
|
+
def change
|
|
5
|
+
create_table :solid_observer_metrics do |t|
|
|
6
|
+
t.string :metric_name, null: false, limit: 50
|
|
7
|
+
t.bigint :value, null: false, default: 0
|
|
8
|
+
t.datetime :period_start, null: false
|
|
9
|
+
t.string :period_type, null: false, limit: 10
|
|
10
|
+
|
|
11
|
+
t.index [:metric_name, :period_start, :period_type],
|
|
12
|
+
unique: true,
|
|
13
|
+
name: "idx_solid_observer_metrics_unique"
|
|
14
|
+
end
|
|
15
|
+
end
|
|
16
|
+
end
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
class CreateSolidObserverStorageInfo < ActiveRecord::Migration[8.0]
|
|
4
|
+
def change
|
|
5
|
+
create_table :solid_observer_storage_info do |t|
|
|
6
|
+
t.bigint :db_size_bytes, null: false
|
|
7
|
+
t.bigint :event_count, null: false
|
|
8
|
+
t.datetime :recorded_at, null: false
|
|
9
|
+
|
|
10
|
+
t.index :recorded_at
|
|
11
|
+
end
|
|
12
|
+
end
|
|
13
|
+
end
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "rails/generators"
|
|
4
|
+
|
|
5
|
+
module SolidObserver
|
|
6
|
+
module Generators
|
|
7
|
+
class InstallGenerator < Rails::Generators::Base
|
|
8
|
+
source_root File.expand_path("templates", __dir__)
|
|
9
|
+
|
|
10
|
+
desc "Install SolidObserver with initializer and database configuration"
|
|
11
|
+
|
|
12
|
+
def create_initializer
|
|
13
|
+
template "initializer.rb.tt", "config/initializers/solid_observer.rb"
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def add_database_configuration
|
|
17
|
+
%w[development test production].each do |env|
|
|
18
|
+
config_block = <<-YAML
|
|
19
|
+
solid_observer_queue:
|
|
20
|
+
<<: *default
|
|
21
|
+
database: storage/#{env}_solid_observer_queue.sqlite3
|
|
22
|
+
YAML
|
|
23
|
+
inject_into_file "config/database.yml", config_block, after: /^#{env}:\n(?: .*\n)*/
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def show_instructions
|
|
28
|
+
say "\n"
|
|
29
|
+
print_banner
|
|
30
|
+
say "\n"
|
|
31
|
+
say "Next steps:", :yellow
|
|
32
|
+
say " 1. Review configuration in config/initializers/solid_observer.rb"
|
|
33
|
+
say " 2. Install migrations: bin/rails solid_observer:install:migrations"
|
|
34
|
+
say " 3. Create database: bin/rails db:create"
|
|
35
|
+
say " 4. Run migrations: bin/rails db:migrate"
|
|
36
|
+
say " 5. Restart your Rails server"
|
|
37
|
+
say "\n"
|
|
38
|
+
say "Documentation: https://solid.observer", :cyan
|
|
39
|
+
say "GitHub: https://github.com/bart-oz/solid_observer", :cyan
|
|
40
|
+
say "\n"
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
private
|
|
44
|
+
|
|
45
|
+
def print_banner
|
|
46
|
+
banner = <<~BANNER
|
|
47
|
+
|
|
48
|
+
███████╗ ██████╗ ██╗ ██╗██████╗
|
|
49
|
+
██╔════╝██╔═══██╗██║ ██║██╔══██╗
|
|
50
|
+
███████╗██║ ██║██║ ██║██║ ██║
|
|
51
|
+
╚════██║██║ ██║██║ ██║██║ ██║
|
|
52
|
+
███████║╚██████╔╝███████╗██║██████╔╝
|
|
53
|
+
╚══════╝ ╚═════╝ ╚══════╝╚═╝╚═════╝
|
|
54
|
+
|
|
55
|
+
██████╗ ██████╗ ███████╗███████╗██████╗ ██╗ ██╗███████╗██████╗
|
|
56
|
+
██╔═══██╗██╔══██╗██╔════╝██╔════╝██╔══██╗██║ ██║██╔════╝██╔══██╗
|
|
57
|
+
██║ ██║██████╔╝███████╗█████╗ ██████╔╝██║ ██║█████╗ ██████╔╝
|
|
58
|
+
██║ ██║██╔══██╗╚════██║██╔══╝ ██╔══██╗╚██╗ ██╔╝██╔══╝ ██╔══██╗
|
|
59
|
+
╚██████╔╝██████╔╝███████║███████╗██║ ██║ ╚████╔╝ ███████╗██║ ██║
|
|
60
|
+
╚═════╝ ╚═════╝ ╚══════╝╚══════╝╚═╝ ╚═╝ ╚═══╝ ╚══════╝╚═╝ ╚═╝
|
|
61
|
+
|
|
62
|
+
Observe your Solid Stack like a pro! 🔭
|
|
63
|
+
v#{SolidObserver::VERSION}
|
|
64
|
+
|
|
65
|
+
BANNER
|
|
66
|
+
|
|
67
|
+
banner.each_line { |line| say line.chomp, :cyan }
|
|
68
|
+
say " ✓ SolidObserver installed successfully!", :green
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
end
|