source_monitor 0.3.0 → 0.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (54) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/skills/sm-architecture/SKILL.md +233 -0
  3. data/.claude/skills/sm-architecture/reference/extraction-patterns.md +192 -0
  4. data/.claude/skills/sm-architecture/reference/module-map.md +194 -0
  5. data/.claude/skills/sm-configuration-setting/SKILL.md +264 -0
  6. data/.claude/skills/sm-configuration-setting/reference/settings-catalog.md +248 -0
  7. data/.claude/skills/sm-configuration-setting/reference/settings-pattern.md +297 -0
  8. data/.claude/skills/sm-configure/SKILL.md +153 -0
  9. data/.claude/skills/sm-configure/reference/configuration-reference.md +321 -0
  10. data/.claude/skills/sm-dashboard-widget/SKILL.md +344 -0
  11. data/.claude/skills/sm-dashboard-widget/reference/dashboard-patterns.md +304 -0
  12. data/.claude/skills/sm-domain-model/SKILL.md +188 -0
  13. data/.claude/skills/sm-domain-model/reference/model-graph.md +114 -0
  14. data/.claude/skills/sm-domain-model/reference/table-structure.md +348 -0
  15. data/.claude/skills/sm-engine-migration/SKILL.md +395 -0
  16. data/.claude/skills/sm-engine-migration/reference/migration-conventions.md +255 -0
  17. data/.claude/skills/sm-engine-test/SKILL.md +302 -0
  18. data/.claude/skills/sm-engine-test/reference/test-helpers.md +259 -0
  19. data/.claude/skills/sm-engine-test/reference/test-patterns.md +411 -0
  20. data/.claude/skills/sm-event-handler/SKILL.md +265 -0
  21. data/.claude/skills/sm-event-handler/reference/events-api.md +229 -0
  22. data/.claude/skills/sm-health-rule/SKILL.md +327 -0
  23. data/.claude/skills/sm-health-rule/reference/health-system.md +269 -0
  24. data/.claude/skills/sm-host-setup/SKILL.md +223 -0
  25. data/.claude/skills/sm-host-setup/reference/initializer-template.md +195 -0
  26. data/.claude/skills/sm-host-setup/reference/setup-checklist.md +134 -0
  27. data/.claude/skills/sm-job/SKILL.md +263 -0
  28. data/.claude/skills/sm-job/reference/job-conventions.md +245 -0
  29. data/.claude/skills/sm-model-extension/SKILL.md +287 -0
  30. data/.claude/skills/sm-model-extension/reference/extension-api.md +317 -0
  31. data/.claude/skills/sm-pipeline-stage/SKILL.md +254 -0
  32. data/.claude/skills/sm-pipeline-stage/reference/completion-handlers.md +152 -0
  33. data/.claude/skills/sm-pipeline-stage/reference/entry-processing.md +191 -0
  34. data/.claude/skills/sm-pipeline-stage/reference/feed-fetcher-architecture.md +198 -0
  35. data/.claude/skills/sm-scraper-adapter/SKILL.md +284 -0
  36. data/.claude/skills/sm-scraper-adapter/reference/adapter-contract.md +167 -0
  37. data/.claude/skills/sm-scraper-adapter/reference/example-adapter.md +274 -0
  38. data/.vbw-planning/.notification-log.jsonl +102 -0
  39. data/.vbw-planning/.session-log.jsonl +505 -0
  40. data/AGENTS.md +20 -57
  41. data/CHANGELOG.md +19 -0
  42. data/CLAUDE.md +44 -1
  43. data/CONTRIBUTING.md +5 -5
  44. data/Gemfile.lock +20 -21
  45. data/README.md +18 -5
  46. data/VERSION +1 -0
  47. data/docs/deployment.md +1 -1
  48. data/docs/setup.md +4 -4
  49. data/lib/source_monitor/setup/skills_installer.rb +94 -0
  50. data/lib/source_monitor/setup/workflow.rb +17 -2
  51. data/lib/source_monitor/version.rb +1 -1
  52. data/lib/tasks/source_monitor_setup.rake +58 -0
  53. data/source_monitor.gemspec +1 -0
  54. metadata +39 -1
@@ -0,0 +1,321 @@
1
+ # Configuration Reference
2
+
3
+ Complete reference for every SourceMonitor configuration setting.
4
+
5
+ Source: `lib/source_monitor/configuration.rb` and `lib/source_monitor/configuration/*.rb`
6
+
7
+ ## Top-Level Settings
8
+
9
+ Defined on `SourceMonitor::Configuration`:
10
+
11
+ | Setting | Type | Default | Description |
12
+ |---|---|---|---|
13
+ | `queue_namespace` | String | `"source_monitor"` | Prefix for queue names and instrumentation keys |
14
+ | `fetch_queue_name` | String | `"source_monitor_fetch"` | Queue name for fetch jobs |
15
+ | `scrape_queue_name` | String | `"source_monitor_scrape"` | Queue name for scrape jobs |
16
+ | `fetch_queue_concurrency` | Integer | `2` | Advisory concurrency for fetch queue |
17
+ | `scrape_queue_concurrency` | Integer | `2` | Advisory concurrency for scrape queue |
18
+ | `recurring_command_job_class` | String/Class/nil | `nil` | Override Solid Queue recurring task job class |
19
+ | `job_metrics_enabled` | Boolean | `true` | Toggle queue metrics on dashboard |
20
+ | `mission_control_enabled` | Boolean | `false` | Show Mission Control link on dashboard |
21
+ | `mission_control_dashboard_path` | String/Proc/nil | `nil` | Path or callable returning MC URL |
22
+
23
+ ### Methods
24
+
25
+ | Method | Signature | Description |
26
+ |---|---|---|
27
+ | `queue_name_for` | `(role) -> String` | Returns resolved queue name with host prefix |
28
+ | `concurrency_for` | `(role) -> Integer` | Returns concurrency for `:fetch` or `:scrape` |
29
+
30
+ ---
31
+
32
+ ## HTTP Settings (`config.http`)
33
+
34
+ Class: `SourceMonitor::Configuration::HTTPSettings`
35
+
36
+ | Setting | Type | Default | Description |
37
+ |---|---|---|---|
38
+ | `timeout` | Integer | `15` | Total request timeout (seconds) |
39
+ | `open_timeout` | Integer | `5` | Connection open timeout (seconds) |
40
+ | `max_redirects` | Integer | `5` | Maximum redirects to follow |
41
+ | `user_agent` | String | `"SourceMonitor/<version>"` | User-Agent header |
42
+ | `proxy` | String/Hash/nil | `nil` | HTTP proxy configuration |
43
+ | `headers` | Hash | `{}` | Extra headers merged into every request |
44
+ | `retry_max` | Integer | `4` | Maximum retry attempts |
45
+ | `retry_interval` | Float | `0.5` | Initial retry delay (seconds) |
46
+ | `retry_interval_randomness` | Float | `0.5` | Randomness factor for retry jitter |
47
+ | `retry_backoff_factor` | Integer | `2` | Exponential backoff multiplier |
48
+ | `retry_statuses` | Array/nil | `nil` | HTTP status codes to retry (nil = use defaults) |
49
+
50
+ ```ruby
51
+ config.http.timeout = 30
52
+ config.http.proxy = "http://proxy.example.com:8080"
53
+ config.http.headers = { "X-Custom" => "value" }
54
+ config.http.retry_statuses = [429, 500, 502, 503, 504]
55
+ ```
56
+
57
+ ---
58
+
59
+ ## Fetching Settings (`config.fetching`)
60
+
61
+ Class: `SourceMonitor::Configuration::FetchingSettings`
62
+
63
+ Controls adaptive fetch scheduling.
64
+
65
+ | Setting | Type | Default | Description |
66
+ |---|---|---|---|
67
+ | `min_interval_minutes` | Integer | `5` | Minimum fetch interval (minutes) |
68
+ | `max_interval_minutes` | Integer | `1440` | Maximum fetch interval (24 hours) |
69
+ | `increase_factor` | Float | `1.25` | Multiplier when source trends slow (no new items) |
70
+ | `decrease_factor` | Float | `0.75` | Multiplier when new items arrive |
71
+ | `failure_increase_factor` | Float | `1.5` | Multiplier on consecutive failures |
72
+ | `jitter_percent` | Float | `0.1` | Random jitter (+/-10%, 0 disables) |
73
+
74
+ ```ruby
75
+ config.fetching.min_interval_minutes = 10
76
+ config.fetching.max_interval_minutes = 720 # 12 hours
77
+ config.fetching.jitter_percent = 0.15 # +/-15%
78
+ ```
79
+
80
+ ---
81
+
82
+ ## Health Settings (`config.health`)
83
+
84
+ Class: `SourceMonitor::Configuration::HealthSettings`
85
+
86
+ Tunes automatic pause/resume heuristics per source.
87
+
88
+ | Setting | Type | Default | Description |
89
+ |---|---|---|---|
90
+ | `window_size` | Integer | `20` | Number of fetch attempts to evaluate |
91
+ | `healthy_threshold` | Float | `0.8` | Success ratio for "healthy" badge |
92
+ | `warning_threshold` | Float | `0.5` | Success ratio for "warning" badge |
93
+ | `auto_pause_threshold` | Float | `0.2` | Auto-pause source below this ratio |
94
+ | `auto_resume_threshold` | Float | `0.6` | Auto-resume source above this ratio |
95
+ | `auto_pause_cooldown_minutes` | Integer | `60` | Grace period before re-enabling |
96
+
97
+ ```ruby
98
+ config.health.window_size = 50
99
+ config.health.auto_pause_threshold = 0.1
100
+ config.health.auto_pause_cooldown_minutes = 120
101
+ ```
102
+
103
+ ---
104
+
105
+ ## Scraper Registry (`config.scrapers`)
106
+
107
+ Class: `SourceMonitor::Configuration::ScraperRegistry`
108
+
109
+ Manages scraper adapter registration. Adapters must inherit from `SourceMonitor::Scrapers::Base`.
110
+
111
+ ### Methods
112
+
113
+ | Method | Signature | Description |
114
+ |---|---|---|
115
+ | `register` | `(name, adapter)` | Register adapter by name (symbol/string) |
116
+ | `unregister` | `(name)` | Remove adapter by name |
117
+ | `adapter_for` | `(name) -> Class/nil` | Look up adapter class |
118
+ | `each` | `(&block)` | Iterate registered adapters |
119
+
120
+ Names are normalized to lowercase alphanumeric + underscore.
121
+
122
+ Adapters can be a Class or a String (constantized lazily).
123
+
124
+ ```ruby
125
+ config.scrapers.register(:custom, MyApp::Scrapers::Custom)
126
+ config.scrapers.register(:premium, "MyApp::Scrapers::Premium")
127
+ config.scrapers.unregister(:readability)
128
+ ```
129
+
130
+ ---
131
+
132
+ ## Retention Settings (`config.retention`)
133
+
134
+ Class: `SourceMonitor::Configuration::RetentionSettings`
135
+
136
+ Global defaults inherited by sources with blank retention fields.
137
+
138
+ | Setting | Type | Default | Description |
139
+ |---|---|---|---|
140
+ | `items_retention_days` | Integer/nil | `nil` | Prune items older than N days (nil = keep forever) |
141
+ | `max_items` | Integer/nil | `nil` | Keep only N most recent items (nil = unlimited) |
142
+ | `strategy` | Symbol | `:destroy` | Pruning strategy: `:destroy` or `:soft_delete` |
143
+
144
+ ```ruby
145
+ config.retention.items_retention_days = 90
146
+ config.retention.max_items = 1000
147
+ config.retention.strategy = :soft_delete
148
+ ```
149
+
150
+ ---
151
+
152
+ ## Scraping Settings (`config.scraping`)
153
+
154
+ Class: `SourceMonitor::Configuration::ScrapingSettings`
155
+
156
+ | Setting | Type | Default | Description |
157
+ |---|---|---|---|
158
+ | `max_in_flight_per_source` | Integer/nil | `25` | Max concurrent scrapes per source |
159
+ | `max_bulk_batch_size` | Integer/nil | `100` | Max items per bulk scrape enqueue |
160
+
161
+ Values are normalized to positive integers. Set to `nil` to disable limits.
162
+
163
+ ```ruby
164
+ config.scraping.max_in_flight_per_source = 50
165
+ config.scraping.max_bulk_batch_size = 200
166
+ ```
167
+
168
+ ---
169
+
170
+ ## Events (`config.events`)
171
+
172
+ Class: `SourceMonitor::Configuration::Events`
173
+
174
+ Register lifecycle callbacks. See the `sm-event-handler` skill for full event documentation.
175
+
176
+ ### Callback Methods
177
+
178
+ | Method | Signature | Description |
179
+ |---|---|---|
180
+ | `after_item_created` | `(handler=nil, &block)` | Called after a new item is created from a feed entry |
181
+ | `after_item_scraped` | `(handler=nil, &block)` | Called after an item is scraped for content |
182
+ | `after_fetch_completed` | `(handler=nil, &block)` | Called after a feed fetch finishes |
183
+ | `register_item_processor` | `(processor=nil, &block)` | Register a post-processing pipeline step |
184
+ | `callbacks_for` | `(name) -> Array` | Retrieve callbacks for an event |
185
+ | `item_processors` | `-> Array` | Retrieve registered item processors |
186
+ | `reset!` | `-> void` | Clear all callbacks and processors |
187
+
188
+ Handlers must respond to `#call`. Can be a Proc, lambda, or any callable object.
189
+
190
+ ```ruby
191
+ config.events.after_item_created ->(event) { log(event.item) }
192
+ config.events.after_fetch_completed do |event|
193
+ StatsTracker.record(event.source, event.status)
194
+ end
195
+ ```
196
+
197
+ ---
198
+
199
+ ## Models (`config.models`)
200
+
201
+ Class: `SourceMonitor::Configuration::Models`
202
+
203
+ Per-model extension points for host apps.
204
+
205
+ | Setting | Type | Default | Description |
206
+ |---|---|---|---|
207
+ | `table_name_prefix` | String | `"sourcemon_"` | Table name prefix for all engine tables |
208
+
209
+ ### Model Accessors
210
+
211
+ | Method | Key | Engine Model |
212
+ |---|---|---|
213
+ | `config.models.source` | `:source` | `SourceMonitor::Source` |
214
+ | `config.models.item` | `:item` | `SourceMonitor::Item` |
215
+ | `config.models.fetch_log` | `:fetch_log` | `SourceMonitor::FetchLog` |
216
+ | `config.models.scrape_log` | `:scrape_log` | `SourceMonitor::ScrapeLog` |
217
+ | `config.models.health_check_log` | `:health_check_log` | `SourceMonitor::HealthCheckLog` |
218
+ | `config.models.item_content` | `:item_content` | `SourceMonitor::ItemContent` |
219
+ | `config.models.log_entry` | `:log_entry` | `SourceMonitor::LogEntry` |
220
+
221
+ Each accessor returns a `ModelDefinition` with:
222
+
223
+ | Method | Signature | Description |
224
+ |---|---|---|
225
+ | `include_concern` | `(concern=nil, &block)` | Include a concern module |
226
+ | `validate` | `(handler=nil, **options, &block)` | Register a validation |
227
+
228
+ ```ruby
229
+ config.models.table_name_prefix = "sm_"
230
+ config.models.source.include_concern "MyApp::SourceTagging"
231
+ config.models.item.validate :check_content_length
232
+ ```
233
+
234
+ ---
235
+
236
+ ## Realtime Settings (`config.realtime`)
237
+
238
+ Class: `SourceMonitor::Configuration::RealtimeSettings`
239
+
240
+ | Setting | Type | Default | Description |
241
+ |---|---|---|---|
242
+ | `adapter` | Symbol | `:solid_cable` | One of `:solid_cable`, `:redis`, `:async` |
243
+ | `redis_url` | String/nil | `nil` | Redis URL when using `:redis` adapter |
244
+
245
+ ### Solid Cable Options (`config.realtime.solid_cable`)
246
+
247
+ | Setting | Type | Default | Description |
248
+ |---|---|---|---|
249
+ | `polling_interval` | String | `"0.1.seconds"` | Polling frequency |
250
+ | `message_retention` | String | `"1.day"` | How long to retain messages |
251
+ | `autotrim` | Boolean | `true` | Auto-trim old messages |
252
+ | `silence_polling` | Boolean | `true` | Suppress polling log noise |
253
+ | `use_skip_locked` | Boolean | `true` | Use PostgreSQL SKIP LOCKED |
254
+ | `trim_batch_size` | Integer/nil | `nil` | Batch size for trim operations |
255
+ | `connects_to` | Hash/nil | `nil` | Multi-database routing |
256
+
257
+ ### Methods
258
+
259
+ | Method | Returns | Description |
260
+ |---|---|---|
261
+ | `action_cable_config` | Hash | Full configuration hash for cable.yml |
262
+
263
+ ```ruby
264
+ config.realtime.adapter = :redis
265
+ config.realtime.redis_url = "redis://localhost:6379/1"
266
+
267
+ # Or with Solid Cable tuning:
268
+ config.realtime.adapter = :solid_cable
269
+ config.realtime.solid_cable.polling_interval = "0.05.seconds"
270
+ config.realtime.solid_cable.connects_to = { database: { writing: :cable } }
271
+ ```
272
+
273
+ ---
274
+
275
+ ## Authentication Settings (`config.authentication`)
276
+
277
+ Class: `SourceMonitor::Configuration::AuthenticationSettings`
278
+
279
+ | Setting | Type | Default | Description |
280
+ |---|---|---|---|
281
+ | `current_user_method` | Symbol/nil | `nil` | Controller method to get current user |
282
+ | `user_signed_in_method` | Symbol/nil | `nil` | Controller method to check sign-in status |
283
+
284
+ ### Methods
285
+
286
+ | Method | Signature | Description |
287
+ |---|---|---|
288
+ | `authenticate_with` | `(handler=nil, &block)` | Set authentication handler |
289
+ | `authorize_with` | `(handler=nil, &block)` | Set authorization handler |
290
+
291
+ Handlers can be:
292
+ - **Symbol**: Invoked as `controller.public_send(handler)`
293
+ - **Callable**: Called with `callable.call(controller)` (or `instance_exec` if arity is 0)
294
+
295
+ ```ruby
296
+ # Symbol handler (Devise)
297
+ config.authentication.authenticate_with :authenticate_user!
298
+
299
+ # Callable handler
300
+ config.authentication.authorize_with ->(controller) {
301
+ controller.current_user&.feature_enabled?(:source_monitor)
302
+ }
303
+
304
+ # Block handler (instance_exec on controller)
305
+ config.authentication.authorize_with do
306
+ redirect_to root_path unless current_user&.admin?
307
+ end
308
+ ```
309
+
310
+ ---
311
+
312
+ ## Environment Variables
313
+
314
+ | Variable | Purpose |
315
+ |---|---|
316
+ | `SOLID_QUEUE_SKIP_RECURRING` | Skip loading `config/recurring.yml` |
317
+ | `SOLID_QUEUE_RECURRING_SCHEDULE_FILE` | Alternative schedule file path |
318
+ | `SOFT_DELETE` | Override retention strategy in rake tasks |
319
+ | `SOURCE_IDS` / `SOURCE_ID` | Scope cleanup rake tasks to specific sources |
320
+ | `FETCH_LOG_DAYS` / `SCRAPE_LOG_DAYS` | Retention windows for log cleanup |
321
+ | `SOURCE_MONITOR_SETUP_TELEMETRY` | Enable setup verification telemetry logging |
@@ -0,0 +1,344 @@
1
+ ---
2
+ name: sm-dashboard-widget
3
+ description: Creates dashboard widgets with queries, presenters, and Turbo broadcasts for Source Monitor. Use when building dashboard metrics, stats panels, or real-time monitoring displays.
4
+ allowed-tools: Read, Write, Edit, Bash, Glob, Grep
5
+ disable-model-invocation: true
6
+ ---
7
+
8
+ # SourceMonitor Dashboard Widget
9
+
10
+ ## Overview
11
+
12
+ The SourceMonitor dashboard lives at `DashboardController#index` and is composed of widgets: stat cards, recent activity feeds, job queue metrics, fetch schedules, and quick actions. Each widget follows a query-presenter-view pattern with optional Turbo Stream broadcasting for real-time updates.
13
+
14
+ ## Architecture
15
+
16
+ ```
17
+ Controller (app/controllers/source_monitor/dashboard_controller.rb)
18
+ |
19
+ v
20
+ Queries (lib/source_monitor/dashboard/queries.rb) -- facade
21
+ |-- StatsQuery -- single SQL for source counts + item/fetch totals
22
+ |-- RecentActivityQuery -- UNION ALL across fetch_logs, scrape_logs, items
23
+ |-- UpcomingFetchSchedule -- groups active sources into time-window buckets
24
+ |-- job_metrics -- delegates to Jobs::SolidQueueMetrics
25
+ |-- quick_actions -- static QuickAction structs
26
+ |
27
+ v
28
+ Presenters
29
+ |-- RecentActivityPresenter -- maps Event structs to view-model hashes
30
+ |-- QuickActionsPresenter -- maps QuickAction structs to path-resolved hashes
31
+ |
32
+ v
33
+ Views (app/views/source_monitor/dashboard/)
34
+ |-- index.html.erb -- layout shell with turbo_stream_from
35
+ |-- _stats.html.erb -- grid of stat cards
36
+ |-- _stat_card.html.erb -- single stat card partial (collection render)
37
+ |-- _recent_activity.html.erb
38
+ |-- _fetch_schedule.html.erb
39
+ |-- _job_metrics.html.erb
40
+ |
41
+ v
42
+ TurboBroadcaster (lib/source_monitor/dashboard/turbo_broadcaster.rb)
43
+ |-- registers event callbacks (after_fetch_completed, after_item_created)
44
+ |-- broadcasts replace_to for stats, recent_activity, fetch_schedule
45
+ ```
46
+
47
+ ## Creating a New Dashboard Widget
48
+
49
+ ### Step 1: Define the Query
50
+
51
+ Create a query class under `lib/source_monitor/dashboard/queries/`. Follow the existing pattern:
52
+
53
+ ```ruby
54
+ # frozen_string_literal: true
55
+
56
+ module SourceMonitor
57
+ module Dashboard
58
+ class Queries
59
+ class MyWidgetQuery
60
+ def initialize(reference_time:)
61
+ @reference_time = reference_time
62
+ end
63
+
64
+ def call
65
+ # Return a hash, array of structs, or value object
66
+ {
67
+ metric_a: compute_metric_a,
68
+ metric_b: compute_metric_b
69
+ }
70
+ end
71
+
72
+ private
73
+
74
+ attr_reader :reference_time
75
+
76
+ def compute_metric_a
77
+ # Use quoted_table_name for SQL safety
78
+ SourceMonitor::Source.connection.select_value(<<~SQL.squish).to_i
79
+ SELECT COUNT(*)
80
+ FROM #{SourceMonitor::Source.quoted_table_name}
81
+ WHERE some_condition
82
+ SQL
83
+ end
84
+
85
+ def compute_metric_b
86
+ SourceMonitor::Item.where("created_at >= ?", reference_time.beginning_of_day).count
87
+ end
88
+ end
89
+ end
90
+ end
91
+ end
92
+ ```
93
+
94
+ Key patterns:
95
+ - Constructor takes `reference_time:` for time-relative queries
96
+ - `call` returns the result (hash, array, or value object)
97
+ - Use `quoted_table_name` for all SQL references
98
+ - Use `connection.exec_query` for multi-column results
99
+ - Use `connection.select_value` for single scalar values
100
+ - Use ActiveRecord scopes when raw SQL is not needed
101
+
102
+ ### Step 2: Register in the Queries Facade
103
+
104
+ Add a method in `lib/source_monitor/dashboard/queries.rb`:
105
+
106
+ ```ruby
107
+ def my_widget
108
+ cache.fetch(:my_widget) do
109
+ measure(:my_widget) do
110
+ MyWidgetQuery.new(reference_time:).call
111
+ end
112
+ end
113
+ end
114
+ ```
115
+
116
+ The `cache` prevents duplicate computation within a single request. The `measure` wrapper:
117
+ 1. Records timing via `ActiveSupport::Notifications.instrument`
118
+ 2. Publishes gauge metrics via `SourceMonitor::Metrics.gauge`
119
+
120
+ Add a require at the top of the file:
121
+ ```ruby
122
+ require "source_monitor/dashboard/queries/my_widget_query"
123
+ ```
124
+
125
+ ### Step 3: Create a Presenter (if needed)
126
+
127
+ Presenters live in `lib/source_monitor/dashboard/`. They transform query results into view-friendly hashes, resolving route paths using `url_helpers`.
128
+
129
+ ```ruby
130
+ # frozen_string_literal: true
131
+
132
+ module SourceMonitor
133
+ module Dashboard
134
+ class MyWidgetPresenter
135
+ def initialize(data, url_helpers:)
136
+ @data = data
137
+ @url_helpers = url_helpers
138
+ end
139
+
140
+ def to_a
141
+ data.map { |item| build_view_model(item) }
142
+ end
143
+
144
+ private
145
+
146
+ attr_reader :data, :url_helpers
147
+
148
+ def build_view_model(item)
149
+ {
150
+ label: item.name,
151
+ value: item.count,
152
+ path: url_helpers.source_path(item.source_id)
153
+ }
154
+ end
155
+ end
156
+ end
157
+ end
158
+ ```
159
+
160
+ Presenter conventions:
161
+ - Constructor takes raw data + `url_helpers:` keyword
162
+ - `to_a` returns array of plain hashes for the view
163
+ - `url_helpers` comes from `SourceMonitor::Engine.routes.url_helpers`
164
+
165
+ ### Step 4: Wire into the Controller
166
+
167
+ In `app/controllers/source_monitor/dashboard_controller.rb`:
168
+
169
+ ```ruby
170
+ def index
171
+ queries = SourceMonitor::Dashboard::Queries.new
172
+ url_helpers = SourceMonitor::Engine.routes.url_helpers
173
+
174
+ # existing assigns...
175
+ @my_widget = MyWidgetPresenter.new(queries.my_widget, url_helpers:).to_a
176
+ end
177
+ ```
178
+
179
+ ### Step 5: Create the View Partial
180
+
181
+ Create `app/views/source_monitor/dashboard/_my_widget.html.erb`:
182
+
183
+ ```erb
184
+ <div id="source_monitor_dashboard_my_widget" class="rounded-lg border border-slate-200 bg-white shadow-sm">
185
+ <div class="border-b border-slate-200 px-5 py-4">
186
+ <h2 class="text-lg font-medium">My Widget</h2>
187
+ <p class="mt-1 text-xs text-slate-500">Description of what this widget shows.</p>
188
+ </div>
189
+ <div class="divide-y divide-slate-100">
190
+ <%% if my_widget.any? %>
191
+ <%% my_widget.each do |item| %>
192
+ <div class="flex items-center justify-between px-5 py-4">
193
+ <div class="text-sm font-medium text-slate-900"><%%= item[:label] %></div>
194
+ <div class="text-sm text-slate-600"><%%= item[:value] %></div>
195
+ </div>
196
+ <%% end %>
197
+ <%% else %>
198
+ <div class="px-5 py-6 text-sm text-slate-500">No data available.</div>
199
+ <%% end %>
200
+ </div>
201
+ </div>
202
+ ```
203
+
204
+ Important: The outer div must have an `id` prefixed with `source_monitor_dashboard_` for Turbo Stream targeting.
205
+
206
+ Render in `index.html.erb`:
207
+ ```erb
208
+ <%%= render "my_widget", my_widget: @my_widget %>
209
+ ```
210
+
211
+ ### Step 6: Add Turbo Stream Broadcasting (optional)
212
+
213
+ In `lib/source_monitor/dashboard/turbo_broadcaster.rb`, add a broadcast call inside `broadcast_dashboard_updates`:
214
+
215
+ ```ruby
216
+ Turbo::StreamsChannel.broadcast_replace_to(
217
+ STREAM_NAME,
218
+ target: "source_monitor_dashboard_my_widget",
219
+ html: render_partial(
220
+ "source_monitor/dashboard/my_widget",
221
+ my_widget: MyWidgetPresenter.new(
222
+ queries.my_widget,
223
+ url_helpers:
224
+ ).to_a
225
+ )
226
+ )
227
+ ```
228
+
229
+ The broadcaster is triggered by event callbacks registered in `setup!`. Existing triggers:
230
+ - `after_fetch_completed` -- fires after each feed fetch
231
+ - `after_item_created` -- fires when a new item is persisted
232
+
233
+ To add a new trigger, register another callback:
234
+ ```ruby
235
+ register_callback(:after_scrape_completed, scrape_callback)
236
+ ```
237
+
238
+ ### Step 7: Add Metrics Recording
239
+
240
+ In the `record_metrics` method of `queries.rb`, add a case for your widget:
241
+
242
+ ```ruby
243
+ when :my_widget
244
+ SourceMonitor::Metrics.gauge(:dashboard_my_widget_count, result.size)
245
+ ```
246
+
247
+ ## Existing Widget Reference
248
+
249
+ ### Stat Cards
250
+ - Query: `StatsQuery` returns `{ total_sources:, active_sources:, failed_sources:, total_items:, fetches_today: }`
251
+ - View: `_stats.html.erb` renders `_stat_card.html.erb` as a collection
252
+ - Turbo target: `source_monitor_dashboard_stats`
253
+
254
+ ### Recent Activity
255
+ - Query: `RecentActivityQuery` uses UNION ALL across fetch_logs, scrape_logs, items
256
+ - Struct: `RecentActivity::Event` (keyword_init Struct)
257
+ - Presenter: `RecentActivityPresenter` maps events to `{ label:, description:, status:, type:, time:, path: }`
258
+ - View: `_recent_activity.html.erb`
259
+ - Turbo target: `source_monitor_dashboard_recent_activity`
260
+
261
+ ### Job Metrics
262
+ - Query: `job_metrics` delegates to `Jobs::SolidQueueMetrics.call`
263
+ - Returns array of `{ role:, queue_name:, summary: }` hashes
264
+ - View: `_job_metrics.html.erb`
265
+ - No Turbo broadcasting (refreshed on page load)
266
+
267
+ ### Fetch Schedule
268
+ - Query: `UpcomingFetchSchedule` groups sources into time-window buckets
269
+ - Struct: `UpcomingFetchSchedule::Group` with `:key, :label, :sources, :window_start, :window_end`
270
+ - View: `_fetch_schedule.html.erb`
271
+ - Turbo target: `source_monitor_dashboard_fetch_schedule`
272
+
273
+ ### Quick Actions
274
+ - Static `QuickAction` structs defined in `Queries::QUICK_ACTIONS`
275
+ - Presenter: `QuickActionsPresenter` resolves route names to paths
276
+ - View: inline in `index.html.erb`
277
+
278
+ ## Data Structures
279
+
280
+ ### RecentActivity::Event (Struct)
281
+ ```ruby
282
+ Struct.new(
283
+ :type, :id, :occurred_at, :success, :items_created,
284
+ :items_updated, :scraper_adapter, :item_title, :item_url,
285
+ :source_name, :source_id, keyword_init: true
286
+ )
287
+ ```
288
+
289
+ ### UpcomingFetchSchedule::Group (Struct)
290
+ ```ruby
291
+ Struct.new(
292
+ :key, :label, :min_minutes, :max_minutes,
293
+ :window_start, :window_end, :include_unscheduled, :sources,
294
+ keyword_init: true
295
+ )
296
+ ```
297
+
298
+ ### QuickAction (Struct)
299
+ ```ruby
300
+ Struct.new(:label, :description, :route_name, keyword_init: true)
301
+ ```
302
+
303
+ ## Turbo Stream Setup
304
+
305
+ The dashboard view subscribes to the broadcast channel:
306
+ ```erb
307
+ <%%= turbo_stream_from SourceMonitor::Dashboard::TurboBroadcaster::STREAM_NAME %>
308
+ ```
309
+
310
+ Stream name: `"source_monitor_dashboard"`
311
+
312
+ The broadcaster uses `DashboardController.render(partial:, locals:)` to render partials outside of a request context.
313
+
314
+ ## Caching
315
+
316
+ The `Queries` class uses an in-memory `Cache` (simple hash store) scoped to the instance. Each `Queries.new` gets a fresh cache, so data is fresh per request but not duplicated within one request.
317
+
318
+ ## File Locations
319
+
320
+ | Component | Path |
321
+ |-----------|------|
322
+ | Controller | `app/controllers/source_monitor/dashboard_controller.rb` |
323
+ | Queries facade | `lib/source_monitor/dashboard/queries.rb` |
324
+ | StatsQuery | `lib/source_monitor/dashboard/queries/stats_query.rb` |
325
+ | RecentActivityQuery | `lib/source_monitor/dashboard/queries/recent_activity_query.rb` |
326
+ | UpcomingFetchSchedule | `lib/source_monitor/dashboard/upcoming_fetch_schedule.rb` |
327
+ | RecentActivity::Event | `lib/source_monitor/dashboard/recent_activity.rb` |
328
+ | RecentActivityPresenter | `lib/source_monitor/dashboard/recent_activity_presenter.rb` |
329
+ | QuickAction | `lib/source_monitor/dashboard/quick_action.rb` |
330
+ | QuickActionsPresenter | `lib/source_monitor/dashboard/quick_actions_presenter.rb` |
331
+ | TurboBroadcaster | `lib/source_monitor/dashboard/turbo_broadcaster.rb` |
332
+ | Views | `app/views/source_monitor/dashboard/` |
333
+
334
+ ## Testing
335
+
336
+ Dashboard queries should be tested with integration tests that verify SQL correctness against real records. See `test/lib/source_monitor/dashboard/` for existing test patterns.
337
+
338
+ Presenters can be unit-tested with mock url_helpers:
339
+ ```ruby
340
+ class FakeUrlHelpers
341
+ def source_path(id) = "/source_monitor/sources/#{id}"
342
+ def fetch_log_path(id) = "/source_monitor/fetch_logs/#{id}"
343
+ end
344
+ ```