source_monitor 0.12.0 → 0.12.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.claude/commands/release.md +70 -47
- data/.claude/skills/sm-architecture/reference/module-map.md +32 -0
- data/.claude/skills/sm-domain-model/reference/model-graph.md +42 -1
- data/.claude/skills/sm-job/reference/job-conventions.md +46 -40
- data/.claude/skills/sm-upgrade/reference/version-history.md +19 -0
- data/CHANGELOG.md +7 -0
- data/Gemfile.lock +1 -1
- data/README.md +6 -3
- data/VERSION +1 -1
- data/docs/setup.md +2 -2
- data/docs/upgrade.md +23 -0
- data/lib/source_monitor/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d460ce01dd1453813fe455bde15c1b8fc2eb6954ca20ff4bf7599b55480c98d5
|
|
4
|
+
data.tar.gz: 16e5903aa483ba06cc092b08a8148edc3050a3830cd4dda42abfbca3d1cab25f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 20a4b8a80f4a861d8ea7e543e4cd90e31709bf9a4d504cf40059375fa439f5d5dd3803bb6ad133d50c75dfe50ee42cfd0913cae66ea41422df1473710493bb5c
|
|
7
|
+
data.tar.gz: ae82ac7575c31e93839bbe48c309a71f57cc1ac91ecd67a4f7690d978be459459abcdafb554fad36e98ff0cdcc424133f6a77f4f756fdfbaf6e11b22c8c9ce27
|
data/.claude/commands/release.md
CHANGED
|
@@ -26,7 +26,8 @@ These are real issues encountered in previous releases. Each step below accounts
|
|
|
26
26
|
9. **ESLint browser globals**: Any JS file using browser APIs (MutationObserver, requestAnimationFrame, cancelAnimationFrame, IntersectionObserver, etc.) MUST declare them with a `/* global ... */` comment at the top. ESLint's `no-undef` rule in CI will reject them otherwise.
|
|
27
27
|
10. **Diff coverage rescue paths**: Every `rescue`/fallback/error handling branch in changed source code needs test coverage. Common blind spots: `rescue StandardError => e` logging, `rescue URI::InvalidURIError` returning nil, fallback `false` returns. Write targeted tests for these BEFORE creating the release commit.
|
|
28
28
|
11. **Zsh glob nomatch**: Commands like `rm -f *.gem` fail in zsh when no files match. Always use `rm -f *.gem 2>/dev/null || true` or check existence first with `ls`.
|
|
29
|
-
12. **Documentation drift**: Features, config options, and behavioral changes often land across milestone work without corresponding doc updates.
|
|
29
|
+
12. **Documentation drift**: Features, config options, and behavioral changes often land across milestone work without corresponding doc updates. Step 4 now spawns agents to both audit AND fix all stale docs automatically. In v0.9.x, 14 files needed updates; in v0.12.0, 7 files were stale (README version refs, docs/setup version refs, docs/upgrade missing section, 4 skill references). Always assume docs are stale.
|
|
30
|
+
13. **test-coverage script health_suite**: The `bin/test-coverage` script runs a second "health_suite" pass with specific test files. If you add new test files for health/service classes, you MUST add them to the `TARGETED_TESTS` array in `bin/test-coverage` or SimpleCov's merged report will show 0% coverage for those files even though the main suite covers them. In v0.12.0, `source_health_check_orchestrator_test.rb` showed 31% coverage on CI despite 100% locally because it wasn't in the health_suite list.
|
|
30
31
|
|
|
31
32
|
## Step 1: Git Hygiene
|
|
32
33
|
|
|
@@ -114,54 +115,76 @@ The changelog follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) f
|
|
|
114
115
|
- Insert the new versioned entry immediately after the `## [Unreleased]` block and before the previous release entry.
|
|
115
116
|
- Preserve all existing entries below.
|
|
116
117
|
|
|
117
|
-
## Step 4: Documentation
|
|
118
|
+
## Step 4: Documentation Update (Mandatory)
|
|
118
119
|
|
|
119
|
-
|
|
120
|
+
**This step is NON-OPTIONAL.** Every release MUST update all documentation to reflect the current codebase state. Documentation drift is the #1 blind spot in milestone-based work — in v0.9.x, 14 files needed updates; in v0.12.0, 7 files were stale. Always assume docs are stale and verify.
|
|
120
121
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
122
|
+
### 4a. Gather what changed
|
|
123
|
+
|
|
124
|
+
```
|
|
125
|
+
git log vPREVIOUS..HEAD --oneline --no-merges
|
|
126
|
+
git diff vPREVIOUS..HEAD --name-only -- lib/ app/ config/
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Summarize the key changes: new classes, extracted services, new components, new model methods, new migrations, config changes, breaking changes.
|
|
130
|
+
|
|
131
|
+
### 4b. Spawn Explore agent to audit all docs
|
|
132
|
+
|
|
133
|
+
Spawn an Explore agent (subagent_type: "Explore", thoroughness: "very thorough") to compare documentation against the actual source code. The agent MUST check ALL of these files:
|
|
134
|
+
|
|
135
|
+
**Project docs:**
|
|
136
|
+
|
|
137
|
+
| File | What to verify |
|
|
138
|
+
|------|---------------|
|
|
139
|
+
| `README.md` | Version references match new version, feature descriptions current, install instructions accurate |
|
|
140
|
+
| `docs/setup.md` | Version references match, setup steps accurate |
|
|
141
|
+
| `docs/upgrade.md` | Has upgrade section for THIS version with migrations, action items, breaking changes |
|
|
142
|
+
| `docs/configuration.md` | All config options documented, defaults match source code |
|
|
143
|
+
| `docs/deployment.md` | Worker/queue descriptions match current queues and job assignments |
|
|
144
|
+
| `docs/troubleshooting.md` | Covers known failure modes from recent changes |
|
|
145
|
+
|
|
146
|
+
**Skills reference files** (engine-specific documentation for Claude Code):
|
|
147
|
+
|
|
148
|
+
| Skill Reference | What to verify |
|
|
149
|
+
|----------------|---------------|
|
|
150
|
+
| `sm-configure/reference/configuration-reference.md` | All config settings and their defaults |
|
|
151
|
+
| `sm-configuration-setting/reference/settings-catalog.md` | Settings catalog with types, defaults, descriptions |
|
|
152
|
+
| `sm-job/reference/job-conventions.md` | Queue names, job assignments, concurrency defaults, service class delegation |
|
|
153
|
+
| `sm-pipeline-stage/reference/completion-handlers.md` | Pipeline handler code matches actual implementation |
|
|
154
|
+
| `sm-upgrade/reference/version-history.md` | Version transition notes for the new release |
|
|
155
|
+
| `sm-host-setup/reference/initializer-template.md` | Initializer template shows all configurable options |
|
|
156
|
+
| `sm-architecture/reference/module-map.md` | All modules, classes, components, presenters listed |
|
|
157
|
+
| `sm-domain-model/reference/model-graph.md` | All model methods, validations, scopes, associations |
|
|
158
|
+
|
|
159
|
+
The Explore agent should output a structured report: files that are current, files that need updates (with specific stale content), and what each update should contain.
|
|
160
|
+
|
|
161
|
+
### 4c. Spawn Docs agent to fix all stale files
|
|
162
|
+
|
|
163
|
+
Based on the Explore agent's report, spawn a Docs agent (subagent_type: "vbw:vbw-docs", model: "sonnet") to update ALL stale files. The Docs agent MUST:
|
|
164
|
+
|
|
165
|
+
- Read each file before editing (targeted edits, not full rewrites)
|
|
166
|
+
- Update version references to the new release version
|
|
167
|
+
- Add upgrade section for this version (migrations, action items, breaking changes)
|
|
168
|
+
- Update module maps, model graphs, job conventions to reflect current source code
|
|
169
|
+
- Add entries for any new classes, components, concerns, or model methods
|
|
170
|
+
|
|
171
|
+
### 4d. Verify and report
|
|
172
|
+
|
|
173
|
+
After the Docs agent completes, verify all files were updated:
|
|
174
|
+
```
|
|
175
|
+
git diff --name-only # Should show the updated doc files
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Report:
|
|
179
|
+
```
|
|
180
|
+
Documentation Update: Updated N files.
|
|
181
|
+
- <file>: <what was updated>
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
If the Explore agent found 0 stale files, report:
|
|
185
|
+
```
|
|
186
|
+
Documentation Update: All files current. No updates needed.
|
|
187
|
+
```
|
|
165
188
|
|
|
166
189
|
Do NOT commit documentation updates separately -- they will be included in the single release commit in Step 7.
|
|
167
190
|
|
|
@@ -79,6 +79,7 @@ Complete module tree with each module's responsibility.
|
|
|
79
79
|
|--------|------|----------------|
|
|
80
80
|
| `EntryNormalizer` | `import_sessions/entry_normalizer.rb` | Normalize OPML entries to standard format |
|
|
81
81
|
| `HealthCheckBroadcaster` | `import_sessions/health_check_broadcaster.rb` | Broadcast health check progress via Turbo Streams |
|
|
82
|
+
| `HealthCheckUpdater` | `import_sessions/health_check_updater.rb` | Service delegated to by ImportSessionHealthCheckJob; runs checks and broadcasts |
|
|
82
83
|
|
|
83
84
|
### Jobs
|
|
84
85
|
|
|
@@ -104,6 +105,7 @@ Complete module tree with each module's responsibility.
|
|
|
104
105
|
|--------|------|----------------|
|
|
105
106
|
| `Sanitizable` | `models/sanitizable.rb` | `sanitizes_string_attributes`, `sanitizes_hash_attributes` class methods |
|
|
106
107
|
| `UrlNormalizable` | `models/url_normalizable.rb` | `normalizes_urls`, `validates_url_format` class methods |
|
|
108
|
+
| `SetSource` | `models/set_source.rb` | Controller concern: resolves and assigns `@source` from params before actions |
|
|
107
109
|
|
|
108
110
|
### Scrapers (Scraper Adapters)
|
|
109
111
|
|
|
@@ -123,6 +125,7 @@ Complete module tree with each module's responsibility.
|
|
|
123
125
|
| `ItemScraper` | `scraping/item_scraper.rb` | Scrape a single item |
|
|
124
126
|
| `ItemScraper::AdapterResolver` | `scraping/item_scraper/adapter_resolver.rb` | Select scraper adapter for a source |
|
|
125
127
|
| `ItemScraper::Persistence` | `scraping/item_scraper/persistence.rb` | Save scrape results to ItemContent |
|
|
128
|
+
| `Runner` | `scraping/runner.rb` | Service delegated to by ScrapeItemJob; orchestrates state, scraping, and logging |
|
|
126
129
|
| `BulkSourceScraper` | `scraping/bulk_source_scraper.rb` | Scrape all pending items for a source |
|
|
127
130
|
| `BulkResultPresenter` | `scraping/bulk_result_presenter.rb` | Format bulk scrape results |
|
|
128
131
|
| `State` | `scraping/state.rb` | Track scraping state per source |
|
|
@@ -150,6 +153,7 @@ Complete module tree with each module's responsibility.
|
|
|
150
153
|
|--------|------|----------------|
|
|
151
154
|
| `SourceHealthMonitor` | `health/source_health_monitor.rb` | Calculate rolling success rate, update health_status |
|
|
152
155
|
| `SourceHealthCheck` | `health/source_health_check.rb` | Perform HTTP health check on a source |
|
|
156
|
+
| `SourceHealthCheckOrchestrator` | `health/source_health_check_orchestrator.rb` | Service delegated to by SourceHealthCheckJob; coordinates check, logging, and broadcasting |
|
|
153
157
|
| `SourceHealthReset` | `health/source_health_reset.rb` | Reset health state for a source |
|
|
154
158
|
| `ImportSourceHealthCheck` | `health/import_source_health_check.rb` | Health check for import session sources |
|
|
155
159
|
|
|
@@ -182,6 +186,34 @@ Complete module tree with each module's responsibility.
|
|
|
182
186
|
| `Verification::ActionCableVerifier` | `setup/verification/action_cable_verifier.rb` | Verify Action Cable setup |
|
|
183
187
|
| `Verification::TelemetryLogger` | `setup/verification/telemetry_logger.rb` | Log setup telemetry |
|
|
184
188
|
|
|
189
|
+
### Favicons
|
|
190
|
+
|
|
191
|
+
| Module | File | Responsibility |
|
|
192
|
+
|--------|------|----------------|
|
|
193
|
+
| `Favicons::Discoverer` | `favicons/discoverer.rb` | Multi-strategy favicon discovery (direct, HTML parsing, Google API) |
|
|
194
|
+
| `Favicons::Fetcher` | `favicons/fetcher.rb` | Service delegated to by FaviconFetchJob; guards, discovers, and attaches favicon |
|
|
195
|
+
|
|
196
|
+
### Images
|
|
197
|
+
|
|
198
|
+
| Module | File | Responsibility |
|
|
199
|
+
|--------|------|----------------|
|
|
200
|
+
| `Images::Processor` | `images/processor.rb` | Service delegated to by DownloadContentImagesJob; downloads and attaches content images via Active Storage |
|
|
201
|
+
|
|
202
|
+
### ViewComponents
|
|
203
|
+
|
|
204
|
+
| Component | File | Responsibility |
|
|
205
|
+
|-----------|------|----------------|
|
|
206
|
+
| `StatusBadgeComponent` | `app/components/source_monitor/status_badge_component.rb` | Renders a colored status badge for health_status values |
|
|
207
|
+
| `IconComponent` | `app/components/source_monitor/icon_component.rb` | Renders consistent inline SVG or CSS icons |
|
|
208
|
+
| `FilterDropdownComponent` | `app/components/source_monitor/filter_dropdown_component.rb` | Renders filter dropdown menus for sources/items index |
|
|
209
|
+
|
|
210
|
+
### Presenters
|
|
211
|
+
|
|
212
|
+
| Presenter | File | Responsibility |
|
|
213
|
+
|-----------|------|----------------|
|
|
214
|
+
| `SourceDetailsPresenter` | `app/presenters/source_monitor/source_details_presenter.rb` | View-specific formatting for source detail pages (SimpleDelegator over Source) |
|
|
215
|
+
| `SourcesFilterPresenter` | `app/presenters/source_monitor/sources_filter_presenter.rb` | Formats filter state and option lists for the sources index |
|
|
216
|
+
|
|
185
217
|
### Other
|
|
186
218
|
|
|
187
219
|
| Module | File | Responsibility |
|
|
@@ -112,4 +112,45 @@ Source has a `type` column for potential STI subclassing but it is not actively
|
|
|
112
112
|
|--------|--------|-------|-------|
|
|
113
113
|
| Source | `items_count` | Item | Only counts active (non-deleted) items |
|
|
114
114
|
|
|
115
|
-
The counter is decremented manually during `soft_delete!` and
|
|
115
|
+
The counter is decremented manually during `soft_delete!` and **incremented** during `restore!`. Both operations update `items_count` atomically. Use `reset_items_counter!` to recalculate from scratch if the counter drifts.
|
|
116
|
+
|
|
117
|
+
## Model Methods (v0.12.0+)
|
|
118
|
+
|
|
119
|
+
### Source.enable_scraping!(ids)
|
|
120
|
+
|
|
121
|
+
Bulk-enables scraping for a list of source IDs:
|
|
122
|
+
|
|
123
|
+
```ruby
|
|
124
|
+
Source.enable_scraping!([1, 2, 3])
|
|
125
|
+
# Sets scraping_enabled = true for each source in the list
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Use case: enabling scraping on a filtered set of sources from the dashboard or a rake task.
|
|
129
|
+
|
|
130
|
+
### Item#restore!
|
|
131
|
+
|
|
132
|
+
Reverses a soft delete. Symmetric counterpart to `soft_delete!`:
|
|
133
|
+
|
|
134
|
+
```ruby
|
|
135
|
+
item.restore!
|
|
136
|
+
# Clears deleted_at, increments source.items_count
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
After `restore!`, the item re-enters the `:items` (active) association and the source counter cache reflects the change.
|
|
140
|
+
|
|
141
|
+
### health_status Validation
|
|
142
|
+
|
|
143
|
+
As of v0.12.0, `health_status` is validated against the four permitted values. Assigning an unrecognized value raises a validation error:
|
|
144
|
+
|
|
145
|
+
```ruby
|
|
146
|
+
source.health_status = "healthy" # invalid -- was removed in 0.11.0
|
|
147
|
+
source.valid? # => false
|
|
148
|
+
source.errors[:health_status] # => ["is not included in the list"]
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Permitted values: `"working"`, `"declining"`, `"improving"`, `"failing"`.
|
|
152
|
+
|
|
153
|
+
## Schema Notes (v0.12.0)
|
|
154
|
+
|
|
155
|
+
- **Composite indexes on log tables:** `sourcemon_fetch_logs`, `sourcemon_scrape_logs`, and `sourcemon_health_check_logs` now have composite indexes on `(source_id, created_at)` to improve log query performance on busy sources.
|
|
156
|
+
- **health_status default alignment:** The `health_status` column default on `sourcemon_sources` was updated to `"working"` (previously `"healthy"`, a removed value) to match the four-value enum added in v0.11.0.
|
|
@@ -113,7 +113,7 @@ Notable patterns:
|
|
|
113
113
|
|
|
114
114
|
### Worker Job (ScrapeItemJob)
|
|
115
115
|
|
|
116
|
-
Demonstrates
|
|
116
|
+
Demonstrates shallow delegation to a service class:
|
|
117
117
|
|
|
118
118
|
```ruby
|
|
119
119
|
class ScrapeItemJob < ApplicationJob
|
|
@@ -121,35 +121,18 @@ class ScrapeItemJob < ApplicationJob
|
|
|
121
121
|
discard_on ActiveJob::DeserializationError
|
|
122
122
|
|
|
123
123
|
def perform(item_id)
|
|
124
|
-
log("job:start", item_id: item_id)
|
|
125
124
|
item = Item.includes(:source).find_by(id: item_id)
|
|
126
125
|
return unless item
|
|
127
126
|
|
|
128
|
-
|
|
129
|
-
unless source&.scraping_enabled?
|
|
130
|
-
log("job:skipped_scraping_disabled", item: item)
|
|
131
|
-
Scraping::State.clear_inflight!(item)
|
|
132
|
-
return
|
|
133
|
-
end
|
|
134
|
-
|
|
135
|
-
Scraping::State.mark_processing!(item)
|
|
136
|
-
Scraping::ItemScraper.new(item:, source:).call
|
|
137
|
-
log("job:completed", item: item, status: item.scrape_status)
|
|
138
|
-
rescue StandardError => error
|
|
139
|
-
log("job:error", item: item, error: error.message)
|
|
140
|
-
Scraping::State.mark_failed!(item)
|
|
141
|
-
raise
|
|
142
|
-
ensure
|
|
143
|
-
Scraping::State.clear_inflight!(item) if item
|
|
127
|
+
Scraping::Runner.new(item: item).call
|
|
144
128
|
end
|
|
145
129
|
end
|
|
146
130
|
```
|
|
147
131
|
|
|
148
132
|
Notable patterns:
|
|
149
|
-
-
|
|
150
|
-
-
|
|
151
|
-
-
|
|
152
|
-
- Structured JSON logging at each stage
|
|
133
|
+
- Job body is deserialization + delegation only
|
|
134
|
+
- All business logic (state management, scraping, logging) lives in `Scraping::Runner`
|
|
135
|
+
- `includes(:source)` prevents N+1 query before handing off to the service
|
|
153
136
|
|
|
154
137
|
### Scheduling Job (ScheduleFetchesJob)
|
|
155
138
|
|
|
@@ -168,7 +151,7 @@ end
|
|
|
168
151
|
|
|
169
152
|
### Lightweight Fetch Job (FaviconFetchJob)
|
|
170
153
|
|
|
171
|
-
Demonstrates
|
|
154
|
+
Demonstrates shallow delegation with guard clause:
|
|
172
155
|
|
|
173
156
|
```ruby
|
|
174
157
|
class FaviconFetchJob < ApplicationJob
|
|
@@ -178,23 +161,19 @@ class FaviconFetchJob < ApplicationJob
|
|
|
178
161
|
def perform(source_id)
|
|
179
162
|
source = Source.find_by(id: source_id)
|
|
180
163
|
return unless source
|
|
181
|
-
return unless should_fetch?(source)
|
|
182
164
|
|
|
183
|
-
|
|
184
|
-
attach_favicon(source, result) if result.success?
|
|
165
|
+
Favicons::Fetcher.new(source: source).call
|
|
185
166
|
end
|
|
186
167
|
end
|
|
187
168
|
```
|
|
188
169
|
|
|
189
170
|
Notable patterns:
|
|
190
|
-
-
|
|
191
|
-
- Uses `Favicons::Discoverer` service with 3-strategy cascade (direct `/favicon.ico`, HTML parsing, Google API)
|
|
192
|
-
- Failed attempts tracked in source `metadata` JSONB (`favicon_last_attempted_at`) for retry cooldown
|
|
171
|
+
- Job contains only lookup + delegation — guard clauses and discovery strategy cascade live in `Favicons::Fetcher`
|
|
193
172
|
- Graceful degradation: host apps without Active Storage never enqueue this job
|
|
194
173
|
|
|
195
174
|
### Broadcast Job (SourceHealthCheckJob)
|
|
196
175
|
|
|
197
|
-
Demonstrates result broadcasting:
|
|
176
|
+
Demonstrates shallow delegation with result broadcasting:
|
|
198
177
|
|
|
199
178
|
```ruby
|
|
200
179
|
class SourceHealthCheckJob < ApplicationJob
|
|
@@ -205,21 +184,48 @@ class SourceHealthCheckJob < ApplicationJob
|
|
|
205
184
|
source = Source.find_by(id: source_id)
|
|
206
185
|
return unless source
|
|
207
186
|
|
|
208
|
-
|
|
209
|
-
broadcast_outcome(source, result)
|
|
210
|
-
result
|
|
211
|
-
rescue StandardError => error
|
|
212
|
-
record_unexpected_failure(source, error) if source
|
|
213
|
-
broadcast_outcome(source, nil, error) if source
|
|
214
|
-
nil
|
|
187
|
+
Health::SourceHealthCheckOrchestrator.new(source: source).call
|
|
215
188
|
end
|
|
216
189
|
end
|
|
217
190
|
```
|
|
218
191
|
|
|
219
192
|
Notable patterns:
|
|
220
|
-
-
|
|
221
|
-
-
|
|
222
|
-
- Returns nil on
|
|
193
|
+
- Job body is lookup + delegation only
|
|
194
|
+
- Broadcasting, logging, and error handling live in `Health::SourceHealthCheckOrchestrator`
|
|
195
|
+
- Returns nil on missing source; orchestrator handles nil-on-error (health checks are non-critical)
|
|
196
|
+
|
|
197
|
+
## Shallow Delegation Pattern
|
|
198
|
+
|
|
199
|
+
As of v0.12.0, five maintenance jobs delegate entirely to dedicated service classes. Jobs contain only deserialization and delegation — no business logic.
|
|
200
|
+
|
|
201
|
+
| Job | Service Class |
|
|
202
|
+
|-----|---------------|
|
|
203
|
+
| `ScrapeItemJob` | `Scraping::Runner` |
|
|
204
|
+
| `DownloadContentImagesJob` | `Images::Processor` |
|
|
205
|
+
| `FaviconFetchJob` | `Favicons::Fetcher` |
|
|
206
|
+
| `SourceHealthCheckJob` | `Health::SourceHealthCheckOrchestrator` |
|
|
207
|
+
| `ImportSessionHealthCheckJob` | `ImportSessions::HealthCheckUpdater` |
|
|
208
|
+
|
|
209
|
+
**Why this pattern:**
|
|
210
|
+
- Jobs are the transport mechanism, not the behavior container.
|
|
211
|
+
- Service classes are unit-testable without ActiveJob infrastructure.
|
|
212
|
+
- Future pipeline changes (e.g., calling the same logic synchronously) require no job changes.
|
|
213
|
+
|
|
214
|
+
**Template for new jobs:**
|
|
215
|
+
|
|
216
|
+
```ruby
|
|
217
|
+
class MyJob < ApplicationJob
|
|
218
|
+
source_monitor_queue :maintenance
|
|
219
|
+
discard_on ActiveJob::DeserializationError
|
|
220
|
+
|
|
221
|
+
def perform(record_id)
|
|
222
|
+
record = MyModel.find_by(id: record_id)
|
|
223
|
+
return unless record
|
|
224
|
+
|
|
225
|
+
MyNamespace::MyService.new(record: record).call
|
|
226
|
+
end
|
|
227
|
+
end
|
|
228
|
+
```
|
|
223
229
|
|
|
224
230
|
## _later / _now Naming Convention
|
|
225
231
|
|
|
@@ -2,6 +2,25 @@
|
|
|
2
2
|
|
|
3
3
|
Version-specific migration notes for each major/minor version transition. Agents should reference this file when guiding users through multi-version upgrades.
|
|
4
4
|
|
|
5
|
+
## 0.11.x to 0.12.0
|
|
6
|
+
|
|
7
|
+
**Key changes:**
|
|
8
|
+
- 2 new migrations: composite indexes on `sourcemon_fetch_logs`, `sourcemon_scrape_logs`, and `sourcemon_health_check_logs` (on `source_id, created_at`), and `health_status` column default corrected to `"working"`.
|
|
9
|
+
- 5 background jobs extracted to service classes: `ScrapeItemJob` -> `Scraping::Runner`, `DownloadContentImagesJob` -> `Images::Processor`, `FaviconFetchJob` -> `Favicons::Fetcher`, `SourceHealthCheckJob` -> `Health::SourceHealthCheckOrchestrator`, `ImportSessionHealthCheckJob` -> `ImportSessions::HealthCheckUpdater`. Job arguments and queue assignments are unchanged.
|
|
10
|
+
- New ViewComponents: `StatusBadgeComponent`, `IconComponent`, `FilterDropdownComponent`.
|
|
11
|
+
- New presenters: `SourceDetailsPresenter`, `SourcesFilterPresenter`.
|
|
12
|
+
- New model methods: `Source.enable_scraping!(ids)`, `Item#restore!`, `health_status` validation against the 4 permitted values.
|
|
13
|
+
|
|
14
|
+
**Action items:**
|
|
15
|
+
1. Copy and run migrations:
|
|
16
|
+
```bash
|
|
17
|
+
bin/rails source_monitor:install:migrations
|
|
18
|
+
bin/rails db:migrate
|
|
19
|
+
```
|
|
20
|
+
2. No breaking changes -- all existing initializer configuration and job interfaces remain valid.
|
|
21
|
+
3. No configuration changes required.
|
|
22
|
+
4. ViewComponents and presenters are available for use in custom views but are not required.
|
|
23
|
+
|
|
5
24
|
## 0.10.2 to 0.11.0
|
|
6
25
|
|
|
7
26
|
**Key changes:**
|
data/CHANGELOG.md
CHANGED
|
@@ -15,6 +15,13 @@ All notable changes to this project are documented below. The format follows [Ke
|
|
|
15
15
|
|
|
16
16
|
- No unreleased changes yet.
|
|
17
17
|
|
|
18
|
+
## [0.12.1] - 2026-03-15
|
|
19
|
+
|
|
20
|
+
### Documentation
|
|
21
|
+
- Updated README, docs/setup, docs/upgrade with v0.12.0 version references and upgrade guide
|
|
22
|
+
- Updated 4 skill references (sm-job, sm-architecture, sm-domain-model, sm-upgrade) for service class extractions, new components, and model methods
|
|
23
|
+
- Improved `/release` command: Step 4 now spawns agents to automatically audit AND fix all stale docs before every release
|
|
24
|
+
|
|
18
25
|
## [0.12.0] - 2026-03-15
|
|
19
26
|
|
|
20
27
|
### Added
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
|
@@ -9,8 +9,8 @@ SourceMonitor is a production-ready Rails 8 mountable engine for ingesting, norm
|
|
|
9
9
|
In your host Rails app:
|
|
10
10
|
|
|
11
11
|
```bash
|
|
12
|
-
bundle add source_monitor --version "~> 0.
|
|
13
|
-
# or add `gem "source_monitor", "~> 0.
|
|
12
|
+
bundle add source_monitor --version "~> 0.12.0"
|
|
13
|
+
# or add `gem "source_monitor", "~> 0.12.0"` manually, then run:
|
|
14
14
|
bundle install
|
|
15
15
|
```
|
|
16
16
|
|
|
@@ -25,6 +25,9 @@ This exposes `bin/source_monitor` (via Bundler binstubs) so you can run the guid
|
|
|
25
25
|
- Extensible scraper adapters (Readability included) with per-source settings and structured result metadata
|
|
26
26
|
- Declarative configuration DSL covering queues, HTTP, retention, events, model extensions, authentication, and realtime transports
|
|
27
27
|
- First-class observability through ActiveSupport notifications and `SourceMonitor::Metrics` counters/gauges
|
|
28
|
+
- ViewComponent UI primitives: `StatusBadgeComponent` and `IconComponent` for consistent badge and icon rendering in custom views
|
|
29
|
+
- Presenter layer: `SourceDetailsPresenter` and `SourcesFilterPresenter` for view-specific formatting without coupling controllers to display logic
|
|
30
|
+
- Shallow delegation service layer: five background jobs (ScrapeItemJob, DownloadContentImagesJob, FaviconFetchJob, SourceHealthCheckJob, ImportSessionHealthCheckJob) extracted to dedicated service classes, keeping job bodies to deserialization + delegation only
|
|
28
31
|
|
|
29
32
|
## Requirements
|
|
30
33
|
- Ruby 4.0+ (we recommend [rbenv](https://github.com/rbenv/rbenv) for local development, but use whatever Ruby version manager suits your environment—asdf, chruby, rvm, or container-based workflows all work fine)
|
|
@@ -43,7 +46,7 @@ This exposes `bin/source_monitor` (via Bundler binstubs) so you can run the guid
|
|
|
43
46
|
Before running any SourceMonitor commands inside your host app, add the gem and install dependencies:
|
|
44
47
|
|
|
45
48
|
```bash
|
|
46
|
-
bundle add source_monitor --version "~> 0.
|
|
49
|
+
bundle add source_monitor --version "~> 0.12.0"
|
|
47
50
|
# or edit your Gemfile, then run
|
|
48
51
|
bundle install
|
|
49
52
|
```
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.12.
|
|
1
|
+
0.12.1
|
data/docs/setup.md
CHANGED
|
@@ -18,8 +18,8 @@ This guide consolidates the new guided installer, verification commands, and rol
|
|
|
18
18
|
Run these commands inside your host Rails application before invoking the guided workflow:
|
|
19
19
|
|
|
20
20
|
```bash
|
|
21
|
-
bundle add source_monitor --version "~> 0.
|
|
22
|
-
# or add gem "source_monitor", "~> 0.
|
|
21
|
+
bundle add source_monitor --version "~> 0.12.0"
|
|
22
|
+
# or add gem "source_monitor", "~> 0.12.0" to Gemfile manually
|
|
23
23
|
bundle install
|
|
24
24
|
```
|
|
25
25
|
|
data/docs/upgrade.md
CHANGED
|
@@ -46,6 +46,29 @@ If a removed option raises an error (`SourceMonitor::DeprecatedOptionError`), yo
|
|
|
46
46
|
|
|
47
47
|
## Version-Specific Notes
|
|
48
48
|
|
|
49
|
+
### Upgrading to 0.12.0
|
|
50
|
+
|
|
51
|
+
**What changed:**
|
|
52
|
+
- 2 new migrations: composite indexes on log tables and `health_status` default value alignment.
|
|
53
|
+
- 5 background jobs extracted to service classes (`Scraping::Runner`, `Images::Processor`, `Favicons::Fetcher`, `Health::SourceHealthCheckOrchestrator`, `ImportSessions::HealthCheckUpdater`). Jobs now contain only deserialization and delegation — business logic lives in `lib/`.
|
|
54
|
+
- New ViewComponents: `StatusBadgeComponent` and `IconComponent` for consistent badge and icon rendering.
|
|
55
|
+
- New presenters: `SourceDetailsPresenter` and `SourcesFilterPresenter` for view-specific formatting.
|
|
56
|
+
- New model methods: `Source.enable_scraping!(ids)` bulk-enables scraping for a list of source IDs; `Item#restore!` reverses a soft delete and updates the source counter cache; `health_status` now validated against the 4 permitted values (`working`, `declining`, `improving`, `failing`).
|
|
57
|
+
|
|
58
|
+
**Upgrade steps:**
|
|
59
|
+
```bash
|
|
60
|
+
bundle update source_monitor
|
|
61
|
+
bin/rails source_monitor:install:migrations
|
|
62
|
+
bin/rails db:migrate
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
**Notes:**
|
|
66
|
+
- No breaking changes. All existing initializer configuration remains valid.
|
|
67
|
+
- No configuration changes required for this release.
|
|
68
|
+
- The 5 service extractions are internal. Job class names and their public interfaces (queue, arguments) are unchanged.
|
|
69
|
+
- New ViewComponents and presenters are available for custom view integration but are not required by default templates.
|
|
70
|
+
- `Item#restore!` is the symmetric counterpart to `soft_delete!` — it clears `deleted_at` and increments the source `items_count` counter cache.
|
|
71
|
+
|
|
49
72
|
### Upgrading to 0.11.0
|
|
50
73
|
|
|
51
74
|
**What changed:**
|