source_monitor 0.12.3 → 0.12.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.claude/skills/sm-job/reference/job-conventions.md +3 -2
- data/.claude/skills/sm-upgrade/reference/version-history.md +13 -0
- data/CHANGELOG.md +9 -0
- data/Gemfile.lock +1 -1
- data/README.md +3 -3
- data/VERSION +1 -1
- data/app/controllers/source_monitor/dashboard_controller.rb +1 -1
- data/app/controllers/source_monitor/import_history_dismissals_controller.rb +5 -2
- data/app/jobs/source_monitor/fetch_feed_job.rb +24 -3
- data/app/views/source_monitor/logs/index.html.erb +1 -1
- data/app/views/source_monitor/sources/index.html.erb +1 -1
- data/docs/setup.md +2 -2
- data/docs/upgrade.md +18 -0
- data/lib/source_monitor/scraping/state.rb +2 -2
- data/lib/source_monitor/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c9f29bb075dfefc28a290e718ca9e983b489e7b62e5c8654778bb3c4c0c8df91
|
|
4
|
+
data.tar.gz: e404576eb8a66613e653257216039a4b67eef5b1d8614bc22742396b0b5014cb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b91d1d3f7d0b9f5f12612e8764b423b2be5a164aa3a52abe7c3ce48e587891b2f45b37ed281a5dd70e14ca3735c0abc2c89427c1f450900f9fa3650a052b08f1
|
|
7
|
+
data.tar.gz: cbe4936aa544c0a345b803ef8ffb11ff69b3237676f4ce34cdddf06259855c0150569cef5ca400f939044387466f36ba5bc0c41b18d712ebd4d7e54afd95cbd4
|
|
@@ -58,13 +58,14 @@ The most complex job, demonstrating retry strategy integration:
|
|
|
58
58
|
|
|
59
59
|
```ruby
|
|
60
60
|
class FetchFeedJob < ApplicationJob
|
|
61
|
-
|
|
61
|
+
FETCH_CONCURRENCY_BASE_WAIT = 30.seconds
|
|
62
|
+
FETCH_CONCURRENCY_MAX_WAIT = 5.minutes
|
|
62
63
|
EARLY_EXECUTION_LEEWAY = 30.seconds
|
|
63
64
|
|
|
64
65
|
source_monitor_queue :fetch
|
|
65
66
|
|
|
66
67
|
discard_on ActiveJob::DeserializationError
|
|
67
|
-
|
|
68
|
+
# ConcurrencyError: exponential backoff (30s * 2^attempt) with 25% jitter, discards after 5 attempts
|
|
68
69
|
|
|
69
70
|
def perform(source_id, force: false)
|
|
70
71
|
source = Source.find_by(id: source_id)
|
|
@@ -2,6 +2,19 @@
|
|
|
2
2
|
|
|
3
3
|
Version-specific migration notes for each major/minor version transition. Agents should reference this file when guiding users through multi-version upgrades.
|
|
4
4
|
|
|
5
|
+
## 0.12.3 to 0.12.4
|
|
6
|
+
|
|
7
|
+
**Key changes:**
|
|
8
|
+
- Bug fix: ScrapeItemJob with_lock compatibility (assign_attributes → reload)
|
|
9
|
+
- Bug fix: FetchFeedJob advisory lock exponential backoff + graceful discard
|
|
10
|
+
- Bug fix: OPML import dismissal (all instead of latest only)
|
|
11
|
+
- Bug fix: Dashboard pagination regex for group keys
|
|
12
|
+
- Bug fix: Filter dropdown Stimulus controller declaration
|
|
13
|
+
|
|
14
|
+
**Action items:**
|
|
15
|
+
1. `bundle update source_monitor`
|
|
16
|
+
2. No migrations, config changes, or breaking changes.
|
|
17
|
+
|
|
5
18
|
## 0.12.2 to 0.12.3
|
|
6
19
|
|
|
7
20
|
**Key changes:**
|
data/CHANGELOG.md
CHANGED
|
@@ -15,6 +15,15 @@ All notable changes to this project are documented below. The format follows [Ke
|
|
|
15
15
|
|
|
16
16
|
- No unreleased changes yet.
|
|
17
17
|
|
|
18
|
+
## [0.12.4] - 2026-03-17
|
|
19
|
+
|
|
20
|
+
### Fixed
|
|
21
|
+
- ScrapeItemJob failing with "Locking a record with unpersisted changes" on Rails 8.1.2 — replaced `assign_attributes` with `reload` in scraping state machine
|
|
22
|
+
- FetchFeedJob permanently failing on advisory lock contention — switched to exponential backoff with jitter and graceful discard on exhaustion
|
|
23
|
+
- OPML import dismissal only hiding the latest notification — now dismisses all undismissed import histories for the user
|
|
24
|
+
- Dashboard pagination not working — schedule group keys were silently dropped by overly restrictive param whitelist regex
|
|
25
|
+
- Source filter dropdowns not auto-submitting — missing `filter-submit` Stimulus controller declaration on search forms (also fixed on logs index)
|
|
26
|
+
|
|
18
27
|
## [0.12.3] - 2026-03-16
|
|
19
28
|
|
|
20
29
|
### Added
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
|
@@ -9,8 +9,8 @@ SourceMonitor is a production-ready Rails 8 mountable engine for ingesting, norm
|
|
|
9
9
|
In your host Rails app:
|
|
10
10
|
|
|
11
11
|
```bash
|
|
12
|
-
bundle add source_monitor --version "~> 0.12.
|
|
13
|
-
# or add `gem "source_monitor", "~> 0.12.
|
|
12
|
+
bundle add source_monitor --version "~> 0.12.4"
|
|
13
|
+
# or add `gem "source_monitor", "~> 0.12.4"` manually, then run:
|
|
14
14
|
bundle install
|
|
15
15
|
```
|
|
16
16
|
|
|
@@ -46,7 +46,7 @@ This exposes `bin/source_monitor` (via Bundler binstubs) so you can run the guid
|
|
|
46
46
|
Before running any SourceMonitor commands inside your host app, add the gem and install dependencies:
|
|
47
47
|
|
|
48
48
|
```bash
|
|
49
|
-
bundle add source_monitor --version "~> 0.12.
|
|
49
|
+
bundle add source_monitor --version "~> 0.12.4"
|
|
50
50
|
# or edit your Gemfile, then run
|
|
51
51
|
bundle install
|
|
52
52
|
```
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.12.
|
|
1
|
+
0.12.4
|
|
@@ -33,7 +33,7 @@ module SourceMonitor
|
|
|
33
33
|
raw = params.fetch(:schedule_pages, {})
|
|
34
34
|
return {} unless raw.respond_to?(:permit)
|
|
35
35
|
|
|
36
|
-
permitted_keys = raw.keys.select { |k| k.to_s.match?(/\
|
|
36
|
+
permitted_keys = raw.keys.select { |k| k.to_s.match?(/\A[\d+\-]+\z/) }
|
|
37
37
|
raw.permit(*permitted_keys).to_h
|
|
38
38
|
end
|
|
39
39
|
end
|
|
@@ -3,8 +3,11 @@
|
|
|
3
3
|
module SourceMonitor
|
|
4
4
|
class ImportHistoryDismissalsController < ApplicationController
|
|
5
5
|
def create
|
|
6
|
-
|
|
7
|
-
|
|
6
|
+
user_id = source_monitor_current_user&.id
|
|
7
|
+
# Verify the specified import history belongs to this user (authorization check)
|
|
8
|
+
ImportHistory.where(user_id: user_id).find(params[:import_history_id])
|
|
9
|
+
# Dismiss all undismissed import histories for this user so older ones don't resurface
|
|
10
|
+
ImportHistory.where(user_id: user_id).not_dismissed.update_all(dismissed_at: Time.current)
|
|
8
11
|
|
|
9
12
|
respond_to do |format|
|
|
10
13
|
format.turbo_stream do
|
|
@@ -2,7 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
module SourceMonitor
|
|
4
4
|
class FetchFeedJob < ApplicationJob
|
|
5
|
-
|
|
5
|
+
FETCH_CONCURRENCY_BASE_WAIT = 30.seconds
|
|
6
|
+
FETCH_CONCURRENCY_MAX_WAIT = 5.minutes
|
|
6
7
|
EARLY_EXECUTION_LEEWAY = 30.seconds
|
|
7
8
|
|
|
8
9
|
source_monitor_queue :fetch
|
|
@@ -39,13 +40,33 @@ module SourceMonitor
|
|
|
39
40
|
else
|
|
40
41
|
attempt = executions
|
|
41
42
|
if attempt < SCHEDULED_CONCURRENCY_MAX_ATTEMPTS
|
|
42
|
-
retry_job wait:
|
|
43
|
+
retry_job wait: concurrency_backoff_wait(attempt)
|
|
43
44
|
else
|
|
44
|
-
|
|
45
|
+
log_concurrency_exhausted
|
|
46
|
+
@source&.update_columns(fetch_status: "idle") if @source&.fetch_status == "queued"
|
|
45
47
|
end
|
|
46
48
|
end
|
|
47
49
|
end
|
|
48
50
|
|
|
51
|
+
def concurrency_backoff_wait(attempt)
|
|
52
|
+
base = FETCH_CONCURRENCY_BASE_WAIT.to_i
|
|
53
|
+
exponential = base * (2**attempt)
|
|
54
|
+
capped = [ exponential, FETCH_CONCURRENCY_MAX_WAIT.to_i ].min
|
|
55
|
+
jitter = rand(0..(capped * 0.25).to_i)
|
|
56
|
+
(capped + jitter).seconds
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def log_concurrency_exhausted
|
|
60
|
+
return unless defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger
|
|
61
|
+
|
|
62
|
+
Rails.logger.info(
|
|
63
|
+
"[SourceMonitor::FetchFeedJob] Concurrency retries exhausted for source #{@source_id}, " \
|
|
64
|
+
"discarding (another worker is fetching this source)"
|
|
65
|
+
)
|
|
66
|
+
rescue StandardError
|
|
67
|
+
nil
|
|
68
|
+
end
|
|
69
|
+
|
|
49
70
|
def log_force_fetch_skipped
|
|
50
71
|
return unless defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger
|
|
51
72
|
|
|
@@ -52,7 +52,7 @@
|
|
|
52
52
|
</div>
|
|
53
53
|
</div>
|
|
54
54
|
|
|
55
|
-
<%= form_with url: source_monitor.logs_path, method: :get, html: { class: "rounded-lg border border-slate-200 bg-white p-4 shadow-sm", data: { turbo_frame: "source_monitor_logs" } } do |form| %>
|
|
55
|
+
<%= form_with url: source_monitor.logs_path, method: :get, html: { class: "rounded-lg border border-slate-200 bg-white p-4 shadow-sm", data: { turbo_frame: "source_monitor_logs", controller: "filter-submit" } } do |form| %>
|
|
56
56
|
<%= form.hidden_field :status, value: @filter_set.status %>
|
|
57
57
|
<%= form.hidden_field :log_type, value: @filter_set.log_type %>
|
|
58
58
|
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
selected_bucket: @selected_fetch_interval_bucket,
|
|
20
20
|
search_params: @search_params %>
|
|
21
21
|
|
|
22
|
-
<%= search_form_for @q, url: source_monitor.sources_path, method: :get, html: { class: "flex flex-wrap items-end gap-3", data: { turbo_frame: "source_monitor_sources_table" } } do |form| %>
|
|
22
|
+
<%= search_form_for @q, url: source_monitor.sources_path, method: :get, html: { class: "flex flex-wrap items-end gap-3", data: { turbo_frame: "source_monitor_sources_table", controller: "filter-submit" } } do |form| %>
|
|
23
23
|
<div class="flex-1 min-w-[12rem]">
|
|
24
24
|
<%= form.label @search_field, "Search sources", class: "sr-only" %>
|
|
25
25
|
<div class="flex rounded-md shadow-sm">
|
data/docs/setup.md
CHANGED
|
@@ -18,8 +18,8 @@ This guide consolidates the new guided installer, verification commands, and rol
|
|
|
18
18
|
Run these commands inside your host Rails application before invoking the guided workflow:
|
|
19
19
|
|
|
20
20
|
```bash
|
|
21
|
-
bundle add source_monitor --version "~> 0.12.
|
|
22
|
-
# or add gem "source_monitor", "~> 0.12.
|
|
21
|
+
bundle add source_monitor --version "~> 0.12.4"
|
|
22
|
+
# or add gem "source_monitor", "~> 0.12.4" to Gemfile manually
|
|
23
23
|
bundle install
|
|
24
24
|
```
|
|
25
25
|
|
data/docs/upgrade.md
CHANGED
|
@@ -69,6 +69,24 @@ bin/rails db:migrate
|
|
|
69
69
|
- New ViewComponents and presenters are available for custom view integration but are not required by default templates.
|
|
70
70
|
- `Item#restore!` is the symmetric counterpart to `soft_delete!` — it clears `deleted_at` and increments the source `items_count` counter cache.
|
|
71
71
|
|
|
72
|
+
### Upgrading to 0.12.4
|
|
73
|
+
|
|
74
|
+
**What changed:**
|
|
75
|
+
- Bug fix: ScrapeItemJob Rails 8.1.2 with_lock compatibility (assign_attributes → reload)
|
|
76
|
+
- Bug fix: FetchFeedJob exponential backoff for advisory lock contention
|
|
77
|
+
- Bug fix: OPML import dismissal now dismisses all undismissed histories
|
|
78
|
+
- Bug fix: Dashboard pagination regex for schedule group keys
|
|
79
|
+
- Bug fix: Source/logs filter dropdowns auto-submit via Stimulus controller
|
|
80
|
+
|
|
81
|
+
**Upgrade steps:**
|
|
82
|
+
```bash
|
|
83
|
+
bundle update source_monitor
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
**Notes:**
|
|
87
|
+
- No breaking changes, migrations, or configuration changes required.
|
|
88
|
+
- Patch fix release.
|
|
89
|
+
|
|
72
90
|
### Upgrading to 0.12.3
|
|
73
91
|
|
|
74
92
|
**What changed:**
|
|
@@ -32,7 +32,7 @@ module SourceMonitor
|
|
|
32
32
|
next unless in_flight?(record.scrape_status)
|
|
33
33
|
|
|
34
34
|
record.update_columns(scrape_status: nil)
|
|
35
|
-
record.
|
|
35
|
+
record.reload
|
|
36
36
|
end
|
|
37
37
|
|
|
38
38
|
broadcast_item(item) if broadcast
|
|
@@ -48,7 +48,7 @@ module SourceMonitor
|
|
|
48
48
|
with_item(item, lock:) do |record|
|
|
49
49
|
attributes = { scrape_status: status }.merge(extra.compact)
|
|
50
50
|
record.update_columns(attributes)
|
|
51
|
-
record.
|
|
51
|
+
record.reload
|
|
52
52
|
end
|
|
53
53
|
|
|
54
54
|
broadcast_item(item) if broadcast
|