source_monitor 0.11.1 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.claude/commands/rails-audit.md +77 -0
- data/CHANGELOG.md +50 -0
- data/CLAUDE.md +2 -2
- data/Gemfile.lock +7 -20
- data/RAILS_AUDIT.md +424 -0
- data/VERSION +1 -1
- data/app/assets/builds/source_monitor/application.css +4 -24
- data/app/assets/builds/source_monitor/application.js +57 -89
- data/app/assets/builds/source_monitor/application.js.map +4 -4
- data/app/assets/javascripts/source_monitor/application.js +3 -6
- data/app/assets/javascripts/source_monitor/controllers/dropdown_controller.js +6 -86
- data/app/assets/javascripts/source_monitor/controllers/filter_submit_controller.js +13 -0
- data/app/assets/javascripts/source_monitor/controllers/modal_controller.js +56 -0
- data/app/assets/javascripts/source_monitor/controllers/notification_controller.js +3 -13
- data/app/components/source_monitor/application_component.rb +10 -0
- data/app/components/source_monitor/filter_dropdown_component.rb +62 -0
- data/app/components/source_monitor/icon_component.rb +140 -0
- data/app/components/source_monitor/status_badge_component.html.erb +8 -0
- data/app/components/source_monitor/status_badge_component.rb +96 -0
- data/app/controllers/concerns/source_monitor/sanitizes_search_params.rb +4 -0
- data/app/controllers/concerns/source_monitor/set_source.rb +13 -0
- data/app/controllers/source_monitor/application_controller.rb +17 -0
- data/app/controllers/source_monitor/bulk_scrape_enablements_controller.rb +6 -10
- data/app/controllers/source_monitor/dashboard_controller.rb +5 -1
- data/app/controllers/source_monitor/import_history_dismissals_controller.rb +1 -1
- data/app/controllers/source_monitor/import_sessions_controller.rb +30 -9
- data/app/controllers/source_monitor/item_scrapes_controller.rb +70 -0
- data/app/controllers/source_monitor/items_controller.rb +2 -69
- data/app/controllers/source_monitor/source_bulk_scrapes_controller.rb +1 -4
- data/app/controllers/source_monitor/source_favicon_fetches_controller.rb +2 -12
- data/app/controllers/source_monitor/source_fetches_controller.rb +1 -6
- data/app/controllers/source_monitor/source_health_checks_controller.rb +9 -16
- data/app/controllers/source_monitor/source_health_resets_controller.rb +1 -6
- data/app/controllers/source_monitor/source_retries_controller.rb +1 -6
- data/app/controllers/source_monitor/source_scrape_tests_controller.rb +2 -4
- data/app/controllers/source_monitor/source_turbo_responses.rb +1 -3
- data/app/controllers/source_monitor/sources_controller.rb +15 -20
- data/app/helpers/source_monitor/application_helper.rb +15 -31
- data/app/helpers/source_monitor/health_badge_helper.rb +8 -0
- data/app/jobs/source_monitor/download_content_images_job.rb +1 -59
- data/app/jobs/source_monitor/favicon_fetch_job.rb +1 -58
- data/app/jobs/source_monitor/fetch_feed_job.rb +2 -52
- data/app/jobs/source_monitor/import_opml_job.rb +6 -145
- data/app/jobs/source_monitor/import_session_health_check_job.rb +15 -76
- data/app/jobs/source_monitor/item_cleanup_job.rb +5 -0
- data/app/jobs/source_monitor/log_cleanup_job.rb +13 -2
- data/app/jobs/source_monitor/schedule_fetches_job.rb +8 -0
- data/app/jobs/source_monitor/scrape_item_job.rb +6 -52
- data/app/jobs/source_monitor/source_health_check_job.rb +1 -72
- data/app/models/concerns/source_monitor/loggable.rb +12 -0
- data/app/models/source_monitor/fetch_log.rb +0 -8
- data/app/models/source_monitor/health_check_log.rb +0 -8
- data/app/models/source_monitor/import_history.rb +14 -0
- data/app/models/source_monitor/import_session.rb +2 -0
- data/app/models/source_monitor/item.rb +15 -0
- data/app/models/source_monitor/item_content.rb +4 -3
- data/app/models/source_monitor/scrape_log.rb +4 -6
- data/app/models/source_monitor/source.rb +28 -19
- data/app/presenters/source_monitor/base_presenter.rb +19 -0
- data/app/presenters/source_monitor/source_details_presenter.rb +61 -0
- data/app/presenters/source_monitor/sources_filter_presenter.rb +61 -0
- data/app/views/source_monitor/dashboard/_recent_activity.html.erb +3 -3
- data/app/views/source_monitor/dashboard/_stat_card.html.erb +2 -1
- data/app/views/source_monitor/dashboard/_stats.html.erb +5 -7
- data/app/views/source_monitor/items/_details.html.erb +11 -14
- data/app/views/source_monitor/items/index.html.erb +10 -35
- data/app/views/source_monitor/logs/index.html.erb +20 -41
- data/app/views/source_monitor/shared/_form_errors.html.erb +14 -0
- data/app/views/source_monitor/source_scrape_tests/_result.html.erb +1 -29
- data/app/views/source_monitor/source_scrape_tests/_result_content.html.erb +33 -0
- data/app/views/source_monitor/source_scrape_tests/show.html.erb +1 -29
- data/app/views/source_monitor/sources/_bulk_scrape_enable_modal.html.erb +2 -2
- data/app/views/source_monitor/sources/_bulk_scrape_modal.html.erb +7 -5
- data/app/views/source_monitor/sources/_details.html.erb +24 -52
- data/app/views/source_monitor/sources/_health_status_badge.html.erb +4 -6
- data/app/views/source_monitor/sources/_row.html.erb +7 -18
- data/app/views/source_monitor/sources/edit.html.erb +1 -10
- data/app/views/source_monitor/sources/index.html.erb +26 -46
- data/app/views/source_monitor/sources/new.html.erb +1 -10
- data/config/routes.rb +1 -1
- data/db/migrate/20260313120000_add_composite_indexes_to_log_tables.rb +14 -0
- data/db/migrate/20260314120000_align_health_status_default.rb +11 -0
- data/lib/source_monitor/analytics/sources_index_metrics.rb +15 -0
- data/lib/source_monitor/dashboard/queries/recent_activity_query.rb +10 -4
- data/lib/source_monitor/dashboard/turbo_broadcaster.rb +21 -5
- data/lib/source_monitor/favicons/fetcher.rb +86 -0
- data/lib/source_monitor/fetching/cloudflare_bypass.rb +14 -5
- data/lib/source_monitor/fetching/completion/event_publisher.rb +12 -0
- data/lib/source_monitor/fetching/completion/follow_up_handler.rb +15 -2
- data/lib/source_monitor/fetching/completion/retention_handler.rb +11 -3
- data/lib/source_monitor/fetching/feed_fetcher.rb +2 -21
- data/lib/source_monitor/fetching/fetch_runner.rb +12 -3
- data/lib/source_monitor/fetching/retry_orchestrator.rb +102 -0
- data/lib/source_monitor/fetching/stalled_fetch_reconciler.rb +9 -0
- data/lib/source_monitor/health/source_health_check_orchestrator.rb +95 -0
- data/lib/source_monitor/health.rb +1 -0
- data/lib/source_monitor/images/downloader.rb +6 -7
- data/lib/source_monitor/images/processor.rb +98 -0
- data/lib/source_monitor/import_sessions/health_check_updater.rb +95 -0
- data/lib/source_monitor/import_sessions/opml_importer.rb +163 -0
- data/lib/source_monitor/items/item_creator.rb +0 -21
- data/lib/source_monitor/logs/query.rb +20 -0
- data/lib/source_monitor/queries/scrape_candidates_query.rb +30 -0
- data/lib/source_monitor/queries.rb +7 -0
- data/lib/source_monitor/scheduler.rb +5 -0
- data/lib/source_monitor/scraping/bulk_result_presenter.rb +11 -8
- data/lib/source_monitor/scraping/runner.rb +52 -0
- data/lib/source_monitor/scraping/scheduler.rb +5 -0
- data/lib/source_monitor/scraping/state.rb +4 -2
- data/lib/source_monitor/security/parameter_sanitizer.rb +7 -0
- data/lib/source_monitor/version.rb +1 -1
- data/lib/source_monitor.rb +7 -0
- data/source_monitor.gemspec +1 -0
- metadata +47 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6cb1aa0acdd0eddd3b7b6a51e4e57861420a37e99398440f990cda73ee87b900
|
|
4
|
+
data.tar.gz: 916c1d6d11edd1b41422405c91b5b36100622f0c0b512ed6c7aa660a0625327e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e96387f3a085b20a56b9e97b97b4f0321737dd4f1ad046f364c7bd777f8d7cdf8bf30591a81db24c4ed7fba38742f80c97e3d31e9f794a2a18741dca568c7dee
|
|
7
|
+
data.tar.gz: 6672afcc7ac248c75520c638099b0a0dce8fe5e3a6e49621587de3d090a2fde15d88b57fb801d4eb9fe294366a7688a9d0ce0d45045f6e2860d6435e68a03e93
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Rails Best Practices Audit
|
|
2
|
+
|
|
3
|
+
Perform a comprehensive Rails best practices audit of the entire codebase. Use the rails-specific skills and agents to identify opportunities to simplify, refactor, and align with Rails conventions ("the Rails Way").
|
|
4
|
+
|
|
5
|
+
## Usage
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
/rails-audit # Full audit of entire codebase
|
|
9
|
+
/rails-audit models only # Scope to specific layer
|
|
10
|
+
/rails-audit --changed-only # Only audit changed files (git diff)
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## Instructions
|
|
14
|
+
|
|
15
|
+
Launch a team of agents in parallel to explore the codebase across multiple dimensions. Each agent should use the relevant rails skills (see CLAUDE.md Skill Catalog) and search the web for best practices when skills are insufficient.
|
|
16
|
+
|
|
17
|
+
### Agent 1: Models & Concerns (rails-review agent)
|
|
18
|
+
- Fat-model anti-patterns: models doing too much vs. missing scopes/validations
|
|
19
|
+
- Concern hygiene: single-purpose? overused? should any be model methods?
|
|
20
|
+
- Business logic placement: logic in controllers/services that belongs in models
|
|
21
|
+
- ActiveRecord anti-patterns: raw SQL where scopes suffice, missing `includes`/`preload`
|
|
22
|
+
- Missing validations, unnecessary callbacks, state management patterns
|
|
23
|
+
- Check for state-as-records pattern compliance (booleans vs. state records)
|
|
24
|
+
|
|
25
|
+
### Agent 2: Controllers & Routes (rails-review agent)
|
|
26
|
+
- Everything-is-CRUD compliance: custom actions that should be separate resources
|
|
27
|
+
- Business logic leaking into controllers
|
|
28
|
+
- Strong parameters, before_actions, proper response handling
|
|
29
|
+
- RESTful route compliance (no custom `member`/`collection` verbs)
|
|
30
|
+
- Controller concerns: well-focused or kitchen-sink?
|
|
31
|
+
|
|
32
|
+
### Agent 3: Services, Jobs & Pipeline (rails-review agent)
|
|
33
|
+
- Service objects: single-responsibility, Result pattern compliance
|
|
34
|
+
- Job shallowness: jobs should only deserialize + delegate, no business logic
|
|
35
|
+
- Service objects that should be model methods or concerns (< 3 models = model method)
|
|
36
|
+
- Pipeline stages: consolidation opportunities, error handling
|
|
37
|
+
- Query objects: are complex queries properly extracted?
|
|
38
|
+
|
|
39
|
+
### Agent 4: Views, Frontend & Hotwire (rails-review agent)
|
|
40
|
+
- Turbo Frame/Stream best practices
|
|
41
|
+
- Stimulus controllers: small, focused, one behavior each
|
|
42
|
+
- View logic that should be presenters (SimpleDelegator) or ViewComponents
|
|
43
|
+
- Tailwind CSS patterns: repeated utility groups that should be components
|
|
44
|
+
- Partial organization and reuse
|
|
45
|
+
|
|
46
|
+
### Agent 5: Testing & Quality (rails-review agent)
|
|
47
|
+
- Test DRYness: repeated setup that should be helpers
|
|
48
|
+
- Factory helper consistency (`create_source!`, etc.)
|
|
49
|
+
- Testing behavior vs. implementation
|
|
50
|
+
- Missing coverage patterns (validations, scopes, edge cases)
|
|
51
|
+
- Test isolation and parallel-safety
|
|
52
|
+
|
|
53
|
+
## Output
|
|
54
|
+
|
|
55
|
+
Produce a markdown file at `RAILS_AUDIT.md` in the project root with:
|
|
56
|
+
|
|
57
|
+
```markdown
|
|
58
|
+
# Rails Best Practices Audit — [date]
|
|
59
|
+
|
|
60
|
+
## Executive Summary
|
|
61
|
+
[High-level findings count by severity]
|
|
62
|
+
|
|
63
|
+
## Findings by Category
|
|
64
|
+
|
|
65
|
+
### Category Name
|
|
66
|
+
#### Finding Title
|
|
67
|
+
- **Severity:** high/medium/low
|
|
68
|
+
- **File(s):** `path/to/file.rb:line_number`
|
|
69
|
+
- **Current:** [what it does now]
|
|
70
|
+
- **Recommended:** [what it should do, with code example if helpful]
|
|
71
|
+
- **Rationale:** [why this is better, link to Rails convention]
|
|
72
|
+
- **Effort:** quick (< 30 min) / short (< 2 hrs) / medium (< 1 day) / large (> 1 day)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Sort findings within each category by severity (high first), then by effort (quick first).
|
|
76
|
+
|
|
77
|
+
$ARGUMENTS
|
data/CHANGELOG.md
CHANGED
|
@@ -15,6 +15,56 @@ All notable changes to this project are documented below. The format follows [Ke
|
|
|
15
15
|
|
|
16
16
|
- No unreleased changes yet.
|
|
17
17
|
|
|
18
|
+
## [0.12.0] - 2026-03-15
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
- `StatusBadgeComponent` for unified status badge rendering across all views
|
|
22
|
+
- `IconComponent` with named icon registry replacing inline SVGs
|
|
23
|
+
- `SourceDetailsPresenter` and `SourcesFilterPresenter` for view formatting
|
|
24
|
+
- `FilterDropdownComponent` for reusable filter dropdowns
|
|
25
|
+
- `SetSource` controller concern eliminating duplication across 7 controllers
|
|
26
|
+
- `rescue_from ActiveRecord::RecordNotFound` with Turbo-aware 404 handling
|
|
27
|
+
- `Source.enable_scraping!` class method for bulk enablement
|
|
28
|
+
- `Item#restore!` with counter cache symmetry to `soft_delete!`
|
|
29
|
+
- `Source` health_status inclusion validation and scraping scopes
|
|
30
|
+
- Modal accessibility: `role="dialog"`, `aria-modal`, focus trapping via `inert`
|
|
31
|
+
- Logs index Turbo Frame for partial page updates on filter/pagination
|
|
32
|
+
- Missing controller tests for FetchLogsController and ScrapeLogsController
|
|
33
|
+
- Date range scopes and composite indexes on Loggable concern
|
|
34
|
+
- Result structs for completion handlers (EventPublisher, RetentionHandler, FollowUpHandler)
|
|
35
|
+
- Transient vs fatal error classification in FaviconFetchJob and DownloadContentImagesJob
|
|
36
|
+
- SharedLoggableTests module for consistent log model testing
|
|
37
|
+
- TEST_CONVENTIONS.md documenting testing patterns
|
|
38
|
+
|
|
39
|
+
### Fixed
|
|
40
|
+
- LogCleanupJob now cascades deletes to LogEntry records preventing orphaned rows
|
|
41
|
+
- `health_status` default aligned between model (`working`) and database
|
|
42
|
+
- Swallowed exceptions in `Scraping::State` and pipeline rescue blocks now log warnings
|
|
43
|
+
- Pagination tests use `clean_source_monitor_tables!` instead of `Source.destroy_all` for parallel safety
|
|
44
|
+
- `fallback_user_id` guarded behind `Rails.env.development?` to prevent engine writing to host-app tables
|
|
45
|
+
- Dashboard GC pressure reduced from SolidQueue data accumulation
|
|
46
|
+
- Deadlock rescue added to 4 jobs
|
|
47
|
+
|
|
48
|
+
### Changed
|
|
49
|
+
- **5 jobs extracted to service classes** (shallow delegation pattern):
|
|
50
|
+
- `ScrapeItemJob` → `Scraping::Runner` (73→21 lines)
|
|
51
|
+
- `DownloadContentImagesJob` → `Images::Processor` (95→16 lines)
|
|
52
|
+
- `FaviconFetchJob` → `Favicons::Fetcher` (94→16 lines)
|
|
53
|
+
- `SourceHealthCheckJob` → `Health::SourceHealthCheckOrchestrator` (85→16 lines)
|
|
54
|
+
- `ImportSessionHealthCheckJob` → `ImportSessions::HealthCheckUpdater` (99→32 lines)
|
|
55
|
+
- Duplicated scrape rate-limiting consolidated (removed from ScrapeItemJob, kept in Enqueuer)
|
|
56
|
+
- `sync_log_entry` callback consolidated into Loggable concern (removed from 3 models)
|
|
57
|
+
- `ItemScrapesController` extracted from Items#scrape for CRUD compliance
|
|
58
|
+
- FetchFeedJob retry/circuit-breaker logic extracted to RetryOrchestrator service
|
|
59
|
+
- Favicon cooldown logic moved from controller to Source model
|
|
60
|
+
- Import step dispatch refactored to handler registry pattern
|
|
61
|
+
- 30 backward-compatibility forwarding methods removed from FeedFetcher and ItemCreator
|
|
62
|
+
- `Images::Downloader` uses `HTTP.client` instead of raw Faraday (respects proxy/SSL config)
|
|
63
|
+
- `CloudflareBypass` limited to 1 UA rotation attempt with per-attempt timeout
|
|
64
|
+
- Dashboard stats split into per-stat Turbo Streams for granular updates
|
|
65
|
+
- Test factory helpers centralized into ModelFactories module
|
|
66
|
+
- System tests refactored to shared base class with SystemTestHelpers
|
|
67
|
+
|
|
18
68
|
## [0.11.1] - 2026-03-13
|
|
19
69
|
|
|
20
70
|
### Fixed
|
data/CLAUDE.md
CHANGED
|
@@ -4,8 +4,8 @@
|
|
|
4
4
|
|
|
5
5
|
## Active Context
|
|
6
6
|
|
|
7
|
-
**Last shipped:**
|
|
8
|
-
**Next action:** /vbw:vibe to
|
|
7
|
+
**Last shipped:** rails-audit-and-refactoring (7 phases, 30 plans)
|
|
8
|
+
**Next action:** /vbw:vibe to start new work
|
|
9
9
|
|
|
10
10
|
## Key Decisions
|
|
11
11
|
|
data/Gemfile.lock
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
source_monitor (0.
|
|
4
|
+
source_monitor (0.12.0)
|
|
5
5
|
cssbundling-rails (~> 1.4)
|
|
6
6
|
faraday (~> 2.9)
|
|
7
7
|
faraday-follow_redirects (~> 0.4)
|
|
@@ -16,6 +16,7 @@ PATH
|
|
|
16
16
|
solid_cable (>= 3.0, < 4.0)
|
|
17
17
|
solid_queue (>= 0.3, < 3.0)
|
|
18
18
|
turbo-rails (~> 2.0)
|
|
19
|
+
view_component (>= 3.0, < 4.0)
|
|
19
20
|
|
|
20
21
|
GEM
|
|
21
22
|
remote: https://rubygems.org/
|
|
@@ -177,6 +178,7 @@ GEM
|
|
|
177
178
|
net-smtp
|
|
178
179
|
marcel (1.1.0)
|
|
179
180
|
matrix (0.4.3)
|
|
181
|
+
method_source (1.1.0)
|
|
180
182
|
mini_magick (5.3.1)
|
|
181
183
|
logger
|
|
182
184
|
mini_mime (1.1.5)
|
|
@@ -207,18 +209,7 @@ GEM
|
|
|
207
209
|
racc (~> 1.4)
|
|
208
210
|
nokogiri (1.19.0-arm-linux-musl)
|
|
209
211
|
racc (~> 1.4)
|
|
210
|
-
nokogiri (1.19.0-arm64-darwin)
|
|
211
|
-
racc (~> 1.4)
|
|
212
|
-
nokogiri (1.19.0-x86_64-darwin)
|
|
213
|
-
racc (~> 1.4)
|
|
214
|
-
nokogiri (1.19.0-x86_64-linux-gnu)
|
|
215
|
-
racc (~> 1.4)
|
|
216
|
-
nokogiri (1.19.0-x86_64-linux-musl)
|
|
217
|
-
racc (~> 1.4)
|
|
218
212
|
nokolexbor (0.6.2)
|
|
219
|
-
nokolexbor (0.6.2-arm64-darwin)
|
|
220
|
-
nokolexbor (0.6.2-x86_64-darwin)
|
|
221
|
-
nokolexbor (0.6.2-x86_64-linux)
|
|
222
213
|
ostruct (0.6.3)
|
|
223
214
|
parallel (1.27.0)
|
|
224
215
|
parser (3.3.10.1)
|
|
@@ -227,10 +218,6 @@ GEM
|
|
|
227
218
|
pg (1.6.3)
|
|
228
219
|
pg (1.6.3-aarch64-linux)
|
|
229
220
|
pg (1.6.3-aarch64-linux-musl)
|
|
230
|
-
pg (1.6.3-arm64-darwin)
|
|
231
|
-
pg (1.6.3-x86_64-darwin)
|
|
232
|
-
pg (1.6.3-x86_64-linux)
|
|
233
|
-
pg (1.6.3-x86_64-linux-musl)
|
|
234
221
|
pp (0.6.3)
|
|
235
222
|
prettyprint
|
|
236
223
|
prettyprint (0.2.0)
|
|
@@ -375,6 +362,10 @@ GEM
|
|
|
375
362
|
uri (1.1.1)
|
|
376
363
|
useragent (0.16.11)
|
|
377
364
|
vcr (6.4.0)
|
|
365
|
+
view_component (3.24.0)
|
|
366
|
+
activesupport (>= 5.2.0, < 8.2)
|
|
367
|
+
concurrent-ruby (~> 1)
|
|
368
|
+
method_source (~> 1.0)
|
|
378
369
|
webmock (3.26.1)
|
|
379
370
|
addressable (>= 2.8.0)
|
|
380
371
|
crack (>= 0.3.2)
|
|
@@ -394,11 +385,7 @@ PLATFORMS
|
|
|
394
385
|
aarch64-linux-musl
|
|
395
386
|
arm-linux-gnu
|
|
396
387
|
arm-linux-musl
|
|
397
|
-
arm64-darwin
|
|
398
388
|
ruby
|
|
399
|
-
x86_64-darwin
|
|
400
|
-
x86_64-linux-gnu
|
|
401
|
-
x86_64-linux-musl
|
|
402
389
|
|
|
403
390
|
DEPENDENCIES
|
|
404
391
|
brakeman
|
data/RAILS_AUDIT.md
ADDED
|
@@ -0,0 +1,424 @@
|
|
|
1
|
+
# Rails Best Practices Audit — 2026-03-14
|
|
2
|
+
|
|
3
|
+
## Executive Summary
|
|
4
|
+
|
|
5
|
+
5 parallel agents audited the entire SourceMonitor engine codebase across models, controllers, services/jobs/pipeline, views/frontend, and testing layers.
|
|
6
|
+
|
|
7
|
+
| Severity | Remaining | Already Fixed |
|
|
8
|
+
|----------|-----------|---------------|
|
|
9
|
+
| **HIGH** | 6 | 0 |
|
|
10
|
+
| **MEDIUM** | 17 | 10 |
|
|
11
|
+
| **LOW** | 21 | 9 |
|
|
12
|
+
| **Total** | **44** | **19** |
|
|
13
|
+
|
|
14
|
+
**Overall verdict:** The codebase is well-structured for an engine of this complexity. Routes follow CRUD conventions, concerns are well-scoped, and the Hotwire frontend has solid fundamentals. Recent milestone work (phases 01-06) already resolved ~19 findings. The remaining gaps are primarily **business logic in jobs** (violates "shallow jobs" convention) and **duplicated logic** across pipeline layers.
|
|
15
|
+
|
|
16
|
+
> **Note:** 19 findings from the initial 63 were already addressed by commits in the recent ui-fixes-and-smart-scraping milestone. These are marked ~~strikethrough~~ below. The counts above reflect only unresolved findings.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Table of Contents
|
|
21
|
+
|
|
22
|
+
- [HIGH Severity Findings](#high-severity-findings)
|
|
23
|
+
- [MEDIUM Severity Findings](#medium-severity-findings)
|
|
24
|
+
- [LOW Severity Findings](#low-severity-findings)
|
|
25
|
+
- [Top 10 Actions](#top-10-actions-prioritized-by-impacteffort-ratio)
|
|
26
|
+
- [Positive Observations](#positive-observations)
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## HIGH Severity Findings
|
|
31
|
+
|
|
32
|
+
### H1. LogCleanupJob orphans LogEntry records
|
|
33
|
+
|
|
34
|
+
- **File(s):** `app/jobs/source_monitor/log_cleanup_job.rb:42-49`
|
|
35
|
+
- **Current:** Uses `batch.delete_all` on FetchLog/ScrapeLog records. These models have `has_one :log_entry, dependent: :destroy`, but `delete_all` skips callbacks, orphaning LogEntry records.
|
|
36
|
+
- **Recommended:** Delete LogEntry records first by `loggable_type`/`loggable_id`, then delete the log records. Or use `destroy_in_batches`.
|
|
37
|
+
- **Rationale:** Orphaned LogEntry records accumulate over time, consuming disk space and corrupting the unified logs view. The `dependent: :destroy` declaration shows cascade was intended.
|
|
38
|
+
- **Effort:** short
|
|
39
|
+
|
|
40
|
+
### H2. ImportOpmlJob contains 160 lines of business logic
|
|
41
|
+
|
|
42
|
+
- **File(s):** `app/jobs/source_monitor/import_opml_job.rb:14-157`
|
|
43
|
+
- **Current:** The job contains entry selection, deduplication, source creation, attribute building, broadcast logic, and error aggregation. This is multi-model orchestration (Source, ImportHistory, ImportSession) in a job.
|
|
44
|
+
- **Recommended:** Extract to `SourceMonitor::ImportSessions::OPMLImporter` service. The job becomes a 5-line delegation.
|
|
45
|
+
- **Rationale:** Violates "shallow jobs: only deserialization + delegation." Import logic cannot be invoked synchronously (console, tests) without going through ActiveJob. Spans 3+ models, qualifying for a service object.
|
|
46
|
+
- **Effort:** medium
|
|
47
|
+
|
|
48
|
+
### H3. ScrapeItemJob contains rate-limiting, state management, and deferral logic
|
|
49
|
+
|
|
50
|
+
- **File(s):** `app/jobs/source_monitor/scrape_item_job.rb:14-57`
|
|
51
|
+
- **Current:** Checks scraping-enabled status, computes time-until-scrape-allowed, manages state transitions (`mark_processing!`, `mark_failed!`, `clear_inflight!`), and re-enqueues itself with a delay.
|
|
52
|
+
- **Recommended:** Move pre-flight checks and state management into `Scraping::Runner`. Job becomes a one-liner delegation.
|
|
53
|
+
- **Rationale:** Rate-limiting in `time_until_scrape_allowed` duplicates near-identical logic in `Scraping::Enqueuer#time_rate_limited?`. Two places to maintain the same business rule.
|
|
54
|
+
- **Effort:** medium
|
|
55
|
+
|
|
56
|
+
### H4. DownloadContentImagesJob contains multi-model orchestration
|
|
57
|
+
|
|
58
|
+
- **File(s):** `app/jobs/source_monitor/download_content_images_job.rb:17-49`
|
|
59
|
+
- **Current:** Builds ItemContent, downloads images, creates ActiveStorage blobs, rewrites HTML, and updates the item.
|
|
60
|
+
- **Recommended:** Extract to `SourceMonitor::Images::Processor`. Job delegates with a single call.
|
|
61
|
+
- **Rationale:** Multi-model orchestration (Item, ItemContent, ActiveStorage::Blob) belongs in a pipeline class, not a job.
|
|
62
|
+
- **Effort:** short
|
|
63
|
+
|
|
64
|
+
### H5. Scrape rate-limiting duplicated in two places
|
|
65
|
+
|
|
66
|
+
- **File(s):** `app/jobs/source_monitor/scrape_item_job.rb:47-57` and `lib/source_monitor/scraping/enqueuer.rb:129-143`
|
|
67
|
+
- **Current:** Both compute time since last scrape vs. `min_scrape_interval`. The Enqueuer defers the job, and the Job re-checks and defers again.
|
|
68
|
+
- **Recommended:** Remove the check from `ScrapeItemJob`. The Enqueuer already handles deferral at enqueue time. If race conditions are a concern, have the job delegate to a runner that calls the Enqueuer.
|
|
69
|
+
- **Rationale:** Redundant DB queries and divergence risk if one is updated without the other.
|
|
70
|
+
- **Effort:** quick
|
|
71
|
+
|
|
72
|
+
### H6. `Source.destroy_all` in pagination tests is not parallel-safe
|
|
73
|
+
|
|
74
|
+
- **File(s):** `test/controllers/source_monitor/sources_controller_test.rb:252,264,276,286,297,311,322`
|
|
75
|
+
- **Current:** Seven tests call `Source.destroy_all` to get a clean slate for pagination counting. With thread-based parallelism, this can race with other threads.
|
|
76
|
+
- **Recommended:** Scope assertions to test-created records using a naming pattern or tracking IDs. Or use a dedicated test class with proper isolation.
|
|
77
|
+
- **Rationale:** Violates the project's own documented isolation rule in `TEST_CONVENTIONS.md` section 6.
|
|
78
|
+
- **Effort:** medium
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## MEDIUM Severity Findings
|
|
83
|
+
|
|
84
|
+
### Models & Concerns
|
|
85
|
+
|
|
86
|
+
#### M1. `health_status` default mismatch between model and database
|
|
87
|
+
|
|
88
|
+
- **File(s):** `app/models/source_monitor/source.rb:37` vs `db/schema.rb:384`
|
|
89
|
+
- **Current:** Model declares `attribute :health_status, :string, default: "working"` but schema has `default: "healthy"`. A Source created in Ruby gets `"working"`, one via raw SQL gets `"healthy"`.
|
|
90
|
+
- **Recommended:** Align the defaults. Add `validates :health_status, inclusion: { in: HEALTH_STATUS_VALUES }`.
|
|
91
|
+
- **Effort:** quick
|
|
92
|
+
|
|
93
|
+
#### M2. Missing `health_status` validation
|
|
94
|
+
|
|
95
|
+
- **File(s):** `app/models/source_monitor/source.rb:44-51`
|
|
96
|
+
- **Current:** `fetch_status` has an inclusion validation; `health_status` has none. Any arbitrary string can be stored.
|
|
97
|
+
- **Recommended:** Add `HEALTH_STATUS_VALUES = %w[healthy working declining failing].freeze` and `validates :health_status, inclusion: { in: HEALTH_STATUS_VALUES }`.
|
|
98
|
+
- **Effort:** quick
|
|
99
|
+
|
|
100
|
+
#### M3. `Item#soft_delete!` counter cache fragility
|
|
101
|
+
|
|
102
|
+
- **File(s):** `app/models/source_monitor/item.rb:69-83`
|
|
103
|
+
- **Current:** Manually calls `Source.decrement_counter(:items_count, source_id)` after `update_columns`. No corresponding `restore!` method to re-increment.
|
|
104
|
+
- **Recommended:** Add a `restore!` method for symmetry. Consider extracting soft-delete into a concern.
|
|
105
|
+
- **Effort:** short
|
|
106
|
+
|
|
107
|
+
#### M4. Duplicated `sync_log_entry` callback across 3 log models
|
|
108
|
+
|
|
109
|
+
- **File(s):** `app/models/source_monitor/fetch_log.rb:28`, `scrape_log.rb:20`, `health_check_log.rb:20`
|
|
110
|
+
- **Current:** All three define identical `after_save :sync_log_entry` callbacks.
|
|
111
|
+
- **Recommended:** Move into the `Loggable` concern.
|
|
112
|
+
- **Effort:** quick
|
|
113
|
+
|
|
114
|
+
### Controllers & Routes
|
|
115
|
+
|
|
116
|
+
#### M5. No `rescue_from ActiveRecord::RecordNotFound`
|
|
117
|
+
|
|
118
|
+
- **File(s):** `app/controllers/source_monitor/application_controller.rb`
|
|
119
|
+
- **Current:** No rescue_from handlers. A missing record raises 500 in production.
|
|
120
|
+
- **Recommended:** Add a Turbo-aware RecordNotFound handler that renders a toast + 404.
|
|
121
|
+
- **Rationale:** As a mountable engine, SourceMonitor should handle its own common exceptions gracefully.
|
|
122
|
+
- **Effort:** short
|
|
123
|
+
|
|
124
|
+
#### M6. Duplicated `set_source` across 7 controllers
|
|
125
|
+
|
|
126
|
+
- **File(s):** `source_fetches_controller.rb`, `source_retries_controller.rb`, `source_bulk_scrapes_controller.rb`, `source_health_checks_controller.rb`, `source_health_resets_controller.rb`, `source_favicon_fetches_controller.rb`, `source_scrape_tests_controller.rb`
|
|
127
|
+
- **Current:** Each defines identical `def set_source; @source = Source.find(params[:source_id]); end`.
|
|
128
|
+
- **Recommended:** Extract to a `SetSource` concern.
|
|
129
|
+
- **Effort:** quick
|
|
130
|
+
|
|
131
|
+
#### M7. `fallback_user_id` creates users in host-app tables
|
|
132
|
+
|
|
133
|
+
- **File(s):** `app/controllers/source_monitor/import_sessions_controller.rb:244-276`
|
|
134
|
+
- **Current:** When no authenticated user exists, creates a "guest" user by introspecting column schema.
|
|
135
|
+
- **Recommended:** Guard behind `Rails.env.development?` or remove entirely. An engine should never create records in host-app tables.
|
|
136
|
+
- **Effort:** short
|
|
137
|
+
|
|
138
|
+
#### M8. ImportSessions controller concerns contain significant business logic
|
|
139
|
+
|
|
140
|
+
- **File(s):** `app/controllers/source_monitor/import_sessions/opml_parser.rb` (128 lines), `entry_annotation.rb` (187 lines), `health_check_management.rb` (112 lines), `bulk_configuration.rb` (106 lines)
|
|
141
|
+
- **Current:** XML parsing, URL validation, duplicate detection, job enqueueing, database locking — all in controller concerns.
|
|
142
|
+
- **Recommended:** Extract pure-domain parts to `lib/` or `app/services/` classes. Controller concerns become thin wrappers.
|
|
143
|
+
- **Effort:** large
|
|
144
|
+
|
|
145
|
+
#### ~~M9. `SourcesController#index` has 47 lines of query orchestration~~ RESOLVED
|
|
146
|
+
|
|
147
|
+
- ~~Resolved by `a6d7148` (extract sources index metrics) + `795b7b8` (SourcesFilterPresenter) + `cafefc2` (FilterDropdownComponent)~~
|
|
148
|
+
|
|
149
|
+
#### M10. `BulkScrapeEnablementsController` contains business logic
|
|
150
|
+
|
|
151
|
+
- **File(s):** `app/controllers/source_monitor/bulk_scrape_enablements_controller.rb:13-19`
|
|
152
|
+
- **Current:** `update_all` with field combination for enabling scraping is in the controller.
|
|
153
|
+
- **Recommended:** Extract to `Source.enable_scraping!(ids)` class method.
|
|
154
|
+
- **Effort:** quick
|
|
155
|
+
|
|
156
|
+
#### M11. Excessive `update_column` usage in ImportSessions flow
|
|
157
|
+
|
|
158
|
+
- **File(s):** `app/controllers/source_monitor/import_sessions_controller.rb` (11 calls)
|
|
159
|
+
- **Current:** Skips validations for `current_step` and `selected_source_ids` changes.
|
|
160
|
+
- **Recommended:** Encapsulate in model methods like `ImportSession#advance_to!(step)`.
|
|
161
|
+
- **Effort:** short
|
|
162
|
+
|
|
163
|
+
### Services, Jobs & Pipeline
|
|
164
|
+
|
|
165
|
+
#### M12. FaviconFetchJob contains cooldown and attachment logic
|
|
166
|
+
|
|
167
|
+
- **File(s):** `app/jobs/source_monitor/favicon_fetch_job.rb:17-42`
|
|
168
|
+
- **Current:** Cooldown checking duplicated with `SourceUpdater#enqueue_favicon_fetch_if_needed`.
|
|
169
|
+
- **Recommended:** Extract to `Favicons::FetchService`. Consolidate cooldown in `Favicons::CooldownCheck`.
|
|
170
|
+
- **Effort:** short
|
|
171
|
+
|
|
172
|
+
#### M13. ImportSessionHealthCheckJob contains lock management
|
|
173
|
+
|
|
174
|
+
- **File(s):** `app/jobs/source_monitor/import_session_health_check_job.rb:18-63`
|
|
175
|
+
- **Current:** Acquires row lock, merges results, updates state, broadcasts.
|
|
176
|
+
- **Recommended:** Extract to `ImportSessions::HealthCheckUpdater`.
|
|
177
|
+
- **Effort:** short
|
|
178
|
+
|
|
179
|
+
#### M14. SourceHealthCheckJob contains broadcast and toast formatting
|
|
180
|
+
|
|
181
|
+
- **File(s):** `app/jobs/source_monitor/source_health_check_job.rb:29-83`
|
|
182
|
+
- **Current:** `toast_payload`, `broadcast_outcome`, `trigger_fetch_if_degraded` are all presentation/side-effect logic.
|
|
183
|
+
- **Recommended:** Move into `Health::SourceHealthCheckOrchestrator`.
|
|
184
|
+
- **Effort:** short
|
|
185
|
+
|
|
186
|
+
#### ~~M15. Inconsistent Result pattern across pipeline classes~~ PARTIALLY RESOLVED
|
|
187
|
+
|
|
188
|
+
- ~~`e03723d` added Result structs to completion handlers; `5bd538a` wired Result usage in FetchRunner~~
|
|
189
|
+
- **Remaining:** `FeedFetcher::Result` still lacks `success?`. No shared base `SourceMonitor::Result` class exists yet.
|
|
190
|
+
- **Effort:** medium
|
|
191
|
+
|
|
192
|
+
#### ~~M16. Retry logic split across 4 locations~~ RESOLVED
|
|
193
|
+
|
|
194
|
+
- ~~Resolved by `4ff8884` (extract FetchFeedJob retry orchestrator service)~~
|
|
195
|
+
|
|
196
|
+
#### M17. Swallowed exceptions in ensure/rescue blocks
|
|
197
|
+
|
|
198
|
+
- **File(s):** `feed_fetcher.rb:331`, `scraping/state.rb:68`, `source_health_check_job.rb:46-47`
|
|
199
|
+
- **Current:** `rescue StandardError => nil` silently swallows failures.
|
|
200
|
+
- **Recommended:** Add `Rails.logger.warn` in rescue blocks.
|
|
201
|
+
- **Effort:** quick
|
|
202
|
+
|
|
203
|
+
#### M18. StalledFetchReconciler uses fragile PG JSON operator on SolidQueue internals
|
|
204
|
+
|
|
205
|
+
- **File(s):** `lib/source_monitor/fetching/stalled_fetch_reconciler.rb:107`
|
|
206
|
+
- **Current:** `where("arguments::jsonb -> 'arguments' ->> 0 = ?", source.id.to_s)` — fragile if SolidQueue changes serialization.
|
|
207
|
+
- **Recommended:** Add version comment and regression test.
|
|
208
|
+
- **Effort:** quick
|
|
209
|
+
|
|
210
|
+
### Views & Frontend
|
|
211
|
+
|
|
212
|
+
#### M19. Database queries executed in view templates
|
|
213
|
+
|
|
214
|
+
- **File(s):** `items/_details.html.erb:6-7`, `sources/_bulk_scrape_modal.html.erb:3-11`
|
|
215
|
+
- **Current:** `item.scrape_logs.order(...).limit(5)` and similar queries directly in ERB.
|
|
216
|
+
- **Recommended:** Move to controllers and pass as locals, or extend presenters.
|
|
217
|
+
- **Note:** `sources/_details.html.erb` was partially addressed by `SourceDetailsPresenter` (`7c2604b`), but items and bulk scrape modal still have inline queries.
|
|
218
|
+
- **Effort:** short
|
|
219
|
+
|
|
220
|
+
#### M20. StatusBadge markup duplicated 12+ times
|
|
221
|
+
|
|
222
|
+
- **File(s):** `sources/_row.html.erb`, `sources/_details.html.erb`, `dashboard/_recent_activity.html.erb`, `items/_details.html.erb`, `items/index.html.erb`, `logs/index.html.erb`
|
|
223
|
+
- **Current:** Hand-crafted `<span class="inline-flex items-center rounded-full ...">` with conditional spinners.
|
|
224
|
+
- **Recommended:** Create a `StatusBadgeComponent`.
|
|
225
|
+
- **Note:** `IconComponent` was added (`caa4e69`) for SVG icons, but status badges are a separate pattern that still needs extraction.
|
|
226
|
+
- **Effort:** medium
|
|
227
|
+
|
|
228
|
+
#### ~~M21. Missing SourceDetailsPresenter~~ PARTIALLY RESOLVED
|
|
229
|
+
|
|
230
|
+
- ~~`SourceDetailsPresenter` added in `7c2604b`~~
|
|
231
|
+
- **Remaining:** `ItemDetailsPresenter`, `FetchLogPresenter`, `ScrapeLogPresenter`, `SourceRowPresenter` still missing.
|
|
232
|
+
- **Effort:** medium (each)
|
|
233
|
+
|
|
234
|
+
#### M22. Modal missing `role="dialog"` and `aria-modal`
|
|
235
|
+
|
|
236
|
+
- **File(s):** `sources/_bulk_scrape_modal.html.erb`, `sources/_bulk_scrape_enable_modal.html.erb`
|
|
237
|
+
- **Current:** Modal panels are plain `<div>` elements.
|
|
238
|
+
- **Recommended:** Add `role="dialog"`, `aria-modal="true"`, `aria-labelledby` pointing to heading.
|
|
239
|
+
- **Effort:** quick
|
|
240
|
+
|
|
241
|
+
#### M23. Modal controller lacks focus trapping
|
|
242
|
+
|
|
243
|
+
- **File(s):** `app/assets/javascripts/source_monitor/controllers/modal_controller.js`
|
|
244
|
+
- **Current:** Handles Escape and backdrop click but doesn't trap focus. Users can Tab out.
|
|
245
|
+
- **Recommended:** Implement focus trapping with `inert` attribute on background elements.
|
|
246
|
+
- **Rationale:** WCAG 2.1 SC 2.4.3 requires meaningful focus order.
|
|
247
|
+
- **Effort:** medium
|
|
248
|
+
|
|
249
|
+
#### M24. Logs index missing Turbo Frame for filter/pagination
|
|
250
|
+
|
|
251
|
+
- **File(s):** `app/views/source_monitor/logs/index.html.erb`
|
|
252
|
+
- **Current:** No Turbo Frame wrapping. `form_with` uses `local: true` disabling Turbo. Full page reload on filter.
|
|
253
|
+
- **Recommended:** Wrap table+pagination in a Turbo Frame. Sources and Items both use this pattern.
|
|
254
|
+
- **Effort:** medium
|
|
255
|
+
|
|
256
|
+
#### M25. Button styles inconsistent across templates
|
|
257
|
+
|
|
258
|
+
- **File(s):** Virtually every template
|
|
259
|
+
- **Current:** 5-6 button variants with inconsistent `font-semibold` vs `font-medium`, padding variations.
|
|
260
|
+
- **Recommended:** Extract button variants to `@apply` CSS classes or a `ButtonComponent`.
|
|
261
|
+
- **Effort:** medium
|
|
262
|
+
|
|
263
|
+
#### M26. `ApplicationHelper` is 333 lines with mixed concerns
|
|
264
|
+
|
|
265
|
+
- **File(s):** `app/helpers/source_monitor/application_helper.rb`
|
|
266
|
+
- **Current:** 20+ methods spanning badges, favicons, pagination, URLs, formatting.
|
|
267
|
+
- **Recommended:** Split into focused helper modules: `StatusBadgeHelper`, `FaviconHelper`, `PaginationHelper`, `FetchIntervalHelper`, `ExternalLinkHelper`.
|
|
268
|
+
- **Note:** `SourcesFilterPresenter` (`795b7b8`) and `FilterDropdownComponent` (`cafefc2`) extracted some filter logic, but the helper itself is still large.
|
|
269
|
+
- **Effort:** medium
|
|
270
|
+
|
|
271
|
+
### Testing
|
|
272
|
+
|
|
273
|
+
#### ~~M27. `create_item!` factory underused~~ PARTIALLY RESOLVED
|
|
274
|
+
|
|
275
|
+
- ~~`18692f6` centralized factory helpers into ModelFactories module~~
|
|
276
|
+
- **Remaining:** Many test files still use manual `Item.create!` instead of the now-centralized `create_item!`. Migration to the shared factories is incomplete.
|
|
277
|
+
- **Effort:** medium
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
281
|
+
## LOW Severity Findings
|
|
282
|
+
|
|
283
|
+
### Controllers & Routes
|
|
284
|
+
|
|
285
|
+
#### L1. `new` action delegates to `create` in ImportSessionsController (GET creates records)
|
|
286
|
+
- `import_sessions_controller.rb:35-37` — quick
|
|
287
|
+
|
|
288
|
+
#### L2. `BulkScrapeEnablementsController` accesses params without strong params wrapper
|
|
289
|
+
- `bulk_scrape_enablements_controller.rb:6` — quick
|
|
290
|
+
|
|
291
|
+
#### L3. `SourceScrapeTestsController#create` builds result hash inline (should be presenter)
|
|
292
|
+
- `source_scrape_tests_controller.rb:14-24` — short
|
|
293
|
+
|
|
294
|
+
#### L4. `SourceHealthChecksController` embeds Tailwind classes in controller
|
|
295
|
+
- `source_health_checks_controller.rb:25-31` — quick
|
|
296
|
+
|
|
297
|
+
#### L5. Inconsistent Turbo Stream response patterns (StreamResponder vs raw arrays)
|
|
298
|
+
- Multiple controllers — short
|
|
299
|
+
|
|
300
|
+
#### ~~L6. Broad `rescue StandardError` in action controllers~~ RESOLVED
|
|
301
|
+
- ~~Resolved by `6bcd0ac` and `19bb3b8` (transient vs fatal error classification in FaviconFetchJob and DownloadContentImagesJob) + `911c17e` (deadlock rescue + error logging)~~
|
|
302
|
+
|
|
303
|
+
#### L7. `SanitizesSearchParams` uses `to_unsafe_h` without documentation
|
|
304
|
+
- `concerns/source_monitor/sanitizes_search_params.rb:45` — quick
|
|
305
|
+
|
|
306
|
+
#### L8. `SourcesController#update` contains conditional job-enqueue logic
|
|
307
|
+
- `sources_controller.rb:94-109` — short
|
|
308
|
+
|
|
309
|
+
### Models
|
|
310
|
+
|
|
311
|
+
#### L9. Ransacker subqueries could be extracted to `Source::SearchableAttributes` concern
|
|
312
|
+
- `source.rb:79-103` — short
|
|
313
|
+
|
|
314
|
+
#### L10. Missing `scraping_enabled` / `scraping_disabled` scopes
|
|
315
|
+
- `source.rb` — quick
|
|
316
|
+
|
|
317
|
+
#### L11. `ItemContent#compute_feed_word_count` reaches through association (minor Demeter violation)
|
|
318
|
+
- `item_content.rb:33-39` — quick
|
|
319
|
+
|
|
320
|
+
#### L12. `ImportHistory` missing chronological validation and JSONB attribute declarations
|
|
321
|
+
- `import_history.rb` — quick
|
|
322
|
+
|
|
323
|
+
### Services, Jobs & Pipeline
|
|
324
|
+
|
|
325
|
+
#### ~~L13. `FetchFeedJob#should_run?` scheduling logic~~ PARTIALLY RESOLVED
|
|
326
|
+
- ~~`4ff8884` extracted retry orchestrator, reducing job logic. `should_run?` guard still exists but is simpler.~~
|
|
327
|
+
|
|
328
|
+
#### L14. Backward-compatibility forwarding methods in FeedFetcher (12) and ItemCreator (18)
|
|
329
|
+
- `feed_fetcher.rb:393-404`, `item_creator.rb:179-197` — medium
|
|
330
|
+
|
|
331
|
+
#### L15. FeedFetcher constants duplicated from AdaptiveInterval
|
|
332
|
+
- `feed_fetcher.rb:30-36` — quick
|
|
333
|
+
|
|
334
|
+
#### L16. Inconsistent logger guard pattern (20+ occurrences of full guard)
|
|
335
|
+
- Nearly every pipeline file — short
|
|
336
|
+
|
|
337
|
+
#### L17. `Images::Downloader` creates raw Faraday connection instead of `HTTP.client`
|
|
338
|
+
- `images/downloader.rb:46-58` — quick
|
|
339
|
+
|
|
340
|
+
#### L18. `Logs::Query` is good; `Scheduler` queries could be extracted
|
|
341
|
+
- `scheduler.rb:55-89`, `scraping/scheduler.rb:38-45` — short
|
|
342
|
+
|
|
343
|
+
#### L19. CloudflareBypass tries all 4 user agents sequentially (could cause 60s+ fetch)
|
|
344
|
+
- `fetching/cloudflare_bypass.rb:39-49` — quick
|
|
345
|
+
|
|
346
|
+
### Views & Frontend
|
|
347
|
+
|
|
348
|
+
#### L20. Items and Logs index pagination not using shared `_pagination.html.erb` partial
|
|
349
|
+
- `items/index.html.erb:124-146`, `logs/index.html.erb:186-209` — short
|
|
350
|
+
|
|
351
|
+
#### L21. Scrape test result markup duplicated between show page and modal
|
|
352
|
+
- `source_scrape_tests/show.html.erb:8-56`, `_result.html.erb:12-60` — quick
|
|
353
|
+
|
|
354
|
+
#### L22. Card panel pattern repeated ~20 times (potential `PanelComponent`)
|
|
355
|
+
- Various dashboard/sources/items views — medium
|
|
356
|
+
|
|
357
|
+
#### L23. Error display duplicated across new/edit views (should be `_form_errors` partial)
|
|
358
|
+
- `sources/new.html.erb:4-13`, `edit.html.erb:4-13` — quick
|
|
359
|
+
|
|
360
|
+
#### ~~L24. Dropdown controller registers global click listener eagerly~~ RESOLVED
|
|
361
|
+
- ~~Resolved by `491fae1` (simplify dropdown controller and remove JS globals)~~
|
|
362
|
+
|
|
363
|
+
#### ~~L25. Notification controller has dead `applyLevelDelay()` method~~ RESOLVED
|
|
364
|
+
- ~~Resolved by `15e7d53` (remove dead JS error delay override and document constants)~~
|
|
365
|
+
|
|
366
|
+
#### ~~L26. Dismiss button SVG missing `aria-label`, should use `IconComponent`~~ PARTIALLY RESOLVED
|
|
367
|
+
- ~~`4c56789` replaced inline SVGs with IconComponent. Check if this specific dismiss button was included.~~
|
|
368
|
+
|
|
369
|
+
#### L27. `FilterDropdownComponent` uses inline `onchange` instead of Stimulus action
|
|
370
|
+
- `filter_dropdown_component.rb:48,55` — short
|
|
371
|
+
|
|
372
|
+
### Testing
|
|
373
|
+
|
|
374
|
+
#### L28. Duplicated `configure_authentication` helper across 4 test files
|
|
375
|
+
- 4 test files — quick
|
|
376
|
+
|
|
377
|
+
#### ~~L29. Duplicated SolidQueue table purge logic~~ PARTIALLY RESOLVED
|
|
378
|
+
- ~~`54617b8` created SystemTestHelpers module with shared purge method. Some lib tests may still inline it.~~
|
|
379
|
+
|
|
380
|
+
#### L30. No test files for FetchLogsController, ScrapeLogsController, ImportHistory model
|
|
381
|
+
- Missing test files — short
|
|
382
|
+
|
|
383
|
+
---
|
|
384
|
+
|
|
385
|
+
## Top 10 Actions (prioritized by impact/effort ratio)
|
|
386
|
+
|
|
387
|
+
| # | Finding | Severity | Effort | Category |
|
|
388
|
+
|---|---------|----------|--------|----------|
|
|
389
|
+
| 1 | **H1** — Fix LogCleanupJob orphaned LogEntry records | HIGH | short | Data integrity |
|
|
390
|
+
| 2 | **H5** — Remove duplicated scrape rate-limiting from ScrapeItemJob | HIGH | quick | DRY |
|
|
391
|
+
| 3 | **M6** — Extract `set_source` into shared concern | MEDIUM | quick | DRY |
|
|
392
|
+
| 4 | **M1+M2** — Align `health_status` default + add validation | MEDIUM | quick | Correctness |
|
|
393
|
+
| 5 | **M4** — Move `sync_log_entry` callback into Loggable concern | MEDIUM | quick | DRY |
|
|
394
|
+
| 6 | **M5** — Add `rescue_from RecordNotFound` | MEDIUM | short | Robustness |
|
|
395
|
+
| 7 | **H4** — Extract DownloadContentImagesJob orchestration | HIGH | short | Convention |
|
|
396
|
+
| 8 | **H6** — Fix pagination test parallel-safety | HIGH | medium | Test reliability |
|
|
397
|
+
| 9 | **H2** — Extract ImportOpmlJob business logic to service | HIGH | medium | Convention |
|
|
398
|
+
| 10 | **M20** — Create StatusBadgeComponent | MEDIUM | medium | DRY/Consistency |
|
|
399
|
+
|
|
400
|
+
> **Previously in Top 10, now resolved:** M9 (SourcesController#index metrics — extracted to presenters), M16 (retry logic consolidation — extracted to RetryOrchestrator service)
|
|
401
|
+
|
|
402
|
+
---
|
|
403
|
+
|
|
404
|
+
## Positive Observations
|
|
405
|
+
|
|
406
|
+
The audit identified many areas where the codebase excels:
|
|
407
|
+
|
|
408
|
+
- **CRUD route design** is textbook — every action is a resource (`source_fetches`, `source_retries`, etc.)
|
|
409
|
+
- **Loggable concern** is exemplary single-purpose shared behavior
|
|
410
|
+
- **Boolean usage** correctly follows state-as-records convention (all booleans are technical flags)
|
|
411
|
+
- **Association declarations** include `inverse_of`, `dependent: :destroy`, and thorough indexing
|
|
412
|
+
- **Factory helpers** (`ModelFactories`) have good defaults with `SecureRandom` for parallel safety
|
|
413
|
+
- **VCR/WebMock separation** is clean (VCR for real feeds, WebMock for controlled scenarios)
|
|
414
|
+
- **Source turbo responses** concern is well-focused on response rendering
|
|
415
|
+
- **Table styling** is consistent across all views
|
|
416
|
+
- **Test conventions** are documented in `TEST_CONVENTIONS.md` with clear guidance
|
|
417
|
+
- **Thread-safe config reset** with `SourceMonitor.reset_configuration!`
|
|
418
|
+
- **Active Storage guarding** with `if defined?(ActiveStorage)` checks
|
|
419
|
+
- **Strong params** via `Sources::Params.sanitize` with explicit allowlist
|
|
420
|
+
- **Pipeline architecture** — all service objects are justified multi-model orchestrators
|
|
421
|
+
|
|
422
|
+
---
|
|
423
|
+
|
|
424
|
+
*Generated by `/rails-audit` command. Re-run to refresh findings.*
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.12.0
|