source_monitor 0.3.3 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (68) hide show
  1. checksums.yaml +4 -4
  2. data/.claude/commands/release.md +101 -58
  3. data/.claude/skills/sm-configure/SKILL.md +13 -2
  4. data/.claude/skills/sm-configure/reference/configuration-reference.md +33 -0
  5. data/.claude/skills/sm-host-setup/SKILL.md +15 -1
  6. data/.claude/skills/sm-host-setup/reference/setup-checklist.md +33 -0
  7. data/.claude/skills/sm-job/SKILL.md +1 -1
  8. data/.vbw-planning/REQUIREMENTS.md +22 -0
  9. data/.vbw-planning/ROADMAP.md +125 -0
  10. data/.vbw-planning/STATE.md +43 -0
  11. data/.vbw-planning/config.json +3 -1
  12. data/.vbw-planning/discovery.json +3 -1
  13. data/.vbw-planning/phases/01-generator-steps/01-CONTEXT.md +33 -0
  14. data/.vbw-planning/phases/01-generator-steps/01-VERIFICATION.md +86 -0
  15. data/.vbw-planning/phases/01-generator-steps/PLAN-01-SUMMARY.md +61 -0
  16. data/.vbw-planning/phases/01-generator-steps/PLAN-01.md +380 -0
  17. data/.vbw-planning/phases/02-verification/02-VERIFICATION.md +78 -0
  18. data/.vbw-planning/phases/02-verification/PLAN-01-SUMMARY.md +46 -0
  19. data/.vbw-planning/phases/02-verification/PLAN-01.md +500 -0
  20. data/.vbw-planning/phases/03-docs-alignment/03-VERIFICATION.md +89 -0
  21. data/.vbw-planning/phases/03-docs-alignment/PLAN-01-SUMMARY.md +48 -0
  22. data/.vbw-planning/phases/03-docs-alignment/PLAN-01.md +456 -0
  23. data/.vbw-planning/phases/04-dashboard-ux/04-VERIFICATION.md +129 -0
  24. data/.vbw-planning/phases/04-dashboard-ux/PLAN-01-SUMMARY.md +70 -0
  25. data/.vbw-planning/phases/04-dashboard-ux/PLAN-01.md +747 -0
  26. data/.vbw-planning/phases/05-active-storage-images/05-VERIFICATION.md +156 -0
  27. data/.vbw-planning/phases/05-active-storage-images/PLAN-01-SUMMARY.md +69 -0
  28. data/.vbw-planning/phases/05-active-storage-images/PLAN-01.md +455 -0
  29. data/.vbw-planning/phases/05-active-storage-images/PLAN-02-SUMMARY.md +39 -0
  30. data/.vbw-planning/phases/05-active-storage-images/PLAN-02.md +488 -0
  31. data/.vbw-planning/phases/06-netflix-feed-fix/06-VERIFICATION.md +100 -0
  32. data/.vbw-planning/phases/06-netflix-feed-fix/PLAN-01-SUMMARY.md +37 -0
  33. data/.vbw-planning/phases/06-netflix-feed-fix/PLAN-01.md +345 -0
  34. data/CHANGELOG.md +31 -0
  35. data/Gemfile.lock +1 -1
  36. data/VERSION +1 -1
  37. data/app/assets/builds/source_monitor/application.css +9 -0
  38. data/app/helpers/source_monitor/application_helper.rb +38 -0
  39. data/app/jobs/source_monitor/download_content_images_job.rb +72 -0
  40. data/app/models/source_monitor/item_content.rb +2 -0
  41. data/app/views/source_monitor/dashboard/_recent_activity.html.erb +9 -0
  42. data/app/views/source_monitor/items/_details.html.erb +2 -2
  43. data/app/views/source_monitor/logs/index.html.erb +9 -0
  44. data/app/views/source_monitor/sources/_details.html.erb +2 -2
  45. data/app/views/source_monitor/sources/_row.html.erb +1 -1
  46. data/docs/setup.md +10 -1
  47. data/docs/troubleshooting.md +38 -7
  48. data/lib/generators/source_monitor/install/install_generator.rb +101 -0
  49. data/lib/source_monitor/configuration/http_settings.rb +7 -1
  50. data/lib/source_monitor/configuration/images_settings.rb +37 -0
  51. data/lib/source_monitor/configuration.rb +3 -1
  52. data/lib/source_monitor/dashboard/queries/recent_activity_query.rb +16 -7
  53. data/lib/source_monitor/dashboard/recent_activity.rb +1 -0
  54. data/lib/source_monitor/dashboard/recent_activity_presenter.rb +15 -2
  55. data/lib/source_monitor/fetching/feed_fetcher/entry_processor.rb +13 -0
  56. data/lib/source_monitor/http.rb +23 -0
  57. data/lib/source_monitor/images/content_rewriter.rb +81 -0
  58. data/lib/source_monitor/images/downloader.rb +82 -0
  59. data/lib/source_monitor/logs/table_presenter.rb +25 -0
  60. data/lib/source_monitor/setup/procfile_patcher.rb +31 -0
  61. data/lib/source_monitor/setup/queue_config_patcher.rb +84 -0
  62. data/lib/source_monitor/setup/verification/recurring_schedule_verifier.rb +102 -0
  63. data/lib/source_monitor/setup/verification/runner.rb +1 -1
  64. data/lib/source_monitor/setup/verification/solid_queue_verifier.rb +1 -1
  65. data/lib/source_monitor/setup/workflow.rb +10 -0
  66. data/lib/source_monitor/version.rb +1 -1
  67. data/lib/source_monitor.rb +8 -0
  68. metadata +31 -1
@@ -0,0 +1,89 @@
1
+ ---
2
+ phase: 3
3
+ plan: "01"
4
+ tier: standard
5
+ result: PASS
6
+ passed: 22
7
+ failed: 0
8
+ total: 22
9
+ date: 2026-02-12
10
+ ---
11
+
12
+ ## Must-Have Checks
13
+
14
+ | # | Truth/Condition | Status | Evidence |
15
+ |---|----------------|--------|----------|
16
+ | 1 | grep 'Procfile.dev' .claude/skills/sm-host-setup/SKILL.md returns matches describing automatic patching, not manual steps | PASS | 4 matches found: lines 76 (automation note), 107 (patches action), 193 (patcher file), 236 (auto-handled checklist) |
17
+ | 2 | grep 'recurring_schedule' .claude/skills/sm-host-setup/SKILL.md returns matches describing automatic wiring, not manual configuration | PASS | 4 matches found: lines 77, 110-111 (dispatcher patching action), 237 (auto-handled checklist) |
18
+ | 3 | grep 'automatically' .claude/skills/sm-host-setup/reference/setup-checklist.md returns matches in Phase 6 worker config section | PASS | 3 matches found: lines 79, 105 (Phase 6a), 118 (Phase 6b) |
19
+ | 4 | grep 'recurring_schedule' .claude/skills/sm-configure/SKILL.md returns matches referencing automatic dispatcher wiring | PASS | 1 match found: line 150 checklist item marked [x] with "(handled by install generator)" |
20
+ | 5 | grep 'automatically' docs/setup.md returns matches describing generator automation of Procfile.dev and queue.yml | PASS | 4 matches found: lines 52, 109, 110, 111 describing automatic worker configuration |
21
+ | 6 | grep 'RecurringScheduleVerifier' docs/troubleshooting.md returns matches in recurring jobs and diagnostics sections | PASS | 1 match found: line 30 Issue #4 diagnostics section |
22
+ | 7 | The sm-host-setup SKILL.md "What the Install Generator Does" section lists 5 actions (not 3) | PASS | Line 93: "performs five actions" with all 5 listed (mount, initializer, recurring jobs, Procfile.dev, queue.yml dispatcher) |
23
+ | 8 | The sm-host-setup SKILL.md checklist shows Procfile.dev and dispatcher items as auto-handled (not manual checkboxes) | PASS | Lines 236-237: both marked [x] with "(handled by generator)" |
24
+ | 9 | The setup-checklist.md Phase 6a and 6b sections note that the generator handles these automatically | PASS | Phase 6a title (line 103): "(Automatic)", Phase 6b title (line 116): "(Automatic)" with generator behavior descriptions |
25
+ | 10 | The docs/setup.md manual installation steps 6a and 6b note that the generator handles these automatically | PASS | Lines 110-111: Procfile.dev and recurring schedule bullets describe generator automation with verification guidance |
26
+ | 11 | The docs/troubleshooting.md Issue #4 and #5 mention running the generator as the primary fix | PASS | Issue #4 line 26: "Primary fix" with generator command; Issue #5 line 43: "Primary fix" with generator command |
27
+
28
+ ## Artifact Checks
29
+
30
+ | Artifact | Exists | Contains | Status |
31
+ |----------|--------|----------|--------|
32
+ | .claude/skills/sm-host-setup/SKILL.md | YES | "Patches Procfile.dev" | PASS |
33
+ | .claude/skills/sm-host-setup/reference/setup-checklist.md | YES | "generator handles this automatically" | PASS |
34
+ | .claude/skills/sm-configure/SKILL.md | YES | "recurring_schedule" | PASS |
35
+ | docs/setup.md | YES | "generator automatically" | PASS |
36
+ | docs/troubleshooting.md | YES | "RecurringScheduleVerifier" | PASS |
37
+
38
+ ## Key Link Checks
39
+
40
+ | From | To | Via | Status |
41
+ |------|----|----|--------|
42
+ | .claude/skills/sm-host-setup/SKILL.md | REQ-21 | sm-host-setup skill reflects new generator capabilities (5 actions, auto Procfile.dev and queue.yml) | PASS |
43
+ | .claude/skills/sm-configure/SKILL.md | REQ-21 | sm-configure skill references automatic recurring_schedule wiring | PASS |
44
+ | docs/setup.md | REQ-21 | Setup docs updated to note generator handles both automatically | PASS |
45
+ | docs/troubleshooting.md | REQ-21 | Troubleshooting updated with improved diagnostics (RecurringScheduleVerifier) | PASS |
46
+ | .claude/skills/sm-host-setup/reference/setup-checklist.md | REQ-21 | Setup checklist reflects automation (Phase 6a/6b Automatic) | PASS |
47
+
48
+ ## Anti-Pattern Scan
49
+
50
+ | Pattern | Found | Location | Severity |
51
+ |---------|-------|----------|----------|
52
+ | Manual "add a jobs: entry to Procfile.dev" instructions | NO | Searched .claude/skills and docs | INFO |
53
+ | Manual "add recurring_schedule to" instructions | NO | All replaced with generator automation notes | INFO |
54
+ | Missing re-run guidance after automation note | NO | All automation sections include re-run instructions | INFO |
55
+
56
+ ## Convention Compliance
57
+
58
+ | Convention | File | Status | Detail |
59
+ |------------|------|--------|--------|
60
+ | Maintenance rule: skills aligned with engine code | All 6 modified files | PASS | Skills updated to match generator capabilities from Phase 1 |
61
+ | RuboCop check | N/A | N/A | Markdown files not subject to RuboCop |
62
+
63
+ ## Summary
64
+
65
+ **Tier:** standard (22 checks executed)
66
+
67
+ **Result:** PASS
68
+
69
+ **Passed:** 22/22
70
+
71
+ **Failed:** None
72
+
73
+ **Details:**
74
+
75
+ All must_haves verified successfully:
76
+ - All 6 files modified as specified in the plan
77
+ - sm-host-setup SKILL.md expanded from 3 to 5 generator actions
78
+ - Procfile.dev and dispatcher checklist items marked as auto-handled
79
+ - Phase 6a/6b in setup-checklist.md now titled "(Automatic)" with generator behavior descriptions
80
+ - RecurringScheduleVerifier referenced in setup-checklist.md and troubleshooting.md Issue #4
81
+ - sm-configure SKILL.md added dispatcher recurring_schedule checklist item
82
+ - sm-job SKILL.md mentions automatic queue.yml patching
83
+ - docs/setup.md guided and manual sections describe generator automation
84
+ - docs/troubleshooting.md Issues #4 and #5 use generator as primary fix with diagnostics sections
85
+ - No manual "add this to your file" instructions found (all replaced with automation notes)
86
+ - Commit 7978d61 exists with correct message and 6 file changes
87
+ - All key links to REQ-21 verified
88
+
89
+ Phase 3 execution completed with 100% alignment between generator capabilities (Phase 1), verification tooling (Phase 2), and all consumer-facing documentation.
@@ -0,0 +1,48 @@
1
+ # PLAN-01 Summary: skills-docs-alignment
2
+
3
+ ## Status: COMPLETE
4
+
5
+ ## What Was Done
6
+
7
+ ### Task 1: Update sm-host-setup SKILL.md
8
+ - Replaced manual Procfile.dev/queue.yml comments (lines 76-80) with automation note and re-run guidance
9
+ - Expanded "What the Install Generator Does" from 3 to 5 actions: added Procfile.dev patching and queue.yml dispatcher patching
10
+ - Marked Procfile.dev and dispatcher checklist items as `[x]` with "(handled by generator)"
11
+ - Added `procfile_patcher.rb` and `queue_config_patcher.rb` to Key Source Files table
12
+
13
+ ### Task 2: Update setup-checklist.md
14
+ - Phase 6a renamed to "Procfile.dev for Development (Automatic)" with generator behavior description
15
+ - Phase 6b renamed to "Recurring Schedule Dispatcher Wiring (Automatic)" with generator behavior description
16
+ - Both sections include verification guidance and "re-run" instructions
17
+ - Phase 6b references RecurringScheduleVerifier diagnostics
18
+ - Checklist items marked `[x]` with "(handled by generator)"
19
+
20
+ ### Task 3: Update sm-configure and sm-job Skills
21
+ - sm-configure SKILL.md: updated queue names checklist entry, added `[x]` dispatcher `recurring_schedule` item
22
+ - sm-job SKILL.md: recurring jobs section now mentions automatic `config/queue.yml` patching and idempotency
23
+
24
+ ### Task 4: Update docs/setup.md
25
+ - Guided Setup: replaced manual Procfile.dev and queue.yml instructions with 3-bullet automatic summary and re-run guidance
26
+ - Manual Installation Quick Reference: steps 6a/6b descriptions changed to "Handled by generator"
27
+ - Step-by-step details: Procfile.dev and recurring schedule bullets now describe generator automation with verification guidance
28
+
29
+ ### Task 5: Update docs/troubleshooting.md
30
+ - Issue #4 (Recurring Jobs Not Running): added "Primary fix" with generator command, "Diagnostics" with RecurringScheduleVerifier, kept manual fix as fallback
31
+ - Issue #5 (Jobs Not Processing with bin/dev): added "Primary fix" with generator command, "Diagnostics" with SolidQueueVerifier Procfile.dev suggestion
32
+
33
+ ## Files Modified
34
+ - `.claude/skills/sm-host-setup/SKILL.md` (5-action generator, auto-handled checklist, patcher files in table)
35
+ - `.claude/skills/sm-host-setup/reference/setup-checklist.md` (automated Phase 6a/6b, RecurringScheduleVerifier)
36
+ - `.claude/skills/sm-configure/SKILL.md` (dispatcher recurring_schedule checklist item)
37
+ - `.claude/skills/sm-job/SKILL.md` (automatic queue.yml patching mention)
38
+ - `docs/setup.md` (generator automation in guided and manual sections)
39
+ - `docs/troubleshooting.md` (generator-first remediation, verifier diagnostics)
40
+
41
+ ## Commit
42
+ - `7978d61` docs(03-docs-alignment): update skills and docs for generator automation
43
+
44
+ ## Requirements Satisfied
45
+ - REQ-21: All sm-* skills and documentation updated to reflect generator automation of Procfile.dev patching, queue.yml dispatcher wiring, and RecurringScheduleVerifier diagnostics
46
+
47
+ ## Deviations
48
+ None. All tasks executed as specified in the plan.
@@ -0,0 +1,456 @@
1
+ ---
2
+ phase: 3
3
+ plan: "01"
4
+ title: skills-docs-alignment
5
+ type: execute
6
+ wave: 1
7
+ depends_on: []
8
+ cross_phase_deps:
9
+ - phase: 1
10
+ artifact: "lib/generators/source_monitor/install/install_generator.rb"
11
+ reason: "Phase 1 added patch_procfile_dev and configure_queue_dispatcher -- docs must reflect these"
12
+ - phase: 2
13
+ artifact: "lib/source_monitor/setup/verification/recurring_schedule_verifier.rb"
14
+ reason: "Phase 2 added RecurringScheduleVerifier -- troubleshooting and verification docs must reference it"
15
+ autonomous: true
16
+ effort_override: thorough
17
+ skills_used: []
18
+ files_modified:
19
+ - .claude/skills/sm-host-setup/SKILL.md
20
+ - .claude/skills/sm-host-setup/reference/setup-checklist.md
21
+ - .claude/skills/sm-configure/SKILL.md
22
+ - .claude/skills/sm-job/SKILL.md
23
+ - docs/setup.md
24
+ - docs/troubleshooting.md
25
+ must_haves:
26
+ truths:
27
+ - "grep 'Procfile.dev' .claude/skills/sm-host-setup/SKILL.md returns matches that describe automatic patching, not manual steps"
28
+ - "grep 'recurring_schedule' .claude/skills/sm-host-setup/SKILL.md returns matches that describe automatic wiring, not manual configuration"
29
+ - "grep 'automatically' .claude/skills/sm-host-setup/reference/setup-checklist.md returns matches in the Phase 6 worker config section"
30
+ - "grep 'recurring_schedule' .claude/skills/sm-configure/SKILL.md returns matches referencing automatic dispatcher wiring"
31
+ - "grep 'automatically' docs/setup.md returns matches describing generator automation of Procfile.dev and queue.yml"
32
+ - "grep 'RecurringScheduleVerifier\\|recurring schedule' docs/troubleshooting.md returns matches in the recurring jobs and diagnostics sections"
33
+ - "The sm-host-setup SKILL.md 'What the Install Generator Does' section lists 5 actions (not 3)"
34
+ - "The sm-host-setup SKILL.md checklist shows Procfile.dev and dispatcher items as auto-handled (not manual checkboxes)"
35
+ - "The setup-checklist.md Phase 6a and 6b sections note that the generator handles these automatically"
36
+ - "The docs/setup.md manual installation steps 6a and 6b note that the generator handles these automatically"
37
+ - "The docs/troubleshooting.md Issue #4 and #5 mention running the generator as the primary fix"
38
+ artifacts:
39
+ - path: ".claude/skills/sm-host-setup/SKILL.md"
40
+ provides: "Updated skill reflecting 5-action generator with auto Procfile.dev and queue.yml patching"
41
+ contains: "Patches Procfile.dev"
42
+ - path: ".claude/skills/sm-host-setup/reference/setup-checklist.md"
43
+ provides: "Updated checklist with automated Phase 6a/6b"
44
+ contains: "generator handles this automatically"
45
+ - path: ".claude/skills/sm-configure/SKILL.md"
46
+ provides: "Updated configure skill referencing automatic dispatcher wiring"
47
+ contains: "recurring_schedule"
48
+ - path: "docs/setup.md"
49
+ provides: "Updated setup docs noting generator automation"
50
+ contains: "generator automatically"
51
+ - path: "docs/troubleshooting.md"
52
+ provides: "Updated troubleshooting with generator-first remediation and RecurringScheduleVerifier reference"
53
+ contains: "RecurringScheduleVerifier"
54
+ key_links:
55
+ - from: ".claude/skills/sm-host-setup/SKILL.md"
56
+ to: "REQ-21"
57
+ via: "sm-host-setup skill reflects new generator capabilities"
58
+ - from: ".claude/skills/sm-configure/SKILL.md"
59
+ to: "REQ-21"
60
+ via: "sm-configure skill references automatic recurring_schedule wiring"
61
+ - from: "docs/setup.md"
62
+ to: "REQ-21"
63
+ via: "Setup docs updated to note generator handles both automatically"
64
+ - from: "docs/troubleshooting.md"
65
+ to: "REQ-21"
66
+ via: "Troubleshooting updated with improved diagnostics"
67
+ - from: ".claude/skills/sm-host-setup/reference/setup-checklist.md"
68
+ to: "REQ-21"
69
+ via: "Setup checklist reflects automation"
70
+ ---
71
+ <objective>
72
+ Update all sm-* skills and documentation files to reflect that the install generator now automatically handles Procfile.dev patching and queue.yml dispatcher wiring (added in Phase 1). Remove manual instructions that are now automated. Update troubleshooting to reference the RecurringScheduleVerifier (added in Phase 2). REQ-21.
73
+ </objective>
74
+ <context>
75
+ @lib/generators/source_monitor/install/install_generator.rb -- The generator now has 5 public methods executed in order: (1) add_routes_mount, (2) create_initializer, (3) configure_recurring_jobs, (4) patch_procfile_dev, (5) configure_queue_dispatcher, plus print_next_steps. The docs currently describe only 3 actions. The Procfile.dev step creates the file with web: + jobs: entries if missing, appends jobs: if present without it, or skips if already there. The queue.yml step adds recurring_schedule to dispatchers or creates a default dispatcher section.
76
+
77
+ @lib/source_monitor/setup/workflow.rb -- The guided workflow now calls procfile_patcher.patch and queue_config_patcher.patch after the install generator runs and before verification. Both patchers are unconditional (no user prompt).
78
+
79
+ @lib/source_monitor/setup/verification/recurring_schedule_verifier.rb -- New verifier (Phase 2) that checks SolidQueue recurring tasks are registered. Warns when no SM tasks found, errors when SolidQueue unavailable. Wired into Runner.default_verifiers.
80
+
81
+ @lib/source_monitor/setup/verification/solid_queue_verifier.rb -- Remediation now mentions Procfile.dev (Phase 2 change).
82
+
83
+ @.claude/skills/sm-host-setup/SKILL.md -- Lines 76-80 still have manual Procfile/queue comments in the Manual Step-by-Step section. Lines 93-109 describe only 3 generator actions. Lines 230-231 have manual checklist items.
84
+
85
+ @.claude/skills/sm-host-setup/reference/setup-checklist.md -- Phase 6a (lines 103-115) and Phase 6b (lines 115-127) describe manual Procfile.dev and recurring schedule wiring.
86
+
87
+ @.claude/skills/sm-configure/SKILL.md -- Line 149 has queue names checklist item. Needs a note about automatic recurring_schedule wiring.
88
+
89
+ @.claude/skills/sm-job/SKILL.md -- Lines 162-171 describe recurring jobs. Could mention that the generator also wires the dispatcher automatically.
90
+
91
+ @docs/setup.md -- Lines 54-67 have manual Procfile.dev and queue.yml guidance that should note the generator handles these.
92
+
93
+ @docs/troubleshooting.md -- Issues #4 (lines 23-34) and #5 (lines 36-44) should recommend re-running the generator as the primary fix and mention bin/source_monitor verify for diagnostics.
94
+
95
+ **Rationale:** Phase 1 automated Procfile.dev and queue.yml patching. Phase 2 added RecurringScheduleVerifier for diagnostics. All docs and skills still describe the pre-Phase-1 manual workflow. This plan updates every consumer-facing document to reflect the current automated behavior, eliminating confusion for new users.
96
+ </context>
97
+ <tasks>
98
+ <task type="auto">
99
+ <name>update-sm-host-setup-skill</name>
100
+ <files>
101
+ .claude/skills/sm-host-setup/SKILL.md
102
+ </files>
103
+ <action>
104
+ Update the sm-host-setup SKILL.md in three areas:
105
+
106
+ **1. Manual Step-by-Step section (lines 73-84):**
107
+
108
+ Replace the manual comments on lines 76-80:
109
+ ```
110
+ # 5a. If your host uses bin/dev (foreman/overmind), add a jobs: entry to Procfile.dev:
111
+ # jobs: bundle exec rake solid_queue:start
112
+
113
+ # 5b. Ensure your dispatcher config in config/queue.yml includes
114
+ # recurring_schedule: config/recurring.yml so recurring jobs are loaded.
115
+ ```
116
+
117
+ With a note that these are now handled automatically:
118
+ ```
119
+ # Note: The generator automatically patches Procfile.dev with a jobs: entry
120
+ # and adds recurring_schedule to your queue.yml dispatcher config.
121
+ # Re-run the generator if these were not applied: bin/rails generate source_monitor:install
122
+ ```
123
+
124
+ **2. "What the Install Generator Does" section (lines 93-109):**
125
+
126
+ Update from "performs three actions" to "performs five actions" and add items 4 and 5:
127
+
128
+ 4. **Patches Procfile.dev** with a `jobs:` entry for Solid Queue:
129
+ Creates the file with `web:` and `jobs:` entries if it does not exist. Appends a `jobs:` entry if the file exists but lacks one. Skips if a `jobs:` entry is already present.
130
+
131
+ 5. **Patches queue.yml dispatcher** with `recurring_schedule: config/recurring.yml`:
132
+ Adds the `recurring_schedule` key to each dispatcher entry in `config/queue.yml`. If no dispatchers section exists, creates a default one. Skips if `recurring_schedule` is already configured. Skips if `config/queue.yml` does not exist.
133
+
134
+ **3. Checklist section (lines 222-236):**
135
+
136
+ Change lines 230-231 from manual checkboxes:
137
+ ```
138
+ - [ ] `Procfile.dev` includes `jobs:` entry for Solid Queue (for `bin/dev` usage)
139
+ - [ ] Dispatcher config includes `recurring_schedule: config/recurring.yml`
140
+ ```
141
+
142
+ To auto-handled items:
143
+ ```
144
+ - [x] `Procfile.dev` includes `jobs:` entry for Solid Queue (handled by generator)
145
+ - [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
146
+ ```
147
+
148
+ Also update the Key Source Files table to add the two new setup helpers:
149
+ ```
150
+ | `lib/source_monitor/setup/procfile_patcher.rb` | Procfile.dev patching for guided workflow |
151
+ | `lib/source_monitor/setup/queue_config_patcher.rb` | Queue config patching for guided workflow |
152
+ ```
153
+ </action>
154
+ <verify>
155
+ Read the modified file. Confirm: (a) manual comments on lines 76-80 are replaced with automation note, (b) "What the Install Generator Does" lists 5 actions, (c) checklist items for Procfile.dev and dispatcher are marked [x] with "(handled by generator)", (d) Key Source Files table includes the two new patcher files.
156
+ </verify>
157
+ <done>
158
+ sm-host-setup SKILL.md updated to reflect 5-action generator with automatic Procfile.dev and queue.yml handling. Manual steps replaced with automation notes. Checklist items marked as auto-handled.
159
+ </done>
160
+ </task>
161
+ <task type="auto">
162
+ <name>update-setup-checklist</name>
163
+ <files>
164
+ .claude/skills/sm-host-setup/reference/setup-checklist.md
165
+ </files>
166
+ <action>
167
+ Update the setup-checklist.md to reflect that Phase 6a and 6b are now automated by the generator.
168
+
169
+ **Phase 6a (lines 103-114):**
170
+
171
+ Replace the manual instruction block:
172
+ ```
173
+ ### Phase 6a: Wire Procfile.dev for Development
174
+
175
+ Most Rails 8 apps use `bin/dev` (via foreman or overmind) to start all processes. The host app's `Procfile.dev` must include a `jobs:` entry so Solid Queue workers start alongside the web server:
176
+
177
+ ```
178
+ # Procfile.dev
179
+ web: bin/rails server -p 3000
180
+ jobs: bundle exec rake solid_queue:start
181
+ ```
182
+
183
+ Without this line, `bin/dev` will start the web server but jobs will never process.
184
+ ```
185
+
186
+ With:
187
+ ```
188
+ ### Phase 6a: Procfile.dev for Development (Automatic)
189
+
190
+ The install generator automatically patches `Procfile.dev` with a `jobs:` entry for Solid Queue. If no `Procfile.dev` exists, it creates one with `web:` and `jobs:` entries. If the file exists but lacks a `jobs:` entry, it appends one. This is idempotent -- re-running the generator is safe.
191
+
192
+ Verify after running the generator:
193
+ ```
194
+ # Expected Procfile.dev content:
195
+ web: bin/rails server -p 3000
196
+ jobs: bundle exec rake solid_queue:start
197
+ ```
198
+
199
+ If the entry is missing, re-run: `bin/rails generate source_monitor:install`
200
+ ```
201
+
202
+ **Phase 6b (lines 115-127):**
203
+
204
+ Replace the manual instruction block:
205
+ ```
206
+ ### Phase 6b: Wire Recurring Schedule into Dispatcher
207
+
208
+ The install generator creates `config/recurring.yml` with SourceMonitor's recurring jobs, but the dispatcher must explicitly reference this file. In `config/queue.yml` (or `config/solid_queue.yml`), add `recurring_schedule` to the dispatchers section:
209
+
210
+ ```yaml
211
+ dispatchers:
212
+ - polling_interval: 1
213
+ batch_size: 500
214
+ recurring_schedule: config/recurring.yml
215
+ ```
216
+
217
+ Without this key, Solid Queue's dispatcher will not load recurring jobs even though the file exists. Sources will never auto-fetch and cleanup jobs will never fire.
218
+ ```
219
+
220
+ With:
221
+ ```
222
+ ### Phase 6b: Recurring Schedule Dispatcher Wiring (Automatic)
223
+
224
+ The install generator automatically patches `config/queue.yml` dispatchers with `recurring_schedule: config/recurring.yml`. If no dispatchers section exists, it creates a default one. This is idempotent -- re-running the generator is safe.
225
+
226
+ Verify after running the generator:
227
+ ```yaml
228
+ # Expected in config/queue.yml under dispatchers:
229
+ dispatchers:
230
+ - polling_interval: 1
231
+ batch_size: 500
232
+ recurring_schedule: config/recurring.yml
233
+ ```
234
+
235
+ If the key is missing, re-run: `bin/rails generate source_monitor:install`
236
+
237
+ **Diagnostics:** Run `bin/source_monitor verify` to check that recurring tasks are registered. The RecurringScheduleVerifier will warn if no SourceMonitor recurring tasks are found in Solid Queue.
238
+ ```
239
+
240
+ **Checklist items (lines 128-131):**
241
+
242
+ Update the checklist items to reflect automation:
243
+ ```
244
+ - [x] `Procfile.dev` includes a `jobs:` entry for Solid Queue (handled by generator)
245
+ - [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
246
+ ```
247
+ </action>
248
+ <verify>
249
+ Read the modified file. Confirm: (a) Phase 6a title includes "(Automatic)", (b) Phase 6b title includes "(Automatic)", (c) both sections describe generator behavior not manual steps, (d) both include "re-run" guidance, (e) Phase 6b mentions RecurringScheduleVerifier, (f) checklist items marked [x] with "(handled by generator)".
250
+ </verify>
251
+ <done>
252
+ setup-checklist.md updated with automated Phase 6a/6b sections. Manual steps replaced with generator behavior descriptions and verification guidance.
253
+ </done>
254
+ </task>
255
+ <task type="auto">
256
+ <name>update-sm-configure-and-sm-job-skills</name>
257
+ <files>
258
+ .claude/skills/sm-configure/SKILL.md
259
+ .claude/skills/sm-job/SKILL.md
260
+ </files>
261
+ <action>
262
+ **sm-configure SKILL.md:**
263
+
264
+ In the Checklist section (line 149), update:
265
+ ```
266
+ - [ ] Queue names match `config/solid_queue.yml` entries
267
+ ```
268
+
269
+ To:
270
+ ```
271
+ - [ ] Queue names match `config/queue.yml` (or `config/solid_queue.yml`) entries
272
+ - [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by install generator)
273
+ ```
274
+
275
+ This adds a new checklist item noting the automatic dispatcher wiring.
276
+
277
+ **sm-job SKILL.md:**
278
+
279
+ In the Recurring Jobs section (lines 162-171), update the text after the table:
280
+ ```
281
+ These run automatically with `bin/dev` or `bin/jobs`. If you need to customize, edit `config/recurring.yml` directly.
282
+ ```
283
+
284
+ To:
285
+ ```
286
+ The install generator automatically configures `config/recurring.yml` with these entries AND patches the `config/queue.yml` dispatcher with `recurring_schedule: config/recurring.yml` so recurring jobs load on startup. Both steps are idempotent. If you need to customize schedules, edit `config/recurring.yml` directly.
287
+ ```
288
+ </action>
289
+ <verify>
290
+ Read both modified files. Confirm: (a) sm-configure checklist has new dispatcher item marked [x], (b) sm-job recurring jobs section mentions automatic queue.yml patching.
291
+ </verify>
292
+ <done>
293
+ sm-configure and sm-job skills updated to reference automatic dispatcher wiring by the install generator.
294
+ </done>
295
+ </task>
296
+ <task type="auto">
297
+ <name>update-setup-docs</name>
298
+ <files>
299
+ docs/setup.md
300
+ </files>
301
+ <action>
302
+ Update docs/setup.md to reflect generator automation in two areas:
303
+
304
+ **1. Guided Setup section (lines 48-67):**
305
+
306
+ Replace the manual Procfile.dev and queue.yml guidance:
307
+ ```
308
+ 3. **Start background workers:**
309
+ ```bash
310
+ bin/rails solid_queue:start
311
+ ```
312
+ Recurring jobs (fetch scheduling, scraping, cleanup) are automatically configured in `config/recurring.yml` by the install generator. They'll run automatically with `bin/dev` or `bin/jobs`.
313
+
314
+ **For development with `bin/dev`:** Ensure `Procfile.dev` includes a `jobs:` entry so Solid Queue workers start alongside the web server:
315
+ ```
316
+ jobs: bundle exec rake solid_queue:start
317
+ ```
318
+
319
+ **For recurring jobs:** Ensure the dispatcher in `config/queue.yml` (or `config/solid_queue.yml`) references the recurring schedule:
320
+ ```yaml
321
+ dispatchers:
322
+ - polling_interval: 1
323
+ batch_size: 500
324
+ recurring_schedule: config/recurring.yml
325
+ ```
326
+ Without this key, Solid Queue will not load recurring jobs even though the file exists.
327
+ ```
328
+
329
+ With:
330
+ ```
331
+ 3. **Start background workers:**
332
+ ```bash
333
+ bin/rails solid_queue:start
334
+ ```
335
+ The install generator automatically handles all worker configuration:
336
+ - **Recurring jobs** are configured in `config/recurring.yml` (fetch scheduling, scraping, cleanup).
337
+ - **Procfile.dev** is patched with a `jobs:` entry so `bin/dev` starts Solid Queue alongside the web server.
338
+ - **Queue dispatcher** is patched with `recurring_schedule: config/recurring.yml` in `config/queue.yml` so recurring jobs load on startup.
339
+
340
+ All three steps are idempotent. If any configuration is missing, re-run: `bin/rails generate source_monitor:install`
341
+ ```
342
+
343
+ **2. Manual Installation step 6a/6b (lines 104-106, 119-120):**
344
+
345
+ In the Quick Reference table, update steps 6a and 6b descriptions:
346
+ - Step 6a: Change "Add `jobs:` line to `Procfile.dev`" to "Handled by generator (patches `Procfile.dev`)"
347
+ - Step 6b: Change "Add `recurring_schedule` to dispatcher config" to "Handled by generator (patches `config/queue.yml`)"
348
+
349
+ In the step-by-step details (line 119-120), update the bullet points:
350
+ ```
351
+ - **Procfile.dev:** If your host uses `bin/dev` (foreman/overmind), add a `jobs:` entry to `Procfile.dev`: `jobs: bundle exec rake solid_queue:start`. Without this, `bin/dev` will not start Solid Queue workers.
352
+ - **Recurring schedule:** Ensure the dispatcher in `config/queue.yml` (or `config/solid_queue.yml`) includes `recurring_schedule: config/recurring.yml`. Without this key, recurring jobs will not load even though the file exists.
353
+ ```
354
+
355
+ To:
356
+ ```
357
+ - **Procfile.dev:** The generator automatically patches `Procfile.dev` with a `jobs:` entry for Solid Queue. Verify the file contains `jobs: bundle exec rake solid_queue:start` after running the generator.
358
+ - **Recurring schedule:** The generator automatically patches `config/queue.yml` dispatchers with `recurring_schedule: config/recurring.yml`. Verify the key is present after running the generator.
359
+ ```
360
+ </action>
361
+ <verify>
362
+ Read the modified file. Confirm: (a) Guided Setup section describes 3 automatic steps (recurring.yml, Procfile.dev, queue dispatcher), (b) manual steps 6a/6b reference generator automation, (c) "re-run" guidance is present, (d) no leftover manual "add this to" instructions for Procfile.dev or queue.yml.
363
+ </verify>
364
+ <done>
365
+ docs/setup.md updated to note the generator handles Procfile.dev and queue.yml configuration automatically in both guided and manual sections.
366
+ </done>
367
+ </task>
368
+ <task type="auto">
369
+ <name>update-troubleshooting-docs</name>
370
+ <files>
371
+ docs/troubleshooting.md
372
+ </files>
373
+ <action>
374
+ Update docs/troubleshooting.md issues #4 and #5 to reference generator automation and the RecurringScheduleVerifier.
375
+
376
+ **Issue #4: Recurring Jobs Not Running (lines 23-35):**
377
+
378
+ Update to mention running the generator as the primary fix and the RecurringScheduleVerifier for diagnostics:
379
+
380
+ ```
381
+ ## 4. Recurring Jobs Not Running
382
+
383
+ - **Symptoms:** Fetch scheduling, scrape scheduling, and cleanup jobs never fire. Sources never auto-fetch on their configured intervals.
384
+ - **Primary fix:** Re-run the install generator, which automatically patches the dispatcher config:
385
+ ```bash
386
+ bin/rails generate source_monitor:install
387
+ ```
388
+ - **Diagnostics:** Run `bin/source_monitor verify` to check recurring task registration. The RecurringScheduleVerifier will report whether SourceMonitor recurring tasks are loaded into Solid Queue.
389
+ - **Manual check:** Verify `config/queue.yml` includes `recurring_schedule: config/recurring.yml` under the `dispatchers:` section. Without this key, Solid Queue's dispatcher will not load the recurring schedule even though `config/recurring.yml` exists.
390
+ - **Manual fix (if generator cannot patch):**
391
+ ```yaml
392
+ dispatchers:
393
+ - polling_interval: 1
394
+ batch_size: 500
395
+ recurring_schedule: config/recurring.yml
396
+ ```
397
+ ```
398
+
399
+ **Issue #5: Jobs Not Processing with bin/dev (lines 36-44):**
400
+
401
+ Update to mention running the generator as the primary fix:
402
+
403
+ ```
404
+ ## 5. Jobs Not Processing with bin/dev
405
+
406
+ - **Symptoms:** `bin/dev` starts the web server but jobs never run. Running `bin/rails solid_queue:start` manually works fine.
407
+ - **Primary fix:** Re-run the install generator, which automatically patches `Procfile.dev`:
408
+ ```bash
409
+ bin/rails generate source_monitor:install
410
+ ```
411
+ - **Diagnostics:** Run `bin/source_monitor verify` to check Solid Queue worker status. The SolidQueueVerifier will suggest Procfile.dev if no workers are detected.
412
+ - **Manual check:** Verify `Procfile.dev` includes a `jobs:` line:
413
+ ```
414
+ jobs: bundle exec rake solid_queue:start
415
+ ```
416
+ - Most Rails 8 apps use foreman or overmind via `bin/dev`. Without a `jobs:` entry, the process manager only starts the web server and asset watchers -- Solid Queue workers are not launched.
417
+ ```
418
+ </action>
419
+ <verify>
420
+ Read the modified file. Confirm: (a) Issue #4 has "Primary fix" mentioning the generator, (b) Issue #4 mentions RecurringScheduleVerifier, (c) Issue #5 has "Primary fix" mentioning the generator, (d) Issue #5 mentions SolidQueueVerifier and Procfile.dev, (e) both issues have "Diagnostics" sections referencing `bin/source_monitor verify`, (f) manual fixes are still present as fallback.
421
+ </verify>
422
+ <done>
423
+ docs/troubleshooting.md updated with generator-first remediation, RecurringScheduleVerifier diagnostics, and SolidQueueVerifier Procfile.dev suggestion for issues #4 and #5.
424
+ </done>
425
+ </task>
426
+ </tasks>
427
+ <verification>
428
+ 1. `grep -n 'performs five actions' .claude/skills/sm-host-setup/SKILL.md` returns a match
429
+ 2. `grep -n 'Patches Procfile.dev' .claude/skills/sm-host-setup/SKILL.md` returns a match
430
+ 3. `grep -n 'Patches queue.yml' .claude/skills/sm-host-setup/SKILL.md` returns a match
431
+ 4. `grep -n 'handled by generator' .claude/skills/sm-host-setup/SKILL.md` returns matches for both checklist items
432
+ 5. `grep -n 'Automatic' .claude/skills/sm-host-setup/reference/setup-checklist.md` returns matches for Phase 6a and 6b
433
+ 6. `grep -n 'handled by generator' .claude/skills/sm-host-setup/reference/setup-checklist.md` returns matches for checklist items
434
+ 7. `grep -n 'RecurringScheduleVerifier' .claude/skills/sm-host-setup/reference/setup-checklist.md` returns a match
435
+ 8. `grep -n 'recurring_schedule' .claude/skills/sm-configure/SKILL.md` returns a match in the checklist
436
+ 9. `grep -n 'patches.*queue.yml' .claude/skills/sm-job/SKILL.md` returns a match in the recurring jobs section
437
+ 10. `grep -n 'generator automatically' docs/setup.md` returns matches in both guided and manual sections
438
+ 11. `grep -n 'RecurringScheduleVerifier' docs/troubleshooting.md` returns a match in Issue #4
439
+ 12. `grep -n 'Primary fix' docs/troubleshooting.md` returns matches in Issues #4 and #5
440
+ 13. No manual "add a jobs: entry to Procfile.dev" instructions remain in any updated file (verified by grep absence)
441
+ </verification>
442
+ <success_criteria>
443
+ - sm-host-setup SKILL.md describes 5 generator actions including Procfile.dev and queue.yml patching (REQ-21)
444
+ - sm-host-setup SKILL.md checklist marks Procfile.dev and dispatcher items as auto-handled (REQ-21)
445
+ - setup-checklist.md Phase 6a/6b describe generator automation, not manual steps (REQ-21)
446
+ - setup-checklist.md references RecurringScheduleVerifier for diagnostics (REQ-21)
447
+ - sm-configure SKILL.md checklist references automatic recurring_schedule wiring (REQ-21)
448
+ - sm-job SKILL.md recurring jobs section mentions automatic queue.yml patching (REQ-21)
449
+ - docs/setup.md guided and manual sections note generator handles Procfile.dev and queue.yml (REQ-21)
450
+ - docs/troubleshooting.md Issue #4 recommends generator as primary fix and mentions RecurringScheduleVerifier (REQ-21)
451
+ - docs/troubleshooting.md Issue #5 recommends generator as primary fix and mentions SolidQueueVerifier (REQ-21)
452
+ - All manual "add this to your file" instructions replaced with "generator handles this" + verification guidance (REQ-21)
453
+ </success_criteria>
454
+ <output>
455
+ .vbw-planning/phases/03-docs-alignment/PLAN-01-SUMMARY.md
456
+ </output>