source_monitor 0.3.2 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 24dc578d91173b255c4d1a4ecc7d85542c5c2a86daabac9ee779b73c4f012ed5
-  data.tar.gz: 43f2ae2005109c50a5c3298c315a48092926411e02a8a4715459e39304324e14
+  metadata.gz: 69576ecf67e1a978b29bbc96f594a01cd33c7e5510600f696bb5aca1cf5c0740
+  data.tar.gz: a1d42ffe1fc8339e0657c915cdaf1765fe9518659978a20ad8a18a9861823145
 SHA512:
-  metadata.gz: 640830590b4e1638b0fd2e90c337f49f41c788948abd7a2552338a15e80b13560da320d449ec55fad12a6a9724916af2e1a0942129ca396e9c11f0ddb3abcc89
-  data.tar.gz: 5dae4fbfe5864a33fd2895a275f2511d014e5a94ccf49a19bc1f4d0c284bd9045a561c003e945fec3acf9565b116f868616c45566310857675f3b02d3768f0e8
+  metadata.gz: 2a3a5c00b0803cf7bef974d5b9507519221a00794a6f17398674e089221d1102fe4fef1b2e2d7cdecd30d3b9e9b0f44f50f62b04ebb46c96df7a9f3a52710fac
+  data.tar.gz: 1ec0ec3676faa8055c984be855b0e89096c47a9b59eabc5aeba5d806e00f70a8e098ad61c4daa707114e0570bd2c655cd853896524f3855e01bedd29491eaade

data/.claude/agent-memory/vbw-vbw-dev/MEMORY.md

@@ -0,0 +1,34 @@
+# VBW Dev Agent Memory
+
+## Project: source_monitor
+- Rails engine gem for RSS/feed monitoring
+- Ruby 3.4.4, Rails 8.x
+- Test suite: 473 tests via `bin/rails test` (takes ~76 seconds)
+- RuboCop uses `rubocop-rails-omakase` base config
+
+## Key Learnings
+
+### Shell/Bash in zsh
+- `!` in zsh inline scripts causes `command not found` errors. Use `case` statements instead of `if ! ...` patterns.
+- Use `IFS= read -r` when reading filenames from pipes to handle edge cases.
+
+### RuboCop scope vs git scope
+- RuboCop inspects ALL Ruby-like files (Gemfile, Rakefile, .gemspec, .rake, bin/*, config.ru), not just `.rb` files.
+- `git ls-files -- '*.rb'` only matches `.rb` extensions. Plans scoped to `.rb` files may miss RuboCop violations in non-`.rb` Ruby files.
+- `test/tmp/` contains untracked generated Rails app templates. These are NOT git-tracked but RuboCop scans them unless excluded.
+
+### File structure
+- `test/lib/tmp/install_generator/` contains test fixtures (1 tracked file: `config/initializers/source_monitor.rb`)
+- `test/tmp/host_app_template_*` directories are generated test artifacts, not git-tracked
+- `.rubocop.yml` is at project root, inherits from `rubocop-rails-omakase`
+
+### Frozen string literal pragma
+- Completed in commit `5f02db8` -- 113 files modified
+- Added `test/tmp/**/*` to RuboCop exclude list
+
+### RuboCop omakase ruleset details
+- Only 45 cops enabled (out of 775 available)
+- ALL Metrics cops disabled (ClassLength, MethodLength, BlockLength, etc.)
+- After frozen_string_literal fix, codebase had zero remaining violations
+- No `.rubocop.yml` exclusions needed for large files since Metrics cops are off
+- Plan 02 completed with no code changes required

data/.claude/agent-memory/vbw-vbw-lead/MEMORY.md

@@ -0,0 +1,49 @@
+# Lead Agent Memory -- SourceMonitor
+
+## Project Quick Facts
+- Rails 8 engine, Ruby 3.4+, PostgreSQL-only
+- 325 git-tracked Ruby files, 130 test files
+- Coverage baseline: 2328 uncovered lines in config/coverage_baseline.json
+- RuboCop: rubocop-rails-omakase, config at .rubocop.yml
+- CI: .github/workflows/ci.yml (lint, security, test, release-verify, nightly profiling)
+- Large files: FeedFetcher (627), Configuration (655), ImportSessionsController (792)
+
+## Phase 1 Findings
+- 98 git-tracked .rb files missing frozen_string_literal (out of 325)
+- Breakdown: app/(5), lib/(24), test/(43 non-dummy), test/dummy/(22), config/(1), db/migrate/(4)
+- test/tmp/ is NOT git-tracked (generated by install_generator tests) -- exclude from changes
+- test/lib/tmp/install_generator/config/initializers/source_monitor.rb IS tracked and already has pragma
+- bin/rubocop wrapper forces --config .rubocop.yml
+- Phase 1 criterion #3 (10% coverage shrink) not addressed by plans 01/02 -- noted for future
+
+## Phase 2 Planning Insights
+- Coverage baseline.json has 2117 uncovered lines across 105 files (not 2328 -- that was line count)
+- Top 7 files account for ~743 uncovered lines (35% of total)
+- FeedFetcher: 245 uncovered, 12 existing tests, uses VCR+WebMock
+- ItemCreator: 228 uncovered, 8 existing tests, needs mock entries for Atom/JSON branches
+- Configuration: 94 uncovered, 5 existing tests, ~12 nested settings classes
+- Dashboard::Queries: 66 uncovered, 7 existing tests, uses raw SQL + Cache
+- BulkSourceScraper: 66 uncovered, 6 existing tests, ActiveJob test adapter
+- Broadcaster: 48 uncovered, NO existing test file -- needs creation
+- SourcesIndexMetrics: 34 uncovered, 3 existing tests
+- All 5 Phase 2 plans are Wave 1 (no inter-plan dependencies, no file conflicts)
+- Testing main files will indirectly cover supporting files (retry_policy, fetch_error, state, enqueuer, etc.)
+- Broadcaster tests need Turbo::StreamsChannel stubs -- no full Action Cable required
+- 50% coverage target requires ~1059 lines covered; direct plans target ~630, rest from indirect
+
+## Phase 4 Planning Insights
+- 3 plans: conventions-audit (wave 1), item-creator-extraction (wave 1), final-verification (wave 2)
+- SourcesController has dead `fetch`/`retry` methods (leftover from Phase 3 CRUD extraction)
+- ImportSessionsController `new` and `create` are byte-for-byte identical
+- 4 RuboCop violations in db/migrate/20260210204022_add_composite_index_to_log_entries.rb
+- Duplicate test: test/controllers/concerns/ vs test/controllers/source_monitor/concerns/
+- ItemCreator at 601 lines is the last file over 300 lines -- extract to entry_parser + content_extractor
+- Coverage baseline still at 2117 uncovered (not regenerated since Phase 1) -- must regenerate
+- 60% reduction target: at most 847 uncovered lines
+- dashboard/queries.rb (356) and application_helper.rb (346) are near 300 but acceptable as view/query code
+- `bin/update-coverage-baseline` requires `COVERAGE=1 bin/rails test` first
+
+## Bash/Shell Notes
+- zsh on macOS -- `!` in shell conditionals causes `command not found` errors
+- Use `case` statements instead of `if ! grep` patterns in zsh
+- git ls-files piped to while loops works reliably for file enumeration

data/.claude/commands/release.md

@@ -0,0 +1,255 @@
+# Release: PR, CI, Merge, and Gem Build
+
+Orchestrate a full release cycle for the source_monitor gem. This command handles changelog generation, version bumping, PR creation, CI monitoring, auto-merge on success, release tagging, and gem build with push instructions.
+
+## Inputs
+
+- `$ARGUMENTS` -- Optional: version bump description or release notes summary. If empty, derive from commits since last tag.
+
+## Known Gotchas (Read Before Starting)
+
+These are real issues encountered in previous releases. Each step below accounts for them, but keep them in mind:
+
+1. **Two version files**: Both `lib/source_monitor/version.rb` AND the top-level `VERSION` file must be bumped. The VBW pre-push hook checks for changes to `VERSION` (the top-level file).
+2. **Gemfile.lock sync**: After bumping the version in `version.rb`, you MUST run `bundle install` to update `Gemfile.lock`. CI runs `bundle install --frozen` which fails if the lockfile is stale.
+3. **VBW volatile files**: Files in `.vbw-planning/` (`.cost-ledger.json`, `.notification-log.jsonl`, `.session-log.jsonl`, `.hook-errors.log`) are continuously modified by VBW hooks. They should be in `.gitignore`. If they aren't, add them before proceeding.
+4. **Pre-push hook**: The VBW pre-push hook at `.git/hooks/pre-push` requires the `VERSION` file to appear in the diff for any push. For new branches, it compares the commit against the working tree -- any dirty files will trigger a false positive. For force-pushes to existing branches where `VERSION` hasn't changed since the last push, use `--no-verify`.
+5. **Single squashed commit**: Always create ONE commit on the release branch with ALL changes (version bump, changelog, Gemfile.lock, any fixes). Multiple commits cause pre-push hook issues.
+6. **Diff coverage CI gate**: The `test` CI job enforces diff coverage. Any changed lines in source code (not just test files) must have test coverage. If you change source code during the release (e.g., bug fixes), you must add corresponding tests.
+7. **Local main divergence after merge**: After the PR merges, local main will have different commits than origin/main (pre-squash vs merged). You must `git reset --hard origin/main` to sync -- this requires user approval since the sandbox blocks it.
+
+## Step 1: Git Hygiene
+
+Run these checks. If ANY fail, STOP and report the issue to the user.
+
+1. **VBW volatile files gitignored**: Check that `.gitignore` contains these entries:
+   ```
+   .vbw-planning/.cost-ledger.json
+   .vbw-planning/.notification-log.jsonl
+   .vbw-planning/.session-log.jsonl
+   .vbw-planning/.hook-errors.log
+   ```
+   If any are missing, add them. If any of these files are tracked by git, remove them from tracking with `git rm --cached <file>`.
+
+2. **Working tree clean**: `git status --porcelain` must be empty (ignoring the VBW volatile files which should now be gitignored). If not, list the dirty files and ask the user whether to commit, stash, or abort.
+
+3. **On main branch**: `git branch --show-current` must be `main`. If not, ask the user if they want to continue from the current branch or switch.
+
+4. **Fetch latest**: `git fetch origin main`
+
+5. **Up to date with remote**: Compare `git rev-parse HEAD` with `git rev-parse origin/main`. If behind, ask the user whether to pull.
+
+6. **No unpushed commits**: Compare local HEAD with `origin/main`. If ahead, note how many commits are unpushed -- these will be included in the release.
+
+Report a summary:
+```
+Git Status:
+  Branch: main
+  Clean: yes
+  VBW files gitignored: yes
+  Synced with origin: yes
+  Unpushed commits: N
+```
+
+## Step 2: Version Check
+
+1. Read `lib/source_monitor/version.rb` to get the current VERSION constant.
+2. Read the top-level `VERSION` file to verify it matches. If they differ, sync them before proceeding.
+3. Read the latest git tag with `git tag --sort=-v:refname | head -1`.
+4. If the VERSION matches the latest tag (e.g., both are `0.3.2`), the version hasn't been bumped. Ask the user:
+   - "Current version is X.Y.Z which already has a tag. What should the new version be?"
+   - Offer options: patch (X.Y.Z+1), minor (X.Y+1.0), major (X+1.0.0), or custom.
+   - Update BOTH `lib/source_monitor/version.rb` AND the top-level `VERSION` file with the new version.
+5. If VERSION is ahead of the latest tag, proceed with the current version.
+
+Store the release version for later steps. Do NOT commit yet -- that happens after the changelog is updated.
+
+## Step 3: Update CHANGELOG.md
+
+The changelog follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) format. The file is at `CHANGELOG.md` in the project root.
+
+1. **Gather commit history** since the last tag:
+   ```
+   git log vPREVIOUS..HEAD --oneline --no-merges
+   ```
+2. **Categorize commits** into Keep a Changelog sections based on commit prefixes and content:
+   - `feat:`, `add:` --> **Added**
+   - `fix:`, `bugfix:` --> **Fixed**
+   - `chore:`, `refactor:`, `perf:` --> **Changed**
+   - `docs:` --> **Documentation** (only include if substantive)
+   - `BREAKING:` or `!:` --> **Breaking Changes** (at the top)
+   - `remove:`, `deprecate:` --> **Removed**
+   - Skip merge commits, version bumps, and CI-only changes.
+   - If `$ARGUMENTS` was provided, use it to inform/supplement the categorization.
+
+3. **Draft the changelog entry** and present it to the user for review:
+   ```
+   ## [X.Y.Z] - YYYY-MM-DD
+
+   ### Added
+   - <items>
+
+   ### Fixed
+   - <items>
+
+   ### Changed
+   - <items>
+   ```
+   Each bullet should be a concise, user-facing description (not a raw commit message). Consolidate related commits into single bullets where it makes sense.
+
+4. **Ask the user to approve** the changelog entry. Offer to edit if they want changes.
+
+5. **Write the entry** into `CHANGELOG.md`:
+   - Replace the `## [Unreleased]` section contents with `- No unreleased changes yet.`
+   - Insert the new versioned entry immediately after the `## [Unreleased]` block and before the previous release entry.
+   - Preserve all existing entries below.
+
+## Step 4: Sync Gemfile.lock
+
+**CRITICAL**: After updating `version.rb`, the gemspec version changes and `Gemfile.lock` becomes stale.
+
+1. Run `bundle install` to update `Gemfile.lock`.
+2. Verify the output shows the new version: `Using source_monitor X.Y.Z (was X.Y.Z-1)`.
+3. If `bundle install` fails, resolve the issue before proceeding.
+
+## Step 5: Create Release Branch with Single Squashed Commit
+
+**IMPORTANT**: All release changes MUST be in a single commit on the release branch. This avoids pre-push hook issues where individual commits are checked for VERSION changes.
+
+1. Create the release branch from main: `git checkout -b release/vX.Y.Z`
+2. Stage ALL release files in one commit:
+   ```
+   git add lib/source_monitor/version.rb VERSION CHANGELOG.md Gemfile.lock
+   ```
+   Also stage any other files that were changed (updated skills, docs, etc.).
+3. Create a single commit:
+   ```
+   chore: release vX.Y.Z
+   ```
+4. Push the branch: `git push -u origin release/vX.Y.Z`
+   - If the pre-push hook blocks with a false positive (e.g., VBW files dirty in working tree despite being gitignored), use `git push -u --no-verify origin release/vX.Y.Z`. This is safe because we've verified VERSION is in the commit.
+5. If the push fails for other reasons, diagnose and fix before proceeding.
+
+## Step 6: Create PR
+
+1. Create the PR using `gh pr create`:
+   - Title: `Release vX.Y.Z`
+   - Body format:
+     ```
+     ## Release vX.Y.Z
+
+     <paste the CHANGELOG.md entry content here>
+
+     ### Release Checklist
+     - [x] Version bumped in `lib/source_monitor/version.rb` and `VERSION`
+     - [x] CHANGELOG.md updated
+     - [x] Gemfile.lock synced
+     - [ ] CI passes (lint, security, test, release_verification)
+
+     ---
+     Auto-generated release PR by `/release` command.
+     ```
+   - Base: `main`
+2. Report the PR URL to the user.
+
+## Step 7: Monitor CI Pipeline
+
+Poll the CI status using repeated `gh pr checks <PR_NUMBER>` calls. The CI has 4 required jobs: `lint`, `security`, `test`, `release_verification` (release_verification only runs after test passes).
+
+1. Tell the user: "Monitoring CI pipeline for PR #N... This typically takes 3-5 minutes."
+2. Wait 30 seconds before first poll (give CI time to start).
+3. Poll with `gh pr checks <PR_NUMBER>` every 60 seconds, up to 15 minutes.
+4. After each poll, report progress if any jobs completed or failed.
+
+### If CI PASSES (all checks green):
+
+Continue to Step 8.
+
+### If CI FAILS:
+
+1. Get the failure details: `gh pr checks <PR_NUMBER>` to identify which jobs failed.
+2. For each failed job, fetch the log summary. Note: logs are only available after the **entire run** completes (not just one job). If `gh run view <RUN_ID> --log-failed` says "still in progress", wait and retry:
+   ```
+   gh run view <RUN_ID> --log-failed | tail -80
+   ```
+3. **Common failure: diff coverage** -- If the `test` job fails on "Enforce diff coverage", it means changed source lines lack test coverage. Read the error to identify uncovered files/lines, write tests, and add them to the release commit.
+4. **Common failure: Gemfile.lock frozen** -- If `bundle install` fails in CI with "frozen mode", you forgot to run `bundle install` locally (Step 4).
+5. Present failure details to the user and ask what to do:
+   - "Fix the issues and re-push" -- Fix issues, amend the commit (`git commit --amend --no-edit`), force push (`git push --force-with-lease --no-verify origin release/vX.Y.Z`), and restart CI monitoring.
+   - "Close the PR and abort" -- Close the PR, delete the branch, switch back to main.
+   - "Investigate manually" -- Stop and let the user handle it.
+
+**Note on force pushes**: When force-pushing the release branch after amending, always use `--no-verify` because the pre-push hook will see the diff between old and new branch tips, and `VERSION` won't appear as changed (it's the same in both). This is expected and safe.
+
+## Step 8: Auto-Merge PR
+
+Once CI is green:
+
+1. Merge the PR: `gh pr merge <PR_NUMBER> --merge --delete-branch`
+
+2. **Sync local main with remote** (this is the tricky part):
+   - The merge creates a merge commit on origin/main that doesn't exist locally.
+   - Local main may have different commits (pre-squash) than what was merged.
+   - Switch to main: `git checkout main`
+   - Try `git pull origin main`. If it fails with conflicts or divergence:
+     - Ask the user to run `git reset --hard origin/main` (the sandbox blocks this command).
+     - Explain this is safe because the PR is merged and all changes are on origin/main.
+   - Verify with `git log --oneline -3` that local matches remote.
+
+3. Report: "PR #N merged successfully."
+
+## Step 9: Tag the Release
+
+1. Verify you're on main and synced with origin.
+2. Create an annotated tag:
+   ```
+   git tag -a vX.Y.Z -m "Release vX.Y.Z"
+   ```
+3. Push the tag: `git push origin vX.Y.Z`
+4. Create a GitHub release from the tag:
+   ```
+   gh release create vX.Y.Z --title "vX.Y.Z" --notes "<changelog entry from Step 3>"
+   ```
+5. Report the release URL.
+
+## Step 10: Build the Gem
+
+1. Clean any old gem files: `ls source_monitor-*.gem` and remove them if found (don't error if none exist).
+2. Build the gem: `gem build source_monitor.gemspec`
+3. Verify the gem was built: check for `source_monitor-X.Y.Z.gem` in the project root.
+4. Show the file size: `ls -la source_monitor-X.Y.Z.gem`
+
+## Step 11: Gem Push Instructions
+
+Present the final instructions to the user:
+
+```
+Release vX.Y.Z Complete!
+
+  Git tag:    vX.Y.Z (pushed)
+  GitHub:     <release URL>
+  PR:         <PR URL> (merged)
+  Gem built:  source_monitor-X.Y.Z.gem
+
+To publish to RubyGems:
+
+  gem push source_monitor-X.Y.Z.gem
+
+You'll be prompted for your RubyGems OTP code (check your authenticator app).
+The gem has `rubygems_mfa_required` enabled, so the OTP is mandatory.
+
+After pushing, verify at:
+  https://rubygems.org/gems/source_monitor/versions/X.Y.Z
+```
+
+Do NOT run `gem push` automatically -- always let the user handle the OTP-protected push manually.
+
+## Error Recovery
+
+- If any step fails unexpectedly, report what happened and where things stand.
+- If a release branch already exists, ask the user whether to reuse or recreate it.
+- If the tag already exists, skip tagging and inform the user.
+- If the pre-push hook blocks, check whether `VERSION` (top-level) is in the commit diff. If it is and the hook still blocks (working tree drift from VBW), use `--no-verify`.
+- If local main diverges from origin after merge, ask the user to run `git reset --hard origin/main`.
+- Always leave the user on the `main` branch in a clean state when possible.
+- If `gem build` warns about duplicate URIs (homepage_uri/source_code_uri), this is cosmetic and safe to ignore.

data/.claude/skills/sm-configure/SKILL.md

@@ -29,7 +29,7 @@ After the block executes, `ModelExtensions.reload!` runs automatically to apply
 
 ## Configuration Sections
 
-The `config` object (`SourceMonitor::Configuration`) has 10 sub-sections plus top-level queue/job settings:
+The `config` object (`SourceMonitor::Configuration`) has 11 sub-sections plus top-level queue/job settings:
 
 | Section | Accessor | Class |
 |---|---|---|
@@ -44,6 +44,7 @@ The `config` object (`SourceMonitor::Configuration`) has 10 sub-sections plus to
 | Models | `config.models` | `Models` |
 | Realtime | `config.realtime` | `RealtimeSettings` |
 | Authentication | `config.authentication` | `AuthenticationSettings` |
+| Images | `config.images` | `ImagesSettings` |
 
 See `reference/configuration-reference.md` for every setting with types, defaults, and examples.
 
@@ -69,6 +70,14 @@ config.authentication.authenticate_with :authenticate_user!
 config.authentication.authorize_with ->(c) { c.current_user&.admin? }
 ```
 
+### Image Downloads (Active Storage)
+```ruby
+config.images.download_to_active_storage = true
+config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
+config.images.download_timeout = 15
+config.images.allowed_content_types = %w[image/jpeg image/png image/webp]
+```
+
 ### Events
 ```ruby
 config.events.after_item_created { |e| Notifier.new_item(e.item) }
@@ -116,6 +125,7 @@ SourceMonitor.mission_control_dashboard_path # Resolved MC path or nil
 | `lib/source_monitor/configuration/model_definition.rb` | Per-model definition |
 | `lib/source_monitor/configuration/realtime_settings.rb` | Action Cable settings |
 | `lib/source_monitor/configuration/authentication_settings.rb` | Auth settings |
+| `lib/source_monitor/configuration/images_settings.rb` | Image download settings |
 | `lib/source_monitor/configuration/validation_definition.rb` | Validation wrapper |
 
 ## References
@@ -146,7 +156,8 @@ end
 ## Checklist
 
 - [ ] Initializer exists at `config/initializers/source_monitor.rb`
-- [ ] Queue names match `config/solid_queue.yml` entries
+- [ ] Queue names match `config/queue.yml` (or `config/solid_queue.yml`) entries
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by install generator)
 - [ ] Authentication hooks configured for host auth system
 - [ ] HTTP timeouts appropriate for target feeds
 - [ ] Retention policy set for production

data/.claude/skills/sm-configure/reference/configuration-reference.md

@@ -309,6 +309,39 @@ end
 
 ---
 
+## Images Settings (`config.images`)
+
+Class: `SourceMonitor::Configuration::ImagesSettings`
+
+Controls background downloading of inline images from feed content to Active Storage.
+
+**Prerequisite:** The host app must have Active Storage installed (`rails active_storage:install` + migrations).
+
+| Setting | Type | Default | Description |
+|---|---|---|---|
+| `download_to_active_storage` | Boolean | `false` | Enable background image downloading for new items |
+| `max_download_size` | Integer | `10485760` (10 MB) | Maximum image file size in bytes; larger images are skipped |
+| `download_timeout` | Integer | `30` | HTTP timeout for image downloads in seconds |
+| `allowed_content_types` | Array | `["image/jpeg", "image/png", "image/gif", "image/webp", "image/svg+xml"]` | Permitted MIME types for downloaded images |
+
+### Helper Method
+
+| Method | Returns | Description |
+|---|---|---|
+| `download_enabled?` | Boolean | Returns `true` when `download_to_active_storage` is truthy |
+
+```ruby
+# Enable image downloading with custom limits
+config.images.download_to_active_storage = true
+config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
+config.images.download_timeout = 15
+config.images.allowed_content_types = %w[image/jpeg image/png image/webp]
+```
+
+When enabled, `DownloadContentImagesJob` is automatically enqueued after new items are created from feed entries. The job downloads inline `<img>` images from `item.content`, attaches them to `item_content.images` via Active Storage, and rewrites the HTML with Active Storage serving URLs. Failed downloads gracefully preserve the original image URL.
+
+---
+
 ## Environment Variables
 
 | Variable | Purpose |

data/.claude/skills/sm-host-setup/SKILL.md

@@ -61,7 +61,7 @@ bin/source_monitor install --yes
 gem "source_monitor", "~> 0.3.0"  # in Gemfile
 bundle install
 
-# 2. Run the install generator
+# 2. Run the install generator (mounts engine, creates initializer, configures recurring jobs)
 bin/rails generate source_monitor:install --mount-path=/source_monitor
 
 # 3. Copy engine migrations
@@ -70,9 +70,13 @@ bin/rails railties:install:migrations FROM=source_monitor
 # 4. Apply migrations
 bin/rails db:migrate
 
-# 5. Start background workers
+# 5. Start background workers (recurring jobs configured automatically in config/recurring.yml)
 bin/rails solid_queue:start
 
+# Note: The generator automatically patches Procfile.dev with a jobs: entry
+# and adds recurring_schedule to your queue.yml dispatcher config.
+# Re-run the generator if these were not applied: bin/rails generate source_monitor:install
+
 # 6. Verify
 bin/source_monitor verify
 ```
@@ -86,7 +90,7 @@ gem "source_monitor", github: "dchuk/source_monitor"
 
 ## What the Install Generator Does
 
-The generator (`SourceMonitor::Generators::InstallGenerator`) performs two actions:
+The generator (`SourceMonitor::Generators::InstallGenerator`) performs five actions:
 
 1. **Mounts the engine** in `config/routes.rb`:
    ```ruby
@@ -97,6 +101,15 @@ The generator (`SourceMonitor::Generators::InstallGenerator`) performs two actio
 2. **Creates the initializer** at `config/initializers/source_monitor.rb`:
    Uses the template at `lib/generators/source_monitor/install/templates/source_monitor.rb.tt`. Skips if the file already exists.
 
+3. **Configures recurring jobs** in `config/recurring.yml`:
+   Adds entries for `ScheduleFetchesJob` (every minute), scrape scheduling (every 2 minutes), `ItemCleanupJob` (2am daily), and `LogCleanupJob` (3am daily). If the file doesn't exist, creates it with `default: &default` and environment sections. If it exists, merges entries without overwriting. Skips if SourceMonitor entries are already present.
+
+4. **Patches Procfile.dev** with a `jobs:` entry for Solid Queue:
+   Creates the file with `web:` and `jobs:` entries if it does not exist. Appends a `jobs:` entry if the file exists but lacks one. Skips if a `jobs:` entry is already present.
+
+5. **Patches queue.yml dispatcher** with `recurring_schedule: config/recurring.yml`:
+   Adds the `recurring_schedule` key to each dispatcher entry in `config/queue.yml`. If no dispatchers section exists, creates a default one. Skips if `recurring_schedule` is already configured. Skips if `config/queue.yml` does not exist.
+
 Re-running the generator is safe and idempotent.
 
 ## Post-Install Configuration
@@ -177,6 +190,8 @@ config.authentication.user_signed_in_method = :user_signed_in?
 | `lib/generators/source_monitor/install/install_generator.rb` | Rails generator |
 | `lib/generators/source_monitor/install/templates/source_monitor.rb.tt` | Initializer template |
 | `lib/source_monitor/setup/initializer_patcher.rb` | Post-install patching |
+| `lib/source_monitor/setup/procfile_patcher.rb` | Procfile.dev patching for guided workflow |
+| `lib/source_monitor/setup/queue_config_patcher.rb` | Queue config patching for guided workflow |
 | `lib/source_monitor/setup/verification/runner.rb` | Verification runner |
 | `lib/source_monitor/engine.rb` | Engine configuration and initializers |
 | `docs/setup.md` | Full setup documentation |
@@ -217,6 +232,9 @@ end
 - [ ] `bundle install` completed
 - [ ] Install generator ran (`bin/rails generate source_monitor:install`)
 - [ ] Engine migrations copied and applied
+- [ ] Recurring jobs configured in `config/recurring.yml`
+- [x] `Procfile.dev` includes `jobs:` entry for Solid Queue (handled by generator)
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
 - [ ] Solid Queue workers started
 - [ ] Authentication hooks configured in initializer
 - [ ] `bin/source_monitor verify` passes

data/.claude/skills/sm-host-setup/reference/setup-checklist.md

@@ -39,6 +39,7 @@ bin/rails generate source_monitor:install --mount-path=/source_monitor
 Verify after running:
 - [ ] `config/routes.rb` contains `mount SourceMonitor::Engine, at: "/source_monitor"`
 - [ ] `config/initializers/source_monitor.rb` exists
+- [ ] `config/recurring.yml` contains SourceMonitor recurring job entries
 
 ## Phase 4: Database Setup
 
@@ -75,6 +76,8 @@ end
 
 ## Phase 6: Configure Workers
 
+The install generator automatically configures recurring jobs in `config/recurring.yml` (fetch scheduling, scrape scheduling, item cleanup, log cleanup). These run automatically with `bin/dev` or `bin/jobs`.
+
 Ensure `config/solid_queue.yml` (or equivalent) includes the SourceMonitor queues:
 
 ```yaml
@@ -83,6 +86,7 @@ production:
   dispatchers:
     - polling_interval: 1
       batch_size: 500
+      recurring_schedule: config/recurring.yml
   workers:
     - queues: "source_monitor_fetch"
       threads: 2
@@ -96,7 +100,39 @@ production:
 bin/rails solid_queue:start
 ```
 
+### Phase 6a: Procfile.dev for Development (Automatic)
+
+The install generator automatically patches `Procfile.dev` with a `jobs:` entry for Solid Queue. If no `Procfile.dev` exists, it creates one with `web:` and `jobs:` entries. If the file exists but lacks a `jobs:` entry, it appends one. This is idempotent -- re-running the generator is safe.
+
+Verify after running the generator:
+```
+# Expected Procfile.dev content:
+web: bin/rails server -p 3000
+jobs: bundle exec rake solid_queue:start
+```
+
+If the entry is missing, re-run: `bin/rails generate source_monitor:install`
+
+### Phase 6b: Recurring Schedule Dispatcher Wiring (Automatic)
+
+The install generator automatically patches `config/queue.yml` dispatchers with `recurring_schedule: config/recurring.yml`. If no dispatchers section exists, it creates a default one. This is idempotent -- re-running the generator is safe.
+
+Verify after running the generator:
+```yaml
+# Expected in config/queue.yml under dispatchers:
+dispatchers:
+  - polling_interval: 1
+    batch_size: 500
+    recurring_schedule: config/recurring.yml
+```
+
+If the key is missing, re-run: `bin/rails generate source_monitor:install`
+
+**Diagnostics:** Run `bin/source_monitor verify` to check that recurring tasks are registered. The RecurringScheduleVerifier will warn if no SourceMonitor recurring tasks are found in Solid Queue.
+
 - [ ] Queue configuration includes `source_monitor_fetch` and `source_monitor_scrape`
+- [x] `Procfile.dev` includes a `jobs:` entry for Solid Queue (handled by generator)
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
 - [ ] Workers started and processing
 
 ## Phase 7: Verify Installation

data/.claude/skills/sm-job/SKILL.md

@@ -159,15 +159,16 @@ SourceMonitor.queue_name(:fetch)  # => "source_monitor_fetch"
 
 ### Recurring Jobs
 
-ScheduleFetchesJob is typically configured as a recurring job in `config/recurring.yml`:
-
-```yaml
-production:
-  schedule_fetches:
-    class: SourceMonitor::ScheduleFetchesJob
-    schedule: every 1 minute
-    queue: source_monitor_fetch
-```
+The install generator (`bin/rails generate source_monitor:install`) automatically configures these recurring jobs in `config/recurring.yml`:
+
+| Job | Schedule |
+|-----|----------|
+| `SourceMonitor::ScheduleFetchesJob` | every minute |
+| `SourceMonitor::Scraping::Scheduler.run` | every 2 minutes |
+| `SourceMonitor::ItemCleanupJob` | at 2am every day |
+| `SourceMonitor::LogCleanupJob` | at 3am every day |
+
+The install generator automatically configures `config/recurring.yml` with these entries AND patches the `config/queue.yml` dispatcher with `recurring_schedule: config/recurring.yml` so recurring jobs load on startup. Both steps are idempotent. If you need to customize schedules, edit `config/recurring.yml` directly.
 
 ## Retry Policies
 

data/.gitignore

@@ -15,3 +15,7 @@
 /app/assets/builds/*
 !/app/assets/builds/.keep
 !/app/assets/builds/source_monitor/
+.vbw-planning/.cost-ledger.json
+.vbw-planning/.notification-log.jsonl
+.vbw-planning/.session-log.jsonl
+.vbw-planning/.hook-errors.log

data/.vbw-planning/REQUIREMENTS.md

@@ -32,6 +32,28 @@ Core value: Drop-in Rails engine for feed monitoring, content scraping, and oper
 - [ ] **REQ-14**: Audit and fix any RuboCop violations against omakase ruleset
 - [ ] **REQ-15**: Ensure all models, controllers, and service objects follow Rails conventions
 
+### Generator Enhancements
+
+- [ ] **REQ-16**: Install generator patches `Procfile.dev` with a `jobs:` entry for Solid Queue
+- [ ] **REQ-17**: Install generator patches queue config dispatcher with `recurring_schedule: config/recurring.yml`
+- [ ] **REQ-18**: Guided workflow (`Setup::Workflow`) integrates both new generator steps
+- [ ] **REQ-19**: `RecurringScheduleVerifier` checks that recurring tasks are registered with Solid Queue
+- [ ] **REQ-20**: `SolidQueueVerifier` remediation suggests `Procfile.dev` when workers not detected
+- [ ] **REQ-21**: Skills and documentation updated to reflect automated Procfile.dev and recurring_schedule setup
+
+### Dashboard UX
+
+- [ ] **REQ-22**: Fetch logs show source URL for both success and failure entries on the dashboard
+- [ ] **REQ-23**: Dashboard links to sources and items are clickable and open in a new tab
+
+### Active Storage Image Downloads
+
+- [ ] **REQ-24**: Configurable option to download inline images from items to Active Storage instead of loading from source
+
+### Feed Compatibility
+
+- [ ] **REQ-25**: Investigate and fix failing fetch for Netflix Tech Blog feed (https://netflixtechblog.com/feed)
+
 ## v2 Requirements
 
 - [ ] **REQ-XX**: Improve optional dependency loading with clear error messages