source_monitor 0.3.3 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ae2bf8e389d7c36d4f7061a13f7f4bb213c8365408f9d0c1fb818ea25b385ff7
-  data.tar.gz: a3b8b537ea6bb06251edbc53a203d9c9e9c20a1d555679a722f00ff62eef9967
+  metadata.gz: 69576ecf67e1a978b29bbc96f594a01cd33c7e5510600f696bb5aca1cf5c0740
+  data.tar.gz: a1d42ffe1fc8339e0657c915cdaf1765fe9518659978a20ad8a18a9861823145
 SHA512:
-  metadata.gz: 418df72b1757e7a941e645e171d00f012072c5f2974352851015cc3501adf6a546703a0b3cc38c35de7f1d6034b05c4bf13ce162815640a609612d395a40ac3d
-  data.tar.gz: bef273caaf974979aa4a9e5dce7c4b0043efa47149a0b3f2bcba80874f7b3ff303e5f4672e90ab176d79306a4085ff6ed692f883709a37490e7df279595803e7
+  metadata.gz: 2a3a5c00b0803cf7bef974d5b9507519221a00794a6f17398674e089221d1102fe4fef1b2e2d7cdecd30d3b9e9b0f44f50f62b04ebb46c96df7a9f3a52710fac
+  data.tar.gz: 1ec0ec3676faa8055c984be855b0e89096c47a9b59eabc5aeba5d806e00f70a8e098ad61c4daa707114e0570bd2c655cd853896524f3855e01bedd29491eaade

data/.claude/commands/release.md

@@ -6,34 +6,61 @@ Orchestrate a full release cycle for the source_monitor gem. This command handle
 
 - `$ARGUMENTS` -- Optional: version bump description or release notes summary. If empty, derive from commits since last tag.
 
+## Known Gotchas (Read Before Starting)
+
+These are real issues encountered in previous releases. Each step below accounts for them, but keep them in mind:
+
+1. **Two version files**: Both `lib/source_monitor/version.rb` AND the top-level `VERSION` file must be bumped. The VBW pre-push hook checks for changes to `VERSION` (the top-level file).
+2. **Gemfile.lock sync**: After bumping the version in `version.rb`, you MUST run `bundle install` to update `Gemfile.lock`. CI runs `bundle install --frozen` which fails if the lockfile is stale.
+3. **VBW volatile files**: Files in `.vbw-planning/` (`.cost-ledger.json`, `.notification-log.jsonl`, `.session-log.jsonl`, `.hook-errors.log`) are continuously modified by VBW hooks. They should be in `.gitignore`. If they aren't, add them before proceeding.
+4. **Pre-push hook**: The VBW pre-push hook at `.git/hooks/pre-push` requires the `VERSION` file to appear in the diff for any push. For new branches, it compares the commit against the working tree -- any dirty files will trigger a false positive. For force-pushes to existing branches where `VERSION` hasn't changed since the last push, use `--no-verify`.
+5. **Single squashed commit**: Always create ONE commit on the release branch with ALL changes (version bump, changelog, Gemfile.lock, any fixes). Multiple commits cause pre-push hook issues.
+6. **Diff coverage CI gate**: The `test` CI job enforces diff coverage. Any changed lines in source code (not just test files) must have test coverage. If you change source code during the release (e.g., bug fixes), you must add corresponding tests.
+7. **Local main divergence after merge**: After the PR merges, local main will have different commits than origin/main (pre-squash vs merged). You must `git reset --hard origin/main` to sync -- this requires user approval since the sandbox blocks it.
+
 ## Step 1: Git Hygiene
 
 Run these checks. If ANY fail, STOP and report the issue to the user.
 
-1. **Working tree clean**: `git status --porcelain` must be empty. If not, list the dirty files and ask the user whether to commit, stash, or abort.
-2. **On main branch**: `git branch --show-current` must be `main`. If not, ask the user if they want to continue from the current branch or switch.
-3. **Fetch latest**: `git fetch origin main`
-4. **Up to date with remote**: Compare `git rev-parse HEAD` with `git rev-parse origin/main`. If behind, ask the user whether to pull.
-5. **No unpushed commits**: Compare local HEAD with `origin/main`. If ahead, note how many commits are unpushed -- these will be included in the PR.
+1. **VBW volatile files gitignored**: Check that `.gitignore` contains these entries:
+   ```
+   .vbw-planning/.cost-ledger.json
+   .vbw-planning/.notification-log.jsonl
+   .vbw-planning/.session-log.jsonl
+   .vbw-planning/.hook-errors.log
+   ```
+   If any are missing, add them. If any of these files are tracked by git, remove them from tracking with `git rm --cached <file>`.
+
+2. **Working tree clean**: `git status --porcelain` must be empty (ignoring the VBW volatile files which should now be gitignored). If not, list the dirty files and ask the user whether to commit, stash, or abort.
+
+3. **On main branch**: `git branch --show-current` must be `main`. If not, ask the user if they want to continue from the current branch or switch.
+
+4. **Fetch latest**: `git fetch origin main`
+
+5. **Up to date with remote**: Compare `git rev-parse HEAD` with `git rev-parse origin/main`. If behind, ask the user whether to pull.
+
+6. **No unpushed commits**: Compare local HEAD with `origin/main`. If ahead, note how many commits are unpushed -- these will be included in the release.
 
 Report a summary:
 ```
 Git Status:
   Branch: main
   Clean: yes
+  VBW files gitignored: yes
   Synced with origin: yes
   Unpushed commits: N
 ```
 
 ## Step 2: Version Check
 
-1. Read `lib/source_monitor/version.rb` to get the current VERSION.
-2. Read the latest git tag with `git tag --sort=-v:refname | head -1`.
-3. If the VERSION matches the latest tag (e.g., both are `0.3.2`), the version hasn't been bumped. Ask the user:
+1. Read `lib/source_monitor/version.rb` to get the current VERSION constant.
+2. Read the top-level `VERSION` file to verify it matches. If they differ, sync them before proceeding.
+3. Read the latest git tag with `git tag --sort=-v:refname | head -1`.
+4. If the VERSION matches the latest tag (e.g., both are `0.3.2`), the version hasn't been bumped. Ask the user:
    - "Current version is X.Y.Z which already has a tag. What should the new version be?"
    - Offer options: patch (X.Y.Z+1), minor (X.Y+1.0), major (X+1.0.0), or custom.
-   - Update `lib/source_monitor/version.rb` with the new version.
-4. If VERSION is ahead of the latest tag, proceed with the current version.
+   - Update BOTH `lib/source_monitor/version.rb` AND the top-level `VERSION` file with the new version.
+5. If VERSION is ahead of the latest tag, proceed with the current version.
 
 Store the release version for later steps. Do NOT commit yet -- that happens after the changelog is updated.
 
@@ -77,29 +104,35 @@ The changelog follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) f
    - Insert the new versioned entry immediately after the `## [Unreleased]` block and before the previous release entry.
    - Preserve all existing entries below.
 
-## Step 4: Commit Version Bump + Changelog
+## Step 4: Sync Gemfile.lock
 
-Once both `lib/source_monitor/version.rb` (if changed) and `CHANGELOG.md` are updated:
+**CRITICAL**: After updating `version.rb`, the gemspec version changes and `Gemfile.lock` becomes stale.
 
-1. Stage the changed files:
-   ```
-   git add lib/source_monitor/version.rb CHANGELOG.md
-   ```
-2. Create a single commit:
+1. Run `bundle install` to update `Gemfile.lock`.
+2. Verify the output shows the new version: `Using source_monitor X.Y.Z (was X.Y.Z-1)`.
+3. If `bundle install` fails, resolve the issue before proceeding.
+
+## Step 5: Create Release Branch with Single Squashed Commit
+
+**IMPORTANT**: All release changes MUST be in a single commit on the release branch. This avoids pre-push hook issues where individual commits are checked for VERSION changes.
+
+1. Create the release branch from main: `git checkout -b release/vX.Y.Z`
+2. Stage ALL release files in one commit:
    ```
-   chore: bump version to X.Y.Z and update changelog
+   git add lib/source_monitor/version.rb VERSION CHANGELOG.md Gemfile.lock
    ```
-   If only the changelog changed (version was already bumped), use:
+   Also stage any other files that were changed (updated skills, docs, etc.).
+3. Create a single commit:
    ```
-   chore: update changelog for vX.Y.Z release
+   chore: release vX.Y.Z
    ```
+4. Push the branch: `git push -u origin release/vX.Y.Z`
+   - If the pre-push hook blocks with a false positive (e.g., VBW files dirty in working tree despite being gitignored), use `git push -u --no-verify origin release/vX.Y.Z`. This is safe because we've verified VERSION is in the commit.
+5. If the push fails for other reasons, diagnose and fix before proceeding.
 
-## Step 5: Create Release Branch and PR
+## Step 6: Create PR
 
-1. Create a release branch: `git checkout -b release/vX.Y.Z`
-2. Push the branch: `git push -u origin release/vX.Y.Z`
-3. Generate a PR body from the new CHANGELOG.md entry (use the entry written in Step 3, not raw commits).
-4. Create the PR using `gh pr create`:
+1. Create the PR using `gh pr create`:
    - Title: `Release vX.Y.Z`
    - Body format:
      ```
@@ -108,79 +141,85 @@ Once both `lib/source_monitor/version.rb` (if changed) and `CHANGELOG.md` are up
      <paste the CHANGELOG.md entry content here>
 
      ### Release Checklist
-     - [x] Version bumped in `lib/source_monitor/version.rb`
+     - [x] Version bumped in `lib/source_monitor/version.rb` and `VERSION`
      - [x] CHANGELOG.md updated
+     - [x] Gemfile.lock synced
      - [ ] CI passes (lint, security, test, release_verification)
 
      ---
      Auto-generated release PR by `/release` command.
      ```
    - Base: `main`
-5. Report the PR URL to the user.
+2. Report the PR URL to the user.
 
-## Step 6: Monitor CI Pipeline
+## Step 7: Monitor CI Pipeline
 
-Poll the CI status using `gh pr checks <PR_NUMBER> --watch` or repeated `gh pr checks <PR_NUMBER>` calls. The CI has 4 required jobs: `lint`, `security`, `test`, `release_verification`.
+Poll the CI status using repeated `gh pr checks <PR_NUMBER>` calls. The CI has 4 required jobs: `lint`, `security`, `test`, `release_verification` (release_verification only runs after test passes).
 
 1. Tell the user: "Monitoring CI pipeline for PR #N... This typically takes 3-5 minutes."
-2. Poll with `gh pr checks <PR_NUMBER>` every 30 seconds, up to 15 minutes.
-3. After each poll, report progress if any jobs completed.
+2. Wait 30 seconds before first poll (give CI time to start).
+3. Poll with `gh pr checks <PR_NUMBER>` every 60 seconds, up to 15 minutes.
+4. After each poll, report progress if any jobs completed or failed.
 
 ### If CI PASSES (all checks green):
 
-Continue to Step 7.
+Continue to Step 8.
 
 ### If CI FAILS:
 
 1. Get the failure details: `gh pr checks <PR_NUMBER>` to identify which jobs failed.
-2. For each failed job, fetch the log summary:
+2. For each failed job, fetch the log summary. Note: logs are only available after the **entire run** completes (not just one job). If `gh run view <RUN_ID> --log-failed` says "still in progress", wait and retry:
    ```
-   gh run view <RUN_ID> --log-failed | tail -50
+   gh run view <RUN_ID> --log-failed | tail -80
    ```
-3. Present to the user:
-   ```
-   CI Failed on PR #N
-
-   Failed Jobs:
-   - <job_name>: <brief summary of failure>
+3. **Common failure: diff coverage** -- If the `test` job fails on "Enforce diff coverage", it means changed source lines lack test coverage. Read the error to identify uncovered files/lines, write tests, and add them to the release commit.
+4. **Common failure: Gemfile.lock frozen** -- If `bundle install` fails in CI with "frozen mode", you forgot to run `bundle install` locally (Step 4).
+5. Present failure details to the user and ask what to do:
+   - "Fix the issues and re-push" -- Fix issues, amend the commit (`git commit --amend --no-edit`), force push (`git push --force-with-lease --no-verify origin release/vX.Y.Z`), and restart CI monitoring.
+   - "Close the PR and abort" -- Close the PR, delete the branch, switch back to main.
+   - "Investigate manually" -- Stop and let the user handle it.
 
-   Log excerpt:
-   <relevant log lines>
-   ```
-4. Ask the user what to do:
-   - "Fix the issues and re-push" -- Help fix the issues, commit, push, and restart CI monitoring
-   - "Close the PR and abort" -- Close the PR, delete the branch, switch back to main
-   - "Investigate manually" -- Stop and let the user handle it
+**Note on force pushes**: When force-pushing the release branch after amending, always use `--no-verify` because the pre-push hook will see the diff between old and new branch tips, and `VERSION` won't appear as changed (it's the same in both). This is expected and safe.
 
-## Step 7: Auto-Merge PR
+## Step 8: Auto-Merge PR
 
 Once CI is green:
 
 1. Merge the PR: `gh pr merge <PR_NUMBER> --merge --delete-branch`
-2. Switch back to main and pull: `git checkout main && git pull origin main`
+
+2. **Sync local main with remote** (this is the tricky part):
+   - The merge creates a merge commit on origin/main that doesn't exist locally.
+   - Local main may have different commits (pre-squash) than what was merged.
+   - Switch to main: `git checkout main`
+   - Try `git pull origin main`. If it fails with conflicts or divergence:
+     - Ask the user to run `git reset --hard origin/main` (the sandbox blocks this command).
+     - Explain this is safe because the PR is merged and all changes are on origin/main.
+   - Verify with `git log --oneline -3` that local matches remote.
+
 3. Report: "PR #N merged successfully."
 
-## Step 8: Tag the Release
+## Step 9: Tag the Release
 
-1. Create an annotated tag:
+1. Verify you're on main and synced with origin.
+2. Create an annotated tag:
    ```
    git tag -a vX.Y.Z -m "Release vX.Y.Z"
    ```
-2. Push the tag: `git push origin vX.Y.Z`
-3. Create a GitHub release from the tag:
+3. Push the tag: `git push origin vX.Y.Z`
+4. Create a GitHub release from the tag:
    ```
    gh release create vX.Y.Z --title "vX.Y.Z" --notes "<changelog entry from Step 3>"
    ```
-4. Report the release URL.
+5. Report the release URL.
 
-## Step 9: Build the Gem
+## Step 10: Build the Gem
 
-1. Clean any old gem files: remove any `source_monitor-*.gem` files in the project root.
+1. Clean any old gem files: `ls source_monitor-*.gem` and remove them if found (don't error if none exist).
 2. Build the gem: `gem build source_monitor.gemspec`
 3. Verify the gem was built: check for `source_monitor-X.Y.Z.gem` in the project root.
-4. Show the gem contents summary: `gem spec source_monitor-X.Y.Z.gem | head -30`
+4. Show the file size: `ls -la source_monitor-X.Y.Z.gem`
 
-## Step 10: Gem Push Instructions
+## Step 11: Gem Push Instructions
 
 Present the final instructions to the user:
 
@@ -189,6 +228,7 @@ Release vX.Y.Z Complete!
 
   Git tag:    vX.Y.Z (pushed)
   GitHub:     <release URL>
+  PR:         <PR URL> (merged)
   Gem built:  source_monitor-X.Y.Z.gem
 
 To publish to RubyGems:
@@ -209,4 +249,7 @@ Do NOT run `gem push` automatically -- always let the user handle the OTP-protec
 - If any step fails unexpectedly, report what happened and where things stand.
 - If a release branch already exists, ask the user whether to reuse or recreate it.
 - If the tag already exists, skip tagging and inform the user.
+- If the pre-push hook blocks, check whether `VERSION` (top-level) is in the commit diff. If it is and the hook still blocks (working tree drift from VBW), use `--no-verify`.
+- If local main diverges from origin after merge, ask the user to run `git reset --hard origin/main`.
 - Always leave the user on the `main` branch in a clean state when possible.
+- If `gem build` warns about duplicate URIs (homepage_uri/source_code_uri), this is cosmetic and safe to ignore.

data/.claude/skills/sm-configure/SKILL.md

@@ -29,7 +29,7 @@ After the block executes, `ModelExtensions.reload!` runs automatically to apply
 
 ## Configuration Sections
 
-The `config` object (`SourceMonitor::Configuration`) has 10 sub-sections plus top-level queue/job settings:
+The `config` object (`SourceMonitor::Configuration`) has 11 sub-sections plus top-level queue/job settings:
 
 | Section | Accessor | Class |
 |---|---|---|
@@ -44,6 +44,7 @@ The `config` object (`SourceMonitor::Configuration`) has 10 sub-sections plus to
 | Models | `config.models` | `Models` |
 | Realtime | `config.realtime` | `RealtimeSettings` |
 | Authentication | `config.authentication` | `AuthenticationSettings` |
+| Images | `config.images` | `ImagesSettings` |
 
 See `reference/configuration-reference.md` for every setting with types, defaults, and examples.
 
@@ -69,6 +70,14 @@ config.authentication.authenticate_with :authenticate_user!
 config.authentication.authorize_with ->(c) { c.current_user&.admin? }
 ```
 
+### Image Downloads (Active Storage)
+```ruby
+config.images.download_to_active_storage = true
+config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
+config.images.download_timeout = 15
+config.images.allowed_content_types = %w[image/jpeg image/png image/webp]
+```
+
 ### Events
 ```ruby
 config.events.after_item_created { |e| Notifier.new_item(e.item) }
@@ -116,6 +125,7 @@ SourceMonitor.mission_control_dashboard_path # Resolved MC path or nil
 | `lib/source_monitor/configuration/model_definition.rb` | Per-model definition |
 | `lib/source_monitor/configuration/realtime_settings.rb` | Action Cable settings |
 | `lib/source_monitor/configuration/authentication_settings.rb` | Auth settings |
+| `lib/source_monitor/configuration/images_settings.rb` | Image download settings |
 | `lib/source_monitor/configuration/validation_definition.rb` | Validation wrapper |
 
 ## References
@@ -146,7 +156,8 @@ end
 ## Checklist
 
 - [ ] Initializer exists at `config/initializers/source_monitor.rb`
-- [ ] Queue names match `config/solid_queue.yml` entries
+- [ ] Queue names match `config/queue.yml` (or `config/solid_queue.yml`) entries
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by install generator)
 - [ ] Authentication hooks configured for host auth system
 - [ ] HTTP timeouts appropriate for target feeds
 - [ ] Retention policy set for production

data/.claude/skills/sm-configure/reference/configuration-reference.md

@@ -309,6 +309,39 @@ end
 
 ---
 
+## Images Settings (`config.images`)
+
+Class: `SourceMonitor::Configuration::ImagesSettings`
+
+Controls background downloading of inline images from feed content to Active Storage.
+
+**Prerequisite:** The host app must have Active Storage installed (`rails active_storage:install` + migrations).
+
+| Setting | Type | Default | Description |
+|---|---|---|---|
+| `download_to_active_storage` | Boolean | `false` | Enable background image downloading for new items |
+| `max_download_size` | Integer | `10485760` (10 MB) | Maximum image file size in bytes; larger images are skipped |
+| `download_timeout` | Integer | `30` | HTTP timeout for image downloads in seconds |
+| `allowed_content_types` | Array | `["image/jpeg", "image/png", "image/gif", "image/webp", "image/svg+xml"]` | Permitted MIME types for downloaded images |
+
+### Helper Method
+
+| Method | Returns | Description |
+|---|---|---|
+| `download_enabled?` | Boolean | Returns `true` when `download_to_active_storage` is truthy |
+
+```ruby
+# Enable image downloading with custom limits
+config.images.download_to_active_storage = true
+config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
+config.images.download_timeout = 15
+config.images.allowed_content_types = %w[image/jpeg image/png image/webp]
+```
+
+When enabled, `DownloadContentImagesJob` is automatically enqueued after new items are created from feed entries. The job downloads inline `<img>` images from `item.content`, attaches them to `item_content.images` via Active Storage, and rewrites the HTML with Active Storage serving URLs. Failed downloads gracefully preserve the original image URL.
+
+---
+
 ## Environment Variables
 
 | Variable | Purpose |

data/.claude/skills/sm-host-setup/SKILL.md

@@ -73,6 +73,10 @@ bin/rails db:migrate
 # 5. Start background workers (recurring jobs configured automatically in config/recurring.yml)
 bin/rails solid_queue:start
 
+# Note: The generator automatically patches Procfile.dev with a jobs: entry
+# and adds recurring_schedule to your queue.yml dispatcher config.
+# Re-run the generator if these were not applied: bin/rails generate source_monitor:install
+
 # 6. Verify
 bin/source_monitor verify
 ```
@@ -86,7 +90,7 @@ gem "source_monitor", github: "dchuk/source_monitor"
 
 ## What the Install Generator Does
 
-The generator (`SourceMonitor::Generators::InstallGenerator`) performs three actions:
+The generator (`SourceMonitor::Generators::InstallGenerator`) performs five actions:
 
 1. **Mounts the engine** in `config/routes.rb`:
    ```ruby
@@ -100,6 +104,12 @@ The generator (`SourceMonitor::Generators::InstallGenerator`) performs three act
 3. **Configures recurring jobs** in `config/recurring.yml`:
    Adds entries for `ScheduleFetchesJob` (every minute), scrape scheduling (every 2 minutes), `ItemCleanupJob` (2am daily), and `LogCleanupJob` (3am daily). If the file doesn't exist, creates it with `default: &default` and environment sections. If it exists, merges entries without overwriting. Skips if SourceMonitor entries are already present.
 
+4. **Patches Procfile.dev** with a `jobs:` entry for Solid Queue:
+   Creates the file with `web:` and `jobs:` entries if it does not exist. Appends a `jobs:` entry if the file exists but lacks one. Skips if a `jobs:` entry is already present.
+
+5. **Patches queue.yml dispatcher** with `recurring_schedule: config/recurring.yml`:
+   Adds the `recurring_schedule` key to each dispatcher entry in `config/queue.yml`. If no dispatchers section exists, creates a default one. Skips if `recurring_schedule` is already configured. Skips if `config/queue.yml` does not exist.
+
 Re-running the generator is safe and idempotent.
 
 ## Post-Install Configuration
@@ -180,6 +190,8 @@ config.authentication.user_signed_in_method = :user_signed_in?
 | `lib/generators/source_monitor/install/install_generator.rb` | Rails generator |
 | `lib/generators/source_monitor/install/templates/source_monitor.rb.tt` | Initializer template |
 | `lib/source_monitor/setup/initializer_patcher.rb` | Post-install patching |
+| `lib/source_monitor/setup/procfile_patcher.rb` | Procfile.dev patching for guided workflow |
+| `lib/source_monitor/setup/queue_config_patcher.rb` | Queue config patching for guided workflow |
 | `lib/source_monitor/setup/verification/runner.rb` | Verification runner |
 | `lib/source_monitor/engine.rb` | Engine configuration and initializers |
 | `docs/setup.md` | Full setup documentation |
@@ -221,6 +233,8 @@ end
 - [ ] Install generator ran (`bin/rails generate source_monitor:install`)
 - [ ] Engine migrations copied and applied
 - [ ] Recurring jobs configured in `config/recurring.yml`
+- [x] `Procfile.dev` includes `jobs:` entry for Solid Queue (handled by generator)
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
 - [ ] Solid Queue workers started
 - [ ] Authentication hooks configured in initializer
 - [ ] `bin/source_monitor verify` passes

data/.claude/skills/sm-host-setup/reference/setup-checklist.md

@@ -86,6 +86,7 @@ production:
   dispatchers:
     - polling_interval: 1
       batch_size: 500
+      recurring_schedule: config/recurring.yml
   workers:
     - queues: "source_monitor_fetch"
       threads: 2
@@ -99,7 +100,39 @@ production:
 bin/rails solid_queue:start
 ```
 
+### Phase 6a: Procfile.dev for Development (Automatic)
+
+The install generator automatically patches `Procfile.dev` with a `jobs:` entry for Solid Queue. If no `Procfile.dev` exists, it creates one with `web:` and `jobs:` entries. If the file exists but lacks a `jobs:` entry, it appends one. This is idempotent -- re-running the generator is safe.
+
+Verify after running the generator:
+```
+# Expected Procfile.dev content:
+web: bin/rails server -p 3000
+jobs: bundle exec rake solid_queue:start
+```
+
+If the entry is missing, re-run: `bin/rails generate source_monitor:install`
+
+### Phase 6b: Recurring Schedule Dispatcher Wiring (Automatic)
+
+The install generator automatically patches `config/queue.yml` dispatchers with `recurring_schedule: config/recurring.yml`. If no dispatchers section exists, it creates a default one. This is idempotent -- re-running the generator is safe.
+
+Verify after running the generator:
+```yaml
+# Expected in config/queue.yml under dispatchers:
+dispatchers:
+  - polling_interval: 1
+    batch_size: 500
+    recurring_schedule: config/recurring.yml
+```
+
+If the key is missing, re-run: `bin/rails generate source_monitor:install`
+
+**Diagnostics:** Run `bin/source_monitor verify` to check that recurring tasks are registered. The RecurringScheduleVerifier will warn if no SourceMonitor recurring tasks are found in Solid Queue.
+
 - [ ] Queue configuration includes `source_monitor_fetch` and `source_monitor_scrape`
+- [x] `Procfile.dev` includes a `jobs:` entry for Solid Queue (handled by generator)
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
 - [ ] Workers started and processing
 
 ## Phase 7: Verify Installation

data/.claude/skills/sm-job/SKILL.md

@@ -168,7 +168,7 @@ The install generator (`bin/rails generate source_monitor:install`) automaticall
 | `SourceMonitor::ItemCleanupJob` | at 2am every day |
 | `SourceMonitor::LogCleanupJob` | at 3am every day |
 
-These run automatically with `bin/dev` or `bin/jobs`. If you need to customize, edit `config/recurring.yml` directly.
+The install generator automatically configures `config/recurring.yml` with these entries AND patches the `config/queue.yml` dispatcher with `recurring_schedule: config/recurring.yml` so recurring jobs load on startup. Both steps are idempotent. If you need to customize schedules, edit `config/recurring.yml` directly.
 
 ## Retry Policies
 

data/.vbw-planning/REQUIREMENTS.md

@@ -32,6 +32,28 @@ Core value: Drop-in Rails engine for feed monitoring, content scraping, and oper
 - [ ] **REQ-14**: Audit and fix any RuboCop violations against omakase ruleset
 - [ ] **REQ-15**: Ensure all models, controllers, and service objects follow Rails conventions
 
+### Generator Enhancements
+
+- [ ] **REQ-16**: Install generator patches `Procfile.dev` with a `jobs:` entry for Solid Queue
+- [ ] **REQ-17**: Install generator patches queue config dispatcher with `recurring_schedule: config/recurring.yml`
+- [ ] **REQ-18**: Guided workflow (`Setup::Workflow`) integrates both new generator steps
+- [ ] **REQ-19**: `RecurringScheduleVerifier` checks that recurring tasks are registered with Solid Queue
+- [ ] **REQ-20**: `SolidQueueVerifier` remediation suggests `Procfile.dev` when workers not detected
+- [ ] **REQ-21**: Skills and documentation updated to reflect automated Procfile.dev and recurring_schedule setup
+
+### Dashboard UX
+
+- [ ] **REQ-22**: Fetch logs show source URL for both success and failure entries on the dashboard
+- [ ] **REQ-23**: Dashboard links to sources and items are clickable and open in a new tab
+
+### Active Storage Image Downloads
+
+- [ ] **REQ-24**: Configurable option to download inline images from items to Active Storage instead of loading from source
+
+### Feed Compatibility
+
+- [ ] **REQ-25**: Investigate and fix failing fetch for Netflix Tech Blog feed (https://netflixtechblog.com/feed)
+
 ## v2 Requirements
 
 - [ ] **REQ-XX**: Improve optional dependency loading with clear error messages

data/.vbw-planning/ROADMAP.md

@@ -0,0 +1,125 @@
+<!-- VBW ROADMAP -- Phase decomposition with requirement mapping -->
+<!-- Created during /vbw scope -->
+
+# SourceMonitor Generator Enhancements Roadmap
+
+**Milestone:** generator-enhancements
+**Goal:** Make the install generator and verification suite catch the two most common host-app setup failures: missing Procfile.dev jobs entry and missing recurring_schedule dispatcher wiring.
+
+## Phases
+
+1. [x] ~~Phase 0: Documentation Gaps~~ (shipped via quick fix ea788ea)
+2. [x] Phase 1: Install Generator Steps (Procfile.dev + Queue Config)
+3. [x] Phase 2: Recurring Schedule Verifier
+4. [x] Phase 3: Skills & Documentation Alignment
+5. [x] Phase 4: Dashboard UX Improvements
+6. [x] Phase 5: Active Storage Image Downloads
+7. [x] Phase 6: Netflix Feed Investigation
+
+## Phase Details
+
+### Phase 1: Install Generator Steps
+
+**Goal:** Add two new idempotent steps to the install generator: (a) patch `Procfile.dev` with a `jobs:` entry for Solid Queue, and (b) patch the queue config dispatcher with `recurring_schedule: config/recurring.yml`.
+
+**Requirements:** REQ-16, REQ-17, REQ-18
+
+**Success Criteria:**
+- `bin/rails generate source_monitor:install` patches Procfile.dev when present (idempotent, skip if entry exists)
+- `bin/rails generate source_monitor:install` patches queue.yml dispatcher with recurring_schedule (idempotent)
+- Both steps wired into `Setup::Workflow` for the guided installer
+- Generator tests cover: fresh file, existing file with entry, existing file without entry, missing file
+- `bin/rails test` passes, RuboCop clean
+
+### Phase 2: Recurring Schedule Verifier
+
+**Goal:** Add a `RecurringScheduleVerifier` to the verification suite that checks whether recurring tasks are actually registered with Solid Queue dispatchers, and enhance the existing `SolidQueueVerifier` to suggest Procfile.dev when workers aren't detected.
+
+**Requirements:** REQ-19, REQ-20
+
+**Success Criteria:**
+- `bin/source_monitor verify` checks that recurring tasks are registered (not just that workers heartbeat)
+- Warning when no recurring tasks found with actionable remediation message
+- SolidQueueVerifier remediation mentions Procfile.dev for `bin/dev` users
+- Verifier tests with mocked Solid Queue state
+- `bin/rails test` passes, RuboCop clean
+
+### Phase 3: Skills & Documentation Alignment
+
+**Goal:** Update all `sm-*` skills and docs to reflect that the generator now automatically handles Procfile.dev and recurring_schedule wiring. Remove manual steps that are now automated.
+
+**Requirements:** REQ-21
+
+**Success Criteria:**
+- sm-host-setup skill reflects new generator capabilities (auto-patching, not manual steps)
+- sm-configure skill references the automatic recurring_schedule wiring
+- docs/setup.md updated to note the generator handles both automatically
+- docs/troubleshooting.md updated with improved diagnostics
+- setup-checklist.md reflects automation (checked by default, not manual)
+
+### Phase 4: Dashboard UX Improvements
+
+**Goal:** Show source URLs in fetch log entries for both successes and failures on the dashboard, and make links to sources and items clickable (opening in a new tab).
+
+**Requirements:** REQ-22, REQ-23
+
+**Success Criteria:**
+- Fetch log entries on the dashboard display the source URL alongside the existing summary
+- Both success and failure fetch logs show the URL
+- Source names and item titles are clickable links that open in a new tab
+- Existing dashboard layout is preserved
+- `bin/rails test` passes, RuboCop clean
+
+### Phase 5: Active Storage Image Downloads
+
+**Goal:** Add a configurable option to download inline images from feed items to Active Storage instead of loading them directly from the source URL. This prevents broken images when sources go offline and improves page load performance.
+
+**Requirements:** REQ-24
+
+**Success Criteria:**
+- New configuration option (`config.images.download_to_active_storage` or similar) defaults to `false`
+- When enabled, inline images in item content are detected and downloaded to Active Storage
+- Original image URLs are replaced with Active Storage URLs in the stored content
+- Images that fail to download gracefully fall back to original URLs
+- Configuration is documented in sm-configure skill
+- `bin/rails test` passes, RuboCop clean
+
+### Phase 6: Netflix Feed Investigation
+
+**Goal:** Investigate and fix the failing fetch for `https://netflixtechblog.com/feed`. Determine whether the issue is in the feed parser, HTTP client configuration, or content format, and apply appropriate fixes.
+
+**Requirements:** REQ-25
+
+**Success Criteria:**
+- Root cause identified and documented
+- Fix applied (parser, HTTP client, or configuration change)
+- Netflix Tech Blog feed fetches successfully
+- No regressions in other feed types
+- `bin/rails test` passes, RuboCop clean
+
+## Progress
+
+| Phase | Status | Plans |
+|-------|--------|-------|
+| 0 | Complete | - |
+| 1 | Complete | PLAN-01 (5 tasks, 4 commits) |
+| 2 | Complete | PLAN-01 (5 tasks, 1 commit) |
+| 3 | Complete | PLAN-01 (5 tasks, 1 commit) |
+| 4 | Complete | PLAN-01 (5 tasks, 5 commits) |
+| 5 | Complete | PLAN-01 (4 tasks, 5 commits) + PLAN-02 (4 tasks, 4 commits) |
+| 6 | Complete | PLAN-01 (5 tasks, 5 commits) |
+
+## Requirement Mapping
+
+| REQ | Phase | Description |
+|-----|-------|-------------|
+| REQ-16 | 1 | Generator patches Procfile.dev with jobs: entry |
+| REQ-17 | 1 | Generator patches queue config with recurring_schedule |
+| REQ-18 | 1 | Guided workflow integrates both new steps |
+| REQ-19 | 2 | RecurringScheduleVerifier checks recurring task registration |
+| REQ-20 | 2 | SolidQueueVerifier remediation mentions Procfile.dev |
+| REQ-21 | 3 | Skills and docs reflect automated setup |
+| REQ-22 | 4 | Fetch logs show source URL on dashboard |
+| REQ-23 | 4 | Dashboard links clickable in new tab |
+| REQ-24 | 5 | Download inline images to Active Storage |
+| REQ-25 | 6 | Fix Netflix Tech Blog feed fetch |