source_monitor 0.3.3 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ae2bf8e389d7c36d4f7061a13f7f4bb213c8365408f9d0c1fb818ea25b385ff7
-  data.tar.gz: a3b8b537ea6bb06251edbc53a203d9c9e9c20a1d555679a722f00ff62eef9967
+  metadata.gz: e63143dd784f79d378752978f534b36c122eec15264e8adf13cfcb994ca222cf
+  data.tar.gz: 04caa4a25e233c9520950697729efdad586770d6fba3fa00ba7369cbca9763ec
 SHA512:
-  metadata.gz: 418df72b1757e7a941e645e171d00f012072c5f2974352851015cc3501adf6a546703a0b3cc38c35de7f1d6034b05c4bf13ce162815640a609612d395a40ac3d
-  data.tar.gz: bef273caaf974979aa4a9e5dce7c4b0043efa47149a0b3f2bcba80874f7b3ff303e5f4672e90ab176d79306a4085ff6ed692f883709a37490e7df279595803e7
+  metadata.gz: 27bddd52129b5341906ebc67a7b86e259df8d0da671622b9271d1eab83cf214c1478e8674953d0eb20c8bd4bcb9b9257fc03261e4b061bd6f6d8c59d5a591ddb
+  data.tar.gz: b74f078deff1add84247e4fd0377d76ef026b1a2cee7b9808a252a2140507057d62a0be5c786c10f8e2e90e327a88717af211502850de797ece0ace895a822f7

data/.claude/commands/release.md

@@ -6,34 +6,61 @@ Orchestrate a full release cycle for the source_monitor gem. This command handle
 
 - `$ARGUMENTS` -- Optional: version bump description or release notes summary. If empty, derive from commits since last tag.
 
+## Known Gotchas (Read Before Starting)
+
+These are real issues encountered in previous releases. Each step below accounts for them, but keep them in mind:
+
+1. **Two version files**: Both `lib/source_monitor/version.rb` AND the top-level `VERSION` file must be bumped. The VBW pre-push hook checks for changes to `VERSION` (the top-level file).
+2. **Gemfile.lock sync**: After bumping the version in `version.rb`, you MUST run `bundle install` to update `Gemfile.lock`. CI runs `bundle install --frozen` which fails if the lockfile is stale.
+3. **VBW volatile files**: Files in `.vbw-planning/` (`.cost-ledger.json`, `.notification-log.jsonl`, `.session-log.jsonl`, `.hook-errors.log`) are continuously modified by VBW hooks. They should be in `.gitignore`. If they aren't, add them before proceeding.
+4. **Pre-push hook**: The VBW pre-push hook at `.git/hooks/pre-push` requires the `VERSION` file to appear in the diff for any push. For new branches, it compares the commit against the working tree -- any dirty files will trigger a false positive. For force-pushes to existing branches where `VERSION` hasn't changed since the last push, use `--no-verify`.
+5. **Single squashed commit**: Always create ONE commit on the release branch with ALL changes (version bump, changelog, Gemfile.lock, any fixes). Multiple commits cause pre-push hook issues.
+6. **Diff coverage CI gate**: The `test` CI job enforces diff coverage. Any changed lines in source code (not just test files) must have test coverage. If you change source code during the release (e.g., bug fixes), you must add corresponding tests.
+7. **Local main divergence after merge**: After the PR merges, local main will have different commits than origin/main (pre-squash vs merged). You must `git reset --hard origin/main` to sync -- this requires user approval since the sandbox blocks it.
+
 ## Step 1: Git Hygiene
 
 Run these checks. If ANY fail, STOP and report the issue to the user.
 
-1. **Working tree clean**: `git status --porcelain` must be empty. If not, list the dirty files and ask the user whether to commit, stash, or abort.
-2. **On main branch**: `git branch --show-current` must be `main`. If not, ask the user if they want to continue from the current branch or switch.
-3. **Fetch latest**: `git fetch origin main`
-4. **Up to date with remote**: Compare `git rev-parse HEAD` with `git rev-parse origin/main`. If behind, ask the user whether to pull.
-5. **No unpushed commits**: Compare local HEAD with `origin/main`. If ahead, note how many commits are unpushed -- these will be included in the PR.
+1. **VBW volatile files gitignored**: Check that `.gitignore` contains these entries:
+   ```
+   .vbw-planning/.cost-ledger.json
+   .vbw-planning/.notification-log.jsonl
+   .vbw-planning/.session-log.jsonl
+   .vbw-planning/.hook-errors.log
+   ```
+   If any are missing, add them. If any of these files are tracked by git, remove them from tracking with `git rm --cached <file>`.
+
+2. **Working tree clean**: `git status --porcelain` must be empty (ignoring the VBW volatile files which should now be gitignored). If not, list the dirty files and ask the user whether to commit, stash, or abort.
+
+3. **On main branch**: `git branch --show-current` must be `main`. If not, ask the user if they want to continue from the current branch or switch.
+
+4. **Fetch latest**: `git fetch origin main`
+
+5. **Up to date with remote**: Compare `git rev-parse HEAD` with `git rev-parse origin/main`. If behind, ask the user whether to pull.
+
+6. **No unpushed commits**: Compare local HEAD with `origin/main`. If ahead, note how many commits are unpushed -- these will be included in the release.
 
 Report a summary:
 ```
 Git Status:
   Branch: main
   Clean: yes
+  VBW files gitignored: yes
   Synced with origin: yes
   Unpushed commits: N
 ```
 
 ## Step 2: Version Check
 
-1. Read `lib/source_monitor/version.rb` to get the current VERSION.
-2. Read the latest git tag with `git tag --sort=-v:refname | head -1`.
-3. If the VERSION matches the latest tag (e.g., both are `0.3.2`), the version hasn't been bumped. Ask the user:
+1. Read `lib/source_monitor/version.rb` to get the current VERSION constant.
+2. Read the top-level `VERSION` file to verify it matches. If they differ, sync them before proceeding.
+3. Read the latest git tag with `git tag --sort=-v:refname | head -1`.
+4. If the VERSION matches the latest tag (e.g., both are `0.3.2`), the version hasn't been bumped. Ask the user:
    - "Current version is X.Y.Z which already has a tag. What should the new version be?"
    - Offer options: patch (X.Y.Z+1), minor (X.Y+1.0), major (X+1.0.0), or custom.
-   - Update `lib/source_monitor/version.rb` with the new version.
-4. If VERSION is ahead of the latest tag, proceed with the current version.
+   - Update BOTH `lib/source_monitor/version.rb` AND the top-level `VERSION` file with the new version.
+5. If VERSION is ahead of the latest tag, proceed with the current version.
 
 Store the release version for later steps. Do NOT commit yet -- that happens after the changelog is updated.
 
@@ -77,29 +104,35 @@ The changelog follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) f
    - Insert the new versioned entry immediately after the `## [Unreleased]` block and before the previous release entry.
    - Preserve all existing entries below.
 
-## Step 4: Commit Version Bump + Changelog
+## Step 4: Sync Gemfile.lock
 
-Once both `lib/source_monitor/version.rb` (if changed) and `CHANGELOG.md` are updated:
+**CRITICAL**: After updating `version.rb`, the gemspec version changes and `Gemfile.lock` becomes stale.
 
-1. Stage the changed files:
-   ```
-   git add lib/source_monitor/version.rb CHANGELOG.md
-   ```
-2. Create a single commit:
+1. Run `bundle install` to update `Gemfile.lock`.
+2. Verify the output shows the new version: `Using source_monitor X.Y.Z (was X.Y.Z-1)`.
+3. If `bundle install` fails, resolve the issue before proceeding.
+
+## Step 5: Create Release Branch with Single Squashed Commit
+
+**IMPORTANT**: All release changes MUST be in a single commit on the release branch. This avoids pre-push hook issues where individual commits are checked for VERSION changes.
+
+1. Create the release branch from main: `git checkout -b release/vX.Y.Z`
+2. Stage ALL release files in one commit:
    ```
-   chore: bump version to X.Y.Z and update changelog
+   git add lib/source_monitor/version.rb VERSION CHANGELOG.md Gemfile.lock
    ```
-   If only the changelog changed (version was already bumped), use:
+   Also stage any other files that were changed (updated skills, docs, etc.).
+3. Create a single commit:
    ```
-   chore: update changelog for vX.Y.Z release
+   chore: release vX.Y.Z
    ```
+4. Push the branch: `git push -u origin release/vX.Y.Z`
+   - If the pre-push hook blocks with a false positive (e.g., VBW files dirty in working tree despite being gitignored), use `git push -u --no-verify origin release/vX.Y.Z`. This is safe because we've verified VERSION is in the commit.
+5. If the push fails for other reasons, diagnose and fix before proceeding.
 
-## Step 5: Create Release Branch and PR
+## Step 6: Create PR
 
-1. Create a release branch: `git checkout -b release/vX.Y.Z`
-2. Push the branch: `git push -u origin release/vX.Y.Z`
-3. Generate a PR body from the new CHANGELOG.md entry (use the entry written in Step 3, not raw commits).
-4. Create the PR using `gh pr create`:
+1. Create the PR using `gh pr create`:
    - Title: `Release vX.Y.Z`
    - Body format:
      ```
@@ -108,79 +141,85 @@ Once both `lib/source_monitor/version.rb` (if changed) and `CHANGELOG.md` are up
      <paste the CHANGELOG.md entry content here>
 
      ### Release Checklist
-     - [x] Version bumped in `lib/source_monitor/version.rb`
+     - [x] Version bumped in `lib/source_monitor/version.rb` and `VERSION`
      - [x] CHANGELOG.md updated
+     - [x] Gemfile.lock synced
      - [ ] CI passes (lint, security, test, release_verification)
 
      ---
      Auto-generated release PR by `/release` command.
      ```
    - Base: `main`
-5. Report the PR URL to the user.
+2. Report the PR URL to the user.
 
-## Step 6: Monitor CI Pipeline
+## Step 7: Monitor CI Pipeline
 
-Poll the CI status using `gh pr checks <PR_NUMBER> --watch` or repeated `gh pr checks <PR_NUMBER>` calls. The CI has 4 required jobs: `lint`, `security`, `test`, `release_verification`.
+Poll the CI status using repeated `gh pr checks <PR_NUMBER>` calls. The CI has 4 required jobs: `lint`, `security`, `test`, `release_verification` (release_verification only runs after test passes).
 
 1. Tell the user: "Monitoring CI pipeline for PR #N... This typically takes 3-5 minutes."
-2. Poll with `gh pr checks <PR_NUMBER>` every 30 seconds, up to 15 minutes.
-3. After each poll, report progress if any jobs completed.
+2. Wait 30 seconds before first poll (give CI time to start).
+3. Poll with `gh pr checks <PR_NUMBER>` every 60 seconds, up to 15 minutes.
+4. After each poll, report progress if any jobs completed or failed.
 
 ### If CI PASSES (all checks green):
 
-Continue to Step 7.
+Continue to Step 8.
 
 ### If CI FAILS:
 
 1. Get the failure details: `gh pr checks <PR_NUMBER>` to identify which jobs failed.
-2. For each failed job, fetch the log summary:
+2. For each failed job, fetch the log summary. Note: logs are only available after the **entire run** completes (not just one job). If `gh run view <RUN_ID> --log-failed` says "still in progress", wait and retry:
    ```
-   gh run view <RUN_ID> --log-failed | tail -50
+   gh run view <RUN_ID> --log-failed | tail -80
    ```
-3. Present to the user:
-   ```
-   CI Failed on PR #N
-
-   Failed Jobs:
-   - <job_name>: <brief summary of failure>
+3. **Common failure: diff coverage** -- If the `test` job fails on "Enforce diff coverage", it means changed source lines lack test coverage. Read the error to identify uncovered files/lines, write tests, and add them to the release commit.
+4. **Common failure: Gemfile.lock frozen** -- If `bundle install` fails in CI with "frozen mode", you forgot to run `bundle install` locally (Step 4).
+5. Present failure details to the user and ask what to do:
+   - "Fix the issues and re-push" -- Fix issues, amend the commit (`git commit --amend --no-edit`), force push (`git push --force-with-lease --no-verify origin release/vX.Y.Z`), and restart CI monitoring.
+   - "Close the PR and abort" -- Close the PR, delete the branch, switch back to main.
+   - "Investigate manually" -- Stop and let the user handle it.
 
-   Log excerpt:
-   <relevant log lines>
-   ```
-4. Ask the user what to do:
-   - "Fix the issues and re-push" -- Help fix the issues, commit, push, and restart CI monitoring
-   - "Close the PR and abort" -- Close the PR, delete the branch, switch back to main
-   - "Investigate manually" -- Stop and let the user handle it
+**Note on force pushes**: When force-pushing the release branch after amending, always use `--no-verify` because the pre-push hook will see the diff between old and new branch tips, and `VERSION` won't appear as changed (it's the same in both). This is expected and safe.
 
-## Step 7: Auto-Merge PR
+## Step 8: Auto-Merge PR
 
 Once CI is green:
 
 1. Merge the PR: `gh pr merge <PR_NUMBER> --merge --delete-branch`
-2. Switch back to main and pull: `git checkout main && git pull origin main`
+
+2. **Sync local main with remote** (this is the tricky part):
+   - The merge creates a merge commit on origin/main that doesn't exist locally.
+   - Local main may have different commits (pre-squash) than what was merged.
+   - Switch to main: `git checkout main`
+   - Try `git pull origin main`. If it fails with conflicts or divergence:
+     - Ask the user to run `git reset --hard origin/main` (the sandbox blocks this command).
+     - Explain this is safe because the PR is merged and all changes are on origin/main.
+   - Verify with `git log --oneline -3` that local matches remote.
+
 3. Report: "PR #N merged successfully."
 
-## Step 8: Tag the Release
+## Step 9: Tag the Release
 
-1. Create an annotated tag:
+1. Verify you're on main and synced with origin.
+2. Create an annotated tag:
    ```
    git tag -a vX.Y.Z -m "Release vX.Y.Z"
    ```
-2. Push the tag: `git push origin vX.Y.Z`
-3. Create a GitHub release from the tag:
+3. Push the tag: `git push origin vX.Y.Z`
+4. Create a GitHub release from the tag:
    ```
    gh release create vX.Y.Z --title "vX.Y.Z" --notes "<changelog entry from Step 3>"
    ```
-4. Report the release URL.
+5. Report the release URL.
 
-## Step 9: Build the Gem
+## Step 10: Build the Gem
 
-1. Clean any old gem files: remove any `source_monitor-*.gem` files in the project root.
+1. Clean any old gem files: `ls source_monitor-*.gem` and remove them if found (don't error if none exist).
 2. Build the gem: `gem build source_monitor.gemspec`
 3. Verify the gem was built: check for `source_monitor-X.Y.Z.gem` in the project root.
-4. Show the gem contents summary: `gem spec source_monitor-X.Y.Z.gem | head -30`
+4. Show the file size: `ls -la source_monitor-X.Y.Z.gem`
 
-## Step 10: Gem Push Instructions
+## Step 11: Gem Push Instructions
 
 Present the final instructions to the user:
 
@@ -189,6 +228,7 @@ Release vX.Y.Z Complete!
 
   Git tag:    vX.Y.Z (pushed)
   GitHub:     <release URL>
+  PR:         <PR URL> (merged)
   Gem built:  source_monitor-X.Y.Z.gem
 
 To publish to RubyGems:
@@ -209,4 +249,7 @@ Do NOT run `gem push` automatically -- always let the user handle the OTP-protec
 - If any step fails unexpectedly, report what happened and where things stand.
 - If a release branch already exists, ask the user whether to reuse or recreate it.
 - If the tag already exists, skip tagging and inform the user.
+- If the pre-push hook blocks, check whether `VERSION` (top-level) is in the commit diff. If it is and the hook still blocks (working tree drift from VBW), use `--no-verify`.
+- If local main diverges from origin after merge, ask the user to run `git reset --hard origin/main`.
 - Always leave the user on the `main` branch in a clean state when possible.
+- If `gem build` warns about duplicate URIs (homepage_uri/source_code_uri), this is cosmetic and safe to ignore.

data/.claude/skills/sm-configure/SKILL.md

@@ -29,7 +29,7 @@ After the block executes, `ModelExtensions.reload!` runs automatically to apply
 
 ## Configuration Sections
 
-The `config` object (`SourceMonitor::Configuration`) has 10 sub-sections plus top-level queue/job settings:
+The `config` object (`SourceMonitor::Configuration`) has 11 sub-sections plus top-level queue/job settings:
 
 | Section | Accessor | Class |
 |---|---|---|
@@ -44,6 +44,7 @@ The `config` object (`SourceMonitor::Configuration`) has 10 sub-sections plus to
 | Models | `config.models` | `Models` |
 | Realtime | `config.realtime` | `RealtimeSettings` |
 | Authentication | `config.authentication` | `AuthenticationSettings` |
+| Images | `config.images` | `ImagesSettings` |
 
 See `reference/configuration-reference.md` for every setting with types, defaults, and examples.
 
@@ -69,6 +70,14 @@ config.authentication.authenticate_with :authenticate_user!
 config.authentication.authorize_with ->(c) { c.current_user&.admin? }
 ```
 
+### Image Downloads (Active Storage)
+```ruby
+config.images.download_to_active_storage = true
+config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
+config.images.download_timeout = 15
+config.images.allowed_content_types = %w[image/jpeg image/png image/webp]
+```
+
 ### Events
 ```ruby
 config.events.after_item_created { |e| Notifier.new_item(e.item) }
@@ -116,6 +125,7 @@ SourceMonitor.mission_control_dashboard_path # Resolved MC path or nil
 | `lib/source_monitor/configuration/model_definition.rb` | Per-model definition |
 | `lib/source_monitor/configuration/realtime_settings.rb` | Action Cable settings |
 | `lib/source_monitor/configuration/authentication_settings.rb` | Auth settings |
+| `lib/source_monitor/configuration/images_settings.rb` | Image download settings |
 | `lib/source_monitor/configuration/validation_definition.rb` | Validation wrapper |
 
 ## References
@@ -146,7 +156,8 @@ end
 ## Checklist
 
 - [ ] Initializer exists at `config/initializers/source_monitor.rb`
-- [ ] Queue names match `config/solid_queue.yml` entries
+- [ ] Queue names match `config/queue.yml` (or `config/solid_queue.yml`) entries
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by install generator)
 - [ ] Authentication hooks configured for host auth system
 - [ ] HTTP timeouts appropriate for target feeds
 - [ ] Retention policy set for production

data/.claude/skills/sm-configure/reference/configuration-reference.md

@@ -309,6 +309,39 @@ end
 
 ---
 
+## Images Settings (`config.images`)
+
+Class: `SourceMonitor::Configuration::ImagesSettings`
+
+Controls background downloading of inline images from feed content to Active Storage.
+
+**Prerequisite:** The host app must have Active Storage installed (`rails active_storage:install` + migrations).
+
+| Setting | Type | Default | Description |
+|---|---|---|---|
+| `download_to_active_storage` | Boolean | `false` | Enable background image downloading for new items |
+| `max_download_size` | Integer | `10485760` (10 MB) | Maximum image file size in bytes; larger images are skipped |
+| `download_timeout` | Integer | `30` | HTTP timeout for image downloads in seconds |
+| `allowed_content_types` | Array | `["image/jpeg", "image/png", "image/gif", "image/webp", "image/svg+xml"]` | Permitted MIME types for downloaded images |
+
+### Helper Method
+
+| Method | Returns | Description |
+|---|---|---|
+| `download_enabled?` | Boolean | Returns `true` when `download_to_active_storage` is truthy |
+
+```ruby
+# Enable image downloading with custom limits
+config.images.download_to_active_storage = true
+config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
+config.images.download_timeout = 15
+config.images.allowed_content_types = %w[image/jpeg image/png image/webp]
+```
+
+When enabled, `DownloadContentImagesJob` is automatically enqueued after new items are created from feed entries. The job downloads inline `<img>` images from `item.content`, attaches them to `item_content.images` via Active Storage, and rewrites the HTML with Active Storage serving URLs. Failed downloads gracefully preserve the original image URL.
+
+---
+
 ## Environment Variables
 
 | Variable | Purpose |

data/.claude/skills/sm-host-setup/SKILL.md

@@ -12,7 +12,7 @@ Guides integration of the SourceMonitor engine into a host Rails application.
 
 - Adding SourceMonitor to a new or existing Rails 8 host app
 - Troubleshooting a broken installation
-- Re-running setup after upgrading the gem
+- Re-running setup after upgrading the gem (see also: `sm-upgrade` skill for full upgrade workflow)
 - Rolling back the engine from a host app
 
 ## Prerequisites
@@ -73,6 +73,10 @@ bin/rails db:migrate
 # 5. Start background workers (recurring jobs configured automatically in config/recurring.yml)
 bin/rails solid_queue:start
 
+# Note: The generator automatically patches Procfile.dev with a jobs: entry
+# and adds recurring_schedule to your queue.yml dispatcher config.
+# Re-run the generator if these were not applied: bin/rails generate source_monitor:install
+
 # 6. Verify
 bin/source_monitor verify
 ```
@@ -86,7 +90,7 @@ gem "source_monitor", github: "dchuk/source_monitor"
 
 ## What the Install Generator Does
 
-The generator (`SourceMonitor::Generators::InstallGenerator`) performs three actions:
+The generator (`SourceMonitor::Generators::InstallGenerator`) performs five actions:
 
 1. **Mounts the engine** in `config/routes.rb`:
    ```ruby
@@ -100,6 +104,12 @@ The generator (`SourceMonitor::Generators::InstallGenerator`) performs three act
 3. **Configures recurring jobs** in `config/recurring.yml`:
    Adds entries for `ScheduleFetchesJob` (every minute), scrape scheduling (every 2 minutes), `ItemCleanupJob` (2am daily), and `LogCleanupJob` (3am daily). If the file doesn't exist, creates it with `default: &default` and environment sections. If it exists, merges entries without overwriting. Skips if SourceMonitor entries are already present.
 
+4. **Patches Procfile.dev** with a `jobs:` entry for Solid Queue:
+   Creates the file with `web:` and `jobs:` entries if it does not exist. Appends a `jobs:` entry if the file exists but lacks one. Skips if a `jobs:` entry is already present.
+
+5. **Patches queue.yml dispatcher** with `recurring_schedule: config/recurring.yml`:
+   Adds the `recurring_schedule` key to each dispatcher entry in `config/queue.yml`. If no dispatchers section exists, creates a default one. Skips if `recurring_schedule` is already configured. Skips if `config/queue.yml` does not exist.
+
 Re-running the generator is safe and idempotent.
 
 ## Post-Install Configuration
@@ -180,6 +190,8 @@ config.authentication.user_signed_in_method = :user_signed_in?
 | `lib/generators/source_monitor/install/install_generator.rb` | Rails generator |
 | `lib/generators/source_monitor/install/templates/source_monitor.rb.tt` | Initializer template |
 | `lib/source_monitor/setup/initializer_patcher.rb` | Post-install patching |
+| `lib/source_monitor/setup/procfile_patcher.rb` | Procfile.dev patching for guided workflow |
+| `lib/source_monitor/setup/queue_config_patcher.rb` | Queue config patching for guided workflow |
 | `lib/source_monitor/setup/verification/runner.rb` | Verification runner |
 | `lib/source_monitor/engine.rb` | Engine configuration and initializers |
 | `docs/setup.md` | Full setup documentation |
@@ -190,6 +202,7 @@ config.authentication.user_signed_in_method = :user_signed_in?
 - `docs/setup.md` -- Complete setup workflow documentation
 - `docs/configuration.md` -- Configuration reference
 - `docs/troubleshooting.md` -- Common issues and fixes
+- `sm-upgrade` skill -- Upgrade workflow for gem version updates
 
 ## Testing
 
@@ -197,6 +210,7 @@ After setup, verify with:
 1. `bin/source_monitor verify` -- Checks Solid Queue and Action Cable
 2. Visit the mount path in browser -- Dashboard should load
 3. Create a source and trigger "Fetch Now" -- Validates end-to-end
+4. For subsequent gem updates, use `bin/source_monitor upgrade` -- see the `sm-upgrade` skill
 
 Optional system test for host apps using Devise:
 ```ruby
@@ -221,6 +235,8 @@ end
 - [ ] Install generator ran (`bin/rails generate source_monitor:install`)
 - [ ] Engine migrations copied and applied
 - [ ] Recurring jobs configured in `config/recurring.yml`
+- [x] `Procfile.dev` includes `jobs:` entry for Solid Queue (handled by generator)
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
 - [ ] Solid Queue workers started
 - [ ] Authentication hooks configured in initializer
 - [ ] `bin/source_monitor verify` passes

data/.claude/skills/sm-host-setup/reference/setup-checklist.md

@@ -86,6 +86,7 @@ production:
   dispatchers:
     - polling_interval: 1
       batch_size: 500
+      recurring_schedule: config/recurring.yml
   workers:
     - queues: "source_monitor_fetch"
       threads: 2
@@ -99,7 +100,39 @@ production:
 bin/rails solid_queue:start
 ```
 
+### Phase 6a: Procfile.dev for Development (Automatic)
+
+The install generator automatically patches `Procfile.dev` with a `jobs:` entry for Solid Queue. If no `Procfile.dev` exists, it creates one with `web:` and `jobs:` entries. If the file exists but lacks a `jobs:` entry, it appends one. This is idempotent -- re-running the generator is safe.
+
+Verify after running the generator:
+```
+# Expected Procfile.dev content:
+web: bin/rails server -p 3000
+jobs: bundle exec rake solid_queue:start
+```
+
+If the entry is missing, re-run: `bin/rails generate source_monitor:install`
+
+### Phase 6b: Recurring Schedule Dispatcher Wiring (Automatic)
+
+The install generator automatically patches `config/queue.yml` dispatchers with `recurring_schedule: config/recurring.yml`. If no dispatchers section exists, it creates a default one. This is idempotent -- re-running the generator is safe.
+
+Verify after running the generator:
+```yaml
+# Expected in config/queue.yml under dispatchers:
+dispatchers:
+  - polling_interval: 1
+    batch_size: 500
+    recurring_schedule: config/recurring.yml
+```
+
+If the key is missing, re-run: `bin/rails generate source_monitor:install`
+
+**Diagnostics:** Run `bin/source_monitor verify` to check that recurring tasks are registered. The RecurringScheduleVerifier will warn if no SourceMonitor recurring tasks are found in Solid Queue.
+
 - [ ] Queue configuration includes `source_monitor_fetch` and `source_monitor_scrape`
+- [x] `Procfile.dev` includes a `jobs:` entry for Solid Queue (handled by generator)
+- [x] Dispatcher config includes `recurring_schedule: config/recurring.yml` (handled by generator)
 - [ ] Workers started and processing
 
 ## Phase 7: Verify Installation

data/.claude/skills/sm-job/SKILL.md

@@ -168,7 +168,7 @@ The install generator (`bin/rails generate source_monitor:install`) automaticall
 | `SourceMonitor::ItemCleanupJob` | at 2am every day |
 | `SourceMonitor::LogCleanupJob` | at 3am every day |
 
-These run automatically with `bin/dev` or `bin/jobs`. If you need to customize, edit `config/recurring.yml` directly.
+The install generator automatically configures `config/recurring.yml` with these entries AND patches the `config/queue.yml` dispatcher with `recurring_schedule: config/recurring.yml` so recurring jobs load on startup. Both steps are idempotent. If you need to customize schedules, edit `config/recurring.yml` directly.
 
 ## Retry Policies
 

data/.claude/skills/sm-upgrade/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: sm-upgrade
+description: Use when upgrading SourceMonitor to a new gem version, including CHANGELOG review, running the upgrade command, interpreting verification results, and handling configuration deprecations.
+allowed-tools: Read, Write, Edit, Bash, Glob, Grep
+---
+
+# sm-upgrade: Gem Upgrade Workflow
+
+Guides AI agents through upgrading SourceMonitor in a host Rails application after a new gem version is released. Covers CHANGELOG review, running the upgrade command, interpreting verification results, handling deprecation warnings, and resolving common edge cases.
+
+## When to Use
+
+- Host app is updating the source_monitor gem version in Gemfile
+- User reports deprecation warnings after a gem update
+- User wants to know what changed between versions
+- Troubleshooting a broken upgrade
+- Migrating configuration after breaking changes
+
+## Prerequisites
+
+| Requirement | How to Check |
+|---|---|
+| Existing SourceMonitor installation | `cat .source_monitor_version` or check `config/initializers/source_monitor.rb` |
+| Access to CHANGELOG.md | Bundled with gem: `bundle show source_monitor` then read CHANGELOG.md |
+| Current gem version known | `grep source_monitor Gemfile.lock` |
+
+## Upgrade Workflow
+
+1. **Review CHANGELOG** -- Read `CHANGELOG.md` in the gem source. Identify changes between the current installed version (from `.source_monitor_version` or `Gemfile.lock`) and the target version. Focus on Added, Changed, Fixed, Removed sections. Flag any breaking changes or deprecation notices.
+2. **Update Gemfile** -- Bump the version constraint in the host app's Gemfile: `gem "source_monitor", "~> X.Y"`. Run `bundle update source_monitor`.
+3. **Run the upgrade command** -- `bin/source_monitor upgrade`. This automatically: detects the version change via `.source_monitor_version` marker, copies new migrations, re-runs the install generator (idempotent), runs the full verification suite.
+4. **Run database migrations** -- `bin/rails db:migrate` if the upgrade command copied new migrations.
+5. **Handle deprecation warnings** -- If the configure block in the initializer uses deprecated options, Rails logger will show warnings at boot. Read each warning, identify the replacement, update the initializer. See `sm-configure` skill for configuration reference.
+6. **Run verification** -- `bin/source_monitor verify` to confirm all checks pass.
+7. **Restart processes** -- Restart web server and Solid Queue workers to pick up the new version.
+
+## Interpreting Upgrade Results
+
+The upgrade command runs 4 verification checks automatically:
+
+| Check | OK | Failure | Fix |
+|---|---|---|---|
+| PendingMigrations | All engine migrations present | Missing migrations need copying | `bin/rails db:migrate` |
+| SolidQueue | Workers running | Workers not detected | Start Solid Queue workers via `Procfile.dev` or `bin/rails solid_queue:start` |
+| RecurringSchedule | Tasks registered | Tasks missing from dispatcher | Re-run generator or check `config/queue.yml` |
+| ActionCable | Adapter configured | No production adapter | Configure Solid Cable or Redis adapter |
+
+## Handling Deprecation Warnings
+
+The deprecation framework uses two severity levels:
+
+- **`:warning`** -- Option renamed. The old name still works but logs a deprecation message. Update the initializer to use the new option name. The message includes the replacement path.
+- **`:error`** -- Option removed. Using the old name raises `SourceMonitor::DeprecatedOptionError`. You must remove or replace the option before the app can boot.
+
+Pattern for fixing deprecations:
+
+1. Read the deprecation message (logged to `Rails.logger` or raised as an error)
+2. Find the replacement option path in the message
+3. Update `config/initializers/source_monitor.rb`
+4. Restart and verify the warning is gone
+
+Example deprecation message:
+```
+[SourceMonitor] DEPRECATION: 'http.old_option' was deprecated in v0.5.0 and replaced by 'http.new_option'.
+```
+
+## Edge Cases
+
+- **First install (no `.source_monitor_version` file):** Upgrade command treats this as a version change and runs the full workflow. Safe to run on first install.
+- **Same version (no change):** Command reports "Already up to date (vX.Y.Z)" and exits cleanly.
+- **Skipped versions (e.g., 0.2.0 to 0.4.0):** Read all CHANGELOG sections between the two versions. Multiple migrations may need running.
+- **Failed verification after upgrade:** Read the verification output. Most common: pending migrations (run `db:migrate`), missing Solid Queue workers (start workers), stale recurring schedule (re-run generator).
+- **Custom scrapers or event handlers:** Check CHANGELOG for API changes in `Scrapers::Base` or event callback signatures.
+
+## Key Source Files
+
+| File | Purpose |
+|---|---|
+| `lib/source_monitor/setup/upgrade_command.rb` | Upgrade orchestrator |
+| `lib/source_monitor/setup/cli.rb` | CLI entry point (`bin/source_monitor upgrade`) |
+| `lib/source_monitor/setup/verification/runner.rb` | Verification runner (4 verifiers) |
+| `lib/source_monitor/configuration/deprecation_registry.rb` | Deprecation framework |
+| `CHANGELOG.md` | Version history (Keep a Changelog format) |
+| `.source_monitor_version` | Version marker in host app root |
+
+## References
+
+- `docs/upgrade.md` -- Human-readable upgrade guide
+- `docs/setup.md` -- Initial setup documentation
+- `docs/troubleshooting.md` -- Common issues and fixes
+- `sm-host-setup` skill -- Initial installation workflow
+- `sm-configure` skill -- Configuration reference for updating deprecated options
+
+## Checklist
+
+- [ ] CHANGELOG reviewed for version range
+- [ ] Gemfile updated and `bundle update source_monitor` run
+- [ ] `bin/source_monitor upgrade` completed successfully
+- [ ] Database migrations applied if needed
+- [ ] Deprecation warnings addressed in initializer
+- [ ] `bin/source_monitor verify` passes all checks
+- [ ] Web server and workers restarted