source_monitor 0.3.1 → 0.3.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b8348a9154458260c8b6a3c3dfdeea65e8327b9d7a48dd6c1a2fbe36f9f1bf46
4
- data.tar.gz: 2d56c477bc1191f4f505645f9f9f23fceaa1f15d1fd065e5032ac2d40ae3c0d3
3
+ metadata.gz: ae2bf8e389d7c36d4f7061a13f7f4bb213c8365408f9d0c1fb818ea25b385ff7
4
+ data.tar.gz: a3b8b537ea6bb06251edbc53a203d9c9e9c20a1d555679a722f00ff62eef9967
5
5
  SHA512:
6
- metadata.gz: 5f875810a4a5e53aae8ec0609e0ab86a3a97ba8661826de6dcd8cf3abd3091f3e4838d8c301ed0f962da0c5849123f8a1fa7c2614e5341126753a8556a0d6fce
7
- data.tar.gz: 50a7007476a9b989af9bfe989bc758a9febbc80478c541995044e1a6fd55fe54736520f643ca00845de57a4d5fc5ccfe1d83dd7994affff7530a3cd692f6fb96
6
+ metadata.gz: 418df72b1757e7a941e645e171d00f012072c5f2974352851015cc3501adf6a546703a0b3cc38c35de7f1d6034b05c4bf13ce162815640a609612d395a40ac3d
7
+ data.tar.gz: bef273caaf974979aa4a9e5dce7c4b0043efa47149a0b3f2bcba80874f7b3ff303e5f4672e90ab176d79306a4085ff6ed692f883709a37490e7df279595803e7
@@ -0,0 +1,34 @@
1
+ # VBW Dev Agent Memory
2
+
3
+ ## Project: source_monitor
4
+ - Rails engine gem for RSS/feed monitoring
5
+ - Ruby 3.4.4, Rails 8.x
6
+ - Test suite: 473 tests via `bin/rails test` (takes ~76 seconds)
7
+ - RuboCop uses `rubocop-rails-omakase` base config
8
+
9
+ ## Key Learnings
10
+
11
+ ### Shell/Bash in zsh
12
+ - `!` in zsh inline scripts causes `command not found` errors. Use `case` statements instead of `if ! ...` patterns.
13
+ - Use `IFS= read -r` when reading filenames from pipes to handle edge cases.
14
+
15
+ ### RuboCop scope vs git scope
16
+ - RuboCop inspects ALL Ruby-like files (Gemfile, Rakefile, .gemspec, .rake, bin/*, config.ru), not just `.rb` files.
17
+ - `git ls-files -- '*.rb'` only matches `.rb` extensions. Plans scoped to `.rb` files may miss RuboCop violations in non-`.rb` Ruby files.
18
+ - `test/tmp/` contains untracked generated Rails app templates. These are NOT git-tracked but RuboCop scans them unless excluded.
19
+
20
+ ### File structure
21
+ - `test/lib/tmp/install_generator/` contains test fixtures (1 tracked file: `config/initializers/source_monitor.rb`)
22
+ - `test/tmp/host_app_template_*` directories are generated test artifacts, not git-tracked
23
+ - `.rubocop.yml` is at project root, inherits from `rubocop-rails-omakase`
24
+
25
+ ### Frozen string literal pragma
26
+ - Completed in commit `5f02db8` -- 113 files modified
27
+ - Added `test/tmp/**/*` to RuboCop exclude list
28
+
29
+ ### RuboCop omakase ruleset details
30
+ - Only 45 cops enabled (out of 775 available)
31
+ - ALL Metrics cops disabled (ClassLength, MethodLength, BlockLength, etc.)
32
+ - After frozen_string_literal fix, codebase had zero remaining violations
33
+ - No `.rubocop.yml` exclusions needed for large files since Metrics cops are off
34
+ - Plan 02 completed with no code changes required
@@ -0,0 +1,49 @@
1
+ # Lead Agent Memory -- SourceMonitor
2
+
3
+ ## Project Quick Facts
4
+ - Rails 8 engine, Ruby 3.4+, PostgreSQL-only
5
+ - 325 git-tracked Ruby files, 130 test files
6
+ - Coverage baseline: 2328 uncovered lines in config/coverage_baseline.json
7
+ - RuboCop: rubocop-rails-omakase, config at .rubocop.yml
8
+ - CI: .github/workflows/ci.yml (lint, security, test, release-verify, nightly profiling)
9
+ - Large files: FeedFetcher (627), Configuration (655), ImportSessionsController (792)
10
+
11
+ ## Phase 1 Findings
12
+ - 98 git-tracked .rb files missing frozen_string_literal (out of 325)
13
+ - Breakdown: app/(5), lib/(24), test/(43 non-dummy), test/dummy/(22), config/(1), db/migrate/(4)
14
+ - test/tmp/ is NOT git-tracked (generated by install_generator tests) -- exclude from changes
15
+ - test/lib/tmp/install_generator/config/initializers/source_monitor.rb IS tracked and already has pragma
16
+ - bin/rubocop wrapper forces --config .rubocop.yml
17
+ - Phase 1 criterion #3 (10% coverage shrink) not addressed by plans 01/02 -- noted for future
18
+
19
+ ## Phase 2 Planning Insights
20
+ - Coverage baseline.json has 2117 uncovered lines across 105 files (not 2328 -- that was line count)
21
+ - Top 7 files account for ~743 uncovered lines (35% of total)
22
+ - FeedFetcher: 245 uncovered, 12 existing tests, uses VCR+WebMock
23
+ - ItemCreator: 228 uncovered, 8 existing tests, needs mock entries for Atom/JSON branches
24
+ - Configuration: 94 uncovered, 5 existing tests, ~12 nested settings classes
25
+ - Dashboard::Queries: 66 uncovered, 7 existing tests, uses raw SQL + Cache
26
+ - BulkSourceScraper: 66 uncovered, 6 existing tests, ActiveJob test adapter
27
+ - Broadcaster: 48 uncovered, NO existing test file -- needs creation
28
+ - SourcesIndexMetrics: 34 uncovered, 3 existing tests
29
+ - All 5 Phase 2 plans are Wave 1 (no inter-plan dependencies, no file conflicts)
30
+ - Testing main files will indirectly cover supporting files (retry_policy, fetch_error, state, enqueuer, etc.)
31
+ - Broadcaster tests need Turbo::StreamsChannel stubs -- no full Action Cable required
32
+ - 50% coverage target requires ~1059 lines covered; direct plans target ~630, rest from indirect
33
+
34
+ ## Phase 4 Planning Insights
35
+ - 3 plans: conventions-audit (wave 1), item-creator-extraction (wave 1), final-verification (wave 2)
36
+ - SourcesController has dead `fetch`/`retry` methods (leftover from Phase 3 CRUD extraction)
37
+ - ImportSessionsController `new` and `create` are byte-for-byte identical
38
+ - 4 RuboCop violations in db/migrate/20260210204022_add_composite_index_to_log_entries.rb
39
+ - Duplicate test: test/controllers/concerns/ vs test/controllers/source_monitor/concerns/
40
+ - ItemCreator at 601 lines is the last file over 300 lines -- extract to entry_parser + content_extractor
41
+ - Coverage baseline still at 2117 uncovered (not regenerated since Phase 1) -- must regenerate
42
+ - 60% reduction target: at most 847 uncovered lines
43
+ - dashboard/queries.rb (356) and application_helper.rb (346) are near 300 but acceptable as view/query code
44
+ - `bin/update-coverage-baseline` requires `COVERAGE=1 bin/rails test` first
45
+
46
+ ## Bash/Shell Notes
47
+ - zsh on macOS -- `!` in shell conditionals causes `command not found` errors
48
+ - Use `case` statements instead of `if ! grep` patterns in zsh
49
+ - git ls-files piped to while loops works reliably for file enumeration
@@ -0,0 +1,212 @@
1
+ # Release: PR, CI, Merge, and Gem Build
2
+
3
+ Orchestrate a full release cycle for the source_monitor gem. This command handles changelog generation, version bumping, PR creation, CI monitoring, auto-merge on success, release tagging, and gem build with push instructions.
4
+
5
+ ## Inputs
6
+
7
+ - `$ARGUMENTS` -- Optional: version bump description or release notes summary. If empty, derive from commits since last tag.
8
+
9
+ ## Step 1: Git Hygiene
10
+
11
+ Run these checks. If ANY fail, STOP and report the issue to the user.
12
+
13
+ 1. **Working tree clean**: `git status --porcelain` must be empty. If not, list the dirty files and ask the user whether to commit, stash, or abort.
14
+ 2. **On main branch**: `git branch --show-current` must be `main`. If not, ask the user if they want to continue from the current branch or switch.
15
+ 3. **Fetch latest**: `git fetch origin main`
16
+ 4. **Up to date with remote**: Compare `git rev-parse HEAD` with `git rev-parse origin/main`. If behind, ask the user whether to pull.
17
+ 5. **No unpushed commits**: Compare local HEAD with `origin/main`. If ahead, note how many commits are unpushed -- these will be included in the PR.
18
+
19
+ Report a summary:
20
+ ```
21
+ Git Status:
22
+ Branch: main
23
+ Clean: yes
24
+ Synced with origin: yes
25
+ Unpushed commits: N
26
+ ```
27
+
28
+ ## Step 2: Version Check
29
+
30
+ 1. Read `lib/source_monitor/version.rb` to get the current VERSION.
31
+ 2. Read the latest git tag with `git tag --sort=-v:refname | head -1`.
32
+ 3. If the VERSION matches the latest tag (e.g., both are `0.3.2`), the version hasn't been bumped. Ask the user:
33
+ - "Current version is X.Y.Z which already has a tag. What should the new version be?"
34
+ - Offer options: patch (X.Y.Z+1), minor (X.Y+1.0), major (X+1.0.0), or custom.
35
+ - Update `lib/source_monitor/version.rb` with the new version.
36
+ 4. If VERSION is ahead of the latest tag, proceed with the current version.
37
+
38
+ Store the release version for later steps. Do NOT commit yet -- that happens after the changelog is updated.
39
+
40
+ ## Step 3: Update CHANGELOG.md
41
+
42
+ The changelog follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) format. The file is at `CHANGELOG.md` in the project root.
43
+
44
+ 1. **Gather commit history** since the last tag:
45
+ ```
46
+ git log vPREVIOUS..HEAD --oneline --no-merges
47
+ ```
48
+ 2. **Categorize commits** into Keep a Changelog sections based on commit prefixes and content:
49
+ - `feat:`, `add:` --> **Added**
50
+ - `fix:`, `bugfix:` --> **Fixed**
51
+ - `chore:`, `refactor:`, `perf:` --> **Changed**
52
+ - `docs:` --> **Documentation** (only include if substantive)
53
+ - `BREAKING:` or `!:` --> **Breaking Changes** (at the top)
54
+ - `remove:`, `deprecate:` --> **Removed**
55
+ - Skip merge commits, version bumps, and CI-only changes.
56
+ - If `$ARGUMENTS` was provided, use it to inform/supplement the categorization.
57
+
58
+ 3. **Draft the changelog entry** and present it to the user for review:
59
+ ```
60
+ ## [X.Y.Z] - YYYY-MM-DD
61
+
62
+ ### Added
63
+ - <items>
64
+
65
+ ### Fixed
66
+ - <items>
67
+
68
+ ### Changed
69
+ - <items>
70
+ ```
71
+ Each bullet should be a concise, user-facing description (not a raw commit message). Consolidate related commits into single bullets where it makes sense.
72
+
73
+ 4. **Ask the user to approve** the changelog entry. Offer to edit if they want changes.
74
+
75
+ 5. **Write the entry** into `CHANGELOG.md`:
76
+ - Replace the `## [Unreleased]` section contents with `- No unreleased changes yet.`
77
+ - Insert the new versioned entry immediately after the `## [Unreleased]` block and before the previous release entry.
78
+ - Preserve all existing entries below.
79
+
80
+ ## Step 4: Commit Version Bump + Changelog
81
+
82
+ Once both `lib/source_monitor/version.rb` (if changed) and `CHANGELOG.md` are updated:
83
+
84
+ 1. Stage the changed files:
85
+ ```
86
+ git add lib/source_monitor/version.rb CHANGELOG.md
87
+ ```
88
+ 2. Create a single commit:
89
+ ```
90
+ chore: bump version to X.Y.Z and update changelog
91
+ ```
92
+ If only the changelog changed (version was already bumped), use:
93
+ ```
94
+ chore: update changelog for vX.Y.Z release
95
+ ```
96
+
97
+ ## Step 5: Create Release Branch and PR
98
+
99
+ 1. Create a release branch: `git checkout -b release/vX.Y.Z`
100
+ 2. Push the branch: `git push -u origin release/vX.Y.Z`
101
+ 3. Generate a PR body from the new CHANGELOG.md entry (use the entry written in Step 3, not raw commits).
102
+ 4. Create the PR using `gh pr create`:
103
+ - Title: `Release vX.Y.Z`
104
+ - Body format:
105
+ ```
106
+ ## Release vX.Y.Z
107
+
108
+ <paste the CHANGELOG.md entry content here>
109
+
110
+ ### Release Checklist
111
+ - [x] Version bumped in `lib/source_monitor/version.rb`
112
+ - [x] CHANGELOG.md updated
113
+ - [ ] CI passes (lint, security, test, release_verification)
114
+
115
+ ---
116
+ Auto-generated release PR by `/release` command.
117
+ ```
118
+ - Base: `main`
119
+ 5. Report the PR URL to the user.
120
+
121
+ ## Step 6: Monitor CI Pipeline
122
+
123
+ Poll the CI status using `gh pr checks <PR_NUMBER> --watch` or repeated `gh pr checks <PR_NUMBER>` calls. The CI has 4 required jobs: `lint`, `security`, `test`, `release_verification`.
124
+
125
+ 1. Tell the user: "Monitoring CI pipeline for PR #N... This typically takes 3-5 minutes."
126
+ 2. Poll with `gh pr checks <PR_NUMBER>` every 30 seconds, up to 15 minutes.
127
+ 3. After each poll, report progress if any jobs completed.
128
+
129
+ ### If CI PASSES (all checks green):
130
+
131
+ Continue to Step 7.
132
+
133
+ ### If CI FAILS:
134
+
135
+ 1. Get the failure details: `gh pr checks <PR_NUMBER>` to identify which jobs failed.
136
+ 2. For each failed job, fetch the log summary:
137
+ ```
138
+ gh run view <RUN_ID> --log-failed | tail -50
139
+ ```
140
+ 3. Present to the user:
141
+ ```
142
+ CI Failed on PR #N
143
+
144
+ Failed Jobs:
145
+ - <job_name>: <brief summary of failure>
146
+
147
+ Log excerpt:
148
+ <relevant log lines>
149
+ ```
150
+ 4. Ask the user what to do:
151
+ - "Fix the issues and re-push" -- Help fix the issues, commit, push, and restart CI monitoring
152
+ - "Close the PR and abort" -- Close the PR, delete the branch, switch back to main
153
+ - "Investigate manually" -- Stop and let the user handle it
154
+
155
+ ## Step 7: Auto-Merge PR
156
+
157
+ Once CI is green:
158
+
159
+ 1. Merge the PR: `gh pr merge <PR_NUMBER> --merge --delete-branch`
160
+ 2. Switch back to main and pull: `git checkout main && git pull origin main`
161
+ 3. Report: "PR #N merged successfully."
162
+
163
+ ## Step 8: Tag the Release
164
+
165
+ 1. Create an annotated tag:
166
+ ```
167
+ git tag -a vX.Y.Z -m "Release vX.Y.Z"
168
+ ```
169
+ 2. Push the tag: `git push origin vX.Y.Z`
170
+ 3. Create a GitHub release from the tag:
171
+ ```
172
+ gh release create vX.Y.Z --title "vX.Y.Z" --notes "<changelog entry from Step 3>"
173
+ ```
174
+ 4. Report the release URL.
175
+
176
+ ## Step 9: Build the Gem
177
+
178
+ 1. Clean any old gem files: remove any `source_monitor-*.gem` files in the project root.
179
+ 2. Build the gem: `gem build source_monitor.gemspec`
180
+ 3. Verify the gem was built: check for `source_monitor-X.Y.Z.gem` in the project root.
181
+ 4. Show the gem contents summary: `gem spec source_monitor-X.Y.Z.gem | head -30`
182
+
183
+ ## Step 10: Gem Push Instructions
184
+
185
+ Present the final instructions to the user:
186
+
187
+ ```
188
+ Release vX.Y.Z Complete!
189
+
190
+ Git tag: vX.Y.Z (pushed)
191
+ GitHub: <release URL>
192
+ Gem built: source_monitor-X.Y.Z.gem
193
+
194
+ To publish to RubyGems:
195
+
196
+ gem push source_monitor-X.Y.Z.gem
197
+
198
+ You'll be prompted for your RubyGems OTP code (check your authenticator app).
199
+ The gem has `rubygems_mfa_required` enabled, so the OTP is mandatory.
200
+
201
+ After pushing, verify at:
202
+ https://rubygems.org/gems/source_monitor/versions/X.Y.Z
203
+ ```
204
+
205
+ Do NOT run `gem push` automatically -- always let the user handle the OTP-protected push manually.
206
+
207
+ ## Error Recovery
208
+
209
+ - If any step fails unexpectedly, report what happened and where things stand.
210
+ - If a release branch already exists, ask the user whether to reuse or recreate it.
211
+ - If the tag already exists, skip tagging and inform the user.
212
+ - Always leave the user on the `main` branch in a clean state when possible.
@@ -61,7 +61,7 @@ bin/source_monitor install --yes
61
61
  gem "source_monitor", "~> 0.3.0" # in Gemfile
62
62
  bundle install
63
63
 
64
- # 2. Run the install generator
64
+ # 2. Run the install generator (mounts engine, creates initializer, configures recurring jobs)
65
65
  bin/rails generate source_monitor:install --mount-path=/source_monitor
66
66
 
67
67
  # 3. Copy engine migrations
@@ -70,7 +70,7 @@ bin/rails railties:install:migrations FROM=source_monitor
70
70
  # 4. Apply migrations
71
71
  bin/rails db:migrate
72
72
 
73
- # 5. Start background workers
73
+ # 5. Start background workers (recurring jobs configured automatically in config/recurring.yml)
74
74
  bin/rails solid_queue:start
75
75
 
76
76
  # 6. Verify
@@ -86,7 +86,7 @@ gem "source_monitor", github: "dchuk/source_monitor"
86
86
 
87
87
  ## What the Install Generator Does
88
88
 
89
- The generator (`SourceMonitor::Generators::InstallGenerator`) performs two actions:
89
+ The generator (`SourceMonitor::Generators::InstallGenerator`) performs three actions:
90
90
 
91
91
  1. **Mounts the engine** in `config/routes.rb`:
92
92
  ```ruby
@@ -97,6 +97,9 @@ The generator (`SourceMonitor::Generators::InstallGenerator`) performs two actio
97
97
  2. **Creates the initializer** at `config/initializers/source_monitor.rb`:
98
98
  Uses the template at `lib/generators/source_monitor/install/templates/source_monitor.rb.tt`. Skips if the file already exists.
99
99
 
100
+ 3. **Configures recurring jobs** in `config/recurring.yml`:
101
+ Adds entries for `ScheduleFetchesJob` (every minute), scrape scheduling (every 2 minutes), `ItemCleanupJob` (2am daily), and `LogCleanupJob` (3am daily). If the file doesn't exist, creates it with `default: &default` and environment sections. If it exists, merges entries without overwriting. Skips if SourceMonitor entries are already present.
102
+
100
103
  Re-running the generator is safe and idempotent.
101
104
 
102
105
  ## Post-Install Configuration
@@ -217,6 +220,7 @@ end
217
220
  - [ ] `bundle install` completed
218
221
  - [ ] Install generator ran (`bin/rails generate source_monitor:install`)
219
222
  - [ ] Engine migrations copied and applied
223
+ - [ ] Recurring jobs configured in `config/recurring.yml`
220
224
  - [ ] Solid Queue workers started
221
225
  - [ ] Authentication hooks configured in initializer
222
226
  - [ ] `bin/source_monitor verify` passes
@@ -39,6 +39,7 @@ bin/rails generate source_monitor:install --mount-path=/source_monitor
39
39
  Verify after running:
40
40
  - [ ] `config/routes.rb` contains `mount SourceMonitor::Engine, at: "/source_monitor"`
41
41
  - [ ] `config/initializers/source_monitor.rb` exists
42
+ - [ ] `config/recurring.yml` contains SourceMonitor recurring job entries
42
43
 
43
44
  ## Phase 4: Database Setup
44
45
 
@@ -75,6 +76,8 @@ end
75
76
 
76
77
  ## Phase 6: Configure Workers
77
78
 
79
+ The install generator automatically configures recurring jobs in `config/recurring.yml` (fetch scheduling, scrape scheduling, item cleanup, log cleanup). These run automatically with `bin/dev` or `bin/jobs`.
80
+
78
81
  Ensure `config/solid_queue.yml` (or equivalent) includes the SourceMonitor queues:
79
82
 
80
83
  ```yaml
@@ -159,15 +159,16 @@ SourceMonitor.queue_name(:fetch) # => "source_monitor_fetch"
159
159
 
160
160
  ### Recurring Jobs
161
161
 
162
- ScheduleFetchesJob is typically configured as a recurring job in `config/recurring.yml`:
163
-
164
- ```yaml
165
- production:
166
- schedule_fetches:
167
- class: SourceMonitor::ScheduleFetchesJob
168
- schedule: every 1 minute
169
- queue: source_monitor_fetch
170
- ```
162
+ The install generator (`bin/rails generate source_monitor:install`) automatically configures these recurring jobs in `config/recurring.yml`:
163
+
164
+ | Job | Schedule |
165
+ |-----|----------|
166
+ | `SourceMonitor::ScheduleFetchesJob` | every minute |
167
+ | `SourceMonitor::Scraping::Scheduler.run` | every 2 minutes |
168
+ | `SourceMonitor::ItemCleanupJob` | at 2am every day |
169
+ | `SourceMonitor::LogCleanupJob` | at 3am every day |
170
+
171
+ These run automatically with `bin/dev` or `bin/jobs`. If you need to customize, edit `config/recurring.yml` directly.
171
172
 
172
173
  ## Retry Policies
173
174
 
data/.gitignore CHANGED
@@ -15,3 +15,7 @@
15
15
  /app/assets/builds/*
16
16
  !/app/assets/builds/.keep
17
17
  !/app/assets/builds/source_monitor/
18
+ .vbw-planning/.cost-ledger.json
19
+ .vbw-planning/.notification-log.jsonl
20
+ .vbw-planning/.session-log.jsonl
21
+ .vbw-planning/.hook-errors.log
data/AGENTS.md CHANGED
@@ -1,13 +1,13 @@
1
1
  # Repository Guidelines
2
2
 
3
- refer to ./ai/project_overview.md for full scope of this project
3
+ Refer to `CLAUDE.md` for project conventions and skills catalog.
4
4
 
5
- use rbenv for all ruby and bundler/gem commands, not the system ruby
5
+ Use rbenv for all ruby and bundler/gem commands, not the system ruby.
6
6
 
7
7
  ## Contribution Workflow
8
8
 
9
9
  - Treat the engine like any external contributor would: no direct commits to `main`.
10
- - Before writing code, branch off the latest `origin/main` (use a descriptive `feature/` or `bugfix/` prefix) and open a draft PR on `github.com/dchuk/source_monitor` referencing the active `.ai/tasks.md` slice.
10
+ - Before writing code, branch off the latest `origin/main` (use a descriptive `feature/` or `bugfix/` prefix) and open a draft PR on `github.com/dchuk/source_monitor`.
11
11
  - Push early and often to that branch so history stays visible; keep commits scoped and rebases local to your branch only.
12
12
  - Move the PR out of draft once tests pass and the slice is ready for review; request at least one review and wait for all CI jobs (lint, security, Rails tests + diff coverage) to succeed before merging.
13
13
  - The `test` job enforces diff coverage via `bin/check-diff-coverage`; if legitimate gaps remain after new code paths, refresh `config/coverage_baseline.json` by running `bin/test-coverage` followed by `bin/update-coverage-baseline` on the updated branch and commit the regenerated baseline.
@@ -59,7 +59,7 @@ Run `bin/setup` to install gems, prepare the dummy database, and compile Tailwin
59
59
 
60
60
  ## Coding Style & Naming Conventions
61
61
 
62
- Use two-space indentation and Ruby 3.3 syntax. Keep engine classes under the `SourceMonitor::` namespace; new modules should mirror their directory, e.g., `lib/source_monitor/fetching/pipeline.rb`. Favor service objects ending in `Service`, jobs ending in `Job`, and background channels ending in `Channel`. Rails defaults handle formatting, but run `bundle exec rubocop` (configured via `.rubocop.yml`) before opening a PR. For views, stick with ERB and Tailwind utility classes.
62
+ Use two-space indentation and Ruby 4.0+ syntax. Keep engine classes under the `SourceMonitor::` namespace; new modules should mirror their directory, e.g., `lib/source_monitor/fetching/pipeline.rb`. Favor service objects ending in `Service`, jobs ending in `Job`, and background channels ending in `Channel`. Rails defaults handle formatting, but run `bundle exec rubocop` (configured via `.rubocop.yml`) before opening a PR. For views, stick with ERB and Tailwind utility classes.
63
63
 
64
64
  ## Testing Guidelines
65
65
 
@@ -67,66 +67,29 @@ MiniTest drives coverage (`test/models`, `test/controllers`, `test/system`). Nam
67
67
 
68
68
  ## Commit & Pull Request Guidelines
69
69
 
70
- Adopt imperative commit messages in the format `scope: action`, e.g., `sources: enforce URL normalization`. Group unrelated work into separate commits. PRs should describe context, summarise the slice delivered, list validation steps (`bin/rails test`, manual fetch run), and reference roadmap items from `.ai/tasks.md`. Include screenshots or console output when altering UI or background jobs. Request at least one review and ensure CI completes before merge.
70
+ Adopt imperative commit messages in the format `scope: action`, e.g., `sources: enforce URL normalization`. Group unrelated work into separate commits. PRs should describe context, summarise the slice delivered, and list validation steps (`bin/rails test`, manual fetch run). Include screenshots or console output when altering UI or background jobs. Request at least one review and ensure CI completes before merge.
71
71
 
72
72
  ## Security & Configuration Tips
73
73
 
74
74
  Store secrets (API keys, webhook tokens) in `config/credentials/` and never commit plain-text values. When adding HTTP endpoints or webhooks, default to Solid Queue middleware for retries and respect the allowlist in `config/source_monitor.yml`. Document new environment variables in `config/application.yml.sample` and call out any migrations that impact host apps.
75
75
 
76
- # Clean coding guidelines
76
+ ## Clean Coding Principles
77
77
 
78
- Object Orientated Design: You should write code that embraces change. Here’s how…
78
+ - **SRP**: Classes and methods should have a single responsibility.
79
+ - **DRY**: Avoid duplication; changes should only need one edit.
80
+ - **Depend on behaviour, not data**: Wrap instance variables in methods (`attr_reader`); use Struct for data structures.
81
+ - **Minimise dependencies**: Use dependency injection, encapsulate external messages, prefer hash arguments.
82
+ - **Depend on things that change less often than you do.**
79
83
 
80
- SRP: A class should do the smallest possible useful thing; it should have a single responsibility. A class that has more than one responsibility is difficult to reuse. SRP requires that a class be cohesive, that everything the class does be highly related to its purpose. A class that is easy to reuse will make the application easier to change.
84
+ ## Claude Code Skills
81
85
 
82
- Methods, like classes, should have a single responsibility. All of the same reasons apply, having just one responsibility makes them easy to change and easy to reuse.
86
+ SourceMonitor ships 14 engine-specific Claude Code skills (`sm-*` prefix) covering the domain model, configuration DSL, pipeline stages, testing conventions, and more. Skills are distributed with the gem and installed into `.claude/skills/` via rake tasks:
83
87
 
84
- DRY: DRY code tolerates change because any change in behaviour can be made by changing code in just one place.
88
+ ```bash
89
+ bin/rails source_monitor:skills:install # Consumer skills (host app integration)
90
+ bin/rails source_monitor:skills:contributor # Contributor skills (engine development)
91
+ bin/rails source_monitor:skills:all # All skills
92
+ bin/rails source_monitor:skills:remove # Remove all sm-* skills
93
+ ```
85
94
 
86
- Depend on behaviour not data:
87
-
88
- Hide instance variables by wrapping them in a method, an attr_reader. The method changes a variable from data ( which is referenced all over) to behaviour (which is defined once). This means if you need to adjust the data you only have to make a change in one place.
89
-
90
- Hide data structures using the Struct class.
91
-
92
- Minimise Dependencies: An object depends on another object if, when one object changes, the other might be forced to change in turn. An object has a dependency when it knows:
93
-
94
- The name of another class.
95
-
96
- The name of a message that it intends to send to someone other than self.
97
-
98
- The arguments that a message requires.
99
-
100
- The order of those arguments.
101
-
102
- Dependency Injection: Addresses the first dependency. It decouples two objects by moving the creation of object_A outside of object_B eg:
103
-
104
- object_A.new(1,2,object_B.new)
105
-
106
- Encapsulation: Addresses the second dependency. Create wrapper methods so that external messages are isolated in one place. The wrapper method is called throughout the class, rather than sending messages externally.
107
-
108
- Use hashes for initialization arguments: this addresses the final dependencies.
109
-
110
- Depend on things that change less often than you do
111
-
112
- ## Agent Notes (2025-10-08)
113
-
114
- - Finished roadmap section 05.03 (structured fetch error handling). Added `SourceMonitor::Fetching::FetchError` hierarchy, expanded `FeedFetcher` error handling, and ensured instrumentation payloads include success/error context.
115
- - Fetch attempts now always create `FetchLog` records and update `Source` failure state; added MiniTest coverage for timeout, HTTP 404, and malformed feed scenarios.
116
- - Relevant tests: `bundle exec rails test test/lib/source_monitor/fetching/feed_fetcher_test.rb` and full suite via `bundle exec rails test` (both passing).
117
- - Next roadmap item is 05.04 (not started). Untracked tmp dir `test/lib/tmp` can be safely ignored if it reappears.
118
-
119
- ## Agent Notes (2025-10-09)
120
-
121
- - Introduced `SourceMonitor::Scrapers::Base`, establishing the scraper adapter contract and Result object. Subclasses merge default, source, and invocation settings (HashWithIndifferentAccess) and must return a Result with status/html/content/metadata. Use `SourceMonitor::Scrapers::Base.call(item:, source:, settings:, http:)` to execute adapters.
122
- - Added Readability scraper adapter leveraging SourceMonitor::HTTP, Nokolexbor, and ruby-readability. Supports override selectors via `scrape_settings[:selectors]`, records extraction metadata including strategy and inferred status, and returns structured failure results on HTTP errors.
123
- - Extracted scraper HTTP fetching and parsing into dedicated collaborators (`SourceMonitor::Scrapers::Fetchers::HttpFetcher` and `SourceMonitor::Scrapers::Parsers::ReadabilityParser`). The adapter now orchestrates these services, simplifying future parser additions while keeping failure metadata consistent.
124
- - Added Readability scraper adapter leveraging SourceMonitor::HTTP, Nokolexbor, and ruby-readability. Supports override selectors via `scrape_settings[:selectors]`, records extraction metadata including strategy and inferred status, and returns structured failure results on HTTP errors.
125
- - Phase 07.03 complete: introduced `SourceMonitor::ItemContent` table with one-to-one association, virtual accessors on `Item`, and migration that backfills existing scraped data while dropping legacy columns. Clearing both fields now removes the content row. Run `bundle exec rails db:migrate` in host apps after upgrading.
126
- - Phase 07.04 complete: added `SourceMonitor::Scraping::ItemScraper` service with log recording, manual scrape controls in the item admin, and a dedicated scraping configuration section on the source form. Item show page now highlights content comparisons, status badges, and recent scrape errors. System coverage updated to exercise the manual scraping flow.
127
- - Phase 08.02 complete: created `SourceMonitor::FetchFeedJob`, introduced `SourceMonitor::Fetching::FetchRunner` with Postgres advisory locking, and added a temporary `ScrapeItemJob` stub so new items from auto-scrape sources are queued with `scrape_status` set to `pending`. Manual fetch UI now reuses the runner, and tests cover job retry behavior plus follow-up scraping enqueue (`bundle exec rails test test/lib/source_monitor/fetching/fetch_runner_test.rb`, `bundle exec rails test test/jobs/source_monitor/fetch_feed_job_test.rb`, and `bundle exec rails test test/lib/source_monitor/fetching/feed_fetcher_test.rb`).
128
- - Phase 08.03 complete: implemented `SourceMonitor::Scraping::Enqueuer` with deduplication/auto-scrape guards, promoted `SourceMonitor::ScrapeItemJob` to execute `ItemScraper`, and updated the manual item view to enqueue jobs instead of running scrapes inline. Added unit coverage for the enqueuer/job plus refreshed the manual scrape system test (`bundle exec rails test test/lib/source_monitor/scraping/enqueuer_test.rb test/jobs/source_monitor/scrape_item_job_test.rb test/system/items_test.rb test/lib/source_monitor/fetching/fetch_runner_test.rb`).
129
- - Phase 08.04 complete: manual "Fetch Now" now queues `FetchFeedJob` via `FetchRunner.enqueue`, dashboard shows live queue metrics from `SourceMonitor::Jobs::Visibility`, and optional Mission Control link appears when configured. System tests updated to assert job enqueues (`test/system/sources_test.rb`, `test/system/items_test.rb`, `test/system/dashboard_test.rb`); run `bundle exec rails test test/system` after UI tweaks.
130
- - Install generator now also creates `config/initializers/source_monitor.rb` with documented defaults for queue settings, metrics, and Mission Control hooks. Update the initializer if future configuration options are added.
131
- - `SourceMonitor.mission_control_dashboard_path` now validates that the configured path exists before rendering a dashboard link, preventing 404s when Mission Control isn't mounted yet. Template comments outline how to mount the engine when enabling it.
132
- - `test/dummy/bin/dev` now runs the Rails server with `BUNDLE_GEMFILE` pointing at the dummy app, so dummy-only gems (like mission_control-jobs) load correctly while the Tailwind watcher still uses the engine's Gemfile.
95
+ See `CLAUDE.md` for the full skills catalog and usage details.
data/CHANGELOG.md CHANGED
@@ -15,6 +15,28 @@ All notable changes to this project are documented below. The format follows [Ke
15
15
 
16
16
  - No unreleased changes yet.
17
17
 
18
+ ## [0.3.3] - 2026-02-11
19
+
20
+ ### Fixed
21
+
22
+ - Added missing `recurring.yml` configuration to the install generator so host apps get Solid Queue recurring job config on install.
23
+ - Fixed YAML alias parsing in install generator so merging into existing `recurring.yml` files with `<<: *default` anchors works correctly.
24
+
25
+ ### Changed
26
+
27
+ - Updated `sm-host-setup` and `sm-job` skills to reflect latest conventions.
28
+ - Updated setup documentation with current installation steps.
29
+
30
+ ## [0.3.2] - 2026-02-10
31
+
32
+ ### Fixed
33
+
34
+ - Updated README, AGENTS.md, CONTRIBUTING.md, and docs/ to reflect v0.3.1 changes (Ruby 4.0+, gem version refs, skills system documentation).
35
+ - Replaced stale `.ai/` references with `CLAUDE.md` and `AGENTS.md` across all docs.
36
+ - Corrected `SOURCE_MONITOR_TEST_WORKERS` env var to `PARALLEL_WORKERS` in CONTRIBUTING.md.
37
+ - Removed historical agent development notes from AGENTS.md.
38
+ - Condensed verbose clean coding guidelines into concise bullet summary.
39
+
18
40
  ## [0.3.1] - 2026-02-10
19
41
 
20
42
  ### Added
data/CLAUDE.md CHANGED
@@ -45,7 +45,7 @@ Run /vbw:help for all commands.
45
45
 
46
46
  | Layer | Technology |
47
47
  |-------|------------|
48
- | Ruby | 3.4+ |
48
+ | Ruby | 4.0+ |
49
49
  | Rails | 8.x |
50
50
  | Testing | Minitest (no fixtures -- uses factory helpers + WebMock/VCR) |
51
51
  | Authorization | Host app responsibility (mountable engine) |
data/CONTRIBUTING.md CHANGED
@@ -4,7 +4,7 @@ This project ships each roadmap slice behind a fast, reliable test loop. The not
4
4
 
5
5
  ## Test Suite Expectations
6
6
 
7
- - **Fast feedback:** run `bundle exec rake app:test:smoke` for unit, helper, and job coverage before pushing. Use `SOURCE_MONITOR_TEST_WORKERS=1` locally when profiling failures for determinism.
7
+ - **Fast feedback:** run `bundle exec rake app:test:smoke` for unit, helper, and job coverage before pushing. Use `PARALLEL_WORKERS=1` locally when profiling failures for determinism.
8
8
  - **Full validation:** continue to run `bundle exec rails test` (or `bin/test-coverage` in CI) before marking a slice ready for review.
9
9
  - **Background jobs:** the default adapter is `:test`. Switch to inline execution only for the precise block that needs it via `with_inline_jobs { ... }`; never flip the global adapter.
10
10
  - **Database usage:** prefer transactional fixtures over manual `delete_all`. Reach for `setup_once` (TestProf’s `before_all`) when immutable data can be shared safely across examples.
@@ -13,16 +13,16 @@ This project ships each roadmap slice behind a fast, reliable test loop. The not
13
13
  ## Performance Profiling
14
14
 
15
15
  - **Local commands:**
16
- - `TAG_PROF=type SOURCE_MONITOR_TEST_WORKERS=1 bundle exec rails test`
17
- - `EVENT_PROF=sql.active_record SOURCE_MONITOR_TEST_WORKERS=1 bundle exec rails test`
18
- - `TEST_STACK_PROF=1 SOURCE_MONITOR_TEST_WORKERS=1 bundle exec rails test test/integration`
16
+ - `TAG_PROF=type PARALLEL_WORKERS=1 bundle exec rails test`
17
+ - `EVENT_PROF=sql.active_record PARALLEL_WORKERS=1 bundle exec rails test`
18
+ - `TEST_STACK_PROF=1 PARALLEL_WORKERS=1 bundle exec rails test test/integration`
19
19
  - **CI automation:** a scheduled `profiling` workflow runs the commands above nightly, archives `tmp/test_prof`, and applies guardrails via `bin/check-test-prof-metrics` (suite ≤ 80s, integration ≤ 35s, DB time ≤ 5s). Failures block the job so regressions surface quickly.
20
20
 
21
21
  ## Code Review Checklist
22
22
 
23
23
  Before approval, verify that the changes:
24
24
 
25
- 1. Avoid unnecessary database persistence (`build_stubbed`/test doubles over `.create` when assertions allow).
25
+ 1. Avoid unnecessary database persistence (test doubles over `.create` when assertions allow).
26
26
  2. Share heavy setup with `setup_once` or helper memoization instead of per-test rebuilds.
27
27
  3. Scope inline job execution to `with_inline_jobs { ... }` blocks—no suite-wide adapter swaps.
28
28
  4. Keep new or modified system specs lean, relying on lower-level coverage when UI is not essential.
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- source_monitor (0.3.1)
4
+ source_monitor (0.3.3)
5
5
  cssbundling-rails (~> 1.4)
6
6
  faraday (~> 2.9)
7
7
  faraday-follow_redirects (~> 0.4)