source_monitor 0.7.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,156 @@
1
+ ---
2
+ phase: "02"
3
+ plan: "01"
4
+ title: "Split FeedFetcherTest into Concern-Based Classes"
5
+ wave: 1
6
+ depends_on: []
7
+ must_haves:
8
+ - "REQ-PERF-01: FeedFetcherTest split into 6+ independent test files by concern"
9
+ - "Original feed_fetcher_test.rb deleted or replaced with require-only shim"
10
+ - "Each new test file is independently runnable with PARALLEL_WORKERS=1"
11
+ - "All 71 FeedFetcherTest tests pass individually and in full suite"
12
+ - "Shared build_source and body_digest helpers extracted to shared module"
13
+ - "RuboCop zero offenses on all new/modified test files"
14
+ skills_used: []
15
+ ---
16
+
17
+ # Plan 01: Split FeedFetcherTest into Concern-Based Classes
18
+
19
+ ## Objective
20
+
21
+ Split the monolithic `FeedFetcherTest` (71 tests, 84.8s, 64% of total runtime) into 6+ smaller test classes by concern. This is the single highest-impact optimization: Minitest parallelizes by class, so one 71-test class gets assigned to one worker. Splitting enables parallel distribution across all CPU cores.
22
+
23
+ ## Context
24
+
25
+ - `@` `test/lib/source_monitor/fetching/feed_fetcher_test.rb` -- 1350-line monolithic test class (the file to split)
26
+ - `@` `test/test_helper.rb` -- base test setup with `create_source!` and `clean_source_monitor_tables!`
27
+ - `@` `test/test_prof.rb` -- TestProf `before_all` and `setup_once` support
28
+
29
+ **Rationale:** The test file already has section comments (Task 1-6) that map to concern groups. The `build_source` and `body_digest` private helpers are shared across all tests and must be extracted to a module.
30
+
31
+ ## Tasks
32
+
33
+ ### Task 1: Create shared helper module for FeedFetcher tests
34
+
35
+ Create `test/lib/source_monitor/fetching/feed_fetcher_test_helper.rb`:
36
+
37
+ ```ruby
38
+ # frozen_string_literal: true
39
+
40
+ module SourceMonitor
41
+ module Fetching
42
+ module FeedFetcherTestHelper
43
+ private
44
+
45
+ def build_source(name:, feed_url:, fetch_interval_minutes: 360, adaptive_fetching_enabled: true)
46
+ create_source!(
47
+ name: name,
48
+ feed_url: feed_url,
49
+ fetch_interval_minutes: fetch_interval_minutes,
50
+ adaptive_fetching_enabled: adaptive_fetching_enabled
51
+ )
52
+ end
53
+
54
+ def body_digest(body)
55
+ Digest::SHA256.hexdigest(body)
56
+ end
57
+ end
58
+ end
59
+ end
60
+ ```
61
+
62
+ ### Task 2: Create the 6 split test files
63
+
64
+ Split the 71 tests into these files, each requiring `test_helper` and `feed_fetcher_test_helper`:
65
+
66
+ 1. **`feed_fetcher_success_test.rb`** (~13 tests) -- Success paths: RSS/Atom/JSON fetching, log entries, instrumentation notifications, ETag/304 handling, Netflix cassette test. Tests: "continues processing when an item creation fails", "fetches an RSS feed and records log entries", "reuses etag and handles 304", "parses rss atom and json feeds via feedjira", "fetches Netflix Tech Blog feed via Medium RSS".
67
+
68
+ 2. **`feed_fetcher_error_handling_test.rb`** (~12 tests) -- Error wrapping and connection failures: all Faraday error type wrapping, AIA certificate resolution tests (retry on SSL, nil resolve, non-SSL skip), generic Faraday::Error, unexpected StandardError, HTTPError from ClientError, re-raise without double-wrap. Tests from "Task 2" section plus AIA tests.
69
+
70
+ 3. **`feed_fetcher_adaptive_interval_test.rb`** (~8 tests) -- Adaptive fetch interval: decrease on content change, increase on no change, configured settings, min/max bounds, failure increase with backoff, disabled adaptive fetching. Tests from the interval section.
71
+
72
+ 4. **`feed_fetcher_retry_circuit_test.rb`** (~7 tests) -- Retry strategy and circuit breaker: reset on success, reset on 304, apply retry state, circuit open when exhausted, next_fetch_at earliest logic, policy error handling. Tests from "Task 1" section.
73
+
74
+ 5. **`feed_fetcher_entry_processing_test.rb`** (~7 tests) -- Entry processing: empty entries, error normalization (with guid, without guid), created/updated tracking, unchanged items, entries_digest fallback (url, title), failure result empty processing. Tests from "Task 4" section plus "process_feed_entries tracks created and updated" and "unchanged items".
75
+
76
+ 6. **`feed_fetcher_utilities_test.rb`** (~16 tests) -- Utility methods: jitter_offset, adjusted_interval_with_jitter, body_digest, updated_metadata, feed_signature_changed?, configured_seconds, configured_positive, configured_non_negative, interval_minutes_for, parse_http_time, extract_numeric. Tests from "Task 5" section plus "Task 3" header tests (If-Modified-Since, custom_headers, ETag update, Last-Modified update/304/unparseable).
77
+
78
+ Each file follows this pattern:
79
+ ```ruby
80
+ # frozen_string_literal: true
81
+
82
+ require "test_helper"
83
+ require "faraday"
84
+ require "uri"
85
+ require "digest"
86
+ require_relative "feed_fetcher_test_helper"
87
+
88
+ module SourceMonitor
89
+ module Fetching
90
+ class FeedFetcherSuccessTest < ActiveSupport::TestCase
91
+ include FeedFetcherTestHelper
92
+ # tests here...
93
+ end
94
+ end
95
+ end
96
+ ```
97
+
98
+ ### Task 3: Delete original feed_fetcher_test.rb
99
+
100
+ After all 6 new files are created and verified, delete the original `test/lib/source_monitor/fetching/feed_fetcher_test.rb`. Do NOT keep a shim -- the new files are self-contained.
101
+
102
+ ### Task 4: Verify all tests pass and lint clean
103
+
104
+ Run verification:
105
+ ```bash
106
+ # Run each new file individually
107
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_success_test.rb
108
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_error_handling_test.rb
109
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_adaptive_interval_test.rb
110
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_retry_circuit_test.rb
111
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_entry_processing_test.rb
112
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_utilities_test.rb
113
+
114
+ # Full suite
115
+ bin/rails test
116
+
117
+ # Lint
118
+ bin/rubocop test/lib/source_monitor/fetching/
119
+ ```
120
+
121
+ Ensure the total test count remains 1031+ (no tests lost or duplicated).
122
+
123
+ ## Files
124
+
125
+ | Action | Path |
126
+ |--------|------|
127
+ | CREATE | `test/lib/source_monitor/fetching/feed_fetcher_test_helper.rb` |
128
+ | CREATE | `test/lib/source_monitor/fetching/feed_fetcher_success_test.rb` |
129
+ | CREATE | `test/lib/source_monitor/fetching/feed_fetcher_error_handling_test.rb` |
130
+ | CREATE | `test/lib/source_monitor/fetching/feed_fetcher_adaptive_interval_test.rb` |
131
+ | CREATE | `test/lib/source_monitor/fetching/feed_fetcher_retry_circuit_test.rb` |
132
+ | CREATE | `test/lib/source_monitor/fetching/feed_fetcher_entry_processing_test.rb` |
133
+ | CREATE | `test/lib/source_monitor/fetching/feed_fetcher_utilities_test.rb` |
134
+ | DELETE | `test/lib/source_monitor/fetching/feed_fetcher_test.rb` |
135
+
136
+ ## Verification
137
+
138
+ ```bash
139
+ # Individual file runs (PARALLEL_WORKERS=1 due to PG fork segfault on single files)
140
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_success_test.rb
141
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_utilities_test.rb
142
+
143
+ # Full suite (all 1031+ tests pass)
144
+ bin/rails test
145
+
146
+ # Lint
147
+ bin/rubocop test/lib/source_monitor/fetching/
148
+ ```
149
+
150
+ ## Success Criteria
151
+
152
+ - 6+ new test files exist in `test/lib/source_monitor/fetching/`
153
+ - Original `feed_fetcher_test.rb` deleted
154
+ - `grep -c "class Feed" test/lib/source_monitor/fetching/*_test.rb` shows 6+ classes
155
+ - All 1031+ tests pass in full suite
156
+ - Each file runnable independently with PARALLEL_WORKERS=1
@@ -0,0 +1,33 @@
1
+ ---
2
+ plan: "02"
3
+ phase: "02"
4
+ title: "Log Level Reduction and Integration Test Tagging"
5
+ status: complete
6
+ commits:
7
+ - hash: edbfe23
8
+ message: "perf(02-02): reduce test log IO and add test:fast rake task"
9
+ tasks_completed: 4
10
+ tasks_total: 4
11
+ files_modified:
12
+ - test/dummy/config/environments/test.rb
13
+ files_created:
14
+ - lib/tasks/test_fast.rake
15
+ ---
16
+
17
+ ## What Was Built
18
+
19
+ - Set `config.log_level = :warn` in test environment to eliminate ~95MB of debug log IO per test run
20
+ - Created `lib/tasks/test_fast.rake` providing `test:fast` rake task that excludes integration/ and system/ directories
21
+ - Verified all 4 integration test files already in `test/integration/` (no moves needed)
22
+ - Full suite: 1033 runs, 0 failures; Fast mode: 1022 runs, 0 failures
23
+
24
+ ## Files Modified
25
+
26
+ - `test/dummy/config/environments/test.rb` — added `config.log_level = :warn` after `config.cache_store = :null_store`
27
+ - `lib/tasks/test_fast.rake` — new rake task `test:fast` using Dir glob to exclude integration and system test files
28
+
29
+ ## Deviations
30
+
31
+ - Plan specified `--exclude-pattern` flag for minitest but this flag does not exist in Rails/Minitest. Replaced with Dir glob approach that rejects `test/integration/` and `test/system/` paths (DEVN-01 Minor).
32
+ - Also excluded `test/system/` from fast mode since `bin/rails test` already excludes system tests by default — this makes `test:fast` equivalent to `bin/rails test` minus integration tests.
33
+ - Rake task accessible as `bundle exec rake app:test:fast` from engine root (engine prefixes with `app:`).
@@ -0,0 +1,120 @@
1
+ ---
2
+ phase: "02"
3
+ plan: "02"
4
+ title: "Log Level Reduction and Integration Test Tagging"
5
+ wave: 1
6
+ depends_on: []
7
+ must_haves:
8
+ - "REQ-PERF-02: config.log_level = :warn in test/dummy/config/environments/test.rb"
9
+ - "REQ-PERF-03: Integration tests tagged so --exclude-pattern can skip them"
10
+ - "host_install_flow_test.rb and release_packaging_test.rb moved under test/integration/"
11
+ - "bin/rails test --exclude-pattern='**/integration/**' excludes slow integration tests"
12
+ - "bin/rails test runs all tests including integration (default behavior preserved)"
13
+ - "RuboCop zero offenses on modified files"
14
+ skills_used: []
15
+ ---
16
+
17
+ # Plan 02: Log Level Reduction and Integration Test Tagging
18
+
19
+ ## Objective
20
+
21
+ Two quick wins that reduce test wall-clock time: (1) eliminate 95MB of debug log IO by setting test log level to `:warn` (saves 5-15s), and (2) ensure integration tests are organized under `test/integration/` so developers can exclude the 31s of subprocess-spawning tests during iterative development using `--exclude-pattern`.
22
+
23
+ ## Context
24
+
25
+ - `@` `test/dummy/config/environments/test.rb` -- currently has no explicit log_level (defaults to :debug)
26
+ - `@` `test/integration/host_install_flow_test.rb` -- slow integration test (subprocess spawning, ~15s)
27
+ - `@` `test/integration/release_packaging_test.rb` -- slow integration test (gem build + subprocess, ~15s)
28
+ - `@` `test/integration/engine_mounting_test.rb` -- fast integration test (route checks)
29
+ - `@` `test/integration/navigation_test.rb` -- empty placeholder test
30
+ - `@` `test/test_helper.rb` -- contains DEFAULT_TEST_EXCLUDE but does NOT need modification
31
+
32
+ **Rationale:** The research found 95MB of :debug log output during tests. Setting :warn eliminates this IO without losing any test coverage. Integration tests already live under `test/integration/` -- we just need to verify the exclude pattern works and document it.
33
+
34
+ ## Tasks
35
+
36
+ ### Task 1: Set test log level to :warn
37
+
38
+ In `test/dummy/config/environments/test.rb`, add after the `config.cache_store = :null_store` line:
39
+
40
+ ```ruby
41
+ # Reduce log IO in tests -- :debug generates ~95MB of output.
42
+ config.log_level = :warn
43
+ ```
44
+
45
+ This is a single-line addition. The research confirmed this saves 5-15s per run by eliminating disk IO for debug/info log messages.
46
+
47
+ ### Task 2: Verify integration test directory organization
48
+
49
+ Confirm all 4 integration test files are properly under `test/integration/`:
50
+ - `test/integration/host_install_flow_test.rb` (slow -- subprocess spawning)
51
+ - `test/integration/release_packaging_test.rb` (slow -- gem build)
52
+ - `test/integration/engine_mounting_test.rb` (fast -- route assertions)
53
+ - `test/integration/navigation_test.rb` (empty placeholder)
54
+
55
+ No file moves needed -- they are already in the correct location. The `--exclude-pattern` flag works with glob patterns on the file path.
56
+
57
+ ### Task 3: Add Rake task for fast test runs
58
+
59
+ Create `lib/tasks/test_fast.rake` with a convenience task:
60
+
61
+ ```ruby
62
+ # frozen_string_literal: true
63
+
64
+ namespace :test do
65
+ desc "Run tests excluding slow integration tests"
66
+ task fast: :environment do
67
+ $stdout.puts "Running tests excluding integration/ directory..."
68
+ system(
69
+ "bin/rails", "test",
70
+ "--exclude-pattern", "**/integration/**",
71
+ exception: true
72
+ )
73
+ end
74
+ end
75
+ ```
76
+
77
+ This provides `bin/rails test:fast` as a developer convenience that excludes the integration directory.
78
+
79
+ ### Task 4: Verify both test modes work
80
+
81
+ Run verification:
82
+ ```bash
83
+ # Full suite (all tests including integration)
84
+ bin/rails test
85
+
86
+ # Fast mode (excluding integration)
87
+ bin/rails test --exclude-pattern="**/integration/**"
88
+
89
+ # Lint modified files
90
+ bin/rubocop test/dummy/config/environments/test.rb lib/tasks/test_fast.rake
91
+ ```
92
+
93
+ Verify the fast mode excludes the integration tests and completes significantly faster.
94
+
95
+ ## Files
96
+
97
+ | Action | Path |
98
+ |--------|------|
99
+ | MODIFY | `test/dummy/config/environments/test.rb` |
100
+ | CREATE | `lib/tasks/test_fast.rake` |
101
+
102
+ ## Verification
103
+
104
+ ```bash
105
+ # Full suite passes
106
+ bin/rails test
107
+
108
+ # Exclude pattern works (fewer tests, no integration)
109
+ bin/rails test --exclude-pattern="**/integration/**"
110
+
111
+ # Lint
112
+ bin/rubocop test/dummy/config/environments/test.rb lib/tasks/test_fast.rake
113
+ ```
114
+
115
+ ## Success Criteria
116
+
117
+ - `grep "log_level" test/dummy/config/environments/test.rb` shows `:warn`
118
+ - `bin/rails test` runs all 1031+ tests (no regressions)
119
+ - `bin/rails test --exclude-pattern="**/integration/**"` runs successfully with fewer tests
120
+ - `lib/tasks/test_fast.rake` exists and is syntactically valid
@@ -0,0 +1,30 @@
1
+ ---
2
+ phase: 2
3
+ plan: 3
4
+ status: complete
5
+ ---
6
+ # Plan 03 Summary: Adopt before_all in DB-Heavy Test Files
7
+
8
+ ## Tasks Completed
9
+ - [x] Task 1: Convert sources_index_metrics_test.rb to setup_once (17 read-only tests)
10
+ - [x] Task 2: Convert 3 single-test files to setup_once for consistency
11
+ - [x] Task 3: Verify all converted files individually (PARALLEL_WORKERS=1)
12
+ - [x] Task 4: Full suite verification (1033 tests, 0 failures) and lint (0 offenses)
13
+
14
+ ## Commits
15
+ - 912665f: perf(02-03): adopt setup_once/before_all in DB-heavy test files
16
+
17
+ ## Files Modified
18
+ - test/lib/source_monitor/analytics/sources_index_metrics_test.rb (modified)
19
+ - test/lib/source_monitor/analytics/source_activity_rates_test.rb (modified)
20
+ - test/lib/source_monitor/analytics/source_fetch_interval_distribution_test.rb (modified)
21
+ - test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb (modified)
22
+
23
+ ## What Was Built
24
+ - Converted `sources_index_metrics_test.rb` from per-test setup to `setup_once` for shared fixture creation (3 sources + 3 items), following the reference pattern from `query_test.rb` (store IDs in setup_once, re-find records in per-test setup)
25
+ - Kept `travel_to`/`travel_back` in regular setup/teardown for thread-local time safety
26
+ - Converted 3 single-test files to `setup_once` for `clean_source_monitor_tables!` (functionally identical for single-test classes but normalizes the pattern)
27
+ - `setup_once` usage increased from 1 file to 5 files across the test suite
28
+
29
+ ## Deviations
30
+ - None
@@ -0,0 +1,154 @@
1
+ ---
2
+ phase: "02"
3
+ plan: "03"
4
+ title: "Adopt before_all in DB-Heavy Test Files"
5
+ wave: 1
6
+ depends_on: []
7
+ must_haves:
8
+ - "REQ-PERF-05: Top DB-heavy test files converted from per-test setup to setup_once/before_all"
9
+ - "sources_index_metrics_test.rb converted to setup_once (17 tests, shared read-only fixtures)"
10
+ - "Additional eligible files converted where safe (read-only shared data)"
11
+ - "Only read-only test data shared via setup_once (tests that mutate data keep per-test setup)"
12
+ - "All converted tests pass individually with PARALLEL_WORKERS=1"
13
+ - "Full test suite passes with no isolation regressions"
14
+ - "RuboCop zero offenses on modified files"
15
+ skills_used: []
16
+ ---
17
+
18
+ # Plan 03: Adopt before_all in DB-Heavy Test Files
19
+
20
+ ## Objective
21
+
22
+ Convert eligible DB-heavy test files from per-test `setup` to `setup_once`/`before_all` for shared fixture creation. The `setup_once` helper (alias for `before_all`) is already wired up in `test/test_prof.rb` but only used in 1 of 54 eligible files. This saves ~3-5s by eliminating redundant database INSERT/DELETE cycles.
23
+
24
+ ## Context
25
+
26
+ - `@` `test/test_prof.rb` -- `setup_once` (alias for `before_all`) already configured and included in `ActiveSupport::TestCase`
27
+ - `@` `test/lib/source_monitor/logs/query_test.rb` -- only existing user of `setup_once` (reference pattern)
28
+ - `@` `test/lib/source_monitor/analytics/sources_index_metrics_test.rb` -- 17 tests, shared read-only fixtures. **PRIMARY candidate: creates 3 sources + 3 items in setup, all tests only query this data.**
29
+ - `@` `test/lib/source_monitor/analytics/source_activity_rates_test.rb` -- 1 test, uses `clean_source_monitor_tables!`
30
+ - `@` `test/lib/source_monitor/analytics/source_fetch_interval_distribution_test.rb` -- 1 test, uses `clean_source_monitor_tables!`
31
+ - `@` `test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb` -- 1 test, uses `clean_source_monitor_tables!`
32
+
33
+ **Safety analysis performed:**
34
+ - `sources_index_metrics_test.rb`: SAFE. All 17 tests construct `SourcesIndexMetrics.new(...)` and call read-only query methods. No test creates, updates, or deletes records.
35
+ - `source_activity_rates_test.rb`: SAFE but minimal benefit (1 test, setup runs once either way).
36
+ - `source_fetch_interval_distribution_test.rb`: SAFE but minimal benefit (1 test).
37
+ - `upcoming_fetch_schedule_test.rb`: SAFE but minimal benefit (1 test).
38
+ - `dashboard/queries_test.rb`: NOT SAFE. Each test creates its own sources and checks specific counts. Shared state would cause pollution.
39
+ - `health/source_health_monitor_test.rb`: NOT SAFE. Tests mutate `@source` via `SourceHealthMonitor.call`.
40
+ - `items/item_creator_test.rb`: NOT SAFE. Tests create items on shared source and check counts.
41
+
42
+ **Rationale:** `before_all` wraps fixture creation in a SAVEPOINT, shared across all tests in the class. After all tests run, the savepoint rolls back. This only works when tests are read-only on the shared data. The `sources_index_metrics_test.rb` is the highest-value candidate with 17 read-only tests sharing the same 3 sources + 3 items.
43
+
44
+ ## Tasks
45
+
46
+ ### Task 1: Convert sources_index_metrics_test.rb to setup_once (PRIMARY)
47
+
48
+ This is the highest-impact conversion. Convert `test/lib/source_monitor/analytics/sources_index_metrics_test.rb`:
49
+
50
+ Replace:
51
+ ```ruby
52
+ setup do
53
+ clean_source_monitor_tables!
54
+ travel_to Time.current.change(usec: 0)
55
+ @fast_source = create_source!(name: "Fast", fetch_interval_minutes: 30)
56
+ # ... fixture creation
57
+ end
58
+ ```
59
+
60
+ With:
61
+ ```ruby
62
+ setup_once do
63
+ clean_source_monitor_tables!
64
+ @fast_source = create_source!(name: "Fast", fetch_interval_minutes: 30)
65
+ # ... same fixture creation, but now runs once for all 17 tests
66
+ end
67
+ ```
68
+
69
+ **Important:** The `travel_to` call must stay in a regular `setup` block because `travel_to` affects the thread-local time for each test independently:
70
+ ```ruby
71
+ setup_once do
72
+ clean_source_monitor_tables!
73
+ # fixture creation here
74
+ end
75
+
76
+ setup do
77
+ travel_to Time.current.change(usec: 0)
78
+ end
79
+
80
+ teardown do
81
+ travel_back
82
+ end
83
+ ```
84
+
85
+ Wait -- `travel_to` inside `setup_once` would freeze time for the SAVEPOINT transaction but tests need consistent time for assertions. Actually, the fixtures are created with relative timestamps (`1.day.ago`, `2.days.ago`) which depend on `Time.current`. If `travel_to` is in `setup_once`, the timestamps are fixed at creation time, which is fine since tests read them as-is. But `travel_back` in teardown would only run once after all tests, and the `travel_to` in `setup_once` persists through all tests.
86
+
87
+ Safest approach: Move `travel_to` into `setup_once` and remove the teardown's `travel_back` (before_all handles cleanup). Add a regular `setup` with `travel_to` at the same frozen time to ensure each test sees consistent time.
88
+
89
+ Actually, the simplest safe approach: keep `travel_to` and `travel_back` in regular `setup`/`teardown`, and only put the DB operations in `setup_once`. The fixtures use relative timestamps (`1.day.ago`) which will be slightly different each test, but since the tests only compare relative values (bucket labels, activity rates), this is fine.
90
+
91
+ ### Task 2: Convert single-test analytics files to setup_once
92
+
93
+ Convert these 3 files for consistency (minimal performance benefit but establishes the pattern):
94
+
95
+ 1. **`test/lib/source_monitor/analytics/source_activity_rates_test.rb`** -- Replace `setup { clean_source_monitor_tables! }` with `setup_once { clean_source_monitor_tables! }`
96
+ 2. **`test/lib/source_monitor/analytics/source_fetch_interval_distribution_test.rb`** -- Same pattern
97
+ 3. **`test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb`** -- Same pattern
98
+
99
+ For single-test classes, `setup` and `setup_once` are functionally identical, so this is a no-op in terms of performance but normalizes the codebase to use the `setup_once` pattern for table cleaning.
100
+
101
+ ### Task 3: Verify all converted files individually
102
+
103
+ Run each converted file with PARALLEL_WORKERS=1 to confirm no regressions:
104
+ ```bash
105
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/analytics/sources_index_metrics_test.rb
106
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/analytics/source_activity_rates_test.rb
107
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/analytics/source_fetch_interval_distribution_test.rb
108
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb
109
+ ```
110
+
111
+ If any file fails due to test isolation issues, revert it to per-test setup and document why.
112
+
113
+ ### Task 4: Full suite verification and lint
114
+
115
+ ```bash
116
+ # Full suite (all 1031+ tests pass)
117
+ bin/rails test
118
+
119
+ # Lint all modified files
120
+ bin/rubocop test/lib/source_monitor/analytics/ test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb
121
+ ```
122
+
123
+ Ensure total test count remains 1031+ and no failures occur.
124
+
125
+ ## Files
126
+
127
+ | Action | Path |
128
+ |--------|------|
129
+ | MODIFY | `test/lib/source_monitor/analytics/sources_index_metrics_test.rb` |
130
+ | MODIFY | `test/lib/source_monitor/analytics/source_activity_rates_test.rb` |
131
+ | MODIFY | `test/lib/source_monitor/analytics/source_fetch_interval_distribution_test.rb` |
132
+ | MODIFY | `test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb` |
133
+
134
+ ## Verification
135
+
136
+ ```bash
137
+ # Individual file runs
138
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/analytics/sources_index_metrics_test.rb
139
+ PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb
140
+
141
+ # Full suite (all 1031+ tests pass)
142
+ bin/rails test
143
+
144
+ # Lint
145
+ bin/rubocop test/lib/source_monitor/analytics/ test/lib/source_monitor/dashboard/upcoming_fetch_schedule_test.rb
146
+ ```
147
+
148
+ ## Success Criteria
149
+
150
+ - `grep -r "setup_once" test/lib/source_monitor/` shows 5+ files (up from 1)
151
+ - `sources_index_metrics_test.rb` uses `setup_once` for fixture creation
152
+ - All 1031+ tests pass in full suite
153
+ - No test isolation regressions in parallel runs
154
+ - Each converted file passes individually with PARALLEL_WORKERS=1
@@ -0,0 +1,28 @@
1
+ ---
2
+ phase: 2
3
+ plan: 4
4
+ status: complete
5
+ ---
6
+ # Plan 04 Summary: Switch Default Parallelism to Threads
7
+
8
+ ## Tasks Completed
9
+ - [x] Task 1: Switch parallelize to always use `with: :threads` (not just coverage mode)
10
+ - [x] Task 2: Add thread-safety comment to reset_configuration! setup block
11
+ - [x] Task 3: Verify single-file runs work without PARALLEL_WORKERS=1 (3 files tested, all pass)
12
+ - [x] Task 4: Full suite verification (1033 tests, 0 failures, 2 consecutive runs, 0 flaky)
13
+
14
+ ## Commits
15
+ - eceb06d: perf(test): switch default parallelism from forks to threads
16
+
17
+ ## Files Modified
18
+ - test/test_helper.rb (modified)
19
+
20
+ ## What Was Built
21
+ - Unified parallelism to always use `with: :threads` instead of fork-based (forks only used in coverage mode previously)
22
+ - Worker count logic preserved: COVERAGE=1 forces 1 worker, otherwise respects SOURCE_MONITOR_TEST_WORKERS env var or defaults to :number_of_processors
23
+ - PG fork segfault on single-file runs eliminated — verified with feed_fetcher_success_test.rb, source_test.rb, and sources_controller_test.rb all passing without PARALLEL_WORKERS=1
24
+ - Added thread-safety comment explaining why reset_configuration! is safe under thread parallelism
25
+ - Note: TestProf emits `before_all is not implemented for parallalization with threads` warning — cosmetic only, before_all works correctly since single-file runs stay below parallelization threshold and full suite distributes by class
26
+
27
+ ## Deviations
28
+ - None
@@ -0,0 +1,133 @@
1
+ ---
2
+ phase: "02"
3
+ plan: "04"
4
+ title: "Switch Default Parallelism to Threads"
5
+ wave: 2
6
+ depends_on: ["PLAN-01"]
7
+ must_haves:
8
+ - "REQ-PERF-04: Default parallelism switched from forks to threads"
9
+ - "test_helper.rb parallelize call uses 'with: :threads' for all modes"
10
+ - "Thread safety verified for reset_configuration! (no data races)"
11
+ - "All 1031+ tests pass with thread-based parallelism"
12
+ - "PG fork segfault on single-file runs eliminated"
13
+ - "PARALLEL_WORKERS env var still respected"
14
+ - "RuboCop zero offenses on modified files"
15
+ skills_used: []
16
+ ---
17
+
18
+ # Plan 04: Switch Default Parallelism to Threads
19
+
20
+ ## Objective
21
+
22
+ Switch the default test parallelism from fork-based to thread-based. This eliminates the PG fork segfault that forces `PARALLEL_WORKERS=1` on single-file runs, and enables the FeedFetcherTest split (Plan 01) to actually parallelize across workers. Thread-based parallelism is already proven working in coverage mode (`COVERAGE=1`).
23
+
24
+ ## Context
25
+
26
+ - `@` `test/test_helper.rb` -- current parallelism configuration (forks by default, threads only for coverage)
27
+ - `@` `.vbw-planning/phases/02-test-performance/02-RESEARCH.md` -- research confirming thread parallelism works in coverage mode
28
+ - `@` `test/test_prof.rb` -- TestProf setup (thread-compatible)
29
+
30
+ **Rationale:** The current code uses `parallelize(workers: worker_count)` which defaults to fork-based parallelism. This causes PG segfaults on single-file runs and prevents the FeedFetcherTest split from distributing across workers (since forks copy the process and the PG connection). Thread-based parallelism is already proven (used with COVERAGE=1) and avoids these issues.
31
+
32
+ **Dependency on Plan 01:** Plan 01 splits FeedFetcherTest into 6+ classes. Without the split, thread parallelism still cannot distribute the 71-test monolith across workers. The split must complete first for the parallelism switch to realize its full benefit.
33
+
34
+ **Risk: Thread safety of `reset_configuration!`** -- The global `setup` block calls `SourceMonitor.reset_configuration!` before every test. With threads, multiple tests may call this simultaneously. Since `reset_configuration!` replaces the entire `@configuration` instance, and each test reads config after setup, this is safe as long as no test modifies config mid-test while another test is reading it. The research confirmed this is pure Ruby assignment (microseconds). If any flaky failures appear, we add a `Mutex` around the reset.
35
+
36
+ ## Tasks
37
+
38
+ ### Task 1: Switch parallelize to threads
39
+
40
+ In `test/test_helper.rb`, replace the parallelism block:
41
+
42
+ ```ruby
43
+ # BEFORE:
44
+ if ENV["COVERAGE"]
45
+ parallelize(workers: 1, with: :threads)
46
+ else
47
+ worker_count = ENV.fetch("SOURCE_MONITOR_TEST_WORKERS", :number_of_processors)
48
+ worker_count = worker_count.to_i if worker_count.is_a?(String) && !worker_count.empty?
49
+ worker_count = :number_of_processors if worker_count.respond_to?(:zero?) && worker_count.zero?
50
+ parallelize(workers: worker_count)
51
+ end
52
+ ```
53
+
54
+ ```ruby
55
+ # AFTER:
56
+ worker_count = if ENV["COVERAGE"]
57
+ 1
58
+ else
59
+ count = ENV.fetch("SOURCE_MONITOR_TEST_WORKERS", :number_of_processors)
60
+ count = count.to_i if count.is_a?(String) && !count.empty?
61
+ count = :number_of_processors if count.respond_to?(:zero?) && count.zero?
62
+ count
63
+ end
64
+ parallelize(workers: worker_count, with: :threads)
65
+ ```
66
+
67
+ Key change: Always use `with: :threads` (not just for coverage). Worker count logic stays the same.
68
+
69
+ ### Task 2: Add thread-safety comment to reset_configuration
70
+
71
+ Add a comment in the `setup` block explaining thread safety:
72
+
73
+ ```ruby
74
+ setup do
75
+ # Thread-safe: reset_configuration! replaces @configuration atomically.
76
+ # Each test gets a fresh config object. No concurrent mutation risk since
77
+ # tests read config only after their own setup completes.
78
+ SourceMonitor.reset_configuration!
79
+ end
80
+ ```
81
+
82
+ ### Task 3: Verify single-file runs work without PARALLEL_WORKERS=1
83
+
84
+ The main benefit of thread-based parallelism: single-file runs no longer segfault.
85
+
86
+ ```bash
87
+ # These should now work WITHOUT PARALLEL_WORKERS=1
88
+ bin/rails test test/lib/source_monitor/fetching/feed_fetcher_success_test.rb
89
+ bin/rails test test/models/source_monitor/source_test.rb
90
+ bin/rails test test/controllers/source_monitor/sources_controller_test.rb
91
+ ```
92
+
93
+ ### Task 4: Full suite verification
94
+
95
+ ```bash
96
+ # Full suite with thread parallelism
97
+ bin/rails test
98
+
99
+ # Verify worker count is respected
100
+ SOURCE_MONITOR_TEST_WORKERS=4 bin/rails test
101
+
102
+ # Lint
103
+ bin/rubocop test/test_helper.rb
104
+ ```
105
+
106
+ Ensure all 1031+ tests pass with zero failures. Watch for flaky tests that might indicate thread-safety issues. If any test fails intermittently, check if it modifies global state (module-level variables, class variables, or singletons) and fix the isolation.
107
+
108
+ ## Files
109
+
110
+ | Action | Path |
111
+ |--------|------|
112
+ | MODIFY | `test/test_helper.rb` |
113
+
114
+ ## Verification
115
+
116
+ ```bash
117
+ # Single-file run (no PARALLEL_WORKERS=1 needed)
118
+ bin/rails test test/models/source_monitor/source_test.rb
119
+
120
+ # Full suite
121
+ bin/rails test
122
+
123
+ # Lint
124
+ bin/rubocop test/test_helper.rb
125
+ ```
126
+
127
+ ## Success Criteria
128
+
129
+ - `grep "with: :threads" test/test_helper.rb` shows the threads configuration
130
+ - `bin/rails test` passes all 1031+ tests
131
+ - Single-file runs work without PARALLEL_WORKERS=1 workaround
132
+ - No flaky test failures in 2 consecutive full suite runs
133
+ - Full suite completes in <70s locally (down from 133s)