source_monitor 0.5.3 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 21aedfc8713e8179626900475c725eeb1fca98e169b5477d1ee45e557cdebf7e
-  data.tar.gz: 30e532cc201cf97c31209010125de15643dc9e53529e6aedbbbf07e67fee7d9d
+  metadata.gz: b65480547bf48a4cabf2d1c98dbd6c965a6b7342c3da362b3987b1bed3e59a5d
+  data.tar.gz: 775abb18c5c94b5cf11e78e01c296a618c7ef884cb328f4f5f886c2d144c2f75
 SHA512:
-  metadata.gz: cc9da6548d8c8873e0aa989b21cd9fb85b76c14ab9a1335952e873a0be3694674c8aca74cbdb03cdb05e3c00db3638d519147f9410ee13439a6b71b31fecf29d
-  data.tar.gz: 34e16d32cfef7444234f812c4c27359a7f73d964859630c79bccccdeda43eb65a50669639e17ffecf9de492f82dbf654706986ac489d65635fcc9ad964615133
+  metadata.gz: f75e313708962d167d7b362ed4f8af42be433e28d5a9e1aa59f290d82ed12103800abaf2f904098211c351f7c3b9265af05363d2364f477e22af3e7de2bc9755
+  data.tar.gz: ab0e7911a85c744f632d2fad3dbeb671ddb55b3e1f4cc0ed3a0dbc859e2dad21ccacdec8aff7ee45445cf84f3cfb3ddaf2ba803a7e533cacdc14b8cb3ee61ab6

data/.gitignore

@@ -22,3 +22,10 @@
 .vbw-planning/.claude-md-migrated
 .vbw-planning/.watchdog-pid
 .vbw-planning/.watchdog.log
+.vbw-planning/.agent-pids
+.vbw-planning/.agent-panes
+.vbw-planning/.active-agent
+.vbw-planning/.active-agent-count
+.vbw-planning/.todo-flat-migrated
+/codebase_analysis.md
+*.gem

data/.vbw-planning/ROADMAP.md

@@ -0,0 +1,32 @@
+# Roadmap
+
+## Milestone: aia-ssl-fix
+
+### Phases
+
+1. [x] **AIA Certificate Resolution** -- Fix SSL failures for feeds with missing intermediate certificates by implementing AIA (Authority Information Access) resolution
+
+### Phase Details
+
+#### Phase 1: AIA Certificate Resolution
+
+**Goal:** Implement automatic AIA intermediate certificate fetching so feeds like netflixtechblog.com (served via Medium/AWS with wrong intermediates) succeed without manual cert configuration.
+
+**Requirements:**
+- REQ-AIA-01: Create AIAResolver module with thread-safe cache and 1-hour TTL
+- REQ-AIA-02: Add cert_store: parameter to HTTP.client for custom cert stores
+- REQ-AIA-03: On Faraday::SSLError, attempt AIA resolution before failing
+- REQ-AIA-04: Best-effort only -- never make things worse (rescue StandardError -> nil)
+
+**Success Criteria:**
+- [ ] AIAResolver.resolve(hostname) fetches leaf cert, extracts AIA URL, downloads intermediate
+- [ ] HTTP.client(cert_store:) accepts and uses custom cert stores
+- [ ] FeedFetcher retries once with AIA-resolved cert store on SSL failure
+- [ ] All existing tests pass (1003+), new tests cover AIA paths
+- [ ] RuboCop zero offenses, Brakeman zero warnings
+
+### Progress
+
+| Phase | Status | Plans | Completed |
+|-------|--------|-------|-----------|
+| 1. AIA Certificate Resolution | Planned | 3 | 0 |

data/.vbw-planning/STATE.md

@@ -0,0 +1,27 @@
+# State
+
+## Current Position
+
+- **Milestone:** aia-ssl-fix
+- **Phase:** 1 -- AIA Certificate Resolution
+- **Status:** Complete
+- **Progress:** 100%
+
+## Decisions
+
+| Decision | Date | Context |
+|----------|------|---------|
+| Single-phase milestone for AIA fix | 2026-02-17 | Complete plan already validated; no scoping needed |
+| 3 plans with wave parallelism | 2026-02-17 | Plans 01+02 (wave 1, disjoint files), Plan 03 (wave 2, integration) |
+
+## Todos
+
+## Metrics
+
+- **Started:** 2026-02-17
+- **Phases:** 1
+- **Plans:** 3
+- **Tests at start:** 1003
+- **Tests at end:** 1025
+- **Commits:** 4 (f60e9bf, 4c9568a, 9c38bc3, e68a6b0)
+- **Plans completed:** 3/3

data/.vbw-planning/milestones/default/STATE.md

@@ -50,7 +50,6 @@ Progress: [##########] 100%
 - [phase-4]: Fix-everything approach for public API convention violations
 - [phase-4]: 3 files slightly exceed 300 lines (entry_parser 390, queries 356, application_helper 346) -- all single-responsibility, cannot be split further
 
-### Pending Todos
 
 None
 

data/.vbw-planning/phases/01-aia-certificate-resolution/.context-dev.md

@@ -0,0 +1,17 @@
+## Phase 1 Context
+
+### Goal
+Not available
+
+### Codebase Map Available
+Codebase mapping exists in `.vbw-planning/codebase/`. Key files:
+- `ARCHITECTURE.md`
+- `CONCERNS.md`
+- `PATTERNS.md`
+- `DEPENDENCIES.md`
+- `STRUCTURE.md`
+- `CONVENTIONS.md`
+- `TESTING.md`
+- `STACK.md`
+
+Read CONVENTIONS.md, PATTERNS.md, STRUCTURE.md, and DEPENDENCIES.md first to bootstrap codebase understanding.

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-01-SUMMARY.md

@@ -0,0 +1,26 @@
+---
+phase: 1
+plan: 1
+status: complete
+---
+# Plan 01 Summary: AIA Resolver Module
+
+## Tasks Completed
+- [x] Task 1: Created lib/source_monitor/http/aia_resolver.rb
+- [x] Task 2: Created test/lib/source_monitor/http/aia_resolver_test.rb
+
+## Commits
+- 4c9568a: feat(1-1): add AIA intermediate certificate resolver
+
+## Files Modified
+- lib/source_monitor/http/aia_resolver.rb (created)
+- test/lib/source_monitor/http/aia_resolver_test.rb (created)
+
+## What Was Built
+- `SourceMonitor::HTTP::AIAResolver` module with thread-safe cached resolution of missing intermediate SSL certificates via AIA (Authority Information Access) X.509 extension
+- Public API: `resolve(hostname)`, `enhanced_cert_store(certs)`, `clear_cache!`, `cache_size`
+- Private methods: `fetch_leaf_certificate` (VERIFY_NONE + SNI), `extract_aia_url` (uses `cert.ca_issuer_uris`), `download_certificate` (DER-first, PEM fallback)
+- 11 unit tests covering all public/private methods, caching, TTL expiration, and error handling
+
+## Deviations
+- None

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-01.md

@@ -0,0 +1,71 @@
+---
+phase: 1
+plan: 1
+title: "AIA Resolver Module"
+wave: 1
+depends_on: []
+must_haves:
+  - AIAResolver module with resolve, enhanced_cert_store, clear_cache!, cache_size
+  - Thread-safe Mutex + Hash cache with 1-hour TTL per hostname
+  - fetch_leaf_certificate with VERIFY_NONE and SNI support
+  - extract_aia_url using cert.ca_issuer_uris (not regex)
+  - download_certificate with DER-first, PEM-fallback parsing
+  - All methods rescue StandardError and return nil
+  - Unit tests covering all public and private methods
+---
+
+# Plan 01: AIA Resolver Module
+
+## Goal
+
+Create `SourceMonitor::HTTP::AIAResolver` -- a standalone module that resolves missing intermediate certificates via the AIA (Authority Information Access) extension in X.509 certificates.
+
+## Tasks
+
+### Task 1: Create lib/source_monitor/http/aia_resolver.rb
+
+Create new module `SourceMonitor::HTTP::AIAResolver` with class methods:
+
+**Public API:**
+- `resolve(hostname, port: 443)` -- Entry point. Checks cache first, then: fetch leaf cert -> extract AIA URL -> download intermediate. Returns `OpenSSL::X509::Certificate` or `nil`.
+- `enhanced_cert_store(additional_certs)` -- Builds `OpenSSL::X509::Store` with `set_default_paths` plus extra certs from the array.
+- `clear_cache!` -- Clears the hostname cache (for testing).
+- `cache_size` -- Returns number of cached entries (for testing).
+
+**Private methods:**
+- `fetch_leaf_certificate(hostname, port)` -- TCP+SSL connect with `VERIFY_NONE` to get the server's leaf cert. 5s connect timeout. Uses `ssl_socket.hostname=` for SNI.
+- `extract_aia_url(cert)` -- Uses Ruby's built-in `cert.ca_issuer_uris` method. Returns first URI string or nil.
+- `download_certificate(url)` -- Plain HTTP GET (AIA URLs are always HTTP, not HTTPS). 5s timeout. Parses DER body as `OpenSSL::X509::Certificate`, falls back to PEM on failure.
+
+**Cache:** `Mutex` + `Hash` keyed by hostname. Each entry stores `{ cert:, expires_at: }` with 1-hour TTL.
+
+**Safety:** All methods rescue `StandardError` and return `nil`. This is best-effort -- never makes things worse.
+
+### Task 2: Create test/lib/source_monitor/http/aia_resolver_test.rb
+
+Unit tests:
+- `extract_aia_url` with cert that has AIA extension returns URL
+- `extract_aia_url` with cert without AIA returns nil
+- `download_certificate` with DER body parses correctly (WebMock stub)
+- `download_certificate` returns nil on HTTP 404 (WebMock)
+- `download_certificate` returns nil on timeout (WebMock)
+- `enhanced_cert_store` returns store with added certs
+- `enhanced_cert_store` handles empty array gracefully
+- Cache: resolve stores result, second call returns cached
+- Cache: expired entries are re-fetched
+- `clear_cache!` empties the cache
+- `resolve` returns nil when hostname unreachable (stub fetch_leaf_certificate)
+
+## Files
+
+| Action | Path |
+|--------|------|
+| CREATE | `lib/source_monitor/http/aia_resolver.rb` |
+| CREATE | `test/lib/source_monitor/http/aia_resolver_test.rb` |
+
+## Verification
+
+```bash
+PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/http/aia_resolver_test.rb
+bin/rubocop lib/source_monitor/http/aia_resolver.rb test/lib/source_monitor/http/aia_resolver_test.rb
+```

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-02-SUMMARY.md

@@ -0,0 +1,16 @@
+---
+phase: 1
+plan: 2
+status: complete
+commit: f60e9bf
+---
+
+## What Was Built
+- Added `cert_store:` keyword parameter to `HTTP.client` for custom OpenSSL cert stores
+- Added `autoload :AIAResolver` to HTTP module
+- Plumbed cert_store through `configure_request` -> `configure_ssl` with fallback to `default_cert_store`
+- 2 new tests: custom cert_store usage, ssl_ca_file takes precedence over cert_store
+
+## Files Modified
+- `lib/source_monitor/http.rb` — autoload, cert_store param, SSL plumbing
+- `test/lib/source_monitor/http_test.rb` — 2 new cert_store tests

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-02.md

@@ -0,0 +1,56 @@
+---
+phase: 1
+plan: 2
+title: "HTTP Module cert_store Parameter"
+wave: 1
+depends_on: []
+must_haves:
+  - Add autoload :AIAResolver to module HTTP
+  - Add cert_store keyword to client method
+  - Pass cert_store through configure_request to configure_ssl
+  - configure_ssl uses cert_store when no ssl_ca_file/ssl_ca_path
+  - Tests for cert_store parameter usage
+---
+
+# Plan 02: HTTP Module cert_store Parameter
+
+## Goal
+
+Extend `SourceMonitor::HTTP.client` to accept an optional `cert_store:` parameter, enabling callers (like FeedFetcher's AIA retry) to provide a custom `OpenSSL::X509::Store` with additional certificates.
+
+## Tasks
+
+### Task 1: Modify lib/source_monitor/http.rb
+
+1. Add autoload inside `module HTTP` (after RETRY_STATUSES):
+   ```ruby
+   autoload :AIAResolver, "source_monitor/http/aia_resolver"
+   ```
+
+2. Add `cert_store: nil` keyword to `client` method signature.
+
+3. Pass `cert_store:` through `configure_request` to `configure_ssl`:
+   - Add `cert_store:` parameter to `configure_request`
+   - Pass it to `configure_ssl(connection, settings, cert_store:)`
+
+4. In `configure_ssl`: when no `ssl_ca_file` or `ssl_ca_path` is set, use `cert_store || default_cert_store`.
+
+### Task 2: Add tests to test/lib/source_monitor/http_test.rb
+
+Add 2 tests:
+- `cert_store: param is used when no ssl_ca_file or ssl_ca_path` -- pass a custom store, verify `connection.ssl.cert_store` is the custom store
+- `cert_store: is ignored when ssl_ca_file is set` -- configure ssl_ca_file, pass cert_store, verify ca_file takes precedence
+
+## Files
+
+| Action | Path |
+|--------|------|
+| MODIFY | `lib/source_monitor/http.rb` |
+| MODIFY | `test/lib/source_monitor/http_test.rb` |
+
+## Verification
+
+```bash
+PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/http_test.rb
+bin/rubocop lib/source_monitor/http.rb test/lib/source_monitor/http_test.rb
+```

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-03-SUMMARY.md

@@ -0,0 +1,17 @@
+---
+phase: 1
+plan: 3
+status: complete
+commit: 9c38bc3
+---
+
+## What Was Built
+- Wired AIA certificate resolution into FeedFetcher's SSL error handling
+- On `Faraday::SSLError`, attempts intermediate cert recovery via `AIAResolver.resolve` before raising
+- Guard flag `@aia_attempted` prevents infinite recursion; `rescue StandardError => nil` ensures recovery never makes things worse
+- Tags `instrumentation_payload[:aia_resolved] = true` on successful AIA recovery
+- 3 integration tests: success retry path, nil fallback to ConnectionError, non-SSL skip
+
+## Files Modified
+- `lib/source_monitor/fetching/feed_fetcher.rb` — split SSL rescue, add `attempt_aia_recovery`
+- `test/lib/source_monitor/fetching/feed_fetcher_test.rb` — 3 AIA resolution tests

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-03.md

@@ -0,0 +1,98 @@
+---
+phase: 1
+plan: 3
+title: "FeedFetcher AIA Retry Integration"
+wave: 2
+depends_on: [1, 2]
+must_haves:
+  - Separate Faraday::SSLError rescue from Faraday::ConnectionFailed
+  - On SSLError attempt AIA resolution once (aia_attempted flag)
+  - Parse hostname from source.feed_url for AIA resolve
+  - If intermediate found rebuild connection with enhanced cert store and retry
+  - If nil raise ConnectionError as before
+  - Tag successful recoveries with aia_resolved in instrumentation
+  - Integration tests for all AIA retry paths
+  - Full test suite passes (1003+ tests)
+  - RuboCop zero offenses
+  - Brakeman zero warnings
+---
+
+# Plan 03: FeedFetcher AIA Retry Integration
+
+## Goal
+
+Wire AIA resolution into FeedFetcher's error handling so SSL failures automatically attempt intermediate certificate recovery before giving up.
+
+## Tasks
+
+### Task 1: Modify lib/source_monitor/fetching/feed_fetcher.rb
+
+Modify `perform_fetch` (lines 77-90):
+
+1. **Split rescue clause:** Separate `Faraday::SSLError` from `Faraday::ConnectionFailed` into its own rescue:
+   ```ruby
+   rescue Faraday::ConnectionFailed => error
+     raise ConnectionError.new(error.message, original_error: error)
+   rescue Faraday::SSLError => error
+     attempt_aia_recovery(error) || raise(ConnectionError.new(error.message, original_error: error))
+   ```
+
+2. **Add `attempt_aia_recovery` private method:**
+   - Guard: return nil if `@aia_attempted` is true (prevents recursion)
+   - Set `@aia_attempted = true`
+   - Parse hostname from `URI.parse(source.feed_url).host`
+   - Call `SourceMonitor::HTTP::AIAResolver.resolve(hostname)`
+   - If intermediate found:
+     - Build enhanced cert store via `AIAResolver.enhanced_cert_store([intermediate])`
+     - Rebuild `@connection = SourceMonitor::HTTP.client(cert_store: store, headers: request_headers)`
+     - Return `perform_request` (the retry)
+   - If nil: return nil (caller raises ConnectionError)
+   - Rescue StandardError -> nil (never make retry worse)
+
+3. **Tag instrumentation:** In the `handle_response` path after successful AIA retry, the `instrumentation_payload[:aia_resolved] = true` will naturally flow through since `perform_fetch` calls `handle_response` on the retried response.
+
+### Task 2: Add tests to test/lib/source_monitor/fetching/feed_fetcher_test.rb
+
+Add 3 tests under a new section `# -- AIA Certificate Resolution --`:
+
+1. **SSL error + AIA resolve succeeds -> fetch succeeds:**
+   - First stub: raise `Faraday::SSLError`
+   - Stub `AIAResolver.resolve` to return a mock certificate
+   - Stub `AIAResolver.enhanced_cert_store` to return a store
+   - Second stub (after retry): return 200 with RSS body
+   - Assert result.status == :fetched
+
+2. **SSL error + AIA resolve returns nil -> ConnectionError:**
+   - Stub to raise `Faraday::SSLError`
+   - Stub `AIAResolver.resolve` to return nil
+   - Assert result.status == :failed
+   - Assert result.error is ConnectionError
+
+3. **Non-SSL ConnectionError -> AIA not attempted:**
+   - Stub to raise `Faraday::ConnectionFailed`
+   - Verify `AIAResolver.resolve` was NOT called
+   - Assert result.status == :failed
+   - Assert result.error is ConnectionError
+
+### Task 3: Run full verification
+
+1. `PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_test.rb`
+2. `bin/rails test` (full suite)
+3. `bin/rubocop`
+4. `bin/brakeman --no-pager`
+
+## Files
+
+| Action | Path |
+|--------|------|
+| MODIFY | `lib/source_monitor/fetching/feed_fetcher.rb` |
+| MODIFY | `test/lib/source_monitor/fetching/feed_fetcher_test.rb` |
+
+## Verification
+
+```bash
+PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_test.rb
+bin/rails test
+bin/rubocop
+bin/brakeman --no-pager
+```

data/CHANGELOG.md

@@ -15,6 +15,35 @@ All notable changes to this project are documented below. The format follows [Ke
 
 - No unreleased changes yet.
 
+## [0.7.0] - 2026-02-18
+
+### Fixed
+
+- **False "updated" counts on unchanged feed items.** ItemCreator now checks for significant attribute changes before saving. Items with no real changes return a new `:unchanged` status instead of `:updated`, eliminating unnecessary database writes and misleading dashboard statistics.
+- **Redundant entry processing on unchanged feeds.** When a feed's body SHA-256 signature matches the previous fetch, entry processing is now skipped entirely (like the existing 304 Not Modified path), avoiding unnecessary parsing, DB lookups, and saves.
+- **Adaptive interval not backing off for stable feeds.** The `content_changed` signal for adaptive fetch scheduling now uses an item-level content hash (sorted entry IDs) instead of the raw XML body hash. This prevents cosmetic feed changes (e.g., `<lastBuildDate>` updates) from defeating interval backoff, allowing stable feeds to correctly increase their fetch interval.
+
+### Testing
+
+- 1,031 tests, 3,300 assertions, 0 failures.
+- RuboCop: 0 offenses.
+- Brakeman: 0 warnings.
+
+## [0.6.0] - 2026-02-17
+
+### Added
+
+- AIA (Authority Information Access) certificate resolution for SSL failures. When feed fetching or scraping encounters `certificate verify failed` errors due to missing intermediate certificates, the engine now automatically fetches the missing intermediate via AIA URLs and retries the request. This fixes feeds hosted on servers with incomplete certificate chains (e.g., Medium/Netflix Tech Blog on AWS).
+- `SourceMonitor::HTTP::AIAResolver` module with thread-safe hostname-keyed cache (1-hour TTL), SNI support, and DER/PEM certificate parsing.
+- `cert_store:` parameter on `SourceMonitor::HTTP.client` for passing custom certificate stores.
+- Brakeman ignore configuration (`config/brakeman.ignore`) for the intentional `VERIFY_NONE` in the AIA resolver's leaf certificate fetch.
+
+### Testing
+
+- 1,028 tests, 0 failures (up from 1,003 in 0.5.x).
+- RuboCop: 0 offenses.
+- Brakeman: 0 warnings (1 intentional ignore).
+
 ## [0.5.3] - 2026-02-16
 
 ### Fixed

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    source_monitor (0.5.3)
+    source_monitor (0.7.0)
       cssbundling-rails (~> 1.4)
       faraday (~> 2.9)
       faraday-follow_redirects (~> 0.4)

data/VERSION

@@ -1 +1 @@
-0.5.3
+0.7.0

data/config/brakeman.ignore

@@ -0,0 +1,17 @@
+{
+  "ignored_warnings": [
+    {
+      "warning_type": "SSL Verification Bypass",
+      "warning_code": 71,
+      "fingerprint": "17da2beb8f8ecf05b0ca1e3da89ee27d593395c711197f2f7dd38df759ea3720",
+      "check_name": "SSLVerify",
+      "message": "SSL certificate verification was bypassed",
+      "file": "lib/source_monitor/http/aia_resolver.rb",
+      "line": 77,
+      "code": "OpenSSL::SSL::SSLContext.new.verify_mode = OpenSSL::SSL::VERIFY_NONE",
+      "note": "Intentional: AIA resolver must connect without verification to fetch the leaf certificate from servers with broken certificate chains. This is the core purpose of the module -- it only uses VERIFY_NONE to read the cert, never to transmit data."
+    }
+  ],
+  "updated": "2026-02-17",
+  "brakeman_version": "8.0.2"
+}

data/lib/source_monitor/fetching/feed_fetcher/entry_processor.rb

@@ -14,6 +14,7 @@ module SourceMonitor
           return FeedFetcher::EntryProcessingResult.new(
             created: 0,
             updated: 0,
+            unchanged: 0,
             failed: 0,
             items: [],
             errors: [],
@@ -23,6 +24,7 @@ module SourceMonitor
 
           created = 0
           updated = 0
+          unchanged = 0
           failed = 0
           items = []
           created_items = []
@@ -39,6 +41,8 @@ module SourceMonitor
                 created_items << result.item
                 SourceMonitor::Events.after_item_created(item: result.item, source:, entry:, result: result)
                 enqueue_image_download(result.item)
+              elsif result.unchanged?
+                unchanged += 1
               else
                 updated += 1
                 updated_items << result.item
@@ -52,6 +56,7 @@ module SourceMonitor
           FeedFetcher::EntryProcessingResult.new(
             created:,
             updated:,
+            unchanged:,
             failed:,
             items:,
             errors: errors.compact,

data/lib/source_monitor/fetching/feed_fetcher/source_updater.rb

@@ -11,7 +11,7 @@ module SourceMonitor
           @adaptive_interval = adaptive_interval
         end
 
-        def update_source_for_success(response, duration_ms, feed, feed_signature)
+        def update_source_for_success(response, duration_ms, feed, feed_signature, content_changed: nil, entries_digest: nil)
           attributes = {
             last_fetched_at: Time.current,
             last_fetch_duration_ms: duration_ms,
@@ -31,8 +31,10 @@ module SourceMonitor
             attributes[:last_modified] = parsed_time if parsed_time
           end
 
-          adaptive_interval.apply_adaptive_interval!(attributes, content_changed: feed_signature_changed?(feed_signature))
-          attributes[:metadata] = updated_metadata(feed_signature: feed_signature)
+          # Use explicit content_changed if provided, otherwise fall back to feed signature comparison
+          changed = content_changed.nil? ? feed_signature_changed?(feed_signature) : content_changed
+          adaptive_interval.apply_adaptive_interval!(attributes, content_changed: changed)
+          attributes[:metadata] = updated_metadata(feed_signature: feed_signature, entries_digest: entries_digest)
           reset_retry_state!(attributes)
           source.update!(attributes)
         end
@@ -111,10 +113,11 @@ module SourceMonitor
           (source.metadata || {}).fetch("last_feed_signature", nil) != feed_signature
         end
 
-        def updated_metadata(feed_signature: nil)
+        def updated_metadata(feed_signature: nil, entries_digest: nil)
           metadata = (source.metadata || {}).dup
           metadata.delete("dynamic_fetch_interval_seconds")
           metadata["last_feed_signature"] = feed_signature if feed_signature.present?
+          metadata["last_entries_digest"] = entries_digest if entries_digest.present?
           metadata
         end
 

data/lib/source_monitor/fetching/feed_fetcher.rb

@@ -17,6 +17,7 @@ module SourceMonitor
       EntryProcessingResult = Struct.new(
         :created,
         :updated,
+        :unchanged,
         :failed,
         :items,
         :errors,
@@ -81,8 +82,11 @@ module SourceMonitor
         raise error
       rescue Faraday::TimeoutError => error
         raise TimeoutError.new(error.message, original_error: error)
-      rescue Faraday::ConnectionFailed, Faraday::SSLError => error
+      rescue Faraday::ConnectionFailed => error
         raise ConnectionError.new(error.message, original_error: error)
+      rescue Faraday::SSLError => error
+        attempt_aia_recovery(error, started_at, instrumentation_payload) ||
+          raise(ConnectionError.new(error.message, original_error: error))
       rescue Faraday::ClientError => error
         raise build_http_error_from_faraday(error)
       rescue Faraday::Error => error
@@ -120,11 +124,28 @@ module SourceMonitor
       def handle_success(response, started_at, instrumentation_payload)
         duration_ms = source_updater.elapsed_ms(started_at)
         body = response.body
+        feed_body_signature = body_digest(body)
         feed = parse_feed(body, response)
-        processing = entry_processor.process_feed_entries(feed)
 
-        feed_body_signature = body_digest(body)
-        source_updater.update_source_for_success(response, duration_ms, feed, feed_body_signature)
+        if source_updater.feed_signature_changed?(feed_body_signature)
+          processing = entry_processor.process_feed_entries(feed)
+          content_changed = entries_digest_changed?(feed)
+        else
+          processing = EntryProcessingResult.new(
+            created: 0,
+            updated: 0,
+            unchanged: 0,
+            failed: 0,
+            items: [],
+            errors: [],
+            created_items: [],
+            updated_items: []
+          )
+          content_changed = false
+        end
+
+        feed_entries_digest = entries_digest(feed)
+        source_updater.update_source_for_success(response, duration_ms, feed, feed_body_signature, content_changed: content_changed, entries_digest: feed_entries_digest)
         source_updater.create_fetch_log(
           response: response,
           duration_ms: duration_ms,
@@ -177,6 +198,7 @@ module SourceMonitor
           item_processing: EntryProcessingResult.new(
             created: 0,
             updated: 0,
+            unchanged: 0,
             failed: 0,
             items: [],
             errors: [],
@@ -227,6 +249,7 @@ module SourceMonitor
           item_processing: EntryProcessingResult.new(
             created: 0,
             updated: 0,
+            unchanged: 0,
             failed: 0,
             items: [],
             errors: [],
@@ -236,6 +259,24 @@ module SourceMonitor
         )
       end
 
+      def attempt_aia_recovery(_error, started_at, instrumentation_payload)
+        return if @aia_attempted
+
+        @aia_attempted = true
+        hostname = URI.parse(source.feed_url).host
+        intermediate = SourceMonitor::HTTP::AIAResolver.resolve(hostname)
+        return unless intermediate
+
+        store = SourceMonitor::HTTP::AIAResolver.enhanced_cert_store([ intermediate ])
+        @connection = SourceMonitor::HTTP.client(cert_store: store, headers: request_headers)
+        instrumentation_payload[:aia_resolved] = true
+
+        response = perform_request
+        handle_response(response, started_at, instrumentation_payload)
+      rescue StandardError
+        nil
+      end
+
       def build_http_error_from_faraday(error)
         response_hash = error.response || {}
         headers = response_hash[:headers] || response_hash[:response_headers] || {}
@@ -256,6 +297,32 @@ module SourceMonitor
         Digest::SHA256.hexdigest(body)
       end
 
+      def entries_digest(feed)
+        return if feed.nil? || !feed.respond_to?(:entries)
+
+        ids = Array(feed.entries).map do |entry|
+          if entry.respond_to?(:entry_id) && entry.entry_id.present?
+            entry.entry_id
+          elsif entry.respond_to?(:url) && entry.url.present?
+            entry.url
+          elsif entry.respond_to?(:title) && entry.title.present?
+            entry.title
+          end
+        end.compact.sort
+
+        return if ids.empty?
+
+        Digest::SHA256.hexdigest(ids.join("\0"))
+      end
+
+      def entries_digest_changed?(feed)
+        digest = entries_digest(feed)
+        return false if digest.nil?
+
+        stored = (source.metadata || {}).fetch("last_entries_digest", nil)
+        stored != digest
+      end
+
       def adaptive_interval
         @adaptive_interval ||= AdaptiveInterval.new(source: source, jitter_proc: jitter_proc)
       end

data/lib/source_monitor/http/aia_resolver.rb

@@ -0,0 +1,128 @@
+# frozen_string_literal: true
+
+require "openssl"
+require "net/http"
+require "socket"
+
+module SourceMonitor
+  module HTTP
+    module AIAResolver
+      CONNECT_TIMEOUT = 5
+      DOWNLOAD_TIMEOUT = 5
+      CACHE_TTL = 3600 # 1 hour
+
+      class << self
+        def resolve(hostname, port: 443)
+          cached = cache_lookup(hostname)
+          return cached if cached
+
+          cert = fetch_leaf_certificate(hostname, port)
+          return unless cert
+
+          url = extract_aia_url(cert)
+          return unless url
+
+          intermediate = download_certificate(url)
+          return unless intermediate
+
+          cache_store(hostname, intermediate)
+          intermediate
+        rescue StandardError
+          nil
+        end
+
+        def enhanced_cert_store(additional_certs)
+          store = OpenSSL::X509::Store.new
+          store.set_default_paths
+
+          Array(additional_certs).each do |cert|
+            store.add_cert(cert)
+          rescue OpenSSL::X509::StoreError
+            # Already in store or invalid -- skip
+          end
+
+          store
+        end
+
+        def clear_cache!
+          @mutex.synchronize { @cache.clear }
+        end
+
+        def cache_size
+          @mutex.synchronize { @cache.size }
+        end
+
+        private
+
+        def cache_lookup(hostname)
+          @mutex.synchronize do
+            entry = @cache[hostname]
+            return unless entry
+            return entry[:cert] if entry[:expires_at] > Time.now
+
+            @cache.delete(hostname)
+            nil
+          end
+        end
+
+        def cache_store(hostname, cert)
+          @mutex.synchronize do
+            @cache[hostname] = { cert: cert, expires_at: Time.now + CACHE_TTL }
+          end
+        end
+
+        def fetch_leaf_certificate(hostname, port)
+          tcp = Socket.tcp(hostname, port, connect_timeout: CONNECT_TIMEOUT)
+          ssl_context = OpenSSL::SSL::SSLContext.new
+          ssl_context.verify_mode = OpenSSL::SSL::VERIFY_NONE
+
+          ssl = OpenSSL::SSL::SSLSocket.new(tcp, ssl_context)
+          ssl.hostname = hostname
+          ssl.connect
+
+          ssl.peer_cert
+        rescue StandardError
+          nil
+        ensure
+          ssl&.close rescue nil # rubocop:disable Style/RescueModifier
+          tcp&.close rescue nil # rubocop:disable Style/RescueModifier
+        end
+
+        def extract_aia_url(cert)
+          return unless cert.respond_to?(:ca_issuer_uris)
+
+          uris = cert.ca_issuer_uris
+          return if uris.nil? || uris.empty?
+
+          uris.first.to_s
+        rescue StandardError
+          nil
+        end
+
+        def download_certificate(url)
+          uri = URI.parse(url)
+          http = Net::HTTP.new(uri.host, uri.port)
+          http.open_timeout = DOWNLOAD_TIMEOUT
+          http.read_timeout = DOWNLOAD_TIMEOUT
+
+          response = http.get(uri.request_uri)
+          return unless response.is_a?(Net::HTTPSuccess)
+
+          body = response.body
+          parse_certificate(body)
+        rescue StandardError
+          nil
+        end
+
+        def parse_certificate(body)
+          OpenSSL::X509::Certificate.new(body) # tries DER first, then PEM
+        rescue OpenSSL::X509::CertificateError
+          nil
+        end
+      end
+
+      @mutex = Mutex.new
+      @cache = {}
+    end
+  end
+end

data/lib/source_monitor/http.rb

@@ -9,6 +9,8 @@ require "active_support/core_ext/object/blank"
 
 module SourceMonitor
   module HTTP
+    autoload :AIAResolver, "source_monitor/http/aia_resolver"
+
     DEFAULT_TIMEOUT = 15
     DEFAULT_OPEN_TIMEOUT = 5
     DEFAULT_MAX_REDIRECTS = 5
@@ -16,7 +18,7 @@ module SourceMonitor
     RETRY_STATUSES = [ 429, 500, 502, 503, 504 ].freeze
 
     class << self
-      def client(proxy: nil, headers: {}, timeout: nil, open_timeout: nil, retry_requests: true)
+      def client(proxy: nil, headers: {}, timeout: nil, open_timeout: nil, retry_requests: true, cert_store: nil)
         settings = SourceMonitor.config.http
 
         effective_proxy = resolve_proxy(proxy, settings)
@@ -30,14 +32,15 @@ module SourceMonitor
             timeout: effective_timeout,
             open_timeout: effective_open_timeout,
             settings: settings,
-            enable_retry: retry_requests
+            enable_retry: retry_requests,
+            cert_store: cert_store
           )
         end
       end
 
       private
 
-      def configure_request(connection, headers, timeout:, open_timeout:, settings:, enable_retry:) # rubocop:disable Metrics/MethodLength
+      def configure_request(connection, headers, timeout:, open_timeout:, settings:, enable_retry:, cert_store: nil) # rubocop:disable Metrics/MethodLength
         if enable_retry
           connection.request :retry,
                              max: settings.retry_max || 4,
@@ -58,7 +61,7 @@ module SourceMonitor
           connection.headers[key] = value
         end
 
-        configure_ssl(connection, settings)
+        configure_ssl(connection, settings, cert_store: cert_store)
 
         connection.adapter Faraday.default_adapter
       end
@@ -67,7 +70,7 @@ module SourceMonitor
       # fail to verify certificate chains that depend on intermediate CAs
       # (e.g., Medium/Netflix on AWS). OpenSSL::X509::Store#set_default_paths
       # loads all system-trusted CAs including intermediates.
-      def configure_ssl(connection, settings)
+      def configure_ssl(connection, settings, cert_store: nil)
         connection.ssl.verify = settings.ssl_verify != false
 
         if settings.ssl_ca_file
@@ -75,7 +78,7 @@ module SourceMonitor
         elsif settings.ssl_ca_path
           connection.ssl.ca_path = settings.ssl_ca_path
         else
-          connection.ssl.cert_store = default_cert_store
+          connection.ssl.cert_store = cert_store || default_cert_store
         end
       end
 

data/lib/source_monitor/items/item_creator.rb

@@ -21,6 +21,10 @@ module SourceMonitor
         def updated?
           status == :updated
         end
+
+        def unchanged?
+          status == :unchanged
+        end
       end
 
       FINGERPRINT_SEPARATOR = "\u0000".freeze
@@ -46,8 +50,15 @@ module SourceMonitor
         existing_item, matched_by = existing_item_for(attributes, raw_guid_present: raw_guid.present?)
 
         if existing_item
-          updated_item = update_existing_item(existing_item, attributes, matched_by)
-          return Result.new(item: updated_item, status: :updated, matched_by: matched_by)
+          apply_attributes(existing_item, attributes)
+          instrument_duplicate(existing_item, matched_by)
+          if significant_changes?(existing_item)
+            existing_item.save!
+            return Result.new(item: existing_item, status: :updated, matched_by: matched_by)
+          else
+            existing_item.reload if existing_item.changed?
+            return Result.new(item: existing_item, status: :unchanged, matched_by: matched_by)
+          end
         end
 
         create_new_item(attributes, raw_guid_present: raw_guid.present?)
@@ -100,7 +111,7 @@ module SourceMonitor
 
       def update_existing_item(existing_item, attributes, matched_by)
         apply_attributes(existing_item, attributes)
-        existing_item.save!
+        existing_item.save! if significant_changes?(existing_item)
         instrument_duplicate(existing_item, matched_by)
         existing_item
       end
@@ -117,8 +128,15 @@ module SourceMonitor
       def handle_concurrent_duplicate(attributes, raw_guid_present:)
         matched_by = raw_guid_present ? :guid : :fingerprint
         existing = find_conflicting_item(attributes, matched_by)
-        updated = update_existing_item(existing, attributes, matched_by)
-        Result.new(item: updated, status: :updated, matched_by: matched_by)
+        apply_attributes(existing, attributes)
+        instrument_duplicate(existing, matched_by)
+        if significant_changes?(existing)
+          existing.save!
+          Result.new(item: existing, status: :updated, matched_by: matched_by)
+        else
+          existing.reload if existing.changed?
+          Result.new(item: existing, status: :unchanged, matched_by: matched_by)
+        end
       end
 
       def find_conflicting_item(attributes, matched_by)
@@ -131,6 +149,10 @@ module SourceMonitor
         end
       end
 
+      # Attributes that should not trigger an "updated" status when they change.
+      # Metadata contains feedjira object references that differ between parses.
+      IGNORED_CHANGE_ATTRIBUTES = %w[metadata].freeze
+
       def apply_attributes(record, attributes)
         attributes = attributes.dup
         metadata = attributes.delete(:metadata)
@@ -138,6 +160,10 @@ module SourceMonitor
         record.metadata = metadata if metadata
       end
 
+      def significant_changes?(record)
+        (record.changed - IGNORED_CHANGE_ATTRIBUTES).any?
+      end
+
       def build_attributes
         entry_parser.parse
       end

data/lib/source_monitor/scrapers/fetchers/http_fetcher.rb

@@ -10,6 +10,7 @@ module SourceMonitor
 
         def initialize(http: SourceMonitor::HTTP)
           @http = http
+          @aia_attempted = false
         end
 
         def fetch(url:, settings: nil)
@@ -25,6 +26,11 @@ module SourceMonitor
               message: "Non-success HTTP status"
             )
           end
+        rescue Faraday::SSLError => error
+          result = attempt_aia_recovery(url, settings)
+          return result if result
+
+          Result.new(status: :failed, error: error.class.name, message: error.message)
         rescue Faraday::ClientError => error
           Result.new(
             status: :failed,
@@ -40,13 +46,34 @@ module SourceMonitor
 
         attr_reader :http
 
-        def connection(settings)
+        def attempt_aia_recovery(url, settings)
+          return if @aia_attempted
+
+          @aia_attempted = true
+          hostname = URI.parse(url).host
+          intermediate = SourceMonitor::HTTP::AIAResolver.resolve(hostname)
+          return unless intermediate
+
+          store = SourceMonitor::HTTP::AIAResolver.enhanced_cert_store([ intermediate ])
+          response = connection(settings, cert_store: store).get(url)
+
+          if success_status?(response.status)
+            Result.new(status: :success, body: response.body, headers: response.headers, http_status: response.status)
+          else
+            Result.new(status: :failed, http_status: response.status, error: "http_error", message: "Non-success HTTP status")
+          end
+        rescue StandardError
+          nil
+        end
+
+        def connection(settings, cert_store: nil)
           normalized = normalize_settings(settings)
           http.client(
             proxy: normalized[:proxy],
             headers: normalized[:headers],
             timeout: normalized[:timeout] || SourceMonitor::HTTP::DEFAULT_TIMEOUT,
-            open_timeout: normalized[:open_timeout] || SourceMonitor::HTTP::DEFAULT_OPEN_TIMEOUT
+            open_timeout: normalized[:open_timeout] || SourceMonitor::HTTP::DEFAULT_OPEN_TIMEOUT,
+            cert_store: cert_store
           )
         end
 

data/lib/source_monitor/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module SourceMonitor
-  VERSION = "0.5.3"
+  VERSION = "0.7.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: source_monitor
 version: !ruby/object:Gem::Version
-  version: 0.5.3
+  version: 0.7.0
 platform: ruby
 authors:
 - dchuk
@@ -343,7 +343,9 @@ files:
 - ".rubocop.yml"
 - ".ruby-version"
 - ".vbw-planning/PROJECT.md"
+- ".vbw-planning/ROADMAP.md"
 - ".vbw-planning/SHIPPED.md"
+- ".vbw-planning/STATE.md"
 - ".vbw-planning/codebase/ARCHITECTURE.md"
 - ".vbw-planning/codebase/CONCERNS.md"
 - ".vbw-planning/codebase/CONVENTIONS.md"
@@ -425,6 +427,13 @@ files:
 - ".vbw-planning/milestones/upgrade-assurance/phases/03-upgrade-skill-docs/03-VERIFICATION.md"
 - ".vbw-planning/milestones/upgrade-assurance/phases/03-upgrade-skill-docs/PLAN-01-SUMMARY.md"
 - ".vbw-planning/milestones/upgrade-assurance/phases/03-upgrade-skill-docs/PLAN-01.md"
+- ".vbw-planning/phases/01-aia-certificate-resolution/.context-dev.md"
+- ".vbw-planning/phases/01-aia-certificate-resolution/PLAN-01-SUMMARY.md"
+- ".vbw-planning/phases/01-aia-certificate-resolution/PLAN-01.md"
+- ".vbw-planning/phases/01-aia-certificate-resolution/PLAN-02-SUMMARY.md"
+- ".vbw-planning/phases/01-aia-certificate-resolution/PLAN-02.md"
+- ".vbw-planning/phases/01-aia-certificate-resolution/PLAN-03-SUMMARY.md"
+- ".vbw-planning/phases/01-aia-certificate-resolution/PLAN-03.md"
 - AGENTS.md
 - CHANGELOG.md
 - CLAUDE.md
@@ -540,6 +549,7 @@ files:
 - app/views/source_monitor/sources/index.html.erb
 - app/views/source_monitor/sources/new.html.erb
 - app/views/source_monitor/sources/show.html.erb
+- config/brakeman.ignore
 - config/coverage_baseline.json
 - config/initializers/feedjira.rb
 - config/routes.rb
@@ -630,6 +640,7 @@ files:
 - lib/source_monitor/health/source_health_monitor.rb
 - lib/source_monitor/health/source_health_reset.rb
 - lib/source_monitor/http.rb
+- lib/source_monitor/http/aia_resolver.rb
 - lib/source_monitor/images/content_rewriter.rb
 - lib/source_monitor/images/downloader.rb
 - lib/source_monitor/import_sessions/entry_normalizer.rb