source_monitor 0.7.0 → 0.8.0

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-01-SUMMARY.md

@@ -1,26 +0,0 @@
----
-phase: 1
-plan: 1
-status: complete
----
-# Plan 01 Summary: AIA Resolver Module
-
-## Tasks Completed
-- [x] Task 1: Created lib/source_monitor/http/aia_resolver.rb
-- [x] Task 2: Created test/lib/source_monitor/http/aia_resolver_test.rb
-
-## Commits
-- 4c9568a: feat(1-1): add AIA intermediate certificate resolver
-
-## Files Modified
-- lib/source_monitor/http/aia_resolver.rb (created)
-- test/lib/source_monitor/http/aia_resolver_test.rb (created)
-
-## What Was Built
-- `SourceMonitor::HTTP::AIAResolver` module with thread-safe cached resolution of missing intermediate SSL certificates via AIA (Authority Information Access) X.509 extension
-- Public API: `resolve(hostname)`, `enhanced_cert_store(certs)`, `clear_cache!`, `cache_size`
-- Private methods: `fetch_leaf_certificate` (VERIFY_NONE + SNI), `extract_aia_url` (uses `cert.ca_issuer_uris`), `download_certificate` (DER-first, PEM fallback)
-- 11 unit tests covering all public/private methods, caching, TTL expiration, and error handling
-
-## Deviations
-- None

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-01.md

@@ -1,71 +0,0 @@
----
-phase: 1
-plan: 1
-title: "AIA Resolver Module"
-wave: 1
-depends_on: []
-must_haves:
-  - AIAResolver module with resolve, enhanced_cert_store, clear_cache!, cache_size
-  - Thread-safe Mutex + Hash cache with 1-hour TTL per hostname
-  - fetch_leaf_certificate with VERIFY_NONE and SNI support
-  - extract_aia_url using cert.ca_issuer_uris (not regex)
-  - download_certificate with DER-first, PEM-fallback parsing
-  - All methods rescue StandardError and return nil
-  - Unit tests covering all public and private methods
----
-
-# Plan 01: AIA Resolver Module
-
-## Goal
-
-Create `SourceMonitor::HTTP::AIAResolver` -- a standalone module that resolves missing intermediate certificates via the AIA (Authority Information Access) extension in X.509 certificates.
-
-## Tasks
-
-### Task 1: Create lib/source_monitor/http/aia_resolver.rb
-
-Create new module `SourceMonitor::HTTP::AIAResolver` with class methods:
-
-**Public API:**
-- `resolve(hostname, port: 443)` -- Entry point. Checks cache first, then: fetch leaf cert -> extract AIA URL -> download intermediate. Returns `OpenSSL::X509::Certificate` or `nil`.
-- `enhanced_cert_store(additional_certs)` -- Builds `OpenSSL::X509::Store` with `set_default_paths` plus extra certs from the array.
-- `clear_cache!` -- Clears the hostname cache (for testing).
-- `cache_size` -- Returns number of cached entries (for testing).
-
-**Private methods:**
-- `fetch_leaf_certificate(hostname, port)` -- TCP+SSL connect with `VERIFY_NONE` to get the server's leaf cert. 5s connect timeout. Uses `ssl_socket.hostname=` for SNI.
-- `extract_aia_url(cert)` -- Uses Ruby's built-in `cert.ca_issuer_uris` method. Returns first URI string or nil.
-- `download_certificate(url)` -- Plain HTTP GET (AIA URLs are always HTTP, not HTTPS). 5s timeout. Parses DER body as `OpenSSL::X509::Certificate`, falls back to PEM on failure.
-
-**Cache:** `Mutex` + `Hash` keyed by hostname. Each entry stores `{ cert:, expires_at: }` with 1-hour TTL.
-
-**Safety:** All methods rescue `StandardError` and return `nil`. This is best-effort -- never makes things worse.
-
-### Task 2: Create test/lib/source_monitor/http/aia_resolver_test.rb
-
-Unit tests:
-- `extract_aia_url` with cert that has AIA extension returns URL
-- `extract_aia_url` with cert without AIA returns nil
-- `download_certificate` with DER body parses correctly (WebMock stub)
-- `download_certificate` returns nil on HTTP 404 (WebMock)
-- `download_certificate` returns nil on timeout (WebMock)
-- `enhanced_cert_store` returns store with added certs
-- `enhanced_cert_store` handles empty array gracefully
-- Cache: resolve stores result, second call returns cached
-- Cache: expired entries are re-fetched
-- `clear_cache!` empties the cache
-- `resolve` returns nil when hostname unreachable (stub fetch_leaf_certificate)
-
-## Files
-
-| Action | Path |
-|--------|------|
-| CREATE | `lib/source_monitor/http/aia_resolver.rb` |
-| CREATE | `test/lib/source_monitor/http/aia_resolver_test.rb` |
-
-## Verification
-
-```bash
-PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/http/aia_resolver_test.rb
-bin/rubocop lib/source_monitor/http/aia_resolver.rb test/lib/source_monitor/http/aia_resolver_test.rb
-```

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-02-SUMMARY.md

@@ -1,16 +0,0 @@
----
-phase: 1
-plan: 2
-status: complete
-commit: f60e9bf
----
-
-## What Was Built
-- Added `cert_store:` keyword parameter to `HTTP.client` for custom OpenSSL cert stores
-- Added `autoload :AIAResolver` to HTTP module
-- Plumbed cert_store through `configure_request` -> `configure_ssl` with fallback to `default_cert_store`
-- 2 new tests: custom cert_store usage, ssl_ca_file takes precedence over cert_store
-
-## Files Modified
-- `lib/source_monitor/http.rb` — autoload, cert_store param, SSL plumbing
-- `test/lib/source_monitor/http_test.rb` — 2 new cert_store tests

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-02.md

@@ -1,56 +0,0 @@
----
-phase: 1
-plan: 2
-title: "HTTP Module cert_store Parameter"
-wave: 1
-depends_on: []
-must_haves:
-  - Add autoload :AIAResolver to module HTTP
-  - Add cert_store keyword to client method
-  - Pass cert_store through configure_request to configure_ssl
-  - configure_ssl uses cert_store when no ssl_ca_file/ssl_ca_path
-  - Tests for cert_store parameter usage
----
-
-# Plan 02: HTTP Module cert_store Parameter
-
-## Goal
-
-Extend `SourceMonitor::HTTP.client` to accept an optional `cert_store:` parameter, enabling callers (like FeedFetcher's AIA retry) to provide a custom `OpenSSL::X509::Store` with additional certificates.
-
-## Tasks
-
-### Task 1: Modify lib/source_monitor/http.rb
-
-1. Add autoload inside `module HTTP` (after RETRY_STATUSES):
-   ```ruby
-   autoload :AIAResolver, "source_monitor/http/aia_resolver"
-   ```
-
-2. Add `cert_store: nil` keyword to `client` method signature.
-
-3. Pass `cert_store:` through `configure_request` to `configure_ssl`:
-   - Add `cert_store:` parameter to `configure_request`
-   - Pass it to `configure_ssl(connection, settings, cert_store:)`
-
-4. In `configure_ssl`: when no `ssl_ca_file` or `ssl_ca_path` is set, use `cert_store || default_cert_store`.
-
-### Task 2: Add tests to test/lib/source_monitor/http_test.rb
-
-Add 2 tests:
-- `cert_store: param is used when no ssl_ca_file or ssl_ca_path` -- pass a custom store, verify `connection.ssl.cert_store` is the custom store
-- `cert_store: is ignored when ssl_ca_file is set` -- configure ssl_ca_file, pass cert_store, verify ca_file takes precedence
-
-## Files
-
-| Action | Path |
-|--------|------|
-| MODIFY | `lib/source_monitor/http.rb` |
-| MODIFY | `test/lib/source_monitor/http_test.rb` |
-
-## Verification
-
-```bash
-PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/http_test.rb
-bin/rubocop lib/source_monitor/http.rb test/lib/source_monitor/http_test.rb
-```

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-03-SUMMARY.md

@@ -1,17 +0,0 @@
----
-phase: 1
-plan: 3
-status: complete
-commit: 9c38bc3
----
-
-## What Was Built
-- Wired AIA certificate resolution into FeedFetcher's SSL error handling
-- On `Faraday::SSLError`, attempts intermediate cert recovery via `AIAResolver.resolve` before raising
-- Guard flag `@aia_attempted` prevents infinite recursion; `rescue StandardError => nil` ensures recovery never makes things worse
-- Tags `instrumentation_payload[:aia_resolved] = true` on successful AIA recovery
-- 3 integration tests: success retry path, nil fallback to ConnectionError, non-SSL skip
-
-## Files Modified
-- `lib/source_monitor/fetching/feed_fetcher.rb` — split SSL rescue, add `attempt_aia_recovery`
-- `test/lib/source_monitor/fetching/feed_fetcher_test.rb` — 3 AIA resolution tests

data/.vbw-planning/phases/01-aia-certificate-resolution/PLAN-03.md

@@ -1,98 +0,0 @@
----
-phase: 1
-plan: 3
-title: "FeedFetcher AIA Retry Integration"
-wave: 2
-depends_on: [1, 2]
-must_haves:
-  - Separate Faraday::SSLError rescue from Faraday::ConnectionFailed
-  - On SSLError attempt AIA resolution once (aia_attempted flag)
-  - Parse hostname from source.feed_url for AIA resolve
-  - If intermediate found rebuild connection with enhanced cert store and retry
-  - If nil raise ConnectionError as before
-  - Tag successful recoveries with aia_resolved in instrumentation
-  - Integration tests for all AIA retry paths
-  - Full test suite passes (1003+ tests)
-  - RuboCop zero offenses
-  - Brakeman zero warnings
----
-
-# Plan 03: FeedFetcher AIA Retry Integration
-
-## Goal
-
-Wire AIA resolution into FeedFetcher's error handling so SSL failures automatically attempt intermediate certificate recovery before giving up.
-
-## Tasks
-
-### Task 1: Modify lib/source_monitor/fetching/feed_fetcher.rb
-
-Modify `perform_fetch` (lines 77-90):
-
-1. **Split rescue clause:** Separate `Faraday::SSLError` from `Faraday::ConnectionFailed` into its own rescue:
-   ```ruby
-   rescue Faraday::ConnectionFailed => error
-     raise ConnectionError.new(error.message, original_error: error)
-   rescue Faraday::SSLError => error
-     attempt_aia_recovery(error) || raise(ConnectionError.new(error.message, original_error: error))
-   ```
-
-2. **Add `attempt_aia_recovery` private method:**
-   - Guard: return nil if `@aia_attempted` is true (prevents recursion)
-   - Set `@aia_attempted = true`
-   - Parse hostname from `URI.parse(source.feed_url).host`
-   - Call `SourceMonitor::HTTP::AIAResolver.resolve(hostname)`
-   - If intermediate found:
-     - Build enhanced cert store via `AIAResolver.enhanced_cert_store([intermediate])`
-     - Rebuild `@connection = SourceMonitor::HTTP.client(cert_store: store, headers: request_headers)`
-     - Return `perform_request` (the retry)
-   - If nil: return nil (caller raises ConnectionError)
-   - Rescue StandardError -> nil (never make retry worse)
-
-3. **Tag instrumentation:** In the `handle_response` path after successful AIA retry, the `instrumentation_payload[:aia_resolved] = true` will naturally flow through since `perform_fetch` calls `handle_response` on the retried response.
-
-### Task 2: Add tests to test/lib/source_monitor/fetching/feed_fetcher_test.rb
-
-Add 3 tests under a new section `# -- AIA Certificate Resolution --`:
-
-1. **SSL error + AIA resolve succeeds -> fetch succeeds:**
-   - First stub: raise `Faraday::SSLError`
-   - Stub `AIAResolver.resolve` to return a mock certificate
-   - Stub `AIAResolver.enhanced_cert_store` to return a store
-   - Second stub (after retry): return 200 with RSS body
-   - Assert result.status == :fetched
-
-2. **SSL error + AIA resolve returns nil -> ConnectionError:**
-   - Stub to raise `Faraday::SSLError`
-   - Stub `AIAResolver.resolve` to return nil
-   - Assert result.status == :failed
-   - Assert result.error is ConnectionError
-
-3. **Non-SSL ConnectionError -> AIA not attempted:**
-   - Stub to raise `Faraday::ConnectionFailed`
-   - Verify `AIAResolver.resolve` was NOT called
-   - Assert result.status == :failed
-   - Assert result.error is ConnectionError
-
-### Task 3: Run full verification
-
-1. `PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_test.rb`
-2. `bin/rails test` (full suite)
-3. `bin/rubocop`
-4. `bin/brakeman --no-pager`
-
-## Files
-
-| Action | Path |
-|--------|------|
-| MODIFY | `lib/source_monitor/fetching/feed_fetcher.rb` |
-| MODIFY | `test/lib/source_monitor/fetching/feed_fetcher_test.rb` |
-
-## Verification
-
-```bash
-PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher_test.rb
-bin/rails test
-bin/rubocop
-bin/brakeman --no-pager
-```