source_monitor 0.3.0 → 0.3.2

data/.claude/skills/sm-configuration-setting/reference/settings-pattern.md

@@ -0,0 +1,297 @@
+# Step-by-Step: Adding a Configuration Setting
+
+## Scenario A: Add to Existing Section
+
+### Example: Add `stale_threshold_hours` to FetchingSettings
+
+**Step 1: Edit the settings file**
+
+```ruby
+# lib/source_monitor/configuration/fetching_settings.rb
+class FetchingSettings
+  attr_accessor :min_interval_minutes,
+    :max_interval_minutes,
+    :increase_factor,
+    :decrease_factor,
+    :failure_increase_factor,
+    :jitter_percent,
+    :stale_threshold_hours        # ADD: new accessor
+
+  def reset!
+    @min_interval_minutes = 5
+    @max_interval_minutes = 24 * 60
+    @increase_factor = 1.25
+    @decrease_factor = 0.75
+    @failure_increase_factor = 1.5
+    @jitter_percent = 0.1
+    @stale_threshold_hours = 48   # ADD: default value
+  end
+end
+```
+
+**Step 2: Write tests**
+
+```ruby
+# test/lib/source_monitor/configuration_test.rb
+
+test "stale_threshold_hours has correct default" do
+  assert_equal 48, SourceMonitor.config.fetching.stale_threshold_hours
+end
+
+test "stale_threshold_hours can be overridden" do
+  SourceMonitor.configure do |config|
+    config.fetching.stale_threshold_hours = 72
+  end
+  assert_equal 72, SourceMonitor.config.fetching.stale_threshold_hours
+end
+
+test "stale_threshold_hours resets with configuration" do
+  SourceMonitor.configure do |config|
+    config.fetching.stale_threshold_hours = 72
+  end
+  SourceMonitor.reset_configuration!
+  assert_equal 48, SourceMonitor.config.fetching.stale_threshold_hours
+end
+```
+
+**Step 3: Run tests**
+
+```bash
+PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/configuration_test.rb
+```
+
+---
+
+## Scenario B: Add Setting with Validation
+
+### Example: Add validated `strategy` to a section
+
+**Step 1: Edit the settings file with custom setter**
+
+```ruby
+class MySettings
+  VALID_MODES = %i[fast balanced thorough].freeze
+
+  attr_reader :mode
+
+  def initialize
+    reset!
+  end
+
+  def mode=(value)
+    normalized = value&.to_sym
+    unless VALID_MODES.include?(normalized)
+      raise ArgumentError, "Invalid mode #{value.inspect}. Must be one of: #{VALID_MODES.join(', ')}"
+    end
+    @mode = normalized
+  end
+
+  def reset!
+    @mode = :balanced
+  end
+end
+```
+
+**Step 2: Write tests for all paths**
+
+```ruby
+test "mode defaults to balanced" do
+  assert_equal :balanced, SourceMonitor.config.my_section.mode
+end
+
+test "mode accepts valid values" do
+  %i[fast balanced thorough].each do |mode|
+    SourceMonitor.configure do |config|
+      config.my_section.mode = mode
+    end
+    assert_equal mode, SourceMonitor.config.my_section.mode
+  end
+end
+
+test "mode accepts string values" do
+  SourceMonitor.configure do |config|
+    config.my_section.mode = "fast"
+  end
+  assert_equal :fast, SourceMonitor.config.my_section.mode
+end
+
+test "mode rejects invalid values" do
+  assert_raises(ArgumentError, /Invalid mode/) do
+    SourceMonitor.configure do |config|
+      config.my_section.mode = :invalid
+    end
+  end
+end
+```
+
+---
+
+## Scenario C: Add Setting with Normalization
+
+### Example: Numeric setting that normalizes edge cases
+
+Follow the `ScrapingSettings` pattern:
+
+```ruby
+class MySettings
+  DEFAULT_LIMIT = 50
+
+  attr_reader :limit
+
+  def initialize
+    reset!
+  end
+
+  def limit=(value)
+    @limit = normalize_numeric(value)
+  end
+
+  def reset!
+    @limit = DEFAULT_LIMIT
+  end
+
+  private
+
+  def normalize_numeric(value)
+    return nil if value.nil?
+    return nil if value == ""
+    integer = value.respond_to?(:to_i) ? value.to_i : value
+    integer.positive? ? integer : nil
+  end
+end
+```
+
+**Test normalization edge cases:**
+
+```ruby
+test "limit normalizes string to integer" do
+  SourceMonitor.configure { |c| c.my_section.limit = "10" }
+  assert_equal 10, SourceMonitor.config.my_section.limit
+end
+
+test "limit normalizes nil to nil" do
+  SourceMonitor.configure { |c| c.my_section.limit = nil }
+  assert_nil SourceMonitor.config.my_section.limit
+end
+
+test "limit normalizes empty string to nil" do
+  SourceMonitor.configure { |c| c.my_section.limit = "" }
+  assert_nil SourceMonitor.config.my_section.limit
+end
+
+test "limit normalizes zero to nil" do
+  SourceMonitor.configure { |c| c.my_section.limit = 0 }
+  assert_nil SourceMonitor.config.my_section.limit
+end
+
+test "limit normalizes negative to nil" do
+  SourceMonitor.configure { |c| c.my_section.limit = -5 }
+  assert_nil SourceMonitor.config.my_section.limit
+end
+```
+
+---
+
+## Scenario D: Create a New Settings Section
+
+### Step 1: Create the settings file
+
+```ruby
+# lib/source_monitor/configuration/notifications_settings.rb
+# frozen_string_literal: true
+
+module SourceMonitor
+  class Configuration
+    class NotificationsSettings
+      attr_accessor :enabled, :channels, :throttle_seconds
+
+      def initialize
+        reset!
+      end
+
+      def reset!
+        @enabled = true
+        @channels = []
+        @throttle_seconds = 60
+      end
+    end
+  end
+end
+```
+
+### Step 2: Add require and reader to Configuration
+
+```ruby
+# lib/source_monitor/configuration.rb
+
+# At the top, add require:
+require "source_monitor/configuration/notifications_settings"
+
+# In the class:
+attr_reader :http, :scrapers, :retention, :events, :models,
+  :realtime, :fetching, :health, :authentication, :scraping,
+  :notifications
+
+# In initialize:
+def initialize
+  # ... existing ...
+  @notifications = NotificationsSettings.new
+end
+```
+
+### Step 3: Write tests
+
+```ruby
+# test/lib/source_monitor/configuration_test.rb
+
+test "notifications settings have correct defaults" do
+  settings = SourceMonitor.config.notifications
+  assert_equal true, settings.enabled
+  assert_equal [], settings.channels
+  assert_equal 60, settings.throttle_seconds
+end
+
+test "notifications settings can be configured" do
+  SourceMonitor.configure do |config|
+    config.notifications.enabled = false
+    config.notifications.channels = [:email]
+    config.notifications.throttle_seconds = 30
+  end
+
+  settings = SourceMonitor.config.notifications
+  assert_equal false, settings.enabled
+  assert_equal [:email], settings.channels
+  assert_equal 30, settings.throttle_seconds
+end
+
+test "notifications reset restores defaults" do
+  SourceMonitor.configure do |config|
+    config.notifications.enabled = false
+  end
+  SourceMonitor.reset_configuration!
+  assert_equal true, SourceMonitor.config.notifications.enabled
+end
+```
+
+### Step 4: Run tests
+
+```bash
+PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/configuration_test.rb
+```
+
+---
+
+## Checklist for Any New Setting
+
+- [ ] Attribute added to settings class (attr_accessor or custom setter)
+- [ ] Default value set in `reset!` (or `initialize` for classes without `reset!`)
+- [ ] require statement added (if new file)
+- [ ] Reader method exposed on Configuration (if new section)
+- [ ] Initialization in Configuration#initialize (if new section)
+- [ ] Tests: default value is correct
+- [ ] Tests: value can be overridden
+- [ ] Tests: reset restores default
+- [ ] Tests: validation raises ArgumentError (if applicable)
+- [ ] Tests: edge cases for normalization (if applicable)
+- [ ] All tests pass: `PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/configuration_test.rb`
+- [ ] RuboCop clean: `bin/rubocop lib/source_monitor/configuration/`

data/.claude/skills/sm-configure/SKILL.md

@@ -0,0 +1,153 @@
+---
+name: sm-configure
+description: Use when configuring SourceMonitor engine settings via the DSL, including queue settings, HTTP client, fetching, health, scrapers, retention, scraping controls, events, model extensions, realtime, and authentication.
+allowed-tools: Read, Write, Edit, Bash, Glob, Grep
+---
+
+# sm-configure: Engine Configuration DSL
+
+Comprehensive reference for configuring SourceMonitor via `SourceMonitor.configure`.
+
+## When to Use
+
+- Adding or modifying settings in `config/initializers/source_monitor.rb`
+- Understanding what configuration options are available
+- Debugging configuration-related issues
+- Setting up environment-specific overrides
+
+## Configuration Entry Point
+
+All configuration lives inside the `configure` block in the host app's initializer:
+
+```ruby
+SourceMonitor.configure do |config|
+  # settings here
+end
+```
+
+After the block executes, `ModelExtensions.reload!` runs automatically to apply any model changes. Restart web and worker processes after changes.
+
+## Configuration Sections
+
+The `config` object (`SourceMonitor::Configuration`) has 10 sub-sections plus top-level queue/job settings:
+
+| Section | Accessor | Class |
+|---|---|---|
+| Top-level | `config.*` | `Configuration` |
+| HTTP | `config.http` | `HTTPSettings` |
+| Fetching | `config.fetching` | `FetchingSettings` |
+| Health | `config.health` | `HealthSettings` |
+| Scrapers | `config.scrapers` | `ScraperRegistry` |
+| Retention | `config.retention` | `RetentionSettings` |
+| Scraping | `config.scraping` | `ScrapingSettings` |
+| Events | `config.events` | `Events` |
+| Models | `config.models` | `Models` |
+| Realtime | `config.realtime` | `RealtimeSettings` |
+| Authentication | `config.authentication` | `AuthenticationSettings` |
+
+See `reference/configuration-reference.md` for every setting with types, defaults, and examples.
+
+## Quick Examples
+
+### Queue Configuration
+```ruby
+config.queue_namespace = "source_monitor"
+config.fetch_queue_name = "source_monitor_fetch"
+config.fetch_queue_concurrency = 4
+```
+
+### HTTP Client
+```ruby
+config.http.timeout = 30
+config.http.proxy = ENV["HTTP_PROXY"]
+config.http.retry_max = 3
+```
+
+### Authentication (Devise)
+```ruby
+config.authentication.authenticate_with :authenticate_user!
+config.authentication.authorize_with ->(c) { c.current_user&.admin? }
+```
+
+### Events
+```ruby
+config.events.after_item_created { |e| Notifier.new_item(e.item) }
+config.events.register_item_processor ->(ctx) { Indexer.index(ctx.item) }
+```
+
+### Model Extensions
+```ruby
+config.models.table_name_prefix = "sm_"
+config.models.source.include_concern "MyApp::SourceExtension"
+config.models.item.validate :custom_check
+```
+
+### Realtime
+```ruby
+config.realtime.adapter = :redis
+config.realtime.redis_url = ENV["REDIS_URL"]
+```
+
+## Helper APIs
+
+```ruby
+SourceMonitor.config                         # Current configuration
+SourceMonitor.configure { |c| ... }          # Set configuration
+SourceMonitor.reset_configuration!           # Revert to defaults (for tests)
+SourceMonitor.events                         # Shortcut to config.events
+SourceMonitor.queue_name(:fetch)             # Resolved queue name
+SourceMonitor.queue_concurrency(:scrape)     # Resolved concurrency
+SourceMonitor.mission_control_dashboard_path # Resolved MC path or nil
+```
+
+## Key Source Files
+
+| File | Purpose |
+|---|---|
+| `lib/source_monitor/configuration.rb` | Main Configuration class |
+| `lib/source_monitor/configuration/http_settings.rb` | HTTP client settings |
+| `lib/source_monitor/configuration/fetching_settings.rb` | Adaptive scheduling |
+| `lib/source_monitor/configuration/health_settings.rb` | Health monitoring |
+| `lib/source_monitor/configuration/scraper_registry.rb` | Scraper adapter registry |
+| `lib/source_monitor/configuration/retention_settings.rb` | Item retention |
+| `lib/source_monitor/configuration/scraping_settings.rb` | Scraping controls |
+| `lib/source_monitor/configuration/events.rb` | Event callbacks |
+| `lib/source_monitor/configuration/models.rb` | Model extensions config |
+| `lib/source_monitor/configuration/model_definition.rb` | Per-model definition |
+| `lib/source_monitor/configuration/realtime_settings.rb` | Action Cable settings |
+| `lib/source_monitor/configuration/authentication_settings.rb` | Auth settings |
+| `lib/source_monitor/configuration/validation_definition.rb` | Validation wrapper |
+
+## References
+
+- `reference/configuration-reference.md` -- Complete settings reference
+- `docs/configuration.md` -- Official configuration documentation
+- `lib/generators/source_monitor/install/templates/source_monitor.rb.tt` -- Initializer template
+
+## Testing
+
+Reset configuration between tests:
+```ruby
+setup do
+  SourceMonitor.reset_configuration!
+end
+```
+
+Test custom configuration:
+```ruby
+test "custom queue name" do
+  SourceMonitor.configure do |config|
+    config.fetch_queue_name = "custom_fetch"
+  end
+  assert_equal "custom_fetch", SourceMonitor.queue_name(:fetch)
+end
+```
+
+## Checklist
+
+- [ ] Initializer exists at `config/initializers/source_monitor.rb`
+- [ ] Queue names match `config/solid_queue.yml` entries
+- [ ] Authentication hooks configured for host auth system
+- [ ] HTTP timeouts appropriate for target feeds
+- [ ] Retention policy set for production
+- [ ] Workers restarted after configuration changes