source_monitor 0.1.1

data/docs/configuration.md

@@ -0,0 +1,170 @@
+# Configuration & API Reference
+
+All configuration lives in `SourceMonitor.configure`—the install generator creates `config/initializers/source_monitor.rb` with sensible defaults and inline documentation. This guide expands on every namespace so you know which knobs to turn in production.
+
+```ruby
+SourceMonitor.configure do |config|
+  # customize settings here
+end
+```
+
+Restart your application whenever you change these settings. The engine reloads model extensions automatically when the block runs, but background processes need a restart to pick up queue name or adapter changes.
+
+## Asset Bundling
+
+- SourceMonitor ships with npm-based bundling via `cssbundling-rails` and `jsbundling-rails`; follow `.ai/engine-asset-configuration.md:11-113` when adjusting dependency versions or adding new entrypoints.
+- Keep bundled outputs under the namespaced directories (`app/assets/builds/source_monitor`, `images/source_monitor`, `svgs/source_monitor`) so host apps avoid collisions. See `.ai/engine-asset-configuration.md:32-44` for the layout recommendations.
+- Use `rbenv exec bundle exec rake app:source_monitor:assets:build` (or `npm run build`) after tweaking Tailwind/Stimulus code to refresh the builds, and rely on `test/dummy/bin/dev` to watch `build:css:watch` + `build:js:watch` during development.
+- Sprockets hosts receive automatic precompile coverage; if a host prefers manifest-based precompilation, adapt the approach documented in `.ai/engine-asset-configuration.md:114-143`.
+
+## Queue & Worker Settings
+
+- `config.queue_namespace` – prefix applied to queue names (`"source_monitor"` by default)
+- `config.fetch_queue_name` / `config.scrape_queue_name` – base queue names before the host's `ActiveJob.queue_name_prefix` is applied
+- `config.fetch_queue_concurrency` / `config.scrape_queue_concurrency` – advisory values Solid Queue uses for per-queue limits
+- `config.queue_name_for(:fetch | :scrape)` – helper that respects the host's queue prefix
+
+Use the helpers exposed on `SourceMonitor`:
+
+```ruby
+SourceMonitor.queue_name(:fetch)    # => "source_monitor_fetch"
+SourceMonitor.queue_concurrency(:scrape) # => 2
+```
+
+## Job Metrics & Mission Control
+
+- `config.job_metrics_enabled` – toggles Solid Queue metrics collection for the dashboard cards (default `true`)
+- `config.mission_control_enabled` – surfaces the Mission Control link on the dashboard when `true`
+- `config.mission_control_dashboard_path` – host route helper or callable returning the Mission Control path/URL; left blank by default
+
+The helper `SourceMonitor.mission_control_dashboard_path` performs a routing check so the dashboard only renders links that resolve.
+
+## HTTP Client Settings
+
+`config.http` maps directly onto Faraday's middleware options.
+
+- `timeout` – total request timeout in seconds (default `15`)
+- `open_timeout` – connection open timeout in seconds (`5`)
+- `max_redirects` – maximum redirects to follow (`5`)
+- `user_agent` – defaults to `SourceMonitor/<version>`
+- `proxy` – hash or URL to configure proxy usage
+- `headers` – hash (or callables) merged into every request
+- `retry_max`, `retry_interval`, `retry_interval_randomness`, `retry_backoff_factor`, `retry_statuses` – mapped to `faraday-retry`
+
+## Fetching Behaviour
+
+`config.fetching` controls adaptive scheduling.
+
+- `min_interval_minutes` / `max_interval_minutes` – enforce floor/ceiling for automatic schedule adjustments (defaults: `5` and `1440`)
+- `increase_factor` / `decrease_factor` – multipliers when a source trends slow/fast
+- `failure_increase_factor` – multiplier applied on consecutive failures
+- `jitter_percent` – random jitter applied to next fetch time (0.1 = ±10%)
+
+## Retention Defaults
+
+`config.retention` sets global defaults that sources inherit when their fields are blank.
+
+- `items_retention_days` – prune items older than this many days (`nil` = retain forever)
+- `max_items` – keep only the most recent N items (`nil` = unlimited)
+- `strategy` – `:destroy` or `:soft_delete` (defaults to `:destroy` in the configuration class; the installer comments demonstrate `:soft_delete`)
+
+The retention pruner runs after every successful fetch and inside nightly cleanup jobs.
+
+## Scraper Registry
+
+Register adapters that inherit from `SourceMonitor::Scrapers::Base`:
+
+```ruby
+config.scrapers.register(:readability, SourceMonitor::Scrapers::Readability)
+config.scrapers.register(:custom, "MyApp::Scrapers::Premium" )
+```
+
+Adapters receive merged settings (`default -> source -> invocation`), and must return a `SourceMonitor::Scrapers::Result` object. Use `config.scrapers.unregister(:custom)` to remove overrides.
+
+## Events & Item Processors
+
+Respond to lifecycle events without monkey patching:
+
+```ruby
+config.events.after_item_created do |event|
+  Analytics.track_new_item(event.item, source: event.source)
+end
+
+config.events.after_fetch_completed do |event|
+  Rails.logger.info("Feed #{event.source.name} finished with #{event.status}")
+end
+
+config.events.register_item_processor ->(context) {
+  SearchIndexer.index(context.item)
+}
+```
+
+Event structs expose `item`, `source`, `entry`, `result`, `status`, and `occurred_at`. Item processors run after events and receive an `ItemProcessorContext` with the same shape.
+
+## Model Extensions
+
+`config.models` lets host apps customise engine models at load time.
+
+- `config.models.table_name_prefix` – override the default `sourcemon_` prefix
+- `config.models.source.include_concern "MyApp::SourceMonitor::SourceExtensions"` – mix in concerns before models load
+- `config.models.source.validate :ensure_metadata_rules` – register validations (blocks or symbols)
+- Equivalent hooks exist for items, fetch logs, scrape logs, and item content via `config.models.item`, etc.
+
+The engine reloads model extensions whenever configuration runs so code reloading in development continues to work.
+
+## Realtime Settings
+
+`config.realtime` governs Action Cable transport.
+
+- `config.realtime.adapter` – one of `:solid_cable`, `:redis`, or `:async`
+- `config.realtime.redis_url` – optional Redis URL when using the Redis adapter
+- `config.realtime.solid_cable` – yields options: `polling_interval`, `message_retention`, `autotrim`, `use_skip_locked`, `trim_batch_size`, `connects_to` (hash for multi-database setups), `silence_polling`
+
+Call `config.realtime.action_cable_config` if you need a full hash for environment-specific `cable.yml` generation.
+
+## Authentication Helpers
+
+Protect the dashboard with host-specific auth in one place:
+
+```ruby
+config.authentication.authenticate_with :authenticate_admin!
+config.authentication.authorize_with ->(controller) {
+  controller.current_user&.feature_enabled?(:source_monitor)
+}
+config.authentication.current_user_method = :current_user
+config.authentication.user_signed_in_method = :user_signed_in?
+```
+
+Handlers can be symbols (invoked on the controller) or callables. Return `false` or raise to deny access.
+
+## Health Model
+
+`config.health` tunes automatic pause/resume heuristics.
+
+- `window_size` – number of fetch attempts to evaluate (default `20`)
+- `healthy_threshold` / `warning_threshold` – ratios that drive UI badges
+- `auto_pause_threshold` / `auto_resume_threshold` – percentages that trigger automatic toggling
+- `auto_pause_cooldown_minutes` – grace period before re-enabling a source
+
+## Helper APIs
+
+- `SourceMonitor.configure` – run-time configuration entry point
+- `SourceMonitor.reset_configuration!` – revert to defaults (useful in tests)
+- `SourceMonitor.events` – direct access to the events registry
+- `SourceMonitor.queue_name(role)` / `SourceMonitor.queue_concurrency(role)` – convenience helpers
+- `SourceMonitor::Metrics.snapshot` – inspect counters/gauges (great for health checks)
+
+## Environment Variables
+
+The engine honours several environment variables out of the box:
+
+- `SOLID_QUEUE_SKIP_RECURRING` – skip loading `config/recurring.yml`
+- `SOLID_QUEUE_RECURRING_SCHEDULE_FILE` – alternative schedule file path
+- `SOFT_DELETE` / `SOURCE_IDS` / `SOURCE_ID` – overrides for item cleanup rake tasks
+- `FETCH_LOG_DAYS` / `SCRAPE_LOG_DAYS` – retention windows for log cleanup
+
+## After Changing Configuration
+
+1. Restart web and worker processes so Solid Queue picks up new queue names/adapters.
+2. Re-run `npm run build` (engine root) and `bin/rails assets:precompile` if you adjust engine assets per `.ai/engine-asset-configuration.md:11-188`.
+3. Keep a regression test per configuration extension—for example, ensure custom validations are exercised in MiniTest.

data/docs/deployment.md

@@ -0,0 +1,63 @@
+# Deployment Guide
+
+This guide captures the production considerations for running SourceMonitor inside a host Rails application. Pair it with your platform playbook (Heroku, Render, Kubernetes, etc.) for environment-specific instructions.
+
+## Build & Release Pipeline
+
+> **Ruby version management in production:** Use rbenv, asdf, or the Ruby version baked into your container image, depending on your deployment platform. The commands below are shown without version manager prefixes—adjust for your environment (e.g., `rbenv exec bundle install`, `asdf exec bundle install`, or bare `bundle install` in Docker).
+
+1. **Install dependencies** – use `bundle install` and `npm install` during build steps.
+2. **Copy and run migrations** – always run `bin/rails railties:install:migrations FROM=source_monitor` before `bin/rails db:migrate` so new engine tables ship with each release.
+3. **Precompile assets** – `bin/rails assets:precompile` pulls in SourceMonitor's bundled CSS/JS outputs and Stimulus controllers. Fail the build if `source_monitor:assets:verify` raises.
+4. **Run quality gates** – `bin/rubocop`, `bin/brakeman --no-pager`, `bin/lint-assets`, and `bin/test-coverage` mirror the repository CI setup.
+
+## Process Model
+
+SourceMonitor assumes the standard Rails 8 process split:
+
+- **Web** – your application server (Puma) serving the mounted engine and Action Cable. When using Solid Cable, no separate Redis process is required.
+- **Worker** – at least one Solid Queue worker (`bin/rails solid_queue:start`). Scale horizontally to match feed volume and retention pruning needs. Use queue selectors if you dedicate workers to `source_monitor_fetch` or `source_monitor_scrape`.
+- **Scheduler/Recurring** – optional process invoking `bin/jobs --recurring_schedule_file=config/recurring.yml` so the bundled recurring tasks enqueue fetch/scrape/cleanup jobs. Disable with `SOLID_QUEUE_SKIP_RECURRING=true` when another scheduler handles cron-style jobs.
+
+## Database & Storage
+
+- **Primary Database** – hosts sources, items, logs, and Solid Cable messages. Ensure autovacuum keeps up; retention pruning helps bound growth.
+- **Solid Queue Database (optional)** – create a dedicated connection via `rails solid_queue:install` if queue load warrants isolation. Update the generated config before running migrations.
+- **Backups** – include Solid Queue tables in backups if they share the primary database; they store job state.
+
+## Observability & Alerting
+
+- Subscribe to ActiveSupport notifications (`source_monitor.fetch.finish`, `source_monitor.scheduler.run`, `source_monitor.dashboard.*`) to emit logs or metrics into your monitoring stack.
+- Scrape `SourceMonitor::Metrics.snapshot` periodically (e.g., via a health check controller) to track counters and gauges in Prometheus or StatsD.
+- Mission Control integration becomes useful once queues exceed a few hundred jobs; enable it when your platform already hosts the Mission Control UI.
+
+## Security & Authentication
+
+- Lock down the engine routes with authentication hooks (`config.authentication.authenticate_with` / `authorize_with`).
+- Configure HTTPS for Action Cable if you expose Solid Cable over the public internet.
+- Store API keys for authenticated feeds in encrypted credentials and inject them via per-source custom headers.
+
+## Scaling Guidelines
+
+- Increase `config.fetch_queue_concurrency` and the number of Solid Queue workers as source volume grows.
+- Adjust `config.fetching` multipliers to smooth out noisy feeds; raising `failure_increase_factor` slows retries for consistently failing sources.
+- Use `config.retention` to cap database growth; nightly cleanup jobs can run on separate workers if pruning becomes heavy.
+
+## Rolling Upgrades
+
+1. Merge and deploy code.
+2. Run migrations before restarting workers so new tables/columns exist before jobs enqueue.
+3. Restart Solid Queue workers after deploy so they load updated configuration and code.
+4. Verify the dashboard loads, queue counts appear, and Mission Control links resolve when enabled.
+
+## Disaster Recovery Checklist
+
+- Restore the database, then replay any external feed data if necessary (SourceMonitor deduplicates using GUID/fingerprint).
+- Resume Solid Queue workers and monitor `fetch_failed_total` gauge for anomalies.
+- Rebuild assets if you deploy to ephemeral filesystems.
+
+Keep this guide alongside your platform runbooks so teams can confidently deploy and operate SourceMonitor in any environment.
+
+## Container Reference Stack
+
+The repository ships a reusable Docker stack under `examples/docker` that mirrors the recommended process model. It builds a Ruby 3.3 image with Node, mounts your generated example via `APP_PATH`, and launches three services (`web`, `worker`, `scheduler`) alongside Postgres and Redis. Use it to trial production settings locally or as a baseline for ECS/Kubernetes manifests.

data/docs/gh-cli-workflow.md

@@ -0,0 +1,44 @@
+# GitHub CLI Workflow for Branches & Pull Requests
+
+Use this checklist whenever you create or land a slice. It keeps `gh` commands predictable and avoids redo steps.
+
+## 1. Prep local branch
+1. `git checkout main && git fetch origin && git reset --hard origin/main`
+2. `git checkout -b <feature|bugfix>/<description>`
+3. Do the work, run tests/lint, then `git commit -am "scope: summary"`
+4. Push: `git push -u origin <branch>`
+
+## 2. Open a PR (as draft)
+1. `gh pr create --base main --head <branch> --title "scope: summary" --body "## Summary\n..." --draft`
+2. `gh pr view --json url,isDraft` (confirm association & draft status)
+
+## 3. Iterate on the PR
+1. Additional commits → `git push`
+2. `gh pr status` shows open PRs for the current branch.
+
+## 4. Move PR out of draft
+1. Check draft flag: `gh pr view <number> --json isDraft`
+2. If true, run `gh pr ready <number>`.
+3. Request reviews: `gh pr ready <number> --reviewer user1,user2`
+
+## 5. Merge steps
+1. Ensure PR is not draft & CI green: `gh pr checks <number>`
+2. Merge via GitHub (avoids local fast-forward issues):
+   ```bash
+   gh pr merge <number> --squash --delete-branch --auto
+   ```
+   `--auto` queues the merge once requirements are satisfied. Drop `--auto` only if you intend to merge immediately and have maintainer privileges.
+
+## 6. Local cleanup
+1. `git checkout main`
+2. `git pull --ff-only origin main`
+3. Delete local branch: `git branch -d <branch>`
+4. Optional: `git remote prune origin`
+
+## 7. Handy status commands
+- `gh pr status` – current branch PR state
+- `gh pr view <number> --json url,state,isDraft` – PR summary
+- `gh pr checks <number>` – CI state
+- `gh pr list --author @me` – open PRs authored by you
+
+Keep this document in `docs/gh-cli-workflow.md` and update it as our process evolves.

data/docs/installation.md

@@ -0,0 +1,144 @@
+# Installation Guide
+
+SourceMonitor installs like any other Rails engine, but it ships enough infrastructure (background jobs, realtime broadcasting, configuration DSL) that it is worth walking through the full setup. This guide assumes you are adding the engine to an existing Rails 8 host application.
+
+## Prerequisites
+
+- Ruby 3.4.4 (we recommend [rbenv](https://github.com/rbenv/rbenv) for local development: `rbenv install 3.4.4 && rbenv local 3.4.4`, but asdf, chruby, rvm, or container-managed Ruby all work equally well—choose whatever fits your environment)
+- Rails 8.0.2.1 or newer
+- PostgreSQL 13 or newer (the engine migrations rely on JSONB, SKIP LOCKED, and advisory locks)
+- Node.js 18+ and npm or Yarn for asset linting/builds
+- Solid Queue and Solid Cable gems available (they ship with Rails 8, but make sure they are not removed)
+- Optional: Mission Control Jobs if you plan to surface the dashboard shortcut, Redis if you intend to switch realtime adapters
+
+> **Command prefixes:** All commands below are shown without `rbenv exec`. If you use rbenv, prefix `bundle` and `bin/rails` commands with `rbenv exec`. For asdf, no prefix is needed. In Docker/container environments, run commands directly inside the container.
+
+## Quick Reference
+
+| Step | Command | Purpose |
+| --- | --- | --- |
+| 1 | `gem "source_monitor", github: "dchuk/source_monitor"` | Add the engine to your Gemfile |
+| 2 | `bundle install` | Install Ruby dependencies |
+| 3 | `bin/rails generate source_monitor:install --mount-path=/source_monitor` | Mount the engine and create the initializer |
+| 4 | `bin/rails railties:install:migrations FROM=source_monitor` | Copy engine migrations (idempotent) |
+| 5 | `bin/rails db:migrate` | Apply schema updates, including Solid Queue tables |
+| 6 | `bin/rails solid_queue:start` | Ensure jobs process via Solid Queue |
+| 7 | `bin/jobs --recurring_schedule_file=config/recurring.yml` | Start recurring scheduler (optional but recommended) |
+
+## 1. Add the Gem
+
+In your host application's `Gemfile`:
+
+```ruby
+gem "source_monitor", github: "dchuk/source_monitor"
+```
+
+Then install dependencies:
+
+```bash
+bundle install
+```
+
+If you vendor node tooling for linting/assets in the host app, run `npm install` as well.
+
+## 2. Run the Install Generator
+
+The generator mounts the engine and drops an initializer stub for you to configure.
+
+```bash
+bin/rails generate source_monitor:install --mount-path=/source_monitor
+```
+
+Key outputs:
+
+- Adds `mount SourceMonitor::Engine, at: "/source_monitor"` to your routes (change the path with `--mount-path`)
+- Creates `config/initializers/source_monitor.rb` with documented configuration defaults
+- The generator is idempotent: re-running it will detect existing mounts/initializers and skip overwriting your customizations
+- Update your navigation or admin layout to link to the mount path so teammates can discover the dashboard.
+
+## 3. Copy Engine Migrations
+
+SourceMonitor relies on several tables (sources, items, fetch logs, scrape logs, Solid Cable messages, retention helpers) plus an optional Solid Queue schema. Copy them into your host application with:
+
+```bash
+bin/rails railties:install:migrations FROM=source_monitor
+```
+
+This command is idempotent—re-run it when upgrading to pick up new migrations.
+
+## 4. Run Database Migrations
+
+Apply the new schema:
+
+```bash
+bin/rails db:migrate
+```
+
+If you prefer a dedicated database for Solid Queue, run `bin/rails solid_queue:install` beforehand and point the generated config at your queue database. Otherwise the engine-provided migration keeps Solid Queue tables in the primary database.
+
+> Tip: Solid Queue tables must be present before you start the dashboard. If your host app already ran `solid_queue:install`, delete the engine-provided migration before running `db:migrate` to avoid duplication.
+
+## 5. Wire Action Cable (if needed)
+
+SourceMonitor defaults to Solid Cable for realtime. Rails expects `ApplicationCable::Connection` and `ApplicationCable::Channel` to exist in your host app:
+
+```ruby
+# app/channels/application_cable/connection.rb
+module ApplicationCable
+  class Connection < ActionCable::Connection::Base; end
+end
+
+# app/channels/application_cable/channel.rb
+module ApplicationCable
+  class Channel < ActionCable::Channel::Base; end
+end
+```
+
+Verify `config/cable.yml` allows the adapter you choose. To switch to Redis, update `config/initializers/source_monitor.rb` with `config.realtime.adapter = :redis` and provide `config.realtime.redis_url`.
+
+## 6. Configure Background Workers
+
+Solid Queue becomes the default Active Job adapter when the host app still uses `:async`. Keep at least one worker process running:
+
+```bash
+# long-running process
+bin/rails solid_queue:start
+```
+
+SourceMonitor respects explicit queue adapter overrides. If your host sets `config.active_job.queue_adapter` (for example, to `:inline` or `:sidekiq`), the engine leaves that configuration in place.
+
+For recurring schedules, add a process that runs the Solid Queue CLI with the engine's schedule file:
+
+```bash
+bin/jobs --recurring_schedule_file=config/recurring.yml
+```
+
+Set `SOLID_QUEUE_SKIP_RECURRING=true` in environments that manage recurring tasks elsewhere.
+
+## 7. Review Configuration Defaults
+
+Open `config/initializers/source_monitor.rb` and adjust queue namespaces, HTTP timeouts, scraping adapters, retention limits, authentication hooks, and Mission Control integration to match your environment. The [configuration reference](configuration.md) covers every option with examples.
+
+## 8. Verify the Installation
+
+1. Start your Rails server: `bin/rails server`
+2. Ensure a Solid Queue worker is running (see step 6)
+3. Visit the mount path (`/source_monitor` by default)
+4. Create a source and trigger `Fetch Now`—watch fetch logs appear and dashboard metrics update
+
+If you encounter issues, consult the [troubleshooting guide](troubleshooting.md).
+
+## Next Steps
+
+- Explore the admin UI and dashboards
+- Integrate custom scraper adapters or item processors via `SourceMonitor.configure`
+- Set up monitoring for the Solid Queue queues using Mission Control or your preferred observability stack
+
+## Host Compatibility Matrix
+
+| Host Scenario | Status | Notes |
+| --- | --- | --- |
+| Rails 8 full-stack app | ✅ Supported | Default generator flow (mount + initializer) |
+| Rails 8 API-only app (`--api`) | ✅ Supported | Generator mounts engine; ensure you provide a UI entry point if needed |
+| Dedicated Solid Queue database | ✅ Supported | Run `bin/rails solid_queue:install` in the host app before copying SourceMonitor migrations |
+| Redis-backed Action Cable | ✅ Supported | Set `config.realtime.adapter = :redis` and provide `config.realtime.redis_url`; existing `config/cable.yml` entries are preserved |

data/docs/troubleshooting.md

@@ -0,0 +1,76 @@
+# Troubleshooting Guide
+
+This guide lists common issues you might encounter while installing, upgrading, or operating SourceMonitor, along with concrete steps to resolve them.
+
+## 1. Mount Path Returns 404
+
+- Ensure the install generator added `mount SourceMonitor::Engine, at: "/source_monitor"` (or your custom path) inside the host `config/routes.rb`.
+- Restart the Rails server after modifying routes.
+- Confirm your host application routes are reloaded (run `rbenv exec bin/rails routes | grep source_monitor`).
+
+## 2. Migrations or Solid Queue Tables Are Missing
+
+- Run `rbenv exec bin/rails railties:install:migrations FROM=source_monitor` followed by `rbenv exec bin/rails db:migrate`.
+- If you see duplicate migration timestamps, remove the older copy before rerunning the installer.
+- For Solid Queue tables, verify the host database contains the `solid_queue_*` tables shipped with the engine migration or run `rbenv exec bin/rails solid_queue:install` for a dedicated queue database.
+
+## 3. Dashboard Metrics Show "Unavailable"
+
+- Solid Queue metrics require the `solid_queue` tables—see issue 2 above.
+- Ensure at least one Solid Queue worker is running; the dashboard reads visibility data via `SourceMonitor::Jobs::Visibility`.
+- When using mission control integration, keep `config.mission_control_dashboard_path` pointing at a valid route helper; otherwise the dashboard hides the link.
+
+## 4. Realtime Updates Do Not Stream
+
+- Confirm Action Cable is mounted and `ApplicationCable` classes exist (see installation guide).
+- In production, verify WebSocket proxy settings allow the `/cable` endpoint.
+- When switching to Redis, add `config.realtime.adapter = :redis` and `config.realtime.redis_url` in the initializer, then restart web and worker processes.
+- For Solid Cable, check that the `solid_cable_messages` table exists and that no other process clears it unexpectedly.
+
+## 5. Fetch Jobs Keep Failing
+
+- Review the most recent fetch log entry for the source; it stores the HTTP status, error class, and error message.
+- Increase `config.http.timeout` or `config.http.retry_max` if the feed is slow or prone to transient errors.
+- Supply custom headers or basic auth credentials via the source form when feeds require authentication.
+- Check for TLS issues on self-signed feeds; you may need to configure Faraday with custom SSL options.
+
+## 6. Scraping Returns "Failed"
+
+- Confirm the source has scraping enabled and the configured adapter exists.
+- Override selectors in the source's scrape settings if the default Readability extraction misses key elements.
+- Inspect the scrape log to see the adapter status and content length. Logs store the HTTP status and any exception raised by the adapter.
+- Retry manually from the item detail page after fixing selectors.
+
+## 7. Cleanup Rake Tasks Fail
+
+- Pass numeric values for `FETCH_LOG_DAYS` or `SCRAPE_LOG_DAYS` environment variables (e.g., `FETCH_LOG_DAYS=30`).
+- Ensure workers or the console environment have permission to soft delete (`SOFT_DELETE=true`) if you expect tombstones.
+- If job classes cannot load, verify `SourceMonitor.configure` ran before calling `rake source_monitor:cleanup:*`.
+
+## 8. Test Suite Cannot Launch a Browser
+
+- System tests rely on Selenium + Chrome. Install Chrome/Chromium and set `SELENIUM_CHROME_BINARY` if the binary lives in a non-standard path.
+- You can run `rbenv exec bin/test-coverage --verbose` to inspect failures with additional logging.
+
+## 9. Mission Control Jobs Link Returns 404
+
+- Mount `MissionControl::Jobs::Engine` in your host routes (for example, `mount MissionControl::Jobs::Engine, at: "/mission_control"`).
+- Keep `config.mission_control_enabled = true` **and** `config.mission_control_dashboard_path` pointing at that mounted route helper. Call `SourceMonitor.mission_control_dashboard_path` in the Rails console to confirm it resolves.
+- When hosting Mission Control in a separate app, provide a full URL instead of a route helper and ensure CORS/WebSocket settings allow the dashboard iframe.
+
+## 10. Tailwind Build Fails or Admin UI Loads Without Styles
+
+- Running `test/dummy/bin/dev` before configuring the bundling pipeline will serve the admin UI without Tailwind styles or Stimulus behaviours. This happens because the engine no longer ships precompiled assets; see `.ai/engine-asset-configuration.md:11-44` for the required npm setup.
+- Fix by running `npm install` followed by `npm run build` inside the engine root so that `app/assets/builds/source_monitor/application.css` and `application.js` exist. The Rake task `app:source_monitor:assets:build` wraps the same scripts for CI usage.
+- When the UI is still unstyled, confirm the dummy app can read the namespaced asset directories noted in `.ai/engine-asset-configuration.md:32-44` and restart `bin/dev` so the CSS/JS watchers reconnect.
+
+## Still Stuck?
+
+Collect the following and open an issue or start a discussion:
+
+- Engine version (`SourceMonitor::VERSION`)
+- Host Rails version and Ruby version
+- Relevant configuration snippet from `config/initializers/source_monitor.rb`
+- Recent fetch/scrape log entries or stack traces
+
+We track known issues and roadmap items in `.ai/tasks.md`; check the next slice to see if your problem is already scheduled.

data/eslint.config.mjs

@@ -0,0 +1,27 @@
+import js from "@eslint/js";
+
+export default [
+  {
+    ignores: [
+      "node_modules/**",
+      "vendor/assets/**",
+      "app/assets/builds/**"
+    ]
+  },
+  js.configs.recommended,
+  {
+    files: [ "app/assets/javascripts/**/*.js" ],
+    languageOptions: {
+      sourceType: "module",
+      globals: {
+        window: "readonly",
+        document: "readonly",
+        CustomEvent: "readonly"
+      }
+    },
+    rules: {
+      "no-unused-vars": [ "error", { args: "none", ignoreRestSiblings: true } ],
+      "no-console": [ "warn", { allow: [ "warn", "error" ] } ]
+    }
+  }
+];

data/lib/generators/source_monitor/install/install_generator.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/base"
+
+module SourceMonitor
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      class_option :mount_path,
+        type: :string,
+        default: "/source_monitor",
+        desc: "Path the engine will mount at inside the host application's routes"
+
+      def add_routes_mount
+        mount_path = normalized_mount_path
+        return if engine_already_mounted?(mount_path)
+
+        route %(mount SourceMonitor::Engine, at: "#{mount_path}")
+      end
+
+      def create_initializer
+        initializer_path = "config/initializers/source_monitor.rb"
+        destination = File.join(destination_root, initializer_path)
+
+        if File.exist?(destination)
+          say_status :skip, initializer_path, :yellow
+          return
+        end
+
+        template "source_monitor.rb.tt", initializer_path
+      end
+
+      def print_next_steps
+        say_status :info,
+          "Next steps: review docs/installation.md for install walkthroughs and docs/troubleshooting.md for common fixes.",
+          :green
+      end
+
+      private
+
+      def engine_already_mounted?(mount_path)
+        routes_path = File.join(destination_root, "config/routes.rb")
+        return false unless File.exist?(routes_path)
+
+        routes_content = File.read(routes_path)
+        routes_content.include?("mount SourceMonitor::Engine, at: \"#{mount_path}\"") ||
+          routes_content.include?("mount SourceMonitor::Engine")
+      end
+
+      def normalized_mount_path
+        raw_path = options.key?(:mount_path) ? options[:mount_path] : "/source_monitor"
+        path = (raw_path && !raw_path.strip.empty?) ? raw_path.strip : "/source_monitor"
+        path.start_with?("/") ? path : "/#{path}"
+      end
+    end
+  end
+end