source_monitor 0.1.1 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 90320cedd371ba574e94a72530899c9c186ee1e90f6b27dd221dca6e5831a626
-  data.tar.gz: 67db4a9fb817b386cbdffd675c44ec8e32cfa806ad4be284e95a80756180cf82
+  metadata.gz: 3831f3ec3fa841e1e15fb16fc7ab780177d35353fce96975338c2e7aa129bd9f
+  data.tar.gz: ac970a0b226c1b8c1c9a266a892754941d92173b8753ac37d2a4cd89f1895fd9
 SHA512:
-  metadata.gz: 1f63265ddc4b7f5a2989a91c04b61645e573ed71b68aa0348197f0f0afb34d8f36b9d8127613d7c63b25df39d7e20603c31cfac445b9f431188bdaf4724f46ee
-  data.tar.gz: 6ceed78ef9db2de1d61904745c326016529a06f5ad7c56c07f0428bebf4d6c237a769532ec0e6ca5be795970833389e67c9a503b47283e48aa7af524a321d969
+  metadata.gz: 46dc5a034c7b7c294877d124a59b632aec2cfbc420544835b06d0809a7e8d574930ffc86cd5bfbceb436516a0bc88b55ffdb34a38a0bb873ae3892bb3615b503
+  data.tar.gz: 542c9c82713daadb6f0cfa4151ab99df55440a8e86a553ee46b2223806096f6d36d1e1987890f50aaad83132498c674793cae11c11752a897daca6c36efb1a94

data/CHANGELOG.md

@@ -15,6 +15,23 @@ All notable changes to this project are documented below. The format follows [Ke
 
 - No unreleased changes yet.
 
+## [0.1.2] - 2025-11-13
+
+### Added
+
+- Guided setup workflow (`bin/source_monitor install`) with dependency checks, Gemfile automation, migration deduplication, initializer patching, and Devise prompt support.
+- Reusable verification tooling (`bin/source_monitor verify` / `bin/rails source_monitor:setup:verify`) plus Solid Queue/Action Cable verifiers, JSON output, and optional telemetry logging.
+- Fresh documentation (`docs/setup.md`, rollout checklist in `docs/deployment.md`, validation log) outlining prerequisites, rollback steps, and CI adoption guidance.
+
+### Fixed
+
+- Enforced coverage/diff checks in CI, added `bin/check-setup-tests`, and expanded test suites so new setup workflow files stay covered.
+- Hardened host harness env defaults to avoid `root` Postgres role errors during CI/app template generation.
+
+### Testing
+
+- Full suite (`bin/rails test`), `bin/test-coverage`, `bin/check-diff-coverage`, and `bin/rubocop` all pass on Ruby 3.4.4.
+
 ## [0.1.1] - 2025-11-09
 
 ### Changed

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    source_monitor (0.1.1)
+    source_monitor (0.1.2)
       cssbundling-rails (~> 1.4)
       faraday (~> 2.9)
       faraday-follow_redirects (~> 0.4)

data/README.md

@@ -20,25 +20,29 @@ SourceMonitor is a production-ready Rails 8 mountable engine for ingesting, norm
 
 ## Quick Start (Host Application)
 
-> **Note on commands:** The examples below use bare `bundle` and `bin/rails` commands. If you use rbenv, prefix them with `rbenv exec` (e.g., `rbenv exec bundle install`). For Docker/container environments, run commands directly or via your container runtime. Adjust to match your Ruby version management approach.
+> **Command prefixes:** Examples below show bare `bundle`, `bin/rails`, and `bin/source_monitor`. If you use rbenv/asdf or containerized tooling, prefix/adjust commands accordingly so they run inside your Ruby environment.
 
-### Initial Install
-1. Add `gem "source_monitor", github: "dchuk/source_monitor"` to your Gemfile and run `bundle install`.
-2. Install the engine: `bin/rails generate source_monitor:install --mount-path=/source_monitor` (updates routes, drops the initializer, prints doc links).
-3. Copy migrations: `bin/rails railties:install:migrations FROM=source_monitor`.
-4. Apply migrations: `bin/rails db:migrate` (creates sources/items/logs tables, Solid Cable messages, and Solid Queue schema when required).
-5. Install frontend tooling if you plan to extend engine assets: `npm install`.
-6. Start background workers: `bin/rails solid_queue:start` (or your preferred process manager).
-7. Boot your app and visit the mount path you chose (default `/source_monitor`) to explore the dashboard and confirm Solid Queue metrics render.
+### Recommended: Guided Workflow
+1. **Optional prerequisite check:** `bin/rails source_monitor:setup:check`
+2. **Run the guided installer:** `bin/source_monitor install --yes`
+   - Prompts for the mount path (default `/source_monitor`), adds the gem entry when missing, runs `bundle install`, `npm install` (when `package.json` exists), copies/deduplicates migrations, patches the initializer, and runs verification.
+3. **Start workers / scheduler:** `bin/rails solid_queue:start` and, if you use recurring jobs, `bin/jobs --recurring_schedule_file=config/recurring.yml`.
+4. **Verify anytime:** `bin/source_monitor verify` (also exposed as `bin/rails source_monitor:setup:verify`). The command prints a human summary plus JSON so CI can gate on Solid Queue and Action Cable health.
+5. **Visit the dashboard** at the chosen mount path and trigger “Fetch Now” on a source to confirm everything is wired.
 
-Detailed instructions, optional flags, and verification steps live in [docs/installation.md](docs/installation.md), with troubleshooting advice in [docs/troubleshooting.md](docs/troubleshooting.md).
+See [docs/setup.md](docs/setup.md) for the full workflow (prereq table, rollback steps, telemetry flag, Devise system test template).
+
+### Manual Install (Advanced)
+Prefer explicit Rails generator steps or need to customize each phase? The same document covers a full **Manual Installation** section so you can copy/paste each command into bespoke pipelines.
+
+Troubleshooting advice lives in [docs/troubleshooting.md](docs/troubleshooting.md).
 
 ### Upgrading SourceMonitor
 1. Bump the gem version in your host `Gemfile` and run `bundle install` (or `bundle update source_monitor` when targeting a specific release).
 2. Re-run `bin/rails railties:install:migrations FROM=source_monitor` and then `bin/rails db:migrate` to pick up schema changes.
 3. Compare your `config/initializers/source_monitor.rb` against the newly generated template for configuration diffs (new queue knobs, HTTP options, etc.).
 4. Review release notes for optional integrations—when enabling Mission Control, ensure `mission_control-jobs` stays mounted and linked via `config.mission_control_dashboard_path`.
-5. Smoke test Solid Queue workers, Action Cable, and admin UI flows after the upgrade.
+5. Smoke test Solid Queue workers, Action Cable, and admin UI flows after the upgrade, and run `bin/source_monitor verify` so CI/deploys confirm workers/cable health before rollout.
 
 ## Example Applications
 - `examples/basic_host/template.rb` – Minimal host that seeds a Rails blog source and redirects `/` to the dashboard.

data/config/coverage_baseline.json

@@ -132,21 +132,21 @@
     82
   ],
   "lib/source_monitor.rb": [
-    28,
-    97,
-    113,
-    117,
-    121,
-    125,
-    126,
-    127,
-    129,
-    131,
-    137,
-    141,
-    143,
+    29,
+    116,
+    132,
+    136,
+    140,
     144,
-    146
+    145,
+    146,
+    148,
+    150,
+    156,
+    160,
+    162,
+    163,
+    165
   ],
   "lib/source_monitor/analytics/source_activity_rates.rb": [
     44
@@ -458,28 +458,28 @@
     121
   ],
   "lib/source_monitor/engine.rb": [
-    47,
-    51,
-    52,
+    49,
     53,
     54,
-    59,
-    60,
+    55,
+    56,
     61,
     62,
     63,
     64,
     65,
-    68,
-    69,
+    66,
+    67,
     70,
-    73,
-    74,
+    71,
+    72,
+    75,
     76,
-    77,
-    96,
-    97,
-    99
+    78,
+    79,
+    98,
+    99,
+    101
   ],
   "lib/source_monitor/events.rb": [
     9,
@@ -823,10 +823,6 @@
     32,
     33,
     37,
-    41,
-    42,
-    45,
-    46,
     50,
     52,
     53,
@@ -843,8 +839,6 @@
     69,
     70,
     71,
-    77,
-    79,
     84,
     85,
     89,
@@ -932,8 +926,7 @@
     35,
     39,
     43,
-    44,
-    49
+    44
   ],
   "lib/source_monitor/items/item_creator.rb": [
     16,
@@ -1268,6 +1261,13 @@
     77,
     80
   ],
+  "lib/source_monitor/jobs/fetch_failure_subscriber.rb": [
+    62,
+    98,
+    111,
+    120,
+    122
+  ],
   "lib/source_monitor/jobs/solid_queue_metrics.rb": [
     20,
     27,
@@ -1330,24 +1330,9 @@
     43,
     48,
     49,
-    54,
     56,
-    60,
-    64,
-    65,
     70,
     71,
-    76,
-    80,
-    85,
-    86,
-    88,
-    90,
-    91,
-    92,
-    93,
-    94,
-    95,
     102,
     103,
     105,
@@ -1397,13 +1382,8 @@
     117
   ],
   "lib/source_monitor/metrics.rb": [
-    10,
-    14,
     18,
     22,
-    26,
-    30,
-    31,
     38,
     39,
     43,
@@ -1412,16 +1392,7 @@
     46,
     48,
     51,
-    52,
-    56,
-    57,
-    59,
-    60,
-    61,
-    62,
-    63,
-    70,
-    74
+    52
   ],
   "lib/source_monitor/model_extensions.rb": [
     53,
@@ -1465,11 +1436,6 @@
     93
   ],
   "lib/source_monitor/realtime/broadcaster.rb": [
-    33,
-    34,
-    35,
-    37,
-    38,
     42,
     43,
     44,
@@ -1511,20 +1477,11 @@
     148,
     149,
     150,
-    155,
     167,
-    171,
-    180,
     187,
-    191,
-    193,
     195,
-    199,
-    202,
-    204,
     206,
     210,
-    214,
     231,
     233
   ],
@@ -1534,34 +1491,6 @@
   "lib/source_monitor/release/runner.rb": [
     68
   ],
-  "lib/source_monitor/scheduler.rb": [
-    12,
-    16,
-    17,
-    21,
-    23,
-    24,
-    25,
-    26,
-    28,
-    29,
-    32,
-    34,
-    43,
-    45,
-    46,
-    47,
-    48,
-    51,
-    55,
-    63,
-    64,
-    68,
-    70,
-    71,
-    72,
-    74
-  ],
   "lib/source_monitor/scrapers/base.rb": [
     29,
     33,
@@ -1991,6 +1920,293 @@
     19,
     23
   ],
+  "lib/source_monitor/setup/bundle_installer.rb": [
+    5,
+    9
+  ],
+  "lib/source_monitor/setup/cli.rb": [
+    11,
+    14,
+    15,
+    20,
+    21,
+    27,
+    28,
+    29,
+    33,
+    37,
+    41,
+    43,
+    47
+  ],
+  "lib/source_monitor/setup/dependency_checker.rb": [
+    23,
+    27,
+    31,
+    35,
+    43,
+    47,
+    48,
+    50,
+    54,
+    58,
+    62,
+    66,
+    70,
+    75,
+    79,
+    85,
+    86,
+    87,
+    89,
+    100,
+    102,
+    106,
+    107,
+    109,
+    113,
+    114,
+    116,
+    121,
+    126,
+    130,
+    137,
+    144,
+    151,
+    158
+  ],
+  "lib/source_monitor/setup/detectors.rb": [
+    7,
+    11,
+    13,
+    17,
+    18,
+    20,
+    22,
+    26,
+    27,
+    29,
+    31,
+    35,
+    36,
+    42,
+    44,
+    46,
+    50,
+    52,
+    55,
+    59,
+    61,
+    63
+  ],
+  "lib/source_monitor/setup/gemfile_editor.rb": [
+    9,
+    13,
+    15,
+    16,
+    18,
+    19,
+    25
+  ],
+  "lib/source_monitor/setup/initializer_patcher.rb": [
+    9,
+    13,
+    15,
+    16,
+    17,
+    19,
+    20,
+    21,
+    25,
+    27,
+    28,
+    30,
+    31,
+    32,
+    34,
+    37,
+    38,
+    46,
+    50
+  ],
+  "lib/source_monitor/setup/install_generator.rb": [
+    5,
+    9
+  ],
+  "lib/source_monitor/setup/migration_installer.rb": [
+    10,
+    11,
+    15,
+    16,
+    17,
+    25,
+    29,
+    30,
+    32,
+    36
+  ],
+  "lib/source_monitor/setup/node_installer.rb": [
+    7,
+    8,
+    12,
+    14,
+    15,
+    23
+  ],
+  "lib/source_monitor/setup/prompter.rb": [
+    7,
+    8,
+    12,
+    14,
+    15,
+    16,
+    20,
+    22,
+    23,
+    24,
+    26
+  ],
+  "lib/source_monitor/setup/requirements.rb": [
+    8,
+    12,
+    16,
+    18,
+    19,
+    21,
+    25,
+    26,
+    28,
+    36,
+    40,
+    44,
+    46
+  ],
+  "lib/source_monitor/setup/shell_runner.rb": [
+    7,
+    8,
+    10
+  ],
+  "lib/source_monitor/setup/verification/action_cable_verifier.rb": [
+    6,
+    7,
+    8,
+    12,
+    14,
+    16,
+    18,
+    21,
+    29,
+    33,
+    35,
+    39,
+    41,
+    45,
+    46,
+    49,
+    50,
+    53,
+    57,
+    58,
+    59,
+    62,
+    66,
+    70,
+    74
+  ],
+  "lib/source_monitor/setup/verification/printer.rb": [
+    6,
+    10,
+    11,
+    12,
+    13,
+    15
+  ],
+  "lib/source_monitor/setup/verification/result.rb": [
+    15,
+    19,
+    23,
+    28,
+    41,
+    45,
+    46,
+    48,
+    52,
+    57,
+    63
+  ],
+  "lib/source_monitor/setup/verification/runner.rb": [
+    6,
+    10,
+    11,
+    19
+  ],
+  "lib/source_monitor/setup/verification/solid_queue_verifier.rb": [
+    8,
+    9,
+    10,
+    14,
+    15,
+    17,
+    19,
+    20,
+    22,
+    25,
+    33,
+    37,
+    39,
+    43,
+    45,
+    49,
+    50,
+    54,
+    58,
+    62,
+    66,
+    70
+  ],
+  "lib/source_monitor/setup/verification/telemetry_logger.rb": [
+    9,
+    13,
+    14,
+    15,
+    24,
+    25,
+    27
+  ],
+  "lib/source_monitor/setup/workflow.rb": [
+    12,
+    13,
+    19,
+    20,
+    22,
+    38,
+    39,
+    40,
+    41,
+    42,
+    43,
+    44,
+    45,
+    46,
+    47,
+    51,
+    52,
+    54,
+    56,
+    57,
+    58,
+    59,
+    60,
+    61,
+    63,
+    64,
+    67,
+    84,
+    88,
+    92,
+    93,
+    95
+  ],
   "lib/source_monitor/turbo_streams/stream_responder.rb": [
     35,
     36,
@@ -2006,5 +2222,32 @@
     1,
     2,
     3
+  ],
+  "lib/tasks/source_monitor_assets.rake": [
+    7,
+    12
+  ],
+  "lib/tasks/source_monitor_setup.rake": [
+    31
+  ],
+  "lib/tasks/source_monitor_tasks.rake": [
+    5,
+    7,
+    8,
+    9,
+    10,
+    13,
+    14,
+    17,
+    22,
+    23,
+    24,
+    26
+  ],
+  "lib/tasks/test_smoke.rake": [
+    6,
+    7,
+    8,
+    10
   ]
 }

data/docs/deployment.md

@@ -61,3 +61,11 @@ Keep this guide alongside your platform runbooks so teams can confidently deploy
 ## Container Reference Stack
 
 The repository ships a reusable Docker stack under `examples/docker` that mirrors the recommended process model. It builds a Ruby 3.3 image with Node, mounts your generated example via `APP_PATH`, and launches three services (`web`, `worker`, `scheduler`) alongside Postgres and Redis. Use it to trial production settings locally or as a baseline for ECS/Kubernetes manifests.
+
+## Setup Workflow Rollout Checklist
+
+1. **Adopt the CLI** – add `bin/source_monitor install --yes` (or an equivalent rake task wrapper) to your internal onboarding docs for both greenfield apps and existing hosts. Pair it with `bin/source_monitor verify` so operators can re-check queue/Action Cable health after deployments.
+2. **CI Gate** – extend your deploy pipeline to invoke `bin/source_monitor verify` right after migrations. Fail the build when the command exits non-zero and surface the JSON blob produced by the printer for diagnostics. This step replaces ad-hoc “did you start Solid Queue?” questions before merging.
+3. **Telemetry Opt-In** – set `SOURCE_MONITOR_SETUP_TELEMETRY=true` in staging/pre-prod so setup runs append logs to `log/source_monitor_setup.log`. Ship the file as a build artifact for debugging failed installs.
+4. **Rollback Ready** – reference `docs/setup.md#rollback-steps` in your release checklist so anyone piloting the workflow knows how to revert gem/routes/initializer changes if the experiment needs to pause.
+5. **Worker Coverage** – ensure at least one Solid Queue worker is running (or emits a heartbeat) before moving the PR out of draft; otherwise verification exits with a warning and CI will fail.

data/docs/gh-cli-workflow.md

@@ -6,7 +6,11 @@ Use this checklist whenever you create or land a slice. It keeps `gh` commands p
 1. `git checkout main && git fetch origin && git reset --hard origin/main`
 2. `git checkout -b <feature|bugfix>/<description>`
 3. Do the work, run tests/lint, then `git commit -am "scope: summary"`
-4. Push: `git push -u origin <branch>`
+4. Run the coverage gate locally before pushing:
+   - `bin/test-coverage`
+   - `bin/check-diff-coverage`
+   - If new code is intentionally uncovered, refresh the baseline with `bin/update-coverage-baseline` and commit the updated `config/coverage_baseline.json`.
+5. Push: `git push -u origin <branch>`
 
 ## 2. Open a PR (as draft)
 1. `gh pr create --base main --head <branch> --title "scope: summary" --body "## Summary\n..." --draft`