source_monitor 0.1.3 → 0.2.0

data/tasks/completed/engine-asset-configuration.md

@@ -0,0 +1,203 @@
+# Rails Engine Asset Pipeline Guide
+
+## Supporting Both Sprockets and Propshaft
+
+### Core Principle
+
+Support both asset pipelines to maximize compatibility with parent Rails apps. Propshaft is the modern standard, but Sprockets remains widely used in existing applications.
+
+---
+
+## Setup: Asset Bundling
+
+Use `cssbundling-rails` and `jsbundling-rails` for maximum flexibility.
+
+```ruby
+# Add to gemspec
+gem.add_dependency "cssbundling-rails"
+gem.add_dependency "jsbundling-rails"
+```
+
+**Note:** Generators don't work inside engines. Install in a dummy app first, then manually move files to the engine.
+
+```bash
+cd test/dummy
+rails javascript:install:esbuild
+rails css:install:tailwind
+# Move generated files back to engine root
+```
+
+---
+
+## Asset Organization
+
+### Directory Structure
+
+```
+app/assets/
+  ├── builds/ENGINE_NAME/     # Bundled CSS/JS (auto-exposed)
+  ├── images/ENGINE_NAME/     # Image files
+  ├── svgs/ENGINE_NAME/       # SVG files
+  └── config/                 # Manifest files (optional)
+```
+
+**Rule:** Always namespace asset directories with your engine name to prevent conflicts.
+
+---
+
+## Configuring engine.rb
+
+### Basic Setup (Works for Both Pipelines)
+
+```ruby
+module YourEngine
+  class Engine < ::Rails::Engine
+    isolate_namespace YourEngine
+
+    initializer "your_engine.assets" do |app|
+      if app.config.respond_to?(:assets)
+        # Expose asset directories
+        app.config.assets.paths << Engine.root.join("app", "assets", "builds").to_s
+        app.config.assets.paths << Engine.root.join("app", "assets", "images").to_s
+        app.config.assets.paths << Engine.root.join("app", "assets", "svgs").to_s
+      end
+    end
+  end
+end
+```
+
+This configuration:
+
+- Exposes the `builds` directory (bundled assets)
+- Exposes the `images` directory
+- Exposes the `svgs` directory
+- Works automatically with Propshaft
+- Requires additional configuration for Sprockets (see below)
+
+---
+
+## Sprockets-Specific Configuration
+
+For Sprockets to properly precompile assets referenced in CSS (via `url()`), use one of these approaches:
+
+### Option 1: Programmatic Precompile (Recommended)
+
+```ruby
+initializer "your_engine.assets" do |app|
+  if app.config.respond_to?(:assets)
+    # Add paths
+    app.config.assets.paths << Engine.root.join("app", "assets", "builds").to_s
+    app.config.assets.paths << Engine.root.join("app", "assets", "images").to_s
+    app.config.assets.paths << Engine.root.join("app", "assets", "svgs").to_s
+
+    # Configure precompilation for Sprockets
+    asset_paths = [
+      ["app", "assets", "images"],
+      ["app", "assets", "images", "your_engine"],
+      ["app", "assets", "svgs"],
+    ]
+
+    paths_to_precompile = asset_paths.flat_map do |path|
+      Dir[Engine.root.join(*path, "**", "*")].filter_map do |file|
+        next unless File.file?(file)
+        Pathname.new(file).relative_path_from(Engine.root.join(*path)).to_s
+      end
+    end
+
+    app.config.assets.precompile += paths_to_precompile
+  end
+end
+```
+
+**Critical:** Paths must be relative from the asset directory (e.g., `your_engine/logo.png`, not `app/assets/images/your_engine/logo.png`).
+
+### Option 2: Manual Precompile Array
+
+```ruby
+initializer "your_engine.assets" do |app|
+  if app.config.respond_to?(:assets)
+    app.config.assets.paths << Engine.root.join("app", "assets", "images").to_s
+    app.config.assets.precompile += %w[ your_engine/logo.png your_engine/icon.svg ]
+  end
+end
+```
+
+### Option 3: Manifest File
+
+Create `app/assets/config/your_engine_manifest.js`:
+
+```javascript
+//= link_tree ../builds
+//= link_tree ../images
+//= link_tree ../svgs
+```
+
+Then in `engine.rb`:
+
+```ruby
+initializer "your_engine.assets" do |app|
+  if defined?(::Sprockets)
+    app.config.assets.precompile += %w[your_engine_manifest.js]
+  end
+end
+```
+
+---
+
+## Using Assets
+
+### In Views
+
+```erb
+<%= image_tag 'your_engine/logo.png' %>
+<%= image_tag 'your_engine/icon.svg' %>
+```
+
+### In CSS
+
+```css
+.logo {
+	background-image: url("your_engine/logo.png");
+}
+```
+
+**Propshaft behavior:** Automatically finds assets and adds digest hashes.
+**Sprockets behavior:** Requires files to be in the precompile array.
+
+---
+
+## Key Differences Between Pipelines
+
+| Aspect                   | Propshaft                | Sprockets                         |
+| ------------------------ | ------------------------ | --------------------------------- |
+| Asset finding            | Automatic from paths     | Requires precompile configuration |
+| CSS url() references     | Works automatically      | Needs explicit precompile setup   |
+| Configuration complexity | Minimal                  | More involved                     |
+| Prefix flexibility       | Can work with or without | Requires namespace prefix         |
+
+---
+
+## Testing Checklist
+
+- [ ] Test in a Sprockets-based app
+- [ ] Test in a Propshaft-based app
+- [ ] Verify `image_tag` works in views
+- [ ] Verify `url()` references work in CSS
+- [ ] Confirm digest hashes are applied in production
+- [ ] Check for asset path conflicts with parent app
+
+---
+
+## Quick Decision Tree
+
+**Q: Do bundled assets in `app/assets/builds` need configuration?**
+A: No, they're auto-exposed to both pipelines.
+
+**Q: Do images/SVGs need configuration?**
+A: Yes. Add paths with `app.config.assets.paths <<`. For Sprockets, also add to precompile array.
+
+**Q: Are CSS `url()` references working?**
+A: Propshaft: automatic. Sprockets: add files to precompile array.
+
+**Q: Should I namespace asset directories?**
+A: Yes, always use `your_engine/` prefix to prevent conflicts.

data/tasks/completed/opml-import-wizard/opml-import-wizard-product-brief.md

@@ -0,0 +1,58 @@
+## Objective
+
+Enable users to bulk import feed sources via a wizard workflow that guides them through uploading an OPML file, previewing and selecting sources, running health checks, and configuring settings before importing—all from the Sources index page.
+
+## Project Goal
+
+Allow users to successfully import at least 90% of valid, healthy feeds from an OPML file in a single workflow, with less than 5% error rate due to invalid or duplicate sources.
+
+## User Journey
+
+1. **Entry Point**
+    - User navigates to the Sources index page.
+    - User clicks the new "Import OPML" button.
+
+2. **Step 1: Upload OPML**
+    - User is presented with a wizard step to upload an OPML file.
+    - The system accepts any OPML file and parses it, handling errors gracefully.
+
+3. **Step 2: Preview & Select Sources**
+    - User sees a table listing all parsed sources from the OPML file.
+    - The table includes a column indicating whether each source is "Already Imported."
+    - Filters at the top allow toggling between "All," "New Sources," and "Existing Sources."
+    - User can unselect any feeds they do not wish to import.
+    - Selections persist as the user navigates between wizard steps.
+
+4. **Step 3: Health Check**
+    - The system runs a health check on each selected source’s feed URL.
+    - Results are displayed in a table with a green check for healthy sources and a red X for unhealthy sources.
+    - Unhealthy sources are unselected by default, but users can re-select them and proceed if desired.
+
+5. **Step 4: Configure Settings**
+    - User is presented with source settings (same options as single feed creation) to apply uniformly to all selected sources.
+    - No per-feed override; settings apply to all selected feeds.
+
+6. **Step 5: Confirm & Import**
+    - User reviews the list of sources to be imported and confirms.
+    - Import is initiated as a background job.
+    - User is redirected to the Sources index page with a static confirmation message listing imported sources.
+    - Fetching results and source health updates are shown in real-time via existing UI features.
+
+## Features 
+
+### In Scope
+- OPML File Upload
+- Wizard Workflow (Multi-step)
+- Source Preview Table with Filters
+- Duplicate Source Detection
+- Source Selection Persistence
+- Health Check for Feeds
+- Bulk Source Settings Configuration
+- Background Job for Import
+- Static Import Confirmation
+
+### Out of Scope
+- Per-feed Settings Overrides
+- Real-time Import Progress Bar
+- OPML File Size or Structure Restrictions
+- Advanced Error Recovery for Import Failures

data/tasks/completed/opml-import-wizard/opml-import-wizard-tech-brief.md

@@ -0,0 +1,75 @@
+### Technical Summary
+
+Implement a multi-step OPML import wizard accessible from the Sources index page, enabling admin users to upload an OPML file, preview and select sources, run health checks, configure bulk settings, and confirm import, with results stored for later viewing as import history.  
+This extends SourceMonitor’s existing sources management, background job orchestration (Solid Queue), Turbo Streams real-time UI, and follows established Rails engine, controller, and asset pipeline patterns.
+
+### System Design
+
+#### **Frontend**
+- Add "Import OPML" button to the Sources index page, linking to a dedicated wizard route.
+- Wizard UI uses a sidebar for step navigation (Upload, Preview, Health Check, Configure, Confirm), with current step highlighted.
+- Each step is rendered as a separate view, using Turbo Frames for partial updates and state transitions.
+- File upload uses standard Rails form helpers, with immediate synchronous parsing and error feedback.
+- Preview table paginates for large source lists, supports filters ("All", "New Sources", "Existing Sources"), and disables selection for duplicates/malformed entries.
+- Selection state and bulk settings persist across steps using a temporary ImportSession record.
+- Health check results update in real-time via Turbo Streams; unhealthy sources are unselected by default but can be re-selected.
+- Bulk settings form reuses the single source creation partial, applying settings uniformly to all selected sources.
+- Confirmation step displays a summary table of sources and settings; import progress and results are shown via Turbo Streams and static messaging.
+- Accessibility, Tailwind styling, and Turbo conventions are followed throughout.
+
+#### **Backend**
+- Introduce an ImportSession model to persist wizard state (uploaded file, parsed sources, selections, settings) per user/session.
+- OPML file is parsed synchronously in the controller; malformed entries are marked and excluded from selection.
+- Duplicate detection matches only on feed URL, referencing existing sources in the database.
+- Health checks for selected sources are enqueued as individual Solid Queue jobs; results are stored and broadcast via Turbo Streams.
+- Import confirmation triggers a background job that creates sources individually, reporting errors per source and skipping duplicates.
+- Import job results (successes, failures, skipped duplicates) are stored in an ImportHistory record for later viewing.
+- Controllers follow SourceMonitor’s established RESTful and Turbo Stream response patterns, with strong parameter sanitization and error handling.
+- If the user refreshes or exits the wizard, the ImportSession is discarded and progress is lost; navigation away triggers a JS warning.
+
+#### **Data & Storage**
+- Add `import_sessions` table to persist wizard state (user reference, OPML file metadata, parsed sources, selections, bulk settings, step state).
+- Add `import_histories` table to store completed import results (timestamp, user, sources imported, failures, skipped duplicates, error details).
+- No changes to existing `sourcemon_sources` table; duplicate detection uses feed URL matching.
+- All new tables follow SourceMonitor’s naming, indexing, and migration conventions.
+
+#### **Background Jobs**
+- Health check jobs use existing Solid Queue infrastructure, leveraging SourceMonitor’s health check logic.
+- Import job creates sources individually, logs errors per source, and updates ImportHistory.
+- Job orchestration, retry, and error handling follow established patterns in app/jobs/source_monitor/*.
+
+#### **Integrations**
+- No new external integrations; leverages existing Feedjira (OPML parsing), Faraday (feed health checks), and Turbo Streams (real-time UI).
+- All background jobs and real-time updates use Solid Queue and Solid Cable/Redis as configured.
+
+#### **Security**
+- Wizard and import actions restricted to authenticated admin users via SourceMonitor’s authentication hooks.
+- All data access and mutations use strong parameter sanitization and follow engine security conventions.
+
+#### **Testing**
+- Unit, integration, and system tests cover wizard flow, OPML parsing, selection logic, health checks, bulk settings, import job, and import history.
+- Use Minitest and VCR/WebMock for HTTP and feed parsing fixtures.
+- Regression tests for edge cases: malformed OPML, all sources already imported, all health checks fail, large file uploads, concurrent imports.
+- Turbo Stream and accessibility behaviors validated in system tests.
+
+### Data Model / Schema Changes
+
+| Table              | Column                | Type         | Description                                                        |
+|--------------------|----------------------|--------------|--------------------------------------------------------------------|
+| `import_sessions`  | `user_id`            | UUID (FK)    | References the admin user running the wizard                       |
+|                    | `opml_file_metadata` | JSONB        | Stores OPML file info (filename, size, upload timestamp)           |
+|                    | `parsed_sources`     | JSONB        | Array of parsed sources with status (valid, malformed, duplicate)  |
+|                    | `selected_source_ids`| JSONB        | Array of selected source identifiers                               |
+|                    | `bulk_settings`      | JSONB        | Bulk source settings to apply                                      |
+|                    | `current_step`       | String       | Tracks wizard step for session                                     |
+|                    | `created_at`         | Timestamp    | Creation time                                                      |
+|                    | `updated_at`         | Timestamp    | Last update time                                                   |
+| `import_histories` | `user_id`            | UUID (FK)    | References the admin user who performed the import                 |
+|                    | `imported_sources`   | JSONB        | Array of successfully imported sources                             |
+|                    | `failed_sources`     | JSONB        | Array of sources that failed to import with error details          |
+|                    | `skipped_duplicates` | JSONB        | Array of sources skipped due to duplication                        |
+|                    | `bulk_settings`      | JSONB        | Settings applied to imported sources                               |
+|                    | `started_at`         | Timestamp    | Import start time                                                  |
+|                    | `completed_at`       | Timestamp    | Import completion time                                             |
+
+All new tables and columns follow SourceMonitor’s migration, indexing, and naming standards.

data/tasks/completed/opml-import-wizard/task-01/instructions.md

@@ -0,0 +1,81 @@
+- **Context**
+  - Implement the entrypoint and shell for the multi-step OPML Import Wizard described in the PRD / Tech Brief. The wizard is launched from the Sources index page and contains steps: Upload, Preview, Health Check, Configure, Confirm. The shell must provide navigation UI, per‑step persistence, JS guard against accidental navigation/refresh, admin-only access, and server-side incremental persistence via a new ImportSession model (integer user_id, JSONB fields).
+  - This task does NOT implement file parsing, preview table, health checks, settings form, or the import/background job — those are separate tasks that depend on this shell.
+
+- **Goals**
+  - Add an "Import OPML" action on the Sources index that navigates to a dedicated wizard route.
+  - Create a multi-step wizard shell: left sidebar listing steps (Upload, Preview, Health Check, Configure, Confirm) with current step highlighted, and the main content area rendered per step.
+  - Provide per-step navigation with state persisted server-side to an ImportSession record so subsequent steps can update state.
+  - Show a browser JS warning when the user attempts to refresh or navigate away from the wizard.
+  - Provide a cancel/exit action that deletes the ImportSession and returns the user to the Sources index.
+  - Enforce admin-only access on the wizard endpoints and follow engine UI conventions (Tailwind, accessibility, Turbo Frames).
+
+- **Technical Guidelines**
+  - Routes & Controller
+    - Add engine routes for the wizard under the engine namespace (e.g., GET/POST /import_opml or /sources/import_opml) exposing step navigation. Use a Wicked-style incremental pattern (one controller that renders step views based on a step param or separate RESTful actions per step) and render step content inside Turbo Frames.
+    - Controller must enforce admin-only access using the engine’s existing authentication hooks (follow patterns used by SourcesController).
+    - Provide controller actions:
+      - new / create to start a new ImportSession and redirect to the first step,
+      - show/edit endpoints for step rendering and updates (or a step param on a single action),
+      - cancel/destroy to delete ImportSession and redirect to Sources index.
+    - All form submissions/step transitions should save partial state to ImportSession (see model below) and respond with Turbo Frame updates.
+  - ImportSession model & migration
+    - Implement ImportSession ActiveRecord model and DB migration following engine conventions.
+    - Use Rails’ default integer primary keys and integer user reference: t.references :user, foreign_key: true (not UUID).
+    - Include JSONB columns for server-side state:
+      - opml_file_metadata (JSONB)
+      - parsed_sources (JSONB) — array of parsed entries with status
+      - selected_source_ids (JSONB) — array of selected parsed entry identifiers
+      - bulk_settings (JSONB)
+      - current_step (string)
+    - Add timestamps (created_at, updated_at).
+    - Ensure migration and model naming follow repository conventions and include indices if needed for lookups by user.
+  - Views / UI Shell
+    - Add an "Import OPML" button/link to the Sources index page that navigates to the wizard entry route. Follow existing Sources index patterns (use Turbo Frame if appropriate).
+    - Create a wizard layout view that:
+      - Renders a left sidebar listing steps (Upload, Preview, Health Check, Configure, Confirm), highlights the active step, and uses accessible semantics (nav, aria-current on active step).
+      - Renders the step content region inside a named Turbo Frame so step content can be replaced without full-page reloads.
+      - Follows Tailwind styling and the engine’s accessibility patterns.
+    - Each step should be a separate partial/view (initially stubbed with placeholders) rendered into the Turbo Frame. The Upload step must include the file input form that posts to the ImportSession controller (actual parsing implemented in another task).
+  - Client behavior and navigation safety
+    - Implement a browser JS beforeunload warning when the user is in the wizard to prevent accidental refresh/navigate away. Use unobtrusive Stimulus/Turbolinks/Turbo patterns consistent with the repo (avoid breaking existing global behaviors).
+    - Provide explicit Cancel/Exit action in the UI which triggers server-side deletion of the ImportSession and redirects to Sources index. The Cancel action must not require the beforeunload confirmation to proceed.
+    - Ensure per-step navigation (next/back links) saves state to ImportSession via standard Rails form submissions or Turbo Frame form submissions so selections/text persist while the ImportSession exists.
+  - Persistence semantics & edge conditions
+    - Persist wizard state incrementally to ImportSession on each step transition or save action.
+    - Enforce admin-only access on import session resources.
+    - Do not implement parsing/preview/health logic here; parsed_sources may be initialized empty; Upload step should accept a file param and store metadata in opml_file_metadata (actual parsing implemented in next task).
+    - The UI should be designed so if the user refreshes (explicit browser reload), behavior conforms to the PRD requirement: show a fresh wizard state (documented for the implementing developer). Also show in UI a JS warning to prevent accidental refresh. (Concrete refresh reset behavior will be implemented per higher-level product decision when wiring parsing/session expiry.)
+  - Conventions & quality
+    - Use existing Turbo Frames, Turbo Streams and presenter patterns used across the engine. Use Tailwind classes and existing layout components for consistency.
+    - Sanitize and permit only required params in controller (use engine security/parameter sanitizer patterns).
+    - Add basic unit/controller/view tests to assert:
+      - Route and action presence,
+      - ImportSession creation with user_id integer,
+      - Cancel deletes the session and redirects,
+      - Sidebar renders with active step highlight,
+      - beforeunload warning is registered while wizard open.
+    - Internationalization/messages: follow repo pattern (simple English strings ok if consistent with existing views).
+    - Add migration, model, controller, views and routes in same PR; keep implementation minimal and extensible for follow-on tasks.
+
+- **Out of scope**
+  - Do not implement OPML parsing, source preview table, duplicate detection, health checks, bulk settings form fields, or background import job logic in this task.
+  - Do not implement persistence lifecycles beyond basic create/update/delete of ImportSession (e.g., expiry policies, garbage collection of sessions, or advanced session reconciliation).
+  - Do not implement per-feed settings overrides, real-time import progress bar, or import history viewing (those are separate tasks).
+  - Do not change sourcemon_sources table or alter existing source creation logic here.
+
+- **Suggested research**
+  - Inspect Sources index and its patterns to add the Import OPML entrypoint:
+    - app/views/source_monitor/sources/index.html.erb
+    - app/controllers/source_monitor/sources_controller.rb
+  - Review engine routing and controller conventions:
+    - config/routes.rb (engine)
+    - app/controllers/source_monitor/application_controller.rb (auth pattern)
+  - Review turbo frame and presenter/responder patterns:
+    - lib/source_monitor/turbo_streams/stream_responder.rb
+    - app/assets/javascripts/source_monitor/turbo_actions.js
+    - app/views/layouts/source_monitor/application.html.erb
+  - Find examples of model/migration conventions and JSONB usage:
+    - existing migrations under db/migrate/ and models using JSONB (e.g., sourcemon_* tables referenced in research)
+  - Authentication hooks / admin-only enforcement:
+    - lib/source_monitor/configuration.rb and controller auth checks used in SourcesController (follow same authorization approach).

data/tasks/completed/opml-import-wizard/task-01/requirements.md

@@ -0,0 +1,19 @@
+#### Goals  
+
+- Add an entry point on the Sources index page labeled "Import OPML" that navigates to a dedicated wizard route.  
+- Implement a multi-step wizard shell with a left sidebar listing steps (Upload, Preview, Health Check, Configure, Confirm) and highlighting the current step.  
+- Provide persistent per-step navigation and a JS warning when the user attempts to refresh or navigate away.  
+- Implement server-side incremental persistence for wizard state via an ImportSession ActiveRecord model so later steps can update state.
+
+#### Technical Considerations  
+
+- Use a dedicated engine route and controller that follow a Wicked-style incremental update pattern for steps and render step content inside Turbo Frames.  
+- Persist wizard state to an ImportSession record. Implement the ImportSession model and migration to reference the host user using standard integer-based ActiveRecord IDs (i.e., integer primary keys and a user reference using integer IDs). Ensure migrations use the default Rails integer id conventions (for example, t.references :user, foreign_key: true or equivalent) rather than UUID columns.  
+- Store core placeholders in ImportSession (opml_file_metadata, parsed_sources, selected_source_ids, bulk_settings, current_step) as JSONB where appropriate.  
+- Enforce admin-only access checks using existing SourceMonitor authentication hooks.  
+- Provide graceful cancel/exit behavior that deletes the ImportSession and returns the user to the Sources index.  
+- UI/UX must follow engine conventions: Tailwind styling, accessibility, and Turbo/Turbo Frames patterns used elsewhere in the codebase.
+
+#### Dependencies  
+
+- None

data/tasks/completed/opml-import-wizard/task-02/instructions.md

@@ -0,0 +1,83 @@
+**Context**
+
+- This task implements the "Upload" step of the OPML import wizard for the SourceMonitor engine.
+- The wizard shell and ImportSession persistence are provided by the wizard shell task; this task must wire the upload form and synchronous parsing into that flow.
+- Parsing should use the repo's existing feed parsing libraries (Feedjira/Nokogiri patterns) and persist parsed results into ImportSession.parsed_sources (JSONB). Malformed entries must be flagged and excluded from selection in the Preview step.
+
+**Goals**
+
+- Add an upload form for the Upload step that accepts an OPML file and validates file type before parsing.
+- Parse the uploaded OPML file synchronously in the controller action handling the Upload step.
+- Extract feed entries and persist them into ImportSession.parsed_sources as JSONB.
+- Mark malformed/unparsable entries with a status and an error message so they are not selectable in Preview.
+- Prevent navigation to Preview until ImportSession.parsed_sources contains at least one valid parsed entry; surface actionable error messages for invalid file / parse failures.
+- Follow SourceMonitor engine conventions (strong params, user scoping, security, Tailwind/Turbo forms).
+
+**Technical Guidelines**
+
+- Controller & route
+  - Implement a POST action on the wizard Upload step controller (the wizard route provided by the shell) that receives the uploaded file and the ImportSession identifier (or creates/loads ImportSession for current admin).
+  - Enforce admin-only access via existing SourceMonitor authentication hooks.
+  - Use strong parameter sanitization for the file param and ImportSession id.
+
+- File handling & validation
+  - Accept standard Rails file upload types (ActionDispatch::Http::UploadedFile). Validate the upload exists and is non-empty.
+  - Perform a content-type check for XML/OPML (e.g., application/xml, text/xml, text/x-opml, application/opml) but do not rely solely on content-type: if content-type is absent or generic, attempt XML parsing anyway.
+  - Read the uploaded file via tempfile or uploaded_file.read in a memory-conscious way. Follow engine file handling safety conventions (tempfiles, no persistent file writes unless required by ImportSession metadata).
+  - Persist basic OPML file metadata into ImportSession.opml_file_metadata (filename, size, uploaded_at).
+
+- Synchronous parsing
+  - Parse synchronously in the controller request handling the Upload step—do not enqueue parsing to background jobs.
+  - Reuse existing parsing patterns: prefer Feedjira for feed handling when applicable and Nokogiri for XML traversal (OPML is XML). Use Feedjira/Nokogiri idioms already present in the repo (inspect lib/source_monitor/fetching/feed_fetcher.rb and feedjira init).
+  - Traverse OPML outlines to extract feed entries; for each candidate entry, normalize and extract at minimum: feed URL (required), title (if present), website/HTML URL (if present), and any obvious metadata.
+  - For entries that cannot be parsed (malformed XML snippet, missing feed URL, invalid URL), produce an entry object with a status (e.g., "malformed") and an error message describing the issue.
+  - For successfully parsed entries produce an entry object with status "valid" and extracted fields. Do not perform duplicate detection here (preview task handles duplicates), but include the feed_url so preview can detect duplicates by comparing against existing sourcemon_sources.
+
+- Persisting parsed results
+  - Store the array of parsed entry objects into ImportSession.parsed_sources (JSONB). Also update ImportSession.opml_file_metadata and ImportSession.current_step if appropriate.
+  - Parsed entries must include:
+    - stable temporary id or index (so UI can refer to rows)
+    - feed_url (string)
+    - title (string|null)
+    - website_url (string|null)
+    - status: "valid" | "malformed"
+    - error: string (present when status == "malformed")
+    - any other minimal metadata useful to Preview (e.g., raw_outline_index, line/position optional)
+  - Ensure ImportSession is scoped to the current user (user_id integer) and use standard ActiveRecord integer primary key conventions (do not change user id type).
+
+- Errors & UX constraints
+  - If the uploaded file is not valid XML / OPML, return an actionable error message on the Upload view (do not crash). Do not advance to Preview.
+  - If parsing succeeds but yields zero valid entries, block progression to Preview and show a clear message instructing the user to upload a different OPML or correct the file.
+  - If parsing yields some valid entries, persist them and redirect/render the Preview step UI via Turbo Frames (the wizard shell controls the step rendering). Ensure selections are not created yet—only parsed_sources persisted.
+
+- Security & limits
+  - Respect standard engine security: only admin users can upload; operate on ImportSession scoped to current user.
+  - Use tempfile APIs provided by Rails for file reading; do not dangerously evaluate XML. Use safe Nokogiri parsing options (no external entity expansion).
+  - Do not implement custom file size limits in this task (out of scope), but parse in a way that does not load huge files into excessive memory when possible.
+
+- Testing guidance (what to cover)
+  - Controller tests for successful upload and parsing storing parsed_sources with both valid and malformed entries.
+  - Tests for invalid file types and malformed XML returning appropriate errors and not advancing to Preview.
+  - Verify ImportSession is updated (opml_file_metadata and parsed_sources) and is scoped to current user.
+  - Use existing repo test helpers and VCR/webmock patterns if parsing makes external requests (should not for pure OPML parsing).
+
+**Out of scope**
+
+- Preview UI/table rendering, filters or pagination (Preview task handles these).
+- Duplicate detection or marking duplicates as part of parsing (Preview task handles matching feed_url against existing sourcemon_sources).
+- Backgrounding parsing or streaming large-file parsing—parsing must be synchronous per requirements.
+- Per-feed settings, health checks, or import confirmation logic.
+- File storage beyond keeping metadata in ImportSession.opml_file_metadata.
+
+**Suggested research (inspect before implementing)**
+
+- Existing feed parsing patterns:
+  - lib/source_monitor/fetching/feed_fetcher.rb (Feedjira usage and parsing patterns)
+  - config/initializers/feedjira.rb (any repo customizations)
+- Wizard shell controller/layout and ImportSession model/location created by the wizard shell task:
+  - Routes and controller names for the wizard Upload step (app/controllers/... likely under source_monitor/import_sessions or import_wizard controller).
+  - ImportSession model/migration shape (fields: parsed_sources JSONB, opml_file_metadata JSONB, user_id integer).
+- Existing examples of file upload handling and strong param patterns across the engine (SourcesController create/update forms and controllers under app/controllers/source_monitor).
+- Nokogiri safe parsing patterns used in repo (search for Nokogiri usage) to avoid unsafe XML parsing options.
+
+Implement only the Upload step controller action, the upload form view for the Upload step, synchronous parsing and persistence to ImportSession.parsed_sources, and the error/flow control that blocks Preview until at least one valid parsed entry exists.

data/tasks/completed/opml-import-wizard/task-02/requirements.md

@@ -0,0 +1,18 @@
+#### Goals  
+
+- Implement the Upload step form that accepts OPML files and validates file type before parsing.  
+- Parse OPML synchronously in the request, extract feed entries, and persist parsed entries into the ImportSession.parsed_sources JSONB field.  
+- Mark malformed or unparsable entries clearly (status + error message) so they are excluded from selection in Preview.  
+- Block navigation to the Preview step until at least one valid parsed entry exists.
+
+#### Technical Considerations  
+
+- Use existing Feedjira/Nokogiri parsing patterns where available; parse synchronously in controller action handling the Upload step and store parsed results in ImportSession.parsed_sources.  
+- Validate content-type and fallback to XML parsing based on file contents; present actionable error messages for invalid files or malformed XML.  
+- Ensure the ImportSession model and migration reference the user via standard integer-based ActiveRecord IDs (i.e., integer primary keys and an integer user reference). Do not introduce UUID columns for user references—use the Rails default integer id and foreign key conventions.  
+- For large files, ensure the request handles a reasonable file size and the preview UI will paginate large parsed lists (pagination handled in Preview task). Synchronous parsing chosen deliberately for immediate feedback.  
+- Respect security and file handling conventions in the engine (tempfiles, permitted params, user scoping).
+
+#### Dependencies  
+
+- Implement OPML Import Wizard Shell (needed for routes, ImportSession persistence and step navigation)

data/tasks/completed/opml-import-wizard/task-03/instructions.md

@@ -0,0 +1,58 @@
+- **Context**
+  - This task implements the Preview step of the OPML import wizard (part of the "Preview and Select Sources" user story). The Upload step will have populated ImportSession.parsed_sources (JSONB) with entries parsed from the OPML and ImportSession exists for the current admin user. The engine uses Turbo Frames, Turbo Streams, Tailwind, server-side pagination helpers, and a Source model/table (sourcemon_sources) for duplicate detection. Selection state for the wizard must persist in ImportSession.selected_source_ids (JSONB) while the ImportSession exists.
+
+- **Goals**
+  - Render a paginated Preview table of parsed OPML entries with columns: feed URL, title, and “Already Imported” status.
+  - Mark duplicate entries (feed URL matched against existing sourcemon_sources) as “Already Imported” and make them non-selectable.
+  - Show malformed/parsing-error entries (from parsed_sources) with inline error indicator and make them non-selectable.
+  - Provide filters: All, New Sources, Existing Sources (server-side filtering).
+  - Allow admin to select/unselect valid entries; persist selections across filter changes and step navigation by updating ImportSession.selected_source_ids.
+  - Prevent proceeding to the next wizard step when no sources are selected and surface a clear warning.
+
+- **Technical Guidelines**
+  - Rendering & UI
+    - Implement the Preview step view as a Turbo Frame (consistent with engine conventions used for sources index tables). The table content must be server-rendered (HTML partials) and replaceable via turbo_frame_tag updates.
+    - Table columns: feed URL (clickable/visible text), title (as parsed), and “Already Imported” status (visual badge/icon). Malformed rows must show a clear inline error message/toolip and a disabled checkbox.
+    - Use Tailwind classes and accessibility patterns consistent with other engine tables and forms.
+    - Add filter controls (All / New / Existing) that trigger full-frame GET requests (targeting the preview Turbo Frame) and support pagination & stateful checkbox persistence.
+  - Data & Selection Persistence
+    - Persist and read selection state in ImportSession.selected_source_ids (JSONB). Store a stable identifier for each parsed entry that can be used across requests. If parsed_sources includes a unique temp id, use it; otherwise use feed_url as the canonical identifier for selection storage (feed_url is required for duplicate detection).
+    - When rendering rows, mark checkboxes checked if their identifier appears in ImportSession.selected_source_ids.
+    - Update ImportSession.selected_source_ids via server endpoints (e.g., POST/PATCH from the preview frame) whenever a user toggles selection or submits the preview form. The controller actions must sanitize params and scope updates to the current ImportSession and current admin user.
+    - Ensure ImportSession model uses integer user_id as configured in the tech brief. (This task reads/updates ImportSession only; ensure permission checks enforce admin-only access.)
+  - Duplicate & Malformed Handling
+    - Duplicate detection: treat a parsed entry as an "existing" duplicate if its feed URL exactly matches an existing source record (sourcemon_sources). Mark duplicates visually and make their selection control disabled.
+    - Malformed entries: rely on parser-set flags in ImportSession.parsed_sources to identify malformed entries; render error details inline and disable selection.
+  - Filters & Pagination
+    - Implement server-side filtering logic:
+      - All => show all parsed_sources (including malformed and duplicates).
+      - New Sources => parsed entries that are valid and not duplicate.
+      - Existing Sources => parsed entries flagged as duplicate (already imported).
+    - Implement server-side pagination for the preview table using existing pagination helpers/patterns used elsewhere in the engine (do not load entire list into browser for very large files).
+    - Preserve selection state across pagination and filter changes by reading/writing ImportSession.selected_source_ids (i.e., selections are global to the ImportSession, not only per page).
+  - Navigation & Progression Block
+    - Provide UI validation that blocks “Next” navigation when ImportSession.selected_source_ids is empty (or contains no enabled/valid entries). Show a clear, inline warning message in the Preview step explaining at least one valid source must be selected.
+  - Security & Conventions
+    - Enforce admin-only access for all preview endpoints using existing SourceMonitor authentication hooks.
+    - Use strong parameter sanitization for any incoming params (filter, page, selection updates).
+    - Follow the engine’s Turbo/Tailwind/Accessibility patterns and Turbo Frame naming conventions so later steps and Turbo Streams integrate cleanly.
+  - Integration points (do not reimplement)
+    - Read parsed entries from ImportSession.parsed_sources (populated by the Upload/Parsing step).
+    - For duplicate detection, query the canonical Source model/table (sourcemon_sources / SourceMonitor::Source) — match by feed URL only.
+    - Use existing pagination helpers and table rendering conventions used by the Sources index.
+
+- **Out of scope**
+  - Health check job orchestration, Turbo Stream live broadcasting for health — that is handled in the Health Check step.
+  - Upload/parsing implementation (Add OPML Upload & Synchronous Parsing) and ImportSession creation — assume ImportSession.parsed_sources exists with required fields.
+  - Per-feed settings UI or per-feed overrides.
+  - Client-only selection persistence (all persistence must be server-backed in ImportSession.selected_source_ids).
+  - Retry semantics for failed persistence or selection race-handling beyond simple idempotent updates and param sanitization.
+
+- **Suggested research**
+  - Inspect ImportSession model and its parsed_sources schema to confirm the entry shape and any temporary identifiers (or confirm to use feed_url as identifier).
+  - Review the wizard controller/routes for Preview step (ImportSession/wizard controller) to determine the Turbo Frame name and route endpoints for updating selection and paginated frame rendering.
+  - Review existing sources index/table implementation and pagination helpers to follow consistent Turbo Frame patterns and CSS classes:
+    - app/views/source_monitor/sources/index.html.erb and its table partials
+    - existing pagination helpers under lib/source_monitor/pagination or helpers used in sources index
+  - Inspect Source model / table for feed_url column and any scopes used for matching (sourcemon_sources / SourceMonitor::Source).
+  - Check authentication hooks and parameter sanitizer patterns used by controllers to ensure admin-only access and strong param handling (e.g., SourceMonitor.config.authentication, SourceMonitor::Security::ParameterSanitizer).

data/tasks/completed/opml-import-wizard/task-03/requirements.md

@@ -0,0 +1,18 @@
+#### Goals  
+
+- Render a Preview step showing a paginated table of parsed sources with columns: feed URL, title, and "Already Imported" status.  
+- Detect duplicates by matching feed URL against existing sourcemon_sources and mark duplicates as non-selectable.  
+- Provide filters to toggle between "All", "New Sources", and "Existing Sources" and ensure selections persist across filter changes and step navigation.  
+- Prevent progression to the next step if all sources are deselected and show a clear warning.
+
+#### Technical Considerations  
+
+- Implement the preview UI using Turbo Frames for the table and follow existing sources index patterns (search form targeting a turbo frame, Tailwind styling and accessibility conventions).  
+- Store user selection state in ImportSession.selected_source_ids (JSONB) so selections persist across wizard steps and page navigations while the ImportSession exists. Ensure the ImportSession model uses an integer-based user reference (standard Rails user_id integer foreign key).  
+- Duplicate detection uses only feed URL matching (as decided); malformed entries flagged by parsing step must be disabled for selection.  
+- Implement server-side pagination for large parsed lists using existing pagination helpers used in the engine.
+
+#### Dependencies  
+
+- Add OPML Upload & Synchronous Parsing (parsing fills parsed_sources used to render preview)  
+- Implement OPML Import Wizard Shell (routes, ImportSession persistence and wizard navigation)