mailer-log 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 62eb9e79690dbc8bfda9c264bdebc0d1da51e29f5df9d54d549e9fef7420cf3d
+  data.tar.gz: c9ff2c0973cdfd3514c6f9b40e102a4aa0784e21290b9b5eee0bfe51ceea9048
+SHA512:
+  metadata.gz: 90f81eac98bbffe31bec481a156d160597a93b4e54d510c8d61784fd02aa38807723ce5cf4b6aaa622c1bd45b30be65c2a11e4a118d887757513ba67ee571cec
+  data.tar.gz: fab117cb20cb2368872faf733b6718fbffe1fafc51be4b58409d2f66a58cc632b20fe362194940de6dad562416db200de1c54c3dcc722fed803b3d9ffed6a7f0

data/CHANGELOG.md

@@ -0,0 +1,22 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2024-12-14
+
+### Added
+
+- Initial release
+- Rails engine for capturing all outgoing emails
+- PostgreSQL storage for emails with full headers, body, and metadata
+- Vue.js 3 admin UI with Tailwind CSS for browsing emails
+- Mailgun webhook integration for delivery tracking (delivered, opened, clicked, bounced)
+- Polymorphic `accountable` association for linking emails to business objects
+- Configurable call stack capture for debugging
+- JSON API for programmatic access
+- Filter and search capabilities (by recipient, sender, subject, mailer, status, date range)
+- Email preview with iframe rendering
+- Delivery events timeline

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 TrafficRunners
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,286 @@
+# MailerLog
+
+Rails engine for logging all outgoing emails with Mailgun webhook integration.
+
+## Features
+
+- Automatic capture of all outgoing emails (body, headers, mailer class/action)
+- Call stack saving for debugging (shows where the email was triggered from)
+- Mailgun webhook integration for delivery status tracking
+- Modern Vue 3 + Tailwind CSS admin UI
+- Email preview in sandboxed iframe
+- Polymorphic association with Organization/Account for filtering by organization
+- Configurable retention period with automatic cleanup
+
+## Installation
+
+### 1. Add to Gemfile
+
+```ruby
+gem 'mailer_log'
+```
+
+### 2. Run install generator
+
+```bash
+bundle install
+rails generate mailer_log:install
+rails db:migrate
+```
+
+This will:
+- Create `config/initializers/mailer_log.rb` with configuration
+- Copy migration to `db/migrate/`
+
+### 3. Add routes
+
+```ruby
+# config/routes.rb
+Rails.application.routes.draw do
+  mount MailerLog::Engine, at: '/mailer_log'
+end
+```
+
+### 4. Build the frontend (if needed)
+
+The engine includes pre-built Vue.js frontend assets. If you need to rebuild:
+
+```bash
+# From host application
+rake mailer_log:build_frontend
+
+# Or directly
+cd path/to/mailer_log/frontend
+npm install
+npm run build
+```
+
+### 5. Configure initializer
+
+Edit `config/initializers/mailer_log.rb` to customize settings:
+
+```ruby
+MailerLog.configure do |config|
+  # Email retention period (default 1 year)
+  config.retention_period = 1.year
+
+  # Key for Mailgun webhook verification
+  config.webhook_signing_key = ENV['MAILGUN_WEBHOOK_SIGNING_KEY']
+
+  # Capture call stack (useful for debugging)
+  config.capture_call_stack = Rails.env.development?
+
+  # Optional: Additional authentication for admin UI
+  # config.authenticate_with do |controller|
+  #   controller.send(:require_super_admin!)
+  # end
+
+  # Optional: Resolver for organization association
+  # config.resolve_accountable do |email_record|
+  #   User.find_by(email: email_record.to_addresses&.first)&.organization
+  # end
+end
+```
+
+## Mailgun Webhook Setup
+
+### 1. Get Webhook Signing Key
+
+1. Log in to [Mailgun Dashboard](https://app.mailgun.com/)
+2. Go to **Sending** → **Webhooks**
+3. Copy the **HTTP webhook signing key**
+4. Add to ENV: `MAILGUN_WEBHOOK_SIGNING_KEY=your_key_here`
+
+### 2. Configure Webhook URL
+
+1. In Mailgun Dashboard, go to **Webhooks**
+2. Add webhook for each event:
+   - URL: `https://your-app.com/admin/email_log/webhooks/mailgun`
+   - Events: `delivered`, `opened`, `clicked`, `bounced`, `failed`, `dropped`, `complained`
+
+### 3. For Each Domain
+
+If using multiple domains (white-label), configure webhooks for each.
+
+## Usage
+
+### Admin UI
+
+After installation, accessible at: `/admin/email_log/admin/emails`
+
+**Features:**
+- List of all sent emails with pagination
+- Filtering by: recipient, sender, subject, mailer class, status, date
+- Email details view: headers, body preview, delivery events
+- Call stack view (where in code the email was triggered)
+
+### Email Statuses
+
+| Status | Description |
+|--------|-------------|
+| `pending` | Email created, waiting to be sent |
+| `sent` | Email sent to Mailgun |
+| `delivered` | Email delivered to recipient |
+| `opened` | Email opened (tracking pixel) |
+| `clicked` | Link in email clicked |
+| `bounced` | Email not delivered (hard bounce) |
+| `complained` | Recipient marked as spam |
+
+### Programmatic Access
+
+```ruby
+# Find all emails for a user
+MailerLog::Email.recipient('user@example.com')
+
+# Find emails by mailer class
+MailerLog::Email.by_mailer('UsersMailer')
+
+# Find undelivered emails
+MailerLog::Email.by_status('bounced')
+
+# Last 10 emails
+MailerLog::Email.recent.limit(10)
+
+# Emails with events
+email = MailerLog::Email.find(id)
+email.events.each do |event|
+  puts "#{event.event_type} at #{event.occurred_at}"
+end
+```
+
+### Cleaning Up Old Records
+
+Add to Sidekiq cron or call periodically:
+
+```ruby
+# Run manually
+MailerLog::CleanupJob.perform_now
+
+# Via Sidekiq
+MailerLog::CleanupJob.perform_later
+
+# In sidekiq-cron (config/schedule.yml or sidekiq.yml)
+cleanup_mailer_log:
+  cron: '0 3 * * *'  # Daily at 3:00 AM
+  class: MailerLog::CleanupJob
+  queue: low_priority
+```
+
+## Configuration
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `retention_period` | `1.year` | Email retention period |
+| `webhook_signing_key` | `nil` | Key for Mailgun webhook verification |
+| `capture_call_stack` | `true` | Capture call stack |
+| `call_stack_depth` | `20` | Call stack depth |
+| `admin_layout` | `'application'` | Layout for admin views |
+| `per_page` | `25` | Records per page |
+
+## Troubleshooting
+
+### Emails Not Being Saved
+
+1. Check that engine is included in Gemfile and bundle install was run
+2. Check that migrations are executed: `rails db:migrate:status`
+3. Check logs for errors in `MailerLog::MailObserver`
+
+### Webhooks Not Working
+
+1. Check `MAILGUN_WEBHOOK_SIGNING_KEY` in ENV
+2. Check that URL is accessible externally (not localhost)
+3. Check logs for signature verification errors
+4. In Mailgun Dashboard, check webhook status (failed/success)
+
+### Events Not Linking to Emails
+
+1. Check that `message_id` is being saved correctly
+2. Check that header `X-Mailer-Log-Tracking-ID` is passed to Mailgun
+3. Webhook may arrive before email is saved — retry mechanism is used
+
+## Frontend Development
+
+The admin UI is built with Vue 3 + Vite + Tailwind CSS.
+
+### Development Mode
+
+For hot-reload during development:
+
+```bash
+# Terminal 1: Start Rails server
+rails server
+
+# Terminal 2: Start Vite dev server
+cd path/to/mailer_log/frontend
+npm run dev
+```
+
+The Vue app will automatically use the Vite dev server at `http://localhost:5173` when assets are not built.
+
+### Building for Production
+
+```bash
+rake mailer_log:build_frontend
+```
+
+Built assets are stored in `public/mailer_log/assets/`.
+
+### Frontend Structure
+
+```
+frontend/
+├── src/
+│   ├── api/           # API client functions
+│   ├── assets/        # CSS and static assets
+│   ├── components/    # Reusable Vue components
+│   ├── views/         # Page components
+│   ├── App.vue        # Root component
+│   └── main.js        # Entry point
+├── index.html         # HTML template
+├── package.json
+├── tailwind.config.js
+└── vite.config.js
+```
+
+### Customizing Components
+
+You can override Vue components (like the navbar) to match your application's style.
+
+1. Create an overrides directory in your host application:
+```bash
+mkdir -p app/javascript/mailer_log_overrides
+```
+
+2. Copy and customize the component:
+```bash
+cp path/to/mailer_log/frontend/src/components/AppNavbar.vue \
+   app/javascript/mailer_log_overrides/AppNavbar.vue
+```
+
+3. Build with the override path:
+```bash
+MAILER_LOG_OVERRIDES_PATH=/path/to/app/javascript/mailer_log_overrides \
+  rake mailer_log:build_frontend
+```
+
+**Overridable components:**
+- `AppNavbar.vue` - Header/navigation bar
+
+Example custom navbar that matches host application style:
+```vue
+<template>
+  <nav class="your-app-navbar">
+    <router-link to="/">Email Log</router-link>
+    <!-- Your custom navigation items -->
+  </nav>
+</template>
+
+<script setup>
+// You can import from @mailer-log/ alias to reuse engine components
+// import StatusBadge from '@mailer-log/components/StatusBadge.vue'
+</script>
+```
+
+## License
+
+MIT

data/app/assets/stylesheets/mailer_log/emails.scss

@@ -0,0 +1,18 @@
+// MailerLog Email Views Styles
+
+.mailer-log-email-preview {
+  width: 100%;
+  height: 500px;
+  border: 1px solid #ddd;
+  background: #fff;
+}
+
+.mailer-log-scrollable-content {
+  max-height: 500px;
+  overflow: auto;
+}
+
+.mailer-log-scrollable-content-sm {
+  max-height: 300px;
+  overflow: auto;
+}

data/app/controllers/mailer_log/admin/emails_controller.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module MailerLog
+  module Admin
+    class EmailsController < MailerLog::AdminController
+      has_scope :recipient
+      has_scope :sender
+      has_scope :subject_search
+      has_scope :by_mailer, as: :mailer
+      has_scope :by_status, as: :status
+      has_scope :date_from
+      has_scope :date_to
+
+      def index
+        @emails = apply_scopes(MailerLog::Email.order(created_at: :desc))
+          .page(params[:page])
+          .per(params[:per] || 25)
+        @mailers = MailerLog::Email.distinct.pluck(:mailer_class).compact.sort
+      end
+
+      def show
+        @email = MailerLog::Email.find(params[:id])
+      end
+
+      def preview
+        @email = MailerLog::Email.find(params[:id])
+        # rubocop:disable Rails/OutputSafety
+        render html: @email.html_body&.html_safe, layout: false
+        # rubocop:enable Rails/OutputSafety
+      end
+
+    end
+  end
+end

data/app/controllers/mailer_log/admin_controller.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+module MailerLog
+  class AdminController < ::AdminsController
+    prepend HasScope
+
+    # Expose main app routes to views for navbar/layout links
+    helper Rails.application.routes.url_helpers
+
+    before_action :authenticate_mailer_log!
+
+    # Make main_app proxy available in views
+    helper_method :main_app
+
+    # Override url_options to prevent main app routes from being prefixed
+    # with the engine's mount path
+    def url_options
+      script_name = request.env['SCRIPT_NAME']
+      return super unless script_name&.include?('mailer_log') || script_name&.include?('email_log')
+
+      super.merge(script_name: '')
+    end
+
+    private
+
+    def authenticate_mailer_log!
+      return unless MailerLog.configuration.authenticate_with_proc
+
+      MailerLog.configuration.authenticate_with_proc.call(self)
+    end
+
+    def mailer_log_engine
+      MailerLog::Engine.routes.url_helpers
+    end
+    helper_method :mailer_log_engine
+  end
+end

data/app/controllers/mailer_log/api/emails_controller.rb

@@ -0,0 +1,88 @@
+# frozen_string_literal: true
+
+module MailerLog
+  module Api
+    class EmailsController < MailerLog::AdminController
+      skip_before_action :verify_authenticity_token, only: [:index, :show]
+
+      def index
+        emails = apply_filters(MailerLog::Email.order(created_at: :desc))
+          .page(params[:page])
+          .per(params[:per] || 25)
+
+        render json: {
+          emails: emails.map { |e| email_json(e) },
+          total_count: emails.total_count,
+          total_pages: emails.total_pages,
+          current_page: emails.current_page
+        }
+      end
+
+      def show
+        email = MailerLog::Email.find(params[:id])
+
+        render json: {
+          email: email_json(email, include_body: true, include_events: true)
+        }
+      end
+
+      private
+
+      def apply_filters(scope)
+        scope = scope.recipient(params[:recipient]) if params[:recipient].present?
+        scope = scope.sender(params[:sender]) if params[:sender].present?
+        scope = scope.subject_search(params[:subject_search]) if params[:subject_search].present?
+        scope = scope.by_mailer(params[:mailer]) if params[:mailer].present?
+        scope = scope.by_status(params[:status]) if params[:status].present?
+        scope = scope.date_from(params[:date_from]) if params[:date_from].present?
+        scope = scope.date_to(params[:date_to]) if params[:date_to].present?
+        scope
+      end
+
+      def email_json(email, include_body: false, include_events: false)
+        data = {
+          id: email.id,
+          tracking_id: email.tracking_id,
+          message_id: email.message_id,
+          mailer_class: email.mailer_class,
+          mailer_action: email.mailer_action,
+          from_address: email.from_address,
+          to_addresses: email.to_addresses,
+          cc_addresses: email.cc_addresses,
+          bcc_addresses: email.bcc_addresses,
+          subject: email.subject,
+          domain: email.domain,
+          status: email.status,
+          delivered_at: email.delivered_at,
+          opened_at: email.opened_at,
+          clicked_at: email.clicked_at,
+          bounced_at: email.bounced_at,
+          created_at: email.created_at,
+          updated_at: email.updated_at
+        }
+
+        if include_body
+          data[:html_body] = email.html_body
+          data[:text_body] = email.text_body
+          data[:headers] = email.headers
+          data[:call_stack] = email.call_stack
+        end
+
+        if include_events
+          data[:events] = email.events.recent.map do |event|
+            {
+              id: event.id,
+              event_type: event.event_type,
+              occurred_at: event.occurred_at,
+              recipient: event.recipient,
+              ip_address: event.ip_address,
+              user_agent: event.user_agent
+            }
+          end
+        end
+
+        data
+      end
+    end
+  end
+end

data/app/controllers/mailer_log/api/mailers_controller.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module MailerLog
+  module Api
+    class MailersController < MailerLog::AdminController
+      skip_before_action :verify_authenticity_token
+
+      def index
+        mailers = MailerLog::Email.distinct.pluck(:mailer_class).compact.sort
+
+        render json: { mailers: mailers }
+      end
+    end
+  end
+end

data/app/controllers/mailer_log/application_controller.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+module MailerLog
+  class ApplicationController < ActionController::Base
+    protect_from_forgery with: :exception
+  end
+end

data/app/controllers/mailer_log/assets_controller.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module MailerLog
+  class AssetsController < ActionController::Base
+    skip_forgery_protection
+
+    def show
+      file_path = MailerLog::Engine.root.join('public', 'mailer_log', 'assets', params[:path])
+
+      if File.exist?(file_path) && file_path.to_s.start_with?(MailerLog::Engine.root.join('public', 'mailer_log').to_s)
+        content_type = content_type_for(file_path)
+        send_file file_path, type: content_type, disposition: 'inline'
+      else
+        head :not_found
+      end
+    end
+
+    private
+
+    def content_type_for(file_path)
+      extension = File.extname(file_path).downcase
+      case extension
+      when '.js'
+        'application/javascript'
+      when '.css'
+        'text/css'
+      when '.map'
+        'application/json'
+      when '.svg'
+        'image/svg+xml'
+      when '.png'
+        'image/png'
+      when '.jpg', '.jpeg'
+        'image/jpeg'
+      when '.woff', '.woff2'
+        'font/woff2'
+      else
+        'application/octet-stream'
+      end
+    end
+  end
+end

data/app/controllers/mailer_log/spa_controller.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module MailerLog
+  class SpaController < MailerLog::AdminController
+    helper MailerLog::SpaHelper
+
+    layout false
+
+    def index
+      render :index
+    end
+  end
+end