@bernierllc/email-webhook-events 1.0.0 → 1.2.0

package/CHANGELOG.md

@@ -0,0 +1,50 @@
+# Change Log
+
+All notable changes to this project will be documented in this file.
+See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
+
+# 1.2.0 (2025-12-25)
+
+
+### Bug Fixes
+
+* **ci:** add lerna version step before publish ([3d20300](https://github.com/bernierllc/tools/commit/3d203002143bf353fffafe4f8a78a99009567347))
+* update neverhub-adapter deps to workspace:* and publish 0.1.2 ([f0e3d04](https://github.com/bernierllc/tools/commit/f0e3d04d8d4f094e3bb899ddf81e93243d16e2c2))
+
+
+### Features
+
+* **email-webhook-events:** migrate to Vitest and add integration testing ([c45c75c](https://github.com/bernierllc/tools/commit/c45c75ce2db9cc9f45fe1be6c73e8018a427727c))
+
+
+
+
+
+# 1.1.0 (2025-12-25)
+
+
+### Bug Fixes
+
+* update neverhub-adapter deps to workspace:* and publish 0.1.2 ([f0e3d04](https://github.com/bernierllc/tools/commit/f0e3d04d8d4f094e3bb899ddf81e93243d16e2c2))
+
+
+### Features
+
+* **email-webhook-events:** migrate to Vitest and add integration testing ([c45c75c](https://github.com/bernierllc/tools/commit/c45c75ce2db9cc9f45fe1be6c73e8018a427727c))
+
+
+
+
+
+# @bernierllc/email-webhook-events
+
+## 1.0.1
+
+### Patch Changes
+
+- [`c45c75c`](https://github.com/bernierllc/tools/commit/c45c75ce2db9cc9f45fe1be6c73e8018a427727c) - Migrate test infrastructure from Jest to Vitest and add integration testing support
+  - Switch from Jest to Vitest for improved ESM compatibility
+  - Add integration test infrastructure using ngrok for real SendGrid webhook testing
+  - Add @bernierllc/email-manager as dependency for integration tests
+  - Capture real webhook payloads for accurate test fixtures
+  - Update README with integration testing documentation

package/README.md

@@ -321,6 +321,48 @@ EMAIL_WEBHOOK_RATE_LIMIT_ENABLED=false
 EMAIL_WEBHOOK_RATE_LIMIT_MAX_PER_SECOND=100
 ```
 
+## Integration Testing
+
+This package includes integration tests that use real SendGrid webhooks.
+
+### Prerequisites
+
+1. SendGrid account with API key
+2. Paid ngrok account with auth token
+3. Test email address
+4. `.env.test` file configured with:
+   - `SENDGRID_API_KEY`
+   - `SENDGRID_WEBHOOK_SECRET` (optional)
+   - `TEST_EMAIL`
+   - `NGROK_AUTH_TOKEN`
+   - `NGROK_RESERVED_DOMAIN` (optional, for consistent URLs)
+   - `WEBHOOK_SERVER_PORT` (optional, defaults to 3000)
+
+### Running Integration Tests
+
+1. Ensure `.env.test` file is configured with all required variables
+2. Set `RUN_INTEGRATION_TESTS=true` environment variable
+3. Configure SendGrid webhook URL (first time only, or use reserved domain):
+   - Run test once to get webhook URL from output, or
+   - Use reserved domain: `https://your-reserved-domain.ngrok.io/webhooks/sendgrid`
+   - Go to SendGrid dashboard → Settings → Mail Settings → Event Webhook
+   - Set webhook URL and enable all event types
+4. Run tests: `npm test -- sendgrid-webhook-integration`
+
+Tests automatically:
+- Start webhook server
+- Create ngrok tunnel (programmatically)
+- Send test emails via @bernierllc/email-manager
+- Capture and process webhooks
+- Save payloads to `__tests__/fixtures/sendgrid-webhooks/`
+
+### Captured Payloads
+
+Real webhook payloads are captured in `__tests__/fixtures/sendgrid-webhooks/`
+and used to generate accurate mocks for unit tests.
+
+See `__tests__/fixtures/sendgrid-webhooks/README.md` for more details.
+
 ## Integration Status
 
 - **Logger**: integrated - Uses MockLogger (pending @bernierllc/logger publication)

package/__tests__/fixtures/sendgrid-webhooks/README.md

@@ -0,0 +1,63 @@
+# SendGrid Webhook Fixtures
+
+This directory contains webhook payloads based on REAL SendGrid webhook structures captured during integration testing.
+
+> ⚠️ **SANITIZED DATA**: All email addresses, IPs, message IDs, signatures, and other identifiable data have been replaced with fake test values. The **structure** matches real SendGrid payloads but all values are safe for version control.
+
+## Structure
+
+Each event type has its own JSON file containing an array of captured webhook payloads:
+
+- `delivered.json` - Email delivered events
+- `open.json` - Email opened events
+- `click.json` - Link clicked events
+- `bounce.json` - Email bounced events
+- `spamreport.json` - Spam complaint events
+- `unsubscribe.json` - Unsubscribe events
+- `deferred.json` - Email delivery deferred events
+- `dropped.json` - Email dropped events
+
+## File Format
+
+Each file contains an array of captured webhook objects:
+
+```json
+[
+  {
+    "headers": {
+      "x-twilio-email-event-webhook-signature": "...",
+      "content-type": "application/json",
+      ...
+    },
+    "body": {
+      "event": "delivered",
+      "email": "user@example.com",
+      "timestamp": 1234567890,
+      "sg_message_id": "...",
+      ...
+    },
+    "timestamp": "2025-01-27T12:00:00.000Z",
+    "eventType": "delivered"
+  }
+]
+```
+
+## Usage
+
+These fixtures are automatically generated during integration tests and can be used to:
+
+1. Generate accurate mocks for unit tests
+2. Validate webhook payload structure
+3. Test normalizer implementations with real data
+4. Debug webhook processing issues
+
+## Generation
+
+Fixtures are automatically captured when running integration tests:
+
+```bash
+npm test -- sendgrid-webhook-integration
+```
+
+The webhook server automatically saves all received webhooks to the appropriate fixture files.
+

package/__tests__/fixtures/sendgrid-webhooks/deferred.json

@@ -0,0 +1,31 @@
+[
+  {
+    "headers": {
+      "host": "example-webhook.ngrok.app",
+      "user-agent": "SendGrid Event API",
+      "content-length": "498",
+      "accept-encoding": "gzip",
+      "content-type": "application/json;charset=utf-8",
+      "x-forwarded-for": "192.0.2.30",
+      "x-forwarded-host": "example-webhook.ngrok.app",
+      "x-forwarded-proto": "https",
+      "x-twilio-email-event-webhook-signature": "FAKE_SIGNATURE_MEQCIG5UlgMJ_DO_NOT_USE_IN_PRODUCTION=",
+      "x-twilio-email-event-webhook-timestamp": "1700000300"
+    },
+    "body": {
+      "attempt": "2",
+      "domain": "example.com",
+      "email": "deferred-recipient@example.com",
+      "event": "deferred",
+      "from": "Test Sender <sender@example.com>",
+      "response": "temporarily unable to deliver - server busy, please retry later",
+      "sg_event_id": "FAKE_DEFERRED_EVENT_001",
+      "sg_message_id": "FAKE_MSG_DEFERRED_001.recvd-1234567890-pqrst-1-1234567B-14.0",
+      "smtp-id": "<FAKE_MSG_DEFERRED_001@geopod-ismtpd-test>",
+      "timestamp": 1700000300,
+      "tls": 0
+    },
+    "timestamp": "2025-01-01T12:05:00.000Z",
+    "eventType": "deferred"
+  }
+]

package/__tests__/fixtures/sendgrid-webhooks/delivered.json

@@ -0,0 +1,83 @@
+[
+  {
+    "headers": {
+      "host": "example-webhook.ngrok.app",
+      "user-agent": "SendGrid Event API",
+      "content-length": "405",
+      "accept-encoding": "gzip",
+      "content-type": "application/json;charset=utf-8",
+      "x-forwarded-for": "192.0.2.1",
+      "x-forwarded-host": "example-webhook.ngrok.app",
+      "x-forwarded-proto": "https",
+      "x-twilio-email-event-webhook-signature": "FAKE_SIGNATURE_MEUCIB5bSBgVWU_DO_NOT_USE_IN_PRODUCTION=",
+      "x-twilio-email-event-webhook-timestamp": "1700000000"
+    },
+    "body": {
+      "email": "test-recipient@example.com",
+      "event": "delivered",
+      "ip": "192.0.2.10",
+      "response": "250 2.0.0 OK  1700000000 example-mail-server.example.com - gsmtp",
+      "sg_event_id": "FAKE_DELIVERED_EVENT_001",
+      "sg_message_id": "FAKE_MSG_001.recvd-1234567890-abcde-1-12345678-37.0",
+      "smtp-id": "<FAKE_MSG_001@geopod-ismtpd-test>",
+      "timestamp": 1700000000,
+      "tls": 1
+    },
+    "timestamp": "2025-01-01T12:00:00.000Z",
+    "eventType": "delivered"
+  },
+  {
+    "headers": {
+      "host": "example-webhook.ngrok.app",
+      "user-agent": "SendGrid Event API",
+      "content-length": "405",
+      "accept-encoding": "gzip",
+      "content-type": "application/json;charset=utf-8",
+      "x-forwarded-for": "192.0.2.2",
+      "x-forwarded-host": "example-webhook.ngrok.app",
+      "x-forwarded-proto": "https",
+      "x-twilio-email-event-webhook-signature": "FAKE_SIGNATURE_MEUCICpeBdSY_DO_NOT_USE_IN_PRODUCTION=",
+      "x-twilio-email-event-webhook-timestamp": "1700000100"
+    },
+    "body": {
+      "email": "test-recipient@example.com",
+      "event": "delivered",
+      "ip": "192.0.2.11",
+      "response": "250 2.0.0 OK  1700000100 example-mail-server.example.com - gsmtp",
+      "sg_event_id": "FAKE_DELIVERED_EVENT_002",
+      "sg_message_id": "FAKE_MSG_002.recvd-1234567890-fghij-1-12345679-10.0",
+      "smtp-id": "<FAKE_MSG_002@geopod-ismtpd-test>",
+      "timestamp": 1700000100,
+      "tls": 1
+    },
+    "timestamp": "2025-01-01T12:01:40.000Z",
+    "eventType": "delivered"
+  },
+  {
+    "headers": {
+      "host": "example-webhook.ngrok.app",
+      "user-agent": "SendGrid Event API",
+      "content-length": "405",
+      "accept-encoding": "gzip",
+      "content-type": "application/json;charset=utf-8",
+      "x-forwarded-for": "192.0.2.3",
+      "x-forwarded-host": "example-webhook.ngrok.app",
+      "x-forwarded-proto": "https",
+      "x-twilio-email-event-webhook-signature": "FAKE_SIGNATURE_MEQCIAektsAA_DO_NOT_USE_IN_PRODUCTION=",
+      "x-twilio-email-event-webhook-timestamp": "1700000200"
+    },
+    "body": {
+      "email": "another-recipient@example.com",
+      "event": "delivered",
+      "ip": "192.0.2.12",
+      "response": "250 2.0.0 OK  1700000200 example-mail-server.example.com - gsmtp",
+      "sg_event_id": "FAKE_DELIVERED_EVENT_003",
+      "sg_message_id": "FAKE_MSG_003.recvd-1234567890-klmno-1-1234567A-10.0",
+      "smtp-id": "<FAKE_MSG_003@geopod-ismtpd-test>",
+      "timestamp": 1700000200,
+      "tls": 1
+    },
+    "timestamp": "2025-01-01T12:03:20.000Z",
+    "eventType": "delivered"
+  }
+]

package/__tests__/fixtures/sendgrid-webhooks/open.json

@@ -0,0 +1,83 @@
+[
+  {
+    "headers": {
+      "host": "example-webhook.ngrok.app",
+      "user-agent": "SendGrid Event API",
+      "content-length": "416",
+      "accept-encoding": "gzip",
+      "content-type": "application/json;charset=utf-8",
+      "x-forwarded-for": "192.0.2.20",
+      "x-forwarded-host": "example-webhook.ngrok.app",
+      "x-forwarded-proto": "https",
+      "x-twilio-email-event-webhook-signature": "FAKE_SIGNATURE_MEQCIHN5S5nZ_DO_NOT_USE_IN_PRODUCTION=",
+      "x-twilio-email-event-webhook-timestamp": "1700000050"
+    },
+    "body": {
+      "email": "test-recipient@example.com",
+      "event": "open",
+      "ip": "192.0.2.100",
+      "sg_content_type": "html",
+      "sg_event_id": "FAKE_OPEN_EVENT_001",
+      "sg_machine_open": false,
+      "sg_message_id": "FAKE_MSG_001.recvd-1234567890-abcde-1-12345678-1B.0",
+      "timestamp": 1700000050,
+      "useragent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+    },
+    "timestamp": "2025-01-01T12:00:50.000Z",
+    "eventType": "open"
+  },
+  {
+    "headers": {
+      "host": "example-webhook.ngrok.app",
+      "user-agent": "SendGrid Event API",
+      "content-length": "416",
+      "accept-encoding": "gzip",
+      "content-type": "application/json;charset=utf-8",
+      "x-forwarded-for": "192.0.2.21",
+      "x-forwarded-host": "example-webhook.ngrok.app",
+      "x-forwarded-proto": "https",
+      "x-twilio-email-event-webhook-signature": "FAKE_SIGNATURE_MEQCIFFHHjr5_DO_NOT_USE_IN_PRODUCTION=",
+      "x-twilio-email-event-webhook-timestamp": "1700000150"
+    },
+    "body": {
+      "email": "test-recipient@example.com",
+      "event": "open",
+      "ip": "192.0.2.101",
+      "sg_content_type": "html",
+      "sg_event_id": "FAKE_OPEN_EVENT_002",
+      "sg_machine_open": false,
+      "sg_message_id": "FAKE_MSG_002.recvd-1234567890-fghij-1-12345679-10.0",
+      "timestamp": 1700000150,
+      "useragent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+    },
+    "timestamp": "2025-01-01T12:02:30.000Z",
+    "eventType": "open"
+  },
+  {
+    "headers": {
+      "host": "example-webhook.ngrok.app",
+      "user-agent": "SendGrid Event API",
+      "content-length": "366",
+      "accept-encoding": "gzip",
+      "content-type": "application/json;charset=utf-8",
+      "x-forwarded-for": "192.0.2.22",
+      "x-forwarded-host": "example-webhook.ngrok.app",
+      "x-forwarded-proto": "https",
+      "x-twilio-email-event-webhook-signature": "FAKE_SIGNATURE_MEUCIQC7bY6P_DO_NOT_USE_IN_PRODUCTION=",
+      "x-twilio-email-event-webhook-timestamp": "1700000250"
+    },
+    "body": {
+      "email": "another-recipient@example.com",
+      "event": "open",
+      "ip": "192.0.2.102",
+      "sg_content_type": "html",
+      "sg_event_id": "FAKE_OPEN_EVENT_003",
+      "sg_machine_open": true,
+      "sg_message_id": "FAKE_MSG_003.recvd-1234567890-klmno-1-1234567A-10.0",
+      "timestamp": 1700000250,
+      "useragent": "Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko Firefox/11.0 (via ggpht.com GoogleImageProxy)"
+    },
+    "timestamp": "2025-01-01T12:04:10.000Z",
+    "eventType": "open"
+  }
+]

package/__tests__/integration/ngrok-tunnel-manager.ts

@@ -0,0 +1,88 @@
+/*
+Copyright (c) 2025 Bernier LLC
+
+This file is licensed to the client under a limited-use license.
+The client may use and modify this code *only within the scope of the project it was delivered for*.
+Redistribution or use in other products or commercial offerings is not permitted without written consent from Bernier LLC.
+*/
+
+import { connect, disconnect } from '@ngrok/ngrok';
+
+export interface NgrokTunnelConfig {
+  port: number;
+  domain?: string; // Reserved domain (paid account feature)
+  authtoken?: string;
+}
+
+export class NgrokTunnelManager {
+  private listener: any;
+  private publicUrl: string | null = null;
+
+  async start(config: NgrokTunnelConfig): Promise<string> {
+    const connectOptions: any = {
+      addr: config.port,
+      authtoken: config.authtoken || process.env.NGROK_AUTH_TOKEN
+    };
+
+    // Use reserved domain if available (paid account feature)
+    if (config.domain) {
+      connectOptions.domain = config.domain;
+    }
+
+    try {
+      // Disconnect any existing tunnels first (in case of previous failed runs)
+      try {
+        await disconnect();
+        console.log('Disconnected existing ngrok tunnels');
+      } catch {
+        // Ignore errors if no tunnels exist
+      }
+
+      this.listener = await connect(connectOptions);
+      this.publicUrl = this.listener.url();
+      
+      if (!this.publicUrl) {
+        throw new Error('Failed to get ngrok tunnel URL');
+      }
+      
+      console.log(`ngrok tunnel established: ${this.publicUrl}`);
+      return this.publicUrl;
+    } catch (error) {
+      const errorMessage = error instanceof Error ? error.message : String(error);
+      throw new Error(`Failed to start ngrok tunnel: ${errorMessage}`);
+    }
+  }
+
+  async stop(): Promise<void> {
+    if (this.listener) {
+      try {
+        // Use disconnect with the URL, or close the listener directly
+        const url = this.listener.url();
+        if (url) {
+          const { disconnect } = await import('@ngrok/ngrok');
+          await disconnect(url);
+        }
+        this.listener = null;
+        this.publicUrl = null;
+        console.log('ngrok tunnel closed');
+      } catch (error) {
+        const errorMessage = error instanceof Error ? error.message : String(error);
+        console.warn(`Error closing ngrok tunnel: ${errorMessage}`);
+      }
+    }
+  }
+
+  getUrl(): string | null {
+    return this.publicUrl;
+  }
+
+  // Get tunnel statistics (paid account feature)
+  async getStats(): Promise<any> {
+    if (!this.listener) {
+      throw new Error('Tunnel not started');
+    }
+    // Access tunnel metrics via ngrok API
+    return this.listener;
+  }
+}
+