payload-plugin-newsletter 0.20.6 → 0.21.0

package/CHANGELOG.md

@@ -1,3 +1,74 @@
+## [0.21.0] - 2025-01-31
+
+### Added
+- **Webhook Support**: Real-time webhook integration for Broadcast provider
+  - Automatic updates for subscriber events (subscribed/unsubscribed)
+  - Real-time broadcast status updates (scheduled, in_progress, sent, etc.)
+  - HMAC-SHA256 signature verification for security
+  - Webhook configuration UI in Newsletter Settings
+  - Manual webhook registration with setup instructions
+
+### Changed
+- **BREAKING**: Removed polling-based unsubscribe synchronization
+  - Webhooks now provide real-time updates instead of scheduled polling
+  - Removed `unsubscribeSync` configuration option
+  - Removed `afterUnsubscribeSync` hook
+  - Removed sync job infrastructure
+
+### Security
+- Added webhook signature verification with timestamp validation
+- Webhook secrets stored securely in Newsletter Settings
+
+### Migration Guide
+If you were using unsubscribe sync:
+1. Remove any `unsubscribeSync` configuration from your plugin setup
+2. Configure webhooks in your Broadcast dashboard (see README)
+3. Add the webhook secret to your Newsletter Settings
+
+## [0.20.7] - 2025-08-02
+
+### Added
+- **Comprehensive Email Wrapper System** - Unified email rendering that uses the same wrapper for previews and actual sends
+  - Preview component now passes ALL form fields to the preview endpoint as `documentData`
+  - Custom wrappers receive complete document data, not just content
+  - Consistent HTML output between preview and actual email sends
+  - Support for accessing any custom fields added to broadcasts collection
+  - Wrapper is applied during actual send operations, not just previews
+
+### Enhanced
+- **Email Preview Component** - Builds document data from all form fields for wrapper access
+  - Collects all field values using `useFormFields` hook
+  - Passes complete form state to preview endpoint
+  - Enables custom wrappers to access metadata like slug, issueNumber, custom fields
+
+- **Preview Endpoint** - Updated to accept and forward documentData to email wrapper
+  - Receives documentData from preview component
+  - Passes through to convertToEmailSafeHtml function
+  - Maintains backward compatibility with existing implementations
+
+- **Email Wrapper Interface** - Enhanced type definitions for better developer experience
+  - Added `EmailWrapperOptions` interface with documentData support
+  - Updated `CustomEmailWrapper` type for consistency
+  - Generic `Record<string, any>` allows any custom fields
+
+- **Broadcast Sync Hooks** - Now apply wrapper during actual email sends
+  - Create operation uses wrapper when syncing to provider
+  - Update operation maintains wrapper consistency
+  - Deferred creates (from draft to published) include wrapper
+  - Same HTML template used for both preview and actual sends
+
+### Technical Changes
+- Updated `convertToEmailSafeHtml` to accept documentData in options
+- Modified all sync operations in Broadcasts collection to use email preview config
+- Wrapper customization now available throughout the email lifecycle
+- No breaking changes - wrapper remains optional with sensible defaults
+
+### Benefits
+- **Flexibility** - Users can pass any custom fields without plugin modifications
+- **Consistency** - Identical HTML in preview and actual email sends
+- **Future Proof** - Add new fields to broadcasts without updating plugin code
+- **No Breaking Changes** - Existing implementations continue to work unchanged
+
 ## [0.20.6] - 2025-08-01
 
 ### Fixed

package/README.md

@@ -17,7 +17,7 @@ A complete newsletter management plugin for [Payload CMS](https://github.com/pay
 - 🌍 **Internationalization** - Multi-language support built-in
 - 📊 **Analytics Ready** - UTM tracking and signup metadata collection
 - ⚙️ **Admin UI Configuration** - Manage email settings through Payload admin panel
-- 🔄 **Bidirectional Sync** - Sync unsubscribes from email services back to Payload
+- 🔄 **Real-time Webhook Sync** - Receive subscriber and broadcast events from email services via webhooks
 - 👁️ **Email Preview** - Real-time preview with desktop/mobile views (v0.9.0+)
 - ✅ **Email Validation** - Built-in validation for email client compatibility (v0.9.0+)
 - 📝 **Email-Safe Editor** - Rich text editor limited to email-compatible features (v0.9.0+)
@@ -654,27 +654,44 @@ This adds a "Newsletter Scheduling" group to your articles with:
 - Audience segment selection
 - Send status tracking
 
-## Unsubscribe Sync
+## Webhook Configuration (Broadcast)
 
-The plugin supports bidirectional synchronization of unsubscribe states between Payload and your email service:
+The plugin supports real-time webhook integration with Broadcast for instant updates:
 
-```typescript
-features: {
-  unsubscribeSync: {
-    enabled: true,
-    schedule: '0 * * * *', // Hourly sync
-    queue: 'newsletter-sync' // Optional custom queue name
-  }
-}
-```
+### Automatic Updates
+
+When configured, the plugin automatically receives and processes:
+- **Subscriber Events**: `subscribed`, `unsubscribed`
+- **Broadcast Events**: All status changes (`scheduled`, `in_progress`, `sent`, etc.)
+
+### Setup Instructions
+
+1. **Save your Newsletter Settings** in the Payload admin to generate a webhook URL
+2. **Configure in Broadcast**:
+   - Go to your Broadcast dashboard → "Webhook Endpoints"
+   - Click "Add Webhook Endpoint"
+   - Paste the webhook URL from Payload
+   - Select events:
+     - Subscriber Events: `subscribed`, `unsubscribed`
+     - Broadcast Events: All
+   - Create the webhook and copy the webhook secret
+3. **Add the webhook secret** to your Newsletter Settings in Payload
+4. **Save and verify** the webhook connection
+
+### Security
+
+Webhooks are secured with:
+- HMAC-SHA256 signature verification
+- Timestamp validation (5-minute window)
+- Secret key stored in Newsletter Settings
+
+### Data Flow
 
-This feature:
-- Polls your email service for unsubscribed users
-- Updates their status in Payload automatically
-- Supports both Broadcast and Resend providers
-- Can run on a schedule or be triggered manually
+- **Subscriber events** update the subscriber's status and metadata
+- **Broadcast events** update the broadcast's status and delivery metrics
+- All updates happen in real-time without polling
 
-For more details, see the [Unsubscribe Sync documentation](./docs/unsubscribe-sync.md).
+**Note**: Email engagement events (opens, clicks) remain in Broadcast for analytics.
 
 ## Email Providers
 

package/dist/admin.d.ts

@@ -22,4 +22,6 @@ interface EmailPreviewProps {
 }
 declare const EmailPreview: React.FC<EmailPreviewProps>;
 
-export { BroadcastInlinePreview, type BroadcastInlinePreviewProps, EmailPreview, type EmailPreviewProps, StatusBadge, type StatusBadgeProps };
+declare const WebhookConfiguration: React.FC;
+
+export { BroadcastInlinePreview, type BroadcastInlinePreviewProps, EmailPreview, type EmailPreviewProps, StatusBadge, type StatusBadgeProps, WebhookConfiguration };

package/dist/admin.js

@@ -22,10 +22,20 @@ var BroadcastInlinePreview = () => {
         setLoading(false);
         return;
       }
+      const documentData = {};
+      Object.entries(fields || {}).forEach(([key, field]) => {
+        if (field && typeof field === "object" && "value" in field) {
+          documentData[key] = field.value;
+        }
+      });
       const response = await fetch("/api/broadcasts/preview", {
         method: "POST",
         headers: { "Content-Type": "application/json" },
-        body: JSON.stringify({ content: contentValue })
+        body: JSON.stringify({
+          content: contentValue,
+          documentData
+          // Pass all form data
+        })
       });
       if (!response.ok) {
         throw new Error(`Preview failed: ${response.statusText}`);
@@ -178,8 +188,153 @@ var EmailPreview = ({
     ] })
   ] });
 };
+
+// src/admin/components/WebhookConfiguration.tsx
+import { useState as useState2 } from "react";
+import { useFormFields as useFormFields2 } from "@payloadcms/ui";
+import { Button } from "@payloadcms/ui";
+import { toast } from "@payloadcms/ui";
+import { jsx as jsx4, jsxs as jsxs3 } from "react/jsx-runtime";
+var WebhookConfiguration = () => {
+  const [showInstructions, setShowInstructions] = useState2(false);
+  const [verifying, setVerifying] = useState2(false);
+  const fields = useFormFields2(([fields2]) => ({
+    webhookUrl: fields2?.broadcastSettings?.webhookUrl,
+    webhookStatus: fields2?.broadcastSettings?.webhookStatus,
+    lastWebhookReceived: fields2?.broadcastSettings?.lastWebhookReceived
+  }));
+  const handleVerify = async () => {
+    setVerifying(true);
+    try {
+      const response = await fetch("/api/newsletter/webhooks/verify", {
+        method: "POST",
+        headers: {
+          "Content-Type": "application/json"
+        }
+      });
+      const data = await response.json();
+      if (data.success) {
+        toast.success(data.message || "Webhook verified");
+      } else {
+        toast.error(data.error || "Verification failed");
+      }
+    } catch (error) {
+      toast.error("Failed to verify webhook");
+    } finally {
+      setVerifying(false);
+    }
+  };
+  const getStatusColor = (status) => {
+    switch (status) {
+      case "verified":
+        return "green";
+      case "configured":
+        return "yellow";
+      case "error":
+        return "red";
+      default:
+        return "gray";
+    }
+  };
+  const getStatusLabel = (status) => {
+    switch (status) {
+      case "verified":
+        return "Verified \u2713";
+      case "configured":
+        return "Configured";
+      case "error":
+        return "Error";
+      default:
+        return "Not Configured";
+    }
+  };
+  return /* @__PURE__ */ jsx4("div", { className: "field-type", children: /* @__PURE__ */ jsxs3("div", { style: { marginBottom: "1rem" }, children: [
+    /* @__PURE__ */ jsx4("label", { className: "field-label", children: "Webhook Configuration" }),
+    /* @__PURE__ */ jsxs3("div", { style: {
+      background: "#f5f5f5",
+      padding: "1rem",
+      borderRadius: "4px",
+      marginTop: "0.5rem"
+    }, children: [
+      /* @__PURE__ */ jsxs3("div", { style: { marginBottom: "0.5rem" }, children: [
+        /* @__PURE__ */ jsx4("strong", { children: "Webhook URL:" }),
+        /* @__PURE__ */ jsx4("code", { style: {
+          display: "block",
+          padding: "0.5rem",
+          background: "#fff",
+          border: "1px solid #ddd",
+          borderRadius: "4px",
+          marginTop: "0.25rem",
+          fontSize: "0.875rem"
+        }, children: fields.webhookUrl || "Save settings to generate URL" })
+      ] }),
+      /* @__PURE__ */ jsxs3("div", { style: { marginBottom: "0.5rem" }, children: [
+        /* @__PURE__ */ jsx4("strong", { children: "Status:" }),
+        " ",
+        /* @__PURE__ */ jsx4("span", { style: { color: getStatusColor(fields.webhookStatus || "not_configured") }, children: getStatusLabel(fields.webhookStatus || "not_configured") })
+      ] }),
+      fields.lastWebhookReceived && /* @__PURE__ */ jsxs3("div", { style: { fontSize: "0.875rem", color: "#666" }, children: [
+        "Last event: ",
+        new Date(fields.lastWebhookReceived).toLocaleString()
+      ] })
+    ] }),
+    /* @__PURE__ */ jsxs3("div", { style: { marginTop: "1rem", display: "flex", gap: "0.5rem" }, children: [
+      /* @__PURE__ */ jsxs3(
+        Button,
+        {
+          onClick: () => setShowInstructions(!showInstructions),
+          buttonStyle: "secondary",
+          size: "small",
+          children: [
+            showInstructions ? "Hide" : "Show",
+            " Instructions"
+          ]
+        }
+      ),
+      /* @__PURE__ */ jsx4(
+        Button,
+        {
+          onClick: handleVerify,
+          buttonStyle: "primary",
+          size: "small",
+          disabled: verifying || !fields.webhookUrl,
+          children: verifying ? "Verifying..." : "Verify Webhook"
+        }
+      )
+    ] }),
+    showInstructions && /* @__PURE__ */ jsxs3("div", { style: {
+      marginTop: "1rem",
+      padding: "1rem",
+      background: "#f0f8ff",
+      border: "1px solid #b0d4ff",
+      borderRadius: "4px"
+    }, children: [
+      /* @__PURE__ */ jsx4("h4", { style: { marginTop: 0 }, children: "Configure Broadcast Webhook" }),
+      /* @__PURE__ */ jsxs3("ol", { children: [
+        /* @__PURE__ */ jsx4("li", { children: "Copy the webhook URL above" }),
+        /* @__PURE__ */ jsx4("li", { children: "Go to your Broadcast dashboard" }),
+        /* @__PURE__ */ jsx4("li", { children: 'Navigate to "Webhook Endpoints"' }),
+        /* @__PURE__ */ jsx4("li", { children: 'Click "Add Webhook Endpoint"' }),
+        /* @__PURE__ */ jsx4("li", { children: "Paste the URL" }),
+        /* @__PURE__ */ jsxs3("li", { children: [
+          "Select these events:",
+          /* @__PURE__ */ jsxs3("ul", { children: [
+            /* @__PURE__ */ jsx4("li", { children: "Subscriber Events: subscribed, unsubscribed" }),
+            /* @__PURE__ */ jsx4("li", { children: "Broadcast Events: All" })
+          ] })
+        ] }),
+        /* @__PURE__ */ jsx4("li", { children: 'Click "Create Webhook"' }),
+        /* @__PURE__ */ jsx4("li", { children: "Copy the webhook secret shown" }),
+        /* @__PURE__ */ jsx4("li", { children: "Paste it in the Webhook Secret field below" }),
+        /* @__PURE__ */ jsx4("li", { children: "Save these settings" }),
+        /* @__PURE__ */ jsx4("li", { children: 'Click "Verify Webhook" to test the connection' })
+      ] })
+    ] })
+  ] }) });
+};
 export {
   BroadcastInlinePreview,
   EmailPreview,
-  StatusBadge
+  StatusBadge,
+  WebhookConfiguration
 };

package/dist/collections.cjs

@@ -979,7 +979,8 @@ async function convertToEmailSafeHtml(editorState, options) {
     if (options.customWrapper) {
       return await Promise.resolve(options.customWrapper(sanitizedHtml, {
         preheader: options.preheader,
-        subject: options.subject
+        subject: options.subject,
+        documentData: options.documentData
       }));
     }
     return wrapInEmailTemplate(sanitizedHtml, options.preheader);
@@ -1866,7 +1867,7 @@ var createBroadcastPreviewEndpoint = (config, _collectionSlug) => {
     handler: async (req) => {
       try {
         const data = await (req.json?.() || Promise.resolve({}));
-        const { content, preheader, subject } = data;
+        const { content, preheader, subject, documentData } = data;
         if (!content) {
           return Response.json({
             success: false,
@@ -1882,6 +1883,8 @@ var createBroadcastPreviewEndpoint = (config, _collectionSlug) => {
           preheader,
           subject,
           mediaUrl,
+          documentData,
+          // Pass all document data
           customBlockConverter: config.customizations?.broadcasts?.customBlockConverter,
           customWrapper: emailPreviewConfig?.customWrapper
         });
@@ -2151,6 +2154,14 @@ var createBroadcastsCollection = (pluginConfig) => {
           condition: (data) => hasProviders && data?.providerId
         }
       },
+      {
+        name: "externalId",
+        type: "text",
+        admin: {
+          readOnly: true,
+          description: "External ID for webhook integration"
+        }
+      },
       {
         name: "providerData",
         type: "json",
@@ -2159,6 +2170,77 @@ var createBroadcastsCollection = (pluginConfig) => {
           condition: () => false
           // Hidden by default
         }
+      },
+      // Webhook tracking fields
+      {
+        name: "webhookData",
+        type: "group",
+        label: "Webhook Data",
+        admin: {
+          condition: () => false
+          // Hidden by default, used for webhook tracking
+        },
+        fields: [
+          {
+            name: "lastWebhookEvent",
+            type: "text",
+            admin: {
+              readOnly: true
+            }
+          },
+          {
+            name: "lastWebhookEventAt",
+            type: "date",
+            admin: {
+              readOnly: true
+            }
+          },
+          {
+            name: "hasWarnings",
+            type: "checkbox",
+            defaultValue: false
+          },
+          {
+            name: "failureReason",
+            type: "text"
+          },
+          {
+            name: "sentCount",
+            type: "number"
+          },
+          {
+            name: "totalCount",
+            type: "number"
+          },
+          {
+            name: "failedCount",
+            type: "number"
+          },
+          {
+            name: "remainingCount",
+            type: "number"
+          },
+          {
+            name: "sendingStartedAt",
+            type: "date"
+          },
+          {
+            name: "failedAt",
+            type: "date"
+          },
+          {
+            name: "abortedAt",
+            type: "date"
+          },
+          {
+            name: "abortReason",
+            type: "text"
+          },
+          {
+            name: "pausedAt",
+            type: "date"
+          }
+        ]
       }
     ],
     hooks: {
@@ -2181,7 +2263,14 @@ var createBroadcastsCollection = (pluginConfig) => {
               const provider = new BroadcastApiProvider2(providerConfig);
               req.payload.logger.info("Populating media fields and converting content to HTML...");
               const populatedContent = await populateMediaFields(doc.contentSection?.content, req.payload, pluginConfig);
+              const emailPreviewConfig = pluginConfig.customizations?.broadcasts?.emailPreview;
               const htmlContent = await convertToEmailSafeHtml(populatedContent, {
+                wrapInTemplate: emailPreviewConfig?.wrapInTemplate ?? true,
+                customWrapper: emailPreviewConfig?.customWrapper,
+                preheader: doc.contentSection?.preheader,
+                subject: doc.subject,
+                documentData: doc,
+                // Pass entire document
                 customBlockConverter: pluginConfig.customizations?.broadcasts?.customBlockConverter
               });
               if (!htmlContent || htmlContent.trim() === "") {
@@ -2281,7 +2370,14 @@ var createBroadcastsCollection = (pluginConfig) => {
                 }
                 req.payload.logger.info("Creating broadcast in provider (deferred from initial create)...");
                 const populatedContent = await populateMediaFields(doc.contentSection?.content, req.payload, pluginConfig);
+                const emailPreviewConfig = pluginConfig.customizations?.broadcasts?.emailPreview;
                 const htmlContent = await convertToEmailSafeHtml(populatedContent, {
+                  wrapInTemplate: emailPreviewConfig?.wrapInTemplate ?? true,
+                  customWrapper: emailPreviewConfig?.customWrapper,
+                  preheader: doc.contentSection?.preheader,
+                  subject: doc.subject,
+                  documentData: doc,
+                  // Pass entire document
                   customBlockConverter: pluginConfig.customizations?.broadcasts?.customBlockConverter
                 });
                 if (!htmlContent || htmlContent.trim() === "") {
@@ -2343,7 +2439,14 @@ var createBroadcastsCollection = (pluginConfig) => {
                   }
                   if (JSON.stringify(doc.contentSection?.content) !== JSON.stringify(previousDoc?.contentSection?.content)) {
                     const populatedContent = await populateMediaFields(doc.contentSection?.content, req.payload, pluginConfig);
+                    const emailPreviewConfig = pluginConfig.customizations?.broadcasts?.emailPreview;
                     updates.content = await convertToEmailSafeHtml(populatedContent, {
+                      wrapInTemplate: emailPreviewConfig?.wrapInTemplate ?? true,
+                      customWrapper: emailPreviewConfig?.customWrapper,
+                      preheader: doc.contentSection?.preheader,
+                      subject: doc.subject,
+                      documentData: doc,
+                      // Pass entire document
                       customBlockConverter: pluginConfig.customizations?.broadcasts?.customBlockConverter
                     });
                   }
@@ -2779,6 +2882,15 @@ var createSubscribersCollection = (pluginConfig) => {
       type: "date",
       hidden: true
     },
+    // External ID for webhook integration
+    {
+      name: "externalId",
+      type: "text",
+      admin: {
+        description: "ID from email service provider",
+        readOnly: true
+      }
+    },
     // Subscription status
     {
       name: "subscriptionStatus",
@@ -2794,6 +2906,14 @@ var createSubscribersCollection = (pluginConfig) => {
         description: "Current subscription status"
       }
     },
+    {
+      name: "subscribedAt",
+      type: "date",
+      admin: {
+        description: "When the user subscribed",
+        readOnly: true
+      }
+    },
     {
       name: "unsubscribedAt",
       type: "date",
@@ -2803,6 +2923,15 @@ var createSubscribersCollection = (pluginConfig) => {
         readOnly: true
       }
     },
+    {
+      name: "unsubscribeReason",
+      type: "text",
+      admin: {
+        condition: (data) => data?.subscriptionStatus === "unsubscribed",
+        description: "Reason for unsubscribing",
+        readOnly: true
+      }
+    },
     // Email preferences
     {
       name: "emailPreferences",