payload-wordpress-migrator 0.0.22

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Brightscout
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,586 @@
+# Payload WordPress Migrator Plugin
+
+A comprehensive PayloadCMS plugin for WordPress migration - migrate and manage WordPress content directly in your Payload admin dashboard with a powerful React-based interface.
+
+## 🚀 Features
+
+- 📊 **Interactive Migration Dashboard**: Real-time progress tracking with a modern React interface
+- 🔄 **WordPress API Integration**: Secure REST API connectivity with application password authentication
+- 📋 **Smart Job Management**: Create, monitor, pause, resume, and retry migration jobs
+- 🎯 **Content Type Discovery**: Automatic detection of posts, pages, media, categories, and custom post types
+- 🔗 **Advanced Field Mapping**: Visual field mapping interface with dot-notation support
+- 🖼️ **Intelligent Media Migration**: Auto-import featured images and content media with duplicate detection
+- 🧱 **Gutenberg to Lexical**: Convert WordPress blocks to PayloadCMS Lexical format
+- ⚡ **Performance Optimized**: Configurable batch processing with background job execution
+- 🔄 **Resume & Retry**: Intelligent job recovery with progress preservation
+- 🔒 **Production Ready**: Comprehensive error handling and SSL certificate management
+
+## 📦 Installation
+
+```bash
+# npm
+npm install payload-wordpress-migrator
+
+# yarn
+yarn add payload-wordpress-migrator
+
+# pnpm
+pnpm add payload-wordpress-migrator
+```
+
+## 🚀 Quick Start
+
+### Basic Setup
+
+Add the plugin to your Payload config:
+
+```typescript
+import { buildConfig } from 'payload'
+import { payloadWordPressMigrator } from 'payload-wordpress-migrator'
+
+export default buildConfig({
+  // ... your existing config
+  plugins: [
+    payloadWordPressMigrator({
+      wpSiteUrl: process.env.WORDPRESS_API_URL,
+      wpUsername: process.env.WORDPRESS_USERNAME,
+      wpPassword: process.env.WORDPRESS_APP_PASSWORD,
+      collections: {
+        posts: { wpPostType: 'post', enableBlocks: true },
+        pages: { wpPostType: 'page', enableBlocks: true },
+        media: { wpPostType: 'media' },
+      },
+      enableMediaDownload: true,
+      migrationBatchSize: 25,
+    }),
+  ],
+})
+```
+
+### Environment Variables
+
+Add these to your `.env` file:
+
+```env
+WORDPRESS_API_URL=https://your-wordpress-site.com
+WORDPRESS_USERNAME=your-username
+WORDPRESS_APP_PASSWORD=your-application-password
+```
+
+### WordPress Setup
+
+#### Application Password Setup
+
+1. In WordPress admin, go to **Users** → **Profile**
+2. Scroll down to **Application Passwords**
+3. Enter a name (e.g., "Payload Migration")
+4. Click **Add New Application Password**
+5. Copy the generated password (save it securely)
+6. Use this password as `WORDPRESS_APP_PASSWORD`
+
+#### REST API Requirements
+
+- WordPress 5.6+ (for Application Passwords)
+- REST API enabled (default in most installations)
+- User with appropriate permissions for content access
+
+### Configuration Options
+
+```typescript
+type PayloadWordPressMigratorConfig = {
+  // WordPress Connection
+  wpSiteUrl?: string // WordPress site URL
+  wpUsername?: string // WordPress username
+  wpPassword?: string // WordPress application password
+
+  // Collection Configuration
+  collections?: Partial<Record<CollectionSlug, WordPressCollectionMapping>>
+
+  // Plugin Control
+  disabled?: boolean // Disable plugin functionality
+  disableDashboard?: boolean // Disable dashboard UI
+
+  // Performance Settings
+  migrationBatchSize?: number // Batch size (default: 10)
+  enableAutoSync?: boolean // Auto-sync on startup
+
+  // Media Migration
+  enableMediaDownload?: boolean // Download actual files (default: false)
+  maxMediaFileSize?: number // Max file size in bytes (default: 10MB)
+  allowedMediaTypes?: string[] // Allowed MIME types
+  allowSelfSignedCerts?: boolean // Allow self-signed SSL (dev only)
+}
+
+type WordPressCollectionMapping = {
+  wpPostType: string // WordPress content type
+  fieldMapping?: Record<string, string> // Custom field mapping
+  enableBlocks?: boolean // Convert Gutenberg blocks
+  customFields?: string[] // ACF field names to migrate
+  importContentMedia?: boolean // Auto-import content images
+}
+```
+
+## 📋 Plugin Architecture
+
+The plugin creates the following collections and components in your PayloadCMS instance:
+
+### Collections Added
+
+1. **`wordpress-migration`**: Core collection for managing migration jobs
+   - Job configuration and status tracking
+   - Progress monitoring with detailed logs
+   - Built-in dashboard interface
+
+2. **Enhanced Target Collections**: Automatically adds WordPress metadata fields to configured collections:
+   - `migratedFromWordPress.wpPostId`: Original WordPress post ID
+   - `migratedFromWordPress.wpPostType`: WordPress content type
+   - `migratedFromWordPress.migrationDate`: Migration timestamp
+
+### UI Components
+
+- **Migration Dashboard**: React-based interface with real-time updates
+- **WordPress Site Configuration**: Secure credential management with localStorage persistence
+- **Content Type Selector**: Dynamic dropdown populated from WordPress API discovery
+- **Field Mapping Interface**: Visual field mapping with source/destination field analysis
+- **Job Monitor**: Live progress tracking with success/failure metrics
+
+### API Endpoints
+
+The plugin adds these endpoints to your PayloadCMS API:
+
+```
+POST   /api/wordpress/test-connection      # Test WordPress connectivity
+GET    /api/wordpress/migration-summary    # Dashboard data with caching
+POST   /api/wordpress/discover-content     # Content type discovery
+GET    /api/wordpress/migration-jobs       # Job status retrieval
+POST   /api/wordpress/migration-jobs       # Start migration jobs
+PUT    /api/wordpress/migration-jobs       # Pause/resume jobs
+DELETE /api/wordpress/migration-jobs       # Delete jobs
+POST   /api/wordpress/content-fields       # Analyze WordPress fields
+GET    /api/collections/:slug/fields       # Analyze PayloadCMS fields
+```
+
+## 🎯 Content Migration Features
+
+### Supported Content Types
+
+| Content Type          | Features                                                | Notes                             |
+| --------------------- | ------------------------------------------------------- | --------------------------------- |
+| **Posts**             | ✅ Content, metadata, categories, tags, featured images | Full Gutenberg block conversion   |
+| **Pages**             | ✅ Content, metadata, featured images, custom fields    | Hierarchical structure support    |
+| **Media**             | ✅ File downloads, metadata, alt text, captions         | MIME type filtering & size limits |
+| **Categories**        | ✅ Names, descriptions, hierarchy, counts               | Taxonomy relationships            |
+| **Tags**              | ✅ Names, descriptions, post associations               | Tag cloud support                 |
+| **Users**             | ✅ User data, roles, capabilities, avatars              | Permission mapping                |
+| **Custom Post Types** | ✅ Auto-discovery, custom fields, metadata              | ACF integration                   |
+
+### Advanced Content Processing
+
+- **HTML to Lexical Conversion**: Sophisticated converter handles complex HTML structures
+- **Gutenberg Block Processing**: Converts blocks to PayloadCMS block format
+- **ACF Field Migration**: Automatic Advanced Custom Fields detection and migration
+- **Media Auto-Import**: Scans content for WordPress media URLs and imports automatically
+- **Relationship Preservation**: Maintains post-category, post-tag, and other relationships
+
+## 🖼️ Media Migration
+
+### Migration Modes
+
+The plugin supports two modes for media migration:
+
+1. **Metadata Only** (default): Migrates media titles, descriptions, alt text, and URLs
+2. **Full File Migration**: Downloads files from WordPress and uploads them to PayloadCMS
+
+### Enabling Media File Migration
+
+```typescript
+payloadWordPressMigrator({
+  enableMediaDownload: true, // Enable file downloads
+  maxMediaFileSize: 50 * 1024 * 1024, // 50MB limit
+  allowedMediaTypes: [
+    'image/jpeg',
+    'image/png',
+    'image/gif',
+    'image/webp',
+    'application/pdf',
+    'video/mp4',
+  ],
+  allowSelfSignedCerts: true, // For development only
+})
+```
+
+### Auto-Import Features
+
+#### Featured Image Auto-Import
+
+- Automatically detects and imports featured images using WordPress `_embed` data
+- Fallback support when `_embed` data is unavailable
+- Creates PayloadCMS media records with actual file downloads
+- Links imported media to posts/pages via `featuredImage` field
+
+#### Content Media Auto-Import
+
+- Scans HTML content for WordPress media URLs (`wp-content/uploads`)
+- Downloads and imports images referenced in post/page content
+- Preserves alt text and title attributes
+- Smart duplicate detection to avoid re-importing existing files
+
+#### MediaBlock Conversion
+
+When migrating content with `enableMediaDownload: true`, the plugin now converts HTML `<img>` tags to proper PayloadCMS MediaBlocks:
+
+- **HTML Images → MediaBlocks**: Converts inline images to structured MediaBlock components
+- **Media Import Integration**: Automatically downloads and links media files to MediaBlocks
+- **Rich Editor Support**: MediaBlocks display properly in PayloadCMS Lexical editor
+- **Frontend Rendering**: Works seamlessly with PayloadCMS RichText component
+- **Metadata Preservation**: Maintains alt text, titles, and dimensions for accessibility
+
+#### Configuration Example
+
+```typescript
+collections: {
+  posts: {
+    wpPostType: 'post',
+    importContentMedia: true,    // Enable content media auto-import
+  },
+  media: {
+    wpPostType: 'media'         // Required for media storage
+  }
+},
+enableMediaDownload: true       // Required for auto-import
+```
+
+### Migration Workflow Options
+
+1. **Recommended**: Migrate posts/pages only - media will be auto-imported
+2. **Alternative**: Migrate media first, then posts/pages (avoids duplicates)
+
+### Performance Considerations
+
+- Files processed in configurable batches to prevent memory issues
+- Large files streamed to minimize memory usage
+- Failed downloads don't stop the entire migration process
+- Media auto-import runs in parallel for optimal performance
+- Smart duplicate detection prevents re-importing existing media
+
+## 🔧 Advanced Configuration
+
+### Complete Configuration Example
+
+```typescript
+import { buildConfig } from 'payload'
+import { payloadWordPressMigrator } from 'payload-wordpress-migrator'
+
+export default buildConfig({
+  plugins: [
+    payloadWordPressMigrator({
+      // WordPress Connection
+      wpSiteUrl: process.env.WORDPRESS_API_URL,
+      wpUsername: process.env.WORDPRESS_USERNAME,
+      wpPassword: process.env.WORDPRESS_APP_PASSWORD,
+
+      // Collection Configuration
+      collections: {
+        posts: {
+          wpPostType: 'post',
+          enableBlocks: true,
+          importContentMedia: true,
+          customFields: ['_yoast_wpseo_title', '_yoast_wpseo_metadesc'],
+          fieldMapping: {
+            'title.rendered': 'title',
+            'content.rendered': 'content',
+            'excerpt.rendered': 'excerpt',
+          },
+        },
+        pages: {
+          wpPostType: 'page',
+          enableBlocks: true,
+          importContentMedia: true,
+          disableHtmlConversion: false, // Enable HTML to Lexical conversion
+        },
+        media: {
+          wpPostType: 'media',
+        },
+        categories: {
+          wpPostType: 'category',
+        },
+        products: {
+          // Custom post type example
+          wpPostType: 'product',
+          enableBlocks: false,
+          customFields: ['_price', '_stock_status', '_featured'],
+        },
+      },
+
+      // Performance & Batch Settings
+      migrationBatchSize: 25,
+      enableAutoSync: false,
+
+      // Media Configuration
+      enableMediaDownload: true,
+      maxMediaFileSize: 50 * 1024 * 1024, // 50MB
+      allowedMediaTypes: [
+        'image/jpeg',
+        'image/png',
+        'image/gif',
+        'image/webp',
+        'image/svg+xml',
+        'application/pdf',
+        'video/mp4',
+        'video/webm',
+        'audio/mpeg',
+      ],
+
+      // Development Settings
+      allowSelfSignedCerts: process.env.NODE_ENV === 'development',
+
+      // Plugin Control
+      disabled: false,
+      disableDashboard: false,
+    }),
+  ],
+})
+```
+
+### Field Mapping Configuration
+
+The plugin supports sophisticated field mapping with dot notation:
+
+```typescript
+fieldMapping: {
+  // WordPress field path → PayloadCMS field path
+  'title.rendered': 'title',
+  'content.rendered': 'content',
+  'meta._yoast_wpseo_title': 'seo.title',
+  'acf.hero_image': 'hero.image',
+  'custom_field_name': 'mappedField'
+}
+```
+
+## 🎛️ Migration Dashboard
+
+The plugin provides a comprehensive React-based dashboard accessible at `/admin/collections/wordpress-migration`:
+
+### Dashboard Features
+
+- **Site Configuration**: Secure WordPress credentials management
+- **Content Discovery**: Automatic scan and analysis of available content
+- **Job Creation**: Visual job setup with field mapping interface
+- **Real-time Monitoring**: Live progress updates with success/failure metrics
+- **Job Controls**: Start, pause, resume, retry operations
+- **Error Reporting**: Detailed logs and error analysis
+
+### Dashboard Screenshots
+
+The interface includes:
+
+1. **Configuration Panel**: WordPress site setup and testing
+2. **Content Overview**: Discovered content types with item counts
+3. **Job Management**: Active and completed migration jobs
+4. **Progress Tracking**: Visual progress bars with detailed statistics
+5. **Log Viewer**: Real-time migration logs and error reporting
+
+## 🔌 Programmatic API
+
+The plugin exports utilities that can be used outside the dashboard for scripted or custom migrations.
+
+### WordPress Client
+
+```typescript
+import { WordPressClient } from 'payload-wordpress-migrator/dist/utils/wordpress/index.js'
+
+const client = new WordPressClient({
+  wpSiteUrl: 'https://example.com',
+  wpUsername: 'admin',
+  wpPassword: 'xxxx xxxx xxxx xxxx',
+})
+
+// Discover available content types
+const contentTypes = await client.discoverContent()
+
+// Fetch fields for a specific content type
+const fields = await client.fetchContentFields('post')
+```
+
+### Content Transformation
+
+```typescript
+import { transformWordPressContent } from 'payload-wordpress-migrator/dist/utils/content/index.js'
+
+const payloadData = await transformWordPressContent(
+  wpItem,       // WordPress REST API item
+  'post',       // Content type
+  jobConfig,    // Job configuration (batchSize, enableBlocks, fieldMapping)
+  payload,      // Payload instance
+  pluginOptions // Plugin config
+)
+```
+
+### HTML to Lexical Conversion
+
+```typescript
+import { convertHtmlToLexical } from 'payload-wordpress-migrator/dist/utils/lexical/index.js'
+
+const lexicalContent = convertHtmlToLexical('<p>Hello <strong>world</strong></p>')
+// Returns a Lexical root node ready for PayloadCMS rich text fields
+```
+
+### Field Mapping
+
+```typescript
+import { applyFieldMapping } from 'payload-wordpress-migrator/dist/utils/content/index.js'
+
+const mappedData = await applyFieldMapping(
+  payloadData,  // Transformed Payload data
+  wpItem,       // Original WordPress item
+  fieldMapping, // { fieldMappings: [{ sourceField, destinationField }] }
+  payload       // Payload instance (needed for relationship ID conversion)
+)
+```
+
+> **Note:** These utilities are imported from internal paths (`dist/utils/...`). The public package exports (`payload-wordpress-migrator`) expose only the plugin function and types. Internal APIs may change between minor versions.
+
+## 🛠️ Development & Extension
+
+### Plugin Structure
+
+```
+src/
+├── components/                        # React UI components
+│   ├── dashboard/                     # Dashboard sub-components
+│   │   ├── SiteConfigPanel.tsx        # WordPress connection form
+│   │   ├── StatsOverview.tsx          # Summary stat cards
+│   │   ├── JobsTable.tsx              # Migration jobs table
+│   │   ├── JobActionButtons.tsx       # Per-job action buttons
+│   │   ├── LogViewer.tsx              # Combined log display
+│   │   ├── useMigrationDashboard.ts   # State, effects, and handlers
+│   │   └── types.ts                   # Dashboard-specific types
+│   ├── MigrationDashboardClient.tsx   # Dashboard composition shell
+│   ├── ContentTypeSelect.tsx          # Content type picker
+│   ├── FieldMappingConfiguration.tsx  # Field mapper
+│   └── SimpleFieldMapping.tsx         # Simple mapping UI
+├── utils/
+│   ├── wordpress/client.ts            # WordPress REST API client
+│   ├── migration/orchestrator.ts      # Job execution engine
+│   ├── content/transformer.ts         # WP → Payload content transform
+│   ├── lexical/htmlToLexicalConverter.ts  # HTML → Lexical conversion
+│   └── ...                            # Media, fields, helpers, endpoints
+└── index.ts                           # Plugin entry point
+```
+
+### Error Handling
+
+The plugin provides comprehensive error handling:
+
+- **Connection Errors**: WordPress API connectivity issues
+- **Authentication Errors**: Invalid credentials or permissions
+- **Content Errors**: Malformed content or missing fields
+- **Media Errors**: Download failures or file size limits
+- **Database Errors**: PayloadCMS creation failures
+
+## 📊 Migration Best Practices
+
+### Pre-Migration Checklist
+
+1. **✅ WordPress Setup**
+   - Generate application password
+   - Verify REST API access
+   - Check user permissions
+   - Test SSL certificate
+
+2. **✅ PayloadCMS Configuration**
+   - Configure target collections
+   - Set up field mappings
+   - Test media collection setup
+   - Verify storage configuration
+
+3. **✅ Content Analysis**
+   - Run content discovery scan
+   - Review field mapping options
+   - Identify custom post types
+   - Plan migration sequence
+
+### Migration Strategy
+
+**Recommended Migration Order:**
+
+1. **Categories & Tags** (establishes taxonomy)
+2. **Media Files** (creates media library)
+3. **Pages** (static content first)
+4. **Posts** (blog content with relationships)
+5. **Users** (author information)
+6. **Custom Post Types** (specialized content)
+
+**Performance Optimization:**
+
+- Start with smaller batch sizes (10-25) for initial testing
+- Increase batch size (50-100) for production migrations
+- Monitor memory usage during large media migrations
+- Use resume functionality for interrupted migrations
+
+### Troubleshooting
+
+| Problem | Cause | Solution |
+|---------|-------|----------|
+| **"Authentication failed"** | Wrong credentials, or REST API disabled | Verify you're using an **Application Password** (not your login password). Confirm the REST API is accessible at `https://your-site.com/wp-json/` |
+| **Content types not discovered** | WordPress permalinks set to "Plain" | In WordPress Admin > Settings > Permalinks, switch to any structure other than "Plain" |
+| **Custom post types not found** | CPT not exposed to REST API | Add `'show_in_rest' => true` to your `register_post_type()` call |
+| **SSL certificate errors** | Self-signed or expired certificate | Set `allowSelfSignedCerts: true` for development. In production, use a valid certificate |
+| **Media download fails** | WordPress behind CDN or firewall | Ensure media file URLs (`wp-content/uploads/...`) are accessible from the server running Payload |
+| **`"The following path cannot be queried: migratedFromWordPress"`** | Target collection not in plugin config | Add the collection to the `collections` option so the metadata fields get injected |
+| **Large migration runs out of memory** | Batch size too high for available RAM | Reduce `migrationBatchSize` (try 5) and `migrationConcurrency` (set to 1) |
+| **Lexical content looks wrong** | HTML conversion edge case | Set `disableHtmlConversion: true` on the collection mapping as a workaround, and report the specific HTML that fails |
+| **Duplicate items after resume** | `migratedFromWordPress.wpPostId` field missing | Ensure the collection is listed in the plugin `collections` config — the field is auto-added |
+| **ACF fields not migrated** | Field mapping not configured | Add ACF field names to `customFields` in the collection mapping, or configure explicit `fieldMapping` in the job |
+| **`"MIME type not allowed"` or `"File too large"`** | Media file exceeds limits | Expand `allowedMediaTypes` and increase `maxMediaFileSize` in plugin config |
+
+## 🔧 Technical Requirements
+
+- **Node.js**: ^18.20.2 || >=20.9.0
+- **PayloadCMS**: ^3.29.0
+- **Package Manager**: pnpm ^9 || ^10
+- **WordPress**: 5.6+ with REST API enabled
+- **Memory**: 512MB+ recommended for media migrations
+- **Storage**: Adequate space for media file imports
+
+## 📈 Performance Considerations
+
+- **Batch Processing**: Configurable batch sizes prevent memory overflow
+- **Background Jobs**: Non-blocking migration execution
+- **Progress Persistence**: Resume interrupted migrations
+- **Memory Management**: Efficient file streaming for large media
+- **Cache Management**: Intelligent caching with invalidation
+- **Parallel Processing**: Concurrent media imports for optimal speed
+
+## 🤝 Contributing
+
+We welcome contributions! Areas for enhancement:
+
+- Additional content type support
+- Enhanced field mapping UI
+- Performance optimizations
+- Advanced content transformations
+- Integration tests
+- Documentation improvements
+
+## 📝 License
+
+MIT License - see [LICENSE](LICENSE) file for details.
+
+## 🎯 Roadmap
+
+- [ ] WordPress multisite support
+- [ ] Advanced Custom Fields Pro support
+- [ ] WooCommerce product migration
+- [ ] Automated testing suite
+- [ ] Docker development environment
+- [ ] Migration templates and presets
+- [ ] Webhook integration for real-time sync
+
+---
+
+**Author**: [Igor Abdulovic](mailto:igor.abdulovic@brightscout.com) at [Brightscout](https://brightscout.com)
+
+**Repository**: [GitHub](https://github.com/Brightscout/payload-wordpress-migrator)
+
+**Issues**: [GitHub Issues](https://github.com/Brightscout/payload-wordpress-migrator/issues)

package/dist/components/BeforeDashboardClient.d.ts

@@ -0,0 +1,14 @@
+type HubSpotForm = {
+    guid: string;
+    name: string;
+    stats: {
+        conversionRate: number;
+        submissions: number;
+        views: number;
+    };
+};
+type BeforeDashboardClientProps = {
+    forms: HubSpotForm[];
+};
+export declare const BeforeDashboardClient: ({ forms: initialForms }: BeforeDashboardClientProps) => import("react/jsx-runtime").JSX.Element;
+export {};