appwrite-utils-cli 0.10.86 → 1.0.2

package/.appwrite/import/README.md

@@ -0,0 +1,148 @@
+# Import Configurations
+
+This directory contains YAML-based import configurations that demonstrate the powerful import capabilities of Appwrite Utils CLI 1.0.0.
+
+## Available Import Configurations
+
+### 1. Users Import (`users-import.yaml`)
+- **Purpose**: Import user data with avatar file downloads
+- **Features**:
+  - Downloads avatar images from picsum.photos URLs
+  - Creates user accounts in Appwrite Auth
+  - Validates email addresses and phone numbers
+  - Handles user deduplication based on email
+  - Converts social links and preferences JSON
+- **Data Source**: `importData/users.json` (12 sample users)
+
+### 2. Categories Import (`categories-import.yaml`)
+- **Purpose**: Import hierarchical category structure
+- **Features**:
+  - Downloads cover images from URLs
+  - Handles parent-child relationships
+  - Supports category hierarchy up to 5 levels
+  - SEO metadata handling
+- **Data Source**: `importData/categories.json` (10 categories with hierarchy)
+
+### 3. Posts Import (`posts-import.yaml`)
+- **Purpose**: Import blog posts with relationships and media
+- **Features**:
+  - Downloads featured images and gallery images
+  - Links posts to authors (Users collection)
+  - Links posts to categories (Categories collection)
+  - Handles arrays (tags, gallery)
+  - Rich content with HTML support
+  - SEO metadata and social sharing images
+- **Data Source**: `importData/posts.json` (10 blog posts)
+
+## File Handling Features
+
+All import configurations demonstrate advanced file handling:
+
+- **URL Downloads**: Automatically downloads images from picsum.photos
+- **File Naming**: Uses template patterns like `{user_id}_avatar`
+- **Multiple Files**: Supports gallery arrays with multiple images
+- **File Validation**: Handles missing or invalid URLs gracefully
+- **Storage Integration**: Uploads to appropriate Appwrite storage buckets
+
+## Usage
+
+### Run Individual Imports
+```bash
+# Import users with avatars
+npx appwrite-utils-cli appwrite-migrate --import --collectionIds Users
+
+# Import categories with hierarchy
+npx appwrite-utils-cli appwrite-migrate --import --collectionIds Categories
+
+# Import posts with relationships
+npx appwrite-utils-cli appwrite-migrate --import --collectionIds Posts
+```
+
+### Run All Imports
+```bash
+# Import all collections in order
+npx appwrite-utils-cli appwrite-migrate --import --dbIds main
+```
+
+## Import Order
+
+For best results, import in this order due to relationships:
+
+1. **Users** (no dependencies)
+2. **Categories** (self-referential relationships)
+3. **Posts** (depends on Users and Categories)
+
+## Configuration Highlights
+
+### Rate Limiting
+- **Users**: 25 items per batch (avatar downloads)
+- **Categories**: 50 items per batch (cover images)
+- **Posts**: 20 items per batch (multiple images per post)
+
+### Validation
+- Email validation for users
+- URL validation for file downloads
+- Required field validation
+- Length constraints for text fields
+
+### Relationships
+- **Posts → Users**: Author relationship via userId
+- **Posts → Categories**: Category relationship via categoryId
+- **Categories → Categories**: Parent-child hierarchy
+
+### File Downloads
+- **User avatars**: 500x500 images from picsum.photos
+- **Category covers**: 800x400 images from picsum.photos
+- **Post images**: Various sizes (1200x600, 800x400, etc.)
+- **Social images**: 1200x630 for social media sharing
+
+## Testing
+
+### Dry Run
+Test configurations without actual import:
+```bash
+npx appwrite-utils-cli appwrite-migrate --import --dryRun
+```
+
+### Validation Only
+Validate data without importing:
+```bash
+npx appwrite-utils-cli appwrite-migrate --writeData
+```
+
+## Customization
+
+These configurations serve as examples. You can:
+
+1. **Modify data sources**: Update file paths in YAML configs
+2. **Add/remove fields**: Modify attribute mappings
+3. **Change validation rules**: Add or remove validation constraints
+4. **Adjust batch sizes**: Optimize for your server capacity
+5. **Add custom converters**: Extend data transformation logic
+
+## File Structure
+
+```
+.appwrite/
+├── collections/           # YAML collection definitions
+│   ├── Users.yaml
+│   ├── Posts.yaml
+│   └── Categories.yaml
+├── importData/           # Source data files
+│   ├── users.json
+│   ├── posts.json
+│   └── categories.json
+├── import/               # Import configurations
+│   ├── users-import.yaml
+│   ├── posts-import.yaml
+│   └── categories-import.yaml
+└── config.yaml          # Main project configuration
+```
+
+## Support
+
+For questions about the import system:
+- Check the main README.md for CLI usage
+- Review YAML schema files in `.yaml_schemas/`
+- See example configurations in this directory
+- Test with `--dryRun` flag before actual import

package/.appwrite/import/categories-import.yaml

@@ -0,0 +1,129 @@
+# yaml-language-server: $schema=../.yaml_schemas/import-config.schema.json
+# Categories Import Configuration with Hierarchy Support
+
+source:
+  file: "importData/categories.json"
+  basePath: "RECORDS"
+  type: "json"
+
+target:
+  collection: "Categories"
+  type: "create"
+  primaryKey: "category_id"
+  createUsers: false
+
+mapping:
+  attributes:
+    # Core Fields
+    - oldKey: "category_id"
+      targetKey: "categoryId"
+      converters: ["anyToString"]
+      validation:
+        - rule: "required"
+          params: ["{categoryId}"]
+
+    - oldKey: "name"
+      targetKey: "name"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "required"
+          params: ["{name}"]
+        - rule: "minLength"
+          params: ["{name}", "2"]
+
+    - oldKey: "slug"
+      targetKey: "slug"
+      converters: ["anyToString", "stringToLowerCase", "stringTrim"]
+
+    - oldKey: "description"
+      targetKey: "description"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "maxLength"
+          params: ["{description}", "500"]
+
+    # Hierarchy
+    - oldKey: "parent_id"
+      targetKey: "parentId"
+      converters: ["anyToString"]
+
+    - oldKey: "level"
+      targetKey: "level"
+      converters: ["anyToNumber"]
+      validation:
+        - rule: "numeric"
+          params: ["{level}"]
+        - rule: "positive"
+          params: ["{level}"]
+
+    - oldKey: "sort_order"
+      targetKey: "sortOrder"
+      converters: ["anyToNumber"]
+
+    # Visual & Branding
+    - oldKey: "color_code"
+      targetKey: "colorCode"
+      converters: ["anyToString"]
+
+    - oldKey: "icon"
+      targetKey: "icon"
+      converters: ["anyToString"]
+
+    # Cover Image with URL Download
+    - oldKey: "cover_image_url"
+      targetKey: "coverImage"
+      fileData:
+        path: "{cover_image_url}"
+        name: "{category_id}_cover"
+      afterImport:
+        - action: "createFileAndUpdateField"
+          params: ["{dbId}", "{collId}", "{docId}", "coverImage", "{bucketId}", "{filePath}", "{fileName}"]
+
+    # Status
+    - oldKey: "is_active"
+      targetKey: "isActive"
+      converters: ["anyToBoolean"]
+
+    - oldKey: "is_featured"
+      targetKey: "isFeatured"
+      converters: ["anyToBoolean"]
+
+    # Statistics
+    - oldKey: "post_count"
+      targetKey: "postCount"
+      converters: ["anyToNumber"]
+
+    - oldKey: "total_post_count"
+      targetKey: "totalPostCount"
+      converters: ["anyToNumber"]
+
+    # SEO
+    - oldKey: "meta_title"
+      targetKey: "metaTitle"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "maxLength"
+          params: ["{metaTitle}", "60"]
+
+    - oldKey: "meta_description"
+      targetKey: "metaDescription"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "maxLength"
+          params: ["{metaDescription}", "160"]
+
+    - oldKey: "guidelines"
+      targetKey: "guidelines"
+      converters: ["anyToString", "stringTrim"]
+
+  relationships:
+    # Parent-Child Relationship Resolution
+    - sourceField: "parent_id"
+      targetField: "parentId"
+      targetCollection: "Categories"
+
+options:
+  batchSize: 50
+  skipValidation: false
+  dryRun: false
+  continueOnError: true

package/.appwrite/import/posts-import.yaml

@@ -0,0 +1,208 @@
+# yaml-language-server: $schema=../.yaml_schemas/import-config.schema.json
+# Posts Import Configuration with Relationships and File Handling
+
+source:
+  file: "importData/posts.json"
+  basePath: "RECORDS"
+  type: "json"
+
+target:
+  collection: "Posts"
+  type: "create"
+  primaryKey: "post_id"
+  createUsers: false
+
+mapping:
+  attributes:
+    # Core Content
+    - oldKey: "post_id"
+      targetKey: "postId"
+      converters: ["anyToString"]
+      validation:
+        - rule: "required"
+          params: ["{postId}"]
+
+    - oldKey: "title"
+      targetKey: "title"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "required"
+          params: ["{title}"]
+        - rule: "minLength"
+          params: ["{title}", "5"]
+        - rule: "maxLength"
+          params: ["{title}", "200"]
+
+    - oldKey: "slug"
+      targetKey: "slug"
+      converters: ["anyToString", "stringToLowerCase", "stringTrim"]
+
+    - oldKey: "content"
+      targetKey: "content"
+      converters: ["anyToString"]
+      validation:
+        - rule: "required"
+          params: ["{content}"]
+        - rule: "minLength"
+          params: ["{content}", "100"]
+
+    - oldKey: "excerpt"
+      targetKey: "excerpt"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "maxLength"
+          params: ["{excerpt}", "500"]
+
+    # Author Relationship (handled in relationships section)
+    - oldKey: "author_id"
+      targetKey: "authorId"
+      converters: ["anyToString"]
+      validation:
+        - rule: "required"
+          params: ["{authorId}"]
+
+    # Category Relationship (handled in relationships section)  
+    - oldKey: "category_id"
+      targetKey: "categoryId"
+      converters: ["anyToString"]
+
+    # Featured Image with URL Download
+    - oldKey: "featured_image_url"
+      targetKey: "featuredImage"
+      fileData:
+        path: "{featured_image_url}"
+        name: "{post_id}_featured"
+      afterImport:
+        - action: "createFileAndUpdateField"
+          params: ["{dbId}", "{collId}", "{docId}", "featuredImage", "{bucketId}", "{filePath}", "{fileName}"]
+
+    # Gallery Images (multiple files)
+    - oldKey: "gallery_urls"
+      targetKey: "gallery"
+      fileData:
+        path: "{gallery_urls}"
+        name: "{post_id}_gallery_{index}"
+      afterImport:
+        - action: "createFileAndUpdateField"
+          params: ["{dbId}", "{collId}", "{docId}", "gallery", "{bucketId}", "{filePath}", "{fileName}"]
+
+    # Tags Array
+    - oldKey: "tags"
+      targetKey: "tags"
+      converters: ["anyToStringArray"]
+
+    # Status and Publishing
+    - oldKey: "status"
+      targetKey: "status"
+      converters: ["anyToString", "stringToLowerCase"]
+      validation:
+        - rule: "required"
+          params: ["{status}"]
+
+    - oldKey: "published_at"
+      targetKey: "publishedAt"
+      converters: ["anyToDate"]
+
+    - oldKey: "scheduled_at"
+      targetKey: "scheduledAt"
+      converters: ["anyToDate"]
+
+    # Engagement Metrics
+    - oldKey: "view_count"
+      targetKey: "viewCount"
+      converters: ["anyToNumber"]
+      validation:
+        - rule: "numeric"
+          params: ["{viewCount}"]
+        - rule: "positive"
+          params: ["{viewCount}"]
+
+    - oldKey: "like_count"
+      targetKey: "likeCount"
+      converters: ["anyToNumber"]
+      validation:
+        - rule: "numeric"
+          params: ["{likeCount}"]
+        - rule: "positive"
+          params: ["{likeCount}"]
+
+    - oldKey: "comment_count"
+      targetKey: "commentCount"
+      converters: ["anyToNumber"]
+      validation:
+        - rule: "numeric"
+          params: ["{commentCount}"]
+        - rule: "positive"
+          params: ["{commentCount}"]
+
+    - oldKey: "share_count"
+      targetKey: "shareCount"
+      converters: ["anyToNumber"]
+      validation:
+        - rule: "numeric"
+          params: ["{shareCount}"]
+        - rule: "positive"
+          params: ["{shareCount}"]
+
+    # SEO Fields
+    - oldKey: "meta_title"
+      targetKey: "metaTitle"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "maxLength"
+          params: ["{metaTitle}", "60"]
+
+    - oldKey: "meta_description"
+      targetKey: "metaDescription"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "maxLength"
+          params: ["{metaDescription}", "160"]
+
+    # Social Image with URL Download
+    - oldKey: "social_image_url"
+      targetKey: "socialImage"
+      fileData:
+        path: "{social_image_url}"
+        name: "{post_id}_social"
+      afterImport:
+        - action: "createFileAndUpdateField"
+          params: ["{dbId}", "{collId}", "{docId}", "socialImage", "{bucketId}", "{filePath}", "{fileName}"]
+
+    # Reading Time
+    - oldKey: "reading_time"
+      targetKey: "readingTime"
+      converters: ["anyToNumber"]
+      validation:
+        - rule: "numeric"
+          params: ["{readingTime}"]
+        - rule: "positive"
+          params: ["{readingTime}"]
+
+    # Featured Status
+    - oldKey: "is_featured"
+      targetKey: "isFeatured"
+      converters: ["anyToBoolean"]
+
+    - oldKey: "is_pinned"
+      targetKey: "isPinned"
+      converters: ["anyToBoolean"]
+
+  relationships:
+    # Author Relationship - Link to Users collection
+    - sourceField: "author_id"
+      targetField: "userId"
+      targetCollection: "Users"
+      fieldToSet: "authorId"
+
+    # Category Relationship - Link to Categories collection
+    - sourceField: "category_id"
+      targetField: "categoryId"
+      targetCollection: "Categories"
+      fieldToSet: "categoryId"
+
+options:
+  batchSize: 20
+  skipValidation: false
+  dryRun: false
+  continueOnError: true

package/.appwrite/import/users-import.yaml

@@ -0,0 +1,130 @@
+# yaml-language-server: $schema=../.yaml_schemas/import-config.schema.json
+# User Import Configuration with Avatar File Handling
+
+source:
+  file: "importData/users.json"
+  basePath: "RECORDS"
+  type: "json"
+
+target:
+  collection: "Users"
+  type: "create"
+  primaryKey: "user_id"
+  createUsers: true
+
+mapping:
+  attributes:
+    # Core Identity Fields
+    - oldKey: "user_id"
+      targetKey: "userId"
+      converters: ["anyToString"]
+      validation:
+        - rule: "required"
+          params: ["{userId}"]
+
+    - oldKey: "name"
+      targetKey: "name"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "required"
+          params: ["{name}"]
+        - rule: "minLength"
+          params: ["{name}", "2"]
+
+    - oldKey: "email"
+      targetKey: "email"
+      converters: ["anyToString", "stringToLowerCase", "stringTrim"]
+      validation:
+        - rule: "required"
+          params: ["{email}"]
+        - rule: "email"
+          params: ["{email}"]
+
+    - oldKey: "username"
+      targetKey: "username"
+      converters: ["anyToString", "stringToLowerCase", "stringTrim"]
+      validation:
+        - rule: "minLength"
+          params: ["{username}", "3"]
+
+    # Avatar File Handling with URL Download
+    - oldKey: "avatar_url"
+      targetKey: "avatar"
+      fileData:
+        path: "{avatar_url}"
+        name: "{user_id}_avatar"
+      afterImport:
+        - action: "createFileAndUpdateField"
+          params: ["{dbId}", "{collId}", "{docId}", "avatar", "{bucketId}", "{filePath}", "{fileName}"]
+
+    # Contact Information
+    - oldKey: "phone"
+      targetKey: "phone"
+      converters: ["anyToString"]
+      validation:
+        - rule: "phone"
+          params: ["{phone}"]
+
+    - oldKey: "bio"
+      targetKey: "bio"
+      converters: ["anyToString", "stringTrim"]
+      validation:
+        - rule: "maxLength"
+          params: ["{bio}", "500"]
+
+    # Dates
+    - oldKey: "date_of_birth"
+      targetKey: "dateOfBirth"
+      converters: ["anyToDate"]
+
+    - oldKey: "last_login_at"
+      targetKey: "lastLoginAt"
+      converters: ["anyToDate"]
+
+    # Location
+    - oldKey: "location"
+      targetKey: "location"
+      converters: ["anyToString", "stringTrim"]
+
+    # User Management
+    - oldKey: "role"
+      targetKey: "role"
+      converters: ["anyToString", "stringToLowerCase"]
+      validation:
+        - rule: "required"
+          params: ["{role}"]
+
+    - oldKey: "is_active"
+      targetKey: "isActive"
+      converters: ["anyToBoolean"]
+
+    - oldKey: "is_verified"
+      targetKey: "isVerified"
+      converters: ["anyToBoolean"]
+
+    # Statistics
+    - oldKey: "profile_views"
+      targetKey: "profileViews"
+      converters: ["anyToNumber"]
+      validation:
+        - rule: "numeric"
+          params: ["{profileViews}"]
+        - rule: "positive"
+          params: ["{profileViews}"]
+
+    # JSON Fields
+    - oldKey: "social_links"
+      targetKey: "socialLinks"
+      converters: ["anyToString"]
+
+    - oldKey: "preferences"
+      targetKey: "preferences"
+      converters: ["anyToString"]
+
+  relationships: []
+
+options:
+  batchSize: 25
+  skipValidation: false
+  dryRun: false
+  continueOnError: true