cloudflare-images-migrator 1.0.0__py3-none-any.whl

cloudflare_images_migrator-1.0.0.dist-info/METADATA

@@ -0,0 +1,474 @@
+Metadata-Version: 2.4
+Name: cloudflare-images-migrator
+Version: 1.0.0
+Summary: Enterprise-grade tool to migrate images to Cloudflare Images with persistent tracking and duplicate detection
+Home-page: https://github.com/mariolqn/Cloudflare-Images-Migrator
+Author: Mario Quinones
+Author-email: Mario Lemos Quirino Neto <mariolqn@users.noreply.github.com>
+License: MIT
+Project-URL: Homepage, https://github.com/mariolqn/Cloudflare-Images-Migrator
+Project-URL: Bug Tracker, https://github.com/mariolqn/Cloudflare-Images-Migrator/issues
+Project-URL: Documentation, https://github.com/mariolqn/Cloudflare-Images-Migrator#readme
+Project-URL: Source Code, https://github.com/mariolqn/Cloudflare-Images-Migrator
+Keywords: cloudflare,images,migration,cdn,optimization,duplicate-detection,enterprise,security,tracking,automation
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Multimedia :: Graphics
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Archiving :: Backup
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Operating System :: OS Independent
+Classifier: Environment :: Console
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.28.0
+Requires-Dist: pillow>=9.0.0
+Requires-Dist: tqdm>=4.64.0
+Requires-Dist: colorama>=0.4.4
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: python-magic>=0.4.24
+Requires-Dist: beautifulsoup4>=4.11.0
+Requires-Dist: python-dotenv>=0.19.0
+Provides-Extra: dev
+Requires-Dist: pytest>=6.0; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: flake8; extra == "dev"
+Requires-Dist: mypy; extra == "dev"
+Dynamic: author
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: requires-python
+
+# Cloudflare Images Migration Tool
+
+A powerful Python tool to automatically migrate all images in a codebase to Cloudflare Images, replacing local and external image references with optimized Cloudflare URLs.
+
+## Features
+
+### 🔒 **Enterprise Security & Quality**
+- **Advanced Threat Detection**: Multi-layer security validation with malware scanning
+- **File Signature Verification**: Magic byte validation and MIME type checking  
+- **Content Security Scanning**: Deep analysis for malicious patterns and XSS threats
+- **URL Security Validation**: HTTPS enforcement and domain reputation checking
+- **EXIF Data Sanitization**: Removes potentially sensitive metadata
+- **Rate Limiting Protection**: Prevents abuse and ensures compliance
+- **Premium Image Optimization**: AI-powered quality enhancement and compression
+- **Responsive Variants**: Automatic generation of multiple image sizes
+- **Enterprise Audit Logging**: SOX, GDPR, HIPAA, PCI DSS compliance support
+
+### 📊 **Enterprise Tracking & Deduplication**
+- **Persistent Duplicate Detection**: Never re-uploads the same image across sessions
+- **Comprehensive Tracking Database**: SQLite database with full metadata storage
+- **CSV Export Support**: Export migration data for spreadsheet analysis
+- **Session Tracking**: Unique migration IDs for audit trails
+- **Cross-Session Statistics**: Performance metrics across multiple runs
+- **Multi-Level Duplicate Detection**: File hash, URL hash, and path matching
+- **Cloudflare Library Integration**: Checks existing images before uploading
+
+### 🚀 **Core Migration Features**  
+- **Enhanced Image Detection**: Finds badges, GitHub assets, CDN images, and traditional formats
+- **Multi-Format Parsing**: Supports HTML, CSS, JavaScript, TypeScript, Markdown, JSON, and more
+- **Batch Processing**: Efficiently uploads multiple images with progress tracking
+- **Smart Deduplication**: Avoids duplicate uploads using content hashing
+- **Zip Support**: Works with both directories and zip files
+- **Safe Operation**: Creates backups and supports dry-run mode
+- **Colored Logging**: Beautiful console output with progress bars
+- **Flexible Configuration**: Environment variables, config files, or command-line options
+
+## Installation
+
+1. **Clone or download this repository**
+2. **Install dependencies**:
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+## Prerequisites
+
+Before using the tool, you need:
+
+1. **Cloudflare Account ID**: Found in your Cloudflare dashboard
+2. **Cloudflare API Token**: Create one with Images permissions
+
+### Getting Your Credentials
+
+1. **Account ID**: 
+   - Go to [Cloudflare Dashboard](https://dash.cloudflare.com/)
+   - Select your account
+   - Find "Account ID" in the right sidebar
+
+2. **API Token**:
+   - Go to [Cloudflare API Tokens](https://dash.cloudflare.com/profile/api-tokens)
+   - Click "Create Token"
+   - Use "Custom token" template
+   - Add permissions: `Cloudflare Images:Edit`
+   - Choose your account in "Account Resources"
+
+## Usage
+
+### Basic Usage
+
+```bash
+python main.py /path/to/your/codebase --account-id YOUR_ACCOUNT_ID --api-token YOUR_API_TOKEN
+```
+
+### Tracking & Statistics
+
+```bash
+# Show comprehensive statistics
+python main.py /path/to/your/codebase --show-stats
+
+# Export migration data to CSV
+python main.py /path/to/your/codebase --export-csv ./migration-data.csv
+
+# View statistics without running migration
+python main.py --show-stats
+```
+
+### Using Environment Variables
+
+```bash
+export CLOUDFLARE_ACCOUNT_ID="your_account_id"
+export CLOUDFLARE_API_TOKEN="your_api_token"
+python main.py /path/to/your/codebase
+```
+
+### Using Configuration File
+
+Create a `config.yaml` file:
+
+```yaml
+account_id: "your_account_id"
+api_token: "your_api_token"
+dry_run: false
+backup: true
+batch_size: 10
+```
+
+Then run:
+
+```bash
+python main.py /path/to/your/codebase --config-file config.yaml
+```
+
+## Command Line Options
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `--account-id` | Cloudflare Account ID | Required |
+| `--api-token` | Cloudflare API Token | Required |
+| `--output-dir` | Output directory for results | Auto-generated |
+| `--dry-run` | Preview changes without making them | False |
+| `--backup/--no-backup` | Create backup before modifying | True |
+| `--config-file` | Path to configuration file | None |
+| `--verbose, -v` | Enable verbose logging | False |
+| `--file-types` | Comma-separated file extensions | Auto-detected |
+| `--exclude-dirs` | Comma-separated directories to exclude | Common build dirs |
+| `--show-stats` | Display comprehensive statistics | False |
+| `--export-csv` | Export migration data to CSV file | None |
+
+## Examples
+
+### Tracking & Statistics Examples
+```bash
+# View comprehensive statistics
+python main.py --show-stats
+
+# Export migration history to CSV
+python main.py --export-csv ./my-migration-data.csv
+
+# Run migration with statistics display
+python main.py ./my-website --show-stats --verbose
+```
+
+### Dry Run (Preview Changes)
+```bash
+python main.py ./my-website --dry-run --verbose
+```
+
+### Process Specific File Types
+```bash
+python main.py ./project --file-types=".html,.css,.js"
+```
+
+### Exclude Directories
+```bash
+python main.py ./app --exclude-dirs="node_modules,dist,build"
+```
+
+### Process Zip File
+```bash
+python main.py ./website-backup.zip --output-dir ./migrated-website
+```
+
+### Enterprise Security Examples
+```bash
+# Maximum security with audit report
+python main.py ./project --security-level enterprise --generate-security-report
+
+# Premium quality optimization
+python main.py ./images --optimization-level aggressive --security-level enterprise
+
+# Compliance-focused migration with tracking
+python main.py ./app --security-level enterprise --backup --verbose --show-stats
+```
+
+## Supported File Types
+
+The tool automatically detects and processes these file types:
+
+- **Web Files**: `.html`, `.htm`, `.css`, `.js`, `.jsx`, `.ts`, `.tsx`
+- **Documentation**: `.md`, `.markdown`
+- **Configuration**: `.json`, `.xml`, `.yaml`, `.yml`
+- **Stylesheets**: `.scss`, `.sass`, `.less`
+
+## Enhanced Image Detection
+
+The tool now detects a wide variety of image types:
+
+### **Traditional Images**
+- PNG, JPEG, GIF, WebP, SVG, BMP, ICO with file extensions
+
+### **Badge Services**
+- Shield.io badges (`img.shields.io`)
+- GitHub status badges
+- NPM badges
+- Travis CI badges
+- And many more badge services
+
+### **GitHub Assets**
+- GitHub raw content (`raw.githubusercontent.com`)
+- GitHub user content (`user-images.githubusercontent.com`)
+- GitHub assets (`github.com/*/assets/`)
+
+### **CDN Images**
+- Images served from CDNs without traditional extensions
+- Image-like paths (containing 'icon', 'logo', 'banner', etc.)
+- URLs with image-like query parameters
+
+## Supported Image Formats
+
+- PNG, JPEG, GIF, WebP, SVG, BMP, ICO
+- Maximum file size: 10MB
+- Maximum dimensions: 12,000 pixels
+- Maximum area: 100 megapixels
+
+## Configuration File Format
+
+Create a `config.yaml` file with these options:
+
+```yaml
+# Required
+account_id: "your_cloudflare_account_id"
+api_token: "your_cloudflare_api_token"
+
+# Optional
+dry_run: false
+backup: true
+batch_size: 10
+retry_count: 3
+timeout: 30
+
+# File processing
+file_types:
+  - ".html"
+  - ".css"
+  - ".js"
+  - ".jsx"
+  - ".ts"
+  - ".tsx"
+  - ".md"
+  - ".json"
+
+exclude_dirs:
+  - "node_modules"
+  - ".git"
+  - "dist"
+  - "build"
+```
+
+## How It Works
+
+1. **📁 Preparation**: Extracts zip files if needed, creates backups
+2. **🔍 Enhanced Scanning**: Recursively scans files for all types of image references
+3. **🎯 Duplicate Detection**: Checks against existing Cloudflare Images and local database
+4. **🚀 Smart Upload**: Uploads only new, unique images to Cloudflare Images
+5. **📊 Tracking**: Records all metadata in persistent SQLite database
+6. **✏️ Replacement**: Replaces old references with Cloudflare URLs
+7. **📋 Comprehensive Report**: Provides detailed migration statistics and history
+
+## Tracking & Statistics
+
+### **Migration Database**
+The tool maintains a persistent SQLite database (`cloudflare_images.db`) that tracks:
+- Every image processed across all sessions
+- File metadata (size, dimensions, format, quality)
+- Upload timestamps and session IDs
+- Cloudflare URLs and variants
+- File hashes for duplicate detection
+- Performance metrics and statistics
+
+### **Statistics Display**
+Use `--show-stats` to see:
+- **Session Statistics**: Current migration metrics
+- **Total Statistics**: All-time migration history
+- **Performance Metrics**: Average upload times, success rates
+- **Duplicate Prevention**: Images skipped due to existing uploads
+- **Size Optimization**: Compression ratios and savings
+
+### **CSV Export**
+Export your migration data with `--export-csv filename.csv`:
+- Complete migration history
+- File metadata and performance metrics
+- Timestamps and session tracking
+- Cloudflare URLs and variants
+- Perfect for reporting and analysis
+
+## Output
+
+### **Enhanced Output Example**
+
+```
+15:30:45 - INFO - Starting migration of: ./my-website
+15:30:45 - INFO - ✓ Successfully connected to Cloudflare Images API
+15:30:45 - INFO - ✓ Loaded 1,250 existing Cloudflare Images for duplicate detection
+15:30:45 - INFO - ✓ Connected to tracking database (3,847 images tracked)
+15:30:45 - INFO - Created backup: ./my-website_backup
+15:30:45 - INFO - Phase 1: Enhanced scanning for image references...
+Scanning files: 100%|██████████| 150/150 [00:02<00:00, 75.0it/s]
+15:30:47 - INFO - Found 156 image references (45 unique, 111 duplicates)
+15:30:47 - INFO - Phase 2: Smart uploading with duplicate detection...
+Uploading images: 100%|██████████| 23/23 [00:15<00:00, 1.5it/s]
+15:31:02 - INFO - Upload complete: 23/23 images uploaded (100.0% success rate)
+15:31:02 - INFO - Phase 3: Replacing image references in code...
+Modifying files: 100%|██████████| 12/12 [00:01<00:00, 10.0it/s]
+
+==================================================
+Migration Summary
+==================================================
+Files processed: 150
+Images found: 156 (45 unique, 111 duplicates)
+Images uploaded: 23 (0 skipped as duplicates)
+Files modified: 12
+URLs replaced: 156
+Average file size reduction: 45.2%
+Time elapsed: 0:00:17
+
+Session Statistics:
+- New images uploaded: 23
+- Duplicate images skipped: 0
+- Security threats blocked: 0
+- Quality optimizations applied: 23
+
+Total Statistics (All Sessions):
+- Total images tracked: 3,870
+- Total migrations completed: 47
+- Total file size saved: 2.3 GB
+- Average success rate: 99.2%
+==================================================
+
+✓ Tracking data saved to: cloudflare_images.db
+✓ Migration completed successfully!
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **"Failed to connect to Cloudflare Images API"**
+   - Check your account ID and API token
+   - Ensure API token has Images permissions
+   - Verify internet connection
+
+2. **"Database locked" errors**
+   - Ensure no other migration processes are running
+   - Check database file permissions
+   - Consider running with `--verbose` for more details
+
+3. **"File too large" errors**
+   - Cloudflare Images has a 10MB limit per image
+   - Consider optimizing large images before migration
+
+4. **"No image references found"**
+   - Check if file types are supported
+   - Use `--verbose` to see detailed scanning logs
+   - Verify exclude directories aren't too broad
+
+5. **Duplicate detection issues**
+   - Check if images were previously uploaded with different settings
+   - Use `--show-stats` to see duplicate detection statistics
+   - Consider clearing the tracking database if needed
+
+### Advanced Troubleshooting
+
+```bash
+# View detailed statistics
+python main.py --show-stats --verbose
+
+# Export data for analysis
+python main.py --export-csv debug-data.csv
+
+# Check database status
+python main.py --show-stats  # Shows database statistics
+```
+
+### Getting Help
+
+- Use `--verbose` flag for detailed logging
+- Check the generated log file for full operation details
+- Use `--show-stats` to understand tracking database status
+- Export CSV data for external analysis
+
+## Environment Variables
+
+You can set these environment variables instead of using command-line options:
+
+- `CLOUDFLARE_ACCOUNT_ID` or `CF_ACCOUNT_ID`
+- `CLOUDFLARE_API_TOKEN` or `CF_API_TOKEN`
+
+## Security Notes
+
+- Never commit API tokens to version control
+- Use environment variables or config files with proper permissions
+- The tool only reads from source files and uploads to Cloudflare
+- Backups are created locally for safety
+- Tracking database contains metadata only, no sensitive data
+
+## Database Management
+
+The tool creates a `cloudflare_images.db` SQLite database to track:
+- All processed images and their metadata
+- Migration sessions and timestamps
+- Duplicate detection hashes
+- Performance statistics
+
+### Database Location
+- Default: `./cloudflare_images.db` in the current directory
+- Contains no sensitive data (API tokens, personal info)
+- Safe to backup and share for analysis
+
+## Contributing
+
+This tool is designed to be extensible:
+
+- Add new file type parsers in `src/parsers.py`
+- Extend configuration options in `src/config.py`
+- Add new utility functions in `src/utils.py`
+- Enhance tracking features in `src/image_tracker.py`
+
+## License
+
+This project is provided as-is for educational and practical use.
+
+---
+
+**Made with ❤️ for the Cloudflare Images community** 

cloudflare_images_migrator-1.0.0.dist-info/RECORD

@@ -0,0 +1,17 @@
+cloudflare_images_migrator-1.0.0.dist-info/licenses/LICENSE,sha256=q_GOanS_whxguhI63ykZMgb0fFhm42mVN1IGXR9tUdo,1081
+src/__init__.py,sha256=bwslfUk2uqWfV7_zaL2ps3Uu5CgfPQwNkyolEyV_AtA,35
+src/audit.py,sha256=nfd1U2JL8eNlAylCiRdtw9zSoacYrGJG_AHysIHLK7k,24170
+src/cloudflare_client.py,sha256=RcoLzrlaE91XXJtlWml4WFsYk2gCXYrvN8ts-G8Xrfs,30923
+src/config.py,sha256=nApsCEK7ktXoZ8PYR1lWy1Mh9aKcJ5kWSR4cJHtLqQM,5851
+src/image_tracker.py,sha256=zEsnUjkJqUIsguoNV3dd9Du2XfFtIwUYWJaNeuD_Z74,18336
+src/logger.py,sha256=s23PzChAcdPCo4ea6hPW55zN31VItWwdkHOAe6d6yZM,5656
+src/migrator.py,sha256=FzmtBIltd6gLgeHNO0x7fmh4YrlrZfbGXpmWD0ZI_IM,21245
+src/parsers.py,sha256=bPs-d9VipElW5sB_Pv1a-trCINaes4kV19u9mPbBPe8,24834
+src/quality.py,sha256=_LsiAekbsZqE7FUySLGNOqx3_TczTVweZSv1hvS0IBU,22225
+src/security.py,sha256=fLKFmJnmvUK0ESECfzQMG9JNgyi-HuxAveroeAHJtok,20968
+src/utils.py,sha256=HIquvYns9dkG-JXuLC9AxhAmoMYPJTMQVuF36zoDQvM,8572
+cloudflare_images_migrator-1.0.0.dist-info/METADATA,sha256=wJNY1TiIKU9iKc40ub59BghgsRMt1_Q-7MRNQAWM8q8,15564
+cloudflare_images_migrator-1.0.0.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+cloudflare_images_migrator-1.0.0.dist-info/entry_points.txt,sha256=37pDOXye2Gcvq0RGjeK_umDvwFF5oj74fL3YAHBPFkc,87
+cloudflare_images_migrator-1.0.0.dist-info/top_level.txt,sha256=74rtVfumQlgAPzR5_2CgYN24MB0XARCg0t-gzk6gTrM,4
+cloudflare_images_migrator-1.0.0.dist-info/RECORD,,

cloudflare_images_migrator-1.0.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (80.9.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

cloudflare_images_migrator-1.0.0.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+cf-images-migrate = main:main
+cloudflare-images-migrator = main:main

cloudflare_images_migrator-1.0.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Mario Lemos Quirino Neto
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

cloudflare_images_migrator-1.0.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+src

src/__init__.py

@@ -0,0 +1 @@
+# Cloudflare Images Migration Tool