django-cfg 1.4.106__py3-none-any.whl → 1.4.108__py3-none-any.whl

django_cfg/modules/django_admin/RESOURCE_CONFIG_ENHANCEMENT.md

@@ -0,0 +1,350 @@
+# ResourceConfig & BackgroundTaskConfig Enhancement
+
+## Summary
+
+Enhanced django-admin module with declarative import/export configuration and background task support.
+
+## New Features
+
+### 1. ResourceConfig
+
+Declarative configuration for django-import-export Resource classes.
+
+**Location:** `django_cfg/modules/django_admin/config/resource_config.py`
+
+**Features:**
+- Field selection and exclusion
+- Import ID fields configuration
+- Import/export behavior control
+- Hook support (before/after import, per-row hooks)
+- Export field ordering
+- Batch processing options
+
+**Example:**
+```python
+from django_cfg.modules.django_admin import ResourceConfig
+
+resource_config = ResourceConfig(
+    fields=['host', 'port', 'username', 'password'],
+    exclude=['metadata', 'config'],
+    import_id_fields=['host', 'port'],
+    skip_unchanged=True,
+    after_import_row='apps.myapp.tasks.test_after_import',
+    batch_size=100,
+)
+```
+
+### 2. BackgroundTaskConfig
+
+Configuration for background task processing.
+
+**Location:** `django_cfg/modules/django_admin/config/background_task_config.py`
+
+**Features:**
+- Task runner selection (rearq, celery, django_q, sync)
+- Batch size configuration
+- Timeout settings
+- Retry policy
+- Priority levels
+
+**Example:**
+```python
+from django_cfg.modules.django_admin import BackgroundTaskConfig
+
+background_config = BackgroundTaskConfig(
+    enabled=True,
+    task_runner='rearq',
+    batch_size=50,
+    timeout=300,
+    retry_on_failure=True,
+    max_retries=3,
+)
+```
+
+### 3. Enhanced AdminConfig
+
+**Updated:** `django_cfg/modules/django_admin/config/admin_config.py`
+
+**New Fields:**
+```python
+class AdminConfig(BaseModel):
+    # ... existing fields ...
+
+    # Import/Export enhancement
+    resource_config: Optional[ResourceConfig] = None
+
+    # Background task processing
+    background_task_config: Optional[BackgroundTaskConfig] = None
+```
+
+### 4. Enhanced PydanticAdmin
+
+**Updated:** `django_cfg/modules/django_admin/base/pydantic_admin.py`
+
+**Key Changes:**
+- `_generate_resource_class()` now uses ResourceConfig if provided
+- Auto-generates Resource with hooks support
+- Dynamically attaches hook methods to Resource class
+
+**Hook Support:**
+- `before_import` - Called before import starts
+- `after_import` - Called after import completes
+- `before_import_row` - Called before each row import
+- `after_import_row` - Called after each row import (★ most useful)
+
+## Usage Example
+
+### Complete Proxy Admin with Import/Export
+
+```python
+from django_cfg.modules.django_admin import (
+    ActionConfig,
+    AdminConfig,
+    BackgroundTaskConfig,
+    ResourceConfig,
+)
+
+proxy_config = AdminConfig(
+    model=Proxy,
+
+    # Enable import/export with ResourceConfig
+    import_export_enabled=True,
+    resource_config=ResourceConfig(
+        fields=[
+            'host', 'port', 'proxy_type', 'proxy_mode',
+            'username', 'password',
+            'provider', 'country',
+        ],
+        exclude=['metadata', 'config', 'last_error'],
+        import_id_fields=['host', 'port', 'provider'],
+        skip_unchanged=True,
+        # Auto-test after import
+        after_import_row='apps.proxies.tasks.after_import_row_test_proxy',
+    ),
+
+    # Background task configuration
+    background_task_config=BackgroundTaskConfig(
+        enabled=True,
+        task_runner='rearq',
+        batch_size=50,
+        timeout=300,
+    ),
+
+    # Admin actions
+    actions=[
+        ActionConfig(
+            name='test_selected_proxies',
+            description='Test selected proxies',
+            variant='warning',
+            icon='speed',
+            handler='apps.proxies.admin.actions.test_selected_proxies',
+        ),
+    ],
+
+    list_display=['host', 'port', 'status', 'success_rate'],
+)
+
+@admin.register(Proxy)
+class ProxyAdmin(PydanticAdmin):
+    config = proxy_config
+```
+
+### Hook Implementation
+
+```python
+# apps/proxies/tasks.py
+
+def after_import_row_test_proxy(row, row_result, **kwargs):
+    """Hook called after each proxy import."""
+
+    # Skip if dry run
+    if kwargs.get('dry_run'):
+        return
+
+    # Only test new proxies
+    if row_result.import_type == 'new':
+        proxy = row_result.instance
+
+        # Queue async test
+        from api.workers import get_worker
+        worker = get_worker()
+        worker.enqueue_task(
+            'apps.proxies.tasks.test_proxy_async',
+            proxy_id=str(proxy.id)
+        )
+```
+
+## Benefits
+
+### Before (Manual Resource Class)
+```python
+from import_export import resources
+
+class ProxyResource(resources.ModelResource):
+    class Meta:
+        model = Proxy
+        fields = ('host', 'port', 'username', 'password')
+        import_id_fields = ['host', 'port']
+        skip_unchanged = True
+
+    def after_import_row(self, row, row_result, **kwargs):
+        # Custom logic here
+        pass
+
+proxy_config = AdminConfig(
+    model=Proxy,
+    import_export_enabled=True,
+    resource_class=ProxyResource,  # Manual class
+)
+```
+
+### After (Declarative ResourceConfig)
+```python
+proxy_config = AdminConfig(
+    model=Proxy,
+    import_export_enabled=True,
+    resource_config=ResourceConfig(
+        fields=['host', 'port', 'username', 'password'],
+        import_id_fields=['host', 'port'],
+        skip_unchanged=True,
+        after_import_row='apps.proxies.tasks.after_import_row_test_proxy',
+    ),
+)
+```
+
+**Advantages:**
+- ✅ No separate Resource class needed
+- ✅ All configuration in one place
+- ✅ Type-safe with Pydantic validation
+- ✅ Reusable across multiple admins
+- ✅ Hook as string path (lazy import)
+- ✅ Auto-generated Resource with full control
+
+## Dependencies Added
+
+**pyproject.toml updates:**
+```toml
+dependencies = [
+    # ... existing ...
+    "pytz>=2025.1",
+    "httpx>=0.28.1,<1.0",
+]
+```
+
+- `pytz` - Timezone support for datetime operations
+- `httpx` - Modern HTTP client for proxy testing
+
+## Files Changed
+
+### django-cfg Core
+1. `config/resource_config.py` - NEW
+2. `config/background_task_config.py` - NEW
+3. `config/admin_config.py` - MODIFIED (added new configs)
+4. `config/__init__.py` - MODIFIED (exports)
+5. `base/pydantic_admin.py` - MODIFIED (Resource generation)
+6. `__init__.py` - MODIFIED (exports)
+7. `pyproject.toml` - MODIFIED (dependencies)
+
+### stockapis Implementation
+1. `apps/proxies/admin/proxy_admin.py` - Uses ResourceConfig
+2. `apps/proxies/admin/actions.py` - NEW (admin actions)
+3. `apps/proxies/services/proxy_tester.py` - NEW (testing logic)
+4. `apps/proxies/tasks.py` - NEW (background tasks)
+5. `apps/proxies/@docs/IMPORT_EXPORT_SETUP.md` - NEW (docs)
+
+## Backward Compatibility
+
+✅ **100% Backward Compatible**
+
+Old code still works:
+```python
+# Old way - still works
+proxy_config = AdminConfig(
+    model=Proxy,
+    import_export_enabled=True,
+    resource_class=MyCustomResource,  # Manual class
+)
+
+# New way - alternative
+proxy_config = AdminConfig(
+    model=Proxy,
+    import_export_enabled=True,
+    resource_config=ResourceConfig(...),  # Declarative
+)
+```
+
+Priority:
+1. If `resource_class` provided → use it (legacy)
+2. If `resource_config` provided → auto-generate Resource
+3. If neither → auto-generate basic Resource
+
+## Testing
+
+```bash
+# Test django-cfg imports
+cd django-cfg-dev
+poetry run python -c "from django_cfg.modules.django_admin import ResourceConfig, BackgroundTaskConfig; print('✅ OK')"
+
+# Test stockapis admin
+cd stockapis
+poetry run python manage.py check --tag admin
+
+# Test in browser
+poetry run python manage.py runserver
+# Visit: http://localhost:8000/admin/proxies/proxy/
+```
+
+## Future Enhancements
+
+- [ ] `ResourceConfig.field_widgets` - Custom widgets for fields
+- [ ] `ResourceConfig.validators` - Custom validation rules
+- [ ] `ResourceConfig.transformers` - Data transformation before import
+- [ ] Progress tracking for large imports
+- [ ] Import history and rollback
+- [ ] Scheduled imports via cron
+- [ ] API endpoint for programmatic imports
+
+## Migration Guide
+
+### For Existing Projects
+
+1. Update django-cfg to latest version
+2. Optional: Replace manual Resource classes with ResourceConfig
+3. Optional: Add BackgroundTaskConfig for async operations
+4. Optional: Add after_import_row hooks for post-import processing
+
+### Example Migration
+
+**Before:**
+```python
+# resources.py
+class ProxyResource(resources.ModelResource):
+    class Meta:
+        model = Proxy
+        fields = ('host', 'port')
+
+# admin.py
+config = AdminConfig(
+    model=Proxy,
+    import_export_enabled=True,
+    resource_class=ProxyResource,
+)
+```
+
+**After:**
+```python
+# admin.py only - no resources.py needed
+config = AdminConfig(
+    model=Proxy,
+    import_export_enabled=True,
+    resource_config=ResourceConfig(
+        fields=['host', 'port'],
+    ),
+)
+```
+
+## Version
+
+- **django-cfg**: 1.4.106+
+- **Python**: 3.12+
+- **Django**: 5.2+

django_cfg/modules/django_admin/__init__.py

@@ -46,6 +46,7 @@ Example:
 from .config import (
     ActionConfig,
     AdminConfig,
+    BackgroundTaskConfig,
     BadgeField,
     BooleanField,
     CurrencyField,
@@ -53,6 +54,7 @@ from .config import (
     FieldConfig,
     FieldsetConfig,
     ImageField,
+    ResourceConfig,
     TextField,
     UserField,
 )
@@ -101,6 +103,8 @@ __all__ = [
     "FieldConfig",
     "FieldsetConfig",
     "ActionConfig",
+    "ResourceConfig",
+    "BackgroundTaskConfig",
     # Specialized Field Types
     "BadgeField",
     "BooleanField",

django_cfg/modules/django_admin/base/pydantic_admin.py

@@ -154,37 +154,92 @@ class PydanticAdminMixin:
 
     @classmethod
     def _generate_resource_class(cls, config: AdminConfig):
-        """Auto-generate a ModelResource class for import/export."""
+        """
+        Auto-generate a ModelResource class for import/export.
+
+        Uses ResourceConfig if provided, otherwise generates basic Resource.
+        """
         from import_export import resources
 
         target_model = config.model
+        resource_config = config.resource_config
 
-        # Get all model fields
-        model_fields = []
-        for field in target_model._meta.get_fields():
-            # Skip relations and auto fields that shouldn't be imported
-            if field.concrete and not field.many_to_many:
-                # Skip password fields for security
-                if 'password' not in field.name.lower():
-                    model_fields.append(field.name)
-
-        # Create dynamic resource class with explicit Meta attributes
+        # Determine fields to include
+        if resource_config and resource_config.fields:
+            # Use explicitly specified fields
+            model_fields = resource_config.fields
+        else:
+            # Auto-detect fields from model
+            model_fields = []
+            for field in target_model._meta.get_fields():
+                # Skip relations and auto fields that shouldn't be imported
+                if field.concrete and not field.many_to_many:
+                    # Skip password fields for security
+                    if 'password' not in field.name.lower():
+                        model_fields.append(field.name)
+
+        # Apply exclusions if specified
+        if resource_config and resource_config.exclude:
+            model_fields = [f for f in model_fields if f not in resource_config.exclude]
+
+        # Build Meta attributes
         meta_attrs = {
             'model': target_model,
             'fields': tuple(model_fields),
-            'import_id_fields': ['id'] if 'id' in model_fields else [],
-            'skip_unchanged': True,
-            'report_skipped': True,
         }
 
+        # Add ResourceConfig settings
+        if resource_config:
+            meta_attrs['import_id_fields'] = resource_config.import_id_fields
+            meta_attrs['skip_unchanged'] = resource_config.skip_unchanged
+            meta_attrs['report_skipped'] = resource_config.report_skipped
+            meta_attrs['use_transactions'] = resource_config.use_transactions
+
+            if resource_config.export_order:
+                meta_attrs['export_order'] = tuple(resource_config.export_order)
+        else:
+            # Default settings
+            meta_attrs['import_id_fields'] = ['id'] if 'id' in model_fields else []
+            meta_attrs['skip_unchanged'] = True
+            meta_attrs['report_skipped'] = True
+
         # Create Meta class
         ResourceMeta = type('Meta', (), meta_attrs)
 
+        # Build Resource class attributes (methods + Meta)
+        resource_attrs = {'Meta': ResourceMeta}
+
+        # Add hooks from ResourceConfig
+        if resource_config:
+            # before_import hook
+            if resource_config.before_import:
+                hook = resource_config.get_callable('before_import')
+                if hook:
+                    resource_attrs['before_import'] = hook
+
+            # after_import hook
+            if resource_config.after_import:
+                hook = resource_config.get_callable('after_import')
+                if hook:
+                    resource_attrs['after_import'] = hook
+
+            # before_import_row hook
+            if resource_config.before_import_row:
+                hook = resource_config.get_callable('before_import_row')
+                if hook:
+                    resource_attrs['before_import_row'] = hook
+
+            # after_import_row hook
+            if resource_config.after_import_row:
+                hook = resource_config.get_callable('after_import_row')
+                if hook:
+                    resource_attrs['after_import_row'] = hook
+
         # Create Resource class
         AutoGeneratedResource = type(
             f'{target_model.__name__}Resource',
             (resources.ModelResource,),
-            {'Meta': ResourceMeta}
+            resource_attrs
         )
 
         return AutoGeneratedResource

django_cfg/modules/django_admin/config/__init__.py

@@ -4,6 +4,7 @@ Configuration models for declarative Django Admin.
 
 from .action_config import ActionConfig
 from .admin_config import AdminConfig
+from .background_task_config import BackgroundTaskConfig
 from .field_config import (
     FieldConfig,
     BadgeField,
@@ -15,12 +16,15 @@ from .field_config import (
     UserField,
 )
 from .fieldset_config import FieldsetConfig
+from .resource_config import ResourceConfig
 
 __all__ = [
     "AdminConfig",
     "FieldConfig",
     "FieldsetConfig",
     "ActionConfig",
+    "ResourceConfig",
+    "BackgroundTaskConfig",
     # Specialized Field Types
     "BadgeField",
     "BooleanField",

django_cfg/modules/django_admin/config/admin_config.py

@@ -8,8 +8,10 @@ from django.db import models
 from pydantic import BaseModel, ConfigDict, Field
 
 from .action_config import ActionConfig
+from .background_task_config import BackgroundTaskConfig
 from .field_config import FieldConfig
 from .fieldset_config import FieldsetConfig
+from .resource_config import ResourceConfig
 
 
 class AdminConfig(BaseModel):
@@ -127,7 +129,17 @@ class AdminConfig(BaseModel):
 
     # Import/Export options
     import_export_enabled: bool = Field(False, description="Enable import/export functionality")
-    resource_class: Optional[Type] = Field(None, description="Resource class for import/export")
+    resource_class: Optional[Type] = Field(None, description="Custom Resource class for import/export")
+    resource_config: Optional[ResourceConfig] = Field(
+        None,
+        description="Declarative resource configuration (alternative to resource_class)"
+    )
+
+    # Background task processing
+    background_task_config: Optional[BackgroundTaskConfig] = Field(
+        None,
+        description="Configuration for background task processing"
+    )
 
     def get_display_field_config(self, field_name: str) -> Optional[FieldConfig]:
         """Get FieldConfig for a specific field."""

django_cfg/modules/django_admin/config/background_task_config.py

@@ -0,0 +1,76 @@
+"""
+Background task configuration for async operations.
+"""
+
+from typing import Literal, Optional
+
+from pydantic import BaseModel, ConfigDict, Field
+
+
+class BackgroundTaskConfig(BaseModel):
+    """
+    Configuration for background task processing.
+
+    Used for async operations like testing imported items,
+    bulk processing, or long-running operations.
+
+    Example:
+        ```python
+        background_config = BackgroundTaskConfig(
+            enabled=True,
+            task_runner='rearq',
+            batch_size=100,
+            timeout=300,
+        )
+        ```
+    """
+
+    model_config = ConfigDict(
+        validate_assignment=True,
+        extra="forbid"
+    )
+
+    enabled: bool = Field(
+        True,
+        description="Enable background task processing"
+    )
+
+    task_runner: Literal['rearq', 'celery', 'django_q', 'sync'] = Field(
+        'rearq',
+        description="Task runner to use for background operations"
+    )
+
+    batch_size: int = Field(
+        100,
+        ge=1,
+        le=10000,
+        description="Number of items to process in each batch"
+    )
+
+    timeout: int = Field(
+        300,
+        ge=1,
+        le=3600,
+        description="Task timeout in seconds (max 1 hour)"
+    )
+
+    retry_on_failure: bool = Field(
+        True,
+        description="Automatically retry failed tasks"
+    )
+
+    max_retries: int = Field(
+        3,
+        ge=0,
+        le=10,
+        description="Maximum number of retry attempts"
+    )
+
+    priority: Literal['high', 'normal', 'low'] = Field(
+        'normal',
+        description="Task priority in queue"
+    )
+
+    def should_use_background(self) -> bool:
+        """Check if background processing should be used."""
+        return self.enabled and self.task_runner != 'sync'