django-cfg 1.5.1__py3-none-any.whl → 1.5.3__py3-none-any.whl

django_cfg/apps/rq/services/models/README.md

@@ -0,0 +1,417 @@
+# RQ Internal Pydantic Models
+
+Внутренние Pydantic модели для бизнес-логики RQ.
+
+**ВАЖНО:** Все модели имеют **плоскую структуру** (никаких nested JSON)!
+
+---
+
+## 📁 Структура
+
+```
+services/models/
+├── __init__.py       # Экспорт всех моделей
+├── job.py            # RQJobModel, JobStatus
+├── worker.py         # RQWorkerModel, WorkerState
+├── queue.py          # RQQueueModel
+├── event.py          # Event models для Centrifugo
+└── README.md         # Этот файл
+```
+
+---
+
+## 🎯 Назначение
+
+### Разделение ответственности:
+
+**`/serializers/`** (DRF Serializers):
+- Для API endpoints (views)
+- OpenAPI schema generation
+- HTTP request/response validation
+
+**`/services/models/`** (Pydantic Models):
+- Внутренняя бизнес-логика
+- Type safety для сервисов
+- Валидация данных из RQ
+- Computed properties и методы
+
+---
+
+## 📝 Модели
+
+### 1. RQJobModel - Job данные
+
+```python
+from django_cfg.apps.rq.services.models import RQJobModel, JobStatus
+
+job = RQJobModel(
+    id="abc123",
+    func_name="myapp.tasks.send_email",
+    queue="default",
+    status=JobStatus.FINISHED,
+    created_at="2025-01-15T10:00:00Z",
+    started_at="2025-01-15T10:00:05Z",
+    ended_at="2025-01-15T10:00:10Z",
+    worker_name="worker1.12345",
+    timeout=180,
+    result_ttl=500,
+    args_json='["user@example.com", "Hello"]',
+    kwargs_json='{"priority": "high"}',
+    result_json='{"sent": true}',
+)
+
+# Properties
+print(job.is_success)  # True
+print(job.get_duration_seconds())  # 5.0
+```
+
+**Поля (все плоские!):**
+- Базовые: `id`, `func_name`, `queue`, `status`
+- Timestamps: `created_at`, `enqueued_at`, `started_at`, `ended_at` (ISO strings)
+- Worker: `worker_name`
+- Config: `timeout`, `result_ttl`, `failure_ttl`
+- Data: `args_json`, `kwargs_json`, `meta_json`, `result_json` (JSON strings)
+- Dependencies: `dependency_ids` (comma-separated)
+
+**Properties:**
+- `is_success` - успешность
+- `is_failed` - провал
+- `is_running` - выполняется
+- `get_duration_seconds()` - длительность
+
+---
+
+### 2. RQWorkerModel - Worker данные
+
+```python
+from django_cfg.apps.rq.services.models import RQWorkerModel, WorkerState
+
+worker = RQWorkerModel(
+    name="worker1.12345",
+    state=WorkerState.BUSY,
+    queues="default,high,low",  # Comma-separated!
+    current_job_id="abc123",
+    birth="2025-01-15T08:00:00Z",
+    last_heartbeat="2025-01-15T10:30:00Z",
+    successful_job_count=450,
+    failed_job_count=5,
+    total_working_time=12500.5,
+)
+
+# Properties
+print(worker.is_alive)  # True if heartbeat < 60s ago
+print(worker.is_busy)  # True
+print(worker.get_uptime_seconds())  # 9000.0
+print(worker.get_queue_list())  # ['default', 'high', 'low']
+print(worker.success_rate)  # 98.9%
+```
+
+**Поля (все плоские!):**
+- Базовые: `name`, `state`, `queues` (comma-separated!)
+- Current: `current_job_id`
+- Timestamps: `birth`, `last_heartbeat` (ISO strings)
+- Stats: `successful_job_count`, `failed_job_count`, `total_working_time`
+
+**Properties:**
+- `is_alive` - жив ли worker (heartbeat < 60s)
+- `is_busy` / `is_idle` - состояние
+- `get_uptime_seconds()` - время работы
+- `get_queue_list()` - список очередей
+- `total_job_count` - всего задач
+- `success_rate` - процент успеха
+
+---
+
+### 3. RQQueueModel - Queue статистика
+
+```python
+from django_cfg.apps.rq.services.models import RQQueueModel
+
+queue = RQQueueModel(
+    name="default",
+    is_async=True,
+    count=45,
+    queued_jobs=45,
+    started_jobs=2,
+    finished_jobs=1250,
+    failed_jobs=12,
+    deferred_jobs=0,
+    scheduled_jobs=5,
+    workers=3,
+    oldest_job_timestamp="2025-01-15T09:15:00Z",
+    connection_host="localhost",
+    connection_port=6379,
+    connection_db=0,
+)
+
+# Properties
+print(queue.total_jobs)  # 1314
+print(queue.completed_jobs)  # 1262
+print(queue.failure_rate)  # 0.95%
+print(queue.is_empty)  # False
+print(queue.has_workers)  # True
+print(queue.is_healthy)  # True
+```
+
+**Поля (все плоские!):**
+- Базовые: `name`, `is_async`, `count`
+- Job counts: `queued_jobs`, `started_jobs`, `finished_jobs`, `failed_jobs`, `deferred_jobs`, `scheduled_jobs`
+- Workers: `workers`
+- Metadata: `oldest_job_timestamp` (ISO string)
+- Connection: `connection_host`, `connection_port`, `connection_db` (flat!)
+
+**Properties:**
+- `total_jobs` - всего задач
+- `completed_jobs` - завершенных
+- `failure_rate` - процент провалов
+- `is_empty` - пустая ли очередь
+- `has_workers` - есть ли workers
+- `is_healthy` - здорова ли очередь
+
+---
+
+### 4. Event Models - для Centrifugo
+
+#### JobEventModel
+
+```python
+from django_cfg.apps.rq.services.models import JobEventModel, EventType
+
+event = JobEventModel(
+    event_type=EventType.JOB_FINISHED,
+    timestamp="2025-01-15T10:00:10Z",
+    job_id="abc123",
+    queue="default",
+    func_name="myapp.tasks.send_email",
+    status="finished",
+    worker_name="worker1.12345",
+    result_json='{"sent": true}',
+    duration_seconds=5.0,
+)
+```
+
+**Channel:** `rq:jobs`
+
+**Поля (все плоские!):**
+- Event: `event_type`, `timestamp`
+- Job: `job_id`, `queue`, `func_name`
+- Status: `status`, `worker_name`
+- Result: `result_json` (JSON string), `error`
+- Timing: `duration_seconds`
+
+#### QueueEventModel
+
+```python
+from django_cfg.apps.rq.services.models import QueueEventModel, EventType
+
+event = QueueEventModel(
+    event_type=EventType.QUEUE_PURGED,
+    timestamp="2025-01-15T10:00:00Z",
+    queue="default",
+    purged_count=45,
+    job_count=0,
+)
+```
+
+**Channel:** `rq:queues`
+
+**Поля (все плоские!):**
+- Event: `event_type`, `timestamp`
+- Queue: `queue`
+- Data: `purged_count`, `job_count`
+
+#### WorkerEventModel
+
+```python
+from django_cfg.apps.rq.services.models import WorkerEventModel, EventType
+
+event = WorkerEventModel(
+    event_type=EventType.WORKER_STARTED,
+    timestamp="2025-01-15T08:00:00Z",
+    worker_name="worker1.12345",
+    queues="default,high,low",  # Comma-separated!
+    state="idle",
+    successful_job_count=0,
+    failed_job_count=0,
+    total_working_time=0.0,
+)
+```
+
+**Channel:** `rq:workers`
+
+**Поля (все плоские!):**
+- Event: `event_type`, `timestamp`
+- Worker: `worker_name`, `queues` (comma-separated!)
+- State: `state`, `current_job_id`
+- Stats: `successful_job_count`, `failed_job_count`, `total_working_time`
+
+---
+
+## 💡 Примеры использования
+
+### Пример 1: Валидация Job данных
+
+```python
+from rq.job import Job
+from django_cfg.apps.rq.services.models import RQJobModel
+
+def validate_job(job: Job) -> RQJobModel:
+    """Validate RQ job with Pydantic."""
+    import json
+
+    return RQJobModel(
+        id=job.id,
+        func_name=job.func_name,
+        queue="default",  # or extract from job
+        status=job.get_status(),
+        created_at=job.created_at.isoformat(),
+        started_at=job.started_at.isoformat() if job.started_at else None,
+        ended_at=job.ended_at.isoformat() if job.ended_at else None,
+        worker_name=job.worker_name,
+        timeout=job.timeout,
+        result_ttl=job.result_ttl,
+        failure_ttl=job.failure_ttl,
+        args_json=json.dumps(list(job.args or [])),
+        kwargs_json=json.dumps(job.kwargs or {}),
+        meta_json=json.dumps(job.meta or {}),
+        result_json=json.dumps(job.result) if job.result else None,
+        exc_info=job.exc_info,
+        dependency_ids=",".join(job._dependency_ids or []),
+    )
+
+# Usage
+job = Job.fetch("abc123", connection=...)
+validated_job = validate_job(job)
+print(validated_job.is_success)
+print(validated_job.get_duration_seconds())
+```
+
+### Пример 2: Публикация события в Centrifugo
+
+```python
+from datetime import datetime
+from django_cfg.apps.rq.services.models import JobEventModel, EventType
+
+def publish_job_completed(job_model: RQJobModel):
+    """Publish job completion event."""
+    from django_cfg.apps.rq.services.centrifugo_publisher import publish_to_channel
+
+    event = JobEventModel(
+        event_type=EventType.JOB_FINISHED,
+        timestamp=datetime.now().isoformat(),
+        job_id=job_model.id,
+        queue=job_model.queue,
+        func_name=job_model.func_name,
+        status=job_model.status,
+        worker_name=job_model.worker_name,
+        result_json=job_model.result_json,
+        duration_seconds=job_model.get_duration_seconds(),
+    )
+
+    # Pydantic validates and serializes to flat JSON
+    publish_to_channel("rq:jobs", event.model_dump())
+```
+
+### Пример 3: Бизнес-логика с типизацией
+
+```python
+from typing import List
+from django_cfg.apps.rq.services.models import RQJobModel
+
+def calculate_avg_duration(jobs: List[RQJobModel]) -> float:
+    """Calculate average job duration with type safety."""
+    durations = [j.get_duration_seconds() for j in jobs if j.get_duration_seconds()]
+
+    if not durations:
+        return 0.0
+
+    return sum(durations) / len(durations)
+
+def get_failed_jobs(jobs: List[RQJobModel]) -> List[RQJobModel]:
+    """Filter failed jobs with type safety."""
+    return [j for j in jobs if j.is_failed]
+
+def group_by_queue(jobs: List[RQJobModel]) -> dict[str, List[RQJobModel]]:
+    """Group jobs by queue with type safety."""
+    result = {}
+    for job in jobs:
+        if job.queue not in result:
+            result[job.queue] = []
+        result[job.queue].append(job)
+    return result
+```
+
+---
+
+## ⚠️ Важные правила
+
+### 1. NO NESTED JSON!
+
+❌ **НЕПРАВИЛЬНО:**
+```python
+class BadJobModel(BaseModel):
+    id: str
+    config: JobConfig  # NESTED!
+    result: dict  # NESTED!
+```
+
+✅ **ПРАВИЛЬНО:**
+```python
+class GoodJobModel(BaseModel):
+    id: str
+    config_timeout: int
+    config_ttl: int
+    result_json: str  # JSON string!
+```
+
+### 2. JSON как строки
+
+Для сложных данных используем JSON strings:
+```python
+args_json: str = '["arg1", "arg2"]'
+kwargs_json: str = '{"key": "value"}'
+result_json: str = '{"success": true}'
+```
+
+### 3. Списки как comma-separated строки
+
+```python
+queues: str = "default,high,low"
+dependency_ids: str = "id1,id2,id3"
+
+# Получение списка:
+queue_list = queues.split(",")
+```
+
+### 4. Timestamps как ISO strings
+
+```python
+created_at: str = "2025-01-15T10:00:00Z"
+
+# Преобразование:
+dt = datetime.fromisoformat(created_at)
+```
+
+---
+
+## 🎯 Когда использовать
+
+**Используй Pydantic models когда:**
+- Нужна валидация данных из RQ
+- Нужна типизация для IDE/mypy
+- Нужны computed properties
+- Нужна бизнес-логика (расчеты, фильтры)
+- Готовишь данные для Centrifugo
+
+**НЕ используй Pydantic models для:**
+- API endpoints (используй DRF Serializers)
+- OpenAPI schema (используй DRF Serializers)
+- HTTP request/response (используй DRF Serializers)
+
+---
+
+## 📚 См. также
+
+- `/serializers/` - DRF Serializers для API
+- `/services/centrifugo_publisher.py` - публикация событий
+- `/services/config_helper.py` - работа с конфигом

django_cfg/apps/rq/services/models/__init__.py

@@ -0,0 +1,30 @@
+"""
+Pydantic models for internal RQ business logic.
+
+These models provide type safety and validation for internal operations,
+separate from DRF serializers used in API views.
+"""
+
+from .job import RQJobModel, JobStatus
+from .worker import RQWorkerModel, WorkerState
+from .queue import RQQueueModel
+from .event import JobEventModel, QueueEventModel, WorkerEventModel, EventType
+
+__all__ = [
+    # Job models
+    'RQJobModel',
+    'JobStatus',
+
+    # Worker models
+    'RQWorkerModel',
+    'WorkerState',
+
+    # Queue models
+    'RQQueueModel',
+
+    # Event models
+    'JobEventModel',
+    'QueueEventModel',
+    'WorkerEventModel',
+    'EventType',
+]

django_cfg/apps/rq/services/models/event.py

@@ -0,0 +1,123 @@
+"""
+Pydantic models for RQ Events (Centrifugo WebSocket).
+
+Internal models for event validation before publishing to Centrifugo.
+NO NESTED JSON - all fields are flat!
+"""
+
+from enum import Enum
+from typing import Optional
+
+from pydantic import BaseModel, Field
+
+
+class EventType(str, Enum):
+    """Event type enumeration for WebSocket events."""
+
+    # Job events
+    JOB_QUEUED = "job_queued"
+    JOB_STARTED = "job_started"
+    JOB_FINISHED = "job_finished"
+    JOB_FAILED = "job_failed"
+    JOB_CANCELED = "job_canceled"
+    JOB_REQUEUED = "job_requeued"
+    JOB_DELETED = "job_deleted"
+
+    # Queue events
+    QUEUE_PURGED = "queue_purged"
+    QUEUE_EMPTIED = "queue_emptied"
+
+    # Worker events
+    WORKER_STARTED = "worker_started"
+    WORKER_STOPPED = "worker_stopped"
+    WORKER_HEARTBEAT = "worker_heartbeat"
+
+
+class JobEventModel(BaseModel):
+    """
+    Job event for Centrifugo publishing.
+
+    FLAT STRUCTURE - no nested objects!
+    Published to channel: rq:jobs
+    """
+
+    # Event meta
+    event_type: EventType = Field(..., description="Event type")
+    timestamp: str = Field(..., description="Event timestamp (ISO 8601)")
+
+    # Job info
+    job_id: str = Field(..., description="Job ID")
+    queue: str = Field(..., description="Queue name")
+    func_name: Optional[str] = Field(None, description="Function name")
+
+    # Status info
+    status: Optional[str] = Field(None, description="Job status")
+    worker_name: Optional[str] = Field(None, description="Worker name")
+
+    # Result/Error (as JSON strings for flat structure)
+    result_json: Optional[str] = Field(None, description="Job result as JSON string")
+    error: Optional[str] = Field(None, description="Error message if failed")
+
+    # Timing
+    duration_seconds: Optional[float] = Field(None, description="Job duration in seconds")
+
+    class Config:
+        """Pydantic config."""
+
+        use_enum_values = True
+
+
+class QueueEventModel(BaseModel):
+    """
+    Queue event for Centrifugo publishing.
+
+    FLAT STRUCTURE - no nested objects!
+    Published to channel: rq:queues
+    """
+
+    # Event meta
+    event_type: EventType = Field(..., description="Event type")
+    timestamp: str = Field(..., description="Event timestamp (ISO 8601)")
+
+    # Queue info
+    queue: str = Field(..., description="Queue name")
+
+    # Event-specific data
+    purged_count: Optional[int] = Field(None, description="Number of jobs purged")
+    job_count: Optional[int] = Field(None, description="Current job count")
+
+    class Config:
+        """Pydantic config."""
+
+        use_enum_values = True
+
+
+class WorkerEventModel(BaseModel):
+    """
+    Worker event for Centrifugo publishing.
+
+    FLAT STRUCTURE - no nested objects!
+    Published to channel: rq:workers
+    """
+
+    # Event meta
+    event_type: EventType = Field(..., description="Event type")
+    timestamp: str = Field(..., description="Event timestamp (ISO 8601)")
+
+    # Worker info
+    worker_name: str = Field(..., description="Worker name")
+    queues: str = Field(..., description="Comma-separated queue names")
+
+    # State info
+    state: Optional[str] = Field(None, description="Worker state (idle/busy/suspended)")
+    current_job_id: Optional[str] = Field(None, description="Current job ID if busy")
+
+    # Stats
+    successful_job_count: Optional[int] = Field(None, description="Successful job count")
+    failed_job_count: Optional[int] = Field(None, description="Failed job count")
+    total_working_time: Optional[float] = Field(None, description="Total working time in seconds")
+
+    class Config:
+        """Pydantic config."""
+
+        use_enum_values = True

django_cfg/apps/rq/services/models/job.py

@@ -0,0 +1,99 @@
+"""
+Pydantic models for RQ Jobs.
+
+Internal models for job validation and business logic.
+NO NESTED JSON - all fields are flat!
+"""
+
+from datetime import datetime
+from enum import Enum
+from typing import Optional
+
+from pydantic import BaseModel, Field
+
+
+class JobStatus(str, Enum):
+    """Job status enumeration."""
+
+    QUEUED = "queued"
+    STARTED = "started"
+    FINISHED = "finished"
+    FAILED = "failed"
+    DEFERRED = "deferred"
+    SCHEDULED = "scheduled"
+    CANCELED = "canceled"
+
+
+class RQJobModel(BaseModel):
+    """
+    Internal model for RQ Job with validation.
+
+    FLAT STRUCTURE - no nested objects!
+    All timestamps are ISO strings, all complex types are flattened.
+    """
+
+    # Basic info
+    id: str = Field(..., description="Job ID")
+    func_name: str = Field(..., description="Function name (e.g., 'myapp.tasks.send_email')")
+    queue: str = Field(..., description="Queue name")
+
+    # Status
+    status: JobStatus = Field(..., description="Current job status")
+
+    # Timestamps (as ISO strings for flat JSON)
+    created_at: str = Field(..., description="Creation timestamp (ISO 8601)")
+    enqueued_at: Optional[str] = Field(None, description="Enqueue timestamp (ISO 8601)")
+    started_at: Optional[str] = Field(None, description="Start timestamp (ISO 8601)")
+    ended_at: Optional[str] = Field(None, description="End timestamp (ISO 8601)")
+
+    # Worker info
+    worker_name: Optional[str] = Field(None, description="Worker name if job is/was running")
+
+    # Configuration (flat!)
+    timeout: Optional[int] = Field(None, description="Job timeout in seconds")
+    result_ttl: Optional[int] = Field(None, description="Result TTL in seconds")
+    failure_ttl: Optional[int] = Field(None, description="Failure TTL in seconds")
+
+    # Result/Error (as strings, not nested!)
+    result_json: Optional[str] = Field(None, description="Job result as JSON string")
+    exc_info: Optional[str] = Field(None, description="Exception info if failed")
+
+    # Args/Kwargs (as JSON strings for flat structure)
+    args_json: str = Field(default="[]", description="Function args as JSON string")
+    kwargs_json: str = Field(default="{}", description="Function kwargs as JSON string")
+    meta_json: str = Field(default="{}", description="Job metadata as JSON string")
+
+    # Dependencies (comma-separated for flat structure)
+    dependency_ids: str = Field(default="", description="Comma-separated dependency job IDs")
+
+    @property
+    def is_success(self) -> bool:
+        """Check if job succeeded."""
+        return self.status == JobStatus.FINISHED
+
+    @property
+    def is_failed(self) -> bool:
+        """Check if job failed."""
+        return self.status == JobStatus.FAILED
+
+    @property
+    def is_running(self) -> bool:
+        """Check if job is running."""
+        return self.status == JobStatus.STARTED
+
+    def get_duration_seconds(self) -> Optional[float]:
+        """Calculate job duration in seconds."""
+        if not self.started_at or not self.ended_at:
+            return None
+
+        try:
+            start = datetime.fromisoformat(self.started_at)
+            end = datetime.fromisoformat(self.ended_at)
+            return (end - start).total_seconds()
+        except (ValueError, TypeError):
+            return None
+
+    class Config:
+        """Pydantic config."""
+
+        use_enum_values = True