plain.jobs 0.33.0__py3-none-any.whl

plain/jobs/CHANGELOG.md

@@ -0,0 +1,186 @@
+# plain-jobs changelog
+
+## [0.33.0](https://github.com/dropseed/plain/releases/plain-jobs@0.33.0) (2025-10-10)
+
+### What's changed
+
+- Renamed package from `plain.worker` to `plain.jobs` ([24219856e0](https://github.com/dropseed/plain/commit/24219856e0))
+
+### Upgrade instructions
+
+- Update any imports from `plain.worker` to `plain.jobs` (e.g., `from plain.worker import Job` becomes `from plain.jobs import Job`)
+- Change worker commands from `plain worker run` to `plain jobs worker`
+- Check updated settings names
+
+## [0.32.0](https://github.com/dropseed/plain/releases/plain-jobs@0.32.0) (2025-10-07)
+
+### What's changed
+
+- Models now use `model_options` instead of `_meta` for accessing model configuration like `package_label` and `model_name` ([73ba469](https://github.com/dropseed/plain/commit/73ba469ba0))
+- Model configuration now uses `model_options = models.Options()` instead of `class Meta` ([17a378d](https://github.com/dropseed/plain/commit/17a378dcfb))
+- QuerySet types now properly use `Self` return type for better type checking ([2578301](https://github.com/dropseed/plain/commit/2578301819))
+- Removed unnecessary type ignore comments now that QuerySet is properly typed ([2578301](https://github.com/dropseed/plain/commit/2578301819))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.31.1](https://github.com/dropseed/plain/releases/plain-jobs@0.31.1) (2025-10-06)
+
+### What's changed
+
+- Updated dependency resolution to use newer compatible versions of `plain` and `plain.models`
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.31.0](https://github.com/dropseed/plain/releases/plain-jobs@0.31.0) (2025-09-25)
+
+### What's changed
+
+- The jobs autodiscovery now includes `app.jobs` modules in addition to package jobs modules ([b0b610d](https://github.com/dropseed/plain/commit/b0b610d461))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.30.0](https://github.com/dropseed/plain/releases/plain-jobs@0.30.0) (2025-09-19)
+
+### What's changed
+
+- The `Job` model has been renamed to `JobProcess` for better clarity ([986c914](https://github.com/dropseed/plain/commit/986c914))
+- The `job_uuid` field in JobResult has been renamed to `job_process_uuid` to match the model rename ([986c914](https://github.com/dropseed/plain/commit/986c914))
+- Admin interface now shows "Job processes" as the section title instead of "Jobs" ([986c914](https://github.com/dropseed/plain/commit/986c914))
+
+### Upgrade instructions
+
+- Run `plain migrate` to apply the database migration that renames the Job model to JobProcess
+- If you have any custom code that directly references the `Job` model (different than the `Job` base class for job type definitions), update it to use `JobProcess` instead
+- If you have any code that accesses the `job_uuid` field on JobResult instances, update it to use `job_process_uuid`
+
+## [0.29.0](https://github.com/dropseed/plain/releases/plain-jobs@0.29.0) (2025-09-12)
+
+### What's changed
+
+- Model managers have been renamed from `.objects` to `.query` ([037a239](https://github.com/dropseed/plain/commit/037a239ef4))
+- Manager functionality has been merged into QuerySet classes ([bbaee93](https://github.com/dropseed/plain/commit/bbaee93839))
+- Models now use `Meta.queryset_class` instead of separate manager configuration ([6b60a00](https://github.com/dropseed/plain/commit/6b60a00731))
+
+### Upgrade instructions
+
+- Update all model queries to use `.query` instead of `.objects` (e.g., `Job.query.all()` becomes `Job.query.all()`)
+
+## [0.28.1](https://github.com/dropseed/plain/releases/plain-jobs@0.28.1) (2025-09-10)
+
+### What's changed
+
+- Fixed log context method in worker middleware to use `include_context` instead of `with_context` ([755f873](https://github.com/dropseed/plain/commit/755f873986))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.28.0](https://github.com/dropseed/plain/releases/plain-jobs@0.28.0) (2025-09-09)
+
+### What's changed
+
+- Improved logging middleware to use context manager pattern for cleaner job context handling ([ea7c953](https://github.com/dropseed/plain/commit/ea7c9537e3))
+- Updated minimum Python requirement to 3.13 ([d86e307](https://github.com/dropseed/plain/commit/d86e307efb))
+- Added explicit nav_icon definitions to admin views to ensure consistent icon display ([2aac07d](https://github.com/dropseed/plain/commit/2aac07de4e))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.27.1](https://github.com/dropseed/plain/releases/plain-jobs@0.27.1) (2025-08-27)
+
+### What's changed
+
+- Jobs are now marked as cancelled when the worker process is killed or fails unexpectedly ([e73ca53](https://github.com/dropseed/plain/commit/e73ca53c3d))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.27.0](https://github.com/dropseed/plain/releases/plain-jobs@0.27.0) (2025-08-22)
+
+### What's changed
+
+- Added support for date and datetime job parameters with proper serialization/deserialization ([7bb5ab0911](https://github.com/dropseed/plain/commit/7bb5ab0911))
+- Improved job priority documentation to clarify that higher numbers run first ([73271b5bf0](https://github.com/dropseed/plain/commit/73271b5bf0))
+- Updated admin interface with consolidated navigation icons at the section level ([5a6479ac79](https://github.com/dropseed/plain/commit/5a6479ac79))
+- Enhanced admin views to use cached object properties for better performance ([bd0507a72c](https://github.com/dropseed/plain/commit/bd0507a72c))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.26.0](https://github.com/dropseed/plain/releases/plain-jobs@0.26.0) (2025-08-19)
+
+### What's changed
+
+- Improved CSRF token handling in admin forms by removing manual `csrf_input` in favor of automatic Sec-Fetch-Site header validation ([955150800c](https://github.com/dropseed/plain/commit/955150800c))
+- Enhanced README documentation with comprehensive examples, table of contents, and detailed sections covering job parameters, scheduling, monitoring, and FAQs ([4ebecd1856](https://github.com/dropseed/plain/commit/4ebecd1856))
+- Updated package description to be more descriptive: "Process background jobs with a database-driven worker" ([4ebecd1856](https://github.com/dropseed/plain/commit/4ebecd1856))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.25.1](https://github.com/dropseed/plain/releases/plain-jobs@0.25.1) (2025-07-23)
+
+### What's changed
+
+- Added Bootstrap icons to admin interface for worker job views ([9e9f8b0](https://github.com/dropseed/plain/commit/9e9f8b0e2ccc3174f05034e6e908bb26345e1a5c))
+- Removed the description field from admin views ([8d2352d](https://github.com/dropseed/plain/commit/8d2352db94277ddd87b6a480783c9f740b6e806f))
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.25.0](https://github.com/dropseed/plain/releases/plain-jobs@0.25.0) (2025-07-22)
+
+### What's changed
+
+- Removed `pk` alias and auto fields in favor of a single automatic `id` PrimaryKeyField ([4b8fa6a](https://github.com/dropseed/plain/commit/4b8fa6aef126a15e48b5f85e0652adf841eb7b5c))
+- Admin interface methods now use `target_ids` parameter instead of `target_pks` for batch actions
+- Model instance registry now uses `.id` instead of `.pk` for Global ID generation
+- Updated database migrations to use `models.PrimaryKeyField()` instead of `models.BigAutoField()`
+
+### Upgrade instructions
+
+- No changes required
+
+## [0.24.0](https://github.com/dropseed/plain/releases/plain-jobs@0.24.0) (2025-07-18)
+
+### What's changed
+
+- Added OpenTelemetry tracing support to job processing system ([b0224d0](https://github.com/dropseed/plain/commit/b0224d0418))
+- Job requests now capture trace context when queued from traced operations
+- Job execution creates proper consumer spans linked to the original producer trace
+- Added `trace_id` and `span_id` fields to JobRequest, Job, and JobResult models for trace correlation
+
+### Upgrade instructions
+
+- Run `plain migrate` to apply new database migration that adds trace context fields to worker tables
+
+## [0.23.0](https://github.com/dropseed/plain/releases/plain-jobs@0.23.0) (2025-07-18)
+
+### What's changed
+
+- Migrations have been reset and consolidated into a single initial migration ([484f1b6e93](https://github.com/dropseed/plain/commit/484f1b6e93))
+
+### Upgrade instructions
+
+- Run `plain migrate --prune plainworker` to remove old migration records and apply the consolidated migration
+
+## [0.22.5](https://github.com/dropseed/plain/releases/plain-jobs@0.22.5) (2025-06-24)
+
+### What's changed
+
+- No functional changes. This release only updates internal documentation (CHANGELOG) and contains no code modifications that impact users ([82710c3](https://github.com/dropseed/plain/commit/82710c3), [9a1963d](https://github.com/dropseed/plain/commit/9a1963d), [e1f5dd3](https://github.com/dropseed/plain/commit/e1f5dd3)).
+
+### Upgrade instructions
+
+- No changes required

plain/jobs/README.md

@@ -0,0 +1,253 @@
+# plain.jobs
+
+**Process background jobs with a database-driven job queue.**
+
+- [Overview](#overview)
+- [Local development](#local-development)
+- [Job parameters](#job-parameters)
+- [Job methods](#job-methods)
+- [Scheduled jobs](#scheduled-jobs)
+- [Admin interface](#admin-interface)
+- [Job history](#job-history)
+- [Monitoring](#monitoring)
+- [FAQs](#faqs)
+- [Installation](#installation)
+
+## Overview
+
+Jobs are defined using the [`Job`](./jobs.py#Job) base class and the `run()` method at a minimum.
+
+```python
+from plain.jobs import Job, register_job
+from plain.email import send_mail
+
+
+@register_job
+class WelcomeUserJob(Job):
+    def __init__(self, user):
+        self.user = user
+
+    def run(self):
+        send_mail(
+            subject="Welcome!",
+            message=f"Hello from Plain, {self.user}",
+            from_email="welcome@plainframework.com",
+            recipient_list=[self.user.email],
+        )
+```
+
+You can then create an instance of the job and call [`run_in_worker()`](./jobs.py#Job.run_in_worker) to enqueue it for a background worker to pick up.
+
+```python
+user = User.query.get(id=1)
+WelcomeUserJob(user).run_in_worker()
+```
+
+Workers are run using the `plain jobs worker` command.
+
+Jobs can be defined in any Python file, but it is suggested to use `app/jobs.py` or `app/{pkg}/jobs.py` as those will be imported automatically so the [`@register_job`](./registry.py#register_job) decorator will fire.
+
+Run database migrations after installation:
+
+```bash
+plain migrate
+```
+
+## Local development
+
+In development, you will typically want to run the worker alongside your app. With [`plain.dev`](/plain-dev/plain/dev/README.md) you can do this by adding it to the `[tool.plain.dev.run]` section of your `pyproject.toml` file. Currently, you will need to use something like [watchfiles](https://pypi.org/project/watchfiles/) to add auto-reloading to the worker.
+
+```toml
+# pyproject.toml
+[tool.plain.dev.run]
+worker = {cmd = "watchfiles --filter python \"plain jobs worker --stats-every 0 --max-processes 2\" ."}
+worker-slow = {cmd = "watchfiles --filter python \"plain jobs worker --queue slow --stats-every 0 --max-processes 2\" ."}
+```
+
+## Job parameters
+
+When calling `run_in_worker()`, you can specify several parameters to control job execution:
+
+```python
+job.run_in_worker(
+    queue="slow",  # Target a specific queue (default: "default")
+    delay=60,  # Delay in seconds (or timedelta/datetime)
+    priority=10,  # Higher numbers run first (default: 0, use negatives for lower priority)
+    retries=3,  # Number of retry attempts (default: 0)
+    unique_key="user-123-welcome",  # Prevent duplicate jobs
+)
+```
+
+For more advanced parameter options, see [`Job.run_in_worker()`](./jobs.py#Job.run_in_worker).
+
+## Job methods
+
+The [`Job`](./jobs.py#Job) base class provides several methods you can override to customize behavior:
+
+```python
+class MyJob(Job):
+    def run(self):
+        # Required: The main job logic
+        pass
+
+    def get_queue(self) -> str:
+        # Specify the default queue for this job type
+        return "default"
+
+    def get_priority(self) -> int:
+        # Set the default priority
+        # Higher numbers run first: 10 > 5 > 0 > -5 > -10
+        # Use positive numbers for high priority, negative for low priority
+        return 0
+
+    def get_retries(self) -> int:
+        # Number of retry attempts on failure
+        return 0
+
+    def get_retry_delay(self, attempt: int) -> int:
+        # Delay in seconds before retry (attempt starts at 1)
+        return 0
+
+    def get_unique_key(self) -> str:
+        # Return a key to prevent duplicate jobs
+        return ""
+```
+
+## Scheduled jobs
+
+You can schedule jobs to run at specific times using the [`Schedule`](./scheduling.py#Schedule) class:
+
+```python
+from plain.jobs import Job, register_job
+from plain.jobs.scheduling import Schedule
+
+@register_job
+class DailyReportJob(Job):
+    schedule = Schedule.from_cron("0 9 * * *")  # Every day at 9 AM
+
+    def run(self):
+        # Generate daily report
+        pass
+```
+
+The `Schedule` class supports standard cron syntax and special strings:
+
+- `@yearly` or `@annually` - Run once a year
+- `@monthly` - Run once a month
+- `@weekly` - Run once a week
+- `@daily` or `@midnight` - Run once a day
+- `@hourly` - Run once an hour
+
+For custom schedules, see [`Schedule`](./scheduling.py#Schedule).
+
+## Admin interface
+
+The jobs package includes admin views for monitoring jobs under the "Jobs" section. The admin interface provides:
+
+- **Requests**: View pending jobs in the queue
+- **Processes**: Monitor currently running jobs
+- **Results**: Review completed and failed job history
+
+Dashboard cards show at-a-glance statistics for successful, errored, lost, and retried jobs.
+
+## Job history
+
+Job execution history is stored in the [`JobResult`](./models.py#JobResult) model. This includes:
+
+- Job class and parameters
+- Start and end times
+- Success/failure status
+- Error messages and tracebacks for failed jobs
+- Worker information
+
+History retention is controlled by the `JOBS_RESULTS_RETENTION` setting (defaults to 7 days):
+
+```python
+# app/settings.py
+JOBS_RESULTS_RETENTION = 60 * 60 * 24 * 30  # 30 days (in seconds)
+```
+
+Job timeout can be configured with `JOBS_TIMEOUT` (defaults to 1 day):
+
+```python
+# app/settings.py
+JOBS_TIMEOUT = 60 * 60 * 24  # 1 day (in seconds)
+```
+
+## Monitoring
+
+Workers report statistics and can be monitored using the `--stats-every` option:
+
+```bash
+# Report stats every 60 seconds
+plain jobs worker --stats-every 60
+```
+
+The worker integrates with OpenTelemetry for distributed tracing. Spans are created for:
+
+- Job scheduling (`run_in_worker`)
+- Job execution
+- Job completion/failure
+
+Jobs can be linked to the originating trace context, allowing you to track jobs initiated from web requests.
+
+## FAQs
+
+#### How do I ensure a job only runs once?
+
+Return a unique key from the `get_unique_key()` method:
+
+```python
+class ProcessUserDataJob(Job):
+    def __init__(self, user_id):
+        self.user_id = user_id
+
+    def get_unique_key(self):
+        return f"process-user-{self.user_id}"
+```
+
+#### Can I run multiple workers?
+
+Yes, you can run multiple worker processes:
+
+```bash
+plain jobs worker --max-processes 4
+```
+
+Or run workers for specific queues:
+
+```bash
+plain jobs worker --queue slow --max-processes 2
+```
+
+#### How do I handle job failures?
+
+Set the number of retries and implement retry delays:
+
+```python
+class MyJob(Job):
+    def get_retries(self):
+        return 3
+
+    def get_retry_delay(self, attempt):
+        # Exponential backoff: 1s, 2s, 4s
+        return 2 ** (attempt - 1)
+```
+
+## Installation
+
+Install the `plain.jobs` package from [PyPI](https://pypi.org/project/plain.jobs/):
+
+```bash
+uv add plain.jobs
+```
+
+Add to your `INSTALLED_PACKAGES`:
+
+```python
+# app/settings.py
+INSTALLED_PACKAGES = [
+    ...
+    "plain.jobs",
+]
+```

plain/jobs/__init__.py

@@ -0,0 +1,4 @@
+from .jobs import Job
+from .registry import register_job
+
+__all__ = ["Job", "register_job"]

plain/jobs/admin.py

@@ -0,0 +1,238 @@
+from __future__ import annotations
+
+from datetime import timedelta
+from typing import Any
+
+from plain import models
+from plain.admin.cards import Card
+from plain.admin.views import (
+    AdminModelDetailView,
+    AdminModelListView,
+    AdminViewset,
+    register_viewset,
+)
+from plain.http import ResponseRedirect
+from plain.runtime import settings
+
+from .models import JobProcess, JobRequest, JobResult
+
+
+def _td_format(td_object: timedelta) -> str:
+    seconds = int(td_object.total_seconds())
+    periods = [
+        ("year", 60 * 60 * 24 * 365),
+        ("month", 60 * 60 * 24 * 30),
+        ("day", 60 * 60 * 24),
+        ("hour", 60 * 60),
+        ("minute", 60),
+        ("second", 1),
+    ]
+
+    strings = []
+    for period_name, period_seconds in periods:
+        if seconds > period_seconds:
+            period_value, seconds = divmod(seconds, period_seconds)
+            has_s = "s" if period_value > 1 else ""
+            strings.append(f"{period_value} {period_name}{has_s}")
+
+    return ", ".join(strings)
+
+
+class SuccessfulJobsCard(Card):
+    title = "Successful"
+    text = "View"
+
+    def get_number(self) -> int:
+        return JobResult.query.successful().count()
+
+    def get_link(self) -> str:
+        return JobResultViewset.ListView.get_view_url() + "?display=Successful"
+
+
+class ErroredJobsCard(Card):
+    title = "Errored"
+    text = "View"
+
+    def get_number(self) -> int:
+        return JobResult.query.errored().count()
+
+    def get_link(self) -> str:
+        return JobResultViewset.ListView.get_view_url() + "?display=Errored"
+
+
+class LostJobsCard(Card):
+    title = "Lost"
+    text = "View"  # TODO make not required - just an icon?
+
+    def get_description(self) -> str:
+        delta = timedelta(seconds=settings.JOBS_TIMEOUT)
+        return f"Jobs are considered lost after {_td_format(delta)}"
+
+    def get_number(self) -> int:
+        return JobResult.query.lost().count()
+
+    def get_link(self) -> str:
+        return JobResultViewset.ListView.get_view_url() + "?display=Lost"
+
+
+class RetriedJobsCard(Card):
+    title = "Retried"
+    text = "View"  # TODO make not required - just an icon?
+
+    def get_number(self) -> int:
+        return JobResult.query.retried().count()
+
+    def get_link(self) -> str:
+        return JobResultViewset.ListView.get_view_url() + "?display=Retried"
+
+
+class WaitingJobsCard(Card):
+    title = "Waiting"
+
+    def get_number(self) -> int:
+        return JobProcess.query.waiting().count()
+
+
+class RunningJobsCard(Card):
+    title = "Running"
+
+    def get_number(self) -> int:
+        return JobProcess.query.running().count()
+
+
+@register_viewset
+class JobRequestViewset(AdminViewset):
+    class ListView(AdminModelListView):
+        nav_section = "Jobs"
+        nav_icon = "gear"
+        model = JobRequest
+        title = "Requests"
+        fields = ["id", "job_class", "priority", "created_at", "start_at", "unique_key"]
+        actions = ["Delete"]
+
+        def perform_action(self, action: str, target_ids: list[int]) -> None:
+            if action == "Delete":
+                JobRequest.query.filter(id__in=target_ids).delete()
+
+    class DetailView(AdminModelDetailView):
+        model = JobRequest
+        title = "Request"
+
+
+@register_viewset
+class JobProcessViewset(AdminViewset):
+    class ListView(AdminModelListView):
+        nav_section = "Jobs"
+        nav_icon = "gear"
+        model = JobProcess
+        title = "Processes"
+        fields = [
+            "id",
+            "job_class",
+            "priority",
+            "created_at",
+            "started_at",
+            "unique_key",
+        ]
+        actions = ["Delete"]
+        cards = [
+            WaitingJobsCard,
+            RunningJobsCard,
+        ]
+
+        def perform_action(self, action: str, target_ids: list[int]) -> None:
+            if action == "Delete":
+                JobProcess.query.filter(id__in=target_ids).delete()
+
+    class DetailView(AdminModelDetailView):
+        model = JobProcess
+        title = "Process"
+
+
+@register_viewset
+class JobResultViewset(AdminViewset):
+    class ListView(AdminModelListView):
+        nav_section = "Jobs"
+        nav_icon = "gear"
+        model = JobResult
+        title = "Results"
+        fields = [
+            "id",
+            "job_class",
+            "priority",
+            "created_at",
+            "status",
+            "retried",
+            "is_retry",
+        ]
+        search_fields = [
+            "uuid",
+            "job_process_uuid",
+            "job_request_uuid",
+            "job_class",
+        ]
+        cards = [
+            SuccessfulJobsCard,
+            ErroredJobsCard,
+            LostJobsCard,
+            RetriedJobsCard,
+        ]
+        filters = [
+            "Successful",
+            "Errored",
+            "Cancelled",
+            "Lost",
+            "Retried",
+        ]
+        actions = [
+            "Retry",
+        ]
+        allow_global_search = False
+
+        def get_initial_queryset(self) -> Any:
+            queryset = super().get_initial_queryset()
+            queryset = queryset.annotate(
+                retried=models.Case(
+                    models.When(retry_job_request_uuid__isnull=False, then=True),
+                    default=False,
+                    output_field=models.BooleanField(),
+                ),
+                is_retry=models.Case(
+                    models.When(retry_attempt__gt=0, then=True),
+                    default=False,
+                    output_field=models.BooleanField(),
+                ),
+            )
+            if self.display == "Successful":
+                return queryset.successful()
+            if self.display == "Errored":
+                return queryset.errored()
+            if self.display == "Cancelled":
+                return queryset.cancelled()
+            if self.display == "Lost":
+                return queryset.lost()
+            if self.display == "Retried":
+                return queryset.retried()
+            return queryset
+
+        def get_fields(self) -> list[str]:
+            fields = super().get_fields()
+            if self.display == "Retried":
+                fields.append("retries")
+                fields.append("retry_attempt")
+            return fields
+
+        def perform_action(self, action: str, target_ids: list[int]) -> None:
+            if action == "Retry":
+                for result in JobResult.query.filter(id__in=target_ids):
+                    result.retry_job(delay=0)
+            else:
+                raise ValueError("Invalid action")
+
+    class DetailView(AdminModelDetailView):
+        model = JobResult
+        title = "Result"
+
+        def post(self) -> ResponseRedirect:
+            self.object.retry_job(delay=0)
+            return ResponseRedirect(".")

plain/jobs/chores.py

@@ -0,0 +1,17 @@
+import datetime
+
+from plain.chores import register_chore
+from plain.runtime import settings
+from plain.utils import timezone
+
+from .models import JobResult
+
+
+@register_chore("jobs")
+def clear_completed() -> str:
+    """Delete all completed job results in all queues."""
+    cutoff = timezone.now() - datetime.timedelta(
+        seconds=settings.JOBS_RESULTS_RETENTION
+    )
+    results = JobResult.query.filter(created_at__lt=cutoff).delete()
+    return f"{results[0]} jobs deleted"