django-tasks-local-db 0.1.0__tar.gz

django_tasks_local_db-0.1.0/LICENSE

@@ -0,0 +1,79 @@
+MIT License
+
+Copyright (c) 2026 Forest Gregg
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+This project incorporates code from the following projects:
+
+## django-tasks-local
+
+MIT License
+
+Copyright (c) 2026 Lincoln Loop
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+## django-tasks-db
+
+Copyright (c) Jake Howard and individual contributors.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+    1. Redistributions of source code must retain the above copyright notice,
+       this list of conditions and the following disclaimer.
+
+    2. Redistributions in binary form must reproduce the above copyright
+       notice, this list of conditions and the following disclaimer in the
+       documentation and/or other materials provided with the distribution.
+
+    3. Neither the name of the copyright holder nor the names of its
+       contributors may be used to endorse or promote products derived from
+       this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

django_tasks_local_db-0.1.0/PKG-INFO

@@ -0,0 +1,53 @@
+Metadata-Version: 2.4
+Name: django-tasks-local-db
+Version: 0.1.0
+Summary: Django Tasks backend combining in-process thread pool execution with database persistence
+License-Expression: MIT
+Requires-Python: >=3.12
+Requires-Dist: django>=6.0
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-django; extra == "dev"
+Description-Content-Type: text/markdown
+
+# django-tasks-local-db
+
+A [Django Tasks](https://docs.djangoproject.com/en/6.0/topics/tasks/) backend that combines in-process thread pool execution with database persistence. Tasks run in your web process without a separate worker, while results are durably stored in the database.
+
+This project is a remix of [django-tasks-local](https://github.com/lincolnloop/django-tasks-local) and [django-tasks-db](https://github.com/RealOrangeOne/django-tasks-db). It takes the in-process executor approach from django-tasks-local and adds DB-backed result storage from django-tasks-db.
+
+## How it works
+
+- `enqueue()` writes a task row to the database, then submits it to an in-process thread pool
+- On completion, the DB row is updated with the result or error
+- On restart, orphaned tasks are recovered from the database and resubmitted
+
+## Installation
+
+```bash
+pip install django-tasks-local-db
+```
+
+## Configuration
+
+```python
+INSTALLED_APPS = [
+    # ...
+    "django_tasks_local_db",
+]
+
+TASKS = {
+    "default": {
+        "BACKEND": "django_tasks_local_db.LocalDBBackend",
+        "OPTIONS": {
+            "MAX_WORKERS": 4,
+        },
+    }
+}
+```
+
+Then run migrations:
+
+```bash
+python manage.py migrate django_tasks_local_db
+```

django_tasks_local_db-0.1.0/README.md

@@ -0,0 +1,41 @@
+# django-tasks-local-db
+
+A [Django Tasks](https://docs.djangoproject.com/en/6.0/topics/tasks/) backend that combines in-process thread pool execution with database persistence. Tasks run in your web process without a separate worker, while results are durably stored in the database.
+
+This project is a remix of [django-tasks-local](https://github.com/lincolnloop/django-tasks-local) and [django-tasks-db](https://github.com/RealOrangeOne/django-tasks-db). It takes the in-process executor approach from django-tasks-local and adds DB-backed result storage from django-tasks-db.
+
+## How it works
+
+- `enqueue()` writes a task row to the database, then submits it to an in-process thread pool
+- On completion, the DB row is updated with the result or error
+- On restart, orphaned tasks are recovered from the database and resubmitted
+
+## Installation
+
+```bash
+pip install django-tasks-local-db
+```
+
+## Configuration
+
+```python
+INSTALLED_APPS = [
+    # ...
+    "django_tasks_local_db",
+]
+
+TASKS = {
+    "default": {
+        "BACKEND": "django_tasks_local_db.LocalDBBackend",
+        "OPTIONS": {
+            "MAX_WORKERS": 4,
+        },
+    }
+}
+```
+
+Then run migrations:
+
+```bash
+python manage.py migrate django_tasks_local_db
+```

django_tasks_local_db-0.1.0/django_tasks_local_db/__init__.py

@@ -0,0 +1,3 @@
+from .backend import LocalDBBackend
+
+__all__ = ["LocalDBBackend"]

django_tasks_local_db-0.1.0/django_tasks_local_db/admin.py

@@ -0,0 +1,55 @@
+from django.contrib import admin
+from django.utils.html import format_html
+
+from .models import DBTaskResult
+
+
+@admin.register(DBTaskResult)
+class DBTaskResultAdmin(admin.ModelAdmin):
+    list_display = [
+        "id",
+        "task_name",
+        "status",
+        "enqueued_at",
+        "started_at",
+        "finished_at",
+        "priority",
+        "queue_name",
+    ]
+    list_filter = ["status", "priority", "queue_name"]
+    readonly_fields = [
+        "id",
+        "task_name",
+        "task_path",
+        "status",
+        "enqueued_at",
+        "started_at",
+        "finished_at",
+        "priority",
+        "queue_name",
+        "backend_name",
+        "args_kwargs",
+        "return_value",
+        "exception_class_path",
+        "formatted_traceback",
+    ]
+    exclude = ["traceback", "worker_ids"]
+
+    def has_add_permission(self, request):
+        return False
+
+    def has_delete_permission(self, request, obj=None):
+        return False
+
+    def has_change_permission(self, request, obj=None):
+        return False
+
+    @admin.display(description="Task name")
+    def task_name(self, obj):
+        return obj.task_name
+
+    @admin.display(description="Traceback")
+    def formatted_traceback(self, obj):
+        if obj.traceback:
+            return format_html("<pre>{}</pre>", obj.traceback)
+        return ""

django_tasks_local_db-0.1.0/django_tasks_local_db/apps.py

@@ -0,0 +1,52 @@
+import logging
+
+from django.apps import AppConfig
+
+logger = logging.getLogger("django_tasks_local_db")
+
+
+class DjangoTasksLocalDbConfig(AppConfig):
+    name = "django_tasks_local_db"
+    verbose_name = "Django Tasks Local DB"
+    default_auto_field = "django.db.models.BigAutoField"
+
+    def ready(self):
+        import threading
+
+        # Defer recovery until after Django is fully initialized.
+        # This avoids "database accessed during app initialization" warnings
+        # and ensures the task backends are fully configured.
+        thread = threading.Thread(target=self._recover_orphaned_tasks, daemon=True)
+        thread.start()
+
+    def _recover_orphaned_tasks(self):
+        import time
+
+        from django.conf import settings
+
+        # Brief delay to ensure Django startup completes
+        time.sleep(0.5)
+
+        from .backend import LocalDBBackend
+
+        tasks_settings = getattr(settings, "TASKS", {})
+        for alias, params in tasks_settings.items():
+            backend_path = params.get("BACKEND", "")
+            if "django_tasks_local_db" not in backend_path:
+                continue
+            try:
+                from django.tasks import task_backends
+
+                backend = task_backends[alias]
+                if isinstance(backend, LocalDBBackend):
+                    recovered = backend.recover_tasks()
+                    if recovered:
+                        logger.info(
+                            "Recovered %d orphaned task(s) for backend '%s'",
+                            recovered,
+                            alias,
+                        )
+            except Exception:
+                logger.exception(
+                    "Failed to recover tasks for backend '%s'", alias
+                )

django_tasks_local_db-0.1.0/django_tasks_local_db/backend.py

@@ -0,0 +1,199 @@
+import logging
+from concurrent.futures import Future, ThreadPoolExecutor
+from typing import Any
+
+from django.db import close_old_connections, transaction
+from django.tasks.backends.base import BaseTaskBackend
+from django.tasks.base import Task, TaskContext, TaskResultStatus
+from django.tasks.exceptions import TaskResultDoesNotExist
+from django.tasks.signals import task_enqueued, task_finished, task_started
+from django.utils.crypto import get_random_string
+from django.utils.json import normalize_json
+from django.utils.module_loading import import_string
+
+from .state import get_executor_state, shutdown_executor
+
+logger = logging.getLogger("django_tasks_local_db")
+
+
+def _execute_task(
+    func_path: str,
+    args: tuple,
+    kwargs: dict,
+    result_id: str,
+    backend_name: str,
+    worker_id: str,
+):
+    """Execute a task function. Runs in the thread pool."""
+    from .models import DBTaskResult
+
+    close_old_connections()
+
+    db_result = DBTaskResult.objects.get(id=result_id)
+    db_result.claim(worker_id)
+
+    task = import_string(func_path)
+    task_result = db_result.task_result
+    backend_type = type(task.get_backend())
+
+    task_started.send(sender=backend_type, task_result=task_result)
+
+    if task.takes_context:
+        raw_return_value = task.call(
+            TaskContext(task_result=task_result), *args, **kwargs
+        )
+    else:
+        raw_return_value = task.call(*args, **kwargs)
+
+    return normalize_json(raw_return_value)
+
+
+class LocalDBBackend(BaseTaskBackend):
+    supports_defer = False
+    supports_async_task = False
+    supports_get_result = True
+    supports_priority = False
+    executor_class = ThreadPoolExecutor
+
+    def __init__(self, alias: str, params: dict[str, Any]) -> None:
+        super().__init__(alias, params)
+        options = params.get("OPTIONS", {})
+        self._max_workers = options.get("MAX_WORKERS", 10)
+        self._name = f"tasks-{alias}"
+        self._worker_id = get_random_string(32)
+        self._state = None
+
+    def _get_state(self):
+        if self._state is None:
+            self._state = get_executor_state(
+                name=self._name,
+                executor_class=self.executor_class,
+                max_workers=self._max_workers,
+            )
+        return self._state
+
+    def enqueue(self, task: Task, args=None, kwargs=None):
+        from .models import DBTaskResult
+
+        self.validate_task(task)
+
+        func_path = task.module_path
+        normalized_args = normalize_json({"args": list(args or ()), "kwargs": dict(kwargs or {})})
+
+        db_result = DBTaskResult.objects.create(
+            args_kwargs=normalized_args,
+            priority=task.priority,
+            task_path=func_path,
+            queue_name=task.queue_name,
+            backend_name=self.alias,
+            exception_class_path="",
+            traceback="",
+        )
+
+        result_id = str(db_result.id)
+
+        task_enqueued.send(type(self), task_result=db_result.task_result)
+
+        state = self._get_state()
+        future = state.executor.submit(
+            _execute_task,
+            func_path,
+            tuple(normalized_args["args"]),
+            normalized_args["kwargs"],
+            result_id,
+            self.alias,
+            self._worker_id,
+        )
+
+        with state.lock:
+            state.futures[result_id] = future
+
+        future.add_done_callback(lambda f: self._on_complete(result_id, f))
+
+        return db_result.task_result
+
+    def _on_complete(self, result_id: str, future: Future) -> None:
+        from .models import DBTaskResult
+
+        try:
+            close_old_connections()
+
+            task_exc = None
+            return_value = None
+            try:
+                return_value = future.result()
+            except Exception as exc:
+                task_exc = exc
+
+            db_result = DBTaskResult.objects.get(id=result_id)
+            if task_exc is not None:
+                db_result.set_failed(task_exc)
+            else:
+                db_result.set_successful(return_value)
+
+            task_finished.send(
+                sender=type(self), task_result=db_result.task_result
+            )
+        finally:
+            state = self._get_state()
+            with state.lock:
+                state.futures.pop(result_id, None)
+
+    def get_result(self, result_id: str):
+        from .models import DBTaskResult
+
+        try:
+            return DBTaskResult.objects.get(id=result_id).task_result
+        except DBTaskResult.DoesNotExist as e:
+            raise TaskResultDoesNotExist(result_id) from e
+
+    def recover_tasks(self) -> int:
+        """Recover orphaned tasks from DB and resubmit to the executor.
+
+        Returns the number of tasks recovered.
+        """
+        from .models import DBTaskResult
+
+        recovered = 0
+        tasks_to_submit = []
+        with transaction.atomic():
+            orphaned = (
+                DBTaskResult.objects.filter(
+                    status__in=[TaskResultStatus.READY, TaskResultStatus.RUNNING]
+                )
+                .filter(backend_name=self.alias)
+                .select_for_update(skip_locked=True)
+            )
+            for db_result in orphaned:
+                result_id = str(db_result.id)
+                db_result.claim(self._worker_id)
+                tasks_to_submit.append((
+                    result_id,
+                    db_result.task_path,
+                    tuple(db_result.args_kwargs["args"]),
+                    db_result.args_kwargs["kwargs"],
+                ))
+                recovered += 1
+                logger.info("Recovered orphaned task %s (%s)", result_id, db_result.task_path)
+
+        for result_id, task_path, args, kwargs in tasks_to_submit:
+            state = self._get_state()
+            future = state.executor.submit(
+                _execute_task,
+                task_path,
+                args,
+                kwargs,
+                result_id,
+                self.alias,
+                self._worker_id,
+            )
+            with state.lock:
+                state.futures[result_id] = future
+            future.add_done_callback(
+                lambda f, rid=result_id: self._on_complete(rid, f)
+            )
+
+        return recovered
+
+    def close(self):
+        shutdown_executor(self._name)

django_tasks_local_db-0.1.0/django_tasks_local_db/migrations/0001_initial.py

@@ -0,0 +1,104 @@
+import datetime
+import uuid
+
+import django.db.models.functions.comparison
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = []
+
+    operations = [
+        migrations.CreateModel(
+            name="DBTaskResult",
+            fields=[
+                (
+                    "id",
+                    models.UUIDField(
+                        default=uuid.uuid4,
+                        editable=False,
+                        primary_key=True,
+                        serialize=False,
+                    ),
+                ),
+                (
+                    "status",
+                    models.CharField(
+                        choices=[
+                            ("READY", "Ready"),
+                            ("RUNNING", "Running"),
+                            ("FAILED", "Failed"),
+                            ("SUCCESSFUL", "Successful"),
+                        ],
+                        default="READY",
+                        max_length=10,
+                    ),
+                ),
+                ("enqueued_at", models.DateTimeField(auto_now_add=True)),
+                ("started_at", models.DateTimeField(null=True)),
+                ("finished_at", models.DateTimeField(null=True)),
+                ("args_kwargs", models.JSONField()),
+                ("priority", models.IntegerField(default=0)),
+                ("task_path", models.TextField()),
+                ("worker_ids", models.JSONField(default=list)),
+                (
+                    "queue_name",
+                    models.CharField(default="default", max_length=32),
+                ),
+                ("backend_name", models.CharField(max_length=32)),
+                (
+                    "run_after",
+                    models.DateTimeField(
+                        default=datetime.datetime(
+                            9999, 1, 1, 0, 0, tzinfo=datetime.timezone.utc
+                        )
+                    ),
+                ),
+                ("return_value", models.JSONField(default=None, null=True)),
+                ("exception_class_path", models.TextField(default="")),
+                ("traceback", models.TextField(default="")),
+            ],
+            options={
+                "ordering": [
+                    django.db.models.expressions.OrderBy(
+                        django.db.models.expressions.F("priority"), descending=True
+                    ),
+                    django.db.models.expressions.OrderBy(
+                        django.db.models.expressions.F("run_after")
+                    ),
+                ],
+            },
+        ),
+        migrations.AddIndex(
+            model_name="dbtaskresult",
+            index=models.Index(
+                "status",
+                django.db.models.expressions.OrderBy(
+                    django.db.models.expressions.F("priority"), descending=True
+                ),
+                django.db.models.expressions.OrderBy(
+                    django.db.models.expressions.F("run_after")
+                ),
+                condition=models.Q(("status", "READY")),
+                name="local_db_ready_idx",
+            ),
+        ),
+        migrations.AddIndex(
+            model_name="dbtaskresult",
+            index=models.Index(
+                fields=["backend_name"], name="django_task_backend_b87c8f_idx"
+            ),
+        ),
+        migrations.AddConstraint(
+            model_name="dbtaskresult",
+            constraint=models.CheckConstraint(
+                condition=models.Q(
+                    ("priority__gte", -100), ("priority__lte", 100)
+                ),
+                name="local_db_priority_range",
+            ),
+        ),
+    ]

django_tasks_local_db-0.1.0/django_tasks_local_db/migrations/0002_alter_dbtaskresult_options_and_more.py

@@ -0,0 +1,102 @@
+# Generated by Django 6.0.2 on 2026-03-03 17:28
+
+import uuid
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('django_tasks_local_db', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='dbtaskresult',
+            options={'ordering': [models.OrderBy(models.F('priority'), descending=True), models.OrderBy(models.F('run_after'))], 'verbose_name': 'task result', 'verbose_name_plural': 'task results'},
+        ),
+        migrations.RenameIndex(
+            model_name='dbtaskresult',
+            new_name='django_task_backend_0535b8_idx',
+            old_name='django_task_backend_b87c8f_idx',
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='args_kwargs',
+            field=models.JSONField(verbose_name='args kwargs'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='backend_name',
+            field=models.CharField(max_length=32, verbose_name='backend name'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='enqueued_at',
+            field=models.DateTimeField(auto_now_add=True, verbose_name='enqueued at'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='exception_class_path',
+            field=models.TextField(default='', verbose_name='exception class path'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='finished_at',
+            field=models.DateTimeField(null=True, verbose_name='finished at'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='id',
+            field=models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False, verbose_name='id'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='priority',
+            field=models.IntegerField(default=0, verbose_name='priority'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='queue_name',
+            field=models.CharField(default='default', max_length=32, verbose_name='queue name'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='return_value',
+            field=models.JSONField(default=None, null=True, verbose_name='return value'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='run_after',
+            field=models.DateTimeField(null=True, verbose_name='run after'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='started_at',
+            field=models.DateTimeField(null=True, verbose_name='started at'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='status',
+            field=models.CharField(choices=[('READY', 'Ready'), ('RUNNING', 'Running'), ('FAILED', 'Failed'), ('SUCCESSFUL', 'Successful')], default='READY', max_length=10, verbose_name='status'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='task_path',
+            field=models.TextField(verbose_name='task path'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='traceback',
+            field=models.TextField(default='', verbose_name='traceback'),
+        ),
+        migrations.AlterField(
+            model_name='dbtaskresult',
+            name='worker_ids',
+            field=models.JSONField(default=list, verbose_name='worker ids'),
+        ),
+        migrations.AddIndex(
+            model_name='dbtaskresult',
+            index=models.Index(fields=['queue_name'], name='django_task_queue_n_7c9efa_idx'),
+        ),
+    ]