django-lambda-tasks 0.6.7__tar.gz → 0.8.0__tar.gz

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/.github/workflows/ci.yml

@@ -30,12 +30,26 @@ jobs:
     strategy:
       matrix:
         python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
+    services:
+      postgres:
+        image: postgres:17
+        env:
+          POSTGRES_DB: lambda_tasks_test
+          POSTGRES_USER: postgres
+          POSTGRES_PASSWORD: postgres
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U postgres"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
     steps:
       - uses: actions/checkout@v4
       - name: Install uv and set the python version
         uses: astral-sh/setup-uv@v6
         with:
           python-version: ${{ matrix.python-version }}
-      - name: Run uv
+      - name: Run tests
         run: |
           uv run pytest --cov-branch --cov-report term-missing --cov=tests/ --cov=lambda_tasks/

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/.kiro/steering/product.md

@@ -143,6 +143,7 @@ class SQSLambdaTaskMessage(BaseModel):
 
 `handler(event, context)` in `handler.py`:
 - Processes each SQS record independently
+- Calls `django.db.close_old_connections()` before each record to close database connections that have become unusable between warm invocations (idle timeouts, RDS failovers, network interruptions)
 - Passes `record["messageId"]` as `message_id` to `execute_immediately()` — this becomes the `TaskRecord` primary key, enabling deduplication on redelivery
 - Returns `{"batchItemFailures": [...]}` for partial-batch failure reporting
 - Only pre-execution failures (malformed message, import error, misconfiguration) are reported as `batchItemFailures` — task logic failures are caught and recorded as `FAILED` TaskRecords without raising

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/.kiro/steering/structure.md

@@ -43,7 +43,7 @@ django-lambda-tasks/
 - `decorators.py` — defines `LambdaTaskWrapper` and `@lambda_task` decorator factory; enforces kwargs-only at decoration time; validates timeouts (> 0, soft < hard) at decoration time; upper-bound timeout validation deferred to execution time based on queue type
 - `models.py` — `TaskRecord` (Django ORM), `SQSLambdaTaskMessage` (Pydantic, SQS schema + execution logic), `SQSLambdaTask` (Pydantic, holds message + routing; `_execute()` inspects queue config to publish to SQS, submit to Batch via `submit_batch_job`, execute eagerly, or submit to the local process pool; `execute_on_commit()` registers `_execute` with `transaction.on_commit`)
 - `local_executor.py` — `ProcessPoolExecutor`-based async local execution; `get_pool()` lazily creates a module-level pool; `submit_task()` fire-and-forget submission; `_execute_in_worker()` deserializes and runs the task in a child process; `_pool_initializer()` calls `django.setup()` per worker (and sets `SIGINT` to `SIG_IGN` so workers ignore Ctrl+C); `_install_shutdown_handlers()` installs main-thread `SIGINT`/`SIGTERM` handlers that release the pool before chaining to the previous handler
-- `handler.py` — Lambda entry point (`handler(event, context)`); cold-start init (memory limit → temporary log handler → `resolve_environment()` → `resolve_secrets_into_env()` → handler removed → conditional `django.setup()`) runs inside the handler on first invocation, guarded by `_cold_start_done` sentinel; processes SQS records independently; returns `batchItemFailures`. Also provides `main()` as the AWS Batch container entry point (`python -m lambda_tasks.handler`) — reads `LAMBDA_TASKS_MESSAGE` env var, constructs a synthetic SQS event, and delegates to `handler()`
+- `handler.py` — Lambda entry point (`handler(event, context)`); cold-start init (memory limit → temporary log handler → `resolve_environment()` → `resolve_secrets_into_env()` → handler removed → conditional `django.setup()`) runs inside the handler on first invocation, guarded by `_cold_start_done` sentinel; calls `db.close_old_connections()` before each SQS record to handle stale connections on warm starts; processes SQS records independently; returns `batchItemFailures`. Also provides `main()` as the AWS Batch container entry point (`python -m lambda_tasks.handler`) — reads `LAMBDA_TASKS_MESSAGE` env var, constructs a synthetic SQS event, and delegates to `handler()`
 - `environment_loader.py` — loads env vars from a Secrets Manager secret at cold start; validates flat JSON format; idempotent via `_loaded` sentinel
 - `secret_loader.py` — resolves `LAMBDA_TASKS_SECRET_*` env vars from Secrets Manager before Django starts; validates format, detects conflicts, batches API calls; idempotent via `_loaded` sentinel
 - `logging.py` — `task_logger` singleton; `message_id` set/cleared around each task execution

django_lambda_tasks-0.6.7/README.md → django_lambda_tasks-0.8.0/PKG-INFO

@@ -1,9 +1,23 @@
+Metadata-Version: 2.5
+Name: django-lambda-tasks
+Version: 0.8.0
+Summary: Run async tasks in a lambda function
+Requires-Python: >=3.10
+Requires-Dist: awslambdaric
+Requires-Dist: boto3
+Requires-Dist: django
+Requires-Dist: pydantic
+Requires-Dist: redis
+Description-Content-Type: text/markdown
+
 # Django Lambda Tasks
 
 A Django library for offloading work to AWS Lambda or AWS Batch outside of the request-response cycle. Tasks are defined with a decorator, enqueued to SQS on transaction commit, and executed by a Lambda handler or Batch container. Task results, status, and metadata are persisted in the Django database.
 
 > **Platform note:** Unix-based systems only (Linux, macOS). Timeout enforcement relies on `SIGALRM`.
 
+> **Database note:** PostgreSQL is required. The `TaskRecord` model uses PostgreSQL-specific features.
+
 ---
 
 ## Installation
@@ -78,6 +92,7 @@ A runnable Django project is included in the [`example/`](example/) directory. I
 
 ```bash
 cd example
+docker compose up -d
 uv run python manage.py migrate
 uv run python manage.py createsuperuser
 uv run python manage.py runserver
@@ -535,6 +550,8 @@ lambda_tasks.handler.handler
 
 The handler performs cold-start initialisation on the first invocation (not at module import time) to avoid Lambda init-duration timeouts. The sequence is: temporary log handler attached → `resolve_environment()` → `resolve_secrets_into_env()` → log handler removed → conditional `django.setup()`. A module-level sentinel ensures this runs only once; subsequent warm invocations skip it entirely.
 
+Before processing each SQS record, the handler calls `django.db.close_old_connections()` to close any database connections that have become unusable between invocations. Lambda containers are reused across invocations (warm starts), but database connections may be closed server-side due to idle timeouts, RDS failovers, or network interruptions. Without this, the first database query in a task would fail with `OperationalError: the connection is closed`.
+
 Ensure the Lambda execution environment has `DJANGO_SETTINGS_MODULE` set and that all task modules are importable (i.e. your application code is on the Python path).
 
 | Environment Variable | Required | Description |

django_lambda_tasks-0.6.7/PKG-INFO → django_lambda_tasks-0.8.0/README.md

@@ -1,21 +1,11 @@
-Metadata-Version: 2.4
-Name: django-lambda-tasks
-Version: 0.6.7
-Summary: Run async tasks in a lambda function
-Requires-Python: >=3.10
-Requires-Dist: awslambdaric
-Requires-Dist: boto3
-Requires-Dist: django
-Requires-Dist: pydantic
-Requires-Dist: redis
-Description-Content-Type: text/markdown
-
 # Django Lambda Tasks
 
 A Django library for offloading work to AWS Lambda or AWS Batch outside of the request-response cycle. Tasks are defined with a decorator, enqueued to SQS on transaction commit, and executed by a Lambda handler or Batch container. Task results, status, and metadata are persisted in the Django database.
 
 > **Platform note:** Unix-based systems only (Linux, macOS). Timeout enforcement relies on `SIGALRM`.
 
+> **Database note:** PostgreSQL is required. The `TaskRecord` model uses PostgreSQL-specific features.
+
 ---
 
 ## Installation
@@ -90,6 +80,7 @@ A runnable Django project is included in the [`example/`](example/) directory. I
 
 ```bash
 cd example
+docker compose up -d
 uv run python manage.py migrate
 uv run python manage.py createsuperuser
 uv run python manage.py runserver
@@ -547,6 +538,8 @@ lambda_tasks.handler.handler
 
 The handler performs cold-start initialisation on the first invocation (not at module import time) to avoid Lambda init-duration timeouts. The sequence is: temporary log handler attached → `resolve_environment()` → `resolve_secrets_into_env()` → log handler removed → conditional `django.setup()`. A module-level sentinel ensures this runs only once; subsequent warm invocations skip it entirely.
 
+Before processing each SQS record, the handler calls `django.db.close_old_connections()` to close any database connections that have become unusable between invocations. Lambda containers are reused across invocations (warm starts), but database connections may be closed server-side due to idle timeouts, RDS failovers, or network interruptions. Without this, the first database query in a task would fail with `OperationalError: the connection is closed`.
+
 Ensure the Lambda execution environment has `DJANGO_SETTINGS_MODULE` set and that all task modules are importable (i.e. your application code is on the Python path).
 
 | Environment Variable | Required | Description |

django_lambda_tasks-0.8.0/docker-compose.yml

@@ -0,0 +1,11 @@
+services:
+  postgres:
+    image: postgres:17
+    environment:
+      POSTGRES_DB: lambda_tasks_test
+      POSTGRES_USER: postgres
+      POSTGRES_PASSWORD: postgres
+    ports:
+      - "5432:5432"
+    tmpfs:
+      - /var/lib/postgresql/data

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/example/README.md

@@ -1,27 +1,34 @@
 # Example App
 
-A minimal Django app demonstrating `django-lambda-tasks` with `LAMBDA_TASKS_EAGER = True`. In eager mode, tasks execute synchronously in-process — no AWS infrastructure required.
+A minimal Django app demonstrating `django-lambda-tasks` with `LAMBDA_TASKS_LOCAL_WORKERS = 2`. Tasks execute in a background process pool — no AWS infrastructure required.
 
 ## Prerequisites
 
 - `uv` installed
+- Docker installed (for PostgreSQL)
 - All commands run from the `example/` directory
 
 ## Setup
 
-**1. Apply migrations**
+**1. Start PostgreSQL**
+
+```bash
+docker compose up -d
+```
+
+**2. Apply migrations**
 
 ```bash
 uv run python manage.py migrate
 ```
 
-**2. Create a superuser** (to access the admin interface)
+**3. Create a superuser** (to access the admin interface)
 
 ```bash
 uv run python manage.py createsuperuser
 ```
 
-**3. Start the development server**
+**4. Start the development server**
 
 ```bash
 uv run python manage.py runserver
@@ -33,7 +40,7 @@ uv run python manage.py runserver
 
 Visit: `http://127.0.0.1:8000/trigger/?name=Alice`
 
-The task runs synchronously and the response confirms it was triggered.
+The task runs in a background worker process and the response confirms it was triggered.
 
 **View task records in the admin**
 

django_lambda_tasks-0.8.0/example/docker-compose.yml

@@ -0,0 +1,14 @@
+services:
+  postgres:
+    image: postgres:17
+    environment:
+      POSTGRES_DB: example
+      POSTGRES_USER: postgres
+      POSTGRES_PASSWORD: postgres
+    ports:
+      - "5432:5432"
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+
+volumes:
+  postgres_data:

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/example/example_project/settings.py

@@ -17,6 +17,7 @@ INSTALLED_APPS = [
     "django.contrib.sessions",
     "django.contrib.messages",
     "django.contrib.staticfiles",
+    "django.contrib.postgres",
     "lambda_tasks",
     "example_app",
 ]
@@ -53,8 +54,12 @@ WSGI_APPLICATION = "example_project.wsgi.application"
 
 DATABASES = {
     "default": {
-        "ENGINE": "django.db.backends.sqlite3",
-        "NAME": BASE_DIR / "db.sqlite3",
+        "ENGINE": "django.db.backends.postgresql",
+        "NAME": "example",
+        "USER": "postgres",
+        "PASSWORD": "postgres",
+        "HOST": "localhost",
+        "PORT": "5432",
     }
 }
 

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/lambda_tasks/checks.py

@@ -1,7 +1,27 @@
 """Django deployment checks for lambda_tasks."""
 
 from django.conf import settings as django_settings
-from django.core.checks import Warning, register
+from django.core.checks import Error, Warning, register
+
+
+@register("lambda_tasks")
+def check_contrib_postgres_installed(
+    *, app_configs: object, **kwargs: object
+) -> list[Error]:
+    """Error if django.contrib.postgres is not in INSTALLED_APPS."""
+    errors: list[Error] = []
+
+    if "django.contrib.postgres" not in django_settings.INSTALLED_APPS:
+        errors.append(
+            Error(
+                "django.contrib.postgres is not in INSTALLED_APPS.",
+                hint="Add 'django.contrib.postgres' to INSTALLED_APPS. "
+                "It is required for PostgreSQL-specific indexes used by lambda_tasks.",
+                id="lambda_tasks.E001",
+            )
+        )
+
+    return errors
 
 
 @register("lambda_tasks", deploy=True)

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/lambda_tasks/handler.py

@@ -21,6 +21,7 @@ import urllib.request
 import uuid
 
 import django
+from django import db
 from django.apps import apps
 
 from lambda_tasks.environment_loader import resolve_environment
@@ -132,6 +133,7 @@ def handler(event: dict, context: object) -> dict:
 
     for record in event["Records"]:
         try:
+            db.close_old_connections()
             SQSLambdaTaskMessage.model_validate_json(
                 record["body"]
             ).execute_immediately(message_id=record["messageId"])

django_lambda_tasks-0.8.0/lambda_tasks/migrations/0003_taskrecord_taskrecord_kwargs_trgm.py

@@ -0,0 +1,29 @@
+# Generated by Django 6.0.3 on 2026-08-17 12:45
+
+import django.contrib.postgres.indexes
+import django.db.models.functions.comparison
+from django.contrib.postgres.operations import TrigramExtension
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("lambda_tasks", "0002_alter_taskrecord_status"),
+    ]
+
+    operations = [
+        TrigramExtension(),
+        migrations.AddIndex(
+            model_name="taskrecord",
+            index=django.contrib.postgres.indexes.GinIndex(
+                django.contrib.postgres.indexes.OpClass(
+                    django.db.models.functions.comparison.Cast(
+                        "kwargs", output_field=models.TextField()
+                    ),
+                    name="gin_trgm_ops",
+                ),
+                name="taskrecord_kwargs_trgm",
+            ),
+        ),
+    ]

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/lambda_tasks/models.py

@@ -8,10 +8,12 @@ import traceback
 import uuid
 
 import boto3
+from django.contrib.postgres.indexes import GinIndex, OpClass
 from django.core.cache import caches
 from django.core.exceptions import ImproperlyConfigured
 from django.db import models, transaction
 from django.db.models import Q
+from django.db.models.functions import Cast
 from django.utils.module_loading import import_string
 from django.utils.timezone import now
 from pydantic import BaseModel, ConfigDict, Field
@@ -74,6 +76,12 @@ class TaskRecord(models.Model):
             models.Index(fields=["task_name"]),
             models.Index(fields=["status"]),
             models.Index(fields=["-start_time"]),
+            GinIndex(
+                OpClass(
+                    Cast("kwargs", output_field=models.TextField()), name="gin_trgm_ops"
+                ),
+                name="taskrecord_kwargs_trgm",
+            ),
         ]
         constraints = [
             models.CheckConstraint(

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/pyproject.toml

@@ -7,7 +7,7 @@ packages = ["lambda_tasks"]
 
 [project]
 name = "django-lambda-tasks"
-version = "0.6.7"
+version = "0.8.0"
 description = "Run async tasks in a lambda function"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -30,6 +30,7 @@ dev = [
     "pytest-xdist",
     "pytest-cov",
     "ty",
+    "psycopg",
 ]
 
 [tool.ty.rules]

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/tests/settings.py

@@ -1,7 +1,11 @@
 DATABASES = {
     "default": {
-        "ENGINE": "django.db.backends.sqlite3",
-        "NAME": ":memory:",
+        "ENGINE": "django.db.backends.postgresql",
+        "NAME": "lambda_tasks_test",
+        "USER": "postgres",
+        "PASSWORD": "postgres",
+        "HOST": "localhost",
+        "PORT": "5432",
     }
 }
 
@@ -9,6 +13,7 @@ INSTALLED_APPS = [
     "django.contrib.admin",
     "django.contrib.contenttypes",
     "django.contrib.auth",
+    "django.contrib.postgres",
     "lambda_tasks",
 ]
 

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/tests/test_batch_worker.py

@@ -1,6 +1,7 @@
 """Tests for lambda_tasks.batch_worker entry point."""
 
 import uuid
+from unittest.mock import patch
 
 import pytest
 
@@ -8,6 +9,14 @@ from lambda_tasks.decorators import lambda_task
 from lambda_tasks.handler import main
 from lambda_tasks.models import SQSLambdaTaskMessage, TaskRecord
 
+
+@pytest.fixture(autouse=True)
+def _patch_close_old_connections():
+    """Prevent close_old_connections from closing the test DB connection."""
+    with patch("lambda_tasks.handler.db.close_old_connections"):
+        yield
+
+
 QUEUES = {
     "default": {"queue_url": "https://sqs.us-east-1.amazonaws.com/000000000000/default"}
 }

django_lambda_tasks-0.8.0/tests/test_checks.py

@@ -0,0 +1,21 @@
+"""Tests for the django.contrib.postgres system check."""
+
+from lambda_tasks.checks import check_contrib_postgres_installed
+
+
+def test_check_passes_when_contrib_postgres_installed(settings):
+    settings.INSTALLED_APPS = [
+        "django.contrib.postgres",
+        "lambda_tasks",
+    ]
+    errors = check_contrib_postgres_installed(app_configs=None)
+    assert errors == []
+
+
+def test_check_errors_when_contrib_postgres_missing(settings):
+    settings.INSTALLED_APPS = [
+        "lambda_tasks",
+    ]
+    errors = check_contrib_postgres_installed(app_configs=None)
+    assert len(errors) == 1
+    assert errors[0].id == "lambda_tasks.E001"

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/tests/test_handler.py

@@ -26,6 +26,14 @@ from hypothesis import strategies as st
 
 from lambda_tasks.handler import handler
 
+
+@pytest.fixture(autouse=True)
+def _patch_close_old_connections():
+    """Prevent close_old_connections from triggering the DB access guard in unit tests."""
+    with patch("lambda_tasks.handler.db.close_old_connections"):
+        yield
+
+
 # ---------------------------------------------------------------------------
 # Signature test
 # ---------------------------------------------------------------------------
@@ -457,3 +465,71 @@ class TestColdStartLogging:
 
         assert lambda_tasks_logger.handlers == []
         assert lambda_tasks_logger.level == logging.NOTSET
+
+
+# ---------------------------------------------------------------------------
+# Stale connection handling
+# ---------------------------------------------------------------------------
+
+
+class TestHandlerClosesOldConnections:
+    def test_close_old_connections_called_before_each_record(self):
+        """close_old_connections() is called before processing each SQS record."""
+        call_order = []
+
+        def _track_close():
+            call_order.append("close")
+
+        def _make_message(body: str):
+            msg = MagicMock()
+
+            def _execute(**kwargs):
+                call_order.append("execute")
+
+            msg.execute_immediately.side_effect = _execute
+            return msg
+
+        records = [
+            _make_record("msg-1", _valid_body()),
+            _make_record("msg-2", _valid_body()),
+        ]
+
+        with (
+            patch(
+                "lambda_tasks.handler.db.close_old_connections",
+                side_effect=_track_close,
+            ),
+            _patch_model_validate(_make_message),
+        ):
+            handler(event={"Records": records}, context=None)
+
+        assert call_order == ["close", "execute", "close", "execute"]
+
+    def test_close_old_connections_called_even_when_record_fails(self):
+        """close_old_connections() is called before each record, including ones that fail."""
+        close_count = 0
+
+        def _track_close():
+            nonlocal close_count
+            close_count += 1
+
+        def _make_message(body: str):
+            msg = MagicMock()
+            msg.execute_immediately.side_effect = RuntimeError("boom")
+            return msg
+
+        records = [
+            _make_record("msg-1", _valid_body()),
+            _make_record("msg-2", _valid_body()),
+        ]
+
+        with (
+            patch(
+                "lambda_tasks.handler.db.close_old_connections",
+                side_effect=_track_close,
+            ),
+            _patch_model_validate(_make_message),
+        ):
+            handler(event={"Records": records}, context=None)
+
+        assert close_count == 2

{django_lambda_tasks-0.6.7 → django_lambda_tasks-0.8.0}/tests/test_models.py

@@ -1096,7 +1096,14 @@ def test_property_4_retry_increments_n_retries(n_retries, exc_type_name):
 @pytest.mark.django_db(transaction=True)
 @given(
     x=st.integers(),
-    label=st.text(min_size=1, max_size=20),
+    label=st.text(
+        min_size=1,
+        max_size=20,
+        alphabet=st.characters(
+            blacklist_characters="\x00",
+            blacklist_categories=("Cs",),
+        ),
+    ),
 )
 @h_settings(max_examples=50, suppress_health_check=[HealthCheck.too_slow])
 def test_property_5_matching_exc_enqueues_retry_same_kwargs(x, label):