django-lambda-tasks 0.9.1__tar.gz → 0.9.2__tar.gz

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/.kiro/steering/product.md

@@ -310,6 +310,8 @@ Behaviour:
 - Builds a `TracerProvider` with an OTLP/HTTP span exporter via a `BatchSpanProcessor`. Spans go **directly** to the AWS X-Ray OTLP traces endpoint (`https://xray.<region>.amazonaws.com/v1/traces`) — no collector, no daemon, no Lambda layer.
 - The exporter's `requests` session SigV4-signs each export with the ambient AWS credentials (task / execution role) via `botocore` — no API key to manage. Unsigned when no credentials are resolvable.
 - Uses the AWS X-Ray id generator (`AwsXRayIdGenerator`) so the X-Ray backend accepts the trace IDs, and the X-Ray propagator (`AwsXRayLambdaPropagator` on Lambda, else `AwsXRayPropagator`). The X-Ray packages (`opentelemetry-sdk-extension-aws`, `opentelemetry-propagator-aws-xray`) are part of the `otel` extra.
+- Library instrumentation: after the provider and propagator are installed, `configure_tracing()` calls `_instrument_libraries()`, which attaches the instrumentors for the libraries the tasks use — **psycopg 3** (SQL queries), **botocore** (every boto3 / aioboto3 call), **requests** and **httpx** (outbound HTTP), and **redis** (cache ops via `django_redis` → `redis-py`). Each instrumentor is wrapped independently and is best-effort: a missing or failing one is logged and skipped, never raised, so one broken package cannot stop the others or take down the worker. These are process-global and idempotent, and emit spans into the global provider (so it must exist first). Django request instrumentation (`DjangoInstrumentor`) is **not** installed here — a task has no Django request; the API (ECS) consumer installs that itself. sqlcommenter (`enable_commenter=True` on the psycopg instrumentor) is left off because it modifies the SQL text sent to PostgreSQL.
+- Exporter self-trace suppression: the OTLP exporter POSTs spans over `requests` to the X-Ray endpoint, so the `requests` instrumentor is configured with `excluded_urls="xray\\..*\\.amazonaws\\.com"` to avoid tracing its own export calls (self-referential noise). The instrumentor also honours the `OTEL_PYTHON_REQUESTS_EXCLUDED_URLS` env var for per-environment overrides. botocore is not used by the exporter, so it needs no exclusion.
 - The OpenTelemetry / `requests` imports are **deferred into the functions** (module scope imports only `os`), so the OTEL packages remain a genuinely optional extra: importing `lambda_tasks.handler` with them uninstalled is a no-op, not an `ImportError`. The disabled path (endpoint unset) never reaches any OTEL import.
 - Lambda path: also calls `AwsLambdaInstrumentor().instrument()` (only when `AWS_LAMBDA_FUNCTION_NAME` is set). Batch path (`main()`): the instrumentor is skipped (no handler to wrap); the provider alone is configured.
 - Flush on shutdown: `main()` calls `TracerProvider.force_flush()` / `shutdown()` before the one-shot process exits so short-lived Batch jobs do not drop buffered spans. The Lambda path needs no manual flush — `AwsLambdaInstrumentor` force-flushes per invocation.

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/.kiro/steering/structure.md

@@ -45,7 +45,7 @@ django-lambda-tasks/
 - `models.py` — `TaskRecord` (Django ORM), `SQSLambdaTaskMessage` (Pydantic, SQS schema + execution logic), `SQSLambdaTask` (Pydantic, holds message + routing; `_execute()` inspects queue config to publish to SQS, submit to Batch via `submit_batch_job`, execute eagerly, or submit to the local process pool; `execute_on_commit()` registers `_execute` with `transaction.on_commit`)
 - `local_executor.py` — `ProcessPoolExecutor`-based async local execution; `get_pool()` lazily creates a module-level pool; `submit_task()` fire-and-forget submission; `_execute_in_worker()` deserializes and runs the task in a child process; `_pool_initializer()` calls `django.setup()` per worker (and sets `SIGINT` to `SIG_IGN` so workers ignore Ctrl+C); `_install_shutdown_handlers()` installs main-thread `SIGINT`/`SIGTERM` handlers that release the pool before chaining to the previous handler
 - `handler.py` — Lambda entry point (`handler(event, context)`); cold-start init (memory limit → temporary log handler → `resolve_environment()` → `resolve_secrets_into_env()` → handler removed → conditional `django.setup()`) runs inside the handler on first invocation, guarded by `_cold_start_done` sentinel; calls `db.close_old_connections()` before each SQS record to handle stale connections on warm starts; processes SQS records independently; returns `batchItemFailures`. At **import time** (before any invocation) it calls `configure_tracing()` and `instrument_lambda_handler()` from `tracing.py` so the Lambda instrumentor can wrap the handler before the runtime invokes it. Also provides `main()` as the AWS Batch container entry point (`python -m lambda_tasks.handler`) — reads `LAMBDA_TASKS_MESSAGE` env var, constructs a synthetic SQS event, delegates to `handler()`, then force-flushes and shuts down the global `TracerProvider` (if any) before the one-shot process exits
-- `tracing.py` — optional OpenTelemetry tracing, driven purely by standard OTEL env vars (no Django settings) so the package stays generic; no-op unless `OTEL_EXPORTER_OTLP_ENDPOINT` (or `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`) is set. All OpenTelemetry / `requests` imports are **deferred into the functions** (only `os` is imported at module scope) so the OTEL packages are a genuinely optional extra — importing `lambda_tasks.handler` without them is a no-op, not an `ImportError`. `configure_tracing()` installs a global `TracerProvider` using the X-Ray id generator (`AwsXRayIdGenerator`) with an OTLP/HTTP exporter whose `requests` session SigV4-signs each export to the AWS X-Ray traces endpoint (unsigned when no credentials resolve), and sets the X-Ray propagator (`AwsXRayLambdaPropagator` on Lambda, else `AwsXRayPropagator`). `instrument_lambda_handler()` calls `AwsLambdaInstrumentor().instrument()` only when tracing is enabled and `AWS_LAMBDA_FUNCTION_NAME` is set (Lambda path); skipped on the Batch path. `flush_tracing()` force-flushes and shuts down the provider (gated; used by the Batch entry point)
+- `tracing.py` — optional OpenTelemetry tracing, driven purely by standard OTEL env vars (no Django settings) so the package stays generic; no-op unless `OTEL_EXPORTER_OTLP_ENDPOINT` (or `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`) is set. All OpenTelemetry / `requests` imports are **deferred into the functions** (only `os` is imported at module scope) so the OTEL packages are a genuinely optional extra — importing `lambda_tasks.handler` without them is a no-op, not an `ImportError`. `configure_tracing()` installs a global `TracerProvider` using the X-Ray id generator (`AwsXRayIdGenerator`) with an OTLP/HTTP exporter whose `requests` session SigV4-signs each export to the AWS X-Ray traces endpoint (unsigned when no credentials resolve), and sets the X-Ray propagator (`AwsXRayLambdaPropagator` on Lambda, else `AwsXRayPropagator`), then calls `_instrument_libraries()` to attach the library instrumentors (psycopg 3, botocore, requests, httpx, redis) — each wrapped independently and best-effort (a missing or failing instrumentor is logged, not raised), and the `requests` instrumentor is given an `excluded_urls` regex for the X-Ray OTLP endpoint so the exporter's own export POSTs are not self-traced. `instrument_lambda_handler()` calls `AwsLambdaInstrumentor().instrument()` only when tracing is enabled and `AWS_LAMBDA_FUNCTION_NAME` is set (Lambda path); skipped on the Batch path. `flush_tracing()` force-flushes and shuts down the provider (gated; used by the Batch entry point)
 - `environment_loader.py` — loads env vars from a Secrets Manager secret at cold start; validates flat JSON format; idempotent via `_loaded` sentinel
 - `secret_loader.py` — resolves `LAMBDA_TASKS_SECRET_*` env vars from Secrets Manager before Django starts; validates format, detects conflicts, batches API calls; idempotent via `_loaded` sentinel
 - `logging.py` — `task_logger` singleton; `message_id` set/cleared around each task execution

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/.kiro/steering/tech.md

@@ -28,6 +28,11 @@ Installed via the `otel` extra (`pip install django-lambda-tasks[otel]`). Enable
 | `opentelemetry-instrumentation-aws-lambda` | Wraps the Lambda handler (`AwsLambdaInstrumentor`) |
 | `opentelemetry-sdk-extension-aws` | X-Ray id generator (`AwsXRayIdGenerator`) |
 | `opentelemetry-propagator-aws-xray` | X-Ray trace propagator |
+| `opentelemetry-instrumentation-psycopg` | SQL query spans (psycopg 3) |
+| `opentelemetry-instrumentation-botocore` | AWS SDK spans — every boto3 / aioboto3 call |
+| `opentelemetry-instrumentation-requests` | Outbound HTTP spans via `requests` |
+| `opentelemetry-instrumentation-httpx` | Outbound HTTP spans via `httpx` |
+| `opentelemetry-instrumentation-redis` | Cache operation spans (`django_redis` → `redis-py`) |
 | `requests` | OTLP/HTTP transport session (SigV4-signed via `botocore`) |
 
 ## Dev Dependencies

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: django-lambda-tasks
-Version: 0.9.1
+Version: 0.9.2
 Summary: Run async tasks in a lambda function
 Requires-Python: >=3.11
 Requires-Dist: awslambdaric

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/lambda_tasks/handler.py

@@ -34,13 +34,6 @@ from lambda_tasks.tracing import (
 
 logger = logging.getLogger(__name__)
 
-# Initialise tracing at import time so the Lambda instrumentor can wrap the
-# handler before the first invocation (which is earlier than the cold-start
-# django.setup() inside handler()). No-op unless OTEL_EXPORTER_OTLP_ENDPOINT
-# is set.
-configure_tracing()
-instrument_lambda_handler()
-
 _cold_start_done: bool = False
 
 
@@ -191,5 +184,15 @@ def main() -> int:
         return 0
 
 
+# Initialise tracing at import time so the Lambda instrumentor can wrap the
+# handler before the runtime invokes it (which is earlier than the cold-start
+# django.setup() inside handler()). This MUST run after `handler` and `main`
+# are defined: wrapt resolves the handler by attribute lookup on this module,
+# and wrapping a name that does not exist yet raises AttributeError. No-op
+# unless OTEL_EXPORTER_OTLP_ENDPOINT is set.
+configure_tracing()
+instrument_lambda_handler()
+
+
 if __name__ == "__main__":
     sys.exit(main())

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/lambda_tasks/tracing.py

@@ -33,6 +33,7 @@ if TYPE_CHECKING:
 logger = logging.getLogger(__name__)
 
 _XRAY_SIGV4_SERVICE_NAME = "xray"
+_LAMBDA_HANDLER = "lambda_tasks.handler.handler"
 
 
 def tracing_is_enabled() -> bool:
@@ -262,8 +263,8 @@ def configure_tracing() -> "TracerProvider | None":
 def instrument_lambda_handler() -> None:
     """Wrap the Lambda handler for tracing. No-op unless enabled.
 
-    Must be called at import time of the handler module, before the runtime
-    invokes the handler.
+    Must be called at import time of the handler module, AFTER `handler` is
+    defined (see the handler module), before the runtime invokes the handler.
     """
     if not tracing_is_enabled():
         return
@@ -275,6 +276,12 @@ def instrument_lambda_handler() -> None:
         )
         return
 
+    # AwsLambdaInstrumentor derives the handler to wrap from
+    # ORIG_HANDLER / _HANDLER. Container images launched via awslambdaric pass
+    # the handler as a CMD arg, which is not exported as _HANDLER, so default
+    # it here. setdefault() leaves any explicitly configured value untouched.
+    os.environ.setdefault("ORIG_HANDLER", _LAMBDA_HANDLER)
+
     try:
         from opentelemetry.instrumentation.aws_lambda import (
             AwsLambdaInstrumentor,

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/pyproject.toml

@@ -7,7 +7,7 @@ packages = ["lambda_tasks"]
 
 [project]
 name = "django-lambda-tasks"
-version = "0.9.1"
+version = "0.9.2"
 description = "Run async tasks in a lambda function"
 readme = "README.md"
 requires-python = ">=3.11"

{django_lambda_tasks-0.9.1 → django_lambda_tasks-0.9.2}/tests/test_tracing.py

@@ -263,6 +263,43 @@ class TestInstrumentLambdaHandler:
             if instrumentor.is_instrumented_by_opentelemetry:
                 instrumentor.uninstrument()
 
+    def test_defaults_orig_handler_when_unset(self, monkeypatch):
+        """With ORIG_HANDLER unset the package defaults it to its own handler
+        so container-image deployments need not set the env var."""
+        _clear_otel_env(monkeypatch)
+        monkeypatch.setenv("OTEL_EXPORTER_OTLP_ENDPOINT", _ENDPOINT)
+        monkeypatch.setenv("AWS_LAMBDA_FUNCTION_NAME", "my-fn")
+        monkeypatch.delenv("ORIG_HANDLER", raising=False)
+        monkeypatch.delenv("_HANDLER", raising=False)
+
+        from opentelemetry.instrumentation.aws_lambda import AwsLambdaInstrumentor
+
+        instrumentor = AwsLambdaInstrumentor()
+        try:
+            tracing.instrument_lambda_handler()
+            assert os.environ["ORIG_HANDLER"] == "lambda_tasks.handler.handler"
+        finally:
+            if instrumentor.is_instrumented_by_opentelemetry:
+                instrumentor.uninstrument()
+
+    def test_leaves_explicit_orig_handler_unchanged(self, monkeypatch):
+        """An explicitly configured ORIG_HANDLER wins — setdefault must not
+        overwrite a consumer that invokes the handler at a different path."""
+        _clear_otel_env(monkeypatch)
+        monkeypatch.setenv("OTEL_EXPORTER_OTLP_ENDPOINT", _ENDPOINT)
+        monkeypatch.setenv("AWS_LAMBDA_FUNCTION_NAME", "my-fn")
+        monkeypatch.setenv("ORIG_HANDLER", "custom.module.my_handler")
+
+        from opentelemetry.instrumentation.aws_lambda import AwsLambdaInstrumentor
+
+        instrumentor = AwsLambdaInstrumentor()
+        try:
+            tracing.instrument_lambda_handler()
+            assert os.environ["ORIG_HANDLER"] == "custom.module.my_handler"
+        finally:
+            if instrumentor.is_instrumented_by_opentelemetry:
+                instrumentor.uninstrument()
+
 
 class TestOptionalDependency:
     """The OTEL packages are a genuinely optional extra.
@@ -383,3 +420,120 @@ class TestLoggingDiagnostics:
         assert any(
             "No AWS region resolvable" in record.message for record in caplog.records
         )
+
+
+class TestHandlerImportOrderingRegression:
+    """Regression test for the import-time ordering bug.
+
+    `instrument_lambda_handler()` must run AFTER `def handler` executes, because
+    AwsLambdaInstrumentor wraps the handler via wrapt attribute lookup on the
+    module. When the setup calls sat at the top of the module (before the def),
+    the lookup raised AttributeError, the best-effort wrapper swallowed it and
+    logged "Failed to instrument the Lambda handler...", and the handler was
+    never wrapped — so no SERVER span was produced.
+
+    Run in a subprocess so the fresh, fully-instrumented import cannot perturb
+    the module singletons the rest of the suite relies on.
+    """
+
+    def test_fresh_import_wraps_handler_without_warning(self):
+        program = (
+            "import logging\n"
+            "records = []\n"
+            "class _Capture(logging.Handler):\n"
+            "    def emit(self, record):\n"
+            "        records.append(record.getMessage())\n"
+            "logging.getLogger('lambda_tasks.tracing').addHandler(_Capture())\n"
+            "logging.getLogger('lambda_tasks.tracing').setLevel(logging.DEBUG)\n"
+            "import lambda_tasks.handler as h\n"
+            "from opentelemetry.instrumentation.aws_lambda import "
+            "AwsLambdaInstrumentor\n"
+            "assert AwsLambdaInstrumentor().is_instrumented_by_opentelemetry, "
+            "'handler was not wrapped'\n"
+            "assert not any('Failed to instrument the Lambda handler' in m "
+            "for m in records), records\n"
+            "print('ok')\n"
+        )
+        result = subprocess.run(
+            [sys.executable, "-c", program],
+            capture_output=True,
+            text=True,
+            env={
+                **os.environ,
+                "OTEL_EXPORTER_OTLP_ENDPOINT": _ENDPOINT,
+                "AWS_LAMBDA_FUNCTION_NAME": "my-fn",
+                "AWS_REGION": "us-east-1",
+                "ORIG_HANDLER": "lambda_tasks.handler.handler",
+                "DJANGO_SETTINGS_MODULE": "",
+            },
+        )
+        assert result.returncode == 0, result.stderr
+        assert "ok" in result.stdout
+
+
+class TestHandlerSpanParenting:
+    """An invocation of the wrapped handler produces a SERVER span whose trace
+    contains the task's inner spans as descendants (single connected trace)."""
+
+    def test_inner_span_descends_from_handler_server_span(self, monkeypatch):
+        _clear_otel_env(monkeypatch)
+        monkeypatch.setenv("OTEL_EXPORTER_OTLP_ENDPOINT", _ENDPOINT)
+        monkeypatch.setenv("AWS_LAMBDA_FUNCTION_NAME", "my-fn")
+        monkeypatch.setenv("AWS_REGION", "us-east-1")
+        monkeypatch.setenv("ORIG_HANDLER", "lambda_tasks.handler.handler")
+
+        from opentelemetry import trace as trace_api
+        from opentelemetry.instrumentation.aws_lambda import AwsLambdaInstrumentor
+        from opentelemetry.sdk.trace import TracerProvider
+        from opentelemetry.sdk.trace.export import SimpleSpanProcessor
+        from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
+            InMemorySpanExporter,
+        )
+
+        exporter = InMemorySpanExporter()
+        provider = TracerProvider()
+        provider.add_span_processor(SimpleSpanProcessor(exporter))
+        trace_api.set_tracer_provider(provider)
+
+        instrumentor = AwsLambdaInstrumentor()
+
+        def _inner_handler(event, context):
+            tracer = trace_api.get_tracer("test")
+            with tracer.start_as_current_span("inner-db-span"):
+                pass
+            return {"batchItemFailures": []}
+
+        monkeypatch.setattr(
+            "lambda_tasks.handler.handler", _inner_handler, raising=False
+        )
+
+        class _Context:
+            function_name = "my-fn"
+            function_version = "$LATEST"
+            invoked_function_arn = (
+                "arn:aws:lambda:us-east-1:123456789012:function:my-fn"
+            )
+            memory_limit_in_mb = 128
+            aws_request_id = "req-1"
+
+        try:
+            instrumentor.instrument()
+            wrapped = __import__("lambda_tasks.handler", fromlist=["handler"]).handler
+            wrapped({"Records": []}, _Context())
+        finally:
+            if instrumentor.is_instrumented_by_opentelemetry:
+                instrumentor.uninstrument()
+
+        spans = exporter.get_finished_spans()
+        server_spans = [s for s in spans if s.kind == trace_api.SpanKind.SERVER]
+        inner_spans = [s for s in spans if s.name == "inner-db-span"]
+
+        assert server_spans, f"no SERVER span produced; got {[s.name for s in spans]}"
+        assert inner_spans, "inner task span was not recorded"
+
+        server = server_spans[0]
+        inner = inner_spans[0]
+
+        assert inner.context.trace_id == server.context.trace_id
+        assert inner.parent is not None
+        assert inner.parent.span_id == server.context.span_id