datadog-async-handler 0.1.2__tar.gz → 0.1.4__tar.gz

datadog_async_handler-0.1.4/CHANGELOG.md

@@ -0,0 +1,64 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.4] - 2026-09-02
+
+### Fixed
+- Fork safety: a process that called `fork()` while the worker thread was inside a TLS
+  send handed the child OpenSSL state mid-update, and the child deadlocked on its own
+  first TLS handshake (observed with Celery prefork workers). `os.register_at_fork`
+  hooks now hold the send lock across `fork()` and give the child a fresh queue, lock,
+  and worker thread. The lock covers only the network attempt, not retry back-off.
+- The worker thread is named `datadog-http-handler`.
+
+## [0.1.2] - 2025-08-25
+
+### Added
+- First stable release of datadog-async-handler
+- High-performance async HTTP logging handler for Datadog
+- Asynchronous batching and background processing
+- Automatic retry logic with exponential backoff
+- Configurable batch size and flush intervals
+- Rich metadata support (service, environment, tags)
+- Multi-site Datadog support (US, EU, etc.)
+- Full type hints and mypy compatibility
+- Comprehensive documentation and examples
+- GitHub Actions CI/CD with trusted PyPI publishing
+
+### Features
+- **Performance**: Non-blocking background processing
+- **Reliability**: Comprehensive retry logic and error handling
+- **Efficiency**: Memory-optimized batching reduces API calls
+- **Integration**: Drop-in replacement for standard logging handlers
+- **Framework Support**: Examples for Django, FastAPI, Flask, Celery
+- **Testing**: Comprehensive test suite with >95% coverage
+
+### Documentation
+- Complete API reference with mkdocstrings
+- Framework integration examples
+- Configuration guide with all parameters
+- Troubleshooting documentation
+- Installation instructions
+
+### Infrastructure
+- Modern Python packaging with pyproject.toml
+- Hatch build system with UV package manager
+- Ruff for formatting and linting
+- MyPy for type checking
+- Pytest for testing
+- MkDocs Material theme for documentation
+- GitHub Actions for CI/CD
+- Trusted publishing to PyPI
+
+## [Unreleased]
+
+### Planned
+- Async/await support for truly asynchronous logging
+- Metrics and tracing integration
+- Advanced filtering and transformation options
+- Performance optimizations
+- Additional framework integrations

{datadog_async_handler-0.1.2 → datadog_async_handler-0.1.4}/PKG-INFO

@@ -1,12 +1,12 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: datadog-async-handler
-Version: 0.1.2
+Version: 0.1.4
 Summary: High-performance async HTTP logging handler for Datadog with batching and retry logic
-Project-URL: Homepage, https://github.com/enlyft/datadog-http-handler
-Project-URL: Documentation, https://enlyft.github.io/datadog-http-handler
-Project-URL: Repository, https://github.com/enlyft/datadog-http-handler.git
-Project-URL: Changelog, https://github.com/enlyft/datadog-http-handler/blob/main/CHANGELOG.md
-Project-URL: Bug Tracker, https://github.com/enlyft/datadog-http-handler/issues
+Project-URL: Homepage, https://github.com/enlyft/datadog-async-handler
+Project-URL: Documentation, https://enlyft.github.io/datadog-async-handler
+Project-URL: Repository, https://github.com/enlyft/datadog-async-handler.git
+Project-URL: Changelog, https://github.com/enlyft/datadog-async-handler/blob/main/CHANGELOG.md
+Project-URL: Bug Tracker, https://github.com/enlyft/datadog-async-handler/issues
 Author-email: Parth <parth@enlyft.com>
 Maintainer-email: Parth <parth@enlyft.com>
 License-Expression: MIT
@@ -53,7 +53,7 @@ Description-Content-Type: text/markdown
 
 # Datadog Async Handler
 
-[![PyPI version](https://badge.fury.io/py/datadog-async-handler.svg)](https://badge.fury.io/py/datadog-async-handler)
+[![PyPI version](https://img.shields.io/pypi/v/datadog-async-handler)](https://pypi.org/project/datadog-async-handler/)
 [![Python versions](https://img.shields.io/pypi/pyversions/datadog-async-handler.svg)](https://pypi.org/project/datadog-async-handler/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![Code style: ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

{datadog_async_handler-0.1.2 → datadog_async_handler-0.1.4}/README.md

@@ -1,6 +1,6 @@
 # Datadog Async Handler
 
-[![PyPI version](https://badge.fury.io/py/datadog-async-handler.svg)](https://badge.fury.io/py/datadog-async-handler)
+[![PyPI version](https://img.shields.io/pypi/v/datadog-async-handler)](https://pypi.org/project/datadog-async-handler/)
 [![Python versions](https://img.shields.io/pypi/pyversions/datadog-async-handler.svg)](https://pypi.org/project/datadog-async-handler/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![Code style: ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

{datadog_async_handler-0.1.2 → datadog_async_handler-0.1.4}/mkdocs.yml

@@ -1,8 +1,8 @@
 site_name: Datadog Async Handler
 site_description: High-performance async HTTP logging handler for Datadog with batching and retry logic
-site_url: https://enlyft.github.io/datadog-http-handler
-repo_url: https://github.com/enlyft/datadog-http-handler
-repo_name: enlyft/datadog-http-handler
+site_url: https://enlyft.github.io/datadog-async-handler
+repo_url: https://github.com/enlyft/datadog-async-handler
+repo_name: enlyft/datadog-async-handler
 
 theme:
   name: material

{datadog_async_handler-0.1.2 → datadog_async_handler-0.1.4}/pyproject.toml

@@ -71,11 +71,11 @@ test = [
 ]
 
 [project.urls]
-Homepage = "https://github.com/enlyft/datadog-http-handler"
-Documentation = "https://enlyft.github.io/datadog-http-handler"
-Repository = "https://github.com/enlyft/datadog-http-handler.git"
-Changelog = "https://github.com/enlyft/datadog-http-handler/blob/main/CHANGELOG.md"
-"Bug Tracker" = "https://github.com/enlyft/datadog-http-handler/issues"
+Homepage = "https://github.com/enlyft/datadog-async-handler"
+Documentation = "https://enlyft.github.io/datadog-async-handler"
+Repository = "https://github.com/enlyft/datadog-async-handler.git"
+Changelog = "https://github.com/enlyft/datadog-async-handler/blob/main/CHANGELOG.md"
+"Bug Tracker" = "https://github.com/enlyft/datadog-async-handler/issues"
 
 [tool.hatch.version]
 source = "vcs"

{datadog_async_handler-0.1.2 → datadog_async_handler-0.1.4}/src/datadog_http_handler/__init__.py

@@ -24,4 +24,4 @@ Example:
 from .handler import DatadogHTTPHandler
 
 __version__ = "0.1.0"
-__all__ = ["DatadogHTTPHandler"]
+__all__ = ["DatadogHTTPHandler", "DatadogJsonFormatter"]

datadog_async_handler-0.1.4/src/datadog_http_handler/formatter.py

@@ -0,0 +1,132 @@
+import json
+import logging
+import os
+from typing import Optional
+from datetime import datetime, timezone
+
+
+class DatadogJsonFormatter(logging.Formatter):
+    """
+    JSON formatter optimized for Datadog log ingestion.
+
+    This formatter creates structured JSON logs with fields that Datadog
+    can automatically parse and index for better searchability and analysis.
+    """
+
+    def __init__(self, service_name: str, version: Optional[str] = None):
+        """
+        Initialize the formatter.
+
+        Args:
+            service_name: Name of the service (e.g., 'auth-service', 'embedding-service')
+            version: Version of the service (optional)
+        """
+        super().__init__()
+        self.service_name = service_name
+        self.version = version or os.getenv("SERVICE_VERSION", "unknown")
+        self.environment = os.getenv("DD_ENV", "development")
+
+    def format(self, record: logging.LogRecord) -> str:
+        """
+        Format a log record as JSON.
+
+        Args:
+            record: The log record to format
+
+        Returns:
+            JSON-formatted log string
+        """
+        # Base log structure for Datadog
+        log_entry = {
+            "timestamp": datetime.fromtimestamp(record.created, tz=timezone.utc).isoformat(),
+            "level": record.levelname,
+            "logger": record.name,
+            "message": record.getMessage(),
+            "service": self.service_name,
+            "version": self.version,
+            "env": self.environment,
+            "process_id": record.process,
+        }
+
+        # --- Datadog correlation IDs (must be at top level) ---
+        # First check nested dd fields
+        if hasattr(record, "dd"):
+            dd_fields = record.dd
+            if "trace_id" in dd_fields:
+                log_entry["dd.trace_id"] = dd_fields["trace_id"]
+            if "span_id" in dd_fields:
+                log_entry["dd.span_id"] = dd_fields["span_id"]
+            if "service" in dd_fields:
+                log_entry["dd.service"] = dd_fields["service"]
+            if "version" in dd_fields:
+                log_entry["dd.version"] = dd_fields["version"]
+            if "env" in dd_fields:
+                log_entry["dd.env"] = dd_fields["env"]
+
+        # Then check for flat Datadog attributes that ddtrace may inject
+        dd_flat_keys = ["dd.trace_id", "dd.span_id", "dd.service", "dd.version", "dd.env"]
+        for dd_key in dd_flat_keys:
+            # Only use flat key if we don't already have this field from record.dd
+            if dd_key not in log_entry and hasattr(record, dd_key):
+                log_entry[dd_key] = getattr(record, dd_key)
+
+        # --- Add source info ---
+        if record.pathname:
+            log_entry["source"] = {
+                "file": record.pathname,
+                "line": record.lineno,
+                "function": record.funcName,
+            }
+
+        # --- Collect remaining extras, except dd (already flattened) ---
+        extra_fields = {
+            k: v
+            for k, v in record.__dict__.items()
+            if k
+            not in {
+                "name",
+                "msg",
+                "args",
+                "levelname",
+                "levelno",
+                "pathname",
+                "filename",
+                "module",
+                "lineno",
+                "funcName",
+                "created",
+                "msecs",
+                "relativeCreated",
+                "thread",
+                "threadName",
+                "processName",
+                "process",
+                "getMessage",
+                "exc_info",
+                "exc_text",
+                "stack_info",
+                "message",
+                "dd",  # skip dd because we handled it
+                "dd.trace_id",  # skip flat dd keys because we handled them
+                "dd.span_id",
+                "dd.service",
+                "dd.version",
+                "dd.env",
+            }
+        }
+        if extra_fields:
+            log_entry["extra"] = extra_fields
+
+        # --- Exception handling ---
+        if record.exc_info:
+            log_entry["exception"] = {
+                "class": record.exc_info[0].__name__ if record.exc_info[0] else None,
+                "message": str(record.exc_info[1]) if record.exc_info[1] else None,
+                "traceback": self.formatException(record.exc_info),
+            }
+
+        if record.stack_info:
+            log_entry["stack_trace"] = record.stack_info
+
+        return json.dumps(log_entry, default=str, ensure_ascii=False)
+

{datadog_async_handler-0.1.2 → datadog_async_handler-0.1.4}/src/datadog_http_handler/handler.py

@@ -9,6 +9,7 @@ import logging
 import os
 import threading
 import time
+import weakref
 from queue import Empty, Queue
 from typing import Optional
 
@@ -18,6 +19,17 @@ from datadog_api_client.v2.model.http_log import HTTPLog
 from datadog_api_client.v2.model.http_log_item import HTTPLogItem
 
 
+def _at_fork(ref: "weakref.ReferenceType[DatadogHTTPHandler]", method: str) -> None:
+    """Dispatch an ``os.register_at_fork`` callback to a handler that is still alive.
+
+    ``os.register_at_fork`` keeps its callables for the life of the process, so the
+    hooks hold only a weak reference and become no-ops once the handler is collected.
+    """
+    handler = ref()
+    if handler is not None:
+        getattr(handler, method)()
+
+
 class DatadogHTTPHandler(logging.Handler):
     """
     High-performance logging handler that sends logs to Datadog via HTTP API.
@@ -26,6 +38,12 @@ class DatadogHTTPHandler(logging.Handler):
     the main application thread. It includes retry logic with exponential backoff,
     comprehensive error handling, and support for all Datadog sites.
 
+    The handler is fork-safe. Its worker thread sends over TLS, and a child forked
+    while that handshake is in flight inherits OpenSSL state mid-update and deadlocks
+    on its own first TLS handshake. ``os.register_at_fork`` hooks hold a send lock
+    across ``fork()`` so no child is forked mid-send, and give the child a fresh
+    queue, lock, and worker thread (threads do not survive ``fork()``).
+
     Args:
         api_key: Datadog API key (or set DD_API_KEY env var)
         site: Datadog site (default: datadoghq.com)
@@ -60,6 +78,9 @@ class DatadogHTTPHandler(logging.Handler):
         >>> logger.addHandler(handler)
     """
 
+    # Bounded so a worker wedged inside C can never freeze the parent's fork() forever.
+    _FORK_WAIT_SECONDS = 15.0
+
     def __init__(
         self,
         api_key: Optional[str] = None,
@@ -103,7 +124,16 @@ class DatadogHTTPHandler(logging.Handler):
         self._log_queue: Queue[HTTPLogItem] = Queue()
         self._stop_event = threading.Event()
         self._worker_thread: Optional[threading.Thread] = None
+        self._send_lock = threading.Lock()
+        self._fork_lock_held = False
         self._start_worker()
+        if hasattr(os, "register_at_fork"):  # POSIX only
+            ref = weakref.ref(self)
+            os.register_at_fork(
+                before=lambda: _at_fork(ref, "_before_fork"),
+                after_in_parent=lambda: _at_fork(ref, "_after_fork_in_parent"),
+                after_in_child=lambda: _at_fork(ref, "_after_fork_in_child"),
+            )
 
     def _setup_api_client(self) -> None:
         """Setup the Datadog API client."""
@@ -117,7 +147,9 @@ class DatadogHTTPHandler(logging.Handler):
     def _start_worker(self) -> None:
         """Start the background worker thread."""
         if self._worker_thread is None or not self._worker_thread.is_alive():
-            self._worker_thread = threading.Thread(target=self._worker, daemon=True)
+            self._worker_thread = threading.Thread(
+                target=self._worker, daemon=True, name="datadog-http-handler"
+            )
             self._worker_thread.start()
 
     def _worker(self) -> None:
@@ -161,7 +193,8 @@ class DatadogHTTPHandler(logging.Handler):
         for attempt in range(self.max_retries + 1):
             try:
                 http_log = HTTPLog(batch)
-                self.logs_api.submit_log(body=http_log)
+                with self._send_lock:  # fork() waits for this, not for back-off
+                    self.logs_api.submit_log(body=http_log)
                 return  # Success
 
             except Exception as e:
@@ -174,6 +207,33 @@ class DatadogHTTPHandler(logging.Handler):
                     # Wait before retry (exponential backoff)
                     time.sleep(2**attempt)
 
+    def _before_fork(self) -> None:
+        """Hold the send lock across ``fork()`` so no child is forked mid-send."""
+        self._fork_lock_held = self._send_lock.acquire(timeout=self._FORK_WAIT_SECONDS)
+        if not self._fork_lock_held:
+            self._handle_error(
+                "forking while a log send is wedged; the child's TLS may deadlock"
+            )
+
+    def _after_fork_in_parent(self) -> None:
+        """Release the lock taken by ``_before_fork``."""
+        if self._fork_lock_held:
+            self._fork_lock_held = False
+            self._send_lock.release()
+
+    def _after_fork_in_child(self) -> None:
+        """Give the child a fresh lock, queue, and worker thread.
+
+        The parent's worker thread does not exist in the child, and the inherited
+        lock may still be held by it.
+        """
+        self._send_lock = threading.Lock()
+        self._fork_lock_held = False
+        self._log_queue = Queue()
+        self._stop_event = threading.Event()
+        self._worker_thread = None
+        self._start_worker()
+
     def _handle_error(self, message: str) -> None:
         """Handle errors that occur during log submission."""
         # Log to stderr to avoid infinite recursion

datadog_async_handler-0.1.2/CHANGELOG.md

@@ -1,48 +0,0 @@
-# Changelog
-
-All notable changes to this project will be documented in this file.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
-and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-
-## [Unreleased]
-
-### Added
-- Initial release of datadog-http-handler
-- Asynchronous HTTP logging handler for Datadog
-- Batching support with configurable batch size and flush intervals
-- Automatic retry logic with exponential backoff
-- Comprehensive error handling and graceful degradation
-- Type hints and mypy compatibility
-- Support for all Datadog sites (US, EU, Government, etc.)
-- Environment variable configuration support
-- Framework integration examples (Django, FastAPI, Flask)
-- Background worker thread for non-blocking log processing
-- Automatic tag enrichment (environment, service, version)
-- Memory-efficient queue management
-- Graceful shutdown handling
-
-### Changed
-- N/A (initial release)
-
-### Deprecated
-- N/A (initial release)
-
-### Removed
-- N/A (initial release)
-
-### Fixed
-- N/A (initial release)
-
-### Security
-- N/A (initial release)
-
-## [0.1.0] - 2025-08-25
-
-### Added
-- Initial development version
-- Core DatadogHTTPHandler implementation
-- Basic test suite
-- Documentation and examples
-- Modern Python packaging with Hatch
-- Comprehensive development tooling (Ruff, mypy, pytest)