datadog-async-handler 0.1.2__py3-none-any.whl

datadog_async_handler-0.1.2.dist-info/METADATA

@@ -0,0 +1,294 @@
+Metadata-Version: 2.4
+Name: datadog-async-handler
+Version: 0.1.2
+Summary: High-performance async HTTP logging handler for Datadog with batching and retry logic
+Project-URL: Homepage, https://github.com/enlyft/datadog-http-handler
+Project-URL: Documentation, https://enlyft.github.io/datadog-http-handler
+Project-URL: Repository, https://github.com/enlyft/datadog-http-handler.git
+Project-URL: Changelog, https://github.com/enlyft/datadog-http-handler/blob/main/CHANGELOG.md
+Project-URL: Bug Tracker, https://github.com/enlyft/datadog-http-handler/issues
+Author-email: Parth <parth@enlyft.com>
+Maintainer-email: Parth <parth@enlyft.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: async,batching,datadog,handler,logging,observability
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Logging
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.9
+Requires-Dist: datadog-api-client<3.0.0,>=2.0.0
+Provides-Extra: async
+Requires-Dist: aiohttp>=3.8.0; extra == 'async'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.8.0; extra == 'dev'
+Requires-Dist: pre-commit>=3.6.0; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
+Requires-Dist: pytest-benchmark>=4.0.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
+Requires-Dist: pytest-mock>=3.12.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: ruff>=0.2.0; extra == 'dev'
+Requires-Dist: tox>=4.0.0; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
+Requires-Dist: mkdocs>=1.5.0; extra == 'docs'
+Requires-Dist: mkdocstrings[python]>=0.24.0; extra == 'docs'
+Provides-Extra: test
+Requires-Dist: pytest-asyncio>=0.23.0; extra == 'test'
+Requires-Dist: pytest-benchmark>=4.0.0; extra == 'test'
+Requires-Dist: pytest-cov>=4.0.0; extra == 'test'
+Requires-Dist: pytest-mock>=3.12.0; extra == 'test'
+Requires-Dist: pytest>=8.0.0; extra == 'test'
+Description-Content-Type: text/markdown
+
+# Datadog Async Handler
+
+[![PyPI version](https://badge.fury.io/py/datadog-async-handler.svg)](https://badge.fury.io/py/datadog-async-handler)
+[![Python versions](https://img.shields.io/pypi/pyversions/datadog-async-handler.svg)](https://pypi.org/project/datadog-async-handler/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Code style: ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+
+A modern, high-performance Python logging handler that sends logs directly to Datadog via HTTP API with asynchronous batching, retry logic, and comprehensive error handling.
+
+## ✨ Features
+
+- **🚀 High Performance**: Asynchronous batching and background processing
+- **🔄 Reliable Delivery**: Automatic retry with exponential backoff
+- **📊 Batching**: Configurable batch size and flush intervals
+- **🏷️ Rich Metadata**: Automatic service, environment, and custom tag support
+- **🔧 Easy Integration**: Drop-in replacement for standard logging handlers
+- **🌐 Multi-Site Support**: Works with all Datadog sites (US, EU, etc.)
+- **📝 Type Safe**: Full type hints and mypy compatibility
+- **⚡ Modern**: Built with Python 3.9+ and latest best practices
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+pip install datadog-async-handler
+```
+
+### Basic Usage
+
+```python
+import logging
+from datadog_http_handler import DatadogHTTPHandler
+
+# Configure the handler
+handler = DatadogHTTPHandler(
+    api_key="your-datadog-api-key",  # or set DD_API_KEY env var
+    service="my-application",
+    source="python",
+    tags="env:production,team:backend"
+)
+
+# Set up logging
+logger = logging.getLogger(__name__)
+logger.addHandler(handler)
+logger.setLevel(logging.INFO)
+
+# Start logging!
+logger.info("Application started successfully", extra={
+    "user_id": "12345",
+    "action": "startup"
+})
+```
+
+### Environment Variables
+
+The handler automatically picks up standard Datadog environment variables:
+
+```bash
+export DD_API_KEY="your-api-key"
+export DD_SERVICE="my-application"
+export DD_ENV="production"
+export DD_VERSION="1.2.3"
+export DD_TAGS="team:backend,component:api"
+export DD_SITE="datadoghq.com"  # or datadoghq.eu, ddog-gov.com, etc.
+```
+
+## 📖 Documentation
+
+### Configuration Options
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `api_key` | `str` | `None` | Datadog API key (required) |
+| `site` | `str` | `"datadoghq.com"` | Datadog site |
+| `service` | `str` | `None` | Service name |
+| `source` | `str` | `"python"` | Log source |
+| `hostname` | `str` | `None` | Hostname |
+| `tags` | `str` | `None` | Comma-separated tags |
+| `batch_size` | `int` | `10` | Number of logs per batch |
+| `flush_interval_seconds` | `float` | `5.0` | Batch flush interval |
+| `timeout_seconds` | `float` | `10.0` | Request timeout |
+| `max_retries` | `int` | `3` | Maximum retry attempts |
+
+### Framework Integration Examples
+
+#### Django
+
+```python
+# settings.py
+LOGGING = {
+    'version': 1,
+    'disable_existing_loggers': False,
+    'handlers': {
+        'datadog': {
+            'class': 'datadog_http_handler.DatadogHTTPHandler',
+            'api_key': 'your-api-key',
+            'service': 'django-app',
+            'source': 'django',
+        },
+    },
+    'root': {
+        'handlers': ['datadog'],
+        'level': 'INFO',
+    },
+}
+```
+
+#### FastAPI
+
+```python
+import logging
+from fastapi import FastAPI
+from datadog_http_handler import DatadogHTTPHandler
+
+app = FastAPI()
+
+# Configure logging
+handler = DatadogHTTPHandler(service="fastapi-app", source="fastapi")
+logging.getLogger().addHandler(handler)
+logging.getLogger().setLevel(logging.INFO)
+
+@app.get("/")
+async def root():
+    logging.info("API endpoint called", extra={"endpoint": "/"})
+    return {"message": "Hello World"}
+```
+
+#### Flask
+
+```python
+import logging
+from flask import Flask
+from datadog_http_handler import DatadogHTTPHandler
+
+app = Flask(__name__)
+
+# Configure logging
+handler = DatadogHTTPHandler(service="flask-app", source="flask")
+app.logger.addHandler(handler)
+app.logger.setLevel(logging.INFO)
+
+@app.route("/")
+def hello():
+    app.logger.info("Flask endpoint called", extra={"endpoint": "/"})
+    return "Hello World!"
+```
+
+## 🔧 Development
+
+### Setup
+
+```bash
+# Clone the repository
+git clone https://github.com/enlyft/datadog-http-handler.git
+cd datadog-http-handler
+
+# Install UV (modern Python package manager)
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Install dependencies
+uv pip install -e ".[dev]"
+
+# Install pre-commit hooks
+pre-commit install
+```
+
+### Running Tests
+
+```bash
+# Run all tests
+hatch run test
+
+# Run with coverage
+hatch run test-cov
+
+# Run specific tests
+hatch run test tests/test_handler.py::test_basic_logging
+```
+
+### Code Quality
+
+```bash
+# Format code
+hatch run format
+
+# Lint code
+hatch run lint
+
+# Type checking
+hatch run type-check
+
+# Run all checks
+hatch run all
+```
+
+## 🤝 Contributing
+
+We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes
+4. Add tests
+5. Run the test suite
+6. Submit a pull request
+
+## 📋 Requirements
+
+- Python 3.9+
+- `datadog-api-client>=2.0.0`
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## 🔗 Links
+
+- [Documentation](https://enlyft.github.io/datadog-http-handler)
+- [PyPI Package](https://pypi.org/project/datadog-async-handler/)
+- [GitHub Repository](https://github.com/enlyft/datadog-http-handler)
+- [Issue Tracker](https://github.com/enlyft/datadog-http-handler/issues)
+- [Datadog Logs API Documentation](https://docs.datadoghq.com/api/latest/logs/)
+
+## 🆚 Comparison with Other Solutions
+
+| Feature | datadog-async-handler | datadog-http-handler | python-datadog | datadog-logger |
+|---------|----------------------|---------------------|----------------|----------------|
+| Async Batching | ✅ | ❌ | ❌ | ❌ |
+| Retry Logic | ✅ | ❌ | ❌ | ❌ |
+| Type Hints | ✅ | ❌ | ❌ | ❌ |
+| Modern Python | ✅ (3.9+) | ❌ (3.6+) | ❌ (2.7+) | ❌ (3.6+) |
+| Official API Client | ✅ | ❌ | ❌ | ❌ |
+| Background Processing | ✅ | ❌ | ❌ | ❌ |
+| Memory Efficient | ✅ | ❌ | ❌ | ❌ |
+| Active Maintenance | ✅ | ❌ (2019) | ✅ | ❌ |
+| Comprehensive Tests | ✅ | ❌ | ✅ | ❌ |
+
+---
+
+Made with ❤️ for the Python and Datadog communities.

datadog_async_handler-0.1.2.dist-info/RECORD

@@ -0,0 +1,7 @@
+datadog_http_handler/__init__.py,sha256=opKEnLnS-qXQ-alt7EPRQQk91E8MEwEznQ4-3k4-92s,744
+datadog_http_handler/handler.py,sha256=_pJAK1iKgVO4mR2dTL9FDlNi4oJVkq_D59R69aH7Sok,10660
+datadog_http_handler/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+datadog_async_handler-0.1.2.dist-info/METADATA,sha256=CQilYl9Lm_A0zDvb6Cqlj5VZQUUVSdBWKNqwLqiOGhc,9017
+datadog_async_handler-0.1.2.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+datadog_async_handler-0.1.2.dist-info/licenses/LICENSE,sha256=E4Va_3PEQq8kxwFMY3NDGGjDbiklFYi-Z6QJI0NEfq8,1061
+datadog_async_handler-0.1.2.dist-info/RECORD,,

datadog_async_handler-0.1.2.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.27.0
+Root-Is-Purelib: true
+Tag: py3-none-any

datadog_async_handler-0.1.2.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Parth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

datadog_http_handler/__init__.py

@@ -0,0 +1,27 @@
+"""
+Datadog HTTP Handler - Modern Python logging handler for Datadog.
+
+This package provides a high-performance, asynchronous logging handler that sends
+logs directly to Datadog's HTTP intake API with batching, retry logic, and
+comprehensive error handling.
+
+Example:
+    Basic usage:
+
+    >>> import logging
+    >>> from datadog_http_handler import DatadogHTTPHandler
+    >>>
+    >>> handler = DatadogHTTPHandler(
+    ...     api_key="your-api-key",
+    ...     service="my-app",
+    ...     tags="env:production"
+    ... )
+    >>> logger = logging.getLogger(__name__)
+    >>> logger.addHandler(handler)
+    >>> logger.info("Hello Datadog!")
+"""
+
+from .handler import DatadogHTTPHandler
+
+__version__ = "0.1.0"
+__all__ = ["DatadogHTTPHandler"]

datadog_http_handler/handler.py

@@ -0,0 +1,309 @@
+"""
+Datadog HTTP logging handler implementation.
+
+This module provides the DatadogHTTPHandler class, a modern Python logging handler
+that sends logs to Datadog via HTTP API with asynchronous batching and retry logic.
+"""
+
+import logging
+import os
+import threading
+import time
+from queue import Empty, Queue
+from typing import Optional
+
+from datadog_api_client import ApiClient, Configuration
+from datadog_api_client.v2.api.logs_api import LogsApi
+from datadog_api_client.v2.model.http_log import HTTPLog
+from datadog_api_client.v2.model.http_log_item import HTTPLogItem
+
+
+class DatadogHTTPHandler(logging.Handler):
+    """
+    High-performance logging handler that sends logs to Datadog via HTTP API.
+
+    This handler batches logs and sends them asynchronously to avoid blocking
+    the main application thread. It includes retry logic with exponential backoff,
+    comprehensive error handling, and support for all Datadog sites.
+
+    Args:
+        api_key: Datadog API key (or set DD_API_KEY env var)
+        site: Datadog site (default: datadoghq.com)
+        service: Service name for logs
+        source: Log source (default: python)
+        hostname: Hostname for logs
+        tags: Tags to add to logs (comma-separated string)
+        batch_size: Number of logs to batch before sending
+        flush_interval_seconds: How often to flush logs (seconds)
+        timeout_seconds: Request timeout
+        max_retries: Maximum retry attempts
+        level: Logging level
+
+    Example:
+        Basic usage:
+
+        >>> import logging
+        >>> handler = DatadogHTTPHandler(
+        ...     api_key="your-api-key",
+        ...     service="my-app",
+        ...     tags="env:production,team:backend"
+        ... )
+        >>> logger = logging.getLogger(__name__)
+        >>> logger.addHandler(handler)
+        >>> logger.info("Application started")
+
+        With environment variables:
+
+        >>> # Set DD_API_KEY, DD_SERVICE, DD_ENV, etc.
+        >>> handler = DatadogHTTPHandler()  # Uses env vars
+        >>> logger = logging.getLogger(__name__)
+        >>> logger.addHandler(handler)
+    """
+
+    def __init__(
+        self,
+        api_key: Optional[str] = None,
+        site: str = "datadoghq.com",
+        service: Optional[str] = None,
+        source: str = "python",
+        hostname: Optional[str] = None,
+        tags: Optional[str] = None,
+        batch_size: int = 10,
+        flush_interval_seconds: float = 5.0,
+        timeout_seconds: float = 10.0,
+        max_retries: int = 3,
+        level: int = logging.NOTSET,
+    ) -> None:
+        """Initialize the Datadog HTTP handler."""
+        super().__init__(level)
+
+        # API Configuration
+        self.api_key = api_key or os.getenv("DD_API_KEY")
+        if not self.api_key:
+            raise ValueError(
+                "Datadog API key is required. Set DD_API_KEY env var or pass api_key parameter."
+            )
+
+        self.site = site or os.getenv("DD_SITE", "datadoghq.com")
+        self.service = service or os.getenv("DD_SERVICE", "unknown")
+        self.source = source
+        self.hostname = hostname or os.getenv("DD_HOSTNAME")
+        self.tags = tags or os.getenv("DD_TAGS", "")
+
+        # Batch settings
+        self.batch_size = max(1, batch_size)  # Ensure at least 1
+        self.flush_interval = max(0.1, flush_interval_seconds)  # Minimum 100ms
+        self.timeout = max(1.0, timeout_seconds)  # Minimum 1 second
+        self.max_retries = max(0, max_retries)  # No negative retries
+
+        # Initialize API client
+        self._setup_api_client()
+
+        # Background processing
+        self._log_queue: Queue[HTTPLogItem] = Queue()
+        self._stop_event = threading.Event()
+        self._worker_thread: Optional[threading.Thread] = None
+        self._start_worker()
+
+    def _setup_api_client(self) -> None:
+        """Setup the Datadog API client."""
+        configuration = Configuration()
+        configuration.api_key["apiKeyAuth"] = self.api_key
+        configuration.server_variables["site"] = self.site
+
+        self.api_client = ApiClient(configuration)
+        self.logs_api = LogsApi(self.api_client)
+
+    def _start_worker(self) -> None:
+        """Start the background worker thread."""
+        if self._worker_thread is None or not self._worker_thread.is_alive():
+            self._worker_thread = threading.Thread(target=self._worker, daemon=True)
+            self._worker_thread.start()
+
+    def _worker(self) -> None:
+        """Background worker that processes log batches."""
+        batch: list[HTTPLogItem] = []
+        last_flush = time.time()
+
+        while not self._stop_event.is_set():
+            try:
+                # Get log from queue with timeout
+                log_item = self._log_queue.get(timeout=0.1)
+                batch.append(log_item)
+
+                # Check if we should flush
+                should_flush = len(batch) >= self.batch_size or (
+                    batch and time.time() - last_flush >= self.flush_interval
+                )
+
+                if should_flush:
+                    self._send_batch(batch)
+                    batch.clear()
+                    last_flush = time.time()
+
+            except Empty:
+                # No logs in queue, check if we should flush existing batch
+                if batch and time.time() - last_flush >= self.flush_interval:
+                    self._send_batch(batch)
+                    batch.clear()
+                    last_flush = time.time()
+                continue
+
+        # Flush remaining logs on shutdown
+        if batch:
+            self._send_batch(batch)
+
+    def _send_batch(self, batch: list[HTTPLogItem]) -> None:
+        """Send a batch of logs to Datadog."""
+        if not batch:
+            return
+
+        for attempt in range(self.max_retries + 1):
+            try:
+                http_log = HTTPLog(batch)
+                self.logs_api.submit_log(body=http_log)
+                return  # Success
+
+            except Exception as e:
+                if attempt == self.max_retries:
+                    # Last attempt failed, give up
+                    self._handle_error(
+                        f"Failed to send logs after {self.max_retries + 1} attempts: {e}"
+                    )
+                else:
+                    # Wait before retry (exponential backoff)
+                    time.sleep(2**attempt)
+
+    def _handle_error(self, message: str) -> None:
+        """Handle errors that occur during log submission."""
+        # Log to stderr to avoid infinite recursion
+        print(f"DatadogHTTPHandler error: {message}", file=__import__("sys").stderr)
+
+    def emit(self, record: logging.LogRecord) -> None:
+        """Emit a log record."""
+        try:
+            # Ensure handler is properly initialized
+            if not hasattr(self, "_log_queue") or not hasattr(self, "api_key"):
+                # Handler not properly initialized, skip logging to avoid errors
+                return
+
+            log_item = self._format_log_item(record)
+            self._log_queue.put(log_item, block=False)
+        except Exception:
+            self.handleError(record)
+
+    def _format_log_item(self, record: logging.LogRecord) -> HTTPLogItem:
+        """Convert a LogRecord to a Datadog HTTPLogItem."""
+        # Format the message
+        message = self.format(record)
+
+        # Build the log item
+        log_item = HTTPLogItem(
+            message=message,
+            ddsource=self.source,
+            service=self.service,
+            hostname=self.hostname,
+        )
+
+        # Add tags
+        tags_list = []
+        if self.tags:
+            tags_list.extend(self.tags.split(","))
+
+        # Add log level as tag
+        tags_list.append(f"level:{record.levelname.lower()}")
+
+        # Add logger name as tag
+        tags_list.append(f"logger:{record.name}")
+
+        # Add environment if available
+        env = os.getenv("DD_ENV")
+        if env:
+            tags_list.append(f"env:{env}")
+
+        # Add version if available
+        version = os.getenv("DD_VERSION")
+        if version:
+            tags_list.append(f"version:{version}")
+
+        # Add any extra fields from the log record
+        if hasattr(record, "__dict__"):
+            for key, value in record.__dict__.items():
+                if key.startswith("dd_") and isinstance(value, (str, int, float, bool)):
+                    # Custom Datadog fields
+                    tags_list.append(f"{key[3:]}:{value}")
+
+        if tags_list:
+            log_item.ddtags = ",".join(tags_list)
+
+        return log_item
+
+    def flush(self) -> None:
+        """Flush any pending logs."""
+        # Signal worker to process remaining logs
+        if (
+            hasattr(self, "_worker_thread")
+            and self._worker_thread
+            and self._worker_thread.is_alive()
+        ):
+            # Wait a bit for the worker to process
+            time.sleep(0.1)
+
+    def close(self) -> None:
+        """Close the handler and cleanup resources."""
+        # Stop the worker
+        if hasattr(self, "_stop_event"):
+            self._stop_event.set()
+
+        # Wait for worker to finish
+        if (
+            hasattr(self, "_worker_thread")
+            and self._worker_thread
+            and self._worker_thread.is_alive()
+        ):
+            self._worker_thread.join(timeout=5.0)
+
+        # Close API client
+        if hasattr(self, "api_client"):
+            self.api_client.close()
+
+        super().close()
+
+    def health_check(self) -> bool:
+        """
+        Perform a health check to verify the handler is working.
+
+        Returns:
+            True if the handler is healthy, False otherwise.
+        """
+        try:
+            # Check if worker thread is alive
+            if not self._worker_thread or not self._worker_thread.is_alive():
+                return False
+
+            # Check if we can create a test log item
+            test_record = logging.LogRecord(
+                name="health_check",
+                level=logging.INFO,
+                pathname="",
+                lineno=0,
+                msg="Health check",
+                args=(),
+                exc_info=None,
+            )
+            self._format_log_item(test_record)
+            return True
+
+        except Exception:
+            return False
+
+    def get_queue_size(self) -> int:
+        """Get the current size of the log queue."""
+        return self._log_queue.qsize() if hasattr(self, "_log_queue") else 0
+
+    def __repr__(self) -> str:
+        """Return a string representation of the handler."""
+        return (
+            f"DatadogHTTPHandler(service={self.service!r}, "
+            f"source={self.source!r}, site={self.site!r})"
+        )