litewave-logger 0.1.0__tar.gz

litewave_logger-0.1.0/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.4
+Name: litewave_logger
+Version: 0.1.0
+Summary: A lightweight logging library with context carry forward
+Author: Litewave
+Author-email: Sonu Sudhakaran <sonu@litewave.ai>
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+Requires-Dist: requests
+Requires-Dist: fastapi
+Dynamic: author
+Dynamic: requires-python
+
+# Litewave Logger
+
+This module provides a centralized and consistent logging solution for Litewave services. It ensures that a `request-id` is maintained across all services, including HTTP requests and Celery tasks, allowing for easy tracing of requests as they propagate through the system.
+
+## Features
+
+- **Centralized Logging**: A single module to configure and manage logging across all services.
+- **Request ID Propagation**: Automatically injects a `request-id` into all log messages.
+- **FastAPI Integration**: Middleware for FastAPI to handle `request-id` for incoming HTTP requests.
+- **Celery Integration**: Signal handlers to propagate the `request-id` to Celery tasks.
+- **Requests Library Patching**: Automatically injects the `request-id` into outgoing HTTP requests made with the `requests` library.
+
+## Installation
+
+1.  Add the `litewave_logger` directory to your Python project.
+2.  Ensure that the dependencies listed in the main `requirements.txt` (`fastapi`, `celery`, `requests`) are installed.
+
+## Usage
+
+To use the `litewave_logger` in your service, follow these steps:
+
+1.  **Initialize the logger**: In your main application file (e.g., `api.py`), import and call the `setup_logging` function. This should be done as early as possible.
+
+    ```python
+    from litewave_logger import setup_logging
+
+    setup_logging()
+    ```
+
+2.  **Add the FastAPI middleware**: If your service is a FastAPI application, add the `RequestIdMiddleware` to your FastAPI app.
+
+    ```python
+    from fastapi import FastAPI
+    from litewave_logger.middleware import RequestIdMiddleware
+
+    app = FastAPI()
+    app.add_middleware(RequestIdMiddleware)
+    ```
+
+3.  **Patch the `requests` library**: To ensure the `request-id` is propagated to other services, patch the `requests` library.
+
+    ```python
+    from litewave_logger.requests import patch_requests
+
+    patch_requests()
+    ```
+
+4.  **Connect Celery signals**: If your service uses Celery, you need to import the Celery signal handlers to ensure they are registered. You don't need to call them directly.
+
+    ```python
+    from litewave_logger.celery import propagate_request_id_to_celery, clear_request_id_after_celery
+    ```
+
+### Example
+
+Here's a complete example of how to integrate the `litewave_logger` into a FastAPI application:
+
+```python
+from fastapi import FastAPI
+from litewave_logger import setup_logging
+from litewave_logger.middleware import RequestIdMiddleware
+from litewave_logger.requests import patch_requests
+
+# These imports are needed to register the signal handlers
+from litewave_logger.celery import propagate_request_id_to_celery, clear_request_id_after_celery
+
+# 1. Initialize logging
+setup_logging()
+
+# 2. Patch requests library
+patch_requests()
+
+app = FastAPI()
+
+# 3. Add RequestIdMiddleware
+app.add_middleware(RequestIdMiddleware)
+
+# Your application code here...
+```

litewave_logger-0.1.0/README.md

@@ -0,0 +1,79 @@
+# Litewave Logger
+
+This module provides a centralized and consistent logging solution for Litewave services. It ensures that a `request-id` is maintained across all services, including HTTP requests and Celery tasks, allowing for easy tracing of requests as they propagate through the system.
+
+## Features
+
+- **Centralized Logging**: A single module to configure and manage logging across all services.
+- **Request ID Propagation**: Automatically injects a `request-id` into all log messages.
+- **FastAPI Integration**: Middleware for FastAPI to handle `request-id` for incoming HTTP requests.
+- **Celery Integration**: Signal handlers to propagate the `request-id` to Celery tasks.
+- **Requests Library Patching**: Automatically injects the `request-id` into outgoing HTTP requests made with the `requests` library.
+
+## Installation
+
+1.  Add the `litewave_logger` directory to your Python project.
+2.  Ensure that the dependencies listed in the main `requirements.txt` (`fastapi`, `celery`, `requests`) are installed.
+
+## Usage
+
+To use the `litewave_logger` in your service, follow these steps:
+
+1.  **Initialize the logger**: In your main application file (e.g., `api.py`), import and call the `setup_logging` function. This should be done as early as possible.
+
+    ```python
+    from litewave_logger import setup_logging
+
+    setup_logging()
+    ```
+
+2.  **Add the FastAPI middleware**: If your service is a FastAPI application, add the `RequestIdMiddleware` to your FastAPI app.
+
+    ```python
+    from fastapi import FastAPI
+    from litewave_logger.middleware import RequestIdMiddleware
+
+    app = FastAPI()
+    app.add_middleware(RequestIdMiddleware)
+    ```
+
+3.  **Patch the `requests` library**: To ensure the `request-id` is propagated to other services, patch the `requests` library.
+
+    ```python
+    from litewave_logger.requests import patch_requests
+
+    patch_requests()
+    ```
+
+4.  **Connect Celery signals**: If your service uses Celery, you need to import the Celery signal handlers to ensure they are registered. You don't need to call them directly.
+
+    ```python
+    from litewave_logger.celery import propagate_request_id_to_celery, clear_request_id_after_celery
+    ```
+
+### Example
+
+Here's a complete example of how to integrate the `litewave_logger` into a FastAPI application:
+
+```python
+from fastapi import FastAPI
+from litewave_logger import setup_logging
+from litewave_logger.middleware import RequestIdMiddleware
+from litewave_logger.requests import patch_requests
+
+# These imports are needed to register the signal handlers
+from litewave_logger.celery import propagate_request_id_to_celery, clear_request_id_after_celery
+
+# 1. Initialize logging
+setup_logging()
+
+# 2. Patch requests library
+patch_requests()
+
+app = FastAPI()
+
+# 3. Add RequestIdMiddleware
+app.add_middleware(RequestIdMiddleware)
+
+# Your application code here...
+```

litewave_logger-0.1.0/litewave_logger/__init__.py

@@ -0,0 +1,45 @@
+import logging
+import uuid
+from contextvars import ContextVar
+
+# Context variable to hold the request ID
+# This is used to store the request ID in the context of the request
+# The value of the request ID is set in the RequestIdMiddleware
+# Magically, this is maintained across multiple requests without it getting mixed up.
+request_id_var = ContextVar("request_id", default=None)
+
+
+class RequestIdFilter(logging.Filter):
+    """
+    A logging filter that injects the request_id from the context variable into the log record.
+    """
+    def filter(self, record):
+        record.request_id = request_id_var.get()
+        return True
+
+
+def setup_logging():
+    """
+    Configures the logging for the application.
+    It sets up a handler, a formatter, and adds the RequestIdFilter.
+    """
+    logger = logging.getLogger()
+    logger.setLevel(logging.INFO)
+
+    # Prevent adding duplicate handlers
+    if not logger.handlers:
+        handler = logging.StreamHandler()
+        formatter = logging.Formatter(
+            "%(asctime)s - %(name)s - %(levelname)s - [%(request_id)s] - %(message)s"
+        )
+        handler.setFormatter(formatter)
+        logger.addHandler(handler)
+
+    # Add the filter to all handlers
+    for h in logger.handlers:
+        if not any(isinstance(f, RequestIdFilter) for f in h.filters):
+            h.addFilter(RequestIdFilter())
+
+    return logger
+
+__all__ = ["setup_logging", "request_id_var", "RequestIdFilter"]

litewave_logger-0.1.0/litewave_logger/celery.py

@@ -0,0 +1,86 @@
+import uuid
+from celery.signals import (
+    task_prerun,
+    task_postrun,
+    before_task_publish,
+    after_setup_task_logger
+)
+from . import request_id_var, RequestIdFilter
+import logging
+
+logger = logging.getLogger(__name__)
+
+# This signal runs in the CLIENT process (e.g., FastAPI app)
+# This is where we tap into the request ID.
+# Set the header for tasks to use the request ID.
+@before_task_publish.connect
+def propagate_request_id_to_celery_header(sender=None, headers=None, body=None, **kwargs):
+    """
+    Celery signal handler executed before a task is published.
+    It retrieves the request ID from the current context and injects it
+    into the task's headers. This propagates the ID from the client to the worker.
+    """
+    request_id = request_id_var.get()
+    if request_id:
+        if headers is None:
+            headers = {}
+        headers['request_id'] = request_id
+        logger.info(f"Injecting request_id {request_id} into Celery task headers.")
+
+# This signal runs in the WORKER process
+# check if header has request_id
+# this works magically because, we are importing the request_id_var from the same module in the worker process.
+@task_prerun.connect
+def set_request_id_from_celery_header(sender, task_id, task, args, kwargs, **extras):
+    """
+    Celery signal handler executed before a task runs in the worker.
+    It retrieves the request ID from the task's headers (if present)
+    and sets it in the worker's context variable.
+    """
+    request_id = task.request.get('request_id')
+    if request_id:
+        request_id_var.set(request_id)
+        logger.info(f"Set request_id {request_id} from Celery task {task.name} headers")
+    else:
+        # Generate a new one if it's a standalone task initiated not from a request
+        request_id_var.set(str(uuid.uuid4()))
+
+@after_setup_task_logger.connect
+def setup_celery_logging(logger, **kwargs):
+    """
+    This function is connected to the `after_setup_task_logger` signal.
+    It modifies the ROOT logger's handlers to ensure that all propagated
+    logs (including those from Celery tasks) are formatted correctly
+    with our custom request_id.
+    """
+    # Celery task logs propagate to the root logger. We need to modify
+    # the handlers on the root logger.
+    root_logger = logging.getLogger()
+
+    for handler in root_logger.handlers:
+        # Check if we've already configured this handler to avoid duplicates
+        if not any(isinstance(f, RequestIdFilter) for f in handler.filters):
+            # 1. Add the filter to inject the request_id
+            handler.addFilter(RequestIdFilter())
+            
+            # 2. Set our custom formatter to display the request_id
+            formatter = logging.Formatter(
+                "%(asctime)s - %(name)s - %(levelname)s - [%(request_id)s] - %(message)s"
+            )
+            handler.setFormatter(formatter)
+
+
+@task_postrun.connect
+def clear_request_id_after_celery(sender, task_id, task, args, kwargs, retval, state, **extras):
+    """
+    Celery signal handler executed after a task completes.
+    It clears the request ID from the context variable in the worker.
+    """
+    request_id_var.set(None)
+
+__all__ = [
+    "propagate_request_id_to_celery_header",
+    "set_request_id_from_celery_header",
+    "clear_request_id_after_celery",
+    "setup_celery_logging" # Expose the new function
+]

litewave_logger-0.1.0/litewave_logger/middleware.py

@@ -0,0 +1,31 @@
+import uuid
+from starlette.middleware.base import BaseHTTPMiddleware
+from starlette.requests import Request
+
+from . import request_id_var
+
+# This middleware is used to handle the request ID.
+# It checks for a 'X-Request-ID' header in the incoming request.
+# If the header is present, its value is used as the request ID.
+# If not, a new UUID is generated.
+# The request ID is then stored in a context variable.
+class RequestIdMiddleware(BaseHTTPMiddleware):
+    """
+    FastAPI middleware to handle the request ID.
+    It checks for a 'X-Request-ID' header in the incoming request.
+    If the header is present, its value is used as the request ID.
+    If not, a new UUID is generated.
+    The request ID is then stored in a context variable.
+    """
+    async def dispatch(self, request: Request, call_next):
+        request_id = request.headers.get("X-Request-ID")
+        if not request_id:
+            request_id = str(uuid.uuid4())
+        
+        request_id_var.set(request_id)
+        
+        response = await call_next(request)
+        response.headers["X-Request-ID"] = request_id_var.get()
+        return response
+
+__all__ = ["RequestIdMiddleware"]

litewave_logger-0.1.0/litewave_logger/requests.py

@@ -0,0 +1,32 @@
+import requests
+from . import request_id_var
+import logging
+
+logger = logging.getLogger(__name__)
+
+# This is used to patch the requests library to automatically inject the 'X-Request-ID' header
+# into all outgoing requests.
+# It is used to ensure that the request ID is propagated to the external services.
+def patch_requests():
+    """
+    Patches the requests library to automatically inject the 'X-Request-ID' header
+    into all outgoing requests.
+    """
+    # Get the original Session class
+    original_session_class = requests.Session
+
+    # Create a new Session class that inherits from the original
+    class PatchedSession(original_session_class):
+        def request(self, method, url, *args, **kwargs):
+            request_id = request_id_var.get()
+            if request_id:
+                headers = kwargs.get("headers", {})
+                headers["X-Request-ID"] = request_id
+                kwargs["headers"] = headers
+                logger.debug(f"Injecting request_id {request_id} into outgoing request to {url}")
+            return super().request(method, url, *args, **kwargs)
+
+    # Replace the original Session class with our patched version
+    requests.Session = PatchedSession
+
+__all__ = ["patch_requests"]

litewave_logger-0.1.0/litewave_logger.egg-info/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.4
+Name: litewave_logger
+Version: 0.1.0
+Summary: A lightweight logging library with context carry forward
+Author: Litewave
+Author-email: Sonu Sudhakaran <sonu@litewave.ai>
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+Requires-Dist: requests
+Requires-Dist: fastapi
+Dynamic: author
+Dynamic: requires-python
+
+# Litewave Logger
+
+This module provides a centralized and consistent logging solution for Litewave services. It ensures that a `request-id` is maintained across all services, including HTTP requests and Celery tasks, allowing for easy tracing of requests as they propagate through the system.
+
+## Features
+
+- **Centralized Logging**: A single module to configure and manage logging across all services.
+- **Request ID Propagation**: Automatically injects a `request-id` into all log messages.
+- **FastAPI Integration**: Middleware for FastAPI to handle `request-id` for incoming HTTP requests.
+- **Celery Integration**: Signal handlers to propagate the `request-id` to Celery tasks.
+- **Requests Library Patching**: Automatically injects the `request-id` into outgoing HTTP requests made with the `requests` library.
+
+## Installation
+
+1.  Add the `litewave_logger` directory to your Python project.
+2.  Ensure that the dependencies listed in the main `requirements.txt` (`fastapi`, `celery`, `requests`) are installed.
+
+## Usage
+
+To use the `litewave_logger` in your service, follow these steps:
+
+1.  **Initialize the logger**: In your main application file (e.g., `api.py`), import and call the `setup_logging` function. This should be done as early as possible.
+
+    ```python
+    from litewave_logger import setup_logging
+
+    setup_logging()
+    ```
+
+2.  **Add the FastAPI middleware**: If your service is a FastAPI application, add the `RequestIdMiddleware` to your FastAPI app.
+
+    ```python
+    from fastapi import FastAPI
+    from litewave_logger.middleware import RequestIdMiddleware
+
+    app = FastAPI()
+    app.add_middleware(RequestIdMiddleware)
+    ```
+
+3.  **Patch the `requests` library**: To ensure the `request-id` is propagated to other services, patch the `requests` library.
+
+    ```python
+    from litewave_logger.requests import patch_requests
+
+    patch_requests()
+    ```
+
+4.  **Connect Celery signals**: If your service uses Celery, you need to import the Celery signal handlers to ensure they are registered. You don't need to call them directly.
+
+    ```python
+    from litewave_logger.celery import propagate_request_id_to_celery, clear_request_id_after_celery
+    ```
+
+### Example
+
+Here's a complete example of how to integrate the `litewave_logger` into a FastAPI application:
+
+```python
+from fastapi import FastAPI
+from litewave_logger import setup_logging
+from litewave_logger.middleware import RequestIdMiddleware
+from litewave_logger.requests import patch_requests
+
+# These imports are needed to register the signal handlers
+from litewave_logger.celery import propagate_request_id_to_celery, clear_request_id_after_celery
+
+# 1. Initialize logging
+setup_logging()
+
+# 2. Patch requests library
+patch_requests()
+
+app = FastAPI()
+
+# 3. Add RequestIdMiddleware
+app.add_middleware(RequestIdMiddleware)
+
+# Your application code here...
+```

litewave_logger-0.1.0/litewave_logger.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+README.md
+pyproject.toml
+setup.py
+litewave_logger/__init__.py
+litewave_logger/celery.py
+litewave_logger/middleware.py
+litewave_logger/requests.py
+litewave_logger.egg-info/PKG-INFO
+litewave_logger.egg-info/SOURCES.txt
+litewave_logger.egg-info/dependency_links.txt
+litewave_logger.egg-info/requires.txt
+litewave_logger.egg-info/top_level.txt

litewave_logger-0.1.0/litewave_logger.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

litewave_logger-0.1.0/litewave_logger.egg-info/requires.txt

@@ -0,0 +1,2 @@
+requests
+fastapi

litewave_logger-0.1.0/litewave_logger.egg-info/top_level.txt

@@ -0,0 +1 @@
+litewave_logger

litewave_logger-0.1.0/pyproject.toml

@@ -0,0 +1,14 @@
+[project]
+name = "litewave_logger"
+version = "0.1.0"
+description = "A lightweight logging library with context carry forward"
+authors = [{name = "Sonu Sudhakaran", email = "sonu@litewave.ai"}]
+dependencies = [
+    "requests", "fastapi"
+]
+requires-python = ">=3.7"
+readme = "README.md"
+
+[build-system]
+requires = ["setuptools", "wheel"]
+build-backend = "setuptools.build_meta"

litewave_logger-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

litewave_logger-0.1.0/setup.py

@@ -0,0 +1,21 @@
+from setuptools import setup, find_packages
+
+setup(
+    name="litewave_logger",
+    version="0.1.0",
+    description="A centralized logging module for Litewave services.",
+    author="Litewave",
+    packages=find_packages(),
+    install_requires=[
+        "fastapi",
+        "starlette",
+        "celery",
+        "requests",
+    ],
+    classifiers=[
+        "Programming Language :: Python :: 3",
+        "License :: OSI Approved :: MIT License",
+        "Operating System :: OS Independent",
+    ],
+    python_requires='>=3.6',
+)