djlogq 1.0.0__py3-none-any.whl

djlogq-1.0.0.dist-info/METADATA

@@ -0,0 +1,242 @@
+Metadata-Version: 2.4
+Name: djlogq
+Version: 1.0.0
+Summary: A reusable Django app for asynchronous, thread-safe logging with rich metadata, admin interface, and API support.
+Author-email: mesake <mesnavunawa@gmail.com>
+License: MIT
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: Django
+Provides-Extra: dev
+Requires-Dist: build==1.2.1; extra == "dev"
+Requires-Dist: nox==2024.4.15; extra == "dev"
+Requires-Dist: twine==5.1.1; extra == "dev"
+
+# Django Async Logger
+
+A reusable Django app that provides asynchronous logging functionality using a separate thread to avoid blocking the main application.
+
+## Features
+
+- **Asynchronous Logging**: All log operations run in a separate thread
+- **Thread-Safe**: Uses a queue system for thread-safe logging
+- **Rich Metadata**: Captures module, function, line number, user ID, request ID, and extra data
+- **Admin Interface**: Beautiful Django admin interface for viewing and managing logs
+- **API Endpoints**: REST API for external logging
+- **Middleware**: Automatic request logging with unique request IDs
+- **Decorators**: Utility decorators for function logging and performance monitoring
+- **Context Managers**: Easy-to-use context managers for operation logging
+- **Configurable**: Customizable queue size, flush intervals, and cleanup policies
+
+## Installation
+
+1. Add the app to your Django project:
+```python
+INSTALLED_APPS = [
+    # ...
+    'logq',
+]
+```
+
+2. Add the middleware to your settings:
+```python
+MIDDLEWARE = [
+    # ...
+    'logq.middleware.AsyncLoggingMiddleware',
+]
+```
+
+3. Run migrations:
+```bash
+python manage.py makemigrations logq
+python manage.py migrate
+```
+
+4. (Optional) Configure logging settings:
+```python
+ASYNC_LOGGING_CONFIG = {
+    'MAX_QUEUE_SIZE': 1000,
+    'FLUSH_INTERVAL': 1.0,  # seconds
+    'AUTO_CLEANUP_DAYS': 30,
+    'ENABLE_REQUEST_LOGGING': True,
+    'IGNORE_PATHS': ['/admin/'],  # paths to ignore for request logging
+}
+```
+
+## Usage
+
+### Basic Logging
+
+```python
+from logq.async_logger import get_async_logger
+
+logger = get_async_logger()
+
+# Different log levels
+logger.debug("Debug message")
+logger.info("Info message")
+logger.warning("Warning message")
+logger.error("Error message")
+logger.critical("Critical message")
+
+# With extra data
+logger.info("User action", extra_data={'action': 'login', 'ip': '192.168.1.1'})
+
+# Log exceptions
+try:
+    # some code that might fail
+    pass
+except Exception as e:
+    logger.exception("An error occurred", exc_info=str(e))
+```
+
+### Function Decorators
+
+```python
+from logq.utils import log_function_call, log_performance
+
+@log_function_call
+def my_function():
+    return "result"
+
+@log_function_call(level='DEBUG')
+def debug_function():
+    return "debug result"
+
+@log_performance(threshold_seconds=0.5)
+def slow_function():
+    time.sleep(1)
+    return "slow result"
+```
+
+### Context Managers
+
+```python
+from logq.utils import LogContext
+
+with LogContext("Processing data", level='INFO'):
+    # do some work
+    time.sleep(0.1)
+    # automatically logs start and completion with timing
+```
+
+### API Logging
+
+```python
+import requests
+import json
+
+# Log via API
+data = {
+    'level': 'INFO',
+    'message': 'External log message',
+    'extra_data': {'source': 'external_service'}
+}
+
+response = requests.post(
+    'http://your-domain/logq/api/log/',
+    data=json.dumps(data),
+    headers={'Content-Type': 'application/json'}
+)
+
+# Retrieve logs via API
+response = requests.get('http://your-domain/logq/api/logs/?limit=10')
+logs = response.json()['logs']
+```
+
+### Admin Interface
+
+Access the admin interface at `/admin/` to view and manage logs. Features include:
+
+- Filter by level, module, timestamp, user ID
+- Search by message, module, function, request ID
+- View extra data in formatted JSON
+- Delete old logs
+- Export functionality
+
+### Management Commands
+
+Clean old logs:
+```bash
+# Delete logs older than 30 days
+python manage.py clean_logs
+
+# Delete logs older than 7 days
+python manage.py clean_logs --days 7
+
+# Delete only DEBUG and INFO logs older than 30 days
+python manage.py clean_logs --level INFO
+
+# Dry run to see what would be deleted
+python manage.py clean_logs --dry-run
+```
+
+## Configuration Options
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `MAX_QUEUE_SIZE` | 1000 | Maximum number of log entries in the queue |
+| `FLUSH_INTERVAL` | 1.0 | How often to flush logs to database (seconds) |
+| `AUTO_CLEANUP_DAYS` | 30 | Days to keep logs before auto-cleanup |
+| `ENABLE_REQUEST_LOGGING` | True | Whether to log all HTTP requests |
+
+## Model Fields
+
+The `LogEntry` model includes:
+
+- `timestamp`: When the log was created
+- `level`: Log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+- `message`: The log message
+- `module`: Python module where the log originated
+- `function`: Function name where the log originated
+- `line_number`: Line number where the log originated
+- `user_id`: ID of the user (if authenticated)
+- `request_id`: Unique request identifier
+- `extra_data`: Additional JSON data
+- `created_at`: When the entry was saved to database
+
+## Performance Considerations
+
+- The logger runs in a separate thread and won't block your main application
+- Log entries are batched and written to the database periodically
+- If the queue is full, new entries are dropped (with console fallback)
+- Consider setting up database indexes for better query performance
+- Use the cleanup command regularly to prevent database bloat
+
+## Thread Safety
+
+The logger is completely thread-safe:
+- Uses a thread-safe queue for communication
+- Database operations are wrapped in transactions
+- Multiple threads can safely call the logger simultaneously
+
+## Customization
+
+You can extend the logger by:
+
+1. Creating custom log levels
+2. Adding new fields to the LogEntry model
+3. Customizing the admin interface
+4. Adding new API endpoints
+5. Creating custom middleware
+
+## Troubleshooting
+
+### Logs not appearing
+- Check that the async logger thread is running
+- Verify database migrations are applied
+- Check for any database connection issues
+
+### Performance issues
+- Reduce `FLUSH_INTERVAL` for more frequent writes
+- Increase `MAX_QUEUE_SIZE` for higher throughput
+- Add database indexes for frequently queried fields
+
+### Memory usage
+- Reduce `MAX_QUEUE_SIZE` if memory is a concern
+- Run cleanup commands more frequently
+- Monitor database size and clean old logs
+
+## License
+
+This project is open source and available under the MIT License. 

djlogq-1.0.0.dist-info/RECORD

@@ -0,0 +1,20 @@
+logq/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+logq/admin.py,sha256=nC8TVXc64G41Mh05ijUSCJmvIDo-kgZHKyyOlmqH-AE,320
+logq/apps.py,sha256=uTCJNk3UWtyAZOpg7nRX0owKI-ts6BtPHARrU7PytQA,346
+logq/async_logger.py,sha256=I8za2lfEhiGrKDq76oZ4DZvvAlXeiRr17bmXT5tXSY4,5216
+logq/middleware.py,sha256=Y9EAnxrmavyDgVklEW893Wh0eeQPKdCvizcxGzPDLBo,3642
+logq/models.py,sha256=Gu8KLNMn7I4XtEUHQPavSSlcESQ-2Pu5JLKsL97t9Gg,1616
+logq/tests.py,sha256=iimhG7nvH5SbFn1og6soBMvCWhxCsuvgwkfaPOr5PC4,11102
+logq/urls.py,sha256=oGwVM9HXsVY_P86sVPuz5xnFofYfmkL8ZSZDhExhJQk,216
+logq/utils.py,sha256=Bg973EhnvQXvFSk1mtf8pcpQqnHiOnQRdaCLM9ibJPw,4204
+logq/views.py,sha256=WJpwiPyfItBbceM1862NcXp_ui4U6WyNBhw2P27mlJ4,3695
+logq/management/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+logq/management/commands/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+logq/management/commands/clean_logs.py,sha256=Cc33EEqGGwsNcvQjwnLbM6kIb0lBJsyDQliia7EEAZo,2416
+logq/migrations/0001_initial.py,sha256=l4f-lUcO7OsABGYiSBp7fdWDt2rLHaIhR78pCKIAAdQ,2007
+logq/migrations/0002_alter_logentry_function_alter_logentry_line_number_and_more.py,sha256=SNBngZmhk9BgcOe8eAojX47V5JKC2V7oW9QtLHWIkFc,750
+logq/migrations/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+djlogq-1.0.0.dist-info/METADATA,sha256=E1B27SP2iNCWCG1UkcpldNH1fhuAItNbRUwyk8JpTV8,6453
+djlogq-1.0.0.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+djlogq-1.0.0.dist-info/top_level.txt,sha256=VWj_EO93x0ix2Y2qm6obWT22L7VPFiZ7lQf0yIaI8do,5
+djlogq-1.0.0.dist-info/RECORD,,

djlogq-1.0.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (80.9.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

djlogq-1.0.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+logq

logq/admin.py

@@ -0,0 +1,12 @@
+from django.contrib import admin
+
+from .models import LogEntry
+
+
+@admin.register(LogEntry)
+class LogEntryAdmin(admin.ModelAdmin):
+    list_display = ('timestamp', 'level', 'message')
+    list_filter = ('level', 'timestamp')
+    search_fields = ('message',)
+    date_hierarchy = 'timestamp'
+    ordering = ('-timestamp',)

logq/apps.py

@@ -0,0 +1,12 @@
+from django.apps import AppConfig
+
+
+class LogqConfig(AppConfig):
+    default_auto_field = 'django.db.models.BigAutoField'
+    name = 'logq'
+    verbose_name = 'Asynchronous Logging'
+
+    def ready(self):
+        """Initialize the async logger when the app is ready."""
+        from .async_logger import get_async_logger
+        get_async_logger()

logq/async_logger.py

@@ -0,0 +1,160 @@
+import threading
+import queue
+import time
+import logging
+import traceback
+import inspect
+from typing import Optional, Dict, Any
+from django.utils import timezone
+from django.db import transaction
+from django.conf import settings
+from .models import LogEntry, LogLevel
+
+
+class AsyncLogger:
+    """
+    Asynchronous logger that runs in a separate thread to avoid blocking the main application.
+    """
+    
+    def __init__(self, max_queue_size: int = None, flush_interval: float = None):
+        # Get configuration from settings
+        config = getattr(settings, 'ASYNC_LOGGING_CONFIG', {})
+        self.max_queue_size = max_queue_size or config.get('MAX_QUEUE_SIZE', 1000)
+        self.flush_interval = flush_interval or config.get('FLUSH_INTERVAL', 1.0)
+        
+        self.queue = queue.Queue(maxsize=self.max_queue_size)
+        self.running = False
+        self.thread = None
+        self._lock = threading.Lock()
+        
+    def start(self):
+        """Start the logging thread."""
+        with self._lock:
+            if not self.running:
+                self.running = True
+                self.thread = threading.Thread(target=self._worker, daemon=True)
+                self.thread.start()
+    
+    def stop(self):
+        """Stop the logging thread."""
+        with self._lock:
+            if self.running:
+                self.running = False
+                if self.thread:
+                    self.thread.join(timeout=5.0)
+    
+    def _worker(self):
+        """Worker thread that processes log entries from the queue."""
+        batch = []
+        last_flush = time.time()
+        
+        while self.running:
+            try:
+                # Try to get a log entry with timeout
+                try:
+                    entry = self.queue.get(timeout=0.1)
+                    batch.append(entry)
+                except queue.Empty:
+                    pass
+                
+                # Flush batch if it's time or batch is getting large
+                current_time = time.time()
+                if (current_time - last_flush >= self.flush_interval or 
+                    len(batch) >= 50):
+                    if batch:
+                        self._flush_batch(batch)
+                        batch = []
+                        last_flush = current_time
+                        
+            except Exception as e:
+                # Log the error to prevent infinite loops
+                print(f"Error in async logger worker: {e}")
+                time.sleep(1)
+        
+        # Flush remaining entries
+        if batch:
+            self._flush_batch(batch)
+    
+    def _flush_batch(self, batch):
+        """Flush a batch of log entries to the database."""
+        try:
+            with transaction.atomic():
+                LogEntry.objects.bulk_create(batch, ignore_conflicts=True)
+        except Exception as e:
+            print(f"Error flushing log batch: {e}")
+    
+    def log(self, level: str, message: str, **kwargs):
+        """Add a log entry to the queue."""
+        if not self.running:
+            return
+        
+        # Get caller information
+        frame = inspect.currentframe().f_back
+        module = frame.f_globals.get('__name__', 'unknown')
+        function = frame.f_code.co_name
+        line_number = frame.f_lineno
+        
+        # Create log entry
+        entry = LogEntry(
+            level=level,
+            message=message,
+            module=module,
+            function=function,
+            line_number=line_number,
+            user_id=kwargs.get('user_id'),
+            request_id=kwargs.get('request_id'),
+            extra_data=kwargs.get('extra_data', {})
+        )
+        
+        try:
+            self.queue.put_nowait(entry)
+        except queue.Full:
+            # If queue is full, log to console as fallback
+            print(f"Log queue full, dropping entry: [{level}] {message}")
+    
+    def debug(self, message: str, **kwargs):
+        self.log(LogLevel.DEBUG, message, **kwargs)
+    
+    def info(self, message: str, **kwargs):
+        self.log(LogLevel.INFO, message, **kwargs)
+    
+    def warning(self, message: str, **kwargs):
+        self.log(LogLevel.WARNING, message, **kwargs)
+    
+    def error(self, message: str, **kwargs):
+        self.log(LogLevel.ERROR, message, **kwargs)
+    
+    def critical(self, message: str, **kwargs):
+        self.log(LogLevel.CRITICAL, message, **kwargs)
+    
+    def exception(self, message: str, exc_info=None, **kwargs):
+        """Log an exception with traceback."""
+        if exc_info is None:
+            exc_info = traceback.format_exc()
+        
+        extra_data = kwargs.get('extra_data', {})
+        extra_data['traceback'] = exc_info
+        
+        self.log(LogLevel.ERROR, message, extra_data=extra_data, **kwargs)
+
+
+# Global logger instance
+_async_logger = None
+
+
+def get_async_logger() -> AsyncLogger:
+    """Get the global async logger instance."""
+    global _async_logger
+    if _async_logger is None:
+        _async_logger = AsyncLogger()
+        _async_logger.start()
+    return _async_logger
+
+
+def stop_async_logger():
+    """Stop the global async logger."""
+    global _async_logger
+    if _async_logger:
+        _async_logger.stop()
+        _async_logger = None
+

logq/management/commands/clean_logs.py

@@ -0,0 +1,69 @@
+
+from django.core.management.base import BaseCommand
+from django.utils import timezone
+from datetime import timedelta
+from django.conf import settings
+from logq.models import LogEntry
+
+
+class Command(BaseCommand):
+    help = 'Clean old log entries from the database'
+    
+    def add_arguments(self, parser):
+        parser.add_argument(
+            '--days',
+            type=int,
+            default=None,
+            help='Delete logs older than this many days (default: from ASYNC_LOGGING_CONFIG)'
+        )
+        parser.add_argument(
+            '--level',
+            type=str,
+            choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'],
+            help='Only delete logs of this level or lower'
+        )
+        parser.add_argument(
+            '--dry-run',
+            action='store_true',
+            help='Show what would be deleted without actually deleting'
+        )
+    
+    def handle(self, *args, **options):
+        # Get default days from settings
+        config = getattr(settings, 'ASYNC_LOGGING_CONFIG', {})
+        default_days = config.get('AUTO_CLEANUP_DAYS', 30)
+        
+        days = options['days'] if options['days'] is not None else default_days
+        level = options['level']
+        dry_run = options['dry_run']
+        
+        # Calculate cutoff date
+        cutoff_date = timezone.now() - timedelta(days=days)
+        
+        # Build query
+        query = LogEntry.objects.filter(timestamp__lt=cutoff_date)
+        
+        if level:
+            # Get level hierarchy
+            level_order = ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']
+            level_index = level_order.index(level)
+            levels_to_delete = level_order[:level_index + 1]
+            query = query.filter(level__in=levels_to_delete)
+        
+        count = query.count()
+        
+        if dry_run:
+            self.stdout.write(
+                self.style.WARNING(
+                    f'Would delete {count} log entries older than {days} days'
+                    + (f' with level {level} or lower' if level else '')
+                )
+            )
+        else:
+            deleted_count = query.delete()[0]
+            self.stdout.write(
+                self.style.SUCCESS(
+                    f'Successfully deleted {deleted_count} log entries older than {days} days'
+                    + (f' with level {level} or lower' if level else '')
+                )
+            )

logq/middleware.py

@@ -0,0 +1,96 @@
+import uuid
+from django.utils.deprecation import MiddlewareMixin
+from .async_logger import get_async_logger
+from django.conf import settings
+
+class AsyncLoggingMiddleware(MiddlewareMixin):
+    """
+    Middleware that automatically logs request information and adds request_id to the request.
+    """
+    
+    def process_request(self, request):
+        # Check if request path is in ignore paths
+        ignore_paths = getattr(settings, 'ASYNC_LOGGING_CONFIG', {}).get('IGNORE_PATHS', [])
+        if any(request.path.startswith(path) for path in ignore_paths):
+            return None
+        
+        # Generate unique request ID
+        request.request_id = str(uuid.uuid4())
+        
+        # Get user ID if authenticated
+        user_id = None
+        if hasattr(request, 'user') and request.user.is_authenticated:
+            user_id = request.user.id
+        
+        # Log request start
+        logger = get_async_logger()
+        logger.info(
+            f"Request started: {request.method} {request.path}",
+            request_id=request.request_id,
+            user_id=user_id,
+            extra_data={
+                'method': request.method,
+                'path': request.path,
+                'query_params': dict(request.GET),
+                'user_agent': request.META.get('HTTP_USER_AGENT', ''),
+                'ip_address': self._get_client_ip(request),
+            }
+        )
+    
+    def process_response(self, request, response):
+        # Check if request path is in ignore paths
+        ignore_paths = getattr(settings, 'ASYNC_LOGGING_CONFIG', {}).get('IGNORE_PATHS', [])
+        if any(request.path.startswith(path) for path in ignore_paths):
+            return response
+        
+        # Get user ID if authenticated
+        user_id = None
+        if hasattr(request, 'user') and request.user.is_authenticated:
+            user_id = request.user.id
+        
+        # Log request completion
+        logger = get_async_logger()
+        logger.info(
+            f"Request completed: {request.method} {request.path} - {response.status_code}",
+            request_id=getattr(request, 'request_id', 'unknown'),
+            user_id=user_id,
+            extra_data={
+                'status_code': response.status_code,
+                'content_length': len(response.content) if hasattr(response, 'content') else 0,
+            }
+        )
+        
+        return response
+    
+    def process_exception(self, request, exception):
+        # Check if request path is in ignore paths
+        ignore_paths = getattr(settings, 'ASYNC_LOGGING_CONFIG', {}).get('IGNORE_PATHS', [])
+        if any(request.path.startswith(path) for path in ignore_paths):
+            return None
+        
+        # Get user ID if authenticated
+        user_id = None
+        if hasattr(request, 'user') and request.user.is_authenticated:
+            user_id = request.user.id
+        
+        # Log exception
+        logger = get_async_logger()
+        logger.exception(
+            f"Request exception: {request.method} {request.path}",
+            exc_info=str(exception),
+            request_id=getattr(request, 'request_id', 'unknown'),
+            user_id=user_id,
+            extra_data={
+                'exception_type': type(exception).__name__,
+                'exception_args': str(exception.args),
+            }
+        )
+    
+    def _get_client_ip(self, request):
+        """Get the client's IP address."""
+        x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
+        if x_forwarded_for:
+            ip = x_forwarded_for.split(',')[0]
+        else:
+            ip = request.META.get('REMOTE_ADDR')
+        return ip

logq/migrations/0001_initial.py

@@ -0,0 +1,37 @@
+# Generated by Django 5.2.4 on 2025-07-15 20:43
+
+import django.utils.timezone
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='LogEntry',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('timestamp', models.DateTimeField(db_index=True, default=django.utils.timezone.now)),
+                ('level', models.CharField(choices=[('DEBUG', 'Debug'), ('INFO', 'Info'), ('WARNING', 'Warning'), ('ERROR', 'Error'), ('CRITICAL', 'Critical')], max_length=10)),
+                ('message', models.TextField()),
+                ('module', models.CharField(max_length=255)),
+                ('function', models.CharField(max_length=255)),
+                ('line_number', models.IntegerField()),
+                ('user_id', models.IntegerField(blank=True, null=True)),
+                ('request_id', models.CharField(blank=True, max_length=255, null=True)),
+                ('extra_data', models.JSONField(blank=True, default=dict)),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+            ],
+            options={
+                'verbose_name': 'Log Entry',
+                'verbose_name_plural': 'Log Entries',
+                'ordering': ['-timestamp'],
+                'indexes': [models.Index(fields=['timestamp'], name='logq_logent_timesta_9e2e5a_idx'), models.Index(fields=['level'], name='logq_logent_level_fa122c_idx'), models.Index(fields=['module'], name='logq_logent_module_f632bb_idx'), models.Index(fields=['function'], name='logq_logent_functio_433e61_idx'), models.Index(fields=['line_number'], name='logq_logent_line_nu_bd9370_idx'), models.Index(fields=['user_id'], name='logq_logent_user_id_651abc_idx'), models.Index(fields=['request_id'], name='logq_logent_request_eda96e_idx')],
+            },
+        ),
+    ]

logq/migrations/0002_alter_logentry_function_alter_logentry_line_number_and_more.py

@@ -0,0 +1,28 @@
+# Generated by Django 5.2.4 on 2025-07-15 21:02
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('logq', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='logentry',
+            name='function',
+            field=models.CharField(blank=True, max_length=255, null=True),
+        ),
+        migrations.AlterField(
+            model_name='logentry',
+            name='line_number',
+            field=models.IntegerField(blank=True, null=True),
+        ),
+        migrations.AlterField(
+            model_name='logentry',
+            name='module',
+            field=models.CharField(blank=True, max_length=255, null=True),
+        ),
+    ]

logq/models.py

@@ -0,0 +1,44 @@
+from django.db import models
+
+from django.utils import timezone
+import json
+
+
+class LogLevel(models.TextChoices):
+    DEBUG = 'DEBUG'
+    INFO = 'INFO'
+    WARNING = 'WARNING'
+    ERROR = 'ERROR'
+    CRITICAL = 'CRITICAL'
+
+class LogEntry(models.Model):
+    timestamp = models.DateTimeField(default=timezone.now, db_index=True)  # index for faster queries
+    level = models.CharField(max_length=10, choices=LogLevel.choices)
+    message = models.TextField()
+    module = models.CharField(max_length=255, null=True, blank=True)
+    function = models.CharField(max_length=255, null=True, blank=True)
+    line_number = models.IntegerField(null=True, blank=True)
+    user_id = models.IntegerField(null=True, blank=True)
+    request_id = models.CharField(max_length=255, null=True, blank=True)
+    extra_data = models.JSONField(default=dict, blank=True)
+    created_at = models.DateTimeField(auto_now_add=True)
+
+
+    class Meta:
+        ordering = ['-timestamp']
+        indexes = [  # index for faster queries
+            models.Index(fields=['timestamp']),
+            models.Index(fields=['level']),
+            models.Index(fields=['module']),
+            models.Index(fields=['function']),
+            models.Index(fields=['line_number']),
+            models.Index(fields=['user_id']),
+            models.Index(fields=['request_id']),
+        ]
+        verbose_name = 'Log Entry'
+        verbose_name_plural = 'Log Entries'
+
+    def __str__(self):
+        """Return a string representation of the log entry."""
+        return f"[{self.level}] {self.timestamp.strftime('%Y-%m-%d %H:%M:%S')} - {self.message[:100]}"
+  

logq/tests.py

@@ -0,0 +1,317 @@
+from django.test import TransactionTestCase, override_settings
+from django.urls import reverse
+from django.contrib.auth.models import User
+from django.utils import timezone
+from django.db import connection
+import json
+import time
+import threading
+from .models import LogEntry, LogLevel
+from .async_logger import AsyncLogger, get_async_logger, stop_async_logger
+
+
+class AsyncLoggerTestCase(TransactionTestCase):
+    def setUp(self):
+        super().setUp()
+        # Stop the global logger to avoid interference
+        stop_async_logger()
+        
+        # Clear all existing logs using raw SQL to ensure complete cleanup
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+        
+        # Create a fresh logger instance for testing
+        self.logger = AsyncLogger(max_queue_size=100, flush_interval=0.1)
+        self.logger.start()
+        time.sleep(0.2)  # Wait for thread to start
+    
+    def tearDown(self):
+        self.logger.stop()
+        time.sleep(0.2)  # Wait for thread to stop
+        
+        # Clear logs after test using raw SQL
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+        
+        super().tearDown()
+    
+    def test_basic_logging(self):
+        """Test basic logging functionality."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        self.logger.info("Test message")
+        time.sleep(0.5)  # Wait longer for flush
+        
+        # Verify we have exactly one log entry
+        self.assertEqual(LogEntry.objects.count(), 1)
+        
+        log_entry = LogEntry.objects.first()
+        self.assertEqual(log_entry.level, LogLevel.INFO)
+        self.assertEqual(log_entry.message, "Test message")
+    
+    def test_all_log_levels(self):
+        """Test all log levels."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        levels = [LogLevel.DEBUG, LogLevel.INFO, LogLevel.WARNING, LogLevel.ERROR, LogLevel.CRITICAL]
+        
+        for level in levels:
+            self.logger.log(level, f"Test {level}")
+        
+        time.sleep(0.5)  # Wait longer for flush
+        
+        entries = LogEntry.objects.all()
+        self.assertEqual(entries.count(), len(levels))
+        
+        for entry in entries:
+            self.assertIn(entry.level, levels)
+    
+    def test_extra_data(self):
+        """Test logging with extra data."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        extra_data = {'user_id': 123, 'action': 'test'}
+        self.logger.info("Test with extra data", extra_data=extra_data)
+        time.sleep(0.5)
+        
+        # Verify we have exactly one log entry
+        self.assertEqual(LogEntry.objects.count(), 1)
+        
+        entry = LogEntry.objects.first()
+        self.assertEqual(entry.extra_data, extra_data)
+    
+    def test_queue_full_handling(self):
+        """Test behavior when queue is full."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        # Fill the queue
+        for i in range(150):  # More than max_queue_size
+            self.logger.info(f"Message {i}")
+        
+        time.sleep(0.5)
+        
+        # Should have some entries but not all due to queue being full
+        entries = LogEntry.objects.count()
+        self.assertGreater(entries, 0)
+        self.assertLessEqual(entries, 100)  # max_queue_size
+
+
+class LogEntryModelTestCase(TransactionTestCase):
+    def setUp(self):
+        super().setUp()
+        # Clear all existing logs
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+    
+    def tearDown(self):
+        # Clear logs after test
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+        super().tearDown()
+    
+    def test_log_entry_creation(self):
+        """Test LogEntry model creation."""
+        entry = LogEntry.objects.create(
+            level=LogLevel.INFO,
+            message="Test message",
+            module="test_module",
+            function="test_function",
+            line_number=42,
+            user_id=123,
+            request_id="test-request-id",
+            extra_data={'key': 'value'}
+        )
+        
+        self.assertEqual(entry.level, LogLevel.INFO)
+        self.assertEqual(entry.message, "Test message")
+        self.assertEqual(entry.module, "test_module")
+        self.assertEqual(entry.function, "test_function")
+        self.assertEqual(entry.line_number, 42)
+        self.assertEqual(entry.user_id, 123)
+        self.assertEqual(entry.request_id, "test-request-id")
+        self.assertEqual(entry.extra_data, {'key': 'value'})
+    
+    def test_log_entry_str_representation(self):
+        """Test string representation of LogEntry."""
+        entry = LogEntry.objects.create(
+            level=LogLevel.ERROR,
+            message="This is a very long message that should be truncated in the string representation",
+            timestamp=timezone.now()
+        )
+        
+        str_repr = str(entry)
+        self.assertIn("[ERROR]", str_repr)
+        self.assertIn("This is a very long message that should be truncated", str_repr[:100])
+
+
+class LoggingAPITestCase(TransactionTestCase):
+    def setUp(self):
+        super().setUp()
+        self.user = User.objects.create_user(username='testuser', password='testpass')
+        # Stop the global logger to avoid interference
+        stop_async_logger()
+        # Clear all existing logs
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+    
+    def tearDown(self):
+        # Clear logs after test
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+        super().tearDown()
+    
+    def test_log_endpoint(self):
+        """Test the log API endpoint."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        data = {
+            'level': 'INFO',
+            'message': 'Test API log',
+            'extra_data': {'source': 'api'}
+        }
+        
+        response = self.client.post(
+            reverse('logq:log_endpoint'),
+            data=json.dumps(data),
+            content_type='application/json'
+        )
+        
+        self.assertEqual(response.status_code, 200)
+        self.assertEqual(response.json()['status'], 'success')
+        
+        # Wait for async processing
+        time.sleep(0.5)
+        
+        # Verify we have exactly one log entry
+        self.assertEqual(LogEntry.objects.count(), 1)
+        
+        entry = LogEntry.objects.first()
+        self.assertEqual(entry.message, 'Test API log')
+        self.assertEqual(entry.extra_data, {'source': 'api'})
+    
+    def test_log_api_view(self):
+        """Test the class-based log API view."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        data = {
+            'level': 'WARNING',
+            'message': 'Test warning',
+            'user_id': self.user.id,
+            'request_id': 'test-123'
+        }
+        
+        response = self.client.post(
+            reverse('logq:log_api'),
+            data=json.dumps(data),
+            content_type='application/json'
+        )
+        
+        self.assertEqual(response.status_code, 200)
+        
+        time.sleep(0.5)
+        
+        # Verify we have exactly one log entry
+        self.assertEqual(LogEntry.objects.count(), 1)
+        
+        entry = LogEntry.objects.first()
+        self.assertEqual(entry.level, LogLevel.WARNING)
+        self.assertEqual(entry.user_id, self.user.id)
+        self.assertEqual(entry.request_id, 'test-123')
+    
+    def test_get_logs_api(self):
+        """Test retrieving logs via API."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        # Create some test logs directly
+        LogEntry.objects.create(level=LogLevel.INFO, message="Test 1")
+        LogEntry.objects.create(level=LogLevel.ERROR, message="Test 2")
+        LogEntry.objects.create(level=LogLevel.DEBUG, message="Test 3")
+        
+        # Verify we have exactly 3 logs
+        self.assertEqual(LogEntry.objects.count(), 3)
+        
+        response = self.client.get(reverse('logq:log_api'))
+        self.assertEqual(response.status_code, 200)
+        
+        data = response.json()
+        self.assertEqual(len(data['logs']), 3)
+        self.assertEqual(data['logs'][0]['message'], "Test 1")
+    
+    def test_invalid_log_level(self):
+        """Test API with invalid log level."""
+        data = {
+            'level': 'INVALID',
+            'message': 'Test message'
+        }
+        
+        response = self.client.post(
+            reverse('logq:log_endpoint'),
+            data=json.dumps(data),
+            content_type='application/json'
+        )
+        
+        self.assertEqual(response.status_code, 400)
+        self.assertIn('Invalid log level', response.json()['error'])
+
+
+@override_settings(ASYNC_LOGGING_CONFIG={'MAX_QUEUE_SIZE': 500, 'FLUSH_INTERVAL': 0.5})
+class ConfigurationTestCase(TransactionTestCase):
+    def setUp(self):
+        super().setUp()
+        # Clear all existing logs
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+    
+    def tearDown(self):
+        # Clear logs after test
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+        super().tearDown()
+    
+    def test_custom_configuration(self):
+        """Test that custom configuration is respected."""
+        logger = AsyncLogger()
+        self.assertEqual(logger.queue.maxsize, 500)
+        self.assertEqual(logger.flush_interval, 0.5)
+
+
+class MiddlewareTestCase(TransactionTestCase):
+    def setUp(self):
+        super().setUp()
+        # Stop the global logger to avoid interference
+        stop_async_logger()
+        # Clear all existing logs
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+    
+    def tearDown(self):
+        # Clear logs after test
+        with connection.cursor() as cursor:
+            cursor.execute("DELETE FROM logq_logentry")
+        super().tearDown()
+    
+    def test_middleware_request_logging(self):
+        """Test that middleware logs requests."""
+        # Verify we start with no logs
+        self.assertEqual(LogEntry.objects.count(), 0)
+        
+        response = self.client.get('/admin/')
+        
+        time.sleep(0.5)
+        
+        entries = LogEntry.objects.all()
+        self.assertGreater(entries.count(), 0)
+        
+        # Should have request start and completion logs
+        start_logs = entries.filter(message__contains="Request started")
+        complete_logs = entries.filter(message__contains="Request completed")
+        
+        self.assertGreater(start_logs.count(), 0)
+        self.assertGreater(complete_logs.count(), 0)

logq/urls.py

@@ -0,0 +1,9 @@
+from django.urls import path
+from . import views
+
+app_name = 'logq'
+
+urlpatterns = [
+    path('api/log/', views.log_endpoint, name='log_endpoint'),
+    path('api/logs/', views.LogAPIView.as_view(), name='log_api'),
+]

logq/utils.py

@@ -0,0 +1,141 @@
+from .async_logger import get_async_logger
+from functools import wraps
+import time
+
+
+def log_function_call(func=None, *, level='INFO'):
+    """
+    Decorator to automatically log function calls.
+    
+    Usage:
+        @log_function_call
+        def my_function():
+            pass
+        
+        @log_function_call(level='DEBUG')
+        def my_debug_function():
+            pass
+    """
+    def decorator(func):
+        @wraps(func)
+        def wrapper(*args, **kwargs):
+            logger = get_async_logger()
+            
+            # Log function entry
+            logger.log(
+                level,
+                f"Entering function: {func.__name__}",
+                extra_data={
+                    'args_count': len(args),
+                    'kwargs_keys': list(kwargs.keys()),
+                    'module': func.__module__,
+                }
+            )
+            
+            start_time = time.time()
+            try:
+                result = func(*args, **kwargs)
+                execution_time = time.time() - start_time
+                
+                # Log successful completion
+                logger.log(
+                    level,
+                    f"Function completed: {func.__name__} (took {execution_time:.3f}s)",
+                    extra_data={'execution_time': execution_time}
+                )
+                
+                return result
+            
+            except Exception as e:
+                execution_time = time.time() - start_time
+                
+                # Log exception
+                logger.exception(
+                    f"Function failed: {func.__name__} (took {execution_time:.3f}s)",
+                    exc_info=str(e),
+                    extra_data={'execution_time': execution_time}
+                )
+                raise
+        
+        return wrapper
+    
+    if func is None:
+        return decorator
+    return decorator(func)
+
+
+def log_performance(threshold_seconds=1.0):
+    """
+    Decorator to log slow function calls.
+    
+    Usage:
+        @log_performance(threshold_seconds=0.5)
+        def my_slow_function():
+            pass
+    """
+    def decorator(func):
+        @wraps(func)
+        def wrapper(*args, **kwargs):
+            start_time = time.time()
+            result = func(*args, **kwargs)
+            execution_time = time.time() - start_time
+            
+            if execution_time > threshold_seconds:
+                logger = get_async_logger()
+                logger.warning(
+                    f"Slow function detected: {func.__name__} took {execution_time:.3f}s",
+                    extra_data={
+                        'execution_time': execution_time,
+                        'threshold': threshold_seconds,
+                        'module': func.__module__,
+                    }
+                )
+            
+            return result
+        return wrapper
+    return decorator
+
+
+class LogContext:
+    """
+    Context manager for logging operations with automatic timing.
+    
+    Usage:
+        with LogContext("Processing data", level='INFO'):
+            # do some work
+            pass
+    """
+    
+    def __init__(self, message, level='INFO', **kwargs):
+        self.message = message
+        self.level = level
+        self.kwargs = kwargs
+        self.logger = get_async_logger()
+        self.start_time = None
+    
+    def __enter__(self):
+        self.start_time = time.time()
+        self.logger.log(
+            self.level,
+            f"Starting: {self.message}",
+            **self.kwargs
+        )
+        return self
+    
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        execution_time = time.time() - self.start_time
+        
+        if exc_type is None:
+            self.logger.log(
+                self.level,
+                f"Completed: {self.message} (took {execution_time:.3f}s)",
+                extra_data={'execution_time': execution_time},
+                **self.kwargs
+            )
+        else:
+            self.logger.exception(
+                f"Failed: {self.message} (took {execution_time:.3f}s)",
+                exc_info=str(exc_val),
+                extra_data={'execution_time': execution_time},
+                **self.kwargs
+            )

logq/views.py

@@ -0,0 +1,103 @@
+from django.http import JsonResponse
+from django.views.decorators.csrf import csrf_exempt
+from django.views.decorators.http import require_http_methods
+from django.utils.decorators import method_decorator
+from django.views import View
+import json
+from .async_logger import get_async_logger
+from .models import LogEntry, LogLevel
+
+
+@csrf_exempt
+@require_http_methods(["POST"])
+def log_endpoint(request):
+    """Simple API endpoint for external logging."""
+    try:
+        data = json.loads(request.body)
+        level = data.get('level', 'INFO')
+        message = data.get('message', '')
+        
+        if not message:
+            return JsonResponse({'error': 'Message is required'}, status=400)
+        
+        if level not in [choice[0] for choice in LogLevel.choices]:
+            return JsonResponse({'error': 'Invalid log level'}, status=400)
+        
+        logger = get_async_logger()
+        logger.log(level, message, extra_data=data.get('extra_data', {}))
+        
+        return JsonResponse({'status': 'success'})
+    
+    except json.JSONDecodeError:
+        return JsonResponse({'error': 'Invalid JSON'}, status=400)
+    except Exception as e:
+        return JsonResponse({'error': str(e)}, status=500)
+
+
+@method_decorator(csrf_exempt, name='dispatch')
+class LogAPIView(View):
+    """Class-based view for more advanced logging operations."""
+    
+    def post(self, request):
+        """Handle POST requests for logging."""
+        try:
+            data = json.loads(request.body)
+            level = data.get('level', 'INFO')
+            message = data.get('message', '')
+            
+            if not message:
+                return JsonResponse({'error': 'Message is required'}, status=400)
+            
+            logger = get_async_logger()
+            logger.log(
+                level, 
+                message, 
+                user_id=data.get('user_id'),
+                request_id=data.get('request_id'),
+                extra_data=data.get('extra_data', {})
+            )
+            
+            return JsonResponse({'status': 'success'})
+        
+        except json.JSONDecodeError:
+            return JsonResponse({'error': 'Invalid JSON'}, status=400)
+        except Exception as e:
+            return JsonResponse({'error': str(e)}, status=500)
+    
+    def get(self, request):
+        """Handle GET requests for retrieving recent logs."""
+        try:
+            limit = int(request.GET.get('limit', 100))
+            level = request.GET.get('level')
+            module = request.GET.get('module')
+            
+            query = LogEntry.objects.all()
+            
+            if level:
+                query = query.filter(level=level)
+            if module:
+                query = query.filter(module__icontains=module)
+            
+            logs = query.order_by('-timestamp')[:limit]
+            
+            log_data = []
+            for log in logs:
+                log_data.append({
+                    'id': log.id,
+                    'timestamp': log.timestamp.isoformat(),
+                    'level': log.level,
+                    'message': log.message,
+                    'module': log.module,
+                    'function': log.function,
+                    'line_number': log.line_number,
+                    'user_id': log.user_id,
+                    'request_id': log.request_id,
+                    'extra_data': log.extra_data,
+                })
+            
+            return JsonResponse({'logs': log_data})
+        
+        except ValueError:
+            return JsonResponse({'error': 'Invalid limit parameter'}, status=400)
+        except Exception as e:
+            return JsonResponse({'error': str(e)}, status=500)