djlogq 1.0.0__tar.gz

djlogq-1.0.0/MANIFEST.in

@@ -0,0 +1,6 @@
+include README.md
+include LICENSE
+recursive-include src/logq/migrations *.py
+recursive-include src/logq/management *.py
+recursive-include src/logq/templates *.html
+recursive-include src/logq/static *

djlogq-1.0.0/PKG-INFO

@@ -0,0 +1,242 @@
+Metadata-Version: 2.4
+Name: djlogq
+Version: 1.0.0
+Summary: A reusable Django app for asynchronous, thread-safe logging with rich metadata, admin interface, and API support.
+Author-email: mesake <mesnavunawa@gmail.com>
+License: MIT
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: Django
+Provides-Extra: dev
+Requires-Dist: build==1.2.1; extra == "dev"
+Requires-Dist: nox==2024.4.15; extra == "dev"
+Requires-Dist: twine==5.1.1; extra == "dev"
+
+# Django Async Logger
+
+A reusable Django app that provides asynchronous logging functionality using a separate thread to avoid blocking the main application.
+
+## Features
+
+- **Asynchronous Logging**: All log operations run in a separate thread
+- **Thread-Safe**: Uses a queue system for thread-safe logging
+- **Rich Metadata**: Captures module, function, line number, user ID, request ID, and extra data
+- **Admin Interface**: Beautiful Django admin interface for viewing and managing logs
+- **API Endpoints**: REST API for external logging
+- **Middleware**: Automatic request logging with unique request IDs
+- **Decorators**: Utility decorators for function logging and performance monitoring
+- **Context Managers**: Easy-to-use context managers for operation logging
+- **Configurable**: Customizable queue size, flush intervals, and cleanup policies
+
+## Installation
+
+1. Add the app to your Django project:
+```python
+INSTALLED_APPS = [
+    # ...
+    'logq',
+]
+```
+
+2. Add the middleware to your settings:
+```python
+MIDDLEWARE = [
+    # ...
+    'logq.middleware.AsyncLoggingMiddleware',
+]
+```
+
+3. Run migrations:
+```bash
+python manage.py makemigrations logq
+python manage.py migrate
+```
+
+4. (Optional) Configure logging settings:
+```python
+ASYNC_LOGGING_CONFIG = {
+    'MAX_QUEUE_SIZE': 1000,
+    'FLUSH_INTERVAL': 1.0,  # seconds
+    'AUTO_CLEANUP_DAYS': 30,
+    'ENABLE_REQUEST_LOGGING': True,
+    'IGNORE_PATHS': ['/admin/'],  # paths to ignore for request logging
+}
+```
+
+## Usage
+
+### Basic Logging
+
+```python
+from logq.async_logger import get_async_logger
+
+logger = get_async_logger()
+
+# Different log levels
+logger.debug("Debug message")
+logger.info("Info message")
+logger.warning("Warning message")
+logger.error("Error message")
+logger.critical("Critical message")
+
+# With extra data
+logger.info("User action", extra_data={'action': 'login', 'ip': '192.168.1.1'})
+
+# Log exceptions
+try:
+    # some code that might fail
+    pass
+except Exception as e:
+    logger.exception("An error occurred", exc_info=str(e))
+```
+
+### Function Decorators
+
+```python
+from logq.utils import log_function_call, log_performance
+
+@log_function_call
+def my_function():
+    return "result"
+
+@log_function_call(level='DEBUG')
+def debug_function():
+    return "debug result"
+
+@log_performance(threshold_seconds=0.5)
+def slow_function():
+    time.sleep(1)
+    return "slow result"
+```
+
+### Context Managers
+
+```python
+from logq.utils import LogContext
+
+with LogContext("Processing data", level='INFO'):
+    # do some work
+    time.sleep(0.1)
+    # automatically logs start and completion with timing
+```
+
+### API Logging
+
+```python
+import requests
+import json
+
+# Log via API
+data = {
+    'level': 'INFO',
+    'message': 'External log message',
+    'extra_data': {'source': 'external_service'}
+}
+
+response = requests.post(
+    'http://your-domain/logq/api/log/',
+    data=json.dumps(data),
+    headers={'Content-Type': 'application/json'}
+)
+
+# Retrieve logs via API
+response = requests.get('http://your-domain/logq/api/logs/?limit=10')
+logs = response.json()['logs']
+```
+
+### Admin Interface
+
+Access the admin interface at `/admin/` to view and manage logs. Features include:
+
+- Filter by level, module, timestamp, user ID
+- Search by message, module, function, request ID
+- View extra data in formatted JSON
+- Delete old logs
+- Export functionality
+
+### Management Commands
+
+Clean old logs:
+```bash
+# Delete logs older than 30 days
+python manage.py clean_logs
+
+# Delete logs older than 7 days
+python manage.py clean_logs --days 7
+
+# Delete only DEBUG and INFO logs older than 30 days
+python manage.py clean_logs --level INFO
+
+# Dry run to see what would be deleted
+python manage.py clean_logs --dry-run
+```
+
+## Configuration Options
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `MAX_QUEUE_SIZE` | 1000 | Maximum number of log entries in the queue |
+| `FLUSH_INTERVAL` | 1.0 | How often to flush logs to database (seconds) |
+| `AUTO_CLEANUP_DAYS` | 30 | Days to keep logs before auto-cleanup |
+| `ENABLE_REQUEST_LOGGING` | True | Whether to log all HTTP requests |
+
+## Model Fields
+
+The `LogEntry` model includes:
+
+- `timestamp`: When the log was created
+- `level`: Log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+- `message`: The log message
+- `module`: Python module where the log originated
+- `function`: Function name where the log originated
+- `line_number`: Line number where the log originated
+- `user_id`: ID of the user (if authenticated)
+- `request_id`: Unique request identifier
+- `extra_data`: Additional JSON data
+- `created_at`: When the entry was saved to database
+
+## Performance Considerations
+
+- The logger runs in a separate thread and won't block your main application
+- Log entries are batched and written to the database periodically
+- If the queue is full, new entries are dropped (with console fallback)
+- Consider setting up database indexes for better query performance
+- Use the cleanup command regularly to prevent database bloat
+
+## Thread Safety
+
+The logger is completely thread-safe:
+- Uses a thread-safe queue for communication
+- Database operations are wrapped in transactions
+- Multiple threads can safely call the logger simultaneously
+
+## Customization
+
+You can extend the logger by:
+
+1. Creating custom log levels
+2. Adding new fields to the LogEntry model
+3. Customizing the admin interface
+4. Adding new API endpoints
+5. Creating custom middleware
+
+## Troubleshooting
+
+### Logs not appearing
+- Check that the async logger thread is running
+- Verify database migrations are applied
+- Check for any database connection issues
+
+### Performance issues
+- Reduce `FLUSH_INTERVAL` for more frequent writes
+- Increase `MAX_QUEUE_SIZE` for higher throughput
+- Add database indexes for frequently queried fields
+
+### Memory usage
+- Reduce `MAX_QUEUE_SIZE` if memory is a concern
+- Run cleanup commands more frequently
+- Monitor database size and clean old logs
+
+## License
+
+This project is open source and available under the MIT License. 

djlogq-1.0.0/README.md

@@ -0,0 +1,228 @@
+# Django Async Logger
+
+A reusable Django app that provides asynchronous logging functionality using a separate thread to avoid blocking the main application.
+
+## Features
+
+- **Asynchronous Logging**: All log operations run in a separate thread
+- **Thread-Safe**: Uses a queue system for thread-safe logging
+- **Rich Metadata**: Captures module, function, line number, user ID, request ID, and extra data
+- **Admin Interface**: Beautiful Django admin interface for viewing and managing logs
+- **API Endpoints**: REST API for external logging
+- **Middleware**: Automatic request logging with unique request IDs
+- **Decorators**: Utility decorators for function logging and performance monitoring
+- **Context Managers**: Easy-to-use context managers for operation logging
+- **Configurable**: Customizable queue size, flush intervals, and cleanup policies
+
+## Installation
+
+1. Add the app to your Django project:
+```python
+INSTALLED_APPS = [
+    # ...
+    'logq',
+]
+```
+
+2. Add the middleware to your settings:
+```python
+MIDDLEWARE = [
+    # ...
+    'logq.middleware.AsyncLoggingMiddleware',
+]
+```
+
+3. Run migrations:
+```bash
+python manage.py makemigrations logq
+python manage.py migrate
+```
+
+4. (Optional) Configure logging settings:
+```python
+ASYNC_LOGGING_CONFIG = {
+    'MAX_QUEUE_SIZE': 1000,
+    'FLUSH_INTERVAL': 1.0,  # seconds
+    'AUTO_CLEANUP_DAYS': 30,
+    'ENABLE_REQUEST_LOGGING': True,
+    'IGNORE_PATHS': ['/admin/'],  # paths to ignore for request logging
+}
+```
+
+## Usage
+
+### Basic Logging
+
+```python
+from logq.async_logger import get_async_logger
+
+logger = get_async_logger()
+
+# Different log levels
+logger.debug("Debug message")
+logger.info("Info message")
+logger.warning("Warning message")
+logger.error("Error message")
+logger.critical("Critical message")
+
+# With extra data
+logger.info("User action", extra_data={'action': 'login', 'ip': '192.168.1.1'})
+
+# Log exceptions
+try:
+    # some code that might fail
+    pass
+except Exception as e:
+    logger.exception("An error occurred", exc_info=str(e))
+```
+
+### Function Decorators
+
+```python
+from logq.utils import log_function_call, log_performance
+
+@log_function_call
+def my_function():
+    return "result"
+
+@log_function_call(level='DEBUG')
+def debug_function():
+    return "debug result"
+
+@log_performance(threshold_seconds=0.5)
+def slow_function():
+    time.sleep(1)
+    return "slow result"
+```
+
+### Context Managers
+
+```python
+from logq.utils import LogContext
+
+with LogContext("Processing data", level='INFO'):
+    # do some work
+    time.sleep(0.1)
+    # automatically logs start and completion with timing
+```
+
+### API Logging
+
+```python
+import requests
+import json
+
+# Log via API
+data = {
+    'level': 'INFO',
+    'message': 'External log message',
+    'extra_data': {'source': 'external_service'}
+}
+
+response = requests.post(
+    'http://your-domain/logq/api/log/',
+    data=json.dumps(data),
+    headers={'Content-Type': 'application/json'}
+)
+
+# Retrieve logs via API
+response = requests.get('http://your-domain/logq/api/logs/?limit=10')
+logs = response.json()['logs']
+```
+
+### Admin Interface
+
+Access the admin interface at `/admin/` to view and manage logs. Features include:
+
+- Filter by level, module, timestamp, user ID
+- Search by message, module, function, request ID
+- View extra data in formatted JSON
+- Delete old logs
+- Export functionality
+
+### Management Commands
+
+Clean old logs:
+```bash
+# Delete logs older than 30 days
+python manage.py clean_logs
+
+# Delete logs older than 7 days
+python manage.py clean_logs --days 7
+
+# Delete only DEBUG and INFO logs older than 30 days
+python manage.py clean_logs --level INFO
+
+# Dry run to see what would be deleted
+python manage.py clean_logs --dry-run
+```
+
+## Configuration Options
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `MAX_QUEUE_SIZE` | 1000 | Maximum number of log entries in the queue |
+| `FLUSH_INTERVAL` | 1.0 | How often to flush logs to database (seconds) |
+| `AUTO_CLEANUP_DAYS` | 30 | Days to keep logs before auto-cleanup |
+| `ENABLE_REQUEST_LOGGING` | True | Whether to log all HTTP requests |
+
+## Model Fields
+
+The `LogEntry` model includes:
+
+- `timestamp`: When the log was created
+- `level`: Log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+- `message`: The log message
+- `module`: Python module where the log originated
+- `function`: Function name where the log originated
+- `line_number`: Line number where the log originated
+- `user_id`: ID of the user (if authenticated)
+- `request_id`: Unique request identifier
+- `extra_data`: Additional JSON data
+- `created_at`: When the entry was saved to database
+
+## Performance Considerations
+
+- The logger runs in a separate thread and won't block your main application
+- Log entries are batched and written to the database periodically
+- If the queue is full, new entries are dropped (with console fallback)
+- Consider setting up database indexes for better query performance
+- Use the cleanup command regularly to prevent database bloat
+
+## Thread Safety
+
+The logger is completely thread-safe:
+- Uses a thread-safe queue for communication
+- Database operations are wrapped in transactions
+- Multiple threads can safely call the logger simultaneously
+
+## Customization
+
+You can extend the logger by:
+
+1. Creating custom log levels
+2. Adding new fields to the LogEntry model
+3. Customizing the admin interface
+4. Adding new API endpoints
+5. Creating custom middleware
+
+## Troubleshooting
+
+### Logs not appearing
+- Check that the async logger thread is running
+- Verify database migrations are applied
+- Check for any database connection issues
+
+### Performance issues
+- Reduce `FLUSH_INTERVAL` for more frequent writes
+- Increase `MAX_QUEUE_SIZE` for higher throughput
+- Add database indexes for frequently queried fields
+
+### Memory usage
+- Reduce `MAX_QUEUE_SIZE` if memory is a concern
+- Run cleanup commands more frequently
+- Monitor database size and clean old logs
+
+## License
+
+This project is open source and available under the MIT License. 

djlogq-1.0.0/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["setuptools >= 40.9.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "djlogq"
+version = "1.0.0"
+description = "A reusable Django app for asynchronous, thread-safe logging with rich metadata, admin interface, and API support."
+readme = "README.md"
+authors = [
+    {name = "mesake", email = "mesnavunawa@gmail.com"}
+]
+license = {text = "MIT"}
+requires-python = ">=3.8"
+dependencies = [
+    "Django"
+]
+
+[project.optional-dependencies]
+dev = [
+    "build==1.2.1",
+    "nox==2024.4.15",
+    "twine==5.1.1",
+]
+
+# Add package discovery configuration
+[tool.setuptools.packages.find]
+where = ["src"]
+
+# Add package data for Django app
+[tool.setuptools.package-data]
+logq = [
+    "migrations/*.py",
+    "management/commands/*.py",
+    "templates/*.html",
+    "static/*",
+]

djlogq-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+