django-webhook-subscriber 1.0.0__tar.gz → 2.1.0__tar.gz

django_webhook_subscriber-2.1.0/PKG-INFO

@@ -0,0 +1,822 @@
+Metadata-Version: 2.4
+Name: django-webhook-subscriber
+Version: 2.1.0
+Summary: A Django package designed to handle webhook creation, management, and delivery.
+Author-email: 42 Portugal <root@42porto.com>
+Classifier: Environment :: Web Environment
+Classifier: Framework :: Django
+Classifier: Framework :: Django :: 5.0
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: Django<5.3,>=5.0
+Requires-Dist: djangorestframework<3.17,>=3.15.0
+Requires-Dist: requests<2.33,>=2.32.3
+Requires-Dist: celery<5.6,>=5.0.0
+Requires-Dist: redis<6.5,>=6.4
+Provides-Extra: test
+Requires-Dist: pytest<8.4,>=8.3.4; extra == "test"
+Requires-Dist: pytest-django<5.0,>=4.9.0; extra == "test"
+Requires-Dist: pytest-cov<6.1,>=6.0.0; extra == "test"
+Requires-Dist: pytest-factoryboy<2.9,>=2.8; extra == "test"
+Dynamic: license-file
+
+# Django Webhook Subscriber
+
+A Django package for managing webhook subscriptions and deliveries with robust retry logic, performance monitoring, and comprehensive admin interface.
+
+## Table of Contents
+
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Configuration](#configuration)
+- [Basic Usage](#basic-usage)
+- [Models Reference](#models-reference)
+- [API Reference](#api-reference)
+- [Admin Interface](#admin-interface)
+- [Management Commands](#management-commands)
+- [Performance Testing](#performance-testing)
+- [Production Deployment](#production-deployment)
+- [Troubleshooting](#troubleshooting)
+- [Contributing](#contributing)
+
+## Installation
+
+### Requirements
+
+- Python 3.8+
+- Django 3.2+
+- Django REST Framework 3.12+
+- Celery 5.0+ (for async webhook delivery)
+- Redis or another cache backend (recommended)
+
+### Install Package
+
+```bash
+pip install django-webhook-subscriber
+```
+
+### Django Settings
+
+Add to your `INSTALLED_APPS`:
+
+```python
+INSTALLED_APPS = [
+    "django_webhook_subscriber",
+    # ... your apps
+]
+```
+
+### Database Migration
+
+```bash
+python manage.py migrate django_webhook_subscriber
+```
+
+### Celery Configuration
+
+Configure Celery for async webhook delivery:
+
+```python
+# settings.py
+CELERY_BROKER_URL = "redis://localhost:6379/0"
+CELERY_RESULT_BACKEND = "redis://localhost:6379/0"
+```
+
+```python
+# celery.py
+from celery import Celery
+import os
+
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
+
+app = Celery("myproject")
+app.config_from_object("django.conf:settings", namespace="CELERY")
+app.autodiscover_tasks()
+```
+
+## Quick Start
+
+### 1. Create a Webhook Subscriber
+
+```python
+from django.contrib.contenttypes.models import ContentType
+from django_webhook_subscriber.models import WebhookSubscriber, WebhookSubscription
+from django.contrib.auth import get_user_model
+
+User = get_user_model()
+
+# Create subscriber for User model
+user_content_type = ContentType.objects.get_for_model(User)
+
+subscriber = WebhookSubscriber.objects.create(
+    name="My External API",
+    description="Receives user events",
+    content_type=user_content_type,
+    target_url="https://api.example.com/webhooks",
+    max_retries=3,
+    retry_delay=60,
+    timeout=30
+)
+
+# Create subscription for 'created' events
+subscription = WebhookSubscription.objects.create(
+    subscriber=subscriber,
+    event_name="created",
+    is_active=True
+)
+```
+
+### 2. Send Webhooks
+
+```python
+from django_webhook_subscriber import send_webhooks
+from django.contrib.auth import get_user_model
+
+User = get_user_model()
+
+# Create a user and send webhook
+user = User.objects.create(name="John Doe", email="john@example.com")
+
+# Send webhook for "created" event
+send_webhooks(user, "created")
+```
+
+### 3. Custom Serialization
+
+```python
+from rest_framework import serializers
+from django.contrib.auth import get_user_model
+
+User = get_user_model()
+
+class UserWebhookSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = User
+        fields = ["id", "name", "email"]
+
+# Update subscriber to use custom serializer
+subscriber.serializer_class = "myapp.serializers.UserWebhookSerializer"
+subscriber.save()
+```
+
+### 4. Raw Payload (No Metadata Wrapper)
+
+By default, webhook payloads include metadata and the serialized data is nested inside a `fields` key:
+
+```json
+{
+    "pk": 1,
+    "event_signal": "created",
+    "source": "auth.user",
+    "timestamp": "2026-02-05T10:30:00Z",
+    "fields": {
+        "id": 1,
+        "name": "John Doe",
+        "email": "john@example.com"
+    }
+}
+```
+
+If you prefer a raw payload containing only the serialized data (without the metadata wrapper), use the `WebhookRawPayloadMixin`:
+
+```python
+from rest_framework import serializers
+from django.contrib.auth import get_user_model
+from django_webhook_subscriber.serializers import WebhookRawPayloadMixin
+
+User = get_user_model()
+
+class UserRawWebhookSerializer(WebhookRawPayloadMixin, serializers.ModelSerializer):
+    class Meta:
+        model = User
+        fields = ["id", "name", "email"]
+
+# Update subscriber to use the raw payload serializer
+subscriber.serializer_class = "myapp.serializers.UserRawWebhookSerializer"
+subscriber.save()
+```
+
+With `WebhookRawPayloadMixin`, the webhook payload will be just the serialized data:
+
+```json
+{
+    "id": 1,
+    "name": "John Doe",
+    "email": "john@example.com"
+}
+```
+
+## Configuration
+
+Configure the package in your Django settings:
+
+<!-- TODO: confirm these are all the settings -->
+
+```python
+WEBHOOK_SUBSCRIBER = {
+    # Log retention
+    "LOG_RETENTION_DAYS": 30,
+    "AUTO_CLEANUP": True,
+
+    # Performance settings
+    "MAX_WEBHOOK_BATCH_SIZE": 50,
+    "WEBHOOK_CACHE_TTL": 300,
+
+    # Defaults for new subscribers
+    "DEFAULT_MAX_RETRIES": 3,
+    "DEFAULT_RETRY_DELAY": 60,
+    "REQUEST_TIMEOUT": 30,
+}
+
+
+# Recommended cache configuration
+CACHES = {
+    'default': {
+        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
+        'LOCATION': 'redis://127.0.0.1:6379/1',
+        'OPTIONS': {
+            'CONNECTION_POOL_KWARGS': {'max_connections': 20},
+        }
+    }
+}
+
+
+# Optional: Disable webhooks globally
+DISABLE_WEBHOOKS = False  # Set to True to disable all webhook sending
+```
+
+## Basic Usage
+
+### Sending Webhooks
+
+The main API is the `send_webhooks()` function:
+
+```python
+from django_webhook_subscriber import send_webhooks
+
+# Basic usage
+send_webhooks(instance, 'event_name')
+```
+
+### Common Integration Patterns
+
+#### Django Signals
+
+```python
+# signals.py
+from django.db.models.signals import post_save
+from django.dispatch import receiver
+from django_webhook_subscriber import send_webhooks
+from django.contrib.auth import get_user_model
+
+User = get_user_model()
+
+@receiver(post_save, sender=User)
+def user_saved(sender, instance, created, **kwargs):
+    event_name = "created" if created else "updated"
+    send_webhooks(instance, event_name)
+
+```
+
+#### Django Lifecycle
+
+```python
+# models.py
+from django_lifecycle import LifecycleModel, hook, AFTER_CREATE, AFTER_UPDATE
+from django_webhook_subscriber import send_webhooks
+
+class User(LifecycleModel):
+    name = models.CharField(max_length=100)
+    email = models.EmailField()
+
+    @hook(AFTER_CREATE)
+    def on_create(self):
+        send_webhooks(self, 'created')
+
+    @hook(AFTER_UPDATE, when='email', has_changed=True)
+    def on_email_update(self):
+        send_webhooks(self, 'email_updated')
+```
+
+#### Manual Integration
+
+```python
+# myapp/views.py
+from django_webhook_subscriber import send_webhooks
+
+class UserViewSet(viewsets.ModelViewSet):
+    def perform_create(self, serializer):
+        user = serializer.save()
+        send_webhooks(user, "created")
+
+    def perform_update(self, serializer):
+        user = serializer.save()
+        send_webhooks(user, "updated")
+```
+
+### Disabling Webhooks Temporarily
+
+```python
+from django_webhook_subscriber.utils import disable_webhooks
+
+# Disable webhooks for bulk operations
+with disable_webhooks():
+    User.objects.bulk_create([...])  # No webhooks sent
+
+# Webhooks resume normal operation after the context
+```
+
+## Models Reference
+
+### WebhookSubscriber
+
+Represents an external service that wants to receive webhooks.
+
+**Key Fields:**
+
+- `name`: Human-readable name for the subscriber
+- `target_url`: Base URL for webhook delivery
+- `content_type`: Django model this subscriber watches
+- `serializer_class`: Optional custom DRF serializer
+- `max_retries`: Maximum delivery attempts
+- `retry_delay`: Seconds between retries
+- `timeout`: Request timeout
+- `auto_disable_after_failures`: Auto-disable threshold
+
+**Methods:**
+
+- `record_success()`: Mark successful delivery
+- `record_failure()`: Mark failed delivery, handle auto-disable
+
+### WebhookSubscription
+
+Individual event subscription for a subscriber.
+
+**Key Fields:**
+
+- `subscriber`: Foreign key to WebhookSubscriber
+- `event_name`: Event to subscribe to (e.g., 'created', 'updated')
+- `custom_endpoint`: Optional endpoint override
+- `success_rate`: Calculated property showing delivery success percentage
+
+**Properties:**
+
+- `endpoint`: Full URL for this subscription
+- `success_rate`: Percentage of successful deliveries
+
+**Methods:**
+
+- `record_delivery_attempt(success, response_text)`: Track delivery stats
+
+### WebhookDeliveryLog
+
+Log of individual webhook delivery attempts.
+
+**Key Fields:**
+
+- `subscription`: Foreign key to WebhookSubscription
+- `attempt_number`: Retry attempt number
+- `response_status`: HTTP response status
+- `response_body`: HTTP response body (truncated)
+- `delivery_duration_ms`: Delivery time in milliseconds
+- `error_message`: Exception details if delivery failed
+
+**Properties:**
+
+- `is_success`: True if delivery was successful (2xx status)
+- `is_client_error`: True for 4xx status codes
+- `is_server_error`: True for 5xx status codes
+
+## API Reference
+
+### Core Functions
+
+#### `send_webhooks(instance, event_name, context=None, **kwargs)`
+
+Send webhooks for a model instance and event.
+
+**Parameters:**
+
+- `instance`: Django model instance
+- `event_name`: String event name
+- `context`: Optional dict with additional data
+- `**kwargs`: Additional arguments
+
+**Returns:**
+
+- Dict with processing summary
+
+**Example:**
+
+```python
+result = send_webhooks(user, 'created')
+# Returns: {'processed': 3, 'batches': 1, 'task_ids': [...]}
+```
+
+#### `clear_webhook_cache(content_type=None, event_name=None)`
+
+Clear cached webhook subscription data.
+
+**Parameters:**
+
+- `content_type`: Optional ContentType to limit clearing
+- `event_name`: Optional event name to limit clearing
+
+#### `get_webhook_cache_stats()`
+
+Get detailed cache statistics.
+
+**Returns:**
+
+- Dict with cache hit ratios, key counts, etc.
+
+### Utility Functions
+
+#### `webhooks_disabled()`
+
+Check if webhooks are currently disabled.
+
+#### `disable_webhooks()` (context manager)
+
+Temporarily disable webhook sending.
+
+#### `generate_secret()`
+
+Generate a new webhook secret key.
+
+## Admin Interface
+
+The package provides a comprehensive Django admin interface:
+
+### WebhookSubscriber Admin
+
+- **Health indicators**: Visual status showing healthy/warning/critical
+- **Performance tracking**: Success rates and failure counts
+- **Connectivity testing**: Test endpoints via admin actions
+- **Cache management**: Clear cache for specific subscribers
+
+**Available Actions:**
+
+- Activate/deactivate subscribers
+- Test endpoint connectivity
+- Reset failure counters
+- Clear cache
+
+### WebhookSubscription Admin
+
+- **Performance metrics**: Success rates with color coding
+- **Recent deliveries**: Visual icons showing last 5 attempts
+- **Statistics management**: Reset performance counters
+
+### WebhookDeliveryLog Admin
+
+- **Detailed logging**: Formatted JSON payloads and responses
+- **Performance analysis**: Delivery times and retry information
+- **Error categorization**: Grouped error analysis
+
+## Management Commands
+
+### Cache Management
+
+```bash
+# Show cache statistics
+python manage.py webhook_cache stats
+
+# Clear all cache
+python manage.py webhook_cache clear
+
+# Clear specific content type
+python manage.py webhook_cache clear --content-type=myapp.User
+
+# Pre-warm cache
+python manage.py webhook_cache warm
+```
+
+### Testing Endpoints
+
+```bash
+# Test all active subscribers
+python manage.py webhook_test --all
+
+# Test with custom payload
+python manage.py webhook_test --subscriber-id=1 --method=POST --payload='{"test": true}'
+```
+
+### Log Management
+
+```bash
+# Clean up old logs
+python manage.py webhook_logs cleanup --days=30
+
+# Show statistics
+python manage.py webhook_logs stats --days=7
+
+# Show recent errors
+python manage.py webhook_logs errors --limit=20
+```
+
+### System Status
+
+```bash
+# Overview
+python manage.py webhook_status
+
+# Detailed per-subscriber status
+python manage.py webhook_status --detailed
+```
+
+### Manual Webhook Sending
+
+```bash
+# Send test webhook
+python manage.py webhook_send myapp.User 123 created
+
+# With custom context
+python manage.py webhook_send myapp.Order 456 paid --context='{"payment_method": "stripe"}'
+```
+
+## Performance Testing
+
+Test your webhook system under load:
+
+```bash
+# Basic performance test
+python manage.py webhook_performance_test \
+    --model=myapp.User \
+    --event=created \
+    --object-count=20 \
+    --concurrent-webhooks=10
+
+# High-load test with delivery measurement
+python manage.py webhook_performance_test \
+    --model=myapp.Order \
+    --event=paid \
+    --object-count=50 \
+    --concurrent-webhooks=20 \
+    --measure-delivery \
+    --timeout=60 \
+    --warmup
+
+# Sustained load test
+python manage.py webhook_performance_test \
+    --model=myapp.Product \
+    --event=updated \
+    --object-count=15 \
+    --batches=5 \
+    --batch-delay=2.0
+```
+
+**Performance Metrics Provided:**
+
+- Send throughput (webhooks/second)
+- Delivery timing statistics
+- Success/failure rates
+- Error categorization
+- Concurrency analysis
+
+## Production Deployment
+
+### Celery Setup
+
+Ensure Celery is properly configured for production:
+
+```python
+# celery.py
+from celery import Celery
+
+app = Celery('myproject')
+app.config_from_object('django.conf:settings', namespace='CELERY')
+
+# Configure for production
+app.conf.update(
+    task_serializer='json',
+    accept_content=['json'],
+    result_serializer='json',
+    timezone='UTC',
+    enable_utc=True,
+    task_track_started=True,
+    worker_hijack_root_logger=False,
+    worker_log_format='[%(asctime)s: %(levelname)s/%(processName)s] %(message)s',
+)
+
+app.autodiscover_tasks()
+```
+
+### Monitoring Setup
+
+Configure logging for webhook monitoring:
+
+```python
+# settings.py
+LOGGING = {
+    'version': 1,
+    'disable_existing_loggers': False,
+    'handlers': {
+        'webhook_file': {
+            'level': 'INFO',
+            'class': 'logging.FileHandler',
+            'filename': '/var/log/django/webhooks.log',
+        },
+    },
+    'loggers': {
+        'django_webhook_subscriber': {
+            'handlers': ['webhook_file'],
+            'level': 'INFO',
+            'propagate': True,
+        },
+    },
+}
+```
+
+### Automated Maintenance
+
+Set up cron jobs for maintenance:
+
+```bash
+# Daily log cleanup at 2 AM
+0 2 * * * cd /path/to/project && python manage.py webhook_logs cleanup
+
+# Weekly health check report
+0 9 * * 1 cd /path/to/project && python manage.py webhook_status --detailed
+
+# Cache warming after deployments
+@reboot cd /path/to/project && python manage.py webhook_cache warm
+```
+
+### Performance Optimization
+
+1. **Use Redis for caching:**
+
+   ```python
+   CACHES = {
+       'default': {
+           'BACKEND': 'django.core.cache.backends.redis.RedisCache',
+           'LOCATION': 'redis://127.0.0.1:6379/1',
+       }
+   }
+   ```
+
+2. **Configure appropriate batch sizes:**
+
+   ```python
+   WEBHOOK_SUBSCRIBER = {
+       'MAX_WEBHOOK_BATCH_SIZE': 25,  # Adjust based on your load
+       'WEBHOOK_CACHE_TTL': 600,      # 10 minutes
+   }
+   ```
+
+3. **Monitor delivery performance:**
+
+   ```bash
+   # Regular performance testing
+   python manage.py webhook_performance_test --model=myapp.User --event=test --object-count=10
+   ```
+
+## Troubleshooting
+
+### Common Issues
+
+#### 1. Webhooks Not Sending
+
+**Check:**
+
+- Webhooks globally enabled: `DISABLE_WEBHOOKS = False`
+- Active subscriptions exist for the model/event
+- Celery workers are running
+
+**Debug:**
+
+```bash
+python manage.py webhook_status --detailed
+python manage.py webhook_cache stats
+```
+
+#### 2. High Delivery Failures
+
+**Check:**
+
+- Endpoint accessibility
+- Network connectivity
+- Authentication headers
+
+**Debug:**
+
+```bash
+python manage.py webhook_test --all
+python manage.py webhook_logs errors --limit=50
+```
+
+#### 3. Performance Issues
+
+**Check:**
+
+- Cache hit ratios
+- Batch sizes
+- Celery worker count
+
+**Debug:**
+
+```bash
+python manage.py webhook_cache stats
+python manage.py webhook_performance_test --model=myapp.User --event=test --object-count=5
+```
+
+#### 4. Memory Usage
+
+**Solutions:**
+
+- Reduce `LOG_RETENTION_DAYS`
+- Increase cleanup frequency
+- Limit response body storage
+
+```bash
+python manage.py webhook_logs cleanup --days=7
+```
+
+### Error Reference
+
+#### Common Error Messages
+
+- **"No subscriptions found"**: No active subscriptions for model/event
+- **"Webhook disabled"**: Global or context-specific disabling active
+- **"Subscription not found"**: Subscription deleted during processing
+- **"Connection error"**: Network connectivity issues
+- **"Timeout"**: Endpoint response too slow
+
+#### Performance Warnings
+
+- **Cache miss ratio > 50%**: Consider warming cache more frequently
+- **Delivery time > 5s**: Endpoint performance issues
+- **Success rate < 95%**: Check endpoint reliability
+
+### Debug Mode
+
+Enable detailed logging for debugging:
+
+```python
+# settings.py
+LOGGING = {
+    'loggers': {
+        'django_webhook_subscriber': {
+            'level': 'DEBUG',
+            'handlers': ['console'],
+        },
+    },
+}
+```
+
+## Contributing
+
+### Development Setup
+
+1. Clone the repository
+2. Install development dependencies: `pip install -e .[dev]`
+3. Run tests: `python -m pytest`
+4. Run linting: `flake8 django_webhook_subscriber/`
+
+### Running Tests
+
+```bash
+# Run all tests
+python -m pytest
+
+# Run specific test file
+python -m pytest tests/test_delivery.py
+
+# Run with coverage
+python -m pytest --cov=django_webhook_subscriber
+```
+
+### Contributing Guidelines
+
+1. Fork the repository
+2. Create a feature branch
+3. Add tests for new functionality
+4. Ensure all tests pass
+5. Submit a pull request
+
+### Reporting Issues
+
+Please include:
+
+- Django version
+- Package version
+- Full error traceback
+- Minimal reproduction case
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](./LICENSE) file for details.