django-activity-audit 1.3.0.dev10__tar.gz → 1.3.0.dev12__tar.gz

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: django-activity-audit
-Version: 1.3.0.dev10
+Version: 1.3.0.dev12
 Summary: A Django package for easy CRUD operation logging and container logs
 Project-URL: Homepage, https://github.com/shree256/django-activity-audit
 Project-URL: Repository, https://github.com/shree256/django-activity-audit

django_activity_audit-1.3.0.dev12/activity_audit/REVIEW.md

@@ -0,0 +1,297 @@
+# activity_audit — Code Review
+
+> Reviewed: 2026-06-08  
+> Reviewer: Claude Code  
+> Scope: Full app audit following bug fixes for `AppRegistryNotReady` and `request_id` thread-local propagation
+
+---
+
+## Recent Fixes (Verified Correct)
+
+| Fix | File | Status |
+|-----|------|--------|
+| `AppRegistryNotReady` — model imports deferred to `unregistered.py`, loaded in `apps.ready()` | `unregistered.py`, `apps.py`, `settings.py` | ✅ Correct |
+| `request_id` not written to `app.log` — captured via `prepare()` in handler (producer thread) before record is queued to background `QueueListener` thread | `handlers.py` | ✅ Correct |
+| `watchfiles` reload loop — log files moved to `/logs/` (outside `/app/`), `watchfiles` logger silenced at WARNING | `config/settings/local.py`, `local.yml` | ✅ Correct |
+
+---
+
+## Critical Issues (Must Fix)
+
+### 1. Cross-request PHI data leak — shared `self.log_data` in middleware
+**File:** `middleware.py:127–140`
+
+`self.log_data` is constructed once in `__init__` and stored on the middleware instance. Middleware instances are shared across all requests. Under concurrent ASGI (multiple coroutines on one instance), request A's `user_id`, `request_repr`, `response_repr` overwrite request B's in the same dict — PHI from one patient's session can be written into another patient's audit record.
+
+**Fix:** Build `log_data` as a local dict inside `__call__` and `__acall__`, not on `self`.
+
+```python
+# Before (in __init__)
+self.log_data = {"service_name": SERVICE_NAME, ...}
+
+# After (top of __call__ and __acall__)
+log_data = {"service_name": SERVICE_NAME, ...}
+```
+
+---
+
+### 2. `QuerySet.bulk_create/bulk_update` monkey-patched N times — wrappers nest
+**File:** `signals.py:129–130, 244–247`
+
+Inside `patch_model_event`, the patching logic does:
+```python
+original_bulk_create = models.QuerySet.bulk_create   # captured per call
+...
+models.QuerySet.bulk_create = bulk_create_with_signals  # assigned globally
+```
+This runs once per audited model. The second model captures the already-patched
+method as its "original", the third wraps the double-patched version, and so on.
+After N models are patched, every `bulk_create` call traverses N nested wrappers.
+
+**Fix:** Extract the `QuerySet` patches into a separate function called exactly once from `setup_model_signals()`, before the per-model loop.
+
+---
+
+### 3. `get_calling_model()` frame-walking is broken — bulk auditing is non-functional
+**File:** `signals.py:57–79, 193–196, 225–228`
+
+```python
+if calling_model == model_class.__name__:
+```
+`calling_model` is `module_name.split(".")[-1]` (e.g. `"views"`, `"tasks"`).
+`model_class.__name__` is a class name (e.g. `"Patient"`, `"Appointment"`).
+These will almost never match, so bulk audit logs are silently never written for virtually all models.
+
+**Fix:** Rethink bulk auditing using Django signals (`post_bulk_create` is unavailable natively, but a custom `QuerySet` mixin on a project-level base queryset is more reliable than frame introspection).
+
+---
+
+### 4. `request.body` access can crash real requests
+**File:** `middleware.py:165–170, 229–234`
+
+Only `json.JSONDecodeError` is caught. The following are unhandled and will 500 the request:
+- `RawPostDataException` — body stream already consumed by DRF/multipart parsers
+- `RequestDataTooBig` — body exceeds `DATA_UPLOAD_MAX_MEMORY_SIZE`
+- `UnicodeDecodeError` — binary upload body
+
+Audit middleware must never break actual requests.
+
+**Fix:**
+```python
+try:
+    body = json.loads(request.body)
+    request_data["body"] = body
+except Exception:
+    pass  # never let audit logging crash the request
+```
+
+---
+
+## Warnings (Should Fix)
+
+### 5. PHI/SSN written into `audit.log` via `instance_to_dict`
+**File:** `signals.py:117–118`
+
+`model_to_dict(instance, fields=[f.name for f in instance._meta.fields])` dumps all
+concrete fields including encrypted `social_security_number` and other PHI directly
+into the audit log file. This directly violates HIPAA rules documented in `CLAUDE.md`:
+> "Never log PHI to console" / "SSN handling — only access via `get_ssn` endpoint"
+
+**Fix:** Add a per-model field deny-list or use a model-level `AUDIT_EXCLUDE_FIELDS`
+attribute. At minimum, exclude all `EncryptedField` instances.
+
+---
+
+### 6. Regex patterns recompiled on every request in `should_log_url`
+**File:** `middleware.py:84–100`
+
+```python
+for unregistered_url in UNREGISTERED_URLS:
+    pattern = re.compile(unregistered_url)   # compiled on every request
+```
+This is in the hot path for every HTTP request.
+
+**Fix:** Compile once at module load:
+```python
+_UNREGISTERED_PATTERNS = [re.compile(p) for p in UNREGISTERED_URLS]
+_REGISTERED_PATTERNS   = [re.compile(p) for p in REGISTERED_URLS]
+```
+
+---
+
+### 7. `clear_request()` not called on early return — thread-local leaks
+**File:** `middleware.py:148–153`
+
+```python
+set_current_request(request)
+set_request_id(request_id)
+
+if not should_log_url(request.path):
+    return self.get_response(request)   # clear_request() never called
+```
+The thread-local `request` and `request_id` persist on the worker thread and
+are seen by the next request handled by that thread.
+
+**Fix:** Use `try/finally`:
+```python
+set_current_request(request)
+set_request_id(request_id)
+try:
+    if not should_log_url(request.path):
+        return self.get_response(request)
+    ...
+finally:
+    clear_request()
+```
+
+---
+
+### 8. Bulk wrappers never call `should_audit` — unregistered models audited on bulk paths
+**File:** `signals.py:179–242`
+
+`bulk_create_with_signals` and `bulk_update_with_signals` have no `should_audit` guard,
+so `Session`, `Permission`, `Migration` etc. are audited on bulk paths but excluded on
+single-save paths. Inconsistent and wasteful.
+
+---
+
+### 9. `SFTPClient.upload` — unconditional assignment clobbers success and real errors
+**File:** `protocols.py:209–237`
+
+```python
+if not error:
+    try:
+        ...upload...
+    except Exception:
+        self.log_payload["error_message"] = str(e)   # real error
+
+# This runs unconditionally — overwrites both the success case and the real exception:
+self.log_payload["error_message"] = f"Path validation failed. Error: {str(error)}"
+```
+On a fully successful upload, `error_message` is set to `"Path validation failed. Error: None"`.
+
+**Fix:** Wrap the final assignment in an `else` block.
+
+---
+
+### 10. `HTTPClient.request` returns `None` on exception and calls `.json()` unconditionally
+**File:** `protocols.py:97–118`
+
+```python
+except Exception:
+    ...
+    return response   # response is None here
+```
+Callers expecting a `requests.Response` will raise `AttributeError`. On the success path,
+`response.json()` is called regardless of `Content-Type`, which will raise for non-JSON
+responses and discard the real response object.
+
+---
+
+### 11. `push_usage_log` imports `get_user_details` through `signals` — unwanted side effect
+**File:** `utils.py:139`
+
+```python
+from .signals import get_user_details
+```
+Importing `signals` triggers `setup_model_signals()` at the bottom of that module as a
+side effect. Import directly from the canonical source:
+```python
+from .middleware import get_user_details
+```
+
+---
+
+### 12. Sentry filter attached at settings-load time — before Sentry is initialised
+**File:** `settings.py:24–27`
+
+```python
+root_logger = logging.getLogger()
+for handler in root_logger.handlers:
+    if handler.__class__.__name__.startswith("Sentry"):
+        handler.addFilter(AuditToSentryFilter())
+```
+At settings-load time Sentry SDK has not yet added its handler, so this loop finds nothing
+and the filter is never attached. Move to `apps.ready()`.
+
+---
+
+## Suggestions (Consider)
+
+### 13. `default_app_config` is dead
+**File:** `__init__.py:1`
+
+Deprecated in Django 3.2, removed in Django 4.1. It is a no-op. Remove it.
+
+---
+
+### 14. Timestamp formatting duplicated across all 4 formatters
+**File:** `formatters.py:42–44` (and 3 other formatters)
+
+```python
+datetime.datetime.fromtimestamp(record.created).strftime(self.timestamp_format)[:-3]
+```
+This block is copy-pasted in `JsonFormatter`, `APIFormatter`, `AuditFormatter`, and
+`LoginFormatter`. Also uses server local time with no timezone. Extract a shared:
+```python
+class BaseAuditFormatter(logging.Formatter):
+    def _timestamp(self, record) -> str:
+        return datetime.datetime.fromtimestamp(
+            record.created, tz=datetime.timezone.utc
+        ).strftime(self.timestamp_format)[:-3]
+```
+
+---
+
+### 15. `apps.ready()` no-op attribute accesses are misleading
+**File:** `apps.py:14–16`
+
+```python
+logger_levels.AUDIT
+logger_levels.API
+logger_levels.LOGIN
+```
+These are bare attribute reads that do nothing — the level registration already
+happened when `logger_levels` was imported. The import itself is the side effect.
+Remove the three lines or replace with a comment.
+
+---
+
+### 16. `AuditToSentryFilter` mutates `record.levelname` — corrupts file logs
+**File:** `settings.py:8–20`
+
+Overwriting `record.levelname = "INFO"` means formatters downstream will log `"INFO"`
+instead of `"AUDIT"` / `"API"` in file logs too (filters run before formatting).
+If Sentry integration is needed, use a `before_send` hook in Sentry config instead.
+
+---
+
+### 17. Minor style issues
+
+| Location | Issue |
+|----------|-------|
+| `middleware.py:64` | `id = str(user.id)` shadows the builtin `id` |
+| `protocols.py:193` | `f"Connection not established"` — f-string with no placeholders |
+| `signals.py:88` | `instance_repr: str` type hint but callers pass `dict` |
+| `middleware.py:21–25` | `MockRequest.__init__` passes `*args, **kwargs` to `object.__init__` which rejects them |
+| `formatters.py`, `signals.py` | Several lines exceed 80-char limit — run `make format` |
+
+---
+
+## Priority Order
+
+| Priority | Issue | File |
+|----------|-------|------|
+| 🔴 P0 | Cross-request PHI data leak (`self.log_data`) | `middleware.py` |
+| 🔴 P0 | PHI/SSN in `audit.log` via `instance_to_dict` | `signals.py` |
+| 🔴 P0 | `request.body` access can crash requests | `middleware.py` |
+| 🟠 P1 | `QuerySet` patch nested N times | `signals.py` |
+| 🟠 P1 | Bulk auditing effectively broken (`get_calling_model`) | `signals.py` |
+| 🟠 P1 | `clear_request()` missing on early return | `middleware.py` |
+| 🟡 P2 | Regex recompiled on every request | `middleware.py` |
+| 🟡 P2 | Bulk wrappers skip `should_audit` | `signals.py` |
+| 🟡 P2 | `SFTPClient.upload` error message clobbered | `protocols.py` |
+| 🟡 P2 | `HTTPClient.request` returns `None` silently | `protocols.py` |
+| 🟡 P2 | `push_usage_log` imports via `signals` | `utils.py` |
+| 🟡 P2 | Sentry filter never attaches | `settings.py` |
+| 🔵 P3 | Timestamp duplication, `default_app_config`, style nits | various |

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/activity_audit/apps.py

@@ -15,5 +15,8 @@ class AuditLoggingConfig(AppConfig):
         logger_levels.API
         logger_levels.LOGIN
 
+        # Populate UNREGISTERED_CLASSES (requires app registry to be ready)
+        from . import unregistered  # noqa
+
         # Initialize signals
         from . import signals  # noqa

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/activity_audit/formatters.py

@@ -47,6 +47,7 @@ class JsonFormatter(logging.Formatter):
             "path": record.pathname,
             "module": record.module,
             "function": record.funcName,
+            "request_id": getattr(record, "request_id", "") or "",
             "message": record.getMessage(),
             "exception": "",
             # "extra": {},
@@ -86,6 +87,7 @@ class APIFormatter(logging.Formatter):
             "service_name",
             "request_type",
             "protocol",
+            "request_id",
             "user_id",
             "user_info",
             "request_repr",
@@ -117,6 +119,7 @@ class AuditFormatter(logging.Formatter):
         audit_fields = [
             "model",
             "event_type",
+            "request_id",
             "instance_id",
             "instance_repr",
             "user_id",

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/activity_audit/handlers.py

@@ -4,6 +4,7 @@ import queue
 from logging.handlers import QueueHandler, QueueListener, RotatingFileHandler
 
 from .formatters import APIFormatter, AuditFormatter, JsonFormatter, LoginFormatter
+from .middleware import get_request_id
 
 
 class BaseAuditHandler(RotatingFileHandler):
@@ -108,6 +109,13 @@ class AsyncBaseAuditHandler(QueueHandler):
         )
         self._listener.start()
 
+    def prepare(self, record):
+        # Capture request_id in the calling thread before the record is queued,
+        # since thread-locals are not accessible from the QueueListener thread.
+        record = super().prepare(record)
+        record.request_id = get_request_id() or ""
+        return record
+
     def close(self):
         self._listener.stop()
         super().close()
@@ -161,6 +169,11 @@ class AsyncJsonHandler(QueueHandler):
         )
         self._listener.start()
 
+    def prepare(self, record):
+        record = super().prepare(record)
+        record.request_id = get_request_id() or ""
+        return record
+
     def close(self):
         self._listener.stop()
         super().close()

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/activity_audit/middleware.py

@@ -3,6 +3,7 @@ import json
 import logging
 import re
 import time
+import uuid
 
 from asgiref.local import Local
 from asgiref.sync import iscoroutinefunction, markcoroutinefunction, sync_to_async
@@ -32,6 +33,14 @@ def set_current_request(request):
     _thread_locals.request = request
 
 
+def get_request_id():
+    return getattr(_thread_locals, "request_id", None)
+
+
+def set_request_id(request_id):
+    _thread_locals.request_id = request_id
+
+
 def get_current_user():
     request = get_current_request()
     if request:
@@ -68,6 +77,8 @@ def get_user_details():
 def clear_request():
     with contextlib.suppress(AttributeError):
         del _thread_locals.request
+    with contextlib.suppress(AttributeError):
+        del _thread_locals.request_id
 
 
 def should_log_url(url):
@@ -119,6 +130,7 @@ class AuditLoggingMiddleware(MiddlewareMixin):
             "service_name": SERVICE_NAME,
             "request_type": REQUEST_TYPES[0],
             "protocol": None,
+            "request_id": "",
             "user_id": "",
             "user_info": {},
             "request_repr": {},
@@ -134,6 +146,8 @@ class AuditLoggingMiddleware(MiddlewareMixin):
         if iscoroutinefunction(self):
             return self.__acall__(request)
         set_current_request(request)
+        request_id = str(uuid.uuid4())
+        set_request_id(request_id)
 
         if not should_log_url(request.path):
             return self.get_response(request)
@@ -184,6 +198,7 @@ class AuditLoggingMiddleware(MiddlewareMixin):
 
         self.log_data["execution_time"] = end_time - start_time
         self.log_data["protocol"] = "https" if request.is_secure() else "http"
+        self.log_data["request_id"] = request_id
         self.log_data["request_repr"] = request_data
         self.log_data["response_repr"] = response_data
 
@@ -195,6 +210,8 @@ class AuditLoggingMiddleware(MiddlewareMixin):
 
     async def __acall__(self, request):
         set_current_request(request)
+        request_id = str(uuid.uuid4())
+        set_request_id(request_id)
 
         if not should_log_url(request.path):
             return await self.get_response(request)
@@ -247,6 +264,7 @@ class AuditLoggingMiddleware(MiddlewareMixin):
 
         self.log_data["execution_time"] = end_time - start_time
         self.log_data["protocol"] = "https" if request.is_secure() else "http"
+        self.log_data["request_id"] = request_id
         self.log_data["request_repr"] = request_data
         self.log_data["response_repr"] = response_data
 

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/activity_audit/settings.py

@@ -1,12 +1,6 @@
 import logging
 
-from django.apps import apps
 from django.conf import settings
-from django.contrib.auth.models import Permission
-from django.contrib.contenttypes.models import ContentType
-from django.contrib.sessions.models import Session
-from django.db.migrations import Migration
-from django.db.migrations.recorder import MigrationRecorder
 
 
 # Handles AUDIT/API level as INFO for Sentry
@@ -32,43 +26,6 @@ for handler in root_logger.handlers:
     if handler.__class__.__name__.startswith("Sentry"):
         handler.addFilter(AuditToSentryFilter())
 
-UNREGISTERED_CLASSES = [
-    Migration,
-    Session,
-    Permission,
-    ContentType,
-    MigrationRecorder.Migration,
-]
-
-# Remove silk models audit logging
-SILK_INSTALLED = apps.is_installed("silk")
-if SILK_INSTALLED:
-    from silk.models import (
-        BaseProfile,
-        Profile,
-        Request,
-        Response,
-        SQLQuery,
-        SQLQueryManager,
-    )
-
-    UNREGISTERED_CLASSES.extend(
-        [
-            Request,
-            Response,
-            SQLQueryManager,
-            SQLQuery,
-            BaseProfile,
-            Profile,
-        ]
-    )
-
-# Import and unregister LogEntry class only if Django Admin app is installed
-if apps.is_installed("django.contrib.admin"):
-    from django.contrib.admin.models import LogEntry
-
-    UNREGISTERED_CLASSES.extend([LogEntry])
-
 # URL patterns to exclude from logging
 UNREGISTERED_URLS = [r"^/admin/", r"^/static/", r"^/favicon.ico$"]
 UNREGISTERED_URLS = getattr(

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/activity_audit/signals.py

@@ -15,8 +15,8 @@ from django.db.models.signals import (
 from django.dispatch import receiver
 from django.forms.models import model_to_dict
 
-from activity_audit.middleware import get_user_details
-from activity_audit.settings import UNREGISTERED_CLASSES
+from activity_audit.middleware import get_request_id, get_user_details
+from activity_audit.unregistered import UNREGISTERED_CLASSES
 
 logger = logging.getLogger("audit.model")
 
@@ -96,6 +96,7 @@ def push_log(
             "model": model,
             "instance_id": str(instance_id),
             "event_type": event_type,
+            "request_id": get_request_id() or "",
             "user_id": user_id,
             "user_info": user_info,
             "instance_repr": instance_repr,

django_activity_audit-1.3.0.dev12/activity_audit/unregistered.py

@@ -0,0 +1,33 @@
+from django.apps import apps
+from django.contrib.auth.models import Permission
+from django.contrib.contenttypes.models import ContentType
+from django.contrib.sessions.models import Session
+from django.db.migrations import Migration
+from django.db.migrations.recorder import MigrationRecorder
+
+UNREGISTERED_CLASSES = [
+    Migration,
+    Session,
+    Permission,
+    ContentType,
+    MigrationRecorder.Migration,
+]
+
+if apps.is_installed("silk"):
+    from silk.models import (
+        BaseProfile,
+        Profile,
+        Request,
+        Response,
+        SQLQuery,
+        SQLQueryManager,
+    )
+
+    UNREGISTERED_CLASSES.extend(
+        [Request, Response, SQLQueryManager, SQLQuery, BaseProfile, Profile]
+    )
+
+if apps.is_installed("django.contrib.admin"):
+    from django.contrib.admin.models import LogEntry
+
+    UNREGISTERED_CLASSES.extend([LogEntry])

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12/docs}/django-activity-audit.md

@@ -620,6 +620,87 @@ INSERT INTO rbp_stag_logs.audit FORMAT JSONEachRow
 
 ---
 
-## Future Scope
+## Part E: Async Handlers
 
-- Async implementation using **`QueueListener`** and **`QueueHandler`**.
+All four handler types have non-blocking async variants. Each async handler subclasses `QueueHandler` — the calling (request) thread only enqueues the log record, and a dedicated background thread owned by a `QueueListener` performs the actual file I/O.
+
+### How It Works
+
+```
+Request thread          Background thread
+──────────────          ─────────────────
+emit(record)
+  └─ queue.put()  ───►  QueueListener.dequeue()
+  (returns immediately)   └─ RotatingFileHandler.emit()
+                               └─ write to file + rotate if needed
+```
+
+### Async Handler Classes
+
+| Async Handler | Wraps | Formatter |
+|---|---|---|
+| `AsyncAPILogHandler` | `APILogHandler` | `APIFormatter` |
+| `AsyncAuditLogHandler` | `AuditLogHandler` | `AuditFormatter` |
+| `AsyncLoginLogHandler` | `LoginLogHandler` | `LoginFormatter` |
+| `AsyncJsonHandler` | `RotatingFileHandler` | `JsonFormatter` |
+
+Each handler starts its `QueueListener` thread on `__init__` and stops it cleanly on `close()` (called automatically by Django on shutdown).
+
+### Utility Functions
+
+Async counterparts match the signatures of their sync equivalents:
+
+```python
+from activity_audit import (
+    get_async_json_handler,
+    get_async_api_handler,
+    get_async_audit_handler,
+    get_async_login_handler,
+)
+```
+
+#### `get_async_json_handler`
+
+```python
+get_async_json_handler(
+    level="DEBUG",
+    filename="audit_logs/app.log",
+    max_bytes=1024 * 1024 * 10,  # 10MB
+    backup_count=5,
+)
+```
+
+> Note: unlike `get_json_handler`, there is no `formatter` parameter — `JsonFormatter` is embedded directly on the inner handler so it applies on the background thread.
+
+#### `get_async_api_handler`
+
+```python
+get_async_api_handler(filename="audit_logs/api.log")
+```
+
+#### `get_async_audit_handler`
+
+```python
+get_async_audit_handler(filename="audit_logs/audit.log")
+```
+
+#### `get_async_login_handler`
+
+```python
+get_async_login_handler(filename="audit_logs/login.log")
+```
+
+### Example Django LOGGING Configuration
+
+```python
+LOGGING = {
+    "version": 1,
+    "handlers": {
+        "app_file": get_async_json_handler(level="INFO", filename="audit_logs/app.log"),
+        "api_file": get_async_api_handler(filename="audit_logs/api.log"),
+        "audit_file": get_async_audit_handler(filename="audit_logs/audit.log"),
+        "login_file": get_async_login_handler(filename="audit_logs/login.log"),
+    },
+    ...
+}
+```

{django_activity_audit-1.3.0.dev10 → django_activity_audit-1.3.0.dev12}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "django-activity-audit"
-version = "1.3.0.dev10"
+version = "1.3.0.dev12"
 description = "A Django package for easy CRUD operation logging and container logs"
 authors = [
     { name = "Shreeshan", email = "shreeshan256@gmail.com" }