django-activity-audit 1.3.0.dev11__tar.gz → 1.3.0.dev13__tar.gz

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: django-activity-audit
-Version: 1.3.0.dev11
+Version: 1.3.0.dev13
 Summary: A Django package for easy CRUD operation logging and container logs
 Project-URL: Homepage, https://github.com/shree256/django-activity-audit
 Project-URL: Repository, https://github.com/shree256/django-activity-audit

django_activity_audit-1.3.0.dev13/activity_audit/REVIEW.md

@@ -0,0 +1,297 @@
+# activity_audit — Code Review
+
+> Reviewed: 2026-06-08  
+> Reviewer: Claude Code  
+> Scope: Full app audit following bug fixes for `AppRegistryNotReady` and `request_id` thread-local propagation
+
+---
+
+## Recent Fixes (Verified Correct)
+
+| Fix | File | Status |
+|-----|------|--------|
+| `AppRegistryNotReady` — model imports deferred to `unregistered.py`, loaded in `apps.ready()` | `unregistered.py`, `apps.py`, `settings.py` | ✅ Correct |
+| `request_id` not written to `app.log` — captured via `prepare()` in handler (producer thread) before record is queued to background `QueueListener` thread | `handlers.py` | ✅ Correct |
+| `watchfiles` reload loop — log files moved to `/logs/` (outside `/app/`), `watchfiles` logger silenced at WARNING | `config/settings/local.py`, `local.yml` | ✅ Correct |
+
+---
+
+## Critical Issues (Must Fix)
+
+### 1. Cross-request PHI data leak — shared `self.log_data` in middleware
+**File:** `middleware.py:127–140`
+
+`self.log_data` is constructed once in `__init__` and stored on the middleware instance. Middleware instances are shared across all requests. Under concurrent ASGI (multiple coroutines on one instance), request A's `user_id`, `request_repr`, `response_repr` overwrite request B's in the same dict — PHI from one patient's session can be written into another patient's audit record.
+
+**Fix:** Build `log_data` as a local dict inside `__call__` and `__acall__`, not on `self`.
+
+```python
+# Before (in __init__)
+self.log_data = {"service_name": SERVICE_NAME, ...}
+
+# After (top of __call__ and __acall__)
+log_data = {"service_name": SERVICE_NAME, ...}
+```
+
+---
+
+### 2. `QuerySet.bulk_create/bulk_update` monkey-patched N times — wrappers nest
+**File:** `signals.py:129–130, 244–247`
+
+Inside `patch_model_event`, the patching logic does:
+```python
+original_bulk_create = models.QuerySet.bulk_create   # captured per call
+...
+models.QuerySet.bulk_create = bulk_create_with_signals  # assigned globally
+```
+This runs once per audited model. The second model captures the already-patched
+method as its "original", the third wraps the double-patched version, and so on.
+After N models are patched, every `bulk_create` call traverses N nested wrappers.
+
+**Fix:** Extract the `QuerySet` patches into a separate function called exactly once from `setup_model_signals()`, before the per-model loop.
+
+---
+
+### 3. `get_calling_model()` frame-walking is broken — bulk auditing is non-functional
+**File:** `signals.py:57–79, 193–196, 225–228`
+
+```python
+if calling_model == model_class.__name__:
+```
+`calling_model` is `module_name.split(".")[-1]` (e.g. `"views"`, `"tasks"`).
+`model_class.__name__` is a class name (e.g. `"Patient"`, `"Appointment"`).
+These will almost never match, so bulk audit logs are silently never written for virtually all models.
+
+**Fix:** Rethink bulk auditing using Django signals (`post_bulk_create` is unavailable natively, but a custom `QuerySet` mixin on a project-level base queryset is more reliable than frame introspection).
+
+---
+
+### 4. `request.body` access can crash real requests
+**File:** `middleware.py:165–170, 229–234`
+
+Only `json.JSONDecodeError` is caught. The following are unhandled and will 500 the request:
+- `RawPostDataException` — body stream already consumed by DRF/multipart parsers
+- `RequestDataTooBig` — body exceeds `DATA_UPLOAD_MAX_MEMORY_SIZE`
+- `UnicodeDecodeError` — binary upload body
+
+Audit middleware must never break actual requests.
+
+**Fix:**
+```python
+try:
+    body = json.loads(request.body)
+    request_data["body"] = body
+except Exception:
+    pass  # never let audit logging crash the request
+```
+
+---
+
+## Warnings (Should Fix)
+
+### 5. PHI/SSN written into `audit.log` via `instance_to_dict`
+**File:** `signals.py:117–118`
+
+`model_to_dict(instance, fields=[f.name for f in instance._meta.fields])` dumps all
+concrete fields including encrypted `social_security_number` and other PHI directly
+into the audit log file. This directly violates HIPAA rules documented in `CLAUDE.md`:
+> "Never log PHI to console" / "SSN handling — only access via `get_ssn` endpoint"
+
+**Fix:** Add a per-model field deny-list or use a model-level `AUDIT_EXCLUDE_FIELDS`
+attribute. At minimum, exclude all `EncryptedField` instances.
+
+---
+
+### 6. Regex patterns recompiled on every request in `should_log_url`
+**File:** `middleware.py:84–100`
+
+```python
+for unregistered_url in UNREGISTERED_URLS:
+    pattern = re.compile(unregistered_url)   # compiled on every request
+```
+This is in the hot path for every HTTP request.
+
+**Fix:** Compile once at module load:
+```python
+_UNREGISTERED_PATTERNS = [re.compile(p) for p in UNREGISTERED_URLS]
+_REGISTERED_PATTERNS   = [re.compile(p) for p in REGISTERED_URLS]
+```
+
+---
+
+### 7. `clear_request()` not called on early return — thread-local leaks
+**File:** `middleware.py:148–153`
+
+```python
+set_current_request(request)
+set_request_id(request_id)
+
+if not should_log_url(request.path):
+    return self.get_response(request)   # clear_request() never called
+```
+The thread-local `request` and `request_id` persist on the worker thread and
+are seen by the next request handled by that thread.
+
+**Fix:** Use `try/finally`:
+```python
+set_current_request(request)
+set_request_id(request_id)
+try:
+    if not should_log_url(request.path):
+        return self.get_response(request)
+    ...
+finally:
+    clear_request()
+```
+
+---
+
+### 8. Bulk wrappers never call `should_audit` — unregistered models audited on bulk paths
+**File:** `signals.py:179–242`
+
+`bulk_create_with_signals` and `bulk_update_with_signals` have no `should_audit` guard,
+so `Session`, `Permission`, `Migration` etc. are audited on bulk paths but excluded on
+single-save paths. Inconsistent and wasteful.
+
+---
+
+### 9. `SFTPClient.upload` — unconditional assignment clobbers success and real errors
+**File:** `protocols.py:209–237`
+
+```python
+if not error:
+    try:
+        ...upload...
+    except Exception:
+        self.log_payload["error_message"] = str(e)   # real error
+
+# This runs unconditionally — overwrites both the success case and the real exception:
+self.log_payload["error_message"] = f"Path validation failed. Error: {str(error)}"
+```
+On a fully successful upload, `error_message` is set to `"Path validation failed. Error: None"`.
+
+**Fix:** Wrap the final assignment in an `else` block.
+
+---
+
+### 10. `HTTPClient.request` returns `None` on exception and calls `.json()` unconditionally
+**File:** `protocols.py:97–118`
+
+```python
+except Exception:
+    ...
+    return response   # response is None here
+```
+Callers expecting a `requests.Response` will raise `AttributeError`. On the success path,
+`response.json()` is called regardless of `Content-Type`, which will raise for non-JSON
+responses and discard the real response object.
+
+---
+
+### 11. `push_usage_log` imports `get_user_details` through `signals` — unwanted side effect
+**File:** `utils.py:139`
+
+```python
+from .signals import get_user_details
+```
+Importing `signals` triggers `setup_model_signals()` at the bottom of that module as a
+side effect. Import directly from the canonical source:
+```python
+from .middleware import get_user_details
+```
+
+---
+
+### 12. Sentry filter attached at settings-load time — before Sentry is initialised
+**File:** `settings.py:24–27`
+
+```python
+root_logger = logging.getLogger()
+for handler in root_logger.handlers:
+    if handler.__class__.__name__.startswith("Sentry"):
+        handler.addFilter(AuditToSentryFilter())
+```
+At settings-load time Sentry SDK has not yet added its handler, so this loop finds nothing
+and the filter is never attached. Move to `apps.ready()`.
+
+---
+
+## Suggestions (Consider)
+
+### 13. `default_app_config` is dead
+**File:** `__init__.py:1`
+
+Deprecated in Django 3.2, removed in Django 4.1. It is a no-op. Remove it.
+
+---
+
+### 14. Timestamp formatting duplicated across all 4 formatters
+**File:** `formatters.py:42–44` (and 3 other formatters)
+
+```python
+datetime.datetime.fromtimestamp(record.created).strftime(self.timestamp_format)[:-3]
+```
+This block is copy-pasted in `JsonFormatter`, `APIFormatter`, `AuditFormatter`, and
+`LoginFormatter`. Also uses server local time with no timezone. Extract a shared:
+```python
+class BaseAuditFormatter(logging.Formatter):
+    def _timestamp(self, record) -> str:
+        return datetime.datetime.fromtimestamp(
+            record.created, tz=datetime.timezone.utc
+        ).strftime(self.timestamp_format)[:-3]
+```
+
+---
+
+### 15. `apps.ready()` no-op attribute accesses are misleading
+**File:** `apps.py:14–16`
+
+```python
+logger_levels.AUDIT
+logger_levels.API
+logger_levels.LOGIN
+```
+These are bare attribute reads that do nothing — the level registration already
+happened when `logger_levels` was imported. The import itself is the side effect.
+Remove the three lines or replace with a comment.
+
+---
+
+### 16. `AuditToSentryFilter` mutates `record.levelname` — corrupts file logs
+**File:** `settings.py:8–20`
+
+Overwriting `record.levelname = "INFO"` means formatters downstream will log `"INFO"`
+instead of `"AUDIT"` / `"API"` in file logs too (filters run before formatting).
+If Sentry integration is needed, use a `before_send` hook in Sentry config instead.
+
+---
+
+### 17. Minor style issues
+
+| Location | Issue |
+|----------|-------|
+| `middleware.py:64` | `id = str(user.id)` shadows the builtin `id` |
+| `protocols.py:193` | `f"Connection not established"` — f-string with no placeholders |
+| `signals.py:88` | `instance_repr: str` type hint but callers pass `dict` |
+| `middleware.py:21–25` | `MockRequest.__init__` passes `*args, **kwargs` to `object.__init__` which rejects them |
+| `formatters.py`, `signals.py` | Several lines exceed 80-char limit — run `make format` |
+
+---
+
+## Priority Order
+
+| Priority | Issue | File |
+|----------|-------|------|
+| 🔴 P0 | Cross-request PHI data leak (`self.log_data`) | `middleware.py` |
+| 🔴 P0 | PHI/SSN in `audit.log` via `instance_to_dict` | `signals.py` |
+| 🔴 P0 | `request.body` access can crash requests | `middleware.py` |
+| 🟠 P1 | `QuerySet` patch nested N times | `signals.py` |
+| 🟠 P1 | Bulk auditing effectively broken (`get_calling_model`) | `signals.py` |
+| 🟠 P1 | `clear_request()` missing on early return | `middleware.py` |
+| 🟡 P2 | Regex recompiled on every request | `middleware.py` |
+| 🟡 P2 | Bulk wrappers skip `should_audit` | `signals.py` |
+| 🟡 P2 | `SFTPClient.upload` error message clobbered | `protocols.py` |
+| 🟡 P2 | `HTTPClient.request` returns `None` silently | `protocols.py` |
+| 🟡 P2 | `push_usage_log` imports via `signals` | `utils.py` |
+| 🟡 P2 | Sentry filter never attaches | `settings.py` |
+| 🔵 P3 | Timestamp duplication, `default_app_config`, style nits | various |

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/__init__.py

@@ -8,7 +8,9 @@ from activity_audit.utils import (
     get_async_login_handler,
     get_audit_handler,
     get_console_formatter,
-    get_json_formatter,
+    get_app_formatter,
+    get_api_formatter,
+    get_audit_formatter,
     get_json_handler,
     get_login_handler,
 )
@@ -17,7 +19,9 @@ from . import logger_levels
 
 __all__ = [
     "get_console_formatter",
-    "get_json_formatter",
+    "get_app_formatter",
+    "get_api_formatter",
+    "get_audit_formatter",
     "get_json_handler",
     "get_api_handler",
     "get_audit_handler",

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/apps.py

@@ -15,5 +15,8 @@ class AuditLoggingConfig(AppConfig):
         logger_levels.API
         logger_levels.LOGIN
 
+        # Populate UNREGISTERED_CLASSES (requires app registry to be ready)
+        from . import unregistered  # noqa
+
         # Initialize signals
         from . import signals  # noqa

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/constants.py

@@ -1,3 +1,13 @@
+import enum
+
+
+class LogType(str, enum.Enum):
+    APP = "app"
+    API = "api"
+    AUDIT = "audit"
+    LOGIN = "login"
+
+
 CONSOLE_FORMAT = (
     "%(levelname)s %(asctime)s %(pathname)s %(module)s %(funcName)s %(message)s"
 )
@@ -6,3 +16,4 @@ REQUEST_TYPES = [
     "internal",
     "external",
 ]
+

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/formatters.py

@@ -4,8 +4,7 @@ import json
 import logging
 import uuid
 
-from activity_audit.middleware import get_request_id
-
+from .constants import LogType
 
 def _json_default(obj):
     """
@@ -31,7 +30,7 @@ def _json_default(obj):
     return str(obj)
 
 
-class JsonFormatter(logging.Formatter):
+class AppFormatter(logging.Formatter):
     def __init__(self, timestamp_format: str = "%Y-%m-%d %H:%M:%S.%f"):
         super().__init__()
         self.timestamp_format = timestamp_format
@@ -49,20 +48,16 @@ class JsonFormatter(logging.Formatter):
             "path": record.pathname,
             "module": record.module,
             "function": record.funcName,
-            "request_id": get_request_id() or "",
+            "request_id": getattr(record, "request_id", "") or "",
             "message": record.getMessage(),
             "exception": "",
-            # "extra": {},
+            "log_type": LogType.APP,
         }
 
         # Add exception info if present for ERROR
         if record.exc_info:
             log_data["exception"] = "{}".format(self.formatException(record.exc_info))
 
-        # Add extra fields if present
-        # if hasattr(record, "extra"):
-        #     log_data.update(record.extra)
-
         return json.dumps(log_data, default=_json_default)
 
 
@@ -82,6 +77,7 @@ class APIFormatter(logging.Formatter):
             "level": record.levelname,
             "name": record.name,
             "message": record.getMessage(),
+            "log_type": LogType.API,
         }
 
         # Add all audit-specific fields if they exist
@@ -116,6 +112,7 @@ class AuditFormatter(logging.Formatter):
             "level": record.levelname,
             "name": record.name,
             "message": record.getMessage(),
+            "log_type": LogType.AUDIT,
         }
 
         audit_fields = [

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/handlers.py

@@ -3,7 +3,8 @@ import queue
 
 from logging.handlers import QueueHandler, QueueListener, RotatingFileHandler
 
-from .formatters import APIFormatter, AuditFormatter, JsonFormatter, LoginFormatter
+from .formatters import APIFormatter, AuditFormatter, AppFormatter, LoginFormatter
+from .middleware import get_request_id
 
 
 class BaseAuditHandler(RotatingFileHandler):
@@ -108,6 +109,13 @@ class AsyncBaseAuditHandler(QueueHandler):
         )
         self._listener.start()
 
+    def prepare(self, record):
+        # Capture request_id in the calling thread before the record is queued,
+        # since thread-locals are not accessible from the QueueListener thread.
+        record = super().prepare(record)
+        record.request_id = get_request_id() or ""
+        return record
+
     def close(self):
         self._listener.stop()
         super().close()
@@ -137,7 +145,7 @@ class AsyncLoginLogHandler(AsyncBaseAuditHandler):
 class AsyncJsonHandler(QueueHandler):
     """
     Non-blocking handler for general JSON logs. Wraps a RotatingFileHandler
-    with JsonFormatter on the background thread.
+    with AppFormatter on the background thread.
     """
 
     def __init__(
@@ -155,12 +163,17 @@ class AsyncJsonHandler(QueueHandler):
         sync_handler = RotatingFileHandler(
             filename, mode, maxBytes, backupCount, encoding, delay
         )
-        sync_handler.setFormatter(JsonFormatter())
+        sync_handler.setFormatter(AppFormatter())
         self._listener = QueueListener(
             log_queue, sync_handler, respect_handler_level=True
         )
         self._listener.start()
 
+    def prepare(self, record):
+        record = super().prepare(record)
+        record.request_id = get_request_id() or ""
+        return record
+
     def close(self):
         self._listener.stop()
         super().close()

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/settings.py

@@ -1,12 +1,6 @@
 import logging
 
-from django.apps import apps
 from django.conf import settings
-from django.contrib.auth.models import Permission
-from django.contrib.contenttypes.models import ContentType
-from django.contrib.sessions.models import Session
-from django.db.migrations import Migration
-from django.db.migrations.recorder import MigrationRecorder
 
 
 # Handles AUDIT/API level as INFO for Sentry
@@ -32,43 +26,6 @@ for handler in root_logger.handlers:
     if handler.__class__.__name__.startswith("Sentry"):
         handler.addFilter(AuditToSentryFilter())
 
-UNREGISTERED_CLASSES = [
-    Migration,
-    Session,
-    Permission,
-    ContentType,
-    MigrationRecorder.Migration,
-]
-
-# Remove silk models audit logging
-SILK_INSTALLED = apps.is_installed("silk")
-if SILK_INSTALLED:
-    from silk.models import (
-        BaseProfile,
-        Profile,
-        Request,
-        Response,
-        SQLQuery,
-        SQLQueryManager,
-    )
-
-    UNREGISTERED_CLASSES.extend(
-        [
-            Request,
-            Response,
-            SQLQueryManager,
-            SQLQuery,
-            BaseProfile,
-            Profile,
-        ]
-    )
-
-# Import and unregister LogEntry class only if Django Admin app is installed
-if apps.is_installed("django.contrib.admin"):
-    from django.contrib.admin.models import LogEntry
-
-    UNREGISTERED_CLASSES.extend([LogEntry])
-
 # URL patterns to exclude from logging
 UNREGISTERED_URLS = [r"^/admin/", r"^/static/", r"^/favicon.ico$"]
 UNREGISTERED_URLS = getattr(

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/signals.py

@@ -16,7 +16,7 @@ from django.dispatch import receiver
 from django.forms.models import model_to_dict
 
 from activity_audit.middleware import get_request_id, get_user_details
-from activity_audit.settings import UNREGISTERED_CLASSES
+from activity_audit.unregistered import UNREGISTERED_CLASSES
 
 logger = logging.getLogger("audit.model")
 

django_activity_audit-1.3.0.dev13/activity_audit/unregistered.py

@@ -0,0 +1,33 @@
+from django.apps import apps
+from django.contrib.auth.models import Permission
+from django.contrib.contenttypes.models import ContentType
+from django.contrib.sessions.models import Session
+from django.db.migrations import Migration
+from django.db.migrations.recorder import MigrationRecorder
+
+UNREGISTERED_CLASSES = [
+    Migration,
+    Session,
+    Permission,
+    ContentType,
+    MigrationRecorder.Migration,
+]
+
+if apps.is_installed("silk"):
+    from silk.models import (
+        BaseProfile,
+        Profile,
+        Request,
+        Response,
+        SQLQuery,
+        SQLQueryManager,
+    )
+
+    UNREGISTERED_CLASSES.extend(
+        [Request, Response, SQLQueryManager, SQLQuery, BaseProfile, Profile]
+    )
+
+if apps.is_installed("django.contrib.admin"):
+    from django.contrib.admin.models import LogEntry
+
+    UNREGISTERED_CLASSES.extend([LogEntry])

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/activity_audit/utils.py

@@ -7,9 +7,19 @@ def get_console_formatter() -> dict:
     }
 
 
-def get_json_formatter() -> dict:
+def get_app_formatter() -> dict:
     return {
-        "()": "activity_audit.formatters.JsonFormatter",
+        "()": "activity_audit.formatters.AppFormatter",
+    }
+
+def get_api_formatter() -> dict:
+    return {
+        "()": "activity_audit.formatters.APIFormatter"
+    }
+
+def get_audit_formatter() -> dict:
+    return {
+        "()": "activity_audit.formatters.AuditFormatter"
     }
 
 
@@ -30,6 +40,7 @@ def get_json_handler(
     }
 
 
+
 def get_api_handler(
     filename: str = "audit_logs/api.log",
 ) -> dict:

{django_activity_audit-1.3.0.dev11 → django_activity_audit-1.3.0.dev13}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "django-activity-audit"
-version = "1.3.0.dev11"
+version = "1.3.0.dev13"
 description = "A Django package for easy CRUD operation logging and container logs"
 authors = [
     { name = "Shreeshan", email = "shreeshan256@gmail.com" }