django-field-audit 1.2.4__tar.gz → 1.2.5__tar.gz

{django-field-audit-1.2.4 → django-field-audit-1.2.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: django-field-audit
-Version: 1.2.4
+Version: 1.2.5
 Summary: Audit Field Changes on Django Models
 Home-page: https://github.com/dimagi/django-field-audit
 Maintainer: Joel Miller
@@ -151,13 +151,16 @@ details:
 - Specifying `audit_special_queryset_writes=True` (step **1** above) without
   setting the default manager to an instance of `AuditingManager` (step **2**
   above) will raise an exception when the model class is evaluated.
-- At this time, only the `QuerySet.delete()` and `QuerySet.update()` "special"
-  write methods can actually perform change auditing when called with 
-  `audit_action=AuditAction.AUDIT`. The other two methods are currently not
-  implemented and will raise `NotImplementedError` if called with that action.
-  Implementing these remaining methods remains a task for the future, see
-  **TODO** below. All four methods do support `audit_action=AuditAction.IGNORE`
-  usage, however.
+- At this time, `QuerySet.delete()`, `QuerySet.update()`,
+  and `QuerySet.bulk_create()` "special" write methods can actually perform
+  change auditing when called with `audit_action=AuditAction.AUDIT`. 
+  `QuerySet.bulk_update()` is not currently implemented and will raise
+  `NotImplementedError` if called with that action. Implementing this remaining
+  method remains a task for the future, see **TODO** below. All four methods do
+  support `audit_action=AuditAction.IGNORE` usage, however.
+- All audited methods use transactions to ensure changes to audited models
+  are only committed to the database if audit events are successfully created
+  and saved as well.
 
 #### Bootstrap events for models with existing records
 
@@ -276,11 +279,9 @@ twine upload dist/*
 ## TODO
 
 - Implement auditing for the remaining "special" QuerySet write operations:
-  - `bulk_create()`
   - `bulk_update()`
 - Write full library documentation using github.io.
 - Switch to `pytest` to support Python 3.10.
-- Wrap audited DB write methods in Django's transaction.atomic context manager
 
 ### Backlog
 

{django-field-audit-1.2.4 → django-field-audit-1.2.5}/README.md

@@ -124,13 +124,16 @@ details:
 - Specifying `audit_special_queryset_writes=True` (step **1** above) without
   setting the default manager to an instance of `AuditingManager` (step **2**
   above) will raise an exception when the model class is evaluated.
-- At this time, only the `QuerySet.delete()` and `QuerySet.update()` "special"
-  write methods can actually perform change auditing when called with 
-  `audit_action=AuditAction.AUDIT`. The other two methods are currently not
-  implemented and will raise `NotImplementedError` if called with that action.
-  Implementing these remaining methods remains a task for the future, see
-  **TODO** below. All four methods do support `audit_action=AuditAction.IGNORE`
-  usage, however.
+- At this time, `QuerySet.delete()`, `QuerySet.update()`,
+  and `QuerySet.bulk_create()` "special" write methods can actually perform
+  change auditing when called with `audit_action=AuditAction.AUDIT`. 
+  `QuerySet.bulk_update()` is not currently implemented and will raise
+  `NotImplementedError` if called with that action. Implementing this remaining
+  method remains a task for the future, see **TODO** below. All four methods do
+  support `audit_action=AuditAction.IGNORE` usage, however.
+- All audited methods use transactions to ensure changes to audited models
+  are only committed to the database if audit events are successfully created
+  and saved as well.
 
 #### Bootstrap events for models with existing records
 
@@ -249,11 +252,9 @@ twine upload dist/*
 ## TODO
 
 - Implement auditing for the remaining "special" QuerySet write operations:
-  - `bulk_create()`
   - `bulk_update()`
 - Write full library documentation using github.io.
 - Switch to `pytest` to support Python 3.10.
-- Wrap audited DB write methods in Django's transaction.atomic context manager
 
 ### Backlog
 

{django-field-audit-1.2.4 → django-field-audit-1.2.5}/django_field_audit.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: django-field-audit
-Version: 1.2.4
+Version: 1.2.5
 Summary: Audit Field Changes on Django Models
 Home-page: https://github.com/dimagi/django-field-audit
 Maintainer: Joel Miller
@@ -151,13 +151,16 @@ details:
 - Specifying `audit_special_queryset_writes=True` (step **1** above) without
   setting the default manager to an instance of `AuditingManager` (step **2**
   above) will raise an exception when the model class is evaluated.
-- At this time, only the `QuerySet.delete()` and `QuerySet.update()` "special"
-  write methods can actually perform change auditing when called with 
-  `audit_action=AuditAction.AUDIT`. The other two methods are currently not
-  implemented and will raise `NotImplementedError` if called with that action.
-  Implementing these remaining methods remains a task for the future, see
-  **TODO** below. All four methods do support `audit_action=AuditAction.IGNORE`
-  usage, however.
+- At this time, `QuerySet.delete()`, `QuerySet.update()`,
+  and `QuerySet.bulk_create()` "special" write methods can actually perform
+  change auditing when called with `audit_action=AuditAction.AUDIT`. 
+  `QuerySet.bulk_update()` is not currently implemented and will raise
+  `NotImplementedError` if called with that action. Implementing this remaining
+  method remains a task for the future, see **TODO** below. All four methods do
+  support `audit_action=AuditAction.IGNORE` usage, however.
+- All audited methods use transactions to ensure changes to audited models
+  are only committed to the database if audit events are successfully created
+  and saved as well.
 
 #### Bootstrap events for models with existing records
 
@@ -276,11 +279,9 @@ twine upload dist/*
 ## TODO
 
 - Implement auditing for the remaining "special" QuerySet write operations:
-  - `bulk_create()`
   - `bulk_update()`
 - Write full library documentation using github.io.
 - Switch to `pytest` to support Python 3.10.
-- Wrap audited DB write methods in Django's transaction.atomic context manager
 
 ### Backlog
 

{django-field-audit-1.2.4 → django-field-audit-1.2.5}/field_audit/__init__.py

@@ -1,3 +1,3 @@
 from .field_audit import audit_fields  # noqa: F401
 
-__version__ = "1.2.4"
+__version__ = "1.2.5"

{django-field-audit-1.2.4 → django-field-audit-1.2.5}/field_audit/field_audit.py

@@ -1,7 +1,7 @@
 import contextvars
 from functools import wraps
 
-from django.db import models
+from django.db import models, router, transaction
 
 from .utils import get_fqcn
 
@@ -116,15 +116,18 @@ def _decorate_db_write(func):
         # - https://stackoverflow.com/questions/907695/
         is_create = is_save and self._state.adding
         object_pk = self.pk if is_delete else None
-        ret = func(self, *args, **kw)
-        AuditEvent.audit_field_changes(
-            self,
-            is_create,
-            is_delete,
-            request.get(),
-            object_pk,
-        )
-        return ret
+
+        db = router.db_for_write(type(self))
+        with transaction.atomic(using=db):
+            ret = func(self, *args, **kw)
+            AuditEvent.audit_field_changes(
+                self,
+                is_create,
+                is_delete,
+                request.get(),
+                object_pk,
+            )
+            return ret
     is_save = func.__name__ == "save"
     is_delete = func.__name__ == "delete"
     if not is_save and not is_delete:

{django-field-audit-1.2.4 → django-field-audit-1.2.5}/field_audit/management/commands/bootstrap_field_audit_events.py

@@ -86,7 +86,7 @@ class Command(BaseCommand):
             stream.write(f"done ({count})")
 
     def do_bootstrap(self, model_class, bootstrap_method, **bootstrap_kw):
-        field_names = self.get_field_names(model_class)
+        field_names = AuditEvent.field_names(model_class)
         if not field_names:
             raise CommandError(
                 f"invalid fields ({field_names!r}) for model: {model_class}"
@@ -103,14 +103,6 @@ class Command(BaseCommand):
         "top-up": top_up_missing,
     }
 
-    @staticmethod
-    def get_field_names(model_class):
-        """Extract the field names from a model class.
-
-        TODO: expose a method on the AuditEvent class for doing this.
-        """
-        return AuditEvent._field_names(model_class())
-
     @contextmanager
     def bootstrap_action_log(self, *args, **kw):
         end = kw.pop("end", "\n")

{django-field-audit-1.2.4 → django-field-audit-1.2.5}/field_audit/models.py

@@ -4,7 +4,7 @@ from functools import wraps
 from itertools import islice
 
 from django.conf import settings
-from django.db import models, router, transaction
+from django.db import models, transaction
 
 from .const import BOOTSTRAP_BATCH_SIZE
 from .utils import class_import_helper
@@ -236,12 +236,12 @@ class AuditEvent(models.Model):
         setattr(model_class, cls.ATTACH_FIELD_NAMES_AT, field_names)
 
     @classmethod
-    def _field_names(cls, instance):
-        """Returns the audit field names stored on the model instance's class
+    def field_names(cls, model_class):
+        """Returns the audit field names stored on the audited Model class
 
-        :param instance: instance of a Model subclass being audited for changes
+        :param model_class: a Django Model class under audit
         """
-        return getattr(instance.__class__, cls.ATTACH_FIELD_NAMES_AT)
+        return getattr(model_class, cls.ATTACH_FIELD_NAMES_AT)
 
     @staticmethod
     def get_field_value(instance, field_name):
@@ -270,7 +270,7 @@ class AuditEvent(models.Model):
                 f"refusing to overwrite {cls.ATTACH_INIT_VALUES_AT!r} "
                 f"on model instance: {instance}"
             )
-        field_names = cls._field_names(instance)
+        field_names = cls.field_names(instance)
         init_values = {f: cls.get_field_value(instance, f) for f in field_names}
         setattr(instance, cls.ATTACH_INIT_VALUES_AT, init_values)
 
@@ -293,19 +293,79 @@ class AuditEvent(models.Model):
 
     @classmethod
     def audit_field_changes(cls, *args, **kw):
-        """Convenience method that calls ``make_audit_event()`` and saves the
-        event (if one is returned).
+        """Convenience method that calls ``make_audit_event_from_instance()``
+        and saves the event (if one is returned).
 
-        All [keyword] arguments are passed directly to ``make_audit_event()``,
-        see that method for usage.
+        All [keyword] arguments are passed directly to
+        ``make_audit_event_from_instance()``, see that method for usage.
         """
-        event = cls.make_audit_event(*args, **kw)
+        event = cls.make_audit_event_from_instance(*args, **kw)
         if event is not None:
             event.save()
 
     @classmethod
-    def make_audit_event(cls, instance, is_create, is_delete,
-                         request, object_pk=None, init_values=None):
+    def get_delta_from_instance(cls, instance, is_create, is_delete):
+        """
+        Returns a dictionary representing the delta of an instance of a model
+        being audited for changes.
+
+        :param instance: instance of a Model subclass to be audited for changes
+        :param is_create: whether or not the audited event creates a new DB
+            record (setting ``True`` implies that ``instance`` is changing)
+        :param is_delete: whether or not the audited event deletes an existing
+            DB record (setting ``True`` implies that ``instance`` is changing)
+        :returns: {field_name: {'old': old_value, 'new': new_value}, ...}
+        :raises: ``AssertionError`` if both is_create and is_delete are true
+        """
+        assert not (is_create and is_delete),\
+            "is_create and is_delete cannot both be true"
+        fields_to_audit = cls.field_names(instance)
+        # fetch (and reset for next db write operation) initial values
+        old_values = {} if is_create else cls.reset_initial_values(instance)
+        new_values = {} if is_delete else \
+            {f: cls.get_field_value(instance, f) for f in fields_to_audit}
+        return cls.create_delta(old_values, new_values)
+
+    @staticmethod
+    def create_delta(old_values, new_values):
+        """
+        Compares two dictionaries and creates a delta between the two
+
+        :param old_values: {field_name: field_value, ...} representing the
+        values prior to a change
+        :param new_values: {field_name: field_value, ...} representing the
+        values after a change
+        :returns: {field_name: {'old': old_value, 'new': new_value}, ...}
+        :raises: ``AssertionError`` if both old_values and new_values are empty
+        do not match
+        """
+        assert old_values or new_values, \
+            "Must provide a non-empty value for either old_values or new_values"
+
+        changed_fields = old_values.keys() if old_values else new_values.keys()
+        if old_values and new_values:
+            changed_fields = new_values.keys()
+
+        delta = {}
+        for field_name in changed_fields:
+            if not old_values:
+                delta[field_name] = {"new": new_values[field_name]}
+            elif not new_values:
+                delta[field_name] = {"old": old_values[field_name]}
+            else:
+                try:
+                    old_value = old_values[field_name]
+                except KeyError:
+                    delta[field_name] = {"new": new_values[field_name]}
+                else:
+                    if old_value != new_values[field_name]:
+                        delta[field_name] = {"old": old_value,
+                                             "new": new_values[field_name]}
+        return delta
+
+    @classmethod
+    def make_audit_event_from_instance(cls, instance, is_create, is_delete,
+                                       request, object_pk=None):
         """Factory method for creating a new ``AuditEvent`` for an instance of a
         model that's being audited for changes.
 
@@ -320,9 +380,6 @@ class AuditEvent(models.Model):
             ``is_delete == True``, that is, when the instance itself no longer
             references its pre-delete primary key. It is ambiguous to set this
             when ``is_delete == False``, and doing so will raise an exception.
-        :param init_values: (Optional) dictionary of initial values for audited
-            fields on the provided instance. If not provided, will use
-            AuditEvent.ATTACH_INIT_VALUES_AT attribute.
         :returns: an unsaved ``AuditEvent`` instance (or ``None`` if
             ``instance`` has not changed)
         :raises: ``ValueError`` on invalid use of the ``object_pk`` argument
@@ -333,37 +390,52 @@ class AuditEvent(models.Model):
                     "'object_pk' arg is ambiguous when 'is_delete == False'"
                 )
             object_pk = instance.pk
-        # fetch (and reset for next db write operation) initial values
-        fields_to_audit = init_values.keys() if init_values else \
-            cls._field_names(instance)
-        init_values = init_values or cls.reset_initial_values(instance)
-        delta = {}
-        for field_name in fields_to_audit:
-            value = cls.get_field_value(instance, field_name)
-            if is_create:
-                delta[field_name] = {"new": value}
-            elif is_delete:
-                delta[field_name] = {"old": value}
-            else:
-                try:
-                    init_value = init_values[field_name]
-                except KeyError:
-                    delta[field_name] = {"new": value}
-                else:
-                    if init_value != value:
-                        delta[field_name] = {"old": init_value, "new": value}
+
+        delta = cls.get_delta_from_instance(instance, is_create, is_delete)
+
         if delta:
-            from .auditors import audit_dispatcher
-            from .field_audit import get_audited_class_path
-            change_context = audit_dispatcher.dispatch(request)
-            return cls(
-                object_class_path=get_audited_class_path(type(instance)),
-                object_pk=object_pk,
-                change_context=cls._change_context_db_value(change_context),
-                is_create=is_create,
-                is_delete=is_delete,
-                delta=delta,
-            )
+            return cls.create_audit_event(object_pk, type(instance), delta,
+                                          is_create, is_delete, request)
+
+    @classmethod
+    def make_audit_event_from_values(cls, old_values, new_values, object_pk,
+                                     object_cls, request):
+        """Factory method for creating a new ``AuditEvent`` based on old and new
+        values.
+
+        :param old_values: {field_name: field_value, ...} representing the
+        values prior to a change
+        :param new_values: {field_name: field_value, ...} representing the
+        values after a change
+        :param object_pk: primary key of the instance
+        :param object_cls: class type of the object being audited
+        :param request: the request object responsible for the change (or
+            ``None`` if there is no request)
+        :returns: an unsaved ``AuditEvent`` instance (or ``None`` if
+            no difference between ``old_values`` and ``new_values``)
+        """
+        is_create = not old_values
+        is_delete = not new_values
+        delta = AuditEvent.create_delta(old_values, new_values)
+        if delta:
+            return AuditEvent.create_audit_event(object_pk, object_cls, delta,
+                                                 is_create, is_delete, request)
+
+    @classmethod
+    def create_audit_event(cls, object_pk, object_cls, delta, is_create,
+                           is_delete, request):
+        from .auditors import audit_dispatcher
+        from .field_audit import get_audited_class_path
+        change_context = audit_dispatcher.dispatch(request)
+        object_cls_path = get_audited_class_path(object_cls)
+        return cls(
+            object_class_path=object_cls_path,
+            object_pk=object_pk,
+            change_context=cls._change_context_db_value(change_context),
+            is_create=is_create,
+            is_delete=is_delete,
+            delta=delta,
+        )
 
     @classmethod
     def bootstrap_existing_model_records(cls, model_class, field_names,
@@ -490,7 +562,14 @@ def validate_audit_action(func):
     :raises: ``InvalidAuditActionError`` or ``UnsetAuditActionError``
     """
     @wraps(func)
-    def wrapper(self, *args, audit_action=AuditAction.RAISE, **kw):
+    def wrapper(self, *args, **kw):
+        try:
+            audit_action = kw["audit_action"]
+        except KeyError:
+            raise UnsetAuditActionError(
+                f"{type(self).__name__}.{func.__name__}() requires an audit "
+                "action as a keyword argument."
+            )
         if audit_action not in AuditAction:
             raise InvalidAuditActionError(
                 "The 'audit_action' argument must be a value of 'AuditAction', "
@@ -501,7 +580,7 @@ def validate_audit_action(func):
                 f"{type(self).__name__}.{func.__name__}() requires an audit "
                 "action"
             )
-        return func(self, *args, audit_action=audit_action, **kw)
+        return func(self, *args, **kw)
     return wrapper
 
 
@@ -532,13 +611,23 @@ class AuditingQuerySet(models.QuerySet):
     """
 
     @validate_audit_action
-    def bulk_create(self, *args, audit_action=AuditAction.RAISE, **kw):
-        if audit_action is AuditAction.IGNORE:
-            return super().bulk_create(*args, **kw)
-        else:
-            raise NotImplementedError(
-                "Change auditing is not implemented for bulk_create()."
-            )
+    def bulk_create(self, objs, *, audit_action=AuditAction.RAISE, **kw):
+        if audit_action is AuditAction.IGNORE or not objs:
+            return super().bulk_create(objs, **kw)
+        assert audit_action is AuditAction.AUDIT, audit_action
+
+        from .field_audit import request
+        request = request.get()
+
+        with transaction.atomic(using=self.db):
+            created_objs = super().bulk_create(objs, **kw)
+            audit_events = []
+            for obj in created_objs:
+                audit_events.append(
+                    AuditEvent.make_audit_event_from_instance(
+                        obj, True, False, request))
+            AuditEvent.objects.bulk_create(audit_events)
+            return created_objs
 
     @validate_audit_action
     def bulk_update(self, *args, audit_action=AuditAction.RAISE, **kw):
@@ -557,52 +646,66 @@ class AuditingQuerySet(models.QuerySet):
         from .field_audit import request
         request = request.get()
         audit_events = []
-        for instance in self:
-            # make_audit_event() will never return None because delete=True
-            audit_events.append(AuditEvent.make_audit_event(
-                instance,
-                False,
-                True,
-                request,
-                instance.pk,
-            ))
-        value = super().delete()
-        if audit_events:
-            # write the audit events _after_ the delete succeeds
-            AuditEvent.objects.bulk_create(audit_events)
-        return value
+        fields_to_fetch = set(AuditEvent.field_names(self.model)) | {'pk'}
+        current_values = {}
+        for values_for_instance in self.values(*fields_to_fetch):
+            pk = values_for_instance.pop('pk')
+            current_values[pk] = values_for_instance
+
+        for pk, current_values_for_pk in current_values.items():
+            audit_event = AuditEvent.make_audit_event_from_values(
+                current_values_for_pk,
+                {},
+                pk,
+                self.model,
+                request
+            )
+            audit_events.append(audit_event)
+
+        with transaction.atomic(using=self.db):
+            value = super().delete()
+            if audit_events:
+                # write the audit events _after_ the delete succeeds
+                AuditEvent.objects.bulk_create(audit_events)
+            return value
 
     @validate_audit_action
-    def update(self, audit_action=AuditAction.RAISE, **kw):
+    def update(self, *, audit_action=AuditAction.RAISE, **kw):
+        """
+        In order to determine the old and new values of the records matched by
+        the queryset, a fetch of audited values for the matched records is
+        performed, resulting in one fetch of the current values, one update of
+        the matched records, and one bulk creation of audit events.
+        """
         if audit_action is AuditAction.IGNORE:
             return super().update(**kw)
         assert audit_action is AuditAction.AUDIT, audit_action
 
         fields_to_update = set(kw.keys())
-        audited_fields = set(
-            getattr(self.model, AuditEvent.ATTACH_FIELD_NAMES_AT)
-        )
+        audited_fields = set(AuditEvent.field_names(self.model))
         fields_to_audit = fields_to_update & audited_fields
         if not fields_to_audit:
             # no audited fields are changing
             return super().update(**kw)
 
-        values_to_fetch = fields_to_update | {"pk"}
+        new_values = {field: kw[field] for field in fields_to_audit}
+
         old_values = {}
+        values_to_fetch = fields_to_update | {"pk"}
         for value in self.values(*values_to_fetch):
             pk = value.pop('pk')
             old_values[pk] = value
 
-        with transaction.atomic(using=router.db_for_write(self.model)):
+        with transaction.atomic(using=self.db):
             rows = super().update(**kw)
             # create and write the audit events _after_ the update succeeds
             from .field_audit import request
             request = request.get()
             audit_events = []
-            for instance in self:
-                init_values = old_values[instance.pk]
-                audit_event = AuditEvent.make_audit_event(
-                    instance, False, False, request, init_values=init_values
+
+            for pk, old_values_for_pk in old_values.items():
+                audit_event = AuditEvent.make_audit_event_from_values(
+                    old_values_for_pk, new_values, pk, self.model, request
                 )
                 if audit_event:
                     audit_events.append(audit_event)