django-field-audit 1.4.0__py3-none-any.whl

django_field_audit-1.4.0.dist-info/METADATA

@@ -0,0 +1,362 @@
+Metadata-Version: 2.4
+Name: django-field-audit
+Version: 1.4.0
+Summary: Audit Field Changes on Django Models
+Author-email: Joel Miller <jmiller@dimagi.com>, Simon Kelly <simongdkelly@gmail.com>, Graham Herceg <gherceg@dimagi.com>, Chris Smit <chris.smit@dimagi.com>, Daniel Miller <millerdev@gmail.com>
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Web Environment
+Classifier: Framework :: Django
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+License-File: LICENSE
+Requires-Dist: django>=3.2
+Project-URL: Homepage, https://github.com/dimagi/django-field-audit
+Project-URL: Repository, https://github.com/dimagi/django-field-audit
+
+# Audit Field Changes on Django Models
+
+[![tests][tests_badge]][tests_link]
+[![coverage][coverage_badge]][coverage_link]
+[![pypi package][pypi_badge]][pypi_link]
+
+[tests_badge]: https://github.com/dimagi/django-field-audit/actions/workflows/tests.yml/badge.svg
+[tests_link]: https://github.com/dimagi/django-field-audit/actions/workflows/tests.yml
+[coverage_badge]: https://github.com/dimagi/django-field-audit/raw/coverage-badge/coverage.svg
+[coverage_link]: https://github.com/dimagi/django-field-audit/actions/workflows/coverage.yml
+[pypi_badge]: https://badge.fury.io/py/django-field-audit.svg
+[pypi_link]: https://pypi.org/project/django-field-audit/
+
+A Django app for auditing field changes on database models.
+
+## Installation
+```
+pip install django-field-audit
+```
+
+## Documentation
+
+<!--
+The [django-field-audit documentation][docs] shows how to use this library to
+audit field changes on Django Models.
+
+[docs]: https://dimagi.github.io/django-field-audit/
+-->
+
+### Django Settings
+
+To enable the app, add it to your Django `INSTALLED_APPS` configuration and run
+migrations. Settings example:
+
+```python
+INSTALLED_APPS = [
+    # ...
+    "field_audit",
+]
+```
+
+The "auditor chain" (see `FIELD_AUDIT_AUDITORS` in the **Custom settings** table
+below) is configured out of the box with the default auditors. If
+`change_context` auditing is desired for authenticated Django requests, add the
+app middleware to your Django `MIDDLEWARE` configuration. For example:
+
+```python
+MIDDLEWARE = [
+    # ...
+    "field_audit.middleware.FieldAuditMiddleware",
+]
+```
+
+The audit chain can be updated to use custom auditors (subclasses of
+`field_audit.auditors.BaseAuditor`). If `change_context` auditing is not
+desired, the audit chain can be cleared to avoid extra processing:
+
+```python
+FIELD_AUDIT_AUDITORS = []
+```
+
+#### Custom settings details
+
+| Name                              | Description                                                    | Default value when unset
+|:----------------------------------|:---------------------------------------------------------------|:------------------------
+| `FIELD_AUDIT_AUDITEVENT_MANAGER`  | A custom manager to use for the `AuditEvent` Model.            | `field_audit.models.DefaultAuditEventManager`
+| `FIELD_AUDIT_AUDITORS`            | A custom list of auditors for acquiring `change_context` info. | `["field_audit.auditors.RequestAuditor", "field_audit.auditors.SystemUserAuditor"]`
+| `FIELD_AUDIT_SERVICE_CLASS`       | A custom service class for audit logic implementation.         | `field_audit.services.AuditService`
+
+### Custom Audit Service
+
+The audit logic has been extracted into a separate `AuditService` class to improve separation of concerns and enable easier customization of audit behavior. Users can provide custom audit implementations by subclassing `AuditService` and configuring the `FIELD_AUDIT_SERVICE_CLASS` setting.
+
+#### Creating a Custom Audit Service
+
+```python
+# myapp/audit.py
+
+from field_audit import AuditService
+
+class CustomAuditService(AuditService):
+    def get_field_value(self, instance, field_name, bootstrap=False):
+        # Custom logic for extracting field values
+        value = super().get_field_value(instance, field_name, bootstrap)
+        
+        # Example: custom serialization or transformation
+        if field_name == 'sensitive_field':
+            value = '[REDACTED]'
+            
+        return value
+```
+
+Then configure it in your Django settings:
+
+```python
+# settings.py
+
+FIELD_AUDIT_SERVICE_CLASS = 'myapp.audit.CustomAuditService'
+```
+
+#### Backward Compatibility
+
+The original `AuditEvent` class methods are maintained for backward compatibility but are now deprecated in favor of the service-based approach. These methods will issue deprecation warnings and delegate to the configured audit service.
+
+### Model Auditing
+
+To begin auditing Django models, import the `field_audit.audit_fields` decorator
+and decorate models specifying which fields should be audited for changes.
+Example code:
+
+```python
+# flight/models.py
+
+from django.db import models
+from field_audit import audit_fields
+
+
+@audit_fields("tail_number", "make_model", "operated_by")
+class Aircraft(models.Model):
+    id = AutoField(primary_key=True)
+    tail_number = models.CharField(max_length=32, unique=True)
+    make_model = models.CharField(max_length=64)
+    operated_by = models.CharField(max_length=64)
+```
+
+#### Audited DB write operations
+
+By default, Model and QuerySet methods are audited, with the exception of four
+"special" QuerySet methods:
+
+| DB Write Method               | Audited
+|:------------------------------|:-------
+| `Model.delete()`              | Yes
+| `Model.save()`                | Yes
+| `QuerySet.bulk_create()`      | No
+| `QuerySet.bulk_update()`      | No
+| `QuerySet.create()`           | Yes (via `Model.save()`)
+| `QuerySet.delete()`           | No
+| `QuerySet.get_or_create()`    | Yes (via `QuerySet.create()`)
+| `QuerySet.update()`           | No
+| `QuerySet.update_or_create()` | Yes (via `QuerySet.get_or_create()` and `Model.save()`)
+
+#### Auditing Special QuerySet Writes
+
+Auditing for the four "special" QuerySet methods that perform DB writes (labeled
+**No** in the table above) _can_ be enabled. This requires three extra usage
+details:
+
+> **Warning**
+> Enabling auditing on these QuerySet methods might have significant
+> performance implications, especially on large datasets, since audit events are
+> constructed in memory and bulk written to the database.
+
+1. Enable the feature by calling the audit decorator specifying
+   `@audit_fields(..., audit_special_queryset_writes=True)`.
+2. Configure the model class so its default manager is an instance of
+   `field_audit.models.AuditingManager`.
+3. All calls to the four "special" QuerySet write methods require an extra
+   `audit_action` keyword argument whose value is one of:
+   - `field_audit.models.AuditAction.AUDIT`
+   - `field_audit.models.AuditAction.IGNORE`
+
+##### Important Notes
+
+- Specifying `audit_special_queryset_writes=True` (step **1** above) without
+  setting the default manager to an instance of `AuditingManager` (step **2**
+  above) will raise an exception when the model class is evaluated.
+- At this time, `QuerySet.delete()`, `QuerySet.update()`,
+  and `QuerySet.bulk_create()` "special" write methods can actually perform
+  change auditing when called with `audit_action=AuditAction.AUDIT`. 
+  `QuerySet.bulk_update()` is not currently implemented and will raise
+  `NotImplementedError` if called with that action. Implementing this remaining
+  method remains a task for the future, see **TODO** below. All four methods do
+  support `audit_action=AuditAction.IGNORE` usage, however.
+- All audited methods use transactions to ensure changes to audited models
+  are only committed to the database if audit events are successfully created
+  and saved as well.
+
+### Auditing Many-to-Many fields
+
+Many-to-Many field changes are automatically audited through Django signals when
+included in the `@audit_fields` decorator. Changes to M2M relationships generate
+audit events immediately without requiring `save()` calls.
+
+```python
+# Example model with audited M2M field
+@audit_fields("name", "title", "certifications")
+class CrewMember(models.Model):
+    name = models.CharField(max_length=256)
+    title = models.CharField(max_length=64)
+    certifications = models.ManyToManyField('Certification', blank=True)
+```
+
+#### Supported M2M operations
+
+All standard M2M operations create audit events:
+
+```python
+crew_member = CrewMember.objects.create(name='Test Pilot', title='Captain')
+cert1 = Certification.objects.create(name='PPL', certification_type='Private')
+
+crew_member.certifications.add(cert1)         # Creates audit event
+crew_member.certifications.remove(cert1)      # Creates audit event
+crew_member.certifications.set([cert1])       # Creates audit event
+crew_member.certifications.clear()            # Creates audit event
+```
+
+#### M2M audit event structure
+
+M2M changes use specific delta structures in audit events:
+
+- **Add**: `{'certifications': {'add': [1, 2]}}`
+- **Remove**: `{'certifications': {'remove': [2]}}`
+- **Clear**: `{'certifications': {'remove': [1, 2]}}`
+- **Create** / **Bootstrap**: `{'certifications': {'new': []}}`
+
+#### Bootstrap events for models with existing records
+
+In the scenario where auditing is enabled for a model with existing data, it can
+be valuable to generate "bootstrap" audit events for all of the existing model
+records in order to ensure that there is at least one audit event record for
+every model instance that currently exists.  There is a migration utility for
+performing this bootstrap operation. Example code:
+
+```python
+# flight/migrations/0002_bootstrap_aircarft_auditing.py
+
+from django.db import migrations, models
+from field_audit.utils import run_bootstrap
+
+from flight.models import Aircraft
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('flight', '0001_initial'),
+    ]
+
+    operations = [
+        run_bootstrap(Aircraft, ["tail_number", "make_model", "operated_by"])
+    ]
+```
+
+##### Bootstrap events via management command
+
+If bootstrapping is not suitable during migrations, there is a management command for
+performing the same operation.  The management command does not accept arbitrary
+field names for bootstrap records, and uses the fields configured by the
+existing `audit_fields(...)` decorator on the model. Example (analogous to
+migration action shown above):
+
+```sh
+manage.py bootstrap_field_audit_events init Aircraft
+```
+
+Additionally, if a post-migration bootstrap "top up" action is needed, the
+the management command can also perform this action. A "top up" operation
+creates bootstrap audit events for any existing model records which do not have
+a "create" or "bootstrap" `AuditEvent` record. Note that the management command
+is currently the only way to "top up" bootstrap audit events. Example:
+
+```sh
+manage.py bootstrap_field_audit_events top-up Aircraft
+```
+
+### Using with SQLite
+
+This app uses Django's `JSONField` which means if you intend to use the app with
+a SQLite database, the SQLite `JSON1` extension is required. If your system's
+Python `sqlite3` library doesn't ship with this extension enabled, see
+[this article](https://code.djangoproject.com/wiki/JSON1Extension) for details
+on how to enable it.
+
+
+## Contributing
+
+All feature and bug contributions are expected to be covered by tests.
+
+### Setup for developers
+
+This project uses [uv](https://docs.astral.sh/uv/) for dependency management. Install uv and then install the project dependencies:
+
+```shell
+cd django-field-audit
+uv sync
+```
+
+### Running tests
+
+**Note**: By default, local tests use an in-memory SQLite database. Ensure that
+your local Python's `sqlite3` library ships with the `JSON1` extension enabled
+(see [Using with SQLite](#using-with-sqlite)).
+
+- Tests
+  ```shell
+  uv run pytest
+  ```
+
+- Style check
+  ```shell
+  ruff check
+  ```
+
+- Coverage
+  ```shell
+  uv run coverage run -m pytest
+  uv run coverage report -m
+  ```
+
+### Adding migrations
+
+The example `manage.py` is available for making new migrations.
+
+```shell
+uv run python example/manage.py makemigrations field_audit
+```
+
+### Publishing a new version to PyPI
+
+Push a new tag to Github using the format vX.Y.Z where X.Y.Z matches the version
+in [`__init__.py`](field_audit/__init__.py). Also ensure that the changelog is up to date.
+
+Publishing is automated with [Github Actions](.github/workflows/pypi.yml).
+
+## TODO
+
+- Implement auditing for the remaining "special" QuerySet write operations:
+  - `bulk_update()`
+- Write full library documentation using github.io.
+
+### Backlog
+
+- Add to optimization for `instance.save(save_fields=[...])` [maybe].
+- Support adding new audit fields on the same model at different times (instead
+  of raising `AlreadyAudited`) [maybe].
+

django_field_audit-1.4.0.dist-info/RECORD

@@ -0,0 +1,20 @@
+field_audit/__init__.py,sha256=3_ZnH0SaqrZCTiCJjHrE8ELZlIoot1h4OJ5gEqIaQRk,143
+field_audit/apps.py,sha256=04NYTi54zEuYPAhxEvsS61hmoPMIMVfPQKi4DOa9nE0,265
+field_audit/auditors.py,sha256=V5Ese7w4V_WTLklIm5EOsjOJ3t4Iwp69AmgwxNXD33Y,4292
+field_audit/const.py,sha256=U7c5Y4a5YjmvaZjsUpDhdTOvkcoaS5aqFf_L3C5GtYk,1135
+field_audit/field_audit.py,sha256=lTEjaQFD1U8bVa6O-67wvbFY4riRcayc_4kCNjRFgao,8895
+field_audit/middleware.py,sha256=JQMdM1vITddHFCqIX_M8_UDIvbCKU8BhY_sWccGVS-Y,468
+field_audit/models.py,sha256=z7-Srvg7pjmyz_g3NrRosPNsvtbPNy17Ec1vHOXXp48,32514
+field_audit/services.py,sha256=x_xoqmlKmGN4KD_LPsuUuoFJVmrgkwdugS_tgV6rUbM,16393
+field_audit/utils.py,sha256=Jhal9wjUjZcQJsrXrOQoAxaIl62EhJbkfn2xPhiOBAM,3483
+field_audit/management/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+field_audit/management/commands/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+field_audit/management/commands/bootstrap_field_audit_events.py,sha256=0onszv7o3J03Y-NGjp8-74YVVtdyJkzcOUIJ6hs3hmQ,4327
+field_audit/migrations/0001_initial.py,sha256=UXnmSkG8ZZmh-4DrqYACH9LSJ0Mh74nW0u_yKdS1ctg,1297
+field_audit/migrations/0002_add_is_bootstrap_column.py,sha256=lkPNMk9Kb6IeaX9E08Snar-0_zr1epVrW9VSq24f9Ns,972
+field_audit/migrations/0003_alter_auditevent_change_context_and_more.py,sha256=0MxLTzKAGCsIc6674-RZc6wGImbHpy5D069fTPbCjak,933
+field_audit/migrations/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+django_field_audit-1.4.0.dist-info/licenses/LICENSE,sha256=DUmNjtND8byIKaTpjoksSUVazUJ_-3sPWOyyLiOvyDs,1508
+django_field_audit-1.4.0.dist-info/WHEEL,sha256=G2gURzTEtmeR8nrdXUJfNiB3VYVxigPQ-bEQujpNiNs,82
+django_field_audit-1.4.0.dist-info/METADATA,sha256=m9ZPI1oBrdeWrvg4YwcfR5OTu0ETldVlYt6Zx_TIb2U,13035
+django_field_audit-1.4.0.dist-info/RECORD,,

django_field_audit-1.4.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: flit 3.12.0
+Root-Is-Purelib: true
+Tag: py3-none-any

django_field_audit-1.4.0.dist-info/licenses/LICENSE

@@ -0,0 +1,24 @@
+Copyright (c) 2022, Dimagi Inc., and individual contributors.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+    * Redistributions of source code must retain the above copyright
+      notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+      notice, this list of conditions and the following disclaimer in the
+      documentation and/or other materials provided with the distribution.
+    * Neither the name Dimagi, nor the names of its contributors, may be used
+      to endorse or promote products derived from this software without
+      specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL DIMAGI INC. BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

field_audit/__init__.py

@@ -0,0 +1,4 @@
+from .field_audit import audit_fields  # noqa: F401
+from .services import AuditService, get_audit_service  # noqa: F401
+
+__version__ = "1.4.0"

field_audit/apps.py

@@ -0,0 +1,11 @@
+from django.apps import AppConfig
+
+
+class FieldAuditConfig(AppConfig):
+
+    name = "field_audit"
+    default_auto_field = "django.db.models.BigAutoField"
+
+    def ready(self):
+        from .auditors import audit_dispatcher
+        audit_dispatcher.setup_auditors()

field_audit/auditors.py

@@ -0,0 +1,125 @@
+from getpass import getuser
+from subprocess import DEVNULL, CalledProcessError, check_output
+
+from .models import (
+    USER_TYPE_PROCESS,
+    USER_TYPE_REQUEST,
+    USER_TYPE_TTY,
+)
+from .utils import class_import_helper
+
+__all__ = [
+    "BaseAuditor",
+    "RequestAuditor",
+    "SystemUserAuditor",
+    "audit_dispatcher",
+]
+
+
+class _AuditDispatcher:
+    """Dispatcher for the Audit API used to get "changed by" information when
+    creating new ``AuditEvent`` records (i.e. when an audited model is changed).
+
+    An instance of this class maintains the auditor chain that defines which
+    (and in what order) ``BaseAuditor`` subclass instances are used to acquire
+    "changed by" information.
+
+    The auditor chain can be customized by defining a list of 'BaseAuditor'
+    subclass paths via the ``FIELD_AUDIT_AUDITORS`` settings attribute.
+    """
+
+    def setup_auditors(self):
+        """Populate the auditors chain, possibly defined in settings.
+
+        This method is called at app ready time, and must be called before the
+        ``dispatch()`` method can be used.
+        """
+        from django.conf import settings
+        auditors_attr = "FIELD_AUDIT_AUDITORS"
+        if hasattr(settings, auditors_attr):
+            self.auditors = []
+            for auditor_path in getattr(settings, auditors_attr):
+                auditor_class = class_import_helper(
+                    auditor_path,
+                    f"{auditors_attr!r} item",
+                    BaseAuditor,
+                )
+                self.auditors.append(auditor_class())
+        else:
+            self.auditors = [RequestAuditor(), SystemUserAuditor()]
+
+    def dispatch(self, request):
+        """Cycles through the auditors chain and returns the first non-None
+        value returned by a call to ``auditor.change_context(request)``.
+
+        :param request: Django request to be audited (or ``None``).
+        :returns: JSON-serializable value (or ``None`` if chain is exhausted).
+        """
+        for auditor in self.auditors:
+            change_context = auditor.change_context(request)
+            if change_context is not None:
+                return change_context
+        return None
+
+
+audit_dispatcher = _AuditDispatcher()
+
+
+class BaseAuditor:
+    """Abstract class for the Auditor API. Subclasses are used to return
+    "changed by" information associated with an event which they know how to
+    audit.
+
+    BaseAuditor subclasses must define the following:
+    - ``change_context()`` method that returns a JSON-serializable object of
+      information for events it knows how to audit (or ``None`` otherwise).
+    """
+
+    def change_context(self, request):
+        raise NotImplementedError("change_context() is abstract")
+
+
+class RequestAuditor(BaseAuditor):
+    """Auditor class for getting users from authenticated requests."""
+
+    def change_context(self, request):
+        if request is None:
+            # cannot provide a request user without a request
+            return None
+        if request.user.is_authenticated:
+            return {
+                "user_type": USER_TYPE_REQUEST,
+                "username": request.user.username,
+            }
+        # short-circuit the audit chain for not-None requests
+        return {}
+
+
+class SystemUserAuditor(BaseAuditor):
+    """Auditor class for getting OS usernames."""
+
+    def __init__(self):
+        self.has_who_bin = True
+
+    def change_context(self, request):
+        username = None
+        if self.has_who_bin:
+            try:
+                # get owner of STDIN file on login sessions (e.g. SSH)
+                output = check_output(["who", "-m"], stderr=DEVNULL)
+            except (FileNotFoundError, CalledProcessError):
+                self.has_who_bin = False
+            else:
+                if output:
+                    try:
+                        username = output.split()[0].decode("utf-8")
+                        user_type = USER_TYPE_TTY
+                    except (IndexError, UnicodeDecodeError):
+                        pass
+        if not username:
+            # no TTY user, get owner of the current process
+            username = getuser()
+            user_type = USER_TYPE_PROCESS
+        if username:
+            return {"user_type": user_type, "username": username}
+        return None

field_audit/const.py

@@ -0,0 +1,19 @@
+
+# Number of records to bulk fetch/insert per batch for bootstrap operations.
+#
+# Benchmark testing of this value was performed by bootstrapping a table with
+# five (5) columns and ~2.6 million rows. Two benchmark runs were performed with
+# the database reset and restarted between runs. The first benchmark used
+# 'batch_size=1000' and completed in 6min 15sec, the second benchmark used
+# 'batch_size=10000' and completed in 6min 12sec (less than 1% difference in
+# runtime). There was no noticeable difference in database resource usage
+# between the two runs, but the Django process consistently used about 160MiB
+# more memory for the duration of the second benchmark compared to the first.
+# Given that the overall runtime was relatively unaffected between two tests
+# whose batch size differed by an order of magnitude, the lower value seems like
+# the better default due to the lower Django resource usage.
+#
+# Installations with noticeable database connection latency may prefer to
+# specify a higher value on their bootstrap operations in order to optimize for
+# fewer round-trips to the database.
+BOOTSTRAP_BATCH_SIZE = 1000