django-log-formatter-asim 1.1.0a4__py3-none-any.whl → 1.2.0__py3-none-any.whl

django_log_formatter_asim-1.2.0.dist-info/METADATA

@@ -0,0 +1,399 @@
+Metadata-Version: 2.4
+Name: django-log-formatter-asim
+Version: 1.2.0
+Summary: Formats Django logs in ASIM format.
+License: MIT
+License-File: LICENSE
+Author: Department for Business and Trade Platform Team
+Author-email: sre-team@digital.trade.gov.uk
+Requires-Python: >=3.9,<4
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: ddtrace (>=3.2.1,<5)
+Requires-Dist: django (>=3,<5) ; python_version == "3.9"
+Requires-Dist: django (>=3,<6) ; python_version >= "3.10" and python_version < "4"
+Requires-Dist: django-ipware (>=7.0.1,<8.0.0)
+Description-Content-Type: text/markdown
+
+# Django ASIM log formatter
+
+The library formats Django logs in [ASIM format](https://learn.microsoft.com/en-us/azure/sentinel/normalization).
+
+Mapping to the format may not be complete, but best effort has been made to create logical field mappings.
+
+## Installation
+
+``` shell
+pip install django-log-formatter-asim
+```
+
+## Usage
+
+This package provides the following ASIM functionality:
+
+- A Python [logging.Formatter] implementation.
+- A module of functions `django_log_formatter_asim.events` which generate ASIM event log entries.
+
+[logging.Formatter]: https://docs.python.org/3/library/logging.html#formatter-objects
+
+### `logging.Formatter` setup
+
+Using the formatter in a Django logging configuration:
+
+``` python
+from django_log_formatter_asim import ASIMFormatter
+
+LOGGING = {
+    ...
+    "formatters": {
+        "asim_formatter": {
+            "()": ASIMFormatter,
+        },
+    },
+    'handlers': {
+        'asim': {
+            'formatter': 'asim_formatter',
+            ...
+        },
+    },
+    "root": {
+        "handlers": ["asim"],
+        ...
+    }
+    "loggers": {
+        "django": {
+            "handlers": ["asim"],
+            "propagate": False
+            ...
+        },
+    },
+}
+```
+
+In this example we assign the ASIM formatter to a `handler` and ensure both `root` and `django` loggers use this `handler`.
+We then set `propagate` to `False` on the `django` logger, to avoid duplicating logs at the root level.
+
+### Settings
+
+`DLFA_LOG_PERSONALLY_IDENTIFIABLE_INFORMATION` - the formatter checks this setting to see if personally identifiable information should be logged. If this is not set to true, only the user's id is logged.
+
+`DLFA_TRACE_HEADERS` - used for defining custom zipkin headers, the defaults is `("X-Amzn-Trace-Id")`, but for applications hosted in GOV.UK PaaS you should use `("X-B3-TraceId", "X-B3-SpanId")`. If you are running your application in both places side by side during migration, the following should work in your Django settings:
+
+```python
+from dbt_copilot_python.utility import is_copilot
+
+if is_copilot():
+   DLFA_TRACE_HEADERS = ("X-B3-TraceId", "X-B3-SpanId")
+```
+
+`DLFA_INCLUDE_RAW_LOG` - By default the original unformatted log is not included in the ASIM formatted log. You can enable that by setting this to `True` and it will be included in `AddidtionalFields.RawLog`.
+
+> [!WARNING]
+> Setting `DLFA_INCLUDE_RAW_LOG` to `True` will cause additional private fields to be output to your logs.
+> This could include secrets, such as AWS Access Keys, private HTTP Request data, or personally identifiable information.
+> This setting is not recommended for a production environment.
+
+### Serialisation behaviour
+
+The package provides one `logging.Formatter` class, `ASIMFormatter` which routes log messages to a serialiser
+which generates a python dict which the formatter converts to a JSON string and prints to standard output.
+
+It has a generic serialiser called `ASIMRootFormatter` and a custom serlializer for log messages where the
+logger is `django.request`.
+
+``` python
+    ASIM_FORMATTERS = {
+        "root": ASIMRootFormatter,
+        "django.request": ASIMRequestFormatter,
+    }
+```
+
+#### ASIMRootFormatter
+
+This serialiser outputs the following ASIM fields.
+
+- `EventSchema` = `ProcessEvent`
+- `ActingAppType` = `Django`
+- `AdditionalFields[DjangoLogFormatterAsimVersion]`
+- `EventSchemaVersion`
+- `EventMessage`
+- `EventCount`
+- `EventStartTime`
+- `EventEndTime`
+- `EventType`
+- `EventResult`
+- `EventSeverity`
+- `EventOriginalSeverity`
+
+Additionally, the following DataDog fields where available:
+
+- `dd.trace_id`
+- `dd.span_id`
+- `env`
+- `service`
+- `version`
+
+
+#### ASIMRequestFormatter
+
+This serialiser outputs the following ASIM fields in addition to the ones from ASIMRootFormatter.
+It is coupled to the datastructure provided by the `django.request` logger.
+The `django.request` logger only outputs requests where the response code is 4xx/5xx.
+
+- `SrcIpAddr` and `IpAddr`
+- `SrcPortNumber`
+- `SrcUserId` and `SrcUsername`
+- `HttpUserAgent`
+- `AdditionalFields["TraceHeaders"][trace_header_name]` - See `DLFA_TRACE_HEADERS` setting for more information.
+
+#### Creating a custom serialiser
+
+If you wish to create your own ASIM serialiser, you can inherit from `ASIMRootFormatter` and call
+`super().get_log_dict()` to get the base level logging data for augmentation:
+
+``` python
+    class MyASIMFormatter(ASIMRootFormatter):
+        def get_log_dict(self):
+            log_dict = super().get_log_dict()
+
+            # Customise logger event
+
+            return log_dict
+```
+
+This serialiser can then be added to `ASIM_FORMATTERS`...
+
+```python
+ASIM_FORMATTERS["my_logger"] = MyASIMFormatter
+```
+
+
+### ASIM Events
+
+The events mostly follow the Microsoft schema but have been tailored to Department of Business and Trade needs.
+
+Events are designed for simple integrate into your Django app.
+Each will take additional information from the [Django HttpRequest object][django-request].
+
+[django-request]: https://docs.djangoproject.com/en/5.2/ref/request-response/#httprequest-objects
+
+#### Authentication event
+
+Following the [ASIM Authentication Schema](https://learn.microsoft.com/en-us/azure/sentinel/normalization-schema-authentication).
+
+```python
+# Example usage
+from django_log_formatter_asim.events import log_authentication
+
+log_authentication(
+    request,
+    event=log_authentication.Event.Logoff,
+    result=log_authentication.Result.Success,
+    login_method=log_authentication.LoginMethod.UsernamePassword,
+)
+
+# Example JSON printed to standard output
+{
+    # Values provided as arguments
+    "EventType": "Logoff",
+    "EventResult": "Success",
+    "LogonMethod": "Username & Password",
+
+    # Calculated / Hard coded fields
+    "EventStartTime": "2025-07-02T08:15:20+00:00",
+    "EventSeverity": "Informational",
+    "EventOriginalType": "001c",
+    "EventSchema": "Authentication",
+    "EventSchemaVersion": "0.1.4",
+
+    # Taken from Django HttpRequest object
+    "HttpHost": "WebServer.local",
+    "SrcIpAddr": "192.168.1.101",
+    "TargetUrl": "https://WebServer.local/steel",
+    "TargetSessionId": "def456",
+    "TargetUsername": "Adrian"
+
+    # Taken from DBT Platform environment variables
+    "TargetAppName": "export-analytics-frontend",
+}
+```
+
+#### File Activity event
+
+Following the [ASIM File Event Schema](https://learn.microsoft.com/en-us/azure/sentinel/normalization-schema-file-event).
+
+```python
+# Example usage
+from django_log_formatter_asim.events import log_file_activity
+
+log_file_activity(
+    request,
+    event=log_file_activity.Event.FileCopied,
+    result=log_file_activity.Result.Success,
+    file={
+        "path": "/tmp/copied.txt",
+        "content_type": "text/plain",
+        "extension": "txt",
+        "name": "copied.txt",
+        "sha256": "6798b7a132f37a0474002dec538ec52bdcd5f7b76e49e52c8a3d2016ca8d1d18",
+        "size": 14,
+    },
+    # source_file is only necessary if the event is one of FileRenamed, FileMoved, FileCopied, FolderMoved
+    source_file={
+        "path": "/tmp/original.txt",
+        "content_type": "text/plain",
+        "extension": "txt",
+        "name": "original.txt",
+        "sha256": "6798b7a132f37a0474002dec538ec52bdcd5f7b76e49e52c8a3d2016ca8d1d18",
+        "size": 14,
+    },
+)
+
+# Example JSON printed to standard output
+{
+    # Values provided as arguments
+    "EventType": "FileCopied",
+    "EventResult": "Success",
+
+    "TargetFilePath": "/tmp/copied.txt",
+    "TargetFileName": "copied.txt",
+    "TargetFileExtension": "txt",
+    "TargetFileMimeType": "text/plain",
+    "TargetFileSHA256": "6798b7a132f37a0474002dec538ec52bdcd5f7b76e49e52c8a3d2016ca8d1d18",
+    "TargetFileSize": 14,
+
+    "SrcFilePath": "/tmp/original.txt",
+    "SrcFileName": "original.txt",
+    "SrcFileExtension": "txt",
+    "SrcFileMimeType": "text/plain",
+    "SrcFileSHA256": "6798b7a132f37a0474002dec538ec52bdcd5f7b76e49e52c8a3d2016ca8d1d18",
+    "SrcFileSize": 14,
+
+    # Calculated / Hard coded fields
+    "EventStartTime": "2025-07-30T11:05:09.406460+00:00",
+    "EventSchema": "FileEvent",
+    "EventSchemaVersion": "0.2.1",
+    "EventSeverity": "Informational",
+
+    # Taken from Django HttpRequest object
+    "HttpHost": "WebServer.local",
+    "SrcIpAddr": "192.168.1.101",
+    "TargetUrl": "https://WebServer.local/steel",
+    "TargetUsername": "Adrian"
+
+    # Taken from DBT Platform environment variables
+    "TargetAppName": "export-analytics-frontend",
+}
+```
+
+#### Account Management event
+
+Following the [ASIM User Management Schema](https://learn.microsoft.com/en-us/azure/sentinel/normalization-schema-user-management).
+
+
+```python
+# Example usage
+from django_log_formatter_asim.events import log_account_management
+
+log_account_management(
+    request,
+    event=log_account_management.Event.UserCreated,
+    result=log_account_management.Result.Success,
+    account={
+        "username": "Roger",
+    },
+)
+
+# Example JSON printed to standard output
+{
+    # Values provided as arguments
+    "EventType": "UserCreated",
+    "EventResult": "Success",
+    "TargetUsername": "Roger",
+
+    # Calculated / Hard coded fields
+    "EventStartTime": "2025-07-30T11:05:09.406460+00:00",
+    "EventSchema": "UserManagement",
+    "EventSchemaVersion": "0.1.1",
+    "EventSeverity": "Informational",
+
+    # Taken from Django HttpRequest object
+    "HttpHost": "WebServer.local",
+    "SrcIpAddr": "192.168.1.101",
+    "TargetUrl": "https://WebServer.local/admin/create-user",
+    "ActorUsername": "Adrian"
+
+    # Taken from DBT Platform environment variables
+    "TargetAppName": "export-analytics-frontend",
+}
+```
+
+## Dependencies
+
+This package uses [Django IPware](https://github.com/un33k/django-ipware) for IP address capture.
+
+This package is compatible with [Django User Agents](https://pypi.org/project/django-user-agents) which, when used, will enhance logged user agent information.
+
+## Contributing to the `django-log-formatter-asim` package
+
+### Getting started
+
+1. Clone the repository:
+
+   ```
+   git clone https://github.com/uktrade/django-log-formatter-asim.git && cd django-log-formatter-asim
+   ```
+
+2. Install the required dependencies:
+
+   ```
+   pip install poetry && poetry install && poetry run pre-commit install
+   ```
+
+### Testing
+
+#### Automated testing
+
+Run `poetry run pytest` in the root directory to run all tests.
+
+Or, run `poetry run tox` in the root directory to run all tests for multiple Python versions. See the [`tox` configuration file](tox.ini).
+
+### Publishing
+
+1. Acquire API token from [Passman](https://passman.ci.uktrade.digital/secret/cc82a3f7-ddfa-4312-ab56-1ff8528dadc8/).
+   - Request access from the SRE team.
+   - _Note: You will need access to the `platform` group in Passman._
+2. Run `poetry config pypi-token.pypi <token>` to add the token to your Poetry configuration.
+
+Update the version, as the same version cannot be published to PyPI.
+
+```
+poetry version patch
+```
+
+More options for the `version` command can be found in the [Poetry documentation](https://python-poetry.org/docs/cli/#version). For example, for a minor version bump: `poetry version minor`.
+
+Build the Python package.
+
+```
+poetry build
+```
+
+Publish the Python package.
+
+_Note: Make sure your Pull Request (PR) is approved and contains the version upgrade in `pyproject.toml` before publishing the package._
+
+```
+poetry publish
+```
+
+Check the [PyPI Release history](https://pypi.org/project/django-log-formatter-asim/#history) to make sure the package has been updated.
+
+For an optional manual check, install the package locally and test everything works as expected.
+

django_log_formatter_asim-1.2.0.dist-info/RECORD

@@ -0,0 +1,11 @@
+django_log_formatter_asim/__init__.py,sha256=ftbSWdejSnWK_MpT3RSfTgcs3E-661FM-e3ugnoiBqA,7133
+django_log_formatter_asim/ecs.py,sha256=SSg3A5pfS5E-Hm7AXQnN1RrtXclgq07oSBxPCRn5gDg,536
+django_log_formatter_asim/events/__init__.py,sha256=K78BzhkNJpI2AdpaiMmf9FrBQTurEdBFX1Xx2hy-TGg,270
+django_log_formatter_asim/events/account_management.py,sha256=k6uPs9XcchCW2G3lcbdeXviSVWjJ4pK52mrYekwSLVQ,3936
+django_log_formatter_asim/events/authentication.py,sha256=8Kn5_8WF-DunmgpqWPVhaNh3k1qYVCwLbQ2I7WPmHj8,6966
+django_log_formatter_asim/events/common.py,sha256=CWC6Bo3R7yOY7NElJb1jxcD1el72Hl4CBZhFYgT0tIs,3874
+django_log_formatter_asim/events/file_activity.py,sha256=hTscMhGyaYVsLzqMcw-aQB5ks4G2VpB9Pkw63LwxqaI,7793
+django_log_formatter_asim-1.2.0.dist-info/METADATA,sha256=7oGXunm1_kCub5urCbcxS9CCS-c8gy-iykk8rYrQsmE,12476
+django_log_formatter_asim-1.2.0.dist-info/WHEEL,sha256=zp0Cn7JsFoX2ATtOhtaFYIiE2rmFAD4OcMhtUki8W3U,88
+django_log_formatter_asim-1.2.0.dist-info/licenses/LICENSE,sha256=dP79lN73--7LMApnankTGLqDbImXg8iYFqWgnExGkGk,1090
+django_log_formatter_asim-1.2.0.dist-info/RECORD,,

{django_log_formatter_asim-1.1.0a4.dist-info → django_log_formatter_asim-1.2.0.dist-info}/WHEEL

@@ -1,4 +1,4 @@
 Wheel-Version: 1.0
-Generator: poetry-core 2.1.3
+Generator: poetry-core 2.2.1
 Root-Is-Purelib: true
 Tag: py3-none-any

django_log_formatter_asim-1.1.0a4.dist-info/METADATA

@@ -1,176 +0,0 @@
-Metadata-Version: 2.3
-Name: django-log-formatter-asim
-Version: 1.1.0a4
-Summary: Formats Django logs in ASIM format.
-License: MIT
-Author: Department for Business and Trade Platform Team
-Author-email: sre-team@digital.trade.gov.uk
-Requires-Python: >=3.9,<4
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Requires-Dist: ddtrace (>=3.2.1,<4.0.0)
-Requires-Dist: django (>=3,<5) ; python_version == "3.9"
-Requires-Dist: django (>=3,<6) ; python_version >= "3.10" and python_version < "4"
-Requires-Dist: django-ipware (>=7.0.1,<8.0.0)
-Description-Content-Type: text/markdown
-
-# Django ASIM log formatter
-
-The library formats Django logs in [ASIM format](https://learn.microsoft.com/en-us/azure/sentinel/normalization).
-
-Mapping to the format may not be complete, but best effort has been made to create logical field mappings.
-
-If you need to amend the mapping, you can implement a custom formatter.
-
-## Installation
-
-``` shell
-pip install django-log-formatter-asim
-```
-
-## Usage
-
-Using in a Django logging configuration:
-
-``` python
-from django_log_formatter_asim import ASIMFormatter
-
-LOGGING = {
-    ...
-    "formatters": {
-        "asim_formatter": {
-            "()": ASIMFormatter,
-        },
-    },
-    'handlers': {
-        'asim': {
-            'formatter': 'asim_formatter',
-            ...
-        },
-    },
-    "root": {
-        "handlers": ["asim"],
-        ...
-    }
-    "loggers": {
-        "django": {
-            "handlers": ["asim"],
-            "propagate": False
-            ...
-        },
-    },
-}
-```
-In this example we assign the ASIM formatter to a `handler` and ensure both `root` and `django` loggers use this `handler`. We then set `propagate` to `False` on the `django` logger, to avoid duplicating logs at the root level. 
-
-## Dependencies
-
-This package uses [Django IPware](https://github.com/un33k/django-ipware) for IP address capture.
-
-This package is compatible with [Django User Agents](https://pypi.org/project/django-user-agents) which, when used, will enhance logged user agent information.
-
-## Settings
-
-`DLFA_LOG_PERSONALLY_IDENTIFIABLE_INFORMATION` - the formatter checks this setting to see if personally identifiable information should be logged. If this is not set to true, only the user's id is logged.
-
-`DLFA_TRACE_HEADERS` - used for defining custom zipkin headers, the defaults is `("X-Amzn-Trace-Id")`, but for applications hosted in GOV.UK PaaS you should use `("X-B3-TraceId", "X-B3-SpanId")`. If you are running your application in both places side by side during migration, the following should work in your Django settings:
-
-`DLFA_INCLUDE_RAW_LOG` - By default the original unformatted log is not included in the ASIM formatted log. You can enable that by setting this to `True` and it will be included in `AddidtionalFields.RawLog`.
-
-```python
-from dbt_copilot_python.utility import is_copilot
-
-if is_copilot():
-   DLFA_TRACE_HEADERS = ("X-B3-TraceId", "X-B3-SpanId")
-```
-
-## Formatter classes
-
-``` python
-    ASIM_FORMATTERS = {
-        "root": ASIMSystemFormatter,
-        "django.request": ASIMRequestFormatter,
-    }
-```
-
-The default class for other loggers is:
-
-``` python
-    ASIMSystemFormatter
-```
-
-## Creating a custom formatter
-
-If you wish to create your own ASIM formatter, you can inherit from ASIMSystemFormatter and call _get_event_base to get the base level logging data for use in augmentation:
-
-``` python
-    class ASIMSystemFormatter(ASIMFormatterBase):
-        def get_event(self):
-            logger_event = self._get_event_base()
-
-            # Customise logger event
-
-            return logger_event
-```
-
-## Contributing to the `django-log-formatter-asim` package
-
-### Getting started
-
-1. Clone the repository:
-
-   ```
-   git clone https://github.com/uktrade/django-log-formatter-asim.git && cd django-log-formatter-asim
-   ```
-
-2. Install the required dependencies:
-
-   ```
-   pip install poetry && poetry install && poetry run pre-commit install
-   ```
-
-### Testing
-
-#### Automated testing
-
-Run `poetry run pytest` in the root directory to run all tests.
-
-Or, run `poetry run tox` in the root directory to run all tests for multiple Python versions. See the [`tox` configuration file](tox.ini).
-
-### Publishing
-
-1. Acquire API token from [Passman](https://passman.ci.uktrade.digital/secret/cc82a3f7-ddfa-4312-ab56-1ff8528dadc8/).
-   - Request access from the SRE team.
-   - _Note: You will need access to the `platform` group in Passman._
-2. Run `poetry config pypi-token.pypi <token>` to add the token to your Poetry configuration.
-
-Update the version, as the same version cannot be published to PyPI.
-
-```
-poetry version patch
-```
-
-More options for the `version` command can be found in the [Poetry documentation](https://python-poetry.org/docs/cli/#version). For example, for a minor version bump: `poetry version minor`.
-
-Build the Python package.
-
-```
-poetry build
-```
-
-Publish the Python package.
-
-_Note: Make sure your Pull Request (PR) is approved and contains the version upgrade in `pyproject.toml` before publishing the package._
-
-```
-poetry publish
-```
-
-Check the [PyPI Release history](https://pypi.org/project/django-log-formatter-asim/#history) to make sure the package has been updated.
-
-For an optional manual check, install the package locally and test everything works as expected.

django_log_formatter_asim-1.1.0a4.dist-info/RECORD

@@ -1,10 +0,0 @@
-django_log_formatter_asim/__init__.py,sha256=ftbSWdejSnWK_MpT3RSfTgcs3E-661FM-e3ugnoiBqA,7133
-django_log_formatter_asim/ecs.py,sha256=SSg3A5pfS5E-Hm7AXQnN1RrtXclgq07oSBxPCRn5gDg,536
-django_log_formatter_asim/events/__init__.py,sha256=th8AEFNM-J5lNlO-d8Lk465jXqplE3IoTwj4DlscwYo,92
-django_log_formatter_asim/events/authentication.py,sha256=IjGTPb8Ucky-NtHGFe2CLwER-57lnMYoemjswWJE8Jk,6985
-django_log_formatter_asim/events/common.py,sha256=9LeiEn11FJowf2QiI5MLXlGIbY4MVRChABVcfchX8ko,1676
-django_log_formatter_asim/events/file_activity.py,sha256=fSNXWCr0De6EZDfoz-JQgQMQce4Wf9Kn2XeUPHW7oI0,7471
-django_log_formatter_asim-1.1.0a4.dist-info/LICENSE,sha256=dP79lN73--7LMApnankTGLqDbImXg8iYFqWgnExGkGk,1090
-django_log_formatter_asim-1.1.0a4.dist-info/METADATA,sha256=ygNuO5M_cSJoctnfCzAI6J4N8miKlpbeaEliD3ubmxs,5581
-django_log_formatter_asim-1.1.0a4.dist-info/WHEEL,sha256=b4K_helf-jlQoXBBETfwnf4B04YC67LOev0jo4fX5m8,88
-django_log_formatter_asim-1.1.0a4.dist-info/RECORD,,