django-log-formatter-asim 1.1.0a4__tar.gz → 1.2.0a0__tar.gz

{django_log_formatter_asim-1.1.0a4 → django_log_formatter_asim-1.2.0a0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: django-log-formatter-asim
-Version: 1.1.0a4
+Version: 1.2.0a0
 Summary: Formats Django logs in ASIM format.
 License: MIT
 Author: Department for Business and Trade Platform Team
@@ -25,8 +25,6 @@ The library formats Django logs in [ASIM format](https://learn.microsoft.com/en-
 
 Mapping to the format may not be complete, but best effort has been made to create logical field mappings.
 
-If you need to amend the mapping, you can implement a custom formatter.
-
 ## Installation
 
 ``` shell
@@ -35,7 +33,16 @@ pip install django-log-formatter-asim
 
 ## Usage
 
-Using in a Django logging configuration:
+This package provides the following ASIM functionality:
+
+- A Python [logging.Formatter] implementation.
+- A module of functions `django_log_formatter_asim.events` which generate ASIM event log entries.
+
+[logging.Formatter]: https://docs.python.org/3/library/logging.html#formatter-objects
+
+### `logging.Formatter` setup
+
+Using the formatter in a Django logging configuration:
 
 ``` python
 from django_log_formatter_asim import ASIMFormatter
@@ -66,15 +73,61 @@ LOGGING = {
     },
 }
 ```
-In this example we assign the ASIM formatter to a `handler` and ensure both `root` and `django` loggers use this `handler`. We then set `propagate` to `False` on the `django` logger, to avoid duplicating logs at the root level. 
 
-## Dependencies
+In this example we assign the ASIM formatter to a `handler` and ensure both `root` and `django` loggers use this `handler`.
+We then set `propagate` to `False` on the `django` logger, to avoid duplicating logs at the root level.
 
-This package uses [Django IPware](https://github.com/un33k/django-ipware) for IP address capture.
+### ASIM Events
 
-This package is compatible with [Django User Agents](https://pypi.org/project/django-user-agents) which, when used, will enhance logged user agent information.
+The events mostly follow the Microsoft schema but have been tailored to Department of Business and Trade needs.
+
+Events are designed for simple integrate into your Django app.
+Each will take additional information from the [Django HttpRequest object][django-request].
+
+[django-request]: https://docs.djangoproject.com/en/5.2/ref/request-response/#httprequest-objects
+
+#### Authentication event
 
-## Settings
+Following the [ASIM Authentication Schema](https://learn.microsoft.com/en-us/azure/sentinel/normalization-schema-authentication).
+
+```python
+# Example usage
+from django_log_formatter_asim.events import log_authentication
+
+log_authentication(
+    request,
+    event=log_authentication.Event.Logoff,
+    result=log_authentication.Result.Success,
+    login_method=log_authentication.LoginMethod.UsernamePassword,
+)
+
+# Example JSON printed to standard output
+{
+    # Values provided as arguments
+    "EventType": "Logoff",
+    "EventResult": "Success",
+    "LogonMethod": "Username & Password",
+
+    # Calculated / Hard coded fields
+    "EventStartTime": "2025-07-02T08:15:20+00:00",
+    "EventSeverity": "Informational",
+    "EventOriginalType": "001c",
+    "EventSchema": "Authentication",
+    "EventSchemaVersion": "0.1.4",
+
+    # Taken from Django HttpRequest object
+    "HttpHost": "WebServer.local",
+    "SrcIpAddr": "192.168.1.101",
+    "TargetUrl": "https://WebServer.local/steel",
+    "TargetSessionId": "def456",
+    "TargetUsername": "Adrian"
+
+    # Taken from DBT Platform environment variables
+    "TargetAppName": "export-analytics-frontend",
+}
+```
+
+### Settings
 
 `DLFA_LOG_PERSONALLY_IDENTIFIABLE_INFORMATION` - the formatter checks this setting to see if personally identifiable information should be logged. If this is not set to true, only the user's id is logged.
 
@@ -89,7 +142,7 @@ if is_copilot():
    DLFA_TRACE_HEADERS = ("X-B3-TraceId", "X-B3-SpanId")
 ```
 
-## Formatter classes
+### Formatter classes
 
 ``` python
     ASIM_FORMATTERS = {
@@ -104,7 +157,7 @@ The default class for other loggers is:
     ASIMSystemFormatter
 ```
 
-## Creating a custom formatter
+### Creating a custom `logging.Formatter`
 
 If you wish to create your own ASIM formatter, you can inherit from ASIMSystemFormatter and call _get_event_base to get the base level logging data for use in augmentation:
 
@@ -118,6 +171,12 @@ If you wish to create your own ASIM formatter, you can inherit from ASIMSystemFo
             return logger_event
 ```
 
+## Dependencies
+
+This package uses [Django IPware](https://github.com/un33k/django-ipware) for IP address capture.
+
+This package is compatible with [Django User Agents](https://pypi.org/project/django-user-agents) which, when used, will enhance logged user agent information.
+
 ## Contributing to the `django-log-formatter-asim` package
 
 ### Getting started

{django_log_formatter_asim-1.1.0a4 → django_log_formatter_asim-1.2.0a0}/README.md

@@ -4,8 +4,6 @@ The library formats Django logs in [ASIM format](https://learn.microsoft.com/en-
 
 Mapping to the format may not be complete, but best effort has been made to create logical field mappings.
 
-If you need to amend the mapping, you can implement a custom formatter.
-
 ## Installation
 
 ``` shell
@@ -14,7 +12,16 @@ pip install django-log-formatter-asim
 
 ## Usage
 
-Using in a Django logging configuration:
+This package provides the following ASIM functionality:
+
+- A Python [logging.Formatter] implementation.
+- A module of functions `django_log_formatter_asim.events` which generate ASIM event log entries.
+
+[logging.Formatter]: https://docs.python.org/3/library/logging.html#formatter-objects
+
+### `logging.Formatter` setup
+
+Using the formatter in a Django logging configuration:
 
 ``` python
 from django_log_formatter_asim import ASIMFormatter
@@ -45,15 +52,61 @@ LOGGING = {
     },
 }
 ```
-In this example we assign the ASIM formatter to a `handler` and ensure both `root` and `django` loggers use this `handler`. We then set `propagate` to `False` on the `django` logger, to avoid duplicating logs at the root level. 
 
-## Dependencies
+In this example we assign the ASIM formatter to a `handler` and ensure both `root` and `django` loggers use this `handler`.
+We then set `propagate` to `False` on the `django` logger, to avoid duplicating logs at the root level.
 
-This package uses [Django IPware](https://github.com/un33k/django-ipware) for IP address capture.
+### ASIM Events
 
-This package is compatible with [Django User Agents](https://pypi.org/project/django-user-agents) which, when used, will enhance logged user agent information.
+The events mostly follow the Microsoft schema but have been tailored to Department of Business and Trade needs.
+
+Events are designed for simple integrate into your Django app.
+Each will take additional information from the [Django HttpRequest object][django-request].
+
+[django-request]: https://docs.djangoproject.com/en/5.2/ref/request-response/#httprequest-objects
+
+#### Authentication event
 
-## Settings
+Following the [ASIM Authentication Schema](https://learn.microsoft.com/en-us/azure/sentinel/normalization-schema-authentication).
+
+```python
+# Example usage
+from django_log_formatter_asim.events import log_authentication
+
+log_authentication(
+    request,
+    event=log_authentication.Event.Logoff,
+    result=log_authentication.Result.Success,
+    login_method=log_authentication.LoginMethod.UsernamePassword,
+)
+
+# Example JSON printed to standard output
+{
+    # Values provided as arguments
+    "EventType": "Logoff",
+    "EventResult": "Success",
+    "LogonMethod": "Username & Password",
+
+    # Calculated / Hard coded fields
+    "EventStartTime": "2025-07-02T08:15:20+00:00",
+    "EventSeverity": "Informational",
+    "EventOriginalType": "001c",
+    "EventSchema": "Authentication",
+    "EventSchemaVersion": "0.1.4",
+
+    # Taken from Django HttpRequest object
+    "HttpHost": "WebServer.local",
+    "SrcIpAddr": "192.168.1.101",
+    "TargetUrl": "https://WebServer.local/steel",
+    "TargetSessionId": "def456",
+    "TargetUsername": "Adrian"
+
+    # Taken from DBT Platform environment variables
+    "TargetAppName": "export-analytics-frontend",
+}
+```
+
+### Settings
 
 `DLFA_LOG_PERSONALLY_IDENTIFIABLE_INFORMATION` - the formatter checks this setting to see if personally identifiable information should be logged. If this is not set to true, only the user's id is logged.
 
@@ -68,7 +121,7 @@ if is_copilot():
    DLFA_TRACE_HEADERS = ("X-B3-TraceId", "X-B3-SpanId")
 ```
 
-## Formatter classes
+### Formatter classes
 
 ``` python
     ASIM_FORMATTERS = {
@@ -83,7 +136,7 @@ The default class for other loggers is:
     ASIMSystemFormatter
 ```
 
-## Creating a custom formatter
+### Creating a custom `logging.Formatter`
 
 If you wish to create your own ASIM formatter, you can inherit from ASIMSystemFormatter and call _get_event_base to get the base level logging data for use in augmentation:
 
@@ -97,6 +150,12 @@ If you wish to create your own ASIM formatter, you can inherit from ASIMSystemFo
             return logger_event
 ```
 
+## Dependencies
+
+This package uses [Django IPware](https://github.com/un33k/django-ipware) for IP address capture.
+
+This package is compatible with [Django User Agents](https://pypi.org/project/django-user-agents) which, when used, will enhance logged user agent information.
+
 ## Contributing to the `django-log-formatter-asim` package
 
 ### Getting started

{django_log_formatter_asim-1.1.0a4 → django_log_formatter_asim-1.2.0a0}/django_log_formatter_asim/events/authentication.py

@@ -2,6 +2,7 @@ import datetime
 import json
 import os
 from enum import Enum
+from hashlib import sha3_512
 from typing import Literal
 from typing import Optional
 from typing import TypedDict
@@ -30,7 +31,7 @@ class AuthenticationLoginMethod(str, Enum):
     ExternalIDP = "External IdP"
 
 
-class AuthenticationUser(TypedDict):
+class AuthenticationUser(TypedDict, total=False):
     """Dictionary to represent properties of the users session."""
 
     """What type of role best describes this Authentication event."""
@@ -109,17 +110,37 @@ def log_authentication(
 
     See also: https://learn.microsoft.com/en-us/azure/sentinel/normalization-schema-authentication
     """
-    if user == None:
-        user = {}
-    if server == None:
-        server = {}
-    if client == None:
-        client = {}
-
-    event_created = time_generated or datetime.datetime.now(tz=datetime.timezone.utc)
 
+    _log_authentication(
+        request,
+        event,
+        result,
+        login_method,
+        user={} if user == None else user,
+        server={} if server == None else server,
+        client={} if client == None else client,
+        event_created=time_generated or datetime.datetime.now(tz=datetime.timezone.utc),
+        severity=severity,
+        result_details=result_details,
+        message=message,
+    )
+
+
+def _log_authentication(
+    request: HttpRequest,
+    event: AuthenticationEvent,
+    result: Result,
+    login_method: AuthenticationLoginMethod,
+    user: AuthenticationUser,
+    server: Server,
+    client: Client,
+    event_created: datetime.datetime,
+    severity: Optional[Severity] = None,
+    result_details: Optional[str] = None,
+    message: Optional[str] = None,
+):
     log = {
-        "EventCreated": event_created.isoformat(),  # TODO: Should this really be EventCreated, or TimeGenerated
+        "EventStartTime": event_created.isoformat(),
         "EventSeverity": severity or _default_severity(result),
         "EventOriginalType": _event_code(event, result),
         "EventType": event,
@@ -141,7 +162,7 @@ def log_authentication(
         log["TargetAppName"] = app_name
 
     if container_id := _get_container_id():
-        log["ContainerId"] = container_id
+        log["TargetContainerId"] = container_id
 
     if "ip_address" in client:
         log["SrcIpAddr"] = client["ip_address"]
@@ -157,9 +178,9 @@ def log_authentication(
         log["TargetUserType"] = user["role"]
 
     if "sessionId" in user:
-        log["TargetSessionId"] = user["sessionId"]
-    elif request.session.session_key:
-        log["TargetSessionId"] = request.session.session_key
+        log["TargetSessionId"] = _cryptographically_hash(user["sessionId"])
+    elif hasattr(request, "session") and request.session.session_key:
+        log["TargetSessionId"] = _cryptographically_hash(request.session.session_key)
 
     if "username" in user:
         log["TargetUsername"] = user["username"]
@@ -184,6 +205,12 @@ log_authentication.LoginMethod = AuthenticationLoginMethod
 log_authentication.Severity = Severity
 
 
+def _cryptographically_hash(data: Optional[str]) -> Optional[str]:
+    if data is None:
+        return None
+    return sha3_512(data.encode("UTF-8")).hexdigest()
+
+
 def _event_code(event: AuthenticationEvent, result: Result) -> str:
     if event == AuthenticationEvent.Logon:
         if result == Result.Success:

{django_log_formatter_asim-1.1.0a4 → django_log_formatter_asim-1.2.0a0}/django_log_formatter_asim/events/common.py

@@ -19,7 +19,7 @@ class Severity(str, Enum):
     High = "High"
 
 
-class Client(TypedDict):
+class Client(TypedDict, total=False):
     """Dictionary to represent properties of the HTTP Client."""
 
     """Internet Protocol Address of the client making the Authentication
@@ -29,7 +29,7 @@ class Client(TypedDict):
     requested_url: Optional[str]
 
 
-class Server(TypedDict):
+class Server(TypedDict, total=False):
     """Dictionary to represent properties of the HTTP Server."""
 
     """

{django_log_formatter_asim-1.1.0a4 → django_log_formatter_asim-1.2.0a0}/django_log_formatter_asim/events/file_activity.py

@@ -31,24 +31,30 @@ class FileActivityEvent(str, Enum):
     FolderModified = "FolderModified"
 
 
-class FileActivityFile(TypedDict):
-    """Dictionary to represent properties of the target file."""
+class FileActivityFileBase(TypedDict):
+    """Mandatory field definitions of FileActivityFile."""
 
     """
-    The full, normalized path of the target file, including the folder or location,
-    the file name, and the extension.
+    The full, normalized path of the target file, including the folder or
+    location, the file name, and the extension.
     """
     path: str
+
+
+class FileActivityFile(FileActivityFileBase, total=False):
+    """Dictionary to represent properties of either the target or source
+    file."""
+
     """
     The name of the target file, without a path or a location, but with an
     extension if available. This field should be similar to the final element in
-    the TargetFilePath field.
+    the *FilePath field.
 
     Defaults to extracting the name based off the path if not provided.
     """
     name: Optional[str]
     """
-    The target file extension.
+    The file extension.
 
     Defaults to extracting the extension based off the path if not provided.
     """
@@ -59,13 +65,13 @@ class FileActivityFile(TypedDict):
     Allowed values are listed in the IANA Media Types repository.
     """
     content_type: Optional[str]
-    """The SHA256 value of the target file."""
+    """The SHA256 value of the file."""
     sha256: Optional[str]
-    """The size of the target file in bytes."""
+    """The size of the file in bytes."""
     size: Optional[int]
 
 
-class FileActivityUser(TypedDict):
+class FileActivityUser(TypedDict, total=False):
     """
     A unique identifier for the user.
 
@@ -80,6 +86,7 @@ def log_file_activity(
     event: FileActivityEvent,
     result: Result,
     file: FileActivityFile,
+    source_file: Optional[FileActivityFile] = None,
     user: Optional[FileActivityUser] = None,
     server: Optional[Server] = None,
     client: Optional[Client] = None,
@@ -111,7 +118,10 @@ def log_file_activity(
                         - FolderModified
     :param result: What outcome did the action have, either "Success", "Failure", "Partial", "NA"
     :param file: Dictionary containing information on the target of this File event see
-                   FileActivityFile for more details.
+                 FileActivityFile for more details.
+    :param source_file: Dictionary containing information on the source of this File event,
+                        this MUST be used for a FileRenamed, FileMoved, FileCopied, FolderMoved
+                        operation. See FileActivityFile for more details.
     :param user: Dictionary containing information on the logged in users username.
     :param server: Dictionary containing information on the server servicing this File event
                    see Server class for more details.
@@ -129,45 +139,49 @@ def log_file_activity(
 
     See also: https://learn.microsoft.com/en-us/azure/sentinel/normalization-schema-file-event
     """
-    if user == None:
-        user = {}
-    if server == None:
-        server = {}
-    if client == None:
-        client = {}
-
-    event_created = time_generated or datetime.datetime.now(tz=datetime.timezone.utc)
 
+    _log_file_activity(
+        request,
+        event,
+        result,
+        file,
+        source_file,
+        user={} if user == None else user,
+        server={} if server == None else server,
+        client={} if client == None else client,
+        event_created=time_generated or datetime.datetime.now(tz=datetime.timezone.utc),
+        severity=severity,
+        result_details=result_details,
+        message=message,
+    )
+
+
+def _log_file_activity(
+    request: HttpRequest,
+    event: FileActivityEvent,
+    result: Result,
+    file: FileActivityFile,
+    source_file: Optional[FileActivityFile],
+    user: FileActivityUser,
+    server: Server,
+    client: Client,
+    event_created: datetime.datetime,
+    severity: Optional[Severity] = None,
+    result_details: Optional[str] = None,
+    message: Optional[str] = None,
+):
     log = {
         "EventSchema": "FileEvent",
         "EventSchemaVersion": "0.2.1",
         "EventType": event,
         "EventResult": result,
-        "EventCreated": event_created.isoformat(),  # TODO: Should this really be EventCreated, or TimeGenerated
+        "EventStartTime": event_created.isoformat(),
         "EventSeverity": severity or _default_severity(result),
-        "TargetFilePath": file["path"],
     }
 
-    if "name" in file:
-        log["TargetFileName"] = file["name"]
-    else:
-        log["TargetFileName"] = os.path.basename(file["path"])
-
-    if "extension" in file:
-        log["TargetFileExtension"] = file["extension"]
-    else:
-        file_name_parts = list(filter(None, log["TargetFileName"].split(".", 1)))
-        if len(file_name_parts) > 1:
-            log["TargetFileExtension"] = file_name_parts[1]
-
-    if "content_type" in file:
-        log["TargetFileMimeType"] = file["content_type"]
-
-    if "sha256" in file:
-        log["TargetFileSHA256"] = file["sha256"]
-
-    if "size" in file:
-        log["TargetFileSize"] = file["size"]
+    log.update(_generate_file_attributes(file, "Target"))
+    if source_file:
+        log.update(_generate_file_attributes(source_file, "Src"))
 
     if "domain_name" in server:
         log["HttpHost"] = server["domain_name"]
@@ -181,7 +195,7 @@ def log_file_activity(
         log["TargetAppName"] = app_name
 
     if container_id := _get_container_id():
-        log["ContainerId"] = container_id
+        log["TargetContainerId"] = container_id
 
     if "ip_address" in client:
         log["SrcIpAddr"] = client["ip_address"]
@@ -210,6 +224,33 @@ def log_file_activity(
     print(json.dumps(log), flush=True)
 
 
+def _generate_file_attributes(file: FileActivityFile, prefix: str) -> dict:
+    log = {prefix + "FilePath": file["path"]}
+
+    if "name" in file:
+        log[prefix + "FileName"] = file["name"]
+    else:
+        log[prefix + "FileName"] = os.path.basename(file["path"])
+
+    if "extension" in file:
+        log[prefix + "FileExtension"] = file["extension"]
+    else:
+        file_name_parts = list(filter(None, log[prefix + "FileName"].split(".", 1)))
+        if len(file_name_parts) > 1:
+            log[prefix + "FileExtension"] = file_name_parts[1]
+
+    if "content_type" in file:
+        log[prefix + "FileMimeType"] = file["content_type"]
+
+    if "sha256" in file:
+        log[prefix + "FileSHA256"] = file["sha256"]
+
+    if "size" in file:
+        log[prefix + "FileSize"] = file["size"]
+
+    return log
+
+
 log_file_activity.Event = FileActivityEvent
 log_file_activity.Result = Result
 log_file_activity.Severity = Severity

{django_log_formatter_asim-1.1.0a4 → django_log_formatter_asim-1.2.0a0}/pyproject.toml

@@ -3,7 +3,7 @@ line-length = 100
 
 [tool.poetry]
 name = "django-log-formatter-asim"
-version = "1.1.0a4"
+version = "1.2.0a0"
 description = "Formats Django logs in ASIM format."
 authors = ["Department for Business and Trade Platform Team <sre-team@digital.trade.gov.uk>"]
 license = "MIT"