open-api-framework 0.12.0__tar.gz → 0.13.0__tar.gz

{open_api_framework-0.12.0 → open_api_framework-0.13.0}/CHANGELOG.rst

@@ -1,6 +1,49 @@
 Changelog
 =========
 
+0.13.0 (2025-09-18)
+-------------------
+
+**New features**
+
+* [#178] Add logging envvars:
+
+    * ``LOG_FORMAT_CONSOLE``
+    * ``ENABLE_STRUCTLOG_REQUESTS``
+
+* [#178] Specify optional dependency groups:
+
+    * **structlog**: ``structlog``, ``django-structlog``
+    * **structlog-celery**: ``structlog``, ``django-structlog[celery]``
+
+* [#178] Add option to configure whether to use structlog or standard logging (default: stdlib logging). 
+  To enable structlog, downstream projects can do the following in their base settings:
+
+    .. code-block:: python
+
+        os.environ["_USE_STRUCTLOG"] = "True"
+
+        from open_api_framework.conf.base import *  # noqa
+
+**Bugfixes/QOL**
+
+* Fix casting DB_PORT "" to int
+
+**Maintenance**
+
+* Pin ```django-csp``` to 4.0 or higher.
+
+.. warning::
+   
+   The CSP environment variables function the same as with 3.8, but if any changes are made to 
+   CSP settings for downstream projects (like adding extra values to directives via the old 3.8 settings), 
+   manual action is needed to make sure it works with 4.0
+   (`see documentation <https://django-csp.readthedocs.io/en/latest/configuration.html#migrating-from-django-csp-3-8>`_)
+
+**Documentation**
+
+* [#171] Add documentation for connection pooling behaviour.
+
 0.12.0 (2025-07-15)
 -------------------
 

{open_api_framework-0.12.0/open_api_framework.egg-info → open_api_framework-0.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: open-api-framework
-Version: 0.12.0
+Version: 0.13.0
 Summary: A metapackage for registration components, that bundles the dependencies shared between these components and provides generic settings
 Author-email: Maykin Media <support@maykinmedia.nl>
 License: Copyright © Maykin 2024
@@ -74,7 +74,7 @@ Requires-Dist: django-markup>=1.8.1; extra == "markup"
 Provides-Extra: geo
 Requires-Dist: djangorestframework-gis>=1.0; extra == "geo"
 Provides-Extra: csp
-Requires-Dist: django-csp>=3.8; extra == "csp"
+Requires-Dist: django-csp>=4.0; extra == "csp"
 Provides-Extra: commonground
 Requires-Dist: commonground-api-common>=2.7.0; extra == "commonground"
 Provides-Extra: inclusions
@@ -85,12 +85,18 @@ Provides-Extra: server
 Requires-Dist: uwsgi>=2.0.23; extra == "server"
 Provides-Extra: redis
 Requires-Dist: django-redis>=5.4.0; extra == "redis"
+Provides-Extra: structlog
+Requires-Dist: structlog>=25.4.0; extra == "structlog"
+Requires-Dist: django-structlog>=9.1.1; extra == "structlog"
+Provides-Extra: structlog-celery
+Requires-Dist: structlog>=25.4.0; extra == "structlog-celery"
+Requires-Dist: django-structlog[celery]>=9.1.1; extra == "structlog-celery"
 Dynamic: license-file
 
 Open API Framework
 ==================
 
-:Version: 0.12.0
+:Version: 0.13.0
 :Source: https://github.com/maykinmedia/open-api-framework
 :Keywords: metapackage, dependencies
 

{open_api_framework-0.12.0 → open_api_framework-0.13.0}/README.rst

@@ -1,7 +1,7 @@
 Open API Framework
 ==================
 
-:Version: 0.12.0
+:Version: 0.13.0
 :Source: https://github.com/maykinmedia/open-api-framework
 :Keywords: metapackage, dependencies
 

{open_api_framework-0.12.0 → open_api_framework-0.13.0}/open_api_framework/conf/base.py

@@ -7,6 +7,7 @@ from django.urls import reverse_lazy
 
 import sentry_sdk
 from corsheaders.defaults import default_headers as default_cors_headers
+from csp.constants import NONCE, NONE, SELF
 from log_outgoing_requests.formatters import HttpFormatter
 from notifications_api_common.settings import *  # noqa
 
@@ -133,7 +134,11 @@ DATABASES = {
             auto_display_default=False,
         ),
         "PORT": config(
-            "DB_PORT", 5432, group="Database", help_text="port number of the database"
+            "DB_PORT",
+            5432,
+            group="Database",
+            help_text="port number of the database",
+            cast=lambda s: s and int(s),
         ),
     }
 }
@@ -543,6 +548,15 @@ if LOG_QUERIES and not DEBUG:
         RuntimeWarning,
     )
 
+LOG_FORMAT_CONSOLE = config(
+    "LOG_FORMAT_CONSOLE",
+    default="json",
+    help_text=(
+        "The format for the console logging handler, possible options: ``json``, ``plain_console``."
+    ),
+    group="Logging",
+)
+
 CELERY_LOGLEVEL = config(
     "CELERY_LOGLEVEL",
     default="INFO",
@@ -551,156 +565,373 @@ CELERY_LOGLEVEL = config(
     group="Celery",
 )
 
+_USE_STRUCTLOG = config("_USE_STRUCTLOG", default=False, add_to_docs=False)
+
+# XXX: this should be renamed to `LOG_REQUESTS` in the next major release
+ENABLE_STRUCTLOG_REQUESTS = config(
+    "ENABLE_STRUCTLOG_REQUESTS",
+    default=True,
+    help_text=("enable structured logging of requests"),
+    group="Logging",
+)
+
+
 LOGGING_DIR = Path(BASE_DIR) / "log"
 
-logging_root_handlers = ["console"] if LOG_STDOUT else ["project"]
-logging_django_handlers = ["console"] if LOG_STDOUT else ["django"]
+if _USE_STRUCTLOG:
+    import structlog
 
-LOGGING = {
-    "version": 1,
-    "disable_existing_loggers": False,
-    "formatters": {
-        "verbose": {
-            "format": "%(asctime)s %(levelname)s %(name)s %(module)s %(process)d %(thread)d  %(message)s"
-        },
-        "timestamped": {"format": "%(asctime)s %(levelname)s %(name)s  %(message)s"},
-        "simple": {"format": "%(levelname)s  %(message)s"},
-        "performance": {"format": "%(asctime)s %(process)d | %(thread)d | %(message)s"},
-        "db": {"format": "%(asctime)s | %(message)s"},
-        "outgoing_requests": {"()": HttpFormatter},
-    },
-    "filters": {
-        "require_debug_false": {"()": "django.utils.log.RequireDebugFalse"},
-    },
-    "handlers": {
-        "mail_admins": {
-            "level": "ERROR",
-            "filters": ["require_debug_false"],
-            "class": "django.utils.log.AdminEmailHandler",
-        },
-        "null": {"level": "DEBUG", "class": "logging.NullHandler"},
-        "console": {
-            "level": LOG_LEVEL,
-            "class": "logging.StreamHandler",
-            "formatter": "timestamped",
-        },
-        "console_db": {
-            "level": "DEBUG",
-            "class": "logging.StreamHandler",
-            "formatter": "db",
-        },
-        "celery_console": {
-            "level": CELERY_LOGLEVEL,
-            "class": "logging.StreamHandler",
-            "formatter": "timestamped",
-        },
-        "celery_file": {
-            "level": CELERY_LOGLEVEL,
-            "class": "logging.handlers.RotatingFileHandler",
-            "filename": Path(LOGGING_DIR) / "celery.log",
-            "formatter": "verbose",
-            "maxBytes": 1024 * 1024 * 10,  # 10 MB
-            "backupCount": 10,
-        },
-        "django": {
-            "level": LOG_LEVEL,
-            "class": "logging.handlers.RotatingFileHandler",
-            "filename": Path(LOGGING_DIR) / "django.log",
-            "formatter": "verbose",
-            "maxBytes": 1024 * 1024 * 10,  # 10 MB
-            "backupCount": 10,
-        },
-        "project": {
-            "level": LOG_LEVEL,
-            "class": "logging.handlers.RotatingFileHandler",
-            "filename": Path(LOGGING_DIR) / f"{PROJECT_DIRNAME}.log",
-            "formatter": "verbose",
-            "maxBytes": 1024 * 1024 * 10,  # 10 MB
-            "backupCount": 10,
-        },
-        "performance": {
-            "level": "INFO",
-            "class": "logging.handlers.RotatingFileHandler",
-            "filename": Path(LOGGING_DIR) / "performance.log",
-            "formatter": "performance",
-            "maxBytes": 1024 * 1024 * 10,  # 10 MB
-            "backupCount": 10,
-        },
-        "requests": {
-            "level": "DEBUG",
-            "class": "logging.handlers.RotatingFileHandler",
-            "filename": Path(LOGGING_DIR) / "requests.log",
-            "formatter": "timestamped",
-            "maxBytes": 1024 * 1024 * 10,  # 10 MB
-            "backupCount": 10,
-        },
-        "log_outgoing_requests": {
-            "level": "DEBUG",
-            "formatter": "outgoing_requests",
-            "class": "logging.StreamHandler",  # to write to stdout
-        },
-        "save_outgoing_requests": {
-            "level": "DEBUG",
-            # enabling saving to database
-            "class": "log_outgoing_requests.handlers.DatabaseOutgoingRequestsHandler",
-        },
-    },
-    "loggers": {
-        "": {
-            "handlers": logging_root_handlers,
-            "level": "ERROR",
-            "propagate": False,
-        },
-        PROJECT_DIRNAME: {
-            "handlers": logging_root_handlers,
-            "level": LOG_LEVEL,
-            "propagate": True,
-        },
-        "mozilla_django_oidc": {
-            "handlers": logging_root_handlers,
-            "level": LOG_LEVEL,
+    INSTALLED_APPS += [
+        "django_structlog",
+    ]
+
+    if ENABLE_STRUCTLOG_REQUESTS:
+        MIDDLEWARE.insert(
+            MIDDLEWARE.index("django.contrib.auth.middleware.AuthenticationMiddleware")
+            + 1,
+            "django_structlog.middlewares.RequestMiddleware",
+        )
+
+    logging_root_handlers = ["console"] if LOG_STDOUT else ["json_file"]
+
+    LOGGING = {
+        "version": 1,
+        "disable_existing_loggers": False,
+        "formatters": {
+            # structlog - foreign_pre_chain handles logs coming from stdlib logging module,
+            # while the `structlog.configure` call handles everything coming from structlog.
+            # They are mutually exclusive.
+            "json": {
+                "()": structlog.stdlib.ProcessorFormatter,
+                "processor": structlog.processors.JSONRenderer(),
+                "foreign_pre_chain": [
+                    structlog.contextvars.merge_contextvars,
+                    structlog.processors.TimeStamper(fmt="iso"),
+                    structlog.stdlib.add_logger_name,
+                    structlog.stdlib.add_log_level,
+                    structlog.stdlib.PositionalArgumentsFormatter(),
+                ],
+            },
+            "plain_console": {
+                "()": structlog.stdlib.ProcessorFormatter,
+                "processor": structlog.dev.ConsoleRenderer(),
+                "foreign_pre_chain": [
+                    structlog.contextvars.merge_contextvars,
+                    structlog.processors.TimeStamper(fmt="iso"),
+                    structlog.stdlib.add_logger_name,
+                    structlog.stdlib.add_log_level,
+                    structlog.stdlib.PositionalArgumentsFormatter(),
+                ],
+            },
+            "verbose": {
+                "format": "%(asctime)s %(levelname)s %(name)s %(module)s %(process)d %(thread)d  %(message)s"
+            },
+            "timestamped": {
+                "format": "%(asctime)s %(levelname)s %(name)s  %(message)s"
+            },
+            "simple": {"format": "%(levelname)s  %(message)s"},
+            "performance": {
+                "format": "%(asctime)s %(process)d | %(thread)d | %(message)s"
+            },
+            "db": {"format": "%(asctime)s | %(message)s"},
+            "outgoing_requests": {"()": HttpFormatter},
         },
-        f"{PROJECT_DIRNAME}.utils.middleware": {
-            "handlers": logging_root_handlers,
-            "level": LOG_LEVEL,
-            "propagate": False,
+        # TODO can be removed?
+        "filters": {
+            "require_debug_false": {"()": "django.utils.log.RequireDebugFalse"},
         },
-        "vng_api_common": {
-            "handlers": ["console"],
-            "level": LOG_LEVEL,
-            "propagate": True,
+        "handlers": {
+            # TODO can be removed?
+            "mail_admins": {
+                "level": "ERROR",
+                "filters": ["require_debug_false"],
+                "class": "django.utils.log.AdminEmailHandler",
+            },
+            "null": {"level": "DEBUG", "class": "logging.NullHandler"},
+            "console": {
+                "level": LOG_LEVEL,
+                "class": "logging.StreamHandler",
+                "formatter": LOG_FORMAT_CONSOLE,
+            },
+            "console_db": {
+                "level": "DEBUG",
+                "class": "logging.StreamHandler",
+                "formatter": "db",
+            },
+            # replaces the "django" and "project" handlers - in containerized applications
+            # the best practices is to log to stdout (use the console handler).
+            "json_file": {
+                "level": LOG_LEVEL,  # always debug might be better?
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / "application.jsonl",
+                "formatter": "json",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "performance": {
+                "level": "INFO",
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / "performance.log",
+                "formatter": "performance",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "requests": {
+                "level": "DEBUG",
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / "requests.log",
+                "formatter": "timestamped",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "log_outgoing_requests": {
+                "level": "DEBUG",
+                "formatter": "outgoing_requests",
+                "class": "logging.StreamHandler",  # to write to stdout
+            },
+            "save_outgoing_requests": {
+                "level": "DEBUG",
+                # enabling saving to database
+                "class": "log_outgoing_requests.handlers.DatabaseOutgoingRequestsHandler",
+            },
         },
-        "django.db.backends": {
-            "handlers": ["console_db"] if LOG_QUERIES else [],
-            "level": "DEBUG",
-            "propagate": False,
+        "loggers": {
+            "": {
+                "handlers": logging_root_handlers,
+                "level": "ERROR",
+                "propagate": False,
+            },
+            PROJECT_DIRNAME: {
+                "handlers": logging_root_handlers,
+                "level": LOG_LEVEL,
+                "propagate": False,
+            },
+            "mozilla_django_oidc": {
+                "handlers": logging_root_handlers,
+                "level": LOG_LEVEL,
+            },
+            f"{PROJECT_DIRNAME}.utils.middleware": {
+                "handlers": logging_root_handlers,
+                "level": LOG_LEVEL,
+                "propagate": False,
+            },
+            "vng_api_common": {
+                "handlers": ["console"],
+                "level": LOG_LEVEL,
+                "propagate": True,
+            },
+            "django.db.backends": {
+                "handlers": ["console_db"] if LOG_QUERIES else [],
+                "level": "DEBUG",
+                "propagate": False,
+            },
+            "django.request": {
+                "handlers": logging_root_handlers,
+                "level": LOG_LEVEL,
+                "propagate": False,
+            },
+            # suppress django.server request logs because those are already emitted by
+            # django-structlog middleware
+            "django.server": {
+                "handlers": ["console"],
+                "level": "WARNING" if ENABLE_STRUCTLOG_REQUESTS else "INFO",
+                "propagate": False,
+            },
+            "django.template": {
+                "handlers": ["console"],
+                "level": "INFO",
+                "propagate": False,
+            },
+            "log_outgoing_requests": {
+                "handlers": (
+                    ["log_outgoing_requests", "save_outgoing_requests"]
+                    if LOG_REQUESTS
+                    else []
+                ),
+                "level": "DEBUG",
+                "propagate": True,
+            },
+            "django_structlog": {
+                "handlers": logging_root_handlers,
+                "level": "INFO",
+                "propagate": False,
+            },
         },
-        "django.request": {
-            "handlers": logging_django_handlers,
-            "level": LOG_LEVEL,
-            "propagate": True,
+    }
+
+    structlog.configure(
+        processors=[
+            structlog.contextvars.merge_contextvars,
+            structlog.stdlib.filter_by_level,
+            structlog.processors.TimeStamper(fmt="iso"),
+            structlog.stdlib.add_logger_name,
+            structlog.stdlib.add_log_level,
+            structlog.stdlib.PositionalArgumentsFormatter(),
+            structlog.processors.StackInfoRenderer(),
+            structlog.processors.format_exc_info,
+            structlog.processors.UnicodeDecoder(),
+            structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
+        ],
+        logger_factory=structlog.stdlib.LoggerFactory(),
+        cache_logger_on_first_use=True,
+    )
+
+    # Optional django-structlog settings
+    DJANGO_STRUCTLOG_IP_LOGGING_ENABLED = False
+    DJANGO_STRUCTLOG_CELERY_ENABLED = True
+else:
+    logging_root_handlers = ["console"] if LOG_STDOUT else ["project"]
+    logging_django_handlers = ["console"] if LOG_STDOUT else ["django"]
+
+    LOGGING = {
+        "version": 1,
+        "disable_existing_loggers": False,
+        "formatters": {
+            "verbose": {
+                "format": "%(asctime)s %(levelname)s %(name)s %(module)s %(process)d %(thread)d  %(message)s"
+            },
+            "timestamped": {
+                "format": "%(asctime)s %(levelname)s %(name)s  %(message)s"
+            },
+            "simple": {"format": "%(levelname)s  %(message)s"},
+            "performance": {
+                "format": "%(asctime)s %(process)d | %(thread)d | %(message)s"
+            },
+            "db": {"format": "%(asctime)s | %(message)s"},
+            "outgoing_requests": {"()": HttpFormatter},
         },
-        "django.template": {
-            "handlers": ["console"],
-            "level": "INFO",
-            "propagate": False,
+        "filters": {
+            "require_debug_false": {"()": "django.utils.log.RequireDebugFalse"},
         },
-        "log_outgoing_requests": {
-            "handlers": (
-                ["log_outgoing_requests", "save_outgoing_requests"]
-                if LOG_REQUESTS
-                else []
-            ),
-            "level": "DEBUG",
-            "propagate": True,
+        "handlers": {
+            "mail_admins": {
+                "level": "ERROR",
+                "filters": ["require_debug_false"],
+                "class": "django.utils.log.AdminEmailHandler",
+            },
+            "null": {"level": "DEBUG", "class": "logging.NullHandler"},
+            "console": {
+                "level": LOG_LEVEL,
+                "class": "logging.StreamHandler",
+                "formatter": "timestamped",
+            },
+            "console_db": {
+                "level": "DEBUG",
+                "class": "logging.StreamHandler",
+                "formatter": "db",
+            },
+            "celery_console": {
+                "level": CELERY_LOGLEVEL,
+                "class": "logging.StreamHandler",
+                "formatter": "timestamped",
+            },
+            "celery_file": {
+                "level": CELERY_LOGLEVEL,
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / "celery.log",
+                "formatter": "verbose",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "django": {
+                "level": LOG_LEVEL,
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / "django.log",
+                "formatter": "verbose",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "project": {
+                "level": LOG_LEVEL,
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / f"{PROJECT_DIRNAME}.log",
+                "formatter": "verbose",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "performance": {
+                "level": "INFO",
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / "performance.log",
+                "formatter": "performance",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "requests": {
+                "level": "DEBUG",
+                "class": "logging.handlers.RotatingFileHandler",
+                "filename": Path(LOGGING_DIR) / "requests.log",
+                "formatter": "timestamped",
+                "maxBytes": 1024 * 1024 * 10,  # 10 MB
+                "backupCount": 10,
+            },
+            "log_outgoing_requests": {
+                "level": "DEBUG",
+                "formatter": "outgoing_requests",
+                "class": "logging.StreamHandler",  # to write to stdout
+            },
+            "save_outgoing_requests": {
+                "level": "DEBUG",
+                # enabling saving to database
+                "class": "log_outgoing_requests.handlers.DatabaseOutgoingRequestsHandler",
+            },
         },
-        "celery": {
-            "handlers": ["celery_console"] if LOG_STDOUT else ["celery_file"],
-            "level": CELERY_LOGLEVEL,
-            "propagate": True,
+        "loggers": {
+            "": {
+                "handlers": logging_root_handlers,
+                "level": "ERROR",
+                "propagate": False,
+            },
+            PROJECT_DIRNAME: {
+                "handlers": logging_root_handlers,
+                "level": LOG_LEVEL,
+                "propagate": True,
+            },
+            "mozilla_django_oidc": {
+                "handlers": logging_root_handlers,
+                "level": LOG_LEVEL,
+            },
+            f"{PROJECT_DIRNAME}.utils.middleware": {
+                "handlers": logging_root_handlers,
+                "level": LOG_LEVEL,
+                "propagate": False,
+            },
+            "vng_api_common": {
+                "handlers": ["console"],
+                "level": LOG_LEVEL,
+                "propagate": True,
+            },
+            "django.db.backends": {
+                "handlers": ["console_db"] if LOG_QUERIES else [],
+                "level": "DEBUG",
+                "propagate": False,
+            },
+            "django.request": {
+                "handlers": logging_django_handlers,
+                "level": LOG_LEVEL,
+                "propagate": True,
+            },
+            "django.template": {
+                "handlers": ["console"],
+                "level": "INFO",
+                "propagate": False,
+            },
+            "log_outgoing_requests": {
+                "handlers": (
+                    ["log_outgoing_requests", "save_outgoing_requests"]
+                    if LOG_REQUESTS
+                    else []
+                ),
+                "level": "DEBUG",
+                "propagate": True,
+            },
+            "celery": {
+                "handlers": ["celery_console"] if LOG_STDOUT else ["celery_file"],
+                "level": CELERY_LOGLEVEL,
+                "propagate": True,
+            },
         },
-    },
-}
+    }
 
 #
 # AUTH settings - user accounts, passwords, backends...
@@ -1153,95 +1384,86 @@ LOG_OUTGOING_REQUESTS_MAX_AGE = config(
 #
 # NOTE: make sure values are a tuple or list, and to quote special values like 'self'
 
-# ideally we'd use BASE_URI but it'd have to be lazy or cause issues
-CSP_DEFAULT_SRC = ["'self'"] + config(
-    "CSP_EXTRA_DEFAULT_SRC",
-    default=[],
-    split=True,
-    group="Content Security Policy",
-    help_text="Extra default source URLs for CSP other than ``self``. "
-    "Used for ``img-src``, ``style-src`` and ``script-src``",
-)
-
-
-CSP_REPORT_URI = config(
-    "CSP_REPORT_URI",
-    None,
-    group="Content Security Policy",
-    help_text="URI of the``report-uri`` directive.",
-)
-CSP_REPORT_PERCENTAGE = config(
-    "CSP_REPORT_PERCENTAGE",
-    0,
-    group="Content Security Policy",
-    help_text="Percentage of requests that get the ``report-uri`` directive.",
-)  # float between 0 and 1
-
-CSP_FORM_ACTION = (
-    config(
-        "CSP_FORM_ACTION",
-        default=["\"'self'\""]
-        + config(
-            "CSP_EXTRA_FORM_ACTION",
-            default=[],
-            split=True,
-            group="Content Security Policy",
-            help_text="Add additional ``form-action`` source to the default ",
-        ),
-        split=True,
-        group="Content Security Policy",
-        help_text="Override the default ``form-action`` source",
-    )
-    + CORS_ALLOWED_ORIGINS
-)
 
-CSP_IMG_SRC = (
-    CSP_DEFAULT_SRC
-    + ["data:", "cdn.redoc.ly"]  # used by DRF spectacular
-    + config(
-        "CSP_EXTRA_IMG_SRC",
+def get_content_security_policy():
+    # ideally we'd use BASE_URI but it'd have to be lazy or cause issues
+    csp_default_src = [SELF] + config(
+        "CSP_EXTRA_DEFAULT_SRC",
         default=[],
         split=True,
         group="Content Security Policy",
-        help_text="Extra ``img-src`` sources for CSP other than ``CSP_DEFAULT_SRC``.",
+        help_text="Extra default source URLs for CSP other than ``self``. Used for ``img-src``, ``style-src`` and ``script-src``.",
     )
-)
-
-# affects <object> and <embed> tags, block everything by default but allow deploy-time
-# overrides.
-CSP_OBJECT_SRC = config(
-    "CSP_OBJECT_SRC",
-    default=["\"'none'\""],
-    split=True,
-    group="Content Security Policy",
-    help_text="``object-src`` urls",
-)
-
-# we must include this explicitly, otherwise the style-src only includes the nonce because
-# of CSP_INCLUDE_NONCE_IN
-CSP_STYLE_SRC = CSP_DEFAULT_SRC + [
-    "'unsafe-inline'",
-    "fonts.googleapis.com",
-]  # used by DRF spectacular
-CSP_SCRIPT_SRC = CSP_DEFAULT_SRC + ["'unsafe-inline'"]
-CSP_FONT_SRC = ("'self'", "fonts.gstatic.com")
-CSP_WORKER_SRC = ("'self'", "blob:")
-
-# firefox does not get the nonce from default-src, see
-# https://stackoverflow.com/a/63376012
-CSP_INCLUDE_NONCE_IN = ["style-src", "script-src"]
-
-# directives that don't fallback to default-src
-CSP_BASE_URI = ["'self'"]
-
-# Frame directives do not fall back to default-src
-CSP_FRAME_ANCESTORS = ["'none'"]  # equivalent to X-Frame-Options: deny
-CSP_FRAME_SRC = ["'self'"]
-# CSP_NAVIGATE_TO = ["'self'"]  # this will break all outgoing links etc  # too much & tricky, see note on MDN
-# CSP_SANDBOX # too much
+    return {
+        "DIRECTIVES": {
+            "default-src": [SELF]
+            + config(
+                "CSP_EXTRA_DEFAULT_SRC",
+                default=[],
+                split=True,
+                group="Content Security Policy",
+                help_text="Extra default source URLs for CSP other than ``self``. Used for ``img-src``, ``style-src`` and ``script-src``.",
+            ),
+            "form-action": config(
+                "CSP_FORM_ACTION",
+                default=["\"'self'\""]
+                + config(
+                    "CSP_EXTRA_FORM_ACTION",
+                    default=[],
+                    split=True,
+                    group="Content Security Policy",
+                    help_text="Additional `form-action` sources.",
+                ),
+                split=True,
+                group="Content Security Policy",
+                help_text="Override the default `form-action` sources.",
+            )
+            + CORS_ALLOWED_ORIGINS,
+            "img-src": csp_default_src
+            + ["data:", "cdn.redoc.ly"]
+            + config(
+                "CSP_EXTRA_IMG_SRC",
+                default=[],
+                split=True,
+                group="Content Security Policy",
+                help_text="Extra `img-src` sources.",
+            ),
+            "object-src": config(
+                "CSP_OBJECT_SRC",
+                default=["\"'none'\""],
+                split=True,
+                group="Content Security Policy",
+                help_text="`object-src` sources.",
+            ),
+            "style-src": csp_default_src
+            + [NONCE, "'unsafe-inline'", "fonts.googleapis.com"],
+            "script-src": csp_default_src + [NONCE, "'unsafe-inline'"],
+            "font-src": [SELF, "fonts.gstatic.com"],
+            "worker-src": [SELF, "blob:"],
+            "base-uri": [SELF],
+            "frame-ancestors": [NONE],
+            "frame-src": [SELF],
+            "upgrade-insecure-requests": False,  # Enable only in production
+            "report-uri": config(
+                "CSP_REPORT_URI",
+                None,
+                group="Content Security Policy",
+                help_text="URI for CSP report-uri directive.",
+            ),
+        },
+        # Envvar used for django-csp==3.8 was a float between 0 and 1, while django-csp==4.0
+        # expects a percentage (between 0 and 100)
+        "REPORT_PERCENTAGE": config(
+            "CSP_REPORT_PERCENTAGE",
+            0.0,
+            group="Content Security Policy",
+            help_text="Fraction (between 0 and 1) of requests to include report-uri directive.",
+        )
+        * 100,
+    }
 
-CSP_UPGRADE_INSECURE_REQUESTS = False  # TODO enable on production?
 
+CONTENT_SECURITY_POLICY = get_content_security_policy()
 
 #
 # Django Solo

{open_api_framework-0.12.0 → open_api_framework-0.13.0/open_api_framework.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: open-api-framework
-Version: 0.12.0
+Version: 0.13.0
 Summary: A metapackage for registration components, that bundles the dependencies shared between these components and provides generic settings
 Author-email: Maykin Media <support@maykinmedia.nl>
 License: Copyright © Maykin 2024
@@ -74,7 +74,7 @@ Requires-Dist: django-markup>=1.8.1; extra == "markup"
 Provides-Extra: geo
 Requires-Dist: djangorestframework-gis>=1.0; extra == "geo"
 Provides-Extra: csp
-Requires-Dist: django-csp>=3.8; extra == "csp"
+Requires-Dist: django-csp>=4.0; extra == "csp"
 Provides-Extra: commonground
 Requires-Dist: commonground-api-common>=2.7.0; extra == "commonground"
 Provides-Extra: inclusions
@@ -85,12 +85,18 @@ Provides-Extra: server
 Requires-Dist: uwsgi>=2.0.23; extra == "server"
 Provides-Extra: redis
 Requires-Dist: django-redis>=5.4.0; extra == "redis"
+Provides-Extra: structlog
+Requires-Dist: structlog>=25.4.0; extra == "structlog"
+Requires-Dist: django-structlog>=9.1.1; extra == "structlog"
+Provides-Extra: structlog-celery
+Requires-Dist: structlog>=25.4.0; extra == "structlog-celery"
+Requires-Dist: django-structlog[celery]>=9.1.1; extra == "structlog-celery"
 Dynamic: license-file
 
 Open API Framework
 ==================
 
-:Version: 0.12.0
+:Version: 0.13.0
 :Source: https://github.com/maykinmedia/open-api-framework
 :Keywords: metapackage, dependencies
 

{open_api_framework-0.12.0 → open_api_framework-0.13.0}/open_api_framework.egg-info/SOURCES.txt

@@ -30,5 +30,6 @@ open_api_framework/templatetags/doc_tags.py
 open_api_framework/templatetags/open_api_framework.py
 tests/test_admin.py
 tests/test_config_helpers.py
+tests/test_csp.py
 tests/test_generate_envvar_docs.py
 tests/test_settings.py

{open_api_framework-0.12.0 → open_api_framework-0.13.0}/open_api_framework.egg-info/requires.txt

@@ -32,7 +32,7 @@ django-cors-headers>=4.4.0
 pytest-cov
 
 [csp]
-django-csp>=3.8
+django-csp>=4.0
 
 [docs]
 sphinx
@@ -60,6 +60,14 @@ bleach>=6.1.0
 [server]
 uwsgi>=2.0.23
 
+[structlog]
+structlog>=25.4.0
+django-structlog>=9.1.1
+
+[structlog-celery]
+structlog>=25.4.0
+django-structlog[celery]>=9.1.1
+
 [tests]
 pytest
 pytest-django

{open_api_framework-0.12.0 → open_api_framework-0.13.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "open-api-framework"
-version = "0.12.0"
+version = "0.13.0"
 description = "A metapackage for registration components, that bundles the dependencies shared between these components and provides generic settings"
 authors = [
     {name = "Maykin Media", email = "support@maykinmedia.nl"}
@@ -88,7 +88,7 @@ geo = [
     "djangorestframework-gis>=1.0"
 ]
 csp = [
-    "django-csp>=3.8"
+    "django-csp>=4.0"
 ]
 commonground = [
     "commonground-api-common>=2.7.0"
@@ -105,6 +105,15 @@ server = [
 redis = [
     "django-redis>=5.4.0"
 ]
+structlog = [
+    "structlog>=25.4.0",
+    "django-structlog>=9.1.1"
+]
+structlog-celery = [
+    "structlog>=25.4.0",
+    "django-structlog[celery]>=9.1.1"
+]
+
 
 [tool.setuptools.packages.find]
 include = ["open_api_framework*"]
@@ -120,7 +129,7 @@ env = [
 ]
 
 [tool.bumpversion]
-current_version = "0.12.0"
+current_version = "0.13.0"
 files = [
     {filename = "pyproject.toml"},
     {filename = "README.rst"},

open_api_framework-0.13.0/tests/test_csp.py

@@ -0,0 +1,66 @@
+import testapp.settings
+
+
+def test_csp_default(client):
+    response = client.get("/dummy/")
+
+    csp_header = response.headers.get("Content-Security-Policy")
+
+    assert csp_header is not None
+    assert "default-src 'self'" in csp_header
+    assert "base-uri 'self'" in csp_header
+    assert "object-src 'none'" in csp_header
+    assert "worker-src 'self' blob:" in csp_header
+    assert "style-src 'self' 'unsafe-inline' fonts.googleapis.com" in csp_header
+    assert "frame-ancestors 'none'" in csp_header
+    assert "script-src 'self' 'unsafe-inline'" in csp_header
+    assert "font-src 'self' fonts.gstatic.com" in csp_header
+    assert "img-src 'self' data: cdn.redoc.ly" in csp_header
+    assert "form-action 'self'" in csp_header
+    assert "frame-src 'self'" in csp_header
+
+
+def test_csp_with_envvars_configured(client, monkeypatch, settings):
+    monkeypatch.setenv("CSP_EXTRA_DEFAULT_SRC", "https://default.local")
+    monkeypatch.setenv("CSP_EXTRA_FORM_ACTION", "https://form-action.local")
+    monkeypatch.setenv("CSP_EXTRA_IMG_SRC", "https://img.local")
+    monkeypatch.setenv("CSP_OBJECT_SRC", "https://object.local")
+    monkeypatch.setenv("CSP_REPORT_URI", "https://report-uri.local")
+    monkeypatch.setenv("CSP_REPORT_PERCENTAGE", "0.5")
+
+    # Unfortunately it's not possible to reload settings with envvars, so we manually
+    # recompute the CSP setting instead
+    settings.CONTENT_SECURITY_POLICY = testapp.settings.get_content_security_policy()
+    try:
+        response = client.get("/dummy/")
+
+        csp_header = response.headers.get("Content-Security-Policy")
+
+        # Number between 0 and 1 should be converted to number between 0 and 100
+        assert settings.CONTENT_SECURITY_POLICY["REPORT_PERCENTAGE"] == 50
+
+        assert csp_header is not None
+        assert "default-src 'self' https://default.local" in csp_header
+        assert "base-uri 'self'" in csp_header
+        assert "object-src https://object.local" in csp_header
+        assert "worker-src 'self' blob:" in csp_header
+        assert (
+            "style-src 'self' https://default.local 'unsafe-inline' fonts.googleapis.com"
+            in csp_header
+        )
+        assert "frame-ancestors 'none'" in csp_header
+        assert "script-src 'self' https://default.local 'unsafe-inline'" in csp_header
+        assert "font-src 'self' fonts.gstatic.com" in csp_header
+        assert (
+            "img-src 'self' https://default.local data: cdn.redoc.ly https://img.local"
+            in csp_header
+        )
+        assert "form-action 'self'" in csp_header
+        assert "frame-src 'self'" in csp_header
+    finally:
+        # Manually revert the CSP back to the default, to make sure the envvar docs
+        # do not use the envvar values used in this test
+        monkeypatch.undo()
+        settings.CONTENT_SECURITY_POLICY = (
+            testapp.settings.get_content_security_policy()
+        )

{open_api_framework-0.12.0 → open_api_framework-0.13.0}/tests/test_generate_envvar_docs.py

@@ -52,6 +52,8 @@ Logging
 * ``LOG_LEVEL``: control the verbosity of logging output. Available values are ``CRITICAL``, ``ERROR``, ``WARNING``, ``INFO`` and ``DEBUG``. Defaults to: ``INFO``.
 * ``LOG_QUERIES``: enable (query) logging at the database backend level. Note that you must also set ``DEBUG=1``, which should be done very sparingly!. Defaults to: ``False``.
 * ``LOG_REQUESTS``: enable logging of the outgoing requests. This must be enabled along with `LOG_OUTGOING_REQUESTS_DB_SAVE` to save outgoing request logs in the database. Defaults to: ``False``.
+* ``LOG_FORMAT_CONSOLE``: The format for the console logging handler, possible options: ``json``, ``plain_console``. Defaults to: ``json``.
+* ``ENABLE_STRUCTLOG_REQUESTS``: enable structured logging of requests. Defaults to: ``True``.
 * ``LOG_OUTGOING_REQUESTS_EMIT_BODY``: Whether or not outgoing request bodies should be logged. Defaults to: ``True``.
 * ``LOG_OUTGOING_REQUESTS_DB_SAVE``: Whether or not outgoing request logs should be saved to the database. Defaults to: ``False``.
 * ``LOG_OUTGOING_REQUESTS_DB_SAVE_BODY``: Whether or not outgoing request bodies should be saved to the database. Defaults to: ``True``.
@@ -87,12 +89,12 @@ Content Security Policy
 -----------------------
 
 * ``CSP_EXTRA_DEFAULT_SRC``: Extra default source URLs for CSP other than ``self``. Used for ``img-src``, ``style-src`` and ``script-src``. Defaults to: ``[]``.
-* ``CSP_REPORT_URI``: URI of the``report-uri`` directive. Defaults to: ``None``.
-* ``CSP_REPORT_PERCENTAGE``: Percentage of requests that get the ``report-uri`` directive. Defaults to: ``0``.
-* ``CSP_EXTRA_FORM_ACTION``: Add additional ``form-action`` source to the default . Defaults to: ``[]``.
-* ``CSP_FORM_ACTION``: Override the default ``form-action`` source. Defaults to: ``['"\\'self\\'"']``.
-* ``CSP_EXTRA_IMG_SRC``: Extra ``img-src`` sources for CSP other than ``CSP_DEFAULT_SRC``. Defaults to: ``[]``.
-* ``CSP_OBJECT_SRC``: ``object-src`` urls. Defaults to: ``['"\\'none\\'"']``.
+* ``CSP_EXTRA_FORM_ACTION``: Additional `form-action` sources. Defaults to: ``[]``.
+* ``CSP_FORM_ACTION``: Override the default `form-action` sources. Defaults to: ``['"\\'self\\'"']``.
+* ``CSP_EXTRA_IMG_SRC``: Extra `img-src` sources. Defaults to: ``[]``.
+* ``CSP_OBJECT_SRC``: `object-src` sources. Defaults to: ``['"\\'none\\'"']``.
+* ``CSP_REPORT_URI``: URI for CSP report-uri directive. Defaults to: ``None``.
+* ``CSP_REPORT_PERCENTAGE``: Fraction (between 0 and 1) of requests to include report-uri directive. Defaults to: ``0.0``.
 
 
 Optional