django-aqueduct 0.1.0__tar.gz

django_aqueduct-0.1.0/.gitignore

@@ -0,0 +1,38 @@
+# Python
+__pycache__/
+*.py[cod]
+*.pyo
+*.pyd
+.Python
+*.egg
+*.egg-info/
+dist/
+build/
+.eggs/
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# uv
+uv.lock
+
+# Testing / coverage
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+
+# mypy
+.mypy_cache/
+
+# IDEs
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db

django_aqueduct-0.1.0/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2025, MIT Open Learning Engineering
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

django_aqueduct-0.1.0/PKG-INFO

@@ -0,0 +1,326 @@
+Metadata-Version: 2.4
+Name: django-aqueduct
+Version: 0.1.0
+Summary: Structured, typed, auditable Django settings management powered by Pydantic
+Project-URL: Homepage, https://github.com/mitodl/django-aqueduct
+Project-URL: Repository, https://github.com/mitodl/django-aqueduct
+Project-URL: Issues, https://github.com/mitodl/django-aqueduct/issues
+Author-email: MIT Open Learning Engineering <odl-devops@mit.edu>
+License-Expression: BSD-3-Clause
+License-File: LICENSE
+Keywords: configuration,django,kubernetes,pydantic,settings,vault
+Classifier: Development Status :: 3 - Alpha
+Classifier: Framework :: Django
+Classifier: Framework :: Django :: 5.0
+Classifier: Framework :: Django :: 5.1
+Classifier: Framework :: Django :: 5.2
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Internet :: WWW/HTTP
+Requires-Python: >=3.12
+Requires-Dist: django>=5.0
+Requires-Dist: pydantic-settings>=2.0
+Requires-Dist: pydantic>=2.0
+Provides-Extra: aws
+Requires-Dist: boto3>=1.26; extra == 'aws'
+Provides-Extra: mitol
+Requires-Dist: mitol-django-common>=2023.1.1; extra == 'mitol'
+Provides-Extra: vault
+Requires-Dist: hvac>=2.0; extra == 'vault'
+Description-Content-Type: text/markdown
+
+# django-aqueduct
+
+Structured, typed, auditable Django settings management powered by Pydantic.
+
+`django-aqueduct` channels configuration from multiple sources — environment variables, YAML files, HashiCorp Vault, AWS SSM Parameter Store — into a single typed, validated model, making settings auditable and K8s-friendly without changing any application code.
+
+[![CI](https://github.com/mitodl/django-aqueduct/actions/workflows/ci.yml/badge.svg)](https://github.com/mitodl/django-aqueduct/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/django-aqueduct)](https://pypi.org/project/django-aqueduct/)
+[![Python](https://img.shields.io/pypi/pyversions/django-aqueduct)](https://pypi.org/project/django-aqueduct/)
+[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](LICENSE)
+
+---
+
+## Installation
+
+```bash
+pip install django-aqueduct
+
+# Optional extras
+pip install django-aqueduct[vault]   # HashiCorp Vault support (hvac)
+pip install django-aqueduct[aws]     # AWS SSM Parameter Store (boto3)
+pip install django-aqueduct[mitol]   # mitol-django-common EnvParser integration
+```
+
+Add to `INSTALLED_APPS`:
+
+```python
+INSTALLED_APPS = [
+    ...
+    "django_aqueduct",
+]
+```
+
+---
+
+## Quickstart
+
+### Step 1 — Generate a scaffold
+
+Point `generate_aqueduct_settings` at your existing settings module:
+
+```bash
+python manage.py generate_aqueduct_settings \
+    --modules myapp.settings.common \
+    --output src/myapp/settings_model.py
+```
+
+This emits a typed `AqueductSettings(BaseSettings)` class with every
+`UPPERCASE` name from your settings module as a Pydantic field, grouped
+under section comments by source module.
+
+### Step 2 — Refine the scaffold
+
+Open `settings_model.py` and:
+
+- Fix any `# TODO: refine type` annotations
+- Add `model_validator` methods to derive complex objects from primitives:
+
+```python
+from pydantic import model_validator
+import dj_database_url
+
+class AqueductSettings(BaseSettings):
+    DATABASE_URL: str = Field(default="sqlite:///db.sqlite3")
+
+    # Derived — populated by the validator below
+    DATABASES: dict[str, Any] = Field(default_factory=dict)
+
+    @model_validator(mode="after")
+    def build_databases(self) -> "AqueductSettings":
+        self.DATABASES = {"default": dj_database_url.parse(self.DATABASE_URL)}
+        return self
+```
+
+### Step 3 — Wire the shim
+
+Replace your host settings file with a thin shim:
+
+```python
+# myapp/settings/production.py
+from django_aqueduct import configure_django_settings
+from myapp.settings_model import AqueductSettings
+
+configure_django_settings(AqueductSettings)
+```
+
+That's it. `DJANGO_SETTINGS_MODULE` stays the same. All existing
+`django.conf.settings.FOO` access in application code continues to work
+with zero changes.
+
+---
+
+## Kubernetes deployment pattern
+
+In Kubernetes, configuration typically arrives from multiple sources:
+
+| Source | Typical content |
+|--------|----------------|
+| Pod environment variables | Non-secret config from ConfigMaps |
+| Vault (Kubernetes SA auth) | Database passwords, API keys |
+| AWS SSM Parameter Store | Secrets in AWS-hosted deployments |
+
+Configure all three in your settings model:
+
+```python
+from django_aqueduct import configure_django_settings
+from django_aqueduct.sources.vault import VaultSettingsSource
+from django_aqueduct.sources.aws_ssm import AWSParameterStoreSource
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+
+class ProductionSettings(BaseSettings):
+    model_config = SettingsConfigDict(extra="allow")
+
+    SECRET_KEY: str = Field(...)
+    DATABASE_URL: str = Field(...)
+
+    @classmethod
+    def settings_customise_sources(cls, settings_cls, **kwargs):
+        return (
+            # 1. Environment variables (from K8s ConfigMaps)
+            kwargs["env_settings"],
+            # 2. Vault via Kubernetes SA — reads JWT from default mount path
+            #    /var/run/secrets/kubernetes.io/serviceaccount/token
+            VaultSettingsSource(
+                settings_cls,
+                vault_url="https://vault.example.com",
+                vault_path="myapp/production",
+                auth_method="kubernetes",
+                role="myapp",
+                # Optional: custom JWT path for projected service accounts
+                # jwt_path="/var/run/secrets/custom/token",
+            ),
+        )
+
+
+# myapp/settings/production.py
+configure_django_settings(ProductionSettings)
+```
+
+### Vault authentication methods
+
+| Method | When to use |
+|--------|-------------|
+| `"token"` | Local dev, CI with a static token |
+| `"oidc"` | Interactive / browser-based login |
+| `"kubernetes"` | Production K8s — uses the pod's service account JWT |
+
+```python
+# Token auth (dev/CI)
+VaultSettingsSource(settings_cls, ..., auth_method="token", vault_token="s.xxx")
+
+# OIDC (interactive)
+VaultSettingsSource(settings_cls, ..., auth_method="oidc", role="myapp")
+
+# Kubernetes SA (production) — custom JWT path
+VaultSettingsSource(
+    settings_cls,
+    ...,
+    auth_method="kubernetes",
+    role="myapp",
+    jwt_path="/var/run/secrets/tokens/vault",  # projected SA token
+)
+```
+
+### AWS SSM Parameter Store
+
+```python
+from django_aqueduct.sources.aws_ssm import AWSParameterStoreSource
+
+# All parameters under /myapp/production/ are fetched with full pagination.
+# The prefix is stripped: /myapp/production/SECRET_KEY → SECRET_KEY
+AWSParameterStoreSource(
+    settings_cls,
+    path_prefix="/myapp/production/",
+    region_name="us-east-1",
+)
+```
+
+---
+
+## Adapter modes
+
+### Option A — Shim settings file (recommended)
+
+`DJANGO_SETTINGS_MODULE` stays unchanged. The settings file becomes a
+thin shim:
+
+```python
+# myapp/settings/production.py
+from django_aqueduct import configure_django_settings
+from myapp.settings_model import ProductionSettings
+
+configure_django_settings(ProductionSettings)
+```
+
+Works with gunicorn, Celery, pytest-django, management commands, and every
+other tool that reads `DJANGO_SETTINGS_MODULE` — no changes required.
+
+### Option B — Programmatic configure (greenfield)
+
+For new projects or container-native apps where you control all entry points
+and want no `DJANGO_SETTINGS_MODULE`:
+
+```python
+# manage.py or WSGI/ASGI entry point — call before django.setup()
+from django_aqueduct import configure_django_programmatic
+from myapp.settings_model import AppSettings
+
+configure_django_programmatic(AppSettings)
+
+import django
+django.setup()
+```
+
+---
+
+## edx-platform migration walkthrough
+
+edx-platform's `lms/envs/production.py` currently loads a YAML file and
+applies hundreds of lines of post-processing. With `django-aqueduct`:
+
+1. **Generate the scaffold** from `common.py`:
+
+   ```bash
+   python manage.py generate_aqueduct_settings \
+       --modules lms.envs.common \
+       --output lms/envs/settings_model.py
+   ```
+
+2. **Review** `settings_model.py` — fix `# TODO: refine type` entries,
+   move `derive_settings` logic into `@model_validator` methods.
+
+3. **Replace** `lms/envs/production.py`:
+
+   ```python
+   # lms/envs/production.py
+   from django_aqueduct import configure_django_settings
+   from lms.envs.settings_model import LMSSettings
+
+   configure_django_settings(LMSSettings)
+   ```
+
+4. Set `DJANGO_SETTINGS_MODULE=lms.envs.production` as before.
+   All LMS app code using `from django.conf import settings` is unchanged.
+
+---
+
+## `[mitol]` extra — EnvParser integration
+
+If your project uses `mitol-django-common`'s `EnvParser`, install the
+`[mitol]` extra and pass `--include-envparser` to the generator:
+
+```bash
+pip install django-aqueduct[mitol]
+
+python manage.py generate_aqueduct_settings \
+    --modules myapp.settings \
+    --include-envparser
+```
+
+The `EnvParserInspector` reads the global `env._configured_vars` registry
+and emits precisely-typed fields for every `get_string`/`get_bool`/`get_int`
+call, preserving `description`, `required`, and `dev_only` metadata.
+
+---
+
+## Contributing
+
+```bash
+git clone https://github.com/mitodl/django-aqueduct
+cd django-aqueduct
+uv sync
+uv run pytest
+uv run mypy src/django_aqueduct
+```
+
+Install pre-commit hooks with [prek](https://prek.j178.dev):
+
+```bash
+pip install prek
+prek install
+```
+
+Please open an issue before submitting a pull request for significant changes.
+
+---
+
+## License
+
+BSD-3-Clause © MIT Open Learning Engineering

django_aqueduct-0.1.0/README.md

@@ -0,0 +1,291 @@
+# django-aqueduct
+
+Structured, typed, auditable Django settings management powered by Pydantic.
+
+`django-aqueduct` channels configuration from multiple sources — environment variables, YAML files, HashiCorp Vault, AWS SSM Parameter Store — into a single typed, validated model, making settings auditable and K8s-friendly without changing any application code.
+
+[![CI](https://github.com/mitodl/django-aqueduct/actions/workflows/ci.yml/badge.svg)](https://github.com/mitodl/django-aqueduct/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/django-aqueduct)](https://pypi.org/project/django-aqueduct/)
+[![Python](https://img.shields.io/pypi/pyversions/django-aqueduct)](https://pypi.org/project/django-aqueduct/)
+[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](LICENSE)
+
+---
+
+## Installation
+
+```bash
+pip install django-aqueduct
+
+# Optional extras
+pip install django-aqueduct[vault]   # HashiCorp Vault support (hvac)
+pip install django-aqueduct[aws]     # AWS SSM Parameter Store (boto3)
+pip install django-aqueduct[mitol]   # mitol-django-common EnvParser integration
+```
+
+Add to `INSTALLED_APPS`:
+
+```python
+INSTALLED_APPS = [
+    ...
+    "django_aqueduct",
+]
+```
+
+---
+
+## Quickstart
+
+### Step 1 — Generate a scaffold
+
+Point `generate_aqueduct_settings` at your existing settings module:
+
+```bash
+python manage.py generate_aqueduct_settings \
+    --modules myapp.settings.common \
+    --output src/myapp/settings_model.py
+```
+
+This emits a typed `AqueductSettings(BaseSettings)` class with every
+`UPPERCASE` name from your settings module as a Pydantic field, grouped
+under section comments by source module.
+
+### Step 2 — Refine the scaffold
+
+Open `settings_model.py` and:
+
+- Fix any `# TODO: refine type` annotations
+- Add `model_validator` methods to derive complex objects from primitives:
+
+```python
+from pydantic import model_validator
+import dj_database_url
+
+class AqueductSettings(BaseSettings):
+    DATABASE_URL: str = Field(default="sqlite:///db.sqlite3")
+
+    # Derived — populated by the validator below
+    DATABASES: dict[str, Any] = Field(default_factory=dict)
+
+    @model_validator(mode="after")
+    def build_databases(self) -> "AqueductSettings":
+        self.DATABASES = {"default": dj_database_url.parse(self.DATABASE_URL)}
+        return self
+```
+
+### Step 3 — Wire the shim
+
+Replace your host settings file with a thin shim:
+
+```python
+# myapp/settings/production.py
+from django_aqueduct import configure_django_settings
+from myapp.settings_model import AqueductSettings
+
+configure_django_settings(AqueductSettings)
+```
+
+That's it. `DJANGO_SETTINGS_MODULE` stays the same. All existing
+`django.conf.settings.FOO` access in application code continues to work
+with zero changes.
+
+---
+
+## Kubernetes deployment pattern
+
+In Kubernetes, configuration typically arrives from multiple sources:
+
+| Source | Typical content |
+|--------|----------------|
+| Pod environment variables | Non-secret config from ConfigMaps |
+| Vault (Kubernetes SA auth) | Database passwords, API keys |
+| AWS SSM Parameter Store | Secrets in AWS-hosted deployments |
+
+Configure all three in your settings model:
+
+```python
+from django_aqueduct import configure_django_settings
+from django_aqueduct.sources.vault import VaultSettingsSource
+from django_aqueduct.sources.aws_ssm import AWSParameterStoreSource
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+
+class ProductionSettings(BaseSettings):
+    model_config = SettingsConfigDict(extra="allow")
+
+    SECRET_KEY: str = Field(...)
+    DATABASE_URL: str = Field(...)
+
+    @classmethod
+    def settings_customise_sources(cls, settings_cls, **kwargs):
+        return (
+            # 1. Environment variables (from K8s ConfigMaps)
+            kwargs["env_settings"],
+            # 2. Vault via Kubernetes SA — reads JWT from default mount path
+            #    /var/run/secrets/kubernetes.io/serviceaccount/token
+            VaultSettingsSource(
+                settings_cls,
+                vault_url="https://vault.example.com",
+                vault_path="myapp/production",
+                auth_method="kubernetes",
+                role="myapp",
+                # Optional: custom JWT path for projected service accounts
+                # jwt_path="/var/run/secrets/custom/token",
+            ),
+        )
+
+
+# myapp/settings/production.py
+configure_django_settings(ProductionSettings)
+```
+
+### Vault authentication methods
+
+| Method | When to use |
+|--------|-------------|
+| `"token"` | Local dev, CI with a static token |
+| `"oidc"` | Interactive / browser-based login |
+| `"kubernetes"` | Production K8s — uses the pod's service account JWT |
+
+```python
+# Token auth (dev/CI)
+VaultSettingsSource(settings_cls, ..., auth_method="token", vault_token="s.xxx")
+
+# OIDC (interactive)
+VaultSettingsSource(settings_cls, ..., auth_method="oidc", role="myapp")
+
+# Kubernetes SA (production) — custom JWT path
+VaultSettingsSource(
+    settings_cls,
+    ...,
+    auth_method="kubernetes",
+    role="myapp",
+    jwt_path="/var/run/secrets/tokens/vault",  # projected SA token
+)
+```
+
+### AWS SSM Parameter Store
+
+```python
+from django_aqueduct.sources.aws_ssm import AWSParameterStoreSource
+
+# All parameters under /myapp/production/ are fetched with full pagination.
+# The prefix is stripped: /myapp/production/SECRET_KEY → SECRET_KEY
+AWSParameterStoreSource(
+    settings_cls,
+    path_prefix="/myapp/production/",
+    region_name="us-east-1",
+)
+```
+
+---
+
+## Adapter modes
+
+### Option A — Shim settings file (recommended)
+
+`DJANGO_SETTINGS_MODULE` stays unchanged. The settings file becomes a
+thin shim:
+
+```python
+# myapp/settings/production.py
+from django_aqueduct import configure_django_settings
+from myapp.settings_model import ProductionSettings
+
+configure_django_settings(ProductionSettings)
+```
+
+Works with gunicorn, Celery, pytest-django, management commands, and every
+other tool that reads `DJANGO_SETTINGS_MODULE` — no changes required.
+
+### Option B — Programmatic configure (greenfield)
+
+For new projects or container-native apps where you control all entry points
+and want no `DJANGO_SETTINGS_MODULE`:
+
+```python
+# manage.py or WSGI/ASGI entry point — call before django.setup()
+from django_aqueduct import configure_django_programmatic
+from myapp.settings_model import AppSettings
+
+configure_django_programmatic(AppSettings)
+
+import django
+django.setup()
+```
+
+---
+
+## edx-platform migration walkthrough
+
+edx-platform's `lms/envs/production.py` currently loads a YAML file and
+applies hundreds of lines of post-processing. With `django-aqueduct`:
+
+1. **Generate the scaffold** from `common.py`:
+
+   ```bash
+   python manage.py generate_aqueduct_settings \
+       --modules lms.envs.common \
+       --output lms/envs/settings_model.py
+   ```
+
+2. **Review** `settings_model.py` — fix `# TODO: refine type` entries,
+   move `derive_settings` logic into `@model_validator` methods.
+
+3. **Replace** `lms/envs/production.py`:
+
+   ```python
+   # lms/envs/production.py
+   from django_aqueduct import configure_django_settings
+   from lms.envs.settings_model import LMSSettings
+
+   configure_django_settings(LMSSettings)
+   ```
+
+4. Set `DJANGO_SETTINGS_MODULE=lms.envs.production` as before.
+   All LMS app code using `from django.conf import settings` is unchanged.
+
+---
+
+## `[mitol]` extra — EnvParser integration
+
+If your project uses `mitol-django-common`'s `EnvParser`, install the
+`[mitol]` extra and pass `--include-envparser` to the generator:
+
+```bash
+pip install django-aqueduct[mitol]
+
+python manage.py generate_aqueduct_settings \
+    --modules myapp.settings \
+    --include-envparser
+```
+
+The `EnvParserInspector` reads the global `env._configured_vars` registry
+and emits precisely-typed fields for every `get_string`/`get_bool`/`get_int`
+call, preserving `description`, `required`, and `dev_only` metadata.
+
+---
+
+## Contributing
+
+```bash
+git clone https://github.com/mitodl/django-aqueduct
+cd django-aqueduct
+uv sync
+uv run pytest
+uv run mypy src/django_aqueduct
+```
+
+Install pre-commit hooks with [prek](https://prek.j178.dev):
+
+```bash
+pip install prek
+prek install
+```
+
+Please open an issue before submitting a pull request for significant changes.
+
+---
+
+## License
+
+BSD-3-Clause © MIT Open Learning Engineering