psyplus 1.0.1__tar.gz → 1.2.0__tar.gz

{psyplus-1.0.1 → psyplus-1.2.0}/PKG-INFO

@@ -1,21 +1,22 @@
 Metadata-Version: 2.1
 Name: psyplus
-Version: 1.0.1
+Version: 1.2.0
 Summary: Generate, read and document pydantic-settings config files in YAML
 Keywords: DZD,pydantic,pydantic-settings,yaml,config
 Author-Email: Tim Bleimehl <bleimehl@helmholtz-munich.de>
 License: MIT
 Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Project-URL: Source, https://github.com/DZD-eV-Diabetes-Research/pydantic-settings-yaml-plus
 Project-URL: Issues, https://github.com/DZD-eV-Diabetes-Research/pydantic-settings-yaml-plus/issues
-Requires-Python: >=3.12
-Requires-Dist: pydantic>=2.0
-Requires-Dist: pydantic-settings>=2.3
-Requires-Dist: ruamel.yaml>=0.17
-Requires-Dist: pyyaml>=6.0
+Requires-Python: >=3.11
+Requires-Dist: pydantic>=2.10
+Requires-Dist: pydantic-settings>=2.13.1
+Requires-Dist: ruamel.yaml>=0.17.5
+Requires-Dist: pyyaml>=6.0.1
 Provides-Extra: test
 Requires-Dist: pytest>=8.0; extra == "test"
 Requires-Dist: pytest-cov>=5.0; extra == "test"
@@ -23,14 +24,14 @@ Description-Content-Type: text/markdown
 
 # pydantic-settings-yaml-plus
 
-A helper library that builds upon [pydantic-settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/) to **generate**, **read**, and **document** YAML config files — with comment headers that explain every setting's type, default value, allowed values, environment variable override, and description.
+A helper library that builds upon [pydantic-settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/) to **generate**, **read**, and **document** YAML config files, with comment headers that explain every setting's type, default value, allowed values, environment variable override, and description.
 
 - [pydantic-settings-yaml-plus](#pydantic-settings-yaml-plus)
   - [Features](#features)
   - [Installation](#installation)
   - [Quick start](#quick-start)
-    - [1 — Define your settings model](#1--define-your-settings-model)
-    - [2 — Generate the YAML config file](#2--generate-the-yaml-config-file)
+    - [1. Define your settings model](#1-define-your-settings-model)
+    - [2. Generate the YAML config file](#2-generate-the-yaml-config-file)
   - [Generated YAML output](#generated-yaml-output)
   - [Reading the config](#reading-the-config)
   - [Generating Markdown documentation](#generating-markdown-documentation)
@@ -44,6 +45,7 @@ A helper library that builds upon [pydantic-settings](https://docs.pydantic.dev/
     - [Useful `Field()` kwargs](#useful-field-kwargs)
     - [Configuring env vars](#configuring-env-vars)
   - [Environment variable overrides](#environment-variable-overrides)
+    - [Single list elements](#single-list-elements)
   - [Development](#development)
     - [Smoke test](#smoke-test)
     - [Running tests with coverage](#running-tests-with-coverage)
@@ -53,11 +55,12 @@ A helper library that builds upon [pydantic-settings](https://docs.pydantic.dev/
 
 ## Features
 
-- **Generate documented YAML** — each field gets a comment block with its type, required flag, default value, allowed enum values, corresponding environment variable, description, and examples.
-- **Nested models** — `BaseModel` sub-models are rendered as indented YAML sections with full comment headers.
-- **Read back with env-var override** — `load()` loads the YAML file and still applies environment variable overrides using pydantic-settings' standard machinery.
-- **Markdown docs** — generate a Markdown reference document straight from your settings class.
-- **Zero boilerplate** — define your settings in a standard `pydantic_settings.BaseSettings` subclass; psyplus does the rest.
+- **Generate documented YAML:** each field gets a comment block with its type, required flag, default value, allowed enum values, corresponding environment variable, description, and examples.
+- **Nested models:** `BaseModel` sub-models are rendered as indented YAML sections with full comment headers.
+- **Read back with env-var override:** `load()` loads the YAML file and still applies environment variable overrides using pydantic-settings' standard machinery.
+- **Single list elements from env vars:** `load()` also reads env vars like `APP_SERVERS__0__HOST` that change one element of a list from the YAML file.
+- **Markdown docs:** generate a Markdown reference document straight from your settings class.
+- **Zero boilerplate:** define your settings in a standard `pydantic_settings.BaseSettings` subclass; psyplus does the rest.
 
 ---
 
@@ -67,13 +70,13 @@ A helper library that builds upon [pydantic-settings](https://docs.pydantic.dev/
 pip install psyplus
 ```
 
-**Requirements:** Python ≥ 3.12, pydantic ≥ 2.0, pydantic-settings ≥ 2.3
+**Requirements:** Python ≥ 3.11, pydantic ≥ 2.10, pydantic-settings ≥ 2.13.1, ruamel.yaml ≥ 0.17.5, PyYAML ≥ 6.0.1
 
 ---
 
 ## Quick start
 
-### 1 — Define your settings model
+### 1. Define your settings model
 
 ```python
 # config.py
@@ -128,7 +131,7 @@ class MyAppConfig(BaseSettings):
 
 > **Note on `model_config`:** use `SettingsConfigDict` (pydantic-settings v2 style) rather than the old inner `class Config`. Both work, but `SettingsConfigDict` is preferred.
 
-### 2 — Generate the YAML config file
+### 2. Generate the YAML config file
 
 ```python
 from psyplus import YamlSettingsPlus
@@ -179,7 +182,7 @@ storage_dir: /home/user/.config/myapp
 # Required:    True
 # Env-var:     'APP_ADMIN_PW'
 # Description: The init password for the admin account
-admin_pw: ''
+admin_pw:
 
 # ## database_server ###
 # Type:        Object (DatabaseServerSettings)
@@ -218,22 +221,28 @@ database_server:
   # Type:        List of str
   # Required:    True
   # Env-var:     'APP_DATABASE_SERVER__DATABASE_NAMES'
+  #              'APP_DATABASE_SERVER__DATABASE_NAMES__<list-index>' (one element, 0 is the first; psyplus load() only)
   # Description: The names of the databases to use
   # Example:
   #  >database_names:
   #  >- mydb
   #  >- theotherdb
-  database_names: []
+  database_names:
 
 # ## init_values ###
 # Type:     Dictionary of (str, str)
 # Required: True
 # Env-var:  'APP_INIT_VALUES'
-init_values: {}
+#           'APP_INIT_VALUES__<dict-key>' sets one entry, its key arrives in lower case
+init_values:
 ```
 
-Required fields without defaults are written with an empty placeholder (`''`, `[]`, `{}`).  
-The `Required: True` comment tells the user they must fill these in before the app will validate.
+Required fields without a default are written with an empty value (`admin_pw:`, which YAML reads as null),
+whatever their type. A required sub-model without a default shows its fields, so the structure is
+visible, with its own required fields empty. The `Required: True` comment marks every value to fill in.
+Until that is done, `load()` raises a pydantic `ValidationError` that names each empty required field,
+so an app does not start with an unfilled config. (A required `Optional[...]` field is the exception:
+null is a valid value for it.)
 
 ---
 
@@ -250,6 +259,8 @@ print(config.database_server.host)   # → "localhost" (or whatever is in the YA
 print(config.log_level)              # → "INFO"
 ```
 
+`load()` returns an instance of your model class itself (`type(config) is MyAppConfig`), validated once, exactly as if you had called `MyAppConfig()`. It compares equal to and pickles like any other `MyAppConfig` instance.
+
 `load()` honours environment variable overrides through pydantic-settings' standard source priority: **env vars beat YAML values**. For example:
 
 ```bash
@@ -258,10 +269,10 @@ export APP_LOG_LEVEL=DEBUG
 
 …will make `config.log_level == "DEBUG"` even if the YAML file says `INFO`.
 
-> **Tip:** if you prefer to keep psyplus out of your runtime path, configure `yaml_file` directly in your model and call it without psyplus:
+> **Tip:** if you prefer to keep psyplus out of your runtime path, configure `yaml_file` directly in your model and add a YAML source (setting `yaml_file` alone is not enough, pydantic-settings does not read it by default):
 >
 > ```python
-> from pydantic_settings import BaseSettings, SettingsConfigDict
+> from pydantic_settings import BaseSettings, SettingsConfigDict, YamlConfigSettingsSource
 >
 > class MyAppConfig(BaseSettings):
 >     model_config = SettingsConfigDict(
@@ -270,8 +281,19 @@ export APP_LOG_LEVEL=DEBUG
 >         env_nested_delimiter="__",
 >     )
 >
-> config = MyAppConfig()  # reads YAML + env vars automatically
+>     @classmethod
+>     def settings_customise_sources(
+>         cls, settings_cls, init_settings, env_settings, dotenv_settings, file_secret_settings
+>     ):
+>         return (
+>             init_settings, env_settings, dotenv_settings, file_secret_settings,
+>             YamlConfigSettingsSource(settings_cls),
+>         )
+>
+> config = MyAppConfig()  # reads YAML + env vars
 > ```
+>
+> Such a model still works with `load()`: psyplus then reads its own `file_path` in place of `yaml_file` (see [Environment variable overrides](#environment-variable-overrides)).
 
 ---
 
@@ -294,6 +316,11 @@ md = YamlSettingsPlus(MyAppConfig).render_markdown()
 
 Both produce a document with a property table for every field: type, required flag, default, allowed values, environment variable, description, and examples.
 
+The fields of a nested model get their own sections, headed by their full path (`database.host`,
+`servers[*].port`). A field that holds one of several models (`Union[Cat, Dog]`) gets one section
+per model, labelled `option 1 of 2` and so on. Markdown has no headings below level 6, so deeper
+fields stay at level 6 and the path in the heading shows where they belong.
+
 ---
 
 ## Example gallery
@@ -304,7 +331,7 @@ The table below links the bundled test models to their pre-generated output so y
 |---|---|---|---|---|
 | `SimpleModel` | [tests/models/simple.py](tests/models/simple.py) | Flat settings: scalars, Optional, Literal, List, Dict, Annotated metadata, `default_factory` | [SimpleModel.yaml](smoke_output/SimpleModel.yaml) | [SimpleModel.md](smoke_output/SimpleModel.md) |
 | `NestedModel` | [tests/models/nested.py](tests/models/nested.py) | Nested `BaseModel` sub-field, optional sub-model, env-var path generation with `env_nested_delimiter` | [NestedModel.yaml](smoke_output/NestedModel.yaml) | [NestedModel.md](smoke_output/NestedModel.md) |
-| `MultiNestedModel` | [tests/models/nested.py](tests/models/nested.py) | `List[BaseModel]` and `Dict[str, BaseModel]` fields — YAML renders each item, Markdown recurses into the item schema | [MultiNestedModel.yaml](smoke_output/MultiNestedModel.yaml) | [MultiNestedModel.md](smoke_output/MultiNestedModel.md) |
+| `MultiNestedModel` | [tests/models/nested.py](tests/models/nested.py) | `List[BaseModel]` and `Dict[str, BaseModel]` fields; YAML renders each item, Markdown recurses into the item schema | [MultiNestedModel.yaml](smoke_output/MultiNestedModel.yaml) | [MultiNestedModel.md](smoke_output/MultiNestedModel.md) |
 | `MyAppConfig` | [tests/models/readme_example.py](tests/models/readme_example.py) | The full README quick-start model with nested database settings and field examples | [MyAppConfig.yaml](smoke_output/MyAppConfig.yaml) | [MyAppConfig.md](smoke_output/MyAppConfig.md) |
 
 These files are regenerated automatically by the smoke script (`pdm run smoke`) and committed to the repository, so they are always in sync with the current code.
@@ -328,7 +355,7 @@ These files are regenerated automatically by the smoke script (`pdm run smoke`)
 | `generate_markdown_doc(output_path)` | Write a Markdown reference document to `output_path`. |
 | `render_yaml() → str` | Render the documented YAML as a string without touching the filesystem. |
 | `render_markdown() → str` | Render the Markdown reference document as a string without touching the filesystem. |
-| `load() → BaseSettings` | Load `file_path`, apply env var overrides, return a validated settings instance. |
+| `load() → model` | Load `file_path`, apply env var overrides (including [single list elements](#single-list-elements)), return a validated instance of `model`. |
 
 ### `YamlFileGenerator(settings_class, indent_size=2)`
 
@@ -368,7 +395,8 @@ psyplus works with any standard `pydantic_settings.BaseSettings` subclass. The r
 | `title="…"` | Appended to the `## field_name - Title ###` header |
 | `examples=[…]` | `# Example:` block with rendered YAML |
 | `default=…` | `# Default:` comment row |
-| `default_factory=lambda: …` | `# Default:` not shown (factory evaluated at generation time) |
+| `default_factory=lambda: …` | `# Default:` not shown (factory evaluated at generation time, its result is the value in the YAML file) |
+| `default_factory=lambda data: …` | Same, the factory gets the values of the fields defined before it. If it needs a required value without a default, the field is written empty with a note to remove the key |
 | `max_length=…` / `gt=…` / etc. | `# Constraints:` comment row |
 
 ### Configuring env vars
@@ -394,6 +422,26 @@ For nested models the env var path is built as:
 # e.g. MYAPP_DATABASE__HOST  (prefix="MYAPP_", delimiter="__")
 ```
 
+What pydantic-settings can set through env vars, and what the generated files show:
+
+| Field | Env var example | Behaviour |
+|---|---|---|
+| Field of a nested model, any depth | `MYAPP_DATABASE__SERVER__HOST=db` | Sets only this value. The other values from the YAML file stay. |
+| Whole nested model | `MYAPP_DATABASE='{"name": "x"}'` | JSON, merged into the values from the YAML file. |
+| List (`List[str]`, `List[SubModel]`, tuple, set) | `MYAPP_HOSTS='["a", "b"]'` | JSON, replaces the whole list. |
+| One list element, or a field of it | `MYAPP_HOSTS__1=b`, `MYAPP_SERVERS__0__PORT=8080` | Only with `load()`, see [Single list elements](#single-list-elements). `<list-index>` stands for the position, 0 is the first. Changes this element of the list from the YAML file, the other elements stay. |
+| Dict value (`Dict[str, ...]`) | `MYAPP_LABELS__team=core` | `<dict-key>` stands for the key. Env var names are case-insensitive by default, so the key arrives in lower case, and the generated files say so next to the env var of the dict field. Keys in a JSON value keep their case. |
+
+Only the root model's `env_prefix` and `env_nested_delimiter` count. A nested `BaseSettings`
+sub-model's own `env_prefix` is not used for its fields. Without `env_nested_delimiter`, nested
+fields have no env var of their own and can only be set through the JSON value of the top-level
+field. With `case_sensitive=True`, the env var names keep the case of the field names. With
+`env_nested_max_split`, only fields that many levels below the top-level field have an env var
+(a list index counts as a level).
+A sub-model that defaults to null has no fields in the YAML file, so its comment lists the env
+vars of its fields under `Env-vars of the sub-fields:`. For a field that holds one of several
+models (`Union[Cat, Dog]`), this list covers the fields of every model.
+
 #### Nullable fields
 
 A nullable field (`Optional[int]`, `int | None`) shows its type as `int or null`. By default,
@@ -416,18 +464,102 @@ For a nullable field whose default is not null, the `Env-var:` hint says which c
 without it. A field that defaults to null gets no hint, since leaving the env var unset already
 gives null.
 
+### Fields with an alias
+
+pydantic-settings reads the YAML file and the env vars by a field's validation alias, so psyplus
+uses it as the YAML key, in the YAML path and in the env var name. That covers `alias=`,
+`validation_alias=` and an `alias_generator` such as `to_camel`, also with `populate_by_name=True`.
+A `serialization_alias` changes nothing.
+
+```python
+class Database(BaseModel):
+    host_name: str = Field("localhost", alias="hostName")
+
+class MyConfig(BaseSettings):
+    model_config = SettingsConfigDict(env_prefix="MYAPP_", env_nested_delimiter="__")
+
+    log_level: str = Field("INFO", alias="logLevel")
+    # → YAML key: logLevel, Env-var: 'LOGLEVEL'
+    database: Database = Database()
+    # → YAML path: database.hostName, Env-var: 'MYAPP_DATABASE__HOSTNAME'
+```
+
+| Field | YAML key | Env var |
+|---|---|---|
+| `alias="logLevel"` or `validation_alias="logLevel"` | `logLevel` | `LOGLEVEL` |
+| `validation_alias=AliasChoices("a", "b")` | `a`, the first string choice | `A` |
+| `validation_alias=AliasPath("database", "host")` | `host` inside a `database` map | none |
+| `serialization_alias="x"` only | the field name | as without an alias |
+
+* **Prefix:** pydantic-settings puts `env_prefix` in front of an alias only when the model sets
+  `env_prefix_target="alias"` or `"all"`. With the default `"variable"` only field names get it.
+  What counts is the top-level field: `MYAPP_DATABASE__HOSTNAME` above gets the prefix because
+  `database` has no alias.
+* **Case:** without `case_sensitive=True` env var names arrive in lower case, and
+  pydantic-settings matches them back to a camelCase key only along plain sub-model fields. Below a
+  dict, a Union of models, or an `Optional` sub-model deeper down, a key with upper case letters can
+  not be set through its own env var, so none is shown there. The JSON value of the field above
+  still works.
+* **`populate_by_name`:** below a top-level field with an alias, the fields and dict entries have
+  no env var of their own. pydantic-settings files them under the field name, which clashes with the alias key
+  from the YAML file. The JSON value of the top-level field still works.
+* **`AliasPath`:** a path of names is written as nested maps, which several fields may share. A
+  path with a list index (`AliasPath("servers", 0)`) or one that clashes with another key is not
+  written; its comment block says where to add it. No env var sets an `AliasPath` field:
+  pydantic-settings would read each element of the path as an env var of its own.
+* **Alias and validation alias differ:** pydantic-settings reads a top-level field from the YAML
+  file only under its `alias`. When validation expects another name (`alias="a",
+  validation_alias="b"`, an `alias_generator` next to an explicit `validation_alias`, or
+  `validate_by_alias=False`), no key works, so the field is left out of the YAML file. A note in
+  the YAML file and in the Markdown doc says so. Set it through its env var instead.
+
 ---
 
 ## Environment variable overrides
 
-`load()` creates a thin dynamic subclass that adds the YAML file as a pydantic-settings source while keeping the normal env var source with higher priority. The effective source priority (highest → lowest) is:
+`load()` uses the same sources pydantic-settings uses for your model and adds the YAML file as the lowest priority source, just above the field defaults. For a model without its own `settings_customise_sources` the effective source priority (highest → lowest) is:
 
 1. **Environment variables** (respects `env_prefix` / `env_nested_delimiter`)
-2. **YAML file** values
-3. **Model defaults**
+2. **`.env` file** (if the model sets `env_file`)
+3. **Secrets directory** (if the model sets `secrets_dir`)
+4. **YAML file** values
+5. **Model defaults**
 
 No changes to your settings class are required.
 
+If your model defines `settings_customise_sources`, `load()` respects it: the YAML file is added after the sources it returns, and a source it leaves out stays out. A model that drops `env_settings`, for example, ignores env vars in `load()` too. If it already returns a `YamlConfigSettingsSource`, `load()` adds no second one but reads `file_path` through it, at the position your model chose.
+
+Command line parsing configured in `model_config` (`cli_parse_args`, `cli_settings_source`) also applies to `load()`, with the command line beating all other sources.
+
+A missing YAML file is not an error: `load()` then returns the model defaults plus whatever the env vars, the `.env` file and the secrets directory set.
+
+### Single list elements
+
+pydantic-settings itself can only replace a whole list through an env var (`APP_SERVERS='[{"host": "a"}]'`). `load()` also reads env vars that address one element by its position, in the environment and in the `.env` file:
+
+```bash
+export APP_SERVERS__0__HOST=db.internal   # field of the first element
+export APP_TAGS__1=beta                   # second element of a List[str]
+export APP_GROUPS__admins__0=ann          # list inside a dict
+export APP_MATRIX__1__0=7                 # list inside a list
+```
+
+The generated YAML comments and Markdown docs show these names with `<list-index>` in place of the position (`APP_SERVERS__<list-index>__HOST`). They work with `load()` only: a model instantiated directly (`MyAppConfig()`) reads them the way pydantic-settings does, which fails list validation.
+
+The env var changes the list that the lower priority sources provide (usually the YAML file) instead of replacing it:
+
+- **Index below the list length:** changes that element. The fields of an element model are merged into it (`APP_SERVERS__0__HOST` keeps the element's `port`), the same goes for a JSON object (`APP_SERVERS__0='{"host": "x"}'`). Any other value replaces the element.
+- **Index equal to the list length:** adds an element. Several new elements need consecutive indices (2, 3, 4 for a list of two). A new element that is a model needs its required fields set too.
+- **Index further beyond the end:** `load()` fails with an error that names the env var.
+- **List not in the YAML file:** the env var changes the field's default list. A field without a default (or with `null`) starts as an empty list, so its indices start at 0. A default from a `default_factory` that takes the other values (`lambda data: ...`) can not be changed this way: `load()` fails with an error, put the list in the YAML file or set it as a whole.
+- **Aliases:** the list is found in the YAML file under the field name or any of its aliases, and the changed list stays under that key.
+- **Whole list and elements together:** an index env var changes the whole list value from the same place (`APP_TAGS='["a", "b"]'` plus `APP_TAGS__1=c` gives `["a", "c"]`). A whole value that is not a JSON list there (a comma separated string for a `NoDecode` field, for example) can not be changed this way, `load()` fails with an error. A whole list from a higher priority source replaces everything below it, as usual.
+- **Not a list index:** `-1` or `01` where a list is expected is an error that names the env var. Other names (`APP_TAGS__NAME`) are left to pydantic-settings, as without psyplus.
+- **A list or a dict (`Union[List[str], Dict[str, str]]`):** a number is a list index only where the value it changes is a list. Where it is a dict, the number is a dict key, as pydantic-settings reads it.
+- **Tuples and sets** work like lists. For a set, the index counts the elements in the order the YAML file lists them.
+
+Only the env and `.env` sources that `load()` hands to your model's `settings_customise_sources` (`env_settings`, `dotenv_settings`) read list indices. A source your model builds itself (`EnvSettingsSource(settings_cls, env_prefix=...)`) works as pydantic-settings makes it, without list index support. Under `load()`, `env_settings` and `dotenv_settings` are subclasses of `EnvSettingsSource` and `DotEnvSettingsSource`: `isinstance()` checks on them work, `type(...) is EnvSettingsSource` does not.
+
 ---
 
 ## Development
@@ -456,7 +588,7 @@ To visually inspect the generated YAML and Markdown for all built-in test models
 pdm run smoke
 ```
 
-This writes four files per model (YAML config template + Markdown reference) to `smoke_output/` and prints a preview of each to the terminal. The output is committed to the repository and linked from the [Example gallery](#example-gallery) section — re-run the script and commit the results whenever the generator logic changes.
+This writes two files per model (YAML config template + Markdown reference) to `smoke_output/` and prints a preview of each to the terminal. The output is committed to the repository and linked from the [Example gallery](#example-gallery) section. Re-run the script and commit the results whenever the generator logic changes.
 
 ```
 smoke_output/