nested-config 2.0.3__tar.gz

nested_config-2.0.3/CHANGELOG.md

@@ -0,0 +1,57 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [2.0.3] - 2024-04-15
+
+- Fix typing issue regression for Pydantic < 2.0 introduced in last release
+- Move package to `src` directory
+
+## [2.0.2] - 2024-04-12
+
+- Generalize handling of lists and dicts such that if the source config value and the
+  model annotation are both lists, recursively evaluate each item. This addresses the
+  situation where there may be a dict in the source config that corresponds to a Pydantic
+  model and that dict contains paths to other configs.
+
+## [2.0.1] - 2024-04-10
+
+- Make dependency specifications more generous
+- Use `yaml.safe_load`
+- Test minimum dependency versions in CI
+
+## [2.0.0] - 2024-04-09
+
+### Changed
+
+- Project renamed from **pydantic-plus** to **nested-config**
+
+### Added
+
+- Can find paths to other config files and parse them using their respective Pydantic
+  models using `validate_config` or `BaseModel` (this is the main functionality now).
+- Pydantic 2.0 compatibility.
+- Can validate any config file. TOML and JSON built in, YAML optional, others can be
+  added.
+- Validators for `PurePath` and `PureWindowsPath`
+- Simplify JSON encoder specification to work for all `PurePaths`
+- pytest and mypy checks, checked with GitLab CI/CD
+
+## [1.1.3] - 2021-07-30
+
+- Add README
+- Simplify PurePosixPath validator
+- Export `TomlParsingError` from rtoml for downstream exception handling (without needing to explicitly
+  import rtoml).
+
+[Unreleased]: https://gitlab.com/osu-nrsg/nested-config/-/compare/v2.0.3...master
+[2.0.3]: https://gitlab.com/osu-nrsg/nested-config/-/compare/v2.0.2...v2.0.3
+[2.0.2]: https://gitlab.com/osu-nrsg/nested-config/-/compare/v2.0.1...v2.0.2
+[2.0.1]: https://gitlab.com/osu-nrsg/nested-config/-/compare/v2.0.0...v2.0.1
+[2.0.0]: https://gitlab.com/osu-nrsg/nested-config/-/compare/v1.1.3...v2.0.0
+[1.1.3]: https://gitlab.com/osu-nrsg/nested-config/-/tags/v1.1.3

nested_config-2.0.3/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Randall Pittman, Oregon State University
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

nested_config-2.0.3/PKG-INFO

@@ -0,0 +1,208 @@
+Metadata-Version: 2.1
+Name: nested-config
+Version: 2.0.3
+Summary: Parse configuration files that include paths to other config files into Pydantic modelinstances. Also support pathlib.PurePath on Pydantic 1.8+.
+Home-page: https://gitlab.com/osu-nrsg/nested-config
+License: MIT
+Keywords: pydantic,config,configuration files
+Author: Randall Pittman
+Author-email: pittmara@oregonstate.edu
+Requires-Python: >=3.8,<4.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Framework :: Pydantic
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Provides-Extra: yaml
+Requires-Dist: pydantic (>=1.8,<3.0.0)
+Requires-Dist: pyyaml (>=5.1.0,<7.0.0) ; extra == "yaml"
+Requires-Dist: single-version (>=1.6.0,<2.0.0)
+Requires-Dist: tomli (>=2.0.0,<3.0.0) ; python_version < "3.11"
+Requires-Dist: typing-extensions (>=4.6.0,<5.0.0)
+Project-URL: Changes, https://gitlab.com/osu-nrsg/nested-config/-/blob/master/CHANGELOG.md
+Project-URL: Repository, https://gitlab.com/osu-nrsg/nested-config
+Description-Content-Type: text/markdown
+
+# nested-config README
+
+**nested-config** provides for parsing configuration files that include paths to other
+config files into [Pydantic](https://github.com/samuelcolvin/pydantic/) model instances.
+It also supports validating and JSON-encoding `pathlib.PurePath` on Pydantic 1.8+.
+
+## Usage
+
+### Config loading
+
+**nested-config** may be used in your project in two main ways.
+
+1. You may simply call `nested_config.validate_config()` with a config file path and a
+   Pydantic model which may or may not include nested Pydantic models. If there are nested
+   models and the config file has string values for those fields, those values are
+   interpreted as paths to other config files and those are recursively read into their
+   respective Pydantic models using `validate_config()`. The `default_suffix` kwarg allows
+   for specifying the file suffix (extension) to assume if the config file has no suffix
+   or its suffix is not in the `nested_config.config_dict_loaders` dict.
+
+   Example including mixed configuration file types and `default_suffix` (Note that PyYAML
+   is an extra dependency required for parsing yaml files):
+
+   **house.yaml**
+
+   ```yaml
+   name: my house
+   dimensions: dimensions
+   ```
+
+   **dimensions** (TOML type)
+
+   ```toml
+   length = 10
+   width = 20
+   ```
+
+   **parse_house.py**
+
+   ```python
+   import pydantic
+   import yaml
+
+   from nested_config import validate_config
+
+   class Dimensions(pydantic.BaseModel):
+       length: int
+       width: int
+
+
+   class House(pydantic.BaseModel):
+       name: str
+       dimensions: Dimensions
+
+
+   house = validate_config("house.yaml", House, default_suffix=".toml")
+   house  # House(name='my house', dimensions=Dimensions(length=10, width=20))
+   ```
+
+2. Alternatively, you can use `nested_config.BaseModel` which subclasses
+   `pydantic.BaseModel` and adds a `from_config` classmethod:
+
+   **house.toml**
+
+   ```toml
+   name = "my house"
+   dimensions = "dimensions.toml"
+   ```
+
+   **dimensions.toml**
+
+   ```toml
+   length = 12.6
+   width = 25.3
+   ```
+
+   **parse_house.py**
+
+   ```python
+   import nested_config
+
+   class Dimensions(nested_config.BaseModel):
+       length: float
+       width: float
+
+
+   class House(nested_config.BaseModel):
+       name: str
+       dimensions: Dimensions
+
+
+   house = House.from_config("house.toml", House)
+   house  # House(name='my house', dimensions=Dimensions(length=12.6, width=25.3))
+   ```
+
+   In this case, if you need to specify a default loader, just use
+   `nested_config.set_default_loader(suffix)` before using `BaseModel.from_config()`.
+
+See [tests](https://gitlab.com/osu-nrsg/nested-config/-/tree/master/tests) for more
+detailed use-cases.
+
+### Included loaders
+
+**nested-config** automatically loads the following files based on extension:
+
+| Format | Extensions(s) | Library                                    |
+| ------ | ------------- | ------------------------------------------ |
+| JSON   | .json         | `json` (stdlib)                            |
+| TOML   | .toml         | `tomllib` (Python 3.11+ stdlib) or `tomli` |
+| YAML   | .yaml, .yml   | `pyyaml` (extra dependency[^yaml-extra])   |
+
+### Adding loaders
+
+To add a loader for another file extension, simply update the `config_dict_loaders` dict:
+
+```python
+import nested_config
+from nested_config import ConfigDict  # alias for dict[str, Any]
+
+def dummy_loader(config_path: Path) -> ConfigDict:
+    return {"a": 1, "b": 2}
+
+nested_config.config_dict_loaders[".dmy"] = dummy_loader
+
+# or add another extension for an existing loader
+nested_config.config_dict_loaders[".jsn"] = nested_config.config_dict_loaders[".json"]
+```
+
+### `PurePath` handling
+
+A bonus feature of **nested-config** is that it provides for validation and JSON encoding
+of `pathlib.PurePath` and its subclasses in Pydantic <2.0 (this is built into Pydantic
+2.0+). All that is needed is an import of `nested_config`. Example:
+
+```python
+from pathlib import PurePosixPath
+
+import nested_config
+import pydantic
+
+
+class RsyncDestination(pydantic.BaseModel):
+    remote_server: str
+    remote_path: PurePosixPath
+
+
+dest = RsyncDestination(remote_server="rsync.example.com", remote_path="/data/incoming")
+
+dest  # RsyncDestination(remote_server='rsync.example.com', remote_path=PurePosixPath('/data/incoming'))
+dest.json()  # '{"remote_server":"rsync.example.com","remote_path":"/data/incoming"}'
+
+```
+
+## Pydantic 1.0/2.0 Compatibility
+
+nested-config is runtime compatible with Pydantic 1.8+ and Pydantic 2.0.
+
+The follow table gives info on how to configure the [mypy](https://www.mypy-lang.org/) and
+[Pyright](https://microsoft.github.io/pyright) type checkers to properly work, depending
+on the version of Pydantic you are using.
+
+| Pydantic Version | [mypy config][1]            | mypy cli                    | [Pyright config][2]                         |
+|------------------|-----------------------------|-----------------------------|---------------------------------------------|
+| 2.0+             | `always_false = PYDANTIC_1` | `--always-false PYDANTIC_1` | `defineConstant = { "PYDANTIC_1" = false }` |
+| 1.8-1.10         | `always_true = PYDANTIC_1`  | `--always-true PYDANTIC_1`  | `defineConstant = { "PYDANTIC_1" = true }`  |
+
+## Footnotes
+
+[^yaml-extra]: Install `pyyaml` separately with `pip` or install **nested-config** with
+               `pip install nested-config[yaml]`.
+
+[1]: https://mypy.readthedocs.io/en/latest/config_file.html
+[2]: https://microsoft.github.io/pyright/#/configuration
+

nested_config-2.0.3/README.md

@@ -0,0 +1,173 @@
+# nested-config README
+
+**nested-config** provides for parsing configuration files that include paths to other
+config files into [Pydantic](https://github.com/samuelcolvin/pydantic/) model instances.
+It also supports validating and JSON-encoding `pathlib.PurePath` on Pydantic 1.8+.
+
+## Usage
+
+### Config loading
+
+**nested-config** may be used in your project in two main ways.
+
+1. You may simply call `nested_config.validate_config()` with a config file path and a
+   Pydantic model which may or may not include nested Pydantic models. If there are nested
+   models and the config file has string values for those fields, those values are
+   interpreted as paths to other config files and those are recursively read into their
+   respective Pydantic models using `validate_config()`. The `default_suffix` kwarg allows
+   for specifying the file suffix (extension) to assume if the config file has no suffix
+   or its suffix is not in the `nested_config.config_dict_loaders` dict.
+
+   Example including mixed configuration file types and `default_suffix` (Note that PyYAML
+   is an extra dependency required for parsing yaml files):
+
+   **house.yaml**
+
+   ```yaml
+   name: my house
+   dimensions: dimensions
+   ```
+
+   **dimensions** (TOML type)
+
+   ```toml
+   length = 10
+   width = 20
+   ```
+
+   **parse_house.py**
+
+   ```python
+   import pydantic
+   import yaml
+
+   from nested_config import validate_config
+
+   class Dimensions(pydantic.BaseModel):
+       length: int
+       width: int
+
+
+   class House(pydantic.BaseModel):
+       name: str
+       dimensions: Dimensions
+
+
+   house = validate_config("house.yaml", House, default_suffix=".toml")
+   house  # House(name='my house', dimensions=Dimensions(length=10, width=20))
+   ```
+
+2. Alternatively, you can use `nested_config.BaseModel` which subclasses
+   `pydantic.BaseModel` and adds a `from_config` classmethod:
+
+   **house.toml**
+
+   ```toml
+   name = "my house"
+   dimensions = "dimensions.toml"
+   ```
+
+   **dimensions.toml**
+
+   ```toml
+   length = 12.6
+   width = 25.3
+   ```
+
+   **parse_house.py**
+
+   ```python
+   import nested_config
+
+   class Dimensions(nested_config.BaseModel):
+       length: float
+       width: float
+
+
+   class House(nested_config.BaseModel):
+       name: str
+       dimensions: Dimensions
+
+
+   house = House.from_config("house.toml", House)
+   house  # House(name='my house', dimensions=Dimensions(length=12.6, width=25.3))
+   ```
+
+   In this case, if you need to specify a default loader, just use
+   `nested_config.set_default_loader(suffix)` before using `BaseModel.from_config()`.
+
+See [tests](https://gitlab.com/osu-nrsg/nested-config/-/tree/master/tests) for more
+detailed use-cases.
+
+### Included loaders
+
+**nested-config** automatically loads the following files based on extension:
+
+| Format | Extensions(s) | Library                                    |
+| ------ | ------------- | ------------------------------------------ |
+| JSON   | .json         | `json` (stdlib)                            |
+| TOML   | .toml         | `tomllib` (Python 3.11+ stdlib) or `tomli` |
+| YAML   | .yaml, .yml   | `pyyaml` (extra dependency[^yaml-extra])   |
+
+### Adding loaders
+
+To add a loader for another file extension, simply update the `config_dict_loaders` dict:
+
+```python
+import nested_config
+from nested_config import ConfigDict  # alias for dict[str, Any]
+
+def dummy_loader(config_path: Path) -> ConfigDict:
+    return {"a": 1, "b": 2}
+
+nested_config.config_dict_loaders[".dmy"] = dummy_loader
+
+# or add another extension for an existing loader
+nested_config.config_dict_loaders[".jsn"] = nested_config.config_dict_loaders[".json"]
+```
+
+### `PurePath` handling
+
+A bonus feature of **nested-config** is that it provides for validation and JSON encoding
+of `pathlib.PurePath` and its subclasses in Pydantic <2.0 (this is built into Pydantic
+2.0+). All that is needed is an import of `nested_config`. Example:
+
+```python
+from pathlib import PurePosixPath
+
+import nested_config
+import pydantic
+
+
+class RsyncDestination(pydantic.BaseModel):
+    remote_server: str
+    remote_path: PurePosixPath
+
+
+dest = RsyncDestination(remote_server="rsync.example.com", remote_path="/data/incoming")
+
+dest  # RsyncDestination(remote_server='rsync.example.com', remote_path=PurePosixPath('/data/incoming'))
+dest.json()  # '{"remote_server":"rsync.example.com","remote_path":"/data/incoming"}'
+
+```
+
+## Pydantic 1.0/2.0 Compatibility
+
+nested-config is runtime compatible with Pydantic 1.8+ and Pydantic 2.0.
+
+The follow table gives info on how to configure the [mypy](https://www.mypy-lang.org/) and
+[Pyright](https://microsoft.github.io/pyright) type checkers to properly work, depending
+on the version of Pydantic you are using.
+
+| Pydantic Version | [mypy config][1]            | mypy cli                    | [Pyright config][2]                         |
+|------------------|-----------------------------|-----------------------------|---------------------------------------------|
+| 2.0+             | `always_false = PYDANTIC_1` | `--always-false PYDANTIC_1` | `defineConstant = { "PYDANTIC_1" = false }` |
+| 1.8-1.10         | `always_true = PYDANTIC_1`  | `--always-true PYDANTIC_1`  | `defineConstant = { "PYDANTIC_1" = true }`  |
+
+## Footnotes
+
+[^yaml-extra]: Install `pyyaml` separately with `pip` or install **nested-config** with
+               `pip install nested-config[yaml]`.
+
+[1]: https://mypy.readthedocs.io/en/latest/config_file.html
+[2]: https://microsoft.github.io/pyright/#/configuration

nested_config-2.0.3/pyproject.toml

@@ -0,0 +1,58 @@
+[tool.poetry]
+name = "nested-config"
+version = "2.0.3"
+description = """\
+Parse configuration files that include paths to other config files into Pydantic model\
+ instances. Also support pathlib.PurePath on Pydantic 1.8+.\
+"""
+authors = ["Randall Pittman <pittmara@oregonstate.edu>"]
+readme = "README.md"
+license = "MIT"
+repository = "https://gitlab.com/osu-nrsg/nested-config"
+keywords = ["pydantic", "config", "configuration files"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Framework :: Pydantic",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+]
+include = ["CHANGELOG.md"]
+
+[tool.poetry.urls]
+Changes = "https://gitlab.com/osu-nrsg/nested-config/-/blob/master/CHANGELOG.md"
+
+[tool.poetry.dependencies]
+python = "^3.8"
+pydantic = ">=1.8,<3.0.0"
+single-version = "^1.6.0"
+tomli = {version = "^2.0.0", python = "<3.11"}
+typing-extensions = "^4.6.0"
+pyyaml = {version = ">=5.1.0,<7.0.0", optional = true}
+
+[tool.poetry.group.dev.dependencies]
+ipython = "^7.22.0"
+ruff = "^0.3.5"
+pytest = "^8.1.1"
+mypy = "^1.9.0"
+types-pyyaml = ">=5.1.0"
+
+[tool.poetry.extras]
+yaml = ["pyyaml"]
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.ruff]
+line-length = 90
+
+[tool.ruff.lint.extend-per-file-ignores]
+"__init__.py" = ["F401"]
+
+[tool.pyright]
+defineConstant = { "PYDANTIC_1" = false }  # change to true for Pydantic <2.0

nested_config-2.0.3/setup.py

@@ -0,0 +1,41 @@
+# -*- coding: utf-8 -*-
+from setuptools import setup
+
+package_dir = \
+{'': 'src'}
+
+packages = \
+['nested_config']
+
+package_data = \
+{'': ['*']}
+
+install_requires = \
+['pydantic>=1.8,<3.0.0',
+ 'single-version>=1.6.0,<2.0.0',
+ 'typing-extensions>=4.6.0,<5.0.0']
+
+extras_require = \
+{':python_version < "3.11"': ['tomli>=2.0.0,<3.0.0'],
+ 'yaml': ['pyyaml>=5.1.0,<7.0.0']}
+
+setup_kwargs = {
+    'name': 'nested-config',
+    'version': '2.0.3',
+    'description': 'Parse configuration files that include paths to other config files into Pydantic modelinstances. Also support pathlib.PurePath on Pydantic 1.8+.',
+    'long_description': '# nested-config README\n\n**nested-config** provides for parsing configuration files that include paths to other\nconfig files into [Pydantic](https://github.com/samuelcolvin/pydantic/) model instances.\nIt also supports validating and JSON-encoding `pathlib.PurePath` on Pydantic 1.8+.\n\n## Usage\n\n### Config loading\n\n**nested-config** may be used in your project in two main ways.\n\n1. You may simply call `nested_config.validate_config()` with a config file path and a\n   Pydantic model which may or may not include nested Pydantic models. If there are nested\n   models and the config file has string values for those fields, those values are\n   interpreted as paths to other config files and those are recursively read into their\n   respective Pydantic models using `validate_config()`. The `default_suffix` kwarg allows\n   for specifying the file suffix (extension) to assume if the config file has no suffix\n   or its suffix is not in the `nested_config.config_dict_loaders` dict.\n\n   Example including mixed configuration file types and `default_suffix` (Note that PyYAML\n   is an extra dependency required for parsing yaml files):\n\n   **house.yaml**\n\n   ```yaml\n   name: my house\n   dimensions: dimensions\n   ```\n\n   **dimensions** (TOML type)\n\n   ```toml\n   length = 10\n   width = 20\n   ```\n\n   **parse_house.py**\n\n   ```python\n   import pydantic\n   import yaml\n\n   from nested_config import validate_config\n\n   class Dimensions(pydantic.BaseModel):\n       length: int\n       width: int\n\n\n   class House(pydantic.BaseModel):\n       name: str\n       dimensions: Dimensions\n\n\n   house = validate_config("house.yaml", House, default_suffix=".toml")\n   house  # House(name=\'my house\', dimensions=Dimensions(length=10, width=20))\n   ```\n\n2. Alternatively, you can use `nested_config.BaseModel` which subclasses\n   `pydantic.BaseModel` and adds a `from_config` classmethod:\n\n   **house.toml**\n\n   ```toml\n   name = "my house"\n   dimensions = "dimensions.toml"\n   ```\n\n   **dimensions.toml**\n\n   ```toml\n   length = 12.6\n   width = 25.3\n   ```\n\n   **parse_house.py**\n\n   ```python\n   import nested_config\n\n   class Dimensions(nested_config.BaseModel):\n       length: float\n       width: float\n\n\n   class House(nested_config.BaseModel):\n       name: str\n       dimensions: Dimensions\n\n\n   house = House.from_config("house.toml", House)\n   house  # House(name=\'my house\', dimensions=Dimensions(length=12.6, width=25.3))\n   ```\n\n   In this case, if you need to specify a default loader, just use\n   `nested_config.set_default_loader(suffix)` before using `BaseModel.from_config()`.\n\nSee [tests](https://gitlab.com/osu-nrsg/nested-config/-/tree/master/tests) for more\ndetailed use-cases.\n\n### Included loaders\n\n**nested-config** automatically loads the following files based on extension:\n\n| Format | Extensions(s) | Library                                    |\n| ------ | ------------- | ------------------------------------------ |\n| JSON   | .json         | `json` (stdlib)                            |\n| TOML   | .toml         | `tomllib` (Python 3.11+ stdlib) or `tomli` |\n| YAML   | .yaml, .yml   | `pyyaml` (extra dependency[^yaml-extra])   |\n\n### Adding loaders\n\nTo add a loader for another file extension, simply update the `config_dict_loaders` dict:\n\n```python\nimport nested_config\nfrom nested_config import ConfigDict  # alias for dict[str, Any]\n\ndef dummy_loader(config_path: Path) -> ConfigDict:\n    return {"a": 1, "b": 2}\n\nnested_config.config_dict_loaders[".dmy"] = dummy_loader\n\n# or add another extension for an existing loader\nnested_config.config_dict_loaders[".jsn"] = nested_config.config_dict_loaders[".json"]\n```\n\n### `PurePath` handling\n\nA bonus feature of **nested-config** is that it provides for validation and JSON encoding\nof `pathlib.PurePath` and its subclasses in Pydantic <2.0 (this is built into Pydantic\n2.0+). All that is needed is an import of `nested_config`. Example:\n\n```python\nfrom pathlib import PurePosixPath\n\nimport nested_config\nimport pydantic\n\n\nclass RsyncDestination(pydantic.BaseModel):\n    remote_server: str\n    remote_path: PurePosixPath\n\n\ndest = RsyncDestination(remote_server="rsync.example.com", remote_path="/data/incoming")\n\ndest  # RsyncDestination(remote_server=\'rsync.example.com\', remote_path=PurePosixPath(\'/data/incoming\'))\ndest.json()  # \'{"remote_server":"rsync.example.com","remote_path":"/data/incoming"}\'\n\n```\n\n## Pydantic 1.0/2.0 Compatibility\n\nnested-config is runtime compatible with Pydantic 1.8+ and Pydantic 2.0.\n\nThe follow table gives info on how to configure the [mypy](https://www.mypy-lang.org/) and\n[Pyright](https://microsoft.github.io/pyright) type checkers to properly work, depending\non the version of Pydantic you are using.\n\n| Pydantic Version | [mypy config][1]            | mypy cli                    | [Pyright config][2]                         |\n|------------------|-----------------------------|-----------------------------|---------------------------------------------|\n| 2.0+             | `always_false = PYDANTIC_1` | `--always-false PYDANTIC_1` | `defineConstant = { "PYDANTIC_1" = false }` |\n| 1.8-1.10         | `always_true = PYDANTIC_1`  | `--always-true PYDANTIC_1`  | `defineConstant = { "PYDANTIC_1" = true }`  |\n\n## Footnotes\n\n[^yaml-extra]: Install `pyyaml` separately with `pip` or install **nested-config** with\n               `pip install nested-config[yaml]`.\n\n[1]: https://mypy.readthedocs.io/en/latest/config_file.html\n[2]: https://microsoft.github.io/pyright/#/configuration\n',
+    'author': 'Randall Pittman',
+    'author_email': 'pittmara@oregonstate.edu',
+    'maintainer': 'None',
+    'maintainer_email': 'None',
+    'url': 'https://gitlab.com/osu-nrsg/nested-config',
+    'package_dir': package_dir,
+    'packages': packages,
+    'package_data': package_data,
+    'install_requires': install_requires,
+    'extras_require': extras_require,
+    'python_requires': '>=3.8,<4.0',
+}
+
+
+setup(**setup_kwargs)

nested_config-2.0.3/src/nested_config/__init__.py

@@ -0,0 +1,31 @@
+"""nested_config - This package does two things:
+
+1. It adds the ability to parse config files into Pydantic model instances, including
+   config files that include string path references to other config files in place of
+   sub-model instances.
+
+       my_obj = validate_config("my_config.toml", MyConfigModel, loader=toml.load)
+
+2. It adds PurePath, PurePosixPath, and PureWindowsPath validation and JSON-encoding to
+   Pydantic v1 (these are already included in Pydantic 2.)
+"""
+
+from nested_config._validators import (
+    patch_pydantic_validators as _patch_pydantic_validators,
+)
+from nested_config.base_model import BaseModel
+from nested_config.json import (
+    patch_pydantic_json_encoders as _patch_pydantic_json_encoders,
+)
+from nested_config.loaders import (
+    ConfigLoaderError,
+    NoLoaderError,
+    config_dict_loaders,
+    set_default_loader,
+)
+from nested_config.parsing import ispydmodel, validate_config
+from nested_config.version import __version__
+
+# We always patch the validators, but in the future this may be made optional
+_patch_pydantic_validators()
+_patch_pydantic_json_encoders()

nested_config-2.0.3/src/nested_config/_compat.py

@@ -0,0 +1,61 @@
+"""_compat.py - Functions and types to assist with Pydantic 1/2 compatibility"""
+
+from typing import Any, Dict, Optional, Protocol, Type
+
+import pydantic
+import pydantic.fields
+from setuptools._vendor.packaging.version import Version  # type: ignore
+from typing_extensions import TypeAlias
+
+from nested_config._types import PydModelT
+
+PYDANTIC_1 = Version(pydantic.VERSION) < Version("2.0")
+
+
+class HasAnnotation(Protocol):
+    """Protocol will allow some Pydantic 2.0 compatibility down the road"""
+
+    annotation: Optional[Type[Any]]
+
+
+if PYDANTIC_1:
+    FieldInfo_: TypeAlias = pydantic.fields.ModelField
+else:
+    FieldInfo_: TypeAlias = pydantic.fields.FieldInfo
+ModelFields: TypeAlias = Dict[str, FieldInfo_]
+
+
+def get_modelfield_annotation(model: Type[pydantic.BaseModel], field_name: str):
+    # "annotation" exists in pydantic 1.10, but not 1.8 or 1.9
+    field = get_model_fields(model)[field_name]
+    return get_field_annotation(field)
+
+
+def get_field_annotation(field: FieldInfo_):
+    # "annotation" exists in pydantic 1.10, but not 1.8 or 1.9
+    if PYDANTIC_1:
+        return field.outer_type_
+    else:
+        return field.annotation
+
+
+def get_model_fields(model: Type[pydantic.BaseModel]) -> ModelFields:
+    if PYDANTIC_1:
+        return model.__fields__
+    else:
+        return model.model_fields
+
+
+def parse_obj(model: Type[PydModelT], obj: Any) -> PydModelT:
+    if PYDANTIC_1:
+        return model.parse_obj(obj)
+    else:
+        return model.model_validate(obj)
+
+
+def dump_json(model: pydantic.BaseModel) -> str:
+    """Compatibility of json dump function for testing"""
+    if PYDANTIC_1:
+        return model.json()
+    else:
+        return model.model_dump_json()

nested_config-2.0.3/src/nested_config/_types.py

@@ -0,0 +1,26 @@
+"""_types.py - Type aliases and type-checking functions"""
+
+import sys
+from pathlib import Path
+from typing import Any, Callable, Dict, Type, TypeVar, Union
+
+import pydantic
+from typing_extensions import TypeAlias, TypeGuard
+
+ConfigDict: TypeAlias = Dict[str, Any]
+PathLike: TypeAlias = Union[Path, str]
+PydModelT = TypeVar("PydModelT", bound=pydantic.BaseModel)
+ConfigDictLoader: TypeAlias = Callable[[Path], ConfigDict]
+
+
+if sys.version_info >= (3, 10):
+    from types import UnionType
+
+    UNION_TYPES = [Union, UnionType]
+else:
+    UNION_TYPES = [Union]
+
+
+def ispydmodel(klass, cls: Type[PydModelT]) -> TypeGuard[Type[PydModelT]]:
+    """Exception-safe issubclass for pydantic BaseModel types"""
+    return isinstance(klass, type) and issubclass(klass, cls)

nested_config-2.0.3/src/nested_config/_validators.py

@@ -0,0 +1,51 @@
+"""_validators.py - For Pydantic 1.8-1.10, extends the built-in validators to include
+PurePath & subclasses"""
+
+from pathlib import PurePath, PurePosixPath, PureWindowsPath
+from typing import Any, Type, TypeVar
+
+import pydantic
+import pydantic.errors
+import pydantic.validators
+
+from nested_config._compat import PYDANTIC_1
+
+PathT = TypeVar("PathT", bound=PurePath)
+
+
+def _path_validator(v: Any, type: Type[PathT]) -> PathT:
+    """Attempt to convert a value to a PurePosixPath"""
+    if isinstance(v, type):
+        return v
+    try:
+        return type(v)
+    except TypeError:
+        # n.b. this error only exists in Pydantic < 2.0
+        raise pydantic.errors.PathError from None
+
+
+def pure_path_validator(v: Any):
+    return _path_validator(v, type=PurePath)
+
+
+def pure_posix_path_validator(v: Any):
+    return _path_validator(v, type=PurePosixPath)
+
+
+def pure_windows_path_validator(v: Any):
+    return _path_validator(v, type=PureWindowsPath)
+
+
+def patch_pydantic_validators():
+    if PYDANTIC_1:
+        # These are already included in pydantic 2+
+        pydantic.validators._VALIDATORS.extend(
+            [
+                (PurePosixPath, [pure_posix_path_validator]),
+                (PureWindowsPath, [pure_windows_path_validator]),
+                (
+                    PurePath,
+                    [pure_path_validator],
+                ),  # last because others are more specific
+            ]
+        )

nested_config-2.0.3/src/nested_config/base_model.py

@@ -0,0 +1,58 @@
+"""base_model.py
+
+Pydantic BaseModel extended a bit:
+  - PurePosixPath json encoding and validation
+  - from_toml and from_tomls classmethods
+"""
+
+from pathlib import Path
+from typing import Type
+
+import pydantic
+
+from nested_config import parsing
+from nested_config._compat import parse_obj
+from nested_config._types import PathLike, PydModelT
+from nested_config.loaders import load_config
+
+
+class BaseModel(pydantic.BaseModel):
+    """Extends pydantic.BaseModel with from_config classmethod to load a config file into
+    the model."""
+
+    @classmethod
+    def from_config(
+        cls: Type[PydModelT], toml_path: PathLike, convert_strpaths=True
+    ) -> PydModelT:
+        """Create Pydantic model from a TOML file
+
+        Parameters
+        ----------
+        toml_path
+            Path to the TOML file
+        convert_strpaths
+            If True, every string value [a] in the dict from the parsed TOML file that
+            corresponds to a Pydantic model field [b] in the base model will be
+            interpreted as a path to another TOML file and an attempt will be made to
+            parse that TOML file [a] and make it into an object of that [b] model type,
+            and so on, recursively.
+
+        Returns
+        -------
+        An object of this class
+
+        Raises
+        -------
+        NoLoaderError
+            No loader is available for the config file extension
+        ConfigLoaderError
+            There was a problem loading a config file with its loader
+        pydantic.ValidationError
+            The data fields or types in the file do not match the model.
+        """
+        toml_path = Path(toml_path)
+        if convert_strpaths:
+            return parsing.validate_config(toml_path, cls)
+        # otherwise just load the config as-is
+        config_dict = load_config(toml_path)
+        return parse_obj(cls, config_dict)

nested_config-2.0.3/src/nested_config/json.py

@@ -0,0 +1,13 @@
+"""json.py - For PYDANTIC_1, adds a json encoder for PurePath objects"""
+
+from pathlib import PurePath
+
+import pydantic.json
+
+from nested_config._compat import PYDANTIC_1
+
+
+def patch_pydantic_json_encoders():
+    if PYDANTIC_1:
+        # These are already in pydantic 2+
+        pydantic.json.ENCODERS_BY_TYPE[PurePath] = str

nested_config-2.0.3/src/nested_config/loaders.py

@@ -0,0 +1,115 @@
+"""loaders.py - Manage config file loaders"""
+
+import contextlib
+import json
+import sys
+from pathlib import Path
+from typing import Dict, Optional
+
+if sys.version_info < (3, 11):
+    from tomli import load as toml_load_fobj
+else:
+    from tomllib import load as toml_load_fobj
+
+from nested_config._types import ConfigDict, ConfigDictLoader, PathLike
+
+YAML_INSTALLED = False
+with contextlib.suppress(ImportError):
+    import yaml  # type: ignore
+
+    YAML_INSTALLED = True
+
+
+class NoLoaderError(Exception):
+    def __init__(self, suffix: str):
+        super().__init__(f"There is no loader for file extension {suffix}")
+
+
+class ConfigLoaderError(Exception):
+    def __init__(self, config_path: Path) -> None:
+        super().__init__(f"There was a problem loading config file {config_path}")
+
+
+def toml_load(path: PathLike) -> ConfigDict:
+    """Load a TOML config file"""
+    with open(path, "rb") as fobj:
+        return toml_load_fobj(fobj)
+
+
+def json_load(path: PathLike) -> ConfigDict:
+    """Load a JSON config file"""
+    with open(path, "rb") as fobj:
+        return json.load(fobj)
+
+
+config_dict_loaders: Dict[str, ConfigDictLoader] = {
+    ".toml": toml_load,
+    ".json": json_load,
+}
+"""Mapping of config file extension to config file loader"""
+
+
+if YAML_INSTALLED:
+
+    def yaml_load(path: PathLike) -> ConfigDict:
+        """Load a YAML config file (safely)"""
+        with open(path, "r") as fobj:
+            return yaml.safe_load(fobj)
+
+    config_dict_loaders[".yaml"] = yaml_load
+    config_dict_loaders[".yml"] = yaml_load
+
+
+def _get_loader(config_path: Path):
+    """Get the loader for the specified suffix, or a loader from default suffix"""
+    try:
+        try:
+            return config_dict_loaders[config_path.suffix]
+        except KeyError:
+            if default_loader := _get_default_loader():
+                return default_loader
+            raise
+    except KeyError:
+        raise NoLoaderError(config_path.suffix) from None
+
+
+def _get_default_loader() -> Optional[ConfigDictLoader]:
+    try:
+        return config_dict_loaders["DEFAULT"]
+    except KeyError:
+        return None
+
+
+def set_default_loader(default_suffix: str):
+    try:
+        config_dict_loaders["DEFAULT"] = config_dict_loaders[default_suffix]
+    except KeyError:
+        raise NoLoaderError(default_suffix) from None
+
+
+def load_config(config_path: Path) -> ConfigDict:
+    """Select a loader based on the suffix (extension) of the config file and try to load
+    the config using that loader. E.g. for .toml, use the TOML loader.
+
+    Inputs
+    ------
+    config_path
+        Path to the config file
+
+    Returns
+    -------
+    ConfigDict
+        A mapping of the data stored in the config file
+
+    Raises
+    ------
+    NoLoaderError (via _get_loader)
+        No loader could be found for the suffix (or default suffix, if provided)
+    ConfigLoaderError
+        There was an error running the loader (e.g. in tomllib or yaml or json)
+    """
+    loader = _get_loader(config_path)
+    try:
+        return loader(config_path)
+    except Exception as ex:
+        raise ConfigLoaderError(config_path) from ex

nested_config-2.0.3/src/nested_config/parsing.py

@@ -0,0 +1,167 @@
+"""parsing.py - Functions to parse config files (e.g. TOML) into Pydantic model instances,
+possibly with nested models specified by string paths."""
+
+import typing
+from pathlib import Path
+from typing import Optional, Type
+
+import pydantic
+
+from nested_config import _compat
+from nested_config._types import (
+    UNION_TYPES,
+    ConfigDict,
+    PathLike,
+    PydModelT,
+    ispydmodel,
+)
+from nested_config.loaders import load_config, set_default_loader
+
+
+def validate_config(
+    config_path: PathLike,
+    model: Type[PydModelT],
+    *,
+    default_suffix: Optional[str] = None,
+) -> PydModelT:
+    """Load a config file into a Pydantic model. The config file may contain string paths
+    where nested models would be expected. These are preparsed into their respective
+    models.
+
+    If paths to nested models are relative, they are assumed to be relative to the path of
+    their parent config file.
+
+    Input
+    -----
+    config_path
+        A string or pathlib.Path to the config file to parse
+    model
+        The Pydantic model to use for creating the config object
+    default_suffix
+        If there is no loader for the config file suffix (or the config file has no
+        suffix) try to load the config with the loader specified by this extension, e.g.
+        '.toml' or '.yml'
+    Returns
+    -------
+    A Pydantic object of the type specified by the model input.
+
+    Raises
+    ------
+    NoLoaderError
+        No loader is available for the config file extension
+    ConfigLoaderError
+        There was a problem loading a config file with its loader
+    pydantic.ValidationError
+        The data fields or types in the file do not match the model.
+
+    """
+    if default_suffix:
+        set_default_loader(default_suffix)
+    # Input arg coercion
+    config_path = Path(config_path)
+    # Get the config dict and the model fields
+    config_dict = load_config(config_path)
+    # preparse the config (possibly loading nested configs)
+    config_dict = _preparse_config_dict(config_dict, model, config_path)
+    # Create and validate the config object
+    return _compat.parse_obj(model, config_dict)
+
+
+def _preparse_config_dict(
+    config_dict: ConfigDict, model: Type[pydantic.BaseModel], config_path: Path
+):
+    return {
+        key: _preparse_config_value(
+            value, _compat.get_modelfield_annotation(model, key), config_path
+        )
+        for key, value in config_dict.items()
+    }
+
+
+def _preparse_config_value(field_value, field_annotation, config_path: Path):
+    """Check if a model field contains a path to another model and parse it accordingly"""
+    # If the annotation is optional, get the enclosed annotation
+    field_annotation = _get_optional_ann(field_annotation)
+    # ###
+    # N cases:
+    # 1. Config value is not a string, list, or dict
+    # 2. Config value is a dict, model expects a model
+    # 3. Config value is a string, model expects a model
+    # 4. Config value is a list, model expects a list of some type
+    # 5. Config value is a dict, model expects a dict with values of a particular model
+    #    type
+    # 6. A string, list, or dict that doesn't match cases 2-5
+    # ###
+
+    # 1.
+    if not isinstance(field_value, (str, list, dict)):
+        return field_value
+    # 2.
+    if isinstance(field_value, dict) and ispydmodel(field_annotation, pydantic.BaseModel):
+        return _preparse_config_dict(field_value, field_annotation, config_path)
+    # 3.
+    if isinstance(field_value, str) and ispydmodel(field_annotation, pydantic.BaseModel):
+        return _parse_path_str_into_pydmodel(field_value, field_annotation, config_path)
+    # 4.
+    if isinstance(field_value, list) and (
+        listval_annotation := _get_list_value_ann(field_annotation)
+    ):
+        return [
+            _preparse_config_value(li, listval_annotation, config_path)
+            for li in field_value
+        ]
+    # 5.
+    if isinstance(field_value, dict) and (
+        dictval_annotation := _get_dict_value_ann(field_annotation)
+    ):
+        return {
+            key: _preparse_config_value(value, dictval_annotation, config_path)
+            for key, value in field_value.items()
+        }
+    # 6.
+    return field_value
+
+
+def _parse_path_str_into_pydmodel(
+    path_str: str, model: Type[PydModelT], parent_path: Path
+) -> PydModelT:
+    """Convert a path string to a path (possibly relative to a parent config path) and
+    create an instance of a Pydantic model"""
+    path = Path(path_str)
+    if not path.is_absolute():
+        # Assume it's relative to the parent config path
+        path = parent_path.parent / path
+    if not path.is_file():
+        raise FileNotFoundError(
+            f"Config file '{parent_path}' contains a path to another config file"
+            f" '{path_str}' that could not be found."
+        )
+    return validate_config(path, model)
+
+
+def _get_optional_ann(annotation):
+    """Convert a possibly Optional annotation to its underlying annotation"""
+    annotation_origin = typing.get_origin(annotation)
+    annotation_args = typing.get_args(annotation)
+    if annotation_origin in UNION_TYPES and annotation_args[1] is type(None):
+        return annotation_args[0]
+    return annotation
+
+
+def _get_list_value_ann(annotation):
+    """Get the internal annotation of a typed list, if any. Otherwise return None."""
+    annotation_origin = typing.get_origin(annotation)
+    annotation_args = typing.get_args(annotation)
+    if annotation_origin is list and len(annotation_args) > 0:
+        return annotation_args[0]
+    return None
+
+
+def _get_dict_value_ann(annotation):
+    """Get the internal annotation of a dict's value type, if any. Otherwise return
+    None."""
+    annotation_origin = typing.get_origin(annotation)
+    annotation_args = typing.get_args(annotation)
+    if annotation_origin is dict and len(annotation_args) > 1:
+        return annotation_args[1]
+    return None

nested_config-2.0.3/src/nested_config/version.py

@@ -0,0 +1,10 @@
+"""version.py - Get __version__ from Poetry-generated pyproject.toml or installed
+package version"""
+
+from pathlib import Path
+
+import single_version  # type: ignore
+
+__version__ = single_version.get_version(
+    __name__.split(".")[0], Path(__file__).parent.parent
+)