granular-configuration-language 2.0.0__tar.gz

granular_configuration_language-2.0.0/PKG-INFO

@@ -0,0 +1,131 @@
+Metadata-Version: 2.3
+Name: granular-configuration-language
+Version: 2.0.0
+Summary: This configuration utility library allows your code use YAML as a configuration language for internal and external parties, allowing configuration to be crafted from multiple sources and merged just before use, with added YAML Tags that run lazily for added functionality.
+License: MIT
+Author: Eric Jensen
+Author-email: eric.jensen42@gmail.com
+Requires-Python: >=3.10
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Natural Language :: English
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Utilities
+Classifier: Typing :: Typed
+Requires-Dist: python-dateutil ; python_version < "3.11"
+Requires-Dist: python-jsonpath
+Requires-Dist: ruamel.yaml (>=0.18)
+Project-URL: Documentation, https://lifedox.github.io/granular-configuration-language/README.html
+Project-URL: Homepage, https://lifedox.github.io/granular-configuration-language/README.html
+Project-URL: Repository, https://github.com/lifedox/granular-configuration-language
+Description-Content-Type: text/markdown
+
+# `granular-configuration-language`
+
+[![Coverage badge](https://raw.githubusercontent.com/lifedox/granular-configuration-language/python-coverage-comment-action-data/badge.svg)](https://github.com/lifedox/granular-configuration-language/tree/python-coverage-comment-action-data) ![Testing workflow](https://github.com/lifedox/granular-configuration-language/actions/workflows/testing.yaml/badge.svg?event=push) ![codeql workflow](https://github.com/lifedox/granular-configuration-language/actions/workflows/codeql-analysis.yaml/badge.svg?event=push)
+
+> ⚠️ **This library is meant for trusted configuration files.** ⚠️
+
+Get started or read more at the [documentation site](https://lifedox.github.io/granular-configuration-language/doc-spec/getting_started.html).
+
+## Why does this exist?
+
+This library exists to allow your code use YAML as a configuration language for internal and external parties, allowing configuration to be crafted from multiple sources and merged just before use, with added [YAML Tags](https://lifedox.github.io/granular-configuration-language/doc-spec/yaml.html) that run lazily for added functionality, and plugin support for creating custom YAML Tags.
+
+Some use cases:
+
+- You are writing a library to help connect to some databases. You want users to easily changes settings and defined databases by name.
+  - Conceptual Example:
+    - Library Code:
+      ```python
+      CONFIG = LazyLoadConfiguration(
+          Path(__file___).parent / "config.yaml",
+          "./database-util-config.yaml",
+          "~/configs/database-util-config.yaml",
+          base_path="database-util",
+          env_location_var_name="ORG_COMMON_CONFIG_LOCATIONS",
+      )
+      ```
+    - Library configuration:
+      ```yaml
+      database-util:
+        common_settings:
+          use_decimal: true
+          encryption_type: secure
+        databases: {} # Empty mapping, for users define
+      ```
+    - User application configuration:
+      ```yaml
+      database-util:
+        common_settings:
+          use_decimal: false
+        databases:
+          datebase1:
+            location: http://somewhere
+            user: !Mask ${DB_USERNAME}
+            password: !Mask ${DB_PASSWORD}
+      ```
+- You are deploying an application that has multiple deployment types with specific settings.
+  - Conceptual Example:
+    - Library Code:
+      ```python
+      CONFIG = LazyLoadConfiguration(
+          Path(__file___).parent / "config.yaml",
+          "./database-util-config.yaml",
+          base_path="app",
+      )
+      ```
+    - Base configuration:
+      ```yaml
+      app:
+        log_as: really cool app name
+        log_to: nowhere
+      ```
+    - AWS Lambda deploy:
+      ```yaml
+      app:
+        log_to: std_out
+      ```
+    - Server deploy:
+      ```yaml
+      app:
+        log_to: !Sub file://var/log/${$.app.log_as}.log
+      ```
+- You are writing a [`pytest`](https://docs.pytest.org/en/stable/) plugin that create test data using named fixtures configured by the user.
+  - Conceptual Examples:
+    - Library Code:
+      ```python
+      CONFIG = LazyLoadConfiguration(
+          Path(__file___).parent / "fixture_config.yaml",
+          *Path().rglob("fixture_config.yaml"),
+          base_path="fixture-gen",
+      ).config
+      #
+      for name, spec in CONFIG.fixtures:
+          generate_fixture(name, spec)
+      ```
+    - Library configuration:
+      ```yaml
+      fixture-gen:
+        fixtures: {} # Empty mapping, for users define
+      ```
+    - User application configuration:
+      ```yaml
+      fixture-gen:
+        fixtures:
+          fixture1:
+            api: does something
+      ```
+
+## Why the long name?
+
+- It's "granular" because you can specify settings across multiple files at a fine granularity for overriding values.
+- It is meant for trusted "configuration" files.
+- Including "language" make it clear that this is not the source of configuration.
+  - A valid piece of feedback was that it sounded like it was the source for configuration, not the processing of generic configuration files.
+  - "Format" sounded weirder than "language".
+

granular_configuration_language-2.0.0/README.md

@@ -0,0 +1,104 @@
+# `granular-configuration-language`
+
+[![Coverage badge](https://raw.githubusercontent.com/lifedox/granular-configuration-language/python-coverage-comment-action-data/badge.svg)](https://github.com/lifedox/granular-configuration-language/tree/python-coverage-comment-action-data) ![Testing workflow](https://github.com/lifedox/granular-configuration-language/actions/workflows/testing.yaml/badge.svg?event=push) ![codeql workflow](https://github.com/lifedox/granular-configuration-language/actions/workflows/codeql-analysis.yaml/badge.svg?event=push)
+
+> ⚠️ **This library is meant for trusted configuration files.** ⚠️
+
+Get started or read more at the [documentation site](https://lifedox.github.io/granular-configuration-language/doc-spec/getting_started.html).
+
+## Why does this exist?
+
+This library exists to allow your code use YAML as a configuration language for internal and external parties, allowing configuration to be crafted from multiple sources and merged just before use, with added [YAML Tags](https://lifedox.github.io/granular-configuration-language/doc-spec/yaml.html) that run lazily for added functionality, and plugin support for creating custom YAML Tags.
+
+Some use cases:
+
+- You are writing a library to help connect to some databases. You want users to easily changes settings and defined databases by name.
+  - Conceptual Example:
+    - Library Code:
+      ```python
+      CONFIG = LazyLoadConfiguration(
+          Path(__file___).parent / "config.yaml",
+          "./database-util-config.yaml",
+          "~/configs/database-util-config.yaml",
+          base_path="database-util",
+          env_location_var_name="ORG_COMMON_CONFIG_LOCATIONS",
+      )
+      ```
+    - Library configuration:
+      ```yaml
+      database-util:
+        common_settings:
+          use_decimal: true
+          encryption_type: secure
+        databases: {} # Empty mapping, for users define
+      ```
+    - User application configuration:
+      ```yaml
+      database-util:
+        common_settings:
+          use_decimal: false
+        databases:
+          datebase1:
+            location: http://somewhere
+            user: !Mask ${DB_USERNAME}
+            password: !Mask ${DB_PASSWORD}
+      ```
+- You are deploying an application that has multiple deployment types with specific settings.
+  - Conceptual Example:
+    - Library Code:
+      ```python
+      CONFIG = LazyLoadConfiguration(
+          Path(__file___).parent / "config.yaml",
+          "./database-util-config.yaml",
+          base_path="app",
+      )
+      ```
+    - Base configuration:
+      ```yaml
+      app:
+        log_as: really cool app name
+        log_to: nowhere
+      ```
+    - AWS Lambda deploy:
+      ```yaml
+      app:
+        log_to: std_out
+      ```
+    - Server deploy:
+      ```yaml
+      app:
+        log_to: !Sub file://var/log/${$.app.log_as}.log
+      ```
+- You are writing a [`pytest`](https://docs.pytest.org/en/stable/) plugin that create test data using named fixtures configured by the user.
+  - Conceptual Examples:
+    - Library Code:
+      ```python
+      CONFIG = LazyLoadConfiguration(
+          Path(__file___).parent / "fixture_config.yaml",
+          *Path().rglob("fixture_config.yaml"),
+          base_path="fixture-gen",
+      ).config
+      #
+      for name, spec in CONFIG.fixtures:
+          generate_fixture(name, spec)
+      ```
+    - Library configuration:
+      ```yaml
+      fixture-gen:
+        fixtures: {} # Empty mapping, for users define
+      ```
+    - User application configuration:
+      ```yaml
+      fixture-gen:
+        fixtures:
+          fixture1:
+            api: does something
+      ```
+
+## Why the long name?
+
+- It's "granular" because you can specify settings across multiple files at a fine granularity for overriding values.
+- It is meant for trusted "configuration" files.
+- Including "language" make it clear that this is not the source of configuration.
+  - A valid piece of feedback was that it sounded like it was the source for configuration, not the processing of generic configuration files.
+  - "Format" sounded weirder than "language".

granular_configuration_language-2.0.0/granular_configuration_language/__init__.py

@@ -0,0 +1,12 @@
+# isort:skip_file
+import granular_configuration_language.yaml.classes  # Needs to import before _configuration to prevent circular import
+from granular_configuration_language._configuration import Configuration, MutableConfiguration
+from granular_configuration_language._lazy_load_configuration import (
+    LazyLoadConfiguration,
+    LazyLoadConfiguration as LLC,
+    MutableLazyLoadConfiguration,
+)
+from granular_configuration_language._merge import merge  # depends on LazyLoadConfiguration
+
+from granular_configuration_language._json import json_default
+from granular_configuration_language.yaml import Masked, Placeholder

granular_configuration_language-2.0.0/granular_configuration_language/_base_path.py

@@ -0,0 +1,23 @@
+from __future__ import annotations
+
+import collections.abc as tabc
+
+
+class BasePathPart(str):
+    pass
+
+
+class BasePath(tuple[BasePathPart]):
+    pass
+
+
+def read_base_path(base_path: str | tabc.Sequence[str] | None) -> BasePath:
+    if isinstance(base_path, str):
+        if base_path.startswith("/"):
+            return BasePath(map(BasePathPart, filter(None, base_path.split("/"))))
+        else:
+            return BasePath((BasePathPart(base_path),))
+    elif base_path:
+        return BasePath(map(BasePathPart, base_path))
+    else:
+        return BasePath()

granular_configuration_language-2.0.0/granular_configuration_language/_build.py

@@ -0,0 +1,94 @@
+from __future__ import annotations
+
+import collections.abc as tabc
+import typing as typ
+from functools import partial
+from itertools import chain
+from pathlib import Path
+
+from granular_configuration_language import Configuration
+from granular_configuration_language._load import load_file
+from granular_configuration_language._s import setter_secret
+from granular_configuration_language._utils import consume
+from granular_configuration_language.yaml import LazyRoot
+from granular_configuration_language.yaml.load import obj_pairs_func
+
+_C = typ.TypeVar("_C", bound=Configuration)
+
+
+def _merge_into_base(configuration_type: typ.Type[_C], base_dict: _C, from_dict: _C) -> None:
+    for key, value in from_dict._raw_items():
+        if isinstance(value, configuration_type) and (key in base_dict):
+            if base_dict.exists(key):
+                new_dict = base_dict[key]
+            else:  # If Placeholder
+                new_dict = configuration_type()
+
+            if isinstance(new_dict, configuration_type):
+                _merge_into_base(configuration_type, new_dict, value)
+                value = new_dict
+
+        base_dict._private_set(key, value, setter_secret)
+
+
+def _merge(configuration_type: typ.Type[_C], base_config: _C, configs: tabc.Iterable[_C]) -> _C:
+    consume(map(partial(_merge_into_base, configuration_type, base_config), configs))
+    return base_config
+
+
+def _load_configs_from_locations(
+    configuration_type: typ.Type[_C], locations: tabc.Iterable[Path], lazy_root: LazyRoot, mutable: bool
+) -> tabc.Iterator[_C]:
+    def configuration_only(
+        configs: tabc.Iterable[_C | typ.Any],
+    ) -> tabc.Iterator[_C]:
+        for config in configs:
+            if isinstance(config, configuration_type):
+                yield config
+
+    _load_file = partial(load_file, lazy_root=lazy_root, mutable=mutable)
+    return configuration_only(map(_load_file, locations))
+
+
+def _inject(*new: Configuration, configs: tabc.Iterator[_C], in_front: bool) -> tabc.Iterator[_C]:
+    cast = typ.cast(_C, new)
+    if in_front:
+        return chain(cast, configs)
+    else:
+        return chain(configs, cast)
+
+
+def _inject_configs(
+    configs: tabc.Iterator[_C],
+    *,
+    before: Configuration | None,
+    after: Configuration | None,
+) -> tabc.Iterator[_C]:
+    if before and isinstance(before, Configuration):
+        configs = _inject(before, configs=configs, in_front=True)
+
+    if after and isinstance(after, Configuration):
+        configs = _inject(after, configs=configs, in_front=False)
+
+    return configs
+
+
+def build_configuration(
+    locations: tabc.Iterable[Path],
+    mutable: bool,
+    *,
+    inject_before: Configuration | None,
+    inject_after: Configuration | None,
+) -> Configuration:
+    configuration_type = obj_pairs_func(mutable)
+    base_config = configuration_type()
+    lazy_root = LazyRoot.with_root(base_config)
+
+    valid_configs = _inject_configs(
+        _load_configs_from_locations(configuration_type, locations, lazy_root, mutable),
+        before=inject_before,
+        after=inject_after,
+    )
+    merged_config = _merge(configuration_type, base_config, valid_configs)
+
+    return merged_config

granular_configuration_language-2.0.0/granular_configuration_language/_cache.py

@@ -0,0 +1,105 @@
+from __future__ import annotations
+
+import collections.abc as tabc
+import dataclasses
+import operator as op
+import typing as typ
+from collections import deque
+from functools import cached_property, reduce
+from threading import Lock
+from weakref import WeakValueDictionary
+
+from granular_configuration_language._base_path import BasePath, read_base_path
+from granular_configuration_language._build import build_configuration
+from granular_configuration_language._configuration import Configuration
+from granular_configuration_language._locations import Locations
+
+
+@dataclasses.dataclass(frozen=False, eq=False, kw_only=True)
+class SharedConfigurationReference:
+    _locations: Locations
+    _mutable_config: bool
+    _inject_before: Configuration | None = None
+    _inject_after: Configuration | None = None
+    __lock: Lock | None = dataclasses.field(repr=False, compare=False, init=False, default_factory=Lock)
+    __notes: deque[NoteOfIntentToRead] = dataclasses.field(repr=False, compare=False, init=False, default_factory=deque)
+
+    def register(self, note: NoteOfIntentToRead) -> None:
+        self.__notes.append(note)
+
+    def __clear_notes(self, caller: NoteOfIntentToRead) -> None:
+        while self.__notes:
+            note = self.__notes.pop()
+            if note is not caller:
+                note._config
+
+    def build(self, caller: NoteOfIntentToRead) -> Configuration:
+        # Making cached_property thread-safe
+        if self.__lock:
+            with self.__lock:
+                self.__config
+                self.__lock = None
+                self.__clear_notes(caller)
+
+        return self.__config
+
+    @cached_property
+    def __config(self) -> Configuration:
+        return build_configuration(
+            self._locations, self._mutable_config, inject_after=self._inject_after, inject_before=self._inject_before
+        )
+
+
+@dataclasses.dataclass(frozen=False, eq=False, kw_only=True)
+class NoteOfIntentToRead:
+    _base_path: BasePath
+    _config_ref: SharedConfigurationReference
+
+    def __post_init__(self) -> None:
+        self._config_ref.register(self)
+
+    @property
+    def config(self) -> Configuration:
+        config = self._config
+        if isinstance(config, Exception):
+            raise config
+        else:
+            return config
+
+    @cached_property
+    def _config(self) -> Configuration | Exception:
+        config = self._config_ref.build(self)
+        try:
+            return reduce(op.getitem, self._base_path, config)
+        except Exception as e:
+            return e
+        finally:
+            del self._config_ref
+
+
+store: typ.Final[WeakValueDictionary[Locations, SharedConfigurationReference]] = WeakValueDictionary()
+
+
+def prepare_to_load_configuration(
+    *,
+    locations: Locations,
+    base_path: str | tabc.Sequence[str] | None,
+    mutable_configuration: bool,
+    disable_cache: bool,
+    inject_before: Configuration | None,
+    inject_after: Configuration | None,
+) -> NoteOfIntentToRead:
+    if disable_cache or mutable_configuration or inject_after or inject_before:
+        shared_config_ref = SharedConfigurationReference(
+            _locations=locations,
+            _mutable_config=mutable_configuration,
+            _inject_after=inject_after,
+            _inject_before=inject_before,
+        )
+    elif locations not in store:
+        shared_config_ref = SharedConfigurationReference(_locations=locations, _mutable_config=mutable_configuration)
+        store[locations] = shared_config_ref
+    else:
+        shared_config_ref = store[locations]
+
+    return NoteOfIntentToRead(_base_path=read_base_path(base_path), _config_ref=shared_config_ref)