modconfig 1.2.1__tar.gz → 1.3.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,178 @@
1
+ Metadata-Version: 2.4
2
+ Name: modconfig
3
+ Version: 1.3.2
4
+ Summary: Simple hierarchic configuration manager for apps
5
+ Author-email: Kirill Klenov <horneds@gmail.com>
6
+ License-Expression: MIT
7
+ Project-URL: Documentation, https://github.com/klen/modconfig
8
+ Project-URL: Source, https://github.com/klen/modconfig
9
+ Project-URL: Issue tracker, https://github.com/klen/modconfig/issues
10
+ Keywords: settings,configuration
11
+ Classifier: Development Status :: 5 - Production/Stable
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Programming Language :: Python
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Programming Language :: Python :: 3.13
18
+ Classifier: Programming Language :: Python :: 3.14
19
+ Classifier: Programming Language :: Python :: 3.15
20
+ Requires-Python: >=3.11
21
+ Description-Content-Type: text/markdown
22
+ License-File: LICENSE
23
+ Dynamic: license-file
24
+
25
+ # modconfig
26
+
27
+ [![Tests](https://github.com/klen/modconfig/workflows/tests/badge.svg)](https://github.com/klen/modconfig/actions)
28
+ [![PyPI](https://img.shields.io/pypi/v/modconfig)](https://pypi.org/project/modconfig/)
29
+ [![Python](https://img.shields.io/pypi/pyversions/modconfig)](https://pypi.org/project/modconfig/)
30
+ [![License](https://img.shields.io/pypi/l/modconfig)](https://opensource.org/licenses/MIT)
31
+
32
+ Simple hierarchic configuration manager for apps.
33
+
34
+ ## Motivation
35
+
36
+ Applications (especially web services) often require certain configuration
37
+ options to depend on the environment an application runs in (development,
38
+ testing, production, etc.). For instance, a database address config option may
39
+ default to a local database server during development, a mock database server
40
+ during testing, and yet another database server during production. It may also
41
+ need to be customizable via an environment variable. `modconfig` approaches
42
+ scenarios like this, and allows you to specify default configuration options
43
+ for various environments and optionally override them by custom environment
44
+ variables.
45
+
46
+ `modconfig` uses Python modules to keep the configuration options. You are not
47
+ locked by format (json, yaml, ini) restrictions and are able to use any Python
48
+ statements/modules to tune your configuration as a pro. It keeps the
49
+ flexibility and makes your configuration very declarative without any magic.
50
+
51
+ ## Requirements
52
+
53
+ - Python >= 3.11
54
+
55
+ ## Installation
56
+
57
+ ```bash
58
+ pip install modconfig
59
+ ```
60
+
61
+ ## Usage
62
+
63
+ For example you have the structure in your app:
64
+
65
+ ```text
66
+ |- myapp/
67
+ | |- __init__.py
68
+ | |- config/
69
+ | | |- __init__.py
70
+ | | |- defaults.py
71
+ | | |- production.py
72
+ | | |- stage.py
73
+ | | |- tests.py
74
+ | | ...
75
+ ```
76
+
77
+ See the [example](https://github.com/klen/modconfig/tree/develop/example) as a
78
+ simple reference.
79
+
80
+ Initialize the config in your app and use it anywhere:
81
+
82
+ ```python
83
+ from modconfig import Config
84
+
85
+ cfg = Config(
86
+ # instead an import path it could be the module itself
87
+ "myapp.config.production",
88
+ # Override any options
89
+ ANY_OPTION1="VALUE",
90
+ ANY_OPTION2="VALUE",
91
+ )
92
+
93
+ assert cfg.DATABASE
94
+ assert cfg.ANY_OPTION1
95
+ ```
96
+
97
+ Option names are case-insensitive: `cfg.DATABASE`, `cfg.database` and
98
+ `cfg["database"]` return the same value. The type of an option comes from the
99
+ first value it receives, or from a bare type hint in the config module
100
+ (`LIMIT: int = 5`); anything set later is converted to that type, and a value
101
+ that cannot be converted is logged and ignored.
102
+
103
+ ### Fallbacks
104
+
105
+ With several modules, `modconfig` uses the first one it can import:
106
+
107
+ ```python
108
+ from modconfig import Config
109
+
110
+ cfg = Config("myapp.config.local", "myapp.config.production", ANY_OPTION1="VALUE")
111
+
112
+ assert cfg.DATABASE
113
+ assert cfg.ANY_OPTION1
114
+ ```
115
+
116
+ ### Environments
117
+
118
+ The module path may be set as an ENV variable:
119
+
120
+ ```python
121
+ import os
122
+
123
+ from modconfig import Config
124
+
125
+ # Let's define an env var
126
+ os.environ["MODCONFIG"] = "myapp.config.production"
127
+
128
+ cfg = Config("env:MODCONFIG", "myapp.config.local")
129
+ assert cfg.DATABASE
130
+ ```
131
+
132
+ ### Custom environment variables
133
+
134
+ Any option may be overridden with an ENV variable. Values are parsed as JSON
135
+ when possible, so dicts and lists work; anything else stays a string.
136
+
137
+ ENV variables for options that do not exist in the config are ignored.
138
+
139
+ To pick up overrides only for your app, give them a prefix:
140
+
141
+ ```python
142
+ cfg = Config("myapp.config.production", config_config={"env_prefix": "APP_"})
143
+ # APP_DATABASE overrides DATABASE, DEMOENV is ignored
144
+ ```
145
+
146
+ The meta options of the config itself (`env_prefix`, `update_from_env`) are
147
+ passed through `config_config`.
148
+
149
+ See [tests.py](https://github.com/klen/modconfig/blob/develop/tests.py) for
150
+ more examples.
151
+
152
+ ## Limitations
153
+
154
+ Configuration modules are Python code, so importing one executes it. Keep them
155
+ as trusted as the application itself.
156
+
157
+ Only the first importable module in the list is used. The rest are fallbacks for
158
+ when an earlier module is missing, not layers that get merged.
159
+
160
+ ENV overrides apply only to options that already exist in the config; unknown
161
+ variables are ignored rather than added.
162
+
163
+ Because an option keeps the type of its first value, replacing an `int` with a
164
+ `str` later does not work: the string is converted back to an `int` or dropped.
165
+
166
+ ## Bug tracker
167
+
168
+ If you have any suggestions, bug reports or annoyances please report them to
169
+ the issue tracker at <https://github.com/klen/modconfig/issues>.
170
+
171
+ ## Contributing
172
+
173
+ Development of the project happens at
174
+ <https://github.com/klen/modconfig>.
175
+
176
+ ## License
177
+
178
+ Licensed under a [MIT license](https://opensource.org/licenses/MIT).
@@ -0,0 +1,154 @@
1
+ # modconfig
2
+
3
+ [![Tests](https://github.com/klen/modconfig/workflows/tests/badge.svg)](https://github.com/klen/modconfig/actions)
4
+ [![PyPI](https://img.shields.io/pypi/v/modconfig)](https://pypi.org/project/modconfig/)
5
+ [![Python](https://img.shields.io/pypi/pyversions/modconfig)](https://pypi.org/project/modconfig/)
6
+ [![License](https://img.shields.io/pypi/l/modconfig)](https://opensource.org/licenses/MIT)
7
+
8
+ Simple hierarchic configuration manager for apps.
9
+
10
+ ## Motivation
11
+
12
+ Applications (especially web services) often require certain configuration
13
+ options to depend on the environment an application runs in (development,
14
+ testing, production, etc.). For instance, a database address config option may
15
+ default to a local database server during development, a mock database server
16
+ during testing, and yet another database server during production. It may also
17
+ need to be customizable via an environment variable. `modconfig` approaches
18
+ scenarios like this, and allows you to specify default configuration options
19
+ for various environments and optionally override them by custom environment
20
+ variables.
21
+
22
+ `modconfig` uses Python modules to keep the configuration options. You are not
23
+ locked by format (json, yaml, ini) restrictions and are able to use any Python
24
+ statements/modules to tune your configuration as a pro. It keeps the
25
+ flexibility and makes your configuration very declarative without any magic.
26
+
27
+ ## Requirements
28
+
29
+ - Python >= 3.11
30
+
31
+ ## Installation
32
+
33
+ ```bash
34
+ pip install modconfig
35
+ ```
36
+
37
+ ## Usage
38
+
39
+ For example you have the structure in your app:
40
+
41
+ ```text
42
+ |- myapp/
43
+ | |- __init__.py
44
+ | |- config/
45
+ | | |- __init__.py
46
+ | | |- defaults.py
47
+ | | |- production.py
48
+ | | |- stage.py
49
+ | | |- tests.py
50
+ | | ...
51
+ ```
52
+
53
+ See the [example](https://github.com/klen/modconfig/tree/develop/example) as a
54
+ simple reference.
55
+
56
+ Initialize the config in your app and use it anywhere:
57
+
58
+ ```python
59
+ from modconfig import Config
60
+
61
+ cfg = Config(
62
+ # instead an import path it could be the module itself
63
+ "myapp.config.production",
64
+ # Override any options
65
+ ANY_OPTION1="VALUE",
66
+ ANY_OPTION2="VALUE",
67
+ )
68
+
69
+ assert cfg.DATABASE
70
+ assert cfg.ANY_OPTION1
71
+ ```
72
+
73
+ Option names are case-insensitive: `cfg.DATABASE`, `cfg.database` and
74
+ `cfg["database"]` return the same value. The type of an option comes from the
75
+ first value it receives, or from a bare type hint in the config module
76
+ (`LIMIT: int = 5`); anything set later is converted to that type, and a value
77
+ that cannot be converted is logged and ignored.
78
+
79
+ ### Fallbacks
80
+
81
+ With several modules, `modconfig` uses the first one it can import:
82
+
83
+ ```python
84
+ from modconfig import Config
85
+
86
+ cfg = Config("myapp.config.local", "myapp.config.production", ANY_OPTION1="VALUE")
87
+
88
+ assert cfg.DATABASE
89
+ assert cfg.ANY_OPTION1
90
+ ```
91
+
92
+ ### Environments
93
+
94
+ The module path may be set as an ENV variable:
95
+
96
+ ```python
97
+ import os
98
+
99
+ from modconfig import Config
100
+
101
+ # Let's define an env var
102
+ os.environ["MODCONFIG"] = "myapp.config.production"
103
+
104
+ cfg = Config("env:MODCONFIG", "myapp.config.local")
105
+ assert cfg.DATABASE
106
+ ```
107
+
108
+ ### Custom environment variables
109
+
110
+ Any option may be overridden with an ENV variable. Values are parsed as JSON
111
+ when possible, so dicts and lists work; anything else stays a string.
112
+
113
+ ENV variables for options that do not exist in the config are ignored.
114
+
115
+ To pick up overrides only for your app, give them a prefix:
116
+
117
+ ```python
118
+ cfg = Config("myapp.config.production", config_config={"env_prefix": "APP_"})
119
+ # APP_DATABASE overrides DATABASE, DEMOENV is ignored
120
+ ```
121
+
122
+ The meta options of the config itself (`env_prefix`, `update_from_env`) are
123
+ passed through `config_config`.
124
+
125
+ See [tests.py](https://github.com/klen/modconfig/blob/develop/tests.py) for
126
+ more examples.
127
+
128
+ ## Limitations
129
+
130
+ Configuration modules are Python code, so importing one executes it. Keep them
131
+ as trusted as the application itself.
132
+
133
+ Only the first importable module in the list is used. The rest are fallbacks for
134
+ when an earlier module is missing, not layers that get merged.
135
+
136
+ ENV overrides apply only to options that already exist in the config; unknown
137
+ variables are ignored rather than added.
138
+
139
+ Because an option keeps the type of its first value, replacing an `int` with a
140
+ `str` later does not work: the string is converted back to an `int` or dropped.
141
+
142
+ ## Bug tracker
143
+
144
+ If you have any suggestions, bug reports or annoyances please report them to
145
+ the issue tracker at <https://github.com/klen/modconfig/issues>.
146
+
147
+ ## Contributing
148
+
149
+ Development of the project happens at
150
+ <https://github.com/klen/modconfig>.
151
+
152
+ ## License
153
+
154
+ Licensed under a [MIT license](https://opensource.org/licenses/MIT).
@@ -0,0 +1,250 @@
1
+ """Hierarchic configuration for applications, kept in Python modules.
2
+
3
+ Options are loaded from one of several modules (development, production,
4
+ staging, ...) and may be overridden by keyword arguments and by OS
5
+ environment variables.
6
+ """
7
+
8
+ import json
9
+ import logging
10
+ import os
11
+ from collections.abc import Callable, Iterator, Mapping
12
+ from importlib import import_module
13
+ from importlib.metadata import PackageNotFoundError, version
14
+ from inspect import getmembers, isbuiltin, isclass, ismodule
15
+ from types import ModuleType
16
+ from typing import Any, get_type_hints
17
+
18
+ try:
19
+ __version__ = version("modconfig")
20
+ except PackageNotFoundError: # running from a source checkout
21
+ __version__ = "0.0.0"
22
+
23
+ __license__ = "MIT"
24
+ __all__ = ("Config",)
25
+
26
+
27
+ logger: logging.Logger = logging.getLogger("scfg")
28
+ identity: Callable[[Any], Any] = lambda v: v # noqa
29
+ types_supported: set[type] = {int, float, bool, str, bytes, object, list, tuple, dict}
30
+
31
+
32
+ class Config:
33
+
34
+ """Application configuration loaded from Python modules and overrides.
35
+
36
+ Options are stored under upper-cased names, so access is case-insensitive:
37
+ ``cfg.database``, ``cfg.DATABASE`` and ``cfg["database"]`` are the same
38
+ option. The type of an option comes from the first value it receives;
39
+ later values are converted to that type, and a value that cannot be
40
+ converted is logged and ignored.
41
+
42
+ Sources are applied in order: modules given on init, option overrides
43
+ (keyword arguments), then the OS environment.
44
+ """
45
+
46
+ __slots__ = "__env_prefix", "__storage", "__annotations"
47
+
48
+ def __init__(
49
+ self,
50
+ *mods: str | ModuleType,
51
+ config_config: Mapping[str, Any] | None = None,
52
+ **options: Any,
53
+ ) -> None:
54
+ """Initialize the config.
55
+
56
+ :param mods: Module objects or import paths. The first one that can be
57
+ imported is used, the rest are fallbacks; a path prefixed with
58
+ ``env:`` is looked up in the OS environment first.
59
+ :param config_config: Meta options of the config itself.
60
+ ``update_from_env`` (True) reads options from the OS environment
61
+ after initialization, ``env_prefix`` ("") limits that to variables
62
+ starting with the given prefix.
63
+ :param options: Option overrides applied after the modules are loaded.
64
+ """
65
+ opts = config_config or {}
66
+
67
+ self.__storage: dict[str, object] = {}
68
+ self.__annotations: dict[str, Callable[[Any], Any]] = {}
69
+ self.__env_prefix: str = opts.get("env_prefix", "")
70
+
71
+ if mods:
72
+ self.update_from_modules(*mods)
73
+
74
+ self.update(**options)
75
+
76
+ if opts.get("update_from_env", True):
77
+ self.update_from_env()
78
+
79
+ def __repr__(self) -> str:
80
+ """Show the loaded options."""
81
+ return "<Config %r>" % self.__storage
82
+
83
+ def __iter__(self) -> Iterator[tuple[str, object]]:
84
+ """Iterate over ``(NAME, value)`` pairs."""
85
+ return iter(self.__storage.items())
86
+
87
+ def __getattr__(self, name: str) -> Any:
88
+ """Look an option up as an attribute.
89
+
90
+ :raises AttributeError: The option is not set.
91
+ """
92
+ try:
93
+ return self.__storage[name.upper()]
94
+ except KeyError:
95
+ raise AttributeError(f"Invalid option: {name}")
96
+
97
+ def __getitem__(self, name: str) -> Any:
98
+ """Look an option up by name.
99
+
100
+ :raises KeyError: The option is not set.
101
+ """
102
+ return self.__storage[name.upper()]
103
+
104
+ def get(self, name: str, default: object = None) -> Any:
105
+ """Look an option up by name, falling back to ``default``.
106
+
107
+ :param name: Option name, case-insensitive.
108
+ :param default: Returned when the option is not set.
109
+ """
110
+ return self.__storage.get(name.upper(), default)
111
+
112
+ def update(self, *mods: str | ModuleType, **options: Any) -> None:
113
+ """Load modules and apply overrides on top of the current options."""
114
+ self.update_from_modules(*mods)
115
+ self.update_from_dict(options, exist_only=False)
116
+
117
+ def update_from_dict(
118
+ self,
119
+ options: Mapping[str, object],
120
+ *,
121
+ exist_only: bool = True,
122
+ prefix: str = "",
123
+ processor: Callable[[Any], Any] = identity,
124
+ annotations: Mapping[str, Callable[[Any], Any]] | None = None,
125
+ ) -> None:
126
+ """Merge options from a mapping into the configuration.
127
+
128
+ Option names are upper-cased and options starting with ``_`` are
129
+ skipped.
130
+
131
+ :param options: Raw ``name -> value`` pairs.
132
+ :param exist_only: Only update options that are already set, which
133
+ keeps unknown names out of the config (used for the environment).
134
+ :param prefix: Only accept names starting with the prefix, and strip it
135
+ before storing.
136
+ :param processor: Called on each raw value before type conversion, e.g.
137
+ to parse JSON.
138
+ :param annotations: Option types by original name, usually module type
139
+ hints; they win over the type of the value.
140
+ """
141
+ annotations = annotations or {}
142
+ prefix = prefix.upper()
143
+ prefix_length = len(prefix)
144
+ for name, value in options.items():
145
+ vtype = annotations.get(name)
146
+ name = name.upper()
147
+ if prefix:
148
+ if not name.startswith(prefix):
149
+ continue
150
+
151
+ name = name[prefix_length:]
152
+
153
+ if not name or name.startswith("_"):
154
+ continue
155
+
156
+ if exist_only and name not in self.__storage:
157
+ continue
158
+
159
+ if name in self.__storage:
160
+ vtype = self.__annotations[name]
161
+
162
+ elif exist_only:
163
+ continue
164
+
165
+ else:
166
+ if not vtype:
167
+ vtype = type(value)
168
+
169
+ vtype = (
170
+ vtype
171
+ if vtype in types_supported and getattr(vtype, "_inst", True)
172
+ else identity
173
+ )
174
+ self.__annotations[name] = vtype
175
+
176
+ try:
177
+ value = processor(value)
178
+ if value is not None:
179
+ value = vtype(value)
180
+ self.__storage[name] = value
181
+ except (ValueError, TypeError):
182
+ logger.warning(
183
+ "Ignored invalid configuration value given for %s: %s", name, value
184
+ )
185
+ continue
186
+
187
+ def update_from_modules(
188
+ self, *mods: str | ModuleType, exist_only: bool = False
189
+ ) -> str | None:
190
+ """Load options from the first available module.
191
+
192
+ :param mods: Module objects or import paths. A path prefixed with
193
+ ``env:`` is looked up in the OS environment first.
194
+ :param exist_only: Only update options that are already set.
195
+ :return: Name of the loaded module, or None when nothing was imported.
196
+ """
197
+ try:
198
+ mod, *fallback = mods
199
+ except ValueError:
200
+ return None
201
+
202
+ fallback = list(fallback)
203
+ while mod:
204
+ try:
205
+ if isinstance(mod, str):
206
+ if mod.lower().startswith("env:"):
207
+ mod = os.environ[mod[4:]]
208
+
209
+ mod = import_module(mod)
210
+
211
+ break
212
+
213
+ except (ImportError, KeyError):
214
+ logger.debug("Ignore invalid configuration module given: %s", mod)
215
+ mod = fallback.pop(0) if fallback else ""
216
+
217
+ else:
218
+ return None
219
+
220
+ members = getmembers(
221
+ mod, lambda v: not (isclass(v) or ismodule(v) or isbuiltin(v))
222
+ )
223
+ self.update_from_dict(
224
+ dict(members), exist_only=exist_only, annotations=get_type_hints(mod)
225
+ )
226
+ return mod.__name__
227
+
228
+ def update_from_env(self, prefix: str | None = None) -> None:
229
+ """Update the configuration from environment variables.
230
+
231
+ Only options that are already set are updated. Values are parsed as
232
+ JSON when possible and stay strings otherwise.
233
+
234
+ :param prefix: Variable name prefix, defaults to the ``env_prefix`` meta
235
+ option; the prefix is stripped from the option names.
236
+ """
237
+
238
+ def processor(value: str) -> Any:
239
+ try:
240
+ return json.loads(value)
241
+ except ValueError:
242
+ return value
243
+
244
+ prefix = (prefix or self.__env_prefix).upper()
245
+ trim = len(prefix)
246
+ options = {
247
+ n[trim:]: v for n, v in os.environ.items() if n.upper().startswith(prefix)
248
+ }
249
+
250
+ self.update_from_dict(options, exist_only=True, processor=processor)