typedconf 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- typedconf-0.1.0/.gitignore +13 -0
- typedconf-0.1.0/.python-version +1 -0
- typedconf-0.1.0/PKG-INFO +327 -0
- typedconf-0.1.0/README.md +313 -0
- typedconf-0.1.0/main_demo01.py +47 -0
- typedconf-0.1.0/main_demo02.py +58 -0
- typedconf-0.1.0/pyproject.toml +29 -0
- typedconf-0.1.0/src/typedconf/__init__.py +42 -0
- typedconf-0.1.0/src/typedconf/core.py +446 -0
- typedconf-0.1.0/tests/conftest.py +9 -0
- typedconf-0.1.0/tests/test_config.py +373 -0
- typedconf-0.1.0/uv.lock +184 -0
|
@@ -0,0 +1 @@
|
|
|
1
|
+
3.12
|
typedconf-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,327 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: typedconf
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: A lightweight, type-safe configuration management library for Python
|
|
5
|
+
Author: the_pi
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Requires-Python: >=3.12
|
|
8
|
+
Requires-Dist: pydantic>=2.11.7
|
|
9
|
+
Provides-Extra: dev
|
|
10
|
+
Requires-Dist: pytest>=9.0.3; extra == 'dev'
|
|
11
|
+
Provides-Extra: toml
|
|
12
|
+
Requires-Dist: tomli-w>=1.2.0; extra == 'toml'
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
|
|
15
|
+
# TypedConf
|
|
16
|
+
|
|
17
|
+
A lightweight, type-safe configuration management library powered by Pydantic. Following the [12-factor application guide](https://12factor.net/config), it centralizes your application configuration/settings by merging data from multiple sources with a defined priority:
|
|
18
|
+
`env > cli > json > toml > payload > defaults`.
|
|
19
|
+
|
|
20
|
+
TypedConf combines the strengths of Pydantic and dynaconf: While it requires developers to create classes to reflect your configuration (unlike [dynaconf](https://www.dynaconf.com/)), this approach reduces heavily your headaches 🤕🤕😠from misstyped property names, wrong types or format, ...
|
|
21
|
+
TypedConf offers comprehensive IDE support, type safety and runtime data validation, powered by [pydantic](https://pydantic.dev/docs/validation/latest/get-started/why/).
|
|
22
|
+
|
|
23
|
+
## Key Features
|
|
24
|
+
|
|
25
|
+
- **Type-Safe:** Built on Pydantic, ensuring configuration values are validated at runtime.
|
|
26
|
+
- **IDE Support:** Full type-hinting and IntelliSense-Support for seamless development.
|
|
27
|
+
- **Nested Support:** Easily handle complex configuration structures.
|
|
28
|
+
- **TOML and JSON Interface:** Load configuration from toml and/or json files.
|
|
29
|
+
- **CLI and Environment Interface:** Load configuration data from CLI interface (`--cfg_myint=1`) and/or ENV varables (`export CFG_MYINT=1`).
|
|
30
|
+
- **Layered Configuration:** Merges configuration data with a clear priority: `env > cli > json > toml > payload > defaults`
|
|
31
|
+
- **Immutability**: Configuration data is readonly (default) after loading.
|
|
32
|
+
- **Self-Documenting:** Generate help text from your configuration-schema definition.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Quick Start
|
|
37
|
+
|
|
38
|
+
Define your configuration schema/model by inheriting from `ConfigModel`.
|
|
39
|
+
This will handle pydantic's parsing and validation while loading data from different sources, in this eyample directly from the payload:
|
|
40
|
+
|
|
41
|
+
```python
|
|
42
|
+
from typedconf import ConfigModel
|
|
43
|
+
|
|
44
|
+
# define configuration schema
|
|
45
|
+
class AppConfig(ConfigModel):
|
|
46
|
+
app_name: str # required field, no default
|
|
47
|
+
port: int = 8080 # default value
|
|
48
|
+
|
|
49
|
+
# load configuration
|
|
50
|
+
conf = AppConfig.load(payload={'app_name':'app'})
|
|
51
|
+
|
|
52
|
+
print(f"Running {conf.app_name} on port {conf.port}") # Running app on port 8080
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## Load configuration from TOML (or JSON)
|
|
56
|
+
|
|
57
|
+
Loading configuration data from source isn't a big deal. Most the time you will load data from a configurations file. Let's say from this toml-file:
|
|
58
|
+
|
|
59
|
+
```toml
|
|
60
|
+
# config.toml v1
|
|
61
|
+
app_name = "myapp"
|
|
62
|
+
port = 2000
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
```python
|
|
66
|
+
from typedconf import ConfigModel
|
|
67
|
+
|
|
68
|
+
# define configuration schema
|
|
69
|
+
class AppConfig(ConfigModel):
|
|
70
|
+
app_name: str
|
|
71
|
+
port: int = 8080
|
|
72
|
+
|
|
73
|
+
# load configuration
|
|
74
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
75
|
+
|
|
76
|
+
print(f"Running {conf.app_name} on port {conf.port}") # Running myapp on port 2000
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Data Validation and nested configuration
|
|
80
|
+
|
|
81
|
+
`ConfigModel` is a [pydantic](https://pydantic.dev/docs/validation/latest/get-started/#pydantic-examples) `BaseModel`. So you can use the `Field` definitions to add descriptions, constraints, or default values. Nested configuration can be applied by nesting `ConfigModel`classes.
|
|
82
|
+
|
|
83
|
+
```python
|
|
84
|
+
from pydantic import Field
|
|
85
|
+
from typedconf import ConfigModel, ConfigError
|
|
86
|
+
|
|
87
|
+
# define configuration schema
|
|
88
|
+
class DatabaseConfig(ConfigModel):
|
|
89
|
+
con: str = Field(..., description="DB connection-string, required field.")
|
|
90
|
+
user: str = Field(..., description="DB username, required field.")
|
|
91
|
+
pwd: str = Field(..., description="DB password, required field.")
|
|
92
|
+
|
|
93
|
+
class AppConfig(ConfigModel):
|
|
94
|
+
app_name: str = Field(..., description="application name, required field.")
|
|
95
|
+
port: int = Field(8080, ge=1000, le=9999, description="application listen on port. Between 1000 and 9999, defaullt=8080")
|
|
96
|
+
db: DatabaseConfig = Field(default_factory=DatabaseConfig, description="database configuration")
|
|
97
|
+
|
|
98
|
+
# Load configuration
|
|
99
|
+
try:
|
|
100
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
101
|
+
print(f"Running {conf.app_name} on port {conf.port}. DB connected {conf.db.user} @ {conf.db.con}")
|
|
102
|
+
except ConfigError as e:
|
|
103
|
+
print(e)
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Loading from our "old" TOML-file will raise a `ConfigError`, because the stored data didn't reflect the new configuration-schema:
|
|
107
|
+
|
|
108
|
+
```text
|
|
109
|
+
3 validation errors for DatabaseConfig
|
|
110
|
+
con Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing
|
|
111
|
+
user Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing
|
|
112
|
+
pwd Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
TOML is perfect for nested configurations using `tables` and JSON requires nested objects to reflect the same structure 🌞.
|
|
116
|
+
However, it's not the best idea to store sensitive or volatile data in a configuration-file. It is way better to [handle this kind of data by cli-interface and/or through environment variables](https://12factor.net/config).
|
|
117
|
+
Let's fix our TOML-file, while keeping the database password secret:
|
|
118
|
+
|
|
119
|
+
```toml
|
|
120
|
+
# config.toml v2
|
|
121
|
+
app_name = "toml-app"
|
|
122
|
+
port = 9090
|
|
123
|
+
|
|
124
|
+
[db]
|
|
125
|
+
con = "postgresql://localhost:5432/mydb"
|
|
126
|
+
user = "db_user_readonly"
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
**Just remember: don't store any sensitive data in configuration-files!**
|
|
130
|
+
|
|
131
|
+
## ENV & CLI Interface
|
|
132
|
+
|
|
133
|
+
Cool, now we can inject the missing (or secret) data through the cli- and env-interface. Both interfaces are enabled per default.
|
|
134
|
+
|
|
135
|
+
```sh
|
|
136
|
+
# cli-interface
|
|
137
|
+
$ python app.py --cfg_db__pwd="secret"
|
|
138
|
+
Running toml-app on port 9090. DB connected db_user_readonly @ postgresql://localhost:5432/mydb
|
|
139
|
+
|
|
140
|
+
# env-interface
|
|
141
|
+
$ export CFG_DB__PWD="secret"
|
|
142
|
+
$ python app.py
|
|
143
|
+
Running toml-app on port 9090. DB connected db_user_readonly @ postgresql://localhost:5432/mydb
|
|
144
|
+
|
|
145
|
+
# mix them
|
|
146
|
+
$ export CFG_DB__USER="db_user_admin"
|
|
147
|
+
$ export CFG_DB__PWD="secret"
|
|
148
|
+
$ CFG_PORT=2525 python app.py --cfg_app_name="cli-app"
|
|
149
|
+
Running cli-app on port 2525. DB connected db_user_admin @ postgresql://localhost:5432/mydb
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
The CLI- and ENV interface follows this convention:
|
|
153
|
+
|
|
154
|
+
- Case-sensitive: cli is *lowercase*, env is *UPPERCASE*
|
|
155
|
+
- CLI uses only long format for the arguments like `--key=val`
|
|
156
|
+
- Prefix: CLI arguments and ENV variables uses a prefix to avoid cross-situations in the shell. Defaults to `cfg_`. The prefix can be changed.
|
|
157
|
+
- Nested configuration will be seperated by `__` (two underscrores)
|
|
158
|
+
- Examples:
|
|
159
|
+
- cli-interface: `--cfg_app_name` or `--cfg_db__user`
|
|
160
|
+
- env-inteface: `CFG_APP_NAME` or `CFG_DB__USER`
|
|
161
|
+
|
|
162
|
+
## Priority Chain
|
|
163
|
+
|
|
164
|
+
TypedConf merges all data sources in a specific order. Higher-priority sources overwrite lower-priority ones:
|
|
165
|
+
|
|
166
|
+
1. **Environment Variables (Highest):** Overrides all other sources - i.e. `export CFG_DB__PWD="abc"`
|
|
167
|
+
2. **CLI Arguments:** Passed via command-line - i.e. `--cfg_db__pwd='abc'`
|
|
168
|
+
3. **JSON Files:** Merged from the provided list in the order specified
|
|
169
|
+
4. **TOML Files:** Merged from the provided list in the order specified
|
|
170
|
+
5. **Payload:** A dictionary passed directly to the load method - i.e. `.load(payload={"db":{"pwd":"abc"}})`
|
|
171
|
+
6. **Defaults (Lowest):** Default values defined in the `ConfigModel` class
|
|
172
|
+
|
|
173
|
+
*Note: The system performs a deep merge, preserving nested structures when partial overrides are provided.*
|
|
174
|
+
|
|
175
|
+
## Utils
|
|
176
|
+
|
|
177
|
+
### Exporting Configurations
|
|
178
|
+
|
|
179
|
+
Export your current configuration instance to JSON or TOML format.
|
|
180
|
+
|
|
181
|
+
```python
|
|
182
|
+
# Export TOML string
|
|
183
|
+
print(conf.dumps_toml())
|
|
184
|
+
|
|
185
|
+
# Export JSON string
|
|
186
|
+
print(conf.dumps_json())
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
*Note: Exporting to TOML requires python package `tomli-w`.*
|
|
190
|
+
|
|
191
|
+
### CLI Help included
|
|
192
|
+
|
|
193
|
+
TypedConf can include a `--help` argument to your application and generates a nice helptext for all field-names based on their types and descriptions. Let's step back to our *nested configuration example* and add some help for the user:
|
|
194
|
+
|
|
195
|
+
```python
|
|
196
|
+
from pydantic import Field
|
|
197
|
+
from typedconf import ConfigModel, ConfigError
|
|
198
|
+
|
|
199
|
+
RFC3986_URI_REGEX = r'^([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})(?::\d+)?([\/\w \.-]*)*\/?$'
|
|
200
|
+
|
|
201
|
+
# define configuration schema
|
|
202
|
+
class DatabaseConfig(ConfigModel):
|
|
203
|
+
con: str = Field(..., pattern=RFC3986_URI_REGEX, description="DB connection-string, required field.")
|
|
204
|
+
user: str = Field(..., description="DB username, required field.")
|
|
205
|
+
pwd: str = Field(..., description="DB password, required field.")
|
|
206
|
+
|
|
207
|
+
class AppConfig(ConfigModel):
|
|
208
|
+
app_name: str = Field(..., description="application name, required field.")
|
|
209
|
+
port: int = Field(8080, gt=1000, lt=9999, description="application listen on port. Between 1000 and 9999, defaullt=8080")
|
|
210
|
+
db: DatabaseConfig = Field(default_factory=DatabaseConfig, description="database configuration")
|
|
211
|
+
|
|
212
|
+
# need some help?
|
|
213
|
+
if AppConfig.user_needs_help():
|
|
214
|
+
print(f"MYAPP\n\nAvailable CLI Parameter\n{AppConfig.get_cli_helptext()}")
|
|
215
|
+
exit(0)
|
|
216
|
+
|
|
217
|
+
# Load configuration
|
|
218
|
+
try:
|
|
219
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
220
|
+
print(f"Running {conf.app_name} on port {conf.port}. DB connected {conf.db.user} @ {conf.db.con}")
|
|
221
|
+
except ConfigError as e:
|
|
222
|
+
print(e)
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
```text
|
|
226
|
+
$ python main.py --help
|
|
227
|
+
MYAPP
|
|
228
|
+
|
|
229
|
+
Available CLI Parameter
|
|
230
|
+
--cfg_app_name (AppConfig.app_name)
|
|
231
|
+
type=str, default=None
|
|
232
|
+
application name, required field.
|
|
233
|
+
|
|
234
|
+
--cfg_db__con (DatabaseConfig.con)
|
|
235
|
+
type=str, default=None
|
|
236
|
+
DB connection-string, required field.
|
|
237
|
+
|
|
238
|
+
--cfg_db__pwd (DatabaseConfig.pwd)
|
|
239
|
+
type=str, default=None
|
|
240
|
+
DB password, required field.
|
|
241
|
+
|
|
242
|
+
--cfg_db__user (DatabaseConfig.user)
|
|
243
|
+
type=str, default=None
|
|
244
|
+
DB username, required field.
|
|
245
|
+
|
|
246
|
+
--cfg_port (AppConfig.port)
|
|
247
|
+
type=int, default=8080
|
|
248
|
+
application listen on port. Between 1000 and 9999, defaullt=8080
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
### Writeable Configuration
|
|
252
|
+
|
|
253
|
+
Set pydantic's `frozen` to False, if you need a *writeable configuration*.
|
|
254
|
+
|
|
255
|
+
```python
|
|
256
|
+
from pydantic import Field
|
|
257
|
+
from typedconf import ConfigModel, ConfigError
|
|
258
|
+
|
|
259
|
+
# define configuration schema
|
|
260
|
+
class DatabaseConfig(ConfigModel):
|
|
261
|
+
model_config = {'frozen': False} # writeable BaseModel
|
|
262
|
+
|
|
263
|
+
con: str = Field(..., description="DB connection-string, required field.")
|
|
264
|
+
user: str = Field(..., description="DB username, required field.")
|
|
265
|
+
pwd: str = Field(..., description="DB password, required field.")
|
|
266
|
+
|
|
267
|
+
class AppConfig(ConfigModel):
|
|
268
|
+
model_config = {'frozen': False} # writeable BaseModel
|
|
269
|
+
|
|
270
|
+
app_name: str = Field(..., description="application name, required field.")
|
|
271
|
+
port: int = Field(8080, ge=1000, le=9999, description="application listen on port. Between 1000 and 9999, defaullt=8080")
|
|
272
|
+
db: DatabaseConfig = Field(default_factory=DatabaseConfig, description="database configuration")
|
|
273
|
+
|
|
274
|
+
# Load configuration
|
|
275
|
+
try:
|
|
276
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
277
|
+
print(f"Running {conf.app_name} on port {conf.port}. DB connected {conf.db.user} @ {conf.db.con}")
|
|
278
|
+
except ConfigError as e:
|
|
279
|
+
print(e)
|
|
280
|
+
|
|
281
|
+
# write configuration (hint: this is instance-memory only!)
|
|
282
|
+
conf.port = 6789
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
Note: ConfigModel sets the pydantic model_config to:
|
|
286
|
+
|
|
287
|
+
- Set ConfigModel to readonly
|
|
288
|
+
- Raise error, when loading unknown extra data
|
|
289
|
+
- Validate default values when loading
|
|
290
|
+
- validate when assigning a new value to a writeable ConfigModel
|
|
291
|
+
|
|
292
|
+
```python
|
|
293
|
+
model_config = {
|
|
294
|
+
"frozen": True,
|
|
295
|
+
"extra": "forbid",
|
|
296
|
+
"validate_default": True,
|
|
297
|
+
"validate_assignment": True,
|
|
298
|
+
}
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
|
|
302
|
+
## TODOs
|
|
303
|
+
- override cli-seperator
|
|
304
|
+
- use (prefix & ) cli-seperator for metadata and fullnames
|
|
305
|
+
|
|
306
|
+
|
|
307
|
+
|
|
308
|
+
## Comparison with Other Configuration Approaches
|
|
309
|
+
|
|
310
|
+
| Feature | TypedConf | dynaconf | raw toml/json |
|
|
311
|
+
|------------------------|------------------------------------|-----------------------------------|-----------------------------------|
|
|
312
|
+
| Type Safety | Yes (Pydantic-based) | No | No |
|
|
313
|
+
| IDE Support | Excellent (Pydantic integration) | Limited | Limited |
|
|
314
|
+
| Nested Configurations | Native support | Native support | Manual handling |
|
|
315
|
+
| Validation | Built-in (Pydantic) | Optional (schema validation) | Manual |
|
|
316
|
+
| CLI Interface | Built-in | Built-in | Manual parsing |
|
|
317
|
+
| Environment Variables | Built-in | Built-in | Manual handling |
|
|
318
|
+
| TOML Support | Yes | Yes | Yes |
|
|
319
|
+
| JSON Support | Yes | Yes | Yes |
|
|
320
|
+
| Help Text Generation | Yes | Limited | No |
|
|
321
|
+
| Immutability | Default (configurable) | Configurable | Manual handling |
|
|
322
|
+
|
|
323
|
+
|
|
324
|
+
|
|
325
|
+
## License
|
|
326
|
+
|
|
327
|
+
This project is licensed under the GPL-3.0 License - see the [LICENSE](LICENSE) file for details.
|
|
@@ -0,0 +1,313 @@
|
|
|
1
|
+
# TypedConf
|
|
2
|
+
|
|
3
|
+
A lightweight, type-safe configuration management library powered by Pydantic. Following the [12-factor application guide](https://12factor.net/config), it centralizes your application configuration/settings by merging data from multiple sources with a defined priority:
|
|
4
|
+
`env > cli > json > toml > payload > defaults`.
|
|
5
|
+
|
|
6
|
+
TypedConf combines the strengths of Pydantic and dynaconf: While it requires developers to create classes to reflect your configuration (unlike [dynaconf](https://www.dynaconf.com/)), this approach reduces heavily your headaches 🤕🤕😠from misstyped property names, wrong types or format, ...
|
|
7
|
+
TypedConf offers comprehensive IDE support, type safety and runtime data validation, powered by [pydantic](https://pydantic.dev/docs/validation/latest/get-started/why/).
|
|
8
|
+
|
|
9
|
+
## Key Features
|
|
10
|
+
|
|
11
|
+
- **Type-Safe:** Built on Pydantic, ensuring configuration values are validated at runtime.
|
|
12
|
+
- **IDE Support:** Full type-hinting and IntelliSense-Support for seamless development.
|
|
13
|
+
- **Nested Support:** Easily handle complex configuration structures.
|
|
14
|
+
- **TOML and JSON Interface:** Load configuration from toml and/or json files.
|
|
15
|
+
- **CLI and Environment Interface:** Load configuration data from CLI interface (`--cfg_myint=1`) and/or ENV varables (`export CFG_MYINT=1`).
|
|
16
|
+
- **Layered Configuration:** Merges configuration data with a clear priority: `env > cli > json > toml > payload > defaults`
|
|
17
|
+
- **Immutability**: Configuration data is readonly (default) after loading.
|
|
18
|
+
- **Self-Documenting:** Generate help text from your configuration-schema definition.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Quick Start
|
|
23
|
+
|
|
24
|
+
Define your configuration schema/model by inheriting from `ConfigModel`.
|
|
25
|
+
This will handle pydantic's parsing and validation while loading data from different sources, in this eyample directly from the payload:
|
|
26
|
+
|
|
27
|
+
```python
|
|
28
|
+
from typedconf import ConfigModel
|
|
29
|
+
|
|
30
|
+
# define configuration schema
|
|
31
|
+
class AppConfig(ConfigModel):
|
|
32
|
+
app_name: str # required field, no default
|
|
33
|
+
port: int = 8080 # default value
|
|
34
|
+
|
|
35
|
+
# load configuration
|
|
36
|
+
conf = AppConfig.load(payload={'app_name':'app'})
|
|
37
|
+
|
|
38
|
+
print(f"Running {conf.app_name} on port {conf.port}") # Running app on port 8080
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Load configuration from TOML (or JSON)
|
|
42
|
+
|
|
43
|
+
Loading configuration data from source isn't a big deal. Most the time you will load data from a configurations file. Let's say from this toml-file:
|
|
44
|
+
|
|
45
|
+
```toml
|
|
46
|
+
# config.toml v1
|
|
47
|
+
app_name = "myapp"
|
|
48
|
+
port = 2000
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
```python
|
|
52
|
+
from typedconf import ConfigModel
|
|
53
|
+
|
|
54
|
+
# define configuration schema
|
|
55
|
+
class AppConfig(ConfigModel):
|
|
56
|
+
app_name: str
|
|
57
|
+
port: int = 8080
|
|
58
|
+
|
|
59
|
+
# load configuration
|
|
60
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
61
|
+
|
|
62
|
+
print(f"Running {conf.app_name} on port {conf.port}") # Running myapp on port 2000
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Data Validation and nested configuration
|
|
66
|
+
|
|
67
|
+
`ConfigModel` is a [pydantic](https://pydantic.dev/docs/validation/latest/get-started/#pydantic-examples) `BaseModel`. So you can use the `Field` definitions to add descriptions, constraints, or default values. Nested configuration can be applied by nesting `ConfigModel`classes.
|
|
68
|
+
|
|
69
|
+
```python
|
|
70
|
+
from pydantic import Field
|
|
71
|
+
from typedconf import ConfigModel, ConfigError
|
|
72
|
+
|
|
73
|
+
# define configuration schema
|
|
74
|
+
class DatabaseConfig(ConfigModel):
|
|
75
|
+
con: str = Field(..., description="DB connection-string, required field.")
|
|
76
|
+
user: str = Field(..., description="DB username, required field.")
|
|
77
|
+
pwd: str = Field(..., description="DB password, required field.")
|
|
78
|
+
|
|
79
|
+
class AppConfig(ConfigModel):
|
|
80
|
+
app_name: str = Field(..., description="application name, required field.")
|
|
81
|
+
port: int = Field(8080, ge=1000, le=9999, description="application listen on port. Between 1000 and 9999, defaullt=8080")
|
|
82
|
+
db: DatabaseConfig = Field(default_factory=DatabaseConfig, description="database configuration")
|
|
83
|
+
|
|
84
|
+
# Load configuration
|
|
85
|
+
try:
|
|
86
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
87
|
+
print(f"Running {conf.app_name} on port {conf.port}. DB connected {conf.db.user} @ {conf.db.con}")
|
|
88
|
+
except ConfigError as e:
|
|
89
|
+
print(e)
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Loading from our "old" TOML-file will raise a `ConfigError`, because the stored data didn't reflect the new configuration-schema:
|
|
93
|
+
|
|
94
|
+
```text
|
|
95
|
+
3 validation errors for DatabaseConfig
|
|
96
|
+
con Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing
|
|
97
|
+
user Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing
|
|
98
|
+
pwd Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
TOML is perfect for nested configurations using `tables` and JSON requires nested objects to reflect the same structure 🌞.
|
|
102
|
+
However, it's not the best idea to store sensitive or volatile data in a configuration-file. It is way better to [handle this kind of data by cli-interface and/or through environment variables](https://12factor.net/config).
|
|
103
|
+
Let's fix our TOML-file, while keeping the database password secret:
|
|
104
|
+
|
|
105
|
+
```toml
|
|
106
|
+
# config.toml v2
|
|
107
|
+
app_name = "toml-app"
|
|
108
|
+
port = 9090
|
|
109
|
+
|
|
110
|
+
[db]
|
|
111
|
+
con = "postgresql://localhost:5432/mydb"
|
|
112
|
+
user = "db_user_readonly"
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
**Just remember: don't store any sensitive data in configuration-files!**
|
|
116
|
+
|
|
117
|
+
## ENV & CLI Interface
|
|
118
|
+
|
|
119
|
+
Cool, now we can inject the missing (or secret) data through the cli- and env-interface. Both interfaces are enabled per default.
|
|
120
|
+
|
|
121
|
+
```sh
|
|
122
|
+
# cli-interface
|
|
123
|
+
$ python app.py --cfg_db__pwd="secret"
|
|
124
|
+
Running toml-app on port 9090. DB connected db_user_readonly @ postgresql://localhost:5432/mydb
|
|
125
|
+
|
|
126
|
+
# env-interface
|
|
127
|
+
$ export CFG_DB__PWD="secret"
|
|
128
|
+
$ python app.py
|
|
129
|
+
Running toml-app on port 9090. DB connected db_user_readonly @ postgresql://localhost:5432/mydb
|
|
130
|
+
|
|
131
|
+
# mix them
|
|
132
|
+
$ export CFG_DB__USER="db_user_admin"
|
|
133
|
+
$ export CFG_DB__PWD="secret"
|
|
134
|
+
$ CFG_PORT=2525 python app.py --cfg_app_name="cli-app"
|
|
135
|
+
Running cli-app on port 2525. DB connected db_user_admin @ postgresql://localhost:5432/mydb
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
The CLI- and ENV interface follows this convention:
|
|
139
|
+
|
|
140
|
+
- Case-sensitive: cli is *lowercase*, env is *UPPERCASE*
|
|
141
|
+
- CLI uses only long format for the arguments like `--key=val`
|
|
142
|
+
- Prefix: CLI arguments and ENV variables uses a prefix to avoid cross-situations in the shell. Defaults to `cfg_`. The prefix can be changed.
|
|
143
|
+
- Nested configuration will be seperated by `__` (two underscrores)
|
|
144
|
+
- Examples:
|
|
145
|
+
- cli-interface: `--cfg_app_name` or `--cfg_db__user`
|
|
146
|
+
- env-inteface: `CFG_APP_NAME` or `CFG_DB__USER`
|
|
147
|
+
|
|
148
|
+
## Priority Chain
|
|
149
|
+
|
|
150
|
+
TypedConf merges all data sources in a specific order. Higher-priority sources overwrite lower-priority ones:
|
|
151
|
+
|
|
152
|
+
1. **Environment Variables (Highest):** Overrides all other sources - i.e. `export CFG_DB__PWD="abc"`
|
|
153
|
+
2. **CLI Arguments:** Passed via command-line - i.e. `--cfg_db__pwd='abc'`
|
|
154
|
+
3. **JSON Files:** Merged from the provided list in the order specified
|
|
155
|
+
4. **TOML Files:** Merged from the provided list in the order specified
|
|
156
|
+
5. **Payload:** A dictionary passed directly to the load method - i.e. `.load(payload={"db":{"pwd":"abc"}})`
|
|
157
|
+
6. **Defaults (Lowest):** Default values defined in the `ConfigModel` class
|
|
158
|
+
|
|
159
|
+
*Note: The system performs a deep merge, preserving nested structures when partial overrides are provided.*
|
|
160
|
+
|
|
161
|
+
## Utils
|
|
162
|
+
|
|
163
|
+
### Exporting Configurations
|
|
164
|
+
|
|
165
|
+
Export your current configuration instance to JSON or TOML format.
|
|
166
|
+
|
|
167
|
+
```python
|
|
168
|
+
# Export TOML string
|
|
169
|
+
print(conf.dumps_toml())
|
|
170
|
+
|
|
171
|
+
# Export JSON string
|
|
172
|
+
print(conf.dumps_json())
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
*Note: Exporting to TOML requires python package `tomli-w`.*
|
|
176
|
+
|
|
177
|
+
### CLI Help included
|
|
178
|
+
|
|
179
|
+
TypedConf can include a `--help` argument to your application and generates a nice helptext for all field-names based on their types and descriptions. Let's step back to our *nested configuration example* and add some help for the user:
|
|
180
|
+
|
|
181
|
+
```python
|
|
182
|
+
from pydantic import Field
|
|
183
|
+
from typedconf import ConfigModel, ConfigError
|
|
184
|
+
|
|
185
|
+
RFC3986_URI_REGEX = r'^([a-zA-Z][a-zA-Z0-9+.-]*:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})(?::\d+)?([\/\w \.-]*)*\/?$'
|
|
186
|
+
|
|
187
|
+
# define configuration schema
|
|
188
|
+
class DatabaseConfig(ConfigModel):
|
|
189
|
+
con: str = Field(..., pattern=RFC3986_URI_REGEX, description="DB connection-string, required field.")
|
|
190
|
+
user: str = Field(..., description="DB username, required field.")
|
|
191
|
+
pwd: str = Field(..., description="DB password, required field.")
|
|
192
|
+
|
|
193
|
+
class AppConfig(ConfigModel):
|
|
194
|
+
app_name: str = Field(..., description="application name, required field.")
|
|
195
|
+
port: int = Field(8080, gt=1000, lt=9999, description="application listen on port. Between 1000 and 9999, defaullt=8080")
|
|
196
|
+
db: DatabaseConfig = Field(default_factory=DatabaseConfig, description="database configuration")
|
|
197
|
+
|
|
198
|
+
# need some help?
|
|
199
|
+
if AppConfig.user_needs_help():
|
|
200
|
+
print(f"MYAPP\n\nAvailable CLI Parameter\n{AppConfig.get_cli_helptext()}")
|
|
201
|
+
exit(0)
|
|
202
|
+
|
|
203
|
+
# Load configuration
|
|
204
|
+
try:
|
|
205
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
206
|
+
print(f"Running {conf.app_name} on port {conf.port}. DB connected {conf.db.user} @ {conf.db.con}")
|
|
207
|
+
except ConfigError as e:
|
|
208
|
+
print(e)
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
```text
|
|
212
|
+
$ python main.py --help
|
|
213
|
+
MYAPP
|
|
214
|
+
|
|
215
|
+
Available CLI Parameter
|
|
216
|
+
--cfg_app_name (AppConfig.app_name)
|
|
217
|
+
type=str, default=None
|
|
218
|
+
application name, required field.
|
|
219
|
+
|
|
220
|
+
--cfg_db__con (DatabaseConfig.con)
|
|
221
|
+
type=str, default=None
|
|
222
|
+
DB connection-string, required field.
|
|
223
|
+
|
|
224
|
+
--cfg_db__pwd (DatabaseConfig.pwd)
|
|
225
|
+
type=str, default=None
|
|
226
|
+
DB password, required field.
|
|
227
|
+
|
|
228
|
+
--cfg_db__user (DatabaseConfig.user)
|
|
229
|
+
type=str, default=None
|
|
230
|
+
DB username, required field.
|
|
231
|
+
|
|
232
|
+
--cfg_port (AppConfig.port)
|
|
233
|
+
type=int, default=8080
|
|
234
|
+
application listen on port. Between 1000 and 9999, defaullt=8080
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
### Writeable Configuration
|
|
238
|
+
|
|
239
|
+
Set pydantic's `frozen` to False, if you need a *writeable configuration*.
|
|
240
|
+
|
|
241
|
+
```python
|
|
242
|
+
from pydantic import Field
|
|
243
|
+
from typedconf import ConfigModel, ConfigError
|
|
244
|
+
|
|
245
|
+
# define configuration schema
|
|
246
|
+
class DatabaseConfig(ConfigModel):
|
|
247
|
+
model_config = {'frozen': False} # writeable BaseModel
|
|
248
|
+
|
|
249
|
+
con: str = Field(..., description="DB connection-string, required field.")
|
|
250
|
+
user: str = Field(..., description="DB username, required field.")
|
|
251
|
+
pwd: str = Field(..., description="DB password, required field.")
|
|
252
|
+
|
|
253
|
+
class AppConfig(ConfigModel):
|
|
254
|
+
model_config = {'frozen': False} # writeable BaseModel
|
|
255
|
+
|
|
256
|
+
app_name: str = Field(..., description="application name, required field.")
|
|
257
|
+
port: int = Field(8080, ge=1000, le=9999, description="application listen on port. Between 1000 and 9999, defaullt=8080")
|
|
258
|
+
db: DatabaseConfig = Field(default_factory=DatabaseConfig, description="database configuration")
|
|
259
|
+
|
|
260
|
+
# Load configuration
|
|
261
|
+
try:
|
|
262
|
+
conf = AppConfig.load(toml_files=['config.toml'])
|
|
263
|
+
print(f"Running {conf.app_name} on port {conf.port}. DB connected {conf.db.user} @ {conf.db.con}")
|
|
264
|
+
except ConfigError as e:
|
|
265
|
+
print(e)
|
|
266
|
+
|
|
267
|
+
# write configuration (hint: this is instance-memory only!)
|
|
268
|
+
conf.port = 6789
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
Note: ConfigModel sets the pydantic model_config to:
|
|
272
|
+
|
|
273
|
+
- Set ConfigModel to readonly
|
|
274
|
+
- Raise error, when loading unknown extra data
|
|
275
|
+
- Validate default values when loading
|
|
276
|
+
- validate when assigning a new value to a writeable ConfigModel
|
|
277
|
+
|
|
278
|
+
```python
|
|
279
|
+
model_config = {
|
|
280
|
+
"frozen": True,
|
|
281
|
+
"extra": "forbid",
|
|
282
|
+
"validate_default": True,
|
|
283
|
+
"validate_assignment": True,
|
|
284
|
+
}
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
|
|
288
|
+
## TODOs
|
|
289
|
+
- override cli-seperator
|
|
290
|
+
- use (prefix & ) cli-seperator for metadata and fullnames
|
|
291
|
+
|
|
292
|
+
|
|
293
|
+
|
|
294
|
+
## Comparison with Other Configuration Approaches
|
|
295
|
+
|
|
296
|
+
| Feature | TypedConf | dynaconf | raw toml/json |
|
|
297
|
+
|------------------------|------------------------------------|-----------------------------------|-----------------------------------|
|
|
298
|
+
| Type Safety | Yes (Pydantic-based) | No | No |
|
|
299
|
+
| IDE Support | Excellent (Pydantic integration) | Limited | Limited |
|
|
300
|
+
| Nested Configurations | Native support | Native support | Manual handling |
|
|
301
|
+
| Validation | Built-in (Pydantic) | Optional (schema validation) | Manual |
|
|
302
|
+
| CLI Interface | Built-in | Built-in | Manual parsing |
|
|
303
|
+
| Environment Variables | Built-in | Built-in | Manual handling |
|
|
304
|
+
| TOML Support | Yes | Yes | Yes |
|
|
305
|
+
| JSON Support | Yes | Yes | Yes |
|
|
306
|
+
| Help Text Generation | Yes | Limited | No |
|
|
307
|
+
| Immutability | Default (configurable) | Configurable | Manual handling |
|
|
308
|
+
|
|
309
|
+
|
|
310
|
+
|
|
311
|
+
## License
|
|
312
|
+
|
|
313
|
+
This project is licensed under the GPL-3.0 License - see the [LICENSE](LICENSE) file for details.
|