paramflow 0.2.1__tar.gz → 0.2.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (28) hide show
  1. paramflow-0.2.3/PKG-INFO +147 -0
  2. paramflow-0.2.3/README.md +129 -0
  3. paramflow-0.2.3/paramflow/__pycache__/params.cpython-312.pyc +0 -0
  4. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/params.py +2 -1
  5. paramflow-0.2.3/paramflow.egg-info/PKG-INFO +147 -0
  6. {paramflow-0.2.1 → paramflow-0.2.3}/setup.py +1 -1
  7. paramflow-0.2.1/PKG-INFO +0 -147
  8. paramflow-0.2.1/README.md +0 -129
  9. paramflow-0.2.1/paramflow/__pycache__/params.cpython-312.pyc +0 -0
  10. paramflow-0.2.1/paramflow.egg-info/PKG-INFO +0 -147
  11. {paramflow-0.2.1 → paramflow-0.2.3}/LICENSE +0 -0
  12. {paramflow-0.2.1 → paramflow-0.2.3}/MANIFEST.in +0 -0
  13. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/__init__.py +0 -0
  14. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/__pycache__/__init__.cpython-312.pyc +0 -0
  15. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/__pycache__/convert.cpython-312.pyc +0 -0
  16. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/__pycache__/frozen.cpython-312.pyc +0 -0
  17. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/__pycache__/frozen_test.cpython-312-pytest-8.3.4.pyc +0 -0
  18. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/__pycache__/params_test.cpython-312-pytest-8.3.4.pyc +0 -0
  19. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/__pycache__/parser.cpython-312.pyc +0 -0
  20. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/convert.py +0 -0
  21. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/frozen.py +0 -0
  22. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow/parser.py +0 -0
  23. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow.egg-info/SOURCES.txt +0 -0
  24. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow.egg-info/dependency_links.txt +0 -0
  25. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow.egg-info/requires.txt +0 -0
  26. {paramflow-0.2.1 → paramflow-0.2.3}/paramflow.egg-info/top_level.txt +0 -0
  27. {paramflow-0.2.1 → paramflow-0.2.3}/pyproject.toml +0 -0
  28. {paramflow-0.2.1 → paramflow-0.2.3}/setup.cfg +0 -0
@@ -0,0 +1,147 @@
1
+ Metadata-Version: 2.2
2
+ Name: paramflow
3
+ Version: 0.2.3
4
+ Home-page: https://github.com/mduszyk/paramflow
5
+ Classifier: Programming Language :: Python :: 3
6
+ Classifier: License :: OSI Approved :: MIT License
7
+ Description-Content-Type: text/markdown
8
+ License-File: LICENSE
9
+ Requires-Dist: pyyaml
10
+ Provides-Extra: dotenv
11
+ Requires-Dist: python-dotenv; extra == "dotenv"
12
+ Dynamic: classifier
13
+ Dynamic: description
14
+ Dynamic: description-content-type
15
+ Dynamic: home-page
16
+ Dynamic: provides-extra
17
+ Dynamic: requires-dist
18
+
19
+ # paramflow
20
+ `paramflow` is a flexible and user-friendly parameter and configuration management library designed for
21
+ machine learning workflows and applications requiring profiles and layered parameters. It enables seamless
22
+ merging of parameters from multiple sources, auto-generates a command-line argument parser,
23
+ and allows for easy parameter overrides.
24
+
25
+ ## Features
26
+ - **Layered configuration**: Merge parameters from files, environment variables, and command-line arguments.
27
+ - **Immutable dictionary**: Provides a read-only dictionary with attribute-style access.
28
+ - **Profile support**: Manage multiple sets of parameters with profile-based layering.
29
+ - **Layered meta-parameters**: `paramflow` configures itself using a layered approach.
30
+ - **Automatic type conversion**: Converts types during merging based on target parameter types.
31
+ - **Command-line argument parsing**: Automatically generates an `argparse` parser from parameter definitions.
32
+ - **Nested Configuration**: Allows for nested configuration and merging.
33
+
34
+ ## Installation
35
+ ```sh
36
+ pip install paramflow
37
+ ```
38
+ Install with `.env` support:
39
+ ```sh
40
+ pip install "paramflow[dotenv]"
41
+ ```
42
+
43
+ ## Basic Usage
44
+ ### Example Configuration File (`params.toml`)
45
+ ```toml
46
+ [default]
47
+ learning_rate = 0.001
48
+ batch_size = 64
49
+ ```
50
+
51
+ ### Loading Parameters in Python (`app.py`)
52
+ ```python
53
+ import paramflow as pf
54
+
55
+ params = pf.load('params.toml')
56
+ print(params.learning_rate) # 0.001
57
+ ```
58
+
59
+ ### Generating Command-line Help
60
+ Running the script with `--help` displays both meta-parameters and parameters:
61
+ ```sh
62
+ python app.py --help
63
+ ```
64
+
65
+ ## Meta-Parameter Layering
66
+ Meta-parameters control how `paramflow.load` reads its own configuration. Layering order:
67
+ 1. `paramflow.load` arguments
68
+ 2. Environment variables (default prefix: `P_`)
69
+ 3. Command-line arguments (`argparse`)
70
+
71
+ ### Activating Profiles
72
+ Via command-line:
73
+ ```sh
74
+ python print_params.py --profile dqn-adam
75
+ ```
76
+ Via environment variable:
77
+ ```sh
78
+ P_PROFILE=dqn-adam python print_params.py
79
+ ```
80
+
81
+ ## Parameter Layering
82
+ Parameters are merged from multiple sources in the following order:
83
+ 1. Configuration files (`.toml`, `.yaml`, `.ini`, `.json`)
84
+ 2. `.env` file (if enabled)
85
+ 3. Environment variables (default prefix: `P_`)
86
+ 4. Command-line arguments (`argparse`)
87
+
88
+ ### Customizing Layering Order
89
+ You can specify the order explicitly:
90
+ ```python
91
+ params = pf.load('params.toml', 'env', '.env', 'args')
92
+ ```
93
+
94
+ ### Overriding Parameters
95
+ Command-line arguments override other sources:
96
+ ```sh
97
+ python print_params.py --profile dqn-adam --learning_rate 0.0002
98
+ ```
99
+
100
+ ## Managing ML Hyperparameter Profiles
101
+ ### Example Configuration (`params.toml`)
102
+ ```toml
103
+ [default]
104
+ learning_rate = 0.00025
105
+ batch_size = 32
106
+ optimizer_class = 'torch.optim.RMSprop'
107
+ optimizer_kwargs = { momentum = 0.95 }
108
+ random_seed = 13
109
+
110
+ [adam]
111
+ learning_rate = 1e-4
112
+ optimizer_class = 'torch.optim.Adam'
113
+ optimizer_kwargs = {}
114
+ ```
115
+
116
+ ### Activating a Profile
117
+ ```sh
118
+ python app.py --profile adam
119
+ ```
120
+ This overrides:
121
+ - `learning_rate` → `1e-4`
122
+ - `optimizer_class` → `torch.optim.Adam`
123
+ - `optimizer_kwargs` → `{}`
124
+
125
+ ## Managing Development Stages
126
+ Profiles can be used to manage configurations for different environments.
127
+
128
+ ### Example Configuration (`params.toml`)
129
+ ```toml
130
+ [default]
131
+ debug = true
132
+ database_url = "mysql://localhost:3306/myapp"
133
+
134
+ [dev]
135
+ database_url = "mysql://dev:3306/myapp"
136
+
137
+ [prod]
138
+ debug = false
139
+ database_url = "mysql://prod:3306/myapp"
140
+ ```
141
+
142
+ ### Activating a Profile
143
+ ```sh
144
+ export P_PROFILE=dev
145
+ python app.py
146
+ ```
147
+
@@ -0,0 +1,129 @@
1
+ # paramflow
2
+ `paramflow` is a flexible and user-friendly parameter and configuration management library designed for
3
+ machine learning workflows and applications requiring profiles and layered parameters. It enables seamless
4
+ merging of parameters from multiple sources, auto-generates a command-line argument parser,
5
+ and allows for easy parameter overrides.
6
+
7
+ ## Features
8
+ - **Layered configuration**: Merge parameters from files, environment variables, and command-line arguments.
9
+ - **Immutable dictionary**: Provides a read-only dictionary with attribute-style access.
10
+ - **Profile support**: Manage multiple sets of parameters with profile-based layering.
11
+ - **Layered meta-parameters**: `paramflow` configures itself using a layered approach.
12
+ - **Automatic type conversion**: Converts types during merging based on target parameter types.
13
+ - **Command-line argument parsing**: Automatically generates an `argparse` parser from parameter definitions.
14
+ - **Nested Configuration**: Allows for nested configuration and merging.
15
+
16
+ ## Installation
17
+ ```sh
18
+ pip install paramflow
19
+ ```
20
+ Install with `.env` support:
21
+ ```sh
22
+ pip install "paramflow[dotenv]"
23
+ ```
24
+
25
+ ## Basic Usage
26
+ ### Example Configuration File (`params.toml`)
27
+ ```toml
28
+ [default]
29
+ learning_rate = 0.001
30
+ batch_size = 64
31
+ ```
32
+
33
+ ### Loading Parameters in Python (`app.py`)
34
+ ```python
35
+ import paramflow as pf
36
+
37
+ params = pf.load('params.toml')
38
+ print(params.learning_rate) # 0.001
39
+ ```
40
+
41
+ ### Generating Command-line Help
42
+ Running the script with `--help` displays both meta-parameters and parameters:
43
+ ```sh
44
+ python app.py --help
45
+ ```
46
+
47
+ ## Meta-Parameter Layering
48
+ Meta-parameters control how `paramflow.load` reads its own configuration. Layering order:
49
+ 1. `paramflow.load` arguments
50
+ 2. Environment variables (default prefix: `P_`)
51
+ 3. Command-line arguments (`argparse`)
52
+
53
+ ### Activating Profiles
54
+ Via command-line:
55
+ ```sh
56
+ python print_params.py --profile dqn-adam
57
+ ```
58
+ Via environment variable:
59
+ ```sh
60
+ P_PROFILE=dqn-adam python print_params.py
61
+ ```
62
+
63
+ ## Parameter Layering
64
+ Parameters are merged from multiple sources in the following order:
65
+ 1. Configuration files (`.toml`, `.yaml`, `.ini`, `.json`)
66
+ 2. `.env` file (if enabled)
67
+ 3. Environment variables (default prefix: `P_`)
68
+ 4. Command-line arguments (`argparse`)
69
+
70
+ ### Customizing Layering Order
71
+ You can specify the order explicitly:
72
+ ```python
73
+ params = pf.load('params.toml', 'env', '.env', 'args')
74
+ ```
75
+
76
+ ### Overriding Parameters
77
+ Command-line arguments override other sources:
78
+ ```sh
79
+ python print_params.py --profile dqn-adam --learning_rate 0.0002
80
+ ```
81
+
82
+ ## Managing ML Hyperparameter Profiles
83
+ ### Example Configuration (`params.toml`)
84
+ ```toml
85
+ [default]
86
+ learning_rate = 0.00025
87
+ batch_size = 32
88
+ optimizer_class = 'torch.optim.RMSprop'
89
+ optimizer_kwargs = { momentum = 0.95 }
90
+ random_seed = 13
91
+
92
+ [adam]
93
+ learning_rate = 1e-4
94
+ optimizer_class = 'torch.optim.Adam'
95
+ optimizer_kwargs = {}
96
+ ```
97
+
98
+ ### Activating a Profile
99
+ ```sh
100
+ python app.py --profile adam
101
+ ```
102
+ This overrides:
103
+ - `learning_rate` → `1e-4`
104
+ - `optimizer_class` → `torch.optim.Adam`
105
+ - `optimizer_kwargs` → `{}`
106
+
107
+ ## Managing Development Stages
108
+ Profiles can be used to manage configurations for different environments.
109
+
110
+ ### Example Configuration (`params.toml`)
111
+ ```toml
112
+ [default]
113
+ debug = true
114
+ database_url = "mysql://localhost:3306/myapp"
115
+
116
+ [dev]
117
+ database_url = "mysql://dev:3306/myapp"
118
+
119
+ [prod]
120
+ debug = false
121
+ database_url = "mysql://prod:3306/myapp"
122
+ ```
123
+
124
+ ### Activating a Profile
125
+ ```sh
126
+ export P_PROFILE=dev
127
+ python app.py
128
+ ```
129
+
@@ -42,6 +42,7 @@ def load(*sources: Tuple[str, ...],
42
42
  'profile_key': profile_key,
43
43
  'default_profile': default_profile,
44
44
  profile_key: profile,
45
+ '__source__': ['pf.load'],
45
46
  }
46
47
  meta_env_parser = EnvParser(ENV_PREFIX, DEFAULT_PROFILE)
47
48
  meta_args_parser = ArgsParser(ARGS_PREFIX, DEFAULT_PROFILE, no_exit=True, descr='Meta-parameters')
@@ -50,7 +51,7 @@ def load(*sources: Tuple[str, ...],
50
51
  meta = freeze(meta)
51
52
 
52
53
  if meta.sources is None or len(meta.sources) == 0:
53
- sys.exit('source meta param is missing')
54
+ sys.exit('sources meta param is missing')
54
55
 
55
56
  sources = list(meta.sources)
56
57
 
@@ -0,0 +1,147 @@
1
+ Metadata-Version: 2.2
2
+ Name: paramflow
3
+ Version: 0.2.3
4
+ Home-page: https://github.com/mduszyk/paramflow
5
+ Classifier: Programming Language :: Python :: 3
6
+ Classifier: License :: OSI Approved :: MIT License
7
+ Description-Content-Type: text/markdown
8
+ License-File: LICENSE
9
+ Requires-Dist: pyyaml
10
+ Provides-Extra: dotenv
11
+ Requires-Dist: python-dotenv; extra == "dotenv"
12
+ Dynamic: classifier
13
+ Dynamic: description
14
+ Dynamic: description-content-type
15
+ Dynamic: home-page
16
+ Dynamic: provides-extra
17
+ Dynamic: requires-dist
18
+
19
+ # paramflow
20
+ `paramflow` is a flexible and user-friendly parameter and configuration management library designed for
21
+ machine learning workflows and applications requiring profiles and layered parameters. It enables seamless
22
+ merging of parameters from multiple sources, auto-generates a command-line argument parser,
23
+ and allows for easy parameter overrides.
24
+
25
+ ## Features
26
+ - **Layered configuration**: Merge parameters from files, environment variables, and command-line arguments.
27
+ - **Immutable dictionary**: Provides a read-only dictionary with attribute-style access.
28
+ - **Profile support**: Manage multiple sets of parameters with profile-based layering.
29
+ - **Layered meta-parameters**: `paramflow` configures itself using a layered approach.
30
+ - **Automatic type conversion**: Converts types during merging based on target parameter types.
31
+ - **Command-line argument parsing**: Automatically generates an `argparse` parser from parameter definitions.
32
+ - **Nested Configuration**: Allows for nested configuration and merging.
33
+
34
+ ## Installation
35
+ ```sh
36
+ pip install paramflow
37
+ ```
38
+ Install with `.env` support:
39
+ ```sh
40
+ pip install "paramflow[dotenv]"
41
+ ```
42
+
43
+ ## Basic Usage
44
+ ### Example Configuration File (`params.toml`)
45
+ ```toml
46
+ [default]
47
+ learning_rate = 0.001
48
+ batch_size = 64
49
+ ```
50
+
51
+ ### Loading Parameters in Python (`app.py`)
52
+ ```python
53
+ import paramflow as pf
54
+
55
+ params = pf.load('params.toml')
56
+ print(params.learning_rate) # 0.001
57
+ ```
58
+
59
+ ### Generating Command-line Help
60
+ Running the script with `--help` displays both meta-parameters and parameters:
61
+ ```sh
62
+ python app.py --help
63
+ ```
64
+
65
+ ## Meta-Parameter Layering
66
+ Meta-parameters control how `paramflow.load` reads its own configuration. Layering order:
67
+ 1. `paramflow.load` arguments
68
+ 2. Environment variables (default prefix: `P_`)
69
+ 3. Command-line arguments (`argparse`)
70
+
71
+ ### Activating Profiles
72
+ Via command-line:
73
+ ```sh
74
+ python print_params.py --profile dqn-adam
75
+ ```
76
+ Via environment variable:
77
+ ```sh
78
+ P_PROFILE=dqn-adam python print_params.py
79
+ ```
80
+
81
+ ## Parameter Layering
82
+ Parameters are merged from multiple sources in the following order:
83
+ 1. Configuration files (`.toml`, `.yaml`, `.ini`, `.json`)
84
+ 2. `.env` file (if enabled)
85
+ 3. Environment variables (default prefix: `P_`)
86
+ 4. Command-line arguments (`argparse`)
87
+
88
+ ### Customizing Layering Order
89
+ You can specify the order explicitly:
90
+ ```python
91
+ params = pf.load('params.toml', 'env', '.env', 'args')
92
+ ```
93
+
94
+ ### Overriding Parameters
95
+ Command-line arguments override other sources:
96
+ ```sh
97
+ python print_params.py --profile dqn-adam --learning_rate 0.0002
98
+ ```
99
+
100
+ ## Managing ML Hyperparameter Profiles
101
+ ### Example Configuration (`params.toml`)
102
+ ```toml
103
+ [default]
104
+ learning_rate = 0.00025
105
+ batch_size = 32
106
+ optimizer_class = 'torch.optim.RMSprop'
107
+ optimizer_kwargs = { momentum = 0.95 }
108
+ random_seed = 13
109
+
110
+ [adam]
111
+ learning_rate = 1e-4
112
+ optimizer_class = 'torch.optim.Adam'
113
+ optimizer_kwargs = {}
114
+ ```
115
+
116
+ ### Activating a Profile
117
+ ```sh
118
+ python app.py --profile adam
119
+ ```
120
+ This overrides:
121
+ - `learning_rate` → `1e-4`
122
+ - `optimizer_class` → `torch.optim.Adam`
123
+ - `optimizer_kwargs` → `{}`
124
+
125
+ ## Managing Development Stages
126
+ Profiles can be used to manage configurations for different environments.
127
+
128
+ ### Example Configuration (`params.toml`)
129
+ ```toml
130
+ [default]
131
+ debug = true
132
+ database_url = "mysql://localhost:3306/myapp"
133
+
134
+ [dev]
135
+ database_url = "mysql://dev:3306/myapp"
136
+
137
+ [prod]
138
+ debug = false
139
+ database_url = "mysql://prod:3306/myapp"
140
+ ```
141
+
142
+ ### Activating a Profile
143
+ ```sh
144
+ export P_PROFILE=dev
145
+ python app.py
146
+ ```
147
+
@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
2
2
 
3
3
  setup(
4
4
  name='paramflow',
5
- version='0.2.1',
5
+ version='0.2.3',
6
6
  packages=find_packages(),
7
7
  install_requires=[
8
8
  "pyyaml",
paramflow-0.2.1/PKG-INFO DELETED
@@ -1,147 +0,0 @@
1
- Metadata-Version: 2.2
2
- Name: paramflow
3
- Version: 0.2.1
4
- Home-page: https://github.com/mduszyk/paramflow
5
- Classifier: Programming Language :: Python :: 3
6
- Classifier: License :: OSI Approved :: MIT License
7
- Description-Content-Type: text/markdown
8
- License-File: LICENSE
9
- Requires-Dist: pyyaml
10
- Provides-Extra: dotenv
11
- Requires-Dist: python-dotenv; extra == "dotenv"
12
- Dynamic: classifier
13
- Dynamic: description
14
- Dynamic: description-content-type
15
- Dynamic: home-page
16
- Dynamic: provides-extra
17
- Dynamic: requires-dist
18
-
19
- # paramflow
20
- A parameter and configuration management library motivated by training machine learning models
21
- and managing configuration for applications that require profiles and layered parameters.
22
- ```paramflow``` is designed for flexibility and ease of use, enabling seamless parameter merging
23
- from multiple sources. It also auto-generates a command-line argument parser and allows for
24
- easy parameter overrides.
25
-
26
- ## Features
27
- - **Layered configuration**: Merge parameters from files, environment variables, and command-line arguments.
28
- - **Immutable dictionary**: Provides a read-only dictionary with attribute-style access.
29
- - **Profile support**: Manage multiple sets of parameters. Layer the chosen profile on top of the default profile.
30
- - **Layered meta-parameters**: ```paramflow``` loads its own configuration using layered approach.
31
- - **Convert types**: Convert types during merging using target parameters as a reference for type conversions.
32
- - **Generate argument parser**: Use parameters defined in files as a reference for generating ```argparse``` parser.
33
-
34
- ## Usage
35
- Install:
36
- ```shell
37
- pip install paramflow
38
- ```
39
- Install with ```.env``` support:
40
- ```shell
41
- pip install "paramflow[dotenv]"
42
- ```
43
-
44
- ```params.toml```
45
- ```toml
46
- [default]
47
- learning_rate = 0.001
48
- batch_size = 64
49
- ```
50
-
51
- ```app.py```
52
- ```python
53
- import paramflow as pf
54
- params = pf.load('params.toml')
55
- print(params.learning_rate)
56
- ```
57
-
58
- ```shell
59
- python app.py --help
60
- usage: app.py [-h] [--learning_rate LEARNING_RATE] [--batch_size BATCH_SIZE]
61
-
62
- options:
63
- -h, --help show this help message and exit
64
- --learning_rate LEARNING_RATE
65
- learning_rate = 0.001
66
- --batch_size BATCH_SIZE
67
- batch_size = 64
68
- ```
69
-
70
- ## Meta-parameter Layering
71
- Meta-parameter layering controls how ```paramflow.load``` reads its own configuration.
72
-
73
- Layering order:
74
- 1. ```paramflow.load``` arguments.
75
- 2. Environment variables (default prefix ```P_```).
76
- 3. Command-line arguments (via ```argparse```).
77
-
78
- Activate profile using command-line arguments:
79
- ```shell
80
- python print_params.py --profile dqn-adam
81
- ```
82
- Activate profile using environment variable:
83
- ```shell
84
- P_PROFILE=dqn-adam python print_params.py
85
- ```
86
-
87
- ## Parameter Layering
88
- Parameter layering merges parameters from multiple sources.
89
-
90
- Layering order:
91
- 1. Configuration files (```.toml```, ```.yaml```, ```.ini```, ```.json```).
92
- 2. ```.env``` file.
93
- 3. Environment variables (default prefix ```P_```).
94
- 4. Command-line arguments (via ```argparse```).
95
-
96
- Layering order can be customized via ```source``` argument to ```param.flow```.
97
- ```python
98
- params = pf.load(['params.toml', 'env', '.env', 'args'])
99
- ```
100
-
101
- Overwrite parameter value:
102
- ```shell
103
- python print_params.py --profile dqn-adam --learning_rate 0.0002
104
- ```
105
-
106
- ## ML hyper-parameters profiles
107
- ```params.toml```
108
- ```toml
109
- [default]
110
- learning_rate = 0.00025
111
- batch_size = 32
112
- optimizer_class = 'torch.optim.RMSprop'
113
- optimizer_kwargs = { momentum = 0.95 }
114
- random_seed = 13
115
-
116
- [adam]
117
- learning_rate = 1e-4
118
- optimizer_class = 'torch.optim.Adam'
119
- optimizer_kwargs = {}
120
- ```
121
- Activating adam profile
122
- ```shell
123
- python app.py --profile adam
124
- ```
125
- will result in overwriting default learning rate with ```1e-4```, default optimizer class with ```torch.optim.Adam```
126
- and default optimizer arguments with and empty dict.
127
-
128
- ## Development stages profiles
129
- Profiles can be used to manage software development stages.
130
- ```params.toml```:
131
- ```toml
132
- [default]
133
- debug = true
134
- database_url = "mysql://localhost:3306/myapp"
135
-
136
- [dev]
137
- database_url = "mysql://dev:3306/myapp"
138
-
139
- [prod]
140
- debug = false
141
- database_url = "mysql://prod:3306/myapp"
142
- ```
143
- Activate prod profile:
144
- ```shell
145
- export P_PROFILE=dev
146
- python app.py
147
- ```
paramflow-0.2.1/README.md DELETED
@@ -1,129 +0,0 @@
1
- # paramflow
2
- A parameter and configuration management library motivated by training machine learning models
3
- and managing configuration for applications that require profiles and layered parameters.
4
- ```paramflow``` is designed for flexibility and ease of use, enabling seamless parameter merging
5
- from multiple sources. It also auto-generates a command-line argument parser and allows for
6
- easy parameter overrides.
7
-
8
- ## Features
9
- - **Layered configuration**: Merge parameters from files, environment variables, and command-line arguments.
10
- - **Immutable dictionary**: Provides a read-only dictionary with attribute-style access.
11
- - **Profile support**: Manage multiple sets of parameters. Layer the chosen profile on top of the default profile.
12
- - **Layered meta-parameters**: ```paramflow``` loads its own configuration using layered approach.
13
- - **Convert types**: Convert types during merging using target parameters as a reference for type conversions.
14
- - **Generate argument parser**: Use parameters defined in files as a reference for generating ```argparse``` parser.
15
-
16
- ## Usage
17
- Install:
18
- ```shell
19
- pip install paramflow
20
- ```
21
- Install with ```.env``` support:
22
- ```shell
23
- pip install "paramflow[dotenv]"
24
- ```
25
-
26
- ```params.toml```
27
- ```toml
28
- [default]
29
- learning_rate = 0.001
30
- batch_size = 64
31
- ```
32
-
33
- ```app.py```
34
- ```python
35
- import paramflow as pf
36
- params = pf.load('params.toml')
37
- print(params.learning_rate)
38
- ```
39
-
40
- ```shell
41
- python app.py --help
42
- usage: app.py [-h] [--learning_rate LEARNING_RATE] [--batch_size BATCH_SIZE]
43
-
44
- options:
45
- -h, --help show this help message and exit
46
- --learning_rate LEARNING_RATE
47
- learning_rate = 0.001
48
- --batch_size BATCH_SIZE
49
- batch_size = 64
50
- ```
51
-
52
- ## Meta-parameter Layering
53
- Meta-parameter layering controls how ```paramflow.load``` reads its own configuration.
54
-
55
- Layering order:
56
- 1. ```paramflow.load``` arguments.
57
- 2. Environment variables (default prefix ```P_```).
58
- 3. Command-line arguments (via ```argparse```).
59
-
60
- Activate profile using command-line arguments:
61
- ```shell
62
- python print_params.py --profile dqn-adam
63
- ```
64
- Activate profile using environment variable:
65
- ```shell
66
- P_PROFILE=dqn-adam python print_params.py
67
- ```
68
-
69
- ## Parameter Layering
70
- Parameter layering merges parameters from multiple sources.
71
-
72
- Layering order:
73
- 1. Configuration files (```.toml```, ```.yaml```, ```.ini```, ```.json```).
74
- 2. ```.env``` file.
75
- 3. Environment variables (default prefix ```P_```).
76
- 4. Command-line arguments (via ```argparse```).
77
-
78
- Layering order can be customized via ```source``` argument to ```param.flow```.
79
- ```python
80
- params = pf.load(['params.toml', 'env', '.env', 'args'])
81
- ```
82
-
83
- Overwrite parameter value:
84
- ```shell
85
- python print_params.py --profile dqn-adam --learning_rate 0.0002
86
- ```
87
-
88
- ## ML hyper-parameters profiles
89
- ```params.toml```
90
- ```toml
91
- [default]
92
- learning_rate = 0.00025
93
- batch_size = 32
94
- optimizer_class = 'torch.optim.RMSprop'
95
- optimizer_kwargs = { momentum = 0.95 }
96
- random_seed = 13
97
-
98
- [adam]
99
- learning_rate = 1e-4
100
- optimizer_class = 'torch.optim.Adam'
101
- optimizer_kwargs = {}
102
- ```
103
- Activating adam profile
104
- ```shell
105
- python app.py --profile adam
106
- ```
107
- will result in overwriting default learning rate with ```1e-4```, default optimizer class with ```torch.optim.Adam```
108
- and default optimizer arguments with and empty dict.
109
-
110
- ## Development stages profiles
111
- Profiles can be used to manage software development stages.
112
- ```params.toml```:
113
- ```toml
114
- [default]
115
- debug = true
116
- database_url = "mysql://localhost:3306/myapp"
117
-
118
- [dev]
119
- database_url = "mysql://dev:3306/myapp"
120
-
121
- [prod]
122
- debug = false
123
- database_url = "mysql://prod:3306/myapp"
124
- ```
125
- Activate prod profile:
126
- ```shell
127
- export P_PROFILE=dev
128
- python app.py
129
- ```
@@ -1,147 +0,0 @@
1
- Metadata-Version: 2.2
2
- Name: paramflow
3
- Version: 0.2.1
4
- Home-page: https://github.com/mduszyk/paramflow
5
- Classifier: Programming Language :: Python :: 3
6
- Classifier: License :: OSI Approved :: MIT License
7
- Description-Content-Type: text/markdown
8
- License-File: LICENSE
9
- Requires-Dist: pyyaml
10
- Provides-Extra: dotenv
11
- Requires-Dist: python-dotenv; extra == "dotenv"
12
- Dynamic: classifier
13
- Dynamic: description
14
- Dynamic: description-content-type
15
- Dynamic: home-page
16
- Dynamic: provides-extra
17
- Dynamic: requires-dist
18
-
19
- # paramflow
20
- A parameter and configuration management library motivated by training machine learning models
21
- and managing configuration for applications that require profiles and layered parameters.
22
- ```paramflow``` is designed for flexibility and ease of use, enabling seamless parameter merging
23
- from multiple sources. It also auto-generates a command-line argument parser and allows for
24
- easy parameter overrides.
25
-
26
- ## Features
27
- - **Layered configuration**: Merge parameters from files, environment variables, and command-line arguments.
28
- - **Immutable dictionary**: Provides a read-only dictionary with attribute-style access.
29
- - **Profile support**: Manage multiple sets of parameters. Layer the chosen profile on top of the default profile.
30
- - **Layered meta-parameters**: ```paramflow``` loads its own configuration using layered approach.
31
- - **Convert types**: Convert types during merging using target parameters as a reference for type conversions.
32
- - **Generate argument parser**: Use parameters defined in files as a reference for generating ```argparse``` parser.
33
-
34
- ## Usage
35
- Install:
36
- ```shell
37
- pip install paramflow
38
- ```
39
- Install with ```.env``` support:
40
- ```shell
41
- pip install "paramflow[dotenv]"
42
- ```
43
-
44
- ```params.toml```
45
- ```toml
46
- [default]
47
- learning_rate = 0.001
48
- batch_size = 64
49
- ```
50
-
51
- ```app.py```
52
- ```python
53
- import paramflow as pf
54
- params = pf.load('params.toml')
55
- print(params.learning_rate)
56
- ```
57
-
58
- ```shell
59
- python app.py --help
60
- usage: app.py [-h] [--learning_rate LEARNING_RATE] [--batch_size BATCH_SIZE]
61
-
62
- options:
63
- -h, --help show this help message and exit
64
- --learning_rate LEARNING_RATE
65
- learning_rate = 0.001
66
- --batch_size BATCH_SIZE
67
- batch_size = 64
68
- ```
69
-
70
- ## Meta-parameter Layering
71
- Meta-parameter layering controls how ```paramflow.load``` reads its own configuration.
72
-
73
- Layering order:
74
- 1. ```paramflow.load``` arguments.
75
- 2. Environment variables (default prefix ```P_```).
76
- 3. Command-line arguments (via ```argparse```).
77
-
78
- Activate profile using command-line arguments:
79
- ```shell
80
- python print_params.py --profile dqn-adam
81
- ```
82
- Activate profile using environment variable:
83
- ```shell
84
- P_PROFILE=dqn-adam python print_params.py
85
- ```
86
-
87
- ## Parameter Layering
88
- Parameter layering merges parameters from multiple sources.
89
-
90
- Layering order:
91
- 1. Configuration files (```.toml```, ```.yaml```, ```.ini```, ```.json```).
92
- 2. ```.env``` file.
93
- 3. Environment variables (default prefix ```P_```).
94
- 4. Command-line arguments (via ```argparse```).
95
-
96
- Layering order can be customized via ```source``` argument to ```param.flow```.
97
- ```python
98
- params = pf.load(['params.toml', 'env', '.env', 'args'])
99
- ```
100
-
101
- Overwrite parameter value:
102
- ```shell
103
- python print_params.py --profile dqn-adam --learning_rate 0.0002
104
- ```
105
-
106
- ## ML hyper-parameters profiles
107
- ```params.toml```
108
- ```toml
109
- [default]
110
- learning_rate = 0.00025
111
- batch_size = 32
112
- optimizer_class = 'torch.optim.RMSprop'
113
- optimizer_kwargs = { momentum = 0.95 }
114
- random_seed = 13
115
-
116
- [adam]
117
- learning_rate = 1e-4
118
- optimizer_class = 'torch.optim.Adam'
119
- optimizer_kwargs = {}
120
- ```
121
- Activating adam profile
122
- ```shell
123
- python app.py --profile adam
124
- ```
125
- will result in overwriting default learning rate with ```1e-4```, default optimizer class with ```torch.optim.Adam```
126
- and default optimizer arguments with and empty dict.
127
-
128
- ## Development stages profiles
129
- Profiles can be used to manage software development stages.
130
- ```params.toml```:
131
- ```toml
132
- [default]
133
- debug = true
134
- database_url = "mysql://localhost:3306/myapp"
135
-
136
- [dev]
137
- database_url = "mysql://dev:3306/myapp"
138
-
139
- [prod]
140
- debug = false
141
- database_url = "mysql://prod:3306/myapp"
142
- ```
143
- Activate prod profile:
144
- ```shell
145
- export P_PROFILE=dev
146
- python app.py
147
- ```
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes