plexosdb 1.3.2__tar.gz → 1.3.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- plexosdb-1.3.4/PKG-INFO +147 -0
- plexosdb-1.3.4/README.md +103 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/pyproject.toml +18 -10
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/db.py +86 -50
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/utils.py +43 -26
- plexosdb-1.3.2/PKG-INFO +0 -83
- plexosdb-1.3.2/README.md +0 -39
- {plexosdb-1.3.2 → plexosdb-1.3.4}/LICENSE.txt +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/__init__.py +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/__main__.py +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/checks.py +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/db_manager.py +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/enums.py +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/exceptions.py +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/py.typed +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/queries/object_properties.sql +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/queries/object_query.sql +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/queries/property.sql +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/queries/property_query.sql +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/queries/simple_object_query.sql +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/schema.sql +0 -0
- {plexosdb-1.3.2 → plexosdb-1.3.4}/src/plexosdb/xml_handler.py +0 -0
plexosdb-1.3.4/PKG-INFO
ADDED
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
Metadata-Version: 2.3
|
|
2
|
+
Name: plexosdb
|
|
3
|
+
Version: 1.3.4
|
|
4
|
+
Summary: SQLite API for plexos XMLs
|
|
5
|
+
Keywords: PLEXOS,Database,SQLite
|
|
6
|
+
Author: Pedro Andres Sanchez Perez, Kodi Obika, mcllerena
|
|
7
|
+
Author-email: Pedro Andres Sanchez Perez <psanchez@nrel.gov>, Kodi Obika <kodi.obika@nrel.gov>, mcllerena <mcllerena@users.noreply.github.com>
|
|
8
|
+
License: BSD 3-Clause License
|
|
9
|
+
|
|
10
|
+
Copyright (c) 2024, Alliance for Sustainable Energy LLC, All rights reserved.
|
|
11
|
+
|
|
12
|
+
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
|
13
|
+
|
|
14
|
+
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
|
15
|
+
|
|
16
|
+
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
|
17
|
+
|
|
18
|
+
* Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
|
|
19
|
+
|
|
20
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
21
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
22
|
+
Classifier: Intended Audience :: Developers
|
|
23
|
+
Classifier: License :: OSI Approved :: BSD License
|
|
24
|
+
Classifier: Topic :: Software Development :: Build Tools
|
|
25
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
26
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
27
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
28
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
29
|
+
Classifier: Topic :: Scientific/Engineering
|
|
30
|
+
Classifier: Topic :: Scientific/Engineering :: Information Analysis
|
|
31
|
+
Classifier: Topic :: Scientific/Engineering :: Interface Engine/Protocol Translator
|
|
32
|
+
Classifier: Topic :: Software Development :: Build Tools
|
|
33
|
+
Classifier: Topic :: Database
|
|
34
|
+
Classifier: Typing :: Typed
|
|
35
|
+
Requires-Dist: loguru
|
|
36
|
+
Maintainer: Pedro Andres Sanchez Perez, Kodi Obika, mcllerena
|
|
37
|
+
Maintainer-email: Pedro Andres Sanchez Perez <psanchez@nrel.gov>, Kodi Obika <kodi.obika@nrel.gov>, mcllerena <mcllerena@users.noreply.github.com>
|
|
38
|
+
Requires-Python: >=3.11, <3.15
|
|
39
|
+
Project-URL: Changelog, https://github.com/NatLabRockies/plexosdb/blob/main/CHANGELOG.md
|
|
40
|
+
Project-URL: Documentation, https://natlabrockies.github.io/plexosdb/
|
|
41
|
+
Project-URL: Issues, https://github.com/NatLabRockies/plexosdb/issues
|
|
42
|
+
Project-URL: Source, https://github.com/NatLabRockies/plexosdb
|
|
43
|
+
Description-Content-Type: text/markdown
|
|
44
|
+
|
|
45
|
+
### plexosdb
|
|
46
|
+
|
|
47
|
+
> SQLite-backed Python API for reading, building, and writing PLEXOS XML models
|
|
48
|
+
>
|
|
49
|
+
> [](https://pypi.python.org/pypi/plexosdb)
|
|
50
|
+
> [](https://pypi.python.org/pypi/plexosdb)
|
|
51
|
+
> [](https://pypi.python.org/pypi/plexosdb)
|
|
52
|
+
> [](https://github.com/NatLabRockies/plexosdb/actions/workflows/CI.yaml)
|
|
53
|
+
> [](https://codecov.io/gh/NatLabRockies/plexosdb)
|
|
54
|
+
> [](https://github.com/astral-sh/ruff)
|
|
55
|
+
> [](https://natlabrockies.github.io/plexosdb/)
|
|
56
|
+
|
|
57
|
+
plexosdb converts PLEXOS XML input files into an in-memory SQLite database,
|
|
58
|
+
giving you a fast, typed Python API to query, create, and modify power system
|
|
59
|
+
models programmatically, then write them back to XML.
|
|
60
|
+
|
|
61
|
+
## Installation
|
|
62
|
+
|
|
63
|
+
```console
|
|
64
|
+
pip install plexosdb
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Or with [uv](https://docs.astral.sh/uv/):
|
|
68
|
+
|
|
69
|
+
```console
|
|
70
|
+
uv add plexosdb
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
**Python version support:** 3.11, 3.12, 3.13, 3.14
|
|
74
|
+
|
|
75
|
+
## Quick Start
|
|
76
|
+
|
|
77
|
+
```python
|
|
78
|
+
from plexosdb import PlexosDB, ClassEnum, CollectionEnum
|
|
79
|
+
|
|
80
|
+
# Load a PLEXOS XML file into an in-memory SQLite database
|
|
81
|
+
db = PlexosDB.from_xml("model.xml")
|
|
82
|
+
|
|
83
|
+
# Query existing objects
|
|
84
|
+
generators = db.get_object_names(ClassEnum.Generator)
|
|
85
|
+
|
|
86
|
+
# Add new objects
|
|
87
|
+
db.add_object(ClassEnum.Generator, name="SolarPV_01", category="Renewables")
|
|
88
|
+
db.add_object(ClassEnum.Node, name="Bus_1")
|
|
89
|
+
|
|
90
|
+
# Create memberships between objects
|
|
91
|
+
db.add_membership(
|
|
92
|
+
CollectionEnum.GeneratorNodes,
|
|
93
|
+
parent_class=ClassEnum.Generator,
|
|
94
|
+
parent_name="SolarPV_01",
|
|
95
|
+
child_class=ClassEnum.Node,
|
|
96
|
+
child_name="Bus_1",
|
|
97
|
+
)
|
|
98
|
+
|
|
99
|
+
# Export the modified model back to XML
|
|
100
|
+
db.to_xml("modified_model.xml")
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## Documentation
|
|
104
|
+
|
|
105
|
+
Full documentation is available at
|
|
106
|
+
[natlabrockies.github.io/plexosdb](https://natlabrockies.github.io/plexosdb/).
|
|
107
|
+
|
|
108
|
+
## Developer Setup
|
|
109
|
+
|
|
110
|
+
plexosdb uses [uv](https://docs.astral.sh/uv/) for dependency management.
|
|
111
|
+
|
|
112
|
+
```console
|
|
113
|
+
git clone https://github.com/NatLabRockies/plexosdb.git
|
|
114
|
+
cd plexosdb
|
|
115
|
+
uv sync --all-groups
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
Install the git hooks so your code is checked before making commits:
|
|
119
|
+
|
|
120
|
+
```console
|
|
121
|
+
uv run prek install
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Run the test suite:
|
|
125
|
+
|
|
126
|
+
```console
|
|
127
|
+
uv run pytest
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
## License
|
|
131
|
+
|
|
132
|
+
This software is released under a BSD-3-Clause
|
|
133
|
+
[License](https://github.com/NatLabRockies/plexosdb/blob/main/LICENSE.txt).
|
|
134
|
+
|
|
135
|
+
This software was developed under software record SWR-24-90 at the National
|
|
136
|
+
Renewable Energy Laboratory ([NREL](https://www.nrel.gov)).
|
|
137
|
+
|
|
138
|
+
## Disclaimer
|
|
139
|
+
|
|
140
|
+
PLEXOS is a registered trademark of Energy Exemplar Pty Ltd. Energy Exemplar Pty
|
|
141
|
+
Ltd. has no affiliation to or participation in this software. Reference herein
|
|
142
|
+
to any specific commercial products, process, or service by trade name,
|
|
143
|
+
trademark, manufacturer, or otherwise, does not necessarily constitute or imply
|
|
144
|
+
its endorsement, recommendation, or favoring by the United States Government or
|
|
145
|
+
Alliance for Sustainable Energy, LLC ("Alliance"). The views and opinions of
|
|
146
|
+
authors expressed in the available or referenced documents do not necessarily
|
|
147
|
+
state or reflect those of the United States Government or Alliance.
|
plexosdb-1.3.4/README.md
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
### plexosdb
|
|
2
|
+
|
|
3
|
+
> SQLite-backed Python API for reading, building, and writing PLEXOS XML models
|
|
4
|
+
>
|
|
5
|
+
> [](https://pypi.python.org/pypi/plexosdb)
|
|
6
|
+
> [](https://pypi.python.org/pypi/plexosdb)
|
|
7
|
+
> [](https://pypi.python.org/pypi/plexosdb)
|
|
8
|
+
> [](https://github.com/NatLabRockies/plexosdb/actions/workflows/CI.yaml)
|
|
9
|
+
> [](https://codecov.io/gh/NatLabRockies/plexosdb)
|
|
10
|
+
> [](https://github.com/astral-sh/ruff)
|
|
11
|
+
> [](https://natlabrockies.github.io/plexosdb/)
|
|
12
|
+
|
|
13
|
+
plexosdb converts PLEXOS XML input files into an in-memory SQLite database,
|
|
14
|
+
giving you a fast, typed Python API to query, create, and modify power system
|
|
15
|
+
models programmatically, then write them back to XML.
|
|
16
|
+
|
|
17
|
+
## Installation
|
|
18
|
+
|
|
19
|
+
```console
|
|
20
|
+
pip install plexosdb
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Or with [uv](https://docs.astral.sh/uv/):
|
|
24
|
+
|
|
25
|
+
```console
|
|
26
|
+
uv add plexosdb
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
**Python version support:** 3.11, 3.12, 3.13, 3.14
|
|
30
|
+
|
|
31
|
+
## Quick Start
|
|
32
|
+
|
|
33
|
+
```python
|
|
34
|
+
from plexosdb import PlexosDB, ClassEnum, CollectionEnum
|
|
35
|
+
|
|
36
|
+
# Load a PLEXOS XML file into an in-memory SQLite database
|
|
37
|
+
db = PlexosDB.from_xml("model.xml")
|
|
38
|
+
|
|
39
|
+
# Query existing objects
|
|
40
|
+
generators = db.get_object_names(ClassEnum.Generator)
|
|
41
|
+
|
|
42
|
+
# Add new objects
|
|
43
|
+
db.add_object(ClassEnum.Generator, name="SolarPV_01", category="Renewables")
|
|
44
|
+
db.add_object(ClassEnum.Node, name="Bus_1")
|
|
45
|
+
|
|
46
|
+
# Create memberships between objects
|
|
47
|
+
db.add_membership(
|
|
48
|
+
CollectionEnum.GeneratorNodes,
|
|
49
|
+
parent_class=ClassEnum.Generator,
|
|
50
|
+
parent_name="SolarPV_01",
|
|
51
|
+
child_class=ClassEnum.Node,
|
|
52
|
+
child_name="Bus_1",
|
|
53
|
+
)
|
|
54
|
+
|
|
55
|
+
# Export the modified model back to XML
|
|
56
|
+
db.to_xml("modified_model.xml")
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Documentation
|
|
60
|
+
|
|
61
|
+
Full documentation is available at
|
|
62
|
+
[natlabrockies.github.io/plexosdb](https://natlabrockies.github.io/plexosdb/).
|
|
63
|
+
|
|
64
|
+
## Developer Setup
|
|
65
|
+
|
|
66
|
+
plexosdb uses [uv](https://docs.astral.sh/uv/) for dependency management.
|
|
67
|
+
|
|
68
|
+
```console
|
|
69
|
+
git clone https://github.com/NatLabRockies/plexosdb.git
|
|
70
|
+
cd plexosdb
|
|
71
|
+
uv sync --all-groups
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Install the git hooks so your code is checked before making commits:
|
|
75
|
+
|
|
76
|
+
```console
|
|
77
|
+
uv run prek install
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Run the test suite:
|
|
81
|
+
|
|
82
|
+
```console
|
|
83
|
+
uv run pytest
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
## License
|
|
87
|
+
|
|
88
|
+
This software is released under a BSD-3-Clause
|
|
89
|
+
[License](https://github.com/NatLabRockies/plexosdb/blob/main/LICENSE.txt).
|
|
90
|
+
|
|
91
|
+
This software was developed under software record SWR-24-90 at the National
|
|
92
|
+
Renewable Energy Laboratory ([NREL](https://www.nrel.gov)).
|
|
93
|
+
|
|
94
|
+
## Disclaimer
|
|
95
|
+
|
|
96
|
+
PLEXOS is a registered trademark of Energy Exemplar Pty Ltd. Energy Exemplar Pty
|
|
97
|
+
Ltd. has no affiliation to or participation in this software. Reference herein
|
|
98
|
+
to any specific commercial products, process, or service by trade name,
|
|
99
|
+
trademark, manufacturer, or otherwise, does not necessarily constitute or imply
|
|
100
|
+
its endorsement, recommendation, or favoring by the United States Government or
|
|
101
|
+
Alliance for Sustainable Energy, LLC ("Alliance"). The views and opinions of
|
|
102
|
+
authors expressed in the available or referenced documents do not necessarily
|
|
103
|
+
state or reflect those of the United States Government or Alliance.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[project]
|
|
2
2
|
name = "plexosdb"
|
|
3
|
-
version = "1.3.
|
|
3
|
+
version = "1.3.4"
|
|
4
4
|
readme = "README.md"
|
|
5
5
|
license = {file = "LICENSE.txt"}
|
|
6
6
|
keywords = ["PLEXOS", "Database", "SQLite"]
|
|
@@ -38,10 +38,10 @@ dependencies = [
|
|
|
38
38
|
]
|
|
39
39
|
|
|
40
40
|
[project.urls]
|
|
41
|
-
Documentation = "https://github.
|
|
42
|
-
Issues = "https://github.com/
|
|
43
|
-
Source = "https://github.com/
|
|
44
|
-
Changelog = "https://github.com/
|
|
41
|
+
Documentation = "https://natlabrockies.github.io/plexosdb/"
|
|
42
|
+
Issues = "https://github.com/NatLabRockies/plexosdb/issues"
|
|
43
|
+
Source = "https://github.com/NatLabRockies/plexosdb"
|
|
44
|
+
Changelog = "https://github.com/NatLabRockies/plexosdb/blob/main/CHANGELOG.md"
|
|
45
45
|
|
|
46
46
|
[build-system]
|
|
47
47
|
requires = ["uv_build>=0.8.22,<0.9.0"]
|
|
@@ -63,9 +63,10 @@ docs = [
|
|
|
63
63
|
]
|
|
64
64
|
dev = [
|
|
65
65
|
"ipython>=9.2.0",
|
|
66
|
-
"
|
|
67
|
-
"
|
|
66
|
+
"prek>=0.3.3",
|
|
67
|
+
"ty>=0.0.23",
|
|
68
68
|
"pytest>=8.3.5",
|
|
69
|
+
"pytest-benchmark>=5.1.0",
|
|
69
70
|
"pytest-coverage>=0.0",
|
|
70
71
|
"ruff>=0.11.5",
|
|
71
72
|
]
|
|
@@ -79,10 +80,16 @@ include-package-data = true
|
|
|
79
80
|
[tool.setuptools.packages.find]
|
|
80
81
|
where = ["src"]
|
|
81
82
|
|
|
82
|
-
[tool.
|
|
83
|
-
|
|
83
|
+
[tool.ty.src]
|
|
84
|
+
include = ["src/plexosdb"]
|
|
84
85
|
exclude = ["tests/"]
|
|
85
86
|
|
|
87
|
+
[tool.ty.environment]
|
|
88
|
+
python-version = "3.11"
|
|
89
|
+
|
|
90
|
+
[tool.ty.terminal]
|
|
91
|
+
output-format = "full"
|
|
92
|
+
|
|
86
93
|
[tool.ruff]
|
|
87
94
|
line-length = 110
|
|
88
95
|
target-version = "py311"
|
|
@@ -157,11 +164,12 @@ markers = [
|
|
|
157
164
|
"checks: Functions that check existence of database entities",
|
|
158
165
|
"getters: Functions that get data",
|
|
159
166
|
"adders: Functions that add data",
|
|
167
|
+
"benchmark: Performance benchmark tests",
|
|
160
168
|
"empty_database: Functions for test empty database",
|
|
161
169
|
"export: Functions that export the database.",
|
|
162
170
|
"listing: Functions that list elements of the database.",
|
|
163
171
|
"object_operations: Operations to objects in the database.",
|
|
164
|
-
"fast: Run test with only
|
|
172
|
+
"fast: Run test with only master_v10.0R2 XML for fast development testing"
|
|
165
173
|
]
|
|
166
174
|
|
|
167
175
|
[tool.coverage.run]
|
|
@@ -13,7 +13,6 @@ import warnings
|
|
|
13
13
|
|
|
14
14
|
from loguru import logger
|
|
15
15
|
|
|
16
|
-
from .checks import check_memberships_from_records
|
|
17
16
|
from .db_manager import SQLiteManager
|
|
18
17
|
from .enums import (
|
|
19
18
|
ClassEnum,
|
|
@@ -31,6 +30,7 @@ from .exceptions import (
|
|
|
31
30
|
)
|
|
32
31
|
from .utils import (
|
|
33
32
|
apply_scenario_tags,
|
|
33
|
+
batched,
|
|
34
34
|
create_membership_record,
|
|
35
35
|
get_system_object_name,
|
|
36
36
|
insert_property_texts,
|
|
@@ -549,7 +549,6 @@ class PlexosDB:
|
|
|
549
549
|
See Also
|
|
550
550
|
--------
|
|
551
551
|
add_membership : Add a single membership between two objects
|
|
552
|
-
check_memberships_from_records : Validate membership records format
|
|
553
552
|
create_membership_record : Helper to create membership record dictionaries
|
|
554
553
|
|
|
555
554
|
Examples
|
|
@@ -585,18 +584,54 @@ class PlexosDB:
|
|
|
585
584
|
>>> db.add_memberships_from_records(records)
|
|
586
585
|
True
|
|
587
586
|
"""
|
|
588
|
-
if not
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
587
|
+
if not records:
|
|
588
|
+
logger.debug("No membership records provided")
|
|
589
|
+
return True
|
|
590
|
+
|
|
591
|
+
if chunksize < 1:
|
|
592
|
+
msg = f"chunksize must be >= 1, received {chunksize}"
|
|
593
|
+
raise ValueError(msg)
|
|
594
|
+
|
|
592
595
|
query = f"""
|
|
593
596
|
INSERT INTO {Schema.Memberships.name}
|
|
594
597
|
(parent_class_id,parent_object_id, collection_id, child_class_id, child_object_id)
|
|
595
598
|
VALUES
|
|
596
|
-
(
|
|
599
|
+
(?, ?, ?, ?, ?)
|
|
597
600
|
"""
|
|
598
|
-
|
|
599
|
-
|
|
601
|
+
error_msg = "Some of the records do not have all the required fields. Check construction of records."
|
|
602
|
+
|
|
603
|
+
def prepare_batch(
|
|
604
|
+
batch_records: Sequence[dict[str, int]],
|
|
605
|
+
) -> list[tuple[int, int, int, int, int]]:
|
|
606
|
+
"""Validate records and map each membership dict to positional SQL parameters."""
|
|
607
|
+
params: list[tuple[int, int, int, int, int]] = []
|
|
608
|
+
for record in batch_records:
|
|
609
|
+
# Keep strict validation semantics: exact keys, no missing or extra fields.
|
|
610
|
+
if len(record) != 5:
|
|
611
|
+
raise KeyError(error_msg)
|
|
612
|
+
try:
|
|
613
|
+
params.append(
|
|
614
|
+
(
|
|
615
|
+
record["parent_class_id"],
|
|
616
|
+
record["parent_object_id"],
|
|
617
|
+
record["collection_id"],
|
|
618
|
+
record["child_class_id"],
|
|
619
|
+
record["child_object_id"],
|
|
620
|
+
)
|
|
621
|
+
)
|
|
622
|
+
except KeyError as exc:
|
|
623
|
+
raise KeyError(error_msg) from exc
|
|
624
|
+
return params
|
|
625
|
+
|
|
626
|
+
with self._db.transaction():
|
|
627
|
+
if chunksize >= len(records):
|
|
628
|
+
query_status = self._db.executemany(query, prepare_batch(records))
|
|
629
|
+
assert query_status
|
|
630
|
+
else:
|
|
631
|
+
for batch in batched(records, chunksize):
|
|
632
|
+
query_status = self._db.executemany(query, prepare_batch(batch))
|
|
633
|
+
assert query_status
|
|
634
|
+
|
|
600
635
|
logger.debug("Added {} memberships.", len(records))
|
|
601
636
|
return True
|
|
602
637
|
|
|
@@ -898,7 +933,7 @@ class PlexosDB:
|
|
|
898
933
|
|
|
899
934
|
with self._db.transaction():
|
|
900
935
|
data_id_map = insert_property_values(self, params, metadata_map=metadata_map)
|
|
901
|
-
apply_scenario_tags(self, params, scenario=scenario, chunksize=chunksize)
|
|
936
|
+
apply_scenario_tags(self, params, scenario=scenario, chunksize=chunksize, data_id_map=data_id_map)
|
|
902
937
|
|
|
903
938
|
if has_datafile_text:
|
|
904
939
|
insert_property_texts(
|
|
@@ -2702,10 +2737,9 @@ class PlexosDB:
|
|
|
2702
2737
|
self,
|
|
2703
2738
|
*object_names: Iterable[str] | str,
|
|
2704
2739
|
object_class: ClassEnum,
|
|
2705
|
-
category: str | None = None,
|
|
2706
2740
|
collection: CollectionEnum | None = None,
|
|
2707
2741
|
) -> list[dict[str, Any]]:
|
|
2708
|
-
"""Retrieve
|
|
2742
|
+
"""Retrieve memberships for the requested object names.
|
|
2709
2743
|
|
|
2710
2744
|
Parameters
|
|
2711
2745
|
----------
|
|
@@ -2715,21 +2749,23 @@ class PlexosDB:
|
|
|
2715
2749
|
object_class : ClassEnum
|
|
2716
2750
|
Class of the objects.
|
|
2717
2751
|
collection : CollectionEnum | None, optional
|
|
2718
|
-
Collection to filter memberships.
|
|
2752
|
+
Collection used to filter memberships. When provided, results are
|
|
2753
|
+
limited to memberships under the System parent class for that
|
|
2754
|
+
collection.
|
|
2719
2755
|
|
|
2720
2756
|
Returns
|
|
2721
2757
|
-------
|
|
2722
|
-
list[
|
|
2723
|
-
|
|
2724
|
-
|
|
2725
|
-
Raises
|
|
2726
|
-
------
|
|
2727
|
-
KeyError
|
|
2728
|
-
If any of the object_names do not exist.
|
|
2758
|
+
list[dict[str, Any]]
|
|
2759
|
+
Membership rows for the requested object names in ``object_class``.
|
|
2760
|
+
Names that do not exist are ignored.
|
|
2729
2761
|
"""
|
|
2730
2762
|
names = normalize_names(*object_names)
|
|
2731
|
-
|
|
2732
|
-
|
|
2763
|
+
if not names:
|
|
2764
|
+
return []
|
|
2765
|
+
|
|
2766
|
+
class_id = self.get_class_id(object_class) # 1 query instead of N
|
|
2767
|
+
|
|
2768
|
+
base_query = """
|
|
2733
2769
|
SELECT
|
|
2734
2770
|
mem.membership_id,
|
|
2735
2771
|
mem.child_class_id,
|
|
@@ -2737,36 +2773,36 @@ class PlexosDB:
|
|
|
2737
2773
|
mem.collection_id,
|
|
2738
2774
|
child_class.name AS class,
|
|
2739
2775
|
collections.name AS collection_name
|
|
2740
|
-
FROM
|
|
2741
|
-
|
|
2742
|
-
INNER JOIN
|
|
2743
|
-
|
|
2744
|
-
|
|
2745
|
-
|
|
2746
|
-
|
|
2747
|
-
|
|
2748
|
-
|
|
2749
|
-
|
|
2750
|
-
|
|
2751
|
-
|
|
2752
|
-
"""
|
|
2753
|
-
conditions = []
|
|
2754
|
-
if len(object_ids) == 1:
|
|
2755
|
-
conditions.append(
|
|
2756
|
-
f"(child_object.object_id = {object_ids[0]} OR parent_object.object_id = {object_ids[0]})"
|
|
2757
|
-
)
|
|
2758
|
-
else:
|
|
2759
|
-
conditions.append(
|
|
2760
|
-
f"(child_object.object_id in {object_ids} OR parent_object.object_id in {object_ids})"
|
|
2761
|
-
)
|
|
2762
|
-
parent_class = ClassEnum.System
|
|
2776
|
+
FROM t_membership AS mem
|
|
2777
|
+
INNER JOIN t_object AS parent_object ON mem.parent_object_id = parent_object.object_id
|
|
2778
|
+
INNER JOIN t_object AS child_object ON mem.child_object_id = child_object.object_id
|
|
2779
|
+
LEFT JOIN t_class AS parent_class ON mem.parent_class_id = parent_class.class_id
|
|
2780
|
+
LEFT JOIN t_class AS child_class ON mem.child_class_id = child_class.class_id
|
|
2781
|
+
LEFT JOIN t_collection AS collections ON mem.collection_id = collections.collection_id
|
|
2782
|
+
WHERE mem.child_class_id = ?
|
|
2783
|
+
AND child_object.name IN ({ph})
|
|
2784
|
+
"""
|
|
2785
|
+
|
|
2786
|
+
extra = ""
|
|
2787
|
+
extra_params: list[Any] = []
|
|
2763
2788
|
if collection:
|
|
2764
|
-
|
|
2765
|
-
|
|
2789
|
+
extra = " AND parent_class.name = ? AND collections.name = ?"
|
|
2790
|
+
extra_params = [ClassEnum.System.value, collection.value]
|
|
2791
|
+
|
|
2792
|
+
# Specify bound parameter limit
|
|
2793
|
+
CHUNK = 900 # noqa: N806
|
|
2794
|
+
all_rows: list[dict[str, Any]] = []
|
|
2795
|
+
for i in range(0, len(names), CHUNK):
|
|
2796
|
+
chunk = names[i : i + CHUNK]
|
|
2797
|
+
ph = ",".join("?" * len(chunk))
|
|
2798
|
+
params = (class_id, *chunk, *extra_params)
|
|
2799
|
+
all_rows.extend(
|
|
2800
|
+
self._db.fetchall_dict(
|
|
2801
|
+
base_query.format(ph=ph) + extra,
|
|
2802
|
+
params,
|
|
2803
|
+
)
|
|
2766
2804
|
)
|
|
2767
|
-
|
|
2768
|
-
query_string += " WHERE " + " AND ".join(conditions)
|
|
2769
|
-
return self._db.fetchall_dict(query_string)
|
|
2805
|
+
return all_rows
|
|
2770
2806
|
|
|
2771
2807
|
def get_metadata(
|
|
2772
2808
|
self,
|
|
@@ -544,20 +544,32 @@ def insert_property_values(
|
|
|
544
544
|
|
|
545
545
|
db._db.executemany("INSERT into t_data(membership_id, property_id, value) values (?,?,?)", params)
|
|
546
546
|
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
547
|
+
last_id = db._db.last_insert_rowid()
|
|
548
|
+
first_id = last_id - len(params) + 1
|
|
549
|
+
|
|
550
|
+
unique_membership_ids = list({mid for mid, _, _ in params})
|
|
551
|
+
membership_to_name: dict[int, str] = {}
|
|
552
|
+
# Specify bound parameter limit
|
|
553
|
+
CHUNK = 900 # noqa: N806
|
|
554
|
+
for i in range(0, len(unique_membership_ids), CHUNK):
|
|
555
|
+
chunk = tuple(unique_membership_ids[i : i + CHUNK])
|
|
556
|
+
ph = ",".join("?" * len(chunk))
|
|
557
|
+
rows = db._db.fetchall(
|
|
558
|
+
f"""SELECT m.membership_id, o.name
|
|
559
|
+
FROM t_membership m
|
|
560
|
+
JOIN t_object o ON m.child_object_id = o.object_id
|
|
561
|
+
WHERE m.membership_id IN ({ph})""",
|
|
562
|
+
chunk,
|
|
563
|
+
)
|
|
564
|
+
for mid, name in rows:
|
|
565
|
+
membership_to_name[mid] = name
|
|
566
|
+
|
|
567
|
+
data_id_map: dict[tuple[int, int, Any], tuple[int, str]] = {}
|
|
568
|
+
for i, (membership_id, property_id, value) in enumerate(params):
|
|
569
|
+
data_id_map[(membership_id, property_id, value)] = (
|
|
570
|
+
first_id + i,
|
|
571
|
+
membership_to_name.get(membership_id, ""),
|
|
572
|
+
)
|
|
561
573
|
|
|
562
574
|
if metadata_map:
|
|
563
575
|
_persist_metadata_for_data(db, metadata_map=metadata_map, data_id_map=data_id_map)
|
|
@@ -572,6 +584,7 @@ def apply_scenario_tags(
|
|
|
572
584
|
*,
|
|
573
585
|
scenario: str,
|
|
574
586
|
chunksize: int,
|
|
587
|
+
data_id_map: dict[tuple[int, int, Any], tuple[int, str]] | None = None,
|
|
575
588
|
) -> None:
|
|
576
589
|
"""Insert scenario tags for property data.
|
|
577
590
|
|
|
@@ -594,18 +607,22 @@ def apply_scenario_tags(
|
|
|
594
607
|
else:
|
|
595
608
|
scenario_id = db.get_scenario_id(scenario)
|
|
596
609
|
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
610
|
+
if data_id_map is not None:
|
|
611
|
+
tag_rows = [(data_id_map[key][0], scenario_id) for key in params if key in data_id_map]
|
|
612
|
+
for batch in batched(tag_rows, chunksize):
|
|
613
|
+
db._db.executemany(
|
|
614
|
+
"INSERT INTO t_tag(data_id, object_id) VALUES (?, ?)",
|
|
615
|
+
list(batch),
|
|
616
|
+
)
|
|
617
|
+
else:
|
|
618
|
+
for batch in batched(params, chunksize):
|
|
619
|
+
scenario_query = f"""
|
|
620
|
+
INSERT into t_tag(data_id, object_id)
|
|
621
|
+
SELECT d.data_id, {scenario_id}
|
|
622
|
+
FROM t_data d
|
|
623
|
+
WHERE d.membership_id = ? AND d.property_id = ? AND d.value = ?
|
|
624
|
+
"""
|
|
625
|
+
db._db.executemany(scenario_query, list(batch))
|
|
609
626
|
|
|
610
627
|
|
|
611
628
|
def insert_property_texts(
|
plexosdb-1.3.2/PKG-INFO
DELETED
|
@@ -1,83 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.3
|
|
2
|
-
Name: plexosdb
|
|
3
|
-
Version: 1.3.2
|
|
4
|
-
Summary: SQLite API for plexos XMLs
|
|
5
|
-
Keywords: PLEXOS,Database,SQLite
|
|
6
|
-
Author: Pedro Andres Sanchez Perez, Kodi Obika, mcllerena
|
|
7
|
-
Author-email: Pedro Andres Sanchez Perez <psanchez@nrel.gov>, Kodi Obika <kodi.obika@nrel.gov>, mcllerena <mcllerena@users.noreply.github.com>
|
|
8
|
-
License: BSD 3-Clause License
|
|
9
|
-
|
|
10
|
-
Copyright (c) 2024, Alliance for Sustainable Energy LLC, All rights reserved.
|
|
11
|
-
|
|
12
|
-
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
|
13
|
-
|
|
14
|
-
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
|
15
|
-
|
|
16
|
-
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
|
17
|
-
|
|
18
|
-
* Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
|
|
19
|
-
|
|
20
|
-
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
21
|
-
Classifier: Development Status :: 5 - Production/Stable
|
|
22
|
-
Classifier: Intended Audience :: Developers
|
|
23
|
-
Classifier: License :: OSI Approved :: BSD License
|
|
24
|
-
Classifier: Topic :: Software Development :: Build Tools
|
|
25
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
26
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
27
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
28
|
-
Classifier: Programming Language :: Python :: 3.14
|
|
29
|
-
Classifier: Topic :: Scientific/Engineering
|
|
30
|
-
Classifier: Topic :: Scientific/Engineering :: Information Analysis
|
|
31
|
-
Classifier: Topic :: Scientific/Engineering :: Interface Engine/Protocol Translator
|
|
32
|
-
Classifier: Topic :: Software Development :: Build Tools
|
|
33
|
-
Classifier: Topic :: Database
|
|
34
|
-
Classifier: Typing :: Typed
|
|
35
|
-
Requires-Dist: loguru
|
|
36
|
-
Maintainer: Pedro Andres Sanchez Perez, Kodi Obika, mcllerena
|
|
37
|
-
Maintainer-email: Pedro Andres Sanchez Perez <psanchez@nrel.gov>, Kodi Obika <kodi.obika@nrel.gov>, mcllerena <mcllerena@users.noreply.github.com>
|
|
38
|
-
Requires-Python: >=3.11, <3.15
|
|
39
|
-
Project-URL: Changelog, https://github.com/NREL/plexosdb/blob/master/CHANGELOG.md
|
|
40
|
-
Project-URL: Documentation, https://github.com/NREL/plexosdb#readme
|
|
41
|
-
Project-URL: Issues, https://github.com/NREL/plexosdb/issues
|
|
42
|
-
Project-URL: Source, https://github.com/NREL/plexosdb
|
|
43
|
-
Description-Content-Type: text/markdown
|
|
44
|
-
|
|
45
|
-
### Database Manager for use with PLEXOS XML files
|
|
46
|
-
[](https://pypi.python.org/pypi/plexosdb)
|
|
47
|
-
[](https://pypi.python.org/pypi/plexosdb)
|
|
48
|
-
[](https://pypi.python.org/pypi/plexosdb)
|
|
49
|
-
[](https://github.com/NREL/plexosdb/actions/workflows/CI.yaml)
|
|
50
|
-
[](https://codecov.io/gh/NREL/plexosdb)
|
|
51
|
-
[](https://github.com/astral-sh/ruff)
|
|
52
|
-
<br/>
|
|
53
|
-
|
|
54
|
-
## Installation
|
|
55
|
-
|
|
56
|
-
```console
|
|
57
|
-
python -m pip install plexosdb
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
## Developer installation
|
|
61
|
-
|
|
62
|
-
```console
|
|
63
|
-
$ pip install -e ".[dev]"
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
Please install `pre-commit` so that your code is checked before making commits.
|
|
67
|
-
|
|
68
|
-
```console
|
|
69
|
-
pre-commit install
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
## License
|
|
73
|
-
|
|
74
|
-
This software is released under a BSD-3-Clause
|
|
75
|
-
[License](https://github.com/NREL/plexosdb/blob/main/LICENSE.txt).
|
|
76
|
-
|
|
77
|
-
This software was developed under software record SWR-24-90 at the National Renewable Energy Laboratory
|
|
78
|
-
([NREL](https://www.nrel.gov)).
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
## Disclaimer
|
|
82
|
-
|
|
83
|
-
PLEXOS is a registered trademark of Energy Exemplar Pty Ltd. Energy Exemplar Pty Ltd. has no affiliation to or participation in this software. Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or Alliance for Sustainable Energy, LLC ("Alliance"). The views and opinions of authors expressed in the available or referenced documents do not necessarily state or reflect those of the United States Government or Alliance.
|
plexosdb-1.3.2/README.md
DELETED
|
@@ -1,39 +0,0 @@
|
|
|
1
|
-
### Database Manager for use with PLEXOS XML files
|
|
2
|
-
[](https://pypi.python.org/pypi/plexosdb)
|
|
3
|
-
[](https://pypi.python.org/pypi/plexosdb)
|
|
4
|
-
[](https://pypi.python.org/pypi/plexosdb)
|
|
5
|
-
[](https://github.com/NREL/plexosdb/actions/workflows/CI.yaml)
|
|
6
|
-
[](https://codecov.io/gh/NREL/plexosdb)
|
|
7
|
-
[](https://github.com/astral-sh/ruff)
|
|
8
|
-
<br/>
|
|
9
|
-
|
|
10
|
-
## Installation
|
|
11
|
-
|
|
12
|
-
```console
|
|
13
|
-
python -m pip install plexosdb
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
## Developer installation
|
|
17
|
-
|
|
18
|
-
```console
|
|
19
|
-
$ pip install -e ".[dev]"
|
|
20
|
-
```
|
|
21
|
-
|
|
22
|
-
Please install `pre-commit` so that your code is checked before making commits.
|
|
23
|
-
|
|
24
|
-
```console
|
|
25
|
-
pre-commit install
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
## License
|
|
29
|
-
|
|
30
|
-
This software is released under a BSD-3-Clause
|
|
31
|
-
[License](https://github.com/NREL/plexosdb/blob/main/LICENSE.txt).
|
|
32
|
-
|
|
33
|
-
This software was developed under software record SWR-24-90 at the National Renewable Energy Laboratory
|
|
34
|
-
([NREL](https://www.nrel.gov)).
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
## Disclaimer
|
|
38
|
-
|
|
39
|
-
PLEXOS is a registered trademark of Energy Exemplar Pty Ltd. Energy Exemplar Pty Ltd. has no affiliation to or participation in this software. Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or Alliance for Sustainable Energy, LLC ("Alliance"). The views and opinions of authors expressed in the available or referenced documents do not necessarily state or reflect those of the United States Government or Alliance.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|