fameio 2.3.0__py3-none-any.whl → 3.0.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- CHANGELOG.md +28 -0
- fameio/__init__.py +4 -1
- fameio/{source/cli → cli}/__init__.py +2 -0
- fameio/{source/cli → cli}/convert_results.py +8 -8
- fameio/{source/cli → cli}/make_config.py +5 -5
- fameio/{source/cli → cli}/options.py +0 -8
- fameio/{source/cli → cli}/parser.py +26 -63
- fameio/input/__init__.py +27 -0
- fameio/input/loader/__init__.py +68 -0
- fameio/input/loader/controller.py +129 -0
- fameio/input/loader/loader.py +109 -0
- fameio/input/metadata.py +149 -0
- fameio/input/resolver.py +44 -0
- fameio/{source → input}/scenario/__init__.py +1 -2
- fameio/{source → input}/scenario/agent.py +24 -38
- fameio/input/scenario/attribute.py +203 -0
- fameio/{source → input}/scenario/contract.py +50 -61
- fameio/{source → input}/scenario/exception.py +8 -13
- fameio/{source → input}/scenario/fameiofactory.py +6 -6
- fameio/{source → input}/scenario/generalproperties.py +22 -47
- fameio/{source → input}/scenario/scenario.py +34 -31
- fameio/input/scenario/stringset.py +48 -0
- fameio/{source → input}/schema/__init__.py +2 -2
- fameio/input/schema/agenttype.py +125 -0
- fameio/input/schema/attribute.py +268 -0
- fameio/{source → input}/schema/java_packages.py +26 -22
- fameio/{source → input}/schema/schema.py +25 -22
- fameio/{source → input}/validator.py +32 -35
- fameio/{source → input}/writer.py +86 -86
- fameio/{source/logs.py → logs.py} +25 -9
- fameio/{source/results → output}/agent_type.py +21 -22
- fameio/{source/results → output}/conversion.py +34 -31
- fameio/{source/results → output}/csv_writer.py +7 -7
- fameio/{source/results → output}/data_transformer.py +24 -24
- fameio/{source/results → output}/input_dao.py +51 -49
- fameio/{source/results → output}/output_dao.py +16 -17
- fameio/{source/results → output}/reader.py +30 -31
- fameio/{source/results → output}/yaml_writer.py +2 -3
- fameio/scripts/__init__.py +2 -2
- fameio/scripts/convert_results.py +16 -15
- fameio/scripts/make_config.py +9 -9
- fameio/{source/series.py → series.py} +28 -26
- fameio/{source/time.py → time.py} +8 -8
- fameio/{source/tools.py → tools.py} +2 -2
- {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/METADATA +277 -72
- fameio-3.0.0.dist-info/RECORD +56 -0
- fameio/source/__init__.py +0 -8
- fameio/source/loader.py +0 -181
- fameio/source/metadata.py +0 -32
- fameio/source/path_resolver.py +0 -34
- fameio/source/scenario/attribute.py +0 -130
- fameio/source/scenario/stringset.py +0 -51
- fameio/source/schema/agenttype.py +0 -132
- fameio/source/schema/attribute.py +0 -203
- fameio/source/schema/exception.py +0 -9
- fameio-2.3.0.dist-info/RECORD +0 -55
- /fameio/{source/results → output}/__init__.py +0 -0
- {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSE.txt +0 -0
- {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSES/Apache-2.0.txt +0 -0
- {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSES/CC-BY-4.0.txt +0 -0
- {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSES/CC0-1.0.txt +0 -0
- {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/WHEEL +0 -0
- {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/entry_points.txt +0 -0
@@ -1,7 +1,7 @@
|
|
1
1
|
Metadata-Version: 2.1
|
2
2
|
Name: fameio
|
3
|
-
Version:
|
4
|
-
Summary:
|
3
|
+
Version: 3.0.0
|
4
|
+
Summary: Tools for input preparation and output digestion of FAME models
|
5
5
|
Home-page: https://gitlab.com/fame-framework/wiki/-/wikis/home
|
6
6
|
License: Apache-2.0
|
7
7
|
Keywords: FAME,fameio,agent-based modelling,energy systems
|
@@ -21,7 +21,7 @@ Classifier: Programming Language :: Python :: 3.10
|
|
21
21
|
Classifier: Programming Language :: Python :: 3.11
|
22
22
|
Classifier: Programming Language :: Python :: 3.12
|
23
23
|
Classifier: Topic :: Scientific/Engineering
|
24
|
-
Requires-Dist: fameprotobuf (>=
|
24
|
+
Requires-Dist: fameprotobuf (>=2.0.2,<3.0.0)
|
25
25
|
Requires-Dist: pandas (>=1.0,<3.0)
|
26
26
|
Requires-Dist: pyyaml (>=6.0,<7.0)
|
27
27
|
Project-URL: Repository, https://gitlab.com/fame-framework/fame-io/
|
@@ -42,17 +42,21 @@ SPDX-License-Identifier: Apache-2.0 -->
|
|
42
42
|

|
43
43
|

|
44
44
|
|
45
|
-
|
46
45
|
# FAME-Io
|
47
|
-
|
46
|
+
|
47
|
+
*Tools for input preparation and output digestion of FAME models*
|
48
|
+
|
49
|
+
FAME-Io compiles input for FAME models in protobuf format and extracts model outputs to human-readable files.
|
48
50
|
Please visit the [FAME-Wiki](https://gitlab.com/fame-framework/wiki/-/wikis/home) to get an explanation of FAME and its components.
|
49
51
|
|
50
52
|
# Installation
|
53
|
+
|
51
54
|
We recommend installing `fameio` using PyPI:
|
52
55
|
|
53
56
|
pip install fameio
|
54
57
|
|
55
|
-
You may also use `pipx`. For detailed information please refer to the
|
58
|
+
You may also use `pipx`. For detailed information please refer to the
|
59
|
+
official `pipx` [documentation](https://github.com/pypa/pipx).
|
56
60
|
|
57
61
|
pipx install fameio
|
58
62
|
|
@@ -60,15 +64,21 @@ You may also use `pipx`. For detailed information please refer to the official `
|
|
60
64
|
See the `pyproject.toml` for a complete listing of dependencies.
|
61
65
|
|
62
66
|
# Usage
|
67
|
+
|
63
68
|
FAME-Io currently offers two main scripts `makeFameRunConfig` and `convertFameResults`.
|
64
69
|
Both are automatically installed with the package.
|
65
70
|
The first one creates a protobuf file for FAME applications using YAML definition files and CSV files.
|
66
71
|
The latter one reads output files from FAME applications in protobuf format and converts them to CSV files.
|
67
72
|
|
68
|
-
You may use the [example data](https://gitlab.com/dlr-ve/esy/amiris/examples) provided for
|
73
|
+
You may use the [example data](https://gitlab.com/dlr-ve/esy/amiris/examples) provided for
|
74
|
+
the [AMIRIS](https://gitlab.com/dlr-ve/esy/amiris/amiris) model which can be used to simulate electricity markets
|
75
|
+
in [Germany](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Germany2019), [Austria](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Austria2019),
|
76
|
+
and a simple [proof-of-concept model](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Simple).
|
69
77
|
|
70
78
|
## Make a FAME run configuration
|
71
|
-
|
79
|
+
|
80
|
+
Digests configuration files in YAML format, combines them with CSV data files and creates a single input file for FAME
|
81
|
+
applications in protobuf format.
|
72
82
|
Call structure:
|
73
83
|
|
74
84
|
makeFameRunConfig -f <path/to/scenario.yaml>
|
@@ -94,7 +104,8 @@ from fameio.scripts.make_config import Options, run as make_config
|
|
94
104
|
make_config({Options.FILE: "path/to/scenario.yaml", })
|
95
105
|
```
|
96
106
|
|
97
|
-
Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function
|
107
|
+
Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function
|
108
|
+
call.
|
98
109
|
|
99
110
|
```python
|
100
111
|
from fameio.scripts.make_config import Options, run as make_config
|
@@ -112,13 +123,13 @@ You can also use the associated argument parser, to extract the run_config dynam
|
|
112
123
|
|
113
124
|
```python
|
114
125
|
from fameio.scripts.make_config import Options, run as make_config
|
115
|
-
from fameio.
|
126
|
+
from fameio.cli.make_config import handle_args
|
116
127
|
|
117
128
|
my_defaults = {Options.FILE: "path/to/scenario.yaml",
|
118
|
-
|
119
|
-
|
120
|
-
|
121
|
-
|
129
|
+
Options.LOG_LEVEL: "info",
|
130
|
+
Options.OUTPUT: "output.pb",
|
131
|
+
Options.LOG_FILE: "scenario.log",
|
132
|
+
}
|
122
133
|
my_arg_string = ['-f', 'my/other/scenario.yaml', '-l', 'error']
|
123
134
|
|
124
135
|
run_config = handle_args(my_arg_string, my_defaults)
|
@@ -126,16 +137,21 @@ make_config(run_config)
|
|
126
137
|
```
|
127
138
|
|
128
139
|
### Scenario YAML
|
140
|
+
|
129
141
|
The "scenario.yaml" file contains all configuration options for a FAME-based simulation.
|
130
|
-
It consists of the sections `Schema`, `GeneralProperties`, `Agents` and `Contracts`, and the optional
|
142
|
+
It consists of the sections `Schema`, `GeneralProperties`, `Agents` and `Contracts`, and the optional
|
143
|
+
section `StringSets`.
|
131
144
|
All of them are described below.
|
132
145
|
|
133
146
|
#### Schema
|
147
|
+
|
134
148
|
The Schema describes a model's components such as its types of agents, their inputs, what data they exchange, etc.
|
135
149
|
It is also used to validate the model inputs provided in the `scenario.yaml`.
|
136
|
-
Since the Schema is valid until the model itself is changed, it is recommended to defined it in a separate file and
|
150
|
+
Since the Schema is valid until the model itself is changed, it is recommended to defined it in a separate file and
|
151
|
+
include the file here.
|
137
152
|
|
138
153
|
Currently, the schema specifies:
|
154
|
+
|
139
155
|
* which type of Agents can be created
|
140
156
|
* what type of input attributes an Agent uses
|
141
157
|
* what type of Products an Agent can send in Contracts, and
|
@@ -144,15 +160,18 @@ Currently, the schema specifies:
|
|
144
160
|
The Schema consists of the sections `JavaPackages` and `AgentTypes`.
|
145
161
|
|
146
162
|
##### JavaPackages
|
163
|
+
|
147
164
|
This section defines the name of the Java packages in which the model code is located.
|
148
165
|
A similar data set was formerly specified in the `fameSetup.yaml`, but is now specified in the schema.
|
149
|
-
Each of the three sections `Agents`, `DataItems`, and `Portables` contain a list of fully qualified java package names
|
166
|
+
Each of the three sections `Agents`, `DataItems`, and `Portables` contain a list of fully qualified java package names
|
167
|
+
of your model's classes.
|
150
168
|
Package names can occur in multiple lists and may overlap.
|
151
169
|
It is not necessary (but possible) to specify the nearest enclosing package for each Agent, DataItem or Portable.
|
152
170
|
Specifying any super-package will also work.
|
153
171
|
Also, package names occur on multiple lists for Agent, DataItem or Portable.
|
154
172
|
|
155
173
|
For example, for a project with all its
|
174
|
+
|
156
175
|
* Agent-derived java classes located in packages below the package named "agents",
|
157
176
|
* DataItem implementation classes in a subpackage named "msg",
|
158
177
|
* Portable implementation classes in a subpackages named "portableItems" and "otherPortables",
|
@@ -170,8 +189,12 @@ JavaPackages:
|
|
170
189
|
- "otherPortables"
|
171
190
|
```
|
172
191
|
|
192
|
+
One can leave out the `DataItems` specifications, but `Agents` and `Portables` are required and must not be empty.
|
193
|
+
|
173
194
|
##### AgentTypes
|
174
|
-
|
195
|
+
|
196
|
+
Here, each type of agent that can be created in your FAME-based application is listed, its attributes and its available
|
197
|
+
Products for Contracts.
|
175
198
|
The structure of this section
|
176
199
|
|
177
200
|
```yaml
|
@@ -183,18 +206,49 @@ AgentTypes:
|
|
183
206
|
MyOtherAttribute:
|
184
207
|
...
|
185
208
|
Products: [ 'Product1', 'Product2', 'Product3' ]
|
209
|
+
Outputs: [ 'Column1', 'Column2', 'Column3' ]
|
210
|
+
Metadata:
|
211
|
+
Some: "Dict with Metadata that you would like to add"
|
186
212
|
MyOtherAgentWithoutProductsOrAttributes:
|
187
213
|
```
|
188
214
|
|
189
215
|
* `MyAgentType` Java's simple class name of the Agent type
|
190
216
|
* `Attributes` indicates that beginning of the attribute definition section for this Agent type
|
191
|
-
* `MyAttribute` Name of an attribute as specified in the corresponding Java source code of this Agent type (annotated
|
217
|
+
* `MyAttribute` Name of an attribute as specified in the corresponding Java source code of this Agent type (annotated
|
218
|
+
with "@Input")
|
192
219
|
* `MyOtherAttribute` Name of another attribute derived from Java source code
|
193
|
-
* `Products` list of Products that this Agent can send in Contracts; derived from Java source code of this
|
220
|
+
* `Products` list or dictionary of Products that this Agent can send in Contracts; derived from Java source code of this
|
221
|
+
Agent type (annotated with "@Product")
|
222
|
+
* `Outputs` list or dictionary of Output columns that this Agent can write to; derived from Java source code of this
|
223
|
+
Agent type (annotated with "@Output")
|
224
|
+
* `Metadata` dictionary with any content that is assigned to this Agent type as additional information
|
194
225
|
* `MyOtherAgentWithoutProductsOrAttributes` an Agent type that requires neither Attributes nor Products
|
195
226
|
|
196
|
-
|
197
|
-
|
227
|
+
Attributes, Products, Outputs and Metadata are optional - there may be useful Agents that require none of them.
|
228
|
+
Products and Outputs can both be lists of Strings, or dictionaries with additional Metadata.
|
229
|
+
For example, you could write the above in the following way:
|
230
|
+
|
231
|
+
```yaml
|
232
|
+
Products:
|
233
|
+
Product1:
|
234
|
+
Metadata:
|
235
|
+
Any: "information you would like to add to Product1 using a dictionary form"
|
236
|
+
Product2:
|
237
|
+
Product3:
|
238
|
+
Outputs:
|
239
|
+
Column1:
|
240
|
+
Column2:
|
241
|
+
ThisEntry: "is ignored, as it is not below the keyword: 'Metadata'"
|
242
|
+
Metadata:
|
243
|
+
My: "Metadata"
|
244
|
+
That: "will be saved to Column2"
|
245
|
+
Column3:
|
246
|
+
```
|
247
|
+
|
248
|
+
Here, "Product1" and "Column2" have additional, optional Metadata assigned to them (using the keyword "Metadata").
|
249
|
+
The other Products and Columns have no metadata assigned to them - which is also ok.
|
250
|
+
|
251
|
+
In the AgentType definition example above attribute definition was not shown explicitly (indicated by `...`).
|
198
252
|
The next example provides details on how to define an attribute:
|
199
253
|
|
200
254
|
```yaml
|
@@ -205,29 +259,48 @@ MySimpleAttribute:
|
|
205
259
|
Values: [ 'AllowedValue1', 'AllowedValue2' ]
|
206
260
|
Default: 'AllowedValue1'
|
207
261
|
Help: 'My help text'
|
262
|
+
Metadata:
|
263
|
+
Go: "here"
|
208
264
|
|
209
265
|
MyComplexAttribute:
|
210
266
|
AttributeType: block
|
211
267
|
NestedAttributes:
|
212
268
|
InnerAttributeA:
|
213
269
|
AttributeType: integer
|
270
|
+
Values:
|
271
|
+
1:
|
272
|
+
Metadata:
|
273
|
+
Explain: "1 is a allowed value"
|
274
|
+
2:
|
275
|
+
Metadata:
|
276
|
+
Comment: "2 is also allowed, but consider using 1"
|
214
277
|
InnerAttributeB:
|
215
278
|
AttributeType: double
|
216
279
|
```
|
217
280
|
|
218
|
-
* `MySimpleAttribute`, `MyDoubleList`, `MyComplexAttribute` Names of the attributes as specified in the Java enum
|
281
|
+
* `MySimpleAttribute`, `MyDoubleList`, `MyComplexAttribute` Names of the attributes as specified in the Java enum
|
282
|
+
annotated with "@Input"
|
219
283
|
* `AttributeType` (required) data type of the attribute; see options in table below
|
220
|
-
* `Mandatory` (optional - true by default) if true: the attribute is required for this agent and validation will fail if
|
284
|
+
* `Mandatory` (optional - true by default) if true: the attribute is required for this agent and validation will fail if
|
285
|
+
the attribute is missing in the scenario **and** no default is provided
|
221
286
|
* `List` (optional - false by default)
|
222
287
|
* `AttributeType: time_series` cannot be true
|
223
288
|
* `AttributeType: block`
|
224
289
|
* if true: any nested element in the scenario must be part of a list element and thus can appear multiple times
|
225
290
|
* if false: any nested element in the scenario can only appear once
|
226
|
-
* any other AttributeType: the attribute is interpreted as list, i.e. multiple values can be assigned to this
|
227
|
-
|
228
|
-
* `
|
229
|
-
|
230
|
-
* `
|
291
|
+
* any other AttributeType: the attribute is interpreted as list, i.e. multiple values can be assigned to this
|
292
|
+
attribute in the scenario
|
293
|
+
* `NestedAttributes` (required only if `AttributeType: block`, otherwise disallowed) starts an inner Attribute
|
294
|
+
definition block - defined Attributes are sub-elements of `MyComplexAttribute`
|
295
|
+
* `Values` (optional - None by default):
|
296
|
+
* if present, defines a list or dictionary of allowed values for this attribute
|
297
|
+
* if a dictionary is used, individual Metadata can be assigned to each allowed value using the `Metadata` keyword
|
298
|
+
* `Default` (optional - None by default):
|
299
|
+
* if present, defines a default value to be used if the scenario does not specify one
|
300
|
+
* must match one of the entries in `Values` in case those are defined
|
301
|
+
* can be a list if the attribute is a list
|
302
|
+
* `Help` (optional - None by default): if present, defines a help text for your Attribute
|
303
|
+
* `Metadata` (optional - None by default): if present, defines additional metadata assigned to the Attribute
|
231
304
|
|
232
305
|
| AttributeType | value |
|
233
306
|
|---------------|-------------------------------------------------------------------------------------------------------------------------|
|
@@ -251,23 +324,18 @@ GeneralProperties:
|
|
251
324
|
StartTime: 2011-12-31_23:58:00
|
252
325
|
StopTime: 2012-12-30_23:58:00
|
253
326
|
RandomSeed: 1
|
254
|
-
Output:
|
255
|
-
Interval: 100
|
256
|
-
Process: 0
|
257
327
|
```
|
258
328
|
|
259
329
|
Parameters:
|
330
|
+
|
260
331
|
* `RunId` an ID that can be given to the simulation; use at your discretion
|
261
332
|
* `StartTime` time stamp in the format YYYY-MM-DD_hh:mm:ss; first moment of the simulation.
|
262
333
|
* `StopTime` time stamp in the format YYYY-MM-DD_hh:mm:ss; last moment of the simulation - i.e. simulation terminates
|
263
334
|
after passing that time stamp
|
264
335
|
* `RandomSeed` seed to initialise random number generation; each value leads to a unique series of random numbers.
|
265
336
|
|
266
|
-
Parameters in section `Output` are deprecated and will be removed in FAME-Io v3.0
|
267
|
-
* `Interval` number of simulation ticks in between write-to-disk events; may be used for performance optimisations;
|
268
|
-
* `Process` id of process that performs write-to-disk operations; leave at 0 to be compatible with single-processes;
|
269
|
-
|
270
337
|
#### Agents
|
338
|
+
|
271
339
|
Specifies all Agents to be created in the simulation in a list. Each Agent has its own entry.
|
272
340
|
Structure:
|
273
341
|
|
@@ -280,17 +348,21 @@ Agents:
|
|
280
348
|
MyInteger: 2
|
281
349
|
MyDouble: 4.2
|
282
350
|
MyTimeSeries: "./path/to/time_series.csv"
|
351
|
+
Metadata:
|
352
|
+
Can: "also be assigned"
|
283
353
|
|
284
354
|
- Type: MyAgentWithoutInputs
|
285
355
|
Id: 2
|
286
356
|
```
|
287
357
|
|
288
358
|
Agent Parameters:
|
359
|
+
|
289
360
|
* `Type` Mandatory; Java's simple class name of the agent to be created
|
290
361
|
* `Id` Mandatory; simulation-unique id of this agent; if two agents have the same ID, the configuration process will
|
291
362
|
stop.
|
292
363
|
* `Attributes` Optional; if the agent has any attributes, specify them here in the format "AttributeName: value"; please
|
293
364
|
see attribute table above
|
365
|
+
* `Metadata` Optional; can be assigned to each instance of an Agent, as well as to each of its Attributes
|
294
366
|
|
295
367
|
The specified `Attributes` for each agent must match the specified `Attributes` options in the linked Schema (see above).
|
296
368
|
For better structure and readability of the `scenario.yaml`, `Attributes` may also be specified in a nested way as demonstrated below.
|
@@ -323,7 +395,102 @@ Attributes:
|
|
323
395
|
|
324
396
|
Here, `MyDoubleList` and `MyListGroup` need to specify `List: true` in the corresponding Schema.
|
325
397
|
The shorter `[]`-notation was used to assign a list of floating-point values to `MyDoubleList`.
|
326
|
-
Nested items `IntValueA` and `IntValueB` of `MyListGroup` are assigned within a list, allowing the specification of
|
398
|
+
Nested items `IntValueA` and `IntValueB` of `MyListGroup` are assigned within a list, allowing the specification of
|
399
|
+
these nested items several times.
|
400
|
+
|
401
|
+
##### Attribute Metadata
|
402
|
+
Metadata can be assigned to any value, list item, or superstructure.
|
403
|
+
To assign Metadata to a primitive value, create a dictionary from it, set the actual value with the inner keyword `Value` and add the keyword `Metadata` like this:
|
404
|
+
|
405
|
+
```yaml
|
406
|
+
ValueWithoutMetadata: 1
|
407
|
+
SameValueWithMetadata:
|
408
|
+
Value: 1
|
409
|
+
Metadata: # describe `SameValueWithMetadata` herein
|
410
|
+
```
|
411
|
+
|
412
|
+
You can assign Metadata to a list of primitive values using the keyword `Values` like this:
|
413
|
+
|
414
|
+
```yaml
|
415
|
+
ValueListWithoutMetadata: [1,2,3]
|
416
|
+
SameValueListWithListMetadata:
|
417
|
+
Values: [1,2,3]
|
418
|
+
Metadata: # describe the whole list of values with Metadata here
|
419
|
+
```
|
420
|
+
|
421
|
+
or specify Metadata for each (or just some) value individually, like this:
|
422
|
+
|
423
|
+
```yaml
|
424
|
+
ValueListWithoutMetadata: [1,2,3]
|
425
|
+
SameValueListWithMetadataAtEachElement:
|
426
|
+
- Value: 1
|
427
|
+
Metadata: # describe this specific value "1" with Metadata here
|
428
|
+
- Value: 2 # this value has no Metadata attached, but you can still use the keyword `Value`
|
429
|
+
- 3 # or use in the actual directly since this value has no Metadata anyway
|
430
|
+
```
|
431
|
+
|
432
|
+
or assign Metadata to both the list and any of its list entries, like this:
|
433
|
+
|
434
|
+
```yaml
|
435
|
+
ValueListWithoutMetadata: [1,2,3]
|
436
|
+
SameValueListWithAllMetadata:
|
437
|
+
Metadata: # Recommendation: place the Metadata of the list first if the list of values is extensive, as in this case
|
438
|
+
Values:
|
439
|
+
- Value: 1
|
440
|
+
Metadata: # describe this specific value "1" with Metadata here
|
441
|
+
- Value: 2
|
442
|
+
Metadata: # describe this specific value "2" with Metadata here
|
443
|
+
- Value: 3
|
444
|
+
Metadata: # describe this specific value "3" with Metadata here
|
445
|
+
```
|
446
|
+
|
447
|
+
You can assign Metadata directly to a nested element by adding the Metadata keyword:
|
448
|
+
|
449
|
+
```yaml
|
450
|
+
NestedItemWithoutMetadata:
|
451
|
+
A: 1
|
452
|
+
B: 2
|
453
|
+
SameNestedItemWithMetadata:
|
454
|
+
A: 1
|
455
|
+
B: 2
|
456
|
+
Metadata: # These Metadata describe `SameNestedItemWithMetadata`
|
457
|
+
```
|
458
|
+
|
459
|
+
Similar to lists of values, you can assign Metadata to a list of nested elements using the `Values` keyword, like this:
|
460
|
+
|
461
|
+
```yaml
|
462
|
+
ListOfNestedItemsWithoutMetadata:
|
463
|
+
- A: 1
|
464
|
+
B: 10
|
465
|
+
- A: 2
|
466
|
+
B: 20
|
467
|
+
SameListOfNestedItemsWithGeneralMetadata:
|
468
|
+
Values:
|
469
|
+
- A: 1
|
470
|
+
B: 10
|
471
|
+
- A: 2
|
472
|
+
B: 20
|
473
|
+
Metadata: # These Metadata describe `SameListOfNestedItemsWithGeneralMetadata` as a whole
|
474
|
+
```
|
475
|
+
|
476
|
+
and, similar to nested elements, you can assign Metadata directly to any list element, like this:
|
477
|
+
|
478
|
+
```yaml
|
479
|
+
ListOfNestedItemsWithoutMetadata:
|
480
|
+
- A: 1
|
481
|
+
B: 10
|
482
|
+
- A: 2
|
483
|
+
B: 20
|
484
|
+
SameListOfNestedItemsWithGeneralMetadata:
|
485
|
+
- A: 1
|
486
|
+
B: 10
|
487
|
+
Metadata: # These Metadata describe the first list item
|
488
|
+
- A: 2
|
489
|
+
B: 20
|
490
|
+
Metadata: # These Metadata describe the second list item
|
491
|
+
```
|
492
|
+
|
493
|
+
Again, you may apply both variants and apply Metadata to the list and each of its items if you wish.
|
327
494
|
|
328
495
|
#### Contracts
|
329
496
|
Specifies all Contracts, i.e. repetitive bilateral transactions in between agents.
|
@@ -337,6 +504,8 @@ Contracts:
|
|
337
504
|
ProductName: ProductOfAgent_1
|
338
505
|
FirstDeliveryTime: -25
|
339
506
|
DeliveryIntervalInSteps: 3600
|
507
|
+
Metadata:
|
508
|
+
Some: "additional information can go here"
|
340
509
|
|
341
510
|
- SenderId: 2
|
342
511
|
ReceiverId: 1
|
@@ -349,17 +518,20 @@ Contracts:
|
|
349
518
|
```
|
350
519
|
|
351
520
|
Contract Parameters:
|
521
|
+
|
352
522
|
* `SenderId` unique ID of agent sending the product
|
353
523
|
* `ReceiverId` unique ID of agent receiving the product
|
354
524
|
* `ProductName` name of the product to be sent
|
355
525
|
* `FirstDeliveryTime` first time of delivery in the format "seconds after the January 1st 2000, 00:00:00"
|
356
526
|
* `DeliveryIntervalInSteps` delay time in between deliveries in seconds
|
527
|
+
* `Metadata` can be assigned to add further helpful information about a Contract
|
357
528
|
* `Attributes` can be set to include additional information as `int`, `float`, `enum`, or `dict` data types
|
358
529
|
|
359
530
|
##### Definition of Multiple Similar Contracts
|
360
531
|
Often, scenarios contain multiple agents of similar type that also have similar chains of contracts.
|
361
532
|
Therefore, FAME-Io supports a compact definition of multiple similar contracts.
|
362
|
-
`SenderId` and `ReceiverId` can both be lists and support One-to-N, N-to-One and N-to-N relations like in the following
|
533
|
+
`SenderId` and `ReceiverId` can both be lists and support One-to-N, N-to-One and N-to-N relations like in the following
|
534
|
+
example:
|
363
535
|
|
364
536
|
```yaml
|
365
537
|
Contracts:
|
@@ -389,7 +561,8 @@ Contracts:
|
|
389
561
|
```
|
390
562
|
|
391
563
|
Combined with YAML anchors complex contract chains can be easily reduced to a minimum of required configuration.
|
392
|
-
The following example is equivalent to the previous one and allows a quick extension of contracts to a new couple of
|
564
|
+
The following example is equivalent to the previous one and allows a quick extension of contracts to a new couple of
|
565
|
+
agents e.g. (4;14):
|
393
566
|
|
394
567
|
```yaml
|
395
568
|
Groups:
|
@@ -417,9 +590,12 @@ Contracts:
|
|
417
590
|
```
|
418
591
|
|
419
592
|
#### StringSets
|
593
|
+
|
420
594
|
This optional section defines values of type `string_set`.
|
421
|
-
In contrast to `enum` values, which are **statically** defined in the `Schema`, `string_set` values can be **dynamically
|
595
|
+
In contrast to `enum` values, which are **statically** defined in the `Schema`, `string_set` values can be **dynamically
|
596
|
+
** defined in this section.
|
422
597
|
If an agent attribute is of type `string_set` and the attribute is set in the `scenario`, then
|
598
|
+
|
423
599
|
1. the section `StringSets` in the `scenario` must contain an entry named exactly like the attribute, and
|
424
600
|
2. the attribute value must be contained in the string set's `Values` declaration.
|
425
601
|
|
@@ -449,12 +625,15 @@ Agents:
|
|
449
625
|
FuelType: OIL
|
450
626
|
```
|
451
627
|
|
452
|
-
Important: If different types of Agents shall refer to the same StringSet, their attributes in schema must have the
|
628
|
+
Important: If different types of Agents shall refer to the same StringSet, their attributes in schema must have the
|
629
|
+
**exact** same name.
|
453
630
|
|
454
631
|
### CSV files
|
632
|
+
|
455
633
|
TIME_SERIES inputs are not directly fed into the Scenario YAML file.
|
456
634
|
Instead, TIME_SERIES reference a CSV file that can be stored some place else.
|
457
635
|
These CSV files follow a specific structure:
|
636
|
+
|
458
637
|
* They should contain exactly two columns - any other columns are ignored.
|
459
638
|
A warning is raised if more than two non-empty columns are detected.
|
460
639
|
* The first column must be a time stamp in form `YYYY-MM-DD_hh:mm:ss`
|
@@ -473,20 +652,24 @@ Exemplary content of a valid CSV file:
|
|
473
652
|
2016-01-01_00:00:00;42 # optional comment on this particular data point
|
474
653
|
2017-01-01_00:00:00;0.1
|
475
654
|
|
476
|
-
Please refer also to the detailed article about `TimeStamps` in
|
655
|
+
Please refer also to the detailed article about `TimeStamps` in
|
656
|
+
the [FAME-Wiki](https://gitlab.com/fame-framework/wiki/-/wikis/TimeStamp).
|
477
657
|
|
478
658
|
### Split and join multiple YAML files
|
659
|
+
|
479
660
|
The user may include other YAML files into a YAML file to divide the content across files as convenient.
|
480
661
|
We explicitly recommend using this feature for the `Schema` and `Contracts` sections.
|
481
662
|
Otherwise, the scenario.yaml may become crowded.
|
482
663
|
|
483
664
|
#### Command: !Include
|
665
|
+
|
484
666
|
To hint YAML to load the content of another file use `!include "path/relative/to/including/yaml/file.yml"`.
|
485
667
|
You can concatenate !include commands and can use !include in the included file as well.
|
486
668
|
The path to the included file is always relative to the file using the !include command.
|
487
669
|
So with the following file structure
|
488
670
|
|
489
671
|
###### file-structure
|
672
|
+
|
490
673
|
```
|
491
674
|
a.yaml
|
492
675
|
folder/b.yaml
|
@@ -497,24 +680,29 @@ folder/deeper_folder/d.yaml
|
|
497
680
|
the following !include commands work
|
498
681
|
|
499
682
|
###### in a.yaml
|
683
|
+
|
500
684
|
```
|
501
685
|
ToBe: !include "folder/b.yaml"
|
502
686
|
OrNot: !include "folder/deeper_folder/d.yaml"
|
503
687
|
```
|
504
688
|
|
505
689
|
###### in b.yaml
|
690
|
+
|
506
691
|
```
|
507
692
|
ThatIs: !include "c.yaml"
|
508
693
|
TheQuestion: !include "deeper_folder/d.yaml"
|
509
694
|
```
|
510
695
|
|
511
696
|
Provided that
|
697
|
+
|
512
698
|
###### in c.yaml
|
699
|
+
|
513
700
|
```
|
514
701
|
Or: maybe
|
515
702
|
```
|
516
703
|
|
517
704
|
###### d.yaml
|
705
|
+
|
518
706
|
```
|
519
707
|
not: "?"
|
520
708
|
```
|
@@ -522,6 +710,7 @@ not: "?"
|
|
522
710
|
the resulting file would look like this:
|
523
711
|
|
524
712
|
###### THe Joined file a.yaml
|
713
|
+
|
525
714
|
```
|
526
715
|
ToBe:
|
527
716
|
ThatIs:
|
@@ -540,6 +729,7 @@ However, you cannot combine the value returned from !include with other values i
|
|
540
729
|
Thus, the following combinations do not work:
|
541
730
|
|
542
731
|
###### caveats.yml
|
732
|
+
|
543
733
|
```
|
544
734
|
!include "file.yaml" # no key assigned
|
545
735
|
|
@@ -553,12 +743,14 @@ List:
|
|
553
743
|
```
|
554
744
|
|
555
745
|
#### Integrate specific nodes of YAML files
|
746
|
+
|
556
747
|
Instead of including *all* content in the included file, you may also pick a specific node within that file.
|
557
748
|
For this use `!include [<relative/path/to/file.yaml>, Path:To:Field:In:Yaml]`.
|
558
749
|
Here, `:` is used in the node-specifying string to select a sequence of nodes to follow - with custom depth.
|
559
750
|
Consider the following two files:
|
560
751
|
|
561
752
|
###### file_to_be_included.yaml
|
753
|
+
|
562
754
|
```yaml
|
563
755
|
Set1:
|
564
756
|
Subset1:
|
@@ -568,6 +760,7 @@ Set2:
|
|
568
760
|
```
|
569
761
|
|
570
762
|
###### including_file.yaml
|
763
|
+
|
571
764
|
```yaml
|
572
765
|
- Type: MyAgentWithInputs
|
573
766
|
Id: 1
|
@@ -577,6 +770,7 @@ Set2:
|
|
577
770
|
Compiling "including_file.yaml" results in
|
578
771
|
|
579
772
|
###### resulting_file.yaml
|
773
|
+
|
580
774
|
```yaml
|
581
775
|
- Type: MyAgentWithInputs
|
582
776
|
Id: 1
|
@@ -585,14 +779,18 @@ Compiling "including_file.yaml" results in
|
|
585
779
|
```
|
586
780
|
|
587
781
|
#### Load multiple files
|
588
|
-
|
782
|
+
|
783
|
+
Using wildcards in the given path (e.g. "path/to/many/*.yaml") will lead to loading multiple files and assigning their
|
784
|
+
content to the same key.
|
589
785
|
You can make use of this feature with or without specifying a node selector.
|
590
786
|
However, the elements to be joined across multiple files must be lists.
|
591
787
|
These lists are then concatenated into a single list and then assigned to the key in the file calling !include.
|
592
|
-
This feature is especially useful for Contracts: You can split the Contracts list into several files and place them in a
|
788
|
+
This feature is especially useful for Contracts: You can split the Contracts list into several files and place them in a
|
789
|
+
separate folder.
|
593
790
|
Then use !include to re-integrate them into your configuration. An example:
|
594
791
|
|
595
792
|
###### my_contract1.yaml
|
793
|
+
|
596
794
|
```
|
597
795
|
Contracts:
|
598
796
|
- ContractA
|
@@ -600,6 +798,7 @@ Contracts:
|
|
600
798
|
```
|
601
799
|
|
602
800
|
###### my_contract2.yaml
|
801
|
+
|
603
802
|
```
|
604
803
|
Contracts:
|
605
804
|
- ContractC
|
@@ -608,6 +807,7 @@ Contracts:
|
|
608
807
|
```
|
609
808
|
|
610
809
|
###### including_file.yaml
|
810
|
+
|
611
811
|
```
|
612
812
|
Contracts: [!include "my_contract*.yaml", "Contracts"]
|
613
813
|
```
|
@@ -615,6 +815,7 @@ Contracts: [!include "my_contract*.yaml", "Contracts"]
|
|
615
815
|
results in
|
616
816
|
|
617
817
|
###### result.yaml
|
818
|
+
|
618
819
|
```
|
619
820
|
Contracts:
|
620
821
|
- ContractA
|
@@ -625,11 +826,13 @@ Contracts:
|
|
625
826
|
```
|
626
827
|
|
627
828
|
#### Ignoring files
|
829
|
+
|
628
830
|
Files that have their name start with "IGNORE_" are not included with the !include command.
|
629
831
|
You will see a debug output to notify you that the file was ignored.
|
630
832
|
Use this to temporarily take files out ouf your configuration without deleting or moving them.
|
631
833
|
|
632
834
|
## Read FAME results
|
835
|
+
|
633
836
|
Takes an output file in protobuf format of FAME-based applications and converts it into files in CSV format.
|
634
837
|
An individual file for each type of Agent is created in a folder named after the protobuf input file.
|
635
838
|
Call structure:
|
@@ -638,35 +841,33 @@ Call structure:
|
|
638
841
|
|
639
842
|
You may also specify any of the following arguments:
|
640
843
|
|
641
|
-
| Command
|
642
|
-
|
643
|
-
| `-l` or `--log` <option>
|
644
|
-
| `-lf` or `--logfile` <file>
|
645
|
-
| `-a` or `--agents` <list-of-agents>
|
646
|
-
| `-o` or `--output`
|
647
|
-
| `-se` or `--single-export`
|
648
|
-
| `-m` or `--memory-saving`
|
649
|
-
| `-cc` or `--complex-column` <option>
|
650
|
-
| `-t` or `--time` <option>
|
651
|
-
| `--input-recovery` or `--no-input-recovery`
|
652
|
-
|
653
|
-
|
654
|
-
|
655
|
-
|
656
|
-
|
657
|
-
|
658
|
-
|
659
|
-
|
|
660
|
-
|
661
|
-
|
662
|
-
| `-sb` or `--steps-before` | Range of TimeSteps before the `focal-point` they get merged to |
|
663
|
-
| `-sa` or `--steps-after` | Range of TimeSteps after the `focal-point` they get merged to |
|
844
|
+
| Command | Action |
|
845
|
+
|-----------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
846
|
+
| `-l` or `--log` <option> | Sets the logging level. Default is `WARNING`. Options are `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. |
|
847
|
+
| `-lf` or `--logfile` <file> | Sets the logging file. Default is `None`. If `None` is provided, all logs get only printed to the console. |
|
848
|
+
| `-a` or `--agents` <list-of-agents> | If specified, only a subset of agents is extracted from the protobuf file. Default is to extract all agents. |
|
849
|
+
| `-o` or `--output` | Sets the path to where the generated output files are written to. If not specified, the folder's name is derived from the input file's name. Folder will be created if it does not exist. |
|
850
|
+
| `-se` or `--single-export` | Enables export of individual agents to individual files, when present. If not present (the default) one file per `AgentType` is created. |
|
851
|
+
| `-m` or `--memory-saving` | When specified, reduces memory usage profile at the cost of runtime. Use only when necessary. |
|
852
|
+
| `-cc` or `--complex-column` <option> | Defines how to deal with complex indexed output columns (if any). `IGNORE` ignores complex columns. `SPLIT` creates a separate file for each complex indexed output column. |
|
853
|
+
| `-t` or `--time` <option> | Option to define conversion of time steps to given format (default=`UTC`) by `-t/--time {UTC, INT, FAME}` |
|
854
|
+
| `--input-recovery` or `--no-input-recovery` | If True, all input data are recovered in addition to the outputs (default=False). |
|
855
|
+
| `-mt` or `--merge-times` <list-of-parameters> | Option to merge `TimeSteps` of a certain range of steps in the output files to associate multiple time steps with a common logical time in your simulation and reduce number of lines in output files |
|
856
|
+
|
857
|
+
The option `--merge-times` requires exactly three integer arguments separated by spaces:
|
858
|
+
|
859
|
+
| Position | Name | Meaning |
|
860
|
+
|----------|--------------|------------------------------------------------------------------------------------------|
|
861
|
+
| First | Focal point | TimeStep on which `steps-before` earlier and `steps-after` later TimeSteps are merged on |
|
862
|
+
| Second | Steps before | Range of TimeSteps before the `focal-point` they get merged to, must be Zero or positive |
|
863
|
+
| Third | Steps after | Range of TimeSteps after the `focal-point` they get merged to, must be Zero or positive |
|
864
|
+
|
664
865
|
|
665
866
|
This could look as follows:
|
666
867
|
|
667
|
-
convertFameResults -f <./path/to/protobuf_file.pb> -l debug -lf <path/to/output.log> -a AgentType1 AgentType2 -o myCsvFolder -m -cc SPLIT merge-times
|
868
|
+
convertFameResults -f <./path/to/protobuf_file.pb> -l debug -lf <path/to/output.log> -a AgentType1 AgentType2 -o myCsvFolder -m -cc SPLIT --merge-times 0 1799 1800
|
668
869
|
|
669
|
-
Make sure that in the range of time steps you specify for merging there is only one value per column in the merged time range.
|
870
|
+
Make sure that in the range of time steps you specify for merging, there is only one value per column in the merged time range.
|
670
871
|
If multiple values per column are merged values will get concatenated and might yield unexpected results.
|
671
872
|
|
672
873
|
You may also call the conversion script from any Python script with:
|
@@ -677,7 +878,8 @@ from fameio.scripts.convert_results import Options, run as convert_results
|
|
677
878
|
convert_results({Options.FILE: "./path/to/protobuf_file.pb"})
|
678
879
|
```
|
679
880
|
|
680
|
-
Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function
|
881
|
+
Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function
|
882
|
+
call.
|
681
883
|
|
682
884
|
```python
|
683
885
|
from fameio.scripts.convert_results import Options, run as convert_results
|
@@ -701,7 +903,7 @@ You can also use the associated argument parser, to extract the run_config dynam
|
|
701
903
|
|
702
904
|
```python
|
703
905
|
from fameio.scripts.convert_results import Options, run as convert_results
|
704
|
-
from fameio.
|
906
|
+
from fameio.cli.convert_results import handle_args
|
705
907
|
|
706
908
|
my_defaults = {Options.FILE: "./path/to/protobuf_file.pb",
|
707
909
|
Options.LOG_LEVEL: "info",
|
@@ -721,6 +923,7 @@ convert_results(run_config)
|
|
721
923
|
```
|
722
924
|
|
723
925
|
## Cite FAME-Io
|
926
|
+
|
724
927
|
If you use FAME-Io for academic work, please cite as follows.
|
725
928
|
|
726
929
|
Bibtex entry:
|
@@ -736,12 +939,14 @@ Bibtex entry:
|
|
736
939
|
```
|
737
940
|
|
738
941
|
## Available Support
|
942
|
+
|
739
943
|
This is a purely scientific project by (at the moment) one research group.
|
740
944
|
Thus, there is no paid technical support available.
|
741
945
|
However, we will give our best to answer your questions and provide support.
|
742
946
|
|
743
947
|
If you experience any trouble with FAME-Io, you may contact the developers via [fame@dlr.de](mailto:fame@dlr.de).
|
744
|
-
Please report bugs and make feature requests by filing issues following the provided templates (see
|
948
|
+
Please report bugs and make feature requests by filing issues following the provided templates (see
|
949
|
+
also [Contribute](CONTRIBUTING.md)).
|
745
950
|
For substantial enhancements, we recommend that you contact us via [fame@dlr.de](mailto:fame@dlr.de) for working
|
746
951
|
together on the code in common projects or towards common publications and thus further develop FAME-Io.
|
747
952
|
|