fameio 2.3.0__py3-none-any.whl → 3.0.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (63) hide show
  1. CHANGELOG.md +28 -0
  2. fameio/__init__.py +4 -1
  3. fameio/{source/cli → cli}/__init__.py +2 -0
  4. fameio/{source/cli → cli}/convert_results.py +8 -8
  5. fameio/{source/cli → cli}/make_config.py +5 -5
  6. fameio/{source/cli → cli}/options.py +0 -8
  7. fameio/{source/cli → cli}/parser.py +26 -63
  8. fameio/input/__init__.py +27 -0
  9. fameio/input/loader/__init__.py +68 -0
  10. fameio/input/loader/controller.py +129 -0
  11. fameio/input/loader/loader.py +109 -0
  12. fameio/input/metadata.py +149 -0
  13. fameio/input/resolver.py +44 -0
  14. fameio/{source → input}/scenario/__init__.py +1 -2
  15. fameio/{source → input}/scenario/agent.py +24 -38
  16. fameio/input/scenario/attribute.py +203 -0
  17. fameio/{source → input}/scenario/contract.py +50 -61
  18. fameio/{source → input}/scenario/exception.py +8 -13
  19. fameio/{source → input}/scenario/fameiofactory.py +6 -6
  20. fameio/{source → input}/scenario/generalproperties.py +22 -47
  21. fameio/{source → input}/scenario/scenario.py +34 -31
  22. fameio/input/scenario/stringset.py +48 -0
  23. fameio/{source → input}/schema/__init__.py +2 -2
  24. fameio/input/schema/agenttype.py +125 -0
  25. fameio/input/schema/attribute.py +268 -0
  26. fameio/{source → input}/schema/java_packages.py +26 -22
  27. fameio/{source → input}/schema/schema.py +25 -22
  28. fameio/{source → input}/validator.py +32 -35
  29. fameio/{source → input}/writer.py +86 -86
  30. fameio/{source/logs.py → logs.py} +25 -9
  31. fameio/{source/results → output}/agent_type.py +21 -22
  32. fameio/{source/results → output}/conversion.py +34 -31
  33. fameio/{source/results → output}/csv_writer.py +7 -7
  34. fameio/{source/results → output}/data_transformer.py +24 -24
  35. fameio/{source/results → output}/input_dao.py +51 -49
  36. fameio/{source/results → output}/output_dao.py +16 -17
  37. fameio/{source/results → output}/reader.py +30 -31
  38. fameio/{source/results → output}/yaml_writer.py +2 -3
  39. fameio/scripts/__init__.py +2 -2
  40. fameio/scripts/convert_results.py +16 -15
  41. fameio/scripts/make_config.py +9 -9
  42. fameio/{source/series.py → series.py} +28 -26
  43. fameio/{source/time.py → time.py} +8 -8
  44. fameio/{source/tools.py → tools.py} +2 -2
  45. {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/METADATA +277 -72
  46. fameio-3.0.0.dist-info/RECORD +56 -0
  47. fameio/source/__init__.py +0 -8
  48. fameio/source/loader.py +0 -181
  49. fameio/source/metadata.py +0 -32
  50. fameio/source/path_resolver.py +0 -34
  51. fameio/source/scenario/attribute.py +0 -130
  52. fameio/source/scenario/stringset.py +0 -51
  53. fameio/source/schema/agenttype.py +0 -132
  54. fameio/source/schema/attribute.py +0 -203
  55. fameio/source/schema/exception.py +0 -9
  56. fameio-2.3.0.dist-info/RECORD +0 -55
  57. /fameio/{source/results → output}/__init__.py +0 -0
  58. {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSE.txt +0 -0
  59. {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSES/Apache-2.0.txt +0 -0
  60. {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSES/CC-BY-4.0.txt +0 -0
  61. {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/LICENSES/CC0-1.0.txt +0 -0
  62. {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/WHEEL +0 -0
  63. {fameio-2.3.0.dist-info → fameio-3.0.0.dist-info}/entry_points.txt +0 -0
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: fameio
3
- Version: 2.3.0
4
- Summary: Python scripts for operation of FAME models
3
+ Version: 3.0.0
4
+ Summary: Tools for input preparation and output digestion of FAME models
5
5
  Home-page: https://gitlab.com/fame-framework/wiki/-/wikis/home
6
6
  License: Apache-2.0
7
7
  Keywords: FAME,fameio,agent-based modelling,energy systems
@@ -21,7 +21,7 @@ Classifier: Programming Language :: Python :: 3.10
21
21
  Classifier: Programming Language :: Python :: 3.11
22
22
  Classifier: Programming Language :: Python :: 3.12
23
23
  Classifier: Topic :: Scientific/Engineering
24
- Requires-Dist: fameprotobuf (>=1.5.0,<2.0.0)
24
+ Requires-Dist: fameprotobuf (>=2.0.2,<3.0.0)
25
25
  Requires-Dist: pandas (>=1.0,<3.0)
26
26
  Requires-Dist: pyyaml (>=6.0,<7.0)
27
27
  Project-URL: Repository, https://gitlab.com/fame-framework/fame-io/
@@ -42,17 +42,21 @@ SPDX-License-Identifier: Apache-2.0 -->
42
42
  ![GitLab last commit](https://img.shields.io/gitlab/last-commit/fame-framework%2Ffame-io)
43
43
  ![GitLab closed issues by-label](https://img.shields.io/gitlab/issues/closed/fame-framework%2Ffame-io)
44
44
 
45
-
46
45
  # FAME-Io
47
- Python scripts for FAME models, generation of protobuf input files and conversion of protobuf output files.
46
+
47
+ *Tools for input preparation and output digestion of FAME models*
48
+
49
+ FAME-Io compiles input for FAME models in protobuf format and extracts model outputs to human-readable files.
48
50
  Please visit the [FAME-Wiki](https://gitlab.com/fame-framework/wiki/-/wikis/home) to get an explanation of FAME and its components.
49
51
 
50
52
  # Installation
53
+
51
54
  We recommend installing `fameio` using PyPI:
52
55
 
53
56
  pip install fameio
54
57
 
55
- You may also use `pipx`. For detailed information please refer to the official `pipx` [documentation](https://github.com/pypa/pipx).
58
+ You may also use `pipx`. For detailed information please refer to the
59
+ official `pipx` [documentation](https://github.com/pypa/pipx).
56
60
 
57
61
  pipx install fameio
58
62
 
@@ -60,15 +64,21 @@ You may also use `pipx`. For detailed information please refer to the official `
60
64
  See the `pyproject.toml` for a complete listing of dependencies.
61
65
 
62
66
  # Usage
67
+
63
68
  FAME-Io currently offers two main scripts `makeFameRunConfig` and `convertFameResults`.
64
69
  Both are automatically installed with the package.
65
70
  The first one creates a protobuf file for FAME applications using YAML definition files and CSV files.
66
71
  The latter one reads output files from FAME applications in protobuf format and converts them to CSV files.
67
72
 
68
- You may use the [example data](https://gitlab.com/dlr-ve/esy/amiris/examples) provided for the [AMIRIS](https://gitlab.com/dlr-ve/esy/amiris/amiris) model which can be used to simulate electricity markets in [Germany](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Germany2019), [Austria](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Austria2019), and a simple [proof-of-concept model](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Simple).
73
+ You may use the [example data](https://gitlab.com/dlr-ve/esy/amiris/examples) provided for
74
+ the [AMIRIS](https://gitlab.com/dlr-ve/esy/amiris/amiris) model which can be used to simulate electricity markets
75
+ in [Germany](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Germany2019), [Austria](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Austria2019),
76
+ and a simple [proof-of-concept model](https://gitlab.com/dlr-ve/esy/amiris/examples/-/tree/main/Simple).
69
77
 
70
78
  ## Make a FAME run configuration
71
- Digests configuration files in YAML format, combines them with CSV data files and creates a single input file for FAME applications in protobuf format.
79
+
80
+ Digests configuration files in YAML format, combines them with CSV data files and creates a single input file for FAME
81
+ applications in protobuf format.
72
82
  Call structure:
73
83
 
74
84
  makeFameRunConfig -f <path/to/scenario.yaml>
@@ -94,7 +104,8 @@ from fameio.scripts.make_config import Options, run as make_config
94
104
  make_config({Options.FILE: "path/to/scenario.yaml", })
95
105
  ```
96
106
 
97
- Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function call.
107
+ Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function
108
+ call.
98
109
 
99
110
  ```python
100
111
  from fameio.scripts.make_config import Options, run as make_config
@@ -112,13 +123,13 @@ You can also use the associated argument parser, to extract the run_config dynam
112
123
 
113
124
  ```python
114
125
  from fameio.scripts.make_config import Options, run as make_config
115
- from fameio.source.cli.make_config import handle_args
126
+ from fameio.cli.make_config import handle_args
116
127
 
117
128
  my_defaults = {Options.FILE: "path/to/scenario.yaml",
118
- Options.LOG_LEVEL: "info",
119
- Options.OUTPUT: "output.pb",
120
- Options.LOG_FILE: "scenario.log",
121
- }
129
+ Options.LOG_LEVEL: "info",
130
+ Options.OUTPUT: "output.pb",
131
+ Options.LOG_FILE: "scenario.log",
132
+ }
122
133
  my_arg_string = ['-f', 'my/other/scenario.yaml', '-l', 'error']
123
134
 
124
135
  run_config = handle_args(my_arg_string, my_defaults)
@@ -126,16 +137,21 @@ make_config(run_config)
126
137
  ```
127
138
 
128
139
  ### Scenario YAML
140
+
129
141
  The "scenario.yaml" file contains all configuration options for a FAME-based simulation.
130
- It consists of the sections `Schema`, `GeneralProperties`, `Agents` and `Contracts`, and the optional section `StringSets`.
142
+ It consists of the sections `Schema`, `GeneralProperties`, `Agents` and `Contracts`, and the optional
143
+ section `StringSets`.
131
144
  All of them are described below.
132
145
 
133
146
  #### Schema
147
+
134
148
  The Schema describes a model's components such as its types of agents, their inputs, what data they exchange, etc.
135
149
  It is also used to validate the model inputs provided in the `scenario.yaml`.
136
- Since the Schema is valid until the model itself is changed, it is recommended to defined it in a separate file and include the file here.
150
+ Since the Schema is valid until the model itself is changed, it is recommended to defined it in a separate file and
151
+ include the file here.
137
152
 
138
153
  Currently, the schema specifies:
154
+
139
155
  * which type of Agents can be created
140
156
  * what type of input attributes an Agent uses
141
157
  * what type of Products an Agent can send in Contracts, and
@@ -144,15 +160,18 @@ Currently, the schema specifies:
144
160
  The Schema consists of the sections `JavaPackages` and `AgentTypes`.
145
161
 
146
162
  ##### JavaPackages
163
+
147
164
  This section defines the name of the Java packages in which the model code is located.
148
165
  A similar data set was formerly specified in the `fameSetup.yaml`, but is now specified in the schema.
149
- Each of the three sections `Agents`, `DataItems`, and `Portables` contain a list of fully qualified java package names of your model's classes.
166
+ Each of the three sections `Agents`, `DataItems`, and `Portables` contain a list of fully qualified java package names
167
+ of your model's classes.
150
168
  Package names can occur in multiple lists and may overlap.
151
169
  It is not necessary (but possible) to specify the nearest enclosing package for each Agent, DataItem or Portable.
152
170
  Specifying any super-package will also work.
153
171
  Also, package names occur on multiple lists for Agent, DataItem or Portable.
154
172
 
155
173
  For example, for a project with all its
174
+
156
175
  * Agent-derived java classes located in packages below the package named "agents",
157
176
  * DataItem implementation classes in a subpackage named "msg",
158
177
  * Portable implementation classes in a subpackages named "portableItems" and "otherPortables",
@@ -170,8 +189,12 @@ JavaPackages:
170
189
  - "otherPortables"
171
190
  ```
172
191
 
192
+ One can leave out the `DataItems` specifications, but `Agents` and `Portables` are required and must not be empty.
193
+
173
194
  ##### AgentTypes
174
- Here, each type of agent that can be created in your FAME-based application is listed, its attributes and its available Products for Contracts.
195
+
196
+ Here, each type of agent that can be created in your FAME-based application is listed, its attributes and its available
197
+ Products for Contracts.
175
198
  The structure of this section
176
199
 
177
200
  ```yaml
@@ -183,18 +206,49 @@ AgentTypes:
183
206
  MyOtherAttribute:
184
207
  ...
185
208
  Products: [ 'Product1', 'Product2', 'Product3' ]
209
+ Outputs: [ 'Column1', 'Column2', 'Column3' ]
210
+ Metadata:
211
+ Some: "Dict with Metadata that you would like to add"
186
212
  MyOtherAgentWithoutProductsOrAttributes:
187
213
  ```
188
214
 
189
215
  * `MyAgentType` Java's simple class name of the Agent type
190
216
  * `Attributes` indicates that beginning of the attribute definition section for this Agent type
191
- * `MyAttribute` Name of an attribute as specified in the corresponding Java source code of this Agent type (annotated with "@Input")
217
+ * `MyAttribute` Name of an attribute as specified in the corresponding Java source code of this Agent type (annotated
218
+ with "@Input")
192
219
  * `MyOtherAttribute` Name of another attribute derived from Java source code
193
- * `Products` list of Products that this Agent can send in Contracts; derived from Java source code of this Agent type (annotated with "@Product")
220
+ * `Products` list or dictionary of Products that this Agent can send in Contracts; derived from Java source code of this
221
+ Agent type (annotated with "@Product")
222
+ * `Outputs` list or dictionary of Output columns that this Agent can write to; derived from Java source code of this
223
+ Agent type (annotated with "@Output")
224
+ * `Metadata` dictionary with any content that is assigned to this Agent type as additional information
194
225
  * `MyOtherAgentWithoutProductsOrAttributes` an Agent type that requires neither Attributes nor Products
195
226
 
196
- Both Attributes and Products are optional - there could be useful Agents that require neither of them.
197
- In the above example attribute definition was not shown (indicated by `...`).
227
+ Attributes, Products, Outputs and Metadata are optional - there may be useful Agents that require none of them.
228
+ Products and Outputs can both be lists of Strings, or dictionaries with additional Metadata.
229
+ For example, you could write the above in the following way:
230
+
231
+ ```yaml
232
+ Products:
233
+ Product1:
234
+ Metadata:
235
+ Any: "information you would like to add to Product1 using a dictionary form"
236
+ Product2:
237
+ Product3:
238
+ Outputs:
239
+ Column1:
240
+ Column2:
241
+ ThisEntry: "is ignored, as it is not below the keyword: 'Metadata'"
242
+ Metadata:
243
+ My: "Metadata"
244
+ That: "will be saved to Column2"
245
+ Column3:
246
+ ```
247
+
248
+ Here, "Product1" and "Column2" have additional, optional Metadata assigned to them (using the keyword "Metadata").
249
+ The other Products and Columns have no metadata assigned to them - which is also ok.
250
+
251
+ In the AgentType definition example above attribute definition was not shown explicitly (indicated by `...`).
198
252
  The next example provides details on how to define an attribute:
199
253
 
200
254
  ```yaml
@@ -205,29 +259,48 @@ MySimpleAttribute:
205
259
  Values: [ 'AllowedValue1', 'AllowedValue2' ]
206
260
  Default: 'AllowedValue1'
207
261
  Help: 'My help text'
262
+ Metadata:
263
+ Go: "here"
208
264
 
209
265
  MyComplexAttribute:
210
266
  AttributeType: block
211
267
  NestedAttributes:
212
268
  InnerAttributeA:
213
269
  AttributeType: integer
270
+ Values:
271
+ 1:
272
+ Metadata:
273
+ Explain: "1 is a allowed value"
274
+ 2:
275
+ Metadata:
276
+ Comment: "2 is also allowed, but consider using 1"
214
277
  InnerAttributeB:
215
278
  AttributeType: double
216
279
  ```
217
280
 
218
- * `MySimpleAttribute`, `MyDoubleList`, `MyComplexAttribute` Names of the attributes as specified in the Java enum annotated with "@Input"
281
+ * `MySimpleAttribute`, `MyDoubleList`, `MyComplexAttribute` Names of the attributes as specified in the Java enum
282
+ annotated with "@Input"
219
283
  * `AttributeType` (required) data type of the attribute; see options in table below
220
- * `Mandatory` (optional - true by default) if true: the attribute is required for this agent and validation will fail if the attribute is missing in the scenario **and** no default is provided
284
+ * `Mandatory` (optional - true by default) if true: the attribute is required for this agent and validation will fail if
285
+ the attribute is missing in the scenario **and** no default is provided
221
286
  * `List` (optional - false by default)
222
287
  * `AttributeType: time_series` cannot be true
223
288
  * `AttributeType: block`
224
289
  * if true: any nested element in the scenario must be part of a list element and thus can appear multiple times
225
290
  * if false: any nested element in the scenario can only appear once
226
- * any other AttributeType: the attribute is interpreted as list, i.e. multiple values can be assigned to this attribute in the scenario
227
- * `NestedAttributes` (required only if `AttributeType: block`, otherwise disallowed) starts an inner Attribute definition block - defined Attributes are sub-elements of `MyComplexAttribute`
228
- * `Values` (optional - None by default): if present defines a list of allowed values for this attribute
229
- * `Default` (optional - None by default): if present defines a default value to be used in case the scenario does not specify it
230
- * `Help` (optional - None by default): if present defines a help text to you attribute
291
+ * any other AttributeType: the attribute is interpreted as list, i.e. multiple values can be assigned to this
292
+ attribute in the scenario
293
+ * `NestedAttributes` (required only if `AttributeType: block`, otherwise disallowed) starts an inner Attribute
294
+ definition block - defined Attributes are sub-elements of `MyComplexAttribute`
295
+ * `Values` (optional - None by default):
296
+ * if present, defines a list or dictionary of allowed values for this attribute
297
+ * if a dictionary is used, individual Metadata can be assigned to each allowed value using the `Metadata` keyword
298
+ * `Default` (optional - None by default):
299
+ * if present, defines a default value to be used if the scenario does not specify one
300
+ * must match one of the entries in `Values` in case those are defined
301
+ * can be a list if the attribute is a list
302
+ * `Help` (optional - None by default): if present, defines a help text for your Attribute
303
+ * `Metadata` (optional - None by default): if present, defines additional metadata assigned to the Attribute
231
304
 
232
305
  | AttributeType | value |
233
306
  |---------------|-------------------------------------------------------------------------------------------------------------------------|
@@ -251,23 +324,18 @@ GeneralProperties:
251
324
  StartTime: 2011-12-31_23:58:00
252
325
  StopTime: 2012-12-30_23:58:00
253
326
  RandomSeed: 1
254
- Output:
255
- Interval: 100
256
- Process: 0
257
327
  ```
258
328
 
259
329
  Parameters:
330
+
260
331
  * `RunId` an ID that can be given to the simulation; use at your discretion
261
332
  * `StartTime` time stamp in the format YYYY-MM-DD_hh:mm:ss; first moment of the simulation.
262
333
  * `StopTime` time stamp in the format YYYY-MM-DD_hh:mm:ss; last moment of the simulation - i.e. simulation terminates
263
334
  after passing that time stamp
264
335
  * `RandomSeed` seed to initialise random number generation; each value leads to a unique series of random numbers.
265
336
 
266
- Parameters in section `Output` are deprecated and will be removed in FAME-Io v3.0
267
- * `Interval` number of simulation ticks in between write-to-disk events; may be used for performance optimisations;
268
- * `Process` id of process that performs write-to-disk operations; leave at 0 to be compatible with single-processes;
269
-
270
337
  #### Agents
338
+
271
339
  Specifies all Agents to be created in the simulation in a list. Each Agent has its own entry.
272
340
  Structure:
273
341
 
@@ -280,17 +348,21 @@ Agents:
280
348
  MyInteger: 2
281
349
  MyDouble: 4.2
282
350
  MyTimeSeries: "./path/to/time_series.csv"
351
+ Metadata:
352
+ Can: "also be assigned"
283
353
 
284
354
  - Type: MyAgentWithoutInputs
285
355
  Id: 2
286
356
  ```
287
357
 
288
358
  Agent Parameters:
359
+
289
360
  * `Type` Mandatory; Java's simple class name of the agent to be created
290
361
  * `Id` Mandatory; simulation-unique id of this agent; if two agents have the same ID, the configuration process will
291
362
  stop.
292
363
  * `Attributes` Optional; if the agent has any attributes, specify them here in the format "AttributeName: value"; please
293
364
  see attribute table above
365
+ * `Metadata` Optional; can be assigned to each instance of an Agent, as well as to each of its Attributes
294
366
 
295
367
  The specified `Attributes` for each agent must match the specified `Attributes` options in the linked Schema (see above).
296
368
  For better structure and readability of the `scenario.yaml`, `Attributes` may also be specified in a nested way as demonstrated below.
@@ -323,7 +395,102 @@ Attributes:
323
395
 
324
396
  Here, `MyDoubleList` and `MyListGroup` need to specify `List: true` in the corresponding Schema.
325
397
  The shorter `[]`-notation was used to assign a list of floating-point values to `MyDoubleList`.
326
- Nested items `IntValueA` and `IntValueB` of `MyListGroup` are assigned within a list, allowing the specification of these nested items several times.
398
+ Nested items `IntValueA` and `IntValueB` of `MyListGroup` are assigned within a list, allowing the specification of
399
+ these nested items several times.
400
+
401
+ ##### Attribute Metadata
402
+ Metadata can be assigned to any value, list item, or superstructure.
403
+ To assign Metadata to a primitive value, create a dictionary from it, set the actual value with the inner keyword `Value` and add the keyword `Metadata` like this:
404
+
405
+ ```yaml
406
+ ValueWithoutMetadata: 1
407
+ SameValueWithMetadata:
408
+ Value: 1
409
+ Metadata: # describe `SameValueWithMetadata` herein
410
+ ```
411
+
412
+ You can assign Metadata to a list of primitive values using the keyword `Values` like this:
413
+
414
+ ```yaml
415
+ ValueListWithoutMetadata: [1,2,3]
416
+ SameValueListWithListMetadata:
417
+ Values: [1,2,3]
418
+ Metadata: # describe the whole list of values with Metadata here
419
+ ```
420
+
421
+ or specify Metadata for each (or just some) value individually, like this:
422
+
423
+ ```yaml
424
+ ValueListWithoutMetadata: [1,2,3]
425
+ SameValueListWithMetadataAtEachElement:
426
+ - Value: 1
427
+ Metadata: # describe this specific value "1" with Metadata here
428
+ - Value: 2 # this value has no Metadata attached, but you can still use the keyword `Value`
429
+ - 3 # or use in the actual directly since this value has no Metadata anyway
430
+ ```
431
+
432
+ or assign Metadata to both the list and any of its list entries, like this:
433
+
434
+ ```yaml
435
+ ValueListWithoutMetadata: [1,2,3]
436
+ SameValueListWithAllMetadata:
437
+ Metadata: # Recommendation: place the Metadata of the list first if the list of values is extensive, as in this case
438
+ Values:
439
+ - Value: 1
440
+ Metadata: # describe this specific value "1" with Metadata here
441
+ - Value: 2
442
+ Metadata: # describe this specific value "2" with Metadata here
443
+ - Value: 3
444
+ Metadata: # describe this specific value "3" with Metadata here
445
+ ```
446
+
447
+ You can assign Metadata directly to a nested element by adding the Metadata keyword:
448
+
449
+ ```yaml
450
+ NestedItemWithoutMetadata:
451
+ A: 1
452
+ B: 2
453
+ SameNestedItemWithMetadata:
454
+ A: 1
455
+ B: 2
456
+ Metadata: # These Metadata describe `SameNestedItemWithMetadata`
457
+ ```
458
+
459
+ Similar to lists of values, you can assign Metadata to a list of nested elements using the `Values` keyword, like this:
460
+
461
+ ```yaml
462
+ ListOfNestedItemsWithoutMetadata:
463
+ - A: 1
464
+ B: 10
465
+ - A: 2
466
+ B: 20
467
+ SameListOfNestedItemsWithGeneralMetadata:
468
+ Values:
469
+ - A: 1
470
+ B: 10
471
+ - A: 2
472
+ B: 20
473
+ Metadata: # These Metadata describe `SameListOfNestedItemsWithGeneralMetadata` as a whole
474
+ ```
475
+
476
+ and, similar to nested elements, you can assign Metadata directly to any list element, like this:
477
+
478
+ ```yaml
479
+ ListOfNestedItemsWithoutMetadata:
480
+ - A: 1
481
+ B: 10
482
+ - A: 2
483
+ B: 20
484
+ SameListOfNestedItemsWithGeneralMetadata:
485
+ - A: 1
486
+ B: 10
487
+ Metadata: # These Metadata describe the first list item
488
+ - A: 2
489
+ B: 20
490
+ Metadata: # These Metadata describe the second list item
491
+ ```
492
+
493
+ Again, you may apply both variants and apply Metadata to the list and each of its items if you wish.
327
494
 
328
495
  #### Contracts
329
496
  Specifies all Contracts, i.e. repetitive bilateral transactions in between agents.
@@ -337,6 +504,8 @@ Contracts:
337
504
  ProductName: ProductOfAgent_1
338
505
  FirstDeliveryTime: -25
339
506
  DeliveryIntervalInSteps: 3600
507
+ Metadata:
508
+ Some: "additional information can go here"
340
509
 
341
510
  - SenderId: 2
342
511
  ReceiverId: 1
@@ -349,17 +518,20 @@ Contracts:
349
518
  ```
350
519
 
351
520
  Contract Parameters:
521
+
352
522
  * `SenderId` unique ID of agent sending the product
353
523
  * `ReceiverId` unique ID of agent receiving the product
354
524
  * `ProductName` name of the product to be sent
355
525
  * `FirstDeliveryTime` first time of delivery in the format "seconds after the January 1st 2000, 00:00:00"
356
526
  * `DeliveryIntervalInSteps` delay time in between deliveries in seconds
527
+ * `Metadata` can be assigned to add further helpful information about a Contract
357
528
  * `Attributes` can be set to include additional information as `int`, `float`, `enum`, or `dict` data types
358
529
 
359
530
  ##### Definition of Multiple Similar Contracts
360
531
  Often, scenarios contain multiple agents of similar type that also have similar chains of contracts.
361
532
  Therefore, FAME-Io supports a compact definition of multiple similar contracts.
362
- `SenderId` and `ReceiverId` can both be lists and support One-to-N, N-to-One and N-to-N relations like in the following example:
533
+ `SenderId` and `ReceiverId` can both be lists and support One-to-N, N-to-One and N-to-N relations like in the following
534
+ example:
363
535
 
364
536
  ```yaml
365
537
  Contracts:
@@ -389,7 +561,8 @@ Contracts:
389
561
  ```
390
562
 
391
563
  Combined with YAML anchors complex contract chains can be easily reduced to a minimum of required configuration.
392
- The following example is equivalent to the previous one and allows a quick extension of contracts to a new couple of agents e.g. (4;14):
564
+ The following example is equivalent to the previous one and allows a quick extension of contracts to a new couple of
565
+ agents e.g. (4;14):
393
566
 
394
567
  ```yaml
395
568
  Groups:
@@ -417,9 +590,12 @@ Contracts:
417
590
  ```
418
591
 
419
592
  #### StringSets
593
+
420
594
  This optional section defines values of type `string_set`.
421
- In contrast to `enum` values, which are **statically** defined in the `Schema`, `string_set` values can be **dynamically** defined in this section.
595
+ In contrast to `enum` values, which are **statically** defined in the `Schema`, `string_set` values can be **dynamically
596
+ ** defined in this section.
422
597
  If an agent attribute is of type `string_set` and the attribute is set in the `scenario`, then
598
+
423
599
  1. the section `StringSets` in the `scenario` must contain an entry named exactly like the attribute, and
424
600
  2. the attribute value must be contained in the string set's `Values` declaration.
425
601
 
@@ -449,12 +625,15 @@ Agents:
449
625
  FuelType: OIL
450
626
  ```
451
627
 
452
- Important: If different types of Agents shall refer to the same StringSet, their attributes in schema must have the **exact** same name.
628
+ Important: If different types of Agents shall refer to the same StringSet, their attributes in schema must have the
629
+ **exact** same name.
453
630
 
454
631
  ### CSV files
632
+
455
633
  TIME_SERIES inputs are not directly fed into the Scenario YAML file.
456
634
  Instead, TIME_SERIES reference a CSV file that can be stored some place else.
457
635
  These CSV files follow a specific structure:
636
+
458
637
  * They should contain exactly two columns - any other columns are ignored.
459
638
  A warning is raised if more than two non-empty columns are detected.
460
639
  * The first column must be a time stamp in form `YYYY-MM-DD_hh:mm:ss`
@@ -473,20 +652,24 @@ Exemplary content of a valid CSV file:
473
652
  2016-01-01_00:00:00;42 # optional comment on this particular data point
474
653
  2017-01-01_00:00:00;0.1
475
654
 
476
- Please refer also to the detailed article about `TimeStamps` in the [FAME-Wiki](https://gitlab.com/fame-framework/wiki/-/wikis/TimeStamp).
655
+ Please refer also to the detailed article about `TimeStamps` in
656
+ the [FAME-Wiki](https://gitlab.com/fame-framework/wiki/-/wikis/TimeStamp).
477
657
 
478
658
  ### Split and join multiple YAML files
659
+
479
660
  The user may include other YAML files into a YAML file to divide the content across files as convenient.
480
661
  We explicitly recommend using this feature for the `Schema` and `Contracts` sections.
481
662
  Otherwise, the scenario.yaml may become crowded.
482
663
 
483
664
  #### Command: !Include
665
+
484
666
  To hint YAML to load the content of another file use `!include "path/relative/to/including/yaml/file.yml"`.
485
667
  You can concatenate !include commands and can use !include in the included file as well.
486
668
  The path to the included file is always relative to the file using the !include command.
487
669
  So with the following file structure
488
670
 
489
671
  ###### file-structure
672
+
490
673
  ```
491
674
  a.yaml
492
675
  folder/b.yaml
@@ -497,24 +680,29 @@ folder/deeper_folder/d.yaml
497
680
  the following !include commands work
498
681
 
499
682
  ###### in a.yaml
683
+
500
684
  ```
501
685
  ToBe: !include "folder/b.yaml"
502
686
  OrNot: !include "folder/deeper_folder/d.yaml"
503
687
  ```
504
688
 
505
689
  ###### in b.yaml
690
+
506
691
  ```
507
692
  ThatIs: !include "c.yaml"
508
693
  TheQuestion: !include "deeper_folder/d.yaml"
509
694
  ```
510
695
 
511
696
  Provided that
697
+
512
698
  ###### in c.yaml
699
+
513
700
  ```
514
701
  Or: maybe
515
702
  ```
516
703
 
517
704
  ###### d.yaml
705
+
518
706
  ```
519
707
  not: "?"
520
708
  ```
@@ -522,6 +710,7 @@ not: "?"
522
710
  the resulting file would look like this:
523
711
 
524
712
  ###### THe Joined file a.yaml
713
+
525
714
  ```
526
715
  ToBe:
527
716
  ThatIs:
@@ -540,6 +729,7 @@ However, you cannot combine the value returned from !include with other values i
540
729
  Thus, the following combinations do not work:
541
730
 
542
731
  ###### caveats.yml
732
+
543
733
  ```
544
734
  !include "file.yaml" # no key assigned
545
735
 
@@ -553,12 +743,14 @@ List:
553
743
  ```
554
744
 
555
745
  #### Integrate specific nodes of YAML files
746
+
556
747
  Instead of including *all* content in the included file, you may also pick a specific node within that file.
557
748
  For this use `!include [<relative/path/to/file.yaml>, Path:To:Field:In:Yaml]`.
558
749
  Here, `:` is used in the node-specifying string to select a sequence of nodes to follow - with custom depth.
559
750
  Consider the following two files:
560
751
 
561
752
  ###### file_to_be_included.yaml
753
+
562
754
  ```yaml
563
755
  Set1:
564
756
  Subset1:
@@ -568,6 +760,7 @@ Set2:
568
760
  ```
569
761
 
570
762
  ###### including_file.yaml
763
+
571
764
  ```yaml
572
765
  - Type: MyAgentWithInputs
573
766
  Id: 1
@@ -577,6 +770,7 @@ Set2:
577
770
  Compiling "including_file.yaml" results in
578
771
 
579
772
  ###### resulting_file.yaml
773
+
580
774
  ```yaml
581
775
  - Type: MyAgentWithInputs
582
776
  Id: 1
@@ -585,14 +779,18 @@ Compiling "including_file.yaml" results in
585
779
  ```
586
780
 
587
781
  #### Load multiple files
588
- Using wildcards in the given path (e.g. "path/to/many/*.yaml") will lead to loading multiple files and assigning their content to the same key.
782
+
783
+ Using wildcards in the given path (e.g. "path/to/many/*.yaml") will lead to loading multiple files and assigning their
784
+ content to the same key.
589
785
  You can make use of this feature with or without specifying a node selector.
590
786
  However, the elements to be joined across multiple files must be lists.
591
787
  These lists are then concatenated into a single list and then assigned to the key in the file calling !include.
592
- This feature is especially useful for Contracts: You can split the Contracts list into several files and place them in a separate folder.
788
+ This feature is especially useful for Contracts: You can split the Contracts list into several files and place them in a
789
+ separate folder.
593
790
  Then use !include to re-integrate them into your configuration. An example:
594
791
 
595
792
  ###### my_contract1.yaml
793
+
596
794
  ```
597
795
  Contracts:
598
796
  - ContractA
@@ -600,6 +798,7 @@ Contracts:
600
798
  ```
601
799
 
602
800
  ###### my_contract2.yaml
801
+
603
802
  ```
604
803
  Contracts:
605
804
  - ContractC
@@ -608,6 +807,7 @@ Contracts:
608
807
  ```
609
808
 
610
809
  ###### including_file.yaml
810
+
611
811
  ```
612
812
  Contracts: [!include "my_contract*.yaml", "Contracts"]
613
813
  ```
@@ -615,6 +815,7 @@ Contracts: [!include "my_contract*.yaml", "Contracts"]
615
815
  results in
616
816
 
617
817
  ###### result.yaml
818
+
618
819
  ```
619
820
  Contracts:
620
821
  - ContractA
@@ -625,11 +826,13 @@ Contracts:
625
826
  ```
626
827
 
627
828
  #### Ignoring files
829
+
628
830
  Files that have their name start with "IGNORE_" are not included with the !include command.
629
831
  You will see a debug output to notify you that the file was ignored.
630
832
  Use this to temporarily take files out ouf your configuration without deleting or moving them.
631
833
 
632
834
  ## Read FAME results
835
+
633
836
  Takes an output file in protobuf format of FAME-based applications and converts it into files in CSV format.
634
837
  An individual file for each type of Agent is created in a folder named after the protobuf input file.
635
838
  Call structure:
@@ -638,35 +841,33 @@ Call structure:
638
841
 
639
842
  You may also specify any of the following arguments:
640
843
 
641
- | Command | Action |
642
- |---------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
643
- | `-l` or `--log` <option> | Sets the logging level. Default is `WARNING`. Options are `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. |
644
- | `-lf` or `--logfile` <file> | Sets the logging file. Default is `None`. If `None` is provided, all logs get only printed to the console. |
645
- | `-a` or `--agents` <list-of-agents> | If specified, only a subset of agents is extracted from the protobuf file. Default is to extract all agents. |
646
- | `-o` or `--output` | Sets the path to where the generated output files are written to. If not specified, the folder's name is derived from the input file's name. Folder will be created if it does not exist. |
647
- | `-se` or `--single-export` | Enables export of individual agents to individual files, when present. If not present (the default) one file per `AgentType` is created. |
648
- | `-m` or `--memory-saving` | When specified, reduces memory usage profile at the cost of runtime. Use only when necessary. |
649
- | `-cc` or `--complex-column` <option> | Defines how to deal with complex indexed output columns (if any). `IGNORE` ignores complex columns. `SPLIT` creates a separate file for each complex indexed output column. |
650
- | `-t` or `--time` <option> | Option to define conversion of time steps to given format (default=`UTC`) by `-t/--time {UTC, INT, FAME}` |
651
- | `--input-recovery` or `--no-input-recovery` | If True, all input data are recovered as well as the outputs (default=False). |
652
-
653
- Additionally, you may merge `TimeSteps` of a certain range of steps in the output files to
654
- i) associate multiple time steps with a common logical time in your simulation
655
- ii) reduce number of lines in output files
656
-
657
- For this, add the option `merge-times` and specify the arguments as follows:
658
-
659
- | Command | Action |
660
- |---------------------------|------------------------------------------------------------------------------------------|
661
- | `-fp` or `--focal-point` | TimeStep on which `steps-before` earlier and `steps-after` later TimeSteps are merged on |
662
- | `-sb` or `--steps-before` | Range of TimeSteps before the `focal-point` they get merged to |
663
- | `-sa` or `--steps-after` | Range of TimeSteps after the `focal-point` they get merged to |
844
+ | Command | Action |
845
+ |-----------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
846
+ | `-l` or `--log` <option> | Sets the logging level. Default is `WARNING`. Options are `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. |
847
+ | `-lf` or `--logfile` <file> | Sets the logging file. Default is `None`. If `None` is provided, all logs get only printed to the console. |
848
+ | `-a` or `--agents` <list-of-agents> | If specified, only a subset of agents is extracted from the protobuf file. Default is to extract all agents. |
849
+ | `-o` or `--output` | Sets the path to where the generated output files are written to. If not specified, the folder's name is derived from the input file's name. Folder will be created if it does not exist. |
850
+ | `-se` or `--single-export` | Enables export of individual agents to individual files, when present. If not present (the default) one file per `AgentType` is created. |
851
+ | `-m` or `--memory-saving` | When specified, reduces memory usage profile at the cost of runtime. Use only when necessary. |
852
+ | `-cc` or `--complex-column` <option> | Defines how to deal with complex indexed output columns (if any). `IGNORE` ignores complex columns. `SPLIT` creates a separate file for each complex indexed output column. |
853
+ | `-t` or `--time` <option> | Option to define conversion of time steps to given format (default=`UTC`) by `-t/--time {UTC, INT, FAME}` |
854
+ | `--input-recovery` or `--no-input-recovery` | If True, all input data are recovered in addition to the outputs (default=False). |
855
+ | `-mt` or `--merge-times` <list-of-parameters> | Option to merge `TimeSteps` of a certain range of steps in the output files to associate multiple time steps with a common logical time in your simulation and reduce number of lines in output files |
856
+
857
+ The option `--merge-times` requires exactly three integer arguments separated by spaces:
858
+
859
+ | Position | Name | Meaning |
860
+ |----------|--------------|------------------------------------------------------------------------------------------|
861
+ | First | Focal point | TimeStep on which `steps-before` earlier and `steps-after` later TimeSteps are merged on |
862
+ | Second | Steps before | Range of TimeSteps before the `focal-point` they get merged to, must be Zero or positive |
863
+ | Third | Steps after | Range of TimeSteps after the `focal-point` they get merged to, must be Zero or positive |
864
+
664
865
 
665
866
  This could look as follows:
666
867
 
667
- convertFameResults -f <./path/to/protobuf_file.pb> -l debug -lf <path/to/output.log> -a AgentType1 AgentType2 -o myCsvFolder -m -cc SPLIT merge-times -fp 0 -sb 1799 -sa 1800
868
+ convertFameResults -f <./path/to/protobuf_file.pb> -l debug -lf <path/to/output.log> -a AgentType1 AgentType2 -o myCsvFolder -m -cc SPLIT --merge-times 0 1799 1800
668
869
 
669
- Make sure that in the range of time steps you specify for merging there is only one value per column in the merged time range.
870
+ Make sure that in the range of time steps you specify for merging, there is only one value per column in the merged time range.
670
871
  If multiple values per column are merged values will get concatenated and might yield unexpected results.
671
872
 
672
873
  You may also call the conversion script from any Python script with:
@@ -677,7 +878,8 @@ from fameio.scripts.convert_results import Options, run as convert_results
677
878
  convert_results({Options.FILE: "./path/to/protobuf_file.pb"})
678
879
  ```
679
880
 
680
- Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function call.
881
+ Similar to the console call you may also specify custom run config arguments and add it in a dictionary to the function
882
+ call.
681
883
 
682
884
  ```python
683
885
  from fameio.scripts.convert_results import Options, run as convert_results
@@ -701,7 +903,7 @@ You can also use the associated argument parser, to extract the run_config dynam
701
903
 
702
904
  ```python
703
905
  from fameio.scripts.convert_results import Options, run as convert_results
704
- from fameio.source.cli.convert_results import handle_args
906
+ from fameio.cli.convert_results import handle_args
705
907
 
706
908
  my_defaults = {Options.FILE: "./path/to/protobuf_file.pb",
707
909
  Options.LOG_LEVEL: "info",
@@ -721,6 +923,7 @@ convert_results(run_config)
721
923
  ```
722
924
 
723
925
  ## Cite FAME-Io
926
+
724
927
  If you use FAME-Io for academic work, please cite as follows.
725
928
 
726
929
  Bibtex entry:
@@ -736,12 +939,14 @@ Bibtex entry:
736
939
  ```
737
940
 
738
941
  ## Available Support
942
+
739
943
  This is a purely scientific project by (at the moment) one research group.
740
944
  Thus, there is no paid technical support available.
741
945
  However, we will give our best to answer your questions and provide support.
742
946
 
743
947
  If you experience any trouble with FAME-Io, you may contact the developers via [fame@dlr.de](mailto:fame@dlr.de).
744
- Please report bugs and make feature requests by filing issues following the provided templates (see also [Contribute](CONTRIBUTING.md)).
948
+ Please report bugs and make feature requests by filing issues following the provided templates (see
949
+ also [Contribute](CONTRIBUTING.md)).
745
950
  For substantial enhancements, we recommend that you contact us via [fame@dlr.de](mailto:fame@dlr.de) for working
746
951
  together on the code in common projects or towards common publications and thus further develop FAME-Io.
747
952