@grnsft/if 0.3.4 → 0.4.0-beta.0

package/.commitlintrc.js

@@ -27,7 +27,6 @@ module.exports = {
         'package',
         'config',
         'mocks',
-        'examples',
         '.github',
         '.husky',
         'scripts',

package/.lintstagedrc.js

@@ -0,0 +1,6 @@
+module.exports = {
+  '*.ts': [
+    'gts lint',
+  ],
+  'package.json': 'fixpack',
+}

package/README.md

@@ -1,38 +1,36 @@
 # Impact Framework
 
+[![Node.js CI](https://github.com/Green-Software-Foundation/if/actions/workflows/nodejs-ci.yml/badge.svg)](https://github.com/Green-Software-Foundation/if/actions/workflows/nodejs-ci.yml)
 
 > [!IMPORTANT]
 > Incubation Project: This project is an incubation project being run inside the Green Software Foundation; as such, we DON’T recommend using it in any critical use case. Incubation projects are experimental, offer no support guarantee, have minimal governance and process, and may be retired at any moment. This project may one day Graduate, in which case this disclaimer will be removed.
 
 **Note** We have recently (March 2024) refactored the IF codebase and introduced some changes affecting both users and developers. You can read our migration guide [HERE](./Refactor-migration-guide.md) to help you update to the latest version!
 
-
 [Impact Framework](https://greensoftwarefoundation.atlassian.net/wiki/spaces/~612dd45e45cd76006a84071a/pages/17072136/Opensource+Impact+Engine+Framework) (IF) is an [Incubation](https://oc.greensoftware.foundation/project-lifecycle.html#incubation) project from the [Open Source Working Group](https://greensoftwarefoundation.atlassian.net/wiki/spaces/~612dd45e45cd76006a84071a/pages/852049/Open+Source+Working+Group) in the [Green Software Foundation](https://greensoftware.foundation/).
 
-
 **Our documentation is online at [if.greensoftware.foundation](https://if.greensoftware.foundation/)**
 
-
 **IF** is a framework to **M**odel, **M**easure, si**M**ulate and **M**onitor the environmental impacts of software
 
 Modern applications are composed of many smaller pieces of software (components) running on many different environments, for example, private cloud, public cloud, bare-metal, virtualized, containerized, mobile, laptops, and desktops.
 
-Every environment requires a different plugin of measurement, and there is no single solution you can use to calculate the environmental impacts for all components across all environments.      
+Every environment requires a different plugin of measurement, and there is no single solution you can use to calculate the environmental impacts for all components across all environments.
 
 The friction to measuring software emissions isn't that we need to know how, it's that we run software on many things and each thing has several different ways to measure.
 
 Read the [specification and design docs](https://if.greensoftware.foundation) to begin.
 
-
 ## Get started
 
-The first thing to understand is that IF is a framework for running plugins. This means that in order to do some calculations, you need to load some plugins from some external resource. We provide a [standard library of plugins](https://github.com/Green-Software-Foundation/if-plugins) and a repository of [community plugins](https://github.com/Green-Software-Foundation/if-unofficial-plugins) to get you started. 
+The first thing to understand is that IF is a framework for running plugins. This means that in order to do some calculations, you need to load some plugins from some external resource. We provide a [standard library of plugins](https://github.com/Green-Software-Foundation/if-plugins) and a repository of [community plugins](https://github.com/Green-Software-Foundation/if-unofficial-plugins) to get you started.
 
 Start by installing framework itself:
 
 ```sh
 npm install -g "@grnsft/if"
 ```
+
 Then installing some plugins:
 
 ```sh
@@ -50,6 +48,7 @@ ie -m <path-to-your-manifest-file>
 ```
 
 Note that above command will not print the final output. In order to print the final output to the console, run `if` using the optional stdout argument:
+
 ```sh
 ie --manifest <path-to-your-manifest-file> --stdout
 ## or using aliases
@@ -74,6 +73,12 @@ ie -m <path-to-your-manifest-file> -o <your-savepath>
 
 The `ie` CLI tool will configure and run the plugins defined in your input `yaml` (`manifest`) and return the results as an output `yaml` (`output`).
 
+Use the `debug` command if you want to diagnose and fix errors in your plugin:
+
+```sh
+ie --manifest <path-to-your-manifest-file> --debug
+```
+
 Use the `help` command if you need guidance about the available commands
 
 ```sh
@@ -92,10 +97,9 @@ Watch this video to learn how to create and run a `manifest`.
 
 [![Watch the walk-through video](https://i3.ytimg.com/vi/R-6eDM8AsvY/maxresdefault.jpg)](https://youtu.be/GW37Qd4AQbU)
 
-
 ## Contributing
 
-To contribute to IF, please fork this repository and raise a pull request from your fork. 
+To contribute to IF, please fork this repository and raise a pull request from your fork.
 
 You can check our issue board for issues tagged `help-wanted`. These are issues that are not currently, actively being worked on by the core team but are well-scoped enough for someone to pick up. We recommend commenting on the issue to start a chat with the core team, then start working on the issue when you have been assigned to it. This process helps to ensure your work is aligned with our roadmap and makes it much more likely that your changes will get merged compared to unsolicited PRs.
 

package/Refactor-migration-guide.md

@@ -116,9 +116,9 @@ There have also been some changes to the structure of manifest files. Some of th
     For example:
     
     ```yaml
-    "sci-m":
-      path: "@grnsft/if-plugins"
-      method: SciM
+    "sci-embodied":
+      path: "builtin"
+      method: SciEmbodied
     ```
 
 - **Global config**
@@ -150,7 +150,7 @@ There have also been some changes to the structure of manifest files. Some of th
           pipeline:
             - teads-curve
             - sci-e
-            - sci-m
+            - sci-embodied
             - sci-o
             - time-sync
             - sci
@@ -174,7 +174,7 @@ There have also been some changes to the structure of manifest files. Some of th
           pipeline:
             - teads-curve
             - sci-e
-            - sci-m
+            - sci-embodied
             - sci-o
             - time-sync
             - sci
@@ -254,7 +254,7 @@ tree:
       pipeline:
         - teads-curve
         - sci-e
-        - sci-m
+        - sci-embodied
         - sci-o
         - time-sync
         - group-by

package/github-processes.md

@@ -3,7 +3,7 @@
 
 - [`if`](https://github.com/Green-Software-Foundation/if)
     - source code for the IF
-- [`if-plugins`](https://github.com/Green-Software-Foundation/if-plugins)
+- [`if-plugins`](https://github.com/Green-Software-Foundation/if-plugins) **DEPRECATED**
     - source code for standard library of plugins
     - IF core team commit to maintaining these plugins 
 - [`if-unofficial-plugins`](https://github.com/Green-Software-Foundation/if-unofficial-plugins)
@@ -110,47 +110,21 @@ We intend to respond to new PRs and issues within 3 days of the ticket being ope
 
 ## Labels
 
-```
-//review status labels 
-awaiting-triage 
-triage-pass 
-awaiting-review 
-review-pass 
-awaiting-qa 
-qa-pass 
-changes-needed 
-
-//change type labels 
-fix 
-feature 
-docs 
-package 
-other 
-
-// size labels 
-small 
-medium 
-large 
-
-// priority labels 
-high-priority 
-med-priority 
-low-priority 
-
-//delay explanation labels 
-abandoned 
-blocked 
-backlogged
-
-// other labels
-project-management
-agenda
-discussion
-help-wanted
-good-first-issue
-```
-
-Each PR is expected to have one label from each category (except `other`) assigned to it at all times.
+| Label            | Used by   | Description                                                                                |
+| ---------------- | --------- | ------------------------------------------------------------------------------------------ |
+| blocked          | all       | The issue is blocked and cannot proceed.                                                   |
+| bug              | all       | Error, flaw or fault                                                                       |
+| core-only        | all       | This issue is reserved for the IF core team only                                           |
+| draft            | all       | The issue is still being written, no need to respond or action on anything.                |
+| good-first-issue | all       | This issue is a good one for someone looking to get involved with this project/initiative. |
+| help-wanted      | all       | This issue can be picked up by anyone in the community                                     |
+| needs-response   | all       | The issue has stalled because someone isn’t responding.                                    |
+| agenda           | core team | Use for any meeting agenda or minutes issue.                                               |
+| EPIC             | core team | Used to denote an issue that represents a whole epic                                       |
+| initiative       | core team | A helper label, needed for GSF internal processes                                          |
+| i-summary        | core team | A helper label, needed for GSF internal processes                                          |
+| i-milestone      | core team | A helper label, needed for GSF internal processes                                          |
+| project-update   | core team | Used to flag project progress updates                                                      |
 
 
 ## Releases

package/manifests/bugs/aggregation-error-wrong-metric.yml

@@ -15,14 +15,14 @@ initialize:
     "sci-e":
       path: "@grnsft/if-plugins"
       method: SciE
-    "sci-m":
-      path: "@grnsft/if-plugins"
-      method: SciM
+    "sci-embodied":
+      path: "builtin"
+      method: SciEmbodied
     "sci-o":
       path: "@grnsft/if-plugins"
       method: SciO
     "sci":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Sci
       global-config:
         functional-unit: "requests"
@@ -44,7 +44,7 @@ tree:
       pipeline:
         - teads-curve
         - sci-e
-        - sci-m
+        - sci-embodied
         - sci-o
         - time-sync
         - sci
@@ -87,7 +87,7 @@ tree:
       pipeline:
         - teads-curve
         - sci-e
-        - sci-m
+        - sci-embodied
         - sci-o
         - time-sync
         - sci

package/manifests/bugs/azure-importer-ignoring-defaults.yml

@@ -15,14 +15,14 @@ initialize:
       method: AzureImporter
       path: "@grnsft/if-unofficial-plugins"
     "try-defaults-1":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Coefficient
       global-config:
         input-parameter: grid/carbon-intensity
         coefficient: 0.1
         output-parameter: grid/carbon-intensity
     "try-defaults-2":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Coefficient
       global-config:
         input-parameter: network/energy
@@ -58,4 +58,4 @@ tree:
       inputs:
         - timestamp: "2024-04-04T08:00:00.001Z"
           duration: 3600
-          energy: 100
+          energy: 100

package/manifests/bugs/azure-importer-incorrect-calculation.yml

@@ -7,14 +7,14 @@ initialize:
       path: '@grnsft/if-unofficial-plugins'
       method: AzureImporter
     try-defaults-1:
-      path: '@grnsft/if-plugins'
+      path: 'builtin'
       method: Coefficient
       global-config:
         input-parameter: grid/carbon-intensity
         coefficient: 0.1
         output-parameter: grid/carbon-intensity
     try-defaults-2:
-      path: '@grnsft/if-plugins'
+      path: 'builtin'
       method: Coefficient
       global-config:
         input-parameter: network/energy

package/manifests/bugs/mock-observations-failure-duration-is-zero.yml

@@ -0,0 +1,34 @@
+name: mock-observation-demo
+description: a manifest demonstrating how to use the mock observations feature
+tags:
+initialize:
+  outputs: ['yaml']
+  plugins:
+    mock-observations:
+      kind: plugin
+      method: MockObservations
+      path: "builtin"
+      global-config:
+        timestamp-from: 2023-07-06T00:00
+        timestamp-to: 2023-07-06T00:10
+        duration: 0
+        components:
+          - cloud/instance-type: A1
+          - cloud/instance-type: B1
+        generators:
+          common:
+            region: uk-west
+            common-key: common-val
+          randint:
+            cpu/utilization:
+              min: 1
+              max: 99
+            memory/utilization:
+              min: 1
+              max: 99
+tree:
+  children:
+    child:
+      pipeline:
+        - mock-observations
+      inputs:

package/manifests/bugs/pipeline-error-uninitialized-plugin.yml

@@ -2,13 +2,13 @@ name: pipeline-uninitialized-plugin-error
 description: a negative test case that fails due to an uninitialized plugin being invoked in a pipeline
 tags:
 initialize:
+ # outputs: ['yaml']
   plugins:
     teads-curve: 
       path: '@grnsft/if-unofficial-plugins'
       method: TeadsCurve
       global-config:
         interpolation: spline
-  outputs: ['yaml']
 tree:
   children:
     child-0:

package/manifests/bugs/pipeline-ordering-error.yml

@@ -9,21 +9,21 @@ initialize:
       global-config:
         interpolation: spline
     "sum":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Sum
       global-config:
         input-parameters:
           - cpu/energy
           - network/energy
         output-parameter: energy
-    "sci-m":
-      path: "@grnsft/if-plugins"
-      method: SciM
+    "sci-embodied":
+      path: "builtin"
+      method: SciEmbodied
     "sci-o":
       path: "@grnsft/if-plugins"
       method: SciO
     "sci":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Sci
       global-config:
         functional-unit: ""
@@ -43,7 +43,7 @@ tree:
         - teads-curve
         - sci-o
         - sum
-        - sci-m
+        - sci-embodied
         - time-sync
         - sci
       config:

package/manifests/bugs/sci-embodied-missing-resources-total.yml

@@ -0,0 +1,23 @@
+name: sci-embodied
+description: receiving incorrect error message when running sci-embodied without `resources-total` issue #716
+tags:
+initialize:
+  #outputs: ['yaml']
+  plugins:
+    "sci-embodied": # a model that calculates m from te, tir, el, rr and rtor
+      method: SciEmbodied
+      path: "builtin"
+tree:
+  children:
+    child:
+      pipeline:
+        - sci-embodied # duration & config -> embodied
+      defaults:
+        device/emissions-embodied: 1533.120 # gCO2eq
+        time-reserved: 3600 # 1hr in seconds
+        device/expected-lifespan: 94608000 # 3 years in seconds
+        resources-reserved: 1
+       # resources-total: 8
+      inputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 3600

package/manifests/examples/generics.yml

@@ -9,7 +9,7 @@ initialize:
       global-config:
         interpolation: spline
     "sum":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Sum
       global-config:
         input-parameters:
@@ -17,14 +17,14 @@ initialize:
           - network/energy
         output-parameter: energy-sum
     "coefficient":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Coefficient
       global-config:
         input-parameter: energy
         coefficient: 2
         output-parameter: energy-doubled
     "multiply":
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Multiply
       global-config:
         input-parameters: ["cpu/utilization", "duration"]

package/manifests/examples/instance-metadata.yml

@@ -0,0 +1,36 @@
+name: csv-demo
+description: null
+tags: null
+initialize:
+  plugins:
+    cloud-instance-metadata:
+      path: builtin
+      method: CSVLookup
+      global-config:
+        filepath: >-
+          https://raw.githubusercontent.com/Green-Software-Foundation/if-data/main/cloud-metdata-azure-instances.csv
+        query:
+          instance-class: cloud/instance-type
+        output: '*'
+    extract-processor-name:
+      path: builtin
+      method: Regex
+      global-config:
+        parameter: cpu-model-name
+        match: /^([^,])+/g
+        output: cpu/name
+  outputs:
+    - yaml
+tree:
+  children:
+    child:
+      pipeline:
+        - cloud-instance-metadata
+        - extract-processor-name
+      inputs:
+        - timestamp: 2023-08-06T00:00
+          duration: 3600
+          cpu/energy: 0.001
+          cloud/provider: gcp
+          cloud/region: asia-east
+          cloud/instance-type: Standard_A1_v2

package/manifests/examples/mock-cpu-util-to-carbon.yml

@@ -1,13 +1,13 @@
 name: mock-cpu-util-to-carbon
 description: a complete pipeline that starts with mocked CPU utilization data and outputs operational carbon in gCO2eq
 initialize:
-  outputs: ['yaml']
+  #outputs: ['yaml']
   plugins: 
     group-by: 
       path: 'builtin'
       method: GroupBy   
     operational-carbon:
-      path: '@grnsft/if-plugins'
+      path: 'builtin'
       method: Multiply
       global-config:
         input-parameters: ['cpu/energy', 'grid/carbon-intensity']
@@ -24,7 +24,7 @@ initialize:
       method: CloudMetadata
       path: "@grnsft/if-plugins"   
     mock-observations:
-      path: '@grnsft/if-plugins'
+      path: 'builtin'
       method: MockObservations
       global-config:
         timestamp-from: '2024-03-05T00:00:00.000Z'

package/manifests/examples/nesting.yml

@@ -19,25 +19,32 @@ initialize:
       global-config:
         interpolation: spline
     sum:
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Sum
       global-config:
         input-parameters:
           - cpu/energy
           - network/energy
         output-parameter: energy
-    sci-m:
-      path: "@grnsft/if-plugins"
-      method: SciM
+    sci-embodied:
+      path: "builtin"
+      method: SciEmbodied
     sci-o:
       path: "@grnsft/if-plugins"
       method: SciO
     sci:
-      path: "@grnsft/if-plugins"
+      path: "builtin"
       method: Sci
       global-config:
         functional-unit: "requests"
-        functional-unit-time: "1 minute"
+    "sum-carbon":
+      path: "builtin"
+      method: Sum
+      global-config:
+        input-parameters:
+          - carbon-operational
+          - carbon-embodied
+        output-parameter: carbon
     time-sync:
       method: TimeSync
       path: "builtin"
@@ -60,8 +67,9 @@ tree:
       pipeline:
         - teads-curve
         - sum
-        - sci-m
+        - sci-embodied
         - sci-o
+        - sum-carbon
         - time-sync
         - sci
       inputs:
@@ -71,24 +79,28 @@ tree:
           duration: 1
           cpu/utilization: 50
           network/energy: 0.000001
+          requests: 50
         - timestamp: "2023-12-12T00:00:01.000Z"
           duration: 5
           cpu/utilization: 20
           cloud/instance-type: A1
           cloud/region: uk-west
           network/energy: 0.000001
+          requests: 60
         - timestamp: "2023-12-12T00:00:06.000Z"
           duration: 7
           cpu/utilization: 15
           cloud/instance-type: A1
           cloud/region: uk-west
           network/energy: 0.000001
+          requests: 70
         - timestamp: "2023-12-12T00:00:13.000Z"
           duration: 30
           cloud/instance-type: A1
           cloud/region: uk-west
           cpu/utilization: 15
           network/energy: 0.000001
+          requests: 55
     child-1:
       defaults:
         cpu/thermal-design-power: 100
@@ -101,8 +113,9 @@ tree:
       pipeline:
         - teads-curve
         - sum
-        - sci-m
+        - sci-embodied
         - sci-o
+        - sum-carbon
         - time-sync
         - sci
       inputs:
@@ -112,24 +125,28 @@ tree:
           duration: 1
           cpu/utilization: 50
           network/energy: 0.000001
+          requests: 10
         - timestamp: "2023-12-12T00:00:01.000Z"
           duration: 5
           cpu/utilization: 20
           cloud/instance-type: A1
           cloud/region: uk-west
           network/energy: 0.000001
+          requests: 90
         - timestamp: "2023-12-12T00:00:06.000Z"
           duration: 7
           cpu/utilization: 15
           cloud/instance-type: A1
           cloud/region: uk-west
           network/energy: 0.000001
+          requests: 30
         - timestamp: "2023-12-12T00:00:13.000Z"
           duration: 30
           cloud/instance-type: A1
           cloud/region: uk-west
           cpu/utilization: 15
           network/energy: 0.000001
+          requests: 22
     child-2:
       children:
         child-2-0:
@@ -144,8 +161,9 @@ tree:
           pipeline:
             - teads-curve
             - sum
-            - sci-m
+            - sci-embodied
             - sci-o
+            - sum-carbon
             - time-sync
             - sci
           inputs:
@@ -155,24 +173,28 @@ tree:
               duration: 1
               cpu/utilization: 50
               network/energy: 0.000001
+              requests: 50
             - timestamp: "2023-12-12T00:00:01.000Z"
               duration: 5
               cpu/utilization: 20
               cloud/instance-type: A1
               cloud/region: uk-west
               network/energy: 0.000001
+              requests: 65
             - timestamp: "2023-12-12T00:00:06.000Z"
               duration: 7
               cpu/utilization: 15
               cloud/instance-type: A1
               cloud/region: uk-west
               network/energy: 0.000001
+              requests: 80
             - timestamp: "2023-12-12T00:00:13.000Z"
               duration: 30
               cloud/instance-type: A1
               cloud/region: uk-west
               cpu/utilization: 15
               network/energy: 0.000001
+              requests: 40
         child-2-1:
           defaults:
             cpu/thermal-design-power: 100
@@ -185,8 +207,9 @@ tree:
           pipeline:
             - teads-curve
             - sum
-            - sci-m
+            - sci-embodied
             - sci-o
+            - sum-carbon
             - time-sync
             - sci
           inputs:
@@ -196,21 +219,25 @@ tree:
               duration: 1
               cpu/utilization: 50
               network/energy: 0.000001
+              requests: 50
             - timestamp: "2023-12-12T00:00:01.000Z"
               duration: 5
               cpu/utilization: 20
               cloud/instance-type: A1
               cloud/region: uk-west
               network/energy: 0.000001
+              requests: 50
             - timestamp: "2023-12-12T00:00:06.000Z"
               duration: 7
               cpu/utilization: 15
               cloud/instance-type: A1
               cloud/region: uk-west
               network/energy: 0.000001
+              requests: 60
             - timestamp: "2023-12-12T00:00:13.000Z"
               duration: 30
               cloud/instance-type: A1
               cloud/region: uk-west
               cpu/utilization: 15
               network/energy: 0.000001
+              requests: 40