@grnsft/if 0.3.4 → 0.4.0-beta.1

package/src/builtins/regex/README.md

@@ -0,0 +1,91 @@
+# Regex
+
+`regex` is a generic plugin to match part of one string in an `input` and extract it into an output.
+
+You provide the name of the value you want to match, and a name to use to add the regex to the output array.
+
+For example, `boavizta-cpu` need `cpu/name` to work, however `cloud-metadata` returns `physical-processor` which usually contains a long string of processors that the instance could be separated by `,`, like so:
+
+```
+Intel® Xeon® Platinum 8272CL,Intel® Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz,Intel® Xeon® E5-2673 v3 2.4 GHz
+```
+
+## Parameters
+
+### Plugin config
+
+- `parameter` - a parameter by a specific configured string
+- `match` - a regex by which needs to match the `parameter`
+- `output` - output parameter name in the input
+
+### Inputs
+
+- `parameter` - as input parameter, must be available in the input array
+
+## Returns
+
+- `output`: the first match of `parameter` with the parameter name with `match` defined in global config.
+
+## Implementation
+
+To run the plugin, you must first create an instance of `Regex`. Then, you can call `execute()`.
+
+```typescript
+
+const globalConfig = {
+  parameter: 'physical-processor',
+  match: '^[^,]+',
+  output: 'cpu/name',
+};
+const regex = Regex(globalConfig);
+
+const input = [
+  {
+    timestamp: '2021-01-01T00:00:00Z',
+    duration: 3600,
+    'physical-processor':
+      'Intel® Xeon® Platinum 8272CL,Intel® Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz,Intel® Xeon® E5-2673 v3 2.4 GHz',
+  },
+];
+```
+
+## Example manifest
+
+IF users will typically call the plugin as part of a pipeline defined in a manifest file. In this case, instantiating the plugin is handled by `if` and does not have to be done explicitly by the user. The following is an example manifest that calls `regex`:
+
+```yaml
+name: regex-demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    regex:
+      method: Regex
+      path: 'builtin'
+      global-config:
+        parameter: physical-processor
+        match: ^[^,]+
+        output: cpu/name
+tree:
+  children:
+    child:
+      pipeline:
+        - regex
+      config:
+        regex:
+      inputs:
+        - timestamp: 2023-08-06T00:00
+          duration: 3600
+          physical-processor: Intel® Xeon® Platinum 8272CL,Intel® Xeon® 8171M 2.1 GHz,Intel® Xeon® E5-2673 v4 2.3 GHz,Intel® Xeon® E5-2673 v3 2.4 GHz
+```
+
+You can run this example by saving it as `manifests/plugins/regex.yml` and executing the following command from the project root:
+
+```sh
+npm i -g @grnsft/if
+if --manifest manifests/examples/regex.yml --output manifests/outputs/regex.yml
+```
+
+The results will be saved to a new `yaml` file in `manifests/outputs`.

package/src/builtins/sci/README.md

@@ -0,0 +1,89 @@
+# SCI (software carbon intensity)
+
+[SCI](https://sci-guide.greensoftware.foundation/) represents the amount of carbon emitted per [functional unit](https://sci-guide.greensoftware.foundation/R/).
+
+## Parameters
+
+### Plugin global config
+
+- `functional-unit`: the name of the functional unit in which to express the carbon impact (required)
+
+
+### Inputs
+
+
+- `carbon`: total carbon in gCO2eq (required)
+- `functional-unit`: whatever `functional-unit` you define in global config also has to be present in each input, for example if you provide `functional-unit: requests` in global config, `requests` must be present in your input data.
+
+## Returns
+
+- `sci`: carbon expressed in terms of the given functional unit
+
+## Calculation
+
+SCI is calculated as:
+
+```pseudocode
+sci = carbon / functional unit
+```
+
+
+## IF Implementation
+
+To run the plugin, you must first create an instance of `Sci`. Then, you can call `execute()` to return `sci`.
+
+```typescript
+import {Sci} from 'builtins';
+
+const sci = Sci({'functional-unit': 'requests'});
+const results = await sci.execute(
+  [
+    {
+      'carbon': 5'
+      duration: 1,
+      requests: 100,
+    },
+  ]
+);
+```
+
+## Example manifest
+
+IF users will typically call the plugin as part of a pipeline defined in a `manifest` file. In this case, instantiating the plugin is handled by `ie` and does not have to be done explicitly by the user.
+
+The following is an example `manifest` that calls `sci`:
+
+```yaml
+name: sci-demo
+description: example invoking sci plugin
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sci:
+      method: Sci
+      path: 'builtin'
+      global-config:
+        functional-unit: 'requests'
+tree:
+  children:
+    child:
+      pipeline:
+        - sci
+      config:
+      inputs:
+        - timestamp: 2023-07-06T00:00
+          carbon: 5
+          duration: 1
+          requests: 100
+```
+
+You can run this example `manifest` by saving it as `./manifests/plugins/sci.yml` and executing the following command from the project root:
+
+```sh
+npm i -g @grnsft/if
+ie --manifest manifests/plugins/sci.yml --output manifests/outputs/sci.yml
+```
+
+The results will be saved to a new `yaml` file.

package/src/builtins/sci-embodied/README.md

@@ -0,0 +1,110 @@
+# SCI-Embodied
+
+Software systems cause emissions through the hardware that they operate on, both through the energy that the physical hardware consumes and the emissions associated with manufacturing the hardware. Embodied carbon refers to the carbon emitted during the manufacture and eventual disposal of a component. It is added to the operational carbon (carbon emitted when a component is used) to give an overall SCI score.
+
+Read more on [embodied carbon](https://github.com/Green-Software-Foundation/sci/blob/main/Software_Carbon_Intensity/Software_Carbon_Intensity_Specification.md#embodied-emissions)
+
+## Parameters
+
+### Plugin config
+
+Not Needed
+
+### Inputs
+
+- `device/emissions-embodied`: the sum of Life Cycle Assessment (LCA) emissions for the component
+- `device/expected-lifespan`: the length of time, in seconds, between a component's manufacture and its disposal
+- `resources-reserved`: the number of resources reserved for use by the software
+- `resources-total`: the total number of resources available
+- `duration`: the amount of time covered by an observation, in this context it is used as the share of the total life span of the hardware reserved for use by an application, in seconds.
+
+> Note that if you have a plugin pipeline that adds `vcpus-allocated` and `vcpus-total` to each observation, such as the `cloud-metadata` plugin, those values will be used **in preference** to the given `resources-reserved` and `resources-total` fields.
+
+## Returns
+
+- `carbon-embodied`: the carbon emitted in manufacturing and disposing of a component, in gCO2eq
+
+## Calculation
+
+To calculate the embodied carbon, `m` for a software application, use the equation:
+
+```
+m = te * ts * rs
+```
+
+Where:
+
+- `device/emissions-embodied` = Total embodied emissions; the sum of Life Cycle Assessment (LCA) emissions for the component.
+
+- `timeShare` = Time-share; the share of the total life span of the hardware reserved for use by an application.
+
+  - `timeShare` is calculated as `duration/'device/expected-lifespan'`, where:
+    - `duration` = the length of time the hardware is reserved for use by the software.
+    - `device/expected-lifespan` = Expected lifespan: the length of time, in seconds, between a component's manufacture and its disposal.
+
+- `resourceShare` = Resource-share; the share of the total available resources of the hardware reserved for use by an application.
+  - `resourceShare` is calculated as `resources-reserved/resources-total`, where:
+    - `resources-reserved` = Resources reserved; the number of resources reserved for use by the software.
+    - `resources-total` = Total Resources; the total number of resources available.
+
+## Implementation
+
+IF implements the plugin based on the logic described above. To run the plugin, you must first create an instance of `SciEmbodied`. Then, you can call `execute()` to return `m`.
+
+## Usage
+
+The following snippet demonstrates how to call the `sci-embodied` plugin from Typescript.
+
+```typescript
+import {SciEmbodied} from 'builtins';
+
+const sciEmbodied = SciEmbodied();
+const results = await sciEmbodied.execute([
+  {
+    'device/emissions-embodied': 200, // in gCO2e for total resource units
+    duration: 60 * 60 * 24 * 30, // time reserved in seconds, can point to another field "duration"
+    'device/expected-lifespan': 60 * 60 * 24 * 365 * 4, // lifespan in seconds (4 years)
+    'resources-reserved': 1, // resource units reserved / used
+    'resources-total': 1, // total resource units available
+  },
+]);
+```
+
+## Example manifest
+
+IF users will typically call the plugin as part of a pipeline defined in a `manifest` file. In this case, instantiating the plugin is handled by `ie` and does not have to be done explicitly by the user. The following is an example `manifest` that calls `sci-embodied`:
+
+```yaml
+name: sci-embodied
+description: simple demo invoking sci-embodied
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sci-embodied:
+      method: SciEmbodied
+      path: 'builtins'
+tree:
+  children:
+    child:
+      pipeline:
+        - sci-embodied # duration & config -> embodied
+      defaults:
+        device/emissions-embodied: 1533.120 # gCO2eq
+        device/expected-lifespan: 3 # 3 years in seconds
+        resources-reserved: 1
+        resources-total: 8
+      inputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 3600
+```
+
+You can run this example `manifest` by executing the following command from the project root:
+
+```sh
+npm i -g @grnsft/if
+ie --manifest manifests/plugins/sci-embodied.yml --output manifests/outputs/sci-embodied.yml
+```
+
+The results will be saved to a new `yaml` file in `./examples/outputs`.

package/src/builtins/shell/README.md

@@ -0,0 +1,130 @@
+# Shell Plugin
+
+The `shell` is a wrapper enabling plugins implemented in any other programming language to be executed as a part of IF pipeline. For example, you might have a standalone plugin written in Python. `shell` spawns a subprocess to execute that Python plugin in a dedicated shell and pipes the results back into IF's Typescript process.
+
+## Parameters
+
+### Plugin global config
+
+The plugin should be initialized as follows:
+
+```
+initialize:
+  plugins:
+    shell:
+      method: Shell
+      path: 'builtin'
+      global-config:
+        command: python3 /usr/local/bin/sampler
+```
+
+The `shell` plugin interface requires a path to the plugin command. This path is provided in the plugin configuration with the name command. The path should be appended by the execution command, for example, if the executable is a binary, the path would be prepended with `./` on a Linux system. If the plugin is a Python script, you can prepend `python`.
+
+- `command`: the path to the plugin executable along with the execution command as it would be entered into a shell.
+
+### Inputs
+
+The parameters included in the `inputs` field in the `manifest` depend entirely on the plugin itself. A typical plugin might expect the following common data to be provided as `inputs`:
+
+- `timestamp`: A timestamp for the specific input
+- `duration`: The length of time these specific inputs cover
+
+## Returns
+
+The specific return types depend on the plugin being invoked. Typically, we would expect some kind of energy or carbon metric as an output, but it is also possible that plugins target different parts of the pipeline, such as data importers, adaptor plugins etc. Therefore, we do not specify return data for external plugins.
+
+## Implementation
+
+To run the plugin, you must first create an instance of `Shell` and call its `execute()` to run the external plugin.
+
+```typescript
+const output = Shell({command: '/usr/local/bin/sampler'});
+const result = await output.execute([
+  {
+    timestamp: '2021-01-01T00:00:00Z',
+    duration: 3600,
+    'cpu/energy': 0.002,
+    'memory/energy': 0.000005,
+  },
+]);
+```
+
+## Considerations
+
+The `shell` is designed to run arbitrary external plugins. This means IF does not necessarily know what calculations are being executed in the external plugin. There is no strict requirement on the return type, as this depends upon the calculations and the position of the external plugin in a plugin pipeline. For example, one external plugin might carry out the entire end-to-end SCI calculation, taking in usage inputs and returning `sci`. In this case, the plugin is expected to return `sci` and it would be the only plugin invoked in the `manifest`.
+
+However, it is also entirely possible to have external plugins that only deliver some small part of the overall pipeline, and rely on IF other plugins to do the rest.
+
+Since the design space for external plugins is so large, it is up to external plugin developers to ensure compatibility with IF built-ins.
+
+## Example manifest
+
+IF users will typically call the shell plugin as part of a pipeline defined in a `manifest` file. In this case, instantiating and configuring the plugin is handled by and does not have to be done explicitly by the user. The following is an example `manifest` that calls an external plugin via `shell`. It assumes the plugin takes `cpu/energy` and `memory/energy` as inputs and returns `energy`:
+
+```yaml
+name: shell-demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sampler:
+      method: Shell
+      path: 'builtin'
+      global-config:
+        command: python3 /usr/local/bin/sampler
+tree:
+  children:
+    child:
+      pipeline:
+        - sampler
+      inputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 1 # Secs
+          cpu/energy: 0.002
+          memory/energy: 0.000005
+```
+
+In this hypothetical example, the plugin is written in Python and invoked by executing `python3 /usr/local/bin/sampler` in a shell.
+The plugin should return an `output` looking as follows:
+
+```yaml
+name: shell-demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sampler:
+      method: Shell
+      path: 'builtin'
+      global-config:
+        command: python3 /usr/local/bin/sampler
+tree:
+  children:
+    child:
+      pipeline:
+        - sampler
+      inputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 1 # Secs
+          cpu/energy: 0.002
+          memory/energy: 0.000005
+      outputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 1 # Secs
+          cpu/energy: 0.002
+          memory/energy: 0.000005
+          energy: 0.02 # added by plugin
+```
+
+You can run this example `manifest` by saving it as `manifests/plugins/shell.yml` and executing the following command from the project root:
+
+```sh
+npm i -g @grnsft/if
+ie --manifest manifests/plugins/shell.yml --output manifests/outputs/shell.yml
+```
+
+The results will be saved to a new `yaml` file.

package/src/builtins/subtract/README.md

@@ -0,0 +1,94 @@
+# Subtract
+
+`subtract` is a generic plugin for doing arithmetic subtractions of two or more values in an `input` array.
+
+You provide the names of the values you want to subtract, and a name to use to add the subtraction to the output array.
+
+For example, you could subtract `cpu/energy` and `network/energy` and name the result `offset/energy`. `offset/energy` would then be added to every observation in your input array as the diff of `cpu/energy` and `network/energy`.
+
+## Parameters
+
+### Plugin config
+
+Two parameters are required in global config: `input-parameters` and `output-parameter`.
+
+`input-parameters`: an array of strings. Each string should match an existing key in the `inputs` array
+`output-parameter`: a string defining the name to use to add the result of the diff to the output array.
+
+### Inputs
+
+All of `input-parameters` must be available in the input array.
+
+## Returns
+
+- `output-parameter`: the subtraction of all `input-parameters` with the parameter name defined by `output-parameter` in global config.
+
+## Calculation
+
+```pseudocode
+output = input0 - input1 - input2 ... - inputN
+```
+
+## Implementation
+
+To run the plugin, you must first create an instance of `Subtract`. Then, you can call `execute()`.
+
+```typescript
+import {Subtract} from 'builtins';
+
+const config = {
+  inputParameters: ['cpu/energy', 'network/energy'],
+  outputParameter: 'offset/energy',
+};
+
+const subtract = Subtract(config);
+const result = subtract subtract.execute([
+  {
+    duration: 3600,
+    timestamp: '2021-01-01T00:00:00Z',
+    'cpu/energy': 0.005,
+    'memory/energy': 0.0001,
+  },
+]);
+```
+
+## Example manifest
+
+IF users will typically call the plugin as part of a pipeline defined in a manifest file. In this case, instantiating the plugin is handled by and does not have to be done explicitly by the user. The following is an example manifest that calls `subtract`:
+
+```yaml
+name: subtract demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    subtract:
+      method: Subtract
+      path: 'builtin'
+      global-config:
+        input-parameters: ['cpu/energy', 'network/energy']
+        output-parameter: 'energy/diff'
+tree:
+  children:
+    child:
+      pipeline:
+        - subtract
+      config:
+        subtract:
+      inputs:
+        - timestamp: 2023-08-06T00:00
+          duration: 3600
+          cpu/energy: 0.003
+          network/energy: 0.001
+```
+
+You can run this example by saving it as `./examples/manifests/test/subrtact.yml` and executing the following command from the project root:
+
+```sh
+npm i -g @grnsft/if
+ie --manifest /manifests/plugins/subtract.yml --output manifests/outputs/subtract.yml
+```
+
+The results will be saved to a new `yaml` file in `manifests/outputs`.

package/src/builtins/sum/README.md

@@ -0,0 +1,91 @@
+# Sum
+
+`sum` is a generic plugin for doing arithmetic sums of two or more values in an `input` array.
+
+You provide the names of the values you want to sum, and a name to use to add the sum to the output array.
+
+For example, you could add `cpu/energy` and `network/energy` and name the result `energy`. `energy` would then be added to every observation in your input array as the sum of `cpu/energy` and `network/energy`.
+
+## Parameters
+
+### Plugin config
+
+Two parameters are required in global config: `input-parameters` and `output-parameter`.
+
+`input-parameters`: an array of strings. Each string should match an existing key in the `inputs` array
+`output-parameter`: a string defining the name to use to add the result of summing the input parameters to the output array.
+
+### Inputs
+
+All of `input-parameters` must be available in the input array.
+
+## Returns
+
+- `output-parameter`: the sum of all `input-parameters` with the parameter name defined by `output-parameter` in global config.
+
+## Calculation
+
+```pseudocode
+output = input0 + input1 + input2 ... inputN
+```
+
+## Implementation
+
+To run the plugin, you must first create an instance of `Sum`. Then, you can call `execute()`.
+
+```typescript
+const config = {
+  inputParameters: ['cpu/energy', 'network/energy'],
+  outputParameter: 'energy',
+};
+
+const sum = Sum(config);
+const result = sum.execute([
+  {
+    timestamp: '2021-01-01T00:00:00Z',
+    duration: 3600,
+    'cpu/energy': 0.001,
+    'memory/energy': 0.0005,
+  },
+]);
+```
+
+## Example manifest
+
+IF users will typically call the plugin as part of a pipeline defined in a manifest file. In this case, instantiating the plugin is handled by and does not have to be done explicitly by the user. The following is an example manifest that calls `sum`:
+
+```yaml
+name: sum demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sum:
+      method: Sum
+      path: 'builtin'
+      global-config:
+        input-parameters: ['cpu/energy', 'network/energy']
+        output-parameter: 'energy'
+tree:
+  children:
+    child:
+      pipeline:
+        - sum
+      config:
+        sum:
+      inputs:
+        - timestamp: 2023-08-06T00:00
+          duration: 3600
+          cpu/energy: 0.001
+          network/energy: 0.001
+```
+
+You can run this example by saving it as `./examples/manifests/sum.yml` and executing the following command from the project root:
+
+```sh
+ie --manifest ./examples/manifests/sum.yml --output ./examples/outputs/sum.yml
+```
+
+The results will be saved to a new `yaml` file in `./examples/outputs`.

package/.env

@@ -1,5 +0,0 @@
-# WATT_TIME_USERNAME=manushak
-# WATT_TIME_PASSWORD=S7Ub&_tNF=s{
-
-WATT_TIME_USERNAME=m_test
-WATT_TIME_PASSWORD=@watt-time17