@grnsft/if 0.3.3-beta.0 → 0.4.0-beta.0

package/src/builtins/shell/README.md

@@ -0,0 +1,130 @@
+# Shell Plugin
+
+The `shell` is a wrapper enabling plugins implemented in any other programming language to be executed as a part of IF pipeline. For example, you might have a standalone plugin written in Python. `shell` spawns a subprocess to execute that Python plugin in a dedicated shell and pipes the results back into IF's Typescript process.
+
+## Parameters
+
+### Plugin global config
+
+The plugin should be initialized as follows:
+
+```
+initialize:
+  plugins:
+    shell:
+      method: Shell
+      path: 'builtin'
+      global-config:
+        command: python3 /usr/local/bin/sampler
+```
+
+The `shell` plugin interface requires a path to the plugin command. This path is provided in the plugin configuration with the name command. The path should be appended by the execution command, for example, if the executable is a binary, the path would be prepended with `./` on a Linux system. If the plugin is a Python script, you can prepend `python`.
+
+- `command`: the path to the plugin executable along with the execution command as it would be entered into a shell.
+
+### Inputs
+
+The parameters included in the `inputs` field in the `manifest` depend entirely on the plugin itself. A typical plugin might expect the following common data to be provided as `inputs`:
+
+- `timestamp`: A timestamp for the specific input
+- `duration`: The length of time these specific inputs cover
+
+## Returns
+
+The specific return types depend on the plugin being invoked. Typically, we would expect some kind of energy or carbon metric as an output, but it is also possible that plugins target different parts of the pipeline, such as data importers, adaptor plugins etc. Therefore, we do not specify return data for external plugins.
+
+## Implementation
+
+To run the plugin, you must first create an instance of `Shell` and call its `execute()` to run the external plugin.
+
+```typescript
+const output = Shell({command: '/usr/local/bin/sampler'});
+const result = await output.execute([
+  {
+    timestamp: '2021-01-01T00:00:00Z',
+    duration: 3600,
+    'cpu/energy': 0.002,
+    'memory/energy': 0.000005,
+  },
+]);
+```
+
+## Considerations
+
+The `shell` is designed to run arbitrary external plugins. This means IF does not necessarily know what calculations are being executed in the external plugin. There is no strict requirement on the return type, as this depends upon the calculations and the position of the external plugin in a plugin pipeline. For example, one external plugin might carry out the entire end-to-end SCI calculation, taking in usage inputs and returning `sci`. In this case, the plugin is expected to return `sci` and it would be the only plugin invoked in the `manifest`.
+
+However, it is also entirely possible to have external plugins that only deliver some small part of the overall pipeline, and rely on IF other plugins to do the rest.
+
+Since the design space for external plugins is so large, it is up to external plugin developers to ensure compatibility with IF built-ins.
+
+## Example manifest
+
+IF users will typically call the shell plugin as part of a pipeline defined in a `manifest` file. In this case, instantiating and configuring the plugin is handled by and does not have to be done explicitly by the user. The following is an example `manifest` that calls an external plugin via `shell`. It assumes the plugin takes `cpu/energy` and `memory/energy` as inputs and returns `energy`:
+
+```yaml
+name: shell-demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sampler:
+      method: Shell
+      path: 'builtin'
+      global-config:
+        command: python3 /usr/local/bin/sampler
+tree:
+  children:
+    child:
+      pipeline:
+        - sampler
+      inputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 1 # Secs
+          cpu/energy: 0.002
+          memory/energy: 0.000005
+```
+
+In this hypothetical example, the plugin is written in Python and invoked by executing `python3 /usr/local/bin/sampler` in a shell.
+The plugin should return an `output` looking as follows:
+
+```yaml
+name: shell-demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sampler:
+      method: Shell
+      path: 'builtin'
+      global-config:
+        command: python3 /usr/local/bin/sampler
+tree:
+  children:
+    child:
+      pipeline:
+        - sampler
+      inputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 1 # Secs
+          cpu/energy: 0.002
+          memory/energy: 0.000005
+      outputs:
+        - timestamp: 2023-07-06T00:00
+          duration: 1 # Secs
+          cpu/energy: 0.002
+          memory/energy: 0.000005
+          energy: 0.02 # added by plugin
+```
+
+You can run this example `manifest` by saving it as `manifests/plugins/shell.yml` and executing the following command from the project root:
+
+```sh
+npm i -g @grnsft/if
+ie --manifest manifests/plugins/shell.yml --output manifests/outputs/shell.yml
+```
+
+The results will be saved to a new `yaml` file.

package/src/builtins/subtract/README.md

@@ -0,0 +1,94 @@
+# Subtract
+
+`subtract` is a generic plugin for doing arithmetic subtractions of two or more values in an `input` array.
+
+You provide the names of the values you want to subtract, and a name to use to add the subtraction to the output array.
+
+For example, you could subtract `cpu/energy` and `network/energy` and name the result `offset/energy`. `offset/energy` would then be added to every observation in your input array as the diff of `cpu/energy` and `network/energy`.
+
+## Parameters
+
+### Plugin config
+
+Two parameters are required in global config: `input-parameters` and `output-parameter`.
+
+`input-parameters`: an array of strings. Each string should match an existing key in the `inputs` array
+`output-parameter`: a string defining the name to use to add the result of the diff to the output array.
+
+### Inputs
+
+All of `input-parameters` must be available in the input array.
+
+## Returns
+
+- `output-parameter`: the subtraction of all `input-parameters` with the parameter name defined by `output-parameter` in global config.
+
+## Calculation
+
+```pseudocode
+output = input0 - input1 - input2 ... - inputN
+```
+
+## Implementation
+
+To run the plugin, you must first create an instance of `Subtract`. Then, you can call `execute()`.
+
+```typescript
+import {Subtract} from 'builtins';
+
+const config = {
+  inputParameters: ['cpu/energy', 'network/energy'],
+  outputParameter: 'offset/energy',
+};
+
+const subtract = Subtract(config);
+const result = subtract subtract.execute([
+  {
+    duration: 3600,
+    timestamp: '2021-01-01T00:00:00Z',
+    'cpu/energy': 0.005,
+    'memory/energy': 0.0001,
+  },
+]);
+```
+
+## Example manifest
+
+IF users will typically call the plugin as part of a pipeline defined in a manifest file. In this case, instantiating the plugin is handled by and does not have to be done explicitly by the user. The following is an example manifest that calls `subtract`:
+
+```yaml
+name: subtract demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    subtract:
+      method: Subtract
+      path: 'builtin'
+      global-config:
+        input-parameters: ['cpu/energy', 'network/energy']
+        output-parameter: 'energy/diff'
+tree:
+  children:
+    child:
+      pipeline:
+        - subtract
+      config:
+        subtract:
+      inputs:
+        - timestamp: 2023-08-06T00:00
+          duration: 3600
+          cpu/energy: 0.003
+          network/energy: 0.001
+```
+
+You can run this example by saving it as `./examples/manifests/test/subrtact.yml` and executing the following command from the project root:
+
+```sh
+npm i -g @grnsft/if
+ie --manifest /manifests/plugins/subtract.yml --output manifests/outputs/subtract.yml
+```
+
+The results will be saved to a new `yaml` file in `manifests/outputs`.

package/src/builtins/sum/README.md

@@ -0,0 +1,91 @@
+# Sum
+
+`sum` is a generic plugin for doing arithmetic sums of two or more values in an `input` array.
+
+You provide the names of the values you want to sum, and a name to use to add the sum to the output array.
+
+For example, you could add `cpu/energy` and `network/energy` and name the result `energy`. `energy` would then be added to every observation in your input array as the sum of `cpu/energy` and `network/energy`.
+
+## Parameters
+
+### Plugin config
+
+Two parameters are required in global config: `input-parameters` and `output-parameter`.
+
+`input-parameters`: an array of strings. Each string should match an existing key in the `inputs` array
+`output-parameter`: a string defining the name to use to add the result of summing the input parameters to the output array.
+
+### Inputs
+
+All of `input-parameters` must be available in the input array.
+
+## Returns
+
+- `output-parameter`: the sum of all `input-parameters` with the parameter name defined by `output-parameter` in global config.
+
+## Calculation
+
+```pseudocode
+output = input0 + input1 + input2 ... inputN
+```
+
+## Implementation
+
+To run the plugin, you must first create an instance of `Sum`. Then, you can call `execute()`.
+
+```typescript
+const config = {
+  inputParameters: ['cpu/energy', 'network/energy'],
+  outputParameter: 'energy',
+};
+
+const sum = Sum(config);
+const result = sum.execute([
+  {
+    timestamp: '2021-01-01T00:00:00Z',
+    duration: 3600,
+    'cpu/energy': 0.001,
+    'memory/energy': 0.0005,
+  },
+]);
+```
+
+## Example manifest
+
+IF users will typically call the plugin as part of a pipeline defined in a manifest file. In this case, instantiating the plugin is handled by and does not have to be done explicitly by the user. The following is an example manifest that calls `sum`:
+
+```yaml
+name: sum demo
+description:
+tags:
+initialize:
+  outputs:
+    - yaml
+  plugins:
+    sum:
+      method: Sum
+      path: 'builtin'
+      global-config:
+        input-parameters: ['cpu/energy', 'network/energy']
+        output-parameter: 'energy'
+tree:
+  children:
+    child:
+      pipeline:
+        - sum
+      config:
+        sum:
+      inputs:
+        - timestamp: 2023-08-06T00:00
+          duration: 3600
+          cpu/energy: 0.001
+          network/energy: 0.001
+```
+
+You can run this example by saving it as `./examples/manifests/sum.yml` and executing the following command from the project root:
+
+```sh
+ie --manifest ./examples/manifests/sum.yml --output ./examples/outputs/sum.yml
+```
+
+The results will be saved to a new `yaml` file in `./examples/outputs`.