@itentialopensource/adapter-vmware_vcenter 0.5.3 → 0.7.0

package/.eslintignore

@@ -2,3 +2,4 @@
 backup/*
 coverage/*
 out/*
+adapter_modifications/*

package/.eslintrc.js

@@ -1,18 +1,18 @@
 module.exports = {
-  'env': {
-    'browser': true,
-    'es6': true,
-    'node': true,
+  env: {
+    browser: true,
+    es6: true,
+    node: true
   },
-  'extends': 'airbnb-base',
-  "plugins": [
-    "json"
+  extends: 'airbnb-base',
+  plugins: [
+    'json'
   ],
-  'parserOptions': {
-    'sourceType': 'module',
+  parserOptions: {
+    sourceType: 'module'
   },
   rules: {
     'max-len': 'warn',
-    'comma-dangle': ['error', 'never'],
-  },
-};
+    'comma-dangle': ['error', 'never']
+  }
+};

package/CHANGELOG.md

@@ -1,4 +1,36 @@
 
+## 0.7.0 [01-24-2022]
+
+* migration to the latest foundation and broker ready
+
+See merge request itentialopensource/adapters/cloud/adapter-vmware_vcenter!15
+
+---
+
+## 0.6.0 [08-02-2021]
+
+* Add call
+
+See merge request itentialopensource/adapters/cloud/adapter-vmware_vcenter!14
+
+---
+
+## 0.5.5 [03-16-2021]
+
+* migration to the latest adapter foundation
+
+See merge request itentialopensource/adapters/cloud/adapter-vmware_vcenter!12
+
+---
+
+## 0.5.4 [07-09-2020]
+
+* migration
+
+See merge request itentialopensource/adapters/cloud/adapter-vmware_vcenter!11
+
+---
+
 ## 0.5.3 [06-25-2020]
 
 * changes to filters

package/README.md

@@ -1,4 +1,4 @@
-# Vmware_vCenter Adapter
+# Adapter for Vmware_vCenter
 
 This adapter is used to integrate the Itential Automation Platform (IAP) with the Vmware_vCenter System. The API for Vmware_vCenter is available at [undefined API URL]. The adapter utilizes the Vmware_vCenter API to provide the integrations that are deemed pertinent to IAP. This ReadMe file is intended to provide information on this adapter.
 
@@ -22,6 +22,24 @@ Release notes can be viewed in CHANGELOG.md or in the [Customer Knowledge Base](
 
 These instructions will help you get a copy of the project on your local machine for development and testing. Reading this section is also helpful for deployments as it provides you with pertinent information on prerequisites and properties.
 
+### Adapter Technical Resources
+
+There is adapter documentation available on the Itential Developer Site [HERE](https://developer.itential.io/adapters-resources/). This documentation includes information and examples that are helpful for:
+
+```text
+Authentication
+Properties
+Code Files
+Action Files
+Schema Files
+Mock Data Files
+Linting and Testing
+Troubleshooting
+```
+
+Others will be added over time.
+Want to build a new adapter? Use the Adapter Builder [HERE](https://adapters.itential.io)
+
 ### Environment Prerequisites
 
 The following is a list of required packages for an adapter.
@@ -40,8 +58,14 @@ The following list of packages are required for Itential product adapters or cus
 | ------- | ------- |
 | @itentialopensource/adapter-utils | Runtime library classes for all adapters;  includes request handling, connection, throttling, and translation. |
 | ajv | Required for validation of adapter properties to integrate with Vmware_vCenter. |
+| axios | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
+| commander | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
 | fs-extra | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
-| readline-sync | Utilized by the testRunner script that comes with the adapter;  helps to test unit and integration functionality. |
+| network-diagnostics | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
+| readline-sync | Utilized by the node script that comes with the adapter;  helps to test unit and integration functionality. |
+| semver | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
+
+Some of the adapter node scripts run testing scripts which require the dev dependencies listed below.
 
 ### Additional Prerequisites for Development and Testing
 
@@ -54,7 +78,9 @@ eslint-config-airbnb-base
 eslint-plugin-import
 eslint-plugin-json
 mocha
+mocha-param
 nyc
+package-json-validator
 testdouble
 winston
 ```
@@ -81,6 +107,7 @@ This section defines **all** the properties that are available for the adapter,
       "base_path": "/",
       "version": "v1",
       "cache_location": "local",
+      "encode_pathvars": true,
       "save_metric": true,
       "stub": false,
       "protocol": "https",
@@ -88,16 +115,18 @@ This section defines **all** the properties that are available for the adapter,
         "auth_method": "basic user_password",
         "username": "username",
         "password": "password",
-        "auth_field": "header.headers.X-AUTH-TOKEN",
-        "auth_field_format": "{token}",
         "token": "token",
         "invalid_token_error": 401,
         "token_timeout": 0,
-        "token_cache": "local"
+        "token_cache": "local",
+        "auth_field": "header.headers.X-AUTH-TOKEN",
+        "auth_field_format": "{token}",
+        "auth_logging": false
       },
       "healthcheck": {
         "type": "startup",
-        "frequency": 300000
+        "frequency": 300000,
+        "query_object": {}
       },
       "request": {
         "number_redirects": 0,
@@ -113,7 +142,8 @@ This section defines **all** the properties that are available for the adapter,
         },
         "healthcheck_on_timeout": false,
         "return_raw": false,
-        "archiving": false
+        "archiving": false,
+        "return_request": false
       },
       "ssl": {
         "ecdhCurve": "",
@@ -132,13 +162,16 @@ This section defines **all** the properties that are available for the adapter,
         "max_in_queue": 1000,
         "concurrent_max": 1,
         "expire_timeout": 0,
-        "avg_runtime": 200
+        "avg_runtime": 200,
+        "priorities": []
       },
       "proxy": {
         "enabled": false,
         "host": "localhost",
         "port": 9999,
-        "protocol": "http"
+        "protocol": "http",
+        "username": "",
+        "password": "",
       },
       "mongo": {
         "host": "",
@@ -171,6 +204,7 @@ These base properties are used to connect to Vmware_vCenter upon the adapter ini
 | base_path | Optional. Used to define part of a path that is consistent for all or most endpoints. It makes the URIs easier to use and maintain but can be overridden on individual calls. An example **base_path** might be `/rest/api`. Default is ``.|
 | version | Optional. Used to set a global version for action endpoints. This makes it faster to update the adapter when endpoints change. As with the base-path, version can be overridden on individual endpoints. Default is ``.|
 | cache\_location | Optional. Used to define where the adapter cache is located. The cache is used to maintain an entity list to improve performance. Storage locally is lost when the adapter is restarted. Storage in Redis is preserved upon adapter restart. Default is none which means no caching of the entity list.|
+| encode\_pathvars | Optional. Used to tell the adapter to encode path variables or not. The default behavior is to encode them so this property can b e used to stop that behavior.|
 | save\_metric | Optional. Used to tell the adapter to save metric information (this does not impact metrics returned on calls). This allows the adapter to gather metrics over time. Metric data can be stored in a database or on the file system.|
 | stub | Optional. Indicates whether the stub should run instead of making calls to Vmware_vCenter (very useful during basic testing). Default is false (which means connect to Vmware_vCenter).|
 | protocol | Optional. Notifies the adapter whether to use HTTP or HTTPS. Default is HTTP.|
@@ -188,12 +222,13 @@ The following properties are used to define the authentication process to Vmware
 | auth\_method | Required. Used to define the type of authentication currently supported. Authentication methods currently supported are: `basic user_password`, `static_token`, `request_token`, and `no_authentication`.|
 | username | Used to authenticate with Vmware_vCenter on every request or when pulling a token that will be used in subsequent requests.|
 | password | Used to authenticate with Vmware_vCenter on every request or when pulling a token that will be used in subsequent requests.|
-| auth\_field | Defines the request field the authentication (e.g., token are basic auth credentials) needs to be placed in order for the calls to work.|
-| auth\_field\_format | Defines the format of the auth\_field. See examples below. Items enclosed in {} inform the adapter to perofrm an action prior to sending the data. It may be to replace the item with a value or it may be to encode the item. |
 | token | Defines a static token that can be used on all requests. Only used with `static_token` as an authentication method (auth\_method).|
 | invalid\_token\_error | Defines the HTTP error that is received when the token is invalid. Notifies the adapter to pull a new token and retry the request. Default is 401.|
 | token\_timeout | Defines how long a token is valid. Measured in milliseconds. Once a dynamic token is no longer valid, the adapter has to pull a new token. If the token\_timeout is set to -1, the adapter will pull a token on every request to Vmware_vCenter. If the timeout\_token is 0, the adapter will use the expiration from the token response to determine when the token is no longer valid.|
 | token\_cache | Used to determine where the token should be stored (local memory or in Redis).|
+| auth\_field | Defines the request field the authentication (e.g., token are basic auth credentials) needs to be placed in order for the calls to work.|
+| auth\_field\_format | Defines the format of the auth\_field. See examples below. Items enclosed in {} inform the adapter to perofrm an action prior to sending the data. It may be to replace the item with a value or it may be to encode the item. |
+| auth\_logging | Setting this true will add some additional logs but this should only be done when trying to debug an issue as certain credential information may be logged out when this is true. |
 
 #### Examples of authentication field format
 
@@ -216,6 +251,7 @@ The healthcheck properties defines the API that runs the healthcheck to tell the
 | ------- | ------- |
 | type | Required. The type of health check to run. |
 | frequency | Required if intermittent. Defines how often the health check should run. Measured in milliseconds. Default is 300000.|
+| query_object | Query parameters to be added to the adapter healthcheck call.|
 
 ### Request Properties
 
@@ -236,6 +272,7 @@ The request section defines properties to help handle requests.
 | healthcheck\_on\_timeout | Required. Defines if the adapter should run a health check on timeout. If set to true, the adapter will abort the request and run a health check until it re-establishes connectivity and then it will re-attempt the request.|
 | return\_raw | Optional. Tells the adapter whether the raw response should be returned as well as the IAP response. This is helpful when running integration tests to save mock data. It does add overhead to the response object so it is not ideal from production.|
 | archiving | Optional flag. Default is false. It archives the request, the results and the various times (wait time, Vmware_vCenter time and overall time) in the `adapterid_results` collection in MongoDB. Although archiving might be desirable, be sure to develop a strategy before enabling this capability. Consider how much to archive and what strategy to use for cleaning up the collection in the database so that it does not become too large, especially if the responses are large.|
+| return\_request | Optional flag. Default is false. Will return the actual request that is made including headers. This should only be used during debugging issues as there could be credentials in the actual request.|
 
 ### SSL Properties
 
@@ -265,6 +302,7 @@ The throttle section is used when requests to Vmware_vCenter must be queued (thr
 | concurrent\_max | Defines the number of requests the adapter can send to Vmware_vCenter at one time (minimum = 1, maximum = 1000). The default is 1 meaning each request must be sent to Vmware_vCenter in a serial manner. |
 | expire\_timeout | Default is 0. Defines a graceful timeout of the request session. After a request has completed, the adapter will wait additional time prior to sending the next request. Measured in milliseconds (minimum = 0, maximum = 60000).|
 | average\_runtime | Represents the approximate average of how long it takes Vmware_vCenter to handle each request. Measured in milliseconds (minimum = 50, maximum = 60000). Default is 200. This metric has performance implications. If the runtime number is set too low, it puts extra burden on the CPU and memory as the requests will continually try to run. If the runtime number is set too high, requests may wait longer than they need to before running. The number does not need to be exact but your throttling strategy depends heavily on this number being within reason. If averages range from 50 to 250 milliseconds you might pick an average run-time somewhere in the middle so that when Vmware_vCenter performance is exceptional you might run a little slower than you might like, but when it is poor you still run efficiently.|
+| priorities | An array of priorities and how to handle them in relation to the throttle queue. Array of objects that include priority value and percent of queue to put the item ex { value: 1, percent: 10 }|
 
 ### Proxy Properties
 
@@ -276,6 +314,8 @@ The proxy section defines the properties to utilize when Vmware_vCenter is behin
 | host | Host information for the proxy server. Required if `enabled` is true.|
 | port | Port information for the proxy server. Required if `enabled` is true.|
 | protocol | The protocol (i.e., http, https, etc.) used to connect to the proxy. Default is http.|
+| username | If there is authentication for the proxy, provide the username here.|
+| password | If there is authentication for the proxy, provide the password here.|
 
 ### Mongo Properties
 
@@ -309,6 +349,7 @@ Unit Testing includes testing basic adapter functionality as well as error condi
 node utils/testRunner --unit
 
 npm run test:unit
+npm run test:baseunit
 ```
 
 To add new unit tests, edit the `test/unit/adapterTestUnit.js` file. The tests that are already in this file should provide guidance for adding additional tests.
@@ -344,6 +385,23 @@ Test should also be written to clean up after themselves. However, it is importa
 
 > **Reminder**: Do not check in code with actual credentials to systems.
 
+## Adapter Node Scripts
+
+There are several node scripts that now accompany the adapter. These scripts are provided to make several activities easier. Each of these scripts are described below.
+
+| Run | Description |
+| ------- | ------- |
+| npm run adapter:install | Provides an easier way to install the adapter.|
+| npm run adapter:checkMigrate | Checks whether your adapter can and should be migrated to the latest foundation.|
+| npm run adapter:findPath | Can be used to see if the adapter supports a particular API call.|
+| npm run adapter:migrate | Provides an easier way to migrate your adapter after you download the migration zip from Itential DevSite|
+| npm run adapter:update | Provides an easier way to update your adapter after you download the migration zip from Itential DevSite|
+| npm run adapter:revert | Allows you to revert after a migration or update if it resulted in issues.|
+| npm run troubleshoot | Provides a way to troubleshoot the adapter - runs connectivity, healthcheck and basic get.|
+| npm run connectivity | Provides a connectivity check to the Vmware_vCenter system.|
+| npm run healthcheck | Checks whether the configured healthcheck call works to Vmware_vCenter.|
+| npm run basicget | Checks whether the basic get calls works to Vmware_vCenter.|
+
 ## Installing an Itential Product Adapter
 
 If you have App-Artifact installed in IAP, you can follow the instruction for that application to install the adapter into IAP. If not, follow these instructions.
@@ -356,24 +414,21 @@ if the @itentialopensource directory does not exist, create it:
    mkdir @itentialopensource
 ```
 
-1. Clone the adapter into your IAP environment.
+2. Clone the adapter into your IAP environment.
 
 ```bash
 cd \@itentialopensource
 git clone git@gitlab.com:\@itentialopensource/adapters/adapter-vmware_vcenter
 ```
 
-1. Install the dependencies for the adapter.
+3. Run the adapter install script.
 
 ```bash
 cd adapter-vmware_vcenter
-npm install
+npm run adapter:install
 ```
 
-1. If you are running IAP 2019.1 or older, add the adapter properties for Vmware_vCenter (created from Adapter Builder) to the `properties.json` file for your Itential build. You will need to change the credentials and possibly the host information below.
-[Vmware_vCenter sample properties](sampleProperties.json). If you are running IAP 2019.2 the adapter properties need to go into the database. You can review IAP documentation for how to do this.
-
-1. Restart IAP
+4. Restart IAP
 
 ```bash
 systemctl restart pronghorn
@@ -393,7 +448,7 @@ Depending on where your code is located, this process is different.
 Adapter should be placed into: /opt/pronghorn/current/node_modules/\@itentialopensource
 ```
 
-1. Follow Steps 3-5 (above) to install an Itential adapter to load your properties, dependencies and restart IAP.
+2. Follow Steps 3-4 (above) to install an Itential adapter to load your properties, dependencies and restart IAP.
 
 ## Using this Adapter
 
@@ -421,11 +476,6 @@ The `encryptProperty` call will take the provided property and technique, and re
 encryptProperty(property, technique, callback)
 ```
 
-The `getQueue` call will return the requests that are waiting in the queue if throttling is enabled.
-```js
-getQueue(callback)
-```
-
 The `addEntityCache` call will take the entities and add the list to the entity cache to expedite performance.
 ```js
 addEntityCache(entityType, entities, key, callback)
@@ -446,39 +496,169 @@ The `verifyCapability` call verifies the adapter can perform the provided action
 verifyCapability(entityType, actionType, entityId, callback)
 ```
 
-The `updateEntityCache` call will update the entity cache. 
+The `updateEntityCache` call will update the entity cache.
 ```js
 updateEntityCache()
 ```
 
+The `updateAdapterConfiguration` call provides the ability to update the adapter configuration from IAP - includes actions, schema, mockdata and other configurations.
+```js
+updateAdapterConfiguration(configFile, changes, entity, type, action, callback)
+```
+
+The `suspend` call provides the ability to suspend the adapter and either have requests rejected or put into a queue to be processed after the adapter is resumed.
+```js
+suspend(mode, callback)
+```
+
+The `unsuspend` call provides the ability to resume a suspended adapter. Any requests in queue will be processed before new requests.
+```js
+unsuspend(callback)
+```
+
+The `findPath` call provides the ability to see if a particular API path is supported by the adapter.
+```js
+findPath(apiPath, callback)
+```
+
+The `troubleshoot` call can be used to check on the performance of the adapter - it checks connectivity, healthcheck and basic get calls.
+```js
+troubleshoot(props, persistFlag, adapter, callback)
+```
+
+The `runHealthcheck` call will return the results of a healthcheck.
+```js
+runHealthcheck(adapter, callback)
+```
+
+The `runConnectivity` call will return the results of a connectivity check.
+```js
+runConnectivity(callback)
+```
+
+The `runBasicGet` call will return the results of running basic get API calls.
+```js
+runBasicGet(callback)
+```
+
+The `getQueue` call will return the requests that are waiting in the queue if throttling is enabled.
+```js
+getQueue(callback)
+```
+
 ### Specific Adapter Calls
 
 Specific adapter calls are built based on the API of the Vmware_vCenter. The Adapter Builder creates the proper method comments for generating JS-DOC for the adapter. This is the best way to get information on the calls.
 
+
+## Extending/Enhancing the Adapter
+
+### Adding a Second Instance of an Adapter
+
+You can add a second instance of this adapter without adding new code on the file system. To do this go into the IAP Admin Essentials and add a new service config for this adapter. The two instances of the adapter should have unique ids. In addition, they should point to different instances of the other system. For example, they should be configured to talk to different hosts.
+
+### Adding Adapter Calls
+
+There are multiple ways to add calls to an existing adapter.
+
+The easiest way would be to use the Adapter Builder update process. This process takes in a Swagger or OpenAPI document, allows you to select the calls you want to add and then generates a zip file that can be used to update the adapter. Once you have the zip file simple put it in the adapter direcctory and execute `npm run adapter:update`.
+
+```bash
+mv updatePackage.zip adapter-vmware_vcenter
+cd adapter-vmware_vcenter
+npm run adapter:update
+```
+
+If you do not have a Swagger or OpenAPI document, you can use a Postman Collection and convert that to an OpenAPI document using APIMatic and then follow the first process.
+
+If you want to manually update the adapter that can also be done the key thing is to make sure you update all of the right files. Within the entities directory you will find 1 or more entities. You can create a new entity or add to an existing entity. Each entity has an action.json file, any new call will need to be put in the action.json file. It will also need to be added to the enum for the ph_request_type in the appropriate schema files. Once this configuration is complete you will need to add the call to the adapter.js file and in order to make it available as a workflow task in IAP, it should also be added to the pronghorn.json file. You can optionally add it to the unit and integration test files. There is more information on how to work on each of these files in the Adapter Technical Resources on Dev Site [HERE](https://developer.itential.io/adapters-resources/)
+
+```text
+Files to update
+* entities/<entity>/action.json: add an action
+* entities/<entity>/schema.json (or the schema defined on the action): add action to the enum for ph_request_type
+* adapter.js: add the new method and make sure it calls the proper entity and action
+* pronghorn.json: add the new method
+* test/unit/adapterTestUnit.js (optional but best practice): add unit test(s) - function is there, any required parameters error when not passed in
+* test/integration/adapterTestIntegration.js (optional but best practice): add integration test
+```
+
+### Adding Adapter Properties
+
+While changing adapter properties is done in the service instance configuration section of IAP, adding properties has to be done in the adapter. To add a property you should edit the propertiesSchema.json with the proper information for the property. In addition, you should modify the sampleProperties to have the new property in it.
+
+```text
+Files to update
+* propertiesSchema.json: add the new property and how it is defined
+* sampleProperties: add the new property with a default value
+* test/unit/adapterTestUnit.js (optional but best practice): add the property to the global properties
+* test/integration/adapterTestIntegration.js (optional but best practice): add the property to the global properties
+```
+
+### Changing Adapter Authentication
+
+Often an adapter is built before knowing the authentication and authentication process can also change over time. The adapter supports many different kinds of authentication but it does require configuration. Some forms of authentication can be defined entirely with the adapter properties but others require configuration.
+
+```text
+Files to update
+* entities/<entity>/action.json: change the getToken action as needed
+* entities/<entity>/schemaTokenReq.json: add input parameters (external name is name in other system)
+* entities/<entity>/schemaTokenResp.json: add response parameters (external name is name in other system)
+* propertiesSchema.json: add any new property and how it is defined
+* sampleProperties: add any new property with a default value
+* test/unit/adapterTestUnit.js (optional but best practice): add the property to the global properties
+* test/integration/adapterTestIntegration.js (optional but best practice): add the property to the global properties
+```
+
+### Enhancing Adapter Integration Tests
+
+The adapter integration tests are written to be able to test in either stub (standalone) mode or integrated to the other system. However, if integrating to the other system, you may need to provide better data than what the adapter provides by default as that data is likely to fail for create and update. To provide better data, edit the adapter integration test file. Make sure you do not remove the marker and keep custom code below the marker so you do not impact future migrations. Once the edits are complete, run the integration test as it instructs you to above. When you run integrated to the other system, you can also save mockdata for future use by changing the isSaveMockData flag to true.
+
+```text
+Files to update
+* test/integration/adapterTestIntegration.js: add better data for the create and update calls so that they will not fail.
+```
+
+As mentioned previously, for most of these changes as well as other possible changes, there is more information on how to work on an adapter in the Adapter Technical Resources on Dev Site [HERE](https://developer.itential.io/adapters-resources/)
+
 ## Troubleshooting the Adapter
 
+Run `npm run troubleshoot` to start the interactive troubleshooting process. The command allows user to verify and update connection, authentication as well as healthcheck configuration. After that it will test these properties by sending HTTP request to the endpoint. If the tests pass, it will persist these changes into IAP.
+
+User also have the option to run individual command to perform specific test
+
+- `npm run healthcheck` will perform a healthcheck request of with current setting.
+- `npm run basicget` will perform some non-parameter GET request with current setting.
+- `npm run connectivity` will perform networking diagnostics of the adatper endpoint.
+
 ### Connectivity Issues
 
-1. Verify the adapter properties are set up correctly.
+1. You can run the adapter troubleshooting script which will check connectivity, run the healthcheck and run basic get calls.
+
+```bash
+npm run troubleshoot
+```
+
+2. Verify the adapter properties are set up correctly.
 
 ```text
 Go into the Itential Platform GUI and verify/update the properties
 ```
 
-1. Verify there is connectivity between the Itential Platform Server and Vmware_vCenter Server.
+3. Verify there is connectivity between the Itential Platform Server and Vmware_vCenter Server.
 
 ```text
 ping the ip address of Vmware_vCenter server
 try telnet to the ip address port of Vmware_vCenter
 ```
 
-1. Verify the credentials provided for Vmware_vCenter.
+4. Verify the credentials provided for Vmware_vCenter.
 
 ```text
 login to Vmware_vCenter using the provided credentials
 ```
 
-1. Verify the API of the call utilized for Vmware_vCenter Healthcheck.
+5. Verify the API of the call utilized for Vmware_vCenter Healthcheck.
 
 ```text
 Go into the Itential Platform GUI and verify/update the properties