@itentialopensource/adapter-netbox 0.6.1 → 0.7.0

package/.eslintignore

@@ -2,3 +2,4 @@
 backup/*
 coverage/*
 out/*
+adapter_modifications/*

package/CHANGELOG.md

@@ -1,4 +1,36 @@
 
+## 0.7.0 [02-14-2022]
+
+* migration and added a call for graphql
+
+See merge request itentialopensource/adapters/inventory/adapter-netbox!13
+
+---
+
+## 0.6.4 [07-13-2021]
+
+* Updates healthcheck endpoint to use /api/status ADAPT-758
+
+See merge request itentialopensource/adapters/inventory/adapter-netbox!12
+
+---
+
+## 0.6.3 [03-17-2021]
+
+* fix descriptions and summaries
+
+See merge request itentialopensource/adapters/inventory/adapter-netbox!11
+
+---
+
+## 0.6.2 [03-10-2021]
+
+* migration to the latest adapter foundation
+
+See merge request itentialopensource/adapters/inventory/adapter-netbox!10
+
+---
+
 ## 0.6.1 [01-28-2021]
 
 * update a schema so a parameter can be either string or object

package/README.md

@@ -1,4 +1,4 @@
-# Netbox Adapter
+# Adapter for Netbox
 
 This adapter is used to integrate the Itential Automation Platform (IAP) with the Netbox System. The API for Netbox is available at [undefined API URL]. The adapter utilizes the Netbox API to provide the integrations that are deemed pertinent to IAP. This ReadMe file is intended to provide information on this adapter.
 
@@ -58,8 +58,14 @@ The following list of packages are required for Itential product adapters or cus
 | ------- | ------- |
 | @itentialopensource/adapter-utils | Runtime library classes for all adapters;  includes request handling, connection, throttling, and translation. |
 | ajv | Required for validation of adapter properties to integrate with Netbox. |
+| axios | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
+| commander | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
 | fs-extra | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
-| readline-sync | Utilized by the testRunner script that comes with the adapter;  helps to test unit and integration functionality. |
+| network-diagnostics | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
+| readline-sync | Utilized by the node script that comes with the adapter;  helps to test unit and integration functionality. |
+| semver | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
+
+Some of the adapter node scripts run testing scripts which require the dev dependencies listed below.
 
 ### Additional Prerequisites for Development and Testing
 
@@ -72,7 +78,9 @@ eslint-config-airbnb-base
 eslint-plugin-import
 eslint-plugin-json
 mocha
+mocha-param
 nyc
+package-json-validator
 testdouble
 winston
 ```
@@ -112,11 +120,13 @@ This section defines **all** the properties that are available for the adapter,
         "token_timeout": 0,
         "token_cache": "local",
         "auth_field": "header.headers.X-AUTH-TOKEN",
-        "auth_field_format": "{token}"
+        "auth_field_format": "{token}",
+        "auth_logging": false
       },
       "healthcheck": {
         "type": "startup",
-        "frequency": 300000
+        "frequency": 300000,
+        "query_object": {}
       },
       "request": {
         "number_redirects": 0,
@@ -132,7 +142,8 @@ This section defines **all** the properties that are available for the adapter,
         },
         "healthcheck_on_timeout": false,
         "return_raw": false,
-        "archiving": false
+        "archiving": false,
+        "return_request": false
       },
       "ssl": {
         "ecdhCurve": "",
@@ -158,7 +169,9 @@ This section defines **all** the properties that are available for the adapter,
         "enabled": false,
         "host": "localhost",
         "port": 9999,
-        "protocol": "http"
+        "protocol": "http",
+        "username": "",
+        "password": "",
       },
       "mongo": {
         "host": "",
@@ -215,6 +228,7 @@ The following properties are used to define the authentication process to Netbox
 | token\_cache | Used to determine where the token should be stored (local memory or in Redis).|
 | auth\_field | Defines the request field the authentication (e.g., token are basic auth credentials) needs to be placed in order for the calls to work.|
 | auth\_field\_format | Defines the format of the auth\_field. See examples below. Items enclosed in {} inform the adapter to perofrm an action prior to sending the data. It may be to replace the item with a value or it may be to encode the item. |
+| auth\_logging | Setting this true will add some additional logs but this should only be done when trying to debug an issue as certain credential information may be logged out when this is true. |
 
 #### Examples of authentication field format
 
@@ -237,6 +251,7 @@ The healthcheck properties defines the API that runs the healthcheck to tell the
 | ------- | ------- |
 | type | Required. The type of health check to run. |
 | frequency | Required if intermittent. Defines how often the health check should run. Measured in milliseconds. Default is 300000.|
+| query_object | Query parameters to be added to the adapter healthcheck call.|
 
 ### Request Properties
 
@@ -257,6 +272,7 @@ The request section defines properties to help handle requests.
 | healthcheck\_on\_timeout | Required. Defines if the adapter should run a health check on timeout. If set to true, the adapter will abort the request and run a health check until it re-establishes connectivity and then it will re-attempt the request.|
 | return\_raw | Optional. Tells the adapter whether the raw response should be returned as well as the IAP response. This is helpful when running integration tests to save mock data. It does add overhead to the response object so it is not ideal from production.|
 | archiving | Optional flag. Default is false. It archives the request, the results and the various times (wait time, Netbox time and overall time) in the `adapterid_results` collection in MongoDB. Although archiving might be desirable, be sure to develop a strategy before enabling this capability. Consider how much to archive and what strategy to use for cleaning up the collection in the database so that it does not become too large, especially if the responses are large.|
+| return\_request | Optional flag. Default is false. Will return the actual request that is made including headers. This should only be used during debugging issues as there could be credentials in the actual request.|
 
 ### SSL Properties
 
@@ -298,6 +314,8 @@ The proxy section defines the properties to utilize when Netbox is behind a prox
 | host | Host information for the proxy server. Required if `enabled` is true.|
 | port | Port information for the proxy server. Required if `enabled` is true.|
 | protocol | The protocol (i.e., http, https, etc.) used to connect to the proxy. Default is http.|
+| username | If there is authentication for the proxy, provide the username here.|
+| password | If there is authentication for the proxy, provide the password here.|
 
 ### Mongo Properties
 
@@ -331,6 +349,7 @@ Unit Testing includes testing basic adapter functionality as well as error condi
 node utils/testRunner --unit
 
 npm run test:unit
+npm run test:baseunit
 ```
 
 To add new unit tests, edit the `test/unit/adapterTestUnit.js` file. The tests that are already in this file should provide guidance for adding additional tests.
@@ -366,6 +385,23 @@ Test should also be written to clean up after themselves. However, it is importa
 
 > **Reminder**: Do not check in code with actual credentials to systems.
 
+## Adapter Node Scripts
+
+There are several node scripts that now accompany the adapter. These scripts are provided to make several activities easier. Each of these scripts are described below.
+
+| Run | Description |
+| ------- | ------- |
+| npm run adapter:install | Provides an easier way to install the adapter.|
+| npm run adapter:checkMigrate | Checks whether your adapter can and should be migrated to the latest foundation.|
+| npm run adapter:findPath | Can be used to see if the adapter supports a particular API call.|
+| npm run adapter:migrate | Provides an easier way to migrate your adapter after you download the migration zip from Itential DevSite|
+| npm run adapter:update | Provides an easier way to update your adapter after you download the migration zip from Itential DevSite|
+| npm run adapter:revert | Allows you to revert after a migration or update if it resulted in issues.|
+| npm run troubleshoot | Provides a way to troubleshoot the adapter - runs connectivity, healthcheck and basic get.|
+| npm run connectivity | Provides a connectivity check to the Netbox system.|
+| npm run healthcheck | Checks whether the configured healthcheck call works to Netbox.|
+| npm run basicget | Checks whether the basic get calls works to Netbox.|
+
 ## Installing an Itential Product Adapter
 
 If you have App-Artifact installed in IAP, you can follow the instruction for that application to install the adapter into IAP. If not, follow these instructions.
@@ -378,24 +414,21 @@ if the @itentialopensource directory does not exist, create it:
    mkdir @itentialopensource
 ```
 
-1. Clone the adapter into your IAP environment.
+2. Clone the adapter into your IAP environment.
 
 ```bash
 cd \@itentialopensource
 git clone git@gitlab.com:\@itentialopensource/adapters/adapter-netbox
 ```
 
-1. Install the dependencies for the adapter.
+3. Run the adapter install script.
 
 ```bash
 cd adapter-netbox
-npm install
+npm run adapter:install
 ```
 
-1. If you are running IAP 2019.1 or older, add the adapter properties for Netbox (created from Adapter Builder) to the `properties.json` file for your Itential build. You will need to change the credentials and possibly the host information below.
-[Netbox sample properties](sampleProperties.json). If you are running IAP 2019.2 the adapter properties need to go into the database. You can review IAP documentation for how to do this.
-
-1. Restart IAP
+4. Restart IAP
 
 ```bash
 systemctl restart pronghorn
@@ -415,7 +448,7 @@ Depending on where your code is located, this process is different.
 Adapter should be placed into: /opt/pronghorn/current/node_modules/\@itentialopensource
 ```
 
-1. Follow Steps 3-5 (above) to install an Itential adapter to load your properties, dependencies and restart IAP.
+2. Follow Steps 3-4 (above) to install an Itential adapter to load your properties, dependencies and restart IAP.
 
 ## Using this Adapter
 
@@ -443,11 +476,6 @@ The `encryptProperty` call will take the provided property and technique, and re
 encryptProperty(property, technique, callback)
 ```
 
-The `getQueue` call will return the requests that are waiting in the queue if throttling is enabled.
-```js
-getQueue(callback)
-```
-
 The `addEntityCache` call will take the entities and add the list to the entity cache to expedite performance.
 ```js
 addEntityCache(entityType, entities, key, callback)
@@ -468,39 +496,169 @@ The `verifyCapability` call verifies the adapter can perform the provided action
 verifyCapability(entityType, actionType, entityId, callback)
 ```
 
-The `updateEntityCache` call will update the entity cache. 
+The `updateEntityCache` call will update the entity cache.
 ```js
 updateEntityCache()
 ```
 
+The `updateAdapterConfiguration` call provides the ability to update the adapter configuration from IAP - includes actions, schema, mockdata and other configurations.
+```js
+updateAdapterConfiguration(configFile, changes, entity, type, action, callback)
+```
+
+The `suspend` call provides the ability to suspend the adapter and either have requests rejected or put into a queue to be processed after the adapter is resumed.
+```js
+suspend(mode, callback)
+```
+
+The `unsuspend` call provides the ability to resume a suspended adapter. Any requests in queue will be processed before new requests.
+```js
+unsuspend(callback)
+```
+
+The `findPath` call provides the ability to see if a particular API path is supported by the adapter.
+```js
+findPath(apiPath, callback)
+```
+
+The `troubleshoot` call can be used to check on the performance of the adapter - it checks connectivity, healthcheck and basic get calls.
+```js
+troubleshoot(props, persistFlag, adapter, callback)
+```
+
+The `runHealthcheck` call will return the results of a healthcheck.
+```js
+runHealthcheck(adapter, callback)
+```
+
+The `runConnectivity` call will return the results of a connectivity check.
+```js
+runConnectivity(callback)
+```
+
+The `runBasicGet` call will return the results of running basic get API calls.
+```js
+runBasicGet(callback)
+```
+
+The `getQueue` call will return the requests that are waiting in the queue if throttling is enabled.
+```js
+getQueue(callback)
+```
+
 ### Specific Adapter Calls
 
 Specific adapter calls are built based on the API of the Netbox. The Adapter Builder creates the proper method comments for generating JS-DOC for the adapter. This is the best way to get information on the calls.
 
+
+## Extending/Enhancing the Adapter
+
+### Adding a Second Instance of an Adapter
+
+You can add a second instance of this adapter without adding new code on the file system. To do this go into the IAP Admin Essentials and add a new service config for this adapter. The two instances of the adapter should have unique ids. In addition, they should point to different instances of the other system. For example, they should be configured to talk to different hosts.
+
+### Adding Adapter Calls
+
+There are multiple ways to add calls to an existing adapter.
+
+The easiest way would be to use the Adapter Builder update process. This process takes in a Swagger or OpenAPI document, allows you to select the calls you want to add and then generates a zip file that can be used to update the adapter. Once you have the zip file simple put it in the adapter direcctory and execute `npm run adapter:update`.
+
+```bash
+mv updatePackage.zip adapter-netbox
+cd adapter-netbox
+npm run adapter:update
+```
+
+If you do not have a Swagger or OpenAPI document, you can use a Postman Collection and convert that to an OpenAPI document using APIMatic and then follow the first process.
+
+If you want to manually update the adapter that can also be done the key thing is to make sure you update all of the right files. Within the entities directory you will find 1 or more entities. You can create a new entity or add to an existing entity. Each entity has an action.json file, any new call will need to be put in the action.json file. It will also need to be added to the enum for the ph_request_type in the appropriate schema files. Once this configuration is complete you will need to add the call to the adapter.js file and in order to make it available as a workflow task in IAP, it should also be added to the pronghorn.json file. You can optionally add it to the unit and integration test files. There is more information on how to work on each of these files in the Adapter Technical Resources on Dev Site [HERE](https://developer.itential.io/adapters-resources/)
+
+```text
+Files to update
+* entities/<entity>/action.json: add an action
+* entities/<entity>/schema.json (or the schema defined on the action): add action to the enum for ph_request_type
+* adapter.js: add the new method and make sure it calls the proper entity and action
+* pronghorn.json: add the new method
+* test/unit/adapterTestUnit.js (optional but best practice): add unit test(s) - function is there, any required parameters error when not passed in
+* test/integration/adapterTestIntegration.js (optional but best practice): add integration test
+```
+
+### Adding Adapter Properties
+
+While changing adapter properties is done in the service instance configuration section of IAP, adding properties has to be done in the adapter. To add a property you should edit the propertiesSchema.json with the proper information for the property. In addition, you should modify the sampleProperties to have the new property in it.
+
+```text
+Files to update
+* propertiesSchema.json: add the new property and how it is defined
+* sampleProperties: add the new property with a default value
+* test/unit/adapterTestUnit.js (optional but best practice): add the property to the global properties
+* test/integration/adapterTestIntegration.js (optional but best practice): add the property to the global properties
+```
+
+### Changing Adapter Authentication
+
+Often an adapter is built before knowing the authentication and authentication process can also change over time. The adapter supports many different kinds of authentication but it does require configuration. Some forms of authentication can be defined entirely with the adapter properties but others require configuration.
+
+```text
+Files to update
+* entities/<entity>/action.json: change the getToken action as needed
+* entities/<entity>/schemaTokenReq.json: add input parameters (external name is name in other system)
+* entities/<entity>/schemaTokenResp.json: add response parameters (external name is name in other system)
+* propertiesSchema.json: add any new property and how it is defined
+* sampleProperties: add any new property with a default value
+* test/unit/adapterTestUnit.js (optional but best practice): add the property to the global properties
+* test/integration/adapterTestIntegration.js (optional but best practice): add the property to the global properties
+```
+
+### Enhancing Adapter Integration Tests
+
+The adapter integration tests are written to be able to test in either stub (standalone) mode or integrated to the other system. However, if integrating to the other system, you may need to provide better data than what the adapter provides by default as that data is likely to fail for create and update. To provide better data, edit the adapter integration test file. Make sure you do not remove the marker and keep custom code below the marker so you do not impact future migrations. Once the edits are complete, run the integration test as it instructs you to above. When you run integrated to the other system, you can also save mockdata for future use by changing the isSaveMockData flag to true.
+
+```text
+Files to update
+* test/integration/adapterTestIntegration.js: add better data for the create and update calls so that they will not fail.
+```
+
+As mentioned previously, for most of these changes as well as other possible changes, there is more information on how to work on an adapter in the Adapter Technical Resources on Dev Site [HERE](https://developer.itential.io/adapters-resources/)
+
 ## Troubleshooting the Adapter
 
+Run `npm run troubleshoot` to start the interactive troubleshooting process. The command allows user to verify and update connection, authentication as well as healthcheck configuration. After that it will test these properties by sending HTTP request to the endpoint. If the tests pass, it will persist these changes into IAP.
+
+User also have the option to run individual command to perform specific test
+
+- `npm run healthcheck` will perform a healthcheck request of with current setting.
+- `npm run basicget` will perform some non-parameter GET request with current setting.
+- `npm run connectivity` will perform networking diagnostics of the adatper endpoint.
+
 ### Connectivity Issues
 
-1. Verify the adapter properties are set up correctly.
+1. You can run the adapter troubleshooting script which will check connectivity, run the healthcheck and run basic get calls.
+
+```bash
+npm run troubleshoot
+```
+
+2. Verify the adapter properties are set up correctly.
 
 ```text
 Go into the Itential Platform GUI and verify/update the properties
 ```
 
-1. Verify there is connectivity between the Itential Platform Server and Netbox Server.
+3. Verify there is connectivity between the Itential Platform Server and Netbox Server.
 
 ```text
 ping the ip address of Netbox server
 try telnet to the ip address port of Netbox
 ```
 
-1. Verify the credentials provided for Netbox.
+4. Verify the credentials provided for Netbox.
 
 ```text
 login to Netbox using the provided credentials
 ```
 
-1. Verify the API of the call utilized for Netbox Healthcheck.
+5. Verify the API of the call utilized for Netbox Healthcheck.
 
 ```text
 Go into the Itential Platform GUI and verify/update the properties