@itentialopensource/adapter-datadog 0.5.2 → 0.6.0

package/README.md

@@ -1,303 +1,235 @@
-# Datadog Adapter
+# Adapter for Datadog 
+
+## Table of Contents
+
+Some of the page links in this document and links to other GitLab files do not work in Confluence however, the information is available in other sections of the Confluence material.
+
+- [Overview](./SUMMARY.md)
+- [Versioning](#versioning)
+- [Supported IAP Versions](#supported-iap-versions)
+- [Getting Started](#getting-started)
+  - [Helpful Background Information](#helpful-background-information)
+  - [Prerequisites](#prerequisites)
+  - [How to Install](#how-to-install)
+  - [Testing](#testing)
+- [Configuration](./PROPERTIES.md)
+- [Using this Adapter](./CALLS.md)
+- [Additional Information](#additional-information)
+  - [Enhancements](./ENHANCE.md)
+  - [Contributing](./CONTRIBUTING.md)
+  - [Helpful Links](#helpful-links)
+  - [Node Scripts](#node-scripts)
+- [Troubleshoot](./TROUBLESHOOT.md)
+- [License and Maintainers](#license-and-maintainers)
+- [Product License](#product-license)
+
+## [Overview](./SUMMARY.md)
 
-This adapter is used to integrate the Itential Automation Platform (IAP) with the Datadog System. The API for Datadog is available at [undefined API URL]. The adapter utilizes the Datadog API to provide the integrations that are deemed pertinent to IAP. This ReadMe file is intended to provide information on this adapter.
-
->**Note**: It is possible that some integrations will be supported through the Datadog adapter while other integrations will not.
+## Versioning
 
-Itential provides information on all of its product adapters in the Customer Knowledge Base. Information in the [Customer Knowledge Base](https://itential.atlassian.net/servicedesk/customer/portals) is consistently maintained and goes through documentation reviews. As a result, it should be the first place to go for information.
+Itential Product and opensource adapters utilize SemVer for versioning. The current version of the adapter can be found in the `package.json` file or viewed in the IAP GUI on the System page. All Itential opensource adapters can be found in the <a href="https://gitlab.com/itentialopensource/adapters" target="_blank">Itential OpenSource Repository</a>.
 
-For custom built adapters, it is a starting point to understand what you have built, provide the information for you to be able to update the adapter, and assist you with deploying the adapter into IAP.
+Any release prior to 1.0.0 is a pre-release. Initial builds of adapters are generally set up as pre-releases as there is often work that needs to be done to configure the adapter and make sure the authentication process to Datadog works appropriately.
 
-## Versioning
+Release notes can be viewed in CHANGELOG.md or in the <a href="https://itential.atlassian.net/servicedesk/customer/portals" target="_blank">Customer Knowledge Base</a> for Itential adapters.
 
-Itential Product adapters utilize SemVer for versioning. The current version of the adapter can be found in the `package.json` file or viewed in the IAP GUI on the System page. For Open Source Adapters, the versions available can be found in the [Itential OpenSource Repository](https://www.npmjs.com/search?q=itentialopensource%2Fadapter).
+## Supported IAP Versions
 
-## Release History
+Itential Product adapters are built for particular versions of IAP and packaged with the versions they work with. 
 
-Any release prior to 1.0.0 is a pre-release. Initial builds of adapters are generally set up as pre-releases as there is often work that needs to be done to configure the adapter and make sure the authentication process to Datadog works appropriately.
+Itential opensource adapter as well as custom adapters built with the Itential Adapter Builder work acoss many releases of IAP. As a result, it is not often necessary to modify an adapter when upgrading IAP. If IAP has changes that impact the pronghorn.json, like adding a new required section, this will most likely require changes to all adapters when upgrading IAP.
 
-Release notes can be viewed in CHANGELOG.md or in the [Customer Knowledge Base](https://itential.atlassian.net/servicedesk/customer/portals) for Itential adapters.
+Many of the scripts that come with all adapters built using the Itential Adapter Builder do have some dependencies on IAP or the IAP database schema and so it is possible these scripts could stop working in different versions of IAP. If you notify Itential of any issues, the Adapter Team will attempt to fix the scripts for newer releases of IAP.
 
 ## Getting Started
 
 These instructions will help you get a copy of the project on your local machine for development and testing. Reading this section is also helpful for deployments as it provides you with pertinent information on prerequisites and properties.
 
-### Environment Prerequisites
+### Helpful Background Information
 
-The following is a list of required packages for an adapter.
+There is adapter documentation available on the Itential Developer Site <a href="https://www.itential.com/automation-platform/integrations/adapters-resources/" target="_blank">HERE</a>. This documentation includes information and examples that are helpful for:
 
-```json
-Node.js
-Git
+```text
+Authentication
+Properties
+Code Files
+Action Files
+Schema Files
+Mock Data Files
+Linting and Testing
+Troubleshooting
 ```
 
-### Adapter Prerequisites
-
-The following list of packages are required for Itential product adapters or custom adapters that have been built utilizing the Itential Adapter Builder.
+Others will be added over time.
+Want to build a new adapter? Use the <a href="https://adapters.itential.io" target="_blank">Itential Adapter Builder</a>
 
-| Package | Description |
-| ------- | ------- |
-| @itentialopensource/adapter-utils | Runtime library classes for all adapters;  includes request handling, connection, throttling, and translation. |
-| ajv | Required for validation of adapter properties to integrate with Datadog. |
-| fs-extra | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
-| readline-sync | Utilized by the testRunner script that comes with the adapter;  helps to test unit and integration functionality. |
+### Prerequisites
 
-### Additional Prerequisites for Development and Testing
+The following is a list of required packages for installation on the system the adapter will run on:
 
-If you are developing and testing a custom adapter, or have testing capabilities on an Itential product adapter, you will need to install these packages as well.
+```text
+Node.js
+npm
+Git
+```
 
-```json
+The following list of packages are required for Itential opensource adapters or custom adapters that have been built utilizing the Itential Adapter Builder. You can install these packages by running npm install inside the adapter directory.
+
+<table border="1" class="bordered-table">
+  <tr>
+    <th bgcolor="lightgrey" style="padding:15px"><span style="font-size:12.0pt">Package</span></th>
+    <th bgcolor="lightgrey" style="padding:15px"><span style="font-size:12.0pt">Description</span></th>
+  </tr>
+  <tr>
+    <td style="padding:15px">@itentialopensource/adapter-utils</td>
+    <td style="padding:15px">Runtime library classes for all adapters;  includes request handling, connection, authentication throttling, and translation.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">ajv</td>
+    <td style="padding:15px">Required for validation of adapter properties to integrate with Datadog.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">axios</td>
+    <td style="padding:15px">Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">commander</td>
+    <td style="padding:15px">Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">fs-extra</td>
+    <td style="padding:15px">Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">mocha</td>
+    <td style="padding:15px">Testing library that is utilized by some of the node scripts that are included with the adapter.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">mocha-param</td>
+    <td style="padding:15px">Testing library that is utilized by some of the node scripts that are included with the adapter.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">mongodb</td>
+    <td style="padding:15px">Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">network-diagnostics</td>
+    <td style="padding:15px">Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">nyc</td>
+    <td style="padding:15px">Testing coverage library that is utilized by some of the node scripts that are included with the adapter.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">readline-sync</td>
+    <td style="padding:15px">Utilized by the node script that comes with the adapter;  helps to test unit and integration functionality.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">semver</td>
+    <td style="padding:15px">Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">winston</td>
+    <td style="padding:15px">Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality.</td>
+  </tr>
+</table>
+<br>
+
+If you are developing and testing a custom adapter, or have testing capabilities on an Itential opensource adapter, you will need to install these packages as well.
+
+```text
 chai
 eslint
 eslint-config-airbnb-base
 eslint-plugin-import
 eslint-plugin-json
-mocha
-nyc
+package-json-validator
 testdouble
-winston
 ```
 
-### Creating a Workspace
-
-The following provides a local copy of the repository along with adapter dependencies.
+### How to Install
 
-```json
-git clone git@gitlab.com:\@itentialopensource/adapters/adapter-datadog
-npm install
-```
+1. Set up the name space location in your IAP node_modules.
 
-## Adapter Properties and Descriptions
-
-This section defines **all** the properties that are available for the adapter, including detailed information on what each property is for. If you are not using certain capabilities with this adapter, you do not need to define all of the properties. An example of how the properties for this adapter can be used with tests or IAP are provided in the **Installation** section.
-
-```json
-  {
-    "id": "ALL ADAPTER PROPERTIES!!!",
-    "properties": {
-      "host": "system.access.resolved",
-      "port": 443,
-      "base_path": "/",
-      "version": "v1",
-      "cache_location": "local",
-      "save_metric": true,
-      "stub": false,
-      "protocol": "https",
-      "authentication": {
-        "auth_method": "basic user_password",
-        "username": "username",
-        "password": "password",
-        "auth_field": "header.headers.X-AUTH-TOKEN",
-        "auth_field_format": "{token}",
-        "token": "token",
-        "invalid_token_error": 401,
-        "token_timeout": 0,
-        "token_cache": "local"
-      },
-      "healthcheck": {
-        "type": "startup",
-        "frequency": 300000
-      },
-      "request": {
-        "number_retries": 3,
-        "limit_retry_error": 401,
-        "failover_codes": [404, 405],
-        "attempt_timeout": 5000,
-        "global_request": {
-          "payload": {},
-          "uriOptions": {},
-          "addlHeaders": {},
-          "authData": {}
-        },
-        "healthcheck_on_timeout": false,
-        "return_raw": false,
-        "archiving": false
-      },
-      "ssl": {
-        "ecdhCurve": "",
-        "enabled": false,
-        "accept_invalid_cert": false,
-        "ca_file": "",
-        "secure_protocol": "",
-        "ciphers": ""
-      },
-      "throttle": {
-        "throttle_enabled": false,
-        "number_pronghorns": 1,
-        "sync_async": "sync",
-        "max_in_queue": 1000,
-        "concurrent_max": 1,
-        "expire_timeout": 0,
-        "avg_runtime": 200
-      },
-      "proxy": {
-        "enabled": false,
-        "host": "localhost",
-        "port": 9999,
-        "protocol": "http"
-      },
-      "mongo": {
-        "host": "",
-        "port": 0,
-        "database": "",
-        "username": "",
-        "password": ""
-      }
-    },
-    "type": "YOUR ADAPTER CLASS"
-  }
+```bash
+cd /opt/pronghorn/current/node_modules (* could be in a different place)
+if the @itentialopensource directory does not exist, create it:
+    mkdir @itentialopensource
 ```
 
-### Connection Properties
-
-These base properties are used to connect to Datadog upon the adapter initially coming up. It is important to set these properties appropriately.
-
-| Property | Description |
-| ------- | ------- |
-| host | Required. A fully qualified domain name or IP address.|
-| port | Required. Used to connect to the server.|
-| base_path | Optional. Used to define part of a path that is consistent for all or most endpoints. It makes the URIs easier to use and maintain but can be overridden on individual calls. An example **base_path** might be `/rest/api`. Default is ``.|
-| version | Optional. Used to set a global version for action endpoints. This makes it faster to update the adapter when endpoints change. As with the base-path, version can be overridden on individual endpoints. Default is ``.|
-| cache\_location | Optional. Used to define where the adapter cache is located. The cache is used to maintain an entity list to improve performance. Storage locally is lost when the adapter is restarted. Storage in Redis is preserved upon adapter restart. Default is none which means no caching of the entity list.|
-| save\_metric | Optional. Used to tell the adapter to save metric information (this does not impact metrics returned on calls). This allows the adapter to gather metrics over time. Metric data can be stored in a database or on the file system.|
-| stub | Optional. Indicates whether the stub should run instead of making calls to Datadog (very useful during basic testing). Default is false (which means connect to Datadog).|
-| protocol | Optional. Notifies the adapter whether to use HTTP or HTTPS. Default is HTTP.|
-
-A connectivity check tells IAP the adapter has loaded successfully.
-
-### Authentication Properties
-
-The following properties are used to define the authentication process to Datadog.
-
->**Note**: Depending on the method that is used to authenticate with Datadog, you may not need to set all of the authentication properties.
+2. Clone/unzip/tar the adapter into your IAP environment.
 
-| Property | Description |
-| ------- | ------- |
-| auth\_method | Required. Used to define the type of authentication currently supported. Authentication methods currently supported are: `basic user_password`, `static_token`, `request_token`, and `no_authentication`.|
-| username | Used to authenticate with Datadog on every request or when pulling a token that will be used in subsequent requests.|
-| password | Used to authenticate with Datadog on every request or when pulling a token that will be used in subsequent requests.|
-| auth\_field | Defines the request field the authentication (e.g., token are basic auth credentials) needs to be placed in order for the calls to work.|
-| auth\_field\_format | Defines the format of the auth\_field. See examples below. Items enclosed in {} inform the adapter to perofrm an action prior to sending the data. It may be to replace the item with a value or it may be to encode the item. |
-| token | Defines a static token that can be used on all requests. Only used with `static_token` as an authentication method (auth\_method).|
-| invalid\_token\_error | Defines the HTTP error that is received when the token is invalid. Notifies the adapter to pull a new token and retry the request. Default is 401.|
-| token\_timeout | Defines how long a token is valid. Measured in milliseconds. Once a dynamic token is no longer valid, the adapter has to pull a new token. If the token\_timeout is set to -1, the adapter will pull a token on every request to Datadog. If the timeout\_token is 0, the adapter will use the expiration from the token response to determine when the token is no longer valid.|
-| token\_cache | Used to determine where the token should be stored (local memory or in Redis).|
-
-#### Examples of authentication field format
-
-```json
-"{token}",
-"Token {token}",
-"{username}:{password}",
-"Basic {b64}{username}:{password}{/b64}"
+```bash
+cd \@itentialopensource
+git clone git@gitlab.com:\@itentialopensource/adapters/adapter-datadog
+or
+unzip adapter-datadog.zip
+or
+tar -xvf adapter-datadog.tar
 ```
 
-### Healthcheck Properties
-
-The healthcheck properties defines the API that runs the healthcheck to tell the adapter that it can reach Datadog. There are currently three types of healthchecks.
-
-- None - Not recommended. Adapter will not run a healthcheck. Consequently, unable to determine before making a request if the adapter can reach Datadog.
-- Startup - Adapter will check for connectivity when the adapter initially comes up, but it will not check afterwards.
-- Intermittent - Adapter will check connectivity to Datadog at a frequency defined in the `frequency` property.
-
-| Property | Description |
-| ------- | ------- |
-| type | Required. The type of health check to run. |
-| frequency | Required if intermittent. Defines how often the health check should run. Measured in milliseconds. Default is 300000.|
+3. Run the adapter install script.
 
-### Request Properties
-
-The request section defines properties to help handle requests.
-
-| Property | Description |
-| ------- | ------- |
-| number\_retries | Tells the adapter how many times to retry a request that has either aborted or reached a limit error before giving up and returning an error.|
-| limit\_retry\_error | Optional. Indicates the http error status number to define that no capacity was available and, after waiting a short interval, the adapter can retry the request. Default is 0.|
-| failover\_codes | An array of error codes for which the adapter will send back a failover flag to IAP so that the Platform can attempt the action in another adapter.|
-| attempt\_timeout | Optional. Tells how long the adapter should wait before aborting the attempt. On abort, the adapter will do one of two things: 1) return the error; or 2) if **healthcheck\_on\_timeout** is set to true, it will abort the request and run a Healthcheck until it re-establishes connectivity to Datadog, and then will re-attempt the request that aborted. Default is 5000 milliseconds.|
-| global\_request | Optional. This is information that the adapter can include in all requests to the other system. This is easier to define and maintain than adding this information in either the code (adapter.js) or the action files.|
-| global\_request -> payload | Optional. Defines any information that should be included on all requests sent to the other system that have a payload/body.|
-| global\_request -> uriOptions | Optional. Defines any information that should be sent as untranslated  query options (e.g. page, size) on all requests to the other system.|
-| global\_request -> addlHeaders | Optioonal. Defines any headers that should be sent on all requests to the other system.|
-| global\_request -> authData | Optional. Defines any additional authentication data used to authentice with the other system. This authData needs to be consistent on every request.|
-| healthcheck\_on\_timeout | Required. Defines if the adapter should run a health check on timeout. If set to true, the adapter will abort the request and run a health check until it re-establishes connectivity and then it will re-attempt the request.|
-| return\_raw | Optional. Tells the adapter whether the raw response should be returned as well as the IAP response. This is helpful when running integration tests to save mock data. It does add overhead to the response object so it is not ideal from production.|
-| archiving | Optional flag. Default is false. It archives the request, the results and the various times (wait time, Datadog time and overall time) in the `adapterid_results` collection in MongoDB. Although archiving might be desirable, be sure to develop a strategy before enabling this capability. Consider how much to archive and what strategy to use for cleaning up the collection in the database so that it does not become too large, especially if the responses are large.|
-
-### SSL Properties
+```bash
+cd adapter-datadog
+npm run adapter:install
+```
 
-The SSL section defines the properties utilized for ssl authentication with Datadog. SSL can work two different ways: set the `accept\_invalid\_certs` flag to true (only recommended for lab environments), or provide a `ca\_file`.
+4. Restart IAP
 
-| Property | Description |
-| ------- | ------- |
-| enabled | If SSL is required, set to true. |
-| accept\_invalid\_certs | Defines if the adapter should accept invalid certificates (only recommended for lab environments). Required if SSL is enabled. Default is false.|
-| ca\_file | Defines the path name to the CA file used for SSL. If SSL is enabled and the accept invalid certifications is false, then ca_file is required.|
-| secure\_protocol | Defines the protocol (e.g., SSLv3_method) to use on the SSL request.|
-| ciphers | Required if SSL enabled. Specifies a list of SSL ciphers to use.|
-| ecdhCurve | During testing on some Node 8 environments, you need to set `ecdhCurve` to auto. If you do not, you will receive PROTO errors when attempting the calls. This is the only usage of this property and to our knowledge it only impacts Node 8 and 9. |
+```bash
+systemctl restart pronghorn
+```
 
-### Throttle Properties
+5. Change the adapter service instance configuration (host, port, credentials, etc) in IAP Admin Essentials GUI
 
-The throttle section is used when requests to Datadog must be queued (throttled). All of the properties in this section are optional.
+npm run adapter:install can be dependent on where the adapter is installed and on the version of IAP so it is subject to fail. If this happens you can replace step 3-5 above with these:
 
-| Property | Description |
-| ------- | ------- |
-| throttle\_enabled | Default is false. Defines if the adapter should use throttling o rnot. |
-| number\_pronghorns | Default is 1. Defines if throttling is done in a single Itential instance or whether requests are being throttled across multiple Itential instances (minimum = 1, maximum = 20). Throttling in a single Itential instance uses an in-memory queue so there is less overhead. Throttling across multiple Itential instances requires placing the request and queue information into a shared resource (e.g. database) so that each instance can determine what is running and what is next to run. Throttling across multiple instances requires additional I/O overhead.|
-| sync-async | This property is not used at the current time (it is for future expansion of the throttling engine).|
-| max\_in\_queue | Represents the maximum number of requests the adapter should allow into the queue before rejecting requests (minimum = 1, maximum = 5000). This is not a limit on what the adapter can handle but more about timely responses to requests. The default is currently 1000.|
-| concurrent\_max | Defines the number of requests the adapter can send to Datadog at one time (minimum = 1, maximum = 1000). The default is 1 meaning each request must be sent to Datadog in a serial manner. |
-| expire\_timeout | Default is 0. Defines a graceful timeout of the request session. After a request has completed, the adapter will wait additional time prior to sending the next request. Measured in milliseconds (minimum = 0, maximum = 60000).|
-| average\_runtime | Represents the approximate average of how long it takes Datadog to handle each request. Measured in milliseconds (minimum = 50, maximum = 60000). Default is 200. This metric has performance implications. If the runtime number is set too low, it puts extra burden on the CPU and memory as the requests will continually try to run. If the runtime number is set too high, requests may wait longer than they need to before running. The number does not need to be exact but your throttling strategy depends heavily on this number being within reason. If averages range from 50 to 250 milliseconds you might pick an average run-time somewhere in the middle so that when Datadog performance is exceptional you might run a little slower than you might like, but when it is poor you still run efficiently.|
+3. Install adapter dependencies and check the adapter.
 
-### Proxy Properties
+```bash
+cd adapter-datadog
+npm run install
+npm run lint:errors
+npm run test
+```
 
-The proxy section defines the properties to utilize when Datadog is behind a proxy server.
+4. Restart IAP
 
-| Property | Description |
-| ------- | ------- |
-| enabled | Required. Default is false. If Datadog is behind a proxy server, set enabled flag to true. |
-| host | Host information for the proxy server. Required if `enabled` is true.|
-| port | Port information for the proxy server. Required if `enabled` is true.|
-| protocol | The protocol (i.e., http, https, etc.) used to connect to the proxy. Default is http.|
+```bash
+systemctl restart pronghorn
+```
 
-### Mongo Properties
+5. Create an adapter service instance configuration in IAP Admin Essentials GUI
 
-The mongo section defines the properties used to connect to a Mongo database. Mongo can be used for throttling as well as to persist metric data. If not provided, metrics will be stored in the file system.
+6. Copy the properties from the sampleProperties.json and paste them into the service instance configuration in the inner/second properties field.
 
-| Property | Description |
-| ------- | ------- |
-| host | Optional. Host information for the mongo server.|
-| port | Optional. Port information for the mongo server.|
-| database | Optional. The database for the adapter to use for its data.|
-| username | Optional. If credentials are required to access mongo, this is the user to login as.|
-| password | Optional. If credentials are required to access mongo, this is the password to login with.|
+7. Change the adapter service instance configuration (host, port, credentials, etc) in IAP Admin Essentials GUI
 
-## Testing an Itential Product Adapter
+### Testing
 
-Mocha is generally used to test all Itential Product Adapters. There are unit tests as well as integration tests performed. Integration tests can generally be run as standalone using mock data and running the adapter in stub mode, or as integrated. When running integrated, every effort is made to prevent environmental failures, however there is still a possibility.
+Mocha is generally used to test all Itential Opensource Adapters. There are unit tests as well as integration tests performed. Integration tests can generally be run as standalone using mock data and running the adapter in stub mode, or as integrated. When running integrated, every effort is made to prevent environmental failures, however there is still a possibility.
 
-### Unit Testing
+#### Unit Testing
 
 Unit Testing includes testing basic adapter functionality as well as error conditions that are triggered in the adapter prior to any integration. There are two ways to run unit tests. The prefered method is to use the testRunner script; however, both methods are provided here.
 
-
-```json
+```bash
 node utils/testRunner --unit
 
 npm run test:unit
+npm run test:baseunit
 ```
 
 To add new unit tests, edit the `test/unit/adapterTestUnit.js` file. The tests that are already in this file should provide guidance for adding additional tests.
 
-### Integration Testing - Standalone
+#### Integration Testing - Standalone
 
 Standalone Integration Testing requires mock data to be provided with the entities. If this data is not provided, standalone integration testing will fail. When the adapter is set to run in stub mode (setting the stub property to true), the adapter will run through its code up to the point of making the request. It will then retrieve the mock data and return that as if it had received that data as the response from Datadog. It will then translate the data so that the adapter can return the expected response to the rest of the Itential software. Standalone is the default integration test.
 
 Similar to unit testing, there are two ways to run integration tests. Using the testRunner script is better because it prevents you from having to edit the test script; it will also resets information after testing is complete so that credentials are not saved in the file.
 
-```json
+```bash
 node utils/testRunner
   answer no at the first prompt
 
@@ -306,13 +238,13 @@ npm run test:integration
 
 To add new integration tests, edit the `test/integration/adapterTestIntegration.js` file. The tests that are already in this file should provide guidance for adding additional tests.
 
-### Integration Testing
+#### Integration Testing
 
 Integration Testing requires connectivity to Datadog. By using the testRunner script it prevents you from having to edit the integration test. It also resets the integration test after the test is complete so that credentials are not saved in the file.
 
 > **Note**: These tests have been written as a best effort to make them work in most environments. However, the Adapter Builder often does not have the necessary information that is required to set up valid integration tests. For example, the order of the requests can be very important and data is often required for `creates` and `updates`. Hence, integration tests may have to be enhanced before they will work (integrate) with Datadog. Even after tests have been set up properly, it is possible there are environmental constraints that could result in test failures. Some examples of possible environmental issues are customizations that have been made within Datadog which change order dependencies or required data.
 
-```json
+```bash
 node utils/testRunner
 answer yes at the first prompt
 answer all other questions on connectivity and credentials
@@ -322,164 +254,84 @@ Test should also be written to clean up after themselves. However, it is importa
 
 > **Reminder**: Do not check in code with actual credentials to systems.
 
-## Installing an Itential Product Adapter
-
-If you have App-Artifact installed in IAP, you can follow the instruction for that application to install the adapter into IAP. If not, follow these instructions.
-
-1. Set up the name space location in your IAP node_modules.
-
-```json
-cd /opt/pronghorn/current/node_modules
-if the @itentialopensource directory does not exist, create it:
-   mkdir @itentialopensource
-```
-
-1. Clone the adapter into your IAP environment.
-
-```json
-cd \@itentialopensource
-git clone git@gitlab.com:\@itentialopensource/adapters/adapter-datadog
-```
-
-1. Install the dependencies for the adapter.
-
-```json
-cd adapter-datadog
-npm install
-```
-
-1. If you are running IAP 2019.1 or older, add the adapter properties for Datadog (created from Adapter Builder) to the `properties.json` file for your Itential build. You will need to change the credentials and possibly the host information below.
-[Datadog sample properties](sampleProperties.json). If you are running IAP 2019.2 the adapter properties need to go into the database. You can review IAP documentation for how to do this.
-
-1. Restart IAP
-
-```json
-systemctl restart pronghorn
-```
-
-## Installing a Custom Adapter
-
-If you built this as a custom adapter through the Adapter Builder, it is recommended you go through setting up a development environment and testing the adapter before installing it. There is often configuration and authentication work that is required before the adapter will work in IAP.
-
-1. Move the adapter into the IAP `node_modules` directory.
-
-```json
-Depending on where your code is located, this process is different.
-    Could be a tar, move, untar
-    Could be a git clone of a repository
-    Could also be a cp -R from a coding directory
-Adapter should be placed into: /opt/pronghorn/current/node_modules/\@itentialopensource
-```
-
-1. Follow Steps 3-5 (above) to install an Itential adapter to load your properties, dependencies and restart IAP.
-
-## Using this Adapter
-
-The `adapter.js` file contains the calls the adapter makes available to the rest of the Itential Platform. The API detailed for these calls should be available through JSDOC. The following is a brief summary of the calls.
-
-### Generic Adapter Calls
-
-```json
-connect()
-The connect call is run when the Adapter is first loaded by he Itential Platform. It validates the properties have been provided correctly.
-```
-
-```json
-healthCheck(callback)
-Insures that the adapter can communicate with Datadog. The actual call that is used is defined in the adapter properties.
-```
-
-```json
-refreshProperties(properties)
-Provides the adapter the ability to accept property changes without having to restart the adapter.
-```
-
-```json
-encryptProperty(property, technique, callback)
-Will take the provided property and technique, and return the property encrypted with the technique. This allows the property to be used in the adapterProps section for the credential password so that the password does not have to be in clear text. The adapter will decrypt the property as needed for communications with Datadog.
-```
-
-```json
-getQueue(callback)
-Will return the requests that are waiting in the queue if throttling is enabled.
-```
-
-```json
-addEntityCache(entityType, entities, key, callback)
-Will take the entities and add the list to the entity cache to expedite performance.
-```
-
-```json
-capabilityResults(results, callback)
-Will take the results from a verifyCompatibility and put them in the format to be passed back to the Itential Platform.
-```
-
-```json
-hasEntity(entityType, entityId, callback)
-Verifies the adapter has the specific entity.
-```
-
-```json
-verifyCapability(entityType, actionType, entityId, callback)
-Verifies the adapter can perform the provided action on the specific entity.
-```
-
-```json
-updateEntityCache()
-Call to update the entity cache.
-```
-
-### Specific Adapter Calls
-
-Specific adapter calls are built based on the API of the Datadog. The Adapter Builder creates the proper method comments for generating JS-DOC for the adapter. This is the best way to get information on the calls.
-
-## Troubleshooting the Adapter
-
-### Connectivity Issues
-
-1. Verify the adapter properties are set up correctly.
-
-```json
-Go into the Itential Platform GUI and verify/update the properties
-```
-
-1. Verify there is connectivity between the Itential Platform Server and Datadog Server.
-
-```json
-ping the ip address of Datadog server
-try telnet to the ip address port of Datadog
-```
-
-1. Verify the credentials provided for Datadog.
-
-```json
-login to Datadog using the provided credentials
-```
-
-1. Verify the API of the call utilized for Datadog Healthcheck.
-
-```json
-Go into the Itential Platform GUI and verify/update the properties
-```
-
-### Functional Issues
-
-Adapter logs are located in `/var/log/pronghorn`. In older releases of the Itential Platform, there is a `pronghorn.log` file which contains logs for all of the Itential Platform. In newer versions, adapters are logging into their own files.
-
-## Contributing to Datadog
-
-Please check out the [Contributing Guidelines](./CONTRIBUTING.md).
-
-## License & Maintainers
-
-### Maintained By
-
-```json
-Itential Product Adapters are maintained by the Itential Adapter Team.
-Itential OpenSource Adapters are maintained by the community at large.
+## [Configuration](./PROPERTIES.md)
+
+## [Using this Adapter](./CALLS.md)
+
+### [Authentication](./AUTH.md)
+
+## Additional Information
+
+### [Enhancements](./ENHANCE.md)
+
+### [Contributing](./CONTRIBUTING.md)
+
+### Helpful Links
+
+<a href="https://www.itential.com/automation-platform/integrations/adapters-resources/" target="_blank">Adapter Technical Resources</a>
+
+### Node Scripts
+
+There are several node scripts that now accompany the adapter. These scripts are provided to make several activities easier. Many of these scripts can have issues with different versions of IAP as they have dependencies on IAP and Mongo. If you have issues with the scripts please report them to the Itential Adapter Team. Each of these scripts are described below.
+
+<table border="1" class="bordered-table">
+  <tr>
+    <th bgcolor="lightgrey" style="padding:15px"><span style="font-size:12.0pt">Run</span></th>
+    <th bgcolor="lightgrey" style="padding:15px"><span style="font-size:12.0pt">Description</span></th>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run adapter:install</td>
+    <td style="padding:15px">Provides an easier way to install the adapter.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run adapter:checkMigrate</td>
+    <td style="padding:15px">Checks whether your adapter can and should be migrated to the latest foundation.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run adapter:findPath</td>
+    <td style="padding:15px">Can be used to see if the adapter supports a particular API call.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run adapter:migrate</td>
+    <td style="padding:15px">Provides an easier way to update your adapter after you download the migration zip from Itential DevSite.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run adapter:update</td>
+    <td style="padding:15px">Provides an easier way to update your adapter after you download the update zip from Itential DevSite.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run adapter:revert</td>
+    <td style="padding:15px">Allows you to revert after a migration or update if it resulted in issues.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run troubleshoot</td>
+    <td style="padding:15px">Provides a way to troubleshoot the adapter - runs connectivity, healthcheck and basic get.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run connectivity</td>
+    <td style="padding:15px">Provides a connectivity check to the Datadog system.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run healthcheck</td>
+    <td style="padding:15px">Checks whether the configured healthcheck call works to Datadog.</td>
+  </tr>
+  <tr>
+    <td style="padding:15px">npm run basicget</td>
+    <td style="padding:15px">Checks whether the basic get calls works to Datadog.</td>
+  </tr>
+</table>
+<br>
+
+## [Troubleshoot](./TROUBLESHOOT.md)
+
+## License and Maintainers
+
+```text
+Itential Product Adapters are maintained by the Itential Product Team.
+Itential OpenSource Adapters are maintained by the Itential Adapter Team and the community at large.
 Custom Adapters are maintained by other sources.
 ```
 
-### Product License
+## Product License
 
 [Apache 2.0](./LICENSE)