@itentialopensource/adapter-kafkav2 0.16.8 → 0.17.0

package/AUTH.md

@@ -0,0 +1,26 @@
+## Authenticating Kafka Adapter 
+
+This document will go through the steps for authenticating the Kafkav2 adapter with. Properly configuring the properties for an adapter in IAP is critical for getting the adapter online. You can read more about adapter authentication <a href="https://docs.itential.com/opensource/docs/authentication" target="_blank">HERE</a>. 
+
+### Database Library Authentication
+The Kafka adapter supports SASL Authentication for Kafka server. If you change authentication methods, you should change this section accordingly and merge it back into the adapter repository.
+
+STEPS  
+1. Ensure you have access to a Kafka server and that it is running
+2. Follow the steps in the README.md to import the adapter into IAP if you have not already done so
+3. Use the properties below for the ```properties.client.sasl``` field
+```json
+"sasl": {
+          "mechanism": "PLAIN",
+          "username": "foo",
+          "password": "bar"
+        }
+
+```
+For more details on sasl authentication follow the CONFIGURATION.md
+4. Restart the adapter. If your properties were set correctly, the adapter should go online. 
+
+### Troubleshooting
+- Make sure you copied over the correct username and password.
+- Turn on debug level logs for the adapter in IAP Admin Essentials.
+- Investigate the logs

package/CHANGELOG.md

@@ -1,4 +1,12 @@
 
+## 0.17.0 [01-08-2024]
+
+* Migration changes
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!18
+
+---
+
 ## 0.16.8 [12-10-2023]
 
 * Fix bug to check allowed partition

package/CODE_OF_CONDUCT.md

@@ -8,19 +8,19 @@ In the interest of fostering an open and welcoming environment, we as contributo
 
 Examples of behavior that contributes to creating a positive environment include:
 
-* Using welcoming and inclusive language
-* Being respectful of differing viewpoints and experiences
-* Gracefully accepting constructive criticism
-* Focusing on what is best for the community
-* Showing empathy towards other community members
+- Using welcoming and inclusive language
+- Being respectful of differing viewpoints and experiences
+- Gracefully accepting constructive criticism
+- Focusing on what is best for the community
+- Showing empathy towards other community members
 
 Examples of unacceptable behavior by participants include:
 
-* The use of sexualized language or imagery and unwelcome sexual attention or advances
-* Trolling, insulting/derogatory comments, and personal or political attacks
-* Public or private harassment
-* Publishing others' private information, such as a physical or electronic address, without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a professional setting
+- The use of sexualized language or imagery and unwelcome sexual attention or advances
+- Trolling, insulting/derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or electronic address, without explicit permission
+- Other conduct which could reasonably be considered inappropriate in a professional setting
 
 ## Our Responsibilities
 
@@ -34,15 +34,10 @@ This Code of Conduct applies both within project spaces and in public spaces whe
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at [support@itential.com](mailto:support@itential.com). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at support@itential.com. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
 
 Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
 
 ## Attribution
 
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
-
-[homepage]: http://contributor-covenant.org
-[version]: http://contributor-covenant.org/version/1/4/
-
-_return to [README](../README.md)_
+This Code of Conduct is adapted from the <a href="http://contributor-covenant.org" target="_blank">Contributor Covenant</a>, version 1.4, available at <a href="http://contributor-covenant.org/version/1/4/" target="_blank">version</a>

package/CONTRIBUTING.md

@@ -1,158 +1,13 @@
 # Contributing
 
-:+1: :tada: First off, thanks for taking the time to contribute! :tada: :+1:
+First off, thanks for taking the time to contribute!
 
 The following is a set of rules for contributing.
 
-## Table Of Contents
-
-[Code of Conduct](#code-of-conduct)
-
-[How Can I Contribute?](#how-can-i-contribute)
-
-* [Reporting Bugs](#reporting-bugs)
-* [Your First Code Contribution](#your-first-code-contribution)
-* [Merge Requests](#merge-requests)
-
-[Styleguides](#styleguides)
-
-* [Git Commit Messages](#git-commit-messages)
-* [JavaScript Styleguide](#javascript-styleguide)
-* [Tests Styleguide](#tests-styleguide)
-* [Documentation Styleguide](#documentation-styleguide)
-
 ## Code of Conduct
 
-This project and everyone participating in it is governed by the [Code of Conduct](./CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [support@itential.com](mailto:support@itential.com).
+This project and everyone participating in it is governed by the Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to support@itential.com.
 
 ## How to Contribute
 
-### Reporting Bugs
-
-This section guides you through submitting a bug report. Following these guidelines helps maintainers understand your report :pencil:, reproduce the behavior :computer: :left_right_arrow: :computer:, and find related reports :mag_right:.
-
-Before creating bug reports, please check [this list](#before-submitting-a-bug-report) as you might find out that you don't need to create one. When you are creating a bug report, please [include as many details as possible](#how-do-i-submit-a-good-bug-report). If you have an Itential support contract, please create an Itential Service Desk ticket. If you don't have an Itential support contract, please send an email of the issue to [support@itential.com](mailto:support@itential.com).
-
-#### Before Submitting A Bug Report
-
-* **Check the [User Guide](README.md).** You might be able to find the cause of the problem and fix things yourself. Most importantly, check if you can reproduce the problem __in the latest version__.
-* **Work through the troubleshooting process.** Troubleshooting will include changing the log level of Itential Automation Platform (IAP) and adapters and checking the logs to see what the issues are. These logs should be included in any ticket you open for this issue.
-* **Check for resolutions to the issue.** Check the [Itential Knowledge Base](https://itential.atlassian.net/servicedesk/customer/kb/view/286883841?applicationId=605994d2-2cb2-3174-af59-ed5b23ff5fd5&spaceKey=PKB&src=1187380921) to see if there is a known resolution for the issue you are having.
-* **Ask around in chat if you are an Itential employee** to see if others are experiencing the same issue.
-
-#### How to Submit a (Good) Bug Report
-
-Bugs are tracked through the Itential Service Desk. Explain the problem and include additional details to help maintainers reproduce the problem:
-
-* **Use a clear and descriptive title** for the issue to identify the problem.
-* **Describe the exact steps which reproduce the problem** in as much detail as possible. For example, start by explaining how you configured the adapter (e.g., which properties you used and how they were set) or how you are using an artifact.
-* **Provide specific examples to demonstrate the steps**. Include logs, or copy/paste snippets, in your examples.
-* **Describe the behavior you observed after following the steps** and point out what exactly is the problem with that behavior.
-* **Explain which behavior you expected to see instead and why.**
-* **Include screenshots and animated GIFs** which show you following the described steps and clearly demonstrate the problem. You can use [this tool](https://www.cockos.com/licecap/) to record GIFs on macOS and Windows. Use [this tool](https://github.com/colinkeenan/silentcast) or [program](https://github.com/rhcarvalho/byzanz-guiz) on Linux.
-* **If the problem wasn't triggered by a specific action**, describe what you were doing before the problem happened and share more information using the guidelines below.
-
-Provide more context by answering these questions:
-
-* **Did the problem start happening recently** (e.g. after updating to a new version/tag) or was this always a problem?
-* If the problem started happening recently, **can you reproduce the problem in an older version/tag?** What's the most recent version in which the problem does not happen?
-* **Can you reliably reproduce the issue?** If not, provide details about how often the problem happens and under which conditions it normally happens.
-
-Include details about your configuration and environment:
-
-* **Which version of the adapter/artifact are you using?** You can get the exact version by checking the project version in the package.json file.
-* **What's the name and version of the OS you're using**?
-* **Are you running or using IAP in a virtual machine?** If so, which VM software are you using and which operating systems and versions are used for the host and guest?
-* **Are there firewalls between IAP and any other system you are integrating with?** If so, have you tried running curl from the system IAP is on to the other system to verify you have connectivity between the systems?
-
-### Your First Code Contribution
-
-#### Local development
-
-This project can be developed locally on all operating systems. For instructions on how to do this, follow the steps highlighted in the [User Guide](README.md). The [User Guide](README.md)provides many more details on cloning the repository for local development, how to lint and test your changes. 
-
-#### Development Process
-
-The following provides a local copy of the repository along with dependencies.
-
-```json
-go to the repository you are interested in working on
-click on the Clone button to get the link(s) you can use to clone the repository
-git clone <link to clone the project>
-npm install
-```
-
-Prior to making changes you should create a branch to work in. The branch should be named with the type of change (major, minor or patch) and then a Jira issue number or a description of the issue.
-
-```json
-git checkout -b <name_of_your_new_branch>
-```
-
-Make all of the desired changes.
-
-> **Note:** All code changes should be accompanied by additional Unit and Integration tests to minimize the likelihood that any changes will negatively impact the adapter/artifact as well as to ensure the desired functionality is achieved.
-
-| Files | Changes |
-| ------- | ------- |
-| .codeclimate.yml, .eslintignore, .eslintrc.js, .gitignore, .gitlab-ci.yml, .jshintrc, .npmignore | These are foundational files that are used in linting and building of the adapter. Changes to these files are heavily scrutinized and may be implemented in a different manner so that the changes can be implemented across all adapters.|
-| CODE_OF_CONDUCT.md, CONTRIBUTING.md, LICENSE, README.md | These are license and process files for the repository. Changes to these files may require legal review and may also be implemented in a different manner so that the changes can be implemented across all adapters.|
-| utils/*, adapterBase.js | Changes to these files will rarely be accepted. These are process files that need to be changed through the builder process and migration.|
-| adapter.js | Changes to this file are code changes. They should be accompanied by the appropriate unit and integration tests. If new methods are added, they should also be added to the pronghorn.json file, otherwise integration tests are likely to fail.|
-| error.json, propertiesSchema.json | These files are utilized by the Itential Runtime libraries. You can make changes to these files, but the changes should be appending of new errors or properties. Removal of errors and properties can result in rejection of the changes.|
-| pacakge.json, package-lock.json | The package-lock.json is automatically generated and should only be updated when the dependencies inside of the package.json are modified. You can make changes to the package.json to add additional scripts, add new dependencies, modify names, etc. However, changing the scripts or dependencies that are Itential specific will be overriden during the acceptance process.|
-| pronghorn.json | Changes to this file are made for integration into IAP. The most common change here is adding new methods that shoud be exposed to the IAP Workflow.|
-| entities/* | Changes to these files include adding new capabilities, updating existing capabilities, or removing deprecated capabilities in the adapter. Translation configuration can also be modified here.|
-| test/* | Changes to these files include adding, updating, or removing unit and integration tests in the adapter. Tests should never be removed unless the methods that are being tested are removed also. Both the unit and integration test file should have a marker that identifies where modification can take place. Modifications above that marker are likely to be rejected or implemented in a different manner.|
-
-> **Note:** It is a best practice to lint and test your code prior to committing changes into the repository. You can lint your code by running **npm run lint** and test it by running **node utils/testRunner.sh**. However, there are pre-commit hooks that will run these scripts when you commit your changes.
-
-Commit the changes to the repository.
-
-```json
-git commit -a -m "<descriptive message>" 
-```
-
-> **Note:** There are pre-commit hooks in place. If the pre-commit hooks fail, you will need to address those issues before moving forward.
-
-Push the changes into the repository. This should only be done after the commit has successfully completed.
-
-```json
-git push origin <name_of_your_new_branch>
-```
-
-Create a [Merge Request](#merge-request).
-
-### Merge Requests
-
-* Fill out the provided merge request template.
-* Reference related issues and merge requests liberally.
-* Include screenshots and animated GIFs in your merge request whenever possible.
-* Follow the project [Styleguide](#styleguides).
-* End all files with a newline.
-
-## Styleguide
-
-### Git Commit Messages
-
-* Use the present tense ("Add feature" not "Added feature")
-* Use the imperative mood ("Move cursor to..." not "Moves cursor to...")
-* Consider starting the commit message with an applicable emoji _(not required)_:
-    * :art: `:art:` When improving the format/structure of the code.
-    * :racehorse: `:racehorse:` When improving performance
-    * :non-potable_water: `:non-potable_water:` When plugging memory leaks.
-    * :memo: `:memo:` When writing documentation.
-    * :penguin: `:penguin:` When fixing something on Linux.
-    * :apple: `:apple:` When fixing something on macOS.
-    * :checkered_flag: `:checkered_flag:` When fixing something on Windows.
-    * :bug: `:bug:` When fixing a bug.
-    * :fire: `:fire:` When removing code or files.
-    * :green_heart: `:green_heart:` When fixing the CI build.
-    * :white_check_mark: `:white_check_mark:` When adding tests,
-    * :lock: `:lock:` When dealing with security.
-    * :arrow_up: `:arrow_up:` When upgrading dependencies.
-    * :arrow_down: `:arrow_down:` When downgrading dependencies.
-    * :shirt: `:shirt:` When removing linter warnings.
-
-__Avoid installing unnecessary packages.__ Do not install packages just because they might be "nice to have". Itential Opensource projects are supposed to be minimal, specific, and compact by design.
-
-_return to [README](./README.md)_
+Follow the contributing guide (here)[https://gitlab.com/itentialopensource/adapters/contributing-guide]

package/ENHANCE.md

@@ -0,0 +1,69 @@
+## Enhancements
+
+### Adding a Second Instance of an Adapter
+
+You can add a second instance of this adapter without adding new code on the file system. To do this go into the IAP Admin Essentials and add a new service config for this adapter. The two instances of the adapter should have unique ids. In addition, they should point to different instances (unique host and port) of the other system.
+
+### Adding Adapter Calls
+
+There are multiple ways to add calls to an existing adapter.
+
+The easiest way would be to use the Adapter Builder update process. This process takes in a Swagger or OpenAPI document, allows you to select the calls you want to add and then generates a zip file that can be used to update the adapter. Once you have the zip file simply put it in the adapter directory and execute `npm run adapter:update`.
+
+```bash
+mv updatePackage.zip adapter-kafka
+cd adapter-kafka
+npm run adapter:update
+```
+
+If you do not have a Swagger or OpenAPI document, you can use a Postman Collection and convert that to an OpenAPI document using APIMatic and then follow the first process.
+
+If you want to manually update the adapter that can also be done the key thing is to make sure you update all of the right files. Within the entities directory you will find 1 or more entities. You can create a new entity or add to an existing entity. Each entity has an action.json file, any new call will need to be put in the action.json file. It will also need to be added to the enum for the ph_request_type in the appropriate schema files. Once this configuration is complete you will need to add the call to the adapter.js file and, in order to make it available as a workflow task in IAP, it should also be added to the pronghorn.json file. You can optionally add it to the unit and integration test files. There is more information on how to work on each of these files in the <a href="https://docs.itential.com/opensource/docs/adapters" target="_blank">Adapter Technical Resources</a> on our Documentation Site.
+
+```text
+Files to update
+* entities/<entity>/action.json: add an action
+* entities/<entity>/schema.json (or the schema defined on the action): add action to the enum for ph_request_type
+* adapter.js: add the new method and make sure it calls the proper entity and action
+* pronghorn.json: add the new method
+* test/unit/adapterTestUnit.js (optional but best practice): add unit test(s) - function is there, any required parameters error when not passed in
+* test/integration/adapterTestIntegration.js (optional but best practice): add integration test
+```
+
+### Adding Adapter Properties
+
+While changing adapter properties is done in the service instance configuration section of IAP, adding properties has to be done in the adapter. To add a property you should edit the propertiesSchema.json with the proper information for the property. In addition, you should modify the sampleProperties to have the new property in it.
+
+```text
+Files to update
+* propertiesSchema.json: add the new property and how it is defined
+* sampleProperties: add the new property with a default value
+* test/unit/adapterTestUnit.js (optional but best practice): add the property to the global properties
+* test/integration/adapterTestIntegration.js (optional but best practice): add the property to the global properties
+```
+
+### Changing Adapter Authentication
+
+Often an adapter is built before knowing the authentication and authentication processes can also change over time. The adapter supports many different kinds of authentication but it does require configuration. Some forms of authentication can be defined entirely with the adapter properties but others require configuration.
+
+```text
+Files to update
+* entities/.system/action.json: change the getToken action as needed
+* entities/.system/schemaTokenReq.json: add input parameters (external name is name in other system)
+* entities/.system/schemaTokenResp.json: add response parameters (external name is name in other system)
+* propertiesSchema.json: add any new property and how it is defined
+* sampleProperties: add any new property with a default value
+* test/unit/adapterTestUnit.js (optional but best practice): add the property to the global properties
+* test/integration/adapterTestIntegration.js (optional but best practice): add the property to the global properties
+```
+
+### Enhancing Adapter Integration Tests
+
+The adapter integration tests are written to be able to test in either stub (standalone) mode or integrated to the other system. However, if integrating to the other system, you may need to provide better data than what the adapter provides by default as that data is likely to fail for create and update. To provide better data, edit the adapter integration test file. Make sure you do not remove the marker and keep custom code below the marker so you do not impact future migrations. Once the edits are complete, run the integration test as it instructs you to above. When you run integrated to the other system, you can also save mockdata for future use by changing the isSaveMockData flag to true.
+
+```text
+Files to update
+* test/integration/adapterTestIntegration.js: add better data for the create and update calls so that they will not fail.
+```
+
+As mentioned previously, for most of these changes as well as other possible changes, there is more information on how to work on an adapter in the <a href="https://docs.itential.com/opensource/docs/adapters" target="_blank">Adapter Technical Resources</a> on our Documentation Site.

package/changelogs/CHANGELOG.md

@@ -0,0 +1,72 @@
+
+## 0.16.8 [12-10-2023]
+
+* Fix bug to check allowed partition
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!17
+
+---
+
+## 0.16.7 [09-27-2023]
+
+* Big fixes logLevel and adapter restart
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!15
+
+---
+
+## 0.16.6 [09-18-2023]
+
+* Big fixes logLevel and adapter restart
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!15
+
+---
+
+## 0.16.5 [09-08-2023]
+
+* Big fixes logLevel and adapter restart
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!15
+
+---
+
+## 0.16.4 [09-08-2023]
+
+* Big fixes logLevel and adapter restart
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!15
+
+---
+
+## 0.16.3 [09-05-2023]
+
+* Big fixes logLevel and adapter restart
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!15
+
+---
+
+## 0.16.2 [08-29-2023]
+
+* Big fixes logLevel and adapter restart
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!15
+
+---
+
+## 0.16.1 [08-24-2023]
+
+* Handle empty topics file
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!13
+
+---
+
+## 0.16.0 [08-23-2023]
+
+* Add logic to clean topics.json from service config
+
+See merge request itentialopensource/adapters/notification-messaging/adapter-kafkav2!11
+
+---

package/error.json

@@ -96,6 +96,12 @@
       "displayString": "Invalid properties: $VARIABLE$",
       "recommendation": "Verify the properties for the adapter"
     },
+    {
+      "key": "Missing Properties",
+      "icode": "AD.306",
+      "displayString": "Property $VARIABLE$ is required",
+      "recommendation": "Please provide the required property"
+    },
     {
       "key": "Query Not Translated",
       "icode": "AD.310",
@@ -168,6 +174,12 @@
       "displayString": "Failure response received for $VARIABLE$",
       "recommendation": "Check the reason for failure in the stack trace"
     },
+    {
+      "key": "Suspended Adapter",
+      "icode": "AD.600",
+      "displayString": "Adapter is suspended",
+      "recommendation": "Check if external system is functional and unsuspend if appropriate"
+    },
     {
       "key": "Caught Exception",
       "icode": "AD.900",

package/metadata.json

@@ -0,0 +1,71 @@
+{
+    "name": "adapter-kafkav2",
+    "vendor": "kafkav2",
+    "product": "kafkav2",
+    "osVersion": [],
+    "apiVersions": [],
+    "iapVersions": [
+        "2021.1.x",
+        "2021.2.x",
+        "2022.1.x",
+        "2023.1.x"
+    ],
+    "method": "Library",
+    "type": "Adapter",
+    "domains": [],
+    "tags": [],
+    "useCases": [],
+    "deprecated": {
+        "isDeprecated": false
+    },
+    "brokerSince": "",
+    "documentation": {
+        "storeLink": "",
+        "npmLink": "https://www.npmjs.com/package/@itentialopensource/adapter-kafkav2",
+        "repoLink": "https://gitlab.com/itentialopensource/adapters/notification-messaging/adapter-kafkav2",
+        "docLink": "",
+        "demoLinks": [],
+        "trainingLinks": [],
+        "faqLink": "https://docs.itential.com/opensource/docs/troubleshooting-an-adapter",
+        "contributeLink": "https://gitlab.com/itentialopensource/adapters/contributing-guide",
+        "issueLink": "https://itential.atlassian.net/servicedesk/customer/portals",
+        "webLink": "",
+        "vendorLink": "",
+        "productLink": "",
+        "apiLinks": []
+    },
+    "assets": [],
+    "relatedItems": {
+        "adapters": [],
+        "integrations": [],
+        "ecosystemApplications": [],
+        "workflowProjects": [
+            {
+                "name": "Receive Message Using Kafkav2 Adapter",
+                "webName": "Receive Message Using Kafkav2 Adapter",
+                "overview": "This pre-built is configured to run by an event trigger named `Kafka Message Event Trigger` in the Operations Manager application. This event type is configured as a Kafka message from the KafkaV2 adapter. When a Kafka message is received, the trigger will start the workflow `Main Kafka Receiver`. This workflow contains two child jobs. One of the child jobs will run based on the evaluation task in the workflow. The evaluation task is configured to do the following: - If the message string value contains `workflowA`, the automation will take the success path and trigger `workflow A`- If the message string value contains `workflowB`, the automation will take the failure path and trigger `workflow B`",
+                "isDependency": false,
+                "versions": [
+                    "0.1.2"
+                ],
+                "repoLink": "https://gitlab.com/itentialopensource/pre-built-automations/receive-message-using-kafkav2-adapter",
+                "docLink": "",
+                "webLink": "https://www.itential.com/automations/receive-message-using-kafkav2-adapter/"
+            },
+            {
+                "name": "Send Message With Kafkav2 Adapter",
+                "webName": "Send Message With Kafkav2 Adapter",
+                "overview": "This pre-built is designed to send a Kafka message using the Kafka v2 adapter. This automation consists of a manual trigger in Operations Manager, which uses a JSON form to allow the user to create a message under a specific topic. The output of this JSON form will be mapped to the variables required by the workflow `Kafka Message Producer` to send the Kafka message.",
+                "isDependency": false,
+                "versions": [
+                    "0.1.2"
+                ],
+                "repoLink": "https://gitlab.com/itentialopensource/pre-built-automations/send-message-with-kafkav2-adapter",
+                "docLink": "",
+                "webLink": "https://www.itential.com/automations/send-message-with-kafkav2-adapter/"
+            }
+        ],
+        "transformationProjects": [],
+        "exampleProjects": []
+    }
+}

package/package.json

@@ -1,14 +1,15 @@
 {
   "name": "@itentialopensource/adapter-kafkav2",
-  "version": "0.16.8",
+  "version": "0.17.0",
   "description": "Itential adapter to connect to kafka",
   "main": "adapter.js",
   "wizardVersion": "2.35.0",
   "engineVersion": "1.40.2",
   "scripts": {
     "artifactize": "npm i && node utils/packModificationScript.js",
-    "preinstall": "node utils/setup.js && npm install --package-lock-only --ignore-scripts",
-    "lint": "eslint . --ext .json --ext .js",
+    "preinstall": "node utils/setup.js",
+    "deinstall": "node utils/removeHooks.js",
+    "lint": "node --max_old_space_size=4096 ./node_modules/eslint/bin/eslint.js . --ext .json --ext .js",
     "lint:errors": "node --max_old_space_size=4096 ./node_modules/eslint/bin/eslint.js . --ext .json --ext .js --quiet",
     "test:baseunit": "mocha test/unit/adapterBaseTestUnit.js --LOG=error",
     "test:unit": "mocha test/unit/adapterTestUnit.js --LOG=error",

package/propertiesDecorators.json

@@ -0,0 +1,14 @@
+[
+    {
+      "type": "encryption",
+      "pointer": "/authentication/password"
+    },
+    {
+      "type": "encryption",
+      "pointer": "/authentication/token"
+    },
+    {
+      "type": "encryption",
+      "pointer": "/mongo/password"
+    }
+  ]

package/utils/adapterInfo.js

@@ -0,0 +1,206 @@
+#!/usr/bin/env node
+/* @copyright Itential, LLC 2019 */
+/* eslint global-require:warn */
+/* eslint import/no-dynamic-require:warn */
+/* eslint prefer-destructuring:warn */
+
+const path = require('path');
+const fs = require('fs-extra');
+
+/**
+ * This script will determine the information about the adapter and store
+ * it into a file in the adapter.
+ */
+
+/**
+ * get adapter information
+ */
+function adapterInfo() {
+  // set the base pase of the adapter - tool shoud be one level up in utils
+  let adaptdir = __dirname;
+  const infoRes = {};
+
+  if (adaptdir.endsWith('/utils')) {
+    adaptdir = adaptdir.substring(0, adaptdir.length - 6);
+  }
+  const pack = require(`${adaptdir}/package.json`);
+  infoRes.version = pack.version;
+
+  let configCount = 0;
+  if (fs.existsSync(`${adaptdir}/pronghorn.json`)) {
+    const cFile = fs.readFileSync(`${adaptdir}/pronghorn.json`, 'utf8');
+    configCount += cFile.split('\n').length;
+  } else {
+    console.log('Missing - pronghorn.json');
+  }
+  if (fs.existsSync(`${adaptdir}/propertiesSchema.json`)) {
+    const cFile = fs.readFileSync(`${adaptdir}/propertiesSchema.json`, 'utf8');
+    configCount += cFile.split('\n').length;
+  } else {
+    console.log('Missing - propertiesSchema.json');
+  }
+  if (fs.existsSync(`${adaptdir}/error.json`)) {
+    const cFile = fs.readFileSync(`${adaptdir}/error.json`, 'utf8');
+    configCount += cFile.split('\n').length;
+  } else {
+    console.log('Missing - error.json');
+  }
+  const entitydir = path.join(adaptdir, '/entities');
+  if (fs.existsSync(entitydir) && fs.statSync(entitydir).isDirectory()) {
+    const entities = fs.readdirSync(entitydir);
+    // need to go through each entity in the entities directory
+    for (let e = 0; e < entities.length; e += 1) {
+      if (fs.statSync(`${entitydir}/${entities[e]}`).isDirectory()) {
+        const cfiles = fs.readdirSync(entitydir);
+        for (let c = 0; c < cfiles.length; c += 1) {
+          if (cfiles[c].endsWith('.json')) {
+            const ccFile = fs.readFileSync(`${entitydir}/${entities[e]}/${cfiles[c]}`, 'utf8');
+            configCount += ccFile.split('\n').length;
+          }
+        }
+      }
+    }
+  } else {
+    console.log('Could not find the entities directory');
+  }
+  infoRes.configLines = configCount;
+
+  let scodeCount = 0;
+  if (fs.existsSync(`${adaptdir}/utils/artifactize.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/artifactize.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/artifactize.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/basicGet.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/basicGet.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/basicGet.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/checkMigrate.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/checkMigrate.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/checkMigrate.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/findPath.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/findPath.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/findPath.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/modify.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/modify.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/modify.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/packModificationScript.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/packModificationScript.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/packModificationScript.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/setup.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/setup.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/setup.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/tbScript.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/tbScript.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/tbScript.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/tbUtils.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/tbUtils.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/tbUtils.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/testRunner.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/testRunner.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/testRunner.js');
+  }
+  if (fs.existsSync(`${adaptdir}/utils/troubleshootingAdapter.js`)) {
+    const sFile = fs.readFileSync(`${adaptdir}/utils/troubleshootingAdapter.js`, 'utf8');
+    scodeCount += sFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/troubleshootingAdapter.js');
+  }
+  infoRes.scriptLines = scodeCount;
+
+  let codeCount = 0;
+  if (fs.existsSync(`${adaptdir}/adapter.js`)) {
+    const aFile = fs.readFileSync(`${adaptdir}/adapter.js`, 'utf8');
+    codeCount += aFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/adapter.js');
+  }
+  if (fs.existsSync(`${adaptdir}/adapterBase.js`)) {
+    const aFile = fs.readFileSync(`${adaptdir}/adapterBase.js`, 'utf8');
+    codeCount += aFile.split('\n').length;
+  } else {
+    console.log('Missing - utils/adapterBase.js');
+  }
+  infoRes.codeLines = codeCount;
+
+  let tcodeCount = 0;
+  let ttestCount = 0;
+  if (fs.existsSync(`${adaptdir}/test/integration/adapterTestBasicGet.js`)) {
+    const tFile = fs.readFileSync(`${adaptdir}/test/integration/adapterTestBasicGet.js`, 'utf8');
+    tcodeCount += tFile.split('\n').length;
+    ttestCount += tFile.split('it(\'').length;
+  } else {
+    console.log('Missing - test/integration/adapterTestBasicGet.js');
+  }
+  if (fs.existsSync(`${adaptdir}/test/integration/adapterTestConnectivity.js`)) {
+    const tFile = fs.readFileSync(`${adaptdir}/test/integration/adapterTestConnectivity.js`, 'utf8');
+    tcodeCount += tFile.split('\n').length;
+    ttestCount += tFile.split('it(\'').length;
+  } else {
+    console.log('Missing - test/integration/adapterTestConnectivity.js');
+  }
+  if (fs.existsSync(`${adaptdir}/test/integration/adapterTestIntegration.js`)) {
+    const tFile = fs.readFileSync(`${adaptdir}/test/integration/adapterTestIntegration.js`, 'utf8');
+    tcodeCount += tFile.split('\n').length;
+    ttestCount += tFile.split('it(\'').length;
+  } else {
+    console.log('Missing - test/integration/adapterTestIntegration.js');
+  }
+  if (fs.existsSync(`${adaptdir}/test/unit/adapterBaseTestUnit.js`)) {
+    const tFile = fs.readFileSync(`${adaptdir}/test/unit/adapterBaseTestUnit.js`, 'utf8');
+    tcodeCount += tFile.split('\n').length;
+    ttestCount += tFile.split('it(\'').length;
+  } else {
+    console.log('Missing - test/unit/adapterBaseTestUnit.js');
+  }
+  if (fs.existsSync(`${adaptdir}/test/unit/adapterTestUnit.js`)) {
+    const tFile = fs.readFileSync(`${adaptdir}/test/unit/adapterTestUnit.js`, 'utf8');
+    tcodeCount += tFile.split('\n').length;
+    ttestCount += tFile.split('it(\'').length;
+  } else {
+    console.log('Missing - test/unit/adapterTestUnit.js');
+  }
+  infoRes.testLines = tcodeCount;
+  infoRes.testCases = ttestCount;
+  infoRes.totalCodeLines = scodeCount + codeCount + tcodeCount;
+
+  if (fs.existsSync(`${adaptdir}/pronghorn.json`)) {
+    // Read the entity schema from the file system
+    const phFile = path.join(adaptdir, '/pronghorn.json');
+    const prong = require(phFile);
+    infoRes.wfTasks = prong.methods.length;
+  } else {
+    console.log('Missing - pronghorn.json');
+  }
+
+  console.log(JSON.stringify(infoRes));
+  fs.writeFileSync(`${adaptdir}/report/adapterInfo.json`, JSON.stringify(infoRes, null, 2));
+}
+
+adapterInfo();

package/utils/artifactize.js

@@ -1,8 +1,8 @@
 #!/usr/bin/env node
 /* @copyright Itential, LLC 2019 */
 
-const fs = require('fs-extra');
 const path = require('path');
+const fs = require('fs-extra');
 
 async function createBundle(adapterOldDir) {
   // set directories

package/utils/packModificationScript.js

@@ -1,9 +1,9 @@
 #!/usr/bin/env node
 /* @copyright Itential, LLC 2019 */
 
-const fs = require('fs-extra');
 const path = require('path');
 const { spawnSync } = require('child_process');
+const fs = require('fs-extra');
 const { createBundle } = require('./artifactize');
 
 const nodeEntryPath = path.resolve('.');

package/utils/removeHooks.js

@@ -0,0 +1,20 @@
+const fs = require('fs');
+
+/**
+ * This script will uninstall pre-commit or pre-push hooks in case there's ever a need to
+ * commit/push something that has issues
+ */
+
+const precommitPath = '.git/hooks/pre-commit';
+const prepushPath = '.git/hooks/pre-push';
+fs.unlink(precommitPath, (err) => {
+  if (err && err.code !== 'ENOENT') {
+    console.log(`${err.message}`);
+  }
+});
+
+fs.unlink(prepushPath, (err) => {
+  if (err && err.code !== 'ENOENT') {
+    console.log(`${err.message}`);
+  }
+});

package/utils/testRunner.js

@@ -1,9 +1,9 @@
 #!/usr/bin/env node
 /* @copyright Itential, LLC 2019 */
 
+const execute = require('child_process').exec;
 const fs = require('fs-extra');
 const rl = require('readline-sync');
-const execute = require('child_process').exec;
 
 /**
  * This script will determine the type of integration test to run
@@ -47,11 +47,11 @@ function replaceTestVars(test) {
   let intTest = fs.readFileSync(test, 'utf8');
 
   // replace stub variable but check if it exists first
-  let sindex = intTest.indexOf('const stub');
+  let sindex = intTest.indexOf('samProps.stub');
   let eindex = intTest.indexOf(';', sindex);
   let replStr = intTest.substring(sindex, eindex + 1);
   if (sindex > -1) {
-    intTest = intTest.replace(replStr, `const stub = ${stub};`);
+    intTest = intTest.replace(replStr, `samProps.stub = ${stub};`);
   }
 
   // replace isRapidFail variable but check if it exists first
@@ -71,46 +71,46 @@ function replaceTestVars(test) {
   }
 
   // replace host variable
-  sindex = intTest.indexOf('const host');
+  sindex = intTest.indexOf('samProps.host');
   eindex = intTest.indexOf(';', sindex);
   replStr = intTest.substring(sindex, eindex + 1);
-  intTest = intTest.replace(replStr, `const host = '${host}';`);
+  intTest = intTest.replace(replStr, `samProps.host = '${host}';`);
 
   // replace username variable
-  sindex = intTest.indexOf('const username');
+  sindex = intTest.indexOf('samProps.authentication.username');
   eindex = intTest.indexOf(';', sindex);
   replStr = intTest.substring(sindex, eindex + 1);
-  intTest = intTest.replace(replStr, `const username = '${username}';`);
+  intTest = intTest.replace(replStr, `samProps.authentication.username = '${username}';`);
 
   // replace password variable
-  sindex = intTest.indexOf('const password');
+  sindex = intTest.indexOf('samProps.authentication.password');
   eindex = intTest.indexOf(';', sindex);
   replStr = intTest.substring(sindex, eindex + 1);
-  intTest = intTest.replace(replStr, `const password = '${password}';`);
+  intTest = intTest.replace(replStr, `samProps.authentication.password = '${password}';`);
 
   // replace protocol variable
-  sindex = intTest.indexOf('const protocol');
+  sindex = intTest.indexOf('samProps.protocol');
   eindex = intTest.indexOf(';', sindex);
   replStr = intTest.substring(sindex, eindex + 1);
-  intTest = intTest.replace(replStr, `const protocol = '${protocol}';`);
+  intTest = intTest.replace(replStr, `samProps.protocol = '${protocol}';`);
 
   // replace port variable
-  sindex = intTest.indexOf('const port');
+  sindex = intTest.indexOf('samProps.port');
   eindex = intTest.indexOf(';', sindex);
   replStr = intTest.substring(sindex, eindex + 1);
-  intTest = intTest.replace(replStr, `const port = ${port};`);
+  intTest = intTest.replace(replStr, `samProps.port = ${port};`);
 
   // replace sslenable variable
-  sindex = intTest.indexOf('const sslenable');
+  sindex = intTest.indexOf('samProps.ssl.enabled');
   eindex = intTest.indexOf(';', sindex);
   replStr = intTest.substring(sindex, eindex + 1);
-  intTest = intTest.replace(replStr, `const sslenable = ${sslenable};`);
+  intTest = intTest.replace(replStr, `samProps.ssl.enabled = ${sslenable};`);
 
   // replace sslinvalid variable
-  sindex = intTest.indexOf('const sslinvalid');
+  sindex = intTest.indexOf('samProps.ssl.accept_invalid_cert');
   eindex = intTest.indexOf(';', sindex);
   replStr = intTest.substring(sindex, eindex + 1);
-  intTest = intTest.replace(replStr, `const sslinvalid = ${sslinvalid};`);
+  intTest = intTest.replace(replStr, `samProps.ssl.accept_invalid_cert = ${sslinvalid};`);
 
   console.log(`Updates to ${test} complete`);
   fs.writeFileSync(test, intTest);

package/workflows/kafka_test.json

@@ -1 +0,0 @@
-{"name":"kafka_test","tasks":{"3678":{"name":"resumeConsumer","summary":"Resume the consumer","description":"Resume the consumer","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"adapter_id":"Kafka"},"outgoing":{"result":null}},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.6906740535549399,"y":0.1601423487544484},"6409":{"name":"pauseTopics","summary":"PauseTopics the consumer","description":"PauseTopics the consumer","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"topics":["test6"],"adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.16528162511542013,"y":0.12455516014234876,"scheduled":false},"9815":{"name":"resumeTopics","summary":"ResumeTopics the consumer","description":"ResumeTopics the consumer","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"topics":["test6"],"adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.407202216066482,"y":0.3718861209964413,"scheduled":false},"workflow_start":{"name":"workflow_start","groups":[],"x":0,"y":0.5},"workflow_end":{"name":"workflow_end","groups":[],"x":1,"y":0.5},"e07":{"name":"createTopics","summary":"create topics on the Kafka server","description":"create topics on the Kafka server","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"topics":[{"topic":"test6","partitions":1,"replicationFactor":1}],"adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.15881809787626963,"y":0.5889679715302492,"scheduled":false},"c5dc":{"name":"subscribe","summary":"subscribe topics to current consumer","description":"subscribe topics to current consumer","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"topics":["test6"],"event":"event01","adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.1634349030470914,"y":0.3665480427046263,"scheduled":false},"31e1":{"name":"sendMessage","summary":"send message","description":"send message","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"payloads":[{"topic":"test6","messages":"hello world66","partitions":1}],"adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.3961218836565097,"y":0.11743772241992882,"scheduled":false},"ef3c":{"name":"unsubscribe","summary":"unsubscribe topics from current consumer","description":"remove topics from current consumer","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"topics":["test6"],"event":"event01","adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.6989843028624192,"y":0.6281138790035588,"scheduled":false},"593e":{"name":"pauseConsumer","summary":"Pause the consumer","description":"Pause the consumer","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.6906740535549399,"y":0.3879003558718861,"scheduled":false},"b417":{"name":"setOffset","summary":"Set offset of the given topic","description":"Set offset of the given topic","location":"Adapter","locationType":"Kafka","app":"Kafka","displayName":"Kafka","type":"automatic","variables":{"incoming":{"topic":"test6","partition":1,"offset":1,"adapter_id":"Kafka"},"outgoing":{"result":null},"error":""},"deprecated":false,"matched":[],"groups":[],"actor":"Pronghorn","x":0.41920590951061865,"y":0.6387900355871886,"scheduled":false}},"transitions":{"3678":{"workflow_end":{"type":"standard","state":"success"}},"5982":{},"6409":{"31e1":{"type":"standard","state":"success"}},"9815":{"b417":{"type":"standard","state":"success"}},"workflow_start":{"e07":{"type":"standard","state":"success"}},"workflow_end":{},"f78a":{},"e07":{"c5dc":{"type":"standard","state":"success"}},"c5dc":{"6409":{"type":"standard","state":"success"}},"31e1":{"9815":{"type":"standard","state":"success"}},"ef3c":{"593e":{"type":"standard","state":"success"}},"2e22":{},"593e":{"3678":{"type":"standard","state":"success"}},"b417":{"ef3c":{"type":"standard","state":"success"}}},"font_size":12,"created":"2019-09-30T21:33:32.271Z","created_by":{"provenance":"Local AAA","username":"admin@pronghorn","firstname":"admin","inactive":false,"email":""},"groups":[],"last_updated":"2019-10-15T18:35:53.047Z","last_updated_by":{"provenance":"Local AAA","username":"admin@pronghorn","firstname":"admin","inactive":false,"email":""},"type":"automation","description":null}