@transcend-io/cli 4.26.3 → 4.29.0

package/README.md

@@ -224,26 +224,36 @@ This command can be helpful if you are looking to:
 
 In order to use this cli, you will first need to generate an API key on the Transcend Admin Dashboard (https://app.transcend.io/infrastructure/api-keys).
 
-The API key needs the following scopes:
-
-- View Data Map
-- View Identity Verification Settings
-- View Data Subject Request Settings
-- View API Keys
-- View Email Templates
+The API key permissions for this command vary based on the value to the `resources` argument. See the table below to understand the necessary permissions for the resources you are attempting to pull.
+
+| Key              | Description                                                                                                                          | Scope                                            | Is Default | Link                                                                                                                                                                                                  |
+| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------ | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| apiKeys          | API Key definitions assigned to Data Silos. API keys cannot be created through the cli, but you can map API key usage to Data Silos. | View API Keys                                    | true       | [Infrastructure -> API keys](https://app.transcend.io/infrastructure/api-keys)                                                                                                                        |
+| templates        | Email templates. Only template titles can be created and mapped to other resources.                                                  | View Email Templates                             | true       | [Privacy Requests -> Email Templates](https://app.transcend.io/privacy-requests/email-templates)                                                                                                      |
+| dataSilos        | The Data Silo/Integration definitions.                                                                                               | View Data Map,View Data Subject Request Settings | true       | [Data Mapping -> Data Inventory -> Data Silos](https://app.transcend.io/data-map/data-inventory/) and [Infrastucture -> Integrations](https://app.transcend.io/infrastructure/integrationsdata-silos) |
+| enrichers        | The Privacy Request enricher configurations.                                                                                         | View Identity Verification Settings              | true       | [Privacy Requests -> Identifiers](https://app.transcend.io/privacy-requests/identifiers)                                                                                                              |
+| businessEntities | The business entities in the data inventory.                                                                                         | View Data Map                                    | false      | [Data Mapping -> Data Inventory -> Business Entities](https://app.transcend.io/data-map/data-inventory/business-entities)                                                                             |
+| identifiers      | The Privacy Request identifier configurations.                                                                                       | View Identity Verification Settings              | false      | [Privacy Requests -> Identifiers](https://app.transcend.io/privacy-requests/identifiers)                                                                                                              |
+| actions          | The Privacy Request action settings.                                                                                                 | View Data Subject Request Settings               | false      | [Privacy Requests -> Request Settings](https://app.transcend.io/privacy-requests/settings)                                                                                                            |
+| dataSubjects     | The Privacy Request data subject settings.                                                                                           | View Data Subject Request Settings               | false      | [Privacy Requests -> Request Settings](https://app.transcend.io/privacy-requests/settings)                                                                                                            |
+| attributes       | Attribute definitions that define extra metadata for each table in the Admin Dashboard.                                              | View Global Attributes                           | false      | [Infrastructure -> Attributes](https://app.transcend.io/infrastructure/attributes)                                                                                                                    |
+| dataFlows        | Consent Manager Data Flow definitions.                                                                                               | View Data Map Data Flows                         | false      | [Consent Manager -> Data Flows](https://app.transcend.io/consent-manager/data-flows/approved)                                                                                                         |
+| cookies          | Consent Manager Cookie definitions.                                                                                                  | View Data Flows                                  | false      | [Consent Manager -> Cookies](https://app.transcend.io/consent-manager/cookies/approved)                                                                                                               |
+| consentManager   | Consent Manager general settings, including domain list.                                                                             | View Consent Manager                             | false      | [Consent Manager -> Developer Settings](https://app.transcend.io/consent-manager/developer-settings)                                                                                                  |
 
 _Note: The scopes for tr-push are comprehensive of the scopes for tr-pull_
 
 #### Arguments
 
-| Argument     | Description                                                                   | Type                | Default                  | Required |
-| ------------ | ----------------------------------------------------------------------------- | ------------------- | ------------------------ | -------- |
-| auth         | The Transcend API capable of fetching the configuration                       | string              | N/A                      | true     |
-| file         | Path to the YAML file to pull into                                            | string - file-path  | ./transcend.yml          | false    |
-| transcendUrl | URL of the Transcend backend. Use https://api.us.transcend.io for US hosting. | string - URL        | https://api.transcend.io | false    |
-| dataSiloIds  | The UUIDs of the data silos that should be pulled into the YAML file.         | list(string - UUID) | N/A                      | false    |
-| pageSize     | The page size to use when paginating over the API                             | number              | 50                       | false    |
-| debug        | Set to true to include debug logs while pulling the configuration             | boolean             | false                    | false    |
+| Argument     | Description                                                                              | Type                | Default                               | Required |
+| ------------ | ---------------------------------------------------------------------------------------- | ------------------- | ------------------------------------- | -------- |
+| auth         | The Transcend API capable of fetching the configuration                                  | string              | N/A                                   | true     |
+| resources    | The different resource types to pull in (see table above for breakdown of each resource) | string              | apiKeys,templates,dataSilos,enrichers | false    |
+| file         | Path to the YAML file to pull into                                                       | string - file-path  | ./transcend.yml                       | false    |
+| transcendUrl | URL of the Transcend backend. Use https://api.us.transcend.io for US hosting.            | string - URL        | https://api.transcend.io              | false    |
+| dataSiloIds  | The UUIDs of the data silos that should be pulled into the YAML file.                    | list(string - UUID) | N/A                                   | false    |
+| pageSize     | The page size to use when paginating over the API                                        | number              | 50                                    | false    |
+| debug        | Set to true to include debug logs while pulling the configuration                        | boolean             | false                                 | false    |
 
 #### Usage
 
@@ -271,6 +281,30 @@ Or a specific types of data silo(s) can be pulled in:
 tr-pull --auth=$TRANSCEND_API_KEY --integrationNames=salesforce,snowflake
 ```
 
+Specifying the resource types to pull in (the following resources are the default resources):
+
+```sh
+tr-pull --auth=$TRANSCEND_API_KEY --resources=apiKeys,templates,dataSilos,enrichers
+```
+
+Pull in data flow and cookie resources instead (see [this example](./examples/data-flows-cookies.yml)):
+
+```sh
+tr-pull --auth=$TRANSCEND_API_KEY --resources=dataFlows,cookies
+```
+
+Pull in attribute definitions only (see [this example](./examples/attributes.yml)):
+
+```sh
+tr-pull --auth=$TRANSCEND_API_KEY --resources=attributes
+```
+
+Pull everything:
+
+```sh
+tr-pull --auth=$TRANSCEND_API_KEY --resources=all
+```
+
 Or with a specific page size (max 100)
 
 ```sh
@@ -293,14 +327,22 @@ Given a transcend.yml file, sync the contents up to your connected services view
 
 In order to use this cli, you will first need to generate an API key on the Transcend Admin Dashboard (https://app.transcend.io/infrastructure/api-keys).
 
-The API key needs the following scopes:
-
-- Manage Data Map
-- Manage Request Identity Verification
-- Connect Data Silos
-- Manage Data Subject Request Settings
-- View API Keys
-- Manage Email Templates
+The API key needs the following scopes when pushing the various resource types:
+
+| Key              | Description                                                                                                   | Scope                                 | Is Default | Link                                                                                                                                                                                                  |
+| ---------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| apiKeys          | API Key definitions. API keys cannot be created through the cli, but you can map API key usage to Data Silos. | View API Keys                         | true       | [Infrastructure -> API keys](https://app.transcend.io/infrastructure/api-keys)                                                                                                                        |
+| templates        | Email templates. Only template titles can be created and mapped to other resources.                           | Manage Email Templates                | true       | [Privacy Requests -> Email Templates](https://app.transcend.io/privacy-requests/email-templates)                                                                                                      |
+| dataSilos        | The Data Silo/Integration definitions.                                                                        | Manage Data Map,Connect Data Silos    | true       | [Data Mapping -> Data Inventory -> Data Silos](https://app.transcend.io/data-map/data-inventory/) and [Infrastucture -> Integrations](https://app.transcend.io/infrastructure/integrationsdata-silos) |
+| enrichers        | The Privacy Request enricher configurations.                                                                  | Manage Request Identity Verification  | true       | [Privacy Requests -> Identifiers](https://app.transcend.io/privacy-requests/identifiers)                                                                                                              |
+| businessEntities | The business entities in the data inventory.                                                                  | Manage Data Map                       | false      | [Data Mapping -> Data Inventory -> Business Entities](https://app.transcend.io/data-map/data-inventory/business-entities)                                                                             |
+| identifiers      | The Privacy Request identifier configurations.                                                                | Manage Identity Verification Settings | false      | [Privacy Requests -> Identifiers](https://app.transcend.io/privacy-requests/identifiers)                                                                                                              |
+| actions          | The Privacy Request action settings.                                                                          | Manage Data Subject Request Settings  | false      | [Privacy Requests -> Request Settings](https://app.transcend.io/privacy-requests/settings)                                                                                                            |
+| dataSubjects     | The Privacy Request data subject settings.                                                                    | Manage Data Subject Request Settings  | false      | [Privacy Requests -> Request Settings](https://app.transcend.io/privacy-requests/settings)                                                                                                            |
+| attributes       | Attribute definitions that define extra metadata for each table in the Admin Dashboard.                       | Manage Global Attributes              | false      | [Infrastructure -> Attributes](https://app.transcend.io/infrastructure/attributes)                                                                                                                    |
+| dataFlows        | Consent Manager Data Flow definitions.                                                                        | Manage Data Flows                     | false      | [Consent Manager -> Data Flows](https://app.transcend.io/consent-manager/data-flows/approved)                                                                                                         |
+| cookies          | Consent Manager Cookie definitions.                                                                           | Manage Data Flows                     | false      | [Consent Manager -> Cookies](https://app.transcend.io/consent-manager/cookies/approved)                                                                                                               |
+| consentManager   | Consent Manager general settings, including domain list.                                                      | Deploy Consent Manager                | false      | [Consent Manager -> Developer Settings](https://app.transcend.io/consent-manager/developer-settings)                                                                                                  |
 
 #### Arguments
 
@@ -356,7 +398,7 @@ jobs:
   deploy:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
 
       - name: Setup Node.js
         uses: actions/setup-node@v2

package/build/cli-pull.js

@@ -10,6 +10,7 @@ const colors_1 = __importDefault(require("colors"));
 const graphql_1 = require("./graphql");
 const readTranscendYaml_1 = require("./readTranscendYaml");
 const constants_1 = require("./constants");
+const VALID_RESOURCES = Object.values(graphql_1.TranscendPullResource);
 /**
  * Sync data silo configuration from Transcend down locally to disk
  *
@@ -21,7 +22,7 @@ const constants_1 = require("./constants");
  */
 async function main() {
     // Parse command line arguments
-    const { file = './transcend.yml', transcendUrl = 'https://api.transcend.io', dataSiloIds = '', integrationNames = '', pageSize = '', debug = '', auth, } = (0, yargs_parser_1.default)(process.argv.slice(2));
+    const { file = './transcend.yml', transcendUrl = 'https://api.transcend.io', dataSiloIds = '', integrationNames = '', resources = graphql_1.DEFAULT_TRANSCEND_PULL_RESOURCES.join(','), pageSize = '', debug = '', auth, } = (0, yargs_parser_1.default)(process.argv.slice(2));
     // Ensure auth is passed
     if (!auth) {
         logger_1.logger.error(colors_1.default.red('A Transcend API key must be provided. You can specify using --auth=asd123'));
@@ -29,6 +30,21 @@ async function main() {
     }
     // Create a GraphQL client
     const client = (0, graphql_1.buildTranscendGraphQLClient)(transcendUrl, auth);
+    // Parse request actions
+    let splitResources = [];
+    if (!resources) {
+        logger_1.logger.error(colors_1.default.red(`Missing required parameter "resources". e.g. --resources=${VALID_RESOURCES.join(',')}`));
+        process.exit(1);
+    }
+    splitResources =
+        resources === 'all'
+            ? VALID_RESOURCES
+            : resources.split(',');
+    const invalidResources = splitResources.filter((resource) => !VALID_RESOURCES.includes(resource));
+    if (invalidResources.length > 0) {
+        logger_1.logger.error(colors_1.default.red(`Received invalid resources values: "${invalidResources.join(',')}". Allowed: ${VALID_RESOURCES.join(',')}`));
+        process.exit(1);
+    }
     // Sync to Disk
     try {
         const configuration = await (0, graphql_1.pullTranscendConfiguration)(client, {
@@ -36,6 +52,7 @@ async function main() {
             integrationNames: integrationNames
                 .split(',')
                 .filter((x) => !!x),
+            resources: splitResources,
             pageSize: pageSize ? parseInt(pageSize, 10) : 50,
             debug: debug === 'true',
         });

package/build/cli-pull.js.map

@@ -1 +1 @@
-{"version":3,"file":"cli-pull.js","sourceRoot":"","sources":["../src/cli-pull.ts"],"names":[],"mappings":";;;;;;AAEA,gEAAiC;AACjC,qCAAkC;AAClC,oDAA4B;AAC5B,uCAGmB;AACnB,2DAAyD;AACzD,2CAAsD;AAEtD;;;;;;;;GAQG;AACH,KAAK,UAAU,IAAI;IACjB,+BAA+B;IAC/B,MAAM,EACJ,IAAI,GAAG,iBAAiB,EACxB,YAAY,GAAG,0BAA0B,EACzC,WAAW,GAAG,EAAE,EAChB,gBAAgB,GAAG,EAAE,EACrB,QAAQ,GAAG,EAAE,EACb,KAAK,GAAG,EAAE,EACV,IAAI,GACL,GAAG,IAAA,sBAAK,EAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAEjC,wBAAwB;IACxB,IAAI,CAAC,IAAI,EAAE;QACT,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,2EAA2E,CAC5E,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,0BAA0B;IAC1B,MAAM,MAAM,GAAG,IAAA,qCAA2B,EAAC,YAAY,EAAE,IAAI,CAAC,CAAC;IAE/D,eAAe;IACf,IAAI;QACF,MAAM,aAAa,GAAG,MAAM,IAAA,oCAA0B,EAAC,MAAM,EAAE;YAC7D,WAAW,EAAG,WAAsB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YAClE,gBAAgB,EAAG,gBAA2B;iBAC3C,KAAK,CAAC,GAAG,CAAC;iBACV,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YACrB,QAAQ,EAAE,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE;YAChD,KAAK,EAAE,KAAK,KAAK,MAAM;SACxB,CAAC,CAAC;QAEH,eAAM,CAAC,IAAI,CAAC,gBAAM,CAAC,OAAO,CAAC,kCAAkC,IAAI,MAAM,CAAC,CAAC,CAAC;QAC1E,IAAA,sCAAkB,EAAC,IAAI,EAAE,aAAa,CAAC,CAAC;KACzC;IAAC,OAAO,GAAG,EAAE;QACZ,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CAAC,yCAAyC,GAAG,CAAC,OAAO,EAAE,CAAC,CACnE,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,mBAAmB;IACnB,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,KAAK,CACV,4CAA4C,IAAI,aAAa,mCAAuB,EAAE,CACvF,CACF,CAAC;AACJ,CAAC;AAED,IAAI,EAAE,CAAC"}
+{"version":3,"file":"cli-pull.js","sourceRoot":"","sources":["../src/cli-pull.ts"],"names":[],"mappings":";;;;;;AAEA,gEAAiC;AACjC,qCAAkC;AAClC,oDAA4B;AAC5B,uCAKmB;AACnB,2DAAyD;AACzD,2CAAsD;AAEtD,MAAM,eAAe,GAAG,MAAM,CAAC,MAAM,CAAC,+BAAqB,CAAC,CAAC;AAE7D;;;;;;;;GAQG;AACH,KAAK,UAAU,IAAI;IACjB,+BAA+B;IAC/B,MAAM,EACJ,IAAI,GAAG,iBAAiB,EACxB,YAAY,GAAG,0BAA0B,EACzC,WAAW,GAAG,EAAE,EAChB,gBAAgB,GAAG,EAAE,EACrB,SAAS,GAAG,0CAAgC,CAAC,IAAI,CAAC,GAAG,CAAC,EACtD,QAAQ,GAAG,EAAE,EACb,KAAK,GAAG,EAAE,EACV,IAAI,GACL,GAAG,IAAA,sBAAK,EAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAEjC,wBAAwB;IACxB,IAAI,CAAC,IAAI,EAAE;QACT,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,2EAA2E,CAC5E,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,0BAA0B;IAC1B,MAAM,MAAM,GAAG,IAAA,qCAA2B,EAAC,YAAY,EAAE,IAAI,CAAC,CAAC;IAE/D,wBAAwB;IACxB,IAAI,cAAc,GAA4B,EAAE,CAAC;IACjD,IAAI,CAAC,SAAS,EAAE;QACd,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,4DAA4D,eAAe,CAAC,IAAI,CAC9E,GAAG,CACJ,EAAE,CACJ,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IACD,cAAc;QACZ,SAAS,KAAK,KAAK;YACjB,CAAC,CAAC,eAAe;YACjB,CAAC,CAAE,SAAS,CAAC,KAAK,CAAC,GAAG,CAA6B,CAAC;IACxD,MAAM,gBAAgB,GAAG,cAAc,CAAC,MAAM,CAC5C,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,eAAe,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAClD,CAAC;IACF,IAAI,gBAAgB,CAAC,MAAM,GAAG,CAAC,EAAE;QAC/B,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CACR,uCAAuC,gBAAgB,CAAC,IAAI,CAC1D,GAAG,CACJ,eAAe,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAC5C,CACF,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,eAAe;IACf,IAAI;QACF,MAAM,aAAa,GAAG,MAAM,IAAA,oCAA0B,EAAC,MAAM,EAAE;YAC7D,WAAW,EAAG,WAAsB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YAClE,gBAAgB,EAAG,gBAA2B;iBAC3C,KAAK,CAAC,GAAG,CAAC;iBACV,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YACrB,SAAS,EAAE,cAAc;YACzB,QAAQ,EAAE,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE;YAChD,KAAK,EAAE,KAAK,KAAK,MAAM;SACxB,CAAC,CAAC;QAEH,eAAM,CAAC,IAAI,CAAC,gBAAM,CAAC,OAAO,CAAC,kCAAkC,IAAI,MAAM,CAAC,CAAC,CAAC;QAC1E,IAAA,sCAAkB,EAAC,IAAI,EAAE,aAAa,CAAC,CAAC;KACzC;IAAC,OAAO,GAAG,EAAE;QACZ,eAAM,CAAC,KAAK,CACV,gBAAM,CAAC,GAAG,CAAC,yCAAyC,GAAG,CAAC,OAAO,EAAE,CAAC,CACnE,CAAC;QACF,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;KACjB;IAED,mBAAmB;IACnB,eAAM,CAAC,IAAI,CACT,gBAAM,CAAC,KAAK,CACV,4CAA4C,IAAI,aAAa,mCAAuB,EAAE,CACvF,CACF,CAAC;AACJ,CAAC;AAED,IAAI,EAAE,CAAC"}