mcdev 4.1.8 → 4.1.10

package/.github/ISSUE_TEMPLATE/bug.yml

@@ -39,6 +39,8 @@ body:
           label: Version
           description: What version of our software are you running? (mcdev --version)
           options:
+              - 4.1.10
+              - 4.1.9
               - 4.1.8
               - 4.1.7
               - 4.1.6

package/README.md

@@ -51,7 +51,7 @@ Accenture Salesforce Marketing Cloud DevTools (mcdev) is a rapid deployment/roll
     - [6.2.8. createDeltaPkg](#628-createdeltapkg)
 - [7. Advanced Configuration](#7-advanced-configuration)
   - [7.1. Config Options](#71-config-options)
-  - [7.2. Metadata specific settings & options](#72-metadata-specific-settings--options)
+  - [7.2. Metadata-specific settings & options](#72-metadata-specific-settings--options)
     - [7.2.1. Retention Policy fields in Data Extensions](#721-retention-policy-fields-in-data-extensions)
     - [7.2.2. Adding/Updating Fields on existing Data Extensions](#722-addingupdating-fields-on-existing-data-extensions)
     - [7.2.3. Renaming fields of a Data Extensions](#723-renaming-fields-of-a-data-extensions)
@@ -123,7 +123,7 @@ If you experience issues installing Accenture SFMC DevTools, please check out th
 1. Install Accenture SFMC DevTools by running `npm install -g mcdev` (prefix with `sudo` on MacOS)
    - If you get an error, please see the below troubleshooting section.
 
-When completed run `mcdev --version` and it will show you which version you installed (e.g. `4.1.8`).
+When completed run `mcdev --version` and it will show you which version you installed (e.g. `4.1.10`).
 
 > **_Side note for proud nerds_:**
 >
@@ -203,7 +203,7 @@ Click on `Install All` to quickly get things ready or review the recommendations
 
 ![Recommended extension prompt](img/README.md/vscode-recommendations-2.png)
 
-The "Workspace Recommendations" were defined by Accenture SFMC DevTools. Clicking on the little cloud icon will install all at once. The "Other Recommendations" are auto-generated by VS Code and are not controlled by Accenture SFMC DevTools. You _may_ look through those as well, but they might also be completely irrelevant for you.
+The "Workspace Recommendations" were defined by Accenture SFMC DevTools. Clicking on the little cloud icon will install all at once. The "Other Recommendations" are auto-generated by VS Code and are not controlled by Accenture SFMC DevTools. You _may_ look through those as well, but they might also be completely irrelevant to you.
 
 **Node modules:**
 
@@ -277,10 +277,10 @@ _Note: Regardless of which tag or branch you install_
 **Install specific version (using a version tag on npm):**
 
 ```bash
-npm install -g mcdev@4.1.8
+npm install -g mcdev@4.1.10
 ```
 
-**Warning**: When you used the above method to install Accenture SFMC DevTools for a specific version or tag, trying to [update Accenture SFMC DevTools](#updating-mcdev) might not download the most recently published official version but instead stay on the version or branch you previously selected (in the above examples: develop, 4.1.8)!
+**Warning**: When you used the above method to install Accenture SFMC DevTools for a specific version or tag, trying to [update Accenture SFMC DevTools](#updating-mcdev) might not download the most recently published official version but instead stay on the version or branch you previously selected (in the above examples: develop, 4.1.10)!
 
 > **Note**: The version is currently _not_ updated on the developer branch until a new release is published. Hence, you will not see a change if you run `mcdev --version`.
 
@@ -288,7 +288,7 @@ npm install -g mcdev@4.1.8
 
 <a id="markdown-using-custom-clis" name="using-custom-clis"></a>
 
-Some users of Accenture SFMC DevTools prefer to use git bash or other CLIs instead of the operating system's default. Please note that some of the functionality of Accenture SFMC DevTools but also of other tools like the Node package manager (npm) do not necessarily function properly in these.
+Some users of Accenture SFMC DevTools prefer to use git bash or other CLIs instead of the operating system's default. Please note that some of the functionality of Accenture SFMC DevTools but also other tools like the Node package manager (npm) do not necessarily function properly in these.
 
 If you encounter problems, we strongly recommend first trying it in the default CLI.
 
@@ -476,9 +476,20 @@ Example URL: `https://mcg123abcysykllg-0321cbs8bbt64.auth.marketingcloudapis.com
 > You can run this command without the interactive wizard asking questions using the `--skipInteraction` (or short`--yes`/`--y`) flag. In this case, you need to provide a few values in the command:
 >
 > ```bash
-> mcdev init --y.credentialName "yourCustomCredentialName" --y.client_id "yourClientIdHere" --y.client_secret "yourClientSecretHere" --y.auth_url "https://yourTenantSubdomainHere.auth.marketingcloudapis.com/" --y.gitRemoteUrl "https://my.git.server.com/myrepo.git" --y.account_id 00000000 --y.backupBUs "yes" --y.gitPush "yes"
+> mcdev init --y.credentialName "yourCustomCredentialName" --y.client_id "yourClientIdHere" --y.client_secret "yourClientSecretHere" --y.auth_url "https://yourTenantSubdomainHere.auth.marketingcloudapis.com/" --y.gitRemoteUrl "https://my.git.server.com/myrepo.git" --y.account_id 00000000 --y.downloadBUs "true" --y.gitPush "true"
 > ```
 
+| CLI Argument         | Description                              |
+| -------------------- | ---------------------------------------- |
+| `--y.credentialName` | Credential name of your Business Unit    |
+| `--y.client_id`      | Salesforce Marketing Cloud client ID     |
+| `--y.client_secret`  | Salesforce Marketing Cloud client secret |
+| `--y.auth_url`       | Marketing cloud auth url                 |
+| `--y.gitRemoteUrl`   | URL of your git remote repo              |
+| `--y.account_id`     | MID of the parent BU                     |
+| `--y.downloadBUs`    | Download all the BUs: `true` or `false`  |
+| `--y.gitPush`        | Do the first git push: `true` or `false` |
+
 #### 6.1.2. upgrade
 
 <a id="markdown-upgrade" name="upgrade"></a>
@@ -859,7 +870,7 @@ _Alias:_ `mcdev bd`
 The `buildDefinition` command allows to prepare the deployments to one or multiple targets based on templates and [Market Configuration](#market-configuration).
 After you have created your templates via `retrieveAsTemplate` (or manually) in your `template/dataExtension/` folder run this command to create the final deployable files in your respective `retrieve/<business unit>/` folder.
 
-This allows you to double-check if you actually changed something by comparing the before and after using your favorite Git client. You then have to manually copy the files you want to deploy into the respective `deploy/` folder.
+This allows you to double-check if you changed something by comparing the before and after using your favorite Git client. You then have to manually copy the files you want to deploy into the respective `deploy/` folder.
 
 > **Note**: Before using this command, you need to configure your markets first! Check out our guide on [Market Configuration](#market-configuration) to understand how to use templating and prepare your market config.
 
@@ -1119,7 +1130,7 @@ The central config in `.mcdevrc.json` holds multiple adjustable settings:
 | metaDataTypes.documentOnRetrieve         | ['role','dataExtension']                     | automatically executes `document` for selected types                                                                        |
 | metaDataTypes.retrieve                   | _changes with each release_                  | check [Metadata Type Support](#metadata-type-support) for current list                                                      |
 
-### 7.2. Metadata specific settings & options
+### 7.2. Metadata-specific settings & options
 
 <a id="markdown-metadata-specific-settings" name="metadata-specific-settings"></a>
 
@@ -1130,7 +1141,7 @@ The way the retention policy is saved is a bit misleading and hence we wanted to
 | Field                                | Description                                                                                                                                                                                                                                                                                                                        | Values                                                                 |
 | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- | ------------ |
 | **DataRetentionPeriod**              | this field should print the value of the unit of measure but it unfortunately is off by one (e.g. showing "weeks" instead of "months"). Also, it seems to have no impact on what's stored.<br> We therefore excluded it from retrieve/deploy                                                                                       | -                                                                      |
-| **DataRetentionPeriodUnitOfMeasure** | represents drop down for "period after" selection                                                                                                                                                                                                                                                                                  | 6: years<br>5: months<br>4: weeks<br>2: days                           |
+| **DataRetentionPeriodUnitOfMeasure** | represents drop-down for "period after" selection                                                                                                                                                                                                                                                                                  | 6: years<br>5: months<br>4: weeks<br>2: days                           |
 | **DataRetentionPeriodLength**        | represents number field for "period after" selection                                                                                                                                                                                                                                                                               | min: 1<br>max: 999                                                     |
 | **RowBasedRetention**                | only true if "delete individual records" is selected, otherwise false                                                                                                                                                                                                                                                              | true / false                                                           |
 | **ResetRetentionPeriodOnImport**     | true if "Reset period on import" is checked.                                                                                                                                                                                                                                                                                       | This option is always false if "delete individual records" is selected | true / false |
@@ -1158,7 +1169,7 @@ There are a few rules to keep in mind when playing with Data Extensions fields:
 
 #### 7.2.3. Renaming fields of a Data Extensions
 
-With a small addition to the Data Extension's JSON it is possible to rename fields via MC DevTools. Imagine the following Data Extension:
+With a small addition to the Data Extension's JSON, it is possible to rename fields via MC DevTools. Imagine the following Data Extension:
 
 ```json
 {
@@ -1273,7 +1284,7 @@ Here is a simple example with one DEV BU, 1 QA BU and 2 PROD BUs:
 }
 ```
 
-Way more complex example with dedicated "Parent" BUs per environment (DEV, QA, PROD) and multiple country-specific BUs for QA and for PROD:
+Way more complex example with dedicated "Parent" BUs per environment (DEV, QA, PROD) and multiple country-specific BUs for QA and PROD:
 
 ```json
 // example market config in your .mcdevrc.json
@@ -1494,7 +1505,7 @@ Assuming you cloned Accenture SFMC DevTools into `C:\repos\sfmc-devtools\` (or `
 
 This should tell npm to create a symlink to your cloned local directory, allowing you to see updates you make in your mcdev repo instantly.
 
-To test your new **global** developer setup, run `mcdev --version` in CLI which should return the current version (e.g. `4.1.8`). Then, go into your mcdev repo and update the version with the suffix `-dev`, e.g. to `4.1.8-dev` and then run `mcdev --version` again to verify that your change propagates instantly.
+To test your new **global** developer setup, run `mcdev --version` in CLI which should return the current version (e.g. `4.1.10`). Then, go into your mcdev repo and update the version with the suffix `-dev`, e.g. to `4.1.10-dev` and then run `mcdev --version` again to verify that your change propagates instantly.
 
 > **Not recommended:** Alternatively, you can install it locally only by opening a terminal in your project directory and executing `npm install --save-dev "C:\repos\sfmc-devtools"`
 > To run the local version you need to prepend "npx" before your commands, e.g. `npx mcdev --version`
@@ -1532,7 +1543,7 @@ The following explains how you _could_ install it locally for certain edge cases
 4. Afterwards, install Accenture SFMC DevTools by running `npm install --save-dev mcdev`
    - If you get an error, please see the below troubleshooting section.
 
-When completed run `mcdev --version` and it will show you which version you installed (e.g. `4.1.8`).
+When completed run `mcdev --version` and it will show you which version you installed (e.g. `4.1.10`).
 
 ### 9.3. NPM Scripts
 

package/boilerplate/gitignore-template

@@ -1,9 +1,9 @@
 *.DS_Store
-deploy/
-docs/badKeys/
-docs/deltaPackage/
-logs/
 node_modules/
-template/
+/deploy/
+/docs/badKeys/
+/docs/deltaPackage/
+/logs/
+/template/
 **/QueryStudioResults at*
 .mcdev-auth.json

package/docs/dist/documentation.md

@@ -4440,6 +4440,7 @@ Retrieves metadata from a business unit and saves it to the local filesystem.
 * [Retriever](#Retriever)
     * [new Retriever(properties, buObject)](#new_Retriever_new)
     * [.retrieve(metadataTypes, [namesOrKeys], [templateVariables], [changelogOnly])](#Retriever+retrieve) ⇒ <code>Promise.&lt;TYPE.MultiMetadataTypeList&gt;</code>
+    * [._getTypeDependencies(metadataTypes)](#Retriever+_getTypeDependencies) ⇒ <code>Array.&lt;TYPE.SupportedMetadataTypes&gt;</code>
 
 <a name="new_Retriever_new"></a>
 
@@ -4467,6 +4468,18 @@ Retrieve metadata of specified types into local file system and Retriever.metada
 | [templateVariables] | <code>TYPE.TemplateMap</code> | Object of values which can be replaced (in case of templating) |
 | [changelogOnly] | <code>boolean</code> | skip saving, only create json in memory |
 
+<a name="Retriever+_getTypeDependencies"></a>
+
+### retriever.\_getTypeDependencies(metadataTypes) ⇒ <code>Array.&lt;TYPE.SupportedMetadataTypes&gt;</code>
+helper for [retrieve](retrieve) to get all dependencies of the given types
+
+**Kind**: instance method of [<code>Retriever</code>](#Retriever)  
+**Returns**: <code>Array.&lt;TYPE.SupportedMetadataTypes&gt;</code> - unique list dependent metadata types  
+
+| Param | Type | Description |
+| --- | --- | --- |
+| metadataTypes | <code>Array.&lt;TYPE.SupportedMetadataTypes&gt;</code> | list of metadata types to retrieve; can include subtypes! |
+
 <a name="Util"></a>
 
 ## Util

package/lib/Retriever.js

@@ -2,6 +2,7 @@
 
 const TYPE = require('../types/mcdev.d');
 const MetadataTypeInfo = require('./MetadataTypeInfo');
+const MetadataDefinitions = require('./MetadataTypeDefinitions');
 const Util = require('./util/util');
 const File = require('./util/file');
 const cache = require('./util/cache');
@@ -63,6 +64,9 @@ class Retriever {
             reset: '\x1B[0m',
             dim: '\x1B[2m',
         };
+        // ensure we know which real dependencies we have to ensure we cache those completely
+        const dependencies = this._getTypeDependencies(metadataTypes);
+
         for (const metadataType of Util.getMetadataHierachy(metadataTypes)) {
             const [type, subType] = metadataType.split('-');
             // if types were added by getMetadataHierachy() for caching, make sure the key-list is set to [null] for them which will retrieve all
@@ -94,6 +98,18 @@ class Retriever {
                         )
                     );
                 } else {
+                    let cacheResult = null;
+                    if (
+                        Object.keys(typeKeyMap[metadataType])[0] !== null &&
+                        (dependencies.includes(type) || dependencies.includes(metadataType))
+                    ) {
+                        // if we have a key-list and the type is a dependency, we need to cache the whole type
+                        Util.logger.info(`Caching dependent Metadata: ${metadataType}`);
+                        cacheResult = await MetadataTypeInfo[type].retrieveForCache(
+                            this.buObject,
+                            subType
+                        );
+                    }
                     Util.logger.info(
                         `Retrieving: ${metadataType}` +
                             (typeKeyMap[metadataType][0] !== null
@@ -115,6 +131,10 @@ class Retriever {
                                   )
                               )
                           ));
+                    if (Array.isArray(result) && cacheResult !== null) {
+                        // if we have a key-list and the type is a dependency, we need to cache the whole type
+                        result = [cacheResult, ...result];
+                    }
                     if (changelogOnly) {
                         // add folder to changelog
                         for (const key of Object.keys(result.metadata)) {
@@ -124,10 +144,11 @@ class Retriever {
                 }
                 if (result) {
                     if (Array.isArray(result)) {
-                        cache.setMetadata(
-                            type,
-                            result.filter((el) => !!el).map((element) => element.metadata)[0]
-                        );
+                        for (const result_i of result) {
+                            if (result_i?.metadata && Object.keys(result_i.metadata).length) {
+                                cache.mergeMetadata(type, result_i.metadata);
+                            }
+                        }
                         if (metadataTypes.includes(type) || metadataTypes.includes(metadataType)) {
                             retrieveChangelog[type] = result
                                 .filter((el) => !!el)
@@ -148,6 +169,27 @@ class Retriever {
         }
         return retrieveChangelog;
     }
+
+    /**
+     * helper for {@link retrieve} to get all dependencies of the given types
+     *
+     * @param {TYPE.SupportedMetadataTypes[]} metadataTypes list of metadata types to retrieve; can include subtypes!
+     * @returns {TYPE.SupportedMetadataTypes[]} unique list dependent metadata types
+     */
+    _getTypeDependencies(metadataTypes) {
+        let dependencies = [];
+        for (const metadataType of metadataTypes) {
+            const type = metadataType.split('-')[0];
+            // if they have dependencies then add a dependency pair for each type
+            if (MetadataDefinitions[type].dependencies.length > 0) {
+                dependencies.push(
+                    ...MetadataDefinitions[type].dependencies.map((dep) => dep.split('-')[0])
+                );
+            }
+        }
+        dependencies = [...new Set(dependencies)];
+        return dependencies;
+    }
 }
 
 module.exports = Retriever;

package/lib/metadataTypes/MetadataType.js

@@ -770,6 +770,11 @@ class MetadataType {
         }
 
         if (retrieveDir) {
+            // defined colors for optionally printing the keys we filtered by
+            const color = {
+                reset: '\x1B[0m',
+                dim: '\x1B[2m',
+            };
             const savedMetadata = await this.saveResults(
                 results,
                 retrieveDir,
@@ -779,7 +784,10 @@ class MetadataType {
             Util.logger.info(
                 `Downloaded: ${overrideType || this.definition.type} (${
                     Object.keys(savedMetadata).length
-                })`
+                })` +
+                    (singleRetrieve !== null
+                        ? ` ${color.dim}(Key: ${singleRetrieve})${color.reset}`
+                        : '')
             );
         }
 
@@ -800,7 +808,6 @@ class MetadataType {
         const bodyIteratorField = this.definition.bodyIteratorField;
         const keyField = this.definition.keyField;
         const metadataStructure = {};
-
         if (body !== null) {
             if (Array.isArray(body)) {
                 // in some cases data is just an array
@@ -816,7 +823,6 @@ class MetadataType {
             } else if (singleRetrieve) {
                 // some types will return a single item intead of an array if the key is supported by their api
                 metadataStructure[singleRetrieve] = body;
-
                 return metadataStructure;
             }
             if (
@@ -827,6 +833,8 @@ class MetadataType {
                 const single = {};
                 single[singleRetrieve] = metadataStructure[singleRetrieve];
                 return single;
+            } else if (singleRetrieve) {
+                return {};
             }
         }
         return metadataStructure;

package/lib/metadataTypes/definitions/Asset.definition.js

@@ -460,7 +460,7 @@ module.exports = {
     ],
     binarySubtypes: ['archive', 'audio', 'document', 'image', 'rawimage', 'video'],
     extendedSubTypes: {
-        asset: ['webpage', 'webtemplate', 'jsonmessage', 'icemailformblock'],
+        asset: ['webpage', 'webtemplate', 'jsonmessage', 'icemailformblock', 'asset'],
         image: [
             'ai',
             'psd',
@@ -486,6 +486,7 @@ module.exports = {
             'pbm',
             'ppm',
             'svg',
+            'image',
         ],
         rawimage: [
             '3fr',
@@ -523,6 +524,7 @@ module.exports = {
             'sr2',
             'srw',
             'x3f',
+            'rawimage',
         ],
         video: [
             '3gp',
@@ -551,6 +553,7 @@ module.exports = {
             'wmv',
             'rm',
             'ogv',
+            'video',
         ],
         document: [
             'indd',
@@ -596,6 +599,7 @@ module.exports = {
             'key',
             'keynote',
             'pez',
+            'document',
         ],
         audio: [
             'aac',
@@ -613,6 +617,7 @@ module.exports = {
             'ra',
             'vox',
             'voc',
+            'audio',
         ],
         archive: [
             '7z',
@@ -630,6 +635,7 @@ module.exports = {
             'tar',
             'zip',
             'gpg',
+            'archive',
         ],
         code: [
             'htm',
@@ -651,8 +657,9 @@ module.exports = {
             'as',
             'fla',
             'eml',
+            'code',
         ],
-        textfile: ['text', 'txt'],
+        textfile: ['text', 'txt', 'textfile'],
         block: [
             'freeformblock',
             'textblock',
@@ -667,9 +674,10 @@ module.exports = {
             'socialfollowblock',
             'buttonblock',
             'layoutblock',
+            'block',
         ],
-        message: ['templatebasedemail', 'htmlemail', 'textonlyemail'],
-        template: ['defaulttemplate'],
+        template: ['defaulttemplate', 'template'],
+        message: ['templatebasedemail', 'htmlemail', 'textonlyemail', 'message'],
         other: [
             'smartcaptureblock',
             'smartcaptureformfieldblock',
@@ -688,6 +696,21 @@ module.exports = {
         ],
     },
     typeMapping: {
+        asset: 1,
+        file: 2, // not inherited on Asset Types
+        block: 3,
+        template: 4,
+        message: 5,
+        custom: 6, // not inherited on Asset Types
+        default: 7, // not inherited on Asset Types
+        image: 8,
+        rawimage: 9,
+        video: 10,
+        document: 11,
+        audio: 12,
+        archive: 13,
+        code: 14,
+        textfile: 15,
         ai: 16,
         psd: 17,
         pdd: 18,

package/lib/util/init.git.js

@@ -110,7 +110,7 @@ const Init = {
                         },
                     ]);
                 }
-                if (skipInteraction.gitPush === 'yes' || responses.gitPush) {
+                if (skipInteraction.gitPush === 'true' || responses?.gitPush) {
                     Util.execSync('git', ['push', '-u', 'origin', 'master']);
                 }
             } else if (remoteBranchesExist === true) {

package/lib/util/init.js

@@ -202,7 +202,7 @@ const Init = {
                 },
             ]);
         }
-        if (skipInteraction.backupBUs === 'yes' || responses.initialRetrieveAll) {
+        if (skipInteraction.downloadBUs === 'true' || responses?.initialRetrieveAll) {
             Util.execSync('mcdev', ['retrieve', bu]);
 
             if (gitStatus === 'init') {

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "mcdev",
-    "version": "4.1.8",
+    "version": "4.1.10",
     "description": "Accenture Salesforce Marketing Cloud DevTools",
     "author": "Accenture: joern.berkefeld, douglas.midgley, robert.zimmermann, maciej.barnas",
     "license": "MIT",

package/test/mockRoot/.mcdevrc.json

@@ -63,5 +63,5 @@
             "triggeredSendDefinition"
         ]
     },
-    "version": "4.1.8"
+    "version": "4.1.10"
 }