cumulusci 3.52.0.dev0__tar.gz → 3.53.0__tar.gz

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/HISTORY.rst

@@ -2,7 +2,48 @@
 History
 =======
 
+3.53.0 (2022-02-24)
+-------------------
+
+Critical Changes
+
+- CumulusCI now ships with Snowfakery 3.0. See the `Snowfakery release notes <https://github.com/SFDO-Tooling/Snowfakery/releases/tag/v3.0.0>`__. Snowfakery 3.0 ships with opt-in features that will become the default in June 2022; please `test your recipes <https://snowfakery.readthedocs.io/en/latest/#snowfakery-3>`__  now with ``snowfakery_version: 3`` to ensure compatibility.
+
+Changes
+
+- CumulusCI now has an ``update_data`` task that allows updates to existing datasets, in coordination with Snowfakery 3.0.
+- Orgs connected using ``cci org connect`` now keep track of which ``connected_app`` service was used to connect them, and will use the same ``connected_app`` service when getting a fresh access token, even if it isn't currently the default ``connected_app`` service. This makes it easier to use CumulusCI with multiple ``connected_app`` services. (Orgs that were connected before this update will continue to use the default ``connected_app`` service.)
+- When connecting a custom ``connected_app`` service, a default ``login_url`` can be specified (to use a login URL other than https://login.salesforce.com when connecting orgs using this ``connected_app`` service).
+- The ``deploy_marketing_cloud_package`` task now utilizes the new API endpoint for deployments.
+- CumulusCI commands are more resilient in the face of corrupted or impossible to decode org config files.
+- We added documentation about managing Robot Framework locators
+- CumulusCI will now warn Windows users if long pathname support is not set up correctly.
+
+Issues Closed
+
+- Fixed a bug in the JSON format output of cci service list --json
+
+3.52.0 (2022-02-03)
+-------------------
+
+Changes
+
+* Flow steps can now be replaced with the same syntax for all step types! (Current step types are: 'task' or 'flow'). See the [replacing a flow step](https://cumulusci.readthedocs.io/en/latest/config.html?highlight=override#replace-a-flow-step) docs for more details. CumulusCI is still compatible with the old syntax which required setting the current step type to ``None`` when replacing with a step of a differing type. (#3043)
+* Whenever possible, while running the ``robot_libdoc`` task libdoc generates relative pathnames when creating CSV output (#3058)
+* Added a new option ``--preview`` to the ``robot_libdoc`` task. When set to ``true`` it automatically opens a browser window to the generated documentation. (#3057)
+* The robot keyword ``Locate element by label`` has been removed from the Salesforce.py library. This wasn't designed to be a keyword but was accidentally exported as one. If you want to find an input or textarea element by its label you can use a locator of the form ``label:<text>`` (eg: ``label:First Name``) (#3048)
+* Updated to `Snowfakery 2.5.0 <https://github.com/SFDO-Tooling/Snowfakery/releases/tag/v2.5.0>`__
+* If you have Python code that is importing CumulusCI's config classes, some of them have been reorganized into modules with a snake_case name. The old CamelCase imports should still work, but the new names are preferred for consistency with other parts of the codebase.
+
+Issues Closed
+
+* Fixed an issue where overriding flow steps was not working as documented. (#3043)
+* We fixed a bug where the ``Run Task`` and ``Run Task Class`` robot keywords throw an error like ``AttributeError: module 'robot.api.logger' has no attribute 'log'`` in rare cases (#3053)
+* Fixed a bug in the `snowfakery` task which caused Unique IDs to not always be actually unique. (#3059)
+
+
 3.51.1 (2022-01-25)
+-------------------
 
 Issues Closed
 

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: cumulusci
-Version: 3.52.0.dev0
+Version: 3.53.0
 Summary: Build and release tools for Salesforce developers
 Home-page: https://github.com/SFDO-Tooling/CumulusCI
 Author: Salesforce.org
@@ -57,9 +57,7 @@ Description: =========
         Questions?
         ----------
         
-        Salesforce.org (http://salesforce.org/) customers and partners can ask in the `CumulusCI group in the Power of Us Hub <https://powerofus.force.com/s/group/0F980000000PSRHCA4/cumulusci-cci>`_.
-        
-        Others can ask in the `CumulusCI group in the Trailblazer Community <https://success.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F9300000009M9ZCAU>`_.
+        Ask in the `CumulusCI group in the Trailblazer Community <https://success.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F9300000009M9ZCAU>`_.
         
         *Please note:* CumulusCI is distributed under an `open source license <https://github.com/SFDO-Tooling/CumulusCI/blob/main/LICENSE>`_ and is not covered by the Salesforce Master Subscription Agreement.
         
@@ -68,7 +66,48 @@ Description: =========
         History
         =======
         
+        3.53.0 (2022-02-24)
+        -------------------
+        
+        Critical Changes
+        
+        - CumulusCI now ships with Snowfakery 3.0. See the `Snowfakery release notes <https://github.com/SFDO-Tooling/Snowfakery/releases/tag/v3.0.0>`__. Snowfakery 3.0 ships with opt-in features that will become the default in June 2022; please `test your recipes <https://snowfakery.readthedocs.io/en/latest/#snowfakery-3>`__  now with ``snowfakery_version: 3`` to ensure compatibility.
+        
+        Changes
+        
+        - CumulusCI now has an ``update_data`` task that allows updates to existing datasets, in coordination with Snowfakery 3.0.
+        - Orgs connected using ``cci org connect`` now keep track of which ``connected_app`` service was used to connect them, and will use the same ``connected_app`` service when getting a fresh access token, even if it isn't currently the default ``connected_app`` service. This makes it easier to use CumulusCI with multiple ``connected_app`` services. (Orgs that were connected before this update will continue to use the default ``connected_app`` service.)
+        - When connecting a custom ``connected_app`` service, a default ``login_url`` can be specified (to use a login URL other than https://login.salesforce.com when connecting orgs using this ``connected_app`` service).
+        - The ``deploy_marketing_cloud_package`` task now utilizes the new API endpoint for deployments.
+        - CumulusCI commands are more resilient in the face of corrupted or impossible to decode org config files.
+        - We added documentation about managing Robot Framework locators
+        - CumulusCI will now warn Windows users if long pathname support is not set up correctly.
+        
+        Issues Closed
+        
+        - Fixed a bug in the JSON format output of cci service list --json
+        
+        3.52.0 (2022-02-03)
+        -------------------
+        
+        Changes
+        
+        * Flow steps can now be replaced with the same syntax for all step types! (Current step types are: 'task' or 'flow'). See the [replacing a flow step](https://cumulusci.readthedocs.io/en/latest/config.html?highlight=override#replace-a-flow-step) docs for more details. CumulusCI is still compatible with the old syntax which required setting the current step type to ``None`` when replacing with a step of a differing type. (#3043)
+        * Whenever possible, while running the ``robot_libdoc`` task libdoc generates relative pathnames when creating CSV output (#3058)
+        * Added a new option ``--preview`` to the ``robot_libdoc`` task. When set to ``true`` it automatically opens a browser window to the generated documentation. (#3057)
+        * The robot keyword ``Locate element by label`` has been removed from the Salesforce.py library. This wasn't designed to be a keyword but was accidentally exported as one. If you want to find an input or textarea element by its label you can use a locator of the form ``label:<text>`` (eg: ``label:First Name``) (#3048)
+        * Updated to `Snowfakery 2.5.0 <https://github.com/SFDO-Tooling/Snowfakery/releases/tag/v2.5.0>`__
+        * If you have Python code that is importing CumulusCI's config classes, some of them have been reorganized into modules with a snake_case name. The old CamelCase imports should still work, but the new names are preferred for consistency with other parts of the codebase.
+        
+        Issues Closed
+        
+        * Fixed an issue where overriding flow steps was not working as documented. (#3043)
+        * We fixed a bug where the ``Run Task`` and ``Run Task Class`` robot keywords throw an error like ``AttributeError: module 'robot.api.logger' has no attribute 'log'`` in rare cases (#3053)
+        * Fixed a bug in the `snowfakery` task which caused Unique IDs to not always be actually unique. (#3059)
+        
+        
         3.51.1 (2022-01-25)
+        -------------------
         
         Issues Closed
         

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/README.rst

@@ -49,8 +49,6 @@ from minute 36 through minute 54.
 Questions?
 ----------
 
-Salesforce.org (http://salesforce.org/) customers and partners can ask in the `CumulusCI group in the Power of Us Hub <https://powerofus.force.com/s/group/0F980000000PSRHCA4/cumulusci-cci>`_.
-
-Others can ask in the `CumulusCI group in the Trailblazer Community <https://success.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F9300000009M9ZCAU>`_.
+Ask in the `CumulusCI group in the Trailblazer Community <https://success.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F9300000009M9ZCAU>`_.
 
 *Please note:* CumulusCI is distributed under an `open source license <https://github.com/SFDO-Tooling/CumulusCI/blob/main/LICENSE>`_ and is not covered by the Salesforce Master Subscription Agreement.

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/cumulusci/cli/cci.py

@@ -25,7 +25,12 @@ from .project import project
 from .runtime import CliRuntime, pass_runtime
 from .service import service
 from .task import task
-from .utils import check_latest_version, get_installed_version, get_latest_final_version
+from .utils import (
+    check_latest_version,
+    get_installed_version,
+    get_latest_final_version,
+    warn_if_no_long_paths,
+)
 
 SUGGEST_ERROR_COMMAND = (
     """Run this command for more information about debugging errors: cci error --help"""
@@ -120,7 +125,7 @@ def handle_exception(
             traceback.print_exc(file=log_file)  # log stacktrace silently
 
     if should_show_stacktraces and not isinstance(error, USAGE_ERRORS):
-        raise error
+        error_console.print_exception()
 
 
 def connection_error_message(console: Console):
@@ -142,6 +147,7 @@ def show_version_info():
     console.print(f"CumulusCI version: {cumulusci.__version__} ({sys.argv[0]})")
     console.print(f"Python version: {sys.version.split()[0]} ({sys.executable})")
     console.print()
+    warn_if_no_long_paths(console=console)
 
     current_version = get_installed_version()
     latest_version = get_latest_final_version()

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/cumulusci/cli/error.py

@@ -8,6 +8,7 @@ import click
 import github3
 
 import cumulusci
+from cumulusci.cli.utils import warn_if_no_long_paths, win32_long_paths_enabled
 from cumulusci.core.exceptions import CumulusCIException
 from cumulusci.core.github import check_github_scopes, create_gist, get_github_api
 
@@ -40,7 +41,8 @@ def error():
     """
 
 
-CCI_LOGFILE_PATH = Path.home() / ".cumulusci" / "logs" / "cci.log"
+def get_logfile_path():
+    return Path.home() / ".cumulusci" / "logs" / "cci.log"
 
 
 @error.command(
@@ -48,11 +50,13 @@ CCI_LOGFILE_PATH = Path.home() / ".cumulusci" / "logs" / "cci.log"
     help="Outputs the most recent traceback (if one exists in the most recent log)",
 )
 def error_info():
-    if not CCI_LOGFILE_PATH.is_file():
-        click.echo(f"No logfile found at: {CCI_LOGFILE_PATH}")
+    warn_if_no_long_paths()
+    logfile_path = get_logfile_path()
+    if not logfile_path.is_file():
+        click.echo(f"No logfile found at: {logfile_path}")
         return
 
-    traceback = get_traceback(CCI_LOGFILE_PATH.read_text(encoding="utf-8"))
+    traceback = get_traceback(logfile_path.read_text(encoding="utf-8"))
     click.echo(traceback)
 
 
@@ -60,7 +64,7 @@ def get_traceback(log_content: str) -> str:
     """Returns the the traceback in a logfile if it exists."""
     stacktrace_start = "Traceback (most recent call last):"
     if stacktrace_start not in log_content:
-        return f"\nNo stacktrace found in: {CCI_LOGFILE_PATH}\n"
+        return f"\nNo stacktrace found in: {get_logfile_path()}\n"
 
     stacktrace = ""
     for i, line in enumerate(reversed(log_content.split("\n")), 1):
@@ -73,13 +77,14 @@ def get_traceback(log_content: str) -> str:
 
 @error.command(name="gist", help="Creates a GitHub gist from the latest logfile")
 @pass_runtime(require_project=False, require_keychain=True)
-def gist(runtime):
-    if CCI_LOGFILE_PATH.is_file():
-        log_content = CCI_LOGFILE_PATH.read_text(encoding="utf-8")
+def error_gist(runtime):
+    logfile_path = get_logfile_path()
+    if logfile_path.is_file():
+        log_content = logfile_path.read_text(encoding="utf-8")
     else:
         log_not_found_msg = """No logfile to open at path: {}
         Please ensure you're running this command from the same directory you were experiencing an issue."""
-        error_msg = log_not_found_msg.format(CCI_LOGFILE_PATH)
+        error_msg = log_not_found_msg.format(logfile_path)
         click.echo(error_msg)
         raise CumulusCIException(error_msg)
 
@@ -115,4 +120,6 @@ def get_context_info():
     info.append(f"CumulusCI version: {cumulusci.__version__}")
     info.append(f"Python version: {sys.version.split()[0]} ({sys.executable})")
     info.append(f"Environment Info: {host_info.system} / {host_info.machine}")
+    if host_info.system == "Windows":
+        info.append(f"Windows long path support enabled: {win32_long_paths_enabled()}")
     return "\n".join(info)

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/cumulusci/cli/org.py

@@ -11,7 +11,12 @@ from rich.console import Console
 from cumulusci.cli.ui import CliTable, SimpleSalesforceUIHelpers
 from cumulusci.core.config import OrgConfig, ScratchOrgConfig
 from cumulusci.core.exceptions import OrgNotFound
-from cumulusci.oauth.client import OAuth2Client, OAuth2ClientConfig
+from cumulusci.oauth.client import (
+    PROD_LOGIN_URL,
+    SANDBOX_LOGIN_URL,
+    OAuth2Client,
+    OAuth2ClientConfig,
+)
 from cumulusci.salesforce_api.utils import get_simple_salesforce_connection
 from cumulusci.utils import parse_api_datetime
 
@@ -109,27 +114,35 @@ def org_browser(runtime, org_name, path, url_only):
     org_config.save()
 
 
-def setup_client(runtime, login_url=None, sandbox=None) -> OAuth2Client:
-    """Provides an OAuth2Client for Connecting and Org"""
-    connected_app = runtime.keychain.get_service("connected_app")
-    base_uri = "https://{}.salesforce.com"
-    base_uri = login_url or base_uri.format("test" if sandbox else "login")
-    auth_uri = base_uri + "/services/oauth2/authorize"
-    token_uri = base_uri + "/services/oauth2/token"
+def setup_client(connected_app, login_url=None, sandbox=None) -> OAuth2Client:
+    """Provides an OAuth2Client for connecting an Org"""
+    if login_url:
+        base_uri = login_url
+    elif sandbox:
+        base_uri = SANDBOX_LOGIN_URL
+    elif connected_app.login_url:
+        base_uri = connected_app.login_url
+    else:
+        base_uri = PROD_LOGIN_URL
+    base_uri = base_uri.rstrip("/")
 
     sf_client_config = OAuth2ClientConfig(
         client_id=connected_app.client_id,
         client_secret=connected_app.client_secret,
         redirect_uri=connected_app.callback_url,
-        auth_uri=auth_uri,
-        token_uri=token_uri,
+        auth_uri=f"{base_uri}/services/oauth2/authorize",
+        token_uri=f"{base_uri}/services/oauth2/token",
         scope="web full refresh_token",
     )
     return OAuth2Client(sf_client_config)
 
 
 def connect_org_to_keychain(
-    client: OAuth2Client, runtime, global_org: bool, org_name: str
+    client: OAuth2Client,
+    runtime,
+    global_org: bool,
+    org_name: str,
+    connected_app: str,
 ) -> None:
     """Use the given client to authorize into an org, and save the
     new OrgConfig to the keychain."""
@@ -137,6 +150,7 @@ def connect_org_to_keychain(
 
     global_org = global_org or runtime.project_config is None
     org_config = OrgConfig(oauth_dict, org_name, runtime.keychain, global_org)
+    org_config.config["connected_app"] = connected_app
     org_config.load_userinfo()
     org_config._load_orginfo()
     if org_config.organization_sobject["TrialExpirationDate"] is None:
@@ -153,6 +167,12 @@ def connect_org_to_keychain(
     name="connect", help="Connects a new org's credentials using OAuth Web Flow"
 )
 @orgname_option_or_argument(required=True)
+@click.option(
+    "--connected-app",
+    "--connected_app",
+    "connected_app_name",
+    help="Name of the connected_app service to use.",
+)
 @click.option(
     "--sandbox", is_flag=True, help="If set, connects to a Salesforce sandbox org"
 )
@@ -171,7 +191,9 @@ def connect_org_to_keychain(
     is_flag=True,
 )
 @pass_runtime(require_project=False, require_keychain=True)
-def org_connect(runtime, org_name, sandbox, login_url, default, global_org):
+def org_connect(
+    runtime, org_name, sandbox, login_url, default, global_org, connected_app_name=None
+):
     runtime.check_org_overwrite(org_name)
 
     if login_url and ".lightning." in login_url:
@@ -180,8 +202,15 @@ def org_connect(runtime, org_name, sandbox, login_url, default, global_org):
             "Use the my.salesforce.com version instead"
         )
 
-    sf_client = setup_client(runtime, login_url, sandbox)
-    connect_org_to_keychain(sf_client, runtime, global_org, org_name)
+    connected_app_name = (
+        connected_app_name or runtime.keychain.get_default_service_name("connected_app")
+    )
+    click.echo(f"Connecting org using the {connected_app_name} connected app...")
+    connected_app = runtime.keychain.get_service("connected_app", connected_app_name)
+    sf_client = setup_client(connected_app, login_url, sandbox)
+    connect_org_to_keychain(
+        sf_client, runtime, global_org, org_name, connected_app_name
+    )
 
     if default and runtime.project_config is not None:
         runtime.keychain.set_default_org(org_name)
@@ -273,6 +302,7 @@ def org_info(runtime, org_name, print_json):
         ui_key_set = {
             "config_file",
             "config_name",
+            "connected_app",
             "created",
             "date_created",
             "days",
@@ -318,12 +348,22 @@ def org_info(runtime, org_name, print_json):
 )
 @pass_runtime(require_project=False, require_keychain=True)
 def org_list(runtime, json_flag, plain):
+    def _get_org_safe(org):
+        try:
+            return runtime.keychain.get_org(org)
+        except Exception as e:
+            click.echo(f"Cannot load org config for `{org}`: {e}")
+
     plain = plain or runtime.universal_config.cli__plain_output
-    org_configs = {
-        org: runtime.keychain.get_org(org) for org in runtime.keychain.list_orgs()
-    }
+    org_configs = ((org, _get_org_safe(org)) for org in runtime.keychain.list_orgs())
+    org_configs = {org: org_config for org, org_config in org_configs if org_config}
+
     json_data = {}
-    default_org_name, _ = runtime.keychain.get_default_org()
+    try:
+        default_org_name, _ = runtime.keychain.get_default_org()
+    except Exception:  # pragma: no cover
+        default_org_name = None
+
     for org, org_config in org_configs.items():
         is_org_default = org == default_org_name
         row_data = {"is_default": is_org_default, "name": org}
@@ -339,7 +379,12 @@ def org_list(runtime, json_flag, plain):
     console.print(scratch_table, justify="left")
     console.print(persistent_table, justify="left")
 
-    runtime.keychain.cleanup_org_cache_dirs()
+    try:
+        runtime.keychain.cleanup_org_cache_dirs()
+    except Exception:
+        click.echo(
+            "Cannot cleanup org cache dirs, perhaps due to org config files which cannot be decrypted."
+        )
 
 
 def _make_tables(json_data: dict):

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/cumulusci/cli/service.py

@@ -1,4 +1,3 @@
-import json
 import os
 from pathlib import Path
 from typing import Callable, Optional
@@ -35,7 +34,7 @@ def service_list(runtime, plain, print_json):
 
     console = Console()
     if print_json:
-        console.print(json.dumps(services))
+        console.print_json(data=services)
         return None
 
     configured_services = runtime.keychain.list_services()
@@ -84,17 +83,43 @@ class ConnectServiceCommand(click.MultiCommand):
         return sorted(services.keys())
 
     def _build_param(self, attribute: str, details: dict) -> click.Option:
-        req = details["required"]
+        required = details.get("required", False)
         default_factory: Optional[Callable] = self._get_callable_default(
             details.get("default_factory")
         )
-        prompt = None if default_factory else req
+        default = details.get("default")
+        if default is not None:
+            # This gives the user a chance to change the default
+            # (but only if they didn't specify it as a command line option)
+            default_factory = lambda: default  # noqa
+            prompt = True
+            # Since there's a default value,
+            # we don't need to indicate this option as required in help,
+            required = False
+        elif default_factory:
+            # If there's a function to calculate the default,
+            # we'll call it instead of prompting the user.
+            # This provides a hook for collecting the value in other ways,
+            # such as via an oauth flow.
+            prompt = None
+        else:
+            # If there's no default, we prompt the user only if the option is required.
+            prompt = required
+
+        # Make sure the description is included in the prompt
+        description = details.get("description")
+        if prompt:
+            prompt = attribute
+            if description:
+                prompt += f" ({description})"
 
         kwargs = {
             "prompt": prompt,
-            "required": req,
-            "help": details.get("description"),
+            "required": required,
+            "help": description,
             "default": default_factory,
+            # If there is a preset default, this causes it to be shown in help.
+            "show_default": default,
         }
         return click.Option((f"--{attribute}",), **kwargs)
 
@@ -138,7 +163,7 @@ class ConnectServiceCommand(click.MultiCommand):
                 f"Sorry, I don't know about the '{service_type}' service."
             )
 
-        attributes = service_config["attributes"].items()
+        attributes = service_config.get("attributes", {}).items()
         params = [self._build_param(attr, cnfg) for attr, cnfg in attributes]
         params.extend(self._get_default_options(runtime))
 

{cumulusci-3.52.0.dev0 → cumulusci-3.53.0}/cumulusci/cli/tests/test_cci.py

@@ -114,6 +114,7 @@ def test_main__cci_show_stacktraces(
     init_logger,
     get_tempfile_logger,
     tee,
+    capsys,
 ):
     runtime = mock.Mock()
     runtime.universal_config.cli__show_stacktraces = True
@@ -121,7 +122,7 @@ def test_main__cci_show_stacktraces(
     cli.side_effect = Exception
     get_tempfile_logger.return_value = (mock.Mock(), "tempfile.log")
 
-    with pytest.raises(Exception):
+    with pytest.raises(SystemExit):
         cci.main(["cci"])
 
     check_latest_version.assert_called_once()
@@ -129,6 +130,8 @@ def test_main__cci_show_stacktraces(
     CliRuntime.assert_called_once()
     cli.assert_called_once()
     post_mortem.assert_not_called()
+    captured = capsys.readouterr()
+    assert "Traceback (most recent call last)" in captured.err
 
 
 @mock.patch("cumulusci.cli.cci.tee_stdout_stderr")
@@ -413,6 +416,12 @@ def test_version__latest(capsys):
     assert "You have the latest version of CumulusCI" in console_output
 
 
+@mock.patch("cumulusci.cli.cci.warn_if_no_long_paths")
+def test_version__win_path_warning(warn_if):
+    run_click_command(cci.version)
+    warn_if.assert_called_once()
+
+
 @mock.patch("code.interact")
 def test_shell(interact):
     run_click_command(cci.shell)