streamsets 6.6.0.dev0__tar.gz → 6.6.0rc1__tar.gz

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: streamsets
-Version: 6.6.0.dev0
+Version: 6.6.0rc1
 Summary: A Python SDK for StreamSets
 Author: StreamSets Inc
 Classifier: Development Status :: 5 - Production/Stable

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/docs/api/sch_api.rst

@@ -205,6 +205,16 @@ Pipelines
 .. autoclass:: streamsets.sdk.sch_models.StPipelineBuilder
     :members:
 
+Projects
+^^^^^^^^
+
+.. autoclass:: streamsets.sdk.sch_models.Project
+    :members:
+.. autoclass:: streamsets.sdk.sch_models.Projects
+    :members:
+.. autoclass:: streamsets.sdk.sch_models.ProjectBuilder
+    :members:
+
 Protection Methods
 ^^^^^^^^^^^^^^^^^^
 .. autoclass:: streamsets.sdk.sch_models.ProtectionMethod

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/docs/build/editing_pipelines.rst

@@ -216,6 +216,27 @@ Setting the configuration value is as simple as directly setting the value in-me
 .. note::
   The same workflow can be followed to access/edit configuration values of :py:class:`streamsets.sdk.sch_models.Pipeline` objects
 
+Some configuration values have a predefined set of options to choose from. You can get a list of these using the :py:meth:`streamsets.sdk.sch.PipelineBuilder.get_stage_configuration_options` method, as shown below:
+
+.. code-block:: python
+
+    pipeline_builder.get_stage_configuration_options(
+        config_name="JSON Content",         # UI name of the configuration field
+        stage_name="Dev Raw Data Source",   # Name of the stage
+        config_name_type="label",           # Type of the config_name parameter (Default: label)
+        stage_name_type="label",            # Type of the stage_name parameter (Default: label)
+    )
+**Output:**
+
+.. code-block:: python:
+
+    [ChooseOption(UI_VALUE='JSON array of objects', SDK_VALUE='ARRAY_OBJECTS'), ChooseOption(UI_VALUE='Multiple JSON objects', SDK_VALUE='MULTIPLE_OBJECTS')]
+
+The list will match the values shown in the `JSON Content` UI dropdown.
+
+.. image:: ../_static/images/build/value_chooser__JSON_content.png
+|
+|
 Once you have edited your :py:class:`streamsets.sdk.sch_models.Pipeline` or :py:class:`streamsets.sdk.sch_models.SchSdcStage`, the changes must be published to Control Hub.
 This can be done by taking the updated :py:class:`streamsets.sdk.sch_models.Pipeline` instance and passing it into the :py:meth:`streamsets.sdk.sch.publish_pipeline` method as seen below:
 

streamsets-6.6.0rc1/docs/manage/projects/projects.rst

@@ -0,0 +1,242 @@
+Managing Projects
+=================
+|
+
+The SDK allows you to manage projects for your organization. This includes creating or deleting projects, managing users and groups
+that have access to a project.
+
+Creating A Project
+~~~~~~~~~~~~~~~~~~
+
+To create a Project on the Platform UI, you should go to Manage > Projects and click on the ``+`` icon.
+
+.. image:: ../../_static/images/manage/projects/create_project.png
+|
+
+To create a Project via the SDK, you need to instantiate a new instance of :py:class:`streamsets.sdk.sch_models.ProjectBuilder`
+to create a new instance of :py:class:`streamsets.sdk.sch_models.Project`.
+
+To instantiate the builder instance use :py:meth:`streamsets.sdk.ControlHub.get_project_builder` method.
+You can then call the :py:meth:`streamsets.sdk.sch_models.ProjectBuilder.build` method to create the new Project by supplying the ``name``
+and the optional ``description`` parameter.
+
+After instantiating the object, you can add it to Platform via the :py:meth:`streamsets.sdk.ControlHub.add_project`.
+
+.. code-block:: python
+
+    # Instantiate the ProjectBuilder
+    project_builder = sch.get_project_builder()
+
+    # build the project by passing name, and optionally a description
+    project = project_builder.build(name='Secret project', decription='Nothing to see')
+
+    # add the project to Platform
+    sch.add_project(project)
+
+Manage User And Group Access For Project
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Managing the project roles for a user or group is explained in the :ref:`Project Roles<project_roles>` section.
+
+Adding Users and Groups To Project
+----------------------------------
+
+After creating a project in the step above, we are now ready to add users and groups to a project.
+
+To do this you can use the following methods, :py:meth:`streamsets.sdk.sch_models.Project.add_user`
+and :py:meth:`streamsets.sdk.sch_models.Project.add_group`.
+
+You can pass one or more instances of :py:class:`streamsets.sdk.sch_models.User` or :py:class:`streamsets.sdk.sch_models.Group`
+to the respective methods to add them to a project.
+
+.. code-block:: python
+
+    # get a user to add to a project
+    user = sch.users.get(email_address="some.person@company.com")
+
+    # add the user to a project
+    project.add_user(user)
+
+    # you can add multiple users in a single call
+    project.add_user(user1, user2)
+
+    # get a group to add to a project
+    group = sch.groups.get(display_name='many-persons')
+
+    # add the group to the project
+    project.add_group(group)
+
+    # you can add multiple groups in a single call
+    project.add_group(group1, group2)
+
+Listing Users and Groups In Project
+-----------------------------------
+
+To list the users and groups in a project you can use the :py:attr:`streamsets.sdk.sch_models.Project.users` and
+:py:attr:`streamsets.sdk.sch_models.Project.groups` respectively.
+
+.. code-block:: python
+
+    # list users
+    project.users
+    # Output: [User(name=...), User(name...)]
+
+    # list groups
+    project.groups
+    # Output: [Group(display_name=...), Group(display_name=...)]
+
+Both attributes return a :py:class:`streamsets.sdk.utils.SeekableList` and it is possible to filter for users or groups based on their attributes.
+
+Removing Users and Groups From Project
+--------------------------------------
+
+To remove a user or group from a project you can use the following methods: :py:meth:`streamsets.sdk.sch_models.Project.remove_user`
+and :py:meth:`streamsets.sdk.sch_models.Project.remove_group`.
+
+You can pass one or more instances of :py:class:`streamsets.sdk.sch_models.User` or :py:class:`streamsets.sdk.sch_models.Group`
+to the respective methods to remove them from a project.
+
+.. code-block:: python
+
+    # remove a user
+    project.remove_user(user)
+
+    # remove multiple users
+    project.remove_user(user1, user2)
+
+    # remove a group
+    project.remove_group(group)
+
+    # remove multiple groups
+    project.remove_group(group1, group2)
+
+Listing Projects
+~~~~~~~~~~~~~~~~
+
+You can use the :py:attr:`streamsets.sdk.ControlHub.projects` to list all the projects in your Organization.
+This returns an instance of :py:class:`streamsets.sdk.utils.SeekableList` containing instances of
+:py:class:`streamsets.sdk.sch_models.Project`.
+
+.. code-block:: python
+
+    # list projects
+    sch.projects
+    # Output: [Project(name=..), Project(name=...)]
+
+    # get a particular project
+    project_x = sch.projects.get(name='project x')
+
+Switching Project And Organization Views
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the Platform UI, you start with a organization-level view by default. You can switch to a particular project's view by clicking
+on the option on the top bar of the screen and choosing which project to switch to. You can also click on the organization
+to switch back to an organization-level view.
+
+.. image:: ../../_static/images/manage/projects/switch_projects.png
+|
+
+In the SDK, we maintain a similar experience to the UI by starting with an organization-level view. We then allow the user to choose a project and switch to that project's view.
+
+This is achieved with the :py:attr:`streamsets.sdk.ControlHub.current_project` property. This property defaults to ``None``
+and can then be set to an instance of :py:class:`streamsets.sdk.sch_models.Project` to get that project's view.
+It can then be reset to ``None`` to return to an organization-level view.
+
+.. code-block:: python
+
+    # initialise a ControlHub instance
+    sch = ControlHub(...)
+
+    # the default view is always organization-level
+    sch.current_project is None
+    # Output: True
+
+    # get a project that we can switch to
+    project = sch.projects.get(name='my project')
+
+    # switch to the project view
+    sch.current_project = project
+
+    # SDK usage is the same inside a project when this attribute is set
+
+    # switch to organization-level view
+    sch.current_project = None
+
+Updating A Project
+~~~~~~~~~~~~~~~~~~
+
+Updating a project refers to updating the ``name`` and ``description`` of a project. We can do this by updating the values of those
+attributes of a :py:class:`streamsets.sdk.sch_models.Project` instance and then calling the :py:meth:`streamsets.sdk.ControlHub.update_project`
+method to update it on Platform.
+
+.. code-block:: python
+
+    # get a project to update
+    project = sch.projects.get(name='please update this name')
+
+    # set a new name or description
+    project.name = 'fantastic new name'
+    project.description = 'some description'
+
+    # update it on platform
+    sch.update_project(project)
+
+Deleting A Project
+~~~~~~~~~~~~~~~~~~
+
+A project can be deleted by passing a :py:class:`streamsets.sdk.sch_models.Project` instance to the
+:py:meth:`streamsets.sdk.ControlHub.delete_project` method.
+
+.. code-block:: python
+
+    # get a project to delete
+    project = sch.projects.get(name='delete this')
+
+    # call delete on platform
+    sch.delete_project(project)
+
+    # you can delete multiple projects at the same time
+    projects_to_delete = sch.projects.get_all(description='unused')
+    sch.delete_project(*projects_to_delete)
+
+Bringing It All Together
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The complete scripts from this section can be found below.
+
+.. code-block:: python
+
+    # Retrieve all projects
+    sch.projects
+
+    # switching to project's view
+    project = sch.projects.get(name='some project')
+    sch.current_project = project
+
+    # switching to an organization-level view
+    sch.current_project = None
+
+    # Creating a project
+    project_builder = sch.get_project_builder()
+    project = project_builder.build(name='some name', description='optional description')
+    sch.add_project(project)
+
+    # Updating a project's name or description
+    project.name = 'new name'
+    project.description = 'new description'
+    sch.update_project(project)
+
+    # Deleting a project
+    sch.delete_project(project)
+
+    # Retrieve users/groups in a project
+    project.users
+    project.groups
+
+    # Add users/groups to a project
+    # project.add_user(user1, user2)
+    # project.add_group(group1, group2)
+
+    # Remove users/groups from a project
+    # project.remove_user(user1, user2)
+    # project.remove_group(group1, group2)

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/docs/manage/users_and_groups/managing_users_and_groups.rst

@@ -77,7 +77,7 @@ method:
 
     user_builder = sch.get_user_builder()
     user = user_builder.build(email_address='johndeer@test.com')
-    user.roles = ['Connection Editor', 'Connection User', 'Topology Editor', 'Topology User']
+    user.organization_roles = ['Connection Editor', 'Connection User', 'Topology Editor', 'Topology User']
     # Add user to three groups that exist in the organization: all, beta-testers, pipeline operators
     group_one = sch.groups.get(display_name='all')
     group_two = sch.groups.get(display_name='beta-testers')
@@ -87,27 +87,27 @@ method:
 
 .. note::
   The :py:meth:`streamsets.sdk.ControlHub.invite_user` method will automatically update the invited user's in-memory
-  representation, including adding the user's ``id``, ``roles``, and ``groups``.
+  representation, including adding the user's ``id``, ``organization_roles``, and ``groups``.
 
 Updating An Existing User
 -------------------------
 .. _updating_users:
 
 Updating an existing user in the UI is done by expanding the user's details, making necessary changes to attributes like
-``roles`` or ``groups``, and saving the changes:
+``organization_roles`` or ``groups``, and saving the changes:
 
 .. image:: ../../_static/images/manage/users_and_groups/update_user_details.png
 
 |
-It is also possible to update a user's attributes, like ``roles`` or ``groups``, from the SDK. Simply retrieve the user
+It is also possible to update a user's attributes, like ``organization_roles`` or ``groups``, from the SDK. Simply retrieve the user
 you wish to update, modify the desired attribute(s), and then pass the user object to the :py:meth:`streamsets.sdk.ControlHub.update_user()`
 method:
 
 .. code-block:: python
 
     user = sch.users.get(email_address='mitch@streamsets.com')
-    # Set the user's roles to be the following
-    user.roles = ['Engine Administrator', 'Job Operator', 'Pipeline Editor', 'Deployment Manager']
+    # Set the user's organization roles to be the following
+    user.organization_roles = ['Engine Administrator', 'Job Operator', 'Pipeline Editor', 'Deployment Manager']
     # Add the user to a new group
     new_group = sch.groups.get(display_name='new-group')
     user.groups.append(new_group)
@@ -208,7 +208,7 @@ example have been removed.
     # ---- Adding Users ----
     user_builder = sch.get_user_builder()
     user = user_builder.build(email_address='johndeer@test.com')
-    user.roles = ['Connection Editor', 'Connection User', 'Topology Editor', 'Topology User']
+    user.organization_roles = ['Connection Editor', 'Connection User', 'Topology Editor', 'Topology User']
     # Add user to three groups that exist in the organization: all, beta-testers, pipeline operators
     group_one = sch.groups.get(display_name='all')
     group_two = sch.groups.get(display_name='beta-testers')
@@ -218,8 +218,8 @@ example have been removed.
 
     # ---- Updating An Existing User ----
     user = sch.users.get(email_address='mitch@streamsets.com')
-    # Set the user's roles to be the following
-    user.roles = ['Engine Administrator', 'Job Operator', 'Pipeline Editor', 'Deployment Manager']
+    # Set the user's organization roles to be the following
+    user.organization_roles = ['Engine Administrator', 'Job Operator', 'Pipeline Editor', 'Deployment Manager']
     # Add the user to a new group
     new_group = sch.groups.get(display_name='new-group')
     user.groups.append(new_group)
@@ -321,7 +321,7 @@ SDK, use the :py:class:`streamsets.sdk.sch_models.GroupBuilder` class. Retrieve
     user_two = sch.users.get(email_address='kramer@streamsets.com')
     group.users = [user_one, user_two]
     # Add the 'Pipeline User' role in addition to the defaults
-    group.roles.append('Pipeline User')
+    group.organization_roles.append('Pipeline User')
     response = sch.add_group(group)
 
 .. note::
@@ -332,12 +332,12 @@ Updating Groups
 .. _updating_groups:
 
 Updating an existing group in the Platform UI is done by expanding the group's details, making necessary changes
-to attributes like ``roles`` or ``users``, and saving the changes:
+to attributes like ``organization_roles`` or ``users``, and saving the changes:
 
 .. image:: ../../_static/images/manage/users_and_groups/update_group_details.png
 
 |
-It is also possible to update a group's attributes, like ``roles`` or ``users``, from the SDK. Simply retrieve the
+It is also possible to update a group's attributes, like ``organization_roles`` or ``users``, from the SDK. Simply retrieve the
 group you wish to update, modify the desired attribute(s), and then pass the group object to the :py:meth:`streamsets.sdk.ControlHub.update_group()`
 method:
 
@@ -346,7 +346,7 @@ method:
     group = sch.groups.get(display_name='example-group')
     user_to_add = sch.users.get(email_address='constanza@streamsets.com')
     group.users.append(user_to_add)
-    group.roles.remove('Engine Administrator')
+    group.organization_roles.remove('Engine Administrator')
     response = sch.update_group(group)
 
 .. note::
@@ -403,14 +403,14 @@ example have been removed.
     user_two = sch.users.get(email_address='kramer@streamsets.com')
     group.users = [user_one, user_two]
     # Add the 'Pipeline User' role in addition to the defaults
-    group.roles.append('Pipeline User')
+    group.organization_roles.append('Pipeline User')
     response = sch.add_group(group)
 
     # ---- Updating Groups ----
     group = sch.groups.get(display_name='example-group')
     user_to_add = sch.users.get(email_address='constanza@streamsets.com')
     group.users.append(user_to_add)
-    group.roles.remove('Engine Administrator')
+    group.organization_roles.remove('Engine Administrator')
     response = sch.update_group(group)
 
     # ---- Deleting Groups ----

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/docs/manage/users_and_groups/permissions.rst

@@ -59,7 +59,7 @@ instance the following steps can be taken:
 .. note::
   The ``subject_id`` format for an ACL permission varies based on whether it pertains to a user or group. A user's
   ``subject_id`` will be in the format ``user.id@organization.id``, while a group's ``subject_id`` will be in the format
-  ``group.id``.
+  ``group.group_id``.
 
 
 You can inspect an ACL definition's actions to see the level of access a particular user or group has to the resource:
@@ -135,7 +135,7 @@ method to add it to the object that owns the ACL:
     # Build the new permission definition for the subject_id (id), subject_type (user or group) and the
     # actions to allow for this subject.
     user_permission = acl.permission_builder.build(subject_id=user.id, subject_type='USER', actions=actions)
-    group_permission = acl.permission_builder.build(subject_id=group.id, subject_type='GROUP', actions=actions)
+    group_permission = acl.permission_builder.build(subject_id=group.group_id, subject_type='GROUP', actions=actions)
 
     # Add the permission definition to the ACL
     acl.add_permission(user_permission)
@@ -143,7 +143,7 @@ method to add it to the object that owns the ACL:
 
     # Show that the permission definition was correctly added to the ACL
     pipeline.acl.permissions.get(subject_id=user.id)
-    pipeline.acl.permissions.get(subject_id=group.id)
+    pipeline.acl.permissions.get(subject_id=group.group_id)
 
 
 **Output:**
@@ -153,7 +153,7 @@ method to add it to the object that owns the ACL:
     # pipeline.acl.permissions.get(subject_id=user.id)
     <Permission (resource_id=b99b5d55-380d-45a5-b8f1-0c9345fb662f:791759af-e8b5-11eb-8015-e592a7dbb2d0, subject_type=USER, subject_id=aa172288-c804-11ec-ba8b-4930c98e80a9@791759af-e8b5-11eb-8015-e592a7dbb2d0)>
 
-    # pipeline.acl.permissions.get(subject_id=group.id)
+    # pipeline.acl.permissions.get(subject_id=group.group_id)
     <Permission (resource_id=b99b5d55-380d-45a5-b8f1-0c9345fb662f:791759af-e8b5-11eb-8015-e592a7dbb2d0, subject_type=GROUP, subject_id=new_group@791759af-e8b5-11eb-8015-e592a7dbb2d0)>
 
 Updating Existing ACL Permissions on an Object
@@ -170,7 +170,7 @@ the actions as needed:
 
     # Retrieve the permission definition for the subject to be modified
     group = sch.groups.get(display_name='new-group')
-    permission = pipeline.acl.permissions.get(subject_id=group.id)
+    permission = pipeline.acl.permissions.get(subject_id=group.group_id)
 
     # Create a list of new actions that the permission definition will use
     updated_actions = ['READ']
@@ -179,13 +179,13 @@ the actions as needed:
     permission.actions = updated_actions
 
     # Show that the permission definition was correctly added to the ACL
-    pipeline.acl.permissions.get(subject_id=group.id).actions
+    pipeline.acl.permissions.get(subject_id=group.group_id).actions
 
 **Output:**
 
 .. code-block:: python
 
-    # pipeline.acl.permissions.get(subject_id=group.id).actions
+    # pipeline.acl.permissions.get(subject_id=group.group_id).actions
     ['READ']
 
 .. note::

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/docs/manage/users_and_groups/roles.rst

@@ -11,14 +11,14 @@ Selecting a User's Roles at Creation Time
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 As shown in the :ref:`Inviting Users<inviting_users>` section, a User's roles can be selected at creation time in the
-SDK by setting the the ``roles`` attribute equal to a list of roles for that user. You can then pass the :py:class:`streamsets.sdk.sch_models.User`
+SDK by setting the the ``organization_roles`` attribute equal to a list of roles for that user. You can then pass the :py:class:`streamsets.sdk.sch_models.User`
 instance to the :py:meth:`streamsets.sdk.ControlHub.invite_user` method:
 
 .. code-block:: python
 
     user_builder = sch.get_user_builder()
     user = user_builder.build(email_address='johndeer@test.com')
-    user.roles = ['Connection Editor', 'Connection User', 'Topology Editor', 'Topology User']
+    user.organization_roles = ['Connection Editor', 'Connection User', 'Topology Editor', 'Topology User']
     response = sch.invite_user(user)
 
 The SDK utilizes the same naming conventions for the roles as the Platform UI. A list of the available roles
@@ -29,14 +29,14 @@ Updating an Existing User's Roles
 
 As shown in the :ref:`Updating An Existing User<updating_users>` section, you can also update the roles of an existing
 user in the SDK. Similar to selecting roles at user creation time, you can update an existing user's roles by
-setting the the ``roles`` attribute equal to a list of roles the user should have and passing the updated :py:class:`streamsets.sdk.sch_models.User`
+setting the the ``organization_roles`` attribute equal to a list of roles the user should have and passing the updated :py:class:`streamsets.sdk.sch_models.User`
 instance to the :py:meth:`streamsets.sdk.ControlHub.update_user()` method:
 
 .. code-block:: python
 
     user = sch.users.get(email_address='kramer@streamsets.com')
     # Set the user's roles to be the following
-    user.roles = ['Engine Administrator', 'Job Operator', 'Pipeline Editor', 'Deployment Manager']
+    user.organization_roles = ['Engine Administrator', 'Job Operator', 'Pipeline Editor', 'Deployment Manager']
     response = sch.update_user(user)
 
 Selecting a Group's Roles at Creation Time
@@ -52,8 +52,8 @@ to the :py:meth:`streamsets.sdk.ControlHub.add_group` method:
     group_builder = sch.get_group_builder()
     group = group_builder.build(display_name='example-group', group_id='example_group')
     # Add the 'Pipeline User' role and remove the 'Engine Administrator' role
-    group.roles.append('Pipeline User')
-    group.roles.remove('Engine Administrator')
+    group.organization_roles.append('Pipeline User')
+    group.organization_roles.remove('Engine Administrator')
     response = sch.add_group(group)
 
 Updating an Existing Group's Roles
@@ -68,9 +68,40 @@ instance to the :py:meth:`streamsets.sdk.ControlHub.update_group` method:
 .. code-block:: python
 
     group = sch.groups.get(display_name='example-group')
-    group.roles.remove('Deployment Manager')
+    group.organization_roles.remove('Deployment Manager')
     response = sch.update_group(group)
 
 .. note::
   Being able to add or remove only one role at a time is a known limitation for the :py:class:`streamsets.sdk.sch_models.Group`
   class. It will be improved and expanded in a future release.
+
+Project Roles
+~~~~~~~~~~~~~
+
+.. _project_roles:
+
+If the Projects feature is enabled in your organization, you will be able to grant Users and Groups project-level
+roles. These roles behave the same as organization roles, but grant users a role across all projects they are
+assigned to. You can further control this by assigning them appropriate permissions to a resource.
+
+In the UI, you can see a section for project roles.
+
+.. image:: ../../_static/images/manage/users_and_groups/project_roles.png
+
+This behaves exactly like organization roles.
+
+In the SDK, you can use :py:attr:`streamsets.sdk.sch_models.User.project_roles` and :py:attr:`streamsets.sdk.sch_models.Group.project_roles`
+to modify project roles for a user or group. The usage and behavior is the same as that of :py:attr:`streamsets.sdk.sch_models.User.organization_roles`
+and :py:attr:`streamsets.sdk.sch_models.Group.organization_roles`.
+
+.. code-block:: python
+
+    # give a user the "Connection Editor" role.
+    user = sch.users.get(email_address="user.in@project")
+    user.project_roles.append("Connection Editor")
+    sch.update_user(user)
+
+    # remove the "Deployment Manager" role from a group
+    group = sch.groups.get(display_name="not deployment managers")
+    group.project_roles.remove("Deployment Manager")
+    sch.update_group(group)

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/docs/manage.rst

@@ -9,4 +9,5 @@ Manage
    :hidden:
 
    manage/legacy_kubernetes_integration
+   manage/projects/projects
    manage/users_and_groups/users_and_groups

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/streamsets/sdk/__version__.py

@@ -2,4 +2,4 @@
 #  PID 5900-BAF
 #  Copyright StreamSets Inc., an IBM Company 2024
 
-__version__ = "6.6.0-dev"
+__version__ = "6.6.0rc1"

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/streamsets/sdk/constants.py

@@ -375,8 +375,8 @@ ST_PIPELINE_BW_COMPATIBILITY = {
             'values': {True: 'PROVISION_CLUSTER', False: 'EXISTING_CLUSTER'},
         },
     },
+    # Transformer
     '6.0.0': {
-        # Transformer
         'transformerEmrConnection.masterInstanceType': {'name': 'transformerEmrConnection.primaryInstanceType'},
         'transformerEmrConnection.masterInstanceTypeCustom': {
             'name': 'transformerEmrConnection.primaryInstanceTypeCustom'
@@ -395,6 +395,9 @@ ST_PIPELINE_BW_COMPATIBILITY = {
         'sdcEmrConnection.slaveInstanceTypeCustom': {'name': 'sdcEmrConnection.secondaryInstanceTypeCustom'},
         'sdcEmrConnection.slaveSecurityGroup': {'name': 'sdcEmrConnection.secondarySecurityGroup'},
     },
+    '6.1.0': {
+        'googleCloudConfig.masterType': {'name': 'googleCloudConfig.primaryType'},
+    },
 }
 
 

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/streamsets/sdk/exceptions.py

@@ -113,6 +113,15 @@ class MultipleIssuesError(Exception):
         return str('{}: {}'.format(self.message, self.errors))
 
 
+class ProjectAccessError(Exception):
+    """User does not have access to the current Project."""
+
+    def __init__(self, code, message):
+        super().__init__(f"{code}: {message}")
+        self.code = code
+        self.message = message
+
+
 class UnprocessableEntityError(HTTPError):
     """Unprocessable Entity Error (HTTP 422)."""
 

{streamsets-6.6.0.dev0 → streamsets-6.6.0rc1}/streamsets/sdk/models.py

@@ -137,11 +137,9 @@ class Configuration:
             for config_item_index, config_item in enumerate(config_list):
                 if not isinstance(config_item, dict):
                     raise TypeError('A Configuration must be a list of dictionaries')
-                if self.property_key not in config_item or self.property_value not in config_item:
+                if self.property_key not in config_item:
                     raise TypeError(
-                        'Configuration {} does not contain property_key:{} or property_value:{}'.format(
-                            config_item, self.property_key, self.property_value
-                        )
+                        'Configuration {} does not contain property_key:{}'.format(config_item, self.property_key)
                     )
                 configuration_index_map[config_item[self.property_key]] = (config_item_index, config_list_index)
 
@@ -270,7 +268,7 @@ class Configuration:
         elif config.get('type') == 'integer':
             return int(config[self.property_value])
         else:
-            return config[self.property_value]
+            return config.get(self.property_value, None)
 
 
 class BaseModel: