streamsets 6.7.0.dev0__tar.gz → 7.0.0__tar.gz

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/.gitignore

@@ -1,4 +1,6 @@
-# Copyright 2017 StreamSets Inc.
+#  IBM Confidential
+#  PID 5900-BAF
+#  Copyright StreamSets Inc., an IBM Company 2024
 
 __pycache__/
 *.py[cod]
@@ -64,3 +66,4 @@ target/
 
 # pyenv python configuration file
 .python-version*.pyc
+/.python-version

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.8-slim
+FROM python:3.11-slim
 RUN apt-get update && \
     apt-get -y install \
     gcc \

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/HISTORY.rst

@@ -1,5 +1,53 @@
   .. warning::
-    Support for Python versions below 3.8 will be dropped in the next major release of Streamsets SDK (7.0.0)
+    Support for Python versions below 3.10 has been dropped as of the Streamsets SDK (7.0.0) release.
+
+7.0.0 (April 2026)
+-----------
+
+**New Features**
+
+* **Python Code Generator** - Added :py:class:`streamsets.sdk.codegen.generator.PythonGenerator` to convert existing pipelines to Python SDK code. Supports pipeline definitions from ``.zip`` archives, ``.json`` files, and :py:class:`streamsets.sdk.sch_models.Pipeline` objects. See :ref:`build/code_generation:Code Generation` for details.
+
+* **HTTP Retry Management** - Introduced comprehensive HTTP retry mechanism with exponential backoff and jitter. The new :py:mod:`streamsets.sdk.retry` module provides configurable retry policies for different HTTP status codes, with special handling for rate limiting (429). See :ref:`manage/http_retry_management:HTTP Retry Management` for configuration details.
+
+* **Engine Type Standardization** - Centralized all engine type references into :py:class:`streamsets.sdk.utils.EngineType` enumeration. Engine types are now consistently referenced as ``COLLECTOR``, ``TRANSFORMER``, and ``SNOWPARK`` across the SDK. See :ref:`usage/engine_types:A Note on Engine Types` for migration guidance.
+
+**Breaking Changes**
+
+* **Dropped Python 3.8 and 3.9 Support** - Python versions below 3.10 are no longer supported. Please upgrade to Python 3.10 or higher before upgrading to SDK 7.0.0.
+
+* **Removed Deprecated Features** - This major release removes all features that were previously marked as deprecated. Please update your code to use the replacement features before upgrading.
+
+**Removed API Methods (sch_api.py):**
+
+* Removed ``get_all_registered_executor_labels()`` - Use ``get_all_registered_engine_labels()`` instead
+* Removed ``get_all_registered_executor_versions()`` - Use ``get_all_registered_engine_versions()`` instead
+* Removed ``get_all_executor_stats()`` - Use ``get_all_engine_stats()`` instead
+* Removed ``get_executor_cpu_usage_time_series()`` - Use ``get_engine_cpu_usage_time_series()`` instead
+* Removed ``get_executor_memory_usage_time_series()`` - Use ``get_engine_memory_usage_time_series()`` instead
+
+**Removed Model Properties (sch_models.py):**
+
+* Removed ``User.roles`` - Use ``User.organization_roles`` instead
+* Removed ``Group.group_id`` - Use ``Group.id`` instead
+* Removed ``Group.roles`` - Use ``Group.organization_roles`` instead
+* Removed ``Pipeline.pipeline_id`` - Use ``Pipeline.id`` instead
+* Removed ``Pipeline.executor_type`` - Use ``Pipeline.engine_type`` instead
+* Removed ``Pipeline.sdc_id`` - Use ``Pipeline.engine_id`` instead
+* Removed ``Job.job_id`` - Use ``Job.id`` instead
+* Removed ``Job.executor_type`` - Use ``Job.engine_type`` instead
+* Removed ``Topology.topology_id`` - Use ``Topology.id`` instead
+* Removed ``Environment.environment_id`` - Use ``Environment.id`` instead
+* Removed ``Deployment.deployment_id`` - Use ``Deployment.id`` instead
+* Removed ``Deployment.engine_instances`` - Use ``Deployment.desired_instances`` instead
+
+**Removed ControlHub Methods (sch.py):**
+
+* Removed ``ControlHub.publish_scheduled_task()`` - Use ``ControlHub.add_scheduled_task()`` instead
+
+**Migration Guide:**
+
+All removed features have direct replacements. Simply update your code to use the new method/property names as indicated above. The functionality remains identical, only the names have changed to provide better consistency across the SDK.
 
 History
 =======

{streamsets-6.7.0.dev0/streamsets.egg-info → streamsets-7.0.0}/PKG-INFO

@@ -1,24 +1,21 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: streamsets
-Version: 6.7.0.dev0
+Version: 7.0.0
 Summary: A Python SDK for StreamSets
 Author: StreamSets Inc
 Classifier: Development Status :: 5 - Production/Stable
-Classifier: Programming Language :: Python :: 3.4
-Classifier: Programming Language :: Python :: 3.5
-Classifier: Programming Language :: Python :: 3.6
-Classifier: Programming Language :: Python :: 3.7
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
-Requires-Python: >=3.4
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Requires-Python: >=3.10
 Description-Content-Type: text/x-rst
 Requires-Dist: dpath==1.5.0
 Requires-Dist: inflection
 Requires-Dist: PyYAML
-Requires-Dist: requests
+Requires-Dist: requests>=2.33.0
 
 =========================
 StreamSets SDK for Python
@@ -26,10 +23,10 @@ StreamSets SDK for Python
 
 |Version| |Python|
 
-.. |Python| image:: https://shields.io/pypi/pyversions/streamsets.svg?style=flat
+.. |Python| image:: https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue
     :target: https://pypi.python.org/pypi/streamsets/
     :alt: Python Versions
-.. |Version| image:: http://img.shields.io/pypi/v/streamsets.svg?style=flat
+.. |Version| image:: https://img.shields.io/badge/pypi-v7.0.0-blue?style=flat
     :target: https://pypi.python.org/pypi/streamsets/
     :alt: Package Version
 

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/README.rst

@@ -4,10 +4,10 @@ StreamSets SDK for Python
 
 |Version| |Python|
 
-.. |Python| image:: https://shields.io/pypi/pyversions/streamsets.svg?style=flat
+.. |Python| image:: https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue
     :target: https://pypi.python.org/pypi/streamsets/
     :alt: Python Versions
-.. |Version| image:: http://img.shields.io/pypi/v/streamsets.svg?style=flat
+.. |Version| image:: https://img.shields.io/badge/pypi-v7.0.0-blue?style=flat
     :target: https://pypi.python.org/pypi/streamsets/
     :alt: Package Version
 

streamsets-7.0.0/build-publish/Jenkinsfile

@@ -0,0 +1,161 @@
+pipeline {
+    agent {
+        label 'testframework-2g-2c'
+    }
+
+    parameters {
+        string(
+            name: 'BRANCH',
+            defaultValue: 'master',
+            description: 'The branch of the SDK repository to build from.'
+        )
+        string(
+            name: 'PYPI_VERSION',
+            defaultValue: '',
+            description: 'A version identifier used for publishing to PyPI. Example: 5.2.0b1'
+        )
+        choice(
+            name: 'PYPI_INSTANCE',
+            choices: ['test', 'production'],
+            description: '''Which PyPI instance to publish to.
+            <p><b>WARNING: Publishing permanently uses that version number. Be very careful when publishing to production.</b></p>'''
+        )
+        string(
+            name: 'BRANCH_TAG',
+            defaultValue: '',
+            description: '''<p>This is a postfix to the git tag.</p>
+            <p>Default is: "v$PYPI_VERSION", i.e. "v5.2.0b1"</p>
+            <p>When BRANCH_TAG is given: PyPI release "v$PYPI_VERSION+$BRANCH_TAG"</p>'''
+        )
+    }
+
+    environment {
+        SDK_VERSION_FILE='python/streamsets/sdk/__version__.py'
+        VENV_DIR = "${env.WORKSPACE}/venv"
+    }
+
+    stages {
+        stage('Build & Publish in Container') {
+            agent {
+                docker {
+                    image 'python:3.10'
+                    reuseNode true
+                    args '--userns=keep-id --privileged --env CONTAINER_HOST=unix:///run/podman/podman.sock --volume $XDG_RUNTIME_DIR/podman/podman.sock:/run/podman/podman.sock'
+                }
+            }
+            stages {
+
+                stage('Check Parameters') {
+                    steps {
+                        script {
+                            if (!env.PYPI_VERSION?.trim()) {
+                                error "No version provided via the PYPI_VERSION environment variable."
+                            }
+
+                            withCredentials([
+                                string(credentialsId: 'pypi-test-token', variable: 'PYPI_TOKEN_TEST'),
+                                string(credentialsId: 'pypi-prod-token', variable: 'PYPI_TOKEN_PROD')
+                            ]) {
+                                if (env.PYPI_INSTANCE == 'production') {
+                                    env.TWINE_USERNAME = '__token__'
+                                    env.TWINE_PASSWORD = PYPI_TOKEN_PROD  // pragma: allowlist secret
+                                } else if (env.PYPI_INSTANCE == 'test') {
+                                    env.TWINE_REPOSITORY_URL = 'https://test.pypi.org/legacy/'
+                                    env.TWINE_USERNAME = '__token__'
+                                    env.TWINE_PASSWORD = PYPI_TOKEN_TEST  // pragma: allowlist secret
+                                } else {
+                                    error "Invalid or empty value for PYPI_INSTANCE. Valid options are: {production, test}."
+                                }
+                            }
+                        }
+                    }
+                }
+
+                stage('Checkout SDK from GitHub') {
+                    steps {
+                        dir('sdk') {
+                            git(
+                                url: 'https://github.ibm.com/ibmstreamsets/sdk.git',
+                                credentialsId: 'github-streamsetsci',
+                                branch: "${params.BRANCH}"
+                            )
+                        }
+                    }
+                }
+
+                stage('Ensure git doesnt scream') {
+                    steps {
+                        sh 'git config --global --add safe.directory ${WORKSPACE}'
+                    }
+                }
+
+                stage('Install Requirements') {
+                    steps {
+                        sh '''
+                            python3 -m venv "$VENV_DIR"
+                            . "$VENV_DIR/bin/activate"
+                            export PATH="$HOME/.local/bin:$PATH"
+                            pip install --upgrade pip wheel twine pre-commit
+
+                            cd python
+                            git config --unset-all core.hooksPath || true
+                            make setup
+                            cd ../
+                        '''
+                    }
+                }
+
+                stage('Update Version') {
+                    steps {
+                        sh '''
+                            . "$VENV_DIR/bin/activate"
+                            echo "Updating version number to: ${PYPI_VERSION}"
+                            sed -i.old "s/\\".*\\"/\\"${PYPI_VERSION}\\"/" ${SDK_VERSION_FILE}
+                            rm ${SDK_VERSION_FILE}.old
+                        '''
+                    }
+                }
+
+                stage('Build') {
+                    steps {
+                        sh '''
+                            . "$VENV_DIR/bin/activate"
+                            cd python
+                            echo 'Building'
+                            make build
+                        '''
+                    }
+                }
+
+                stage('Upload') {
+                    steps {
+                        sh '''
+                            echo ""
+                            echo "Releasing version ${PYPI_VERSION} to the ${PYPI_INSTANCE} PyPI."
+                            echo ""
+                            . "$VENV_DIR/bin/activate"
+                            cd python
+                            set +e
+                            twine upload --non-interactive dist/*
+                            RESULT=$?
+                            set -e
+                            if [ $RESULT -ne 0 ]; then
+                                echo "Twine failed to execute. Cleaning up and bailing." >&2
+                                rm -rf dist/ build/ streamsets_testframework.egg-info/
+                                exit 1
+                            fi
+                        '''
+                    }
+                }
+
+                stage('Cleanup') {
+                    steps {
+                        sh '''
+                            rm -rf dist/ build/ streamsets_testframework.egg-info/ "$VENV_DIR"
+                        '''
+                    }
+                }
+            }
+        }
+    }
+}

streamsets-7.0.0/build_requirements.txt

@@ -0,0 +1,2 @@
+setuptools==80.9.0
+build==1.0.3

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/docs/api/sch_api.rst

@@ -110,6 +110,13 @@ Engines
 .. autoclass:: streamsets.sdk.sch_models.Engines
     :members:
 
+EngineType
+^^^^^^^^^^
+.. autoclass:: streamsets.sdk.utils.EngineType
+    :members:
+.. autoclass:: streamsets.sdk.utils.DeploymentEngineType
+    :members:
+
 Environments
 ^^^^^^^^^^^^
 .. autoclass:: streamsets.sdk.sch_models.AWSEnvironment
@@ -246,6 +253,11 @@ ProvisioningAgents
 .. autoclass:: streamsets.sdk.sch_models.ProvisioningAgents
     :members:
 
+Python Generator
+^^^^^^^^^^^^^^^^
+.. autoclass:: streamsets.sdk.codegen.generator.PythonGenerator
+    :members:
+
 Roles
 ^^^^^
 .. autoclass:: streamsets.sdk.sch_models.Roles

streamsets-7.0.0/docs/build/code_generation.rst

@@ -0,0 +1,52 @@
+Code Generation
+===============
+|
+:py:class:`streamsets.sdk.codegen.generator.PythonGenerator` is an entry point that converts existing pipelines to Python SDK code.
+Supported pipeline definition sources are:
+
+* :py:class:`streamsets.sdk.sch_models.Pipeline` object
+* ``.zip`` archive
+* ``.json`` file
+
+Prerequisites
+~~~~~~~~~~~~~
+
+To properly generate a script, a few preparatory steps are required:
+
+1. Make sure your Control Hub Credential Id is exported as an environment variable e.g. ``SCH_CREDENTIAL_ID``.
+2. Make sure your Control Hub Token is exported as an environment variable e.g. ``SCH_TOKEN``.
+
+These environments variables will be used by the generated script during execution.
+
+Example usage
+~~~~~~~~~~~~~
+
+The example below demonstrates how to regenerate Python SDK code from pipeline exported to ``pipeline.zip`` archive.
+
+.. code-block:: python
+
+    from streamsets.sdk.codegen import PythonGenerator
+
+
+    generator = PythonGenerator(
+        source="/path/to/file/pipeline.zip",
+        destination="/tmp/output.py"
+    )
+    generator.save()
+
+|
+Another example demonstrates how to regenerate Python SDK code from the :py:class:`streamsets.sdk.sch_models.Pipeline` object.
+Additionally, configuration options were used to use different environment variable names that stores the Control Hub credentials.
+
+.. code-block:: python
+
+    from streamsets.sdk.codegen import PythonGenerator
+
+
+    generator = PythonGenerator(
+        source="/path/to/file/pipeline.zip",
+        destination="/tmp/output.py",
+        sch_credential_id="SCH_PROD_CRED_ID",
+        sch_token="SCH_PROD_TOKEN"
+    )
+    generator.save()

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/docs/build/creating_pipelines.rst

@@ -24,31 +24,31 @@ To accomplish the same task and create a pipeline using the SDK, the first step
 configuration on your behalf by building the initial JSON representation of the pipeline, and setting default values for
 essential properties (instead of requiring each to be set manually). Use the :py:meth:`streamsets.sdk.ControlHub.get_pipeline_builder`
 method to instantiate the builder object by passing in the ``engine_type`` for the pipeline you plan to create -
-available engine types are ``'data_collector'``, ``'snowflake'``, or ``'transformer'``.
+available engine types are ``'COLLECTOR'``, ``'SNOWPARK'``, or ``'TRANSFORMER'``.
 
 Instantiating a :py:class:`streamsets.sdk.sch_models.PipelineBuilder` instance for either
-the ``'data_collector'`` or ``'transformer'`` engine types requires the Authoring Engine be specified for the pipeline.
+the ``'COLLECTOR'`` or ``'TRANSFORMER'`` engine types requires the Authoring Engine be specified for the pipeline.
 It can be passed into the builder's instantiation via the ``engine_id`` parameter:
 
 .. code-block:: python
 
     sdc = sch.engines.get(engine_url='<data_collector_url>')
-    # Available engine types are 'data_collector', 'snowflake', or 'transformer'
-    pipeline_builder = sch.get_pipeline_builder(engine_type='data_collector', engine_id=sdc.id)
+    # Available engine types are 'COLLECTOR', 'SNOWPARK', or 'TRANSFORMER'
+    pipeline_builder = sch.get_pipeline_builder(engine_type='COLLECTOR', engine_id=sdc.id)
 
-The ``'transformer'`` engine type follows the same conventions:
+The ``'TRANSFORMER'`` engine type follows the same conventions:
 
 .. code-block:: python
 
     transformer = sch.engines.get(engine_url='<transformer_url>', engine_type='TRANSFORMER')
-    pipeline_builder = sch.get_pipeline_builder(engine_type='transformer', engine_id=transformer.id)
+    pipeline_builder = sch.get_pipeline_builder(engine_type='TRANSFORMER', engine_id=transformer.id)
 
 On the other hand, when instantiating a :py:class:`streamsets.sdk.sch_models.PipelineBuilder` instance for the
-``'snowflake'`` engine type, the ``engine_id`` parameter should not be specified:
+``'SNOWPARK'`` engine type, the ``engine_id`` parameter should not be specified:
 
 .. code-block:: python
 
-    pipeline_builder = sch.get_pipeline_builder(engine_type='snowflake')
+    pipeline_builder = sch.get_pipeline_builder(engine_type='SNOWPARK')
 
 .. _adding-stages-to-pipeline-builder:
 
@@ -437,7 +437,7 @@ the :py:meth:`streamsets.sdk.sch_models.PipelineBuilder.build` method:
 
 .. code-block:: python
 
-    <Pipeline (pipeline_id=None, commit_id=None, name=My first pipeline, version=None)>
+    <Pipeline (id=None, commit_id=None, name=My first pipeline, version=None)>
 
 When building a Transformer for Snowflake pipeline, there are 4 parameters required by the Pipeline Canvas UI, seen
 below:
@@ -504,9 +504,9 @@ Commands that only served to verify some output from the example have been remov
 
     sch = ControlHub(credential_id='<credential_id>', token='<token>')
     sdc = sch.engines.get(engine_url='<data_collector_url>')
-    pipeline_builder = sch.get_pipeline_builder(engine_type='data_collector', engine_id=sdc.id)
+    pipeline_builder = sch.get_pipeline_builder(engine_type='COLLECTOR', engine_id=sdc.id)
     #transformer = sch.engines.get(engine_url='<transformer_url>', engine_type='TRANSFORMER')
-    #pipeline_builder = sch.get_pipeline_builder(engine_type='transformer', engine_id=transformer.id)
+    #pipeline_builder = sch.get_pipeline_builder(engine_type='TRANSFORMER', engine_id=transformer.id)
 
     dev_raw_data_source = pipeline_builder.add_stage('Dev Raw Data Source')
     trash = pipeline_builder.add_stage('Trash')
@@ -545,7 +545,7 @@ Transformer For Snowflake:
     from streamsets.sdk import ControlHub
 
     sch = ControlHub(credential_id='<credential_id>', token='<token>')
-    pipeline_builder = sch.get_pipeline_builder(engine_type='snowflake')
+    pipeline_builder = sch.get_pipeline_builder(engine_type='SNOWPARK')
 
     snowflake_query_origin = pipeline_builder.add_stage('Snowflake Query')
     trash = pipeline_builder.add_stage('Trash')

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/docs/build/editing_pipelines.rst

@@ -24,7 +24,7 @@ This attribute returns a :py:class:`streamsets.sdk.utils.SeekableList` of :py:cl
 
     sch.pipelines
 
-Alternatively, you can retrieve specific pipelines by specifying ``pipeline_id``, ``name``, ``version``, or ``commit_id`` to filter the pipeline results
+Alternatively, you can retrieve specific pipelines by specifying ``id``, ``name``, ``version``, or ``commit_id`` to filter the pipeline results
 when calling the :py:meth:`streamsets.sdk.utils.SeekableList.get` or :py:meth:`streamsets.sdk.utils.SeekableList.get_all` methods:
 
 .. code-block:: python
@@ -40,14 +40,14 @@ when calling the :py:meth:`streamsets.sdk.utils.SeekableList.get` or :py:meth:`s
 .. code-block:: python
 
     # pipeline
-    <Pipeline (pipeline_id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin, commit_id=491cf010-da8c-4e63-9918-3f5ef3b182f6:admin, name=Test Pipeline, version=1)>
+    <Pipeline (id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin, commit_id=491cf010-da8c-4e63-9918-3f5ef3b182f6:admin, name=Test Pipeline, version=1)>
 
     # all_version_1_pipelines
-    [<Pipeline (pipeline_id=88d58863-7e8b-4831-a929-8c56db629483:admin,
+    [<Pipeline (id=88d58863-7e8b-4831-a929-8c56db629483:admin,
                 commit_id=600a7709-6a13-4e9b-b4cf-6780f057680a:admin,
                 name=Test Pipeline,
                 version=1)>,
-     <Pipeline (pipeline_id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin,
+     <Pipeline (id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin,
                 commit_id=491cf010-da8c-4e63-9918-3f5ef3b182f6:admin,
                 name=Test Pipeline 2,
                 version=1)>]

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/docs/build/pipeline_fragments.rst

@@ -28,8 +28,8 @@ method to instantiate the builder object, and pass in the relevant ``engine_type
 ``fragment`` parameter.
 
 .. note::
-  Similar to pipelines, the ``engine_id`` is **required** for the ``'data_collector'`` and ``'transformer'``
-  engine types when creating a pipeline fragment but should not be specified for the ``'snowflake'`` engine type.
+  Similar to pipelines, the ``engine_id`` is **required** for the ``'COLLECTOR'`` and ``'TRANSFORMER'``
+  engine types when creating a pipeline fragment but should not be specified for the ``'SNOWPARK'`` engine type.
   Additionally, ``fragment=True`` **must** be supplied when creating a pipeline fragment.
 
 The following code creates a fragment identical to the one seen in the screenshot above:
@@ -38,7 +38,7 @@ The following code creates a fragment identical to the one seen in the screensho
 
     # Initialize fragment builder
     sdc = sch.engines.get(engine_url='<data_collector_url>')
-    pipeline_builder = sch.get_pipeline_builder(engine_type='data_collector', engine_id=sdc.id, fragment=True)
+    pipeline_builder = sch.get_pipeline_builder(engine_type='COLLECTOR', engine_id=sdc.id, fragment=True)
 
     # Add stages to the pipeline builder
     dev_data_generator = pipeline_builder.add_stage('Dev Data Generator')
@@ -75,17 +75,17 @@ when calling :py:meth:`streamsets.sdk.utils.SeekableList.get` or :py:meth:`strea
 
 .. code-block:: python
 
-    [<Pipeline (pipeline_id=88d58863-7e8b-4831-a929-8c56db629483:admin,
+    [<Pipeline (id=88d58863-7e8b-4831-a929-8c56db629483:admin,
                 commit_id=600a7709-6a13-4e9b-b4cf-6780f057680a:admin,
                 name=Dev as fragment,
                 version=1)>,
-     <Pipeline (pipeline_id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin,
+     <Pipeline (id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin,
                 commit_id=491cf010-da8c-4e63-9918-3f5ef3b182f6:admin,
                 name=Test Fragment,
                 version=1)>]
 
 Alternatively, you can retrieve a specific pipeline fragment the same way you would any other pipeline: by specifying
-``pipeline_id``, ``name``, or ``commit_id`` to filter the pipeline results:
+``id`` , ``name``, or ``commit_id`` to filter the pipeline results:
 
 .. code-block:: python
 
@@ -98,7 +98,7 @@ Alternatively, you can retrieve a specific pipeline fragment the same way you wo
 .. code-block:: python
 
     # pipeline_fragment
-    <Pipeline (pipeline_id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin, commit_id=491cf010-da8c-4e63-9918-3f5ef3b182f6:admin, name=Test Fragment, version=1)>
+    <Pipeline (id=5b67c7dc-729b-43cc-bee7-072d3feb184b:admin, commit_id=491cf010-da8c-4e63-9918-3f5ef3b182f6:admin, name=Test Fragment, version=1)>
 
     # pipeline_fragment.fragment
     True
@@ -123,7 +123,7 @@ screenshot above:
 .. code-block:: python
 
     sdc = sch.engines.get(engine_url='<data_collector_url>')
-    pipeline_builder = sch.get_pipeline_builder(engine_type='data_collector', engine_id=sdc.id)
+    pipeline_builder = sch.get_pipeline_builder(engine_type='COLLECTOR', engine_id=sdc.id)
 
     # Retrieve the fragment object to add to the pipeline
     fragment = sch.pipelines.get(fragment=True, name='Test Fragment')
@@ -169,7 +169,7 @@ parameter when calling :py:meth:`streamsets.sdk.utils.SeekableList.get` or :py:m
 
 .. code-block:: python
 
-    [<Pipeline (pipeline_id=0e1a42c9-7ce3-4295-84dd-ff53a7b313c3:admin,
+    [<Pipeline (id=0e1a42c9-7ce3-4295-84dd-ff53a7b313c3:admin,
                 commit_id=f3479d83-6e52-4f85-824c-e8ef4185d8f6:admin,
                 name=Test Pipeline,
                 version=1)>]
@@ -223,7 +223,7 @@ example have been removed, as have any overlapping/redundant commands.
 
     # ---- CREATING THE PIPELINE FRAGMENT ----
     # Initialize fragment builder
-    pipeline_builder = sch.get_pipeline_builder(engine_type='data_collector', engine_id=sdc.id, fragment=True)
+    pipeline_builder = sch.get_pipeline_builder(engine_type='COLLECTOR', engine_id=sdc.id, fragment=True)
     # Add stages to the pipeline builder
     dev_data_generator = pipeline_builder.add_stage('Dev Data Generator')
     expression_evaluator = pipeline_builder.add_stage('Expression Evaluator')
@@ -235,7 +235,7 @@ example have been removed, as have any overlapping/redundant commands.
     sch.publish_pipeline(fragment)
 
     # ---- ADDING THE FRAGMENT TO A PIPELINE ----
-    pipeline_builder = sch.get_pipeline_builder(engine_type='data_collector', engine_id=sdc.id)
+    pipeline_builder = sch.get_pipeline_builder(engine_type='COLLECTOR', engine_id=sdc.id)
     # Retrieve the fragment object to add to the pipeline
     fragment = sch.pipelines.get(fragment=True, name='Test Fragment')
     # Add the fragment to the pipeline builder, which returns a Stage object

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/docs/build.rst

@@ -8,6 +8,7 @@ Build
    :maxdepth: 3
    :hidden:
 
+   build/code_generation
    build/creating_pipelines
    build/editing_pipelines
    build/pipeline_fragments

{streamsets-6.7.0.dev0 → streamsets-7.0.0}/docs/conf.py

@@ -56,7 +56,7 @@ master_doc = 'index'
 
 # General information about the project.
 project = u'StreamSets Platform SDK for Python'
-copyright = u'2023 StreamSets'
+copyright = u'Copyright StreamSets Inc., an IBM Company 2024'
 
 # The version info for the project you're documenting, acts as replacement
 # for |version| and |release|, also used in various other places throughout