teradataml 20.0.0.4__py3-none-any.whl → 20.0.0.6__py3-none-any.whl

teradataml/README.md

@@ -6,7 +6,7 @@ For community support, please visit the [Teradata Community](https://support.ter
 
 For Teradata customer support, please visit [Teradata Support](https://support.teradata.com/csm).
 
-Copyright 2024, Teradata. All Rights Reserved.
+Copyright 2025, Teradata. All Rights Reserved.
 
 ### Table of Contents
 * [Release Notes](#release-notes)
@@ -17,10 +17,179 @@ Copyright 2024, Teradata. All Rights Reserved.
 
 ## Release Notes:
 
+#### teradataml 20.00.00.06
+* ##### New Features/Functionality
+  * ###### teradataml: SDK
+    * Added new client `teradataml.sdk.Client` which can be used by user to make REST calls through SDK.
+    * New exception added in `teradataml`, specifically for REST APIs `TeradatamlRestException` that has attribute `json_resonse` providing proper printable json.
+    * Exposed three different ways of authentication through `Client`.
+      * Client credentials Authentication through `ClientCredentialsAuth` class.
+      * Device code Authentication through `DeviceCodeAuth` class.
+      * Bearer Authentication through `BearerAuth` class.
+
+  * ###### teradataml: ModelOps SDK
+    * `teradataml` exposes Python interfaces for all the REST APIs provided by Teradata Vantage ModelOps.
+    * Added support for `blueprint()` method which prints available classes in `modelops` module.
+    * Added new client `ModelOpsClient` with some additional function compared to `teradataml.sdk.Client`.
+    * teradataml classes are added for the schema in ModelOps OpenAPI specification.
+    ```python
+    >>> from teradataml.sdk.modelops import ModelOpsClient, Projects
+    >>> from teradataml.common.exceptions import TeradatamlRestException
+    >>> from teradataml.sdk import DeviceCodeAuth, BearerAuth, ClientCredentialsAuth # Authentication related classes.
+    >>> from teradataml.sdk.modelops import models # All classes related to OpenAPI schema are present in this module.
+
+    # Print available classes in modelops module.
+    >>> from teradataml.sdk.modelops import blueprint
+    >>> blueprint()
+
+    # Create ClientCredentialsAuth object and create ModelOpsClient object.
+    >>> cc_obj = ClientCredentialsAuth(auth_client_id="<client_id>",
+                                       auth_client_secret="<client_secret>",
+                                       auth_token_url="https://<example.com>/token")
+    >>> client = ModelOpsClient(base_url="<base_url>", auth=cc_obj, ssl_verify=False)
+
+    # Create Projects object.
+    >>> p = Projects(client=client)
+
+    # Create project using `body` argument taking object of ProjectRequestBody.
+    >>> project_paylod = {
+            "name": "dummy_project",
+            "description": "dummy_project created for testing",
+            "groupId": "<group_ID>",
+            "gitRepositoryUrl": "/app/built-in/empty",
+            "branch": "<branch>"
+        }
+    >>> p.create_project(body=models.ProjectRequestBody(**project_payload))
+    ```
+
+  * ###### teradataml: Functions
+    * `get_formatters()` - Get the formatters for NUMERIC, DATE and CHAR types.
+
+  * ###### teradataml: DataFrame Methods
+    * `get_snapshot()` - Gets the snapshot data of a teradataml DataFrame created on OTF table for a given snapshot id or timestamp.
+    * `from_pandas()`: Creates a teradataml DataFrame from a pandas DataFrame.
+    * `from_records()`: Creates a teradataml DataFrame from a list.
+    * `from_dict()`: Creates a teradataml DataFrame from a dictionary.
+
+  * ###### teradataml: DataFrame Property
+    * `history` - Returns snapshot history for a DataFrame created on OTF table.
+    * `manifests` - Returns manifest information for a DataFrame created on OTF table.
+    * `partitions` - Returns partition information for a DataFrame created on OTF table.
+    * `snapshots` - Returns snapshot information for a DataFrame created on OTF table.
+
+  * ###### teradataml DataFrameColumn a.k.a. ColumnExpression
+    * `DataFrameColumn.rlike()` - Function to match a string against a regular expression pattern.
+    * `DataFrameColumn.substring_index()` - Function to return the substring from a column before a specified 
+    delimiter, up to a given occurrence count.
+    * `DataFrameColumn.count_delimiters()` - Function to count the total number of occurrences of a specified delimiter.
+
+* ##### Updates
+  * ###### teradataml DataFrameColumn a.k.a. ColumnExpression
+    * `DataFrameColumn.like()`
+      * Added argument `escape_char` to specify the escape character for the LIKE pattern.
+      * Argument `pattern` now accepts DataFrameColumn as input.
+    * `DataFrameColumn.ilike()`
+      * Added argument `escape_char` to specify the escape character for the ILIKE pattern.
+      * Argument `pattern` now accepts DataFrameColumn as input.
+    * `DataFrameColumn.parse_url()` - Added argument `key` to extract a specific query parameter when `url_part` is set to "QUERY".
+
+  * ###### teradataml: DataFrame function
+    * `groupby()`, `cube()` and `rollup()`
+       * Added argument `include_grouping_columns` to include aggregations on the grouping column(s).
+    * `DataFrame()`: New argument `data`, that accepts input data to create a teradataml DataFrame, is added.
+
+  * ###### General functions
+    * `set_auth_token()`
+      * New keyword argument `auth_url` accepts the endpoint URL for a keycloak server.
+      * New keyword argument `rest_client` accepts name of the service for which keycloak token is to be generated.
+      * New keyword argument `validate_jwt` accepts the boolean flag to decide whether to validate generated JWT token or not.
+      * New keyword argument `valid_from` accepts the epoch seconds representing time from which JWT token will be valid.
+
+  * ###### teradataml Options
+    * Configuration Options
+      * `configure.use_short_object_name`
+        Specifies whether to use a shorter name for temporary database objects which are created by teradataml internally.
+
+  * ###### BYOM Function
+    * Supports special characters.
+
+#### teradataml 20.00.00.05
+* ##### New Features/Functionality
+  * ##### teradataml: AutoML
+      * New methods added for `AutoML()`, `AutoRegressor()` and `AutoClassifier()`:
+        * `get_persisted_tables()` - List the persisted tables created during AutoML execution.
+        * `visualize()` - Generates visualizations to analyze and understand the underlying patterns in the data.
+  
+  * ##### AutoDataPrep - Automated Data Preparation
+    AutoDataPrep simplifies the data preparation process by automating the different aspects of 
+    data cleaning and transformation, enabling seamless exploration, transformation, and optimization of datasets.
+    * `AutoDataPrep`
+      * Methods of AutoDataPrep
+        * `__init__()` - Instantiate an object of AutoDataPrep with given parameters.
+        * `fit()` - Perform fit on specified data and target column.
+        * `get_data()` - Retrieve the data after AutoDataPrep.
+        * `load()` - Load the saved datasets from Teradata Vantage.
+        * `deploy()` - Persist the datasets generated by AutoDataPrep in Teradata Vantage.
+        * `delete_data()` - Deletes the deployed dataset from the Teradata Vantage.
+        * `visualize()` - Generates visualizations to analyze and understand the underlying patterns in the data.
+
+  * ##### teradataml: SQLE Engine Analytic Functions
+    * New Analytics Database Analytic Functions:
+      * `Apriori()`
+      * `NERExtractor()`
+      * `TextMorph()`
+  
+  * ##### teradataml: Functions
+    * `td_range()` - Creates a DataFrame with a specified range of numbers.
+
+  * ##### teradataml DataFrameColumn a.k.a. ColumnExpression
+    * `DataFrameColumn.to_number()` - Function converts a string-like representation of a number to NUMBER type.
+
+* ##### Updates
+  * ###### teradataml: DataFrame function
+    * `DataFrame.agg()`: User can request for different percentiles while running agg function.
+    * New argument `debug` is added to `DataFrame.map_row()`, `DataFrame.map_partition()`, `DataFrame.apply()` and `udf()`. During the execution of these functions, teradataml internally generates scripts, which are garbage collected implicitly. To debug the failures, this argument allows user to control the garbage collection of the script. When set to False (default), script generated is garbage collected, otherwise script is not garbage collected and displays the path to the script, and user is responsible to remove the script if required.
+    * `map_row()`, `map_partition()` and `apply()`
+      * Raises a TeradataMlException, if the Python interpreter major version is different between the Vantage Python environment and the local user environment.
+      * Displays a warning, if `dill` package version is different between the Vantage Python environment and the local user environment. 
+    * `DataFrame.describe()`: Argument `include` is no longer supported.
+    * `assign()` - Optimized SQL query to enhance the performance for consecutive assign calls.
+
+  * ###### teradataml: Context Creation
+    * `create_context()`
+      * Enables user to set the authentication token while creating the connection. This authentication token is required to access services running on Teradata Vantage.
+      * New argument `sql_timeout` is added to specify timeout for SQL statement execution triggered from the current session.
+
+  * ###### teradataml: UAF Functions
+    * Integer type value is now accepted as a valid value for function arguments accepting float type.
+
+  * ###### General functions
+    * `set_auth_token()`
+      * Added argument `kid` to accept the name of the key used while generating `pem_file`.
+      * New keyword argument `auth_mech` accepts the authentication mechanism to be used for generating authentication token.
+      * Basic authentication is now supported as well. New keyword argument `password` accepts password for database user in such case.
+    * `copy_to_sql()` and `read_csv()` support the VECTOR data type.
+
+  * ###### Open Analytics Framework (OpenAF) APIs:
+    * `create_env()`:
+      * Supports creation of conda R environment.
+
+  * ###### teradataml DataFrameColumn a.k.a. ColumnExpression
+    * _String Functions_
+      * `DataFrameColumn.substr()` - Arguments `start_pos` and `length` now accept DataFrameColumn as input. 
+      * `DataFrameColumn.to_char()` - Argument `formatter` now accepts DataFrameColumn as input.
+
+  * ###### teradataml: SQLE Engine Analytic Functions
+    * Updated Analytics Database Analytic Functions:
+      * `SMOTE()` is now supported on 17.20.00.00 as well.
+      * `TextParser()`
+        * New arguments added: `enforce_token_limit`, `delimiter_regex`, `doc_id_column`,
+        `list_positions`, `token_frequency`, `output_by_word`
+
 #### teradataml 20.00.00.04
 * ##### New Features/Functionality
     * ###### teradataml OTF Support:
-      * This release has enabled the support for accessing OTF data from teradataml. 
+      * This release has enabled the support for accessing OTF data from teradataml.
       * User can now create a teradataml DataFrame on OTF table, allowing user to use teradataml functions.
         * Example usage below:
           * Creation of view on OTF/datalake table is not supported. Hence, user has to set `configure.temp_object_type` to `VT` using below-mentioned statement.
@@ -48,7 +217,7 @@ Copyright 2024, Teradata. All Rights Reserved.
                ```
   * ###### teradataml: DataFrame
     * Introduced a new feature 'Exploratory Data Analysis UI' (EDA-UI), which enhances
-      the user experience of teradataml with Jupyter notebook. EDA-UI is displayed by default 
+      the user experience of teradataml with Jupyter notebook. EDA-UI is displayed by default
       when a teradataml DataFrame is printed in the Jupyter notebook.
     * User can control the EDA-UI using a new configuration option `display.enable_ui`.
       It can be disabled by setting `display.enable_ui` to False.
@@ -65,7 +234,7 @@ Copyright 2024, Teradata. All Rights Reserved.
         * `td_lightgbm.load()` - Load the deployed model in the current session.
 
   * ###### FeatureStore
-    * New function `FeatureStore.delete()` is added to drop the Feature Store and corresponding repo from Vantage. 
+    * New function `FeatureStore.delete()` is added to drop the Feature Store and corresponding repo from Vantage.
 
   * ###### Database Utility
     * `db_python_version_diff()` - Identifies the Python interpreter major version difference between the interpreter installed on Vantage vs interpreter on the local user environment.
@@ -78,13 +247,13 @@ Copyright 2024, Teradata. All Rights Reserved.
       * Configuration Options
         * `configure.temp_object_type` - Allows user to choose between creating volatile tables or views for teradataml internal use. By default, teradataml internally creates the views for some of the operations. Now, with new configuration option, user can opt to create Volatile tables instead of views. This provides greater flexibility for users who lack the necessary permissions to create view or need to create views on tables without WITH GRANT permissions.
       * Display Options
-        * `display.enable_ui` - Specifies whether to display exploratory data analysis UI when DataFrame is printed. By default, this option is enabled (True), allowing exploratory data analysis UI to be displayed. When set to False, exploratory data analysis UI is hidden. 
+        * `display.enable_ui` - Specifies whether to display exploratory data analysis UI when DataFrame is printed. By default, this option is enabled (True), allowing exploratory data analysis UI to be displayed. When set to False, exploratory data analysis UI is hidden.
 
 * ##### Updates
   * ###### teradataml: DataFrame function
     * `describe()`
       * New argument added: `pivot`.
-      * When argument `pivot` is set to False, Non-numeric columns are no longer supported for generating statistics. 
+      * When argument `pivot` is set to False, Non-numeric columns are no longer supported for generating statistics.
         Use `CategoricalSummary` and `ColumnSummary`.
     * `fillna()` - Accepts new argument `partition_column` to partition the data and impute null values accordingly.
     * Optimised performance for `DataFrame.plot()`.
@@ -94,9 +263,9 @@ Copyright 2024, Teradata. All Rights Reserved.
   * ###### teradataml: DataFrame Utilities
     * `in_schema()`: New argument `datalake_name` added to accept datalake name.
 
-  * ###### Table Operator 
-    * `Apply()` no longer looks at authentication token by default. Authentication token is now required only if user want to update backend Open Analytics Framework service.
-  
+  * ###### Table Operator
+    * `Apply()` no longer looks at authentication token by default. Authentication token is now required only if user wants to consume Open Analytics Framework REST APIs.
+
   * ###### Hyper Parameter Tuner
     * `GridSearch()` and `RandomSearch()` now displays a message to refer to `get_error_log()` api when model training fails in HPT.
 
@@ -104,7 +273,7 @@ Copyright 2024, Teradata. All Rights Reserved.
     * Configuration Options
       * `configure.indb_install_location`
         Determines the installation location of the In-DB Python package based on the installed RPM version.
-  
+
   * ###### teradataml Context Creation
     * `create_context()` - Enables user to create connection using either parameters set in environment or config file, in addition to previous method. Newly added options help users to hide the sensitive data from the script.
 
@@ -113,7 +282,7 @@ Copyright 2024, Teradata. All Rights Reserved.
 
   * ###### OpensourceML
     * Raises a TeradataMlException, if the Python interpreter major version is different between the Vantage Python environment and the local user environment.
-    * Displays a warning, if specific Python package versions are different between the Vantage Python environment and the local user environment. 
+    * Displays a warning, if specific Python package versions are different between the Vantage Python environment and the local user environment.
 
   * ###### Database Utility
     * `db_list_tables()`: New argument `datalake_name` added to accept datalake name to list tables from.
@@ -131,10 +300,10 @@ Copyright 2024, Teradata. All Rights Reserved.
   * `DataFrame.set_index()` no longer modifies the original DataFrame's index when argument `append` is used.
   * `concat()` function now supports the DataFrame with column name starts with digit or contains special characters or contains reserved keywords.
   * `create_env()` proceeds to install other files even if current file installation fails.
-  * Corrected the error message being raised in `create_env()` when authentication is not set.
+  * Corrected the error message being raised in `create_env()` when authentication token is not set.
   * Added missing argument `charset` for Vantage Analytic Library functions.
   * New argument `seed` is added to `AutoML`, `AutoRegressor` and `AutoClassifier` to ensure consistency on result.
-  * Analytic functions now work even if name of columns for underlying tables is non-ascii characters.
+  * Analytic functions now work even if name of columns for underlying tables has non-ascii characters.
 
 #### teradataml 20.00.00.03
 

teradataml/__init__.py

@@ -63,6 +63,7 @@ from teradataml.opensource import *
 
 # Import AutoML
 from teradataml.automl import AutoML, AutoRegressor, AutoClassifier
+from teradataml.automl.autodataprep import AutoDataPrep
 
 # Import global variable representing session_queryband.
 from teradataml.telemetry_utils.queryband import session_queryband
@@ -72,6 +73,6 @@ session_queryband.configure_queryband_parameters(app_name="TDML", app_version=__
 # Import functions.
 from teradataml.dataframe.functions import *
 
-# Import FeatureStore and VectorStore
+# Import FeatureStore.
 from teradataml.store import *
 

teradataml/_version.py

@@ -1,6 +1,6 @@
 # ##################################################################
 # 
-# Copyright 2024 Teradata. All rights reserved.
+# Copyright 2025 Teradata. All rights reserved.
 # TERADATA CONFIDENTIAL AND TRADE SECRET
 # 
 # Primary Owner: Pankaj Purandare (PankajVinod.Purandare@teradata.com)
@@ -8,4 +8,4 @@
 # 
 # ################################################################## 
 
-version = "20.00.00.04"
+version = "20.00.00.06"

teradataml/analytics/analytic_function_executor.py

@@ -137,6 +137,7 @@ class _AnlyticFunctionExecutor:
             else:
                 py_types = (py_types, additional_valid_types)
 
+
         argument_info = [func_arg_name,
                          func_arg_value,
                          not argument.is_required(),
@@ -489,7 +490,7 @@ class _AnlyticFunctionExecutor:
                 arg_value = self._spl_func_obj._add_square_bracket(arg_value)
 
                 # Check if there are columns with non-ASCII characters.
-                if UtilFuncs._is_ascii(arg_value):
+                if UtilFuncs._is_non_ascii(arg_value):
                     arg_value = UtilFuncs._teradata_quote_arg(arg_value, "\"", False)
                 # Handling special case for Teradata reserved keywords or column names with spaces.
                 # If argument is a string or list of strings, then add quotes to the string.
@@ -1949,12 +1950,9 @@ class _UAFFunctionExecutor(_SQLEFunctionExecutor):
                 # a list and of the required size.
                 if argument.get_match_length_of_arguments():
                     required_length = argument.get_required_length()
-                    if (isinstance(arg_value, list) and len(arg_value) != required_length) or\
-                       (not isinstance(arg_value, list)):
-                        raise TeradataMlException(Messages.get_message(
-                                                  MessageCodes.INVALID_LIST_LENGTH).format(lang_name,
-                                                                                           required_length),
-                                                  MessageCodes.INVALID_LIST_LENGTH)
+
+                    _Validators._valid_list_length(arg_value=arg_value, arg_name=lang_name,
+                                                     required_length=required_length)
 
                 # Perform the checks which are specific to argument(_AnlyFuncArgument) type.
                 # Check lower bound and upper bound for numeric arguments.
@@ -2283,12 +2281,9 @@ class _StoredProcedureExecutor(_UAFFunctionExecutor):
                 # a list and of the required size.
                 if argument.get_match_length_of_arguments():
                     required_length = argument.get_required_length()
-                    if (isinstance(arg_value, list) and len(arg_value) != required_length) or\
-                       (not isinstance(arg_value, list)):
-                        raise TeradataMlException(Messages.get_message(
-                                                  MessageCodes.INVALID_LIST_LENGTH).format(lang_name,
-                                                                                           required_length),
-                                                  MessageCodes.INVALID_LIST_LENGTH)
+
+                    _Validators._valid_list_length(arg_value=arg_value, arg_name=lang_name,
+                                                     required_length=required_length)
 
                 # Perform the checks which are specific to argument(_AnlyFuncArgument) type.
                 # Check lower bound and upper bound for numeric arguments.

teradataml/analytics/json_parser/analytic_functions_argument.py

@@ -1701,6 +1701,10 @@ class _AnlyFuncArgumentUAF(_AnlyFuncArgumentBaseUAF):
             py_type = _Dtypes._anly_json_type_to_python_type(self.__data_type)
             py_types = py_type if isinstance(py_type, tuple) else (py_type,)
 
+        # If argument is float and int is not in the list of valid types, then add int to the list.
+        if float in py_types and int not in py_types:
+            py_types = (int,) + py_types
+            
         # If lists are allowed, add list type also.
         if self.__allows_lists and (list not in py_types):
             py_types = py_types + (list,)

teradataml/analytics/sqle/__init__.py

@@ -5,7 +5,19 @@ from teradataml.analytics.meta_class import _AnalyticFunction
 from teradataml.analytics.meta_class import _common_init, _common_dir
 from teradataml.analytics.json_parser.utils import _get_associated_parent_classes
 
-_sqle_functions = ['ANOVA',
+_sqle_functions = [
+                  'AIAnalyzeSentiment',
+                  'AITextTranslate',
+                  'AIDetectLanguage',
+                  'AITextClassifier',
+                  'AIAskLLM',
+                  'AITextEmbeddings',
+                  'AITextSummarize',
+                  'AIExtractKeyPhrases',
+                  'AIRecognizePIIEntities',
+                  'AIMaskPII',
+                  'AIRecognizeEntities',
+                  'ANOVA',
                   'Antiselect',
                   'Attribution',
                   'BincodeFit',
@@ -99,6 +111,9 @@ _sqle_functions = ['ANOVA',
                   'HNSW',
                   'HNSWPredict',
                   'HNSWSummary',
+                  'TextMorph',
+                  'NERExtractor',
+                  'Apriori'
                   ]
 
 for func in _sqle_functions:

teradataml/analytics/utils.py

@@ -411,6 +411,9 @@ class FuncSpecialCaseHandler():
         self._single_quote_arg = lambda arg_value: "'{0}'".format(arg_value)
         # Quote "arg_value" when value is 'NONE'.
         self._single_quote_arg_value_NONE = lambda arg_value: "'{0}'".format(arg_value) if arg_value == 'NONE' else arg_value
+        # Remove quotes from "arg_value".
+        self._remove_quotes = lambda arg_value, *args, **kwargs: arg_value.replace("'", "") if isinstance(arg_value, str) else arg_value
+        
 
         # Initialize special function handle dictionary.
         self.__handlers = {"Antiselect": {"exclude": self._enclose_square_brackets_add_quote},
@@ -455,9 +458,65 @@ class FuncSpecialCaseHandler():
                                       "mode": self._single_quote_arg},
                            "Matrix2Image": {"type": self._single_quote_arg,
                                             "colormap": self._single_quote_arg
-                                           }
+                                           },
+                           "AIAnalyzeSentiment": {"authorization": self._remove_quotes},
+                           "AITextTranslate": {"authorization": self._remove_quotes},
+                           "AITextSummarize": {"authorization": self._remove_quotes},
+                           "AI_TextEmbeddings": {"authorization": self._remove_quotes},
+                           "AITextClassifier": {"authorization": self._remove_quotes},
+                           "AIRecognizePIIEntities": {"authorization": self._remove_quotes},
+                           "AIRecognizeEntities": {"authorization": self._remove_quotes},
+                           "AIMaskPII:": {"authorization": self._remove_quotes},
+                           "AIExtractKeyPhrases": {"authorization": self._remove_quotes},
+                           "AIDetectLanguage": {"authorization": self._remove_quotes},
+                           "AIAskLLM": {"authorization": self._remove_quotes},
+                           "DataikuPredict": {"accumulate": self._handle_byom_args},
+                           "DataRobotPredict": {"accumulate": self._handle_byom_args},
+                           "H2OPredict": {"accumulate": self._handle_byom_args},
+                           "ONNXEmbeddings": {"accumulate": self._handle_byom_args},
+                           "ONNXPredict": {"accumulate": self._handle_byom_args},
+                           "PMMLPredict": {"accumulate": self._handle_byom_args}
                            }
 
+    def _handle_byom_args(self, arg_value, *args, **kwargs):
+        """
+        DESCRIPTION:
+            Function to handle the special function arguments for BYOM functions.
+            Special character(s) are required to be in single quote for BYOM
+            functions. For other SQLE functions, special character(s) to be in double
+            quote. Hence removing double quotes and adding single quotes for
+            special character(s) in the argument value.
+
+        PARAMETERS:
+            arg_value
+                Required Argument.
+                Specifies the argument value to handle special function arguments.
+                Types: str or list of str(s)
+
+        RETURNS:
+            str
+
+        RAISES:
+            None
+
+        EXAMPLES:
+            # Example 1: Handle the special function arguments for BYOM functions.
+            >>> self._handle_byom_args('"alice.deauth"')
+            # result: 'alice.deauth'
+        """
+        arg_values = []
+        if isinstance(arg_value, list):
+            arg_values = (val.strip('"') for val in arg_value)
+            return ", ".join(["'{}'".format(val) for val in arg_values])
+
+        if isinstance(arg_value, str):
+            # If argument value is a string, remove quotes.
+            arg_value = arg_value.strip('"')
+            return "'{}'".format(arg_value)
+
+        return arg_value
+
+
     # Setter method for argument.
     def set_arg_name(self, argument):
         """