teradataml 20.0.0.2__py3-none-any.whl → 20.0.0.4__py3-none-any.whl

{teradataml-20.0.0.2.dist-info → teradataml-20.0.0.4.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: teradataml
-Version: 20.0.0.2
+Version: 20.0.0.4
 Summary: Teradata Vantage Python package for Advanced Analytics
 Home-page: http://www.teradata.com/
 Author: Teradata Corporation
@@ -17,8 +17,8 @@ Classifier: Topic :: Database :: Front-Ends
 Classifier: License :: Other/Proprietary License
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
-Requires-Dist: teradatasql (>=17.10.0.11)
-Requires-Dist: teradatasqlalchemy (>=20.0.0.2)
+Requires-Dist: teradatasql (>=20.0.0.19)
+Requires-Dist: teradatasqlalchemy (>=20.0.0.4)
 Requires-Dist: pandas (>=0.22)
 Requires-Dist: psutil
 Requires-Dist: requests (>=2.25.1)
@@ -28,6 +28,9 @@ Requires-Dist: imbalanced-learn (>=0.8.0)
 Requires-Dist: pyjwt (>=2.8.0)
 Requires-Dist: cryptography (>=42.0.5)
 Requires-Dist: sqlalchemy (>=2.0)
+Requires-Dist: lightgbm (>=3.3.3)
+Requires-Dist: python-dotenv
+Requires-Dist: teradatamlwidgets (>=20.0.0.5)
 
 ## Teradata Python package for Advanced Analytics.
 
@@ -47,6 +50,306 @@ Copyright 2024, Teradata. All Rights Reserved.
 * [License](#license)
 
 ## Release Notes:
+
+#### teradataml 20.00.00.04
+* ##### New Features/Functionality
+    * ###### teradataml OTF Support:
+      * This release has enabled the support for accessing OTF data from teradataml. 
+      * User can now create a teradataml DataFrame on OTF table, allowing user to use teradataml functions.
+        * Example usage below:
+          * Creation of view on OTF/datalake table is not supported. Hence, user has to set `configure.temp_object_type` to `VT` using below-mentioned statement.
+            ```configure.temp_object_type = "VT"```
+          * User needs to provide additional information about datalake while creating the DataFrame. There are two approaches to provide datalake information
+            * Approach 1: Using `in_schema()`
+              ```
+              >>> from teradataml.dataframe.dataframe import in_schema
+              # Create an in_schema object to privide additional information about datalake.
+              >>> in_schema_tbl = in_schema(schema_name="datalake_db",
+              ...                           table_name="datalake_table_name",
+              ...                           datalake_name="datalake")
+              >>> otf_df = DataFrame(in_schema_tbl)
+                ```
+            * Approach 2: Using `DataFrame.from_table()`
+              ```
+               >>> otf_df = DataFrame.from_table(table_name = "datalake_table_name",
+               ...                               schema_name="datalake_db",
+               ...                               datalake_name="datalake")
+              ```
+          * Once this DataFrame is created, users can use any DataFrame method or analytics features/functionality from teradataml with it. Visit Limitations and considerations section in _Teradata Python Package User Guide_ to check the supportability.
+            *  Note: All further operations create volatile tables in local database.
+               ```
+                >>> new_df = otf_df.assign(new_col=otf_df.existing_col*2)
+               ```
+  * ###### teradataml: DataFrame
+    * Introduced a new feature 'Exploratory Data Analysis UI' (EDA-UI), which enhances
+      the user experience of teradataml with Jupyter notebook. EDA-UI is displayed by default 
+      when a teradataml DataFrame is printed in the Jupyter notebook.
+    * User can control the EDA-UI using a new configuration option `display.enable_ui`.
+      It can be disabled by setting `display.enable_ui` to False.
+    * New Function
+      * `get_output()` is added to get the result of Analytic function when executed from EDA UI.
+
+  * ###### OpensourceML
+    * `td_lightgbm` - A teradataml OpenSourceML module
+      * `deploy()` - User can now deploy the models created by lightgbm `Booster` and `sklearn` modules. Deploying the model stores the model in Vantage for future use with `td_lightgbm`.
+        * `td_lightgbm.deploy()` - Deploy the lightgbm `Booster` or any `scikit-learn` model trained outside Vantage.
+        * `td_lightgbm.train().deploy()` - Deploys the lightgbm `Booster` object trained within Vantage.
+        * `td_lightgbm.<sklearn_class>().deploy()` - Deploys lightgbm's sklearn class object created/trained within Vantage.
+      * `load()` - User can load the deployed models back in the current session. This allows user to use the lightgbm functions with the `td_lightgbm` module.
+        * `td_lightgbm.load()` - Load the deployed model in the current session.
+
+  * ###### FeatureStore
+    * New function `FeatureStore.delete()` is added to drop the Feature Store and corresponding repo from Vantage. 
+
+  * ###### Database Utility
+    * `db_python_version_diff()` - Identifies the Python interpreter major version difference between the interpreter installed on Vantage vs interpreter on the local user environment.
+    * `db_python_package_version_diff()` - Identifies the Python package version difference between the packages installed on Vantage vs the local user environment.
+
+  * ###### BYOM Function
+    * `ONNXEmbeddings()` - Calculate embeddings values in Vantage using an embeddings model that has been created outside Vantage and stored in ONNX format.
+
+  * ###### teradataml Options
+      * Configuration Options
+        * `configure.temp_object_type` - Allows user to choose between creating volatile tables or views for teradataml internal use. By default, teradataml internally creates the views for some of the operations. Now, with new configuration option, user can opt to create Volatile tables instead of views. This provides greater flexibility for users who lack the necessary permissions to create view or need to create views on tables without WITH GRANT permissions.
+      * Display Options
+        * `display.enable_ui` - Specifies whether to display exploratory data analysis UI when DataFrame is printed. By default, this option is enabled (True), allowing exploratory data analysis UI to be displayed. When set to False, exploratory data analysis UI is hidden. 
+
+* ##### Updates
+  * ###### teradataml: DataFrame function
+    * `describe()`
+      * New argument added: `pivot`.
+      * When argument `pivot` is set to False, Non-numeric columns are no longer supported for generating statistics. 
+        Use `CategoricalSummary` and `ColumnSummary`.
+    * `fillna()` - Accepts new argument `partition_column` to partition the data and impute null values accordingly.
+    * Optimised performance for `DataFrame.plot()`.
+      * `DataFrame.plot()` will not regenerate the image when run more than once with same arguments.
+    * `DataFrame.from_table()`: New argument `datalake_name` added to accept datalake name while creating DataFrame on datalake table.
+
+  * ###### teradataml: DataFrame Utilities
+    * `in_schema()`: New argument `datalake_name` added to accept datalake name.
+
+  * ###### Table Operator 
+    * `Apply()` no longer looks at authentication token by default. Authentication token is now required only if user want to update backend Open Analytics Framework service.
+
+  * ###### Hyper Parameter Tuner
+    * `GridSearch()` and `RandomSearch()` now displays a message to refer to `get_error_log()` api when model training fails in HPT.
+
+  * ###### teradataml Options
+    * Configuration Options
+      * `configure.indb_install_location`
+        Determines the installation location of the In-DB Python package based on the installed RPM version.
+
+  * ###### teradataml Context Creation
+    * `create_context()` - Enables user to create connection using either parameters set in environment or config file, in addition to previous method. Newly added options help users to hide the sensitive data from the script.
+
+  * ###### Open Analytics Framework
+    * Enhanced the `create_env()` to display a message when an invalid base_env is passed, informing users that the default base_env is being used.
+
+  * ###### OpensourceML
+    * Raises a TeradataMlException, if the Python interpreter major version is different between the Vantage Python environment and the local user environment.
+    * Displays a warning, if specific Python package versions are different between the Vantage Python environment and the local user environment. 
+
+  * ###### Database Utility
+    * `db_list_tables()`: New argument `datalake_name` added to accept datalake name to list tables from.
+    * `db_drop_table()`:
+      * New argument `datalake_name` added to accept datalake name to drop tables from.
+      * New argument `purge` added to specify whether to use `PURGE ALL` or `NO PURGE` clause while dropping table.
+
+* ##### Bug Fixes
+  * `td_lightgbm` OpensourceML module: In multi model case, `td_lightgbm.Dataset().add_features_from()` function should add features of one partition in first Dataset to features of the same partition in second Dataset. This is not the case before and this function fails. Fixed this now.
+  * Fixed a minor bug in the `Shap()` and converted argument `training_method` to required argument.
+  * Fixed PCA-related warnings in `AutoML`.
+  * `AutoML` no longer fails when data with all categorical columns are provided.
+  * Fixed `AutoML` issue with upsampling method.
+  * Excluded the identifier column from outlier processing in `AutoML`.
+  * `DataFrame.set_index()` no longer modifies the original DataFrame's index when argument `append` is used.
+  * `concat()` function now supports the DataFrame with column name starts with digit or contains special characters or contains reserved keywords.
+  * `create_env()` proceeds to install other files even if current file installation fails.
+  * Corrected the error message being raised in `create_env()` when authentication is not set.
+  * Added missing argument `charset` for Vantage Analytic Library functions.
+  * New argument `seed` is added to `AutoML`, `AutoRegressor` and `AutoClassifier` to ensure consistency on result.
+  * Analytic functions now work even if name of columns for underlying tables is non-ascii characters.
+
+#### teradataml 20.00.00.03
+
+* teradataml no longer supports setting the `auth_token` using `set_config_params()`. Users should use `set_auth_token()` to set the token. 
+
+* ##### New Features/Functionality
+  * ###### teradataml: DataFrame
+    * New Function
+      * `alias()` - Creates a DataFrame with alias name.
+    * New Properties
+      * `db_object_name` - Get the underlying database object name, on which DataFrame is created.
+
+  * ###### teradataml: GeoDataFrame
+    * New Function
+      * `alias()` - Creates a GeoDataFrame with alias name.
+
+  * ###### teradataml: DataFrameColumn a.k.a. ColumnExpression
+    * _Arithmetic Functions_
+      * `DataFrameColumn.isnan()` - Function evaluates expression to determine if the floating-point
+                                    argument is a NaN (Not-a-Number) value.
+      * `DataFrameColumn.isinf()` - Function evaluates expression to determine if the floating-point
+                                    argument is an infinite number.
+      * `DataFrameColumn.isfinite()` - Function evaluates expression to determine if it is a finite
+                                       floating value.
+
+  * ###### FeatureStore - handles feature management within the Vantage environment
+    * FeatureStore Components
+      * Feature - Represents a feature which is used in ML Modeling. 
+      * Entity - Represents the columns which serves as uniqueness for the data used in ML Modeling. 
+      * DataSource - Represents the source of Data.
+      * FeatureGroup - Collection of Feature, Entity and DataSource.
+        * Methods
+          * `apply()` - Adds Feature, Entity, DataSource to a FeatureGroup.
+          * `from_DataFrame()` - Creates a FeatureGroup from teradataml DataFrame.
+          * `from_query()` - Creates a FeatureGroup using a SQL query.
+          * `remove()` - Removes Feature, Entity, or DataSource from a FeatureGroup.
+          * `reset_labels()` - Removes the labels assigned to the FeatureGroup, that are set using `set_labels()`.
+          * `set_labels()` - Sets the Features as labels for a FeatureGroup.
+        * Properties
+          * `features` - Get the features of a FeatureGroup.
+          * `labels` - Get the labels of FeatureGroup.
+    * FeatureStore 
+      * Methods
+        * `apply()` - Adds Feature, Entity, DataSource, FeatureGroup to FeatureStore.
+        * `archive_data_source()` - Archives a specified DataSource from a FeatureStore.
+        * `archive_entity()` - Archives a specified Entity from a FeatureStore.
+        * `archive_feature()` - Archives a specified Feature from a FeatureStore.
+        * `archive_feature_group()` - Archives a specified FeatureGroup from a FeatureStore. Method archives underlying Feature, Entity, DataSource also.
+        * `delete_data_source()` - Deletes an archived DataSource.
+        * `delete_entity()` - Deletes an archived Entity.
+        * `delete_feature()` - Deletes an archived Feature.
+        * `delete_feature_group()` - Deletes an archived FeatureGroup. 
+        * `get_data_source()` - Get the DataSources associated with FeatureStore.
+        * `get_dataset()` - Get the teradataml DataFrame based on Features, Entities and DataSource from FeatureGroup.
+        * `get_entity()` - Get the Entity associated with FeatureStore.
+        * `get_feature()` - Get the Feature associated with FeatureStore.
+        * `get_feature_group()` - Get the FeatureGroup associated with FeatureStore.
+        * `list_data_sources()` - List DataSources.
+        * `list_entities()` - List Entities.
+        * `list_feature_groups()` - List FeatureGroups.
+        * `list_features()` - List Features.
+        * `list_repos()` - List available repos which are configured for FeatureStore. 
+        * `repair()` - Repairs the underlying FeatureStore schema on database.  
+        * `set_features_active()` - Marks the Features as active.
+        * `set_features_inactive()` - Marks the Features as inactive.
+        * `setup()` - Setup the FeatureStore for a repo.
+      * Property
+        * `repo` - Property for FeatureStore repo.
+        * `grant` - Property to Grant access on FeatureStore to user.
+        * `revoke` - Property to Revoke access on FeatureStore from user.
+
+  * ###### teradataml: Table Operator Functions
+    * `Image2Matrix()` - Converts an image into a matrix.
+
+  * ###### teradataml: SQLE Engine Analytic Functions
+    * New Analytics Database Analytic Functions:
+      * `CFilter()`
+      * `NaiveBayes()`
+      * `TDNaiveBayesPredict()`
+      * `Shap()`
+      * `SMOTE()`
+
+    * ###### teradataml: Unbounded Array Framework (UAF) Functions
+      * New Unbounded Array Framework(UAF) Functions:
+        * `CopyArt()`
+
+  * ###### General functions
+    * Vantage File Management Functions
+      * `list_files()` - List the installed files in Database.
+
+  * ###### OpensourceML: LightGBM
+    * teradataml adds support for lightGBM package through `OpensourceML` (`OpenML`) feature.
+      The following functionality is added in the current release:
+      * `td_lightgbm` - Interface object to run lightgbm functions and classes through Teradata Vantage.
+      Example usage below:
+        ```
+        from teradataml import td_lightgbm, DataFrame
+
+        df_train = DataFrame("multi_model_classification")
+
+        feature_columns = ["col1", "col2", "col3", "col4"]
+        label_columns = ["label"]
+        part_columns = ["partition_column_1", "partition_column_2"]
+
+        df_x = df_train.select(feature_columns)
+        df_y = df_train.select(label_columns)
+
+        # Dataset creation.
+        # Single model case.
+        obj_s = td_lightgbm.Dataset(df_x, df_y, silent=True, free_raw_data=False)
+
+        # Multi model case.
+        obj_m = td_lightgbm.Dataset(df_x, df_y, free_raw_data=False, partition_columns=part_columns)
+        obj_m_v = td_lightgbm.Dataset(df_x, df_y, free_raw_data=False, partition_columns=part_columns)
+
+        ## Model training.
+        # Single model case.
+        opt = td_lightgbm.train(params={}, train_set = obj_s, num_boost_round=30)
+
+        opt.predict(data=df_x, num_iteration=20, pred_contrib=True)
+
+        # Multi model case.
+        opt = td_lightgbm.train(params={}, train_set = obj_m, num_boost_round=30,
+                                callbacks=[td_lightgbm.record_evaluation(rec)],
+                                valid_sets=[obj_m_v, obj_m_v])
+
+        # Passing `label` argument to get it returned in output DataFrame.
+        opt.predict(data=df_x, label=df_y, num_iteration=20)
+
+        ```
+      * Added support for accessing scikit-learn APIs using exposed inteface object `td_lightgbm`.
+
+    Refer Teradata Python Package User Guide for more details of this feature, arguments, usage, examples and supportability in Vantage.
+
+  * ###### teradataml: Functions
+    * `register()` - Registers a user defined function (UDF).
+    * `call_udf()` - Calls a registered user defined function (UDF) and returns ColumnExpression.
+    * `list_udfs()` - List all the UDFs registered using 'register()' function.
+    * `deregister()` - Deregisters a user defined function (UDF).
+
+  * ###### teradataml: Options
+    * Configuration Options
+      * `table_operator` - Specifies the name of table operator.
+
+* ##### Updates
+  * ###### General functions
+    * `set_auth_token()` - Added `base_url` parameter which accepts the CCP url. 
+                           'ues_url' will be deprecated in future and users
+                           will need to specify 'base_url' instead.
+
+  * ###### teradataml: DataFrame function
+     * `join()`
+       * Now supports compound ColumExpression having more than one binary operator in `on` argument.
+       * Now supports ColumExpression containing FunctionExpression(s) in `on` argument.
+       * self-join now expects aliased DataFrame in `other` argument.
+
+  * ###### teradataml: GeoDataFrame function
+     * `join()`
+       * Now supports compound ColumExpression having more than one binary operator in `on` argument.
+       * Now supports ColumExpression containing FunctionExpression(s) in `on` argument.
+       * self-join now expects aliased DataFrame in `other` argument.
+
+  * ###### teradataml: Unbounded Array Framework (UAF) Functions
+    * `SAX()` - Default value added for `window_size` and `output_frequency`.
+    * `DickeyFuller()`
+      * Supports TDAnalyticResult as input.
+      * Default value added for `max_lags`.
+      * Removed parameter `drift_trend_formula`.
+      * Updated permitted values for `algorithm`.
+
+  * ##### teradataml: AutoML
+    * `AutoML`, `AutoRegressor` and `AutoClassifier`
+      * Now supports DECIMAL datatype as input.
+
+  * ##### teradataml: SQLE Engine Analytic Functions
+    * `TextParser()`
+      * Argument name `covert_to_lowercase` changed to `convert_to_lowercase`.
+
+* ##### Bug Fixes
+  * `db_list_tables()` now returns correct results when '%' is used.
+
 #### teradataml 20.00.00.02
 
 * teradataml will no longer be supported with SQLAlchemy < 2.0.
@@ -115,6 +418,10 @@ Copyright 2024, Teradata. All Rights Reserved.
         * `ues_url`
         * `auth_token`
 
+  * #### teradata DataFrame
+    * `to_pandas()` - Function returns the pandas dataframe with Decimal columns types as float instead of object.
+                      If user want datatype to be object, set argument `coerce_float` to False.
+
   * ###### Database Utility
       * `list_td_reserved_keywords()` - Accepts a list of strings as argument.
 
@@ -133,7 +440,7 @@ Copyright 2024, Teradata. All Rights Reserved.
 * ##### Bug Fixes
   * KNN `predict()` function can now predict on test data which does not contain target column.
   * Metrics functions are supported on the Lake system.
-  * The following OpensourceML functions from different sklearn modules are fixed.
+  * The following OpensourceML functions from different sklearn modules in single model case are fixed.
     * `sklearn.ensemble`:
       * ExtraTreesClassifier - `apply()`
       * ExtraTreesRegressor - `apply()`
@@ -146,12 +453,21 @@ Copyright 2024, Teradata. All Rights Reserved.
       * Nystroem - `transform()`, `fit_transform()`
       * PolynomialCountSketch - `transform()`, `fit_transform()`
       * RBFSampler - `transform()`, `fit_transform()`
-    * `sklearn.neighbours`:
+    * `sklearn.neighbors`:
       * KNeighborsTransformer - `transform()`, `fit_transform()`
       * RadiusNeighborsTransformer - `transform()`, `fit_transform()`
     * `sklearn.preprocessing`:
       * KernelCenterer - `transform()`
       * OneHotEncoder - `transform()`, `inverse_transform()`
+  * The following OpensourceML functions from different sklearn modules in multi model case are fixed.
+    * `sklearn.feature_selection`:
+      * SelectFpr - `transform()`, `fit_transform()`, `inverse_transform()`
+      * SelectFdr - `transform()`, `fit_transform()`, `inverse_transform()`
+      * SelectFromModel - `transform()`, `fit_transform()`, `inverse_transform()`
+      * SelectFwe - `transform()`, `fit_transform()`, `inverse_transform()`
+      * RFECV - `transform()`, `fit_transform()`, `inverse_transform()`
+    * `sklearn.clustering`:
+      * Birch - `transform()`, `fit_transform()`
   * OpensourceML returns teradataml objects for model attributes and functions instead of sklearn
     objects so that the user can perform further operations like `score()`, `predict()` etc on top
     of the returned objects.