snowflake-ml-python 1.7.4__tar.gz → 1.7.5__tar.gz

{snowflake_ml_python-1.7.4 → snowflake_ml_python-1.7.5}/CHANGELOG.md

@@ -1,27 +1,53 @@
 # Release History
 
-## 1.7.4
+## 1.7.5
+
+- Support Python 3.12.
+- Explainability: Support native and snowml sklearn pipeline
+
+### Bug Fixes
+
+- Registry: Fixed a compatibility issue when using `snowflake-ml-python` 1.7.0 or greater to save a `tensorflow.keras`
+  model with `keras` 2.x, if `relax_version` is set or default to True, and newer version of `snowflake-ml-python`
+  is available in Snowflake Anaconda Channel, model could not be run in Snowflake. If you have such model, you could
+  use the latest version of `snowflake-ml-python` and call `ModelVersion.load` to load it back, and re-log it.
+  Alternatively, you can prevent this issue by setting `relax_version=False` when saving the model.
+- Registry: Removed the validation that disallows data that does not have non-null values being passed to
+  `ModelVersion.run`.
+- ML Job (PrPr): No longer require CREATE STAGE privilege if `stage_name` points to an existing stage
+- ML Job (PrPr): Fixed a bug causing some payload source and entrypoint path
+  combinations to be erroneously rejected with
+  `ValueError(f"{self.entrypoint} must be a subpath of {self.source}")`
+- ML Job (PrPr): Fixed a bug in Ray cluster startup config which caused certain Runtime APIs to fail
+
+### Behavior Change
+
+### New Features
+
+- Registry: Added support for handling Hugging Face model configurations with auto-mapping functionality.
+- Registry: Added support for `keras` 3.x model with `tensorflow` and `pytorch` backend
+- ML Job (PrPr): Support any serializable (pickleable) argument for `@remote` decorated functions
+
+## 1.7.4 (01-28-2025)
 
 - FileSet: The `snowflake.ml.fileset.FileSet` has been deprecated and will be removed in a future version.
   Use [snowflake.ml.dataset.Dataset](https://docs.snowflake.com/en/developer-guide/snowflake-ml/dataset) and
   [snowflake.ml.data.DataConnector](https://docs.snowflake.com/en/developer-guide/snowpark-ml/reference/latest/api/data/snowflake.ml.data.data_connector.DataConnector)
   instead.
+- Registry: `ModelVersion.run` on a service would require redeploying the service once account opts into nested function.
 
 ### Bug Fixes
 
 - Registry: Fixed an issue that the hugging face pipeline is loaded using incorrect dtype.
 - Registry: Fixed an issue that only 1 row is used when infer the model signature in the modeling model.
 
-### Behavior Changes
-
-- Registry: `ModelVersion.run` on a service would require redeploying the service once account opts into nested function.
-
 ### New Features
 
 - Add new `snowflake.ml.jobs` preview API for running headless workloads on SPCS using
   [Container Runtime for ML](https://docs.snowflake.com/en/developer-guide/snowflake-ml/container-runtime-ml)
 - Added `guardrails` option to Cortex `complete` function, enabling
   [Cortex Guard](https://docs.snowflake.com/en/user-guide/snowflake-cortex/llm-functions#cortex-guard) support
+- Model Monitoring: Expose Model Monitoring Python API by default.
 
 ## 1.7.3 (2025-01-08)
 
@@ -29,6 +55,7 @@
 - Bumped the requirements of `fsspec` and `s3fs` to `>=2024.6.1,<2026`
 - Bumped the requirement of `mlflow` to `>=2.16.0, <3`
 - Registry: Support 500+ features for model registry
+- Feature Store: Add support for `cluster_by` for feature views.
 
 ### Bug Fixes
 
@@ -146,6 +173,10 @@ class ExamplePipelineModel(custom_model.CustomModel):
 - Data Connector: Add the option of passing a `None` sized batch to `to_torch_dataset` for better
 interoperability with PyTorch DataLoader.
 - Model Registry: Support [pandas.CategoricalDtype](https://pandas.pydata.org/docs/reference/api/pandas.CategoricalDtype.html#pandas-categoricaldtype)
+  - Limitations:
+    - The native categorical data handling handling by XGBoost using `enable_categorical=True` is not supported.
+    Instead please use [`sklearn.pipeline`](https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html)
+    to preprocess the categorical datatype and log the pipeline with the XGBoost model.
 - Registry: It is now possible to pass `signatures` and `sample_input_data` at the same time to capture background
 data from explainablity and data lineage.
 

{snowflake_ml_python-1.7.4/snowflake_ml_python.egg-info → snowflake_ml_python-1.7.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: snowflake-ml-python
-Version: 1.7.4
+Version: 1.7.5
 Summary: The machine learning client library that is used for interacting with Snowflake to build machine learning solutions.
 Author-email: "Snowflake, Inc" <support@snowflake.com>
 License: 
@@ -223,13 +223,14 @@ Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Database
 Classifier: Topic :: Software Development
 Classifier: Topic :: Software Development :: Libraries
 Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Topic :: Scientific/Engineering :: Information Analysis
-Requires-Python: <3.12,>=3.9
+Requires-Python: <3.13,>=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
 Requires-Dist: absl-py<2,>=0.15
@@ -257,39 +258,40 @@ Requires-Dist: typing-extensions<5,>=4.1.0
 Requires-Dist: xgboost<3,>=1.7.3
 Provides-Extra: all
 Requires-Dist: catboost<2,>=1.2.0; extra == "all"
-Requires-Dist: huggingface_hub<0.26; extra == "all"
+Requires-Dist: keras<4,>=2.0.0; extra == "all"
 Requires-Dist: lightgbm<5,>=4.1.0; extra == "all"
 Requires-Dist: mlflow<3,>=2.16.0; extra == "all"
-Requires-Dist: peft<1,>=0.5.0; extra == "all"
-Requires-Dist: sentence-transformers<3,>=2.2.2; extra == "all"
-Requires-Dist: sentencepiece<1,>=0.1.95; extra == "all"
+Requires-Dist: sentence-transformers<3,>=2.7.0; extra == "all"
+Requires-Dist: sentencepiece<0.2.0,>=0.1.95; extra == "all"
 Requires-Dist: shap<1,>=0.46.0; extra == "all"
-Requires-Dist: tensorflow<3,>=2.12.0; extra == "all"
-Requires-Dist: tokenizers<1,>=0.10; extra == "all"
-Requires-Dist: torch<2.3.0,>=2.0.1; extra == "all"
+Requires-Dist: tensorflow<3,>=2.17.0; extra == "all"
+Requires-Dist: tokenizers<1,>=0.15.1; extra == "all"
+Requires-Dist: torch<3,>=2.0.1; extra == "all"
 Requires-Dist: torchdata<1,>=0.4; extra == "all"
-Requires-Dist: transformers<5,>=4.32.1; extra == "all"
+Requires-Dist: transformers<5,>=4.37.2; extra == "all"
 Provides-Extra: catboost
 Requires-Dist: catboost<2,>=1.2.0; extra == "catboost"
+Provides-Extra: keras
+Requires-Dist: keras<4,>=2.0.0; extra == "keras"
+Requires-Dist: tensorflow<3,>=2.17.0; extra == "keras"
+Requires-Dist: torch<3,>=2.0.1; extra == "keras"
 Provides-Extra: lightgbm
 Requires-Dist: lightgbm<5,>=4.1.0; extra == "lightgbm"
-Provides-Extra: llm
-Requires-Dist: peft<1,>=0.5.0; extra == "llm"
 Provides-Extra: mlflow
 Requires-Dist: mlflow<3,>=2.16.0; extra == "mlflow"
 Provides-Extra: shap
 Requires-Dist: shap<1,>=0.46.0; extra == "shap"
 Provides-Extra: tensorflow
-Requires-Dist: tensorflow<3,>=2.12.0; extra == "tensorflow"
+Requires-Dist: tensorflow<3,>=2.17.0; extra == "tensorflow"
 Provides-Extra: torch
-Requires-Dist: torch<2.3.0,>=2.0.1; extra == "torch"
+Requires-Dist: torch<3,>=2.0.1; extra == "torch"
 Requires-Dist: torchdata<1,>=0.4; extra == "torch"
 Provides-Extra: transformers
-Requires-Dist: huggingface_hub<0.26; extra == "transformers"
-Requires-Dist: sentence-transformers<3,>=2.2.2; extra == "transformers"
-Requires-Dist: sentencepiece<1,>=0.1.95; extra == "transformers"
-Requires-Dist: tokenizers<1,>=0.10; extra == "transformers"
-Requires-Dist: transformers<5,>=4.32.1; extra == "transformers"
+Requires-Dist: sentence-transformers<3,>=2.7.0; extra == "transformers"
+Requires-Dist: sentencepiece<0.2.0,>=0.1.95; extra == "transformers"
+Requires-Dist: tokenizers<1,>=0.15.1; extra == "transformers"
+Requires-Dist: torch<3,>=2.0.1; extra == "transformers"
+Requires-Dist: transformers<5,>=4.37.2; extra == "transformers"
 
 # Snowpark ML
 
@@ -346,7 +348,7 @@ If you don't have a Snowflake account yet, you can [sign up for a 30-day free tr
 Follow the [installation instructions](https://docs.snowflake.com/en/developer-guide/snowpark-ml/index#installing-snowpark-ml)
 in the Snowflake documentation.
 
-Python versions 3.9 to 3.11 are supported. You can use [miniconda](https://docs.conda.io/en/latest/miniconda.html) or
+Python versions 3.9 to 3.12 are supported. You can use [miniconda](https://docs.conda.io/en/latest/miniconda.html) or
 [anaconda](https://www.anaconda.com/) to create a Conda environment (recommended),
 or [virtualenv](https://docs.python.org/3/tutorial/venv.html) to create a virtual environment.
 
@@ -399,28 +401,54 @@ NOTE: Version 1.7.0 is used as example here. Please choose the the latest versio
 
 # Release History
 
-## 1.7.4
+## 1.7.5
+
+- Support Python 3.12.
+- Explainability: Support native and snowml sklearn pipeline
+
+### Bug Fixes
+
+- Registry: Fixed a compatibility issue when using `snowflake-ml-python` 1.7.0 or greater to save a `tensorflow.keras`
+  model with `keras` 2.x, if `relax_version` is set or default to True, and newer version of `snowflake-ml-python`
+  is available in Snowflake Anaconda Channel, model could not be run in Snowflake. If you have such model, you could
+  use the latest version of `snowflake-ml-python` and call `ModelVersion.load` to load it back, and re-log it.
+  Alternatively, you can prevent this issue by setting `relax_version=False` when saving the model.
+- Registry: Removed the validation that disallows data that does not have non-null values being passed to
+  `ModelVersion.run`.
+- ML Job (PrPr): No longer require CREATE STAGE privilege if `stage_name` points to an existing stage
+- ML Job (PrPr): Fixed a bug causing some payload source and entrypoint path
+  combinations to be erroneously rejected with
+  `ValueError(f"{self.entrypoint} must be a subpath of {self.source}")`
+- ML Job (PrPr): Fixed a bug in Ray cluster startup config which caused certain Runtime APIs to fail
+
+### Behavior Change
+
+### New Features
+
+- Registry: Added support for handling Hugging Face model configurations with auto-mapping functionality.
+- Registry: Added support for `keras` 3.x model with `tensorflow` and `pytorch` backend
+- ML Job (PrPr): Support any serializable (pickleable) argument for `@remote` decorated functions
+
+## 1.7.4 (01-28-2025)
 
 - FileSet: The `snowflake.ml.fileset.FileSet` has been deprecated and will be removed in a future version.
   Use [snowflake.ml.dataset.Dataset](https://docs.snowflake.com/en/developer-guide/snowflake-ml/dataset) and
   [snowflake.ml.data.DataConnector](https://docs.snowflake.com/en/developer-guide/snowpark-ml/reference/latest/api/data/snowflake.ml.data.data_connector.DataConnector)
   instead.
+- Registry: `ModelVersion.run` on a service would require redeploying the service once account opts into nested function.
 
 ### Bug Fixes
 
 - Registry: Fixed an issue that the hugging face pipeline is loaded using incorrect dtype.
 - Registry: Fixed an issue that only 1 row is used when infer the model signature in the modeling model.
 
-### Behavior Changes
-
-- Registry: `ModelVersion.run` on a service would require redeploying the service once account opts into nested function.
-
 ### New Features
 
 - Add new `snowflake.ml.jobs` preview API for running headless workloads on SPCS using
   [Container Runtime for ML](https://docs.snowflake.com/en/developer-guide/snowflake-ml/container-runtime-ml)
 - Added `guardrails` option to Cortex `complete` function, enabling
   [Cortex Guard](https://docs.snowflake.com/en/user-guide/snowflake-cortex/llm-functions#cortex-guard) support
+- Model Monitoring: Expose Model Monitoring Python API by default.
 
 ## 1.7.3 (2025-01-08)
 
@@ -428,6 +456,7 @@ NOTE: Version 1.7.0 is used as example here. Please choose the the latest versio
 - Bumped the requirements of `fsspec` and `s3fs` to `>=2024.6.1,<2026`
 - Bumped the requirement of `mlflow` to `>=2.16.0, <3`
 - Registry: Support 500+ features for model registry
+- Feature Store: Add support for `cluster_by` for feature views.
 
 ### Bug Fixes
 
@@ -545,6 +574,10 @@ class ExamplePipelineModel(custom_model.CustomModel):
 - Data Connector: Add the option of passing a `None` sized batch to `to_torch_dataset` for better
 interoperability with PyTorch DataLoader.
 - Model Registry: Support [pandas.CategoricalDtype](https://pandas.pydata.org/docs/reference/api/pandas.CategoricalDtype.html#pandas-categoricaldtype)
+  - Limitations:
+    - The native categorical data handling handling by XGBoost using `enable_categorical=True` is not supported.
+    Instead please use [`sklearn.pipeline`](https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html)
+    to preprocess the categorical datatype and log the pipeline with the XGBoost model.
 - Registry: It is now possible to pass `signatures` and `sample_input_data` at the same time to capture background
 data from explainablity and data lineage.
 

{snowflake_ml_python-1.7.4 → snowflake_ml_python-1.7.5}/README.md

@@ -53,7 +53,7 @@ If you don't have a Snowflake account yet, you can [sign up for a 30-day free tr
 Follow the [installation instructions](https://docs.snowflake.com/en/developer-guide/snowpark-ml/index#installing-snowpark-ml)
 in the Snowflake documentation.
 
-Python versions 3.9 to 3.11 are supported. You can use [miniconda](https://docs.conda.io/en/latest/miniconda.html) or
+Python versions 3.9 to 3.12 are supported. You can use [miniconda](https://docs.conda.io/en/latest/miniconda.html) or
 [anaconda](https://www.anaconda.com/) to create a Conda environment (recommended),
 or [virtualenv](https://docs.python.org/3/tutorial/venv.html) to create a virtual environment.
 

{snowflake_ml_python-1.7.4 → snowflake_ml_python-1.7.5}/pyproject.toml

@@ -5,8 +5,8 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "snowflake-ml-python"
 description = "The machine learning client library that is used for interacting with Snowflake to build machine learning solutions."
-classifiers = [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Environment :: Other Environment", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Topic :: Database", "Topic :: Software Development", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Application Frameworks", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Scientific/Engineering :: Information Analysis",]
-requires-python = ">=3.9, <3.12"
+classifiers = [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Environment :: Other Environment", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Topic :: Database", "Topic :: Software Development", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Application Frameworks", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Scientific/Engineering :: Information Analysis",]
+requires-python = ">=3.9, <3.13"
 dynamic = [ "version", "readme",]
 dependencies = [ "absl-py>=0.15,<2", "anyio>=3.5.0,<5", "cachetools>=3.1.1,<6", "cloudpickle>=2.0.0", "cryptography", "fsspec[http]>=2024.6.1,<2026", "importlib_resources>=6.1.1, <7", "numpy>=1.23,<2", "packaging>=20.9,<25", "pandas>=1.0.0,<3", "pyarrow", "pyjwt>=2.0.0, <3", "pytimeparse>=1.1.8,<2", "pyyaml>=6.0,<7", "retrying>=1.3.3,<2", "s3fs>=2024.6.1,<2026", "scikit-learn>=1.4,<1.6", "scipy>=1.9,<2", "snowflake-connector-python[pandas]>=3.5.0,<4", "snowflake-snowpark-python>=1.17.0,<2,!=1.26.0", "sqlparse>=0.4,<1", "typing-extensions>=4.1.0,<5", "xgboost>=1.7.3,<3",]
 [[project.authors]]
@@ -24,15 +24,15 @@ Issues = "https://github.com/snowflakedb/snowflake-ml-python/issues"
 Changelog = "https://github.com/snowflakedb/snowflake-ml-python/blob/master/CHANGELOG.md"
 
 [project.optional-dependencies]
-all = [ "catboost>=1.2.0, <2", "huggingface_hub<0.26", "lightgbm>=4.1.0, <5", "mlflow>=2.16.0, <3", "peft>=0.5.0,<1", "sentence-transformers>=2.2.2,<3", "sentencepiece>=0.1.95,<1", "shap>=0.46.0,<1", "tensorflow>=2.12.0,<3", "tokenizers>=0.10,<1", "torch>=2.0.1,<2.3.0", "torchdata>=0.4,<1", "transformers>=4.32.1,<5",]
+all = [ "catboost>=1.2.0, <2", "keras>=2.0.0,<4", "lightgbm>=4.1.0, <5", "mlflow>=2.16.0, <3", "sentence-transformers>=2.7.0,<3", "sentencepiece>=0.1.95,<0.2.0", "shap>=0.46.0,<1", "tensorflow>=2.17.0,<3", "tokenizers>=0.15.1,<1", "torch>=2.0.1,<3", "torchdata>=0.4,<1", "transformers>=4.37.2,<5",]
 catboost = [ "catboost>=1.2.0, <2",]
+keras = [ "keras>=2.0.0,<4", "tensorflow>=2.17.0,<3", "torch>=2.0.1,<3",]
 lightgbm = [ "lightgbm>=4.1.0, <5",]
-llm = [ "peft>=0.5.0,<1",]
 mlflow = [ "mlflow>=2.16.0, <3",]
 shap = [ "shap>=0.46.0,<1",]
-tensorflow = [ "tensorflow>=2.12.0,<3",]
-torch = [ "torch>=2.0.1,<2.3.0", "torchdata>=0.4,<1",]
-transformers = [ "huggingface_hub<0.26", "sentence-transformers>=2.2.2,<3", "sentencepiece>=0.1.95,<1", "tokenizers>=0.10,<1", "transformers>=4.32.1,<5",]
+tensorflow = [ "tensorflow>=2.17.0,<3",]
+torch = [ "torch>=2.0.1,<3", "torchdata>=0.4,<1",]
+transformers = [ "sentence-transformers>=2.7.0,<3", "sentencepiece>=0.1.95,<0.2.0", "tokenizers>=0.15.1,<1", "torch>=2.0.1,<3", "transformers>=4.37.2,<5",]
 
 [tool.setuptools.package-data]
 "*" = [ "*",]

{snowflake_ml_python-1.7.4 → snowflake_ml_python-1.7.5}/snowflake/ml/_internal/env_utils.py

@@ -12,7 +12,7 @@ import yaml
 from packaging import requirements, specifiers, version
 
 import snowflake.connector
-from snowflake.ml._internal import env as snowml_env
+from snowflake.ml._internal import env as snowml_env, relax_version_strategy
 from snowflake.ml._internal.utils import query_result_checker
 from snowflake.snowpark import context, exceptions, session
 
@@ -56,6 +56,8 @@ def _validate_pip_requirement_string(req_str: str) -> requirements.Requirement:
 
         if r.name == "python":
             raise ValueError("Don't specify python as a dependency, use python version argument instead.")
+        if r.name == "cuda":
+            raise ValueError("Don't specify cuda as a dependency, use cuda version argument instead.")
     except requirements.InvalidRequirement:
         raise ValueError(f"Invalid package requirement {req_str} found.")
 
@@ -313,19 +315,14 @@ def get_package_spec_with_supported_ops_only(req: requirements.Requirement) -> r
     return new_req
 
 
-def relax_requirement_version(req: requirements.Requirement) -> requirements.Requirement:
-    """Relax version specifier from a requirement. It detects any ==x.y.z in specifiers and replaced with
-    >=x.y, <(x+1)
-
-    Args:
-        req: The requirement that version specifier to be removed.
-
-    Returns:
-        A new requirement object after relaxations.
-    """
-    new_req = copy.deepcopy(req)
+def _relax_specifier_set(
+    specifier_set: specifiers.SpecifierSet, strategy: relax_version_strategy.RelaxVersionStrategy
+) -> specifiers.SpecifierSet:
+    if strategy == relax_version_strategy.RelaxVersionStrategy.NO_RELAX:
+        return specifier_set
+    specifier_set = copy.deepcopy(specifier_set)
     relaxed_specifier_set = set()
-    for spec in new_req.specifier._specs:
+    for spec in specifier_set._specs:
         if spec.operator != "==":
             relaxed_specifier_set.add(spec)
             continue
@@ -337,9 +334,40 @@ def relax_requirement_version(req: requirements.Requirement) -> requirements.Req
             relaxed_specifier_set.add(spec)
             continue
         assert pinned_version is not None
-        relaxed_specifier_set.add(specifiers.Specifier(f">={pinned_version.major}.{pinned_version.minor}"))
-        relaxed_specifier_set.add(specifiers.Specifier(f"<{pinned_version.major + 1}"))
-    new_req.specifier._specs = frozenset(relaxed_specifier_set)
+        if strategy == relax_version_strategy.RelaxVersionStrategy.PATCH:
+            relaxed_specifier_set.add(specifiers.Specifier(f">={pinned_version.major}.{pinned_version.minor}"))
+            relaxed_specifier_set.add(specifiers.Specifier(f"<{pinned_version.major}.{pinned_version.minor+1}"))
+        elif strategy == relax_version_strategy.RelaxVersionStrategy.MINOR:
+            relaxed_specifier_set.add(specifiers.Specifier(f">={pinned_version.major}.{pinned_version.minor}"))
+            relaxed_specifier_set.add(specifiers.Specifier(f"<{pinned_version.major + 1}"))
+        elif strategy == relax_version_strategy.RelaxVersionStrategy.MAJOR:
+            relaxed_specifier_set.add(specifiers.Specifier(f">={pinned_version.major}"))
+            relaxed_specifier_set.add(specifiers.Specifier(f"<{pinned_version.major + 1}"))
+    specifier_set._specs = frozenset(relaxed_specifier_set)
+    return specifier_set
+
+
+def relax_requirement_version(req: requirements.Requirement) -> requirements.Requirement:
+    """Relax version specifier from a requirement. It detects any ==x.y.z in specifiers and replaced with relaxed
+    version specifier based on the strategy defined in RELAX_VERSION_STRATEGY_MAP.
+
+    NO_RELAX: No relaxation.
+    PATCH: >=x.y, <x.(y+1)
+    MINOR (default): >=x.y, <(x+1)
+    MAJOR: >=x, <(x+1)
+
+
+    Args:
+        req: The requirement that version specifier to be removed.
+
+    Returns:
+        A new requirement object after relaxations.
+    """
+    new_req = copy.deepcopy(req)
+    strategy = relax_version_strategy.RELAX_VERSION_STRATEGY_MAP.get(
+        req.name, relax_version_strategy.RelaxVersionStrategy.MINOR
+    )
+    new_req.specifier = _relax_specifier_set(new_req.specifier, strategy)
     return new_req
 
 
@@ -431,10 +459,11 @@ def save_conda_env_file(
     path: pathlib.Path,
     conda_chan_deps: DefaultDict[str, List[requirements.Requirement]],
     python_version: str,
+    cuda_version: Optional[str] = None,
     default_channel_override: str = SNOWFLAKE_CONDA_CHANNEL_URL,
 ) -> None:
     """Generate conda.yml file given a dict of dependencies after validation.
-    The channels part of conda.yml file will contains Snowflake Anaconda Channel, nodefaults and all channel names
+    The channels part of conda.yml file will contain Snowflake Anaconda Channel, nodefaults and all channel names
     in keys of the dict, ordered by the number of the packages which belongs to.
     The dependencies part of conda.yml file will contains requirements specifications. If the requirements is in the
     value list whose key is DEFAULT_CHANNEL_NAME, then the channel won't be specified explicitly. Otherwise, it will be
@@ -443,7 +472,8 @@ def save_conda_env_file(
     Args:
         path: Path to the conda.yml file.
         conda_chan_deps: Dict of conda dependencies after validated.
-        python_version: A string 'major.minor' showing python version relate to model.
+        python_version: A string 'major.minor' for the model's python version.
+        cuda_version: A string 'major.minor' for the model's cuda version.
         default_channel_override: The default channel to be put in the first place of the channels section.
     """
     assert path.suffix in [".yml", ".yaml"], "Conda environment file should have extension of yml or yaml."
@@ -461,6 +491,10 @@ def save_conda_env_file(
 
     env["channels"] = [default_channel_override] + channels + [_NODEFAULTS]
     env["dependencies"] = [f"python=={python_version}.*"]
+
+    if cuda_version is not None:
+        env["dependencies"].extend([f"nvidia::cuda=={cuda_version}.*"])
+
     for chan, reqs in conda_chan_deps.items():
         env["dependencies"].extend(
             [f"{chan}::{str(req)}" if chan != DEFAULT_CHANNEL_NAME else str(req) for req in reqs]
@@ -487,7 +521,12 @@ def save_requirements_file(path: pathlib.Path, pip_deps: List[requirements.Requi
 
 def load_conda_env_file(
     path: pathlib.Path,
-) -> Tuple[DefaultDict[str, List[requirements.Requirement]], Optional[List[requirements.Requirement]], Optional[str]]:
+) -> Tuple[
+    DefaultDict[str, List[requirements.Requirement]],
+    Optional[List[requirements.Requirement]],
+    Optional[str],
+    Optional[str],
+]:
     """Read conda.yml file to get a dict of dependencies after validation.
     The channels part of conda.yml file will be processed with following rules:
     1. If it is Snowflake Anaconda Channel, ignore as it is default.
@@ -515,7 +554,7 @@ def load_conda_env_file(
         and a string 'major.minor.patchlevel' of python version.
     """
     if not path.exists():
-        return collections.defaultdict(list), None, None
+        return collections.defaultdict(list), None, None, None
 
     with open(path, encoding="utf-8") as f:
         env = yaml.safe_load(stream=f)
@@ -526,6 +565,7 @@ def load_conda_env_file(
     pip_deps = []
 
     python_version = None
+    cuda_version = None
 
     channels = env.get("channels", [])
     if len(channels) >= 1:
@@ -541,6 +581,9 @@ def load_conda_env_file(
             # ver is str: python w/ specifier
             if ver:
                 python_version = ver
+            elif dep.startswith("nvidia::cuda"):
+                r = requirements.Requirement(dep.split("nvidia::")[1])
+                cuda_version = list(r.specifier)[0].version.strip(".*")
             elif ver is None:
                 deps.append(dep)
         elif isinstance(dep, dict) and "pip" in dep:
@@ -555,7 +598,7 @@ def load_conda_env_file(
         if channel not in conda_dep_dict:
             conda_dep_dict[channel] = []
 
-    return conda_dep_dict, pip_deps_list if pip_deps_list else None, python_version
+    return conda_dep_dict, pip_deps_list if pip_deps_list else None, python_version, cuda_version
 
 
 def load_requirements_file(path: pathlib.Path) -> List[requirements.Requirement]:

snowflake_ml_python-1.7.5/snowflake/ml/_internal/relax_version_strategy.py

@@ -0,0 +1,16 @@
+from enum import Enum
+
+
+class RelaxVersionStrategy(Enum):
+    NO_RELAX = "no_relax"
+    PATCH = "patch"
+    MINOR = "minor"
+    MAJOR = "major"
+
+
+RELAX_VERSION_STRATEGY_MAP = {
+    # The version of cloudpickle should not be relaxed as it is used for serialization.
+    "cloudpickle": RelaxVersionStrategy.NO_RELAX,
+    # The version of scikit-learn should be relaxed only in patch version as it has breaking changes in minor version.
+    "scikit-learn": RelaxVersionStrategy.PATCH,
+}

{snowflake_ml_python-1.7.4 → snowflake_ml_python-1.7.5}/snowflake/ml/_internal/telemetry.py

@@ -4,6 +4,9 @@ import enum
 import functools
 import inspect
 import operator
+import sys
+import time
+import traceback
 import types
 from typing import (
     Any,
@@ -75,6 +78,8 @@ class TelemetryField(enum.Enum):
     KEY_FUNC_PARAMS = "func_params"
     KEY_ERROR_INFO = "error_info"
     KEY_ERROR_CODE = "error_code"
+    KEY_STACK_TRACE = "stack_trace"
+    KEY_DURATION = "duration"
     KEY_VERSION = "version"
     KEY_PYTHON_VERSION = "python_version"
     KEY_OS = "operating_system"
@@ -435,6 +440,7 @@ def send_api_usage_telemetry(
 
     # noqa: DAR402
     """
+    start_time = time.perf_counter()
 
     if subproject is not None and subproject_extractor is not None:
         raise ValueError("Specifying both subproject and subproject_extractor is not allowed")
@@ -555,8 +561,16 @@ def send_api_usage_telemetry(
                         )
                 else:
                     me = e
+
                 telemetry_args["error"] = repr(me)
                 telemetry_args["error_code"] = me.error_code
+                # exclude telemetry frames
+                excluded_frames = 2
+                tb = traceback.extract_tb(sys.exc_info()[2])
+                formatted_tb = "".join(traceback.format_list(tb[excluded_frames:]))
+                formatted_exception = traceback.format_exception_only(*sys.exc_info()[:2])[0]  # error type + message
+                telemetry_args["stack_trace"] = formatted_tb + formatted_exception
+
                 me.original_exception._snowflake_ml_handled = True  # type: ignore[attr-defined]
                 if e is not me:
                     raise  # Directly raise non-wrapped exceptions to preserve original stacktrace
@@ -565,6 +579,7 @@ def send_api_usage_telemetry(
                 else:
                     raise me.original_exception from e
             finally:
+                telemetry_args["duration"] = time.perf_counter() - start_time  # type: ignore[assignment]
                 telemetry.send_function_usage_telemetry(**telemetry_args)
                 global _log_counter
                 _log_counter += 1
@@ -718,12 +733,14 @@ class _SourceTelemetryClient:
         self,
         func_name: str,
         function_category: str,
+        duration: float,
         func_params: Optional[Dict[str, Any]] = None,
         api_calls: Optional[List[Dict[str, Any]]] = None,
         sfqids: Optional[List[Any]] = None,
         custom_tags: Optional[Dict[str, Union[bool, int, str, float]]] = None,
         error: Optional[str] = None,
         error_code: Optional[str] = None,
+        stack_trace: Optional[str] = None,
     ) -> None:
         """
         Send function usage telemetry message.
@@ -731,12 +748,14 @@ class _SourceTelemetryClient:
         Args:
             func_name: Function name.
             function_category: Function category.
+            duration: Function duration.
             func_params: Function parameters.
             api_calls: API calls.
             sfqids: Snowflake query IDs.
             custom_tags: Custom tags.
             error: Error.
             error_code: Error code.
+            stack_trace: Error stack trace.
         """
         data: Dict[str, Any] = {
             TelemetryField.KEY_FUNC_NAME.value: func_name,
@@ -755,11 +774,13 @@ class _SourceTelemetryClient:
         message: Dict[str, Any] = {
             **self._create_basic_telemetry_data(telemetry_type),
             TelemetryField.KEY_DATA.value: data,
+            TelemetryField.KEY_DURATION.value: duration,
         }
 
         if error:
             message[TelemetryField.KEY_ERROR_INFO.value] = error
             message[TelemetryField.KEY_ERROR_CODE.value] = error_code
+            message[TelemetryField.KEY_STACK_TRACE.value] = stack_trace
 
         self._send(message)
 

{snowflake_ml_python-1.7.4 → snowflake_ml_python-1.7.5}/snowflake/ml/data/_internal/arrow_ingestor.py

@@ -116,7 +116,7 @@ class ArrowIngestor(data_ingestor.DataIngestor):
     def to_pandas(self, limit: Optional[int] = None) -> pd.DataFrame:
         ds = self._get_dataset(shuffle=False)
         table = ds.to_table() if limit is None else ds.head(num_rows=limit)
-        return table.to_pandas()
+        return table.to_pandas(split_blocks=True, self_destruct=True)
 
     def _get_dataset(self, shuffle: bool) -> pds.Dataset:
         format = self._format

{snowflake_ml_python-1.7.4 → snowflake_ml_python-1.7.5}/snowflake/ml/feature_store/feature_store.py

@@ -144,6 +144,7 @@ _LIST_FEATURE_VIEW_SCHEMA = StructType(
         StructField("refresh_mode", StringType()),
         StructField("scheduling_state", StringType()),
         StructField("warehouse", StringType()),
+        StructField("cluster_by", StringType()),
     ]
 )
 
@@ -1832,6 +1833,12 @@ class FeatureStore:
                 WAREHOUSE = {warehouse}
                 REFRESH_MODE = {feature_view.refresh_mode}
                 INITIALIZE = {feature_view.initialize}
+            """
+            if feature_view.cluster_by:
+                cluster_by_clause = f"CLUSTER BY ({', '.join(feature_view.cluster_by)})"
+                query += f"{cluster_by_clause}"
+
+            query += f"""
                 AS {feature_view.query}
             """
             self._session.sql(query).collect(block=block, statement_params=self._telemetry_stmp)
@@ -2249,6 +2256,7 @@ class FeatureStore:
         values.append(row["refresh_mode"] if "refresh_mode" in row else None)
         values.append(row["scheduling_state"] if "scheduling_state" in row else None)
         values.append(row["warehouse"] if "warehouse" in row else None)
+        values.append(json.dumps(self._extract_cluster_by_columns(row["cluster_by"])) if "cluster_by" in row else None)
         output_values.append(values)
 
     def _lookup_feature_view_metadata(self, row: Row, fv_name: str) -> Tuple[_FeatureViewMetadata, str]:
@@ -2335,6 +2343,7 @@ class FeatureStore:
                 owner=row["owner"],
                 infer_schema_df=infer_schema_df,
                 session=self._session,
+                cluster_by=self._extract_cluster_by_columns(row["cluster_by"]),
             )
             return fv
         else:
@@ -2625,3 +2634,12 @@ class FeatureStore:
             )
 
         return feature_view
+
+    @staticmethod
+    def _extract_cluster_by_columns(cluster_by_clause: str) -> List[str]:
+        # Use regex to extract elements inside the parentheses.
+        match = re.search(r"\((.*?)\)", cluster_by_clause)
+        if match:
+            # Handle both quoted and unquoted column names.
+            return re.findall(identifier.SF_IDENTIFIER_RE, match.group(1))
+        return []