snowflake-ml-python 1.0.1__py3-none-any.whl → 1.0.3__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (196) hide show
  1. snowflake/ml/_internal/env_utils.py +2 -1
  2. snowflake/ml/_internal/file_utils.py +35 -40
  3. snowflake/ml/_internal/telemetry.py +5 -8
  4. snowflake/ml/_internal/utils/identifier.py +74 -7
  5. snowflake/ml/_internal/utils/uri.py +7 -2
  6. snowflake/ml/model/_core_requirements.py +1 -1
  7. snowflake/ml/model/_deploy_client/image_builds/base_image_builder.py +15 -0
  8. snowflake/ml/model/_deploy_client/image_builds/client_image_builder.py +259 -0
  9. snowflake/ml/model/_deploy_client/image_builds/docker_context.py +89 -0
  10. snowflake/ml/model/_deploy_client/image_builds/gunicorn_run.sh +24 -0
  11. snowflake/ml/model/_deploy_client/image_builds/inference_server/main.py +118 -0
  12. snowflake/ml/model/_deploy_client/image_builds/templates/dockerfile_template +40 -0
  13. snowflake/ml/model/_deploy_client/snowservice/deploy.py +199 -0
  14. snowflake/ml/model/_deploy_client/snowservice/deploy_options.py +88 -0
  15. snowflake/ml/model/_deploy_client/snowservice/templates/service_spec_template +24 -0
  16. snowflake/ml/model/_deploy_client/utils/constants.py +47 -0
  17. snowflake/ml/model/_deploy_client/utils/snowservice_client.py +178 -0
  18. snowflake/ml/model/_deploy_client/warehouse/deploy.py +25 -28
  19. snowflake/ml/model/_deploy_client/warehouse/infer_template.py +7 -4
  20. snowflake/ml/model/_deployer.py +14 -27
  21. snowflake/ml/model/_env.py +4 -4
  22. snowflake/ml/model/_handlers/_base.py +3 -1
  23. snowflake/ml/model/_handlers/custom.py +14 -2
  24. snowflake/ml/model/_handlers/pytorch.py +186 -0
  25. snowflake/ml/model/_handlers/sklearn.py +14 -8
  26. snowflake/ml/model/_handlers/snowmlmodel.py +14 -9
  27. snowflake/ml/model/_handlers/torchscript.py +180 -0
  28. snowflake/ml/model/_handlers/xgboost.py +19 -9
  29. snowflake/ml/model/_model.py +27 -21
  30. snowflake/ml/model/_model_meta.py +33 -19
  31. snowflake/ml/model/model_signature.py +446 -66
  32. snowflake/ml/model/type_hints.py +28 -15
  33. snowflake/ml/modeling/calibration/calibrated_classifier_cv.py +79 -43
  34. snowflake/ml/modeling/cluster/affinity_propagation.py +79 -43
  35. snowflake/ml/modeling/cluster/agglomerative_clustering.py +79 -43
  36. snowflake/ml/modeling/cluster/birch.py +79 -43
  37. snowflake/ml/modeling/cluster/bisecting_k_means.py +79 -43
  38. snowflake/ml/modeling/cluster/dbscan.py +79 -43
  39. snowflake/ml/modeling/cluster/feature_agglomeration.py +79 -43
  40. snowflake/ml/modeling/cluster/k_means.py +79 -43
  41. snowflake/ml/modeling/cluster/mean_shift.py +79 -43
  42. snowflake/ml/modeling/cluster/mini_batch_k_means.py +79 -43
  43. snowflake/ml/modeling/cluster/optics.py +79 -43
  44. snowflake/ml/modeling/cluster/spectral_biclustering.py +79 -43
  45. snowflake/ml/modeling/cluster/spectral_clustering.py +79 -43
  46. snowflake/ml/modeling/cluster/spectral_coclustering.py +79 -43
  47. snowflake/ml/modeling/compose/column_transformer.py +79 -43
  48. snowflake/ml/modeling/compose/transformed_target_regressor.py +79 -43
  49. snowflake/ml/modeling/covariance/elliptic_envelope.py +79 -43
  50. snowflake/ml/modeling/covariance/empirical_covariance.py +79 -43
  51. snowflake/ml/modeling/covariance/graphical_lasso.py +79 -43
  52. snowflake/ml/modeling/covariance/graphical_lasso_cv.py +79 -43
  53. snowflake/ml/modeling/covariance/ledoit_wolf.py +79 -43
  54. snowflake/ml/modeling/covariance/min_cov_det.py +79 -43
  55. snowflake/ml/modeling/covariance/oas.py +79 -43
  56. snowflake/ml/modeling/covariance/shrunk_covariance.py +79 -43
  57. snowflake/ml/modeling/decomposition/dictionary_learning.py +79 -43
  58. snowflake/ml/modeling/decomposition/factor_analysis.py +79 -43
  59. snowflake/ml/modeling/decomposition/fast_ica.py +79 -43
  60. snowflake/ml/modeling/decomposition/incremental_pca.py +79 -43
  61. snowflake/ml/modeling/decomposition/kernel_pca.py +79 -43
  62. snowflake/ml/modeling/decomposition/mini_batch_dictionary_learning.py +79 -43
  63. snowflake/ml/modeling/decomposition/mini_batch_sparse_pca.py +79 -43
  64. snowflake/ml/modeling/decomposition/pca.py +79 -43
  65. snowflake/ml/modeling/decomposition/sparse_pca.py +79 -43
  66. snowflake/ml/modeling/decomposition/truncated_svd.py +79 -43
  67. snowflake/ml/modeling/discriminant_analysis/linear_discriminant_analysis.py +79 -43
  68. snowflake/ml/modeling/discriminant_analysis/quadratic_discriminant_analysis.py +79 -43
  69. snowflake/ml/modeling/ensemble/ada_boost_classifier.py +79 -43
  70. snowflake/ml/modeling/ensemble/ada_boost_regressor.py +79 -43
  71. snowflake/ml/modeling/ensemble/bagging_classifier.py +79 -43
  72. snowflake/ml/modeling/ensemble/bagging_regressor.py +79 -43
  73. snowflake/ml/modeling/ensemble/extra_trees_classifier.py +79 -43
  74. snowflake/ml/modeling/ensemble/extra_trees_regressor.py +79 -43
  75. snowflake/ml/modeling/ensemble/gradient_boosting_classifier.py +79 -43
  76. snowflake/ml/modeling/ensemble/gradient_boosting_regressor.py +79 -43
  77. snowflake/ml/modeling/ensemble/hist_gradient_boosting_classifier.py +79 -43
  78. snowflake/ml/modeling/ensemble/hist_gradient_boosting_regressor.py +79 -43
  79. snowflake/ml/modeling/ensemble/isolation_forest.py +79 -43
  80. snowflake/ml/modeling/ensemble/random_forest_classifier.py +79 -43
  81. snowflake/ml/modeling/ensemble/random_forest_regressor.py +79 -43
  82. snowflake/ml/modeling/ensemble/stacking_regressor.py +79 -43
  83. snowflake/ml/modeling/ensemble/voting_classifier.py +79 -43
  84. snowflake/ml/modeling/ensemble/voting_regressor.py +79 -43
  85. snowflake/ml/modeling/feature_selection/generic_univariate_select.py +79 -43
  86. snowflake/ml/modeling/feature_selection/select_fdr.py +79 -43
  87. snowflake/ml/modeling/feature_selection/select_fpr.py +79 -43
  88. snowflake/ml/modeling/feature_selection/select_fwe.py +79 -43
  89. snowflake/ml/modeling/feature_selection/select_k_best.py +79 -43
  90. snowflake/ml/modeling/feature_selection/select_percentile.py +79 -43
  91. snowflake/ml/modeling/feature_selection/sequential_feature_selector.py +79 -43
  92. snowflake/ml/modeling/feature_selection/variance_threshold.py +79 -43
  93. snowflake/ml/modeling/gaussian_process/gaussian_process_classifier.py +79 -43
  94. snowflake/ml/modeling/gaussian_process/gaussian_process_regressor.py +79 -43
  95. snowflake/ml/modeling/impute/iterative_imputer.py +79 -43
  96. snowflake/ml/modeling/impute/knn_imputer.py +79 -43
  97. snowflake/ml/modeling/impute/missing_indicator.py +79 -43
  98. snowflake/ml/modeling/kernel_approximation/additive_chi2_sampler.py +79 -43
  99. snowflake/ml/modeling/kernel_approximation/nystroem.py +79 -43
  100. snowflake/ml/modeling/kernel_approximation/polynomial_count_sketch.py +79 -43
  101. snowflake/ml/modeling/kernel_approximation/rbf_sampler.py +79 -43
  102. snowflake/ml/modeling/kernel_approximation/skewed_chi2_sampler.py +79 -43
  103. snowflake/ml/modeling/kernel_ridge/kernel_ridge.py +79 -43
  104. snowflake/ml/modeling/lightgbm/lgbm_classifier.py +79 -43
  105. snowflake/ml/modeling/lightgbm/lgbm_regressor.py +79 -43
  106. snowflake/ml/modeling/linear_model/ard_regression.py +79 -43
  107. snowflake/ml/modeling/linear_model/bayesian_ridge.py +79 -43
  108. snowflake/ml/modeling/linear_model/elastic_net.py +79 -43
  109. snowflake/ml/modeling/linear_model/elastic_net_cv.py +79 -43
  110. snowflake/ml/modeling/linear_model/gamma_regressor.py +79 -43
  111. snowflake/ml/modeling/linear_model/huber_regressor.py +79 -43
  112. snowflake/ml/modeling/linear_model/lars.py +79 -43
  113. snowflake/ml/modeling/linear_model/lars_cv.py +79 -43
  114. snowflake/ml/modeling/linear_model/lasso.py +79 -43
  115. snowflake/ml/modeling/linear_model/lasso_cv.py +79 -43
  116. snowflake/ml/modeling/linear_model/lasso_lars.py +79 -43
  117. snowflake/ml/modeling/linear_model/lasso_lars_cv.py +79 -43
  118. snowflake/ml/modeling/linear_model/lasso_lars_ic.py +79 -43
  119. snowflake/ml/modeling/linear_model/linear_regression.py +79 -43
  120. snowflake/ml/modeling/linear_model/logistic_regression.py +79 -43
  121. snowflake/ml/modeling/linear_model/logistic_regression_cv.py +79 -43
  122. snowflake/ml/modeling/linear_model/multi_task_elastic_net.py +79 -43
  123. snowflake/ml/modeling/linear_model/multi_task_elastic_net_cv.py +79 -43
  124. snowflake/ml/modeling/linear_model/multi_task_lasso.py +79 -43
  125. snowflake/ml/modeling/linear_model/multi_task_lasso_cv.py +79 -43
  126. snowflake/ml/modeling/linear_model/orthogonal_matching_pursuit.py +79 -43
  127. snowflake/ml/modeling/linear_model/passive_aggressive_classifier.py +79 -43
  128. snowflake/ml/modeling/linear_model/passive_aggressive_regressor.py +79 -43
  129. snowflake/ml/modeling/linear_model/perceptron.py +79 -43
  130. snowflake/ml/modeling/linear_model/poisson_regressor.py +79 -43
  131. snowflake/ml/modeling/linear_model/ransac_regressor.py +79 -43
  132. snowflake/ml/modeling/linear_model/ridge.py +79 -43
  133. snowflake/ml/modeling/linear_model/ridge_classifier.py +79 -43
  134. snowflake/ml/modeling/linear_model/ridge_classifier_cv.py +79 -43
  135. snowflake/ml/modeling/linear_model/ridge_cv.py +79 -43
  136. snowflake/ml/modeling/linear_model/sgd_classifier.py +79 -43
  137. snowflake/ml/modeling/linear_model/sgd_one_class_svm.py +79 -43
  138. snowflake/ml/modeling/linear_model/sgd_regressor.py +79 -43
  139. snowflake/ml/modeling/linear_model/theil_sen_regressor.py +79 -43
  140. snowflake/ml/modeling/linear_model/tweedie_regressor.py +79 -43
  141. snowflake/ml/modeling/manifold/isomap.py +79 -43
  142. snowflake/ml/modeling/manifold/mds.py +79 -43
  143. snowflake/ml/modeling/manifold/spectral_embedding.py +79 -43
  144. snowflake/ml/modeling/manifold/tsne.py +79 -43
  145. snowflake/ml/modeling/metrics/classification.py +6 -1
  146. snowflake/ml/modeling/metrics/regression.py +517 -9
  147. snowflake/ml/modeling/mixture/bayesian_gaussian_mixture.py +79 -43
  148. snowflake/ml/modeling/mixture/gaussian_mixture.py +79 -43
  149. snowflake/ml/modeling/model_selection/grid_search_cv.py +79 -43
  150. snowflake/ml/modeling/model_selection/randomized_search_cv.py +79 -43
  151. snowflake/ml/modeling/multiclass/one_vs_one_classifier.py +79 -43
  152. snowflake/ml/modeling/multiclass/one_vs_rest_classifier.py +79 -43
  153. snowflake/ml/modeling/multiclass/output_code_classifier.py +79 -43
  154. snowflake/ml/modeling/naive_bayes/bernoulli_nb.py +79 -43
  155. snowflake/ml/modeling/naive_bayes/categorical_nb.py +79 -43
  156. snowflake/ml/modeling/naive_bayes/complement_nb.py +79 -43
  157. snowflake/ml/modeling/naive_bayes/gaussian_nb.py +79 -43
  158. snowflake/ml/modeling/naive_bayes/multinomial_nb.py +79 -43
  159. snowflake/ml/modeling/neighbors/k_neighbors_classifier.py +79 -43
  160. snowflake/ml/modeling/neighbors/k_neighbors_regressor.py +79 -43
  161. snowflake/ml/modeling/neighbors/kernel_density.py +79 -43
  162. snowflake/ml/modeling/neighbors/local_outlier_factor.py +79 -43
  163. snowflake/ml/modeling/neighbors/nearest_centroid.py +79 -43
  164. snowflake/ml/modeling/neighbors/nearest_neighbors.py +79 -43
  165. snowflake/ml/modeling/neighbors/neighborhood_components_analysis.py +79 -43
  166. snowflake/ml/modeling/neighbors/radius_neighbors_classifier.py +79 -43
  167. snowflake/ml/modeling/neighbors/radius_neighbors_regressor.py +79 -43
  168. snowflake/ml/modeling/neural_network/bernoulli_rbm.py +79 -43
  169. snowflake/ml/modeling/neural_network/mlp_classifier.py +79 -43
  170. snowflake/ml/modeling/neural_network/mlp_regressor.py +79 -43
  171. snowflake/ml/modeling/pipeline/pipeline.py +24 -0
  172. snowflake/ml/modeling/preprocessing/one_hot_encoder.py +18 -19
  173. snowflake/ml/modeling/preprocessing/ordinal_encoder.py +2 -0
  174. snowflake/ml/modeling/preprocessing/polynomial_features.py +79 -43
  175. snowflake/ml/modeling/semi_supervised/label_propagation.py +79 -43
  176. snowflake/ml/modeling/semi_supervised/label_spreading.py +79 -43
  177. snowflake/ml/modeling/svm/linear_svc.py +79 -43
  178. snowflake/ml/modeling/svm/linear_svr.py +79 -43
  179. snowflake/ml/modeling/svm/nu_svc.py +79 -43
  180. snowflake/ml/modeling/svm/nu_svr.py +79 -43
  181. snowflake/ml/modeling/svm/svc.py +79 -43
  182. snowflake/ml/modeling/svm/svr.py +79 -43
  183. snowflake/ml/modeling/tree/decision_tree_classifier.py +79 -43
  184. snowflake/ml/modeling/tree/decision_tree_regressor.py +79 -43
  185. snowflake/ml/modeling/tree/extra_tree_classifier.py +79 -43
  186. snowflake/ml/modeling/tree/extra_tree_regressor.py +79 -43
  187. snowflake/ml/modeling/xgboost/xgb_classifier.py +79 -43
  188. snowflake/ml/modeling/xgboost/xgb_regressor.py +79 -43
  189. snowflake/ml/modeling/xgboost/xgbrf_classifier.py +79 -43
  190. snowflake/ml/modeling/xgboost/xgbrf_regressor.py +79 -43
  191. snowflake/ml/registry/model_registry.py +123 -121
  192. snowflake/ml/version.py +1 -1
  193. {snowflake_ml_python-1.0.1.dist-info → snowflake_ml_python-1.0.3.dist-info}/METADATA +50 -8
  194. snowflake_ml_python-1.0.3.dist-info/RECORD +259 -0
  195. snowflake_ml_python-1.0.1.dist-info/RECORD +0 -246
  196. {snowflake_ml_python-1.0.1.dist-info → snowflake_ml_python-1.0.3.dist-info}/WHEEL +0 -0
@@ -1,6 +1,7 @@
1
1
  import inspect
2
2
  import json
3
3
  import os
4
+ import posixpath
4
5
  import sys
5
6
  import tempfile
6
7
  import types
@@ -8,7 +9,6 @@ import zipfile
8
9
  from typing import TYPE_CHECKING, Any, Dict, List, Optional, cast
9
10
  from uuid import uuid1
10
11
 
11
- import cloudpickle as cp
12
12
  from absl import logging
13
13
 
14
14
  from snowflake import connector, snowpark
@@ -35,7 +35,7 @@ _DEFAULT_REGISTRY_NAME: str = "_SYSTEM_MODEL_REGISTRY"
35
35
  _DEFAULT_SCHEMA_NAME: str = "_SYSTEM_MODEL_REGISTRY_SCHEMA"
36
36
  _MODELS_TABLE_NAME: str = "_SYSTEM_REGISTRY_MODELS"
37
37
  _METADATA_TABLE_NAME: str = "_SYSTEM_REGISTRY_METADATA"
38
- _DEPLOYMENT_TABLE_NAME: str = "_SYSTEM_REGISRTRY_DEPLOYMENTS"
38
+ _DEPLOYMENT_TABLE_NAME: str = "_SYSTEM_REGISTRY_DEPLOYMENTS"
39
39
 
40
40
  # Metadata operation types.
41
41
  _SET_METADATA_OPERATION: str = "SET"
@@ -84,9 +84,11 @@ def create_model_registry(
84
84
  """
85
85
 
86
86
  # These might be exposed as parameters in the future.
87
- registry_table_name = _MODELS_TABLE_NAME
88
- metadata_table_name = _METADATA_TABLE_NAME
89
- deployment_table_name = _DEPLOYMENT_TABLE_NAME
87
+ database_name = identifier.get_inferred_name(database_name)
88
+ schema_name = identifier.get_inferred_name(schema_name)
89
+ registry_table_name = identifier.get_inferred_name(_MODELS_TABLE_NAME)
90
+ metadata_table_name = identifier.get_inferred_name(_METADATA_TABLE_NAME)
91
+ deployment_table_name = identifier.get_inferred_name(_DEPLOYMENT_TABLE_NAME)
90
92
  statement_params = telemetry.get_function_usage_statement_params(
91
93
  project=_TELEMETRY_PROJECT,
92
94
  subproject=_TELEMETRY_SUBPROJECT,
@@ -130,14 +132,14 @@ def _create_registry_database(
130
132
  database_name: Desired name of the model registry database.
131
133
  statement_params: Function usage statement parameters used in sql query executions.
132
134
  """
133
- registry_databases = session.sql(f"SHOW DATABASES LIKE '{database_name}'").collect(
135
+ registry_databases = session.sql(f"SHOW DATABASES LIKE '{identifier.get_unescaped_names(database_name)}'").collect(
134
136
  statement_params=statement_params
135
137
  )
136
138
  if len(registry_databases) > 0:
137
139
  logging.warning(f"The database {database_name} already exists. Skipping creation.")
138
140
  return
139
141
 
140
- session.sql(f'CREATE DATABASE "{database_name}"').collect(statement_params=statement_params)
142
+ session.sql(f"CREATE DATABASE {database_name}").collect(statement_params=statement_params)
141
143
 
142
144
 
143
145
  def _create_registry_schema(
@@ -154,31 +156,31 @@ def _create_registry_schema(
154
156
  statement_params: Function usage statement parameters used in sql query executions.
155
157
  """
156
158
  # The default PUBLIC schema is created by default so it might already exist even in a new database.
157
- registry_schemas = session.sql(f"SHOW SCHEMAS LIKE '{schema_name}' IN DATABASE \"{database_name}\"").collect(
158
- statement_params=statement_params
159
- )
159
+ registry_schemas = session.sql(
160
+ f"SHOW SCHEMAS LIKE '{identifier.get_unescaped_names(schema_name)}' IN DATABASE {database_name}"
161
+ ).collect(statement_params=statement_params)
160
162
 
161
163
  if len(registry_schemas) > 0:
162
- logging.warning(f'The schmea "{database_name}"."{schema_name}" already exists. Skipping creation.')
164
+ logging.warning(
165
+ f"The schema {_get_fully_qualified_schema_name(database_name, schema_name)}already exists. "
166
+ + "Skipping creation."
167
+ )
163
168
  return
164
169
 
165
- session.sql(f'CREATE SCHEMA "{database_name}"."{schema_name}"').collect(statement_params=statement_params)
170
+ session.sql(f"CREATE SCHEMA {_get_fully_qualified_schema_name(database_name, schema_name)}").collect(
171
+ statement_params=statement_params
172
+ )
166
173
 
167
174
 
168
175
  def _get_fully_qualified_schema_name(database_name: str, schema_name: str) -> str:
169
- return ".".join(
170
- [
171
- f"{identifier.quote_name_without_upper_casing(database_name)}",
172
- f"{identifier.quote_name_without_upper_casing(schema_name)}",
173
- ]
174
- )
176
+ return ".".join([database_name, schema_name])
175
177
 
176
178
 
177
179
  def _get_fully_qualified_table_name(database_name: str, schema_name: str, table_name: str) -> str:
178
180
  return ".".join(
179
181
  [
180
182
  _get_fully_qualified_schema_name(database_name, schema_name),
181
- f"{identifier.quote_name_without_upper_casing(table_name)}",
183
+ table_name,
182
184
  ]
183
185
  )
184
186
 
@@ -292,10 +294,10 @@ def _create_registry_views(
292
294
  )
293
295
 
294
296
  # Create views on most recent metadata items.
295
- metadata_view_name_prefix = metadata_table_name + "_LAST_"
297
+ metadata_view_name_prefix = identifier.concat_names([metadata_table_name, "_LAST_"])
296
298
  metadata_view_template = formatting.unwrap(
297
- """CREATE OR REPLACE VIEW "{database}"."{schema}"."{attribute_view}" COPY GRANTS AS
298
- SELECT DISTINCT MODEL_ID, {select_expression} AS {final_attribute_name} FROM "{metadata_table}"
299
+ """CREATE OR REPLACE VIEW {database}.{schema}.{attribute_view} COPY GRANTS AS
300
+ SELECT DISTINCT MODEL_ID, {select_expression} AS {final_attribute_name} FROM {metadata_table}
299
301
  WHERE ATTRIBUTE_NAME = '{attribute_name}'"""
300
302
  )
301
303
 
@@ -303,7 +305,7 @@ def _create_registry_views(
303
305
  metadata_view_names = []
304
306
  metadata_select_fields = []
305
307
  for attribute_name in _LIST_METADATA_ATTRIBUTE:
306
- view_name = f"{metadata_view_name_prefix}{attribute_name}"
308
+ view_name = identifier.concat_names([metadata_view_name_prefix, attribute_name])
307
309
  select_expression = f"(LAST_VALUE(VALUE) OVER (PARTITION BY MODEL_ID ORDER BY SEQUENCE_ID))['{attribute_name}']"
308
310
  sql = metadata_view_template.format(
309
311
  database=database_name,
@@ -316,14 +318,12 @@ def _create_registry_views(
316
318
  )
317
319
  session.sql(sql).collect(statement_params=statement_params)
318
320
  metadata_view_names.append(view_name)
319
- metadata_select_fields.append(
320
- f"{identifier.quote_name_without_upper_casing(view_name)}.{attribute_name} AS {attribute_name}"
321
- )
321
+ metadata_select_fields.append(f"{view_name}.{attribute_name} AS {attribute_name}")
322
322
 
323
323
  # Create a special view for the registration timestamp.
324
324
  attribute_name = _METADATA_ATTRIBUTE_REGISTRATION
325
- final_attribute_name = attribute_name + "_TIMESTAMP"
326
- view_name = f"{metadata_view_name_prefix}{attribute_name}"
325
+ final_attribute_name = identifier.concat_names([attribute_name, "_TIMESTAMP"])
326
+ view_name = identifier.concat_names([metadata_view_name_prefix, attribute_name])
327
327
  create_registration_view_sql = metadata_view_template.format(
328
328
  database=database_name,
329
329
  schema=schema_name,
@@ -335,13 +335,11 @@ def _create_registry_views(
335
335
  )
336
336
  session.sql(create_registration_view_sql).collect(statement_params=statement_params)
337
337
  metadata_view_names.append(view_name)
338
- metadata_select_fields.append(
339
- f"{identifier.quote_name_without_upper_casing(view_name)}.{final_attribute_name} AS {final_attribute_name}"
340
- )
338
+ metadata_select_fields.append(f"{view_name}.{final_attribute_name} AS {final_attribute_name}")
341
339
 
342
340
  metadata_views_join = " ".join(
343
341
  [
344
- 'LEFT JOIN "{view}" ON ("{view}".MODEL_ID = "{registry_table}".ID)'.format(
342
+ "LEFT JOIN {view} ON ({view}.MODEL_ID = {registry_table}.ID)".format(
345
343
  view=view, registry_table=registry_table_name
346
344
  )
347
345
  for view in metadata_view_names
@@ -349,12 +347,12 @@ def _create_registry_views(
349
347
  )
350
348
 
351
349
  # Create view to combine all attributes.
352
- registry_view_name = registry_table_name + "_VIEW"
350
+ registry_view_name = identifier.concat_names([registry_table_name, "_VIEW"])
353
351
  metadata_select_fields_formatted = ",".join(metadata_select_fields)
354
352
  session.sql(
355
- f"""CREATE OR REPLACE VIEW {fully_qualified_schema_name}."{registry_view_name}" COPY GRANTS AS
356
- SELECT "{registry_table_name}".*, {metadata_select_fields_formatted}
357
- FROM "{registry_table_name}" {metadata_views_join}"""
353
+ f"""CREATE OR REPLACE VIEW {fully_qualified_schema_name}.{registry_view_name} COPY GRANTS AS
354
+ SELECT {registry_table_name}.*, {metadata_select_fields_formatted}
355
+ FROM {registry_table_name} {metadata_views_join}"""
358
356
  ).collect(statement_params=statement_params)
359
357
 
360
358
 
@@ -377,8 +375,9 @@ def _create_active_permanent_deployment_view(
377
375
 
378
376
  # Create a view on active permanent deployments
379
377
  # Active deployments are those whose last operation is not DROP.
378
+ active_deployments_view_name = identifier.concat_names([deployment_table_name, "_VIEW"])
380
379
  active_deployments_view_expr = f"""
381
- CREATE OR REPLACE VIEW {fully_qualified_schema_name}."{deployment_table_name}_VIEW"
380
+ CREATE OR REPLACE VIEW {fully_qualified_schema_name}.{active_deployments_view_name}
382
381
  COPY GRANTS AS
383
382
  SELECT
384
383
  DEPLOYMENT_NAME,
@@ -417,14 +416,14 @@ class ModelRegistry:
417
416
  database_name: Desired name of the model registry database.
418
417
  schema_name: Desired name of the schema used by this model registry inside the database.
419
418
  """
420
- self._name = database_name
421
- self._schema = schema_name
422
- self._registry_table = _MODELS_TABLE_NAME
423
- self._registry_table_view = self._registry_table + "_VIEW"
424
- self._metadata_table = _METADATA_TABLE_NAME
425
- self._deployment_table = _DEPLOYMENT_TABLE_NAME
426
- self._permanent_deployment_view = self._deployment_table + "_VIEW"
427
- self._permanent_deployment_stage = self._deployment_table + "_STAGE"
419
+ self._name = identifier.get_inferred_name(database_name)
420
+ self._schema = identifier.get_inferred_name(schema_name)
421
+ self._registry_table = identifier.get_inferred_name(_MODELS_TABLE_NAME)
422
+ self._registry_table_view = identifier.concat_names([self._registry_table, "_VIEW"])
423
+ self._metadata_table = identifier.get_inferred_name(_METADATA_TABLE_NAME)
424
+ self._deployment_table = identifier.get_inferred_name(_DEPLOYMENT_TABLE_NAME)
425
+ self._permanent_deployment_view = identifier.concat_names([self._deployment_table, "_VIEW"])
426
+ self._permanent_deployment_stage = identifier.concat_names([self._deployment_table, "_STAGE"])
428
427
 
429
428
  self._session = session
430
429
 
@@ -441,23 +440,39 @@ class ModelRegistry:
441
440
  # Check that the required tables exist and are accessible by the current role.
442
441
 
443
442
  query_result_checker.SqlResultValidator(
444
- self._session, query=f"SHOW DATABASES LIKE '{self._name}'"
443
+ self._session, query=f"SHOW DATABASES LIKE '{identifier.get_unescaped_names(self._name)}'"
445
444
  ).has_dimensions(expected_rows=1).validate()
446
445
 
447
446
  query_result_checker.SqlResultValidator(
448
- self._session, query=f"SHOW SCHEMAS LIKE '{self._schema}' IN DATABASE \"{self._name}\""
447
+ self._session,
448
+ query=f"SHOW SCHEMAS LIKE '{identifier.get_unescaped_names(self._schema)}' IN DATABASE {self._name}",
449
449
  ).has_dimensions(expected_rows=1).validate()
450
450
 
451
451
  query_result_checker.SqlResultValidator(
452
- self._session, query=f"SHOW TABLES LIKE '{self._registry_table}' IN {self._fully_qualified_schema_name()}"
452
+ self._session,
453
+ query=formatting.unwrap(
454
+ f"""
455
+ SHOW TABLES LIKE '{identifier.get_unescaped_names(self._registry_table)}'
456
+ IN {self._fully_qualified_schema_name()}"""
457
+ ),
453
458
  ).has_dimensions(expected_rows=1).validate()
454
459
 
455
460
  query_result_checker.SqlResultValidator(
456
- self._session, query=f"SHOW TABLES LIKE '{self._metadata_table}' IN {self._fully_qualified_schema_name()}"
461
+ self._session,
462
+ query=formatting.unwrap(
463
+ f"""
464
+ SHOW TABLES LIKE '{identifier.get_unescaped_names(self._metadata_table)}'
465
+ IN {self._fully_qualified_schema_name()}"""
466
+ ),
457
467
  ).has_dimensions(expected_rows=1).validate()
458
468
 
459
469
  query_result_checker.SqlResultValidator(
460
- self._session, query=f"SHOW TABLES LIKE '{self._deployment_table}' IN {self._fully_qualified_schema_name()}"
470
+ self._session,
471
+ query=formatting.unwrap(
472
+ f"""
473
+ SHOW TABLES LIKE '{identifier.get_unescaped_names(self._deployment_table)}'
474
+ IN {self._fully_qualified_schema_name()}"""
475
+ ),
461
476
  ).has_dimensions(expected_rows=1).validate()
462
477
 
463
478
  # TODO(zzhu): Also check validity of views. Consider checking schema as well.
@@ -825,7 +840,7 @@ class ModelRegistry:
825
840
  before setting the metadata attribute. False by default meaning that by default we will check.
826
841
 
827
842
  Raises:
828
- DataError: Failed to set the meatdata attribute.
843
+ DataError: Failed to set the metadata attribute.
829
844
  KeyError: The target model doesn't exist
830
845
  """
831
846
  selected_models = self._list_selected_models(id=id, model_name=model_name, model_version=model_version)
@@ -955,13 +970,13 @@ class ModelRegistry:
955
970
  # Check if directory or file and adapt accordingly.
956
971
  # TODO: Unify and explicit about compression for both file and directory.
957
972
  if os.path.isfile(path):
958
- self._session.file.put(path, f"{fully_qualified_model_stage_name}/data")
973
+ self._session.file.put(path, posixpath.join(fully_qualified_model_stage_name, "data"))
959
974
  elif os.path.isdir(path):
960
975
  with file_utils.zip_file_or_directory_to_stream(path, path) as input_stream:
961
976
  self._session._conn.upload_stream(
962
977
  input_stream=input_stream,
963
978
  stage_location=fully_qualified_model_stage_name,
964
- dest_filename=f"{os.path.basename(path)}.zip",
979
+ dest_filename=f"{posixpath.basename(path)}.zip",
965
980
  dest_prefix="",
966
981
  source_compression="DEFLATE",
967
982
  compress_data=False,
@@ -1067,7 +1082,7 @@ class ModelRegistry:
1067
1082
  """
1068
1083
  # Explicitly not calling collect.
1069
1084
  return self._session.sql(
1070
- 'SELECT * FROM "{database}"."{schema}"."{view}"'.format(
1085
+ "SELECT * FROM {database}.{schema}.{view}".format(
1071
1086
  database=self._name, schema=self._schema, view=self._registry_table_view
1072
1087
  )
1073
1088
  )
@@ -1124,7 +1139,9 @@ class ModelRegistry:
1124
1139
  try:
1125
1140
  del model_tags[tag_name]
1126
1141
  except KeyError:
1127
- raise connector.DataError(f"Model id {id} has not tag named {tag_name}. Full list of tags: {model_tags}")
1142
+ raise connector.DataError(
1143
+ f"Model {model_name}/{model_version} has no tag named {tag_name}. Full list of tags: {model_tags}"
1144
+ )
1128
1145
 
1129
1146
  self._set_metadata_attribute(
1130
1147
  _METADATA_ATTRIBUTE_TAGS, model_tags, model_name=model_name, model_version=model_version
@@ -1227,7 +1244,7 @@ class ModelRegistry:
1227
1244
  result = self._get_metadata_attribute(
1228
1245
  _METADATA_ATTRIBUTE_DESCRIPTION, model_name=model_name, model_version=model_version
1229
1246
  )
1230
- return None if result is None else str(result)
1247
+ return None if result is None else json.loads(result)
1231
1248
 
1232
1249
  @telemetry.send_api_usage_telemetry(
1233
1250
  project=_TELEMETRY_PROJECT,
@@ -1453,6 +1470,8 @@ class ModelRegistry:
1453
1470
  pip_requirements: Optional[List[str]] = None,
1454
1471
  signatures: Optional[Dict[str, model_signature.ModelSignature]] = None,
1455
1472
  sample_input_data: Optional[Any] = None,
1473
+ code_paths: Optional[List[str]] = None,
1474
+ options: Optional[model_types.ModelSaveOption] = None,
1456
1475
  ) -> str:
1457
1476
  """Uploads and register a model to the Model Registry.
1458
1477
 
@@ -1472,6 +1491,8 @@ class ModelRegistry:
1472
1491
  signatures: Signatures of the model, which is a mapping from target method name to signatures of input and
1473
1492
  output, which could be inferred by calling `infer_signature` method with sample input data.
1474
1493
  sample_input_data: Sample of the input data for the model.
1494
+ code_paths: Directory of code to import when loading and deploying the model.
1495
+ options: Additional options when saving the model.
1475
1496
 
1476
1497
  Raises:
1477
1498
  TypeError: Raised when both signatures and sample_input_data is not presented. Will be captured locally.
@@ -1490,60 +1511,50 @@ class ModelRegistry:
1490
1511
  raise connector.DataError(f"Model {model_name}/{model_version} already exists. Unable to log the model.")
1491
1512
  with tempfile.TemporaryDirectory() as tmpdir:
1492
1513
  model = cast(model_types.SupportedModelType, model)
1493
- try:
1494
- if signatures:
1495
- model_api.save_model(
1496
- name=model_name,
1497
- model_dir_path=tmpdir,
1498
- model=model,
1499
- signatures=signatures,
1500
- metadata=tags,
1501
- conda_dependencies=conda_dependencies,
1502
- pip_requirements=pip_requirements,
1503
- )
1504
- elif sample_input_data is not None:
1505
- model_api.save_model(
1506
- name=model_name,
1507
- model_dir_path=tmpdir,
1508
- model=model,
1509
- metadata=tags,
1510
- conda_dependencies=conda_dependencies,
1511
- pip_requirements=pip_requirements,
1512
- sample_input=sample_input_data,
1513
- )
1514
- elif isinstance(model, base.BaseEstimator):
1515
- model_api.save_model(
1516
- name=model_name,
1517
- model_dir_path=tmpdir,
1518
- model=model,
1519
- metadata=tags,
1520
- conda_dependencies=conda_dependencies,
1521
- pip_requirements=pip_requirements,
1522
- )
1523
- else:
1524
- raise TypeError("Either signature or sample input data should exist for native model packaging.")
1525
- return self._log_model_path(
1526
- model_name=model_name,
1527
- model_version=model_version,
1528
- path=tmpdir,
1529
- type="snowflake_native",
1530
- description=description,
1531
- tags=tags, # TODO: Inherent model type enum.
1514
+ if signatures:
1515
+ model_metadata = model_api.save_model(
1516
+ name=model_name,
1517
+ model_dir_path=tmpdir,
1518
+ model=model,
1519
+ signatures=signatures,
1520
+ metadata=tags,
1521
+ conda_dependencies=conda_dependencies,
1522
+ pip_requirements=pip_requirements,
1523
+ code_paths=code_paths,
1524
+ options=options,
1532
1525
  )
1533
- except (AttributeError, TypeError):
1534
- pass
1535
-
1536
- with tempfile.NamedTemporaryFile(delete=True) as local_model_file:
1537
- cp.dump(model, local_model_file)
1538
- local_model_file.flush()
1539
-
1526
+ elif sample_input_data is not None:
1527
+ model_metadata = model_api.save_model(
1528
+ name=model_name,
1529
+ model_dir_path=tmpdir,
1530
+ model=model,
1531
+ metadata=tags,
1532
+ conda_dependencies=conda_dependencies,
1533
+ pip_requirements=pip_requirements,
1534
+ sample_input=sample_input_data,
1535
+ code_paths=code_paths,
1536
+ options=options,
1537
+ )
1538
+ elif isinstance(model, base.BaseEstimator):
1539
+ model_metadata = model_api.save_model(
1540
+ name=model_name,
1541
+ model_dir_path=tmpdir,
1542
+ model=model,
1543
+ metadata=tags,
1544
+ conda_dependencies=conda_dependencies,
1545
+ pip_requirements=pip_requirements,
1546
+ code_paths=code_paths,
1547
+ options=options,
1548
+ )
1549
+ else:
1550
+ raise TypeError("Either signature or sample input data should exist for native model packaging.")
1540
1551
  return self._log_model_path(
1541
1552
  model_name=model_name,
1542
1553
  model_version=model_version,
1543
- path=local_model_file.name,
1544
- type=model.__class__.__name__,
1554
+ path=tmpdir,
1555
+ type=model_metadata.model_type,
1545
1556
  description=description,
1546
- tags=tags,
1557
+ tags=tags, # TODO: Inherent model type enum.
1547
1558
  )
1548
1559
 
1549
1560
  @telemetry.send_api_usage_telemetry(
@@ -1565,22 +1576,13 @@ class ModelRegistry:
1565
1576
  restored_model = None
1566
1577
  with tempfile.TemporaryDirectory() as local_model_directory:
1567
1578
  self._session.file.get(remote_model_path, local_model_directory)
1568
- is_native_model_format = False
1569
- local_path = os.path.join(local_model_directory, os.path.basename(remote_model_path))
1570
- try:
1571
- if zipfile.is_zipfile(local_path):
1572
- extracted_dir = os.path.join(local_model_directory, "extracted")
1573
- with zipfile.ZipFile(local_path, "r") as myzip:
1574
- if len(myzip.namelist()) > 1:
1575
- myzip.extractall(extracted_dir)
1576
- restored_model, _meta = model_api.load_model(model_dir_path=extracted_dir)
1577
- is_native_model_format = True
1578
- except TypeError:
1579
- pass
1580
- if not is_native_model_format:
1581
- file_path = os.path.join(local_model_directory, os.path.basename(os.path.basename(remote_model_path)))
1582
- with open(file_path, mode="r+b") as model_file:
1583
- restored_model = cp.load(model_file)
1579
+ local_path = os.path.join(local_model_directory, posixpath.basename(remote_model_path))
1580
+ if zipfile.is_zipfile(local_path):
1581
+ extracted_dir = os.path.join(local_model_directory, "extracted")
1582
+ with zipfile.ZipFile(local_path, "r") as myzip:
1583
+ if len(myzip.namelist()) > 1:
1584
+ myzip.extractall(extracted_dir)
1585
+ restored_model, _ = model_api.load_model(model_dir_path=extracted_dir)
1584
1586
 
1585
1587
  return restored_model
1586
1588
 
snowflake/ml/version.py CHANGED
@@ -1 +1 @@
1
- VERSION="1.0.1"
1
+ VERSION="1.0.3"
@@ -1,11 +1,15 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: snowflake-ml-python
3
- Version: 1.0.1
4
- Description-Content-Type: text/markdown
5
3
  Author: Snowflake, Inc
6
4
  Author-email: support@snowflake.com
7
5
  Home-page: https://github.com/snowflakedb/snowflake-ml-python
8
6
  License: Apache License, Version 2.0
7
+ Description-Content-Type: text/markdown
8
+ Summary: The machine learning client library that is used for interacting with Snowflake to build machine learning solutions.
9
+ Project-URL: Changelog, https://github.com/snowflakedb/snowflake-ml-python/blob/main/CHANGELOG.md
10
+ Project-URL: Documentation, https://docs.snowflake.com/developer-guide/snowpark-ml
11
+ Project-URL: Issues, https://github.com/snowflakedb/snowflake-ml-python/issues
12
+ Project-URL: Source, https://github.com/snowflakedb/snowflake-ml-python
9
13
  Classifier: Development Status :: 3 - Alpha
10
14
  Classifier: Environment :: Console
11
15
  Classifier: Environment :: Other Environment
@@ -25,17 +29,15 @@ Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
25
29
  Classifier: Topic :: Software Development :: Libraries :: Python Modules
26
30
  Classifier: Topic :: Scientific/Engineering :: Information Analysis
27
31
  Requires-Python: >=3.8,<4
28
- Summary: The machine learning client library that is used for interacting with Snowflake to build machine learning solutions.
29
32
  Requires-Dist: absl-py>=0.15,<2
30
33
  Requires-Dist: anyio>=3.5.0,<4
31
34
  Requires-Dist: cloudpickle
32
- Requires-Dist: cryptography>=3.1.0,<39.0.0
33
35
  Requires-Dist: fsspec[http]>=2022.11,<=2023.1
34
36
  Requires-Dist: numpy>=1.23,<2
35
37
  Requires-Dist: packaging>=20.9,<24
36
38
  Requires-Dist: pandas>=1.0.0,<2
37
39
  Requires-Dist: pyyaml>=6.0,<7
38
- Requires-Dist: scikit-learn>=1.2.1,<2
40
+ Requires-Dist: scikit-learn>=1.2.1,<1.3
39
41
  Requires-Dist: scipy>=1.9,<2
40
42
  Requires-Dist: snowflake-connector-python[pandas]>=3.0.3,<4
41
43
  Requires-Dist: snowflake-snowpark-python>=1.4.0,<2
@@ -52,6 +54,7 @@ Provides-Extra: tensorflow
52
54
  Requires-Dist: tensorflow>=2.9,<3; extra == 'tensorflow'
53
55
  Provides-Extra: torch
54
56
  Requires-Dist: torchdata>=0.4,<1; extra == 'torch'
57
+ Version: 1.0.3
55
58
 
56
59
  # Snowpark ML
57
60
 
@@ -77,9 +80,7 @@ Snowpark MLOps complements the Snowpark ML Development API, and provides model m
77
80
 
78
81
  During PrPr, we are iterating on API without backward compatibility guarantees. It is better to recreate your registry everytime you update the package. This means, at this time, you cannot use the registry for production use.
79
82
 
80
- - [Documentation](http://docs.snowflake.com/developer-guide/snowpark/python/snowpark-ml-modeling)
81
- - [Issues](https://github.com/snowflakedb/snowflake-ml-python/issues)
82
- - [Source](https://github.com/snowflakedb/snowflake-ml-python/)
83
+ - [Documentation](https://docs.snowflake.com/developer-guide/snowpark-ml)
83
84
 
84
85
  ## Getting started
85
86
  ### Have your Snowflake account ready
@@ -96,6 +97,47 @@ pip install snowflake-ml-python
96
97
  ```
97
98
  # Release History
98
99
 
100
+ ## 1.0.3 (2023-07-14)
101
+
102
+ ### Behavior Changes
103
+ - Model Registry: When predicting a model whose output is a list of NumPy ndarray, the output would not be flattened, instead, every ndarray will act as a feature(column) in the output.
104
+
105
+ ### New Features
106
+ - Model Registry: Added support save/load/deploy PyTorch models (`torch.nn.Module` and `torch.jit.ScriptModule`).
107
+
108
+ ### Bug Fixes
109
+
110
+ - Model Registry: Fix an issue that when database or schema name provided to `create_model_registry` contains special characters, the model registry cannot be created.
111
+ - Model Registry: Fix an issue that `get_model_description` returns with additional quotes.
112
+ - Model Registry: Fix incorrect error message when attempting to remove a unset tag of a model.
113
+ - Model Registry: Fix a typo in the default deployment table name.
114
+ - Model Registry: Snowpark dataframe for sample input or input for `predict` method that contains a column with Snowflake `NUMBER(precision, scale)` data type where `scale = 0` will not lead to error, and will now correctly recognized as `INT64` data type in model signature.
115
+ - Model Registry: Fix an issue that prevent model logged in the system whose default encoding is not UTF-8 compatible from deploying.
116
+ - Model Registry: Added earlier and better error message when any file name in the model or the file name of model itself contains characters that are unable to be encoded using ASCII. It is currently not supported to deploy such a model.
117
+
118
+ ## 1.0.2 (2023-06-22)
119
+
120
+ ### Behavior Changes
121
+ - Model Registry: Prohibit non-snowflake-native models from being logged.
122
+ - Model Registry: `_use_local_snowml` parameter in options of `deploy()` has been removed.
123
+ - Model Registry: A default `False` `embed_local_ml_library` parameter has been added to the options of `log_model()`. With this set to `False` (default), the version of the local snowflake-ml-python library will be recorded and used when deploying the model. With this set to `True`, local snowflake-ml-python library will be embedded into the logged model, and will be used when you load or deploy the model.
124
+
125
+ ### New Features
126
+ - Model Registry: A new optional argument named `code_paths` has been added to the arguments of `log_model()` for users to specify additional code paths to be imported when loading and deploying the model.
127
+ - Model Registry: A new optional argument named `options` has been added to the arguments of `log_model()` to specify any additional options when saving the model.
128
+ - Model Development: Added metrics:
129
+ - d2_absolute_error_score
130
+ - d2_pinball_score
131
+ - explained_variance_score
132
+ - mean_absolute_error
133
+ - mean_absolute_percentage_error
134
+ - mean_squared_error
135
+
136
+ ### Bug Fixes
137
+
138
+ - Model Development: `accuracy_score()` now works when given label column names are lists of a single value.
139
+
140
+
99
141
  ## 1.0.1 (2023-06-16)
100
142
  ### Behavior Changes
101
143