databricks-sdk 0.49.0__tar.gz → 0.51.0__tar.gz

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/CHANGELOG.md

@@ -1,5 +1,87 @@
 # Version changelog
 
+## Release v0.51.0
+
+### New Features and Improvements
+* Enabled asynchronous token refreshes by default. A new `disable_async_token_refresh` configuration option has been added to allow disabling this feature if necessary ([#952](https://github.com/databricks/databricks-sdk-py/pull/952)).
+  To disable asynchronous token refresh, set the environment variable `DATABRICKS_DISABLE_ASYNC_TOKEN_REFRESH=true` or configure it within your configuration object.
+  The previous `enable_experimental_async_token_refresh` option has been removed as asynchronous refresh is now the default behavior.
+* Introduce support for Databricks Workload Identity Federation in GitHub workflows ([933](https://github.com/databricks/databricks-sdk-py/pull/933)).
+  See README.md for instructions.
+* [Breaking] Users running their workflows in GitHub Actions, which use Cloud native authentication and also have a `DATABRICKS_CLIENT_ID` and `DATABRICKS_HOST`
+  environment variables set may see their authentication start failing due to the order in which the SDK tries different authentication methods.
+
+### API Changes
+* Added [w.alerts_v2](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/sql/alerts_v2.html) workspace-level service.
+* Added `update_ncc_azure_private_endpoint_rule_public()` method for [a.network_connectivity](https://databricks-sdk-py.readthedocs.io/en/latest/account/settings/network_connectivity.html) account-level service.
+* Added `update_endpoint_budget_policy()` and `update_endpoint_custom_tags()` methods for [w.vector_search_endpoints](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_endpoints.html) workspace-level service.
+* Added `created_at`, `created_by` and `metastore_id` fields for `databricks.sdk.service.catalog.SetArtifactAllowlist`.
+* Added `node_type_flexibility` field for `databricks.sdk.service.compute.EditInstancePool`.
+* Added `page_size` and `page_token` fields for `databricks.sdk.service.compute.GetEvents`.
+* Added `next_page_token` and `prev_page_token` fields for `databricks.sdk.service.compute.GetEventsResponse`.
+* Added `node_type_flexibility` field for `databricks.sdk.service.compute.GetInstancePool`.
+* Added `node_type_flexibility` field for `databricks.sdk.service.compute.InstancePoolAndStats`.
+* Added `effective_performance_target` field for `databricks.sdk.service.jobs.RepairHistoryItem`.
+* Added `performance_target` field for `databricks.sdk.service.jobs.RepairRun`.
+* [Breaking] Added `network_connectivity_config` field for `databricks.sdk.service.settings.CreateNetworkConnectivityConfigRequest`.
+* [Breaking] Added `private_endpoint_rule` field for `databricks.sdk.service.settings.CreatePrivateEndpointRuleRequest`.
+* Added `domain_names` field for `databricks.sdk.service.settings.NccAzurePrivateEndpointRule`.
+* Added `auto_resolve_display_name` field for `databricks.sdk.service.sql.CreateAlertRequest`.
+* Added `auto_resolve_display_name` field for `databricks.sdk.service.sql.CreateQueryRequest`.
+* Added `budget_policy_id` field for `databricks.sdk.service.vectorsearch.CreateEndpoint`.
+* Added `custom_tags` and `effective_budget_policy_id` fields for `databricks.sdk.service.vectorsearch.EndpointInfo`.
+* Added `create_clean_room`, `execute_clean_room_task` and `modify_clean_room` enum values for `databricks.sdk.service.catalog.Privilege`.
+* Added `dns_resolution_error` and `gcp_denied_by_org_policy` enum values for `databricks.sdk.service.compute.TerminationReasonCode`.
+* Added `disabled` enum value for `databricks.sdk.service.jobs.TerminationCodeCode`.
+* Added `expired` enum value for `databricks.sdk.service.settings.NccAzurePrivateEndpointRuleConnectionState`.
+* [Breaking] Changed `create_network_connectivity_configuration()` and `create_private_endpoint_rule()` methods for [a.network_connectivity](https://databricks-sdk-py.readthedocs.io/en/latest/account/settings/network_connectivity.html) account-level service with new required argument order.
+* [Breaking] Changed `create_index()` method for [w.vector_search_indexes](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_indexes.html) workspace-level service to return `databricks.sdk.service.vectorsearch.VectorIndex` dataclass.
+* [Breaking] Changed `delete_data_vector_index()` method for [w.vector_search_indexes](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_indexes.html) workspace-level service . HTTP method/verb has changed.
+* [Breaking] Changed `delete_data_vector_index()` method for [w.vector_search_indexes](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_indexes.html) workspace-level service with new required argument order.
+* [Breaking] Changed `databricks.sdk.service.vectorsearch.List` dataclass to.
+* [Breaking] Changed `workload_size` field for `databricks.sdk.service.serving.ServedModelInput` to type `str` dataclass.
+* [Breaking] Changed `group_id` field for `databricks.sdk.service.settings.NccAzurePrivateEndpointRule` to type `str` dataclass.
+* [Breaking] Changed `target_services` field for `databricks.sdk.service.settings.NccAzureServiceEndpointRule` to type `databricks.sdk.service.settings.EgressResourceTypeList` dataclass.
+* [Breaking] Changed `data_array` field for `databricks.sdk.service.vectorsearch.ResultData` to type `databricks.sdk.service.vectorsearch.ListValueList` dataclass.
+* [Breaking] Changed waiter for [VectorSearchEndpointsAPI.create_endpoint](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_endpoints.html#databricks.sdk.service.vectorsearch.VectorSearchEndpointsAPI.create_endpoint) method.
+* [Breaking] Removed `name` and `region` fields for `databricks.sdk.service.settings.CreateNetworkConnectivityConfigRequest`.
+* [Breaking] Removed `group_id` and `resource_id` fields for `databricks.sdk.service.settings.CreatePrivateEndpointRuleRequest`.
+* [Breaking] Removed `null_value` field for `databricks.sdk.service.vectorsearch.Value`.
+* [Breaking] Removed `large`, `medium` and `small` enum values for `databricks.sdk.service.serving.ServedModelInputWorkloadSize`.
+* [Breaking] Removed `blob`, `dfs`, `mysql_server` and `sql_server` enum values for `databricks.sdk.service.settings.NccAzurePrivateEndpointRuleGroupId`.
+
+
+## Release v0.50.0
+
+### API Changes
+* Added [w.enable_export_notebook](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/settings/settings/enable_export_notebook.html) workspace-level service, [w.enable_notebook_table_clipboard](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/settings/settings/enable_notebook_table_clipboard.html) workspace-level service and [w.enable_results_downloading](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/settings/settings/enable_results_downloading.html) workspace-level service.
+* Added `get_credentials_for_trace_data_download()` and `get_credentials_for_trace_data_upload()` methods for [w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/ml/experiments.html) workspace-level service.
+* Added `get_download_full_query_result()` method for [w.genie](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/dashboards/genie.html) workspace-level service.
+* Added `get_published_dashboard_token_info()` method for [w.lakeview_embedded](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/dashboards/lakeview_embedded.html) workspace-level service.
+* Added `binding_workspace_ids` field for `databricks.sdk.service.billing.BudgetPolicy`.
+* Added `download_id` field for `databricks.sdk.service.dashboards.GenieGenerateDownloadFullQueryResultResponse`.
+* Added `dashboard_output` field for `databricks.sdk.service.jobs.RunOutput`.
+* Added `dashboard_task` and `power_bi_task` fields for `databricks.sdk.service.jobs.RunTask`.
+* Added `dashboard_task` and `power_bi_task` fields for `databricks.sdk.service.jobs.SubmitTask`.
+* Added `dashboard_task` and `power_bi_task` fields for `databricks.sdk.service.jobs.Task`.
+* Added `include_features` field for `databricks.sdk.service.ml.CreateForecastingExperimentRequest`.
+* Added `models` field for `databricks.sdk.service.ml.LogInputs`.
+* Added `dataset_digest`, `dataset_name` and `model_id` fields for `databricks.sdk.service.ml.LogMetric`.
+* Added `dataset_digest`, `dataset_name`, `model_id` and `run_id` fields for `databricks.sdk.service.ml.Metric`.
+* Added `model_inputs` field for `databricks.sdk.service.ml.RunInputs`.
+* Added `client_application` field for `databricks.sdk.service.sql.QueryInfo`.
+* Added `geography` and `geometry` enum values for `databricks.sdk.service.catalog.ColumnTypeName`.
+* Added `allocation_timeout_no_healthy_and_warmed_up_clusters`, `docker_container_creation_exception`, `docker_image_too_large_for_instance_exception` and `docker_invalid_os_exception` enum values for `databricks.sdk.service.compute.TerminationReasonCode`.
+* Added `standard` enum value for `databricks.sdk.service.jobs.PerformanceTarget`.
+* Added `can_view` enum value for `databricks.sdk.service.sql.WarehousePermissionLevel`.
+* [Breaking] Changed `generate_download_full_query_result()` method for [w.genie](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/dashboards/genie.html) workspace-level service . Method path has changed.
+* [Breaking] Changed waiter for [CommandExecutionAPI.create](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/compute/command_execution.html#databricks.sdk.service.compute.CommandExecutionAPI.create) method.
+* [Breaking] Changed waiter for [CommandExecutionAPI.execute](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/compute/command_execution.html#databricks.sdk.service.compute.CommandExecutionAPI.execute) method.
+* [Breaking] Removed `error`, `status` and `transient_statement_id` fields for `databricks.sdk.service.dashboards.GenieGenerateDownloadFullQueryResultResponse`.
+* [Breaking] Removed `balanced` and `cost_optimized` enum values for `databricks.sdk.service.jobs.PerformanceTarget`.
+* [Breaking] Removed [PipelinesAPI.wait_get_pipeline_running](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/pipelines/pipelines.html#databricks.sdk.service.pipelines.PipelinesAPI.wait_get_pipeline_running) method.
+
+
 ## Release v0.49.0
 
 ### API Changes

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: databricks-sdk
-Version: 0.49.0
+Version: 0.51.0
 Summary: Databricks SDK for Python (Beta)
 Project-URL: Documentation, https://databricks-sdk-py.readthedocs.io
 Keywords: databricks,sdk
@@ -180,18 +180,18 @@ Depending on the Databricks authentication method, the SDK uses the following in
 
 ### Databricks native authentication
 
-By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks basic (username/password) authentication (`auth_type="basic"` argument).
+By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks Workload Identity Federation (WIF) authentication using OIDC (`auth_type="github-oidc"` argument).
 
 - For Databricks token authentication, you must provide `host` and `token`; or their environment variable or `.databrickscfg` file field equivalents.
-- For Databricks basic authentication, you must provide `host`, `username`, and `password` _(for AWS workspace-level operations)_; or `host`, `account_id`, `username`, and `password` _(for AWS, Azure, or GCP account-level operations)_; or their environment variable or `.databrickscfg` file field equivalents.
-
-| Argument     | Description | Environment variable |
-|--------------|-------------|-------------------|
-| `host`       | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` |     
-| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_. | `DATABRICKS_ACCOUNT_ID` |
-| `token`      | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` |
-| `username`   | _(String)_ The Databricks username part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_USERNAME` |
-| `password`   | _(String)_ The Databricks password part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_PASSWORD` |
+- For Databricks OIDC authentication, you must provide the `host`, `client_id` and `token_audience` _(optional)_ either directly, through the corresponding environment variables, or in your `.databrickscfg` configuration file.
+
+| Argument         | Description                                                                                                                                                                                                                                                               | Environment variable    |
+|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
+| `host`           | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint.                                                                                                                                                      | `DATABRICKS_HOST`       |     
+| `account_id`     | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_.  | `DATABRICKS_ACCOUNT_ID` |
+| `token`          | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_.                                                                                                                                       | `DATABRICKS_TOKEN`      |
+| `client_id`      | _(String)_ The Databricks Service Principal Application ID.                                                                                                                                                                                                               | `DATABRICKS_CLIENT_ID`  |
+| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier.                                                                                                                                         | `TOKEN_AUDIENCE`        |
 
 For example, to use Databricks token authentication:
 

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/README.md

@@ -126,18 +126,18 @@ Depending on the Databricks authentication method, the SDK uses the following in
 
 ### Databricks native authentication
 
-By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks basic (username/password) authentication (`auth_type="basic"` argument).
+By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks Workload Identity Federation (WIF) authentication using OIDC (`auth_type="github-oidc"` argument).
 
 - For Databricks token authentication, you must provide `host` and `token`; or their environment variable or `.databrickscfg` file field equivalents.
-- For Databricks basic authentication, you must provide `host`, `username`, and `password` _(for AWS workspace-level operations)_; or `host`, `account_id`, `username`, and `password` _(for AWS, Azure, or GCP account-level operations)_; or their environment variable or `.databrickscfg` file field equivalents.
-
-| Argument     | Description | Environment variable |
-|--------------|-------------|-------------------|
-| `host`       | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` |     
-| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_. | `DATABRICKS_ACCOUNT_ID` |
-| `token`      | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` |
-| `username`   | _(String)_ The Databricks username part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_USERNAME` |
-| `password`   | _(String)_ The Databricks password part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_PASSWORD` |
+- For Databricks OIDC authentication, you must provide the `host`, `client_id` and `token_audience` _(optional)_ either directly, through the corresponding environment variables, or in your `.databrickscfg` configuration file.
+
+| Argument         | Description                                                                                                                                                                                                                                                               | Environment variable    |
+|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
+| `host`           | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint.                                                                                                                                                      | `DATABRICKS_HOST`       |     
+| `account_id`     | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_.  | `DATABRICKS_ACCOUNT_ID` |
+| `token`          | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_.                                                                                                                                       | `DATABRICKS_TOKEN`      |
+| `client_id`      | _(String)_ The Databricks Service Principal Application ID.                                                                                                                                                                                                               | `DATABRICKS_CLIENT_ID`  |
+| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier.                                                                                                                                         | `TOKEN_AUDIENCE`        |
 
 For example, to use Databricks token authentication:
 

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/databricks/sdk/__init__.py

@@ -86,19 +86,21 @@ from databricks.sdk.service.settings import (
     AibiDashboardEmbeddingApprovedDomainsAPI, AutomaticClusterUpdateAPI,
     ComplianceSecurityProfileAPI, CredentialsManagerAPI,
     CspEnablementAccountAPI, DefaultNamespaceAPI, DisableLegacyAccessAPI,
-    DisableLegacyDbfsAPI, DisableLegacyFeaturesAPI, EnableIpAccessListsAPI,
-    EnhancedSecurityMonitoringAPI, EsmEnablementAccountAPI, IpAccessListsAPI,
-    NetworkConnectivityAPI, NotificationDestinationsAPI, PersonalComputeAPI,
+    DisableLegacyDbfsAPI, DisableLegacyFeaturesAPI, EnableExportNotebookAPI,
+    EnableIpAccessListsAPI, EnableNotebookTableClipboardAPI,
+    EnableResultsDownloadingAPI, EnhancedSecurityMonitoringAPI,
+    EsmEnablementAccountAPI, IpAccessListsAPI, NetworkConnectivityAPI,
+    NotificationDestinationsAPI, PersonalComputeAPI,
     RestrictWorkspaceAdminsAPI, SettingsAPI, TokenManagementAPI, TokensAPI,
     WorkspaceConfAPI)
 from databricks.sdk.service.sharing import (ProvidersAPI,
                                             RecipientActivationAPI,
                                             RecipientsAPI, SharesAPI)
 from databricks.sdk.service.sql import (AlertsAPI, AlertsLegacyAPI,
-                                        DashboardsAPI, DashboardWidgetsAPI,
-                                        DataSourcesAPI, DbsqlPermissionsAPI,
-                                        QueriesAPI, QueriesLegacyAPI,
-                                        QueryHistoryAPI,
+                                        AlertsV2API, DashboardsAPI,
+                                        DashboardWidgetsAPI, DataSourcesAPI,
+                                        DbsqlPermissionsAPI, QueriesAPI,
+                                        QueriesLegacyAPI, QueryHistoryAPI,
                                         QueryVisualizationsAPI,
                                         QueryVisualizationsLegacyAPI,
                                         RedashConfigAPI, StatementExecutionAPI,
@@ -168,6 +170,7 @@ class WorkspaceClient:
         product_version="0.0.0",
         credentials_strategy: Optional[CredentialsStrategy] = None,
         credentials_provider: Optional[CredentialsStrategy] = None,
+        token_audience: Optional[str] = None,
         config: Optional[client.Config] = None,
     ):
         if not config:
@@ -196,6 +199,7 @@ class WorkspaceClient:
                 debug_headers=debug_headers,
                 product=product,
                 product_version=product_version,
+                token_audience=token_audience,
             )
         self._config = config.copy()
         self._dbutils = _make_dbutils(self._config)
@@ -205,6 +209,7 @@ class WorkspaceClient:
         self._account_access_control_proxy = service.iam.AccountAccessControlProxyAPI(self._api_client)
         self._alerts = service.sql.AlertsAPI(self._api_client)
         self._alerts_legacy = service.sql.AlertsLegacyAPI(self._api_client)
+        self._alerts_v2 = service.sql.AlertsV2API(self._api_client)
         self._apps = service.apps.AppsAPI(self._api_client)
         self._artifact_allowlists = service.catalog.ArtifactAllowlistsAPI(self._api_client)
         self._catalogs = service.catalog.CatalogsAPI(self._api_client)
@@ -287,7 +292,7 @@ class WorkspaceClient:
         self._service_principals = service.iam.ServicePrincipalsAPI(self._api_client)
         self._serving_endpoints = serving_endpoints
         serving_endpoints_data_plane_token_source = DataPlaneTokenSource(
-            self._config.host, self._config.oauth_token, not self._config.enable_experimental_async_token_refresh
+            self._config.host, self._config.oauth_token, self._config.disable_async_token_refresh
         )
         self._serving_endpoints_data_plane = service.serving.ServingEndpointsDataPlaneAPI(
             self._api_client, serving_endpoints, serving_endpoints_data_plane_token_source
@@ -344,6 +349,11 @@ class WorkspaceClient:
         """The alerts API can be used to perform CRUD operations on alerts."""
         return self._alerts_legacy
 
+    @property
+    def alerts_v2(self) -> service.sql.AlertsV2API:
+        """TODO: Add description."""
+        return self._alerts_v2
+
     @property
     def apps(self) -> service.apps.AppsAPI:
         """Apps run directly on a customer’s Databricks instance, integrate with their data, use and extend Databricks services, and enable users to interact through single sign-on."""
@@ -860,6 +870,7 @@ class AccountClient:
         product_version="0.0.0",
         credentials_strategy: Optional[CredentialsStrategy] = None,
         credentials_provider: Optional[CredentialsStrategy] = None,
+        token_audience: Optional[str] = None,
         config: Optional[client.Config] = None,
     ):
         if not config:
@@ -888,6 +899,7 @@ class AccountClient:
                 debug_headers=debug_headers,
                 product=product,
                 product_version=product_version,
+                token_audience=token_audience,
             )
         self._config = config.copy()
         self._api_client = client.ApiClient(self._config)

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/databricks/sdk/config.py

@@ -61,6 +61,7 @@ class Config:
     host: str = ConfigAttribute(env="DATABRICKS_HOST")
     account_id: str = ConfigAttribute(env="DATABRICKS_ACCOUNT_ID")
     token: str = ConfigAttribute(env="DATABRICKS_TOKEN", auth="pat", sensitive=True)
+    token_audience: str = ConfigAttribute(env="DATABRICKS_TOKEN_AUDIENCE", auth="github-oidc")
     username: str = ConfigAttribute(env="DATABRICKS_USERNAME", auth="basic")
     password: str = ConfigAttribute(env="DATABRICKS_PASSWORD", auth="basic", sensitive=True)
     client_id: str = ConfigAttribute(env="DATABRICKS_CLIENT_ID", auth="oauth")
@@ -95,9 +96,7 @@ class Config:
     max_connections_per_pool: int = ConfigAttribute()
     databricks_environment: Optional[DatabricksEnvironment] = None
 
-    enable_experimental_async_token_refresh: bool = ConfigAttribute(
-        env="DATABRICKS_ENABLE_EXPERIMENTAL_ASYNC_TOKEN_REFRESH"
-    )
+    disable_async_token_refresh: bool = ConfigAttribute(env="DATABRICKS_DISABLE_ASYNC_TOKEN_REFRESH")
 
     enable_experimental_files_api_client: bool = ConfigAttribute(env="DATABRICKS_ENABLE_EXPERIMENTAL_FILES_API_CLIENT")
     files_api_client_download_max_total_recovers = None

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/databricks/sdk/credentials_provider.py

@@ -23,6 +23,7 @@ from google.oauth2 import service_account  # type: ignore
 from .azure import add_sp_management_token, add_workspace_id_header
 from .oauth import (ClientCredentials, OAuthClient, Refreshable, Token,
                     TokenCache, TokenSource)
+from .oidc_token_supplier import GitHubOIDCTokenSupplier
 
 CredentialsProvider = Callable[[], Dict[str, str]]
 
@@ -191,7 +192,7 @@ def oauth_service_principal(cfg: "Config") -> Optional[CredentialsProvider]:
         token_url=oidc.token_endpoint,
         scopes=["all-apis"],
         use_header=True,
-        disable_async=not cfg.enable_experimental_async_token_refresh,
+        disable_async=cfg.disable_async_token_refresh,
     )
 
     def inner() -> Dict[str, str]:
@@ -291,7 +292,7 @@ def azure_service_principal(cfg: "Config") -> CredentialsProvider:
             token_url=f"{aad_endpoint}{cfg.azure_tenant_id}/oauth2/token",
             endpoint_params={"resource": resource},
             use_params=True,
-            disable_async=not cfg.enable_experimental_async_token_refresh,
+            disable_async=cfg.disable_async_token_refresh,
         )
 
     _ensure_host_present(cfg, token_source_for)
@@ -314,6 +315,58 @@ def azure_service_principal(cfg: "Config") -> CredentialsProvider:
     return OAuthCredentialsProvider(refreshed_headers, token)
 
 
+@oauth_credentials_strategy("github-oidc", ["host", "client_id"])
+def databricks_wif(cfg: "Config") -> Optional[CredentialsProvider]:
+    """
+    DatabricksWIFCredentials uses a Token Supplier to get a JWT Token and exchanges
+    it for a Databricks Token.
+
+    Supported suppliers:
+    - GitHub OIDC
+    """
+    supplier = GitHubOIDCTokenSupplier()
+
+    audience = cfg.token_audience
+    if audience is None and cfg.is_account_client:
+        audience = cfg.account_id
+    if audience is None and not cfg.is_account_client:
+        audience = cfg.oidc_endpoints.token_endpoint
+
+    # Try to get an idToken. If no supplier returns a token, we cannot use this authentication mode.
+    id_token = supplier.get_oidc_token(audience)
+    if not id_token:
+        return None
+
+    def token_source_for(audience: str) -> TokenSource:
+        id_token = supplier.get_oidc_token(audience)
+        if not id_token:
+            # Should not happen, since we checked it above.
+            raise Exception("Cannot get OIDC token")
+        params = {
+            "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
+            "subject_token": id_token,
+            "grant_type": "urn:ietf:params:oauth:grant-type:token-exchange",
+        }
+        return ClientCredentials(
+            client_id=cfg.client_id,
+            client_secret="",  # we have no (rotatable) secrets in OIDC flow
+            token_url=cfg.oidc_endpoints.token_endpoint,
+            endpoint_params=params,
+            scopes=["all-apis"],
+            use_params=True,
+            disable_async=cfg.disable_async_token_refresh,
+        )
+
+    def refreshed_headers() -> Dict[str, str]:
+        token = token_source_for(audience).token()
+        return {"Authorization": f"{token.token_type} {token.access_token}"}
+
+    def token() -> Token:
+        return token_source_for(audience).token()
+
+    return OAuthCredentialsProvider(refreshed_headers, token)
+
+
 @oauth_credentials_strategy("github-oidc-azure", ["host", "azure_client_id"])
 def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
     if "ACTIONS_ID_TOKEN_REQUEST_TOKEN" not in os.environ:
@@ -325,16 +378,8 @@ def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
     if not cfg.is_azure:
         return None
 
-    # See https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-cloud-providers
-    headers = {"Authorization": f"Bearer {os.environ['ACTIONS_ID_TOKEN_REQUEST_TOKEN']}"}
-    endpoint = f"{os.environ['ACTIONS_ID_TOKEN_REQUEST_URL']}&audience=api://AzureADTokenExchange"
-    response = requests.get(endpoint, headers=headers)
-    if not response.ok:
-        return None
-
-    # get the ID Token with aud=api://AzureADTokenExchange sub=repo:org/repo:environment:name
-    response_json = response.json()
-    if "value" not in response_json:
+    token = GitHubOIDCTokenSupplier().get_oidc_token("api://AzureADTokenExchange")
+    if not token:
         return None
 
     logger.info(
@@ -344,7 +389,7 @@ def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
     params = {
         "client_assertion_type": "urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
         "resource": cfg.effective_azure_login_app_id,
-        "client_assertion": response_json["value"],
+        "client_assertion": token,
     }
     aad_endpoint = cfg.arm_environment.active_directory_endpoint
     if not cfg.azure_tenant_id:
@@ -357,7 +402,7 @@ def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
         token_url=f"{aad_endpoint}{cfg.azure_tenant_id}/oauth2/token",
         endpoint_params=params,
         use_params=True,
-        disable_async=not cfg.enable_experimental_async_token_refresh,
+        disable_async=cfg.disable_async_token_refresh,
     )
 
     def refreshed_headers() -> Dict[str, str]:
@@ -694,7 +739,7 @@ class DatabricksCliTokenSource(CliTokenSource):
             token_type_field="token_type",
             access_token_field="access_token",
             expiry_field="expiry",
-            disable_async=not cfg.enable_experimental_async_token_refresh,
+            disable_async=cfg.disable_async_token_refresh,
         )
 
     @staticmethod
@@ -927,6 +972,7 @@ class DefaultCredentials:
             basic_auth,
             metadata_service,
             oauth_service_principal,
+            databricks_wif,
             azure_service_principal,
             github_oidc_azure,
             azure_cli,

databricks_sdk-0.51.0/databricks/sdk/oidc_token_supplier.py

@@ -0,0 +1,28 @@
+import os
+from typing import Optional
+
+import requests
+
+
+class GitHubOIDCTokenSupplier:
+    """
+    Supplies OIDC tokens from GitHub Actions.
+    """
+
+    def get_oidc_token(self, audience: str) -> Optional[str]:
+        if "ACTIONS_ID_TOKEN_REQUEST_TOKEN" not in os.environ or "ACTIONS_ID_TOKEN_REQUEST_URL" not in os.environ:
+            # not in GitHub actions
+            return None
+        # See https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-cloud-providers
+        headers = {"Authorization": f"Bearer {os.environ['ACTIONS_ID_TOKEN_REQUEST_TOKEN']}"}
+        endpoint = f"{os.environ['ACTIONS_ID_TOKEN_REQUEST_URL']}&audience={audience}"
+        response = requests.get(endpoint, headers=headers)
+        if not response.ok:
+            return None
+
+        # get the ID Token with aud=api://AzureADTokenExchange sub=repo:org/repo:environment:name
+        response_json = response.json()
+        if "value" not in response_json:
+            return None
+
+        return response_json["value"]

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/databricks/sdk/service/apps.py

@@ -1173,12 +1173,12 @@ class AppsAPI:
             attempt += 1
         raise TimeoutError(f"timed out after {timeout}: {status_message}")
 
-    def create(self, *, app: Optional[App] = None, no_compute: Optional[bool] = None) -> Wait[App]:
+    def create(self, app: App, *, no_compute: Optional[bool] = None) -> Wait[App]:
         """Create an app.
 
         Creates a new app.
 
-        :param app: :class:`App` (optional)
+        :param app: :class:`App`
         :param no_compute: bool (optional)
           If true, the app will not be started after creation.
 
@@ -1198,9 +1198,7 @@ class AppsAPI:
         op_response = self._api.do("POST", "/api/2.0/apps", query=query, body=body, headers=headers)
         return Wait(self.wait_get_app_active, response=App.from_dict(op_response), name=op_response["name"])
 
-    def create_and_wait(
-        self, *, app: Optional[App] = None, no_compute: Optional[bool] = None, timeout=timedelta(minutes=20)
-    ) -> App:
+    def create_and_wait(self, app: App, *, no_compute: Optional[bool] = None, timeout=timedelta(minutes=20)) -> App:
         return self.create(app=app, no_compute=no_compute).result(timeout=timeout)
 
     def delete(self, name: str) -> App:
@@ -1221,14 +1219,14 @@ class AppsAPI:
         res = self._api.do("DELETE", f"/api/2.0/apps/{name}", headers=headers)
         return App.from_dict(res)
 
-    def deploy(self, app_name: str, *, app_deployment: Optional[AppDeployment] = None) -> Wait[AppDeployment]:
+    def deploy(self, app_name: str, app_deployment: AppDeployment) -> Wait[AppDeployment]:
         """Create an app deployment.
 
         Creates an app deployment for the app with the supplied name.
 
         :param app_name: str
           The name of the app.
-        :param app_deployment: :class:`AppDeployment` (optional)
+        :param app_deployment: :class:`AppDeployment`
 
         :returns:
           Long-running operation waiter for :class:`AppDeployment`.
@@ -1249,7 +1247,7 @@ class AppsAPI:
         )
 
     def deploy_and_wait(
-        self, app_name: str, *, app_deployment: Optional[AppDeployment] = None, timeout=timedelta(minutes=20)
+        self, app_name: str, app_deployment: AppDeployment, timeout=timedelta(minutes=20)
     ) -> AppDeployment:
         return self.deploy(app_deployment=app_deployment, app_name=app_name).result(timeout=timeout)
 
@@ -1466,7 +1464,7 @@ class AppsAPI:
     def stop_and_wait(self, name: str, timeout=timedelta(minutes=20)) -> App:
         return self.stop(name=name).result(timeout=timeout)
 
-    def update(self, name: str, *, app: Optional[App] = None) -> App:
+    def update(self, name: str, app: App) -> App:
         """Update an app.
 
         Updates the app with the supplied name.
@@ -1474,7 +1472,7 @@ class AppsAPI:
         :param name: str
           The name of the app. The name must contain only lowercase alphanumeric characters and hyphens. It
           must be unique within the workspace.
-        :param app: :class:`App` (optional)
+        :param app: :class:`App`
 
         :returns: :class:`App`
         """

{databricks_sdk-0.49.0 → databricks_sdk-0.51.0}/databricks/sdk/service/billing.py

@@ -364,6 +364,10 @@ class BudgetConfigurationFilterWorkspaceIdClause:
 class BudgetPolicy:
     """Contains the BudgetPolicy details."""
 
+    binding_workspace_ids: Optional[List[int]] = None
+    """List of workspaces that this budget policy will be exclusively bound to. An empty binding
+    implies that this budget policy is open to any workspace in the account."""
+
     custom_tags: Optional[List[compute.CustomPolicyTag]] = None
     """A list of tags defined by the customer. At most 20 entries are allowed per policy."""
 
@@ -378,6 +382,8 @@ class BudgetPolicy:
     def as_dict(self) -> dict:
         """Serializes the BudgetPolicy into a dictionary suitable for use as a JSON request body."""
         body = {}
+        if self.binding_workspace_ids:
+            body["binding_workspace_ids"] = [v for v in self.binding_workspace_ids]
         if self.custom_tags:
             body["custom_tags"] = [v.as_dict() for v in self.custom_tags]
         if self.policy_id is not None:
@@ -389,6 +395,8 @@ class BudgetPolicy:
     def as_shallow_dict(self) -> dict:
         """Serializes the BudgetPolicy into a shallow dictionary of its immediate attributes."""
         body = {}
+        if self.binding_workspace_ids:
+            body["binding_workspace_ids"] = self.binding_workspace_ids
         if self.custom_tags:
             body["custom_tags"] = self.custom_tags
         if self.policy_id is not None:
@@ -401,6 +409,7 @@ class BudgetPolicy:
     def from_dict(cls, d: Dict[str, Any]) -> BudgetPolicy:
         """Deserializes the BudgetPolicy from a dictionary."""
         return cls(
+            binding_workspace_ids=d.get("binding_workspace_ids", None),
             custom_tags=_repeated_dict(d, "custom_tags", compute.CustomPolicyTag),
             policy_id=d.get("policy_id", None),
             policy_name=d.get("policy_name", None),
@@ -1864,7 +1873,7 @@ class BudgetPolicyAPI:
             query["page_token"] = json["next_page_token"]
 
     def update(
-        self, policy_id: str, *, limit_config: Optional[LimitConfig] = None, policy: Optional[BudgetPolicy] = None
+        self, policy_id: str, policy: BudgetPolicy, *, limit_config: Optional[LimitConfig] = None
     ) -> BudgetPolicy:
         """Update a budget policy.
 
@@ -1872,10 +1881,10 @@ class BudgetPolicyAPI:
 
         :param policy_id: str
           The Id of the policy. This field is generated by Databricks and globally unique.
+        :param policy: :class:`BudgetPolicy`
+          Contains the BudgetPolicy details.
         :param limit_config: :class:`LimitConfig` (optional)
           DEPRECATED. This is redundant field as LimitConfig is part of the BudgetPolicy
-        :param policy: :class:`BudgetPolicy` (optional)
-          Contains the BudgetPolicy details.
 
         :returns: :class:`BudgetPolicy`
         """