databricks-sdk 0.50.0__tar.gz → 0.51.0__tar.gz

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/CHANGELOG.md

@@ -1,5 +1,56 @@
 # Version changelog
 
+## Release v0.51.0
+
+### New Features and Improvements
+* Enabled asynchronous token refreshes by default. A new `disable_async_token_refresh` configuration option has been added to allow disabling this feature if necessary ([#952](https://github.com/databricks/databricks-sdk-py/pull/952)).
+  To disable asynchronous token refresh, set the environment variable `DATABRICKS_DISABLE_ASYNC_TOKEN_REFRESH=true` or configure it within your configuration object.
+  The previous `enable_experimental_async_token_refresh` option has been removed as asynchronous refresh is now the default behavior.
+* Introduce support for Databricks Workload Identity Federation in GitHub workflows ([933](https://github.com/databricks/databricks-sdk-py/pull/933)).
+  See README.md for instructions.
+* [Breaking] Users running their workflows in GitHub Actions, which use Cloud native authentication and also have a `DATABRICKS_CLIENT_ID` and `DATABRICKS_HOST`
+  environment variables set may see their authentication start failing due to the order in which the SDK tries different authentication methods.
+
+### API Changes
+* Added [w.alerts_v2](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/sql/alerts_v2.html) workspace-level service.
+* Added `update_ncc_azure_private_endpoint_rule_public()` method for [a.network_connectivity](https://databricks-sdk-py.readthedocs.io/en/latest/account/settings/network_connectivity.html) account-level service.
+* Added `update_endpoint_budget_policy()` and `update_endpoint_custom_tags()` methods for [w.vector_search_endpoints](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_endpoints.html) workspace-level service.
+* Added `created_at`, `created_by` and `metastore_id` fields for `databricks.sdk.service.catalog.SetArtifactAllowlist`.
+* Added `node_type_flexibility` field for `databricks.sdk.service.compute.EditInstancePool`.
+* Added `page_size` and `page_token` fields for `databricks.sdk.service.compute.GetEvents`.
+* Added `next_page_token` and `prev_page_token` fields for `databricks.sdk.service.compute.GetEventsResponse`.
+* Added `node_type_flexibility` field for `databricks.sdk.service.compute.GetInstancePool`.
+* Added `node_type_flexibility` field for `databricks.sdk.service.compute.InstancePoolAndStats`.
+* Added `effective_performance_target` field for `databricks.sdk.service.jobs.RepairHistoryItem`.
+* Added `performance_target` field for `databricks.sdk.service.jobs.RepairRun`.
+* [Breaking] Added `network_connectivity_config` field for `databricks.sdk.service.settings.CreateNetworkConnectivityConfigRequest`.
+* [Breaking] Added `private_endpoint_rule` field for `databricks.sdk.service.settings.CreatePrivateEndpointRuleRequest`.
+* Added `domain_names` field for `databricks.sdk.service.settings.NccAzurePrivateEndpointRule`.
+* Added `auto_resolve_display_name` field for `databricks.sdk.service.sql.CreateAlertRequest`.
+* Added `auto_resolve_display_name` field for `databricks.sdk.service.sql.CreateQueryRequest`.
+* Added `budget_policy_id` field for `databricks.sdk.service.vectorsearch.CreateEndpoint`.
+* Added `custom_tags` and `effective_budget_policy_id` fields for `databricks.sdk.service.vectorsearch.EndpointInfo`.
+* Added `create_clean_room`, `execute_clean_room_task` and `modify_clean_room` enum values for `databricks.sdk.service.catalog.Privilege`.
+* Added `dns_resolution_error` and `gcp_denied_by_org_policy` enum values for `databricks.sdk.service.compute.TerminationReasonCode`.
+* Added `disabled` enum value for `databricks.sdk.service.jobs.TerminationCodeCode`.
+* Added `expired` enum value for `databricks.sdk.service.settings.NccAzurePrivateEndpointRuleConnectionState`.
+* [Breaking] Changed `create_network_connectivity_configuration()` and `create_private_endpoint_rule()` methods for [a.network_connectivity](https://databricks-sdk-py.readthedocs.io/en/latest/account/settings/network_connectivity.html) account-level service with new required argument order.
+* [Breaking] Changed `create_index()` method for [w.vector_search_indexes](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_indexes.html) workspace-level service to return `databricks.sdk.service.vectorsearch.VectorIndex` dataclass.
+* [Breaking] Changed `delete_data_vector_index()` method for [w.vector_search_indexes](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_indexes.html) workspace-level service . HTTP method/verb has changed.
+* [Breaking] Changed `delete_data_vector_index()` method for [w.vector_search_indexes](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_indexes.html) workspace-level service with new required argument order.
+* [Breaking] Changed `databricks.sdk.service.vectorsearch.List` dataclass to.
+* [Breaking] Changed `workload_size` field for `databricks.sdk.service.serving.ServedModelInput` to type `str` dataclass.
+* [Breaking] Changed `group_id` field for `databricks.sdk.service.settings.NccAzurePrivateEndpointRule` to type `str` dataclass.
+* [Breaking] Changed `target_services` field for `databricks.sdk.service.settings.NccAzureServiceEndpointRule` to type `databricks.sdk.service.settings.EgressResourceTypeList` dataclass.
+* [Breaking] Changed `data_array` field for `databricks.sdk.service.vectorsearch.ResultData` to type `databricks.sdk.service.vectorsearch.ListValueList` dataclass.
+* [Breaking] Changed waiter for [VectorSearchEndpointsAPI.create_endpoint](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vectorsearch/vector_search_endpoints.html#databricks.sdk.service.vectorsearch.VectorSearchEndpointsAPI.create_endpoint) method.
+* [Breaking] Removed `name` and `region` fields for `databricks.sdk.service.settings.CreateNetworkConnectivityConfigRequest`.
+* [Breaking] Removed `group_id` and `resource_id` fields for `databricks.sdk.service.settings.CreatePrivateEndpointRuleRequest`.
+* [Breaking] Removed `null_value` field for `databricks.sdk.service.vectorsearch.Value`.
+* [Breaking] Removed `large`, `medium` and `small` enum values for `databricks.sdk.service.serving.ServedModelInputWorkloadSize`.
+* [Breaking] Removed `blob`, `dfs`, `mysql_server` and `sql_server` enum values for `databricks.sdk.service.settings.NccAzurePrivateEndpointRuleGroupId`.
+
+
 ## Release v0.50.0
 
 ### API Changes

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: databricks-sdk
-Version: 0.50.0
+Version: 0.51.0
 Summary: Databricks SDK for Python (Beta)
 Project-URL: Documentation, https://databricks-sdk-py.readthedocs.io
 Keywords: databricks,sdk
@@ -180,18 +180,18 @@ Depending on the Databricks authentication method, the SDK uses the following in
 
 ### Databricks native authentication
 
-By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks basic (username/password) authentication (`auth_type="basic"` argument).
+By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks Workload Identity Federation (WIF) authentication using OIDC (`auth_type="github-oidc"` argument).
 
 - For Databricks token authentication, you must provide `host` and `token`; or their environment variable or `.databrickscfg` file field equivalents.
-- For Databricks basic authentication, you must provide `host`, `username`, and `password` _(for AWS workspace-level operations)_; or `host`, `account_id`, `username`, and `password` _(for AWS, Azure, or GCP account-level operations)_; or their environment variable or `.databrickscfg` file field equivalents.
-
-| Argument     | Description | Environment variable |
-|--------------|-------------|-------------------|
-| `host`       | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` |     
-| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_. | `DATABRICKS_ACCOUNT_ID` |
-| `token`      | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` |
-| `username`   | _(String)_ The Databricks username part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_USERNAME` |
-| `password`   | _(String)_ The Databricks password part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_PASSWORD` |
+- For Databricks OIDC authentication, you must provide the `host`, `client_id` and `token_audience` _(optional)_ either directly, through the corresponding environment variables, or in your `.databrickscfg` configuration file.
+
+| Argument         | Description                                                                                                                                                                                                                                                               | Environment variable    |
+|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
+| `host`           | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint.                                                                                                                                                      | `DATABRICKS_HOST`       |     
+| `account_id`     | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_.  | `DATABRICKS_ACCOUNT_ID` |
+| `token`          | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_.                                                                                                                                       | `DATABRICKS_TOKEN`      |
+| `client_id`      | _(String)_ The Databricks Service Principal Application ID.                                                                                                                                                                                                               | `DATABRICKS_CLIENT_ID`  |
+| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier.                                                                                                                                         | `TOKEN_AUDIENCE`        |
 
 For example, to use Databricks token authentication:
 

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/README.md

@@ -126,18 +126,18 @@ Depending on the Databricks authentication method, the SDK uses the following in
 
 ### Databricks native authentication
 
-By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks basic (username/password) authentication (`auth_type="basic"` argument).
+By default, the Databricks SDK for Python initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Databricks Workload Identity Federation (WIF) authentication using OIDC (`auth_type="github-oidc"` argument).
 
 - For Databricks token authentication, you must provide `host` and `token`; or their environment variable or `.databrickscfg` file field equivalents.
-- For Databricks basic authentication, you must provide `host`, `username`, and `password` _(for AWS workspace-level operations)_; or `host`, `account_id`, `username`, and `password` _(for AWS, Azure, or GCP account-level operations)_; or their environment variable or `.databrickscfg` file field equivalents.
-
-| Argument     | Description | Environment variable |
-|--------------|-------------|-------------------|
-| `host`       | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` |     
-| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_. | `DATABRICKS_ACCOUNT_ID` |
-| `token`      | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` |
-| `username`   | _(String)_ The Databricks username part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_USERNAME` |
-| `password`   | _(String)_ The Databricks password part of basic authentication. Only possible when `Host` is `*.cloud.databricks.com` _(AWS)_. | `DATABRICKS_PASSWORD` |
+- For Databricks OIDC authentication, you must provide the `host`, `client_id` and `token_audience` _(optional)_ either directly, through the corresponding environment variables, or in your `.databrickscfg` configuration file.
+
+| Argument         | Description                                                                                                                                                                                                                                                               | Environment variable    |
+|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
+| `host`           | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint.                                                                                                                                                      | `DATABRICKS_HOST`       |     
+| `account_id`     | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_.  | `DATABRICKS_ACCOUNT_ID` |
+| `token`          | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_.                                                                                                                                       | `DATABRICKS_TOKEN`      |
+| `client_id`      | _(String)_ The Databricks Service Principal Application ID.                                                                                                                                                                                                               | `DATABRICKS_CLIENT_ID`  |
+| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier.                                                                                                                                         | `TOKEN_AUDIENCE`        |
 
 For example, to use Databricks token authentication:
 

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/databricks/sdk/__init__.py

@@ -97,10 +97,10 @@ from databricks.sdk.service.sharing import (ProvidersAPI,
                                             RecipientActivationAPI,
                                             RecipientsAPI, SharesAPI)
 from databricks.sdk.service.sql import (AlertsAPI, AlertsLegacyAPI,
-                                        DashboardsAPI, DashboardWidgetsAPI,
-                                        DataSourcesAPI, DbsqlPermissionsAPI,
-                                        QueriesAPI, QueriesLegacyAPI,
-                                        QueryHistoryAPI,
+                                        AlertsV2API, DashboardsAPI,
+                                        DashboardWidgetsAPI, DataSourcesAPI,
+                                        DbsqlPermissionsAPI, QueriesAPI,
+                                        QueriesLegacyAPI, QueryHistoryAPI,
                                         QueryVisualizationsAPI,
                                         QueryVisualizationsLegacyAPI,
                                         RedashConfigAPI, StatementExecutionAPI,
@@ -170,6 +170,7 @@ class WorkspaceClient:
         product_version="0.0.0",
         credentials_strategy: Optional[CredentialsStrategy] = None,
         credentials_provider: Optional[CredentialsStrategy] = None,
+        token_audience: Optional[str] = None,
         config: Optional[client.Config] = None,
     ):
         if not config:
@@ -198,6 +199,7 @@ class WorkspaceClient:
                 debug_headers=debug_headers,
                 product=product,
                 product_version=product_version,
+                token_audience=token_audience,
             )
         self._config = config.copy()
         self._dbutils = _make_dbutils(self._config)
@@ -207,6 +209,7 @@ class WorkspaceClient:
         self._account_access_control_proxy = service.iam.AccountAccessControlProxyAPI(self._api_client)
         self._alerts = service.sql.AlertsAPI(self._api_client)
         self._alerts_legacy = service.sql.AlertsLegacyAPI(self._api_client)
+        self._alerts_v2 = service.sql.AlertsV2API(self._api_client)
         self._apps = service.apps.AppsAPI(self._api_client)
         self._artifact_allowlists = service.catalog.ArtifactAllowlistsAPI(self._api_client)
         self._catalogs = service.catalog.CatalogsAPI(self._api_client)
@@ -289,7 +292,7 @@ class WorkspaceClient:
         self._service_principals = service.iam.ServicePrincipalsAPI(self._api_client)
         self._serving_endpoints = serving_endpoints
         serving_endpoints_data_plane_token_source = DataPlaneTokenSource(
-            self._config.host, self._config.oauth_token, not self._config.enable_experimental_async_token_refresh
+            self._config.host, self._config.oauth_token, self._config.disable_async_token_refresh
         )
         self._serving_endpoints_data_plane = service.serving.ServingEndpointsDataPlaneAPI(
             self._api_client, serving_endpoints, serving_endpoints_data_plane_token_source
@@ -346,6 +349,11 @@ class WorkspaceClient:
         """The alerts API can be used to perform CRUD operations on alerts."""
         return self._alerts_legacy
 
+    @property
+    def alerts_v2(self) -> service.sql.AlertsV2API:
+        """TODO: Add description."""
+        return self._alerts_v2
+
     @property
     def apps(self) -> service.apps.AppsAPI:
         """Apps run directly on a customer’s Databricks instance, integrate with their data, use and extend Databricks services, and enable users to interact through single sign-on."""
@@ -862,6 +870,7 @@ class AccountClient:
         product_version="0.0.0",
         credentials_strategy: Optional[CredentialsStrategy] = None,
         credentials_provider: Optional[CredentialsStrategy] = None,
+        token_audience: Optional[str] = None,
         config: Optional[client.Config] = None,
     ):
         if not config:
@@ -890,6 +899,7 @@ class AccountClient:
                 debug_headers=debug_headers,
                 product=product,
                 product_version=product_version,
+                token_audience=token_audience,
             )
         self._config = config.copy()
         self._api_client = client.ApiClient(self._config)

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/databricks/sdk/config.py

@@ -61,6 +61,7 @@ class Config:
     host: str = ConfigAttribute(env="DATABRICKS_HOST")
     account_id: str = ConfigAttribute(env="DATABRICKS_ACCOUNT_ID")
     token: str = ConfigAttribute(env="DATABRICKS_TOKEN", auth="pat", sensitive=True)
+    token_audience: str = ConfigAttribute(env="DATABRICKS_TOKEN_AUDIENCE", auth="github-oidc")
     username: str = ConfigAttribute(env="DATABRICKS_USERNAME", auth="basic")
     password: str = ConfigAttribute(env="DATABRICKS_PASSWORD", auth="basic", sensitive=True)
     client_id: str = ConfigAttribute(env="DATABRICKS_CLIENT_ID", auth="oauth")
@@ -95,9 +96,7 @@ class Config:
     max_connections_per_pool: int = ConfigAttribute()
     databricks_environment: Optional[DatabricksEnvironment] = None
 
-    enable_experimental_async_token_refresh: bool = ConfigAttribute(
-        env="DATABRICKS_ENABLE_EXPERIMENTAL_ASYNC_TOKEN_REFRESH"
-    )
+    disable_async_token_refresh: bool = ConfigAttribute(env="DATABRICKS_DISABLE_ASYNC_TOKEN_REFRESH")
 
     enable_experimental_files_api_client: bool = ConfigAttribute(env="DATABRICKS_ENABLE_EXPERIMENTAL_FILES_API_CLIENT")
     files_api_client_download_max_total_recovers = None

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/databricks/sdk/credentials_provider.py

@@ -23,6 +23,7 @@ from google.oauth2 import service_account  # type: ignore
 from .azure import add_sp_management_token, add_workspace_id_header
 from .oauth import (ClientCredentials, OAuthClient, Refreshable, Token,
                     TokenCache, TokenSource)
+from .oidc_token_supplier import GitHubOIDCTokenSupplier
 
 CredentialsProvider = Callable[[], Dict[str, str]]
 
@@ -191,7 +192,7 @@ def oauth_service_principal(cfg: "Config") -> Optional[CredentialsProvider]:
         token_url=oidc.token_endpoint,
         scopes=["all-apis"],
         use_header=True,
-        disable_async=not cfg.enable_experimental_async_token_refresh,
+        disable_async=cfg.disable_async_token_refresh,
     )
 
     def inner() -> Dict[str, str]:
@@ -291,7 +292,7 @@ def azure_service_principal(cfg: "Config") -> CredentialsProvider:
             token_url=f"{aad_endpoint}{cfg.azure_tenant_id}/oauth2/token",
             endpoint_params={"resource": resource},
             use_params=True,
-            disable_async=not cfg.enable_experimental_async_token_refresh,
+            disable_async=cfg.disable_async_token_refresh,
         )
 
     _ensure_host_present(cfg, token_source_for)
@@ -314,6 +315,58 @@ def azure_service_principal(cfg: "Config") -> CredentialsProvider:
     return OAuthCredentialsProvider(refreshed_headers, token)
 
 
+@oauth_credentials_strategy("github-oidc", ["host", "client_id"])
+def databricks_wif(cfg: "Config") -> Optional[CredentialsProvider]:
+    """
+    DatabricksWIFCredentials uses a Token Supplier to get a JWT Token and exchanges
+    it for a Databricks Token.
+
+    Supported suppliers:
+    - GitHub OIDC
+    """
+    supplier = GitHubOIDCTokenSupplier()
+
+    audience = cfg.token_audience
+    if audience is None and cfg.is_account_client:
+        audience = cfg.account_id
+    if audience is None and not cfg.is_account_client:
+        audience = cfg.oidc_endpoints.token_endpoint
+
+    # Try to get an idToken. If no supplier returns a token, we cannot use this authentication mode.
+    id_token = supplier.get_oidc_token(audience)
+    if not id_token:
+        return None
+
+    def token_source_for(audience: str) -> TokenSource:
+        id_token = supplier.get_oidc_token(audience)
+        if not id_token:
+            # Should not happen, since we checked it above.
+            raise Exception("Cannot get OIDC token")
+        params = {
+            "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
+            "subject_token": id_token,
+            "grant_type": "urn:ietf:params:oauth:grant-type:token-exchange",
+        }
+        return ClientCredentials(
+            client_id=cfg.client_id,
+            client_secret="",  # we have no (rotatable) secrets in OIDC flow
+            token_url=cfg.oidc_endpoints.token_endpoint,
+            endpoint_params=params,
+            scopes=["all-apis"],
+            use_params=True,
+            disable_async=cfg.disable_async_token_refresh,
+        )
+
+    def refreshed_headers() -> Dict[str, str]:
+        token = token_source_for(audience).token()
+        return {"Authorization": f"{token.token_type} {token.access_token}"}
+
+    def token() -> Token:
+        return token_source_for(audience).token()
+
+    return OAuthCredentialsProvider(refreshed_headers, token)
+
+
 @oauth_credentials_strategy("github-oidc-azure", ["host", "azure_client_id"])
 def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
     if "ACTIONS_ID_TOKEN_REQUEST_TOKEN" not in os.environ:
@@ -325,16 +378,8 @@ def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
     if not cfg.is_azure:
         return None
 
-    # See https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-cloud-providers
-    headers = {"Authorization": f"Bearer {os.environ['ACTIONS_ID_TOKEN_REQUEST_TOKEN']}"}
-    endpoint = f"{os.environ['ACTIONS_ID_TOKEN_REQUEST_URL']}&audience=api://AzureADTokenExchange"
-    response = requests.get(endpoint, headers=headers)
-    if not response.ok:
-        return None
-
-    # get the ID Token with aud=api://AzureADTokenExchange sub=repo:org/repo:environment:name
-    response_json = response.json()
-    if "value" not in response_json:
+    token = GitHubOIDCTokenSupplier().get_oidc_token("api://AzureADTokenExchange")
+    if not token:
         return None
 
     logger.info(
@@ -344,7 +389,7 @@ def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
     params = {
         "client_assertion_type": "urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
         "resource": cfg.effective_azure_login_app_id,
-        "client_assertion": response_json["value"],
+        "client_assertion": token,
     }
     aad_endpoint = cfg.arm_environment.active_directory_endpoint
     if not cfg.azure_tenant_id:
@@ -357,7 +402,7 @@ def github_oidc_azure(cfg: "Config") -> Optional[CredentialsProvider]:
         token_url=f"{aad_endpoint}{cfg.azure_tenant_id}/oauth2/token",
         endpoint_params=params,
         use_params=True,
-        disable_async=not cfg.enable_experimental_async_token_refresh,
+        disable_async=cfg.disable_async_token_refresh,
     )
 
     def refreshed_headers() -> Dict[str, str]:
@@ -694,7 +739,7 @@ class DatabricksCliTokenSource(CliTokenSource):
             token_type_field="token_type",
             access_token_field="access_token",
             expiry_field="expiry",
-            disable_async=not cfg.enable_experimental_async_token_refresh,
+            disable_async=cfg.disable_async_token_refresh,
         )
 
     @staticmethod
@@ -927,6 +972,7 @@ class DefaultCredentials:
             basic_auth,
             metadata_service,
             oauth_service_principal,
+            databricks_wif,
             azure_service_principal,
             github_oidc_azure,
             azure_cli,

databricks_sdk-0.51.0/databricks/sdk/oidc_token_supplier.py

@@ -0,0 +1,28 @@
+import os
+from typing import Optional
+
+import requests
+
+
+class GitHubOIDCTokenSupplier:
+    """
+    Supplies OIDC tokens from GitHub Actions.
+    """
+
+    def get_oidc_token(self, audience: str) -> Optional[str]:
+        if "ACTIONS_ID_TOKEN_REQUEST_TOKEN" not in os.environ or "ACTIONS_ID_TOKEN_REQUEST_URL" not in os.environ:
+            # not in GitHub actions
+            return None
+        # See https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-cloud-providers
+        headers = {"Authorization": f"Bearer {os.environ['ACTIONS_ID_TOKEN_REQUEST_TOKEN']}"}
+        endpoint = f"{os.environ['ACTIONS_ID_TOKEN_REQUEST_URL']}&audience={audience}"
+        response = requests.get(endpoint, headers=headers)
+        if not response.ok:
+            return None
+
+        # get the ID Token with aud=api://AzureADTokenExchange sub=repo:org/repo:environment:name
+        response_json = response.json()
+        if "value" not in response_json:
+            return None
+
+        return response_json["value"]

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/databricks/sdk/service/apps.py

@@ -1173,12 +1173,12 @@ class AppsAPI:
             attempt += 1
         raise TimeoutError(f"timed out after {timeout}: {status_message}")
 
-    def create(self, *, app: Optional[App] = None, no_compute: Optional[bool] = None) -> Wait[App]:
+    def create(self, app: App, *, no_compute: Optional[bool] = None) -> Wait[App]:
         """Create an app.
 
         Creates a new app.
 
-        :param app: :class:`App` (optional)
+        :param app: :class:`App`
         :param no_compute: bool (optional)
           If true, the app will not be started after creation.
 
@@ -1198,9 +1198,7 @@ class AppsAPI:
         op_response = self._api.do("POST", "/api/2.0/apps", query=query, body=body, headers=headers)
         return Wait(self.wait_get_app_active, response=App.from_dict(op_response), name=op_response["name"])
 
-    def create_and_wait(
-        self, *, app: Optional[App] = None, no_compute: Optional[bool] = None, timeout=timedelta(minutes=20)
-    ) -> App:
+    def create_and_wait(self, app: App, *, no_compute: Optional[bool] = None, timeout=timedelta(minutes=20)) -> App:
         return self.create(app=app, no_compute=no_compute).result(timeout=timeout)
 
     def delete(self, name: str) -> App:
@@ -1221,14 +1219,14 @@ class AppsAPI:
         res = self._api.do("DELETE", f"/api/2.0/apps/{name}", headers=headers)
         return App.from_dict(res)
 
-    def deploy(self, app_name: str, *, app_deployment: Optional[AppDeployment] = None) -> Wait[AppDeployment]:
+    def deploy(self, app_name: str, app_deployment: AppDeployment) -> Wait[AppDeployment]:
         """Create an app deployment.
 
         Creates an app deployment for the app with the supplied name.
 
         :param app_name: str
           The name of the app.
-        :param app_deployment: :class:`AppDeployment` (optional)
+        :param app_deployment: :class:`AppDeployment`
 
         :returns:
           Long-running operation waiter for :class:`AppDeployment`.
@@ -1249,7 +1247,7 @@ class AppsAPI:
         )
 
     def deploy_and_wait(
-        self, app_name: str, *, app_deployment: Optional[AppDeployment] = None, timeout=timedelta(minutes=20)
+        self, app_name: str, app_deployment: AppDeployment, timeout=timedelta(minutes=20)
     ) -> AppDeployment:
         return self.deploy(app_deployment=app_deployment, app_name=app_name).result(timeout=timeout)
 
@@ -1466,7 +1464,7 @@ class AppsAPI:
     def stop_and_wait(self, name: str, timeout=timedelta(minutes=20)) -> App:
         return self.stop(name=name).result(timeout=timeout)
 
-    def update(self, name: str, *, app: Optional[App] = None) -> App:
+    def update(self, name: str, app: App) -> App:
         """Update an app.
 
         Updates the app with the supplied name.
@@ -1474,7 +1472,7 @@ class AppsAPI:
         :param name: str
           The name of the app. The name must contain only lowercase alphanumeric characters and hyphens. It
           must be unique within the workspace.
-        :param app: :class:`App` (optional)
+        :param app: :class:`App`
 
         :returns: :class:`App`
         """

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/databricks/sdk/service/billing.py

@@ -1873,7 +1873,7 @@ class BudgetPolicyAPI:
             query["page_token"] = json["next_page_token"]
 
     def update(
-        self, policy_id: str, *, limit_config: Optional[LimitConfig] = None, policy: Optional[BudgetPolicy] = None
+        self, policy_id: str, policy: BudgetPolicy, *, limit_config: Optional[LimitConfig] = None
     ) -> BudgetPolicy:
         """Update a budget policy.
 
@@ -1881,10 +1881,10 @@ class BudgetPolicyAPI:
 
         :param policy_id: str
           The Id of the policy. This field is generated by Databricks and globally unique.
+        :param policy: :class:`BudgetPolicy`
+          Contains the BudgetPolicy details.
         :param limit_config: :class:`LimitConfig` (optional)
           DEPRECATED. This is redundant field as LimitConfig is part of the BudgetPolicy
-        :param policy: :class:`BudgetPolicy` (optional)
-          Contains the BudgetPolicy details.
 
         :returns: :class:`BudgetPolicy`
         """

{databricks_sdk-0.50.0 → databricks_sdk-0.51.0}/databricks/sdk/service/catalog.py

@@ -6675,6 +6675,7 @@ class Privilege(Enum):
     BROWSE = "BROWSE"
     CREATE = "CREATE"
     CREATE_CATALOG = "CREATE_CATALOG"
+    CREATE_CLEAN_ROOM = "CREATE_CLEAN_ROOM"
     CREATE_CONNECTION = "CREATE_CONNECTION"
     CREATE_EXTERNAL_LOCATION = "CREATE_EXTERNAL_LOCATION"
     CREATE_EXTERNAL_TABLE = "CREATE_EXTERNAL_TABLE"
@@ -6695,9 +6696,11 @@ class Privilege(Enum):
     CREATE_VIEW = "CREATE_VIEW"
     CREATE_VOLUME = "CREATE_VOLUME"
     EXECUTE = "EXECUTE"
+    EXECUTE_CLEAN_ROOM_TASK = "EXECUTE_CLEAN_ROOM_TASK"
     MANAGE = "MANAGE"
     MANAGE_ALLOWLIST = "MANAGE_ALLOWLIST"
     MODIFY = "MODIFY"
+    MODIFY_CLEAN_ROOM = "MODIFY_CLEAN_ROOM"
     READ_FILES = "READ_FILES"
     READ_PRIVATE_FILES = "READ_PRIVATE_FILES"
     READ_VOLUME = "READ_VOLUME"
@@ -7360,6 +7363,15 @@ class SetArtifactAllowlist:
     artifact_type: Optional[ArtifactType] = None
     """The artifact type of the allowlist."""
 
+    created_at: Optional[int] = None
+    """Time at which this artifact allowlist was set, in epoch milliseconds."""
+
+    created_by: Optional[str] = None
+    """Username of the user who set the artifact allowlist."""
+
+    metastore_id: Optional[str] = None
+    """Unique identifier of parent metastore."""
+
     def as_dict(self) -> dict:
         """Serializes the SetArtifactAllowlist into a dictionary suitable for use as a JSON request body."""
         body = {}
@@ -7367,6 +7379,12 @@ class SetArtifactAllowlist:
             body["artifact_matchers"] = [v.as_dict() for v in self.artifact_matchers]
         if self.artifact_type is not None:
             body["artifact_type"] = self.artifact_type.value
+        if self.created_at is not None:
+            body["created_at"] = self.created_at
+        if self.created_by is not None:
+            body["created_by"] = self.created_by
+        if self.metastore_id is not None:
+            body["metastore_id"] = self.metastore_id
         return body
 
     def as_shallow_dict(self) -> dict:
@@ -7376,6 +7394,12 @@ class SetArtifactAllowlist:
             body["artifact_matchers"] = self.artifact_matchers
         if self.artifact_type is not None:
             body["artifact_type"] = self.artifact_type
+        if self.created_at is not None:
+            body["created_at"] = self.created_at
+        if self.created_by is not None:
+            body["created_by"] = self.created_by
+        if self.metastore_id is not None:
+            body["metastore_id"] = self.metastore_id
         return body
 
     @classmethod
@@ -7384,6 +7408,9 @@ class SetArtifactAllowlist:
         return cls(
             artifact_matchers=_repeated_dict(d, "artifact_matchers", ArtifactMatcher),
             artifact_type=_enum(d, "artifact_type", ArtifactType),
+            created_at=d.get("created_at", None),
+            created_by=d.get("created_by", None),
+            metastore_id=d.get("metastore_id", None),
         )
 
 
@@ -10467,7 +10494,15 @@ class ArtifactAllowlistsAPI:
         res = self._api.do("GET", f"/api/2.1/unity-catalog/artifact-allowlists/{artifact_type.value}", headers=headers)
         return ArtifactAllowlistInfo.from_dict(res)
 
-    def update(self, artifact_type: ArtifactType, artifact_matchers: List[ArtifactMatcher]) -> ArtifactAllowlistInfo:
+    def update(
+        self,
+        artifact_type: ArtifactType,
+        artifact_matchers: List[ArtifactMatcher],
+        *,
+        created_at: Optional[int] = None,
+        created_by: Optional[str] = None,
+        metastore_id: Optional[str] = None,
+    ) -> ArtifactAllowlistInfo:
         """Set an artifact allowlist.
 
         Set the artifact allowlist of a certain artifact type. The whole artifact allowlist is replaced with
@@ -10478,12 +10513,24 @@ class ArtifactAllowlistsAPI:
           The artifact type of the allowlist.
         :param artifact_matchers: List[:class:`ArtifactMatcher`]
           A list of allowed artifact match patterns.
+        :param created_at: int (optional)
+          Time at which this artifact allowlist was set, in epoch milliseconds.
+        :param created_by: str (optional)
+          Username of the user who set the artifact allowlist.
+        :param metastore_id: str (optional)
+          Unique identifier of parent metastore.
 
         :returns: :class:`ArtifactAllowlistInfo`
         """
         body = {}
         if artifact_matchers is not None:
             body["artifact_matchers"] = [v.as_dict() for v in artifact_matchers]
+        if created_at is not None:
+            body["created_at"] = created_at
+        if created_by is not None:
+            body["created_by"] = created_by
+        if metastore_id is not None:
+            body["metastore_id"] = metastore_id
         headers = {
             "Accept": "application/json",
             "Content-Type": "application/json",
@@ -12354,12 +12401,12 @@ class OnlineTablesAPI:
             attempt += 1
         raise TimeoutError(f"timed out after {timeout}: {status_message}")
 
-    def create(self, *, table: Optional[OnlineTable] = None) -> Wait[OnlineTable]:
+    def create(self, table: OnlineTable) -> Wait[OnlineTable]:
         """Create an Online Table.
 
         Create a new Online Table.
 
-        :param table: :class:`OnlineTable` (optional)
+        :param table: :class:`OnlineTable`
           Online Table information.
 
         :returns:
@@ -12377,7 +12424,7 @@ class OnlineTablesAPI:
             self.wait_get_online_table_active, response=OnlineTable.from_dict(op_response), name=op_response["name"]
         )
 
-    def create_and_wait(self, *, table: Optional[OnlineTable] = None, timeout=timedelta(minutes=20)) -> OnlineTable:
+    def create_and_wait(self, table: OnlineTable, timeout=timedelta(minutes=20)) -> OnlineTable:
         return self.create(table=table).result(timeout=timeout)
 
     def delete(self, name: str):