azure-ai-evaluation 1.0.1__tar.gz → 1.1.0__tar.gz

{azure_ai_evaluation-1.0.1 → azure_ai_evaluation-1.1.0}/CHANGELOG.md

@@ -1,10 +1,16 @@
 # Release History
 
+## 1.1.0 (2024-12-12)
+
+### Bugs Fixed
+- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Studio.
+- Fixed `AttributeError: 'NoneType' object has no attribute 'get'` while running simulator with 1000+ results
+
 ## 1.0.1 (2024-11-15)
 
 ### Bugs Fixed
-- Fixed `[remote]` extra to be needed only when tracking results in Azure AI Studio.
 - Removing `azure-ai-inference` as dependency.
+- Fixed `AttributeError: 'NoneType' object has no attribute 'get'` while running simulator with 1000+ results
 
 ## 1.0.0 (2024-11-13)
 
@@ -16,6 +22,7 @@
 - Fixed an issue where the `output_path` parameter in the `evaluate` API did not support relative path.
 - Output of adversarial simulators are of type `JsonLineList` and the helper function `to_eval_qr_json_lines` now outputs context from both user and assistant turns along with `category` if it exists in the conversation
 - Fixed an issue where during long-running simulations, API token expires causing "Forbidden" error. Instead, users can now set an environment variable `AZURE_TOKEN_REFRESH_INTERVAL` to refresh the token more frequently to prevent expiration and ensure continuous operation of the simulation.
+- Fixed an issue with the `ContentSafetyEvaluator` that caused parallel execution of sub-evaluators to fail. Parallel execution is now enabled by default again, but can still be disabled via the '_parallel' boolean keyword argument during class initialization.
 - Fix `evaluate` function not producing aggregated metrics if ANY values to be aggregated were None, NaN, or
 otherwise difficult to process. Such values are ignored fully, so the aggregated metric of `[1, 2, 3, NaN]`
 would be 2, not 1.5.

{azure_ai_evaluation-1.0.1/azure_ai_evaluation.egg-info → azure_ai_evaluation-1.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: azure-ai-evaluation
-Version: 1.0.1
+Version: 1.1.0
 Summary: Microsoft Azure Evaluation Library for Python
 Home-page: https://github.com/Azure/azure-sdk-for-python
 Author: Microsoft Corporation
@@ -28,8 +28,7 @@ Requires-Dist: pyjwt>=2.8.0
 Requires-Dist: azure-identity>=1.16.0
 Requires-Dist: azure-core>=1.30.2
 Requires-Dist: nltk>=3.9.1
-Provides-Extra: remote
-Requires-Dist: promptflow-azure<2.0.0,>=1.15.0; extra == "remote"
+Requires-Dist: azure-storage-blob>=12.10.0
 
 # Azure AI Evaluation client library for Python
 
@@ -359,13 +358,13 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
 [evaluate_dataset]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk#evaluate-on-test-dataset-using-evaluate
 [evaluators]: https://learn.microsoft.com/python/api/azure-ai-evaluation/azure.ai.evaluation?view=azure-python-preview
 [evaluate_api]: https://learn.microsoft.com/python/api/azure-ai-evaluation/azure.ai.evaluation?view=azure-python-preview#azure-ai-evaluation-evaluate
-[evaluate_app]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/evaluate_app
+[evaluate_app]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Supported_Evaluation_Targets/Evaluate_App_Endpoint
 [evaluation_tsg]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/evaluation/azure-ai-evaluation/TROUBLESHOOTING.md
 [ai_studio]: https://learn.microsoft.com/azure/ai-studio/what-is-ai-studio
 [ai_project]: https://learn.microsoft.com/azure/ai-studio/how-to/create-projects?tabs=ai-studio
 [azure_openai]: https://learn.microsoft.com/azure/ai-services/openai/
-[evaluate_models]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/evaluate_endpoints
-[custom_evaluators]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/evaluate_custom
+[evaluate_models]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Supported_Evaluation_Targets/Evaluate_Base_Model_Endpoint
+[custom_evaluators]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Supported_Evaluation_Metrics/Custom_Evaluators
 [evaluate_samples]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate
 [evaluation_metrics]: https://learn.microsoft.com/azure/ai-studio/concepts/evaluation-metrics-built-in
 [performance_and_quality_evaluators]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk#performance-and-quality-evaluators
@@ -373,18 +372,23 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
 [composite_evaluators]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk#composite-evaluators
 [adversarial_simulation_docs]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data#generate-adversarial-simulations-for-safety-evaluation
 [adversarial_simulation_scenarios]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data#supported-adversarial-simulation-scenarios
-[adversarial_simulation]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/simulate_adversarial
-[simulate_with_conversation_starter]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/simulate_conversation_starter
+[adversarial_simulation]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Simulators/Simulate_Adversarial_Data
+[simulate_with_conversation_starter]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Simulators/Simulate_Context-Relevant_Data/Simulate_From_Conversation_Starter
 [adversarial_jailbreak]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data#simulating-jailbreak-attacks
 
-
 # Release History
 
+## 1.1.0 (2024-12-12)
+
+### Bugs Fixed
+- Removed `[remote]` extra. This is no longer needed when tracking results in Azure AI Studio.
+- Fixed `AttributeError: 'NoneType' object has no attribute 'get'` while running simulator with 1000+ results
+
 ## 1.0.1 (2024-11-15)
 
 ### Bugs Fixed
-- Fixed `[remote]` extra to be needed only when tracking results in Azure AI Studio.
 - Removing `azure-ai-inference` as dependency.
+- Fixed `AttributeError: 'NoneType' object has no attribute 'get'` while running simulator with 1000+ results
 
 ## 1.0.0 (2024-11-13)
 
@@ -396,6 +400,7 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
 - Fixed an issue where the `output_path` parameter in the `evaluate` API did not support relative path.
 - Output of adversarial simulators are of type `JsonLineList` and the helper function `to_eval_qr_json_lines` now outputs context from both user and assistant turns along with `category` if it exists in the conversation
 - Fixed an issue where during long-running simulations, API token expires causing "Forbidden" error. Instead, users can now set an environment variable `AZURE_TOKEN_REFRESH_INTERVAL` to refresh the token more frequently to prevent expiration and ensure continuous operation of the simulation.
+- Fixed an issue with the `ContentSafetyEvaluator` that caused parallel execution of sub-evaluators to fail. Parallel execution is now enabled by default again, but can still be disabled via the '_parallel' boolean keyword argument during class initialization.
 - Fix `evaluate` function not producing aggregated metrics if ANY values to be aggregated were None, NaN, or
 otherwise difficult to process. Such values are ignored fully, so the aggregated metric of `[1, 2, 3, NaN]`
 would be 2, not 1.5.

{azure_ai_evaluation-1.0.1 → azure_ai_evaluation-1.1.0}/README.md

@@ -326,13 +326,13 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
 [evaluate_dataset]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk#evaluate-on-test-dataset-using-evaluate
 [evaluators]: https://learn.microsoft.com/python/api/azure-ai-evaluation/azure.ai.evaluation?view=azure-python-preview
 [evaluate_api]: https://learn.microsoft.com/python/api/azure-ai-evaluation/azure.ai.evaluation?view=azure-python-preview#azure-ai-evaluation-evaluate
-[evaluate_app]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/evaluate_app
+[evaluate_app]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Supported_Evaluation_Targets/Evaluate_App_Endpoint
 [evaluation_tsg]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/evaluation/azure-ai-evaluation/TROUBLESHOOTING.md
 [ai_studio]: https://learn.microsoft.com/azure/ai-studio/what-is-ai-studio
 [ai_project]: https://learn.microsoft.com/azure/ai-studio/how-to/create-projects?tabs=ai-studio
 [azure_openai]: https://learn.microsoft.com/azure/ai-services/openai/
-[evaluate_models]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/evaluate_endpoints
-[custom_evaluators]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/evaluate_custom
+[evaluate_models]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Supported_Evaluation_Targets/Evaluate_Base_Model_Endpoint
+[custom_evaluators]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Supported_Evaluation_Metrics/Custom_Evaluators
 [evaluate_samples]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate
 [evaluation_metrics]: https://learn.microsoft.com/azure/ai-studio/concepts/evaluation-metrics-built-in
 [performance_and_quality_evaluators]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk#performance-and-quality-evaluators
@@ -340,6 +340,6 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
 [composite_evaluators]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk#composite-evaluators
 [adversarial_simulation_docs]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data#generate-adversarial-simulations-for-safety-evaluation
 [adversarial_simulation_scenarios]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data#supported-adversarial-simulation-scenarios
-[adversarial_simulation]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/simulate_adversarial
-[simulate_with_conversation_starter]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/simulate_conversation_starter
-[adversarial_jailbreak]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data#simulating-jailbreak-attacks
+[adversarial_simulation]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Simulators/Simulate_Adversarial_Data
+[simulate_with_conversation_starter]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios/evaluate/Simulators/Simulate_Context-Relevant_Data/Simulate_From_Conversation_Starter
+[adversarial_jailbreak]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/simulator-interaction-data#simulating-jailbreak-attacks

{azure_ai_evaluation-1.0.1 → azure_ai_evaluation-1.1.0}/TROUBLESHOOTING.md

@@ -26,6 +26,10 @@ This guide walks you through how to investigate failures, common errors in the `
 - Ensure that you assign the proper permissions to the storage account linked to your Azure AI Studio hub. This can be done with the following command. More information can be found [here](https://aka.ms/credentialleshub).
 
     ```Shell
+    # <mySubscriptionID>: Subscription ID of the Azure AI Studio hub's linked storage account (available in Azure AI hub resource view in Azure Portal).
+    # <myResourceGroupName>: Resource group of the Azure AI Studio hub's linked storage account.
+    # <user-id>: User object ID for role assignment (retrieve with "az ad user show" command).
+
     az role assignment create --role "Storage Blob Data Contributor" --scope /subscriptions/<mySubscriptionID>/resourceGroups/<myResourceGroupName> --assignee-principal-type User --assignee-object-id "<user-id>"
     ```
 
@@ -50,7 +54,7 @@ The Adversarial simulator does not support selecting individual harms, instead w
 ### Simulator is slow
 
 Identify the type of simulations being run (adversarial or non-adversarial).
-Adjust parameters such as `api_call_retry_sleep_sec`, `api_call_delay_sec`, and `concurrent_async_task`. Please note that rate limits to llm calls can be both tokens per minute and requests per minute. 
+Adjust parameters such as `api_call_retry_sleep_sec`, `api_call_delay_sec`, and `concurrent_async_task`. Please note that rate limits to llm calls can be both tokens per minute and requests per minute.
 
 ## Logging
 

azure_ai_evaluation-1.1.0/azure/ai/evaluation/_azure/_clients.py

@@ -0,0 +1,188 @@
+# ---------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# ---------------------------------------------------------
+
+from logging import Logger
+from typing import Any, Dict, Final, Optional, Set, Union, cast
+from threading import Lock
+from urllib.parse import quote
+from json.decoder import JSONDecodeError
+
+from azure.core.credentials import TokenCredential, AzureSasCredential
+from azure.core.rest import HttpResponse
+from azure.ai.evaluation._exceptions import ErrorBlame, ErrorCategory, ErrorTarget, EvaluationException
+from azure.ai.evaluation._http_utils import HttpPipeline, get_http_client
+from azure.ai.evaluation._azure._token_manager import AzureMLTokenManager
+from azure.ai.evaluation.simulator._model_tools._identity_manager import TokenScope
+from ._models import BlobStoreInfo, Workspace
+
+
+API_VERSION: Final[str] = "2024-10-01"
+QUERY_KEY_API_VERSION: Final[str] = "api-version"
+PATH_ML_WORKSPACES = ("providers", "Microsoft.MachineLearningServices", "workspaces")
+
+
+class LiteMLClient:
+    """A lightweight Azure ML API client.
+
+    :param subscription_id: Azure subscription ID
+    :type subscription_id: str
+    :param resource_group: Azure resource group name
+    :type resource_group: str
+    :param logger: Logger object
+    :type logger: logging.Logger
+    :keyword credential: Azure credentials
+    :paramtype credential: TokenCredential
+    :keyword kwargs: Additional keyword arguments
+    :paramtype kwargs: Dict
+    :keyword str api_version: The API version. Default is 2024-10-01
+    """
+
+    def __init__(
+        self,
+        subscription_id: str,
+        resource_group: str,
+        logger: Logger,
+        credential: Optional[TokenCredential] = None,
+        **kwargs: Any,
+    ) -> None:
+        subscription_id = quote(subscription_id, safe="")
+        resource_group = quote(resource_group, safe="")
+
+        self._base_url: Final[str] = (
+            f"https://management.azure.com/subscriptions/{subscription_id}/resourceGroups/{resource_group}"
+        )
+        self._logger: Final[Logger] = logger
+        self._api_version: Final[str] = kwargs.get("api_version", API_VERSION)
+        self._http_client: Final[HttpPipeline] = get_http_client(**kwargs)
+        self._lock: Final[Lock] = Lock()
+
+        # things that can change under lock
+        self._token_manager: Optional[AzureMLTokenManager] = None
+        self._credential: Optional[TokenCredential] = credential
+
+    def get_token(self) -> str:
+        return self._get_token_manager().get_token()
+
+    def get_credential(self) -> TokenCredential:
+        # load the token manager to get the credential if needed
+        self._get_token_manager()
+        return cast(TokenCredential, self._credential)
+
+    def workspace_get_default_datastore(self, workspace_name: str, include_credentials: bool = False) -> BlobStoreInfo:
+        # 1. Get the default blob store
+        # REST API documentation:
+        # https://learn.microsoft.com/rest/api/azureml/datastores/list?view=rest-azureml-2024-10-01
+        url = self._generate_path(  # pylint: disable=specify-parameter-names-in-call
+            *PATH_ML_WORKSPACES, workspace_name, "datastores"
+        )
+        headers = self._get_headers()
+
+        stores_response = self._http_client.request(
+            method="GET",
+            url=url,
+            params={QUERY_KEY_API_VERSION: self._api_version, "isDefault": True, "count": 1, "orderByAsc": "false"},
+            headers=headers,
+        )
+        self._throw_on_http_error(stores_response, "list default workspace datastore")
+
+        json = stores_response.json()["value"][0]
+        props_json = json["properties"]
+        name = json["name"]
+        account_name = props_json["accountName"]
+        endpoint = props_json["endpoint"]
+        container_name = props_json["containerName"]
+
+        # 2. Get the SAS token to use for accessing the blob store
+        # REST API documentation:
+        # https://learn.microsoft.com/rest/api/azureml/datastores/list-secrets?view=rest-azureml-2024-10-01
+        blob_store_credential: Optional[Union[AzureSasCredential, str]] = None
+        if include_credentials:
+            url = self._generate_path(
+                *PATH_ML_WORKSPACES, workspace_name, "datastores", "workspaceblobstore", "listSecrets"
+            )
+            secrets_response = self._http_client.request(
+                method="POST",
+                url=url,
+                params={
+                    QUERY_KEY_API_VERSION: self._api_version,
+                },
+                headers=headers,
+            )
+            self._throw_on_http_error(secrets_response, "workspace datastore secrets")
+
+            secrets_json = secrets_response.json()
+            secrets_type = secrets_json["secretsType"].lower()
+
+            if secrets_type == "sas":
+                blob_store_credential = AzureSasCredential(secrets_json["sasToken"])
+            elif secrets_type == "accountkey":
+                # To support olders versions of azure-storage-blob better, we return a string here instead of
+                # an AzureNamedKeyCredential
+                blob_store_credential = secrets_json["key"]
+            else:
+                raise EvaluationException(
+                    message=f"The '{account_name}' blob store does not use a recognized credential type.",
+                    internal_message=f"The credential type is '{secrets_type}'",
+                    target=ErrorTarget.EVALUATE,
+                    category=ErrorCategory.INVALID_VALUE,
+                    blame=ErrorBlame.SYSTEM_ERROR,
+                )
+
+        return BlobStoreInfo(name, account_name, endpoint, container_name, blob_store_credential)
+
+    def workspace_get_info(self, workspace_name: str) -> Workspace:
+        # https://learn.microsoft.com/rest/api/azureml/workspaces/get?view=rest-azureml-2024-10-01
+        workspace_response = self._http_client.request(
+            "GET",
+            self._generate_path(*PATH_ML_WORKSPACES, workspace_name),
+            params={QUERY_KEY_API_VERSION: self._api_version},
+            headers=self._get_headers(),
+        )
+
+        self._throw_on_http_error(workspace_response, f"get '{workspace_name}' workspace")
+        workspace = Workspace.deserialize(workspace_response)
+        return workspace
+
+    def _get_token_manager(self) -> AzureMLTokenManager:
+        # Lazy init since getting credentials in the constructor can take a long time in some situations
+        if self._token_manager is None:
+            with self._lock:
+                if self._token_manager is None:
+                    self._token_manager = AzureMLTokenManager(
+                        TokenScope.DEFAULT_AZURE_MANAGEMENT.value, self._logger, credential=self._credential
+                    )
+                    self._credential = self._token_manager.credential
+
+        return self._token_manager
+
+    @staticmethod
+    def _throw_on_http_error(response: HttpResponse, description: str, valid_status: Optional[Set[int]] = None) -> None:
+        if valid_status and (response.status_code in valid_status):
+            return
+        if response.status_code >= 200 and response.status_code < 300:
+            # nothing to see here, move along
+            return
+
+        additional_info: Optional[str] = None
+        try:
+            error_json = response.json()["error"]
+            additional_info = f"({error_json['code']}) {error_json['message']}"
+        except (JSONDecodeError, ValueError, KeyError):
+            pass
+
+        raise EvaluationException(
+            message=f"The {description} request failed with HTTP {response.status_code}",
+            target=ErrorTarget.EVALUATE,
+            category=ErrorCategory.FAILED_EXECUTION,
+            blame=ErrorBlame.SYSTEM_ERROR,
+            internal_message=additional_info,
+        )
+
+    def _generate_path(self, *paths: str) -> str:
+        sanitized_paths = [quote(path, safe="") for path in paths]
+        url = self._base_url + "/" + str.join("/", sanitized_paths)
+        return url
+
+    def _get_headers(self) -> Dict[str, str]:
+        return {"Authorization": f"Bearer {self.get_token()}", "Content-Type": "application/json"}

azure_ai_evaluation-1.1.0/azure/ai/evaluation/_azure/_models.py

@@ -0,0 +1,227 @@
+# ---------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# ---------------------------------------------------------
+
+# pylint: disable=too-many-instance-attributes
+# pylint: disable=too-many-locals
+# pylint: disable=line-too-long
+
+from typing import Dict, List, NamedTuple, Optional, Union
+from msrest.serialization import Model
+from azure.core.credentials import AzureSasCredential
+
+
+class BlobStoreInfo(NamedTuple):
+    name: str
+    account_name: str
+    endpoint: str
+    container_name: str
+    credential: Optional[Union[AzureSasCredential, str]]
+
+
+class WorkspaceHubConfig(Model):
+    """WorkspaceHub's configuration object."""
+
+    _attribute_map = {
+        "additional_workspace_storage_accounts": {"key": "additionalWorkspaceStorageAccounts", "type": "[str]"},
+        "default_workspace_resource_group": {"key": "defaultWorkspaceResourceGroup", "type": "str"},
+    }
+
+    def __init__(
+        self,
+        *,
+        additional_workspace_storage_accounts: Optional[List[str]] = None,
+        default_workspace_resource_group: Optional[str] = None,
+        **kwargs
+    ):
+        super(WorkspaceHubConfig, self).__init__(**kwargs)
+        self.additional_workspace_storage_accounts = additional_workspace_storage_accounts
+        self.default_workspace_resource_group = default_workspace_resource_group
+
+
+class Workspace(Model):
+    """An object that represents a machine learning workspace.
+
+    Variables are only populated by the server, and will be ignored when sending a request."""
+
+    _validation = {
+        "id": {"readonly": True},
+        "name": {"readonly": True},
+        "type": {"readonly": True},
+        #'system_data': {'readonly': True},
+        "agents_endpoint_uri": {"readonly": True},
+        "ml_flow_tracking_uri": {"readonly": True},
+        #'notebook_info': {'readonly': True},
+        "private_endpoint_connections": {"readonly": True},
+        #'private_link_count': {'readonly': True},
+        "provisioning_state": {"readonly": True},
+        "service_provisioned_resource_group": {"readonly": True},
+        "storage_hns_enabled": {"readonly": True},
+        "tenant_id": {"readonly": True},
+        "workspace_id": {"readonly": True},
+    }
+
+    _attribute_map = {
+        "id": {"key": "id", "type": "str"},
+        "name": {"key": "name", "type": "str"},
+        "type": {"key": "type", "type": "str"},
+        #'system_data': {'key': 'systemData', 'type': 'SystemData'},
+        #'identity': {'key': 'identity', 'type': 'ManagedServiceIdentity'},
+        "kind": {"key": "kind", "type": "str"},
+        "location": {"key": "location", "type": "str"},
+        #'sku': {'key': 'sku', 'type': 'Sku'},
+        "tags": {"key": "tags", "type": "{str}"},
+        "agents_endpoint_uri": {"key": "properties.agentsEndpointUri", "type": "str"},
+        "allow_public_access_when_behind_vnet": {"key": "properties.allowPublicAccessWhenBehindVnet", "type": "bool"},
+        "allow_role_assignment_on_rg": {"key": "properties.allowRoleAssignmentOnRG", "type": "bool"},
+        "application_insights": {"key": "properties.applicationInsights", "type": "str"},
+        "associated_workspaces": {"key": "properties.associatedWorkspaces", "type": "[str]"},
+        "container_registries": {"key": "properties.containerRegistries", "type": "[str]"},
+        "container_registry": {"key": "properties.containerRegistry", "type": "str"},
+        "description": {"key": "properties.description", "type": "str"},
+        "discovery_url": {"key": "properties.discoveryUrl", "type": "str"},
+        "enable_data_isolation": {"key": "properties.enableDataIsolation", "type": "bool"},
+        "enable_service_side_cmk_encryption": {"key": "properties.enableServiceSideCMKEncryption", "type": "bool"},
+        "enable_simplified_cmk": {"key": "properties.enableSimplifiedCmk", "type": "bool"},
+        "enable_software_bill_of_materials": {"key": "properties.enableSoftwareBillOfMaterials", "type": "bool"},
+        #'encryption': {'key': 'properties.encryption', 'type': 'EncryptionProperty'},
+        "existing_workspaces": {"key": "properties.existingWorkspaces", "type": "[str]"},
+        #'feature_store_settings': {'key': 'properties.featureStoreSettings', 'type': 'FeatureStoreSettings'},
+        "friendly_name": {"key": "properties.friendlyName", "type": "str"},
+        "hbi_workspace": {"key": "properties.hbiWorkspace", "type": "bool"},
+        "hub_resource_id": {"key": "properties.hubResourceId", "type": "str"},
+        "image_build_compute": {"key": "properties.imageBuildCompute", "type": "str"},
+        "ip_allowlist": {"key": "properties.ipAllowlist", "type": "[str]"},
+        "key_vault": {"key": "properties.keyVault", "type": "str"},
+        "key_vaults": {"key": "properties.keyVaults", "type": "[str]"},
+        #'managed_network': {'key': 'properties.managedNetwork', 'type': 'ManagedNetworkSettings'},
+        "ml_flow_tracking_uri": {"key": "properties.mlFlowTrackingUri", "type": "str"},
+        #'network_acls': {'key': 'properties.networkAcls', 'type': 'NetworkAcls'},
+        #'notebook_info': {'key': 'properties.notebookInfo', 'type': 'NotebookResourceInfo'},
+        "primary_user_assigned_identity": {"key": "properties.primaryUserAssignedIdentity", "type": "str"},
+        "private_endpoint_connections": {
+            "key": "properties.privateEndpointConnections",
+            "type": "[PrivateEndpointConnection]",
+        },
+        "private_link_count": {"key": "properties.privateLinkCount", "type": "int"},
+        "provision_network_now": {"key": "properties.provisionNetworkNow", "type": "bool"},
+        "provisioning_state": {"key": "properties.provisioningState", "type": "str"},
+        #'public_network_access': {'key': 'properties.publicNetworkAccess', 'type': 'str'},
+        #'serverless_compute_settings': {'key': 'properties.serverlessComputeSettings', 'type': 'ServerlessComputeSettings'},
+        #'service_managed_resources_settings': {'key': 'properties.serviceManagedResourcesSettings', 'type': 'ServiceManagedResourcesSettings'},
+        "service_provisioned_resource_group": {"key": "properties.serviceProvisionedResourceGroup", "type": "str"},
+        #'shared_private_link_resources': {'key': 'properties.sharedPrivateLinkResources', 'type': '[SharedPrivateLinkResource]'},
+        "soft_delete_retention_in_days": {"key": "properties.softDeleteRetentionInDays", "type": "int"},
+        "storage_account": {"key": "properties.storageAccount", "type": "str"},
+        "storage_accounts": {"key": "properties.storageAccounts", "type": "[str]"},
+        "storage_hns_enabled": {"key": "properties.storageHnsEnabled", "type": "bool"},
+        #'system_datastores_auth_mode': {'key': 'properties.systemDatastoresAuthMode', 'type': 'str'},
+        "tenant_id": {"key": "properties.tenantId", "type": "str"},
+        "v1_legacy_mode": {"key": "properties.v1LegacyMode", "type": "bool"},
+        "workspace_hub_config": {"key": "properties.workspaceHubConfig", "type": "WorkspaceHubConfig"},
+        "workspace_id": {"key": "properties.workspaceId", "type": "str"},
+    }
+
+    def __init__(
+        self,
+        *,
+        # system_data: Optional[SystemData] = None,
+        # identity: Optional["ManagedServiceIdentity"] = None,
+        kind: Optional[str] = None,
+        location: Optional[str] = None,
+        # sku: Optional["Sku"] = None,
+        tags: Optional[Dict[str, str]] = None,
+        allow_public_access_when_behind_vnet: Optional[bool] = None,
+        allow_role_assignment_on_rg: Optional[bool] = None,
+        application_insights: Optional[str] = None,
+        associated_workspaces: Optional[List[str]] = None,
+        container_registries: Optional[List[str]] = None,
+        container_registry: Optional[str] = None,
+        description: Optional[str] = None,
+        discovery_url: Optional[str] = None,
+        enable_data_isolation: Optional[bool] = None,
+        enable_service_side_cmk_encryption: Optional[bool] = None,
+        enable_simplified_cmk: Optional[bool] = None,
+        enable_software_bill_of_materials: Optional[bool] = None,
+        # encryption: Optional["EncryptionProperty"] = None,
+        existing_workspaces: Optional[List[str]] = None,
+        # feature_store_settings: Optional["FeatureStoreSettings"] = None,
+        friendly_name: Optional[str] = None,
+        hbi_workspace: Optional[bool] = None,
+        hub_resource_id: Optional[str] = None,
+        image_build_compute: Optional[str] = None,
+        ip_allowlist: Optional[List[str]] = None,
+        key_vault: Optional[str] = None,
+        key_vaults: Optional[List[str]] = None,
+        # managed_network: Optional["ManagedNetworkSettings"] = None,
+        # network_acls: Optional["NetworkAcls"] = None,
+        primary_user_assigned_identity: Optional[str] = None,
+        provision_network_now: Optional[bool] = None,
+        # public_network_access: Optional[Union[str, "PublicNetworkAccessType"]] = None,
+        # serverless_compute_settings: Optional["ServerlessComputeSettings"] = None,
+        # service_managed_resources_settings: Optional["ServiceManagedResourcesSettings"] = None,
+        # shared_private_link_resources: Optional[List["SharedPrivateLinkResource"]] = None,
+        soft_delete_retention_in_days: Optional[int] = None,
+        storage_account: Optional[str] = None,
+        storage_accounts: Optional[List[str]] = None,
+        # system_datastores_auth_mode: Optional[Union[str, "SystemDatastoresAuthMode"]] = None,
+        v1_legacy_mode: Optional[bool] = None,
+        workspace_hub_config: Optional["WorkspaceHubConfig"] = None,
+        **kwargs
+    ):
+        super(Workspace, self).__init__(**kwargs)
+        self.id: Optional[str] = None
+        self.name: Optional[str] = None
+        self.type: Optional[str] = None
+        # self.system_data = system_data
+        # self.identity = identity
+        self.kind = kind
+        self.location = location
+        # self.sku = sku
+        self.tags = tags
+        self.agents_endpoint_uri = None
+        self.allow_public_access_when_behind_vnet = allow_public_access_when_behind_vnet
+        self.allow_role_assignment_on_rg = allow_role_assignment_on_rg
+        self.application_insights = application_insights
+        self.associated_workspaces = associated_workspaces
+        self.container_registries = container_registries
+        self.container_registry = container_registry
+        self.description = description
+        self.discovery_url = discovery_url
+        self.enable_data_isolation = enable_data_isolation
+        self.enable_service_side_cmk_encryption = enable_service_side_cmk_encryption
+        self.enable_simplified_cmk = enable_simplified_cmk
+        self.enable_software_bill_of_materials = enable_software_bill_of_materials
+        # self.encryption = encryption
+        self.existing_workspaces = existing_workspaces
+        # self.feature_store_settings = feature_store_settings
+        self.friendly_name = friendly_name
+        self.hbi_workspace = hbi_workspace
+        self.hub_resource_id = hub_resource_id
+        self.image_build_compute = image_build_compute
+        self.ip_allowlist = ip_allowlist
+        self.key_vault = key_vault
+        self.key_vaults = key_vaults
+        # self.managed_network = managed_network
+        self.ml_flow_tracking_uri = None
+        # self.network_acls = network_acls
+        # self.notebook_info = None
+        self.primary_user_assigned_identity = primary_user_assigned_identity
+        self.private_endpoint_connections = None
+        self.private_link_count = None
+        self.provision_network_now = provision_network_now
+        self.provisioning_state = None
+        # self.public_network_access = public_network_access
+        # self.serverless_compute_settings = serverless_compute_settings
+        # self.service_managed_resources_settings = service_managed_resources_settings
+        self.service_provisioned_resource_group = None
+        # self.shared_private_link_resources = shared_private_link_resources
+        self.soft_delete_retention_in_days = soft_delete_retention_in_days
+        self.storage_account = storage_account
+        self.storage_accounts = storage_accounts
+        self.storage_hns_enabled = None
+        # self.system_datastores_auth_mode = system_datastores_auth_mode
+        self.tenant_id = None
+        self.v1_legacy_mode = v1_legacy_mode
+        self.workspace_hub_config = workspace_hub_config
+        self.workspace_id = None