semantic-link-labs 0.8.3__tar.gz → 0.8.4__tar.gz

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: semantic-link-labs
-Version: 0.8.3
+Version: 0.8.4
 Summary: Semantic Link Labs for Microsoft Fabric
 Author: Microsoft Corporation
 License: MIT License
@@ -15,7 +15,7 @@ Classifier: Framework :: Jupyter
 Requires-Python: <3.12,>=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: semantic-link-sempy>=0.8.0
+Requires-Dist: semantic-link-sempy>=0.8.1
 Requires-Dist: anytree
 Requires-Dist: powerbiclient
 Requires-Dist: polib
@@ -27,7 +27,7 @@ Requires-Dist: pytest>=8.2.1; extra == "test"
 # Semantic Link Labs
 
 [![PyPI version](https://badge.fury.io/py/semantic-link-labs.svg)](https://badge.fury.io/py/semantic-link-labs)
-[![Read The Docs](https://readthedocs.org/projects/semantic-link-labs/badge/?version=0.8.3&style=flat)](https://readthedocs.org/projects/semantic-link-labs/)
+[![Read The Docs](https://readthedocs.org/projects/semantic-link-labs/badge/?version=0.8.4&style=flat)](https://readthedocs.org/projects/semantic-link-labs/)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 [![Downloads](https://static.pepy.tech/badge/semantic-link-labs)](https://pepy.tech/project/semantic-link-labs)
 
@@ -37,6 +37,10 @@ Requires-Dist: pytest>=8.2.1; extra == "test"
 
 Semantic Link Labs is a Python library designed for use in [Microsoft Fabric notebooks](https://learn.microsoft.com/fabric/data-engineering/how-to-use-notebook). This library extends the capabilities of [Semantic Link](https://learn.microsoft.com/fabric/data-science/semantic-link-overview) offering additional functionalities to seamlessly integrate and work alongside it. The goal of Semantic Link Labs is to simplify technical processes, empowering people to focus on higher level activities and allowing tasks that are better suited for machines to be efficiently handled without human intervention.
 
+If you encounter any issues, please [raise a bug](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=).
+
+If you have ideas for new features/functions, please [request a feature](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=feature_request.md&title=).
+
 ## Featured Scenarios
 * Semantic Models
     * [Migrating an import/DirectQuery semantic model to Direct Lake](https://github.com/microsoft/semantic-link-labs?tab=readme-ov-file#direct-lake-migration)
@@ -47,26 +51,46 @@ Semantic Link Labs is a Python library designed for use in [Microsoft Fabric not
     * [Check Direct Lake Guardrails](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.get_lakehouse_tables)
     * [Refresh](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Semantic%20Model%20Refresh.ipynb), [clear cache](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.clear_cache), [backup](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.backup_semantic_model), [restore](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.restore_semantic_model), [copy backup files](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.copy_semantic_model_backup_file), [move/deploy across workspaces](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.deploy_semantic_model)
     * [Run DAX queries which impersonate a user](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.evaluate_dax_impersonation)
-    * [Manage Query Scale Out](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Query%20Scale%20Out.ipynb).
+    * [Manage Query Scale Out](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Query%20Scale%20Out.ipynb)
     * [Auto-generate descriptions for any/all measures in bulk](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.tom.html#sempy_labs.tom.TOMWrapper.generate_measure_descriptions)
+    * [Warm the cache of a Direct Lake semantic model after a refresh (using columns currently in memory)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.directlake.html#sempy_labs.directlake.warm_direct_lake_cache_isresident)
+    * [Warm the cache of a Direct Lake semantic model (via perspective)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.directlake.html#sempy_labs.directlake.warm_direct_lake_cache_perspective)
+    * [Visualize a refresh](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Semantic%20Model%20Refresh.ipynb)
 * Reports
     * [Report Best Practice Analyzer (BPA)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.run_report_bpa)
     * [View report metadata](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Report%20Analysis.ipynb)
     * [View semantic model objects most frequently used in Power BI reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.list_semantic_model_object_report_usage)
     * [View broken reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.list_report_semantic_model_objects)
+    * [Set a report theme](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.ReportWrapper.set_theme)
+    * [Migrate report-level measures to the semantic model](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.ReportWrapper.migrate_report_level_measures)
     * [Rebind reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.report_rebind)
 * Capacities
     * [Migrating a Power BI Premium capacity (P sku) to a Fabric capacity (F sku)](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Capacity%20Migration.ipynb)
+    * [Migrating a Fabric Trial capacity (FT sku) to a Fabric capacity (F sku)](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Capacity%20Migration.ipynb)
     * [Create](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.create_fabric_capacity)/[update](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.update_fabric_capacity)/[suspend](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.suspend_fabric_capacity)/[resume](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.resume_fabric_capacity) Fabric capacities.
+* Lakehouses
+    * [Optimize lakehouse tables](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.optimize_lakehouse_tables)
+    * [Vacuum lakehouse tables](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.vacuum_lakehouse_tables)
 * APIs
     * Wrapper functions for [Power BI](https://learn.microsoft.com/rest/api/power-bi/), [Fabric](https://learn.microsoft.com/rest/api/fabric/articles/using-fabric-apis), and [Azure (Fabric Capacity)](https://learn.microsoft.com/rest/api/microsoftfabric/fabric-capacities?view=rest-microsoftfabric-2023-11-01) APIs
 
+## Helper Notebooks
 
-### Check out the [helper notebooks](https://github.com/microsoft/semantic-link-labs/tree/main/notebooks) for getting started!
+Check out the [helper notebooks](https://github.com/microsoft/semantic-link-labs/tree/main/notebooks) for getting started! 
+Run the code below to load all the helper notebooks to the workspace of your choice at once.
+```python
+import sempy_labs as labs
+import requests
 
-If you encounter any issues, please [raise a bug](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=).
+workspace_name = None # Update this to the workspace in which you want to save the notebooks
+api_url = "https://api.github.com/repos/microsoft/semantic-link-labs/contents/notebooks"
+response = requests.get(api_url)
+files = response.json()
+notebook_files = {file['name'][:-6]: file['html_url'] for file in files if file['name'].endswith('.ipynb')}
 
-If you have ideas for new features/functions, please [request a feature](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=feature_request.md&title=).
+for file_name, file_url in notebook_files.items():
+    labs.import_notebook_from_web(notebook_name=file_name, url=file_url, workspace=workspace_name)
+```
 
 ## Install the library in a Fabric notebook
 ```python
@@ -85,7 +109,7 @@ from sempy_labs import ConnectWarehouse
 from sempy_labs import ConnectLakehouse
 ```
 
-## Load semantic-link-labs into a custom [Fabric environment](https://learn.microsoft.com/fabric/data-engineering/create-and-use-environment)
+## Load Semantic Link Labs into a custom [Fabric environment](https://learn.microsoft.com/fabric/data-engineering/create-and-use-environment)
 An even better way to ensure the semantic-link-labs library is available in your workspace/notebooks is to load it as a library in a custom Fabric environment. If you do this, you will not have to run the above '%pip install' code every time in your notebook. Please follow the steps below.
 
 #### Create a custom environment
@@ -106,6 +130,7 @@ An even better way to ensure the semantic-link-labs library is available in your
 2. Select your newly created environment within the 'Environment' drop down in the navigation bar at the top of the notebook
 
 ## Version History
+* [0.8.4](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.4) (October 30, 2024)
 * [0.8.3](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.3) (October 14, 2024)
 * [0.8.2](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.2) (October 2, 2024)
 * [0.8.1](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.1) (October 2, 2024)

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/README.md

@@ -1,7 +1,7 @@
 # Semantic Link Labs
 
 [![PyPI version](https://badge.fury.io/py/semantic-link-labs.svg)](https://badge.fury.io/py/semantic-link-labs)
-[![Read The Docs](https://readthedocs.org/projects/semantic-link-labs/badge/?version=0.8.3&style=flat)](https://readthedocs.org/projects/semantic-link-labs/)
+[![Read The Docs](https://readthedocs.org/projects/semantic-link-labs/badge/?version=0.8.4&style=flat)](https://readthedocs.org/projects/semantic-link-labs/)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 [![Downloads](https://static.pepy.tech/badge/semantic-link-labs)](https://pepy.tech/project/semantic-link-labs)
 
@@ -11,6 +11,10 @@
 
 Semantic Link Labs is a Python library designed for use in [Microsoft Fabric notebooks](https://learn.microsoft.com/fabric/data-engineering/how-to-use-notebook). This library extends the capabilities of [Semantic Link](https://learn.microsoft.com/fabric/data-science/semantic-link-overview) offering additional functionalities to seamlessly integrate and work alongside it. The goal of Semantic Link Labs is to simplify technical processes, empowering people to focus on higher level activities and allowing tasks that are better suited for machines to be efficiently handled without human intervention.
 
+If you encounter any issues, please [raise a bug](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=).
+
+If you have ideas for new features/functions, please [request a feature](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=feature_request.md&title=).
+
 ## Featured Scenarios
 * Semantic Models
     * [Migrating an import/DirectQuery semantic model to Direct Lake](https://github.com/microsoft/semantic-link-labs?tab=readme-ov-file#direct-lake-migration)
@@ -21,26 +25,46 @@ Semantic Link Labs is a Python library designed for use in [Microsoft Fabric not
     * [Check Direct Lake Guardrails](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.get_lakehouse_tables)
     * [Refresh](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Semantic%20Model%20Refresh.ipynb), [clear cache](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.clear_cache), [backup](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.backup_semantic_model), [restore](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.restore_semantic_model), [copy backup files](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.copy_semantic_model_backup_file), [move/deploy across workspaces](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.deploy_semantic_model)
     * [Run DAX queries which impersonate a user](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.evaluate_dax_impersonation)
-    * [Manage Query Scale Out](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Query%20Scale%20Out.ipynb).
+    * [Manage Query Scale Out](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Query%20Scale%20Out.ipynb)
     * [Auto-generate descriptions for any/all measures in bulk](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.tom.html#sempy_labs.tom.TOMWrapper.generate_measure_descriptions)
+    * [Warm the cache of a Direct Lake semantic model after a refresh (using columns currently in memory)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.directlake.html#sempy_labs.directlake.warm_direct_lake_cache_isresident)
+    * [Warm the cache of a Direct Lake semantic model (via perspective)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.directlake.html#sempy_labs.directlake.warm_direct_lake_cache_perspective)
+    * [Visualize a refresh](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Semantic%20Model%20Refresh.ipynb)
 * Reports
     * [Report Best Practice Analyzer (BPA)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.run_report_bpa)
     * [View report metadata](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Report%20Analysis.ipynb)
     * [View semantic model objects most frequently used in Power BI reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.list_semantic_model_object_report_usage)
     * [View broken reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.list_report_semantic_model_objects)
+    * [Set a report theme](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.ReportWrapper.set_theme)
+    * [Migrate report-level measures to the semantic model](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.ReportWrapper.migrate_report_level_measures)
     * [Rebind reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.report_rebind)
 * Capacities
     * [Migrating a Power BI Premium capacity (P sku) to a Fabric capacity (F sku)](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Capacity%20Migration.ipynb)
+    * [Migrating a Fabric Trial capacity (FT sku) to a Fabric capacity (F sku)](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Capacity%20Migration.ipynb)
     * [Create](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.create_fabric_capacity)/[update](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.update_fabric_capacity)/[suspend](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.suspend_fabric_capacity)/[resume](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.resume_fabric_capacity) Fabric capacities.
+* Lakehouses
+    * [Optimize lakehouse tables](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.optimize_lakehouse_tables)
+    * [Vacuum lakehouse tables](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.vacuum_lakehouse_tables)
 * APIs
     * Wrapper functions for [Power BI](https://learn.microsoft.com/rest/api/power-bi/), [Fabric](https://learn.microsoft.com/rest/api/fabric/articles/using-fabric-apis), and [Azure (Fabric Capacity)](https://learn.microsoft.com/rest/api/microsoftfabric/fabric-capacities?view=rest-microsoftfabric-2023-11-01) APIs
 
+## Helper Notebooks
 
-### Check out the [helper notebooks](https://github.com/microsoft/semantic-link-labs/tree/main/notebooks) for getting started!
+Check out the [helper notebooks](https://github.com/microsoft/semantic-link-labs/tree/main/notebooks) for getting started! 
+Run the code below to load all the helper notebooks to the workspace of your choice at once.
+```python
+import sempy_labs as labs
+import requests
 
-If you encounter any issues, please [raise a bug](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=).
+workspace_name = None # Update this to the workspace in which you want to save the notebooks
+api_url = "https://api.github.com/repos/microsoft/semantic-link-labs/contents/notebooks"
+response = requests.get(api_url)
+files = response.json()
+notebook_files = {file['name'][:-6]: file['html_url'] for file in files if file['name'].endswith('.ipynb')}
 
-If you have ideas for new features/functions, please [request a feature](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=feature_request.md&title=).
+for file_name, file_url in notebook_files.items():
+    labs.import_notebook_from_web(notebook_name=file_name, url=file_url, workspace=workspace_name)
+```
 
 ## Install the library in a Fabric notebook
 ```python
@@ -59,7 +83,7 @@ from sempy_labs import ConnectWarehouse
 from sempy_labs import ConnectLakehouse
 ```
 
-## Load semantic-link-labs into a custom [Fabric environment](https://learn.microsoft.com/fabric/data-engineering/create-and-use-environment)
+## Load Semantic Link Labs into a custom [Fabric environment](https://learn.microsoft.com/fabric/data-engineering/create-and-use-environment)
 An even better way to ensure the semantic-link-labs library is available in your workspace/notebooks is to load it as a library in a custom Fabric environment. If you do this, you will not have to run the above '%pip install' code every time in your notebook. Please follow the steps below.
 
 #### Create a custom environment
@@ -80,6 +104,7 @@ An even better way to ensure the semantic-link-labs library is available in your
 2. Select your newly created environment within the 'Environment' drop down in the navigation bar at the top of the notebook
 
 ## Version History
+* [0.8.4](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.4) (October 30, 2024)
 * [0.8.3](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.3) (October 14, 2024)
 * [0.8.2](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.2) (October 2, 2024)
 * [0.8.1](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.1) (October 2, 2024)

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/docs/source/conf.py

@@ -13,7 +13,7 @@ sys.path.insert(0, os.path.abspath('../../src/'))
 project = 'semantic-link-labs'
 copyright = '2024, Microsoft and community'
 author = 'Microsoft and community'
-release = '0.8.3'
+release = '0.8.4'
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/environment.yml

@@ -6,7 +6,7 @@ dependencies:
 - pytest-cov
 - pytest-mock
 - pip:
-  - semantic-link-sempy>=0.8.0
+  - semantic-link-sempy>=0.8.1
   - azure-identity==1.7.1
   - azure-storage-blob>=12.9.0
   - pandas-stubs

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/notebooks/Capacity Migration.ipynb

@@ -1 +1 @@
-{"cells":[{"cell_type":"markdown","id":"5c27dfd1-4fe0-4a97-92e6-ddf78889aa93","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Install the latest .whl package\n","\n","Check [here](https://pypi.org/project/semantic-link-labs/) to see the latest version."]},{"cell_type":"code","execution_count":null,"id":"d5cae9db-cef9-48a8-a351-9c5fcc99645c","metadata":{"jupyter":{"outputs_hidden":true,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["%pip install semantic-link-labs"]},{"cell_type":"markdown","id":"2856d26d","metadata":{},"source":["### Requirements\n","* Must have an Azure Subscription\n","* Must [register an App](https://ms.portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade)\n","    * Permissions: Will need the Contributor role at the scope where the resources will be created, which is often the subscription level\n","* Azure Key Vault\n","    * [Set up](https://learn.microsoft.com/azure/key-vault/secrets/quick-create-portal) within the Azure Subscription\n","    * Save secrets for the Tenant ID, Client ID (Application ID), Client Secret\n","    * Permissions: Ensure the user who will be executing the notebooks has “Key Vault Secrets User”\n","* Fabric Permissions\n","    * User should be a tenant admin. This ensures they have the necessary authority to execute and manage the notebooks without encountering permission issues.\n","\n","### Result\n","* F skus are created for each (specified) capacity\n","    * Within the same region as the P SKU\n","    * Equivalent SKU size as the P SKU\n","    * Same admins as listed on the P SKU\n","    * All workspaces are migrated to the corresponding new capacity\n","    * Capacity settings from the P SKU are transferred to the F SKU\n","        * Capacity settings\n","        * Notification settings\n","        * Access settings\n","        * Disaster recovery settings\n","        * Spark settings\n","        * Delegated tenant settings\n","* The names of the newly created F SKU capacities will be an alphanumeric lowercase version of the P SKU capacity name, suffixed with 'fsku'. As an example: \"My capacity_3!\" -> \"mycapacity3fsku\"."]},{"cell_type":"markdown","id":"b195eae8","metadata":{},"source":["### Import the library and set the initial parameters"]},{"cell_type":"code","execution_count":null,"id":"1344e286","metadata":{},"outputs":[],"source":["import sempy_labs as labs\n","import sempy_labs.admin as admin\n","\n","azure_subscription_id = '' # Enter your Azure subscription ID\n","key_vault_uri = '' # Enter your Azure Key Vault URI\n","key_vault_tenant_id = '' # Enter the name of the Azure Key Vault secret which stores your Tenant ID\n","key_vault_client_id = '' # Enter the name of the Azure Key Vault secret which stores your Client ID (Application ID)\n","key_vault_client_secret = '' # Enter the name of the Azure Key Vault secret which stores your Client Secret\n","resource_group = '' # Enter the name of the resource group (to be used to create the new F skus)"]},{"cell_type":"markdown","id":"5a3fe6e8-b8aa-4447-812b-7931831e07fe","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Migrate a single P SKU -> F SKU\n","Set the 'capacities' parameter to the single P SKU."]},{"cell_type":"code","execution_count":null,"id":"3655dd88","metadata":{},"outputs":[],"source":["labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    capacities = 'CapacityA',\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"175a59b8","metadata":{},"source":["### Migrate a list of P SKUs to F SKUs\n","Set the 'capacities' parameter to a list of P SKUs."]},{"cell_type":"code","execution_count":null,"id":"3a7a80ec","metadata":{},"outputs":[],"source":["labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    capacities = ['CapacityA', 'CapacityB', 'CapacityC'],\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"30438799","metadata":{},"source":["### Migrate all P SKUs to F SKUs\n","Set the 'capacities' parameter to None."]},{"cell_type":"code","execution_count":null,"id":"315c2dc7","metadata":{},"outputs":[],"source":["labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    capacities = None,\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"1d8e73b2","metadata":{},"source":["### Migrate a list of P SKUs to F SKUs; associate each capacity with a specific resource group\n","This process ensures that each F SKU is created within the resource group specified in the resource_group_mapping dictionary."]},{"cell_type":"code","execution_count":null,"id":"2854bf8a","metadata":{},"outputs":[],"source":["resource_group_mapping = {\n","    \"CapacityA\": \"ResourceGroupA\",\n","    \"CapacityB\": \"ResourceGroupA\",\n","    \"CapacityC\": \"ResourceGroupB\",\n","}\n","\n","labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group_mapping,\n","    capacities = ['CapacityA', 'CapacityB', 'CapacityC'],\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"c3f497c8","metadata":{},"source":["### Migrate a single source capacity to a target capacity (target capacity already created)"]},{"cell_type":"code","execution_count":null,"id":"a4f0b5a2","metadata":{},"outputs":[],"source":["source_capacity = '' # Enter the Source capacity name\n","target_capacity = '' # Enter the Target capacity name (already exists) \n","\n","admin.assign_workspaces_to_capacity(\n","    source_capacity=source_capacity,\n","    target_capacity=target_capacity,\n","    workspace=None,\n",")\n","\n","# Optionally migrate settings\n","\"\"\"\n","labs.migrate_capacity_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_access_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_delegated_tenant_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_disaster_recovery_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_notification_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_spark_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","\"\"\""]},{"cell_type":"markdown","id":"e0db744b","metadata":{},"source":["### Migrate a list of source capacities to target capacities (target capacity already created)"]},{"cell_type":"code","execution_count":null,"id":"0e04d519","metadata":{},"outputs":[],"source":["capacity_mapping = {\n","    \"capacitya\": \"capacityanew\", # Format is \"Source SKU\": \"Target SKU\"\n","    \"capacityb\": \"capacitybnew\",\n","    \"capacityc\": \"capacitycnew\",\n","}\n","\n","capacities = list(capacity_mapping.keys())\n","\n","for capacity in capacities:\n","    admin.assign_workspaces_to_capacity(\n","        source_capacity=capacity,\n","        target_capacity=capacity_mapping.get(capacity),\n","        workspace=None,\n","    )\n","    # Optionally migrate settings\n","    \"\"\"\n","    labs.migrate_capacity_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_access_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_delegated_tenant_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_disaster_recovery_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_notification_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_spark_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    \"\"\"\n"]},{"cell_type":"markdown","id":"67787c3a","metadata":{},"source":["### Migrate a Fabric Trial Capacity (FT1) to a Fabric Capacity (F SKU)"]},{"cell_type":"code","execution_count":null,"id":"4efb55b2","metadata":{},"outputs":[],"source":["labs.migrate_fabric_trial_capacity(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    source_capacity = '', # The name of the Trial capacity.\n","    target_capacity = '', # The name of Fabric capacity (if it does not exist it will be created).\n","    target_capacity_sku = \"F64\", # Defaults to F64 but can specify the required SKU.\n","    target_capacity_region = None, # Setting this not None will create the Fabric capacity in the same region as the Trial capacity.\n","    target_capacity_admin_members = None, # Setting this to None will use the same admin members as the Trial capacity.\n",")"]}],"metadata":{"kernel_info":{"name":"synapse_pyspark"},"kernelspec":{"display_name":"Synapse PySpark","language":"Python","name":"synapse_pyspark"},"language_info":{"name":"python"},"microsoft":{"language":"python"},"nteract":{"version":"nteract-front-end@1.0.0"},"spark_compute":{"compute_id":"/trident/default"},"synapse_widget":{"state":{},"version":"0.1"},"widgets":{}},"nbformat":4,"nbformat_minor":5}
+{"cells":[{"cell_type":"markdown","id":"5c27dfd1-4fe0-4a97-92e6-ddf78889aa93","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Install the latest .whl package\n","\n","Check [here](https://pypi.org/project/semantic-link-labs/) to see the latest version."]},{"cell_type":"code","execution_count":null,"id":"d5cae9db-cef9-48a8-a351-9c5fcc99645c","metadata":{"jupyter":{"outputs_hidden":true,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["%pip install semantic-link-labs"]},{"cell_type":"markdown","id":"2856d26d","metadata":{},"source":["### Requirements\n","* Must have an Azure Subscription\n","* Must [register an App](https://ms.portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade)\n","    * Permissions: Will need the Contributor role at the scope where the resources will be created, which is often the subscription level\n","* Azure Key Vault\n","    * [Set up](https://learn.microsoft.com/azure/key-vault/secrets/quick-create-portal) within the Azure Subscription\n","    * Save secrets for the Tenant ID, Client ID (Application ID), Client Secret\n","    * Permissions: Ensure the user who will be executing the notebooks has “Key Vault Secrets User”\n","* Fabric Permissions\n","    * User should be a tenant admin. This ensures they have the necessary authority to execute and manage the notebooks without encountering permission issues.\n","\n","### Result\n","* F skus are created for each (specified) capacity\n","    * Within the same region as the P SKU\n","    * Equivalent SKU size as the P SKU\n","    * Same admins as listed on the P SKU\n","    * All workspaces are migrated to the corresponding new capacity\n","    * Capacity settings from the P SKU are transferred to the F SKU\n","        * Capacity settings\n","        * Notification settings\n","        * Access settings\n","        * Disaster recovery settings\n","        * Spark settings\n","        * Delegated tenant settings\n","* The names of the newly created F SKU capacities will be an alphanumeric lowercase version of the P SKU capacity name, suffixed with 'fsku'. As an example: \"My capacity_3!\" -> \"mycapacity3fsku\"."]},{"cell_type":"markdown","id":"b195eae8","metadata":{},"source":["### Import the library and set the initial parameters"]},{"cell_type":"code","execution_count":null,"id":"1344e286","metadata":{},"outputs":[],"source":["import sempy_labs as labs\n","import sempy_labs.admin as admin\n","\n","azure_subscription_id = '' # Enter your Azure subscription ID\n","key_vault_uri = '' # Enter your Azure Key Vault URI\n","key_vault_tenant_id = '' # Enter the name of the Azure Key Vault secret which stores your Tenant ID\n","key_vault_client_id = '' # Enter the name of the Azure Key Vault secret which stores your Client ID (Application ID)\n","key_vault_client_secret = '' # Enter the name of the Azure Key Vault secret which stores your Client Secret\n","resource_group = '' # Enter the name of the resource group (to be used to create the new F skus)"]},{"cell_type":"markdown","id":"5a3fe6e8-b8aa-4447-812b-7931831e07fe","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Migrate a single P SKU -> F SKU\n","Set the 'capacities' parameter to the single P SKU."]},{"cell_type":"code","execution_count":null,"id":"3655dd88","metadata":{},"outputs":[],"source":["labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    capacities = 'CapacityA',\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"175a59b8","metadata":{},"source":["### Migrate a list of P SKUs to F SKUs\n","Set the 'capacities' parameter to a list of P SKUs."]},{"cell_type":"code","execution_count":null,"id":"3a7a80ec","metadata":{},"outputs":[],"source":["labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    capacities = ['CapacityA', 'CapacityB', 'CapacityC'],\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"30438799","metadata":{},"source":["### Migrate all P SKUs to F SKUs\n","Set the 'capacities' parameter to None."]},{"cell_type":"code","execution_count":null,"id":"315c2dc7","metadata":{},"outputs":[],"source":["labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    capacities = None,\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"1d8e73b2","metadata":{},"source":["### Migrate a list of P SKUs to F SKUs; associate each capacity with a specific resource group\n","This process ensures that each F SKU is created within the resource group specified in the resource_group_mapping dictionary."]},{"cell_type":"code","execution_count":null,"id":"2854bf8a","metadata":{},"outputs":[],"source":["resource_group_mapping = {\n","    \"CapacityA\": \"ResourceGroupA\",\n","    \"CapacityB\": \"ResourceGroupA\",\n","    \"CapacityC\": \"ResourceGroupB\",\n","}\n","\n","labs.migrate_capacities(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group_mapping,\n","    capacities = ['CapacityA', 'CapacityB', 'CapacityC'],\n","    p_sku_only = True,\n",")"]},{"cell_type":"markdown","id":"c3f497c8","metadata":{},"source":["### Migrate a single source capacity to a target capacity (target capacity already created)"]},{"cell_type":"code","execution_count":null,"id":"a4f0b5a2","metadata":{},"outputs":[],"source":["source_capacity = '' # Enter the Source capacity name\n","target_capacity = '' # Enter the Target capacity name (already exists) \n","\n","admin.assign_workspaces_to_capacity(\n","    source_capacity=source_capacity,\n","    target_capacity=target_capacity,\n","    workspace=None,\n",")\n","\n","# Optionally migrate settings\n","\"\"\"\n","labs.migrate_capacity_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_access_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_delegated_tenant_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_disaster_recovery_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_notification_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","labs.migrate_spark_settings(\n","    source_capacity=source_capacity, \n","    target_capacity=target_capacity\n",")\n","\"\"\""]},{"cell_type":"markdown","id":"e0db744b","metadata":{},"source":["### Migrate a list of source capacities to target capacities (target capacity already created)"]},{"cell_type":"code","execution_count":null,"id":"0e04d519","metadata":{},"outputs":[],"source":["capacity_mapping = {\n","    \"capacitya\": \"capacityanew\", # Format is \"Source SKU\": \"Target SKU\"\n","    \"capacityb\": \"capacitybnew\",\n","    \"capacityc\": \"capacitycnew\",\n","}\n","\n","capacities = list(capacity_mapping.keys())\n","\n","for capacity in capacities:\n","    admin.assign_workspaces_to_capacity(\n","        source_capacity=capacity,\n","        target_capacity=capacity_mapping.get(capacity),\n","        workspace=None,\n","    )\n","    # Optionally migrate settings\n","    \"\"\"\n","    labs.migrate_capacity_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_access_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_delegated_tenant_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_disaster_recovery_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_notification_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    labs.migrate_spark_settings(\n","        source_capacity=source_capacity, \n","        target_capacity=target_capacity\n","    )\n","    \"\"\"\n"]},{"cell_type":"markdown","id":"67787c3a","metadata":{},"source":["### Migrate a Fabric Trial Capacity (FT1) to a Fabric Capacity (F SKU)"]},{"cell_type":"code","execution_count":null,"id":"4efb55b2","metadata":{},"outputs":[],"source":["labs.migrate_fabric_trial_capacity(\n","    azure_subscription_id = azure_subscription_id,\n","    key_vault_uri = key_vault_uri,\n","    key_vault_tenant_id = key_vault_tenant_id,\n","    key_vault_client_id = key_vault_client_id,\n","    key_vault_client_secret = key_vault_client_secret,\n","    resource_group = resource_group,\n","    source_capacity = '', # The name of the Trial capacity.\n","    target_capacity = '', # The name of Fabric capacity (if it does not exist it will be created).\n","    target_capacity_sku = \"F64\", # Defaults to F64 but can specify the required SKU.\n","    target_capacity_admin_members = None, # Setting this to None will use the same admin members as the Trial capacity.\n",")"]}],"metadata":{"kernel_info":{"name":"synapse_pyspark"},"kernelspec":{"display_name":"Synapse PySpark","language":"Python","name":"synapse_pyspark"},"language_info":{"name":"python"},"microsoft":{"language":"python"},"nteract":{"version":"nteract-front-end@1.0.0"},"spark_compute":{"compute_id":"/trident/default"},"synapse_widget":{"state":{},"version":"0.1"},"widgets":{}},"nbformat":4,"nbformat_minor":5}

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/notebooks/Migration to Direct Lake.ipynb

@@ -1 +1 @@
-{"cells":[{"cell_type":"markdown","id":"5c27dfd1-4fe0-4a97-92e6-ddf78889aa93","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Install the latest .whl package\n","\n","Check [here](https://pypi.org/project/semantic-link-labs/) to see the latest version."]},{"cell_type":"code","execution_count":null,"id":"d5cae9db-cef9-48a8-a351-9c5fcc99645c","metadata":{"jupyter":{"outputs_hidden":true,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["%pip install semantic-link-labs"]},{"cell_type":"markdown","id":"969a29bf","metadata":{},"source":["### Import the library and set initial parameters"]},{"cell_type":"code","execution_count":null,"id":"29c923f8","metadata":{},"outputs":[],"source":["import sempy_labs as labs\n","from sempy_labs import migration, directlake\n","import sempy_labs.report as rep\n","\n","dataset_name = '' #Enter the import/DQ semantic model name\n","workspace_name = None #Enter the workspace of the import/DQ semantic model. It set to none it will use the current workspace.\n","new_dataset_name = '' #Enter the new Direct Lake semantic model name\n","new_dataset_workspace_name = None #Enter the workspace where the Direct Lake model will be created. If set to None it will use the current workspace.\n","lakehouse_name = None #Enter the lakehouse to be used for the Direct Lake model. If set to None it will use the lakehouse attached to the notebook.\n","lakehouse_workspace_name = None #Enter the lakehouse workspace. If set to None it will use the new_dataset_workspace_name."]},{"cell_type":"markdown","id":"5a3fe6e8-b8aa-4447-812b-7931831e07fe","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Create the [Power Query Template](https://learn.microsoft.com/power-query/power-query-template) file\n","\n","This encapsulates all of the semantic model's Power Query logic into a single file."]},{"cell_type":"code","execution_count":null,"id":"cde43b47-4ecc-46ae-9125-9674819c7eab","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["migration.create_pqt_file(dataset = dataset_name, workspace = workspace_name)"]},{"cell_type":"markdown","id":"bf945d07-544c-4934-b7a6-cfdb90ca725e","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Import the Power Query Template to Dataflows Gen2\n","\n","- Open the [OneLake file explorer](https://www.microsoft.com/download/details.aspx?id=105222) and sync your files (right click -> Sync from OneLake)\n","\n","- Navigate to your lakehouse. From this window, create a new Dataflows Gen2 and import the Power Query Template file from OneLake (OneLake -> Workspace -> Lakehouse -> Files...), and publish the Dataflows Gen2.\n","\n","<div class=\"alert alert-block alert-info\">\n","<b>Important!</b> Make sure to create the Dataflows Gen2 from within the lakehouse window. That will ensure that all the tables automatically map to that lakehouse as the destination. Otherwise, you will have to manually map each table to its destination individually.\n","</div>"]},{"cell_type":"markdown","id":"9975db7d","metadata":{},"source":["### Create the Direct Lake model based on the import/DQ semantic model\n","\n","Calculated columns are not migrated to the Direct Lake model as they are not supported in Direct Lake mode."]},{"cell_type":"code","execution_count":null,"id":"0a3616b5-566e-414e-a225-fb850d6418dc","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["import time\n","labs.create_blank_semantic_model(dataset = new_dataset_name, workspace = new_dataset_workspace_name, overwrite=False)\n","\n","migration.migrate_calc_tables_to_lakehouse(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name,\n","    lakehouse = lakehouse_name,\n","    lakehouse_workspace = lakehouse_workspace_name\n",")\n","migration.migrate_tables_columns_to_semantic_model(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name,\n","    lakehouse = lakehouse_name,\n","    lakehouse_workspace = lakehouse_workspace_name\n",")\n","migration.migrate_calc_tables_to_semantic_model(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name,\n","    lakehouse = lakehouse_name,\n","    lakehouse_workspace = lakehouse_workspace_name\n",")\n","migration.migrate_model_objects_to_semantic_model(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name\n",")\n","migration.migrate_field_parameters(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name\n",")\n","time.sleep(2)\n","labs.refresh_semantic_model(dataset = new_dataset_name, workspace = new_dataset_workspace_name)\n","migration.refresh_calc_tables(dataset = new_dataset_name, workspace = new_dataset_workspace_name)\n","labs.refresh_semantic_model(dataset = new_dataset_name, workspace = new_dataset_workspace_name)"]},{"cell_type":"markdown","id":"bb98bb13","metadata":{},"source":["### Show migrated/unmigrated objects"]},{"cell_type":"code","execution_count":null,"id":"5db2f22c","metadata":{},"outputs":[],"source":["migration.migration_validation(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name, \n","    workspace = workspace_name, \n","    new_dataset_workspace = new_dataset_workspace_name\n",")"]},{"cell_type":"markdown","id":"fa244e9d-87c2-4a66-a7e0-be539a0ac7de","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Rebind all reports using the old semantic model to the new Direct Lake semantic model"]},{"cell_type":"code","execution_count":null,"id":"d4e867cc","metadata":{},"outputs":[],"source":["rep.report_rebind_all(\n","    dataset = dataset_name,\n","    dataset_workspace = workspace_name,\n","    new_dataset = new_dataset_name,\n","    new_dataset_workpace = new_dataset_workspace_name,\n","    report_workspace = None\n",")"]},{"cell_type":"markdown","id":"3365d20d","metadata":{},"source":["### Rebind reports one-by-one (optional)"]},{"cell_type":"code","execution_count":null,"id":"056b7180-d7ac-492c-87e7-ac7d0e4bb929","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["report_name = '' # Enter report name which you want to rebind to the new Direct Lake model\n","\n","rep.report_rebind(\n","    report = report_name,\n","    dataset = new_dataset_name,\n","    report_workspace=workspace_name,\n","    dataset_workspace = new_dataset_workspace_name)"]},{"cell_type":"markdown","id":"526f2327","metadata":{},"source":["### Show unsupported objects"]},{"cell_type":"code","execution_count":null,"id":"a47376d7","metadata":{},"outputs":[],"source":["dfT, dfC, dfR = directlake.show_unsupported_direct_lake_objects(dataset = dataset_name, workspace = workspace_name)\n","\n","print('Calculated Tables are not supported...')\n","display(dfT)\n","print(\"Learn more about Direct Lake limitations here: https://learn.microsoft.com/power-bi/enterprise/directlake-overview#known-issues-and-limitations\")\n","print('Calculated columns are not supported. Columns of binary data type are not supported.')\n","display(dfC)\n","print('Columns used for relationship cannot be of data type datetime and they also must be of the same data type.')\n","display(dfR)"]},{"cell_type":"markdown","id":"ed08ba4c","metadata":{},"source":["### Schema check between semantic model tables/columns and lakehouse tables/columns\n","\n","This will list any tables/columns which are in the new semantic model but do not exist in the lakehouse"]},{"cell_type":"code","execution_count":null,"id":"03889ba4","metadata":{},"outputs":[],"source":["directlake.direct_lake_schema_compare(dataset = new_dataset_name, workspace = new_dataset_workspace_name)"]},{"cell_type":"markdown","id":"2229963b","metadata":{},"source":["### Show calculated tables which have been migrated to the Direct Lake semantic model as regular tables"]},{"cell_type":"code","execution_count":null,"id":"dd537d90","metadata":{},"outputs":[],"source":["directlake.list_direct_lake_model_calc_tables(dataset = new_dataset_name, workspace = new_dataset_workspace_name)"]}],"metadata":{"kernel_info":{"name":"synapse_pyspark"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.3"},"microsoft":{"language":"python"},"nteract":{"version":"nteract-front-end@1.0.0"},"spark_compute":{"compute_id":"/trident/default"},"synapse_widget":{"state":{},"version":"0.1"},"widgets":{}},"nbformat":4,"nbformat_minor":5}
+{"cells":[{"cell_type":"markdown","id":"5c27dfd1-4fe0-4a97-92e6-ddf78889aa93","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Install the latest .whl package\n","\n","Check [here](https://pypi.org/project/semantic-link-labs/) to see the latest version."]},{"cell_type":"code","execution_count":null,"id":"d5cae9db-cef9-48a8-a351-9c5fcc99645c","metadata":{"jupyter":{"outputs_hidden":true,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["%pip install semantic-link-labs"]},{"cell_type":"markdown","id":"969a29bf","metadata":{},"source":["### Import the library and set initial parameters"]},{"cell_type":"code","execution_count":null,"id":"29c923f8","metadata":{},"outputs":[],"source":["import sempy_labs as labs\n","from sempy_labs import migration, directlake\n","import sempy_labs.report as rep\n","\n","dataset_name = '' #Enter the import/DQ semantic model name\n","workspace_name = None #Enter the workspace of the import/DQ semantic model. It set to none it will use the current workspace.\n","new_dataset_name = '' #Enter the new Direct Lake semantic model name\n","new_dataset_workspace_name = None #Enter the workspace where the Direct Lake model will be created. If set to None it will use the current workspace.\n","lakehouse_name = None #Enter the lakehouse to be used for the Direct Lake model. If set to None it will use the lakehouse attached to the notebook.\n","lakehouse_workspace_name = None #Enter the lakehouse workspace. If set to None it will use the new_dataset_workspace_name."]},{"cell_type":"markdown","id":"5a3fe6e8-b8aa-4447-812b-7931831e07fe","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Create the [Power Query Template](https://learn.microsoft.com/power-query/power-query-template) file\n","\n","This encapsulates all of the semantic model's Power Query logic into a single file."]},{"cell_type":"code","execution_count":null,"id":"cde43b47-4ecc-46ae-9125-9674819c7eab","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["migration.create_pqt_file(dataset = dataset_name, workspace = workspace_name)"]},{"cell_type":"markdown","id":"bf945d07-544c-4934-b7a6-cfdb90ca725e","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Import the Power Query Template to Dataflows Gen2\n","\n","- Open the [OneLake file explorer](https://www.microsoft.com/download/details.aspx?id=105222) and sync your files (right click -> Sync from OneLake)\n","\n","- Navigate to your lakehouse. From this window, create a new Dataflows Gen2 and import the Power Query Template file from OneLake (OneLake -> Workspace -> Lakehouse -> Files...), and publish the Dataflows Gen2.\n","\n","<div class=\"alert alert-block alert-info\">\n","<b>Important!</b> Make sure to create the Dataflows Gen2 from within the lakehouse window. That will ensure that all the tables automatically map to that lakehouse as the destination. Otherwise, you will have to manually map each table to its destination individually.\n","</div>"]},{"cell_type":"markdown","id":"9975db7d","metadata":{},"source":["### Create the Direct Lake model based on the import/DQ semantic model\n","\n","Calculated columns are not migrated to the Direct Lake model as they are not supported in Direct Lake mode."]},{"cell_type":"code","execution_count":null,"id":"0a3616b5-566e-414e-a225-fb850d6418dc","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["import time\n","labs.create_blank_semantic_model(dataset = new_dataset_name, workspace = new_dataset_workspace_name, overwrite=False)\n","\n","migration.migrate_calc_tables_to_lakehouse(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name,\n","    lakehouse = lakehouse_name,\n","    lakehouse_workspace = lakehouse_workspace_name\n",")\n","migration.migrate_tables_columns_to_semantic_model(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name,\n","    lakehouse = lakehouse_name,\n","    lakehouse_workspace = lakehouse_workspace_name\n",")\n","migration.migrate_calc_tables_to_semantic_model(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name,\n","    lakehouse = lakehouse_name,\n","    lakehouse_workspace = lakehouse_workspace_name\n",")\n","migration.migrate_model_objects_to_semantic_model(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name\n",")\n","migration.migrate_field_parameters(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name,\n","    workspace = workspace_name,\n","    new_dataset_workspace = new_dataset_workspace_name\n",")\n","time.sleep(2)\n","labs.refresh_semantic_model(dataset = new_dataset_name, workspace = new_dataset_workspace_name)\n","migration.refresh_calc_tables(dataset = new_dataset_name, workspace = new_dataset_workspace_name)\n","labs.refresh_semantic_model(dataset = new_dataset_name, workspace = new_dataset_workspace_name)"]},{"cell_type":"markdown","id":"bb98bb13","metadata":{},"source":["### Show migrated/unmigrated objects"]},{"cell_type":"code","execution_count":null,"id":"5db2f22c","metadata":{},"outputs":[],"source":["migration.migration_validation(\n","    dataset = dataset_name,\n","    new_dataset = new_dataset_name, \n","    workspace = workspace_name, \n","    new_dataset_workspace = new_dataset_workspace_name\n",")"]},{"cell_type":"markdown","id":"fa244e9d-87c2-4a66-a7e0-be539a0ac7de","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Rebind all reports using the old semantic model to the new Direct Lake semantic model"]},{"cell_type":"code","execution_count":null,"id":"d4e867cc","metadata":{},"outputs":[],"source":["rep.report_rebind_all(\n","    dataset = dataset_name,\n","    dataset_workspace = workspace_name,\n","    new_dataset = new_dataset_name,\n","    new_dataset_workpace = new_dataset_workspace_name,\n","    report_workspace = None\n",")"]},{"cell_type":"markdown","id":"3365d20d","metadata":{},"source":["### Rebind reports one-by-one (optional)"]},{"cell_type":"code","execution_count":null,"id":"056b7180-d7ac-492c-87e7-ac7d0e4bb929","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["report_name = '' # Enter report name which you want to rebind to the new Direct Lake model\n","\n","rep.report_rebind(\n","    report = report_name,\n","    dataset = new_dataset_name,\n","    report_workspace=workspace_name,\n","    dataset_workspace = new_dataset_workspace_name)"]},{"cell_type":"markdown","id":"526f2327","metadata":{},"source":["### Show unsupported objects"]},{"cell_type":"code","execution_count":null,"id":"a47376d7","metadata":{},"outputs":[],"source":["dfT, dfC, dfR = directlake.show_unsupported_direct_lake_objects(dataset = dataset_name, workspace = workspace_name)\n","\n","print('Calculated Tables are not supported...')\n","display(dfT)\n","print(\"Learn more about Direct Lake limitations here: https://learn.microsoft.com/power-bi/enterprise/directlake-overview#known-issues-and-limitations\")\n","print('Calculated columns are not supported. Columns of binary data type are not supported.')\n","display(dfC)\n","print('Columns used for relationship must be of the same data type.')\n","display(dfR)"]},{"cell_type":"markdown","id":"ed08ba4c","metadata":{},"source":["### Schema check between semantic model tables/columns and lakehouse tables/columns\n","\n","This will list any tables/columns which are in the new semantic model but do not exist in the lakehouse"]},{"cell_type":"code","execution_count":null,"id":"03889ba4","metadata":{},"outputs":[],"source":["directlake.direct_lake_schema_compare(dataset = new_dataset_name, workspace = new_dataset_workspace_name)"]},{"cell_type":"markdown","id":"2229963b","metadata":{},"source":["### Show calculated tables which have been migrated to the Direct Lake semantic model as regular tables"]},{"cell_type":"code","execution_count":null,"id":"dd537d90","metadata":{},"outputs":[],"source":["directlake.list_direct_lake_model_calc_tables(dataset = new_dataset_name, workspace = new_dataset_workspace_name)"]}],"metadata":{"kernel_info":{"name":"synapse_pyspark"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.3"},"microsoft":{"language":"python"},"nteract":{"version":"nteract-front-end@1.0.0"},"spark_compute":{"compute_id":"/trident/default"},"synapse_widget":{"state":{},"version":"0.1"},"widgets":{}},"nbformat":4,"nbformat_minor":5}

semantic_link_labs-0.8.4/notebooks/Model Optimization.ipynb

@@ -0,0 +1 @@
+{"cells":[{"cell_type":"markdown","id":"5c27dfd1-4fe0-4a97-92e6-ddf78889aa93","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Install the latest .whl package\n","\n","Check [here](https://pypi.org/project/semantic-link-labs/) to see the latest version."]},{"cell_type":"code","execution_count":null,"id":"d5cae9db-cef9-48a8-a351-9c5fcc99645c","metadata":{"jupyter":{"outputs_hidden":true,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["%pip install semantic-link-labs"]},{"cell_type":"markdown","id":"cd8de5a0","metadata":{},"source":["### Import the library"]},{"cell_type":"code","execution_count":null,"id":"5cc6eedf","metadata":{},"outputs":[],"source":["import sempy_labs as labs\n","from sempy_labs import lakehouse as lake\n","from sempy_labs import directlake\n","import sempy_labs.report as rep\n","\n","dataset_name = ''\n","workspace_name = None"]},{"cell_type":"markdown","id":"5a3fe6e8-b8aa-4447-812b-7931831e07fe","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Vertipaq Analyzer"]},{"cell_type":"code","execution_count":null,"id":"cde43b47-4ecc-46ae-9125-9674819c7eab","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["labs.vertipaq_analyzer(dataset=dataset_name, workspace=workspace_name)"]},{"cell_type":"markdown","id":"419a348f","metadata":{},"source":["Export the Vertipaq Analyzer results to a .zip file in your lakehouse"]},{"cell_type":"code","execution_count":null,"id":"8aa239b3","metadata":{},"outputs":[],"source":["labs.vertipaq_analyzer(dataset=dataset_name, workspace=workspace_name, export='zip')"]},{"cell_type":"markdown","id":"2dce0f4f","metadata":{},"source":["Export the Vertipaq Analyzer results to append to delta tables in your lakehouse."]},{"cell_type":"code","execution_count":null,"id":"aef93fc8","metadata":{},"outputs":[],"source":["labs.vertipaq_analyzer(dataset=dataset_name, workspace=workspace_name, export='table')"]},{"cell_type":"markdown","id":"1c62a802","metadata":{},"source":["Visualize the contents of an exported Vertipaq Analyzer .zip file."]},{"cell_type":"code","execution_count":null,"id":"9e349954","metadata":{},"outputs":[],"source":["labs.import_vertipaq_analyzer(folder_path='', file_name='')"]},{"cell_type":"markdown","id":"456ce0ff","metadata":{},"source":["### Best Practice Analzyer\n","\n","This runs the [standard rules](https://github.com/microsoft/Analysis-Services/tree/master/BestPracticeRules) for semantic models posted on Microsoft's GitHub."]},{"cell_type":"code","execution_count":null,"id":"0a3616b5-566e-414e-a225-fb850d6418dc","metadata":{"jupyter":{"outputs_hidden":false,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["labs.run_model_bpa(dataset=dataset_name, workspace=workspace_name)"]},{"cell_type":"markdown","id":"6fb32a58","metadata":{},"source":["This runs the Best Practice Analyzer and exports the results to the 'modelbparesults' delta table in your Fabric lakehouse."]},{"cell_type":"code","execution_count":null,"id":"677851c3","metadata":{},"outputs":[],"source":["labs.run_model_bpa(dataset=dataset_name, workspace=workspace_name, export=True)"]},{"cell_type":"markdown","id":"64968a31","metadata":{},"source":["This runs the Best Practice Analyzer with the rules translated into Italian."]},{"cell_type":"code","execution_count":null,"id":"3c7d89e2","metadata":{},"outputs":[],"source":["labs.run_model_bpa(dataset=dataset_name, workspace=workspace_name, language ='italian')"]},{"cell_type":"markdown","id":"255c30bb","metadata":{},"source":["<div class=\"alert alert-block alert-info\">\n","<b>Note:</b> For analyzing model BPA results at scale, see the Best Practice Analyzer Report notebook (link below).\n","</div>\n","\n","[Best Practice Analyzer Notebook](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Best%20Practice%20Analyzer%20Report.ipynb)"]},{"cell_type":"markdown","id":"bab18a61","metadata":{},"source":["### Run BPA using your own best practice rules"]},{"cell_type":"code","execution_count":null,"id":"59b89387","metadata":{},"outputs":[],"source":["import sempy\n","sempy.fabric._client._utils._init_analysis_services()\n","import Microsoft.AnalysisServices.Tabular as TOM\n","import pandas as pd\n","\n","dataset_name = ''\n","workspace_name = ''\n","\n","rules = pd.DataFrame(\n","    [\n","        (\n","            \"Performance\",\n","            \"Table\",\n","            \"Warning\",\n","            \"Rule name...\",\n","            lambda obj, tom: tom.is_calculated_table(table_name=obj.Name),\n","            'Rule description...',\n","            '',\n","        ),\n","        (\n","            \"Performance\",\n","            \"Column\",\n","            \"Warning\",\n","            \"Do not use floating point data types\",\n","            lambda obj, tom: obj.DataType == TOM.DataType.Double,\n","            'The \"Double\" floating point data type should be avoided, as it can result in unpredictable roundoff errors and decreased performance in certain scenarios. Use \"Int64\" or \"Decimal\" where appropriate (but note that \"Decimal\" is limited to 4 digits after the decimal sign).',\n","        )\n","    ],\n","    columns=[\n","            \"Category\",\n","            \"Scope\",\n","            \"Severity\",\n","            \"Rule Name\",\n","            \"Expression\",\n","            \"Description\",\n","            \"URL\",\n","        ],\n",")\n","\n","labs.run_model_bpa(dataset=dataset_name, workspace=workspace_name, rules=rules)"]},{"cell_type":"markdown","id":"8126a1a1","metadata":{},"source":["### Direct Lake\n","\n","Check if any lakehouse tables will hit the [Direct Lake guardrails](https://learn.microsoft.com/power-bi/enterprise/directlake-overview#fallback)."]},{"cell_type":"code","execution_count":null,"id":"e7397b15","metadata":{},"outputs":[],"source":["lake.get_lakehouse_tables(lakehouse=None, workspace=None, extended=True, count_rows=False)"]},{"cell_type":"code","execution_count":null,"id":"b30074cf","metadata":{},"outputs":[],"source":["lake.get_lakehouse_tables(lakehouse=None, workspace=None, extended=True, count_rows=False, export=True)"]},{"cell_type":"markdown","id":"99b84f2b","metadata":{},"source":["Check if any tables in a Direct Lake semantic model will fall back to DirectQuery."]},{"cell_type":"code","execution_count":null,"id":"f837be58","metadata":{},"outputs":[],"source":["directlake.check_fallback_reason(dataset=dataset_name, workspace=workspace_name)"]},{"cell_type":"markdown","id":"8f6df93e","metadata":{},"source":["### [OPTIMIZE](https://docs.delta.io/latest/optimizations-oss.html) your lakehouse delta tables."]},{"cell_type":"code","execution_count":null,"id":"e0262c9e","metadata":{},"outputs":[],"source":["lake.optimize_lakehouse_tables(tables=['', ''], lakehouse=None, workspace=workspace_name)"]},{"cell_type":"markdown","id":"0091d6a0","metadata":{},"source":["Refresh/reframe your Direct Lake semantic model and restore the columns which were in memory prior to the refresh."]},{"cell_type":"code","execution_count":null,"id":"77eef082","metadata":{},"outputs":[],"source":["directlake.warm_direct_lake_cache_isresident(dataset=dataset_name, workspace=workspace_name)"]},{"cell_type":"markdown","id":"dae1a210","metadata":{},"source":["Ensure a warm cache for your users by putting the columns of a Direct Lake semantic model into memory based on the contents of a [perspective](https://learn.microsoft.com/analysis-services/tabular-models/perspectives-ssas-tabular?view=asallproducts-allversions).\n","\n","Perspectives can be created either in [Tabular Editor 3](https://github.com/TabularEditor/TabularEditor3/releases/latest) or in [Tabular Editor 2](https://github.com/TabularEditor/TabularEditor/releases/latest) using the [Perspective Editor](https://www.elegantbi.com/post/perspectiveeditor)."]},{"cell_type":"code","execution_count":null,"id":"43297001","metadata":{},"outputs":[],"source":["directlake.warm_direct_lake_cache_perspective(\n","    dataset=dataset_name,\n","    workspace=workspace_name,\n","    perspective='',\n","    add_dependencies=True\n",")"]}],"metadata":{"kernel_info":{"name":"synapse_pyspark"},"kernelspec":{"display_name":"Synapse PySpark","language":"Python","name":"synapse_pyspark"},"language_info":{"name":"python"},"microsoft":{"language":"python"},"nteract":{"version":"nteract-front-end@1.0.0"},"spark_compute":{"compute_id":"/trident/default"},"synapse_widget":{"state":{},"version":"0.1"},"widgets":{}},"nbformat":4,"nbformat_minor":5}

semantic_link_labs-0.8.4/notebooks/SQL.ipynb

@@ -0,0 +1 @@
+{"cells":[{"cell_type":"markdown","id":"5c27dfd1-4fe0-4a97-92e6-ddf78889aa93","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Install the latest .whl package\n","\n","Check [here](https://pypi.org/project/semantic-link-labs/) to see the latest version."]},{"cell_type":"code","execution_count":null,"id":"d5cae9db-cef9-48a8-a351-9c5fcc99645c","metadata":{"jupyter":{"outputs_hidden":true,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["%pip install semantic-link-labs"]},{"cell_type":"markdown","id":"b195eae8","metadata":{},"source":["### Import the library and necessary packages"]},{"cell_type":"code","execution_count":null,"id":"1344e286","metadata":{},"outputs":[],"source":["import sempy_labs as labs\n","from sempy_labs import ConnectWarehouse\n","from sempy_labs import ConnectLakehouse\n","\n","lakehouse_name = ''\n","lakehouse_workspace_name = ''\n","warehouse_name = ''\n","warehouse_workspace_name = ''"]},{"cell_type":"markdown","id":"55e5ca67","metadata":{},"source":["### Run a SQL query (or queries) against a Fabric warehouse"]},{"cell_type":"code","execution_count":null,"id":"a9f984e9","metadata":{},"outputs":[],"source":["with ConnectWarehouse(warehouse=warehouse_name, workspace=warehouse_workspace_name) as sql:\n","    df = sql.query(\"SELECT * FROM Product\")\n","    display(df)"]},{"cell_type":"code","execution_count":null,"id":"865ac4a1","metadata":{},"outputs":[],"source":["with ConnectWarehouse(warehouse=warehouse_name, workspace=warehouse_workspace_name) as sql:\n","    dfs = sql.query([\"SELECT * FROM Product\", \"SELECT DISTINCT [Category] FROM Product\"])\n","\n","for df in dfs:\n","    display(df)"]},{"cell_type":"markdown","id":"bca53cd8","metadata":{},"source":["#### See the tables in a warehouse"]},{"cell_type":"code","execution_count":null,"id":"9af2cce7","metadata":{},"outputs":[],"source":["labs.get_warehouse_tables(warehouse=warehouse_name, workspace=warehouse_workspace_name)"]},{"cell_type":"markdown","id":"765f99ae","metadata":{},"source":["#### See the columns in each table in a warehouse"]},{"cell_type":"code","execution_count":null,"id":"1fabe168","metadata":{},"outputs":[],"source":["labs.get_warehouse_columns(warehouse=warehouse_name, workspace=warehouse_workspace_name)"]},{"cell_type":"markdown","id":"634700c3","metadata":{},"source":["### Run a T-SQL query (or queries) against a Fabric warehouse"]},{"cell_type":"code","execution_count":null,"id":"5dbf34f3","metadata":{},"outputs":[],"source":["with ConnectWarehouse(warehouse=warehouse_name, workspace=warehouse_workspace_name) as sql:\n","    sql.query(\"CREATE SCHEMA [Business]\")"]},{"cell_type":"code","execution_count":null,"id":"ec8ddb81","metadata":{},"outputs":[],"source":["with ConnectWarehouse(warehouse=warehouse_name, workspace=warehouse_workspace_name) as sql:\n","    sql.query([\"CREATE SCHEMA [Business]\", \"CREATE SCHEMA [Marketing]\"])"]},{"cell_type":"markdown","id":"d5b090da","metadata":{},"source":["### Run a SQL query (or queries) against a Fabric lakehouse"]},{"cell_type":"code","execution_count":null,"id":"4dca7f4a","metadata":{},"outputs":[],"source":["with ConnectLakehouse(lakehouse=lakehouse_name, workspace=lakehouse_workspace_name) as sql:\n","    df = sql.query(\"SELECT * FROM Product\")\n","    display(df)"]},{"cell_type":"code","execution_count":null,"id":"b9606ae8","metadata":{},"outputs":[],"source":["with ConnectLakehouse(lakehouse=lakehouse_name, workspace=lakehouse_workspace_name) as sql:\n","    dfs = sql.query([\"SELECT * FROM Product\", \"SELECT DISTINCT [Category] FROM Product\"])\n","\n","for df in dfs:\n","    display(df)"]}],"metadata":{"kernel_info":{"name":"synapse_pyspark"},"kernelspec":{"display_name":"Synapse PySpark","language":"Python","name":"synapse_pyspark"},"language_info":{"name":"python"},"microsoft":{"language":"python"},"nteract":{"version":"nteract-front-end@1.0.0"},"spark_compute":{"compute_id":"/trident/default"},"synapse_widget":{"state":{},"version":"0.1"},"widgets":{}},"nbformat":4,"nbformat_minor":5}

semantic_link_labs-0.8.4/notebooks/Semantic Model Refresh.ipynb

@@ -0,0 +1 @@
+{"cells":[{"cell_type":"markdown","id":"5c27dfd1-4fe0-4a97-92e6-ddf78889aa93","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Install the latest .whl package\n","\n","Check [here](https://pypi.org/project/semantic-link-labs/) to see the latest version."]},{"cell_type":"code","execution_count":null,"id":"d5cae9db-cef9-48a8-a351-9c5fcc99645c","metadata":{"jupyter":{"outputs_hidden":true,"source_hidden":false},"nteract":{"transient":{"deleting":false}}},"outputs":[],"source":["%pip install semantic-link-labs"]},{"cell_type":"markdown","id":"b195eae8","metadata":{},"source":["### Import the library and set the initial parameters"]},{"cell_type":"code","execution_count":null,"id":"1344e286","metadata":{},"outputs":[],"source":["import sempy.fabric as fabric\n","import sempy_labs as labs\n","dataset = '' # Enter your dataset name\n","workspace = None # Enter your workspace name (if set to None it will use the workspace in which the notebook is running)"]},{"cell_type":"markdown","id":"5a3fe6e8-b8aa-4447-812b-7931831e07fe","metadata":{"nteract":{"transient":{"deleting":false}}},"source":["### Refresh a semantic model"]},{"cell_type":"code","execution_count":null,"id":"9e349954","metadata":{},"outputs":[],"source":["labs.refresh_semantic_model(dataset=dataset, workspace=workspace)"]},{"cell_type":"markdown","id":"65db8187","metadata":{},"source":["#### Visualize the refresh of a semantic model\n","The resulting dataframe returns the trace logs capturing the details of the refresh operation"]},{"cell_type":"code","execution_count":null,"id":"24de391e","metadata":{},"outputs":[],"source":["df = labs.refresh_semantic_model(dataset=dataset, workspace=workspace, visualize=True)"]},{"cell_type":"markdown","id":"113b04a7","metadata":{},"source":["#### Refresh specific tables"]},{"cell_type":"code","execution_count":null,"id":"b4e1296b","metadata":{},"outputs":[],"source":["labs.refresh_semantic_model(dataset=dataset, workspace=workspace, tables=['Sales', 'Geography'])"]},{"cell_type":"markdown","id":"7f94b13a","metadata":{},"source":["#### Refresh specific partitions"]},{"cell_type":"code","execution_count":null,"id":"17565d35","metadata":{},"outputs":[],"source":["labs.refresh_semantic_model(dataset=dataset, workspace=workspace, partitions=[\"'Sales'[Sales FY20]\", \"'Sales'[Sales FY21]\"])"]},{"cell_type":"markdown","id":"aab5ca7c","metadata":{},"source":["#### Refresh a combination of tables and partitions"]},{"cell_type":"code","execution_count":null,"id":"e5818bd1","metadata":{},"outputs":[],"source":["labs.refresh_semantic_model(dataset=dataset, workspace=workspace, tables=['Geography', 'Calendar'], partitions=[\"'Sales'[Sales FY20]\", \"'Sales'[Sales FY21]\"])"]},{"cell_type":"markdown","id":"7f7074ea","metadata":{},"source":["#### Clear the values of a table"]},{"cell_type":"code","execution_count":null,"id":"3b1eb772","metadata":{},"outputs":[],"source":["labs.refresh_semantic_model(dataset=dataset, workspace=workspace, refresh_type='clearValues')"]},{"cell_type":"markdown","id":"29afede1","metadata":{},"source":["### View semantic model refreshes"]},{"cell_type":"code","execution_count":null,"id":"95c52cc0","metadata":{},"outputs":[],"source":["fabric.list_refresh_requests(dataset=dataset, workspace=workspace)"]},{"cell_type":"markdown","id":"fa7c525c","metadata":{},"source":["### Cancel a semantic model refresh"]},{"cell_type":"code","execution_count":null,"id":"5bb6f79f","metadata":{},"outputs":[],"source":["labs.cancel_dataset_refresh(dataset=dataset, workspace=workspace)"]},{"cell_type":"code","execution_count":null,"id":"acd34900","metadata":{},"outputs":[],"source":["# Specify the request_id based on a value from list_refresh_requests\n","labs.cancel_dataset_refresh(dataset=dataset, workspace=workspace, request_id='')"]},{"cell_type":"markdown","id":"26300103","metadata":{},"source":["### View a semantic model's refresh history"]},{"cell_type":"code","execution_count":null,"id":"60cec3f8","metadata":{},"outputs":[],"source":["labs.get_semantic_model_refresh_history(dataset=dataset, workspace=workspace)"]},{"cell_type":"markdown","id":"538d5f1e","metadata":{},"source":["### View details of a specific semantic model refresh"]},{"cell_type":"code","execution_count":null,"id":"c043f9bc","metadata":{},"outputs":[],"source":["labs.get_semantic_model_refresh_history(dataset=dataset, workspace=workspace, request_id='')"]}],"metadata":{"kernel_info":{"name":"synapse_pyspark"},"kernelspec":{"display_name":"Synapse PySpark","language":"Python","name":"synapse_pyspark"},"language_info":{"name":"python"},"microsoft":{"language":"python"},"nteract":{"version":"nteract-front-end@1.0.0"},"spark_compute":{"compute_id":"/trident/default"},"synapse_widget":{"state":{},"version":"0.1"},"widgets":{}},"nbformat":4,"nbformat_minor":5}

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/pyproject.toml

@@ -7,7 +7,7 @@ name="semantic-link-labs"
 authors = [
     { name = "Microsoft Corporation" },
 ]
-version="0.8.3"
+version="0.8.4"
 description="Semantic Link Labs for Microsoft Fabric"
 readme="README.md"
 requires-python=">=3.10,<3.12"
@@ -23,7 +23,7 @@ classifiers = [
 license= { text = "MIT License" }
 
 dependencies = [
-    "semantic-link-sempy>=0.8.0",
+    "semantic-link-sempy>=0.8.1",
     "anytree",
     "powerbiclient",
     "polib",

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/src/semantic_link_labs.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: semantic-link-labs
-Version: 0.8.3
+Version: 0.8.4
 Summary: Semantic Link Labs for Microsoft Fabric
 Author: Microsoft Corporation
 License: MIT License
@@ -15,7 +15,7 @@ Classifier: Framework :: Jupyter
 Requires-Python: <3.12,>=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: semantic-link-sempy>=0.8.0
+Requires-Dist: semantic-link-sempy>=0.8.1
 Requires-Dist: anytree
 Requires-Dist: powerbiclient
 Requires-Dist: polib
@@ -27,7 +27,7 @@ Requires-Dist: pytest>=8.2.1; extra == "test"
 # Semantic Link Labs
 
 [![PyPI version](https://badge.fury.io/py/semantic-link-labs.svg)](https://badge.fury.io/py/semantic-link-labs)
-[![Read The Docs](https://readthedocs.org/projects/semantic-link-labs/badge/?version=0.8.3&style=flat)](https://readthedocs.org/projects/semantic-link-labs/)
+[![Read The Docs](https://readthedocs.org/projects/semantic-link-labs/badge/?version=0.8.4&style=flat)](https://readthedocs.org/projects/semantic-link-labs/)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 [![Downloads](https://static.pepy.tech/badge/semantic-link-labs)](https://pepy.tech/project/semantic-link-labs)
 
@@ -37,6 +37,10 @@ Requires-Dist: pytest>=8.2.1; extra == "test"
 
 Semantic Link Labs is a Python library designed for use in [Microsoft Fabric notebooks](https://learn.microsoft.com/fabric/data-engineering/how-to-use-notebook). This library extends the capabilities of [Semantic Link](https://learn.microsoft.com/fabric/data-science/semantic-link-overview) offering additional functionalities to seamlessly integrate and work alongside it. The goal of Semantic Link Labs is to simplify technical processes, empowering people to focus on higher level activities and allowing tasks that are better suited for machines to be efficiently handled without human intervention.
 
+If you encounter any issues, please [raise a bug](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=).
+
+If you have ideas for new features/functions, please [request a feature](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=feature_request.md&title=).
+
 ## Featured Scenarios
 * Semantic Models
     * [Migrating an import/DirectQuery semantic model to Direct Lake](https://github.com/microsoft/semantic-link-labs?tab=readme-ov-file#direct-lake-migration)
@@ -47,26 +51,46 @@ Semantic Link Labs is a Python library designed for use in [Microsoft Fabric not
     * [Check Direct Lake Guardrails](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.get_lakehouse_tables)
     * [Refresh](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Semantic%20Model%20Refresh.ipynb), [clear cache](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.clear_cache), [backup](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.backup_semantic_model), [restore](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.restore_semantic_model), [copy backup files](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.copy_semantic_model_backup_file), [move/deploy across workspaces](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.deploy_semantic_model)
     * [Run DAX queries which impersonate a user](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.evaluate_dax_impersonation)
-    * [Manage Query Scale Out](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Query%20Scale%20Out.ipynb).
+    * [Manage Query Scale Out](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Query%20Scale%20Out.ipynb)
     * [Auto-generate descriptions for any/all measures in bulk](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.tom.html#sempy_labs.tom.TOMWrapper.generate_measure_descriptions)
+    * [Warm the cache of a Direct Lake semantic model after a refresh (using columns currently in memory)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.directlake.html#sempy_labs.directlake.warm_direct_lake_cache_isresident)
+    * [Warm the cache of a Direct Lake semantic model (via perspective)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.directlake.html#sempy_labs.directlake.warm_direct_lake_cache_perspective)
+    * [Visualize a refresh](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Semantic%20Model%20Refresh.ipynb)
 * Reports
     * [Report Best Practice Analyzer (BPA)](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.run_report_bpa)
     * [View report metadata](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Report%20Analysis.ipynb)
     * [View semantic model objects most frequently used in Power BI reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.list_semantic_model_object_report_usage)
     * [View broken reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.list_report_semantic_model_objects)
+    * [Set a report theme](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.ReportWrapper.set_theme)
+    * [Migrate report-level measures to the semantic model](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.ReportWrapper.migrate_report_level_measures)
     * [Rebind reports](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.report.html#sempy_labs.report.report_rebind)
 * Capacities
     * [Migrating a Power BI Premium capacity (P sku) to a Fabric capacity (F sku)](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Capacity%20Migration.ipynb)
+    * [Migrating a Fabric Trial capacity (FT sku) to a Fabric capacity (F sku)](https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Capacity%20Migration.ipynb)
     * [Create](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.create_fabric_capacity)/[update](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.update_fabric_capacity)/[suspend](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.suspend_fabric_capacity)/[resume](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.html#sempy_labs.resume_fabric_capacity) Fabric capacities.
+* Lakehouses
+    * [Optimize lakehouse tables](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.optimize_lakehouse_tables)
+    * [Vacuum lakehouse tables](https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.lakehouse.html#sempy_labs.lakehouse.vacuum_lakehouse_tables)
 * APIs
     * Wrapper functions for [Power BI](https://learn.microsoft.com/rest/api/power-bi/), [Fabric](https://learn.microsoft.com/rest/api/fabric/articles/using-fabric-apis), and [Azure (Fabric Capacity)](https://learn.microsoft.com/rest/api/microsoftfabric/fabric-capacities?view=rest-microsoftfabric-2023-11-01) APIs
 
+## Helper Notebooks
 
-### Check out the [helper notebooks](https://github.com/microsoft/semantic-link-labs/tree/main/notebooks) for getting started!
+Check out the [helper notebooks](https://github.com/microsoft/semantic-link-labs/tree/main/notebooks) for getting started! 
+Run the code below to load all the helper notebooks to the workspace of your choice at once.
+```python
+import sempy_labs as labs
+import requests
 
-If you encounter any issues, please [raise a bug](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=).
+workspace_name = None # Update this to the workspace in which you want to save the notebooks
+api_url = "https://api.github.com/repos/microsoft/semantic-link-labs/contents/notebooks"
+response = requests.get(api_url)
+files = response.json()
+notebook_files = {file['name'][:-6]: file['html_url'] for file in files if file['name'].endswith('.ipynb')}
 
-If you have ideas for new features/functions, please [request a feature](https://github.com/microsoft/semantic-link-labs/issues/new?assignees=&labels=&projects=&template=feature_request.md&title=).
+for file_name, file_url in notebook_files.items():
+    labs.import_notebook_from_web(notebook_name=file_name, url=file_url, workspace=workspace_name)
+```
 
 ## Install the library in a Fabric notebook
 ```python
@@ -85,7 +109,7 @@ from sempy_labs import ConnectWarehouse
 from sempy_labs import ConnectLakehouse
 ```
 
-## Load semantic-link-labs into a custom [Fabric environment](https://learn.microsoft.com/fabric/data-engineering/create-and-use-environment)
+## Load Semantic Link Labs into a custom [Fabric environment](https://learn.microsoft.com/fabric/data-engineering/create-and-use-environment)
 An even better way to ensure the semantic-link-labs library is available in your workspace/notebooks is to load it as a library in a custom Fabric environment. If you do this, you will not have to run the above '%pip install' code every time in your notebook. Please follow the steps below.
 
 #### Create a custom environment
@@ -106,6 +130,7 @@ An even better way to ensure the semantic-link-labs library is available in your
 2. Select your newly created environment within the 'Environment' drop down in the navigation bar at the top of the notebook
 
 ## Version History
+* [0.8.4](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.4) (October 30, 2024)
 * [0.8.3](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.3) (October 14, 2024)
 * [0.8.2](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.2) (October 2, 2024)
 * [0.8.1](https://github.com/microsoft/semantic-link-labs/releases/tag/0.8.1) (October 2, 2024)

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/src/semantic_link_labs.egg-info/SOURCES.txt

@@ -55,6 +55,7 @@ src/sempy_labs/_icons.py
 src/sempy_labs/_kql_databases.py
 src/sempy_labs/_kql_querysets.py
 src/sempy_labs/_list_functions.py
+src/sempy_labs/_managed_private_endpoints.py
 src/sempy_labs/_mirrored_warehouses.py
 src/sempy_labs/_ml_experiments.py
 src/sempy_labs/_ml_models.py
@@ -72,6 +73,7 @@ src/sempy_labs/_sql.py
 src/sempy_labs/_translations.py
 src/sempy_labs/_vertipaq.py
 src/sempy_labs/_warehouses.py
+src/sempy_labs/_workloads.py
 src/sempy_labs/_workspace_identity.py
 src/sempy_labs/_workspaces.py
 src/sempy_labs/_bpa_translation/_model/_translations_am-ET.po
@@ -144,6 +146,7 @@ src/sempy_labs/migration/_refresh_calc_tables.py
 src/sempy_labs/report/_BPAReportTemplate.json
 src/sempy_labs/report/__init__.py
 src/sempy_labs/report/_generate_report.py
+src/sempy_labs/report/_paginated.py
 src/sempy_labs/report/_report_bpa.py
 src/sempy_labs/report/_report_bpa_rules.py
 src/sempy_labs/report/_report_functions.py

{semantic_link_labs-0.8.3 → semantic_link_labs-0.8.4}/src/semantic_link_labs.egg-info/requires.txt

@@ -1,4 +1,4 @@
-semantic-link-sempy>=0.8.0
+semantic-link-sempy>=0.8.1
 anytree
 powerbiclient
 polib