msdev-kit 0.1.2__tar.gz → 0.1.4__tar.gz

{msdev_kit-0.1.2 → msdev_kit-0.1.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: msdev-kit
-Version: 0.1.2
+Version: 0.1.4
 Summary: Microsoft developer toolkit: Fabric, MS Graph, and SharePoint
 License: MIT
 Author: Bernardo Rufino
@@ -36,12 +36,6 @@ Microsoft developer toolkit for Python: Fabric/Power BI, MS Graph (Entra), and S
 [![Python](https://img.shields.io/pypi/pyversions/msdev-kit)](https://pypi.org/project/msdev-kit/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
 
-## Table of Contents
-
-[Installation](https://github.com/Bernardo-Rufino/msdev-kit#installation) · [Quick Start](https://github.com/Bernardo-Rufino/msdev-kit#quick-start) · [Authentication](https://github.com/Bernardo-Rufino/msdev-kit#authentication) · [Fabric & Power BI](https://github.com/Bernardo-Rufino/msdev-kit#fabric--power-bi) · [MS Graph (Entra)](https://github.com/Bernardo-Rufino/msdev-kit#ms-graph-entra) · [SharePoint](https://github.com/Bernardo-Rufino/msdev-kit#sharepoint) · [Limitations](https://github.com/Bernardo-Rufino/msdev-kit#limitations)
-
----
-
 ## Installation
 
 ```shell
@@ -145,12 +139,12 @@ auth = Auth(tenant_id, client_id, client_secret)
 
 Fabric classes take a token string — call `auth.get_token('fabric')` or `auth.get_token('pbi')` depending on the API.
 
-- [Workspace](https://github.com/Bernardo-Rufino/msdev-kit#workspace) — workspaces, users, permissions
-- [Dataset](https://github.com/Bernardo-Rufino/msdev-kit#dataset) — semantic models, DAX queries, permissions
-- [Report](https://github.com/Bernardo-Rufino/msdev-kit#report) — metadata, definitions, visuals, measures
-- [Dataflow](https://github.com/Bernardo-Rufino/msdev-kit#dataflow) — Gen1, Gen2, Gen2 CI/CD management
-- [Pipeline](https://github.com/Bernardo-Rufino/msdev-kit#pipeline) — Data Pipeline management
-- [Other modules](https://github.com/Bernardo-Rufino/msdev-kit#other-modules) — Capacity, Admin, KQL, Notebook, Database
+- [Workspace](#workspace) — workspaces, users, permissions
+- [Dataset](#dataset) — semantic models, DAX queries, permissions
+- [Report](#report) — metadata, definitions, visuals, measures
+- [Dataflow](#dataflow) — Gen1, Gen2, Gen2 CI/CD management
+- [Pipeline](#pipeline) — Data Pipeline management
+- [Other modules](#other-modules) — Capacity, Admin, KQL, Notebook, Database
 
 ### Workspace
 
@@ -210,7 +204,7 @@ pages = rpt.list_report_pages(workspace_id, report_id)
 | `get_report_metadata(workspace_id, report_id)` | Get metadata for a specific report. |
 | `get_report_name(workspace_id, report_id)` | Get a report's display name. |
 | `list_report_pages(workspace_id, report_id)` | List all pages in a report. |
-| `get_report_json_pages_and_visuals(json_data, workspace_id, report_id)` | Parse a PBIR-Legacy report JSON and extract pages/visuals into a DataFrame. |
+| `get_legacy_report_pages_and_visuals(json_data, workspace_id, report_id)` | Parse a PBIR-Legacy report JSON and extract pages/visuals into a DataFrame. |
 | `get_legacy_report_json(workspace_id, report_id, operations)` | Get and decode the full report definition for PBIR-Legacy reports. |
 | `export_report(workspace_id, report_id, ...)` | Export a report as a `.pbix` file. |
 | `get_report_measures(workspace_id, report_id, operations)` | Extract report-level measures and generate a DAX Query View script. |

{msdev_kit-0.1.2 → msdev_kit-0.1.4}/README.md

@@ -6,12 +6,6 @@ Microsoft developer toolkit for Python: Fabric/Power BI, MS Graph (Entra), and S
 [![Python](https://img.shields.io/pypi/pyversions/msdev-kit)](https://pypi.org/project/msdev-kit/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
 
-## Table of Contents
-
-[Installation](https://github.com/Bernardo-Rufino/msdev-kit#installation) · [Quick Start](https://github.com/Bernardo-Rufino/msdev-kit#quick-start) · [Authentication](https://github.com/Bernardo-Rufino/msdev-kit#authentication) · [Fabric & Power BI](https://github.com/Bernardo-Rufino/msdev-kit#fabric--power-bi) · [MS Graph (Entra)](https://github.com/Bernardo-Rufino/msdev-kit#ms-graph-entra) · [SharePoint](https://github.com/Bernardo-Rufino/msdev-kit#sharepoint) · [Limitations](https://github.com/Bernardo-Rufino/msdev-kit#limitations)
-
----
-
 ## Installation
 
 ```shell
@@ -115,12 +109,12 @@ auth = Auth(tenant_id, client_id, client_secret)
 
 Fabric classes take a token string — call `auth.get_token('fabric')` or `auth.get_token('pbi')` depending on the API.
 
-- [Workspace](https://github.com/Bernardo-Rufino/msdev-kit#workspace) — workspaces, users, permissions
-- [Dataset](https://github.com/Bernardo-Rufino/msdev-kit#dataset) — semantic models, DAX queries, permissions
-- [Report](https://github.com/Bernardo-Rufino/msdev-kit#report) — metadata, definitions, visuals, measures
-- [Dataflow](https://github.com/Bernardo-Rufino/msdev-kit#dataflow) — Gen1, Gen2, Gen2 CI/CD management
-- [Pipeline](https://github.com/Bernardo-Rufino/msdev-kit#pipeline) — Data Pipeline management
-- [Other modules](https://github.com/Bernardo-Rufino/msdev-kit#other-modules) — Capacity, Admin, KQL, Notebook, Database
+- [Workspace](#workspace) — workspaces, users, permissions
+- [Dataset](#dataset) — semantic models, DAX queries, permissions
+- [Report](#report) — metadata, definitions, visuals, measures
+- [Dataflow](#dataflow) — Gen1, Gen2, Gen2 CI/CD management
+- [Pipeline](#pipeline) — Data Pipeline management
+- [Other modules](#other-modules) — Capacity, Admin, KQL, Notebook, Database
 
 ### Workspace
 
@@ -180,7 +174,7 @@ pages = rpt.list_report_pages(workspace_id, report_id)
 | `get_report_metadata(workspace_id, report_id)` | Get metadata for a specific report. |
 | `get_report_name(workspace_id, report_id)` | Get a report's display name. |
 | `list_report_pages(workspace_id, report_id)` | List all pages in a report. |
-| `get_report_json_pages_and_visuals(json_data, workspace_id, report_id)` | Parse a PBIR-Legacy report JSON and extract pages/visuals into a DataFrame. |
+| `get_legacy_report_pages_and_visuals(json_data, workspace_id, report_id)` | Parse a PBIR-Legacy report JSON and extract pages/visuals into a DataFrame. |
 | `get_legacy_report_json(workspace_id, report_id, operations)` | Get and decode the full report definition for PBIR-Legacy reports. |
 | `export_report(workspace_id, report_id, ...)` | Export a report as a `.pbix` file. |
 | `get_report_measures(workspace_id, report_id, operations)` | Extract report-level measures and generate a DAX Query View script. |

{msdev_kit-0.1.2 → msdev_kit-0.1.4}/msdev_kit/__init__.py

@@ -1,3 +1,3 @@
 from .auth import Auth
 
-__version__ = "0.1.2"
+__version__ = "0.1.4"

{msdev_kit-0.1.2 → msdev_kit-0.1.4}/msdev_kit/fabric/report.py

@@ -144,6 +144,22 @@ class Report:
         return report_name
 
 
+    @staticmethod
+    def _get_nested_value(data: dict, path: List[Any], default: Any = None) -> Any:
+        """Safely traverses a nested dict/list structure."""
+        current = data
+        for key in path:
+            if isinstance(current, dict):
+                current = current.get(key)
+            elif isinstance(current, list) and isinstance(key, int) and len(current) > key:
+                current = current[key]
+            else:
+                return default
+            if current is None:
+                return default
+        return current
+
+
     def list_report_pages(
                 self,
                 workspace_id: str = '',
@@ -198,7 +214,7 @@ class Report:
                 return {'message': {'error': error_message, 'content': response}}
 
 
-    def get_report_json_pages_and_visuals(
+    def get_legacy_report_pages_and_visuals(
                     self,
                     json_data: str,
                     workspace_id: str,
@@ -214,21 +230,6 @@ class Report:
         Returns:
             Dict: status message and content.
         """
-        def get_nested_value(data: dict, path: List[Any], default: Any = None) -> Any:
-            """Safely traverses a nested dictionary/list structure."""
-            current = data
-            for key in path:
-                if isinstance(current, dict):
-                    current = current.get(key)
-                elif isinstance(current, list) and isinstance(key, int) and len(current) > key:
-                    current = current[key]
-                else:
-                    return default
-
-                if current is None:
-                    return default
-            return current
-
         # List to hold flat dictionary records for the final DataFrame
         report_records: List[Dict[str, Any]] = []
 
@@ -241,7 +242,7 @@ class Report:
                 print("Error: Invalid JSON format.")
                 return pd.DataFrame()
 
-        sections = get_nested_value(data, ['config', 'sections'])
+        sections = self._get_nested_value(data, ['config', 'sections'])
         if not sections:
             sections = data.get('sections', [])
 
@@ -252,7 +253,7 @@ class Report:
 
             for vc in visual_containers:
                 # The 'name' property inside 'config' is the unique Visual ID
-                visual_id = get_nested_value(vc, ['config', 'name'], 'No ID')
+                visual_id = self._get_nested_value(vc, ['config', 'name'], 'No ID')
                 visual_type = 'Unknown'
                 visual_title = 'No Title'
                 vc_config = vc.get('config', {})
@@ -281,14 +282,12 @@ class Report:
                             ['text', 0, 'properties', 'text', 'expr', 'Literal', 'Value'],
                             # Path 4: Text Visual/Button Label (sometimes the second text object)
                             ['text', 1, 'properties', 'text', 'expr', 'Literal', 'Value'],
-                            # Path 5: Text Visual/Button Label (sometimes the second text object)
-                            ['text', 1, 'properties', 'text', 'expr', 'Literal', 'Value']
                         ]
 
                         # Check on Objects
                         found_title = 'No Title'
                         for path in title_paths:
-                            title_value = get_nested_value(objects, path)
+                            title_value = self._get_nested_value(objects, path)
 
                             if isinstance(title_value, str) and title_value:
                                 found_title = title_value.strip("'").replace('\'', "'")
@@ -296,7 +295,7 @@ class Report:
 
                         # Check on vcObjects
                         for path in title_paths:
-                            title_value = get_nested_value(vc_objects, path)
+                            title_value = self._get_nested_value(vc_objects, path)
 
                             if isinstance(title_value, str) and title_value:
                                 found_title = title_value.strip("'").replace('\'', "'")
@@ -308,8 +307,8 @@ class Report:
                 title_expression = None
                 if visual_title == visual_type:
                     # Check for dynamic title expression
-                    title_obj = get_nested_value(objects, ['title', 0, 'properties', 'text', 'expr'])
-                    if title_obj and not get_nested_value(title_obj, ['Literal', 'Value']):
+                    title_obj = self._get_nested_value(objects, ['title', 0, 'properties', 'text', 'expr'])
+                    if title_obj and not self._get_nested_value(title_obj, ['Literal', 'Value']):
                         # Capture the full expression structure (DAX)
                         title_expression = str(title_obj)
 
@@ -335,6 +334,214 @@ class Report:
         return df
 
 
+    def get_pbir_report_pages_and_visuals(
+                self,
+                parts: List[Dict],
+                workspace_id: str,
+                report_id: str) -> pd.DataFrame:
+        """
+        Parses a PBIR report definition's parts array to extract pages and visual
+        details, and returns the result as a Pandas DataFrame.
+
+        Args:
+            parts (list): the 'parts' list from the Fabric API definition response
+                (report_content['definition']['parts']).
+            workspace_id (str): workspace id.
+            report_id (str): report id.
+
+        Returns:
+            pd.DataFrame: one row per visual with columns report_id, pageIndex,
+                pageName, visual_id, type, title, title_expression.
+        """
+        title_paths = [
+            ['title', 0, 'properties', 'text', 'expr', 'Literal', 'Value'],
+            ['general', 0, 'properties', 'title', 'expr', 'Literal', 'Value'],
+            ['text', 0, 'properties', 'text', 'expr', 'Literal', 'Value'],
+            ['text', 1, 'properties', 'text', 'expr', 'Literal', 'Value'],
+        ]
+
+        # Index parts by path for O(1) lookup
+        parts_by_path = {p['path']: p for p in parts}
+
+        # Decode pages.json to get ordered page IDs
+        pages_meta_part = parts_by_path.get('definition/pages/pages.json')
+        if not pages_meta_part:
+            return pd.DataFrame()
+
+        pages_meta = json.loads(base64.b64decode(pages_meta_part['payload']).decode('utf-8'))
+        page_order = pages_meta.get('pageOrder', [])
+
+        report_records: List[Dict[str, Any]] = []
+
+        for page_index, page_id in enumerate(page_order, start=1):
+            page_part = parts_by_path.get(f'definition/pages/{page_id}/page.json')
+            if not page_part:
+                continue
+            page_data = json.loads(base64.b64decode(page_part['payload']).decode('utf-8'))
+            page_name = page_data.get('displayName', page_id)
+
+            prefix = f'definition/pages/{page_id}/visuals/'
+            visual_parts = [
+                p for p in parts
+                if p['path'].startswith(prefix) and p['path'].endswith('/visual.json')
+            ]
+
+            for vp in visual_parts:
+                vc = json.loads(base64.b64decode(vp['payload']).decode('utf-8'))
+                visual_id = vc.get('name', 'No ID')
+                visual_type = 'Unknown'
+                visual_title = 'No Title'
+                title_expression = None
+
+                visual_group = vc.get('visualGroup')
+                if visual_group:
+                    visual_type = 'Visual Group (Container)'
+                    visual_title = visual_group.get('displayName', 'Visual Group')
+                else:
+                    visual_data = vc.get('visual', {})
+                    visual_type = visual_data.get('visualType', 'Generic Visual')
+                    objects = visual_data.get('objects', {})
+
+                    found_title = 'No Title'
+                    for path in title_paths:
+                        title_value = self._get_nested_value(objects, path)
+                        if isinstance(title_value, str) and title_value:
+                            found_title = title_value.strip("'").replace('\'', "'")
+                            break
+
+                    visual_title = found_title if found_title != 'No Title' else visual_type
+
+                    if visual_title == visual_type:
+                        title_obj = self._get_nested_value(
+                            objects, ['title', 0, 'properties', 'text', 'expr']
+                        )
+                        if title_obj and not self._get_nested_value(title_obj, ['Literal', 'Value']):
+                            title_expression = str(title_obj)
+
+                report_records.append({
+                    'report_id': report_id,
+                    'pageIndex': page_index,
+                    'pageName': page_name,
+                    'visual_id': visual_id,
+                    'type': visual_type,
+                    'title': visual_title,
+                    'title_expression': title_expression,
+                })
+
+        report_name = (
+            self.get_report_name(workspace_id, report_id)
+            .replace(' ', '').replace('(', '').replace(')', '').strip()
+        )
+        df = pd.DataFrame(report_records)
+        df.sort_values(by=['pageIndex', 'title'], inplace=True)
+        df.to_excel(f'{self.data_dir}/pages_and_visuals/{report_name}.xlsx', index=False)
+        return df
+
+
+    def get_report_pages_and_visuals(
+            self,
+            report_definition: Dict,
+            workspace_id: str,
+            report_id: str) -> pd.DataFrame:
+        """
+        Dispatcher. Detects the report format and delegates to the appropriate
+        pages-and-visuals parser.
+
+        Args:
+            report_definition (dict): dict with 'format' and 'parts' keys, as
+                returned by report_content['definition'] from the Fabric API.
+            workspace_id (str): workspace id.
+            report_id (str): report id.
+
+        Returns:
+            pd.DataFrame: one row per visual. Empty DataFrame for unknown formats.
+        """
+        fmt = report_definition.get('format', '').lower()
+        parts = report_definition.get('parts', [])
+
+        if fmt == 'pbir':
+            return self.get_pbir_report_pages_and_visuals(parts, workspace_id, report_id)
+
+        elif fmt == 'pbir-legacy':
+            report_payload = None
+            for part in parts:
+                if part.get('path') == 'report.json':
+                    report_payload = part.get('payload')
+                    break
+
+            if not report_payload:
+                print('get_report_pages_and_visuals: report.json not found in PBIR-Legacy parts.')
+                return pd.DataFrame()
+
+            decoded = base64.b64decode(report_payload).decode('utf-8')
+            json_data = json.loads(decoded)
+
+            if isinstance(json_data.get('config'), str):
+                json_data['config'] = json.loads(json_data['config'])
+
+            return self.get_legacy_report_pages_and_visuals(json_data, workspace_id, report_id)
+
+        else:
+            print(f'get_report_pages_and_visuals: unsupported format "{fmt}".')
+            return pd.DataFrame()
+
+
+    def get_report_definition(
+                self,
+                workspace_id: str = '',
+                report_id: str = '',
+                operations: Operations = None) -> Dict:
+        """
+        Fetch the raw report definition for any format (PBIR or PBIR-Legacy).
+
+        Returns the definition dict as-is from the Fabric API, without any
+        format-specific decoding. Pass the returned 'content' directly to
+        get_report_pages_and_visuals.
+
+        Args:
+            workspace_id (str): workspace id where the report is.
+            report_id (str): report id.
+            operations (Operations): Operations class instance.
+
+        Returns:
+            Dict: status message and content. On success, content is a dict
+                with 'format' and 'parts' keys.
+        """
+        if not workspace_id:
+            return {'message': 'Missing workspace id, please check.', 'content': ''}
+        if not report_id:
+            return {'message': 'Missing report id, please check.', 'content': ''}
+
+        request_url = f'{self.main_fabric_url}/workspaces/{workspace_id}/reports/{report_id}/getDefinition'
+        r = requests.post(url=request_url, headers=self.headers)
+        status = r.status_code
+        response = json.loads(r.content)
+        print(f'status_code={status}')
+
+        if status != 202:
+            error_message = response.get('error', {}).get('message', 'Unknown error')
+            return {'message': {'error': error_message}, 'content': response}
+
+        operation_id = r.headers.get('x-ms-operation-id')
+        print(f'operation_id={operation_id}')
+
+        while True:
+            active_operation_state = operations.get_operation_state(operation_id)
+            print('Operation state:', active_operation_state)
+            if active_operation_state:
+                operation_state = active_operation_state.get('operation_state', '')
+                if operation_state in ('Succeeded', 'Failed'):
+                    sleep(1)
+                    break
+            sleep(1)
+
+        report_content = operations.get_operation_result(operation_id).get('content', '')
+        report_definition = report_content.get('definition')
+        print(f'Report format: {report_definition.get("format")}')
+
+        return {'message': 'Success', 'content': report_definition}
+
+
     def get_legacy_report_json(
                 self,
                 workspace_id: str = '',
@@ -1007,6 +1214,4 @@ class Report:
                 'users_migrated': users_migrated,
                 'users_failed': users_failed
             }
-        }
-
-
+        }

{msdev_kit-0.1.2 → msdev_kit-0.1.4}/msdev_kit/fabric/workspace.py

@@ -135,6 +135,43 @@ class Workspace:
             return {'message': {'error': error_message, 'content': response}}
 
 
+    def list_workspaces(self, admin: bool = False, top: int = 5000) -> Dict:
+        """
+        List all workspaces the authenticated SPN has access to.
+
+        Args:
+            admin (bool, optional): use the admin API to list all tenant workspaces.
+                Requires tenant-level admin or read-all permissions. Defaults to False.
+            top (int, optional): max results per page (admin only). Defaults to 5000.
+
+        Returns:
+            Dict: status message and content.
+        """
+        if admin:
+            all_workspaces = []
+            request_url = self.main_url + f'/admin/groups?$top={top}'
+            while request_url:
+                r = requests.get(url=request_url, headers=self.headers)
+                if r.status_code != 200:
+                    body = json.loads(r.content)
+                    return {'message': {'error': body['error']['message'], 'content': body}}
+                body = json.loads(r.content)
+                all_workspaces.extend(body.get('value', []))
+                request_url = body.get('odata.nextLink', None)
+            response = all_workspaces
+        else:
+            r = requests.get(url=self.main_url + '/groups', headers=self.headers)
+            if r.status_code != 200:
+                body = json.loads(r.content)
+                return {'message': {'error': body['error']['message'], 'content': body}}
+            response = json.loads(r.content).get('value', [])
+
+        df = pd.DataFrame(response)
+        df.to_excel(f'{self.workspace_dir}/workspaces_all.xlsx', index=False)
+
+        return {'message': 'Success', 'content': response}
+
+
     def get_workspace_details(
                 self, 
                 workspace_id: str = '') -> Dict:

{msdev_kit-0.1.2 → msdev_kit-0.1.4}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "msdev-kit"
 description = "Microsoft developer toolkit: Fabric, MS Graph, and SharePoint"
-version = "0.1.2"
+version = "0.1.4"
 requires-python = ">=3.10"
 readme = "README.md"
 license = {text = "MIT"}