pvw-cli 1.2.3__py3-none-any.whl → 1.2.5__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of pvw-cli might be problematic. Click here for more details.

@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: pvw-cli
3
- Version: 1.2.3
3
+ Version: 1.2.5
4
4
  Summary: Microsoft Purview CLI with comprehensive automation capabilities
5
5
  Author-email: AYOUB KEBAILI <keayoub@msn.com>
6
6
  Maintainer-email: AYOUB KEBAILI <keayoub@msn.com>
@@ -56,26 +56,28 @@ Requires-Dist: pytest-asyncio>=0.20.0; extra == "test"
56
56
  Requires-Dist: pytest-cov>=4.0.0; extra == "test"
57
57
  Requires-Dist: requests-mock>=1.9.0; extra == "test"
58
58
 
59
- # PURVIEW CLI v1.2.3 - Microsoft Purview Automation & Data Governance
60
-
61
- > **LATEST UPDATE v1.2.3 (October 2025):**
62
- > - **🔍 FIXED: Search API Integration** - Fixed `suggest` and `autocomplete` API payload format (HTTP 400 errors resolved)
63
- > - **📊 ENHANCED: Collection Display** - Improved collection name detection in search results with proper fallback logic
64
- > - **✅ VALIDATED: All Search Commands** - Comprehensive testing of query, browse, suggest, find-table operations
65
- > - **📦 NEW: Bulk Term Import/Export** - Import multiple terms from CSV/JSON with dry-run support
66
- > - **🗑️ NEW: Bulk Delete Scripts** - PowerShell and Python scripts for bulk term deletion
67
- > - **📊 NEW: Multiple Output Formats** - `--output` flag supports table, json, and jsonc formats
68
- > - **🔧 NEW: PowerShell Integration** - Plain JSON output works with `ConvertFrom-Json`
69
- > - **🚀 Complete Data Product CRUD** - Full update and delete support with smart partial updates
70
- > - **🏥 Health Monitoring API** - Automated governance health checks and recommendations
71
- > - **🔄 Workflow Management** - Approval workflows and business process automation
72
- > - **🚀 Complete Microsoft Purview Unified Catalog (UC)** - Full governance domains, glossary terms, data products, OKRs, and CDEs
59
+ # PURVIEW CLI v1.2.5 - Microsoft Purview Automation & Data Governance
60
+
61
+ > **LATEST UPDATE v1.2.5 (October 2025):**
62
+ >
63
+ > - **[NEW]** Lineage CSV Import - Bulk import lineage relationships from CSV files with validation and sample generation
64
+ > - **[FIXED]** Search API Integration - Fixed `suggest` and `autocomplete` API payload format (HTTP 400 errors resolved)
65
+ > - **[ENHANCED]** Collection Display - Improved collection name detection in search results with proper fallback logic
66
+ > - **[OK]** All Search Commands - Comprehensive testing of query, browse, suggest, find-table operations
67
+ > - **[NEW]** Bulk Term Import/Export - Import multiple terms from CSV/JSON with dry-run support
68
+ > - **[NEW]** Bulk Delete Scripts - PowerShell and Python scripts for bulk term deletion
69
+ > - **[NEW]** Multiple Output Formats - `--output` flag supports table, json, and jsonc formats
70
+ > - **[NEW]** PowerShell Integration - Plain JSON output works with `ConvertFrom-Json`
71
+ > - **[OK]** Complete Data Product CRUD - Full update and delete support with smart partial updates
72
+ > - **[OK]** Health Monitoring API - Automated governance health checks and recommendations
73
+ > - **[OK]** Workflow Management - Approval workflows and business process automation
74
+ > - **[OK]** Complete Microsoft Purview Unified Catalog (UC) - Full governance domains, glossary terms, data products, OKRs, and CDEs
73
75
 
74
76
  ---
75
77
 
76
78
  ## What is PVW CLI?
77
79
 
78
- **PVW CLI v1.2.3** is a modern, full-featured command-line interface and Python library for Microsoft Purview. It enables automation and management of *all major Purview APIs* including:
80
+ **PVW CLI v1.2.5** is a modern, full-featured command-line interface and Python library for Microsoft Purview. It enables automation and management of *all major Purview APIs* including:
79
81
 
80
82
  - **Unified Catalog (UC) Management** - Complete governance domains, glossary terms, data products, OKRs, CDEs
81
83
  - **Bulk Operations** - Import/export terms from CSV/JSON, bulk delete scripts with progress tracking
@@ -90,9 +92,18 @@ Requires-Dist: requests-mock>=1.9.0; extra == "test"
90
92
 
91
93
  The CLI is designed for data engineers, stewards, architects, and platform teams to automate, scale, and enhance their Microsoft Purview experience.
92
94
 
95
+ ### NEW: MCP Server for AI Assistants
96
+
97
+ **[NEW]** Model Context Protocol (MCP) server enables LLM-powered data governance workflows!
98
+
99
+ - Natural language interface to Purview catalog
100
+ - 20+ tools for AI assistants (Claude, Cline, etc.)
101
+ - Automate complex multi-step operations
102
+ - See `mcp/README.md` for setup instructions
103
+
93
104
  ---
94
105
 
95
- ## Getting Started
106
+ ## Getting Started
96
107
 
97
108
  Follow this short flow to get PVW CLI installed and running quickly.
98
109
 
@@ -144,8 +155,8 @@ Follow this short flow to get PVW CLI installed and running quickly.
144
155
 
145
156
  3. Authenticate
146
157
 
147
- - Run `az login` (recommended), or
148
- - Provide Service Principal credentials via environment variables.
158
+ - Run `az login` (recommended), or
159
+ - Provide Service Principal credentials via environment variables.
149
160
 
150
161
  4. Try a few commands:
151
162
 
@@ -161,14 +172,15 @@ Follow this short flow to get PVW CLI installed and running quickly.
161
172
  pvw uc --help
162
173
  ```
163
174
 
164
- For more advanced usage, see the documentation in `doc/` or the project docs: https://pvw-cli.readthedocs.io/
175
+ For more advanced usage, see the documentation in `doc/` or the project docs: <https://pvw-cli.readthedocs.io/>
165
176
 
166
177
  ---
167
178
 
168
179
  ## Overview
169
180
 
170
- **PVW CLI v1.2.3** is a modern command-line interface and Python library for Microsoft Purview, enabling:
181
+ **PVW CLI v1.2.5** is a modern command-line interface and Python library for Microsoft Purview, enabling:
171
182
 
183
+ - **MCP Server** - Natural language interface for AI assistants (Claude, Cline)
172
184
  - Advanced data catalog search and discovery
173
185
  - Bulk import/export of entities, glossary terms, and lineage
174
186
  - Real-time monitoring and analytics
@@ -296,6 +308,7 @@ If you are signed in to Azure in Visual Studio or VS Code, `DefaultAzureCredenti
296
308
  ---
297
309
 
298
310
  **Note:**
311
+
299
312
  - The CLI will try all supported authentication methods in order. The first one that works will be used.
300
313
  - For most automation and CI/CD scenarios, service principal authentication is recommended.
301
314
  - For local development, Azure CLI authentication is easiest.
@@ -381,11 +394,13 @@ pvw uc dataproduct list --domain-id "abc-123" --output json
381
394
  ### Migration from Old --json Flag
382
395
 
383
396
  **Old (deprecated):**
397
+
384
398
  ```bash
385
399
  pvw uc term list --domain-id "abc-123" --json
386
400
  ```
387
401
 
388
402
  **New (recommended):**
403
+
389
404
  ```bash
390
405
  pvw uc term list --domain-id "abc-123" --output json # Plain JSON for scripting
391
406
  pvw uc term list --domain-id "abc-123" --output jsonc # Colored JSON (old behavior)
@@ -400,21 +415,25 @@ Before using PVW CLI, you need to set three essential environment variables. Her
400
415
  ### 🔍 **How to Find Your Purview Values**
401
416
 
402
417
  #### **1. PURVIEW_ACCOUNT_NAME**
418
+
403
419
  - This is your Purview account name as it appears in Azure Portal
404
420
  - Example: `kaydemopurview`
405
421
 
406
- #### **2. PURVIEW_ACCOUNT_ID**
422
+ #### **2. PURVIEW_ACCOUNT_ID**
423
+
407
424
  - This is the GUID that identifies your Purview account for Unified Catalog APIs
408
- - **✅ Important: For most Purview deployments, this is your Azure Tenant ID**
425
+ - **Important: For most Purview deployments, this is your Azure Tenant ID**
409
426
 
410
427
  - **Method 1 - Get your Tenant ID (recommended):**
411
428
 
412
429
  **Bash/Command Prompt:**
430
+
413
431
  ```bash
414
432
  az account show --query tenantId -o tsv
415
433
  ```
416
434
 
417
435
  **PowerShell:**
436
+
418
437
  ```powershell
419
438
  az account show --query tenantId -o tsv
420
439
  # Or store directly in environment variable:
@@ -422,9 +441,11 @@ Before using PVW CLI, you need to set three essential environment variables. Her
422
441
  ```
423
442
 
424
443
  - **Method 2 - Azure CLI (extract from Atlas endpoint):**
444
+
425
445
  ```bash
426
446
  az purview account show --name YOUR_ACCOUNT_NAME --resource-group YOUR_RG --query endpoints.catalog -o tsv
427
447
  ```
448
+
428
449
  Extract the GUID from the URL (before `-api.purview-service.microsoft.com`)
429
450
 
430
451
  - **Method 3 - Azure Portal:**
@@ -433,12 +454,14 @@ Before using PVW CLI, you need to set three essential environment variables. Her
433
454
  3. Extract GUID from: `https://GUID-api.purview-service.microsoft.com/catalog`
434
455
 
435
456
  #### **3. PURVIEW_RESOURCE_GROUP**
457
+
436
458
  - The Azure resource group containing your Purview account
437
459
  - Example: `fabric-artifacts`
438
460
 
439
461
  ### 📋 **Setting the Variables**
440
462
 
441
463
  **Windows Command Prompt:**
464
+
442
465
  ```cmd
443
466
  set PURVIEW_ACCOUNT_NAME=your-purview-account
444
467
  set PURVIEW_ACCOUNT_ID=your-purview-account-id
@@ -446,6 +469,7 @@ set PURVIEW_RESOURCE_GROUP=your-resource-group
446
469
  ```
447
470
 
448
471
  **Windows PowerShell:**
472
+
449
473
  ```powershell
450
474
  $env:PURVIEW_ACCOUNT_NAME="your-purview-account"
451
475
  $env:PURVIEW_ACCOUNT_ID="your-purview-account-id"
@@ -453,6 +477,7 @@ $env:PURVIEW_RESOURCE_GROUP="your-resource-group"
453
477
  ```
454
478
 
455
479
  **Linux/macOS:**
480
+
456
481
  ```bash
457
482
  export PURVIEW_ACCOUNT_NAME=your-purview-account
458
483
  export PURVIEW_ACCOUNT_ID=your-purview-account-id
@@ -460,6 +485,7 @@ export PURVIEW_RESOURCE_GROUP=your-resource-group
460
485
  ```
461
486
 
462
487
  **Permanent (Windows Command Prompt):**
488
+
463
489
  ```cmd
464
490
  setx PURVIEW_ACCOUNT_NAME "your-purview-account"
465
491
  setx PURVIEW_ACCOUNT_ID "your-purview-account-id"
@@ -467,17 +493,19 @@ setx PURVIEW_RESOURCE_GROUP "your-resource-group"
467
493
  ```
468
494
 
469
495
  **Permanent (Windows PowerShell):**
496
+
470
497
  ```powershell
471
498
  [Environment]::SetEnvironmentVariable("PURVIEW_ACCOUNT_NAME", "your-purview-account", "User")
472
499
  [Environment]::SetEnvironmentVariable("PURVIEW_ACCOUNT_ID", "your-purview-account-id", "User")
473
500
  [Environment]::SetEnvironmentVariable("PURVIEW_RESOURCE_GROUP", "your-resource-group", "User")
474
501
  ```
475
502
 
476
- ### 🔧 **Debug Environment Issues**
503
+ ### **Debug Environment Issues**
477
504
 
478
505
  If you experience issues with environment variables between different terminals, use these debug commands:
479
506
 
480
507
  **Command Prompt/Bash:**
508
+
481
509
  ```bash
482
510
  # Run this to check your current environment
483
511
  python -c "
@@ -489,6 +517,7 @@ print('PURVIEW_RESOURCE_GROUP:', os.getenv('PURVIEW_RESOURCE_GROUP'))
489
517
  ```
490
518
 
491
519
  **PowerShell:**
520
+
492
521
  ```powershell
493
522
  # Check environment variables in PowerShell
494
523
  python -c "
@@ -514,14 +543,15 @@ The PVW CLI provides advanced search using the latest Microsoft Purview Discover
514
543
  - Use autocomplete and suggestion endpoints
515
544
  - Perform faceted, time-based, and entity-type-specific queries
516
545
 
517
- **✅ v1.2.3 Improvements:**
546
+ **v1.2.5 Improvements:**
547
+
518
548
  - Fixed `suggest` and `autocomplete` API payload format (removed empty filter causing HTTP 400 errors)
519
549
  - Enhanced collection display with robust type checking and fallback logic
520
550
  - All search commands validated and working correctly (query, browse, suggest, find-table)
521
551
 
522
552
  ### CLI Usage Examples
523
553
 
524
- #### 🎯 **Multiple Output Formats**
554
+ #### **Multiple Output Formats**
525
555
 
526
556
  ```bash
527
557
  # 1. Table Format (Default) - Quick overview
@@ -541,7 +571,7 @@ pvw search query --keywords="customer" --limit=5 --show-ids
541
571
  # → Table format + entity GUIDs for copy/paste into update commands
542
572
  ```
543
573
 
544
- #### 🔍 **Search Operations**
574
+ #### **Search Operations**
545
575
 
546
576
  ```bash
547
577
  # Basic search for assets with keyword 'customer'
@@ -559,12 +589,12 @@ pvw search autocomplete --keywords="ord" --limit=3
559
589
  # Get search suggestions (fuzzy matching)
560
590
  pvw search suggest --keywords="prod" --limit=2
561
591
 
562
- **⚠️ IMPORTANT - Command Line Quoting:**
592
+ **IMPORTANT - Command Line Quoting:**
563
593
  ```cmd
564
- # CORRECT - Use quotes around keywords
594
+ # [OK] CORRECT - Use quotes around keywords
565
595
  pvw search query --keywords="customer" --limit=5
566
596
 
567
- # CORRECT - For wildcard searches, use quotes
597
+ # [OK] CORRECT - For wildcard searches, use quotes
568
598
  pvw search query --keywords="*" --limit=5
569
599
 
570
600
  # ❌ WRONG - Don't use unquoted * (shell expands to file names)
@@ -586,7 +616,7 @@ pvw search query --keywords="audit" --createdAfter="2024-01-01" --limit=1
586
616
  pvw search query --keywords="finance" --entityTypes="Files,Tables" --limit=2
587
617
  ```
588
618
 
589
- #### 💡 **Usage Scenarios**
619
+ #### **Usage Scenarios**
590
620
 
591
621
  - **Daily browsing**: Use default table format for quick scans
592
622
  - **Understanding assets**: Use `--detailed` for rich information panels
@@ -629,7 +659,7 @@ See [`doc/commands/unified-catalog.md`](doc/commands/unified-catalog.md) for com
629
659
 
630
660
  ### Quick UC Examples
631
661
 
632
- #### 🏛️ **Governance Domains Management**
662
+ #### **Governance Domains Management**
633
663
 
634
664
  ```bash
635
665
  # List all governance domains
@@ -645,7 +675,7 @@ pvw uc domain get --domain-id "abc-123-def-456"
645
675
  pvw uc domain update --domain-id "abc-123" --description "Updated financial governance"
646
676
  ```
647
677
 
648
- #### 📖 **Glossary Terms in UC**
678
+ #### **Glossary Terms in UC**
649
679
 
650
680
  ```bash
651
681
  # List all terms in a domain
@@ -686,14 +716,16 @@ pvw uc term import-json --json-file "samples/json/term/uc_terms_bulk_example.jso
686
716
  ```
687
717
 
688
718
  **Bulk Import Features:**
689
- - ✅ Import from CSV or JSON files
690
- - Dry-run mode to preview before importing
691
- - Support for multiple owners (Entra ID Object IDs), acronyms, and resources
692
- - Progress tracking with Rich console output
693
- - Detailed error messages and summary reports
694
- - Sequential POST requests (no native bulk endpoint available)
719
+
720
+ - [OK] Import from CSV or JSON files
721
+ - [OK] Dry-run mode to preview before importing
722
+ - [OK] Support for multiple owners (Entra ID Object IDs), acronyms, and resources
723
+ - [OK] Progress tracking with Rich console output
724
+ - [OK] Detailed error messages and summary reports
725
+ - [OK] Sequential POST requests (no native bulk endpoint available)
695
726
 
696
727
  **CSV Format Example:**
728
+
697
729
  ```csv
698
730
  name,description,status,acronym,owner_id,resource_name,resource_url
699
731
  Customer Acquisition Cost,Cost to acquire new customer,Draft,CAC,<guid>,Metrics Guide,https://docs.example.com
@@ -701,6 +733,7 @@ Monthly Recurring Revenue,Predictable monthly revenue,Draft,MRR,<guid>,Finance D
701
733
  ```
702
734
 
703
735
  **JSON Format Example:**
736
+
704
737
  ```json
705
738
  {
706
739
  "terms": [
@@ -718,9 +751,10 @@ Monthly Recurring Revenue,Predictable monthly revenue,Draft,MRR,<guid>,Finance D
718
751
  ```
719
752
 
720
753
  **Important Notes:**
754
+
721
755
  - ⚠️ **Owner IDs must be Entra ID Object IDs (GUIDs)**, not email addresses
722
756
  - ⚠️ **Terms cannot be "Published" in unpublished domains** - use "Draft" status
723
- - Sample files available: `samples/csv/uc_terms_bulk_example.csv`, `samples/json/term/uc_terms_bulk_example.json`
757
+ - [OK] Sample files available: `samples/csv/uc_terms_bulk_example.csv`, `samples/json/term/uc_terms_bulk_example.json`
724
758
  - 📖 Complete documentation: [`doc/commands/unified-catalog/term-bulk-import.md`](doc/commands/unified-catalog/term-bulk-import.md)
725
759
 
726
760
  **🗑️ Bulk Delete (NEW)**
@@ -744,14 +778,15 @@ python scripts/delete_all_uc_terms_v2.py --domain-id "abc-123" --force
744
778
  ```
745
779
 
746
780
  **Bulk Delete Features:**
747
- - ✅ Interactive confirmation prompts (type "DELETE" to confirm)
748
- - ✅ Beautiful progress display with colors
749
- - ✅ Success/failure tracking per term
750
- - ✅ Detailed summary reports
751
- - ✅ Rate limiting (200ms delay between deletes)
752
- - ✅ Graceful error handling and Ctrl+C support
753
781
 
754
- #### 📦 **Data Products Management**
782
+ - [OK] Interactive confirmation prompts (type "DELETE" to confirm)
783
+ - [OK] Beautiful progress display with colors
784
+ - [OK] Success/failure tracking per term
785
+ - [OK] Detailed summary reports
786
+ - [OK] Rate limiting (200ms delay between deletes)
787
+ - [OK] Graceful error handling and Ctrl+C support
788
+
789
+ #### **Data Products Management**
755
790
 
756
791
  ```bash
757
792
  # List all data products in a domain
@@ -789,7 +824,7 @@ pvw uc dataproduct delete --product-id "prod-789"
789
824
  pvw uc dataproduct delete --product-id "prod-789" --yes
790
825
  ```
791
826
 
792
- #### 🎯 **Objectives & Key Results (OKRs)**
827
+ #### **Objectives & Key Results (OKRs)**
793
828
 
794
829
  ```bash
795
830
  # List objectives for a domain
@@ -810,7 +845,7 @@ pvw uc objective update \
810
845
  --status "in-progress"
811
846
  ```
812
847
 
813
- #### 🔑 **Critical Data Elements (CDEs)**
848
+ #### **Critical Data Elements (CDEs)**
814
849
 
815
850
  ```bash
816
851
  # List critical data elements
@@ -831,7 +866,7 @@ pvw uc cde link \
831
866
  --asset-id "ea3412c3-7387-4bc1-9923-11f6f6f60000"
832
867
  ```
833
868
 
834
- #### 🏥 **Health Monitoring (NEW)**
869
+ #### **Health Monitoring (NEW)**
835
870
 
836
871
  Monitor governance health and get automated recommendations to improve your data governance posture.
837
872
 
@@ -864,6 +899,7 @@ pvw uc health query --json
864
899
  ```
865
900
 
866
901
  **Health Finding Types:**
902
+
867
903
  - Missing glossary terms on data products (High)
868
904
  - Data products without OKRs (Medium)
869
905
  - Missing data quality scores (Medium)
@@ -871,7 +907,7 @@ pvw uc health query --json
871
907
  - Description quality issues (Medium)
872
908
  - Business domains without critical data entities (Medium)
873
909
 
874
- #### 🔄 **Workflow Management (NEW)**
910
+ #### **Workflow Management (NEW)**
875
911
 
876
912
  Manage approval workflows and business process automation in Purview.
877
913
 
@@ -905,12 +941,13 @@ pvw workflow list --json
905
941
  ```
906
942
 
907
943
  **Workflow Use Cases:**
944
+
908
945
  - Data access request approvals
909
946
  - Glossary term certification workflows
910
947
  - Data product publishing approvals
911
948
  - Classification review processes
912
949
 
913
- #### 🔄 **Integrated Workflow Example**
950
+ #### **Integrated Workflow Example**
914
951
 
915
952
  ```bash
916
953
  # 1. Discover assets to govern
@@ -935,7 +972,7 @@ pvw uc objective create --definition "Ensure 100% PII classification compliance"
935
972
 
936
973
  PVW CLI provides comprehensive entity management capabilities for updating Purview assets like descriptions, classifications, and custom attributes.
937
974
 
938
- ### 🔄 **Entity Update Examples**
975
+ ### **Entity Update Examples**
939
976
 
940
977
  #### **Update Asset Descriptions**
941
978
 
@@ -982,7 +1019,7 @@ pvw entity add-classification \
982
1019
  --classification "MICROSOFT.PERSONAL.EMAIL"
983
1020
  ```
984
1021
 
985
- ### 🔍 **Discovery to Update Workflow**
1022
+ ### **Discovery to Update Workflow**
986
1023
 
987
1024
  ```bash
988
1025
  # 1. Find assets that need updates
@@ -1003,6 +1040,90 @@ pvw search query --keywords="FOUND_GUID" --detailed
1003
1040
 
1004
1041
  ---
1005
1042
 
1043
+ ## Lineage CSV Import & Management
1044
+
1045
+ PVW CLI provides powerful lineage management capabilities including CSV-based bulk import for automating data lineage creation.
1046
+
1047
+ ### **Lineage CSV Import**
1048
+
1049
+ Import lineage relationships from CSV files to automate the creation of data flow documentation in Microsoft Purview.
1050
+
1051
+ #### **CSV Format**
1052
+
1053
+ The CSV file must contain the following columns:
1054
+
1055
+ **Required columns:**
1056
+
1057
+ - `source_entity_guid` - GUID of the source entity
1058
+ - `target_entity_guid` - GUID of the target entity
1059
+
1060
+ **Optional columns:**
1061
+
1062
+ - `relationship_type` - Type of relationship (default: "Process")
1063
+ - `process_name` - Name of the transformation process
1064
+ - `description` - Description of the transformation
1065
+ - `confidence_score` - Confidence score (0-1)
1066
+ - `owner` - Process owner
1067
+ - `metadata` - Additional JSON metadata
1068
+
1069
+ **Example CSV:**
1070
+
1071
+ ```csv
1072
+ source_entity_guid,target_entity_guid,relationship_type,process_name,description,confidence_score,owner,metadata
1073
+ dcfc99ed-c74d-49aa-bd0b-72f6f6f60000,1db9c650-acfb-4914-8bc5-1cf6f6f60000,Process,Transform_Product_Data,Transform product data for analytics,0.95,data-engineering,"{""tool"": ""Azure Data Factory""}"
1074
+ ```
1075
+
1076
+ #### **Lineage Commands**
1077
+
1078
+ ```bash
1079
+ # Validate CSV format before import (no API calls)
1080
+ pvw lineage validate lineage_data.csv
1081
+
1082
+ # Import lineage relationships from CSV
1083
+ pvw lineage import lineage_data.csv
1084
+
1085
+ # Generate sample CSV file with examples
1086
+ pvw lineage sample output.csv --num-samples 10 --template detailed
1087
+
1088
+ # View available CSV templates
1089
+ pvw lineage templates
1090
+ ```
1091
+
1092
+ #### **Available Templates**
1093
+
1094
+ - **`basic`** - Minimal columns (source, target, process name)
1095
+ - **`detailed`** - All columns including metadata and confidence scores
1096
+ - **`qualified_names`** - Use qualified names instead of GUIDs
1097
+
1098
+ #### **Workflow Example**
1099
+
1100
+ ```bash
1101
+ # 1. Find entity GUIDs using search
1102
+ pvw search find-table --name "Product" --schema "dbo" --id-only
1103
+
1104
+ # 2. Create CSV file with lineage relationships
1105
+ # (use the GUIDs from step 1)
1106
+
1107
+ # 3. Validate CSV format
1108
+ pvw lineage validate my_lineage.csv
1109
+ # Output: SUCCESS: Lineage validation passed (5 rows, 8 columns)
1110
+
1111
+ # 4. Import to Purview
1112
+ pvw lineage import my_lineage.csv
1113
+ # Output: SUCCESS: Lineage import completed successfully
1114
+ ```
1115
+
1116
+ #### **Advanced Features**
1117
+
1118
+ - **GUID Validation**: Automatic validation of GUID format with helpful error messages
1119
+ - **Process Entity Creation**: Creates intermediate "Process" entities to link source→target relationships
1120
+ - **Metadata Support**: Add custom JSON metadata to each lineage relationship
1121
+ - **Dry-Run Validation**: Validate CSV format locally before making API calls
1122
+
1123
+ **For detailed documentation, see:** [`doc/guides/lineage-csv-import.md`](doc/guides/lineage-csv-import.md)
1124
+
1125
+ ---
1126
+
1006
1127
  ## Data Product Management (Legacy)
1007
1128
 
1008
1129
  PVW CLI also includes the original `data-product` command group for backward compatibility with traditional data product lifecycle management.
@@ -1032,15 +1153,18 @@ pvw data-product show-lineage --qualified-name="product.test.1"
1032
1153
  ## Core Features
1033
1154
 
1034
1155
  - **Unified Catalog (UC)**: Complete modern data governance (NEW)
1156
+
1035
1157
  ```bash
1036
1158
  # Manage governance domains, terms, data products, OKRs, CDEs
1037
1159
  pvw uc domain list
1038
1160
  pvw uc term create --name "Customer" --domain-id "abc-123"
1039
1161
  pvw uc objective create --definition "Improve quality" --domain-id "abc-123"
1040
1162
  ```
1163
+
1041
1164
  - **Discovery Query/Search**: Flexible, advanced search for all catalog assets
1042
1165
  - **Entity Management**: Bulk import/export, update, and validation
1043
1166
  - **Glossary Management**: Import/export terms, assign terms in bulk
1167
+
1044
1168
  ```bash
1045
1169
  # List all terms in a glossary
1046
1170
  pvw glossary list-terms --glossary-guid "your-glossary-guid"
@@ -1048,7 +1172,20 @@ pvw data-product show-lineage --qualified-name="product.test.1"
1048
1172
  # Create and manage glossary terms
1049
1173
  pvw glossary create-term --payload-file term.json
1050
1174
  ```
1051
- - **Lineage Operations**: Lineage discovery, CSV-based bulk lineage
1175
+
1176
+ - **Lineage Operations**: Lineage discovery, CSV-based bulk lineage import/export
1177
+
1178
+ ```bash
1179
+ # Import lineage relationships from CSV
1180
+ pvw lineage import lineage_data.csv
1181
+
1182
+ # Validate CSV format before import
1183
+ pvw lineage validate lineage_data.csv
1184
+
1185
+ # Generate sample CSV file
1186
+ pvw lineage sample output.csv --num-samples 10
1187
+ ```
1188
+
1052
1189
  - **Monitoring & Analytics**: Real-time dashboards, metrics, and reporting
1053
1190
  - **Plugin System**: Extensible with custom plugins
1054
1191
 
@@ -1060,14 +1197,14 @@ PVW CLI provides comprehensive automation for all major Microsoft Purview APIs,
1060
1197
 
1061
1198
  ### Supported API Groups
1062
1199
 
1063
- - **Unified Catalog**: Complete governance domains, glossary terms, data products, OKRs, CDEs management
1064
- - **Health Monitoring**: Automated governance health checks and recommendations NEW
1065
- - **Workflows**: Approval workflows and business process automation NEW
1066
- - **Data Map**: Full entity and lineage management
1067
- - **Discovery**: Advanced search, browse, and query capabilities
1068
- - **Collections**: Collection and account management
1069
- - **Management**: Administrative operations
1070
- - **Scan**: Data source scanning and configuration
1200
+ - **Unified Catalog**: Complete governance domains, glossary terms, data products, OKRs, CDEs management [OK]
1201
+ - **Health Monitoring**: Automated governance health checks and recommendations [OK] NEW
1202
+ - **Workflows**: Approval workflows and business process automation [OK] NEW
1203
+ - **Data Map**: Full entity and lineage management [OK]
1204
+ - **Discovery**: Advanced search, browse, and query capabilities [OK]
1205
+ - **Collections**: Collection and account management [OK]
1206
+ - **Management**: Administrative operations [OK]
1207
+ - **Scan**: Data source scanning and configuration [OK]
1071
1208
 
1072
1209
  ### API Version Support
1073
1210
 
@@ -1079,6 +1216,7 @@ PVW CLI provides comprehensive automation for all major Microsoft Purview APIs,
1079
1216
  - Scan: **2018-12-01-preview**
1080
1217
 
1081
1218
  For the latest API documentation and updates, see:
1219
+
1082
1220
  - [Microsoft Purview REST API reference](https://learn.microsoft.com/en-us/rest/api/purview/)
1083
1221
  - [Atlas 2.2 API documentation](https://learn.microsoft.com/en-us/purview/data-gov-api-atlas-2-2)
1084
1222
  - [Azure Updates](https://azure.microsoft.com/updates/) for new releases
@@ -1094,9 +1232,18 @@ PVW CLI includes comprehensive sample files and scripts for bulk operations:
1094
1232
  ### Bulk Import Samples
1095
1233
 
1096
1234
  - **CSV Samples:** `samples/csv/uc_terms_bulk_example.csv` (8 sample terms)
1097
- - **JSON Samples:**
1235
+ - **JSON Samples:**
1098
1236
  - `samples/json/term/uc_terms_bulk_example.json` (8 data management terms)
1099
1237
  - `samples/json/term/uc_terms_sample.json` (8 business terms)
1238
+ - **Lineage CSV Samples:** `samples/csv/lineage_example.csv` - Multiple lineage relationships with metadata
1239
+
1240
+ ### Lineage Documentation
1241
+
1242
+ - **Comprehensive Guide:** `doc/guides/lineage-csv-import.md` - Complete lineage CSV import documentation
1243
+ - CSV format specification with required/optional columns
1244
+ - Command examples for validate, import, sample, templates
1245
+ - Workflow recommendations and troubleshooting
1246
+ - Advanced scenarios with metadata and multiple transformations
1100
1247
 
1101
1248
  ### Bulk Delete Scripts
1102
1249
 
@@ -1137,27 +1284,31 @@ PVW CLI includes comprehensive sample files and scripts for bulk operations:
1137
1284
 
1138
1285
  ## Recent Updates (October 2025)
1139
1286
 
1140
- ### Bulk Term Import/Export
1287
+ ### Bulk Term Import/Export
1288
+
1141
1289
  - Import multiple terms from CSV or JSON files
1142
1290
  - Dry-run mode for validation before import
1143
1291
  - Support for owners (Entra ID GUIDs), acronyms, resources
1144
1292
  - Progress tracking and detailed error reporting
1145
1293
  - 100% success rate in testing (8/8 terms)
1146
1294
 
1147
- ### PowerShell & Scripting Integration
1295
+ ### PowerShell & Scripting Integration
1296
+
1148
1297
  - New `--output` parameter with table/json/jsonc formats
1149
1298
  - Plain JSON works with PowerShell's `ConvertFrom-Json`
1150
1299
  - Compatible with jq, Python json module, and other tools
1151
1300
  - Migration from deprecated `--json` flag
1152
1301
 
1153
- ### Bulk Delete Scripts
1302
+ ### Bulk Delete Scripts
1303
+
1154
1304
  - PowerShell script with interactive confirmation ("DELETE" to confirm)
1155
1305
  - Python script with Rich progress bars
1156
1306
  - Beautiful UI with colored output
1157
1307
  - Success/failure tracking per term
1158
1308
  - Rate limiting (200ms delay)
1159
1309
 
1160
- ### Critical Fixes (v1.2.3)
1310
+ ### Critical Fixes (v1.2.5)
1311
+
1161
1312
  - **Search API Suggest/Autocomplete:** Fixed HTTP 400 errors by removing empty filter objects from payload
1162
1313
  - **Collection Display:** Enhanced collection name detection with proper fallback logic (isinstance checks)
1163
1314
  - **Owner ID Format:** Must use Entra ID Object IDs (GUIDs), not email addresses
@@ -1169,29 +1320,34 @@ PVW CLI includes comprehensive sample files and scripts for bulk operations:
1169
1320
 
1170
1321
  ## Key Features Summary
1171
1322
 
1172
- ### 🚀 **Unified Catalog (UC) - Complete Management**
1323
+ ### **Unified Catalog (UC) - Complete Management**
1324
+
1173
1325
  - Governance domains, glossary terms, data products
1174
1326
  - Objectives & Key Results (OKRs), Critical Data Elements (CDEs)
1175
1327
  - Health monitoring and workflow automation
1176
1328
  - Full CRUD operations with smart partial updates
1177
1329
 
1178
- ### 📦 **Bulk Operations**
1330
+ ### **Bulk Operations**
1331
+
1179
1332
  - CSV/JSON import with dry-run validation
1180
1333
  - PowerShell and Python bulk delete scripts
1181
1334
  - Progress tracking and error handling
1182
1335
  - Sample files and templates included
1183
1336
 
1184
- ### 📊 **Multiple Output Formats**
1337
+ ### **Multiple Output Formats**
1338
+
1185
1339
  - Table format for human viewing (default)
1186
1340
  - Plain JSON for PowerShell/bash scripting
1187
1341
  - Colored JSON for visual inspection
1188
1342
 
1189
- ### 🔧 **Automation & Integration**
1343
+ ### **Automation & Integration**
1344
+
1190
1345
  - Azure CLI, Service Principal, Managed Identity auth
1191
1346
  - Works in local development, CI/CD, and production
1192
1347
  - Compatible with PowerShell, bash, Python, jq
1193
1348
 
1194
- ### 📚 **Comprehensive Documentation**
1349
+ ### **Comprehensive Documentation**
1350
+
1195
1351
  - Complete API coverage documentation
1196
1352
  - Jupyter notebook examples
1197
1353
  - Troubleshooting guides
@@ -1214,9 +1370,10 @@ See [LICENSE](LICENSE) file for details.
1214
1370
 
1215
1371
  ---
1216
1372
 
1217
- **PVW CLI v1.2.3 empowers data engineers, stewards, and architects to automate, scale, and enhance their Microsoft Purview experience with powerful command-line and programmatic capabilities.**
1373
+ **PVW CLI v1.2.5 empowers data engineers, stewards, and architects to automate, scale, and enhance their Microsoft Purview experience with powerful command-line and programmatic capabilities.**
1374
+
1375
+ **Latest in v1.2.5:**
1218
1376
 
1219
- **Latest in v1.2.3:**
1220
1377
  - Fixed Search API suggest/autocomplete (HTTP 400 errors resolved)
1221
1378
  - Enhanced collection display with robust fallback logic
1222
1379
  - Comprehensive search command validation