pvw-cli 1.2.3__py3-none-any.whl → 1.2.5__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of pvw-cli might be problematic. Click here for more details.
- purviewcli/__init__.py +1 -1
- purviewcli/cli/lineage.py +4 -1
- purviewcli/cli/types.py +333 -0
- purviewcli/cli/unified_catalog.py +1329 -9
- purviewcli/client/_entity.py +74 -17
- purviewcli/client/_lineage.py +206 -44
- purviewcli/client/_types.py +31 -0
- purviewcli/client/_unified_catalog.py +646 -30
- purviewcli/client/endpoints.py +75 -0
- {pvw_cli-1.2.3.dist-info → pvw_cli-1.2.5.dist-info}/METADATA +232 -75
- {pvw_cli-1.2.3.dist-info → pvw_cli-1.2.5.dist-info}/RECORD +14 -14
- {pvw_cli-1.2.3.dist-info → pvw_cli-1.2.5.dist-info}/WHEEL +0 -0
- {pvw_cli-1.2.3.dist-info → pvw_cli-1.2.5.dist-info}/entry_points.txt +0 -0
- {pvw_cli-1.2.3.dist-info → pvw_cli-1.2.5.dist-info}/top_level.txt +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pvw-cli
|
|
3
|
-
Version: 1.2.
|
|
3
|
+
Version: 1.2.5
|
|
4
4
|
Summary: Microsoft Purview CLI with comprehensive automation capabilities
|
|
5
5
|
Author-email: AYOUB KEBAILI <keayoub@msn.com>
|
|
6
6
|
Maintainer-email: AYOUB KEBAILI <keayoub@msn.com>
|
|
@@ -56,26 +56,28 @@ Requires-Dist: pytest-asyncio>=0.20.0; extra == "test"
|
|
|
56
56
|
Requires-Dist: pytest-cov>=4.0.0; extra == "test"
|
|
57
57
|
Requires-Dist: requests-mock>=1.9.0; extra == "test"
|
|
58
58
|
|
|
59
|
-
# PURVIEW CLI v1.2.
|
|
60
|
-
|
|
61
|
-
> **LATEST UPDATE v1.2.
|
|
62
|
-
>
|
|
63
|
-
> -
|
|
64
|
-
> -
|
|
65
|
-
> -
|
|
66
|
-
> -
|
|
67
|
-
> -
|
|
68
|
-
> -
|
|
69
|
-
> -
|
|
70
|
-
> -
|
|
71
|
-
> -
|
|
72
|
-
> -
|
|
59
|
+
# PURVIEW CLI v1.2.5 - Microsoft Purview Automation & Data Governance
|
|
60
|
+
|
|
61
|
+
> **LATEST UPDATE v1.2.5 (October 2025):**
|
|
62
|
+
>
|
|
63
|
+
> - **[NEW]** Lineage CSV Import - Bulk import lineage relationships from CSV files with validation and sample generation
|
|
64
|
+
> - **[FIXED]** Search API Integration - Fixed `suggest` and `autocomplete` API payload format (HTTP 400 errors resolved)
|
|
65
|
+
> - **[ENHANCED]** Collection Display - Improved collection name detection in search results with proper fallback logic
|
|
66
|
+
> - **[OK]** All Search Commands - Comprehensive testing of query, browse, suggest, find-table operations
|
|
67
|
+
> - **[NEW]** Bulk Term Import/Export - Import multiple terms from CSV/JSON with dry-run support
|
|
68
|
+
> - **[NEW]** Bulk Delete Scripts - PowerShell and Python scripts for bulk term deletion
|
|
69
|
+
> - **[NEW]** Multiple Output Formats - `--output` flag supports table, json, and jsonc formats
|
|
70
|
+
> - **[NEW]** PowerShell Integration - Plain JSON output works with `ConvertFrom-Json`
|
|
71
|
+
> - **[OK]** Complete Data Product CRUD - Full update and delete support with smart partial updates
|
|
72
|
+
> - **[OK]** Health Monitoring API - Automated governance health checks and recommendations
|
|
73
|
+
> - **[OK]** Workflow Management - Approval workflows and business process automation
|
|
74
|
+
> - **[OK]** Complete Microsoft Purview Unified Catalog (UC) - Full governance domains, glossary terms, data products, OKRs, and CDEs
|
|
73
75
|
|
|
74
76
|
---
|
|
75
77
|
|
|
76
78
|
## What is PVW CLI?
|
|
77
79
|
|
|
78
|
-
**PVW CLI v1.2.
|
|
80
|
+
**PVW CLI v1.2.5** is a modern, full-featured command-line interface and Python library for Microsoft Purview. It enables automation and management of *all major Purview APIs* including:
|
|
79
81
|
|
|
80
82
|
- **Unified Catalog (UC) Management** - Complete governance domains, glossary terms, data products, OKRs, CDEs
|
|
81
83
|
- **Bulk Operations** - Import/export terms from CSV/JSON, bulk delete scripts with progress tracking
|
|
@@ -90,9 +92,18 @@ Requires-Dist: requests-mock>=1.9.0; extra == "test"
|
|
|
90
92
|
|
|
91
93
|
The CLI is designed for data engineers, stewards, architects, and platform teams to automate, scale, and enhance their Microsoft Purview experience.
|
|
92
94
|
|
|
95
|
+
### NEW: MCP Server for AI Assistants
|
|
96
|
+
|
|
97
|
+
**[NEW]** Model Context Protocol (MCP) server enables LLM-powered data governance workflows!
|
|
98
|
+
|
|
99
|
+
- Natural language interface to Purview catalog
|
|
100
|
+
- 20+ tools for AI assistants (Claude, Cline, etc.)
|
|
101
|
+
- Automate complex multi-step operations
|
|
102
|
+
- See `mcp/README.md` for setup instructions
|
|
103
|
+
|
|
93
104
|
---
|
|
94
105
|
|
|
95
|
-
## Getting Started
|
|
106
|
+
## Getting Started
|
|
96
107
|
|
|
97
108
|
Follow this short flow to get PVW CLI installed and running quickly.
|
|
98
109
|
|
|
@@ -144,8 +155,8 @@ Follow this short flow to get PVW CLI installed and running quickly.
|
|
|
144
155
|
|
|
145
156
|
3. Authenticate
|
|
146
157
|
|
|
147
|
-
|
|
148
|
-
|
|
158
|
+
- Run `az login` (recommended), or
|
|
159
|
+
- Provide Service Principal credentials via environment variables.
|
|
149
160
|
|
|
150
161
|
4. Try a few commands:
|
|
151
162
|
|
|
@@ -161,14 +172,15 @@ Follow this short flow to get PVW CLI installed and running quickly.
|
|
|
161
172
|
pvw uc --help
|
|
162
173
|
```
|
|
163
174
|
|
|
164
|
-
For more advanced usage, see the documentation in `doc/` or the project docs: https://pvw-cli.readthedocs.io
|
|
175
|
+
For more advanced usage, see the documentation in `doc/` or the project docs: <https://pvw-cli.readthedocs.io/>
|
|
165
176
|
|
|
166
177
|
---
|
|
167
178
|
|
|
168
179
|
## Overview
|
|
169
180
|
|
|
170
|
-
**PVW CLI v1.2.
|
|
181
|
+
**PVW CLI v1.2.5** is a modern command-line interface and Python library for Microsoft Purview, enabling:
|
|
171
182
|
|
|
183
|
+
- **MCP Server** - Natural language interface for AI assistants (Claude, Cline)
|
|
172
184
|
- Advanced data catalog search and discovery
|
|
173
185
|
- Bulk import/export of entities, glossary terms, and lineage
|
|
174
186
|
- Real-time monitoring and analytics
|
|
@@ -296,6 +308,7 @@ If you are signed in to Azure in Visual Studio or VS Code, `DefaultAzureCredenti
|
|
|
296
308
|
---
|
|
297
309
|
|
|
298
310
|
**Note:**
|
|
311
|
+
|
|
299
312
|
- The CLI will try all supported authentication methods in order. The first one that works will be used.
|
|
300
313
|
- For most automation and CI/CD scenarios, service principal authentication is recommended.
|
|
301
314
|
- For local development, Azure CLI authentication is easiest.
|
|
@@ -381,11 +394,13 @@ pvw uc dataproduct list --domain-id "abc-123" --output json
|
|
|
381
394
|
### Migration from Old --json Flag
|
|
382
395
|
|
|
383
396
|
**Old (deprecated):**
|
|
397
|
+
|
|
384
398
|
```bash
|
|
385
399
|
pvw uc term list --domain-id "abc-123" --json
|
|
386
400
|
```
|
|
387
401
|
|
|
388
402
|
**New (recommended):**
|
|
403
|
+
|
|
389
404
|
```bash
|
|
390
405
|
pvw uc term list --domain-id "abc-123" --output json # Plain JSON for scripting
|
|
391
406
|
pvw uc term list --domain-id "abc-123" --output jsonc # Colored JSON (old behavior)
|
|
@@ -400,21 +415,25 @@ Before using PVW CLI, you need to set three essential environment variables. Her
|
|
|
400
415
|
### 🔍 **How to Find Your Purview Values**
|
|
401
416
|
|
|
402
417
|
#### **1. PURVIEW_ACCOUNT_NAME**
|
|
418
|
+
|
|
403
419
|
- This is your Purview account name as it appears in Azure Portal
|
|
404
420
|
- Example: `kaydemopurview`
|
|
405
421
|
|
|
406
|
-
#### **2. PURVIEW_ACCOUNT_ID**
|
|
422
|
+
#### **2. PURVIEW_ACCOUNT_ID**
|
|
423
|
+
|
|
407
424
|
- This is the GUID that identifies your Purview account for Unified Catalog APIs
|
|
408
|
-
-
|
|
425
|
+
- **Important: For most Purview deployments, this is your Azure Tenant ID**
|
|
409
426
|
|
|
410
427
|
- **Method 1 - Get your Tenant ID (recommended):**
|
|
411
428
|
|
|
412
429
|
**Bash/Command Prompt:**
|
|
430
|
+
|
|
413
431
|
```bash
|
|
414
432
|
az account show --query tenantId -o tsv
|
|
415
433
|
```
|
|
416
434
|
|
|
417
435
|
**PowerShell:**
|
|
436
|
+
|
|
418
437
|
```powershell
|
|
419
438
|
az account show --query tenantId -o tsv
|
|
420
439
|
# Or store directly in environment variable:
|
|
@@ -422,9 +441,11 @@ Before using PVW CLI, you need to set three essential environment variables. Her
|
|
|
422
441
|
```
|
|
423
442
|
|
|
424
443
|
- **Method 2 - Azure CLI (extract from Atlas endpoint):**
|
|
444
|
+
|
|
425
445
|
```bash
|
|
426
446
|
az purview account show --name YOUR_ACCOUNT_NAME --resource-group YOUR_RG --query endpoints.catalog -o tsv
|
|
427
447
|
```
|
|
448
|
+
|
|
428
449
|
Extract the GUID from the URL (before `-api.purview-service.microsoft.com`)
|
|
429
450
|
|
|
430
451
|
- **Method 3 - Azure Portal:**
|
|
@@ -433,12 +454,14 @@ Before using PVW CLI, you need to set three essential environment variables. Her
|
|
|
433
454
|
3. Extract GUID from: `https://GUID-api.purview-service.microsoft.com/catalog`
|
|
434
455
|
|
|
435
456
|
#### **3. PURVIEW_RESOURCE_GROUP**
|
|
457
|
+
|
|
436
458
|
- The Azure resource group containing your Purview account
|
|
437
459
|
- Example: `fabric-artifacts`
|
|
438
460
|
|
|
439
461
|
### 📋 **Setting the Variables**
|
|
440
462
|
|
|
441
463
|
**Windows Command Prompt:**
|
|
464
|
+
|
|
442
465
|
```cmd
|
|
443
466
|
set PURVIEW_ACCOUNT_NAME=your-purview-account
|
|
444
467
|
set PURVIEW_ACCOUNT_ID=your-purview-account-id
|
|
@@ -446,6 +469,7 @@ set PURVIEW_RESOURCE_GROUP=your-resource-group
|
|
|
446
469
|
```
|
|
447
470
|
|
|
448
471
|
**Windows PowerShell:**
|
|
472
|
+
|
|
449
473
|
```powershell
|
|
450
474
|
$env:PURVIEW_ACCOUNT_NAME="your-purview-account"
|
|
451
475
|
$env:PURVIEW_ACCOUNT_ID="your-purview-account-id"
|
|
@@ -453,6 +477,7 @@ $env:PURVIEW_RESOURCE_GROUP="your-resource-group"
|
|
|
453
477
|
```
|
|
454
478
|
|
|
455
479
|
**Linux/macOS:**
|
|
480
|
+
|
|
456
481
|
```bash
|
|
457
482
|
export PURVIEW_ACCOUNT_NAME=your-purview-account
|
|
458
483
|
export PURVIEW_ACCOUNT_ID=your-purview-account-id
|
|
@@ -460,6 +485,7 @@ export PURVIEW_RESOURCE_GROUP=your-resource-group
|
|
|
460
485
|
```
|
|
461
486
|
|
|
462
487
|
**Permanent (Windows Command Prompt):**
|
|
488
|
+
|
|
463
489
|
```cmd
|
|
464
490
|
setx PURVIEW_ACCOUNT_NAME "your-purview-account"
|
|
465
491
|
setx PURVIEW_ACCOUNT_ID "your-purview-account-id"
|
|
@@ -467,17 +493,19 @@ setx PURVIEW_RESOURCE_GROUP "your-resource-group"
|
|
|
467
493
|
```
|
|
468
494
|
|
|
469
495
|
**Permanent (Windows PowerShell):**
|
|
496
|
+
|
|
470
497
|
```powershell
|
|
471
498
|
[Environment]::SetEnvironmentVariable("PURVIEW_ACCOUNT_NAME", "your-purview-account", "User")
|
|
472
499
|
[Environment]::SetEnvironmentVariable("PURVIEW_ACCOUNT_ID", "your-purview-account-id", "User")
|
|
473
500
|
[Environment]::SetEnvironmentVariable("PURVIEW_RESOURCE_GROUP", "your-resource-group", "User")
|
|
474
501
|
```
|
|
475
502
|
|
|
476
|
-
###
|
|
503
|
+
### **Debug Environment Issues**
|
|
477
504
|
|
|
478
505
|
If you experience issues with environment variables between different terminals, use these debug commands:
|
|
479
506
|
|
|
480
507
|
**Command Prompt/Bash:**
|
|
508
|
+
|
|
481
509
|
```bash
|
|
482
510
|
# Run this to check your current environment
|
|
483
511
|
python -c "
|
|
@@ -489,6 +517,7 @@ print('PURVIEW_RESOURCE_GROUP:', os.getenv('PURVIEW_RESOURCE_GROUP'))
|
|
|
489
517
|
```
|
|
490
518
|
|
|
491
519
|
**PowerShell:**
|
|
520
|
+
|
|
492
521
|
```powershell
|
|
493
522
|
# Check environment variables in PowerShell
|
|
494
523
|
python -c "
|
|
@@ -514,14 +543,15 @@ The PVW CLI provides advanced search using the latest Microsoft Purview Discover
|
|
|
514
543
|
- Use autocomplete and suggestion endpoints
|
|
515
544
|
- Perform faceted, time-based, and entity-type-specific queries
|
|
516
545
|
|
|
517
|
-
|
|
546
|
+
**v1.2.5 Improvements:**
|
|
547
|
+
|
|
518
548
|
- Fixed `suggest` and `autocomplete` API payload format (removed empty filter causing HTTP 400 errors)
|
|
519
549
|
- Enhanced collection display with robust type checking and fallback logic
|
|
520
550
|
- All search commands validated and working correctly (query, browse, suggest, find-table)
|
|
521
551
|
|
|
522
552
|
### CLI Usage Examples
|
|
523
553
|
|
|
524
|
-
####
|
|
554
|
+
#### **Multiple Output Formats**
|
|
525
555
|
|
|
526
556
|
```bash
|
|
527
557
|
# 1. Table Format (Default) - Quick overview
|
|
@@ -541,7 +571,7 @@ pvw search query --keywords="customer" --limit=5 --show-ids
|
|
|
541
571
|
# → Table format + entity GUIDs for copy/paste into update commands
|
|
542
572
|
```
|
|
543
573
|
|
|
544
|
-
####
|
|
574
|
+
#### **Search Operations**
|
|
545
575
|
|
|
546
576
|
```bash
|
|
547
577
|
# Basic search for assets with keyword 'customer'
|
|
@@ -559,12 +589,12 @@ pvw search autocomplete --keywords="ord" --limit=3
|
|
|
559
589
|
# Get search suggestions (fuzzy matching)
|
|
560
590
|
pvw search suggest --keywords="prod" --limit=2
|
|
561
591
|
|
|
562
|
-
|
|
592
|
+
**IMPORTANT - Command Line Quoting:**
|
|
563
593
|
```cmd
|
|
564
|
-
#
|
|
594
|
+
# [OK] CORRECT - Use quotes around keywords
|
|
565
595
|
pvw search query --keywords="customer" --limit=5
|
|
566
596
|
|
|
567
|
-
#
|
|
597
|
+
# [OK] CORRECT - For wildcard searches, use quotes
|
|
568
598
|
pvw search query --keywords="*" --limit=5
|
|
569
599
|
|
|
570
600
|
# ❌ WRONG - Don't use unquoted * (shell expands to file names)
|
|
@@ -586,7 +616,7 @@ pvw search query --keywords="audit" --createdAfter="2024-01-01" --limit=1
|
|
|
586
616
|
pvw search query --keywords="finance" --entityTypes="Files,Tables" --limit=2
|
|
587
617
|
```
|
|
588
618
|
|
|
589
|
-
####
|
|
619
|
+
#### **Usage Scenarios**
|
|
590
620
|
|
|
591
621
|
- **Daily browsing**: Use default table format for quick scans
|
|
592
622
|
- **Understanding assets**: Use `--detailed` for rich information panels
|
|
@@ -629,7 +659,7 @@ See [`doc/commands/unified-catalog.md`](doc/commands/unified-catalog.md) for com
|
|
|
629
659
|
|
|
630
660
|
### Quick UC Examples
|
|
631
661
|
|
|
632
|
-
####
|
|
662
|
+
#### **Governance Domains Management**
|
|
633
663
|
|
|
634
664
|
```bash
|
|
635
665
|
# List all governance domains
|
|
@@ -645,7 +675,7 @@ pvw uc domain get --domain-id "abc-123-def-456"
|
|
|
645
675
|
pvw uc domain update --domain-id "abc-123" --description "Updated financial governance"
|
|
646
676
|
```
|
|
647
677
|
|
|
648
|
-
####
|
|
678
|
+
#### **Glossary Terms in UC**
|
|
649
679
|
|
|
650
680
|
```bash
|
|
651
681
|
# List all terms in a domain
|
|
@@ -686,14 +716,16 @@ pvw uc term import-json --json-file "samples/json/term/uc_terms_bulk_example.jso
|
|
|
686
716
|
```
|
|
687
717
|
|
|
688
718
|
**Bulk Import Features:**
|
|
689
|
-
|
|
690
|
-
-
|
|
691
|
-
-
|
|
692
|
-
-
|
|
693
|
-
-
|
|
694
|
-
-
|
|
719
|
+
|
|
720
|
+
- [OK] Import from CSV or JSON files
|
|
721
|
+
- [OK] Dry-run mode to preview before importing
|
|
722
|
+
- [OK] Support for multiple owners (Entra ID Object IDs), acronyms, and resources
|
|
723
|
+
- [OK] Progress tracking with Rich console output
|
|
724
|
+
- [OK] Detailed error messages and summary reports
|
|
725
|
+
- [OK] Sequential POST requests (no native bulk endpoint available)
|
|
695
726
|
|
|
696
727
|
**CSV Format Example:**
|
|
728
|
+
|
|
697
729
|
```csv
|
|
698
730
|
name,description,status,acronym,owner_id,resource_name,resource_url
|
|
699
731
|
Customer Acquisition Cost,Cost to acquire new customer,Draft,CAC,<guid>,Metrics Guide,https://docs.example.com
|
|
@@ -701,6 +733,7 @@ Monthly Recurring Revenue,Predictable monthly revenue,Draft,MRR,<guid>,Finance D
|
|
|
701
733
|
```
|
|
702
734
|
|
|
703
735
|
**JSON Format Example:**
|
|
736
|
+
|
|
704
737
|
```json
|
|
705
738
|
{
|
|
706
739
|
"terms": [
|
|
@@ -718,9 +751,10 @@ Monthly Recurring Revenue,Predictable monthly revenue,Draft,MRR,<guid>,Finance D
|
|
|
718
751
|
```
|
|
719
752
|
|
|
720
753
|
**Important Notes:**
|
|
754
|
+
|
|
721
755
|
- ⚠️ **Owner IDs must be Entra ID Object IDs (GUIDs)**, not email addresses
|
|
722
756
|
- ⚠️ **Terms cannot be "Published" in unpublished domains** - use "Draft" status
|
|
723
|
-
-
|
|
757
|
+
- [OK] Sample files available: `samples/csv/uc_terms_bulk_example.csv`, `samples/json/term/uc_terms_bulk_example.json`
|
|
724
758
|
- 📖 Complete documentation: [`doc/commands/unified-catalog/term-bulk-import.md`](doc/commands/unified-catalog/term-bulk-import.md)
|
|
725
759
|
|
|
726
760
|
**🗑️ Bulk Delete (NEW)**
|
|
@@ -744,14 +778,15 @@ python scripts/delete_all_uc_terms_v2.py --domain-id "abc-123" --force
|
|
|
744
778
|
```
|
|
745
779
|
|
|
746
780
|
**Bulk Delete Features:**
|
|
747
|
-
- ✅ Interactive confirmation prompts (type "DELETE" to confirm)
|
|
748
|
-
- ✅ Beautiful progress display with colors
|
|
749
|
-
- ✅ Success/failure tracking per term
|
|
750
|
-
- ✅ Detailed summary reports
|
|
751
|
-
- ✅ Rate limiting (200ms delay between deletes)
|
|
752
|
-
- ✅ Graceful error handling and Ctrl+C support
|
|
753
781
|
|
|
754
|
-
|
|
782
|
+
- [OK] Interactive confirmation prompts (type "DELETE" to confirm)
|
|
783
|
+
- [OK] Beautiful progress display with colors
|
|
784
|
+
- [OK] Success/failure tracking per term
|
|
785
|
+
- [OK] Detailed summary reports
|
|
786
|
+
- [OK] Rate limiting (200ms delay between deletes)
|
|
787
|
+
- [OK] Graceful error handling and Ctrl+C support
|
|
788
|
+
|
|
789
|
+
#### **Data Products Management**
|
|
755
790
|
|
|
756
791
|
```bash
|
|
757
792
|
# List all data products in a domain
|
|
@@ -789,7 +824,7 @@ pvw uc dataproduct delete --product-id "prod-789"
|
|
|
789
824
|
pvw uc dataproduct delete --product-id "prod-789" --yes
|
|
790
825
|
```
|
|
791
826
|
|
|
792
|
-
####
|
|
827
|
+
#### **Objectives & Key Results (OKRs)**
|
|
793
828
|
|
|
794
829
|
```bash
|
|
795
830
|
# List objectives for a domain
|
|
@@ -810,7 +845,7 @@ pvw uc objective update \
|
|
|
810
845
|
--status "in-progress"
|
|
811
846
|
```
|
|
812
847
|
|
|
813
|
-
####
|
|
848
|
+
#### **Critical Data Elements (CDEs)**
|
|
814
849
|
|
|
815
850
|
```bash
|
|
816
851
|
# List critical data elements
|
|
@@ -831,7 +866,7 @@ pvw uc cde link \
|
|
|
831
866
|
--asset-id "ea3412c3-7387-4bc1-9923-11f6f6f60000"
|
|
832
867
|
```
|
|
833
868
|
|
|
834
|
-
####
|
|
869
|
+
#### **Health Monitoring (NEW)**
|
|
835
870
|
|
|
836
871
|
Monitor governance health and get automated recommendations to improve your data governance posture.
|
|
837
872
|
|
|
@@ -864,6 +899,7 @@ pvw uc health query --json
|
|
|
864
899
|
```
|
|
865
900
|
|
|
866
901
|
**Health Finding Types:**
|
|
902
|
+
|
|
867
903
|
- Missing glossary terms on data products (High)
|
|
868
904
|
- Data products without OKRs (Medium)
|
|
869
905
|
- Missing data quality scores (Medium)
|
|
@@ -871,7 +907,7 @@ pvw uc health query --json
|
|
|
871
907
|
- Description quality issues (Medium)
|
|
872
908
|
- Business domains without critical data entities (Medium)
|
|
873
909
|
|
|
874
|
-
####
|
|
910
|
+
#### **Workflow Management (NEW)**
|
|
875
911
|
|
|
876
912
|
Manage approval workflows and business process automation in Purview.
|
|
877
913
|
|
|
@@ -905,12 +941,13 @@ pvw workflow list --json
|
|
|
905
941
|
```
|
|
906
942
|
|
|
907
943
|
**Workflow Use Cases:**
|
|
944
|
+
|
|
908
945
|
- Data access request approvals
|
|
909
946
|
- Glossary term certification workflows
|
|
910
947
|
- Data product publishing approvals
|
|
911
948
|
- Classification review processes
|
|
912
949
|
|
|
913
|
-
####
|
|
950
|
+
#### **Integrated Workflow Example**
|
|
914
951
|
|
|
915
952
|
```bash
|
|
916
953
|
# 1. Discover assets to govern
|
|
@@ -935,7 +972,7 @@ pvw uc objective create --definition "Ensure 100% PII classification compliance"
|
|
|
935
972
|
|
|
936
973
|
PVW CLI provides comprehensive entity management capabilities for updating Purview assets like descriptions, classifications, and custom attributes.
|
|
937
974
|
|
|
938
|
-
###
|
|
975
|
+
### **Entity Update Examples**
|
|
939
976
|
|
|
940
977
|
#### **Update Asset Descriptions**
|
|
941
978
|
|
|
@@ -982,7 +1019,7 @@ pvw entity add-classification \
|
|
|
982
1019
|
--classification "MICROSOFT.PERSONAL.EMAIL"
|
|
983
1020
|
```
|
|
984
1021
|
|
|
985
|
-
###
|
|
1022
|
+
### **Discovery to Update Workflow**
|
|
986
1023
|
|
|
987
1024
|
```bash
|
|
988
1025
|
# 1. Find assets that need updates
|
|
@@ -1003,6 +1040,90 @@ pvw search query --keywords="FOUND_GUID" --detailed
|
|
|
1003
1040
|
|
|
1004
1041
|
---
|
|
1005
1042
|
|
|
1043
|
+
## Lineage CSV Import & Management
|
|
1044
|
+
|
|
1045
|
+
PVW CLI provides powerful lineage management capabilities including CSV-based bulk import for automating data lineage creation.
|
|
1046
|
+
|
|
1047
|
+
### **Lineage CSV Import**
|
|
1048
|
+
|
|
1049
|
+
Import lineage relationships from CSV files to automate the creation of data flow documentation in Microsoft Purview.
|
|
1050
|
+
|
|
1051
|
+
#### **CSV Format**
|
|
1052
|
+
|
|
1053
|
+
The CSV file must contain the following columns:
|
|
1054
|
+
|
|
1055
|
+
**Required columns:**
|
|
1056
|
+
|
|
1057
|
+
- `source_entity_guid` - GUID of the source entity
|
|
1058
|
+
- `target_entity_guid` - GUID of the target entity
|
|
1059
|
+
|
|
1060
|
+
**Optional columns:**
|
|
1061
|
+
|
|
1062
|
+
- `relationship_type` - Type of relationship (default: "Process")
|
|
1063
|
+
- `process_name` - Name of the transformation process
|
|
1064
|
+
- `description` - Description of the transformation
|
|
1065
|
+
- `confidence_score` - Confidence score (0-1)
|
|
1066
|
+
- `owner` - Process owner
|
|
1067
|
+
- `metadata` - Additional JSON metadata
|
|
1068
|
+
|
|
1069
|
+
**Example CSV:**
|
|
1070
|
+
|
|
1071
|
+
```csv
|
|
1072
|
+
source_entity_guid,target_entity_guid,relationship_type,process_name,description,confidence_score,owner,metadata
|
|
1073
|
+
dcfc99ed-c74d-49aa-bd0b-72f6f6f60000,1db9c650-acfb-4914-8bc5-1cf6f6f60000,Process,Transform_Product_Data,Transform product data for analytics,0.95,data-engineering,"{""tool"": ""Azure Data Factory""}"
|
|
1074
|
+
```
|
|
1075
|
+
|
|
1076
|
+
#### **Lineage Commands**
|
|
1077
|
+
|
|
1078
|
+
```bash
|
|
1079
|
+
# Validate CSV format before import (no API calls)
|
|
1080
|
+
pvw lineage validate lineage_data.csv
|
|
1081
|
+
|
|
1082
|
+
# Import lineage relationships from CSV
|
|
1083
|
+
pvw lineage import lineage_data.csv
|
|
1084
|
+
|
|
1085
|
+
# Generate sample CSV file with examples
|
|
1086
|
+
pvw lineage sample output.csv --num-samples 10 --template detailed
|
|
1087
|
+
|
|
1088
|
+
# View available CSV templates
|
|
1089
|
+
pvw lineage templates
|
|
1090
|
+
```
|
|
1091
|
+
|
|
1092
|
+
#### **Available Templates**
|
|
1093
|
+
|
|
1094
|
+
- **`basic`** - Minimal columns (source, target, process name)
|
|
1095
|
+
- **`detailed`** - All columns including metadata and confidence scores
|
|
1096
|
+
- **`qualified_names`** - Use qualified names instead of GUIDs
|
|
1097
|
+
|
|
1098
|
+
#### **Workflow Example**
|
|
1099
|
+
|
|
1100
|
+
```bash
|
|
1101
|
+
# 1. Find entity GUIDs using search
|
|
1102
|
+
pvw search find-table --name "Product" --schema "dbo" --id-only
|
|
1103
|
+
|
|
1104
|
+
# 2. Create CSV file with lineage relationships
|
|
1105
|
+
# (use the GUIDs from step 1)
|
|
1106
|
+
|
|
1107
|
+
# 3. Validate CSV format
|
|
1108
|
+
pvw lineage validate my_lineage.csv
|
|
1109
|
+
# Output: SUCCESS: Lineage validation passed (5 rows, 8 columns)
|
|
1110
|
+
|
|
1111
|
+
# 4. Import to Purview
|
|
1112
|
+
pvw lineage import my_lineage.csv
|
|
1113
|
+
# Output: SUCCESS: Lineage import completed successfully
|
|
1114
|
+
```
|
|
1115
|
+
|
|
1116
|
+
#### **Advanced Features**
|
|
1117
|
+
|
|
1118
|
+
- **GUID Validation**: Automatic validation of GUID format with helpful error messages
|
|
1119
|
+
- **Process Entity Creation**: Creates intermediate "Process" entities to link source→target relationships
|
|
1120
|
+
- **Metadata Support**: Add custom JSON metadata to each lineage relationship
|
|
1121
|
+
- **Dry-Run Validation**: Validate CSV format locally before making API calls
|
|
1122
|
+
|
|
1123
|
+
**For detailed documentation, see:** [`doc/guides/lineage-csv-import.md`](doc/guides/lineage-csv-import.md)
|
|
1124
|
+
|
|
1125
|
+
---
|
|
1126
|
+
|
|
1006
1127
|
## Data Product Management (Legacy)
|
|
1007
1128
|
|
|
1008
1129
|
PVW CLI also includes the original `data-product` command group for backward compatibility with traditional data product lifecycle management.
|
|
@@ -1032,15 +1153,18 @@ pvw data-product show-lineage --qualified-name="product.test.1"
|
|
|
1032
1153
|
## Core Features
|
|
1033
1154
|
|
|
1034
1155
|
- **Unified Catalog (UC)**: Complete modern data governance (NEW)
|
|
1156
|
+
|
|
1035
1157
|
```bash
|
|
1036
1158
|
# Manage governance domains, terms, data products, OKRs, CDEs
|
|
1037
1159
|
pvw uc domain list
|
|
1038
1160
|
pvw uc term create --name "Customer" --domain-id "abc-123"
|
|
1039
1161
|
pvw uc objective create --definition "Improve quality" --domain-id "abc-123"
|
|
1040
1162
|
```
|
|
1163
|
+
|
|
1041
1164
|
- **Discovery Query/Search**: Flexible, advanced search for all catalog assets
|
|
1042
1165
|
- **Entity Management**: Bulk import/export, update, and validation
|
|
1043
1166
|
- **Glossary Management**: Import/export terms, assign terms in bulk
|
|
1167
|
+
|
|
1044
1168
|
```bash
|
|
1045
1169
|
# List all terms in a glossary
|
|
1046
1170
|
pvw glossary list-terms --glossary-guid "your-glossary-guid"
|
|
@@ -1048,7 +1172,20 @@ pvw data-product show-lineage --qualified-name="product.test.1"
|
|
|
1048
1172
|
# Create and manage glossary terms
|
|
1049
1173
|
pvw glossary create-term --payload-file term.json
|
|
1050
1174
|
```
|
|
1051
|
-
|
|
1175
|
+
|
|
1176
|
+
- **Lineage Operations**: Lineage discovery, CSV-based bulk lineage import/export
|
|
1177
|
+
|
|
1178
|
+
```bash
|
|
1179
|
+
# Import lineage relationships from CSV
|
|
1180
|
+
pvw lineage import lineage_data.csv
|
|
1181
|
+
|
|
1182
|
+
# Validate CSV format before import
|
|
1183
|
+
pvw lineage validate lineage_data.csv
|
|
1184
|
+
|
|
1185
|
+
# Generate sample CSV file
|
|
1186
|
+
pvw lineage sample output.csv --num-samples 10
|
|
1187
|
+
```
|
|
1188
|
+
|
|
1052
1189
|
- **Monitoring & Analytics**: Real-time dashboards, metrics, and reporting
|
|
1053
1190
|
- **Plugin System**: Extensible with custom plugins
|
|
1054
1191
|
|
|
@@ -1060,14 +1197,14 @@ PVW CLI provides comprehensive automation for all major Microsoft Purview APIs,
|
|
|
1060
1197
|
|
|
1061
1198
|
### Supported API Groups
|
|
1062
1199
|
|
|
1063
|
-
- **Unified Catalog**: Complete governance domains, glossary terms, data products, OKRs, CDEs management
|
|
1064
|
-
- **Health Monitoring**: Automated governance health checks and recommendations
|
|
1065
|
-
- **Workflows**: Approval workflows and business process automation
|
|
1066
|
-
- **Data Map**: Full entity and lineage management
|
|
1067
|
-
- **Discovery**: Advanced search, browse, and query capabilities
|
|
1068
|
-
- **Collections**: Collection and account management
|
|
1069
|
-
- **Management**: Administrative operations
|
|
1070
|
-
- **Scan**: Data source scanning and configuration
|
|
1200
|
+
- **Unified Catalog**: Complete governance domains, glossary terms, data products, OKRs, CDEs management [OK]
|
|
1201
|
+
- **Health Monitoring**: Automated governance health checks and recommendations [OK] NEW
|
|
1202
|
+
- **Workflows**: Approval workflows and business process automation [OK] NEW
|
|
1203
|
+
- **Data Map**: Full entity and lineage management [OK]
|
|
1204
|
+
- **Discovery**: Advanced search, browse, and query capabilities [OK]
|
|
1205
|
+
- **Collections**: Collection and account management [OK]
|
|
1206
|
+
- **Management**: Administrative operations [OK]
|
|
1207
|
+
- **Scan**: Data source scanning and configuration [OK]
|
|
1071
1208
|
|
|
1072
1209
|
### API Version Support
|
|
1073
1210
|
|
|
@@ -1079,6 +1216,7 @@ PVW CLI provides comprehensive automation for all major Microsoft Purview APIs,
|
|
|
1079
1216
|
- Scan: **2018-12-01-preview**
|
|
1080
1217
|
|
|
1081
1218
|
For the latest API documentation and updates, see:
|
|
1219
|
+
|
|
1082
1220
|
- [Microsoft Purview REST API reference](https://learn.microsoft.com/en-us/rest/api/purview/)
|
|
1083
1221
|
- [Atlas 2.2 API documentation](https://learn.microsoft.com/en-us/purview/data-gov-api-atlas-2-2)
|
|
1084
1222
|
- [Azure Updates](https://azure.microsoft.com/updates/) for new releases
|
|
@@ -1094,9 +1232,18 @@ PVW CLI includes comprehensive sample files and scripts for bulk operations:
|
|
|
1094
1232
|
### Bulk Import Samples
|
|
1095
1233
|
|
|
1096
1234
|
- **CSV Samples:** `samples/csv/uc_terms_bulk_example.csv` (8 sample terms)
|
|
1097
|
-
- **JSON Samples:**
|
|
1235
|
+
- **JSON Samples:**
|
|
1098
1236
|
- `samples/json/term/uc_terms_bulk_example.json` (8 data management terms)
|
|
1099
1237
|
- `samples/json/term/uc_terms_sample.json` (8 business terms)
|
|
1238
|
+
- **Lineage CSV Samples:** `samples/csv/lineage_example.csv` - Multiple lineage relationships with metadata
|
|
1239
|
+
|
|
1240
|
+
### Lineage Documentation
|
|
1241
|
+
|
|
1242
|
+
- **Comprehensive Guide:** `doc/guides/lineage-csv-import.md` - Complete lineage CSV import documentation
|
|
1243
|
+
- CSV format specification with required/optional columns
|
|
1244
|
+
- Command examples for validate, import, sample, templates
|
|
1245
|
+
- Workflow recommendations and troubleshooting
|
|
1246
|
+
- Advanced scenarios with metadata and multiple transformations
|
|
1100
1247
|
|
|
1101
1248
|
### Bulk Delete Scripts
|
|
1102
1249
|
|
|
@@ -1137,27 +1284,31 @@ PVW CLI includes comprehensive sample files and scripts for bulk operations:
|
|
|
1137
1284
|
|
|
1138
1285
|
## Recent Updates (October 2025)
|
|
1139
1286
|
|
|
1140
|
-
###
|
|
1287
|
+
### Bulk Term Import/Export
|
|
1288
|
+
|
|
1141
1289
|
- Import multiple terms from CSV or JSON files
|
|
1142
1290
|
- Dry-run mode for validation before import
|
|
1143
1291
|
- Support for owners (Entra ID GUIDs), acronyms, resources
|
|
1144
1292
|
- Progress tracking and detailed error reporting
|
|
1145
1293
|
- 100% success rate in testing (8/8 terms)
|
|
1146
1294
|
|
|
1147
|
-
###
|
|
1295
|
+
### PowerShell & Scripting Integration
|
|
1296
|
+
|
|
1148
1297
|
- New `--output` parameter with table/json/jsonc formats
|
|
1149
1298
|
- Plain JSON works with PowerShell's `ConvertFrom-Json`
|
|
1150
1299
|
- Compatible with jq, Python json module, and other tools
|
|
1151
1300
|
- Migration from deprecated `--json` flag
|
|
1152
1301
|
|
|
1153
|
-
###
|
|
1302
|
+
### Bulk Delete Scripts
|
|
1303
|
+
|
|
1154
1304
|
- PowerShell script with interactive confirmation ("DELETE" to confirm)
|
|
1155
1305
|
- Python script with Rich progress bars
|
|
1156
1306
|
- Beautiful UI with colored output
|
|
1157
1307
|
- Success/failure tracking per term
|
|
1158
1308
|
- Rate limiting (200ms delay)
|
|
1159
1309
|
|
|
1160
|
-
###
|
|
1310
|
+
### Critical Fixes (v1.2.5)
|
|
1311
|
+
|
|
1161
1312
|
- **Search API Suggest/Autocomplete:** Fixed HTTP 400 errors by removing empty filter objects from payload
|
|
1162
1313
|
- **Collection Display:** Enhanced collection name detection with proper fallback logic (isinstance checks)
|
|
1163
1314
|
- **Owner ID Format:** Must use Entra ID Object IDs (GUIDs), not email addresses
|
|
@@ -1169,29 +1320,34 @@ PVW CLI includes comprehensive sample files and scripts for bulk operations:
|
|
|
1169
1320
|
|
|
1170
1321
|
## Key Features Summary
|
|
1171
1322
|
|
|
1172
|
-
###
|
|
1323
|
+
### **Unified Catalog (UC) - Complete Management**
|
|
1324
|
+
|
|
1173
1325
|
- Governance domains, glossary terms, data products
|
|
1174
1326
|
- Objectives & Key Results (OKRs), Critical Data Elements (CDEs)
|
|
1175
1327
|
- Health monitoring and workflow automation
|
|
1176
1328
|
- Full CRUD operations with smart partial updates
|
|
1177
1329
|
|
|
1178
|
-
###
|
|
1330
|
+
### **Bulk Operations**
|
|
1331
|
+
|
|
1179
1332
|
- CSV/JSON import with dry-run validation
|
|
1180
1333
|
- PowerShell and Python bulk delete scripts
|
|
1181
1334
|
- Progress tracking and error handling
|
|
1182
1335
|
- Sample files and templates included
|
|
1183
1336
|
|
|
1184
|
-
###
|
|
1337
|
+
### **Multiple Output Formats**
|
|
1338
|
+
|
|
1185
1339
|
- Table format for human viewing (default)
|
|
1186
1340
|
- Plain JSON for PowerShell/bash scripting
|
|
1187
1341
|
- Colored JSON for visual inspection
|
|
1188
1342
|
|
|
1189
|
-
###
|
|
1343
|
+
### **Automation & Integration**
|
|
1344
|
+
|
|
1190
1345
|
- Azure CLI, Service Principal, Managed Identity auth
|
|
1191
1346
|
- Works in local development, CI/CD, and production
|
|
1192
1347
|
- Compatible with PowerShell, bash, Python, jq
|
|
1193
1348
|
|
|
1194
|
-
###
|
|
1349
|
+
### **Comprehensive Documentation**
|
|
1350
|
+
|
|
1195
1351
|
- Complete API coverage documentation
|
|
1196
1352
|
- Jupyter notebook examples
|
|
1197
1353
|
- Troubleshooting guides
|
|
@@ -1214,9 +1370,10 @@ See [LICENSE](LICENSE) file for details.
|
|
|
1214
1370
|
|
|
1215
1371
|
---
|
|
1216
1372
|
|
|
1217
|
-
**PVW CLI v1.2.
|
|
1373
|
+
**PVW CLI v1.2.5 empowers data engineers, stewards, and architects to automate, scale, and enhance their Microsoft Purview experience with powerful command-line and programmatic capabilities.**
|
|
1374
|
+
|
|
1375
|
+
**Latest in v1.2.5:**
|
|
1218
1376
|
|
|
1219
|
-
**Latest in v1.2.3:**
|
|
1220
1377
|
- Fixed Search API suggest/autocomplete (HTTP 400 errors resolved)
|
|
1221
1378
|
- Enhanced collection display with robust fallback logic
|
|
1222
1379
|
- Comprehensive search command validation
|