pvw-cli 1.2.5__tar.gz → 1.3.4__tar.gz

{pvw_cli-1.2.5 → pvw_cli-1.3.4}/PKG-INFO

@@ -1,15 +1,15 @@
 Metadata-Version: 2.4
 Name: pvw-cli
-Version: 1.2.5
+Version: 1.3.4
 Summary: Microsoft Purview CLI with comprehensive automation capabilities
 Author-email: AYOUB KEBAILI <keayoub@msn.com>
 Maintainer-email: AYOUB KEBAILI <keayoub@msn.com>
 License-Expression: MIT
-Project-URL: Homepage, https://github.com/Keayoub/Purview_cli
-Project-URL: Documentation, https://github.com/Keayoub/Purview_cli/wiki
-Project-URL: Repository, https://github.com/Keayoub/Purview_cli.git
-Project-URL: Bug Tracker, https://github.com/Keayoub/Purview_cli/issues
-Project-URL: Source, https://github.com/Keayoub/Purview_cli
+Project-URL: Homepage, https://github.com/Keayoub/pvw-cli
+Project-URL: Documentation, https://github.com/Keayoub/pvw-cli/wiki
+Project-URL: Repository, https://github.com/Keayoub/pvw-cli.git
+Project-URL: Bug Tracker, https://github.com/Keayoub/pvw-cli/issues
+Project-URL: Source, https://github.com/Keayoub/pvw-cli
 Keywords: azure,purview,cli,data,catalog,governance,automation,pvw
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
@@ -56,37 +56,74 @@ Requires-Dist: pytest-asyncio>=0.20.0; extra == "test"
 Requires-Dist: pytest-cov>=4.0.0; extra == "test"
 Requires-Dist: requests-mock>=1.9.0; extra == "test"
 
-# PURVIEW CLI v1.2.5 - Microsoft Purview Automation & Data Governance
+# PURVIEW CLI v1.3.4 - Microsoft Purview Automation & Data Governance
 
-> **LATEST UPDATE v1.2.5 (October 2025):**
+[![Version](https://img.shields.io/badge/version-1.3.3-blue.svg)](https://github.com/Keayoub/pvw-cli/releases/tag/v1.3.4)
+[![API Coverage](https://img.shields.io/badge/UC%20API%20Coverage-86%25-green.svg)](https://github.com/Keayoub/pvw-cli)
+[![Lineage](https://img.shields.io/badge/Lineage-Enhanced-green.svg)](https://github.com/Keayoub/pvw-cli)
+[![Status](https://img.shields.io/badge/status-stable-success.svg)](https://github.com/Keayoub/pvw-cli)
+
+> **LATEST UPDATE v1.3.4 (November 3, 2025):**
+>
+> **🔗 Advanced Lineage Features & Column-Level Mapping**
+>
+> **New Lineage Capabilities:**
+> - **[NEW]** Column-level lineage with multi-target support (1 source → N targets)
+> - **[NEW]** Direct lineage (UI-style) - No visible Process entity
+> - **[NEW]** Dual-mode CSV import - Automatic detection of Process vs Direct lineage
+> - **[NEW]** Column mapping in direct relationships - Granular data flow tracking
+> - **[NEW]** Enhanced error handling with SSL retry strategies
+>
+> **New CLI Commands:**
+> ```bash
+> pvw lineage create-column   # Column-level lineage (Process-based)
+> pvw lineage create-direct   # Direct lineage (UI-style, no Process)
+> pvw lineage list-column     # List column lineages
+> pvw lineage delete-column   # Delete column lineage
+> ```
+>
+> **What's New:**
+> - ✅ Column mapping visible in Purview UI
+> - ✅ Compatible with manual UI lineage creation
+> - ✅ Type validation (prevent invalid lineage)
+> - ✅ Batch CSV import with 5 sample files
+> - ✅ Comprehensive documentation & examples
 >
-> - **[NEW]** Lineage CSV Import - Bulk import lineage relationships from CSV files with validation and sample generation
-> - **[FIXED]** Search API Integration - Fixed `suggest` and `autocomplete` API payload format (HTTP 400 errors resolved)
-> - **[ENHANCED]** Collection Display - Improved collection name detection in search results with proper fallback logic
-> - **[OK]** All Search Commands - Comprehensive testing of query, browse, suggest, find-table operations
-> - **[NEW]** Bulk Term Import/Export - Import multiple terms from CSV/JSON with dry-run support
-> - **[NEW]** Bulk Delete Scripts - PowerShell and Python scripts for bulk term deletion
-> - **[NEW]** Multiple Output Formats - `--output` flag supports table, json, and jsonc formats
-> - **[NEW]** PowerShell Integration - Plain JSON output works with `ConvertFrom-Json`
-> - **[OK]** Complete Data Product CRUD - Full update and delete support with smart partial updates
-> - **[OK]** Health Monitoring API - Automated governance health checks and recommendations
-> - **[OK]** Workflow Management - Approval workflows and business process automation
-> - **[OK]** Complete Microsoft Purview Unified Catalog (UC) - Full governance domains, glossary terms, data products, OKRs, and CDEs
+> **Previous Releases:**
+> - **v1.2.6** - Initial lineage improvements
+> - **v1.2.5** - 86% UC API Coverage (45/52 operations) with Relationships, Query, Policies APIs
+>
+> **[Full Release Notes v1.3.4](releases/v1.3.4.md)** | **[v1.2.5 Release Notes](releases/v1.2.5.md)** | **[Migration Guide](releases/v1.3.4.md#migration-guide)**
 
 ---
 
 ## What is PVW CLI?
 
-**PVW CLI v1.2.5** is a modern, full-featured command-line interface and Python library for Microsoft Purview. It enables automation and management of *all major Purview APIs* including:
+**PVW CLI v1.3.4** is a modern, full-featured command-line interface and Python library for Microsoft Purview. It enables automation and management of *all major Purview APIs* with **86% Unified Catalog API coverage** (45 of 52 operations).
+
+### Key Capabilities
+
+**Unified Catalog (UC) Management - 86% Complete**
+- Complete governance domains, glossary terms, data products, OKRs, CDEs
+- Relationships API - Link data products/CDEs/terms to entities and columns
+- Query APIs - Advanced OData filtering with multi-criteria search
+- Policy Management - Complete CRUD for governance and RBAC policies
+- Custom Metadata & Attributes - Extensible business metadata and attributes
 
-- **Unified Catalog (UC) Management** - Complete governance domains, glossary terms, data products, OKRs, CDEs
-- **Bulk Operations** - Import/export terms from CSV/JSON, bulk delete scripts with progress tracking
-- **Scriptable Output** - Multiple output formats (table, json, jsonc) for PowerShell/bash automation
+**Data Operations**
 - Entity management (create, update, bulk, import/export)
-- Glossary and term management
-- Lineage operations
+- Lineage operations with interactive creation and CSV import
+- Advanced search and discovery with fixed suggest/autocomplete
+- Business metadata with proper scope configuration
+
+**Automation & Scripting**
+- Bulk Operations - Import/export from CSV/JSON with dry-run support
+- Scriptable Output - Multiple formats (table, json, jsonc) for PowerShell/bash
+- 80+ usage examples and 15+ comprehensive guides
+- PowerShell integration with ConvertFrom-Json support
+
+**Legacy API Support**
 - Collection and account management
-- Advanced search and discovery
 - Data product management (legacy compatibility)
 - Classification, label, and status management
 
@@ -103,6 +140,114 @@ The CLI is designed for data engineers, stewards, architects, and platform teams
 
 ---
 
+## What's New in Recent Releases
+
+### v1.3.4 (November 3, 2025) - Advanced Lineage Features
+
+**Column-Level Lineage & Direct Relationships:**
+- Column-level lineage with multi-target support (1→N)
+- Direct lineage creation (UI-style, no visible Process)
+- Dual-mode CSV import with automatic type detection
+- Column mapping in direct relationships
+- Enhanced error handling with SSL retry strategies
+
+**New Commands:**
+```bash
+pvw lineage create-column   # Column lineage (Process-based)
+pvw lineage create-direct   # Direct lineage (UI-style)
+pvw lineage list-column     # List column lineages
+pvw lineage delete-column   # Delete lineage
+```
+
+**CSV Import Examples:**
+```csv
+# Direct lineage with column mapping
+source_entity_guid,target_entity_guid,relationship_type,column_mapping
+guid1,guid2,direct_lineage_dataset_dataset,"[{""Source"":""ID"",""Sink"":""ID""}]"
+```
+
+**[Full v1.3.4 Release Notes](releases/v1.3.4.md)**
+
+---
+
+### v1.2.5 (October 30, 2025) - 86% UC API Coverage
+
+Version 1.2.5 achieves **86% coverage** of the Microsoft Purview Unified Catalog API with **35 new operations**:
+
+| Resource Type | Coverage | Operations | Status |
+|--------------|----------|------------|---------|
+| **Business Domains** | 100% | 5/5 | ✅ Complete |
+| **Data Products** | 90% | 9/10 | ⚠️ 1 missing (Facets) |
+| **Glossary Terms** | 73% | 8/11 | ⚠️ 3 missing |
+| **Objectives & Key Results** | 92% | 11/12 | ⚠️ 1 missing |
+| **Critical Data Elements** | 90% | 9/10 | ⚠️ 1 missing |
+| **Policies** | 100% | 5/5 | ✅ Complete |
+| **Relationships** | 100% | 6/6 | ✅ Complete |
+| **Query** | 100% | 4/4 | ✅ Complete |
+| **Custom Metadata** | 100% | 5/5 | ✅ Complete |
+| **Custom Attributes** | 100% | 5/5 | ✅ Complete |
+| **TOTAL** | **86%** | **45/52** | 🎯 **A- Grade** |
+
+### 🚀 New APIs Implemented
+
+1. **Relationships API (6 operations)**
+   ```bash
+   # Link data product to entity
+   pvw uc dataproduct link-entity --id <dp-id> --entity-id <guid>
+   
+   # Link CDE to column
+   pvw uc cde link-entity --id <cde-id> --entity-id <guid> --column-qualified-name "..."
+   ```
+
+2. **Query APIs (4 operations)**
+   ```bash
+   # Advanced OData filtering
+   pvw uc term query --domain-ids "finance" --status Approved --top 50
+   
+   # Multi-criteria search with pagination
+   pvw uc dataproduct query --keywords "customer,revenue" --skip 10 --top 25
+   ```
+
+3. **Policy Management (5 operations)**
+   ```bash
+   # Complete policy CRUD
+   pvw uc policy list
+   pvw uc policy create --payload-file policy.json
+   pvw uc policy update --id <policy-id> --payload-file updated.json
+   ```
+
+4. **Custom Metadata (5 operations)**
+   ```bash
+   # Business metadata via Atlas API
+   pvw uc custom-metadata import --file metadata.csv
+   pvw uc custom-metadata add --guid <entity-guid> --name "BusinessConcept"
+   ```
+
+5. **Custom Attributes (5 operations)**
+   ```bash
+   # Extensible attribute definitions
+   pvw uc custom-attribute create --name "Department" --type String
+   pvw uc custom-attribute list
+   ```
+
+### 🔧 Major Fixes & Improvements
+
+- **Lineage Management Overhaul** - Complete rewrite with interactive PowerShell script, real entity support, and proper Process entities
+- **Search API Fixed** - Resolved HTTP 400 errors in suggest and autocomplete endpoints
+- **Business Metadata Scope** - Fixed Business Concept attributes on Glossary Terms with proper applicableEntityTypes
+- **Architecture Refactoring** - Unified endpoints dictionary, zero hardcoded URLs, complete consistency
+
+### 📚 Documentation (3,500+ lines)
+
+- 15+ new guides including relationships, query APIs, lineage creation, business metadata
+- 80+ usage examples across all new features
+- Complete API coverage gap analysis
+- Roadmap to 100% with implementation plans
+
+**[View Full Release Notes](releases/v1.3.4.md)**
+
+---
+
 ## Getting Started
 
 Follow this short flow to get PVW CLI installed and running quickly.
@@ -176,9 +321,100 @@ For more advanced usage, see the documentation in `doc/` or the project docs: <h
 
 ---
 
+## Quick Start Examples
+
+### v1.3.4 - Column-Level Lineage
+
+```bash
+# Create column-level lineage (Process-based)
+pvw lineage create-column \
+  --process-name "ETL_Sales_Transform" \
+  --source-table-guid "9ebbd583-4987-4d1b-b4f5-d8f6f6f60000" \
+  --target-table-guids "c88126ba-5fb5-4d33-bbe2-5ff6f6f60000" \
+  --column-mapping "ProductID:ProductID,Name:Name"
+
+# Create direct lineage (UI-style, no visible Process)
+pvw lineage create-direct \
+  --source-guid "9ebbd583-4987-4d1b-b4f5-d8f6f6f60000" \
+  --target-guid "c88126ba-5fb5-4d33-bbe2-5ff6f6f60000" \
+  --column-mapping "ProductID:ProductID,Name:Name,Amount:TotalAmount"
+
+# Import lineage from CSV (automatic type detection)
+pvw lineage import samples/csv/lineage_with_columns.csv
+
+# List column lineages
+pvw lineage list-column --format table
+
+# Delete column lineage
+pvw lineage delete-column --process-guid <guid> --force
+```
+
+### v1.2.5 - Relationships API
+
+```bash
+# Link data product to SQL table
+pvw uc dataproduct link-entity \
+  --id "dp-sales-2024" \
+  --entity-id "4fae348b-e960-42f7-834c-38f6f6f60000" \
+  --type-name "azure_sql_table"
+
+# Link CDE to specific column
+pvw uc cde link-entity \
+  --id "cde-customer-email" \
+  --entity-id "ea3412c3-7387-4bc1-9923-11f6f6f60000" \
+  --column-qualified-name "mssql://server/db/schema/table#EmailAddress"
+
+# List all linked entities
+pvw uc dataproduct list-entities --id "dp-sales-2024"
+```
+
+### v1.2.5 - Query APIs
+
+```bash
+# Query terms by domain and status
+pvw uc term query --domain-ids "finance,sales" --status Approved --top 50
+
+# Query data products with keywords
+pvw uc dataproduct query --keywords "customer,revenue" --skip 0 --top 25
+
+# Query CDEs by domain with pagination
+pvw uc cde query --domain-ids "compliance" --orderby "name" --top 100
+```
+
+### v1.2.5 - Policy Management
+
+```bash
+# List all policies
+pvw uc policy list
+
+# Create new policy
+pvw uc policy create --payload-file policy-rbac.json
+
+# Update existing policy
+pvw uc policy update --id "policy-001" --payload-file updated.json
+```
+
+### v1.2.5 - Custom Metadata
+
+```bash
+# Import business metadata from CSV
+pvw uc custom-metadata import --file business_concept.csv
+
+# Add metadata to entity
+pvw uc custom-metadata add \
+  --guid "4fae348b-e960-42f7-834c-38f6f6f60000" \
+  --name "BusinessConcept" \
+  --attributes '{"Department":"Sales"}'
+
+# Create custom attribute
+pvw uc custom-attribute create --name "Department" --type String
+```
+
+---
+
 ## Overview
 
-**PVW CLI v1.2.5** is a modern command-line interface and Python library for Microsoft Purview, enabling:
+**PVW CLI v1.3.4** is a modern command-line interface and Python library for Microsoft Purview, enabling:
 
 - **MCP Server** - Natural language interface for AI assistants (Claude, Cline)
 - Advanced data catalog search and discovery
@@ -543,7 +779,7 @@ The PVW CLI provides advanced search using the latest Microsoft Purview Discover
 - Use autocomplete and suggestion endpoints
 - Perform faceted, time-based, and entity-type-specific queries
 
-**v1.2.5 Improvements:**
+**v1.3.4 Improvements:**
 
 - Fixed `suggest` and `autocomplete` API payload format (removed empty filter causing HTTP 400 errors)
 - Enhanced collection display with robust type checking and fallback logic
@@ -1307,7 +1543,7 @@ PVW CLI includes comprehensive sample files and scripts for bulk operations:
 - Success/failure tracking per term
 - Rate limiting (200ms delay)
 
-### Critical Fixes (v1.2.5)
+### Critical Fixes (v1.3.4)
 
 - **Search API Suggest/Autocomplete:** Fixed HTTP 400 errors by removing empty filter objects from payload
 - **Collection Display:** Enhanced collection name detection with proper fallback logic (isinstance checks)
@@ -1370,9 +1606,9 @@ See [LICENSE](LICENSE) file for details.
 
 ---
 
-**PVW CLI v1.2.5 empowers data engineers, stewards, and architects to automate, scale, and enhance their Microsoft Purview experience with powerful command-line and programmatic capabilities.**
+**PVW CLI v1.3.4 empowers data engineers, stewards, and architects to automate, scale, and enhance their Microsoft Purview experience with powerful command-line and programmatic capabilities.**
 
-**Latest in v1.2.5:**
+**Latest in v1.3.4:**
 
 - Fixed Search API suggest/autocomplete (HTTP 400 errors resolved)
 - Enhanced collection display with robust fallback logic