dpdp-erasure-cli 1.0.11 → 1.1.0

package/README.md

@@ -1,31 +1,38 @@
-# Operator CLI Reference Guide (`dpdp-erasure-cli`)
+# dpdp-erasure-cli
 
 [![npm version](https://badge.fury.io/js/dpdp-erasure-cli.svg)](https://badge.fury.io/js/dpdp-erasure-cli)
 
-The **Operator CLI** (`dpdp-erasure-cli`) is the primary interface for data engineers, DevOps, and privacy operators to manage the DPDP Erasure Engine. 
+**The DPDP Erasure Engine CLI** is an automated, AI-assisted privacy toolkit that helps you securely discover, map, and cryptographically shred PII (Personally Identifiable Information) in your database. 
 
-It is used to introspect databases, classify PII, manage compliance manifests, sign them cryptographically, and simulate safe erasure operations locally.
+It acts as the control plane for the [DPDP Erasure Engine](https://github.com/devxdh/dpdp-erasure-engine), allowing Data Protection Officers (DPOs) and Software Engineers to effortlessly comply with global privacy laws (DPDP, GDPR, CCPA) without writing manual SQL deletion scripts.
+
+---
+
+## 🎯 What does it do?
+
+Manually deleting a user across dozens of microservice tables is dangerous and prone to failure. `dpdp-erasure-cli` solves this by:
+
+1. **Introspection & NLP Mapping:** Safely scans your live database (using `TABLESAMPLE` block sampling) to find hidden PII in text columns, JSON blobs, and orphaned tables.
+2. **DAG Compilation:** Maps your entire Foreign Key graph to figure out the exact order tables must be deleted to avoid database constraint violations.
+3. **Drafting a Manifest:** Automatically generates a `compliance.worker.yml` erasure plan that handles HMAC redaction vs. hard deletion.
+4. **Cryptographic Signatures:** Locks the manifest using Ed25519 signatures so production deletion rules cannot be silently altered.
+5. **Dry-Run Simulations:** Tests the erasure locally in a rolled-back PostgreSQL transaction to prove it works before you deploy.
 
 ---
 
 ## 🚀 Installation
 
-This CLI requires [Bun](https://bun.sh/) for native cryptographic bindings and high-performance execution. Ensure you have Bun installed, then install the package globally:
+This CLI relies on [Bun](https://bun.sh/) for native cryptographic bindings and high-performance execution.
 
 ```bash
 npm install -g dpdp-erasure-cli
 ```
 
-*Alternatively, if running from the monorepo root:*
-```bash
-bun run --cwd ./apps/worker cli <command>
-```
-
 ---
 
-## 🛠️ Interactive Console
+## 🛠️ Interactive Setup
 
-If you run the CLI without any arguments, it will launch an interactive wizard to guide you through the available operations:
+Don't want to memorize commands? Just run the CLI with no arguments to launch the interactive wizard:
 
 ```bash
 dpdp-cli
@@ -33,108 +40,56 @@ dpdp-cli
 
 ---
 
-## 📚 Core Commands & Configuration Guide
+## 📚 Quick Start Guide
+
+Setting up your database for privacy compliance follows this simple 5-step workflow:
 
-### 1. `introspect` (The Core Command)
-Safely analyze your database's Foreign Key (FK) Directed Acyclic Graph (DAG) offline and draft a comprehensive PII mapping manifest (`compliance.worker.yml`). This is the first step in setting up the engine.
+### 1. Introspect Your Database
+Safely analyze your schema to discover PII and draft the deletion manifest. The AI will even find logical links if you don't use strict Foreign Keys!
 
 ```bash
 dpdp-cli introspect \
-  --url postgres://user:pass@localhost:5432/app_db \
+  --url "postgres://user:pass@localhost:5432/app_db" \
   --root public.users \
   --schema public \
-  --output ./compliance.worker.yml.draft
+  --output ./compliance.worker.yml
 ```
 
-**Options:**
-*   `-u, --url <url>`: PostgreSQL Connection DSN.
-*   `-r, --root <table>`: The root table containing the user/subject identifier (e.g., `public.users`).
-*   `-s, --schema <schema>`: The target PostgreSQL schema (defaults to `public`).
-*   `-o, --output <path>`: Where to write the generated YAML draft (defaults to `compliance.worker.yml.draft`).
-*   `-d, --max-depth <depth>`: Limit for recursive Foreign Key traversal (default: `32`).
-*   `--sample-percent <percent>`: Percentage of data to sample using `TABLESAMPLE` for PII detection (default: `1`).
-*   `--threshold <score>`: Confidence score required to flag a column as PII (default: `0.75`).
-*   `--report <path>`: Write a readable Markdown report of the findings.
-
----
-
-### 2. `scan` (Quick PII Check)
-A lightweight, metadata-only schema scan that looks for potential PII columns based purely on naming conventions, without the heavy block sampling used by `introspect`.
-
-```bash
-dpdp-cli scan --url "postgres://user:pass@localhost:5432/app_db" --schema public
-```
-
----
-
-### 3. `keygen` (Security Provisioning)
-Provisions secure Ed25519 cryptographic keys required to sign your configuration manifest.
+### 2. Review and Attest
+Open the generated `compliance.worker.yml`. Review the `targets` and `join` conditions. Once you are confident, sign off by updating the `legal_attestation` block.
 
+### 3. Generate Security Keys
+Create a private/public keypair to securely sign your manifest for production environments.
 ```bash
 dpdp-cli keygen
 ```
-*This generates a private key file (e.g., `worker.pkcs8.key`) and a public key.*
-
----
-
-### 4. `sign` (Cryptographic Manifest Lock)
-To prevent unauthorized changes to data erasure rules in production, the manifest must be cryptographically signed by a Data Protection Officer (DPO) or Lead Engineer.
 
+### 4. Cryptographically Sign the Manifest
+Lock down the rules to prevent unauthorized changes in your CI/CD pipeline.
 ```bash
 dpdp-cli sign --config ./compliance.worker.yml --key ./worker.pkcs8.key
 ```
-*This generates a detached signature file (e.g., `compliance.worker.yml.sig`). The worker will fail to boot if this signature does not match the manifest.*
-
----
-
-### 5. `check-integrity` & `verify-schema` (CI/CD Gates)
-These commands are designed for CI/CD pipelines to ensure the live database schema matches the legal attestation hash stored in the signed manifest.
 
+### 5. Simulate an Erasure (Dry-Run)
+Test the erasure on a specific user. This command runs entirely within an isolated transaction that is automatically rolled back, so it is 100% safe.
 ```bash
-# Verify the compiled DAG and live schema hash
-dpdp-cli check-integrity --url "postgres://.../app_db" --config ./compliance.worker.yml
-
-# Check only the live schema hash against the legal attestation
-dpdp-cli verify-schema --url "postgres://.../app_db" --config ./compliance.worker.yml
+dpdp-cli dry-run --id "user_12345" --url "postgres://user:pass@localhost:5432/app_db" --config ./compliance.worker.yml
 ```
-*If a developer adds a new column to the database without updating and re-signing the manifest, these commands will exit with a non-zero status.*
 
 ---
 
-### 6. `dry-run` (Safe Erasure Simulation)
-Simulates a PII vault and redaction operation for a specific user. It runs inside an isolated transaction that is automatically rolled back.
+## 🔒 CI/CD Integrity Checks
 
-```bash
-dpdp-cli dry-run --id "user_12345" --url "postgres://.../app_db" --config ./compliance.worker.yml
-```
-*This is the recommended safety check to help ensure your configuration captures related PII without breaking foreign keys.*
-
----
-
-### 7. `graph` (Dependency Visualization)
-Visualizes the recursive table dependencies (FK DAG) for a specific root table, helping you understand how data cascades down from a user.
+You can use the CLI in your GitHub Actions or GitLab CI to fail builds if a developer modifies the database schema without updating the signed compliance manifest:
 
 ```bash
-dpdp-cli graph --table public.users --url "postgres://.../app_db"
+dpdp-cli check-integrity --url "postgres://..." --config ./compliance.worker.yml
 ```
 
 ---
 
-## ⚙️ Standard Workflow Example
-
-Setting up the engine generally follows this workflow:
+## 📖 Deep Dive
 
-1.  **Introspect** the database to generate a draft manifest:
-    `dpdp-cli introspect -u postgres://... -r public.users -s public -o compliance.worker.yml`
-2.  **Review & Tweak** the `compliance.worker.yml` manually (fix false positives, add missing logical links, select masking actions like `HMAC` or `SET NULL`).
-3.  **Generate Keys** for signing:
-    `dpdp-cli keygen`
-4.  **Sign** the finalized manifest:
-    `dpdp-cli sign -c compliance.worker.yml -k worker.pkcs8.key`
-5.  **Dry-Run** an erasure to verify it behaves as expected:
-    `dpdp-cli dry-run -i "test_user_id" -u postgres://... -c compliance.worker.yml`
-6.  **Deploy** the signed manifest and the detached `.sig` file to your production Worker.
-
----
+Want to understand the cryptographic shredding architecture under the hood? Read our full documentation at the main repository:
 
-For architectural details and deep-dives into how the cryptographic shredding works, refer to the [Main Documentation](https://github.com/devxdh/dpdp-erasure-engine/tree/main/apps/docs).
+**[DPDP Erasure Engine GitHub Repository](https://github.com/devxdh/dpdp-erasure-engine)**

package/compliance.worker.yml

@@ -0,0 +1,164 @@
+# AUTO-GENERATED BY INTROSPECTOR
+# REVIEW REQUIRED: DPO must validate every table, join condition, and PII column before production use.
+# Generated At: 2026-06-12T05:19:07.235Z
+
+legal_attestation:
+  dpo_identifier: PENDING_REVIEW
+  configuration_version: introspector-draft
+  legal_review_date: PENDING_REVIEW
+  schema_hash: ea9e816d30fcee6bd4f322f1fb769e853c2e7ee5b19263aaa54f5ef80189212c
+  generated_by: compliance-introspector-v1
+  acknowledgment: PENDING_REVIEW
+
+legal_disclaimer:
+  text: "Auto-generated by Compliance Worker. The DPO/Developer is responsible for verifying all logical links and PII mappings."
+
+rules:
+  - id: dpdp_standard
+    root_table: public.users
+    max_depth: 32
+    targets:
+      - table: public.users
+        # Introspector Confidence: 0.950 (email)
+        # Introspector Confidence: 0.920 (indian_mobile)
+        pii_columns: [email, phone_number]
+      - table: public.kyc_documents
+        parent: public.users
+        join: "public.users.id = public.kyc_documents.user_id"
+        parent_columns: [id]
+        child_columns: [user_id]
+        pii_columns: []
+      - table: public.orders
+        parent: public.users
+        join: "public.users.id = public.orders.user_id"
+        parent_columns: [id]
+        child_columns: [user_id]
+        pii_columns: []
+      - table: public.support_tickets
+        parent: public.users
+        join: "public.users.id = public.support_tickets.user_id"
+        parent_columns: [id]
+        child_columns: [user_id]
+        # Introspector Confidence: 0.900 (indian_mobile)
+        pii_columns: [description]
+        primary_key_columns: [id]
+        action: redact
+        mutation_rules:
+          description: HMAC
+      - table: public.user_addresses
+        parent: public.users
+        join: "public.users.id = public.user_addresses.user_id"
+        parent_columns: [id]
+        child_columns: [user_id]
+        # Introspector Confidence: 0.780 (indian_pin_code)
+        pii_columns: [pincode]
+        primary_key_columns: [id]
+        action: redact
+        mutation_rules:
+          pincode: HMAC
+      - table: public.user_devices
+        parent: public.users
+        join: "public.users.id = public.user_devices.user_id"
+        parent_columns: [id]
+        child_columns: [user_id]
+        # Introspector Confidence: 0.820 (metadata)
+        pii_columns: [last_ip_address]
+        primary_key_columns: [id]
+        action: redact
+        mutation_rules:
+          last_ip_address: HMAC
+      - table: public.user_preferences
+        parent: public.users
+        join: "public.users.id = public.user_preferences.user_id"
+        parent_columns: [id]
+        child_columns: [user_id]
+        pii_columns: []
+      - table: public.order_items
+        parent: public.orders
+        join: "public.orders.id = public.order_items.order_id"
+        parent_columns: [id]
+        child_columns: [order_id]
+        pii_columns: []
+      - table: public.payments
+        parent: public.orders
+        join: "public.orders.id = public.payments.order_id"
+        parent_columns: [id]
+        child_columns: [order_id]
+        pii_columns: []
+      - table: public.ticket_messages
+        parent: public.support_tickets
+        join: "public.support_tickets.id = public.ticket_messages.ticket_id"
+        parent_columns: [id]
+        child_columns: [ticket_id]
+        # Introspector Confidence: 0.900 (indian_mobile)
+        pii_columns: [message_body]
+        primary_key_columns: [id]
+        action: redact
+        mutation_rules:
+          message_body: HMAC
+      - table: public.abandoned_carts
+        parent: public.users
+        join: "LOGICAL_LINK (customer_id)"
+        parent_columns: [customer_id]
+        child_columns: [customer_id]
+        pii_columns: []
+      - table: public.audit_logs
+        parent: public.users
+        join: "LOGICAL_LINK (actor_id)"
+        parent_columns: [actor_id]
+        child_columns: [actor_id]
+        # Introspector Confidence: 0.820 (ipv4)
+        pii_columns: [ip_address]
+        primary_key_columns: [id]
+        action: redact
+        mutation_rules:
+          ip_address: HMAC
+      - table: public.legacy_crm_notes
+        parent: public.users
+        join: "LOGICAL_LINK (client_id)"
+        parent_columns: [client_id]
+        child_columns: [client_id]
+        # Introspector Confidence: 0.950 (email, indian_mobile)
+        pii_columns: [agent_notes]
+        primary_key_columns: [id]
+        action: redact
+        mutation_rules:
+          agent_notes: HMAC
+      - table: public.marketing_campaign_clicks
+        parent: public.users
+        join: "LOGICAL_LINK (target_email)"
+        parent_columns: [target_email]
+        child_columns: [target_email]
+        # Introspector Confidence: 0.950 (email)
+        pii_columns: [target_email]
+        primary_key_columns: [id]
+        action: redact
+        mutation_rules:
+          target_email: HMAC
+      - table: public.third_party_telemetry
+        parent: public.users
+        join: "LOGICAL_LINK (user_uuid)"
+        parent_columns: [user_uuid]
+        child_columns: [user_uuid]
+        pii_columns: []
+
+# [Potential Logical Link] public.users.customer_id <-> public.abandoned_carts.customer_id - Table exposes customer_id which conceptually maps to the root entity.
+# [Potential Logical Link] public.users.actor_id <-> public.audit_logs.actor_id - Table exposes actor_id which conceptually maps to the root entity.
+# [Potential Logical Link] public.kyc_documents.user_id <-> public.orders.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.kyc_documents.user_id <-> public.support_tickets.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.kyc_documents.user_id <-> public.user_addresses.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.kyc_documents.user_id <-> public.user_devices.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.kyc_documents.user_id <-> public.user_preferences.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.orders.user_id <-> public.support_tickets.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.orders.user_id <-> public.user_addresses.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.orders.user_id <-> public.user_devices.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.orders.user_id <-> public.user_preferences.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.support_tickets.user_id <-> public.user_addresses.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.support_tickets.user_id <-> public.user_devices.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.support_tickets.user_id <-> public.user_preferences.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.user_addresses.user_id <-> public.user_devices.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.user_addresses.user_id <-> public.user_preferences.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.user_devices.user_id <-> public.user_preferences.user_id - Both tables expose user_id but no physical foreign key was found.
+# [Potential Logical Link] public.users.client_id <-> public.legacy_crm_notes.client_id - Table exposes client_id which conceptually maps to the root entity.
+# [Potential Logical Link] public.users.target_email <-> public.marketing_campaign_clicks.target_email - Table exposes target_email which conceptually maps to the root entity.
+# [Potential Logical Link] public.users.user_uuid <-> public.third_party_telemetry.user_uuid - Table exposes user_uuid which conceptually maps to the root entity.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dpdp-erasure-cli",
-  "version": "1.0.11",
+  "version": "1.1.0",
   "license": "Apache-2.0",
   "keywords": [
     "dpdp",
@@ -54,4 +54,4 @@
     "postgres": "^3.4.9",
     "zod": "^4.4.2"
   }
-}
+}

package/report.json

@@ -0,0 +1,370 @@
+{
+  "summary": {
+    "rootTable": "public.users",
+    "generatedAt": "2026-06-12T05:19:07.235Z",
+    "schemaHash": "ea9e816d30fcee6bd4f322f1fb769e853c2e7ee5b19263aaa54f5ef80189212c",
+    "targetCount": 15,
+    "tablesWithPii": 8,
+    "piiColumnCount": 9,
+    "highConfidenceCount": 6,
+    "reviewRequiredCount": 3,
+    "potentialLogicalLinkCount": 20
+  },
+  "findings": [
+    {
+      "table": "public.legacy_crm_notes",
+      "column": "agent_notes",
+      "dataType": "text",
+      "confidence": 0.95,
+      "metadataScore": 0,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "email",
+        "indian_mobile"
+      ]
+    },
+    {
+      "table": "public.marketing_campaign_clicks",
+      "column": "target_email",
+      "dataType": "character varying",
+      "confidence": 0.95,
+      "metadataScore": 0.92,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "email"
+      ]
+    },
+    {
+      "table": "public.users",
+      "column": "email",
+      "dataType": "character varying",
+      "confidence": 0.95,
+      "metadataScore": 0.92,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "email"
+      ]
+    },
+    {
+      "table": "public.users",
+      "column": "phone_number",
+      "dataType": "character varying",
+      "confidence": 0.92,
+      "metadataScore": 0.92,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "indian_mobile"
+      ]
+    },
+    {
+      "table": "public.support_tickets",
+      "column": "description",
+      "dataType": "text",
+      "confidence": 0.9,
+      "metadataScore": 0,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "indian_mobile"
+      ]
+    },
+    {
+      "table": "public.ticket_messages",
+      "column": "message_body",
+      "dataType": "text",
+      "confidence": 0.9,
+      "metadataScore": 0,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "indian_mobile"
+      ]
+    },
+    {
+      "table": "public.audit_logs",
+      "column": "ip_address",
+      "dataType": "inet",
+      "confidence": 0.82,
+      "metadataScore": 0.82,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "ipv4"
+      ]
+    },
+    {
+      "table": "public.user_devices",
+      "column": "last_ip_address",
+      "dataType": "inet",
+      "confidence": 0.82,
+      "metadataScore": 0.82,
+      "contentMatchRatio": 0,
+      "sampleSize": 0,
+      "matchedSignatures": []
+    },
+    {
+      "table": "public.user_addresses",
+      "column": "pincode",
+      "dataType": "character varying",
+      "confidence": 0.78,
+      "metadataScore": 0.62,
+      "contentMatchRatio": 1,
+      "sampleSize": 50,
+      "matchedSignatures": [
+        "indian_pin_code"
+      ]
+    }
+  ],
+  "potentialLogicalLinks": [
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "users"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "abandoned_carts"
+      },
+      "column": "customer_id",
+      "reason": "Table exposes customer_id which conceptually maps to the root entity."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "users"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "audit_logs"
+      },
+      "column": "actor_id",
+      "reason": "Table exposes actor_id which conceptually maps to the root entity."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "kyc_documents"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "orders"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "kyc_documents"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "support_tickets"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "kyc_documents"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_addresses"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "kyc_documents"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_devices"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "kyc_documents"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_preferences"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "orders"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "support_tickets"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "orders"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_addresses"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "orders"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_devices"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "orders"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_preferences"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "support_tickets"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_addresses"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "support_tickets"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_devices"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "support_tickets"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_preferences"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "user_addresses"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_devices"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "user_addresses"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_preferences"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "user_devices"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "user_preferences"
+      },
+      "column": "user_id",
+      "reason": "Both tables expose user_id but no physical foreign key was found."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "users"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "legacy_crm_notes"
+      },
+      "column": "client_id",
+      "reason": "Table exposes client_id which conceptually maps to the root entity."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "users"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "marketing_campaign_clicks"
+      },
+      "column": "target_email",
+      "reason": "Table exposes target_email which conceptually maps to the root entity."
+    },
+    {
+      "sourceTable": {
+        "schema": "public",
+        "table": "users"
+      },
+      "targetTable": {
+        "schema": "public",
+        "table": "third_party_telemetry"
+      },
+      "column": "user_uuid",
+      "reason": "Table exposes user_uuid which conceptually maps to the root entity."
+    }
+  ],
+  "nextSteps": [
+    "Review every PII column and potential logical link with the application owner.",
+    "Copy reviewed targets into compliance.worker.yml and complete legal_attestation.",
+    "Run compliance-worker check-integrity before allowing live worker boot.",
+    "Sign the reviewed manifest with compliance-worker sign after DPO approval."
+  ]
+}

package/report.md

@@ -0,0 +1,57 @@
+# Compliance Introspector Report
+
+## Summary
+
+- Root table: `public.users`
+- Generated at: `2026-06-12T05:19:07.235Z`
+- Schema hash: `ea9e816d30fcee6bd4f322f1fb769e853c2e7ee5b19263aaa54f5ef80189212c`
+- DAG targets: 15
+- Tables with PII: 8
+- PII columns: 9
+- High-confidence findings: 6
+- Review-required findings: 3
+- Potential logical links: 20
+
+## PII Findings
+
+| Table | Column | Type | Confidence | Metadata | Content | Signatures |
+| --- | --- | --- | ---: | ---: | ---: | --- |
+| `public.legacy_crm_notes` `agent_notes` `text` 0.950 0.000 1.000 email, indian_mobile |
+| `public.marketing_campaign_clicks` `target_email` `character varying` 0.950 0.920 1.000 email |
+| `public.users` `email` `character varying` 0.950 0.920 1.000 email |
+| `public.users` `phone_number` `character varying` 0.920 0.920 1.000 indian_mobile |
+| `public.support_tickets` `description` `text` 0.900 0.000 1.000 indian_mobile |
+| `public.ticket_messages` `message_body` `text` 0.900 0.000 1.000 indian_mobile |
+| `public.audit_logs` `ip_address` `inet` 0.820 0.820 1.000 ipv4 |
+| `public.user_devices` `last_ip_address` `inet` 0.820 0.820 0.000 metadata |
+| `public.user_addresses` `pincode` `character varying` 0.780 0.620 1.000 indian_pin_code |
+
+## Potential Logical Links
+
+- `public.users.customer_id` <-> `public.abandoned_carts.customer_id`: Table exposes customer_id which conceptually maps to the root entity.
+- `public.users.actor_id` <-> `public.audit_logs.actor_id`: Table exposes actor_id which conceptually maps to the root entity.
+- `public.kyc_documents.user_id` <-> `public.orders.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.kyc_documents.user_id` <-> `public.support_tickets.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.kyc_documents.user_id` <-> `public.user_addresses.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.kyc_documents.user_id` <-> `public.user_devices.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.kyc_documents.user_id` <-> `public.user_preferences.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.orders.user_id` <-> `public.support_tickets.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.orders.user_id` <-> `public.user_addresses.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.orders.user_id` <-> `public.user_devices.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.orders.user_id` <-> `public.user_preferences.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.support_tickets.user_id` <-> `public.user_addresses.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.support_tickets.user_id` <-> `public.user_devices.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.support_tickets.user_id` <-> `public.user_preferences.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.user_addresses.user_id` <-> `public.user_devices.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.user_addresses.user_id` <-> `public.user_preferences.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.user_devices.user_id` <-> `public.user_preferences.user_id`: Both tables expose user_id but no physical foreign key was found.
+- `public.users.client_id` <-> `public.legacy_crm_notes.client_id`: Table exposes client_id which conceptually maps to the root entity.
+- `public.users.target_email` <-> `public.marketing_campaign_clicks.target_email`: Table exposes target_email which conceptually maps to the root entity.
+- `public.users.user_uuid` <-> `public.third_party_telemetry.user_uuid`: Table exposes user_uuid which conceptually maps to the root entity.
+
+## Next Steps
+
+- Review every PII column and potential logical link with the application owner.
+- Copy reviewed targets into compliance.worker.yml and complete legal_attestation.
+- Run compliance-worker check-integrity before allowing live worker boot.
+- Sign the reviewed manifest with compliance-worker sign after DPO approval.

package/src/modules/engine/vault/satellite-mutation.ts

@@ -52,9 +52,9 @@ async function hardDeleteSatelliteRows(
 
   while (true) {
     const tenantFilter = tenantId ? tx` AND tenant_id = ${tenantId}` : tx``;
-    const deletedRows = await tx<{ id: string | number }[]>`
+    const deletedRows = await tx<{ ctid: string }[]>`
       WITH batch AS (
-        SELECT id
+        SELECT ctid
         FROM ${tx(appSchema)}.${tx(tableName)}
         WHERE ${tx(lookupColumn)} = ${lookupValue}
         ${tenantFilter}
@@ -62,8 +62,8 @@ async function hardDeleteSatelliteRows(
         FOR UPDATE SKIP LOCKED
       )
       DELETE FROM ${tx(appSchema)}.${tx(tableName)}
-      WHERE id IN (SELECT id FROM batch)
-      RETURNING id
+      WHERE ctid IN (SELECT ctid FROM batch)
+      RETURNING ctid
     `;
 
     if (deletedRows.length === 0) {

package/src/modules/engine/vault/satellite.ts

@@ -3,7 +3,7 @@ import type { Tsql } from "@/types";
 import { assertIdentifier } from "@/utils";
 
 interface SatelliteRowId {
-  id: number;
+  ctid: string;
 }
 
 async function yieldWorkerEventLoop(): Promise<void> {
@@ -78,7 +78,7 @@ export async function redactSatelliteTable(
     const tenantFilter = tenantId ? tx` AND tenant_id = ${tenantId}` : tx``;
     const updatedRows = await tx<SatelliteRowId[]>`
       WITH batch AS (
-        SELECT id
+        SELECT ctid
         FROM ${tx(schema)}.${tx(table)}
         WHERE ${tx(safeLookupColumn)} = ${lookupValue}
         ${tenantFilter}
@@ -87,8 +87,8 @@ export async function redactSatelliteTable(
       )
       UPDATE ${tx(schema)}.${tx(table)}
       SET ${tx(safeLookupColumn)} = ${newHmacValue}
-      WHERE id IN (SELECT id FROM batch)
-      RETURNING id
+      WHERE ctid IN (SELECT ctid FROM batch)
+      RETURNING ctid
     `;
 
     if (updatedRows.length === 0) {

package/src/modules/introspector/classifier.ts

@@ -137,6 +137,7 @@ const METADATA_PATTERNS: Array<{ pattern: RegExp; score: number }> = [
   { pattern: /(^|_)(driving_license|driving_licence|license_number|licence_number|dl_number|dl_no)($|_)/i, score: MEDIUM_METADATA_SCORE },
   { pattern: /(^|_)(address|street|postal_code|zip_code|pin_code|pincode)($|_)/i, score: WEAK_METADATA_SCORE },
   { pattern: /(^|_)(device_fingerprint|device_id|advertising_id|gaid|idfa)($|_)/i, score: WEAK_METADATA_SCORE },
+  { pattern: /(^|_)(document_number|identity_number|id_number)($|_)/i, score: WEAK_METADATA_SCORE },
 ];
 
 function qualifiedKey(table: QualifiedTable): string {
@@ -336,12 +337,23 @@ function classifyLeafDetailed(value: string, columnName: string = ""): ContentSi
   const bytes = textEncoder.encode(value.trim());
   try {
     const normalized = textDecoder.decode(bytes).trim();
-    return CONTENT_SIGNATURES
-      .filter((signature) =>
-        signatureHasMetadataSupport(signature, columnName) &&
-        signature.pattern.test(normalized) &&
-        (!signature.validate || signature.validate(normalized))
-      );
+    // Split into tokens and strip leading/trailing punctuation so regexes can match substrings
+    const tokens = normalized.split(/\s+/).map((t) => t.replace(/^[^\w\+]+|[^\w]+$/g, ""));
+    const candidates = Array.from(new Set([normalized, ...tokens])).filter((t) => t.length > 0);
+
+    const matches = new Set<ContentSignature>();
+    for (const candidate of candidates) {
+      for (const signature of CONTENT_SIGNATURES) {
+        if (
+          signatureHasMetadataSupport(signature, columnName) &&
+          signature.pattern.test(candidate) &&
+          (!signature.validate || signature.validate(candidate))
+        ) {
+          matches.add(signature);
+        }
+      }
+    }
+    return Array.from(matches);
   } finally {
     bytes.fill(0);
   }

package/src/modules/introspector/dag.ts

@@ -236,7 +236,7 @@ export async function discoverPotentialLogicalLinks(
   const byColumn = new Map<string, QualifiedTable[]>();
   for (const row of rows) {
     const normalized = row.column_name.toLowerCase();
-    if (!/^(?:user_id|account_id|customer_id|member_id|subject_id|.*_user_id)$/.test(normalized)) {
+    if (!/^(?:user_id|account_id|customer_id|client_id|actor_id|user_uuid|member_id|subject_id|.*_user_id|target_email|user_email)$/.test(normalized)) {
       continue;
     }
 
@@ -247,7 +247,25 @@ export async function discoverPotentialLogicalLinks(
 
   const links: PotentialLogicalLink[] = [];
   const emitted = new Set<string>();
+
   for (const [column, tables] of byColumn.entries()) {
+    for (const table of tables) {
+      // Explicitly link any orphan table that has an identity-like column to the root table
+      if (table.schema === root.schema && table.table === root.table) {
+        continue;
+      }
+      const key = physicalLinkKey(root, table, column);
+      if (!physicalLinks.has(key) && !emitted.has(key)) {
+        emitted.add(key);
+        links.push({
+          sourceTable: root,
+          targetTable: table,
+          column,
+          reason: `Table exposes ${column} which conceptually maps to the root entity.`,
+        });
+      }
+    }
+    
     if (tables.length < 2) {
       continue;
     }

package/src/modules/introspector/run.ts

@@ -38,14 +38,6 @@ export async function runIntrospector(options: RunIntrospectorOptions): Promise<
     maxDepth,
   });
 
-  const classifiedColumns = await classifyDagTargets({
-    sql: options.sql,
-    targets: dag,
-    samplePercent: options.samplePercent,
-    sampleLimit: options.sampleLimit,
-    threshold: options.threshold,
-  });
-
   const [schemaHash, potentialLogicalLinks] = await Promise.all([
     detectSchemaDrift(options.sql, root.schema),
     discoverPotentialLogicalLinks(
@@ -58,7 +50,57 @@ export async function runIntrospector(options: RunIntrospectorOptions): Promise<
     ),
   ]);
 
-  const targets: IntrospectorTargetDraft[] = dag.map((target) => ({
+  const dagTableKeys = new Set(dag.map((t) => targetKey(t.table.schema, t.table.table)));
+  const logicalTargets: typeof dag = [];
+
+  for (const link of potentialLogicalLinks) {
+    const sourceKey = targetKey(link.sourceTable.schema, link.sourceTable.table);
+    const targetKeyStr = targetKey(link.targetTable.schema, link.targetTable.table);
+
+    let parentCol = link.column;
+    let childCol = link.column;
+    
+    // Attempt intelligent primary key mapping for orphaned root links
+    if (link.sourceTable.schema === root.schema && link.sourceTable.table === root.table) {
+      parentCol = ["target_email", "user_email", "email_address"].includes(link.column) ? "email" : "id";
+    }
+
+    if (dagTableKeys.has(sourceKey) && !dagTableKeys.has(targetKeyStr)) {
+      dagTableKeys.add(targetKeyStr);
+      logicalTargets.push({
+        table: link.targetTable,
+        parentTable: link.sourceTable,
+        constraintName: null,
+        childColumns: [childCol],
+        parentColumns: [parentCol],
+        depth: maxDepth,
+        fkCondition: `${formatQualifiedTable(link.sourceTable)}.${parentCol} = ${formatQualifiedTable(link.targetTable)}.${childCol}`,
+      });
+    } else if (dagTableKeys.has(targetKeyStr) && !dagTableKeys.has(sourceKey)) {
+      dagTableKeys.add(sourceKey);
+      logicalTargets.push({
+        table: link.sourceTable,
+        parentTable: link.targetTable,
+        constraintName: null,
+        childColumns: [childCol],
+        parentColumns: [parentCol],
+        depth: maxDepth,
+        fkCondition: `${formatQualifiedTable(link.targetTable)}.${parentCol} = ${formatQualifiedTable(link.sourceTable)}.${childCol}`,
+      });
+    }
+  }
+
+  const fullTargets = [...dag, ...logicalTargets];
+
+  const classifiedColumns = await classifyDagTargets({
+    sql: options.sql,
+    targets: fullTargets,
+    samplePercent: options.samplePercent,
+    sampleLimit: options.sampleLimit,
+    threshold: options.threshold,
+  });
+
+  const targets: IntrospectorTargetDraft[] = fullTargets.map((target) => ({
     table: target.table,
     parentTable: target.parentTable,
     fkCondition: target.fkCondition,

package/src/modules/introspector/yaml.ts

@@ -83,6 +83,20 @@ export function renderIntrospectorYaml(draft: IntrospectorDraft): string {
     "legal_disclaimer:",
     "  text: \"Auto-generated by Compliance Worker. The DPO/Developer is responsible for verifying all logical links and PII mappings.\"",
     "",
+    "# ===================================================================================",
+    "# HOW TO READ THIS MANIFEST:",
+    "# - 'targets': The list of tables the worker will delete/redact from.",
+    "# - 'parent': The table that owns this data. The worker deletes parent-first or child-first depending on the DB constraints.",
+    "# - 'join': The SQL condition used to link the child table to the parent table.",
+    "# - 'pii_columns': Columns identified as containing Personal Identifiable Information.",
+    "# - 'action': 'redact' (anonymizes the row but keeps it) or implicitly 'delete' (removes the row entirely).",
+    "#",
+    "# IMPORTANT:",
+    "# 1. Review all 'join' conditions. If the Introspector guessed a join for an orphaned table, verify it.",
+    "# 2. Review all 'pii_columns' to ensure no sensitive columns were missed.",
+    "# 3. Replace 'PENDING_REVIEW' in legal_attestation once verified.",
+    "# ===================================================================================",
+    "",
     "rules:",
     "  - id: dpdp_standard",
     `    root_table: ${yamlScalar(formatQualifiedTable(draft.root))}`,