sql-dag-flow 0.2.0__tar.gz → 0.3.0__tar.gz

{sql_dag_flow-0.2.0/src/sql_dag_flow.egg-info → sql_dag_flow-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sql-dag-flow
-Version: 0.2.0
+Version: 0.3.0
 Summary: A sophisticated SQL lineage visualization tool for Medallion Architectures.
 Author-email: Flavio Sandoval <dsandovalflavio@gmail.com>
 License: MIT
@@ -46,8 +46,6 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
 *   **1. Legacy Code Audit & Refactoring**:
     *   *The Problem*: You join a new project with 200+ undocumented SQL scripts. Nobody knows what breaks what.
     *   *The Solution*: Run `sql-dag-flow` to instantly map the "spaghetti" dependencies. Identify orphan tables, circular dependencies, and the impact of changing a Silver layer table.
-*   **2. Automated Architecture Documentation**:
-    *   *The Problem*: Architecture diagrams in Lucidchart or Visio are always outdated.
     *   *The Solution*: Generate interactive pipeline visualizations (ETL/ELT) to include in your Pull Requests, Wikis, or client deliverables.
 *   **3. Medallion Architecture Validation**:
     *   *The Problem*: It's hard to verify if the logical separation of layers (Bronze → Silver → Gold) is being respected.
@@ -63,6 +61,10 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
 *   **Medallion Architecture Support**: Automatically categorizes and colors nodes based on folder structure (Bronze, Silver, Gold).
 *   **Discovery Mode**: Visualize "Ghost Nodes" (missing files or external tables) and create them with a click.
 *   **CTE Visualization**: Detects internal Common Table Expressions and displays them as distinct Pink nodes.
+*   **Smart Layout (New 🧠)**:
+    *   Powered by **ELK (Eclipse Layout Kernel)**.
+    *   Minimizes edge crossings and optimizes flow direction.
+    *   Intelligent "Port" handling for cleaner connections.
 
 ### 🎮 Interactive Graph
 *   **Smart Context Menu**:
@@ -70,8 +72,9 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
     *   **Select Tree**: One-click selection of an entire dependency chain for easy movement.
     *   **Hide/Show**: Toggle visibility of individual nodes or full branches.
 *   **Advanced Navigation**:
-    *   **Sidebar**: Grouped list of nodes with usage counts and click-to-center navigation.
-    *   **Details Panel**: View formatted SQL code, schema info, and configure node settings.
+    *   **Sidebar**: Grouped list of nodes with toggle between **By Layer** and **By Project/Dataset** views.
+    *   **SQL Content Search**: Search inside SQL file content across all nodes — find WHERE clauses, JOINs, or any keyword.
+    *   **Details Panel**: View formatted SQL code, schema preview (DDL, CTAS, Views), and node configuration.
 
 ### 📝 Notes & Annotations
 *   **Center Placement**: New notes spawn exactly in the center of your view.
@@ -80,6 +83,12 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
     *   **Transparent & Borderless**:Create clean, floating text labels without boxes.
     *   **Groups**: Create visual containers to group related nodes.
 
+### 📊 Discovery & Analysis Tools (New ✨)
+*   **Business Rule Extraction**: Automatically detects and displays WHERE filters, CASE logic, HAVING clauses, and aggregations from each SQL model.
+*   **Complexity Scoring**: Weighted metric per node (JOINs×3, CTEs×2, Subqueries×3, Filters×1, CASE×2, Aggregations×1, UNIONs×2) with color-coded badges (🟢 Low, 🟡 Medium, 🟠 High, 🔴 Very High). Toggleable via ⚡ button.
+*   **Statistics Panel**: Centered popup with layer distribution bars, edge/source/sink/orphan counts, project/dataset tree, and architecture health validation.
+*   **Schema Preview**: Extracts columns from DDL, CTAS (`CREATE TABLE AS SELECT`), and `CREATE VIEW` statements.
+
 ### ⚙️ Customization
 *   **Premium UI**:
     *   **Themes**: Toggle between Light and Dark modes.
@@ -112,7 +121,7 @@ Install easily via `pip`:
 pip install sql-dag-flow
 ```
 
-To update to the latest version (**v0.2.0**):
+To update to the latest version (**v0.3.0**):
 
 ```bash
 pip install --upgrade sql-dag-flow

{sql_dag_flow-0.2.0 → sql_dag_flow-0.3.0}/README.md

@@ -21,8 +21,6 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
 *   **1. Legacy Code Audit & Refactoring**:
     *   *The Problem*: You join a new project with 200+ undocumented SQL scripts. Nobody knows what breaks what.
     *   *The Solution*: Run `sql-dag-flow` to instantly map the "spaghetti" dependencies. Identify orphan tables, circular dependencies, and the impact of changing a Silver layer table.
-*   **2. Automated Architecture Documentation**:
-    *   *The Problem*: Architecture diagrams in Lucidchart or Visio are always outdated.
     *   *The Solution*: Generate interactive pipeline visualizations (ETL/ELT) to include in your Pull Requests, Wikis, or client deliverables.
 *   **3. Medallion Architecture Validation**:
     *   *The Problem*: It's hard to verify if the logical separation of layers (Bronze → Silver → Gold) is being respected.
@@ -38,6 +36,10 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
 *   **Medallion Architecture Support**: Automatically categorizes and colors nodes based on folder structure (Bronze, Silver, Gold).
 *   **Discovery Mode**: Visualize "Ghost Nodes" (missing files or external tables) and create them with a click.
 *   **CTE Visualization**: Detects internal Common Table Expressions and displays them as distinct Pink nodes.
+*   **Smart Layout (New 🧠)**:
+    *   Powered by **ELK (Eclipse Layout Kernel)**.
+    *   Minimizes edge crossings and optimizes flow direction.
+    *   Intelligent "Port" handling for cleaner connections.
 
 ### 🎮 Interactive Graph
 *   **Smart Context Menu**:
@@ -45,8 +47,9 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
     *   **Select Tree**: One-click selection of an entire dependency chain for easy movement.
     *   **Hide/Show**: Toggle visibility of individual nodes or full branches.
 *   **Advanced Navigation**:
-    *   **Sidebar**: Grouped list of nodes with usage counts and click-to-center navigation.
-    *   **Details Panel**: View formatted SQL code, schema info, and configure node settings.
+    *   **Sidebar**: Grouped list of nodes with toggle between **By Layer** and **By Project/Dataset** views.
+    *   **SQL Content Search**: Search inside SQL file content across all nodes — find WHERE clauses, JOINs, or any keyword.
+    *   **Details Panel**: View formatted SQL code, schema preview (DDL, CTAS, Views), and node configuration.
 
 ### 📝 Notes & Annotations
 *   **Center Placement**: New notes spawn exactly in the center of your view.
@@ -55,6 +58,12 @@ Specially optimized for the **Medallion Architecture** (Bronze, Silver, Gold) an
     *   **Transparent & Borderless**:Create clean, floating text labels without boxes.
     *   **Groups**: Create visual containers to group related nodes.
 
+### 📊 Discovery & Analysis Tools (New ✨)
+*   **Business Rule Extraction**: Automatically detects and displays WHERE filters, CASE logic, HAVING clauses, and aggregations from each SQL model.
+*   **Complexity Scoring**: Weighted metric per node (JOINs×3, CTEs×2, Subqueries×3, Filters×1, CASE×2, Aggregations×1, UNIONs×2) with color-coded badges (🟢 Low, 🟡 Medium, 🟠 High, 🔴 Very High). Toggleable via ⚡ button.
+*   **Statistics Panel**: Centered popup with layer distribution bars, edge/source/sink/orphan counts, project/dataset tree, and architecture health validation.
+*   **Schema Preview**: Extracts columns from DDL, CTAS (`CREATE TABLE AS SELECT`), and `CREATE VIEW` statements.
+
 ### ⚙️ Customization
 *   **Premium UI**:
     *   **Themes**: Toggle between Light and Dark modes.
@@ -87,7 +96,7 @@ Install easily via `pip`:
 pip install sql-dag-flow
 ```
 
-To update to the latest version (**v0.2.0**):
+To update to the latest version (**v0.3.0**):
 
 ```bash
 pip install --upgrade sql-dag-flow

{sql_dag_flow-0.2.0 → sql_dag_flow-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "sql-dag-flow"
-version = "0.2.0"
+version = "0.3.0"
 description = "A sophisticated SQL lineage visualization tool for Medallion Architectures."
 readme = "README.md"
 requires-python = ">=3.8"

{sql_dag_flow-0.2.0 → sql_dag_flow-0.3.0}/src/sql_dag_flow/main.py

@@ -10,6 +10,7 @@ import json
 import webbrowser
 import threading
 import time
+import shutil
 from .parser import parse_sql_files, build_graph
 
 app = FastAPI()
@@ -193,6 +194,51 @@ def create_file(request: CreateFileRequest):
     except Exception as e:
         raise HTTPException(status_code=500, detail=str(e))
 
+class MoveFileRequest(BaseModel):
+    current_path: str
+    target_layer: str
+
+@app.post("/files/move")
+def move_file(request: MoveFileRequest):
+    try:
+        target_layer = request.target_layer.strip()
+        if ".." in request.current_path:
+             raise HTTPException(status_code=400, detail="Invalid path")
+             
+        current_path = os.path.join(CURRENT_DIRECTORY, request.current_path)
+        if not os.path.exists(current_path):
+             raise HTTPException(status_code=404, detail="Original file not found")
+             
+        filename = os.path.basename(current_path)
+        dir_name = os.path.dirname(current_path)
+        
+        parent_dir_name = os.path.basename(dir_name).lower()
+        if parent_dir_name in ["bronze", "bronce", "silver", "gold", "other"]:
+            new_dir = os.path.dirname(dir_name)
+            new_dir = os.path.join(new_dir, target_layer)
+        else:
+            new_dir = os.path.join(dir_name, target_layer)
+            
+        new_full_path = os.path.join(new_dir, filename)
+        
+        if new_full_path == current_path:
+             return {"message": "File already in target layer", "path": request.current_path}
+             
+        if os.path.exists(new_full_path):
+             raise HTTPException(status_code=400, detail="A file with this name already exists in the target layer folder")
+             
+        os.makedirs(new_dir, exist_ok=True)
+        shutil.move(current_path, new_full_path)
+        
+        # Normalize response relative path 
+        rel_new_path = os.path.relpath(new_full_path, CURRENT_DIRECTORY).replace(os.sep, '/')
+        return {"message": f"File moved to {target_layer}", "path": rel_new_path}
+
+    except HTTPException as he:
+        raise he
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
 # Serve Static Files (Frontend)
 if os.path.exists(STATIC_DIR):
     app.mount("/assets", StaticFiles(directory=os.path.join(STATIC_DIR, "assets")), name="assets")

{sql_dag_flow-0.2.0 → sql_dag_flow-0.3.0}/src/sql_dag_flow/parser.py

@@ -130,6 +130,10 @@ def parse_sql_files(directory, allowed_subfolders=None, dialect="bigquery"):
                     if create_node and create_node.this:
                         # sqlglot represents the target as an exp.Table or exp.Schema
                         target_exp = create_node.this
+                        # Handle Schema wrapping (DDL with column definitions)
+                        # e.g. CREATE TABLE project.dataset.table (id INT64, name STRING)
+                        if isinstance(target_exp, exp.Schema):
+                            target_exp = target_exp.this
                         if isinstance(target_exp, exp.Table):
                             target_table_name = target_exp.name
                             dataset = target_exp.db or "default"
@@ -154,17 +158,56 @@ def parse_sql_files(directory, allowed_subfolders=None, dialect="bigquery"):
                          if parent_dir.lower() not in ["bronze", "bronce", "silver", "gold", "other"] and dataset == "default":
                              dataset = parent_dir
                     
-                    dependencies = set()
+                    dependencies = {}  # dep_name -> dep_type
                     
-                    # 1. Identify CTEs defined in the query to exclude them from dependencies
+                    # 1. Identify CTEs defined in the query and their internal dependencies
                     defined_ctes = {}
+                    cte_deps = {}  # cte_name -> {full_dep_name: dep_type}
+                    
                     for cte in parsed.find_all(exp.CTE):
-                        if cte.alias_or_name:
-                            # Extract the full CTE definition as SQL string
-                            # We can use cte.sql() or just the inner query
-                            # User likely wants the full "name AS ( ... )" or just the inner query
-                            # Let's give the full CTE expression for context
-                            defined_ctes[cte.alias_or_name] = cte.sql(dialect=dialect, pretty=True)
+                        cte_name = cte.alias_or_name
+                        if not cte_name:
+                            continue
+                        defined_ctes[cte_name] = cte.sql(dialect=dialect, pretty=True)
+                        
+                        # Extract tables referenced INSIDE this CTE definition
+                        cte_internal_deps = {}
+                        cte_join_tables = set()
+                        for j in cte.find_all(exp.Join):
+                            jt = j.find(exp.Table)
+                            if jt:
+                                cte_join_tables.add(jt.name)
+                        
+                        for t in cte.find_all(exp.Table):
+                            t_name = t.name
+                            # Skip self-references and references to other CTEs in the same query
+                            if t_name == target_table_name or t_name in defined_ctes:
+                                continue
+                            t_full = t_name
+                            if t.db:
+                                t_full = f"{t.db}.{t_name}"
+                                if t.catalog:
+                                    t_full = f"{t.catalog}.{t.db}.{t_name}"
+                            cte_internal_deps[t_full] = "JOIN" if t_name in cte_join_tables else "FROM"
+                        
+                        cte_deps[cte_name] = cte_internal_deps
+                    
+                    # Collect all table names that appear inside CTE definitions
+                    # These should NOT be direct dependencies of the parent model
+                    tables_inside_ctes = set()
+                    for cte_name, ct in cte_deps.items():
+                        for dep_key in ct:
+                            tables_inside_ctes.add(dep_key.split(".")[-1])
+                    
+                    # Detect JOIN tables for labeling (only at the top-level query, not inside CTEs)
+                    join_tables = set()
+                    for join_node in parsed.find_all(exp.Join):
+                        # Check this join is not inside a CTE
+                        parent_cte = join_node.find_ancestor(exp.CTE)
+                        if parent_cte is None:
+                            join_table = join_node.find(exp.Table)
+                            if join_table:
+                                join_tables.add(join_table.name)
                     
                     # Find all tables referenced in the query
                     for table in parsed.find_all(exp.Table):
@@ -180,22 +223,97 @@ def parse_sql_files(directory, allowed_subfolders=None, dialect="bigquery"):
                         if dep_name == target_table_name:
                             continue
                             
-                        # Internal CTE references
+                        # Internal CTE references (main query references a CTE)
                         if dep_name in defined_ctes:
-                            # Add strictly as a CTE dependency so we can visualize it if desired
-                            dependencies.add(f"cte:{filename_base}:{dep_name}")
+                            dependencies[f"cte:{filename_base}:{dep_name}"] = "CTE"
                             continue
-
-                        # If we haven't found a CREATE statement, this might just be a SELECT
-                        # and we treat the filename as the target.
                         
-                        dependencies.add(full_name)
-                        # REMOVED: partial match addition to prevent double counting in visual metadata
-                        # matches are now handled in build_graph via fuzzy lookup
+                        # Skip tables that belong inside a CTE definition
+                        # These will be wired as CTE_node deps in build_graph
+                        if dep_name in tables_inside_ctes:
+                            continue
+
+                        # Regular external dependency at the main query level
+                        dep_type = "JOIN" if dep_name in join_tables else "FROM"
+                        dependencies[full_name] = dep_type
+                    
+                    # ===== Business Rule Extraction =====
+                    business_rules = {
+                        "filters": [],       # WHERE conditions
+                        "case_logic": [],    # CASE statements
+                        "having": [],        # HAVING conditions
+                        "aggregations": [],  # Aggregate functions
+                    }
+                    
+                    # Extract WHERE clauses
+                    for where_node in parsed.find_all(exp.Where):
+                        try:
+                            where_sql = where_node.this.sql(dialect=dialect, pretty=False)
+                            business_rules["filters"].append(where_sql)
+                        except Exception:
+                            pass
+                    
+                    # Extract CASE statements
+                    for case_node in parsed.find_all(exp.Case):
+                        try:
+                            case_sql = case_node.sql(dialect=dialect, pretty=False)
+                            # Try to get the alias if available
+                            parent = case_node.parent
+                            alias = ""
+                            if hasattr(parent, 'alias') and parent.alias:
+                                alias = parent.alias
+                            label = f"{alias}: {case_sql}" if alias else case_sql
+                            business_rules["case_logic"].append(label)
+                        except Exception:
+                            pass
+                    
+                    # Extract HAVING clauses
+                    for having_node in parsed.find_all(exp.Having):
+                        try:
+                            having_sql = having_node.this.sql(dialect=dialect, pretty=False)
+                            business_rules["having"].append(having_sql)
+                        except Exception:
+                            pass
+                    
+                    # Extract aggregate functions
+                    agg_types = (exp.Count, exp.Sum, exp.Avg, exp.Min, exp.Max)
+                    for agg_node in parsed.find_all(*agg_types):
+                        try:
+                            agg_sql = agg_node.sql(dialect=dialect, pretty=False)
+                            parent = agg_node.parent
+                            alias = ""
+                            if hasattr(parent, 'alias') and parent.alias:
+                                alias = parent.alias
+                            label = f"{alias}: {agg_sql}" if alias else agg_sql
+                            business_rules["aggregations"].append(label)
+                        except Exception:
+                            pass
+                    
+                    # ===== Complexity Score =====
+                    # Weights: JOIN=3, CTE=2, Subquery=3, WHERE=1, CASE=2, Aggregation=1, UNION=2
+                    complexity_breakdown = {
+                        "joins": len(list(parsed.find_all(exp.Join))),
+                        "ctes": len(defined_ctes),
+                        "subqueries": len(list(parsed.find_all(exp.Subquery))),
+                        "filters": len(business_rules["filters"]),
+                        "case_statements": len(business_rules["case_logic"]),
+                        "aggregations": len(business_rules["aggregations"]),
+                        "unions": len(list(parsed.find_all(exp.Union))),
+                    }
+                    
+                    complexity_score = (
+                        complexity_breakdown["joins"] * 3 +
+                        complexity_breakdown["ctes"] * 2 +
+                        complexity_breakdown["subqueries"] * 3 +
+                        complexity_breakdown["filters"] * 1 +
+                        complexity_breakdown["case_statements"] * 2 +
+                        complexity_breakdown["aggregations"] * 1 +
+                        complexity_breakdown["unions"] * 2
+                    )
+                    
+                    complexity_breakdown["score"] = complexity_score
                              
                     tables[filename_base] = { 
-                        # Use filename_base as unique ID for the graph to avoid ambiguity
-                        # Visual label can be the actual table name
                         "id": filename_base,
                         "label": target_table_name,
                         "layer": layer,
@@ -203,9 +321,12 @@ def parse_sql_files(directory, allowed_subfolders=None, dialect="bigquery"):
                         "project": project,
                         "dataset": dataset,
                         "path": filepath,
-                        "dependencies": list(dependencies),
+                        "dependencies": dependencies,
                         "content": sql_content,
-                        "ctes": defined_ctes
+                        "ctes": defined_ctes,
+                        "cte_deps": cte_deps,
+                        "business_rules": business_rules,
+                        "complexity": complexity_breakdown
                     }
                 except Exception as e:
                     print(f"Error parsing {filepath}: {e}")
@@ -217,7 +338,7 @@ def parse_sql_files(directory, allowed_subfolders=None, dialect="bigquery"):
                         "project": "n/a",
                         "dataset": "n/a",
                         "path": filepath,
-                        "dependencies": [],
+                        "dependencies": {},
                         "error": str(e),
                         "content": sql_content
                     }
@@ -260,7 +381,10 @@ def build_graph(tables, discovery_mode=False):
 
     # Create edges first (conceptually) to count dependencies
     for source_id, data in tables.items():
-        for dep in data["dependencies"]:
+        deps = data["dependencies"]
+        # Support both dict (name->type) and list (legacy) formats
+        dep_items = deps.items() if isinstance(deps, dict) else [(d, "FROM") for d in deps]
+        for dep, dep_type in dep_items:
             target_id = lookup.get(dep)
             
             # Fuzzy lookup: if exact match fails, try splitting by dot and matching last part (table name)
@@ -274,26 +398,23 @@ def build_graph(tables, discovery_mode=False):
                     "source": target_id,
                     "target": source_id,
                     "animated": True,
+                    "label": dep_type,
                     "style": {"stroke": "#b1b1b7"}
                 })
                 incoming_edges_count[source_id] = incoming_edges_count.get(source_id, 0) + 1
             else:
-                 # Discovery Mode: Handle missing dependencies OR CTEs
-                 if discovery_mode:
-                     
-                     # 1. Handle CTEs
-                     if dep.startswith("cte:"):
-                         # Format: cte:filename_base:cte_name
-                         parts = dep.split(":")
-                         if len(parts) >= 3:
-                             # Reconstruct in case name had colons (unlikely but safe)
-                             cte_name = ":".join(parts[2:])
-                             
-                             # CTE ID is the dependency string itself to be unique per file
+                 # Handle CTE dependency references
+                 if dep.startswith("cte:"):
+                     parts = dep.split(":")
+                     if len(parts) >= 3:
+                         cte_name = ":".join(parts[2:])
+                         cte_internal_deps = tables[source_id].get("cte_deps", {}).get(cte_name, {})
+                         
+                         if discovery_mode:
+                             # Discovery Mode: Create CTE ghost node with incoming edges
                              cte_id = dep
                              
                              if cte_id not in missing_nodes:
-                                 # Retrieve SQL content if available
                                  cte_content = f"-- CTE: {cte_name}"
                                  if source_id in tables and "ctes" in tables[source_id]:
                                      if cte_name in tables[source_id]["ctes"]:
@@ -302,30 +423,89 @@ def build_graph(tables, discovery_mode=False):
                                  missing_nodes[cte_id] = {
                                      "id": cte_id,
                                      "label": cte_name,
-                                     "layer": "cte",  # Special layer for CTEs
-                                     "type": "cte",   # Special type for CTEs
+                                     "layer": "cte",
+                                     "type": "cte",
                                      "project": "internal",
                                      "dataset": "cte",
                                      "path": "internal",
-                                     "dependencies": [],
+                                     "dependencies": {},
                                      "content": cte_content
                                  }
                                  
-                             # Edge from CTE to Table
+                                 # Wire CTE's internal dependencies as incoming edges
+                                 for inner_dep, inner_type in cte_internal_deps.items():
+                                     inner_target = lookup.get(inner_dep)
+                                     if not inner_target and "." in inner_dep:
+                                         inner_target = lookup.get(inner_dep.split(".")[-1])
+                                     
+                                     if inner_target:
+                                         edges.append({
+                                             "id": f"{inner_target}-{cte_id}",
+                                             "source": inner_target,
+                                             "target": cte_id,
+                                             "animated": True,
+                                             "label": inner_type,
+                                             "style": {"stroke": "#E91E63"}
+                                         })
+                                         incoming_edges_count[cte_id] = incoming_edges_count.get(cte_id, 0) + 1
+                                     else:
+                                         # Create ghost node for missing CTE dep
+                                         ghost_id = inner_dep
+                                         if ghost_id not in missing_nodes:
+                                             dep_parts = ghost_id.split('.')
+                                             ghost_project, ghost_dataset, ghost_table = "default", "default", ghost_id
+                                             if len(dep_parts) == 3:
+                                                 ghost_project, ghost_dataset, ghost_table = dep_parts
+                                             elif len(dep_parts) == 2:
+                                                 ghost_dataset, ghost_table = dep_parts
+                                             missing_nodes[ghost_id] = {
+                                                 "id": ghost_id, "label": ghost_table,
+                                                 "layer": "external", "type": "table",
+                                                 "project": ghost_project, "dataset": ghost_dataset,
+                                                 "path": "discovered", "dependencies": {},
+                                                 "content": "-- Discovered dependency (via CTE)"
+                                             }
+                                         edges.append({
+                                             "id": f"{ghost_id}-{cte_id}",
+                                             "source": ghost_id, "target": cte_id,
+                                             "animated": True, "label": inner_type,
+                                             "style": {"stroke": "#ff9f1c", "strokeDasharray": "5,5"}
+                                         })
+                                         incoming_edges_count[cte_id] = incoming_edges_count.get(cte_id, 0) + 1
+                                 
+                             # Edge from CTE to parent Table
                              edges.append({
                                 "id": f"{cte_id}-{source_id}",
                                 "source": cte_id,
                                 "target": source_id,
                                 "animated": True,
-                                "style": {"stroke": "#E91E63", "strokeDasharray": "2,2"} # Pink dashed for CTEs
+                                "label": "CTE",
+                                "style": {"stroke": "#E91E63", "strokeDasharray": "2,2"}
                              })
-                             # CTEs generally don't have incoming edges in this parser implementation yet
-                             # but we count for the target
                              incoming_edges_count[source_id] = incoming_edges_count.get(source_id, 0) + 1
-                         continue
+                         else:
+                             # Non-discovery: Flatten CTE deps as direct edges to parent
+                             for inner_dep, inner_type in cte_internal_deps.items():
+                                 inner_target = lookup.get(inner_dep)
+                                 if not inner_target and "." in inner_dep:
+                                     inner_target = lookup.get(inner_dep.split(".")[-1])
+                                 if inner_target and inner_target != source_id:
+                                     edge_id = f"{inner_target}-{source_id}"
+                                     # Avoid duplicate edges
+                                     if not any(e["id"] == edge_id for e in edges):
+                                         edges.append({
+                                             "id": edge_id,
+                                             "source": inner_target,
+                                             "target": source_id,
+                                             "animated": True,
+                                             "label": inner_type,
+                                             "style": {"stroke": "#b1b1b7"}
+                                         })
+                                         incoming_edges_count[source_id] = incoming_edges_count.get(source_id, 0) + 1
+                     continue
 
-                     # 2. Handle missing external nodes
-                     if not target_id:
+                 # Handle missing external nodes (discovery mode only)
+                 if discovery_mode and not target_id:
                          # Create a unique ID for the missing node
                          # Use the full dependency name as the ID
                          ghost_id = dep