sql-glider 0.1.10__tar.gz → 0.1.12__tar.gz

{sql_glider-0.1.10 → sql_glider-0.1.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sql-glider
-Version: 0.1.10
+Version: 0.1.12
 Summary: SQL Utility Toolkit for better understanding, use, and governance of your queries in a native environment.
 Project-URL: Homepage, https://github.com/rycowhi/sql-glider/
 Project-URL: Repository, https://github.com/rycowhi/sql-glider/

sql_glider-0.1.12/plans/2026-01-28-sparksql-table-extraction.md

@@ -0,0 +1,58 @@
+# SparkSQL Table Extraction for Unsupported Statement Types
+
+**Status:** Planned
+**Date:** 2026-01-28
+
+## Overview
+
+The `analyze_tables()` method in `LineageAnalyzer` currently extracts tables from a subset of statement types (SELECT, INSERT, CREATE, DELETE, DROP, TRUNCATE, CACHE). Several SparkSQL-specific statement types reference tables but are not captured during table extraction. This plan adds table extraction support for these missing types.
+
+Column lineage is **not affected** — these statements contain no SELECT and cannot produce column-level lineage. The goal is to ensure `sqlglider tables overview` reports all tables referenced in a SQL file.
+
+## Statements to Add
+
+| Statement | SQLGlot Expression | Table Location | Proposed Usage |
+|-----------|-------------------|----------------|----------------|
+| `UNCACHE TABLE t` | `exp.Uncache` | `expr.this` | `INPUT` |
+| `REFRESH TABLE t` | `exp.Refresh` | `expr.this` | `INPUT` |
+| `LOAD DATA INPATH '...' INTO TABLE t` | `exp.LoadData` | `expr.this` | `OUTPUT` |
+| `ALTER TABLE t ...` | `exp.Alter` | `expr.this` | `OUTPUT` |
+| `ANALYZE TABLE t COMPUTE STATISTICS` | `exp.Analyze` | `expr.this` | `INPUT` |
+
+### Usage Rationale
+
+- **UNCACHE / REFRESH / ANALYZE**: Read-oriented metadata operations on an existing table → `INPUT`
+- **LOAD DATA**: Writes data into a table → `OUTPUT`
+- **ALTER TABLE**: Modifies table structure → `OUTPUT`
+
+## Implementation Steps
+
+- [ ] Add extraction logic to `_get_target_table_info()` in [analyzer.py](src/sqlglider/lineage/analyzer.py) for each new expression type
+- [ ] Add each type to the `_get_statement_type()` type_map for readable skip messages
+- [ ] Add entries to `_is_target_table()` where applicable (LOAD DATA, ALTER)
+- [ ] Ensure `_get_target_and_select()` returns `None` gracefully for these types (they have no SELECT)
+- [ ] Add unit tests in [test_analyzer.py](tests/sqlglider/lineage/test_analyzer.py):
+  - Table extraction returns correct table name and usage for each type
+  - Column lineage correctly skips these with appropriate message
+  - Parameterized test covering all five statement types
+- [ ] Verify graph build handles these gracefully (skipped queries warning)
+- [ ] Run full test suite and coverage check
+
+## Files to Modify
+
+- `src/sqlglider/lineage/analyzer.py` — extraction logic
+- `tests/sqlglider/lineage/test_analyzer.py` — unit tests
+
+## Testing Strategy
+
+- Parameterized tests with SparkSQL syntax for each statement type
+- Verify `analyze_tables()` returns correct table name, usage, and object type
+- Verify `analyze_queries()` adds these to `skipped_queries` with clear reason
+- Ensure no regressions in existing tests
+- Coverage threshold (80%) maintained
+
+## Notes
+
+- These are all parsed by sqlglot's Spark dialect parser, so no custom parsing is needed
+- Some of these (SHOW, DESCRIBE, EXPLAIN) parse as `exp.Command` — those are intentionally excluded since they don't reference tables in a structured way
+- INSERT OVERWRITE and multi-INSERT patterns may warrant separate investigation

sql_glider-0.1.12/plans/2026-01-29-no-star-flag.md

@@ -0,0 +1,47 @@
+# Add `--no-star` Flag
+
+**Status:** Completed
+
+## Overview
+Add `--no-star` flag to `lineage` and `graph build` commands. When set, analysis fails if `SELECT *` or `t.*` cannot be resolved to actual columns.
+
+## Changes
+
+### 1. `src/sqlglider/utils/config.py` — Add to ConfigSettings
+- [x] Add `no_star: Optional[bool] = None`
+
+### 2. `src/sqlglider/lineage/analyzer.py` — Add parameter + enforce
+- [x] Add `no_star: bool = False` to `__init__`, store as `self._no_star`
+- [x] Add `StarResolutionError` exception class (distinct from `ValueError` to avoid being swallowed by skipped-query handler)
+- [x] DML/DDL path: raise `StarResolutionError` before fallback for bare `*` and `t.*`
+- [x] DQL path: add star handling for both bare `*` and `t.*` with same error behavior
+- [x] Re-raise `StarResolutionError` in `analyze_queries` instead of treating as skipped query
+
+### 3. `src/sqlglider/graph/builder.py` — Pass through
+- [x] Add `no_star: bool = False` to `__init__`, store as `self.no_star`
+- [x] Pass to `LineageAnalyzer(sql_content, dialect=file_dialect, no_star=self.no_star)`
+
+### 4. `src/sqlglider/cli.py` — Add CLI options
+- [x] `lineage` command: Add `no_star: bool = typer.Option(False, "--no-star", ...)`
+- [x] Resolve: `no_star = no_star or config.no_star or False`
+- [x] Pass to `LineageAnalyzer(sql, dialect=dialect, no_star=no_star)`
+- [x] `graph_build` command: same option, passed to `GraphBuilder(..., no_star=no_star)`
+
+### 5. `tests/sqlglider/lineage/test_analyzer.py` — Tests
+- [x] Test bare `SELECT *` with `no_star=True` raises `StarResolutionError`
+- [x] Test `SELECT t.*` with `no_star=True` raises `StarResolutionError`
+- [x] Test resolvable star (via CTE) still works with `no_star=True`
+- [x] Test resolvable qualified star (via CTE) still works with `no_star=True`
+- [x] Test default (`no_star=False`) still falls back to `table.*`
+
+## Implementation Notes
+
+### Deviations from original plan
+- Used `StarResolutionError` instead of `ValueError` because `analyze_queries` catches `ValueError` to handle unsupported statement types (skipped queries). A plain `ValueError` would be silently swallowed.
+- Added star handling in the DQL (plain SELECT) code path in addition to the DML/DDL path. The original plan only addressed the DML/DDL path, but plain `SELECT *` queries go through a different branch in `get_output_columns`.
+- Resolvable star tests use CTEs instead of `CREATE TABLE` with explicit columns, since `_extract_schema_from_statement` only handles `CREATE ... AS SELECT`, not DDL with column definitions.
+
+## Verification
+- `uv run pytest` — 597 passed, 1 skipped, coverage 80.48%
+- `uv run basedpyright src/` — 0 errors
+- `uv run ruff check` — all checks passed

sql_glider-0.1.12/plans/2026-01-29-resolve-schema.md

@@ -0,0 +1,49 @@
+# `--resolve-schema` Flag and Catalog Integration
+
+**Status:** Completed
+
+## Overview
+
+Add `--resolve-schema` flag to `graph build` that runs a two-pass process: first extracting schema from all files, then running lineage analysis with the full schema available. Optionally, `--catalog-type` fills schema gaps by pulling DDL from a remote catalog.
+
+## Design Decisions
+
+- **Types are not required** — SQLGlot only needs column names for star expansion; types are stored as `"UNKNOWN"`
+- **Two-pass approach** — Pass 1 extracts schema from all files (order-independent), Pass 2 runs lineage with full schema
+- **Opt-in via `--resolve-schema`** — default behavior unchanged
+- **Catalog fills gaps only** — file-derived schema always wins over catalog-sourced schema
+- **`--catalog-type` requires `--resolve-schema`** — validated at CLI level
+
+## Implementation
+
+- [x] Add `schema` param to `LineageAnalyzer.__init__()` — pre-populates `_file_schema`
+- [x] Add `extract_schema_only()` and `get_extracted_schema()` methods to `LineageAnalyzer`
+- [x] Create `src/sqlglider/utils/schema.py` with `parse_ddl_to_schema()` for DDL column extraction
+- [x] Add `resolve_schema`, `catalog_type`, `catalog_config` to `GraphBuilder`
+- [x] Implement `_extract_schemas()` for pass 1 and `_fill_schema_from_catalog()` for catalog gap-filling
+- [x] Two-pass flow in `add_files()` and `add_manifest()`
+- [x] Add `--resolve-schema` and `--catalog-type` CLI flags to `graph build`
+- [x] Add `resolve_schema` to `ConfigSettings`
+- [x] Tests: 25 new tests (schema parsing, analyzer schema param, cross-file resolution, catalog mocking)
+
+## Files Modified
+
+- `src/sqlglider/lineage/analyzer.py` — schema param, extraction methods
+- `src/sqlglider/graph/builder.py` — two-pass processing, catalog integration
+- `src/sqlglider/cli.py` — CLI flags
+- `src/sqlglider/utils/config.py` — config setting
+- `src/sqlglider/utils/schema.py` — **new** DDL parsing utility
+- `tests/sqlglider/utils/test_schema.py` — **new**
+- `tests/sqlglider/graph/test_builder.py` — resolve schema + catalog tests
+- `tests/sqlglider/lineage/test_analyzer.py` — schema param tests
+
+## Verification
+
+- 617 passed, 1 skipped
+- Coverage: 80.43%
+- basedpyright: 0 errors
+- ruff: all checks passed
+
+## Known Limitations
+
+- Cross-file CTAS chains with `SELECT *` (view B depends on view A via star) may not resolve if both are in separate files and the schema extraction pass processes B before A. This is rare in practice.

{sql_glider-0.1.10 → sql_glider-0.1.12}/src/sqlglider/_version.py

@@ -28,7 +28,7 @@ version_tuple: VERSION_TUPLE
 commit_id: COMMIT_ID
 __commit_id__: COMMIT_ID
 
-__version__ = version = '0.1.10'
-__version_tuple__ = version_tuple = (0, 1, 10)
+__version__ = version = '0.1.12'
+__version_tuple__ = version_tuple = (0, 1, 12)
 
 __commit_id__ = commit_id = None

{sql_glider-0.1.10 → sql_glider-0.1.12}/src/sqlglider/cli.py

@@ -12,7 +12,7 @@ from sqlglot.errors import ParseError
 from typing_extensions import Annotated
 
 from sqlglider.global_models import AnalysisLevel, NodeFormat
-from sqlglider.lineage.analyzer import LineageAnalyzer
+from sqlglider.lineage.analyzer import LineageAnalyzer, SchemaResolutionError
 from sqlglider.lineage.formatters import (
     CsvFormatter,
     JsonFormatter,
@@ -166,6 +166,11 @@ def lineage(
         exists=True,
         help="Path to variables file (JSON or YAML)",
     ),
+    no_star: bool = typer.Option(
+        False,
+        "--no-star",
+        help="Fail if SELECT * cannot be resolved to actual columns",
+    ),
 ) -> None:
     """
     Analyze column or table lineage for a SQL file.
@@ -207,6 +212,7 @@ def lineage(
     level_str = level or config.level or "column"
     output_format = output_format or config.output_format or "text"
     templater = templater or config.templater  # None means no templating
+    no_star = no_star or config.no_star or False
     # Validate and convert level to enum
     try:
         analysis_level = AnalysisLevel(level_str)
@@ -261,7 +267,7 @@ def lineage(
         )
 
         # Create analyzer
-        analyzer = LineageAnalyzer(sql, dialect=dialect)
+        analyzer = LineageAnalyzer(sql, dialect=dialect, no_star=no_star)
 
         # Unified lineage analysis (handles both single and multi-query files)
         results = analyzer.analyze_queries(
@@ -990,6 +996,40 @@ def graph_build(
         exists=True,
         help="Path to variables file (JSON or YAML)",
     ),
+    no_star: bool = typer.Option(
+        False,
+        "--no-star",
+        help="Fail if SELECT * cannot be resolved to actual columns",
+    ),
+    resolve_schema: bool = typer.Option(
+        False,
+        "--resolve-schema",
+        help="Extract schema from all files before lineage analysis, "
+        "enabling cross-file star resolution",
+    ),
+    catalog_type: Optional[str] = typer.Option(
+        None,
+        "--catalog-type",
+        "-c",
+        help="Catalog provider for pulling DDL of tables not found in files "
+        "(requires --resolve-schema). E.g. 'databricks'",
+    ),
+    dump_schema: Optional[Path] = typer.Option(
+        None,
+        "--dump-schema",
+        help="Dump resolved schema to file (requires --resolve-schema)",
+    ),
+    dump_schema_format: Optional[str] = typer.Option(
+        None,
+        "--dump-schema-format",
+        help="Format for dumped schema: 'text' (default), 'json', or 'csv'",
+    ),
+    strict_schema: bool = typer.Option(
+        False,
+        "--strict-schema",
+        help="Fail if any column's table cannot be identified during schema extraction "
+        "(requires --resolve-schema)",
+    ),
 ) -> None:
     """
     Build a lineage graph from SQL files.
@@ -1024,6 +1064,38 @@ def graph_build(
     config = load_config()
     dialect = dialect or config.dialect or "spark"
     templater = templater or config.templater  # None means no templating
+    no_star = no_star or config.no_star or False
+    resolve_schema = resolve_schema or config.resolve_schema or False
+    strict_schema = strict_schema or config.strict_schema or False
+
+    if strict_schema and not resolve_schema:
+        err_console.print("[red]Error:[/red] --strict-schema requires --resolve-schema")
+        raise typer.Exit(1)
+
+    if catalog_type and not resolve_schema:
+        err_console.print("[red]Error:[/red] --catalog-type requires --resolve-schema")
+        raise typer.Exit(1)
+
+    # Resolve dump_schema options from config
+    dump_schema = dump_schema or (
+        Path(config.dump_schema) if config.dump_schema else None
+    )
+    dump_schema_format = dump_schema_format or config.dump_schema_format or "text"
+
+    if dump_schema and not resolve_schema:
+        err_console.print("[red]Error:[/red] --dump-schema requires --resolve-schema")
+        raise typer.Exit(1)
+
+    if dump_schema_format not in ("text", "json", "csv"):
+        err_console.print(
+            f"[red]Error:[/red] Invalid --dump-schema-format '{dump_schema_format}'. "
+            "Use 'text', 'json', or 'csv'."
+        )
+        raise typer.Exit(1)
+
+    # Only inherit catalog_type from config when resolve_schema is active
+    if resolve_schema and not catalog_type:
+        catalog_type = config.catalog_type
 
     # Validate and convert node format to enum
     try:
@@ -1076,10 +1148,22 @@ def graph_build(
         sql_preprocessor = _preprocess
 
     try:
+        # Build catalog config from config file if available
+        catalog_config_dict = None
+        if catalog_type and config.catalog:
+            provider_config = getattr(config.catalog, catalog_type, None)
+            if provider_config:
+                catalog_config_dict = provider_config.model_dump(exclude_none=True)
+
         builder = GraphBuilder(
             node_format=node_format_enum,
             dialect=dialect,
             sql_preprocessor=sql_preprocessor,
+            no_star=no_star,
+            resolve_schema=resolve_schema,
+            catalog_type=catalog_type,
+            catalog_config=catalog_config_dict,
+            strict_schema=strict_schema,
         )
 
         # Process manifest if provided
@@ -1102,6 +1186,17 @@ def graph_build(
                     raise typer.Exit(1)
             builder.add_files(all_files, dialect=dialect)
 
+        # Dump resolved schema if requested
+        if dump_schema:
+            from sqlglider.graph.formatters import format_schema
+
+            schema_content = format_schema(builder.resolved_schema, dump_schema_format)
+            dump_schema.write_text(schema_content, encoding="utf-8")
+            console.print(
+                f"[green]Schema dumped to {dump_schema} "
+                f"({len(builder.resolved_schema)} table(s))[/green]"
+            )
+
         # Build and save graph
         graph = builder.build()
         save_graph(graph, output)
@@ -1111,6 +1206,10 @@ def graph_build(
             f"({graph.metadata.total_nodes} nodes, {graph.metadata.total_edges} edges)"
         )
 
+    except SchemaResolutionError as e:
+        err_console.print(f"[red]Error:[/red] {e}")
+        raise typer.Exit(1)
+
     except FileNotFoundError as e:
         err_console.print(f"[red]Error:[/red] {e}")
         raise typer.Exit(1)

{sql_glider-0.1.10 → sql_glider-0.1.12}/src/sqlglider/graph/builder.py

@@ -16,8 +16,9 @@ from sqlglider.graph.models import (
     LineageGraph,
     Manifest,
 )
-from sqlglider.lineage.analyzer import LineageAnalyzer
+from sqlglider.lineage.analyzer import LineageAnalyzer, SchemaResolutionError
 from sqlglider.utils.file_utils import read_sql_file
+from sqlglider.utils.schema import parse_ddl_to_schema
 
 console = Console(stderr=True)
 
@@ -33,6 +34,11 @@ class GraphBuilder:
         node_format: NodeFormat = NodeFormat.QUALIFIED,
         dialect: str = "spark",
         sql_preprocessor: Optional[SqlPreprocessor] = None,
+        no_star: bool = False,
+        resolve_schema: bool = False,
+        catalog_type: Optional[str] = None,
+        catalog_config: Optional[Dict[str, object]] = None,
+        strict_schema: bool = False,
     ):
         """
         Initialize the graph builder.
@@ -43,15 +49,32 @@ class GraphBuilder:
             sql_preprocessor: Optional function to preprocess SQL before analysis.
                              Takes (sql: str, file_path: Path) and returns processed SQL.
                              Useful for templating (e.g., Jinja2 rendering).
+            no_star: If True, fail when SELECT * cannot be resolved to columns
+            resolve_schema: If True, run a schema extraction pass across all
+                files before lineage analysis so that schema from any file is
+                available when analyzing every other file.
+            catalog_type: Optional catalog provider name (e.g. "databricks").
+                When set together with resolve_schema, DDL is pulled from the
+                catalog for tables whose schema could not be inferred from files.
+            catalog_config: Optional provider-specific configuration dict
+                passed to the catalog's configure() method.
+            strict_schema: If True, fail during schema extraction when an
+                unqualified column cannot be attributed to a table.
         """
         self.node_format = node_format
         self.dialect = dialect
         self.sql_preprocessor = sql_preprocessor
+        self.no_star = no_star
+        self.resolve_schema = resolve_schema
+        self.catalog_type = catalog_type
+        self.catalog_config = catalog_config
+        self.strict_schema = strict_schema
         self.graph: rx.PyDiGraph = rx.PyDiGraph()
         self._node_index_map: Dict[str, int] = {}  # identifier -> rustworkx node index
         self._source_files: Set[str] = set()
         self._edge_set: Set[tuple] = set()  # (source, target) for dedup
         self._skipped_files: List[tuple[str, str]] = []  # (file_path, reason)
+        self._resolved_schema: Dict[str, Dict[str, str]] = {}  # accumulated schema
 
     def add_file(
         self,
@@ -82,7 +105,12 @@ class GraphBuilder:
             if self.sql_preprocessor:
                 sql_content = self.sql_preprocessor(sql_content, file_path)
 
-            analyzer = LineageAnalyzer(sql_content, dialect=file_dialect)
+            analyzer = LineageAnalyzer(
+                sql_content,
+                dialect=file_dialect,
+                no_star=self.no_star,
+                schema=self._resolved_schema if self._resolved_schema else None,
+            )
             results = analyzer.analyze_queries(level=AnalysisLevel.COLUMN)
 
             # Print warnings for any skipped queries within the file
@@ -204,23 +232,37 @@ class GraphBuilder:
             entry_dialect = entry.dialect or dialect or self.dialect
             files_with_dialects.append((file_path, entry_dialect))
 
-        # Process with progress
-        if files_with_dialects:
-            total = len(files_with_dialects)
-            with Progress(
-                TextColumn("[progress.description]{task.description}"),
-                BarColumn(),
-                TaskProgressColumn(),
-                console=console,
-                transient=False,
-            ) as progress:
-                task = progress.add_task("Parsing", total=total)
-                for i, (file_path, file_dialect) in enumerate(
-                    files_with_dialects, start=1
-                ):
-                    console.print(f"Parsing file {i}/{total}: {file_path.name}")
-                    self.add_file(file_path, file_dialect)
-                    progress.advance(task)
+        if not files_with_dialects:
+            return self
+
+        # Two-pass schema resolution
+        if self.resolve_schema:
+            console.print("[blue]Pass 1: Extracting schema from files[/blue]")
+            file_paths_only = [fp for fp, _ in files_with_dialects]
+            self._resolved_schema = self._extract_schemas(file_paths_only, dialect)
+            if self.catalog_type:
+                self._resolved_schema = self._fill_schema_from_catalog(
+                    self._resolved_schema, file_paths_only, dialect
+                )
+            console.print(
+                f"[blue]Schema resolved for "
+                f"{len(self._resolved_schema)} table(s)[/blue]"
+            )
+
+        total = len(files_with_dialects)
+        description = "Pass 2: Analyzing lineage" if self.resolve_schema else "Parsing"
+        with Progress(
+            TextColumn("[progress.description]{task.description}"),
+            BarColumn(),
+            TaskProgressColumn(),
+            console=console,
+            transient=False,
+        ) as progress:
+            task = progress.add_task(description, total=total)
+            for i, (file_path, file_dialect) in enumerate(files_with_dialects, start=1):
+                console.print(f"Parsing file {i}/{total}: {file_path.name}")
+                self.add_file(file_path, file_dialect)
+                progress.advance(task)
 
         return self
 
@@ -244,8 +286,24 @@ class GraphBuilder:
         if not file_paths:
             return self
 
+        # Two-pass schema resolution: extract schema from all files first
+        if self.resolve_schema:
+            console.print("[blue]Pass 1: Extracting schema from files[/blue]")
+            self._resolved_schema = self._extract_schemas(file_paths, dialect)
+            if self.catalog_type:
+                self._resolved_schema = self._fill_schema_from_catalog(
+                    self._resolved_schema, file_paths, dialect
+                )
+            console.print(
+                f"[blue]Schema resolved for "
+                f"{len(self._resolved_schema)} table(s)[/blue]"
+            )
+
         if show_progress:
             total = len(file_paths)
+            description = (
+                "Pass 2: Analyzing lineage" if self.resolve_schema else "Parsing"
+            )
             with Progress(
                 TextColumn("[progress.description]{task.description}"),
                 BarColumn(),
@@ -253,7 +311,7 @@ class GraphBuilder:
                 console=console,
                 transient=False,
             ) as progress:
-                task = progress.add_task("Parsing", total=total)
+                task = progress.add_task(description, total=total)
                 for i, file_path in enumerate(file_paths, start=1):
                     console.print(f"Parsing file {i}/{total}: {file_path.name}")
                     self.add_file(file_path, dialect)
@@ -263,6 +321,129 @@ class GraphBuilder:
                 self.add_file(file_path, dialect)
         return self
 
+    def _extract_schemas(
+        self,
+        file_paths: List[Path],
+        dialect: Optional[str] = None,
+    ) -> Dict[str, Dict[str, str]]:
+        """Run schema extraction pass across all files.
+
+        Parses each file and extracts schema from CREATE TABLE/VIEW
+        statements without performing lineage analysis.
+
+        Args:
+            file_paths: SQL files to extract schema from
+            dialect: SQL dialect override
+
+        Returns:
+            Accumulated schema dict from all files
+        """
+        schema: Dict[str, Dict[str, str]] = {}
+        total = len(file_paths)
+        with Progress(
+            TextColumn("[progress.description]{task.description}"),
+            BarColumn(),
+            TaskProgressColumn(),
+            console=console,
+            transient=False,
+        ) as progress:
+            task = progress.add_task("Pass 1: Extracting schema", total=total)
+            for i, file_path in enumerate(file_paths, start=1):
+                console.print(f"Extracting schema {i}/{total}: {file_path.name}")
+                file_dialect = dialect or self.dialect
+                try:
+                    sql_content = read_sql_file(file_path)
+                    if self.sql_preprocessor:
+                        sql_content = self.sql_preprocessor(sql_content, file_path)
+                    analyzer = LineageAnalyzer(
+                        sql_content,
+                        dialect=file_dialect,
+                        schema=schema,
+                        strict_schema=self.strict_schema,
+                    )
+                    file_schema = analyzer.extract_schema_only()
+                    schema.update(file_schema)
+                except SchemaResolutionError:
+                    raise
+                except Exception:
+                    # Schema extraction failures are non-fatal; the file
+                    # will be reported during the lineage pass if it also fails.
+                    pass
+                progress.advance(task)
+        return schema
+
+    def _fill_schema_from_catalog(
+        self,
+        schema: Dict[str, Dict[str, str]],
+        file_paths: List[Path],
+        dialect: Optional[str] = None,
+    ) -> Dict[str, Dict[str, str]]:
+        """Pull DDL from catalog for tables not yet in schema.
+
+        Extracts all table names referenced across the files, identifies
+        those missing from the schema, and fetches their DDL from the
+        configured catalog provider.
+
+        Args:
+            schema: Schema dict already populated from file extraction
+            file_paths: SQL files to scan for table references
+            dialect: SQL dialect override
+
+        Returns:
+            Updated schema dict with catalog-sourced entries added
+        """
+        from sqlglider.catalog import get_catalog
+
+        catalog = get_catalog(self.catalog_type)  # type: ignore[arg-type]
+        if self.catalog_config:
+            catalog.configure(self.catalog_config)
+
+        # Collect all referenced table names across files
+        all_tables: Set[str] = set()
+        for file_path in file_paths:
+            file_dialect = dialect or self.dialect
+            try:
+                sql_content = read_sql_file(file_path)
+                if self.sql_preprocessor:
+                    sql_content = self.sql_preprocessor(sql_content, file_path)
+                analyzer = LineageAnalyzer(sql_content, dialect=file_dialect)
+                tables_results = analyzer.analyze_tables()
+                for result in tables_results:
+                    for table_info in result.tables:
+                        # Skip CTEs — they don't exist in catalogs
+                        from sqlglider.lineage.analyzer import ObjectType
+
+                        if table_info.object_type != ObjectType.CTE:
+                            all_tables.add(table_info.name)
+            except Exception:
+                pass
+
+        # Find tables missing from schema
+        missing = [t for t in all_tables if t not in schema]
+        if not missing:
+            return schema
+
+        console.print(
+            f"[blue]Pulling DDL from {self.catalog_type} "
+            f"for {len(missing)} table(s)...[/blue]"
+        )
+
+        ddl_results = catalog.get_ddl_batch(missing)
+        file_dialect = dialect or self.dialect
+        for table_name, ddl in ddl_results.items():
+            if ddl.startswith("ERROR:"):
+                console.print(
+                    f"[yellow]Warning:[/yellow] Could not pull DDL "
+                    f"for {table_name}: {ddl}"
+                )
+                continue
+            parsed_schema = parse_ddl_to_schema(ddl, dialect=file_dialect)
+            for name, cols in parsed_schema.items():
+                if name not in schema:
+                    schema[name] = cols
+
+        return schema
+
     def _ensure_node(
         self,
         identifier: str,
@@ -343,6 +524,11 @@ class GraphBuilder:
         """Get mapping from node identifiers to rustworkx indices."""
         return self._node_index_map.copy()
 
+    @property
+    def resolved_schema(self) -> Dict[str, Dict[str, str]]:
+        """Get the resolved schema dictionary from schema extraction pass."""
+        return self._resolved_schema.copy()
+
     @property
     def skipped_files(self) -> List[tuple[str, str]]:
         """Get list of files that were skipped during graph building."""