sql-glider 0.1.20__tar.gz → 0.1.23__tar.gz

sql_glider-0.1.23/.github/workflows/docs.yml

@@ -0,0 +1,54 @@
+name: Docs
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "docs/**"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v4
+        with:
+          version: "latest"
+
+      - name: Set up Python
+        run: uv python install 3.11
+
+      - name: Install dependencies
+        run: uv sync --group docs
+
+      - name: Build docs
+        run: uv run zensical build
+        working-directory: docs
+
+      - name: Upload Pages artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: docs/site
+
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

{sql_glider-0.1.20 → sql_glider-0.1.23}/.gitignore

@@ -151,8 +151,9 @@ venv.bak/
 # Rope project settings
 .ropeproject
 
-# mkdocs documentation
+# Documentation build output
 /site
+docs/site/
 
 # mypy
 .mypy_cache/

{sql_glider-0.1.20 → sql_glider-0.1.23}/ARCHITECTURE.md

@@ -23,6 +23,7 @@ sql-glider/
 │       │   ├── builder.py           # GraphBuilder for creating graphs from SQL
 │       │   ├── merge.py             # GraphMerger for combining graphs
 │       │   ├── query.py             # GraphQuerier for upstream/downstream analysis
+│       │   ├── diagram_formatters.py # Mermaid and DOT diagram output formatters
 │       │   └── serialization.py     # JSON save/load, rustworkx conversion
 │       ├── lineage/
 │       │   ├── __init__.py          # Lineage module exports
@@ -447,6 +448,15 @@ sqlglider graph merge --glob "*.json" -o merged.json
 # Query lineage
 sqlglider graph query graph.json --upstream orders.customer_id
 sqlglider graph query graph.json --downstream customers.id -f json
+
+# Query with diagram output (Mermaid or DOT)
+sqlglider graph query graph.json --upstream orders.customer_id -f mermaid
+sqlglider graph query graph.json --downstream customers.id -f dot
+
+# Visualize entire graph as diagram
+sqlglider graph visualize graph.json              # Mermaid (default)
+sqlglider graph visualize graph.json -f dot       # DOT/Graphviz
+sqlglider graph visualize graph.json -o lineage.mmd
 ```
 
 ### 5. Dissection Module (`dissection/`)

{sql_glider-0.1.20 → sql_glider-0.1.23}/CLAUDE.md

@@ -67,6 +67,7 @@ src/sqlglider/
 │   ├── builder.py            # Build lineage graphs from SQL files
 │   ├── merge.py              # Merge multiple graphs
 │   ├── query.py              # Query upstream/downstream lineage
+│   ├── diagram_formatters.py # Mermaid and DOT diagram output formatters
 │   ├── models.py             # Graph data models (Pydantic)
 │   └── serialization.py      # JSON save/load for graphs
 ├── lineage/
@@ -268,6 +269,22 @@ uv run sqlglider graph query graph.json --upstream orders.total -f json
 
 # Query with CSV output
 uv run sqlglider graph query graph.json --downstream customers.id -f csv
+
+# Query with Mermaid diagram output
+uv run sqlglider graph query graph.json --upstream orders.total -f mermaid
+
+# Query with DOT (Graphviz) diagram output
+uv run sqlglider graph query graph.json --downstream customers.id -f dot
+
+# Visualize entire graph as Mermaid diagram
+uv run sqlglider graph visualize graph.json
+
+# Visualize entire graph as DOT diagram
+uv run sqlglider graph visualize graph.json -f dot
+
+# Save diagram to file
+uv run sqlglider graph visualize graph.json -o lineage.mmd
+uv run sqlglider graph visualize graph.json -f dot -o lineage.dot
 ```
 
 ### SQL Templating

{sql_glider-0.1.20 → sql_glider-0.1.23}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sql-glider
-Version: 0.1.20
+Version: 0.1.23
 Summary: SQL Utility Toolkit for better understanding, use, and governance of your queries in a native environment.
 Project-URL: Homepage, https://github.com/rycowhi/sql-glider/
 Project-URL: Repository, https://github.com/rycowhi/sql-glider/
@@ -32,8 +32,12 @@ Description-Content-Type: text/markdown
 
 # SQL Glider
 
+![SQL Glider Logo](./docs/docs/static/sqlglider-logo-transparent.png)
+
 SQL Utility Toolkit for better understanding, use, and governance of your queries in a native environment.
 
+**[Read the docs](https://rycowhi.github.io/sql-glider/)**
+
 ## Overview
 
 SQL Glider provides powerful column-level and table-level lineage analysis for SQL queries using SQLGlot. It operates on standalone SQL files without requiring a full project setup, making it perfect for ad-hoc analysis, data governance, and understanding query dependencies.
@@ -409,6 +413,17 @@ uv run sqlglider graph query graph.json --upstream orders.customer_id
 
 # Query downstream dependencies (find all columns affected by a source)
 uv run sqlglider graph query graph.json --downstream customers.id
+
+# Query with Mermaid diagram output
+uv run sqlglider graph query graph.json --upstream orders.customer_id -f mermaid
+
+# Query with DOT (Graphviz) diagram output
+uv run sqlglider graph query graph.json --downstream customers.id -f dot
+
+# Visualize entire graph as a diagram
+uv run sqlglider graph visualize graph.json                    # Mermaid (default)
+uv run sqlglider graph visualize graph.json -f dot             # DOT/Graphviz
+uv run sqlglider graph visualize graph.json -o lineage.mmd     # Save to file
 ```
 
 **Example Upstream Query Output:**
@@ -796,6 +811,31 @@ src/sqlglider/
     └── file_utils.py         # File I/O utilities
 ```
 
+## Documentation
+
+Project documentation is built with [Zensical](https://zensical.org/) and lives in the `docs/` directory:
+
+```
+docs/
+├── zensical.toml    # Site configuration (name, theme, features)
+└── docs/            # Markdown content
+    └── index.md     # Landing page
+```
+
+To preview docs locally:
+
+```bash
+cd docs && uv run zensical serve
+```
+
+To build the static site:
+
+```bash
+cd docs && uv run zensical build
+```
+
+The built site outputs to `docs/site/` (git-ignored). Documentation is automatically deployed to GitHub Pages on pushes to `main` via the `docs.yml` workflow.
+
 ## Publishing
 
 SQL Glider is configured for publishing to both TestPyPI and PyPI using `uv`.

{sql_glider-0.1.20 → sql_glider-0.1.23}/README.md

@@ -1,7 +1,11 @@
 # SQL Glider
 
+![SQL Glider Logo](./docs/docs/static/sqlglider-logo-transparent.png)
+
 SQL Utility Toolkit for better understanding, use, and governance of your queries in a native environment.
 
+**[Read the docs](https://rycowhi.github.io/sql-glider/)**
+
 ## Overview
 
 SQL Glider provides powerful column-level and table-level lineage analysis for SQL queries using SQLGlot. It operates on standalone SQL files without requiring a full project setup, making it perfect for ad-hoc analysis, data governance, and understanding query dependencies.
@@ -377,6 +381,17 @@ uv run sqlglider graph query graph.json --upstream orders.customer_id
 
 # Query downstream dependencies (find all columns affected by a source)
 uv run sqlglider graph query graph.json --downstream customers.id
+
+# Query with Mermaid diagram output
+uv run sqlglider graph query graph.json --upstream orders.customer_id -f mermaid
+
+# Query with DOT (Graphviz) diagram output
+uv run sqlglider graph query graph.json --downstream customers.id -f dot
+
+# Visualize entire graph as a diagram
+uv run sqlglider graph visualize graph.json                    # Mermaid (default)
+uv run sqlglider graph visualize graph.json -f dot             # DOT/Graphviz
+uv run sqlglider graph visualize graph.json -o lineage.mmd     # Save to file
 ```
 
 **Example Upstream Query Output:**
@@ -764,6 +779,31 @@ src/sqlglider/
     └── file_utils.py         # File I/O utilities
 ```
 
+## Documentation
+
+Project documentation is built with [Zensical](https://zensical.org/) and lives in the `docs/` directory:
+
+```
+docs/
+├── zensical.toml    # Site configuration (name, theme, features)
+└── docs/            # Markdown content
+    └── index.md     # Landing page
+```
+
+To preview docs locally:
+
+```bash
+cd docs && uv run zensical serve
+```
+
+To build the static site:
+
+```bash
+cd docs && uv run zensical build
+```
+
+The built site outputs to `docs/site/` (git-ignored). Documentation is automatically deployed to GitHub Pages on pushes to `main` via the `docs.yml` workflow.
+
 ## Publishing
 
 SQL Glider is configured for publishing to both TestPyPI and PyPI using `uv`.

sql_glider-0.1.23/docs/.github/workflows/docs.yml

@@ -0,0 +1,29 @@
+name: Documentation
+on:
+  push:
+    branches:
+      - master
+      - main
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+jobs:
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/configure-pages@v5
+      - uses: actions/checkout@v5
+      - uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+      - run: pip install zensical
+      - run: zensical build --clean
+      - uses: actions/upload-pages-artifact@v4
+        with:
+          path: site
+      - uses: actions/deploy-pages@v4
+        id: deployment

sql_glider-0.1.23/docs/docs/catalogs.md

@@ -0,0 +1,190 @@
+---
+icon: lucide/database
+---
+
+# Catalogs
+
+SQL Glider can connect to remote data catalogs to pull DDL definitions for the tables referenced in your SQL. This is useful when you want to enrich lineage analysis with real schema information, validate queries against production table definitions, or simply collect DDL for documentation.
+
+## How It Works
+
+The `tables pull` command reads your SQL, extracts the table names, then fetches the `CREATE TABLE` DDL for each one from a remote catalog. CTEs are automatically excluded since they don't exist in any remote catalog.
+
+```bash
+# Pull DDL for all tables referenced in a query
+sqlglider tables pull query.sql --catalog-type databricks
+```
+
+Output:
+
+```sql
+-- Table: my_catalog.my_schema.customers
+CREATE TABLE my_catalog.my_schema.customers (
+    customer_id BIGINT,
+    name STRING,
+    email STRING,
+    created_at TIMESTAMP
+)
+USING DELTA;
+```
+
+You can also write each table's DDL to a separate file:
+
+```bash
+sqlglider tables pull query.sql -c databricks -o ./ddl/
+```
+
+This creates one `.sql` file per table in the `./ddl/` directory.
+
+## Built-in: Databricks
+
+SQL Glider ships with a Databricks Unity Catalog provider. It requires the Databricks SDK as an optional dependency:
+
+```bash
+pip install sql-glider[databricks]
+```
+
+### Authentication
+
+The Databricks catalog uses the Databricks SDK's unified authentication, which tries the following sources in order:
+
+1. Explicit host and token from `sqlglider.toml`
+2. Environment variables (`DATABRICKS_HOST`, `DATABRICKS_TOKEN`)
+3. Databricks CLI profile (`~/.databrickscfg`)
+4. OAuth M2M via `DATABRICKS_CLIENT_ID` / `DATABRICKS_CLIENT_SECRET`
+5. Azure CLI or Google Cloud auth for cloud-hosted workspaces
+
+A **warehouse ID** is always required. Set it in config or via the `DATABRICKS_WAREHOUSE_ID` environment variable.
+
+### Configuration
+
+Configure Databricks in `sqlglider.toml`:
+
+```toml
+[sqlglider]
+catalog_type = "databricks"
+
+[sqlglider.catalog.databricks]
+warehouse_id = "abc123def456"
+profile = "my-workspace"       # optional: Databricks CLI profile
+host = "https://my.databricks.com"  # optional if using profile or env vars
+token = "dapi..."              # optional: prefer OAuth or profile instead
+```
+
+Or use environment variables:
+
+```bash
+export DATABRICKS_HOST="https://my.databricks.com"
+export DATABRICKS_TOKEN="dapi..."
+export DATABRICKS_WAREHOUSE_ID="abc123def456"
+```
+
+### Usage
+
+```bash
+# Pull DDL to stdout
+sqlglider tables pull query.sql -c databricks
+
+# Pull DDL to folder
+sqlglider tables pull query.sql -c databricks -o ./ddl/
+
+# Combine with templating
+sqlglider tables pull query.sql -c databricks --templater jinja --var schema=prod
+
+# From stdin
+echo "SELECT * FROM my_catalog.my_schema.users" | sqlglider tables pull -c databricks
+
+# List available catalog providers
+sqlglider tables pull --list
+```
+
+## Writing a Custom Catalog Provider
+
+You can create your own catalog provider as a Python package and register it as a plugin.
+
+### 1. Implement the Catalog Class
+
+Subclass `sqlglider.catalog.base.Catalog` and implement three methods:
+
+```python
+from typing import Any, Dict, List, Optional
+
+from sqlglider.catalog.base import Catalog, CatalogError
+
+
+class SnowflakeCatalog(Catalog):
+    @property
+    def name(self) -> str:
+        return "snowflake"
+
+    def configure(self, config: Optional[Dict[str, Any]] = None) -> None:
+        """Set up connection details from config or environment."""
+        config = config or {}
+        self._account = config.get("account")
+        self._warehouse = config.get("warehouse")
+        # Validate required settings
+        if not self._account:
+            raise CatalogError("Snowflake account is required")
+
+    def get_ddl(self, table_name: str) -> str:
+        """Fetch DDL for a single table."""
+        try:
+            # Connect and run: GET_DDL('TABLE', table_name)
+            return ddl_string
+        except Exception as e:
+            raise CatalogError(f"Failed to fetch DDL for {table_name}: {e}") from e
+
+    def get_ddl_batch(self, table_names: List[str]) -> Dict[str, str]:
+        """Fetch DDL for multiple tables.
+
+        Return a dict mapping table names to DDL strings.
+        For tables that fail, prefix the value with "ERROR: ".
+        """
+        results: Dict[str, str] = {}
+        for table in table_names:
+            try:
+                results[table] = self.get_ddl(table)
+            except CatalogError as e:
+                results[table] = f"ERROR: {e}"
+        return results
+```
+
+Key points:
+
+- **`name`** — the identifier users pass to `--catalog-type`
+- **`configure()`** — called after instantiation with settings from `sqlglider.toml` and CLI; validate required config here
+- **`get_ddl()`** — fetch DDL for one table; raise `CatalogError` on failure
+- **`get_ddl_batch()`** — fetch DDL for many tables; prefix failures with `"ERROR: "` so the batch continues
+
+### 2. Register via Entry Points
+
+In your package's `pyproject.toml`:
+
+```toml
+[project.entry-points."sqlglider.catalogs"]
+snowflake = "my_package.catalog:SnowflakeCatalog"
+```
+
+### 3. Add Configuration Support
+
+Users can configure your catalog in `sqlglider.toml`. The config section name matches the catalog name:
+
+```toml
+[sqlglider]
+catalog_type = "snowflake"
+
+[sqlglider.catalog.snowflake]
+account = "xy12345.us-east-1"
+warehouse = "COMPUTE_WH"
+```
+
+!!! note
+    For the config section to be loaded automatically, the core `ConfigSettings` model needs to know about your provider. For third-party plugins, users can also pass configuration through environment variables in your `configure()` method.
+
+### 4. Use It
+
+After installing your package:
+
+```bash
+sqlglider tables pull query.sql --catalog-type snowflake
+```