sqlnbfmt 0.2.0__tar.gz → 0.3.1__tar.gz

{sqlnbfmt-0.2.0/sqlnbfmt.egg-info → sqlnbfmt-0.3.1}/PKG-INFO

@@ -1,9 +1,14 @@
 Metadata-Version: 2.4
 Name: sqlnbfmt
-Version: 0.2.0
+Version: 0.3.1
 Summary: A tool to format SQL code in Jupyter notebooks.
 Author-email: Qing <qingye779@gmail.com>
 License: MIT
+Project-URL: Homepage, https://github.com/flyersworder/sqlnbfmt
+Project-URL: Repository, https://github.com/flyersworder/sqlnbfmt
+Project-URL: Changelog, https://github.com/flyersworder/sqlnbfmt/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/flyersworder/sqlnbfmt/issues
+Keywords: sql,jupyter,notebook,formatter,sqlglot,pre-commit
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
@@ -21,16 +26,20 @@ Dynamic: license-file
 [![License](https://img.shields.io/pypi/l/sqlnbfmt.svg)](https://github.com/flyersworder/sqlnbfmt/blob/main/LICENSE)
 [![Python Versions](https://img.shields.io/pypi/pyversions/sqlnbfmt.svg)](https://pypi.org/project/sqlnbfmt/)
 
+> **Quick start:** `pip install sqlnbfmt && sqlnbfmt notebook.ipynb`
+
 A SQL formatter designed specifically for Jupyter Notebooks. `sqlnbfmt` automatically formats SQL queries embedded in code cells, including both Python strings and SQL magic cells (`%%sql`), helping you maintain clean and consistent code.
 
 ## Features
 
-- 🎯 **Smart SQL Detection**: Automatically identifies and formats SQL queries in code cells and magic SQL cells
-- 🌳 **AST-Powered**: Uses Abstract Syntax Tree parsing for accurate SQL string identification
-- 🔒 **Safe Formatting**: Preserves Python comments, query parameters (e.g., `%s`, `?`), and SQL comments
-- ⚙️ **Highly Configurable**: Customize formatting through YAML configuration
-- 🔄 **Pre-commit Ready**: Seamlessly integrates with pre-commit hooks
-- 📦 **Lightweight**: Only three runtime dependencies (sqlglot, nbformat, pyyaml)
+- **Zero-config**: Works out of the box with sensible defaults — no config file needed
+- **Smart SQL Detection**: Automatically identifies and formats SQL queries in code cells and magic SQL cells
+- **AST-Powered**: Uses Abstract Syntax Tree parsing for accurate SQL string identification
+- **Safe Formatting**: Preserves Python comments, query parameters (e.g., `%s`, `?`), and SQL comments
+- **CI-Friendly**: `--check` mode exits non-zero when formatting is needed; `--diff` shows what would change
+- **Skip Hints**: Add `# sqlnbfmt: skip` to any cell to exclude it from formatting
+- **Pre-commit Ready**: Seamlessly integrates with pre-commit hooks
+- **Lightweight**: Only three runtime dependencies (sqlglot, nbformat, pyyaml)
 
 ## Installation
 
@@ -47,9 +56,23 @@ Format a single notebook:
 sqlnbfmt path/to/your_notebook.ipynb
 ```
 
-Format all notebooks in a directory:
+Check formatting without modifying files (useful in CI):
+```bash
+sqlnbfmt --check path/to/your_notebook.ipynb
+```
+
+Show a diff of what would change:
 ```bash
-sqlnbfmt path/to/notebooks/
+sqlnbfmt --diff path/to/your_notebook.ipynb
+```
+
+### Skipping Cells
+
+Add a `# sqlnbfmt: skip` comment anywhere in a cell to skip formatting:
+
+```python
+# sqlnbfmt: skip
+query = "select * from my_special_table where id = 1"
 ```
 
 ### Pre-commit Integration
@@ -63,17 +86,15 @@ pip install pre-commit
 ```yaml
 repos:
   - repo: https://github.com/flyersworder/sqlnbfmt
-    rev: v0.2.0
+    rev: v0.3.0
     hooks:
       - id: sqlnbfmt
-        name: sqlnbfmt
         types: [jupyter]
-        args: [--config, config.yaml, --dialect, postgres]
 ```
-Please run the following command for help:
 
-```bash
-sqlnbfmt -h
+All arguments are optional. To specify a dialect or custom config:
+```yaml
+        args: [--dialect, postgres, --config, config.yaml]
 ```
 
 3. Install the hook:
@@ -86,8 +107,21 @@ pre-commit install
 pre-commit run --all-files
 ```
 
+### CI Usage
+
+Use `--check` in GitHub Actions to enforce formatting:
+
+```yaml
+- name: Check SQL notebook formatting
+  run: |
+    pip install sqlnbfmt
+    sqlnbfmt --check **/*.ipynb
+```
+
 ## Configuration
 
+`sqlnbfmt` works without any configuration file. A `config.yaml` is only needed to override defaults.
+
 Create a `config.yaml` file to customize formatting behavior. [Here](https://github.com/flyersworder/sqlnbfmt/blob/main/config.yaml) is a template.
 
 ### Configuration Options
@@ -95,8 +129,8 @@ Create a `config.yaml` file to customize formatting behavior. [Here](https://git
 | Option | Description | Default |
 |--------|-------------|---------|
 | `sql_keywords` | SQL keywords to recognize and format | Common SQL keywords |
-| `function_names` | Python functions containing SQL code | `[]` |
-| `sql_decorators` | Decorators indicating SQL code | `[]` |
+| `function_names` | Python functions containing SQL code | `read_sql`, `execute`, etc. |
+| `sql_decorators` | Decorators indicating SQL code | `query`, `sql_query`, etc. |
 | `single_line_threshold` | Maximum length before splitting SQL | 80 |
 | `indent_width` | Number of spaces for indentation | 4 |
 
@@ -125,6 +159,19 @@ ORDER BY
 """)
 ```
 
+## Troubleshooting
+
+**SQL not being formatted?**
+- Ensure the string contains at least 2 SQL keywords or a recognizable pattern like `SELECT...FROM`
+- Check that the function name is in the recognized list (use `--config` to add custom ones)
+
+**Comments being modified?**
+- Python comments (`#`) are preserved. SQL comments (`--`) inside strings are converted to `/* */` block comments by sqlglot.
+
+**Pre-commit hook fails?**
+- Make sure the `rev` matches the installed version
+- Run `pre-commit autoupdate` to get the latest version
+
 ## Contributing
 
 We welcome contributions! Here's how to get started:
@@ -146,6 +193,13 @@ source .venv/bin/activate  # On Windows: .venv\Scripts\activate
 pytest
 ```
 
+4. Add eval cases: see `tests/eval/generate_fixtures.py` for examples. Run `python tests/eval/generate_fixtures.py` to regenerate fixtures.
+
+5. Install dev pre-commit hooks:
+```bash
+pre-commit install
+```
+
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

{sqlnbfmt-0.2.0 → sqlnbfmt-0.3.1}/README.md

@@ -4,16 +4,20 @@
 [![License](https://img.shields.io/pypi/l/sqlnbfmt.svg)](https://github.com/flyersworder/sqlnbfmt/blob/main/LICENSE)
 [![Python Versions](https://img.shields.io/pypi/pyversions/sqlnbfmt.svg)](https://pypi.org/project/sqlnbfmt/)
 
+> **Quick start:** `pip install sqlnbfmt && sqlnbfmt notebook.ipynb`
+
 A SQL formatter designed specifically for Jupyter Notebooks. `sqlnbfmt` automatically formats SQL queries embedded in code cells, including both Python strings and SQL magic cells (`%%sql`), helping you maintain clean and consistent code.
 
 ## Features
 
-- 🎯 **Smart SQL Detection**: Automatically identifies and formats SQL queries in code cells and magic SQL cells
-- 🌳 **AST-Powered**: Uses Abstract Syntax Tree parsing for accurate SQL string identification
-- 🔒 **Safe Formatting**: Preserves Python comments, query parameters (e.g., `%s`, `?`), and SQL comments
-- ⚙️ **Highly Configurable**: Customize formatting through YAML configuration
-- 🔄 **Pre-commit Ready**: Seamlessly integrates with pre-commit hooks
-- 📦 **Lightweight**: Only three runtime dependencies (sqlglot, nbformat, pyyaml)
+- **Zero-config**: Works out of the box with sensible defaults — no config file needed
+- **Smart SQL Detection**: Automatically identifies and formats SQL queries in code cells and magic SQL cells
+- **AST-Powered**: Uses Abstract Syntax Tree parsing for accurate SQL string identification
+- **Safe Formatting**: Preserves Python comments, query parameters (e.g., `%s`, `?`), and SQL comments
+- **CI-Friendly**: `--check` mode exits non-zero when formatting is needed; `--diff` shows what would change
+- **Skip Hints**: Add `# sqlnbfmt: skip` to any cell to exclude it from formatting
+- **Pre-commit Ready**: Seamlessly integrates with pre-commit hooks
+- **Lightweight**: Only three runtime dependencies (sqlglot, nbformat, pyyaml)
 
 ## Installation
 
@@ -30,9 +34,23 @@ Format a single notebook:
 sqlnbfmt path/to/your_notebook.ipynb
 ```
 
-Format all notebooks in a directory:
+Check formatting without modifying files (useful in CI):
+```bash
+sqlnbfmt --check path/to/your_notebook.ipynb
+```
+
+Show a diff of what would change:
 ```bash
-sqlnbfmt path/to/notebooks/
+sqlnbfmt --diff path/to/your_notebook.ipynb
+```
+
+### Skipping Cells
+
+Add a `# sqlnbfmt: skip` comment anywhere in a cell to skip formatting:
+
+```python
+# sqlnbfmt: skip
+query = "select * from my_special_table where id = 1"
 ```
 
 ### Pre-commit Integration
@@ -46,17 +64,15 @@ pip install pre-commit
 ```yaml
 repos:
   - repo: https://github.com/flyersworder/sqlnbfmt
-    rev: v0.2.0
+    rev: v0.3.0
     hooks:
       - id: sqlnbfmt
-        name: sqlnbfmt
         types: [jupyter]
-        args: [--config, config.yaml, --dialect, postgres]
 ```
-Please run the following command for help:
 
-```bash
-sqlnbfmt -h
+All arguments are optional. To specify a dialect or custom config:
+```yaml
+        args: [--dialect, postgres, --config, config.yaml]
 ```
 
 3. Install the hook:
@@ -69,8 +85,21 @@ pre-commit install
 pre-commit run --all-files
 ```
 
+### CI Usage
+
+Use `--check` in GitHub Actions to enforce formatting:
+
+```yaml
+- name: Check SQL notebook formatting
+  run: |
+    pip install sqlnbfmt
+    sqlnbfmt --check **/*.ipynb
+```
+
 ## Configuration
 
+`sqlnbfmt` works without any configuration file. A `config.yaml` is only needed to override defaults.
+
 Create a `config.yaml` file to customize formatting behavior. [Here](https://github.com/flyersworder/sqlnbfmt/blob/main/config.yaml) is a template.
 
 ### Configuration Options
@@ -78,8 +107,8 @@ Create a `config.yaml` file to customize formatting behavior. [Here](https://git
 | Option | Description | Default |
 |--------|-------------|---------|
 | `sql_keywords` | SQL keywords to recognize and format | Common SQL keywords |
-| `function_names` | Python functions containing SQL code | `[]` |
-| `sql_decorators` | Decorators indicating SQL code | `[]` |
+| `function_names` | Python functions containing SQL code | `read_sql`, `execute`, etc. |
+| `sql_decorators` | Decorators indicating SQL code | `query`, `sql_query`, etc. |
 | `single_line_threshold` | Maximum length before splitting SQL | 80 |
 | `indent_width` | Number of spaces for indentation | 4 |
 
@@ -108,6 +137,19 @@ ORDER BY
 """)
 ```
 
+## Troubleshooting
+
+**SQL not being formatted?**
+- Ensure the string contains at least 2 SQL keywords or a recognizable pattern like `SELECT...FROM`
+- Check that the function name is in the recognized list (use `--config` to add custom ones)
+
+**Comments being modified?**
+- Python comments (`#`) are preserved. SQL comments (`--`) inside strings are converted to `/* */` block comments by sqlglot.
+
+**Pre-commit hook fails?**
+- Make sure the `rev` matches the installed version
+- Run `pre-commit autoupdate` to get the latest version
+
 ## Contributing
 
 We welcome contributions! Here's how to get started:
@@ -129,6 +171,13 @@ source .venv/bin/activate  # On Windows: .venv\Scripts\activate
 pytest
 ```
 
+4. Add eval cases: see `tests/eval/generate_fixtures.py` for examples. Run `python tests/eval/generate_fixtures.py` to regenerate fixtures.
+
+5. Install dev pre-commit hooks:
+```bash
+pre-commit install
+```
+
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

{sqlnbfmt-0.2.0 → sqlnbfmt-0.3.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "sqlnbfmt"
-version = "0.2.0"
+version = "0.3.1"
 description = "A tool to format SQL code in Jupyter notebooks."
 authors = [
   { name = "Qing", email = "qingye779@gmail.com" }
@@ -8,6 +8,7 @@ authors = [
 license = { text = "MIT" }
 readme = "README.md"
 requires-python = ">=3.12"
+keywords = ["sql", "jupyter", "notebook", "formatter", "sqlglot", "pre-commit"]
 classifiers = [
     "Programming Language :: Python :: 3",
     "Programming Language :: Python :: 3.12",
@@ -19,6 +20,15 @@ dependencies = [
     "sqlglot>=25.27.0",
 ]
 
+[project.urls]
+Homepage = "https://github.com/flyersworder/sqlnbfmt"
+Repository = "https://github.com/flyersworder/sqlnbfmt"
+Changelog = "https://github.com/flyersworder/sqlnbfmt/blob/main/CHANGELOG.md"
+Issues = "https://github.com/flyersworder/sqlnbfmt/issues"
+
+[project.scripts]
+sqlnbfmt = "sqlnbfmt.formatter:main"
+
 [dependency-groups]
 dev = [
     "pytest-cov>=5.0.0",
@@ -30,9 +40,6 @@ dev = [
 requires = ["setuptools>=61.0", "wheel"]
 build-backend = "setuptools.build_meta"
 
-[project.scripts]
-sqlnbfmt = "sqlnbfmt.formatter:main"
-
 [tool.pytest.ini_options]
 markers = [
     "integration: mark a test as an integration test."

{sqlnbfmt-0.2.0 → sqlnbfmt-0.3.1}/sqlnbfmt/formatter.py

@@ -1,8 +1,10 @@
 import ast
+import copy
+import difflib
 import logging
 import re
 import sys
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Set, Optional, Dict, Union, Tuple
 
@@ -10,6 +12,55 @@ import nbformat
 import yaml
 from sqlglot import parse_one, errors
 
+DEFAULT_SQL_KEYWORDS = frozenset(
+    {
+        "SELECT",
+        "FROM",
+        "WHERE",
+        "INSERT",
+        "UPDATE",
+        "DELETE",
+        "WITH",
+        "JOIN",
+        "CREATE",
+        "DROP",
+        "ALTER",
+        "TRUNCATE",
+        "UNION",
+        "EXCEPT",
+        "INTERSECT",
+        "GROUP BY",
+        "ORDER BY",
+        "HAVING",
+        "LIMIT",
+        "OFFSET",
+        "DISTINCT",
+    }
+)
+
+DEFAULT_FUNCTION_NAMES = frozenset(
+    {
+        "read_sql",
+        "read_sql_query",
+        "read_sql_table",
+        "execute",
+        "executescript",
+        "fetchall",
+        "fetchone",
+        "fetchmany",
+        "execute_query",
+    }
+)
+
+DEFAULT_SQL_DECORATORS = frozenset(
+    {
+        "sql_decorator",
+        "query",
+        "sql_query",
+        "db_query",
+    }
+)
+
 
 @dataclass
 class SQLReplacement:
@@ -24,9 +75,13 @@ class SQLReplacement:
 class FormattingConfig:
     """Configuration for SQL formatting."""
 
-    sql_keywords: Set[str]
-    function_names: Set[str]
-    sql_decorators: Set[str]
+    sql_keywords: Set[str] = field(default_factory=lambda: set(DEFAULT_SQL_KEYWORDS))
+    function_names: Set[str] = field(
+        default_factory=lambda: set(DEFAULT_FUNCTION_NAMES)
+    )
+    sql_decorators: Set[str] = field(
+        default_factory=lambda: set(DEFAULT_SQL_DECORATORS)
+    )
     single_line_threshold: int = 80
     indent_width: int = 4
 
@@ -49,8 +104,11 @@ def setup_logging(level: str = "INFO") -> logging.Logger:
     return logger
 
 
-def load_config(config_path: Union[str, Path] = "config.yaml") -> FormattingConfig:
-    """Loads configuration from a YAML file."""
+def load_config(config_path: Optional[Union[str, Path]] = None) -> FormattingConfig:
+    """Loads configuration from a YAML file, or returns defaults if no path given."""
+    if config_path is None:
+        return FormattingConfig()
+
     config_file = Path(config_path)
     if not config_file.is_file():
         raise FileNotFoundError(f"Configuration file '{config_file}' not found.")
@@ -58,10 +116,16 @@ def load_config(config_path: Union[str, Path] = "config.yaml") -> FormattingConf
     try:
         with open(config_file) as file:
             config = yaml.safe_load(file)
+            if config is None:
+                return FormattingConfig()
             return FormattingConfig(
-                sql_keywords=set(config.get("sql_keywords", [])),
-                function_names=set(config.get("function_names", [])),
-                sql_decorators=set(config.get("sql_decorators", [])),
+                sql_keywords=set(config.get("sql_keywords", DEFAULT_SQL_KEYWORDS)),
+                function_names=set(
+                    config.get("function_names", DEFAULT_FUNCTION_NAMES)
+                ),
+                sql_decorators=set(
+                    config.get("sql_decorators", DEFAULT_SQL_DECORATORS)
+                ),
                 single_line_threshold=config.get("formatting_options", {}).get(
                     "single_line_threshold", 80
                 ),
@@ -314,7 +378,7 @@ class SQLStringFormatter(ast.NodeVisitor):
         """Constructs the replacement string literal text."""
         prefix = "f" if is_fstring else ""
         if in_function and "\n" in formatted_sql:
-            indent = " " * 4
+            indent = " " * self.config.indent_width
             lines = formatted_sql.split("\n")
             indented_lines = [indent + line if line.strip() else line for line in lines]
             indented_sql = "\n".join(indented_lines)
@@ -369,119 +433,159 @@ class SQLStringFormatter(ast.NodeVisitor):
         return ".".join(reversed(parts))
 
 
-def process_notebook(
-    notebook_path: Union[str, Path],
+def _has_skip_hint(source: str) -> bool:
+    """Check if any comment line contains the sqlnbfmt: skip directive.
+
+    Only dedicated comment lines are recognised (the line must start with ``#``
+    after stripping whitespace).  Inline trailing comments like
+    ``x = 1  # sqlnbfmt: skip`` are intentionally ignored so that the directive
+    cannot be triggered accidentally from within an expression or string.
+    """
+    return any(
+        line.strip().startswith("#") and "sqlnbfmt: skip" in line
+        for line in source.split("\n")
+    )
+
+
+def _format_cells(
+    notebook: nbformat.NotebookNode,
     config: FormattingConfig,
     dialect: Optional[str],
     logger: logging.Logger,
 ) -> bool:
-    """Processes a Jupyter notebook."""
-    try:
-        notebook = nbformat.read(notebook_path, as_version=4)
-        changed = False
+    """Format cells in-place. Returns True if any cell was modified."""
+    changed = False
 
-        for cell in notebook.cells:
-            if cell.cell_type != "code":
-                continue
+    for cell_index, cell in enumerate(notebook.cells):
+        if cell.cell_type != "code":
+            continue
 
-            original_code = cell.source
-            if not original_code.strip():
-                continue
+        original_code = cell.source
+        if not original_code.strip():
+            continue
 
-            lines = original_code.split("\n")
+        if _has_skip_hint(original_code):
+            logger.debug(f"[Cell {cell_index}] Skipping cell (sqlnbfmt: skip)")
+            continue
 
-            # Initialize variables to track magic commands
-            magic_cmd = None
-            magic_cmd_index = None
+        lines = original_code.split("\n")
 
-            # Iterate through lines to find the first non-comment magic command
-            for idx, line in enumerate(lines):
-                stripped = line.strip()
+        # Initialize variables to track magic commands
+        magic_cmd = None
+        magic_cmd_index = None
 
-                if not stripped:
-                    continue  # Skip empty lines
+        # Iterate through lines to find the first non-comment magic command
+        for idx, line in enumerate(lines):
+            stripped = line.strip()
 
-                if stripped.startswith("#"):
-                    continue  # Skip comment lines
+            if not stripped:
+                continue  # Skip empty lines
 
-                if stripped.startswith("%%sql") or stripped.startswith("%sql"):
-                    magic_cmd = stripped.split()[0]
-                    magic_cmd_index = idx
-                    break  # Magic command found
+            if stripped.startswith("#"):
+                continue  # Skip comment lines
 
-                else:
-                    break  # Non-magic, non-comment line found
+            if stripped.startswith("%%sql") or stripped.startswith("%sql"):
+                magic_cmd = stripped.split()[0]
+                magic_cmd_index = idx
+                break  # Magic command found
+
+            else:
+                break  # Non-magic, non-comment line found
 
-            if magic_cmd:
-                is_cell_magic = magic_cmd.startswith("%%sql")
+        if magic_cmd:
+            is_cell_magic = magic_cmd.startswith("%%sql")
 
+            if is_cell_magic:
+                # Cell magic: SQL code starts from the next line
+                sql_code = "\n".join(lines[magic_cmd_index + 1 :]).strip()
+            else:
+                # Line magic: SQL code is on the same line after the magic command
+                sql_code = lines[magic_cmd_index][len(magic_cmd) :].strip()
+
+            try:
+                formatted_sql = format_sql_code(
+                    sql_code,
+                    dialect,
+                    config,
+                    is_magic_command=True,
+                    is_cell_magic=is_cell_magic,
+                )
+
+                # Reconstruct the cell content
                 if is_cell_magic:
-                    # Cell magic: SQL code starts from the next line
-                    sql_code = "\n".join(lines[magic_cmd_index + 1 :]).strip()
+                    # Preserve comments before the magic command
+                    pre_magic = "\n".join(lines[:magic_cmd_index])
+                    if pre_magic:
+                        new_content = f"{pre_magic}\n{magic_cmd}\n{formatted_sql}"
+                    else:
+                        new_content = f"{magic_cmd}\n{formatted_sql}"
                 else:
-                    # Line magic: SQL code is on the same line after the magic command
-                    sql_code = lines[magic_cmd_index][len(magic_cmd) :].strip()
-
-                try:
-                    formatted_sql = format_sql_code(
-                        sql_code,
-                        dialect,
-                        config,
-                        is_magic_command=True,
-                        is_cell_magic=is_cell_magic,
+                    # Preserve comments before the magic command
+                    pre_magic = "\n".join(lines[:magic_cmd_index])
+                    if pre_magic:
+                        new_content = f"{pre_magic}\n{magic_cmd} {formatted_sql}"
+                    else:
+                        new_content = f"{magic_cmd} {formatted_sql}"
+
+                if new_content != original_code:
+                    cell.source = new_content
+                    changed = True
+
+            except SQLFormattingError as e:
+                logger.warning(f"[Cell {cell_index}] SQL magic formatting skipped: {e}")
+
+            continue  # Move to the next cell after handling magic command
+
+        # Handle regular Python code with surgical string replacement
+        try:
+            tree = ast.parse(original_code)
+            line_offsets = build_line_offsets(original_code)
+            formatter = SQLStringFormatter(config, dialect, logger, line_offsets)
+            formatter.visit(tree)
+            if formatter.replacements:
+                result = original_code
+                for rep in sorted(
+                    formatter.replacements,
+                    key=lambda r: r.start_offset,
+                    reverse=True,
+                ):
+                    result = (
+                        result[: rep.start_offset]
+                        + rep.new_text
+                        + result[rep.end_offset :]
                     )
+                if result != original_code:
+                    cell.source = result
+                    changed = True
+        except SyntaxError:
+            logger.warning(
+                f"[Cell {cell_index}] Failed to parse cell:\n{original_code}"
+            )
+            continue
 
-                    # Reconstruct the cell content
-                    if is_cell_magic:
-                        # Preserve comments before the magic command
-                        pre_magic = "\n".join(lines[:magic_cmd_index])
-                        if pre_magic:
-                            new_content = f"{pre_magic}\n{magic_cmd}\n{formatted_sql}"
-                        else:
-                            new_content = f"{magic_cmd}\n{formatted_sql}"
-                    else:
-                        # Preserve comments before the magic command
-                        pre_magic = "\n".join(lines[:magic_cmd_index])
-                        if pre_magic:
-                            new_content = f"{pre_magic}\n{magic_cmd} {formatted_sql}"
-                        else:
-                            new_content = f"{magic_cmd} {formatted_sql}"
+    return changed
 
-                    if new_content != original_code:
-                        cell.source = new_content
-                        changed = True
 
-                except SQLFormattingError as e:
-                    logger.warning(f"SQL magic formatting skipped: {e}")
+def process_notebook(
+    notebook_path: Union[str, Path],
+    config: FormattingConfig,
+    dialect: Optional[str],
+    logger: logging.Logger,
+    check_only: bool = False,
+) -> bool:
+    """Processes a Jupyter notebook.
 
-                continue  # Move to the next cell after handling magic command
+    Args:
+        check_only: If True, detect changes but don't write the file.
 
-            # Handle regular Python code with surgical string replacement
-            try:
-                tree = ast.parse(original_code)
-                line_offsets = build_line_offsets(original_code)
-                formatter = SQLStringFormatter(config, dialect, logger, line_offsets)
-                formatter.visit(tree)
-                if formatter.replacements:
-                    result = original_code
-                    for rep in sorted(
-                        formatter.replacements,
-                        key=lambda r: r.start_offset,
-                        reverse=True,
-                    ):
-                        result = (
-                            result[: rep.start_offset]
-                            + rep.new_text
-                            + result[rep.end_offset :]
-                        )
-                    if result != original_code:
-                        cell.source = result
-                        changed = True
-            except SyntaxError:
-                logger.warning(f"Failed to parse cell:\n{original_code}")
-                continue
-
-        if changed:
+    Returns:
+        True if changes were made (or would be made in check mode).
+    """
+    try:
+        notebook = nbformat.read(notebook_path, as_version=4)
+        changed = _format_cells(notebook, config, dialect, logger)
+
+        if changed and not check_only:
             nbformat.write(notebook, notebook_path)
             logger.info(f"Updated notebook: {notebook_path}")
 
@@ -492,6 +596,36 @@ def process_notebook(
         raise
 
 
+def diff_notebook(
+    notebook_path: Union[str, Path],
+    config: FormattingConfig,
+    dialect: Optional[str],
+    logger: logging.Logger,
+) -> str:
+    """Returns a unified diff of formatting changes for a notebook."""
+    notebook_path = Path(notebook_path)
+    original_nb = nbformat.read(notebook_path, as_version=4)
+    formatted_nb = copy.deepcopy(original_nb)
+
+    _format_cells(formatted_nb, config, dialect, logger)
+
+    # Build unified diff cell-by-cell
+    diff_lines = []
+    for i, (orig_cell, fmt_cell) in enumerate(
+        zip(original_nb.cells, formatted_nb.cells)
+    ):
+        if orig_cell.source != fmt_cell.source:
+            diff_lines.extend(
+                difflib.unified_diff(
+                    orig_cell.source.splitlines(keepends=True),
+                    fmt_cell.source.splitlines(keepends=True),
+                    fromfile=f"{notebook_path} Cell [{i}] original",
+                    tofile=f"{notebook_path} Cell [{i}] formatted",
+                )
+            )
+    return "".join(diff_lines)
+
+
 def main():
     """Main entry point for the SQL formatter."""
     import argparse
@@ -501,7 +635,7 @@ def main():
         "notebooks", nargs="+", type=Path, help="Notebook paths to process"
     )
     parser.add_argument(
-        "--config", type=Path, default="config.yaml", help="Configuration file path"
+        "--config", type=Path, default=None, help="Configuration file path (optional)"
     )
     parser.add_argument(
         "--dialect", type=str, help="SQL dialect (e.g., mysql, postgres)"
@@ -512,27 +646,66 @@ def main():
         default="INFO",
         help="Logging level",
     )
+    parser.add_argument(
+        "--check",
+        action="store_true",
+        help="Check formatting without modifying files (exit code 1 if changes needed)",
+    )
+    parser.add_argument(
+        "--diff",
+        action="store_true",
+        help="Print a unified diff of formatting changes",
+    )
 
     args = parser.parse_args()
     logger = setup_logging(args.log_level)
 
     try:
         config = load_config(args.config)
-        changed = False
+    except FileNotFoundError as e:
+        logger.error(
+            f"{e}\nHint: --config is optional; omit it to use built-in defaults."
+        )
+        sys.exit(1)
+
+    try:
         changed_files = []
 
         for notebook in args.notebooks:
-            if process_notebook(notebook, config, args.dialect, logger):
-                changed = True
-                changed_files.append(notebook)
-
-        # Print summary
-        if changed:
-            logger.info("Changes made to the following notebooks:")
-            for file in changed_files:
-                logger.info(f"  - {file}")
-        else:
-            logger.info("No changes needed. All notebooks are properly formatted.")
+            if args.check or args.diff:
+                # In check/diff mode, use diff_notebook to get both the
+                # diff output and whether changes are needed (single pass).
+                diff_output = diff_notebook(notebook, config, args.dialect, logger)
+                needs_formatting = bool(diff_output)
+
+                if args.diff and diff_output:
+                    print(diff_output, end="")
+
+                if needs_formatting:
+                    changed_files.append(notebook)
+            else:
+                # Normal mode: format in place
+                if process_notebook(notebook, config, args.dialect, logger):
+                    changed_files.append(notebook)
+
+        if args.check:
+            if changed_files:
+                logger.error(f"{len(changed_files)} notebook(s) would be reformatted:")
+                for f in changed_files:
+                    logger.error(f"  - {f}")
+                sys.exit(1)
+            else:
+                logger.info("All notebooks are properly formatted.")
+                sys.exit(0)
+
+        # Print summary for normal mode
+        if not args.diff:
+            if changed_files:
+                logger.info("Changes made to the following notebooks:")
+                for file in changed_files:
+                    logger.info(f"  - {file}")
+            else:
+                logger.info("No changes needed. All notebooks are properly formatted.")
 
         sys.exit(0)
 

{sqlnbfmt-0.2.0 → sqlnbfmt-0.3.1/sqlnbfmt.egg-info}/PKG-INFO

@@ -1,9 +1,14 @@
 Metadata-Version: 2.4
 Name: sqlnbfmt
-Version: 0.2.0
+Version: 0.3.1
 Summary: A tool to format SQL code in Jupyter notebooks.
 Author-email: Qing <qingye779@gmail.com>
 License: MIT
+Project-URL: Homepage, https://github.com/flyersworder/sqlnbfmt
+Project-URL: Repository, https://github.com/flyersworder/sqlnbfmt
+Project-URL: Changelog, https://github.com/flyersworder/sqlnbfmt/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/flyersworder/sqlnbfmt/issues
+Keywords: sql,jupyter,notebook,formatter,sqlglot,pre-commit
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
@@ -21,16 +26,20 @@ Dynamic: license-file
 [![License](https://img.shields.io/pypi/l/sqlnbfmt.svg)](https://github.com/flyersworder/sqlnbfmt/blob/main/LICENSE)
 [![Python Versions](https://img.shields.io/pypi/pyversions/sqlnbfmt.svg)](https://pypi.org/project/sqlnbfmt/)
 
+> **Quick start:** `pip install sqlnbfmt && sqlnbfmt notebook.ipynb`
+
 A SQL formatter designed specifically for Jupyter Notebooks. `sqlnbfmt` automatically formats SQL queries embedded in code cells, including both Python strings and SQL magic cells (`%%sql`), helping you maintain clean and consistent code.
 
 ## Features
 
-- 🎯 **Smart SQL Detection**: Automatically identifies and formats SQL queries in code cells and magic SQL cells
-- 🌳 **AST-Powered**: Uses Abstract Syntax Tree parsing for accurate SQL string identification
-- 🔒 **Safe Formatting**: Preserves Python comments, query parameters (e.g., `%s`, `?`), and SQL comments
-- ⚙️ **Highly Configurable**: Customize formatting through YAML configuration
-- 🔄 **Pre-commit Ready**: Seamlessly integrates with pre-commit hooks
-- 📦 **Lightweight**: Only three runtime dependencies (sqlglot, nbformat, pyyaml)
+- **Zero-config**: Works out of the box with sensible defaults — no config file needed
+- **Smart SQL Detection**: Automatically identifies and formats SQL queries in code cells and magic SQL cells
+- **AST-Powered**: Uses Abstract Syntax Tree parsing for accurate SQL string identification
+- **Safe Formatting**: Preserves Python comments, query parameters (e.g., `%s`, `?`), and SQL comments
+- **CI-Friendly**: `--check` mode exits non-zero when formatting is needed; `--diff` shows what would change
+- **Skip Hints**: Add `# sqlnbfmt: skip` to any cell to exclude it from formatting
+- **Pre-commit Ready**: Seamlessly integrates with pre-commit hooks
+- **Lightweight**: Only three runtime dependencies (sqlglot, nbformat, pyyaml)
 
 ## Installation
 
@@ -47,9 +56,23 @@ Format a single notebook:
 sqlnbfmt path/to/your_notebook.ipynb
 ```
 
-Format all notebooks in a directory:
+Check formatting without modifying files (useful in CI):
+```bash
+sqlnbfmt --check path/to/your_notebook.ipynb
+```
+
+Show a diff of what would change:
 ```bash
-sqlnbfmt path/to/notebooks/
+sqlnbfmt --diff path/to/your_notebook.ipynb
+```
+
+### Skipping Cells
+
+Add a `# sqlnbfmt: skip` comment anywhere in a cell to skip formatting:
+
+```python
+# sqlnbfmt: skip
+query = "select * from my_special_table where id = 1"
 ```
 
 ### Pre-commit Integration
@@ -63,17 +86,15 @@ pip install pre-commit
 ```yaml
 repos:
   - repo: https://github.com/flyersworder/sqlnbfmt
-    rev: v0.2.0
+    rev: v0.3.0
     hooks:
       - id: sqlnbfmt
-        name: sqlnbfmt
         types: [jupyter]
-        args: [--config, config.yaml, --dialect, postgres]
 ```
-Please run the following command for help:
 
-```bash
-sqlnbfmt -h
+All arguments are optional. To specify a dialect or custom config:
+```yaml
+        args: [--dialect, postgres, --config, config.yaml]
 ```
 
 3. Install the hook:
@@ -86,8 +107,21 @@ pre-commit install
 pre-commit run --all-files
 ```
 
+### CI Usage
+
+Use `--check` in GitHub Actions to enforce formatting:
+
+```yaml
+- name: Check SQL notebook formatting
+  run: |
+    pip install sqlnbfmt
+    sqlnbfmt --check **/*.ipynb
+```
+
 ## Configuration
 
+`sqlnbfmt` works without any configuration file. A `config.yaml` is only needed to override defaults.
+
 Create a `config.yaml` file to customize formatting behavior. [Here](https://github.com/flyersworder/sqlnbfmt/blob/main/config.yaml) is a template.
 
 ### Configuration Options
@@ -95,8 +129,8 @@ Create a `config.yaml` file to customize formatting behavior. [Here](https://git
 | Option | Description | Default |
 |--------|-------------|---------|
 | `sql_keywords` | SQL keywords to recognize and format | Common SQL keywords |
-| `function_names` | Python functions containing SQL code | `[]` |
-| `sql_decorators` | Decorators indicating SQL code | `[]` |
+| `function_names` | Python functions containing SQL code | `read_sql`, `execute`, etc. |
+| `sql_decorators` | Decorators indicating SQL code | `query`, `sql_query`, etc. |
 | `single_line_threshold` | Maximum length before splitting SQL | 80 |
 | `indent_width` | Number of spaces for indentation | 4 |
 
@@ -125,6 +159,19 @@ ORDER BY
 """)
 ```
 
+## Troubleshooting
+
+**SQL not being formatted?**
+- Ensure the string contains at least 2 SQL keywords or a recognizable pattern like `SELECT...FROM`
+- Check that the function name is in the recognized list (use `--config` to add custom ones)
+
+**Comments being modified?**
+- Python comments (`#`) are preserved. SQL comments (`--`) inside strings are converted to `/* */` block comments by sqlglot.
+
+**Pre-commit hook fails?**
+- Make sure the `rev` matches the installed version
+- Run `pre-commit autoupdate` to get the latest version
+
 ## Contributing
 
 We welcome contributions! Here's how to get started:
@@ -146,6 +193,13 @@ source .venv/bin/activate  # On Windows: .venv\Scripts\activate
 pytest
 ```
 
+4. Add eval cases: see `tests/eval/generate_fixtures.py` for examples. Run `python tests/eval/generate_fixtures.py` to regenerate fixtures.
+
+5. Install dev pre-commit hooks:
+```bash
+pre-commit install
+```
+
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

{sqlnbfmt-0.2.0 → sqlnbfmt-0.3.1}/tests/test_eval.py

@@ -14,7 +14,6 @@ import pytest
 from sqlnbfmt.formatter import load_config, process_notebook
 
 EVAL_DIR = Path(__file__).resolve().parent / "eval"
-CONFIG_PATH = Path(__file__).resolve().parent.parent / "config.yaml"
 DIALECT = "mysql"
 
 
@@ -54,7 +53,7 @@ def test_eval(case_name, logger, tmp_path):
     shutil.copy2(case_dir / "input.ipynb", work_path)
 
     # Load config and run formatter
-    config = load_config(CONFIG_PATH)
+    config = load_config()
     process_notebook(work_path, config, DIALECT, logger)
 
     # Read formatted and expected

{sqlnbfmt-0.2.0 → sqlnbfmt-0.3.1}/tests/test_formatter.py

@@ -3,7 +3,7 @@ import nbformat
 from nbformat import v4 as nbf
 import logging
 
-from sqlnbfmt.formatter import process_notebook, load_config
+from sqlnbfmt.formatter import process_notebook, load_config, diff_notebook
 
 
 # Fixture for temporary notebook path
@@ -216,7 +216,7 @@ def test_sql_formatter(temp_nb_path, logger, input_cells, expected_cells):
     nbformat.write(nb, temp_nb_path)
 
     # Load configuration
-    config = load_config("config.yaml")
+    config = load_config()
     dialect = "mysql"  # or your preferred SQL dialect
 
     # Run the formatter on the notebook
@@ -235,3 +235,229 @@ def test_sql_formatter(temp_nb_path, logger, input_cells, expected_cells):
 
     # Clean up
     temp_nb_path.unlink()
+
+
+def test_skip_hint(tmp_path, logger):
+    """Cell with # sqlnbfmt: skip should remain untouched."""
+    nb = nbf.new_notebook()
+    original_source = (
+        '# sqlnbfmt: skip\nquery = "select id, name from users where active = 1"'
+    )
+    nb.cells = [nbf.new_code_cell(original_source)]
+    nb_path = tmp_path / "skip_test.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    changed = process_notebook(nb_path, config, "mysql", logger)
+    assert not changed
+
+    result_nb = nbformat.read(nb_path, as_version=4)
+    assert result_nb.cells[0].source == original_source
+
+
+def test_check_only_unformatted(tmp_path, logger):
+    """--check on unformatted notebook returns True but file unchanged."""
+    nb = nbf.new_notebook()
+    original_source = 'query = "select id, name from users where active = 1"'
+    nb.cells = [nbf.new_code_cell(original_source)]
+    nb_path = tmp_path / "check_test.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    changed = process_notebook(nb_path, config, "mysql", logger, check_only=True)
+    assert changed
+
+    # File should be unchanged
+    result_nb = nbformat.read(nb_path, as_version=4)
+    assert result_nb.cells[0].source == original_source
+
+
+def test_check_only_formatted(tmp_path, logger):
+    """--check on already-formatted notebook returns False."""
+    nb = nbf.new_notebook()
+    nb.cells = [
+        nbf.new_code_cell(
+            'query = """\nSELECT\n  id,\n  name\nFROM users\nWHERE\n  active = 1\n"""'
+        )
+    ]
+    nb_path = tmp_path / "check_formatted.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    changed = process_notebook(nb_path, config, "mysql", logger, check_only=True)
+    assert not changed
+
+
+def test_diff_notebook_unformatted(tmp_path, logger):
+    """diff_notebook returns non-empty diff for unformatted notebook."""
+    nb = nbf.new_notebook()
+    nb.cells = [
+        nbf.new_code_cell('query = "select id, name from users where active = 1"')
+    ]
+    nb_path = tmp_path / "diff_test.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    diff_output = diff_notebook(nb_path, config, "mysql", logger)
+    assert diff_output  # non-empty
+    assert "---" in diff_output
+    assert "+++" in diff_output
+
+
+def test_diff_notebook_formatted(tmp_path, logger):
+    """diff_notebook returns empty string for already-formatted notebook."""
+    nb = nbf.new_notebook()
+    nb.cells = [
+        nbf.new_code_cell(
+            'query = """\nSELECT\n  id,\n  name\nFROM users\nWHERE\n  active = 1\n"""'
+        )
+    ]
+    nb_path = tmp_path / "diff_formatted.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    diff_output = diff_notebook(nb_path, config, "mysql", logger)
+    assert diff_output == ""
+
+
+def test_load_config_defaults():
+    """load_config() without args returns defaults."""
+    config = load_config()
+    assert "SELECT" in config.sql_keywords
+    assert "read_sql" in config.function_names
+    assert config.indent_width == 4
+
+
+def test_load_config_missing_file():
+    """load_config with missing path raises FileNotFoundError."""
+    import pytest
+
+    with pytest.raises(FileNotFoundError):
+        load_config("/nonexistent/config.yaml")
+
+
+def test_load_config_empty_file(tmp_path):
+    """load_config with empty YAML file returns defaults (not AttributeError)."""
+    empty_config = tmp_path / "empty.yaml"
+    empty_config.write_text("")
+    config = load_config(empty_config)
+    assert "SELECT" in config.sql_keywords
+    assert "read_sql" in config.function_names
+
+
+def test_load_config_partial_override(tmp_path):
+    """Config file with only indent_width still gets default keywords."""
+    partial_config = tmp_path / "partial.yaml"
+    partial_config.write_text("formatting_options:\n  indent_width: 2\n")
+    config = load_config(partial_config)
+    assert config.indent_width == 2
+    assert "SELECT" in config.sql_keywords
+    assert "read_sql" in config.function_names
+
+
+def test_skip_hint_in_string_not_triggered(tmp_path, logger):
+    """Skip hint inside a string literal should NOT skip the cell."""
+    nb = nbf.new_notebook()
+    original_source = (
+        'msg = "Use # sqlnbfmt: skip to disable"\n'
+        'query = "select id, name from users where active = 1"'
+    )
+    nb.cells = [nbf.new_code_cell(original_source)]
+    nb_path = tmp_path / "skip_in_string.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    changed = process_notebook(nb_path, config, "mysql", logger)
+    assert changed  # SQL should still be formatted
+
+
+def test_skip_hint_magic_cell(tmp_path, logger):
+    """Skip hint in a magic command cell should skip formatting."""
+    nb = nbf.new_notebook()
+    original_source = "# sqlnbfmt: skip\n%%sql\nselect * from users where active = 1"
+    nb.cells = [nbf.new_code_cell(original_source)]
+    nb_path = tmp_path / "skip_magic.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    changed = process_notebook(nb_path, config, "mysql", logger)
+    assert not changed
+
+    result_nb = nbformat.read(nb_path, as_version=4)
+    assert result_nb.cells[0].source == original_source
+
+
+def test_check_diff_combined(tmp_path, logger):
+    """--check --diff combined: no duplicate files, diff is printed."""
+    nb = nbf.new_notebook()
+    nb.cells = [
+        nbf.new_code_cell('query = "select id, name from users where active = 1"')
+    ]
+    nb_path = tmp_path / "combined.ipynb"
+    nbformat.write(nb, nb_path)
+
+    config = load_config()
+    # Simulate the combined logic from main()
+    diff_output = diff_notebook(nb_path, config, "mysql", logger)
+    assert diff_output  # has diff
+    # In the refactored main(), diff_notebook is used as the single source
+    # of truth for both --check and --diff, so no duplication possible.
+
+
+def test_main_diff_only_exits_zero(tmp_path):
+    """--diff alone always exits 0 even when changes are needed."""
+    import subprocess
+
+    nb = nbf.new_notebook()
+    nb.cells = [
+        nbf.new_code_cell('query = "select id, name from users where active = 1"')
+    ]
+    nb_path = tmp_path / "diff_exit.ipynb"
+    nbformat.write(nb, nb_path)
+
+    result = subprocess.run(
+        ["sqlnbfmt", "--diff", str(nb_path)],
+        capture_output=True,
+        text=True,
+    )
+    assert result.returncode == 0
+    assert "---" in result.stdout  # diff was printed
+
+
+def test_main_check_exits_one(tmp_path):
+    """--check exits 1 when formatting is needed."""
+    import subprocess
+
+    nb = nbf.new_notebook()
+    nb.cells = [
+        nbf.new_code_cell('query = "select id, name from users where active = 1"')
+    ]
+    nb_path = tmp_path / "check_exit.ipynb"
+    nbformat.write(nb, nb_path)
+
+    result = subprocess.run(
+        ["sqlnbfmt", "--check", str(nb_path)],
+        capture_output=True,
+        text=True,
+    )
+    assert result.returncode == 1
+
+
+def test_main_check_diff_exits_one(tmp_path):
+    """--check --diff combined exits 1 and prints diff."""
+    import subprocess
+
+    nb = nbf.new_notebook()
+    nb.cells = [
+        nbf.new_code_cell('query = "select id, name from users where active = 1"')
+    ]
+    nb_path = tmp_path / "check_diff_exit.ipynb"
+    nbformat.write(nb, nb_path)
+
+    result = subprocess.run(
+        ["sqlnbfmt", "--check", "--diff", str(nb_path)],
+        capture_output=True,
+        text=True,
+    )
+    assert result.returncode == 1
+    assert "---" in result.stdout  # diff was printed