sql-decomposer 0.1.0__tar.gz

sql_decomposer-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 sql_decomposer contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

sql_decomposer-0.1.0/PKG-INFO

@@ -0,0 +1,116 @@
+Metadata-Version: 2.4
+Name: sql-decomposer
+Version: 0.1.0
+Summary: Extract repeated SQL subqueries into temporary tables using sqlglot.
+Author: sql_decomposer contributors
+License-Expression: MIT
+Keywords: sql,sqlglot,decomposer,query-optimization
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: sqlglot>=23.0
+Provides-Extra: dev
+Requires-Dist: build>=1.2.2; extra == "dev"
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: twine>=5.0; extra == "dev"
+Dynamic: license-file
+
+# sql_decomposer
+
+`sql_decomposer` extracts repeated SQL subqueries into temporary tables.
+It can help reduce duplication in large analytical queries and produce a more readable SQL script.
+
+## Features
+
+- Parses SQL safely with `sqlglot`.
+- Detects repeated `SELECT` subqueries.
+- Rewrites repeated blocks to `SELECT * FROM <temp_table>`.
+- Provides a Python API and a CLI.
+
+## Installation
+
+From source:
+
+```bash
+pip install .
+```
+
+From PyPI (after release):
+
+```bash
+pip install sql-decomposer
+```
+
+## CLI usage
+
+```bash
+python -m sql_decomposer input.sql output.sql --min-count 2 --temp-prefix __temp
+```
+
+Or with console script:
+
+```bash
+sql-decomposer input.sql output.sql
+```
+
+Options:
+
+- `--dialect`: optional sqlglot dialect (`postgres`, `mysql`, etc.)
+- `--min-count`: minimum repetition count to extract (default: `2`)
+- `--temp-prefix`: generated temp table prefix (default: `__temp`)
+
+## Python API
+
+```python
+from sql_decomposer import decompose_sql
+
+sql = "SELECT * FROM (SELECT id FROM users) t1 JOIN (SELECT id FROM users) t2 ON t1.id=t2.id"
+result = decompose_sql(sql, min_count=2, temp_prefix="tmp")
+print(result)
+```
+
+## Development
+
+Install dev dependencies:
+
+```bash
+pip install -e ".[dev]"
+```
+
+Run tests:
+
+```bash
+pytest
+```
+
+Build artifacts:
+
+```bash
+python -m build
+```
+
+Validate package metadata:
+
+```bash
+twine check dist/*
+```
+
+## GitHub and PyPI release checklist
+
+1. Create repository named `sql_decomposer` on GitHub.
+2. Push code and enable Actions.
+3. Create a PyPI project `sql-decomposer`.
+4. Add `PYPI_API_TOKEN` as a GitHub Actions secret.
+5. Tag a release (`v0.1.0`) to trigger publish workflow.
+
+## License
+
+MIT License. See `LICENSE`.

sql_decomposer-0.1.0/README.md

@@ -0,0 +1,91 @@
+# sql_decomposer
+
+`sql_decomposer` extracts repeated SQL subqueries into temporary tables.
+It can help reduce duplication in large analytical queries and produce a more readable SQL script.
+
+## Features
+
+- Parses SQL safely with `sqlglot`.
+- Detects repeated `SELECT` subqueries.
+- Rewrites repeated blocks to `SELECT * FROM <temp_table>`.
+- Provides a Python API and a CLI.
+
+## Installation
+
+From source:
+
+```bash
+pip install .
+```
+
+From PyPI (after release):
+
+```bash
+pip install sql-decomposer
+```
+
+## CLI usage
+
+```bash
+python -m sql_decomposer input.sql output.sql --min-count 2 --temp-prefix __temp
+```
+
+Or with console script:
+
+```bash
+sql-decomposer input.sql output.sql
+```
+
+Options:
+
+- `--dialect`: optional sqlglot dialect (`postgres`, `mysql`, etc.)
+- `--min-count`: minimum repetition count to extract (default: `2`)
+- `--temp-prefix`: generated temp table prefix (default: `__temp`)
+
+## Python API
+
+```python
+from sql_decomposer import decompose_sql
+
+sql = "SELECT * FROM (SELECT id FROM users) t1 JOIN (SELECT id FROM users) t2 ON t1.id=t2.id"
+result = decompose_sql(sql, min_count=2, temp_prefix="tmp")
+print(result)
+```
+
+## Development
+
+Install dev dependencies:
+
+```bash
+pip install -e ".[dev]"
+```
+
+Run tests:
+
+```bash
+pytest
+```
+
+Build artifacts:
+
+```bash
+python -m build
+```
+
+Validate package metadata:
+
+```bash
+twine check dist/*
+```
+
+## GitHub and PyPI release checklist
+
+1. Create repository named `sql_decomposer` on GitHub.
+2. Push code and enable Actions.
+3. Create a PyPI project `sql-decomposer`.
+4. Add `PYPI_API_TOKEN` as a GitHub Actions secret.
+5. Tag a release (`v0.1.0`) to trigger publish workflow.
+
+## License
+
+MIT License. See `LICENSE`.

sql_decomposer-0.1.0/pyproject.toml

@@ -0,0 +1,46 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "sql-decomposer"
+version = "0.1.0"
+description = "Extract repeated SQL subqueries into temporary tables using sqlglot."
+readme = "README.md"
+requires-python = ">=3.9"
+license = "MIT"
+authors = [
+  { name = "sql_decomposer contributors" }
+]
+keywords = ["sql", "sqlglot", "decomposer", "query-optimization"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.9",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Topic :: Database",
+]
+dependencies = [
+  "sqlglot>=23.0",
+]
+
+[project.optional-dependencies]
+dev = [
+  "build>=1.2.2",
+  "pytest>=8.0",
+  "twine>=5.0",
+]
+
+[project.scripts]
+sql-decomposer = "sql_decomposer.__main__:main"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["sql_decomposer*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+python_files = ["test_package.py"]

sql_decomposer-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

sql_decomposer-0.1.0/sql_decomposer/__init__.py

@@ -0,0 +1,4 @@
+from sql_decomposer.decomposer import decompose_sql
+
+__version__ = "0.1.0"
+__all__ = ["decompose_sql"]

sql_decomposer-0.1.0/sql_decomposer/__main__.py

@@ -0,0 +1,56 @@
+import argparse
+from pathlib import Path
+
+from sql_decomposer.decomposer import decompose_sql
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="sql_decomposer",
+        description="Decompose repeated SQL subqueries and save result to file.",
+    )
+    parser.add_argument("input", type=Path, help="Path to input .sql file")
+    parser.add_argument("output", type=Path, help="Path to output .sql file")
+    parser.add_argument(
+        "--dialect",
+        default=None,
+        help='Optional sqlglot dialect (e.g. "postgres", "mysql")',
+    )
+    parser.add_argument(
+        "--min-count",
+        type=int,
+        default=2,
+        help="Minimum repetition count for extraction (default: 2)",
+    )
+    parser.add_argument(
+        "--temp-prefix",
+        default="__temp",
+        help='Prefix for generated temp tables (default: "__temp")',
+    )
+    return parser
+
+
+def main() -> int:
+    parser = build_parser()
+    args = parser.parse_args()
+
+    if not args.input.exists():
+        parser.error(f"Input file does not exist: {args.input}")
+    if not args.input.is_file():
+        parser.error(f"Input path is not a file: {args.input}")
+
+    sql_text = args.input.read_text(encoding="utf-8")
+    decomposed = decompose_sql(
+        sql=sql_text,
+        dialect=args.dialect,
+        min_count=args.min_count,
+        temp_prefix=args.temp_prefix,
+    )
+
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(decomposed, encoding="utf-8")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

sql_decomposer-0.1.0/sql_decomposer/decomposer.py

@@ -0,0 +1,156 @@
+"""
+SQL Decomposer — extracts repeated subqueries from a SQL statement
+into temporary tables, reducing duplication.
+"""
+
+import sqlglot
+from sqlglot import exp
+from typing import Optional
+
+
+def _collect_sql_frequencies(
+    ast: exp.Expression, dialect: Optional[str]
+) -> tuple[dict[str, int], set[str]]:
+    """Collect SQL string frequencies and SELECT-only SQL strings."""
+    sql_counter: dict[str, int] = {}
+    select_sqls: set[str] = set()
+
+    for walked in ast.walk():
+        node = walked[0] if isinstance(walked, tuple) else walked
+        node_sql = node.sql(dialect=dialect)
+        if not node_sql:
+            continue
+
+        sql_counter[node_sql] = sql_counter.get(node_sql, 0) + 1
+        if isinstance(node, exp.Select):
+            select_sqls.add(node_sql)
+
+    return sql_counter, select_sqls
+
+
+def _build_temp_table_map(
+    sql_counter: dict[str, int],
+    select_sqls: set[str],
+    root_sql: str,
+    min_count: int,
+    temp_prefix: str,
+) -> dict[str, str]:
+    """
+    Build stable mapping: repeated SELECT SQL -> temp table name.
+
+    Ordering is deterministic:
+      1) higher repetition count first
+      2) longer SQL first (helps nested/repeated patterns)
+    """
+    sorted_items = sorted(
+        sql_counter.items(),
+        key=lambda item: (item[1], len(item[0])),
+        reverse=True,
+    )
+
+    ordered_candidates = [
+        sql_key
+        for sql_key, count in sorted_items
+        if count >= min_count and sql_key != root_sql and sql_key in select_sqls
+    ]
+
+    temp_table_map: dict[str, str] = {}
+    working_sql = root_sql
+
+    # Keep legacy "longer-first replacement" semantics for candidate filtering:
+    # once a larger repeated SELECT is selected, nested candidates it contains
+    # may disappear and therefore should not be extracted separately.
+    for sql_key in ordered_candidates:
+        if sql_key not in working_sql:
+            continue
+
+        temp_name = f"{temp_prefix}_{len(temp_table_map) + 1}"
+        temp_table_map[sql_key] = temp_name
+        working_sql = working_sql.replace(sql_key, f"SELECT * FROM {temp_name}")
+
+    return temp_table_map
+
+
+def _rewrite_selects_with_temp_tables(
+    ast: exp.Expression, dialect: Optional[str], temp_table_map: dict[str, str]
+) -> exp.Expression:
+    """Replace repeated SELECT nodes with SELECT * FROM <temp_table>."""
+    if not temp_table_map:
+        return ast
+
+    def _replace(node: exp.Expression) -> exp.Expression:
+        if isinstance(node, exp.Select):
+            node_sql = node.sql(dialect=dialect)
+            temp_name = temp_table_map.get(node_sql)
+            if temp_name:
+                return exp.select("*").from_(temp_name)
+        return node
+
+    return ast.transform(_replace)
+
+
+def decompose_sql(
+    sql: str,
+    dialect: Optional[str] = None,
+    min_count: int = 2,
+    temp_prefix: str = "__temp",
+) -> str:
+    """
+    Decompose a SQL query by extracting repeated sub-SELECT expressions
+    into CREATE TEMPORARY TABLE statements.
+
+    Algorithm:
+      1. Parse *sql* into an AST via sqlglot.
+      2. Walk every node; convert each node back to a SQL string and
+         count occurrences in a dictionary.
+      3. Regenerate the full SQL from the AST.
+      4. Sort the dictionary by count DESC, then by SQL length DESC
+         (longer expressions first — avoids partial-substring collisions).
+      5-6. For each entry that qualifies (is a SELECT, appears ≥ *min_count*
+         times, is not the root query), create a temp-table definition and
+         replace the subquery text in the working SQL string.
+      7. Append the (possibly rewritten) final SQL.
+      8. Return the ordered list of statements.
+
+    Args:
+        sql:         Source SQL string.
+        dialect:     sqlglot dialect name (e.g. "postgres", "mysql").
+        min_count:   Minimum number of occurrences for extraction (default 2).
+        temp_prefix: Naming prefix for generated temp tables.
+
+    Returns:
+        SQL script as a single string: zero or more CREATE TEMPORARY TABLE
+        statements followed by the final (rewritten) query.
+    """
+    # ── Step 1: parse ────────────────────────────────────────────────
+    ast = sqlglot.parse_one(sql, dialect=dialect)
+
+    # ── Step 2: walk all nodes, count SQL representations ────────────
+    sql_counter, select_sqls = _collect_sql_frequencies(ast, dialect)
+
+    # ── Step 3: full SQL from AST ────────────────────────────────────
+    result_sql = ast.sql(dialect=dialect)
+
+    # ── Step 4: build SELECT->temp table mapping ─────────────────────
+    temp_table_map = _build_temp_table_map(
+        sql_counter=sql_counter,
+        select_sqls=select_sqls,
+        root_sql=result_sql,
+        min_count=min_count,
+        temp_prefix=temp_prefix,
+    )
+
+    # ── Steps 5-6: create temp tables and rewrite AST ────────────────
+    statements: list[str] = []
+
+    for sql_key, temp_name in temp_table_map.items():
+        statements.append(f"CREATE TEMPORARY TABLE {temp_name} AS {sql_key}")
+
+    rewritten_ast = _rewrite_selects_with_temp_tables(ast, dialect, temp_table_map)
+    result_sql = rewritten_ast.sql(dialect=dialect)
+
+    # ── Step 7: append final rewritten query ─────────────────────────
+    statements.append(result_sql)
+
+    # ── Step 8 ───────────────────────────────────────────────────────
+    return "\n".join(statements)

sql_decomposer-0.1.0/sql_decomposer.egg-info/PKG-INFO

@@ -0,0 +1,116 @@
+Metadata-Version: 2.4
+Name: sql-decomposer
+Version: 0.1.0
+Summary: Extract repeated SQL subqueries into temporary tables using sqlglot.
+Author: sql_decomposer contributors
+License-Expression: MIT
+Keywords: sql,sqlglot,decomposer,query-optimization
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: sqlglot>=23.0
+Provides-Extra: dev
+Requires-Dist: build>=1.2.2; extra == "dev"
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: twine>=5.0; extra == "dev"
+Dynamic: license-file
+
+# sql_decomposer
+
+`sql_decomposer` extracts repeated SQL subqueries into temporary tables.
+It can help reduce duplication in large analytical queries and produce a more readable SQL script.
+
+## Features
+
+- Parses SQL safely with `sqlglot`.
+- Detects repeated `SELECT` subqueries.
+- Rewrites repeated blocks to `SELECT * FROM <temp_table>`.
+- Provides a Python API and a CLI.
+
+## Installation
+
+From source:
+
+```bash
+pip install .
+```
+
+From PyPI (after release):
+
+```bash
+pip install sql-decomposer
+```
+
+## CLI usage
+
+```bash
+python -m sql_decomposer input.sql output.sql --min-count 2 --temp-prefix __temp
+```
+
+Or with console script:
+
+```bash
+sql-decomposer input.sql output.sql
+```
+
+Options:
+
+- `--dialect`: optional sqlglot dialect (`postgres`, `mysql`, etc.)
+- `--min-count`: minimum repetition count to extract (default: `2`)
+- `--temp-prefix`: generated temp table prefix (default: `__temp`)
+
+## Python API
+
+```python
+from sql_decomposer import decompose_sql
+
+sql = "SELECT * FROM (SELECT id FROM users) t1 JOIN (SELECT id FROM users) t2 ON t1.id=t2.id"
+result = decompose_sql(sql, min_count=2, temp_prefix="tmp")
+print(result)
+```
+
+## Development
+
+Install dev dependencies:
+
+```bash
+pip install -e ".[dev]"
+```
+
+Run tests:
+
+```bash
+pytest
+```
+
+Build artifacts:
+
+```bash
+python -m build
+```
+
+Validate package metadata:
+
+```bash
+twine check dist/*
+```
+
+## GitHub and PyPI release checklist
+
+1. Create repository named `sql_decomposer` on GitHub.
+2. Push code and enable Actions.
+3. Create a PyPI project `sql-decomposer`.
+4. Add `PYPI_API_TOKEN` as a GitHub Actions secret.
+5. Tag a release (`v0.1.0`) to trigger publish workflow.
+
+## License
+
+MIT License. See `LICENSE`.

sql_decomposer-0.1.0/sql_decomposer.egg-info/SOURCES.txt

@@ -0,0 +1,13 @@
+LICENSE
+README.md
+pyproject.toml
+sql_decomposer/__init__.py
+sql_decomposer/__main__.py
+sql_decomposer/decomposer.py
+sql_decomposer.egg-info/PKG-INFO
+sql_decomposer.egg-info/SOURCES.txt
+sql_decomposer.egg-info/dependency_links.txt
+sql_decomposer.egg-info/entry_points.txt
+sql_decomposer.egg-info/requires.txt
+sql_decomposer.egg-info/top_level.txt
+tests/test_package.py

sql_decomposer-0.1.0/sql_decomposer.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

sql_decomposer-0.1.0/sql_decomposer.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+sql-decomposer = sql_decomposer.__main__:main

sql_decomposer-0.1.0/sql_decomposer.egg-info/requires.txt

@@ -0,0 +1,6 @@
+sqlglot>=23.0
+
+[dev]
+build>=1.2.2
+pytest>=8.0
+twine>=5.0

sql_decomposer-0.1.0/sql_decomposer.egg-info/top_level.txt

@@ -0,0 +1 @@
+sql_decomposer

sql_decomposer-0.1.0/tests/test_package.py

@@ -0,0 +1,82 @@
+from pathlib import Path
+
+import pytest
+
+from sql_decomposer import decompose_sql
+
+
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+
+
+def _read(rel_path: str) -> str:
+    return (PROJECT_ROOT / rel_path).read_text(encoding="utf-8").strip()
+
+
+@pytest.mark.parametrize(
+    ("input_fixture", "expected_fixture"),
+    [
+        (
+            "tests/sql_big_analytics_query.sql",
+            "decomposed_sql/sql_big_analytics_query.sql",
+        ),
+        (
+            "tests/sql_complex_hr_query.sql",
+            "decomposed_sql/sql_complex_hr_query.sql",
+        ),
+        (
+            "tests/sql_financial_report_four_repeats.sql",
+            "decomposed_sql/sql_financial_report_four_repeats.sql",
+        ),
+        (
+            "tests/sql_flat_query_no_subqueries.sql",
+            "decomposed_sql/sql_flat_query_no_subqueries.sql",
+        ),
+        ("tests/sql_min_count_one.sql", "decomposed_sql/sql_min_count_one.sql"),
+        (
+            "tests/sql_nested_repeated_subqueries.sql",
+            "decomposed_sql/sql_nested_repeated_subqueries.sql",
+        ),
+        ("tests/sql_no_repetition.sql", "decomposed_sql/sql_no_repetition.sql"),
+        (
+            "tests/sql_simple_two_identical_subqueries.sql",
+            "decomposed_sql/sql_simple_two_identical_subqueries.sql",
+        ),
+        (
+            "tests/sql_triple_repeated_subquery.sql",
+            "decomposed_sql/sql_triple_repeated_subquery.sql",
+        ),
+        (
+            "tests/sql_two_different_repeated_subqueries.sql",
+            "decomposed_sql/sql_two_different_repeated_subqueries.sql",
+        ),
+    ],
+)
+def test_decompose_sql_matches_expected_fixtures(
+    input_fixture: str, expected_fixture: str
+) -> None:
+    sql = _read(input_fixture)
+    expected = _read(expected_fixture)
+    assert decompose_sql(sql) == expected
+
+
+def test_decompose_sql_does_not_replace_string_literals() -> None:
+    sql = """
+        SELECT
+            'SELECT id FROM users WHERE active = 1' AS query_text,
+            a.id
+        FROM (
+            SELECT id FROM users WHERE active = 1
+        ) AS a
+        JOIN (
+            SELECT id FROM users WHERE active = 1
+        ) AS b ON a.id = b.id
+    """
+    result = decompose_sql(sql)
+
+    assert (
+        "CREATE TEMPORARY TABLE __temp_1 AS SELECT id FROM users WHERE active = 1"
+        in result
+    )
+    assert "'SELECT id FROM users WHERE active = 1' AS query_text" in result
+    assert "(SELECT * FROM __temp_1) AS a" in result
+    assert "(SELECT * FROM __temp_1) AS b" in result