semql 0.1.0__tar.gz

semql-0.1.0/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Nikhil Pallamreddy
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

semql-0.1.0/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.4
+Name: semql
+Version: 0.1.0
+Summary: Semantic data layer: SemanticQuery → backend SQL with authorisation, row-level scoping, time-spine fill, and a typed four-role LLM prompt pipeline.
+Author: Nikhil Pallamreddy
+Author-email: Nikhil Pallamreddy <nikhil.pallamreddy+git@gmail.com>
+License-Expression: BSD-3-Clause
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Database
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Dist: pydantic>=2.13.4
+Requires-Dist: sqlglot>=30.9.0
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/npalladium/semql
+Project-URL: Repository, https://github.com/npalladium/semql
+Project-URL: Issues, https://github.com/npalladium/semql/issues
+Description-Content-Type: text/markdown
+
+# semql
+
+Pure-Python compiler from a semantic spec to backend SQL. Define
+cubes (dimensions, measures, time-dimensions, joins) once; emit
+correct, parameterised SQL for Postgres, ClickHouse, DuckDB and
+(via the strategy seam) Snowflake / BigQuery.
+
+`semql` does **no I/O**: catalogues are Python data; the compiler
+returns SQL + bound params; running the SQL is the caller's job.
+Prompt-fragment rendering for LLM planners ships in the core
+(`semql.prompt`). Sibling packages add MCP exposure (`semql-mcp`)
+and ER diagrams (`semql-erd`).
+
+## Install
+
+```sh
+pip install semql
+```
+
+## Quick start
+
+```python
+from semql import (
+    Backend,
+    Catalog,
+    Cube,
+    Dimension,
+    Measure,
+    SemanticQuery,
+)
+
+orders = Cube(
+    name="orders",
+    backend=Backend.POSTGRES,
+    table="orders",
+    alias="o",
+    measures=[
+        Measure(name="revenue", sql="{o}.amount", agg="sum", unit="currency"),
+    ],
+    dimensions=[
+        Dimension(name="region", sql="{o}.region", type="string"),
+    ],
+)
+
+catalog = Catalog([orders])
+compiled = catalog.compile(
+    SemanticQuery(measures=["orders.revenue"], dimensions=["orders.region"]),
+)
+# compiled.sql, compiled.params, compiled.columns, compiled.backend
+```
+
+The `{o}` placeholder in a cube's `sql` is its alias; the compiler
+resolves it (along with `{schema}`-style context placeholders and
+`{ctx.X}` row-level-security placeholders) at compile time.
+
+## What lives in the box
+
+| Surface | Module |
+|---|---|
+| Cube / Measure / Dimension / TimeDimension / Join | `semql.model` |
+| SemanticQuery / Filter / TimeWindow / CompareWindow | `semql.spec` |
+| Catalog wrapper (validation, prompt, compile entry) | `semql.catalog` |
+| Compiler — sqlglot AST → dialect SQL | `semql.compile` |
+| Collect-all static validator | `semql.validate` |
+| Reflection cubes (catalog_cubes, ...) | `semql.introspect` |
+| Planner / router prompt fragments | `semql.prompt` |
+| Backend strategies + sqlglot dialect adapter | `semql.backend`, `semql.dialect` |
+| Visualisation decision (chart type, axes, formats) | `semql.visualize` |
+| `is_safe_select` post-hoc SQL guard | `semql.safe` |
+| Structured error hierarchy | `semql.errors` |
+
+## Features
+
+- **Compare windows** — `CompareWindow(mode="previous_period")` wraps
+  the inner query in `current` / `prior` CTEs joined via `FULL OUTER
+  JOIN` and emits `{m}_current` / `{m}_prior` / `{m}_delta` /
+  `{m}_pct_change` columns per measure.
+- **Tenancy** — per-cube `SCHEMA` (default; `{tenant_schema}`
+  substitution) or `DISCRIMINATOR` (compiler wraps the source in a
+  subquery with `WHERE tenancy_column = $tenant`).
+- **Row-level security** — `Cube.security_sql` AND-composes with
+  tenancy inside the isolation subquery; `{ctx.X}` placeholders bind
+  as parameters, never inline as literals.
+- **MCP-ready** — `Catalog.prompt()` produces the planner system-prompt
+  fragment; `semql-mcp` wraps it as a server.
+- **Pluggable backends** — `BackendStrategy` Protocol lets out-of-tree
+  Snowflake / BigQuery adapters slot in without forking the compiler.
+
+## Philosophy
+
+See `PHILOSOPHY.md` at the repo root. Highlights:
+- Correct SQL, not optimal. The query planner is the database's job.
+- The emitted SQL must read like something a human could have written.
+- `compile()` fails at the first problem; `validate()` collects them all.
+- The catalogue is data; reflection isn't an afterthought.
+
+## Status
+
+Pre-v1. The shape is stable, but minor names / fields may move before
+the v1 contract locks. Tests pin every public behaviour the README
+documents.

semql-0.1.0/README.md

@@ -0,0 +1,101 @@
+# semql
+
+Pure-Python compiler from a semantic spec to backend SQL. Define
+cubes (dimensions, measures, time-dimensions, joins) once; emit
+correct, parameterised SQL for Postgres, ClickHouse, DuckDB and
+(via the strategy seam) Snowflake / BigQuery.
+
+`semql` does **no I/O**: catalogues are Python data; the compiler
+returns SQL + bound params; running the SQL is the caller's job.
+Prompt-fragment rendering for LLM planners ships in the core
+(`semql.prompt`). Sibling packages add MCP exposure (`semql-mcp`)
+and ER diagrams (`semql-erd`).
+
+## Install
+
+```sh
+pip install semql
+```
+
+## Quick start
+
+```python
+from semql import (
+    Backend,
+    Catalog,
+    Cube,
+    Dimension,
+    Measure,
+    SemanticQuery,
+)
+
+orders = Cube(
+    name="orders",
+    backend=Backend.POSTGRES,
+    table="orders",
+    alias="o",
+    measures=[
+        Measure(name="revenue", sql="{o}.amount", agg="sum", unit="currency"),
+    ],
+    dimensions=[
+        Dimension(name="region", sql="{o}.region", type="string"),
+    ],
+)
+
+catalog = Catalog([orders])
+compiled = catalog.compile(
+    SemanticQuery(measures=["orders.revenue"], dimensions=["orders.region"]),
+)
+# compiled.sql, compiled.params, compiled.columns, compiled.backend
+```
+
+The `{o}` placeholder in a cube's `sql` is its alias; the compiler
+resolves it (along with `{schema}`-style context placeholders and
+`{ctx.X}` row-level-security placeholders) at compile time.
+
+## What lives in the box
+
+| Surface | Module |
+|---|---|
+| Cube / Measure / Dimension / TimeDimension / Join | `semql.model` |
+| SemanticQuery / Filter / TimeWindow / CompareWindow | `semql.spec` |
+| Catalog wrapper (validation, prompt, compile entry) | `semql.catalog` |
+| Compiler — sqlglot AST → dialect SQL | `semql.compile` |
+| Collect-all static validator | `semql.validate` |
+| Reflection cubes (catalog_cubes, ...) | `semql.introspect` |
+| Planner / router prompt fragments | `semql.prompt` |
+| Backend strategies + sqlglot dialect adapter | `semql.backend`, `semql.dialect` |
+| Visualisation decision (chart type, axes, formats) | `semql.visualize` |
+| `is_safe_select` post-hoc SQL guard | `semql.safe` |
+| Structured error hierarchy | `semql.errors` |
+
+## Features
+
+- **Compare windows** — `CompareWindow(mode="previous_period")` wraps
+  the inner query in `current` / `prior` CTEs joined via `FULL OUTER
+  JOIN` and emits `{m}_current` / `{m}_prior` / `{m}_delta` /
+  `{m}_pct_change` columns per measure.
+- **Tenancy** — per-cube `SCHEMA` (default; `{tenant_schema}`
+  substitution) or `DISCRIMINATOR` (compiler wraps the source in a
+  subquery with `WHERE tenancy_column = $tenant`).
+- **Row-level security** — `Cube.security_sql` AND-composes with
+  tenancy inside the isolation subquery; `{ctx.X}` placeholders bind
+  as parameters, never inline as literals.
+- **MCP-ready** — `Catalog.prompt()` produces the planner system-prompt
+  fragment; `semql-mcp` wraps it as a server.
+- **Pluggable backends** — `BackendStrategy` Protocol lets out-of-tree
+  Snowflake / BigQuery adapters slot in without forking the compiler.
+
+## Philosophy
+
+See `PHILOSOPHY.md` at the repo root. Highlights:
+- Correct SQL, not optimal. The query planner is the database's job.
+- The emitted SQL must read like something a human could have written.
+- `compile()` fails at the first problem; `validate()` collects them all.
+- The catalogue is data; reflection isn't an afterthought.
+
+## Status
+
+Pre-v1. The shape is stable, but minor names / fields may move before
+the v1 contract locks. Tests pin every public behaviour the README
+documents.

semql-0.1.0/pyproject.toml

@@ -0,0 +1,35 @@
+[project]
+name = "semql"
+version = "0.1.0"
+description = "Semantic data layer: SemanticQuery → backend SQL with authorisation, row-level scoping, time-spine fill, and a typed four-role LLM prompt pipeline."
+readme = "README.md"
+license = "BSD-3-Clause"
+license-files = ["LICENSE"]
+authors = [
+    { name = "Nikhil Pallamreddy", email = "nikhil.pallamreddy+git@gmail.com" }
+]
+requires-python = ">=3.12"
+dependencies = [
+    "pydantic>=2.13.4",
+    "sqlglot>=30.9.0",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Database",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+    "Typing :: Typed",
+]
+
+[project.urls]
+Homepage = "https://github.com/npalladium/semql"
+Repository = "https://github.com/npalladium/semql"
+Issues = "https://github.com/npalladium/semql/issues"
+
+[build-system]
+requires = ["uv_build>=0.11.19,<0.12.0"]
+build-backend = "uv_build"

semql-0.1.0/src/semql/__init__.py

@@ -0,0 +1,153 @@
+"""Public surface of the semantic layer.
+
+Most users only need ``Catalog``, ``Cube``, the field types
+(``Measure`` / ``Dimension`` / ``TimeDimension``), and
+``SemanticQuery``. The rest is exported for callers building their own
+tooling on top of the compiler (custom validators, MCP servers,
+prompt rendering, etc.).
+"""
+
+from __future__ import annotations
+
+from semql.catalog import Catalog
+from semql.compile import MAX_UNGROUPED_ROWS, Compiled, compile_query
+from semql.docs import render_catalog_markdown
+from semql.errors import (
+    CompileError,
+    CrossBackendError,
+    FilterTypeError,
+    JoinPathError,
+    PhaseDeferredError,
+    PlaceholderError,
+    ResolveError,
+    SemQLError,
+    UnknownIdentifierError,
+)
+from semql.introspect import (
+    CATALOG_CUBES,
+    CATALOG_DIMENSIONS,
+    CATALOG_MEASURES,
+    META_CUBES,
+    ResolvedQuery,
+    iter_cubes,
+    iter_fields,
+    iter_joins,
+    resolve_field,
+    resolve_query,
+)
+from semql.model import (
+    AggLiteral,
+    AuthContext,
+    Backend,
+    BaseField,
+    ChartTypeLiteral,
+    Cube,
+    Dimension,
+    DimTypeLiteral,
+    FormatLiteral,
+    GranularityLiteral,
+    Join,
+    Measure,
+    ScopePredicate,
+    Segment,
+    TimeDimension,
+    View,
+)
+from semql.plan import (
+    DrilldownSuggestion,
+    DrilldownSuggestions,
+    Presentation,
+    QueryIntent,
+    QueryPlan,
+    QueryStep,
+    RouterDecision,
+    RouterPath,
+)
+from semql.prompt import (
+    build_drilldown_prompt_fragment,
+    build_planner_prompt_fragment,
+    build_presenter_prompt_fragment,
+    build_query_generator_prompt_fragment,
+    build_router_prompt_fragment,
+    render_catalogue_block,
+)
+from semql.safe import is_safe_select
+from semql.spec import (
+    BoolExpr,
+    CompareWindow,
+    Filter,
+    FilterOp,
+    SemanticQuery,
+    TimeWindow,
+)
+from semql.validate import ValidationError, validate
+from semql.visualize import VizColumn, VizDecision, decide_visualization
+
+__all__ = [
+    "AggLiteral",
+    "AuthContext",
+    "Backend",
+    "BaseField",
+    "BoolExpr",
+    "CATALOG_CUBES",
+    "CATALOG_DIMENSIONS",
+    "CATALOG_MEASURES",
+    "Catalog",
+    "ChartTypeLiteral",
+    "CompareWindow",
+    "CompileError",
+    "Compiled",
+    "CrossBackendError",
+    "Cube",
+    "DimTypeLiteral",
+    "Dimension",
+    "DrilldownSuggestion",
+    "DrilldownSuggestions",
+    "Filter",
+    "FilterOp",
+    "FilterTypeError",
+    "FormatLiteral",
+    "GranularityLiteral",
+    "Join",
+    "JoinPathError",
+    "MAX_UNGROUPED_ROWS",
+    "META_CUBES",
+    "Measure",
+    "PhaseDeferredError",
+    "PlaceholderError",
+    "Presentation",
+    "QueryIntent",
+    "QueryPlan",
+    "QueryStep",
+    "ResolveError",
+    "ResolvedQuery",
+    "RouterDecision",
+    "RouterPath",
+    "ScopePredicate",
+    "Segment",
+    "SemQLError",
+    "SemanticQuery",
+    "TimeDimension",
+    "TimeWindow",
+    "UnknownIdentifierError",
+    "ValidationError",
+    "View",
+    "VizColumn",
+    "VizDecision",
+    "build_drilldown_prompt_fragment",
+    "build_planner_prompt_fragment",
+    "build_presenter_prompt_fragment",
+    "build_query_generator_prompt_fragment",
+    "build_router_prompt_fragment",
+    "compile_query",
+    "decide_visualization",
+    "is_safe_select",
+    "iter_cubes",
+    "iter_fields",
+    "iter_joins",
+    "render_catalog_markdown",
+    "render_catalogue_block",
+    "resolve_field",
+    "resolve_query",
+    "validate",
+]

semql-0.1.0/src/semql/__main__.py

@@ -0,0 +1,110 @@
+"""``python -m semql.compile``  CLI.
+
+Compiles a ``SemanticQuery`` JSON spec against a Catalog declared
+in a Python module and prints the SQL + params to stdout. Useful
+for ad-hoc cube authoring (no need to write a runner script) and
+as a smoke target in CI.
+
+    python -m semql.compile --catalog mypkg.catalogs:default \\
+        '{"measures": ["orders.revenue"], "dimensions": ["orders.region"]}'
+
+The ``--catalog`` arg is ``module.path:attr`` — the module is
+imported, the named attribute must be a ``Catalog`` instance.
+Context substitutions for ``{schema}`` / ``{tenant}`` placeholders
+go through repeated ``--context key=value`` flags.
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib
+import json
+import sys
+from typing import Any
+
+from semql import Catalog, SemanticQuery
+
+
+def _load_catalog(spec: str) -> Catalog:
+    """Resolve ``module.path:attr`` into the named ``Catalog``."""
+    if ":" not in spec:
+        raise SystemExit(
+            f"--catalog must be 'module.path:attr', got {spec!r}. "
+            "Example: --catalog mypkg.catalogs:default"
+        )
+    module_path, attr = spec.rsplit(":", 1)
+    try:
+        module = importlib.import_module(module_path)
+    except ModuleNotFoundError as exc:
+        raise SystemExit(f"Could not import {module_path!r}: {exc}") from exc
+    try:
+        catalog = getattr(module, attr)
+    except AttributeError as exc:
+        raise SystemExit(f"Module {module_path!r} has no attribute {attr!r}.") from exc
+    if not isinstance(catalog, Catalog):
+        raise SystemExit(f"{spec!r} resolved to {type(catalog).__name__}, not semql.Catalog.")
+    return catalog
+
+
+def _parse_context(pairs: list[str]) -> dict[str, str]:
+    """Parse repeated ``--context key=value`` flags into a dict."""
+    out: dict[str, str] = {}
+    for pair in pairs:
+        if "=" not in pair:
+            raise SystemExit(f"--context expects key=value, got {pair!r}.")
+        k, v = pair.split("=", 1)
+        out[k] = v
+    return out
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(
+        prog="python -m semql.compile",
+        description="Compile a SemanticQuery JSON spec to SQL.",
+    )
+    parser.add_argument(
+        "--catalog",
+        required=True,
+        help="Catalog locator: module.path:attr (e.g. mypkg.catalogs:default).",
+    )
+    parser.add_argument(
+        "--context",
+        action="append",
+        default=[],
+        help="Compile-time context pair (key=value). May be repeated.",
+    )
+    parser.add_argument(
+        "--params-format",
+        choices=("comment", "json"),
+        default="comment",
+        help="How to print the params: as a SQL comment (default) or a JSON line.",
+    )
+    parser.add_argument(
+        "spec",
+        help="SemanticQuery as a JSON string. Use '-' to read from stdin.",
+    )
+    args = parser.parse_args(argv)
+
+    raw_spec = sys.stdin.read() if args.spec == "-" else args.spec
+    try:
+        spec_dict: dict[str, Any] = json.loads(raw_spec)
+    except json.JSONDecodeError as exc:
+        raise SystemExit(f"--spec must be JSON: {exc}") from exc
+    query = SemanticQuery.model_validate(spec_dict)
+
+    catalog = _load_catalog(args.catalog)
+    ctx = _parse_context(args.context)
+
+    compiled = catalog.compile(query, context=ctx)
+
+    print(compiled.sql)
+    if args.params_format == "json":
+        print(json.dumps(compiled.params, default=str))
+    else:
+        print(f"-- params: {json.dumps(compiled.params, default=str)}")
+        print(f"-- columns: {compiled.columns}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

semql-0.1.0/src/semql/_resolve.py

@@ -0,0 +1,63 @@
+"""Shared identifier resolution for the semantic layer.
+
+Both `compile.py` and `visualize.py` parse the planner's `cube.field`
+references against the catalogue. Keeping a single resolver here makes
+the validation, regex shape, and error class consistent across both
+modules.
+"""
+
+from __future__ import annotations
+
+import re
+
+from semql.errors import ResolveError, UnknownIdentifierError, closest_match
+from semql.model import Cube, Dimension, Measure, TimeDimension
+
+_QUALIFIED_RE = re.compile(r"^([a-z_][a-z0-9_]*)\.([a-z_][a-z0-9_]*)$", re.IGNORECASE)
+
+
+def split(qualified: str) -> tuple[str, str]:
+    m = _QUALIFIED_RE.match(qualified)
+    if not m:
+        raise ResolveError(f"Field reference must be 'cube.field', got: {qualified!r}")
+    return m.group(1), m.group(2)
+
+
+def resolve_field(
+    qualified: str,
+    catalog: dict[str, Cube],
+) -> tuple[Cube, Measure | Dimension | TimeDimension]:
+    cube_name, field_name = split(qualified)
+    if cube_name not in catalog:
+        hint = closest_match(cube_name, catalog.keys())
+        known = ", ".join(sorted(catalog))
+        suffix = f" Did you mean {hint!r}?" if hint else ""
+        raise UnknownIdentifierError(
+            f"Unknown cube: {cube_name!r}. Known cubes: {known}.{suffix}",
+            kind="cube",
+            name=cube_name,
+            hint=hint,
+        )
+    cube = catalog[cube_name]
+    for m in cube.measures:
+        if m.name == field_name:
+            return cube, m
+    for d in cube.dimensions:
+        if d.name == field_name:
+            return cube, d
+    for td in cube.time_dimensions:
+        if td.name == field_name:
+            return cube, td
+    hint = closest_match(field_name, cube.field_names())
+    known = ", ".join(sorted(cube.field_names()))
+    suffix = f" Did you mean {hint!r}?" if hint else ""
+    raise UnknownIdentifierError(
+        f"Unknown field {field_name!r} on cube {cube_name!r}. Known fields: {known}.{suffix}",
+        kind="field",
+        name=field_name,
+        cube=cube_name,
+        hint=hint,
+    )
+
+
+__all__ = ["ResolveError", "UnknownIdentifierError", "split", "resolve_field"]