analytics-toolkit 1.3.8.2__tar.gz → 1.3.10.7__tar.gz

{analytics_toolkit-1.3.8.2 → analytics_toolkit-1.3.10.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: analytics-toolkit
-Version: 1.3.8.2
+Version: 1.3.10.7
 Summary: Shared Python utilities for SQL, Excel, and date helpers.
 Author: analytics_toolkit contributors
 License: MIT License
@@ -75,7 +75,7 @@ Python toolkit for AB-test analysis, SQL workflows, Excel reports, and date help
 
 [Documentation Overview](https://github.com/Karapsin/analytics_toolkit/blob/main/docs/README.md)
 
-**Version:** `1.3.8.2`<br>
+**Version:** `1.3.10.7`<br>
 **Depends:** Python (`>=3.8,<3.15`)<br>
 **Imports:** [clickhouse-connect](https://pypi.org/project/clickhouse-connect/) (`>=0.5.14,<1`), [lz4](https://pypi.org/project/lz4/) (`>=4.3.2,<5`), [numpy](https://pypi.org/project/numpy/) (`>=1.24.2,<2`), [openpyxl](https://pypi.org/project/openpyxl/) (`>=3.1.1,<4`), [orjson](https://pypi.org/project/orjson/) (`>=3.8.7,<4`), [pandas](https://pypi.org/project/pandas/) (`>=1.4.4,<3`), [psycopg2-binary](https://pypi.org/project/psycopg2-binary/) (`>=2.9.5,<3`), [python-dateutil](https://pypi.org/project/python-dateutil/) (`>=2.8.2,<3`), [pytz](https://pypi.org/project/pytz/) (`>=2022.7`), [requests](https://pypi.org/project/requests/) (`>=2.28.2,<3`), [scipy](https://pypi.org/project/scipy/) (`>=1.10.1,<2`), [sqlglot](https://pypi.org/project/sqlglot/) (`>=20,<31`), [sqlparse](https://pypi.org/project/sqlparse/) (`>=0.4.3,<1`), [tqdm](https://pypi.org/project/tqdm/) (`>=4.65.0,<5`), [trino](https://pypi.org/project/trino/) (`>=0.320,<1`), [zstandard](https://pypi.org/project/zstandard/) (`>=0.20.0,<1`)<br>
 **Suggests:** [apache-airflow](https://pypi.org/project/apache-airflow/) (`>=2.4,<3`; optional extra `airflow`)<br>
@@ -103,6 +103,7 @@ pip install git+https://github.com/Karapsin/analytics_toolkit.git
 
 - `analytics_toolkit.ab_utils`: AB-test metric comparison helpers.
 - `analytics_toolkit.sql`: SQL read, execute, load, and transfer helpers.
+- `analytics_toolkit.sql_format`: SQL formatting, CTE rewrite, and Greenplum temp-table rewrite helpers.
 - `analytics_toolkit.excel`: Excel report helpers for long-format dataframes.
 - `analytics_toolkit.dates`: date and period helpers.
 - `analytics_toolkit.general`: shared logging and file path helpers.
@@ -150,6 +151,25 @@ rows = sql.transfer(
 - `sql.load_df`: load a pandas dataframe into a configured backend table.
 - `sql.transfer`: move rows from a source query to a target table across backends.
 
+## SQL Formatting
+
+[SQL formatting guide](https://github.com/Karapsin/analytics_toolkit/blob/main/docs/modules/sql_format/index.md)
+
+`sql_format.format_sql`, `sql_format.rewrite_with_ctes`, and
+`sql_format.gp_rewrite_to_temp_tables` transform SQL text locally without
+opening database connections. `GROUP BY` and `ORDER BY` clauses use SELECT-list
+ordinals by default, with expression-based formatting available through
+`group_by_format="expressions"` and `order_by_format="expressions"`.
+
+```python
+from analytics_toolkit import sql_format
+
+formatted = sql_format.format_sql(
+    "select user_id, amount from orders where amount > 100",
+    dialect="postgres",
+)
+```
+
 ## AB Metrics
 
 [All AB functions](https://github.com/Karapsin/analytics_toolkit/blob/main/docs/modules/ab_utils/functions/index.md)

{analytics_toolkit-1.3.8.2 → analytics_toolkit-1.3.10.7}/README.md

@@ -6,7 +6,7 @@ Python toolkit for AB-test analysis, SQL workflows, Excel reports, and date help
 
 [Documentation Overview](https://github.com/Karapsin/analytics_toolkit/blob/main/docs/README.md)
 
-**Version:** `1.3.8.2`<br>
+**Version:** `1.3.10.7`<br>
 **Depends:** Python (`>=3.8,<3.15`)<br>
 **Imports:** [clickhouse-connect](https://pypi.org/project/clickhouse-connect/) (`>=0.5.14,<1`), [lz4](https://pypi.org/project/lz4/) (`>=4.3.2,<5`), [numpy](https://pypi.org/project/numpy/) (`>=1.24.2,<2`), [openpyxl](https://pypi.org/project/openpyxl/) (`>=3.1.1,<4`), [orjson](https://pypi.org/project/orjson/) (`>=3.8.7,<4`), [pandas](https://pypi.org/project/pandas/) (`>=1.4.4,<3`), [psycopg2-binary](https://pypi.org/project/psycopg2-binary/) (`>=2.9.5,<3`), [python-dateutil](https://pypi.org/project/python-dateutil/) (`>=2.8.2,<3`), [pytz](https://pypi.org/project/pytz/) (`>=2022.7`), [requests](https://pypi.org/project/requests/) (`>=2.28.2,<3`), [scipy](https://pypi.org/project/scipy/) (`>=1.10.1,<2`), [sqlglot](https://pypi.org/project/sqlglot/) (`>=20,<31`), [sqlparse](https://pypi.org/project/sqlparse/) (`>=0.4.3,<1`), [tqdm](https://pypi.org/project/tqdm/) (`>=4.65.0,<5`), [trino](https://pypi.org/project/trino/) (`>=0.320,<1`), [zstandard](https://pypi.org/project/zstandard/) (`>=0.20.0,<1`)<br>
 **Suggests:** [apache-airflow](https://pypi.org/project/apache-airflow/) (`>=2.4,<3`; optional extra `airflow`)<br>
@@ -34,6 +34,7 @@ pip install git+https://github.com/Karapsin/analytics_toolkit.git
 
 - `analytics_toolkit.ab_utils`: AB-test metric comparison helpers.
 - `analytics_toolkit.sql`: SQL read, execute, load, and transfer helpers.
+- `analytics_toolkit.sql_format`: SQL formatting, CTE rewrite, and Greenplum temp-table rewrite helpers.
 - `analytics_toolkit.excel`: Excel report helpers for long-format dataframes.
 - `analytics_toolkit.dates`: date and period helpers.
 - `analytics_toolkit.general`: shared logging and file path helpers.
@@ -81,6 +82,25 @@ rows = sql.transfer(
 - `sql.load_df`: load a pandas dataframe into a configured backend table.
 - `sql.transfer`: move rows from a source query to a target table across backends.
 
+## SQL Formatting
+
+[SQL formatting guide](https://github.com/Karapsin/analytics_toolkit/blob/main/docs/modules/sql_format/index.md)
+
+`sql_format.format_sql`, `sql_format.rewrite_with_ctes`, and
+`sql_format.gp_rewrite_to_temp_tables` transform SQL text locally without
+opening database connections. `GROUP BY` and `ORDER BY` clauses use SELECT-list
+ordinals by default, with expression-based formatting available through
+`group_by_format="expressions"` and `order_by_format="expressions"`.
+
+```python
+from analytics_toolkit import sql_format
+
+formatted = sql_format.format_sql(
+    "select user_id, amount from orders where amount > 100",
+    dialect="postgres",
+)
+```
+
 ## AB Metrics
 
 [All AB functions](https://github.com/Karapsin/analytics_toolkit/blob/main/docs/modules/ab_utils/functions/index.md)

analytics_toolkit-1.3.10.7/analytics_toolkit/__init__.py

@@ -0,0 +1,3 @@
+"""analytics_toolkit package."""
+
+__all__ = ["ab_utils", "dates", "excel", "general", "sql", "sql_format"]

{analytics_toolkit-1.3.8.2 → analytics_toolkit-1.3.10.7}/analytics_toolkit/ab_utils/__init__.py

@@ -1,9 +1,10 @@
 """Utilities for AB-test related workflows."""
 
 from .metrics import (
-    MdePlanningOptions,
     RatioMetricSpec,
-    compute_mde_only,
+    compute_mde,
+    compute_mde_from_sql,
+    compute_mde_sql_native,
     compute_test_metrics,
     format_ab_metrics,
     parallel_compute_metrics,
@@ -12,9 +13,10 @@ from .metrics import (
 from .split import do_split
 
 __all__ = [
-    "MdePlanningOptions",
     "RatioMetricSpec",
-    "compute_mde_only",
+    "compute_mde",
+    "compute_mde_from_sql",
+    "compute_mde_sql_native",
     "compute_test_metrics",
     "do_split",
     "format_ab_metrics",

{analytics_toolkit-1.3.8.2 → analytics_toolkit-1.3.10.7}/analytics_toolkit/ab_utils/api.py

@@ -41,7 +41,7 @@ def compute_test_metrics(
     bootstrap_progress: bool = False,
     pre_exp_metrics_df: pd.DataFrame | None = None,
     outliers_quantile: float = 0.999,
-    outliers_policy: str = "truncate",
+    outliers_policy: str = "non_zero_truncate",
 ) -> pd.DataFrame:
     """Compute per-metric experiment comparison statistics.
 

{analytics_toolkit-1.3.8.2 → analytics_toolkit-1.3.10.7}/analytics_toolkit/ab_utils/metrics.py

@@ -3,7 +3,12 @@ from __future__ import annotations
 from .api import compute_test_metrics
 from .formatter import format_ab_metrics
 from .parallel import parallel_compute_metrics, parallel_compute_metrics_from_sql
-from .planning import MdePlanningOptions, RatioMetricSpec, compute_mde_only
+from .planning import (
+    RatioMetricSpec,
+    compute_mde,
+    compute_mde_from_sql,
+    compute_mde_sql_native,
+)
 from .bootstrap import (
     _apply_multiple_comparisons_adjustment,
     _compute_bootstrap_family_max_statistics,
@@ -87,9 +92,10 @@ from .validation import (
 __all__ = [
     "DEFAULT_ALPHA",
     "DEFAULT_POWER",
-    "MdePlanningOptions",
     "RatioMetricSpec",
-    "compute_mde_only",
+    "compute_mde",
+    "compute_mde_from_sql",
+    "compute_mde_sql_native",
     "compute_test_metrics",
     "format_ab_metrics",
     "parallel_compute_metrics",

{analytics_toolkit-1.3.8.2 → analytics_toolkit-1.3.10.7}/analytics_toolkit/ab_utils/outliers.py

@@ -50,7 +50,11 @@ def _build_outlier_context(
         return {
             "kind": "mean",
             "metric_key": metric_key,
-            "cutoff": _compute_outlier_cutoff(values, outliers_quantile),
+            "cutoff": _compute_outlier_cutoff(
+                values,
+                outliers_quantile,
+                outliers_policy,
+            ),
             "policy": outliers_policy,
         }
 
@@ -84,13 +88,23 @@ def _build_outlier_context(
         "kind": "ratio",
         "level": ratio_spec["level"],
         "metric_key": metric_key,
-        "cutoff": _compute_outlier_cutoff(values, outliers_quantile),
+        "cutoff": _compute_outlier_cutoff(
+            values,
+            outliers_quantile,
+            outliers_policy,
+        ),
         "policy": outliers_policy,
     }
 
 
-def _compute_outlier_cutoff(values: pd.Series, outliers_quantile: float) -> float:
+def _compute_outlier_cutoff(
+    values: pd.Series,
+    outliers_quantile: float,
+    outliers_policy: str,
+) -> float:
     nonmissing_values = values.dropna()
+    if outliers_policy == "non_zero_truncate":
+        nonmissing_values = nonmissing_values[nonmissing_values != 0]
     if nonmissing_values.empty:
         return math.nan
     return float(nonmissing_values.quantile(outliers_quantile))
@@ -105,7 +119,7 @@ def _apply_outliers_to_values(
     if outlier_mask.any() and outlier_context is not None:
         cutoff = float(outlier_context["cutoff"])
         policy = str(outlier_context["policy"])
-        if policy == "truncate":
+        if policy in {"truncate", "non_zero_truncate"}:
             transformed.loc[outlier_mask] = cutoff
         else:
             transformed.loc[outlier_mask] = np.nan
@@ -127,7 +141,7 @@ def _apply_outliers_to_agg_ratio_components(
     if outlier_mask.any() and outlier_context is not None:
         cutoff = float(outlier_context["cutoff"])
         policy = str(outlier_context["policy"])
-        if policy == "truncate":
+        if policy in {"truncate", "non_zero_truncate"}:
             transformed_numerator.loc[outlier_mask] = cutoff * denominator.loc[outlier_mask]
         else:
             transformed_numerator.loc[outlier_mask] = np.nan