al-data-utils 0.3.1__tar.gz → 0.3.3__tar.gz

{al_data_utils-0.3.1/src/al_data_utils.egg-info → al_data_utils-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: al-data-utils
-Version: 0.3.1
+Version: 0.3.3
 Summary: Reusable utilities for data engineering (Airflow, Azure, Snowflake, SQL, config).
 Author-email: AL Data Labs <pypi@aldatalabs.com>
 License-Expression: MIT
@@ -12,6 +12,7 @@ Classifier: Topic :: Software Development :: Libraries
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: pyyaml
 Dynamic: license-file
 
 # al-data-utils
@@ -20,40 +21,31 @@ Reusable Python utilities for data and automation workflows.
 
 This repository contains independent utility modules for sync operations, git automation, SQL work, logging helpers, and MSSQL schema snapshot generation.
 
-## Python and Setup
-
-- Python: 3.10+
-- Packaging: setuptools (src layout)
-
-Install locally in editable mode:
+## Install
 
 ```bash
-pip install -e .
+pip install al-data-utils
 ```
 
-## Project Structure
-
-- src/al_data_utils
-- examples
-- development_guides
-- tools
+Python 3.10+. Runtime dependencies (`pyyaml`, `sqlalchemy`, `pandas`, `requests`, `msal`, ...) are expected from the host environment, typically an Airflow image.
 
-Main utility groups under src/al_data_utils:
+## Modules
 
-- airflow_utils
-- backup_utils
-- config_utils
-- files_sync_utils
-- geo_utils
-- git_utils
-- logging_utils
-- ones_api_utils
-- sql_utils
-- tg_utils
+- `airflow_utils` - Airflow Variable / connection helpers
+- `azure_as_utils` - Azure Analysis Services model refresh (REST + MSAL)
+- `backup_utils` - MSSQL schema snapshot to filesystem
+- `config_utils` - YAML config loading
+- `files_sync_utils` - folder inventory, sync plan, copy, retention
+- `geo_utils` - geo helpers
+- `git_utils` - git automation wrappers
+- `logging_utils` - unified logging
+- `ones_api_utils` - 1C/ОneS API client
+- `sql_utils` - `UnifiedSQLClient` (MSSQL / PostgreSQL / Snowflake via SQLAlchemy)
+- `tg_utils` - Telegram helpers
 
 ## Examples
 
-Runnable examples are available in the `examples` folder for the main utility groups.
+Runnable examples are available in the `examples` folder of the repository for the main utility groups.
 Use them as reference implementations for local scripts and automation jobs.
 
 ## MSSQL Schema Snapshot
@@ -145,13 +137,6 @@ Notes:
 
 `git_utils` wraps common git automation operations with consistent behavior and error handling.
 
-## Development Guides
-
-- development_guides/dev_guide.txt
-- development_guides/auth_key_guide.txt
-
-These guides contain release flow notes and auth-related instructions.
-
 ## Changelog
 
-See CHANGELOG.md for release notes.
+See `CHANGELOG.md` in the repository for release notes.

{al_data_utils-0.3.1 → al_data_utils-0.3.3}/README.md

@@ -1,141 +1,125 @@
-# al-data-utils
-
-Reusable Python utilities for data and automation workflows.
-
-This repository contains independent utility modules for sync operations, git automation, SQL work, logging helpers, and MSSQL schema snapshot generation.
-
-## Python and Setup
-
-- Python: 3.10+
-- Packaging: setuptools (src layout)
-
-Install locally in editable mode:
-
-```bash
-pip install -e .
-```
-
-## Project Structure
-
-- src/al_data_utils
-- examples
-- development_guides
-- tools
-
-Main utility groups under src/al_data_utils:
-
-- airflow_utils
-- backup_utils
-- config_utils
-- files_sync_utils
-- geo_utils
-- git_utils
-- logging_utils
-- ones_api_utils
-- sql_utils
-- tg_utils
-
-## Examples
-
-Runnable examples are available in the `examples` folder for the main utility groups.
-Use them as reference implementations for local scripts and automation jobs.
-
-## MSSQL Schema Snapshot
-
-The snapshot tool exports SQL objects from MSSQL into a filesystem layout and syncs staging output to destination folders.
-
-Current behavior highlights:
-
-- Supports single DB and multi-DB runs in one execution.
-- Always writes outputs into per-DB folders.
-- Uses staging first, then syncs to final output.
-- Supports cleanup of staging folders.
-- Supports sync log retention cleanup.
-- Supports include and exclude filters per object type.
-- Supports DataGrip template override files.
-
-Output shape:
-
-```text
-<MSSQL_OUT_DIR>/<DatabaseName>/schemas/<schema>/
-	tables/
-	views/
-	routines/
-	sequences/
-	synonyms/
-```
-
-### Snapshot Environment Variables
-
-Core connection and output:
-
-- MSSQL_OUT_DIR
-- MSSQL_SERVER
-- MSSQL_DATABASE
-- MSSQL_DATABASES
-- MSSQL_AUTH_MODE
-- MSSQL_USER
-- MSSQL_PASSWORD
-- MSSQL_DRIVER
-
-Snapshot behavior:
-
-- MSSQL_ENABLE_TABLE_DEFINITIONS
-- MSSQL_ENABLE_INDEXES
-- MSSQL_ENABLE_FOREIGN_KEYS
-- MSSQL_ENABLE_VIEWS
-- MSSQL_ENABLE_STORED_PROCEDURES
-- MSSQL_ENABLE_FUNCTIONS
-- MSSQL_ENABLE_SEQUENCES
-- MSSQL_ENABLE_SYNONYMS
-- MSSQL_CLEANUP_STAGING
-- MSSQL_SYNC_LOG_RETENTION_DAYS
-
-Filters:
-
-- MSSQL_INCLUDE_SCHEMAS
-- MSSQL_EXCLUDE_SCHEMAS
-- MSSQL_INCLUDE_TABLES
-- MSSQL_EXCLUDE_TABLES
-- MSSQL_INCLUDE_VIEWS
-- MSSQL_EXCLUDE_VIEWS
-- MSSQL_INCLUDE_STORED_PROCEDURES
-- MSSQL_EXCLUDE_STORED_PROCEDURES
-- MSSQL_INCLUDE_FUNCTIONS
-- MSSQL_EXCLUDE_FUNCTIONS
-- MSSQL_INCLUDE_SEQUENCES
-- MSSQL_EXCLUDE_SEQUENCES
-- MSSQL_INCLUDE_SYNONYMS
-- MSSQL_EXCLUDE_SYNONYMS
-
-Notes:
-
-- If MSSQL_DATABASES is set, it is treated as a comma-separated list.
-- If MSSQL_DATABASES is empty, MSSQL_DATABASE is used.
-- If MSSQL_DATABASE contains commas, it is also treated as a list for backward compatibility.
-
-## File Sync Utilities
-
-`files_sync_utils` provides:
-
-- Inventory building
-- Plan generation
-- Copy and update execution
-- Extra destination cleanup rules
-- Activity CSV logging
-- Log retention cleanup
-
-## Git Utilities
-
-`git_utils` wraps common git automation operations with consistent behavior and error handling.
-
-## Development Guides
-
-- development_guides/dev_guide.txt
-- development_guides/auth_key_guide.txt
-
-These guides contain release flow notes and auth-related instructions.
-
-## Changelog
-
-See CHANGELOG.md for release notes.
+# al-data-utils
+
+Reusable Python utilities for data and automation workflows.
+
+This repository contains independent utility modules for sync operations, git automation, SQL work, logging helpers, and MSSQL schema snapshot generation.
+
+## Install
+
+```bash
+pip install al-data-utils
+```
+
+Python 3.10+. Runtime dependencies (`pyyaml`, `sqlalchemy`, `pandas`, `requests`, `msal`, ...) are expected from the host environment, typically an Airflow image.
+
+## Modules
+
+- `airflow_utils` - Airflow Variable / connection helpers
+- `azure_as_utils` - Azure Analysis Services model refresh (REST + MSAL)
+- `backup_utils` - MSSQL schema snapshot to filesystem
+- `config_utils` - YAML config loading
+- `files_sync_utils` - folder inventory, sync plan, copy, retention
+- `geo_utils` - geo helpers
+- `git_utils` - git automation wrappers
+- `logging_utils` - unified logging
+- `ones_api_utils` - 1C/ОneS API client
+- `sql_utils` - `UnifiedSQLClient` (MSSQL / PostgreSQL / Snowflake via SQLAlchemy)
+- `tg_utils` - Telegram helpers
+
+## Examples
+
+Runnable examples are available in the `examples` folder of the repository for the main utility groups.
+Use them as reference implementations for local scripts and automation jobs.
+
+## MSSQL Schema Snapshot
+
+The snapshot tool exports SQL objects from MSSQL into a filesystem layout and syncs staging output to destination folders.
+
+Current behavior highlights:
+
+- Supports single DB and multi-DB runs in one execution.
+- Always writes outputs into per-DB folders.
+- Uses staging first, then syncs to final output.
+- Supports cleanup of staging folders.
+- Supports sync log retention cleanup.
+- Supports include and exclude filters per object type.
+- Supports DataGrip template override files.
+
+Output shape:
+
+```text
+<MSSQL_OUT_DIR>/<DatabaseName>/schemas/<schema>/
+	tables/
+	views/
+	routines/
+	sequences/
+	synonyms/
+```
+
+### Snapshot Environment Variables
+
+Core connection and output:
+
+- MSSQL_OUT_DIR
+- MSSQL_SERVER
+- MSSQL_DATABASE
+- MSSQL_DATABASES
+- MSSQL_AUTH_MODE
+- MSSQL_USER
+- MSSQL_PASSWORD
+- MSSQL_DRIVER
+
+Snapshot behavior:
+
+- MSSQL_ENABLE_TABLE_DEFINITIONS
+- MSSQL_ENABLE_INDEXES
+- MSSQL_ENABLE_FOREIGN_KEYS
+- MSSQL_ENABLE_VIEWS
+- MSSQL_ENABLE_STORED_PROCEDURES
+- MSSQL_ENABLE_FUNCTIONS
+- MSSQL_ENABLE_SEQUENCES
+- MSSQL_ENABLE_SYNONYMS
+- MSSQL_CLEANUP_STAGING
+- MSSQL_SYNC_LOG_RETENTION_DAYS
+
+Filters:
+
+- MSSQL_INCLUDE_SCHEMAS
+- MSSQL_EXCLUDE_SCHEMAS
+- MSSQL_INCLUDE_TABLES
+- MSSQL_EXCLUDE_TABLES
+- MSSQL_INCLUDE_VIEWS
+- MSSQL_EXCLUDE_VIEWS
+- MSSQL_INCLUDE_STORED_PROCEDURES
+- MSSQL_EXCLUDE_STORED_PROCEDURES
+- MSSQL_INCLUDE_FUNCTIONS
+- MSSQL_EXCLUDE_FUNCTIONS
+- MSSQL_INCLUDE_SEQUENCES
+- MSSQL_EXCLUDE_SEQUENCES
+- MSSQL_INCLUDE_SYNONYMS
+- MSSQL_EXCLUDE_SYNONYMS
+
+Notes:
+
+- If MSSQL_DATABASES is set, it is treated as a comma-separated list.
+- If MSSQL_DATABASES is empty, MSSQL_DATABASE is used.
+- If MSSQL_DATABASE contains commas, it is also treated as a list for backward compatibility.
+
+## File Sync Utilities
+
+`files_sync_utils` provides:
+
+- Inventory building
+- Plan generation
+- Copy and update execution
+- Extra destination cleanup rules
+- Activity CSV logging
+- Log retention cleanup
+
+## Git Utilities
+
+`git_utils` wraps common git automation operations with consistent behavior and error handling.
+
+## Changelog
+
+See `CHANGELOG.md` in the repository for release notes.

{al_data_utils-0.3.1 → al_data_utils-0.3.3}/pyproject.toml

@@ -1,58 +1,60 @@
-[project]
-name = "al-data-utils"
-version = "0.3.1"
-description = "Reusable utilities for data engineering (Airflow, Azure, Snowflake, SQL, config)."
-authors = [{ name = "AL Data Labs", email = "pypi@aldatalabs.com" }]
-license = "MIT"
-license-files = ["LICENSE"]
-keywords = ["airflow", "data-engineering", "azure", "snowflake", "sql"]
-classifiers = [
-    "Programming Language :: Python :: 3",
-    "Operating System :: OS Independent",
-    "Topic :: Software Development :: Libraries",
-]
-readme = "README.md"
-requires-python = ">=3.10"
-dependencies = [] # keep empty for now, using airflow environment's dependencies
-
-[project.urls]
-Homepage = "https://aldatalabs.com"
-
-
-[build-system]
-requires = ["setuptools>=77"]
-build-backend = "setuptools.build_meta"
-
-[tool.setuptools]
-package-dir = {"" = "src"}
-
-[tool.setuptools.packages.find]
-where = ["src"]
-
-# Ruff configuration
-[tool.ruff]
-# Exclude a variety of commonly ignored directories.
-exclude = [
-    ".git",
-    ".venv",
-]
-
-# Assume Python 3.10
-target-version = "py310"
-
-# Same as Black.
-line-length = 88
-indent-width = 4
-
-
-[tool.ruff.format]
-quote-style = "single"
-
-# Like Black, indent with spaces, rather than tabs.
-indent-style = "space"
-
-# Like Black, respect magic trailing commas.
-skip-magic-trailing-comma = true
-
-# Like Black, automatically detect the appropriate line ending.
+[project]
+name = "al-data-utils"
+version = "0.3.3"
+description = "Reusable utilities for data engineering (Airflow, Azure, Snowflake, SQL, config)."
+authors = [{ name = "AL Data Labs", email = "pypi@aldatalabs.com" }]
+license = "MIT"
+license-files = ["LICENSE"]
+keywords = ["airflow", "data-engineering", "azure", "snowflake", "sql"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Operating System :: OS Independent",
+    "Topic :: Software Development :: Libraries",
+]
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+    "pyyaml",
+] # other runtime deps (sqlalchemy, pandas, requests, msal, airflow) come from the host environment
+
+[project.urls]
+Homepage = "https://aldatalabs.com"
+
+
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+# Ruff configuration
+[tool.ruff]
+# Exclude a variety of commonly ignored directories.
+exclude = [
+    ".git",
+    ".venv",
+]
+
+# Assume Python 3.10
+target-version = "py310"
+
+# Same as Black.
+line-length = 88
+indent-width = 4
+
+
+[tool.ruff.format]
+quote-style = "single"
+
+# Like Black, indent with spaces, rather than tabs.
+indent-style = "space"
+
+# Like Black, respect magic trailing commas.
+skip-magic-trailing-comma = true
+
+# Like Black, automatically detect the appropriate line ending.
 line-ending = "auto"

{al_data_utils-0.3.1 → al_data_utils-0.3.3}/src/al_data_utils/sql_utils/sql_utils.py

@@ -89,7 +89,7 @@ class UnifiedSQLClient:
         else:
             raise ValueError(f'Unsupported dialect: {dialect}')
 
-        self.engine = create_engine(self.connection_string, connect_args=connect_args)
+        self.engine = create_engine(self.connection_string, connect_args=connect_args, pool_pre_ping=True)
 
     @staticmethod
     def parse_dialect(value: str | SQLDialect) -> SQLDialect:

{al_data_utils-0.3.1 → al_data_utils-0.3.3/src/al_data_utils.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: al-data-utils
-Version: 0.3.1
+Version: 0.3.3
 Summary: Reusable utilities for data engineering (Airflow, Azure, Snowflake, SQL, config).
 Author-email: AL Data Labs <pypi@aldatalabs.com>
 License-Expression: MIT
@@ -12,6 +12,7 @@ Classifier: Topic :: Software Development :: Libraries
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: pyyaml
 Dynamic: license-file
 
 # al-data-utils
@@ -20,40 +21,31 @@ Reusable Python utilities for data and automation workflows.
 
 This repository contains independent utility modules for sync operations, git automation, SQL work, logging helpers, and MSSQL schema snapshot generation.
 
-## Python and Setup
-
-- Python: 3.10+
-- Packaging: setuptools (src layout)
-
-Install locally in editable mode:
+## Install
 
 ```bash
-pip install -e .
+pip install al-data-utils
 ```
 
-## Project Structure
-
-- src/al_data_utils
-- examples
-- development_guides
-- tools
+Python 3.10+. Runtime dependencies (`pyyaml`, `sqlalchemy`, `pandas`, `requests`, `msal`, ...) are expected from the host environment, typically an Airflow image.
 
-Main utility groups under src/al_data_utils:
+## Modules
 
-- airflow_utils
-- backup_utils
-- config_utils
-- files_sync_utils
-- geo_utils
-- git_utils
-- logging_utils
-- ones_api_utils
-- sql_utils
-- tg_utils
+- `airflow_utils` - Airflow Variable / connection helpers
+- `azure_as_utils` - Azure Analysis Services model refresh (REST + MSAL)
+- `backup_utils` - MSSQL schema snapshot to filesystem
+- `config_utils` - YAML config loading
+- `files_sync_utils` - folder inventory, sync plan, copy, retention
+- `geo_utils` - geo helpers
+- `git_utils` - git automation wrappers
+- `logging_utils` - unified logging
+- `ones_api_utils` - 1C/ОneS API client
+- `sql_utils` - `UnifiedSQLClient` (MSSQL / PostgreSQL / Snowflake via SQLAlchemy)
+- `tg_utils` - Telegram helpers
 
 ## Examples
 
-Runnable examples are available in the `examples` folder for the main utility groups.
+Runnable examples are available in the `examples` folder of the repository for the main utility groups.
 Use them as reference implementations for local scripts and automation jobs.
 
 ## MSSQL Schema Snapshot
@@ -145,13 +137,6 @@ Notes:
 
 `git_utils` wraps common git automation operations with consistent behavior and error handling.
 
-## Development Guides
-
-- development_guides/dev_guide.txt
-- development_guides/auth_key_guide.txt
-
-These guides contain release flow notes and auth-related instructions.
-
 ## Changelog
 
-See CHANGELOG.md for release notes.
+See `CHANGELOG.md` in the repository for release notes.

{al_data_utils-0.3.1 → al_data_utils-0.3.3}/src/al_data_utils.egg-info/SOURCES.txt

@@ -5,6 +5,7 @@ src/al_data_utils/__init__.py
 src/al_data_utils.egg-info/PKG-INFO
 src/al_data_utils.egg-info/SOURCES.txt
 src/al_data_utils.egg-info/dependency_links.txt
+src/al_data_utils.egg-info/requires.txt
 src/al_data_utils.egg-info/top_level.txt
 src/al_data_utils/airflow_utils/__init__.py
 src/al_data_utils/airflow_utils/airflow_utils.py

al_data_utils-0.3.3/src/al_data_utils.egg-info/requires.txt

@@ -0,0 +1 @@
+pyyaml