table-validator 0.1.0__py3-none-any.whl

table_validator-0.1.0.dist-info/METADATA

@@ -0,0 +1,190 @@
+Metadata-Version: 2.4
+Name: table-validator
+Version: 0.1.0
+Summary: CLI tool for validating data migrations between Azure (Blob Storage / SQL Database) and Databricks Delta Lake catalogs
+License: MIT
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: typer>=0.12
+Requires-Dist: rich>=13.0
+Requires-Dist: questionary>=2.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: pyodbc>=4.0
+Requires-Dist: databricks-sql-connector>=3.0
+Requires-Dist: pandas>=2.0
+Requires-Dist: numpy>=1.24
+Requires-Dist: azure-storage-blob>=12.14
+Requires-Dist: openpyxl>=3.1
+Requires-Dist: pyarrow>=14.0
+Requires-Dist: PyYAML>=6.0
+Requires-Dist: python-dotenv>=1.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: ruff>=0.4; extra == "dev"
+Requires-Dist: black>=24.0; extra == "dev"
+Dynamic: license-file
+
+# table-validator
+
+Cross-platform data migration validator. Compares a source table against a
+target Databricks table/catalog and reports whether the migration is
+correct: matching schema, row counts, column statistics, and (where a
+difference is found) the exact row/column that changed. Produces a
+multi-sheet Excel report and a pass/fail summary.
+
+Currently supported sources (target is always Databricks):
+
+- Databricks catalog → Databricks catalog
+- Azure Blob Storage (CSV / Excel / Parquet) → Databricks table
+- Azure SQL Database → Databricks catalog
+
+More source platforms can be added behind the same connector/validator
+interfaces described below.
+
+## Install
+
+```bash
+pip install table-validator
+```
+
+From a local checkout (the directory containing `pyproject.toml`):
+
+```bash
+pip install .
+```
+
+For local development (editable install, so code changes take effect
+without reinstalling):
+
+```bash
+pip install -e ".[dev]"
+```
+
+Either way, this installs the `tablevalidator` command on your PATH.
+
+## CLI usage
+
+```bash
+tablevalidator info
+```
+
+Prints what the tool does and the commands below, in the order you'd
+normally run them.
+
+```bash
+tablevalidator configure
+```
+
+Interactive wizard that walks you through:
+
+1. Azure Storage account + container (optional, skip if you don't have a Blob source) and account key
+2. Azure SQL server + database (optional, skip if you don't have a SQL source) and username/password
+3. Databricks workspace URL, SQL Warehouse HTTP path, and personal access token
+4. Source table (catalog / schema / table)
+5. Target table (catalog / schema / table)
+6. Which validations to run (catalog / schema / column / row)
+
+```bash
+tablevalidator validate
+```
+
+Runs the comparison using the saved configuration and writes
+`validation_report.xlsx` in the current directory, printing a pass/fail
+summary to the console. Exit code is `0` if the overall result is PASS,
+non-zero otherwise (useful in CI). Useful flags:
+
+```bash
+tablevalidator validate --config-path /path/to/config.yaml --output /path/to/report.xlsx
+```
+
+```bash
+tablevalidator open
+```
+
+Opens the most recently generated report in your default spreadsheet app.
+
+## Quickstart (Python API)
+
+Everything the CLI does is available as a library, built from the same
+public API exported by `table_validator/__init__.py`:
+
+```python
+from table_validator import (
+    load_config,
+    CatalogValidator,
+    CatalogValidationRequest,
+    DatabricksConnector,
+)
+from table_validator.auth.databricks_auth import get_databricks_token
+
+# Non-secret settings from ~/.table_validator/config.yaml
+# (see `tablevalidator configure`); secrets from ~/.table_validator/.env.
+config = load_config()
+token = get_databricks_token(config)
+
+# DatabricksConnector wants a bare hostname, not the full workspace URL
+host = config.databricks.workspace_url.replace("https://", "").split("/")[0]
+
+databricks = DatabricksConnector(
+    host=host,
+    token=token,
+    http_path=config.databricks.http_path,
+)
+
+validator = CatalogValidator(databricks)
+
+request = CatalogValidationRequest(
+    source_catalog="source_catalog_name",
+    target_catalog="target_catalog_name",
+    schemas=["sales"],          # optional: restrict scope
+    primary_keys={"sales.orders": ["order_id"]},  # optional: enables row-level diffing
+)
+
+result = validator.compare_catalogs(request)
+
+print(result.status)  # PASS / FAIL / ERROR / SKIPPED
+```
+
+Other public entry points exported from `table_validator`:
+
+- `AzureCsvValidator` / `AzureSqlValidator` — the Blob-CSV and Azure-SQL
+  equivalents of `CatalogValidator`, returning the same
+  `CatalogValidationResponse` shape.
+- `BlobCatalogValidator` — validates every file in an Azure Blob container
+  against like-named Databricks tables.
+- `AzureConnector` / `AzureSqlConnector` — the Azure-side connectors, for
+  building your own validation flow against a connector directly.
+- `ValidatorConfig`, `default_config`, `save_config`, `require_config`,
+  `ConfigNotFoundError` — the same config load/save layer the CLI wizard
+  uses, if you want to construct or persist configuration programmatically.
+
+## Where config and secrets are stored
+
+Everything lives outside the repo, under your home directory:
+
+- `~/.table_validator/config.yaml` — non-secret configuration (table
+  references, workspace URL, which validations are enabled). Safe to
+  inspect or version-control separately if you want.
+- `~/.table_validator/.env` — credentials (Azure Storage key, Azure SQL
+  username/password, Databricks personal access token), written in
+  plaintext and restricted to owner read/write only (`chmod 600`,
+  best-effort on Windows since NTFS doesn't map POSIX permission bits).
+  **Never commit this file or add it inside the project repo** — it isn't,
+  by construction, since it's written under your home directory rather
+  than the working directory.
+
+A future version will replace manual credential entry with Azure CLI /
+Service Principal auth and Databricks CLI / OAuth login, without changing
+the config file format or any command usage above.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

table_validator-0.1.0.dist-info/RECORD

@@ -0,0 +1,30 @@
+table_validator/__init__.py,sha256=3Hvlk-yRCL8HAwONirbtPA3POjE5qcSZz2oU_bQCN5o,1447
+table_validator/models.py,sha256=7hr_KjxNYhJmsc0PAc9-woDf__Wp2j1XAfU2Vo5BjfQ,32116
+table_validator/auth/__init__.py,sha256=Spncoe3HL7NI6OgaRBmIwJMKeMUdVLrP4lBaIcBNYOY,70
+table_validator/auth/azure_auth.py,sha256=Dximt9w57P4g7MjoJBSOGdxCyowrarqyix8BCRvSqnU,1805
+table_validator/auth/databricks_auth.py,sha256=C71kUC9ivlqgyQU-8O7PCE-dwJlmhaZ0lcsT7D5EL3k,1086
+table_validator/cli/__init__.py,sha256=ofbwCuoGUlklWwECq86anrbcn1u6hISyKQHnlrDjdMY,67
+table_validator/cli/main.py,sha256=1_-fSv_bVf8HvfWDc9vTDtsODfgCo0olXjegO8ydNhs,30302
+table_validator/cli/partition_prompt.py,sha256=iyWOtHymg9nPDJnVSPoK_imLNBlVIwO3RSdPB3RA2Vo,2623
+table_validator/cli/summary_table.py,sha256=xeIqkATlmlyuYkmpJnh5xk7cKRU-eR1lqUfkpwHS2ck,5634
+table_validator/cli/wizard.py,sha256=8K_Z8xqnzG9Wgx5rekhmq3okq99eVo6ZIMSO-dQD86A,17601
+table_validator/config/__init__.py,sha256=VkYHiQaykkfks5p75tA0gD8gG4wTJsZnQphmdRIT-oA,91
+table_validator/config/manager.py,sha256=wz7htUQ5bwFg1AakG_57U3Zu9OhkPDXY0nFFCgXiOdo,2861
+table_validator/config/schema.py,sha256=QFV4T0qPTUThbs0qXvRl8rvLq98Le63bufDAfls4E6s,6405
+table_validator/connectors/__init__.py,sha256=W9LEtRaS6Ia2GQKMYjRdRCiDB47sEShVxIgmiH3_wrQ,91
+table_validator/connectors/azure_connector.py,sha256=1cWb9jD2zdBHqfZg6qEKLXodyc4m4x-a6OXuIXIFETM,29306
+table_validator/connectors/databricks_connector.py,sha256=dUwggjdJbq7H38umLCF_zwEvUq43KWna_mZzJyIZxgs,43640
+table_validator/engine/__init__.py,sha256=6SoRLYTB6tY0uorhYkjKgBGCSzIE7IsCJw57VoTLC0Q,73
+table_validator/engine/comparison_engine.py,sha256=fxEu4ojofugQ8rtTxjgOAoxGqovELjuw4sNzlYbGXiQ,16781
+table_validator/reports/__init__.py,sha256=z9jtmU9c3NzeDxOFw1jTmHee10mZD2FVuFTqoOu4S1s,64
+table_validator/reports/excel_report.py,sha256=6vZSglCvePqM5SimcKEraFwSpOwfa7CRz7EiMIAJDNQ,39940
+table_validator/validators/__init__.py,sha256=uwh7itYLr6KQ_Fh9U0Z1OoTCdP4LIGXHNM22A2vVVHs,85
+table_validator/validators/blob_discovery.py,sha256=kM9rjneIQt5pfR5dSmITa2kcK1FMCMvK6GjPNU7QXE8,19008
+table_validator/validators/catalog_validator.py,sha256=UDublQOGGfrjDddylV58LqqfxR1zUUzZ4LTQsix9dSE,81635
+table_validator/validators/row_validator.py,sha256=kpl2HkRcbHomGsXCzDaKcDuNWSTpqIkt5CtctA8N7sE,72307
+table_validator-0.1.0.dist-info/licenses/LICENSE,sha256=6vZlKQZ-cKinHW_IxFFav-By29-j61dLdYoVAq-cVj8,1065
+table_validator-0.1.0.dist-info/METADATA,sha256=4iuetgMdhDzTpoobejeMtZnViZmX_gpTBW2mtRJdhLU,6029
+table_validator-0.1.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+table_validator-0.1.0.dist-info/entry_points.txt,sha256=2M-96rIihIW-reYiIdmNDElLfYC_PziuJKMGg_hR0KI,64
+table_validator-0.1.0.dist-info/top_level.txt,sha256=svExNN_O7_T2bOza83z2qeBMzcmPg6RruFIRDZJbcRI,16
+table_validator-0.1.0.dist-info/RECORD,,

table_validator-0.1.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

table_validator-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+tablevalidator = table_validator.cli.main:app

table_validator-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mourya S
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

table_validator-0.1.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+table_validator