exadata-validator 0.0.1__tar.gz → 1.2.0__tar.gz

exadata_validator-1.2.0/.gitignore

@@ -0,0 +1,156 @@
+# Intelij
+.idea/
+
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+#temp files
+*~
+*.swp
+*.swo
+*#*
+
+# vim stuff
+.vim/
+pyrightconfig.json
+.vimspector.json
+
+# macos
+**/.DS_Store
+
+# mip deployment config file
+.deployment.toml
+/configs
+/smpc_configs
+.stored_data/
+*.db
+/tests/test_data/.data_paths/
+/tests/test_data/localworker*/
+/logs/
+
+# LLMs
+.codex

exadata_validator-1.2.0/AGENTS.md

@@ -0,0 +1,47 @@
+# Agent Instructions: exadata-validator
+
+This directory is a separate uv-managed package for validating data-model folders
+using DuckDB. Follow the root `AGENTS.md` first, then these package-specific
+rules.
+
+## Package Overview
+
+`exadata-validator` exposes a CLI through `validator.commands:entry` and can
+also run as `python -m validator`.
+
+## Commands
+
+Run commands from `exadata-validator/`.
+
+Install:
+
+```bash
+uv sync --all-groups
+```
+
+Test:
+
+```bash
+uv run pytest -q
+```
+
+Build:
+
+```bash
+uv build
+```
+
+Smoke test an installed package as CI does:
+
+```bash
+python -m validator validate-data-model tests/data/success/data_model_longitudinal_v_1_0
+```
+
+## Rules
+
+- Keep dependency and lockfile changes local to this package unless the root
+  package also intentionally changes.
+- Preserve the CLI contract documented in `README.md`.
+- Add or update tests under `tests/` for validation behavior changes.
+- Treat CSV/CDE validation semantics as public behavior for package users.
+- Do not mix Exaflow service deployment concerns into this package.

{exadata_validator-0.0.1 → exadata_validator-1.2.0}/PKG-INFO

@@ -1,20 +1,18 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: exadata-validator
-Version: 0.0.1
+Version: 1.2.0
 Summary: Validate data-model folders using DuckDB.
 Author: Exaflow Team
-Requires-Python: >=3.10,<3.11
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Requires-Dist: click (>=8.1,<8.2)
-Requires-Dist: duckdb (>=1.1,<1.2)
+Requires-Python: <3.11,>=3.10
+Requires-Dist: click<8.2.0,>=8.1.0
+Requires-Dist: duckdb<1.2.0,>=1.1.0
 Description-Content-Type: text/markdown
 
 # exadata-validator
 
 `exadata-validator` validates data-model folders using DuckDB.
 
-## Install With pip
+## Install with pip
 
 ```bash
 python -m venv .venv
@@ -28,21 +26,21 @@ Validate a data model folder:
 exadata-validator validate-data-model /path/to/data_model_folder
 ```
 
-## Develop With Poetry
-
-From the repository root:
+Run as a Python module:
 
 ```bash
-cd data-validator/exaflow-data-validator
-poetry install
+python -m validator validate-data-model /path/to/data_model_folder
 ```
 
-Then run the same CLI through Poetry:
+Useful options:
 
 ```bash
-poetry run exadata-validator validate-data-model /path/to/data_model_folder
+exadata-validator validate-data-model /path/to/data_model_folder --fail-fast
+exadata-validator validate-data-model /path/to/data_model_folder --format html --output report.html
 ```
 
+By default the command collects and reports all validation errors in text format. If `--format html` is used without `--output`, the report is written under `/tmp`.
+
 Use `exadata-validator validate-data-model --help` for reporting, output, and threading options.
 
 ## Folder Layout
@@ -58,4 +56,3 @@ Use `exadata-validator validate-data-model --help` for reporting, output, and th
 
 - CSV validation queries files directly with DuckDB and uses fused aggregate checks to reduce scan overhead.
 - Folder-level dataset uniqueness is enforced across all CSV files via SQL using normalized codes (`trim + lower`).
-

{exadata_validator-0.0.1 → exadata_validator-1.2.0}/README.md

@@ -2,7 +2,7 @@
 
 `exadata-validator` validates data-model folders using DuckDB.
 
-## Install With pip
+## Install with pip
 
 ```bash
 python -m venv .venv
@@ -16,21 +16,21 @@ Validate a data model folder:
 exadata-validator validate-data-model /path/to/data_model_folder
 ```
 
-## Develop With Poetry
-
-From the repository root:
+Run as a Python module:
 
 ```bash
-cd data-validator/exaflow-data-validator
-poetry install
+python -m validator validate-data-model /path/to/data_model_folder
 ```
 
-Then run the same CLI through Poetry:
+Useful options:
 
 ```bash
-poetry run exadata-validator validate-data-model /path/to/data_model_folder
+exadata-validator validate-data-model /path/to/data_model_folder --fail-fast
+exadata-validator validate-data-model /path/to/data_model_folder --format html --output report.html
 ```
 
+By default the command collects and reports all validation errors in text format. If `--format html` is used without `--output`, the report is written under `/tmp`.
+
 Use `exadata-validator validate-data-model --help` for reporting, output, and threading options.
 
 ## Folder Layout

exadata_validator-1.2.0/pyproject.toml

@@ -0,0 +1,31 @@
+[project]
+name = "exadata-validator"
+version = "1.2.0"
+description = "Validate data-model folders using DuckDB."
+readme = "README.md"
+authors = [
+    { name = "Exaflow Team" },
+]
+requires-python = ">=3.10,<3.11"
+dependencies = [
+    "click>=8.1.0,<8.2.0",
+    "duckdb>=1.1.0,<1.2.0",
+]
+
+[project.scripts]
+exadata-validator = "validator.commands:entry"
+
+[dependency-groups]
+dev = [
+    "pytest>=8.4.0,<8.5.0",
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["validator"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

exadata_validator-1.2.0/tests/conftest.py

@@ -0,0 +1,26 @@
+import json
+from pathlib import Path
+
+import pytest
+
+TEST_DIR = Path(__file__).resolve().parent
+DATA_DIR = TEST_DIR / "data"
+
+SUCCESS_MODEL_DIR = DATA_DIR / "success" / "data_model_v_1_0"
+FAIL_MODEL_DIR = DATA_DIR / "fail" / "data_model_v_1_0"
+
+SUCCESS_METADATA_PATH = SUCCESS_MODEL_DIR / "CDEsMetadata.json"
+FAIL_METADATA_PATH = FAIL_MODEL_DIR / "CDEsMetadata.json"
+
+LONGITUDINAL_SUCCESS_DIR = DATA_DIR / "success" / "data_model_longitudinal_v_1_0"
+LONGITUDINAL_FAIL_DIR = DATA_DIR / "fail" / "data_model_longitudinal_v_1_0"
+
+
+@pytest.fixture
+def success_metadata():
+    return json.loads(SUCCESS_METADATA_PATH.read_text(encoding="utf-8"))
+
+
+@pytest.fixture
+def fail_metadata():
+    return json.loads(FAIL_METADATA_PATH.read_text(encoding="utf-8"))

exadata_validator-1.2.0/tests/data/fail/data_model_longitudinal_v_1_0/CDEsMetadata.json

@@ -0,0 +1,45 @@
+{
+        "code": "data_model_longitudinal",
+        "label": "The Data Model Longitudinal",
+        "version": "1.0",
+        "longitudinal": true,
+        "variables": [
+            {
+                "isCategorical": true,
+                "code": "dataset",
+                "sql_type": "text",
+                "description": "",
+                "enumerations": [
+                    {"code": "dataset_longitudinal", "label": "Dataset Longitudinal"}
+                ],
+                "label": "Dataset",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": false,
+                "code": "subjectid",
+                "sql_type": "text",
+                "description": "",
+                "label": "subjectid",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": true,
+                "label": "visitid",
+                "code": "visitid",
+                "sql_type": "text",
+                "enumerations": [
+                    {"code": "BL", "label": "Base line"},
+                    {"code": "FL1", "label": "Follow up 1"},
+                    {"code": "FL2", "label": "Follow up 2"}
+                ],
+                "description": "",
+                "methodology": "",
+                "type": "nominal"
+            }
+        ],
+        "groups": [
+        ]
+    }

exadata_validator-1.2.0/tests/data/fail/data_model_longitudinal_v_1_0/dataset.csv

@@ -0,0 +1,5 @@
+subjectid,visitid,dataset
+"subjectid1","BL","dataset_longitudinal"
+"subjectid2","FL1","dataset_longitudinal"
+"subjectid2","FL1","dataset_longitudinal"
+"subjectid2","FL2","dataset_longitudinal"

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/CDEsMetadata.json

@@ -0,0 +1,88 @@
+{
+        "code": "data_model",
+        "label": "The Data Model",
+        "version": "1.0",
+        "variables": [
+            {
+                "isCategorical": false,
+                "code": "var1",
+                "sql_type": "text",
+                "description": "",
+                "label": "Variable 1",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": false,
+                "label": "subjectcode",
+                "code": "subjectcode",
+                "sql_type": "text",
+                "description": "",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": true,
+                "code": "var2",
+                "sql_type": "text",
+                "description": "",
+                "enumerations": [
+                    {"code": "l1", "label": "Level1"},
+                    {"code": "l2", "label": "Level2"}
+                ],
+                "label": "Variable 2",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": true,
+                "code": "dataset",
+                "sql_type": "text",
+                "description": "",
+                "enumerations": [
+                    {"code": "valid_dataset", "label": "Valid dataset"},
+                    {"code": "dataset_is_not_unique", "label": "Dataset with more one value"}
+                ],
+                "label": "Dataset",
+                "methodology": "",
+                "type": "nominal"
+            }
+        ],
+        "groups": [
+            {
+                "name": "group",
+                "label": "The Group",
+                "variables": [
+                    {
+                        "isCategorical": false,
+                        "code": "var3",
+                        "sql_type": "real",
+                        "minValue": 5,
+                        "maxValue": 60,
+                        "description": "",
+                        "label": "Variable 3",
+                        "methodology": "",
+                        "type": "nominal"
+                    }
+                ],
+                "groups": [
+                    {
+                        "name": "inner_group",
+                        "label": "The Inner Group",
+                        "variables": [
+                            {
+                                "isCategorical": false,
+                                "code": "var4",
+                                "sql_type": "int",
+                                "units": "years",
+                                "description": "",
+                                "label": "Variable 4",
+                                "methodology": "",
+                                "type": "nominal"
+                            }
+                        ]
+                    }
+                ]
+            }
+        ]
+    }

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/column_not_present_in_cdes.csv

@@ -0,0 +1,6 @@
+row_id,var1,var3,dataset,non_existing_col
+1,l1,,valid_dataset,1
+2,l2,22,valid_dataset,2
+3,l1,23,valid_dataset,3
+4,l1,24,valid_dataset,4
+5,l2,25,valid_dataset,5

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/dataset_exceeds_max.csv

@@ -0,0 +1,6 @@
+subjectcode,var2,var3,var4,dataset
+1,l1,100,,valid_dataset
+2,l2,12,22,valid_dataset
+3,l1,13,23,valid_dataset
+4,l1,14,24,valid_dataset
+5,l2,15,25,valid_dataset

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/dataset_exceeds_min.csv

@@ -0,0 +1,6 @@
+subjectcode,var2,var3,var4,dataset
+1,l1,0,,valid_dataset
+2,l2,12,22,valid_dataset
+3,l1,13,23,valid_dataset
+4,l1,14,24,valid_dataset
+5,l2,15,25,valid_dataset

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/dataset_is_not_unique.csv

@@ -0,0 +1,6 @@
+subjectcode,var1,var3,dataset
+1,l1,,valid_dataset
+2,l2,22,valid_dataset
+3,l1,23,valid_dataset
+4,l1,24,valid_dataset
+5,l2,25,dataset_is_not_unique

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/duplication_column_row_id.csv

@@ -0,0 +1,6 @@
+subjectcode,var1,var3,dataset
+1,l1,,valid_dataset
+2,l2,22,valid_dataset
+3,l1,23,valid_dataset
+4,l1,24,valid_dataset
+1,l2,25,valid_dataset

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/invalid_enum.csv

@@ -0,0 +1,6 @@
+subjectcode,var2,var3,dataset
+1,l3,,valid_dataset
+2,l2,22,valid_dataset
+3,l1,23,valid_dataset
+4,l1,24,valid_dataset
+5,l2,25,valid_dataset

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/invalid_type1.csv

@@ -0,0 +1,6 @@
+subjectcode,var1,var3,dataset
+1,l10,,valid_dataset
+2,l20,invalid,valid_dataset
+3,l10,23,valid_dataset
+4,l10,24,valid_dataset
+5,l20,25,valid_dataset

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/missing_column_dataset.csv

@@ -0,0 +1,6 @@
+subjectcode,var1,var3
+1,l1,
+2,l2,22
+3,l1,23
+4,l1,24
+5,l2,25

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/missing_column_row_id.csv

@@ -0,0 +1,6 @@
+subjectcode,var1,var3,dataset
+1,l1,,valid_dataset
+2,l2,22,valid_dataset
+2,l1,23,valid_dataset
+2,l1,24,valid_dataset
+3,l2,25,valid_dataset

exadata_validator-1.2.0/tests/data/fail/data_model_v_1_0/missing_column_subjectcode.csv

@@ -0,0 +1,6 @@
+row_id,var1,var3,dataset
+1,l1,,valid_dataset
+2,l2,22,valid_dataset
+3,l1,23,valid_dataset
+4,l1,24,valid_dataset
+5,l2,25,valid_dataset

exadata_validator-1.2.0/tests/data/success/data_model1_v_1_0/CDEsMetadata.json

@@ -0,0 +1,88 @@
+{
+        "code": "data_model1",
+        "label": "The Data Model",
+        "version": "1.0",
+        "variables": [
+            {
+                "isCategorical": false,
+                "code": "var1",
+                "sql_type": "text",
+                "description": "",
+                "label": "Variable 1",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": false,
+                "label": "subjectcode",
+                "code": "subjectcode",
+                "sql_type": "text",
+                "description": "",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": true,
+                "code": "var2",
+                "sql_type": "text",
+                "description": "",
+                "enumerations": [
+                    {"code": "l10", "label": "Level10"},
+                    {"code": "l20", "label": "Level20"}
+                ],
+                "label": "Variable 2",
+                "methodology": "",
+                "type": "nominal"
+            },
+            {
+                "isCategorical": true,
+                "code": "dataset",
+                "sql_type": "text",
+                "description": "",
+                "enumerations": [
+                    {"code": "dataset10", "label": "Dataset 10"},
+                    {"code": "dataset20", "label": "Dataset 20"}
+                ],
+                "label": "Dataset",
+                "methodology": "",
+                "type": "nominal"
+            }
+        ],
+        "groups": [
+            {
+                "name": "group",
+                "label": "The Group",
+                "variables": [
+                    {
+                        "isCategorical": false,
+                        "code": "var3",
+                        "sql_type": "real",
+                        "minValue": 0,
+                        "maxValue": 100,
+                        "description": "",
+                        "label": "Variable 3",
+                        "methodology": "",
+                        "type": "nominal"
+                    }
+                ],
+                "groups": [
+                    {
+                        "name": "inner_group",
+                        "label": "The Inner Group",
+                        "variables": [
+                            {
+                                "isCategorical": false,
+                                "code": "var4",
+                                "sql_type": "int",
+                                "units": "years",
+                                "description": "",
+                                "label": "Variable 4",
+                                "methodology": "",
+                                "type": "nominal"
+                            }
+                        ]
+                    }
+                ]
+            }
+        ]
+    }

exadata_validator-1.2.0/tests/data/success/data_model1_v_1_0/dataset10.csv

@@ -0,0 +1,6 @@
+subjectcode,var1,var3,dataset
+1,l10,,dataset10
+1,l20,22,dataset10
+2,l10,23,dataset10
+4,l10,24,dataset10
+5,l20,25,dataset10

exadata_validator-1.2.0/tests/data/success/data_model1_v_1_0/dataset20.csv

@@ -0,0 +1,6 @@
+subjectcode,var2,var3,var4,dataset
+1,l10,11,,dataset20
+2,l20,12,22,dataset20
+2,l10,13,23,dataset20
+4,l10,14,24,dataset20
+4,l20,15,25,dataset20