lims-data-quality 0.1.0__tar.gz

lims_data_quality-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sri Gorantla
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

lims_data_quality-0.1.0/PKG-INFO

@@ -0,0 +1,132 @@
+Metadata-Version: 2.4
+Name: lims-data-quality
+Version: 0.1.0
+Summary: Validate lab/LIMS data files before they hit your pipeline
+Author: Sri Gorantla
+License: MIT
+Project-URL: Homepage, https://github.com/sriranga13/lims-data-quality
+Keywords: lims,lab,data-quality,validation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas>=2.0
+Requires-Dist: openpyxl>=3.1
+Dynamic: license-file
+
+# lims-data-quality
+
+Validate lab/LIMS data files before they hit your pipeline — schema checks, row-level error reports, audit trails.
+
+`lims-dq` checks CSV/Excel exports against a JSON schema (required columns, dtypes, ranges, regex ID patterns, allowed values) and tells you exactly which row, which column, and why it failed. Every run can append a tamper-evident audit entry (timestamp, file hash, rule set version, pass/fail counts) — a nod to 21 CFR Part 11 style traceability.
+
+## Install
+
+```bash
+pip install lims-data-quality
+```
+
+Or from source:
+
+```bash
+git clone https://github.com/sriranga13/lims-data-quality.git
+cd lims-data-quality
+pip install -e .
+```
+
+Requires Python 3.10+.
+
+## Quickstart
+
+1. Describe your file with a schema (`schema.json`):
+
+```json
+{
+  "name": "lims-export",
+  "version": "1.0.0",
+  "columns": {
+    "sample_id":     { "dtype": "string", "required": true, "pattern": "^SMP-[0-9]{6}$" },
+    "concentration": { "dtype": "float",  "required": true, "min": 0.0, "max": 100.0 },
+    "unit":          { "dtype": "string", "required": true, "allowed": ["mg/L", "ug/mL", "ng/uL"] },
+    "analyzed_at":   { "dtype": "date",   "required": false },
+    "replicates":    { "dtype": "int",    "required": false, "min": 1, "max": 12 }
+  }
+}
+```
+
+2. Validate:
+
+```bash
+lims-dq validate samples.csv --schema schema.json
+```
+
+Sample output:
+
+```
+lims-dq report: samples.csv
+ruleset: lims-export v1.0.0
+rows checked: 5 | passed: 3 | failed: 2 | errors: 7
+
+  row  column             rule               message
+------------------------------------------------------------------------
+    4  analyzed_at        dtype              'not-a-date' is not a recognizable date
+    4  concentration      constraint         '150.0' is above maximum 100.0
+    4  replicates         constraint         '0' is below minimum 1
+    4  sample_id          constraint         'BAD-ID' does not match pattern '^SMP-[0-9]{6}$'
+    4  unit               constraint         'kg' is not one of ['mg/L', 'ug/mL', 'ng/uL']
+    5  concentration      required           required value is missing
+    5  replicates         constraint         '13' is above maximum 12
+```
+
+3. Machine-readable report and audit trail:
+
+```bash
+lims-dq validate samples.csv --schema schema.json \
+    --report report.json \
+    --audit-log audit.jsonl
+```
+
+- `--report` writes the full report (summary counts + every error) as JSON.
+- `--audit-log` appends one JSON-lines entry per run: UTC timestamp, SHA-256 of the input file, rule set name/version, rows checked/passed/failed, error count. Re-running against a modified file produces a different hash, so entries are tamper-evident.
+
+Exit codes: `0` = all rows pass, `1` = validation failures, `2` = usage/file errors.
+
+## Schema reference
+
+| Key        | Applies to        | Meaning                                          |
+|------------|-------------------|--------------------------------------------------|
+| `dtype`    | all               | `string`, `int`, `float`, or `date`              |
+| `required` | all               | missing values fail (missing column fails too)   |
+| `pattern`  | all               | regex the full value must match                  |
+| `min`/`max`| `int`, `float`    | inclusive numeric bounds                          |
+| `allowed`  | all               | value must be one of the listed strings          |
+
+Columns present in the file but absent from the schema are flagged as `unexpected_column` errors unless you pass `--no-strict`.
+
+## Use as a library
+
+```python
+from lims_dq import load_schema, validate_dataframe, ErrorReport
+
+ruleset = load_schema("schema.json")
+import pandas as pd
+df = pd.read_csv("samples.csv", dtype=str)
+errors = validate_dataframe(df, ruleset)
+report = ErrorReport("samples.csv", ruleset.name, ruleset.version, len(df), errors)
+print(report.to_text())   # or report.to_json()
+```
+
+## Development
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e . && pip install pytest
+pytest
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

lims_data_quality-0.1.0/README.md

@@ -0,0 +1,113 @@
+# lims-data-quality
+
+Validate lab/LIMS data files before they hit your pipeline — schema checks, row-level error reports, audit trails.
+
+`lims-dq` checks CSV/Excel exports against a JSON schema (required columns, dtypes, ranges, regex ID patterns, allowed values) and tells you exactly which row, which column, and why it failed. Every run can append a tamper-evident audit entry (timestamp, file hash, rule set version, pass/fail counts) — a nod to 21 CFR Part 11 style traceability.
+
+## Install
+
+```bash
+pip install lims-data-quality
+```
+
+Or from source:
+
+```bash
+git clone https://github.com/sriranga13/lims-data-quality.git
+cd lims-data-quality
+pip install -e .
+```
+
+Requires Python 3.10+.
+
+## Quickstart
+
+1. Describe your file with a schema (`schema.json`):
+
+```json
+{
+  "name": "lims-export",
+  "version": "1.0.0",
+  "columns": {
+    "sample_id":     { "dtype": "string", "required": true, "pattern": "^SMP-[0-9]{6}$" },
+    "concentration": { "dtype": "float",  "required": true, "min": 0.0, "max": 100.0 },
+    "unit":          { "dtype": "string", "required": true, "allowed": ["mg/L", "ug/mL", "ng/uL"] },
+    "analyzed_at":   { "dtype": "date",   "required": false },
+    "replicates":    { "dtype": "int",    "required": false, "min": 1, "max": 12 }
+  }
+}
+```
+
+2. Validate:
+
+```bash
+lims-dq validate samples.csv --schema schema.json
+```
+
+Sample output:
+
+```
+lims-dq report: samples.csv
+ruleset: lims-export v1.0.0
+rows checked: 5 | passed: 3 | failed: 2 | errors: 7
+
+  row  column             rule               message
+------------------------------------------------------------------------
+    4  analyzed_at        dtype              'not-a-date' is not a recognizable date
+    4  concentration      constraint         '150.0' is above maximum 100.0
+    4  replicates         constraint         '0' is below minimum 1
+    4  sample_id          constraint         'BAD-ID' does not match pattern '^SMP-[0-9]{6}$'
+    4  unit               constraint         'kg' is not one of ['mg/L', 'ug/mL', 'ng/uL']
+    5  concentration      required           required value is missing
+    5  replicates         constraint         '13' is above maximum 12
+```
+
+3. Machine-readable report and audit trail:
+
+```bash
+lims-dq validate samples.csv --schema schema.json \
+    --report report.json \
+    --audit-log audit.jsonl
+```
+
+- `--report` writes the full report (summary counts + every error) as JSON.
+- `--audit-log` appends one JSON-lines entry per run: UTC timestamp, SHA-256 of the input file, rule set name/version, rows checked/passed/failed, error count. Re-running against a modified file produces a different hash, so entries are tamper-evident.
+
+Exit codes: `0` = all rows pass, `1` = validation failures, `2` = usage/file errors.
+
+## Schema reference
+
+| Key        | Applies to        | Meaning                                          |
+|------------|-------------------|--------------------------------------------------|
+| `dtype`    | all               | `string`, `int`, `float`, or `date`              |
+| `required` | all               | missing values fail (missing column fails too)   |
+| `pattern`  | all               | regex the full value must match                  |
+| `min`/`max`| `int`, `float`    | inclusive numeric bounds                          |
+| `allowed`  | all               | value must be one of the listed strings          |
+
+Columns present in the file but absent from the schema are flagged as `unexpected_column` errors unless you pass `--no-strict`.
+
+## Use as a library
+
+```python
+from lims_dq import load_schema, validate_dataframe, ErrorReport
+
+ruleset = load_schema("schema.json")
+import pandas as pd
+df = pd.read_csv("samples.csv", dtype=str)
+errors = validate_dataframe(df, ruleset)
+report = ErrorReport("samples.csv", ruleset.name, ruleset.version, len(df), errors)
+print(report.to_text())   # or report.to_json()
+```
+
+## Development
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e . && pip install pytest
+pytest
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

lims_data_quality-0.1.0/pyproject.toml

@@ -0,0 +1,35 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "lims-data-quality"
+version = "0.1.0"
+description = "Validate lab/LIMS data files before they hit your pipeline"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "Sri Gorantla" }]
+keywords = ["lims", "lab", "data-quality", "validation"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+]
+dependencies = [
+    "pandas>=2.0",
+    "openpyxl>=3.1",
+]
+
+[project.urls]
+Homepage = "https://github.com/sriranga13/lims-data-quality"
+
+[project.scripts]
+lims-dq = "lims_dq.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

lims_data_quality-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

lims_data_quality-0.1.0/src/lims_data_quality.egg-info/PKG-INFO

@@ -0,0 +1,132 @@
+Metadata-Version: 2.4
+Name: lims-data-quality
+Version: 0.1.0
+Summary: Validate lab/LIMS data files before they hit your pipeline
+Author: Sri Gorantla
+License: MIT
+Project-URL: Homepage, https://github.com/sriranga13/lims-data-quality
+Keywords: lims,lab,data-quality,validation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas>=2.0
+Requires-Dist: openpyxl>=3.1
+Dynamic: license-file
+
+# lims-data-quality
+
+Validate lab/LIMS data files before they hit your pipeline — schema checks, row-level error reports, audit trails.
+
+`lims-dq` checks CSV/Excel exports against a JSON schema (required columns, dtypes, ranges, regex ID patterns, allowed values) and tells you exactly which row, which column, and why it failed. Every run can append a tamper-evident audit entry (timestamp, file hash, rule set version, pass/fail counts) — a nod to 21 CFR Part 11 style traceability.
+
+## Install
+
+```bash
+pip install lims-data-quality
+```
+
+Or from source:
+
+```bash
+git clone https://github.com/sriranga13/lims-data-quality.git
+cd lims-data-quality
+pip install -e .
+```
+
+Requires Python 3.10+.
+
+## Quickstart
+
+1. Describe your file with a schema (`schema.json`):
+
+```json
+{
+  "name": "lims-export",
+  "version": "1.0.0",
+  "columns": {
+    "sample_id":     { "dtype": "string", "required": true, "pattern": "^SMP-[0-9]{6}$" },
+    "concentration": { "dtype": "float",  "required": true, "min": 0.0, "max": 100.0 },
+    "unit":          { "dtype": "string", "required": true, "allowed": ["mg/L", "ug/mL", "ng/uL"] },
+    "analyzed_at":   { "dtype": "date",   "required": false },
+    "replicates":    { "dtype": "int",    "required": false, "min": 1, "max": 12 }
+  }
+}
+```
+
+2. Validate:
+
+```bash
+lims-dq validate samples.csv --schema schema.json
+```
+
+Sample output:
+
+```
+lims-dq report: samples.csv
+ruleset: lims-export v1.0.0
+rows checked: 5 | passed: 3 | failed: 2 | errors: 7
+
+  row  column             rule               message
+------------------------------------------------------------------------
+    4  analyzed_at        dtype              'not-a-date' is not a recognizable date
+    4  concentration      constraint         '150.0' is above maximum 100.0
+    4  replicates         constraint         '0' is below minimum 1
+    4  sample_id          constraint         'BAD-ID' does not match pattern '^SMP-[0-9]{6}$'
+    4  unit               constraint         'kg' is not one of ['mg/L', 'ug/mL', 'ng/uL']
+    5  concentration      required           required value is missing
+    5  replicates         constraint         '13' is above maximum 12
+```
+
+3. Machine-readable report and audit trail:
+
+```bash
+lims-dq validate samples.csv --schema schema.json \
+    --report report.json \
+    --audit-log audit.jsonl
+```
+
+- `--report` writes the full report (summary counts + every error) as JSON.
+- `--audit-log` appends one JSON-lines entry per run: UTC timestamp, SHA-256 of the input file, rule set name/version, rows checked/passed/failed, error count. Re-running against a modified file produces a different hash, so entries are tamper-evident.
+
+Exit codes: `0` = all rows pass, `1` = validation failures, `2` = usage/file errors.
+
+## Schema reference
+
+| Key        | Applies to        | Meaning                                          |
+|------------|-------------------|--------------------------------------------------|
+| `dtype`    | all               | `string`, `int`, `float`, or `date`              |
+| `required` | all               | missing values fail (missing column fails too)   |
+| `pattern`  | all               | regex the full value must match                  |
+| `min`/`max`| `int`, `float`    | inclusive numeric bounds                          |
+| `allowed`  | all               | value must be one of the listed strings          |
+
+Columns present in the file but absent from the schema are flagged as `unexpected_column` errors unless you pass `--no-strict`.
+
+## Use as a library
+
+```python
+from lims_dq import load_schema, validate_dataframe, ErrorReport
+
+ruleset = load_schema("schema.json")
+import pandas as pd
+df = pd.read_csv("samples.csv", dtype=str)
+errors = validate_dataframe(df, ruleset)
+report = ErrorReport("samples.csv", ruleset.name, ruleset.version, len(df), errors)
+print(report.to_text())   # or report.to_json()
+```
+
+## Development
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e . && pip install pytest
+pytest
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

lims_data_quality-0.1.0/src/lims_data_quality.egg-info/SOURCES.txt

@@ -0,0 +1,20 @@
+LICENSE
+README.md
+pyproject.toml
+src/lims_data_quality.egg-info/PKG-INFO
+src/lims_data_quality.egg-info/SOURCES.txt
+src/lims_data_quality.egg-info/dependency_links.txt
+src/lims_data_quality.egg-info/entry_points.txt
+src/lims_data_quality.egg-info/requires.txt
+src/lims_data_quality.egg-info/top_level.txt
+src/lims_dq/__init__.py
+src/lims_dq/audit.py
+src/lims_dq/cli.py
+src/lims_dq/report.py
+src/lims_dq/schema.py
+src/lims_dq/validators.py
+tests/test_audit.py
+tests/test_cli.py
+tests/test_report.py
+tests/test_schema.py
+tests/test_validators.py

lims_data_quality-0.1.0/src/lims_data_quality.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

lims_data_quality-0.1.0/src/lims_data_quality.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+lims-dq = lims_dq.cli:main

lims_data_quality-0.1.0/src/lims_data_quality.egg-info/requires.txt

@@ -0,0 +1,2 @@
+pandas>=2.0
+openpyxl>=3.1

lims_data_quality-0.1.0/src/lims_data_quality.egg-info/top_level.txt

@@ -0,0 +1 @@
+lims_dq

lims_data_quality-0.1.0/src/lims_dq/__init__.py

@@ -0,0 +1,21 @@
+"""lims-dq: validate lab/LIMS data files before they enter your pipeline."""
+
+from .audit import AuditEntry, read_audit_log, write_audit_log
+from .report import ErrorReport, ValidationError
+from .schema import Rule, RuleSet, SchemaError, load_schema
+from .validators import validate_dataframe
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "AuditEntry",
+    "ErrorReport",
+    "Rule",
+    "RuleSet",
+    "SchemaError",
+    "ValidationError",
+    "load_schema",
+    "read_audit_log",
+    "validate_dataframe",
+    "write_audit_log",
+]

lims_data_quality-0.1.0/src/lims_dq/audit.py

@@ -0,0 +1,88 @@
+"""Minimal audit trail: who validated what, when, against which rules.
+
+Each validation run appends one JSON-lines entry recording the UTC
+timestamp, the SHA-256 hash of the input file, the rule set name and
+version, and the pass/fail counts. The file hash makes the entry
+tamper-evident: re-running against a modified file produces a different
+hash. A nod to 21 CFR Part 11 style traceability — not a compliance
+certification.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+from dataclasses import asdict, dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+
+@dataclass
+class AuditEntry:
+    timestamp_utc: str
+    filename: str
+    file_sha256: str
+    ruleset_name: str
+    ruleset_version: str
+    rows_checked: int
+    rows_passed: int
+    rows_failed: int
+    error_count: int
+    passed: bool
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+
+def sha256_of_file(path: str | Path) -> str:
+    digest = hashlib.sha256()
+    with open(path, "rb") as fh:
+        for chunk in iter(lambda: fh.read(65536), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def build_entry(
+    *,
+    filename: str | Path,
+    ruleset_name: str,
+    ruleset_version: str,
+    rows_checked: int,
+    rows_passed: int,
+    rows_failed: int,
+    error_count: int,
+) -> AuditEntry:
+    path = Path(filename)
+    return AuditEntry(
+        timestamp_utc=datetime.now(timezone.utc).isoformat(timespec="seconds"),
+        filename=path.name,
+        file_sha256=sha256_of_file(path),
+        ruleset_name=ruleset_name,
+        ruleset_version=ruleset_version,
+        rows_checked=rows_checked,
+        rows_passed=rows_passed,
+        rows_failed=rows_failed,
+        error_count=error_count,
+        passed=error_count == 0,
+    )
+
+
+def write_audit_log(entry: AuditEntry, log_path: str | Path) -> None:
+    """Append one entry (as a single JSON line) to the audit log."""
+    log_path = Path(log_path)
+    if log_path.parent != Path("."):
+        log_path.parent.mkdir(parents=True, exist_ok=True)
+    with open(log_path, "a", encoding="utf-8") as fh:
+        fh.write(json.dumps(entry.to_dict()) + "\n")
+
+
+def read_audit_log(log_path: str | Path) -> list[dict[str, Any]]:
+    """Read all entries from a JSON-lines audit log."""
+    entries = []
+    with open(log_path, encoding="utf-8") as fh:
+        for line in fh:
+            line = line.strip()
+            if line:
+                entries.append(json.loads(line))
+    return entries