tablassert 7.5.0__tar.gz → 7.5.2__tar.gz

tablassert-7.5.2/AGENTS.md

@@ -0,0 +1,57 @@
+# AGENTS.md — Tablassert
+
+## Fast Start
+
+- Python package, not a monorepo. Main code lives in `src/tablassert/`; tests live in `tests/`.
+- Install with `uv sync`. Three optional extras:
+  - `--extra qc` / `--extra qc-cuda` — installs `onnxruntime` / `onnxruntime-gpu` for QC (strict runtime behavior below).
+  - `--extra rt` — runtime-compatible Polars build for CPUs missing required SIMD instructions.
+- CLI entrypoint is `tablassert.cli:APP`. Real user commands:
+  - `uv run tablassert build <graph.yaml>` — 6 pipeline stages.
+  - `uv run tablassert validate <table.yaml>` — 3 stages, syntax-only.
+
+## Source of Truth
+
+- When prose docs and code disagree, `src/tablassert/models.py` and `src/tablassert/cli.py` are authoritative. Docs drift: some pages claim shard files are `{0..11}.duckdb` or that only the `rt` extra exists; the code uses `SHARDS = 10` (so `0..9`) and exposes `qc`/`qc-cuda`/`rt` extras.
+
+## Verify Changes
+
+- Match the repo hooks before finishing: `uv run ruff check --fix .`, `uv run ruff format .`, `uv run pyright`, `uv run pytest`.
+- Full hook run: `uv run pre-commit run --all-files` (ruff, ruff-format, pyright, pytest).
+- Focused test runs:
+  - Single test: `uv run pytest tests/test_lib.py::test_name`
+  - By keyword: `uv run pytest -k "pattern"`
+  - With print output: `uv run pytest -s tests/test_lib.py`
+- Docs build: `uv run --group dev mkdocs build`.
+
+## High-Value Structure
+
+- `src/tablassert/cli.py` is the wiring layer: `build()` → `build_pipeline()`, `validate()` → `validate_pipeline()`.
+- `src/tablassert/ingests.py` loads YAML and expands table configs into section dicts.
+- `src/tablassert/lib.py` is the core pipeline:
+  - `Tcode.collect()` builds the per-section operation list.
+  - `compile_subgraph()` executes that list into parquet.
+  - `compile_graph()` aggregates subgraph parquets into KGX NDJSON.
+  - `resolve_many()` is the direct library API for batch entity resolution.
+- Entity resolution uses DuckDB shard files at `<datassert>/data/{0..9}.duckdb`, opened read-only. `datassert` is a required `Path` field on the `Graph` model (not a fixed location); `src/tablassert/fullmap.py` hardcodes `SHARDS = 10`.
+
+## Repo-Specific Gotchas
+
+- Heavy dependencies are lazy-loaded per module with `TYPE_CHECKING` + `lazy_loader`. Follow the existing pattern instead of importing heavy packages eagerly. Lazy-loaded: polars, duckdb, orjson, xxhash, polars_hash, yaml, httpx, pyexcel, onnxruntime, sentence_transformers.
+- `tests/conftest.py` autouse-mocks `httpx.head`, so model-URL validation tests never hit the network unless a test opts in.
+- Network-dependent tests are marked `@pytest.mark.network`; GPU QC tests are marked with both `network` and `gpu` in `tests/test_qc.py`.
+- QC runtime selection is strict in `src/tablassert/qc.py`: if `onnxruntime-gpu` is installed but `CUDAExecutionProvider` is unavailable, the code raises (error 06) instead of falling back to CPU. Install `tablassert[qc]` for CPU-only.
+- Downloader behavior in `src/tablassert/downloader.py` is two-path: `direct()` via `httpx` for known file URLs, `browser()` via Playwright fallback for browser-only sources. Keep payload-validation and cleanup tests intact when changing it.
+
+## Conventions That Matter Here
+
+- Start every module with `from __future__ import annotations`.
+- Annotate every variable, including locals.
+- Use `Optional[T]` / `Union[...]`, not `T | None`. Use `Path`, not `str`, for filesystem paths. Use `# pyright: ignore` to silence lazy-load false positives.
+- No docstrings on functions. Use comment markers on the line above: `# ?` description, `# !` warning, `# *` pipeline stage, `# TODO:` todo.
+- Pydantic models inherit `TablaBase` (`extra="forbid"`, `validate_assignment=True`). Required fields use `Field(...)`; optional use `Optional[T] = Field(None)`. Enums extend `str, Enum` and live in `enums.py`.
+- Per-module logger: `from tablassert.log import cat; logger = cat("MODULE")` (e.g. `cat("FULLMAP")`). The CLI uses the root `from tablassert.log import logger`. Raise `RuntimeError` for failures; `logger.warning()` for non-fatal issues.
+
+## Side Effects
+
+- The package writes working artifacts to hidden dirs in the repo root, each defined in code: `.storassert/` (`utils.STORE`), `.logassert/` (`log.LOGASSERT`), `.cachassert/` (`models.CACHE`), `.onnxassert/` (`qc.MODEL`).

{tablassert-7.5.0 → tablassert-7.5.2}/CHANGELOG.md

@@ -2,6 +2,25 @@
 
 All notable changes to this project are documented in this file.
 
+## 7.5.2 - 2026-07-01
+
+### Changed
+- `sig()` in `lib.py` now emits a third significance label, `"INCONCLUSIVE"`, for p-values that fall between the existing significance `cutoff` (default `0.05`, inclusive) and a new `threshold` parameter (default `0.10`, exclusive). A p-value `p` is now mapped as: null → `"UNSURE"`; `p <= cutoff` → `"YES"`; `cutoff < p < threshold` → `"INCONCLUSIVE"`; `p >= threshold` → `"NO"`. The upper bound is exclusive so `0.10` (and the `0.1` `NO` cases in the existing tests) continue to map to `"NO"`. `threshold` is added to `sig()`'s signature alongside `cutoff`; the function remains wired into `Tcode.collect()` at its default arguments, so builds are unaffected unless a caller overrides the new bound.
+
+### Added
+- One regression test in `test_lib.py` (`test_sig_marks_inconclusive_band`) asserting all four bands in a single frame: a value at/below cutoff (`YES`), a value in the inconclusive range (`INCONCLUSIVE`), and a value at the exclusive threshold (`NO`).
+
+## 7.5.1 - 2026-07-01
+
+### Changed
+- Numeric annotation columns are now coerced and emitted as controlled-notation strings in NDJSON output instead of raw values. Two new pipeline steps wired into `Tcode.collect()` (`lib.py`): `clean_numeric()` lazily casts matching columns to `Float64` with `strict=False` (non-numeric entries drop to null), and `format_numeric()` renders them as strings — p-value columns (any name containing `"p value"`, case-insensitive) in scientific notation (`{:.4e}`), and `relationship strength` / `sample size` in decimal general format (`{:.4g}`, ≥4 significant figures). Non-matching columns are left untouched, and nulls are subsequently dropped by `strip_nulls()`. `math_op()` now also casts with `strict=False` so it tolerates residual junk in numeric annotation columns. `format_numeric()` formats via numpy-backed batch conversion rather than `map_elements` for throughput.
+- Removed dead `pl.Config(set_fmt_float=...)` and `pl.Config(float_precision=...)` context managers from `compile_graph()` (`lib.py`); they were no-ops for NDJSON serialization (`write_ndjson` emits raw f64 via serde shortest-repr and ignores float display options), and the `fmt`/`precision` parameters of `compile_graph()` were removed alongside them.
+- Fixed an off-by-one in the build/validate progress bar so each section loop now shows the configuration currently being processed instead of the last-completed one. `PipelineProgress.section_loop()` (`progress.py`) previously returned a single `advance(info)` callback that set the description and ticked the completed counter together, called after each item's work — so while section *K* ran the bar still displayed section *K−1*. It now returns a `(start, advance)` pair: `start(info)` updates the description to the in-flight item without incrementing, and `advance()` ticks the counter afterwards (so the counter never claims an in-flight item is complete). All five call sites in `cli.py` (TCode build, Collect, Subgraph, Graph, Validate) were updated to `start(...)` before the work and `advance()` after; the long-running Collect and Subgraph stages continue to show the full `format_section_oneline()` summary (including the `CONFIG` name) of the in-flight section.
+
+### Added
+- Fifteen regression tests in `test_lib.py` covering `numeric_columns()` detection (p-value substring, exact-name match, case-insensitivity), `clean_numeric()` (parse/coerce numeric and scientific notation, null out non-numeric junk, leave non-matching columns untouched, noop, idempotent on Float64), `format_numeric()` (scientific notation for p-value, decimal general format for relationship strength/sample size, null preservation, floating-point-noise cleaning, noop), null-stripped NDJSON rows, `compile_graph()` NDJSON emission, and `sig()` operating over a cleaned Float64 p-value column.
+- Three regression tests in `tests/test_progress.py` pinning the new two-callback contract: `start` shows the in-flight item with the counter still at zero, `advance` ticks the counter without altering the description, and a `start`/`advance` cycle keeps the description synced to the current item rather than the previous one.
+
 ## 7.5.0 - 2026-07-01
 
 ### Changed

{tablassert-7.5.0 → tablassert-7.5.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tablassert
-Version: 7.5.0
+Version: 7.5.2
 Summary: Extract knowledge assertions from tabular data into NCATS Translator-compliant KGX NDJSON — declaratively, with entity resolution and quality control built in.
 Project-URL: Homepage, https://github.com/SkyeAv/Tablassert
 Project-URL: Source, https://github.com/SkyeAv/Tablassert

tablassert-7.5.2/docs/changelog.md

@@ -0,0 +1,13 @@
+# Changelog
+
+The canonical release history lives in the repository root at [`CHANGELOG.md`](https://github.com/SkyeAv/Tablassert/blob/main/CHANGELOG.md).
+
+## Current Release Notes
+
+## 7.5.2 - 2026-07-01
+
+### Changed
+- `sig()` in `lib.py` now emits a third significance label, `"INCONCLUSIVE"`, for p-values that fall between the existing significance `cutoff` (default `0.05`, inclusive) and a new `threshold` parameter (default `0.10`, exclusive). A p-value `p` is now mapped as: null → `"UNSURE"`; `p <= cutoff` → `"YES"`; `cutoff < p < threshold` → `"INCONCLUSIVE"`; `p >= threshold` → `"NO"`. The upper bound is exclusive so `0.10` (and the `0.1` `NO` cases in the existing tests) continue to map to `"NO"`. `threshold` is added to `sig()`'s signature alongside `cutoff`; the function remains wired into `Tcode.collect()` at its default arguments, so builds are unaffected unless a caller overrides the new bound.
+
+### Added
+- One regression test in `test_lib.py` (`test_sig_marks_inconclusive_band`) asserting all four bands in a single frame: a value at/below cutoff (`YES`), a value in the inconclusive range (`INCONCLUSIVE`), and a value at the exclusive threshold (`NO`).

{tablassert-7.5.0 → tablassert-7.5.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "tablassert"
-version = "7.5.0"
+version = "7.5.2"
 description = "Extract knowledge assertions from tabular data into NCATS Translator-compliant KGX NDJSON — declaratively, with entity resolution and quality control built in."
 authors = [
     { name = "Skye Lane Goetz", email = "sgoetz@isbscience.org" }
@@ -92,7 +92,7 @@ qc-cuda = [
 dev = [
     "mkdocs>=1.6.1",
     "pre-commit>=4.5.1",
-    "pyright>=1.1.408",
+    "pyright>=1.1.411",
     "pytest>=9.0.2",
     "pytest-cov>=7.1.0",
     "ruff>=0.15.6",

{tablassert-7.5.0 → tablassert-7.5.2}/src/tablassert/cli.py

@@ -59,20 +59,20 @@ def build_pipeline(graph_configuration_file: Path, progress: "PipelineProgress")
 
     # * Build TCode (3/6)
     progress.stage(f"Building TCode | Sections: {n}")
-    advance = progress.section_loop(n, "TCode")
+    start, advance = progress.section_loop(n, "TCode")
     tcode: list[Tcode] = []
     for idx, s in enumerate(sections, start=1):
+        h: str = mkhash(s)
+        start(f"#{idx} | CONFIG: {Path(s['config']).name} | HASH: {h}")
         try:
             tcode.append(
-                Tcode.model_validate(
-                    {**s, "number": idx, "store": (STORE / f"{mkhash(s)}.parquet"), "log": g.log, "qc": g.qc}
-                )
+                Tcode.model_validate({**s, "number": idx, "store": (STORE / f"{h}.parquet"), "log": g.log, "qc": g.qc})
             )
         except pydantic.ValidationError as e:
             raise RuntimeError(
-                f"02 | FAILED VALIDATION | CONFIG: {graph_configuration_file} | IDX: {idx} | HASH: {mkhash(s)} | PYDANTIC: {flatten_pydantic_error(e)}"
+                f"02 | FAILED VALIDATION | CONFIG: {graph_configuration_file} | IDX: {idx} | HASH: {h} | PYDANTIC: {flatten_pydantic_error(e)}"
             ) from e
-        advance(format_section_oneline(tcode[-1]))
+        advance()
 
     with ExitStack() as stack:
         conns: list[object] = [
@@ -82,25 +82,28 @@ def build_pipeline(graph_configuration_file: Path, progress: "PipelineProgress")
 
         # * Collect Instructions (4/6)
         progress.stage(f"Collecting Instructions | Sections: {n}")
-        advance = progress.section_loop(n, "Collect")
+        start, advance = progress.section_loop(n, "Collect")
         instructions: list[Any] = []
         for x in tcode:
+            start(format_section_oneline(x))
             instructions.append(x.collect(conns, g.pubmed_db, g.pmc_db))  # pyright: ignore
-            advance(format_section_oneline(x))
+            advance()
 
         # * Build Subgraphs (5/6)
         progress.stage(f"Building Subgraphs | Sections: {n}")
-        advance = progress.section_loop(n, "Subgraph")
+        start, advance = progress.section_loop(n, "Subgraph")
         subgraphs: list[Path] = []
         for x, op in zip(tcode, instructions):
+            start(format_section_oneline(x))
             subgraphs.append(op if isinstance(op, Path) else compile_subgraph(op))
-            advance(format_section_oneline(x))
+            advance()
 
     # * Compile Graph (6/6)
     progress.stage(f"Compiling Graph | Sections: {n}")
-    advance = progress.section_loop(1, "Graph")
+    start, advance = progress.section_loop(1, "Graph")
+    start(f"NAME: {g.name} | VERSION: {g.version}")
     compile_graph(subgraphs, g.name, g.version)
-    advance(f"NAME: {g.name} | VERSION: {g.version}")
+    advance()
 
     logger.info(f"BUILD DONE | SECTIONS: {n} | NAME: {g.name} | VERSION: {g.version}")
 
@@ -123,16 +126,17 @@ def validate_pipeline(table_configuration_file: Path, progress: "PipelineProgres
 
     # * Validate Section Syntax (3/3)
     progress.stage(f"Validating Section Syntax | Sections: {n}")
-    advance = progress.section_loop(n, "Validate")
+    start, advance = progress.section_loop(n, "Validate")
     for idx, s in enumerate(sections, start=1):
         h: str = mkhash(s)
+        start(f"#{idx} | HASH: {h}")
         try:
             Tcode.model_validate({**s, "number": idx, "store": (STORE / f"{h}.parquet")})
         except pydantic.ValidationError as e:
             raise RuntimeError(
                 f"02 | FAILED VALIDATION | CONFIG: {table_configuration_file} | IDX: {idx} | HASH: {h} | PYDANTIC: {flatten_pydantic_error(e)}"
             ) from e
-        advance(f"#{idx} | HASH: {h}")
+        advance()
 
     logger.info(f"VALIDATE DONE | SECTIONS: {n} | CONFIG: {table_configuration_file.name}")
 

{tablassert-7.5.0 → tablassert-7.5.2}/src/tablassert/lib.py

@@ -5,7 +5,7 @@ import operator
 from collections.abc import Iterable
 from contextlib import ExitStack
 from functools import reduce
-from operator import add, eq, le
+from operator import add, eq, le, lt
 from os.path import basename
 from pathlib import Path
 from typing import TYPE_CHECKING, Any, Callable, Literal, Optional, Self, Union
@@ -24,11 +24,13 @@ from tablassert.utils import namespace_uuid
 
 if TYPE_CHECKING:
     import duckdb
+    import numpy as np
     import orjson
     import polars as pl
     import xxhash
 else:
     duckdb = Lazy.load("duckdb")
+    np = Lazy.load("numpy")
     orjson = Lazy.load("orjson")
     pl = Lazy.load("polars")
     xxhash = Lazy.load("xxhash")
@@ -56,8 +58,9 @@ def math_op(
 ) -> pl.LazyFrame:
     # ? Transform Values In A Column With The Math Module
     # ! Collection Point: Required For map_elements
+    # * strict=False tolerates residual non numeric junk in numeric annotation columns
     df: pl.DataFrame = lf.collect()
-    expr: pl.Expr = pl.col(col).cast(pl.Float64)
+    expr: pl.Expr = pl.col(col).cast(pl.Float64, strict=False)
     attr: Callable[[Any], Any] = getattr(math, func)
     df = df.with_columns(
         expr.map_elements(
@@ -67,6 +70,38 @@ def math_op(
     return df.lazy()
 
 
+def numeric_columns(names: list[str]) -> list[str]:
+    # ? Returns Column Names That Should Be Coerced And Formatted As Numbers
+    # * P Value Columns By Substring Plus Exact Relationship Strength And Sample Size
+    exact: set[str] = {"relationship strength", "sample size"}
+    return [c for c in names if ("p value" in c.lower()) or (c in exact)]
+
+
+def clean_numeric(lf: pl.LazyFrame) -> pl.LazyFrame:
+    # ? Coerces Numeric Annotation Columns To Float64 Dropping Non Numeric Values To Null
+    # * Only Touches P Value Relationship Strength And Sample Size Columns
+    cols: list[str] = numeric_columns(lf.collect_schema().names())
+    if not cols:
+        return lf
+    return lf.with_columns([pl.col(c).cast(pl.Float64, strict=False) for c in cols])
+
+
+def format_numeric(lf: pl.LazyFrame) -> pl.LazyFrame:
+    # ? Formats Numeric Annotation Columns As Strings With Controlled Notation
+    # ! Collection Point: numpy Batch Formatting Required For Notation Control
+    # * P Value Columns Use Scientific Notation Others Use Decimal General Format
+    df: pl.DataFrame = lf.collect()
+    cols: list[str] = numeric_columns(df.columns)
+    for c in cols:
+        df = df.with_columns(pl.col(c).cast(pl.Float64, strict=False).alias(c))
+        mask: object = df[c].is_null().to_numpy()
+        arr: object = df[c].to_numpy()
+        fmt: str = "{:.4e}" if "p value" in c.lower() else "{:.4g}"
+        formatted: list[Optional[str]] = [None if m else fmt.format(float(v)) for v, m in zip(arr, mask)]  # pyright: ignore
+        df = df.with_columns(pl.Series(c, formatted))
+    return df.lazy()
+
+
 def prefix(lf: pl.LazyFrame, col: str, prefix: str) -> pl.LazyFrame:
     expr: pl.Expr = add(pl.lit(prefix), pl.col(col).cast(pl.String))
     return lf.with_columns(expr.alias(col))
@@ -97,6 +132,7 @@ def explode(lf: pl.LazyFrame, col: str, delimiter: str) -> pl.LazyFrame:
 def sig(
     lf: pl.LazyFrame,
     cutoff: float = 0.05,  # pyright: ignore
+    threshold: float = 0.10,
     col: str = "p value",
     out: str = "significant",
 ) -> pl.LazyFrame:
@@ -110,7 +146,13 @@ def sig(
         expr: pl.Expr = pl.col(chosen).cast(pl.Float64, strict=False)
         cond: pl.Expr = le(expr, cutoff)
         cutoff: pl.Expr = (
-            pl.when(expr.is_null()).then(pl.lit("UNSURE")).when(cond).then(pl.lit("YES")).otherwise(pl.lit("NO"))
+            pl.when(expr.is_null())
+            .then(pl.lit("UNSURE"))
+            .when(cond)
+            .then(pl.lit("YES"))
+            .when(lt(expr, threshold))
+            .then(pl.lit("INCONCLUSIVE"))
+            .otherwise(pl.lit("NO"))
         )
         return lf.with_columns(cutoff.alias(out))
     else:
@@ -347,6 +389,7 @@ class Tcode(Section):
                 if self.source.reindex
                 else None,
                 [op for x in self.annotations for op in self.encoding(x, x.annotation)] if self.annotations else None,
+                (clean_numeric, ()),
                 self.node(self.statement.subject, "subject", conns),
                 self.node(self.statement.object, "object", conns),
                 (value, ("predicate", add("biolink:", self.statement.predicate))),
@@ -367,6 +410,7 @@ class Tcode(Section):
                 (with_captions, (pmc_db, self.provenance.publication, str(self.source.url))) if pmc_db else None,
                 (sig, ()),
                 (trim, ()),
+                (format_numeric, ()),
                 (to_store, (self.store, self.config.name)),
             ]
             return self.clean(tcode)
@@ -462,7 +506,7 @@ def dedup_stream(p_in: Path, is_edges: bool) -> None:
     p_in.unlink()
 
 
-def compile_graph(subgraphs: list[Path], name: str, version: str, fmt: str = "mixed", precision: int = 4) -> None:
+def compile_graph(subgraphs: list[Path], name: str, version: str) -> None:
     # ? Aggregates Parquets For NDJSON KGX Export Using Lazy Scan
     p: Path = Path(f"./{name}_{version}.tmp")
 
@@ -497,11 +541,9 @@ def compile_graph(subgraphs: list[Path], name: str, version: str, fmt: str = "mi
             eagernode.write_ndjson(f)
 
     with e.open("a") as f:
-        with pl.Config(set_fmt_float=fmt):  # pyright: ignore
-            with pl.Config(float_precision=precision):
-                for subedge in subedges:
-                    eageredge: pl.DataFrame = subedge.collect().unique()
-                    eageredge.write_ndjson(f)
+        for subedge in subedges:
+            eageredge: pl.DataFrame = subedge.collect().unique()
+            eageredge.write_ndjson(f)
 
     dedup_stream(e, is_edges=True)
     dedup_stream(n, is_edges=False)

{tablassert-7.5.0 → tablassert-7.5.2}/src/tablassert/progress.py

@@ -106,15 +106,23 @@ class PipelineProgress(AbstractContextManager["PipelineProgress"]):
         )
         self.end_section_task()
 
-    def section_loop(self: "PipelineProgress", total: int, label: str) -> Callable[[str], None]:
+    def section_loop(
+        self: "PipelineProgress", total: int, label: str
+    ) -> tuple[Callable[[str], None], Callable[[], None]]:
+        # ? Returns A start Callback To Mark The In Flight Item And An advance Callback To Tick The Counter
+        # ! start Updates The Description Without Incrementing So The Bar Shows The Item Being Processed
         self.end_section_task()
-        self.section_task = self.progress.add_task(description=f"{label.upper()} | WORKING", total=total)
+        self.section_task = self.progress.add_task(description=f"{label.upper()} | STARTING", total=total)
 
-        def advance(info: str) -> None:
+        def start(info: str) -> None:
             assert self.section_task is not None
-            self.progress.update(self.section_task, description=f"{label.upper()} | {info}", advance=1)
+            self.progress.update(self.section_task, description=f"{label.upper()} | {info}")
 
-        return advance
+        def advance() -> None:
+            assert self.section_task is not None
+            self.progress.update(self.section_task, advance=1)
+
+        return start, advance
 
     def end_section_task(self: "PipelineProgress") -> None:
         if self.section_task is not None:

{tablassert-7.5.0 → tablassert-7.5.2}/tests/test_lib.py

@@ -7,7 +7,7 @@ import polars as pl
 
 import tablassert.lib as lib
 from tablassert.ingests import from_yaml
-from tablassert.lib import Tcode, idxname, label_edge, strip_nulls
+from tablassert.lib import Tcode, clean_numeric, format_numeric, idxname, label_edge, numeric_columns, strip_nulls
 
 
 # ? idxname Converts Single Letter Columns
@@ -356,3 +356,167 @@ def test_sig_marks_null_as_unsure() -> None:
     lf: pl.LazyFrame = pl.DataFrame({"p value": [None, 0.01, 0.1]}).lazy()
     result: pl.DataFrame = lib.sig(lf).collect()
     assert list(result["significant"]) == ["UNSURE", "YES", "NO"]
+
+
+# ? sig Marks P-Values Between Cutoff And Threshold As INCONCLUSIVE
+def test_sig_marks_inconclusive_band() -> None:
+    lf: pl.LazyFrame = pl.DataFrame({"p value": [0.01, 0.07, 0.1]}).lazy()
+    result: pl.DataFrame = lib.sig(lf).collect()
+    assert list(result["significant"]) == ["YES", "INCONCLUSIVE", "NO"]
+
+
+# ? numeric_columns Matches Any Column With P Value In The Name
+def test_numeric_columns_matches_p_value_substring() -> None:
+    names: list[str] = ["p value", "adjusted p value", "log p value", "subject"]
+    result: list[str] = numeric_columns(names)
+    assert result == ["p value", "adjusted p value", "log p value"]
+    assert "subject" not in result
+
+
+# ? numeric_columns Matches Exact Relationship Strength And Sample Size Names
+def test_numeric_columns_matches_exact_names() -> None:
+    names: list[str] = ["relationship strength", "sample size", "cohort"]
+    result: list[str] = numeric_columns(names)
+    assert "relationship strength" in result
+    assert "sample size" in result
+    assert "cohort" not in result
+
+
+# ? numeric_columns Is Case Insensitive On The P Value Substring
+def test_numeric_columns_case_insensitive() -> None:
+    names: list[str] = ["P Value", "P VALUE"]
+    result: list[str] = numeric_columns(names)
+    assert result == ["P Value", "P VALUE"]
+
+
+# ? clean_numeric Coerces Numeric And Scientific Notation Strings To Float64
+def test_clean_numeric_parses_numeric_and_scientific() -> None:
+    lf: pl.LazyFrame = pl.DataFrame(
+        {"p value": ["1e-8", "0.05", "450"], "sample size": ["1200", "0.42", "-1.2"]}
+    ).lazy()
+    result: pl.DataFrame = clean_numeric(lf).collect()
+    assert result.schema["p value"] == pl.Float64
+    assert result.schema["sample size"] == pl.Float64
+    assert result["p value"].to_list() == [1e-8, 0.05, 450.0]
+    assert result["sample size"].to_list() == [1200.0, 0.42, -1.2]
+
+
+# ? clean_numeric Drops Non Numeric Entries To Null
+def test_clean_numeric_nulls_non_numeric() -> None:
+    lf: pl.LazyFrame = pl.DataFrame(
+        {"p value": ["1e-8", "N/A", "", "<0.001", "abc"], "relationship strength": ["0.85", "n/a", "NULL", "x", "y"]}
+    ).lazy()
+    result: pl.DataFrame = clean_numeric(lf).collect()
+    assert result["p value"].to_list() == [1e-8, None, None, None, None]
+    assert result["relationship strength"].to_list() == [0.85, None, None, None, None]
+
+
+# ? clean_numeric Leaves Non Matching Columns Untouched
+def test_clean_numeric_leaves_non_matching_untouched() -> None:
+    lf: pl.LazyFrame = pl.DataFrame(
+        {"subject": ["BRCA1", "TP53"], "assertion method": ["ANOVA", "t-test"], "p value": ["0.05", "1e-8"]}
+    ).lazy()
+    result: pl.DataFrame = clean_numeric(lf).collect()
+    assert result.schema["subject"] == pl.String
+    assert result.schema["assertion method"] == pl.String
+    assert result.schema["p value"] == pl.Float64
+    assert result["subject"].to_list() == ["BRCA1", "TP53"]
+    assert result["assertion method"].to_list() == ["ANOVA", "t-test"]
+
+
+# ? clean_numeric Is A Noop When No Numeric Columns Are Present
+def test_clean_numeric_noop_without_numeric_columns() -> None:
+    lf: pl.LazyFrame = pl.DataFrame({"subject": ["BRCA1"], "cohort": ["adult"]}).lazy()
+    result: pl.DataFrame = clean_numeric(lf).collect()
+    assert result.schema["subject"] == pl.String
+    assert result.schema["cohort"] == pl.String
+
+
+# ? clean_numeric Is Idempotent On Already Float64 Columns
+def test_clean_numeric_idempotent_on_float64() -> None:
+    lf: pl.LazyFrame = pl.DataFrame({"p value": [1e-8, 0.05]}).lazy()
+    once: pl.DataFrame = clean_numeric(lf).collect()
+    twice: pl.DataFrame = clean_numeric(once.lazy()).collect()
+    assert twice["p value"].to_list() == [1e-8, 0.05]
+    assert twice.schema["p value"] == pl.Float64
+
+
+# ? format_numeric Renders P Value Columns In Scientific Notation
+def test_format_numeric_p_value_scientific() -> None:
+    lf: pl.LazyFrame = pl.DataFrame(
+        {"p value": ["1e-8", "0.05", "0.001"], "adjusted p value": ["0.0001", "0.1", "0.2"]}
+    ).lazy()
+    result: pl.DataFrame = format_numeric(clean_numeric(lf)).collect()
+    assert result["p value"].to_list() == ["1.0000e-08", "5.0000e-02", "1.0000e-03"]
+    assert result["adjusted p value"].to_list() == ["1.0000e-04", "1.0000e-01", "2.0000e-01"]
+    assert result.schema["p value"] == pl.String
+
+
+# ? format_numeric Renders Relationship Strength And Sample Size In Decimal General Format
+def test_format_numeric_decimal_general() -> None:
+    lf: pl.LazyFrame = pl.DataFrame(
+        {"relationship strength": ["0.85", "0.42", "0.1234"], "sample size": ["450", "1200", "7"]}
+    ).lazy()
+    result: pl.DataFrame = format_numeric(clean_numeric(lf)).collect()
+    assert result["relationship strength"].to_list() == ["0.85", "0.42", "0.1234"]
+    assert result["sample size"].to_list() == ["450", "1200", "7"]
+
+
+# ? format_numeric Preserves Nulls As Null
+def test_format_numeric_preserves_nulls() -> None:
+    lf: pl.LazyFrame = pl.DataFrame({"p value": ["1e-8", "N/A", "0.05"]}).lazy()
+    result: pl.DataFrame = format_numeric(clean_numeric(lf)).collect()
+    assert result["p value"].to_list() == ["1.0000e-08", None, "5.0000e-02"]
+
+
+# ? format_numeric Cleans Floating Point Noise To Four Significant Figures
+def test_format_numeric_cleans_float_noise() -> None:
+    lf: pl.LazyFrame = pl.DataFrame({"relationship strength": ["0.85000000001", "0.41999999999"]}).lazy()
+    result: pl.DataFrame = format_numeric(clean_numeric(lf)).collect()
+    assert result["relationship strength"].to_list() == ["0.85", "0.42"]
+
+
+# ? format_numeric Is A Noop When No Numeric Columns Are Present
+def test_format_numeric_noop_without_numeric_columns() -> None:
+    lf: pl.LazyFrame = pl.DataFrame({"subject": ["BRCA1"], "cohort": ["adult"]}).lazy()
+    result: pl.DataFrame = format_numeric(lf).collect()
+    assert result["subject"].to_list() == ["BRCA1"]
+    assert result.schema["subject"] == pl.String
+
+
+# ? Cleaned And Formatted Null Numeric Values Are Stripped From NDJSON Rows
+def test_format_numeric_nulls_stripped_from_ndjson_rows() -> None:
+    lf: pl.LazyFrame = pl.DataFrame(
+        {"subject": ["BRCA1", "TP53"], "p value": ["1e-8", "N/A"], "relationship strength": ["0.85", "0.42"]}
+    ).lazy()
+    formatted: pl.DataFrame = format_numeric(clean_numeric(lf)).collect()
+    rows: list[dict[str, Any]] = [strip_nulls(r) for r in formatted.iter_rows(named=True)]
+    assert rows[0] == {"subject": "BRCA1", "p value": "1.0000e-08", "relationship strength": "0.85"}
+    assert "p value" not in rows[1]
+    assert rows[1]["subject"] == "TP53"
+    assert rows[1]["relationship strength"] == "0.42"
+
+
+# ? compile_graph Emits Edges And Nodes After Float Formatting Config Removal
+def test_compile_graph_emits_ndjson(monkeypatch: Any, tmp_path: Path) -> None:
+    monkeypatch.chdir(tmp_path)
+    sub: Path = tmp_path / "sub.parquet"
+    pl.DataFrame(
+        {"subject": ["A", "B"], "object": ["X", "Y"], "predicate": ["r", "r"], "p value": ["1.0000e-08", "5.0000e-02"]}
+    ).write_parquet(sub)
+    lib.compile_graph([sub], "smoke", "1.0.0")
+    edges: list[str] = (tmp_path / "smoke_1.0.0.edges.ndjson").read_text().strip().splitlines()
+    nodes: list[str] = (tmp_path / "smoke_1.0.0.nodes.ndjson").read_text().strip().splitlines()
+    assert len(edges) == 2
+    assert all('"uuid"' in line for line in edges)
+    flat: str = "\n".join(edges)
+    assert '"p value":"1.0000e-08"' in flat
+    assert len(nodes) >= 1
+
+
+# ? sig Computes Significance On A Cleaned Float64 P Value Column
+def test_sig_works_on_cleaned_float64() -> None:
+    lf: pl.LazyFrame = pl.DataFrame({"p value": ["1e-8", "0.5", "N/A"]}).lazy()
+    cleaned: pl.LazyFrame = clean_numeric(lf)
+    result: pl.DataFrame = lib.sig(cleaned).collect()
+    assert result["significant"].to_list() == ["YES", "NO", "UNSURE"]

tablassert-7.5.2/tests/test_progress.py

@@ -0,0 +1,48 @@
+from __future__ import annotations
+
+from typing import Any
+
+from tablassert.progress import PipelineProgress
+
+
+# ? Helper To Read The Active Section Task Fields
+def _section_task(p: PipelineProgress) -> Any:
+    return next(t for t in p.progress.tasks if t.id == p.section_task)
+
+
+# ? start Marks The In Flight Item In The Description Without Ticking The Counter
+def test_start_shows_in_flight_item_without_incrementing() -> None:
+    p: PipelineProgress = PipelineProgress(total_stages=1)
+    start, advance = p.section_loop(3, "Collect")
+    task: Any = _section_task(p)
+    assert task.completed == 0
+    start("CONFIG: my_table.yaml")
+    task = _section_task(p)
+    assert "my_table.yaml" in task.description
+    assert task.completed == 0
+    assert callable(advance)
+
+
+# ? advance Ticks The Counter Without Changing The In Flight Description
+def test_advance_ticks_counter_without_changing_description() -> None:
+    p: PipelineProgress = PipelineProgress(total_stages=1)
+    start, advance = p.section_loop(3, "Collect")
+    start("CONFIG: my_table.yaml")
+    advance()
+    task: Any = _section_task(p)
+    assert task.completed == 1
+    assert "my_table.yaml" in task.description
+    assert callable(start)
+
+
+# ? Repeated start advance Cycles Keep Description Synced To The Current Item Not The Last
+def test_start_advance_cycle_shows_current_not_previous() -> None:
+    p: PipelineProgress = PipelineProgress(total_stages=1)
+    start, advance = p.section_loop(3, "Subgraph")
+    start("CONFIG: first.yaml")
+    advance()
+    start("CONFIG: second.yaml")
+    task: Any = _section_task(p)
+    assert task.completed == 1
+    assert "second.yaml" in task.description
+    assert "first.yaml" not in task.description

{tablassert-7.5.0 → tablassert-7.5.2}/uv.lock

@@ -1737,15 +1737,15 @@ wheels = [
 
 [[package]]
 name = "pyright"
-version = "1.1.409"
+version = "1.1.411"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
     { name = "nodeenv" },
     { name = "typing-extensions" },
 ]
-sdist = { url = "https://files.pythonhosted.org/packages/51/4e/3aa27f74211522dba7e9cbc3e74de779c6d4b654c54e50a4840623be8014/pyright-1.1.409.tar.gz", hash = "sha256:986ee05beca9e077c165758ad123667c679e050059a2546aa02473930394bc93", size = 4430434, upload-time = "2026-04-23T11:02:03.799Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/7e/ab/265f7dc69d28113ebba19092e57b075f41543b2ed048429c5f56e2b88eac/pyright-1.1.411.tar.gz", hash = "sha256:d885a0551f2e763b089a02702174e7f4ba77548cddabc972ab86d1f7f1b0f998", size = 4112861, upload-time = "2026-06-25T02:14:06.37Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/16/6b/330d8ebae582b30c2959a1ef4c3bc344ebde48c2ff0c3f113c4710735e11/pyright-1.1.409-py3-none-any.whl", hash = "sha256:aa3ea228cab90c845c7a60d28db7a844c04315356392aa09fafcee98c8c22fb3", size = 6438161, upload-time = "2026-04-23T11:02:01.309Z" },
+    { url = "https://files.pythonhosted.org/packages/0a/49/385be530a6a5b78d1cbcd5c2e38debc8959a2fc6bdb716f4e581002979fc/pyright-1.1.411-py3-none-any.whl", hash = "sha256:dc7c72a8e2700c55baa127554040e067041ea53ccfd50bf96308cc4291c7d5d9", size = 6181526, upload-time = "2026-06-25T02:14:04.691Z" },
 ]
 
 [[package]]
@@ -2340,7 +2340,7 @@ wheels = [
 
 [[package]]
 name = "tablassert"
-version = "7.5.0"
+version = "7.5.2"
 source = { editable = "." }
 dependencies = [
     { name = "cyclopts" },
@@ -2422,7 +2422,7 @@ provides-extras = ["rt", "qc", "qc-cuda"]
 dev = [
     { name = "mkdocs", specifier = ">=1.6.1" },
     { name = "pre-commit", specifier = ">=4.5.1" },
-    { name = "pyright", specifier = ">=1.1.408" },
+    { name = "pyright", specifier = ">=1.1.411" },
     { name = "pytest", specifier = ">=9.0.2" },
     { name = "pytest-cov", specifier = ">=7.1.0" },
     { name = "ruff", specifier = ">=0.15.6" },

tablassert-7.5.0/AGENTS.md

@@ -1,50 +0,0 @@
-# AGENTS.md — Tablassert
-
-## Fast Start
-
-- Python package, not a monorepo. Main code lives in `src/tablassert/`; tests live in `tests/`.
-- Install with `uv sync`. QC is not available unless you install an extra: `uv sync --extra qc` or `uv sync --extra qc-cuda`.
-- CLI entrypoint is `tablassert.cli:APP`. Real user commands are:
-  - `uv run tablassert build <graph.yaml>`
-  - `uv run tablassert validate <table.yaml>`
-
-## Verify Changes
-
-- Match the repo hooks before finishing: `uv run ruff check --fix .`, `uv run ruff format .`, `uv run pyright`, `uv run pytest`.
-- Full hook run: `uv run pre-commit run --all-files`.
-- Focused test runs:
-  - Single test: `uv run pytest tests/test_lib.py::test_name`
-  - By keyword: `uv run pytest -k "pattern"`
-  - With print output: `uv run pytest -s tests/test_lib.py`
-- Docs build: `uv run --group dev mkdocs build`
-
-## High-Value Structure
-
-- `src/tablassert/cli.py` is the wiring layer: `build()` calls `build_pipeline()`, `validate()` calls `validate_pipeline()`.
-- `src/tablassert/ingests.py` loads YAML and expands table configs into section dicts.
-- `src/tablassert/lib.py` is the core pipeline:
-  - `Tcode.collect()` builds the per-section operation list.
-  - `compile_subgraph()` executes that list into parquet.
-  - `compile_graph()` aggregates subgraph parquets into KGX NDJSON.
-  - `resolve_many()` is the direct library API for batch entity resolution.
-- Entity resolution uses DuckDB shard files under `<datassert>/data/`. `src/tablassert/fullmap.py` hardcodes `SHARDS = 10`.
-
-## Repo-Specific Gotchas
-
-- Heavy dependencies are lazy-loaded per module with `TYPE_CHECKING` + `lazy_loader`. Follow the existing pattern instead of importing heavy packages eagerly.
-- `tests/conftest.py` autouse-mocks `httpx.head`, so model URL validation tests do not hit the network unless a test is explicitly marked otherwise.
-- Network-dependent tests are marked `@pytest.mark.network`; GPU QC tests are marked with both `network` and `gpu` in `tests/test_qc.py`.
-- QC runtime selection is strict in `src/tablassert/qc.py`: if `onnxruntime-gpu` is installed but `CUDAExecutionProvider` is unavailable, the code raises instead of falling back to CPU.
-- Downloader behavior in `src/tablassert/downloader.py` is two-path: direct `httpx` fetch for known file URLs, headless-browser fallback for browser-only sources. Keep tests around payload validation and cleanup intact when changing it.
-
-## Conventions That Matter Here
-
-- Start every module with `from __future__ import annotations`.
-- Annotate locals, not just function signatures.
-- Use `Optional[T]` / `Union[...]`, not `T | None`.
-- Prefer `Path` over raw path strings.
-- Function docs are usually `# ?` comments above the code, not docstrings.
-
-## Side Effects
-
-- The package writes working artifacts to hidden directories in the repo root: `.storassert/`, `.logassert/`, `.cachassert/`, and `.onnxassert/`.

tablassert-7.5.0/docs/changelog.md

@@ -1,25 +0,0 @@
-# Changelog
-
-The canonical release history lives in the repository root at [`CHANGELOG.md`](https://github.com/SkyeAv/Tablassert/blob/main/CHANGELOG.md).
-
-## Current Release Notes
-
-## 7.4.14 - 2026-06-30
-
-### Changes
-
-- Extended `sig()` in `lib.py` to select a p-value column by fuzzy matching rather than requiring an exact `"p value"` name. All schema columns whose names contain the substring `"p value"` are now considered candidates; `fuzz.ratio` (rapidfuzz) scores each against the literal `"p value"` and the highest-scoring column is used to compute the `"significant"` output. An exact `"p value"` column scores 100 and is always preferred; columns like `"adjusted p value"` or `"log p value"` are used only when no exact match is present. If no column contains the substring the function continues to emit `"UNSURE"` for all rows.
-- Added five regression tests in `test_lib.py` covering: exact-match preference, non-exact fallback, closest-match selection among multiple non-exact candidates, no-p-value column (UNSURE), and null value handling.
-
-## 7.4.13 - 2026-06-30
-
-### Changes
-
-- Removed the datassert prevalidation failure for unresolved `statement.subject` / `statement.object` literal encodings under `method: value`. Graph builds no longer abort during `Tcode.model_validate(...)` for cases like `"Incertae Sedis XI"`; unresolved literal values are now allowed through config validation so downstream runtime handling can decide whether they map or get filtered.
-- Added a regression test at the `Tcode.model_validate(...)` layer covering an unresolved `method: value` subject encoding, matching the build-time validation path reported in the field.
-
-## 7.4.12 - 2026-06-29
-
-### Changes
-
-- Expanded the placeholder-term filter regex in `distinct()` (`fullmap.py`) to drop additional non-informative terms during entity resolution. The `bad` pattern now also excludes `not applicable`, `p value`, `variable`, `result`, `exposure`, `expression`, and `symbol` alongside the existing `none`, `nan`, `na`, `null`, and `unknown`, preventing these generic column-header-like values from being sent through resolution and producing spurious CURIE mappings.