eval-toolkit 1.4.0__tar.gz → 1.5.0__tar.gz

{eval_toolkit-1.4.0 → eval_toolkit-1.5.0}/CHANGELOG.md

@@ -5,6 +5,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.5.0] — 2026-05-29 — Tier-2 `eda` layer (#83) + schema-aware `HFDatasetsLoader` (#85)
+
+Tier-2 / `loaders` ADDITIVE per [ADR 0003](docs/source/adr/0003-stability-contract-and-gate3-methodology.md) — backward-compatible.
+
+- **`eda` Job-1 integrity gate (#83):** `audit_dataset` / `DataAudit` / `SplitSummary` + the
+  `class_balance` / `no_cross_split_leakage` / `context_window_fit` gates + the §B2 obfuscation
+  prevalence module.
+- **schema-aware `HFDatasetsLoader` (#85):** load real-world dataset schemas without column
+  guessing — `feature_cols` + `feature_join` (join multiple columns into one feature; NaN-safe),
+  `label_map` (remap raw labels → int; fail-fast `ValueError` lists unmapped values), `revision`
+  (pin the HF dataset SHA). All new params default to the prior behavior; a missing feature/label
+  column raises `KeyError` listing the observed columns.
+
 ## [1.4.0] — 2026-05-26 — `audit_citation_alignment` Layer 2 + Layer 3 (closes #82); shared `_narrative` helpers (ADR 0007)
 
 Tier-1 ADDITIVE per [ADR 0003](docs/source/adr/0003-stability-contract-and-gate3-methodology.md).

{eval_toolkit-1.4.0 → eval_toolkit-1.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eval-toolkit
-Version: 1.4.0
+Version: 1.5.0
 Summary: Reusable evaluation contracts for binary classification: metrics, bootstrap CIs, calibration, artifacts, and evidence gates.
 Project-URL: Homepage, https://github.com/brandon-behring/eval-toolkit
 Project-URL: Documentation, https://brandon-behring.github.io/eval-toolkit/
@@ -60,6 +60,9 @@ Requires-Dist: sphinx-autodoc-typehints>=2.0; extra == 'docs'
 Requires-Dist: sphinx-copybutton>=0.5; extra == 'docs'
 Requires-Dist: sphinx-design>=0.6; extra == 'docs'
 Requires-Dist: sphinx>=7.3; extra == 'docs'
+Provides-Extra: eda
+Requires-Dist: matplotlib>=3.8; extra == 'eda'
+Requires-Dist: pandas>=2.0; extra == 'eda'
 Provides-Extra: embeddings
 Requires-Dist: sentence-transformers>=3.0; extra == 'embeddings'
 Provides-Extra: losses

{eval_toolkit-1.4.0 → eval_toolkit-1.5.0}/pyproject.toml

@@ -74,6 +74,14 @@ probes = ["torch>=2.0", "transformers>=4.40"]
 # (granular extras — losses callers should not have to install the larger
 # transformers stack). Shares the torch version pin with [probes].
 losses = ["torch>=2.0"]
+# v1.5.0 (feat/eda-data-audit): eval_toolkit.eda Job-1 integrity-gate layer.
+# Tier-2 surface (ADR 0003) — torch-free by design. pandas powers the
+# DataFrameLoader reuse path; matplotlib is reserved for the EDA layer's
+# future profiling plots. Intentionally NO sentence-transformers / torch:
+# the near-dup / cross-split checks use the lexical TfidfCosineStrategy and
+# token-length quantiles take a caller-supplied tokenizer (no transformers
+# import in this module). NOT folded into [all] / [dev] — opt-in only.
+eda = ["pandas>=2.0", "matplotlib>=3.8"]
 # NO-OP extra kept for backward compatibility (R3 at v0.49.0).
 #
 # jsonschema>=4.21 moved to base deps at v0.16.0; this extra has been a

{eval_toolkit-1.4.0 → eval_toolkit-1.5.0}/src/eval_toolkit/_version.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "1.4.0"
+__version__ = "1.5.0"

eval_toolkit-1.5.0/src/eval_toolkit/eda/__init__.py

@@ -0,0 +1,80 @@
+"""``eval_toolkit.eda`` — EDA-first dataset integrity gating (Tier-2 surface).
+
+This subpackage is the **Job-1 integrity gate** of an EDA-first research
+program: thin, composable, torch-free per-split profiling + dataset-soundness
+gates, built by reusing the v1.4.0 :mod:`eval_toolkit.leakage`,
+:mod:`~eval_toolkit.text_dedup`, :mod:`~eval_toolkit.claims`, and
+:mod:`~eval_toolkit.artifacts` primitives.
+
+Stability tier
+--------------
+Public access is ``eval_toolkit.eda.*`` — **Tier-2** per ADR 0003. This layer
+is intentionally evolvable and is **not** part of the v2.0-frozen top-level
+:mod:`eval_toolkit` surface; nothing here is added to the package-root
+``_EXPORTS`` / ``__all__``. Import explicitly::
+
+    from eval_toolkit.eda import audit_dataset, DataAudit, SplitSummary
+
+Scope (deliberately narrow)
+--------------------------
+Integrity gating only: row counts, class balance, text-length quantiles,
+dedup / cross-split leakage. **No** embeddings, semantic similarity,
+contamination scoring, or UMAP — those distribution-shift concerns are
+deferred to a future ``distribution_shift`` module.
+"""
+
+from __future__ import annotations
+
+from eval_toolkit.eda.data_audit import (
+    DEFAULT_MAX_NEG_POS_RATIO,
+    DEFAULT_MIN_NEG_POS_RATIO,
+    DEFAULT_PCT_OVER_CONTEXT_THRESHOLD,
+    EDA_AUDIT_SCHEMA_VERSION,
+    DataAudit,
+    SplitSummary,
+    Tokenizer,
+    audit_dataset,
+    class_balance,
+    length_quantiles,
+    summarize_split,
+)
+from eval_toolkit.eda.obfuscation import (
+    BASE64_ENTROPY_THRESHOLD,
+    HEX_ENTROPY_THRESHOLD,
+    ObfuscationProfile,
+    analyze_obfuscation,
+    count_invisible_chars,
+    has_high_entropy_alnum_run,
+    has_rot13_marker,
+    is_leeted_token,
+    leetspeak_counts,
+    nfkc_changed,
+    nfkc_char_delta,
+    shannon_entropy,
+)
+
+__all__ = [
+    "BASE64_ENTROPY_THRESHOLD",
+    "DEFAULT_MAX_NEG_POS_RATIO",
+    "DEFAULT_MIN_NEG_POS_RATIO",
+    "DEFAULT_PCT_OVER_CONTEXT_THRESHOLD",
+    "EDA_AUDIT_SCHEMA_VERSION",
+    "HEX_ENTROPY_THRESHOLD",
+    "DataAudit",
+    "ObfuscationProfile",
+    "SplitSummary",
+    "Tokenizer",
+    "analyze_obfuscation",
+    "audit_dataset",
+    "class_balance",
+    "count_invisible_chars",
+    "has_high_entropy_alnum_run",
+    "has_rot13_marker",
+    "is_leeted_token",
+    "leetspeak_counts",
+    "length_quantiles",
+    "nfkc_char_delta",
+    "nfkc_changed",
+    "shannon_entropy",
+    "summarize_split",
+]