zekan 0.1.0__tar.gz

zekan-0.1.0/.gitignore

@@ -0,0 +1,56 @@
+# Python
+__pycache__/
+*.py[cod]
+*.pyo
+*.pyd
+*.so
+*.egg
+*.egg-info/
+dist/
+build/
+.eggs/
+.installed.cfg
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.whl
+
+# Virtual environments
+venv/
+.venv/
+env/
+.env/
+ENV/
+
+# Zekan cache
+.zekan/
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+
+# Type checking
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Ruff
+.ruff_cache/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Local scratch artifacts
+scratch/

zekan-0.1.0/COVERAGE_MAP.md

@@ -0,0 +1,241 @@
+# Coverage Map
+
+What Zekan checks, what it does not, and where the boundaries of what it
+does check are known to sit. This is not a list of apologies — it is the
+thing a tool that measures trust has no business hiding. Every claim below
+is sourced to committed code or a committed calibration/validation
+document.
+
+---
+
+## 1. What runs in a real audit
+
+Every `zekan audit` invocation runs two independent layers:
+
+### The A/B/C severity engine
+
+Measures `fixable_leakage` from columns you declared in
+`forbidden_after_prediction`, with a permutation-null detection gate. See
+[`METHODOLOGY.md`](METHODOLOGY.md) for the full definition. This is the
+only layer that can change the verdict (`PASS`/`NOTE`/`WARN`/`FAIL`/
+`UNCONFIRMED_HIGH_DAMAGE`).
+
+### Nine structural probes (annotate-only — never change the verdict)
+
+Registered in `zekan/severity/audit.py::_build_probe_registry`, run on
+every audit alongside the engine:
+
+| probe | catches | annotate-only |
+|---|---|---|
+| `probe_forbidden_entity_level_aggregate` | a forbidden column that is constant *within* each entity — a value the model could learn as an entity ID in disguise | yes |
+| `probe_raw_duplicates` | exact-duplicate rows anywhere in the dataset | yes |
+| `probe_cross_fold_duplicates` | a test-fold row that is byte-identical to a training-fold row | yes |
+| `probe_undeclared_feature_screen` (Upgrade 1) | non-forbidden features with a high standalone (univariate, temporal-fold) association with the target — a ranked candidate list, not a verdict | yes |
+| `probe_near_bijection` (Upgrade H) | a non-forbidden column whose values determine the target almost exactly (Theil's U ≥ 0.99) — a near-perfect value-to-label partition, the shape of an ID leaking the answer | yes |
+| `probe_terminal_row_presence` | a non-forbidden column populated *only* on an entity's last observed row — the shape of a post-outcome field (settlement amount, disposition code) nobody declared forbidden | yes |
+| `probe_entity_contamination_risk` | whether the same entity recurs across more than one `prediction_time` period — the structural condition under which a *random* (non-grouped) split would leak that entity into both train and test | yes (`confirmed=False`, advisory) |
+| `probe_splitter_contract_violation` | an internal self-check: verifies Zekan's own random-grouped splitter never placed the same entity in both train and test of a fold. If it fires, the bug is in Zekan, not your data | yes (internal integrity check, not a data finding) |
+| `probe_forbidden_zero_cooccurrence` | a *declared forbidden* column with zero non-missing rows where the target is positive — the declared leak is structurally incapable of having contributed to the measured `fixable_leakage`, whatever number came back. See [`FORBIDDEN_ZERO_COOCCURRENCE_CHECK.md`](FORBIDDEN_ZERO_COOCCURRENCE_CHECK.md) | yes |
+
+None of these nine can turn a clean `fl` into a WARN/FAIL. Six are
+target-free by design (`probe_forbidden_entity_level_aggregate`,
+`probe_raw_duplicates`, `probe_cross_fold_duplicates`,
+`probe_terminal_row_presence`, `probe_entity_contamination_risk`,
+`probe_splitter_contract_violation`); three read the target because that is
+exactly what they measure — `probe_undeclared_feature_screen`'s univariate
+association and `probe_near_bijection`'s value-to-label determinism are
+both, by definition, feature-vs-target relationships;
+`probe_forbidden_zero_cooccurrence` reads the target's own value (positive
+vs. negative row) rather than any feature-vs-target statistical
+relationship — a deductive non-missing-count fact, not a model score.
+`probe_splitter_contract_violation`
+reads neither the target nor forbidden-column content — it checks only
+entity_id positions across the random-grouped fold indices Zekan itself
+built. Every finding from these probes appears under
+`structural_annotations` in the JSON export and in a dedicated section of
+both human-readable formatters, never folded silently into the headline
+verdict.
+
+**A real, near-universal consequence worth stating plainly**:
+`probe_entity_contamination_risk` will fire `WARN` on almost every dataset
+Zekan is designed for — its trigger condition (an entity recurring across
+more than one period) is the same "recurring over time" structure this
+README's own fit criteria require. Measured directly against the Freddie
+Mac Frame C data: 1,902 of 1,905 entities (99.8%) trigger it. This is not
+noise or miscalibration; it is the advisory correctly describing the
+structural risk of ever running a *non-grouped* split on this shape of
+data, restated on every audit because the risk is present on every audit
+of this shape of data.
+
+**`structural_coverage_complete`** (a top-level JSON field, and a caveat
+line in both formatters when false) tracks whether every probe that was
+*supposed* to run actually did — a probe that crashed internally, or
+that needed temporal folds the data couldn't supply, sets this false and
+lists why, rather than letting a clean-looking PASS pass over an
+unchecked category silently.
+
+---
+
+## 2. A masking defect this document caused to be found, and its fix
+
+Writing §1 above (first published `f3a7ffe`) surfaced a real defect:
+`probe_entity_contamination_risk` and `probe_splitter_contract_violation`
+were complete, individually-tested modules, **not** registered in
+`_build_probe_registry()` — no real `zekan audit` run had ever called
+either. `structural_coverage_complete` stayed `True` throughout, because
+that field's computation only ever considers registered probes; an
+unregistered probe is invisible to it by construction, not merely
+untested. Full record, including why this is a worse defect than the
+structural-coverage-caveat fix (§1's `structural_coverage_complete`
+mechanism itself), how it was found (writing this document, not a test),
+and why it does not change any conclusion in the Freddie Mac validation
+study: `DATASET2_ADDENDUM_15_UNREGISTERED_STRUCTURAL_PROBES.md`.
+
+**Fixed.** Both probes are now registered (§1's table). Wiring
+`probe_splitter_contract_violation` required more than adding a registry
+line: its own signature (`folds` first, ahead of `df`/`contract`) had
+drifted from the registry's calling convention, and — found only by
+measuring, not by reading the signature alone — the probe validates the
+*random-grouped* splitter's contract specifically (no entity ever appears
+in both train and test), a property the *temporal* splitter deliberately
+does **not** uphold (`temporal_expanding_folds`'s own docstring: entity
+recurrence across train/test "is allowed... this is not leakage"). Giving
+it the temporal folds `_run_structural_probes` already had available
+would have made it report `internal_fail` — "every evaluation result from
+this run is unreliable" — on essentially every real audit of the
+recurring-entity data Zekan is built for. The fix threads the
+random-grouped folds through as a distinct, additive capability
+(`SeverityResult.random_folds`, `_ProbeSpec.needs_random_folds`) rather
+than reusing the temporal ones. Addendum 15 §2 and §4 have the full
+before/after reasoning and the direct measurement
+(`entity_overlap_count`: 100/100 entities under temporal folds, 0/0 under
+random-grouped, on the same panel).
+
+**Regression guard added**: a test walks `zekan/detectors/` dynamically
+(`pkgutil`, not a hand-maintained module list — a hand-maintained list is
+how this defect happened) and fails if any `probe_*` function exists
+without a matching entry in `_build_probe_registry()`
+(`tests/test_structural_probe_wiring.py::test_every_implemented_probe_module_is_registered`).
+
+## 3. Declared in the schema, never implemented
+
+`zekan/detectors/schema.py`'s `IssueType` enum reserves a few categories
+that carry a full coverage-classification entry but no implementing probe
+anywhere in the codebase: `ENTITY_CONTAMINATION` (the *confirmed*,
+cross-fold-observed variant — distinct from the risk-advisory probe in
+§2), `WRONG_SPLIT_STRATEGY`, `SUSPECTED_UNDECLARED_LEAK` (Upgrade 1's
+retired lower-confidence tier — killed on calibration grounds, see
+`UPGRADE1_CALIBRATION.md`), and `CODE_STRUCTURAL_LEAK`, which the schema
+itself labels `SourceLayer.OUT_OF_SCOPE`. These are planned scope
+markers, not silent gaps — the enum's own docstring calls it "the
+scope-declaration forcing function," and `OUT_OF_SCOPE` is a real,
+readable value a JSON consumer can check for, not an omission.
+
+---
+
+## 4. Known boundaries of what the live probes actually catch
+
+Findings below come from the Freddie Mac second-dataset validation study
+(`zekan/benchmark/results/DATASET2_FREDDIEMAC_VALIDATION_STUDY.md`) —
+every threshold Zekan ships was calibrated on one dataset
+(Diabetes-130); this study is the record of what did and did not
+transfer to an independently structured second one.
+
+### The terminal-row check's miss set: event-stamp vs. persistent-state fields
+
+`probe_terminal_row_presence` catches columns populated **exclusively**
+on an entity's last row (`terminal_row_fraction == 1.0` exactly — a
+deductive, not inductive, criterion by design,
+`TERMINAL_ROW_PRESENCE_PREREGISTRATION.md` §9.3). Validated against a
+hand-authored, independent ground-truth list of 24 known post-outcome
+Freddie Mac columns: **16 of 24 hit exactly**, **0 false positives**
+(study doc §6). The 8 misses are not random — every one is
+outcome-adjacent for a reason *other than* row position:
+
+| miss | why it's missed |
+|---|---|
+| `MODIFICATION FLAG`, `PAYMENT DEFERRAL FLAG`, `INTEREST RATE STEP INDICATOR`, `CURRENT PERIOD MODIFICATION COSTS` | **persistent state, not an event stamp** — these recur across many rows of an affected entity's life once set (e.g. ~35 rows/loan for the modification-cost field), not a single terminal record |
+| `BORROWER ASSISTANCE PLAN`, `DELINQUENCY DUE TO DISASTER` | current-period distress flags genuinely forward-associated with the target (93%/91% concentration) — real signal, not a row-position artifact |
+| `MORTGAGE INSURANCE CANCELLATION INDICATOR` | dense on 1,893 of 1,905 entities — the opposite of terminal-only |
+| `BANKRUPTCY CRAMDOWN COSTS` | the check's own named stress anchor: `terminal_row_fraction = 0.535`, a genuinely mixed pattern (a bankruptcy event that sometimes, not always, coincides with termination) — plausibly a real partial leak that no current structural check catches, recorded as an accepted limit, not discovered later as a surprise |
+
+**The general shape of the miss set: a column whose leak-relevant state is
+written once and persists across many subsequent rows (a flag, a status,
+a running cost total) looks nothing like a terminal-only field to a
+row-position check.** This is a real, principled boundary of the
+detection mechanism, not a bug — the check is deductive precisely because
+it only claims what row position alone can prove.
+
+### One row per entity: the terminal-row check correctly recognizes it has nothing to check
+
+On a frame where every entity contributes exactly one row (no lifecycle
+for "terminal" to mean anything relative to), `probe_terminal_row_presence`
+returns no findings — this frame shape was measured to make the check
+fire on 40 of 40 candidates with zero discrimination before this guard
+existed (`DATASET2_ADDENDUM_12_FRAME_T_FALSIFIED_INJECTED_ANCHOR.md` §3);
+it is now recognized as **inapplicable**, not silently vacuous, and does
+not count against `structural_coverage_complete`
+(`TERMINAL_ROW_PRESENCE_PREREGISTRATION.md` §10). Zekan's core severity
+engine likewise needs recurring per-entity observations over time — see
+the README's "Does Zekan fit my project?" section — so this boundary
+applies to the same class of one-row-per-item data the engine itself
+already declines to fit.
+
+### The screenability gate is anti-correlated with leak risk
+
+`probe_undeclared_feature_screen` (Upgrade 1) requires a minimum
+non-missing count and minority-class count before it will score a
+candidate feature at all (`_MIN_NONMISSING_COUNT`,
+`_MIN_MINORITY_CLASS_COUNT`). Measured on the Freddie Mac frame: this
+gate excluded 16 of 63 candidate columns, and **15 of those 16 were
+post-outcome/post-liquidation columns**
+(`DATASET2_ADDENDUM_06_FRAME_D_PROBE_SEARCH.md` Finding 1) — the gate's
+own sparsity floor excludes precisely the highest-risk class of column,
+*because* outcome-derived fields are sparse for the same reason the
+outcome itself is rare. The terminal-row check (above) closes part of
+this specific gap for one leak shape (event-stamp fields); the gate's
+general anti-correlation with risk remains, un-addressed, for any
+candidate feature the terminal-row check's own narrower criterion
+doesn't happen to also catch.
+
+### Upgrade H's `U >= 0.99` remains untested on a second dataset
+
+The near-bijection check has never fired on any real Freddie Mac feature,
+across two independent runs
+(`DATASET2_ADDENDUM_06_FRAME_D_PROBE_SEARCH.md`,
+`DATASET2_ADDENDUM_07_FRAME_D_DROPPED.md`). This is recorded as
+**untested, not passed** — the one pre-identified real candidate for a
+near-bijective leak (DDLPI, a field the source data guide's own
+definition says delinquency status is computed from) was ruled out on
+data-quality grounds before it could reach the check: of the 27,524 rows
+where the target actually fires, DDLPI is populated on only 47 (0.17%)
+— too sparse for any screen to evaluate, not a clean negative result
+(study doc §4a). The `0.99` threshold's separation margin
+(`UPGRADE_H_CALIBRATION.md`) has only ever been measured against
+Diabetes-130.
+
+---
+
+## 5. What Zekan does not attempt, by design (v1 scope)
+
+Stated in the audit's own "Scope note" (printed on every run) and in the
+README's fit criteria:
+
+- Combined or multi-feature leakage — the undeclared-feature screen is
+  strictly univariate.
+- Notebooks, user split code, or external train/test artifacts — Zekan
+  evaluates under the contract you declare, not your actual pipeline code.
+- One-row-per-item data with no repeated entity structure over time — the
+  core engine needs recurring per-entity observations to build temporal
+  folds at all.
+- Regression targets and multi-class targets — v1 is yes/no prediction
+  only.
+
+---
+
+## 6. See also
+
+[`METHODOLOGY.md`](METHODOLOGY.md) — what `fixable_leakage` measures, why
+detection and severity are separate questions, and why the calibrated
+floors (`warn_floor`/`fail_floor`) were shown not to transfer as absolute
+values to a second dataset.

zekan-0.1.0/FORBIDDEN_ZERO_COOCCURRENCE_CHECK.md

@@ -0,0 +1,94 @@
+# Forbidden Zero-Cooccurrence Check
+
+Dated 2026-08-20.
+
+## Motivation
+
+Frame P (`zekan/benchmark/results/DATASET2_ADDENDUM_10_FRAME_P_SEVERITY_DILUTION.md`)
+declared 25 forbidden columns and Zekan returned `fixable_leakage=0.0085` --
+a number that was **correct but meaningless**. That addendum's table A2
+measured that 16 of those 25 columns (the arithmetic loss/disposition
+family: `ACTUAL LOSS` and its components, `ZERO BALANCE CODE/EFFECTIVE
+DATE/REMOVAL UPB`, `DELINQUENT ACCRUED INTEREST`, `DUE DATE OF LAST PAID
+INSTALLMENT (DDLPI)`, `CUMULATIVE MODIFICATION COSTS`, `UNDERWRITING DEFECT
+AND MAJOR SERVICING DEFECT SETTLEMENT DATE`) were **0.000000 populated on
+every one of the frame's 4,567 positive rows**. A column that is never
+non-missing on a positive row carries no information distinguishing
+positives from negatives -- no leak declared through it could possibly have
+contributed to the measured B-C gap on that frame, no matter what the
+arithmetic landed on. Zekan already had every number needed to notice this
+(non-missing counts, target values) and said nothing. This check closes
+that gap.
+
+## Criterion
+
+No pre-registration measurement was needed before implementing this check
+-- the claim is deductive, not statistical:
+
+For each column in `forbidden_after_prediction`, count non-missing rows
+where `target == 1` (the positive class; this project's targets are
+0/1-encoded binary columns throughout, the same convention
+`near_bijection_probe.py` already relies on). If that count is **zero**,
+the declared leak cannot contribute to `B - C` on this frame: the column
+carries no information distinguishing positives, so any severity number
+computed with it among the forbidden set is structurally pinned toward
+whatever the *other* forbidden columns alone would have produced.
+
+Counting only -- no model, no folds, no target relationship beyond
+non-missing presence at the row level.
+
+## Behavior
+
+- **Annotate-only.** Never changes `policy_decision.verdict`, same as every
+  other structural probe in `zekan/severity/audit.py::_build_probe_registry`.
+- `confirmed=True`, scoped to the structural **fact** (zero co-occurrence)
+  -- never to severity. It says nothing about whether *other* declared
+  forbidden columns on the same frame carried a real, measurable leak.
+- Message states meaning first: `"'{col}' is never populated on any row
+  where {target} is positive, so declaring it forbidden cannot change the
+  measured severity on this frame"`.
+- Action line points at the contract or the target's own definition --
+  deliberately nothing about imputation, since imputing a value that was
+  never observed on a positive row would manufacture information that does
+  not exist, not recover it.
+- Reports the count and the positive-row total as evidence
+  (`n_non_missing_on_positive_rows` / `n_positive_rows`).
+- When **every** declared forbidden column meets the criterion, that is a
+  distinct, stronger statement: the entire declared forbidden set is
+  structurally incapable of contributing to `fixable_leakage` on this
+  frame, so the audit could not have found anything through it, whatever
+  number came back. Every per-column record carries `all_forbidden_zero`
+  in its evidence, and its `what` text states the stronger claim directly
+  rather than leaving the reader to infer it from N separate identical-
+  looking records.
+
+See `zekan/detectors/forbidden_zero_cooccurrence_probe.py` for the
+implementation and `zekan/detectors/schema.py`'s `FORBIDDEN_ZERO_COOCCURRENCE`
+registry row / `ForbiddenZeroCooccurrenceDetail` for the typed evidence.
+
+## Interaction with the structural coverage caveat (020a46a)
+
+`zekan/severity/audit.py::_compute_structural_coverage` (introduced in
+`020a46a`) marks a category as a coverage **gap** in exactly two cases: a
+probe was silently skipped because its declared capability precondition
+wasn't met (`needs_folds`/`needs_random_folds` and the fold set wasn't
+available), or a probe raised and was isolated as `PROBE_FAILED`. Neither
+condition applies here: `probe_forbidden_zero_cooccurrence` declares no
+capability flags (same registration shape as
+`probe_forbidden_entity_level_aggregate`) and runs unconditionally to
+completion on every audit.
+
+**Decision: this is NOT a coverage gap.** The check ran and returned a
+record -- that record being "the declared forbidden set is structurally
+incapable of measuring anything here" is the *informative* case
+(`_compute_structural_coverage`'s own docstring: "a probe that ran and
+returned real IssueRecords -- zero, one, or every candidate flagged --
+counts as covered, full stop... never an informativeness check"), the same
+bucket `probe_terminal_row_presence` already occupies when it fires on
+every single candidate on a degenerate cross-sectional frame
+(`DATASET2_ADDENDUM_12_FRAME_T_FALSIFIED_INJECTED_ANCHOR.md` section 3;
+`tests/test_structural_coverage.py::test_vacuous_but_completed_probe_is_not_a_gap`).
+The finding itself -- loud, in `structural_annotations`, `status="fail"`,
+`confirmed=True` -- is the warning. `structural_coverage_complete` stays
+`True` when this is the only thing that fired. Verified directly:
+`tests/test_forbidden_zero_cooccurrence_probe.py::test_all_zero_is_not_a_coverage_gap`.

zekan-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Zekan contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.