catstat 0.1.1__tar.gz

catstat-0.1.1/.claude/skills/benchmark-harness/SKILL.md

@@ -0,0 +1,53 @@
+---
+name: benchmark-harness
+description: >-
+  Run catstat's benchmark harness reproducibly, persist results to the ledger, compare against the
+  committed baseline, and draft a verdict. Invoke for any perf-relevant change, to establish or
+  refresh a baseline, or as the benchmark step of the self-improvement loop. Enforces >=5 reps,
+  median+spread, pinned seeds/versions/SHA, and never changes a default by itself. Outputs a
+  before/after table and a filled docs/verdicts/ entry.
+---
+
+You run the **measurement harness** and turn numbers into a decision. You never change a default
+on your own — that requires a written verdict backed by repeated runs.
+
+## When to use
+- Any perf-relevant change (backend, group-by, dispatch, conversion path).
+- Establishing or refreshing a committed baseline.
+- The benchmark step of the self-improvement loop.
+
+## When NOT to use
+- Correctness-only changes (use `leakage-audit` / `sklearn-compat`).
+- To justify a default change from a single run or a microbenchmark presented as end-to-end.
+
+## Required inputs
+- `--size {small,medium,large}`, `--backend {cpu,gpu}`, `--reps N` (≥5), and a committed baseline
+  JSON to compare against.
+
+## Commands
+```bash
+python3 benchmarks/run_benchmarks.py --backend cpu --reps 5 --out benchmarks/results/<run>.jsonl
+python3 benchmarks/compare_results.py benchmarks/results/<run>.jsonl benchmarks/results/baseline-cpu.json
+python3 scripts/summarize_benchmark_results.py benchmarks/results/<run>.jsonl
+```
+GPU runs go through `scripts/colab_gpu_parity.sh` (Phase 2), not local.
+
+## Files to inspect
+`benchmarks/datasets.py`, `benchmarks/run_benchmarks.py`, `benchmarks/ledger.py`,
+`benchmarks/compare_results.py`, `benchmarks/results/baseline-cpu.json`,
+`docs/verdicts/TEMPLATE-verdict.md`, and the harness design `docs/proposals/evaluation-harness-design.md`.
+
+## Failure modes to catch
+- Fewer than 5 reps; reporting mean without spread.
+- Comparing across different seeds, package versions, or git SHAs.
+- Reporting a microbenchmark win as an end-to-end win.
+- Updating the committed baseline without a verdict.
+- Bundling a harness change with a behavior change in one diff (result becomes un-attributable —
+  keep harness changes in a separate commit).
+- Timing only `fit_transform` wall time without separating `fit` / `transform` / conversion.
+
+## Final report format
+A before/after table (per case: `fit_s`, `transform_s`, `fit_transform_s` as median+spread, peak
+memory), regressions/improvements vs the §8 thresholds (harness doc), parity status if relevant,
+and a filled `docs/verdicts/YYYY-MM-DD-<topic>-verdict.md` with a keep/change/revert decision.
+Update the committed baseline **only** if the verdict says so.

catstat-0.1.1/.claude/skills/leakage-audit/SKILL.md

@@ -0,0 +1,54 @@
+---
+name: leakage-audit
+description: >-
+  Prove that catstat's fit_transform is out-of-fold and that no target information leaks into the
+  encoded features. Invoke for ANY change touching the cross-fit, smoothing, or transform path
+  (_cross_fit.py, _smoothing.py, _base.py transform, fold assignment) before keeping the change.
+  Runs tests/test_cross_fit_no_leakage.py, independently reconstructs each fold's encoding from its
+  complement, and checks the noise-trap. Reports PASS/FAIL with the exact offending path on failure.
+---
+
+You are the **leakage auditor**. The single question: *does any target information leak into a
+`fit_transform` output, directly or via an implementation detail?* Leakage safety is `catstat`'s
+#1 invariant — you sign off before any cross-fit/smoothing change is kept.
+
+## When to use
+- Any diff touching `_cross_fit.py`, `_smoothing.py`, the transform path in `_base.py`, or fold
+  assignment.
+- Before keeping such a change, and whenever a new statistic's OOF behavior is added.
+
+## When NOT to use
+- Pure docs / benchmark / naming changes.
+- Unsupervised `CountEncoder`/`FrequencyEncoder` logic — there is no target, so only run the
+  `fit_transform == fit().transform()` equivalence check (there is nothing to leak).
+
+## Required inputs
+- The diff/PR scope.
+- A seeded dataset with known signal **and** a noise-trap (category independent of `y`); use
+  `benchmarks/datasets.py::make_leakage_trap` and a signal generator.
+
+## Commands
+```bash
+PYTHONPATH=src python3 -m pytest tests/test_cross_fit_no_leakage.py -q
+# ad hoc OOF reconstruction: for each fold, recompute the encoding from the fold's COMPLEMENT and
+# assert it equals the value fit_transform produced for that fold's rows (must be exact on CPU).
+```
+
+## Files to inspect
+`_cross_fit.py`, `_smoothing.py`, `_base.py` (transform), `tests/test_cross_fit_no_leakage.py`,
+and `docs/proposals/target-encoder-library-design.md` §8.
+
+## Failure modes to catch
+- Per-fold statistics that secretly include the held-out fold.
+- `smooth="auto"` variance computed on the full data instead of per fold.
+- Row order scrambled on merge (the produced value lands on the wrong row).
+- Unknown/global fallback drawn from the *transformed* set instead of training folds.
+- An example or test that uses `fit().transform()` on the training set.
+
+## Final report format
+`PASS` / `FAIL`, plus:
+- OOF-reconstruction result (exact match per fold? yes/no).
+- Noise-trap: correlation of the OOF feature with `y` on held-out rows (should be ≈ 0).
+- Which traps were checked and the asymmetry check (`fit_transform ≠ fit().transform()` with signal).
+- On `FAIL`: the exact file/line and which invariant it violates. Escalate non-trivial fixes; do
+  not "make the test pass" by weakening it.

catstat-0.1.1/.claude/skills/release-prep/SKILL.md

@@ -0,0 +1,50 @@
+---
+name: release-prep
+description: >-
+  Prepare a catstat PyPI release: confirm the green gate, bump the version in both pyproject.toml
+  and __init__.py (kept in sync), update CHANGELOG.md, build sdist+wheel, and run twine check +
+  a clean-venv smoke install. Follows docs/publishing_checklist.md. Prepares and verifies artifacts
+  only — never uploads or tags (the maintainer holds PyPI/GitHub credentials).
+---
+
+You prepare a release and **verify it builds and installs**, stopping short of publishing.
+
+## When to use
+- Cutting a new `catstat` version (after features have landed and the gate is green).
+
+## When NOT to use
+- Mid-development; or to change library behavior (that's a normal PR, not a release).
+
+## Required inputs
+- The target version (SemVer). Confirm what changed since the last tag (read `git log` + the
+  unreleased CHANGELOG section).
+
+## Steps / commands
+1. `bash scripts/check.sh` must be green; coverage ≥ floor.
+2. Bump version in **both** `pyproject.toml` `project.version` and `src/catstat/__init__.py`
+   `__version__` — they MUST match (a mismatch is a release bug).
+3. Move the CHANGELOG `[Unreleased]` items under a dated `[X.Y.Z]` heading; keep it honest
+   (Added / Changed / Fixed / Known limitations).
+4. Build + verify:
+   ```bash
+   python3 -m build
+   python3 -m twine check dist/*
+   python3 -m pip install dist/catstat-*.whl   # in a fresh venv
+   python3 -c "import catstat; print(catstat.__version__)"
+   ```
+5. Hand off: report that artifacts are ready; the maintainer runs `twine upload` + `git tag`.
+
+## Files to inspect
+`pyproject.toml`, `src/catstat/__init__.py`, `CHANGELOG.md`, `docs/publishing_checklist.md`,
+`README.md`, `LICENSE`.
+
+## Failure modes to catch
+- Version mismatch between pyproject and `__init__`.
+- `twine check` warnings (README won't render, missing metadata).
+- Wheel missing `py.typed` or the package data.
+- Building from a dirty tree (uncommitted changes) or a non-green gate.
+- Uploading from this environment (don't — no credentials; it's the maintainer's step).
+
+## Final report format
+Version old→new, the CHANGELOG section, `twine check` result, the smoke-install import line, and an
+explicit "ready to `twine upload` + tag vX.Y.Z" hand-off (or the blocking issue).

catstat-0.1.1/.claude/skills/sklearn-compat/SKILL.md

@@ -0,0 +1,51 @@
+---
+name: sklearn-compat
+description: >-
+  Verify that catstat's encoders behave as well-mannered scikit-learn transformers. Invoke for any
+  change to the public classes, constructor params, fitted attributes, feature names, or output
+  handling, and before a release. Runs tests/test_sklearn_compat.py and spot-checks clone,
+  get/set_params, Pipeline, ColumnTransformer, set_output, and get_feature_names_out. Reports
+  PASS/FAIL per check plus the documented subset of check_estimator that applies.
+---
+
+You verify **scikit-learn protocol compliance** for `catstat`'s public encoders. Full
+`check_estimator` compliance is unrealistic for supervised, multi-output transformers — target a
+**documented subset** and be explicit about what does/doesn't apply and why.
+
+## When to use
+- Changes to `TargetEncoder`/`CountEncoder`/`FrequencyEncoder` public surface: constructor params,
+  fitted attributes, feature names, `set_output`/`output=` handling.
+- Before a release.
+
+## When NOT to use
+- Internal backend/perf changes with no public-surface effect.
+
+## Required inputs
+- The class(es) under test; installed `scikit-learn` (record the version — meaningful
+  `check_estimator` coverage needs ≥1.4, which also has `TargetEncoder` for parity).
+
+## Commands
+```bash
+PYTHONPATH=src python3 -m pytest tests/test_sklearn_compat.py -q
+```
+Spot-checks (must all hold): `sklearn.base.clone(enc)`; `enc.get_params()` /
+`enc.set_params(**p)` round-trip; use inside `Pipeline` and `ColumnTransformer`;
+`enc.set_output(transform="pandas")` returns a DataFrame; `enc.get_feature_names_out()` length
+equals the output width.
+
+## Files to inspect
+`target_encoder.py`, `count_encoder.py`, `frequency_encoder.py`, `_base.py`, `_feature_names.py`,
+`tests/test_sklearn_compat.py`.
+
+## Failure modes to catch
+- Constructor mutates or fails to store a param verbatim (breaks `clone`/`get_params`).
+- A fitted attribute missing its trailing underscore, or set before `fit`.
+- `get_feature_names_out` length ≠ number of output columns (esp. multiclass class-expansion and
+  class-agnostic count/frequency not multiplied by `K`).
+- `set_output` / `output=` not honored, or names lost.
+- Silent failure inside `ColumnTransformer` (cuML had this historically — assert it actually works).
+
+## Final report format
+PASS/FAIL per check; the `scikit-learn` version used; and the **documented** list of
+`check_estimator` checks that are (in)applicable to a supervised multi-output transformer, with a
+one-line reason each. On FAIL, the precise attribute/method and the protocol expectation it misses.

catstat-0.1.1/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,30 @@
+---
+name: Bug report
+about: Report a problem with catstat
+title: "[bug] "
+labels: bug
+---
+
+**Describe the bug**
+A clear description of what went wrong.
+
+**To reproduce**
+A minimal, self-contained snippet:
+
+```python
+import pandas as pd
+from catstat import TargetEncoder
+# ...
+```
+
+**Expected behavior**
+What you expected to happen.
+
+**Environment**
+- catstat version:
+- Python version:
+- pandas / numpy / scikit-learn versions:
+- backend (cpu/gpu) and OS:
+
+**Additional context**
+Full traceback, data shape, or anything else relevant.

catstat-0.1.1/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: true
+contact_links:
+  - name: Report a security vulnerability
+    url: https://github.com/Matapanino/catstat/security/advisories/new
+    about: Please report vulnerabilities privately (see SECURITY.md), not as public issues.

catstat-0.1.1/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,22 @@
+---
+name: Feature request
+about: Suggest an idea or new capability for catstat
+title: "[feature] "
+labels: enhancement
+---
+
+**What problem does this solve?**
+The use case or limitation you're hitting.
+
+**Proposed solution**
+What you'd like catstat to do (an API sketch is welcome).
+
+**Alternatives considered**
+Other approaches, or how you work around it today.
+
+**Scope check**
+- Does it fit catstat's scope (leakage-safe statistical categorical encoding)?
+- Is it a new statistic (`stats=`), a backend/IO option, or core behavior?
+
+**Additional context**
+Links, references, or examples.

catstat-0.1.1/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,16 @@
+## Summary
+
+<!-- What does this PR change, and why? -->
+
+## Checklist
+
+- [ ] `bash scripts/check.sh` is green (ruff + pytest + examples).
+- [ ] Tests cover the new behavior (encode correctness, **OOF / no-leakage**, unknown/missing
+      fallback, feature names, determinism — as applicable).
+- [ ] Core invariants intact (leakage safety, smoothing honesty, CPU/GPU parity, public API).
+- [ ] `CHANGELOG.md` updated for user-visible changes; SemVer considered.
+- [ ] Docs updated (`README.md` / `docs/`) where relevant.
+
+## Notes
+
+<!-- Anything reviewers should know: trade-offs, follow-ups, benchmark/verdict links. -->

catstat-0.1.1/.github/workflows/ci.yml

@@ -0,0 +1,34 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install (editable + dev extra)
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[dev]"
+      - name: Lint
+        run: ruff check src tests examples benchmarks scripts
+      - name: Test + coverage
+        # GPU/parity tests carry the `gpu` marker and auto-skip without RAPIDS (CPU runner).
+        run: pytest tests/ -q --cov=catstat --cov-report=term-missing
+      - name: Examples (smoke)
+        run: |
+          python examples/regression_basic.py
+          python examples/binary_classification_basic.py
+          python examples/multiclass_classification_basic.py
+          python examples/count_frequency_basic.py

catstat-0.1.1/.github/workflows/docs.yml

@@ -0,0 +1,51 @@
+name: Docs
+
+# Build the API reference (pdoc) and publish it to GitHub Pages on each push to main.
+# One-time setup: repo Settings -> Pages -> Source: "GitHub Actions".
+
+on:
+  push:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow one concurrent Pages deployment; let an in-progress run finish.
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    name: Build API docs
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install (docs extra)
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[docs]"
+      - name: Build with pdoc
+        run: bash scripts/build_docs.sh site
+      - name: Upload Pages artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: site
+
+  deploy:
+    name: Deploy to GitHub Pages
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy
+        id: deployment
+        uses: actions/deploy-pages@v4

catstat-0.1.1/.github/workflows/release.yml

@@ -0,0 +1,65 @@
+name: Release
+
+# Build and publish to PyPI when a version tag (e.g. v0.1.1) is pushed.
+# Publishing uses PyPI Trusted Publishing (OIDC): no API token is stored in the repo.
+# One-time setup and the manual fallback are documented in docs/publishing_checklist.md.
+
+on:
+  push:
+    tags: ["v*"]
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    name: Build distributions
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Verify tag matches package version
+        env:
+          TAG: ${{ github.ref_name }}
+        run: |
+          python - <<'PY'
+          import os, pathlib, re, sys
+          tag = os.environ["TAG"].lstrip("v")
+          pp = re.search(r'(?m)^version = "([^"]+)"',
+                         pathlib.Path("pyproject.toml").read_text()).group(1)
+          init = re.search(r'(?m)^__version__ = "([^"]+)"',
+                           pathlib.Path("src/catstat/__init__.py").read_text()).group(1)
+          if not (tag == pp == init):
+              sys.exit(f"Version mismatch: tag={tag!r} pyproject={pp!r} __init__={init!r}")
+          print(f"OK: version {tag} matches the tag, pyproject.toml, and __init__.py.")
+          PY
+      - name: Build sdist + wheel
+        run: |
+          python -m pip install --upgrade pip build twine
+          python -m build
+          python -m twine check dist/*
+      - name: Upload dist artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    name: Publish to PyPI (Trusted Publishing)
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/project/catstat/
+    permissions:
+      id-token: write   # OIDC token for PyPI Trusted Publishing — no stored API token
+    steps:
+      - name: Download dist artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

catstat-0.1.1/.gitignore

@@ -0,0 +1,25 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+build/
+dist/
+.eggs/
+
+# Test / lint / coverage caches
+.pytest_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# Benchmark scratch (committed baselines live in benchmarks/results/baseline-*.json)
+benchmarks/results/run.json
+benchmarks/results/_tmp_*.json
+
+# API docs build output (published by .github/workflows/docs.yml)
+site/
+
+# Env / editor
+.venv/
+venv/
+.DS_Store

catstat-0.1.1/CHANGELOG.md

@@ -0,0 +1,60 @@
+# Changelog
+
+All notable changes to `catstat` are documented here. Format follows
+[Keep a Changelog](https://keepachangelog.com/); versioning is [SemVer](https://semver.org/).
+
+## [0.1.1] — 2026-06-26
+
+### Changed
+- Releases now build and publish to PyPI automatically on a `v*` tag, via GitHub Actions and PyPI
+  **Trusted Publishing** (OIDC — no stored API token). See `docs/publishing_checklist.md`.
+- `TargetEncoder` / `CountEncoder` / `FrequencyEncoder` now advertise scikit-learn estimator tags
+  via both `__sklearn_tags__` (sklearn ≥ 1.6) and `_more_tags` (< 1.6): categorical/string input,
+  `allow_nan` (NaN is learned as a level under `handle_missing="value"`), and `requires_y` for the
+  supervised encoder — so `check_estimator` skips inapplicable checks.
+
+### Documentation
+- Rewrote the README: status badges, an honest CPU/GPU status, install + extras, a statistics/
+  feature table, a "leakage-safe by design" note, and a link to the API reference.
+- Published an API reference built with `pdoc` (`scripts/build_docs.sh`), deployed to GitHub Pages
+  via `.github/workflows/docs.yml`.
+
+### Fixed
+- `cols="auto"` now selects pandas `StringDtype` columns, so auto-detection works on pandas ≥ 3.0
+  (where string columns default to `StringDtype` rather than `object`). (KI-022)
+- Fitted estimators are now picklable; a cached backend module previously raised
+  `TypeError: cannot pickle 'module' object`. (KI-012)
+
+## [0.1.0] — 2026-06-26
+
+First public release. Leakage-safe, sklearn-compatible statistical categorical encoding with one
+API across CPU (pandas/numpy) and, optionally, GPU (cuDF/CuPy).
+
+### Added
+- **`TargetEncoder`** — supervised, leakage-safe. `fit_transform` is cross-fitted; `transform`
+  uses full-data encodings for new data. Regression, binary, and multiclass (one-vs-rest) targets.
+- **`CountEncoder` / `FrequencyEncoder`** — unsupervised category-prevalence encoders.
+- **Statistics** (`stats=`): `mean`, `count`, `frequency`, `var`, `std`, `median`, `min`, `max`,
+  `skew`, and **custom `(name, callable)` aggregations** (quantiles, IQR, …). Only mean/probability
+  are smoothed (m-estimate fixed; empirical-Bayes `smooth="auto"`); other statistics fall back to
+  the global value for small/unseen categories and never blend.
+- **Cross-fitting schemes** (`scheme=`): `kfold` (default, out-of-fold), `loo` (leave-one-out),
+  `ordered` (CatBoost-style ordered target statistics). loo/ordered apply to the mean.
+- **Multi-column**: `multi_feature_mode="independent"` (default) or `"combination"` (joint).
+- **Missing / unseen handling**: `handle_missing` / `handle_unknown` ∈ {`value`, `return_nan`,
+  `error`}, with per-statistic fallbacks (count/frequency → 0; mean → global; etc.).
+- **Backends**: `backend="cpu"` (default via `auto`), `backend="gpu"` (cuDF/CuPy, validated
+  CPU/GPU-allclose on a Colab T4, incl. missing). `auto` resolves to CPU for now — the current GPU
+  path is not yet faster than CPU up to 1M rows (see `docs/known_issues.md` KI-020).
+- **Output**: `output` ∈ {`auto`, `numpy`, `pandas`, `polars`}; sklearn `set_output`,
+  `get_feature_names_out`, `Pipeline` / `ColumnTransformer` compatibility.
+- Test suite (88 tests), synthetic benchmark harness, Colab GPU-parity loop, and CI
+  (Python 3.10–3.12).
+
+### Known limitations
+- GPU `auto` disabled pending an on-device redesign (KI-020); `combination` and `skew`/custom
+  aggregations run on CPU. sklearn-parity tests require `scikit-learn>=1.4`. See
+  `docs/known_issues.md`.
+
+[0.1.1]: https://github.com/Matapanino/catstat/releases/tag/v0.1.1
+[0.1.0]: https://github.com/Matapanino/catstat/releases/tag/v0.1.0