evoforest-tab 0.1.0__tar.gz

evoforest_tab-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 the tabmap authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

evoforest_tab-0.1.0/PKG-INFO

@@ -0,0 +1,119 @@
+Metadata-Version: 2.4
+Name: evoforest-tab
+Version: 0.1.0
+Summary: Evolved universal tabular feature map + closed-form ridge: an interpretable, training-free, local in-context learner for tabular data.
+License: MIT
+Keywords: tabular,in-context-learning,feature-map,tabpfn,ridge,automl
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=1.13
+Requires-Dist: numpy>=1.21
+Requires-Dist: pyyaml>=5.4
+Provides-Extra: sklearn
+Requires-Dist: scikit-learn>=1.0; extra == "sklearn"
+Provides-Extra: examples
+Requires-Dist: scikit-learn>=1.0; extra == "examples"
+Requires-Dist: pandas>=1.3; extra == "examples"
+Dynamic: license-file
+
+# tabmap — EvoForest-Tab: an evolved universal tabular feature map
+
+`tabmap` is the reference implementation of **EvoForest-Tab** (the EvoForest computation-search framework specialized to tabular data).
+
+`tabmap` is an interpretable, training-free, **local** in-context learner for tabular data: an
+evolved universal feature map `φ: row → ℝᴷ` (16 transform families over rank-gauss, count-encoding,
+and categorical-mask channels) paired with a per-dataset **closed-form Bayesian-ridge head**. Given a
+labeled *support* set and an unlabeled *query* set, it predicts in a single SVD solve — no gradient
+descent, no per-dataset tuning, no GPU. It is competitive with gradient boosting and with the
+published **TabPFN-v2** tabular foundation model, while remaining free to run and fully inspectable.
+
+This repository accompanies the paper *"Evolving a Universal Tabular Feature Map: Interpretable,
+Closed-Form In-Context Learning Competitive with Tabular Foundation Models"* and is **stand-alone**:
+the deployment pipeline (feature map + ridge) depends only on `torch`, `numpy`, and `pyyaml`.
+
+## Install
+```bash
+pip install -e .            # editable; or: pip install .
+# deps: torch, numpy, pyyaml  (+ scikit-learn for the estimator base classes & examples)
+```
+
+## Usage (scikit-learn style)
+```python
+from evoforest_tab import TabMapClassifier, TabMapRegressor
+
+clf = TabMapClassifier(n_estimators=6).fit(X_support, y_support)   # X: ndarray or DataFrame
+proba = clf.predict_proba(X_query)                                  # in-context: query needed to fit φ channels
+pred  = clf.predict(X_query)
+
+reg = TabMapRegressor(n_estimators=6).fit(X_support, y_support)
+yhat = reg.predict(X_query)
+```
+Notes:
+- It is an **in-context** learner: `predict` builds the (label-free, transductive) channels over the
+  pooled support+query rows, so the query rows are needed at prediction time (as with TabPFN).
+- `n_estimators` is the random-feature ensemble size (averaged decorrelated seed-variants of `φ`);
+  `n_estimators=1` is the single map, `6` is the paper default (variance reduction toward the kernel limit).
+- `cat_features=[...]` marks categorical columns (indices or DataFrame names); omitted → auto-detected.
+- No class-count ceiling (unlike TabPFN-v2's ≤10 classes); runs on CPU in milliseconds.
+
+## What's inside
+```
+tabmap/
+  _channels.py   raw rows  -> input channels (col-z, rank-gauss, count-encoding, categorical mask), nan-safe
+  _genome.py     evaluate the evolved genome (champion.yaml) -> feature matrix Phi; seed-variants for the ensemble
+  _ridge.py      closed-form Bayesian-ridge head (evidence-maximized lambda), single SVD solve
+  estimator.py   TabMapClassifier / TabMapRegressor (sklearn API) + K-seed ensemble
+  champion.yaml  the evolved 16-family genome (the deployment artifact)
+examples/quickstart.py
+reproduce/        scripts + cached TabPFN-v2 predictions to reproduce the paper's experiments
+tests/
+```
+
+## Reproducing the paper
+See [`reproduce/README.md`](reproduce/README.md). The cached TabPFN-v2 cloud predictions are included
+so the head-to-head and routing experiments reproduce **without** any API key.
+
+## Contributing this method upstream
+`tabmap` is designed to drop into the tabular ML ecosystem. Best integration targets (most aligned first):
+
+| Repo | Why it fits | Integration |
+|---|---|---|
+| **PriorLabs/tabpfn-extensions** | community extensions around TabPFN; our method is a free/local **complementary** in-context learner and a natural **cost-aware router** companion (route hard datasets to TabPFN, the rest to `tabmap`) | add as an extension module + a routing utility (`sklearn`-compatible) |
+| **scikit-learn-contrib** | `TabMapClassifier`/`TabMapRegressor` already follow the estimator API | publish as a standalone `scikit-learn-contrib` project |
+| **skrub** (ex dirty-cat) | tabular feature engineering / encoders; our channels (rank-gauss, count-encoding) + `φ` are a drop-in `TransformerMixin` featurizer | contribute `TabMapEncoder` (transform-only) |
+| **pyg-team/pytorch-frame** | deep tabular; `φ` is a fixed featurizer usable as an input stem | add as an `encoder`/`stype` transform |
+| **autogluon / TabArena** | leaderboard model implementations | submit `tabmap` as a model for the TabArena living benchmark |
+
+The estimator's sklearn-compatible surface (`fit`/`predict`/`predict_proba`, `get_params`) is the
+contribution-ready API; the transform-only `build_channels`+`build_phi` path serves the encoder use-cases.
+
+
+## Combining with a foundation model (e.g. TabPFN)
+`StackedTabularEnsemble` combines TabMap with any in-context base model (such as TabPFN's client) into a
+single, stronger predictor -- the paper's complementarity result (our map tends to win classification,
+TabPFN regression; combining beats either alone). Three methods: `blend` (50/50), `compwt`
+(label-free, weight each model by its support-cross-validated competence), `meta` (a learned ridge head
+over the models' out-of-fold support predictions; most robust). All are leakage-safe and in-context
+(weights/head fit on support, no query labels).
+
+```python
+from evoforest_tab import TabMapClassifier, StackedTabularEnsemble
+from tabpfn_client import TabPFNClassifier            # or any sklearn-surface in-context model
+
+ens = StackedTabularEnsemble(
+        [TabMapClassifier(n_estimators=6), TabPFNClassifier()],
+        task="classification", method="meta",          # "meta" | "compwt" | "blend"
+      ).fit(X_support, y_support)
+proba = ens.predict_proba(X_query)
+```
+The learned head (`meta`) is robust whether the two models are evenly matched or one dominates; the
+label-free `compwt` is a close, deployable second with no meta-learner. See `examples/combine_tabpfn.py`.
+
+## Citation
+If you use this library, please cite the accompanying paper *"Evolving a Universal Tabular Feature Map:
+Interpretable, Closed-Form In-Context Learning Competitive with Tabular Foundation Models."* (anonymized
+for review; see `../tabular_paper/`).
+
+## License
+MIT (see `LICENSE`).

evoforest_tab-0.1.0/README.md

@@ -0,0 +1,100 @@
+# tabmap — EvoForest-Tab: an evolved universal tabular feature map
+
+`tabmap` is the reference implementation of **EvoForest-Tab** (the EvoForest computation-search framework specialized to tabular data).
+
+`tabmap` is an interpretable, training-free, **local** in-context learner for tabular data: an
+evolved universal feature map `φ: row → ℝᴷ` (16 transform families over rank-gauss, count-encoding,
+and categorical-mask channels) paired with a per-dataset **closed-form Bayesian-ridge head**. Given a
+labeled *support* set and an unlabeled *query* set, it predicts in a single SVD solve — no gradient
+descent, no per-dataset tuning, no GPU. It is competitive with gradient boosting and with the
+published **TabPFN-v2** tabular foundation model, while remaining free to run and fully inspectable.
+
+This repository accompanies the paper *"Evolving a Universal Tabular Feature Map: Interpretable,
+Closed-Form In-Context Learning Competitive with Tabular Foundation Models"* and is **stand-alone**:
+the deployment pipeline (feature map + ridge) depends only on `torch`, `numpy`, and `pyyaml`.
+
+## Install
+```bash
+pip install -e .            # editable; or: pip install .
+# deps: torch, numpy, pyyaml  (+ scikit-learn for the estimator base classes & examples)
+```
+
+## Usage (scikit-learn style)
+```python
+from evoforest_tab import TabMapClassifier, TabMapRegressor
+
+clf = TabMapClassifier(n_estimators=6).fit(X_support, y_support)   # X: ndarray or DataFrame
+proba = clf.predict_proba(X_query)                                  # in-context: query needed to fit φ channels
+pred  = clf.predict(X_query)
+
+reg = TabMapRegressor(n_estimators=6).fit(X_support, y_support)
+yhat = reg.predict(X_query)
+```
+Notes:
+- It is an **in-context** learner: `predict` builds the (label-free, transductive) channels over the
+  pooled support+query rows, so the query rows are needed at prediction time (as with TabPFN).
+- `n_estimators` is the random-feature ensemble size (averaged decorrelated seed-variants of `φ`);
+  `n_estimators=1` is the single map, `6` is the paper default (variance reduction toward the kernel limit).
+- `cat_features=[...]` marks categorical columns (indices or DataFrame names); omitted → auto-detected.
+- No class-count ceiling (unlike TabPFN-v2's ≤10 classes); runs on CPU in milliseconds.
+
+## What's inside
+```
+tabmap/
+  _channels.py   raw rows  -> input channels (col-z, rank-gauss, count-encoding, categorical mask), nan-safe
+  _genome.py     evaluate the evolved genome (champion.yaml) -> feature matrix Phi; seed-variants for the ensemble
+  _ridge.py      closed-form Bayesian-ridge head (evidence-maximized lambda), single SVD solve
+  estimator.py   TabMapClassifier / TabMapRegressor (sklearn API) + K-seed ensemble
+  champion.yaml  the evolved 16-family genome (the deployment artifact)
+examples/quickstart.py
+reproduce/        scripts + cached TabPFN-v2 predictions to reproduce the paper's experiments
+tests/
+```
+
+## Reproducing the paper
+See [`reproduce/README.md`](reproduce/README.md). The cached TabPFN-v2 cloud predictions are included
+so the head-to-head and routing experiments reproduce **without** any API key.
+
+## Contributing this method upstream
+`tabmap` is designed to drop into the tabular ML ecosystem. Best integration targets (most aligned first):
+
+| Repo | Why it fits | Integration |
+|---|---|---|
+| **PriorLabs/tabpfn-extensions** | community extensions around TabPFN; our method is a free/local **complementary** in-context learner and a natural **cost-aware router** companion (route hard datasets to TabPFN, the rest to `tabmap`) | add as an extension module + a routing utility (`sklearn`-compatible) |
+| **scikit-learn-contrib** | `TabMapClassifier`/`TabMapRegressor` already follow the estimator API | publish as a standalone `scikit-learn-contrib` project |
+| **skrub** (ex dirty-cat) | tabular feature engineering / encoders; our channels (rank-gauss, count-encoding) + `φ` are a drop-in `TransformerMixin` featurizer | contribute `TabMapEncoder` (transform-only) |
+| **pyg-team/pytorch-frame** | deep tabular; `φ` is a fixed featurizer usable as an input stem | add as an `encoder`/`stype` transform |
+| **autogluon / TabArena** | leaderboard model implementations | submit `tabmap` as a model for the TabArena living benchmark |
+
+The estimator's sklearn-compatible surface (`fit`/`predict`/`predict_proba`, `get_params`) is the
+contribution-ready API; the transform-only `build_channels`+`build_phi` path serves the encoder use-cases.
+
+
+## Combining with a foundation model (e.g. TabPFN)
+`StackedTabularEnsemble` combines TabMap with any in-context base model (such as TabPFN's client) into a
+single, stronger predictor -- the paper's complementarity result (our map tends to win classification,
+TabPFN regression; combining beats either alone). Three methods: `blend` (50/50), `compwt`
+(label-free, weight each model by its support-cross-validated competence), `meta` (a learned ridge head
+over the models' out-of-fold support predictions; most robust). All are leakage-safe and in-context
+(weights/head fit on support, no query labels).
+
+```python
+from evoforest_tab import TabMapClassifier, StackedTabularEnsemble
+from tabpfn_client import TabPFNClassifier            # or any sklearn-surface in-context model
+
+ens = StackedTabularEnsemble(
+        [TabMapClassifier(n_estimators=6), TabPFNClassifier()],
+        task="classification", method="meta",          # "meta" | "compwt" | "blend"
+      ).fit(X_support, y_support)
+proba = ens.predict_proba(X_query)
+```
+The learned head (`meta`) is robust whether the two models are evenly matched or one dominates; the
+label-free `compwt` is a close, deployable second with no meta-learner. See `examples/combine_tabpfn.py`.
+
+## Citation
+If you use this library, please cite the accompanying paper *"Evolving a Universal Tabular Feature Map:
+Interpretable, Closed-Form In-Context Learning Competitive with Tabular Foundation Models."* (anonymized
+for review; see `../tabular_paper/`).
+
+## License
+MIT (see `LICENSE`).

evoforest_tab-0.1.0/evoforest_tab/__init__.py

@@ -0,0 +1,28 @@
+"""evoforest_tab: EvoForest-Tab -- an evolved universal tabular feature map + closed-form ridge head.
+
+An interpretable, training-free, local in-context learner competitive with tabular foundation
+models. See README. Main entry points:
+
+    from evoforest_tab import EvoForestTabClassifier, EvoForestTabRegressor
+"""
+from .estimator import TabMapClassifier, TabMapRegressor
+from ._genome import load_genome, build_phi, seed_variant, DEFAULT_GENOME
+from ._channels import build_channels
+from .combine import StackedTabularEnsemble
+from ._module import EvoForestTabModule
+from .inductive import (
+    EvoForestTabInductiveClassifier, EvoForestTabInductiveRegressor, EvoForestTabTransformer,
+    TabMapInductiveClassifier, TabMapInductiveRegressor, TabMapTransformer,
+)
+
+# brand-consistent names matching the paper (the TabMap* names remain as aliases)
+EvoForestTabClassifier = TabMapClassifier
+EvoForestTabRegressor = TabMapRegressor
+
+__all__ = ["EvoForestTabClassifier", "EvoForestTabRegressor",
+           "TabMapClassifier", "TabMapRegressor", "build_channels",
+           "build_phi", "load_genome", "seed_variant", "DEFAULT_GENOME",
+           "StackedTabularEnsemble", "EvoForestTabModule",
+           "EvoForestTabInductiveClassifier", "EvoForestTabInductiveRegressor", "EvoForestTabTransformer",
+           "TabMapInductiveClassifier", "TabMapInductiveRegressor", "TabMapTransformer"]
+__version__ = "0.1.0"

evoforest_tab-0.1.0/evoforest_tab/_channels.py

@@ -0,0 +1,122 @@
+"""Channel construction: raw table rows -> the input channels the feature map reads.
+
+Faithful to the development pipeline. All channels are UNSUPERVISED and TRANSDUCTIVE (computed over the
+pooled support+query rows, label-free) so the map is leakage-safe in the in-context (support->query)
+setting. Categoricals are ordinal-encoded; missing values are nan-safe; columns are padded/capped to Dmax.
+"""
+import math
+
+import numpy as np
+import torch
+
+DMAX_DEFAULT = 100
+
+
+def _nan_col_zscore(X: torch.Tensor) -> torch.Tensor:
+    """Per-column z-score (nan-safe); nan -> 0 after standardizing."""
+    nan = torch.isnan(X)
+    Xf = torch.where(nan, torch.zeros_like(X), X)
+    cnt = (~nan).sum(0).clamp(min=1).to(X.dtype)
+    mean = Xf.sum(0) / cnt
+    var = (torch.where(nan, torch.zeros_like(X), (X - mean) ** 2)).sum(0) / cnt
+    z = (X - mean) / (var.sqrt() + 1e-6)
+    return torch.where(nan, torch.zeros_like(z), z)
+
+
+def _col_rankgauss(X: torch.Tensor) -> torch.Tensor:
+    """Per-column rank -> ~N(0,1) (nan ranked last, then zeroed). Transductive, label-free."""
+    N = X.shape[0]
+    filled = torch.where(torch.isnan(X), torch.full_like(X, float("inf")), X)
+    ranks = filled.argsort(0).argsort(0).to(X.dtype)
+    u = (ranks + 0.5) / N
+    z = math.sqrt(2.0) * torch.erfinv((2 * u - 1).clamp(-1 + 1e-6, 1 - 1e-6))
+    return torch.where(torch.isnan(X), torch.zeros_like(z), z)
+
+
+def _col_freq(Xm: np.ndarray) -> np.ndarray:
+    """Per-column count/frequency encoding: each cell -> fraction of rows sharing its value."""
+    n, D = Xm.shape
+    Fr = np.zeros_like(Xm, dtype=np.float64)
+    for j in range(D):
+        _, inv, counts = np.unique(Xm[:, j], return_inverse=True, return_counts=True)
+        Fr[:, j] = counts[inv] / n
+    return Fr
+
+
+def _pad(X: torch.Tensor, Dmax: int) -> torch.Tensor:
+    if X.shape[1] >= Dmax:
+        return X[:, :Dmax]
+    return torch.cat([X, torch.zeros(X.shape[0], Dmax - X.shape[1], dtype=X.dtype)], dim=1)
+
+
+def _to_ordinal(X, cat_features):
+    """Coerce a 2-D array (numpy object/float or pandas DataFrame) to a float matrix with categoricals
+    ordinal-encoded (nan preserved), returning (Xm float64, cat_mask bool)."""
+    try:
+        import pandas as pd
+        is_df = isinstance(X, pd.DataFrame)
+    except ImportError:
+        is_df = False
+    if is_df:
+        cols, cat = [], []
+        for ci, c in enumerate(X.columns):
+            s = X[c]
+            auto_cat = str(s.dtype) in ("category", "object", "bool")
+            user_cat = cat_features is not None and (ci in cat_features or c in cat_features)
+            if auto_cat or user_cat:
+                codes = s.astype("category").cat.codes.to_numpy().astype(np.float64)
+                codes[codes < 0] = np.nan
+                cols.append(codes); cat.append(True)
+            else:
+                cols.append(s.to_numpy(dtype=np.float64)); cat.append(False)
+        return np.column_stack(cols), np.array(cat, dtype=bool)
+    Xm = np.asarray(X, dtype=object)
+    n, D = Xm.shape
+    out = np.zeros((n, D), dtype=np.float64)
+    cat = np.zeros(D, dtype=bool)
+    for j in range(D):
+        col = Xm[:, j]
+        user_cat = cat_features is not None and j in cat_features
+        is_numeric = np.issubdtype(np.asarray(col).dtype, np.number)
+        try:
+            fcol = col.astype(np.float64)
+            numeric_ok = True
+        except (ValueError, TypeError):
+            numeric_ok = False
+        if user_cat or not numeric_ok or (not is_numeric):
+            uniq = {v: i for i, v in enumerate(sorted(set(map(str, col))))}
+            out[:, j] = np.array([uniq[str(v)] for v in col], dtype=np.float64)
+            cat[j] = True
+        else:
+            out[:, j] = fcol
+    return out, cat
+
+
+def build_channels(X, cat_features=None, Dmax: int = DMAX_DEFAULT, device="cpu"):
+    """Build the input-channel dict from raw rows X (n, d). Returns tensors padded to Dmax.
+
+    X: numpy array or pandas DataFrame (the POOLED support+query rows).
+    cat_features: indices (or names) of categorical columns; if None, auto-detect by dtype/cardinality.
+    """
+    Xm, cat = _to_ordinal(X, cat_features)
+    # median-impute, drop constant columns (matches the dev pipeline)
+    med = np.nanmedian(Xm, axis=0)
+    inds = np.where(np.isnan(Xm))
+    Xm[inds] = np.take(med, inds[1])
+    keep = Xm.std(0) > 1e-9
+    Xm = Xm[:, keep]; cat = cat[keep]
+    if Xm.shape[1] == 0:
+        raise ValueError("no non-constant columns after preprocessing")
+    # cap to Dmax by top variance
+    if Xm.shape[1] > Dmax:
+        top = np.argsort(-Xm.std(0))[:Dmax]
+        Xm = Xm[:, top]; cat = cat[top]
+    n, D = Xm.shape
+    Xt = torch.from_numpy(Xm).float().to(device)
+    x = _pad(_nan_col_zscore(Xt), Dmax)
+    xrank = _pad(_col_rankgauss(Xt), Dmax)
+    x_freq = _pad(_nan_col_zscore(torch.from_numpy(_col_freq(Xm)).float().to(device)), Dmax)
+    fmask = torch.zeros(n, Dmax, device=device); fmask[:, :min(D, Dmax)] = 1.0
+    is_cat = torch.zeros(n, Dmax, device=device)
+    is_cat[:, :min(D, Dmax)] = torch.from_numpy(cat[:min(D, Dmax)].astype(np.float32)).to(device)
+    return {"x": x, "xrank": xrank, "fmask": fmask, "is_cat": is_cat, "x_freq": x_freq, "Dmax": Dmax}

evoforest_tab-0.1.0/evoforest_tab/_genome.py

@@ -0,0 +1,64 @@
+"""Evaluate the evolved genome (a list of feature-lambdas) on the input channels to produce the
+feature matrix Phi. The genome is the deployment artifact; each lambda is a small, inspectable
+expression over the channels with fixed seeded random projections."""
+import math
+import os
+import re
+
+import numpy as np
+import torch
+import torch.nn.functional as F
+import yaml
+
+_G = {"torch": torch, "F": F, "np": np, "math": math}
+_HERE = os.path.dirname(os.path.abspath(__file__))
+DEFAULT_GENOME = os.path.join(_HERE, "champion.yaml")
+
+
+def load_genome(path: str = DEFAULT_GENOME) -> dict:
+    return yaml.safe_load(open(path))
+
+
+def seed_variant(genome: dict, offset: int) -> dict:
+    """Offset every manual_seed(N) -> manual_seed(N+offset): a decorrelated random-feature draw of the
+    SAME architecture (used for the variance-reducing ensemble)."""
+    out = [re.sub(r"manual_seed\((\d+)\)", lambda m: f"manual_seed({int(m.group(1)) + offset})", lam)
+           for lam in genome["output"]]
+    g = dict(genome); g["output"] = out
+    return g
+
+
+def build_phi(genome: dict, channels: dict) -> torch.Tensor:
+    """Apply the genome's output lambdas to the channel dict and stack into a normalized (n, K) matrix."""
+    gg = {k: eval(v, _G) for k, v in (genome.get("@globals", {}) or {}).items()}
+    n = channels["x"].shape[0]
+    signals = []
+    for lam in genome["output"]:
+        fn = eval(lam, _G)
+        na = fn.__code__.co_argcount
+        an = fn.__code__.co_varnames[:na]
+        args = [channels if a == "input" else gg if a == "globals" else None for a in an]
+        signals.append(fn(*args))
+    cols = []
+    for sig in signals:
+        if not torch.is_tensor(sig):
+            sig = torch.as_tensor(sig)
+        sig = sig.float()
+        if sig.dim() >= 2 and sig.shape[0] == n and sig.shape[1] > 1:
+            sub = [sig[:, c] for c in range(sig.shape[1])]
+        else:
+            sig = sig.squeeze()
+            if sig.dim() != 1 or sig.shape[0] != n:
+                continue
+            sub = [sig]
+        for col in sub:
+            col = torch.where(torch.isfinite(col), col, torch.zeros_like(col))
+            if col.std() < 1e-12:
+                continue
+            cols.append(col)
+    if not cols:
+        raise ValueError("genome produced no usable feature columns")
+    Phi = torch.stack(cols, dim=1)
+    mu = Phi.mean(0, keepdim=True)
+    sigma = Phi.std(0, keepdim=True).clamp(min=1e-12)
+    return (Phi - mu) / sigma

evoforest_tab-0.1.0/evoforest_tab/_module.py

@@ -0,0 +1,152 @@
+"""Compile the evolved genome (champion.yaml) into a self-contained ``torch.nn.Module``.
+
+The genome is the *architecture spec* produced by EvoForest search. In the reference path each family's
+seeded random projection is re-drawn on every forward via ``torch.randn(..., generator=manual_seed(N))``
+inside an ``eval``'d lambda. That is fine for a frozen feature map but (a) cannot be fine-tuned, (b) re-draws
+on every call, and (c) is not a saveable artifact.
+
+``EvoForestTabModule`` fixes all three: it materializes every seeded ``randn``/``rand`` draw **once** as a
+frozen ``nn.Parameter`` (``requires_grad=False`` by default), so the module is a self-contained, saveable,
+HuggingFace-publishable checkpoint whose ``forward`` is byte-identical to the reference evaluation -- and the
+random features become fine-tunable simply by calling :meth:`unfreeze_random_features`. The deterministic
+families (rank/stats/frequency) carry no parameters and are reproduced exactly.
+"""
+from __future__ import annotations
+
+import math
+
+import numpy as np
+import torch
+import torch.nn.functional as F
+from torch import nn
+
+from ._genome import load_genome, DEFAULT_GENOME
+
+
+class _SeededWeightBank(nn.Module):
+    """Materialize seeded ``torch.randn``/``torch.rand`` draws as frozen Parameters, keyed by (op, seed, shape)."""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.weights = nn.ParameterDict()
+
+    @staticmethod
+    def _key(op: str, shape, seed: int) -> str:
+        return f"{op}__seed{int(seed)}__" + "x".join(str(int(s)) for s in shape)
+
+    def get(self, op: str, shape, seed: int, device=None) -> torch.Tensor:
+        key = self._key(op, shape, seed)
+        if key not in self.weights:
+            gen = torch.Generator().manual_seed(int(seed))            # CPU draw == reference on CPU
+            draw = (torch.randn if op == "randn" else torch.rand)(*shape, generator=gen)
+            self.weights[key] = nn.Parameter(draw, requires_grad=False)  # frozen by default
+        w = self.weights[key]
+        return w.to(device) if device is not None else w
+
+
+class _TorchProxy:
+    """A stand-in for the ``torch`` namespace inside the genome lambdas: seeded ``randn``/``rand`` are routed
+    to the materialized weight bank; every other attribute delegates to real ``torch``."""
+
+    def __init__(self, bank: _SeededWeightBank) -> None:
+        object.__setattr__(self, "_bank", bank)
+
+    def __getattr__(self, name):
+        return getattr(torch, name)
+
+    def randn(self, *shape, generator=None, device=None, dtype=None):
+        if generator is None:
+            return torch.randn(*shape, device=device, dtype=dtype)
+        return self._bank.get("randn", shape, generator.initial_seed(), device)
+
+    def rand(self, *shape, generator=None, device=None, dtype=None):
+        if generator is None:
+            return torch.rand(*shape, device=device, dtype=dtype)
+        return self._bank.get("rand", shape, generator.initial_seed(), device)
+
+
+class EvoForestTabModule(nn.Module):
+    """Evolved EvoForest-Tab feature map phi as a compiled, fine-tunable, saveable ``nn.Module``.
+
+    Parameters
+    ----------
+    genome : dict | None
+        The parsed genome (defaults to the released champion).
+    dmax : int
+        Padded channel width the module is built for (random-projection shapes depend only on this).
+
+    Notes
+    -----
+    ``forward(channels)`` takes the channel dict from :func:`evoforest_tab.build_channels` and returns the
+    normalized ``(n, K)`` feature matrix, byte-identical to :func:`evoforest_tab.build_phi`. Random-feature
+    weights are frozen by default; call :meth:`unfreeze_random_features` to fine-tune them (e.g. at BSC), then
+    ``state_dict()`` is a publishable derivative checkpoint.
+    """
+
+    def __init__(self, genome: dict | None = None, dmax: int = 100) -> None:
+        super().__init__()
+        genome = genome if genome is not None else load_genome(DEFAULT_GENOME)
+        self.output_src = list(genome["output"])
+        self.dmax = dmax
+        self.bank = _SeededWeightBank()
+        self._proxy = _TorchProxy(self.bank)
+        env = {"torch": self._proxy, "F": F, "np": np, "math": math}
+        self._fns = [eval(src, env) for src in self.output_src]                # proxy-bound lambdas
+        with torch.no_grad():                                                  # eager-materialize the bank
+            self._materialize()
+
+    # ------------------------------------------------------------------ build / fine-tune controls
+    def _materialize(self) -> None:
+        dummy = self._dummy_channels()
+        for fn in self._fns:
+            try:
+                fn(dummy)                       # fires the seeded randn/rand -> banks the weights
+            except Exception:                   # noqa: BLE001 - dummy may degenerate post-draw; weights are banked
+                pass
+
+    def _dummy_channels(self) -> dict:
+        z = torch.randn(4, self.dmax)
+        return {"x": z, "xrank": z.clone(), "x_freq": z.clone(),
+                "fmask": torch.ones(4, self.dmax), "is_cat": torch.zeros(4, self.dmax), "Dmax": self.dmax}
+
+    def unfreeze_random_features(self) -> "EvoForestTabModule":
+        for p in self.bank.weights.values():
+            p.requires_grad_(True)
+        return self
+
+    def freeze_random_features(self) -> "EvoForestTabModule":
+        for p in self.bank.weights.values():
+            p.requires_grad_(False)
+        return self
+
+    @property
+    def n_random_parameters(self) -> int:
+        return sum(p.numel() for p in self.bank.weights.values())
+
+    # ------------------------------------------------------------------ forward (mirrors build_phi exactly)
+    def forward(self, channels: dict) -> torch.Tensor:
+        n = channels["x"].shape[0]
+        cols = []
+        for fn in self._fns:
+            sig = fn(channels)
+            if not torch.is_tensor(sig):
+                sig = torch.as_tensor(sig)
+            sig = sig.float()
+            if sig.dim() >= 2 and sig.shape[0] == n and sig.shape[1] > 1:
+                sub = [sig[:, c] for c in range(sig.shape[1])]
+            else:
+                sig = sig.squeeze()
+                if sig.dim() != 1 or sig.shape[0] != n:
+                    continue
+                sub = [sig]
+            for col in sub:
+                col = torch.where(torch.isfinite(col), col, torch.zeros_like(col))
+                if col.std() < 1e-12:
+                    continue
+                cols.append(col)
+        if not cols:
+            raise ValueError("genome produced no usable feature columns")
+        phi = torch.stack(cols, dim=1)
+        mu = phi.mean(0, keepdim=True)
+        sigma = phi.std(0, keepdim=True).clamp(min=1e-12)
+        return (phi - mu) / sigma

evoforest_tab-0.1.0/evoforest_tab/_ridge.py

@@ -0,0 +1,45 @@
+"""Closed-form ridge head with Bayesian evidence-maximized regularization (MacKay/Tipping).
+
+The head is fit on the support features and applied to the query features in a single SVD solve --
+no gradient descent, no per-dataset hyperparameter grid. lambda is set by evidence maximization,
+which is markedly more stable than a leave-one-out grid in the few-shot (K>>n) regime.
+"""
+import torch
+
+LAM_FLOOR = 1e-2
+
+
+def evidence_lambda(U, S, Y, n_iter: int = 60) -> float:
+    """Bayesian-ridge lambda = alpha/beta by evidence maximization, reusing the support SVD (U,S),
+    with alpha,beta shared across Y's columns and a floor for the K>n interpolation regime."""
+    s2 = S ** 2
+    UtY = U.transpose(0, 1) @ Y
+    n, M = Y.shape
+    ytot = (Y ** 2).sum(); proj = (UtY ** 2).sum()
+    lam = torch.tensor(1.0, dtype=S.dtype, device=S.device)
+    for _ in range(n_iter):
+        h = s2 / (s2 + lam)
+        d = S / (s2 + lam)
+        wsq = ((d.unsqueeze(1) * UtY) ** 2).sum()
+        rss = (ytot - proj) + (((1 - h).unsqueeze(1) * UtY) ** 2).sum()
+        gamma = h.sum()
+        alpha = (M * gamma) / (wsq + 1e-12)
+        beta = (M * n - M * gamma) / (rss + 1e-12)
+        lam_new = (alpha / (beta + 1e-12)).clamp(min=LAM_FLOOR, max=1e8)
+        if (lam_new - lam).abs() < 1e-3 * lam:
+            lam = lam_new
+            break
+        lam = lam_new
+    return float(lam)
+
+
+def ridge_scores(Phi_s, Y, Phi_q, lam=None):
+    """Standardize by support stats, solve ridge in closed form, return query scores (nq, M)."""
+    mu = Phi_s.mean(0, keepdim=True); sd = Phi_s.std(0, keepdim=True).clamp(min=1e-8)
+    Phi_s = (Phi_s - mu) / sd; Phi_q = (Phi_q - mu) / sd
+    U, S, Vt = torch.linalg.svd(Phi_s, full_matrices=False)
+    UY = U.transpose(0, 1) @ Y
+    if lam is None:
+        lam = evidence_lambda(U, S, Y)
+    W = Vt.transpose(0, 1) @ ((S / (S ** 2 + lam)).unsqueeze(1) * UY)
+    return Phi_q @ W