lighttar 0.1.0__tar.gz

lighttar-0.1.0/.gitignore

@@ -0,0 +1,8 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+.pytest_cache/
+.ruff_cache/
+build/
+dist/

lighttar-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Eilam Shapira
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

lighttar-0.1.0/PKG-INFO

@@ -0,0 +1,114 @@
+Metadata-Version: 2.5
+Name: lighttar
+Version: 0.1.0
+Summary: LightTAR: target-aware tabular features from frozen encoder layers, in closed form. A scikit-learn step for tabular models.
+Project-URL: Homepage, https://github.com/eilamshapira/LightTAR
+Project-URL: Repository, https://github.com/eilamshapira/LightTAR
+Project-URL: Issues, https://github.com/eilamshapira/LightTAR/issues
+Author-email: Eilam Shapira <eilamshapira@campus.technion.ac.il>
+Maintainer-email: Eilam Shapira <eilamshapira@campus.technion.ac.il>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: feature extraction,in-context learning,partial least squares,scikit-learn,tabular,tabular foundation models,text embeddings
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.22
+Requires-Dist: pandas>=1.5
+Requires-Dist: scikit-learn>=1.2
+Provides-Extra: all
+Requires-Dist: lightgbm>=4.0; extra == 'all'
+Requires-Dist: pillow>=9; extra == 'all'
+Requires-Dist: torch>=2.0; extra == 'all'
+Requires-Dist: transformers>=4.30; extra == 'all'
+Requires-Dist: transformers>=4.56; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: lightgbm>=4.0; extra == 'dev'
+Requires-Dist: pillow>=9; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Requires-Dist: torch>=2.0; extra == 'dev'
+Requires-Dist: transformers>=4.30; extra == 'dev'
+Requires-Dist: transformers>=4.56; extra == 'dev'
+Provides-Extra: gate
+Requires-Dist: lightgbm>=4.0; extra == 'gate'
+Provides-Extra: image
+Requires-Dist: pillow>=9; extra == 'image'
+Requires-Dist: torch>=2.0; extra == 'image'
+Requires-Dist: transformers>=4.56; extra == 'image'
+Provides-Extra: text
+Requires-Dist: torch>=2.0; extra == 'text'
+Requires-Dist: transformers>=4.30; extra == 'text'
+Description-Content-Type: text/markdown
+
+# LightTAR
+
+LightTAR replaces text or image columns in a pandas DataFrame with numeric features for a tabular model. By default, each selected column contributes up to 30 PCA features from the last encoder layer and 5 supervised features learned from the last and middle layers. The encoder stays frozen.
+
+## Install
+
+Requires Python 3.10 or later. For the text example below, install `lighttar[text]`.
+
+| Use case | Command |
+| --- | --- |
+| Precomputed embeddings | `pip install lighttar` |
+| Text columns | `pip install "lighttar[text]"` |
+| Image columns | `pip install "lighttar[image]"` |
+| Text, images, and the LightGBM gate | `pip install "lighttar[all]"` |
+
+To use the LightGBM gate with precomputed embeddings, run `pip install "lighttar[gate]"`. You can also combine extras, for example `pip install "lighttar[text,gate]"`.
+
+## Text
+
+The first run downloads the text encoder (`intfloat/e5-small-v2`). Pass the text column names explicitly, or use `LightTAR()` to detect them. Other columns pass through unchanged.
+
+```python
+import pandas as pd
+from sklearn.linear_model import LogisticRegression
+from sklearn.pipeline import make_pipeline
+
+from lighttar import LightTAR
+
+train = pd.DataFrame({"review": ["Very helpful", "Hard to use", "Works well", "Broke immediately"]})
+test = pd.DataFrame({"review": ["Easy to use", "Stopped working"]})
+
+model = make_pipeline(LightTAR(columns=["review"]), LogisticRegression())
+model.fit(train, [1, 0, 1, 0])
+predictions = model.predict(test)
+```
+
+## Images
+
+For a column of image filenames, use this transformer in the same pipeline. The default DINOv3 model requires access on Hugging Face.
+
+```python
+from lighttar import ImageLayerEncoder, LightTAR
+
+features = LightTAR(columns=["photo"], encoder=ImageLayerEncoder(image_folder="images"))
+```
+
+## Precomputed embeddings
+
+Pass one array per split with shape `(rows, layers, embedding_dim)`, with the last encoder layer first. A single layer can also be passed as `(rows, embedding_dim)`.
+
+```python
+from lighttar import LightTARProjection
+
+projection = LightTARProjection()
+Z_train = projection.fit_transform(H_train, y_train)
+Z_test = projection.transform(H_test)
+```
+
+## Optional gate
+
+Use `GatedLightTAR` in place of `LightTAR` to choose between LightTAR and PCA using cross-validation. After fitting, `gate_decision_` records the choice.

lighttar-0.1.0/README.md

@@ -0,0 +1,61 @@
+# LightTAR
+
+LightTAR replaces text or image columns in a pandas DataFrame with numeric features for a tabular model. By default, each selected column contributes up to 30 PCA features from the last encoder layer and 5 supervised features learned from the last and middle layers. The encoder stays frozen.
+
+## Install
+
+Requires Python 3.10 or later. For the text example below, install `lighttar[text]`.
+
+| Use case | Command |
+| --- | --- |
+| Precomputed embeddings | `pip install lighttar` |
+| Text columns | `pip install "lighttar[text]"` |
+| Image columns | `pip install "lighttar[image]"` |
+| Text, images, and the LightGBM gate | `pip install "lighttar[all]"` |
+
+To use the LightGBM gate with precomputed embeddings, run `pip install "lighttar[gate]"`. You can also combine extras, for example `pip install "lighttar[text,gate]"`.
+
+## Text
+
+The first run downloads the text encoder (`intfloat/e5-small-v2`). Pass the text column names explicitly, or use `LightTAR()` to detect them. Other columns pass through unchanged.
+
+```python
+import pandas as pd
+from sklearn.linear_model import LogisticRegression
+from sklearn.pipeline import make_pipeline
+
+from lighttar import LightTAR
+
+train = pd.DataFrame({"review": ["Very helpful", "Hard to use", "Works well", "Broke immediately"]})
+test = pd.DataFrame({"review": ["Easy to use", "Stopped working"]})
+
+model = make_pipeline(LightTAR(columns=["review"]), LogisticRegression())
+model.fit(train, [1, 0, 1, 0])
+predictions = model.predict(test)
+```
+
+## Images
+
+For a column of image filenames, use this transformer in the same pipeline. The default DINOv3 model requires access on Hugging Face.
+
+```python
+from lighttar import ImageLayerEncoder, LightTAR
+
+features = LightTAR(columns=["photo"], encoder=ImageLayerEncoder(image_folder="images"))
+```
+
+## Precomputed embeddings
+
+Pass one array per split with shape `(rows, layers, embedding_dim)`, with the last encoder layer first. A single layer can also be passed as `(rows, embedding_dim)`.
+
+```python
+from lighttar import LightTARProjection
+
+projection = LightTARProjection()
+Z_train = projection.fit_transform(H_train, y_train)
+Z_test = projection.transform(H_test)
+```
+
+## Optional gate
+
+Use `GatedLightTAR` in place of `LightTAR` to choose between LightTAR and PCA using cross-validation. After fitting, `gate_decision_` records the choice.

lighttar-0.1.0/pyproject.toml

@@ -0,0 +1,67 @@
+[build-system]
+requires = ["hatchling>=1.27"]
+build-backend = "hatchling.build"
+
+[project]
+name = "lighttar"
+dynamic = ["version"]
+description = "LightTAR: target-aware tabular features from frozen encoder layers, in closed form. A scikit-learn step for tabular models."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Eilam Shapira", email = "eilamshapira@campus.technion.ac.il" }]
+maintainers = [{ name = "Eilam Shapira", email = "eilamshapira@campus.technion.ac.il" }]
+keywords = [
+    "tabular", "text embeddings", "partial least squares", "feature extraction",
+    "in-context learning", "tabular foundation models", "scikit-learn",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Science/Research",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Typing :: Typed",
+]
+dependencies = ["numpy>=1.22", "pandas>=1.5", "scikit-learn>=1.2"]
+
+[project.optional-dependencies]
+text = ["torch>=2.0", "transformers>=4.30"]
+image = ["torch>=2.0", "transformers>=4.56", "Pillow>=9"]
+gate = ["lightgbm>=4.0"]
+all = ["lighttar[text,image,gate]"]
+dev = ["lighttar[all]", "pytest>=7", "ruff>=0.6", "build>=1.2"]
+
+[project.urls]
+Homepage = "https://github.com/eilamshapira/LightTAR"
+Repository = "https://github.com/eilamshapira/LightTAR"
+Issues = "https://github.com/eilamshapira/LightTAR/issues"
+
+[tool.hatch.version]
+path = "src/lighttar/_version.py"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/lighttar"]
+
+[tool.hatch.build.targets.sdist]
+include = ["/src/lighttar", "/tests", "/README.md", "/LICENSE", "/pyproject.toml"]
+exclude = ["**/__pycache__"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-ra"
+
+[tool.ruff]
+line-length = 120
+target-version = "py310"
+src = ["src", "tests"]
+
+[tool.ruff.lint]
+select = ["E", "F", "W", "B", "I", "UP"]
+ignore = []

lighttar-0.1.0/src/lighttar/__init__.py

@@ -0,0 +1,19 @@
+"""LightTAR: target-aware tabular features from frozen encoder layers, in closed form."""
+
+from ._version import __version__
+from .core import LightTARProjection
+from .encoder import LayerEncoder, PrecomputedEncoder
+from .image import ImageLayerEncoder, detect_image_columns
+from .table import GatedLightTAR, LightTAR, detect_text_columns
+
+__all__ = [
+    "LightTAR",
+    "GatedLightTAR",
+    "LightTARProjection",
+    "LayerEncoder",
+    "PrecomputedEncoder",
+    "ImageLayerEncoder",
+    "detect_text_columns",
+    "detect_image_columns",
+    "__version__",
+]

lighttar-0.1.0/src/lighttar/_arms.py

@@ -0,0 +1,93 @@
+"""The two arms of ``GatedLightTAR``: the frozen pipeline (raw last layer -> PCA-30) and LightTAR from
+moment sums shared across the inner folds."""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass
+
+import numpy as np
+from sklearn.base import BaseEstimator, TransformerMixin
+
+from .core import _moment_sums, _split_layers, raw_target
+
+BASELINE = "frozen"
+CANDIDATE = "lighttar"
+FROZEN_DIM = 30  # the frozen baseline's PCA dimension
+FROZEN_SEED = 42  # the frozen baseline's PCA seed (the SEED of MulTaBench's frozen e5 + PCA-30 pipeline)
+
+
+def _first_layer32(H) -> np.ndarray:
+    """The first layer of ``(n, d)``, ``(n, L, d)`` or a list of layers, as a float32 numpy array."""
+    return np.asarray(_split_layers(H, as_numpy=True)[0][0], dtype=np.float32)
+
+
+class FrozenPCA(BaseEstimator, TransformerMixin):
+    """The standard frozen pipeline on one column: scikit-learn PCA-30 of the raw first layer (the encoder's
+    last layer by default), fitted and applied on float32 states exactly as the upstream pipeline does
+    (a float64 fit rotates near-degenerate trailing components)."""
+
+    def fit(self, H, y=None) -> FrozenPCA:
+        from sklearn.decomposition import PCA
+
+        X = _first_layer32(H)
+        self.n_components_ = max(1, min(FROZEN_DIM, X.shape[1], len(X) - 1))
+        self.pca_ = PCA(n_components=self.n_components_, random_state=FROZEN_SEED).fit(X)
+        return self
+
+    def transform(self, H) -> np.ndarray:
+        return self.pca_.transform(_first_layer32(H)).astype(np.float32, copy=False)
+
+
+@dataclass
+class ColumnStates:
+    """One column's float64 layers, its float32 first layer, and the moment sums of all its rows (shared by
+    the inner folds)."""
+
+    mats: list
+    first32: np.ndarray
+    Y: np.ndarray
+    classes: np.ndarray | None
+    total: np.ndarray
+
+    def rows(self, idx) -> list:
+        return [m[idx] for m in self.mats]
+
+
+def _frozen_arm(cs: ColumnStates, tr, va, head) -> tuple[np.ndarray, np.ndarray]:
+    fz = FrozenPCA().fit(cs.first32[tr])
+    return fz.transform(cs.first32[tr]), fz.transform(cs.first32[va])
+
+
+def _lighttar_arm(cs: ColumnStates, tr, va, head) -> tuple[np.ndarray, np.ndarray]:
+    """The projection of the inner-train rows from shared sums: all rows minus the inner-val part."""
+    A = cs.total - _moment_sums(cs.rows(va), cs.Y[va])
+    proj = head()._fit_from_sums(A, [m.shape[1] for m in cs.mats], cs.classes)
+    return proj.transform(cs.rows(tr)), proj.transform(cs.rows(va))
+
+
+ARMS: dict[str, Callable] = {BASELINE: _frozen_arm, CANDIDATE: _lighttar_arm}
+
+
+def prepare_column(H, y, classification: bool) -> ColumnStates:
+    """The per-column work shared by all inner folds: float64 layers, float32 first layer, total sums."""
+    raw = _split_layers(H, as_numpy=True)[0]
+    mats = [np.asarray(m, dtype=np.float64) for m in raw]
+    Y, classes = raw_target(y, classification)
+    if not classification:
+        Y = Y - Y.mean()  # the sums are shift-invariant in y; centering keeps them well conditioned
+    return ColumnStates(mats, np.asarray(raw[0], dtype=np.float32), Y, classes, _moment_sums(mats, Y))
+
+
+def fold_arms(cs: ColumnStates, tr, va, head: Callable) -> dict:
+    """``{arm: (Z_inner_train, Z_inner_val)}`` of one column on one inner fold; ``head()`` makes an unfitted
+    ``LightTARProjection``. The inner-train rows must be the complement of the inner-val rows."""
+    return {arm: fn(cs, tr, va, head) for arm, fn in ARMS.items()}
+
+
+def column_arms(H, y, splits, classification: bool, head: Callable) -> list[dict]:
+    """``fold_arms`` of one column for every inner fold, sequentially."""
+    if not splits:
+        return []
+    cs = prepare_column(H, y, classification)
+    return [fold_arms(cs, tr, va, head) for tr, va in splits]

lighttar-0.1.0/src/lighttar/_columns.py

@@ -0,0 +1,82 @@
+"""Text-column detection and the cell texts handed to an encoder."""
+
+from __future__ import annotations
+
+from numbers import Number
+
+import numpy as np
+import pandas as pd
+
+MIN_TEXT_UNIQUE = 100
+MIN_TEXT_UNIQUE_RATIO = 0.8
+MAX_NUMERIC_FOR_CATEGORICAL = 50
+MISSING_TEXT = "Unknown Value"
+
+
+def _valid(values) -> list:
+    out = []
+    for v in values:
+        if isinstance(v, str):
+            out.append(v)
+        elif v is None or _isna(v) or (isinstance(v, Number) and not np.isfinite(v)):
+            continue
+        else:
+            out.append(v)
+    return out
+
+
+def _isna(v) -> bool:
+    try:
+        return bool(pd.isna(v))
+    except (TypeError, ValueError):
+        return False
+
+
+def _is_numeric_value(v) -> bool:
+    if v is None:
+        return False
+    if isinstance(v, str):
+        return v.isdigit()
+    if isinstance(v, (int, float)):
+        return True
+    try:
+        float(v)
+        return True
+    except (TypeError, ValueError):
+        return False
+
+
+def is_numerical(s) -> bool:
+    """Numerical column: it has valid values and either a numeric (or bool) dtype or more than 50 distinct
+    valid values of which at most one is non-numeric."""
+    from pandas.api.types import is_numeric_dtype
+
+    if is_numeric_dtype(s.dtype):
+        return bool(np.isfinite(s.to_numpy(dtype=float, na_value=np.nan)).any())
+    vals = _valid(s.tolist())
+    if not vals:
+        return False
+    uniq = set(vals)
+    return len(uniq) > MAX_NUMERIC_FOR_CATEGORICAL and sum(not _is_numeric_value(v) for v in uniq) <= 1
+
+
+def is_text(s) -> bool:
+    """A non-numerical column is text if its values as strings (missing values as "Unknown Value") have at
+    least 100 distinct values or a distinct ratio of at least 0.8; otherwise it is categorical."""
+    vals = column_texts(s)
+    n = len(set(vals))
+    return bool(vals) and (n >= MIN_TEXT_UNIQUE or n / len(vals) >= MIN_TEXT_UNIQUE_RATIO)
+
+
+def detect_text_columns(df) -> list:
+    """The text columns of a DataFrame: neither datetime nor ``is_numerical``, and ``is_text``."""
+    from pandas.api.types import is_datetime64_any_dtype
+
+    return [
+        c for c in df.columns if not is_datetime64_any_dtype(df[c].dtype) and not is_numerical(df[c]) and is_text(df[c])
+    ]
+
+
+def column_texts(s) -> list:
+    """Values as strings, with missing values as "Unknown Value"."""
+    return s.astype(object).fillna(MISSING_TEXT).astype(str).tolist()