scm2cpp-lasso 0.5.2__tar.gz → 0.5.3.1__tar.gz

{scm2cpp_lasso-0.5.2 → scm2cpp_lasso-0.5.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.5.2
+Version: 0.5.3.1
 Summary: Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included
 Author: Hirotaka Niitsuma
 License: MIT
@@ -70,9 +70,14 @@ from scm2cpp_lasso import cuda_available
 lambdas = model.lambda_grid(num=100)        # from lambda_max down
 path = model.fit_path(lambdas)              # warm-started, sequential
 grid = model.fit_path_batch(lambdas)        # every lambda from zero
+one = model.fit(lambdas[60])                # a single lambda, if you must
 print("GPU:", cuda_available())
 ```
 
+`fit` is the one-element special case of `fit_path` -- the Gram matrix
+was built in the constructor, so a whole warm-started path costs barely
+more than one penalty, which is why the path is the primary interface.
+
 The objective is scikit-learn's, with `fit_intercept=False`:
 
     (1 / 2 nobs) ||y - X b||^2 + lam ||b||_1
@@ -84,6 +89,58 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Against the other lassos pip can install
+
+`bench/lasso-compare.py` in the repository runs the same
+cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
+p=200, n=1800 -- against every lasso a `pip install` produces:
+scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+Every solver runs at its own default tolerance (ours at the tighter
+1e-8), each gets one untimed warm-up fit, cuML's data is on the device
+before the clock starts, and the last column is each solver's
+objective minus ours at the smallest lambda -- a fast row with a loose
+answer would show as one:
+
+| solver                                 | time    | objective gap |
+|----------------------------------------|---------|---------------|
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
+| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
+| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
+| celer per lambda                       | 17.3 s  | 0             |
+| skglm per lambda                       | 16.4 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+
+celer and skglm are built for very large sparse designs, where their
+screening rules dominate; at this size they pay their setup per fit.
+cuML parallelises *within* one fit, which wins when a fit is large; at
+this size its launch overhead dominates, while our GPU row
+parallelises *across* lambdas -- one CUDA thread per penalty, the axis
+a cross-validation grid actually offers.
+
+`CovLassoCV` is scikit-learn's `LassoCV` over this machinery -- same
+grid construction, same contiguous folds, same minimum-mean-MSE choice
+-- with two structural savings.  A fold's training Gram is a
+*subtraction* (the Gram is additive over rows, so G - Xf'Xf costs one
+fold's work where rebuilding costs four folds'), and once the Grams
+exist nothing downstream ever touches the n rows again -- which is why
+the gap grows with n.  On the GPU every fold and every alpha can run
+as one thread of a single launch; that launch replicates each fold's
+Gram per alpha, so it is chosen only while the replication stays under
+512 MB, the CPU warm path taking over at large p.  Measured at cv=5,
+100 alphas, one CPU core:
+
+| n       | p    | `CovLassoCV` | sklearn `LassoCV` |
+|---------|------|--------------|-------------------|
+| 1,800   | 200  | 0.23 s       | 0.20 s            |
+| 5,000   | 1000 | 1.1 s        | 3.4 s             |
+| 100,000 | 200  | 0.44 s       | 5.7 s             |
+| 100,000 | 500  | 0.90 s       | 8.0 s             |
+
+At n=100,000 the chosen alpha is identical and the coefficients agree
+to 7e-7; at small n a near-tie can leave the picks one grid step
+apart, which sklearn resolves our way once its tolerance is
+tightened.  GPU and CPU agree to 1e-14 wherever both run.
+
 ## Which method
 
 `fit_path` walks a single path, each lambda starting from the previous

{scm2cpp_lasso-0.5.2 → scm2cpp_lasso-0.5.3.1}/README.md

@@ -52,9 +52,14 @@ from scm2cpp_lasso import cuda_available
 lambdas = model.lambda_grid(num=100)        # from lambda_max down
 path = model.fit_path(lambdas)              # warm-started, sequential
 grid = model.fit_path_batch(lambdas)        # every lambda from zero
+one = model.fit(lambdas[60])                # a single lambda, if you must
 print("GPU:", cuda_available())
 ```
 
+`fit` is the one-element special case of `fit_path` -- the Gram matrix
+was built in the constructor, so a whole warm-started path costs barely
+more than one penalty, which is why the path is the primary interface.
+
 The objective is scikit-learn's, with `fit_intercept=False`:
 
     (1 / 2 nobs) ||y - X b||^2 + lam ||b||_1
@@ -66,6 +71,58 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Against the other lassos pip can install
+
+`bench/lasso-compare.py` in the repository runs the same
+cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
+p=200, n=1800 -- against every lasso a `pip install` produces:
+scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+Every solver runs at its own default tolerance (ours at the tighter
+1e-8), each gets one untimed warm-up fit, cuML's data is on the device
+before the clock starts, and the last column is each solver's
+objective minus ours at the smallest lambda -- a fast row with a loose
+answer would show as one:
+
+| solver                                 | time    | objective gap |
+|----------------------------------------|---------|---------------|
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
+| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
+| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
+| celer per lambda                       | 17.3 s  | 0             |
+| skglm per lambda                       | 16.4 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+
+celer and skglm are built for very large sparse designs, where their
+screening rules dominate; at this size they pay their setup per fit.
+cuML parallelises *within* one fit, which wins when a fit is large; at
+this size its launch overhead dominates, while our GPU row
+parallelises *across* lambdas -- one CUDA thread per penalty, the axis
+a cross-validation grid actually offers.
+
+`CovLassoCV` is scikit-learn's `LassoCV` over this machinery -- same
+grid construction, same contiguous folds, same minimum-mean-MSE choice
+-- with two structural savings.  A fold's training Gram is a
+*subtraction* (the Gram is additive over rows, so G - Xf'Xf costs one
+fold's work where rebuilding costs four folds'), and once the Grams
+exist nothing downstream ever touches the n rows again -- which is why
+the gap grows with n.  On the GPU every fold and every alpha can run
+as one thread of a single launch; that launch replicates each fold's
+Gram per alpha, so it is chosen only while the replication stays under
+512 MB, the CPU warm path taking over at large p.  Measured at cv=5,
+100 alphas, one CPU core:
+
+| n       | p    | `CovLassoCV` | sklearn `LassoCV` |
+|---------|------|--------------|-------------------|
+| 1,800   | 200  | 0.23 s       | 0.20 s            |
+| 5,000   | 1000 | 1.1 s        | 3.4 s             |
+| 100,000 | 200  | 0.44 s       | 5.7 s             |
+| 100,000 | 500  | 0.90 s       | 8.0 s             |
+
+At n=100,000 the chosen alpha is identical and the coefficients agree
+to 7e-7; at small n a near-tie can leave the picks one grid step
+apart, which sklearn resolves our way once its tolerance is
+tightened.  GPU and CPU agree to 1e-14 wherever both run.
+
 ## Which method
 
 `fit_path` walks a single path, each lambda starting from the previous

{scm2cpp_lasso-0.5.2 → scm2cpp_lasso-0.5.3.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scm2cpp-lasso"
-version = "0.5.2"
+version = "0.5.3.1"
 description = "Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included"
 readme = "README.md"
 requires-python = ">=3.8"

{scm2cpp_lasso-0.5.2 → scm2cpp_lasso-0.5.3.1}/src/scm2cpp_lasso/__init__.py

@@ -31,7 +31,7 @@ from ._libfind import load_batch_lib
 
 __all__ = ["CovLasso", "CovRidge", "CovLogistic", "CovGroupLasso",
            "cuda_available", "kernel"]
-__version__ = "0.5.2"
+__version__ = "0.5.3.1"
 
 _BATCH = load_batch_lib()
 _DP = ctypes.POINTER(ctypes.c_double)
@@ -257,7 +257,10 @@ def _batch_descend_multi(grams, corrs, lam, nobs, p, tol=1e-8, chunk=20,
     beta = np.zeros((B, p))
     c = np.ascontiguousarray(corrs.copy())
     g = np.ascontiguousarray(grams)
-    lams = np.full(B, float(lam))
+    # lam may be one penalty for every thread or one per thread; the
+    # kernel always reads a per-thread array
+    lams = (np.ascontiguousarray(lam, dtype=np.float64)
+            if np.ndim(lam) else np.full(B, float(lam)))
     if _BATCH is not None and not force_cpu:
         dp = ctypes.POINTER(ctypes.c_double)
         rc = _BATCH.scm2cpp_batch_descend(
@@ -270,11 +273,12 @@ def _batch_descend_multi(grams, corrs, lam, nobs, p, tol=1e-8, chunk=20,
     prev = np.empty(p)
     for b in range(B):
         gb, cb, bb = g[b], c[b], beta[b]
+        lam_b = float(lams[b])
         swept = 0
         while swept < max_sweeps:
             prev[:] = bb
-            kernel_fn(gb, cb, bb, float(lam) * l1_ratio,
-                      float(lam) * (1.0 - l1_ratio), chunk,
+            kernel_fn(gb, cb, bb, lam_b * l1_ratio,
+                      lam_b * (1.0 - l1_ratio), chunk,
                       float(nobs), p)
             swept += chunk
             if np.max(np.abs(bb - prev)) < tol * max(
@@ -283,6 +287,126 @@ def _batch_descend_multi(grams, corrs, lam, nobs, p, tol=1e-8, chunk=20,
     return beta
 
 
+
+class CovLassoCV:
+    """Cross-validated lasso, scikit-learn's LassoCV semantics.
+
+    The lambda grid comes from the full data (lambda_max down, the
+    sklearn construction), the folds are contiguous splits in order
+    (sklearn's KFold default), each fold's whole path is solved from
+    zero -- which is exactly the workload fit_path_batch exists for,
+    one GPU thread per lambda -- and validation error is one matrix
+    product per fold.  alpha_ minimises the mean validation MSE, ties
+    going to the larger penalty as in scikit-learn, and coef_ is the
+    refit on all rows at alpha_.
+
+    Attributes after fit: alphas_, mse_path_ (num, cv), alpha_, coef_.
+    """
+
+    def __init__(self, cv=5, num=100, eps=1e-3, tol=1e-8, l1_ratio=1.0,
+                 force_cpu=False):
+        self.cv = int(cv)
+        self.num = int(num)
+        self.eps = float(eps)
+        self.tol = float(tol)
+        self.l1_ratio = float(l1_ratio)
+        self.force_cpu = force_cpu
+
+    def fit(self, X, y):
+        X = np.ascontiguousarray(X, dtype=np.float64)
+        y = np.ascontiguousarray(y, dtype=np.float64)
+        n = len(y)
+        full = CovLasso(X, y)
+        self.alphas_ = full.lambda_grid(num=self.num, eps=self.eps,
+                                        l1_ratio=self.l1_ratio)
+        # contiguous folds in row order, sizes as even as they come --
+        # sklearn's KFold(shuffle=False)
+        bounds = np.linspace(0, n, self.cv + 1).astype(int)
+        self.mse_path_ = np.empty((self.num, self.cv))
+        # The single GPU launch replicates each fold's Gram once per
+        # alpha; at large p that replication is gigabytes of transfer
+        # and the CPU warm path wins, so the choice is made by the
+        # replication's size rather than by the GPU's mere presence.
+        gram_bytes = self.cv * self.num * full.p * full.p * 8
+        gpu = (cuda_available() and not self.force_cpu
+               and gram_bytes <= (1 << 29))          # 512 MB
+        G = full.g.reshape(full.p, full.p)
+        if gpu:
+            return self._fit_gpu(X, y, full, G, bounds, n)
+        for k in range(self.cv):
+            lo, hi = bounds[k], bounds[k + 1]
+            Xf, yf = X[lo:hi], y[lo:hi]
+            # The Gram is additive over rows, so a fold's training Gram
+            # is a subtraction: G_full - Xf'Xf costs O(n_fold p^2) where
+            # rebuilding from the complement costs O((n - n_fold) p^2).
+            # Over five folds that is the whole Gram work twice instead
+            # of four times, and the Gram is the dominant O(n p^2) step.
+            m = CovLasso(gram=(G - Xf.T @ Xf).ravel(),
+                         corr=full.c0 - Xf.T @ yf,
+                         nobs=n - (hi - lo))
+            # On the GPU every lambda is its own thread, so the batch is
+            # the right shape; on the CPU the warm-started path is: after
+            # the Gram, each lambda resumes where the previous stopped
+            # and costs O(p) per moving coordinate. That warm path is
+            # what makes searching the alpha grid cheap -- the grid
+            # costs barely more than its hardest single alpha.
+            if gpu:
+                path = m.fit_path_batch(self.alphas_, tol=self.tol,
+                                        l1_ratio=self.l1_ratio)
+            else:
+                path = m.fit_path(self.alphas_, tol=self.tol,
+                                  l1_ratio=self.l1_ratio)
+            resid = Xf @ path.T - yf[:, None]
+            self.mse_path_[:, k] = np.mean(resid * resid, axis=0)
+        mean = self.mse_path_.mean(axis=1)
+        # alphas_ descends, so argmin's first hit is the larger penalty
+        self.alpha_ = float(self.alphas_[int(np.argmin(mean))])
+        self.coef_ = full.fit(self.alpha_, tol=self.tol,
+                              l1_ratio=self.l1_ratio)
+        return self
+
+    def _fit_gpu(self, X, y, full, G, bounds, n):
+        # Every fold and every alpha is an independent problem, so all of
+        # them go to the device as one batch: cv * num threads, each with
+        # its fold's Gram (the per-thread-Gram path the bootstrap added)
+        # and its own penalty. Fold sizes may differ by a row; the kernel
+        # only ever uses lam through the product lam * nobs, so each
+        # thread's nobs is absorbed into its lambda and nobs is passed
+        # as 1 -- exact, not approximate.
+        p_ = full.p
+        num, cv = self.num, self.cv
+        grams = np.empty((cv, p_ * p_))
+        corrs = np.empty((cv, p_))
+        nobs_k = np.empty(cv)
+        for k in range(cv):
+            lo, hi = bounds[k], bounds[k + 1]
+            Xf, yf = X[lo:hi], y[lo:hi]
+            grams[k] = (G - Xf.T @ Xf).ravel()
+            corrs[k] = full.c0 - Xf.T @ yf
+            nobs_k[k] = n - (hi - lo)
+        gr = np.repeat(grams, num, axis=0)
+        cr = np.repeat(corrs, num, axis=0)
+        lams = (np.tile(self.alphas_, cv) * np.repeat(nobs_k, num))
+        betas = _batch_descend_multi(gr, cr, lams, 1.0, p_,
+                                     tol=self.tol,
+                                     l1_ratio=self.l1_ratio,
+                                     force_cpu=self.force_cpu,
+                                     kernel_fn=kernel.enet_descend)
+        betas = betas.reshape(cv, num, p_)
+        for k in range(cv):
+            lo, hi = bounds[k], bounds[k + 1]
+            resid = X[lo:hi] @ betas[k].T - y[lo:hi, None]
+            self.mse_path_[:, k] = np.mean(resid * resid, axis=0)
+        mean = self.mse_path_.mean(axis=1)
+        self.alpha_ = float(self.alphas_[int(np.argmin(mean))])
+        self.coef_ = full.fit(self.alpha_, tol=self.tol,
+                              l1_ratio=self.l1_ratio)
+        return self
+
+    def predict(self, X):
+        return np.ascontiguousarray(X, dtype=np.float64) @ self.coef_
+
+
 class CovRidge:
     """Ridge regression over a design given by its Gram matrix.
 

{scm2cpp_lasso-0.5.2 → scm2cpp_lasso-0.5.3.1}/src/scm2cpp_lasso.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.5.2
+Version: 0.5.3.1
 Summary: Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included
 Author: Hirotaka Niitsuma
 License: MIT
@@ -70,9 +70,14 @@ from scm2cpp_lasso import cuda_available
 lambdas = model.lambda_grid(num=100)        # from lambda_max down
 path = model.fit_path(lambdas)              # warm-started, sequential
 grid = model.fit_path_batch(lambdas)        # every lambda from zero
+one = model.fit(lambdas[60])                # a single lambda, if you must
 print("GPU:", cuda_available())
 ```
 
+`fit` is the one-element special case of `fit_path` -- the Gram matrix
+was built in the constructor, so a whole warm-started path costs barely
+more than one penalty, which is why the path is the primary interface.
+
 The objective is scikit-learn's, with `fit_intercept=False`:
 
     (1 / 2 nobs) ||y - X b||^2 + lam ||b||_1
@@ -84,6 +89,58 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Against the other lassos pip can install
+
+`bench/lasso-compare.py` in the repository runs the same
+cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
+p=200, n=1800 -- against every lasso a `pip install` produces:
+scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+Every solver runs at its own default tolerance (ours at the tighter
+1e-8), each gets one untimed warm-up fit, cuML's data is on the device
+before the clock starts, and the last column is each solver's
+objective minus ours at the smallest lambda -- a fast row with a loose
+answer would show as one:
+
+| solver                                 | time    | objective gap |
+|----------------------------------------|---------|---------------|
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
+| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
+| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
+| celer per lambda                       | 17.3 s  | 0             |
+| skglm per lambda                       | 16.4 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+
+celer and skglm are built for very large sparse designs, where their
+screening rules dominate; at this size they pay their setup per fit.
+cuML parallelises *within* one fit, which wins when a fit is large; at
+this size its launch overhead dominates, while our GPU row
+parallelises *across* lambdas -- one CUDA thread per penalty, the axis
+a cross-validation grid actually offers.
+
+`CovLassoCV` is scikit-learn's `LassoCV` over this machinery -- same
+grid construction, same contiguous folds, same minimum-mean-MSE choice
+-- with two structural savings.  A fold's training Gram is a
+*subtraction* (the Gram is additive over rows, so G - Xf'Xf costs one
+fold's work where rebuilding costs four folds'), and once the Grams
+exist nothing downstream ever touches the n rows again -- which is why
+the gap grows with n.  On the GPU every fold and every alpha can run
+as one thread of a single launch; that launch replicates each fold's
+Gram per alpha, so it is chosen only while the replication stays under
+512 MB, the CPU warm path taking over at large p.  Measured at cv=5,
+100 alphas, one CPU core:
+
+| n       | p    | `CovLassoCV` | sklearn `LassoCV` |
+|---------|------|--------------|-------------------|
+| 1,800   | 200  | 0.23 s       | 0.20 s            |
+| 5,000   | 1000 | 1.1 s        | 3.4 s             |
+| 100,000 | 200  | 0.44 s       | 5.7 s             |
+| 100,000 | 500  | 0.90 s       | 8.0 s             |
+
+At n=100,000 the chosen alpha is identical and the coefficients agree
+to 7e-7; at small n a near-tie can leave the picks one grid step
+apart, which sklearn resolves our way once its tolerance is
+tightened.  GPU and CPU agree to 1e-14 wherever both run.
+
 ## Which method
 
 `fit_path` walks a single path, each lambda starting from the previous