scm2cpp-lasso 0.7.0__tar.gz → 0.8.0__tar.gz

{scm2cpp_lasso-0.7.0 → scm2cpp_lasso-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.7.0
+Version: 0.8.0
 Summary: Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included
 Author: Hirotaka Niitsuma
 License: MIT
@@ -89,12 +89,85 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Two kernels: `CovLasso` and `DerivedLasso`
+
+Everything above, and the cross-validation, bootstrap and GPU methods
+below, runs on one kernel: numpy forms the Gram matrix, and a
+hand-written coordinate-descent sweep over it
+(`examples/kernel-only/lasso-cov.scm` in the repository, translated to
+C++) does the rest.  That is the usable one.
+
+`DerivedLasso` is the other kernel, and it is there to show the
+translator's work rather than to be the better estimator.  Its source
+is `examples/kernel-only/lasso-design.scm`: the residual-form
+coordinate descent -- the algorithm inside scikit-learn's
+`Lasso(precompute=False)` -- written as flat loops over X exactly as
+numpy holds it (n rows of p, `x[i*p+j]`), the residual copied from y
+and the column norms taken inside, with no Gram matrix anywhere in
+it.  The translator's `--derive` turns
+that into the Gram form: it raises the loops to an array algebra,
+notices that the residual is only ever updated by a scaled column and
+read in dot products against columns, replaces it by the memo `c =
+X'r` kept current through a Gram matrix it hoists in front of the
+penalty path, folds the column norms into that Gram's diagonal, folds
+the three products into one `dsyrk` and two `dgemv` calls (`--blas`), and puts the whole thing under
+`(if (< p n) ..)` with the sweep as written in the else branch --
+scikit-learn's `precompute='auto'` rule, derived rather than written.
+The C++ of that derivation, as the translator emitted it, is what
+`DerivedLasso` calls; numpy prepares nothing -- X and y go in as they
+are, no transposed copy, no norms.
+
+```python
+from scm2cpp_lasso import DerivedLasso, derived_backend
+
+model = DerivedLasso(X, y)                   # X and y; the kernel builds its own Gram
+path = model.fit_path(model.lambda_grid())   # one call, the whole warm-started path
+print(derived_backend())                     # 'cublas', 'blas' or 'loops': what this machine built
+gpu = DerivedLasso(X, y, backend="cublas")   # insist on one
+```
+
+A lambda is done when no coefficient moved by more than `tol`
+(absolute, default 1e-8) in a sweep, or at `max_sweeps`; `tol=0.0`
+asks for the exact fixed point.  Three translations of the one
+derivation are committed and `pip` builds what the machine allows:
+`--derive --cublas` where the CUDA toolkit is (the three products on
+the device through cuBLAS, X uploaded once per `fit_path` call --
+taken by default when a device answers), `--derive --blas` where
+`cblas.h` and an OpenBLAS are, and `--derive` alone otherwise, the
+same program with its Gram matrix as a scalar loop nest, correct but
+slow on large designs.  `check-derived.py` in the package directory runs
+the two kernels and scikit-learn on the same path and prints the
+differences and the times; on AR(1) designs (rho 0.9), 20 penalties
+from lambda_max to 0.01 lambda_max, one core, best of three (load
+about 3), whole calls with the Gram matrix wherever it is built:
+
+| n x p                                   | 1,800 x 200 | 5,000 x 1000 | 100,000 x 200 | 100,000 x 500 |
+|-----------------------------------------|-------------|--------------|---------------|---------------|
+| `DerivedLasso`, cuBLAS (upload included) | 0.003 s     | 0.024 s      | 0.052 s       | 0.11 s        |
+| `DerivedLasso`, CBLAS                   | 0.003 s     | 0.076 s      | 0.12 s        | 0.44 s        |
+| `CovLasso(X, y).fit_path`, tol 1e-8     | 0.004 s     | 0.080 s      | 0.11 s        | 0.42 s        |
+| sklearn `lasso_path(precompute=True)`, tol 1e-4 | 0.005 s     | 0.10 s       | 0.24 s        | 0.76 s        |
+
+The CBLAS run is the same `dsyrk` and the same sweeps as `CovLasso`'s
+numpy Gram and path, and times the same; the cuBLAS run is the same
+program with the three products on the device, the upload of X
+counted, and is 4.2x, 4.5x and 6.7x ahead of scikit-learn at the
+three larger shapes -- the figures of the repository's equal-work
+table, now reached from the package with nothing in front of the
+kernel: the constructor keeps X and y and the kernel takes the
+residual and the column norms itself (the norms folded into the Gram
+diagonal by the derivation).  Against a reference driven to
+tol=1e-12, `DerivedLasso` lands within 5e-8 on every backend (its
+tolerance is absolute, on one sweep's largest move), `CovLasso`
+within 2e-10 (it tests every 20 sweeps and overshoots), and
+scikit-learn at its default within 1e-3.
+
 ## Against the other lassos pip can install
 
 `bench/lasso-compare.py` in the repository runs the same
 cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
 p=200, n=1800 -- against every lasso a `pip install` produces:
-scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+scikit-learn 1.9.1, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
 Every solver runs at its own default tolerance (ours at the tighter
 1e-8), each gets one untimed warm-up fit, cuML's data is on the device
 before the clock starts, and the last column is each solver's
@@ -103,12 +176,15 @@ answer would show as one:
 
 | solver                                 | time    | objective gap |
 |----------------------------------------|---------|---------------|
-| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
-| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
-| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
-| celer per lambda                       | 17.3 s  | 0             |
-| skglm per lambda                       | 16.4 s  | +2.8e-17      |
-| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.91 s  | 0             |
+| scm2cpp-lasso, GPU, one block/lambda   | 0.044 s | 0             |
+| sklearn `Lasso.fit` per lambda         | 9.6 s   | +1.6e-09      |
+| celer per lambda                       | 13.5 s  | 0             |
+| skglm per lambda                       | 14.5 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 53.6 s  | +9.1e-07      |
+
+(One sitting on an otherwise idle machine, best of three runs; cuML
+26.08 on CUDA 12.9.)
 
 celer and skglm are built for very large sparse designs, where their
 screening rules dominate; at this size they pay their setup per fit.
@@ -134,25 +210,26 @@ times the alternatives (one thread per cell, warm runs of several
 alphas per thread or block) side by side, and the block per cell wins
 at every size tried.  The GPU is the default when a device answers
 (`force_cpu` takes the warm CPU path).  Measured at cv=5, 100 alphas,
-one CPU core, sklearn 1.9.0, best of three runs, whole estimator
+one CPU core, sklearn 1.9.1, best of three runs, whole estimator
 including the Grams:
 
 | n       | p    | CPU    | CUDA   | sklearn `LassoCV` |
 |---------|------|--------|--------|-------------------|
-| 1,800   | 200  | 0.36 s | 0.03 s | 0.23 s            |
-| 5,000   | 1000 | 3.9 s  | 0.39 s | 2.7 s             |
-| 100,000 | 200  | 0.75 s | 0.60 s | 3.9 s             |
-| 100,000 | 500  | 2.0 s  | 1.8 s  | 12.6 s            |
-
-(This table was taken on a machine sharing its cores with other jobs,
-load 14 on 20 threads, so its absolute times run some 2-3x above the
-tables earlier in this README; the ratios are the point.)  The grid
+| 1,800   | 200  | 0.21 s | 0.02 s | 0.10 s            |
+| 5,000   | 1000 | 1.48 s | 0.25 s | 0.90 s            |
+| 100,000 | 200  | 0.49 s | 0.34 s | 1.72 s            |
+| 100,000 | 500  | 1.32 s | 1.09 s | 4.82 s            |
+
+(Taken on an otherwise idle machine, load 2-3 on 20 threads.  sklearn's
+`LassoCV` at its default `precompute='auto'` takes the Gram route at
+every size here, n > p, rebuilding each fold's Gram from that fold's
+rows where we subtract it from the whole.)  The grid
 launch itself takes 9 ms at n=1,800 and 64 ms at n=5,000, p=1000; at
 n=100,000 the CUDA column is nearly all Gram products, which the CPU
 path pays as well.  Large n is where the machinery pays either way:
 nothing downstream of the Grams touches the 100,000 rows again.  At
 n=5,000, p=1000 sklearn's raw-X descent beats our Gram-heavy CPU path,
-and the GPU makes up the difference seven times over.  CPU and CUDA
+and the GPU makes up the difference three and a half times over.  CPU and CUDA
 choose the same alpha everywhere and agree to 1e-14; against sklearn
 the pick is exact at n=5,000 and one grid step apart on a near-tie
 elsewhere (mean-MSE relative gap at most 2.6e-4), and sklearn at
@@ -178,21 +255,24 @@ the CV pairs share the 100-alpha grid:
 
 | estimator             | n       | p   | ours   | ours `n_jobs=5` | ours CUDA | sklearn |
 |-----------------------|---------|-----|--------|-----------------|-----------|---------|
-| MultiTaskLasso        | 100,000 | 200 | 0.15 s | --              | --        | 0.80 s  |
-| MultiTaskLasso        | 100,000 | 500 | 0.53 s | --              | --        | 1.9 s   |
-| MultiTaskLassoCV      | 1,800   | 200 | 1.5 s  | 0.33 s          | 0.08 s    | 0.51 s  |
-| MultiTaskLassoCV      | 100,000 | 200 | 1.6 s  | 0.62 s          | 1.7 s     | 76 s    |
-| MultiTaskElasticNetCV | 1,800   | 200 | 1.6 s  | 0.51 s          | 0.10 s    | 1.0 s   |
-| MultiTaskElasticNetCV | 100,000 | 200 | 1.6 s  | 0.68 s          | 1.9 s     | 164 s   |
-
-(`MultiTaskElasticNet` single fits time the same as `MultiTaskLasso`'s.
-The CUDA column was timed on the busier day of the `CovLassoCV` table
-above, when the sequential CPU rows read 1.5 / 2.0 / 1.6 / 2.1 s and
-sklearn's 0.56 / 114 / 0.99 / 241 s.)  At n=1,800 our sequential CV
+| MultiTaskLasso        | 100,000 | 200 | 0.14 s | --              | --        | 0.76 s  |
+| MultiTaskLasso        | 100,000 | 500 | 0.53 s | --              | --        | 1.7 s   |
+| MultiTaskLassoCV      | 1,800   | 200 | 1.4 s  | 0.51 s          | 0.07 s    | 0.42 s  |
+| MultiTaskLassoCV      | 100,000 | 200 | 1.6 s  | 0.64 s          | 1.3 s     | 74 s    |
+| MultiTaskElasticNetCV | 1,800   | 200 | 1.4 s  | 0.49 s          | 0.08 s    | 0.75 s  |
+| MultiTaskElasticNetCV | 100,000 | 200 | 1.6 s  | 0.68 s          | 1.3 s     | 162 s   |
+
+(`MultiTaskElasticNet` single fits time the same as `MultiTaskLasso`'s;
+every column was taken in one sitting on the idle machine of the
+`CovLassoCV` table above, sklearn 1.9.1.  scikit-learn has no Gram
+route for its multi-task estimators -- its path code sets
+`precompute=False` for 2-D targets, "No Gram variant of multi-task
+exists right now" -- so its column is raw-X descent at every n, which
+is why it grows with n where `LassoCV`'s did not.)  At n=1,800 our sequential CV
 is slower than sklearn's: the default tol=1e-8 runs more sweeps than
 sklearn's 1e-4 dual-gap stop, and at that size there are no rows to
 save -- and the GPU grid takes the whole thing in a tenth of a second.
-At n=100,000 the Gram route is 48x and 100x ahead, essentially
+At n=100,000 the Gram route is 46x and 100x ahead, essentially
 unchanged from n=1,800 in absolute time; the CUDA column there is
 the Gram products, the same ones the CPU path forms.  Coefficients agree with scikit-learn's to 1e-15 at tight
 tolerance, and the CV pair picks the same alpha with coefficients to
@@ -294,12 +374,15 @@ neither installs the other.
 ## Where this comes from
 
 The C++ this compiles is committed to `python/scm2cpp-lasso/` in the
-scm2cpp repository, translated from
-`examples/kernel-only/lasso-cov.scm`.  That repository also derives the
-covariance-update solver automatically from a naive one by finite
-differencing; this package is the derived kernel, packaged.  To refresh
-the committed C++ after changing the Scheme, run `regenerate.sh` --
-that step, and only that step, needs Racket.
+scm2cpp repository.  `CovLasso` and the rest run on the translation of
+`examples/kernel-only/lasso-cov.scm`, a covariance-update sweep
+written by hand at the Gram level; `DerivedLasso` runs on the
+translation of `examples/kernel-only/lasso-design.scm`, the naive
+residual-form descent over X as numpy holds it, with the translator's `--derive --blas`
+deriving the same covariance form from it.  The two are the same
+sweep over the same `dsyrk`, and they time the same.  To refresh the
+committed C++ after changing the Scheme, run `regenerate.sh` -- that
+step, and only that step, needs Racket.
 
 ## License
 

{scm2cpp_lasso-0.7.0 → scm2cpp_lasso-0.8.0}/README.md

@@ -71,12 +71,85 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Two kernels: `CovLasso` and `DerivedLasso`
+
+Everything above, and the cross-validation, bootstrap and GPU methods
+below, runs on one kernel: numpy forms the Gram matrix, and a
+hand-written coordinate-descent sweep over it
+(`examples/kernel-only/lasso-cov.scm` in the repository, translated to
+C++) does the rest.  That is the usable one.
+
+`DerivedLasso` is the other kernel, and it is there to show the
+translator's work rather than to be the better estimator.  Its source
+is `examples/kernel-only/lasso-design.scm`: the residual-form
+coordinate descent -- the algorithm inside scikit-learn's
+`Lasso(precompute=False)` -- written as flat loops over X exactly as
+numpy holds it (n rows of p, `x[i*p+j]`), the residual copied from y
+and the column norms taken inside, with no Gram matrix anywhere in
+it.  The translator's `--derive` turns
+that into the Gram form: it raises the loops to an array algebra,
+notices that the residual is only ever updated by a scaled column and
+read in dot products against columns, replaces it by the memo `c =
+X'r` kept current through a Gram matrix it hoists in front of the
+penalty path, folds the column norms into that Gram's diagonal, folds
+the three products into one `dsyrk` and two `dgemv` calls (`--blas`), and puts the whole thing under
+`(if (< p n) ..)` with the sweep as written in the else branch --
+scikit-learn's `precompute='auto'` rule, derived rather than written.
+The C++ of that derivation, as the translator emitted it, is what
+`DerivedLasso` calls; numpy prepares nothing -- X and y go in as they
+are, no transposed copy, no norms.
+
+```python
+from scm2cpp_lasso import DerivedLasso, derived_backend
+
+model = DerivedLasso(X, y)                   # X and y; the kernel builds its own Gram
+path = model.fit_path(model.lambda_grid())   # one call, the whole warm-started path
+print(derived_backend())                     # 'cublas', 'blas' or 'loops': what this machine built
+gpu = DerivedLasso(X, y, backend="cublas")   # insist on one
+```
+
+A lambda is done when no coefficient moved by more than `tol`
+(absolute, default 1e-8) in a sweep, or at `max_sweeps`; `tol=0.0`
+asks for the exact fixed point.  Three translations of the one
+derivation are committed and `pip` builds what the machine allows:
+`--derive --cublas` where the CUDA toolkit is (the three products on
+the device through cuBLAS, X uploaded once per `fit_path` call --
+taken by default when a device answers), `--derive --blas` where
+`cblas.h` and an OpenBLAS are, and `--derive` alone otherwise, the
+same program with its Gram matrix as a scalar loop nest, correct but
+slow on large designs.  `check-derived.py` in the package directory runs
+the two kernels and scikit-learn on the same path and prints the
+differences and the times; on AR(1) designs (rho 0.9), 20 penalties
+from lambda_max to 0.01 lambda_max, one core, best of three (load
+about 3), whole calls with the Gram matrix wherever it is built:
+
+| n x p                                   | 1,800 x 200 | 5,000 x 1000 | 100,000 x 200 | 100,000 x 500 |
+|-----------------------------------------|-------------|--------------|---------------|---------------|
+| `DerivedLasso`, cuBLAS (upload included) | 0.003 s     | 0.024 s      | 0.052 s       | 0.11 s        |
+| `DerivedLasso`, CBLAS                   | 0.003 s     | 0.076 s      | 0.12 s        | 0.44 s        |
+| `CovLasso(X, y).fit_path`, tol 1e-8     | 0.004 s     | 0.080 s      | 0.11 s        | 0.42 s        |
+| sklearn `lasso_path(precompute=True)`, tol 1e-4 | 0.005 s     | 0.10 s       | 0.24 s        | 0.76 s        |
+
+The CBLAS run is the same `dsyrk` and the same sweeps as `CovLasso`'s
+numpy Gram and path, and times the same; the cuBLAS run is the same
+program with the three products on the device, the upload of X
+counted, and is 4.2x, 4.5x and 6.7x ahead of scikit-learn at the
+three larger shapes -- the figures of the repository's equal-work
+table, now reached from the package with nothing in front of the
+kernel: the constructor keeps X and y and the kernel takes the
+residual and the column norms itself (the norms folded into the Gram
+diagonal by the derivation).  Against a reference driven to
+tol=1e-12, `DerivedLasso` lands within 5e-8 on every backend (its
+tolerance is absolute, on one sweep's largest move), `CovLasso`
+within 2e-10 (it tests every 20 sweeps and overshoots), and
+scikit-learn at its default within 1e-3.
+
 ## Against the other lassos pip can install
 
 `bench/lasso-compare.py` in the repository runs the same
 cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
 p=200, n=1800 -- against every lasso a `pip install` produces:
-scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+scikit-learn 1.9.1, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
 Every solver runs at its own default tolerance (ours at the tighter
 1e-8), each gets one untimed warm-up fit, cuML's data is on the device
 before the clock starts, and the last column is each solver's
@@ -85,12 +158,15 @@ answer would show as one:
 
 | solver                                 | time    | objective gap |
 |----------------------------------------|---------|---------------|
-| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
-| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
-| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
-| celer per lambda                       | 17.3 s  | 0             |
-| skglm per lambda                       | 16.4 s  | +2.8e-17      |
-| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.91 s  | 0             |
+| scm2cpp-lasso, GPU, one block/lambda   | 0.044 s | 0             |
+| sklearn `Lasso.fit` per lambda         | 9.6 s   | +1.6e-09      |
+| celer per lambda                       | 13.5 s  | 0             |
+| skglm per lambda                       | 14.5 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 53.6 s  | +9.1e-07      |
+
+(One sitting on an otherwise idle machine, best of three runs; cuML
+26.08 on CUDA 12.9.)
 
 celer and skglm are built for very large sparse designs, where their
 screening rules dominate; at this size they pay their setup per fit.
@@ -116,25 +192,26 @@ times the alternatives (one thread per cell, warm runs of several
 alphas per thread or block) side by side, and the block per cell wins
 at every size tried.  The GPU is the default when a device answers
 (`force_cpu` takes the warm CPU path).  Measured at cv=5, 100 alphas,
-one CPU core, sklearn 1.9.0, best of three runs, whole estimator
+one CPU core, sklearn 1.9.1, best of three runs, whole estimator
 including the Grams:
 
 | n       | p    | CPU    | CUDA   | sklearn `LassoCV` |
 |---------|------|--------|--------|-------------------|
-| 1,800   | 200  | 0.36 s | 0.03 s | 0.23 s            |
-| 5,000   | 1000 | 3.9 s  | 0.39 s | 2.7 s             |
-| 100,000 | 200  | 0.75 s | 0.60 s | 3.9 s             |
-| 100,000 | 500  | 2.0 s  | 1.8 s  | 12.6 s            |
-
-(This table was taken on a machine sharing its cores with other jobs,
-load 14 on 20 threads, so its absolute times run some 2-3x above the
-tables earlier in this README; the ratios are the point.)  The grid
+| 1,800   | 200  | 0.21 s | 0.02 s | 0.10 s            |
+| 5,000   | 1000 | 1.48 s | 0.25 s | 0.90 s            |
+| 100,000 | 200  | 0.49 s | 0.34 s | 1.72 s            |
+| 100,000 | 500  | 1.32 s | 1.09 s | 4.82 s            |
+
+(Taken on an otherwise idle machine, load 2-3 on 20 threads.  sklearn's
+`LassoCV` at its default `precompute='auto'` takes the Gram route at
+every size here, n > p, rebuilding each fold's Gram from that fold's
+rows where we subtract it from the whole.)  The grid
 launch itself takes 9 ms at n=1,800 and 64 ms at n=5,000, p=1000; at
 n=100,000 the CUDA column is nearly all Gram products, which the CPU
 path pays as well.  Large n is where the machinery pays either way:
 nothing downstream of the Grams touches the 100,000 rows again.  At
 n=5,000, p=1000 sklearn's raw-X descent beats our Gram-heavy CPU path,
-and the GPU makes up the difference seven times over.  CPU and CUDA
+and the GPU makes up the difference three and a half times over.  CPU and CUDA
 choose the same alpha everywhere and agree to 1e-14; against sklearn
 the pick is exact at n=5,000 and one grid step apart on a near-tie
 elsewhere (mean-MSE relative gap at most 2.6e-4), and sklearn at
@@ -160,21 +237,24 @@ the CV pairs share the 100-alpha grid:
 
 | estimator             | n       | p   | ours   | ours `n_jobs=5` | ours CUDA | sklearn |
 |-----------------------|---------|-----|--------|-----------------|-----------|---------|
-| MultiTaskLasso        | 100,000 | 200 | 0.15 s | --              | --        | 0.80 s  |
-| MultiTaskLasso        | 100,000 | 500 | 0.53 s | --              | --        | 1.9 s   |
-| MultiTaskLassoCV      | 1,800   | 200 | 1.5 s  | 0.33 s          | 0.08 s    | 0.51 s  |
-| MultiTaskLassoCV      | 100,000 | 200 | 1.6 s  | 0.62 s          | 1.7 s     | 76 s    |
-| MultiTaskElasticNetCV | 1,800   | 200 | 1.6 s  | 0.51 s          | 0.10 s    | 1.0 s   |
-| MultiTaskElasticNetCV | 100,000 | 200 | 1.6 s  | 0.68 s          | 1.9 s     | 164 s   |
-
-(`MultiTaskElasticNet` single fits time the same as `MultiTaskLasso`'s.
-The CUDA column was timed on the busier day of the `CovLassoCV` table
-above, when the sequential CPU rows read 1.5 / 2.0 / 1.6 / 2.1 s and
-sklearn's 0.56 / 114 / 0.99 / 241 s.)  At n=1,800 our sequential CV
+| MultiTaskLasso        | 100,000 | 200 | 0.14 s | --              | --        | 0.76 s  |
+| MultiTaskLasso        | 100,000 | 500 | 0.53 s | --              | --        | 1.7 s   |
+| MultiTaskLassoCV      | 1,800   | 200 | 1.4 s  | 0.51 s          | 0.07 s    | 0.42 s  |
+| MultiTaskLassoCV      | 100,000 | 200 | 1.6 s  | 0.64 s          | 1.3 s     | 74 s    |
+| MultiTaskElasticNetCV | 1,800   | 200 | 1.4 s  | 0.49 s          | 0.08 s    | 0.75 s  |
+| MultiTaskElasticNetCV | 100,000 | 200 | 1.6 s  | 0.68 s          | 1.3 s     | 162 s   |
+
+(`MultiTaskElasticNet` single fits time the same as `MultiTaskLasso`'s;
+every column was taken in one sitting on the idle machine of the
+`CovLassoCV` table above, sklearn 1.9.1.  scikit-learn has no Gram
+route for its multi-task estimators -- its path code sets
+`precompute=False` for 2-D targets, "No Gram variant of multi-task
+exists right now" -- so its column is raw-X descent at every n, which
+is why it grows with n where `LassoCV`'s did not.)  At n=1,800 our sequential CV
 is slower than sklearn's: the default tol=1e-8 runs more sweeps than
 sklearn's 1e-4 dual-gap stop, and at that size there are no rows to
 save -- and the GPU grid takes the whole thing in a tenth of a second.
-At n=100,000 the Gram route is 48x and 100x ahead, essentially
+At n=100,000 the Gram route is 46x and 100x ahead, essentially
 unchanged from n=1,800 in absolute time; the CUDA column there is
 the Gram products, the same ones the CPU path forms.  Coefficients agree with scikit-learn's to 1e-15 at tight
 tolerance, and the CV pair picks the same alpha with coefficients to
@@ -276,12 +356,15 @@ neither installs the other.
 ## Where this comes from
 
 The C++ this compiles is committed to `python/scm2cpp-lasso/` in the
-scm2cpp repository, translated from
-`examples/kernel-only/lasso-cov.scm`.  That repository also derives the
-covariance-update solver automatically from a naive one by finite
-differencing; this package is the derived kernel, packaged.  To refresh
-the committed C++ after changing the Scheme, run `regenerate.sh` --
-that step, and only that step, needs Racket.
+scm2cpp repository.  `CovLasso` and the rest run on the translation of
+`examples/kernel-only/lasso-cov.scm`, a covariance-update sweep
+written by hand at the Gram level; `DerivedLasso` runs on the
+translation of `examples/kernel-only/lasso-design.scm`, the naive
+residual-form descent over X as numpy holds it, with the translator's `--derive --blas`
+deriving the same covariance form from it.  The two are the same
+sweep over the same `dsyrk`, and they time the same.  To refresh the
+committed C++ after changing the Scheme, run `regenerate.sh` -- that
+step, and only that step, needs Racket.
 
 ## License
 

{scm2cpp_lasso-0.7.0 → scm2cpp_lasso-0.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scm2cpp-lasso"
-version = "0.7.0"
+version = "0.8.0"
 description = "Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included"
 readme = "README.md"
 requires-python = ">=3.8"
@@ -31,4 +31,4 @@ package-dir = {"" = "src"}
 where = ["src"]
 
 [tool.setuptools.package-data]
-scm2cpp_lasso = ["_generated/*.hpp", "_generated/*.cpp", "*.cu"]
+scm2cpp_lasso = ["_generated/*.hpp", "_generated/*.cpp", "_generated/derived_*/*.hpp", "_generated/derived_*/*.cpp", "*.cu"]

scm2cpp_lasso-0.8.0/setup.py

@@ -0,0 +1,151 @@
+"""Build the translated kernel, and the CUDA path when nvcc is here.
+
+The C++ this compiles is committed, not generated at install time, so
+building the package needs a C++17 compiler and nothing else -- no
+Racket, no scm2cpp.  A CBLAS (cblas.h plus openblas) at install time
+gives the derived kernel its BLAS variant; without one the loop
+variant is built.  regenerate.sh is how the committed sources are
+refreshed, and it is the only thing that needs the translator.
+
+The CUDA library is optional in the strongest sense: no nvcc, no
+device, or a failed nvcc all leave a working CPU package behind, and
+the batch path notices at import and falls back.
+"""
+import os
+import shutil
+import subprocess
+import sys
+from pathlib import Path
+
+from setuptools import Extension, setup
+from setuptools.command.build_ext import build_ext
+
+HERE = Path(__file__).resolve().parent
+# setuptools wants sources and include paths relative to this file and
+# separated by forward slashes, on every platform
+PKG_REL = "src/scm2cpp_lasso"
+GEN_REL = PKG_REL + "/_generated"
+PKG = HERE / "src" / "scm2cpp_lasso"
+GEN = PKG / "_generated"
+
+CXXFLAGS = ["-O2", "-std=c++17"] if os.name != "nt" else ["/O2", "/std:c++17"]
+
+kernel = Extension(
+    "scm2cpp_lasso._lasso_cov",
+    sources=[GEN_REL + "/lasso_cov_capi.cpp"],
+    include_dirs=[GEN_REL],
+    extra_compile_args=CXXFLAGS,
+)
+
+
+def cblas_flags():
+    """How to compile and link against a CBLAS here, or None.
+
+    pkg-config for openblas where it answers, else the bare -lopenblas
+    if a test program compiles and links against it.  The derived
+    kernel's BLAS variant needs cblas.h and a library; without them the
+    loop variant is built instead (same answers, the Gram as a scalar
+    loop nest).
+    """
+    if os.name == "nt" or os.environ.get("SCM2CPP_NO_BLAS"):
+        return None
+    cxx = os.environ.get("CXX", "c++")
+    candidates = []
+    try:
+        pc = subprocess.run(["pkg-config", "--cflags", "--libs", "openblas"],
+                            capture_output=True, text=True, check=True)
+        candidates.append(pc.stdout.split())
+    except (subprocess.CalledProcessError, OSError):
+        pass
+    candidates.append(["-lopenblas"])
+    probe = "#include <cblas.h>\nint main(){double a=1,b=0;" \
+            "cblas_dgemv(CblasRowMajor,CblasNoTrans,1,1,1.0,&a,1,&a,1,0.0,&b,1);" \
+            "return b==1?0:1;}\n"
+    for flags in candidates:
+        try:
+            r = subprocess.run([cxx, "-x", "c++", "-", "-o", os.devnull] + flags,
+                               input=probe, capture_output=True, text=True)
+        except OSError:
+            return None
+        if r.returncode == 0:
+            return flags
+    return None
+
+
+# The derived kernel: lasso-design.scm, the plain residual-form
+# coordinate descent over X as numpy holds it, turned into the Gram form by the translator
+# (--derive) with its products as CBLAS calls (--blas).  Which variant
+# is built is visible in the extension's name; _libfind reports it.
+_blas = cblas_flags()
+if _blas is not None:
+    derived = Extension(
+        "scm2cpp_lasso._lasso_derived_blas",
+        sources=[GEN_REL + "/derived_blas/lasso_design_capi.cpp"],
+        include_dirs=[GEN_REL + "/derived_blas", GEN_REL],
+        extra_compile_args=CXXFLAGS + [f for f in _blas if f.startswith("-I")
+                                       or f.startswith("-D")],
+        extra_link_args=[f for f in _blas if not f.startswith("-I")
+                         and not f.startswith("-D")],
+    )
+else:
+    derived = Extension(
+        "scm2cpp_lasso._lasso_derived_loops",
+        sources=[GEN_REL + "/derived_loops/lasso_design_capi.cpp"],
+        include_dirs=[GEN_REL + "/derived_loops", GEN_REL],
+        extra_compile_args=(["-O3", "-std=c++17"] if os.name != "nt"
+                            else ["/O2", "/std:c++17"]),
+    )
+
+
+class BuildWithCuda(build_ext):
+    def build_extensions(self):
+        # the kernel is C++ compiled through the usual machinery, so
+        # wheels and cross-compilation behave as they always do
+        super().build_extensions()
+        if os.environ.get("SCM2CPP_NO_CUDA"):
+            return
+        nvcc = shutil.which("nvcc")
+        if not nvcc:
+            return
+        out = Path(self.build_lib) / "scm2cpp_lasso" / "libscm2cpp_batch.so"
+        out.parent.mkdir(parents=True, exist_ok=True)
+        cmd = [nvcc, "-O2", "-std=c++17", "-shared",
+               "-Xcompiler", "-fPIC",
+               "-I", str(PKG), "-I", str(GEN),
+               str(PKG / "batch_capi.cu"), "-o", str(out),
+               "-Wno-deprecated-gpu-targets", "-diag-suppress", "174"]
+        try:
+            subprocess.run(cmd, check=True)
+            print("scm2cpp_lasso: built the CUDA batch path")
+        except (subprocess.CalledProcessError, OSError) as exc:
+            print(f"scm2cpp_lasso: no CUDA batch path ({exc}); "
+                  "the CPU solver is unaffected", file=sys.stderr)
+
+
+# The cuBLAS variant of the derived kernel: host C++ against the CUDA
+# runtime and cuBLAS, so it builds wherever the toolkit's headers and
+# libraries are, device or not; DerivedLasso uses it only when a device
+# answers.  Skipped silently without the toolkit.
+def cuda_home():
+    if os.environ.get("SCM2CPP_NO_CUDA"):
+        return None
+    nvcc = shutil.which("nvcc")
+    if not nvcc:
+        return None
+    home = Path(nvcc).resolve().parent.parent
+    return home if (home / "include" / "cublas_v2.h").exists() else None
+
+
+_cuda = cuda_home()
+extensions = [kernel, derived]
+if _cuda is not None:
+    extensions.append(Extension(
+        "scm2cpp_lasso._lasso_derived_cublas",
+        sources=[GEN_REL + "/derived_cublas/lasso_design_capi.cpp"],
+        include_dirs=[GEN_REL + "/derived_cublas", GEN_REL, str(_cuda / "include")],
+        library_dirs=[str(_cuda / "lib64"), str(_cuda / "lib")],
+        libraries=["cublas", "cudart"],
+        extra_compile_args=CXXFLAGS,
+    ))
+
+setup(ext_modules=extensions, cmdclass={"build_ext": BuildWithCuda})