scm2cpp-learn 0.18.0__tar.gz → 0.20.0__tar.gz

{scm2cpp_learn-0.18.0/src/scm2cpp_learn.egg-info → scm2cpp_learn-0.20.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: scm2cpp-learn
-Version: 0.18.0
-Summary: scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
+Version: 0.20.0
+Summary: scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, PLS, CCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
 Author: Hirotaka Niitsuma
 License: MIT
 Project-URL: Homepage, https://github.com/niitsuma/scm2cpp
@@ -206,9 +206,10 @@ package picks the fastest one present.
 
 | class | scikit-learn | Scheme kernel | builds | sklearnex accelerates |
 |---|---|---|---|---|
-| `GramLasso` | Lasso(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
-| `GramElasticNet` | ElasticNet(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
-| `GramRidge` | Ridge, fit(X, y) with the intercept, one or many targets | `gram.scm` (Cholesky; dposv) | loops / BLAS | Ridge |
+| `GramLasso` | Lasso(precompute=True), fit(X, y) with the intercept, dense or sparse X | `lasso-cov.scm` | CPU | no |
+| `GramElasticNet` | ElasticNet(precompute=True), fit(X, y) with the intercept, dense or sparse X | `lasso-cov.scm` | CPU | no |
+| `GramRidge` | Ridge, fit(X, y) with the intercept, one or many targets, dense or sparse X | `gram.scm` (Cholesky; dposv) | loops / BLAS | Ridge |
+| `GramLinearRegression` | LinearRegression, dense or sparse X, sample_weight | `linreg.scm` (Cholesky; dposv, dpocon) | loops / BLAS | no |
 | `GramRidgeCV` | RidgeCV (cv=None), leave-one-out over every alpha from one eigenproblem | `ridge-gcv.scm` | loops / BLAS | no |
 | `GramMultiTaskLasso` | MultiTaskLasso, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramMultiTaskElasticNet` | MultiTaskElasticNet, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
@@ -256,7 +257,9 @@ package picks the fastest one present.
 | `CovPCA` | PCA(svd_solver='covariance_eigh') | `pca.scm` | loops / BLAS / cuBLAS | PCA |
 | `GramKernelPCA` | KernelPCA (no inverse_transform) | `kernel-pca.scm` | loops / BLAS | no |
 | `GramLDA` | LinearDiscriminantAnalysis (svd, lsqr, eigen) | `lda.scm` | loops / BLAS | no |
-| `GramTruncatedSVD` | TruncatedSVD on dense X (exact, as arpack) | `truncated-svd.scm` | loops / BLAS | no |
+| `GramTruncatedSVD` | TruncatedSVD on dense or sparse X (exact, as arpack) | `truncated-svd.scm`, `sparse.scm` | loops / BLAS | no |
+| `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD` | PLSRegression, PLSCanonical, CCA, PLSSVD | `pls.scm` | loops / BLAS | no |
+| `GramFactorAnalysis` | FactorAnalysis (exact, as svd_method='lapack'; rotations) | `fa.scm` | loops / BLAS | no |
 | `f_regression, f_classif` | f_regression, f_classif | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `SelectKBest, SelectPercentile` | SelectKBest, SelectPercentile | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `GramRFE` | RFE(Ridge), RFE(LinearRegression) | `rfe.scm` | loops / BLAS / cuBLAS | inside: Ridge |
@@ -714,6 +717,143 @@ The exact route costs O(n p^2 + p^3); with few components of many
 features (2,000 here) the randomized one does less work, and on one core
 it is faster, at its accuracy. Sparse X is not taken yet.
 
+## PLS and CCA: `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD`
+
+scikit-learn's `cross_decomposition` estimators over the cross products
+(`examples/kernel-only/pls.scm`). NIPALS multiplies the blocks only by
+weight vectors, and each such product is one of Cxx = X'X, Cxy = X'Y,
+Cyy = Y'Y times a vector: the Y score is Y c for a coefficient vector c,
+so X'(Y c) = Cxy c. The deflations are rank-one updates of the three
+matrices (with a = Cxx w, b = Cxy'w and s2 = w'Cxx w, regression mode is
+Cxx -= a a'/s2, Cxy -= a b'/s2, Cyy -= b b'/s2). So X and Y are read
+once, for the three products (dsyrk and dgemm), and every power
+iteration after that costs O(p^2 + pq + q^2) where scikit-learn's costs
+O(n(p + q)). CCA's mode B takes pinv(X) y = Cxx^+ X'y, Cxx^+ from its
+eigenproblem; PLSSVD is the SVD of Cxy.
+
+```python
+from scm2cpp_learn import GramPLSRegression, GramCCA
+pls = GramPLSRegression(n_components=5).fit(X, Y)
+pls.predict(X), pls.transform(X), pls.x_scores_
+```
+
+Weights, loadings, rotations, coefficients, scores and transforms agree
+with scikit-learn's to 1e-13, with the same iteration counts. One cut
+differs: a deflated block loses a rank exactly, which its SVD shows as a
+singular value near eps but its Gram as an eigenvalue near eps
+lam_max, so mode B's pinv also drops eigenvalues under 1e3 eps lam_max
+(singular values under about 5e-7 of the largest, where scikit-learn's
+cut is 2.2e-10). Measured (`bench/eigen-compare.py`, the MKL build, best
+of three):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| PLSRegression, 200,000 x 100 -> 5, 5 components | 5.64 s | 0.26 s | 21x | 19x |
+| PLSCanonical, 200,000 x 100 -> 5, 5 components | 5.63 s | 0.14 s | 40x | 59x |
+| CCA, 200,000 x 100 -> 5, 5 components | 31.9 s | 0.14 s | 220x | 190x |
+| PLSSVD, 200,000 x 100 -> 5, 5 components | 0.28 s | 0.14 s | 2.0x | 4.0x |
+| PLSRegression, 50,000 x 500 -> 10, 10 components | 45.5 s | 0.75 s | 61x | 57x |
+| PLSCanonical, 50,000 x 500 -> 10, 10 components | 45.3 s | 0.60 s | 76x | 100x |
+| CCA, 50,000 x 500 -> 10, 10 components | 132 s | 1.21 s | 109x | 151x |
+| PLSSVD, 50,000 x 500 -> 10, 10 components | 0.34 s | 0.34 s | 1.0x | 3.4x |
+
+PLSSVD is one SVD of X'Y on both sides, so only the product differs.
+
+## Factor analysis: `GramFactorAnalysis`
+
+scikit-learn's `FactorAnalysis` over the covariance
+(`examples/kernel-only/fa.scm`). Every iteration of scikit-learn's takes
+the SVD of the n x p matrix Xc / (sqrt(psi) sqrt(n)); its squared
+singular values and right singular vectors are the eigenpairs of
+D^-1 C D^-1, C = Xc'Xc / n and D = diag(sqrt psi). So C is formed once
+(dsyrk) and each iteration is the top `n_components` eigenpairs of a
+p x p matrix (dsyevr), then scikit-learn's own updates of the
+components, the log-likelihood and psi. The eigenproblem is exact: what
+`svd_method="lapack"` computes, which the fit matches to 1e-13 with the
+same iterations; scikit-learn's default `"randomized"` approximates it
+(here to 1e-6 in psi). Rotations (varimax, quartimax), `transform`,
+`score`, `get_covariance` and `get_precision` are scikit-learn's.
+
+```python
+from scm2cpp_learn import GramFactorAnalysis
+fa = GramFactorAnalysis(n_components=10).fit(X)
+fa.components_, fa.noise_variance_, fa.transform(X)
+```
+
+Measured (`bench/eigen-compare.py`, the MKL build, best of two; load
+average 10 to 16 from other jobs):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| FactorAnalysis(randomized), 200,000 x 100, 10 factors | 7.06 s | 0.11 s | 62x | 56x |
+| FactorAnalysis(lapack), 200,000 x 100, 10 factors | 28.4 s | 0.10 s | 286x | 219x |
+| FactorAnalysis(randomized), 50,000 x 500, 20 factors | 4.37 s | 0.33 s | 13x | 28x |
+| FactorAnalysis(lapack), 50,000 x 500, 20 factors | 22.0 s | 0.33 s | 67x | 106x |
+
+## Least squares: `GramLinearRegression`
+
+scikit-learn's `LinearRegression` over the Gram matrix
+(`examples/kernel-only/linreg.scm`): Xc'Xc and Xc'y in one pass, then
+the normal equations of the column-scaled Gram by Cholesky (dposv),
+where scikit-learn runs LAPACK's gelsd on the n x p design. The normal
+equations square the condition number, so the answer is taken only
+when the scaled Gram's condition (dpocon's estimate) is at most 1e6,
+which bounds the relative error near 1e-10, and the design has full
+rank at `tol`; otherwise, and for `positive=True`, the fit is
+scikit-learn's own (gelsd, or nnls). `rank_` and `singular_` come from
+the Gram's eigenvalues when first read. A sparse X is taken (below).
+
+Measured (`bench/eigen-compare.py`, as above):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| LinearRegression, 500,000 x 100 | 7.25 s | 0.39 s | 19x | 19x |
+| LinearRegression, 200,000 x 500 | 10.9 s | 1.15 s | 9.5x | 50x |
+| LinearRegression, 200,000 x 100, 10 targets | 2.83 s | 0.17 s | 17x | 18x |
+| LinearRegression, 20,000 x 2,000 | 12.3 s | 1.41 s | 8.7x | 123x |
+
+The coefficients agree to 1e-13.
+
+## Sparse X
+
+`GramLasso`, `GramElasticNet`, `GramRidge`, `GramLinearRegression` and
+`GramTruncatedSVD` take a scipy.sparse X (any format; CSR reaches the
+kernels). X'X is formed by `examples/kernel-only/sparse.scm`'s
+`csr-gram` -- each row's outer product into the upper triangle, rows in
+order, blocks of rows on threads into their own G's -- and X'y and X B
+by `csr-mm`; the intercept is the n xm xm' correction, as scikit-learn's
+sparse coordinate descent keeps X uncentred too. After X'X the fits are
+the dense ones, so the answers equal the dense X's to 1e-15 and are
+exact: scikit-learn's sparse Ridge and LinearRegression are iterative
+(sparse_cg, lsqr) and agree with its own dense answers only to their
+tolerance (1e-5 and 1e-7 here).
+
+Measured (`bench/sparse-compare.py`, the MKL build, best of two; load
+average 10 to 29):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| Lasso, 200,000 x 200, 5% dense | 0.16 s | 0.067 s | 2.4x | 2.6x |
+| ElasticNet, 200,000 x 200, 5% dense | 0.16 s | 0.059 s | 2.6x | 3.4x |
+| Ridge, 200,000 x 200, 5% dense | 0.056 s | 0.064 s | 0.9x | 0.9x |
+| LinearRegression, 200,000 x 200, 5% dense | 0.074 s | 0.076 s | 1.0x | 1.3x |
+| TruncatedSVD (randomized), 200,000 x 200, 5% dense | 0.86 s | 0.11 s | 8.1x | 6.2x |
+| TruncatedSVD (arpack), 200,000 x 200, 5% dense | 1.54 s | 0.11 s | 15x | 13x |
+| Lasso, 100,000 x 1,000, 1% dense | 0.14 s | 0.069 s | 2.1x | 3.0x |
+| Ridge, 100,000 x 1,000, 1% dense | 0.043 s | 0.082 s | 0.5x | 0.6x |
+| TruncatedSVD (arpack), 100,000 x 1,000, 1% dense | 1.41 s | 0.16 s | 8.7x | 8.7x |
+| Lasso, 50,000 x 2,000, 0.5% dense | 0.11 s | 0.15 s | 0.7x | 1.2x |
+| Ridge, 50,000 x 2,000, 0.5% dense | 0.024 s | 0.23 s | 0.1x | 0.2x |
+| TruncatedSVD (randomized), 50,000 x 2,000, 0.5% dense | 0.16 s | 1.49 s | 0.1x | 0.6x |
+
+The Gram route pays p^2 memory and p^3 for the solve or the
+eigenproblem whatever the sparsity, where scikit-learn's iterative
+sparse solvers pay a few passes over the nonzeros. So it wins for
+moderate p (a few hundred) and for TruncatedSVD against arpack, and
+loses for p in the thousands with few nonzeros per row: there, Ridge
+and LinearRegression are better left to scikit-learn's sparse solvers
+(at their tolerance), and the randomized TruncatedSVD to scikit-learn.
+
 ## Bayesian ridge
 
 `CovBayesianRidge` is scikit-learn's `BayesianRidge` -- the same model,

scm2cpp_learn-0.18.0/PKG-INFO → scm2cpp_learn-0.20.0/README.md

@@ -1,21 +1,3 @@
-Metadata-Version: 2.4
-Name: scm2cpp-learn
-Version: 0.18.0
-Summary: scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
-Author: Hirotaka Niitsuma
-License: MIT
-Project-URL: Homepage, https://github.com/niitsuma/scm2cpp
-Project-URL: Source, https://github.com/niitsuma/scm2cpp/tree/master/python/scm2cpp-learn
-Keywords: scikit-learn,lasso,elastic-net,ridge,svm,logistic-regression,pca,lda,coordinate-descent,lapack,cuda,compiler
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: C++
-Classifier: Topic :: Scientific/Engineering :: Mathematics
-Classifier: Intended Audience :: Science/Research
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-Requires-Dist: numpy>=1.17
-
 # scm2cpp-learn
 
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/niitsuma/scm2cpp/blob/master/notebooks/compare-sklearn.ipynb)
@@ -206,9 +188,10 @@ package picks the fastest one present.
 
 | class | scikit-learn | Scheme kernel | builds | sklearnex accelerates |
 |---|---|---|---|---|
-| `GramLasso` | Lasso(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
-| `GramElasticNet` | ElasticNet(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
-| `GramRidge` | Ridge, fit(X, y) with the intercept, one or many targets | `gram.scm` (Cholesky; dposv) | loops / BLAS | Ridge |
+| `GramLasso` | Lasso(precompute=True), fit(X, y) with the intercept, dense or sparse X | `lasso-cov.scm` | CPU | no |
+| `GramElasticNet` | ElasticNet(precompute=True), fit(X, y) with the intercept, dense or sparse X | `lasso-cov.scm` | CPU | no |
+| `GramRidge` | Ridge, fit(X, y) with the intercept, one or many targets, dense or sparse X | `gram.scm` (Cholesky; dposv) | loops / BLAS | Ridge |
+| `GramLinearRegression` | LinearRegression, dense or sparse X, sample_weight | `linreg.scm` (Cholesky; dposv, dpocon) | loops / BLAS | no |
 | `GramRidgeCV` | RidgeCV (cv=None), leave-one-out over every alpha from one eigenproblem | `ridge-gcv.scm` | loops / BLAS | no |
 | `GramMultiTaskLasso` | MultiTaskLasso, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramMultiTaskElasticNet` | MultiTaskElasticNet, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
@@ -256,7 +239,9 @@ package picks the fastest one present.
 | `CovPCA` | PCA(svd_solver='covariance_eigh') | `pca.scm` | loops / BLAS / cuBLAS | PCA |
 | `GramKernelPCA` | KernelPCA (no inverse_transform) | `kernel-pca.scm` | loops / BLAS | no |
 | `GramLDA` | LinearDiscriminantAnalysis (svd, lsqr, eigen) | `lda.scm` | loops / BLAS | no |
-| `GramTruncatedSVD` | TruncatedSVD on dense X (exact, as arpack) | `truncated-svd.scm` | loops / BLAS | no |
+| `GramTruncatedSVD` | TruncatedSVD on dense or sparse X (exact, as arpack) | `truncated-svd.scm`, `sparse.scm` | loops / BLAS | no |
+| `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD` | PLSRegression, PLSCanonical, CCA, PLSSVD | `pls.scm` | loops / BLAS | no |
+| `GramFactorAnalysis` | FactorAnalysis (exact, as svd_method='lapack'; rotations) | `fa.scm` | loops / BLAS | no |
 | `f_regression, f_classif` | f_regression, f_classif | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `SelectKBest, SelectPercentile` | SelectKBest, SelectPercentile | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `GramRFE` | RFE(Ridge), RFE(LinearRegression) | `rfe.scm` | loops / BLAS / cuBLAS | inside: Ridge |
@@ -714,6 +699,143 @@ The exact route costs O(n p^2 + p^3); with few components of many
 features (2,000 here) the randomized one does less work, and on one core
 it is faster, at its accuracy. Sparse X is not taken yet.
 
+## PLS and CCA: `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD`
+
+scikit-learn's `cross_decomposition` estimators over the cross products
+(`examples/kernel-only/pls.scm`). NIPALS multiplies the blocks only by
+weight vectors, and each such product is one of Cxx = X'X, Cxy = X'Y,
+Cyy = Y'Y times a vector: the Y score is Y c for a coefficient vector c,
+so X'(Y c) = Cxy c. The deflations are rank-one updates of the three
+matrices (with a = Cxx w, b = Cxy'w and s2 = w'Cxx w, regression mode is
+Cxx -= a a'/s2, Cxy -= a b'/s2, Cyy -= b b'/s2). So X and Y are read
+once, for the three products (dsyrk and dgemm), and every power
+iteration after that costs O(p^2 + pq + q^2) where scikit-learn's costs
+O(n(p + q)). CCA's mode B takes pinv(X) y = Cxx^+ X'y, Cxx^+ from its
+eigenproblem; PLSSVD is the SVD of Cxy.
+
+```python
+from scm2cpp_learn import GramPLSRegression, GramCCA
+pls = GramPLSRegression(n_components=5).fit(X, Y)
+pls.predict(X), pls.transform(X), pls.x_scores_
+```
+
+Weights, loadings, rotations, coefficients, scores and transforms agree
+with scikit-learn's to 1e-13, with the same iteration counts. One cut
+differs: a deflated block loses a rank exactly, which its SVD shows as a
+singular value near eps but its Gram as an eigenvalue near eps
+lam_max, so mode B's pinv also drops eigenvalues under 1e3 eps lam_max
+(singular values under about 5e-7 of the largest, where scikit-learn's
+cut is 2.2e-10). Measured (`bench/eigen-compare.py`, the MKL build, best
+of three):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| PLSRegression, 200,000 x 100 -> 5, 5 components | 5.64 s | 0.26 s | 21x | 19x |
+| PLSCanonical, 200,000 x 100 -> 5, 5 components | 5.63 s | 0.14 s | 40x | 59x |
+| CCA, 200,000 x 100 -> 5, 5 components | 31.9 s | 0.14 s | 220x | 190x |
+| PLSSVD, 200,000 x 100 -> 5, 5 components | 0.28 s | 0.14 s | 2.0x | 4.0x |
+| PLSRegression, 50,000 x 500 -> 10, 10 components | 45.5 s | 0.75 s | 61x | 57x |
+| PLSCanonical, 50,000 x 500 -> 10, 10 components | 45.3 s | 0.60 s | 76x | 100x |
+| CCA, 50,000 x 500 -> 10, 10 components | 132 s | 1.21 s | 109x | 151x |
+| PLSSVD, 50,000 x 500 -> 10, 10 components | 0.34 s | 0.34 s | 1.0x | 3.4x |
+
+PLSSVD is one SVD of X'Y on both sides, so only the product differs.
+
+## Factor analysis: `GramFactorAnalysis`
+
+scikit-learn's `FactorAnalysis` over the covariance
+(`examples/kernel-only/fa.scm`). Every iteration of scikit-learn's takes
+the SVD of the n x p matrix Xc / (sqrt(psi) sqrt(n)); its squared
+singular values and right singular vectors are the eigenpairs of
+D^-1 C D^-1, C = Xc'Xc / n and D = diag(sqrt psi). So C is formed once
+(dsyrk) and each iteration is the top `n_components` eigenpairs of a
+p x p matrix (dsyevr), then scikit-learn's own updates of the
+components, the log-likelihood and psi. The eigenproblem is exact: what
+`svd_method="lapack"` computes, which the fit matches to 1e-13 with the
+same iterations; scikit-learn's default `"randomized"` approximates it
+(here to 1e-6 in psi). Rotations (varimax, quartimax), `transform`,
+`score`, `get_covariance` and `get_precision` are scikit-learn's.
+
+```python
+from scm2cpp_learn import GramFactorAnalysis
+fa = GramFactorAnalysis(n_components=10).fit(X)
+fa.components_, fa.noise_variance_, fa.transform(X)
+```
+
+Measured (`bench/eigen-compare.py`, the MKL build, best of two; load
+average 10 to 16 from other jobs):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| FactorAnalysis(randomized), 200,000 x 100, 10 factors | 7.06 s | 0.11 s | 62x | 56x |
+| FactorAnalysis(lapack), 200,000 x 100, 10 factors | 28.4 s | 0.10 s | 286x | 219x |
+| FactorAnalysis(randomized), 50,000 x 500, 20 factors | 4.37 s | 0.33 s | 13x | 28x |
+| FactorAnalysis(lapack), 50,000 x 500, 20 factors | 22.0 s | 0.33 s | 67x | 106x |
+
+## Least squares: `GramLinearRegression`
+
+scikit-learn's `LinearRegression` over the Gram matrix
+(`examples/kernel-only/linreg.scm`): Xc'Xc and Xc'y in one pass, then
+the normal equations of the column-scaled Gram by Cholesky (dposv),
+where scikit-learn runs LAPACK's gelsd on the n x p design. The normal
+equations square the condition number, so the answer is taken only
+when the scaled Gram's condition (dpocon's estimate) is at most 1e6,
+which bounds the relative error near 1e-10, and the design has full
+rank at `tol`; otherwise, and for `positive=True`, the fit is
+scikit-learn's own (gelsd, or nnls). `rank_` and `singular_` come from
+the Gram's eigenvalues when first read. A sparse X is taken (below).
+
+Measured (`bench/eigen-compare.py`, as above):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| LinearRegression, 500,000 x 100 | 7.25 s | 0.39 s | 19x | 19x |
+| LinearRegression, 200,000 x 500 | 10.9 s | 1.15 s | 9.5x | 50x |
+| LinearRegression, 200,000 x 100, 10 targets | 2.83 s | 0.17 s | 17x | 18x |
+| LinearRegression, 20,000 x 2,000 | 12.3 s | 1.41 s | 8.7x | 123x |
+
+The coefficients agree to 1e-13.
+
+## Sparse X
+
+`GramLasso`, `GramElasticNet`, `GramRidge`, `GramLinearRegression` and
+`GramTruncatedSVD` take a scipy.sparse X (any format; CSR reaches the
+kernels). X'X is formed by `examples/kernel-only/sparse.scm`'s
+`csr-gram` -- each row's outer product into the upper triangle, rows in
+order, blocks of rows on threads into their own G's -- and X'y and X B
+by `csr-mm`; the intercept is the n xm xm' correction, as scikit-learn's
+sparse coordinate descent keeps X uncentred too. After X'X the fits are
+the dense ones, so the answers equal the dense X's to 1e-15 and are
+exact: scikit-learn's sparse Ridge and LinearRegression are iterative
+(sparse_cg, lsqr) and agree with its own dense answers only to their
+tolerance (1e-5 and 1e-7 here).
+
+Measured (`bench/sparse-compare.py`, the MKL build, best of two; load
+average 10 to 29):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| Lasso, 200,000 x 200, 5% dense | 0.16 s | 0.067 s | 2.4x | 2.6x |
+| ElasticNet, 200,000 x 200, 5% dense | 0.16 s | 0.059 s | 2.6x | 3.4x |
+| Ridge, 200,000 x 200, 5% dense | 0.056 s | 0.064 s | 0.9x | 0.9x |
+| LinearRegression, 200,000 x 200, 5% dense | 0.074 s | 0.076 s | 1.0x | 1.3x |
+| TruncatedSVD (randomized), 200,000 x 200, 5% dense | 0.86 s | 0.11 s | 8.1x | 6.2x |
+| TruncatedSVD (arpack), 200,000 x 200, 5% dense | 1.54 s | 0.11 s | 15x | 13x |
+| Lasso, 100,000 x 1,000, 1% dense | 0.14 s | 0.069 s | 2.1x | 3.0x |
+| Ridge, 100,000 x 1,000, 1% dense | 0.043 s | 0.082 s | 0.5x | 0.6x |
+| TruncatedSVD (arpack), 100,000 x 1,000, 1% dense | 1.41 s | 0.16 s | 8.7x | 8.7x |
+| Lasso, 50,000 x 2,000, 0.5% dense | 0.11 s | 0.15 s | 0.7x | 1.2x |
+| Ridge, 50,000 x 2,000, 0.5% dense | 0.024 s | 0.23 s | 0.1x | 0.2x |
+| TruncatedSVD (randomized), 50,000 x 2,000, 0.5% dense | 0.16 s | 1.49 s | 0.1x | 0.6x |
+
+The Gram route pays p^2 memory and p^3 for the solve or the
+eigenproblem whatever the sparsity, where scikit-learn's iterative
+sparse solvers pay a few passes over the nonzeros. So it wins for
+moderate p (a few hundred) and for TruncatedSVD against arpack, and
+loses for p in the thousands with few nonzeros per row: there, Ridge
+and LinearRegression are better left to scikit-learn's sparse solvers
+(at their tolerance), and the randomized TruncatedSVD to scikit-learn.
+
 ## Bayesian ridge
 
 `CovBayesianRidge` is scikit-learn's `BayesianRidge` -- the same model,

{scm2cpp_learn-0.18.0 → scm2cpp_learn-0.20.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scm2cpp-learn"
-version = "0.18.0"
-description = "scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included"
+version = "0.20.0"
+description = "scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, PLS, CCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included"
 readme = "README.md"
 requires-python = ">=3.8"
 license = {text = "MIT"}