TensorFox 1.2.1__tar.gz → 1.2.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {TensorFox-1.2.1/modules/TensorFox.egg-info → tensorfox-1.2.2}/PKG-INFO +34 -11
- TensorFox-1.2.1/PKG-INFO → tensorfox-1.2.2/README.md +14 -21
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/Auxiliar.py +40 -1
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/GaussNewton.py +25 -9
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/MultilinearAlgebra.py +14 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/TensorFox.py +9 -6
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/TensorLearning.py +1 -1
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/__init__.py +2 -2
- TensorFox-1.2.1/README.md → tensorfox-1.2.2/modules/TensorFox.egg-info/PKG-INFO +44 -6
- {TensorFox-1.2.1 → tensorfox-1.2.2}/setup.py +1 -1
- {TensorFox-1.2.1 → tensorfox-1.2.2}/LICENSE +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/Alternating_Least_Squares.py +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/Compression.py +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/Conversion.py +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/Critical.py +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/Display.py +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox/Initialization.py +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox.egg-info/SOURCES.txt +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox.egg-info/dependency_links.txt +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox.egg-info/requires.txt +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/modules/TensorFox.egg-info/top_level.txt +0 -0
- {TensorFox-1.2.1 → tensorfox-1.2.2}/setup.cfg +0 -0
|
@@ -1,15 +1,32 @@
|
|
|
1
|
-
Metadata-Version: 2.
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
2
|
Name: TensorFox
|
|
3
|
-
Version: 1.2.
|
|
3
|
+
Version: 1.2.2
|
|
4
4
|
Summary: Tensor Fox is a high performance package of multilinear algebra and tensor routines, with focus on the Canonical Polyadic Decomposition (CPD), also called PARAFAC or CANDECOMP.
|
|
5
5
|
Home-page: https://github.com/felipebottega/Tensor-Fox
|
|
6
6
|
Author: Felipe Bottega Diniz
|
|
7
7
|
Author-email: felipebottega@gmail.com
|
|
8
8
|
License: GNU
|
|
9
9
|
Keywords: Tensor Fox Canonical Polyadic Decomposition CPD PARAFAC CANDECOMP Multilinear Learning
|
|
10
|
-
Platform: UNKNOWN
|
|
11
10
|
Description-Content-Type: text/markdown
|
|
12
11
|
License-File: LICENSE
|
|
12
|
+
Requires-Dist: numpy>=1.21.0
|
|
13
|
+
Requires-Dist: pandas>=1.2.3
|
|
14
|
+
Requires-Dist: scipy>=1.6.2
|
|
15
|
+
Requires-Dist: scikit-learn>=0.24.1
|
|
16
|
+
Requires-Dist: matplotlib>=3.3.4
|
|
17
|
+
Requires-Dist: numba>=0.53.1
|
|
18
|
+
Requires-Dist: IPython>=7.31.1
|
|
19
|
+
Requires-Dist: sparse_dot_mkl>=0.7
|
|
20
|
+
Dynamic: author
|
|
21
|
+
Dynamic: author-email
|
|
22
|
+
Dynamic: description
|
|
23
|
+
Dynamic: description-content-type
|
|
24
|
+
Dynamic: home-page
|
|
25
|
+
Dynamic: keywords
|
|
26
|
+
Dynamic: license
|
|
27
|
+
Dynamic: license-file
|
|
28
|
+
Dynamic: requires-dist
|
|
29
|
+
Dynamic: summary
|
|
13
30
|
|
|
14
31
|
# Tensor Fox
|
|
15
32
|
|
|
@@ -41,12 +58,19 @@ The only pre-requisite is to have **Python 3** installed. After this you just ne
|
|
|
41
58
|
|
|
42
59
|
pip install TensorFox
|
|
43
60
|
|
|
44
|
-
The package `sparse_dot_mkl` won't work properly with `pip` (we explain in more detail at the end of this text). This package is responsible for accelerating
|
|
61
|
+
The package `sparse_dot_mkl` won't work properly with `pip` (we explain in more detail at the end of this text). This package is responsible for accelerating sparse matrix multiplications. Tensor Fox still works without this package but it will use Scipy dot instead. I recommend installing [Miniconda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) and then create an env with the following:
|
|
45
62
|
|
|
46
63
|
conda create --name tfx_env --channel defaults jupyter numpy pandas scipy scikit-learn matplotlib numba IPython sparse_dot_mkl
|
|
64
|
+
conda install -c conda-forge sparse_dot_mkl
|
|
47
65
|
|
|
48
66
|
After this you may install TensorFox and everything will be fine. The channel must be `defaults`, otherwise Numpy won't be linked against MKL (see https://numpy.org/install/), which is necessary to work with `sparse_dot_mkl`. Finally, note that `jupyter` is optional, only if you want to work with jupyter notebooks.
|
|
49
67
|
|
|
68
|
+
### Tensor Fox Standalone Environment (only for Windows)
|
|
69
|
+
|
|
70
|
+
If you prefer not to install Tensor Fox via pip, [this release](https://github.com/felipebottega/Tensor-Fox/releases/) provides a portable standalone environment with Python and all required dependencies already included. Simply extract the archive and follow the instructions in this release to start using Tensor Fox.
|
|
71
|
+
|
|
72
|
+
The standalone environment is intended to simplify setup and provide a reproducible execution environment. It contains the exact dependency versions used during development, helping ensure consistent behavior across compatible Windows systems.
|
|
73
|
+
|
|
50
74
|
### Creating a tensor
|
|
51
75
|
|
|
52
76
|
Let's start importing Tensor Fox and other necessary modules.
|
|
@@ -187,7 +211,7 @@ In this section we summarize all the features Tensor Fox has to offer. As alread
|
|
|
187
211
|
|
|
188
212
|
### Sparse Dot MKL requirements
|
|
189
213
|
|
|
190
|
-
This package requires the MKL runtime linking library `libmkl_rt.so` (or `libmkl_rt.dylib` for OSX, or `mkl_rt.dll` for WIN). If the MKL library cannot be loaded an `ImportError` will be raised when the package is first imported. MKL is distributed with the full version of conda, and can be installed into Miniconda with `conda install -c intel mkl`. Alternatively, you may
|
|
214
|
+
This package requires the MKL runtime linking library `libmkl_rt.so` (or `libmkl_rt.dylib` for OSX, or `mkl_rt.dll` for WIN). If the MKL library cannot be loaded an `ImportError` will be raised when the package is first imported. MKL is distributed with the full version of conda, and can be installed into Miniconda with `conda install -c intel mkl`. Alternatively, you may need to add the path to MKL shared objects to `LD_LIBRARY_PATH` (e.g. `export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64:$LD_LIBRARY_PATH`). There are some known bugs in MKL v2019 which may cause intermittent segfaults. Updating to MKL v2020 is highly recommended if you encounter any problems.
|
|
191
215
|
|
|
192
216
|
### Sparse Dot MKL environment variable
|
|
193
217
|
|
|
@@ -226,9 +250,8 @@ This project is licensed under the GNU GENERAL PUBLIC LICENSE - see the [LICENSE
|
|
|
226
250
|
8) S. Rabanser, O. Shchur, and S. Günnemann, *Introduction to Tensor Decompositions and their Applications in Machine Learning*, arXiv:1711.10781v1 (2017).
|
|
227
251
|
9) A. H. Phan, P. Tichavsky, and A. Cichoki, *Low Complexity Damped Gauss-Newton Algorithm for CANDECOMP/PARAFAC*, SIAM Journal on Matrix Analysis and Applications, 34 (1), 126-147 (2013).
|
|
228
252
|
10) L. De Lathauwer, B. De Moor, and J. Vandewalle, *A Multilinear Singular Value Decomposition*, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278.
|
|
229
|
-
11)
|
|
230
|
-
12)
|
|
231
|
-
13)
|
|
232
|
-
14)
|
|
233
|
-
|
|
234
|
-
|
|
253
|
+
11) F. Bottega Diniz, PhD Thesis, https://www.pgmat.im.ufrj.br/index.php/pt-br/teses-e-dissertacoes/teses/2019-1/281-tensor-decompositions-and-algorithms-with-applications-to-tensor-learning/file
|
|
254
|
+
12) Tensorlab - https://www.tensorlab.net/
|
|
255
|
+
13) Tensor Toolbox - http://www.sandia.gov/~tgkolda/TensorToolbox/
|
|
256
|
+
14) Tensorly - https://github.com/tensorly/
|
|
257
|
+
15) TensorBox - https://github.com/phananhhuy/TensorBox
|
|
@@ -1,16 +1,3 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: TensorFox
|
|
3
|
-
Version: 1.2.1
|
|
4
|
-
Summary: Tensor Fox is a high performance package of multilinear algebra and tensor routines, with focus on the Canonical Polyadic Decomposition (CPD), also called PARAFAC or CANDECOMP.
|
|
5
|
-
Home-page: https://github.com/felipebottega/Tensor-Fox
|
|
6
|
-
Author: Felipe Bottega Diniz
|
|
7
|
-
Author-email: felipebottega@gmail.com
|
|
8
|
-
License: GNU
|
|
9
|
-
Keywords: Tensor Fox Canonical Polyadic Decomposition CPD PARAFAC CANDECOMP Multilinear Learning
|
|
10
|
-
Platform: UNKNOWN
|
|
11
|
-
Description-Content-Type: text/markdown
|
|
12
|
-
License-File: LICENSE
|
|
13
|
-
|
|
14
1
|
# Tensor Fox
|
|
15
2
|
|
|
16
3
|
Tensor Fox is a high performance package of multilinear algebra and tensor routines, with focus on the Canonical Polyadic Decomposition (CPD), also called PARAFAC or CANDECOMP. We recommend [the wikipedia article](https://en.wikipedia.org/wiki/Tensor_rank_decomposition) for a introduction to this decomposition. In this package you will find an efficient CPD solver, a great variety of parameters for fine tuning, support to sparse tensors, routines to test models and tensor behavior, rank estimate, statistics, and many more!
|
|
@@ -41,12 +28,19 @@ The only pre-requisite is to have **Python 3** installed. After this you just ne
|
|
|
41
28
|
|
|
42
29
|
pip install TensorFox
|
|
43
30
|
|
|
44
|
-
The package `sparse_dot_mkl` won't work properly with `pip` (we explain in more detail at the end of this text). This package is responsible for accelerating
|
|
31
|
+
The package `sparse_dot_mkl` won't work properly with `pip` (we explain in more detail at the end of this text). This package is responsible for accelerating sparse matrix multiplications. Tensor Fox still works without this package but it will use Scipy dot instead. I recommend installing [Miniconda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) and then create an env with the following:
|
|
45
32
|
|
|
46
33
|
conda create --name tfx_env --channel defaults jupyter numpy pandas scipy scikit-learn matplotlib numba IPython sparse_dot_mkl
|
|
34
|
+
conda install -c conda-forge sparse_dot_mkl
|
|
47
35
|
|
|
48
36
|
After this you may install TensorFox and everything will be fine. The channel must be `defaults`, otherwise Numpy won't be linked against MKL (see https://numpy.org/install/), which is necessary to work with `sparse_dot_mkl`. Finally, note that `jupyter` is optional, only if you want to work with jupyter notebooks.
|
|
49
37
|
|
|
38
|
+
### Tensor Fox Standalone Environment (only for Windows)
|
|
39
|
+
|
|
40
|
+
If you prefer not to install Tensor Fox via pip, [this release](https://github.com/felipebottega/Tensor-Fox/releases/) provides a portable standalone environment with Python and all required dependencies already included. Simply extract the archive and follow the instructions in this release to start using Tensor Fox.
|
|
41
|
+
|
|
42
|
+
The standalone environment is intended to simplify setup and provide a reproducible execution environment. It contains the exact dependency versions used during development, helping ensure consistent behavior across compatible Windows systems.
|
|
43
|
+
|
|
50
44
|
### Creating a tensor
|
|
51
45
|
|
|
52
46
|
Let's start importing Tensor Fox and other necessary modules.
|
|
@@ -187,7 +181,7 @@ In this section we summarize all the features Tensor Fox has to offer. As alread
|
|
|
187
181
|
|
|
188
182
|
### Sparse Dot MKL requirements
|
|
189
183
|
|
|
190
|
-
This package requires the MKL runtime linking library `libmkl_rt.so` (or `libmkl_rt.dylib` for OSX, or `mkl_rt.dll` for WIN). If the MKL library cannot be loaded an `ImportError` will be raised when the package is first imported. MKL is distributed with the full version of conda, and can be installed into Miniconda with `conda install -c intel mkl`. Alternatively, you may
|
|
184
|
+
This package requires the MKL runtime linking library `libmkl_rt.so` (or `libmkl_rt.dylib` for OSX, or `mkl_rt.dll` for WIN). If the MKL library cannot be loaded an `ImportError` will be raised when the package is first imported. MKL is distributed with the full version of conda, and can be installed into Miniconda with `conda install -c intel mkl`. Alternatively, you may need to add the path to MKL shared objects to `LD_LIBRARY_PATH` (e.g. `export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64:$LD_LIBRARY_PATH`). There are some known bugs in MKL v2019 which may cause intermittent segfaults. Updating to MKL v2020 is highly recommended if you encounter any problems.
|
|
191
185
|
|
|
192
186
|
### Sparse Dot MKL environment variable
|
|
193
187
|
|
|
@@ -226,9 +220,8 @@ This project is licensed under the GNU GENERAL PUBLIC LICENSE - see the [LICENSE
|
|
|
226
220
|
8) S. Rabanser, O. Shchur, and S. Günnemann, *Introduction to Tensor Decompositions and their Applications in Machine Learning*, arXiv:1711.10781v1 (2017).
|
|
227
221
|
9) A. H. Phan, P. Tichavsky, and A. Cichoki, *Low Complexity Damped Gauss-Newton Algorithm for CANDECOMP/PARAFAC*, SIAM Journal on Matrix Analysis and Applications, 34 (1), 126-147 (2013).
|
|
228
222
|
10) L. De Lathauwer, B. De Moor, and J. Vandewalle, *A Multilinear Singular Value Decomposition*, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278.
|
|
229
|
-
11)
|
|
230
|
-
12)
|
|
231
|
-
13)
|
|
232
|
-
14)
|
|
233
|
-
|
|
234
|
-
|
|
223
|
+
11) F. Bottega Diniz, PhD Thesis, https://www.pgmat.im.ufrj.br/index.php/pt-br/teses-e-dissertacoes/teses/2019-1/281-tensor-decompositions-and-algorithms-with-applications-to-tensor-learning/file
|
|
224
|
+
12) Tensorlab - https://www.tensorlab.net/
|
|
225
|
+
13) Tensor Toolbox - http://www.sandia.gov/~tgkolda/TensorToolbox/
|
|
226
|
+
14) Tensorly - https://github.com/tensorly/
|
|
227
|
+
15) TensorBox - https://github.com/phananhhuy/TensorBox
|
|
@@ -168,6 +168,7 @@ def output_info(T1, Tsize, T1_approx,
|
|
|
168
168
|
self.improv = [improv_main, improv_refine]
|
|
169
169
|
self.gradients = [gradients_main, gradients_refine]
|
|
170
170
|
self.stop = [stop_main, stop_refine]
|
|
171
|
+
self.conditioning = options.conditioning
|
|
171
172
|
self.options = options
|
|
172
173
|
|
|
173
174
|
def stop_msg(self):
|
|
@@ -281,6 +282,7 @@ def make_options(options):
|
|
|
281
282
|
self.gpu = False
|
|
282
283
|
self.mkl_dot = True
|
|
283
284
|
self.svd_n_components = None
|
|
285
|
+
self.conditioning = []
|
|
284
286
|
|
|
285
287
|
temp_options = temp_options()
|
|
286
288
|
|
|
@@ -351,6 +353,8 @@ def make_options(options):
|
|
|
351
353
|
temp_options.mkl_dot = options.mkl_dot
|
|
352
354
|
if 'svd_n_components' in dir(options):
|
|
353
355
|
temp_options.svd_n_components = options.svd_n_components
|
|
356
|
+
if 'conditioning' in dir(options):
|
|
357
|
+
temp_options.conditioning = options.conditioning
|
|
354
358
|
|
|
355
359
|
# If gpu is True, the variable mlsvd_method is set to 'gpu', which is a special strategy aiming to minimize the
|
|
356
360
|
# memory size of the data passed to the GPU. This strategy is based on the classic MLSVD method. In the case the
|
|
@@ -591,7 +595,7 @@ def gen_rand_sparse_tensor(dims, R, nnz):
|
|
|
591
595
|
|
|
592
596
|
Output
|
|
593
597
|
------
|
|
594
|
-
data: float 1-D
|
|
598
|
+
data: float 1-D array
|
|
595
599
|
data[i] is the nonzero value of the tensor at index idxs[i, :].
|
|
596
600
|
idxs: int 2-D array
|
|
597
601
|
Let nnz be the number of nonzero entries of the tensor. Then idxs is an array
|
|
@@ -641,7 +645,42 @@ def gen_rand_sparse_tensor(dims, R, nnz):
|
|
|
641
645
|
for l in range(L):
|
|
642
646
|
if (j[l], r) in orig_factors[l].keys():
|
|
643
647
|
p *= orig_factors[l][j[l], r]
|
|
648
|
+
else:
|
|
649
|
+
p = 0
|
|
650
|
+
break
|
|
644
651
|
tmp += p
|
|
645
652
|
data.append(tmp)
|
|
653
|
+
|
|
654
|
+
# Examine the values to remove possible remaining zeros.
|
|
655
|
+
for i in reversed(range(len(data))):
|
|
656
|
+
if data[i] == 0:
|
|
657
|
+
del data[i]
|
|
658
|
+
del idxs[i]
|
|
646
659
|
|
|
647
660
|
return array(data), array(idxs), array(dims), orig_factors
|
|
661
|
+
|
|
662
|
+
|
|
663
|
+
def sparse2txt(idxs, data, filename, start=1):
|
|
664
|
+
"""
|
|
665
|
+
Converts a sparse tensor in text format and saves it. The format is such that
|
|
666
|
+
each row contains a sequence of the index values and the tensor value.
|
|
667
|
+
|
|
668
|
+
Input
|
|
669
|
+
-----
|
|
670
|
+
idxs, data: float 1-D arrays
|
|
671
|
+
The same arrays of the sparse tensor used in Tensor Fox.
|
|
672
|
+
filename: str
|
|
673
|
+
The name of the text file where the tensor wil be saved.
|
|
674
|
+
start: int
|
|
675
|
+
In Tensor Fox the indexes starts at zero, but the same may not be true for
|
|
676
|
+
other applications where the text file will be used. This parameter is a
|
|
677
|
+
value to be added on the indexes.
|
|
678
|
+
"""
|
|
679
|
+
|
|
680
|
+
with open(filename, 'w') as f:
|
|
681
|
+
for i, d in zip(idxs, data):
|
|
682
|
+
i = i + start
|
|
683
|
+
line = ''.join([f'{x} ' for x in (i + 1)]) + str(d) + '\n'
|
|
684
|
+
f.write(line)
|
|
685
|
+
|
|
686
|
+
return
|
|
@@ -115,6 +115,7 @@ def dGN(T, factors, R, options):
|
|
|
115
115
|
|
|
116
116
|
# The program is encouraged to make more iterations for small problems.
|
|
117
117
|
if R * sum(dims) <= 100:
|
|
118
|
+
print("-> Small problem detected (R * sum(dims) <= 100). All the convergence tolerances (tol, tol_step, tol_improv, tol_grad) have been set to zero.", file=sys.stderr)
|
|
118
119
|
tol = 0
|
|
119
120
|
tol_step = 0
|
|
120
121
|
tol_improv = 0
|
|
@@ -157,7 +158,7 @@ def dGN(T, factors, R, options):
|
|
|
157
158
|
'| {:^10}'.format('Predicted error'),
|
|
158
159
|
'| {:^10}'.format('# Inner iterations'))
|
|
159
160
|
|
|
160
|
-
|
|
161
|
+
# START GAUSS-NEWTON ITERATIONS
|
|
161
162
|
|
|
162
163
|
for it in range(maxiter):
|
|
163
164
|
# Keep the previous value of x and error to compare with the new ones in the next iteration.
|
|
@@ -165,10 +166,10 @@ def dGN(T, factors, R, options):
|
|
|
165
166
|
old_error = error
|
|
166
167
|
|
|
167
168
|
# Computation of the Gauss-Newton iteration formula to obtain the new point x + y, where x is the
|
|
168
|
-
# previous point and y is the new step obtained as the solution of min_y |Ay - b
|
|
169
|
+
# previous point and y is the new step obtained as the solution of min_y |Ay - b|.
|
|
169
170
|
inner_parameters = damp, inner_method, cg_maxiter, cg_factor, cg_tol, tol_jump, symm, factors_norm, fix_mode
|
|
170
171
|
T1_approx, factors, x, y, grad, itn, residualnorm, error = \
|
|
171
|
-
compute_step(Tsize, Tl, T1_approx, factors, orig_factors, data, x, y, inner_parameters, it, old_error)
|
|
172
|
+
compute_step(Tsize, Tl, T1_approx, factors, orig_factors, data, x, y, inner_parameters, it, old_error, options)
|
|
172
173
|
|
|
173
174
|
# Update gain ratio and damping parameter.
|
|
174
175
|
gain_ratio = update_gain_ratio(damp, old_error, error, Tsize, old_x, x, grad)
|
|
@@ -251,7 +252,7 @@ def dGN(T, factors, R, options):
|
|
|
251
252
|
return best_factors, step_sizes, errors, improv, gradients, stop
|
|
252
253
|
|
|
253
254
|
|
|
254
|
-
def compute_step(Tsize, Tl, T1_approx, factors, orig_factors, data, x, y, inner_parameters, it, old_error):
|
|
255
|
+
def compute_step(Tsize, Tl, T1_approx, factors, orig_factors, data, x, y, inner_parameters, it, old_error, options):
|
|
255
256
|
"""
|
|
256
257
|
This function uses the chosen inner method to compute the next step.
|
|
257
258
|
"""
|
|
@@ -265,10 +266,10 @@ def compute_step(Tsize, Tl, T1_approx, factors, orig_factors, data, x, y, inner_
|
|
|
265
266
|
# Call the inner method.
|
|
266
267
|
if inner_method == 'cg':
|
|
267
268
|
cg_maxiter = 1 + (L-2) * int(cg_factor * randint(1 + it**0.4, 2 + it**0.9))
|
|
268
|
-
y, grad, JT_J_grad, itn, residualnorm = cg(Tl, factors, data, y, damp, cg_maxiter, cg_tol)
|
|
269
|
+
y, grad, JT_J_grad, itn, residualnorm = cg(Tl, factors, data, y, damp, cg_maxiter, cg_tol, options)
|
|
269
270
|
|
|
270
271
|
elif inner_method == 'cg_static':
|
|
271
|
-
y, grad, JT_J_grad, itn, residualnorm = cg(Tl, factors, data, y, damp, cg_maxiter, cg_tol)
|
|
272
|
+
y, grad, JT_J_grad, itn, residualnorm = cg(Tl, factors, data, y, damp, cg_maxiter, cg_tol, options)
|
|
272
273
|
|
|
273
274
|
elif inner_method == 'als':
|
|
274
275
|
factors = alsq.als_iteration(Tl, factors, fix_mode)
|
|
@@ -312,7 +313,7 @@ def compute_step(Tsize, Tl, T1_approx, factors, orig_factors, data, x, y, inner_
|
|
|
312
313
|
return T1_approx, factors, x, y, grad, itn, residualnorm, error
|
|
313
314
|
|
|
314
315
|
|
|
315
|
-
def cg(Tl, factors, data, y, damp, maxiter, tol):
|
|
316
|
+
def cg(Tl, factors, data, y, damp, maxiter, tol, options):
|
|
316
317
|
"""
|
|
317
318
|
Conjugate gradient algorithm specialized to the tensor case.
|
|
318
319
|
"""
|
|
@@ -355,6 +356,16 @@ def cg(Tl, factors, data, y, damp, maxiter, tol):
|
|
|
355
356
|
# CG iterations.
|
|
356
357
|
y, itn, residualnorm = cg_iterations(factors, P1, P2, A, B, P_VT_W, result, result_tmp, M, P,
|
|
357
358
|
Gamma, damp, z, residual_cg, residualnorm, y, tol, maxiter, dims, sum_dims)
|
|
359
|
+
|
|
360
|
+
# Save condition numbers of intermediate matrices. Only for display = 4, since these are heavy computations.
|
|
361
|
+
if options.display == 4 and not options.refine:
|
|
362
|
+
H = hessian(factors, P1, P2, sum_dims)
|
|
363
|
+
Hd = H + damp * diag(Gamma)
|
|
364
|
+
Hp = M[:, None] * Hd * M[None, :]
|
|
365
|
+
cond_H = np.linalg.cond(H)
|
|
366
|
+
cond_Hd = np.linalg.cond(Hd)
|
|
367
|
+
cond_Hp = np.linalg.cond(Hp)
|
|
368
|
+
options.conditioning.append([cond_H, cond_Hd, cond_Hp, damp])
|
|
358
369
|
|
|
359
370
|
return M * y, grad, JT_J_grad, itn + 1, residualnorm
|
|
360
371
|
|
|
@@ -688,8 +699,13 @@ def hessian(factors, P1, P2, sum_dims):
|
|
|
688
699
|
tmp2 = zeros((R*dims[l], R*dims[ll]))
|
|
689
700
|
tmp2 = compute_blocks(tmp2, fortran_factors[l], vec_factors[ll], tuple(dims), R, l, ll)
|
|
690
701
|
# Blocks H_{l, ll} and H_{ll, l}.
|
|
691
|
-
|
|
692
|
-
|
|
702
|
+
if tmp1.shape[0] == tmp1.shape[1]:
|
|
703
|
+
H[sum_dims[l]:sum_dims[l+1], sum_dims[ll]:sum_dims[ll+1]] = \
|
|
704
|
+
mlinalg.hadamard(tmp1, tmp2, H[sum_dims[l]:sum_dims[l+1], sum_dims[ll]:sum_dims[ll+1]])
|
|
705
|
+
else:
|
|
706
|
+
H[sum_dims[l]:sum_dims[l+1], sum_dims[ll]:sum_dims[ll+1]] = \
|
|
707
|
+
mlinalg.hadamard_rect(tmp1, tmp2, H[sum_dims[l]:sum_dims[l+1], sum_dims[ll]:sum_dims[ll+1]])
|
|
708
|
+
|
|
693
709
|
H[sum_dims[ll]:sum_dims[ll+1], sum_dims[l]:sum_dims[l+1]] = H[sum_dims[l]:sum_dims[l+1], sum_dims[ll]:sum_dims[ll+1]].T
|
|
694
710
|
|
|
695
711
|
return H
|
|
@@ -288,6 +288,20 @@ def hadamard(A, B, M):
|
|
|
288
288
|
M[r, rr] = A[r, rr]*B[r, rr]
|
|
289
289
|
|
|
290
290
|
return M
|
|
291
|
+
|
|
292
|
+
@njit(nogil=True)
|
|
293
|
+
def hadamard_rect(A, B, M):
|
|
294
|
+
"""
|
|
295
|
+
Hadamard for rectangular matrices.
|
|
296
|
+
"""
|
|
297
|
+
|
|
298
|
+
nrows, ncols = A.shape
|
|
299
|
+
|
|
300
|
+
for i in range(nrows):
|
|
301
|
+
for j in range(ncols):
|
|
302
|
+
M[i, j] = A[i, j] * B[i, j]
|
|
303
|
+
|
|
304
|
+
return M
|
|
291
305
|
|
|
292
306
|
|
|
293
307
|
def cond(factors):
|
|
@@ -212,6 +212,9 @@ def cpd(T, R, options=False):
|
|
|
212
212
|
tensors, where L is the number of dimensions (modes), R is the rank and dims[l] is the l-th dimension of
|
|
213
213
|
the tensors. The number of components for each dimension is an upper bound for the dimension of the
|
|
214
214
|
compressed tensor. Higher values may lead to better approximations with a cost on computational cost.
|
|
215
|
+
conditioning: list
|
|
216
|
+
Save condition numbers of intermediate matrices in the CG method. Only for display = 4 and it is not applied
|
|
217
|
+
to the refine stage, since these are heavy computations.
|
|
215
218
|
|
|
216
219
|
It is not necessary to create 'options' with all parameters described above. Any missing parameter is assigned to
|
|
217
220
|
its default value automatically. For a tutorial about the options, check the Tensor Fox tutorial at
|
|
@@ -301,13 +304,13 @@ def cpd(T, R, options=False):
|
|
|
301
304
|
if prod(array(S.shape) == array(dims)):
|
|
302
305
|
if tol_mlsvd == -1:
|
|
303
306
|
print(' No compression and no truncation requested by user')
|
|
304
|
-
print(' Working with dimensions', tuple(dims))
|
|
307
|
+
print(' Working with dimensions', tuple(map(int, dims)))
|
|
305
308
|
else:
|
|
306
309
|
print(' No compression detected')
|
|
307
|
-
print(' Working with dimensions', tuple(dims))
|
|
310
|
+
print(' Working with dimensions', tuple(map(int, dims)))
|
|
308
311
|
else:
|
|
309
312
|
print(' Compression detected')
|
|
310
|
-
print(' Compressing from', tuple(dims), 'to', S.shape)
|
|
313
|
+
print(' Compressing from', tuple(map(int, dims)), 'to', S.shape)
|
|
311
314
|
if display > 2 or display < -1:
|
|
312
315
|
print(' Compression relative error = {:7e}'.format(best_error))
|
|
313
316
|
print()
|
|
@@ -531,13 +534,13 @@ def tricpd(T, R, options):
|
|
|
531
534
|
if prod(array(dims_cmpr) == array(dims)):
|
|
532
535
|
if tol_mlsvd == -1:
|
|
533
536
|
print(' No compression and no truncation requested by user')
|
|
534
|
-
print(' Working with dimensions', tuple(dims))
|
|
537
|
+
print(' Working with dimensions', tuple(map(int, dims)))
|
|
535
538
|
else:
|
|
536
539
|
print(' No compression detected')
|
|
537
|
-
print(' Working with dimensions', tuple(dims))
|
|
540
|
+
print(' Working with dimensions', tuple(map(int, dims)))
|
|
538
541
|
else:
|
|
539
542
|
print(' Compression detected')
|
|
540
|
-
print(' Compressing from', tuple(dims), 'to', S.shape)
|
|
543
|
+
print(' Compressing from', tuple(map(int, dims)), 'to', S.shape)
|
|
541
544
|
if display > 2:
|
|
542
545
|
print(' Compression relative error = {:7e}'.format(best_error))
|
|
543
546
|
|
|
@@ -11,7 +11,7 @@ import TensorFox.TensorFox as tfx
|
|
|
11
11
|
|
|
12
12
|
def normalization(X, method='mean_normalization'):
|
|
13
13
|
"""
|
|
14
|
-
Normalize data by shifting all data by a certain constant, and dividing all data by
|
|
14
|
+
Normalize data by shifting all data by a certain constant, and dividing all data by other certain constant. The
|
|
15
15
|
program returns the normalized data X = (X - mu)/sigma. In the case the user want to return to the original format,
|
|
16
16
|
set X = sigma*X + mu.
|
|
17
17
|
|
|
@@ -1,3 +1,33 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: TensorFox
|
|
3
|
+
Version: 1.2.2
|
|
4
|
+
Summary: Tensor Fox is a high performance package of multilinear algebra and tensor routines, with focus on the Canonical Polyadic Decomposition (CPD), also called PARAFAC or CANDECOMP.
|
|
5
|
+
Home-page: https://github.com/felipebottega/Tensor-Fox
|
|
6
|
+
Author: Felipe Bottega Diniz
|
|
7
|
+
Author-email: felipebottega@gmail.com
|
|
8
|
+
License: GNU
|
|
9
|
+
Keywords: Tensor Fox Canonical Polyadic Decomposition CPD PARAFAC CANDECOMP Multilinear Learning
|
|
10
|
+
Description-Content-Type: text/markdown
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Requires-Dist: numpy>=1.21.0
|
|
13
|
+
Requires-Dist: pandas>=1.2.3
|
|
14
|
+
Requires-Dist: scipy>=1.6.2
|
|
15
|
+
Requires-Dist: scikit-learn>=0.24.1
|
|
16
|
+
Requires-Dist: matplotlib>=3.3.4
|
|
17
|
+
Requires-Dist: numba>=0.53.1
|
|
18
|
+
Requires-Dist: IPython>=7.31.1
|
|
19
|
+
Requires-Dist: sparse_dot_mkl>=0.7
|
|
20
|
+
Dynamic: author
|
|
21
|
+
Dynamic: author-email
|
|
22
|
+
Dynamic: description
|
|
23
|
+
Dynamic: description-content-type
|
|
24
|
+
Dynamic: home-page
|
|
25
|
+
Dynamic: keywords
|
|
26
|
+
Dynamic: license
|
|
27
|
+
Dynamic: license-file
|
|
28
|
+
Dynamic: requires-dist
|
|
29
|
+
Dynamic: summary
|
|
30
|
+
|
|
1
31
|
# Tensor Fox
|
|
2
32
|
|
|
3
33
|
Tensor Fox is a high performance package of multilinear algebra and tensor routines, with focus on the Canonical Polyadic Decomposition (CPD), also called PARAFAC or CANDECOMP. We recommend [the wikipedia article](https://en.wikipedia.org/wiki/Tensor_rank_decomposition) for a introduction to this decomposition. In this package you will find an efficient CPD solver, a great variety of parameters for fine tuning, support to sparse tensors, routines to test models and tensor behavior, rank estimate, statistics, and many more!
|
|
@@ -28,12 +58,19 @@ The only pre-requisite is to have **Python 3** installed. After this you just ne
|
|
|
28
58
|
|
|
29
59
|
pip install TensorFox
|
|
30
60
|
|
|
31
|
-
The package `sparse_dot_mkl` won't work properly with `pip` (we explain in more detail at the end of this text). This package is responsible for accelerating
|
|
61
|
+
The package `sparse_dot_mkl` won't work properly with `pip` (we explain in more detail at the end of this text). This package is responsible for accelerating sparse matrix multiplications. Tensor Fox still works without this package but it will use Scipy dot instead. I recommend installing [Miniconda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) and then create an env with the following:
|
|
32
62
|
|
|
33
63
|
conda create --name tfx_env --channel defaults jupyter numpy pandas scipy scikit-learn matplotlib numba IPython sparse_dot_mkl
|
|
64
|
+
conda install -c conda-forge sparse_dot_mkl
|
|
34
65
|
|
|
35
66
|
After this you may install TensorFox and everything will be fine. The channel must be `defaults`, otherwise Numpy won't be linked against MKL (see https://numpy.org/install/), which is necessary to work with `sparse_dot_mkl`. Finally, note that `jupyter` is optional, only if you want to work with jupyter notebooks.
|
|
36
67
|
|
|
68
|
+
### Tensor Fox Standalone Environment (only for Windows)
|
|
69
|
+
|
|
70
|
+
If you prefer not to install Tensor Fox via pip, [this release](https://github.com/felipebottega/Tensor-Fox/releases/) provides a portable standalone environment with Python and all required dependencies already included. Simply extract the archive and follow the instructions in this release to start using Tensor Fox.
|
|
71
|
+
|
|
72
|
+
The standalone environment is intended to simplify setup and provide a reproducible execution environment. It contains the exact dependency versions used during development, helping ensure consistent behavior across compatible Windows systems.
|
|
73
|
+
|
|
37
74
|
### Creating a tensor
|
|
38
75
|
|
|
39
76
|
Let's start importing Tensor Fox and other necessary modules.
|
|
@@ -174,7 +211,7 @@ In this section we summarize all the features Tensor Fox has to offer. As alread
|
|
|
174
211
|
|
|
175
212
|
### Sparse Dot MKL requirements
|
|
176
213
|
|
|
177
|
-
This package requires the MKL runtime linking library `libmkl_rt.so` (or `libmkl_rt.dylib` for OSX, or `mkl_rt.dll` for WIN). If the MKL library cannot be loaded an `ImportError` will be raised when the package is first imported. MKL is distributed with the full version of conda, and can be installed into Miniconda with `conda install -c intel mkl`. Alternatively, you may
|
|
214
|
+
This package requires the MKL runtime linking library `libmkl_rt.so` (or `libmkl_rt.dylib` for OSX, or `mkl_rt.dll` for WIN). If the MKL library cannot be loaded an `ImportError` will be raised when the package is first imported. MKL is distributed with the full version of conda, and can be installed into Miniconda with `conda install -c intel mkl`. Alternatively, you may need to add the path to MKL shared objects to `LD_LIBRARY_PATH` (e.g. `export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64:$LD_LIBRARY_PATH`). There are some known bugs in MKL v2019 which may cause intermittent segfaults. Updating to MKL v2020 is highly recommended if you encounter any problems.
|
|
178
215
|
|
|
179
216
|
### Sparse Dot MKL environment variable
|
|
180
217
|
|
|
@@ -213,7 +250,8 @@ This project is licensed under the GNU GENERAL PUBLIC LICENSE - see the [LICENSE
|
|
|
213
250
|
8) S. Rabanser, O. Shchur, and S. Günnemann, *Introduction to Tensor Decompositions and their Applications in Machine Learning*, arXiv:1711.10781v1 (2017).
|
|
214
251
|
9) A. H. Phan, P. Tichavsky, and A. Cichoki, *Low Complexity Damped Gauss-Newton Algorithm for CANDECOMP/PARAFAC*, SIAM Journal on Matrix Analysis and Applications, 34 (1), 126-147 (2013).
|
|
215
252
|
10) L. De Lathauwer, B. De Moor, and J. Vandewalle, *A Multilinear Singular Value Decomposition*, SIAM J. Matrix Anal. Appl., 21 (2000), pp. 1253-1278.
|
|
216
|
-
11)
|
|
217
|
-
12)
|
|
218
|
-
13)
|
|
219
|
-
14)
|
|
253
|
+
11) F. Bottega Diniz, PhD Thesis, https://www.pgmat.im.ufrj.br/index.php/pt-br/teses-e-dissertacoes/teses/2019-1/281-tensor-decompositions-and-algorithms-with-applications-to-tensor-learning/file
|
|
254
|
+
12) Tensorlab - https://www.tensorlab.net/
|
|
255
|
+
13) Tensor Toolbox - http://www.sandia.gov/~tgkolda/TensorToolbox/
|
|
256
|
+
14) Tensorly - https://github.com/tensorly/
|
|
257
|
+
15) TensorBox - https://github.com/phananhhuy/TensorBox
|
|
@@ -7,7 +7,7 @@ long_description = (here / 'README.md').read_text(encoding='utf-8')
|
|
|
7
7
|
|
|
8
8
|
setup(
|
|
9
9
|
name='TensorFox',
|
|
10
|
-
version='1.2.
|
|
10
|
+
version='1.2.2',
|
|
11
11
|
description='Tensor Fox is a high performance package of multilinear algebra and tensor routines, with focus on the Canonical Polyadic Decomposition (CPD), also called PARAFAC or CANDECOMP.',
|
|
12
12
|
long_description=long_description,
|
|
13
13
|
long_description_content_type='text/markdown',
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|