libmf 0.1.3 → 0.2.0

data/vendor/libmf/Makefile.win

@@ -1,36 +0,0 @@
-CXX = cl.exe
-CFLAGS = /W4 /nologo /O2 /EHsc /D "_CRT_SECURE_NO_DEPRECATE"
-
-# Choose one instruction set to accelerate LIBMF
-# 1. use SSE
-CPUSET = /D "USESSE"
-# 2. use AVX
-#CPUSET = /D "USEAVX" /arch:AVX
-# 3. no acceleration
-#CPUSET =
-
-# Always use OpenMP because Visual Studio supports it.
-CFLAGS = $(CFLAGS) $(CPUSET) /D "USEOMP" /openmp
-# To disable OpenMP, please use the command below.
-#CFLAGS = $(CFLAGS) $(CPUSET) /D
-
-TARGET = windows
-
-all: $(TARGET)\mf-train.exe $(TARGET)\mf-predict.exe lib
-
-$(TARGET)\mf-predict.exe: mf.h mf-predict.cpp mf.obj
-	$(CXX) $(CFLAGS) mf-predict.cpp mf.obj -Fe$(TARGET)\mf-predict.exe
-
-$(TARGET)\mf-train.exe: mf.h mf-train.cpp mf.obj
-	$(CXX) $(CFLAGS) mf-train.cpp mf.obj -Fe$(TARGET)\mf-train.exe
-
-mf.obj: mf.cpp mf.h
-	$(CXX) $(CFLAGS) -c mf.cpp
-
-lib: mf.cpp mf.def mf.h
-	$(CXX) $(CFLAGS) -LD mf.cpp -Fe$(TARGET)\mf -link -DEF:mf.def
-	
-clean:
-	-erase /Q *.obj *.dll *.lib *.exe $(TARGET)\.
-
-

data/vendor/libmf/README

@@ -1,637 +0,0 @@
-LIBMF is a library for large-scale sparse matrix factorization. For the
-optimization problem it solves and the overall framework, please refer to [3].
-
-
-
-Table of Contents
-=================
-
-- Installation
-- Data Format
-- Model Format
-- Command Line Usage
-- Examples
-- Library Usage
-- SSE, AVX, and OpenMP
-- Building Windows and Mac Binaries
-- References
-
-
-
-Installation
-============
-
-- Requirements
-  To compile LIBMF, a compiler which supports C++11 is required. LIBMF can
-  use SSE, AVX, and OpenMP for acceleration. See Section SSE, AVX, and OpenMP
-  if you want to disable or enable these features.
-
-- Unix & Cygwin
-
-  Type `make' to build `mf-train' and `mf-precict.'
-
-- Windows & Mac
-
-  See `Building Windows and Mac Binaries' to compile. For Windows, pre-built
-  binaries are available in the directory `windows.'
-
-
-
-Data Format
-===========
-
-LIBMF's command-line tool can be used to factorize matrices with real or binary
-values. Each line in the training file stores a tuple,
-
-    <row_idx> <col_idx> <value>
-
-which records an entry of the training matrix. In the `demo' directory, the
-files `real_matrix.tr.txt' and `real_matrix.te.txt' are the training and test
-sets for a demonstration of real-valued matrix factorization (RVMF). For binary
-matrix factorization (BMF), the set of <value> is {-1, 1} as shown in
-`binary_matrix.tr.txt' and `binary_matrix.te.txt.' For one-class MF, all
-<value>'s are positive. See `all_one_matrix.tr.txt' and `all_one_matrix.te.txt'
-as examples.
-
-Note: If the values in the test set are unknown, please put dummy zeros.
-
-
-
-Model Format
-============
-
-    LIBMF factorizes a training matrix `R' into a k-by-m matrix `P' and a
-    k-by-n matrix `Q' such that `R' is approximated by P'Q. After the training
-    process is finished, the two factor matrices `P' and `Q' are stored into a
-    model file. The file starts with a header including:
-
-        `f': the loss function of the solved MF problem
-        `m': the number of rows in the training matrix,
-        `n': the number of columns in the training matrix,
-        `k': the number of latent factors,
-        `b': the average of all elements in the training matrix.
-
-    From the 5th line, the columns of `P' and `Q' are stored line by line. In
-    each line, there are two leading tokens followed by the values of a
-    column. The first token is the name of the stored column, and the second
-    word indicates the type of values. If the second word is `T', the column is
-    a real vector. Otherwise, all values in the column are NaN. For example, if
-
-                            [1 NaN 2]      [-1 -2]
-                        P = |3 NaN 4|, Q = |-3 -4|,
-                            [5 NaN 6]      [-5 -6]
-
-    and the value `b' is 0.5, the content of the model file is:
-
-                        --------model file--------
-                        m 3
-                        n 2
-                        k 3
-                        b 0.5
-                        p0 T 1 3 5
-                        p1 F 0 0 0
-                        p2 T 2 4 6
-                        q0 T -1 -3 -5
-                        q1 T -2 -4 -6
-                        --------------------------
-
-
-
-Command Line Usage
-==================
-
--   `mf-train'
-
-    usage: mf-train [options] training_set_file [model_file]
-
-    options:
-    -l1 <lambda>,<lambda>: set L1-regularization parameters for P and Q.
-        (default 0) If only one value is specified, P and Q share the same
-        lambda.
-    -l2 <lambda>,<lambda>: set L2-regularization parameters for P and Q.
-        (default 0.1) If only one value is specified, P and Q share the same
-        lambda.
-    -f  <loss>: specify loss function (default 0)
-      for real-valued matrix factorization
-         0 -- squared error (L2-norm)
-         1 -- absolute error (L1-norm)
-         2 -- generalized KL-divergence (--nmf is required)
-      for binary matrix factorization
-         5 -- logarithmic error
-         6 -- squared hinge loss
-         7 -- hinge loss
-      for one-class matrix factorization
-        10 -- row-oriented pair-wise logarithmic loss
-        11 -- column-oriented pair-wise logarithmic loss
-        12 -- squared error (L2-norm)
-    -k <dimensions>: set number of dimensions (default 8)
-    -t <iter>: set number of iterations (default 20)
-    -r <eta>: set initial learning rate (default 0.1)
-    -a <alpha>: set coefficient of negative entries' loss (default 1)
-    -c <c>: set value of negative entries (default 0.0001).
-       Every positive entry is assumed to be 1.
-    -s <threads>: set number of threads (default 12)
-    -n <bins>: set number of bins (may be adjusted by LIBMF for speed)
-    -p <path>: set path to the validation set
-    -v <fold>: set number of folds for cross validation
-    --quiet: quiet mode (no outputs)
-    --nmf: perform non-negative matrix factorization
-    --disk: perform disk-level training (will create a buffer file)
-
-    `mf-train' is the main training command of LIBMF. At each iteration, the
-    following information is printed.
-
-        - iter: the index of iteration.
-        - tr_*: * is the evaluation criterion on the training set.
-        - tr_*+: * is the evaluation criterion on the positive entries in the
-          training set.
-        - tr_*-: * is the evaluation criterion on the negative entries in the
-          training set.
-        - va_*: the same criterion on the validation set if `-p' is set
-        - va_*+: * is the evaluation criterion on the positive entries in the
-          validation set.
-        - va_*-: * is the evaluation criterion on the negative entries in the
-          validation set.
-        - obj: objective function value.
-        - reg: regularization term.
-
-    Here `tr_*' and `obj' are estimations because calculating true values
-    can be time-consuming. Different solvers can print different combinations
-    those values.
-
-    For different losses, the criterion to be printed is listed below.
-
-           <loss>: <evaluation criterion>
-        -       0: root mean square error (RMSE)
-        -       1: mean absolute error (MAE)
-        -       2: generalized KL-divergence (KL)
-        -       5: logarithmic loss
-        -   6 & 7: accuracy
-        - 10 & 11: pair-wise logarithmic loss in Bayesian personalized ranking
-        -      12: sum of squared errors. The label of positive entries is 1
-        -          while negative entries' value is set using command line
-        -          option -c.
-
--   `mf-predict'
-
-    usage: mf-predict [options] test_file model_file output_file
-
-    options:
-    -e <criterion>: set the evaluation criterion (default 0)
-         0: root mean square error
-         1: mean absolute error
-         2: generalized KL-divergence
-         5: logarithmic loss
-         6: accuracy
-        10: row-oriented mean percentile rank (row-oriented MPR)
-        11: colum-oriented mean percentile rank (column-oriented MPR)
-        12: row-oriented area under ROC curve (row-oriented AUC)
-        13: column-oriented area under ROC curve (column-oriented AUC)
-
-    `mf-predict' outputs the prediction values of the entries specified in
-    `test_file' to the `output_file.' The selected criterion will be printed
-    as well.
-
-
-
-Examples
-========
-This section gives example commands of LIBMF using the data sets in `demo'
-directory. In `demo,' a shell script `demo.sh' can be run for demonstration.
-
-> mf-train real_matrix.tr.txt model
-
-train a model using the default parameters
-
-> mf-train -l1 0.05 -l2 0.01 real_matrix.tr.txt model
-
-train a model with the following regularization coefficients:
-
-    coefficient of L1-norm regularization on P = 0.05
-    coefficient of L1-norm regularization on Q = 0.05
-    coefficient of L2-norm regularization on P = 0.01
-    coefficient of L2-norm regularization on Q = 0.01
-
-> mf-train -l1 0.015,0 -l2 0.01,0.005 real_matrix.tr.txt model
-
-train a model with the following regularization coefficients:
-
-    coefficient of L1-norm regularization on P = 0.05
-    coefficient of L1-norm regularization on Q = 0
-    coefficient of L2-norm regularization on P = 0.01
-    coefficient of L2-norm regularization on Q = 0.03
-
-> mf-train -f 5 -l1 0,0.02 -k 100 -t 30 -r 0.02 -s 4 binary_matrix.tr.txt model
-
-train a BMF model using logarithmic loss and the following parameters:
-
-    coefficient of L1-norm regularization on P = 0
-    coefficient of L1-norm regularization on Q = 0.01
-    latent factors = 100
-    iterations = 30
-    learning rate = 0.02
-    threads = 4
-
-> mf-train -p real_matrix.te.txt real_matrix.tr.txt model
-
-use real_matrix.te.txt for hold-out validation
-
-> mf-train -v 5 real_matrix.tr.txt
-
-do five fold cross validation
-
-> mf-train -f 2 --nmf real_matrix.tr.txt
-
-do non-negative matrix factorization with generalized KL-divergence
-
-> mf-train --quiet real_matrix.tr.txt
-
-do not print message to screen
-
-> mf-train --disk real_matrix.tr.txt
-
-do disk-level training
-
-> mf-predict real_matrix.te.txt model output
-
-do prediction
-
-> mf-predict -e 1 real_matrix.te.txt model output
-
-do prediction and output MAE
-
-
-
-Library Usage
-=============
-
-These structures and functions are declared in the header file `mf.h.' You need
-to #include `mf.h' in your C/C++ source files and link your program with
-`mf.cpp.' Users can read `mf-train.cpp' and `mf-predict.cpp' as usage examples.
-
-Before predicting test data, we need to construct a model (`mf_model') using
-training data which is either a C structure `mf_problem' or the path to the
-training file. For the first case, the whole data set needs to be fitted into
-memory. For the second case, a binary version of the training file will be
-created, and only some parts of the binary file are loaded at one time. Note
-that a model can also be saved in a file for later use. To evaluate the quality
-of a model, users can call an evaluation function in LIBMF with a `mf_problem'
-and a `mf_model.'
-
-
-There are four public data structures in LIBMF.
-
--   struct mf_node
-    {
-        mf_int u;
-        mf_int v;
-        mf_float r;
-    };
-
-    `mf_node' represents an element in a sparse matrix. `u' represents the row
-    index, `v' represents the column index, and `r' represents the value.
-
-
--   struct mf_problem
-    {
-        mf_int m;
-        mf_int n;
-        mf_long nnz;
-        struct mf_node *R;
-    };
-
-    `mf_problem' represents a sparse matrix. Each element is represented by
-    `mf_node.' `m' represents the number of rows, `n' represents the number of
-    columns, `nnz' represents the number of non-zero elements, and `R' is an
-    array of `mf_node' whose length is `nnz.'
-
-
--   struct mf_parameter
-    {
-        mf_int fun;
-        mf_int k;
-        mf_int nr_threads;
-        mf_int nr_bins;
-        mf_int nr_iters;
-        mf_float lambda_p1;
-        mf_float lambda_p2;
-        mf_float lambda_q1;
-        mf_float lambda_q2;
-        mf_float alpha;
-        mf_float c;
-        mf_float eta;
-        bool do_nmf;
-        bool quiet;
-        bool copy_data;
-    };
-
-    `mf_parameter' represents the parameters used for training. The meaning of
-    each variable is:
-
-    variable      meaning                                    default
-    ================================================================
-    fun           loss function                                    0
-    k             number of latent factors                         8
-    nr_threads    number of threads used                          12
-    nr_bins       number of bins                                  20
-    nr_iters      number of iterations                            20
-    lambda_p1     coefficient of L1-norm regularization on P       0
-    lambda_p2     coefficient of L2-norm regularization on P     0.1
-    lambda_q1     coefficient of L1-norm regularization on Q       0
-    lambda_q2     coefficient of L2-norm regularization on Q     0.1
-    eta           learning rate                                  0.1
-    alpha         importance of negative entries                 0.1
-    c             desired value of negative entries           0.0001
-    do_nmf        perform non-negative MF (NMF)                false
-    quiet         no outputs to stdout                         false
-    copy_data     copy data in training procedure               true
-
-    There are two major algorithm categories in LIBMF. One is for stochastic
-    gradient method and the other one is for coordinate descent method. Both
-    of them support multi-threading. Currently, the only solver used
-    coordinate descent method is implemented for fun=12. All other types of loss
-    functions such as fun=0 may use stochastic gradient method. Notice that
-    when a framework does support the parameters specified, LIBMF may ignore
-    them or throw an error.
-
-    LIBMF's framework for stochastic gradient method:
-
-        In LIBMF, we parallelize the computation by griding the data matrix
-        into nr_bins^2 blocks. According to our experiments, this parameter is
-        not sensitive to both effectiveness and efficiency. In most cases the
-        default value should work well.
-
-        For disk-level training, `nr_bins' controls the memory usage of
-        because one thread accesss an entire block at one time. If `nr_bins'
-        is 4 and `nr_threads' is 1, the expected usage of memory is 25% of the
-        memory to store the whole training matrix.
-
-        Let the training data is a `mf_problem.' By default, at the beginning
-        of the training procedure, the data matrix is copied because it would
-        be modified in the training process. To save memory, `copy_data' can
-        be set to false with the following effects.
-
-            (1) The raw data is directly used without being copied.
-            (2) The order of nodes may be changed.
-            (3) The value in each node may become slightly different.
-
-        Note that `copy_data' is invalid for disk-level training.
-
-        To obtain a parameter with default values, use the function
-        `get_default_parameter.'
-
-        Note that parameter alpha and c are not ignored under this framework. 
-
-    LIBMF's framework for coordinate descent method:
-
-        Currently, only one solver is implemented under this framework. It
-        minimizes the squared errors overall the whole training matrix. Its
-        regularization function is Frobenius norm on the two factor matrices
-        P and Q. Note that the the original training matrix R (m-by-n) is
-        approximated by P^TQ. This solver requires two copies of the original
-        positive entries if `copy_data' is false. That is, if your input data is
-        50MB, LIBMF may need 150MB memory in total for data storage. By
-        setting `copy_data' to false, LIBMF will only make one extra copy.
-        Disk-level training is not supported.
-
-        Parameters recognized by this framework are `fun,' `k,' `nr_threads,'
-        `nr_iters,' `lambda_p2,' `lambda_q2,' `alpha,' `c,' `quiet,' and
-        `copy_data.'
-
-        Unlike the standard C++ thread class used in stochastic gradient
-        method's framework, the parallel computation here relies on OpenMP, so
-        please make sure your complier can support it.
-
-
--   struct mf_model
-    {
-        mf_int fun;
-        mf_int m;
-        mf_int n;
-        mf_int k;
-        mf_float b;
-        mf_float *P;
-        mf_float *Q;
-    };
-
-    `mf_model' is used to store models learned by LIBMF. `fun' indicates the
-    loss function of the solved MF problem. `m' represents the number of rows,
-    `n' represents the number of columns, `k' represents the number of latent
-    factors, and `b' is the average of all elements in the training matrix. `P'
-    is used to store a kxm matrix in column oriented format. For example, if
-    `P' stores a 3x4 matrix, then the content of `P' is:
-
-        P11 P21 P31 P12 P22 P32 P13 P23 P33 P14 P24 P34
-
-    `Q' is used to store a kxn matrix in the same manner.
-
-
-Functions available in LIBMF include:
-
-
--   mf_parameter mf_get_default_param();
-
-    Get default parameters.
-
--   mf_int mf_save_model(struct mf_model const *model, char const *path);
-
-    Save a model. It returns 0 on sucess and 1 on failure.
-
--   struct mf_model* mf_load_model(char const *path);
-
-    Load a model. If the model could not be loaded, a nullptr is returned.
-
--   void mf_destroy_model(struct mf_model **model);
-
-    Destroy a model.
-
--   struct mf_model* mf_train(
-        struct mf_problem const *prob,
-        mf_parameter param);
-
-    Train a model. A nullptr is returned if fail.
-
--   struct mf_model* mf_train_on_disk(
-        char const *tr_path,
-        mf_parameter param);
-
-    Train a model while parts of data is put in disk to reduce memory usage. A
-    nullptr is returned if fail.
-
-    Notice: the model is still fully loaded during the training process.
-
--   struct mf_model* mf_train_with_validation(
-        struct mf_problem const *tr,
-        struct mf_problem const *va,
-        mf_parameter param);
-
-    Train a model with training set `tr' and validation set `va.' The
-    evaluation criterion of the validation set is printed at each iteration.
-
--   struct mf_model* mf_train_with_validation_on_disk(
-        char const *tr_path,
-        char const *va_path,
-        mf_parameter param);
-
-    Train a model using the training file `tr_path' and validation file
-    `va_path' for holdout validation. The same strategy is used to save memory
-    as in `mf_train_on_disk.' It also printed the same information as
-    `mf_train_with_validation.'
-
-    Notice: LIBMF assumes that the model and the validation set can be fully
-    loaded into the memory.
-
--   mf_float mf_cross_validation(
-        struct mf_problem const *prob,
-        mf_int nr_folds,
-        mf_parameter param);
-
-    Do cross validation with `nr_folds' folds.
-
--   mf_float mf_predict(
-        struct mf_model const *model,
-        mf_int p_idx,
-        mf_int q_idx);
-
-    Predict the value at the position (p_idx, q_idx). The predicted value is a
-    real number for RVMF or OCMF. For BMF, the range of the prediction values
-    are {-1, 1}. If `p_idx' or `q_idx' can not be found in the training set,
-    the function returns the average (mode if BMF) of all values in the
-    training matrix.
-
--   mf_double calc_rmse(mf_problem *prob, mf_model *model);
-
-    calculate the RMSE of the model on a test set `prob.' It can be used to
-    evaluate the result of real-valued MF.
-
--   mf_double calc_mae(mf_problem *prob, mf_model *model);
-
-    calculate the MAE of the model on a test set `prob.' It can be used to
-    evaluate the result of real-valued MF.
-
--   mf_double calc_gkl(mf_problem *prob, mf_model *model);
-
-    calculate the Generalized KL-divergence of the model on a test set `prob.'
-    It can be used to evaluate the result of non-negative RVMF.
-
--   calc_logloss(mf_problem *prob, mf_model *model);
-
-    calculate the logarithmic loss of the model on a test `prob.' It can be
-    used to evaluate the result of BMF.
-
--   mf_double calc_accuracy(mf_problem *prob, mf_model *model);
-
-    calculate the accuracy of the model on a test `prob.' It can be used to
-    evaluate the result of BMF.
-
--   mf_double calc_mpr(mf_problem *prob, mf_model *model, bool transpose)
-
-    calculate the MPR of the model on a test `prob.' If `transpose' is `false
-    row-oriented MPR is calculated and otherwise column-oriented MPR. It can be
-    used to evaluate the result of OCMF.
-
--   calc_auc(mf_problem *prob, mf_model *model, bool transpose);
-
-    calculate the row-oriented AUC of the model on a test `prob' if `transpose'
-    is `false.' For column-oriented AUC, set `transpose' to be 'true.' It can
-    be used to evaluate the result of OCMF.
-
-
-
-SSE, AVX, and OpenMP
-====================
-
-LIBMF utilizes SSE instructions to accelerate the computation. If you cannot
-use SSE on your platform, then please comment out
-
-    DFLAG = -DUSESSE
-
-in Makefile to disable SSE.
-
-Some modern CPUs support AVX, which is more powerful than SSE. To enable AVX,
-please comment out
-
-    DFLAG = -DUSESSE
-
-and uncomment the following lines in Makefile.
-
-    DFLAG = -DUSEAVX
-    CFLAGS += -mavx
-
-If OpenMP is not available on your platform, please comment out the following
-lines in Makefile.
-
-    DFLAG += -DUSEOMP
-    CXXFLAGS += -fopenmp
-
-Notice: Please always run `make clean all' if these flags are changed.
-
-
-
-Building Windows and Mac and Binaries
-=====================================
-
--   Windows
-
-    Windows binaries are in the directory `windows.' To build them via
-    command-line tools of Microsoft Visual Studio, use the following steps:
-
-    1. Open a DOS command box (or Developer Command Prompt for Visual Studio)
-    and go to libmf directory. If environment variables of VC++ have not been
-    set, type
-
-    "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\amd64\vcvars64.bat"
-
-    You may have to modify the above command according which version of VC++ or
-    where it is installed.
-
-    2. Type
-
-    nmake -f Makefile.win clean all
-
-    3. (optional) To build shared library mf_c.dll, type
-
-    nmake -f Makefile.win lib
-
--   Mac
-
-    To complie LIBMF on Mac, a GCC complier is required, and users need to
-    slightly modify the Makefile. The following instructions are tested with
-    GCC 4.9.
-
-    1. Set the complier path to your GCC complier. For example, the first
-       line in the Makefile can be
-
-       CXX = g++-4.9
-
-    2. Remove `-march=native' from `CXXFLAGS.' The second line in the Makefile
-       Should be
-
-       CXXFLAGS = -O3 -pthread -std=c++0x
-
-    3. If AVX is enabled, we add `-Wa,-q' to the `CXXFLAGS,' so the previous
-       `CXXFLAGS' becomes
-
-       CXXFLAGS = -O3 -pthread -std=c++0x -Wa,-q
-
-
-
-References
-==========
-
-[1] W.-S. Chin, Y. Zhuang, Y.-C. Juan, and C.-J. Lin. A Fast Parallel
-Stochastic Gradient Method for Matrix Factorization in Shared Memory Systems.
-ACM TIST, 2015. (www.csie.ntu.edu.tw/~cjlin/papers/libmf/libmf_journal.pdf)
-
-[2] W.-S. Chin, Y. Zhuang, Y.-C. Juan, and C.-J. Lin. A Learning-rate Schedule
-for Stochastic Gradient Methods to Matrix Factorization. PAKDD, 2015.
-(www.csie.ntu.edu.tw/~cjlin/papers/libmf/mf_adaptive_pakdd.pdf)
-
-[3] W.-S. Chin, B.-W. Yuan, M.-Y. Yang, Y. Zhuang, Y.-C. Juan, and C.-J. Lin.
-LIBMF: A Library for Parallel Matrix Factorization in Shared-memory Systems.
-JMLR, 2015.
-(www.csie.ntu.edu.tw/~cjlin/papers/libmf/libmf_open_source.pdf)
-
-For any questions and comments, please email:
-
-    cjlin@csie.ntu.edu.tw