rsvm 0.0.1
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +6 -0
- data/Gemfile +6 -0
- data/README.md +89 -0
- data/Rakefile +41 -0
- data/ext/libsvm/COPYRIGHT +31 -0
- data/ext/libsvm/FAQ.html +1837 -0
- data/ext/libsvm/README +748 -0
- data/ext/libsvm/extconf.rb +3 -0
- data/ext/libsvm/svm.cpp +3213 -0
- data/ext/libsvm/svm.h +102 -0
- data/lib/svm.rb +119 -0
- data/lib/svm/cross_validation.rb +39 -0
- data/lib/svm/debug.rb +12 -0
- data/lib/svm/model.rb +68 -0
- data/lib/svm/options.rb +69 -0
- data/lib/svm/problem.rb +151 -0
- data/lib/svm/scaler.rb +82 -0
- data/lib/svm/version.rb +3 -0
- data/rsvm.gemspec +25 -0
- data/test/fixtures/heart_scale.csv +270 -0
- data/test/fixtures/unbalanced.csv +164 -0
- data/test/lib/cross_validation_test.rb +35 -0
- data/test/lib/model_test.rb +84 -0
- data/test/lib/problem_test.rb +39 -0
- data/test/lib/scaler_test.rb +57 -0
- data/test/test_helper.rb +3 -0
- metadata +101 -0
data/ext/libsvm/README
ADDED
@@ -0,0 +1,748 @@
|
|
1
|
+
Libsvm is a simple, easy-to-use, and efficient software for SVM
|
2
|
+
classification and regression. It solves C-SVM classification, nu-SVM
|
3
|
+
classification, one-class-SVM, epsilon-SVM regression, and nu-SVM
|
4
|
+
regression. It also provides an automatic model selection tool for
|
5
|
+
C-SVM classification. This document explains the use of libsvm.
|
6
|
+
|
7
|
+
Libsvm is available at
|
8
|
+
http://www.csie.ntu.edu.tw/~cjlin/libsvm
|
9
|
+
Please read the COPYRIGHT file before using libsvm.
|
10
|
+
|
11
|
+
Table of Contents
|
12
|
+
=================
|
13
|
+
|
14
|
+
- Quick Start
|
15
|
+
- Installation and Data Format
|
16
|
+
- `svm-train' Usage
|
17
|
+
- `svm-predict' Usage
|
18
|
+
- `svm-scale' Usage
|
19
|
+
- Tips on Practical Use
|
20
|
+
- Examples
|
21
|
+
- Precomputed Kernels
|
22
|
+
- Library Usage
|
23
|
+
- Java Version
|
24
|
+
- Building Windows Binaries
|
25
|
+
- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.
|
26
|
+
- MATLAB/OCTAVE Interface
|
27
|
+
- Python Interface
|
28
|
+
- Additional Information
|
29
|
+
|
30
|
+
Quick Start
|
31
|
+
===========
|
32
|
+
|
33
|
+
If you are new to SVM and if the data is not large, please go to
|
34
|
+
`tools' directory and use easy.py after installation. It does
|
35
|
+
everything automatic -- from data scaling to parameter selection.
|
36
|
+
|
37
|
+
Usage: easy.py training_file [testing_file]
|
38
|
+
|
39
|
+
More information about parameter selection can be found in
|
40
|
+
`tools/README.'
|
41
|
+
|
42
|
+
Installation and Data Format
|
43
|
+
============================
|
44
|
+
|
45
|
+
On Unix systems, type `make' to build the `svm-train' and `svm-predict'
|
46
|
+
programs. Run them without arguments to show the usages of them.
|
47
|
+
|
48
|
+
On other systems, consult `Makefile' to build them (e.g., see
|
49
|
+
'Building Windows binaries' in this file) or use the pre-built
|
50
|
+
binaries (Windows binaries are in the directory `windows').
|
51
|
+
|
52
|
+
The format of training and testing data file is:
|
53
|
+
|
54
|
+
<label> <index1>:<value1> <index2>:<value2> ...
|
55
|
+
.
|
56
|
+
.
|
57
|
+
.
|
58
|
+
|
59
|
+
Each line contains an instance and is ended by a '\n' character. For
|
60
|
+
classification, <label> is an integer indicating the class label
|
61
|
+
(multi-class is supported). For regression, <label> is the target
|
62
|
+
value which can be any real number. For one-class SVM, it's not used
|
63
|
+
so can be any number. The pair <index>:<value> gives a feature
|
64
|
+
(attribute) value: <index> is an integer starting from 1 and <value>
|
65
|
+
is a real number. The only exception is the precomputed kernel, where
|
66
|
+
<index> starts from 0; see the section of precomputed kernels. Indices
|
67
|
+
must be in ASCENDING order. Labels in the testing file are only used
|
68
|
+
to calculate accuracy or errors. If they are unknown, just fill the
|
69
|
+
first column with any numbers.
|
70
|
+
|
71
|
+
A sample classification data included in this package is
|
72
|
+
`heart_scale'. To check if your data is in a correct form, use
|
73
|
+
`tools/checkdata.py' (details in `tools/README').
|
74
|
+
|
75
|
+
Type `svm-train heart_scale', and the program will read the training
|
76
|
+
data and output the model file `heart_scale.model'. If you have a test
|
77
|
+
set called heart_scale.t, then type `svm-predict heart_scale.t
|
78
|
+
heart_scale.model output' to see the prediction accuracy. The `output'
|
79
|
+
file contains the predicted class labels.
|
80
|
+
|
81
|
+
For classification, if training data are in only one class (i.e., all
|
82
|
+
labels are the same), then `svm-train' issues a warning message:
|
83
|
+
`Warning: training data in only one class. See README for details,'
|
84
|
+
which means the training data is very unbalanced. The label in the
|
85
|
+
training data is directly returned when testing.
|
86
|
+
|
87
|
+
There are some other useful programs in this package.
|
88
|
+
|
89
|
+
svm-scale:
|
90
|
+
|
91
|
+
This is a tool for scaling input data file.
|
92
|
+
|
93
|
+
svm-toy:
|
94
|
+
|
95
|
+
This is a simple graphical interface which shows how SVM
|
96
|
+
separate data in a plane. You can click in the window to
|
97
|
+
draw data points. Use "change" button to choose class
|
98
|
+
1, 2 or 3 (i.e., up to three classes are supported), "load"
|
99
|
+
button to load data from a file, "save" button to save data to
|
100
|
+
a file, "run" button to obtain an SVM model, and "clear"
|
101
|
+
button to clear the window.
|
102
|
+
|
103
|
+
You can enter options in the bottom of the window, the syntax of
|
104
|
+
options is the same as `svm-train'.
|
105
|
+
|
106
|
+
Note that "load" and "save" consider data in the
|
107
|
+
classification but not the regression case. Each data point
|
108
|
+
has one label (the color) which must be 1, 2, or 3 and two
|
109
|
+
attributes (x-axis and y-axis values) in [0,1].
|
110
|
+
|
111
|
+
Type `make' in respective directories to build them.
|
112
|
+
|
113
|
+
You need Qt library to build the Qt version.
|
114
|
+
(available from http://www.trolltech.com)
|
115
|
+
|
116
|
+
You need GTK+ library to build the GTK version.
|
117
|
+
(available from http://www.gtk.org)
|
118
|
+
|
119
|
+
The pre-built Windows binaries are in the `windows'
|
120
|
+
directory. We use Visual C++ on a 32-bit machine, so the
|
121
|
+
maximal cache size is 2GB.
|
122
|
+
|
123
|
+
`svm-train' Usage
|
124
|
+
=================
|
125
|
+
|
126
|
+
Usage: svm-train [options] training_set_file [model_file]
|
127
|
+
options:
|
128
|
+
-s svm_type : set type of SVM (default 0)
|
129
|
+
0 -- C-SVC
|
130
|
+
1 -- nu-SVC
|
131
|
+
2 -- one-class SVM
|
132
|
+
3 -- epsilon-SVR
|
133
|
+
4 -- nu-SVR
|
134
|
+
-t kernel_type : set type of kernel function (default 2)
|
135
|
+
0 -- linear: u'*v
|
136
|
+
1 -- polynomial: (gamma*u'*v + coef0)^degree
|
137
|
+
2 -- radial basis function: exp(-gamma*|u-v|^2)
|
138
|
+
3 -- sigmoid: tanh(gamma*u'*v + coef0)
|
139
|
+
4 -- precomputed kernel (kernel values in training_set_file)
|
140
|
+
-d degree : set degree in kernel function (default 3)
|
141
|
+
-g gamma : set gamma in kernel function (default 1/num_features)
|
142
|
+
-r coef0 : set coef0 in kernel function (default 0)
|
143
|
+
-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
|
144
|
+
-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
|
145
|
+
-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
|
146
|
+
-m cachesize : set cache memory size in MB (default 100)
|
147
|
+
-e epsilon : set tolerance of termination criterion (default 0.001)
|
148
|
+
-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)
|
149
|
+
-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
|
150
|
+
-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)
|
151
|
+
-v n: n-fold cross validation mode
|
152
|
+
-q : quiet mode (no outputs)
|
153
|
+
|
154
|
+
|
155
|
+
The k in the -g option means the number of attributes in the input data.
|
156
|
+
|
157
|
+
option -v randomly splits the data into n parts and calculates cross
|
158
|
+
validation accuracy/mean squared error on them.
|
159
|
+
|
160
|
+
See libsvm FAQ for the meaning of outputs.
|
161
|
+
|
162
|
+
`svm-predict' Usage
|
163
|
+
===================
|
164
|
+
|
165
|
+
Usage: svm-predict [options] test_file model_file output_file
|
166
|
+
options:
|
167
|
+
-b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0); for one-class SVM only 0 is supported
|
168
|
+
|
169
|
+
model_file is the model file generated by svm-train.
|
170
|
+
test_file is the test data you want to predict.
|
171
|
+
svm-predict will produce output in the output_file.
|
172
|
+
|
173
|
+
`svm-scale' Usage
|
174
|
+
=================
|
175
|
+
|
176
|
+
Usage: svm-scale [options] data_filename
|
177
|
+
options:
|
178
|
+
-l lower : x scaling lower limit (default -1)
|
179
|
+
-u upper : x scaling upper limit (default +1)
|
180
|
+
-y y_lower y_upper : y scaling limits (default: no y scaling)
|
181
|
+
-s save_filename : save scaling parameters to save_filename
|
182
|
+
-r restore_filename : restore scaling parameters from restore_filename
|
183
|
+
|
184
|
+
See 'Examples' in this file for examples.
|
185
|
+
|
186
|
+
Tips on Practical Use
|
187
|
+
=====================
|
188
|
+
|
189
|
+
* Scale your data. For example, scale each attribute to [0,1] or [-1,+1].
|
190
|
+
* For C-SVC, consider using the model selection tool in the tools directory.
|
191
|
+
* nu in nu-SVC/one-class-SVM/nu-SVR approximates the fraction of training
|
192
|
+
errors and support vectors.
|
193
|
+
* If data for classification are unbalanced (e.g. many positive and
|
194
|
+
few negative), try different penalty parameters C by -wi (see
|
195
|
+
examples below).
|
196
|
+
* Specify larger cache size (i.e., larger -m) for huge problems.
|
197
|
+
|
198
|
+
Examples
|
199
|
+
========
|
200
|
+
|
201
|
+
> svm-scale -l -1 -u 1 -s range train > train.scale
|
202
|
+
> svm-scale -r range test > test.scale
|
203
|
+
|
204
|
+
Scale each feature of the training data to be in [-1,1]. Scaling
|
205
|
+
factors are stored in the file range and then used for scaling the
|
206
|
+
test data.
|
207
|
+
|
208
|
+
> svm-train -s 0 -c 5 -t 2 -g 0.5 -e 0.1 data_file
|
209
|
+
|
210
|
+
Train a classifier with RBF kernel exp(-0.5|u-v|^2), C=10, and
|
211
|
+
stopping tolerance 0.1.
|
212
|
+
|
213
|
+
> svm-train -s 3 -p 0.1 -t 0 data_file
|
214
|
+
|
215
|
+
Solve SVM regression with linear kernel u'v and epsilon=0.1
|
216
|
+
in the loss function.
|
217
|
+
|
218
|
+
> svm-train -c 10 -w1 1 -w2 5 -w4 2 data_file
|
219
|
+
|
220
|
+
Train a classifier with penalty 10 = 1 * 10 for class 1, penalty 50 =
|
221
|
+
5 * 10 for class 2, and penalty 20 = 2 * 10 for class 4.
|
222
|
+
|
223
|
+
> svm-train -s 0 -c 100 -g 0.1 -v 5 data_file
|
224
|
+
|
225
|
+
Do five-fold cross validation for the classifier using
|
226
|
+
the parameters C = 100 and gamma = 0.1
|
227
|
+
|
228
|
+
> svm-train -s 0 -b 1 data_file
|
229
|
+
> svm-predict -b 1 test_file data_file.model output_file
|
230
|
+
|
231
|
+
Obtain a model with probability information and predict test data with
|
232
|
+
probability estimates
|
233
|
+
|
234
|
+
Precomputed Kernels
|
235
|
+
===================
|
236
|
+
|
237
|
+
Users may precompute kernel values and input them as training and
|
238
|
+
testing files. Then libsvm does not need the original
|
239
|
+
training/testing sets.
|
240
|
+
|
241
|
+
Assume there are L training instances x1, ..., xL and.
|
242
|
+
Let K(x, y) be the kernel
|
243
|
+
value of two instances x and y. The input formats
|
244
|
+
are:
|
245
|
+
|
246
|
+
New training instance for xi:
|
247
|
+
|
248
|
+
<label> 0:i 1:K(xi,x1) ... L:K(xi,xL)
|
249
|
+
|
250
|
+
New testing instance for any x:
|
251
|
+
|
252
|
+
<label> 0:? 1:K(x,x1) ... L:K(x,xL)
|
253
|
+
|
254
|
+
That is, in the training file the first column must be the "ID" of
|
255
|
+
xi. In testing, ? can be any value.
|
256
|
+
|
257
|
+
All kernel values including ZEROs must be explicitly provided. Any
|
258
|
+
permutation or random subsets of the training/testing files are also
|
259
|
+
valid (see examples below).
|
260
|
+
|
261
|
+
Note: the format is slightly different from the precomputed kernel
|
262
|
+
package released in libsvmtools earlier.
|
263
|
+
|
264
|
+
Examples:
|
265
|
+
|
266
|
+
Assume the original training data has three four-feature
|
267
|
+
instances and testing data has one instance:
|
268
|
+
|
269
|
+
15 1:1 2:1 3:1 4:1
|
270
|
+
45 2:3 4:3
|
271
|
+
25 3:1
|
272
|
+
|
273
|
+
15 1:1 3:1
|
274
|
+
|
275
|
+
If the linear kernel is used, we have the following new
|
276
|
+
training/testing sets:
|
277
|
+
|
278
|
+
15 0:1 1:4 2:6 3:1
|
279
|
+
45 0:2 1:6 2:18 3:0
|
280
|
+
25 0:3 1:1 2:0 3:1
|
281
|
+
|
282
|
+
15 0:? 1:2 2:0 3:1
|
283
|
+
|
284
|
+
? can be any value.
|
285
|
+
|
286
|
+
Any subset of the above training file is also valid. For example,
|
287
|
+
|
288
|
+
25 0:3 1:1 2:0 3:1
|
289
|
+
45 0:2 1:6 2:18 3:0
|
290
|
+
|
291
|
+
implies that the kernel matrix is
|
292
|
+
|
293
|
+
[K(2,2) K(2,3)] = [18 0]
|
294
|
+
[K(3,2) K(3,3)] = [0 1]
|
295
|
+
|
296
|
+
Library Usage
|
297
|
+
=============
|
298
|
+
|
299
|
+
These functions and structures are declared in the header file
|
300
|
+
`svm.h'. You need to #include "svm.h" in your C/C++ source files and
|
301
|
+
link your program with `svm.cpp'. You can see `svm-train.c' and
|
302
|
+
`svm-predict.c' for examples showing how to use them. We define
|
303
|
+
LIBSVM_VERSION and declare `extern int libsvm_version; ' in svm.h, so
|
304
|
+
you can check the version number.
|
305
|
+
|
306
|
+
Before you classify test data, you need to construct an SVM model
|
307
|
+
(`svm_model') using training data. A model can also be saved in
|
308
|
+
a file for later use. Once an SVM model is available, you can use it
|
309
|
+
to classify new data.
|
310
|
+
|
311
|
+
- Function: struct svm_model *svm_train(const struct svm_problem *prob,
|
312
|
+
const struct svm_parameter *param);
|
313
|
+
|
314
|
+
This function constructs and returns an SVM model according to
|
315
|
+
the given training data and parameters.
|
316
|
+
|
317
|
+
struct svm_problem describes the problem:
|
318
|
+
|
319
|
+
struct svm_problem
|
320
|
+
{
|
321
|
+
int l;
|
322
|
+
double *y;
|
323
|
+
struct svm_node **x;
|
324
|
+
};
|
325
|
+
|
326
|
+
where `l' is the number of training data, and `y' is an array containing
|
327
|
+
their target values. (integers in classification, real numbers in
|
328
|
+
regression) `x' is an array of pointers, each of which points to a sparse
|
329
|
+
representation (array of svm_node) of one training vector.
|
330
|
+
|
331
|
+
For example, if we have the following training data:
|
332
|
+
|
333
|
+
LABEL ATTR1 ATTR2 ATTR3 ATTR4 ATTR5
|
334
|
+
----- ----- ----- ----- ----- -----
|
335
|
+
1 0 0.1 0.2 0 0
|
336
|
+
2 0 0.1 0.3 -1.2 0
|
337
|
+
1 0.4 0 0 0 0
|
338
|
+
2 0 0.1 0 1.4 0.5
|
339
|
+
3 -0.1 -0.2 0.1 1.1 0.1
|
340
|
+
|
341
|
+
then the components of svm_problem are:
|
342
|
+
|
343
|
+
l = 5
|
344
|
+
|
345
|
+
y -> 1 2 1 2 3
|
346
|
+
|
347
|
+
x -> [ ] -> (2,0.1) (3,0.2) (-1,?)
|
348
|
+
[ ] -> (2,0.1) (3,0.3) (4,-1.2) (-1,?)
|
349
|
+
[ ] -> (1,0.4) (-1,?)
|
350
|
+
[ ] -> (2,0.1) (4,1.4) (5,0.5) (-1,?)
|
351
|
+
[ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (-1,?)
|
352
|
+
|
353
|
+
where (index,value) is stored in the structure `svm_node':
|
354
|
+
|
355
|
+
struct svm_node
|
356
|
+
{
|
357
|
+
int index;
|
358
|
+
double value;
|
359
|
+
};
|
360
|
+
|
361
|
+
index = -1 indicates the end of one vector. Note that indices must
|
362
|
+
be in ASCENDING order.
|
363
|
+
|
364
|
+
struct svm_parameter describes the parameters of an SVM model:
|
365
|
+
|
366
|
+
struct svm_parameter
|
367
|
+
{
|
368
|
+
int svm_type;
|
369
|
+
int kernel_type;
|
370
|
+
int degree; /* for poly */
|
371
|
+
double gamma; /* for poly/rbf/sigmoid */
|
372
|
+
double coef0; /* for poly/sigmoid */
|
373
|
+
|
374
|
+
/* these are for training only */
|
375
|
+
double cache_size; /* in MB */
|
376
|
+
double eps; /* stopping criteria */
|
377
|
+
double C; /* for C_SVC, EPSILON_SVR, and NU_SVR */
|
378
|
+
int nr_weight; /* for C_SVC */
|
379
|
+
int *weight_label; /* for C_SVC */
|
380
|
+
double* weight; /* for C_SVC */
|
381
|
+
double nu; /* for NU_SVC, ONE_CLASS, and NU_SVR */
|
382
|
+
double p; /* for EPSILON_SVR */
|
383
|
+
int shrinking; /* use the shrinking heuristics */
|
384
|
+
int probability; /* do probability estimates */
|
385
|
+
};
|
386
|
+
|
387
|
+
svm_type can be one of C_SVC, NU_SVC, ONE_CLASS, EPSILON_SVR, NU_SVR.
|
388
|
+
|
389
|
+
C_SVC: C-SVM classification
|
390
|
+
NU_SVC: nu-SVM classification
|
391
|
+
ONE_CLASS: one-class-SVM
|
392
|
+
EPSILON_SVR: epsilon-SVM regression
|
393
|
+
NU_SVR: nu-SVM regression
|
394
|
+
|
395
|
+
kernel_type can be one of LINEAR, POLY, RBF, SIGMOID.
|
396
|
+
|
397
|
+
LINEAR: u'*v
|
398
|
+
POLY: (gamma*u'*v + coef0)^degree
|
399
|
+
RBF: exp(-gamma*|u-v|^2)
|
400
|
+
SIGMOID: tanh(gamma*u'*v + coef0)
|
401
|
+
PRECOMPUTED: kernel values in training_set_file
|
402
|
+
|
403
|
+
cache_size is the size of the kernel cache, specified in megabytes.
|
404
|
+
C is the cost of constraints violation.
|
405
|
+
eps is the stopping criterion. (we usually use 0.00001 in nu-SVC,
|
406
|
+
0.001 in others). nu is the parameter in nu-SVM, nu-SVR, and
|
407
|
+
one-class-SVM. p is the epsilon in epsilon-insensitive loss function
|
408
|
+
of epsilon-SVM regression. shrinking = 1 means shrinking is conducted;
|
409
|
+
= 0 otherwise. probability = 1 means model with probability
|
410
|
+
information is obtained; = 0 otherwise.
|
411
|
+
|
412
|
+
nr_weight, weight_label, and weight are used to change the penalty
|
413
|
+
for some classes (If the weight for a class is not changed, it is
|
414
|
+
set to 1). This is useful for training classifier using unbalanced
|
415
|
+
input data or with asymmetric misclassification cost.
|
416
|
+
|
417
|
+
nr_weight is the number of elements in the array weight_label and
|
418
|
+
weight. Each weight[i] corresponds to weight_label[i], meaning that
|
419
|
+
the penalty of class weight_label[i] is scaled by a factor of weight[i].
|
420
|
+
|
421
|
+
If you do not want to change penalty for any of the classes,
|
422
|
+
just set nr_weight to 0.
|
423
|
+
|
424
|
+
*NOTE* Because svm_model contains pointers to svm_problem, you can
|
425
|
+
not free the memory used by svm_problem if you are still using the
|
426
|
+
svm_model produced by svm_train().
|
427
|
+
|
428
|
+
*NOTE* To avoid wrong parameters, svm_check_parameter() should be
|
429
|
+
called before svm_train().
|
430
|
+
|
431
|
+
struct svm_model stores the model obtained from the training procedure.
|
432
|
+
It is not recommended to directly access entries in this structure.
|
433
|
+
Programmers should use the interface functions to get the values.
|
434
|
+
|
435
|
+
struct svm_model
|
436
|
+
{
|
437
|
+
struct svm_parameter param; /* parameter */
|
438
|
+
int nr_class; /* number of classes, = 2 in regression/one class svm */
|
439
|
+
int l; /* total #SV */
|
440
|
+
struct svm_node **SV; /* SVs (SV[l]) */
|
441
|
+
double **sv_coef; /* coefficients for SVs in decision functions (sv_coef[k-1][l]) */
|
442
|
+
double *rho; /* constants in decision functions (rho[k*(k-1)/2]) */
|
443
|
+
double *probA; /* pairwise probability information */
|
444
|
+
double *probB;
|
445
|
+
|
446
|
+
/* for classification only */
|
447
|
+
|
448
|
+
int *label; /* label of each class (label[k]) */
|
449
|
+
int *nSV; /* number of SVs for each class (nSV[k]) */
|
450
|
+
/* nSV[0] + nSV[1] + ... + nSV[k-1] = l */
|
451
|
+
/* XXX */
|
452
|
+
int free_sv; /* 1 if svm_model is created by svm_load_model*/
|
453
|
+
/* 0 if svm_model is created by svm_train */
|
454
|
+
};
|
455
|
+
|
456
|
+
param describes the parameters used to obtain the model.
|
457
|
+
|
458
|
+
nr_class is the number of classes. It is 2 for regression and one-class SVM.
|
459
|
+
|
460
|
+
l is the number of support vectors. SV and sv_coef are support
|
461
|
+
vectors and the corresponding coefficients, respectively. Assume there are
|
462
|
+
k classes. For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,
|
463
|
+
where alpha's are solutions of the following two class problems:
|
464
|
+
1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k
|
465
|
+
and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j
|
466
|
+
vectors. For example, if there are 4 classes, sv_coef and SV are like:
|
467
|
+
|
468
|
+
+-+-+-+--------------------+
|
469
|
+
|1|1|1| |
|
470
|
+
|v|v|v| SVs from class 1 |
|
471
|
+
|2|3|4| |
|
472
|
+
+-+-+-+--------------------+
|
473
|
+
|1|2|2| |
|
474
|
+
|v|v|v| SVs from class 2 |
|
475
|
+
|2|3|4| |
|
476
|
+
+-+-+-+--------------------+
|
477
|
+
|1|2|3| |
|
478
|
+
|v|v|v| SVs from class 3 |
|
479
|
+
|3|3|4| |
|
480
|
+
+-+-+-+--------------------+
|
481
|
+
|1|2|3| |
|
482
|
+
|v|v|v| SVs from class 4 |
|
483
|
+
|4|4|4| |
|
484
|
+
+-+-+-+--------------------+
|
485
|
+
|
486
|
+
See svm_train() for an example of assigning values to sv_coef.
|
487
|
+
|
488
|
+
rho is the bias term (-b). probA and probB are parameters used in
|
489
|
+
probability outputs. If there are k classes, there are k*(k-1)/2
|
490
|
+
binary problems as well as rho, probA, and probB values. They are
|
491
|
+
aligned in the order of binary problems:
|
492
|
+
1 vs 2, 1 vs 3, ..., 1 vs k, 2 vs 3, ..., 2 vs k, ..., k-1 vs k.
|
493
|
+
|
494
|
+
label contains labels in the training data.
|
495
|
+
|
496
|
+
nSV is the number of support vectors in each class.
|
497
|
+
|
498
|
+
free_sv is a flag used to determine whether the space of SV should
|
499
|
+
be released in free_model_content(struct svm_model*) and
|
500
|
+
free_and_destroy_model(struct svm_model**). If the model is
|
501
|
+
generated by svm_train(), then SV points to data in svm_problem
|
502
|
+
and should not be removed. For example, free_sv is 0 if svm_model
|
503
|
+
is created by svm_train, but is 0 if created by svm_load_model.
|
504
|
+
|
505
|
+
- Function: double svm_predict(const struct svm_model *model,
|
506
|
+
const struct svm_node *x);
|
507
|
+
|
508
|
+
This function does classification or regression on a test vector x
|
509
|
+
given a model.
|
510
|
+
|
511
|
+
For a classification model, the predicted class for x is returned.
|
512
|
+
For a regression model, the function value of x calculated using
|
513
|
+
the model is returned. For an one-class model, +1 or -1 is
|
514
|
+
returned.
|
515
|
+
|
516
|
+
- Function: void svm_cross_validation(const struct svm_problem *prob,
|
517
|
+
const struct svm_parameter *param, int nr_fold, double *target);
|
518
|
+
|
519
|
+
This function conducts cross validation. Data are separated to
|
520
|
+
nr_fold folds. Under given parameters, sequentially each fold is
|
521
|
+
validated using the model from training the remaining. Predicted
|
522
|
+
labels (of all prob's instances) in the validation process are
|
523
|
+
stored in the array called target.
|
524
|
+
|
525
|
+
The format of svm_prob is same as that for svm_train().
|
526
|
+
|
527
|
+
- Function: int svm_get_svm_type(const struct svm_model *model);
|
528
|
+
|
529
|
+
This function gives svm_type of the model. Possible values of
|
530
|
+
svm_type are defined in svm.h.
|
531
|
+
|
532
|
+
- Function: int svm_get_nr_class(const svm_model *model);
|
533
|
+
|
534
|
+
For a classification model, this function gives the number of
|
535
|
+
classes. For a regression or an one-class model, 2 is returned.
|
536
|
+
|
537
|
+
- Function: void svm_get_labels(const svm_model *model, int* label)
|
538
|
+
|
539
|
+
For a classification model, this function outputs the name of
|
540
|
+
labels into an array called label. For regression and one-class
|
541
|
+
models, label is unchanged.
|
542
|
+
|
543
|
+
- Function: double svm_get_svr_probability(const struct svm_model *model);
|
544
|
+
|
545
|
+
For a regression model with probability information, this function
|
546
|
+
outputs a value sigma > 0. For test data, we consider the
|
547
|
+
probability model: target value = predicted value + z, z: Laplace
|
548
|
+
distribution e^(-|z|/sigma)/(2sigma)
|
549
|
+
|
550
|
+
If the model is not for svr or does not contain required
|
551
|
+
information, 0 is returned.
|
552
|
+
|
553
|
+
- Function: double svm_predict_values(const svm_model *model,
|
554
|
+
const svm_node *x, double* dec_values)
|
555
|
+
|
556
|
+
This function gives decision values on a test vector x given a
|
557
|
+
model, and return the predicted label (classification) or
|
558
|
+
the function value (regression).
|
559
|
+
|
560
|
+
For a classification model with nr_class classes, this function
|
561
|
+
gives nr_class*(nr_class-1)/2 decision values in the array
|
562
|
+
dec_values, where nr_class can be obtained from the function
|
563
|
+
svm_get_nr_class. The order is label[0] vs. label[1], ...,
|
564
|
+
label[0] vs. label[nr_class-1], label[1] vs. label[2], ...,
|
565
|
+
label[nr_class-2] vs. label[nr_class-1], where label can be
|
566
|
+
obtained from the function svm_get_labels. The returned value is
|
567
|
+
the predicted class for x. Note that when nr_class = 1, this
|
568
|
+
function does not give any decision value.
|
569
|
+
|
570
|
+
For a regression model, dec_values[0] and the returned value are
|
571
|
+
both the function value of x calculated using the model. For a
|
572
|
+
one-class model, dec_values[0] is the decision value of x, while
|
573
|
+
the returned value is +1/-1.
|
574
|
+
|
575
|
+
- Function: double svm_predict_probability(const struct svm_model *model,
|
576
|
+
const struct svm_node *x, double* prob_estimates);
|
577
|
+
|
578
|
+
This function does classification or regression on a test vector x
|
579
|
+
given a model with probability information.
|
580
|
+
|
581
|
+
For a classification model with probability information, this
|
582
|
+
function gives nr_class probability estimates in the array
|
583
|
+
prob_estimates. nr_class can be obtained from the function
|
584
|
+
svm_get_nr_class. The class with the highest probability is
|
585
|
+
returned. For regression/one-class SVM, the array prob_estimates
|
586
|
+
is unchanged and the returned value is the same as that of
|
587
|
+
svm_predict.
|
588
|
+
|
589
|
+
- Function: const char *svm_check_parameter(const struct svm_problem *prob,
|
590
|
+
const struct svm_parameter *param);
|
591
|
+
|
592
|
+
This function checks whether the parameters are within the feasible
|
593
|
+
range of the problem. This function should be called before calling
|
594
|
+
svm_train() and svm_cross_validation(). It returns NULL if the
|
595
|
+
parameters are feasible, otherwise an error message is returned.
|
596
|
+
|
597
|
+
- Function: int svm_check_probability_model(const struct svm_model *model);
|
598
|
+
|
599
|
+
This function checks whether the model contains required
|
600
|
+
information to do probability estimates. If so, it returns
|
601
|
+
+1. Otherwise, 0 is returned. This function should be called
|
602
|
+
before calling svm_get_svr_probability and
|
603
|
+
svm_predict_probability.
|
604
|
+
|
605
|
+
- Function: int svm_save_model(const char *model_file_name,
|
606
|
+
const struct svm_model *model);
|
607
|
+
|
608
|
+
This function saves a model to a file; returns 0 on success, or -1
|
609
|
+
if an error occurs.
|
610
|
+
|
611
|
+
- Function: struct svm_model *svm_load_model(const char *model_file_name);
|
612
|
+
|
613
|
+
This function returns a pointer to the model read from the file,
|
614
|
+
or a null pointer if the model could not be loaded.
|
615
|
+
|
616
|
+
- Function: void svm_free_model_content(struct svm_model *model_ptr);
|
617
|
+
|
618
|
+
This function frees the memory used by the entries in a model structure.
|
619
|
+
|
620
|
+
- Function: void svm_free_and_destroy_model(struct svm_model **model_ptr_ptr);
|
621
|
+
|
622
|
+
This function frees the memory used by a model and destroys the model
|
623
|
+
structure. It is equivalent to svm_destroy_model, which
|
624
|
+
is deprecated after version 3.0.
|
625
|
+
|
626
|
+
- Function: void svm_destroy_param(struct svm_parameter *param);
|
627
|
+
|
628
|
+
This function frees the memory used by a parameter set.
|
629
|
+
|
630
|
+
- Function: void svm_set_print_string_function(void (*print_func)(const char *));
|
631
|
+
|
632
|
+
Users can specify their output format by a function. Use
|
633
|
+
svm_set_print_string_function(NULL);
|
634
|
+
for default printing to stdout.
|
635
|
+
|
636
|
+
Java Version
|
637
|
+
============
|
638
|
+
|
639
|
+
The pre-compiled java class archive `libsvm.jar' and its source files are
|
640
|
+
in the java directory. To run the programs, use
|
641
|
+
|
642
|
+
java -classpath libsvm.jar svm_train <arguments>
|
643
|
+
java -classpath libsvm.jar svm_predict <arguments>
|
644
|
+
java -classpath libsvm.jar svm_toy
|
645
|
+
java -classpath libsvm.jar svm_scale <arguments>
|
646
|
+
|
647
|
+
Note that you need Java 1.5 (5.0) or above to run it.
|
648
|
+
|
649
|
+
You may need to add Java runtime library (like classes.zip) to the classpath.
|
650
|
+
You may need to increase maximum Java heap size.
|
651
|
+
|
652
|
+
Library usages are similar to the C version. These functions are available:
|
653
|
+
|
654
|
+
public class svm {
|
655
|
+
public static final int LIBSVM_VERSION=311;
|
656
|
+
public static svm_model svm_train(svm_problem prob, svm_parameter param);
|
657
|
+
public static void svm_cross_validation(svm_problem prob, svm_parameter param, int nr_fold, double[] target);
|
658
|
+
public static int svm_get_svm_type(svm_model model);
|
659
|
+
public static int svm_get_nr_class(svm_model model);
|
660
|
+
public static void svm_get_labels(svm_model model, int[] label);
|
661
|
+
public static double svm_get_svr_probability(svm_model model);
|
662
|
+
public static double svm_predict_values(svm_model model, svm_node[] x, double[] dec_values);
|
663
|
+
public static double svm_predict(svm_model model, svm_node[] x);
|
664
|
+
public static double svm_predict_probability(svm_model model, svm_node[] x, double[] prob_estimates);
|
665
|
+
public static void svm_save_model(String model_file_name, svm_model model) throws IOException
|
666
|
+
public static svm_model svm_load_model(String model_file_name) throws IOException
|
667
|
+
public static String svm_check_parameter(svm_problem prob, svm_parameter param);
|
668
|
+
public static int svm_check_probability_model(svm_model model);
|
669
|
+
public static void svm_set_print_string_function(svm_print_interface print_func);
|
670
|
+
}
|
671
|
+
|
672
|
+
The library is in the "libsvm" package.
|
673
|
+
Note that in Java version, svm_node[] is not ended with a node whose index = -1.
|
674
|
+
|
675
|
+
Users can specify their output format by
|
676
|
+
|
677
|
+
your_print_func = new svm_print_interface()
|
678
|
+
{
|
679
|
+
public void print(String s)
|
680
|
+
{
|
681
|
+
// your own format
|
682
|
+
}
|
683
|
+
};
|
684
|
+
svm.svm_set_print_string_function(your_print_func);
|
685
|
+
|
686
|
+
Building Windows Binaries
|
687
|
+
=========================
|
688
|
+
|
689
|
+
Windows binaries are in the directory `windows'. To build them via
|
690
|
+
Visual C++, use the following steps:
|
691
|
+
|
692
|
+
1. Open a DOS command box (or Visual Studio Command Prompt) and change
|
693
|
+
to libsvm directory. If environment variables of VC++ have not been
|
694
|
+
set, type
|
695
|
+
|
696
|
+
"C:\Program Files\Microsoft Visual Studio 10.0\VC\bin\vcvars32.bat"
|
697
|
+
|
698
|
+
You may have to modify the above command according which version of
|
699
|
+
VC++ or where it is installed.
|
700
|
+
|
701
|
+
2. Type
|
702
|
+
|
703
|
+
nmake -f Makefile.win clean all
|
704
|
+
|
705
|
+
3. (optional) To build shared library libsvm.dll, type
|
706
|
+
|
707
|
+
nmake -f Makefile.win lib
|
708
|
+
|
709
|
+
Another way is to build them from Visual C++ environment. See details
|
710
|
+
in libsvm FAQ.
|
711
|
+
|
712
|
+
- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.
|
713
|
+
============================================================================
|
714
|
+
|
715
|
+
See the README file in the tools directory.
|
716
|
+
|
717
|
+
MATLAB/OCTAVE Interface
|
718
|
+
=======================
|
719
|
+
|
720
|
+
Please check the file README in the directory `matlab'.
|
721
|
+
|
722
|
+
Python Interface
|
723
|
+
================
|
724
|
+
|
725
|
+
See the README file in python directory.
|
726
|
+
|
727
|
+
Additional Information
|
728
|
+
======================
|
729
|
+
|
730
|
+
If you find LIBSVM helpful, please cite it as
|
731
|
+
|
732
|
+
Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support
|
733
|
+
vector machines. ACM Transactions on Intelligent Systems and
|
734
|
+
Technology, 2:27:1--27:27, 2011. Software available at
|
735
|
+
http://www.csie.ntu.edu.tw/~cjlin/libsvm
|
736
|
+
|
737
|
+
LIBSVM implementation document is available at
|
738
|
+
http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf
|
739
|
+
|
740
|
+
For any questions and comments, please email cjlin@csie.ntu.edu.tw
|
741
|
+
|
742
|
+
Acknowledgments:
|
743
|
+
This work was supported in part by the National Science
|
744
|
+
Council of Taiwan via the grant NSC 89-2213-E-002-013.
|
745
|
+
The authors thank their group members and users
|
746
|
+
for many helpful discussions and comments. They are listed in
|
747
|
+
http://www.csie.ntu.edu.tw/~cjlin/libsvm/acknowledgements
|
748
|
+
|