Welcome to the MATLAB bindings for liquidSVM. This has only been tested on Linux and MacOS.

% load some generated sample data sets banana_mc, banana_bc, reg
% all of them with train/test split
load data;

%% Least Squares Regression is the default
model = lsSVM(reg_train_x, reg_train_y,'DISPLAY','1');
model.test(reg_test_x,reg_test_y);
model.predict(reg_test_x, '-d 0');

%% For classification specify the MutliClass-Solver (this will convert the Y-Variable
model = mcSVM(banana_mc_train_x, banana_mc_train_y,'DISPLAY','1','folds','3');
model.test(banana_mc_test_x, banana_mc_test_y);

%% Specify further parameters, e.g. 8-fold cross-validation
model = liquidSvmModel(banana_bc_train_x, banana_bc_train_y, 'SCENARIO','MC');
% if liquidSvmModel is used then train and select have to be issued,
% and by that they can be configured
model.train('-d 1 -T 2 -f 3 8');
model.select('-d 1');

model.test(banana_bc_test_x, banana_bc_test_y, '-d', '2', '-T', '-1');
model.test(banana_bc_test_x, banana_bc_test_y, '-d 2 -T -1');
model.test(banana_bc_test_x, banana_bc_test_y);
model.predict(banana_bc_test_x, '-d 1');

%% Quantile Regression
model = qtSVM(reg_train_x, reg_train_y,[.2,.5],'DISPLAY','1');
model.test(reg_test_x,reg_test_y);

%% Expectile Regression
model = exSVM(reg_train_x, reg_train_y,'DISPLAY','1');
model.test(reg_test_x,reg_test_y);

%% Receiver Operating Characteristic curve
model = rocSVM(banana_bc_train_x, banana_bc_train_y,6,'DISPLAY','1');
model.test(banana_bc_test_x,banana_bc_test_y);

%% Neyman-Pearson lemma
model = nplSVM(banana_bc_train_x, banana_bc_train_y, 1, 'DISPLAY','1');
model.test(banana_bc_test_x,banana_bc_test_y);

%% Write a solution (after train and select have been performed)
model = lsSVM(reg_train_x, reg_train_y,'DISPLAY','1');
model.write_solution('reg-1d.sol');

%% read a solution from file
model = liquidSvmModel(reg_train_x, reg_train_y,'SCENARIO','LS','DISPLAY','1');
model.read_solution('reg-1d.sol');
model.test(reg_test_x,reg_test_y);
model.predict(reg_test_x, '-d 0');

The meaning of the configurations in the constructor is described in the next chapter.

Octave

Overview of Configuration Parameters

Specialized configuration parameters

Parameter for multiclass classification determine the multiclass strategy: mc-type=0 : AvA with hinge loss. mc-type=1 : OvA with least squares loss. mc-type=2 : OvA with hinge loss. mc-type=3 : AvA with least squares loss.

Hyperparameter Grid

liquidSVM has a built-in a cross-validation scheme to calculate validation errors for many values of these hyperparameters and then to choose the best pair. Since there are two parameters this means we consider a two-dimensional grid.

For both parameters either specific values can be given or a geometrically spaced grid can be specified.

Note the min and max values are scaled according the the number of samples, the dimensionality of the data sets, the number of folds used, and the estimated diameter of the data set.

`grid_choice`	0	1	2
`gamma_steps`	10	15	20
`lambda_steps`	10	15	20
`min_gamma`	0.2	0.1	0.05
`max_gamma`	5.0	10.0	20.0
`min_lambda`	0.001	0.0001	0.00001
`max_lambda`	0.01	0.01	0.01

Using negative values of grid_choice we create a grid with listed gamma and lambda values:

Adaptive Grid

`grid_choice`	-1
`gammas`	`c(10.0, 5.0, 2.0, 1.0, 0.5, 0.25, 0.1, 0.05)`
`lambdas`	`c(1.0, 0.1, 0.01, 0.001, 0.0001, 0.00001, 0.000001, 0.0000001)`

`grid_choice`	-2
`gammas`	`c(10.0, 5.0, 2.0, 1.0, 0.5, 0.25, 0.1, 0.05)`
`c_values`	`c(0.01, 0.1, 1, 10, 100, 1000, 10000)`

An adaptive grid search can be activated. The higher the values of MAX_LAMBDA_INCREASES and MAX_NUMBER_OF_WORSE_GAMMAS are set the more conservative the search strategy is. The values can be freely modified.

Cells

`ADAPTIVITY_CONTROL`	1	2
`MAX_LAMBDA_INCREASES`	4	3
`MAX_NUMBER_OF_WORSE_GAMMAS`	4	3

A major issue with SVMs is that for larger sample sizes the kernel matrix does not fit into the memory any more. Classically this gives an upper limit for the class of problems that traditional SVMs can handle without significant runtime increase. Furthermore also the time complexity is at least O(n²).

liquidSVM implements two major concepts to circumvent these issues. One is random chunks which is known well in the literature. However we prefer the new alternative of splitting the space into spatial cells and use local SVMs on every cell.

If you specify useCells=TRUE then the sample space X gets partitioned into a number of cells. The training is done first for cell 1 then for cell 2 and so on. Now, to predict the label for a value x ∈ X liquidSVM first finds out to which cell this x belongs and then uses the SVM of that cell to predict a label for it.

This is quite performant, since the complexity in both time and memore are both O(CELLSIZE × n) and this holds both for training as well as testing! It also can be shown that the quality of the solution is comparable, at least for moderate dimensions.

The first parameter values correspond to NO_PARTITION, RANDOM_CHUNK_BY_SIZE, RANDOM_CHUNK_BY_NUMBER, VORONOI_BY_RADIUS, VORONOI_BY_SIZE, OVERLAP_BY_SIZE

Weights

NPL:
WEIGHT_STEPS=10
MIN_WEIGHT=0.001
MAX_WEIGHT=0.5
GEO_WEIGHTS=1

ROC:
WEIGHT_STEPS=9
MAX_WEIGHT=0.9
MIN_WEIGHT=0.1
GEO_WEIGHTS=0

More Advanced Parameters

The following parameters should only employed by experienced users and are self-explanatory for these:

For completeness here are some values that usually get set by the learning scenario

liquidSVM for MATLAB