Stochastic Gradient Descent SVM classifier. More...

#include "ml.hpp"

Inheritance diagram for cv::ml::SVMSGD:

Public Types
enum	MarginType { SOFT_MARGIN, HARD_MARGIN }

enum	SvmsgdType { SGD, ASGD }

Public Types inherited from cv::ml::StatModel
enum	Flags { UPDATE_MODEL = 1, RAW_OUTPUT =1, COMPRESSED_INPUT =2, PREPROCESSED_INPUT =4 }

Public Member Functions
virtual float	getInitialStepSize () const =0
	Parameter initialStepSize of a SVMSGD optimization problem. More...

virtual float	getMarginRegularization () const =0
	Parameter marginRegularization of a SVMSGD optimization problem. More...

virtual int	getMarginType () const =0
	Margin type, one of SVMSGD::MarginType. More...

virtual float	getShift ()=0

virtual float	getStepDecreasingPower () const =0
	Parameter stepDecreasingPower of a SVMSGD optimization problem. More...

virtual int	getSvmsgdType () const =0
	Algorithm type, one of SVMSGD::SvmsgdType. More...

virtual TermCriteria	getTermCriteria () const =0
	Termination criteria of the training algorithm. You can specify the maximum number of iterations (maxCount) and/or how much the error could change between the iterations to make the algorithm continue (epsilon). More...

virtual Mat	getWeights ()=0

virtual void	setInitialStepSize (float InitialStepSize)=0
	Parameter initialStepSize of a SVMSGD optimization problem. More...

virtual void	setMarginRegularization (float marginRegularization)=0
	Parameter marginRegularization of a SVMSGD optimization problem. More...

virtual void	setMarginType (int marginType)=0
	Margin type, one of SVMSGD::MarginType. More...

virtual void	setOptimalParameters (int svmsgdType=SVMSGD::ASGD, int marginType=SVMSGD::SOFT_MARGIN)=0
	Function sets optimal parameters values for chosen SVM SGD model. More...

virtual void	setStepDecreasingPower (float stepDecreasingPower)=0
	Parameter stepDecreasingPower of a SVMSGD optimization problem. More...

virtual void	setSvmsgdType (int svmsgdType)=0
	Algorithm type, one of SVMSGD::SvmsgdType. More...

virtual void	setTermCriteria (const cv::TermCriteria &val)=0
	Termination criteria of the training algorithm. You can specify the maximum number of iterations (maxCount) and/or how much the error could change between the iterations to make the algorithm continue (epsilon). More...

Public Member Functions inherited from cv::ml::StatModel
virtual float	calcError (const Ptr< cv::ml::TrainData > &data, bool test, OutputArray resp) const
	Computes error on the training or test dataset. More...

virtual bool	empty () const
	Returns true if the Algorithm is empty (e.g. in the very beginning or after unsuccessful read. More...

virtual int	getVarCount () const =0
	Returns the number of variables in training samples. More...

virtual bool	isClassifier () const =0
	Returns true if the model is classifier. More...

virtual bool	isTrained () const =0
	Returns true if the model is trained. More...

virtual float	predict (InputArray samples, OutputArray results=noArray(), int flags=0) const =0
	Predicts response(s) for the provided sample(s) More...

virtual bool	train (const Ptr< cv::ml::TrainData > &trainData, int flags=0)
	Trains the statistical model. More...

virtual bool	train (InputArray samples, int layout, InputArray responses)
	Trains the statistical model. More...

Public Member Functions inherited from cv::Algorithm
	Algorithm ()

virtual	~Algorithm ()

virtual void	clear ()
	Clears the algorithm state. More...

virtual String	getDefaultName () const

virtual void	read (const FileNode &fn)
	Reads algorithm parameters from a file storage. More...

virtual void	save (const String &filename) const

virtual void	write (FileStorage &fs) const
	Stores algorithm parameters in a file storage. More...

Static Public Member Functions
static Ptr< SVMSGD >	create ()
	Creates empty model. Use StatModel::train to train the model. Since SVMSGD has several parameters, you may want to find the best parameters for your problem or use setOptimalParameters() to set some default parameters. More...

Static Public Member Functions inherited from cv::ml::StatModel
template<typename _Tp >
static Ptr< _Tp >	train (const Ptr< cv::ml::TrainData > &data, int flags=0)
	Create and train model with default parameters. More...

Static Public Member Functions inherited from cv::Algorithm
template<typename _Tp >
static Ptr< _Tp >	load (const String &filename, const String &objname=String())
	Loads algorithm from the file. More...

template<typename _Tp >
static Ptr< _Tp >	loadFromString (const String &strModel, const String &objname=String())
	Loads algorithm from a String. More...

template<typename _Tp >
static Ptr< _Tp >	read (const FileNode &fn)
	Reads algorithm from the file node. More...

Additional Inherited Members
Protected Member Functions inherited from cv::Algorithm
void	writeFormat (FileStorage &fs) const

Detailed Description

Stochastic Gradient Descent SVM classifier.

SVMSGD provides a fast and easy-to-use implementation of the SVM classifier using the Stochastic Gradient Descent approach, as presented in [15].

The classifier has following parameters:

model type,
margin type,
margin regularization ( \(\lambda\)),
initial step size ( \(\gamma_0\)),
step decreasing power ( \(c\)),
and termination criteria.

The model type may have one of the following values: SGD and ASGD.

SGD is the classic version of SVMSGD classifier: every next step is calculated by the formula
\[w_{t+1} = w_t - \gamma(t) \frac{dQ_i}{dw} |_{w = w_t}\]
where
- \(w_t\) is the weights vector for decision function at step \(t\),
- \(\gamma(t)\) is the step size of model parameters at the iteration \(t\), it is decreased on each step by the formula \(\gamma(t) = \gamma_0 (1 + \lambda \gamma_0 t) ^ {-c}\)
- \(Q_i\) is the target functional from SVM task for sample with number \(i\), this sample is chosen stochastically on each step of the algorithm.
ASGD is Average Stochastic Gradient Descent SVM Classifier. ASGD classifier averages weights vector on each step of algorithm by the formula \(\widehat{w}_{t+1} = \frac{t}{1+t}\widehat{w}_{t} + \frac{1}{1+t}w_{t+1}\)

The recommended model type is ASGD (following [15]).

The margin type may have one of the following values: SOFT_MARGIN or HARD_MARGIN.

You should use HARD_MARGIN type, if you have linearly separable sets.
You should use SOFT_MARGIN type, if you have non-linearly separable sets or sets with outliers.
In the general case (if you know nothing about linear separability of your sets), use SOFT_MARGIN.

The other parameters may be described as follows:

Margin regularization parameter is responsible for weights decreasing at each step and for the strength of restrictions on outliers (the less the parameter, the less probability that an outlier will be ignored). Recommended value for SGD model is 0.0001, for ASGD model is 0.00001.
Initial step size parameter is the initial value for the step size \(\gamma(t)\). You will have to find the best initial step for your problem.
Step decreasing power is the power parameter for \(\gamma(t)\) decreasing by the formula, mentioned above. Recommended value for SGD model is 1, for ASGD model is 0.75.
Termination criteria can be TermCriteria::COUNT, TermCriteria::EPS or TermCriteria::COUNT + TermCriteria::EPS. You will have to find the best termination criteria for your problem.

Note that the parameters margin regularization, initial step size, and step decreasing power should be positive.

To use SVMSGD algorithm do as follows:

first, create the SVMSGD object. The algoorithm will set optimal parameters by default, but you can set your own parameters via functions setSvmsgdType(), setMarginType(), setMarginRegularization(), setInitialStepSize(), and setStepDecreasingPower().
then the SVM model can be trained using the train features and the correspondent labels by the method train().
after that, the label of a new feature vector can be predicted using the method predict().

// Create empty object
cv::Ptr<SVMSGD> svmsgd = SVMSGD::create();
// Train the Stochastic Gradient Descent SVM
svmsgd->train(trainData);
// Predict labels for the new samples
svmsgd->predict(samples, responses);

Member Enumeration Documentation

enum cv::ml::SVMSGD::MarginType

Margin type.

Enumerator
SOFT_MARGIN	General case, suits to the case of non-linearly separable sets, allows outliers.
HARD_MARGIN	More accurate for the case of linearly separable sets.

enum cv::ml::SVMSGD::SvmsgdType

SVMSGD type. ASGD is often the preferable choice.

Enumerator
SGD	Stochastic Gradient Descent.
ASGD	Average Stochastic Gradient Descent.

Member Function Documentation

static Ptr<SVMSGD> cv::ml::SVMSGD::create ( )

static

Creates empty model. Use StatModel::train to train the model. Since SVMSGD has several parameters, you may want to find the best parameters for your problem or use setOptimalParameters() to set some default parameters.

virtual float cv::ml::SVMSGD::getInitialStepSize ( ) const

pure virtual

Parameter initialStepSize of a SVMSGD optimization problem.

See also: setInitialStepSize

virtual float cv::ml::SVMSGD::getMarginRegularization ( ) const

pure virtual

Parameter marginRegularization of a SVMSGD optimization problem.

See also: setMarginRegularization

virtual int cv::ml::SVMSGD::getMarginType ( ) const

pure virtual

Margin type, one of SVMSGD::MarginType.

See also: setMarginType

virtual float cv::ml::SVMSGD::getShift ( )

pure virtual

Returns: the shift of the trained model (decision function f(x) = weights * x + shift).

virtual float cv::ml::SVMSGD::getStepDecreasingPower ( ) const

pure virtual

Parameter stepDecreasingPower of a SVMSGD optimization problem.

See also: setStepDecreasingPower

virtual int cv::ml::SVMSGD::getSvmsgdType ( ) const

pure virtual

Algorithm type, one of SVMSGD::SvmsgdType.

See also: setSvmsgdType

virtual TermCriteria cv::ml::SVMSGD::getTermCriteria ( ) const

pure virtual

Termination criteria of the training algorithm. You can specify the maximum number of iterations (maxCount) and/or how much the error could change between the iterations to make the algorithm continue (epsilon).

See also: setTermCriteria

virtual Mat cv::ml::SVMSGD::getWeights ( )

pure virtual

Returns: the weights of the trained model (decision function f(x) = weights * x + shift).

virtual void cv::ml::SVMSGD::setInitialStepSize ( float InitialStepSize )

pure virtual

Parameter initialStepSize of a SVMSGD optimization problem.

See also: getInitialStepSize

virtual void cv::ml::SVMSGD::setMarginRegularization ( float marginRegularization )

pure virtual

Parameter marginRegularization of a SVMSGD optimization problem.

See also: getMarginRegularization

virtual void cv::ml::SVMSGD::setMarginType ( int marginType )

pure virtual

Margin type, one of SVMSGD::MarginType.

See also: getMarginType

virtual void cv::ml::SVMSGD::setOptimalParameters	(	int	svmsgdType = `SVMSGD::ASGD`,
		int	marginType = `SVMSGD::SOFT_MARGIN`
	)

pure virtual

Function sets optimal parameters values for chosen SVM SGD model.

Parameters

svmsgdType	is the type of SVMSGD classifier.
marginType	is the type of margin constraint.

virtual void cv::ml::SVMSGD::setStepDecreasingPower ( float stepDecreasingPower )

pure virtual

Parameter stepDecreasingPower of a SVMSGD optimization problem.

See also: getStepDecreasingPower

virtual void cv::ml::SVMSGD::setSvmsgdType ( int svmsgdType )

pure virtual

Algorithm type, one of SVMSGD::SvmsgdType.

See also: getSvmsgdType

virtual void cv::ml::SVMSGD::setTermCriteria ( const cv::TermCriteria & val )

pure virtual

Termination criteria of the training algorithm. You can specify the maximum number of iterations (maxCount) and/or how much the error could change between the iterations to make the algorithm continue (epsilon).

See also: getTermCriteria

The documentation for this class was generated from the following file:

ml/include/opencv2/ml.hpp

Public Types

Public Member Functions

Static Public Member Functions

Additional Inherited Members

Detailed Description

Member Enumeration Documentation

Member Function Documentation