Training of C-SVMs for binary classification. More...
#include <shark/Algorithms/Trainers/CSvmTrainer.h>
Inheritance diagram for shark::CSvmTrainer< InputType, CacheType >:
Public Types | |
| typedef CacheType | QpFloatType |
| Convenience typedefs: this and many of the below typedefs build on the class template type CacheType. Simply changing that one template parameter CacheType thus allows to flexibly switch between using float or double as type for caching the kernel values. The default is float, offering sufficient accuracy in the vast majority of cases, at a memory cost of only four bytes. However, the template parameter makes it easy to use double instead, (e.g., in case high accuracy training is needed). More... | |
| typedef AbstractKernelFunction< InputType > | KernelType |
| Public Types inherited from shark::AbstractSvmTrainer< InputType, unsigned int, KernelClassifier< InputType >, AbstractWeightedTrainer< KernelClassifier< InputType > > > | |
| typedef AbstractKernelFunction< InputType > | KernelType |
| Public Types inherited from shark::AbstractWeightedTrainer< KernelClassifier< InputType > > | |
| typedef base_type::ModelType | ModelType |
| typedef base_type::InputType | InputType |
| typedef base_type::LabelType | LabelType |
| typedef base_type::DatasetType | DatasetType |
| typedef WeightedLabeledData< InputType, LabelType > | WeightedDatasetType |
| Public Types inherited from shark::IParameterizable<> | |
| typedef RealVector | ParameterVectorType |
Public Member Functions | |
| CSvmTrainer (KernelType *kernel, double C, bool offset, bool unconstrained=false) | |
| CSvmTrainer (KernelType *kernel, double negativeC, double positiveC, bool offset, bool unconstrained=false) | |
| std::string | name () const |
| From INameable: return the class name. More... | |
| void | setComputeBinaryDerivative (bool compute) |
| void | setMcSvmType (McSvm type) |
| sets the type of the multi-class svm used More... | |
| void | train (KernelClassifier< InputType > &svm, LabeledData< InputType, unsigned int > const &dataset) |
| Train the C-SVM. More... | |
| void | train (KernelClassifier< InputType > &svm, WeightedLabeledData< InputType, unsigned int > const &dataset) |
| Train the C-SVM using weights. More... | |
| RealVector const & | get_db_dParams () const |
| Public Member Functions inherited from shark::AbstractSvmTrainer< InputType, unsigned int, KernelClassifier< InputType >, AbstractWeightedTrainer< KernelClassifier< InputType > > > | |
| AbstractSvmTrainer (KernelType *kernel, double C, bool offset, bool unconstrained=false) | |
| AbstractSvmTrainer (KernelType *kernel, double negativeC, double positiveC, bool offset, bool unconstrained=false) | |
| double | C () const |
| Return the value of the regularization parameter C. More... | |
| void | setC (double C) |
| Set the value of the regularization parameter C. More... | |
| RealVector const & | regularizationParameters () const |
| void | setRegularizationParameters (RealVector const ®ularizers) |
| Set the value of the regularization parameter C. More... | |
| KernelType * | kernel () |
| KernelType const * | kernel () const |
| void | setKernel (KernelType *kernel) |
| bool | isUnconstrained () const |
| bool | trainOffset () const |
| std::size_t | cacheSize () const |
| void | setCacheSize (std::size_t size) |
| RealVector | parameterVector () const |
| get the hyper-parameter vector More... | |
| void | setParameterVector (RealVector const &newParameters) |
| set the vector of hyper-parameters More... | |
| size_t | numberOfParameters () const |
| return the number of hyper-parameters More... | |
| Public Member Functions inherited from shark::AbstractWeightedTrainer< KernelClassifier< InputType > > | |
| virtual void | train (ModelType &model, WeightedDatasetType const &dataset)=0 |
| Executes the algorithm and trains a model on the given weighted data. More... | |
| virtual void | train (ModelType &model, DatasetType const &dataset) |
| Executes the algorithm and trains a model on the given unweighted data. More... | |
| Public Member Functions inherited from shark::INameable | |
| virtual | ~INameable () |
| Public Member Functions inherited from shark::ISerializable | |
| virtual | ~ISerializable () |
| Virtual d'tor. More... | |
| virtual void | read (InArchive &archive) |
| Read the component from the supplied archive. More... | |
| virtual void | write (OutArchive &archive) const |
| Write the component to the supplied archive. More... | |
| void | load (InArchive &archive, unsigned int version) |
| Versioned loading of components, calls read(...). More... | |
| void | save (OutArchive &archive, unsigned int version) const |
| Versioned storing of components, calls write(...). More... | |
| BOOST_SERIALIZATION_SPLIT_MEMBER () | |
| Public Member Functions inherited from shark::QpConfig | |
| QpConfig (bool precomputedFlag=false, bool sparsifyFlag=true) | |
| Constructor. More... | |
| QpStoppingCondition & | stoppingCondition () |
| Read/write access to the stopping condition. More... | |
| QpStoppingCondition const & | stoppingCondition () const |
| Read access to the stopping condition. More... | |
| QpSolutionProperties & | solutionProperties () |
| Access to the solution properties. More... | |
| bool & | precomputeKernel () |
| Flag for using a precomputed kernel matrix. More... | |
| bool const & | precomputeKernel () const |
| Flag for using a precomputed kernel matrix. More... | |
| bool & | sparsify () |
| Flag for sparsifying the model after training. More... | |
| bool const & | sparsify () const |
| Flag for sparsifying the model after training. More... | |
| bool & | shrinking () |
| Flag for shrinking in the decomposition solver. More... | |
| bool const & | shrinking () const |
| Flag for shrinking in the decomposition solver. More... | |
| bool & | s2do () |
| Flag for S2DO (instead of SMO) More... | |
| bool const & | s2do () const |
| Flag for S2DO (instead of SMO) More... | |
| unsigned int & | verbosity () |
| Verbosity level of the solver. More... | |
| unsigned int const & | verbosity () const |
| Verbosity level of the solver. More... | |
| unsigned long long const & | accessCount () const |
| Number of kernel accesses. More... | |
| void | setMinAccuracy (double a) |
| void | setMaxIterations (unsigned long long i) |
| void | setTargetValue (double v) |
| void | setMaxSeconds (double s) |
| Public Member Functions inherited from shark::IParameterizable<> | |
| virtual | ~IParameterizable () |
Additional Inherited Members | |
| Protected Attributes inherited from shark::AbstractSvmTrainer< InputType, unsigned int, KernelClassifier< InputType >, AbstractWeightedTrainer< KernelClassifier< InputType > > > | |
| KernelType * | m_kernel |
| RealVector | m_regularizers |
| Vector of regularization parameters. More... | |
| bool | m_trainOffset |
| bool | m_unconstrained |
| Is log(C) stored internally as a parameter instead of C? If yes, then we get rid of the constraint C > 0 on the level of the parameter interface. More... | |
| std::size_t | m_cacheSize |
| Number of values in the kernel cache. The size of the cache in bytes is the size of one entry (4 for float, 8 for double) times this number. More... | |
| Protected Attributes inherited from shark::QpConfig | |
| QpStoppingCondition | m_stoppingcondition |
| conditions for when to stop the QP solver More... | |
| QpSolutionProperties | m_solutionproperties |
| properties of the approximate solution found by the solver More... | |
| bool | m_precomputedKernelMatrix |
| should the solver use a precomputed kernel matrix? More... | |
| bool | m_sparsify |
| should the trainer sparsify the model after training? More... | |
| bool | m_shrinking |
| should shrinking be used? More... | |
| bool | m_s2do |
| should S2DO be used instead of SMO? More... | |
| unsigned int | m_verbosity |
| verbosity level (currently unused) More... | |
| unsigned long long | m_accessCount |
| kernel access count More... | |
Detailed Description
template<class InputType, class CacheType = float>
class shark::CSvmTrainer< InputType, CacheType >
Training of C-SVMs for binary classification.
The C-SVM is the "standard" support vector machine for binary (two-class) classification. Given are data tuples \( (x_i, y_i) \) with x-component denoting input and y-component denoting the label +1 or -1 (see the tutorial on label conventions; the implementation uses values 0/1), a kernel function k(x, x') and a regularization constant C > 0. Let H denote the kernel induced reproducing kernel Hilbert space of k, and let \( \phi \) denote the corresponding feature map. Then the SVM classifier is the function
\[ h(x) = \mathop{sign} (f(x)) \]
\[ f(x) = \langle w, \phi(x) \rangle + b \]
with coefficients w and b given by the (primal) optimization problem
\[ \min \frac{1}{2} \|w\|^2 + C \sum_i L(y_i, f(x_i)), \]
where \( L(y, f(x)) = \max\{0, 1 - y \cdot f(x)\} \) denotes the hinge loss.
For details refer to the paper:
Support-Vector Networks. Corinna Cortes and Vladimir Vapnik, Machine Learning, vol. 20 (1995), pp. 273-297.
or simply to the Wikipedia article:
http://en.wikipedia.org/wiki/Support_vector_machine
Definition at line 74 of file CSvmTrainer.h.
Member Typedef Documentation
◆ KernelType
| typedef AbstractKernelFunction<InputType> shark::CSvmTrainer< InputType, CacheType >::KernelType |
Definition at line 98 of file CSvmTrainer.h.
◆ QpFloatType
| typedef CacheType shark::CSvmTrainer< InputType, CacheType >::QpFloatType |
Convenience typedefs: this and many of the below typedefs build on the class template type CacheType. Simply changing that one template parameter CacheType thus allows to flexibly switch between using float or double as type for caching the kernel values. The default is float, offering sufficient accuracy in the vast majority of cases, at a memory cost of only four bytes. However, the template parameter makes it easy to use double instead, (e.g., in case high accuracy training is needed).
Definition at line 96 of file CSvmTrainer.h.
Constructor & Destructor Documentation
◆ CSvmTrainer() [1/2]
|
inline |
Constructor
- Parameters
-
kernel kernel function to use for training and prediction C regularization parameter - always the 'true' value of C, even when unconstrained is set offset whether to train the svm with offset term unconstrained when a C-value is given via setParameter, should it be piped through the exp-function before using it in the solver?
Definition at line 105 of file CSvmTrainer.h.
◆ CSvmTrainer() [2/2]
|
inline |
Constructor
- Parameters
-
kernel kernel function to use for training and prediction negativeC regularization parameter of the negative class (label 0) positiveC regularization parameter of the positive class (label 1) offset whether to train the svm with offset term unconstrained when a C-value is given via setParameter, should it be piped through the exp-function before using it in the solver?
Definition at line 115 of file CSvmTrainer.h.
Member Function Documentation
◆ get_db_dParams()
|
inline |
Definition at line 279 of file CSvmTrainer.h.
References shark::QpSparseArray< QpFloatType >::add(), shark::Classifier< KernelExpansion< InputType > >::decisionFunction(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::numberOfClasses(), shark::oneVersusRestProblem(), shark::QpConfig::precomputeKernel(), shark::QpNone, shark::QpSparseArray< QpFloatType >::resize(), shark::QpConfig::s2do(), shark::AbstractSvmTrainer< InputType, unsigned int, KernelClassifier< InputType >, AbstractWeightedTrainer< KernelClassifier< InputType > > >::setCacheSize(), shark::QpSparseArray< QpFloatType >::setDefaultValue(), shark::QpConfig::shrinking(), shark::QpConfig::solutionProperties(), shark::QpSolver< Problem, SelectionStrategy >::solve(), shark::BiasSolver< Matrix >::solve(), shark::BiasSolverSimplex< Matrix >::solve(), shark::QpConfig::sparsify(), shark::QpConfig::stoppingCondition(), shark::CSvmTrainer< InputType, CacheType >::train(), and shark::QpConfig::verbosity().
◆ name()
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 120 of file CSvmTrainer.h.
Referenced by main().
◆ setComputeBinaryDerivative()
|
inline |
Definition at line 123 of file CSvmTrainer.h.
◆ setMcSvmType()
|
inline |
sets the type of the multi-class svm used
Definition at line 128 of file CSvmTrainer.h.
Referenced by main().
◆ train() [1/2]
|
inline |
Train the C-SVM.
Definition at line 134 of file CSvmTrainer.h.
References shark::ADM, shark::ATM, shark::ATS, shark::CS, shark::Classifier< KernelExpansion< InputType > >::decisionFunction(), shark::LabeledData< InputT, LabelT >::element(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::LLW, shark::MMR, shark::numberOfClasses(), shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::OVA, shark::ReinforcedSvm, shark::QpSparseArray< QpFloatType >::width(), and shark::WW.
Referenced by shark::CSvmTrainer< InputType, CacheType >::get_db_dParams(), main(), and run_one_trial().
◆ train() [2/2]
|
inline |
Train the C-SVM using weights.
Definition at line 259 of file CSvmTrainer.h.
References shark::Classifier< KernelExpansion< InputType > >::decisionFunction(), shark::WeightedLabeledData< InputT, LabelT >::inputs(), shark::numberOfClasses(), and SHARK_RUNTIME_CHECK.
The documentation for this class was generated from the following file:
- include/shark/Algorithms/Trainers/CSvmTrainer.h