Budgeted stochastic gradient descent training for kernel-based models. More...
#include <shark/Algorithms/Trainers/Budgeted/KernelBudgetedSGDTrainer.h>
Public Member Functions | |
KernelBudgetedSGDTrainer (KernelType *kernel, const LossType *loss, double C, bool offset, bool unconstrained=false, size_t budgetSize=500, AbstractBudgetMaintenanceStrategy< InputType > *budgetMaintenanceStrategy=NULL, size_t epochs=1, size_t preInitializationMethod=NONE, double minMargin=1.0f) | |
Constructor Note that there is no cache size involved, as merging vectors will always create new ones, which makes caching roughly obsolete. More... | |
size_t | budgetSize () const |
void | setBudgetSize (std::size_t budgetSize) |
AbstractBudgetMaintenanceStrategy< InputType > * | budgetMaintenanceStrategy () const |
void | setBudgetMaintenanceStrategy (AbstractBudgetMaintenanceStrategy< InputType > *budgetMaintenanceStrategy) |
double | minMargin () const |
void | setMinMargin (double minMargin) |
std::string | name () const |
From INameable: return the class name. More... | |
void | train (ClassifierType &classifier, const LabeledData< InputType, unsigned int > &dataset) |
std::size_t | epochs () const |
void | setEpochs (std::size_t value) |
KernelType * | kernel () |
get the kernel function More... | |
const KernelType * | kernel () const |
get the kernel function More... | |
void | setKernel (KernelType *kernel) |
set the kernel function More... | |
bool | isUnconstrained () const |
double | C () const |
return the value of the regularization parameter More... | |
void | setC (double value) |
set the value of the regularization parameter (must be positive) More... | |
bool | trainOffset () const |
check whether the model to be trained should include an offset term More... | |
RealVector | parameterVector () const |
Returns the vector of hyper-parameters. More... | |
void | setParameterVector (RealVector const &newParameters) |
Sets the vector of hyper-parameters. More... | |
size_t | numberOfParameters () const |
Returns the number of hyper-parameters. More... | |
Public Member Functions inherited from shark::AbstractTrainer< KernelClassifier< InputType > > | |
virtual void | train (ModelType &model, DatasetType const &dataset)=0 |
Core of the Trainer interface. More... | |
Public Member Functions inherited from shark::INameable | |
virtual | ~INameable () |
Public Member Functions inherited from shark::ISerializable | |
virtual | ~ISerializable () |
Virtual d'tor. More... | |
virtual void | read (InArchive &archive) |
Read the component from the supplied archive. More... | |
virtual void | write (OutArchive &archive) const |
Write the component to the supplied archive. More... | |
void | load (InArchive &archive, unsigned int version) |
Versioned loading of components, calls read(...). More... | |
void | save (OutArchive &archive, unsigned int version) const |
Versioned storing of components, calls write(...). More... | |
BOOST_SERIALIZATION_SPLIT_MEMBER () | |
Public Member Functions inherited from shark::IParameterizable<> | |
virtual | ~IParameterizable () |
Protected Attributes | |
KernelType * | m_kernel |
pointer to kernel function More... | |
const LossType * | m_loss |
pointer to loss function More... | |
double | m_C |
regularization parameter More... | |
bool | m_offset |
should the resulting model have an offset term? More... | |
bool | m_unconstrained |
should C be stored as log(C) as a parameter? More... | |
std::size_t | m_budgetSize |
AbstractBudgetMaintenanceStrategy< InputType > * | m_budgetMaintenanceStrategy |
std::size_t | m_epochs |
number of training epochs (sweeps over the data), or 0 for default = max(10, C) More... | |
std::size_t | m_preInitializationMethod |
double | m_minMargin |
Budgeted stochastic gradient descent training for kernel-based models.
Definition at line 97 of file KernelBudgetedSGDTrainer.h.
typedef KernelClassifier<InputType> shark::KernelBudgetedSGDTrainer< InputType, CacheType >::ClassifierType |
Definition at line 102 of file KernelBudgetedSGDTrainer.h.
typedef ConstProxyReference<typename Batch<InputType>::type const>::type shark::KernelBudgetedSGDTrainer< InputType, CacheType >::ConstBatchInputReference |
Definition at line 105 of file KernelBudgetedSGDTrainer.h.
typedef LabeledData<InputType, unsigned int>::element_type shark::KernelBudgetedSGDTrainer< InputType, CacheType >::ElementType |
Definition at line 107 of file KernelBudgetedSGDTrainer.h.
typedef KernelMatrix<InputType, QpFloatType> shark::KernelBudgetedSGDTrainer< InputType, CacheType >::KernelMatrixType |
Definition at line 109 of file KernelBudgetedSGDTrainer.h.
typedef AbstractKernelFunction<InputType> shark::KernelBudgetedSGDTrainer< InputType, CacheType >::KernelType |
Definition at line 101 of file KernelBudgetedSGDTrainer.h.
typedef AbstractLoss<unsigned int, RealVector> shark::KernelBudgetedSGDTrainer< InputType, CacheType >::LossType |
Definition at line 104 of file KernelBudgetedSGDTrainer.h.
typedef KernelExpansion<InputType> shark::KernelBudgetedSGDTrainer< InputType, CacheType >::ModelType |
Definition at line 103 of file KernelBudgetedSGDTrainer.h.
typedef PartlyPrecomputedMatrix< KernelMatrixType > shark::KernelBudgetedSGDTrainer< InputType, CacheType >::PartlyPrecomputedMatrixType |
Definition at line 110 of file KernelBudgetedSGDTrainer.h.
typedef CacheType shark::KernelBudgetedSGDTrainer< InputType, CacheType >::QpFloatType |
Definition at line 106 of file KernelBudgetedSGDTrainer.h.
enum shark::KernelBudgetedSGDTrainer::preInitializationMethod |
preinitialization methods
Enumerator | |
---|---|
NONE | |
RANDOM |
Definition at line 115 of file KernelBudgetedSGDTrainer.h.
|
inline |
Constructor Note that there is no cache size involved, as merging vectors will always create new ones, which makes caching roughly obsolete.
[in] | kernel | kernel function to use for training and prediction |
[in] | loss | (sub-)differentiable loss function |
[in] | C | regularization parameter - always the 'true' value of C, even when unconstrained is set |
[in] | offset | whether to train with offset/bias parameter or not |
[in] | unconstrained | when a C-value is given via setParameter, should it be piped through the exp-function before using it in the solver? |
[in] | budgetSize | size of the budget/model that the final solution will have. Note that it might be smaller though. |
[in] | budgetMaintenanceStrategy | object that contains the logic for maintaining the budget size. |
[in] | epochs | number of epochs the SGD solver should run. if zero is given, the size will be the max of 10*datasetsize or C*datasetsize |
[in] | preInitializationMethod | the method to preinitialize the budget. |
[in] | minMargin | the margin every vector has to obey. Usually this is 1. |
Definition at line 134 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_budgetMaintenanceStrategy, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_loss, and SHARK_RUNTIME_CHECK.
|
inline |
return pointer to the budget maintenance strategy
Definition at line 185 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_budgetMaintenanceStrategy.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setBudgetMaintenanceStrategy().
|
inline |
get budget size
Definition at line 167 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_budgetSize.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setBudgetSize().
|
inline |
return the value of the regularization parameter
Definition at line 437 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_C.
|
inline |
Return the number of training epochs. A value of 0 indicates that the default of max(10, C) should be used.
Definition at line 400 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_epochs.
|
inline |
check whether the parameter C is represented as log(C), thus, in a form suitable for unconstrained optimization, in the parameter vector
Definition at line 431 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_unconstrained.
|
inline |
get the kernel function
Definition at line 413 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setKernel().
|
inline |
get the kernel function
Definition at line 418 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel.
|
inline |
return min margin
Definition at line 203 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_minMargin.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setMinMargin().
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 219 of file KernelBudgetedSGDTrainer.h.
|
inlinevirtual |
Returns the number of hyper-parameters.
Reimplemented from shark::IParameterizable<>.
Definition at line 474 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel, and shark::IParameterizable< VectorType >::numberOfParameters().
|
inlinevirtual |
Returns the vector of hyper-parameters.
Reimplemented from shark::IParameterizable<>.
Definition at line 456 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_C, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_unconstrained, and shark::IParameterizable< VectorType >::parameterVector().
|
inline |
set budget maintenance strategy
[in] | budgetMaintenanceStrategy | set strategy to given object. |
Definition at line 194 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::budgetMaintenanceStrategy(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_budgetMaintenanceStrategy.
|
inline |
set budget size
[in] | budgetSize | size of budget. |
Definition at line 176 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::budgetSize(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_budgetSize.
|
inline |
set the value of the regularization parameter (must be positive)
Definition at line 443 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_C, and RANGE_CHECK.
|
inline |
Set the number of training epochs. A value of 0 indicates that the default of max(10, C) should be used.
Definition at line 407 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_epochs.
|
inline |
set the kernel function
Definition at line 423 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::kernel(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel.
|
inline |
set min margin
[in] | minMargin | new min margin. |
Definition at line 212 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_minMargin, and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::minMargin().
|
inlinevirtual |
Sets the vector of hyper-parameters.
Reimplemented from shark::IParameterizable<>.
Definition at line 464 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel, shark::IParameterizable< VectorType >::numberOfParameters(), shark::IParameterizable< VectorType >::setParameterVector(), and SHARK_ASSERT.
|
inline |
Train routine.
[in] | classifier | classifier object for the final solution. |
[in] | dataset | dataset to work with. |
Definition at line 229 of file KernelBudgetedSGDTrainer.h.
References shark::KernelExpansion< InputType >::alpha(), shark::createBatch(), shark::Classifier< Model >::decisionFunction(), shark::random::discrete(), shark::LabeledData< InputT, LabelT >::element(), shark::Data< Type >::elements(), shark::random::globalRng, shark::LabeledData< InputT, LabelT >::labels(), lambda, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_budgetSize, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_C, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_epochs, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_kernel, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_offset, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_preInitializationMethod, shark::KernelBudgetedSGDTrainer< InputType, CacheType >::NONE, shark::numberOfClasses(), shark::LabeledData< InputT, LabelT >::numberOfElements(), and shark::KernelExpansion< InputType >::setStructure().
|
inline |
check whether the model to be trained should include an offset term
Definition at line 450 of file KernelBudgetedSGDTrainer.h.
References shark::KernelBudgetedSGDTrainer< InputType, CacheType >::m_offset.
|
protected |
Definition at line 490 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::budgetMaintenanceStrategy(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::KernelBudgetedSGDTrainer(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setBudgetMaintenanceStrategy().
|
protected |
|
protected |
regularization parameter
Definition at line 482 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::C(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::parameterVector(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setC(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::train().
|
protected |
number of training epochs (sweeps over the data), or 0 for default = max(10, C)
Definition at line 492 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::epochs(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setEpochs(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::train().
|
protected |
pointer to kernel function
Definition at line 480 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::kernel(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::KernelBudgetedSGDTrainer(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::numberOfParameters(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::parameterVector(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setKernel(), shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setParameterVector(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::train().
|
protected |
pointer to loss function
Definition at line 481 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::KernelBudgetedSGDTrainer().
|
protected |
Definition at line 498 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::minMargin(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::setMinMargin().
|
protected |
should the resulting model have an offset term?
Definition at line 483 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::train(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::trainOffset().
|
protected |
Definition at line 495 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::train().
|
protected |
should C be stored as log(C) as a parameter?
Definition at line 484 of file KernelBudgetedSGDTrainer.h.
Referenced by shark::KernelBudgetedSGDTrainer< InputType, CacheType >::isUnconstrained(), and shark::KernelBudgetedSGDTrainer< InputType, CacheType >::parameterVector().