shark::LinearSAGTrainer< InputType, LabelType > Class Template Reference

Stochastic Average Gradient Method for training of linear models,. More...

#include <shark/Algorithms/Trainers/LinearSAGTrainer.h>

+ Inheritance diagram for shark::LinearSAGTrainer< InputType, LabelType >:

Public Types

typedef Base::ModelType ModelType
 
typedef Base::WeightedDatasetType WeightedDatasetType
 
typedef detail::LinearSAGTrainerBase< InputType, LabelType >::LossType LossType
 
- Public Types inherited from shark::AbstractWeightedTrainer< Model, LabelTypeT >
typedef base_type::ModelType ModelType
 
typedef base_type::InputType InputType
 
typedef base_type::LabelType LabelType
 
typedef base_type::DatasetType DatasetType
 
typedef WeightedLabeledData< InputType, LabelTypeWeightedDatasetType
 
- Public Types inherited from shark::AbstractTrainer< Model, LabelTypeT >
typedef Model ModelType
 
typedef ModelType::InputType InputType
 
typedef LabelTypeT LabelType
 
typedef LabeledData< InputType, LabelTypeDatasetType
 
- Public Types inherited from shark::IParameterizable<>
typedef RealVector ParameterVectorType
 

Public Member Functions

 LinearSAGTrainer (LossType const *loss, double lambda=0, bool offset=true)
 Constructor. More...
 
std::string name () const
 From INameable: return the class name. More...
 
void train (ModelType &model, WeightedDatasetType const &dataset)
 Executes the algorithm and trains a model on the given weighted data. More...
 
std::size_t epochs () const
 
void setEpochs (std::size_t value)
 Set the number of training epochs. A value of 0 indicates that the default of max(10, dimensionOfData) should be used. More...
 
double lambda () const
 Return the value of the regularization parameter lambda. More...
 
void setLambda (double lambda)
 Set the value of the regularization parameter lambda. More...
 
bool trainOffset () const
 Check whether the model to be trained should include an offset term. More...
 
void setTrainOffset (bool offset)
 Sets whether the model to be trained should include an offset term. More...
 
RealVector parameterVector () const
 Returns the vector of hyper-parameters(same as lambda) More...
 
void setParameterVector (RealVector const &newParameters)
 Sets the vector of hyper-parameters(same as lambda) More...
 
size_t numberOfParameters () const
 Returns the number of hyper-parameters. More...
 
- Public Member Functions inherited from shark::AbstractWeightedTrainer< Model, LabelTypeT >
virtual void train (ModelType &model, DatasetType const &dataset)
 Executes the algorithm and trains a model on the given unweighted data. More...
 
- Public Member Functions inherited from shark::INameable
virtual ~INameable ()
 
- Public Member Functions inherited from shark::ISerializable
virtual ~ISerializable ()
 Virtual d'tor. More...
 
virtual void read (InArchive &archive)
 Read the component from the supplied archive. More...
 
virtual void write (OutArchive &archive) const
 Write the component to the supplied archive. More...
 
void load (InArchive &archive, unsigned int version)
 Versioned loading of components, calls read(...). More...
 
void save (OutArchive &archive, unsigned int version) const
 Versioned storing of components, calls write(...). More...
 
 BOOST_SERIALIZATION_SPLIT_MEMBER ()
 
- Public Member Functions inherited from shark::IParameterizable<>
virtual ~IParameterizable ()
 

Detailed Description

template<class InputType, class LabelType>
class shark::LinearSAGTrainer< InputType, LabelType >

Stochastic Average Gradient Method for training of linear models,.

Given a differentiable loss function L(f, y) and a model f_j(x)= w_j^Tx+b this trainer solves the regularized risk minimization problem

\[ \min \frac{1}{2} \sum_j \frac{\lambda}{2}\|w_j\|^2 + \frac 1 {\ell} \sum_i L(y_i, f(x_i)), \]

where i runs over training data, j over the model outputs, and lambda > 0 is the regularization parameter.

The algorithm uses averaging of the algorithm to obtain a good estimate of the gradient. Averaging is performed by summing over the last gradient value obtained for each data point. At the beginning this estimate is far off as old gradient values are outdated, but as the algorithm converges, this gives linear convergence on strictly convex functions and O(1/T) convergence on not-strictly convex functions.

The algorithm supports classification and regresseion, dense and sparse inputs and weighted and unweighted datasets Reference: Schmidt, Mark, Nicolas Le Roux, and Francis Bach. "Minimizing finite sums with the stochastic average gradient." arXiv preprint arXiv:1309.2388 (2013).

Definition at line 90 of file LinearSAGTrainer.h.

Member Typedef Documentation

◆ LossType

template<class InputType, class LabelType>
typedef detail::LinearSAGTrainerBase<InputType,LabelType>::LossType shark::LinearSAGTrainer< InputType, LabelType >::LossType

Definition at line 97 of file LinearSAGTrainer.h.

◆ ModelType

template<class InputType, class LabelType>
typedef Base::ModelType shark::LinearSAGTrainer< InputType, LabelType >::ModelType

Definition at line 95 of file LinearSAGTrainer.h.

◆ WeightedDatasetType

template<class InputType, class LabelType>
typedef Base::WeightedDatasetType shark::LinearSAGTrainer< InputType, LabelType >::WeightedDatasetType

Definition at line 96 of file LinearSAGTrainer.h.

Constructor & Destructor Documentation

◆ LinearSAGTrainer()

template<class InputType, class LabelType>
shark::LinearSAGTrainer< InputType, LabelType >::LinearSAGTrainer ( LossType const *  loss,
double  lambda = 0,
bool  offset = true 
)
inline

Constructor.

Parameters
loss(sub-)differentiable loss function
lambdaregularization parameter fort wo-norm regularization, 0 by default
offsetwhether to train with offset/bias parameter or not, default is true

Definition at line 105 of file LinearSAGTrainer.h.

Member Function Documentation

◆ epochs()

template<class InputType, class LabelType>
std::size_t shark::LinearSAGTrainer< InputType, LabelType >::epochs ( ) const
inline

the number of training epochs. A value of 0 indicates that the default of max(10, dimensionOfData) should be used.

Definition at line 124 of file LinearSAGTrainer.h.

◆ lambda()

template<class InputType, class LabelType>
double shark::LinearSAGTrainer< InputType, LabelType >::lambda ( ) const
inline

Return the value of the regularization parameter lambda.

Definition at line 134 of file LinearSAGTrainer.h.

◆ name()

template<class InputType, class LabelType>
std::string shark::LinearSAGTrainer< InputType, LabelType >::name ( ) const
inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 113 of file LinearSAGTrainer.h.

◆ numberOfParameters()

◆ parameterVector()

template<class InputType, class LabelType>
RealVector shark::LinearSAGTrainer< InputType, LabelType >::parameterVector ( ) const
inlinevirtual

Returns the vector of hyper-parameters(same as lambda)

Reimplemented from shark::IParameterizable<>.

Definition at line 150 of file LinearSAGTrainer.h.

◆ setEpochs()

template<class InputType, class LabelType>
void shark::LinearSAGTrainer< InputType, LabelType >::setEpochs ( std::size_t  value)
inline

Set the number of training epochs. A value of 0 indicates that the default of max(10, dimensionOfData) should be used.

Definition at line 129 of file LinearSAGTrainer.h.

Referenced by run().

◆ setLambda()

template<class InputType, class LabelType>
void shark::LinearSAGTrainer< InputType, LabelType >::setLambda ( double  lambda)
inline

Set the value of the regularization parameter lambda.

Definition at line 138 of file LinearSAGTrainer.h.

References lambda.

◆ setParameterVector()

template<class InputType, class LabelType>
void shark::LinearSAGTrainer< InputType, LabelType >::setParameterVector ( RealVector const &  newParameters)
inlinevirtual

Sets the vector of hyper-parameters(same as lambda)

Reimplemented from shark::IParameterizable<>.

Definition at line 156 of file LinearSAGTrainer.h.

References SIZE_CHECK.

◆ setTrainOffset()

template<class InputType, class LabelType>
void shark::LinearSAGTrainer< InputType, LabelType >::setTrainOffset ( bool  offset)
inline

Sets whether the model to be trained should include an offset term.

Definition at line 146 of file LinearSAGTrainer.h.

◆ train()

template<class InputType, class LabelType>
void shark::LinearSAGTrainer< InputType, LabelType >::train ( ModelType model,
WeightedDatasetType const &  dataset 
)
inlinevirtual

Executes the algorithm and trains a model on the given weighted data.

Implements shark::AbstractWeightedTrainer< Model, LabelTypeT >.

Definition at line 117 of file LinearSAGTrainer.h.

References shark::random::globalRng.

Referenced by run().

◆ trainOffset()

template<class InputType, class LabelType>
bool shark::LinearSAGTrainer< InputType, LabelType >::trainOffset ( ) const
inline

Check whether the model to be trained should include an offset term.

Definition at line 142 of file LinearSAGTrainer.h.


The documentation for this class was generated from the following file: