LASSO Regression. More...

#include <shark/Algorithms/Trainers/LassoRegression.h>

Inheritance diagram for shark::LassoRegression< InputVectorType >:

Public Types
typedef LinearModel< InputVectorType >	ModelType

typedef LabeledData< InputVectorType, RealVector >	DataType

Public Types inherited from shark::AbstractTrainer< LinearModel< InputVectorType > >
typedef LinearModel< InputVectorType >	ModelType

typedef ModelType::InputType	InputType

typedef typename LinearModel< InputVectorType > ::OutputType	LabelType

typedef LabeledData< InputType, LabelType >	DatasetType

Public Types inherited from shark::IParameterizable<>
typedef RealVector	ParameterVectorType

Public Member Functions
	LassoRegression (double lambda, double accuracy=0.01)
	Constructor. More...

std::string	name () const
	From INameable: return the class name. More...

double	lambda () const
	Return the current setting of the regularization parameter. More...

void	setLambda (double lambda)
	Set the regularization parameter. More...

double	accuracy () const
	Return the current setting of the accuracy (maximal gradient component of the optimization problem). More...

void	setAccuracy (double accuracy)
	Set the accuracy (maximal gradient component of the optimization problem). More...

RealVector	parameterVector () const
	Get the regularization parameter lambda through the IParameterizable interface. More...

void	setParameterVector (const RealVector &param)
	Set the regularization parameter lambda through the IParameterizable interface. More...

size_t	numberOfParameters () const
	Return the number of parameters (one in this case). More...

void	train (ModelType &model, DataType const &dataset)
	Train a linear model with LASSO regression. More...

Public Member Functions inherited from shark::AbstractTrainer< LinearModel< InputVectorType > >
virtual void	train (ModelType &model, DatasetType const &dataset)=0
	Core of the Trainer interface. More...

Public Member Functions inherited from shark::INameable
virtual	~INameable ()

Public Member Functions inherited from shark::ISerializable
virtual	~ISerializable ()
	Virtual d'tor. More...

virtual void	read (InArchive &archive)
	Read the component from the supplied archive. More...

virtual void	write (OutArchive &archive) const
	Write the component to the supplied archive. More...

void	load (InArchive &archive, unsigned int version)
	Versioned loading of components, calls read(...). More...

void	save (OutArchive &archive, unsigned int version) const
	Versioned storing of components, calls write(...). More...

	BOOST_SERIALIZATION_SPLIT_MEMBER ()

Public Member Functions inherited from shark::IParameterizable<>
virtual	~IParameterizable ()

Protected Attributes
double	m_lambda
	regularization parameter More...

double	m_accuracy
	gradient accuracy More...

Detailed Description

template<class InputVectorType = RealVector>
class shark::LassoRegression< InputVectorType >

LASSO Regression.

LASSO Regression extracts a sparse vector of regression coefficients. The original method amounts to L1-constrained least squares regression, while this implementation uses an L1 penalty instead of a constraint (which is equivalent).

For data vectors \( x_i \) with real-valued labels \( y_i \) the trainer solves the problem \( \min_w \quad \frac{1}{2} \sum_i (w^T x_i - y_i)^2 + \lambda \|w\|_1 \). The target accuracy of the solution is measured in terms of the smallest component of the gradient of the objective function.

The trainer has one template parameter, namely the type of the input vectors \( x_i \). These need to be vector valued, typically either RealVector of CompressedRealVector. The resulting weight vector w is represented by a LinearModel object. Currently model outputs and labels are restricted to a single dimension.

Definition at line 69 of file LassoRegression.h.

Member Typedef Documentation

◆ DataType

template<class InputVectorType = RealVector>

typedef LabeledData<InputVectorType, RealVector> shark::LassoRegression< InputVectorType >::DataType

Definition at line 73 of file LassoRegression.h.

◆ ModelType

template<class InputVectorType = RealVector>

typedef LinearModel<InputVectorType> shark::LassoRegression< InputVectorType >::ModelType

Definition at line 72 of file LassoRegression.h.

Constructor & Destructor Documentation

◆ LassoRegression()

template<class InputVectorType = RealVector>

shark::LassoRegression< InputVectorType >::LassoRegression	(	double	lambda,
		double	accuracy = `0.01`
	)

inline

Constructor.

Parameters

lambda	value of the regularization parameter (see class description)
accuracy	stopping criterion for the iterative solver, maximal gradient component of the objective function (see class description)

Definition at line 79 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_accuracy, shark::LassoRegression< InputVectorType >::m_lambda, and RANGE_CHECK.

Member Function Documentation

◆ accuracy()

template<class InputVectorType = RealVector>

double shark::LassoRegression< InputVectorType >::accuracy ( ) const

inline

Return the current setting of the accuracy (maximal gradient component of the optimization problem).

Definition at line 106 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_accuracy.

Referenced by shark::LassoRegression< InputVectorType >::setAccuracy().

◆ lambda()

template<class InputVectorType = RealVector>

double shark::LassoRegression< InputVectorType >::lambda ( ) const

inline

Return the current setting of the regularization parameter.

Definition at line 93 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_lambda.

Referenced by shark::LassoRegression< InputVectorType >::setLambda().

◆ name()

template<class InputVectorType = RealVector>

std::string shark::LassoRegression< InputVectorType >::name ( ) const

inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 88 of file LassoRegression.h.

◆ numberOfParameters()

template<class InputVectorType = RealVector>

size_t shark::LassoRegression< InputVectorType >::numberOfParameters ( ) const

inlinevirtual

Return the number of parameters (one in this case).

Reimplemented from shark::IParameterizable<>.

Definition at line 133 of file LassoRegression.h.

◆ parameterVector()

template<class InputVectorType = RealVector>

RealVector shark::LassoRegression< InputVectorType >::parameterVector ( ) const

inlinevirtual

Get the regularization parameter lambda through the IParameterizable interface.

Reimplemented from shark::IParameterizable<>.

Definition at line 119 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_lambda.

◆ setAccuracy()

template<class InputVectorType = RealVector>

void shark::LassoRegression< InputVectorType >::setAccuracy ( double accuracy )

inline

Set the accuracy (maximal gradient component of the optimization problem).

Definition at line 112 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::accuracy(), shark::LassoRegression< InputVectorType >::m_accuracy, and RANGE_CHECK.

◆ setLambda()

template<class InputVectorType = RealVector>

void shark::LassoRegression< InputVectorType >::setLambda ( double lambda )

inline

Set the regularization parameter.

Definition at line 99 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::lambda(), shark::LassoRegression< InputVectorType >::m_lambda, and RANGE_CHECK.

◆ setParameterVector()

template<class InputVectorType = RealVector>

void shark::LassoRegression< InputVectorType >::setParameterVector ( const RealVector & param )

inlinevirtual

Set the regularization parameter lambda through the IParameterizable interface.

Reimplemented from shark::IParameterizable<>.

Definition at line 125 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_lambda, RANGE_CHECK, and SIZE_CHECK.

◆ train()

template<class InputVectorType = RealVector>

void shark::LassoRegression< InputVectorType >::train	(	ModelType &	model,
		DataType const &	dataset
	)

inline

Train a linear model with LASSO regression.

Definition at line 139 of file LassoRegression.h.

References shark::inputDimension(), and w.

Referenced by main().

Member Data Documentation

◆ m_accuracy

template<class InputVectorType = RealVector>

double shark::LassoRegression< InputVectorType >::m_accuracy

protected

gradient accuracy

Definition at line 334 of file LassoRegression.h.

Referenced by shark::LassoRegression< InputVectorType >::accuracy(), shark::LassoRegression< InputVectorType >::LassoRegression(), and shark::LassoRegression< InputVectorType >::setAccuracy().

◆ m_lambda

template<class InputVectorType = RealVector>

double shark::LassoRegression< InputVectorType >::m_lambda

protected

regularization parameter

Definition at line 333 of file LassoRegression.h.

Referenced by shark::LassoRegression< InputVectorType >::lambda(), shark::LassoRegression< InputVectorType >::LassoRegression(), shark::LassoRegression< InputVectorType >::parameterVector(), shark::LassoRegression< InputVectorType >::setLambda(), and shark::LassoRegression< InputVectorType >::setParameterVector().

The documentation for this class was generated from the following file:

include/shark/Algorithms/Trainers/LassoRegression.h

Public Types

Public Member Functions

Protected Attributes

Detailed Description

template<class InputVectorType = RealVector> class shark::LassoRegression< InputVectorType >

Member Typedef Documentation

◆ DataType

◆ ModelType

Constructor & Destructor Documentation

◆ LassoRegression()

Member Function Documentation

◆ accuracy()

◆ lambda()

◆ name()

◆ numberOfParameters()

◆ parameterVector()

◆ setAccuracy()

◆ setLambda()

◆ setParameterVector()

◆ train()

Member Data Documentation

◆ m_accuracy

◆ m_lambda

template<class InputVectorType = RealVector>
class shark::LassoRegression< InputVectorType >