Error Function for Autoencoders and TiedAutoencoders which should be trained with sparse activation of the hidden neurons. More...

#include <shark/ObjectiveFunctions/SparseAutoencoderError.h>

Inheritance diagram for shark::SparseAutoencoderError:

Public Types
typedef LabeledData< RealVector, RealVector >	DatasetType

Public Types inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
enum	Feature { HAS_VALUE = 1, HAS_FIRST_DERIVATIVE = 2, HAS_SECOND_DERIVATIVE = 4, CAN_PROPOSE_STARTING_POINT = 8, IS_CONSTRAINED_FEATURE = 16, HAS_CONSTRAINT_HANDLER = 32, CAN_PROVIDE_CLOSEST_FEASIBLE = 64, IS_THREAD_SAFE = 128, IS_NOISY = 256 }
	List of features that are supported by an implementation. More...

typedef PointType	SearchPointType

typedef ResultT	ResultType

typedef boost::mpl::if_< std::is_arithmetic< ResultT >, SearchPointType, RealMatrix >::type	FirstOrderDerivative

typedef TypedFlags< Feature >	Features
	This statement declares the member m_features. See Core/Flags.h for details. More...

typedef TypedFeatureNotAvailableException< Feature >	FeatureNotAvailableException

Public Member Functions
template<class HiddenNeuron , class OutputNeuron >
	SparseAutoencoderError (DatasetType const &dataset, Autoencoder< HiddenNeuron, OutputNeuron > model, AbstractLoss< RealVector, RealVector > loss, double rho=0.5, double beta=0.1)

template<class HiddenNeuron , class OutputNeuron >
	SparseAutoencoderError (DatasetType const &dataset, TiedAutoencoder< HiddenNeuron, OutputNeuron > model, AbstractLoss< RealVector, RealVector > loss, double rho=0.5, double beta=0.1)

SparseAutoencoderError &	operator= (SparseAutoencoderError const &op)

std::string	name () const
	From INameable: return the class name. More...

std::size_t	numberOfVariables () const
	Accesses the number of variables. More...

SearchPointType	proposeStartingPoint () const
	Proposes a starting point in the feasible search space of the function. More...

void	setRegularizer (double factor, SingleObjectiveFunction *regularizer)

double	eval (RealVector const &input) const

ResultType	evalDerivative (SearchPointType const &input, FirstOrderDerivative &derivative) const
	Evaluates the objective function and calculates its gradient. More...

Public Member Functions inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
const Features &	features () const

virtual void	updateFeatures ()

bool	hasValue () const
	returns whether this function can calculate it's function value More...

bool	hasFirstDerivative () const
	returns whether this function can calculate the first derivative More...

bool	hasSecondDerivative () const
	returns whether this function can calculate the second derivative More...

bool	canProposeStartingPoint () const
	returns whether this function can propose a starting point. More...

bool	isConstrained () const
	returns whether this function can return More...

bool	hasConstraintHandler () const
	returns whether this function can return More...

bool	canProvideClosestFeasible () const
	Returns whether this function can calculate thee closest feasible to an infeasible point. More...

bool	isThreadSafe () const
	Returns true, when the function can be usd in parallel threads. More...

bool	isNoisy () const
	Returns true, when the function can be usd in parallel threads. More...

	AbstractObjectiveFunction ()
	Default ctor. More...

virtual	~AbstractObjectiveFunction ()
	Virtual destructor. More...

virtual void	init ()

void	setRng (random::rng_type *rng)
	Sets the Rng used by the objective function. More...

virtual bool	hasScalableDimensionality () const

virtual void	setNumberOfVariables (std::size_t numberOfVariables)
	Adjusts the number of variables if the function is scalable. More...

virtual std::size_t	numberOfObjectives () const

virtual bool	hasScalableObjectives () const

virtual void	setNumberOfObjectives (std::size_t numberOfObjectives)
	Adjusts the number of objectives if the function is scalable. More...

std::size_t	evaluationCounter () const
	Accesses the evaluation counter of the function. More...

AbstractConstraintHandler< SearchPointType > const &	getConstraintHandler () const
	Returns the constraint handler of the function if it has one. More...

virtual bool	isFeasible (const SearchPointType &input) const
	Tests whether a point in SearchSpace is feasible, e.g., whether the constraints are fulfilled. More...

virtual void	closestFeasible (SearchPointType &input) const
	If supported, the supplied point is repaired such that it satisfies all of the function's constraints. More...

virtual ResultType	eval (SearchPointType const &input) const
	Evaluates the objective function for the supplied argument. More...

ResultType	operator() (SearchPointType const &input) const
	Evaluates the function. Useful together with STL-Algorithms like std::transform. More...

virtual ResultType	evalDerivative (SearchPointType const &input, SecondOrderDerivative &derivative) const
	Evaluates the objective function and calculates its gradient. More...

Public Member Functions inherited from shark::INameable
virtual	~INameable ()

Friends
void	swap (SparseAutoencoderError &op1, SparseAutoencoderError &op2)

Additional Inherited Members
Protected Member Functions inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
void	announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler)
	helper function which is called to announce the presence of an constraint handler. More...

Protected Attributes inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
Features	m_features

std::size_t	m_evaluationCounter
	Evaluation counter, default value: 0. More...

AbstractConstraintHandler< SearchPointType > const *	m_constraintHandler

random::rng_type *	mep_rng

Detailed Description

Error Function for Autoencoders and TiedAutoencoders which should be trained with sparse activation of the hidden neurons.

This error function optimizes a Network with respect to some loss function similar to the standard ErrorFunction. Additionally another penalty term is added which enforces a sparse activation pattern of the hidden neurons. Given a target mean activation \( \rho \) the mean activation of hidden neuron j over the whole dataset \( \rho_j\) is interpreted as the activation propability and penalized using the KL-divergence: \( KL(\rho||\rho_j) = \rho log(\frac{\rho}{\rho_j})+(1-\rho) log(\frac{1-\rho}{1-\rho_j}) \)

This Error Function has two meta-parameters: rho governs the desired mean activation and beta the strength of regularization. Another regularizer can be added using setRegularizer as in typical ErrorFunctions.

Definition at line 57 of file SparseAutoencoderError.h.

Member Typedef Documentation

◆ DatasetType

typedef LabeledData<RealVector, RealVector> shark::SparseAutoencoderError::DatasetType

Definition at line 60 of file SparseAutoencoderError.h.

Constructor & Destructor Documentation

◆ SparseAutoencoderError() [1/2]

template<class HiddenNeuron , class OutputNeuron >

shark::SparseAutoencoderError::SparseAutoencoderError	(	DatasetType const &	dataset,
		Autoencoder< HiddenNeuron, OutputNeuron > *	model,
		AbstractLoss< RealVector, RealVector > *	loss,
		double	rho = `0.5`,
		double	beta = `0.1`
	)

◆ SparseAutoencoderError() [2/2]

template<class HiddenNeuron , class OutputNeuron >

shark::SparseAutoencoderError::SparseAutoencoderError	(	DatasetType const &	dataset,
		TiedAutoencoder< HiddenNeuron, OutputNeuron > *	model,
		AbstractLoss< RealVector, RealVector > *	loss,
		double	rho = `0.5`,
		double	beta = `0.1`
	)

Member Function Documentation

◆ eval()

double shark::SparseAutoencoderError::eval ( RealVector const & input ) const

inline

Definition at line 98 of file SparseAutoencoderError.h.

References shark::AbstractObjectiveFunction< PointType, ResultT >::eval(), and shark::AbstractObjectiveFunction< PointType, ResultT >::m_evaluationCounter.

◆ evalDerivative()

ResultType shark::SparseAutoencoderError::evalDerivative	(	SearchPointType const &	input,
		FirstOrderDerivative &	derivative
	)		const

inlinevirtual

Evaluates the objective function and calculates its gradient.

Parameters

[in]	input	The argument to eval the function for.
[out]	derivative	The derivate is placed here.

Returns: The result of evaluating the function for the supplied argument.

Exceptions

FeatureNotAvailableException in the default implementation and if a function does not support this feature.

Reimplemented from shark::AbstractObjectiveFunction< PointType, ResultT >.

Definition at line 105 of file SparseAutoencoderError.h.

References shark::AbstractObjectiveFunction< PointType, ResultT >::evalDerivative(), shark::AbstractObjectiveFunction< PointType, ResultT >::m_evaluationCounter, and remora::noalias().

◆ name()

std::string shark::SparseAutoencoderError::name ( ) const

inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 82 of file SparseAutoencoderError.h.

◆ numberOfVariables()

std::size_t shark::SparseAutoencoderError::numberOfVariables ( ) const

inlinevirtual

Accesses the number of variables.

Implements shark::AbstractObjectiveFunction< PointType, ResultT >.

Definition at line 85 of file SparseAutoencoderError.h.

Referenced by main().

◆ operator=()

SparseAutoencoderError& shark::SparseAutoencoderError::operator= ( SparseAutoencoderError const & op )

inline

Definition at line 76 of file SparseAutoencoderError.h.

◆ proposeStartingPoint()

SearchPointType shark::SparseAutoencoderError::proposeStartingPoint ( ) const

inlinevirtual

Proposes a starting point in the feasible search space of the function.

Returns: The generated starting point.

Exceptions

FeatureNotAvailableException in the default implementation and if a function does not support this feature.

Reimplemented from shark::AbstractObjectiveFunction< PointType, ResultT >.

Definition at line 89 of file SparseAutoencoderError.h.

◆ setRegularizer()

void shark::SparseAutoencoderError::setRegularizer	(	double	factor,
		SingleObjectiveFunction *	regularizer
	)

inline

Definition at line 93 of file SparseAutoencoderError.h.

Referenced by main().

Friends And Related Function Documentation

◆ swap

void swap	(	SparseAutoencoderError &	op1,
		SparseAutoencoderError &	op2
	)

friend

Definition at line 116 of file SparseAutoencoderError.h.

The documentation for this class was generated from the following file:

include/shark/ObjectiveFunctions/SparseAutoencoderError.h

Public Types

Public Member Functions

Friends

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ DatasetType

Constructor & Destructor Documentation

◆ SparseAutoencoderError() [1/2]

◆ SparseAutoencoderError() [2/2]

Member Function Documentation

◆ eval()

◆ evalDerivative()

◆ name()

◆ numberOfVariables()

◆ operator=()

◆ proposeStartingPoint()

◆ setRegularizer()

Friends And Related Function Documentation

◆ swap