Implements k-step Contrastive Divergence described by Hinton et al. (2006). More...

#include <shark/Unsupervised/RBM/GradientApproximations/ContrastiveDivergence.h>

Inheritance diagram for shark::ContrastiveDivergence< Operator >:

Public Types
typedef Operator::RBM	RBM

Public Types inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
enum	Feature { HAS_VALUE = 1, HAS_FIRST_DERIVATIVE = 2, HAS_SECOND_DERIVATIVE = 4, CAN_PROPOSE_STARTING_POINT = 8, IS_CONSTRAINED_FEATURE = 16, HAS_CONSTRAINT_HANDLER = 32, CAN_PROVIDE_CLOSEST_FEASIBLE = 64, IS_THREAD_SAFE = 128, IS_NOISY = 256 }
	List of features that are supported by an implementation. More...

typedef PointType	SearchPointType

typedef ResultT	ResultType

typedef boost::mpl::if_< std::is_arithmetic< ResultT >, SearchPointType, RealMatrix >::type	FirstOrderDerivative

typedef TypedFlags< Feature >	Features
	This statement declares the member m_features. See Core/Flags.h for details. More...

typedef TypedFeatureNotAvailableException< Feature >	FeatureNotAvailableException

Public Member Functions
	ContrastiveDivergence (RBM *rbm)
	The constructor. More...

std::string	name () const
	From INameable: return the class name. More...

void	setData (UnlabeledData< RealVector > const &data)
	Sets the training batch. More...

void	setK (unsigned int k)
	Sets the value of k- the number of steps of the Gibbs Chain. More...

SearchPointType	proposeStartingPoint () const
	Proposes a starting point in the feasible search space of the function. More...

std::size_t	numberOfVariables () const
	Returns the number of variables of the RBM. More...

std::size_t	numBatches () const
	Returns the number of batches of the dataset that are used in every iteration. More...

std::size_t &	numBatches ()
	Returns a reference to the number of batches of the dataset that are used in every iteration. More...

void	setRegularizer (double factor, SingleObjectiveFunction *regularizer)

double	evalDerivative (SearchPointType const &parameter, FirstOrderDerivative &derivative) const
	Gives the CD-k approximation of the log-likelihood gradient. More...

Public Member Functions inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
const Features &	features () const

virtual void	updateFeatures ()

bool	hasValue () const
	returns whether this function can calculate it's function value More...

bool	hasFirstDerivative () const
	returns whether this function can calculate the first derivative More...

bool	hasSecondDerivative () const
	returns whether this function can calculate the second derivative More...

bool	canProposeStartingPoint () const
	returns whether this function can propose a starting point. More...

bool	isConstrained () const
	returns whether this function can return More...

bool	hasConstraintHandler () const
	returns whether this function can return More...

bool	canProvideClosestFeasible () const
	Returns whether this function can calculate thee closest feasible to an infeasible point. More...

bool	isThreadSafe () const
	Returns true, when the function can be usd in parallel threads. More...

bool	isNoisy () const
	Returns true, when the function can be usd in parallel threads. More...

	AbstractObjectiveFunction ()
	Default ctor. More...

virtual	~AbstractObjectiveFunction ()
	Virtual destructor. More...

virtual void	init ()

void	setRng (random::rng_type *rng)
	Sets the Rng used by the objective function. More...

virtual bool	hasScalableDimensionality () const

virtual void	setNumberOfVariables (std::size_t numberOfVariables)
	Adjusts the number of variables if the function is scalable. More...

virtual std::size_t	numberOfObjectives () const

virtual bool	hasScalableObjectives () const

virtual void	setNumberOfObjectives (std::size_t numberOfObjectives)
	Adjusts the number of objectives if the function is scalable. More...

std::size_t	evaluationCounter () const
	Accesses the evaluation counter of the function. More...

AbstractConstraintHandler< SearchPointType > const &	getConstraintHandler () const
	Returns the constraint handler of the function if it has one. More...

virtual bool	isFeasible (const SearchPointType &input) const
	Tests whether a point in SearchSpace is feasible, e.g., whether the constraints are fulfilled. More...

virtual void	closestFeasible (SearchPointType &input) const
	If supported, the supplied point is repaired such that it satisfies all of the function's constraints. More...

virtual ResultType	eval (SearchPointType const &input) const
	Evaluates the objective function for the supplied argument. More...

ResultType	operator() (SearchPointType const &input) const
	Evaluates the function. Useful together with STL-Algorithms like std::transform. More...

virtual ResultType	evalDerivative (SearchPointType const &input, SecondOrderDerivative &derivative) const
	Evaluates the objective function and calculates its gradient. More...

Public Member Functions inherited from shark::INameable
virtual	~INameable ()

Additional Inherited Members
Protected Member Functions inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
void	announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler)
	helper function which is called to announce the presence of an constraint handler. More...

Protected Attributes inherited from shark::AbstractObjectiveFunction< PointType, ResultT >
Features	m_features

std::size_t	m_evaluationCounter
	Evaluation counter, default value: 0. More...

AbstractConstraintHandler< SearchPointType > const *	m_constraintHandler

random::rng_type *	mep_rng

Detailed Description

template<class Operator>
class shark::ContrastiveDivergence< Operator >

Implements k-step Contrastive Divergence described by Hinton et al. (2006).

k-step Contrastive Divergence approximates the gradient by initializing a Gibbs chain with a training example and run it for k steps. The sample gained after k steps than samples is than used to approximate the mean of the RBM distribution in the gradient.

Definition at line 44 of file ContrastiveDivergence.h.

Member Typedef Documentation

◆ RBM

template<class Operator >

typedef Operator::RBM shark::ContrastiveDivergence< Operator >::RBM

Definition at line 46 of file ContrastiveDivergence.h.

Constructor & Destructor Documentation

◆ ContrastiveDivergence()

template<class Operator >

shark::ContrastiveDivergence< Operator >::ContrastiveDivergence ( RBM * rbm )

inline

The constructor.

Parameters

rbm	pointer to the RBM which shell be trained

Definition at line 51 of file ContrastiveDivergence.h.

References shark::AbstractObjectiveFunction< PointType, ResultT >::CAN_PROPOSE_STARTING_POINT, shark::AbstractObjectiveFunction< PointType, ResultT >::HAS_FIRST_DERIVATIVE, shark::AbstractObjectiveFunction< PointType, ResultT >::HAS_VALUE, shark::AbstractObjectiveFunction< PointType, ResultT >::m_features, shark::TypedFlags< Flag >::reset(), and SHARK_ASSERT.

Member Function Documentation

◆ evalDerivative()

template<class Operator >

double shark::ContrastiveDivergence< Operator >::evalDerivative	(	SearchPointType const &	parameter,
		FirstOrderDerivative &	derivative
	)		const

inlinevirtual

Gives the CD-k approximation of the log-likelihood gradient.

Parameters

parameter	the actual parameters of the RBM
derivative	holds later the CD-k approximation of the log-likelihood gradient

Reimplemented from shark::AbstractObjectiveFunction< PointType, ResultT >.

Definition at line 113 of file ContrastiveDivergence.h.

References shark::Data< Type >::batch(), shark::batchEnd(), shark::ContrastiveDivergence< Operator >::numBatches(), shark::Data< Type >::numberOfBatches(), SHARK_NUM_THREADS, SHARK_PARALLEL_FOR, and shark::shuffle().

◆ name()

template<class Operator >

std::string shark::ContrastiveDivergence< Operator >::name ( ) const

inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 62 of file ContrastiveDivergence.h.

◆ numBatches() [1/2]

template<class Operator >

std::size_t shark::ContrastiveDivergence< Operator >::numBatches ( ) const

inline

Returns the number of batches of the dataset that are used in every iteration.

If it is less than all batches, the batches are chosen at random. if it is 0, all batches are used

Definition at line 93 of file ContrastiveDivergence.h.

Referenced by shark::ContrastiveDivergence< Operator >::evalDerivative().

◆ numBatches() [2/2]

template<class Operator >

std::size_t& shark::ContrastiveDivergence< Operator >::numBatches ( )

inline

Returns a reference to the number of batches of the dataset that are used in every iteration.

If it is less than all batches, the batches are chosen at random.if it is 0, all batches are used.

Definition at line 100 of file ContrastiveDivergence.h.

◆ numberOfVariables()

template<class Operator >

std::size_t shark::ContrastiveDivergence< Operator >::numberOfVariables ( ) const

inlinevirtual

Returns the number of variables of the RBM.

Returns: the number of variables of the RBM

Implements shark::AbstractObjectiveFunction< PointType, ResultT >.

Definition at line 86 of file ContrastiveDivergence.h.

◆ proposeStartingPoint()

template<class Operator >

SearchPointType shark::ContrastiveDivergence< Operator >::proposeStartingPoint ( ) const

inlinevirtual

Proposes a starting point in the feasible search space of the function.

Returns: The generated starting point.

Exceptions

FeatureNotAvailableException in the default implementation and if a function does not support this feature.

Reimplemented from shark::AbstractObjectiveFunction< PointType, ResultT >.

Definition at line 79 of file ContrastiveDivergence.h.

◆ setData()

template<class Operator >

void shark::ContrastiveDivergence< Operator >::setData ( UnlabeledData< RealVector > const & data )

inline

Sets the training batch.

Parameters

data	the batch of training data

Definition at line 68 of file ContrastiveDivergence.h.

Referenced by main(), and trainRBM().

◆ setK()

template<class Operator >

void shark::ContrastiveDivergence< Operator >::setK ( unsigned int k )

inline

Sets the value of k- the number of steps of the Gibbs Chain.

Parameters

k	the number of steps

Definition at line 75 of file ContrastiveDivergence.h.

Referenced by main(), and trainRBM().

◆ setRegularizer()

template<class Operator >

void shark::ContrastiveDivergence< Operator >::setRegularizer	(	double	factor,
		SingleObjectiveFunction *	regularizer
	)

inline

Definition at line 104 of file ContrastiveDivergence.h.

Referenced by trainRBM().

The documentation for this class was generated from the following file:

include/shark/Unsupervised/RBM/GradientApproximations/ContrastiveDivergence.h

Public Types

Public Member Functions

Additional Inherited Members

Detailed Description

template<class Operator> class shark::ContrastiveDivergence< Operator >

Member Typedef Documentation

◆ RBM

Constructor & Destructor Documentation

◆ ContrastiveDivergence()

Member Function Documentation

◆ evalDerivative()

◆ name()

◆ numBatches() [1/2]

◆ numBatches() [2/2]

◆ numberOfVariables()

◆ proposeStartingPoint()

◆ setData()

◆ setK()

◆ setRegularizer()

template<class Operator>
class shark::ContrastiveDivergence< Operator >