Layer of binary units taking values in {0,1}. More...

#include <shark/Unsupervised/RBM/Neuronlayers/BinaryLayer.h>

Inheritance diagram for shark::BinaryLayer:

Public Types
typedef BinarySpace	StateSpace
	The state space of this neuron is binary. More...

typedef RealVector	SufficientStatistics
	The sufficient statistics for the Binary Layer store the probability for a neuron to be on. More...

typedef Batch< SufficientStatistics >::type	StatisticsBatch
	Sufficient statistics of a batch of data. More...

Public Types inherited from shark::IParameterizable<>
typedef RealVector	ParameterVectorType

Public Member Functions
const RealVector &	bias () const
	Returns the bias values of the units. More...

RealVector &	bias ()
	Returns the bias values of the units. More...

RealVector const &	baseRate () const
	Returns the base rate of the units. More...

RealVector &	baseRate ()
	Returns the base rate of the units. More...

void	resize (std::size_t newSize)
	Resizes this neuron layer. More...

std::size_t	size () const
	Returns the number of neurons of this layer. More...

template<class Input , class BetaVector >
void	sufficientStatistics (Input const &input, StatisticsBatch &statistics, BetaVector const &beta) const
	Takes the input of the neuron and estimates the expectation of the response of the neuron. For binary neurons the expectation is identical with the conditional probability for the neuron to be on given the state of the connected layer. More...

template<class Matrix , class Rng >
void	sample (StatisticsBatch const &statistics, Matrix &state, double alpha, Rng &rng) const
	Samples from the distribution using either Gibbs- or flip-the-state sampling. More...

template<class Matrix >
RealVector	logProbability (StatisticsBatch const &statistics, Matrix const &state) const
	Computes the log of the probability of the given states in the conditional distribution. More...

template<class Matrix >
Matrix const &	phi (Matrix const &state) const
	Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term. In the case of binary neurons the phi-function is just the identity. More...

RealMatrix const &	expectedPhiValue (StatisticsBatch const &statistics) const
	Returns the conditional expectation of the phi-function given the state of the connected layer, i.e. in this case the probabilities of the neurons having state one. More...

RealMatrix const &	mean (StatisticsBatch const &statistics) const
	Returns the mean given the state of the connected layer, i.e. in this case the probabilities of the neurons having state one. More...

template<class Matrix , class BetaVector >
RealVector	energyTerm (Matrix const &state, BetaVector const &beta) const
	Returns the energy term this neuron adds to the energy function. More...

template<class Input >
double	logMarginalize (Input const &inputs, double beta) const
	Sums over all possible values of the terms of the energy function which depend on the this layer and returns the logarithmic result. More...

template<class Vector , class SampleBatch >
void	expectedParameterDerivative (Vector &derivative, SampleBatch const &samples) const
	Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer. More...

template<class Vector , class SampleBatch , class WeightVector >
void	expectedParameterDerivative (Vector &derivative, SampleBatch const &samples, WeightVector const &weights) const
	Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer. More...

template<class Vector , class SampleBatch >
void	parameterDerivative (Vector &derivative, SampleBatch const &samples) const
	Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. More...

template<class Vector , class SampleBatch , class WeightVector >
void	parameterDerivative (Vector &derivative, SampleBatch const &samples, WeightVector const &weights) const
	Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. More...

RealVector	parameterVector () const
	Returns the vector with the parameters associated with the neurons in the layer, i.e. the bias vector. More...

void	setParameterVector (RealVector const &newParameters)
	Sets the parameters associated with the neurons in the layer, i.e. the bias vector. More...

std::size_t	numberOfParameters () const
	Returns the number of the parameters associated with the neurons in the layer. More...

void	read (InArchive &archive)
	Reads the bias vector from an archive. More...

void	write (OutArchive &archive) const
	Writes the bias vector to an archive. More...

Public Member Functions inherited from shark::ISerializable
virtual	~ISerializable ()
	Virtual d'tor. More...

void	load (InArchive &archive, unsigned int version)
	Versioned loading of components, calls read(...). More...

void	save (OutArchive &archive, unsigned int version) const
	Versioned storing of components, calls write(...). More...

	BOOST_SERIALIZATION_SPLIT_MEMBER ()

Public Member Functions inherited from shark::IParameterizable<>
virtual	~IParameterizable ()

Detailed Description

Layer of binary units taking values in {0,1}.

A neuron in a Binary Layer takes values in {0,1} and the conditional probability to be 1 given the states of the neurons in the connected layer is determined by the sigmoid function and the input it gets from the connected layer.

Definition at line 47 of file BinaryLayer.h.

Member Typedef Documentation

◆ StateSpace

typedef BinarySpace shark::BinaryLayer::StateSpace

The state space of this neuron is binary.

Definition at line 54 of file BinaryLayer.h.

◆ StatisticsBatch

typedef Batch<SufficientStatistics>::type shark::BinaryLayer::StatisticsBatch

Sufficient statistics of a batch of data.

Definition at line 59 of file BinaryLayer.h.

◆ SufficientStatistics

typedef RealVector shark::BinaryLayer::SufficientStatistics

The sufficient statistics for the Binary Layer store the probability for a neuron to be on.

Definition at line 57 of file BinaryLayer.h.

Member Function Documentation

◆ baseRate() [1/2]

RealVector const& shark::BinaryLayer::baseRate ( ) const

inline

Returns the base rate of the units.

The base-rate is the tempered disttribution for beta=0 beta then does a fading between the RBM and the base-rate

Definition at line 76 of file BinaryLayer.h.

◆ baseRate() [2/2]

RealVector& shark::BinaryLayer::baseRate ( )

inline

Returns the base rate of the units.

The base-rate is the tempered disttribution for beta=0 beta then does a fading between the RBM and the base-rate

Definition at line 84 of file BinaryLayer.h.

◆ bias() [1/2]

const RealVector& shark::BinaryLayer::bias ( ) const

inline

Returns the bias values of the units.

Definition at line 62 of file BinaryLayer.h.

◆ bias() [2/2]

RealVector& shark::BinaryLayer::bias ( )

inline

Returns the bias values of the units.

Definition at line 67 of file BinaryLayer.h.

◆ energyTerm()

template<class Matrix , class BetaVector >

RealVector shark::BinaryLayer::energyTerm	(	Matrix const &	state,
		BetaVector const &	beta
	)		const

inline

Returns the energy term this neuron adds to the energy function.

Parameters

state	the state of the neuron layer
beta	the inverse temperature of the i-th state

Returns: the energy term of the neuron layer

Definition at line 224 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ expectedParameterDerivative() [1/2]

template<class Vector , class SampleBatch >

void shark::BinaryLayer::expectedParameterDerivative	(	Vector &	derivative,
		SampleBatch const &	samples
	)		const

inline

Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer.

This function takes a batch of samples and extracts the required informations out of it.

Parameters

derivative	the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samples	the samples from which the informations can be extracted

Definition at line 269 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ expectedParameterDerivative() [2/2]

template<class Vector , class SampleBatch , class WeightVector >

void shark::BinaryLayer::expectedParameterDerivative	(	Vector &	derivative,
		SampleBatch const &	samples,
		WeightVector const &	weights
	)		const

inline

Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer.

This function takes a batch of samples and weights the results

Parameters

derivative	the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samples	the samples from which the informations can be extracted
weights	The weights for alle samples

Definition at line 282 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ expectedPhiValue()

RealMatrix const& shark::BinaryLayer::expectedPhiValue ( StatisticsBatch const & statistics ) const

inline

Returns the conditional expectation of the phi-function given the state of the connected layer, i.e. in this case the probabilities of the neurons having state one.

Parameters

statistics the sufficient statistics of the layer

Definition at line 206 of file BinaryLayer.h.

◆ logMarginalize()

template<class Input >

double shark::BinaryLayer::logMarginalize	(	Input const &	inputs,
		double	beta
	)		const

inline

Sums over all possible values of the terms of the energy function which depend on the this layer and returns the logarithmic result.

This function is called by Energy when the unnormalized marginal probability of the connected layer is to be computed. This function calculates the part which depends on the neurons which are to be marginalized out. (In the case of the binary hidden neuron, this is the term \( \sum_h e^{\vec h^T W \vec v+ \vec h^T \vec c} \)). The rest is calculated by the energy function. In the general case of a hidden layer, this function calculates \( \int_h e^(\phi_h(\vec h)^T W \phi_v(\vec v)+f_h(\vec h) ) \) where f_h is the energy term of this layer.

Parameters

inputs	the inputs of the neurons they get from the other layer
beta	the inverse temperature of the RBM

Returns: the marginal distribution of the connected layer

Definition at line 250 of file BinaryLayer.h.

References size(), SIZE_CHECK, and shark::softPlus().

◆ logProbability()

template<class Matrix >

RealVector shark::BinaryLayer::logProbability	(	StatisticsBatch const &	statistics,
		Matrix const &	state
	)		const

inline

Computes the log of the probability of the given states in the conditional distribution.

Currently it is only possible to compute the case with alpha=0

Parameters

statistics	the statistics of the conditional distribution
state	the state to check

Definition at line 175 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ mean()

RealMatrix const& shark::BinaryLayer::mean ( StatisticsBatch const & statistics ) const

inline

Returns the mean given the state of the connected layer, i.e. in this case the probabilities of the neurons having state one.

Parameters

statistics the sufficient statistics of the layer for a whole batch

Definition at line 213 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ numberOfParameters()

std::size_t shark::BinaryLayer::numberOfParameters ( ) const

inlinevirtual

Returns the number of the parameters associated with the neurons in the layer.

Reimplemented from shark::IParameterizable<>.

Definition at line 322 of file BinaryLayer.h.

References size().

◆ parameterDerivative() [1/2]

template<class Vector , class SampleBatch >

void shark::BinaryLayer::parameterDerivative	(	Vector &	derivative,
		SampleBatch const &	samples
	)		const

inline

Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights.

This function takes a batch of samples and extracts the required informations out of it.

Parameters

derivative	the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samples	the sample from which the informations can be extracted

Definition at line 294 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ parameterDerivative() [2/2]

template<class Vector , class SampleBatch , class WeightVector >

void shark::BinaryLayer::parameterDerivative	(	Vector &	derivative,
		SampleBatch const &	samples,
		WeightVector const &	weights
	)		const

inline

Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights.

This function takes a batch of samples and calculates a weighted derivative

Parameters

derivative	the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samples	the sample from which the informations can be extracted
weights	the weights for the single sample derivatives

Definition at line 306 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ parameterVector()

RealVector shark::BinaryLayer::parameterVector ( ) const

inlinevirtual

Returns the vector with the parameters associated with the neurons in the layer, i.e. the bias vector.

Reimplemented from shark::IParameterizable<>.

Definition at line 312 of file BinaryLayer.h.

◆ phi()

template<class Matrix >

Matrix const& shark::BinaryLayer::phi ( Matrix const & state ) const

inline

Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term. In the case of binary neurons the phi-function is just the identity.

Parameters

state the state matrix of the neuron layer

Returns: the value of the phi-function

Definition at line 196 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ read()

void shark::BinaryLayer::read ( InArchive & archive )

inlinevirtual

Reads the bias vector from an archive.

Parameters

archive the archive

Reimplemented from shark::ISerializable.

Definition at line 329 of file BinaryLayer.h.

◆ resize()

void shark::BinaryLayer::resize ( std::size_t newSize )

inline

Resizes this neuron layer.

Parameters

newSize number of neurons in the layer

Definition at line 91 of file BinaryLayer.h.

◆ sample()

template<class Matrix , class Rng >

void shark::BinaryLayer::sample	(	StatisticsBatch const &	statistics,
		Matrix &	state,
		double	alpha,
		Rng &	rng
	)		const

inline

Samples from the distribution using either Gibbs- or flip-the-state sampling.

For alpha= 0 gibbs sampling is performed. That is the next state for neuron i is directly taken from the conditional distribution of the i-th neuron. In the case of alpha=1, flip-the-state sampling is performed, which takes the last state into account and tries to do deterministically jump into states with higher probability. This is counterbalanced by a higher chance to jump back into a lower probability state in later steps. For alpha between 0 and 1 a mixture of both is performed.

Parameters

statistics	sufficient statistics containing the probabilities of the neurons to be one
state	the state vector that shell hold the sampled states
alpha	factor changing from gibbs to flip-the state sampling. 0<=alpha<=1
rng	the random number generator used for sampling

Definition at line 131 of file BinaryLayer.h.

References shark::random::coinToss(), SHARK_CRITICAL_REGION, size(), and SIZE_CHECK.

◆ setParameterVector()

void shark::BinaryLayer::setParameterVector ( RealVector const & newParameters )

inlinevirtual

Sets the parameters associated with the neurons in the layer, i.e. the bias vector.

Reimplemented from shark::IParameterizable<>.

Definition at line 317 of file BinaryLayer.h.

◆ size()

std::size_t shark::BinaryLayer::size ( ) const

inline

Returns the number of neurons of this layer.

Definition at line 98 of file BinaryLayer.h.

Referenced by energyTerm(), expectedParameterDerivative(), logMarginalize(), logProbability(), mean(), numberOfParameters(), parameterDerivative(), phi(), sample(), and sufficientStatistics().

◆ sufficientStatistics()

template<class Input , class BetaVector >

void shark::BinaryLayer::sufficientStatistics	(	Input const &	input,
		StatisticsBatch &	statistics,
		BetaVector const &	beta
	)		const

inline

Takes the input of the neuron and estimates the expectation of the response of the neuron. For binary neurons the expectation is identical with the conditional probability for the neuron to be on given the state of the connected layer.

Parameters

input	the batch of inputs of the neuron
statistics	sufficient statistics containing the probabilities of the neurons to be one
beta	the inverse Temperature of the RBM (typically 1) for the whole batch

Definition at line 109 of file BinaryLayer.h.

References size(), and SIZE_CHECK.

◆ write()

void shark::BinaryLayer::write ( OutArchive & archive ) const

inlinevirtual

Writes the bias vector to an archive.

Parameters

archive the archive

Reimplemented from shark::ISerializable.

Definition at line 337 of file BinaryLayer.h.

The documentation for this class was generated from the following file:

include/shark/Unsupervised/RBM/Neuronlayers/BinaryLayer.h

Public Types

Public Member Functions

Detailed Description

Member Typedef Documentation

◆ StateSpace

◆ StatisticsBatch

◆ SufficientStatistics

Member Function Documentation

◆ baseRate() [1/2]

◆ baseRate() [2/2]

◆ bias() [1/2]

◆ bias() [2/2]

◆ energyTerm()

◆ expectedParameterDerivative() [1/2]

◆ expectedParameterDerivative() [2/2]

◆ expectedPhiValue()

◆ logMarginalize()

◆ logProbability()

◆ mean()

◆ numberOfParameters()

◆ parameterDerivative() [1/2]

◆ parameterDerivative() [2/2]

◆ parameterVector()

◆ phi()

◆ read()

◆ resize()

◆ sample()

◆ setParameterVector()

◆ size()

◆ sufficientStatistics()

◆ write()