shark::BipolarLayer Class Reference

Layer of bipolar units taking values in {-1,1}. More...

#include <shark/Unsupervised/RBM/Neuronlayers/BipolarLayer.h>

+ Inheritance diagram for shark::BipolarLayer:

Public Types

typedef SymmetricBinarySpace StateSpace
 the state space of this neuron is binary More...
 
typedef RealVector SufficientStatistics
 The sufficient statistics for the Binary Layer store the probability for a neuron to be on. More...
 
typedef Batch< SufficientStatistics >::type StatisticsBatch
 Sufficient statistics of a batch of data. More...
 
- Public Types inherited from shark::IParameterizable<>
typedef RealVector ParameterVectorType
 

Public Member Functions

const RealVector & bias () const
 Returns the bias values of the units. More...
 
RealVector & bias ()
 Returns the bias values of the units. More...
 
void resize (std::size_t newSize)
 Resizes this neuron layer. More...
 
std::size_t size () const
 Returns the number of neurons of this layer. More...
 
template<class Input , class BetaVector >
void sufficientStatistics (Input const &input, StatisticsBatch &statistics, BetaVector const &beta) const
 Takes the input of the neuron and calculates the distribution of the neurons For binary neuronsthis is identical with the conditional probability for the neuron to be on given the state of the connected layer. More...
 
template<class Matrix , class Rng >
void sample (StatisticsBatch const &statistics, Matrix &state, double alpha, Rng &rng) const
 Samples from the distribution using either Gibbs- or flip-the-state sampling. More...
 
template<class Matrix >
RealVector logProbability (StatisticsBatch const &statistics, Matrix const &state) const
 Computes the log of the probability of the given states in the conditional distribution. More...
 
template<class Matrix >
Matrix const & phi (Matrix const &state) const
 Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term. In the case of bipolar neurons the phi-function is just the identity. More...
 
RealMatrix expectedPhiValue (StatisticsBatch const &statistics) const
 Returns the conditional expectation of the phi-function given the state of the connected layer,. More...
 
RealMatrix mean (StatisticsBatch const &statistics) const
 Returns the mean of the distribution. More...
 
template<class Matrix , class BetaVector >
RealVector energyTerm (Matrix const &state, BetaVector const &beta) const
 Returns the energy term this neuron adds to the energy function. More...
 
template<class Input >
double logMarginalize (Input const &inputs, double beta) const
 Sums over all possible values of the terms of the energy function which depend on the this layer and returns the logarithmic result. More...
 
template<class Vector , class SampleBatch >
void expectedParameterDerivative (Vector &derivative, SampleBatch const &samples) const
 Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer. More...
 
template<class Vector , class SampleBatch , class WeightVector >
void expectedParameterDerivative (Vector &derivative, SampleBatch const &samples, WeightVector const &weights) const
 Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer. More...
 
template<class Vector , class SampleBatch >
void parameterDerivative (Vector &derivative, SampleBatch const &samples) const
 Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. More...
 
template<class Vector , class SampleBatch , class WeightVector >
void parameterDerivative (Vector &derivative, SampleBatch const &samples, WeightVector const &weights) const
 Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. More...
 
RealVector parameterVector () const
 Returns the vector with the parameters associated with the neurons in the layer, i.e. the bias vector. More...
 
void setParameterVector (RealVector const &newParameters)
 Sets the parameters associated with the neurons in the layer, i.e. the bias vector. More...
 
std::size_t numberOfParameters () const
 Returns the number of the parameters associated with the neurons in the layer. More...
 
void read (InArchive &archive)
 Reads the bias vector from an archive. More...
 
void write (OutArchive &archive) const
 Writes the bias vector to an archive. More...
 
- Public Member Functions inherited from shark::ISerializable
virtual ~ISerializable ()
 Virtual d'tor. More...
 
void load (InArchive &archive, unsigned int version)
 Versioned loading of components, calls read(...). More...
 
void save (OutArchive &archive, unsigned int version) const
 Versioned storing of components, calls write(...). More...
 
 BOOST_SERIALIZATION_SPLIT_MEMBER ()
 
- Public Member Functions inherited from shark::IParameterizable<>
virtual ~IParameterizable ()
 

Detailed Description

Layer of bipolar units taking values in {-1,1}.

A neuron in a BipolarLayer takes values in {-1,1} and the conditional probability to be 1 given the states of the neurons in the connected layer is determined by the sigmoid function and the input it gets from the connected layer.

Definition at line 45 of file BipolarLayer.h.

Member Typedef Documentation

◆ StateSpace

the state space of this neuron is binary

Definition at line 51 of file BipolarLayer.h.

◆ StatisticsBatch

Sufficient statistics of a batch of data.

Definition at line 56 of file BipolarLayer.h.

◆ SufficientStatistics

The sufficient statistics for the Binary Layer store the probability for a neuron to be on.

Definition at line 54 of file BipolarLayer.h.

Member Function Documentation

◆ bias() [1/2]

const RealVector& shark::BipolarLayer::bias ( ) const
inline

Returns the bias values of the units.

Definition at line 59 of file BipolarLayer.h.

◆ bias() [2/2]

RealVector& shark::BipolarLayer::bias ( )
inline

Returns the bias values of the units.

Definition at line 64 of file BipolarLayer.h.

◆ energyTerm()

template<class Matrix , class BetaVector >
RealVector shark::BipolarLayer::energyTerm ( Matrix const &  state,
BetaVector const &  beta 
) const
inline

Returns the energy term this neuron adds to the energy function.

Parameters
statethe state of the neuron layer
betathe inverse temperature of the i-th state
Returns
the energy term of the neuron layer

Definition at line 206 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ expectedParameterDerivative() [1/2]

template<class Vector , class SampleBatch >
void shark::BipolarLayer::expectedParameterDerivative ( Vector &  derivative,
SampleBatch const &  samples 
) const
inline

Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer.

This function takes a batch of samples and extracts the required informations out of it.

Parameters
derivativethe derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samplesthe samples from which the informations can be extracted

Definition at line 243 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ expectedParameterDerivative() [2/2]

template<class Vector , class SampleBatch , class WeightVector >
void shark::BipolarLayer::expectedParameterDerivative ( Vector &  derivative,
SampleBatch const &  samples,
WeightVector const &  weights 
) const
inline

Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer.

This function takes a batch of samples and weights the results

Parameters
derivativethe derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samplesthe samples from which the informations can be extracted
weightsThe weights for alle samples

Definition at line 257 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ expectedPhiValue()

RealMatrix shark::BipolarLayer::expectedPhiValue ( StatisticsBatch const &  statistics) const
inline

Returns the conditional expectation of the phi-function given the state of the connected layer,.

Parameters
statisticsthe sufficient statistics of the layer

Definition at line 185 of file BipolarLayer.h.

◆ logMarginalize()

template<class Input >
double shark::BipolarLayer::logMarginalize ( Input const &  inputs,
double  beta 
) const
inline

Sums over all possible values of the terms of the energy function which depend on the this layer and returns the logarithmic result.

This function is called by Energy when the unnormalized marginal probability of the connected layer is to be computed. This function calculates the part which depends on the neurons which are to be marginalized out. (In the case of the bipolar hidden neuron, this is the term \( \sum_h e^{\vec h^T W \vec v+ \vec h^T \vec c} \)). The rest is calculated by the energy function.

Parameters
inputsthe inputs of the neurons they get from the other layer
betathe inverse temperature of the RBM
Returns
the marginal distribution of the connected layer

Definition at line 225 of file BipolarLayer.h.

References size(), SIZE_CHECK, and shark::softPlus().

◆ logProbability()

template<class Matrix >
RealVector shark::BipolarLayer::logProbability ( StatisticsBatch const &  statistics,
Matrix const &  state 
) const
inline

Computes the log of the probability of the given states in the conditional distribution.

Currently it is only possible to compute the case with alpha=0

Parameters
statisticsthe statistics of the conditional distribution
statethe state to check

Definition at line 155 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ mean()

RealMatrix shark::BipolarLayer::mean ( StatisticsBatch const &  statistics) const
inline

Returns the mean of the distribution.

Parameters
statisticsthe sufficient statistics of the layer for a whole batch

Definition at line 193 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ numberOfParameters()

std::size_t shark::BipolarLayer::numberOfParameters ( ) const
inlinevirtual

Returns the number of the parameters associated with the neurons in the layer.

Reimplemented from shark::IParameterizable<>.

Definition at line 297 of file BipolarLayer.h.

References size().

◆ parameterDerivative() [1/2]

template<class Vector , class SampleBatch >
void shark::BipolarLayer::parameterDerivative ( Vector &  derivative,
SampleBatch const &  samples 
) const
inline

Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights.

This function takes a batch of samples and extracts the required informations out of it.

Parameters
derivativethe derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samplesthe sample from which the informations can be extracted

Definition at line 269 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ parameterDerivative() [2/2]

template<class Vector , class SampleBatch , class WeightVector >
void shark::BipolarLayer::parameterDerivative ( Vector &  derivative,
SampleBatch const &  samples,
WeightVector const &  weights 
) const
inline

Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights.

This function takes a batch of samples and calculates a weighted derivative

Parameters
derivativethe derivative with respect to the parameters, the result is added on top of it to accumulate derivatives
samplesthe sample from which the informations can be extracted
weightsthe weights for the single sample derivatives

Definition at line 281 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ parameterVector()

RealVector shark::BipolarLayer::parameterVector ( ) const
inlinevirtual

Returns the vector with the parameters associated with the neurons in the layer, i.e. the bias vector.

Reimplemented from shark::IParameterizable<>.

Definition at line 287 of file BipolarLayer.h.

◆ phi()

template<class Matrix >
Matrix const& shark::BipolarLayer::phi ( Matrix const &  state) const
inline

Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term. In the case of bipolar neurons the phi-function is just the identity.

Parameters
statethe state matrix of the neuron layer
Returns
the value of the phi-function

Definition at line 176 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ read()

void shark::BipolarLayer::read ( InArchive archive)
inlinevirtual

Reads the bias vector from an archive.

Parameters
archivethe archive

Reimplemented from shark::ISerializable.

Definition at line 304 of file BipolarLayer.h.

◆ resize()

void shark::BipolarLayer::resize ( std::size_t  newSize)
inline

Resizes this neuron layer.

Parameters
newSizenumber of neurons in the layer

Definition at line 71 of file BipolarLayer.h.

◆ sample()

template<class Matrix , class Rng >
void shark::BipolarLayer::sample ( StatisticsBatch const &  statistics,
Matrix &  state,
double  alpha,
Rng &  rng 
) const
inline

Samples from the distribution using either Gibbs- or flip-the-state sampling.

For alpha= 0 gibbs sampling is performed. That is the next state for neuron i is directly taken from the conditional distribution of the i-th neuron. In the case of alpha=1, flip-the-state sampling is performed, which takes the last state into account and tries to do deterministically jump into states with higher probability. This is counterbalanced by a higher chance to jump back into a lower probability state in later steps. For alpha between 0 and 1 a mixture of both is performed.

Parameters
statisticssufficient statistics containing the probabilities of the neurons to be one
statethe state vector that shell hold the sampled states
alphafactor changing from gibbs to flip-the state sampling. 0<=alpha<=1
rngthe random number generator used for sampling

Definition at line 109 of file BipolarLayer.h.

References shark::random::coinToss(), SHARK_CRITICAL_REGION, size(), and SIZE_CHECK.

◆ setParameterVector()

void shark::BipolarLayer::setParameterVector ( RealVector const &  newParameters)
inlinevirtual

Sets the parameters associated with the neurons in the layer, i.e. the bias vector.

Reimplemented from shark::IParameterizable<>.

Definition at line 292 of file BipolarLayer.h.

◆ size()

std::size_t shark::BipolarLayer::size ( ) const
inline

Returns the number of neurons of this layer.

Definition at line 76 of file BipolarLayer.h.

Referenced by energyTerm(), expectedParameterDerivative(), logMarginalize(), logProbability(), mean(), numberOfParameters(), parameterDerivative(), phi(), sample(), and sufficientStatistics().

◆ sufficientStatistics()

template<class Input , class BetaVector >
void shark::BipolarLayer::sufficientStatistics ( Input const &  input,
StatisticsBatch statistics,
BetaVector const &  beta 
) const
inline

Takes the input of the neuron and calculates the distribution of the neurons For binary neuronsthis is identical with the conditional probability for the neuron to be on given the state of the connected layer.

Parameters
inputthe batch of inputs of the neuron
statisticssufficient statistics containing the probabilities of the neurons to be one
betathe inverse Temperature of the RBM (typically 1) for the whole batch

Definition at line 87 of file BipolarLayer.h.

References size(), and SIZE_CHECK.

◆ write()

void shark::BipolarLayer::write ( OutArchive archive) const
inlinevirtual

Writes the bias vector to an archive.

Parameters
archivethe archive

Reimplemented from shark::ISerializable.

Definition at line 311 of file BipolarLayer.h.


The documentation for this class was generated from the following file: