Layer of bipolar units taking values in {-1,1}. More...
#include <shark/Unsupervised/RBM/Neuronlayers/BipolarLayer.h>
Public Types | |
typedef SymmetricBinarySpace | StateSpace |
the state space of this neuron is binary More... | |
typedef RealVector | SufficientStatistics |
The sufficient statistics for the Binary Layer store the probability for a neuron to be on. More... | |
typedef Batch< SufficientStatistics >::type | StatisticsBatch |
Sufficient statistics of a batch of data. More... | |
Public Types inherited from shark::IParameterizable<> | |
typedef RealVector | ParameterVectorType |
Public Member Functions | |
const RealVector & | bias () const |
Returns the bias values of the units. More... | |
RealVector & | bias () |
Returns the bias values of the units. More... | |
void | resize (std::size_t newSize) |
Resizes this neuron layer. More... | |
std::size_t | size () const |
Returns the number of neurons of this layer. More... | |
template<class Input , class BetaVector > | |
void | sufficientStatistics (Input const &input, StatisticsBatch &statistics, BetaVector const &beta) const |
Takes the input of the neuron and calculates the distribution of the neurons For binary neuronsthis is identical with the conditional probability for the neuron to be on given the state of the connected layer. More... | |
template<class Matrix , class Rng > | |
void | sample (StatisticsBatch const &statistics, Matrix &state, double alpha, Rng &rng) const |
Samples from the distribution using either Gibbs- or flip-the-state sampling. More... | |
template<class Matrix > | |
RealVector | logProbability (StatisticsBatch const &statistics, Matrix const &state) const |
Computes the log of the probability of the given states in the conditional distribution. More... | |
template<class Matrix > | |
Matrix const & | phi (Matrix const &state) const |
Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term. In the case of bipolar neurons the phi-function is just the identity. More... | |
RealMatrix | expectedPhiValue (StatisticsBatch const &statistics) const |
Returns the conditional expectation of the phi-function given the state of the connected layer,. More... | |
RealMatrix | mean (StatisticsBatch const &statistics) const |
Returns the mean of the distribution. More... | |
template<class Matrix , class BetaVector > | |
RealVector | energyTerm (Matrix const &state, BetaVector const &beta) const |
Returns the energy term this neuron adds to the energy function. More... | |
template<class Input > | |
double | logMarginalize (Input const &inputs, double beta) const |
Sums over all possible values of the terms of the energy function which depend on the this layer and returns the logarithmic result. More... | |
template<class Vector , class SampleBatch > | |
void | expectedParameterDerivative (Vector &derivative, SampleBatch const &samples) const |
Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer. More... | |
template<class Vector , class SampleBatch , class WeightVector > | |
void | expectedParameterDerivative (Vector &derivative, SampleBatch const &samples, WeightVector const &weights) const |
Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer. More... | |
template<class Vector , class SampleBatch > | |
void | parameterDerivative (Vector &derivative, SampleBatch const &samples) const |
Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. More... | |
template<class Vector , class SampleBatch , class WeightVector > | |
void | parameterDerivative (Vector &derivative, SampleBatch const &samples, WeightVector const &weights) const |
Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. More... | |
RealVector | parameterVector () const |
Returns the vector with the parameters associated with the neurons in the layer, i.e. the bias vector. More... | |
void | setParameterVector (RealVector const &newParameters) |
Sets the parameters associated with the neurons in the layer, i.e. the bias vector. More... | |
std::size_t | numberOfParameters () const |
Returns the number of the parameters associated with the neurons in the layer. More... | |
void | read (InArchive &archive) |
Reads the bias vector from an archive. More... | |
void | write (OutArchive &archive) const |
Writes the bias vector to an archive. More... | |
Public Member Functions inherited from shark::ISerializable | |
virtual | ~ISerializable () |
Virtual d'tor. More... | |
void | load (InArchive &archive, unsigned int version) |
Versioned loading of components, calls read(...). More... | |
void | save (OutArchive &archive, unsigned int version) const |
Versioned storing of components, calls write(...). More... | |
BOOST_SERIALIZATION_SPLIT_MEMBER () | |
Public Member Functions inherited from shark::IParameterizable<> | |
virtual | ~IParameterizable () |
Layer of bipolar units taking values in {-1,1}.
A neuron in a BipolarLayer takes values in {-1,1} and the conditional probability to be 1 given the states of the neurons in the connected layer is determined by the sigmoid function and the input it gets from the connected layer.
Definition at line 45 of file BipolarLayer.h.
the state space of this neuron is binary
Definition at line 51 of file BipolarLayer.h.
typedef Batch<SufficientStatistics>::type shark::BipolarLayer::StatisticsBatch |
Sufficient statistics of a batch of data.
Definition at line 56 of file BipolarLayer.h.
typedef RealVector shark::BipolarLayer::SufficientStatistics |
The sufficient statistics for the Binary Layer store the probability for a neuron to be on.
Definition at line 54 of file BipolarLayer.h.
|
inline |
Returns the bias values of the units.
Definition at line 59 of file BipolarLayer.h.
|
inline |
Returns the bias values of the units.
Definition at line 64 of file BipolarLayer.h.
|
inline |
Returns the energy term this neuron adds to the energy function.
state | the state of the neuron layer |
beta | the inverse temperature of the i-th state |
Definition at line 206 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inline |
Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer.
This function takes a batch of samples and extracts the required informations out of it.
derivative | the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives |
samples | the samples from which the informations can be extracted |
Definition at line 243 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inline |
Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer.
This function takes a batch of samples and weights the results
derivative | the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives |
samples | the samples from which the informations can be extracted |
weights | The weights for alle samples |
Definition at line 257 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inline |
Returns the conditional expectation of the phi-function given the state of the connected layer,.
statistics | the sufficient statistics of the layer |
Definition at line 185 of file BipolarLayer.h.
|
inline |
Sums over all possible values of the terms of the energy function which depend on the this layer and returns the logarithmic result.
This function is called by Energy when the unnormalized marginal probability of the connected layer is to be computed. This function calculates the part which depends on the neurons which are to be marginalized out. (In the case of the bipolar hidden neuron, this is the term \( \sum_h e^{\vec h^T W \vec v+ \vec h^T \vec c} \)). The rest is calculated by the energy function.
inputs | the inputs of the neurons they get from the other layer |
beta | the inverse temperature of the RBM |
Definition at line 225 of file BipolarLayer.h.
References size(), SIZE_CHECK, and shark::softPlus().
|
inline |
Computes the log of the probability of the given states in the conditional distribution.
Currently it is only possible to compute the case with alpha=0
statistics | the statistics of the conditional distribution |
state | the state to check |
Definition at line 155 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inline |
Returns the mean of the distribution.
statistics | the sufficient statistics of the layer for a whole batch |
Definition at line 193 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inlinevirtual |
Returns the number of the parameters associated with the neurons in the layer.
Reimplemented from shark::IParameterizable<>.
Definition at line 297 of file BipolarLayer.h.
References size().
|
inline |
Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights.
This function takes a batch of samples and extracts the required informations out of it.
derivative | the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives |
samples | the sample from which the informations can be extracted |
Definition at line 269 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inline |
Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights.
This function takes a batch of samples and calculates a weighted derivative
derivative | the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives |
samples | the sample from which the informations can be extracted |
weights | the weights for the single sample derivatives |
Definition at line 281 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inlinevirtual |
Returns the vector with the parameters associated with the neurons in the layer, i.e. the bias vector.
Reimplemented from shark::IParameterizable<>.
Definition at line 287 of file BipolarLayer.h.
|
inline |
Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term. In the case of bipolar neurons the phi-function is just the identity.
state | the state matrix of the neuron layer |
Definition at line 176 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inlinevirtual |
Reads the bias vector from an archive.
archive | the archive |
Reimplemented from shark::ISerializable.
Definition at line 304 of file BipolarLayer.h.
|
inline |
Resizes this neuron layer.
newSize | number of neurons in the layer |
Definition at line 71 of file BipolarLayer.h.
|
inline |
Samples from the distribution using either Gibbs- or flip-the-state sampling.
For alpha= 0 gibbs sampling is performed. That is the next state for neuron i is directly taken from the conditional distribution of the i-th neuron. In the case of alpha=1, flip-the-state sampling is performed, which takes the last state into account and tries to do deterministically jump into states with higher probability. This is counterbalanced by a higher chance to jump back into a lower probability state in later steps. For alpha between 0 and 1 a mixture of both is performed.
statistics | sufficient statistics containing the probabilities of the neurons to be one |
state | the state vector that shell hold the sampled states |
alpha | factor changing from gibbs to flip-the state sampling. 0<=alpha<=1 |
rng | the random number generator used for sampling |
Definition at line 109 of file BipolarLayer.h.
References shark::random::coinToss(), SHARK_CRITICAL_REGION, size(), and SIZE_CHECK.
|
inlinevirtual |
Sets the parameters associated with the neurons in the layer, i.e. the bias vector.
Reimplemented from shark::IParameterizable<>.
Definition at line 292 of file BipolarLayer.h.
|
inline |
Returns the number of neurons of this layer.
Definition at line 76 of file BipolarLayer.h.
Referenced by energyTerm(), expectedParameterDerivative(), logMarginalize(), logProbability(), mean(), numberOfParameters(), parameterDerivative(), phi(), sample(), and sufficientStatistics().
|
inline |
Takes the input of the neuron and calculates the distribution of the neurons For binary neuronsthis is identical with the conditional probability for the neuron to be on given the state of the connected layer.
input | the batch of inputs of the neuron |
statistics | sufficient statistics containing the probabilities of the neurons to be one |
beta | the inverse Temperature of the RBM (typically 1) for the whole batch |
Definition at line 87 of file BipolarLayer.h.
References size(), and SIZE_CHECK.
|
inlinevirtual |
Writes the bias vector to an archive.
archive | the archive |
Reimplemented from shark::ISerializable.
Definition at line 311 of file BipolarLayer.h.