A layer of truncated exponential neurons. More...
#include <shark/Unsupervised/RBM/Neuronlayers/TruncatedExponentialLayer.h>
Public Types | |
typedef RealSpace | StateSpace |
the state space of this neuron is real valued, so it can't be enumerated More... | |
typedef detail::TruncatedExponentialSufficientStatistics< RealVector > | SufficientStatistics |
Stores lambda, the defining parameter of the statistics and also exp(-lambda) since it is used regularly. More... | |
typedef Batch< SufficientStatistics >::type | StatisticsBatch |
Sufficient statistics of a batch of data. More... | |
Public Types inherited from shark::IParameterizable<> | |
typedef RealVector | ParameterVectorType |
Public Member Functions | |
const RealVector & | bias () const |
Returns the bias values of the units. More... | |
RealVector & | bias () |
Returns the bias values of the units. More... | |
void | resize (std::size_t newSize) |
Resizes this neuron layer. More... | |
std::size_t | size () const |
Returns the number of neurons of this layer. More... | |
template<class Input , class BetaVector > | |
void | sufficientStatistics (Input const &input, StatisticsBatch &statistics, BetaVector const &beta) const |
Takes the input of the neuron and calculates the statistics required to sample from the conditional distribution. More... | |
template<class Matrix , class Rng > | |
void | sample (StatisticsBatch const &statistics, Matrix &state, double alpha, Rng &rng) const |
Samples from the truncated exponential distribution using either Gibbs- or flip-the-state sampling given the sufficient statistics (i.e. the parameter lambda and the value of exp(-lambda)) More... | |
template<class Matrix > | |
Matrix const & | phi (Matrix const &state) const |
Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term. More... | |
RealMatrix | expectedPhiValue (StatisticsBatch const &statistics) const |
Returns the conditional expectation of the phi-function given the state of the connected layer. More... | |
RealMatrix | mean (StatisticsBatch const &statistics) const |
Returns the mean of the conditional distribution. More... | |
template<class Matrix , class BetaVector > | |
RealVector | energyTerm (Matrix const &state, BetaVector const &beta) const |
Returns the energy term this neuron adds to the energy function. More... | |
template<class Input > | |
double | logMarginalize (const Input &inputs, double beta) const |
Integrates over the terms of the Energy function which depend on the state of this layer and returns the logarithm of the result. More... | |
template<class Vector , class SampleBatch > | |
void | expectedParameterDerivative (Vector &derivative, SampleBatch const &samples) const |
Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer. More... | |
template<class Vector , class SampleBatch > | |
void | parameterDerivative (Vector &derivative, SampleBatch const &samples) const |
Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. More... | |
RealVector | parameterVector () const |
Returns the vector with the parameters associated with the neurons in the layer. More... | |
void | setParameterVector (RealVector const &newParameters) |
Returns the vector with the parameters associated with the neurons in the layer. More... | |
std::size_t | numberOfParameters () const |
Returns the number of the parameters associated with the neurons in the layer. More... | |
void | read (InArchive &archive) |
Reads the bias parameters from an archive. More... | |
void | write (OutArchive &archive) const |
Writes the bias parameters to an archive. More... | |
Public Member Functions inherited from shark::ISerializable | |
virtual | ~ISerializable () |
Virtual d'tor. More... | |
void | load (InArchive &archive, unsigned int version) |
Versioned loading of components, calls read(...). More... | |
void | save (OutArchive &archive, unsigned int version) const |
Versioned storing of components, calls write(...). More... | |
BOOST_SERIALIZATION_SPLIT_MEMBER () | |
Public Member Functions inherited from shark::IParameterizable<> | |
virtual | ~IParameterizable () |
A layer of truncated exponential neurons.
Truncated exponential distributions arise, when the state space of the binary neurons is extended to the real numbers in [0,1]. The conditional distribution of the state of this neurons given the states of the connecred layer is an exponential distribution restricted to [0,1].
Definition at line 73 of file TruncatedExponentialLayer.h.
the state space of this neuron is real valued, so it can't be enumerated
Definition at line 78 of file TruncatedExponentialLayer.h.
Sufficient statistics of a batch of data.
Definition at line 83 of file TruncatedExponentialLayer.h.
typedef detail::TruncatedExponentialSufficientStatistics<RealVector> shark::TruncatedExponentialLayer::SufficientStatistics |
Stores lambda, the defining parameter of the statistics and also exp(-lambda) since it is used regularly.
Definition at line 81 of file TruncatedExponentialLayer.h.
|
inline |
Returns the bias values of the units.
Definition at line 86 of file TruncatedExponentialLayer.h.
|
inline |
Returns the bias values of the units.
Definition at line 90 of file TruncatedExponentialLayer.h.
|
inline |
Returns the energy term this neuron adds to the energy function.
state | the state of the neuron layer |
beta | the inverse temperature of the i-th state |
Definition at line 197 of file TruncatedExponentialLayer.h.
References SIZE_CHECK.
|
inline |
Calculates the expectation of the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights. The expectation is taken with respect to the conditional probability distribution of the layer given the state of the connected layer.
This function takes a batch of samples and extracts the required informations out of it.
derivative | the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives |
samples | the samples from which the informations can be extracted |
Definition at line 244 of file TruncatedExponentialLayer.h.
References SIZE_CHECK.
|
inline |
Returns the conditional expectation of the phi-function given the state of the connected layer.
statistics | the sufficient statistics of the layer |
Definition at line 171 of file TruncatedExponentialLayer.h.
References shark::batchSize(), shark::mean(), and SIZE_CHECK.
|
inline |
Integrates over the terms of the Energy function which depend on the state of this layer and returns the logarithm of the result.
This function is called by Energy when the unnormalized marginal probability of the connected layer is to be computed. It calculates the part which depends on the neurons which are to be marinalized out. (In the case of the exponential hidden neuron, this is the term \( \log \int_h e^{\vec h^T W \vec v+ \vec h^T \vec c} \)).
inputs | the inputs of the neurons they get from the other layer |
beta | the inverse temperature of the RBM |
Definition at line 217 of file TruncatedExponentialLayer.h.
References SIZE_CHECK.
|
inline |
Returns the mean of the conditional distribution.
statistics | the sufficient statistics defining the conditional distribution |
Definition at line 187 of file TruncatedExponentialLayer.h.
|
inlinevirtual |
Returns the number of the parameters associated with the neurons in the layer.
Reimplemented from shark::IParameterizable<>.
Definition at line 272 of file TruncatedExponentialLayer.h.
|
inline |
Calculates the derivatives of the energy term of this neuron layer with respect to it's parameters - the bias weights.
This function takes a batch of samples and extracts the required informations out of it.
derivative | the derivative with respect to the parameters, the result is added on top of it to accumulate derivatives |
samples | the sample from which the informations can be extracted |
Definition at line 256 of file TruncatedExponentialLayer.h.
References SIZE_CHECK.
|
inlinevirtual |
Returns the vector with the parameters associated with the neurons in the layer.
Reimplemented from shark::IParameterizable<>.
Definition at line 262 of file TruncatedExponentialLayer.h.
|
inline |
Transforms the current state of the neurons for the multiplication with the weight matrix of the RBM, i.e. calculates the value of the phi-function used in the interaction term.
state | the state matrix of the neuron layer |
Definition at line 161 of file TruncatedExponentialLayer.h.
References SIZE_CHECK.
|
inlinevirtual |
Reads the bias parameters from an archive.
Reimplemented from shark::ISerializable.
Definition at line 277 of file TruncatedExponentialLayer.h.
|
inline |
Resizes this neuron layer.
newSize | number of neurons in the layer |
Definition at line 97 of file TruncatedExponentialLayer.h.
|
inline |
Samples from the truncated exponential distribution using either Gibbs- or flip-the-state sampling given the sufficient statistics (i.e. the parameter lambda and the value of exp(-lambda))
The truncated exponential function is defined as
\[ p(x) = \lambda \frac{e^{-lambdax}}{1 - e^{-\lambda}}\]
with x being in the range of [0,1]
For alpha= 0 gibbs sampling is performed. That is the next state for neuron i is directly taken from the conditional distribution of the i-th neuron. In the case of alpha=1, flip-the-state sampling is performed, which takes the last state into account and tries to do deterministically jump into states with higher probability. THIS IS NOT IMPLEMENTED YET and alpha is ignored!
statistics | sufficient statistics for the batch to be computed |
state | the state matrix that will hold the sampled states |
alpha | factor changing from gibbs to flip-the state sampling. 0<=alpha<=1 |
rng | the random number generator used for sampling |
Definition at line 140 of file TruncatedExponentialLayer.h.
References SHARK_CRITICAL_REGION, SIZE_CHECK, and shark::random::truncExp().
|
inlinevirtual |
Returns the vector with the parameters associated with the neurons in the layer.
Reimplemented from shark::IParameterizable<>.
Definition at line 267 of file TruncatedExponentialLayer.h.
|
inline |
Returns the number of neurons of this layer.
Definition at line 102 of file TruncatedExponentialLayer.h.
|
inline |
Takes the input of the neuron and calculates the statistics required to sample from the conditional distribution.
input | the batch of inputs of the neuron |
statistics | sufficient statistics containing the probabilities of the neurons to be one |
beta | the inverse Temperature of the RBM (typically 1) for the whole batch |
Definition at line 112 of file TruncatedExponentialLayer.h.
References SIZE_CHECK.
|
inlinevirtual |
Writes the bias parameters to an archive.
Reimplemented from shark::ISerializable.
Definition at line 281 of file TruncatedExponentialLayer.h.