implements the autoencoder More...
#include <shark/Models/Autoencoder.h>
Public Member Functions | |
Autoencoder () | |
std::string | name () const |
From INameable: return the class name. More... | |
std::size_t | inputSize () const |
Number of input neurons. More... | |
std::size_t | outputSize () const |
Number of output neurons. More... | |
std::size_t | numberOfHiddenNeurons () const |
Total number of hidden neurons. More... | |
RealVector const & | hiddenBias () const |
Returns the hidden bias weight vector. More... | |
RealVector & | hiddenBias () |
Returns the hidden bias weight vector. More... | |
RealVector const & | outputBias () const |
Returns the output bias weight vector. More... | |
RealVector & | outputBias () |
Returns the output bias weight vector. More... | |
RealMatrix const & | encoderMatrix () const |
Weight matrix for the direction input->hidden. More... | |
RealMatrix & | encoderMatrix () |
Weight matrix for the direction input->hidden. More... | |
RealMatrix const & | decoderMatrix () const |
Weight matrix for the direction hidden->output. More... | |
RealMatrix & | decoderMatrix () |
Weight matrix for the direction hidden->output. More... | |
std::size_t | numberOfParameters () const |
Returns the total number of parameters of the network. More... | |
RealVector | parameterVector () const |
returns the vector of used parameters inside the weight matrix More... | |
void | setParameterVector (RealVector const &newParameters) |
uses the values inside the parametervector to set the used values inside the weight matrix More... | |
RealMatrix const & | hiddenResponses (State const &state) const |
Returns the output of all neurons after the last call of eval. More... | |
HiddenNeuron const & | hiddenActivationFunction () const |
Returns the activation function of the hidden units. More... | |
OutputNeuron const & | outputActivationFunction () const |
Returns the activation function of the output units. More... | |
HiddenNeuron & | hiddenActivationFunction () |
Returns the activation function of the hidden units. More... | |
OutputNeuron & | outputActivationFunction () |
Returns the activation function of the output units. More... | |
boost::shared_ptr< State > | createState () const |
Creates an internal state of the model. More... | |
void | evalLayer (std::size_t layer, RealMatrix const &patterns, RealMatrix &outputs) const |
Data< RealVector > | evalLayer (std::size_t layer, Data< RealVector > const &patterns) const |
Returns the response of the i-th layer given the input of that layer. More... | |
Data< RealVector > | encode (Data< RealVector > const &patterns) const |
Data< RealVector > | decode (Data< RealVector > const &patterns) const |
template<class Label > | |
LabeledData< RealVector, Label > | encode (LabeledData< RealVector, Label > const &data) const |
template<class Label > | |
LabeledData< RealVector, Label > | decode (LabeledData< RealVector, Label > const &data) const |
void | eval (RealMatrix const &patterns, RealMatrix &output, State &state) const |
void | weightedParameterDerivative (BatchInputType const &patterns, RealMatrix const &coefficients, State const &state, RealVector &gradient) const |
void | weightedInputDerivative (BatchInputType const &patterns, RealMatrix const &coefficients, State const &state, BatchInputType &inputDerivative) const |
virtual void | weightedDerivatives (BatchInputType const &patterns, BatchOutputType const &coefficients, State const &state, RealVector ¶meterDerivative, BatchInputType &inputDerivative) const |
calculates weighted input and parameter derivative at the same time More... | |
void | setStructure (std::size_t in, std::size_t hidden) |
void | read (InArchive &archive) |
From ISerializable, reads a model from an archive. More... | |
void | write (OutArchive &archive) const |
From ISerializable, writes a model to an archive. More... | |
Public Member Functions inherited from shark::AbstractModel< RealVector, RealVector > | |
AbstractModel () | |
virtual | ~AbstractModel () |
const Features & | features () const |
virtual void | updateFeatures () |
bool | hasFirstParameterDerivative () const |
Returns true when the first parameter derivative is implemented. More... | |
bool | hasSecondParameterDerivative () const |
Returns true when the second parameter derivative is implemented. More... | |
bool | hasFirstInputDerivative () const |
Returns true when the first input derivative is implemented. More... | |
bool | hasSecondInputDerivative () const |
Returns true when the second parameter derivative is implemented. More... | |
bool | isSequential () const |
virtual void | eval (BatchInputType const &patterns, BatchOutputType &outputs) const |
Standard interface for evaluating the response of the model to a batch of patterns. More... | |
virtual void | eval (BatchInputType const &patterns, BatchOutputType &outputs, State &state) const=0 |
Standard interface for evaluating the response of the model to a batch of patterns. More... | |
virtual void | eval (InputType const &pattern, OutputType &output) const |
Standard interface for evaluating the response of the model to a single pattern. More... | |
Data< OutputType > | operator() (Data< InputType > const &patterns) const |
Model evaluation as an operator for a whole dataset. This is a convenience function. More... | |
OutputType | operator() (InputType const &pattern) const |
Model evaluation as an operator for a single pattern. This is a convenience function. More... | |
BatchOutputType | operator() (BatchInputType const &patterns) const |
Model evaluation as an operator for a single pattern. This is a convenience function. More... | |
virtual void | weightedParameterDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, State const &state, RealVector &derivative) const |
calculates the weighted sum of derivatives w.r.t the parameters. More... | |
virtual void | weightedParameterDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, Batch< RealMatrix >::type const &errorHessian, State const &state, RealVector &derivative, RealMatrix &hessian) const |
calculates the weighted sum of derivatives w.r.t the parameters More... | |
virtual void | weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, State const &state, BatchInputType &derivative) const |
calculates the weighted sum of derivatives w.r.t the inputs More... | |
virtual void | weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, typename Batch< RealMatrix >::type const &errorHessian, State const &state, RealMatrix &derivative, Batch< RealMatrix >::type &hessian) const |
calculates the weighted sum of derivatives w.r.t the inputs More... | |
Public Member Functions inherited from shark::IParameterizable | |
virtual | ~IParameterizable () |
Public Member Functions inherited from shark::INameable | |
virtual | ~INameable () |
Public Member Functions inherited from shark::ISerializable | |
virtual | ~ISerializable () |
Virtual d'tor. More... | |
void | load (InArchive &archive, unsigned int version) |
Versioned loading of components, calls read(...). More... | |
void | save (OutArchive &archive, unsigned int version) const |
Versioned storing of components, calls write(...). More... | |
BOOST_SERIALIZATION_SPLIT_MEMBER () | |
Additional Inherited Members | |
Public Types inherited from shark::AbstractModel< RealVector, RealVector > | |
enum | Feature |
typedef RealVector | InputType |
Defines the input type of the model. More... | |
typedef RealVector | OutputType |
Defines the output type of the model. More... | |
typedef Batch< InputType >::type | BatchInputType |
defines the batch type of the input type. More... | |
typedef Batch< OutputType >::type | BatchOutputType |
defines the batch type of the output type More... | |
typedef TypedFlags< Feature > | Features |
typedef TypedFeatureNotAvailableException< Feature > | FeatureNotAvailableException |
Protected Attributes inherited from shark::AbstractModel< RealVector, RealVector > | |
Features | m_features |
implements the autoencoder
The formula is
\[ f(x) = \sigma_2(W^T\sigma_1(Wx+b_1)+b_2)\]
Where \( W, W_2, b_1 \) and \(b_2 \) are the weights and \(\sigma_1\) and \( \sigma_2\) are the activation functions for hidden and output units.
see TiedAutoencoder for the tied weights version where \( W_2=W_1^T \).
Definition at line 46 of file Autoencoder.h.
|
inline |
|
inlinevirtual |
Creates an internal state of the model.
The state is needed when the derivatives are to be calculated. Eval can store a state which is then reused to speed up the calculations of the derivatives. This also allows eval to be evaluated in parallel!
Reimplemented from shark::AbstractModel< RealVector, RealVector >.
Definition at line 165 of file Autoencoder.h.
|
inline |
Definition at line 209 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer().
Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::decode().
|
inline |
Definition at line 221 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::decode(), shark::LabeledData< InputT, LabelT >::inputs(), and shark::LabeledData< InputT, LabelT >::labels().
|
inline |
Weight matrix for the direction hidden->output.
Definition at line 108 of file Autoencoder.h.
Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::write().
|
inline |
Weight matrix for the direction hidden->output.
Definition at line 112 of file Autoencoder.h.
|
inline |
Definition at line 205 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer().
Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::encode().
|
inline |
Definition at line 214 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::encode(), shark::LabeledData< InputT, LabelT >::inputs(), and shark::LabeledData< InputT, LabelT >::labels().
|
inline |
Weight matrix for the direction input->hidden.
Definition at line 99 of file Autoencoder.h.
Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), main(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfHiddenNeurons(), unsupervisedPreTraining(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::write().
|
inline |
Weight matrix for the direction input->hidden.
Definition at line 103 of file Autoencoder.h.
|
inline |
Definition at line 228 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and shark::State::toState().
|
inline |
Definition at line 169 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::decoderMatrix(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::encoderMatrix(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::hiddenBias(), remora::noalias(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputBias(), remora::prod(), remora::repeat(), SIZE_CHECK, and remora::trans().
Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::decode(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::encode(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::eval(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and unsupervisedPreTraining().
|
inline |
Returns the response of the i-th layer given the input of that layer.
this is usefull if only a portion of the network needs to be evaluated be aware that this only works without shortcuts in the network
Definition at line 195 of file Autoencoder.h.
References shark::Data< Type >::batch(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), shark::Data< Type >::numberOfBatches(), SHARK_PARALLEL_FOR, and SIZE_CHECK.
|
inline |
Returns the activation function of the hidden units.
Definition at line 148 of file Autoencoder.h.
|
inline |
Returns the activation function of the hidden units.
Definition at line 157 of file Autoencoder.h.
|
inline |
Returns the hidden bias weight vector.
Definition at line 80 of file Autoencoder.h.
Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and unsupervisedPreTraining().
|
inline |
Returns the hidden bias weight vector.
Definition at line 85 of file Autoencoder.h.
|
inline |
Returns the output of all neurons after the last call of eval.
state | last result of eval |
Definition at line 142 of file Autoencoder.h.
References shark::State::toState().
|
inline |
Number of input neurons.
Definition at line 66 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize().
Referenced by initializeFFNet(), main(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfParameters(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::write().
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 61 of file Autoencoder.h.
Referenced by trainAutoencoderModel().
|
inline |
Total number of hidden neurons.
Definition at line 75 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::encoderMatrix().
Referenced by initializeFFNet(), main(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfParameters(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::write().
|
inlinevirtual |
Returns the total number of parameters of the network.
Reimplemented from shark::IParameterizable.
Definition at line 117 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::inputSize(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfHiddenNeurons().
Referenced by initializeFFNet(), main(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::setParameterVector(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::write().
|
inline |
Returns the activation function of the output units.
Definition at line 152 of file Autoencoder.h.
|
inline |
Returns the activation function of the output units.
Definition at line 161 of file Autoencoder.h.
|
inline |
Returns the output bias weight vector.
Definition at line 90 of file Autoencoder.h.
Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize().
|
inline |
Returns the output bias weight vector.
Definition at line 94 of file Autoencoder.h.
|
inline |
Number of output neurons.
Definition at line 70 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputBias().
Referenced by initializeFFNet(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::inputSize(), main(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::weightedDerivatives(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::weightedInputDerivative(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::weightedParameterDerivative(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::write().
|
inlinevirtual |
returns the vector of used parameters inside the weight matrix
Reimplemented from shark::IParameterizable.
Definition at line 122 of file Autoencoder.h.
References remora::to_vector().
|
inlinevirtual |
From ISerializable, reads a model from an archive.
Reimplemented from shark::AbstractModel< RealVector, RealVector >.
Definition at line 285 of file Autoencoder.h.
|
inlinevirtual |
uses the values inside the parametervector to set the used values inside the weight matrix
Reimplemented from shark::IParameterizable.
Definition at line 126 of file Autoencoder.h.
References remora::noalias(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfParameters(), SIZE_CHECK, remora::subrange(), and remora::to_vector().
Referenced by initializeFFNet(), and trainAutoencoderModel().
|
inline |
Definition at line 275 of file Autoencoder.h.
Referenced by main(), and trainAutoencoderModel().
|
inlinevirtual |
calculates weighted input and parameter derivative at the same time
Sometimes, both derivatives are needed at the same time. But sometimes, when calculating the weighted parameter derivative, the input derivative can be calculated for free. This is for example true for the feed-forward neural networks. However, there exists the obvious default implementation to just calculate the derivatives one after another.
patterns | the patterns to evaluate |
coefficients | the coefficients which are used to calculate the weighted sum |
state | intermediate results stored by eval to sped up calculations of the derivatives |
parameterDerivative | the calculated parameter derivative as sum over all derivates of all patterns |
inputDerivative | the calculated derivative for every pattern |
Reimplemented from shark::AbstractModel< RealVector, RealVector >.
Definition at line 259 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize(), and SIZE_CHECK.
|
inline |
Definition at line 248 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize(), and SIZE_CHECK.
|
inline |
Definition at line 236 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize(), and SIZE_CHECK.
|
inlinevirtual |
From ISerializable, writes a model to an archive.
Reimplemented from shark::AbstractModel< RealVector, RealVector >.
Definition at line 293 of file Autoencoder.h.
References shark::Autoencoder< HiddenNeuron, OutputNeuron >::decoderMatrix(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::encoderMatrix(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::inputSize(), remora::noalias(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfHiddenNeurons(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfParameters(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize(), remora::prod(), remora::subrange(), remora::bindings::sum_rows(), remora::to_matrix(), shark::State::toState(), and remora::trans().