shark::Autoencoder< HiddenNeuron, OutputNeuron > Class Template Reference

implements the autoencoder More...

#include <shark/Models/Autoencoder.h>

+ Inheritance diagram for shark::Autoencoder< HiddenNeuron, OutputNeuron >:

Public Member Functions

 Autoencoder ()
 
std::string name () const
 From INameable: return the class name. More...
 
std::size_t inputSize () const
 Number of input neurons. More...
 
std::size_t outputSize () const
 Number of output neurons. More...
 
std::size_t numberOfHiddenNeurons () const
 Total number of hidden neurons. More...
 
RealVector const & hiddenBias () const
 Returns the hidden bias weight vector. More...
 
RealVector & hiddenBias ()
 Returns the hidden bias weight vector. More...
 
RealVector const & outputBias () const
 Returns the output bias weight vector. More...
 
RealVector & outputBias ()
 Returns the output bias weight vector. More...
 
RealMatrix const & encoderMatrix () const
 Weight matrix for the direction input->hidden. More...
 
RealMatrix & encoderMatrix ()
 Weight matrix for the direction input->hidden. More...
 
RealMatrix const & decoderMatrix () const
 Weight matrix for the direction hidden->output. More...
 
RealMatrix & decoderMatrix ()
 Weight matrix for the direction hidden->output. More...
 
std::size_t numberOfParameters () const
 Returns the total number of parameters of the network. More...
 
RealVector parameterVector () const
 returns the vector of used parameters inside the weight matrix More...
 
void setParameterVector (RealVector const &newParameters)
 uses the values inside the parametervector to set the used values inside the weight matrix More...
 
RealMatrix const & hiddenResponses (State const &state) const
 Returns the output of all neurons after the last call of eval. More...
 
HiddenNeuron const & hiddenActivationFunction () const
 Returns the activation function of the hidden units. More...
 
OutputNeuron const & outputActivationFunction () const
 Returns the activation function of the output units. More...
 
HiddenNeuron & hiddenActivationFunction ()
 Returns the activation function of the hidden units. More...
 
OutputNeuron & outputActivationFunction ()
 Returns the activation function of the output units. More...
 
boost::shared_ptr< StatecreateState () const
 Creates an internal state of the model. More...
 
void evalLayer (std::size_t layer, RealMatrix const &patterns, RealMatrix &outputs) const
 
Data< RealVector > evalLayer (std::size_t layer, Data< RealVector > const &patterns) const
 Returns the response of the i-th layer given the input of that layer. More...
 
Data< RealVector > encode (Data< RealVector > const &patterns) const
 
Data< RealVector > decode (Data< RealVector > const &patterns) const
 
template<class Label >
LabeledData< RealVector, Label > encode (LabeledData< RealVector, Label > const &data) const
 
template<class Label >
LabeledData< RealVector, Label > decode (LabeledData< RealVector, Label > const &data) const
 
void eval (RealMatrix const &patterns, RealMatrix &output, State &state) const
 
void weightedParameterDerivative (BatchInputType const &patterns, RealMatrix const &coefficients, State const &state, RealVector &gradient) const
 
void weightedInputDerivative (BatchInputType const &patterns, RealMatrix const &coefficients, State const &state, BatchInputType &inputDerivative) const
 
virtual void weightedDerivatives (BatchInputType const &patterns, BatchOutputType const &coefficients, State const &state, RealVector &parameterDerivative, BatchInputType &inputDerivative) const
 calculates weighted input and parameter derivative at the same time More...
 
void setStructure (std::size_t in, std::size_t hidden)
 
void read (InArchive &archive)
 From ISerializable, reads a model from an archive. More...
 
void write (OutArchive &archive) const
 From ISerializable, writes a model to an archive. More...
 
- Public Member Functions inherited from shark::AbstractModel< RealVector, RealVector >
 AbstractModel ()
 
virtual ~AbstractModel ()
 
const Featuresfeatures () const
 
virtual void updateFeatures ()
 
bool hasFirstParameterDerivative () const
 Returns true when the first parameter derivative is implemented. More...
 
bool hasSecondParameterDerivative () const
 Returns true when the second parameter derivative is implemented. More...
 
bool hasFirstInputDerivative () const
 Returns true when the first input derivative is implemented. More...
 
bool hasSecondInputDerivative () const
 Returns true when the second parameter derivative is implemented. More...
 
bool isSequential () const
 
virtual void eval (BatchInputType const &patterns, BatchOutputType &outputs) const
 Standard interface for evaluating the response of the model to a batch of patterns. More...
 
virtual void eval (BatchInputType const &patterns, BatchOutputType &outputs, State &state) const=0
 Standard interface for evaluating the response of the model to a batch of patterns. More...
 
virtual void eval (InputType const &pattern, OutputType &output) const
 Standard interface for evaluating the response of the model to a single pattern. More...
 
Data< OutputTypeoperator() (Data< InputType > const &patterns) const
 Model evaluation as an operator for a whole dataset. This is a convenience function. More...
 
OutputType operator() (InputType const &pattern) const
 Model evaluation as an operator for a single pattern. This is a convenience function. More...
 
BatchOutputType operator() (BatchInputType const &patterns) const
 Model evaluation as an operator for a single pattern. This is a convenience function. More...
 
virtual void weightedParameterDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, State const &state, RealVector &derivative) const
 calculates the weighted sum of derivatives w.r.t the parameters. More...
 
virtual void weightedParameterDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, Batch< RealMatrix >::type const &errorHessian, State const &state, RealVector &derivative, RealMatrix &hessian) const
 calculates the weighted sum of derivatives w.r.t the parameters More...
 
virtual void weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, State const &state, BatchInputType &derivative) const
 calculates the weighted sum of derivatives w.r.t the inputs More...
 
virtual void weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, typename Batch< RealMatrix >::type const &errorHessian, State const &state, RealMatrix &derivative, Batch< RealMatrix >::type &hessian) const
 calculates the weighted sum of derivatives w.r.t the inputs More...
 
- Public Member Functions inherited from shark::IParameterizable
virtual ~IParameterizable ()
 
- Public Member Functions inherited from shark::INameable
virtual ~INameable ()
 
- Public Member Functions inherited from shark::ISerializable
virtual ~ISerializable ()
 Virtual d'tor. More...
 
void load (InArchive &archive, unsigned int version)
 Versioned loading of components, calls read(...). More...
 
void save (OutArchive &archive, unsigned int version) const
 Versioned storing of components, calls write(...). More...
 
 BOOST_SERIALIZATION_SPLIT_MEMBER ()
 

Additional Inherited Members

- Public Types inherited from shark::AbstractModel< RealVector, RealVector >
enum  Feature
 
typedef RealVector InputType
 Defines the input type of the model. More...
 
typedef RealVector OutputType
 Defines the output type of the model. More...
 
typedef Batch< InputType >::type BatchInputType
 defines the batch type of the input type. More...
 
typedef Batch< OutputType >::type BatchOutputType
 defines the batch type of the output type More...
 
typedef TypedFlags< FeatureFeatures
 
typedef TypedFeatureNotAvailableException< FeatureFeatureNotAvailableException
 
- Protected Attributes inherited from shark::AbstractModel< RealVector, RealVector >
Features m_features
 

Detailed Description

template<class HiddenNeuron, class OutputNeuron>
class shark::Autoencoder< HiddenNeuron, OutputNeuron >

implements the autoencoder

The formula is

\[ f(x) = \sigma_2(W^T\sigma_1(Wx+b_1)+b_2)\]

Where \( W, W_2, b_1 \) and \(b_2 \) are the weights and \(\sigma_1\) and \( \sigma_2\) are the activation functions for hidden and output units.

see TiedAutoencoder for the tied weights version where \( W_2=W_1^T \).

Definition at line 46 of file Autoencoder.h.

Constructor & Destructor Documentation

◆ Autoencoder()

Member Function Documentation

◆ createState()

template<class HiddenNeuron, class OutputNeuron>
boost::shared_ptr<State> shark::Autoencoder< HiddenNeuron, OutputNeuron >::createState ( ) const
inlinevirtual

Creates an internal state of the model.

The state is needed when the derivatives are to be calculated. Eval can store a state which is then reused to speed up the calculations of the derivatives. This also allows eval to be evaluated in parallel!

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Definition at line 165 of file Autoencoder.h.

◆ decode() [1/2]

template<class HiddenNeuron, class OutputNeuron>
Data<RealVector> shark::Autoencoder< HiddenNeuron, OutputNeuron >::decode ( Data< RealVector > const &  patterns) const
inline

◆ decode() [2/2]

template<class HiddenNeuron, class OutputNeuron>
template<class Label >
LabeledData<RealVector,Label> shark::Autoencoder< HiddenNeuron, OutputNeuron >::decode ( LabeledData< RealVector, Label > const &  data) const
inline

◆ decoderMatrix() [1/2]

template<class HiddenNeuron, class OutputNeuron>
RealMatrix const& shark::Autoencoder< HiddenNeuron, OutputNeuron >::decoderMatrix ( ) const
inline

Weight matrix for the direction hidden->output.

Definition at line 108 of file Autoencoder.h.

Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::write().

◆ decoderMatrix() [2/2]

template<class HiddenNeuron, class OutputNeuron>
RealMatrix& shark::Autoencoder< HiddenNeuron, OutputNeuron >::decoderMatrix ( )
inline

Weight matrix for the direction hidden->output.

Definition at line 112 of file Autoencoder.h.

◆ encode() [1/2]

template<class HiddenNeuron, class OutputNeuron>
Data<RealVector> shark::Autoencoder< HiddenNeuron, OutputNeuron >::encode ( Data< RealVector > const &  patterns) const
inline

◆ encode() [2/2]

template<class HiddenNeuron, class OutputNeuron>
template<class Label >
LabeledData<RealVector,Label> shark::Autoencoder< HiddenNeuron, OutputNeuron >::encode ( LabeledData< RealVector, Label > const &  data) const
inline

◆ encoderMatrix() [1/2]

template<class HiddenNeuron, class OutputNeuron>
RealMatrix const& shark::Autoencoder< HiddenNeuron, OutputNeuron >::encoderMatrix ( ) const
inline

◆ encoderMatrix() [2/2]

template<class HiddenNeuron, class OutputNeuron>
RealMatrix& shark::Autoencoder< HiddenNeuron, OutputNeuron >::encoderMatrix ( )
inline

Weight matrix for the direction input->hidden.

Definition at line 103 of file Autoencoder.h.

◆ eval()

template<class HiddenNeuron, class OutputNeuron>
void shark::Autoencoder< HiddenNeuron, OutputNeuron >::eval ( RealMatrix const &  patterns,
RealMatrix &  output,
State state 
) const
inline

◆ evalLayer() [1/2]

◆ evalLayer() [2/2]

template<class HiddenNeuron, class OutputNeuron>
Data<RealVector> shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer ( std::size_t  layer,
Data< RealVector > const &  patterns 
) const
inline

Returns the response of the i-th layer given the input of that layer.

this is usefull if only a portion of the network needs to be evaluated be aware that this only works without shortcuts in the network

Definition at line 195 of file Autoencoder.h.

References shark::Data< Type >::batch(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), shark::Data< Type >::numberOfBatches(), SHARK_PARALLEL_FOR, and SIZE_CHECK.

◆ hiddenActivationFunction() [1/2]

template<class HiddenNeuron, class OutputNeuron>
HiddenNeuron const& shark::Autoencoder< HiddenNeuron, OutputNeuron >::hiddenActivationFunction ( ) const
inline

Returns the activation function of the hidden units.

Definition at line 148 of file Autoencoder.h.

◆ hiddenActivationFunction() [2/2]

template<class HiddenNeuron, class OutputNeuron>
HiddenNeuron& shark::Autoencoder< HiddenNeuron, OutputNeuron >::hiddenActivationFunction ( )
inline

Returns the activation function of the hidden units.

Definition at line 157 of file Autoencoder.h.

◆ hiddenBias() [1/2]

template<class HiddenNeuron, class OutputNeuron>
RealVector const& shark::Autoencoder< HiddenNeuron, OutputNeuron >::hiddenBias ( ) const
inline

Returns the hidden bias weight vector.

Definition at line 80 of file Autoencoder.h.

Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and unsupervisedPreTraining().

◆ hiddenBias() [2/2]

template<class HiddenNeuron, class OutputNeuron>
RealVector& shark::Autoencoder< HiddenNeuron, OutputNeuron >::hiddenBias ( )
inline

Returns the hidden bias weight vector.

Definition at line 85 of file Autoencoder.h.

◆ hiddenResponses()

template<class HiddenNeuron, class OutputNeuron>
RealMatrix const& shark::Autoencoder< HiddenNeuron, OutputNeuron >::hiddenResponses ( State const &  state) const
inline

Returns the output of all neurons after the last call of eval.

Parameters
statelast result of eval
Returns
Output value of the neurons.

Definition at line 142 of file Autoencoder.h.

References shark::State::toState().

◆ inputSize()

template<class HiddenNeuron, class OutputNeuron>
std::size_t shark::Autoencoder< HiddenNeuron, OutputNeuron >::inputSize ( ) const
inline

◆ name()

template<class HiddenNeuron, class OutputNeuron>
std::string shark::Autoencoder< HiddenNeuron, OutputNeuron >::name ( ) const
inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 61 of file Autoencoder.h.

Referenced by trainAutoencoderModel().

◆ numberOfHiddenNeurons()

template<class HiddenNeuron, class OutputNeuron>
std::size_t shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfHiddenNeurons ( ) const
inline

◆ numberOfParameters()

template<class HiddenNeuron, class OutputNeuron>
std::size_t shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfParameters ( ) const
inlinevirtual

◆ outputActivationFunction() [1/2]

template<class HiddenNeuron, class OutputNeuron>
OutputNeuron const& shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputActivationFunction ( ) const
inline

Returns the activation function of the output units.

Definition at line 152 of file Autoencoder.h.

◆ outputActivationFunction() [2/2]

template<class HiddenNeuron, class OutputNeuron>
OutputNeuron& shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputActivationFunction ( )
inline

Returns the activation function of the output units.

Definition at line 161 of file Autoencoder.h.

◆ outputBias() [1/2]

template<class HiddenNeuron, class OutputNeuron>
RealVector const& shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputBias ( ) const
inline

Returns the output bias weight vector.

Definition at line 90 of file Autoencoder.h.

Referenced by shark::Autoencoder< HiddenNeuron, OutputNeuron >::evalLayer(), and shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize().

◆ outputBias() [2/2]

template<class HiddenNeuron, class OutputNeuron>
RealVector& shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputBias ( )
inline

Returns the output bias weight vector.

Definition at line 94 of file Autoencoder.h.

◆ outputSize()

◆ parameterVector()

template<class HiddenNeuron, class OutputNeuron>
RealVector shark::Autoencoder< HiddenNeuron, OutputNeuron >::parameterVector ( ) const
inlinevirtual

returns the vector of used parameters inside the weight matrix

Reimplemented from shark::IParameterizable.

Definition at line 122 of file Autoencoder.h.

References remora::to_vector().

◆ read()

template<class HiddenNeuron, class OutputNeuron>
void shark::Autoencoder< HiddenNeuron, OutputNeuron >::read ( InArchive archive)
inlinevirtual

From ISerializable, reads a model from an archive.

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Definition at line 285 of file Autoencoder.h.

◆ setParameterVector()

template<class HiddenNeuron, class OutputNeuron>
void shark::Autoencoder< HiddenNeuron, OutputNeuron >::setParameterVector ( RealVector const &  newParameters)
inlinevirtual

uses the values inside the parametervector to set the used values inside the weight matrix

Reimplemented from shark::IParameterizable.

Definition at line 126 of file Autoencoder.h.

References remora::noalias(), shark::Autoencoder< HiddenNeuron, OutputNeuron >::numberOfParameters(), SIZE_CHECK, remora::subrange(), and remora::to_vector().

Referenced by initializeFFNet(), and trainAutoencoderModel().

◆ setStructure()

template<class HiddenNeuron, class OutputNeuron>
void shark::Autoencoder< HiddenNeuron, OutputNeuron >::setStructure ( std::size_t  in,
std::size_t  hidden 
)
inline

Definition at line 275 of file Autoencoder.h.

Referenced by main(), and trainAutoencoderModel().

◆ weightedDerivatives()

template<class HiddenNeuron, class OutputNeuron>
virtual void shark::Autoencoder< HiddenNeuron, OutputNeuron >::weightedDerivatives ( BatchInputType const &  patterns,
BatchOutputType const &  coefficients,
State const &  state,
RealVector &  parameterDerivative,
BatchInputType inputDerivative 
) const
inlinevirtual

calculates weighted input and parameter derivative at the same time

Sometimes, both derivatives are needed at the same time. But sometimes, when calculating the weighted parameter derivative, the input derivative can be calculated for free. This is for example true for the feed-forward neural networks. However, there exists the obvious default implementation to just calculate the derivatives one after another.

Parameters
patternsthe patterns to evaluate
coefficientsthe coefficients which are used to calculate the weighted sum
stateintermediate results stored by eval to sped up calculations of the derivatives
parameterDerivativethe calculated parameter derivative as sum over all derivates of all patterns
inputDerivativethe calculated derivative for every pattern

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Definition at line 259 of file Autoencoder.h.

References shark::Autoencoder< HiddenNeuron, OutputNeuron >::outputSize(), and SIZE_CHECK.

◆ weightedInputDerivative()

template<class HiddenNeuron, class OutputNeuron>
void shark::Autoencoder< HiddenNeuron, OutputNeuron >::weightedInputDerivative ( BatchInputType const &  patterns,
RealMatrix const &  coefficients,
State const &  state,
BatchInputType inputDerivative 
) const
inline

◆ weightedParameterDerivative()

template<class HiddenNeuron, class OutputNeuron>
void shark::Autoencoder< HiddenNeuron, OutputNeuron >::weightedParameterDerivative ( BatchInputType const &  patterns,
RealMatrix const &  coefficients,
State const &  state,
RealVector &  gradient 
) const
inline

◆ write()


The documentation for this class was generated from the following file: