shark::Softmax Class Reference

Softmax function. More...

#include <shark/Models/Softmax.h>

+ Inheritance diagram for shark::Softmax:

Public Member Functions

SHARK_EXPORT_SYMBOL Softmax (size_t inputs)
 Constructor. More...
 
SHARK_EXPORT_SYMBOL Softmax ()
 Constructor. More...
 
std::string name () const
 From INameable: return the class name. More...
 
RealVector parameterVector () const
 Return the parameter vector. More...
 
void setParameterVector (RealVector const &newParameters)
 Set the parameter vector. More...
 
size_t inputSize () const
 
size_t outputSize () const
 
size_t numberOfParameters () const
 Return the number of parameters. More...
 
boost::shared_ptr< StatecreateState () const
 Creates an internal state of the model. More...
 
SHARK_EXPORT_SYMBOL void eval (BatchInputType const &patterns, BatchOutputType &output) const
 Standard interface for evaluating the response of the model to a batch of patterns. More...
 
SHARK_EXPORT_SYMBOL void eval (BatchInputType const &patterns, BatchOutputType &output, State &state) const
 Standard interface for evaluating the response of the model to a batch of patterns. More...
 
SHARK_EXPORT_SYMBOL void weightedParameterDerivative (BatchInputType const &patterns, BatchOutputType const &coefficients, State const &state, RealVector &gradient) const
 calculates the weighted sum of derivatives w.r.t the parameters. More...
 
SHARK_EXPORT_SYMBOL void weightedInputDerivative (BatchInputType const &patterns, RealMatrix const &coefficients, State const &state, BatchOutputType &gradient) const
 
void setStructure (std::size_t inputSize)
 
SHARK_EXPORT_SYMBOL void read (InArchive &archive)
 From ISerializable, reads a model from an archive. More...
 
SHARK_EXPORT_SYMBOL void write (OutArchive &archive) const
 From ISerializable, writes a model to an archive. More...
 
- Public Member Functions inherited from shark::AbstractModel< RealVector, RealVector >
 AbstractModel ()
 
virtual ~AbstractModel ()
 
const Featuresfeatures () const
 
virtual void updateFeatures ()
 
bool hasFirstParameterDerivative () const
 Returns true when the first parameter derivative is implemented. More...
 
bool hasSecondParameterDerivative () const
 Returns true when the second parameter derivative is implemented. More...
 
bool hasFirstInputDerivative () const
 Returns true when the first input derivative is implemented. More...
 
bool hasSecondInputDerivative () const
 Returns true when the second parameter derivative is implemented. More...
 
bool isSequential () const
 
virtual void eval (InputType const &pattern, OutputType &output) const
 Standard interface for evaluating the response of the model to a single pattern. More...
 
Data< OutputTypeoperator() (Data< InputType > const &patterns) const
 Model evaluation as an operator for a whole dataset. This is a convenience function. More...
 
OutputType operator() (InputType const &pattern) const
 Model evaluation as an operator for a single pattern. This is a convenience function. More...
 
BatchOutputType operator() (BatchInputType const &patterns) const
 Model evaluation as an operator for a single pattern. This is a convenience function. More...
 
virtual void weightedParameterDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, Batch< RealMatrix >::type const &errorHessian, State const &state, RealVector &derivative, RealMatrix &hessian) const
 calculates the weighted sum of derivatives w.r.t the parameters More...
 
virtual void weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, State const &state, BatchInputType &derivative) const
 calculates the weighted sum of derivatives w.r.t the inputs More...
 
virtual void weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, typename Batch< RealMatrix >::type const &errorHessian, State const &state, RealMatrix &derivative, Batch< RealMatrix >::type &hessian) const
 calculates the weighted sum of derivatives w.r.t the inputs More...
 
virtual void weightedDerivatives (BatchInputType const &patterns, BatchOutputType const &coefficients, State const &state, RealVector &parameterDerivative, BatchInputType &inputDerivative) const
 calculates weighted input and parameter derivative at the same time More...
 
- Public Member Functions inherited from shark::IParameterizable
virtual ~IParameterizable ()
 
- Public Member Functions inherited from shark::INameable
virtual ~INameable ()
 
- Public Member Functions inherited from shark::ISerializable
virtual ~ISerializable ()
 Virtual d'tor. More...
 
void load (InArchive &archive, unsigned int version)
 Versioned loading of components, calls read(...). More...
 
void save (OutArchive &archive, unsigned int version) const
 Versioned storing of components, calls write(...). More...
 
 BOOST_SERIALIZATION_SPLIT_MEMBER ()
 

Additional Inherited Members

- Public Types inherited from shark::AbstractModel< RealVector, RealVector >
enum  Feature
 
typedef RealVector InputType
 Defines the input type of the model. More...
 
typedef RealVector OutputType
 Defines the output type of the model. More...
 
typedef Batch< InputType >::type BatchInputType
 defines the batch type of the input type. More...
 
typedef Batch< OutputType >::type BatchOutputType
 defines the batch type of the output type More...
 
typedef TypedFlags< FeatureFeatures
 
typedef TypedFeatureNotAvailableException< FeatureFeatureNotAvailableException
 
- Protected Attributes inherited from shark::AbstractModel< RealVector, RealVector >
Features m_features
 

Detailed Description

Softmax function.

Squash an n-dimensional real vector space to the (n-1)-dimensional probability simplex:

\[ f_i(x) = \frac{\exp(x_i)}{\sum_j \exp(x_j)} \]

This also corresponds to the exponential norm of the input.

in the case of n=1, the output is

\[ f_i(x) = \frac{\exp((2i-1)x)}{\exp(x_j)+\exp(-x_j)} \]

and the output dimension is 2.

This convention ensures that all models that are trained via CrossEntropy can be used as input to this model and the output will be the probability of the labels.

Definition at line 65 of file Softmax.h.

Constructor & Destructor Documentation

◆ Softmax() [1/2]

SHARK_EXPORT_SYMBOL shark::Softmax::Softmax ( size_t  inputs)

Constructor.

◆ Softmax() [2/2]

SHARK_EXPORT_SYMBOL shark::Softmax::Softmax ( )

Constructor.

Member Function Documentation

◆ createState()

boost::shared_ptr<State> shark::Softmax::createState ( ) const
inlinevirtual

Creates an internal state of the model.

The state is needed when the derivatives are to be calculated. Eval can store a state which is then reused to speed up the calculations of the derivatives. This also allows eval to be evaluated in parallel!

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Definition at line 103 of file Softmax.h.

References eval(), SHARK_EXPORT_SYMBOL, weightedInputDerivative(), and weightedParameterDerivative().

◆ eval() [1/2]

SHARK_EXPORT_SYMBOL void shark::Softmax::eval ( BatchInputType const &  patterns,
BatchOutputType outputs 
) const
virtual

Standard interface for evaluating the response of the model to a batch of patterns.

Parameters
patternsthe inputs of the model
outputsthe predictions or response of the model to every pattern

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Referenced by createState().

◆ eval() [2/2]

SHARK_EXPORT_SYMBOL void shark::Softmax::eval ( BatchInputType const &  patterns,
BatchOutputType outputs,
State state 
) const
virtual

Standard interface for evaluating the response of the model to a batch of patterns.

Parameters
patternsthe inputs of the model
outputsthe predictions or response of the model to every pattern
stateintermediate results stored by eval which can be reused for derivative computation.

Implements shark::AbstractModel< RealVector, RealVector >.

◆ inputSize()

size_t shark::Softmax::inputSize ( ) const
inline

Definition at line 93 of file Softmax.h.

Referenced by setStructure().

◆ name()

std::string shark::Softmax::name ( ) const
inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 83 of file Softmax.h.

◆ numberOfParameters()

size_t shark::Softmax::numberOfParameters ( ) const
inlinevirtual

Return the number of parameters.

Reimplemented from shark::IParameterizable.

Definition at line 99 of file Softmax.h.

◆ outputSize()

size_t shark::Softmax::outputSize ( ) const
inline

Definition at line 96 of file Softmax.h.

◆ parameterVector()

RealVector shark::Softmax::parameterVector ( ) const
inlinevirtual

Return the parameter vector.

Reimplemented from shark::IParameterizable.

Definition at line 86 of file Softmax.h.

◆ read()

SHARK_EXPORT_SYMBOL void shark::Softmax::read ( InArchive archive)
virtual

From ISerializable, reads a model from an archive.

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Referenced by setStructure().

◆ setParameterVector()

void shark::Softmax::setParameterVector ( RealVector const &  newParameters)
inlinevirtual

Set the parameter vector.

Reimplemented from shark::IParameterizable.

Definition at line 89 of file Softmax.h.

References SIZE_CHECK.

◆ setStructure()

void shark::Softmax::setStructure ( std::size_t  inputSize)
inline

Definition at line 118 of file Softmax.h.

References inputSize(), read(), SHARK_EXPORT_SYMBOL, and write().

◆ weightedInputDerivative()

SHARK_EXPORT_SYMBOL void shark::Softmax::weightedInputDerivative ( BatchInputType const &  patterns,
RealMatrix const &  coefficients,
State const &  state,
BatchOutputType gradient 
) const

Referenced by createState().

◆ weightedParameterDerivative()

SHARK_EXPORT_SYMBOL void shark::Softmax::weightedParameterDerivative ( BatchInputType const &  pattern,
BatchOutputType const &  coefficients,
State const &  state,
RealVector &  derivative 
) const
virtual

calculates the weighted sum of derivatives w.r.t the parameters.

Parameters
patternthe patterns to evaluate
coefficientsthe coefficients which are used to calculate the weighted sum for every pattern
stateintermediate results stored by eval to speed up calculations of the derivatives
derivativethe calculated derivative as sum over all derivates of all patterns

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Referenced by createState().

◆ write()

SHARK_EXPORT_SYMBOL void shark::Softmax::write ( OutArchive archive) const
virtual

From ISerializable, writes a model to an archive.

Reimplemented from shark::AbstractModel< RealVector, RealVector >.

Referenced by setStructure().


The documentation for this class was generated from the following file: