Evidence for model selection of a regularization network/Gaussian process. More...
#include <shark/ObjectiveFunctions/NegativeGaussianProcessEvidence.h>
Inheritance diagram for shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >:
Collaboration diagram for shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >:
Public Types | |
| typedef base_type::DatasetType | DatasetType |
| typedef AbstractKernelFunction < InputType > | KernelType |
| Public Types inherited from shark::SupervisedObjectiveFunction< InputType, LabelType > | |
| typedef LabeledData< InputType, LabelType > | DatasetType |
| Public Types inherited from shark::AbstractObjectiveFunction< VectorSpace< double >, double > | |
| enum | Feature |
| List of features that are supported by an implementation. More... | |
| typedef VectorSpace< double > ::PointType | SearchPointType |
| typedef double | ResultType |
| typedef SearchPointType | FirstOrderDerivative |
| typedef TypedSecondOrderDerivative < SearchPointType, RealMatrix > | SecondOrderDerivative |
| typedef TypedFlags< Feature > | Features |
| This statement declares the member m_features. See Core/Flags.h for details. More... | |
| typedef TypedFeatureNotAvailableException < Feature > | FeatureNotAvailableException |
Public Member Functions | |
| NegativeGaussianProcessEvidence (KernelType *kernel, bool unconstrained=false) | |
| NegativeGaussianProcessEvidence (DatasetType const &dataset, KernelType *kernel, bool unconstrained=false) | |
| std::string | name () const |
| From INameable: return the class name. More... | |
| void | setDataset (DatasetType const &dataset) |
| inherited from SupervisedObjectiveFunction More... | |
| std::size_t | numberOfVariables () const |
| double | eval (const RealVector ¶meters) const |
| double | evalDerivative (const RealVector ¶meters, typename base_type::FirstOrderDerivative &derivative) const |
| void | setThreshold (double d) |
| set threshold value for truncating partial derivatives More... | |
| void | setThresholds (RealVector &c) |
| set threshold values for truncating partial derivatives More... | |
| Public Member Functions inherited from shark::SupervisedObjectiveFunction< InputType, LabelType > | |
| SupervisedObjectiveFunction () | |
| virtual | ~SupervisedObjectiveFunction () |
| Public Member Functions inherited from shark::AbstractObjectiveFunction< VectorSpace< double >, double > | |
| const Features & | features () const |
| virtual void | updateFeatures () |
| bool | hasValue () const |
| returns whether this function can calculate it's function value More... | |
| bool | hasFirstDerivative () const |
| returns whether this function can calculate the first derivative More... | |
| bool | hasSecondDerivative () const |
| returns whether this function can calculate the second derivative More... | |
| bool | canProposeStartingPoint () const |
| returns whether this function can propose a starting point. More... | |
| bool | isConstrained () const |
| returns whether this function can return More... | |
| bool | hasConstraintHandler () const |
| returns whether this function can return More... | |
| bool | canProvideClosestFeasible () const |
| Returns whether this function can calculate thee closest feasible to an infeasible point. More... | |
| bool | isThreadSafe () const |
| Returns true, when the function can be usd in parallel threads. More... | |
| AbstractObjectiveFunction () | |
| Default ctor. More... | |
| virtual | ~AbstractObjectiveFunction () |
| Virtual destructor. More... | |
| virtual void | configure (const PropertyTree &node) |
| Configures the component given a property tree. More... | |
| virtual void | init () |
| virtual std::size_t | numberOfObjectives () const |
| virtual bool | hasScalableObjectives () const |
| virtual void | setNumberOfObjectives (std::size_t numberOfObjectives) |
| Adjusts the number of variables if the function is scalable. More... | |
| std::size_t | evaluationCounter () const |
| Accesses the evaluation counter of the function. More... | |
| AbstractConstraintHandler < SearchPointType > const & | getConstraintHandler () const |
| Returns the constraint handler of the function if it has one. More... | |
| virtual bool | isFeasible (const SearchPointType &input) const |
| Tests whether a point in SearchSpace is feasible, e.g., whether the constraints are fulfilled. More... | |
| virtual void | closestFeasible (SearchPointType &input) const |
| If supported, the supplied point is repaired such that it satisfies all of the function's constraints. More... | |
| virtual void | proposeStartingPoint (SearchPointType &startingPoint) const |
| Proposes a starting point in the feasible search space of the function. More... | |
| virtual ResultType | eval (const SearchPointType &input) const |
| Evaluates the objective function for the supplied argument. More... | |
| ResultType | operator() (const SearchPointType &input) const |
| Evaluates the function. Useful together with STL-Algorithms like std::transform. More... | |
| virtual ResultType | evalDerivative (const SearchPointType &input, FirstOrderDerivative &derivative) const |
| Evaluates the objective function and calculates its gradient. More... | |
| virtual ResultType | evalDerivative (const SearchPointType &input, SecondOrderDerivative &derivative) const |
| Evaluates the objective function and calculates its gradient. More... | |
| Public Member Functions inherited from shark::IConfigurable | |
| virtual | ~IConfigurable () |
| Virtual d'tor. More... | |
| Public Member Functions inherited from shark::INameable | |
| virtual | ~INameable () |
Additional Inherited Members | |
| Protected Member Functions inherited from shark::AbstractObjectiveFunction< VectorSpace< double >, double > | |
| void | announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler) |
| helper function which is called to announce the presence of an constraint handler. More... | |
| Protected Attributes inherited from shark::AbstractObjectiveFunction< VectorSpace< double >, double > | |
| Features | m_features |
| std::size_t | m_evaluationCounter |
| Evaluation counter, default value: 0. More... | |
| AbstractConstraintHandler < SearchPointType > const * | m_constraintHandler |
Detailed Description
template<class InputType = RealVector, class OutputType = RealVector, class LabelType = RealVector>
class shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >
Evidence for model selection of a regularization network/Gaussian process.
Let \(M\) denote the (kernel Gram) covariance matrix and \(t\) the corresponding label vector. For the evidence we have:
\[ E = 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi)] \]
The evidence is also known as marginal (log)likelihood. For details, please see:
C.E. Rasmussen & C.K.I. Williams, Gaussian Processes for Machine Learning, section 5.4, MIT Press, 2006
C.M. Bishop, Pattern Recognition and Machine Learning, section 6.4.3, Springer, 2006
The regularization parameter can be encoded in different ways. The exponential encoding is the proper choice for unconstraint optimization. Be careful not to mix up different encodings between trainer and evidence.
Definition at line 58 of file NegativeGaussianProcessEvidence.h.
Member Typedef Documentation
| typedef base_type::DatasetType shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::DatasetType |
Definition at line 63 of file NegativeGaussianProcessEvidence.h.
| typedef AbstractKernelFunction<InputType> shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::KernelType |
Definition at line 64 of file NegativeGaussianProcessEvidence.h.
Constructor & Destructor Documentation
|
inline |
- Parameters
-
kernel,: pointer to external kernel function unconstrained,: exponential encoding of regularization parameter for unconstraint optimization
Definition at line 68 of file NegativeGaussianProcessEvidence.h.
References shark::AbstractObjectiveFunction< VectorSpace< double >, double >::HAS_FIRST_DERIVATIVE, shark::AbstractKernelFunction< InputTypeT >::hasFirstParameterDerivative(), shark::AbstractObjectiveFunction< VectorSpace< double >, double >::m_features, and shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::setThreshold().
|
inline |
- Parameters
-
dataset,: training data for the Gaussian process kernel,: pointer to external kernel function unconstrained,: exponential encoding of regularization parameter for unconstraint optimization
Definition at line 78 of file NegativeGaussianProcessEvidence.h.
References shark::AbstractObjectiveFunction< VectorSpace< double >, double >::HAS_FIRST_DERIVATIVE, shark::AbstractKernelFunction< InputTypeT >::hasFirstParameterDerivative(), shark::AbstractObjectiveFunction< VectorSpace< double >, double >::m_features, and shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::setThreshold().
Member Function Documentation
|
inline |
Let \(M\) denote the (kernel Gram) covariance matrix and \(t\) the label vector. For the evidence we have:
\[ E= 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi) ] \]
Definition at line 104 of file NegativeGaussianProcessEvidence.h.
References shark::calculateRegularizedKernelMatrix(), shark::choleskyDecomposition(), shark::blas::column(), shark::Data< Type >::elements(), shark::init(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::AbstractObjectiveFunction< VectorSpace< double >, double >::m_evaluationCounter, shark::blas::norm_sqr(), shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::IParameterizable::numberOfParameters(), shark::IParameterizable::setParameterVector(), SHARK_ASSERT, and shark::trace().
|
inline |
Let \(M\) denote the regularized (kernel Gram) covariance matrix. For the evidence we have:
\[ E = 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi) ] \]
For a kernel parameter \(p\) and \(C = \beta^{-1}\) we get the derivatives:
\[ dE/dC = 1/2 \cdot [ -tr(M^{-1}) + (M^{-1} t)^2 ] \]
\[ dE/dp = 1/2 \cdot [ -tr(M^{-1} dM/dp) + t^T (M^{-1} dM/dp M^{-1}) t ] \]
Definition at line 155 of file NegativeGaussianProcessEvidence.h.
References shark::calculateKernelMatrixParameterDerivative(), shark::calculateRegularizedKernelMatrix(), shark::choleskyDecomposition(), shark::blas::column(), shark::Data< Type >::elements(), shark::fast_prod(), shark::init(), shark::blas::inner_prod(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::AbstractObjectiveFunction< VectorSpace< double >, double >::m_evaluationCounter, shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::IParameterizable::numberOfParameters(), shark::blas::outer_prod(), shark::blas::vector< T, A >::resize(), shark::IParameterizable::setParameterVector(), SHARK_ASSERT, shark::blas::vector< T, A >::size(), and shark::trace().
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 90 of file NegativeGaussianProcessEvidence.h.
|
inline |
Definition at line 98 of file NegativeGaussianProcessEvidence.h.
References shark::IParameterizable::numberOfParameters().
|
inlinevirtual |
inherited from SupervisedObjectiveFunction
Implements shark::SupervisedObjectiveFunction< InputType, LabelType >.
Definition at line 94 of file NegativeGaussianProcessEvidence.h.
|
inline |
set threshold value for truncating partial derivatives
Definition at line 238 of file NegativeGaussianProcessEvidence.h.
References shark::IParameterizable::numberOfParameters().
Referenced by shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::NegativeGaussianProcessEvidence().
|
inline |
set threshold values for truncating partial derivatives
Definition at line 243 of file NegativeGaussianProcessEvidence.h.
References SHARK_ASSERT.
The documentation for this class was generated from the following file:
- include/shark/ObjectiveFunctions/NegativeGaussianProcessEvidence.h