Evidence for model selection of a regularization network/Gaussian process. More...
#include <shark/ObjectiveFunctions/NegativeGaussianProcessEvidence.h>
Inheritance diagram for shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >:
Public Types | |
| typedef LabeledData< InputType, LabelType > | DatasetType |
| typedef AbstractKernelFunction< InputType > | KernelType |
| Public Types inherited from shark::AbstractObjectiveFunction< RealVector, double > | |
| enum | Feature |
| List of features that are supported by an implementation. More... | |
| typedef RealVector | SearchPointType |
| typedef double | ResultType |
| typedef boost::mpl::if_< std::is_arithmetic< double >, SearchPointType, RealMatrix >::type | FirstOrderDerivative |
| typedef TypedFlags< Feature > | Features |
| This statement declares the member m_features. See Core/Flags.h for details. More... | |
| typedef TypedFeatureNotAvailableException< Feature > | FeatureNotAvailableException |
Public Member Functions | |
| NegativeGaussianProcessEvidence (DatasetType const &dataset, KernelType *kernel, bool unconstrained=false) | |
| std::string | name () const |
| From INameable: return the class name. More... | |
| std::size_t | numberOfVariables () const |
| Accesses the number of variables. More... | |
| double | eval (const RealVector ¶meters) const |
| double | evalDerivative (const RealVector ¶meters, FirstOrderDerivative &derivative) const |
| void | setThreshold (double d) |
| set threshold value for truncating partial derivatives More... | |
| void | setThresholds (RealVector &c) |
| set threshold values for truncating partial derivatives More... | |
| Public Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double > | |
| const Features & | features () const |
| virtual void | updateFeatures () |
| bool | hasValue () const |
| returns whether this function can calculate it's function value More... | |
| bool | hasFirstDerivative () const |
| returns whether this function can calculate the first derivative More... | |
| bool | hasSecondDerivative () const |
| returns whether this function can calculate the second derivative More... | |
| bool | canProposeStartingPoint () const |
| returns whether this function can propose a starting point. More... | |
| bool | isConstrained () const |
| returns whether this function can return More... | |
| bool | hasConstraintHandler () const |
| returns whether this function can return More... | |
| bool | canProvideClosestFeasible () const |
| Returns whether this function can calculate thee closest feasible to an infeasible point. More... | |
| bool | isThreadSafe () const |
| Returns true, when the function can be usd in parallel threads. More... | |
| bool | isNoisy () const |
| Returns true, when the function can be usd in parallel threads. More... | |
| AbstractObjectiveFunction () | |
| Default ctor. More... | |
| virtual | ~AbstractObjectiveFunction () |
| Virtual destructor. More... | |
| virtual void | init () |
| void | setRng (random::rng_type *rng) |
| Sets the Rng used by the objective function. More... | |
| virtual bool | hasScalableDimensionality () const |
| virtual void | setNumberOfVariables (std::size_t numberOfVariables) |
| Adjusts the number of variables if the function is scalable. More... | |
| virtual std::size_t | numberOfObjectives () const |
| virtual bool | hasScalableObjectives () const |
| virtual void | setNumberOfObjectives (std::size_t numberOfObjectives) |
| Adjusts the number of objectives if the function is scalable. More... | |
| std::size_t | evaluationCounter () const |
| Accesses the evaluation counter of the function. More... | |
| AbstractConstraintHandler< SearchPointType > const & | getConstraintHandler () const |
| Returns the constraint handler of the function if it has one. More... | |
| virtual bool | isFeasible (const SearchPointType &input) const |
| Tests whether a point in SearchSpace is feasible, e.g., whether the constraints are fulfilled. More... | |
| virtual void | closestFeasible (SearchPointType &input) const |
| If supported, the supplied point is repaired such that it satisfies all of the function's constraints. More... | |
| virtual SearchPointType | proposeStartingPoint () const |
| Proposes a starting point in the feasible search space of the function. More... | |
| ResultType | operator() (SearchPointType const &input) const |
| Evaluates the function. Useful together with STL-Algorithms like std::transform. More... | |
| virtual ResultType | evalDerivative (SearchPointType const &input, SecondOrderDerivative &derivative) const |
| Evaluates the objective function and calculates its gradient. More... | |
| Public Member Functions inherited from shark::INameable | |
| virtual | ~INameable () |
Additional Inherited Members | |
| Protected Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double > | |
| void | announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler) |
| helper function which is called to announce the presence of an constraint handler. More... | |
| Protected Attributes inherited from shark::AbstractObjectiveFunction< RealVector, double > | |
| Features | m_features |
| std::size_t | m_evaluationCounter |
| Evaluation counter, default value: 0. More... | |
| AbstractConstraintHandler< SearchPointType > const * | m_constraintHandler |
| random::rng_type * | mep_rng |
Detailed Description
template<class InputType = RealVector, class OutputType = RealVector, class LabelType = RealVector>
class shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >
Evidence for model selection of a regularization network/Gaussian process.
Let \(M\) denote the (kernel Gram) covariance matrix and \(t\) the corresponding label vector. For the evidence we have:
\[ E = 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi)] \]
The evidence is also known as marginal (log)likelihood. For details, please see:
C.E. Rasmussen & C.K.I. Williams, Gaussian Processes for Machine Learning, section 5.4, MIT Press, 2006
C.M. Bishop, Pattern Recognition and Machine Learning, section 6.4.3, Springer, 2006
The regularization parameter can be encoded in different ways. The exponential encoding is the proper choice for unconstraint optimization. Be careful not to mix up different encodings between trainer and evidence.
Definition at line 66 of file NegativeGaussianProcessEvidence.h.
Member Typedef Documentation
◆ DatasetType
| typedef LabeledData<InputType,LabelType> shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::DatasetType |
Definition at line 69 of file NegativeGaussianProcessEvidence.h.
◆ KernelType
| typedef AbstractKernelFunction<InputType> shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::KernelType |
Definition at line 70 of file NegativeGaussianProcessEvidence.h.
Constructor & Destructor Documentation
◆ NegativeGaussianProcessEvidence()
|
inline |
- Parameters
-
dataset training data for the Gaussian process kernel pointer to external kernel function unconstrained exponential encoding of regularization parameter for unconstraint optimization
Definition at line 75 of file NegativeGaussianProcessEvidence.h.
References shark::AbstractObjectiveFunction< RealVector, double >::HAS_FIRST_DERIVATIVE, shark::AbstractKernelFunction< InputTypeT >::hasFirstParameterDerivative(), shark::AbstractObjectiveFunction< RealVector, double >::m_features, and shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::setThreshold().
Member Function Documentation
◆ eval()
|
inlinevirtual |
Let \(M\) denote the (kernel Gram) covariance matrix and \(t\) the label vector. For the evidence we have:
\[ E= 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi) ] \]
Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.
Definition at line 97 of file NegativeGaussianProcessEvidence.h.
References shark::AbstractObjectiveFunction< RealVector, double >::m_evaluationCounter, shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::IParameterizable< VectorType >::numberOfParameters(), shark::IParameterizable< VectorType >::setParameterVector(), and SHARK_ASSERT.
◆ evalDerivative()
|
inlinevirtual |
Let \(M\) denote the regularized (kernel Gram) covariance matrix. For the evidence we have:
\[ E = 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi) ] \]
For a kernel parameter \(p\) and \(C = \beta^{-1}\) we get the derivatives:
\[ dE/dC = 1/2 \cdot [ -tr(M^{-1}) + (M^{-1} t)^2 ] \]
\[ dE/dp = 1/2 \cdot [ -tr(M^{-1} dM/dp) + t^T (M^{-1} dM/dp M^{-1}) t ] \]
Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.
Definition at line 143 of file NegativeGaussianProcessEvidence.h.
References shark::AbstractObjectiveFunction< RealVector, double >::m_evaluationCounter, shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::IParameterizable< VectorType >::numberOfParameters(), shark::IParameterizable< VectorType >::setParameterVector(), and SHARK_ASSERT.
◆ name()
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 88 of file NegativeGaussianProcessEvidence.h.
◆ numberOfVariables()
|
inlinevirtual |
Accesses the number of variables.
Implements shark::AbstractObjectiveFunction< RealVector, double >.
Definition at line 91 of file NegativeGaussianProcessEvidence.h.
References shark::IParameterizable< VectorType >::numberOfParameters().
◆ setThreshold()
|
inline |
set threshold value for truncating partial derivatives
Definition at line 218 of file NegativeGaussianProcessEvidence.h.
References shark::IParameterizable< VectorType >::numberOfParameters().
Referenced by shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::NegativeGaussianProcessEvidence().
◆ setThresholds()
|
inline |
set threshold values for truncating partial derivatives
Definition at line 223 of file NegativeGaussianProcessEvidence.h.
References SHARK_ASSERT.
The documentation for this class was generated from the following file:
- include/shark/ObjectiveFunctions/NegativeGaussianProcessEvidence.h