shark::MarkovPole< HiddenNeuron, OutputNeuron > Class Template Reference

#include <shark/ObjectiveFunctions/Benchmarks/MarkovPole.h>

+ Inheritance diagram for shark::MarkovPole< HiddenNeuron, OutputNeuron >:

Public Member Functions

 MarkovPole (bool single_pole, std::size_t hidden, bool shortcuts, bool bias, bool normalize=true, std::size_t max_pole_evaluations=100000)
 
 ~MarkovPole ()
 
std::string name ()
 
std::size_t numberOfVariables () const
 Returns degrees of freedom. More...
 
SearchPointType proposeStartingPoint () const
 Always proposes to start in a zero vector with appropriate degrees of freedom. More...
 
ResultType eval (const SearchPointType &input) const
 Evaluates weight vector on fitness function. More...
 
- Public Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double >
const Featuresfeatures () const
 
virtual void updateFeatures ()
 
bool hasValue () const
 returns whether this function can calculate it's function value More...
 
bool hasFirstDerivative () const
 returns whether this function can calculate the first derivative More...
 
bool hasSecondDerivative () const
 returns whether this function can calculate the second derivative More...
 
bool canProposeStartingPoint () const
 returns whether this function can propose a starting point. More...
 
bool isConstrained () const
 returns whether this function can return More...
 
bool hasConstraintHandler () const
 returns whether this function can return More...
 
bool canProvideClosestFeasible () const
 Returns whether this function can calculate thee closest feasible to an infeasible point. More...
 
bool isThreadSafe () const
 Returns true, when the function can be usd in parallel threads. More...
 
bool isNoisy () const
 Returns true, when the function can be usd in parallel threads. More...
 
 AbstractObjectiveFunction ()
 Default ctor. More...
 
virtual ~AbstractObjectiveFunction ()
 Virtual destructor. More...
 
virtual void init ()
 
void setRng (random::rng_type *rng)
 Sets the Rng used by the objective function. More...
 
virtual bool hasScalableDimensionality () const
 
virtual void setNumberOfVariables (std::size_t numberOfVariables)
 Adjusts the number of variables if the function is scalable. More...
 
virtual std::size_t numberOfObjectives () const
 
virtual bool hasScalableObjectives () const
 
virtual void setNumberOfObjectives (std::size_t numberOfObjectives)
 Adjusts the number of objectives if the function is scalable. More...
 
std::size_t evaluationCounter () const
 Accesses the evaluation counter of the function. More...
 
AbstractConstraintHandler< SearchPointType > const & getConstraintHandler () const
 Returns the constraint handler of the function if it has one. More...
 
virtual bool isFeasible (const SearchPointType &input) const
 Tests whether a point in SearchSpace is feasible, e.g., whether the constraints are fulfilled. More...
 
virtual void closestFeasible (SearchPointType &input) const
 If supported, the supplied point is repaired such that it satisfies all of the function's constraints. More...
 
ResultType operator() (SearchPointType const &input) const
 Evaluates the function. Useful together with STL-Algorithms like std::transform. More...
 
virtual ResultType evalDerivative (SearchPointType const &input, FirstOrderDerivative &derivative) const
 Evaluates the objective function and calculates its gradient. More...
 
virtual ResultType evalDerivative (SearchPointType const &input, SecondOrderDerivative &derivative) const
 Evaluates the objective function and calculates its gradient. More...
 
- Public Member Functions inherited from shark::INameable
virtual ~INameable ()
 
virtual std::string name () const
 returns the name of the object More...
 

Additional Inherited Members

- Public Types inherited from shark::AbstractObjectiveFunction< RealVector, double >
enum  Feature
 List of features that are supported by an implementation. More...
 
typedef RealVector SearchPointType
 
typedef double ResultType
 
typedef boost::mpl::if_< std::is_arithmetic< double >, SearchPointType, RealMatrix >::type FirstOrderDerivative
 
typedef TypedFlags< FeatureFeatures
 This statement declares the member m_features. See Core/Flags.h for details. More...
 
typedef TypedFeatureNotAvailableException< FeatureFeatureNotAvailableException
 
- Protected Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double >
void announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler)
 helper function which is called to announce the presence of an constraint handler. More...
 
- Protected Attributes inherited from shark::AbstractObjectiveFunction< RealVector, double >
Features m_features
 
std::size_t m_evaluationCounter
 Evaluation counter, default value: 0. More...
 
AbstractConstraintHandler< SearchPointType > const * m_constraintHandler
 
random::rng_type * mep_rng
 

Detailed Description

template<class HiddenNeuron, class OutputNeuron>
class shark::MarkovPole< HiddenNeuron, OutputNeuron >

Uses templates to allow changing the neural network activation function since FFNet uses templates. The FastSigmoidNeuron is recommended, as it gives better results overall. If errors are encountered using a specific neuron, one can try without normalization, as it fixes it in the single pole LogisticNeuron case at least. Class for balancing one or two poles on a cart using a fitness function that decreases the longer the pole(s) balance(s). Based on code written by Verena Heidrich-Meisner for the paper

V. Heidrich-Meisner and C. Igel. Neuroevolution strategies for episodic reinforcement learn-ing. Journal of Algorithms, 64(4):152–168, 2009.

Definition at line 64 of file MarkovPole.h.

Constructor & Destructor Documentation

◆ MarkovPole()

template<class HiddenNeuron , class OutputNeuron >
shark::MarkovPole< HiddenNeuron, OutputNeuron >::MarkovPole ( bool  single_pole,
std::size_t  hidden,
bool  shortcuts,
bool  bias,
bool  normalize = true,
std::size_t  max_pole_evaluations = 100000 
)
inline
Parameters
single_poleIndicates whether the cast has a single pole (true) or two poles (false)
hiddenNumber of hidden neurons in underlying neural network
shortcutsWhether to use shortcuts in neural network
biasWhether to use bias in neural network
normalizeWhether to normalize input before use in neural network
max_pole_evaluationsBalance goal of the function, i.e. number of steps that pole should be able to balance without failure

Definition at line 72 of file MarkovPole.h.

References shark::AbstractObjectiveFunction< RealVector, double >::CAN_PROPOSE_STARTING_POINT, shark::AbstractObjectiveFunction< RealVector, double >::m_evaluationCounter, and shark::AbstractObjectiveFunction< RealVector, double >::m_features.

◆ ~MarkovPole()

template<class HiddenNeuron , class OutputNeuron >
shark::MarkovPole< HiddenNeuron, OutputNeuron >::~MarkovPole ( )
inline

Definition at line 121 of file MarkovPole.h.

Member Function Documentation

◆ eval()

template<class HiddenNeuron , class OutputNeuron >
ResultType shark::MarkovPole< HiddenNeuron, OutputNeuron >::eval ( const SearchPointType input) const
inlinevirtual

◆ name()

template<class HiddenNeuron , class OutputNeuron >
std::string shark::MarkovPole< HiddenNeuron, OutputNeuron >::name ( )
inline

Definition at line 126 of file MarkovPole.h.

◆ numberOfVariables()

template<class HiddenNeuron , class OutputNeuron >
std::size_t shark::MarkovPole< HiddenNeuron, OutputNeuron >::numberOfVariables ( ) const
inlinevirtual

Returns degrees of freedom.

Implements shark::AbstractObjectiveFunction< RealVector, double >.

Definition at line 131 of file MarkovPole.h.

◆ proposeStartingPoint()

template<class HiddenNeuron , class OutputNeuron >
SearchPointType shark::MarkovPole< HiddenNeuron, OutputNeuron >::proposeStartingPoint ( ) const
inlinevirtual

Always proposes to start in a zero vector with appropriate degrees of freedom.

Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.

Definition at line 136 of file MarkovPole.h.


The documentation for this class was generated from the following file: