Class for balancing two poles on a cart using a fitness function that punishes oscillating, i.e. quickly moving the cart back and forth to balance the poles. Based on code written by Verena Heidrich-Meisner for the paper. More...

#include <shark/ObjectiveFunctions/Benchmarks/GruauPole.h>

Inheritance diagram for shark::GruauPole:

Public Member Functions
	GruauPole (std::size_t hidden, bool bias, RecurrentStructure::SigmoidType sigmoidType=RecurrentStructure::FastSigmoid, bool normalize=true, std::size_t max_pole_evaluations=1000)

	~GruauPole ()

std::string	name ()

std::size_t	numberOfVariables () const
	Returns degrees of freedom. More...

SearchPointType	proposeStartingPoint () const
	Always proposes to start in a zero vector with appropriate degrees of freedom. More...

ResultType	eval (const SearchPointType &input) const
	Evaluates weight vector on special fitness function from Gruau paper. More...

ResultType	gruauFit (const SearchPointType &input) const
	Evaluates weight vector on special fitness function from Gruau paper. More...

ResultType	balanceFit (const SearchPointType &input, std::size_t maxEvals=100000)
	Evaluates weight vector on normal balancing function. More...

ResultType	generalFit (const SearchPointType &input, std::size_t maxEvals=1000)
	Evaluates weight vector on normal balancing function using 256 different starting positions. More...

Public Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double >
const Features &	features () const

virtual void	updateFeatures ()

bool	hasValue () const
	returns whether this function can calculate it's function value More...

bool	hasFirstDerivative () const
	returns whether this function can calculate the first derivative More...

bool	hasSecondDerivative () const
	returns whether this function can calculate the second derivative More...

bool	canProposeStartingPoint () const
	returns whether this function can propose a starting point. More...

bool	isConstrained () const
	returns whether this function can return More...

bool	hasConstraintHandler () const
	returns whether this function can return More...

bool	canProvideClosestFeasible () const
	Returns whether this function can calculate thee closest feasible to an infeasible point. More...

bool	isThreadSafe () const
	Returns true, when the function can be usd in parallel threads. More...

bool	isNoisy () const
	Returns true, when the function can be usd in parallel threads. More...

	AbstractObjectiveFunction ()
	Default ctor. More...

virtual	~AbstractObjectiveFunction ()
	Virtual destructor. More...

virtual void	init ()

void	setRng (random::rng_type *rng)
	Sets the Rng used by the objective function. More...

virtual bool	hasScalableDimensionality () const

virtual void	setNumberOfVariables (std::size_t numberOfVariables)
	Adjusts the number of variables if the function is scalable. More...

virtual std::size_t	numberOfObjectives () const

virtual bool	hasScalableObjectives () const

virtual void	setNumberOfObjectives (std::size_t numberOfObjectives)
	Adjusts the number of objectives if the function is scalable. More...

std::size_t	evaluationCounter () const
	Accesses the evaluation counter of the function. More...

AbstractConstraintHandler< SearchPointType > const &	getConstraintHandler () const
	Returns the constraint handler of the function if it has one. More...

virtual bool	isFeasible (const SearchPointType &input) const
	Tests whether a point in SearchSpace is feasible, e.g., whether the constraints are fulfilled. More...

virtual void	closestFeasible (SearchPointType &input) const
	If supported, the supplied point is repaired such that it satisfies all of the function's constraints. More...

ResultType	operator() (SearchPointType const &input) const
	Evaluates the function. Useful together with STL-Algorithms like std::transform. More...

virtual ResultType	evalDerivative (SearchPointType const &input, FirstOrderDerivative &derivative) const
	Evaluates the objective function and calculates its gradient. More...

virtual ResultType	evalDerivative (SearchPointType const &input, SecondOrderDerivative &derivative) const
	Evaluates the objective function and calculates its gradient. More...

Public Member Functions inherited from shark::INameable
virtual	~INameable ()

virtual std::string	name () const
	returns the name of the object More...

Additional Inherited Members
Public Types inherited from shark::AbstractObjectiveFunction< RealVector, double >
enum	Feature
	List of features that are supported by an implementation. More...

typedef RealVector	SearchPointType

typedef double	ResultType

typedef boost::mpl::if_< std::is_arithmetic< double >, SearchPointType, RealMatrix >::type	FirstOrderDerivative

typedef TypedFlags< Feature >	Features
	This statement declares the member m_features. See Core/Flags.h for details. More...

typedef TypedFeatureNotAvailableException< Feature >	FeatureNotAvailableException

Protected Member Functions inherited from shark::AbstractObjectiveFunction< RealVector, double >
void	announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler)
	helper function which is called to announce the presence of an constraint handler. More...

Protected Attributes inherited from shark::AbstractObjectiveFunction< RealVector, double >
Features	m_features

std::size_t	m_evaluationCounter
	Evaluation counter, default value: 0. More...

AbstractConstraintHandler< SearchPointType > const *	m_constraintHandler

random::rng_type *	mep_rng

Detailed Description

Class for balancing two poles on a cart using a fitness function that punishes oscillating, i.e. quickly moving the cart back and forth to balance the poles. Based on code written by Verena Heidrich-Meisner for the paper.

V. Heidrich-Meisner and C. Igel. Neuroevolution strategies for episodic reinforcement learn-ing. Journal of Algorithms, 64(4):152–168, 2009.

Special fitness function from the paper

F. Gruau, D. Whitley, L. Pyeatt, A comparison between cellular encoding and direct encoding for genetic neural networks, in: J.R. Koza, D.E. Goldberg, D.B. Fogel, R.L. Riol (Eds.), Genetic Programming 1996: Proceedings of the First Annual Conference, MIT Press, 1996, pp. 81–89.

Definition at line 73 of file GruauPole.h.

Constructor & Destructor Documentation

◆ GruauPole()

shark::GruauPole::GruauPole	(	std::size_t	hidden,
		bool	bias,
		RecurrentStructure::SigmoidType	sigmoidType = `RecurrentStructure::FastSigmoid`,
		bool	normalize = `true`,
		std::size_t	max_pole_evaluations = `1000`
	)

inline

Parameters

hidden	Number of hidden neurons in underlying neural network
bias	Whether to use bias in neural network
sigmoidType	Activation sigmoid function for neural network
normalize	Whether to normalize input before use in neural network
max_pole_evaluations	Balance goal of the function, i.e. number of steps that pole should be able to balance without failure

Definition at line 81 of file GruauPole.h.

References shark::AbstractObjectiveFunction< RealVector, double >::CAN_PROPOSE_STARTING_POINT, shark::RecurrentStructure::Linear, shark::AbstractObjectiveFunction< RealVector, double >::m_evaluationCounter, shark::AbstractObjectiveFunction< RealVector, double >::m_features, and shark::OnlineRNNet::numberOfParameters().

◆ ~GruauPole()

shark::GruauPole::~GruauPole ( )

inline

Definition at line 127 of file GruauPole.h.

Member Function Documentation

◆ balanceFit()

ResultType shark::GruauPole::balanceFit	(	const SearchPointType &	input,
		std::size_t	maxEvals = `100000`
	)

inline

Evaluates weight vector on normal balancing function.

Parameters

input	Vector to be evaluated.
maxEvals	Balance goal of the function, i.e. number of steps that pole should be able to balance without failure

Returns: Fitness of vector

Definition at line 212 of file GruauPole.h.

References shark::OnlineRNNet::eval(), shark::DoublePole::failure(), shark::DoublePole::getState(), shark::DoublePole::init(), shark::DoublePole::move(), remora::row(), shark::OnlineRNNet::setParameterVector(), and SIZE_CHECK.

◆ eval()

ResultType shark::GruauPole::eval ( const SearchPointType & input ) const

inlinevirtual

Evaluates weight vector on special fitness function from Gruau paper.

Parameters

input Vector to be evaluated.

Returns: Fitness of vector

Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.

Definition at line 154 of file GruauPole.h.

References gruauFit(), shark::AbstractObjectiveFunction< RealVector, double >::m_evaluationCounter, and SIZE_CHECK.

Referenced by generalFit().

◆ generalFit()

ResultType shark::GruauPole::generalFit	(	const SearchPointType &	input,
		std::size_t	maxEvals = `1000`
	)

inline

Evaluates weight vector on normal balancing function using 256 different starting positions.

Parameters

input	Vector to be evaluated.
maxEvals	Balance goal of the function, i.e. number of steps that pole should be able to balance without failure

Returns: Number of trials (out of 256) that did not fail

Definition at line 244 of file GruauPole.h.

References shark::OnlineRNNet::eval(), eval(), shark::DoublePole::failure(), shark::RecurrentStructure::FastSigmoid, shark::DoublePole::getState(), shark::DoublePole::init(), shark::RecurrentStructure::Logistic, shark::DoublePole::move(), remora::row(), shark::OnlineRNNet::setParameterVector(), SIZE_CHECK, and shark::RecurrentStructure::Tanh.

◆ gruauFit()

ResultType shark::GruauPole::gruauFit ( const SearchPointType & input ) const

inline

Evaluates weight vector on special fitness function from Gruau paper.

Parameters

input Vector to be evaluated.

Returns: Fitness of vector

Definition at line 163 of file GruauPole.h.

References shark::OnlineRNNet::eval(), shark::DoublePole::failure(), shark::DoublePole::getJiggle(), shark::DoublePole::getState(), shark::DoublePole::init(), shark::DoublePole::move(), remora::row(), shark::OnlineRNNet::setParameterVector(), and SIZE_CHECK.

Referenced by eval().

◆ name()

std::string shark::GruauPole::name ( )

inline

Definition at line 133 of file GruauPole.h.

◆ numberOfVariables()

std::size_t shark::GruauPole::numberOfVariables ( ) const

inlinevirtual

Returns degrees of freedom.

Implements shark::AbstractObjectiveFunction< RealVector, double >.

Definition at line 138 of file GruauPole.h.

◆ proposeStartingPoint()

SearchPointType shark::GruauPole::proposeStartingPoint ( ) const

inlinevirtual

Always proposes to start in a zero vector with appropriate degrees of freedom.

Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.

Definition at line 143 of file GruauPole.h.

The documentation for this class was generated from the following file:

include/shark/ObjectiveFunctions/Benchmarks/GruauPole.h

Public Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ GruauPole()

◆ ~GruauPole()

Member Function Documentation

◆ balanceFit()

◆ eval()

◆ generalFit()

◆ gruauFit()

◆ name()

◆ numberOfVariables()

◆ proposeStartingPoint()