Tutorials

Contents:

• Tutorials
  • Very first steps
  • Neural Networks
  • Overarching concepts
  • Data Handling
  • Specific Machine-Learning Algorithms
  • Tools
  • For Shark developers

This page lists all tutorials available for the Shark machine learning library. Together with a comprehensive set of example and test programs, they are excellent starting points towards developing and evaluating machine learning software using Shark.

If you are familiar with the Shark architecture and/or just want a concise overview over usage patterns and interfaces, see the quick references.

Also see the guide to the different documentation pages available here.

Very first steps

We first show how to set up either a traditional Makefile or a CMake file for your application program. Then we move on to a simple Hello-World example of what linear binary classification can look like in Shark. The third tutorial illustrates the model-error-optimizer trias often encountered in Shark through a simple regression task.

• Your Shark Programs
• Hello World
• General Optimization Tasks
• Deciding When to Stop

Neural Networks

A very important class of machine-learning models are Neural Networks. This section discusses the creation and training of multi-layer neural networks

• Training Feed-Forward Networks
• Other Activation Functions
• Autoencoders

Overarching concepts

Many generic concepts that Shark implements span across the whole library or are relevant in many different application scenarios. We collect a number of such generic tutorials here which explain these concepts in detail. We believe that these are useful if you want to thoroughly familiarize yourself with Shark. If you are looking for a quick introduction on how to set up a specific algorithm, take a look at the component specific tutorials further down the page.

Before we can introduce the main interface classes of shark, we need to define more properly a few basic terms as well as the design goals.

• Design Goals
• Optimizers and Trainers
• Shark Conventions for Derivatives
• Data Batches

The main interfaces, or concepts in shark are the base classes from which nearly all components are derived. These tutorials are meant as a specification of the interfaces as well as the behavior and are written in a more formal language.

• Models
• Loss and Cost Functions
• Optimizers
• Objective Functions
• Stopping Criteria
• Trainers
• Kernels
• Writing Kernel Functions
• Writing Objective Functions

Data Handling

Since many machine learning algorithms work on real-world datasets, we extensively cover Shark’s Data class as well as common operations on them:

• Data Containers
• Label Formats
• Importing Data
• Creating and Using Subsets of Data
• Normalization of Input Data

Specific Machine-Learning Algorithms

Here come tutorials for some selected algorithms implemented in Shark. It must be said that this is only the tip of the iceberg, many more machine learning algorithms and tools are provided by the library.

Let’s start with some classical methods:

• Principal Component Analysis
• Nearest Neighbor Classification
• Linear Discriminant Analysis
• Linear Regression
• LASSO Regression
• K-Means Clustering

Neural networks and deep learning:

• The RBM Module
• Training Binary Restricted Boltzmann Machines

Tree-based algorithms:

• Random Forest

Kernel methods – support vector machine training and model selection:

• Support Vector Machines: First Steps
• Support Vector Machines: Model Selection Using Cross-Validation and Grid-Search
• Support Vector Machines: Likelihood-based Model Selection
• Linear Kernel Combinations (and a bit of MKL)
• Linear Support Vector Machines
• Kernel Target Alignment
• Kernelized Budgeted Stochastic Gradient Descent

Direct search methods – the covariance matrix adaptation evolution strategy:

• Direct Search Algorithms
• Covariance Matrix Adaptation Evolution Strategy
• Multi-Objective Covariance Matrix Adaptation Evolution Strategy
• Creating Multi-Objective Benchmarks with Shark

Tools

Finally, we present functionality which are not machine learning facilities themselves, but necessary or helpful tools.

Shark provides support for random numbers, the basic concepts are introduce here:

• Random Numbers

Sharks comes with its own solver for Quadratic Programs:

• Quadratic Program Solver

For convenience, Shark provides a statistics class wrapper, as well as generic support for serialization:

• Iterative Calculation of Statistics
• Serialization

For Shark developers

Note that Shark follows a

• Coding Convention.

If you contribute to Shark, you might also find these documents helpful:

• The Shark Build System Setup
• Writing Shark Tutorials
• The Shark Documentation System