BayesOpt
 All Classes Namespaces Files Functions Variables Enumerations Enumerator Groups Pages
Reference Manual

Table of Contents

The library is intended to be both fast and clear for development and research. At the same time, it does everything the "right way". For example:

Testing the installation.

The library includes several test program that can be found in the bin folder. Furthermore, there are test programs respectively under the python and matlab folders. They provide examples of the different interfaces that BayesOpt provide.

First of all, make sure your system finds the correct libraries. In Windows, you can copy the dlls to your working folder or include the lib folder in the path.

In some *nix systems, including Ubuntu, Debian and Mac OS, the library is installed by default in /usr/local/lib/. However, this folder is not included by default in the linker, Python or Matlab paths by default. This is specially critical when building shared libraries (mandatory for Python usage). The script exportlocalpaths.sh makes sure that the folder with the libraries is included in all the necessary paths.

After that, there are 3 steps that should be follow:

Understanding the parameters

BayesOpt relies on a complex and highly configurable mathematical model. In theory, it should work reasonably well for many problems in its default configuration. However, Bayesian optimization shines when we can include as much knowledge as possible about the target function or about the problem. Or, if the knowledge is not available, keep the model as general as possible (to avoid bias). In this part, knowledge about Gaussian processes or nonparametric models in general might be useful.

For example, with the parameters we can select the kind of kernel, mean function or surrogate model that we want to use. With the kernel we can play with the smoothness of the function and it's derivatives. The mean function can be use to model the overall trend (flat, linear, etc.). If we know the overall signal variance we better use a Gaussian process, if we don't, we should use a Student's t process instead.

For that reason, the parameters are bundled in a structure (C/C++/Matlab/Octave) or dictionary (Python), depending on the API that we use. This is a brief explanation of every parameter

Budget parameters

Initialization parameters

Logging parameters

Surrogate model parameters

Hyperparameter learning

Although BayesOpt tries to build a full analytic Bayesian model for the surrogate function, some hyperparameters cannot be estimated in closed form. Currently, the only parameters of BayesOpt models that require special treatment are the kernel hyperparameters. See Section Methods for learning the kernel parameters for a detailed description

Exploration/exploitation parameters

Kernel parameters

Mean function parameters

Using the library

Here we show a brief summary of the different ways to use the library:

C/C++ callback usage

This interface is the most standard approach. Due to the large compatibility with C code with other languages it could also be used for other languages such as Fortran, Ada, etc.

The function to optimize must agree with the template provided in bayesopt.h

double my_function (unsigned int n, const double *x, double *gradient, void *func_data);

Note that the gradient has been included for future compatibility, although in the current implementation, it is not used. You can just ignore it or send a NULL pointer.

The parameters are defined in the bopt_params struct. The easiest way to set the parameters is to use

bopt_params initialize_parameters_to_default(void);

and then, modify the necesary fields.

Once we have set the parameters and the function, we can called the optimizer

int bayes_optimization(int nDim, /* number of dimensions */
eval_func f, /* function to optimize */
void* f_data, /* extra data that is transfered directly to f */
const double *lb, const double *ub, /* bounds */
double *x, /* out: minimizer */
double *minf, /* out: minimum */
bopt_params parameters);

C++ inheritance usage

This is the most straighforward and complete method to use the library. The object that must be optimized must inherit from the BayesOptContinuous or BayesOptDiscrete classes. The objects can be found in bayesoptcont.hpp and bayesoptdisc.hpp respectively.

Then, we just need to override one of the virtual functions called evaluateSample, which can be called with C arrays and uBlas vectors. Since there is no pure virtual functions, you can just redefine your preferred interface.

Experimental: You can also override the checkReachability function to include nonlinear restrictions.

class MyOptimization: public BayesOptContinous
{
public:
MyOptimization(bopt_params param):
BayesOptContinous(param) {}
double evaluateSample( const boost::numeric::ublas::vector<double> &query )
{
// My function here
};
bool checkReachability( const boost::numeric::ublas::vector<double> &query )
{
// My restrictions here
};
};

As with the callback interface, the parameters are defined in the bopt_params struct. The easiest way to set the parameters is to use

bopt_params initialize_parameters_to_default(void);

and then, modify the necesary fields.

Python callback/inheritance usage

The file python/demo_quad.py provides examples of the two Python interfaces.

Parameters: For both interfaces, the parameters are defined as a Python dictionary with the same structure as the bopt_params struct in the C/C++ interface. The enumerate values are replaced by strings without the prefix. For example, the C_EI criteria is replaced by the string "EI" and the M_ZERO mean function is replaced by the string "ZERO".

The parameter dictionary can be initialized using

1 parameters = bayesopt.initialize_params()

however, this is not necesary in general. If any of the parameter is not included in the dictionary, the default value is included instead.

Callback: The callback interface is just a wrapper of the C interface. In this case, the callback function should have the form

1 def my_function (query):

where query is a numpy array and the function returns a double scalar.

The optimization process can be called as

1 y_out, x_out, error = bayesopt.optimize(my_function, n_dimensions, lower_bound, upper_bound, parameters)

where the result is a tuple with the minimum as a numpy array (x_out), the value of the function at the minimum (y_out) and the error code.

Inheritance: The object oriented construction is similar to the C++ interface.

1 class MyModule(bayesoptmodule.BayesOptModule):
2  def evalfunc(self,query):
3  """ My function """

The BayesOptModule include atributes for the parameters (params), number of dimensions (n) and bounds (lb and up).

Then, the optimization process can be called as

1 my_instance = MyModule()
2 # set parameters, bounds and number of dimensions.
3 y_out, x_out, error = my_instance.optimize()

wher the result is a tuple with the minimum as a numpy array (x_out), the value of the function at the minimum (y_out) and the error code.

Matlab/Octave callback usage

The file matlab/runtest.m provides an example of the Matlab/Octave interface.

Parameters: The parameters are defined as a Matlab struct equivalent to bopt_params struct in the C/C++ interface, except for the theta and mu arrays which are replaced by Matlab vectors. Thus, the number of elements (n_theta and n_mu) are not needed. The enumerate values are replaced by strings without the prefix. For example, the C_EI criteria is replaced by the string "EI" and the M_ZERO mean function is replaced by the string "ZERO".

If any of the parameter is not included in the Matlab struct, the default value is automatically included instead.

Callback: The callback interface is just a wrapper of the C interface. In this case, the callback function should have the form

function y = my_function (query):

where query is a Matlab vector and the function returns a scalar.

The optimization process can be called (both in Matlab and Octave) as

[x_out, y_out] = bayesopt('my_function', n_dimensions, parameters, lower_bound, upper_bound)

where the result is the minimum as a vector (x_out) and the value of the function at the minimum (y_out).

In Matlab, but not in Octave, the optimization can also be called with function handlers

[x_out, y_out] = bayesopt(@my_function, n_dimensions, parameters, lower_bound, upper_bound)