GPyOpt: mixing different types of variables¶

Javier Gonzalez, Amazon Research Cambridge.¶

Last updated Monday, July 2017

In this notebook we are going to see how to used GPyOpt to solve optimizaiton problems in which the domain of the fucntion is defined in terms of a variety of continous and discrete variables. To this end we start by loading GPyOpt.

In [ ]:
%pylab inline
import GPyOpt
from numpy.random import seed
import numpy as np
seed(123)


We will use the Alpine1 function, that it is available in the benchmark of functions of the package. This function is defined for arbitrary dimension. In this example we will work in dimension 9. The functional form of the Alpine1 function is:

$$f(x_1,x_2,...,x_9)=\sum_{i=1}^{9} \lvert {x_i \sin \left( x_i \right) + 0.1 x_i} \rvert$$

We load the function from GPyOpt, assuming that noisy free values will be sampled.

In [ ]:
func  = GPyOpt.objective_examples.experimentsNd.alpine1(input_dim=9)


We will consider that variables $x_1$, $x_2$ and $x_3$ are continuous and defined in the interval $[-5,5]$, variable $x_9$ is takes continuous values in the interval $[-1,2]$, variable $x_3$ is discrete and takes values $\{3,8,10\}$, and finally $x_4$ is categorical with three different categories. Next we define this domain to use it in GPyOpt.

In [ ]:
mixed_domain =[{'name': 'var1', 'type': 'continuous', 'domain': (-5,5),'dimensionality': 3},
{'name': 'var3', 'type': 'discrete', 'domain': (3,8,10),'dimensionality': 2},
{'name': 'var4', 'type': 'categorical', 'domain': (0,1,2),'dimensionality': 1},
{'name': 'var5', 'type': 'continuous', 'domain': (-1,2)}]

In [ ]:
myBopt = GPyOpt.methods.BayesianOptimization(f=func.f,                     # Objective function
domain=mixed_domain,          # Box-constraints of the problem
initial_design_numdata = 5,   # Number data initial design
acquisition_type='EI',        # Expected Improvement
exact_feval = True)           # True evaluations, no sample noise


Now, we run the optimization for 20 iterations or a maximum of 60 seconds and we show the convergence plots.

In [ ]:
max_iter = 20       ## maximum number of iterations
max_time = 60       ## maximum allowed time
eps      = 1e-6     ## tolerance, max distance between consicutive evaluations.

myBopt.run_optimization(max_iter,eps=0)

In [ ]:
np.round(myBopt.X,2)

In [ ]:
myBopt.plot_convergence()