display(gif)
Minimize a function that is difficult, expensive, or time-consuming to evaluate.
A flexible family of probability distributions over continuous functions
fig
$f \sim GP(m, k)$ if for any $\mathbf{x} = (x_1, \ldots, x_n)$, $(f(x_1), \ldots, f(x_n))$ has a multivariate normal distribution
If $f \sim GP(0, k)$, $\mathcal{D} = \{(x_1, y_1), \ldots, (x_n, y_n)\}$, where $y_i = f(x_i) + \varepsilon$, $\varepsilon \sim N(0, \sigma^2)$, $f\ |\ \mathcal{D}$ is also a Gaussian process with
fig
fig
fig
fig
fig
fig
fig
fig
fig
fig
fig
fig
fig
fig
fig
fig
display(gif)
fig
fig
So far, we have chose the covariance hyperparameters $\sigma^2$ and $\ell$ by maximizing the marginal (log) likelihood
$$ \begin{align*} \log \mathbb{P}(\mathbf{y}\ |\ \mathbf{x}, \theta) & \propto -\frac{1}{2} \mathbf{y}^{\intercal} \left(k(\mathbf{x}, \mathbf{x}) + \sigma^2 I\right)^{-1} \mathbf{y} - \frac{1}{2} \log \left|k(\mathbf{x}, \mathbf{x}) + \sigma^2 I\right| \end{align*} $$This can result in overfitting and underestimating the uncertainty in our acquisition function.
grid.fig
fig
display(gif)
fig
fig
fig
spearmint
open source Bayesian optimization packagemonetate_img
arochford@monetate.com