This example shows you how to run a local optimisation with the Nelder-Mead downhill simplex method.
The Nelder-Mead method is a classical (deterministic) derivative-free optimisation method. It can be very fast if started near the true solution, but can easily get stuck on difficult problems. Nelder-Mead is essentially sequential in nature and can not easily be parallelised.
import pints
import pints.toy
# Create Rosenbrock error (optimum is at 1,1)
f = pints.toy.RosenbrockError()
# Pick starting point
x0 = [-0.75, 3.5]
x, fx = pints.optimise(f, x0, method=pints.NelderMead)
print('Found: ' + str(x))
Minimising error measure Using Nelder-Mead Running in sequential mode. Iter. Eval. Best Time m:s 0 3 865.9531 0:00.0 1 4 832.5452 0:00.0 2 5 832.5452 0:00.0 3 6 628.243 0:00.0 20 23 4.95828 0:00.0 40 43 3.525867 0:00.0 60 63 2.377579 0:00.0 80 83 1.114115 0:00.0 100 103 0.551 0:00.0 120 123 0.237 0:00.0 140 143 0.0666 0:00.0 160 163 0.00181 0:00.0 180 183 6.96e-06 0:00.0 200 203 2.66e-08 0:00.0 220 223 5.06e-11 0:00.0 240 243 2.43e-15 0:00.0 260 263 5.58e-18 0:00.0 280 283 7.74e-20 0:00.0 300 303 6.66e-23 0:00.0 320 323 1.86e-25 0:00.0 340 343 3.16e-28 0:00.0 360 364 3.08e-31 0:00.0 380 390 0 0:00.0 400 416 0 0:00.0 420 443 0 0:00.0 422 444 0 0:00.0 Halting: No significant change for 200 iterations. Found: [1. 1.]
We can use the ask-and-tell interface to visualise its progress through the search-space:
import numpy as np
import matplotlib.pyplot as plt
# Create figure
fig = plt.figure(figsize=(12, 12))
ax = fig.add_subplot(1, 1, 1)
ax.set_xlabel('x')
ax.set_ylabel('y')
# Show function
x = np.linspace(-1.5, 1.5, 150)
y = np.linspace(-0.5, 4, 225)
X, Y = np.meshgrid(x, y)
Z = [[np.log(f([i, j])) for i in x] for j in y]
levels = np.linspace(np.min(Z), np.max(Z), 20)
ax.contour(X, Y, Z, levels = levels)
# Show initial position
ax.plot(x0[0], x0[1], 'ks', markersize=10)
ax.text(x0[0] + 0.05, x0[1] - 0.03, 'Initial position', fontsize=18)
# Run 400 iterations with ask-and-tell, storing all positions
e = pints.SequentialEvaluator(f)
nm = pints.NelderMead(x0)
path = [x0]
for i in range(400):
xs = nm.ask()
fs = e.evaluate(xs)
nm.tell(fs)
path.append(nm.xbest())
# Plot path
path = np.array(path).T
ax.plot(path[0], path[1], 'x-', color='tab:orange', markersize=15)
plt.show()