GPflow has a config file, gpflowrc
which allows the user to change the default behavious in GPflow. GPflow searches for the file in the following order:
You can also make gpflowrc
a hidden file, if you don't want it clutting your home directory, by renaming as .gpflowrc
.
Usually, one wants to change the settings for an entire project. We recommend adding a gpflowrc
file in the working directory to achieve this effect. It is helpful to add assertions / warnings in code to ensure that the correct settings are used. GPflow also allows settings to be adjusted in code, although this comes with some additional difficulties (see below). This is why we recommend using a settings file.
The most common use cases for changing settings are:
float32
computations on the GPU.By default, the configuration looks like this:
[logging]
# possible levels: CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET
level = WARNING
[verbosity]
tf_compile_verb = False
[dtypes]
float_type = float64
int_type = int32
[numerics]
jitter_level = 1e-6
# quadrature can be set to: allow, warn, error
ekern_quadrature = warn
[profiling]
dump_timeline = False
dump_tensorboard = False
output_file_name = timeline
output_directory = ./
each_time = False
[session]
intra_op_parallelism_threads = 0
inter_op_parallelism_threads = 0
You can access the settings as gpflow.settings
, and the different options are nested under the headings in the file. For example, to see how much jitter is added before attempting Cholesky decomposition:
import gpflow
print(gpflow.settings.jitter)
WARNING: Logging before flag parsing goes to stderr. W0813 20:55:52.087115 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/session_manager.py:31: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. W0813 20:55:52.090715 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/misc.py:27: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead. W0813 20:55:52.241827 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/training/tensorflow_optimizer.py:169: The name tf.train.AdadeltaOptimizer is deprecated. Please use tf.compat.v1.train.AdadeltaOptimizer instead. W0813 20:55:52.242599 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/training/tensorflow_optimizer.py:156: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead. W0813 20:55:52.243191 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/training/tensorflow_optimizer.py:169: The name tf.train.AdagradDAOptimizer is deprecated. Please use tf.compat.v1.train.AdagradDAOptimizer instead. W0813 20:55:52.243880 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/training/tensorflow_optimizer.py:169: The name tf.train.AdagradOptimizer is deprecated. Please use tf.compat.v1.train.AdagradOptimizer instead. W0813 20:55:52.244505 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/training/tensorflow_optimizer.py:169: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead. W0813 20:55:52.256444 4544173504 deprecation_wrapper.py:119] From /Users/artemav/code/GPflow/gpflow/saver/coders.py:80: The name tf.data.Iterator is deprecated. Please use tf.compat.v1.data.Iterator instead.
1e-05
If settings are updated during programme execution (see later), you need to always access the setting through the settings
module, to ensure you obtain the latest value. If you make a module level copy of the variable, you may end up using stale values.
Settings can be modified for an entire session, or for a limited set of statements, using a Python context manager. It is recommeded to use the context manager, as this prevents the change of state unintentionally spilling into other parts of the program. The global settings can be set with gpflow.settings.push(custom_config)
. This pushes the old config to a stack, which can be popped with gpflow.settings.pop()
. Again, we recommend using the context manager.
We first train a model without any jitter. We start by making a copy of the current settings using gpflow.settings.get_settings()
. We can edit the values of settings in this object by simple assignment. We then use the context manager gpflow.settings.temp_settings()
to use those settings for the duration of the model creation.
We will expect this to fail with an InvalidArgumentError
due to the Cholesky failing.
import numpy as np
import numpy.random as rnd
import tensorflow as tf
np.random.seed(0)
X = np.random.randn(100, 1)
Y = np.sin(X) + np.sin(1.5*X) + 0.3 * rnd.randn(*X.shape)
custom_config = gpflow.settings.get_settings()
custom_config.numerics.jitter_level = 0.0
print("Default jitter level: %.2e" % gpflow.settings.jitter)
with gpflow.settings.temp_settings(custom_config):
print("Current jitter level: %.2e" % gpflow.settings.jitter)
m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy())
try:
opt = gpflow.train.ScipyOptimizer()
opt.minimize(m)
except tf.errors.InvalidArgumentError as e:
print("\nAs expected, we encountered the error: `%s`\n" % e.message.split("\n")[0])
print("Reverted jitter level: %.2e" % gpflow.settings.jitter)
Default jitter level: 1.00e-05 Current jitter level: 0.00e+00 As expected, we encountered the error: `Cholesky decomposition was not successful. The input might not be valid.` Reverted jitter level: 1.00e-05
With the default jitter levels reverted, we can optimise the model just fine:
m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy())
gpflow.train.ScipyOptimizer().minimize(m)
GPflow also logs a wide variety of events that happen under the hood. One complex piece of code, is how Gaussian conditioning is handled. Many different cases are handled using multiple dispatch, and sometimes it is complicated to keep track of what code is being run. This is logged with the logger module. To display these debug messages, we need to set the logging level to DEBUG
. This needs to be done in gpflowrc
for it to have effect.
Because of the static graph compilation of TensorFlow, models defined inside of the context manager will still retain the settings outside the context manager. The zero jitter example above demonstrates this. We define the model inside a context manager with a zero jitter setting. When we try to optimise the model outside, the model will use zero jitter, since that was the setting's value at the time of model creation. Hence the Cholesky will fail. If we re-define the model with some jitter, the model will optimise just fine.
GPflow may create multiple tensorflow sessions for a single model. To control the session parameters change the [session] section of the settings. This section may contain any valid TensorFlow ConfigProto setting.
For instance to ensure all tensorflow graphs are run serially set
[session]
intra_op_parallelism_threads = 1
inter_op_parallelism_threads = 1
As per the TensorFlow documentation, a setting of 0 means the system picks an appropriate number of cores to use.