Caching variables in a file

The ipycache package defines a %%cache cell magic allowing you to cache results of long-lasting computations in a pickle file.

In [1]:
import numpy as np
In [2]:
%load_ext ipycache
In [3]:
def long_function(a, b):
    import time
    time.sleep(1)
    return a + b

The following cell will only be executed if the pickle file does not exist (also, there is a --force option to always execute the cell and overwrite the results in the pickle file).

In [4]:
%%cache myvars.pkl a b c
a = 3
b = np.random.randn(3)
c = long_function(a, b)
Saved variables a, b, c to file 'myvars.pkl'.
In [5]:
print(a, b, c)
(3, array([-0.95123539, -0.17912806, -0.50877324]), array([ 2.04876461,  2.82087194,  2.49122676]))
In [6]:
del a, b, c

Now, we execute the same cell again (in practice, the cell only appears once in the notebook, so this simulates multiple executions of the same cell).

In [7]:
%%cache myvars.pkl a b c
a = 3
b = np.random.randn(3)
c = long_function(a, b)
Skipped the cell's code and loaded variables a, b, c from file 'myvars.pkl'.

The cell has been skipped, but the variables have been recovered from the pickle file.

In [8]:
print(a, b, c)
(3, array([-0.95123539, -0.17912806, -0.50877324]), array([ 2.04876461,  2.82087194,  2.49122676]))

Now, let's delete the file.

In [9]:
import os
os.remove('myvars.pkl')

Let's execute the cell once more.

In [10]:
%%cache myvars.pkl a b c
a = 3
b = np.random.randn(3)
c = long_function(a, b)
Saved variables a, b, c to file 'myvars.pkl'.
In [11]:
print(a, b, c)
(3, array([ 0.19778047,  1.81953213, -0.1259566 ]), array([ 3.19778047,  4.81953213,  2.8740434 ]))
In [12]:
os.remove('myvars.pkl')