This is one of the 100 recipes of the IPython Cookbook, the definitive guide to high-performance scientific computing and data science in Python.
randomPython module and NumPy.
import random import numpy as np
We use the
%precision magic (defined in IPython) to show only 3 decimals in the Python output. This is just to alleviate the text.
y, each one containing one million random numbers between 0 and 1.
n = 1000000 x = [random.random() for _ in range(n)] y = [random.random() for _ in range(n)]
xplus the first element of
y, and so on. We use a
forloop in a list comprehension.
z = [x[i] + y[i] for i in range(n)] z[:3]
%timeitmagic command to quickly evaluate the time taken by a single command.
%timeit [x[i] + y[i] for i in range(n)]
np.array()function does just that.
xa = np.array(x) ya = np.array(y)
ya contain the exact same numbers than our original lists
y. Whereas those lists where instances of a built-in class
list, our arrays are instances of a NumPy class
ndarray. Those types are implemented very differently in Python and NumPy. We will see that, in this example, using arrays instead of lists leads to drastic performance improvements.
forloop anymore. In NumPy, adding two arrays means adding the elements of the arrays component by component.
za = xa + ya za[:3]
We see that the list
z and the array
za contain the same elements (the sum of the numbers in
%timeit xa + ya
We observe that this operation is more than one order of magnitude faster in NumPy than in pure Python!
xa. Although this is not an element-wise operation, NumPy is still highly efficient here. The pure Python version uses the built-in
sumfunction on an iterable. The NumPy version uses the
np.sum()function on a NumPy array.
%timeit sum(x) # pure Python %timeit np.sum(xa) # NumPy
We also observe an impressive speedup here.
d = [abs(x[i] - y[j]) for i in range(1000) for j in range(1000)]
da = np.abs(xa[:1000,None] - ya[:1000])
%timeit [abs(x[i] - y[j]) for i in range(1000) for j in range(1000)] %timeit np.abs(xa[:1000, None] - ya[:1000])
Here again, observe observe the significant speedups.