SciPy

Modify by P. Zambelli (pietro.zambelli ing unitn)

Original work from J.R. Johansson

In [14]:
%pylab inline
from IPython.display import Image
Populating the interactive namespace from numpy and matplotlib

Introduction

The SciPy framework builds on top of the low-level NumPy framework for multidimensional arrays, and provides a large number of higher-level scientific algorithms. Some of the topics that SciPy covers are:

Each of these submodules provides a number of functions and classes that can be used to solve problems in their respective topics.

In this lecture we will look at how to use some of these subpackages.

To access the SciPy package in a Python program, we start by importing everything from the scipy module.

In [1]:
from scipy import *

If we only need to use part of the SciPy framework we can selectively include only those modules we are interested in. For example, to include the linear algebra package under the name la, we can do:

In [2]:
import scipy.linalg as la

Special functions

A large number of mathematical special functions are important for many computional physics problems. SciPy provides implementations of a very extensive set of special functions. For details, see the list of functions in the reference documention at http://docs.scipy.org/doc/scipy/reference/special.html#module-scipy.special.

To demonstrate the typical usage of special functions we will look in more detail at the Bessel functions:

In [2]:
#
# The scipy.special module includes a large number of Bessel-functions
# Here we will use the functions jn and yn, which are the Bessel functions 
# of the first and second kind and real-valued order. We also include the 
# function jn_zeros and yn_zeros that gives the zeroes of the functions jn
# and yn.
#
from scipy.special import jn, yn, jn_zeros, yn_zeros
In [3]:
x = linspace(0, 10, 100)

fig, ax = subplots()
for n in range(4):
    ax.plot(x, jn(n, x), label=r"$J_%d(x)$" % n)
ax.legend();
In [4]:
# zeros of Bessel functions
n = 0 # order
m = 4 # number of roots to compute
jn_zeros(n, m)
Out[4]:
array([  2.40482556,   5.52007811,   8.65372791,  11.79153444])

Integration

Numerical integration: quadrature

Numerical evaluation of a function of the type

$\displaystyle \int_a^b f(x) dx$

is called numerical quadrature, or simply quadature. SciPy provides a series of functions for different kind of quadrature, for example the quad, dblquad and tplquad for single, double and triple integrals, respectively.

In [5]:
from scipy.integrate import quad, dblquad, tplquad

The quad function takes a large number of optional arguments, which can be used to fine-tune the behaviour of the function (try help(quad) for details).

The basic usage is as follows:

In [6]:
# define a simple function for the integrand
def f(x):
    return x
In [7]:
x_lower = 0 # the lower limit of x
x_upper = 1 # the upper limit of x

val, abserr = quad(f, x_lower, x_upper)

print("integral value =", val, ", absolute error =", abserr )
integral value = 0.5 , absolute error = 5.551115123125783e-15

If we need to pass extra arguments to integrand function we can use the args keyword argument:

In [8]:
def integrand(x, n):
    """
    Bessel function of first kind and order n. 
    """
    return jn(n, x)


x_lower = 0  # the lower limit of x
x_upper = 10 # the upper limit of x

val, abserr = quad(integrand, x_lower, x_upper, args=(3, ))

print(val, abserr)
0.7366751370811073 9.389126882496403e-13

For simple functions we can use a lambda function (name-less function) instead of explicitly defining a function for the integrand, compute the Gauss Integral:

$\int_{-\infty}^{+\infty} e^{-x^2}\,dx = \sqrt{\pi}$

In [9]:
val, abserr = quad(lambda x: exp(-x ** 2), -Inf, Inf)

print("numerical  =", val, abserr)

analytical = sqrt(pi)
print("analytical =", analytical)
numerical  = 1.7724538509055159 1.4202636780944923e-08
analytical = 1.77245385091

As show in the example above, we can also use 'Inf' or '-Inf' as integral limits.

Higher-dimensional integration works in the same way:

$\int_{0}^{10} \int_{0}^{10} e^{-x^2 -y^2}\,dx,dy$

In [10]:
def integrand(x, y):
    return exp(-x**2-y**2)

x_lower = 0  
x_upper = 10
y_lower = 0
y_upper = 10

print("dblquad val: %g, err: %g" % dblquad(integrand, x_lower, x_upper, lambda x : y_lower, lambda x: y_upper))
dblquad val: 0.785398, err: 1.63823e-13

Note how we had to pass lambda functions for the limits for the y integration, since these in general can be functions of x.

Ordinary differential equations (ODEs)

SciPy provides two different ways to solve ODEs: An API based on the function odeint, and object-oriented API based on the class ode. Usually odeint is easier to get started with, but the ode class offers some finer level of control.

Here we will use the odeint functions. For more information about the class ode, try help(ode). It does pretty much the same thing as odeint, but in an object-oriented fashion.

To use odeint, first import it from the scipy.integrate module:

In [3]:
from scipy.integrate import odeint, ode

A system of ODEs are usually formulated on standard form before it is attacked numerically. The standard form is:

$y' = f(y, t)$

where

$y = [y_1(t),\; y_2(t),\; ...,\; y_n(t)]$

and $f$ is some function that gives the derivatives of the function $y_i(t)$. To solve an ODE we need to know the function $f$ and an initial condition, $y(0)$.

Note that higher-order ODEs can always be written in this form by introducing new variables for the intermediate derivatives.

Once we have defined the Python function f and array y_0 (that is $f$ and $y(0)$ in the mathematical formulation), we can use the odeint function as:

y_t = odeint(f, y_0, t)

where t is and array with time-coordinates for which to solve the ODE problem. y_t is an array with one row for each point in time in t, where each column corresponds to a solution y_i(t) at that point in time.

We will see how we can implement f and y_0 in Python code in the examples below.

Example: double pendulum

Let's consider a physical example: The double compound pendulum, described in some detail here: http://en.wikipedia.org/wiki/Double_pendulum

In [4]:
Image(url='http://upload.wikimedia.org/wikipedia/commons/thumb/c/c9/Double-compound-pendulum-dimensioned.svg/500px-Double-compound-pendulum-dimensioned.svg.png')
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-4-645a2b9bd948> in <module>()
----> 1 Image(url='http://upload.wikimedia.org/wikipedia/commons/thumb/c/c9/Double-compound-pendulum-dimensioned.svg/500px-Double-compound-pendulum-dimensioned.svg.png')

NameError: name 'Image' is not defined

The equations of motion of the pendulum are given on the wiki page:

${\dot \theta_1} = \frac{6}{m\ell^2} \frac{ 2 p_{\theta_1} - 3 \cos(\theta_1-\theta_2) p_{\theta_2}}{16 - 9 \cos^2(\theta_1-\theta_2)}$

${\dot \theta_2} = \frac{6}{m\ell^2} \frac{ 8 p_{\theta_2} - 3 \cos(\theta_1-\theta_2) p_{\theta_1}}{16 - 9 \cos^2(\theta_1-\theta_2)}.$

${\dot p_{\theta_1}} = -\frac{1}{2} m \ell^2 \left [ {\dot \theta_1} {\dot \theta_2} \sin (\theta_1-\theta_2) + 3 \frac{g}{\ell} \sin \theta_1 \right ]$

${\dot p_{\theta_2}} = -\frac{1}{2} m \ell^2 \left [ -{\dot \theta_1} {\dot \theta_2} \sin (\theta_1-\theta_2) + \frac{g}{\ell} \sin \theta_2 \right]$

To make the Python code simpler to follow, let's introduce new variable names and the vector notation: $x = [\theta_1, \theta_2, p_{\theta_1}, p_{\theta_2}]$

${\dot x_1} = \frac{6}{m\ell^2} \frac{ 2 x_3 - 3 \cos(x_1-x_2) x_4}{16 - 9 \cos^2(x_1-x_2)}$

${\dot x_2} = \frac{6}{m\ell^2} \frac{ 8 x_4 - 3 \cos(x_1-x_2) x_3}{16 - 9 \cos^2(x_1-x_2)}$

${\dot x_3} = -\frac{1}{2} m \ell^2 \left [ {\dot x_1} {\dot x_2} \sin (x_1-x_2) + 3 \frac{g}{\ell} \sin x_1 \right ]$

${\dot x_4} = -\frac{1}{2} m \ell^2 \left [ -{\dot x_1} {\dot x_2} \sin (x_1-x_2) + \frac{g}{\ell} \sin x_2 \right]$

In [5]:
g = 9.82
L = 0.5
m = 0.1

def dx(x, t):
    """
    The right-hand side of the pendulum ODE
    """
    x1, x2, x3, x4 = x[0], x[1], x[2], x[3]
    
    dx1 = 6.0/(m*L**2) * (2 * x3 - 3 * cos(x1-x2) * x4)/(16 - 9 * cos(x1-x2)**2)
    dx2 = 6.0/(m*L**2) * (8 * x4 - 3 * cos(x1-x2) * x3)/(16 - 9 * cos(x1-x2)**2)
    dx3 = -0.5 * m * L**2 * ( dx1 * dx2 * sin(x1-x2) + 3 * (g/L) * sin(x1))
    dx4 = -0.5 * m * L**2 * (-dx1 * dx2 * sin(x1-x2) + (g/L) * sin(x2))
    
    return [dx1, dx2, dx3, dx4]
In [6]:
# choose an initial state
x0 = [pi/4, pi/2, 0, 0]
In [7]:
# time coodinate to solve the ODE for: from 0 to 10 seconds
t = linspace(0, 10, 250)
In [8]:
# solve the ODE problem
x = odeint(dx, x0, t)
In [9]:
# plot the angles as a function of time

fig, axes = subplots(1,2, figsize=(12,4))
axes[0].plot(t, x[:, 0], 'r', label="theta1")
axes[0].plot(t, x[:, 1], 'b', label="theta2")


x1 = + L * sin(x[:, 0])
y1 = - L * cos(x[:, 0])

x2 = x1 + L * sin(x[:, 1])
y2 = y1 - L * cos(x[:, 1])
    
axes[1].plot(x1, y1, 'r', label="pendulum1")
axes[1].plot(x2, y2, 'b', label="pendulum2")
axes[1].set_ylim([-1, 0])
axes[1].set_xlim([1, -1]);

Simple annimation of the pendulum motion.

In [10]:
from IPython.display import clear_output
In [11]:
fig, axes = subplots(1,2, figsize=(12,4))
x1 = + L * sin(x[:, 0])
y1 = - L * cos(x[:, 0])
x2 = x1 + L * sin(x[:, 1])
y2 = y1 - L * cos(x[:, 1])

for t_idx, tt in enumerate(t):
    # get scalar
    xs1, xs2, ys1, ys2 = x1[t_idx], x2[t_idx], y1[t_idx], y2[t_idx]
    
    # clear axis
    axes[0].cla() 
    axes[1].cla()
    
    # plot tracks
    axes[0].plot(t[:t_idx], x[:, 0][:t_idx], 'r', alpha=0.5, label="theta1")
    axes[0].plot(t[:t_idx], x[:, 1][:t_idx], 'b', alpha=0.5, label="theta2")
    axes[1].plot(x1[:t_idx], y1[:t_idx], 'r', alpha=0.5, label="pendulum1")
    axes[1].plot(x2[:t_idx], y2[:t_idx], 'b', alpha=0.5, label="pendulum2")
    
    # plot pendulum
    axes[1].plot([0, xs1], [0, ys1], 'r.-')
    axes[1].plot([xs1, xs2], [ys1, ys2], 'b.-')
    
    axes[0].set_ylim([-2.0, 2.0])
    axes[0].set_xlim([0, 10])
    axes[1].set_ylim([-1.5, 0.5])
    axes[1].set_xlim([1, -1])
    
    
    display(fig)
    clear_output()

Example: Damped harmonic oscillator

ODE problems are important in computational physics, so we will look at one more example: the damped harmonic oscillation. This problem is well described on the wiki page: http://en.wikipedia.org/wiki/Damping

The equation of motion for the damped oscillator is:

$\displaystyle \frac{\mathrm{d}^2x}{\mathrm{d}t^2} + 2\zeta\omega_0\frac{\mathrm{d}x}{\mathrm{d}t} + \omega^2_0 x = 0$

where $x$ is the position of the oscillator, $\omega_0$ is the frequency, and $\zeta$ is the damping ratio. To write this second-order ODE on standard form we introduce $p = \frac{\mathrm{d}x}{\mathrm{d}t}$:

$\displaystyle \frac{\mathrm{d}p}{\mathrm{d}t} = - 2\zeta\omega_0 p - \omega^2_0 x$

$\displaystyle \frac{\mathrm{d}x}{\mathrm{d}t} = p$

In the implementation of this example we will add extra arguments to the RHS function for the ODE, rather than using global variables as we did in the previous example. As a consequence of the extra arguments to the RHS, we need to pass an keyword argument args to the odeint function:

In [56]:
def dy(y, t, zeta, w0):
    """
    The right-hand side of the damped oscillator ODE
    """
    x, p = y[0], y[1]
    
    dp = -2 * zeta * w0 * p - w0**2 * x
    dx = p

    return dx, dp
In [57]:
# initial state: 
y0 = 1.0, 0.0
In [58]:
# time coodinate to solve the ODE for
t = linspace(0, 10, 1000)
w0 = 2 * pi * 1.0
In [59]:
# solve the ODE problem for three different values of the damping ratio

y1 = odeint(dy, y0, t, args=(0.0, w0)) # undamped
y2 = odeint(dy, y0, t, args=(0.2, w0)) # under damped
y3 = odeint(dy, y0, t, args=(1.0, w0)) # critial damping
y4 = odeint(dy, y0, t, args=(5.0, w0)) # over damped
In [60]:
fig, ax = subplots()
ax.plot(t, y1[:,0], 'k', label="undamped", linewidth=0.25)
ax.plot(t, y2[:,0], 'r', label="under damped")
ax.plot(t, y3[:,0], 'b', label=r"critical damping")
ax.plot(t, y4[:,0], 'g', label="over damped")
ax.legend();

Linear algebra

The linear algebra module contains a lot of matrix related functions, including linear equation solving, eigenvalue solvers, matrix functions (for example matrix-exponentiation), a number of different decompositions (SVD, LU, cholesky), etc.

Detailed documetation is available at: http://docs.scipy.org/doc/scipy/reference/linalg.html

Here we will look at how to use some of these functions:

Linear equation systems

Linear equation systems on the matrix form

$A x = b$

where $A$ is a matrix and $x,b$ are vectors can be solved like:

In [34]:
A = array([[1,2,3], [4,5,6], [7,8,9]])
b = array([1,2,3])
In [35]:
x = solve(A, b)

x
Out[35]:
array([-0.33333333,  0.66666667,  0.        ])
In [36]:
# check
dot(A, x) - b
Out[36]:
array([ -1.11022302e-16,   0.00000000e+00,   0.00000000e+00])

We can also do the same with

$A X = B$

where $A, B, X$ are matrices:

In [37]:
A = rand(3,3)
B = rand(3,3)
In [38]:
X = solve(A, B)
In [39]:
X
Out[39]:
array([[ 2.28587973,  5.88845235,  1.6750663 ],
       [-4.88205838, -5.26531274, -1.37990347],
       [ 1.75135926, -2.05969998, -0.09859636]])
In [40]:
# check
norm(dot(A, X) - B)
Out[40]:
6.2803698347351007e-16

Eigenvalues and eigenvectors

The eigenvalue problem for a matrix $A$:

$\displaystyle A v_n = \lambda_n v_n$

where $v_n$ is the $n$th eigenvector and $\lambda_n$ is the $n$th eigenvalue.

To calculate eigenvalues of a matrix, use the eigvals and for calculating both eigenvalues and eigenvectors, use the function eig:

In [41]:
evals = eigvals(A)
In [42]:
evals
Out[42]:
array([ 1.06633891+0.j        , -0.12420467+0.10106325j,
       -0.12420467-0.10106325j])
In [43]:
evals, evecs = eig(A)
In [44]:
evals
Out[44]:
array([ 1.06633891+0.j        , -0.12420467+0.10106325j,
       -0.12420467-0.10106325j])
In [45]:
evecs
Out[45]:
array([[ 0.89677688+0.j        , -0.30219843-0.30724366j,
        -0.30219843+0.30724366j],
       [ 0.35446145+0.j        ,  0.79483507+0.j        ,  0.79483507+0.j        ],
       [ 0.26485526+0.j        , -0.20767208+0.37334563j,
        -0.20767208-0.37334563j]])

The eigenvectors corresponding to the $n$th eigenvalue (stored in evals[n]) is the $n$th column in evecs, i.e., evecs[:,n]. To verify this, let's try mutiplying eigenvectors with the matrix and compare to the product of the eigenvector and the eigenvalue:

In [46]:
n = 1

norm(dot(A, evecs[:,n]) - evals[n] * evecs[:,n])
Out[46]:
1.3964254612015911e-16

There are also more specialized eigensolvers, like the eigh for Hermitian matrices.

Matrix operations

In [47]:
# the matrix inverse
inv(A)
Out[47]:
array([[-1.38585633,  1.36837431,  6.03633364],
       [ 3.80855289, -4.76960426, -5.2571037 ],
       [ 0.0689213 ,  2.4652602 , -2.5948838 ]])
In [48]:
# determinant
det(A)
Out[48]:
0.027341548212627968
In [49]:
# norms of various orders
norm(A, ord=2), norm(A, ord=Inf)
Out[49]:
(1.1657807164173386, 1.7872032588446576)

Sparse matrices

Sparse matrices are often useful in numerical simulations dealing with large systems, if the problem can be described in matrix form where the matrices or vectors mostly contains zeros. Scipy has a good support for sparse matrices, with basic linear algebra operations (such as equation solving, eigenvalue calculations, etc).

There are many possible strategies for storing sparse matrices in an efficient way. Some of the most common are the so-called coordinate form (COO), list of list (LIL) form, and compressed-sparse column CSC (and row, CSR). Each format has some advantanges and disadvantages. Most computational algorithms (equation solving, matrix-matrix multiplication, etc) can be efficiently implemented using CSR or CSC formats, but they are not so intuitive and not so easy to initialize. So often a sparse matrix is initially created in COO or LIL format (where we can efficiently add elements to the sparse matrix data), and then converted to CSC or CSR before used in real calcalations.

For more information about these sparse formats, see e.g. http://en.wikipedia.org/wiki/Sparse_matrix

When we create a sparse matrix we have to choose which format it should be stored in. For example,

In [50]:
from scipy.sparse import *
In [51]:
# dense matrix
M = array([[1,0,0,0], [0,3,0,0], [0,1,1,0], [1,0,0,1]]); M
Out[51]:
array([[1, 0, 0, 0],
       [0, 3, 0, 0],
       [0, 1, 1, 0],
       [1, 0, 0, 1]])
In [52]:
# convert from dense to sparse
A = csr_matrix(M); A
Out[52]:
<4x4 sparse matrix of type '<type 'numpy.int64'>'
	with 6 stored elements in Compressed Sparse Row format>
In [53]:
# convert from sparse to dense
A.todense()
Out[53]:
matrix([[1, 0, 0, 0],
        [0, 3, 0, 0],
        [0, 1, 1, 0],
        [1, 0, 0, 1]])

More efficient way to create sparse matrices: create an empty matrix and populate with using matrix indexing (avoids creating a potentially large dense matrix)

In [54]:
A = lil_matrix((4,4)) # empty 4x4 sparse matrix
A[0,0] = 1
A[1,1] = 3
A[2,2] = A[2,1] = 1
A[3,3] = A[3,0] = 1
A
Out[54]:
<4x4 sparse matrix of type '<type 'numpy.float64'>'
	with 6 stored elements in LInked List format>
In [55]:
A.todense()
Out[55]:
matrix([[ 1.,  0.,  0.,  0.],
        [ 0.,  3.,  0.,  0.],
        [ 0.,  1.,  1.,  0.],
        [ 1.,  0.,  0.,  1.]])

Converting between different sparse matrix formats:

In [56]:
A
Out[56]:
<4x4 sparse matrix of type '<type 'numpy.float64'>'
	with 6 stored elements in LInked List format>
In [57]:
A = csr_matrix(A); A
Out[57]:
<4x4 sparse matrix of type '<type 'numpy.float64'>'
	with 6 stored elements in Compressed Sparse Row format>

We can compute with sparse matrices like with dense matrices:

In [59]:
A.todense()
Out[59]:
matrix([[ 1.,  0.,  0.,  0.],
        [ 0.,  3.,  0.,  0.],
        [ 0.,  1.,  1.,  0.],
        [ 1.,  0.,  0.,  1.]])
In [60]:
(A * A).todense()
Out[60]:
matrix([[ 1.,  0.,  0.,  0.],
        [ 0.,  9.,  0.,  0.],
        [ 0.,  4.,  1.,  0.],
        [ 2.,  0.,  0.,  1.]])
In [61]:
dot(A, A).todense()
Out[61]:
matrix([[ 1.,  0.,  0.,  0.],
        [ 0.,  9.,  0.,  0.],
        [ 0.,  4.,  1.,  0.],
        [ 2.,  0.,  0.,  1.]])
In [62]:
v = array([1,2,3,4])[:,newaxis]; v
Out[62]:
array([[1],
       [2],
       [3],
       [4]])
In [63]:
# sparse matrix - dense vector multiplication
A * v
Out[63]:
array([[ 1.],
       [ 6.],
       [ 5.],
       [ 5.]])
In [64]:
# same result with dense matrix - dense vector multiplcation
A.todense() * v
Out[64]:
matrix([[ 1.],
        [ 6.],
        [ 5.],
        [ 5.]])

Optimization

Optimization (finding minima or maxima of a function) is a large field in mathematics, and optimization of complicated functions or in many variables can be rather involved. Here we will only look at a few very simple cases. For a more detailed introduction to optimization with SciPy see:

http://scipy-lectures.github.com/advanced/mathematical_optimization/index.html

To use the optimization module in scipy first include the optimize module:

In [12]:
from scipy import optimize

Finding a minima

Let's first look at how to find the minima of a simple function of a single variable:

In [72]:
def f(x):
    return 4*x**3 + (x-2)**2 + x**4
In [73]:
fig, ax  = subplots()
x = linspace(-5, 3, 100)
ax.plot(x, f(x));

We can use the fmin_bfgs function to find the minima of a function:

In [74]:
x_min = optimize.fmin_bfgs(f, -2)
x_min 
Optimization terminated successfully.
         Current function value: -3.506641
         Iterations: 6
         Function evaluations: 30
         Gradient evaluations: 10
Out[74]:
array([-2.67298167])
In [75]:
optimize.fmin_bfgs(f, 0.5) 
Optimization terminated successfully.
         Current function value: 2.804988
         Iterations: 3
         Function evaluations: 15
         Gradient evaluations: 5
Out[75]:
array([ 0.46961745])

We can also use the brent or fminbound functions. They have a bit different syntax and use different algorithms.

In [70]:
optimize.brent(f)
Out[70]:
0.46961743402759754
In [71]:
optimize.fminbound(f, -4, 2)
Out[71]:
-2.6729822917513886

Finding a solution to a function

To find the root for a function of the form $f(x) = 0$ we can use the fsolve function. It requires an initial guess:

In [16]:
omega_c = 3.0

def f(omega):
    # a transcendental equation: resonance frequencies of a low-Q SQUID terminated microwave resonator
    return tan(2*pi*omega) - omega_c/omega
In [19]:
fig, ax  = subplots(figsize=(10,4))
x = linspace(0, 4, 5000)
y = f(x)
mask = where(abs(y) > 50)
x[mask] = y[mask] = NaN # get rid of vertical line when the function flip sign
ax.plot(x, y)
ax.plot([0, 4], [0, 0], 'k')
ax.set_ylim(-5,5)
ax.grid()
In [105]:
optimize.fsolve(f, 0.1)
Out[105]:
array([ 0.23743014])
In [108]:
optimize.fsolve(f, 0.6)
Out[108]:
array([ 0.71286972])
In [107]:
optimize.fsolve(f, 1.1)
Out[107]:
array([ 1.18990285])

Interpolation

Interpolation is simple and convenient in scipy: The interp1d function, when given arrays describing X and Y data, returns and object that behaves like a function that can be called for an arbitrary value of x (in the range covered by X), and it returns the corresponding interpolated y value:

In [85]:
from scipy.interpolate import *
In [86]:
def f(x):
    return sin(x)
In [87]:
n = arange(0, 10)  
x = linspace(0, 9, 1000)

y_meas = f(n) + 0.1 * randn(len(n)) # simulate measurement with noise
y_real = f(x)

methods = ('linear', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic')
results = {}
for method in methods:
    method_interpolation = interp1d(n, y_meas, kind=method)
    results[method] = method_interpolation(x)
In [90]:
fig, ax = subplots(figsize=(10,4))
ax.plot(n, y_meas, 'bs', label='noisy data')
ax.plot(x, y_real, 'k', lw=2, label='true function')
for method in methods:
    ax.plot(x, results[method], label=method)
ax.legend(loc='best');

Statistics

The scipy.stats module contains a large number of statistical distributions, statistical functions and tests. For a complete documentation of its features, see http://docs.scipy.org/doc/scipy/reference/stats.html.

There is also a very powerful python package for statistical modelling called statsmodels. See http://statsmodels.sourceforge.net for more details.

In [91]:
from scipy import stats
In [92]:
# create a (discreet) random variable with poissionian distribution

X = stats.poisson(3.5) # photon distribution for a coherent state with n=3.5 photons
In [98]:
n = arange(0, 15)

fig, axes = subplots(3,1, sharex=True)

# plot the probability mass function (PMF)
axes[0].step(n, X.pmf(n))

# plot the commulative distribution function (CDF)
axes[1].step(n, X.cdf(n))

# plot histogram of 1000 random realizations of the stochastic variable X
axes[2].hist(X.rvs(size=1000));
In [99]:
# create a (continous) random variable with normal distribution
Y = stats.norm()
In [100]:
x = linspace(-5,5,100)

fig, axes = subplots(3,1, sharex=True)

# plot the probability distribution function (PDF)
axes[0].plot(x, Y.pdf(x))

# plot the commulative distributin function (CDF)
axes[1].plot(x, Y.cdf(x));

# plot histogram of 1000 random realizations of the stochastic variable Y
axes[2].hist(Y.rvs(size=1000), bins=50);

Statistics:

In [101]:
X.mean(), X.std(), X.var() # poission distribution
Out[101]:
(3.5, 1.8708286933869707, 3.5)
In [102]:
Y.mean(), Y.std(), Y.var() # normal distribution
Out[102]:
(0.0, 1.0, 1.0)

Statistical tests

Test if two sets of (independent) random data comes from the same distribution:

In [110]:
r0 = X.rvs(size=1000)
r1 = X.rvs(size=1000)
t_statistic, p_value = stats.ttest_ind(r0, r1)

print("t-statistic =", t_statistic)
print("p-value =", p_value)

#fig, ax = subplots(1,1)
#ax.plot(r0, label='r0')
#ax.plot(r1, label='r1')
t-statistic = 0.06150794316381086
p-value = 0.950960840523

Since the p value is very large we cannot reject the hypothesis that the two sets of random data have different means.

To test if the mean of a single sample of data has mean 0.1 (the true mean is 0.0):

In [115]:
stats.ttest_1samp(Y.rvs(size=1000), 0.1)
Out[115]:
(array(-0.0496376767728931), 0.96042104403930884)

Low p-value means that we can reject the hypothesis that the mean of Y is 0.1.

In [112]:
Y.mean()
Out[112]:
0.0
In [113]:
stats.ttest_1samp(Y.rvs(size=1000), Y.mean())
Out[113]:
(array(0.8240077183607233), 0.41013185202675151)

Further reading