Basic statistics

This notebook is an element of the free risk-engineering.org courseware. It can be distributed under the terms of the Creative Commons Attribution-ShareAlike licence.

Author: Eric Marsden [email protected].


This notebook contains an introduction to use of Python and the NumPy library for basic statistical calculations.

We start by importing the numpy library, which makes it possible to use functions and variables from the library, prefixed by numpy..

In [1]:
import numpy

We can use Python as simple interactive calculator:

In [2]:
2 + 3 + 4
Out[2]:
9

Here we call the sqrt function from the numpy library.

In [3]:
numpy.sqrt(2 + 2)
Out[3]:
2.0

Some useful constants are predefined.

In [4]:
numpy.pi
Out[4]:
3.141592653589793
In [5]:
numpy.sin(numpy.pi)
Out[5]:
1.2246467991473532e-16

The notation e-16 above means $10^{-16}$; the number above is very very small.

We can generate a random number from a uniform distribution between 20 and 30. If you evaluate this several times (press Shift-Enter or press on the triangular Run Cell button in the toolbar above), it will generate a different random number each time.

In [6]:
numpy.random.uniform(20, 30)
Out[6]:
26.044113161787763
In [7]:
numpy.random.uniform(20, 30)
Out[7]:
23.327642694274147

We can generate an array of random numbers by passing a third argument to the numpy.random.uniform function, saying how many random numbers we want. We store the array in a variable named obs.

In [8]:
obs = numpy.random.uniform(20, 30, 10)
obs
Out[8]:
array([ 28.41698187,  27.06306939,  27.40696663,  27.76508674,
        28.15876618,  21.57282656,  28.43294936,  24.73514419,
        23.66207196,  21.20696023])

The builtin function len in Python tells us the length of an array or a list.

In [9]:
len(obs)
Out[9]:
10

We can do arithmetic on arrays, adding them together or subtracting a constant from each element.

In [10]:
obs + obs
Out[10]:
array([ 56.83396374,  54.12613878,  54.81393327,  55.53017349,
        56.31753236,  43.14565313,  56.86589873,  49.47028839,
        47.32414392,  42.41392046])
In [11]:
obs - 25
Out[11]:
array([ 3.41698187,  2.06306939,  2.40696663,  2.76508674,  3.15876618,
       -3.42717344,  3.43294936, -0.26485581, -1.33792804, -3.79303977])

The array has methods, a kind of function that acts on the array.

In [12]:
obs.mean()
Out[12]:
25.842082313088696
In [13]:
obs.sum()
Out[13]:
258.42082313088696
In [14]:
obs.min()
Out[14]:
21.206960229584027

Simple plotting

The matplotlib library allows you to generate many types of plots and statistical graphs in a convenient way. The online gallery shows the variety of plots available, and the documentation is also available online. We import the pyplot component of matplotlib and give it an alias plt. We also ask the Jupyter (previously called "IPython") interface to show us plots inline, directly within this notebook.

In [15]:
import matplotlib.pyplot as plt
%matplotlib inline
%config InlineBackend.figure_formats=['svg']
In [16]:
X = numpy.random.uniform(20, 30, 10)
Y = numpy.random.uniform(50, 100, 10)
plt.scatter(X, Y);
In [17]:
x = numpy.linspace(-2, 10, 100)
plt.plot(x, numpy.sin(x));

We can add two vectors together, assuming that all their dimensions are identical. Our array $x$ has one dimension of size 100. We can add another random vector of size 100 to it, containing numbers drawn from a uniform probability distribution between -0.1 and 0.1 (these represent some random "noise" which is added to our sine curve).

In [18]:
x = numpy.linspace(0, 10, 100)
obs = numpy.sin(x) + numpy.random.uniform(-0.1, 0.1, 100)
plt.plot(x, obs);