# Norms¶

We now consider the different norms discussed in lecture:

$$||\mathbf{v}||_1 = \sum_{k=1}^n |v_k|$$$$||\mathbf{v}||_2 = \sqrt{\sum_{k=1}^n v_k^2}$$$$||\mathbf{v}||_\infty = \max |{v_k}|$$

We can use the inbuilt norm function to calculate norms:

In [4]:
norm([1,2,3])==norm([1,2,3],2)==sqrt(1^2+2^2+3^2)

Out[4]:
true
In [5]:
norm([1,-2,3],1)==1+2+3

Out[5]:
true
In [6]:
norm([1,-2,3],Inf)==3

Out[6]:
true

We will investigate these norms by plotting are the level curves for different norms. First, we discuss how to do a surface plot. The following plots $y*x^2$:

In [3]:
f(x,y)=y*x^2

# this is short hand for
function f(x,y)
y*x^2
end

x=y=linspace(-1.,1.,100)

z=Float64[
f(x[j],y[k])
for k=1:length(y), j=1:length(x)]

using PyPlot

surf(x,y,z)  # 3D plot;


Choosing f to be the 2-norm, we can see how the norm grows:

In [4]:
f(x,y)=norm([x,y],2)

x=y=linspace(-1.,1.,200)

z=Float64[
f(x[j],y[k])
for k=1:length(y), j=1:length(x)]

using PyPlot

surf(x,y,z)  # 3D plot;


It is helpful to plot a contour plot, to see where the curves of constant value are. Here, we see that the 2-norm forms circles:

In [7]:
f(x,y)=norm([x,y],2)

x=y=linspace(-1.,1.,200)

z=Float64[
f(x[j],y[k])
for k=1:length(y), j=1:length(x)]

using PyPlot

contour(x,y,z)  # contour plot of f

gcf()[:set_size_inches](4,4)   # make the plot a square;


The ∞-norm has squares of constant norm:

In [8]:
f(x,y)=norm([x,y],Inf)

x=y=linspace(-1.,1.,200)

z=Float64[
f(x[j],y[k])
for k=1:length(y), j=1:length(x)]

using PyPlot

contour(x,y,z)  # 3D plot;

gcf()[:set_size_inches](4,4)   # make the plot a square


The 1-norm has diamonds:

In [9]:
f(x,y)=norm([x,y],1)

x=y=linspace(-1.,1.,200)

z=Float64[
f(x[j],y[k])
for k=1:length(y), j=1:length(x)]

using PyPlot

contour(x,y,z)  # 3D plot;

gcf()[:set_size_inches](4,4)   # make the plot a square


More generally, the p-norm $$||\mathbf{v}||_p = \left(\sum_{k=1}^n |v_k|^p\right)^{1/p}$$ Is between a circle and a square:

In [10]:
p=5

f(x,y)=norm([x,y],p)

x=y=linspace(-1.,1.,200)

z=Float64[
f(x[j],y[k])
for k=1:length(y), j=1:length(x)]

contour(x,y,z)  # 3D plot;
gcf()[:set_size_inches](4,4)   # make the plot a square


We can also weight the norm using a diagonal matrix. For the 2-norm, this changes circles to ellipses:

In [11]:
f(x,y)=norm([2 0; 0 1]*[x,y],2)

x=y=linspace(-1.,1.,200)

z=Float64[
f(x[j],y[k])
for k=1:length(y), j=1:length(x)]

contour(x,y,z)  # 3D plot;
gcf()[:set_size_inches](4,4)   # make the plot a square


# \ and least squares¶

x=A\b

will find x so that norm(A*x-b) is minimized: that is, we are finding the vector that finds the best approximation to the equation in the 2-norm. This is called the least squares solution.

In [13]:
A=rand(10,5)
b=rand(10)

x=A\b

Out[13]:
5-element Array{Float64,1}:
0.00826066
0.151079
0.881139
0.144133
0.350968  

We thus know norm(A*x-b) should be the smallest possible value achievable by any vector x.

In [14]:
minnorm=norm(A*x-b)

Out[14]:
0.4485301788323935

We can check that this is true by sampling many random vectors r, and double checking that norm(A*r-b) is greater than minnorm. This sort of experiment is known as Monte–Carlo simulation.

In [15]:
randomminnorm=Inf   # start the smallest sampled norm at Inf

for k=1:1000000   # we will do a million trials
r=rand(5)    # create a random vector of size 5
newnrm=norm(A*r-b)     # Check ||A*r-b|| for  the random vector r r
randomminnorm = min(newnrm,randomminnorm)   # the minimal sample norm is the min
end
randomminnorm  # this is greater than minnorm

Out[15]:
0.45086413589047813