MAT 201A Winter | Jing Yan | theuniqueeye@gmail.com
The program aims to use cross-correlation, maximum_filter to detect certain patterns in the image.
By detecting 3 distinctive patterns, I am going to explore the difference in procedure and results of cross-correlation.
Load original image, add the 3 layers and do average.
Load the template image, add the 3 layers and do average, do the threshold.
Do the same threshold to the original image.
Do the correlation of the original image and the template.
Find the position of the biggest correlation value.
Show the correlation in image, set threshold to clear the image.
Use the maximum_filter and minimum_filter to get a clear view of the similarity and difference.
Redo the above steps with different templates, change some values to explore different results.
# import some libraries
%pylab inline
from __future__ import print_function
from __future__ import division
# set the size of image by defalut to (16,9)
rcParams['figure.figsize'] = (16, 9)
Populating the interactive namespace from numpy and matplotlib
# load the original image.
img = imread('Simpsons.png')
imshow(img)
# grid()
<matplotlib.image.AxesImage at 0x11381a7d0>
It's a group picture from Simpson's Family.
I choose this image to do detection for some particular reasons.
The image is quite complicated and has lots of details.
There are also many similar characters which makes the detection harder and more meaningful.
img.shape
(1244, 2400, 3)
img.dtype
dtype('float32')
# by average the sum of R, G, B pixel values,
# I turn the matrix of pixel values into a list of numbers.
imgGray = sum(img.astype(float), axis=2)/3.0
imshow(imgGray)
colorbar()
<matplotlib.colorbar.Colorbar at 0x11c482250>
# try to detect 3 different elements in the original image.
head = imread('head.png')
elephant = imread('elephant.png')
alien = imread('alien.png')
gcf().set_figheight(3)
subplot(131)
title('Homer head')
imshow(head)
subplot(132)
title('Elephant')
imshow(elephant)
subplot(133)
title('Drolling Alien')
imshow(alien)
<matplotlib.image.AxesImage at 0x10dfa5b10>
# load the template_1: << Homer's head >>
# It's much harder to distinguished Homer's head than distinguished an elephant.
# Because the people in image are very similar to each other.
# Thus the setting of threshold of the template is very important.
# You should make sure to render the main character of this template
# and eliminate the excess details. It's a balance.
head = imread('head.png')
gcf().set_figheight(3)
imshow(head)
print(head.shape)
(66, 49, 3)
# average the sum of pixel values of all 3 layers of the template.
headGray = sum(head.astype(float), axis=2)/3.0
gcf().set_figheight(3)
imshow(headGray)
colorbar()
<matplotlib.colorbar.Colorbar at 0x129e794d0>
# setting the threshold to the template. [better version for detect]
headGray = where(headGray > 0.7, 1, -1)
gcf().set_figheight(3)
title('template with threshold 0.7')
imshow(headGray,cmap=cm.gray,interpolation='nearest')
colorbar()
<matplotlib.colorbar.Colorbar at 0x11509e950>
# setting the threshold to the template. [You will fail to detect in future.]
headGray2 = where(headGray > 0.5, 1, -1)
gcf().set_figheight(3)
title('template with threshold 0.5')
imshow(headGray2,cmap=cm.gray,interpolation='nearest')
colorbar()
<matplotlib.colorbar.Colorbar at 0x12aea7a50>
[ notes ] Setting the threshold of the template:¶
The value of threshold is very important.
You should make sure to render the main character of this template
and eliminate the excess details.
For example, if you set the threshold to 0.5 that is the second version,
you might fail to detect anything in the later cross-correlation.
In my point of view, the reason is because there is too many white part
in this template which makes the pattern not that remarkable by comparing.
# setting the same threshold to the original image.
imgGray = where(imgGray > 0.7, 1, -1)
title('original image with threshold 0.7')
imshow(imgGray,cmap=cm.gray,interpolation='nearest')
colorbar()
<matplotlib.colorbar.Colorbar at 0x116f02410>
# Doing [2D cross-correlation] using the template and original image.
from scipy.signal import correlate2d
cc = correlate2d(imgGray, headGray)
title('correlation of Homer and original image')
imshow(cc)
colorbar()
<matplotlib.colorbar.Colorbar at 0x12fea5b10>
# This part is used to get the largest number from cross-correlation,
# and the index of where that point lies in.
# So the real Homer located at point(629,1246) in the coordinate.
max = argmax(cc)
print(max)
unravel_index(max, cc.shape)
1541038
(629, 1246)
# And the minimum number...
# The most different part in the original image to the template.
min = argmin(cc)
print(min)
unravel_index(min, cc.shape)
2192626
(895, 1666)
Something interesting about the choice of threshold:
Except from detecting the element, you can also find the similarity in image by setting different thresholds.
# there are a lot of matches when setting the threshold to 1000,
# which makes sense, because as you look into the orignal picture
# those guys appear are pretty similar.
match = where(cc > 1000, 1, 0)
title('correlation with threshold 1000')
imshow(match,cmap=cm.gray)
<matplotlib.image.AxesImage at 0x10ee9f990>
# Then I try to higher the threshold and set it to 1200.
# There are only three matches.
# And I look into the picture to find out these 3 parts.
# The largest one in the middle is the template that I used.
match = where(cc > 1200, 1, 0)
title('correlation with threshold 1200')
imshow(match,cmap=cm.gray)
<matplotlib.image.AxesImage at 0x10ee9f890>
# The middle one of the correlation looks very similar to Homer.
# He looks really like a fake Homer with a wig.
imshow(np.copy(img[920:1000, 2225:2290]))
gcf().set_figheight(3)
# After I set the threshold to 1400, the only one left is the real Homer.
match = where(cc > 1400, 1, 0)
title('correlation with threshold 1400')
imshow(match,cmap=cm.gray)
<matplotlib.image.AxesImage at 0x116f1acd0>
# use the maximum_filter to show the match up area more clearly.
from scipy.ndimage.filters import maximum_filter
title('correlation with maximum_filter 10*10')
imshow(maximum_filter(cc, (10,10)))
gcf().set_figheight(8)
subplot(121)
title('correlation with maximum_filter 50*50')
imshow(maximum_filter(cc, (50,50))) # set a larger square size
subplot(122)
title('original image')
imshow(img, cmap=cm.gray)
gcf().set_figheight(8)
It is shown clearly that the best match locates in the middle of the oringinal image.
Comparing to simply show the image of correlation, the maximum filter shows the pattern better.
And it seperates the level of similarity larger.
# use the minimun_filter to show the distinctive area.
from scipy.ndimage.filters import minimum_filter
subplot(121)
title('correlation with minimum_filter 50*50')
imshow(minimum_filter(cc, (50,50))) # set a larger square size
subplot(122)
title('original image')
imshow(img, cmap=cm.gray)
gcf().set_figheight(8)
# The most distinctive part are shown in blue in the following image.
# load the template_2: << Elephant >>
elephant = imread('elephant.png')
gcf().set_figheight(3)
imshow(elephant)
print(elephant.shape)
(135, 114, 3)
# average the sum of pixel values of all 3 layers of the template.
elephantGray = sum(elephant.astype(float), axis=2)/3.0
gcf().set_figheight(3)
imshow(elephantGray)
colorbar()
<matplotlib.colorbar.Colorbar at 0x12ae26790>
# setting the threshold to the template.
elephantGray = where(elephantGray > 0.5, 1, -1)
gcf().set_figheight(3)
title('template with threshold 0.5')
imshow(elephantGray,cmap=cm.gray,interpolation='nearest')
colorbar()
<matplotlib.colorbar.Colorbar at 0x12a435410>
# setting the same threshold to the original image.
imgGray = where(imgGray > 0.5, 1, -1)
title('original image with threshold 0.5')
imshow(imgGray,cmap=cm.gray,interpolation='nearest')
colorbar()
<matplotlib.colorbar.Colorbar at 0x118baf790>
# Doing 2D cross-correlation using the template and original image.
from scipy.signal import correlate2d
cc = correlate2d(imgGray, elephantGray)
title('correlation of elephant and original image')
imshow(cc)
colorbar()
<matplotlib.colorbar.Colorbar at 0x116e6ed50>
As the image of elephant is bigger, it takes longer time to do the correlation.
# get the largest number from cross-correlation,
# and the index of where that point lies in.
max = argmax(cc)
print(max)
unravel_index(max, cc.shape)
1431164
(569, 1267)
# set the threshold to make the image of correlation clear.
match = where(cc > 4000, 1, 0)
title('correlation with threshold 4000')
imshow(match,cmap=cm.gray)
<matplotlib.image.AxesImage at 0x11a02b290>
imshow(np.copy(img[1000:1150, 800:930]))
gcf().set_figheight(3)
Beacuse there are a lot of white areas in the image of elephant's head, so it seems similar to a white square.
And in the correlation, it shows up the chief's belly looks also alike to the template.
So the threshold of the template should be set again to lay more emphasis on the colorful part than the white area.
# use the maximum_filter to show the match up area more clearly.
subplot(121)
title('correlation with maximum_filter 50*50')
imshow(maximum_filter(cc, (50,50))) # set a larger square size
subplot(122)
title('original image')
imshow(img, cmap=cm.gray)
gcf().set_figheight(8)
# load the template_3: << Drolling Alien >>
alien = imread('alien.png')
gcf().set_figheight(3)
imshow(alien)
print(alien.shape)
(131, 96, 3)
# average the sum of pixel values of all 3 layers of the template.
alienGray = sum(alien.astype(float), axis=2)/3.0
gcf().set_figheight(3)
imshow(alienGray)
colorbar()
<matplotlib.colorbar.Colorbar at 0x12a62a710>
# setting the threshold to the template.
alienGray = where(alienGray > 0.7, 1, -1)
gcf().set_figheight(3)
title('template with threshold 0.7')
imshow(alienGray,cmap=cm.gray,interpolation='nearest')
colorbar()
<matplotlib.colorbar.Colorbar at 0x118c33590>
Like the Homer's example, the threshold should be set higher (ex, 0.7) for detection.
Otherwise, it will show some wrong results.
# setting the same threshold to the original image.
imgGray = where(imgGray > 0.7, 1, -1)
title('original image with threshold 0.7')
imshow(imgGray,cmap=cm.gray,interpolation='nearest')
colorbar()
<matplotlib.colorbar.Colorbar at 0x12974cc50>
# Doing 2D cross-correlation using the template and original image.
from scipy.signal import correlate2d
cc = correlate2d(imgGray, alienGray)
title('correlation of alien and original image')
imshow(cc)
colorbar()
<matplotlib.colorbar.Colorbar at 0x12a6c6110>
# get the largest number from cross-correlation,
# and the index of where that point lies in.
max = argmax(cc)
print(max)
unravel_index(max, cc.shape)
734506
(294, 976)
# set the threshold to make the image of correlation clear.
match = where(cc > 5000, 1, 0)
title('correlation with threshold 5000')
imshow(match,cmap=cm.gray)
<matplotlib.image.AxesImage at 0x12991d350>
# use the maximum_filter to show the match up area more clearly.
subplot(121)
title('correlation with maximum_filter 50*50')
imshow(maximum_filter(cc, (50,50)))
subplot(122)
imshow(img, cmap=cm.gray)
title('original image')
gcf().set_figheight(8)