MAT-201A ASSIGNMENT 2

Ambika Yadav

Sonification on an Image by using image data as the spectrum on which an STFT was performed. To perform the STFT , the image data is extracted and stored as 1D arrays. These 1D arrays are accessed one window at a time and their corresponding fft is computed and stored in the final_sig.

In [2]:
%pylab inline
from __future__ import print_function
from __future__ import division
import IPython
Populating the interactive namespace from numpy and matplotlib
In [3]:
from scipy.io import wavfile
In [17]:
img = imread('image2.png')
img2= imread('image4.jpg')

Using a spiral grayscale image to create more pronounced sounds

In [13]:
imshow(img)
Out[13]:
<matplotlib.image.AxesImage at 0x10c1de950>

Use the green channel flattened array of the image as the magnitude spectrum.

In [7]:
## Flattening the image (2D) into a (1D) array
sig_mag = array((img[:,:,1]).flat)
sig_phs = (array((img[:,:,0]).flat) + array((img[:,:,2]).flat))

#computing the length of the array
a= len(sig_mag)
# making a window of size 1/100th of the complete length of the arrays
windowsize = a/100
win_start = arange(0, a, windowsize)

mag_spec = []
phs_spec = []
final_sig = [] 

#computing ffts per window and adding on to the final signal final_sig
for i in win_start:
    mag_spec = sig_mag[i : i + windowsize]
    phs_spec = sig_phs[i : i + windowsize]
    X = [np.complex(cos(phs)* mag, -sin(phs)* mag) for mag, phs in zip(mag_spec, phs_spec)]
    x= fft.irfft(X) 
    final_sig = final_sig + list (x*100)
plot(final_sig)
    
    
/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:17: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:18: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
Out[7]:
[<matplotlib.lines.Line2D at 0x10a8859d0>]

Use the sum of blue and red channel flattened array of the image as the phase spectrum.

In [8]:
from IPython.display import Audio
Audio(final_sig,rate=16000)
Out[8]:
In [18]:
imshow(img2)
Out[18]:
<matplotlib.image.AxesImage at 0x111269c50>
In [19]:
## Flattening the image (2D) into a (1D) array
sig_mag = array((img2[:,:,1]).flat)
sig_phs = (array((img2[:,:,0]).flat) + array((img2[:,:,2]).flat))

#computing the length of the array
a= len(sig_mag)
# making a window of size 1/100th of the complete length of the arrays
windowsize = a/100
win_start = arange(0, a, windowsize)

mag_spec = []
phs_spec = []
final_sig = [] 

#computing ffts per window and adding on to the final signal final_sig
for i in win_start:
    mag_spec = sig_mag[i : i + windowsize]
    phs_spec = sig_phs[i : i + windowsize]
    X = [np.complex(cos(phs)* mag, -sin(phs)* mag) for mag, phs in zip(mag_spec, phs_spec)]
    x= fft.irfft(X) 
    final_sig = final_sig + list (x*100)
plot(final_sig)
    
/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:17: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:18: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
Out[19]:
[<matplotlib.lines.Line2D at 0x10bbe6890>]
In [20]:
Audio(final_sig,rate=16000)
Out[20]: