Sonification on an Image by using image data as the spectrum on which an STFT was performed. To perform the STFT , the image data is extracted and stored as 1D arrays. These 1D arrays are accessed one window at a time and their corresponding fft is computed and stored in the final_sig.
%pylab inline
from __future__ import print_function
from __future__ import division
import IPython
Populating the interactive namespace from numpy and matplotlib
from scipy.io import wavfile
img = imread('image2.png')
img2= imread('image4.jpg')
Using a spiral grayscale image to create more pronounced sounds
imshow(img)
<matplotlib.image.AxesImage at 0x10c1de950>
Use the green channel flattened array of the image as the magnitude spectrum.
## Flattening the image (2D) into a (1D) array
sig_mag = array((img[:,:,1]).flat)
sig_phs = (array((img[:,:,0]).flat) + array((img[:,:,2]).flat))
#computing the length of the array
a= len(sig_mag)
# making a window of size 1/100th of the complete length of the arrays
windowsize = a/100
win_start = arange(0, a, windowsize)
mag_spec = []
phs_spec = []
final_sig = []
#computing ffts per window and adding on to the final signal final_sig
for i in win_start:
mag_spec = sig_mag[i : i + windowsize]
phs_spec = sig_phs[i : i + windowsize]
X = [np.complex(cos(phs)* mag, -sin(phs)* mag) for mag, phs in zip(mag_spec, phs_spec)]
x= fft.irfft(X)
final_sig = final_sig + list (x*100)
plot(final_sig)
/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:17: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future /usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:18: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
[<matplotlib.lines.Line2D at 0x10a8859d0>]
Use the sum of blue and red channel flattened array of the image as the phase spectrum.
from IPython.display import Audio
Audio(final_sig,rate=16000)
imshow(img2)
<matplotlib.image.AxesImage at 0x111269c50>
## Flattening the image (2D) into a (1D) array
sig_mag = array((img2[:,:,1]).flat)
sig_phs = (array((img2[:,:,0]).flat) + array((img2[:,:,2]).flat))
#computing the length of the array
a= len(sig_mag)
# making a window of size 1/100th of the complete length of the arrays
windowsize = a/100
win_start = arange(0, a, windowsize)
mag_spec = []
phs_spec = []
final_sig = []
#computing ffts per window and adding on to the final signal final_sig
for i in win_start:
mag_spec = sig_mag[i : i + windowsize]
phs_spec = sig_phs[i : i + windowsize]
X = [np.complex(cos(phs)* mag, -sin(phs)* mag) for mag, phs in zip(mag_spec, phs_spec)]
x= fft.irfft(X)
final_sig = final_sig + list (x*100)
plot(final_sig)
/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:17: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future /usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:18: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
[<matplotlib.lines.Line2D at 0x10bbe6890>]
Audio(final_sig,rate=16000)