This notebook provides a basic tutorial on Active Template Models (ATMs). ATMs are used to perform image alignment driven by a statistical shape model.
The first step in building our ATM will be to import all the annotations of the training set of LFPW database. Luckily, Menpo
's io
module allows us to do exactly that using only a few lines of code:
from pathlib import Path
path_to_lfpw = Path('/vol/atlas/databases/lfpw')
import menpo.io as mio
training_shapes = []
# load landmarked images
for i in mio.import_landmark_files(path_to_lfpw / 'trainset' / '*.pts', verbose=True):
training_shapes.append(i['all'])
Found 811 assets, index the returned LazyList to import.
The previous cell loads all the PTS files of the LFPW.
The Menpo
ecosystem is well equiped with a series of predefined IPython Notebook widgets for the most common data visualization tasks. In order to check if the data has been correctly imported we will use the visualize_pointclouds
widget. Note that menpowidgets must be installed in order to use the widgets functionality.
%matplotlib inline
from menpowidgets import visualize_pointclouds
visualize_pointclouds(training_shapes)
In order to build an ATM, we need to load a template image. This will be the first image of the LFPW training set.
template_image = mio.import_image(path_to_lfpw / 'trainset' / 'image_0001.png')
template_image.view_landmarks();
It is very easy to build the ATM
from menpofit.atm import HolisticATM
# build ATM
atm = HolisticATM(
template_image,
training_shapes,
group='PTS',
verbose=True,
diagonal=150
)
- Computing reference shape Computing batch 0 - Building models - Scale 0: Done - Scale 1: Done
/home/nontas/Documents/Research/menpofit/menpofit/builder.py:338: MenpoFitModelBuilderWarning: The reference shape passed is not a TriMesh or subclass and therefore the reference frame (mask) will be calculated via a Delaunay triangulation. This may cause small triangles and thus suboptimal warps. MenpoFitModelBuilderWarning)
As first class citizens of Menpo
, AAMs can be printed just like any other Menpo
object (e.g. Images
or PointClouds)
:
print(atm)
Holistic Active Template Model - Images warped with DifferentiablePiecewiseAffine transform - Images scaled to diagonal: 150.00 - Scales: [0.5, 1.0] - Scale 0.5 - Holistic feature: no_op - Template shape: (59, 60) - Shape model class: OrthoPDM - 132 shape components - 4 similarity transform parameters - Scale 1.0 - Holistic feature: no_op - Template shape: (112, 113) - Shape model class: OrthoPDM - 132 shape components - 4 similarity transform parameters
We can also visualize the ATM model
atm.view_atm_widget()
In menpofit
, ATMs can be fitted to images by creating Fitter
objects around them.
One of the most popular and well known family of algorithms for fitting ATMs is the one based around the original Lucas-Kanade
algorithm for Image Alignment. In order to fit our AAM using an algorithms of the previous family, Menpo
allows the user to define a LucasKanadeATMFitter
object. Again, using a single line of code!!!
from menpofit.atm import LucasKanadeATMFitter
# define Lucas-Kanade based ATM fitter
fitter = LucasKanadeATMFitter(atm, n_shape=[6, 12])
The previous cell has created a LucasKanadeATMFitter
that will fit images using 3, 6 and 12 shape compenents per level.
Fitting a LucasKanadeATMFitter
to an image is as simple as calling its fit
method. Let's try it by fitting the second image of the LFPW trainset!!!
test_image = mio.import_image(path_to_lfpw / 'trainset' / 'image_0002.png')
test_image.view_widget()
from menpofit.fitter import noisy_shape_from_bounding_box
# obtain groubnd truth (original) landmarks
gt_s = test_image.landmarks['PTS'].lms
# generate initialization landmarks
initial_s = noisy_shape_from_bounding_box(gt_s, gt_s.bounding_box())
# fit image
fitting_result = fitter.fit_from_shape(test_image, initial_s, gt_shape=gt_s)
# print fitting error
print(fitting_result)
Fitting result of 68 landmark points. Initial error: 0.0356 Final error: 0.0220
Let's visualize the result using the widget
fitting_result.view_widget()