imgaug has native support for polygon augmentation (currently in Beta state).
Polygons consist of corner points that are connected by a line, which encapsulates an inner polygon area. For a polygon to be valid, the line must not intersect with itself, must not overlap and the inner area must be non-zero (which also means that a valid polygon must have at least three corner points).
The following classes and methods are relevant for polygon augmentation:
imgaug.augmentables.polys.Polygon(exterior, [label]): Container for a single polygon. The exterior is the "border" of the polygon and is made up of corner points, e.g.
Polygon([(0, 0), (10, 0), (10, 10)]) creates a triangle (each point is given as
(x, y) in absolute (sub-)pixel coordinates.
.exterior(all corner points),
.label(the polygon's class label, may be
.xx(x-coordinates of all corner points),
.xxrounded to integers),
.height(height of the polygon),
.area(area of the polygon) and
.is_valid(returns whether the polygon is valid, i.e. has no self-intersections or overlapping segments).
project(from_shape, to_shape): Project the polygon from image shape
from_shapeonto image shape
to_shape, i.e. change the corner point coordinates. This is useful when resizing images.
find_closest_point_index(x, y, return_distance=False): For a given coordinate, find the index of the closest corner point (according to euclidean distance).
is_fully_within_image(image): Returns whether the whole polygon area is inside the image plane.
is_partly_within_image(image): Returns whether at least some parts of the polygon area are inside the image plane.
is_out_of_image(image, fully=True, partly=False): Returns whether the whole polygon area is outside of the image plane (
fully=True) or some of that area (
partly=True) or either of these cases (
clip_out_of_image(image): Clips off parts of the polygon that are outside of the image plane.
shift(top=None, right=None, bottom=None, left=None): Shift the polygon from the top/right/bottom/left side, i.e. move it around on the image.
draw_on_image(image, color=(0, 255, 0), color_face=None, color_lines=None, color_points=None, alpha=1.0, alpha_face=None, alpha_lines=None, alpha_points=None, size=1, size_lines=None, size_points=None, raise_if_out_of_image=False): Draw the polygon on a given image. This will draw an inner area ("face"), a border ("line") and the corner points ("points").
extract_from_image(image): Extract pixels within the polygon from an image. Returns a rectangle that matches the dimensions of a bounding box around the polygon. All pixels that were not inside the polygon are set to zero (i.e. black).
change_first_point_by_coords(x, y, max_distance=1e-4, raise_if_too_far_away=True): Reorders the points in
.exteriorso that the closest one to a given coordinate becomes the first point.
change_first_point_by_index(point_idx): Reorders the points in
.exteriorso that the one with a given index becomes the first point.
to_shapely_polygon(): Converts this
Polygoninstance to a
to_shapely_line_string(closed=False, interpolate=0): Converts this
Polygoninstance to a
to_bounding_box(): Converts this
Polygoninstance to an
to_keypoints(): Converts this
Polygon's corner points to
from_shapely(polygon_shapely, label=None): Creates an
imgaug.augmentables.polys.Polygoninstance from a
exterior_almost_equals(other_polygon, max_distance=1e-6, points_per_edge=8): Compares this polygon's exterior with another polygon's exterior (may be a
Polygoninstance or its corner points). Returns
Trueif the exteriors are practically identical.
almost_equals(other, max_distance=1e-6, points_per_edge=8): Same as
exterior_almost_equals()but also compares
copy(exterior=None, label=None): Currently an alias for
deepcopy(self, exterior=None, label=None): Creates a deep copy of this polygon instance. If
None, they will be assigned to the new instance.
imgaug.augmentables.polys.PolygonsOnImage(polygons, shape): Container for a set of polygons (
imgaug.augmentables.polys.Polygon instances) placed on a single image. The image's shape must be provided as a tuple via the argument
shape and is required during the augmentation to align polygon and image augmentation (e.g. to sample corresponding crop values).
len(.polygons) == 0).
on(self, image):: Calls
project(...)on each polygon in
draw_on_image(...)on each polygon in
remove_out_of_image(self, fully=True, partly=False): Removes all polygons from
.polygonsthat are partially and/or fully outside of the image plane.
clip_out_of_image()on each polygon in
.polygons. This can increase the number of polygons in
shift(...)on each polygon in
copy(): Currently an alias for
deepcopy(): Creates a deep copy of this instance.
Polygons can be augmented using
augment(images=<image data>, polygons=<data>), which is offered by all augmenters.
<data> is fairly tolerant and accepts many different inputs, e.g. a list of lists of
Polygon or a list of list of xy-coordinate-arrays. Alternatively,
augment_polygons(polygons_on_image) can be used, which is also offered by all augmenters. It expects either a single instance of
PolygonsOnImage or a list of it. It does not accept
Polygon instances, because these lack the necessary image shape information in the form of
Note that only augmentations that change the geometry affect polygons, e.g. affine transformations, cropping or horizontal flipping. Other augmentations, e.g. gaussian noise, only affect images.
To show off the polygon augmentation functionality, we need an example image with some polygons on it. Let's load an image from wikipedia:
import imageio import imgaug as ia %matplotlib inline image = imageio.imread("https://upload.wikimedia.org/wikipedia/commons/9/9a/Meerkat_%28Suricata_suricatta%29_Tswalu.jpg") image = ia.imresize_single_image(image, 0.25) print(image.shape) ia.imshow(image)
(571, 856, 3)
We will now create three polygons, one for each of the three meerkats. Each polygon is created as
exterior is a list of absolute (sub-)pixel xy-coordinates (given as tuples) describing the polygon corner points. Finding these corner points manually is a bit tedious. Usually they are supposed to be part of the dataset.
from imgaug.augmentables.polys import Polygon, PolygonsOnImage # left meerkat meerkat_left = Polygon([ (350, 100), # top left (390, 85), # top (435, 110), # top right (435, 170), (445, 250), # right elbow (430, 290), # right hip (440, 300), (420, 340), (440, 400), (410, 450), # right foot (320, 430), (280, 410), # left foot (300, 350), (300, 210), # left elbow (340, 160), (325, 140) # nose ]) # center meerkat meerkat_center = Polygon([ (430, 120), # left top (nose) (510, 90), # top (550, 95), # top right (570, 120), # ear (600, 230), (600, 450), (560, 510), # bottom right (450, 480), # bottom left (430, 340), (450, 250), # elbow (500, 165), # neck (430, 145) ]) # right meerkat meerkat_right = Polygon([ (610, 95), # nose top (650, 60), # top (680, 50), # top (710, 60), (730, 80), # top right (730, 140), (755, 220), (750, 340), (730, 380), (740, 420), (715, 560), # right foot (690, 550), (680, 470), (640, 530), (590, 500), # left foot (605, 240), # left elbow (655, 130), # neck (620, 120), # mouth, bottom ])
Now that we have the three polygons created, let's draw them on the image. We use the method
Polygon.draw_on_image(image) to do that and
imgaug.imshow() to show the drawn image.
import numpy as np image_polys = np.copy(image) image_polys = meerkat_left.draw_on_image(image_polys, alpha_face=0.2, size_points=7) image_polys = meerkat_center.draw_on_image(image_polys, alpha_face=0.2, size_points=7) image_polys = meerkat_right.draw_on_image(image_polys, alpha_face=0.2, size_points=7) ia.imshow(image_polys)