1. imgaug-doc
  2. notebooks

Augment Polygons

imgaug has native support for polygon augmentation (currently in Beta state).

Polygons consist of corner points that are connected by a line, which encapsulates an inner polygon area. For a polygon to be valid, the line must not intersect with itself, must not overlap and the inner area must be non-zero (which also means that a valid polygon must have at least three corner points).

API

The following classes and methods are relevant for polygon augmentation:

API: Polygon

imgaug.augmentables.polys.Polygon(exterior, [label]): Container for a single polygon. The exterior is the "border" of the polygon and is made up of corner points, e.g. Polygon([(0, 0), (10, 0), (10, 10)]) creates a triangle (each point is given as (x, y) in absolute (sub-)pixel coordinates.

  • Properties offered by Polygon are: .exterior (all corner points), .label (the polygon's class label, may be None), .xx (x-coordinates of all corner points), .yy (analogous), .xx_int (.xx rounded to integers), .yy_int (analogous), ( .height (height of the polygon), .width (analogous), .area (area of the polygon) and .is_valid (returns whether the polygon is valid, i.e. has no self-intersections or overlapping segments).
  • Methods offered by Polygon are:
    • project(from_shape, to_shape): Project the polygon from image shape from_shape onto image shape to_shape, i.e. change the corner point coordinates. This is useful when resizing images.
    • find_closest_point_index(x, y, return_distance=False): For a given coordinate, find the index of the closest corner point (according to euclidean distance).
    • is_fully_within_image(image): Returns whether the whole polygon area is inside the image plane.
    • is_partly_within_image(image): Returns whether at least some parts of the polygon area are inside the image plane.
    • is_out_of_image(image, fully=True, partly=False): Returns whether the whole polygon area is outside of the image plane (fully=True) or some of that area (partly=True) or either of these cases (fully=True, partly=True).
    • clip_out_of_image(image): Clips off parts of the polygon that are outside of the image plane.
    • shift(top=None, right=None, bottom=None, left=None): Shift the polygon from the top/right/bottom/left side, i.e. move it around on the image.
    • draw_on_image(image, color=(0, 255, 0), color_face=None, color_lines=None, color_points=None, alpha=1.0, alpha_face=None, alpha_lines=None, alpha_points=None, size=1, size_lines=None, size_points=None, raise_if_out_of_image=False): Draw the polygon on a given image. This will draw an inner area ("face"), a border ("line") and the corner points ("points").
    • extract_from_image(image): Extract pixels within the polygon from an image. Returns a rectangle that matches the dimensions of a bounding box around the polygon. All pixels that were not inside the polygon are set to zero (i.e. black).
    • change_first_point_by_coords(x, y, max_distance=1e-4, raise_if_too_far_away=True): Reorders the points in .exterior so that the closest one to a given coordinate becomes the first point.
    • change_first_point_by_index(point_idx): Reorders the points in .exterior so that the one with a given index becomes the first point.
    • to_shapely_polygon(): Converts this Polygon instance to a shapely.geometry.Polygon instance.
    • to_shapely_line_string(closed=False, interpolate=0): Converts this Polygon instance to a shapely.geometry.LineString instance.
    • to_bounding_box(): Converts this Polygon instance to an imgaug.augmentables.bbs.BoundingBox instance.
    • to_keypoints(): Converts this Polygon's corner points to imgaug.augmentables.kps.Keypoint instances.
    • from_shapely(polygon_shapely, label=None): Creates an imgaug.augmentables.polys.Polygon instance from a shapely.geometry.Polygon instance.
    • exterior_almost_equals(other_polygon, max_distance=1e-6, points_per_edge=8): Compares this polygon's exterior with another polygon's exterior (may be a Polygon instance or its corner points). Returns True if the exteriors are practically identical.
    • almost_equals(other, max_distance=1e-6, points_per_edge=8): Same as exterior_almost_equals() but also compares .label.
    • copy(exterior=None, label=None): Currently an alias for .deepcopy().
    • deepcopy(self, exterior=None, label=None): Creates a deep copy of this polygon instance. If exterior or label are not None, they will be assigned to the new instance.

API: PolygonsOnImage

imgaug.augmentables.polys.PolygonsOnImage(polygons, shape): Container for a set of polygons (imgaug.augmentables.polys.Polygon instances) placed on a single image. The image's shape must be provided as a tuple via the argument shape and is required during the augmentation to align polygon and image augmentation (e.g. to sample corresponding crop values).

  • Properties of PolygonsOnImage are: .polygons, .shape, .empty (same as len(.polygons) == 0).
  • Methods of PolygonsOnImage are:
    • on(self, image):: Calls project(...) on each polygon in .polygons.
    • draw_on_image(...): Calls draw_on_image(...) on each polygon in .polygons.
    • remove_out_of_image(self, fully=True, partly=False): Removes all polygons from .polygons that are partially and/or fully outside of the image plane.
    • clip_out_of_image(): Calls clip_out_of_image() on each polygon in .polygons. This can increase the number of polygons in .polygons.
    • shift(...): Calls shift(...) on each polygon in .polygons.
    • copy(): Currently an alias for .deepcopy().
    • deepcopy(): Creates a deep copy of this instance.

API: Augmentation Methods

Polygons can be augmented using augment(images=<image data>, polygons=<data>), which is offered by all augmenters. <data> is fairly tolerant and accepts many different inputs, e.g. a list of lists of Polygon or a list of list of xy-coordinate-arrays. Alternatively, augment_polygons(polygons_on_image) can be used, which is also offered by all augmenters. It expects either a single instance of PolygonsOnImage or a list of it. It does not accept Polygon instances, because these lack the necessary image shape information in the form of .shape.

Note that only augmentations that change the geometry affect polygons, e.g. affine transformations, cropping or horizontal flipping. Other augmentations, e.g. gaussian noise, only affect images.

API: Docs

The API contains further details about polygon classes and methods, see e.g. Polygon, PolygonsOnImage, Augmenter.augment() and Augmenter.augment_polygons().

Placing Polygons on an Image

To show off the polygon augmentation functionality, we need an example image with some polygons on it. Let's load an image from wikipedia:

In [1]:
import imageio
import imgaug as ia
%matplotlib inline

image = imageio.imread("https://upload.wikimedia.org/wikipedia/commons/9/9a/Meerkat_%28Suricata_suricatta%29_Tswalu.jpg")
image = ia.imresize_single_image(image, 0.25)
print(image.shape)

ia.imshow(image)

(571, 856, 3)

We will now create three polygons, one for each of the three meerkats. Each polygon is created as Polygon(exterior), where exterior is a list of absolute (sub-)pixel xy-coordinates (given as tuples) describing the polygon corner points. Finding these corner points manually is a bit tedious. Usually they are supposed to be part of the dataset.

In [2]:
from imgaug.augmentables.polys import Polygon, PolygonsOnImage

# left meerkat
meerkat_left = Polygon([
    (350, 100),  # top left
    (390, 85),  # top
    (435, 110),  # top right
    (435, 170),
    (445, 250),  # right elbow
    (430, 290),  # right hip
    (440, 300),
    (420, 340),
    (440, 400),
    (410, 450),  # right foot
    (320, 430),
    (280, 410),  # left foot
    (300, 350),
    (300, 210),  # left elbow
    (340, 160),
    (325, 140)  # nose
])

# center meerkat
meerkat_center = Polygon([
    (430, 120),  # left top (nose)
    (510, 90),  # top
    (550, 95),  # top right
    (570, 120),  # ear
    (600, 230),
    (600, 450),
    (560, 510),  # bottom right
    (450, 480),  # bottom left
    (430, 340),
    (450, 250),  # elbow
    (500, 165),  # neck
    (430, 145)
])

# right meerkat
meerkat_right = Polygon([
    (610, 95),  # nose top
    (650, 60),  # top
    (680, 50),  # top
    (710, 60),
    (730, 80),  # top right
    (730, 140),
    (755, 220),
    (750, 340),
    (730, 380),
    (740, 420),
    (715, 560),  # right foot
    (690, 550),
    (680, 470),
    (640, 530),
    (590, 500),  # left foot
    (605, 240),  # left elbow
    (655, 130),  # neck
    (620, 120),  # mouth, bottom
])

Now that we have the three polygons created, let's draw them on the image. We use the method Polygon.draw_on_image(image) to do that and imgaug.imshow() to show the drawn image.

In [3]:
import numpy as np
image_polys = np.copy(image)
image_polys = meerkat_left.draw_on_image(image_polys, alpha_face=0.2, size_points=7)
image_polys = meerkat_center.draw_on_image(image_polys, alpha_face=0.2, size_points=7)
image_polys = meerkat_right.draw_on_image(image_polys, alpha_face=0.2, size_points=7)
ia.imshow(image_polys)