imgaug
has native support for polygon augmentation (currently in Beta state).
Polygons consist of corner points that are connected by a line, which encapsulates an inner polygon area. For a polygon to be valid, the line must not intersect with itself, must not overlap and the inner area must be non-zero (which also means that a valid polygon must have at least three corner points).
The following classes and methods are relevant for polygon augmentation:
imgaug.augmentables.polys.Polygon(exterior, [label])
: Container for a single polygon. The exterior is the "border" of the polygon and is made up of corner points, e.g. Polygon([(0, 0), (10, 0), (10, 10)])
creates a triangle (each point is given as (x, y)
in absolute (sub-)pixel coordinates.
Polygon
are: .exterior
(all corner points), .label
(the polygon's class label, may be None
), .xx
(x-coordinates of all corner points), .yy
(analogous), .xx_int
(.xx
rounded to integers), .yy_int
(analogous), ( .height
(height of the polygon), .width
(analogous), .area
(area of the polygon) and .is_valid
(returns whether the polygon is valid, i.e. has no self-intersections or overlapping segments).Polygon
are:project(from_shape, to_shape)
: Project the polygon from image shape from_shape
onto image shape to_shape
, i.e. change the corner point coordinates. This is useful when resizing images.find_closest_point_index(x, y, return_distance=False)
: For a given coordinate, find the index of the closest corner point (according to euclidean distance).is_fully_within_image(image)
: Returns whether the whole polygon area is inside the image plane.is_partly_within_image(image)
: Returns whether at least some parts of the polygon area are inside the image plane.is_out_of_image(image, fully=True, partly=False)
: Returns whether the whole polygon area is outside of the image plane (fully=True
) or some of that area (partly=True
) or either of these cases (fully=True, partly=True
).clip_out_of_image(image)
: Clips off parts of the polygon that are outside of the image plane.shift(x=0, y=0)
: Shift the polygon along the x/y axis.draw_on_image(image, color=(0, 255, 0), color_face=None, color_lines=None, color_points=None, alpha=1.0, alpha_face=None, alpha_lines=None, alpha_points=None, size=1, size_lines=None, size_points=None, raise_if_out_of_image=False)
: Draw the polygon on a given image. This will draw an inner area ("face"), a border ("line") and the corner points ("points").extract_from_image(image)
: Extract pixels within the polygon from an image. Returns a rectangle that matches the dimensions of a bounding box around the polygon. All pixels that were not inside the polygon are set to zero (i.e. black).change_first_point_by_coords(x, y, max_distance=1e-4, raise_if_too_far_away=True)
: Reorders the points in .exterior
so that the closest one to a given coordinate becomes the first point.change_first_point_by_index(point_idx)
: Reorders the points in .exterior
so that the one with a given index becomes the first point.subdivide(points_per_edge)
: Interpolate points_per_edge
points on each polygon edge.to_shapely_polygon()
: Converts this Polygon
instance to a shapely.geometry.Polygon
instance.to_shapely_line_string(closed=False, interpolate=0)
: Converts this Polygon
instance to a shapely.geometry.LineString
instance.to_bounding_box()
: Converts this Polygon
instance to an imgaug.augmentables.bbs.BoundingBox
instance.to_keypoints()
: Converts this Polygon
's corner points to imgaug.augmentables.kps.Keypoint
instances.from_shapely(polygon_shapely, label=None)
: Creates an imgaug.augmentables.polys.Polygon
instance from a shapely.geometry.Polygon
instance.exterior_almost_equals(other_polygon, max_distance=1e-6, points_per_edge=8)
: Compares this polygon's exterior with another polygon's exterior (may be a Polygon
instance or its corner points). Returns True
if the exteriors are practically identical.almost_equals(other, max_distance=1e-6, points_per_edge=8)
: Same as exterior_almost_equals()
but also compares .label
.copy(exterior=None, label=None)
: Currently an alias for .deepcopy()
.deepcopy(self, exterior=None, label=None)
: Creates a deep copy of this polygon instance. If exterior
or label
are not None
, they will be assigned to the new instance.imgaug.augmentables.polys.PolygonsOnImage(polygons, shape)
: Container for a set of polygons (imgaug.augmentables.polys.Polygon
instances) placed on a single image. The image's shape must be provided as a tuple via the argument shape
and is required during the augmentation to align polygon and image augmentation (e.g. to sample corresponding crop values).
PolygonsOnImage
are: .polygons
, .shape
, .empty
(same as len(.polygons) == 0
).PolygonsOnImage
are:on(self, image):
: Calls project(...)
on each polygon in .polygons
.draw_on_image(...)
: Calls draw_on_image(...)
on each polygon in .polygons
.remove_out_of_image(self, fully=True, partly=False)
: Removes all polygons from .polygons
that are partially and/or fully outside of the image plane.clip_out_of_image()
: Calls clip_out_of_image()
on each polygon in .polygons
. This can increase the number of polygons in .polygons
.shift(...)
: Calls shift(...)
on each polygon in .polygons
.subdivide(...)
: Calls subdivide(...)
on each polygon in .polygons
.copy()
: Currently an alias for .deepcopy()
.deepcopy()
: Creates a deep copy of this instance.Polygons can be augmented using augment(images=<image data>, polygons=<data>)
, which is offered by all augmenters. <data>
is fairly tolerant and accepts many different inputs, e.g. a list of lists of Polygon
or a list of list of xy-coordinate-arrays. Alternatively, augment_polygons(polygons_on_image)
can be used, which is also offered by all augmenters. It expects either a single instance of PolygonsOnImage
or a list of it. It does not accept Polygon
instances, because these lack the necessary image shape information in the form of .shape
.
Note that only augmentations that change the geometry affect polygons, e.g. affine transformations, cropping or horizontal flipping. Other augmentations, e.g. gaussian noise, only affect images.
The API contains further details about polygon classes and methods, see e.g. Polygon, PolygonsOnImage, Augmenter.augment() and Augmenter.augment_polygons().
To show off the polygon augmentation functionality, we need an example image with some polygons on it. Let's load an image from wikipedia:
import imageio
import imgaug as ia
%matplotlib inline
image = imageio.imread("https://upload.wikimedia.org/wikipedia/commons/9/9a/Meerkat_%28Suricata_suricatta%29_Tswalu.jpg")
image = ia.imresize_single_image(image, 0.25)
print(image.shape)
ia.imshow(image)
We will now create three polygons, one for each of the three meerkats. Each polygon is created as Polygon(exterior)
, where exterior
is a list of absolute (sub-)pixel xy-coordinates (given as tuples) describing the polygon corner points. Finding these corner points manually is a bit tedious. Usually they are supposed to be part of the dataset.
from imgaug.augmentables.polys import Polygon, PolygonsOnImage
# left meerkat
meerkat_left = Polygon([
(350, 100), # top left
(390, 85), # top
(435, 110), # top right
(435, 170),
(445, 250), # right elbow
(430, 290), # right hip
(440, 300),
(420, 340),
(440, 400),
(410, 450), # right foot
(320, 430),
(280, 410), # left foot
(300, 350),
(300, 210), # left elbow
(340, 160),
(325, 140) # nose
])
# center meerkat
meerkat_center = Polygon([
(430, 120), # left top (nose)
(510, 90), # top
(550, 95), # top right
(570, 120), # ear
(600, 230),
(600, 450),
(560, 510), # bottom right
(450, 480), # bottom left
(430, 340),
(450, 250), # elbow
(500, 165), # neck
(430, 145)
])
# right meerkat
meerkat_right = Polygon([
(610, 95), # nose top
(650, 60), # top
(680, 50), # top
(710, 60),
(730, 80), # top right
(730, 140),
(755, 220),
(750, 340),
(730, 380),
(740, 420),
(715, 560), # right foot
(690, 550),
(680, 470),
(640, 530),
(590, 500), # left foot
(605, 240), # left elbow
(655, 130), # neck
(620, 120), # mouth, bottom
])
Now that we have the three polygons created, let's draw them on the image. We use the method Polygon.draw_on_image(image)
to do that and imgaug.imshow()
to show the drawn image.
import numpy as np
image_polys = np.copy(image)
image_polys = meerkat_left.draw_on_image(image_polys, alpha_face=0.2, size_points=7)
image_polys = meerkat_center.draw_on_image(image_polys, alpha_face=0.2, size_points=7)
image_polys = meerkat_right.draw_on_image(image_polys, alpha_face=0.2, size_points=7)
ia.imshow(image_polys)