This notebook demonstrates how to compute feature points for images and visualize them in FiftyOne.
Specifically, we'll compute ORB features using OpenCV, which is installed by default along with FiftyOne.
Make sure you have FiftyOne installed:
!pip install fiftyone
Then collect a directory of images that you'd like to process:
IMAGES_DIR = "/path/to/images"
Loading the images into a FiftyOne dataset is easy:
import fiftyone as fo
dataset = fo.Dataset.from_images_dir(IMAGES_DIR)
100% |██████████████████████████████| 13/13 [20.7ms elapsed, 0s remaining, 629.0 samples/s]
print(dataset)
Name: 2021.01.21.00.16.24 Media type: image Num samples: 13 Persistent: False Info: {} Tags: [] Sample fields: filepath: fiftyone.core.fields.StringField tags: fiftyone.core.fields.ListField(fiftyone.core.fields.StringField) metadata: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
Now let's add the feature points to the dataset.
We'll start by instantiating an OpenCV ORB detector:
import cv2
detector = cv2.ORB_create(nfeatures=500)
and then define a function that will extract the keypoints for an image:
def get_keypoints(image_path, detector):
img = cv2.imread(image_path)
height = img.shape[0]
width = img.shape[1]
keypoints = detector.detect(img, None)
return [(kp.pt[0] / width, kp.pt[1] / height) for kp in keypoints] # convert to [0, 1] x [0, 1]
Now we can write a simple loop to add the keypoints to our FiftyOne dataset in a new keypoints
field of each sample using FiftyOne's keypoints label type:
for sample in dataset:
points = get_keypoints(sample.filepath, detector)
sample["keypoints"] = fo.Keypoint(points=points)
sample.save()
Notice that the dataset now has a keypoints
field containing the ORB featuure points:
print(dataset)
Name: 2021.01.21.00.16.24 Media type: image Num samples: 13 Persistent: False Info: {} Tags: [] Sample fields: filepath: fiftyone.core.fields.StringField tags: fiftyone.core.fields.ListField(fiftyone.core.fields.StringField) metadata: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata) keypoints: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Keypoint)
Now we're ready to launch the FiftyOne App and visualize the keypoints:
session = fo.launch_app(dataset)
You can double-click on an image to expand it:
session.show()
session.freeze() # freeze App to enable notebook sharing