We will use quickstart-groups to gather some LIDAR point cloud data to create our labels:
import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F
dataset = foz.load_zoo_dataset("quickstart-groups")
dataset.group_slice = "pcd"
session = fo.launch_app(dataset)
Dataset already downloaded Loading 'quickstart-groups' 100% |█████████████████| 600/600 [1.7s elapsed, 0s remaining, 365.8 samples/s] Dataset 'quickstart-groups' created
Could not connect session, trying again in 10 seconds
Ah! Those pesky error messages are no good. Since it is point cloud data, we have no thumbnail to display. We can change this by utilizing orthographic projection. This will create a birds eye view of our dataset and add the projection to a slice of our group as well. To learn more, look here!
import fiftyone.utils.utils3d as fou3d
size = (-1, 512)
fou3d.compute_orthographic_projection_images(
dataset,
size,
"/tmp/proj",
shading_mode="height",
out_group_slice="proj"
)
session.view = dataset.view()
100% |█████████████████| 200/200 [11.9s elapsed, 0s remaining, 16.9 samples/s] 100% |█████████████████| 200/200 [76.2ms elapsed, 0s remaining, 2.6K samples/s]
Much better!
To add a 3D detection, we create a fo.Detection()
label and add the label, location in x,y,z, rotation in radians (-pi, pi), and dimensions in width, length, height
dataset.group_slice = "pcd"
sample = dataset.first()
bounding_box = fo.Detection(
label="example",
location=[0,0,0],
rotation=[0, 0, 0],
dimensions=[3,3,3]
)
sample["example"]= bounding_box
sample.save()
dataset.save()
view = dataset.filter_labels(
"example", F("label").is_in(["example"])
)
session.view = view
Play around with changing values to get a feel for the coordinate system:
sample = dataset.first()
bounding_box = fo.Detection(
label="example",
location=[0,2,4],
rotation=[0, 0, 0],
dimensions=[1,2,3]
)
sample["example"]= bounding_box
sample.save()
dataset.save()
view = dataset.filter_labels(
"example", F("label").is_in(["example"])
)
session.view = view
Lets try experimenting with rotation now! Tweak the values to try to get the box to point in different directions!
import numpy as np
from fiftyone import ViewField as F
# x = left+right = orange | y = forward/back = green | z = up+down = blue
bounding_box = fo.Detection(
label="example",
location=[0,0,0],
rotation=[np.pi/4, np.pi/4, 0],
dimensions=[3,2,1]
)
sample["example"] = bounding_box
sample.save()
dataset.save()
view = dataset.filter_labels(
"example", F("label").is_in(["example"])
)
session.view = view
Another type of 3D label is polylines! They are created by defining the coordinates in a list for each point on the line. Here we make a triangle:
label = "lane"
# A list of lists of `[x, y, z]` points in scene coordinates describing
# the vertices of each shape in the polyline
points3d = [[[1, 1, 1], [0, 2, 2]], [[0, 2, 2], [-1, 1, 1]], [[-1, 1, 1], [1, 1, 1]]]
# A set of semantically related 3D polylines
polyline = fo.Polyline(label=label, points3d=points3d,)
sample["polylines"] = polyline
sample.save()
dataset.save()
view = dataset.filter_labels(
"polylines", F("label").is_in(["lane"])
)
session.view = view