PyTorch datasets provide a great starting point for loading complex datasets, letting you define a class to load individual samples from disk and then creating data loaders to efficiently supply the data to your model. Problems arise when you want to start iterating over your dataset itself. PyTorch datasets are fairly rigid and require you to either rewrite them or the underlying data on disk if you want to make any changes to the data you are training or testing your model on. That is where FiftyOne comes in.
PyTorch datasets can synergize well with FiftyOne datasets for hard computer vision problems like classification, object detection, segmentation, and more. The flexibility of FiftyOne datasets lets you easily experiment with and finetune the datasets you use for training and testing to create better-performing models, faster. In this example, I am focusing on object detection since that is one of the most common vision tasks while also being fairly complex. However, these methods work for most machine learning tasks. Specifically, this notebook covers:
To start, we need to install FiftyOne:
If you're working in Google Colab, be sure to enable a GPU runtime before running any cell
!pip install fiftyone
!pip install torch torchvision
We'll also need pytorch, and torchvision, as well as clone the torchvision GitHub repository to use the training and evaluation utilities provided for the Torchvision Object Deteciton Tutorial that we are using to train a basic object detection model.
%%shell
# Download TorchVision repo to use some files from
# references/detection
git clone https://github.com/pytorch/vision.git
cd vision
git checkout v0.3.0
cp references/detection/utils.py ../
cp references/detection/transforms.py ../
cp references/detection/coco_eval.py ../
cp references/detection/engine.py ../
cp references/detection/coco_utils.py ../
Cloning into 'vision'... remote: Enumerating objects: 126, done. remote: Counting objects: 100% (126/126), done. remote: Compressing objects: 100% (115/115), done. remote: Total 18391 (delta 78), reused 42 (delta 7), pack-reused 18265 Receiving objects: 100% (18391/18391), 17.21 MiB | 24.65 MiB/s, done. Resolving deltas: 100% (13365/13365), done. Note: checking out 'v0.3.0'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name> HEAD is now at be376084 version check against PyTorch's CUDA version
Getting your data into FiftyOne is oftentimes actually easier than getting it into a PyTorch dataset. Additionally, once the data is in FiftyOne it is much more flexible allowing you to easily find and access even the most specific subsets of data that you can then use to train or evaluate your model.
If you have data that follows a certain format on disk (for example a directory tree for classification, the COCO detection format, or many more), then you can load it into FiftyOne in one line of code.
In this notebook, I am going to work with COCO-2017 and load it from the FiftyOne Dataset Zoo.
import torch
torch.manual_seed(1)
<torch._C.Generator at 0x7f86bfd5d810>
import fiftyone as fo
import fiftyone.zoo as foz
fo_dataset = foz.load_zoo_dataset("coco-2017", "validation")
Downloading split 'validation' to '/root/fiftyone/coco-2017/validation' Downloading images zip to '/root/fiftyone/coco-2017/tmp-download/val2017.zip' 100% |██████| 6.1Gb/6.1Gb [10.6s elapsed, 0s remaining, 502.3Mb/s] Extracting images to '/root/fiftyone/coco-2017/tmp-download/val2017' Downloading annotations zip to '/root/fiftyone/coco-2017/tmp-download/annotations_trainval2017.zip' 100% |██████| 1.9Gb/1.9Gb [4.1s elapsed, 0s remaining, 502.3Mb/s] Extracting annotations to '/root/fiftyone/coco-2017/tmp-download/annotations/instances_val2017.json' loading annotations into memory... Done (t=0.54s) creating index... index created! 100% |███████████████| 5000/5000 [1.7m elapsed, 0s remaining, 52.2 samples/s] Dataset info written to '/root/fiftyone/coco-2017/info.json' Loading 'coco-2017' split 'validation' 100% |███████████████| 5000/5000 [37.3s elapsed, 0s remaining, 130.1 samples/s] Dataset 'coco-2017-validation' created
We will be needing the height and width of images later in this notebook so we need to compute metadata on our dataset.
fo_dataset.compute_metadata()
100% |███████████████| 5000/5000 [55.2s elapsed, 0s remaining, 142.6 samples/s]
We can create a session and visualize this dataset in the FiftyOne App.
session = fo.launch_app(fo_dataset)
A PyTorch dataset is a class that defines how to load a static dataset and its labels from disk via a simple iterator interface. They differ from FiftyOne datasets which are flexible representations of your data geared towards visualization, querying, and understanding.
Every PyTorch model expects data and labels to pass into it in a certain format. Before being able to write up a PyTorch dataset class, you first need to understand the format that the model requires. Namely, we need to know exactly what format the data loader is expected to output when iterating through the dataset so that we can properly define the __getitem__
method in the PyTorch dataset.
In this example, I am following the Torchvision object detection tutorial and construct a PyTorch dataset to work with their RCNN-based models.
import torch
import fiftyone.utils.coco as fouc
from PIL import Image
class FiftyOneTorchDataset(torch.utils.data.Dataset):
"""A class to construct a PyTorch dataset from a FiftyOne dataset.
Args:
fiftyone_dataset: a FiftyOne dataset or view that will be used for training or testing
transforms (None): a list of PyTorch transforms to apply to images and targets when loading
gt_field ("ground_truth"): the name of the field in fiftyone_dataset that contains the
desired labels to load
classes (None): a list of class strings that are used to define the mapping between
class names and indices. If None, it will use all classes present in the given fiftyone_dataset.
"""
def __init__(
self,
fiftyone_dataset,
transforms=None,
gt_field="ground_truth",
classes=None,
):
self.samples = fiftyone_dataset
self.transforms = transforms
self.gt_field = gt_field
self.img_paths = self.samples.values("filepath")
self.classes = classes
if not self.classes:
# Get list of distinct labels that exist in the view
self.classes = self.samples.distinct(
"%s.detections.label" % gt_field
)
if self.classes[0] != "background":
self.classes = ["background"] + self.classes
self.labels_map_rev = {c: i for i, c in enumerate(self.classes)}
def __getitem__(self, idx):
img_path = self.img_paths[idx]
sample = self.samples[img_path]
metadata = sample.metadata
img = Image.open(img_path).convert("RGB")
boxes = []
labels = []
area = []
iscrowd = []
detections = sample[self.gt_field].detections
for det in detections:
category_id = self.labels_map_rev[det.label]
coco_obj = fouc.COCOObject.from_label(
det, metadata, category_id=category_id,
)
x, y, w, h = coco_obj.bbox
boxes.append([x, y, x + w, y + h])
labels.append(coco_obj.category_id)
area.append(coco_obj.area)
iscrowd.append(coco_obj.iscrowd)
target = {}
target["boxes"] = torch.as_tensor(boxes, dtype=torch.float32)
target["labels"] = torch.as_tensor(labels, dtype=torch.int64)
target["image_id"] = torch.as_tensor([idx])
target["area"] = torch.as_tensor(area, dtype=torch.float32)
target["iscrowd"] = torch.as_tensor(iscrowd, dtype=torch.int64)
if self.transforms is not None:
img, target = self.transforms(img, target)
return img, target
def __len__(self):
return len(self.img_paths)
def get_classes(self):
return self.classes
The following code loads Faster-RCNN with a ResNet50 backbone from Torchvision and modifies the classifier for the number of classes we are training on:
import torchvision
from torchvision.models.detection.faster_rcnn import FastRCNNPredictor
def get_model(num_classes):
# load a model pre-trained pre-trained on COCO
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
# get number of input features for the classifier
in_features = model.roi_heads.box_predictor.cls_score.in_features
# replace the pre-trained head with a new one
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
return model
For this example, we are going to write a simple training loop. This function is going to take a model and our PyTorch datasets as input and use the train_one_epoch()
and evaluate()
functions from the Torchvision object detection code
# Import functions from the torchvision references we cloned
from engine import train_one_epoch, evaluate
import utils
def do_training(model, torch_dataset, torch_dataset_test, num_epochs=4):
# define training and validation data loaders
data_loader = torch.utils.data.DataLoader(
torch_dataset, batch_size=2, shuffle=True, num_workers=2,
collate_fn=utils.collate_fn)
data_loader_test = torch.utils.data.DataLoader(
torch_dataset_test, batch_size=1, shuffle=False, num_workers=2,
collate_fn=utils.collate_fn)
# train on the GPU or on the CPU, if a GPU is not available
device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')
print("Using device %s" % device)
# move model to the right device
model.to(device)
# construct an optimizer
params = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.SGD(params, lr=0.005,
momentum=0.9, weight_decay=0.0005)
# and a learning rate scheduler
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer,
step_size=3,
gamma=0.1)
for epoch in range(num_epochs):
# train for one epoch, printing every 10 iterations
train_one_epoch(model, optimizer, data_loader, device, epoch, print_freq=10)
# update the learning rate
lr_scheduler.step()
# evaluate on the test dataset
evaluate(model, data_loader_test, device=device)
One of the primary ways of interacting with your FiftyOne dataset is through different views into your dataset. These are constructed by applying operations like filtering, sorting, slicing, etc, that result in a specific view into certain labels/samples of your dataset. These operations make it easier to experiment with different subsets of data and continue to finetune your dataset to train better models.
For example, cluttered images make it difficult for models to localize objects. We can use FiftyOne to create a view containing only samples with more than, say, 10 objects. You can perform the same operations on views as datasets, so we can create an instance of our PyTorch dataset from this view:
from fiftyone import ViewField as F
busy_view = fo_dataset.match(F("ground_truth.detections").length() > 10)
busy_torch_dataset = FiftyOneTorchDataset(busy_view)
session.view = busy_view
Another example is if we want to train a model that is used primarily for road vehicle detection. We can easily create training and testing views (and corresponding PyTorch datasets) that only contain the classes car, truck, and bus:
from fiftyone import ViewField as F
vehicles_list = ["car", "truck", "bus"]
vehicles_view = fo_dataset.filter_labels("ground_truth",
F("label").is_in(vehicles_list))
print(len(vehicles_view))
707
session.view = vehicles_view
# From the torchvision references we cloned
import transforms as T
train_transforms = T.Compose([T.ToTensor(), T.RandomHorizontalFlip(0.5)])
test_transforms = T.Compose([T.ToTensor()])
# split the dataset in train and test set
train_view = vehicles_view.take(500, seed=51)
test_view = vehicles_view.exclude([s.id for s in train_view])
# use our dataset and defined transformations
torch_dataset = FiftyOneTorchDataset(train_view, train_transforms,
classes=vehicles_list)
torch_dataset_test = FiftyOneTorchDataset(test_view, test_transforms,
classes=vehicles_list)
In this section, we use the functions and datasets we defined above to initialize, train, and evaluate a model continuing with the vehicle example.
model = get_model(len(vehicles_list)+1)
Downloading: "https://download.pytorch.org/models/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth" to /root/.cache/torch/hub/checkpoints/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth
HBox(children=(FloatProgress(value=0.0, max=167502836.0), HTML(value='')))
do_training(model, torch_dataset, torch_dataset_test, num_epochs=4)
Using device cuda
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [0] [ 0/250] eta: 0:05:48 lr: 0.000025 loss: 2.0424 (2.0424) loss_classifier: 1.3071 (1.3071) loss_box_reg: 0.6820 (0.6820) loss_objectness: 0.0361 (0.0361) loss_rpn_box_reg: 0.0172 (0.0172) time: 1.3925 data: 0.2820 max mem: 3094 Epoch: [0] [ 10/250] eta: 0:02:39 lr: 0.000226 loss: 1.8390 (1.7483) loss_classifier: 1.2389 (1.1791) loss_box_reg: 0.5072 (0.4188) loss_objectness: 0.0438 (0.0689) loss_rpn_box_reg: 0.0219 (0.0814) time: 0.6632 data: 0.0323 max mem: 3506 Epoch: [0] [ 20/250] eta: 0:02:21 lr: 0.000426 loss: 1.2446 (1.3204) loss_classifier: 0.7553 (0.8323) loss_box_reg: 0.2559 (0.3675) loss_objectness: 0.0349 (0.0595) loss_rpn_box_reg: 0.0190 (0.0612) time: 0.5744 data: 0.0070 max mem: 3506 Epoch: [0] [ 30/250] eta: 0:02:12 lr: 0.000627 loss: 0.9709 (1.1985) loss_classifier: 0.5002 (0.7143) loss_box_reg: 0.3701 (0.3838) loss_objectness: 0.0290 (0.0493) loss_rpn_box_reg: 0.0190 (0.0511) time: 0.5688 data: 0.0068 max mem: 3506 Epoch: [0] [ 40/250] eta: 0:02:03 lr: 0.000827 loss: 0.6146 (1.0203) loss_classifier: 0.2428 (0.5860) loss_box_reg: 0.2571 (0.3442) loss_objectness: 0.0233 (0.0463) loss_rpn_box_reg: 0.0165 (0.0437) time: 0.5660 data: 0.0068 max mem: 3506 Epoch: [0] [ 50/250] eta: 0:01:57 lr: 0.001028 loss: 0.6146 (0.9627) loss_classifier: 0.2233 (0.5214) loss_box_reg: 0.2774 (0.3545) loss_objectness: 0.0233 (0.0430) loss_rpn_box_reg: 0.0140 (0.0438) time: 0.5606 data: 0.0067 max mem: 3506 Epoch: [0] [ 60/250] eta: 0:01:53 lr: 0.001229 loss: 0.6338 (0.9093) loss_classifier: 0.2331 (0.4718) loss_box_reg: 0.3477 (0.3581) loss_objectness: 0.0210 (0.0399) loss_rpn_box_reg: 0.0155 (0.0395) time: 0.6124 data: 0.0067 max mem: 3862 Epoch: [0] [ 70/250] eta: 0:01:48 lr: 0.001429 loss: 0.5526 (0.8670) loss_classifier: 0.1829 (0.4322) loss_box_reg: 0.3477 (0.3564) loss_objectness: 0.0244 (0.0405) loss_rpn_box_reg: 0.0119 (0.0380) time: 0.6477 data: 0.0071 max mem: 4211 Epoch: [0] [ 80/250] eta: 0:01:43 lr: 0.001630 loss: 0.4572 (0.8247) loss_classifier: 0.1649 (0.3991) loss_box_reg: 0.2320 (0.3494) loss_objectness: 0.0285 (0.0382) loss_rpn_box_reg: 0.0138 (0.0380) time: 0.6457 data: 0.0072 max mem: 4211 Epoch: [0] [ 90/250] eta: 0:01:38 lr: 0.001830 loss: 0.4399 (0.7853) loss_classifier: 0.1504 (0.3728) loss_box_reg: 0.2342 (0.3415) loss_objectness: 0.0101 (0.0360) loss_rpn_box_reg: 0.0122 (0.0350) time: 0.6507 data: 0.0069 max mem: 4211 Epoch: [0] [100/250] eta: 0:01:32 lr: 0.002031 loss: 0.3349 (0.7592) loss_classifier: 0.1167 (0.3501) loss_box_reg: 0.1766 (0.3313) loss_objectness: 0.0155 (0.0385) loss_rpn_box_reg: 0.0069 (0.0392) time: 0.6604 data: 0.0068 max mem: 4211 Epoch: [0] [110/250] eta: 0:01:27 lr: 0.002232 loss: 0.3243 (0.7329) loss_classifier: 0.1398 (0.3329) loss_box_reg: 0.1679 (0.3222) loss_objectness: 0.0209 (0.0385) loss_rpn_box_reg: 0.0129 (0.0392) time: 0.6575 data: 0.0070 max mem: 4211 Epoch: [0] [120/250] eta: 0:01:21 lr: 0.002432 loss: 0.5199 (0.7187) loss_classifier: 0.1594 (0.3210) loss_box_reg: 0.2173 (0.3169) loss_objectness: 0.0252 (0.0414) loss_rpn_box_reg: 0.0180 (0.0392) time: 0.6643 data: 0.0071 max mem: 4211 Epoch: [0] [130/250] eta: 0:01:15 lr: 0.002633 loss: 0.6149 (0.7129) loss_classifier: 0.2029 (0.3155) loss_box_reg: 0.3126 (0.3191) loss_objectness: 0.0282 (0.0402) loss_rpn_box_reg: 0.0186 (0.0381) time: 0.6511 data: 0.0069 max mem: 4211 Epoch: [0] [140/250] eta: 0:01:08 lr: 0.002833 loss: 0.5098 (0.6932) loss_classifier: 0.1946 (0.3066) loss_box_reg: 0.2652 (0.3109) loss_objectness: 0.0195 (0.0388) loss_rpn_box_reg: 0.0166 (0.0368) time: 0.6045 data: 0.0067 max mem: 4211 Epoch: [0] [150/250] eta: 0:01:02 lr: 0.003034 loss: 0.4439 (0.6817) loss_classifier: 0.1885 (0.3004) loss_box_reg: 0.2257 (0.3060) loss_objectness: 0.0195 (0.0391) loss_rpn_box_reg: 0.0125 (0.0363) time: 0.5870 data: 0.0066 max mem: 4211 Epoch: [0] [160/250] eta: 0:00:56 lr: 0.003235 loss: 0.5237 (0.6770) loss_classifier: 0.1961 (0.2968) loss_box_reg: 0.2549 (0.3057) loss_objectness: 0.0277 (0.0386) loss_rpn_box_reg: 0.0162 (0.0358) time: 0.6235 data: 0.0072 max mem: 4211 Epoch: [0] [170/250] eta: 0:00:49 lr: 0.003435 loss: 0.5588 (0.6670) loss_classifier: 0.2055 (0.2910) loss_box_reg: 0.2656 (0.3019) loss_objectness: 0.0268 (0.0385) loss_rpn_box_reg: 0.0195 (0.0356) time: 0.6290 data: 0.0072 max mem: 4211 Epoch: [0] [180/250] eta: 0:00:43 lr: 0.003636 loss: 0.4949 (0.6569) loss_classifier: 0.1841 (0.2846) loss_box_reg: 0.2374 (0.2987) loss_objectness: 0.0319 (0.0385) loss_rpn_box_reg: 0.0167 (0.0350) time: 0.6007 data: 0.0066 max mem: 4211 Epoch: [0] [190/250] eta: 0:00:37 lr: 0.003837 loss: 0.4744 (0.6498) loss_classifier: 0.1813 (0.2792) loss_box_reg: 0.2171 (0.2947) loss_objectness: 0.0337 (0.0398) loss_rpn_box_reg: 0.0209 (0.0361) time: 0.6212 data: 0.0065 max mem: 4211 Epoch: [0] [200/250] eta: 0:00:30 lr: 0.004037 loss: 0.4094 (0.6356) loss_classifier: 0.1452 (0.2727) loss_box_reg: 0.1871 (0.2890) loss_objectness: 0.0243 (0.0392) loss_rpn_box_reg: 0.0139 (0.0348) time: 0.6045 data: 0.0065 max mem: 4211 Epoch: [0] [210/250] eta: 0:00:24 lr: 0.004238 loss: 0.3599 (0.6242) loss_classifier: 0.1452 (0.2670) loss_box_reg: 0.1616 (0.2835) loss_objectness: 0.0208 (0.0390) loss_rpn_box_reg: 0.0073 (0.0346) time: 0.6035 data: 0.0067 max mem: 4211 Epoch: [0] [220/250] eta: 0:00:18 lr: 0.004438 loss: 0.3919 (0.6147) loss_classifier: 0.1589 (0.2621) loss_box_reg: 0.1741 (0.2798) loss_objectness: 0.0278 (0.0383) loss_rpn_box_reg: 0.0073 (0.0344) time: 0.6041 data: 0.0068 max mem: 4211 Epoch: [0] [230/250] eta: 0:00:12 lr: 0.004639 loss: 0.4691 (0.6127) loss_classifier: 0.1637 (0.2601) loss_box_reg: 0.2205 (0.2802) loss_objectness: 0.0179 (0.0378) loss_rpn_box_reg: 0.0173 (0.0347) time: 0.5973 data: 0.0066 max mem: 4211 Epoch: [0] [240/250] eta: 0:00:06 lr: 0.004840 loss: 0.5367 (0.6078) loss_classifier: 0.1789 (0.2572) loss_box_reg: 0.2651 (0.2790) loss_objectness: 0.0164 (0.0374) loss_rpn_box_reg: 0.0207 (0.0342) time: 0.6496 data: 0.0069 max mem: 4211 Epoch: [0] [249/250] eta: 0:00:00 lr: 0.005000 loss: 0.3983 (0.6044) loss_classifier: 0.1707 (0.2538) loss_box_reg: 0.2203 (0.2755) loss_objectness: 0.0173 (0.0402) loss_rpn_box_reg: 0.0246 (0.0350) time: 0.6465 data: 0.0069 max mem: 4211 Epoch: [0] Total time: 0:02:35 (0.6206 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:08 model_time: 0.1717 (0.1717) evaluator_time: 0.0182 (0.0182) time: 0.3311 data: 0.1394 max mem: 4211 Test: [100/207] eta: 0:00:13 model_time: 0.1220 (0.1176) evaluator_time: 0.0048 (0.0063) time: 0.1298 data: 0.0034 max mem: 4211 Test: [200/207] eta: 0:00:00 model_time: 0.1123 (0.1173) evaluator_time: 0.0035 (0.0061) time: 0.1228 data: 0.0036 max mem: 4211 Test: [206/207] eta: 0:00:00 model_time: 0.1130 (0.1173) evaluator_time: 0.0061 (0.0062) time: 0.1256 data: 0.0036 max mem: 4211 Test: Total time: 0:00:26 (0.1292 s / it) Averaged stats: model_time: 0.1130 (0.1173) evaluator_time: 0.0061 (0.0062) Accumulating evaluation results... DONE (t=0.20s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.062 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.147 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.046 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.059 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.068 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.106 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.079 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.221 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.286 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.271 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.313 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.333
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [1] [ 0/250] eta: 0:03:36 lr: 0.005000 loss: 0.7783 (0.7783) loss_classifier: 0.2188 (0.2188) loss_box_reg: 0.3122 (0.3122) loss_objectness: 0.2095 (0.2095) loss_rpn_box_reg: 0.0379 (0.0379) time: 0.8654 data: 0.3291 max mem: 4211 Epoch: [1] [ 10/250] eta: 0:02:30 lr: 0.005000 loss: 0.4352 (0.4338) loss_classifier: 0.1951 (0.1744) loss_box_reg: 0.1519 (0.1611) loss_objectness: 0.0443 (0.0770) loss_rpn_box_reg: 0.0099 (0.0213) time: 0.6275 data: 0.0362 max mem: 4211 Epoch: [1] [ 20/250] eta: 0:02:21 lr: 0.005000 loss: 0.4352 (0.4672) loss_classifier: 0.1951 (0.1943) loss_box_reg: 0.1854 (0.1962) loss_objectness: 0.0252 (0.0554) loss_rpn_box_reg: 0.0140 (0.0214) time: 0.6041 data: 0.0066 max mem: 4211 Epoch: [1] [ 30/250] eta: 0:02:17 lr: 0.005000 loss: 0.4343 (0.4460) loss_classifier: 0.1785 (0.1889) loss_box_reg: 0.1912 (0.1886) loss_objectness: 0.0244 (0.0472) loss_rpn_box_reg: 0.0160 (0.0213) time: 0.6212 data: 0.0065 max mem: 4211 Epoch: [1] [ 40/250] eta: 0:02:11 lr: 0.005000 loss: 0.3589 (0.4416) loss_classifier: 0.1737 (0.1831) loss_box_reg: 0.1693 (0.1881) loss_objectness: 0.0266 (0.0445) loss_rpn_box_reg: 0.0145 (0.0260) time: 0.6329 data: 0.0066 max mem: 4400 Epoch: [1] [ 50/250] eta: 0:02:04 lr: 0.005000 loss: 0.4596 (0.4623) loss_classifier: 0.1906 (0.1905) loss_box_reg: 0.1693 (0.1995) loss_objectness: 0.0280 (0.0416) loss_rpn_box_reg: 0.0186 (0.0306) time: 0.6211 data: 0.0067 max mem: 4400 Epoch: [1] [ 60/250] eta: 0:01:57 lr: 0.005000 loss: 0.5062 (0.4784) loss_classifier: 0.1928 (0.1908) loss_box_reg: 0.2222 (0.2038) loss_objectness: 0.0280 (0.0451) loss_rpn_box_reg: 0.0186 (0.0387) time: 0.6120 data: 0.0069 max mem: 4400 Epoch: [1] [ 70/250] eta: 0:01:51 lr: 0.005000 loss: 0.3885 (0.4576) loss_classifier: 0.1602 (0.1834) loss_box_reg: 0.1471 (0.1954) loss_objectness: 0.0327 (0.0436) loss_rpn_box_reg: 0.0175 (0.0352) time: 0.6113 data: 0.0068 max mem: 4400 Epoch: [1] [ 80/250] eta: 0:01:45 lr: 0.005000 loss: 0.3554 (0.4643) loss_classifier: 0.1489 (0.1844) loss_box_reg: 0.1471 (0.2005) loss_objectness: 0.0369 (0.0437) loss_rpn_box_reg: 0.0170 (0.0358) time: 0.6318 data: 0.0070 max mem: 4400 Epoch: [1] [ 90/250] eta: 0:01:39 lr: 0.005000 loss: 0.4147 (0.4566) loss_classifier: 0.1612 (0.1827) loss_box_reg: 0.1991 (0.1985) loss_objectness: 0.0163 (0.0406) loss_rpn_box_reg: 0.0125 (0.0348) time: 0.6395 data: 0.0070 max mem: 4400 Epoch: [1] [100/250] eta: 0:01:34 lr: 0.005000 loss: 0.3852 (0.4562) loss_classifier: 0.1597 (0.1812) loss_box_reg: 0.1708 (0.1985) loss_objectness: 0.0162 (0.0407) loss_rpn_box_reg: 0.0125 (0.0358) time: 0.6478 data: 0.0065 max mem: 4400 Epoch: [1] [110/250] eta: 0:01:27 lr: 0.005000 loss: 0.4013 (0.4543) loss_classifier: 0.1404 (0.1778) loss_box_reg: 0.1708 (0.1998) loss_objectness: 0.0290 (0.0407) loss_rpn_box_reg: 0.0287 (0.0360) time: 0.6278 data: 0.0066 max mem: 4400 Epoch: [1] [120/250] eta: 0:01:21 lr: 0.005000 loss: 0.4627 (0.4577) loss_classifier: 0.1577 (0.1810) loss_box_reg: 0.2247 (0.2018) loss_objectness: 0.0277 (0.0394) loss_rpn_box_reg: 0.0148 (0.0354) time: 0.6221 data: 0.0067 max mem: 4400 Epoch: [1] [130/250] eta: 0:01:15 lr: 0.005000 loss: 0.4507 (0.4546) loss_classifier: 0.1595 (0.1793) loss_box_reg: 0.1688 (0.2000) loss_objectness: 0.0304 (0.0399) loss_rpn_box_reg: 0.0142 (0.0355) time: 0.6517 data: 0.0065 max mem: 4400 Epoch: [1] [140/250] eta: 0:01:08 lr: 0.005000 loss: 0.3684 (0.4500) loss_classifier: 0.1507 (0.1780) loss_box_reg: 0.1688 (0.1984) loss_objectness: 0.0249 (0.0386) loss_rpn_box_reg: 0.0216 (0.0349) time: 0.6242 data: 0.0067 max mem: 4400 Epoch: [1] [150/250] eta: 0:01:02 lr: 0.005000 loss: 0.3214 (0.4417) loss_classifier: 0.1337 (0.1745) loss_box_reg: 0.1418 (0.1952) loss_objectness: 0.0187 (0.0375) loss_rpn_box_reg: 0.0112 (0.0346) time: 0.5884 data: 0.0068 max mem: 4400 Epoch: [1] [160/250] eta: 0:00:55 lr: 0.005000 loss: 0.3145 (0.4387) loss_classifier: 0.1253 (0.1726) loss_box_reg: 0.1309 (0.1941) loss_objectness: 0.0194 (0.0372) loss_rpn_box_reg: 0.0238 (0.0348) time: 0.5747 data: 0.0066 max mem: 4400 Epoch: [1] [170/250] eta: 0:00:49 lr: 0.005000 loss: 0.3692 (0.4370) loss_classifier: 0.1361 (0.1720) loss_box_reg: 0.1405 (0.1940) loss_objectness: 0.0231 (0.0370) loss_rpn_box_reg: 0.0238 (0.0339) time: 0.5676 data: 0.0068 max mem: 4400 Epoch: [1] [180/250] eta: 0:00:43 lr: 0.005000 loss: 0.4429 (0.4432) loss_classifier: 0.1848 (0.1734) loss_box_reg: 0.2007 (0.1971) loss_objectness: 0.0277 (0.0368) loss_rpn_box_reg: 0.0147 (0.0359) time: 0.6171 data: 0.0068 max mem: 4400 Epoch: [1] [190/250] eta: 0:00:37 lr: 0.005000 loss: 0.4324 (0.4437) loss_classifier: 0.1839 (0.1742) loss_box_reg: 0.2128 (0.1977) loss_objectness: 0.0277 (0.0362) loss_rpn_box_reg: 0.0144 (0.0357) time: 0.6573 data: 0.0067 max mem: 4400 Epoch: [1] [200/250] eta: 0:00:30 lr: 0.005000 loss: 0.4066 (0.4450) loss_classifier: 0.1546 (0.1747) loss_box_reg: 0.2128 (0.1983) loss_objectness: 0.0236 (0.0357) loss_rpn_box_reg: 0.0137 (0.0362) time: 0.6202 data: 0.0066 max mem: 4400 Epoch: [1] [210/250] eta: 0:00:24 lr: 0.005000 loss: 0.3613 (0.4456) loss_classifier: 0.1479 (0.1745) loss_box_reg: 0.1647 (0.1983) loss_objectness: 0.0265 (0.0360) loss_rpn_box_reg: 0.0116 (0.0368) time: 0.6014 data: 0.0066 max mem: 4400 Epoch: [1] [220/250] eta: 0:00:18 lr: 0.005000 loss: 0.3613 (0.4454) loss_classifier: 0.1539 (0.1747) loss_box_reg: 0.1647 (0.1981) loss_objectness: 0.0215 (0.0354) loss_rpn_box_reg: 0.0175 (0.0372) time: 0.5941 data: 0.0067 max mem: 4400 Epoch: [1] [230/250] eta: 0:00:12 lr: 0.005000 loss: 0.2813 (0.4400) loss_classifier: 0.1284 (0.1726) loss_box_reg: 0.1242 (0.1957) loss_objectness: 0.0129 (0.0348) loss_rpn_box_reg: 0.0175 (0.0369) time: 0.6086 data: 0.0066 max mem: 4400 Epoch: [1] [240/250] eta: 0:00:06 lr: 0.005000 loss: 0.2813 (0.4358) loss_classifier: 0.1191 (0.1710) loss_box_reg: 0.1294 (0.1945) loss_objectness: 0.0161 (0.0342) loss_rpn_box_reg: 0.0126 (0.0361) time: 0.6469 data: 0.0065 max mem: 4400 Epoch: [1] [249/250] eta: 0:00:00 lr: 0.005000 loss: 0.3646 (0.4367) loss_classifier: 0.1397 (0.1709) loss_box_reg: 0.1754 (0.1956) loss_objectness: 0.0184 (0.0348) loss_rpn_box_reg: 0.0110 (0.0355) time: 0.6435 data: 0.0065 max mem: 4400 Epoch: [1] Total time: 0:02:35 (0.6207 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:08 model_time: 0.1723 (0.1723) evaluator_time: 0.0091 (0.0091) time: 0.3287 data: 0.1455 max mem: 4400 Test: [100/207] eta: 0:00:13 model_time: 0.1208 (0.1177) evaluator_time: 0.0029 (0.0056) time: 0.1285 data: 0.0034 max mem: 4400 Test: [200/207] eta: 0:00:00 model_time: 0.1131 (0.1174) evaluator_time: 0.0025 (0.0050) time: 0.1211 data: 0.0033 max mem: 4400 Test: [206/207] eta: 0:00:00 model_time: 0.1138 (0.1174) evaluator_time: 0.0038 (0.0052) time: 0.1240 data: 0.0033 max mem: 4400 Test: Total time: 0:00:26 (0.1283 s / it) Averaged stats: model_time: 0.1138 (0.1174) evaluator_time: 0.0038 (0.0052) Accumulating evaluation results... DONE (t=0.13s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.281 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.497 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.302 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.128 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.259 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.345 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.263 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.474 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.508 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.363 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.509 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.595
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [2] [ 0/250] eta: 0:03:35 lr: 0.005000 loss: 0.2030 (0.2030) loss_classifier: 0.0742 (0.0742) loss_box_reg: 0.0981 (0.0981) loss_objectness: 0.0236 (0.0236) loss_rpn_box_reg: 0.0070 (0.0070) time: 0.8602 data: 0.2654 max mem: 4400 Epoch: [2] [ 10/250] eta: 0:02:32 lr: 0.005000 loss: 0.2678 (0.3346) loss_classifier: 0.1189 (0.1501) loss_box_reg: 0.1305 (0.1487) loss_objectness: 0.0176 (0.0219) loss_rpn_box_reg: 0.0094 (0.0138) time: 0.6360 data: 0.0300 max mem: 4400 Epoch: [2] [ 20/250] eta: 0:02:23 lr: 0.005000 loss: 0.2678 (0.3265) loss_classifier: 0.1222 (0.1480) loss_box_reg: 0.1305 (0.1383) loss_objectness: 0.0154 (0.0217) loss_rpn_box_reg: 0.0147 (0.0186) time: 0.6119 data: 0.0065 max mem: 4400 Epoch: [2] [ 30/250] eta: 0:02:18 lr: 0.005000 loss: 0.3089 (0.3398) loss_classifier: 0.1295 (0.1498) loss_box_reg: 0.1501 (0.1533) loss_objectness: 0.0154 (0.0197) loss_rpn_box_reg: 0.0124 (0.0170) time: 0.6232 data: 0.0066 max mem: 4400 Epoch: [2] [ 40/250] eta: 0:02:10 lr: 0.005000 loss: 0.4012 (0.3535) loss_classifier: 0.1544 (0.1528) loss_box_reg: 0.2280 (0.1654) loss_objectness: 0.0083 (0.0182) loss_rpn_box_reg: 0.0113 (0.0171) time: 0.6179 data: 0.0065 max mem: 4400 Epoch: [2] [ 50/250] eta: 0:02:02 lr: 0.005000 loss: 0.4555 (0.3739) loss_classifier: 0.1544 (0.1556) loss_box_reg: 0.2198 (0.1762) loss_objectness: 0.0121 (0.0189) loss_rpn_box_reg: 0.0126 (0.0231) time: 0.5944 data: 0.0064 max mem: 4400 Epoch: [2] [ 60/250] eta: 0:01:58 lr: 0.005000 loss: 0.4607 (0.3821) loss_classifier: 0.1651 (0.1577) loss_box_reg: 0.2198 (0.1833) loss_objectness: 0.0161 (0.0185) loss_rpn_box_reg: 0.0105 (0.0227) time: 0.6251 data: 0.0064 max mem: 4401 Epoch: [2] [ 70/250] eta: 0:01:52 lr: 0.005000 loss: 0.4061 (0.3902) loss_classifier: 0.1718 (0.1623) loss_box_reg: 0.2065 (0.1846) loss_objectness: 0.0172 (0.0186) loss_rpn_box_reg: 0.0122 (0.0247) time: 0.6442 data: 0.0065 max mem: 4401 Epoch: [2] [ 80/250] eta: 0:01:46 lr: 0.005000 loss: 0.3456 (0.3744) loss_classifier: 0.1480 (0.1568) loss_box_reg: 0.1291 (0.1749) loss_objectness: 0.0167 (0.0186) loss_rpn_box_reg: 0.0134 (0.0241) time: 0.6319 data: 0.0065 max mem: 4401 Epoch: [2] [ 90/250] eta: 0:01:38 lr: 0.005000 loss: 0.3226 (0.3736) loss_classifier: 0.1227 (0.1560) loss_box_reg: 0.1452 (0.1744) loss_objectness: 0.0157 (0.0186) loss_rpn_box_reg: 0.0171 (0.0246) time: 0.5941 data: 0.0066 max mem: 4401 Epoch: [2] [100/250] eta: 0:01:32 lr: 0.005000 loss: 0.3428 (0.3759) loss_classifier: 0.1466 (0.1543) loss_box_reg: 0.1719 (0.1779) loss_objectness: 0.0183 (0.0186) loss_rpn_box_reg: 0.0185 (0.0251) time: 0.5790 data: 0.0066 max mem: 4401 Epoch: [2] [110/250] eta: 0:01:27 lr: 0.005000 loss: 0.3428 (0.3753) loss_classifier: 0.1355 (0.1535) loss_box_reg: 0.1755 (0.1758) loss_objectness: 0.0140 (0.0186) loss_rpn_box_reg: 0.0141 (0.0274) time: 0.6470 data: 0.0065 max mem: 4401 Epoch: [2] [120/250] eta: 0:01:21 lr: 0.005000 loss: 0.4148 (0.3833) loss_classifier: 0.1514 (0.1569) loss_box_reg: 0.1740 (0.1789) loss_objectness: 0.0164 (0.0189) loss_rpn_box_reg: 0.0152 (0.0285) time: 0.6660 data: 0.0067 max mem: 4401 Epoch: [2] [130/250] eta: 0:01:14 lr: 0.005000 loss: 0.4149 (0.3835) loss_classifier: 0.1687 (0.1578) loss_box_reg: 0.1765 (0.1772) loss_objectness: 0.0166 (0.0187) loss_rpn_box_reg: 0.0229 (0.0298) time: 0.6217 data: 0.0066 max mem: 4401 Epoch: [2] [140/250] eta: 0:01:08 lr: 0.005000 loss: 0.4996 (0.3905) loss_classifier: 0.1934 (0.1604) loss_box_reg: 0.1858 (0.1812) loss_objectness: 0.0138 (0.0188) loss_rpn_box_reg: 0.0157 (0.0301) time: 0.5995 data: 0.0066 max mem: 4401 Epoch: [2] [150/250] eta: 0:01:02 lr: 0.005000 loss: 0.3816 (0.3883) loss_classifier: 0.1685 (0.1580) loss_box_reg: 0.1860 (0.1809) loss_objectness: 0.0111 (0.0189) loss_rpn_box_reg: 0.0160 (0.0305) time: 0.6182 data: 0.0065 max mem: 4401 Epoch: [2] [160/250] eta: 0:00:55 lr: 0.005000 loss: 0.3247 (0.3888) loss_classifier: 0.1164 (0.1580) loss_box_reg: 0.1533 (0.1820) loss_objectness: 0.0085 (0.0188) loss_rpn_box_reg: 0.0232 (0.0301) time: 0.6283 data: 0.0065 max mem: 4401 Epoch: [2] [170/250] eta: 0:00:49 lr: 0.005000 loss: 0.3346 (0.3865) loss_classifier: 0.1255 (0.1569) loss_box_reg: 0.1665 (0.1806) loss_objectness: 0.0071 (0.0184) loss_rpn_box_reg: 0.0196 (0.0307) time: 0.6233 data: 0.0066 max mem: 4401 Epoch: [2] [180/250] eta: 0:00:43 lr: 0.005000 loss: 0.3093 (0.3840) loss_classifier: 0.1184 (0.1546) loss_box_reg: 0.1458 (0.1799) loss_objectness: 0.0098 (0.0183) loss_rpn_box_reg: 0.0119 (0.0312) time: 0.6190 data: 0.0066 max mem: 4401 Epoch: [2] [190/250] eta: 0:00:37 lr: 0.005000 loss: 0.3315 (0.3878) loss_classifier: 0.1270 (0.1566) loss_box_reg: 0.1589 (0.1811) loss_objectness: 0.0141 (0.0191) loss_rpn_box_reg: 0.0146 (0.0310) time: 0.6022 data: 0.0070 max mem: 4401 Epoch: [2] [200/250] eta: 0:00:30 lr: 0.005000 loss: 0.3494 (0.3882) loss_classifier: 0.1283 (0.1560) loss_box_reg: 0.1492 (0.1812) loss_objectness: 0.0205 (0.0194) loss_rpn_box_reg: 0.0145 (0.0316) time: 0.5932 data: 0.0074 max mem: 4401 Epoch: [2] [210/250] eta: 0:00:24 lr: 0.005000 loss: 0.4358 (0.3902) loss_classifier: 0.1283 (0.1561) loss_box_reg: 0.1408 (0.1811) loss_objectness: 0.0244 (0.0198) loss_rpn_box_reg: 0.0097 (0.0332) time: 0.6046 data: 0.0070 max mem: 4401 Epoch: [2] [220/250] eta: 0:00:18 lr: 0.005000 loss: 0.4121 (0.3934) loss_classifier: 0.1361 (0.1569) loss_box_reg: 0.2099 (0.1836) loss_objectness: 0.0162 (0.0200) loss_rpn_box_reg: 0.0153 (0.0330) time: 0.6274 data: 0.0066 max mem: 4401 Epoch: [2] [230/250] eta: 0:00:12 lr: 0.005000 loss: 0.3618 (0.3916) loss_classifier: 0.1251 (0.1554) loss_box_reg: 0.2102 (0.1836) loss_objectness: 0.0104 (0.0197) loss_rpn_box_reg: 0.0167 (0.0329) time: 0.6337 data: 0.0065 max mem: 4401 Epoch: [2] [240/250] eta: 0:00:06 lr: 0.005000 loss: 0.2496 (0.3902) loss_classifier: 0.0994 (0.1550) loss_box_reg: 0.1357 (0.1835) loss_objectness: 0.0076 (0.0193) loss_rpn_box_reg: 0.0126 (0.0324) time: 0.6587 data: 0.0068 max mem: 4401 Epoch: [2] [249/250] eta: 0:00:00 lr: 0.005000 loss: 0.2581 (0.3889) loss_classifier: 0.1132 (0.1550) loss_box_reg: 0.1357 (0.1830) loss_objectness: 0.0084 (0.0192) loss_rpn_box_reg: 0.0121 (0.0317) time: 0.6822 data: 0.0068 max mem: 4401 Epoch: [2] Total time: 0:02:36 (0.6244 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:05 model_time: 0.1593 (0.1593) evaluator_time: 0.0059 (0.0059) time: 0.3145 data: 0.1474 max mem: 4401 Test: [100/207] eta: 0:00:13 model_time: 0.1216 (0.1176) evaluator_time: 0.0018 (0.0031) time: 0.1269 data: 0.0034 max mem: 4401 Test: [200/207] eta: 0:00:00 model_time: 0.1130 (0.1175) evaluator_time: 0.0017 (0.0030) time: 0.1205 data: 0.0033 max mem: 4401 Test: [206/207] eta: 0:00:00 model_time: 0.1130 (0.1174) evaluator_time: 0.0024 (0.0032) time: 0.1228 data: 0.0034 max mem: 4401 Test: Total time: 0:00:26 (0.1262 s / it) Averaged stats: model_time: 0.1130 (0.1174) evaluator_time: 0.0024 (0.0032) Accumulating evaluation results... DONE (t=0.09s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.341 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.547 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.363 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.186 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.319 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.433 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.304 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.533 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.553 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.438 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.537 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.680
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [3] [ 0/250] eta: 0:03:52 lr: 0.000500 loss: 0.3354 (0.3354) loss_classifier: 0.1450 (0.1450) loss_box_reg: 0.1744 (0.1744) loss_objectness: 0.0091 (0.0091) loss_rpn_box_reg: 0.0069 (0.0069) time: 0.9287 data: 0.2996 max mem: 4401 Epoch: [3] [ 10/250] eta: 0:02:30 lr: 0.000500 loss: 0.3590 (0.3596) loss_classifier: 0.1284 (0.1324) loss_box_reg: 0.1657 (0.1712) loss_objectness: 0.0196 (0.0204) loss_rpn_box_reg: 0.0134 (0.0354) time: 0.6271 data: 0.0325 max mem: 4401 Epoch: [3] [ 20/250] eta: 0:02:22 lr: 0.000500 loss: 0.3198 (0.3061) loss_classifier: 0.1136 (0.1162) loss_box_reg: 0.1374 (0.1451) loss_objectness: 0.0121 (0.0153) loss_rpn_box_reg: 0.0134 (0.0295) time: 0.6062 data: 0.0064 max mem: 4401 Epoch: [3] [ 30/250] eta: 0:02:20 lr: 0.000500 loss: 0.3198 (0.3471) loss_classifier: 0.1263 (0.1277) loss_box_reg: 0.1078 (0.1740) loss_objectness: 0.0081 (0.0147) loss_rpn_box_reg: 0.0154 (0.0307) time: 0.6434 data: 0.0067 max mem: 4401 Epoch: [3] [ 40/250] eta: 0:02:11 lr: 0.000500 loss: 0.4186 (0.3519) loss_classifier: 0.1500 (0.1302) loss_box_reg: 0.1878 (0.1738) loss_objectness: 0.0082 (0.0146) loss_rpn_box_reg: 0.0171 (0.0333) time: 0.6285 data: 0.0066 max mem: 4401 Epoch: [3] [ 50/250] eta: 0:02:05 lr: 0.000500 loss: 0.2651 (0.3384) loss_classifier: 0.0985 (0.1239) loss_box_reg: 0.1453 (0.1643) loss_objectness: 0.0080 (0.0175) loss_rpn_box_reg: 0.0109 (0.0327) time: 0.6171 data: 0.0068 max mem: 4401 Epoch: [3] [ 60/250] eta: 0:01:58 lr: 0.000500 loss: 0.2385 (0.3261) loss_classifier: 0.0935 (0.1192) loss_box_reg: 0.1164 (0.1576) loss_objectness: 0.0068 (0.0166) loss_rpn_box_reg: 0.0112 (0.0327) time: 0.6215 data: 0.0069 max mem: 4401 Epoch: [3] [ 70/250] eta: 0:01:51 lr: 0.000500 loss: 0.2588 (0.3302) loss_classifier: 0.0986 (0.1237) loss_box_reg: 0.1287 (0.1604) loss_objectness: 0.0084 (0.0164) loss_rpn_box_reg: 0.0108 (0.0298) time: 0.5899 data: 0.0067 max mem: 4401 Epoch: [3] [ 80/250] eta: 0:01:45 lr: 0.000500 loss: 0.2999 (0.3289) loss_classifier: 0.0924 (0.1234) loss_box_reg: 0.1696 (0.1622) loss_objectness: 0.0083 (0.0155) loss_rpn_box_reg: 0.0077 (0.0278) time: 0.6020 data: 0.0067 max mem: 4401 Epoch: [3] [ 90/250] eta: 0:01:39 lr: 0.000500 loss: 0.2691 (0.3220) loss_classifier: 0.0815 (0.1220) loss_box_reg: 0.1510 (0.1586) loss_objectness: 0.0054 (0.0146) loss_rpn_box_reg: 0.0066 (0.0268) time: 0.6222 data: 0.0069 max mem: 4401 Epoch: [3] [100/250] eta: 0:01:32 lr: 0.000500 loss: 0.2969 (0.3313) loss_classifier: 0.0992 (0.1239) loss_box_reg: 0.1518 (0.1621) loss_objectness: 0.0081 (0.0148) loss_rpn_box_reg: 0.0122 (0.0304) time: 0.6207 data: 0.0068 max mem: 4401 Epoch: [3] [110/250] eta: 0:01:26 lr: 0.000500 loss: 0.3117 (0.3274) loss_classifier: 0.0989 (0.1221) loss_box_reg: 0.1554 (0.1603) loss_objectness: 0.0095 (0.0144) loss_rpn_box_reg: 0.0120 (0.0306) time: 0.6052 data: 0.0068 max mem: 4401 Epoch: [3] [120/250] eta: 0:01:19 lr: 0.000500 loss: 0.2960 (0.3287) loss_classifier: 0.1061 (0.1217) loss_box_reg: 0.1614 (0.1621) loss_objectness: 0.0076 (0.0142) loss_rpn_box_reg: 0.0106 (0.0307) time: 0.5862 data: 0.0067 max mem: 4401 Epoch: [3] [130/250] eta: 0:01:13 lr: 0.000500 loss: 0.2624 (0.3192) loss_classifier: 0.0816 (0.1182) loss_box_reg: 0.1548 (0.1582) loss_objectness: 0.0054 (0.0136) loss_rpn_box_reg: 0.0097 (0.0292) time: 0.6022 data: 0.0066 max mem: 4401 Epoch: [3] [140/250] eta: 0:01:07 lr: 0.000500 loss: 0.2624 (0.3223) loss_classifier: 0.0943 (0.1193) loss_box_reg: 0.1460 (0.1601) loss_objectness: 0.0069 (0.0136) loss_rpn_box_reg: 0.0102 (0.0293) time: 0.6332 data: 0.0065 max mem: 4401 Epoch: [3] [150/250] eta: 0:01:01 lr: 0.000500 loss: 0.3056 (0.3211) loss_classifier: 0.1139 (0.1187) loss_box_reg: 0.1678 (0.1602) loss_objectness: 0.0126 (0.0136) loss_rpn_box_reg: 0.0092 (0.0286) time: 0.6537 data: 0.0065 max mem: 4401 Epoch: [3] [160/250] eta: 0:00:55 lr: 0.000500 loss: 0.2875 (0.3213) loss_classifier: 0.0929 (0.1180) loss_box_reg: 0.1600 (0.1617) loss_objectness: 0.0104 (0.0134) loss_rpn_box_reg: 0.0080 (0.0282) time: 0.6329 data: 0.0071 max mem: 4401 Epoch: [3] [170/250] eta: 0:00:49 lr: 0.000500 loss: 0.2942 (0.3189) loss_classifier: 0.0995 (0.1175) loss_box_reg: 0.1735 (0.1613) loss_objectness: 0.0044 (0.0129) loss_rpn_box_reg: 0.0113 (0.0272) time: 0.6375 data: 0.0072 max mem: 4401 Epoch: [3] [180/250] eta: 0:00:43 lr: 0.000500 loss: 0.2773 (0.3176) loss_classifier: 0.1070 (0.1169) loss_box_reg: 0.1463 (0.1608) loss_objectness: 0.0038 (0.0127) loss_rpn_box_reg: 0.0140 (0.0272) time: 0.6413 data: 0.0067 max mem: 4401 Epoch: [3] [190/250] eta: 0:00:37 lr: 0.000500 loss: 0.2640 (0.3150) loss_classifier: 0.1010 (0.1160) loss_box_reg: 0.1205 (0.1588) loss_objectness: 0.0065 (0.0125) loss_rpn_box_reg: 0.0216 (0.0276) time: 0.6065 data: 0.0065 max mem: 4401 Epoch: [3] [200/250] eta: 0:00:31 lr: 0.000500 loss: 0.2203 (0.3109) loss_classifier: 0.0895 (0.1143) loss_box_reg: 0.1031 (0.1571) loss_objectness: 0.0051 (0.0122) loss_rpn_box_reg: 0.0082 (0.0273) time: 0.6428 data: 0.0064 max mem: 4401 Epoch: [3] [210/250] eta: 0:00:24 lr: 0.000500 loss: 0.2134 (0.3106) loss_classifier: 0.0739 (0.1139) loss_box_reg: 0.1254 (0.1575) loss_objectness: 0.0045 (0.0122) loss_rpn_box_reg: 0.0062 (0.0269) time: 0.6389 data: 0.0068 max mem: 4401 Epoch: [3] [220/250] eta: 0:00:18 lr: 0.000500 loss: 0.2134 (0.3077) loss_classifier: 0.0802 (0.1135) loss_box_reg: 0.1171 (0.1559) loss_objectness: 0.0097 (0.0122) loss_rpn_box_reg: 0.0087 (0.0262) time: 0.5995 data: 0.0070 max mem: 4401 Epoch: [3] [230/250] eta: 0:00:12 lr: 0.000500 loss: 0.2028 (0.3019) loss_classifier: 0.0658 (0.1111) loss_box_reg: 0.0987 (0.1533) loss_objectness: 0.0056 (0.0118) loss_rpn_box_reg: 0.0066 (0.0256) time: 0.6096 data: 0.0066 max mem: 4401 Epoch: [3] [240/250] eta: 0:00:06 lr: 0.000500 loss: 0.1990 (0.3019) loss_classifier: 0.0799 (0.1111) loss_box_reg: 0.1058 (0.1530) loss_objectness: 0.0060 (0.0120) loss_rpn_box_reg: 0.0057 (0.0259) time: 0.6291 data: 0.0065 max mem: 4401 Epoch: [3] [249/250] eta: 0:00:00 lr: 0.000500 loss: 0.2175 (0.3016) loss_classifier: 0.0886 (0.1113) loss_box_reg: 0.1263 (0.1528) loss_objectness: 0.0089 (0.0118) loss_rpn_box_reg: 0.0076 (0.0257) time: 0.6131 data: 0.0065 max mem: 4401 Epoch: [3] Total time: 0:02:35 (0.6209 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:08 model_time: 0.1713 (0.1713) evaluator_time: 0.0073 (0.0073) time: 0.3321 data: 0.1515 max mem: 4401 Test: [100/207] eta: 0:00:13 model_time: 0.1211 (0.1176) evaluator_time: 0.0018 (0.0031) time: 0.1284 data: 0.0035 max mem: 4401 Test: [200/207] eta: 0:00:00 model_time: 0.1128 (0.1175) evaluator_time: 0.0016 (0.0030) time: 0.1211 data: 0.0037 max mem: 4401 Test: [206/207] eta: 0:00:00 model_time: 0.1129 (0.1174) evaluator_time: 0.0026 (0.0032) time: 0.1239 data: 0.0037 max mem: 4401 Test: Total time: 0:00:26 (0.1267 s / it) Averaged stats: model_time: 0.1129 (0.1174) evaluator_time: 0.0026 (0.0032) Accumulating evaluation results... DONE (t=0.09s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.359 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.578 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.378 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.215 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.346 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.460 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.313 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.541 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.570 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.436 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.550 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.690
One of the main draws of FiftyOne is the ability to find failure modes of your model. The built-in evaluation protocols will help you find where your model got things right and where it got things wrong. Before we can evaluate the model, we need to run it on our test set and store the results in FiftyOne. Doing this is fairly simple and just requires us to run inference for the test images, get their corresponding FiftyOne samples, and add a new field called predictions
to each sample to store the detections.
import fiftyone as fo
def convert_torch_predictions(preds, det_id, s_id, w, h, classes):
# Convert the outputs of the torch model into a FiftyOne Detections object
dets = []
for bbox, label, score in zip(
preds["boxes"].cpu().detach().numpy(),
preds["labels"].cpu().detach().numpy(),
preds["scores"].cpu().detach().numpy()
):
# Parse prediction into FiftyOne Detection object
x0,y0,x1,y1 = bbox
coco_obj = fouc.COCOObject(det_id, s_id, int(label), [x0, y0, x1-x0, y1-y0])
det = coco_obj.to_detection((w,h), classes)
det["confidence"] = float(score)
dets.append(det)
det_id += 1
detections = fo.Detections(detections=dets)
return detections, det_id
def add_detections(model, torch_dataset, view, field_name="predictions"):
# Run inference on a dataset and add results to FiftyOne
torch.set_num_threads(1)
device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')
print("Using device %s" % device)
model.eval()
model.to(device)
image_paths = torch_dataset.img_paths
classes = torch_dataset.classes
det_id = 0
with fo.ProgressBar() as pb:
for img, targets in pb(torch_dataset):
# Get FiftyOne sample indexed by unique image filepath
img_id = int(targets["image_id"][0])
img_path = image_paths[img_id]
sample = view[img_path]
s_id = sample.id
w = sample.metadata["width"]
h = sample.metadata["height"]
# Inference
preds = model(img.unsqueeze(0).to(device))[0]
detections, det_id = convert_torch_predictions(
preds,
det_id,
s_id,
w,
h,
classes,
)
sample[field_name] = detections
sample.save()
add_detections(model, torch_dataset_test, fo_dataset, field_name="predictions")
Using device cuda 100% |█████████████████| 207/207 [31.5s elapsed, 0s remaining, 6.5 samples/s]
results = fo.evaluate_detections(
test_view,
"predictions",
classes=["car", "bus", "truck"],
eval_key="eval",
compute_mAP=True
)
Evaluating detections... 100% |█████████████████| 207/207 [5.5s elapsed, 0s remaining, 36.5 samples/s] Performing IoU sweep... 100% |█████████████████| 207/207 [8.1s elapsed, 0s remaining, 22.2 samples/s]
The DetectionResults object that is returned stores information like the mAP and contains functions that let you plot confusion matrices, precision-recall curves, and more. Also, these evaluation runs are tracked in FiftyOne and can be managed through functions like list_evaluations().
results.mAP()
0.3598547617444416
results.print_report()
precision recall f1-score support car 0.20 0.87 0.32 610 bus 0.13 0.88 0.23 68 truck 0.10 0.88 0.18 112 micro avg 0.17 0.87 0.28 790 macro avg 0.14 0.88 0.24 790 weighted avg 0.18 0.87 0.29 790
By default, objects are only matched with other objects of the same class. In order to get an interesting confusion matrix, we need to match interclass objects by setting classwise=False
.
results_interclass = fo.evaluate_detections(
test_view,
"predictions",
classes=["car", "bus", "truck"],
compute_mAP=True,
classwise=False
)
Evaluating detections... 100% |█████████████████| 207/207 [6.3s elapsed, 0s remaining, 30.2 samples/s] Performing IoU sweep... 100% |█████████████████| 207/207 [9.7s elapsed, 0s remaining, 16.9 samples/s]
results_interclass.plot_confusion_matrix()
Note that there appears to be confusion between car and truck classes.
The detection evaluation also added the attributes eval_fp
, eval_tp
, and eval_fn
to every predicted detection indicating if it is a false positive, true positive, or false negative.
Let's create a view to find the worst samples by sorting by eval_fp
using the FiftyOne App to visualize the results.
session.view = test_view.sort_by("eval_fp", reverse=True)
Looking through some of these samples, we can see the confusion between the classes "car" and "truck" that we found earlier. However, this seems to be, at least in part, due to annotation errors on vans and SUVs where they are interchangably labeled as "car" and "truck". The example below shows an SUV annotated as "truck" but predicted as "car".
session.view = test_view.sort_by("eval_fp", reverse=True)
It would be best to get this data reannotated to fix these mistakes, but in the meantime, we can easily remedy this by simply creating a new view that remaps the labels car
, truck
, and bus
all to vehicle
and then retraining the model with that. This is only possible because we are backing our data in FiftyOne and loading views into PyTorch as needed. Without FiftyOne, the PyTorch dataset class or the underlying data would need to be changed to remap these classes.
# map labels to single vehicle class
vehicles_map = {c: "vehicle" for c in vehicles_list}
train_map_view = train_view.map_labels("ground_truth", vehicles_map)
test_map_view = test_view.map_labels("ground_truth", vehicles_map)
# use our dataset and defined transformations
torch_map_dataset = FiftyOneTorchDataset(train_map_view, train_transforms)
torch_map_dataset_test = FiftyOneTorchDataset(test_map_view, test_transforms)
# Only 2 classes (background and vehicle)
vehicle_model = get_model(2)
do_training(vehicle_model, torch_map_dataset, torch_map_dataset_test)
Using device cuda
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [0] [ 0/250] eta: 0:04:47 lr: 0.000025 loss: 1.5123 (1.5123) loss_classifier: 0.9323 (0.9323) loss_box_reg: 0.5551 (0.5551) loss_objectness: 0.0071 (0.0071) loss_rpn_box_reg: 0.0178 (0.0178) time: 1.1514 data: 0.2843 max mem: 4401 Epoch: [0] [ 10/250] eta: 0:02:35 lr: 0.000226 loss: 1.1389 (1.1536) loss_classifier: 0.8300 (0.7482) loss_box_reg: 0.3211 (0.3252) loss_objectness: 0.0288 (0.0414) loss_rpn_box_reg: 0.0125 (0.0387) time: 0.6473 data: 0.0319 max mem: 4401 Epoch: [0] [ 20/250] eta: 0:02:28 lr: 0.000426 loss: 1.0465 (1.0374) loss_classifier: 0.5113 (0.5782) loss_box_reg: 0.3211 (0.3764) loss_objectness: 0.0261 (0.0420) loss_rpn_box_reg: 0.0125 (0.0407) time: 0.6196 data: 0.0066 max mem: 4401 Epoch: [0] [ 30/250] eta: 0:02:24 lr: 0.000627 loss: 0.7177 (0.9652) loss_classifier: 0.3202 (0.5004) loss_box_reg: 0.4171 (0.3951) loss_objectness: 0.0187 (0.0346) loss_rpn_box_reg: 0.0168 (0.0350) time: 0.6625 data: 0.0066 max mem: 4401 Epoch: [0] [ 40/250] eta: 0:02:15 lr: 0.000827 loss: 0.5643 (0.8700) loss_classifier: 0.2055 (0.4228) loss_box_reg: 0.3185 (0.3841) loss_objectness: 0.0171 (0.0338) loss_rpn_box_reg: 0.0102 (0.0292) time: 0.6452 data: 0.0066 max mem: 4401 Epoch: [0] [ 50/250] eta: 0:02:07 lr: 0.001028 loss: 0.5635 (0.8148) loss_classifier: 0.1749 (0.3717) loss_box_reg: 0.3185 (0.3814) loss_objectness: 0.0303 (0.0343) loss_rpn_box_reg: 0.0115 (0.0274) time: 0.6009 data: 0.0067 max mem: 4401 Epoch: [0] [ 60/250] eta: 0:01:59 lr: 0.001229 loss: 0.4994 (0.7727) loss_classifier: 0.1450 (0.3337) loss_box_reg: 0.3095 (0.3741) loss_objectness: 0.0259 (0.0373) loss_rpn_box_reg: 0.0115 (0.0276) time: 0.6004 data: 0.0072 max mem: 4401 Epoch: [0] [ 70/250] eta: 0:01:52 lr: 0.001429 loss: 0.4577 (0.7237) loss_classifier: 0.1191 (0.3026) loss_box_reg: 0.2745 (0.3532) loss_objectness: 0.0240 (0.0376) loss_rpn_box_reg: 0.0115 (0.0302) time: 0.5992 data: 0.0073 max mem: 4401 Epoch: [0] [ 80/250] eta: 0:01:45 lr: 0.001630 loss: 0.4577 (0.6851) loss_classifier: 0.1058 (0.2799) loss_box_reg: 0.1997 (0.3380) loss_objectness: 0.0226 (0.0361) loss_rpn_box_reg: 0.0169 (0.0311) time: 0.5929 data: 0.0071 max mem: 4401 Epoch: [0] [ 90/250] eta: 0:01:38 lr: 0.001830 loss: 0.3704 (0.6538) loss_classifier: 0.1109 (0.2630) loss_box_reg: 0.2411 (0.3249) loss_objectness: 0.0264 (0.0361) loss_rpn_box_reg: 0.0129 (0.0298) time: 0.5899 data: 0.0070 max mem: 4401 Epoch: [0] [100/250] eta: 0:01:32 lr: 0.002031 loss: 0.3704 (0.6327) loss_classifier: 0.1113 (0.2498) loss_box_reg: 0.2208 (0.3154) loss_objectness: 0.0264 (0.0357) loss_rpn_box_reg: 0.0142 (0.0318) time: 0.5875 data: 0.0070 max mem: 4401 Epoch: [0] [110/250] eta: 0:01:26 lr: 0.002232 loss: 0.3363 (0.6076) loss_classifier: 0.1264 (0.2381) loss_box_reg: 0.1662 (0.3018) loss_objectness: 0.0287 (0.0369) loss_rpn_box_reg: 0.0120 (0.0308) time: 0.6119 data: 0.0070 max mem: 4401 Epoch: [0] [120/250] eta: 0:01:21 lr: 0.002432 loss: 0.3157 (0.5894) loss_classifier: 0.1040 (0.2283) loss_box_reg: 0.1581 (0.2951) loss_objectness: 0.0128 (0.0353) loss_rpn_box_reg: 0.0135 (0.0307) time: 0.6737 data: 0.0066 max mem: 4533 Epoch: [0] [130/250] eta: 0:01:15 lr: 0.002633 loss: 0.4046 (0.5778) loss_classifier: 0.1315 (0.2218) loss_box_reg: 0.1998 (0.2909) loss_objectness: 0.0128 (0.0353) loss_rpn_box_reg: 0.0149 (0.0298) time: 0.6756 data: 0.0070 max mem: 4533 Epoch: [0] [140/250] eta: 0:01:08 lr: 0.002833 loss: 0.4122 (0.5675) loss_classifier: 0.1386 (0.2157) loss_box_reg: 0.1871 (0.2833) loss_objectness: 0.0280 (0.0372) loss_rpn_box_reg: 0.0151 (0.0313) time: 0.6227 data: 0.0073 max mem: 4533 Epoch: [0] [150/250] eta: 0:01:02 lr: 0.003034 loss: 0.3331 (0.5530) loss_classifier: 0.1093 (0.2089) loss_box_reg: 0.1555 (0.2757) loss_objectness: 0.0371 (0.0377) loss_rpn_box_reg: 0.0119 (0.0308) time: 0.6210 data: 0.0071 max mem: 4533 Epoch: [0] [160/250] eta: 0:00:56 lr: 0.003235 loss: 0.2743 (0.5364) loss_classifier: 0.0888 (0.2014) loss_box_reg: 0.1502 (0.2681) loss_objectness: 0.0178 (0.0367) loss_rpn_box_reg: 0.0075 (0.0302) time: 0.6430 data: 0.0073 max mem: 4533 Epoch: [0] [170/250] eta: 0:00:50 lr: 0.003435 loss: 0.3487 (0.5292) loss_classifier: 0.1035 (0.1986) loss_box_reg: 0.1502 (0.2623) loss_objectness: 0.0256 (0.0374) loss_rpn_box_reg: 0.0131 (0.0310) time: 0.6522 data: 0.0074 max mem: 4533 Epoch: [0] [180/250] eta: 0:00:43 lr: 0.003636 loss: 0.3487 (0.5192) loss_classifier: 0.1153 (0.1948) loss_box_reg: 0.1466 (0.2577) loss_objectness: 0.0216 (0.0363) loss_rpn_box_reg: 0.0206 (0.0305) time: 0.6394 data: 0.0073 max mem: 4533 Epoch: [0] [190/250] eta: 0:00:37 lr: 0.003837 loss: 0.2860 (0.5115) loss_classifier: 0.1059 (0.1900) loss_box_reg: 0.1257 (0.2518) loss_objectness: 0.0165 (0.0382) loss_rpn_box_reg: 0.0163 (0.0314) time: 0.6321 data: 0.0074 max mem: 4533 Epoch: [0] [200/250] eta: 0:00:31 lr: 0.004037 loss: 0.3070 (0.5043) loss_classifier: 0.0995 (0.1862) loss_box_reg: 0.1395 (0.2493) loss_objectness: 0.0185 (0.0380) loss_rpn_box_reg: 0.0135 (0.0309) time: 0.6431 data: 0.0078 max mem: 4533 Epoch: [0] [210/250] eta: 0:00:25 lr: 0.004238 loss: 0.3828 (0.5003) loss_classifier: 0.1123 (0.1837) loss_box_reg: 0.2287 (0.2490) loss_objectness: 0.0276 (0.0372) loss_rpn_box_reg: 0.0117 (0.0304) time: 0.6685 data: 0.0071 max mem: 4722 Epoch: [0] [220/250] eta: 0:00:18 lr: 0.004438 loss: 0.4111 (0.4963) loss_classifier: 0.1251 (0.1814) loss_box_reg: 0.2181 (0.2471) loss_objectness: 0.0243 (0.0376) loss_rpn_box_reg: 0.0092 (0.0302) time: 0.6522 data: 0.0065 max mem: 4722 Epoch: [0] [230/250] eta: 0:00:12 lr: 0.004639 loss: 0.4111 (0.4931) loss_classifier: 0.1093 (0.1783) loss_box_reg: 0.2089 (0.2443) loss_objectness: 0.0484 (0.0397) loss_rpn_box_reg: 0.0213 (0.0308) time: 0.6243 data: 0.0066 max mem: 4722 Epoch: [0] [240/250] eta: 0:00:06 lr: 0.004840 loss: 0.3850 (0.4962) loss_classifier: 0.1090 (0.1783) loss_box_reg: 0.1370 (0.2424) loss_objectness: 0.0574 (0.0430) loss_rpn_box_reg: 0.0196 (0.0326) time: 0.6253 data: 0.0066 max mem: 4722 Epoch: [0] [249/250] eta: 0:00:00 lr: 0.005000 loss: 0.3593 (0.4939) loss_classifier: 0.1267 (0.1766) loss_box_reg: 0.1234 (0.2388) loss_objectness: 0.0706 (0.0445) loss_rpn_box_reg: 0.0209 (0.0340) time: 0.6304 data: 0.0066 max mem: 4722 Epoch: [0] Total time: 0:02:37 (0.6313 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:10 model_time: 0.1596 (0.1596) evaluator_time: 0.0073 (0.0073) time: 0.3383 data: 0.1694 max mem: 4722 Test: [100/207] eta: 0:00:13 model_time: 0.1203 (0.1171) evaluator_time: 0.0024 (0.0046) time: 0.1296 data: 0.0037 max mem: 4722 Test: [200/207] eta: 0:00:00 model_time: 0.1130 (0.1172) evaluator_time: 0.0025 (0.0047) time: 0.1215 data: 0.0033 max mem: 4722 Test: [206/207] eta: 0:00:00 model_time: 0.1128 (0.1171) evaluator_time: 0.0034 (0.0049) time: 0.1248 data: 0.0033 max mem: 4722 Test: Total time: 0:00:26 (0.1281 s / it) Averaged stats: model_time: 0.1128 (0.1171) evaluator_time: 0.0034 (0.0049) Accumulating evaluation results... DONE (t=0.07s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.262 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.507 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.245 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.197 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.308 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.416 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.134 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.360 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.419 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.338 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.451 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.595
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [1] [ 0/250] eta: 0:03:48 lr: 0.005000 loss: 0.2260 (0.2260) loss_classifier: 0.0742 (0.0742) loss_box_reg: 0.1216 (0.1216) loss_objectness: 0.0229 (0.0229) loss_rpn_box_reg: 0.0073 (0.0073) time: 0.9124 data: 0.3390 max mem: 4722 Epoch: [1] [ 10/250] eta: 0:02:36 lr: 0.005000 loss: 0.4380 (0.4907) loss_classifier: 0.1351 (0.1584) loss_box_reg: 0.1715 (0.2241) loss_objectness: 0.0317 (0.0482) loss_rpn_box_reg: 0.0167 (0.0600) time: 0.6532 data: 0.0360 max mem: 4722 Epoch: [1] [ 20/250] eta: 0:02:29 lr: 0.005000 loss: 0.3960 (0.4844) loss_classifier: 0.1281 (0.1492) loss_box_reg: 0.1715 (0.2056) loss_objectness: 0.0358 (0.0673) loss_rpn_box_reg: 0.0177 (0.0623) time: 0.6360 data: 0.0061 max mem: 4722 Epoch: [1] [ 30/250] eta: 0:02:17 lr: 0.005000 loss: 0.3671 (0.4388) loss_classifier: 0.1112 (0.1335) loss_box_reg: 0.1653 (0.1906) loss_objectness: 0.0371 (0.0578) loss_rpn_box_reg: 0.0173 (0.0570) time: 0.6098 data: 0.0066 max mem: 4722 Epoch: [1] [ 40/250] eta: 0:02:10 lr: 0.005000 loss: 0.3153 (0.4099) loss_classifier: 0.0969 (0.1271) loss_box_reg: 0.1344 (0.1804) loss_objectness: 0.0310 (0.0536) loss_rpn_box_reg: 0.0164 (0.0487) time: 0.5919 data: 0.0068 max mem: 4722 Epoch: [1] [ 50/250] eta: 0:02:04 lr: 0.005000 loss: 0.3379 (0.4230) loss_classifier: 0.1128 (0.1333) loss_box_reg: 0.1771 (0.1905) loss_objectness: 0.0257 (0.0509) loss_rpn_box_reg: 0.0170 (0.0482) time: 0.6143 data: 0.0067 max mem: 4722 Epoch: [1] [ 60/250] eta: 0:01:58 lr: 0.005000 loss: 0.3574 (0.4030) loss_classifier: 0.1075 (0.1279) loss_box_reg: 0.1640 (0.1838) loss_objectness: 0.0239 (0.0457) loss_rpn_box_reg: 0.0098 (0.0456) time: 0.6247 data: 0.0066 max mem: 4722 Epoch: [1] [ 70/250] eta: 0:01:51 lr: 0.005000 loss: 0.3245 (0.3916) loss_classifier: 0.0975 (0.1251) loss_box_reg: 0.1371 (0.1810) loss_objectness: 0.0138 (0.0424) loss_rpn_box_reg: 0.0082 (0.0431) time: 0.6243 data: 0.0067 max mem: 4722 Epoch: [1] [ 80/250] eta: 0:01:45 lr: 0.005000 loss: 0.3548 (0.4036) loss_classifier: 0.1278 (0.1334) loss_box_reg: 0.1713 (0.1831) loss_objectness: 0.0274 (0.0433) loss_rpn_box_reg: 0.0204 (0.0439) time: 0.6185 data: 0.0067 max mem: 4722 Epoch: [1] [ 90/250] eta: 0:01:38 lr: 0.005000 loss: 0.3772 (0.3997) loss_classifier: 0.1479 (0.1351) loss_box_reg: 0.1760 (0.1809) loss_objectness: 0.0274 (0.0412) loss_rpn_box_reg: 0.0271 (0.0424) time: 0.6057 data: 0.0066 max mem: 4722 Epoch: [1] [100/250] eta: 0:01:32 lr: 0.005000 loss: 0.3772 (0.4120) loss_classifier: 0.1295 (0.1391) loss_box_reg: 0.2125 (0.1897) loss_objectness: 0.0202 (0.0406) loss_rpn_box_reg: 0.0215 (0.0426) time: 0.6011 data: 0.0069 max mem: 4722 Epoch: [1] [110/250] eta: 0:01:26 lr: 0.005000 loss: 0.4882 (0.4238) loss_classifier: 0.1633 (0.1429) loss_box_reg: 0.2770 (0.1992) loss_objectness: 0.0210 (0.0398) loss_rpn_box_reg: 0.0212 (0.0419) time: 0.6070 data: 0.0071 max mem: 4722 Epoch: [1] [120/250] eta: 0:01:20 lr: 0.005000 loss: 0.4424 (0.4215) loss_classifier: 0.1509 (0.1426) loss_box_reg: 0.2372 (0.1982) loss_objectness: 0.0243 (0.0390) loss_rpn_box_reg: 0.0212 (0.0417) time: 0.6373 data: 0.0067 max mem: 4722 Epoch: [1] [130/250] eta: 0:01:14 lr: 0.005000 loss: 0.3083 (0.4120) loss_classifier: 0.1214 (0.1402) loss_box_reg: 0.1500 (0.1943) loss_objectness: 0.0229 (0.0372) loss_rpn_box_reg: 0.0158 (0.0403) time: 0.6323 data: 0.0068 max mem: 4722 Epoch: [1] [140/250] eta: 0:01:07 lr: 0.005000 loss: 0.3083 (0.4095) loss_classifier: 0.1214 (0.1390) loss_box_reg: 0.1667 (0.1947) loss_objectness: 0.0101 (0.0358) loss_rpn_box_reg: 0.0214 (0.0400) time: 0.5895 data: 0.0070 max mem: 4722 Epoch: [1] [150/250] eta: 0:01:01 lr: 0.005000 loss: 0.3217 (0.4036) loss_classifier: 0.0945 (0.1362) loss_box_reg: 0.1705 (0.1923) loss_objectness: 0.0110 (0.0352) loss_rpn_box_reg: 0.0163 (0.0399) time: 0.6008 data: 0.0068 max mem: 4722 Epoch: [1] [160/250] eta: 0:00:55 lr: 0.005000 loss: 0.3606 (0.3998) loss_classifier: 0.0966 (0.1352) loss_box_reg: 0.1771 (0.1923) loss_objectness: 0.0123 (0.0339) loss_rpn_box_reg: 0.0119 (0.0383) time: 0.6264 data: 0.0074 max mem: 4722 Epoch: [1] [170/250] eta: 0:00:49 lr: 0.005000 loss: 0.3350 (0.3959) loss_classifier: 0.1060 (0.1343) loss_box_reg: 0.1939 (0.1920) loss_objectness: 0.0128 (0.0330) loss_rpn_box_reg: 0.0104 (0.0366) time: 0.6413 data: 0.0078 max mem: 4722 Epoch: [1] [180/250] eta: 0:00:43 lr: 0.005000 loss: 0.3195 (0.3966) loss_classifier: 0.1070 (0.1351) loss_box_reg: 0.1639 (0.1908) loss_objectness: 0.0158 (0.0338) loss_rpn_box_reg: 0.0133 (0.0369) time: 0.6152 data: 0.0073 max mem: 4722 Epoch: [1] [190/250] eta: 0:00:37 lr: 0.005000 loss: 0.2176 (0.3884) loss_classifier: 0.1070 (0.1330) loss_box_reg: 0.1231 (0.1859) loss_objectness: 0.0226 (0.0331) loss_rpn_box_reg: 0.0175 (0.0364) time: 0.5990 data: 0.0070 max mem: 4722 Epoch: [1] [200/250] eta: 0:00:30 lr: 0.005000 loss: 0.2095 (0.3844) loss_classifier: 0.0809 (0.1321) loss_box_reg: 0.0945 (0.1839) loss_objectness: 0.0196 (0.0325) loss_rpn_box_reg: 0.0175 (0.0358) time: 0.6188 data: 0.0070 max mem: 4722 Epoch: [1] [210/250] eta: 0:00:24 lr: 0.005000 loss: 0.2791 (0.3822) loss_classifier: 0.1129 (0.1319) loss_box_reg: 0.1148 (0.1827) loss_objectness: 0.0196 (0.0321) loss_rpn_box_reg: 0.0172 (0.0355) time: 0.6508 data: 0.0070 max mem: 4722 Epoch: [1] [220/250] eta: 0:00:18 lr: 0.005000 loss: 0.4018 (0.3878) loss_classifier: 0.1442 (0.1338) loss_box_reg: 0.1818 (0.1860) loss_objectness: 0.0208 (0.0321) loss_rpn_box_reg: 0.0172 (0.0358) time: 0.6436 data: 0.0070 max mem: 4722 Epoch: [1] [230/250] eta: 0:00:12 lr: 0.005000 loss: 0.4029 (0.3874) loss_classifier: 0.1442 (0.1342) loss_box_reg: 0.1804 (0.1855) loss_objectness: 0.0271 (0.0320) loss_rpn_box_reg: 0.0165 (0.0357) time: 0.6020 data: 0.0072 max mem: 4722 Epoch: [1] [240/250] eta: 0:00:06 lr: 0.005000 loss: 0.4094 (0.3896) loss_classifier: 0.1413 (0.1351) loss_box_reg: 0.1645 (0.1871) loss_objectness: 0.0163 (0.0314) loss_rpn_box_reg: 0.0117 (0.0360) time: 0.6465 data: 0.0075 max mem: 4722 Epoch: [1] [249/250] eta: 0:00:00 lr: 0.005000 loss: 0.4166 (0.3906) loss_classifier: 0.1732 (0.1352) loss_box_reg: 0.1937 (0.1879) loss_objectness: 0.0163 (0.0313) loss_rpn_box_reg: 0.0117 (0.0361) time: 0.6474 data: 0.0070 max mem: 4722 Epoch: [1] Total time: 0:02:35 (0.6214 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:11 model_time: 0.1580 (0.1580) evaluator_time: 0.0092 (0.0092) time: 0.3455 data: 0.1763 max mem: 4722 Test: [100/207] eta: 0:00:13 model_time: 0.1207 (0.1171) evaluator_time: 0.0023 (0.0044) time: 0.1285 data: 0.0037 max mem: 4722 Test: [200/207] eta: 0:00:00 model_time: 0.1136 (0.1171) evaluator_time: 0.0022 (0.0044) time: 0.1219 data: 0.0040 max mem: 4722 Test: [206/207] eta: 0:00:00 model_time: 0.1140 (0.1171) evaluator_time: 0.0034 (0.0047) time: 0.1263 data: 0.0040 max mem: 4722 Test: Total time: 0:00:26 (0.1280 s / it) Averaged stats: model_time: 0.1140 (0.1171) evaluator_time: 0.0034 (0.0047) Accumulating evaluation results... DONE (t=0.07s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.375 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.663 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.403 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.291 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.444 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.508 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.165 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.457 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.517 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.443 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.579 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.620
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [2] [ 0/250] eta: 0:03:53 lr: 0.005000 loss: 0.0637 (0.0637) loss_classifier: 0.0306 (0.0306) loss_box_reg: 0.0283 (0.0283) loss_objectness: 0.0016 (0.0016) loss_rpn_box_reg: 0.0031 (0.0031) time: 0.9345 data: 0.3557 max mem: 4722 Epoch: [2] [ 10/250] eta: 0:02:39 lr: 0.005000 loss: 0.2129 (0.2930) loss_classifier: 0.1072 (0.1128) loss_box_reg: 0.1368 (0.1522) loss_objectness: 0.0115 (0.0165) loss_rpn_box_reg: 0.0088 (0.0114) time: 0.6641 data: 0.0378 max mem: 4722 Epoch: [2] [ 20/250] eta: 0:02:31 lr: 0.005000 loss: 0.2121 (0.2590) loss_classifier: 0.0808 (0.0958) loss_box_reg: 0.0980 (0.1370) loss_objectness: 0.0114 (0.0135) loss_rpn_box_reg: 0.0087 (0.0126) time: 0.6431 data: 0.0063 max mem: 4722 Epoch: [2] [ 30/250] eta: 0:02:24 lr: 0.005000 loss: 0.1546 (0.2640) loss_classifier: 0.0635 (0.0953) loss_box_reg: 0.0903 (0.1413) loss_objectness: 0.0073 (0.0118) loss_rpn_box_reg: 0.0068 (0.0156) time: 0.6540 data: 0.0066 max mem: 4722 Epoch: [2] [ 40/250] eta: 0:02:15 lr: 0.005000 loss: 0.2453 (0.2806) loss_classifier: 0.0893 (0.0993) loss_box_reg: 0.1305 (0.1536) loss_objectness: 0.0066 (0.0124) loss_rpn_box_reg: 0.0067 (0.0154) time: 0.6350 data: 0.0074 max mem: 4722 Epoch: [2] [ 50/250] eta: 0:02:08 lr: 0.005000 loss: 0.3236 (0.2907) loss_classifier: 0.1158 (0.1029) loss_box_reg: 0.1868 (0.1593) loss_objectness: 0.0070 (0.0125) loss_rpn_box_reg: 0.0084 (0.0160) time: 0.6131 data: 0.0078 max mem: 4722 Epoch: [2] [ 60/250] eta: 0:02:01 lr: 0.005000 loss: 0.3636 (0.3096) loss_classifier: 0.1372 (0.1094) loss_box_reg: 0.1964 (0.1661) loss_objectness: 0.0158 (0.0160) loss_rpn_box_reg: 0.0105 (0.0181) time: 0.6274 data: 0.0071 max mem: 4722 Epoch: [2] [ 70/250] eta: 0:01:54 lr: 0.005000 loss: 0.4012 (0.3217) loss_classifier: 0.1363 (0.1135) loss_box_reg: 0.1976 (0.1713) loss_objectness: 0.0220 (0.0169) loss_rpn_box_reg: 0.0148 (0.0199) time: 0.6331 data: 0.0070 max mem: 4722 Epoch: [2] [ 80/250] eta: 0:01:47 lr: 0.005000 loss: 0.4053 (0.3383) loss_classifier: 0.1319 (0.1175) loss_box_reg: 0.2088 (0.1788) loss_objectness: 0.0234 (0.0188) loss_rpn_box_reg: 0.0204 (0.0232) time: 0.6052 data: 0.0070 max mem: 4722 Epoch: [2] [ 90/250] eta: 0:01:41 lr: 0.005000 loss: 0.3911 (0.3339) loss_classifier: 0.1216 (0.1147) loss_box_reg: 0.1711 (0.1777) loss_objectness: 0.0156 (0.0182) loss_rpn_box_reg: 0.0158 (0.0234) time: 0.6115 data: 0.0068 max mem: 4722 Epoch: [2] [100/250] eta: 0:01:34 lr: 0.005000 loss: 0.2519 (0.3337) loss_classifier: 0.0933 (0.1142) loss_box_reg: 0.1363 (0.1777) loss_objectness: 0.0126 (0.0182) loss_rpn_box_reg: 0.0153 (0.0236) time: 0.6373 data: 0.0067 max mem: 4722 Epoch: [2] [110/250] eta: 0:01:28 lr: 0.005000 loss: 0.2847 (0.3340) loss_classifier: 0.0933 (0.1129) loss_box_reg: 0.1560 (0.1786) loss_objectness: 0.0145 (0.0183) loss_rpn_box_reg: 0.0132 (0.0241) time: 0.6365 data: 0.0069 max mem: 4722 Epoch: [2] [120/250] eta: 0:01:21 lr: 0.005000 loss: 0.2334 (0.3217) loss_classifier: 0.0636 (0.1092) loss_box_reg: 0.1219 (0.1716) loss_objectness: 0.0098 (0.0177) loss_rpn_box_reg: 0.0117 (0.0232) time: 0.6018 data: 0.0067 max mem: 4722 Epoch: [2] [130/250] eta: 0:01:15 lr: 0.005000 loss: 0.2218 (0.3242) loss_classifier: 0.0836 (0.1094) loss_box_reg: 0.0942 (0.1714) loss_objectness: 0.0125 (0.0182) loss_rpn_box_reg: 0.0137 (0.0251) time: 0.5867 data: 0.0066 max mem: 4722 Epoch: [2] [140/250] eta: 0:01:08 lr: 0.005000 loss: 0.2659 (0.3248) loss_classifier: 0.0960 (0.1098) loss_box_reg: 0.1178 (0.1704) loss_objectness: 0.0221 (0.0193) loss_rpn_box_reg: 0.0200 (0.0253) time: 0.5923 data: 0.0067 max mem: 4722 Epoch: [2] [150/250] eta: 0:01:02 lr: 0.005000 loss: 0.2990 (0.3265) loss_classifier: 0.1055 (0.1110) loss_box_reg: 0.1368 (0.1713) loss_objectness: 0.0179 (0.0191) loss_rpn_box_reg: 0.0099 (0.0251) time: 0.6004 data: 0.0070 max mem: 4722 Epoch: [2] [160/250] eta: 0:00:56 lr: 0.005000 loss: 0.3695 (0.3301) loss_classifier: 0.1121 (0.1121) loss_box_reg: 0.1810 (0.1731) loss_objectness: 0.0163 (0.0191) loss_rpn_box_reg: 0.0113 (0.0258) time: 0.6330 data: 0.0071 max mem: 4722 Epoch: [2] [170/250] eta: 0:00:49 lr: 0.005000 loss: 0.3311 (0.3337) loss_classifier: 0.0969 (0.1125) loss_box_reg: 0.1938 (0.1741) loss_objectness: 0.0157 (0.0192) loss_rpn_box_reg: 0.0162 (0.0279) time: 0.6402 data: 0.0073 max mem: 4722 Epoch: [2] [180/250] eta: 0:00:43 lr: 0.005000 loss: 0.3311 (0.3343) loss_classifier: 0.0988 (0.1123) loss_box_reg: 0.1752 (0.1751) loss_objectness: 0.0107 (0.0190) loss_rpn_box_reg: 0.0238 (0.0278) time: 0.6250 data: 0.0071 max mem: 4722 Epoch: [2] [190/250] eta: 0:00:37 lr: 0.005000 loss: 0.3159 (0.3355) loss_classifier: 0.1008 (0.1132) loss_box_reg: 0.1315 (0.1762) loss_objectness: 0.0107 (0.0187) loss_rpn_box_reg: 0.0114 (0.0275) time: 0.6239 data: 0.0067 max mem: 4722 Epoch: [2] [200/250] eta: 0:00:31 lr: 0.005000 loss: 0.3492 (0.3401) loss_classifier: 0.1314 (0.1145) loss_box_reg: 0.1712 (0.1779) loss_objectness: 0.0145 (0.0193) loss_rpn_box_reg: 0.0155 (0.0284) time: 0.6070 data: 0.0065 max mem: 4722 Epoch: [2] [210/250] eta: 0:00:24 lr: 0.005000 loss: 0.3398 (0.3361) loss_classifier: 0.1102 (0.1135) loss_box_reg: 0.1528 (0.1758) loss_objectness: 0.0113 (0.0187) loss_rpn_box_reg: 0.0114 (0.0280) time: 0.5989 data: 0.0065 max mem: 4722 Epoch: [2] [220/250] eta: 0:00:18 lr: 0.005000 loss: 0.2471 (0.3361) loss_classifier: 0.0930 (0.1129) loss_box_reg: 0.1283 (0.1754) loss_objectness: 0.0072 (0.0186) loss_rpn_box_reg: 0.0097 (0.0292) time: 0.6036 data: 0.0068 max mem: 4722 Epoch: [2] [230/250] eta: 0:00:12 lr: 0.005000 loss: 0.2549 (0.3378) loss_classifier: 0.1038 (0.1127) loss_box_reg: 0.1310 (0.1763) loss_objectness: 0.0109 (0.0188) loss_rpn_box_reg: 0.0194 (0.0300) time: 0.6081 data: 0.0067 max mem: 4722 Epoch: [2] [240/250] eta: 0:00:06 lr: 0.005000 loss: 0.3443 (0.3367) loss_classifier: 0.0949 (0.1124) loss_box_reg: 0.1797 (0.1764) loss_objectness: 0.0107 (0.0185) loss_rpn_box_reg: 0.0137 (0.0295) time: 0.6298 data: 0.0066 max mem: 4722 Epoch: [2] [249/250] eta: 0:00:00 lr: 0.005000 loss: 0.3654 (0.3391) loss_classifier: 0.1281 (0.1133) loss_box_reg: 0.2087 (0.1779) loss_objectness: 0.0129 (0.0186) loss_rpn_box_reg: 0.0137 (0.0292) time: 0.5967 data: 0.0065 max mem: 4722 Epoch: [2] Total time: 0:02:34 (0.6194 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:13 model_time: 0.1681 (0.1681) evaluator_time: 0.0099 (0.0099) time: 0.3561 data: 0.1761 max mem: 4722 Test: [100/207] eta: 0:00:13 model_time: 0.1202 (0.1168) evaluator_time: 0.0027 (0.0050) time: 0.1313 data: 0.0049 max mem: 4722 Test: [200/207] eta: 0:00:00 model_time: 0.1133 (0.1170) evaluator_time: 0.0028 (0.0050) time: 0.1222 data: 0.0033 max mem: 4722 Test: [206/207] eta: 0:00:00 model_time: 0.1137 (0.1170) evaluator_time: 0.0036 (0.0052) time: 0.1265 data: 0.0035 max mem: 4722 Test: Total time: 0:00:26 (0.1288 s / it) Averaged stats: model_time: 0.1137 (0.1170) evaluator_time: 0.0036 (0.0052) Accumulating evaluation results... DONE (t=0.09s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.395 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.651 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.424 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.296 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.454 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.588 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.173 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.476 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.544 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.454 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.600 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.703
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Epoch: [3] [ 0/250] eta: 0:03:42 lr: 0.000500 loss: 0.2147 (0.2147) loss_classifier: 0.0755 (0.0755) loss_box_reg: 0.1179 (0.1179) loss_objectness: 0.0143 (0.0143) loss_rpn_box_reg: 0.0070 (0.0070) time: 0.8897 data: 0.2764 max mem: 4722 Epoch: [3] [ 10/250] eta: 0:02:37 lr: 0.000500 loss: 0.2830 (0.3095) loss_classifier: 0.1161 (0.1141) loss_box_reg: 0.1569 (0.1598) loss_objectness: 0.0143 (0.0162) loss_rpn_box_reg: 0.0080 (0.0194) time: 0.6581 data: 0.0305 max mem: 4722 Epoch: [3] [ 20/250] eta: 0:02:23 lr: 0.000500 loss: 0.2830 (0.3167) loss_classifier: 0.1070 (0.1103) loss_box_reg: 0.1588 (0.1722) loss_objectness: 0.0136 (0.0172) loss_rpn_box_reg: 0.0091 (0.0170) time: 0.6128 data: 0.0061 max mem: 4722 Epoch: [3] [ 30/250] eta: 0:02:15 lr: 0.000500 loss: 0.2080 (0.2815) loss_classifier: 0.0735 (0.0976) loss_box_reg: 0.1270 (0.1549) loss_objectness: 0.0091 (0.0153) loss_rpn_box_reg: 0.0081 (0.0137) time: 0.5902 data: 0.0065 max mem: 4722 Epoch: [3] [ 40/250] eta: 0:02:10 lr: 0.000500 loss: 0.1987 (0.2955) loss_classifier: 0.0735 (0.0988) loss_box_reg: 0.1267 (0.1667) loss_objectness: 0.0070 (0.0137) loss_rpn_box_reg: 0.0106 (0.0163) time: 0.6134 data: 0.0066 max mem: 4722 Epoch: [3] [ 50/250] eta: 0:02:04 lr: 0.000500 loss: 0.2251 (0.2821) loss_classifier: 0.0764 (0.0932) loss_box_reg: 0.1267 (0.1589) loss_objectness: 0.0055 (0.0126) loss_rpn_box_reg: 0.0151 (0.0174) time: 0.6356 data: 0.0066 max mem: 4722 Epoch: [3] [ 60/250] eta: 0:01:57 lr: 0.000500 loss: 0.1786 (0.2780) loss_classifier: 0.0556 (0.0907) loss_box_reg: 0.0982 (0.1566) loss_objectness: 0.0079 (0.0127) loss_rpn_box_reg: 0.0123 (0.0180) time: 0.6201 data: 0.0067 max mem: 4722 Epoch: [3] [ 70/250] eta: 0:01:51 lr: 0.000500 loss: 0.2088 (0.2767) loss_classifier: 0.0584 (0.0914) loss_box_reg: 0.0982 (0.1552) loss_objectness: 0.0088 (0.0123) loss_rpn_box_reg: 0.0123 (0.0178) time: 0.6088 data: 0.0072 max mem: 4722 Epoch: [3] [ 80/250] eta: 0:01:45 lr: 0.000500 loss: 0.2329 (0.2771) loss_classifier: 0.0765 (0.0907) loss_box_reg: 0.1354 (0.1557) loss_objectness: 0.0077 (0.0118) loss_rpn_box_reg: 0.0201 (0.0190) time: 0.6281 data: 0.0070 max mem: 4722 Epoch: [3] [ 90/250] eta: 0:01:39 lr: 0.000500 loss: 0.2785 (0.2762) loss_classifier: 0.0871 (0.0901) loss_box_reg: 0.1580 (0.1558) loss_objectness: 0.0066 (0.0113) loss_rpn_box_reg: 0.0120 (0.0189) time: 0.6214 data: 0.0066 max mem: 4722 Epoch: [3] [100/250] eta: 0:01:32 lr: 0.000500 loss: 0.2862 (0.2802) loss_classifier: 0.0974 (0.0914) loss_box_reg: 0.1737 (0.1580) loss_objectness: 0.0055 (0.0112) loss_rpn_box_reg: 0.0066 (0.0196) time: 0.6024 data: 0.0066 max mem: 4722 Epoch: [3] [110/250] eta: 0:01:26 lr: 0.000500 loss: 0.2487 (0.2801) loss_classifier: 0.1053 (0.0922) loss_box_reg: 0.1395 (0.1579) loss_objectness: 0.0071 (0.0111) loss_rpn_box_reg: 0.0089 (0.0189) time: 0.6126 data: 0.0066 max mem: 4722 Epoch: [3] [120/250] eta: 0:01:19 lr: 0.000500 loss: 0.2201 (0.2756) loss_classifier: 0.0611 (0.0902) loss_box_reg: 0.1097 (0.1553) loss_objectness: 0.0080 (0.0108) loss_rpn_box_reg: 0.0080 (0.0192) time: 0.5932 data: 0.0066 max mem: 4722 Epoch: [3] [130/250] eta: 0:01:13 lr: 0.000500 loss: 0.2385 (0.2720) loss_classifier: 0.0609 (0.0889) loss_box_reg: 0.1028 (0.1537) loss_objectness: 0.0081 (0.0108) loss_rpn_box_reg: 0.0080 (0.0187) time: 0.5834 data: 0.0065 max mem: 4722 Epoch: [3] [140/250] eta: 0:01:07 lr: 0.000500 loss: 0.2385 (0.2705) loss_classifier: 0.0700 (0.0880) loss_box_reg: 0.1259 (0.1524) loss_objectness: 0.0061 (0.0105) loss_rpn_box_reg: 0.0129 (0.0195) time: 0.5881 data: 0.0067 max mem: 4722 Epoch: [3] [150/250] eta: 0:01:00 lr: 0.000500 loss: 0.2545 (0.2780) loss_classifier: 0.0898 (0.0902) loss_box_reg: 0.1450 (0.1560) loss_objectness: 0.0064 (0.0107) loss_rpn_box_reg: 0.0090 (0.0211) time: 0.5857 data: 0.0067 max mem: 4722 Epoch: [3] [160/250] eta: 0:00:55 lr: 0.000500 loss: 0.3263 (0.2809) loss_classifier: 0.1172 (0.0913) loss_box_reg: 0.1450 (0.1559) loss_objectness: 0.0093 (0.0111) loss_rpn_box_reg: 0.0086 (0.0225) time: 0.6324 data: 0.0065 max mem: 4722 Epoch: [3] [170/250] eta: 0:00:49 lr: 0.000500 loss: 0.1752 (0.2739) loss_classifier: 0.0567 (0.0891) loss_box_reg: 0.0837 (0.1515) loss_objectness: 0.0068 (0.0108) loss_rpn_box_reg: 0.0100 (0.0225) time: 0.6591 data: 0.0066 max mem: 4722 Epoch: [3] [180/250] eta: 0:00:43 lr: 0.000500 loss: 0.1413 (0.2706) loss_classifier: 0.0515 (0.0879) loss_box_reg: 0.0699 (0.1493) loss_objectness: 0.0047 (0.0106) loss_rpn_box_reg: 0.0088 (0.0227) time: 0.6477 data: 0.0065 max mem: 4722 Epoch: [3] [190/250] eta: 0:00:37 lr: 0.000500 loss: 0.1904 (0.2686) loss_classifier: 0.0618 (0.0874) loss_box_reg: 0.1134 (0.1488) loss_objectness: 0.0046 (0.0105) loss_rpn_box_reg: 0.0048 (0.0219) time: 0.6486 data: 0.0065 max mem: 4722 Epoch: [3] [200/250] eta: 0:00:31 lr: 0.000500 loss: 0.2096 (0.2679) loss_classifier: 0.0618 (0.0869) loss_box_reg: 0.1405 (0.1486) loss_objectness: 0.0046 (0.0105) loss_rpn_box_reg: 0.0062 (0.0220) time: 0.6733 data: 0.0067 max mem: 4722 Epoch: [3] [210/250] eta: 0:00:24 lr: 0.000500 loss: 0.2528 (0.2669) loss_classifier: 0.0830 (0.0864) loss_box_reg: 0.1412 (0.1479) loss_objectness: 0.0059 (0.0104) loss_rpn_box_reg: 0.0099 (0.0222) time: 0.6832 data: 0.0068 max mem: 4722 Epoch: [3] [220/250] eta: 0:00:18 lr: 0.000500 loss: 0.2576 (0.2698) loss_classifier: 0.0855 (0.0871) loss_box_reg: 0.1504 (0.1504) loss_objectness: 0.0076 (0.0104) loss_rpn_box_reg: 0.0099 (0.0220) time: 0.6381 data: 0.0068 max mem: 4722 Epoch: [3] [230/250] eta: 0:00:12 lr: 0.000500 loss: 0.2824 (0.2752) loss_classifier: 0.0971 (0.0889) loss_box_reg: 0.1677 (0.1526) loss_objectness: 0.0075 (0.0104) loss_rpn_box_reg: 0.0176 (0.0233) time: 0.5935 data: 0.0068 max mem: 4722 Epoch: [3] [240/250] eta: 0:00:06 lr: 0.000500 loss: 0.2824 (0.2744) loss_classifier: 0.0890 (0.0886) loss_box_reg: 0.1466 (0.1524) loss_objectness: 0.0096 (0.0104) loss_rpn_box_reg: 0.0176 (0.0230) time: 0.5977 data: 0.0072 max mem: 4722 Epoch: [3] [249/250] eta: 0:00:00 lr: 0.000500 loss: 0.2845 (0.2753) loss_classifier: 0.0816 (0.0888) loss_box_reg: 0.1415 (0.1522) loss_objectness: 0.0104 (0.0105) loss_rpn_box_reg: 0.0116 (0.0238) time: 0.6015 data: 0.0070 max mem: 4722 Epoch: [3] Total time: 0:02:35 (0.6210 s / it) creating index... index created!
/usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only " /usr/local/lib/python3.7/dist-packages/pymongo/topology.py:162: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe "MongoClient opened before fork. Create MongoClient only "
Test: [ 0/207] eta: 0:01:11 model_time: 0.1685 (0.1685) evaluator_time: 0.0073 (0.0073) time: 0.3448 data: 0.1672 max mem: 4722 Test: [100/207] eta: 0:00:13 model_time: 0.1207 (0.1173) evaluator_time: 0.0019 (0.0033) time: 0.1277 data: 0.0034 max mem: 4722 Test: [200/207] eta: 0:00:00 model_time: 0.1126 (0.1174) evaluator_time: 0.0018 (0.0033) time: 0.1206 data: 0.0035 max mem: 4722 Test: [206/207] eta: 0:00:00 model_time: 0.1136 (0.1174) evaluator_time: 0.0026 (0.0035) time: 0.1245 data: 0.0035 max mem: 4722 Test: Total time: 0:00:26 (0.1267 s / it) Averaged stats: model_time: 0.1136 (0.1174) evaluator_time: 0.0026 (0.0035) Accumulating evaluation results... DONE (t=0.05s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.428 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.688 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.459 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.315 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.507 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.615 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.182 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.502 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.544 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.441 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.614 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.713
add_detections(vehicle_model, torch_map_dataset_test, test_map_view, field_name="vehicle_predictions")
Using device cuda 100% |█████████████████| 207/207 [34.8s elapsed, 0s remaining, 5.5 samples/s]
vehicle_results = fo.evaluate_detections(
test_map_view,
"vehicle_predictions",
classes=["vehicle"],
eval_key="vehicle_eval",
compute_mAP=True
)
Evaluating detections... 100% |█████████████████| 207/207 [5.4s elapsed, 0s remaining, 36.1 samples/s] Performing IoU sweep... 100% |█████████████████| 207/207 [10.0s elapsed, 0s remaining, 15.6 samples/s]
vehicle_results.mAP()
0.43066431144550577
vehicle_results.print_report()
precision recall f1-score support vehicle 0.18 0.89 0.30 791 micro avg 0.18 0.89 0.30 791 macro avg 0.18 0.89 0.30 791 weighted avg 0.18 0.89 0.30 791
Due to our ability to easily visualize and manage our dataset with FiftyOne, we were able to spot and take action on a dataset issue that would otherwise have gone unnoticed if we only concerned ourselves with dataset-wide evaluation metrics and fixed dataset representations. Through these efforts, we managed to increase the mAP of the model to 43%.
Even though this example workflow may not work in all situations, this kind of class-merging strategy can be effective in cases where more fine-grained discrimination is not called for.
PyTorch and related frameworks provide quick and easy methods to bootstrap your model development and training pipelines. However, they largely overlook the need to massage and finetune datasets to efficiently improve performance. FiftyOne makes it easy to load your datasets into a flexible format that works well with existing tools allowing you to provide better data for training and testing. As they say, "garbage in, garbage out".