Zadání 2018 - Najdi chlupa

Cílem je nalezení zvířete v krátkém videu z fotopasti. Videa jsou k dispozici u cvičících a obsahují statickou scénu, která se pro danou lokalitu nemění. Mohou obsahovat záběry z různých fází dne. Do scény zpravidla vstoupí lesní tvor, občas se však nahrávání spustí bez zjevné příčiny a video je tvora-prosté.

Úkol 1 - anotace

Pro každé objevyvší se chlupaté stvoření určete ručně v každém snímku bounding-box. Tato data budou použita pro vyhodnocení přesnosti lokalizace výsledného algoritmu. Výstup anotace nechť je uložen v souboru ve formátu YAML a načtení ze souboru pomocí balíku ruamel.yaml ať má načtená struktura následující tvar:

In [1]:
anotace_file1 = {
    "path": "cesta_k_souboru/video18.avi",
    "team": ["Miroslav Jirik", "Ivan Gruber"],
    "frames": {
        27: [
            { 
                "x1": 41,
                "x2": 62,
                "y1": 187,
                "y2": 189,
            }
        ]
    }
}

anotace_file2 = {
    "path": "cesta_k_souboru/video21.avi",
    "team": ["Miroslav Jirik", "Ivan Gruber"],
    "note": "",
    "frames": {
        15: [
            { 
                "x1": 14,
                "x2": 26,
                "y1": 78,
                "y2": 98,
            }

        ],
        16: [
            { 
                "x1": 141,
                "x2": 267,
                "y1": 198,
                "y2": 325,
            },
            { 
                "x1": 141,
                "x2": 267,
                "y1": 198,
                "y2": 325,
            }

        ]

    }
}

Práce s YAMLem

In [8]:
from io import open
from ruamel.yaml import YAML
yaml = YAML(typ="unsafe")
filename = "anotace.yaml"

with open(filename, 'wt', encoding="utf-8") as f:
    yaml.dump(anotace_file2, f)
In [9]:
yaml = YAML(typ="unsafe")
with open(filename, encoding="utf-8") as f:
    obj = yaml.load(f)
In [19]:
print(obj["path"])
cesta_k_souboru/video21.avi
In [17]:
print(obj["frames"][16])

print("souradnice x1 prvniho boundingboxu ve framu 16: ", obj["frames"][16][0]["x1"])
[{'x1': 141, 'x2': 267, 'y1': 198, 'y2': 325}, {'x1': 141, 'x2': 267, 'y1': 198, 'y2': 325}]
souradnice x1 prvniho boundingboxu ve framu 16:  141

Práce s videem

In [4]:
# Toto je důležité pro spuštění externího editoru sed3 i ginput
% matplotlib qt

# conda install -c conda-forge -c mjirik imageio ffmpeg jupyter python=3.6 scikit-image
# conda create -n animalwatch -c conda-forge -c mjirik imageio ffmpeg jupyter python=3.6 scikit-image jupyter jupyter_contrib_nbextensions

import glob
import os.path as op
from pprint import pprint
import imageio
import skimage.color
import matplotlib.pyplot as plt
import numpy as np

plt.ioff()

datapath = "e:/data/lynx_lynx/fotopasti_20170825/videa/**/**/**"
fnvideos = glob.glob(op.join(datapath, "*.AVI"))

pprint(fnvideos[:10])
['e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0008.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0010.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0012.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0014.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0020.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0024.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0026.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0028.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0030.AVI',
 'e:/data/lynx_lynx/fotopasti_20170825/videa\\bez '
 'rysa\\lok3\\2016_04_23\\IMAG0032.AVI']
In [5]:
allframes = None
In [30]:
# ivideo = 10
ivideo = 1
step = 5
fn = fnvideos[ivideo]
print(fn)
vid = imageio.get_reader(fn)
frame_size = vid.get_meta_data()["size"]
frame_number = vid.get_length()
print(frame_number)
print(frame_size)


allframes = np.zeros([int(frame_number / step), int(frame_size[1]), int(frame_size[0]), 3], dtype=np.uint8)

for num in range(0, frame_number, step):
    image = vid.get_data(num)
#     gray = skimage.color.rgb2gray(image)
    allframes[int(num/step), :, :, :] = image
e:/data/lynx_lynx/fotopasti_20170825/videa\bez rysa\lok3\2016_04_23\IMAG0010.AVI
900
(1280, 720)
In [10]:
# plt.figure()
In [26]:
allframes.shape
Out[26]:
(180, 720, 1280, 3)

Interaktivita

In [33]:
frame_number = 10
plt.imshow(allframes[frame_number,...])# , cmap="gray")
# left mouse button - add point
# right mouse button - remove last point
# middle mouse button - finish
plt.ginput(-1)
Out[33]:
[(4.661290322580641, 4.97096774193551),
 (564.6612903225807, 709.4870967741934),
 (549.1774193548388, 291.42258064516125),
 (693.6935483870968, 394.6483870967742)]
In [ ]:
def get_iou(bb1, bb2):
    """
    Calculate the Intersection over Union (IoU) of two bounding boxes.
    
    by Martin Thoma

    Parameters
    ----------
    bb1 : dict
        Keys: {'x1', 'x2', 'y1', 'y2'}
        The (x1, y1) position is at the top left corner,
        the (x2, y2) position is at the bottom right corner
    bb2 : dict
        Keys: {'x1', 'x2', 'y1', 'y2'}
        The (x, y) position is at the top left corner,
        the (x2, y2) position is at the bottom right corner

    Returns
    -------
    float
        in [0, 1]
    """
    assert bb1['x1'] < bb1['x2']
    assert bb1['y1'] < bb1['y2']
    assert bb2['x1'] < bb2['x2']
    assert bb2['y1'] < bb2['y2']

    # determine the coordinates of the intersection rectangle
    x_left = max(bb1['x1'], bb2['x1'])
    y_top = max(bb1['y1'], bb2['y1'])
    x_right = min(bb1['x2'], bb2['x2'])
    y_bottom = min(bb1['y2'], bb2['y2'])

    if x_right < x_left or y_bottom < y_top:
        return 0.0

    # The intersection of two axis-aligned bounding boxes is always an
    # axis-aligned bounding box
    intersection_area = (x_right - x_left) * (y_bottom - y_top)

    # compute the area of both AABBs
    bb1_area = (bb1['x2'] - bb1['x1']) * (bb1['y2'] - bb1['y1'])
    bb2_area = (bb2['x2'] - bb2['x1']) * (bb2['y2'] - bb2['y1'])

    # compute the intersection over union by taking the intersection
    # area and dividing it by the sum of prediction + ground-truth
    # areas - the interesection area
    iou = intersection_area / float(bb1_area + bb2_area - intersection_area)
    assert iou >= 0.0
    assert iou <= 1.0
    return iou

Očekávaná struktura modulu

In [9]:
def najdi_chlupa(cesta_k_videu, timestamp, lokalita, jmeno_yaml_souboru):
    """
    :param datestamp: "rrrr-mm-dd-hh:mm"
    """
    pass
toml
[setup]
sdfasl
asdfa