%matplotlib inline
from fastai.gen_doc.nbdoc import *
from fastai import *
from fastai.vision import *
This module contains the classes that define datasets handling Image
objects and their tranformations. As usual, we'll start with a quick overview, before we get in to the detailed API docs.
To get you started as easily as possible, the fastai provides two helper functions to create a DataBunch
object that you can directly use for training a classifier. To demonstrate them you'll first need to download and untar the file by executing the following cell. This will create a data folder containing an MNIST subset in data/mnist_sample
.
path = untar_data(URLs.MNIST_SAMPLE); path
PosixPath('/home/jhoward/.fastai/data/mnist_sample')
There are a number of ways to create an ImageDataBunch
. One common approach is to use Imagenet-style folders (see a ways down the page below for details) with ImageDataBunch.from_folder
:
tfms = get_transforms(do_flip=False)
data = ImageDataBunch.from_folder(path, ds_tfms=tfms, size=24)
Here the datasets will be automatically created in the structure of Imagenet-style folders. The parameters specified:
ds_tfms
(here with do_flip
=False because we don't want to flip numbers),size
of our pictures (here 24).As with all DataBunch
usage, a train_dl
and a valid_dl
are created that are of the type PyTorch DataLoader
.
If you want to have a look at a few images inside a batch, you can use ImageDataBunch.show_batch
. The rows
argument is the number of rows and columns to display.
data.show_batch(rows=3, figsize=(5,5))
The second way to define the data for a classifier requires a structure like this:
path\
train\
test\
labels.csv
where the labels.csv file defines the label(s) of each image in the training set. This is the format you will need to use when each image can have multiple labels. It also works with single labels:
pd.read_csv(path/'labels.csv').head()
name | label | |
---|---|---|
0 | train/3/7463.png | 0 |
1 | train/3/21102.png | 0 |
2 | train/3/31559.png | 0 |
3 | train/3/46882.png | 0 |
4 | train/3/26209.png | 0 |
You can then use ImageDataBunch.from_csv
:
data = ImageDataBunch.from_csv(path, ds_tfms=tfms, size=28)
data.show_batch(rows=3, figsize=(5,5))
An example of multiclassification can be downloaded with the following cell. It's a sample of the planet dataset.
planet = untar_data(URLs.PLANET_SAMPLE)
If we open the labels files, we seach that each image has one or more tags, separated by a space.
df =pd.read_csv(planet/'labels.csv')
df.head()
image_name | tags | |
---|---|---|
0 | train_21983 | partly_cloudy primary |
1 | train_9516 | clear cultivation primary water |
2 | train_12664 | haze primary |
3 | train_36960 | clear primary |
4 | train_5302 | haze primary road |
data = ImageDataBunch.from_csv(planet, folder='train', size=128, suffix='.jpg',
ds_tfms=get_transforms(flip_vert=True, max_lighting=0.1, max_zoom=1.05, max_warp=0.))
The show_batch
method will then print all the labels that correspond to each image.
data.show_batch(rows=3, figsize=(10,8), ds_type=DatasetType.Valid)
You can find more ways to build an ImageDataBunch
without the factory methods in data_block
.
show_doc(ImageDataBunch, doc_string=False)
class
ImageDataBunch
[source]
ImageDataBunch
(train_dl
:DataLoader
,valid_dl
:DataLoader
,test_dl
:Optional
[DataLoader
]=None
,device
:device
=None
,tfms
:Optional
[Collection
[Callable
]]=None
,path
:PathOrStr
='.'
,collate_fn
:Callable
='data_collate'
) ::DataBunch
show_doc(ImageDataBunch.create, arg_comments={
'bs': 'Desired batchsize for the dataloaders',
'num_workers': 'The number of process to launch for data collection',
'ds_tfms': 'Tuple of two lists of transforms (first for training and second for validation and test set)',
'size': 'Target size for those transforms',
'tfms': 'List of transforms to be applied at a batch level (like normalization)',
'device': 'The device on which to put the batches'
})
create
[source]
create
(train_ds
,valid_ds
,test_ds
=None
,path
:PathOrStr
='.'
,bs
:int
=64
,ds_tfms
:Union
[Transform
,Collection
[Transform
],NoneType
]=None
,num_workers
:int
=16
,tfms
:Optional
[Collection
[Callable
]]=None
,device
:device
=None
,collate_fn
:Callable
='data_collate'
,size
:int
=None
,kwargs
) →ImageDataBunch
Factory method. bs
batch size, ds_tfms
for Dataset
, tfms
for DataLoader
.
You don't often need to call this directly yourself; instead, you'll normally use one of the convenience wrappers below. However, these wrappers all accept a kwargs
that is passed to this method, so you can pass any of the above parameters as well.
If you quickly want to get a ImageDataBunch
and train a model, you should process your data to have it in one of the formats the following functions handle.
show_doc(ImageDataBunch.from_folder)
from_folder
[source]
from_folder
(path
:PathOrStr
,train
:PathOrStr
='train'
,valid
:PathOrStr
='valid'
,test
:Union
[Path
,str
,NoneType
]=None
,valid_pct
=None
,kwargs
:Any
) →ImageDataBunch
Create from imagenet style dataset in path
with train
,valid
,test
subfolders (or provide valid_pct
).
"Imagenet-style" datasets look something like this (note that the test folder is optional):
path\
train\
clas1\
clas2\
...
valid\
clas1\
clas2\
...
test\
For example:
data = ImageDataBunch.from_folder(path, ds_tfms=tfms, size=24)
Note that this (and all factory methods in this section) pass any kwargs
to ImageDataBunch.create
.
show_doc(ImageDataBunch.from_csv)
from_csv
[source]
from_csv
(path
:PathOrStr
,folder
:PathOrStr
='.'
,sep
=None
,csv_labels
:PathOrStr
='labels.csv'
,valid_pct
:float
=0.2
,fn_col
:int
=0
,label_col
:int
=1
,test
:Union
[Path
,str
,NoneType
]=None
,suffix
:str
=None
,header
:Union
[int
,str
,NoneType
]='infer'
,kwargs
:Any
) →ImageDataBunch
Create from a csv file.
Create ImageDataBunch
from path
by splitting the data in folder
and labelled in a file csv_labels
between a training and validation set. Use valid_pct
to indicate the percentage of the total images for the validation set. An optional test
folder contains unlabelled data and suffix
contains an optional suffix to add to the filenames in csv_labels
(such as '.jpg').
For example:
data = ImageDataBunch.from_csv(path, ds_tfms=tfms, size=24);
show_doc(ImageDataBunch.from_df)
from_df
[source]
from_df
(path
:PathOrStr
,df
:DataFrame
,folder
:PathOrStr
='.'
,sep
=None
,valid_pct
:float
=0.2
,fn_col
:int
=0
,label_col
:int
=1
,test
:Union
[Path
,str
,NoneType
]=None
,suffix
:str
=None
,kwargs
:Any
) →ImageDataBunch
Create from a DataFrame.
Same as ImageDataBunch.from_csv
, but passing in a DataFrame
instead of a csv file. E.gL
df = pd.read_csv(path/'labels.csv', header='infer')
df.head()
name | label | |
---|---|---|
0 | train/3/7463.png | 0 |
1 | train/3/21102.png | 0 |
2 | train/3/31559.png | 0 |
3 | train/3/46882.png | 0 |
4 | train/3/26209.png | 0 |
data = ImageDataBunch.from_df(path, df, ds_tfms=tfms, size=24)
Different datasets are labeled in many different ways. The following methods can help extract the labels from the dataset in a wide variety of situations. The way they are built in fastai is constructive: there are methods which do a lot for you but apply in specific circumstances and there are methods which do less for you but give you more flexibility.
In this case the hierachy is:
ImageDataBunch.from_name_re
: Gets the labels from the filenames using a regular expressionImageDataBunch.from_name_func
: Gets the labels from the filenames using any functionImageDataBunch.from_lists
: Labels need to be provided as an input in a listshow_doc(ImageDataBunch.from_name_re)
from_name_re
[source]
from_name_re
(path
:PathOrStr
,fnames
:FilePathList
,pat
:str
,valid_pct
:float
=0.2
,test
:str
=None
,kwargs
)
Creates an ImageDataBunch
from fnames
, calling a regular expression (containing one re group) on the file names to get the labels, putting aside valid_pct
for the validation. In the same way as ImageDataBunch.from_csv
, an optional test
folder contains unlabelled data.
Our previously created dataframe contains the labels in the filenames so we can leverage it to test this new method. ImageDataBunch.from_name_re
needs the exact path of each file so we will append the data path to each filename before creating our ImageDataBunch
object.
fn_paths = [path/name for name in df['name']]; fn_paths[:2]
[PosixPath('/home/ubuntu/.fastai/data/mnist_sample/train/3/7463.png'), PosixPath('/home/ubuntu/.fastai/data/mnist_sample/train/3/21102.png')]
pat = r"/(\d)/\d+\.png$"
data = ImageDataBunch.from_name_re(path, fn_paths, pat=pat, ds_tfms=tfms, size=24)
data.classes
['3', '7']
show_doc(ImageDataBunch.from_name_func)
from_name_func
[source]
from_name_func
(path
:PathOrStr
,fnames
:FilePathList
,label_func
:Callable
,valid_pct
:float
=0.2
,test
:str
=None
,kwargs
)
Works in the same way as ImageDataBunch.from_name_re
, but instead of a regular expression it expects a function that will determine how to extract the labels from the filenames. (Note that from_name_re
uses this function in its implementation).
To test it we could build a function with our previous regex. Let's try another, similar approach to show that the labels can be obtained in a different way.
def get_labels(file_path): return '3' if '/3/' in str(file_path) else '7'
data = ImageDataBunch.from_name_func(path, fn_paths, label_func=get_labels, ds_tfms=tfms, size=24)
data.classes
['3', '7']
show_doc(ImageDataBunch.from_lists)
from_lists
[source]
from_lists
(path
:PathOrStr
,fnames
:FilePathList
,labels
:StrList
,valid_pct
:float
=0.2
,test
:str
=None
,kwargs
)
The most flexible factory function; pass in a list of labels
that correspond to each of the filenames in fnames
.
To show an example we have to build the labels list outside our ImageDataBunch
object and give it as an argument when we call from_lists
. Let's use our previously created function to create our labels list.
labels_ls = list(map(get_labels, fn_paths))
data = ImageDataBunch.from_lists(path, fn_paths, labels=labels_ls, ds_tfms=tfms, size=24)
data.classes
['3', '7']
show_doc(ImageDataBunch.show_batch)
show_batch
[source]
show_batch
(rows
:int
=None
,figsize
:Tuple
[int
,int
]=(9, 10)
,ds_type
:DatasetType
=<DatasetType.Valid: 2>
)
Create a rows
by rows
grid of images from dataset ds_type
for a figsize
figure. This function works for all type of computer vision data (see data_block
for more examples).
Once you have your ImageDataBunch
, you can have a quick look at your data by using this:
data.show_batch(rows=3, figsize=(6,6))
show_doc(ImageDataBunch.labels_to_csv)
labels_to_csv
[source]
labels_to_csv
(dest
:str
)
Save file names and labels in data
as CSV to file name dest
.
This is a functional version of ImageDataBunch.show_batch
.
In the next two methods we will use a new dataset, CIFAR. This is because the second method will get the statistics for our dataset and we want to be able to show different statistics per channel. If we were to use MNIST, these statistics would be the same for every channel. White pixels are [255,255,255] and black pixels are [0,0,0] (or in normalized form [1,1,1] and [0,0,0]) so there is no variance between channels.
path = untar_data(URLs.CIFAR); path
PosixPath('/home/ubuntu/.fastai/data/cifar10')
show_doc(channel_view)
channel_view
[source]
channel_view
(x
:Tensor
) →Tensor
Make channel the first axis of x
and flatten remaining axes
data = ImageDataBunch.from_folder(path, ds_tfms=tfms, valid='test', size=24)
def channel_view(x:Tensor)->Tensor:
"Make channel the first axis of `x` and flatten remaining axes"
return x.transpose(0,1).contiguous().view(x.shape[1],-1)
This function takes a tensor and flattens all dimensions except the channels, which it keeps as the first axis. This function is used to feed ImageDataBunch.batch_stats
so that it can get the pixel statistics of a whole batch.
Let's take as an example the dimensions our MNIST batches: 128, 3, 24, 24.
t = torch.Tensor(128, 3, 24, 24)
t.size()
torch.Size([128, 3, 24, 24])
tensor = channel_view(t)
tensor.size()
torch.Size([3, 73728])
show_doc(ImageDataBunch.batch_stats)
batch_stats
[source]
batch_stats
(funcs
:Collection
[Callable
]=None
) →Tensor
Grab a batch of data and call reduction function func
per channel
Gets the statistics of each channel of a batch of data. If no functions are specified, default statistics are mean and standard deviation.
data.batch_stats()
[tensor([0.5047, 0.5339, 0.5599]), tensor([0.2412, 0.2319, 0.2493])]
show_doc(ImageDataBunch.normalize)
normalize
[source]
normalize
(stats
:Collection
[Tensor
]=None
)
Add normalize transform using stats
(defaults to DataBunch.batch_stats
)
Adds the normalize transform to the set of transforms associated with the data. In the fast.ai library we have imagenet_stats
, cifar_stats
and mnist_stats
so we can add normalization easily with any of these datasets. Let's see an example with our dataset of choice: MNIST.
data.normalize(cifar_stats)
data.batch_stats()
[tensor([0.0556, 0.2135, 0.4325]), tensor([0.9765, 0.9543, 0.9552])]
show_doc(show_image_batch, arg_comments={
'dl': 'A dataloader from which to show a sample',
'classes': 'List of classes (for the labels)',
'rows': 'Will make a square of `rows` by `rows` images',
'figsize': 'Size of the graph shown'
})
show_image_batch
[source]
show_image_batch
(dl
:DataLoader
,classes
:StrList
,rows
:int
=None
,figsize
:Tuple
[int
,int
]=(9, 10)
)
Show a few images from a batch.
rows
by rows
imagesThis is a functional version of ImageDataBunch.show_batch
.
You may also want to normalize your data, which can be done by using the following functions.
show_doc(normalize)
normalize
[source]
normalize
(x
:Tensor
,mean
:FloatTensor
,std
:FloatTensor
) →Tensor
Normalize x
with mean
and std
.
show_doc(denormalize)
denormalize
[source]
denormalize
(x
:Tensor
,mean
:FloatTensor
,std
:FloatTensor
) →Tensor
Denormalize x
with mean
and std
.
show_doc(normalize_funcs, doc_string=False)
normalize_funcs
[source]
normalize_funcs
(mean
:FloatTensor
,std
:FloatTensor
) →Tuple
[Callable
,Callable
]
Create normalize
and denormalize
functions using mean
and std
. device
will store them on the device specified. do_y
determines if the target should also be normaized or not.
On MNIST the mean and std are 0.1307 and 0.3081 respectively (looked on Google). If you're using a pretrained model, you'll need to use the normalization that was used to train the model. The imagenet norm and denorm functions are stored as constants inside the library named imagenet_norm
and imagenet_denorm
. If you're training a model on CIFAR-10, you can also use cifar_norm
and cifar_denorm
.
You may sometimes see warnings about clipping input data when plotting normalized data. That's because even although it's denormalized when plotting automatically, sometimes floating point errors may make some values slightly out or the correct range. You can safely ignore these warnings in this case.
data = ImageDataBunch.from_folder(untar_data(URLs.MNIST_SAMPLE),
ds_tfms=tfms, size=24)
data.normalize()
data.show_batch(rows=3, figsize=(6,6))
Depending on the task you are tackling, you'll need one of the following fastai datasets.
show_doc(ImageClassificationDataset, title_level=3)
class
ImageClassificationDataset
[source]
ImageClassificationDataset
(fns
:FilePathList
,labels
:StrList
,classes
:Optional
[ArgStar
]=None
) ::ImageClassificationBase
Dataset
for folders of images in style {folder}/{class}/{images}.
This is the basic dataset for image classification: fns
are the filenames of the images and labels
the corresponding labels. Optionally, classes
contains a name for each possible label.
show_doc(ImageClassificationDataset.from_folder)
from_folder
[source]
from_folder
(folder
:Path
,classes
:Optional
[ArgStar
]=None
,valid_pct
:float
=0.0
,extensions
:StrList
={'.jpg', '.jpe', '.jpeg', '.gif', '.ico', '.rgb', '.xbm', '.ppm', '.xwd', '.svg', '.xpm', '.pgm', '.png', '.bmp', '.tiff', '.ras', '.tif', '.pbm', '.ief', '.pnm'}
) →Union
[ImageClassificationDataset
,List
[ImageClassificationDataset
]]
Dataset of classes
labeled images in folder
. Optional valid_pct
split validation set.
Create an ImageClassificationDataset
automatically from a folder
. If classes
is None, it will be set to the names of the directories in folder
. check_ext
forces the function to only keep filenames with image extensions.
show_doc(ImageClassificationDataset.from_single_folder, doc_string=False)
from_single_folder
[source]
from_single_folder
(folder
:PathOrStr
,classes
:ArgStar
,extensions
:StrList
={'.jpg', '.jpe', '.jpeg', '.gif', '.ico', '.rgb', '.xbm', '.ppm', '.xwd', '.svg', '.xpm', '.pgm', '.png', '.bmp', '.tiff', '.ras', '.tif', '.pbm', '.ief', '.pnm'}
)
Typically used for define a test set. Label all images in folder
with classes[0]
. check_ext
forces the function to only keep filenems with image extensions.
show_doc(ImageMultiDataset, doc_string=False, title_level=3)
class
ImageMultiDataset
[source]
ImageMultiDataset
(fns
:FilePathList
,labels
:StrList
,classes
:Optional
[ArgStar
]=None
) ::ImageClassificationBase
This is the basic dataset for image classification with multiple labels: fns
are the filenames of the images and labels
the corresponding labels (may be more than one for each image). Optionally, classes
contains a name for each possible label.
show_doc(ImageMultiDataset.from_folder, doc_string=False)
from_folder
[source]
from_folder
(path
:PathOrStr
,folder
:PathOrStr
,fns
:Series
,labels
:StrList
,valid_pct
:float
=0.2
,classes
:Optional
[ArgStar
]=None
)
To create an ImageMultiDataset
automatically in path
from a folder
and fns
. If classes
is None, it will be set to the names of the different labels
seen. You can split the images in this folder
in a train/valid dataset if valid_pct
is non-zero. check_ext
forces the function to only keep filenems with image extensions.
show_doc(ImageMultiDataset.from_single_folder, doc_string=False)
from_single_folder
[source]
from_single_folder
(folder
:PathOrStr
,classes
:ArgStar
,extensions
={'.jpg', '.jpe', '.jpeg', '.gif', '.ico', '.rgb', '.xbm', '.ppm', '.xwd', '.svg', '.xpm', '.pgm', '.png', '.bmp', '.tiff', '.ras', '.tif', '.pbm', '.ief', '.pnm'}
)
Typically used for define a test set. Label all images in folder
with classes[0]
. check_ext
forces the function to only keep filenems with image extensions.
To help scan a folder for these Dataset
, we use the following helper function:
show_doc(get_image_files, doc_string=False)
get_image_files
[source]
get_image_files
(c
:PathOrStr
,check_ext
:bool
=True
,recurse
=False
) →FilePathList
Return list of files in c
that are images. check_ext
will filter to keep only the files with image extensions.
show_doc(ImageMultiDataset.get_labels)
get_labels
[source]
get_labels
(idx
:int
) →StrList
Gets the labels of a batch of images of choice. Pass in the batch number/index and will return the labels for each of the examples in that batch.
show_doc(ImageMultiDataset.encode)
show_doc(SegmentationDataset, doc_string=False, title_level=3)
class
SegmentationDataset
[source]
SegmentationDataset
(x
:FilePathList
,y
:FilePathList
,classes
:ArgStar
) ::ImageClassificationBase
This is the basic dataset for image sementation: x
contains the filenames of the images and y
the ones of the masks.
show_doc(ObjectDetectDataset, doc_string=False, title_level=3)
class
ObjectDetectDataset
[source]
ObjectDetectDataset
(x_fns
:FilePathList
,labelled_bbs
:Collection
[Tuple
[Collection
[int
],str
]],classes
:StrList
=None
) ::ImageClassificationBase
This is the basic dataset for object detection: x_fns
contains the filenames of the images, labelled_bbs
the corresponding bounding boxes with their corresponding labels. classes
contains the list of classes.
show_doc(ObjectDetectDataset.from_json)
from_json
[source]
from_json
(folder
,fname
,valid_pct
=None
,classes
=None
)
Create an ObjectDetectDataset
by looking at the images in folder
according to annotations in the json fname
. If valid_pct
is passed, split a training and validation set. classes
is the list of classes.
This factory method uses the following helper function.
show_doc(get_annotations)
get_annotations
[source]
get_annotations
(fname
,prefix
=None
)
Open a COCO style json in fname
and returns the lists of filenames (with maybe prefix
) and labelled bboxes.
show_doc(DatasetTfm, doc_string=False, title_level=3)
Dataset that applies the list of transforms tfms
to every item drawn. If tfms
should be applied to the targets as well, tfm_y
should be True. kwargs
will be passed to apply_tfms
internally.
Then this last function automatizes the process of creating DatasetTfm
:
show_doc(transform_datasets, doc_string=False)
Create train, valid and maybe test DatasetTfm from train_ds
, valid_ds
and maybe test_ds
using tfms
. It should be a tuple containing the transforms for the training set, then for the validation and test set.
show_doc(ImageClassificationBase, title_level=3, doc_string=False)
class
ImageClassificationBase
[source]
ImageClassificationBase
(fns
:FilePathList
,classes
:Optional
[ArgStar
]=None
) ::LabelDataset
Base class for computer vision datasets. Maps classes
to indexes via class2idx
.
The vision application adds a few methods to implement data augmentation in the data block API or create an ImageDataBunch
.
show_doc(SplitDatasetsImage, title_level=3)
class
SplitDatasetsImage
[source]
SplitDatasetsImage
(path
:PathOrStr
,train_ds
:Dataset
,valid_ds
:Dataset
,test_ds
:Optional
[Dataset
]=None
) ::SplitDatasets
A class regrouping train_ds
, a valid_ds
and maybe a train_ds
dataset, inside a path
.
show_doc(SplitDatasetsImage.transform)
transform
[source]
transform
(tfms
:Union
[Transform
,Collection
[Transform
]],kwargs
) →SplitDatasets
Apply tfms
to the underlying datasets, kwargs
are passed to DatasetTfm
.
show_doc(SplitDatasetsImage.databunch)
databunch
[source]
databunch
(path
:PathOrStr
=None
,kwargs
) →ImageDataBunch
Create an ImageDataBunch
from self, path
will override self.path
, kwargs
are passed to ImageDataBunch.create
.
To change the default extensions
in InputList.from_folder
to image extensions, we subclass it to the following:
show_doc(ImageFileList, title_level=3, doc_string=False)
show_doc(ImageFileList.from_folder)
from_folder
[source]
from_folder
(path
:PathOrStr
='.'
,extensions
:StrList
={'.jpg', '.jpe', '.jpeg', '.gif', '.ico', '.rgb', '.xbm', '.ppm', '.xwd', '.svg', '.xpm', '.pgm', '.png', '.bmp', '.tiff', '.ras', '.tif', '.pbm', '.ief', '.pnm'}
,recurse
=True
) →ImageFileList
Get the list of files in path
that have a suffix in extensions
. recurse
determines if we search subfolders.