pip install --upgrade git+https://github.com/rbrtwlz/fastai_object_detection
You can find a detailed documentation with examples here.
This package makes object detection and instance segmentation models available for fastai users by using a callback which converts the batches to the required input.
It comes with a fastai DataLoaders class for object detection, prepared and easy to use models and
some metrics to measure generated bounding boxes (mAP). So you can train a model for object detection
in the simple fastai way with one of the included Learner classes.
All you need is a pandas DataFrame containing the data for each object in the images. In default setting follwing columns are required:
For the image, which contains the object(s):
image_idimage_path
The object's bounding box:
x_miny_minx_maxy_max
The object's class/label:
class_name
If you want to use a model for instance segementation, following columns are additionally required:
mask_path(path to the binary mask, which represents the object in the image)
There are helper functions available, for example for adding the image_path by image_id or to change the bbox format from xywh to x1y1x2y2.
Futhermore there is a CocoData class provided to help you to download images from Microsoft COCO dataset, create the corresponding masks and generate a DataFrame.
Microsoft COCO dataset contains 328,000 annotated images of 91 object categories, so you can pick the categories you want and download just associated images.
Simply use the following line for example to create a dataset for cat and dog detection:
from fastai.vision.all import *
from fastai_object_detection.all import *
path, df = CocoData.create(ds_name="coco-cats-and-dogs", cat_list=["cat", "dog"],
max_images=2000, with_mask=False)
Then you can build DataLoaders, using it's from_df factory method.
dls = ObjectDetectionDataLoaders.from_df(df, bs=2,
item_tfms=[Resize(800, method="pad", pad_mode="zeros")],
batch_tfms=[Normalize.from_stats(*imagenet_stats)])
dls.show_batch()
Now you are ready to create your fasterrcnn_learner to train a FasterRCNN model (with resnet50 backbone). To validate your models predictions you can use metrics like mAP_at_IoU60.
learn = fasterrcnn_learner(dls, fasterrcnn_resnet50,
opt_func=SGD, lr=0.005, wd=0.0005, train_bn=False,
metrics=[mAP_at_IoU40, mAP_at_IoU60])
learn.lr_find()
learn.fit_one_cycle(10, 1e-04)
First import the libraries.
from fastai.vision.all import *
from fastai_object_detection.all import *
Then you can donwload images of the categories you want to detect. If you want to train a instance segmentation model use with_mask=True.
path, df = CocoData.create(ds_name="ds-cats-dogs", cat_list=["cat", "dog"], max_images=500)
After the images were downloaded, you can create DataLoaders with the from_df factory method and show some batches. If the column mask_path is present in your DataFrame, it creates a DataLoader for instance segmentation (images, bounding boxes, labels and masks) otherwise for object detection (images, bounding boxes and labels)
dls = ObjectDetectionDataLoaders.from_df(df, bs=2,
item_tfms=[Resize(800, method="pad", pad_mode="zeros")],
batch_tfms=[Normalize.from_stats(*imagenet_stats)])
dls.show_batch(figsize=(10,10))
Then you can choose which architectur you want to use.
Create a learner and pass a model like fasterrcnn_resnet50 together with dls.
In my experiments it was easier to train using SGD as optimizer rather then Adam. Finally you need metrics to measure the predictions of your model. For bounding boxes the metric "mean average precision" at different IoUs (Intersection over Union) is common.
learn = fasterrcnn_learner(dls, fasterrcnn_resnet50,
opt_func=SGD, lr=0.005, wd=0.0005, train_bn=False,
metrics=[mAP_at_IoU40, mAP_at_IoU60])
learn.freeze()
After freezing the Learner you can search for a learning rate using fastai's LRFinder.
learn.lr_find()
learn.fit_one_cycle(3, 1.2e-03)
After a couple of epochs you can unfreeze the Learner and train the whole model for some extra epochs.
learn.unfreeze()
learn.fit_one_cycle(3, 1.2e-03)