Template Matching using Image Processing
(and using scikit-image)
Template matching is a technique for finding small parts of an image that is highly similar to a template or reference image. This can also be used in finding the location of a template image in a larger image.
Basically, we slide the template image over the input image similar to how we used our spatial and morphological images and compare the template with the captured patch of the original image using a similarity score/ measure.
Template matching is much more explored in OpenCV but luckily we have a simple albeit a bit bare-bones implementation in scikit-image.
First, we take an image and identify a template within the image (but cutting a patch)
We then use a built-in function in scikit-image to perform the convolutions. We generate an embossed version of the original image with some bright dots peeking. The new image corresponds to the “similarity” score between the pixels of the original image and the template image and the bright dots are the pixel locations where the template scores the highest.
from skimage.feature import match_template
fig, ax = plt.subplots(1,1,figsize=(20, 10))
result = match_template(carrier_gray, template)
ax.imshow(result, cmap='viridis');
We can then locate these bright spots using another scikit-image function. We first set a threshold and pass the result through the function
from skimage.feature import peak_local_max
fig, ax = plt.subplots(1,1,figsize=(20, 10))
ax.imshow(carrier_gray)
template_width, template_height = template.shape
for x, y in peak_local_max(result, threshold_abs=0.5):
rect = plt.Rectangle((y, x), template_height, template_width, color='y',
fc='none')
plt.gca().add_patch(rect);
As you can see in the image above, we were able to draw a bounding box on the regions (or planes) in the image that is above the threshold we specified successfully detecting 6 planes matching our template.
However, not all planes that are like the template were detected. First, this is a function of the threshold we chose. If we set the threshold to be 0.40 then we will be able to detect planes that are not the same model as the template, so we settled with 0.50.
Aside from that, there are also planes that are oriented differently that were not detected. Since we are using a template, the algorithm assumes that the image elements follow the same orientation and scale as that of the template. So if the image is oriented differently, smaller or larger, or even has a different pixel intensity then the algorithm may not detect them as highly similar.
This is where preprocessing techniques like histogram equalization, Fourier transform, and white balancing can play a huge part as we need to ensure that the desired regions in the original image will yield a high similarity score if convoluted with the template chosen.
Wrapping Up
Here we present a short and simple object localization solution called Template Matching. However, it is not without any catches. Template matching is scale, intensity, and orientation sensitive. To earn a high enough similarity score, the template must really resemble the object being detected. There are more state-of-the-art algorithms available in OpenCV but unfortunately, they are protected by patents and copyright.
This ends the articles series that I prepared for Image Processing with Python 101. In the next series, we will extend our understanding of Image Processing to Computer Vision tasks.
