Image Enhancements in Python using scikit-image
Understanding White Balancing, Histogram Equalization, and Fourier Transforms for Images
In my previous article, we enumerate the most common algorithms in image processing. In this article, we do a deep dive into Image Enhancements as a way to preprocess our images. Specifically, we will touch on White Balancing, Histogram Equalization, and Fourier Transforms.
Image enhancements, as the name suggests, improve the quality of our input image by correcting colors, correcting contrast, and removing noise or patterns that distort the photo.
White Balancing
White balancing allows us to restore discolorization in photos. We color-correct each image by making white or neutral-colored regions in the real world appear white in the digital image. Intuitively, color correcting the image requires us to have a reference region that we can call as true white — the logic behind the ground-truth algorithm. Overall, there are three common white balancing algorithms available to us:
Ground-truth Algorithm requires us to identify a region in the image which we can call as true white. In the code below, we use a patch image and scale pixel values of the current image with the mean/ max of the patch image. Since the ground-truth algorithm is sensitive to the choice of the patch, we have to be careful about what we use. In the photo grid below, we used different patches for the algorithm and it gave mixed results.
def apply_ground_truth(im, patch, method=’mean’, ax=None):
if method==’max’:
im_wb = (im*1.0/ patch.max(axis=(0, 1))).clip(0, 1)
if method==’mean’:
im_wb = (im*patch.mean()/ im.mean(axis=(0, 1))).clip(0, 255).astype(int)
ax.imshow(im_wb)
return im_wb
From the photo grid, we can hypothesize that only the collar is the true white in the image and that the background/ wall may not be a pure white image considering how choosing it as a true white patch resulted in an overexposure
White Patch Algorithm requires us to set the threshold at which we set the maximum of each of the color channels. In this algorithm, we assume that the true white occurs when each of the color channels (RGB, to be specific) is at its maximum (intensity=255). In the code below, we set a percentile, at which we set the maximum of each color channel of the image. In the succeeding photo, we looked at the effect of the percentile cut on the amount of white balancing in the image. We observe that setting it at p=95 resulted in a fairly acceptable image compared to p=90.
def apply_white_patch(im, p):
im_wb = (img_as_ubyte((im*1.0/np.percentile(im, p, axis=(0,1))).clip(0, 1)))
return im_wb
Gray world algorithm assumes that each pixel from each color channel has an equivalent/ counterpart pixel in the image. In this algorithm, we adjust each color channel so that all of them have the same mean. In the code below, we normalized the entire image such that each color channel will have the same mean. However, we observed that the gray world algorithm fails to correct the color. Instead, it resulted in an image with a bluish undertone.
def apply_gray_world(im):
return (im * (im.mean()/im.mean(axis=(0, 1)))).clip(0, 255).astype(int)
We observe that the gray world algorithm does not work well with monotone images (or any image with a dominant color like the sepia image). This is because the gray world algorithm assumes that the image has the same mean in any channel. Since the pixels for the overcast colors (here, red) will be in the higher range, we overexpose some pixels in the other color channels resulting in color overcasts. (In the image, we observe blue overexposure and a slight green overexposure in the region where the child in red is located).
Histogram Equalization
Sometimes, white balancing does not do the trick because the issue is not a decoloration issue but a contrast (or brightness) issue. In the dark image below, we see a true white region under the frame. Using any of the white balancing techniques above will not yield any significant result because a true white already exists in the image.
As such, we look at the histogram of the image. As expected, the histogram is heavily shifted to the left (low values) implying a very underexposed image.
Histogram Equalization basically tells us to evenly spread the histogram of the image across the entire range of the intensity value (from 0 to 255). A simple approach is to map each of pixel value of the image to the desired distribution. The code below shows us how this can be done
def apply_histogram_correction(im, target_dist, target_bins):
freq, bins = cumulative_distribution(im)
corrected = np.interp(freq, target_dist, target_bins)
return img_as_ubyte(corrected[im].astype(int))
image = rgb2gray(dark_image)
target_bins = np.arange(256)
target_dists = np.linspace(0, 1, len(target_bins))apply_histogram_correction(image, target_dist, target_bins)
We can apply this equalization to every color channel of the image giving us the result below
Fourier Transforms
Like what we discussed previously, images can be treated as a 3-D or a 2-D function F(x, y, z) where z refers to the color spaces. As such, we can interpret images as a superposition of waves or of wave patterns. This opens up the opportunity to use Fourier Transforms to remove any repetitive patterns in the image. For example, the image below contains a lot of lines that may be a result of the texture of the canvas the image was painted on. We can remove this by translating the image to the frequency domain and removing the signals that correspond to the texture/ lines.
Fourier transform can be done using numpy operations as follows:
image_lines_fft = np.fft.fftshift(np.fft.fft2(image_lines))If we do this per color channel, we will end up obtaining something like this
The “star” like elements in the “More” column represents the linear patterns in the image. Removing them simply requires us to mask them, only retaining the middle dot which represents the DC component of the image. Once corrected, we merge the color channels again to obtain the following:
A consequence of removing the lines is making the image blurry. This is expected since the lines provided texture and definition to the image before removing it. In some applications, simply applying a gaussian blur would help us remove these elements, but that is for another discussion.
Wrapping Up
In this article, we explored the many different ways of using Image Enhancement to improve the quality of our image. Next, we look at different morphological operations that we can do using scikit-image.
