Digital Imaging — Scale-invariant Feature Transform, Laplacian Blending, Linear Spatial Filtering, and more...

ECE 558 (Digital Imaging) Projects

Aug 2021 - Dec 2021 — Raleigh, North Carolina

Overview:

The primary application for linear spatial filtering in image processing is usually for applying different distributions of noise, detecting linear signal properties like edges or corners, or detecting 'objects' by performing filtering and using a difference metric to see how disparate the region within the image is to the kernel. In this project, multiple linear operators (Sobel, Roberts, Prewitt, etc.) along with kernels like box & Gaussian were were implemented and tested. Above, you see the original image with the Prewitt operator along the 'x' (horizontal) direction being applied. Respective to each RGB channel, hence the color. After processed, the end result is the leftmost image. Below, the same operator in the 'y' (vertical) direction is applied and the magnitude is calculated on the leftmost image. Below the magnitude along the RGB channels is shown, hence why it is black and white. This is the most common application for these edge detecting operators, as it can be seen how effectively an edge is found.

x gradient ===> y gradient ===> gradient magntiude

I understand there are social implications / movements within the image processing community to stop using this Lena image, however this was one of the primary images used during this class. More can be read in the report below, see the Literature section.

PS: I also did a lot more during all these 3 projects with respect to image processing. Some art may be coming soon :~) — simple / bad examples of what I did w.r.t. LSF + animations are shown below:

A Gaussian 'pyramid' is essentially an image downsizing technique, where a variable amount of blur (but constant with respect to each 'pyramid') is applied to an image before downsampled. This downsizes the image while maintaining the 'look' of the image. The term 'pyramid' is used as a visualization if each successive downsample was tiled ontop of one another, creating a rectangular pyramid of sorts. A Laplacian pyramid can be approximated using a difference of Gaussian's at two respective blur amounts. This Laplacian pyramid was original made as a form of lossy image compression, where the original image can be reconstructed with a couple low-data Laplacian's that held detail of the image, and one single Gaussian (the lowest size of the pyramid) which held the color composition of the image. It was discovered that images can be blended during this reconstruction process by generating the Gaussian / Laplacian pyramids of two images as well as a mask. During reconstruction, the mask and its inverse is simply multiplied by each respective picture and then expanded. This creates a rudimentary blending effect. There is still room for color and texture correction, and this is why the example images shown blend objects with similar colors and textures. More can be read in the report below, see the Literature section.

Implemented blending and GUI for creating mask in MATLAB, all figures are my own.

A scale-invariant feature transform (SIFT) detector tries to detect features of an image invariant to transformations such as scale, rotation, and translation. The idea behind the SIFT detector is to take an image of some environment at different angles, lighting, etc., and to be able to extract key points within the image that align with both images, despite having some amount of linear relative distortion. The first step in finding these SIFT interest points happens to align with the same task of 'blob detection'. A series of linear spatial filters are applied to an image to detect edges of varying sizes. If an edge happens to be relatively circular + compact, the result constructively creates a 'blob', where the local maxima/minima of these blobs is a SIFT interest point. The size of each blob is a function of the types of filters used, the size of the image, and more.

For this project, there was an optional speed test for extra credit. I did not really need the extra credit, but I had already optimized my script / algorithm to such an extent where I decided go all out and compete. More can be read in the 'Algorithm' section of the report, shown below. I placed first in the competition.
More can be read in the report below, see the Literature section.