Project Overview
The goal of this project was to implement the image alignment and stitching pipeline similar to that which is found in modern cell phones. Ultimately, the problem we are trying to solve is this: given a set of images featuring a common scene, how do we best fit them together?
This program takes 5 roughly horizontally aligned images and follows four sequential steps to stitch them together:
These steps are repeated again for each of the third, fourth, and fifth images, stitching in an additional image to the composite each time:
In broad strokes, a single iteration looks like this:
Once the SIFT key points and descriptors are determined for both images, we find the most similar descriptors and pair them together. While some of these pairs are correct, many of them are not.
The so-called ratio test refines this list by removing some ambiguous pairings that are close to other potential matchings. The RANSAC algorithm is an iterative procedure that essentially produces a ‘best guess’ as to what the true mapping is. It takes four ratio-tested pairings at random, computes the homography, projects the key points from one image onto the other, and counts how many key points ended up with their supposed partner. This is repeated for some fixed number of iterations, picking four different ratio-tested pairs each time.
The pairings found from the iteration that produced the most consistent matchings are then used to compute a final homography, allowing us to project on image on to the other.
After compositing the first two images together, the third is stitched into the composite, then the fourth is added in, and finally the fifth. All of these examples consist of five images fit together.
To get the best results possible, some type of blending can be used to remove the left over image ‘seams’ visible due to colour or intensity differences across images. Much of this is due the ‘vignetting’ effect that becomes apparent when we piece together these composites.
The blending technique used here is known as ‘two-band blending’. The idea is to separate the overlapping regions of each image into their respective high and low frequency bands, and then cross-dissolve the separated bands from each image together on different curves better suited to the spectrums. Since the images are horizontally aligned, blending is performed left to right.
With blending, the stitching looks much cleaner in the overlapping regions.
Source code: https://github.com/CurtisBuckoll/CMPT469_P5
The API available at http://www.vlfeat.org was used for SIFT keypoint detection and description.