Project Overview
In computer vision, the ‘shape from shading’ problem serves as one avenue for reconstruction of 3D surfaces from 2D images. This project implements one solution, given some constraints on the problem.
Given three images of the same scene where each image is illuminated with its own point light source, we can procedurally recover approximately the depth from the image plane to objects in the image. For this to work, the lighting must remain static with respect to the camera location and viewpoint after calibration, and furthermore, we must know the locations of the point light sources. from the start. We must also assume that all visible surfaces are close to lambertian reflectors, meaning that viewing angle does not affect the perceived intensities.
The calibration step involves placing a ‘calibration sphere’ in view of the camera, allowing us to capture the lighting conditions of each light - each image consists of only one of the three lights turned on. From these images we can extract (in pixels) the sphere diameter, allowing us to compute direction vectors from the sphere center to each respective light. This can be done by simply finding the brightest point on the sphere, which naturally indicates the direction from the sphere center towards the light source.
From here, we want to generate a table mapping three intensities at the same pixel location across the three images to gradient. This involves looking at the normalized light direction vectors and considering a wide range of potential gradient values from which we can construct a surface normal. The scalar product of the light direction vector and surface normal (but with unit length) give us a scalar between 0 and 1, which depends on the cosine of the angle between the two vectors. Analogously, the brightest parts of the image should have intensity close to 1 coinciding with where the light is hitting the most directly, and where darker parts correspond to surfaces illuminated at more extreme angles. We can therefore interpret this scalar product as approximately the expected pixel intensities at the same small patch across the three images with the same surface normal, assuming pixel intensities live in the interval [0,1]. To fill the table, we iterate across a wide range of possible surface normals where, for each given normal, we obtain three 'intensities' from the scalar products of the normal and the light direction vectors. We use these intensities to map to the surface gradient denoted (p,q) which we can easily extract from the normal.
With the table built for specified lighting conditions, we can input new triples of images, and recovering gradient becomes straightforward: we simply examine pixel intensities at each location across the three images and use these to index the table. Below, each input comes as a set of three images for each separate light, but only of each set is shown.
Once we have surface gradient at all locations, we can use this information to reconstruct the relative depth by means of some numerical integration strategy. In this case, however, a solution to the discrete Poisson equation was implemented. In the resulting 'depth images' that follow, brighter pixels correspond to regions that are closer, and darker pixels correspond to regions that are further away.
Source Code: https://github.com/CurtisBuckoll/CMPT412_A3