Robust shape reconstruction from combined shading and stereo information

In this research, we first show that single-image shape from shading (SFS) algorithms share an inherent limitation in the accuracy of reconstructed surfaces due to the property of the reflectance map. That is, surface orientations can be accurately recovered if they lie along the gradient direction of the reflectance map function, but cannot be easily recovered if lying along the tangential direction. Then, we consider two methods which incorporate stereo information with shading to improve the performance. One is to use multiple images taken under different lightening conditions known as photometric stereo, and the other is to incorporate the height information obtained from images taken from different viewing angles known as the geometric stereo. With photometric stereo, we compensate the weakness of each reflectance map by combining several reflectance maps in a proper way in the gradient space and hence improve the accuracy of the results. With geometric stereo, absolute heights at sparse feature points are obtained and used as constraints on the resulting surface so that the ambiguity can be resolved. Simulation results for several test images are given to show the performance of our new robust algorithms.