Shape From Shading
http://carlotto.us/martianenigmas/Articles/SFS/sfs.html
Shape From Shading
Mark J. Carlotto
Shape-from-shading (also known as photoclinometry) is a method for determining the shape of a surface from its image. For a surface of constant albedo, the brightness at a point (x,y) in the image is related to the gradients (p,q) by the following expression
i(x,y) = a R[p(x,y),q(x,y)]
where R is the reflectance map, p = dz/dx and q = dz/dy are the partial derivatives of the surface in the x- and y-directions, and a is a constant that depends on the albedo, the gain of the imaging system and other factors. The above expression also assumes that any additive offsets, e.g., due to atmospheric scattering, have been removed.
A variety of methods have been developed for inverting the above equation (see Horn 1990). The next section describes a simple method that provides satisfactory results in many planetary imaging scenarios. It is based on some early ideas described by Horn (1977).
Row Integration
The reflectance map depends on the position of the light source, the observer, and the type of surface material (Horn 1981). It can be thought of as a lookup table that gives the brightness as a function of the gradients. For Lambertian surfaces the brightness is proportional to the cosine of the angle between the vector that is normal (perpendicular) to the surface and the vector in the direction of the light source. As noted by Pentland (1988), if the angle between the vector in the direction of the light source and the vector in the direction of the observer are more than 30 degrees apart and the surface is not too rough the reflectance map can be approximated by a linear relationship. If the image is rotated so that the vector that points to the sun is in the x-z plane, it can be shown that
i(x,y) ~ a [sin(s) p(x,y) + cos(s)]
where s is the zenith angle of the sun. The constant scale factor a is difficult to determine directly without ground truth (i.e., ground targets with known albedo and slope). However, since in most images the gradients are more-or-less uniformly distributed in all directions, the expected value of the gradient in the x-direction E[p] ~ 0 and so the average image brightness
E[i] ~ a cos(s).
This then allows us to estimate the scale factor
a = E[i] / cos(s).
The elevation map z(x,y) can be obtained iteratively, row-by-row as
z(x,y) = z(x-1,y) + [i(x,y) - a cos(s)] / a sin(s)
where z(0,y) are the boundary values. If the boundary values z(0,y) are unknown, we can minimize the mean-squared elevation difference between rows by subtracting the average row elevation from the elevations in the row.
Synthetic Image Example
The image below (left) is of that of a conical surface where brightness represents height. The image below (right) is a shaded rendition of the cone viewed from below (azimuth = 180 deg., zenith = 70 deg.) using a Lambertian reflectance map with the sun to the left (azimuth = 270 deg., zenith angle = 45 deg.).
The figure below (left) shows a synthetic image generated with the sun directly overhead (zenith = 0 deg.). The image below (right) is the elevation surface estimated from that image. The poor performance of shape-from-shading algorithms at high sun angles (low zenith angles) is like that experienced by image analysts under similar conditions.
As the zenith angle increases the performance improves. The figure below (left) shows a synthetic image generated with the sun at an azimuth angle of 270 deg. and a zenith angle of 30 deg. The image below (right) is the elevation surface estimated from that image.
As the zenith angle continues to increase the true shape of the surface begins to emerge. The figure below (left) shows a synthetic image generated with the sun at an azimuth angle of 270 deg. and a zenith angle of 60 deg. The image below (right) is the elevation surface estimated from that image.
Comparing perspective views of the original and estimated elevation surface from the last example shows the method capable of recovering the general shape of the surface. The slight tilt in the surface in the direction of the sun is a common artifact of this kind of algorithm.
Lunar Image Example
The image below (right) is a portion of frame LUE50467 acquired by the Clementinespacecraft in its 218-th orbit around the moon. The image has been rotated so that the sun is to the left. The middle image is the elevation surface computed by the shape-from-shading algorithm. One method of validating this result is to compute a shaded rendition from the elevation surface under the same illumination and viewing conditions as the original image. The shaded rendition (right) compares favorably with the original image (left).
Once the elevations are known one can generate simulated perspective views of the scene. By generating two views that are within a few degrees of one another, 3-D renditions in the form of synthetic stereo images can be created. The image below is a synthetic stereo pair generated from directly overhead. It is rendered anaglyphically with the left image encoded in red and the right image in blue as well as side-by-side for those that do not have colored glasses.
To view the images in 3-D, look away from picture and focus at a point in the distance. Then shift your gaze to the picture. You should see three images. Concentrate (but don't focus) on the middle image and try to bring different parts of the view into correspondence. This will take some practice. If you have difficulty seeing 3-D, try moving back from the screen.
The image below is an oblique stereo pair. A parallel projection algorithm is used (Foley and Van Dam 1983) that simulates how the scene would appear at a distance as seen through a zoom lens. The observer is located at a azimuth of 180 deg. and a zenith angle of 45 deg.
Conclusion
When stereo imagery does exist, shape-from-shading provides an alternative method for extracting terrain data. Although its use has been limited primarily to constant-albedo planetary mapping applications (i.e., where the surface is covered more-or-less by the same material) new algorithms currently under development will extend to the general case in terrestrial imaging applications where the albedo is not constant.
References
B. K. P. Horn, "Understanding image intensities," Artificial Intelligence, Vol. 8, pp 201-231, 1977.
B. K. P. Horn, "Hill shading and the reflectance map," Image Understanding Workshop, Palo Alto, CA 1979.
B. K. P. Horn, "Height and gradient from shading," International Journal of Computer Vision, Vol. 5, No. 1, 1990.
A. Pentland, "The transform method for shape-from-shading," MIT Media Lab Vision Sciences Tech. Report 106, July 15, 1988.
J. D. Foley and A. Van Dam, Fundamentals of Interactive Computer Graphics, Addison-Wesley, Reading, MA, 1983.
