3D Fusion of Stereo and Spectral Series Acquired with Camera Arrays

Book chapter


Ioana Gheţa
Michael Heizmann
Jürgen Beyerer


Yufeng Zheng (ed.), Image Fusion and Its Applications, InTech, June 2011.





One of the main requirements of industrial visual inspection is that the information acquisition is accomplished in real-time. Camera arrays are a promising solution since they offer the possibility of simultaneous image acquisition. Moreover, the acquisition parameters of the different cameras can be varied. Due to dropping prices of industrial cameras, a large number of cameras can be employed in a camera array for automated visual inspection.

The advantages offered by camera arrays come with a price. Simultaneously triggering the cameras results in obtaining image series that contain a stereo effect. If more acquisition parameters (e. g., focus, different spectral filters) are varied, the obtained image series are combined image series, i. e., the images differ in more than one effect. For example, if the cameras are equipped with spectral filters, the obtained image series are combined stereo and spectral series, i. e., the images differ due to both the stereo effect and the acquisition in different parts of the spectrum. However, the main advantage of such image series is that they contain different types of information gained simultaneously: in this case, it is spatial information due to the stereo effect and spectral information due to the use of spectral filters. The challenge consists in fusing the image series, since the different types of information in the combined image series cannot be evaluated separately.

The present chapter deals with different methods of fusing combined stereo and spectral images in order to obtain both spatial and spectral information. For obtaining the spatial information, region based image registration methods for the exploitation of the stereo effect are presented. The problem is modeled with energy functionals, which are minimized by state-of-the-art methods, e. g., dynamic programming or graph cuts. With the help of the obtained spatial information (in form of depth maps), the spectral information can be extracted and further employed, e. g., for material classification or an improved object detection.