We address the problem of fusing images from many video cameras or a moving video camera, possibly with external orientation data (e.g. GPS and INS data). The captured images have obvious motion parallax, but they will be aligned and integrated into a few mosaics with a large field-of-view (FOV) that preserve 3D information. We have developed a geometric representation that can re-organize the original perspective images into a set of parallel projections with different oblique viewing angles. In addition to providing a wide field of view, mosaics with various oblique views well represent occlusion regions that cannot be seen in a usual nadir view. Stereo pair(s) can be formed from a pair of mosaics with different oblique viewing angles and thus image-based 3D viewing can be achieved. This representation can be used as both an advanced video interface for surveillance or a pre-processing step for 3D reconstruction. 

The technique has been applied to four important surveillance and inspection applications: (1) three-dimensional (3D) reconstruction and moving target extraction via an airborne camera; (2) 3D scene presentation via a camera on a ground vehicle; (3) Under-vehicle inspection via a 1D array of cameras; and (4) 3D cargo inspection via gammar-ray vision. Video demos and experimental results will be shown.

Joint work with George Wolberg, Weihong Li, Li Zhao and Hao Tang at the City College of New York, Allen Hanson and Edward Riseman at the University of Massachusetts at Amherst. The work has been supported by AFRL, NSF, ARO, DARPA, NYSIA, CUNY and Atlantic Coast Technologies, Inc.