This work pertains to surface measurement and surface tracking of human body parts using video sequences acquired by multiple cameras. Traditionally, and even today, research and commercial applications were concentrated either on the measurement and modeling of the human face and body or on the capture of movement of the whole body and facial expressions. In this work, a method is presented to treat both the surface measurement aspects and the surface tracking aspects in a unique process. The proposed method can be applied to measure either the surface of a static human body part or the moving surface of a dynamic event. In the latter case, the 3-D data gained can be of two different types: surface measurement of the part of interest in the form of a 3-D point cloud for each recorded time step or surface tracking in form of a vector field of 3-D trajectories. The process is composed of seven steps: (1) calibration of the system, i.e., establishing the internal and external orientation of the cameras and the parameters modeling the lens distortions; (2) acquisition of multi-image sets and/or multi-image sequences; (3) matching process, i.e., establishing correspondences in the multi-images; (4) computation of the 3-D point cloud for each matched multi-image set; (5) surface tracking in the multi-image sequences; (6) establishing a 3-D vector field of trajectories (position, velocity, acceleration); and (7) tracking key-points in the vector field of trajectories. In the case of surface measurement, only the first four steps are required. The accurate measurement of a human body part starts with an adequate acquisition of the required data. In the case of static surface measurement are used multi-images (multiple images acquired from different positions in the space describing the same scene), while in the case of dynamic surface measurement and surface tracking are used multi-image sequences (multi-images acquired during a time interval). For the acquisition of multi-images different systems can be applied with differing levels of quality depending on the cameras used. The orientation and calibration processes establish the position and orientation of the camera sensors in 3-D space, the parameters describing the internal geometry of the imaging device, and the parameters modeling the distortions caused by the optical system. A thorough determination of all the parameters is required for accurate measurement using photogrammetric techniques. The goal of the automatic matching process is the determination of a dense set of corresponding points in the multi-images on the part of surface of interest. The process uses a stereo matcher based on least squares matching techniques. The automatic matching process begins by defining several seed points. Starting from them, a dense and robust set of corresponding points covering the entire interested region is generated. The 3-D coordinates of the matched points are then computed by forward ray intersection using the results of the calibration process. The strategy is customized to the characteristics of the surface of the human body. Moreover, it is designed to reduce the required processing time to a minimum. The basic idea of the multi-image tracking process is tracking corresponding points in the multi-images through the sequence and computing their 3-D trajectories. Velocities and accelerations are also computed at each time step. The process is based on least squares matching techniques. These are applied to determine the spatial correspondences between the images acquired simultaneously from different views, as well as to determine the temporal correspondences between subsequent frames. The proposed process can be used to track well defined points on the human body surface. Trajectories of single points, however, are not sufficient to understand and record the motion and movement of a human or the changes of the surface of human body parts. Accordingly, the tracking process is extended to simultaneously track a dense set of points belonging to a common surface. In this case, the result of the tracking process can be considered to be a 3-D vector field of trajectories. To solve additional problems caused by occlusions, lack of texture, loss of tracked points and the appearance of new points, the concept of key-points is introduced. The key-points are 3-D regions, defined in the vector field of trajectories, whose size can vary and whose position is defined by their center of gravity. The key-points are interactively defined in a graphical user interface and tracked simply; the position in the next time step is computed as the mean value of the displacements of all the trajectories contained inside the 3-D region. Graphical user interfaces were developed and implemented for all of the processes employed, including the multi-image acquisition, the calibration and orientation procedures, the automatic matching process and the visualization of the results as 3-D point clouds and 3-D vector field of trajectories. To demonstrate the multiple functionality of the method, three different applications are presented: high accuracy measurement of human faces using five CCD cameras, measurement of a blood vessel branching casting fixed in a rotating frame using three CCD cameras and full body motion capture without markers using video sequences acquired by two or three synchronized CCD cameras.
© Copyright 2003 Veröffentlicht von Institut für Geodäsie und Photogrammetrie an der Eidgenössischen Technischen Hochschule ETH Zürich. Alle Rechte vorbehalten.