This thesis addresses the problem of 3D reconstruction of a non-rigid human subject using a single moving RGB-D camera. A robust solution to this problem is necessary for a broad range of applications, including computer animation, visual effects, sports analytics, health care, and biomechanics. The large range of sudden motion and considerable non-rigid deformations of the human body make this problem extremely challenging. The reconstruction from a single-view is further complicated by self-occlusion and ambiguity between camera and object motion. This thesis addresses three challenges for developing a robust framework for the 3D reconstruction of human subjects: 1. Correlation between camera and object motion hinders the accuracy of nonrigid reconstruction from a moving camera. We explore the use of camera pose for decoupling camera and object motion, thereby improving the 3-D reconstruction of a non-rigid object from a moving camera. 2. Fast articulated motions of human subjects make 3D reconstruction very challenging. We explore utilizing a skeleton prior to provide additional constraints and aid in reconstructing fast-moving humans. A detection guided optimization approach with a validation step is proposed. The validation step corrects tracking failures due to errors in the skeleton prior. The tracking and validation advance sequentially in a hierarchical manner, enabling body-part level motion refinement. 3. The multi-scale nature of non-rigid motion leads to tracking failures and results in noisy reconstructions. A coarse-to-fine tracking approach is proposed for handling the multi-scale motion of human subjects. Finally, we combine all contributions to achieve improvements over state-of-theart methods for reconstructing fast moving human subjects from a single moving RGB-D camera.
© Copyright 2020 All rights reserved.