Real-time, Adaptive Plane Sweeping for Free Viewpoint Navigation in Soccer Scenes

In this dissertation, we present a system to generate a novel viewpoint using a virtual camera, specifically for soccer scenes. We demonstrate the applicability for following players, freezing the scene, generating 3D images, et cetera. The method is demonstrated and investigated for 2 camera arrangements, i.e. a curved and a linear setup, where the distance between the cameras can be up to 10 meters. The virtual camera should be located on a position between the real camera positions. The method is designed to be automatic and has high quality results using high performance rendering. We presented an image-based method to generate the novel viewpoints based on the wellknown plane sweep approach. The method consists of a preparation phase and a rendering phase. In the preparation phase, geometric calibration is performed. Here, we presented a calibration system for large setups using the images of the recordings itself. No specific objects must be placed in the scene, but this is nevertheless possible. We applied feature detection on the input streams and match features between pairs of cameras. We present a method based on graphs that select multicamera feature matches using a voting mechanism. Furthermore, the matches are filtered based on the general direction in which the features appear to move across the different cameras, which is a robust outlier detection. These filtered multicamera feature matches are then used to generate the calibration data. The results demonstrate the quality of the calibration, which is sufficiently high for our method. Due to the automatic nature of the calibration method, we have achieved a convenient and practical solution for multicamera calibration in large scenes. Once the calibration is known, we can start rendering. We demonstrate that normal plane sweeping is not sufficient for soccer scenes due to the high number of artifacts, such as ghost legs, ghost players, and halo effects. Therefore, we propose a depth-aware plane sweep approach. We have shown that the depth values of the artifacts differ from the depth values of the players. This can be used to filter out the artifacts. We determine the initial depth using a plane sweep approach. Next, we filter the depth map using a median-based or histogrambased approach, where each group of pixels is processes independently. The depth is furthermore compared to the depth of the background, eliminating ghost player artifacts. The results show that the artifacts are effectively eliminated in most cases. We employed modern and traditional GPGPU technologies for the complete processing pipeline to develop a scalable and fast solution. The performance is higher than a few frames per second for a single GPU and HD resolution, which makes it practical and affordable to scale up to a realtime solution. The results are visually compared to existing systems, which demonstrates that our method can eliminate many artifacts visible in other systems. Furthermore, a novel plane distribution method is developed to assign more processing power to the depths where there actually are objects and to reduce wasted processing power on empty space. The quality is checked qualitative and it is demonstrated that the difference between a high number of planes and a redistributed low number of planes is negligible, and the difference between a uniformly distributed low number of planes and a redistributed low number of planes is significant. This shows the usefulness of the optimization by reducing the required processing power, while keeping quality levels comparable.

Jaar van publicatie:2014

Toegankelijkheid:Open