Four- dimensional sparse filters for near real-time light field processing

Abstract

Light is a fundamental form of conveying information. Sensing of light through conventional cameras leads to images and videos. In contrast to conventional images and videos, which capture only the directional variation of the intensity of light rays emanating from a scene, light fields capture the spatial variation as well. This richness of information has been exploited to accomplish novel tasks that are not possible with conventional images and videos, such as post-capture digital refocusing and depth filtering. As a result of the massive data volume captured by a light field, the light field processing algorithms require higher memory and computational requirement. This is a major drawback for employing light fields in real-time applications. Hence, there is a need for investigating novel low-complexity light field processing algorithms that can be implemented in real-time applications. In this study, we address this critical research problem using multidimensional linear filter theory to develop novel low-complexity and sparse filters for light field processing. To this end, the work presented in this thesis focus on two major scenarios; light field denoising and volumetric refocusing. First, we present a novel low-complexity light field denoising algorithm, utilizing the sparsity of the region of support of a light field in the frequency domain. It turns out that the proposed filter runs in near real-time, compared to the previously reported light field denoising methods which take minutes. Next, a 4-D sparse filter for volumetric refocusing is presented. The proposed sparse filter provides 72% reduction of computational complexity compared to a non-sparse filter, with negligible distortion in fidelity.