DFG project on Deep Models for Handheld Light Field Acquisition

DFG project on Deep Models for Handheld Light Field Acquisition

Dr. Paramanand Chandramouli received a DFG funding for his project on "Deep Models for Handheld Light Field Acquisition". He will be hosted by the Computer Graphics Chair at University's Center for Sensor Systems (ZESS). Previously, he held a postdoctoral researcher position at the University of Bern, Switzerland, after completing his MS and PhD at the Indian Institute of Technology Madras, India.

Collaboration partners of Dr. Paramanand Chandramouli at ZESS:
Prof. Dr. Andreas Kolb, University of Siegen
Prof. Dr. Michael Möller, University of Siegen

The Project
Image-based rendering (IBR) is a central and interdisciplinary research topic in computer graphics  and computer vision. It comprises the acquisition, representation and synthesis of light fields that are mainly based on a set of previously acquired images, i.e. light field samples. Depending on the approach, additional depth data is used for efficient image interpolation. Even though investigated for more than 20 years, the acquisition of high quality light field data commonly undergoes controlled restrictions with regard to the acquired parameter sub-space and/or dimensions of the underlying, high-dimensional plenoptic function. Therefore, the acquisition of high quality light field data is either restricted to low angular or spatial variations (small baselines) when using, e.g., plenoptic cameras or it requires sophisticated and exhaustive hardware setups comprising dozens of cameras. Recently, learning based approaches have also been used in light field research, mainly ddressing plenoptic cameras, i.e. light fields with small baselines.

This research project aims at a lightweight approach for the online acquisition of light fields for unrestricted scene configurations by incorporating deep learning models. It involves development of novel deep learning models that efficiently represent light fields in a generic way. This will allow for efficient light field storage, processing and reconstruction under different, mainly sparse observation models.

Sample Illustration
From Sparse 4D Light Field to Reconstructed Light Field

Fig.: Illustration of a reconstruction task from sparse measurements