Deep Learning Methods for Global Illumination Transport in Computer Graphics

The global illumination light transport problem is a longstanding problem in computer graphics. Given the description of a 3D virtual scene, a photo-realistic image is generated. The state-of-the-art algorithms are all based Monte Carlo integration techniques. Recent advances in image synthesis have included machine learning principles such that the computational effort per pixel can be reduced. These techniques are often aimed at stochastic variance reduction, and primarily work in the 2D image domain, analogous to many results already achieved by computer vision applications. However, the question arises whether learning algorithms can also operate directly in the 3D scene, by trying to generate the 3D light distribution rather than its 2D image projection. Such an approach could be camera-independent and only scene-dependent, and therefore yield further computational benefits. The research questions are the following:

• What 3D representation is best suited to apply learning algorithms to 3D scene descriptions, specifically for the context of global illumination algorithms?
• What training data would be needed to feed such a learning algorithm?
• Is such an approach only feasible for relatively simple solutions only (e.g. diffuse illumination), or can it be scaled up to more complex light interactions?