Light Field Renderer
My masters dissertation project: Sparse light field sampling and reconstruction
-
Date: 06/01/2022
Languages: C++
Keywords: Dissertation, Graphics, Ray-tracing
Overview
For my masters dissertation, I created a Light Field renderer. A light field consists of a multi-dimensional function that returns a light value for any point inside a real or virtual scene. My dissertation achieves this by ray-tracing from multiple cameras which produce multiple views of the same scene. These views can be attached together to produce a viewable, interactable light field. However, my dissertation is more focused on the efficiency of the rendering process. To increase the light field renderer’s efficiency only a smaller set of pixels are ray-traced from. These known pixels are then used to reconstruct the rest of the light field using one of two techniques: filtering and reprojection. The reconstruction method improves the light field renderer’s efficiency due to only a smaller subset of pixels needing to be fully sampled which is an expensive process
Sparse Light Field Sampling
To determine which pixels to ray-trace from the Sobol sequence is used. The Sobol sequence is a type of quasirandom or low-discrepancy sequence which means that the points in the sequence will also cover a specific space uniformly. Using this sequence the light field renderer produces a set of images from the camera’s views. These images can be seen to only have some pixels coloured in as dictated by the Sobol sequence.
Filtering reconstruction
Once the light field has been sparsely sampled the remaining missing pixels are reconstructed. The filtering method entails finding a missing pixel and determining its colour value using a cross-bilateral filter. This filter uses nearby multiple values from known nearby pixels such as its index and the position of its respective ray’s intersection against the missing pixels values to determine weights. These weights are then applied used to get a weighted average of the colour of the missing pixel. For this process to work the missing pixel is still somewhat sampled initially just to find the values needed for the cross-bilateral filter but not its colour. This produces complete but soft-looking images. This softness reduces as the number of pixels being sampled increases.
Reprojection reconstruction
The other method of reconstruction is reprojection. This works by taking a known pixel and reprojecting rays back towards cameras from the point of the original ray’s intersection. This is done to find missing pixels which colour can then be set to the initial known pixel’s colour. Once all the known pixels have been through this reprojection process there is a high chance that not all the missing pixels have been found and set. Because of this the light field renderer goes through all the missing pixels and ray-traces them in the same way that was done in the initial step. This produces much sharper images than the filtering which to the human eye no different to the original but still do however have slight differences in colour.