Unreal Realities: Non-Photorealistic Rendering in Virtual Reality
Unreal Realities was my honours year research project, published in Create World 2017.
Non-Photorealistic Rendering is a collection of techniques used to render images without trying to make them look realistic. Usually this is done to convey information more clearly (removing light and shade or adding outlines to make shapes easier to read in engineering drawings for example) or with artistic goals (trying to make the image resemble a comic book or sketch). A lot of the traditional uses for NPR (engineering, architecture, medical visualization, games, etc) have been disrupted by Virtual Reality. This raises the question, do NPR techniques work in VR?
The paper explores the importance of Temporal, Object Space and Stereoscopic Coherence. Here are the main takeaways:
Most effects that were already suitable for 3D virtual environments will work in VR unless the effect has Large Spacial Abstractions (such as the color banding on a toon shader or the lines of an outline shader) which are calculated based on the view direction. This is because each eye will calculate a different spacial abstraction, meaning something will be lit in one eye but not the other, which is very uncomfortable to look at.
This image is not stereo coherent. The lighting is coherent (shown as the large red circle) which you can see is in the same position on the sphere, when compared to the texture on the surface.
The specular highlight however has moved between the left and right eye because specularity is calculated based on the view direction.
In this image the specular highlight is stereo coherent. The same parts of the sphere surface are lit in both eyes.
This is achieved by calculating the view direction for both eyes from a global constant situated halfway between both eyes. Since the same value is used for both eyes, the result is stereo coherent.