Foveated streaming of real-Time graphics

Thumbnail Image
Access rights
Journal Title
Journal ISSN
Volume Title
Conference article in proceedings
Degree programme
MMSys 2021 - Proceedings of the 2021 Multimedia Systems Conference
Remote rendering systems comprise powerful servers that render graphics on behalf of low-end client devices and stream the graphics as compressed video, enabling high end gaming and Virtual Reality on those devices. One key challenge with them is the amount of bandwidth required for streaming high quality video. Humans have spatially non-uniform visual acuity: We have sharp central vision but our ability to discern details rapidly decreases with angular distance from the point of gaze. This phenomenon called foveation can be taken advantage of to reduce the need for bandwidth. In this paper, we study three different methods to produce a foveated video stream of real-Time rendered graphics in a remote rendered system: 1) foveated shading as part of the rendering pipeline, 2) foveation as post processing step after rendering and before video encoding, 3) foveated video encoding. We report results from a number of experiments with these methods. They suggest that foveated rendering alone does not help save bandwidth. Instead, the two other methods decrease the resulting video bitrate significantly but they also have different quality per bit and latency profiles, which makes them desirable solutions in slightly different situations.
Funding Information: We would like to thank Enrico Masala for valuable insights into video compression. This work has been financially supported by the Academy of Finland (grant number 332306) and the Nokia Center for Advanced Research . Publisher Copyright: © 2021 ACM.
cloud rendering, foveated rendering, foveated streaming, virtual reality
Other note
Illahi , G K , Siekkinen , M , Kamarainen , T & Yla-Jaaski , A 2021 , Foveated streaming of real-Time graphics . in MMSys 2021 - Proceedings of the 2021 Multimedia Systems Conference . ACM , pp. 215-226 , ACM Multimedia Systems Conference , Virtual, Online , Turkey , 28/09/2021 .