Foveated streaming of real-Time graphics

Loading...
Thumbnail Image

Access rights

openAccess

URL

Journal Title

Journal ISSN

Volume Title

A4 Artikkeli konferenssijulkaisussa

Date

2021-07-15

Major/Subject

Mcode

Degree programme

Language

en

Pages

12
215-226

Series

MMSys 2021 - Proceedings of the 2021 Multimedia Systems Conference

Abstract

Remote rendering systems comprise powerful servers that render graphics on behalf of low-end client devices and stream the graphics as compressed video, enabling high end gaming and Virtual Reality on those devices. One key challenge with them is the amount of bandwidth required for streaming high quality video. Humans have spatially non-uniform visual acuity: We have sharp central vision but our ability to discern details rapidly decreases with angular distance from the point of gaze. This phenomenon called foveation can be taken advantage of to reduce the need for bandwidth. In this paper, we study three different methods to produce a foveated video stream of real-Time rendered graphics in a remote rendered system: 1) foveated shading as part of the rendering pipeline, 2) foveation as post processing step after rendering and before video encoding, 3) foveated video encoding. We report results from a number of experiments with these methods. They suggest that foveated rendering alone does not help save bandwidth. Instead, the two other methods decrease the resulting video bitrate significantly but they also have different quality per bit and latency profiles, which makes them desirable solutions in slightly different situations.

Description

Funding Information: We would like to thank Enrico Masala for valuable insights into video compression. This work has been financially supported by the Academy of Finland (grant number 332306) and the Nokia Center for Advanced Research . Publisher Copyright: © 2021 ACM.

Keywords

cloud rendering, foveated rendering, foveated streaming, virtual reality

Other note

Citation

Illahi, G K, Siekkinen, M, Kamarainen, T & Yla-Jaaski, A 2021, Foveated streaming of real-Time graphics . in MMSys 2021 - Proceedings of the 2021 Multimedia Systems Conference . ACM, pp. 215-226, ACM Multimedia Systems Conference, Virtual, Online, Türkiye, 28/09/2021 . https://doi.org/10.1145/3458305.3463383