Learning to Predict Head Pose in Remotely-Rendered Virtual Reality
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2023-06-07
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
12
Series
MMSys 2023 - Proceedings of the 14th ACM Multimedia Systems Conference, pp. 27-38
Abstract
Accurate characterization of Head Mounted Display (HMD) pose in a virtual scene is essential for rendering immersive graphics in Extended Reality (XR). Remote rendering employs servers in the cloud or at the edge of the network to overcome the computational limitations of either standalone or tethered HMDs. Unfortunately, it increases the latency experienced by the user; for this reason, predicting HMD pose in advance is highly beneficial, as long as it achieves high accuracy. This work provides a thorough characterization of solutions that forecast HMD pose in remotely-rendered virtual reality (VR) by considering six degrees of freedom. Specifically, it provides an extensive evaluation of pose representations, forecasting methods, machine learning models, and the use of multiple modalities along with joint and separate training. In particular, a novel three-point representation of pose is introduced together with a data fusion scheme for long-Term short-Term memory (LSTM) neural networks. Our findings show that machine learning models benefit from using multiple modalities, even though simple statistical models perform surprisingly well. Moreover, joint training is comparable to separate training with carefully chosen pose representation and data fusion strategies.Description
Funding Information: This work has been supported by the Academy of Finland under grant numbers 332306, 332307, and 357533. We would like to thank the CSC – IT Center for Science and the Aalto Science-IT project for provisioning the computational resources used for the evaluation. Publisher Copyright: © 2023 Owner/Author(s).
Keywords
machine learning, pose prediction, virtual reality, VR
Other note
Citation
Illahi, G K, Vaishnav, A, Kämäräinen, T, Siekkinen, M & Di Francesco, M 2023, Learning to Predict Head Pose in Remotely-Rendered Virtual Reality . in MMSys 2023 - Proceedings of the 14th ACM Multimedia Systems Conference . ACM, pp. 27-38, ACM Multimedia Systems Conference, Vancouver, British Columbia, Canada, 07/06/2023 . https://doi.org/10.1145/3587819.3590972