Volumetric data streaming from smartphones

Loading...
Thumbnail Image
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu | Master's thesis
Date
2020-08-18
Department
Major/Subject
Security and Cloud Computing (SECCLO)
Mcode
SCI3084
Degree programme
Master’s Programme in Security and Cloud Computing (SECCLO)
Language
en
Pages
53+0
Series
Abstract
Smartphones are increasingly being equipped with dedicated depth cameras that directly measure the distance to objects in a scene. These cameras enhance the 3D perception of the world and enable a wide range of immersive applications (such as 3D tele-presence and tele-surgery). However, such applications often require real-time processing of depth data (for instance, to carry out 3D reconstruction), which can be infeasible to run on a mobile device. On the other hand, the depth data may be processed on the cloud. However, this requires streaming large amounts of volumetric data. To this end, the emergence of edge computing plays a key role in providing the necessary communication and computational resources required to enable the next wave of immersive applications. The goal of this thesis is to study the feasibility of a real-time 3D tele-presence application that captures depth data from one user's smartphone and presents their volumetric 3D model to the other as a 3D avatar. We leverage a server-class edge device to overcome the limited computational capabilities of smartphones. We implement a mobile application that offloads the captured depth data to the edge server. An application on the edge server performs the 3D reconstruction that can be visualized on another smartphone or a head-mounted display. We evaluate the proposed solution in terms of the reconstruction quality and the overall performance. The experiments prove the feasibility of our solution, while achieving acceptable visual quality and reconstruction frame rate. In addition, we present several insights into designing such an application and highlight open questions for future work.
Description
Supervisor
Di Francesco, Mario
Thesis advisor
Premsankar, Gopika
Keywords
volumetric data, depth sensing, Time-of-Flight, streaming, edge computing, 3D reconstruction
Other note
Citation