Real-time neural network based video super-resolution as a service: design and implementation of a real-time video super-resolution service using public cloud services
Loading...
URL
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
2022-03-21
Department
Major/Subject
Machine Learning, Data Science and Artificial Intelligence
Mcode
SCI3044
Degree programme
Master’s Programme in Computer, Communication and Information Sciences
Language
en
Pages
62
Series
Abstract
Despite the advancements in video streaming, we still find limitations when there is the necessity to stream real-time video in a higher resolution (e.g., in super- resolution) through mobile devices with limited resources. This thesis work aims to give an option to address this challenge through a cloud service. There were two main code components to create this service. The first component was aiortc (e.g., the WebRTC python version), the streaming protocol. The second component was the Efficient Sub-Pixel Convolutional Neural Network (ESPCN)-model, one of the outstanding methods to upscale video at the present time. These two code components were implemented in a virtual machine in the Microsoft Azure cloud environment with a customized configuration. Qualitative as well as quantitative results of this work were obtained and analyzed. To obtain the qualitative results two versions of the ESPCN-model were developed and for the quantitative outcomes three different configurations of HW/SW codecs and CPU/GPU utilization were produced and analyzed. Besides finding and defining the code components mentioned before as optimal to create an efficient real-time video super-resolution service based on the cloud, an- other conclusion of this project is that sending or receiving information (frames) from the CPU to the GPU and vice-versa has a very big negative impact in the efficiency of the whole service. Hence, to limit this CPU-GPU interaction or to only use GPU (e.g., with the NVIDIA Virtual Processing Framework [VPF]) is critical for an efficient service. This issue can be avoided, as the quantitative results show, if a codec that only makes use of the GPU (e.g., a NVIDIA HW codec) is employed. Furthermore, the Azure cloud environment component, enables an efficient execution of the service in diverse mobile devices. In future, the quality measure of the video super-resolution done by the ESPCN- model is suggested as a next step to do.Description
Supervisor
Siekkinen, MattiThesis advisor
Siekkinen, MattiKeywords
super-resolution, video, streaming, aiortc, ESPCN, video processing framework