aalto1 untyped-item.component.html
Study and benchmarking of Artificial Intelligence (AI) model serving systems on edge computation units and cloud environments
Loading...
URL
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
Department
Major/Subject
Mcode
SCI3045
Language
en
Pages
58+3
Series
Abstract
Recently, the development of Edge AI, which brings computation resources and artificial intelligence closer to the network edge, has been growing rapidly. Compared to Cloud AI applications, Edge AI takes advantage of low network latency and reduced bandwidth consumption. However, most of the Edge AI applications are still in the conceptual phase, and thorough implementations are not well achieved. In addition, the performance impact of the AI model serving that moves from cloud to edge devices - which can be resource-constrained - has not been sufficiently investigated. This thesis aims to fill the gap in this respect, by investigating and benchmarking edge-based and cloud-based AI model serving, through the use of existing serving systems such as TensorFlow Serving and Clipper. Furthermore, we rely on the usage of Kubernetes for ensuring more robustness and higher scalability both on edge and cloud - we use a Single-Board Computer as edge device, and Google Kubernetes Engine (GKE) as cloud environment. From the empirical point of view, we characterize the performance of Edge AI and Cloud AI model serving in terms of throughput and response time, by taking into account several AI applications that are based on classical machine learning and deep learning algorithms. Our results show that, as expected, Edge AI benefits from lower network latency, while Cloud AI benefits from higher computational resources availability. However, we found out that the an edge-based approach introduces some advantages when serving high-throughput applications, when caching mechanisms are used, and when a few users (preferably one) query the model serving.