Edge capacity planning for real time compute-intensive applications

Abstract

Cloud computing is a major breakthrough in enabling multi-user scalable web services, process offloading and infrastructure cost savings. However, public clouds impose high network latency which became a bottleneck for real time applications such as mobile augmented reality applications. A widely accepted solution is to move latency sensitive services from the centralized cloud to the edge of the Internet, close to service users. An important prerequisite for deploying applications at the edge is determining initial required edge capacity. However, little has been done to provide reliable estimates of required computing capacity under Quality-of-Service (QoS) constraints. Differently from previous works that focus only on applications' CPU usage, in this paper, we propose a novel, queuing theory based edge capacity planning solution that takes into account both CPU and GPU usages of real-time compute-intensive applications. Our solution satisfies the QoS requirements in terms of response delays while minimizing the number of required edge computing nodes, assuming that the nodes are with fixed CPU/GPU capacity. We demonstrate the applicability and accuracy of our solution through extensive evaluation, including a case study using real-life applications. The results show that our solution maximizes the resource utilization through intelligent combinations of service requests, and can accurately estimate the minimal amount of CPU and GPU capacity required for satisfying the QoS requirements.