Evaluation of Emerging Serverless Platforms

Thumbnail Image
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu | Master's thesis
Security and Cloud Computing
Degree programme
Master’s Programme in Security and Cloud Computing (SECCLO)
66 + 17
Software architectures have continuously evolved since the advent of cloud computing, resulting in several paradigm shifts from virtualization towards microservices. Serverless computing, or Function as a Service (FaaS), has lately emerged as a new model that allows developers to focus on the development of event-driven functions, while leaving operational tasks to the serverless platforms. These functions can elastically scale to meet demand, and users are only charged for the actual execution time and consumed computing resources. Therefore, its popularity has been rising, and there are currently several serverless platforms users can choose. Such platforms include hosted platforms that are fully managed by cloud providers and open-source platforms that can be installed, for instance, on a private infrastructure. The performance of hosted platforms has been evaluated in several research works, whereas the number of studies targeting open-source platforms is still limited. Moreover, the benchmarking tools developed so far are either not actively maintained or do not target open-source platforms. Thus, the goal of the thesis is to develop a benchmarking tool to evaluate hosted and open-source serverless platforms. To this end, we leveraged FaaSTest, an open-source tool that provides a set of tests for benchmarking AWS Lambda and Azure Functions, and extended it to support several open-source serverless platforms: Knative, OpenFaaS, Kubeless, and Fission. We then benchmarked the platforms under different scenarios, including different rates of incoming traffic and computational load of the deployed functions. For the hosted platforms, the results show that AWS Lambda outperformed Azure Functions. For the open-source platforms, the evaluation results were not fully conclusive due to the challenges in configuring a large number of platform-specific features. Despite that, we observed that Knative outperformed the other solutions with a higher ratio of successful responses and lower response time in most scenarios.
Francesco, Mario Di
Thesis advisor
Premsankar, Gopika
serverless, function-as-a-service, benchmark, performance, kubernetes
Other note