Evaluation of Emerging Serverless Platforms

Loading...
Thumbnail Image

URL

Journal Title

Journal ISSN

Volume Title

Perustieteiden korkeakoulu | Master's thesis

Date

2020-08-18

Department

Major/Subject

Security and Cloud Computing

Mcode

SCI3084

Degree programme

Master’s Programme in Security and Cloud Computing (SECCLO)

Language

en

Pages

66 + 17

Series

Abstract

Software architectures have continuously evolved since the advent of cloud computing, resulting in several paradigm shifts from virtualization towards microservices. Serverless computing, or Function as a Service (FaaS), has lately emerged as a new model that allows developers to focus on the development of event-driven functions, while leaving operational tasks to the serverless platforms. These functions can elastically scale to meet demand, and users are only charged for the actual execution time and consumed computing resources. Therefore, its popularity has been rising, and there are currently several serverless platforms users can choose. Such platforms include hosted platforms that are fully managed by cloud providers and open-source platforms that can be installed, for instance, on a private infrastructure. The performance of hosted platforms has been evaluated in several research works, whereas the number of studies targeting open-source platforms is still limited. Moreover, the benchmarking tools developed so far are either not actively maintained or do not target open-source platforms. Thus, the goal of the thesis is to develop a benchmarking tool to evaluate hosted and open-source serverless platforms. To this end, we leveraged FaaSTest, an open-source tool that provides a set of tests for benchmarking AWS Lambda and Azure Functions, and extended it to support several open-source serverless platforms: Knative, OpenFaaS, Kubeless, and Fission. We then benchmarked the platforms under different scenarios, including different rates of incoming traffic and computational load of the deployed functions. For the hosted platforms, the results show that AWS Lambda outperformed Azure Functions. For the open-source platforms, the evaluation results were not fully conclusive due to the challenges in configuring a large number of platform-specific features. Despite that, we observed that Knative outperformed the other solutions with a higher ratio of successful responses and lower response time in most scenarios.

Description

Supervisor

Francesco, Mario Di

Thesis advisor

Premsankar, Gopika

Keywords

serverless, function-as-a-service, benchmark, performance, kubernetes

Other note

Citation