Queueing Analysis of an Ensemble Machine Learning System
Loading...
Access rights
openAccess
acceptedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
15
Series
Analytical and Stochastic Modelling Techniques and Applications - 28th International Conference, ASMTA 2024, Proceedings, pp. 97-111, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) ; Volume 14826 LNCS
Abstract
Recent advances in AI/ML technologies have accelerated the development of various ML applications. One of the major trends in AI/ML application development is the increasing use of multiple ML models to support high-accuracy inference in a complex end-to-end ML serving. However, testing the right configuration of multiple ML models is expensive, and the application requirements for ML inferences are highly dependent on various factors like the quality of ML models, computing resource performance, and data quality. In this context, techniques and methods that help to emulate and analyze ML inference characteristics using queueing theory can reduce the development effort and cost for ML services encapsulating ML models but also the entire ML system. In this paper, we modeled and analyzed a queueing model for an ML system that uses ensemble learning as an inference method with a new rule and clarified the impacts of model design in ensemble learning on the system’s performance. As a result, we demonstrate the usefulness of the analysis for understanding possible configurations and their efficiency in the ML system through queueing analysis and simulation.Description
Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
Keywords
Other note
Citation
Tsutsumi, K, Phung-Duc, T & Truong, L 2024, Queueing Analysis of an Ensemble Machine Learning System. in A Devos, A Horváth & S Rossi (eds), Analytical and Stochastic Modelling Techniques and Applications - 28th International Conference, ASMTA 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14826 LNCS, Springer, pp. 97-111, International Conference on Analytical and Stochastic Modelling Techniques and Applications, Venice, Italy, 14/06/2024. https://doi.org/10.1007/978-3-031-70753-7_7