Optimizing Multiple Consumer-specific Objectives in End-to-End Ensemble Machine Learning Serving

Optimizing the quality of machine learning (ML) services for individual consumers with specific objectives is crucial for improving consumer satisfaction. In this context, end-to-end ensemble ML serving (EEMLS) faces many challenges in selecting and deploying ensembles of ML models on diverse resources across the edge-cloud continuum. This paper provides a method for evaluating the runtime performance of inference services via consumer-defined metrics. We enable ML consumers to define high-level metrics and consider consumer satisfaction in estimating service costs. Moreover, we introduce a time-efficient ensemble selection algorithm to optimize the EEMLS with intricate trade-offs between service quality and costs. Our intensive experiments demonstrate that the algorithm can be executed periodically despite the extensive search space, enabling dedicated optimization for individual consumers in dynamic contexts.

Keywords

ML Serving, Ensemble Selection, Ensemble ML, End-to-End ML, Performance Evaluation

Citation

Nguyen, T, Truong, L, Arcaini, P & Ishikawa, F 2024, ' Optimizing Multiple Consumer-specific Objectives in End-to-End Ensemble Machine Learning Serving ', Paper presented at IEEE/ACM International Conference on Utility and Cloud Computing, Sharjah, United Arab Emirates, 16/12/2024 - 19/12/2024 .

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202411067019

Collections

[article-cris] Perustieteiden korkeakoulu / SCI

Show all metadata

Optimizing Multiple Consumer-specific Objectives in End-to-End Ensemble Machine Learning Serving

Access rights

URL

Journal Title

Journal ISSN

Volume Title

Authors

Date

Department

Major/Subject

Mcode

Degree programme

Language

Pages

Series

Abstract

Description

Keywords

Other note

Citation

Permanent link to this item

Collections