CuLao - Constructing Utilities of Large Language Models in Resource-Constrained Environments
Loading...
Access rights
openAccess
acceptedVersion
URL
Journal Title
Journal ISSN
Volume Title
A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
6
Series
GoodIT '24: Proceedings of the 2024 International Conference on Information Technology for Social Good, pp. 100-104
Abstract
The increasing development and utilization of Large Language Model (LLM) services have demonstrated many benefits in different contexts. However, LLM services are mainly available in the public cloud and require huge computing resources to operate, thus not accessible to many companies, organizations or communities with constrained resources. While research efforts have concentrated on LLMs quantization for resource-constrained computing environments like edge devices, to democratize the availability of LLM services as utilities for such communities requires much more than the optimization of LLM models. In this paper, we introduce CuLao - a framework for constructing utilities from LLMs in resource-constrained environments. Our framework focuses on key requirements of resource-constrained companies, organizations and communities by enabling the provisioning and coordination of LLMs as utilities, based on the availability of open-source LLMs. CuLao provides techniques and tools for abstracting LLMs as services with suitable APIs and coordinating them as utility ensembles in edge infrastructures.Description
Keywords
Other note
Citation
Truong, H-L & Nhu Trang, N N 2024, CuLao - Constructing Utilities of Large Language Models in Resource-Constrained Environments. in GoodIT '24: Proceedings of the 2024 International Conference on Information Technology for Social Good. ACM, pp. 100-104, International Conference on Information Technology for Social Good, Bremen, Germany, 04/09/2024. https://doi.org/10.1145/3677525.3678648