AI-driven business performance assessment

Thumbnail Image

Files

master_Ghafoori_Miran_2025.pdf (4.17 MB)

URL

Journal Title

Journal ISSN

Volume Title

School of Engineering | Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2024-12-31

Department

Major/Subject

Additive Manufacturing for Full Flexibility

Mcode

Degree programme

Master's Programme in Manufacturing

Language

en

Pages

68

Series

Abstract

Artificial Intelligence (AI) continues to transform industries, democratizing access to advanced tools that enable faster decision-making and deeper insights through automation and augmentation. Generative Large Language Models (LLMs) are at the forefront of this shift, offering new possibilities for education. This thesis investigates the integration LLMs to automate performance assessments in business simulation games, in collaboration with a business simulation service provider. This thesis aims to provide on-time personalized feedback to users, supporting experiential learning in digital education. The study employs a three-stage Design Science Research (DSR) methodology, with iterative insights guiding subsequent stages. Early stages revealed limitations and optimization opportunities, including rule-based data preprocessing that reduced token usage by 76% and lowered deployment costs. In the final stage, the artifact achieved 99.25% accuracy in summarizing company KPIs. Explainable errors, false positives, and hallucination in text outputs highlighted the need for further iteration both in the artifact development and evaluation framework. To address these errors, the thesis proposes the addition of hallucination as a distinct error category to existing evaluation frameworks, a critical measurement for generative language model evaluation. Although the artifact does not replace tutors, it has strong potential to enhance feedback efficiency and business intelligence dashboards when guided by educators. Aligning the tool with responsible AI principles ensures scalability, transparency, and cost-effectiveness. This work advances educational technology and neural data-to-text research by demonstrating the viability of LLMs for automated, explainable assessments in augmented analytics.

Description

Supervisor

Salmi, Mika

Thesis advisor

Lainema, Timo

Keywords

business assessment, NLP, LLM, business intelligence, augmented analytics, ERP performance

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202501221601

Collections

[dipl] Insinööritieteiden korkeakoulu / ENG