Hybird prompt optimization without fine-tuning: Enhancing information extraction and translation in vehicle maintenance records
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Electrical Engineering |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
76
Series
Abstract
Unstructured, multilingual maintenance logs hamper automated fault analysis, yet fully fine-tuning large language models (LLMs) is prohibitively costly. We therefore adapt a 70 B-parameter LLaMA-33 model without gradient updates using a two-step Hybrid Prompt Optimization (HPO) scheme that merges expert-written instructions with DSPy’s automatic prompt search. The approach is trained and tested on 1000 real, Volvo Trucks records while running only on CPU. For evaluation we draw 10 disjoint test sets of 50 records each; scores reported here are the mean of those ten runs. Relative to a zero-shot baseline, manual prompts raise structured-field extraction accuracy by 23 %; DSPy adds 27 %; and HPO supplies a further 5 %, yielding a cumulative gain of ≈ 60 %. For full-text translation the same steps deliver 55 %, +2 %, and +2 %, respectively. Paired two-tailed t-tests across the 10 × 50 predictions confirm that all improvements remain significant at α = 0.05. Under the best prompt, the lightweight LLaMA-33-70B reaches 92 % of GPT-4o’s extraction quality and slightly surpasses it in translation—yet consumes only a fraction of the compute budget. These findings show that prompt engineering alone can unlock near–state-of-the-art multilingual extraction and translation, delivering a low-cost alternative to full model fine-tuning for industrial after-sales data.Description
Supervisor
Zhou, QuanThesis advisor
Hedman, AndersLi, Haibo