Captions and Subtitles of video courses in Aalto - An LLM-powered workflow for transcription and translation in domain-specific environments
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Science |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
2024-09-30
Department
Major/Subject
Human-Computer Interaction and Design
Mcode
SCI3020
Degree programme
Master's Programme in ICT Innovation
Language
en
Pages
49
Series
Abstract
The thesis investigates Aalto University's current practices for captioning and subtitling video courses. The results show how the current workflow is riddled with problems that hinder the usability of the software, aggravated by automated tools that produce low-quality results. These problems cause a widespread scarcity of captioned media on the platform, crippling the accessibility of educational content at Aalto University. To resolve these deficiencies the thesis proposes an alternative user interface with improved usability, implementing UI features to facilitate subtitle editing. Moreover, the solution implements an error checking and a suggestion system powered by LLM to further improve the users' efficiency during the manual correction of captions and subtitles. A more efficient and powerful transcription service that leverages the open-source WhisperX is also proposed. Subsequent evaluations show how the proposed solution is competitive compared to the current workflow adopted at Aalto. Results show how the automatic English captions have fewer errors and those errors are readily highlighted by the suggestion systems. The translated subtitles sound natural and have just a few mistakes that are easily caught by the LLM hints. In conclusion, this thesis sheds light on the fallacies of the current workflows. It proposes a solution, with the prospect that a possible full or partial implementation of the ideas suggested in this work will help make a more accessible educational experience at Aalto University.Description
Supervisor
Hellas, ArtoThesis advisor
Logacheva, EvanfiyaLeinonen, Juho
Keywords
trascription, translation, llm, ai, usability, UX, subtitles, caption, university, interface, error-correcting