LLMs’ morphological analyses of complex FST-generated Finnish words

Thumbnail Image

Access rights

openAccess
CC BY
publishedVersion

URL

Journal Title

Journal ISSN

Volume Title

A4 Artikkeli konferenssijulkaisussa
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)

Authors

Date

2024

Department

Department of Information and Communications Engineering
Tietojenkäsittelytieteen laito

Major/Subject

Mcode

Degree programme

Language

en

Pages

13

Series

CMCL 2024 - 13th Edition of the Workshop on Cognitive Modeling and Computational Linguistics, Proceedings of the Workshop, pp. 242-254

Abstract

Rule-based language processing systems have been overshadowed by neural systems in terms of utility, but it remains unclear whether neural NLP systems, in practice, learn the grammar rules that humans use. This work aims to shed light on the issue by evaluating state-of-the-art LLMs in a task of morphological analysis of complex Finnish noun forms. We generate the forms using an FST tool, and they are unlikely to have occurred in the training sets of the LLMs, therefore requiring morphological generalisation capacity. We find that GPT-4-turbo has some difficulties in the task while GPT-3.5turbo struggles and smaller models Llama2-70B and Poro-34B fail nearly completely.

Description

Publisher Copyright: ©2024 Association for Computational Linguistics.

Keywords

Other note

DOI

10.18653/v1/2024.cmcl-1.21

Citation

Moisio, A, Creutz, M & Kurimo, M 2024, LLMs’ morphological analyses of complex FST-generated Finnish words . in T Kuribayashi, G Rambelli, E Takmaz, P Wicke & Y Oseki (eds), CMCL 2024 - 13th Edition of the Workshop on Cognitive Modeling and Computational Linguistics, Proceedings of the Workshop . CMCL 2024 - 13th Edition of the Workshop on Cognitive Modeling and Computational Linguistics, Proceedings of the Workshop, Association for Computational Linguistics, pp. 242-254, Workshop on Cognitive Modeling and Computational Linguistics, Bangkok, Thailand, 15/08/2024 . https://doi.org/10.18653/v1/2024.cmcl-1.21

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202501151250

Collections

[article-cris] Palvelut / Services