Dual-model augmentation for compositional semantic parsing via recursive log-likelihood guidance

Thumbnail Image

Files

master_Sherafati_Mohammad_2026.pdf (10.37 MB)

URL

Journal Title

Journal ISSN

Volume Title

School of Science | Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2025-12-26

Department

Major/Subject

Machine Learning, Data Science and Artificial Intelligence

Mcode

Degree programme

Master's Programme in Life Science Technologies
Master's Programme in Life Science Technologies
Master's Programme in Life Science Technologies

Language

en

Pages

78

Series

Abstract

Compositional generalization remains a significant challenge for neural semantic parsers, particularly in low-resource settings. Data augmentation is a powerful method to address this by improving downstream task performance through expanded training diversity. While Large Language Models (LLMs) offer a promising avenue for such augmentation, their tendency to "hallucinate" invalid structures limits their utility in domains requiring strict syntactic adherence, such as the SMCalFlow dataset. This thesis proposes a Dual-Model Augmentation Cycle to resolve the trade-off between generative diversity and structural correctness. We decouple generation, assigned to a Large Language Model (Gemini), from verification, assigned to a fine-tuned small model (T5). Leveraging the small model as a likelihood estimate of the training distribution, we implement a "Judger" that evaluates synthetic examples using normalized log-likelihood scores. Low-probability sequences trigger an automated self-correction loop, where the Generator refines the output based on targeted, token-level feedback. We evaluate this framework on SMCalFlow, Overnight, and GeoQuery. Results demonstrate that our method significantly outperforms strong heuristic and grammar-induction baselines. On SMCalFlow (32-shot), our approach improves exact match accuracy from 37.0% to 52.0%,surpassing GECA (44.0\%) and QCFG (45.0%). Furthermore, in the ultra-low-resource Overnight domain, our method prevents negative transfer, improving accuracy to 19.0% where heuristic methods degraded performance. Distributional analysis confirms that the Judger effectively aligns the synthetic data with the natural data distribution. These findings suggest that small, specialized models act as indispensable critics for large foundation models, unlocking robust compositional generalization.

Description

Supervisor

Marttinen, Pekka

Thesis advisor

Spilsbury, Sam

Keywords

Large Language Models (LLMs), semantic parsing, data augmentation, compositional generalization, self-correction, low-resource NLP, small model as judger

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202601191437

Collections

[dipl] Perustieteiden korkeakoulu / SCI