Dual-model augmentation for compositional semantic parsing via recursive log-likelihood guidance

Abstract

Compositional generalization remains a significant challenge for neural semantic parsers, particularly in low-resource settings. Data augmentation is a powerful method to address this by improving downstream task performance through expanded training diversity. While Large Language Models (LLMs) offer a promising avenue for such augmentation, their tendency to "hallucinate" invalid structures limits their utility in domains requiring strict syntactic adherence, such as the SMCalFlow dataset.

This thesis proposes a Dual-Model Augmentation Cycle to resolve the trade-off between generative diversity and structural correctness. We decouple generation, assigned to a Large Language Model (Gemini), from verification, assigned to a fine-tuned small model (T5). Leveraging the small model as a likelihood estimate of the training distribution, we implement a "Judger" that evaluates synthetic examples using normalized log-likelihood scores. Low-probability sequences trigger an automated self-correction loop, where the Generator refines the output based on targeted, token-level feedback.

We evaluate this framework on SMCalFlow, Overnight, and GeoQuery. Results demonstrate that our method significantly outperforms strong heuristic and grammar-induction baselines. On SMCalFlow (32-shot), our approach improves exact match accuracy from 37.0% to 52.0%,surpassing GECA (44.0\%) and QCFG (45.0%). Furthermore, in the ultra-low-resource Overnight domain, our method prevents negative transfer, improving accuracy to 19.0% where heuristic methods degraded performance. Distributional analysis confirms that the Judger effectively aligns the synthetic data with the natural data distribution. These findings suggest that small, specialized models act as indispensable critics for large foundation models, unlocking robust compositional generalization.