Adaptive prompt elicitation for text-to-image generation

Abstract

Text-to-image generative models have rapidly advanced in producing high-quality and diverse visual content. However, aligning generated outputs with user intent remains challenging, particularly when users have a specific vision but struggle to articulate it precisely in natural language, leading to tedious trial-and-error refinement. This thesis presents Adaptive Prompt Elicitation, a principled method designed to bridge the gap between user intent and effective prompts for text-to-image generation. Building on an information-theoretic foundation, the approach contributes two key innovations: (1) a prompt elicitation framework that interactively elicits user intent through informative queries and automatically generates optimized prompts; (2) an efficient query selection strategy that maximizes expected alignment gain based on Bayesian experimental design. We evaluate the method on two established benchmarks for complex text-to-image generation—IDEA-Bench and DesignBench—comparing it against automatic prompt optimization, in-context query generation, and non-optimized baselines. Results demonstrate that Adaptive Prompt Elicitation achieves superior effectiveness and efficiency in aligning generated images with target intent while maintaining consistent performance across diverse scenarios. Overall, this work contributes a systematic methodology for interactive prompt optimization, advancing the usability, interpretability, and reliability of text-to-image generation systems.