Graphical User Interface Auto-Completion with Element Constraints
Loading...
URL
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
2020-10-20
Department
Major/Subject
Human-Computer Interaction and Design
Mcode
SCI3020
Degree programme
Master's Programme in ICT Innovation
Language
en
Pages
99 + 6
Series
Abstract
Graphical User Interfaces (GUIs) are often the main touchpoint between a digital product and its users. Consistency is a key usability factor of GUIs that dictates reusing layout patterns throughout the designs of a single system. While visual styles can be easily specified across designs, spatial patterns of related elements often emerge in the creation process. Although these patterns are encoded in existing layouts, a large number is difficult to consider manually. This thesis investigates methods for a design assistance tool that suggests completions for a partial layout. Instead of generating arbitrary completions, we allow the designer to control the next element type and dimensions. The goal of the methods is then to predict the placement of this new element according to the layout patterns of reference designs. Two recently proposed methods for layout completion were evaluated, a Graph Neural Network (GNN) and a Transformer model, as well as a novel approach that leverages a sequence alignment algorithm and a nearest neighbor search (kNN). The methods were tested on handcrafted data sets with explicit layout patterns, as well as larger sets of diverse mobile layouts that lack consistent patterns. The implementation of the GNN mostly fails to predict high-quality results. The transformer model captures general layout structures and works reasonably well for spatial compositions that are close to the training data. The kNN approach achieves the best scores overall. The results suggest that leveraging data sets explicitly via instance-based learning algorithms can outperform neural network approaches for layout design problems. As such, this thesis contributes to establishing smarter design tools for professional designers that increase consistency in the design process.Description
Supervisor
Oulasvirta, AnttiThesis advisor
Leiva, LuisDu, Jingzhou
Keywords
user interfaces, machine learning, layouts, human-computer interaction, design, neural networks