Browsing by Author "Oulasvirta, Antti, Prof., Aalto University, Department of Communications and Networking, Finland"

Filter results by typing the first few letters
Now showing 1 - 3 of 3
  • Assignment Problems for Optimizing Text Input
    (2018) Feit, Anna Maria
     School of Electrical Engineering | Doctoral dissertation (article-based)
    Text input methods are an integral part of our daily interaction with digital devices. However, their design poses a complex problem: for any method, we must decide which input action (a button press, a hand gesture, etc.) produces which symbol (e.g., a character or word). With only 26 symbols and input actions, there are already more than 10^26 distinct solutions, making it impossible to find the best one through manual design. Prior work has shown that we can use optimization methods to search such large design spaces efficiently and automatically find the best solution for a given task and objective. However, work in this domain has been limited mostly to the performance optimization of keyboards. The Ph.D. thesis advances the field of text-entry optimization by enlarging the space of optimizable text-input methods and proposing new criteria for assessing their optimality. Firstly, the design problem is formulated as an assignment problem for integer programming. This enables the use of standard mathematical solvers and algorithms for efficiently finding good solutions. Then, objective functions are developed, for assessing their optimality with respect to motor performance, ergonomics, and learnability. The corresponding models extend beyond interaction with soft keyboards, to consider multi-finger input, novel sensors, and alternative form factors. In addition, the thesis illustrates how to formulate models from prior work in terms of an assignment problem, providing a coherent theoretical basis for text-entry optimization. The proposed objectives are applied in the optimization of three assignment problems: text input with multi-finger gestures in mid-air, text input on a long piano keyboard, and -- for a contribution to the official French keyboard standard -- input of special characters via a physical keyboard. Combining the proposed models offers a multi-objective optimization approach able to capture the complex cognitive and motor processes during typing. Finally, the dissertation discusses future work that is needed to solve the long-standing problem of finding the optimal layout for physical keyboards, in light of empirical evidence that prior models are insufficient to respond to the diverse typing strategies people employ with modern keyboards. The thesis advances the state of the art in text-entry optimization by proposing novel objective functions that quantify the performance, ergonomics and learnabilityof a text input method. The objectives presented are formulated as assignment problems, which can be solved with integer programming via standard mathematical solvers or heuristic algorithms. While the work focused on text input, the assignment problem can be used to model other design problems in HCI (e.g., how best to assign commands to UI controls or distribute UI elements across several devices), for which the same problem formulations, optimization techniques, and even models could be applied.
  • Collaborative Systems for Design Inspiration
    (2020) Koch, Janin
     School of Electrical Engineering | Doctoral dissertation (article-based)
    Inspiration is a crucial activity in design and innovation, in which potential desirable solutions are explored and refined to later provide directions and inspiration in later stages of design. Designers use a plethora of inspirational methods and tools. Among them are mood boards, a visual collage of pictures, text, and objects, that is usually created collaboratively in e.g. fashion design, architecture, and marketing. Mood boards help designers identify, select, and curate visually inspiring content, to express their existing ideas but also to inspire new ideas through their combination. As mood board design becomes increasingly digital, the availability and variety of online material presents ever greater opportunities to assist designers. However, it also poses new challenges. Through interviews with professional designers we identified three of them in particular: 1) turning tacit ideas into expressible search terms, 2) synthesizing and reflecting on visual material, and 3) finding external inspiration. While existing methods for visual inspiration hint at a great potential to support conceptual innovations, the computational support to address these challenges remains lacking. My work contributes knowledge, interaction techniques, and co-creative algorithms to assist users with these challenges. First, it introduces collaborative systems that enrich images with semantic information, to help designers express vague, visual ideas and translate them into usable search terms. Second, to support visual reflection, this thesis introduces multiple levels of semantic abstraction of visual material, to inspire designers to find higher-level concepts in their own work. Third, collaboration is an integral part of physical mood board practice, yet digital mood boards are often crafted at an individual level, which deprives designers of many opportunities to challenge and expand their ideas. An artificial agent was developed that creates mood boards jointly with a designer, based on a cooperative contextual bandit algorithm. This approach conferred it the ability to make its own decisions about whether to explore or exploit the visual contents of the current mood board, and our participants, all professional designers, genuinely valued its contributions. Thanks to a grounding-based interaction approach, it had the ability to justify its contributions and to inquire about sudden changes in the designer's choices. That resulted in a system that was perceived as a contributing agent, rather than merely a tool. Finally, beyond mood board creation with individual designers, the developed collaborative systems also contributed to creative collaborations between human designers. Within these collaborations, artificial agents played a role complementary to that of designers, and were appreciated in particular when ideas were sparse, when designers felt ''stuck',' or had trouble expressing their ideas. My work highlights the immense potential of intelligent collaborative systems for inspiration-seeking and creative processes, and opens new ways to assist designers in the era of digital ideation.
  • Optimization approaches for graphical layout design
    (2022) Shiripour, Morteza
     School of Electrical Engineering | Doctoral dissertation (article-based)
    The graphical user interface (GUI) is the most frequently used method of communicating with computers. These interfaces employ graphical components rather than either pure command lines or natural language for the purpose of interaction. A well-designed GUI is critical for improving the interaction's efficacy and utility. Currently, the process of developing GUIs is primarily a manual one that is time-consuming and difficult, and the final product depends greatly on designer competence. Furthermore, the large design space of alternative designs complicates manual design. For example, the number of distinct positions possible for placing five elements on a common canvas of 1024 X 768 pixels (for simplicity, suppose the canvas is divided into 32 X 24 pixels) is approximately 2.6e+14. Whereas analyzing these solutions manually would demand large amounts of time and effort, the thesis examines the alternative of developing computational methods (e.g., mathematical models, deep learning, and heuristic algorithms) and their integration into the design process, for purposes of addressing the inherent complexity of manual design. Several models and algorithms are proposed that automate aspects of the GUI-design process.In one method, a grid operator is presented to a non-dominated-sorting genetic algorithm for graphical layout problems. These grid operators create some vertical and horizontal lines using the fixed elements. The remaining unfixed elements are then inserted between these lines. This procedure ultimately results in the satisfaction of overlap-related constraints and better element alignment, thus fulfilling one of the main objectives of a designer organizing layout elements. The second method developed, which involves a deep neural network estimating Web pages' visual appeal, could serve objective-function approximation that informs evaluating a given design's aesthetics. Results from a deep neural network developed in the doctoral project attest to the ensuing model's ability to predict the ratings that people with diverse demographic backgrounds would give the Web page shown. For the last method, empirical evaluation of a novel integer-programming formulation for menu systems was designed and implemented. The optimized design exhibited increased efficiency: user interactions were around 25% quicker than with non-optimized commercial designs. Along with the project's contribution, the thesis discusses the future work required for solving GUI problems from both a theoretical and a practical standpoint.