A Proactive Interface for Image Retrieval

No Thumbnail Available

URL

Journal Title

Journal ISSN

Volume Title

Helsinki University of Technology | Diplomityö
Checking the digitized thesis and permission for publishing
Instructions for the author

Authors

Date

2009

Department

Tietotekniikan laitos

Major/Subject

Informaatiotekniikka

Mcode

T-61

Degree programme

Language

en

Pages

xxx + xx

Series

Abstract

This thesis studies interfaces for browsing and searching for images. A novel gaze-based interface was developed which attempts to tailor the set of available choices according to the interest of the user. The system is integrated with PicSOM, a content-based image retrieval engine and is able to interact with various eye tracking devices. It can be used in online exploration of large image databases. Users interact with the system through a zooming interface inspired by the concept of infinite-desktops and Dasher, a tool for predictive text entry. The system computes on-line predictions of relevance of images based on implicit feedback, and when the user zooms in, the images predicted to be the most relevant are brought out. The key novelty is that the relevance feedback is inferred from implicit cues obtained in real-time from the gaze pattern, using an estimator learned during a separate training phase. It is found that there is sufficient amount of information in the gaze patterns to make the estimated relevance feedback a viable choice to complement or ultimately even replace explicit feedback by pointing-and-clicking, although the accuracy is not as good as with explicit feedback. The reliability of the relevance prediction is evaluated first as a stand-alone module, then in integration with the full system. For this purpose we carried out eye tracking experiments with test subjects using our software. The thesis describes the design and implementation of the image navigation system, and an evaluation of its performance. In addition, alternative approaches were explored. A simpler variant of the interface uses clicking or explicit selection with eye movements as feedback. An interface using a static collage-view has also been developed.

Description

Supervisor

Kaski, Samuel

Thesis advisor

Klami, Arto

Keywords

eye movements, proactive interfaces, image retrieval, implicit feedback

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-2020120555467

Collections

[dipl] Teknillinen korkeakoulu / TKK