A Proactive Interface for Image Retrieval

No Thumbnail Available
Journal Title
Journal ISSN
Volume Title
Helsinki University of Technology | Diplomityö
Checking the digitized thesis and permission for publishing
Instructions for the author
Degree programme
xxx + xx
This thesis studies interfaces for browsing and searching for images. A novel gaze-based interface was developed which attempts to tailor the set of available choices according to the interest of the user. The system is integrated with PicSOM, a content-based image retrieval engine and is able to interact with various eye tracking devices. It can be used in online exploration of large image databases. Users interact with the system through a zooming interface inspired by the concept of infinite-desktops and Dasher, a tool for predictive text entry. The system computes on-line predictions of relevance of images based on implicit feedback, and when the user zooms in, the images predicted to be the most relevant are brought out. The key novelty is that the relevance feedback is inferred from implicit cues obtained in real-time from the gaze pattern, using an estimator learned during a separate training phase. It is found that there is sufficient amount of information in the gaze patterns to make the estimated relevance feedback a viable choice to complement or ultimately even replace explicit feedback by pointing-and-clicking, although the accuracy is not as good as with explicit feedback. The reliability of the relevance prediction is evaluated first as a stand-alone module, then in integration with the full system. For this purpose we carried out eye tracking experiments with test subjects using our software. The thesis describes the design and implementation of the image navigation system, and an evaluation of its performance. In addition, alternative approaches were explored. A simpler variant of the interface uses clicking or explicit selection with eye movements as feedback. An interface using a static collage-view has also been developed.
Kaski, Samuel
Thesis advisor
Klami, Arto
eye movements, proactive interfaces, image retrieval, implicit feedback