Enhancing 3D Asset Retrieval with Semantic Search
Loading...
URL
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
2024-05-20
Department
Major/Subject
Computer Science
Mcode
SCI3042
Degree programme
Master’s Programme in Computer, Communication and Information Sciences
Language
en
Pages
45
Series
Abstract
Semantic search goes beyond exact keyword matching and instead focuses on the comprehension of search queries' purpose and the surrounding context to deliver relevant search results. In this work, we focus on integrating hybrid search, which uses semantic search and traditional lexical search together, in a 3D asset retrieval pipeline of a game engine. We use the embedding of the 3D asset's rendering, title, and description to fill our vector search index for the semantic search part. This allows our search pipeline to use the visual embedding of the asset for search in case of missing or inaccurate metadata. We rely on the recent advances in Deep Learning that have led to the training of capable multimodal models that can embed various data modalities in the same embedding space. In this work, we mainly use vision-language models that embed images and text in the same embedding space. We test our proposed hybrid search method on a dataset of 5000+ assets provided and labeled by HypeHype. We demonstrate the benefits of our method compared to a pure lexical search method through an A/B test and report the improvements in click-through rate and average relevant index of the search results.Description
Supervisor
Hämäläinen, PerttuThesis advisor
Relas, AskoKeywords
information retrieval, semantic search, hybrid search, multimodal deep learning, embeddings, vector search