Neural Projection Mapping of Textures

Thumbnail Image

Files

master_Horesovsky_Jan_2021.pdf (165.71 MB)

URL

Journal Title

Journal ISSN

Volume Title

Perustieteiden korkeakoulu | Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2021-06-14

Department

Major/Subject

Computer Science

Mcode

SCI3042

Degree programme

Master’s Programme in Computer, Communication and Information Sciences

Language

en

Pages

81+15

Series

Abstract

Projection mapping is a widespread reality augmentation technique used by artists all over the world. It can be done automatically using a projector and a camera which captures the appearance of a projection in a scene. This appearance is then matched with a desired appearance pixel by pixel. However, this approach is limited by projector brightness when some pixels cannot be made as bright (or dark) as would be required to match the two appearances. In this thesis, we propose a method which focuses on projecting textures and overcomes this problem to a certain extent. Simply put, textures are images with repetitive structure such as pebble beach, tree bark or brick wall. Nowadays, it is possible to automatically generate new examples of a given texture which look the same, but their pixel values may be radically different. This problem is called texture synthesis. Instead of per-pixel matching, we thus use an existing synthesis technique based on a neural network to ensure that the actual appearance of a projection is a realization of the same texture as the desired appearance. As a result, our method is more flexible and thus able to reproduce a larger variety of appearances for a given scene. We guide the reader through three experiments in which we evaluate our method in a software simulation, both on simple flat surfaces and arbitrary 3D scenes. We highlight cases where our method outperforms conventional projection mapping techniques, discuss its limitations and outline steps which are needed to achieve real world deployment.

Description

Supervisor

Lehtinen, Jaakko

Thesis advisor

Lehtinen, Jaakko

Keywords

projection mapping, texture synthesis, convolutional neural networks, optimization, light transport

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202106207443

Collections

[dipl] Perustieteiden korkeakoulu / SCI