aalto1 untyped-item.component.html
Transfer-Plausible Acoustics for Augmented Reality
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Electrical Engineering |
Doctoral thesis (article-based)
| Defence date: 2024-07-26
Electronic archive copy is available via Aalto Thesis Database.
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
89 + app. 101
Series
Aalto University publication series DOCTORAL THESES, 139/2024
Abstract
Augmented reality (AR) telepresence systems aim to present visual and auditory "holograms" of conversation partners via head-mounted displays and transparent headphones. These systems require binaural audio that adapts not only to the user's orientation and position but also to their acoustic environment. Many fundamental technologies for such real-time, binaural auralization systems have been developed over the years. These virtual acoustic systems were often tested in direct comparison to a high-quality reference rendering, so the implied objective for the system's development was often indistinguishability from a reference. However, differences were usually audible in such tests, at least for non-ideal, practically relevant systems. When developing future AR systems, two questions arise: "Why exactly do such discrepancies occur?" and "What are meaningful objectives and evaluation paradigms other than indistinguishability from a reference?" First, finding reasons for discrepancies involves a detailed understanding of specific rendering methods, underlying models, and their violations. Two fundamental properties of a parametric spatial room impulse response processing technique are studied as examples.
Second, as an objective that leads to meaningful AR evaluation paradigms, one option is to assess if auditory illusions are evoked, i.e., whether a listener believes a virtual sound source to be real. This work introduces the transfer-plausibility paradigm, which evaluates if a virtual source creates an auditory illusion, even in the presence of other, real sound sources.
In summary, Publication I and Publication II discuss fundamental properties of spatial room impulse response processing techniques: Publication I shows how direction-of-arrival estimation based on the pseudo intensity vector depends on anisotropy in the late reverberation. Publication II investigates how perceptual roughness can occur in spatial room impulse response rendering based on broadband directional assignment.
Publication III and Publication IV deal with problems more closely related to AR. Publication III proposes an approach for blind spatial room impulse response estimation using a pseudo-reference signal. Publication IV demonstrates auditory modeling-based quantification of impairments caused by so-called transparent headphones used for AR.
Publication V and Publication VI introduce the notion of transfer-plausibility and compare it against other paradigms. The results suggest that even non-ideal virtual acoustic renderings are comparable in transfer-plausibility tests. Publication VII presents an experiment about the inability for self-localization using position-dependent room acoustic differences. The thesis concludes by presenting opportunities for future transfer-plausibility tests and a proposed model for describing differences in experimental paradigms by their sensitivity to auditory similarity, context, and artifacts.
Description
Supervising professor
Lokki, Tapio, Prof., Aalto University, Department of Information and Communications Engineering, FinlandThesis advisor
Schlecht, Sebastian J., Prof., Friedrich-Alexander Universität Erlangen-Nürnberg, GermanyRobinson, Philip, Dr., Reality Labs Research, USA
Other note
Parts
- [Publication 1]: Meyer-Kahlen, N., Schlecht, S.J. Directional distribution of the pseudo intensity vector in anisotropic late reverberation. The Journal of the Acoustical Society of America, 155(2), 1515–1526, February 2024.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202403062615DOI: 10.1121/10.0024960 View at publisher
- [Publication 2]: Meyer-Kahlen, N., Schlecht, S.J., Lokki, T. Perceptual roughness of spatially assigned sparse noise for rendering reverberation. The Journal of the Acoustical Society of America, 150(5), 3521–3531, November 2021.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202204062741DOI: 10.1121/10.0007048 View at publisher
- [Publication 3]: Meyer-Kahlen, N., Schlecht, S.J. Blind directional room impulse response parameterization from relative transfer functions. International Workshop on Acoustic Signal Enhancement (IWAENC), Bamberg, Germany, September 2022.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202301181211DOI: 10.1109/IWAENC53105.2022.9914706 View at publisher
- [Publication 4]: Lladó, P., McKenzie, T., Meyer-Kahlen, N., Schlecht, S.J. Predicting perceptual transparency of head-worn devices. The Journal of the Audio Engineering Society, 70 (7), 585–600, July/August 2022.
Full text in Acris/Aaltodoc: https://urn.fi/URN:NBN:fi:aalto-202208174886DOI: 10.17743/jaes.2022.0024 View at publisher
- [Publication 5]: Wirler, S., Meyer-Kahlen, N, Schlecht, S.J. Towards transfer-plausibility for evaluating mixed reality audio in complex scenes. In AES International Conference on Audio for Virtual and Augmented Reality, Remote, August 2020.
- [Publication 6]: Meyer-Kahlen, N, Schlecht, S.J., Amengual Garí, S., Lokki, T. Testing auditory illusions in augmented reality: plausibility, transfer-plausibility and authenticity. The Journal of the Audio Engineering Society, Submitted, 2024.
- [Publication 7]: Meyer-Kahlen, N., Schlecht, S.J., Lokki, T. Clearly audible differences rarely reveal where you are in a room. The Journal of the Acoustical Society of America, 152 (2), 877–887, August 2022.