Transfer-Plausible Acoustics for Augmented Reality

Abstract

Augmented reality (AR) telepresence systems aim to present visual and auditory "holograms" of conversation partners via head-mounted displays and transparent headphones. These systems require binaural audio that adapts not only to the user's orientation and position but also to their acoustic environment. Many fundamental technologies for such real-time, binaural auralization systems have been developed over the years. These virtual acoustic systems were often tested in direct comparison to a high-quality reference rendering, so the implied objective for the system's development was often indistinguishability from a reference. However, differences were usually audible in such tests, at least for non-ideal, practically relevant systems. When developing future AR systems, two questions arise: "Why exactly do such discrepancies occur?" and "What are meaningful objectives and evaluation paradigms other than indistinguishability from a reference?" First, finding reasons for discrepancies involves a detailed understanding of specific rendering methods, underlying models, and their violations. Two fundamental properties of a parametric spatial room impulse response processing technique are studied as examples.

Second, as an objective that leads to meaningful AR evaluation paradigms, one option is to assess if auditory illusions are evoked, i.e., whether a listener believes a virtual sound source to be real. This work introduces the transfer-plausibility paradigm, which evaluates if a virtual source creates an auditory illusion, even in the presence of other, real sound sources.

In summary, Publication I and Publication II discuss fundamental properties of spatial room impulse response processing techniques: Publication I shows how direction-of-arrival estimation based on the pseudo intensity vector depends on anisotropy in the late reverberation. Publication II investigates how perceptual roughness can occur in spatial room impulse response rendering based on broadband directional assignment.

Publication III and Publication IV deal with problems more closely related to AR. Publication III proposes an approach for blind spatial room impulse response estimation using a pseudo-reference signal. Publication IV demonstrates auditory modeling-based quantification of impairments caused by so-called transparent headphones used for AR.

Publication V and Publication VI introduce the notion of transfer-plausibility and compare it against other paradigms. The results suggest that even non-ideal virtual acoustic renderings are comparable in transfer-plausibility tests. Publication VII presents an experiment about the inability for self-localization using position-dependent room acoustic differences. The thesis concludes by presenting opportunities for future transfer-plausibility tests and a proposed model for describing differences in experimental paradigms by their sensitivity to auditory similarity, context, and artifacts.