Domain-aware deep learning for room acoustics: Parameter estimation, localization, and source separation

Abstract

Acoustics fundamentally shape how sound is produced, transmitted, and perceived, influencing both human experience and machine performance. While acoustic properties can in principle be measured in physical environments, such measurements are costly, impractical at scale, and unavailable in virtual or augmented environments. This makes computational approaches—especially domain-aware methods that integrate machine learning with knowledge of sound propagation and signal processing—essential for modeling, mitigating, and exploiting acoustics across applications in spatial audio rendering, scene analysis, and machine listening.

This thesis investigates deep learning methods for room acoustics and spatial audio tasks, with acoustics alternately treated as the prediction target, as an interfering factor, or as a source of supervisory structure. First, we address acoustic parameter estimation: from geometric representations of scenes, from measured room impulse responses, and at the level of entire two dimensional floorplans informed by calibration signals. These approaches contribute new feature representations, architectures, and multimodal formulations that make estimation more accurate and efficient. Second, we study robustness in sound event localization and detection by proposing Spatial Mixup, a data augmentation technique that modifies directional loudness patterns in ambisonic recordings to improve generalization. Finally, we introduce a weakly supervised framework for source separation of sounds produced by machines that leverages spatial location as a supervisory signal, enabling separation when isolated ground truth data is unavailable.

Together, these works demonstrate how domain-aware learning, which is grounded in both digital signal processing and physical knowledge of sound propagation, can advance the analysis and rendering of acoustics. They also highlight ongoing challenges, including limited dataset diversity, difficulties in generalization to real-world environments, and the need for architectures that explicitly capture acoustic structure. By combining DSP insights with flexible learning frameworks, this thesis contributes toward more robust, interpretable, and perceptually grounded machine listening systems.