Reducing noisy annotations for depression estimation from facial images

Depression has been considered the most dominant mental disorder over the past few years. To help clinicians effectively and efficiently estimate the severity scale of depression, various automated systems based on deep learning have been proposed. To estimate the severity of depression, i.e., the depression severity score (Beck Depression Inventory-II), various deep architectures have been designed to perform regression using the Euclidean loss. However, they do not consider the label distribution, and they do not learn the relationships between the facial images and BDI-II scores, which can be resulting in the noisy labeling for automatic depression estimation (ADE). To mitigate this problem, we propose an automated deep architecture, namely the self-adaptation network (SAN), to improve this uncertain labeling for ADE. Specifically, the architecture consists of four modules: (1) ResNet-18 and ResNet-50 are adopted in the deep feature extraction module (DFEM) to extract informative deep features; (2) a self-attention module (SAM) is adopted to learn the weights from the mini-batch; (3) a square ranking regularization module (SRRM) to create high partitions and low partitions is proposed; and (4) a re-label module (RM) is used to re-label the uncertain annotations for ADE in the low partitions. We conduct extensive experiments on depression databases (i.e., AVEC2013 and AVEC2014) and obtain a performance comparable to the performances of other ADE methods in assessing the severity of depression. More importantly, the proposed method can learn valuable depression patterns from facial videos and obtain a performance comparable to the performances of other methods for depression recognition.

Description

| openaire: EC/H2020/101016775/EU//INTERVENE Funding Information: This work is supported by the Shaanxi Provincial Social Science Foundation (grant 2021K015), the Shaanxi Provincial Natural Science Foundation (grant 2021JQ-824), the Shaanxi Provincial Natural Science Foundation (grant 2022JM-380), the Special Construction Fund for Key Disciplines of Shaanxi Provincial Higher Education, the Scientific Research Program Funded by Shaanxi Provincial Education Department (Program No. 19JS028), and the Scientific Research Program Funded by Shaanxi Provincial Education Department (Program No. 20JG030). This work was also supported by the Academy of Finland (grants 336033, 315896), Business Finland (grant 884/31/2018), and EUH2020 (grant 101016775). Publisher Copyright: © 2022 The Author(s)

Keywords

Affective computing, Depression, Noisy labels, Self-adaptation network (SAN)

DOI

10.1016/j.neunet.2022.05.025

Citation

He, L, Tiwari, P, Lv, C, Wu, W S & Guo, L 2022, ' Reducing noisy annotations for depression estimation from facial images ', Neural Networks, vol. 153, pp. 120-129 . https://doi.org/10.1016/j.neunet.2022.05.025

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202209145595

Collections

[article-cris] Perustieteiden korkeakoulu / SCI

Show all metadata