Distributed neural signatures of natural audiovisual speech and music in the human auditory cortex

 |  Login

Show simple item record

dc.contributor Aalto-yliopisto fi
dc.contributor Aalto University en
dc.contributor.author Salmi, Juha
dc.contributor.author Koistinen, Olli-Pekka
dc.contributor.author Glerean, Enrico
dc.contributor.author Jylänki, Pasi
dc.contributor.author Vehtari, Aki
dc.contributor.author Jääskeläinen, Iiro
dc.contributor.author Mäkelä, Sasu
dc.contributor.author Nummenmaa, Lauri
dc.contributor.author Nummi-Kuisma, Katarina
dc.contributor.author Nummi, Ilari
dc.contributor.author Sams, Mikko
dc.date.accessioned 2017-05-11T09:10:50Z
dc.date.available 2017-05-11T09:10:50Z
dc.date.issued 2017-08-15
dc.identifier.citation Salmi , J , Koistinen , O-P , Glerean , E , Jylänki , P , Vehtari , A , Jääskeläinen , I , Mäkelä , S , Nummenmaa , L , Nummi-Kuisma , K , Nummi , I & Sams , M 2016 , ' Distributed neural signatures of natural audiovisual speech and music in the human auditory cortex ' NeuroImage . DOI: 10.1016/j.neuroimage.2016.12.005 en
dc.identifier.issn 1053-8119
dc.identifier.issn 1095-9572
dc.identifier.other PURE UUID: e7bf4c72-6f7e-4b48-9404-d2d179724acf
dc.identifier.other PURE ITEMURL: https://research.aalto.fi/en/publications/distributed-neural-signatures-of-natural-audiovisual-speech-and-music-in-the-human-auditory-cortex(e7bf4c72-6f7e-4b48-9404-d2d179724acf).html
dc.identifier.other PURE LINK: https://doi.org/10.1016/j.neuroimage.2016.12.005
dc.identifier.other PURE FILEURL: https://research.aalto.fi/files/10289747/1_s2.0_S1053811916307121_main.pdf
dc.identifier.uri https://aaltodoc.aalto.fi/handle/123456789/25931
dc.description.abstract During a conversation or when listening to music, auditory and visual information are combined automatically into audiovisual objects. However, it is still poorly understood how specific type of visual information shapes neural processing of sounds in lifelike stimulus environments. Here we applied multi-voxel pattern analysis to investigate how naturally matching visual input modulates supratemporal cortex activity during processing of naturalistic acoustic speech, singing and instrumental music. Bayesian logistic regression classifiers with sparsity-promoting priors were trained to predict whether the stimulus was audiovisual or auditory, and whether it contained piano playing, speech, or singing. The predictive performances of the classifiers were tested by leaving one participant at a time for testing and training the model using the remaining 15 participants. The signature patterns associated with unimodal auditory stimuli encompassed distributed locations mostly in the middle and superior temporal gyrus(STG/MTG). A pattern regression analysis, based on a continuous acoustic model, revealed that activity in some of these MTG and STG areas were associated with acoustic features present in speech and music stimuli. Concurrent visual stimulus modulated activity in bilateral MTG (speech), lateral aspect of right anterior STG (singing), and bilateral parietal opercular cortex (piano). Our results suggest that specific supratemporal brain areas are involved in processing complex natural speech, singing, and piano playing, and other brain areas located in anterior (facial speech) and posterior (music-related hand actions) supratemporal cortex are influenced by related visual information. Those anterior and posterior supratemporal areas have been linked to stimulus identification and sensory-motor integration, respectively. en
dc.format.mimetype application/pdf
dc.language.iso en en
dc.relation.ispartofseries NeuroImage en
dc.rights openAccess en
dc.subject.other Neurology en
dc.subject.other Cognitive Neuroscience en
dc.subject.other 3112 Neurosciences en
dc.title Distributed neural signatures of natural audiovisual speech and music in the human auditory cortex en
dc.type A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä fi
dc.description.version Peer reviewed en
dc.contributor.department Department of Neuroscience and Biomedical Engineering
dc.contributor.department University of the Arts Helsinki
dc.contributor.department Aalto University
dc.contributor.department Department of Computer Science en
dc.subject.keyword Audiovisual
dc.subject.keyword fMRI
dc.subject.keyword Multi-voxel pattern analysis
dc.subject.keyword Music
dc.subject.keyword Speech
dc.subject.keyword Neurology
dc.subject.keyword Cognitive Neuroscience
dc.subject.keyword 3112 Neurosciences
dc.identifier.urn URN:NBN:fi:aalto-201705114306
dc.identifier.doi 10.1016/j.neuroimage.2016.12.005
dc.date.embargo info:eu-repo/date/embargoEnd/2017-12-06

Files in this item

Files Size Format View

There are no files associated with this item.

This item appears in the following Collection(s)

Show simple item record

Search archive

Advanced Search

article-iconSubmit a publication


My Account