Creating and evaluating embodied interactive experiences: case studies of full-body, sonic and tactile enaction.

 |  Login

Show simple item record

dc.contributor Aalto-yliopisto fi
dc.contributor Aalto University en
dc.contributor.advisor Takala, Tapio, Prof., Aalto University, Department of Computer Science, Finland
dc.contributor.author Pugliese, Roberto
dc.date.accessioned 2015-12-07T10:02:03Z
dc.date.available 2015-12-07T10:02:03Z
dc.date.issued 2015
dc.identifier.isbn 978-952-60-6554-0 (electronic)
dc.identifier.isbn 978-952-60-6553-3 (printed)
dc.identifier.issn 1799-4942 (electronic)
dc.identifier.issn 1799-4934 (printed)
dc.identifier.issn 1799-4934 (ISSN-L)
dc.identifier.uri https://aaltodoc.aalto.fi/handle/123456789/18962
dc.description.abstract This thesis contributes to the field of embodied and multimodal interaction by presenting the development of different original interactive systems. Using a constructive approach, a variety of real-time user interaction situations were designed and tested, two cases of human-virtual character bodily interaction, two interactive sonifications of trampoline jumping, collaborative interaction in mobile music performance and tangible and tactile interaction with virtual sounds. While diverse in terms of application, all the explored interaction techniques belong to the context of augmentation and are grounded in the theory of embodiment and strategies for natural human-computer interaction (HCI). The cases have been contextualized within the umbrella of enaction, a paradigm of cognitive science that addresses the user as an embodied agent situated in an environment and coupled to it through sensorimotor activity. This activity of sensing and action is studied through different modalities: auditory, tactile and visual and combinations of these. The designed applications aim at a natural interaction with the system, being full-body, tangible and spatially aware. Particularly sonic interaction has been explored in the context of music creation, sports and auditory display. These technology-mediated scenarios are evaluated in order to understand what the adopted interaction techniques can bring to the user experience, how they modify impressions and enjoyment. The publications also discuss the enabling technologies used for the development, including motion tracking and programmed hardware for the tactile-sonic interaction and sonic and tangible interaction. Results show that combining full-body interaction with auditory augmentation and sonic interaction can modify the perception, observed behavior and emotion during the experience. Using spatial interaction together with tangible interaction or tactile feedback provides for a multimodal experience of exploring a mixed reality environment where audio can be accessed and manipulated with natural interaction. Embodied and spatial interaction brings playfulness to a mobile music improvisation, shifting the focus of the experience from music-making towards movement-based gaming. Finally, two novel implementations of full-body interaction based on the enactive paradigm are presented. In these designed scenarios of enaction the participant is motion tracked and a virtual character rendered as a stick figure is displayed in front of her on a screen. Results from the user studies show how the involvement of the body is crucial in understanding the behavior of a virtual character or a digital representation of the self in a gaming scenario. en
dc.format.extent 87 + app. 73
dc.format.mimetype application/pdf en
dc.language.iso en en
dc.publisher Aalto University en
dc.publisher Aalto-yliopisto fi
dc.relation.ispartofseries Aalto University publication series DOCTORAL DISSERTATIONS en
dc.relation.ispartofseries 205/2015
dc.relation.haspart [Publication 1]: Roberto Pugliese, Klaus Lehtonen. A framework for motion based bodily enaction with virtual characters. In 11th International Conference on Intelligent Virtual Agents (IVA 2011), Reykjavik, Iceland, Lecture Notes in Computer Science, Volume 6895, pp. 162-168, September 2011. DOI: 10.1007/978-3-642-23974-8_18
dc.relation.haspart [Publication 2]: Roberto Pugliese, Archontis Politis, Tapio Takala. Spatial rendering of audio-tactile feedback for exploration and object interaction in virtual environments. In Proceedings of the 9th Sound and Music Computing Conference, Copenhagen, Denmark, pp. 241-248, July 2012.
dc.relation.haspart [Publication 3]: Roberto Pugliese, Koray Tahiroğlu, Callum Goddard, James Nesfield. A qualitative evaluation of augmented human-human interaction in mobile group improvisation. In Proceedings of the International Conference on New Interfaces for Musical Expression, University of Michigan, Ann Arbor, May 2012.
dc.relation.haspart [Publication 4]: Luca Turchet, Roberto Pugliese, Tapio Takala. Physically based sound synthesis and control of jumping sounds on an elastic trampoline. In Proceedings of ISon 2013, 4th Interactive Sonification Workshop, Fraunhofer IIS, Erlangen, Germany, pp. 87-94, December 2013.
dc.relation.haspart [Publication 5]: Roberto Pugliese, Archontis Politis, Tapio Takala. ATSI: augmented and tangible sonic interaction. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, Stanford, California, USA, ACM, pp. 97-104, January 2015. DOI: 10.1145/2677199.2680550
dc.relation.haspart [Publication 6]: Roberto Pugliese, Tapio Takala. Sonic trampoline: how audio feedback impacts the user’s experience of jumping. MultiMedia, IEEE, Volume 22, no. 1, pp. 74-79, January - March 2015. DOI: 10.1109/MMUL.2015.13
dc.relation.haspart [Publication 7]: Roberto Pugliese, Klaus Forger, Tapio Takala. Game experience when controlling a weak avatar in full-body enaction. In 15th International Conference on Intelligent Virtual Agents (IVA 2015), Delft, Netherlands, Lecture Notes in Computer Science, Volume 9238, pp. 418-431, August´2015. DOI: 10.1007/978-3-319-21996-7_45
dc.subject.other Acoustics en
dc.subject.other Computer science en
dc.title Creating and evaluating embodied interactive experiences: case studies of full-body, sonic and tactile enaction. en
dc.type G5 Artikkeliväitöskirja fi
dc.contributor.school Perustieteiden korkeakoulu fi
dc.contributor.school School of Science en
dc.contributor.department Tietotekniikan laitos fi
dc.contributor.department Department of Computer Science en
dc.subject.keyword embodied interaction en
dc.subject.keyword enaction en
dc.subject.keyword multimodal en
dc.subject.keyword sonic interaction en
dc.subject.keyword full-body en
dc.subject.keyword audio augmented reality en
dc.subject.keyword avatars en
dc.identifier.urn URN:ISBN:978-952-60-6554-0
dc.type.dcmitype text en
dc.type.ontasot Doctoral dissertation (article-based) en
dc.type.ontasot Väitöskirja (artikkeli) fi
dc.contributor.supervisor Takala, Tapio, Prof., Aalto University, Department of Computer Science, Finland
dc.opn Ruttkay, Zsófia, Assoc. Prof., Moholy-Nagy University of Art and Design, Hungary
dc.rev Raisamo, Roope, Prof., University of Tampere, Finland
dc.rev Erkut, Cumhur, Assist. Prof., Aalborg University, Denmark
dc.date.defence 2015-12-11


Files in this item

This item appears in the following Collection(s)

Show simple item record

Search archive


Advanced Search

article-iconSubmit a publication

Browse