Browsing by Author "Okulov, Jaana"

Now showing 1 - 4 of 4

Artificial Aesthetics and Aesthetic Machine Attention
(2022-10) Okulov, Jaana
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
The aesthetics of artificial intelligence is often viewed in relation to the qualities of their generated expressions. However, aesthetics could have a broader role in developing machine perception. One of the main areas of expertise in aesthetics is the understanding of feature-based information, which involves how the aesthetics of sensory features can cause affective changes in the perceiver, and the other way around - how affective states can give rise to certain kinds of aesthetic features. This two-way link between aesthetic features and affects is not yet well-established in the interdisciplinary discussion; however, according to perceptual psychology, it fundamentally constructs the human experience. Machine attention is an emerging technique in machine learning that is most often used in tasks like object detection, visual question answering, and language translation. Modern use of technology most often focuses on creating object-based attention through linguistic catego-ries, although the models could also be utilized for nonverbal attention. This paper proposes the following perceptual conditions for aesthetic machine attention: 1) acknowledging that something appears (aesthetic detection); 2) suspension of judgment (aesthetic recognition); and 3) making the incident explicit with expression (aesthetic identification and amplifica-tion). These aspects are developed through an interdisciplinary reflection of literature from the fields of aesthetics, perceptual psychology, and machine learning. The paper does not aim to give a general account of aesthetic perception but to expand the interdisciplinary theory of aesthetics and specify the role of aesthetics among other disciplines at the heart of the techno-logical development of the human future.
Data as Expression
(2023-06-21) Okulov, Jaana
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
In machine learning literature, the concept of expression is seldom addressed as a general term in relation to information construction and data. The term more often carries a narrower meaning and refers to human bodily, emotional, or artistic dimensions that are recorded to train a model. However, this paper discusses a view in which expression is understood as any human sensory realm that can bring incidents explicit through the act of “pressing out,” as the early etymology of expression indicates. Also, all the sensory data that are used to train a machine learning model and the data the trained model gives as an output are considered meaningful through their expressive mediality—a form that is actively produced and can therefore be subjected to critical phenomenological analysis. This paper contrasts nonverbal expressions with categorical or linguistic expressions and asks, “What do eye movements express when they are used in training a machine learning model? What kind of expression arises in linguistic models? What could be considered aesthetic data (and what would it express)?” For philosopher John Dewey, instinctive reactions in human behavior that for example exhibit mere discharge of an emotion should be separated from purposeful expressions. However, in machine learning, the two Deweyan positions (instinctive and intentional) collapse, as artificial expressions are purely simulations of learned logic. Therefore, the phenomenological question of this paper is traced back to the original stimulus and its human annotators. In this paper, philosopher Don Ihde’s experimental phenomenology explains how appearances can be attended to without subsuming them under any assumptions, whereas art theory—arising especially from philosopher Dieter Mersch’s thinking—provides an understanding of the mediality of expression. This paper introduces examples from machine learning that are not generally considered expressive but are used for regular tasks, such as object detection, and provides an alternative approach from aesthetics and artistic research that understands these modalities as expressive. If data are understood as expressive, it can be critically assessed how current machine learning models constitute knowledge.
Deciphering Multimodal Correspondence using Exploratory Data Analysis
(2023-06-12) Hota, Asutosh
Perustieteiden korkeakoulu | Master's thesis
Artistic and creative processes rely on integrating information from multiple sensory modalities. However, understanding the complex interplay between these modalities and how they correlate remains a challenge. The methods followed in conventional behavioral and psychological experiments have been consistently qualitative and the correlations/correspondence have been traditionally found on the basis of the choices that the human participant thinks (pair-matching). These have proven to be the existential foundation of multimodal correlation studies however, a lack of a quantitative approach limits this experimental methodology to test only a few numbers of participants. Conventional pair/pattern matching experiments may not fully capture the underlying correlations in sensory multimodal data and Exploratory Data Analysis (EDA) based approaches can reveal hidden trends and insights. This thesis proposes Primary Evaluator for Multimodal Correlation (PEMC), a novel framework which provides a data-driven approach for exploring correlations between two or more sensory modalities. The framework emphasizes the importance of EDA techniques in identifying hidden patterns in sensory multimodal data, which may not be captured through conventional pair/pattern matching experiments. Utilizing various EDA techniques, such as dimensionality reduction, unsupervised clustering, and correlation analysis, we propose the Correlation Analyzer (CA), an integral part of PEMC. CA is used to identify correlations between two modalities. PEMC framework tries to conduct a preliminary evaluation of the existence of underlying correlations in sensory data using CA in 3 unique test settings. The results suggest that there exist multimodal correlations and recommend whether more controlled experiments are needed to establish the presence of universal multimodal correlations. In this thesis, we conduct an in-depth analysis of sensory multimodal data extracted from audio responses, pen movement responses, and colour transition data as stimulus data using the PEMC. Our findings reveal moderate to strong correlations in the features of audio and pen movement data in response to colour transition data, providing valuable insights into how different modalities interact and influence each other. Potential limitations of the framework, best practices and many applications of the correlation analysis are also discussed giving directions to future studies.
Quantifying Qualia – Aesthetic Machine Attention in Resisting the Objectifying Tendency of Thought
(2024) Okulov, Jaana
School of Arts, Design and Architecture | Doctoral dissertation (article-based)
My interdisciplinary doctoral thesis Quantifying Qualia – Aesthetic Machine Attention in Resisting the Objectifying Tendency of Thought, conducted at the Department of Art and Media at Aalto University, explores human and algorithmic perception. While language-based approaches are widely developed and utilized in machine learning today, the thesis explores the ethical potential of alternative modes of perception to be manifested in machines and proposes the concept of aesthetic attention to invite perceptual variations from phenomena through how they resonate across the senses. Psychologist Daniel Stern suggests that this dynamic nature of experience, arising from embodiment, represents the earliest stage of development. Consequently, it serves as the primary means for interpersonal communication and also expressing inner experiences later in life. Additionally, affective and aesthetic expressions can be viewed as being rooted in these vitality forms described by Stern. The thesis argues that aesthetically oriented attention has the potential to reorganize perception by delaying the categorical determination of an experience. At the core of my research is the idea that the narrowed cognitive repertoire resulting from perceptual biases can be altered with perceptual strategies aiming to broaden the receptivity for sensory knowledge. My thesis consists of three peer-reviewed articles published in interdisciplinary edited volumes and journals, along with one peer-reviewed unpublished article. These articles redefine philosophical concepts such as aesthetic attention and qualia, making them computable. As a result, a method was developed in interdisciplinary collaboration to generate asemic stimuli algorithmically. This approach also led to the establishment of a research platform that seamlessly integrated both artistic and quantitative research. The artistic conclusion of my thesis is a research process utilizing the platform. During this process, asemic stimuli were annotated with artistic expressions as opposed to the traditional method of using verbal categories for annotating. Multimodal expressions established aesthetic data for a machine attention model to perceive beyond categories. With the research process, I demonstrated how the development of machine learning models that incorporate nonverbal expressions can influence cultures increasingly reliant on algorithmic information processing; future intelligence and ethics are founded on the choices we now make in what is recognized as valuable data.