Generative models for irregular sequential data

Thumbnail Image

Files

master_Safar_Shaghayegh_2021.pdf (15.26 MB)

URL

Journal Title

Journal ISSN

Volume Title

Perustieteiden korkeakoulu | Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2021-08-23

Department

Major/Subject

Machine Learning, Data Science and Artificial Intelligence

Mcode

SCI3044

Degree programme

Master’s Programme in Computer, Communication and Information Sciences

Language

en

Pages

55

Series

Abstract

Generative models are commonly used to generate realistic data for different purposes such as preserving the privacy of sensitive data or performance and unit testing. The majority of applications of fake data generation concerns itself with images, videos, text, or tabular data. A less explored direction is a particular type of sequential data where the time-stamp or position of elements of the sequence are not at constant intervals, i.e., irregular sequential data. Such data can for example contain the logs of events with time stamps in a company, which may be considered as sensitive information. This thesis proposes new generative models for such irregular sequential data. Variational auto encoder (VAE) models are powerful methods to generate non-sequential fake data. Following the huge success of transformer models in natural language processing applications, the thesis studies the use of attention network and applies it in a VAE model. The new network, called VAE-Attention, shows improved performance and better discriminative scores compared to a baselines and a related work. The analysis in this thesis shows that the model is able to generate high-quality irregular sequential data while having a reasonable training cost. The thesis is concluded with discussions about ways to include attention network in VAE models.

Description

Supervisor

Solin, Arno

Thesis advisor

Sakko, Arto

Keywords

generative models, sequential/event data, time-series, variational auto encoders, attention

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202108298599

Collections

[dipl] Perustieteiden korkeakoulu / SCI