aalto1 untyped-item.component.html
Anomaly detection in system event logs
Loading...
URL
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Electronic archive copy is available via Aalto Thesis Database.
Authors
Date
Department
Major/Subject
Mcode
SCI3095
Degree programme
Language
en
Pages
46 + 8
Series
Abstract
In this work, we explore approaches for detecting anomalies in system event logs. We define the system log anomaly detection problem and research existing methods. We apply the methods to a practical task of detecting anomalous events in logs of file behavior analysis sandbox. To validate results and compare methods we calculate quality metrics on a manually labeled dataset. First, we try an approach based on calculating event document frequency and use it as a baseline. We improve it by creating an event normalization algorithm and significantly reducing the number of false positives. After that, we implement a different approach that involves extracting event features and training random forest and logistic regression models to model a probability of an event belonging to a clean or anomalous log. Finally, we create a sequence model based on a recurrent neural network and use it to detect anomalies in event sequences.