Anomaly Detection in System Event Logs

Loading...
Thumbnail Image

URL

Journal Title

Journal ISSN

Volume Title

Perustieteiden korkeakoulu | Master's thesis

Department

Major/Subject

Mcode

SCI3095

Language

en

Pages

46 + 8

Series

Abstract

In this work, we explore approaches for detecting anomalies in system event logs. We define the system log anomaly detection problem and research existing methods. We apply the methods to a practical task of detecting anomalous events in logs of file behavior analysis sandbox. To validate results and compare methods we calculate quality metrics on a manually labeled dataset. First, we try an approach based on calculating event document frequency and use it as a baseline. We improve it by creating an event normalization algorithm and significantly reducing the number of false positives. After that, we implement a different approach that involves extracting event features and training random forest and logistic regression models to model a probability of an event belonging to a clean or anomalous log. Finally, we create a sequence model based on a recurrent neural network and use it to detect anomalies in event sequences.

Description

Supervisor

Ilin, Alexander

Thesis advisor

Kallunki, Jouni

Other note

Citation