Clustering and classification methods for spam analysis

Thumbnail Image

Files

master_Smirnov_Maksim_2018.pdf (2.1 MB)

URL

Journal Title

Journal ISSN

Volume Title

Perustieteiden korkeakoulu | Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2018-10-08

Department

Major/Subject

Security and Cloud Computing

Mcode

SCI3084

Degree programme

Master’s Programme in Computer, Communication and Information Sciences

Language

en

Pages

73+7

Series

Abstract

Spam emails are a major tool for criminals to distribute malware, conduct fraudulent activity, sell counterfeit products, etc. Thus, security companies are interested in researching spam. Unfortunately, due to the spammers' detection-avoidance techniques, most of the existing tools for spam analysis are not able to provide accurate information about spam campaigns. Moreover, they are not able to link together campaigns initiated by the same sender. F-Secure, a cybersecurity company, collects vast amounts of spam for analysis. The threat intelligence collection from these messages currently involves a lot of manual work. In this thesis we apply state-of-the-art data-analysis techniques to increase the level of automation in the analysis process, thus enabling the human experts to focus on high-level information such as campaigns and actors. The thesis discusses a novel method of spam analysis in which email messages are clustered by different characteristics and the clusters are presented as a graph. The graph representation allows the analyst to see evolving campaigns and even connections between related messages which themselves have no features in common. This makes our analysis tool more powerful than previous methods that simply cluster emails to sets. We implemented a proof of concept version of the analysis tool to evaluate the usefulness of the approach. Experiments show that the graph representation and clustering by different features makes it possible to link together large and complex spam campaigns that were previously not detected. The tools also found evidence that different campaigns were likely to be organized by the same spammer. The results indicate that the graph-based approach is able to extract new, useful information about spam campaigns.

Description

Supervisor

Aura, Tuomas

Thesis advisor

Tynninen, Päivi

Keywords

spam campaigns, clustering, mass email, graph analysis, threat intelligence, malware

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-201810175433

Collections

[dipl] Perustieteiden korkeakoulu / SCI