Intrusion detection system for Android: Linux kernel system calls analysis

Abstract

Smartphones provide access to a plethora of private information potentially leading to financial and personal hardship, hence they need to be well protected. With new Android malware obfuscation and evading techniques, including encrypted and downloaded malicious code, current protection approaches using static analysis are becoming less effective. A dynamic solution is needed that protects Android phones in real time. System calls have previously been researched as an effective method for Android dynamic analysis. However, these previous studies concentrated on analysing system calls captured in emulated sandboxed environments, which does not prove the suitability of this approach for real time analysis on the actual device.

This thesis focuses on analysis of Linux kernel system calls on the ARMv8 architecture. Given the limitations of android phones it is necessary to minimise the resources required for the analyses, therefore we focused on the sequencing of system calls. With this approach, we sought a method that could be employed for a real time malware detection directly on Android phones. We also experimented with different data representation feature vectors; histogram, n-gram and co-occurrence matrix. All data collection was carried out on a real Android device as existing Android emulators proved to be unsuitable for emulating a system with the ARMv8 architecture. Moreover, data were collected on a human controlled device since reviewed Android event generators and crawlers did not accurately simulate real human interactions.

The results show that Linux kernel sequencing carry enough information to detect malicious behaviour of malicious applications on the ARMv8 architecture. All feature vectors performed well. In particular, n-gram and co-occurrence matrix achieved excellent results. To reduce the computational complexity of the analysis, we experimented with including only the most commonly occurring system calls. While the accuracy degraded slightly, it was a worthwhile trade off as the computational complexity was substantially reduced.