Noise detection pattern recognition in a univariate time series case

Abstract

The goal of this project is to find a specific time series pattern in diverse univariate time series data. The data comes from memory reports of computing servers. These reports show the amount of memory that is in use in the servers, as well as the total memory available.

The specific pattern that was required to find was noise, which consists of periods of time in which there are high fluctuations of in-use memory.

The algorithms that detect these patterns have been manually developed from scratch, due to the diverse nature of the data with which more control was needed. Some of the developed algorithms are a custom peak and valley detector, a noisy segment calculator by two different methodologies (which will be compared), and a daily seasonality detector.

In the results, we get all the systems with noise of the customer, a score of noisiness of each one, and a ranked list of those from the noisiest to the least noisy. An analysis of the two or three noisiest cases is performed, depending on the methodology used.

Additionally, a deployment of part of the work is done on a tool to assess its feasibility. This proof of concept will potentially allow deploying all current and future developments on this tool.