Aligning Machine Learning for the Lambda Architecture
Perustieteiden korkeakoulu | Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Software Engineering and Business
Master’s Programme in Machine Learning and Data Mining (Macadamia)
AbstractWe live in the era of Big Data. Web logs, internet media, social networks and sensor devices are generating petabytes of data every day. Traditional data storage and analysis methodologies have become insufficient to handle the rapidly increasing amount of data. The development of complex machine learning techniques has led to the proliferation of advanced analytics solutions. This has led to a paradigm shift in the way we store, process and analyze data. The avalanche of data has led to the development of numerous platforms and solutions satisfying various business analytics needs. It becomes imperative for the business practitioners and consultants to choose the right solution which can provide the best performance and maximize the utilization of the data available. In this thesis, we develop and implement a Big Data architectural framework called the Lambda Architecture. It consists of three major components, namely batch data processing, realtime data processing and a reporting layer. We develop and implement analytics use cases using machine learning techniques for each of these layers. The objective is to build a system in which the data storage and processing platforms and the analytics frameworks can be integrated seamlessly.
Thesis advisorLuukkonen, Olli
machine learning, big data, data mining, lambda architecture, internet of things